It seems like there are endless ways to ask questions of participants in surveys.
Variety in question types can be both a blessing and a curse.
Having many ways to ask questions provides better options to the researcher to assess the opinion of the respondent.
But the wrong type of question can fail to capture what’s intended, confuse respondents, or even lead to incorrect decisions.
In surveys, questions can be broadly classified as open-ended (free responses) or closed-ended. Closed-ended questions themselves can be classified into multiple choice questions or rating scales.
Multiple choice questions (e.g. age, education level, or electronic devices owned) are usually more straightforward for researchers with little survey experience. When properly written, they’re also straightforward to the respondents as they usually involve concrete selections.
Rating scales however usually involve asking participants to rate abstract concepts, such as satisfaction, ease, or likelihood to recommend. The item selection can have a big impact on both the responses and interpretation.
There are different ways of classifying rating scales and slight variations can result in different looking rating scales, even though they’re variations on the same scale. For example, our MUIQ platform offers over 30 question types but I’ve identified 15 distinct ones.
I’ve adapted a classification scheme based on our experience at MeasuringU and from the classic text on survey research by Alreck and Settle. Here are 15 scales, in roughly the order of most to least commonly used.
1. Linear Numeric Scale
In a linear numeric scale, participants provide some numeric response to a question or statement. This can include things like satisfaction, ease, brand favorability, feature importance, or likelihood to recommend. The Single Ease Question (SEQ) and likelihood to recommend item used in the NPS are examples of linear numeric scales. Linear numeric scales usually have at least the endpoints labeled. (Labeling, neutral points, and number of response options are the topics of other articles.)
2. Likert
The classic Likert scale has participants agree or disagree (or approve/disapprove) to multiple statements. When numbers are associated with each response option, the Likert item can be seen as a special case of the linear numeric scale. The classic Likert item uses a 5-point response scale, but you can use 7, 9, or other points, too. (Although someone will have a strong opinion about the “right” number of steps.) Because the response scale is about agreement, be sure items are phrases participants can agree or disagree to. The System Usability Scale (SUS), SUPR-Q, and UMUX-Lite use a Likert scale with numbered values.
3. Multiple Rating Matrix
The matrix question is a compact way of presenting multiple linear numeric items and is the typical method for displaying Likert items, too. It’s probably not technically different from a linear numeric scale but I’ve separated it out because they’re so popular for online surveys. For example, when having participants rate their brand attitude, it’s common to use a matrix similar to the following one.
Try not to use too many of these in a survey as it can be overwhelming to respondents.
4. Frequency Scales
Understanding how often people perform (or think they perform) actions helps when product planning as in the example below. When listing the frequency of actions, consider both specific number of times (e.g. every day) as well as more general timeframes (sometimes, always, never—referred to as a verbal frequency scale). Also, be sure the frequencies are sequentially ordered and well understood. For example, is occasional more frequent than sometimes?
When we measure users’ attitudes toward the ease of use of websites or software using the SUS or UMUX-Lite, we ask how frequently participants use the software with a verbal frequency scale similar to the one below. (Frequency of use often predicts attitudes.)
5. Forced Ranking Scale
Forced ranking scales are good for prioritizing product features. Having participants rate their interest on a linear numeric scale may result in the problem of every feature being important because there’s no disincentive for rating everything high.
I recommend keeping the number of items to fewer than 10 when possible and randomize their presentation order. With each option, respondents have to review the list to make a decision on ranking. To rank 20 items, for example, participants need to make 19 passes through the continually shrinking list. This process is easier with a drag-and-drop interface as in the MUIQ item below, but forcing people to rank items they have little opinion on may lead to drop out or error. It gets quite laborious to rank many items. If your list is long, consider a “pick some” question type (see #6).
6. Pick Some (a.k.a Top Task)
When you have a long list for participants to prioritize (e.g. more than 10 and especially more than 20) but don’t want them to have to rank all of the items, have participants select a fixed subset, such as 3 or 5. This is what we do for a top-tasks analysis. Again, it’s important to randomize the order to avoid items near the top being favored. Surprisingly, we’ve found that this crude technique takes a fraction of the time as forced ranking and yields very similar results.
7. Paired Comparison Scale
When you want to force a choice between two alternatives (sort of a mini-rank) such as a preference for a website, brand, or design, use a paired comparison scale.
Paired comparisons can also be used on specific attributes of a website, product, or brand as shown below for different aspects of two rental car websites. They’re also used in advanced question types, such as with the Max-Diff.
Items don’t have to be just text. You can present pictures (like alternate designs) or videos for respondents to select their preference.
8. Comparative Scale/Comparative Intensity
You can have participants rate their preference and strength of preference all in one item using a comparative scale. The scale below asks participants to rate their preference and intensity for two rental car companies on four website attributes.
It can also be used to gauge participants’ preferences toward a known external benchmark such as a famous brand or website (like Amazon).
9. Semantic Differential Scale
When you want to assess where participants fall on a continuum of adjectives or attributes, use a semantic differential scale. You need to provide clear polar opposite terms (like hot to cold)—which can be easy in principle but hard in practice. For this reason, we don’t use these as often and prefer the next two options.
The semantic differential scale below asks participants to rate their experience with Netflix on two items.
10. Adjective Checklist
When assessing brand attitude, the adjective checklist is a staple. It’s also the technique used in the Microsoft Desirability Toolkit.Instead of aligning opposite adjectives, you can list them (usually a mix of positive and negative) for participants to select. Again, randomize the presentation order.
11. Semantic Distance Scale
A way to avoid the problem of having to find polar opposites on the semantic differential scale but still have participants rate each adjective is to have respondents rate an adjective, term, or phrase and provide some level of intensity. It’s sort of a cross between the adjective scale and semantic differential scale.
12. Fixed Sum
When you need responses to add up to a fixed amount, such as 100% or an amount spent (e.g. $100), a fixed sum approach might work. It can be another way to force respondents to decide which features are more important than others and is a popular technique for assessing the importance of new features, or even how participants allocate their budget (like the example below).
You’ll want to minimize the mental load you put on your respondent by displaying a running total of the values they’ve entered and what is left (as shown in the MUIQ example above).
13. Compound Matrix
You can really cram a lot of things into one question by using drop-down lists or text fields instead of radio buttons or checkboxes. The compound matrix has participants rate two dimensions at the same time; for example, the importance of features by device type for online banking.
14. Pictorial/Graphic Scales
Instead of picking a number, participants can respond to pictures, such as stars as done on Amazon and Netflix. The stars represent a quantity that can be averaged similar to linear numeric scales.
The Wong-Baker faces pain scale is a common scale used to assess patient discomfort.
Pictorial scales can be particularly helpful when participants might not speak the target language well or even have trouble communicating (hence its widespread use in medical settings).
15. Visual Analog/Slider Scale
Imagine a linear numeric scale that didn’t have discrete points (e.g. 1 through 7) but instead allowed participants to select any value in between. This is the idea behind the Visual Analog Scale (VAS), often just called a slider scale. The analog is the continuum the slider represents; for example, from extremely difficult to extremely easy in the example below.
The impact of slider scales attributes (labels, showing values, starting position of slider) is still being investigated. We’ve had success with participants understanding and using sliders in MUIQ and plan to continue research with and about them.
FAQs
How do you answer a rating scale? ›
- Good - Fair – Poor.
- Agree – Undecided - Disagree.
- Extremely- Moderately - Not at all.
- Too much - About right - Too little.
Popular Types of Rating Scales
Graphic rating scale. Numerical rating scale. Descriptive rating scale. Comparative rating scale.
The rating scale is a closed-ended survey question used to represent respondent feedback in a comparative form for specific particular features/products/services. It is one of the most established question types for online and offline surveys where survey respondents are expected to rate an attribute or feature.
What is the most common rating scale? ›The most common example is the Likert scale, star rating, and slider. For instance, when you visit an online shopping site, it asks you to rate your shopping experience. It is a popular choice for conducting market research.
How do you explain your performance rating? ›Performance rating is the step in the work measurement in which the analyst observes the worker's performance and records a value representing that performance relative to the analyst's concept of standard performance.
What does rating scale 1 to 10 mean? ›In a 1 to 10 Opinion Scale Survey, a numeric scale is provided where rating options lie from 1 to 10, among which the customers can choose a score to rate their experience. Here, ten represents the most positive experience, and one(or zero) illustrates the most negative experience.
What is a simple rating scale? ›The usual simple rating scale purports to measure direction (i.e. important/unimportant, effective/ineffective) and intensity (i.e. very, somewhat) of attitude or opinion in a single assessment.
What is 5 point rating scale examples? ›The 5-point Likert scale contains 5 response options that will consist of two extreme sides and a neutral option linked to the middle answer options. Examples of a 5-point rating scale for measuring satisfaction are: Very Satisfied, Satisfied, Neutral, Dissatisfied, and Very Dissatisfied.
What is an example of a scale question? ›Here is a typical Likert scale question and answer example: How satisfied are you with the service you have received from [brand, department, service agent]? The respondent might be offered this 5-point Likert scale from which to select a response. Very satisfied.
What is a rating scale quizlet? ›A Rating Scale is similar to a Checklist except that it has more options from which to choose. Rating Scales are used for assessing both environment and development. The Rating Scale is a closed recording method. The Rating Scale is free of rater bias.
What is the rating scale used for measuring performance? ›
The four-point rating scale. Many organizations have used the standard three-point rating scale. However, in our research on the distribution of performance responses, we have found that a 4-point rating scale is often the best option.
What are the 5 rating scale? ›| Scale | Rating |
|---|---|
| 5 points (Pass) | Excellent. Exceptional Mastery. Much more than acceptable. |
| 4 points (Pass) | Very Good. Full Performance Behaviours. Above average. |
| 3 points (Pass) | Good. Acceptable. Satisfactory Average |
| 2 points (Fail) | Weak. Less than Acceptable |
- Scale is the ratio between the actual distance on the ground and the distance shown on the map. - The distance between the school and the home of a student is 10 km. If he shows this 10 km distance by 2 cm on the map, it means, 1cm on the map will show 5km on the ground.
What are the most common scales used in research? ›The four most common types of scale are: nominal, ordinal, interval and ratio. Among the attitude scales used in scientific research, we highlight the Thurstone and the Likert.
What are the common rating scale errors? ›Four of the more common rating errors are strictness or leniency, central tendency, halo effect, and recency of events (Deblieux, 2003; Rothwell, 2012).
How do you answer your own performance evaluation? ›- Reflect on feedback. ...
- Make a list of your top accomplishments and identify areas for improvements. ...
- Gather analytics to show impact. ...
- Make a commitment to improve. ...
- Set a SMART goal for yourself. ...
- Create a plan of action. ...
- Communication. ...
- Job Performance.
2: How do you answer employee appraisal questions? You need to review your past performance and future goals. Make a list of accomplishments and understand your strengths and weaknesses. Listen actively during your appraisal round and answer all the questions honestly.
How do you answer a performance self-evaluation? ›- Be proud. One major goal of the self-evaluation is to highlight your accomplishments and recollect milestones in your professional development. ...
- Be honest and critical. ...
- Continuously strive for growth. ...
- Track your accomplishments. ...
- Be professional.
Mathematically 7/10 is better than average, and therefore we feel like it is a good score. However, when it comes to ratings and reviews, 7/10 is mediocre, because people are generally nice, and rarely willing to give other people or services a truly low score. Ultimately what this leads to is standing still.
What is the scale of 1 to 5 ratings? ›A 1 to 5 rating scale is a simple and effective way to rate the severity or magnitude of something. It typically goes from 1, the lowest rating, to 5, the highest rating. The 1 to 5 scale allows respondents to answer quickly and can be applied to a variety of things, such as pain, temperature, and brightness.
What does scale of 1 to 5 mean? ›
Definition of Scale
For example, a scale of 1:5 means that the size of 1 unit in the drawing would represent 5 units in the real world.
Checklists, rating scales and rubrics are tools that state specific criteria and allow teachers and students to gather information and to make judgements about what students know and can do in relation to the outcomes. They offer systematic ways of collecting data about specific behaviours, knowledge and skills.
What is rating scale model? ›The rating scale model specifies that a set of items share the same rating scale structure. It originates in attitude surveys where the respondent is presented the same response choices for several items. The partial credit model specifies that each item has its own rating scale structure.
How do you make a rating scale? ›- Some basic goals.
- Use a limited number of scale points.
- Always include a middle or neutral alternative in the middle.
- Display answers in order.
- A clear description of what each of the answers means on the scale.
- What to keep in mind.
Step 1: For each question on the questionnaire, calculate the total number of responses for each sentiment level (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree). Step 2: Add the totals, and divide by the total number of respondents: 1 + 0 + 0 + 0 + 5 = 6 / 2 respondents = 3.
Why use a 5-point rating scale? ›The 5-point Likert scale is simple to understand and use for survey administrators and respondents alike. It takes less time and effort to complete than higher-point scales. Fits mobile device screens better than higher-point scales. Respondents have choices without becoming overwhelmed.
How does a 5-point scale work? ›The notion behind the 5-point scale is to take an idea or behavior and break it into five parts to make it easier to understand the different degrees of behavior and, eventually, the consequences of one's behavior.
What is an example of scale measurement? ›Interval scales
For example, temperature measurement is an example of an interval scale: 60°C is colder than 65°C, and the temperature difference is the same as the difference between 50°C and 55°C. In other words, the difference of 5°C in both intervals shares the same interpretation and meaning.
Introduction: There are 4 types of scales, based on the extent to which scale values have the arithmetic properties of true numbers. The arithmetic proper- ties are order, equal intervals, and a true zero point. From the least to the most mathematical, the scale types are nominal, ordinal, interval, and ratio.
What is a scale and examples? ›A scale factor is defined as the ratio between the scale of a given original object and a new object, which is its representation but of a different size (bigger or smaller). For example, if we have a rectangle of sides 2 cm and 4 cm, we can enlarge it by multiplying each side by a number, say 2.
What is quality rating scale? ›
Purpose: The performance quality rating scale (PQRS) is an observational measure of performance quality of client-selected, personally meaningful activities. It has been used inconsistently with different scoring systems, and there have been no formal publications on its psychometric properties.
What rating question means? ›A rating question asks survey respondents to compare different items using a common scale (e.g. “Please rate each of the following objects on a rating scale of 1-10, where 1 is 'not at all important' and 10 is 'very important. '”).
What is the best rating scale for a survey? ›There's more variance in larger scales, which has made the Likert scale the most common survey scale. Dr. Rob Balon advises to “always use the 1–5 scale, with 5 being the positive end and 1 being the negative end.
What are the types of rating scale? ›Types of Rating Scales Rating scales are also of various types. Some of their main types are as follows- (i) Numerical Scales. (ii) Descriptive Scales. (iii) Rank Order Scales (iv) Graphic Scales.
What is a 5 point rating scale questionnaire? ›A 5-point Likert scale is a psychometric response method where respondents can easily answer questions and state their level of agreement in five points. The 5-point Likert scale consists of the below points – (1) Strongly Disagree; (2) Disagree; (3) Neither Agree nor Disagree; (4) Agree; (5) Strongly Agree.
What is a 5 point rating scale called? ›So what is a Likert scale survey question? It's a question that uses a 5 or 7-point scale, sometimes referred to as a satisfaction scale, that ranges from one extreme attitude to another. Typically, the Likert survey question includes a moderate or neutral option in its scale.
How do you explain a scale to students? ›Scale factor is the number used to multiply one object by to get another object that looks the same but is a different size. It makes an exact copy only larger or smaller than the original. Scale Factor multiplies the picture or object just as if you enlarged or shrank it on a copy machine.
What is the common scale called? ›The most common scales in Western music contain seven pitches and are thus called “heptatonic” (meaning “seven tones”). Other scales have fewer notes—five-note “pentatonic” scales are quite common in popular music.
What is the most common measure of scale reliability? ›Cronbach's alpha is the most common measure of internal consistency ("reliability"). It is most commonly used when you have multiple Likert questions in a survey/questionnaire that form a scale and you wish to determine if the scale is reliable.
What is the popular rating scale used by researchers? ›The most commonly used scaling technique in market research surveys is the itemized rating scale, which is a measurement scale that has numbers and or labels associated with each scale point and ordered in a particular position.
What is the most common assessment error? ›
Perhaps the most common mistake made by assessors is tying a property's real estate value to its purchase or sale price. The conflict between the real world of what an owner pays for a property and the hypothetical world of property tax assessments becomes a constant source of tension.
How do you avoid common rating errors? ›- Build Awareness of Rater Bias.
- Use Objective, not Subjective, Ratings.
- Reduce Reliance on Memory.
- Implement 360 Degree Feedback Systems.
- Carefully Monitor Performance Feedback Data.
A 1 to 5 rating scale is a simple and effective way to rate the severity or magnitude of something. It typically goes from 1, the lowest rating, to 5, the highest rating. The 1 to 5 scale allows respondents to answer quickly and can be applied to a variety of things, such as pain, temperature, and brightness.
How do you write a rating scale interview question? ›- Ask your team about their needs. When establishing what criteria a new hire should have, incorporate the needs of the team. ...
- Establish clear ratings for each requirement. ...
- Leave room for comments. ...
- Helps you focus. ...
- Provides a record. ...
- Ensures fairness. ...
- Helps you keep track of candidates.
Generic 4-Star Reviews
"Thank you so much for this review. We really appreciate you being a customer. We're here for you anytime.” “We are so grateful for your review.
The 5-point Likert scale contains 5 response options that will consist of two extreme sides and a neutral option linked to the middle answer options. Examples of a 5-point rating scale for measuring satisfaction are: Very Satisfied, Satisfied, Neutral, Dissatisfied, and Very Dissatisfied.
How do you calculate a 5-point rating scale? ›Step 1: For each question on the questionnaire, calculate the total number of responses for each sentiment level (Strongly Disagree, Disagree, Neutral, Agree, Strongly Agree). Step 2: Add the totals, and divide by the total number of respondents: 1 + 0 + 0 + 0 + 5 = 6 / 2 respondents = 3.
What is a 5-point scale rating? ›A 5-point Likert scale is a psychometric response method where respondents can easily answer questions and state their level of agreement in five points. The 5-point Likert scale consists of the below points – (1) Strongly Disagree; (2) Disagree; (3) Neither Agree nor Disagree; (4) Agree; (5) Strongly Agree.
What does a scale of 1 to 20 mean? ›For example, a scale of “one to twenty” means that one millimetre on a page represents twenty millimetres in real life.
How do I calculate scale? ›The basic formula that is used for calculating the scale factor is, Scale factor = Dimension of the new shape ÷ Dimension of the original shape. In case, if the original figure is scaled up, the formula is written as, Scale factor = Larger figure dimensions ÷ Smaller figure dimensions.
What is a scale of 1 to 10 called? ›
That kind of question is known as a Likert scale. Likert scales are widely used to measure attitudes and opinions with a greater degree of nuance than a simple “yes/no” question.
How do you respond to a 3 star rating? ›An acknowledgment. Greet or acknowledge the reviewer, setting a warm, personal tone for the rest of your response. For example, you can say, “Thanks for sharing your feedback, [Reviewer]!” or, “Hi Joey, thank you for leaving a rating and review for [Business Name].” Your appreciation.
What does 3 stars mean in review? ›Common Sense Media uses a scale of one to five, where 3 stars are "Just fine; solid" and anything lower is "Disappointing" at best. There is no agreement on what the lowest rating should be. Some critics make "one star" or a "half-star" their lowest rating.