A rating response allows participants to respond using a numeric scale.
The default setup is a 5-point likert scale, as in the below image.
If you preview or have launched a study before August 15, 2022, you'll see something like this:
Since August 15, 2022, the rating response has a brand new look and supports multi-row questions via row labels:
¶ Required Properties
¶ type
- Definition: The type of response
- Possible values: For a text response,
typemust be specified as"rating".
{
"type": "rating"
}
This piece of code sets up a rating response in the form of a five-point likert scale. If this is what you want, there is no need to add additional code to this response!
¶ Optional Properties
¶ auto_lock_in
Available since August 15, 2022
- Definition: Whether or not to automatically lock in a participant's response. If set to
false, then participants need to click a seperate Confirm button to lock in their answer(s). We recommend leaving it at its default valuetrueespecially if you are sequentially presenting multiple rows of individual rating scales. - Possible values:
trueorfalse - Default value:
true - See also: row_labels, sequential
{
"type": "rating",
"auto_lock_in": false // this is the default; you don't need to specify this
}
¶ delay
- Definition: Delay the displaying of a response (relative to the latest onset of all stimuli on a given trial) by a certain number of seconds. This setting can be useful for forcing participants to read some text before making a response.
- Possible values: Any numeric value. Fractions are supported.
- Default value:
0
{
"type": "rating",
"delay": 1
}
¶ duration_timer
- Definition: Whether to show a duration timer that indicates the time elapsed since the reponse becomes visible.
- Possible values:
true,false - Default:
false
{
"type": "rating",
"duration_timer": true
}
¶ duration_timer_onset
Relevant only if "duration_timer" is true.
- Definition: When to show the duration timer. When set to
"start", the timer will appear as soon as the response is displayed and animate a stopwatch effect. When set to"end", the duration timer will only appear once the participant has made a reponse with no stopwatch effect. - Possible values:
"start","end" - Default:
start
{
"type": "rating",
"duration_timer": true,
"duration_timer_onset": "start"
}
¶ instruction
- Definition: A string of text providing instructions for participants about the current response.
- Possible values: A string of text
- Default value: "How confident are you in your response?"
- To remove the instruction entirely, set
instructionto""(empty quotes).
{
"type": "rating",
"instruction": "How much do you agree with the above statement?"
}
In this example code, participants will see the instruction "How much do you agree with the above statement?" above the rating scale.
{
"type": "rating",
"instruction": ""
}
In this example code, participants will see no instruction above the text box.
¶ labels
New features added since August 15, 2022
-
Definition: Optional column labels that are associated with each step of the rating response. If not specified, a numeric scale will be displayed.
-
Possible values:
Two options are available for defining the labels:- A list of text labels to cosmetically replace numeric values on the scale. Note that participant responses are still recorded in numeric terms in your output data. The number of labels must be the same as the number steps in the rating scale - for example, if there are 5 steps, the list must contain five quoted text labels, such as
["Strongly Disagree", "Disagree", "Neutral", "Agree", "Strongly Agree"]. - A dictionary that maps text labels to numeric values on the scale (e.g.,
{"Label1": 1, "Label5": 5}). This option allow researchers to specify only the column labels that are relevant (e.g., "Strongly Disagree", "Neutral", "Strongly Agree") while leaving out the labels for interleaving steps on the scale. See example code for details.
- A list of text labels to cosmetically replace numeric values on the scale. Note that participant responses are still recorded in numeric terms in your output data. The number of labels must be the same as the number steps in the rating scale - for example, if there are 5 steps, the list must contain five quoted text labels, such as
-
Default value: no default
{
"type": "rating",
"instruction": "How much do you agree with the above statement?",
"labels": [
"Strongly Disagree",
"Disagree",
"Neutral",
"Agree",
"Strongly Agree"
]
}
Because labels is assigned a list of values, participants will see the names "Strongly Disagree", "Disagree" Neutral", "Agree", "Strongly Agree" in place of the numbers 1-2-3-4-5. In your data, a response of "Strongly Disagree" will be coded as 1, "Disagree" as 2, etc.
Using the dictionary format, we can omit "Disagree" and "Agree" and just present 3 labels:
{
"type": "rating",
"instruction": "How much do you agree with the above statement?",
"labels": {
"Strongly Disagree": 1
"Neutral": 3,
"Strongly Agree": 5
]
}
In this case, participants will see only the three labels defined here, positioned at the corresponding numeric values on the scale (i.e., 1, 3, 5) as defined.
¶ max
- Definition: The endpoint of the numeric rating scale.
- Possible values: Any number (decimals not supported)
- Default value:
5 - See also: min, reverse_scoring, zero_centered
{
"type": "rating",
"min": 3,
"max": 6
}
Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6.
¶ min
- Definition: The starting point of the numeric rating scale.
- Possible values: Any number (decimals not supported)
- Default value:
1 - See also: max, reverse_scoring, zero_centered
{
"type": "rating",
"min": 3,
"max": 6
}
Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6.
¶ parent
- Definition: The name of another response. If specified, then the properties of the other response will be inherited by this response (unless a property of the same name is defined in this response).
- Possible values: The name of another response
Given a response named "response1" as follows:
{
"type": "rating",
"max": "10",
"min": "5"
}
You can avoid repetitive typing by defining a child response, "response2", that inherits all settings of its parent:
{
"type": "rating",
"parent": "response1"
}
¶ reverse_scoring
- Definition: Whether to use reverse scoring in your output data. If set to
true, a response of 1 on a 1-5 scale will appear as 5 in your output data, a response of 2 will appear as 4, and a response of 5 will appear as 1. - Possible values:
trueorfalse - Default value:
false - Note:
reverse_scoringdoes not change the order of the scale that participants see. Even ifreverse_scoringis set totrue, a scale with a minimum vale of 1 and maximum value of 5 will display the options 1-2-3-4-5, in that order. Thereverse_scoringproperty simply affects the scores that are recorded in your output data.
{
"type": "rating",
"min": 3,
"max": 6,
"reverse_scoring": true
}
Because min is set to 3, participants will see a rating scale whose lowest value is 3. Because max is set to 6, the scale's highest value will be 6. And because reverse_scoring is set to true, responses in your output data will be coded in reverse order: a response of 3 will be coded as 6, a response of 4 coded as 5, 5 coded as 4, and 6 coded as 3.
¶ row_labels
Available since August 15, 2022
-
Definition: Optional row labels, each of which corresponds to a separate rating scale in a rating response. This feature can be useful for asking a group of questions in the format of ratings. The option for revealing the individual rows sequentially or all at once can be further tweaked via the sequential property.
-
Possible values: A list of text labels, each of which appears to the left of the corresponding rating scale.
Advanced usage: If you need to conditionally branch your experiment based on participant responses from an individual row within a multi-row rating response, use the
"response_name|row_label_text"format as the response name in a trigger.
-
Default value: No default. A default row label of 1 is always implied, meaning that the rating response will have just one scale, but the default text will be an empty string.
-
See also:: auto_lock_in, sequential
{
"type": "rating",
"instruction": "How do you like your new computer?",
"row_labels": [
"Overall speed",
"Speakers",
"Webcam",
"Weight"
]
}
¶ sequential
Available since August 15, 2022
-
Definition: If multiple row labels are defined, whether or not to present each individual row of rating scale sequentially. When rows are presented sequentially, participants must answer the current question before the next one is revealed.
-
Possible values:
trueorfalse -
Default value:
false -
See also: auto_lock_in, row_labels
Reaction times collected on multi-row rating responses are always with respect to the onset of the first rating scale, even when rating rows are sequentially presented. You can subtract the reaction time of the previous rating row from the reaction time of the row of interest to get its "net" reaction time.
{
"type": "rating",
"instruction": "How do you like your new computer?",
"row_labels": [
"Overall speed",
"Speakers",
"Webcam",
"Weight"
],
"sequential": true
}
¶ step
{
"type": "rating",
"min": 0,
"max": 100,
"step": 10
}
Participants will see an 11-point rating scale with 0 as the min, 100 as the max, and every 10-point increment in between.
¶ target
Available since November 11, 2022 (Grammar Version 3.4.0)
- Definition: A single correct answer, or a list of correct answers, in the form of the underlying numeric values of the ratings (i.e., 1, 2, 3...). All target values must be within the range of the rating scale.
- Possible values: A single numeric value like
1, or a list of values like[3,1,2]in the case of a multi-row response. - Note: When target is a list, participants’ choices must exactly match all listed targets to be considered correct.
{
"type": "rating",
"max": 5
"target": 3 // if 3 is somehow defined as the "correct" rating
}
¶ target_sum
Available since November 11, 2022 (Grammar Version 3.4.0)
- Definition: The required sum of all rating scores selected by participants in a multi-row rating responses. If the sum is not met, the response will flash and prompt the participant to revise their ratings.
- Possible values: A numeric value that is between the smallest possible sum and the largest possible sum of a multi-row response.
- Note: As this feature will prevent participants from going forward in an experiment unless their ratings add up to the target sum, it is highly recommended that clear instructions be provided to participants.
{
"type": "rating",
"row_labels": ["Question 1", "Question 2", "Question 3"],
"max": 5,
"target": 10 // any combination that adds up to 10 will suffice
}
¶ zero_centered
{
"type": "rating",
"max": 6,
"zero_centered": true
}
Because zero_centered is set to true, the rating scale will be centered at 0, spanning from -6 to 6 (the value of max).
¶ Recorded Data
- value: the rating selected by the participant (always numeric)
- rt: reaction time, defined as the number of milliseconds between the start of the presentation of the scale and the moment when the participant responds (however, see the note for an exception in sequential).