360 Degree Feedback Survey Response Scales

When a new 360 degree feedback survey is developed, one of the tasks is to determine which response scale is most appropriate. This Insight white paper describes some of the issues and criteria that should be considered, and provides several examples of possible scales.

Background and Criteria

• The scale is simple and easy to understand.
• To the extent possible, raters have a common understanding about what the scale means — they interpret the scale in a similar fashion.
• The words used to define the levels are clearly ordered (e.g., it may not be clear which is higher —"Excellent" or "Outstanding").
• The scale is suitable for the types of items in the survey (e.g., a scale where one of the levels is "Always" may or may not be appropriate, depending of the types of items).
• The scale leads to variance in ratings (competencies and participants are not all rated the same — there is a good spread in ratings).
• The scale is aligned with the main purpose of feedback (e.g., different scales would be appropriate when the purpose is developmental versus evaluating performance against expectations/ requirements).

Three other general conclusions can be drawn from the research:

1. Increasing the number of rating levels within a 360 degree feedback survey will, up to a point, lead to a greater spread in ratings (which is desirable). In most situations, having at least five levels is preferable. However, scales with more than seven levels can lead to more error being introduced.
2. The labels or anchor terms tend to confuse raters when there are more than six levels, and each level is labeled.
3. In many cases, regardless of the themes or number of levels used, raters tend to inflate their 360 degree feedback ratings, and there is less variance in ratings than one would desire.

General Types of Response Scales

• Simple qualitative — how would you rate the person in this area? (e.g., from "Poor" to "Excellent").
• Extent — to what extent does the statement describe the person? (e.g., from "Very little extent" to "Very great extent").
• Frequency — how often does the person demonstrate effective use of the behavior/skill? (e.g., from "Almost never" to "Almost always").
• Developmental (strength/development need) — how would you rate the person's capabilities in this area? (e.g., from "Significant development need" to "Exemplary; a role model").
• Comparison with others — compared to others with whom I have worked, this person is _____ in this area (e.g., from "Significantly below average" to "Best I've ever worked with").
• Performance — how would you rate his/her performance in this area? (e.g., from "Does not meet expectations" to "Far exceeds expectations"). Note that this theme would make sense for performance appraisal purposes, but probably wouldn't for a development-oriented 360 degree feedback process, where peers and subordinates also provide ratings.

It is possible to construct a response scale that has more than one theme. For example, an anchor point of "Excellent; best I've worked with" combines the qualitative and comparison themes. However, unless this is carefully done, there is a danger of the response scale becoming too complex, and thereby confusing.

As a final comment on general types of response scales, it is important that each level in a scale relate to the same theme(s). It is not appropriate, for example, that some levels of a scale relate to frequency, and other levels relate to capability (developmental). This is a common failing of many "home grown" response scales used in 360 degree feedback initiatives.

Anchor Points

5 - Exceptional strength
4 -
3 - Competent
2 -
1 - Weak

The theme for the 5-point scale above is developmental, and only three of the levels are anchored. It is highly questionable whether adding anchors for the second and fourth levels, using intermediate terms, would lead to more accurate and valid ratings. 360 degree feedback raters implicitly form a mental image of the scale and its meaning, and are able to easily understand that a rating of 4 is halfway between "Competent" and "Exceptional strength."

The response scale below is an example of a 5-point frequency scale where only three points are anchored:

Getting Started

When a new 360 degree feedback survey is being developed, it's usually advisable to pilot a draft of the survey in a focus group with typical raters. In addition to input solicited on the 360 survey items themselves, key questions regarding the response scale include:

• Do people interpret the scale the same way?
• Does it "feel right" to raters for the kinds of items in the survey?
• Does the scale work well for all rater roles that will be included in the 360 degree feedback process (managers, peers, etc.)?
• Will the scale provide adequate discrimination versus everyone being rated the same?

Appendix: Examples of Response Scales

© 2007 Censeo Corporation

Return to Downloads page

Learn more about 360 degree feedback