Measuring What Matters

Rethinking 360? Feedback Rating Scales

Moving Beyond the Limitations of a Five-Point Scale in 360-Degree Feedback

The traditional 360-degree Feedback uses a standard five-point scale, but this can be limiting and may restrict the true Feedback provided by employees.

When the rating scale in a 360-degree feedback tool is in a five-point range, the results are not always as helpful as intended. Here’s an explanation of why a five-point scale may have limitations and an alternative scale that can provide more differentiated and more substantial results.

1. Lack of Differentiation:

A five-point scale, typically ranging from “strongly disagree” to “strongly agree” or from “poor” to “excellent,” provides limited differentiation between performance levels. It lacks nuance. With only five points, it can be challenging for raters to distinguish between different performance levels accurately and effectively. This lack of granularity can result in less precise Feedback and may create challenges in identifying specific areas for improvement.

2. Central Tendency Bias:

A five-point scale often leads to a central tendency bias, where raters tend to avoid extreme ratings and gravitate towards the middle of the scale. This bias can reduce the usefulness of the feedback-by-feedback ratings around the average, making it challenging to identify high performers or areas needing improvement. A review of the five-point scale concluded that the mean score was 3.8 with a standard deviation of 0.6.

3. Difficulty in Differentiating Strengths and Weaknesses:

A five-point scale may not provide enough range to differentiate between the different strengths and weaknesses of an individual. It can limit the ability to capture nuances and specific areas of excellence or developmental needs.

An alternative to the five-point scale is a behaviour-based scale, often called a behaviorally anchored rating scale (BARS). This scale includes a range of behavioural indicators associated with performance based on frequency, allowing providers to provide more accurate and meaningful Feedback. BARS focuses on specific observable behaviours and provides a more comprehensive and detailed performance assessment.

The fix:

1. Enhanced Differentiation:

A behaviour-based scale provides a broader range of specific behavioural indicators associated with different performance levels. This allows raters to differentiate between various performance levels more effectively, providing more precise and actionable Feedback.

2. Eliminate Tendency Bias:

A behaviour-anchored scale helps mitigate central tendency bias by focusing on observable behaviours. Raters are prompted to provide ratings based on the frequency of specific behaviours being demonstrated, rather than general impressions, reducing the tendency to cluster around the middle of the scale.

3. Clearer Identification of the scale using frequency:

A behaviour-based scale allows for a more nuanced assessment of strengths and weaknesses. Detailed behavioural indicators make identifying specific areas of excellence and developmental needs easier, facilitating targeted feedback and improvement plans.

4. Training of Feedback Providers and Receivers

Because of the issue of trust in the process stemming from concerns about confidentiality and anonymity, you need to communicate why the system is safe. All the safeguards may be written out in detail on the web-based application, but did they read it? Filling out the form, and getting it completed, is the mission. The person knows how to complete a Likert- scale. Most just jump in and complete the Feedback without reading the information provided. Since human nature is to avoid harm, the ultimate bad Feedback, a score of one is mainly avoided. Yet, the feedback receiver needs to learn the honest perceptions of others. Over the years, we discovered that taking the feedback providers through a one-hour lunch and learn helps to alleviate the concerns. The session focuses on why honest Feedback is critical, how anonymity is protected, and why the scores are based on frequency, not recency or how well the person exhibited the behaviour one time in the past.

In short, the five-point scale does not provide enough granularity to capture the full range of Feedback. It limits the ability to distinguish between different levels of performance or attitudes. The scale providing the most benefit is the 10-point scale, as it differentiates based on the frequency of the action being demonstrated. Here is an example that our clients frequently use:

I usually refer to the Feedback as a gift quoting Robert Burns “Oh, what a great gift it would be if I could only see myself as others see me” If people are not honest, it is not a very meaningful gift.

Additional Sources:

Landy, F. J., & Farr, J. L. (1980). The measurement of work performance: Methods, theory, and applications. Academic Press.

Smith, P. C., Kendall, L. M., & Hulin, C. L. (1969). The Measurement of Satisfaction in Work and Retirement: A Strategy for the Study of Attitudes. Rand McNally.

Mark, Edwards, R., and Ewen, Ann J; 360 Degree Feedback: The Powerful New Model for Employee Assessment & Performance Improvement, AMACOM (May 20, 1996)

________________________________________________________________

Dr. David S. Cohen, Ed.D. has worked with companies on feedback and performance improvement for over 30 years. He helps build feedback tools specific to the behaviours of the company culture and the job families. His approach assists the feedback receiver with more focused and meaningful data and helps the feedback providers by providing Feedback about behaviours they most likely have experienced.