Using External Norms in Employee Surveys: How Useful Are They, Really?

Employee surveys are conducted to measure employee perceptions about factors related to organizational effectiveness, and the degree to which employees are satisfied, engaged and committed. Presented with statistics about those perceptions and affective responses, managers and HR professionals need to understand which areas are positive and should be leveraged, and which areas are negative and should be improved. One approach is to compare the statistics from the company's employee survey with "external norms" — employee survey results from other companies. This Insight white paper addresses the issue of how useful those external norms really are, and how to properly interpret them when they are used.

What Are External Norms?

Employee survey vendors collect data on employee perceptions from different companies over time, sometimes on similar topic areas. The term "norm" refers to the average of those perceptions or affective responses to particular questions. They are "external" in the sense that they come from outside the company conducting a current survey. The underlying premise is that normative comparisons are helpful in understanding the results from the current survey.

Making normative comparisons is intuitively appealing. In many aspects of personal life and in business situations, we rely on such comparisons. When we list our homes for sale, we refer to market data to know what the price should be. When we determine what the compensation for a job should be, we collect data on what other companies are paying for similar jobs. If the turnover rate in our call center is 30%, we look externally in our industry to see if this is high or low.

Determining What's Positive and Negative

Before examining the advantages and disadvantages of using external norms in an employee survey, let's consider the fundamental issue managers and HR professionals face — how to determine whether the employee survey results for particular items and dimensions are positive, neutral or negative. That determination must be made regardless of the types of statistics that are reported (e.g., percent favorable, percent unfavorable, mean).

For the purpose of this discussion, assume that we're dealing with the employee survey report for Department A, and there are 60 employees who completed the survey. Also assume that the primary statistic reported is the percent favorable. The manager of the department is reviewing his/her survey report and trying to decide which areas are positive, so they perhaps can be leveraged, and which areas are negative so they can be improved. There are six basic ways to look at the data:

1. Absolute values. This is simply the percentage of employees who responded favorably, and it doesn't involve making any comparisons. If only 60% of the employees are favorable on an item like, "My manager provides recognition when people do a good job," that's probably an issue the manager would want to address regardless of how the statistic compared to anything else. Why? Because it's critically important to organizational effectiveness, morale and retention, and improvements can be made quickly and cheaply.
2. Internal normative comparisons. These are the gaps between the target group (Department A in this case) and some kind of internal comparison group — perhaps the total Division to which the department belongs. If the department's employees are significantly higher or lower than the comparison group, and there's not a clear reason why, then it should be flagged as something important to look at.
3. Relative scores. This involves examining which items and dimensions are most positive and most negative, or a ranking of some kind. In general, focusing attention on the most negative areas will yield larger returns than working on other areas.
4. Trend comparisons. This involves examining which items and dimensions have improved or deteriorated over time (and is possible, of course, only when a repeat employee survey is being conducted). This is one of the most valuable comparisons for two reasons: (a) the past employee survey is the most logical baseline for the target group, and (b) it's a good measure of how effective planned actions have been in addressing problem areas.
5. Subgroup comparisons. This involves the manager looking at the employee survey results for subgroups below the target group (e.g., organizational units that comprise Department A). The manager can quickly see where the strengths and weaknesses are in different parts of the organization.
6. External normative comparisons. This is the topic at hand, and it involves the manager comparing the perceptions of Department A employees with the perceptions of employees in other companies.

Employee survey reports typically present the basic statistical results for items and dimensions for the target group. Well-designed reports also present whatever comparisons are used in a manner that makes it very easy for people reading the reports to quickly discern the main points of interest, and do so clearly and concisely. The employee survey reports should help managers and HR representatives get beyond the statistics to a real insight as to the most important strengths and opportunities for improvement.

The Case For External Norms

These are the main arguments made in favor of using external norms in employee surveys:

• Norms are important in understanding what's really positive/negative. For example, if 40% of the employees are not satisfied with their pay, is that dangerously low and something that should be addressed? A company might feel differently if normative data indicated that it was on par with other companies, versus the average being in the 30% range.
• Norms help overcome a problem with ranking. This is related to the first argument, but normative data can help companies prioritize their actions and resource allocation. Employees are typically less favorable on some areas (e.g., pay satisfaction, opportunity for advancement) than they are on other areas (e.g., the work itself, several areas related to supervision). Normative data can provide a context for understanding such differences, so effort is not misdirected to areas that really aren't significant problems.
• Making normative comparisons is aligned with other ways the business is measured. In a highly competitive business environment, companies feel a compelling need to check themselves, not just against their past performance, but also how they're doing relative to the competition. And competition in the labor market isn't just with other companies providing similar products and services, but any company drawing from the same labor pool for talent.

Based on Censeo's experience in conducting hundreds of employee surveys in different industries, we suggest there is an additional argument that's often made (though not always admitted) for including external norms. The person responsible for the employee survey initiative knows the question that's going to come from senior management: "Yeah, we can see all that, but how do we compare with other companies?" Even if it's not a particularly good question, it's going to be asked, and the project manager doesn't want to be left without an answer.

The Case Against External Norms

These are the main arguments made against using external norms in employee surveys:

• Norms aren't accurate or comparable. Unless the items and response scales in the normative database and the current survey are identical, the comparisons won't be accurate. Furthermore, to be comparable, the norms should be from the same industry (or same labor market), the same size companies, the same time period (i.e., not too old), and so forth. The requirements for accuracy and comparability are very rarely met, and oftentimes the norms aren't even close.
• Norms aren't available for current items. The items in the survey should be targeted at the most important areas to measure, and be linked to the business strategies. It is a grave mistake to compromise the objectives for the survey initiative by including some items, and omitting others, just because normative data are available.
• "We have our own standards of excellence." Some companies simply don't care whether they're better or worse than other companies on most areas covered in the survey; their own standards of excellence are more important. They want to identify opportunities for improvement and strive for continuous improvement, regardless of what other companies are doing.
• There are better ways to tell what's positve/negative. The rationale here is that the other five ways of looking at the data (described previously) are preferable to external norms in focusing attention on the most important issues. This is especially true in interpreting the survey results for all organizational units below the total company.

People questioning the value of external norms also make the point that if a high percentage of employees are negative in an area, those employees are still negative regardless of the perceptions of employees in other companies. Going back to the turnover example, if the turnover rate could be decreased from 30% to 25%, the company would reduce costs by an enormous amount. That's true regardless of whether the norm in the industry is 20% or 40%.

So, Who Wins the Argument?

In our opinion, the value of external norms is overrated, and they sometimes result in companies achieving less, not more, positive change as a result of employee survey initiatives. Too often, companies don't ask the right questions, or use an inappropriate response scale, only because they want to make normative comparisons. Worst of all, norms can lead to complacency when managers and HR profession-als fail to act on improvement opportunities because they conclude, "We're no more negative than the average."

However, external norms are not without some value. Censeo collects normative data on 100 key items when clients use some of those items, with either identical wording or essentially the same meaning, and the identical response scale (5-point agree-disagree scale). In those cases, there can be a one-to-one normative comparison on equivalent items, but the greater value is being able to help clients understand the relative relationships among the items (e.g., on average, how much lower are employ-ees on opportunities for advancement than on satisfaction with the work itself?).

In fairness to the "For" side of the argument, we do point out that we have cast the whole discussion assuming that line managers at various levels in the organization are involved in the employee survey process — getting employee survey reports on their people, planning improvement actions, etc. When that kind of survey model is used, the external norms shouldn't even be shared with lower-level managers, because internal norm comparisons are much more useful, and it would be confusing to show both. However, if the purpose of the employee survey is only to take a temperature check of the total company, and not report results for lower organizational levels, then using external norms makes more sense. (Note that there can't be internal norms for the total company, but there can be for all groups below the total company.) However, we would strongly argue that trend comparisons, when they are available in repeat surveys, are far better than external norms.

If You're Going to Use External Norms…

Here are a few suggestions on using external norms in the most effective manner:

• Find the best norms you can, considering both accuracy and comparability, but don't get hung up trying to find the perfect norms. Other factors are much more important.
• If you can, and this is very difficult, find norms that are classified by employment status (exempt, nonexempt, hourly) and manager versus non-manager, in addition to industry, company size, etc.
• In addition to getting information on how employees in other companies responded on average, ask the survey vendor for data on how the best companies are doing.
• Focus on those survey areas where external comparisons will be most interesting. Issues related to pay, benefits and career opportunities are good examples.
• Report the norm comparisons only for the total company, and perhaps for major divisions, not for lower-level organizational units.

We offer two final comments on the subject of using external norms in employee surveys. First, the whole issue has become somewhat self-perpetuating, and for the wrong reasons. Employee survey process owners look for vendors who have norms because they think senior managers will ask the "but how do we compare?" question. Employee survey vendors build large normative databases, even though they may agree they're actually of limited value relative to other comparisons one could make. And then they tout the value of their norms, which further convinces clients that they need them. Second, in spite of the self-perpetuating point, the interest in external norms is decreasing each year. The reason is that companies are realizing they are less important than many other employee survey-related issues.

© 2007 Censeo Corporation

Return to Downloads page

Learn more about employee surveys