Developing an Effective Employee Survey

One of the important phases in an employee survey initiative is developing the employee survey instrument itself. No matter how well other aspects of a survey project are managed, if the employee survey instrument is of poor quality, only limited success will be achieved. This Insight white paper offers suggestions on how to develop an effective employee survey. See the companion Insight white paper, Tips on Making Employee Survey Initiatives Successful, for suggestions on a variety of survey issues beyond the employee survey itself.

The diagram below shows how the employee survey instrument fits into the overall employee survey process. The content in the survey obviously depends on the objectives for the survey, but you also have to consider the employee survey feedback reports, and the actions that will occur after the employee survey reports are delivered. This concept is discussed in more detail later in this document.

Steps in Developing an Employee Survey

The assumption here is that a new employee survey is being developed or an existing employee survey is being substantially revised. Later, suggestions are provided on the types of changes that are appropriate when an employee survey is being repeated.

The best approach to take in developing an employee survey depends on a variety of factors, but the principles underlying the steps listed below will form a good strategy in most circumstances. The steps are briefly described here, and then some of them are covered in more detail later in this document.

1. Determine the employee survey's purpose/objectives. When someone is tasked with developing a new employee survey, there's a strong tendency to start writing survey items, or to start collecting item pools from employee survey vendors. A better approach is to first achieve consensus, particularly among senior management, on answers to questions like, "Why are we doing an employee survey, and what are we trying to accomplish?" "What will be done with the employee survey results and who will have responsibility for following through?"
   
   A strong business case should be built before the project is initiated — how the employee survey will positively impact organizational effectiveness, commitment, retention, etc. The objectives can be broad or narrow, there can be several objectives or just one or two, and there are many different possibilities for how the employee survey results can be used… but the objectives and process must be clearly defined up front.
   
   
2. Identify key areas the employee survey should measure. At this point you're not trying to construct the survey items, or even the dimensions — you just need to identify the major topic areas. The employee survey should tie in with your business strategies and priorities. What can your employees tell you about how well the business is being run and how things can be improved? This may involve fairly broad areas or very specific areas such as a key corporate initiative. Most employee surveys also address areas that are related to employee commitment (engagement, satisfaction, morale, etc.). In particular, look for those factors that impact both quality of work life for employees and organizational effectiveness. For example, perceptions of good career opportunities impact retention, which, in turn, could be important to achieving long-term corporate strategies.
   
   There are several ways one could go about identifying the key areas the employee survey should measure, but collecting data from two stakeholder groups is common — (1) senior managers, to understand the business strategies and priorities, and those issues where employee perceptions are particularly important; and (2) HR leaders, who can provide insight on the areas important to commitment and retention. Sometimes it's also useful to conduct a few focus groups with employees at different levels to understand issues from their perspective.
   
   
3. Draft employee survey content. The employee survey content will include instructions, dimensions, items, the response scale, demographic variables, and open-ended questions. It's better to first identify the main dimensions for the employee survey (based on the outcome from Step 2), and then write the survey items to measure those dimensions. At this point it's appropriate to look for suitable employee survey content from vendors; there's no reason to start with a blank sheet of paper. However, be sure to add/ delete/modify items so the survey will be on target, especially with respect to the terms and language your employees will understand.
   
   Including open-ended items in an employee survey is usually a good idea for a couple of reasons. First, most employees appreciate the opportunity to add their comments in addition to rating the items. Second, the employees often have specific suggestions on how things could be improved, which are helpful during the action planning phase of an employee survey initiative. Be careful, though, not to have too many open-ended items, even if they are optional, because it could decrease the response rate.
   
   
4. Pilot test the employee survey. If most of the items you use are from a vendor, it may not be necessary to pilot the employee survey because the items should already be "tried and proven valid." But if there are a lot of new items in your employee survey, it's a good practice to conduct one or two focus groups to surface any items that might be ambiguous, not interpreted as having the same meaning, difficult to rate, etc. Ask participants to go through the survey and rate each item, flagging any that may be problematic. Debrief afterwards to collect their input.
   
   
5. Get senior management endorsement. Top-level support is critical to success of the employee survey initiative. In fact, the whole survey initiative should be a management-driven process rather than just another "HR program." That's important for success during all phases of am employee survey project, from increasing the response rate to action planning and making change happen based on the employee survey results. Getting senior management's "handprints" on the survey before it's administered will help build the support you need.

Criteria for an Effective Employee Survey

Considering the earlier discussion, it should be understood that the most important criterion is that the employee survey content is aligned with the survey objectives, and that the results help improve organizational effectiveness and employee commitment/engagement. In other words, the employee survey results should be actionable and enable the organization to make improvements — not just provide "nice to know" information. But there are other, more technical, criteria as well. It is beyond the scope of this document to provide an exhaustive treatment of what's actually a rather complex topic, but here are a few of the most important criteria for constructing good employee survey items:

• Clarity. Employees understand the meaning and generally interpret the item similarly. Keeping the reading difficulty level low and writing shorter survey items can help in this regard.
• Ratable. All employees are able to rate the item versus only some employees being familiar with the content area.
• Single focus. The item deals with one topic — not two or more. Statistical results for an item that addresses more than one topic will be ambiguous. For example, "My supervisor sets clear goals and provides positive reinforcement when employees meet the goals."
• Dispersion in responses. There is variance in how employees respond to the item. An item where almost all employees respond at either end of the scale doesn't tell us much about their perceptions, or differences among groups.
• Worded positively. All employee survey items are worded positively so that the same end of the response scale always indicates favorable ratings. A widely held myth is that mixing in some negative items will increase accuracy. Doing so will only confuse both the employees taking the employee survey and the people who have to interpret the survey results.
• Fit with response scale. All items are written so the chosen response scale is applicable and appropriate. Different response scales require different syntax for the items.
• Items and response scale not confounded. The response scale carries all the evaluative component, not the items. For example, with a frequency response scale (e.g., "always" to "never"), an item like, "My supervisor always sets clear goals," would be confounded.

Here are a few suggestions that pertain to the employee survey as a whole:

• Keep the survey as short as possible, and no longer than 100 items or so in any case. Really long employee surveys lead to two problems — employees run out of steam in completing them and the response rate is lower than desired, and people who get the employee survey reports are overwhelmed with too much information. Focus on the most important issues. It's better to cover them in some depth than to measure a larger number of areas more shallowly.
• Group the employee survey items into meaningful dimensions. This is particularly important when dimension results are reported back in addition to individual item results, which is usually the case. Dimension statistics will be meaningless if the dimensions are comprised of widely disparate items.
• Demographic variables (gender, tenure, etc.) are useful in analyzing differences in perceptions across types of employees. However, include only those variables that are absolutely essential to achieving your employee survey objectives. The more demographic variables you have, the more the employees will question the anonymity of their responses.
• Don't compromise your employee survey objectives in an effort to salvage the ability to make external normative comparisons. External norms are of limited value at best, and are meaningful only when the items are very similar to those used in the norm sample, and only when the response scale is identical. Internal norms and trend results are actually much more useful.

As a final comment on the criteria for good employee survey content, the author crafting the items and dimensions should also consider the strategy for the action planning that will occur after the employee survey reports are delivered. Ideally, the survey reports will not only highlight the most significant strengths and opportunities for improvement, but also give managers suggestions on actions they can take to make improvements. For example, Censeo's employee survey reports contain "Suggested Improvement Actions" linked to each employee survey item. While the lists of SIAs are by no means comprehensive, they are enormously helpful in getting line managers headed in the right direction. Employee survey initiatives often result in little real change occurring, and a major reason for that is managers not knowing what to do or how to do it. SIAs can help overcome those barriers.

Response Scales

• Use the same scale for all items in the employee survey. This makes it easier for both the employees completing the survey and those who will interpret the results.
• Use at least a 4-point scale, but no more than a 7-point scale. Fewer than four levels will not yield sufficient variance, and more than seven makes it hard to distinguish differences among the levels, and introduces more error.
  
• Include a mid-point in the scale. For example, with a 5-point agree-disagree scale, the anchors could be strongly agree, agree, neither, disagree, and strongly disagree. Many employees will have "in between" opinions on some issues, and will be uncomfortable when forced to make either a favorable or an unfavorable rating. Furthermore, without a mid-point, they will choose "agree" more often than "disagree," even though they really aren't favorable.
• Choose a summary statistic that will be easy to interpret. (This topic actually relates more to the employee survey reports than to the survey instrument, but you have to consider it as you develop the response scale.) Something simple, like percent favorable or percent unfavorable, works best. Line managers can easily understand the meaning of "65% of your employees were favorable on this item, whereas 80% of the employees in the comparison group were favorable." Other statistics, such as the mean, don't have that kind of intuitive meaning.
• Choose one type of summary statistic to display for each dimension and item — not multiple types. While a statistician might like to see employee survey results reported by percent favorable and unfavorable, mean, median, mode, standard deviation, etc., multiple statistics only confuse line managers, who are your primary target group for the employee survey.

Refer to the Insight white paper titled 360 Degree Feedback Survey Response Scales for a more in-depth treatment of this topic. Most of the points in that document are equally applicable to employee surveys.

Conducting Repeat Surveys

Employee survey programs are most useful when they are part of an ongoing measurement system, rather than just one-time snapshots. This is because trend results are the most powerful way of telling what's improving, and which areas require further attention or different interventions. Some organizations conduct employee surveys annually, but a cycle time of 18-24 months is more common.

Conducting a repeat employee survey doesn't mean the survey has to be identical to surveys used in the past. Some things can be changed, and some things can't be changed. With respect to the items, it's fine to add new items or drop some items, understanding, of course, that trend results for those items won't be available. Sometimes the strategy is to include selected items in every other employee survey. It's also okay to make slight wording improvements to the item syntax and still have accurate trend comparisons, as long as the meaning of the item doesn't change. Open-ended items are also often changed from one employee survey to the next.

The main thing that can't be changed is the response scale, because it's almost impossible to accurately calculate trend statistics. Even if it were possible to convert the scales from the current and past employees surveys to a common metric, the statistics lose their intuitive meaning and are hard to interpret.

Using Consultants in Developing the Employee Survey

© 2007 Censeo Corporation

Return to Downloads page

Learn more about employee surveys