The Feedback Form Fallacy – 8 Reasons why you Shouldn’t Use Feedback Forms in Workshops

Amnon Levav

In SIT Innovation - Systematic Inventive Thinking ? , we often discourage the use of feedback forms in training activities. To clarify: we do believe that it is useful to ask participants what was useful for them and what they think requires improvement. This can refer to logistics, venue and the like, and it is also useful to ask about intentions of using the learnings and the kind of support they think they need in order to do so. It is, on the other hand, not useful to ask for participants' general impression of the trainer(s) and especially not to grade them.

There are several reasons for this:

1. We encourage our facilitators to judge themselves by results, rather than by the opinions of participants. What participants think of you is of course important, but it is reflected in the results anyway.

2. It is often said that judging an activity by its results is OK for so-called "result-oriented projects", say developing a pipeline of novel products, but more tricky for pure training. True, it is more difficult to find the right parameters to measure a training activity, but this does not necessarily mean that an average of participants' grades for their trainers is a useful proxy. In fact, since it is often difficult to define concrete expected results for an innovation training (as the results are "in the brain"), achieving a high score on feedback forms becomes the goal for a trainer, which incentivizes feedback-manipulation rather than goal-oriented behaviors.

3. There are quite a few reasons why our experience shows that using grades as proxy is often misleading and can lead to bad decisions on the part of the facilitator, consciously or not:

• A relatively small number of participants can influence the average strongly up or down (and an average is really not a significant indicator anyway). In a group of 16, for example, of which 10 fill the form, one angry participant can easily lower your average by 0.5-1 points out of 10.
• This is especially the case when – as often happens – only part of the participants fill in the feedback forms;
• Grades put emphasis on differences that in real life mean very little if at all ("I got 8.3, he got only 8.1");
• Grades discourage trainers from taking risks, in the fear of angering even one or two participants. Trainers are incentivized to keep everyone happy, rather challenging them.
• Some specific questions are especially misleading, since results can be interpreted in conflicting manners. If someone gave a "6" in answer to the question how satisfied she is by the length of the training, does it mean that she thinks its "way too long and boring", or "so fascinating I would really have wanted two more days"?
• Another misleading result of grading, when there are multiple trainers, is the comparative grading of two trainers who often have very different roles, unbeknown to the participants.

4. The most important reason is that, as we all know, what gets measured is what gets emphasized and delivered, and since trainings (should) have goals, and their goals are not to make participants happy, it is much more important to educate and encourage trainers to achieve the real goals rather than to aim to be popular with the group. Although this is often a necessary condition for delivering the objectives, it is definitely not sufficient, and therefore placing it at the center of attention creates a distorted incentive.

To summarize: yes, it is useful to learn from participants' experience in a training activity, but no, the best way to do so is not by asking them to grade their trainers. Some constructive ways to do so are:

1. Start by defining precise behavioral changes you wish to achieve and make these your objectives. How and how much do you want them to use their newly acquired learnings.
2. Follow up on how and to what extent participants utilize what they have learned. Do so 2 weeks after the training ends, and also 4 and 6 weeks later.
3. Ask participants to write in their own words, or record themselves saying, what they think they plan to do with what they learned, and what they think they still need in order to do so.
4. Ask participants for their opinions on technical and logistical aspects, with an emphasis on what should be improved.
5. Give participants the space and time to add their thoughts freely.
6. Avoid grades.

?All this obviously implies additional work for those in charge of the training, but, if their objectives and goals are defined correctly, i.e. leading company associates to use what they are trained to use, rather than simply "running" training activities, then this extra time and effort will be worth the extra time and efforts.