Responses to ratings-based surveys are particularly prone to misinterpretation if the surveys don’t allow space for open-ended comments. These comments offer insight into what respondents are really thinking, which may not be obvious from their ratings.
Responses to ratings-based surveys are particularly prone to misinterpretation if the surveys don’t allow space for open-ended comments. These comments offer insight into what respondents are really thinking, which may not be obvious from their ratings.
I gained some personal appreciation of the importance of comments after one of my seminars. The evaluation form my client asked me to use requested ratings on a 10-point scale. When I reviewed the feedback afterwards, the evaluations of two particular participants caught my attention. One rated me a 10 on every survey item. (Yes, 10 was the top score!) The other gave me a string of 7s.
Clearly, one participant thought I had done a great job and the other considered me maybe a C+. But which was which would not have been obvious without their comments.
In particular, the person who gave me the 10s listed several things he’d have liked me to do differently. More of this, less of that, longer, slower, etc. By contrast, the person who gave me 7s offered glowing comments, including that this was the best class he’d ever taken.
I don’t know, maybe the 10-rater felt that even when something is done well, there’s room for improvement; I certainly appreciated his feedback. And maybe the giver of 7s had never before given anyone better than a 6. Without their clarifying comments, I’d have drawn the wrong conclusions about their ratings. This sort of situation makes me question whether ratings alone have any value, except to satisfy management's demand to see numbers.
If you use a rating system to solicit feedback, provide a place for comments at the end. If feasible, also request a brief explanation for some of the items rated. The feedback you receive will help you draw more meaningful conclusions than ratings alone, and that information will help you target, with precision, the things you’d like to change.
(Truth in blog writing compels me to admit, however, that I still like receiving 10s more than 7s ...)
