Even though I’m in the survey business, I don’t generally participate in phone surveys because, quite frankly, they usually take too long and often interrupt my family’s dinner hour. Last night, though, I received a call from a nationally recognized polling service regarding a recent shopping experience. I almost hung up the phone with the usual polite, but definitive “No thank you”, but I had recently had a poor shopping experience with this particular retailer and I wanted to air my grievances.
So I agreed to take the phone survey and proceeded to answer the survey administrator’s long list of yes/no and ratings scale questions. As the survey progressed, I became more and more frustrated. I kept thinking to myself, what do these questions really have to do with me or my shopping experience? I was answering questions like “Do you trust your sales associate’s opinion when shopping?” and “Were you greeted warmly upon entering the store?” At least half of the questions were ones that I honestly could not care less about. I actually laughed out loud when asked “On a scale of 1-5 how much do you agree with this statement: Retailer XYZ is the perfect place for people like me?” Seriously? All I could think was, what do these people know about me? What does ‘people like me’ even mean?
The worst part was that this survey, which was supposed to be about my recent shopping experience, didn’t even address why I was disappointed in that shopping experience until the last two questions. And even then, I only had one open ended answer to describe my frustration. So I hung up the phone feeling like I had completely wasted my time. I spent 15 minutes giving feedback that was essentially meaningless. Sure, I had given the retailer high marks on most questions, but they were mostly questions that really had no bearing on my decision to shop at that store. So, I’m sure those who read my survey will pat themselves on the back for a high rate of customer satisfaction, but they will be missing the mark in a big way.
What’s the point of racking up high satisfaction marks from your customers or employees or other key groups if those numbers are inflated or insignificant? No matter what survey method you use, it’s so important to ask questions that are relevant to your survey respondents. One way to do this is to start with broad questions to get an overall idea of a respondent’s opinion or circumstances and then narrow down to more specific questions based on those answers. Another critical tactic, and one that the creator of my survey missed, is to ask not just how a respondent rates a certain experience or item but also how that rating impacts her decision to purchase, visit, etc. again.
I’m also an advocate of giving respondents significant time and space for open-ended answers. Sure, open-ended responses are difficult to analyze and report on, so I wouldn’t suggest over-using them. But they can also be a treasure trove when it comes to finding out what a respondent really thinks. Analyzing open-ended responses can also help highlight the questions you shouldn’t be asking and the ones that you really need to add. They also help qualify your respondents; those that take the time to fill out open-ended responses are generally more invested in the topic you’re surveying and therefore extra worthy of your attention and follow-up.
There are a lot of ways to ensure that your surveys are relevant and meaningful; I’m only scratching the surface on that topic. While market research and survey design are art forms that take time to perfect, the most important question you can ask isn’t one that will ever appear on your surveys. It’s, “So what?” What do your survey scores really mean in terms of how your customers, employees, etc. really feel about you? Retailer XYZ might be thrilled that I rated their sales associates a 5 in terms of trustworthiness, but they need to ask “So what?” in order for that data to be meaningful. Trustworthy sales associates aren’t going to make me walk back into that store after a poor shopping experience, and that’s really the bottom line.