Is Mystery Shopping a Waste of Money?

Table of Contents

We’ve spoken with hundreds of organisations while building TRACX; from independent retailers to multinational corporations. Business owners and CX leaders told us about their frustration with mystery shopping. They are unhappy with the limitations and want better ways to measure customer experiences.

In this post, we’ll discuss why mystery shopping is no longer a good choice for businesses that care about customer experience. But first, let’s take a look at how the mystery shopper market boomed into the $2bn industry it is today.

The History of Mystery Shopping

Though the term “mystery shopping” was first used in the 1940s, the mystery shopping industry as we’d know it didn’t really take off until the 1970s, when retailers and financial services started to benchmark sales and marketing performance using yearly reporting from mystery shoppers. The primary goal changed from monitoring employee dependability to measuring customer service experience.

In the 1980s, businesses began to focus more on the quality of service, and Mystery Shoppers became the most common technique to assess it. The survey methodologies became more advanced, with researchers starting to ask highly specific questions and developing different standards and norms to establish a benchmark for service quality.

The trend continued into the 90s with the first publication of the American Customer Satisfaction Index (ACSI), the only national cross-industry customer satisfaction index in the United States, in 1994. Before the ACSI, no nationwide measure of quality from the user’s perspective existed.

The Mystery Shopper Industry Today

Fast forward to today, and mystery shoppers aren’t such a mystery anymore. The industry has ballooned into a $2bn market, with over 1.5 million active mystery shoppers worldwide.

Businesses of all sizes use mystery shoppers to measure service quality, ensure compliance with brand standards and regulations, and assess customer experience.

The industry has not escaped digital transformation and the wave of disruption it brings. Mobile devices and online surveys are replacing older methods, such as paper surveys.

Today, 9 out of 10 businesses compete mainly on customer experience. It’s no wonder that companies are desperate to find new ways to measure and improve their customer experience. Mystery shopping became an industry standard, and many companies turned to it to get the answers they needed.

But the market is changing rapidly and we’re seeing a shift in how businesses view the use of mystery shoppers as part of their CX strategy. In a recent report, 23% of businesses surveyed found mystery shopping of little or no use.

The UK Customer Experience Decision-Makers’ Guide 2020-21

The Problems with Mystery Shopping

Despite widespread use, many businesses are finding critical flaws when it comes to using mystery shoppers to measure customer experience.

To remain competitive, business leaders today demand actionable insights based on genuine consumer feedback. They want to understand how they’re performing against their competitors and what’s driving the customer experience.

Widespread smartphone adoption and unlimited data plans have put the power back into the hands of customers. They have access to more information than ever before, allowing them to compare businesses and make informed decisions.

The same technologies that empower customers are enabling businesses to collect feedback in real-time, and to analyse it immediately. It’s now easier than ever to understand what real customers are thinking and feeling.

This is where mystery shoppers begin to fall short. But it’s not the only reason that businesses are starting to question the use of mystery shoppers. ARB has written at length about the problems with secret shoppers in retail banking.

Here are just some of mystery shopping’s problems that businesses have identified:

Mystery shoppers are researchers and auditors, not real customers

The most common complaint we heard from CX professionals was that mystery shoppers were not real customers.

Since they’re not real customers, mystery shoppers can’t provide feedback on how real customers perceive and experience the brand. Instead, they provide feedback on how employees perform when a mystery shopper is present.

Mystery shopping audits measure compliance with brand standards, not customer experience.

Mystery shopping (MS) is a widely used tool to monitor the quality of service and personal selling. In consultative retail settings, assessments of mystery shoppers are supposed to capture the most relevant aspects of salespeople’s service and sales behavior.

Given the important conclusions drawn by managers from MS results, the standard assumption seems to be that assessments of mystery shoppers are strongly related to customer satisfaction and sales performance. However, surprisingly scant empirical evidence supports this assumption.

We test the relationship between MS assessments and customer evaluations and sales performance with large-scale data from three service retail chains. Surprisingly, we do not find a substantial correlation.

The results show that mystery shoppers are not good proxies for real customers.

Do Mystery Shoppers Really Predict Customer Satisfaction and Sales Performance?
Gerald Blessing, ESB Business School, Reutlingen University, Germany
Martin Natter, Department of Business Administration, University of Zurich, Switzerland

Mystery shopper reports can take weeks to complete and deliver

Despite technological improvements, this continues to be a frustration for many companies.

Mystery shoppers are not able to provide real-time feedback, which means businesses have to wait weeks for a report that may or may not be relevant to the current customer experience challenges.

The time it takes to get feedback from mystery shoppers is a major barrier to using them as part of an agile CX strategy.

Mystery shopping is too expensive for most businesses

Most companies that we spoke with performed monthly or quarterly monthly shops.

The cost of a mystery shop is generally set on a “per evaluation basis”. The final price will varying greatly depending on the type of business, its location, and the complexity of the task. This can add up quickly, especially if a business wants to measure more than one experience or customer segment.

We spoke with a major high street brand with over 100 locations across the UK who told us that their mystery shopper program required 2 visits to each location, every month. The brand used mystery shoppers to gather insights on their in-store experience, however, they represented less than 0.01% of the real customers who made a purchase every month.

Employees will try to identify and manipulate mystery shoppers

This is a problem for any customer feedback mechanism, but it’s particularly problematic with mystery shoppers.

Employees will try to identify mystery shoppers and adapt their behaviour as a result. At best, your data will be manipulated. At worst, playing “spot the mystery shopper” will distract employees from serving customers.

This defeats the purpose of mystery shopping as a means to measure the customer experience.

You can’t change the mystery shopping survey once the audit begins

This complaint was raised numerous times by companies we spoke with.

Once a mystery shopping program is underway, it’s often impossible to make changes to the survey. Businesses will have to wait until the next round of mystery shops to address questions that need to be changed.

Without the ability to make changes, businesses are stuck with reports that don’t represent their customer experience strategy and goals.

Alternatives to Mystery Shoppers

The good news is that there are plenty of alternatives to mystery shoppers.

Omnichannel customer experience platforms mean that businesses can measure their CX in real-time, across all channels and customer touchpoints.

Rather than relying on a couple of mystery shopping evaluations, companies can now get feedback from hundreds or thousands of visitors every month.

This data is not only more accurate than mystery shopper feedback, it’s also available immediately after the interaction occurs. There are no delays between collecting the feedback and learning from it.

This is a breakthrough in customer experience research and has led many companies to question the value of mystery shoppers. In fact, they’re now looking at ways to use these new tools as part of an agile customer experience strategy.

There are many new technologies that businesses can use to get feedback from customers, including:

Customer Experience Management Platforms

Customer experience management platforms (CXM) are one of the most effective tools that businesses use to measure their customer experience.

These platforms provide real-time feedback, segmentation, workflow management, action plans (and alerts), staff training modules (including videos) and more. They help businesses to understand their customer’s needs and wants, and track initiatives to improve the customer experience.

Because of these advantages, CXM platforms have quickly become an essential tool for most business’ customer experience strategies.

Social Media Listening Tools

Another excellent way to measure the customer experience is through social media listening tools like Brandwatch, Mention, and Brand24.

These tools allow businesses to see what customers are saying about them on social media. They also provide insights into customer sentiment, as well as the overall tone of the conversation.

The downside of social media listening tools is that they don’t enable you to ask the questions. You can only listen passively to what people are saying online.

Survey Platforms

Customer feedback surveys are a great way to gather insights that can then be used to improve customer experiences.

Survey platforms, like Google Forms, are an easy way to collect feedback from customers about their experience with your business. The most effective survey platforms allow businesses to create customised, targeted questions and create surveys tailored to their needs.

You can get more reliable data than mystery shoppers or social media listening tools, but you need to analyse the data before you can make sense of it. This can be challenging, especially if you’re trying to analyse hundreds or thousands of responses.

QR Code Surveys

QR surveys are a new way to gather customer feedback that’s becoming very popular.

Well suited to brick and mortar businesses, QR code surveys can be placed in physical locations, like windows, menus, or store displays. They provide businesses with data about what their customers enjoy (and don’t) about the experience they have at a business or in specific locations/venues.

The main challenges with QR code surveys are accurately tagging the feedback for analysis and managing the QR codes themselves. Because of this, it’s recommended to use a feedback platform that can manage the QR codes for you.

Employee Feedback Tools

One of the most overlooked sources of customer experience data is employee feedback.

Employee engagement tools don’t just offer insight into how happy employees are with their roles, the company culture, and management. They also provide customer-facing employees with a direct line of communication to share feedback about the customer experience.

By talking to your employees you can learn what they think about customers and what they would like to improve.

Customer Journey Mapping Tools

The final tool we’d recommend is customer journey mapping, which allows businesses to understand the customer experience in its entirety.

Customer journey mapping forces businesses to identify all interactions a customer has with your business, from first contact to post-purchase. You can then begin collecting real customer feedback at key touchpoints to better understand and improve the experience.

Conclusion

Mystery shopping may soon be a thing of the past. The bottom line is that mystery shoppers are no longer the best way to understand and improve customer experience.

The downsides with using mystery shoppers outweigh the benefits. Mystery shoppers have several problems, including their inability to provide real-time feedback, the tiny sample sizes, the costs involved, and the fact that employees can manipulate them.

With businesses able to monitor customer sentiment in real-time on social media and review sites, and collect genuine customer feedback at key touchpoints, there is no need to rely on expensive and outdated solutions like mystery shopping.