Brief

The Value of Prototypes

Creating a pilot or prototype is an essential part of designing a robust, scalable Net Promoter System.

By Rob Markey

First published on Μαΐου 14, 2014
min read


Cast an eye on any modern office and you will see the influence of Herman Miller. There’s the high-design seating, often the company’s own Aerons or Eames executive chairs. The modern open plan office—forerunner of the cubicle—was created by Herman Miller’s Robert Propst way back in the 1960s, and the company continues to be a leader in the office landscape. The Michigan-based office-systems maker is an innovator par excellence, so when it decided to launch a Net Promoter System^SM, it followed the usual protocol for any new venture: It designed prototypes. “We knew going in that there was a boatload we didn’t know,” says Michael Ramirez, the senior executive responsible for NPS® implementation. One thing Herman Miller’s team hadn’t anticipated was the sheer quantity of work involved in closing the loop with customers. Launching the prototype, the team selected 6,000 buyers and sent off three batches of emails asking for feedback, one batch each week. Between 40 and 50 respondents from the first batch raised concerns that required a call-back—a lot of people, given that the team wanted to respond right away. “We were a little overwhelmed in that first week,” remembers Pam Carpenter, who led the prototyping effort. “Everybody on the pilot team basically had to stop doing any other work.” Another early lesson: Since callers rarely reached customers right away, they left voicemails. But Herman Miller didn’t have a roaming call center to handle call-backs, so most team members gave customers their personal cellphone numbers. The result? Return calls from California customers, say, at 11:30pm Michigan time. (Ramirez and Carpenter discuss their experience with prototyping in a podcast available on www.NetPromoterSystem.com.) Designing any Net Promoter System is a challenge. You’re trying to create a new pattern of information flows and action, one that fits seamlessly and inexpensively into your company’s daily operations. It’s virtually impossible to do that—to get all the details right—without employing prototypes as part of the design process. Prototypes serve a crucial purpose in the journey: they reveal issues, opportunities and design details that people sitting at their desks can’t anticipate. They offer an essential tool for building robust, scalable and customer-friendly Net Promoter Systems that will have greater impact on customers and employees. Take the most basic Net Promoter process, asking customers for feedback. Right away, you’ll find yourself confronting dozens of design questions. Which customers will you ask? How will you contact them? Which touchpoints should trigger a request for feedback? What questions will you ask? How will they be worded? If you’re requesting feedback through email, what should the subject line say? Should you send a reminder email? When should you stop accepting responses? The strength of your Net Promoter System rests on finding good answers to questions like these. The answers will determine how many customers respond to your queries, how much you’ll learn from them and how valuable that information will be. You can’t know all the right answers in advance, which is why you have to create and test hypotheses. Martha King, who leads Vanguard Group’s Financial Advisor Services division, recently launched prototypes as part of an effort to extend Vanguard’s Net Promoter approach to her unit. (She discusses the effort in another podcast.) “Are we tagging the right interactions?” she asks. “Are we sure we’ve identified the moments that matter to these advisors? . . . We’re going to learn from the pilot about that.” (For more on what to call your experiments, see below, “Terminology: Pilot vs. prototype.”) And it isn’t just gathering feedback. It’s knowing how to use it. Once you get the feedback, who should see it? In what form? What information needs to be passed from your customer relationship management (CRM) system into the feedback system—about the transaction, the product and the customer? You can answer many such questions just through careful planning and scenario building, but there are some you won’t think of until you begin a prototype. One set of questions is particularly critical: Who should make follow-up calls to close the loop with customers? What criteria should determine that? You may think that supervisors or unit managers should make the calls, but you will need to determine whether the expected call load is reasonable, given the other demands on their time. Maybe the reps themselves should make the calls to most customers instead. Also, do your callers have the skills to manage the interactions with customers successfully, or will they require additional training? How will they get the information they need to maximize the chance of a successful outcome? Herman Miller found that even customers who hadn’t offered much in their original feedback had detailed observations to share once they were talking to a Herman Miller employee on the phone. Yet many employees handling the follow-up calls didn’t have the technical knowledge or skills required to address customers’ concerns without first doing a good deal of research— another key lesson. Companies usually prototype both “relationship” NPS, which gauges the strength of a customer’s overall relationship to the company, and “touchpoint” NPS, which assesses a customer’s satisfaction with specific interactions (see Figure 1). They are likely to run more than one prototype in each area, gradually adding challenges and complexity (see the sidebar, “Tips and tricks for successful prototyping”). Typically, each prototype runs for a couple of months. That’s a function of the cycle time involved: it takes a certain amount of time to get the feedback, process it, investigate root causes and learn how to respond. Each team needs to go through a couple of cycles before it has the information required for scaling up the process. Companies often run multiple overlapping prototypes, starting each one a few weeks after the previous one and comparing results at the end (see Figure 2). That makes for a faster learning curve. Criteria for success A prototype is a kind of experiment, so you naturally want to figure out in advance what you hope to learn from it. A successful prototype should produce at least the following results: Identification of customers whose feedback needs to be solicited, along with specific triggers for soliciting it Baseline results for response rates and quality of feedback, along with specific techniques for improving response Insight into what kind of information needs to flow through the system Learning how to close the loop with clients, including who will do it, when, how they will get the information required and how they will track down root causes of complaints Learning how to deliver the customer feedback to employees who need it and how they can take action to address systemic concerns Enough experience so that members of the central NPS team can see what kind of support they need to offer—and so executives can see what investments will be necessary for success at scale Enthusiasm among the employees participating in the prototype about the potential for the full-scale Net Promoter System to make a real difference for the business In short, once your prototypes are complete, you should have all the information you need to design a scaled-up, reliable, low-cost system. Moreover, results from the prototypes will help you calculate the economics of improvement and build a business case for rolling out the system. We have written elsewhere about the infrastructure required for a mature system; successful prototyping is the interim step that helps you decide what that infrastructure should look like and how it should operate if you are to accomplish your goals for improving your customers’ experience. One last lesson: Successful prototype design requires extensive input from the individuals who will be involved. For example, frontline employees and supervisors often have a gut feel for which interactions are real “moments of truth,” creating promoters if they go well and detractors if they go poorly. These touchpoints will be the best triggers for a request for feedback. One property and casualty insurer brought employees to a workshop to identify the key interactions. The prototype, members decided, would focus on just two: first contact after a loss and settlement. In a B2B situation, the key question for a workshop may not be touchpoints; instead it may be which people in the customer’s organization to survey. Purchasing agents? Finance executives? Users of the product? Sales and customer-service reps can draw up account maps for each customer organization involved in the prototype, analyze individuals’ roles and identify the appropriate people to ask for feedback. Creating an effective Net Promoter System is hard work, mostly because it has to be tailored to an individual company’s business, culture, operational processes and competitive situation. Prototypes let you design your system step by step before you scale it up. They ensure that you’ll learn the right information, channel it to the right people and make your customers’ experience as good as it can possibly be. Rob Markey, global practice leader of Bain's Customer Strategy and Marketing practice, shares some best practices for prototypes in this short video. Tips and tricks for successful prototyping Experienced Net Promoter practitioners have learned a variety of lessons from prototyping the system. For example: Plan on more than one prototype. Schedule at least two. That way, you’ll get more than one shot at refinement and improvement of the system, and you can add complexity in a controlled fashion. Pick your spots. The first prototype should offer the highest likelihood of success—a simple situation with straightforward issues, a good leader, a well-staffed team and so on. Follow-on prototypes should add layers of complexity and difficulty so you can sort out the tougher issues. Increase the level of difficulty only after you’ve mastered the first. Incorporate technology as soon as possible. You can map out plans for a prototype with an Excel spreadsheet or pencil and paper. But bring in whatever technological support you intend to use as soon as possible. Ultimately, you’ll need to work with your company’s CRM data, operational databases or other systems. It’s important to learn early on about the challenges you’ll face. Involve senior executives. There’s a risk in any prototype that middle managers’ natural conservatism will hobble what gets tested and how the information is used. At one financial services company, the supervisors in a call center were reluctant to share feedback from clients with call-center reps because they feared it would be demoralizing. A senior leader had to point out that not sharing the feedback defeated the system’s purpose. Get regular feedback on the prototype from the staff. Don’t make every decision by yourself. Frontline staff and supervisors will have a treasure trove of observations about how the system could work better. Terminology: Pilot vs. prototype Many companies refer to their early Net Promoter experiments as pilots. That’s a fine term, but we prefer prototype, because it implies learning and development over time. Like a product prototype, an NPS prototype elicits low-cost, real-world feedback from the people who will use it before you have scaled it up into a full-blown system. Each subsequent prototype reflects the learnings from previous ones and more closely approximates the final scalable solution. Rob Markey is a partner and director in Bain & Company’s New York office and leads the firm’s Global Customer Strategy and Marketing practice. He is coauthor of the bestseller The Ultimate Question 2.0: How Net Promoter Companies Thrive in a Customer-Driven World.

Cast an eye on any modern office and you will see the influence of Herman Miller. There’s the high-design seating, often the company’s own Aerons or Eames executive chairs. The modern open plan office—forerunner of the cubicle—was created by Herman Miller’s Robert Propst way back in the 1960s, and the company continues to be a leader in the office landscape. The Michigan-based office-systems maker is an innovator par excellence, so when it decided to launch a Net Promoter System^SM, it followed the usual protocol for any new venture: It designed prototypes. “We knew going in that there was a boatload we didn’t know,” says Michael Ramirez, the senior executive responsible for NPS® implementation.

One thing Herman Miller’s team hadn’t anticipated was the sheer quantity of work involved in closing the loop with customers. Launching the prototype, the team selected 6,000 buyers and sent off three batches of emails asking for feedback, one batch each week. Between 40 and 50 respondents from the first batch raised concerns that required a call-back—a lot of people, given that the team wanted to respond right away. “We were a little overwhelmed in that first week,” remembers Pam Carpenter, who led the prototyping effort. “Everybody on the pilot team basically had to stop doing any other work.”

Another early lesson: Since callers rarely reached customers right away, they left voicemails. But Herman Miller didn’t have a roaming call center to handle call-backs, so most team members gave customers their personal cellphone numbers. The result? Return calls from California customers, say, at 11:30pm Michigan time. (Ramirez and Carpenter discuss their experience with prototyping in a podcast available on www.NetPromoterSystem.com.)

Designing any Net Promoter System is a challenge. You’re trying to create a new pattern of information flows and action, one that fits seamlessly and inexpensively into your company’s daily operations. It’s virtually impossible to do that—to get all the details right—without employing prototypes as part of the design process. Prototypes serve a crucial purpose in the journey: they reveal issues, opportunities and design details that people sitting at their desks can’t anticipate. They offer an essential tool for building robust, scalable and customer-friendly Net Promoter Systems that will have greater impact on customers and employees.

Take the most basic Net Promoter process, asking customers for feedback. Right away, you’ll find yourself confronting dozens of design questions. Which customers will you ask? How will you contact them? Which touchpoints should trigger a request for feedback? What questions will you ask? How will they be worded? If you’re requesting feedback through email, what should the subject line say? Should you send a reminder email? When should you stop accepting responses?

The strength of your Net Promoter System rests on finding good answers to questions like these. The answers will determine how many customers respond to your queries, how much you’ll learn from them and how valuable that information will be. You can’t know all the right answers in advance, which is why you have to create and test hypotheses. Martha King, who leads Vanguard Group’s Financial Advisor Services division, recently launched prototypes as part of an effort to extend Vanguard’s Net Promoter approach to her unit. (She discusses the effort in another podcast.) “Are we tagging the right interactions?” she asks. “Are we sure we’ve identified the moments that matter to these advisors? . . . We’re going to learn from the pilot about that.” (For more on what to call your experiments, see below, “Terminology: Pilot vs. prototype.”)

And it isn’t just gathering feedback. It’s knowing how to use it. Once you get the feedback, who should see it? In what form? What information needs to be passed from your customer relationship management (CRM) system into the feedback system—about the transaction, the product and the customer? You can answer many such questions just through careful planning and scenario building, but there are some you won’t think of until you begin a prototype.

One set of questions is particularly critical: Who should make follow-up calls to close the loop with customers? What criteria should determine that? You may think that supervisors or unit managers should make the calls, but you will need to determine whether the expected call load is reasonable, given the other demands on their time. Maybe the reps themselves should make the calls to most customers instead. Also, do your callers have the skills to manage the interactions with customers successfully, or will they require additional training? How will they get the information they need to maximize the chance of a successful outcome? Herman Miller found that even customers who hadn’t offered much in their original feedback had detailed observations to share once they were talking to a Herman Miller employee on the phone. Yet many employees handling the follow-up calls didn’t have the technical knowledge or skills required to address customers’ concerns without first doing a good deal of research— another key lesson.

Companies usually prototype both “relationship” NPS, which gauges the strength of a customer’s overall relationship to the company, and “touchpoint” NPS, which assesses a customer’s satisfaction with specific interactions (see Figure 1). They are likely to run more than one prototype in each area, gradually adding challenges and complexity (see the sidebar, “Tips and tricks for successful prototyping”). Typically, each prototype runs for a couple of months. That’s a function of the cycle time involved: it takes a certain amount of time to get the feedback, process it, investigate root causes and learn how to respond. Each team needs to go through a couple of cycles before it has the information required for scaling up the process. Companies often run multiple overlapping prototypes, starting each one a few weeks after the previous one and comparing results at the end (see Figure 2). That makes for a faster learning curve.

li-the-value-of-prototypes-fig-01a_embed

Criteria for success

A prototype is a kind of experiment, so you naturally want to figure out in advance what you hope to learn from it. A successful prototype should produce at least the following results:

Identification of customers whose feedback needs to be solicited, along with specific triggers for soliciting it
Baseline results for response rates and quality of feedback, along with specific techniques for improving response
Insight into what kind of information needs to flow through the system
Learning how to close the loop with clients, including who will do it, when, how they will get the information required and how they will track down root causes of complaints
Learning how to deliver the customer feedback to employees who need it and how they can take action to address systemic concerns
Enough experience so that members of the central NPS team can see what kind of support they need to offer—and so executives can see what investments will be necessary for success at scale
Enthusiasm among the employees participating in the prototype about the potential for the full-scale Net Promoter System to make a real difference for the business

In short, once your prototypes are complete, you should have all the information you need to design a scaled-up, reliable, low-cost system. Moreover, results from the prototypes will help you calculate the economics of improvement and build a business case for rolling out the system. We have written elsewhere about the infrastructure required for a mature system; successful prototyping is the interim step that helps you decide what that infrastructure should look like and how it should operate if you are to accomplish your goals for improving your customers’ experience.

One last lesson: Successful prototype design requires extensive input from the individuals who will be involved. For example, frontline employees and supervisors often have a gut feel for which interactions are real “moments of truth,” creating promoters if they go well and detractors if they go poorly. These touchpoints will be the best triggers for a request for feedback. One property and casualty insurer brought employees to a workshop to identify the key interactions. The prototype, members decided, would focus on just two: first contact after a loss and settlement. In a B2B situation, the key question for a workshop may not be touchpoints; instead it may be which people in the customer’s organization to survey. Purchasing agents? Finance executives? Users of the product? Sales and customer-service reps can draw up account maps for each customer organization involved in the prototype, analyze individuals’ roles and identify the appropriate people to ask for feedback.

Creating an effective Net Promoter System is hard work, mostly because it has to be tailored to an individual company’s business, culture, operational processes and competitive situation. Prototypes let you design your system step by step before you scale it up. They ensure that you’ll learn the right information, channel it to the right people and make your customers’ experience as good as it can possibly be.

Rob Markey, global practice leader of Bain's Customer Strategy and Marketing practice, shares some best practices for prototypes in this short video.

Tips and tricks for successful prototyping

Experienced Net Promoter practitioners have learned a variety of lessons from prototyping the system. For example:

Plan on more than one prototype. Schedule at least two. That way, you’ll get more than one shot at refinement and improvement of the system, and you can add complexity in a controlled fashion.
Pick your spots. The first prototype should offer the highest likelihood of success—a simple situation with straightforward issues, a good leader, a well-staffed team and so on. Follow-on prototypes should add layers of complexity and difficulty so you can sort out the tougher issues. Increase the level of difficulty only after you’ve mastered the first.
Incorporate technology as soon as possible. You can map out plans for a prototype with an Excel spreadsheet or pencil and paper. But bring in whatever technological support you intend to use as soon as possible. Ultimately, you’ll need to work with your company’s CRM data, operational databases or other systems. It’s important to learn early on about the challenges you’ll face.
Involve senior executives. There’s a risk in any prototype that middle managers’ natural conservatism will hobble what gets tested and how the information is used. At one financial services company, the supervisors in a call center were reluctant to share feedback from clients with call-center reps because they feared it would be demoralizing. A senior leader had to point out that not sharing the feedback defeated the system’s purpose.
Get regular feedback on the prototype from the staff. Don’t make every decision by yourself. Frontline staff and supervisors will have a treasure trove of observations about how the system could work better.

Terminology: Pilot vs. prototype

Many companies refer to their early Net Promoter experiments as pilots. That’s a fine term, but we prefer prototype, because it implies learning and development over time. Like a product prototype, an NPS prototype elicits low-cost, real-world feedback from the people who will use it before you have scaled it up into a full-blown system. Each subsequent prototype reflects the learnings from previous ones and more closely approximates the final scalable solution.

Rob Markey is a partner and director in Bain & Company’s New York office and leads the firm’s Global Customer Strategy and Marketing practice. He is coauthor of the bestseller The Ultimate Question 2.0: How Net Promoter Companies Thrive in a Customer-Driven World.