Case Study – In-product feedback

Vmware collects feedback across products to gauge the experiences using a centralized tool

The story begins with internal stakeholders being unhappy with the existing tool

You can see why, it's interruptive and complex, increasing cognitive load when interacting with the platform

From the design side on our platform team, we weren't happy with it either.

Billy and I had never worked together, so I wanted to let him help define how to proceed to empower him

Before I knew it, he was off and running, coming up with different design ideas for the problem area

A few days later in our 1:1, the truth came out, Billy was upset.

He had come up with a great design based on competitive research, but he couldn't get buy in from our key stakeholder, the Director of Customer Experience.

Seeing that a blocker was developing and Billy was getting frustrated, I offered to help out.

Billy had defined a design pattern that resonated with users on a humanistic, emotional level, as represented by the faces on the feedback widget

However, as a 5-point scale, it had fewer options than the existing product feedback's 7-point scale, which would result in data incompatibility if we were to change it

So how could we move this forward together?

How could we get buy-in from the director of customer experience, the key stakeholder who has owned this area across VMware for many years?

I met with her, hoping to understand her goals and see how we could gain alignment. Her perspective made sense, her goals are to provide a historical perspective of product performance. Ultimately, we were able to align around structuring our solution to meet both business and user needs

We both agreed that helping the business get more feedback would be better for our internal stakeholders. So, if engagement would increase with a re-design, we would have enough reasoning to switch the scale

We aligned around setting up an A/B test to compare engagement with a 7-point numeric scale versus a 5-point emotion-based scale

Once Upon a Time on a Design Team

Billy Gilmore is a designer on the team I lead

After completing a task, a customer effort score is delivered asking the user about the ease of completing it.

Product teams can then use that information to understand the impact of design changes, new features, or technical improvements.

The interaction design across both versions of the A/B test were similar.

Product teams can define additional questions to surface within the feedback widget.

But can add no more than 3 questions

Users can choose to turn off additional feedback requests or can ask VMware to send the request later.

Another enhancement we brought (informed by existing data) was to hide the free-form entry of information, enabling the user to choose to take this action as necessary.

We made this decision based on existing data suggesting that most customers (over 80%) submitted feedback without comments.

The Work

Solutions to identified pain points

Define

Design

Discover

Deliver

I allowed Billy to help me understand how best to add value to his process

And hopped in as necessary to help drive broader alignment across the company.

Although I did collaborate to help out with much of the hands-on work, I let Billy take the spotlight. For me, as a design leader, the success of the team is most important, and I try to understand how best to plug-in to their process to add value.

With my guidance, Billy developed a set of principles based on competitive and academic research.

He then distilled those principles into recommendations to inform how teams could leverage the solution

Then drove alignment across the different product teams, using the competitive, academic, and user research to illustrate the benefits of the solution

Based on feedback from users and internal stakeholders, we defined future-facing, visionary design solutions

Like the ability for users to follow up with a VMware team member to provide more details on their feedback

I helped to drive alignment and insure the solution's compatibility with the overall notifications system which I had defined prior

The initial results indicate that we get more engagement with the emotion-based scale, and illustrates how I use data to inform the best product design solutions for users and the business