Introduction
Gathering and understanding user sentiment is crucial for informing your conceptual or future designs. Measuring user sentiment helps you and your organization make more data-driven decisions and move conversations away from subjective interpretations of people think about UI and UX.
Expectations
When you walk into a restaurant you expect to be seated, be given a menu, and pay at the end. How would you feel if you entered a restaurant and were asked to provide your payment before you took a seat or even looked at a menu? Your past experiences going to a restaurant inform your future expectations when you walk into another.
A user’s expectations are informed by their memory of past experiences, and associations they make with similar products, services, interfaces, and so on.
Measuring whether your experience meets or does not meet their expectations is an important part of measuring the success of your solution. An ideal place and time to start measuring it is after the user has completed a task or flow. Although placing the measurement at the end means we miss insights from users who failed the task, it still provides valuable data to infer their experience.
You can ask this in the format of a likert scale: “Based on your experience today, did our product/service meet your expectations?"
Post-task survey response options:
1 - Far below expectations
2 - Slightly below expectations
3 - Met expectations
4 - Slightly exceeded expectations
5 - Far exceeded expectations
Goals
When a user arrives at your experience, they will have a single or even several goals they want to accomplish. The more complicated and feature-rich your product or experience is, the more goals they may have.
When I was at Buildertrend, a construction software-as-a-service company, there were huge disparities between the smallest user and the largest.
One person I interviewed was an Australian handyman whose job in addition to fixing something was to manage and document jobs as simple as fixing a broken window. On the other end of construction, we had million dollar companies building skyscrapers and other gigantic pieces of infrastructure.
Both used the same exact product with the same features and capabilities available to them; but clearly they both had an entirely different set of goals they had set out to accomplish.
When you have a customer profile, asking them about their goals particularly during the onboarding or early stages of using a product is crucial.
You can ask this in the format of a likert scale: “Since you have started using our software, how have we been able to help achieve your goals?"
Periodic check-in survey response options:
1 - Not achieved at all
2 - Barely achieved
3 - Somewhat achieved
4 - Mostly achieved
5 - Fully achieved
Clarity
Clarity can refer to the ability of the user’s comprehension and ability to understand something, usually like text on the screen, whether they know next steps are, or if the action was completed successfully or not.
Government, medical, financial, and legal experiences are notorious for lacking clarity. A lack of clarity can manifest in multiple ways, such as vague button labels, unclear navigation, or missing confirmation messages after completing an action.
Post-task surveys response options:
1 - Not clear at all
2 - Slightly clear
3 - Somewhat clear
4 - Very clear
5 - Extremely clear
Relevancy or interest
Interest is another subjective measurement to ask from users, but can be helpful in probing ideas and demonstrating concepts to a user. If we don't work to validate ideas and concepts, we as designers could end up designing dead product ideas and businesses waste time and effort. Other times, you run into pre-existing content on the page that is politically contentious for you to remove, and collecting this user feedback can help you work to remove it.
Something to be aware of however is that users have a tendency to want to appease their interviewer and not hurt their feelings. So prefacing before the question is important to help reduce bias in the study, statements like: "We're looking to get honest feedback on this concept I'm going to show you. I did not make this, so you're not gonna hurt my feelings."
You can ask the question a couple different ways:
- “Taking a look at the opportunities available, how interested are you in any of these? Why?”
- “Taking a look at this page, is there anything on this page that you didn’t find interesting or helpful? Why?”
- “Would such a feature/service be helpful or useful to you? Why?
Concept testing interview response options:
1 - Not interested at all
2 - Slightly interested
3 - Somewhat interested
4 - Very relevant/interested
5 - Extremely relevant/interested
Satisfaction
Satisfaction is one of the most generic, but widely used user metrics in business. It is relatively easy to capture, and is easily placed at the end of a phone call, in-app survey, modal, and so on. It’s a widely accepted assumption that happier customers lead to better businesses. How exactly satisfaction directly correlates to profit is another question.
You can ask this question at the end of a purchase flow, after a sign-up flow, you can drop a survey to your users just about anywhere after a task.
Post-task surveys response options:
1 - Not satisfied at all
2 - Slightly satisfied
3 - Somewhat satisfied
4 - Very satisfied
5 - Extremely satisfied
Confidence
Confidence is the user’s sense of assuredness in their ability to do something. How confident are you that you could complete your taxes by yourself without any assistance?
Confidence is a great metric when dealing with situations where you believe there will be perceived or genuine difficulty in the user completing the task.
Post-task interview question response options:
1 - Not confident at all
2 - Slightly confident
3 - Somewhat confident
4 - Very confident
5 - Extremely confident
Ease of Use
Ease of use is the perception of how easy or hard it was to use a particular thing. People prefer to use things that are easier to use rather than those that are harder to use. While this is inherently obvious, the problem really lies with when it’s measured and how to fix it.
You can ask this after a task was completed by following up in a survey or having it as a question in an interview.
Post-task surveys response options:
1 - Very difficult
2 - Slightly difficult
3 - Somewhat easy
4 - Very easy
5 - Extremely easy
Trust
Trust can manifest in a few different ways in business. Trust is most important when the risks are incredibly high. Topics of the high-risk nature would involve decisions in healthcare, childcare, financials, and charity to name a few. If users don’t trust a product or service, they will abandon it.
A defective product you bought off of Amazon was $5 or $10 burned, so the total value of loss is very little. But what about the long-term implications of your health? Would you jeopardize years off of your life just to save more money? How about your child’s future? Do you really trust this organization to invest your money well and set you up for retirement, or are they just trying to make a quick buck?
Trust is easily broken and takes ages to regain.
Concept testing interview response options:
1 - Do not trust at all
2 - Slightly distrust
3 - Somewhat trust
4 - Mostly trust
5 - Completely trust
Conclusion
By evaluating expectations, goals, clarity, trust, and other key factors, you as a UXer can pinpoint areas of improvement and persuade your co-workers and stakeholders. By collecting these measurements, you can begin to create or improve the culture of data-driven decisions at your organization.