How to Set UX Metrics with the Google HEART Framework
Organizations run on metrics. It's pretty hard to make the case to improve usability if you can't translate that initiative into something concrete.
I believe that UX can and should be measured, but it isn't easy. Is there some grand, universal UX metric we can all follow?
First, let’s define UX metrics:
UX metrics are the numbers we use to measure, compare, and improve the user experience over time.
Quantifying the user experience is a hard thing, and you are going to need all the help you can get. That’s where Google’s HEART framework comes in. It provides insight into how a data-mature company might approach metrics on the user experience.
What is the Google HEART Framework?
Like many companies with mature UX practices, Google designed a custom UX scoring system to guide their design work. The HEART framework for UX metrics looks at 5 areas of the user’s experience with Google:
Happiness
Answers: Are users happy about their experience with your product?
Happiness is a measure of the attitude of satisfaction to the user experience. In UX, we measure happiness through self-reported surveys, mostly. Popular survey methodologies in UX are SUS (System Usability Scale), SUPR-Q (Standardized User Experience Percentile Rank Questionnaire), and NPS (Net Promoter Score). Be careful with NPS, though…it has a lot of enemies in UX design.
The key to measuring satisfaction or attitudes lies in regularity. Scores on happiness might not mean much at first, but how they change over time is extremely interesting. Set up a regular interval for data gathering if you want to take this metric seriously.
Engagement
Answers: How much are your users interacting with your product?
Engagement measures how your user is voluntarily engaging with your product. Engagement will be an extremely important metric for design and the business if your tool requires voluntary user behavior. Social media companies are good examples of optimizing engagement metrics such as daily active users and session time.
B2B companies might not find this metric to be helpful. The HEART framework leans towards the B2C side of metrics.
Not all engagement is good; users may be highly engaged with an interface when encountering a software bug. Keep balance in mind when considering engagement metrics. You don’t want to over-optimize on engagement like “watch time” at all costs coughcoughNetflixcough…excuse me.
Adoption
Answers: Are we getting new users?
Adoption is the number of new users gained within a specific time frame. Adoption is always going to be critical to business leaders. It’s the lifeblood of most company strategies. If you’re a designer that knows how to improve adoption, you will always have a job.
On the simple side, adoption can be measured through unique page views for simple sites. For most product teams, adoption will be about new paying customers.
Adoption metrics will overlap highly with sales, marketing, and product marketing, so don’t try to run the show if you’re in UX. Don’t get too caught up on roles and silos because the user doesn’t care whether you’re a marketer or a designer…they want things to solve their problems. Attracting new users is everybody’s job, and a solid user experience will improve adoption in the long run.
Retention
Answers: Are we keeping our users?
Retention is the number of users that stay users within a specific time frame. Most users will “churn” or stop coming back eventually. This dropoff can happen quickly for products with poor UX, and it will be an extremely important metric to watch after releasing new features.
Retention helps balance adoption, and both will be very important to business leaders. While marketing typically owns adoption, retention is often owned by product. So again, don’t take too much credit or make a landgrab on retention. It’s not going to be the responsibility of UX except in certain cases where you’re optimizing the user journey or focusing on a specific drop-off problem.
Task Success
Answers: Can users use a specific part of the user experience?
Task Completion Rate (TSR) and Task Completion Time are the main numbers to count with task success. These metrics will be familiar to anyone who runs usability tests, and just like usability tests, the task will be specific to the study.
Much like Happiness, Task Success will be hard to automate. You may end up looking at different tasks in different places. But as you measure task success, you can build a benchmark for usability that can be very helpful in your company.
Start small here, and don’t try to automate measuring Task Success until you know which tasks are important to measure.
The Goals-Signals-Metrics & UX metrics.
The columns that organize the HEART framework are even more helpful to companies outside Google.
I want to show you how to use the Goals-Signals-Metrics Process developed by Google to set UX metrics for anything. I think it's a great place to start if you've never set a UX metric. Here’s what that might look like.
What are Goals?
The goals are the big-picture targets for your product team. Goals should always be based on outcomes, not outputs. An output example would be completing 10 cards in a sprint. An output example would be increasing your efficiency at work. Output goals are shallow and don’t empower the team. Outcomes allow practitioners to develop expertise over time and form strong team bonds.
I'd recommend that you start with one goal. These goals are usually well-defined quarterly goals for mature product teams, like increasing adoption by 15%. Startups might have goals like improving sales to hit a “break even” status as a startup. Don't make these goals up. Instead, ask your PM what the goals are or facilitate your team in that process.
Example: We want to improve the usability of our app
What are Signals?
Signals tell you whether you're on the right path toward your goal. Based on customer behaviors, a signal is an early sign of success or failure. Signals can tell you when you're reaching the goal: after using the new feature, a user submits a positive app store review. These signals could tell you that you've failed: a user submits a bug report while using the feature.
These won't be outcomes-based like the goals, so it's ok to have outputs like the user finishes 3 tasks a day. Try to pick 2-3 signals at first.
Examples:
Users complete tasks faster after usability updates
We get fewer bug reports submitted after we improve the usability
Users log into the app more often as usability is improved
As you can see, the signals don't have to contain numbers until the next stage, but it's ok if they already sound a bit like metrics. Finding signals can be a fun activity so bring your team in on the action. PMs will be the decision-maker here so defer to their judgment as they know your product metrics best.
What are Metrics?
It's finally time to choose the right metrics by translating your signals into measurable criteria. The metrics should reflect a change over time. Avoid selecting "vanity metrics" or counts that always go up, like increasing follower count. Instead, use ratios or percentages over time, like growing followers by 30% every month. It’s important to make it time-bound and describe the change.
UX Example:
By Q4 2022…
We will reduce the time it takes for users to complete their [main task] by 50%
We will get fewer bug reports in Q4 2022 than in Q4 2021
Our weekly active users go up by 10% every month of 2022
As you can see, the reality is that there is no single UX or product metric that all designers can use. In reality, there are many ways to measure UX. Measuring the user experience will always be a triangulation of methods and metrics.
But if you’ve never set UX metrics, the Goals-Signals-Metrics process is a great place to start. As you experiment, you will naturally develop your methods and approach.
Check out my talk below if you want to learn more!
Learning Resources for Google’s H.E.A.R.T. Framework
Get detailed directions on setting Goals-Signals-Metrics in this Atlassian Playbook
Learn how to combine Goals-Signals-Metrics with the H.E.A.R.T. framework
Watch this video on choosing the right UX metrics by Maria Panagiotidi
Learn about the method that Google recommends in their Design Sprint Kit
Free Masterclass in UX Metrics
If you want to dive deeper, check out this free 60-minute webinar on everything you need to know about measuring the user experience.
