And how it compares to data-informed design & data-inspired design

Reading Time: 13 minutes

A woman takes notes about data in a collage representing testing

The term "data-driven" is being applied to design more and more. Tech people love to talk about the benefits of a data-driven mindset and letting data make decisions. It can feel like a sci-fi cult the way some people worship data.

While it may be a buzzword to some, the term persists. What’s behind this term? How can Product Designers and UX/UI Designers harness data-driven approaches? Should we harness it? Is it even possible to let data drive our design decisions?

Here’s how I define “Data-Driven” for Digital Product Design:

Data-Driven Product Design is a continuous process of testing your ideas with customers and letting the resulting customer data make design decisions even over intuition.

Data-Driven Product Design is a new process for creating products using “as-close-to-real-world-as-possible” data. It’s an iterative approach, turning ideas into experiments and letting the customer decide.

That means data isn’t just an inspiration or a way to check your work. It starts and ends projects because the Product Designer’s role is to generate customer data to drive decisions. You can include behavioral data from testing and attitudinal data from research. These two types of customer data allow Data-Driven Product Designers to de-risk their ideas systematically.

Note: When referring to “data,” we mean qualitative and quantitative data throughout this article. We do this to normalize the power of qualitative data.

Pros: Data-driven product designers are human-centered since they let the customer directly influence what gets built. Customers decide through their behavior in various research and testing activities. Intuition can still play a role in interpreting the data, but you should submit your decisions to the results of customer behavior.

Cons: The iterative nature of letting data make decisions can mean you sometimes miss out on the big-picture view and trends that also have a part to play in design. Sometimes customers don’t show their ideal behaviors in their actions. When we base our decisions on behaviors, we can end up with products that customers want but don’t need. Addictive online shopping is an example where you might want to mix research data with experiment data to ensure that you address the entire landscape of customer needs…not just what they seem to enjoy by their clicks.

How is “data-driven” perceived in design?

Designers may look unfavorably at data-driven approaches because it relies on an experimental mindset involving hard numbers rather than an expert-led intuition from subjective, qualitative data more familiar to traditional design.

Data-Driven Product Designers are still designers, but they look more like scientists than artists. The data-driven mindset might not be compatible with visual and speculative design practices.

I see Data-Driven Product Design as another tool to discover customer opportunities and be a user-centered designer. We teach both qualitative and quantitative data-driven skills in our course.

A quote by Kim Goodwin about measuring design

To get some outside opinions, I asked some Product and UX Designers what they thought of the term [read the thread here]. Some common themes begin to emerge. Here is a summary:

Data-driven design means data science and A/B Tests.
Data-driven design also includes qualitative data.
Data-driven design is a buzzword used by managers to feel objective.
Data-driven design helps us avoid biases in our design work.
We’re all Data-Driven Designers if we do research.

All of these feel true to me.

Qualitative data is data, and when it drives decisions, isn’t that data-driven design? I think so. I have also seen "data-driven" teams fail by cherry-picking data because the CEO or HiPPO (highest-paid person's opinion) didn't like the outcome of a test.

I've also worked with some brilliant data scientists who embody the term "data-driven" and have very high standards for the term that only applies to proving causality and modeling the future.

Where should Product Designers land on the issue?

The Three Ways That Data Influences Design

To understand Data-Driven Product Design, we have to look at the alternatives. There’s a common framework of “data-inspired” vs. “data-informed” vs. “data-driven” that we will apply to design and it’s a mix of data types.

A framework on data's influence on design with intuition vs data

Data-Inspired Design

"Customer data is a source of inspiration, but intuitions make decisions."

Data is “insightful” to you, and it can come from anywhere, which sometimes causes cherry-picking data that confirms your biases. You prioritize a big-picture, future view. Trends, attitudinal data, and insights drive your team.

Example:
Designers interview users periodically, and this attitudinal data indirectly lead them to simplify their app's information architecture. Later, the product manager noticed a drop in the numbers simultaneously as the feature launched. They suspect the customers' opinions they talked to don't match the customer base as a whole. In the end, they don't change the feature because they all agree that the change was an improvement. After all, the senior designer says that it's a best practice, and she's an expert in information architecture.

The Designer's Role in Data-Inspired Teams:

Gather inputs and stay up-to-date on customer trends
Design based on intuition using any research or analytics data you can gather
Measure your work through outputs and company goals

Decision Making: Experience makes decisions even over data.

Useful in the domains of #Strategy #Insights #Inspiration

Data-Informed Design

"Customer data helps you evaluate your design decisions."

Customer data is "pulled" by experts or gathered through usability tests to measure projects. Big-picture goals, quantitative KPIs, and qualitative feedback drive the team.

Example:
You create and launch a feature based on your team's quarterly OKRs that they hope will increase the adoption of their app. After releasing the feature, your team notices a significant drop in adoption on the analytics dashboard the same day as the release. You do some usability tests to figure out where the issue is. You don't find a problem, but you decide to change a few things that users bring up in the interview.

The Designer's Role in Data-Informed Teams:

Use research, testing, and analytics to adapt designs to the customer.
Design based on past research/intuition but evaluate your work through usability and analytics.
Measure your work indirectly through a variety of data points
Keep an eye on customer outcomes using big-picture analytics and team KPIs

Decision-Making: Experience makes decisions unless customer data informs us that we're wrong.

Useful in the domains of #Analytics #Usability #Evaluation

Data-Driven Design

"Customer data shows you what to design next."

Design methods are a valuable way to generate data. Design data de-risks ideas and enhances decision-making for the whole team. Behavioral data from experiments drive the team. Customer data is the start and end of projects because it’s how you determine what to build.

Example:
The last experiment you designed tested a new feature you want to build. You set up a Landing Page Test to see if users would give you their email address for the new feature. They didn’t respond, as you guessed...but in the experiment retro, your team looks at the heat map you added on Hotjar. You notice that every user that gave an email address hovered on the newsletter form at the bottom. You decide to design an experiment to remove all other forms from the page. It passes the test! So you create a prototype to test usability now that you have tested the desirability.

The Designer's Role in Data-Driven Teams:

Design experiments that will generate customer data and enhance decision-making for the whole team.
Glean insights from near-term research and test results
Set up evaluation metrics before designing any experiment or release.
Measure your work directly through real-time behavioral data or near-term attitudinal data
De-risk product ideas in lo-fi concept tests as well as hi-fi A/B tests.

Decision-Making: Experiments make decisions even over experience.

Useful in the domains of #Experimentation #ConceptTesting #Innovation

Learn More

Read another take on the data-inspired vs. data-driven concepts by Alistair Simpson
Download the Experiment Cards to help you build data-driven prototypes by tackling assumptions first.

Learn the difference between Declarative and Behavioral Data in this article by Katie Hagan.
Take our 22-day course called Designing Product Experiments and learn how to test product ideas in all 3 decision-making types.