RESEARCH PATTERN

Product Analytics

Skill theme: 
Basic Quantitative Work
Basic Quantitative Work

Organizations have quantitative data coming in from multiple sources, including anything that's instrumented for measurement in a service, Exploratory Quantitative Analysis, or discrete efforts like A/B Test. We need to integrate this type of data—the language that most of our business counterparts are familiar with—with insights derived from our larger body of qualitative work to build the best picture of reality.

* * *

The challenge

Qualitative data can be mistaken as "weak" data when it's not gathered at scale. Strong insights may not carry weight without understanding their representativeness across a user population. On the other hand, user “truths” that can be derived from quantitative product data (metrics, analytics) are not always easy to interpret or understand, especially without sufficient context.

The approach

Working with existing products, consider what data is already on hand, or what data can be gathered with new instrumentation. Look at how you can correlate the data of behavior at scale with experiential insight you've gathered from the research process: each facet will strengthen the other. Both can be a part of ongoing work: the meaning of the data changes as you build a better understanding of user behavior. 


Therefore, seek existing analytics data when undertaking a project. Review what information will be useful, and evaluate knowledge of existing behavior. Identify the story in each facet of data, and highlight those interesting areas where the story of the overlap is more interesting than either of its parts. In some cases, with ethical care and consideration for participant privacy, you may be able to develop parallel qualitative and quantitative understanding of specific, individual participants as part of a study.

* * *

Review related metrics along with each of your Session Debriefs. Bring all relevant quantitative data up front to you Sensemaking Workshop.

Last updated:
Apr 26, 2020 19:19

Resources

Workshop Data:

Data from our 2019 workshops - 486 research practitioners voting on their top three useful skills / top three desired skills.
ResearchOps community