Developing Product Intuition
Over the last several years, there has been a rapidly growing body of practices associated with using data to create better outcomes for customers of products and services. This is referred to as Data-Driven Product Development (and multiple similar terms).
The Lean Startup increased attention to leveraging data for better decision-making by popularizing the Build-Measure-Learn loop. Digital products and cloud-based services made it easier to instrument, collect usage data, and run experiments like A/B testing that is foundational to trendy Product-Led Growth strategies.
In fact, leveraging data for improving outcomes in product management has become so popular that a new role has been created called Product Operations (aka ProductOps) that largely help organizations create, scale, and leverage data-driven practices.
I am a huge believer in the need to leverage data in product management, I fear that the pendulum is swinging too far towards thinking that data alone is our path to product perfection.
The Truth: Focusing on hard data alone, will limit your odds of success.
I offer four high-level reasons why relying on hard data alone is not the best approach to product success.
- Hard data is not always available. The reality is that it is not always possible to collect the data necessary to support the desired analysis and decision-making process. When it is possible, it can be too costly or take too long to support the needs of your business.
- Issues with data quality and analysis. An article in the Proceedings of the National Academy of Sciences named “ Issues with data and analyses: Errors, underlying themes, and potential solutions “ (March 2018) describes major themes for how data analysis can go wrong. These themes include errors in producing data, managing data, statistical analysis, logical flaws, and in communications.
- Analysis paralysis. Waiting for more data to help make a decision obvious can frequently slow down business. There is rarely sufficient data to make decisions perfectly clear. It is often about weighing best a variety of unclear inputs and probabilities against one another.
- Great data does not equal great decisions. Finally, good data, even great data on the topic under scrutiny does not always produce expected or desirable outcomes when a decision is made.