Two Powerful Frameworks for Finding Data Insight

Many times I’ve worked with a business to ‘find insight’ from their data. I use a wide range of techniques. This post summarises two powerful frameworks for finding data insight, providing structure and a road map to speed things up.

Frameworks are commonplace in Business Strategy and also in Modern Programming. They are widely used because they are effective. The same applies to finding insight.

Many commentators state that the only people who can find insight are those with vast domain-specific experience, combined with good data skills and tools. I somewhat agree. Domain-specific experience can be very useful, but it can also be a roadblock. Good commercial experience that transfers well across industries, along with experience finding insights is, in my view, even more useful. Either way, working within one or more defined frameworks for finding data insight will significantly increase the rate of success.

To find insight, you need to look. You also need to listen and you need to ask. Using a simple example... if you wanted to know how a new product is performing you could try things like:

• Reviewing the sales data;
• Listening to what customers or critics are saying; or
• Asking your front-line sales staff.

Any or all of those steps will help. Headline sales could be low but front-line sales staff might know that initial packaging was poor. Maybe an influential blogger raved about it or wrote it off.

Let’s take it a step further. Maybe you sent out a special email offer related to the new product to your email list.  Who showed interest, who made a purchase?  What common or largely common attributes are shared by the customers who bought? What customers did you expect would buy it?

How do Frameworks for Finding Data Insight Help?

Frameworks provide a game plan. When you're given the broad goal of 'finding actionable insight' frameworks will give you somewhere to start.  Here are two powerful general frameworks that are worth considering.

Finding Insights Using the Scientific Framework

When Scientists seek new knowledge they typically use the scientific method. Very simply this means they generate a theory about how something works and make predictions based on this. They then conduct experiments to test their predictions. If those tests don’t support their predictions they will seek to understand why and update their theory. Then repeat.

To find data insights a similar approach can be adopted.

1. Ensure you have access to a specific data set.
2. Discuss with the business or relevant parties and make a series of predictions about the data. Here are some ideas assuming a list of customer transactions. Predict the...
   • Average transaction size and average number of items in each transaction;
   • Products or services that are normally purchased together;
   • The average time between purchases for customers;
   • Percentage of customers who have only made 1 purchase; and
   • Day of the week or month that has the most transactions.
3. Make sure you have actual predictions written down. If possible get people within the business to also make the same predictions. This can be a great way to increase business engagement.
4. Now get to work and analyze the data. Do this fast. It doesn't need to be perfect.
5. Compare actual results to predictions. Provide some visualizations of the data to those in the business who made predictions. Make these visualizations interactive and allow the business users to drill down and explore. Watch. Areas of interest, surprise, or disagreement are all good.
6. Any actual results that differ from the predictions or any data that doesn’t fit with the predictions should be investigated. This is where insights will emerge.

I’ve found this approach also identifies who knows the data, and who is engaged. Simply exposing people to the data and good simple analysis can lead to interesting questions and valuable insights.

Finding Insight Using the Specific Question Framework.

Another great approach to finding Insight is what I call the Specific Question Method.

As the name implies it involves focusing on just one specific question. The question shouldn’t be general in nature it should be as specific as possible.  I've found it's much easier to engage business people if you ask a specific question about a specific product, customer, or similar. Initially frame your questions based on data you already have.

Here are some examples:

1. The most popular product in terms of sales last month was X. How many of the buyers were new customers and what channel did they buy through?
2. How many of our B2B customers who came to our most recent product launch actually bought the product? Did any buy any other product or have any other subsequent interaction?
3. Who were the top 10% of operational teams based on core productivity measures for each month in the last 6 months?

This method is deceptively simple. I've found a useful approach is to define a question and do some analytics to get the answer or learn more. Then pose the question to management, get their thoughts and then reveal the findings. The reactions of management and subsequent discussion can be very valuable.

The key to this framework is speed, focus, and framing the question in the language of the business.  Make the questions real and topical within the context of the business.  This specific question approach works well in conjunction with a broader Questions Framework.