January 30, 2023
14 Minutes

Tracking What Matters

Analytics and the art of tying your shoes: a practical guide to North Star metrics
Tyler Theofilos
Data Science Lead
Header image

In an era when information is increasingly at our fingertips, it can seem like it’s getting harder for entrepreneurs to measure the health of what really matters in their business. Why is that? For most business owners, it feels like more data has led to more confusion.

On any given day, marketing professionals work with:

  • Ecommerce data
  • Social networks metrics
  • Subscription data
  • Website analytics
  • App data
  • Marketing data
  • Owned channel data (email, SMS)
  • Specialized marketing tools
  • Data warehouses

So how do we build a coherent data story?

Before we dive into how to combine data from so many diverse sources, let’s revisit a famous story about John Wooden, the UCLA coach who won 10 national titles during a miraculous run in the ‘60s and ‘70s.

The first lesson Coach Wooden gave to freshmen was always the same: how to tie their shoes. Why something so rudimentary? Ostensibly, every freshman on his team had been tying their shoes since kindergarten. Surely they didn’t need another lesson.

What Coach Wooden understood is that he could teach his team any number of complicated formations, pick maneuvers, or ball-handling techniques, but none of it would matter at all if they couldn’t play comfortably and confidently for the entirety of a 2-hour game. (And especially not if they were injured by tripping over a loose shoelace).

His success came (in part) from simplicity. From focusing on the essentials.

Finding the Essentials in your business

It turns out that tying your shoes is a lot like choosing a set of metrics and data sources for your business. You can do everything else right, but if you don’t have that handful of simple, easy-to-explain metrics – that everyone trusts – then all the dashboards and reports in the world won’t save you.

In light of this, even some of the world’s best businesses can seem like a rookie with ten shoelaces on at the same time (and only some of them are tied). Different definitions of the same KPI will trip up your business, even if you do everything else right.

Data Fracture

The dirty secret of analytics and reporting is that there’s often too much of it, and usually not the right kind. E-commerce and subscription-based businesses typically build out their data teams with the intent of supporting comprehensive analysis of the most intricate aspects of the customer lifecycle. The thought goes: if I capture every piece of information about my users, I will know how to improve their experience.

Sometimes this results in a vital corpus of data for business owners… but more often, that high ambition can lead to data fracture:

data fracture (noun) /ˈdādə ˈfrakCHər /

  1. a state of data storage and analysis in an organization, where leadership relies on too many data sources that don’t line up with one another and/or can’t be monitored through a single interface

Data fracture is a common symptom we see across many different types of organizations. As the scope of data collection increases, it can become easy to get lost in the sea of dashboards, reports, conversion rates shared over Slack, and (even worse) text messages fired off without context.

And the bigger the organization, the bigger the problem. Businesses end up relying on multiple data feeds, which often contradict each other. Pull the same data in two slightly different ways, and it can take several meetings to figure out why the numbers don’t match – if you ever do. Even experimentation, the Holy Grail of analysis, can become an academic exercise in the absence of clear storytelling or ample data.

Data fracture problems:

  • Data is accessed in multiple separate locations
  • Information does not match up between data sources
  • Metrics are not accurately tracked
  • No single UI exists to view the information
  • There is no clear data owner at the organization
  • Education around data tools is spotty or non-existent
  • Many insights are shared without context over Slack
  • Simple answers require complex SQL coding
  • Analysts are required to translate data into insights
  • Employees are not empowered to discover insights themselves
  • Multiple KPI dashboards exist, none of which are universally trusted
  • Data accuracy is questioned on a regular basis, even if it is correct

How to Find your North Star Metric

To help combat data fracture and align your organization around trusted measurements, thought leaders have long recommended a kind of primary business metric called a North Star metric. This is the single metric that a business can rely on to predict their longterm success (or lack thereof). Steering toward the North Star is one way to guarantee that you’re headed in the right direction.

A few famous North Star metrics:

  • Facebook: New users who get 7 friends in 10 days
  • Airbnb: Total nights booked
  • Salesforce: Average records per account

We believe that the problem with North Stars is they are rarely alone in the sky, and businesses often have too many competing sources for them – a kind of data “double vision.” Gaining consensus on which source to use is the first step.

Supporting Metrics

The second step is to understand that North Stars aren’t everything. While longterm business health is important, a North Star metric can and will shift erratically in the short term. That’s why we recommend looking beyond the North Star, at the surrounding constellation of metrics that can point your ship in the right direction.

We call these supporting metrics. For instance, if your North Star metric is “total nights booked” (as with Airbnb) then your supporting metrics might be unique active customers, total active rentals, or nights booked per renter.

A Practical Example

To get things right, start with the star that tracks not just your own success, but the success of your customers. This will be the North Star metric that keeps your business afloat. And have a reliable, trusted source that’s as accurate and precise as possible. Consensus on this will be vital to maintaining a sense of constancy among your team.

The other supporting metrics we tend to look at are entirely focused on the customer experience. What they want, what matters most to them, and what signals that the digital experience you crafted is helping them achieve that (rather than getting in the way).

As part of these secondary supporting metrics, we like to look at something called Customer Effort. How hard is it for your customer to accomplish what they set out to do? What obstacles – whether UX-related or informational – are getting in their way?

The third supporting metric we like to use is a measurement of customer experience on an emotional level. Feedback scores like NPS or CSAT are vital tools to help you take the pulse of your userbase and catalog their comments and reactions to the user journey.

Pitfalls to Avoid

In our work, a common mistake we see is that clients will pay close attention to the wrong things. Things that pull focus away from those core essentials – things like time spent on site, page views per visitor, or even raw visitor counts. And they’ll have dozens of these, coming from various sources. These numbers can be important to measure the rollout of a new experience, but they are rarely worth caring about long-term.

Typical pitfalls:

  • Caring too much about low-traffic experiences
  • Losing the forest for the trees
  • Mistaking in-page engagement for business health
  • Chasing the latest and greatest metric even if it doesn’t apply to your business model (remember “attention”?)
  • Comparing metrics that can’t be compared
  • Comparing a behavioral segment to all users (correlation not causation)
  • Ignoring important directional indicators if they aren’t perfectly tracked

Finding your own supporting metrics means learning how to ignore what doesn’t matter – even when (and especially when) that’s hard to do. You want your business to be perfect. You want the About Us page on your site to wow your audience and earn their loyalty with a description of your amazing team. But the fact is, that page is not for the customer. It’s for your team, to show them you care, that you value them.

Part of what makes Customer Effort so effective as a measurement tool, is that it is built around a truth that’s difficult for many product teams to accept: what should wow your customers the most is the thing they’re buying, not the site they buy it on.

Which means the metrics that measure the health of your business should be the ones that ensure your site isn’t getting in the way – that the purchase flow delivers on what it promises, that customers know exactly what they’re buying or can find more info easily, that the UI is easy to use and understand, and that customers can make quick and accurate decisions.

Which, at the end of the day, is what you’re trying to do too. Sometimes, that requires less data scope, fewer sources, and greater focus on what matters most – not what matters to your product team, not to your warehouse, not even to your data analysts. But to your customers.

Speaking of which, if you ever need help tying your shoelaces, we’re here to chat.

Share this post