Understanding Metrics (Part 1)

Now that I've explained the larger context of why we need metrics to help us tell our story as part of realizing a strategy for a team (refer to the last post), let's dive into understanding metrics.

A 2-dimensional matrix

I personally look at metrics on a spectrum. Imagine a four-quadrant graph, like this:

On the horizontal axis, we've got the kind of engagement, ranging from passive engagement to active engagement. This is how much someone has chosen to do a specific action, whether it's reading your docs, posting on your forum, liking your social media post, or attending your talk. On the vertical axis, we've got business value, from low to high. If your company's main goal for the year is to build awareness of an open-source project, for example, higher-value activities include anything from starring the repository (something I'd call a "vanity metric," more on that in a minute) to attending a workshop on using the project or even creating a video or writing a blog post on using it. A lower-value activity in that scenario might be appearing in a search simply because people skim those results anyway.

When you put the two together, you can start mapping your metrics on a two-dimensional matrix as shown. An active, high-value metric likely is worth the amount of time, effort, and risk associated with the tactics that affect that metric because you can use it to explain the story of what your team is doing and why. A passive, low-value metric might only be worthwhile as a byproduct of tactics intended to impact some other metric—a minor supporting character, if you will, in your story.

A word on vanity metrics

I think it's important not to get drawn in by what I (and many others both within and outside our industry) call vanity metrics. Vanity metrics are the kinds of metrics that make us feel good: likes on social media, stars on platforms for version control systems (e.g., GitHub, GitLab), or even views on a video. They generally are easily gamed, may not indicate much business value, and can be extremely hollow engagement as they don't often indicate much beyond a fleeting interaction. If you were to use vanity metrics as the sole underpinnings explaining your tactics in light of your overall strategy, examination by leadership will likely poke multiple holes in your plan or will disregard your strategy and tactics altogether.

Vanity metrics can be great when you're getting started to give you that psychological boost of watching numbers go up. They're often an early indicator that a tactic might be working, so long as you understand that they can be false signals and need to be backed up with stronger indicators. For example, while stars on an open-source project are fun to watch and can give you a mental boost, the stronger indicators of traction are things like the number of new contributors, the number of commits from outside your organization, and the engagement stats on new issues or discussions within the repository.

Decisions, Decisions

I'd love to be able to tell you the exact metrics to measure. However, to quote the favorite line of every senior engineer ever, it depends. It depends on what the company goals are. It depends on which organization your team is embedded in. It depends on the strengths of your teammates. When deciding on what to measure, there is no one-size-fits-all-teams approach. You have to have that discussion among your teammates, and you need to consider how you will tell the story of how your team impacts the company's goals. All metrics need to inform you on the performance against a baseline, and all metrics need to help you make decisions about tactics against strategy. In short, which metrics will help you tell if what you're doing is actually impacting the strategy you set forth at the beginning?

In the next edition, we'll dive into some example metrics for different teams and made-up goals to get a better sense of how to define these questions for your team.

What do you think? I want to hear your thoughts! Reach me in any of the following ways:

• Post to me on Mastodon at @nimbinatus@hachyderm.io.
• Find this post on LinkedIn via my profile, comment, (and follow me while you're there!).
• Tweet at me at @nimbinatus. (I'm not on Twitter much these days, to be honest.)

Did you find this useful? If so, share it with a friend so they can subscribe.

If you want to post something about developer relations strategy, metrics, or our industry so you can share with others, drop me a line!