Beware of vanity metrics!

Dictionary says that the word 'vanity' means excessive pride in or admiration of one's own appearance or achievements. The word has another connotation - the quality of being worthless or futile. It is interesting to note that in the phrase, 'Vanity Metrics', the word bears both the meanings. I will clarify that soon. As we are now clear with what 'vanity' literally means, let's see what it has to do with 'metrics'.

Metrics are measures of quantitative (or sometimes qualitative) assessment used for assessing, comparing, and tracking performance or production. Metrics are everywhere, starting from academic report cards to complex health care systems to spacecraft control systems. But in this article, we would restrict ourselves to the measurements an organization does to evaluate how well it is progressing towards its business or technical objectives at various levels. The choice of what and how to measure is critical as there are plenty of options available. A vanity metric is a metric which does not provide a reliable measurement of what we should care about but on the contrary, it gives a sense of complacence that something meaningful is being measured. That means it is of little or no real value, but just a sense of pride that's associated with it.

Why should we be aware of them? It is simply because of its worthlessness. Sometimes, its collection involves a considerable amount of effort as well, which makes it even more avoidable. Why are many of us pulled towards them? In fact, there are quite a few reasons:

We have seen them being collected by our ancestors

This is one of the strongest reasons where not only metrics, but some practices are also adopted 'as-it-is' without looking at its value in the current context.

We find it easy to capture

I cannot resist sharing an interesting short story in this regard. One morning, Tom was desperately searching for a gold coin in the eastern backyard. Harry asked him, 'Where did you see that for the last time?' 'In the storeroom at west,' Tom replied. Harry was utterly surprised and asked, 'Then why the hell are you searching that here?' The answer was hilarious - 'It is dark inside the storeroom, and I find it easy to search here in this bright yard.'

It is presentable

Some metrics are so colorful and jazzy that the audience is carried away by its presentation and the value factor is overlooked.

False sense of governance

There can be a lot more reasons like satisfying power stakeholders or avoiding being non-compliant and so on.

We should keep in mind that in one context a metric can have value addition while in some other context, it can be merely a vanity metric. A lean agile mindset plays an instrumental role in finding value in any measurement we try to make. There is a common symptom of all vanity metrics - they are least actionable. That means, if you have not made any specific meaningful action based on a metric for quite a long time, it is highly likely that you are measuring a vanity metric. Beware of it!