Not Everything That Counts Can Be Counted

Every business needs to gauge its performance. Metrics such as revenue, profit, employee count, and market capitalization are how we compare and assess every business. In recent years, there has been a surge in business metrics across various functions, measuring everything from marketing pipeline and customer satisfaction to R&D defects. 

The obsession with metrics reached its crescendo following the release of John Doerr's book, "Measure What Matters," in 2017. Mr. Doerr asserted that business leaders should measure everything that matters and disregard everything else because if it can’t be measured, it can’t possibly be important. Because he’s done very well for himself, he must be right, no doubt. 

After the publication of Mr. Doerr’s book, numerous companies and consultants eagerly embraced the OKR (Objectives and Key Results) methodology, supplanting KPIs and other previous metric fashion trends. CEOs, inspired by the likes of Google, Netflix, and Amazon, mandated the adoption of OKRs across all departments, converting most meetings into KPI review sessions in which their teams try to interpret the data. 

While utilizing data for decision-making is a must, it poses some challenges. Firstly, data quality is often compromised due to measurement difficulties and human tendencies to fudge it...to look good in those review meetings. Secondly, misinterpretation is common, leading to confusion about causality and overlooking the bigger picture. For instance, in A/B testing, the focus on which alternative performs better might miss the possibility that both alternatives are bad (or that they are both great and we shouldn’t dismiss one of them).  

The third issue with running a business by metrics alone is that not everything is measurable. It is dangerously naïve to dismiss this argument by saying that whatever can’t be measured cannot possibly be important. That’s just wrong. We haven’t figured out yet how to measure creativity or subjective quality. We can’t quantify strategy. We can only guess the likelihood that a plan will work. We don’t know how to measure leadership. There is no metric for user experience. These are all examples of critical aspects of business that are not measurable. 

Sure, we can try to measure these factors using feedback, approval rates, and clicks, but they often yield imprecise and biased insights, capturing a limited perspective. The basic question “Is our product any good?” is not answered by the count of priority A defects. The question “How strong is our brand?” is not answered by the number of LinkedIn followers. And the question “Why did we miss our revenue target?” is not answered with pipeline conversion data. These data points may serve as interesting proxies but just because they can be measured, they don’t give us the full picture. 

Any decision in such immeasurable areas has to rely on experience, judgment, and talent. Unfortunately, these decisions are frequently entrusted not to the individuals with the right expertise but to their managers, usually one or two levels above, who potentially lack the full context required to make the call. BTW, one of the greatest opportunities for AI is right here - let the AI figure out how to make these decisions.

Further, because these unquantifiable areas don’t fit into a dashboard format, very few discussions tend to be devoted to them. Some companies can spend way more resources on analyzing their win rate than on analyzing the strategy that drives the win rate. They spend more time on measuring Sales enablement than on actually enabling the sales reps. They spend more effort on analyzing NPS results than on talking to customers. Focusing on what is measurable can lead to avoiding the real problem. 

I am by no means suggesting that metrics don’t matter. When we have good metrics, we would be foolish not to use them for making well-informed, data-based decisions. But making decisions based on good data is easy. The tough decisions are the ones that have to be made with little or incomplete data. We cannot neglect the non-measurable areas of business just because we can’t track them on a dashboard. 

After all, the great Albert Einstein once said: “Not everything that counts can be counted, and not everything that can be counted counts.” 

The Maslow's Hierarchy of a Strategic CIO

Not that long ago, IT departments were responsible mainly for making sure that knowledge workers got access to whatever information was available and for keeping the lights on. That latter responsibility was all consuming. PCs and enterprise systems were still going through their growing pains and varieties of system failures were all too common. The job of the chief information officer (CIO) was more about troubleshooting and firefighting than anything else. Information Technology (IT) was simply a cost center - just like travel or communication.

Well, things have changed. The systems we work with are, for the most part, reasonably reliable. We don't have to reboot our PCs twice a day just to flush out the memory leaks. 99.999% uptime is not that big of a differentiator for servers, routers, and switches anymore. Users don't call the help desk daily and access to information is not the challenge at hand. Today, we have information. We have a lot of information. In fact, we have way more information than we could ever consume.

Herein comes the change in the mission of IT departments. As organizations came to realize that information emerged as a key source of competitive advantage, they were increasingly looking at their IT departments as a key stakeholder in corporate strategy. All of the sudden, the CIOs got what they were always dreaming about. No longer the troubleshooter, no more the firefighter - the CIO is now the strategist.

What organizations need is the ability to make better decisions - using the right information. They need insight. They also need to apply information to create an impact on their business - to grow revenue, attract new customers, enter new markets, and generate innovation. And finally, they also need to drive productivity and continuously optimize their business processes.

Not to forget, enterprise information must also be secured. As it represents significant intellectual property, it has to be protected from intentional or unintentional misappropriation by internal or external actors. And let us not forget the need to address compliance and information governance requirements and to protect the company from legal exposure.

All these requirements reminded me of the Maslow's Hierarchy of Needs and so I have attempted to map the CIO needs into a similar model:

With all of this, the strategic CIOs have their hands full. They still need to keep the lights on but now, they are major stakeholders in defining corporate strategy. Combine that with all the new technology trends such as mobile devices, social software, and cloud computing and you get the picture of the magnitude of their challenge. But I guess that being strategic is way better than fighting fires all day long, right?