The Perfect Metrics

I think we as a species inherently love to measure things. I take that back. We love to measure everything. I am not a baseball fan, but I find it humorously entertaining the number of statistics that are available for seemingly any situation in baseball. I think it is possible to find the batting average for any player in late innings, with runners in scoring position, for away games with left hander pitchers on the mound. Really? I guess there must be someone interested in all that, but I can’t think of who it might be.

I am a hockey fan and there are a whole new generation of metrics created which I am not sure I entirely understand yet, but are supposed to give a much better measurement of the quality of the hockey players on the ice today. It seems that you can now get statistics for third line shots generated or allowed, for defensive players on offensive zone draws in the third period. Okay. I understand what all that means, but I am not sure if I care. Just drop the puck and skate.

I am not quite sure but it seems that some people are trying to make hockey appear to be more like baseball through the use of more and more arcane and detailed metrics. Unless they allow baseball players to carry their bats with them out into the field when they play, (stealing bases would get a whole lot more interesting) instead of just when that are at bat, or figure out a way to make hockey a whole lot slower and more boring, this would not seem to be a plausible goal.

The roundabout introduction here is that to generate all of these baseball statistics, someone had to measure and record all of these actions and variables. They had to create the metrics. And once they created these metrics it became a challenge to create the perfect metrics to more perfectly measure and reflect the game. After over one hundred years they are still trying. This should convince everyone from the onset that there are no perfect metrics. There are only good metrics, and other measurements.

I have had the opportunity in the past to be involved with many metrics projects, programs and functions during my time in business. It has been both an enlightening and useful process to me. It has helped me on several levels when it comes to the successful leadership of a business. In business as in sports, metrics are in part how we keep score.

Metrics are interesting in that they are indicators of performance. Hockey players with good performance metrics tend to be on good teams. Good teams tend to win more games. Winning is usually thought of as being a good thing. The new, complex metrics associated with Hockey seem to go a long way toward providing supporting evidence for how good and accurate the older simpler metrics associated with Hockey actually are. Interesting how that works.

It also seems to go that if a few metrics provide a reasonable indication of individual or business performance, then as we have noted in baseball, a very large number of metrics should provide a significantly more specific and detailed indication of individual or business performance. This thought process is along the lines of the old adage “If a little is good, a lot must be better.”

To extend the baseball analogy that is like saying if a beer or two is good while watching a game then two cases of beer should be excellent. You can find yourself at the game in a state of unconsciousness, immobility or alcoholism.

Similarly you can find yourself in business with so many metrics and indicators that they will begin to provide too much, or even conflicting indicators to the point that you end up in an immobile situation. Hence the phrase “Paralysis by Analysis”. I think I may prefer to refer to this situation as “metricoholism”, or the over dependence on and addiction to metrics to the point of being dysfunctional.

Metricoholism is the inability to have just one, or even a few meaningful metrics. It’s more along the lines of once you get started measuring things, you can’t stop. Eventually you will have measured everything, but will then have no idea what to do about all that you have measured.

I have found that the value of metrics lies in the talent of the people that are interpreting them. Metrics in and of themselves need to be the indicators of where additional human interaction with the business processes may be required in order to understand the possible underlying issues associated with the numeric measurement anomaly (metric). Good metrics identify the leverage points where analysis and performance modification can have the greatest effect on the business. Good metrics simply point to where the leader must look to understand what is affecting their business’ performance indicator.

There was a recent movie about the use of metrics in sports. It was called “Moneyball”. It was nominally a baseball movie, which meant for me that I would wait for it to be on television before I would watch it. I usually don’t pay money to watch a live baseball game because it is as I said a rather boring game to me. Why would I pay money to watch a movie about a rather boring game?

Just as an aside not all baseball based movies fall into this category. I thought “Field of Dreams” and “Bull Durham” were very entertaining movies, in spite of their baseball based premises. However “The Natural”, not so much.

In any event, Moneyball was the story of how a specific baseball team changed the way the business of the sport was conducted. By changing the way that the humongous amount of data associated with baseball and the baseball players was interpreted, they changed the way players and teams were viewed, built and paid for.

That bears repeating. By changing the way that the standard data (that was available to everyone) about the game and each of players was interpreted, one team changed the way an entire century old sports institution looked at how teams were built and how they should best perform.

The value was not in the data. Everyone had the same data. The value was in how the data was interpreted.

While interpretation of the data is going to be the key to success when it comes to metrics, it is also best to remember what Robert McNamara (one of the original automotive industry “whiz kids” of the 1960’s) said. He said:

“First thing: Get the data.”

The point is that there is a lot of data available. Which data do you go get. If you were a Metricoholic you would end up trying to get all of the data, since partial data would not be satisfactory. Also as previously noted, this would be a mistake. It takes far too much time, money and effort to do this and what are you going to do different with one hundred percent of the data that you wouldn’t do with eighty percent of the data.

That was an oblique reference to the old eighty – twenty rule where you can get eighty percent of the data in twenty percent of the time. If you can get eighty percent of the data reasonably quickly, you can make excellent business decisions from that data, and move on.

Good metrics for a business need to be relatively simple and straight forward. They need to deal with the basic functions and core values of the business, not the ancillary capabilities. Revenue, costs and profitability are good examples of simple metrics that all businesses use. I think there is probably a good reason for that. Performance levels and adherence to service levels are good metrics for service related industries. There are certainly others and they can be customized by business type and industry.

The key and the value to good metrics lie in their simplicity and their interpretation. Complex metrics just provide you complex data that is difficult to interpret. Exhaustive numbers of metrics generate exhaustive amounts of data that requires exhaustive interpretation. No amount of metrics, or process for that matter, can replace the need for talented people who can interpret the data, then decide where and what to act on.

The idea of good metrics should be to create a few indicators that measure the specific core leverage points of a business or organization. They should provide both a historical trend (are they getting better or worse) and a specific snap shot of performance. They should indicate where the interpreter of the information should go look for issues, if they are indicating issues. They should not be expected to indicate what the cause of the problem is, and certainly not what the solution to any particular issue will be.

Almost every business in existence already has some sort of metrics. Some are probably good metrics and some are probably just measuring something. There will also probably be those in the organization that are clamoring for more metrics as a way to improve performance.

However, I have found that good metrics are usually like bitter medicine. They are best and most effective when delivered in small doses, and usually best prescribed by someone outside the organization that does not have a stake hold to protect.

Just like healing oneself, measuring oneself is sometimes a difficult thing to accurately and honestly do as well.