Metrics: Friend or Foe?

The subject of metrics can be one of the most basic (and boring) topics in the world of business management.  But few things you manage are as critical to your success!  If your focus is on metrics that tell you what happened last week or last month, or have little to no visible effect on your customers, or the behavior of your employees, your metrics are not your friend!  Our purpose in this article is to examine our data and our metrics and ask ourselves some critical questions that will lead us to determine if we do indeed have the proper metrics focus.  X-Stream LEAN also believes that there are Four Phases in the life of a metric that determines the effectiveness of the decisions made in driving value as determined by your customer.  Phase I is Data Collection; Phase II is Data Analysis; Phase III is the Decision Making Process; and Phase IV is the Evaluation of the Decision (to include the impact and results of that decision).  Most organizations accomplish Phase I and Phase II very well.  The discriminator in World-Class performance is found in how well Phase III and most importantly, Phase IV are accomplished.

Imagine yourself in the cockpit of a new modern fighter aircraft: alone, at night, in bad weather, and in hostile territory.  You are totally dependent upon the information found on the screens in front of you to know and understand what is happening outside of your aircraft.  Every decision you make can result in life or death and every decision is only as good as your ability to understand the information in front of you on your dashboard.   Not only are you flying the plane and navigating to a target to complete your mission, you are also trying to avoid all of the things nature and your enemy can throw at you to prevent you from succeeding and to destroy you personally.  Not an easy task.  The cockpit of a modern fighter aircraft is one of the most highly advanced, technologically superior information management systems ever created.  The pilot must be able to receive data, develop information, analyze options, and execute decisions that will decide life or death in fractions of a second.  Now think about your Information Management System (IMS) or your Performance Management System (PMS), and your data.  Far too often our digital dashboards and our information systems provide a flood of information, metrics, data, and (semi) current status of our business to the point that it is often overwhelming. How you manage this flood of information and the decisions you make as a result can have the same make-or-break effect on your business as those made by the pilot on his mission! 

Metrics and the data that drive those metrics are the lifeblood of any organization that is intent on succeeding in today’s market.  Your ability to interpret data and develop it into information to be processed into knowledge with sufficient accuracy to allow you to make effective decisions is the foundation upon which your organization achieves success or failure.  At X-Stream LEAN, we believe that World-Class organizations approach their IMS and their metrics management in the same manner as the pilot in our analogy.   They focus on the critical information as opposed to “everything”; they utilize Visual Management Techniques to understand what their metrics and data are telling them; and they make a decision.  The point that establishes them as World-Class performers is the time they spend evaluating the decision to determine if it had the desired effect on the business and if not, WHY NOT?  The pilot knows, good or bad, within seconds the effect of his or her decision.  How long does it take for you to know the results of your critical decisions?

The cockpit example can be a powerful representation of the digital dashboards that many businesses are using today to manage the information they require to guide, direct, and control their operations, but without the immediate stress of life and death.    Because of the technological advancement of software and computer graphics, the Digital Dashboard screens are often a thing of beauty.  They are aesthetically pleasing, and often times they do a very good job of presenting the data in formats that are very easy to understand.  But we can become so involved in the technology and the “Buck Rogers” that we lose sight of the fact that our critical question should be “how important is this piece of data or this metric to our customers?”  When we begin to measure everything, we focus on nothing.  At X-Stream LEAN, we believe this is a critical failure mode in today’s environment and an effect that, with increased implementation of computer aided technology, is becoming worse, not better!  In most businesses, it is far better to focus on a few critical and key metrics that are vital to your customers than it is to focus on a large amount of data that is nice to know.  Even worse is producing reports or documentation to “Cover Your Assets” or producing something that somebody somewhere in your organization asked for years ago and you still produce “just in case”.

So What Can We Do?

There are a few key questions you can ask yourself to determine just how critical any metric or any piece of data really is.  How well does your organization collect the data you use?  Is it the right data?  Is it formatted and processed properly to allow effective analysis?  Does it lead you quickly and effectively to the right answer or solution?  How “current” is the data upon which you are relying for the success of your business or your enterprise?  Would you bet your life on the accuracy or reliability of this information?  If not, why would you bet your business?  We all recognize the importance of our data and our metrics, but far too often we take them for granted.  While we intuitively know how important this is, we often fail to spend enough time up front to ensure that we have the right data, presented in the proper format, developed at the proper time, and delivered to the right person, in order to make the right decision.  We need to know if we are focusing our attention on items of criticality as opposed to just reams of information and reports that are just “nice to know”.

The Four Phases In The Life Of A Successful Metric

With the understanding that data and metrics are crucial in the management of every organization, let us explore each of the four Phases in the life cycle of a typical metric and evaluate how typical, good, and World-Class organizations accomplish each Phase.

 

Phase I: Data Collection OR “A Flood of Biblical Proportions?”

The first phase of the metrics life cycle is one that almost every organization does almost too well.  With the advent of modern technology, we are now “swamped” with data, numbers, and facts.  Organizations have developed a tendency to measure anything that can be measured “just in case”.   This leads to the common “analysis paralysis” that is frequent enough to be a cliché in all modern business schools.  The ability to measure something does NOT mean that the measurement has value or will be used to drive a critical decision.  The drive today to “warehouse” enormous amounts of data is obscuring a simple fact.  You may have terra bytes of data in your “warehouse” and still go bankrupt because you did not have access to that one piece of critical information when you needed it most!

 

It is very common that the most important things for an organization to know are also by their very nature the most difficult to measure.  This difficulty often results in critical decisions being made based on purely financial data because that is the most easily measured metric.  Money is vital and should not be underestimated nor under appreciated.  Business runs on the Big Five (Cost, Sales, Orders, EBIT, and ROI).  However, critical business decisions that truly drive value to the customer are often far more complex and require data and consideration that may not be nearly as easily measured or analyzed.  It is our observation that World-Class organizations expend significantly more effort to do the hard work of figuring out what information the organization does need to make their most critical decisions and how they gather and measure that information.  Typical organizations all too often minimize this aspect because the answers are difficult to obtain and business leaders are overwhelmed with the day-to-day decisions and problems that consume their time each day.  World-Class organizations ensure they have a strong external focus on data collection as well as an internal focus.  Developing a strong “situational awareness” of what is going on outside the organization is just as critical as understanding what is happening “on the floor”.

Phase II: Data Analysis OR “What Do I DO With All This?”

The second phase in the life of a metric is in analyzing the data that has been collected by the organization.  The majority of typical organizations also do this function well.  Nothing receives more scrutiny than the monthly financial report.  It is very easy to compare this month’s performance with last month’s performance and to compare the actual performance achieved against the Plan.  They do this in part because this is a very important aspect of any business and ensuring the organization’s long-term success is vital.  But another reason is that it is very easy to do and unequivocal in its conclusions.  However, business is far more complex than just the numbers contained in the monthly report.  Business is about understanding the customer and what they value!  What metric does your organization use to track this extremely critical business issue?  Your focus should be on developing a successful long-term vision and an effective strategy to achieve that vision!  What specific metric does your organization use to track your progress in this area?  Additionally, business is about knowing the strengths and weaknesses of your competitors and those of your own organization and how you can minimize or eliminate your weaknesses and take advantage of your opponent’s weaknesses in the marketplace.

Analysis of your data is critical in order to be able to understand current reality and making effective adjustments to correct for things that change daily.  This analysis allows the business leader to develop “situational awareness” that gives you the ability to see what is happening right now and to learn valuable lessons about what has happened in the past.  World-Class organizations are moving aggressively toward utilizing more difficult to gather predictive, pro-active, strategic metrics while minimizing the more traditional and easily measured reactive metrics that tend to provide excellent rear-view vision.

How many reports are in your files “just in case” somebody asks for the information?  How many reports does your organization generate that no one reads or uses but is generated because somebody at some time in the distant past “asked a question”?  One of the biggest wastes X-Stream LEAN finds in most organizations is this culture of never questioning reports or data collection.  If you cannot get a clear and unequivocal answer as to who uses it and what they use it for, that data or report is certainly a potential candidate for elimination.  Think about how much money could be saved by an organization and how much time could then be spent on making better decisions that impact your business if we eliminated waste of this magnitude!

This is the point where most businesses begin to falter in terms of World-Class use of metrics.  Decisions are made based on the data available and the analysis of that data.  Historically, a decision will be no better than the foundation it was built upon.  Was the analysis correct or was it flawed because the data that drove the analysis was bad, incomplete, or misinterpreted?  Was the data itself the right data or was it simply the easiest to measure?  Are we inappropriately using inference techniques to drive our decisions because the data we would have liked to have used was just too hard to figure out?  Are we making our decision based on internal and external data?  Have we developed the proper “situational awareness” to make the decision within the proper context?

Having the data needed to make good decisions is assumed by most organizations to just be a given.  Yet careful review reveals that most of the time, people make judgments without solid data to support their decision.  This can be because there was insufficient time to develop the required data (legitimate); the data we did have was thought to be sufficient; the data was too expensive to acquire; or we just could not figure out how to measure it easily or quickly enough to be of use.  The opposite side of this argument is “analysis paralysis”.  This is the act of spending too much time waiting for “perfect data” before making a decision.  As in most things in life, a position somewhere in the middle is most effective. Learning how to recognize when you have ENOUGH information to make critical and effective decisions without waiting until you have “perfect information” is a critical skill for any successful executive, professional manager, or good supervisor.

For a multitude of reasons, bad decisions always have a negative impact on your organization.  Mitigating bad decisions is in everyone’s best interests: your customers, your stockholders, your employees, and your own.  Taking the time to understand what data you need to make decisions and determining how to generate that data is of critical importance to you and your future success.  This aspect of business management is far too important to just accept the status quo and take what you get!  Most organizations are good at making decisions that eventually end up with good results.  World-Class organizations develop a robust decision making process that drives them to good decisions.  This is a major difference!

This is the phase in the life of a metric that very few organizations excel in accomplishing.  Once a decision is made, how does your organization determine the results of that decision?  Major issues like strategy or corrections in the organization to meet financial objectives are usually the only areas where this is even addressed in a rudimentary fashion.  We make thousands of decisions each month from the critical to the mundane.  How many of those do we track to understand what happened as a result of our decision and did the results meet our expectations?  And the most critical question: If not, why?  Was it a bad “gut level” decision”?  Was it the result of a good decision making process that experienced a bad outcome?  Or was it the result of an unknown or unobserved variable that we failed to take into account that was only recognized as critical after the fact?  Each of these questions provides an answer that is “golden” in that something can be learned that will benefit you and your team from that point forward.

We cannot track everything and mundane decisions are not the point of our discussions.  However, we make numerous critical decisions that are important to our business.  How many of those that are important do we follow up on to find out what really happened?  In our experience and in doing some research, X-Stream LEAN has learned that in many cases the results turned out fine and nobody in management ever asked a single question.  Further investigation reveals that the initial decision was “flawed” and required superhuman effort by subordinates and the work force to actually achieve the “good” result!  All of the activity required by the organization to achieve the desired result was “invisible” to the person making the initial decision and all that they could see was the ultimate result.  This “churn” in the organization is extremely costly and this activity seldom shows up on a financial report because “we don’t track that kind of data”.  Yet, as a result of the hidden nature of the activity required to achieve the desired result, few, if any, lessons are learned by those making the original decision.  This is extremely common and particularly frustrating to subordinates and ensures that little, if any, progress is ever made in the organization’s ability to adapt, grow, and develop in a changing world.

World-Class organizations are distinguished by their ability to use the results of bad decisions as effective learning points in their relentless drive to continually improve.  This ensures the long-term development of both their organizations and their people.  That is how we learn, how we grow, and how we develop.   One of my early supervisors allowed me to make a decision of some importance as a new manager.  As I was reviewing the results with him (and preparing to receive my beating), it became very clear to me that he knew precisely how the outcome was going to develop before I made my decision.  I asked him why he allowed me to make the decision knowing full well it would be disastrous!  He calmly informed me that had he overruled me, I would have always believed in my heart my way would have produced better results.  And as a result of my new-found experience, I would be a much better manager from that point forward.  I asked him about the cost of my decision to the organization.  He asked a simple question: How much would a formal off-site course have cost me to attend?  “Your experience here has cost us far less than the class and you will remember the lesson far longer.  It was a good investment on our part to allow you to experience this as you did.”  He was right and I have never forgotten the lesson learned or his wisdom as my mentor.

For too many organizations, data is not important until there is a crisis.  Then, the focus is more on “air cover” than in proper root cause analysis.  Organizations with proper discipline collect the right data, but they do not succumb to the urge to make the data fit their conclusions.  They also do not use data as much for historical purposes as they do to determine future direction.  For X-Stream LEAN, the concept of organizational discipline as it regards metrics is about maintaining the integrity of the data no matter what the message or the conclusion of the result may be.  Manipulating numbers is so commonplace in industry and government it is considered to be unremarkable.  Executives may disagree…but ask the employees how much faith they place in the data shown.  How many of you are familiar with the annual “kabuki dance” involving bonus metrics?   Remember, it is not just data integrity to which we are referring: it has to be message integrity.  Are we willing to hear what the metric data is telling us?

Conclusion:

J. R. McGee

President and CEO

X-Stream LEAN, LLC

www.xstreamlean.com

A Service-Disabled Veteran-Owned Small Business