What gets measured gets improved… or not

On the surface, it seems that measuring key metrics for a business can lead to improvements.  Many businesses will try to quantify their operations instead of relying on qualitative goalposts.  This is probably a good idea in most situations.  However, the real world is messy and difficult.  Sometimes employees will game the metrics used to measure their performance.

Combating shrinkage / employee theft

One of the problems in a food service business is that employees will sometimes steal.  They can sell a food item to a customer but keep the cash.  One way to reduce this type of fraud is to count your inventory and your sales.  If there is not enough sales for the inventory consumed, then there is a problem.  On the surface, this might seem like a fool-proof way to measure and combat fraud.  But sometimes the real world is not so easy.

Suppose that the business sells slushies and the manager keeps track of the number of cups used.  One way to game this system is for the employee to bring in their own cups.  The employee can dig cups out of the garbage and wash them (yes, this is disgusting).  I don’t know what a good solution for this problem is.  But it just goes to show that running a business is hard.

Sometimes public companies don’t report relevant metrics

Virtually all retailers measure inventory shrinkage.  Revenue and inventory are generally tracked by computers and there is typically inventory that goes missing due to shoplifting, employees stealing from the company, etc.  Reducing shrinkage is really difficult and is the reason why some CEOs are better at retailing than others.  When Sam Walton was running Walmart, he didn’t want other companies to know how low Walmart’s shrinkage figures really were.  He thought that other people would try to figure out Walmart’s tricks, such as employing greeters to say hi to all the customers and potential shoplifters.  (It so happens that Walton saw another retailer use greeters and stole their idea.)

Nowadays, I don’t know if Walmart is still good at combating shrinkage.  If you look on the Internet, there are websites that explain how Walmart loses inventory.  These little things won’t be obvious from reading a 10-K because it’s simply not reported.

DaVita and the for-profit dialysis industry

In dialysis, various metrics are used to measure the adequacy of dialysis (e.g. Kt/V).  There are two ways of improving the numbers:

  1. Deliver proper care.
  2. Redo measurements until there is a better number.  As I understand it (I could be wrong), the tests aren’t precise.  If you take several measurements one after another, there will be some variation in the test results.  This allows employees to game the system so that their clinic reports good numbers.

The reason why employees don’t deliver proper care instead of redoing tests is because proper care takes more time.  The employees are often overworked because under-staffing dialysis clinics can increase profits.

Measuring the performance of CEOs

Management teams will often choose the set of metrics that make them look good.

  • Non-GAAP earnings that have “one-time” charges year after year.
  • Silly metrics where stock-based compensation is not an expense.
  • Rosy projections about the future.
  • EBITDA used in inappropriate contexts.
  • Taking credit for factors outside their control (e.g. higher profits due to commodity prices).

Instead of using management’s set of metrics, investors should look at the metrics that track how well management is doing and whether or not they are making decisions that make sense.  Of course, sometimes metrics are flawed and should be looked at with precaution.  The underwriting quality of an insurance company is very important but it can be very difficult to quantify.  The combined ratio of past underwriting may be a poor indicator of the future.

Bonus structures

CEOs and employees will often try to game the bonus structure that is imposed on them.  This can be particularly bad at investment banks with proprietary trading divisions.  Employees will make dumb trades to increase their bonuses.  Investors need to be careful because incentives often drive behaviour.  A flawed incentive structure can allow bad behaviour to proliferate.

However, there are some people in the world that have integrity.  They don’t try to game their bonus structures.

  • Altius Minerals has a dopey bonus structure that rewards reported earnings.  Brian Dalton, the CEO, could easily take his bonuses now by selling a tiny fraction of Altius’ royalties so that Altius can realize mark-to-market gains.  He has not done this.
  • Some CEOs will pass on their bonuses because they feel like they don’t deserve them.  John McCarvel (Crocs) and the late Ken Peak (Contango) have passed on their bonuses in the past.

The bottom line

I think that to truly understand an industry, you need to know the answers to the following questions:

  • Which metrics are the most important?
  • What are the problems with those metrics?
  • Which metrics are nonsense?

An industry should not be considered part of your circle of competence until you can answer those questions.

*Disclosure: Long Crox, Altius, and Contango.  I haven’t been able to sell my Contango shares at a good price but they are on the chopping block.

One thought on “What gets measured gets improved… or not

  1. In part, what you are describing is Goodhart’s Law: “Once a measurement becomes a target, it no longer is a good measurement.”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.