x

Finding the right metrics to avoid analysis paralysis

July 24, 2008

Last Monday, we came across an interesting little blurb in Harvey Schachter’s “Monday Morning Manager” section in The Globe and Mail. Under the section entitled ‘Power Points’, Schacter quotes Jakob Nielsen:

We need a new metric: Businesses need to stop using “unique visitors” as a metric for their websites, as many people drawn to sites these days have come by clicking a link on another site or a search engine and quickly leave. Site tourists who leave a site immediately ratchet up the unique visitor count but don’t contribute long-term value.”

True…unless of course, you’re looking for ad sponsorship — in which case, eyeballs count. Nonetheless, we found Nielsen’s point well made: we often tell our clients they have all the wrong metrics in place.

Laurie Clarke, COO of The Tatham Group says, “Many of our customers have reams and reams of data. We ask them to produce numbers, and they say, ‘sure!’ and plunk piles of paper in front of us. The truth is, most of the time, those numbers are absolutely meaningless.”

Clarke is right. Frequently our customers can tell us exactly how long it takes to reply to a call, or how many calls they process – but to a customer that doesn’t mean a thing. Think about it: do you really care if your phone company answered 50,000 calls or 70,000 calls in one day? Not likely.

That’s why we teach our clients to look for the right metrics. Clarke says, “I can tell you that the metrics we believe in, never exist before we arrive.” The ones she’s talking about are as follows:

  • Cycle Time: This refers to the time it takes for a customer to place an order to the moment the customer receives it. Let’s stick with phones and imagine you order a new handset. Cycle time refers to the moment you place the call to the moment you begin successfully using your new phone.
  • Process Yield:This metric refers to the number of units – whether it’s a mortgage, an insurance policy or a chocolate bar – that have gone through the process once and been 100% quality.
  • Rework: This is a close cousin of process yield. Rework refers to the number of times a product has to be re-done, re-touched or goes backwards in the process in order to be corrected. [Hint: rework is often a huge contributor to long cycle times.]
  • Patterns: This is often a point of contention between The Tatham Approach and Six Sigma. We teach our clients to conduct customer surveys. We believe that you don’t have to talk to 10,000 customers to find out if there’s a problem. Six Sigma disciples argue that data must be statistically significant to be counted, but Clarke says, “It doesn’t take long to figure out when a hundred customers are telling you the same thing, there are probably another 1000 out there thinking the same. And if you have even 100 unhappy customers, you know you’ve got a problem.”
  • Average v. Range: Many customers focus on the average, or the median. We believe the more important figure is the range. If you have one customer who waited for a product for five years, and another who waited for a day, that tells you a lot. The one who waited for five years, means you have a major problem with your process. More importantly, is that it tells you your process is not under control. The one who only waited a day, is the more exciting figure — that’s your hidden capacity. If you can do it for one, you can do it for them all.

Data and measurement are crucial to making effective decisions. But often managers get inundated with too much of it, so they panic and freeze. That’s what we call analysis paralysis: getting caught up in the minutiae of statistics rather than keeping the big picture in mind. And ultimately, the big picture will always reflect the customer. So, put yourself in a customer’s shoes: they don’t care if you’ve refined the process of making a phone down to a few hours, but they do care if they get the product they ordered and they get it on time.

  • Pingback: linkingclever.info » Blog Archive » Finding the right metrics to avoid analysis paralysis()

  • Harvey Schachter

    It wasn’t my comment. I was quoting Jakob Nielsen on his Alertbox, so he deserves intellectual credit for the idea/comment not me:

    · Businesses need to stop using “unique visitors” as a metric for their web sites, as many people drawn to sites these days have come by clicking a link on another site or a search engine and quickly leave. Site tourists who leave a site immediately ratchet up the unique visitor count but don’t contribute long-term value. Source: Jakob Nielsen’s Alertbox

    Harvey Schachter