Can we measure it?

[David Kerridge <dfk@mwfree.net>]

I started working on this post a long time ago, but the
topic has come round again, just when I am ready.

What did WED mean by saying that some things can not
be measured? We can always find numbers of some kind
that reflect what is happening, and many people call these
numbers measurements. Nothing wrong with that, except
that it is confusing when Deming uses the words differently.

Deming, I believe, took a very strict view of measurement.
He was both a statistician and a physicist.

I don't remember him ever explaining what he meant by
"measurable", so I don't want to imply that the following
represents his usage of the words, just mine. I am not
taking a dogmatic view, simply trying to clarify the way
we use words (and numbers)

1   There must be something to measure.

     In other words, the number must correspond to some
     concept which represents a property of the real world.

2   The property must be capable of being expressed
      adequately by numbers.

3    There is an agreed operational definition by which the
      concept gives rise to a number.

4    The number obtained by this definition must not be
      affected by other factors.

5    The number has predictive value: in other words, the
      measurement has "temporal spread."

6    A really useful measurement does more than predict
      future values of the *same* measurement: it predicts other
      properties of the real world, which may not themselves
      be measurable.

These may sound trivial, but check the "measures" used
in your organisation against them. You may be surprised.


A proposed measurement may satisfy these requirements
in some contexts, and not others. If so, we should make
it clear what context we are considering. When we say
"The speed of light" we mean the speed of light in a
vacuum. Without this defined context, the speed of
light is meaningless.

There are many numbers which do *not* satisfy these
six principles, but are valuable if used with care. These I
call "Indicators" rather than measures. For example, if
we want to study customer satisfaction, the number of
complaints is an indicator, not a measure. It fails on (4),
because it is influenced by so many other things. The easy
way to get fewer complaints is to make it difficult to
complain - like passing complaints on to a "customer
service department" and then not replying, or replying
with a standard letter. Or arranging for the "Customer
service" phone number to be always busy.

We can use indicators, with due precautions, when trying
to improve a process. Indicators frequently depend on the
state of the system. As systems change, the indicators will
change. So indicators are not suitable for long-term comparisons,
or comparisons between different systems. This applies even
if they are apparently similar, such as different hospitals,
or different departments within the same organisation.

Indicators are particularly dangerous if we use them to judge
people or departments. If the other factors which affect the
indicator can be influenced by those who are judged, they
will feed us misleading information. If the other factors are
beyond their control, they will be demoralised.

If we could measure everything that mattered, management
would be easy - well, a lot easier. The more we use indicators
rather than measures, the more we must rely on theory. And
it had better be good theory.

There are many indicators of quality. There are measures of
some aspects of quality, such as the hardness of steel. But
*overall* quality cannot be measured at all.

Best wishes
David
dfk@mwfree.net
=======================================================================