Resistance to new ideas (2)*

[David Kerridge <dfkerridge@mac.com>]

In the previous post in this series I gave two examples of resistance
to new ideas in statistical theory. The paper by George Box entitled
"Scientific Statistics, Teaching, Learning and the Computer" gives
more details. It can be downloaded from

http://www.engr.wisc.edu/centers/cqpi/reports/pdfs/r146.pdf

This reference was supplied by Rip Stauffer.

I want to suggest a theory about why there should be such
resistance, and how it relates to our problems of spreading the
Deming Philosophy.

I believe that the three approaches to statistical inference do not
come just from differences about statistics. They correspond to
three different views of what *science* is about. What follows is
not an exact description of what philosophers say (or said) that
scientists should do, but based on my own experience in using
all three approaches, and observing what other scientists do.

1.  Logical/Mathematical

     Science is concerned with logical proof. It therefore
     requires an all or nothing view - a theory is true or false,
     and must be accepted or rejected.

2.  Explanatory

     Science is concerned with explanation - the reasons why
     things happen. Approximate models, like representing
     atoms by billiard balls, are useful, if they make the
     explanation easier to understand.

3   Predictive

     Science is prediction - no more, no less. Prediction must be
     based on observation, and observation defined in terms of action.

These three views of science correspond to different stages of
development. The logical/mathematical view was unquestioned
throughout the middle ages, and reached its peak with Descartes.
Scientific truths were deduced by strict logic, starting with
self-evident axioms, as in Euclidean geometry.

From the time of Isaac Newton, science changed. I believe
(I have not checked the originals) that Newton presented his
ideas in the old format, as deductions from axioms. So successful
was he that some later writers claimed that Newton's physics
is true, not because of observed facts, but by pure logic. But for
most people, what Newton did was to provide an explanation - the
force of gravity.

Explanation need not be exact. As George Box has put it:
"All models are wrong. Some are useful."

At the beginning of the 20th century, both forms of science fell apart.
There were two blows to previous thinking. First of all, many
"self-evident" ideas turned out to be false. An example is Einstein's
demonstration that time is relaive. Secondly, the idea of explanation
itself was called into question.

Quantum Theory, in particular, provides no explanation we can
understand. But it predicts strange and unbelievable outcomes,
and predicts them with amazing accuracy.

Most people are unaware that science has changed. Only those
trained in theoretical physics (like Shewhart and Deming) have
adopted the new philosophy of science. Others are stuck in
the thinking of earlier centuries. And because knowledge is
now so specialised and compartmentalised, few scientists are
aware that different ideas are taken for granted in fields other
than their own. We are dealing, in most cases, with unconscious
assumptions, rather than conscious beliefs. That makes them
far harder to deal with. It seems that many people cannot
face a challenge to what they think is "obvious" - though the
System of Profound Knowledge is one challenge after another.




My theory about the three approaches to statistics is as follows.

1    Neyman and Pearson saw science in the logical/deductive
      mode, which is still common among mathematicians.

2    R A Fisher had extensive experience of biological science.
      He became, in fact the head of the department of Genetics at
      Cambridge. Like most scientists, he saw science as explanation.

3    Walter A Shewhart and W Edwards Deming saw science in
      terms of the new physics of prediction and action.

I started with statistics because the historical record is so striking.
But the other examples are also well documented. Semmelweiss
demonstrated that hygiene saved lives. But nothing was done
until Pasteur explained the reason for it.


I apologise for what may seem to be lengthy theoretical rambling.
But the strange thing about the Deming philosophy is that the most
abstract ideas turn out to have direct practical applications. It is
not surprising that science based on prediction and action is exactly
what we need for management.

In my next instalment I hope to show that this helps us understand
some of the difficulties we face.
--
Best wishes

David

dfkerridge@mac.com