Knowledge and Variation: Some thoughts on Deming and Fisher

[Jonathan Siegel <jmsiegel.info.research@worldnet.att.net>]

I've received some requests to cite or use this post. Here is a revised
version with references and a few emendations that might make it more
suitable.

"The rate of increase in fitness of a population at any time is
proportional to its genetic variance in fitness at that time"

Sir Ronald Fisher, "Fisher's Fundamental Theorem of Natural Selection"
[2]

Corollary:

"The more adapted a population is to its current environment, the less
able it will be to adapt to a new environment"

-- Gerald M. Weinberg [2]

The quality movement has sometimes been associated with the idea that
variation is an inherently bad thing and the aim of a person interested
in improving quality should be to reduce variation wherever it appears.
One of the main recent streams of Deming criticism -- the stream that
showed up in John Dowd's "That Paper" [3] thread a couple of months ago
-- basically associates Deming and the quality movement with conformity
and argues that in some areas of life, education in particular, too much
conformity is a bad thing.

I don't believe this is an accurate criticism of Deming. I believe
Deming would be inclined to agree that reducing variation as such cannot
be the main aim of a business or a society that wishes to survive
long-term. Sometimes variation is a good thing, and it is the duty of a
manager -- and a society -- to increase it. As Bill Towns explained in
his 1997 "Deming as Pragmatist" paper [4], Deming borrowed a great many
of Fisher's ideas.  I think Fisher's Fundamental Theorem plays quite a
role in Deming's discussion of learning and variation.

"If you keep doing what you did before, you will keep getting the same
results."

-- Unknown

But only as long as the world around you stays the same, and it won't
forever.

-- Unknown

In order to learn, or to improve, we have to do something different,
then see if it works. The PDSA cycle is a form of adaptive learning that
represents an accelerated, systematic form of natural selection. It is
Fisher's evolutionary engine set up in the shop and run more
efficiently. I believe the dynamics underlying the PDSA cycle are in key
respects not fundamentally different from those of natural selection,
even though in the shop variation can be intentionally designed. I think
Deming looked to Fisher to understand how it works.

In the PDSA cycle variation represents the fuel that drives the engine,
just as it does in nature. And if Fisher is right, our success will be
related to how much variation we are able to generate in a given amount
of time. Fisher tells us that the more efficiently we are able to
generate and test "something different," the more we will be able to
learn. And in business, as in nature, we have to learn in order to
survive. "Without learning, there is no survival" Deming would teach in
his workshops. "Survival is not compulsory." [5]

This means that business is faced with a paradox similar to one a
species faces in the corollary to Fisher's Fundamental Theorem. To
survive day-to-day, we have to "focus" -- adapt as closely as possible
to our present environment, suppressing characteristics and behavior
that are out of alignment with our current environment's needs.
Experimenting with fuel injectors takes precious resources from the job
of making better carburetors and cuts into this quarter's profits.
Teaching new medical students or studying new techniques makes patient
care more costly than at a facility which does not do these things.
Finding out how a business creates and uses information takes time from
the task of cranking out a database. Exploring a fundamentally new line
of research takes precious time from the several-paper-a-year grind and
risks that one won't get tenure, or that ones institution won't get
rated or ranked as highly and lose funding. These things seem out of
focus with today's needs.

But according to this view of Fisher, perfect adaptation to today's
day-to-day survival needs consumes the capital a society needs to adapt
to the future.  Deming taught that a successful society -- even one that
merely expects to survive for a long time -- has got to find a way to
support these costs and risks. Trying something new risks that one will
fail. Fisher's fundamental theorem teaches that a society that makes
failure an individual responsibility -- a pure negative in a person or
business's adaptation within the society under its current environment
-- cannot survive long, because such a society, as a whole, will be
unable to respond to inevitable change in the whole society's
environment. A society whose members do not sometimes fail will not
produce enough variation to adapt or succeed long-term.

Much of Deming's 14 points [6] are about how to create an environment in
which risk-taking is possible and safe. The key things he taught against
-- competition, fear, awarding on price alone, ranking, etc. -- all
teach against aspects of our traditional business and academic life that
prevent the kind of variation necessary for learning to occur. Many of
the techniques are about how to make learning more efficient -- safer
and more effective. In some ways they are all applications of Fisher's
Fundamental Theorem, ways of getting us to a point where a business or
society can have more of the societal equivalent of "genetic" variance
without going bankrupt.

As in an organism, organizational innovation begins with homeostasis --
stabilizing the internal environment so one can be sure outcomes are due
to deliberate change and not to "chance." This is what the Shewhart
charts [7] are all about. Experimental design adds in an ability to
control change and learn efficiently not found in nature. But I think
this was just a starting point.

What is the societal equivalent of the genetic material that appears in
Fisher's work? I think knowledge may play something like that role. If
one plugs it into Fisher's formulae the equations seem to make sense. I
can't find anywhere where Deming says it, so I can't attribute it to him
directly, but I think much of Deming starts making sense if one takes
the view that a society's store of knowledge has some of the same value,
and serves some of the same purposes and roles, as a population's store
of genetic material.

Knowledge is a cognitive society's real capital, just as genetic
material is in a population where cognitive learning is not possible. It
is the result of adaptive learning, in which knowledge is "invested."
(subjected to risk and variation) in ways that do or do not produce a
return. It is the way that the results of a population's successes and
failures are stored.

Like genetic material in Fisher's scheme, knowledge has to be
distributed among various individuals, but this does not mean that
knowledge is "located" purely in those individuals any more than genetic
material is. Individuals alone can no more produce disseminated
knowledge than they can produce offspring. Interaction is required. For
adaptive purposes, knowledge and genetic materials are waves that should
be thought of as properties of the whole society. So is "success" and
"failure."

Do not let those particles fool you into thinking that the wave does not
still exist. Much of it lies hidden and does not "come into being" until
a population (or society) hits an obstacle that serves the role a
detector does in physics. At that point -- which is when Fisher's
Fundamental Theorem matters -- it becomes all too real. We are all
merely carriers. Treating our capital as mere discrete particles that
can then be thought of as the "property" of individuals as private
individuals (as opposed to trustees) is to do immense harm to a
population -- or a society's -- ability to evolve.

I think this is why Deming spent so much time on education. Education is
the one area in a society's life where variation should be most
encouraged, where risk-taking has the most potential for reward and the
least potential for harm. It is where a society's real capital is
fostered. Fisher's Fundamental Theorem teaches that to force educators,
researchers, and students to adapt too much to their current environment
is to destroy a society's ability to adapt to a new environment.
Effective learning requires intellectual variation. A society unable or
unwilling to support trying things that fail is too focused to survive.
Intellectual monoculture is as big a threat to human survival as is
biological monoculture.

I'm making two claims here:
1. Fisher said that the role of genetic material in a population has
some of the characteristics and obeys some of the same wave/particle
dynamics laws as physics "particles" (Fisher made this association
explicitly,  rejecting what he called the “particulate theory of
genetics”).
2. Much of what Deming said about both knowledge and society makes more
sense if one takes the view that  knowledge in a society plays some of
the same adaptive role as genetic material in a natural population as
described by Fisher (If Deming said this directly anywhere, please let
me know.)

Jonathan Siegel

P.S. This post is deliberately fuzzy. "Knowledge", "capital," “fitness”
and other important terms are not clearly defined. This is deliberate: I
am intentionally putting these terms at risk. This paper is in the early
stages of ideas. When our current ideas are too precise, we have a
harder time generating new ones. One has to relax definitions a bit,
allowing associations that weren't obvious to suggest themselves, then
go through, define terms more rigorously, and find out whether the
suggested associations actually have value or not. The  "relaxed
definition" -- a kind of variation -- is an interesting study in itself.
Many famous discoveries first appeared in dreams. Thinking is a process
of allowing variation in ones head, then rigorously testing the value of
the variation generated -- same as any other kind of adaptive learning.
I suppose new associations between ideas, and the "relaxed" states that
permit them, are not unlike the genetic analogy in other ways as well.
Like all adaptive learning processes, conceptualization obeys Fisher's
Fundamental Theorem -- the more varied ones thinking, the more
successful one is likely to be (the "rigorous testing" part is also very
important and there is a balance between the two). The variation
described in this post has not been rigorously tested, and its value has
not been determined.

References:

[1] Fisher, Robert.A. The Genetical Theory of Natural Selection (1959)

[2] Weinberg, Gerald M. An Introduction To General Systems Thinking
(1975)

[3]. Down, John, “That Paper.” Deming Electronic Network Posting (1999)
http://deming.ces.clemson.edu/pub/den/archive/2000.09/msg00180.html

[4] Towns, William, “ Deming As Pragmatist.” W. Edwards Deming Institute
Conference Paper (1997)
http://deming.eng.clemson.edu/pub/den/deming_as_prag.htm

[5] Deming, W. Edwards 4-Day Quality Seminar, Indianapolis, IN (1992).
Jonathan M. Siegel Personal lecture notes.

[6] Deming, W. Edwards. Out of the Crisis (1986)

[7] Shewhart, Walter A.  Statistical Method from the Viewpoint of
Quality Control (reprinted 1986)