Knowledge and Variation: Some thoughts on Deming and Fisher

[Jonathan Siegel <jmsiegel.info.research@worldnet.att.net>]

"The rate of increase in fitness of a population at any time is proportional to its genetic variance in fitness at that time"

Corollary:

"The more adapted a population is to its current environment, the less able it will be to adapt to a new environment"

-- Sir Robert Fisher, "Fisher's Fundamental Theorem of Natural Selection", aka "Fisher's Fundamental Theorem"

The quality movement has become associated with the idea that variation is an inherently bad thing and the aim of a person interested in improving quality should be to reduce variation wherever it appears.  One of the main recent streams of Deming criticism -- the stream that showed up in John Dowd's "That Paper" thread a couple of months ago -- basically associates Deming and the quality movement with conformity and argues that in some areas of life, education in particular, too much conformity is a bad thing.

I don't believe this is an accurate criticism of Deming. I believe Deming would be inclined to agree that reducing variation as such cannot be the main aim of a business or a society that wishes to survive long-term. Sometimes variation is a good thing, and it is the duty of a manager -- and a society -- to increase it. As Bill Towns explained in his 1997 "Deming as Pragmatist" paper, Deming borrowed a great many of Fisher's ideas.  I think Fisher's Fundamental Theorem plays quite a role in Deming's discussion of
learning and variation.

-- "If you keep doing what you did before, you will keep getting the same results." -- Unknown
-- But only as long as the world around you stays the same, and it won't forever. -- Unknown

In order to learn, or to improve, we have to do something different, then see if it works. The PDSA cycle is a form of adaptive learning that represents an accelerated, systematic form of natural selection. It is Fisher's evolutionary engine set up in the shop and run more efficiently. I believe the dynamics underlying the PDSA cycle are not fundamentally different from those of natural selection.  And I think Deming looked to Fisher to understand how it works.  In the PDSA cycle variation represents the fuel that
drives the engine, just as it does in nature. And if Fisher is right, our success will be related to how much variation we are able to generate in a given amount of time. Fisher tells us that the more efficiently we are able to generate and test "something different," the more we will be able to learn. And in business, as in nature, we have to learn in order to survive. "Without learning, there is no survival" Deming would teach in his workshops. "Survival is not compulsory."

This means that business is faced with a paradox similar to one a species faces in the corollary to Fisher's Fundamental Theorem. To survive day-to-day, we have to "focus" -- adapt as closely as possible to our present environment, suppressing characteristics and behavior that are out of alignment with our current environment's needs. Experimenting with fuel injectors takes precious resources from the job of making better carburetors and cuts into this quarter's profits. Teaching new medical students or studying new
techniques makes patient care more costly than at a facility which does not do these things. Exploring a fundamentally new line of research takes precious time from the several-paper-a-year grind and risks that one won't get tenure, or that ones institution won't get rated or ranked as highly and lose funding. These things seem out of focus with today's needs. But according to Fisher, perfect adaptation to today's day-to-day survival needs consumes the capital a society needs to adapt to the future.  Deming taught that
a successful society -- even one that merely expects to survive for a long time -- has got to find a way to support these costs and risks. Trying something new risks that one will fail. Fisher's fundamental theorem teaches that a society that makes failure an individual responsibility -- a pure negative in a person or business's adaptation within the society under its current environment -- cannot survive long, because such a society, as a whole, will be unable to respond to inevitable change in the whole society's
environment.

Much of Deming's 14 points are about how to create an environment in which risk-taking is possible. The key things he taught against -- competition, fear, awarding on price alone, ranking, etc. -- all teach against aspects of our traditional and academic life that prevent the kind of variation necessary for learning to occur. Many of the techniques are about how to make learning more efficient -- safer and more effective. In some ways they are all applications of Fisher's Fundamental Theorem, ways of getting us to a
point where a business or society can have more of the societal equivalent of "genetic" variance without going bankrupt.

As in an organism, organizational innovation begins with homeostasis -- stabilizing the internal environment so one can be sure outcomes are due to deliberate change and not to "chance." This is what the Shewhart charts and the experimental statistical techniques are all about. But I think this was just a starting point.

What is the societal equivalent of the genetic material that appears in Fisher's work? I think knowledge may play something like that role. If one plugs it into Fisher's formulae the equations seem to make sense. I can't find anywhere where Deming says it, so I can't attribute it to him directly, but I think much of Deming starts making sense if one takes the view that a society's store of knowledge has some of the same value, and serves some of the same purposes and roles, as a population's store of genetic material.
Knowledge is a cognitive society's real capital, just as genetic material is in a population where cognitive learning is not possible. It is the result of adaptive learning, in which knowledge is "invested." (subjected to risk and variation) in ways that do or do not produce a return. It is the way that the results of a population's learning is stored. Like genetic material in Fisher's scheme, knowledge has to be distributed among various individuals, but this does not mean that knowledge is "located" purely in those
individuals any more than genetic material is. Individuals alone can no more produce disseminated knowledge than they can produce offspring. For adaptive purposes, knowledge and genetic materials are waves that should be thought of as properties of the whole society. Don't let those particles fool you into thinking that the wave doesn't still exist. Much of it lies hidden and does not "come into being" until a population (or society) hits an obstacle that serves the role a detector does in physics. At that point --
which is when Fisher's Fundamental Theorem matters -- it becomes all too real. We are all merely carriers. Treating our capital as mere discrete particles that can then be thought of as the "property" of individuals as private individuals (as opposed to trustees)  is to do immense harm to a population -- or a society's -- ability to evolve.

I think this is why Deming spent so much time on education. Education is the one area in a society's life where variation should be most encouraged, where risk-taking has the most potential for reward and the the least potential for harm. It is where a society's real capital is fostered. Fisher's Fundamental Theorem teaches that To force educators, researchers, and students to adapt too much to their current environment is to destroy a society's ability to adapt to a new environment. Effective learning requires
intellectual variation. A society unable or unwilling to support trying things that fail is too focused to survive. Intellectual monoculture is as big a threat to human survival as is biological monoculture.

I'm making two claims here:
1. Fisher said that the role of genetic material in a population has some of the characteristics and obeys some of the same wave/particle dynamics laws as physics "particles" (Fisher made this association explicitly).
2. Much of what Deming said about both knowledge and society makes more sense if one takes the view that  knowledge in a society plays some of the same adaptive role as genetic material in a natural population as described by Fisher (If Deming said this directly anywhere, please let me know.)

Jonathan Siegel

P.S. This post is deliberately fuzzy. "Knowledge", "Capital," and other important terms are not clearly defined. This is deliberate: I am intentionally putting these terms at risk. This paper is in the early stages of ideas. When our current ideas are too precise, we have a harder time generating new ones. One has to relax definitions a bit, allowing associations that weren't obvious to suggest themselves, then go through, define terms more rigorously, and find out whether the suggested associations actually have value
or not. The  "relaxed definition" -- a kind of variation -- is an interesting study in itself. Many famous discoveries first appeared in dreams. Thinking is a process of allowing variation in ones head, then rigorously testing the value of the variation generated -- same as any other kind of adaptive learning. I suppose new associations between ideas, and the "relaxed" states that permit them, are not unlike the genetic analogy in other ways as well. Like all adaptive learning processes, conceptualization obeys Fisher's
Fundamental Theorem -- the more varied ones thinking, the more successful one is likely to be (the "rigorous testing" part is also very important and there is a balance between the two). The variation described in this post has not been rigorously tested, and its value has not been determined.