Re: Chaos and topographic diagrams

["Jonathan Siegel" <jmsiegel.info.research@worldnet.att.net>]

A few issues:

A. Chaos and Manufacturing Control

There are number of books by researchers interested in applying nonlinear
dynamics to problems of control. Professor Francis Moon surveys the field in
his compilation book Dynamics and Chaos in Manufacturing Processes (1997).
Professor Henry Abarbanel devotes considerable attention to control in his
textbook Analysis of Observed Chaotic Data (1996), which includes a detailed
discussion of methods of obtaining a phase space.

Much of this work involves taking observations with lasers that are (1)
frequent (often in milliseconds), (2) precise in both the measurement and
chronometry (often by using lasers etc. and getting measurements accurate to
within optical wavelengths); (3) non-interfering with the system measured
(low-intensity lasers have little effect on machinery in the lab); (4)
calculation- and computer-intense; (5) practical applications would likely
need to be real-time (large numbers of calculations occurring in small
fractions of a second). These requirements are not always feasible and would
dictate an automated approach.

Professor Moon found that, taking tens of thousands of observations over the
space of a few seconds, it was possible in the laboratory to create a model
of the dynamics of a milling machine with sufficient utility to predict the
time series a second or so (a few hundred or thousand observation-ticks)
into the future with reasonable accuracy. Dynamic models of chaotic systems
are extremely data-intensive. But they are not impossible. Lasers,
computers, and other control electronics are quite cheap, and computing
power has been improving exponentially for years. The results appear to be
good enough to promise computerized machine controllers that permit better
performance than statistical control. Professor Abarbanel has warned,
however, not to expect an immediate rush from the lab to the assembly line.
He has characterized efforts to reduce developing a model of chaotic
dynamics to a highly reliable mechanical algorithm as so far unsuccessful,
and said that no fundamental mathemetical modeling theory has been developed
yet  from which a "best" method might be derived. This makes chaotic
modeling an art, in a trial-and-error research stage, requiring modelers
with skill and experience to apply handcrafted models. It is still a
research enterprise, but one well underway.

Statistical control is far more robust and reliable, working under a wider
range of conditions with much simpler and cheaper tools. It does not require
measurements to nearly be as frequent, precise, or non-interfering, and the
calculations are not nearly as intense. It uses and processes less
information. And it tells us less about the future than the approach
Professor Moon is working on. But depending on our loss-function, for many
purposes it may tell us all we need to know.

B. Dynamics and Deming's Views on Statistics

Much of statistics of Neyman and Pearson variety is based on the notion that
there are only two kinds of processes in the universe: deterministic and
stochastic. A lot of the machinary of statistics -- Linear models,
good-ness-of-fit tests, tests of association, and much of hypothesis
testing, for example -- is designed to separate a collection of observables
into these two processes. These are some of the statistical methods that
Deming most criticized.

Now that it is clear that ther are other kinds of processes in the universe
and there is a vocabulary to describe them in general use, we have a clearer
way to express these objections.

Observables from dynamic systems are always only approximately random. There
is always some underlying structure. Similarly any two observables from a
dynamic system are always correlated, however slightly. After all, both can
at least in principle can be phased in a way that preserves the structure of
the complete underlying system. And nothing from a dynamic system with
real-world behavior behaves precisely according to a model -- there is
always some interaction perturbing it.

These three principles underly much of Deming's critique. Some examples:

1. Deming talked a lot about "analytic" statistics, which he defined as
methods intended to predict the future. He called the methods Neyman and
Pearson had devised "enumerative" statictics. I think the issue is
illuminated by treating the fundamental metaphor of Neyman and Pearson-style
statistics as the act of taking a sample from a (otherwise unchanging)  bowl
and inferring from the sample to the "population." What Deming realized is
that to predict the future, the fundamental metaphor needs to be a dynamic
system rather than a "population" or a bowl. In analytic statistics, one
makes predictive models of dynamic systems based on observing the structure
of how data from them varies over time. (This isn't all there is, but I
think the metaphor shift is a very important piece).

Data from dynamic systems always has some structure, even if it
superficially appears random. Deming argued that, for example, data that
appears random when displayed as a histogram may show a great of structure
when analyzed as a time-series. Research into the phase structure of chaotic
systems illuminates why.

2. Tests of association ask whether or not two observables are independent.
Deming argued the question is meaningless: everything is associated, however
slightly. If we collect enough data with fine enough instruments we'll
eventually detect it.  Similarly, goodness-of-fit tests test whether data
perfectly follows a model. Knowing that it doesn't, we understand this
question is meaningless: all we have to do is collect enough data and we'll
discover it doesn't.

3. Linear model inference is also based on taking data as if "from a bowl."
We also know now that most real-world systems are not explained by dividing
the world into simple-model and random components, as linear models theory
assumes.

Jonathan Siegel