Fwd: Rule of Thumb?

[Kromkowski@aol.com]

I'm posting this back-channel to the DEN.
	[Moderator's Note:  Posted with Anton's permission]

My comments are as follows:

I have no problem with the use of professional judgment where there is risk 
of life or serious injury.  (No need for the use of any statistics, just say 
I'm a professional and this is what my judgment is.)  But otherwise, I see 
what you have described as the same kind of rationale for tampering as well 
as labeling and self-fulfilling prophecies in industry, government and 
education.  By what method does one tell the difference between professional 
management judgment and management lack of understanding of variation?

JDKromkowski

>From: "Anton O. Tolman, Ph.D." <ANTON@wsh.state.wy.us>
>Organization:  Wyoming State Hospital
>To: Kromkowski@aol.com
>Date:          Thu, 29 Jul 1999 08:28:18 MST
>Subject:       Re: Rule of Thumb?

JD, I'm replying back channel because there were no other replies to my
message and I'm assuming most are not interested, but you can post this to
the DEN if you'd like.

Regarding my statement about the use of a .05 alpha level.... you said:

> "Based on complex statistical theory!"  No, its not.  Its .05 instead of .01,
> because there might not be enough assessments/research to fill up the
> journals, because its easier, and because not finding a "special cause" isn't
> very sexy!

Actually that is not the case.  I meant that the "complex statistical theory"
had to do with the standard error of the mean and sample data, etc.  The .05
alpha level is a part of that, of course, but the decision of the level is
mine, is traditional, is part of our training.  At the same time, I usually
do select it because in doing assessment work, I make an assumption that if
the test score occurs less than 5% of the time in the normative population,
that it bears further investigation or conclusions.  That is why I mentioned
that at times I vary the alpha rate; I drop it to .01 if I want to be more
sure and the nature of the evaluation calls for more certain conclusions, and
I back it off to .10 if the evaluation is exploratory or may call for further
assessment depending upon initial results.  I recognize that in altering
these alpha rates, I alter my conclusions and my recommendations, but that is
part of being a good professional!

> In one of my very first posts to the DEN, a number of years ago now, I
> loosely used the phrase "rule of thumb", sort of like it has been used here.
> Jean Marie Gogue proceeded to blast me out of the water.  I am very thankful,
> he did.  The heck with "rules of thumb", let's aspire to be more scientific
> than the scientists!

I see no problem in trying to be more scientific than the scientists.  As you
know, most scientific literature, especially in social sciences, is filled
with hypothesis testing as the main driver of publications.  There are very
few, if any control charts, although I have published some articles with run
charts, and very little understanding of SoPK.  We can do better than that!

Anton Tolman, PhD, CPHQ, Psychological Services Manager &
=========================================================================