Re: FW: What is quality?

["Anton O. Tolman, Ph.D." <ANTON@wsh.state.wy.us>]

Wayne Mack wrote:

> In my experience, peer review is ineffective for determining
> (or improving) software quality.  In essence, peer review is
> merely another form of final inspection.  I don't have the
> references in front of me, but _Out of the Crisis_ covers
> final inspection and gives several examples covering its
> shortcomings.

I would agree.  Final inspection is usually of little worth and peer review
is in about the same ballpark.

> My test for quality in software is the question, "Is the
> customer's life better with it than without it?"  Peer
> review leaves the customer out of the loop (refer to the
> diagram in OOTC showing the customer as part of the
> system).  And rather than resolving customer complaints
> after the fact (a poor version of final inspection), customer
> needs should serve as initial input into the system.  (To
> revive a previous thread, Kano describes some excellent
> methodologies for capturing customer requirements.)

I do not really know much about software development at all; I agree that
customer needs should be the initial input into the system and also agree
that peer review usually leaves the customer out.  Peer review, as far as I
understand it, only serves to try and evaluate the level or quality of the
service provided in technical areas that are invisible to the consumer.  One
might argue that "invisible" aspects of service inevitably result in some
outcome related to the customer, and that perhaps focusing attention on those
outcomes is a more useful strategy for evaluating service quality?

> Peer review is merely another form of final inspection
> and is not an effective way of improving quality.

I agree.  I was focusing initially on Peter Drucker's commentary that one of
the problems we face right now in service delivery is that we do not have a
consensus or technology for evaluating all aspects of service quality.
I can be very pleased with my doctor's easy manner and the time he takes to
listen to my concerns.  Maybe he even charges less to boot, and I am very
satisfied.  But then I die suddenly of a heart attack because he was really
incompetent and I never knew it.  Perhaps my death is an "outcome" that
indicates that the level of quality was poor, but my own involvement in the
final inspection is *rather* final!

There is a movement currently going on in healthcare focusing on "outcomes
measurement" and perhaps in a sense, that is part of the goal of trying to
evaluate the quality of service delivery in this field.  I do not know if
there are equivalent movements going on in the legal field or other areas.
However, the major problems I have with using outcomes as an indicator of
service quality are as follows:

1)  an outcome is a delivery -- the "product" is already in the hands of the
customer, therefore outcomes measurement is a variant of inspection
2)  outcomes results do not necessarily inform you of upstream process issues
that are at fault in poor outcomes nor good outcomes; therefore, knowing an
outcome does not necessarily assist in improvement and tends to provoke a
traditional approach, e.g. "our outcomes are lousy, so fire the staff!"
because it is NOT process oriented

At the same time, there is some utility to outcomes which is inherent, for
example in treatment research.  Let's say we are reviewing research outcomes
for using suicidal virus DNA to identify and kill cancerous brain tumor
cells.  We compare this new form of genetic treatment with existing
treatments like surgery and radiation.  What do we measure?  Outcomes like
mortality rate and survivability.  Do these outcomes tell us something about
the "quality" of the intervention?  Most likely they do.  But the tricky part
is not the intervention per se, but the competence of the practitioner using
the intervention.

It seems clear to me that service quality is much more multifaceted than
product quality and needs to involve evaluation of the customer's perceptions
and reactions as to *how* the service was delivered, some form of peer review
to ascertain if the person providing the service is competent at a basic
level (really, peer review is a threshold issue of competency rather than
quality probably), and some form of outcomes evaluation.

Does anyone have any better way to measure and assess service quality?

Anton Tolman, PhD, CPHQ, Psychological Services Manager &
Quality Management Coordinator, Wyoming State Hospital
P.O. Box 177, Evanston, WY  82931-0177
Anton@wsh.state.wy.us    (307) 789-3464
===========================================================================