RE: Web-based, enterprise-wide, SPC-based performance improvement software

["Allen Woods" <allenwoods@jit-software.com>]

Alan,

At the moment, I am working with a bunch of guys who have access to
terabytes of data.  One of the principle activities they face is related
to organizing the data so that it can be used by a variety of reporting
tools that fit into a ....  reporting architecture if you like.  There
is no one reporting tool that can do what is required, let me give you
two reasons why all singing and dancing reporting tools are not as
capable as their designers would like us to think...

In a lot of situations, once people get to the limitations of standard
SQL based reporting, the first technique they bump into is On line
Analytical Processing or OLAP.  Simplistically, OLAP tools generate a
series of related queries and join the results up into something like a
data warehouse sometimes referred to as a data cube or universe.  The
tools themselves have a significant footprint in terms of a data storage
requirement during the Cube/warehouse generation phase.  The footprint
can, in extremis be larger than the original data set.  You can reduce
this with careful adjustment of the design of the original data so that
it is OLAP friendly.  Nevertheless, when you start to use OLAP tools,
you have to plan for cube generation.

You also have to be careful in your query design to differentiate
between data that is statistically significant (for calculation
purposes), and the other stuff.....  Which in terms of volume means that
your interesting stuff is buried deep in the rest and must be filtered

Another aspect of creating query tools of the type you describe is how
you connect to the data you want to ferret around in.  To do that, you
need to understand the meta data (data about data) that describes how
your database is ordered.  In a WINDOWS world, an apparently simple way
to achieve connection is through the use of Open Database Connectivity
for which there are drivers, provided (usually) by the database design
company, that meet ODBC connectivity criteria.  Similar connectivity
standards apply for other operating systems.  If however, there is no
connection mechanism such as ODBC then basically you either know and
understand the way the database designer built his database AND the way
those design principles were applied to the data you are interested or
you are stuffed.

Over the last five years I have been tasked with getting data out of a
variety of what should have been legacy systems with a view to
transferring them into something more up to date.  One avenue to follow
is to use reporting tools to do the delving.  After all, a report is
simply formatted data extracted in an expected form, from a database.  I
haven't found one that can operate as you describe and I've gone though
an awful lot of reporting tools...

Of course aside from the really big problems, there are of course the
little odd ones like...  Different forms of storing date related data
and so on that all by themselves make life difficult for data query
engines.

Your tool might be clever, but to say it can be operated as you describe
is nice however, I'll believe it when I see it.

Allen

Allen Woods
JIT Software Limited
01980 843113
Skype ID apw808
http://www.jit-software.com