Peter Seibel  <peter@gigamonkeys.com> wrote:
+---------------
| Unfortunately it's not clear how to practice evidence-based IT--it
| seems that different software projects are more different than
| different patients a doctor might see.
+---------------

I don't think that's so much the problem as that the industry as a
whole has simply stopped doing any computer *science*. That is, you
can't have "evidence-based IT" if you never collect any evidence!!
As Tom DeMarco began one of his best (IMHO) books[1], "You can't
control what you can't measure." [...or what you don't bother to!]

If you look back at all the papers -- and I *don't* mean from just
academia, but from industrial/commercial/governmental organizations
as well -- that were published in the 50's and into the 80's, there
were *plenty* of serious attempts to quantify the difficulty of the
task of software specification, design, development, and maintenance.
It didn't result in any "magic bullets", but then the problem was and
is *hard*!!

But the Rise of Worse is Better seems to have completely put an end
to any wide-spread attempt to closely examine the *act* of software
development in any meaningful, "scientific" way [by which I mean the
collection of predictive metrics, doing controlled experiments, even
work on cognitive models of programming, etc.].

Pity.


-Rob

[1] Tom DeMarco, "Controlling Software Projects: Management,
    Measurement, and Estimatation", Prentice Hall (1982) ISBN
    0917072324 [trade paperback]. Reprinted 1986, ISBN 0131717111.
    Particularly interesting is his discussion of the social/political
    difficulties in getting good "metrics", and his proposal for
    an independent metrics team that is rewarded by the inverse
    of the integral of the *absolute value* of the difference
    between their estimates at any moment during the project and
    the final overall actual result. [They are free to change their
    estimates at any time, but the previous estimates still apply
    to the interval during which they were in force.]

-----
Rob Warnock			<rpw3@rpw3.org>
627 26th Avenue			<URL:http://rpw3.org/>
San Mateo, CA 94403		(650)572-2607