[Python-Dev] Stability Metrics [was Stability and Change] (original) (raw)

Guido van Rossum guido@python.org
Thu, 11 Apr 2002 10:10:51 -0400

[AMK]

> I've done this ever since whatsnew21. My numbers, surely > underestimates, are: > > 2.1: 117 patches, 136 bugs > 2.2: 527 patches, 683 bugs > 2.2.1: 139 patches, 143 bugs

[RH]

Whoa there! The conversation is now trending toward bugs as a metric of merit rather than demerit. Of course, fixing bugs is good, but not having them in the first place is better.

Before choosing a publicity metric, consider the significance of the metric if it is very large or very small. In the above example, 0 bugs would likely indicate that no maintenance is taking place. Having 10,000 bugs would indicate that the release process was a disaster. Whatever metric is choosen (and I DO think having a stability metric is good), it should have a clear interpretation that a higher number is good and a lower number is bad or vice-versa.

I think it's nice to keep track of this for ourselves (despite the interpretational problems). But I'm strengthening my position about the use of statistics in communicating to the users: I think that would be bad. Statistics can easily be misused, misinterpreted, and countered.

--Guido van Rossum (home page: http://www.python.org/~guido/)