[Python-Dev] Stability Metrics [was Stability and Change] (original) (raw)

Raymond Hettinger python@rcn.com
Thu, 11 Apr 2002 00:46:43 -0400

From: akuchlin@mems-exchange.org

Oh, sure; it's straightforward to write a script to pull log messages from a given branch, and if people religiously put 'bug #NNN' in their log messages, count up the fixed bugs. Handling bugs fixed per time would require more script hacking, but nothing terrifying.

I've done this ever since whatsnew21. My numbers, surely underestimates, are: 2.1: 117 patches, 136 bugs 2.2: 527 patches, 683 bugs 2.2.1: 139 patches, 143 bugs

Whoa there! The conversation is now trending toward bugs as a metric of merit rather than demerit. Of course, fixing bugs is good, but not having them in the first place is better.

Before choosing a publicity metric, consider the significance of the metric if it is very large or very small. In the above example, 0 bugs would likely indicate that no maintenance is taking place. Having 10,000 bugs would indicate that the release process was a disaster.

Whatever metric is choosen (and I DO think having a stability metric is good), it should have a clear interpretation that a higher number is good and a lower number is bad or vice-versa.

it-takes-an-accountant-to-notice-these-things-ly yours,

Raymond Hettinger, CPA