Issue 14769: Add test to automatically detect missing format units in skipitem() (original) (raw)

I recently fixed an old bug: some time ago someone added support for a new "format unit", 'Z', to PyArg_Parse(), but neglected to add matching support for it to skipitem(). Benjamin asked for a regression test. Initially I said, okay, how the hell do I test that? A quick grep showed that there was already a test like that, for the format unit 'C'. I figured adding another one-off test was dumb, and there had to be a better way.

Attached is a patch against trunk that adds a new function to _testcapimodule.c: test_skipitem_parity(). The comment at the top describes says it best:

• This function brute-force tests all** ASCII characters (1 to 127
• inclusive) as format units, checking to see that
• PyArg_ParseTupleAndKeywords() return consistent errors both when
• the unit is attempted to be used and when it is skipped. If the
• format unit doesn't exist, we'll get one of two specific error
• messages (one for used, one for skipped); if it does exist we
• won't get that error--we'll get either no error or some other
• error. If we get the "does not exist" error for one test and
• not for the other, there's a mismatch, and the test fails.
• ** Okay, it actually skips some ASCII characters. Some characters

•  have special funny semantics, and it would be difficult to

•  accomodate them here.

I also removed the old test just for 'C', as this test subsumes that one.

Right now, the test runs to completion without complaint. To test that it's really working, comment out an existing case statement in skipitem(). Or, add a new case statement, for a character that isn't otherwise supported (yet), to the top paragraph of case statements. Doing either of these will cause a failure.

... you happy now, Benjamin?

I guess the answer was "no".

Attached is a new diff as you suggest. It isn't really any shorter--in fact it's about 20% longer. Though I freely admit it is more written-in-Python-y.

The test currently passes. If you add support for a new format unit (like add "case '8':" above "case 'H':" in convertsimple()) without adding the corresponding case to simpleitem(), it fails. If you perform the same operation on simpleitem() but not to convertsimple(), it again fails. So I assert the test is working 100% correctly.