Issue 662807: Port tests to unittest (original) (raw)
Created on 2003-01-05 20:50 by doerwalter, last changed 2022-04-10 16:06 by admin. This issue is now closed.
Messages (51)
Author: Walter Dörwald (doerwalter) * 
Date: 2003-01-05 20:50
This patch ports the three tests test_pow.py, test_charmapcodec.py and test_userdict.py to unittest.
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-05 20:56
Logged In: YES user_id=89016
The patch is hard to read, so I'll upload all three test scripts.
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-07 16:37
Logged In: YES user_id=89016
test_b1.py has been ported too.
Author: Guido van Rossum (gvanrossum) *
Date: 2003-01-08 14:03
Logged In: YES user_id=6380
Two random suggestions:
a blank line before each method, even trivial ones, even the first one
use assertEqual, not assertEquals
BTW, I see you've picked up on the convention that unit test methods should not have doc strings. Good! (But they may have comments.)
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-08 19:05
Logged In: YES user_id=89016
test_b1 and test_b2 are combined into test_builtin now
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-10 16:53
Logged In: YES user_id=89016
test_builtin.py is now updated to test more error situations. This increases the coverage of bltinmodule.c from 75.13% to 92.20%, and it actually revealed one or two potential bugs: http://www.python.org/sf/665761 and http://www.python.org/sf/665835
I'm not 100% sure that test_intern() and test_execfile() do the right thing.
I'm not sure, whether the test script should check for undocumented implementation artefacts, like:
a = 1 self.assert_(min(a, 1L) is a)
but in this way at least we get notified if something is changed unintentionally.
Author: Raymond Hettinger (rhettinger) *
Date: 2003-01-10 17:03
Logged In: YES user_id=80475
Good to hear the news on increasing the coverage.
In general, don't do tests that hardwire implementation details. Test it if it is a documented variable, exposed through all, is a key constact (like the magic numbers in random.py), or a variable that a module user is likely to be relying upon. Otherwise, no -- it should be possible to improve an implementation without crashing the suite.
I'll try to review a few of these over the next few days.
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-10 17:17
Logged In: YES user_id=89016
In general, don't do tests that hardwire implementation details
So should we remove self.assertEquals(reduce(42, "1"), "1") self.assertEquals(reduce(42, "", "1"), "1") from test_filter?
BTW, you should look at test_builtin first, as the others are still simply ports to PyUnit.
Author: Walter Dörwald (doerwalter) *
Date: 2003-01-16 20:47
Logged In: YES user_id=89016
test_unicode is ported and enhanced (coverage goes from 80.81% to 85.05%)
Author: Raymond Hettinger (rhettinger) *
Date: 2003-01-19 14:46
Logged In: YES user_id=80475
All are approved except test_charmapcodec.py -- someone else should look at that one.
Be sure to follow GvR's advice and replace assertEquals with assertEqual.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-03 20:36
Logged In: YES user_id=89016
Here's a new one: test_sys.py tests Python/sysmodule.c. Coverage goes from 68% to 77%.
Author: Guido van Rossum (gvanrossum) *
Date: 2003-02-03 21:10
Logged In: YES user_id=6380
test_sys.py:
I agree that it's not worth testing the code paths that will invoke a custom displayhook or excepthook, but I regret it nevertheless. :-) maybe this deserves a comment?
sys.exit() should also be callable with a string
you could check that the value of the SystemExit exception has the right exit code
Have you checked this with Jython? I don't know if it implements all of these; in particular I doubt it has getrefcount().
I presume you've tested this on Windows?
Sorry, I can't help you with charmapcodec
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-03 22:44
Logged In: YES user_id=89016
OK, here's a new test_sys.py
test_sys.py:
- I agree that it's not worth testing the code paths that will invoke a custom displayhook or excepthook, but I regret it nevertheless. :-) maybe this deserves a comment?
Testing a custom displayhook is now done (via compile(..., "single")/exec). Testing a custom excepthook seems to be trickier. This could probably be done by calling the interpreter recursively via os.system() or os.popen(). I've added a comment for now that this isn't tested. Unfortunately this leaves a large block in Python/pythonrun.c uncovered.
- sys.exit() should also be callable with a string
OK, done.
- you could check that the value of the SystemExit exception has the right exit code
Done.
- Have you checked this with Jython? I don't know if it implements all of these; in particular I doubt it has getrefcount().
I haven't tested Jython yet, but I guess test_sys.py will have to many many exceptions for Jython. I'll try this tomorrow.
- I presume you've tested this on Windows?
Linux & Windows
Author: Guido van Rossum (gvanrossum) *
Date: 2003-02-03 22:56
Logged In: YES user_id=6380
I think you can check this in -- if it fails with Jython, Finn or Samuele will quickly patch it. :-)
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-03 23:13
Logged In: YES user_id=89016
OK, test_sys.py is checked in.
Author: Neal Norwitz (nnorwitz) *
Date: 2003-02-13 03:12
Logged In: YES user_id=33168
MAL, could you look at the test_charmapcodec.py? I think that's the only file outstanding from this patch. It's a pretty straightforward test.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-13 18:02
Logged In: YES user_id=89016
Here's another one: test_userlist has been ported to PyUnit and a few tests have been added to increase coverage.
Author: Guido van Rossum (gvanrossum) *
Date: 2003-02-13 18:08
Logged In: YES user_id=6380
Walter, feel free to check in test_userlist.py!
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-13 18:16
Logged In: YES user_id=89016
OK, checked in as test_userlist.py 1.7.
Assigned back to MAL for the review of test_charmapcodec.
Author: Marc-Andre Lemburg (lemburg) *
Date: 2003-02-14 08:52
Logged In: YES user_id=38388
test_charmapcodec looks OK. Just remove the DOS-lineends before checking it in.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-14 11:30
Logged In: YES user_id=89016
Checked in as: Lib/test/output/test_charmapcodec delete Lib/test/test_charmapcodec.py 1.6
Author: Neal Norwitz (nnorwitz) *
Date: 2003-02-14 20:11
Logged In: YES user_id=33168
Walter, can this patch be closed now?
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-16 09:33
Logged In: YES user_id=89016
I'm currently working on a PyUnit port of the string tests (i.e. str, unicode, UserString and the string module). Uploading the result to this patch would be easier, as it already has a establsihed audience: But I can open a new patch for that if you want.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-17 18:29
Logged In: YES user_id=89016
Here is the next bunch of ports: the string tests have been ported to PyUnit and made as reusable as possible. Tests are now shared between str, unicode, UserString and the string module. As a result of reusing a part of the unicode tests for str, the coverage in stringobject.c goes from 83% to 86%. Furthermore it should help keep the API consistent between str and unicode (Example: "%c" % 0xffffffff raises OverflowError, u"%c" % 0xffffffff raises ValueError)
Raymond can look look through the scripts and check that everything is OK?
Author: Raymond Hettinger (rhettinger) *
Date: 2003-02-21 03:39
Logged In: YES user_id=80475
test_string.py imports sets but does not use it.
the names of the mixin classes could possibly be made clearer so I won't have to search into the comments to find-out which mixins are appropriate for each class.
Overall, it looks like a nice factoring job and ought to go a long ways toward keeping these guys in sync in the future.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-21 13:05
Logged In: YES user_id=89016
Checked in as: Lib/test/string_tests.py 1.27 Lib/test/test_str.py 1.1 Lib/test/test_string.py 1.24 Lib/test/test_unicode.py 1.79 Lib/test/test_userstring.py 1.10 Lib/test/output/test_string delete
I've removed the sets import and renamed the mixin tests to contain the relevant class/module names (e.g. MixinStrStringUserStringTest)
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-25 17:53
Logged In: YES user_id=89016
OK, here are the next few ports: test_ucn and test_unicodedata. I'm not actually sure, whether changing test_unicodedata (which uses the comparison of generated output with expected output) is a good thing, as now updates to the database require manual changes. I've added a few error checks which increase coverage in unicodedata.c from 87% to 95%.
Marc-André can you check if this is OK?
Author: Marc-Andre Lemburg (lemburg) *
Date: 2003-02-26 13:42
Logged In: YES user_id=38388
test_ucn and test_unicodedata look OK.
Author: Walter Dörwald (doerwalter) *
Date: 2003-02-26 15:08
Logged In: YES user_id=89016
Checked in as: Lib/test/test_ucn.py 1.12 Lib/test/test_unicodedata.py 1.7 Lib/test/output/test_ucn delete Lib/test/output/test_unicodedata delete
Author: Walter Dörwald (doerwalter) *
Date: 2003-03-11 19:15
Logged In: YES user_id=89016
Here's the next one: test___all__.py ported to PyUnit and updated.
A better solution might be to replace builtin.import in regrtest.py and test for the all attribute there. Additionally this might allow us to check which modules are imported by regrtest.py and which are not and require additional tests.
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-10 08:46
Logged In: YES user_id=89016
Here's the next one: test_pwd
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-10 09:10
Logged In: YES user_id=89016
Here's an updated test_grp that executes tests equivalent to those in test_pwd
Author: Neal Norwitz (nnorwitz) *
Date: 2003-04-12 01:54
Logged In: YES user_id=33168
question why do you use 3 lines in test_main, rather than the old way? -- test_support.run_unittest(Test)
test___all__ is fine test_grp is fine test_pwd fails for me on Redhat 7.2. It also needs an if __name == 'main': test_main()
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-13 09:07
Logged In: YES user_id=89016
question why do you use 3 lines in test_main, rather than the old way? -- test_support.run_unittest(Test)
The example in Lib/test/README seems to suggest that it should be done this way.
I'll look into the test_pwd problem tomorrow.
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-14 12:31
Logged In: YES user_id=89016
Neal, are you sure the error you get isn't just from the output comparison? Can you post a traceback? test_pwd works for me with Redhat 7.3.
Author: Neal Norwitz (nnorwitz) *
Date: 2003-04-14 23:09
Logged In: YES user_id=33168
Sorry, you are correct. I removed the output/test_pwd and the test passes. This is on redhat 9 now.
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-15 11:30
Logged In: YES user_id=89016
Checked in as: Lib/test/output/test_pwd delete Lib/test/test_pwd.py 1.13 Lib/test/test_grp.py 1.11 Lib/test/test___all__.py 1.29 (with if __name == 'main': test_main() added to test_pwd)
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-23 20:33
Logged In: YES user_id=89016
Here's the next one: test_bool.py ported to PyUnit with a few additional tests. Code coverage for boolobject.c is at 100%.
Would it make sense to move assertIs() and assertIsNot() to unittest.py?
Author: Guido van Rossum (gvanrossum) *
Date: 2003-04-23 21:00
Logged In: YES user_id=6380
I suggest not to mess with unittest.py; it is still externally maintained as PyUnit. Maybe ask the PyUnit developer.
No time to review the test, sorry; if it works, I suggest to just check it in. ;-)
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-25 11:40
Logged In: YES user_id=89016
Checked in as: Lib/test/test_bool.py 1.9
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-28 18:42
Logged In: YES user_id=89016
Here's the next one: test_richcmp.py with a few additional tests to increase code coverage in object.c and listobject.c.
BTW, should we change all tests to use the new test_support.run_classtests()?
Author: Raymond Hettinger (rhettinger) *
Date: 2003-04-28 19:10
Logged In: YES user_id=80475
Yes. test_support.run_classtests() ought to be used everywhere it is applicable. There are a couple of places like test_sets.py where the "suite" variable is used again for other purposed (testing for memory leaks). In those cases, it ought to be left alone until I can write a function that encapsulates the essense of the combined test.
test_richcmp.py looks good. Please add comment lines describing the point of classes and tests whose names are not self explanatory (i.e. misbehavin, weird, etc).
BTW, why is the int to int comparison labeled as useless?
Author: Walter Dörwald (doerwalter) *
Date: 2003-04-29 21:39
Logged In: YES user_id=89016
Checked in as: Lib/test/test_richcmp.py 1.8 Lib/test/output/test_richcmp delete which a few additional tests and comments.
I'll probably work on using run_classtests() tomorrow, but wouldn't it be simpler to add the multiple arguments to the already existing function run_unittest() instead of introducing the new run_classtests()?
BTW, why is the int to int comparison labeled as useless? Because it would check whether comparing two ints gives the same result as comparing the same two ints. ;)
Author: Walter Dörwald (doerwalter) *
Date: 2003-05-01 16:54
Logged In: YES user_id=89016
I've changed test_support.run_unittest() to work like your run_classtests() and changed all test scripts to use this. (see diff2.txt)
I compared the output of ./python Lib/test/regrtest.py -v 2>&1 | grep -v "Ran.*test.*in" and apart from the fact that test_csv seems to be more verbose now, there seem to be no regression bugs. Raymond, can you check, if this is OK?
BTW, there's another check for memory leaks in test_datetime.py. Perhaps it would make sense to combine both and use them globally in regrtest.py (or test_support.py)?
Author: Raymond Hettinger (rhettinger) *
Date: 2003-05-01 17:17
Logged In: YES user_id=80475
My preference was to keep run_classtests() as a separate function. The run_unittest() function was unique in that it called run_suite() with two arguments instead of one. That accounts for the additional verbosity you are seeing. While I prefer the separate run_classtests(), it's up to you to do whichever seems best.
Otherwise, all looks okay.
Author: Walter Dörwald (doerwalter) *
Date: 2003-05-01 17:58
Logged In: YES user_id=89016
OK, I've changed the new run_unittest() to call run_suite() with a second argument if exactly one testcase is passed. I hope that's OK.
Checked in as: test_StringIO.py 1.16 test___all__.py 1.30 test_base64.py 1.6 test_bisect.py 1.9 test_bool.py 1.12 test_builtin.py 1.19 test_bz2.py 1.14 test_calendar.py 1.5 test_call.py 1.4 test_cfgparser.py 1.20 test_charmapcodec.py 1.7 test_codeccallbacks.py 1.12 test_codecs.py 1.9 test_copy.py 1.8 test_cpickle.py 1.15 test_csv.py 1.6 test_dummy_thread.py 1.4 test_enumerate.py 1.4 test_filecmp.py 1.2 test_getargs2.py 1.5 test_grp.py 1.16 test_hexoct.py 1.4 test_hmac.py 1.7 test_isinstance.py 1.7 test_itertools.py 1.6 test_optparse.py 1.2 test_os.py 1.18 test_parser.py 1.17 test_pep277.py 1.6 test_pickle.py 1.18 test_posix.py 1.6 test_pow.py 1.18 test_profilehooks.py 1.10 test_pwd.py 1.17 test_re.py 1.42 test_richcmp.py 1.9 test_sets.py 1.24 test_shelve.py 1.5 test_shutil.py 1.2 test_socket.py 1.64 test_str.py 1.3 test_string.py 1.25 test_strptime.py 1.13 test_support.py 1.52 test_sys.py 1.7 test_tarfile.py 1.8 test_tempfile.py 1.14 test_textwrap.py 1.19 test_timeout.py 1.14 test_trace.py 1.9 test_ucn.py 1.14 test_unicode.py 1.83 test_unicodedata.py 1.9 test_univnewlines.py 1.5 test_urllib.py 1.13 test_urllibnet.py 1.2 test_userdict.py 1.15 test_userlist.py 1.9 test_userstring.py 1.11 test_weakref.py 1.23 test_xpickle.py 1.4 test_zipimport.py 1.8 test_zlib.py 1.23
Author: Walter Dörwald (doerwalter) *
Date: 2003-05-05 20:48
Logged In: YES user_id=89016
And now for something completely different: test_uu.py ported to PyUnit. Raymond can you check that this is OK?
Author: Raymond Hettinger (rhettinger) *
Date: 2003-05-05 22:56
Logged In: YES user_id=80475
Yes. It's fine.
Hey, I think it's time to start a new patch for future changes. This one is getting huge.
Author: Walter Dörwald (doerwalter) *
Date: 2003-05-06 09:01
Logged In: YES user_id=89016
Checked in as Lib/test/test_uu.py 1.6.
OK, I'll open a new patch for the next test script once it's done.
Author: Guido van Rossum (gvanrossum) *
Date: 2003-05-06 12:08
Logged In: YES user_id=6380
So maybe close this one?
Author: Walter Dörwald (doerwalter) *
Date: 2003-05-06 13:10
Logged In: YES user_id=89016
Closed the patch