Issue 22680: Blacklist FunctionTestCase from test discovery (original) (raw)

Created on 2014-10-20 17:14 by pitrou, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
issue22680.patch Evan Hubinger,2015-07-27 21:38 Patch to fix issue 22680. review
Messages (17)
msg229731 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-20 17:14
I just got the following traceback when trying discover without 3.5. It runs fine under 3.4... $ ~/cpython/default/python -m unittest discover -v Traceback (most recent call last): File "/home/antoine/cpython/default/Lib/runpy.py", line 170, in _run_module_as_main "__main__", mod_spec) File "/home/antoine/cpython/default/Lib/runpy.py", line 85, in _run_code exec(code, run_globals) File "/home/antoine/cpython/default/Lib/unittest/__main__.py", line 18, in main(module=None) File "/home/antoine/cpython/default/Lib/unittest/main.py", line 93, in __init__ self.runTests() File "/home/antoine/cpython/default/Lib/unittest/main.py", line 244, in runTests self.result = testRunner.run(self.test) File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 168, in run test(result) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__ return self.run(*args, **kwds) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run test(result) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__ return self.run(*args, **kwds) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run test(result) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 87, in __call__ return self.run(*args, **kwds) File "/home/antoine/cpython/default/Lib/unittest/suite.py", line 125, in run test(result) File "/home/antoine/cpython/default/Lib/unittest/case.py", line 625, in __call__ return self.run(*args, **kwds) File "/home/antoine/cpython/default/Lib/unittest/case.py", line 553, in run result.startTest(self) File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 54, in startTest self.stream.write(self.getDescription(test)) File "/home/antoine/cpython/default/Lib/unittest/runner.py", line 47, in getDescription return '\n'.join((str(test), doc_first_line)) File "/home/antoine/cpython/default/Lib/unittest/case.py", line 1354, in __str__ self._testFunc.__name__) AttributeError: 'str' object has no attribute '__name__'
msg229732 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-10-20 17:16
I assume you mean you get the error *with* 3.5 (not "without"). Does this happen *every time* (i.e. is it trivially reproducible) - or can you provide a repro? This is regression that I would *assume* (a totally lazy assumption) introduced by the new error handling code.
msg229733 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-20 17:19
> I assume you mean you get the error *with* 3.5 (not "without"). Yes, sorry :-) > Does this happen *every time* (i.e. is it trivially reproducible) It happens with the llvmlite repository: https://github.com/numba/llvmlite Since using it requires some compiling, I could try to investigate myself if you tell me what to look for.
msg229734 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-10-20 17:22
As _testFunc is a string and shouldn't be, I'd be very interested to know *what* the string is. That may give us a clue as to where it has come from. (So a try...except...raise that also prints that value would be a good first step.)
msg229735 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-20 17:25
testFunc contains "runTest" (!).
msg229737 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-20 17:33
Ok, apparently it's because I have "from unittest import *" somewhere.
msg230367 - (view) Author: Ezio Melotti (ezio.melotti) * (Python committer) Date: 2014-10-31 18:35
Does removing "from unittest import *" from somewhere fix the issue? Is this a bug in your code, or did that import reveal a bug in unittest?
msg230369 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-31 18:37
> Does removing "from unittest import *" from somewhere fix the issue? Yes. > Is this a bug in your code, or did that import reveal a bug in unittest? Your choice :-)
msg230370 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-31 18:38
btw, in case that's not clear from the traceback, discover was mistaking FunctionTestCase as one of my test classes (while it was just there because of the "import *").
msg230372 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-10-31 18:40
Ah. Test discovery is discovering FunctionTestCase and attempting to instantiate it as a test. And failing. Maybe discovery should special case that class and not treat it as a normal TestCase.
msg230373 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014-10-31 18:41
Or perhaps everything that has as a __module__ something which has the attribute __unittest = True? (the attribute is there for a reason, right?)
msg230374 - (view) Author: Ethan Furman (ethan.furman) * (Python committer) Date: 2014-10-31 18:43
Michael Foord opined: --------------------- > Maybe discovery should special case that class and not treat it as a normal TestCase. Not a bad idea. On the other hand, I don't believe unittest is advertised as supporting 'from ... import *', which *is* usually advised to be a bad idea.
msg230629 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2014-11-04 15:07
This was reported as https://code.google.com/p/unittest-ext/issues/detail?id=71 a while back. I think blacklisting FunctionTestCase in TestLoader is entirely reasonable.
msg230672 - (view) Author: Michael Foord (michael.foord) * (Python committer) Date: 2014-11-05 09:49
I agree.
msg233275 - (view) Author: Martin Panter (martin.panter) * (Python committer) Date: 2015-01-01 06:44
Assuming that FunctionTestCase inherits from TestCase, a fix for Issue 14534 would be useful here. That bug is about avoiding TestCase subclasses being automatically run, which is useful for abstract base test classes.
msg247488 - (view) Author: Evan Hubinger (Evan Hubinger) * Date: 2015-07-27 21:38
I wrote a patch to blacklist FunctionTestCase in TestLoader, and a test to make sure FunctionTestCase doesn't show up in the TestSuite after loading a module that includes it. The test runs successfully. This is my first patch, so feedback would be appreciated if I did anything wrong.
msg248923 - (view) Author: Robert Collins (rbcollins) * (Python committer) Date: 2015-08-20 23:30
Thanks for this. I think that a better approach would be the other linked bug - we can kill many birds with one stone.
History
Date User Action Args
2022-04-11 14:58:09 admin set github: 66870
2015-08-25 07:37:04 martin.panter set dependencies: + Add means to mark unittest.TestCases as "do not load".
2015-08-20 23:30:49 rbcollins set status: open -> closedresolution: duplicatemessages: +
2015-08-19 23:58:50 rbcollins set stage: needs patch -> patch review
2015-07-27 21:38:14 Evan Hubinger set files: + issue22680.patchnosy: + Evan Hubingermessages: + keywords: + patch
2015-07-21 07:40:27 ethan.furman set nosy: - ethan.furman
2015-01-01 06:44:32 martin.panter set messages: +
2015-01-01 06:06:15 martin.panter set nosy: + martin.panter
2014-11-05 11:21:30 ezio.melotti set keywords: + easy
2014-11-05 09:49:13 michael.foord set messages: + title: unittest discovery is fragile -> Blacklist FunctionTestCase from test discovery
2014-11-04 15:07:53 rbcollins set messages: +
2014-10-31 18:44:40 Arfrever set nosy: + Arfrever
2014-10-31 18:43:57 ethan.furman set messages: +
2014-10-31 18:41:53 pitrou set messages: +
2014-10-31 18:40:03 michael.foord set messages: +
2014-10-31 18:38:49 pitrou set messages: +
2014-10-31 18:37:17 pitrou set messages: +
2014-10-31 18:35:23 ezio.melotti set messages: +
2014-10-20 17:53:32 ethan.furman set nosy: + ethan.furman
2014-10-20 17:45:58 barry set nosy: + barry
2014-10-20 17:33:17 pitrou set messages: +
2014-10-20 17:25:55 pitrou set messages: +
2014-10-20 17:22:26 michael.foord set messages: +
2014-10-20 17:19:12 pitrou set messages: +
2014-10-20 17:16:55 michael.foord set messages: +
2014-10-20 17:14:21 pitrou create