Issue 32213: assertRaises and subTest context managers cannot be nested (original) (raw)

Created on 2017-12-04 14:44 by p-ganssle, last changed 2022-04-11 14:58 by admin. This issue is now closed.

Messages (4)

msg307569 - (view)

Author: Paul Ganssle (p-ganssle) * (Python committer)

Date: 2017-12-04 14:44

The TestCase.assertRaises and TestCase.subTest macros apparently don't interact with each other well. To demonstrate, consider the following code:

from unittest import TestCase

class SubTestRaisesTest(TestCase):
    def test_assert_outer(self):
        for to_raise in [True, False]:
            with self.assertRaises(Exception):
                with self.subTest(to_raise=to_raise):
                    if to_raise:
                        raise Exception()

    def test_assert_inner(self):
        for to_raise in [True, False]:
            with self.subTest(to_raise=to_raise):
                with self.assertRaises(Exception):
                    if to_raise:
                        raise Exception()

This actually fails in two different ways.

For test_assert_outer:

-with subtest to_raise=True, the test (correctly) passes. -with subtest to_raise=False, the test (correctly) fails, but the subtest is not actually assigned (no indication of which subtest it was that failed):

======================================================================
FAIL: test_assert_outer (test_bug.SubTestRaisesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File ".../assert_demo/test_bug.py", line 9, in test_assert_outer
    raise Exception()
AssertionError: Exception not raised

For test_assert_inner:

So, to sum up, the behavior that needs to be fixed:

  1. When assertRaises is the outer context, the subtest value needs to be set for the failing tests.

  2. When assertRaises is the inner context, the exception needs to be caught properly and cleared on exit from the assertRaises context.

msg307575 - (view)

Author: R. David Murray (r.david.murray) * (Python committer)

Date: 2017-12-04 16:08

To be pedantic, are not macros, they are context managers :)

Your first case is not something I would have thought of coding. In the to_raise=True case, the subTest is failing because an exception is raised inside its scope. In the to_raise=False case, the subTest scope has already ended before the assertRaises scope completes and raises its error because no exception was raised. That is, there is no subTest in effect to be reported when that failure occurs.

In your second case, when to_raise is False, no exception is raised, so the assertRaises correctly fails, and the subtest reports that to_raise is False. When to_raise is true, the exception is raised, the assertRaises passes and so does the subtest. In other words, I can't reproduce the problem you cite for the second case. Looking at what you pasted, it looks like you confused a test_assert_outer report with a test_report_inner report.

So, as far as I can see, there's nothing broken here, everything is working according to the documentation :)

msg307576 - (view)

Author: Paul Ganssle (p-ganssle) * (Python committer)

Date: 2017-12-04 16:34

@r.david.murray So it is, my mistake. I think I was conflating issues when this first came up, and then I didn't notice that the order the test cases printed in was different than I expected.

msg307580 - (view)

Author: R. David Murray (r.david.murray) * (Python committer)

Date: 2017-12-04 16:58

Yes, that's an easy mistake to make. I avoided it by only running one of the tests cases at a time, and I'll admit I had to think about it for a while to understand what was going on in your first case.

History

Date

User

Action

Args

2022-04-11 14:58:55

admin

set

github: 76394

2017-12-04 16:58:54

r.david.murray

set

resolution: not a bug

2017-12-04 16:58:46

r.david.murray

set

messages: +

2017-12-04 16:34:11

p-ganssle

set

status: open -> closed

messages: +
stage: resolved

2017-12-04 16:08:15

r.david.murray

set

nosy: + r.david.murray
messages: +

2017-12-04 14:44:28

p-ganssle

create