Confusion between message and match parameters in pytest.raises · Issue #3974 · pytest-dev/pytest (original) (raw)

After analyzing several code bases, it appears there is frequent confusion by users between the match and message parameters in pytest.raises (and consequentially pytest.warns).

In most cases, users probably would want match but use message by accident, because is it visually similar and the naming does not differentiate their functionality enough. As a result the written test produces false positive which is quite dangerous.

For instance, here are instances of such errors in popular open-source projects,

• TST Replace unittest.TestCase.fail calls, use pytest.raises instead of legacy constructs pandas-dev/pandas#22681 (comment)
• TST Use assert_raises "match" parameter instead of the "message" parameter. scipy/scipy#9264
• [MRG]Fix pytest message that silently passes no matter if it should scikit-learn/scikit-learn#12001

the fact that these cases ended up in the code despite fairly thorough review is telling.

There was also an earlier issue illustrating this confusion in #3350

Searching on github for "pytest.raises(ValueError, message=" for instance, also shows a number of examples that meant to use "match" instead of "message" (though sometimes it's hard to tell).

One possibility could be to deprecate the message parameter and rename it to something more explicit. I saw there were some deprecations related to the use of node.warn in 3.8, not sure if that's related. However, following the same logic even if pytest.raises(ValueError('expected message')) worked it would still not address all the existing code that currently uses the message parameter incorrectly.

EDIT by @nicoddemus

The message parameter has been deprecated and will be removed in pytest 5.0, please see the docs for an alternative which provides the same functionality:

https://docs.pytest.org/en/latest/deprecations.html#raises-message-deprecated