TST: Disallow bare pytest.raises · Issue #23922 · pandas-dev/pandas (original) (raw)

End users rely on error messages for their debugging purposes. Thus, it is important that we make sure that the correct error messages are surfaced depending on the error triggered.

I would like to propose that we disallow the following:
I think we should convert the following:

with pytest.raises(klass):

Your code here.

And in the spirit of tm.assert_raises_regex, we should ensure that all tests have the following:

with pytest.raises(klass, match=msg): # Your code here.

There are likely a few corner cases that would need to be ironed out, but I think the net benefit of converting this format would be much better for the robustness of our tests.

If we are going to focus on error message quality, then let's test them consistently! 🙂

Want to open to discussion first to see if we are on board with this. If we are, then this issue will be converted to one for doing said conversions and then (maybe) enforcing in our linter.

xref #16521
xref #23550
xref #23853

===============================================================

Running this command:

grep -r -e "pytest.raises([a-zA-Z]*)" pandas/tests -l

yields the following list:

pandas/tests/api/test_types.py
pandas/tests/arithmetic/test_datetime64.py
pandas/tests/arithmetic/test_numeric.py
pandas/tests/arithmetic/test_object.py
pandas/tests/arithmetic/test_period.py
pandas/tests/arithmetic/test_timedelta64.py
pandas/tests/arrays/categorical/test_algos.py
pandas/tests/arrays/categorical/test_analytics.py
pandas/tests/arrays/categorical/test_api.py
pandas/tests/arrays/categorical/test_constructors.py
pandas/tests/arrays/categorical/test_indexing.py
pandas/tests/arrays/categorical/test_operators.py
pandas/tests/arrays/interval/test_ops.py
pandas/tests/arrays/sparse/test_array.py
pandas/tests/arrays/sparse/test_dtype.py
pandas/tests/arrays/sparse/test_libsparse.py
pandas/tests/arrays/test_array.py
pandas/tests/arrays/test_datetimelike.py
pandas/tests/arrays/test_integer.py
pandas/tests/computation/test_compat.py
pandas/tests/computation/test_eval.py
pandas/tests/dtypes/test_cast.py
pandas/tests/dtypes/test_dtypes.py
pandas/tests/extension/base/getitem.py
pandas/tests/extension/base/ops.py
pandas/tests/extension/base/reduce.py
pandas/tests/extension/base/setitem.py
pandas/tests/extension/decimal/test_decimal.py
pandas/tests/extension/json/test_json.py
pandas/tests/extension/test_categorical.py
pandas/tests/extension/test_integer.py
pandas/tests/frame/test_analytics.py
pandas/tests/frame/test_apply.py
pandas/tests/frame/test_arithmetic.py
pandas/tests/frame/test_axis_select_reindex.py
pandas/tests/frame/test_constructors.py
pandas/tests/frame/test_convert_to.py
pandas/tests/frame/test_dtypes.py
pandas/tests/frame/test_duplicates.py
pandas/tests/frame/test_indexing.py
pandas/tests/frame/test_missing.py
pandas/tests/frame/test_mutate_columns.py
pandas/tests/frame/test_operators.py
pandas/tests/frame/test_quantile.py
pandas/tests/frame/test_query_eval.py
pandas/tests/frame/test_reshape.py
pandas/tests/frame/test_sorting.py
pandas/tests/frame/test_timeseries.py
pandas/tests/generic/test_frame.py
pandas/tests/generic/test_generic.py
pandas/tests/generic/test_series.py
pandas/tests/groupby/test_categorical.py
pandas/tests/groupby/test_groupby.py
pandas/tests/groupby/test_timegrouper.py
pandas/tests/indexes/common.py
pandas/tests/indexes/datetimelike.py
pandas/tests/indexes/datetimes/test_arithmetic.py
pandas/tests/indexes/datetimes/test_astype.py
pandas/tests/indexes/datetimes/test_construction.py
pandas/tests/indexes/datetimes/test_date_range.py
pandas/tests/indexes/datetimes/test_indexing.py
pandas/tests/indexes/datetimes/test_timezones.py
pandas/tests/indexes/datetimes/test_tools.py
pandas/tests/indexes/interval/test_astype.py
pandas/tests/indexes/interval/test_interval.py
pandas/tests/indexes/interval/test_interval_new.py
pandas/tests/indexes/interval/test_interval_tree.py
pandas/tests/indexes/multi/test_analytics.py
pandas/tests/indexes/multi/test_compat.py
pandas/tests/indexes/multi/test_constructor.py
pandas/tests/indexes/multi/test_drop.py
pandas/tests/indexes/multi/test_duplicates.py
pandas/tests/indexes/multi/test_get_set.py
pandas/tests/indexes/multi/test_indexing.py
pandas/tests/indexes/multi/test_integrity.py
pandas/tests/indexes/multi/test_monotonic.py
pandas/tests/indexes/multi/test_partial_indexing.py
pandas/tests/indexes/multi/test_sorting.py
pandas/tests/indexes/period/test_arithmetic.py
pandas/tests/indexes/period/test_construction.py
pandas/tests/indexes/period/test_indexing.py
pandas/tests/indexes/period/test_partial_slicing.py
pandas/tests/indexes/period/test_period.py
pandas/tests/indexes/test_base.py
pandas/tests/indexes/test_category.py
pandas/tests/indexes/test_common.py
pandas/tests/indexes/test_numeric.py
pandas/tests/indexes/test_range.py
pandas/tests/indexes/timedeltas/test_arithmetic.py
pandas/tests/indexes/timedeltas/test_construction.py
pandas/tests/indexes/timedeltas/test_indexing.py
pandas/tests/indexing/interval/test_interval.py
pandas/tests/indexing/interval/test_interval_new.py
pandas/tests/indexing/multiindex/test_setitem.py
pandas/tests/indexing/multiindex/test_slice.py
pandas/tests/indexing/test_categorical.py
pandas/tests/indexing/test_indexing.py
pandas/tests/indexing/test_loc.py
pandas/tests/indexing/test_panel.py
pandas/tests/indexing/test_partial.py
pandas/tests/indexing/test_scalar.py
pandas/tests/internals/test_internals.py
pandas/tests/io/formats/test_format.py
pandas/tests/io/formats/test_style.py
pandas/tests/io/formats/test_to_latex.py
pandas/tests/io/json/test_json_table_schema.py
pandas/tests/io/json/test_normalize.py
pandas/tests/io/json/test_pandas.py
pandas/tests/io/json/test_ujson.py
pandas/tests/io/parser/test_network.py
pandas/tests/io/parser/test_python_parser_only.py
pandas/tests/io/sas/test_sas7bdat.py
pandas/tests/io/test_clipboard.py
pandas/tests/io/test_common.py
pandas/tests/io/test_excel.py
pandas/tests/io/test_feather.py
pandas/tests/io/test_gcs.py
pandas/tests/io/test_html.py
pandas/tests/io/test_parquet.py
pandas/tests/io/test_pytables.py
pandas/tests/io/test_sql.py
pandas/tests/io/test_stata.py
pandas/tests/plotting/test_boxplot_method.py
pandas/tests/plotting/test_frame.py
pandas/tests/plotting/test_hist_method.py
pandas/tests/plotting/test_series.py
pandas/tests/reductions/test_reductions.py
pandas/tests/reductions/test_stat_reductions.py
pandas/tests/resample/test_base.py
pandas/tests/resample/test_period_index.py
pandas/tests/resample/test_resample_api.py
pandas/tests/reshape/merge/test_join.py
pandas/tests/reshape/merge/test_merge.py
pandas/tests/reshape/merge/test_merge_asof.py
pandas/tests/reshape/merge/test_multi.py
pandas/tests/reshape/test_concat.py
pandas/tests/reshape/test_melt.py
pandas/tests/reshape/test_pivot.py
pandas/tests/reshape/test_reshape.py
pandas/tests/reshape/test_union_categoricals.py
pandas/tests/scalar/interval/test_interval.py
pandas/tests/scalar/period/test_asfreq.py
pandas/tests/scalar/period/test_period.py
pandas/tests/scalar/timedelta/test_arithmetic.py
pandas/tests/scalar/timedelta/test_construction.py
pandas/tests/scalar/timedelta/test_timedelta.py
pandas/tests/scalar/timestamp/test_comparisons.py
pandas/tests/scalar/timestamp/test_timestamp.py
pandas/tests/scalar/timestamp/test_timezones.py
pandas/tests/scalar/timestamp/test_unary_ops.py
pandas/tests/series/indexing/test_alter_index.py
pandas/tests/series/indexing/test_boolean.py
pandas/tests/series/indexing/test_datetime.py
pandas/tests/series/indexing/test_indexing.py
pandas/tests/series/indexing/test_numeric.py
pandas/tests/series/test_alter_axes.py
pandas/tests/series/test_analytics.py
pandas/tests/series/test_api.py
pandas/tests/series/test_apply.py
pandas/tests/series/test_arithmetic.py
pandas/tests/series/test_asof.py
pandas/tests/series/test_dtypes.py
pandas/tests/series/test_missing.py
pandas/tests/series/test_operators.py
pandas/tests/series/test_replace.py
pandas/tests/series/test_timeseries.py
pandas/tests/series/test_timezones.py
pandas/tests/sparse/frame/test_frame.py
pandas/tests/sparse/test_indexing.py
pandas/tests/test_base.py
pandas/tests/test_common.py
pandas/tests/test_errors.py
pandas/tests/test_lib.py
pandas/tests/test_nanops.py
pandas/tests/test_panel.py
pandas/tests/test_take.py
pandas/tests/test_window.py
pandas/tests/tools/test_numeric.py
pandas/tests/tseries/offsets/test_offsets.py
pandas/tests/tseries/offsets/test_ticks.py
pandas/tests/tseries/test_frequencies.py
pandas/tests/tseries/test_holiday.py
pandas/tests/tslibs/test_array_to_datetime.py
pandas/tests/tslibs/test_liboffsets.py
pandas/tests/tslibs/test_timedeltas.py
pandas/tests/tslibs/test_timezones.py