Testing - cloud-init 25.1.2 documentation (original) (raw)

View this page

Toggle table of contents sidebar

Cloud-init has both unit tests and integration tests. Unit tests can be found at tests/unittests. Integration tests can be found attests/integration_tests. Documentation specifically for integration tests can be found on the Integration testing page, but the guidelines specified below apply to both types of tests.

Cloud-init uses pytest to run its tests, and has tests written both as unittest.TestCase sub-classes and as un-subclassed pytest tests.

Guidelines¶

The following guidelines should be followed.

Test layout¶

• For ease of organisation and greater accessibility for developers unfamiliar with pytest, all cloud-init unit tests must be contained within test classes. In other words, module-level test functions should not be used.
• Since all tests are contained within classes, it is acceptable to mixTestCase test classes and pytest test classes within the same test file.
  • These can be easily distinguished by their definition: pytestclasses will not use inheritance at all (e.g.,TestGetPackageMirrorInfo), whereas TestCase classes will subclass (indirectly) from TestCase (e.g.,TestPrependBaseCommands).
• Unit tests and integration tests are located under cloud-init/tests.
  • For consistency, unit test files should have a matching name and directory location under tests/unittests.
  • E.g., the expected test file for code in cloudinit/path/to/file.pyis tests/unittests/path/to/test_file.py.

pytest tests¶

• pytest test classes should use pytest fixtures to share functionality instead of inheritance.
• pytest tests should use bare assert statements, to take advantage of pytest’s assertion introspection.

Dependency versions¶

Cloud-init supports a range of versions for each of its test dependencies, as well as runtime dependencies. If you are unsure whether a specific feature is supported for a particular dependency, check the lowest-supportedenvironment in tox.ini. This can be run using tox -e lowest-supported. This runs as a Github Actions job when a pull request is submitted or updated.

Mocking and assertions¶

• Variables/parameter names for Mock or MagicMock instances should start with m_ to clearly distinguish them from non-mock variables. For example, m_readurl (which would be a mock forreadurl).
• The assert_* methods that are available on Mock andMagicMock objects should be avoided, as typos in these method names may not raise AttributeError (and so can cause tests to silently pass).
  • An important exception: if a Mock is autospecced then misspelled assertion methods will raise an AttributeError, so these assertion methods may be used on autospecced Mock objects.
• For a non-autospecced Mock, these substitutions can be used (m is assumed to be a Mock):
  • m.assert_any_call(*args, **kwargs) => assert mock.call(*args, **kwargs) in m.call_args_list
  • m.assert_called() => assert 0 != m.call_count
  • m.assert_called_once() => assert 1 == m.call_count
  • m.assert_called_once_with(*args, **kwargs) => assert [mock.call(*args, **kwargs)] == m.call_args_list
  • m.assert_called_with(*args, **kwargs) => assert mock.call(*args, **kwargs) == m.call_args_list[-1]
  • m.assert_has_calls(call_list, any_order=True) => for call in call_list: assert call in m.call_args_list
    * m.assert_has_calls(...) and m.assert_has_calls(..., any_order=False) are not easily replicated in a single statement, so their use when appropriate is acceptable.
  • m.assert_not_called() => assert 0 == m.call_count
• When there are multiple patch calls in a test file for the module it is testing, it may be desirable to capture the shared string prefix for these patch calls in a module-level variable. If used, such variables should be named M_PATH or, for datasource tests, DS_PATH.

Test argument ordering¶

• Test arguments should be ordered as follows:
  • mock.patch arguments. When used as a decorator, mock.patchpartially applies its generated Mock object as the first argument, so these arguments must go first.
  • pytest.mark.parametrize arguments, in the order specified to the parametrize decorator. These arguments are also provided by a decorator, so it’s natural that they sit next to themock.patch arguments.
  • Fixture arguments, alphabetically. These are not provided by a decorator, so they are last, and their order has no defined meaning, so we default to alphabetical.
• It follows from this ordering of test arguments (so that we retain the property that arguments left-to-right correspond to decorators bottom-to-top) that test decorators should be ordered as follows:
  • pytest.mark.parametrize
  • mock.patch