Report integration with Sphinx · Issue #1730 · nedbat/coveragepy (original) (raw)

Is your feature request related to a problem? Please describe.

Usually, all my Python projects are documented using Sphinx. Such a documentation contains also an auto generated Python Class Reference as well as multiple code quality indications / analysis reports. Currently this is: code coverage by coveragypy, unit tests by pytest, static typing analysis by mypy and documentation coverage by docstr_coverage.

Most tools provide a machine readable output format (e.g. in XML or JSON) and almost all are writing or converting them to an HTML report. This can be a single file or multiple files in a directory.

For integration into the overall documentation, multiple CI jobs generate the results and generated HTML pages and then a final CI job combines all artifacts to a combined static website for e.g. GitHub Pages.

In the last days I investigated and prototyped a new Sphinx extension, which integrates a documentation coverage report directly as native Sphinx content into the documentation. This offers some huge benefits:

• reports are part of the overall design / no style break
• reports are also part of other Sphinx backend outputs e.g. PDF documentation
• parts of the report can be referenced by the main documentation
• reports are part of the main navigation / table of content

I would like to discuss if and how to integrate coveragepy into such a Sphinx extension. My extension sphinx-reports is open to more report formats like code coverage.

This is an example from my prototype for documentation coverage:

Describe the solution you'd like
I would like to read and render code coverage results into Sphinx documentation pages.

I would like to address it in 2 steps:

1. Integrate coveragepy's summary page and then link to auto generated coverage pages.
   (I haven't figured out how to write complex Sphinx content like colored source code.)
2. Also write source code files via Sphinx.

The extensions alpha code is here: https://pytooling.github.io/sphinx-reports

Describe alternatives you've considered
I checked, if auto generated HTML code from coveragepy could be copied into a placeholder Sphinx page. While content was transferred, I found huge problems with CSS and Javascript contents. It would results in dozens - if not hundreds - of regex-based rewrite and transformation rules. This can't be maintained.

An other alternative would be coveragepy offering ReST (Restructured Text) output and such files could be referenced by a toctree directive.

Additional context
I believe that expressing the quality of an open-source project is a need to convince others to use and maybe invest time in someone open-source project. It also can shorten some discussions if a library can be used easily and safely.