fail-under should show more precision; confusing when just a small value below threshold · Issue #284 · nedbat/coveragepy (original) (raw)

Skip to content

Navigation Menu

• • GitHub Copilot Write better code with AI
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
• Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
• • Enterprise platform AI-powered developer platform
• Pricing

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sign up

Appearance settings

Description

Originally reported by Jon Dufresne (Bitbucket: jdufresne, GitHub: jdufresne)

If the test coverage is just a small fraction below the fail-under value, the report will fail with a non-zero exit status (good and expected) but the report appears as if everything is ok. As an example I received the following report:

$ coverage report --fail-under=70
Name                                    Stmts   Miss  Cover   Missing
---------------------------------------------------------------------
...  
---------------------------------------------------------------------
TOTAL                                    1816    551    70%   
$ echo $?
2

Doing the math:

>>> (1816.0 - 551.0) / 1816.0
0.6965859030837004

This is indeed below 70%. However at first glance, coverage reports this as 70% coverage (due to rounding I assume) creating confusion for me the user. In the case of failure, I would like to see report provide a more precise number for coverage so that the failure is immediately obvious.

• Bitbucket: https://bitbucket.org/ned/coveragepy/issue/284