Looking for additional help?
See our video tutorial on diagnosing unexpected coverage changes at the bottom of this page.
We have found there are many reasons why coverage may change in unexpected ways.
Coverage changes are coverage to lines of code that are not adjusted in the pull/commit diff. This can occur for a number of reasons.
Codecov will analyze the pull/commit diff, detecting lines of code that changed without being adjusted. A simple example of this is removing tests, which is shown below.
## Unchanged source def string_len(string): return len(string) ## Removed test - def test_my_method(): - assert string_len('jedi') == 4
In the code above, the developer removed a single test which changes the coverage of the method
string_len from being
This change in coverage will be surfaced in the Codecov App through our Changes page.
There are a number of reasons we have found that cause line coverage to change unexpectedly. These can be due to:
- Adding or removing tests.
- Failing to upload coverage reports, or a different number of reports between head and base
- Time sensitive tests.
- Missing coverage reports or failed builds.
- Dependencies changed resulting in a different execute plan.
- Encrypted variables may prevent some execution paths.
- In the case of changes to a PR, the base or head commits may not have had an uploaded report, or may not be the commit you expect
Most unexpected changes occur in the raw coverage report before Codecov processes the uploaded coverage file.
How to review the the coverage report for unexpected changes:
- Proceed to the Builds tab when viewing a single commit.
- Click the Download button on one or more of your uploads.
- Scroll through the file network and discover your uploaded reports.
- Review the report, looking for the line(s) that are not reported accurately.
Codecov stores the raw, unprocessed report data precisely for this issue. Reports are archived for 30 days before removal.
Example repository from video: https://github.com/kensho-technologies/graphql-compiler/pull/806
Updated over 1 year ago