Merging Reports

A common practice in the modern developer's workflow is to test multiple build systems, split up tests in different containers, and group tests based on test focus (unittests vs integration tests). Codecov supports all these setups. Therefore, no setup is required.

Multiple Languages

Codecov supports multiple languages in the same repository. You may upload reports for one or more languages, and Codecov will merge the reports automatically while maintaining the original upload context.

Codecov does not override report data for multiple uploads. We always merge the data. Simply upload all three reports at once, or separately.

script:
  # run my python tests
  - coverage run tests.py
  
  # Your Codecov Uploader approach here
	# See more at https://docs.codecov.com/docs/codecov-uploader
  
  # upload my python reports
	./codecov  -cF python
  # run my javascript tests
  - npm test
  # upload my javascript reports
  ./codecov -cF javascript

Delayed Notifications

Codecov will delay notifications such as Slack, Hipchat, and Pull Request Comments until all reports are uploaded and merged. This prevents posting invalid coverage metrics when the first report is uploaded.

A good example of this is pyca/cryptography, with over 20 containers uploading coverage reports to Codecov. Each container runs tests resulting in ~90% coverage. However, when Codecov merges all reports together the result is 100% coverage. Codecov will submit the notifications once it determines that CI is complete and all reports are merged. The notifications expresses 💯, a number they work hard for. Nice job pyca team! 👍 🎉

Frequently Asked Questions

How does Codecov know when to send notifications?

Codecov will check, every minute, the status of each CI provider. If the status of any provider is pending, we will delay the notifications. If the status of any provider is error or failure, Codecov will cancel the notification entirely. If the status of all providers is success, the notifications will be submitted.

Why did I not receive a notification yet?

• If the CI status is "pending", please wait until the CI is complete.
• If the CI status is "error" or "failed", no notifications will be submitted.
  • Disable CI pass requirement.
  • Disable a specific CI provider.
• If CI is successful:
  • is your CI provider in the list of [[Supported CI Providers]].
  • review [[Commit Build Logs]].
  • check if the reports failed to upload to Codecov.
  • make sure your yaml is valid.

Why did I receive a notification too soon?

If you believe you are receiving a notification too soon, it may be in error. One way to prevent premature notifications, is preventing a notification until N builds, YAML configuration for here.

How do I remove checking a CI status?

You can toggle on and off Require CI to Pass setting in the YAML. More details here