Pull Request Comments

The Codecov pull request comment is a quick overview of how a pull request (and relevant commits) will affect the code coverage. This simple integration allows developers to see the effects of their work, without having to leave the familiar Github UI. The PR comment provides details on the coverage changes in the pull request that help with speeding up code reviews, and ensuring that all incoming features and fixes are well tested. The comment is customizable to adjust the behavior and logic, as well as the actual output, please see below for configuration details.

❓

Codecov Delta

It is important to understand what absolute <relative> (impact) represents. Learn all about the Codecov Delta.

Alternate view: Only show coverage on the git diff ("Patch" coverage)

If you want to focus your work only on the patch you can learn how to toggle on "Patch" only coverage in the git diff here:

PR comment for Git Diff only

Configuration

comment:                  # this is a top-level key
  layout: " diff, flags, files"
  behavior: default
  require_changes: false  # learn more in the Requiring Changes section below
  require_base: false        # [true :: must have a base report to post]
  require_head: true       # [true :: must have a head report to post]
  hide_project_coverage: false # [true :: only show coverage on the git diff]

Layout

Customize your comment layout by choosing the order and types of components to include. The following are components you can add or choose to opt out of. Omitting any of these from your layout will cause them to not show up.

👍

Our most up to date PR Comment uses the following:

comment:
     layout: "condensed_header, condensed_files, condensed_footer" # add "condensed_" to "header", "files" and "footer"
      hide_project_coverage: TRUE # set to true

• Diff is the Coverage Diff of the pull request.
  • Learn more here: Coverage Diff.
• Flags are a list of user defined Flags, and their impact on coverage in the PR.
  • Learn more about setting up Flags.
• Components are a list of user defined Components, and their impact on coverage in the PR.
  • Learn more here: Components.
• Files or Tree are a list of files that are impacted by the pull request (coverage changes, file is new or removed).
• Header is the top of the comment showing the two commits being merged, the change in coverage, and the diff coverage.
• Footer is the bottom of a comment that contains the legend, last updated commits, and docs link.
• Feedback is a section that provides a link to provide feedback to Codecov.

Behavior

Choose the way Codecov submits comments in your pull requests.

• default: update, if exists. Otherwise post new.
• once: update, if exists. Otherwise post new. Skip if deleted.
• new: delete old and post new.

📘

Behavior: default

When updating, the creation date of the comment will stay the same. There will be an edited link with a drop-down where you can check the times updated.

Emails will be submitted to all subscribers of the pull request only when posting a new comment. There is no way to disable email notifications. This is a limitation of the Supported CI Providers.

Requiring Changes

You may desire to change the conditions for the comment to be posted. Available options include:

• false [default] - Post comment even if there's no change in coverage
• true - Only post comment if there are changes in coverage (positive or negative)
• "any_change" - Same as true
• "coverage_drop"[project coverage] - Only post comment if coverage drops when comparing HEAD to BASE. This option respects project status threshold , if configured (i.e. drop needs to be bigger than the configured threshold for the comment to be posted).
• "uncovered_patch" [patch coverage] - Only post comment if the patch has uncovered lines

Additionally it is possible to combine the string options using AND and OR operators. This will combine the requirements according to the operator used.

📘

Comments are always edited, if they exist

Requiring changes are for posting the comment the 1st time. If it was already posted Codecov will always update the comment. Editing a comment doesn't send a new email to subscribers of the pull request.

Examples

# Require any change in coverage to post
comment:
  require_changes: true
  
# Only post if coverage drops
comment:
  require_changes: "coverage_drop"
  
# Only post if coverage drops OR there are uncovered lines of code
comment:
  require_changes: "coverage_drop OR uncovered_patch"
  
# Only post if coverage drops AND there are uncovered lines of code
comment:
 require_changes: "coverage_drop AND uncovered_patch" 

Requiring the Base and/or Head Commit

You can force comments to post on pull requests even if Codecov doesn't have coverage reports for either the base or head commit. This generally isn't recommended since, without base and head coverage report information, the resulting comment is not very helpful.

However, setting both require_base and require_head to false can allow you to test the ability of Codecov to post a comment to your pull requests without needing to upload coverage for both the base and head commit.

comment:
  require_head: false     # default true
  require_base: false     # default true

after_n_builds

By default Codecov will post and/or update the pull request comment after it processes each report uploaded for a particular pull request commit. If your CI process uploads many reports to codecov, this can be confusing for team members viewing the pull request as reports are processing.

To remedy this, you can delay the posting of a pull request comment until a certain number of reports are received and processed by Codecov, using the after_n_builds setting.

Generally, you should set after_n_builds equal to the number of reports you upload per commit. However, this is not required. If after_n_builds is greater than the number of reports you upload for a commit, no comment will post. If it is less, the comment will post when after_n_builds is reached and (by default) continue to update on each additional report upload.

after_n_builds for flags

After_n_builds for flags allows you to specify how many uploads are in each Flag and only receive notifications for that flag after Codecov receives the n uploads for that Flag. Longer form blog on this functionality.

When using flag_management configuration will look like this:

flag_management:
  individual_flags:
    - name: projectA
      paths:
        - projectA/
      after_n_builds: 3
    - name: projectB
      paths:
        - projectB/
      after_n_builds: 2

When using flags to specify your flags setup, configuration will look like so:

flags:
  projectA:
    paths:
      - projectA/
    after_n_builds: 3
  projectB:
    paths:
      - projectB/
    after_n_builds: 2

show_carryforward_flags

For those using the Carryforward Flags feature, Carryforward Flags do not show in the pull request comment by default.

You can configure presence of Carryforward Flags in the flags table via advanced usage of Carryforward Flags.

Disable comment

To disable the comment, simply set the top-level comment key to false in your codecov.yml, as shown below.

comment: false

You can do this in your Team Yaml, to take effect on all your repositories.

❗️

Setting the top-level comment key to true is not a valid configuration, as it will overwrite the default configuration with a boolean value and preclude the sending of any comment.

For more troubleshooting options in Gitlab and Bitbucket, be sure to read about the Team Bot

Filtering Emails

Unfortunately Codecov can't "silently" post a comment. All subscribers of a Pull Request get an email when Codecov posts a comment (only when posting a new comment), and there's nothing we can do about it.

We do understand that this can be distracting to some users. If the [required changes] for the comment don't cover your needs, or if you don't have the permissions to make changes to that, the next best thing is to create a filter in your email client.

For example if you are a GitHub user you can create a filter using from: [email protected] and contains: All modified and coverable lines are covered by tests ✅ to mark as read or delete emails for the Codecov coverage comment that has 100% patch coverage.