[Major] Release Notes for Codecov v5.0

🚧

Codecov v5 is a Major Upgrade

Codecov v5 brings many new features and functionality to Codecov. It is recommended to test v5 in a staging / pre-production environment before updating a working v4.6.X installation.

Upgrading to v5 requires some infrastructure changes, we recommend reviewing our upgrade guide (link TBD)

New Features

Codecov v5 brings many new features to our on-premises release that have been rolling out to our cloud version of Codecov for quite some time. Rather than explain those new features in detail here, we encourage reviewing prior release announcements for those features when they were released on Codecov Cloud.

• Impact Analysis (docs) -- Learn which aspects of your code are executed the most on production, and annotate the Codecov PR comment with extra information when these files are changed.
• Flag Coverage Over Time -- see trends in your coverage on a per Flag basis (see below regarding Self-Hosted specific considerations).
• Components (docs) -- a flexible method for grouping coverage in your repositories based on file paths instead of coverage uploads.
• Removed Code Behavior (docs) -- allows Codecov status checks to pass in those instances where code refactors can lead to removing more covered code than contributed in the PR.
• Public API v2 (docs note these docs are still on our v4.6 branch and are being migrated to v5.0 of our docs)-- a completely redesigned public API that exposes much more data than our prior API in a more user friendly way.
• Manual Critical file Labelling (docs) -- Mark important files in your repository and Codecov will signify when these files are changed in PRs.

Instance Wide Admins and Self-Hosted User Management

A common point of feedback for self-hosted users was the difficulty managing user activation/deactivation in Codecov v4. To address this problem we've completely overhauled user management and moved it to a dedicated admin area of the Codecov application.

In Codecov v5, we have added the concept of Instance Wide Admins. Instance wide admins have access to a new admin area within the Codecov application where they can directly manage, search, and toggle activation status for any user in their organization.

Additionally, we have surfaced the current active user count more directly, so that the current amount of seats available in your install is viewable from the top navigation

Note that if you do not have Instance Wide Admins specified in your codecov install yaml, you will see the following banner in the Codecov application:

Once added via the codecov install yaml, Instance Wide Admins will be able to access Codecov's User Management feature dashboard:

Redesigned User Interface

The Codecov application UI has been overhauled to overcome many shortcomings of the previous design and implementation. In addition to a new look and feel, page load times should be improved for the Commit and Pull Pages, especially for extremely large PRs and commits. In many cases, very large commits and PRs that may have failed to load in the old UI will now render properly and fully.

New affordances have also been made to improve accessibility when viewing code:

New users to Codecov repeatedly reported difficulties interpreting the "Codecov Delta". The new UI moves away from the "Codecov delta" to a more direct statement of coverage change and total coverage:

Improved Error Messaging

Codecov's new UI also provides greatly expanded error messaging throughout. These newly improved errors and warnings should serve to better guide users toward resolutions when things don't go according to plan. For example, a common cause of errors / confusion occurs as a result of test failures or flaky tests in a CI pipeline that result in less coverage being uploaded for subsequent commits. Now, the Codecov application will point out such discrepancies when viewing the Pulls page:

Self-Hosted Considerations for Flags

In order to make use of Codecov's new Flags feature, the TimescaleDB dependency was introduced. Codecov v5 functions fully without this dependency, but features related to to Flag coverage over time, such as the Flags tab for a repository, will not function.

It is highly recommended to upgrade to v5 first then install the TimescaleDB dependency. This approach will allow install maintainers to verify that v5 was installed properly before incorporating an additional infrastructure level dependency.

You can read more about the process of installing TimescaleDB in our documentation.

Required Infrastructure Changes

With our V5 launch, we are introducing 2 new containers and removing 1.

Adding:

• enterprise-frontend: This serves our new UI and any web components that are needed such as the uploader.
• enterprise-gateway: This serves as the entrypoint for the Codecov deployment. It handles routing between the frontend and the API.

Removing:

• enterprise-web: Our old monolith has finally been fully replaced!

After making the needed container changes, you will need to adjust your load balancer or ingress to point at the enterprise-gateway rather than enterprise-web. The Codecov app should take care of the rest.

It is recommended to review the following infrastructure configurations, which have been updated for v5:

• docker-compose (https://github.com/codecov/self-hosted) repositories.

🚧

Required configuration for enterprise-frontend

To function correctly, the frontend needs to know the base and api host for your Codecov install. These should be provided as environment variables. This step is handled automatically by our terraform templates.

- name: CODECOV_BASE_HOST
  value: your.codecov.domain
- name: CODECOV_API_HOST
  value: your.codecov.domain

❗️

DB Changes

As with any upgrade, it is highly recommended to take a database backup prior to upgrading. The V5 upgrade contains a number of internal database improvements including indexes. These will run automatically but it is recommended to perform this upgrade during a period of low traffic if you are not operating in a maintenance window.

The new minimum postgres version is 14. We recommend upgrading to 14.X.

Docker Image Information

The following Docker digests comprise this release:

• codecov/enterprise-gateway:v5.0.0 -
  sha256:81a823a37f95e27ff7d94e2eefcb5a7732bb1ce275785cf894ff5fc6d127c433
• codecov/enterprise-frontend:v5.0.0 -
  sha256:f1848a30e1679384cec639182aec0d66faa7ee112cf1bcaf2d027aa231abc0f1
• codecov/enterprise-worker:v5.0.0 - sha256:9b5903a65ebc922262341b19c5d908dd7a404043d25c4164a3ddf10bcc2ab1dc
• codecov/enterprise-api:v5.0.0 -
  sha256:d474e65c065dc1edbca11357fca30942dc1c3eeb01053baf7d13fe3cba41ef6e

These are also the digests referenced by the latest-stable tag.

Note: due to a user impacting bug in enterprise-gateway and enterprise-frontend a point release was issued and pushed over the existing v5.0.0 tag. The new digest is indicated above, the previous digest(s) are as follows:

• codecov/enterprise-gateway:v5.0.0 - sha256:d54b5d079d3a452f9560980b6f118e4ffc8b8726528c52a143c7d6015032b3a4
• codecov/enterprise-frontend:v5.0.0 -
  sha256:b3761f32b732f7d49d56daf53bb21812d6d6299476cd94c4e6ed906532fe70f2
  sha256:ccc3a52f9f996b99e3c0eed30caf91f5e2edba5aace9a58c4ed080e89056570c
  sha256:755a2e0fd0a725014fa6baf47b0c116413f8807ad05f0aa0b7b289e08d8121d
• codecov/enterprise-api:v5.0.0 - sha256:api:690c985743d4c8c9a895f259c5f25bd01fb1a0c8e9fbcc52e0f6201f49752711
  sha256:8f4ba69cb62e50129df2e4b5aea3867b4097278f947c324a1678d040523ca839