Release Notes for Codecov v4.4.8

❗️

This version introduces potentially breaking changes for on-premises / self-deployed users

If you're upgrading from an earlier version of Codecov Enterprise, be sure to read the section on 4.4.8 Codecov Enteprise-specific changes. You will very likely need to update your configuration.

v4.4.8 ChangeLog

v4.4.8 contains many small and some potentially major changes for Codecov Enterprise users. Many of the major changes impact underlying storage and minio, and provide much greater flexibility in how report storage can be accomplished.

Due to the nature of the changes and their impact, an entire section has been devoted to them after the general changelog.

General 4.4.8 Changes

New

• (Production only) Azure Pipelines now supports tokenless upload, like CircleCI, Travis, and other providers.
• (Production only) Pricing model changed from per repo to per user.
• (Production only) GitHub Student Support added. If you're a GitHub Student, you will no longer count toward the seat count of any organization of which you are a member.
• Changed the database pooled connection model to be more efficient and prevent potential deadlocking issues.
• Improved the mechanism for reporting errors that arise as a result of sync failures that can occur when syncing repositories.
• Improved Codecov's method of parent commit picking. This is being actively iterated on and will continue to improve over time.

Fixes

• Added a lock around the PR commenting task to prevent a multiple comment posting issue.
• Whitelisted additional bitbucket domains to better support bitbucket avatars in our CSP.
• Addressed some usability issues around the commits graph in the repo overview page.
• Fixed a bug that caused codecov to look into deeply nested files in the repository for a codecov.yml.
• Added a check for the conditional-coverage attribute in cobertura reports. This is particularly helpful for those users that upload cobertura reports of xcode 11 projects generated by slather.
• Fixed a bug that caused repositories not to auto activate when first visited by a user.
• Fixed a bug that caused the compare page not to load when comparing commits whose reports were uploaded via docker build scripts.
• Fixed a bug that caused Appveyeor Build URL links to be broken for some projects.
• Fixed a bug that caused repository pages not to load due to applying legacy per repo billing logic to accounts using per user billing.
• Fixed an issue where Gitlab subgroup repositories were not properly associated with users in the group.

Minor

• Removed some extemporaneous log messages.
• Numerous small copy updates.
• Updated footer links to properly navigate to the appropriate static pages.

Codecov Enterprise / Self-Hosted 4.4.8 Changes: Changes to minio

🚧

Update your Minio Container Version

4.4.8 changes were tested against minio version: minio/minio:RELEASE.2019-10-02T21-19-38Z

Earlier versions of minio may work, but it is recommended to update your minio version.

It is now possible to completely remove minio from your Codecov Enterprise infrastructure if you so desire. However, doing so comes with caveats, particularly if you're upgrading your install from an earlier version of Codecov.

If you're upgrading from an earlier version and...

You deploy everything onto a single VM using docker compose and aren't using cloud storage

You have one of two options:

1. Update your codecov.yml with the following:

services:
    minio: 
        host: <codecov-enterprise-install-url>
        port: 9000

and open port 9000 on the VM/server where you are running Codecov Enterprise. If you're running minio in your docker-compose.yml using a different port, then you should use that port number instead of 9000 in the codecov.yml.

This change ensures that the codecov bash uploader can properly resolve presigned PUTs that get generated during the process of uploading reports to codecov.

2. If 1 is not an option, you can update your codecov.yml with the following:

services:
    minio:
        host: minio
        port: 9000
        client_uploads: true

This will force the bash uploader to upload by POSTing coverage reports directly to the running web container, instead of relying on presigned PUTs directly to minio. This approach will result in more load on your web container.

You deploy using docker-compose on one or many virtual machines, but your minio server is backed by S3 or Google Cloud Storage

With 4.4.8 you are able to safely remove minio from your codecov infrastructure. To do so:

1. Edit your docker-compose.yml by removing the minio block and its contents, as well as all references to minio in depends_on lists.
2. Update your codecov.yml as follows for AWS:

services:
    minio:
        host: s3.amazonaws.com
        bucket: <bucket-name>
        region: <bucket-region>
        verify_ssl: true
        access_key_id: <aws-iam-access-key>
        secret_access_key: <aws-iam-secret>

and if Google Cloud Storage is used instead:

services:
    minio:
       host: storage.googleapis.com
       verify_ssl: true
       bucket: <bucket-name>
       region: <bucket-region>
       access_key_id: <gcs-hmac-key>
       secret_access_key: <gcs-hmac-secret>

🚧

Do not include a port number for the host if using a cloud backing store

• host: storage.googleapis.com:443 # This will cause minio to fail.
• port: 443 # This will also fail.

Setting up MinIO with Google Cloud Storage

If you already have HMAC credentials for a Google Cloud Platform service account you would like to use, you can skip this section. Otherwise, please do the following:

1. Navigate to Google Cloud Platform in your browser.
2. Confirm you are in the desired project where you want a new bucket created.
   • Codecov will use this new bucket to store processed coverage reports and raw uploaded reports. Processed reports are in a proprietary machine readable format; they do not contain code or any other sensitive information.
3. Using the side menu, navigate to Storage > Settings.
4. Navigate to the Interoperability tab.
5. Under Service account HMAC, click the Create a key for a service account button.
   • Note: Technically the HMAC credentials of a user account will work, but for IAM and audit logs purposes, a service account is recommended.
6. Within the Select a service account modal, select the service account for which you would like to create HMAC credentials.
   • Note: If you prefer to create a new service account, make sure to assign the account the Storage Admin role.
7. Click Create Key.
8. Store the generated access key and secret in a safe, encrypted place. Once you close the New service account HMAC key, you will not be able to retrieve the secret again.
9. In codecov.yml set the value of services.minio.access_key_id to the newly generated key, and set the value of services.minio.secret_access_key to the newly generated secret

You deploy in kubernetes, AWS fargate, or some other infrastructure solution that is not described here

Generally, if you're using cloud storage to back the archive, you can make the codecov.yml changes described in this article and remove minio as infrastructure.

If you're employing a different solution (such as persistent volume claims in kubernetes), you should keep your minio infrastructure intact, and update your codecov.yml as follows:

services:
    minio:
        host: <public-facing-url-to-minio-infrastructure>
        port: <minio-port> #typically 9000
        client_uploads: True #optional. Use if you cannot specify host and port.

Note that it's incredibly difficult for Codecov staff to provide documented support for all possible methods of deploying Codecov. If you're using a deployment strategy that you don't feel is accurately described here, please reach out to [email protected] for assistance.

Additional minio configuration parameters

For any deployment, the codecov.yml contains other settings to consider as they pertain to minio:

• services.minio.hash_key: unchanged, if you are currently supplying it in the codecov.yml, it should remain in 4.4.8
• services.minio.dsn: unused in 4.4.8, this can be safely removed.

Common Questions and Troubleshooting

The bash uploader fails four times then succeeds

This indicates that the presigned PUTs generated during upload. To troubleshoot, upload with the -v flag and inspect the curl output of the PUT the bash uploader performs. Ensure that the url's domain corresponds to your minio installation or cloud storage path.

You can bypass presigned PUTs by uploading with the -X s3 flag or setting services.minio.client_uploads: True in your installation's codecov.yml.

I cannot get an upload to complete properly even when using -X s3

This indicates a misconfiguration of minio. You can run the uploader with the -v flag and inspect the curl output near the end of the uploader's output for specific failure reasons.

You should double-check your minio settings in the codecov.yml and ensure the permissions of the account identified by your access_key_id and secret_access_key has the proper permissions.

If you confirm your settings are correct and are still having problems, please contact [email protected] for further assistance.

I use minio without a cloud store currently, will 4.4.8 allow me to use cloud storage?

Yes and no. You can switch to cloud storage, but you will lose all the data accumulated by minio so far. This could result in problems when viewing historical commits. It could also lead to issues on PR comparisons in the future if the BASE of the PR was uploaded to codecov before switching to cloud storage.

Generally, codecov recommends a data migration process to extract report data from local storage and upload it to a cloud provider's storage solution. If you think you may want to migrate, please contact [email protected] for assistance.