Ruby Quick Start [Impact Analysis]

This guide is meant to get you started using Impact Analysis with Codecov as quickly as possible on ruby projects

Prerequisites

In order to get started, you will need a repository that is

1. using ruby 3.1+
2. already integrated with Codecov
3. running in a production-like environment
4. generating telemetry data via OpenTelemetry. If your repository is not yet doing this, we recommend checking out the Open Telemetry ruby repository for details on how to get started first.

Getting Started

Step 1: Get your Impact analysis token

In your repository settings page on Codecov, copy the Impact analysis token to be used when uploading telemetry reports

Step 2: Update your dependencies

Add the following dependency by running

gem install codecov_opentelem

Be sure to save down the dependencies to Gemfile or similar.

Step 3: Add in the Codecov OpenTelemetry package

Wherever you currently instantiate your open-telemetry instance, add the following lines of code

require 'codecov_opentelem'

... 

OpenTelemetry::SDK.configure do |c|
    ...
    
    current_env = ENV['CODECOV_OPENTELEMETRY_ENV'] || 'production'
    current_version = ENV['CURRENT_VERSION'] || '0.0.1'
    # export_rate: Percentage of spans that are sampled with execution data.
    # Note that sampling execution data does incur some performance penalty,
    # so 10% is recommended for most services.
    # Values should be between 0 and 1.
    export_rate = 1
  
    # untracked_export_rate: Currently unused.
    # Percentage of spans that are sampled without execution data. 
    # These spans incur a much smaller performance penalty, but do not provide
    # as robust a data set to Codecov, resulting in some functionality being
    # limited.
    # Values should be between 0 and 1.
    untracked_export_rate = 0

    generator, exporter = get_codecov_opentelemetry_instances(
        repository_token: ENV['CODECOV_OPENTELEMETRY_TOKEN'],
        sample_rate: export_rate,
        untracked_export_rate: untracked_export_rate,
        code: "#{current_version}:#{current_env}",
        filters: {
            'file_ignore_regex'=>/\/gems\//,
        },
        version_identifier: current_version,
        environment: current_env,
    )

    c.add_span_processor(generator)
    c.add_span_processor(OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(exporter))
end

Configuration Details

If desired, the filters parameter can also be changed to provide different filtering on any valid OpenTelemetry SpanKind as defined by the specification.

The example shows a filter preventing any information about files with /gems/ in their pathname to be exported to Codecov. This helps prevent unnecessary details about third-party gems from being uploaded.

Step 4: Update the codecov.yaml configuration

In order to get Impact Analysis data in your pull requests, add the following lines of code to your codecov.yaml file

comment:
  layout: "diff,flags,tree,betaprofiling"
  show_critical_paths: true

Commit the above code and run your code in production as usual. Note that it may take 30 minutes for the changes to propagate.

Seeing Impact Analysis in your pull requests

After you have followed the steps above, you will need to run the code in a production-like environment. To see changes in your pull requests, make a code change on

You should be able to see two differences. The first notates Critical on files that have code run frequently in production that are also being changed. The second shows API endpoints that are affected by the pull request.

If you are making changes to code that is not deemed critical, you should still see the following in your pull request comment from Codecov.

Integration Examples

The specifics of how this library is integrated into your project depend on the project itself. This section contains a few common, framework-specific, integration approaches along with the general integration approach at the end.

Note that these examples demonstrate possible ways to incorporate this package into your project. As always, your specific needs may vary.

Rails

In a Rails application, you could place the code snippet in your application's config/initializers/opentelemetry.rb file, like so:

require 'opentelemetry/sdk'
require 'opentelemetry/exporter/otlp'
require 'opentelemetry/instrumentation/all'
require 'codecov_opentelem'

OpenTelemetry::SDK.configure do |c|
    c.service_name = 'Codecov'
    c.use_all() # enables all instrumentation!

    current_env = ENV['CODECOV_OPENTELEMETRY_ENV'] || 'production'
    current_version = ENV['CURRENT_VERSION'] || '0.0.11'
    export_rate = 100
    untracked_export_rate = 0

    generator, exporter = get_codecov_opentelemetry_instances(
        repository_token: ENV['CODECOV_OPENTELEMETRY_TOKEN'],
        sample_rate: export_rate,
        untracked_export_rate: untracked_export_rate,
        code: "#{current_version}:#{current_env}",
        filters: {
            'file_ignore_regex'=>/\/gems\//,
        },
        version_identifier: current_version,
        environment: current_env,
    )

    c.add_span_processor(generator)
    c.add_span_processor(OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(exporter))
end

General Integration

If you are not using Rails, integration is still possible by using the above code snippet. How to do this may vary greatly depending on your use case. In general, though, the code snippet should be placed wherever you would put your application's OpenTelemetry startup code. In lieu of that, this code should be incorporated in such a way that it is part of your application's startup process.

Troubleshooting

Not seeing critical changes? You may need to add path fixes to your codecov.yml configuration.

profiling:
  fixes:
    - "before/::after/"  # move path   e.g., "before/path" => "after/path"
    - "::after/"         # move root   e.g., "path/" => "after/path/"
    - "before/::"        # reduce root e.g., "before/path/" => "path/"

You can also check out our example repository to see a complete setup.