Impact Analysis uses information obtained during your application's runtime to provide contextual information to Pull Requests. Impact Analysis currently supports:
1a. Labelling Pull Requests as Critical, such that important PRs in a list of currently open pull requests is emphasized.
1b. Labelling changed files as Critical in the Codecov PR Comment, such that important changes are the file level are obvious to the PR author and reviewer(s).
- Adding an Impacted Entrypoints table to the Codecov PR Comment, such that changes in a pull request are mapped directly to the most user facing aspect of the application
Impact Analysis was formerly marketed as Runtime Insights
Impact Analysis is available in Open Beta
Impact Analysis is currently in Open Beta (full announcement here). It is a free add-on to Codecov during Open Beta, which is expected to run through September 1st, 2022.
Currently, not every language is able to benefit from Impact Analysis. The current languages and/or frameworks are supported:
- PHP (7.4 or higher) applications using the Laravel Framework
- NodeJS (15.1 or higher)
- Python (3.6 or higher)
- Ruby (v3.1 or higher)
Looking to set up Impact Analysis?
It is most useful to select the Quick Start language from the list above in "What Languages are Supported" or in the the dropdown on the left under "Impact Analysis"
Below is a set of general needs of Impact Analysis in any language
In order to function, Impact Analysis requires the following:
- A profiling token used to identify your repository and authenticate API requests. You can find your token in your repo settings page:
A repository using one of our supported languages that is currently running in a production context.
Varying dependencies that are determined by the language and/or framework being used.
Specific installation instructions will be provided for your language of choice upon acceptance into the Early Access program.
To see Impacted Entrypoints in your pull request comments, "betaprofiling" must be added to the
comment.layout section of your codecov.yml, as follows:
comment: layout: "reach,diff,flags,tree,betaprofiling"
If you wish to add "Critical" markers to impacted files, you must update your codecov.yml as follows:
comment: show_critical_paths: true
These settings are also supported at the Global YAML level.
A Critical Change is any code change in a Pull Request that impacts code that is frequently executed in a production/runtime context. More specifically, if a line of code executed more than <> the average of all executed lines is edited, it is considered a Critical Change.
In order to use Impact Analysis you must have running production infrastructure to send Open Telemetry spans to Codecov. So, for example, an open source library that others ingest as a dependency would not make sense here.
However, on the back of this feature, we are working on something for open source maintainers, so hold tight.
Impact Analysis was built using the emerging Open Telemetry standard. It’s facilitated via a runtime dependency of Open Telemetry. That dependency is added directly to your code, similar to a tool like Sentry, Segment, or Raygun.
Therefore, the security surface area of Impact Analysis is the same as using Open Telemetry generally.
Once deployed, the Impact Analysis dependency sends to Codecov: lines of source executed by users, including file path, file name, line number, and execution count (but not actual source code, similar to a coverage report). In the case of HTTP requests, the request route and HTTP verb.
For Codecov SaaS customers, in the same GCP environment alongside code coverage data uploaded by customers.
Yes, in the same fashion that customer’s code coverage data may be requested for deletion.
Currently customers in open beta for Impact Analysis do not have the ability to download span data that has been uploaded. Allowing for downloading will be a potential feature in the future.
In order to use Impact Analysis, an Impact Analysis consumer library must be installed as a production-level dependency along with any required third-party dependencies required by the library. Specific dependencies vary based on the language of the Impact Analysis library in use; however, key requirements are specified in a dependency manifest file based on the language of the Open Telemetry instrumentation. An exception to this is the PHP consumer library’s requirement of PCOV which must be installed independently.
Updated about 1 month ago