Troubleshooting Automated Test Selection
Common errors when setting up ATS and what to do about them
Got a 4XX status code back from Codecov
The Codecov-api couldn't find some important info related to the information you passed.
Check the status_code
and response_json
fields in the logs for further info.
Commit SHORT_SHA not found.
Example
warning - 2023-09-19 16:05:30,761 -- Got a 4XX status code back from Codecov --- {"status_code": 404, "response_json": {"detail": "Commit 5414191 not found."}}
Codecov could not find the commit associated with the request.
Check that the correct SHA is being passed. For label-analysis make sure to also verify the --base-sha
. Also make sure you call create-commit
for the commit that was not found.
"head_commit": ["No static analysis found"]
[label-analysis context] Codecov could not find static analysis information for the HEAD commit. Make sure to call static-analysis
before calling label-analysis
.
"base_commit": No possible commits have static analysis sent.
[label-analysis context] Codecov could not find a commit (either BASE specified or its parents) that have static analysis information). Make sure to call static-analysis
for the commit that you use as --base-sha
.
"processing_errors"
These are errors that typically didn't prevent ATS from running, but are an indication that we couldn't properly calculate the results.
They can be found in the logs of the CLI, in a line that reads "Received list of tests from Codecov"
# Example with no processing errors
info - 2023-09-19 17:00:53,299 -- Received list of tests from Codecov --- {"processing_errors": []}
# Example with processing errors
info - 2023-09-19 17:00:53,299 -- Received list of tests from Codecov --- {"processing_errors": [{"error_code": "missing data", "error_params": {"extra": {"base_commit": "541419157ae7af0f43b4b08b9c830466636ea031", "head_commit": "22c874ab0ec4cdb2a2e00a34698c100def7bb22f"}]}
"message": "Missing base report"
This indicates that Codecov didn't find a successfully processed coverage report for your BASE commit.
In the error message you can find "base_commit": "LONG_SHA"
. Make sure this value matches the SHA for your BASE commit.
If the commit is correct it can be that you don't have any successfully processed coverage reports for the commit. You can verify by looking up the commit on Codecov and seeing if a report was processed. Here's an example of a commit that did not.
Notice that running create-report
for a commit is not the same as having an upload for it. An upload is typically done using the CLI do-upload
command, or (more commonly) the Codecov Github action (or similar action to other providers).
This error is expected if you never used Codecov before. It will fallback to running all tests in your test suite. However if the error persists, make sure that you are uploading coverage info for Codecov after running label-analysis
.
Updated about 1 year ago