4b - Filtering Coverage by Directory

Currently, we collect coverage for each sub-directory. This is useful for projects that run test suites by directory. Codecov automatically merges reports together, but we may want to know coverage as an aggregate of sub-directories.

.
├── api
│   ├── app.py
│   ├── calculator
│   │   ├── calculator.py
│   │   └── test_calculator.py
│   └── smiles
│       ├── smiles.py
│       └── test_smiles.py
└── codecov.yml

The above shows a simplified version of the repository now. If we wanted to know the total coverage for the api/ directory, we can use Components to filter out coverage for files in that directory.

Adding Components

Components are not dependent on the CI/CD workflow and only need to be specified in your codecov.yml file. Add the following to your file.

comment:
  layout: "header, diff, flags, components"
  
component_management:
  individual_components:
    - component_id: api  # this is an identifier that should not be changed
      name: api  # this is a display name, and can be changed freely
      paths:
        - api

We are creating a Component called api that will encapsulate all of the coverage in that directory. Create a new commit and push it up to GitHub.

git add .
git commit -m 'step4: add Codecov Component'
git push origin step4

Create a new file api/smiles/smiles.py

Now create a test file api/smiles/test_smiles.py

Add a coverage step to CI/CD

Let's add a test and coverage step for the api/smiles directory. In your .github/workflows/ci.yml file, replace the file with the below code

We will now be producing two coverage reports calculator-coverage.xml and smiles-coverage.xml. Codecov will seamlessly merge those reports together. Create a pull request

After CI/CD runs, let's take a look at the PR comment.

Notice the new section Components that shows the total coverage for the api/ directory.

Now, let's take the scenario that only one test suite is run and uploaded. We may want to know the coverage for calculator and `smiles separately. Let's change the GitHub workflow to do this.

name: API workflow

on: [push, pull_request]

jobs:
  build:
    runs-on: ubuntu-latest
    name: Test python API
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-python@v2
      with:
        python-version: '3.11'
    - name: Install requirements
      run: pip install -r api/requirements.txt
    - name: Run tests and collect coverage
      run: pytest --cov
    - name: Upload coverage reports to Codecov with GitHub Action
      uses: codecov/codecov-action@v3

Notice that line 17 is running tests and collecting coverage for the entire repository. Update the codecov.yml file to have two Components.

component_management:
  individual_components:
    - component_id: api-calculator  # this is an identifier that should not be changed
      name: calculator  # this is a display name, and can be changed freely
      paths:
        - api/calculator/
    - component_id: api-smiles  # this is an identifier that should not be changed
      name: smiles  # this is a display name, and can be changed freely
      paths:
        - api/smiles/

Now create a new commit and upload to GitHub.

git add .
git commit -m 'step4: aggregate tests and split Component'
git push origin step4

Now we see that there are two Components that represent the calculator and smiles directory, respectively.

Merge in the pull request when finished.