Removed Code Behavior

How removed code behavior works:

Removed code behavior (RCB) is designed to help maintain a passing CI status so that developers can streamline their codebase without worrying about losing code coverage. In the development practice, code removal can be a good thing, however, removing entire lines can reduce the coverage percentage (which is affected if the ratio of total lines to total covered lines changes).

A few more things to note:

• RCB applies to project coverage only. It doesn’t impact patch coverage.
• RCB is effective only when the Target is set to auto.
• RCB has 4 behaviors:
  • fully_covered_patch
  • removals_only
  • adjust_base (default behavior)
  • off

removed_code_behavior motivating example

Initial scenario:

• Total lines in a project: 10
• Covered lines in a project: 9

Action taken:

Developer removes 3 covered lines and adds 1 covered line. All changes are in a pull request.

• Total lines in the project: 8
• Covered lines in the project: 7 (3 covered lines removed, 1 covered line added)

Project coverage calculation:

• The project coverage percentage is calculated as a ratio of covered lines to total lines in the coverage files. It calculates the percentage of lines covered by the combined test suite.

Initial calculation:

After removing 3 covered lines and adding 1 covered line:

removed_code_behavior options (with mathematical examples)

Option 1: fully_covered_patch

• If the patch coverage is 100% and there are no unexpected changes, the project status should be: pass.

• Indirect changes might be caused by modifications to both new and existing tests, covered and uncovered tests, processing errors, CI errors, and so on.

  Patch coverage	Indirect changes	CI status	Scenario
  100%	False	Pass status	If the patch coverage % either increases, decreases, or remains equal to 100% with no indirect changes, the project status will pass.
  100%	True	Fail status	If the patch coverage % remains at 100% with direct changes, the project status will fail.
  < 100%	True	Fail status	If the patch coverage % either decreases or is less than 100% with indirect changes, the project status will fail.
  < 100%	False	Fail status	If the patch coverage % either decreases or is less than 100% with no indirect changes, the project status will fail.

When looking at the motivating example scenario, the project status check will always pass because the patch (the 1 line that was added) is 100% covered.

Option 2: removals_only

• If changes are due to removed lines of code (i.e. no additions, no unexpected changes), the project status should be: pass

When looking at the motivating example scenario, if status threshold is default/not set or automatic, the project status check will fail because 1 or more line has been added and coverage has decreased from 90% to 87.5%, a reduction of 2.5%.

Option 3: (Default) adjust_base

• Recalculates the base to filter out any code removals. The recalculation removes misses, hits, and partials from lines removed in the diff of the base and uses that to calculate a comparison.

When looking at the motivating example scenario, if status threshold is default/not set or automatic, the project status check will pass because the project coverage has increased from an  adjusted_base of 85.7% to 87.5% (7/8 covered lines), an increase of 1.8%.

Option 4: False or off

• This won’t add any special behavior to project statuses;
  When looking at the motivating example scenario, if status threshold is default/not set or automatic, the project status check will fail because project coverage has gone down from 90% to 87.5%, a reduction of 2.5%.
• Project coverage decreases will always result in a failed status check in this case.