Codecov Delta

Understanding the numbers and syntax

The Codecov Delta (aka Codecov ∆) is our syntax for expressing coverage metrics and their impact on the scope of what is being measured.

Codecov calculations

IDEs and coverage collecting agents report coverage differently. Codecov does not conform to any standard, nor do we make any attempt to express our metrics in the exact same way the IDE, or coverage reports, may express coverage.

  • hit indicates that the source code was executed by the test suite.
  • partial indicates that the source code was not fully executed by the test suite; there are remaining branches that were not executed.
  • miss indicates that the source code was not executed by the test suite.

Coverage is the ratio of hits / (sum of hit + partial + miss). A code base that has 5 lines executed by tests out of 12 total lines will receive a coverage ratio of 41% (rounding down).

Codecov is a reporting tool that is intended to process any coverage report format into a format that is standard across Codecov.

You may change the configuration of how Codecov processes reports and expresses coverage information. Please review our Codecov Yaml to learn about the configuration options.

What is Codecov Delta

absolute <relative> (change)
35% <72%> (+4%)

There are three data points in the Codecov Delta:
1: The absolute scope. e.g., 35%.
2: The relative scope. e.g., <72%>.
3: The change in coverage. e.g., (+4%).

Absolute

The entire project's coverage.

Read as "The entire project coverage is 35%."

Relative

Coverage concerning only lines adjusted in the commit diff (aka the diff coverage)

Read as "The lines I changed in this commit are 72% covered."
Also read as "The coverage of this commit's diff is 72%."

Change

The amount of gain or loss of coverage when compared to the commit's parent.

Read as "This commit increased this project's coverage by 4%."

Examples

532

Project coverage is 84%.
The commit did not adjust any tracked lines.
The commit did not change the coverage ratio.

508

Project coverage is 72%.
The commit diff was 100% covered.
The commit did not change project coverage.

516

Project coverage is 60.74%.
Commit diff was not affected.
Coverage decreased by .14%.
An example of Unexpected Coverage Changes.

526

Project coverage at this commit is 61%.
Commit diff is 100% covered.
Coverage increased by 1%.

528

Project coverage is 50%.
Commit diff is 20% covered.
Coverage dropped by 3%.

514

Project is 80% covered.
Commit diff was not affected.
Coverage dropped 1%.
Example of Coverage Offset.

What is "ø"?

The character ø represents "not affected" or "no change". This value could be found in the and (change) block of the Coverage Delta.

When found in the block, it represents that the coverage diff was not concerning coverage. (e.g a commit that only adjusts lines not tracked by coverage.)

When found in the (change) block, it represents that coverage did not change. (e.g. 0% change. The commit did not affect project coverage.)

Contexts

We use the Codecov Delta throughout the app. It expresses details for the entire project, flags, diffs, folders and files. It is important to understand the context of the delta.

The absolute, relative, and change metrics refer to the context. In the example below, the Codecov Delta appears in the file headers. The Delta's metrics concern only the file itself.

2290

The file reference.go is 79% covered.
In this commit there was 12 lines added which were 100% covered.
It results in the file having 5% more coverage.