Flags allow you to isolate coverage reports for different tests. This is particularly helpful if you have multiple types of tests (e.g., unit, integration, frontend, backend, etc), or you're employing a monorepo setup where you'd like to encapsulate each project's test coverage independently.
For coverage information to be properly tracked per flag, the flags must be applied at upload time.
To apply a flag simply submit coverage reports with the
-F flagname included in the upload command; Codecov will do the rest.
The following example illustrates how you may run specific test groups and upload coverage, flagging each coverage group. This strategy for uploading is preferred as it cleanly associates a single flag with a single report.
# example running unittests only py.test --cov=./ -k tests/unittests/ bash <(curl -s https://codecov.io/bash) -c -F unittests # example running integration tests only py.test --cov=./ -k tests/integration/ bash <(curl -s https://codecov.io/bash) -c -F integration # example running ui tests only npm test bash <(curl -s https://codecov.io/bash) -c -F ui
Remove old reports after uploading (optional)
-c argument to clear the workspace of all coverage reports before running you next set of tests.
Flags must be lowercase, alphanumeric, and not exceed 45 characters
Only lowercase, alphanumeric values are accepted.
Once properly configured, you can toggle flags in the source code overlay to see how tests using different flags interact with your source code and impact coverage.
Source code overlay Toggle
ui coverage reports with one tap.
Navigate with specific flags visible.
Badges and graphs may express specific flags only. More details on this will come soon.
Flags can be used to create custom notifications to your repository provider. This makes it easy to see per-flag converage information alongside pull requests.
You can specify Flags in your Codecov Yaml for statuses and all notifications.
coverage status project defaultoff frontend flagsfrontend backend target50% flagsbackend api target89% flagsapi flags # filter the folder(s) you wish to measure by that flag backend # only include files in the backend folder paths app/backend/ frontend paths app/frontend/ api paths app/api/ tests paths tests/
Custom Statuses only report on specific Flags.
Codecov provides a strategy to isolate specific builds from the master report while maintaining the report's integrity. When reports are not joined into the master report, they will be ignored for comparison, although they will remain accessible for source overlay, api, badges, and graphing.
A nightly build is an excellent example of this feature. What follows is a yaml configuration for a nightly build.
flags nightly joinedfalse
Now that we have configured the flag
nightly to not join into the master report, let's upload a report flagged as
bash <(curl -s https://codecov.io/bash) -F nightly
-F nightly we mark all the coverage report data for this build as
nightly coverage data.
Flags feature on the Codecov Compare page
As of June 2019, Codecov has suspended flags for Codecov.io users on the Codecov Compare page.
Flags were causing the compare page to consistently timeout for a minority of users. The Codecov team is investigating refactoring flags to make the compare page more usable, even with many flags turned on.
The compare page will still work for comparing changed code
The flags feature will still be viewable on each individual commit view
Flags will still be functional on pull and merge request status checks
We will continue to iterate on this feature and push out updates here.