LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage#411
Closed
Janell-Huyck wants to merge 6 commits intoqafrom
Closed
LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage#411Janell-Huyck wants to merge 6 commits intoqafrom
Janell-Huyck wants to merge 6 commits intoqafrom
Conversation
- SimpleCov: enable branch coverage (enable_coverage :branch) - GitHub Actions: set CI=true, path-to-lcov, format lcov; single source for coverage - .coveralls.yml: service_name github (was circleci) - CircleCI: remove coveralls orb, upload step, and COVERALLS_PARALLEL - README: document that coverage is uploaded from CI; use bundle exec rspec/rubocop Co-authored-by: Cursor <cursoragent@cursor.com>
- CI: use MultiFormatter (Lcov + HTML) so lcov.info and index.html are both written - Local: set HTMLFormatter explicitly so index.html is always produced Co-authored-by: Cursor <cursoragent@cursor.com>
Remove branch coverage so Coveralls compares apples-to-apples with qa; branch coverage was showing ~3% lower than qa's line-only metric. Co-authored-by: Cursor <cursoragent@cursor.com>
Remove accidentally committed test DB files from the repo; they remain on disk and are ignored by .gitignore. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Author
|
turns out the problem was that "branch coverage" drops the coverage % by 3%. Fixed in other PR and we're good. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes inconsistent Coveralls results and aligns coverage reporting so that only GitHub Actions reports to Coveralls, with an explicit LCOV path/format. We report line coverage only so the badge compares fairly to the qa branch baseline.
Why results were inconsistent
Wrong service name
.coveralls.ymlhadservice_name: circleciwhile we run CI on GitHub Actions. Coveralls was attributing builds to the wrong service, which can change how it matches jobs, branches, and “latest” coverage and lead to fluctuating or misleading numbers.No explicit path or format for the coverage file
The Coveralls GitHub Action was not told where to find the LCOV file or that it was LCOV. It had to guess, and path resolution (e.g. relative vs workspace) can vary. That could cause some runs to find and upload the file and others to miss it or use the wrong format, producing inconsistent coverage.
Unreliable LCOV generation in CI
SimpleCov only writes
coverage/lcov.infowhenENV['CI']is set. We weren’t settingCIexplicitly in the workflow, so we depended on GitHub’s default. Making it explicit (CI: truein the test step) guarantees the LCOV file is always written in CI.Two CI systems both trying to report coverage
CircleCI still had a Coveralls orb and an upload step pointing at a path we don’t use (
coverage/lcov/application_portfolio.lcov). Our SimpleCov config writes a single file atcoverage/lcov.info, so that CircleCI step could never find the right file. Having two CIs (only one of which we actually use for coverage) and a wrong path added noise and inconsistency.Comparing different metrics
We briefly enabled branch coverage, which reports a stricter (lower) percentage than line coverage. That made this branch look like a ~3% drop vs qa, which reports line coverage. We reverted to line coverage only so the badge compares apples-to-apples with qa.
Changes made
spec/spec_helper.rb)lcov.infofor Coveralls andcoverage/index.htmlfor viewing; locally set HTMLFormatter explicitly soindex.htmlis always generated..github/workflows/main.yml)CI: truein the “Run tests” step so SimpleCov always writes LCOV in CI. Gave the Coveralls step explicitpath-to-lcov: ${{ github.workspace }}/coverage/lcov.infoandformat: lcov.service_namefromcirclecitogithubso builds are attributed to GitHub (Actions)..circleci/config.yml)COVERALLS_PARALLEL, and thecoveralls/uploadstep. Coverage is now reported only from GitHub Actions; CircleCI no longer uploads to Coveralls.bundle exec rspecandbundle exec rubocop -a(removed the obsoletecoveralls reportreference).Result
github, so branch/PR and “latest” coverage should be consistent.coverage/index.htmlis always generated (in CI and locally).After merging, coverage on Coveralls should stabilize and reflect the GitHub Actions test run for each push.