Skip to content

LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage#411

Closed
Janell-Huyck wants to merge 6 commits intoqafrom
LIBAPPPORT-84-coveralls-not-being-consistent
Closed

LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage#411
Janell-Huyck wants to merge 6 commits intoqafrom
LIBAPPPORT-84-coveralls-not-being-consistent

Conversation

@Janell-Huyck
Copy link
Contributor

@Janell-Huyck Janell-Huyck commented Feb 13, 2026

Summary

This PR fixes inconsistent Coveralls results and aligns coverage reporting so that only GitHub Actions reports to Coveralls, with an explicit LCOV path/format. We report line coverage only so the badge compares fairly to the qa branch baseline.


Why results were inconsistent

  1. Wrong service name
    .coveralls.yml had service_name: circleci while we run CI on GitHub Actions. Coveralls was attributing builds to the wrong service, which can change how it matches jobs, branches, and “latest” coverage and lead to fluctuating or misleading numbers.

  2. No explicit path or format for the coverage file
    The Coveralls GitHub Action was not told where to find the LCOV file or that it was LCOV. It had to guess, and path resolution (e.g. relative vs workspace) can vary. That could cause some runs to find and upload the file and others to miss it or use the wrong format, producing inconsistent coverage.

  3. Unreliable LCOV generation in CI
    SimpleCov only writes coverage/lcov.info when ENV['CI'] is set. We weren’t setting CI explicitly in the workflow, so we depended on GitHub’s default. Making it explicit (CI: true in the test step) guarantees the LCOV file is always written in CI.

  4. Two CI systems both trying to report coverage
    CircleCI still had a Coveralls orb and an upload step pointing at a path we don’t use (coverage/lcov/application_portfolio.lcov). Our SimpleCov config writes a single file at coverage/lcov.info, so that CircleCI step could never find the right file. Having two CIs (only one of which we actually use for coverage) and a wrong path added noise and inconsistency.

  5. Comparing different metrics
    We briefly enabled branch coverage, which reports a stricter (lower) percentage than line coverage. That made this branch look like a ~3% drop vs qa, which reports line coverage. We reverted to line coverage only so the badge compares apples-to-apples with qa.


Changes made

Area Change
SimpleCov (spec/spec_helper.rb) Report line coverage only (no branch coverage) so Coveralls matches the qa baseline. In CI use MultiFormatter (LCOV + HTML) so we get both lcov.info for Coveralls and coverage/index.html for viewing; locally set HTMLFormatter explicitly so index.html is always generated.
GitHub Actions (.github/workflows/main.yml) Set CI: true in the “Run tests” step so SimpleCov always writes LCOV in CI. Gave the Coveralls step explicit path-to-lcov: ${{ github.workspace }}/coverage/lcov.info and format: lcov.
.coveralls.yml Updated service_name from circleci to github so builds are attributed to GitHub (Actions).
CircleCI (.circleci/config.yml) Removed the Coveralls orb, COVERALLS_PARALLEL, and the coveralls/upload step. Coverage is now reported only from GitHub Actions; CircleCI no longer uploads to Coveralls.
README Clarified that coverage is uploaded automatically from GitHub Actions and updated local instructions to bundle exec rspec and bundle exec rubocop -a (removed the obsolete coveralls report reference).

Result

  • Single source of truth: Only GitHub Actions reports to Coveralls.
  • Stable upload: Explicit LCOV path and format so the action always finds and sends the same file.
  • Correct service: Coveralls sees builds as github, so branch/PR and “latest” coverage should be consistent.
  • Comparable to qa: Line coverage only, so the badge is comparable to the qa branch baseline (no false “drop” from branch coverage).
  • Local HTML report: coverage/index.html is always generated (in CI and locally).

After merging, coverage on Coveralls should stabilize and reflect the GitHub Actions test run for each push.

Janell-Huyck and others added 2 commits February 13, 2026 15:51
- SimpleCov: enable branch coverage (enable_coverage :branch)
- GitHub Actions: set CI=true, path-to-lcov, format lcov; single source for coverage
- .coveralls.yml: service_name github (was circleci)
- CircleCI: remove coveralls orb, upload step, and COVERALLS_PARALLEL
- README: document that coverage is uploaded from CI; use bundle exec rspec/rubocop

Co-authored-by: Cursor <cursoragent@cursor.com>
- CI: use MultiFormatter (Lcov + HTML) so lcov.info and index.html are both written
- Local: set HTMLFormatter explicitly so index.html is always produced

Co-authored-by: Cursor <cursoragent@cursor.com>
@Janell-Huyck Janell-Huyck changed the title LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage WIP LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage Feb 13, 2026
Janell-Huyck and others added 4 commits February 13, 2026 16:14
Remove branch coverage so Coveralls compares apples-to-apples with qa;
branch coverage was showing ~3% lower than qa's line-only metric.

Co-authored-by: Cursor <cursoragent@cursor.com>
Remove accidentally committed test DB files from the repo; they remain
on disk and are ignored by .gitignore.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Janell-Huyck Janell-Huyck changed the title WIP LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage LIBAPPPORT-84 Fix Coveralls consistency and add branch coverage Feb 13, 2026
@Janell-Huyck
Copy link
Contributor Author

turns out the problem was that "branch coverage" drops the coverage % by 3%. Fixed in other PR and we're good.

@Janell-Huyck Janell-Huyck deleted the LIBAPPPORT-84-coveralls-not-being-consistent branch February 13, 2026 21:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments