split lcov and genhtml cmake opts for the Coverage profile by tevio · Pull Request #224 · bitcoinknots/bitcoin

tevio · 2025-10-16T01:17:01Z

Summary

Introduce HTML_OPTS for genhtml with a sensible default: inherit LCOV_OPTS when HTML_OPTS is not provided. This removes duplication for shared settings while keeping a clear escape hatch for genhtml-specific control.

Background (previous behavior)

The HTML report generator (genhtml) always received LCOV_OPTS.
This worked well for shared runtime config (notably --rc flags like branch_coverage=1) but made it awkward to pass options that apply only to genhtml.

What’s changing

Add a dedicated variable, HTML_OPTS, for genhtml.
Default behavior: if HTML_OPTS is omitted, pass LCOV_OPTS to genhtml (maintains the historical behavior and keeps shared config in sync).
Override behavior: if HTML_OPTS is provided, use it instead of LCOV_OPTS for genhtml.
Empty override: if HTML_OPTS is explicitly set to an empty string, pass no extra options to genhtml and do not inherit LCOV_OPTS.

Why this change

lcov and genhtml are separate tools with partially overlapping options.
Many useful settings are shared (e.g., --rc), but some are genhtml-only (e.g., --title).
This change avoids duplicating shared options while enabling precise control of genhtml when needed.

User-facing behavior

Omitted: genhtml receives LCOV_OPTS automatically.
Provided: genhtml receives HTML_OPTS instead of LCOV_OPTS.
Explicitly empty: genhtml receives no options; nothing is inherited.

Examples

# Single place to enable branch coverage (LCOV 2.x) for both tools
cmake -DLCOV_OPTS="--rc branch_coverage=1" -P build/Coverage.cmake

# genhtml-only customization (does not affect lcov)
cmake -DHTML_OPTS="--title 'Knots Coverage'" -P build/Coverage.cmake

# Intentionally pass no genhtml options (do not inherit LCOV_OPTS)
cmake -DHTML_OPTS="" -P build/Coverage.cmake

Implementation notes

In cmake/script/CoverageInclude.cmake.in:
- Detect whether HTML_OPTS was defined (even if empty) before splitting.
- Set GENHTML_OPTS to HTML_OPTS when defined; otherwise fall back to LCOV_OPTS.
- Build GENHTML_COMMAND with GENHTML_OPTS.

Key snippet:

# HTML_OPTS is optionally passed via -D flag.
# If HTML_OPTS is not provided at all, implicitly pass LCOV_OPTS to genhtml.
# If HTML_OPTS is provided but empty (e.g. -DHTML_OPTS=""), use an empty list
# and do NOT inherit LCOV_OPTS.
set(_HTML_OPTS_WAS_DEFINED FALSE)
if(DEFINED HTML_OPTS)
  set(_HTML_OPTS_WAS_DEFINED TRUE)
endif()
separate_arguments(HTML_OPTS)
if(_HTML_OPTS_WAS_DEFINED)
  set(GENHTML_OPTS ${HTML_OPTS})
else()
  set(GENHTML_OPTS ${LCOV_OPTS})
endif()
set(GENHTML_COMMAND ${GENHTML_EXECUTABLE} --show-details ${GENHTML_OPTS})

Backward compatibility

Matches historical default: when HTML_OPTS is omitted, genhtml still receives LCOV_OPTS.
Existing setups that already provided HTML_OPTS continue to behave as before.
Adds a new, explicit way to pass no genhtml options via -DHTML_OPTS="".

Documentation

Updated doc/developer-notes.md to describe the default inheritance, the explicit override, and the empty override, with examples using a genhtml-only flag (--title).

Impact and risk

Low risk, confined to coverage tooling. Improves consistency for shared settings while enabling genhtml-specific customization and intentional minimalism.

kwsantiago

I recommend moving lines 5-17 to after the find_program() calls (after line 40) to ensure variables exist before use.

cmake/script/CoverageInclude.cmake.in

build the Coverage profile, run the tests and generate the lcov html reports with: cmake -S . -B ${PWD}/cmake-build-debug -DCMAKE_BUILD_TYPE=Coverage cmake --build ${PWD}/cmake-build-debug LCOV_OPTS="--ignore-errors inconsistent,gcov,source,format --base-directory ${PWD}/src --rc geninfo_auto_base=1" GENHTML_OPTS="--ignore-errors unsupported,inconsistent,count,category --filter missing" cmake -P ${PWD}/cmake-build-debug/Coverage.cmake

kwsantiago

Previously genhtml incorrectly received LCOV_OPTS

Now that they are properly separated, you may want to document the behavior change to confirm no one was relying on passing genhtml options via LCOV_OPTS

Also may want to rename HTML_OPTS to GENHTML_OPTS throughout for consistency and clarity.

doc/developer-notes.md

cmake/script/CoverageInclude.cmake.in

tevio · 2025-10-20T21:39:00Z

Previously genhtml incorrectly received LCOV_OPTS

Now that they are properly separated, you may want to document the behavior change to confirm no one was relying on passing genhtml options via LCOV_OPTS

Also may want to rename HTML_OPTS to GENHTML_OPTS throughout for consistency and clarity.

hi @kwsantiago yes I was thinking about that yesterday in the point you make about confirming no one was relying on passing LCOV_OPTS - haven't tried it yet but could solve it by falling back to LCOV_OPTS by default and then only overriding with HTML_OPTS if it's passed and that way no ones existing setups would be affected

…option (not envvar), update doc ref

cmake/script/CoverageInclude.cmake.in

kwsantiago · 2025-10-23T12:42:48Z

doc/developer-notes.md

 ```

+HTML_OPTS can be specified to provide an options override to genhtml from the default LCOV_OPTS, the program that generates the
+html report from lcov: `HTML_OPTS="--exclude boost"`.


Have you tested this?

From my understanding, --exclude boost is a lcov option not a genhtml option, so this example will fail.

it is an option for both lcov and genhtml - so I've used it as a reference, certainly tested it to filter out the boost entries in the coverage report, however, because there is currently no genhtml distribution in homebrew you need to install a custom binary which may or may not support the same options. In my case the version I'm using doesn't support the same options.

might be better to provide a reference in the docs for a compatible genhtml implementation that would officially "work" on macOs/windows (if it exists) that would render this OPTS overriding change irrelevant, but I don't know what that is or would look like currently

doc/developer-notes.md

kwsantiago

Looks good but the _HTML_OPTS_WAS_DEFINED pattern seems overly complex.

kwsantiago · 2025-10-27T17:54:21Z

cmake/script/CoverageInclude.cmake.in

+# If HTML_OPTS is not provided at all, implicitly pass LCOV_OPTS to genhtml.
+# If HTML_OPTS is provided but empty (e.g. -DHTML_OPTS=""), use an empty list
+# and do NOT inherit LCOV_OPTS.
+set(_HTML_OPTS_WAS_DEFINED FALSE)


Why track _HTML_OPTS_WAS_DEFINED separately instead of checking DEFINED HTML_OPTS directly in line 37?

@kwsantiago I've tried removing that and it doesn't cause an obvious issue but because separate_arguments(HTML_OPTS) (called earlier) creates/modifies the HTML_OPTS variable, if you checked if(DEFINED HTML_OPTS) after that, it would always be true, losing the ability to distinguish “user didn’t provide HTML_OPTS” from “user provided it (possibly empty)”. The _HTML_OPTS_WAS_DEFINED flag captured whether it was defined before separate_arguments() ran.

this is probably the cleanest way, setting the default to LCOV_OPTS, but I need to test:

# HTML_OPTS is optionally passed via -D flag. -# If HTML_OPTS is not provided at all, implicitly pass LCOV_OPTS to genhtml. -# If HTML_OPTS is provided but empty (e.g. -DHTML_OPTS=""), use an empty list -# and do NOT inherit LCOV_OPTS. -set(_HTML_OPTS_WAS_DEFINED FALSE) +# Default: inherit LCOV_OPTS. If HTML_OPTS is provided (even if empty), use it instead. +set(GENHTML_OPTS ${LCOV_OPTS}) if(DEFINED HTML_OPTS) - set(_HTML_OPTS_WAS_DEFINED TRUE) -endif() -separate_arguments(HTML_OPTS) -if(_HTML_OPTS_WAS_DEFINED) + separate_arguments(HTML_OPTS) set(GENHTML_OPTS ${HTML_OPTS}) -else() - set(GENHTML_OPTS ${LCOV_OPTS}) endif() set(GENHTML_COMMAND ${GENHTML_EXECUTABLE} --show-details ${GENHTML_OPTS})

@kwsantiago with regards to this change b581b44 I've tested this locally and for brevity just summarised the output here: tl;dr: each mode renders a different result on my machine and the only one that succeeds is when passing the specific HTML_OPTS (not empty)

The three modes:

1) pass specific HTML_OPTS

rm -rf build cmake -S . -B build -DCMAKE_BUILD_TYPE=Coverage cmake --build build

TMPDIR=/tmp LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 cmake -DLCOV_OPTS="--ignore-errors inconsistent,gcov,source,format --base-directory $(pwd -P)/src --rc geninfo_auto_base=1" -DHTML_OPTS="--ignore-errors unsupported,inconsistent,count,category --filter missing" -P build/Coverage.cmake

should complete and generate coverage report:

Overall coverage rate:
source files: 762
lines.......: 87.5% (101483 of 115966 lines)
functions...: 78.2% (33941 of 43392 functions)
Message summary:
200 warning messages:
category: 100
inconsistent: 100
1361 ignore messages:
category: 351
count: 2
inconsistent: 1007
unsupported: 1
➜ bitcoin-knots git:(rcov-and-genhtml)

2) pass empty HTML_OPTS

TMPDIR=/tmp LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 cmake -DLCOV_OPTS="--ignore-errors inconsistent,gcov,source,format --base-directory $(pwd -P)/src --rc geninfo_auto_base=1" -DHTML_OPTS="" -P build/Coverage.cmake

was previously untested but now fails with a corrupt trace error:

Reading tracefile test_bitcoin_coverage.info.
genhtml: ERROR: (corrupt) unable to read trace file 'test_bitcoin_coverage.info': genhtml: ERROR: (inconsistent) "./bitcoin-knots/src/wallet/coincontrol.h":27: function '_ZN6wallet16PreselectedInputC1Ev' is hit but no contained lines are hit.
To skip consistency checks, see the 'check_data_consistency' section in man lcovrc(5).
(use "genhtml --ignore-errors inconsistent ..." to bypass this error)
(use "genhtml --ignore-errors corrupt ..." to bypass this error)
CMake Error at build/Coverage.cmake:46 (execute_process):
execute_process failed command indexes:
1: "Child return code: 1"

2) Do not pass HTML_OPTS (fall through to LCOV_OPTS)

TMPDIR=/tmp LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 cmake -DLCOV_OPTS="--ignore-errors inconsistent,gcov,source,format --base-directory $(pwd -P)/src --rc geninfo_auto_base=1" -P build/Coverage.cmake

should fail as it did, with unknown options:

lcov: WARNING: (count) max_message_count=100 reached for 'inconsistent' messages: no more will be reported.
To increase or decrease this limit use '--rc max_message_count=value'.
(use "lcov --ignore-errors count,count ..." to suppress this warning)
Combining tracefiles.
.. found 2 files to aggregate.
Combining tracefiles.
Merging baseline_filtered.info..1 remaining
Merging test_bitcoin_filtered.info..0 remaining
Writing data to test_bitcoin_coverage.info
Summary coverage rate:
source files: 762
lines.......: 69.4% (80524 of 115966 lines)
functions...: 70.1% (30439 of 43392 functions)
Message summary:
102 warning messages:
count: 1
inconsistent: 100
unsupported: 1
1544 ignore messages:
inconsistent: 1544
genhtml: WARNING: Unknown option: base-directory
Use genhtml --help to get usage information
CMake Error at build/Coverage.cmake:46 (execute_process):
execute_process failed command indexes: 1: "Child return code: 1"

kwsantiago · 2025-10-27T17:54:46Z

cmake/script/CoverageInclude.cmake.in

+if(DEFINED HTML_OPTS)
+  set(_HTML_OPTS_WAS_DEFINED TRUE)
+endif()
 separate_arguments(HTML_OPTS)


Should separate_arguments(HTML_OPTS) be called after checking if it's defined to avoid modifying an undefined variable?

kwsantiago · 2025-10-27T17:55:11Z

doc/developer-notes.md


-HTML_OPTS can be specified to provide an options override to genhtml from the default LCOV_OPTS, the program that generates the
-html report from lcov: `HTML_OPTS="--exclude boost"`.
+HTML_OPTS can override the genhtml options (which default to LCOV_OPTS). If HTML_OPTS is omitted, LCOV_OPTS are implicitly passed to genhtml. If HTML_OPTS is provided but empty (e.g. -DHTML_OPTS=""), no options are passed to genhtml and LCOV_OPTS are not inherited.


May want to add an example showing combined use of both LCOV_OPTS and HTML_OPTS to clarify precedence.

luke-jr · 2025-11-04T00:03:03Z

Is there a reason not to submit this through bitcoin/bitcoin?

kwsantiago · 2025-11-06T21:36:46Z

cmake/script/CoverageInclude.cmake.in

+
+# HTML_OPTS is optionally passed via -D flag.
+# Default: inherit LCOV_OPTS. If HTML_OPTS is provided (even if empty), use it instead.
+set(GENHTML_OPTS ${LCOV_OPTS})


Is there a reason why we can't be consistent with the naming, either GENHTML_OPTS or HTML_OPTS

kwsantiago · 2025-11-06T23:09:37Z

cmake/script/CoverageInclude.cmake.in

+# Default: inherit LCOV_OPTS. If HTML_OPTS is provided (even if empty), use it instead.
+set(GENHTML_OPTS ${LCOV_OPTS})
+if(DEFINED HTML_OPTS)
+  separate_arguments(HTML_OPTS)


What happens if LCOV_OPTS is not defined? Does cmake create an empty variable or error?

kwsantiago · 2025-11-06T23:10:27Z

doc/developer-notes.md

 cmake -DLCOV_OPTS="--rc branch_coverage=1" -P build/Coverage.cmake
 ```

+HTML_OPTS can override the genhtml options (which default to LCOV_OPTS). If HTML_OPTS is omitted, LCOV_OPTS are implicitly passed to genhtml. If HTML_OPTS is provided but empty (e.g. -DHTML_OPTS=""), no options are passed to genhtml and LCOV_OPTS are not inherited.


What if LCOV_OPTS is also not set? Does it default to empty?

tevio · 2025-11-17T17:35:24Z

Is there a reason not to submit this through bitcoin/bitcoin?

hi @luke-jr thanks! well, mainly a lack of motivation to contribute to that project, what would be the benefit please?

split rcov and genhtml cmake opts for the Coverage profile

7aa60c6

tevio force-pushed the rcov-and-genhtml branch from fa8fc63 to 7aa60c6 Compare October 16, 2025 01:27

kwsantiago suggested changes Oct 16, 2025

View reviewed changes

tevio changed the title ~~split rcov and genhtml cmake opts for the Coverage profile~~ split lcov and genhtml cmake opts for the Coverage profile Oct 18, 2025

tevio added 2 commits October 19, 2025 02:01

document the option

c115fc1

remove duplicated lines and inline genhtml seperate_arguments

e2d638f

tevio force-pushed the rcov-and-genhtml branch from e4b821d to 842bf4f Compare October 19, 2025 19:51

tevio requested a review from kwsantiago October 19, 2025 19:57

kwsantiago suggested changes Oct 19, 2025

View reviewed changes

doc/developer-notes.md Outdated Show resolved Hide resolved

cmake/script/CoverageInclude.cmake.in Outdated Show resolved Hide resolved

cmake/script/CoverageInclude.cmake.in Outdated Show resolved Hide resolved

remove ENV overrides, shorten opt name, only allow setting via cmake …

b71b968

…option (not envvar), update doc ref

tevio force-pushed the rcov-and-genhtml branch from 842bf4f to b71b968 Compare October 20, 2025 21:54

make HTML_OPTS an override to LCOV_OPTS, for genhtml

2c2d23e

tevio requested a review from kwsantiago October 23, 2025 05:48

kwsantiago suggested changes Oct 23, 2025

View reviewed changes

kwsantiago reviewed Oct 23, 2025

View reviewed changes

doc/developer-notes.md Outdated Show resolved Hide resolved

tevio added 2 commits October 24, 2025 22:23

adapt commentary and make notes more concise

e3e917f

Default genhtml opts to LCOV_OPTS; respect explicitly empty HTML_OPTS

97a898f

tevio requested a review from kwsantiago October 24, 2025 21:49

kwsantiago suggested changes Oct 27, 2025

View reviewed changes

tevio force-pushed the rcov-and-genhtml branch from 055bb3e to 97a898f Compare October 29, 2025 22:05

simplify logic for genhtml opt passing options, remove unnecessary guard

b581b44

tevio requested a review from kwsantiago November 2, 2025 23:18

kwsantiago reviewed Nov 6, 2025

View reviewed changes

Ari4ka approved these changes Feb 15, 2026

View reviewed changes

Conversation

tevio commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Background (previous behavior)

What’s changing

Why this change

User-facing behavior

Examples

Implementation notes

Backward compatibility

Documentation

Impact and risk

Uh oh!

kwsantiago left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kwsantiago left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tevio commented Oct 20, 2025

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tevio Oct 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

kwsantiago left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tevio Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tevio Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tevio Nov 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

The three modes:

1) pass specific HTML_OPTS

should complete and generate coverage report:

2) pass empty HTML_OPTS

was previously untested but now fails with a corrupt trace error:

2) Do not pass HTML_OPTS (fall through to LCOV_OPTS)

should fail as it did, with unknown options:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luke-jr commented Nov 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tevio commented Nov 17, 2025

Uh oh!

tevio commented Oct 16, 2025 •

edited

Loading

tevio Oct 24, 2025 •

edited

Loading

tevio Oct 29, 2025 •

edited

Loading

tevio Oct 29, 2025 •

edited

Loading

tevio Nov 2, 2025 •

edited

Loading