Skip to content

Add special Github Actions console output #448

@Garados007

Description

@Garados007

Github Actions allows special markup in the console output and has additional features which helps the developer to find errors and reports fast without looking in the respective log files. I suggest using the following features:

  1. Report of all failed WARN_* as warnings using ::warning::
  2. Report of all failed CHECK_* and REQUIRE_* as errors using ::error::
  3. Group the log lines of each test case, test suite and test run using ::group::, ::endgroup::
  4. Create a nicely human readable Markdown summary in the file referenced by the environment variable GITHUB_STEP_SUMMARY
  5. Add output variables with some metrics like number of test cases, errors, ... by writing to the file referenced by the environment variable GITHUB_OUTPUT. Actions after that can use these variables without reading the JUnit file.

Implementing 1, 2 and 4 can greatly improve the testing experience. 3 is only useful if there are lots of lines between the start and end token (maybe we should use them only for test suites and test runs).

I think this additional output should be always active if the environment variable GITHUB_ACTIONS is set to true which is always the case for Github Actions. There is no need to modify RunTest() as the environment variable can easily be set for debugging purpose and there is no other use case.

Remarks:

  • A failed expected error shouldn't produce any warning or error in the Github Actions output. There is no need to highlight them.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions