feat: Add JSON, YAML, and Toon output options for zpm test results

[AshokThangavel]

Feature: Add Multi-Format Output (Toon, JSON, YAML) to zpm test

fixes #971

Summary

This Pull Request includes significant flexibility to the zpm test command by allowing users to define the output format for both terminal display and file generation. This enhancement supports Toon (default), JSON, and YAML formats, improving integration with external reporting tools and overall user experience.

It summarizes only the failed unit tests by default.

1. Terminal Output Control via -f / -output Flag

The new optional flag, -f (or -output), allows users to specify the desired output format of the test results displayed directly in the terminal (stdout).

• Supported Formats: Toon, Json, Yaml
• Default Behavior: If the flag is omitted, the output defaults to the Toon format.

Interaction with Verbose Flag (-v):

• Explicit Output (No -v): If the -v (verbose) flag is not defined, the selected format will be displayed to the terminal.
  • Example Command:

    zpm "zpm test -only" 
    # Output defaults to Toon format summary 

  • Explicit Format Command:

    zpm "zpm test -only -f Json"
    # Output is displayed as JSON summary

• Verbose Override: If the verbose flag (-v) is defined in the command, the -f flag is ignored, and no output summary is displayed (maintaining the existing verbose behavior).

Example Toon Output Structure (Terminal):

Test result summary

unitTest:
  id: 20
  namespace: USER
  duration: 63.107128
  testDateTime: 2025-12-04 13:01:56

results[1]{suiteName,testcaseName,methodName,status,assertAction,assertCounter,assertDescription,assertLocation}:
  (root),Test.PM.Unit.AAFailed,TestDatabaseResources,20,AssertStatusOK,1,"testing => ERROR #5001: test error","TestDatabaseResources+2^Test.PM.Unit.AAFailed.cls"

2. Dedicated Configuration Directives for File Generation (-D)

The existing configuration directive logic (-D) used for file output (e.g., -DUnitTest.JUnitOutput) has been extended to support the new formats to easily generate and store results files.

The following new configuration directives are now supported:

Directive	Output Format	Example Usage
-DUnitTest.JsonOutput	JSON	/path/to/testresult.json
-DUnitTest.YamlOutput	YAML	/path/to/testresult.yaml
-DUnitTest.ToonOutput	Toon	/path/to/testresult.toon

Example File Generation Command:

zpm "zpm test -only -DUnitTest.JsonOutput=/usr/irissys/mgr/user/testresult/testresult.json"

Unit Test Verification

A dedicated unit test class, Test.PM.Unit.TestResultsOPFormatAndFileGenTest, has been created to thoroughly test the new file generation functionality for all three formats.

The test results below confirm that files are successfully generated and then cleaned up:

Counter	Action	Status	Description	Location
3	AssertStatusOK	passed	yaml file generated successfully in /home/.../test-reports/	TestResultFileGeneration+12^...
4	AssertStatusOK	passed	Json file generated successfully in /home/.../test-reports/	TestResultFileGeneration+16^...
5	AssertStatusOK	passed	Toon file generated successfully in /home/.../test-reports/	TestResultFileGeneration+20^...
11	AssertStatusOK	passed	File 'test.Json' has been deleted successfully	ShowGeneratedFilesAndCleaup+14^...
12	AssertStatusOK	passed	File 'test.toon' has been deleted successfully	ShowGeneratedFilesAndCleaup+14^...
13	AssertStatusOK	passed	File 'test.yaml' has been deleted successfully	ShowGeneratedFilesAndCleaup+14^...

[isc-tleavitt]

@AshokThangavel thank you for this! And for your other recent contributions.

Just as a heads up / to set expectations, we have a serious backlog of large PRs right now and the people who most need to review (@isc-kiyer and I) also have a bunch of other non-IPM stuff on our plates. Getting community contribution on a project like this is like a dog chasing it's tail, and now that we've caught it we need to figure out what to do. :) There's a history of outstanding community contribution on IPM, but as part of inclusion in IRIS we've necessarily taken more control.

On this particular PR, big picture (without a thorough review):

• The motivating use case is LLM processing of test results - as such, it's great that TOON is the default, but we don't want to have a list of all the passed assertions. A high level summary of the # of passed/failed assertions/tests and details on the failures would be ideal, with -verbose controlling inclusion of passed assertions in the specified output format.
• The storage of errors that happen at various levels - e.g., an exception thrown during a test, an error %Status or exception thrown from On(Before|After)(AllTests|OneTest) - is nonintuitive. We need to be sure, and unit test, that errors at these levels are captured in output; otherwise you can have tests fail silently and not notice.

[AshokThangavel]

Hi @isc-tleavitt
Thanks again for taking the time to provide feedback despite your heavy workload. I completely understand the challenge of balancing community contributions with internal control, and I appreciate the dedication you both put into managing this project.

Regarding LLM-Friendly Output

• The current implementation of the non-verbose terminal output for all formats (Toon, JSON, YAML) already excludes all passed assertions by default. It only displays or serializes the summary counts and the detailed information for failed assertions
• When Interaction with the -v (verbose) flag is defined, the default behavior of displaying verbose output is maintained. so user must explicitly define -f flag (e.g., zpm "zpm test -v -f Toon"). If -v is present and -f is omitted by default