Include device logs in test artifacts by stefan-silabs · Pull Request #169 · SiliconLabsSoftware/matter_extension

stefan-silabs · 2025-10-29T16:01:34Z

Issue Link:
NOJIRA

Description of Problem/Feature:
Include device logs for more information in case of failures.

Description of Fix/Solution:

Testing Done:
CI

Note

Packages pytest report and device logs into a single tar.gz artifact and updates Jenkins test results URL output for PR vs branch builds.

Jenkins pipeline (jenkins_integration/jenkinsFunctions.groovy):
- Artifacts: Replace single pytest-report-*.html upload with test-results-<app>-<board>.tar.gz containing pytest-report.html and any test_tc*.txt logs; archive the tarball.
- Output: Print tar archive contents for verification.
- Links: Echo different Jenkins test results URLs for PR (change-requests) vs branch builds.

^{Written by Cursor Bugbot for commit 8dab24c. This will update automatically on new commits. Configure here.}

Copilot

Pull Request Overview

This PR adds device log collection to test artifacts in the Jenkins integration to provide more debugging information when test failures occur. The changes also include debugging commands and placeholder logic for conditional branch-based URL generation.

Adds debug commands to list directory contents during test execution
Includes a TODO comment for archiving device logs alongside HTML reports
Comments out the existing test results URL echo statement and adds conditional logic for different branch types

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

jenkins_integration/jenkinsFunctions.groovy

…extension into include_device_logs_for_ci_tests

Copilot

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-30T13:05:48Z

jenkins_integration/jenkinsFunctions.groovy

@@ -157,10 +157,14 @@ def execute_sanity_tests(nomadNode, deviceGroup, deviceGroupId, harnessTemplate,
                            }
                        }
                    }


The wildcard pattern test_tc*.txt for device logs is unclear and could match unintended files. Consider using a more specific pattern or documenting what files this is intended to capture.

Suggested change

}

}

// The pattern 'test_tc*.txt' is intended to capture all test case result files generated by the test framework,

// such as 'test_tc01.txt', 'test_tc02.txt', etc. If the naming convention changes, update this pattern accordingly.

Copilot · 2025-10-30T13:05:48Z

jenkins_integration/jenkinsFunctions.groovy

+                    if(branchName.startsWith("PR-")){
+                        echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/view/change-requests/job/${BRANCH_NAME}/${BUILD_NUMBER}/"
+                    } else {
+                        echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/job/${BRANCH_NAME}/${BUILD_NUMBER}/"
+                    }


The URLs are duplicated with only a minor difference. Consider extracting the common base URL and conditionally adding the 'view/change-requests' segment to reduce duplication.

Suggested change

if(branchName.startsWith("PR-")){

echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/view/change-requests/job/${BRANCH_NAME}/${BUILD_NUMBER}/"

} else {

echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/job/${BRANCH_NAME}/${BUILD_NUMBER}/"

}

def baseUrl = "https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub"

def changeRequestsSegment = branchName.startsWith("PR-") ? "/view/change-requests" : ""

def testResultsUrl = "${baseUrl}${changeRequestsSegment}/job/${BRANCH_NAME}/${BUILD_NUMBER}/"

echo "See test results here: ${testResultsUrl}"

Copilot

Pull Request Overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-10-31T17:05:39Z

jenkins_integration/jenkinsFunctions.groovy

                    }
-                    sh "cp ./reports/pytest-report.html ./reports/pytest-report-${appName}-${board}.html"
-                    archiveArtifacts artifacts: "reports/pytest-report-${appName}-${board}.html"
+                    sh "tar -czf ./reports/test-results-${appName}-${board}.tar.gz -C ./reports pytest-report.html test_tc*.txt || true"


The || true at the end of the tar command will suppress all errors, including cases where pytest-report.html doesn't exist. Consider handling missing files more explicitly or at least logging when the tar command fails.

Suggested change

sh "tar -czf ./reports/test-results-${appName}-${board}.tar.gz -C ./reports pytest-report.html test_tc*.txt || true"

// Check if at least one artifact exists before attempting to tar

def pytestReportExists = fileExists('reports/pytest-report.html')

def testTcFiles = sh(script: "ls reports/test_tc*.txt 2>/dev/null | wc -l", returnStdout: true).trim()

if (pytestReportExists || testTcFiles != "0") {

try {

sh "tar -czf ./reports/test-results-${appName}-${board}.tar.gz -C ./reports pytest-report.html test_tc*.txt"

} catch (err) {

echo "Error creating tarball: ${err}"

}

} else {

echo "Warning: No test report artifacts found to archive (pytest-report.html or test_tc*.txt missing)."

}

…r lock an .slcp Merge in WMN_TOOLS/matter_extension from bugfix/doorlock_silabs_zap to release_2.5-1.4 Squashed commit of the following: commit 07c3af80ba8a5e7105527a28004b0c626ffac83f Author: lpbeliveau-silabs <louis-philip.beliveau@silabs.com> Date: Wed Nov 13 15:03:13 2024 -0500 Updated the Zap file path in our lock an .slcp

…are/matter_extension into include_device_logs_for_ci_tests

cursor

Bug: Missing run_number parameter in SQA artifact upload

The call to upload_artifacts for SQA artifacts is missing the run_number parameter. While commit_sha and workflow_id are passed, run_number defaults to the string "null". This causes the Python script to receive --run_number null as an argument, which may lead to incorrect behavior since run_number is a required parameter that should contain the actual workflow run number available in the run_number variable.

jenkins_integration/Jenkinsfile#L209-L210

matter_extension/jenkins_integration/Jenkinsfile

Lines 209 to 210 in 83456aa

    
           withDockerContainer(image: 'artifactory.silabs.net/gsdk-docker-production/gsdk_nomad_containers/gsdk_ubai:latest') { 
        
               def result = pipelineFunctions.upload_artifacts(sqa=true, commit_sha=commit_sha, workflow_id=workflow_id)

cursor · 2025-11-24T14:13:06Z

jenkins_integration/jenkinsFunctions.groovy

+                        echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/view/change-requests/job/${BRANCH_NAME}/${BUILD_NUMBER}/"
+                    } else {
+                        echo "See test results here: https://jenkins-cbs-iot-matter.silabs.net/job/Matter%20Extension%20GitHub/job/${BRANCH_NAME}/${BUILD_NUMBER}/"
+                    }


Bug: URL uses wrong variable for branch name

The condition on line 362 checks the branchName parameter to determine if it's a PR build, but the URLs on lines 363 and 365 use the BRANCH_NAME environment variable instead. When branchName is set to PR-{number} format (as passed from the caller), BRANCH_NAME remains the actual Git branch name, causing the generated URLs to point to incorrect Jenkins job paths that won't match the PR naming convention.

jenkins_integration/jenkinsFunctions.groovy

…are/matter_extension into include_device_logs_for_ci_tests

check directory contents for dut logs

dcf8d8f

Copilot AI review requested due to automatic review settings October 29, 2025 16:01

stefan-silabs requested a review from a team as a code owner October 29, 2025 16:01

Copilot AI reviewed Oct 29, 2025

View reviewed changes

jenkins_integration/jenkinsFunctions.groovy Outdated Show resolved Hide resolved

jenkins_integration/jenkinsFunctions.groovy Outdated Show resolved Hide resolved

sarthak shaha (Sarthak-Shaha) approved these changes Oct 29, 2025

View reviewed changes

jenkins_integration/jenkinsFunctions.groovy Outdated Show resolved Hide resolved

stefan-silabs added 2 commits October 29, 2025 14:00

zip dut logs and pytest report, fix test results link

11fcea4

Merge branch 'main' of https://github.com/SiliconLabsSoftware/matter_…

79d283a

…extension into include_device_logs_for_ci_tests

Copilot AI review requested due to automatic review settings October 30, 2025 13:05

Copilot AI reviewed Oct 30, 2025

View reviewed changes

stefan-silabs added 2 commits October 31, 2025 09:05

install zip to archive all report files

9b1188b

use tar instead of zip

e608ab0

Copilot AI review requested due to automatic review settings October 31, 2025 17:05

Copilot AI reviewed Oct 31, 2025

View reviewed changes

stefan-silabs added 2 commits November 24, 2025 08:50

Merge branch 'release_2.8-1.5' of https://github.com/SiliconLabsSoftw…

050ce76

…are/matter_extension into include_device_logs_for_ci_tests

update logic

83456aa

stefan-silabs changed the base branch from main to release_2.8-1.5 November 24, 2025 14:00

cursor bot reviewed Nov 24, 2025

View reviewed changes

stefan-silabs closed this Nov 24, 2025

stefan-silabs reopened this Nov 24, 2025

stefan-silabs changed the title ~~(TESTING) Include device logs in test artifacts~~ Include device logs in test artifacts Nov 24, 2025

cursor bot reviewed Nov 24, 2025

View reviewed changes

fix logix

114fe58

cursor bot reviewed Nov 24, 2025

View reviewed changes

jenkins_integration/jenkinsFunctions.groovy Outdated Show resolved Hide resolved

stefan-silabs added 4 commits November 24, 2025 10:35

escape properly for variable interpolation

8dab24c

update to use find instead

89b2559

zip the whoe reports folder

7418a20

Merge branch 'release_2.8-1.5' of https://github.com/SiliconLabsSoftw…

c397c00

…are/matter_extension into include_device_logs_for_ci_tests

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Include device logs in test artifacts#169

Include device logs in test artifacts#169
stefan-silabs wants to merge 12 commits intorelease_2.8-1.5from
include_device_logs_for_ci_tests

stefan-silabs commented Oct 29, 2025 •

edited by cursor bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 30, 2025

Uh oh!

Copilot AI Oct 30, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Oct 31, 2025

Uh oh!

cursor bot left a comment

Uh oh!

cursor bot Nov 24, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

-                    }
+                    }
+                    // The pattern 'test_tc*.txt' is intended to capture all test case result files generated by the test framework,
+                    // such as 'test_tc01.txt', 'test_tc02.txt', etc. If the naming convention changes, update this pattern accordingly.

-                    sh "tar -czf ./reports/test-results-${appName}-${board}.tar.gz -C ./reports pytest-report.html test_tc*.txt || true"
+                    // Check if at least one artifact exists before attempting to tar
+                    def pytestReportExists = fileExists('reports/pytest-report.html')
+                    def testTcFiles = sh(script: "ls reports/test_tc*.txt 2>/dev/null | wc -l", returnStdout: true).trim()
+                    if (pytestReportExists || testTcFiles != "0") {
+                        try {
+                            sh "tar -czf ./reports/test-results-${appName}-${board}.tar.gz -C ./reports pytest-report.html test_tc*.txt"
+                        } catch (err) {
+                            echo "Error creating tarball: ${err}"
+                        }
+                    } else {
+                        echo "Warning: No test report artifacts found to archive (pytest-report.html or test_tc*.txt missing)."
+                    }

	withDockerContainer(image: 'artifactory.silabs.net/gsdk-docker-production/gsdk_nomad_containers/gsdk_ubai:latest') {
	def result = pipelineFunctions.upload_artifacts(sqa=true, commit_sha=commit_sha, workflow_id=workflow_id)

Conversation

stefan-silabs commented Oct 29, 2025 • edited by cursor bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Oct 30, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Bug: Missing run_number parameter in SQA artifact upload

Uh oh!

cursor bot Nov 24, 2025

Choose a reason for hiding this comment

Bug: URL uses wrong variable for branch name

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stefan-silabs commented Oct 29, 2025 •

edited by cursor bot

Loading