Skip to content

Feature/add testing framework#59

Merged
gonzolively merged 11 commits intoproject-ocre:mainfrom
PatrickRobbIOL:feature/add-testing-framework
Apr 16, 2025
Merged

Feature/add testing framework#59
gonzolively merged 11 commits intoproject-ocre:mainfrom
PatrickRobbIOL:feature/add-testing-framework

Conversation

@PatrickRobbIOL
Copy link
Collaborator

Description

This PR adds the Ocre testing framework, along with the first test group (flash validation). The more detailed description of what is being added can be seen in /tests/README.md which is added on one of the commits on this PR.

Type of change

  • Test Coverage update

How Has This Been Tested?

I have run this testing framework and the intial test group (the flash validation test group) directly on the Ocre-Runtime GHA agent by SCP'ing my branch over to the host, and running 'bash beginTests.sh flashValidation' which is currently "working," in that the tests are reporting fails since the ocre-runtime currently on the DUT board is reporting errors. I will provide a section from the flash validation test log here for context.

cat /tmp/flashValidation.log 
Beginning test group Flash Validation
Test the Ocre runtime after the flash is complete

Entering Setup...

Script: Flash Validation Setup
==============================

Setup is complete

Entering Testing...

Test Suite: Runtime Validation Tests
====================================

Test Case: Check Runtime Hello World
====================================

**** READING RESPONSE FROM BREAK ****

I: OSPI flash config is OPI / DTR
I: Read SFDP from octoFlash
I: Read SFDP from octoFlash
E: OST wait timed out
I: chip id 0x6b
I: lps22hh@5d: int on gpio@42021800.02
*** Booting Zephyr OS build 7dcd5844296a ***
I: LittleFS version 2.8, disk version 2.1
I: FS at flash-controller@40022000:0x100000 is 32 0x2000-byte blocks with 512 cycle
I: sizes: rd 16 ; pr 16 ; ca 64 ; la 32
E: failed to create directory (-2)
E: ***** USAGE FAULT *****
E:   Illegal load of EXC_RETURN into PC
E: r0/a1:  0x20000c58  r1/a2:  0x08045211  r2/a3:  0x00000040
E: r3/a4:  0x08029baf r12/ip:  0x08029b79 r14/lr:  0x20000c58
E:  xpsr:  0x0803b400
E: Faulting instruction address (r15/pc): 0x00000017
E: >>> ZEPHYR FATAL ERROR 34: Unknown error on CPU 0
E: Current thread: 0x20002760 (shell_uart)
E: Halting system

**** CLOSING CONNECTION ****


TEST FAILED


Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

z46-dev and others added 5 commits April 15, 2025 19:48
Adds a readme file which describes the Ocre testing framework,
how to add new tests, and gives an example of valid config.json.

Signed-off-by: Patrick Robb <probb@iol.unh.edu>
Add flash validation test cases, and remove placeholder testcases.

Signed-off-by: Patrick Robb <probb@iol.unh.edu>
Signed-off-by: Evan Parker <eparker@iol.unh.edu>
Signed-off-by: Patrick Robb <probb@iol.unh.edu>
@PatrickRobbIOL PatrickRobbIOL force-pushed the feature/add-testing-framework branch from 827f133 to 2827fad Compare April 16, 2025 00:00
@PatrickRobbIOL
Copy link
Collaborator Author

PatrickRobbIOL commented Apr 16, 2025

You can download an example of the testlog artifact here: https://github.com/project-ocre/ocre-runtime/actions/runs/14484551864

@PatrickRobbIOL
Copy link
Collaborator Author

I'm running into some difficulty getting the tests.yaml workflow to run once the build.yaml workflow is complete. My latest commit adds the most common approach to doing this, which is "workflow_run," but I think the test.yaml file needs to exist on the main branch before the workflow_run trigger will work. But I'm not sure how we can merge that the tests.yaml file trigger is correct before merging this PR into main, so it's a bit of a catch 22.

I created another PR just now which was making a pull request from ocre-runtime-my-fok/main to ocre-runtime/main to see if that would resolve the trigger issue, which is die not. I'm going to close that PR (#60) now, since it's stuck in the same place as this PR.

@gonzolively gonzolively merged commit bdd2151 into project-ocre:main Apr 16, 2025
5 of 7 checks passed
kr-t pushed a commit that referenced this pull request Aug 28, 2025
* add CONFIG_OCRE_CONTAINER_MESSAGING check for cleanup subsystems
* remove debug comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants