Add initial test and GHA file for bu585 by mgeeIOL · Pull Request #160 · project-ocre/ocre-runtime

mgeeIOL · 2026-02-05T21:02:23Z

Description

Includes a new basic test group and initial github actions file to establish framework for testing ocre runtime with physical hardware after the major refactor. The test plan tests to confirm the hello-world container works and establishes initial process for testing containers on the b_u585 board. Utilizes similar methods in the GHA workflow as before the major refactor to build the .wasm and build and flash the runtime. `

Type of change

Please delete options that are not relevant.

CI system update
Test Coverage update

How Has This Been Tested?

Test scripts have been run against the board directly on a local workstation. GHA changes will be tested while PR is a draft and changes will be created in future commits to fix any problems that may arise.

Test Configuration (required):

Host OS type, version, and arch: Fedora 41 (x86)
Developer Board make & model: b_u585i_iot02a
Board Tools Installed: STM32CubeProgrammer)

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes

Includes a new basic test group and initial github actions file to establish framework for testing ocre runtime with physical hardware after the major refactor. The test plan tests to confirm the hello-world container works and establishes initial process for testing containers on the b_u585 board. Utilizes similar methods in the GHA workflow as before the major refactor to build the .wasm and build and flash the runtime. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

PatrickRobbIOL · 2026-02-06T21:25:12Z

.github/workflows/hardware-bu585.yml

+          submodules: recursive
+          fetch-depth: 0
+
+      - name: Make python venv


To slightly speed up execution time if we wanted we could keep a venv in the home dir of the user which runs this job instead of making it in a workspace which is cleaned every time. That way it would only re-install when there is a newer version of a pip package. Probably not worth doing though unless these pip installs take a while.

PatrickRobbIOL · 2026-02-06T21:26:10Z

.github/workflows/hardware-bu585.yml

+        run: |
+          cd tests_hw && bash beginTests.sh "supervisor-helloWorld"
+
+      - name: Print Modbus Server Validation Logs


re-label this from modbus to helloworld

PatrickRobbIOL · 2026-02-06T21:27:41Z

.github/workflows/main.yml

      with:
        devcontainer-tag: ${{ needs.zephyr-devcontainer.outputs.devcontainer-tag }}
+
+  hardware-bu585:


can you put an underscore after the B and add the i_iot02a?

PatrickRobbIOL · 2026-02-06T21:29:11Z

tests_hw/groups/supervisor-helloWorld/clean.py

+    conn.send_break(duration=1)
+    time.sleep(5)
+    print("Cleaning up container hello-world")
+    conn.write(b'ocre rm hello-world\n')


do you want to remove the container and also remove the image?

Removing the image should be unnecessary since we wipe the filesystem with every flash.

PatrickRobbIOL · 2026-02-06T21:29:58Z

tests_hw/groups/supervisor-helloWorld/config.json

@@ -0,0 +1,28 @@
+{
+    "name": "Supervisor Sample - Hello World",
+    "description": "Test Hello World Container within ",


within what? And why is container capitalized but within not?

PatrickRobbIOL · 2026-02-06T21:32:51Z

tests_hw/groups/supervisor-helloWorld/setup.py

+
+
+
+


Why 5 empty lines here? We could add a linter at some point if we want the python code to be clean, but I have noticed plenty of extra empty lines in this file and some others, including the GHA main.yml. Not a huge deal but worth trying to keep clean.

Code style, naming conventions, and formatting fixes. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

kr-t

Beside using our own docker container for tests (see the comment), I strongly recommend to checkout the zephyr.yml and run the mini and demo samples. Just the two of those already cover a lot of things we want to test in the first place. I think the order shall be:

run mini
run demo
run other stuff via supervisor (even if we are running hello-world again as the very first supervisor test)

Also, a minor nitpick: if you believe it is ready for a review, please remove in this case [draft] from the title.

kr-t · 2026-02-09T08:35:57Z

.github/workflows/hardware-bu585.yml

+    needs: setup-local-runner
+    runs-on: zephyr-xlarge-runner
+    container:
+      image: ghcr.io/zephyrproject-rtos/ci:v0.26-branch


We have a container made specifically for Ocre; Please check zephyr.yml to see how it is used in other workflows. It would be beneficial to use it since it would reduce the steps here (like getting wasi-sdk for example).

.github/workflows/hardware-bu585.yml

casaroli · 2026-02-09T15:01:00Z

.github/workflows/hardware-bu585.yml

+        run: |
+          python3 -m venv ./.venv
+          source .venv/bin/activate
+          pip install littlefs-python
+          pip install west 
+
+      - name: Setup west environment
+        run: |
+          source .venv/bin/activate
+          west update
+          west zephyr-export
+          west packages pip --install
+          west sdk install -t \
+            x86_64-zephyr-elf \
+            aarch64-zephyr-elf \
+            arm-zephyr-eabi


if you use our container, it already has the venv, zephyr SDK and everything needed to build (and probably run) this.

tests_hw/groups/supervisor-helloWorld/setup.py

Combines mini and demo flash and test steps to simply workflow jobs. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

Integrated use of pexpect into the demo, mini, and hello-world supervisor tests in order to improve consistency with tests. Includes addition of testlib.py file to include common test functionality such as establishing serial connections and formatting test output. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

…ial-bu585-testing

Code style updates, remove unnecessary whitespace and unused imports. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

…ial-bu585-testing

Clear serial buffer at the start and end of every serial connection during test cases to ensure runtime outputs from other github actions runs do not affect other runs. Also includes formatting updates in GHA workflow files and separates the demo and mini flashes to their own jobs outside of the container. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

Refactor test scripts to be using more direct python string comparison rather than using pexect.expect() for output detection. Also includes minor format updates for related GHA file. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

casaroli · 2026-02-24T10:23:50Z

src/ocre/commit_id.h

@@ -0,0 +1,2 @@
+/* Auto-generated file. DO NOT EDIT */


this file should not be included in the PR, please remove it (and also delete it from your workspace)

casaroli · 2026-02-24T10:29:32Z

We merged the formatting fix, so when you can , pleas rebase and all workflows will pass

Removed autogenerated files not relevant to the PR. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

…ial-bu585-testing

Add extra check after pexpect timeout for the ocre shell prompt to make sure the shell prompt is not missed and cause a false negative on the testcase. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

kr-t · 2026-02-26T12:21:48Z

I see that you are using our Ocre devcontainer to build. Great! Can you try removing west setup and see if it continues to work?

I think you don't need it, as per our other zephyr build/run sample.

Removes additional unneeded steps in the GHA actions workflow regarding the west setup which should be unnecessary when using the dev container. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

Aligns the hardware-bu585 west enviornment setup with how other GHA zephyr tests setup and manage the west enviornment. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL · 2026-02-26T21:06:43Z

I see that you are using our Ocre devcontainer to build. Great! Can you try removing west setup and see if it continues to work?

I think you don't need it, as per our other zephyr build/run sample.

After some testing I was able to remove the majority of the west setup. Let me know if you see any other changes that should be made.

casaroli

Nice work. Thank you.

I think this is good for merging and we will monitor and see how it goes, and improve as necessary.

I have some suggestions that I think we should address in next PRs:

thse python scripts are actually testing the mini/demo/supervisor programs: maybe we can extend/generalize these tests out of "hardware" and use it for testing these programs even on local machine (or other linux board) (call it "program tests" instead of "hardware tests")
add python linter
we now have system tests in Zephyr. We should add them to these workflows.

kr-t · 2026-03-02T09:52:52Z

Due to multiple merge commits on this branch, I am squashing this PR. In the future, I'd recommend a bit better commit organization, so we would fast forward the whole branch and have a less compact history.
Thanks for the contribution, it's a very good progress that we will have hardware tests again.

mgeeIOL force-pushed the test-initial-bu585-testing branch 9 times, most recently from b2c86c2 to 0a46424 Compare February 5, 2026 22:12

mgeeIOL force-pushed the test-initial-bu585-testing branch from 0a46424 to 4bbb9a2 Compare February 5, 2026 22:20

Merge branch 'main' into test-initial-bu585-testing

0b6e6f8

PatrickRobbIOL reviewed Feb 6, 2026

View reviewed changes

tests: Address initial pull request comments

b371f3c

Code style, naming conventions, and formatting fixes. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL marked this pull request as ready for review February 6, 2026 22:51

mgeeIOL requested review from kr-t and srberard as code owners February 6, 2026 22:51

kr-t requested a review from casaroli February 9, 2026 08:34

kr-t reviewed Feb 9, 2026

View reviewed changes

casaroli reviewed Feb 9, 2026

View reviewed changes

.github/workflows/hardware-bu585.yml Show resolved Hide resolved

casaroli reviewed Feb 9, 2026

View reviewed changes

tests_hw/groups/supervisor-helloWorld/setup.py Show resolved Hide resolved

casaroli linked an issue Feb 9, 2026 that may be closed by this pull request

Reintroduce testing zephyr application on real hardware (U585 board) #123

Closed

mgeeIOL changed the title ~~[DRAFT] Add initial test and GHA file for bu585~~ Add initial test and GHA file for bu585 Feb 10, 2026

mgeeIOL force-pushed the test-initial-bu585-testing branch from 8a16c49 to 9b901df Compare February 10, 2026 19:50

mgeeIOL force-pushed the test-initial-bu585-testing branch from c7a27da to a946dc3 Compare February 13, 2026 21:21

Simplify GHA workflows

7b38a12

Combines mini and demo flash and test steps to simply workflow jobs. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL force-pushed the test-initial-bu585-testing branch from a946dc3 to 7b38a12 Compare February 13, 2026 21:34

mgeeIOL force-pushed the test-initial-bu585-testing branch from c7cc1ca to 36bb7fe Compare February 17, 2026 21:54

mgeeIOL added 3 commits February 17, 2026 17:06

Merge branch 'main' of github.com:mgeeIOL/ocre-runtime into test-init…

3184b44

…ial-bu585-testing

tests_hw: General Code Cleanup

29a2dd1

Code style updates, remove unnecessary whitespace and unused imports. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

Merge branch 'main' of github.com:mgeeIOL/ocre-runtime into test-init…

b52dbc2

…ial-bu585-testing

mgeeIOL force-pushed the test-initial-bu585-testing branch 2 times, most recently from 82e8e02 to 9894092 Compare February 19, 2026 21:35

mgeeIOL force-pushed the test-initial-bu585-testing branch from 9894092 to bf88000 Compare February 20, 2026 18:34

tests_hw: Refactor test scripts

146ae1c

Refactor test scripts to be using more direct python string comparison rather than using pexect.expect() for output detection. Also includes minor format updates for related GHA file. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL force-pushed the test-initial-bu585-testing branch from ee25e20 to 146ae1c Compare February 20, 2026 20:08

casaroli reviewed Feb 24, 2026

View reviewed changes

mgeeIOL added 3 commits February 24, 2026 12:10

tests_hw: Remove unnecessary files

3b00346

Removed autogenerated files not relevant to the PR. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

Merge branch 'main' of github.com:mgeeIOL/ocre-runtime into test-init…

e2d8983

…ial-bu585-testing

tests_hw: Fix inconsistent supervisor test timeout

ee28ae2

Add extra check after pexpect timeout for the ocre shell prompt to make sure the shell prompt is not missed and cause a false negative on the testcase. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL force-pushed the test-initial-bu585-testing branch from 6cfb041 to 1d5dea5 Compare February 26, 2026 17:13

tests_hw: Simply GHA west setup

5f14ce5

Removes additional unneeded steps in the GHA actions workflow regarding the west setup which should be unnecessary when using the dev container. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL force-pushed the test-initial-bu585-testing branch 2 times, most recently from ece6417 to 3224184 Compare February 26, 2026 18:00

tests_hw: GHA Align workspace setup with other tests

d6b2b1f

Aligns the hardware-bu585 west enviornment setup with how other GHA zephyr tests setup and manage the west enviornment. Signed-off-by: Matthew Gee <mgee@iol.unh.edu>

mgeeIOL force-pushed the test-initial-bu585-testing branch from 3224184 to d6b2b1f Compare February 26, 2026 18:10

casaroli approved these changes Feb 27, 2026

View reviewed changes

kr-t merged commit 1e415a6 into project-ocre:main Mar 2, 2026
34 checks passed

Conversation

mgeeIOL commented Feb 5, 2026

Description

Type of change

How Has This Been Tested?

Checklist:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kr-t left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casaroli commented Feb 24, 2026

Uh oh!

kr-t commented Feb 26, 2026

Uh oh!

mgeeIOL commented Feb 26, 2026

Uh oh!

casaroli left a comment

Choose a reason for hiding this comment

Uh oh!

kr-t commented Mar 2, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants