Resolved GitHub Action Test Error

[itutu-tienday]

Content

• An error occurred in testing with the latest GitHub Action (as of March 2026).
• It appears that the issue was caused by an outdated and fixed version of a certain module (pnwb).

Error log

FAILED studio/tests/app/common/core/snakemake/test_snakemake_executor_lccd.py::test_snakemake_execute
FAILED studio/tests/app/common/core/snakemake/test_snakemake_executor_lccd.py::test_snakemake_delete_dependencies
= 2 failed, 1070 passed, 15 skipped, 2 deselected, 5 warnings in 111.89s (0:01:51) =

Expected Cause

The version of a specific module (pynwb) was outdated and its version was fixed, which apparently caused compatibility issues with the setuptools version.
This resulted in a situation where "the Python 3.9 conda environment was loading the jaraco.context package from the host (Python 3.11)."

Solution

One of the following solutions is possible, but option 1 was chosen for its maintainability.

1. Upgrade the pynwb version (latest version of 2.x)
2. Downgrade the setuptools version (<71, etc.)

Testcase

• Verify that the error is resolved using act (GitHub Action Simulator).

Note

GitHub Actions can be simulated using act.

• https://github.com/nektos/act

Verification will be performed using the above.
*It appears that verification work using act was also performed for a similar issue in the past.

• Adjustments to avoid docker build errors (Dockerfile.test) #910

[milesAraya]

Thank you for finding this!

[itutu-tienday]

Investigation process log (1/3)

The error was reproduced with act (the same error as on GitHub).

• Execution command
  act -j test -P ubuntu-latest=catthehacker/ubuntu:act-latest --container-architecture linux/amd64 --container-daemon-socket /var/run/docker.sock

• Error log

  | FAILED studio/tests/app/common/core/snakemake/test_snakemake_executor_lccd.py::test_snakemake_execute - AssertionError: Workflow pickle not found
  | FAILED studio/tests/app/common/core/snakemake/test_snakemake_executor_lccd.py::test_snakemake_delete_dependencies - AssertionError: Dependencies files not found
  | ==== 2 failed, 1084 passed, 15 skipped, 2 deselected, 5 warnings in 30.16s =====
  

[itutu-tienday]

Investigation process log (2/3)

To isolate the cause, I reviewed the test case with the following adjustments applied:

• Since there seemed to be an error with pca, we prepared a test case that temporarily excluded pca.
  • Temporarily exclude pca from the following file:
    • studio/test_data/output_test/default/smk_exec_lccd/snakemake.yaml

Result

• When pca was excluded, the workflow succeeded, and pytest also completed successfully.
• Evidence log

  [Tests/test]   | 2026-03-24 07:15:54,246 INFO:     [optinist] (pid:1) (task:local) (client:default) snakemake_execute():63 - Starting local execution mode
  [Tests/test]   | 2026-03-24 07:15:54,249 INFO:     [optinist] (pid:1) (task:local) (client:default) snakemake_execute():65 - start snakemake running process.
  [Tests/test]   | 2026-03-24 07:15:54,260 DEBUG:    [optinist] (pid:47) (task:local) (client:default) _snakemake_execute_process():144 - Workflow API created successfully
  [Tests/test]   | 2026-03-24 07:15:54,261 DEBUG:    [optinist] (pid:47) (task:local) (client:default) _snakemake_execute_process():145 - Creating DAG...
  
  ~~~~
  
  job                         count
  ------------------------  -------
  all                             1
  input_0                         1
  lccd_cell_detection_1234        1
  total                           3
  
  ~~~~
  
  [Tests/test]   | 2026-03-24 07:16:07,574 INFO:     [optinist] (pid:47) (task:local) (client:default) _snakemake_execute_process():164 - snakemake_execute succeeded.
  [Tests/test]   | 2026-03-24 07:16:07,575 INFO:     [optinist] (pid:47) (task:local) (client:default) _snakemake_execute_process():177 - snakemake_execute succeeded.
  

Considerations

• It doesn't appear that snakemake itself is unable to process.
• This may be a problem limited to pca.

Further investigation plan

• I will examine the pca code and investigate the point where the processing is terminating abnormally (?).
• Also, the following conda env file appears to lack a version specification for the pca module.
  • studio/app/optinist/wrappers/optinist/conda/optinist.yaml

    - numexpr
    - numba
    - dpca
    

  • This point will also be investigated.

[itutu-tienday]

Investigation process log (3/3)

I have reconfirmed that the following logs have been recorded.

INFO:     [snakemake.logging] (pid:39) (task:local) (client:default) __new__():264 - Activating conda environment: ../../../../../.snakemake/conda/a5ad67595b000c6d50997262f299aae6_
Activating conda environment: ../../../../../.snakemake/conda/a5ad67595b000c6d50997262f299aae6_
ERROR:    [optinist] (pid:158) (task:local) (client:default) main():47 - <class 'TypeError'>: unsupported operand type(s) for |: 'types.GenericAlias' and 'NoneType'
Traceback (most recent call last):
  File "/app/studio/test_data/output/default/smk_exec_lccd/.snakemake/scripts/tmpw77ij71h.func.py", line 25, in main
    from studio.app.common.core.rules.runner import Runner
  File "/app/studio/app/common/core/rules/runner.py", line 25, in <module>
    from studio.app.optinist.core.nwb.nwb_creater import (
  File "/app/studio/app/optinist/core/nwb/nwb_creater.py", line 5, in <module>
    from pynwb import NWBHDF5IO, NWBFile
  File "/app/.snakemake/conda/a5ad67595b000c6d50997262f299aae6_/lib/python3.9/site-packages/pynwb/__init__.py", line 105, in <module>
    __resources = __get_resources()
  File "/app/.snakemake/conda/a5ad67595b000c6d50997262f299aae6_/lib/python3.9/site-packages/pynwb/__init__.py", line 25, in __get_resources
    from pkg_resources import resource_filename
  File "/app/.snakemake/conda/a5ad67595b000c6d50997262f299aae6_/lib/python3.9/site-packages/pkg_resources/__init__.py", line 90, in <module>
    from jaraco.text import drop_comment, join_continuation, yield_lines
  File "/app/.snakemake/conda/a5ad67595b000c6d50997262f299aae6_/lib/python3.9/site-packages/setuptools/_vendor/jaraco/text/__init__.py", line 12, in <module>
    from jaraco.context import ExceptionTrap
  File "/usr/local/lib/python3.11/site-packages/jaraco/context/__init__.py", line 43, in <module>
    type[BaseException] | None,
TypeError: unsupported operand type(s) for |: 'types.GenericAlias' and 'NoneType'

From the above, it can be confirmed that "although the environment is supposed to be Python 3.9, Python 3.10 syntax is being used (unsupported operand type)."

• From this, it was inferred that "a version mismatch of setuptools is being used."
• Furthermore, it was inferred that "some older modules are causing the setuptools version mismatch."
• Ultimately, it was inferred that the cause was an old and fixed version of pynwb.