Skip to content

Support basic Nahual functionality#6

Merged
afermg merged 6 commits intomainfrom
nahual
Sep 11, 2025
Merged

Support basic Nahual functionality#6
afermg merged 6 commits intomainfrom
nahual

Conversation

@afermg
Copy link
Owner

@afermg afermg commented Sep 11, 2025

Nahual is a companion tool that allows us to deploy a multitude of models for segmentation (e.g., cellpose), tracking (e.g., trackastra) or featurization (e.g., Dinov2) in an independent Python environment or server altogether. I have integrated its functionality for multiple reasons:

  • Relieve dependency contraints, such as baby, which requires older versions of many things. Now aliby can evolve independently from baby and still reuse it.
  • Facilitate multiprocessing: Many issues arose when trying to use multiprocessing while using GPUs (e.g., cellpose breaks very quickly). Multithreading works to some degree, but then only a few CPUs are used. Decoupling the steps allows us to
  • Scale up using multiple servers: If an aliby pipeline can contact multiple other processes in other computers it could make use of their GPUs/CPUs and process data much faster.

There is a bit of an overhead of moving data around, but at least for pipelines running within the same server it is not an issue (it may become if we start calling servers with low transfer speeds).

afermg added 6 commits August 23, 2025 18:10
The `run_pipeline` and `run_pipeline_save` functions have been
removed to simplify the pipeline execution flow.

The main entry point is now `run_pipeline_and_post`, which has been
updated for clarity by renaming the `out_dir` parameter to
`output_path`.

Additional dead code and comments have also been removed across
several modules.
@afermg afermg merged commit 9625d27 into main Sep 11, 2025
0 of 2 checks passed
@afermg afermg deleted the nahual branch November 2, 2025 15:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant