rixpress might interest you

[b-rodrigues]

I'm taking the liberty to ping you (sorry) because you contributed to rix, and thought that you might be interested in this latest experiment of mine, called rixpress.
The idea of rixpress is to provide essentially a clone of targets, but letting Nix handle the build of the pipeline. It uses a default.nix generated by rix to provide the environment to all the build steps, and I'm aiming at adding the possibility to easily define python derivations.

I thought you might be interested in at least knowing this exists, but if you want to experiment and provide feedback and/or contribute, that would be more than welcome!

The current state is a first draft, the pipeline in the readme works, but it's definitely not polished yet.

@philipp-baumann @mihem @RichardJActon @jrosell @eliocamp @Kupac

[RichardJActon]

Cool! Are you familiar with BioNix it seems to have similar goals but with Nix and not R as the language to define the pipeline, not sure how active the project is these days.

https://github.com/PapenfussLab/bionix
https://doi.org/10.1093/gigascience/giaa121

One of the limitations of targets' design in my view is that there is a single unified execution environment specification (or was last time I actively used it a couple of years ago). Other pipeline management tools like Nextflow and snakemake allow the specification of separate environments for each step in the pipeline. This can make debugging issues with these environments much easier as they tend to be simpler. It also increases the portability / modularity of individual steps between different pipelines as their dependencies don't need to be disentangled from the rest of the global environment. A lot of the updates to Nextflow over the past few years have been around increasing the modularity of individual components so they are easier to reuse in other workflows. So I would consider having as a design goal having the ability to at least optionally specify separate nix environments for each step.

[b-rodrigues]

Cool! Are you familiar with BioNix it seems to have similar goals but with Nix and not R as the language to define the pipeline, not sure how active the project is these days.

https://github.com/PapenfussLab/bionix https://doi.org/10.1093/gigascience/giaa121

yes, I believe @jbedo is the author of BioNix!

So I would consider having as a design goal having the ability to at least optionally specify separate nix environments for each step.

yep, that’s actually also what I think. For now, this is not the case, but I want to add an optional argument to the function that generate the Nix code to allow different environments to be used.

[b-rodrigues]

So I would consider having as a design goal having the ability to at least optionally specify separate nix environments for each step.

I have a prototype of this with 35b8ba2

It's not 100% because I still need to load the specific R packages for each environment, but that shouldn't be much of an issue.

EDIT: Fully implemented now: 420503d