Have a workflow start without input files?

[hacktobeer]

As stands, a worker needs at least one file to start. Would be nice to support a worker that runs based on config settings only, rather than creating a dummy "input.txt" that tells the worker what to do.

At times, the artifacts we want to analyse are from a dynamic source, such as in Big Query. Instead of being required to have a third party orchestration tool, we wish to have a worker pull artifacts which can then kickstart a workflow.

So the input file could be set to None and the first worker is our worker (with appropriate configuration parameters) that fetches the artifacts from Big Query, provides those as output files for the rest of the workflow like normal.

[stripathi2-sc]

+1 to this

We have the exact same use case - needing to start analysis by pulling logs from BigQuery.