Problem with merge_bams

[tgambin]

Hi, I am trying to run germline_exome and I got error at the merge_bams step executed from sequence_to_bqsr subworkflow:

INFO [workflow alignment] starting step merge
INFO [step merge] start
ERROR Exception on step 'merge'
ERROR [step merge] Cannot make job: Invalid job input record:
the `sorted` field is not valid because
  tried boolean but
    the value is not boolean
INFO [workflow alignment] completed permanentFail

I am new to CWL but I was trying to play with parameters (also trying to add sorted parameter - but then I encountered some JS error). Would you be able to help to fix this error?

[tmooney]

The short answer is that the error you're seeing is due to quoting around the default value here:

analysis-workflows/definitions/tools/merge_bams.cwl

Line 69 in 1750cd5

default: "false"

The longer answer is that the merge_bams.cwl tool as it currently stands only works with Cromwell and not with cwltool. There are some escapes to the bash script that will also be required to make it work with cwltool (but will break it in Cromwell)--everywhere ${...} appears in the script will need to be replaced by \${...} instead.

I'm inclined to replace the bash script here with a perl script to get around this problem so the same CWL works with both runners, but it'll take a bit of time to get that done.

[tgambin]

Great - thanks a lot for explanation! We would like to try to run pipelines using cwl-airflow which is based on cwl-tool. So for now I will try to escape problematic expressions in bash. Do you know if I should expect a lot of issues with other tools/pipelines when running them with cwl-runner instead of Cromwell?

[tgambin]

Thanks again. I wanted to confirm that after escaping "${...}" in few other lines the script is now working with cwl-runner.

[tmooney]

Most tools should be fine. If you run into other issues they're likely to be similar to this--where cromwell's behavior differs from cwltool in small ways. It's our goal to have these workflows also work with cwltool (though we don't currently use it routinely ourselves), so if you do run into more issues like this feel free to let us know!

I've opened #1046 to update this tool to address the boolean value as well as the ${...} in a way that should work with both runners.