Error with STAR-fusion FusionInspector

[kmavrommatis]

Hi,
I am running STAR fusion on a few 100s samples. All of them are 75bp PE reads.
Each sample runs on a cluster node with 8 cores, 64GB RAM without any other job running at the same time.
The total disk available to this job is over 1.5TB and by the end of the t
The command executed is

STAR-Fusion --left_fq $fastq1 --right_fq $fastq2 --genome_lib_dir /scratch/reference/genomes/Homo-sapiens/GRCh38.p2/STAR-Fusion --output_dir $outputDirectory/F1E000FC001.starfusion --CPU 8 --FusionInspector validate --tmpdir /scratch/tmp/kmavrommatis/STARfusion.human/

For few samples STAR-Fusion fails with the following error:

[91528000]   -retrieving read alignments for 118 spanning frags.
Error, cannot open file /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.st
ar.cSorted.dupsMarked.bam at /opt/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 34.
        SAM_reader::_init('SAM_reader=HASH(0x24436b0)') called at /opt/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 24
        SAM_reader::new('SAM_reader', '/scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RN...') called at /opt/STAR-Fusion/FusionInspector/util/get_fusion_SPANNING_reads_from_bam.from_chim_summary.pl
line 484
Traceback (most recent call last):
  File "/opt/STAR-Fusion/FusionInspector/FusionInspector", line 944, in <module>
    FusionInspector().run()
  File "/opt/STAR-Fusion/FusionInspector/FusionInspector", line 740, in run
    pipeliner.run()
  File "/opt/STAR-Fusion/FusionInspector/PyLib/Pipeliner.py", line 53, in run
    run_cmd(cmd.get_cmd())
  File "/opt/STAR-Fusion/FusionInspector/PyLib/Pipeliner.py", line 17, in run_cmd
    subprocess.check_call(cmd, shell=True)
  File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '/opt/STAR-Fusion/FusionInspector/util/get_fusion_SPANNING_reads_from_bam.from_chim_summary.pl --gtf_file /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Se
q/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.gtf --MIN_ALIGN_PER_ID 96 --bam /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/
RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam --junction_info /scratch/tmp/kmavrommatis/STARfusion.huma
n/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam.fusion_junction_info > /scratch/tmp/kma
vrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam.fusion_span_re
ads.sam' returned non-zero exit status 12
Error, cmd: /opt/STAR-Fusion/FusionInspector/FusionInspector --fusions star-fusion.fusion_predictions.abridged.tsv  --out_prefix finspector  --min_junction_reads 1  --min_novel_junction_support 3  --min_spannin
g_frags_only 5  --prep_for_IGV  --max_promiscuity 10  --out_dir /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionIns
pector-validate  --genome_lib_dir /scratch/reference/genomes/Homo-sapiens/GRCh38.p2/STAR-Fusion  --CPU 8  --left_fq /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/CheckFastqQua
lityScore_1533331075/F1E000FC001_R1.fastq.gz --right_fq /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/CheckFastqQualityScore_1533331075/F1E000FC001_R2.fastq.gz  --annotate  di
ed with ret 256 No such file or directory at /opt/STAR-Fusion/PerlLib/Pipeliner.pm line 181.
        Pipeliner::run('Pipeliner=HASH(0x197b628)') called at /opt/STAR-Fusion/STAR-Fusion line 766

Any advise?

Thanks in advance for your help

[brianjohnhaas]

Hi, It looks like the error in this case is: Error, cannot open file /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.st ar.cSorted.dupsMarked.bam Is that file missing, or has the directory structure changed at all during the run? If it was a file system hiccup, you can just rerun the failed commands and they should resume where they left off. best, ~brian

…

On Thu, Aug 16, 2018 at 1:31 PM Kostas Mavrommatis ***@***.***> wrote: Hi, I am running STAR fusion on a few 100s samples. All of them are 75bp PE reads. Each sample runs on a cluster node with 8 cores, 64GB RAM without any other job running at the same time. The total disk available to this job is over 1.5TB and by the end of the t The command executed is STAR-Fusion --left_fq $fastq1 --right_fq $fastq2 --genome_lib_dir /scratch/reference/genomes/Homo-sapiens/GRCh38.p2/STAR-Fusion --output_dir $outputDirectory/F1E000FC001.starfusion --CPU 8 --FusionInspector validate --tmpdir /scratch/tmp/kmavrommatis/STARfusion.human/ For few samples STAR-Fusion fails with the following error: [91528000] -retrieving read alignments for 118 spanning frags. Error, cannot open file /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.st ar.cSorted.dupsMarked.bam at /opt/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 34. SAM_reader::_init('SAM_reader=HASH(0x24436b0)') called at /opt/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 24 SAM_reader::new('SAM_reader', '/scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RN...') called at /opt/STAR-Fusion/FusionInspector/util/get_fusion_SPANNING_reads_from_bam.from_chim_summary.pl line 484 Traceback (most recent call last): File "/opt/STAR-Fusion/FusionInspector/FusionInspector", line 944, in <module> FusionInspector().run() File "/opt/STAR-Fusion/FusionInspector/FusionInspector", line 740, in run pipeliner.run() File "/opt/STAR-Fusion/FusionInspector/PyLib/Pipeliner.py", line 53, in run run_cmd(cmd.get_cmd()) File "/opt/STAR-Fusion/FusionInspector/PyLib/Pipeliner.py", line 17, in run_cmd subprocess.check_call(cmd, shell=True) File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '/opt/STAR-Fusion/FusionInspector/util/get_fusion_SPANNING_reads_from_bam.from_chim_summary.pl --gtf_file /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Se q/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.gtf --MIN_ALIGN_PER_ID 96 --bam /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/ RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam --junction_info /scratch/tmp/kmavrommatis/STARfusion.huma n/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam.fusion_junction_info > /scratch/tmp/kma vrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam.fusion_span_re ads.sam' returned non-zero exit status 12 Error, cmd: /opt/STAR-Fusion/FusionInspector/FusionInspector --fusions star-fusion.fusion_predictions.abridged.tsv --out_prefix finspector --min_junction_reads 1 --min_novel_junction_support 3 --min_spannin g_frags_only 5 --prep_for_IGV --max_promiscuity 10 --out_dir /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/STARfusion.human.hg38_1533331155/F1E000FC001.starfusion/FusionIns pector-validate --genome_lib_dir /scratch/reference/genomes/Homo-sapiens/GRCh38.p2/STAR-Fusion --CPU 8 --left_fq /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/CheckFastqQua lityScore_1533331075/F1E000FC001_R1.fastq.gz --right_fq /scratch/tmp/kmavrommatis/STARfusion.human/31825/DA0000206/RNA-Seq/ProcessedData/CheckFastqQualityScore_1533331075/F1E000FC001_R2.fastq.gz --annotate di ed with ret 256 No such file or directory at /opt/STAR-Fusion/PerlLib/Pipeliner.pm line 181. Pipeliner::run('Pipeliner=HASH(0x197b628)') called at /opt/STAR-Fusion/STAR-Fusion line 766 Any advise? Thanks in advance for your help — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHMVX7-3u5MkuMnLndsisHTLLB4zsTkvks5uRaxfgaJpZM4WAR0U> .

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[kmavrommatis]

Hi,
thanks for the response.
The file/directory structure is not touched by any other operation.

The curious thing is that the "missing" dupsMarked.bam file is actually there, full length and with the correct permissions.
But after having verified that the file exists, I restart the job and it resumes from its last checkpoint and still failed at the same place with the same error

[brianjohnhaas]

this is a curious one. In the code, it's basically trying to run 'samtools view' on the bam file. I can't explain why this might not work unless there's something corrupt with the bam file. You can try seeing if 'samtools view' works ok on that file. The only thing I can recommend is to start over with those samples that failed. If they fail again, maybe you could privately share the data with me to explore directly. https://mega.nz/ is my favorite way of sharing data. (bhaas@broadinstitute.org) best, ~brian

…

On Fri, Aug 17, 2018 at 2:22 PM Kostas Mavrommatis ***@***.***> wrote: Hi, thanks for the response. The file/directory structure is not touched by any other operation. The curious thing is that the "missing" dupsMarked.bam file is actually there, full length and with the correct permissions. But after having verified that the file exists, I restart the job and it resumes from its last checkpoint and still fails at the same place. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHMVX-6x_xiCZzK5ZDXdbNJDT4B4g1qqks5uRwnagaJpZM4WAR0U> .

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[stsergbg]

Hi!

It might be the wrong place since the problem was only STAR-Fusion related (not related to tutorial) but have you managed to identify the source of this particular problem? I've encountered the very same one with STAR-Fusion 1.8 - in an isolated docker container with no other processes PerlLib/SAM_reader.pm fails to read finspector.star.cSorted.dupsMarked.bam

Error, cannot open file /tmp/output_dir/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam at /usr/local/src/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 34.

Sergey

[brianjohnhaas]

Hi Sergey, Were there earlier error messages encountered that might shed some more light on this?

…

On Mon, Oct 19, 2020 at 2:57 AM stsergbg ***@***.***> wrote: Hi! It might be the wrong place since the problem was only STAR-Fusion related (not related to tutorial) but have you managed to identify the source of this particular problem? I've encountered the very same one with STAR-Fusion 1.8 - in an isolated docker container with no other processes PerlLib/SAM_reader.pm fails to read finspector.star.cSorted.dupsMarked.bam Error, cannot open file /tmp/output_dir/FusionInspector-validate/fi_workdir/finspector.star.cSorted.dupsMarked.bam at /usr/local/src/STAR-Fusion/FusionInspector/util/../PerlLib/SAM_reader.pm line 34. Sergey — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZRKX2TLEGGH3NHP3KTAB3SLPPOJANCNFSM4FQBDUKA> .

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[stsergbg]

Sorry for the late reply - this error seems to have been caused by insufficient storage space - increasing resources (disk quota and number of cores) helped me get rid of this error and execute everything successfully.

[brianjohnhaas]

great to hear it's working for you now!

…

On Thu, Oct 22, 2020 at 6:49 AM stsergbg ***@***.***> wrote: Sorry for the late reply - this error seems to have been caused by insufficient storage space - increasing resources (disk quota and number of cores) helped me get rid of this error and execute everything successfully. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZRKX62Z7XKQKR6JHQNE5TSMAE2HANCNFSM4FQBDUKA> .

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>