GISTIC2: Could not access the MCR component cache.

[imdanique]

Description of the bug

Hi, I ran into this issue while running the liquid biopsy workflow. Would appreciate any tips to fix it :)

Caused by:
  Process `DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2 (Run Gistic2)` terminated with an error exit status (255)


Command executed:

  gistic2 \
      -seg 'segments_logR_corrected_gistic.seg' \
      -refgene '/opt/refgenefiles/hg38.UCSC.add_miR.160920.refgene.mat' \
      -ta 0.1 -td 0.1 -rx 0 -conf 0.99 -qvt 0.05 -broad 1 -brlen 0.99 -cap 6 -savegene 1 -savedata 0 \
      -b ./

  mkdir -p gistic_results
  cp all_lesions.conf* {amp,del}_genes.conf* scores.gistic *.pdf  gistic_results/
  cp broad*  gistic_results/

  cat <<-END_VERSIONS > versions.yml
  "DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2":
      GISTIC2: 0.1
  END_VERSIONS

Command exit status:
  255

Command output:
  (empty)

Command error:
  Could not access the MCR component cache.

Work dir:
  /csc/epitkane/extension/projects/EDECA/results/samurai/work/74/81e711dea616f561ce44ae51102960

Container:
  /csc/epitkane/extension/projects/EDECA/results/samurai/work/singularity/quay.io-dincalcilab-gistic-2.0.23-35a282.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

Command used and terminal output

#!/usr/bin/env bash

export TMPDIR=/csc/epitkane/home/dkarabay/projects/EDECA/results/work;

nextflow run ../../src/samurai-1.2.3.1 \
        -profile singularity \
        -c ../slurm.config \
        --input ../samplesheet_bam.csv \
        --outdir SAMURAI_SHORT_VER_RUN_FROM_BAM \
        --genome hg38 \
        --fasta /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa \
        --fai /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa.fai \
        --dict /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa.dict \
        --analysis_type liquid_biopsy \
        --caller ichorcna \
        --binsize 500 \
        --normal_panel /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/HD_ULP_PoN_hg38_500kb_median_normAutosome_median.rds \
        --ichorcna_gc_wig /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/gc_hg38_500kb.wig \
        --ichorcna_map_wig /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/map_hg38_500kb.wig \
        --ichorcna_centromere_file /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/GRCh38.GCA_000001405.2_centromere_acen.txt \
        --compute_signatures TRUE \
        --run_gistic TRUE \
        --gistic_broad_analysis TRUE \
        -resume

Relevant files

No response

System information

No response

[lbeltrame]

Thanks for the report and sorry for the delayed response, I was on holiday until this week. I think @SaraPotente also ran into this error back in the days. You can try injecting in the environment (with a custom configuration file) the variable MCR_CACHE_ROOT to a path where it is writable.

Something like (replace /writable/path with a path you can access, preferably shared if you run SAMURAI on a distributed setup). Make also sure that the container can see it (use docker.runOptions or similar Nextflow variables).

process {
  env {
    MCR_CACHE_ROOT = '/writable/path'
  }
}

[imdanique]

@lbeltrame Thanks!

I tried to supply different paths to MCR_CACHE_ROOT pointing to my nextflow work directory but I guess because the container doesn't see the directory the only thing worked is this:

process {
  env {
    MCR_CACHE_ROOT = '/mcr_cache'
  }
}

singularity {
    runOptions = "--writable-tmpfs -B /path/to/samurai/work/mcr_cache:/mcr_cache"
  }

--writable-tmpfs is crucial and I get the same MCR_CACHE error without it

Not sure if this is a progress, but now I receive this error:

ERROR ~ Error executing process > 'DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2 (Run Gistic2)'

Caused by:
  Process `DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2 (Run Gistic2)` terminated with an error exit status (255)


Command executed:

  gistic2 \
      -seg 'segments_logR_corrected_gistic.seg' \
      -refgene '/opt/refgenefiles/hg38.UCSC.add_miR.160920.refgene.mat' \
      -ta 0.1 -td 0.1 -rx 0 -conf 0.99 -qvt 0.05 -broad 1 -brlen 0.99 -cap 6 -savegene 1 -savedata 0 \
      -b ./

  mkdir -p gistic_results
  cp all_lesions.conf* {amp,del}_genes.conf* scores.gistic *.pdf  gistic_results/
  cp broad*  gistic_results/

  cat <<-END_VERSIONS > versions.yml
  "DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2":
      GISTIC2: 0.1
  END_VERSIONS

Command exit status:
  255

Command output:
  Performing permutation 6 of 25
  Performing permutation 7 of 25
  Performing permutation 8 of 25
  Performing permutation 9 of 25
  Performing permutation 10 of 25
  Performing permutation 11 of 25
  Performing permutation 12 of 25
  Performing permutation 13 of 25
  Performing permutation 14 of 25
  Performing permutation 15 of 25
  Performing permutation 16 of 25
  Performing permutation 17 of 25
  Performing permutation 18 of 25
  Performing permutation 19 of 25
  Performing permutation 20 of 25
  Performing permutation 21 of 25
  Performing permutation 22 of 25
  Performing permutation 23 of 25
  Performing permutation 24 of 25
  Performing permutation 25 of 25
       1

       2

  Warning: Objects of graph2d.lineseries class exist - not clearing this class or any of its superclasses

Command error:
  Error using ghostscript (line 186)
One more CTRL+C to force exitpt. System returned error: -1.Failed to convert to output format; Ghostscript status: -100.**** Unable to open the initial device, quitting.
executor >  slurm (6)
[b6/f590eb] DIN…X (HBP20210170_04_01093_B) | 52 of 52, cached: 52 ✔
[8d/16ea64] DIN…S (HBP20210170_04_03216_B) | 50 of 52, cached: 50
[-        ] DIN…RD:PICARD_COLLECTHSMETRICS -
[bb/1eb633] DIN…S (HBP20210170_04_02696_A) | 49 of 52, cached: 49
[47/ed3200] DIN…A (HBP20210170_04_01722_A) | 52 of 52, cached: 52 ✔
[58/051559] DIN…A (HBP20210170_04_01722_A) | 52 of 52, cached: 52 ✔
[2f/3408e6] DIN…Assemble IchorCNA outputs) | 1 of 1, cached: 1 ✔
[34/da8ca7] DIN… Log2 for GISTIC Analysis) | 1 of 1, cached: 1 ✔
[6c/a6e016] DIN…IN_PLOTS (Concatenate_PDF) | 1 of 1, cached: 1 ✔
[7c/35d8db] DIN…STIC:GISTIC2 (Run Gistic2) | 0 of 1 ✘
[-        ] DIN…TIC:ASSEMBLE_GISTIC_OUTPUT -
[-        ] DIN…AMURAI:RUN_GISTIC:MAFTOOLS -
[-        ] DIN…AB_SAMURAI:SAMURAI:MULTIQC -
Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2 (Run Gistic2)'

Caused by:
  Process `DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2 (Run Gistic2)` terminated with an error exit status (255)


Command executed:

  gistic2 \
      -seg 'segments_logR_corrected_gistic.seg' \
      -refgene '/opt/refgenefiles/hg38.UCSC.add_miR.160920.refgene.mat' \
      -ta 0.1 -td 0.1 -rx 0 -conf 0.99 -qvt 0.05 -broad 1 -brlen 0.99 -cap 6 -savegene 1 -savedata 0 \
      -b ./

  mkdir -p gistic_results
  cp all_lesions.conf* {amp,del}_genes.conf* scores.gistic *.pdf  gistic_results/
  cp broad*  gistic_results/

  cat <<-END_VERSIONS > versions.yml
  "DINCALCILAB_SAMURAI:SAMURAI:RUN_GISTIC:GISTIC2":
      GISTIC2: 0.1
  END_VERSIONS

Command exit status:
  255

Command output:
  Performing permutation 6 of 25
  Performing permutation 7 of 25
  Performing permutation 8 of 25
  Performing permutation 9 of 25
  Performing permutation 10 of 25
  Performing permutation 11 of 25
  Performing permutation 12 of 25
  Performing permutation 13 of 25
  Performing permutation 14 of 25
  Performing permutation 15 of 25
  Performing permutation 16 of 25
  Performing permutation 17 of 25
  Performing permutation 18 of 25
  Performing permutation 19 of 25
  Performing permutation 20 of 25
  Performing permutation 21 of 25
  Performing permutation 22 of 25
  Performing permutation 23 of 25
  Performing permutation 24 of 25
  Performing permutation 25 of 25
  Warning: Objects of graph2d.lineseries class exist - not clearing this class or any of its superclasses

Command error:
  Error using ghostscript (line 186)
  Problem converting PostScript. System returned error: -1.Failed to convert to output format; Ghostscript status: -100.**** Unable to open the initial device, quitting.
#!/usr/bin/env bash
  **** Unable to open the initial device, quitting.

  Artifex Ghostscript 8.54: **** Could not open temporary file /csc/epitkane/home/dkarabay/projects/EDECA/results/work/gs_LIPbdA



  Error in print>LocalPrint (line 307)



  Error in print (line 233)



  Error in save_current_figure (line 38)



  Error in plot_snp_score (line 492)



  Error in gistic_plots (line 219)



  Error in run_focal_gistic (line 347)



  Error in run_gistic20 (line 124)



  Error in run_gistic2_from_seg (line 249)



  Error in gp_gistic2_from_seg (line 97)



  MATLAB:ghostscript:ProblemConvertingPostScriptSystemReturnedError

Work dir:
  /csc/epitkane/extension/projects/EDECA/results/samurai/work/7c/35d8db8bc91729aed178fedf92eb76

Container:
  /csc/epitkane/extension/projects/EDECA/results/samurai/work/singularity/quay.io-dincalcilab-gistic-2.0.23-35a282.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details

[lbeltrame]

Thanks for the comments. What version of singularity/apptainer are you using? Just in case, because I can't reproduce this on our HPC.

Artifex Ghostscript 8.54: **** Could not open temporary file /csc/epitkane/home/dkarabay/projects/EDECA/results/work/gs_LIPbdA

This tells me that either:

1. Outputs from GISTIC weren't written
2. There is a permission error there

[imdanique]

@lbeltrame Sorry for the delayed response, I've been trying to debug if for several days but it seems that I'm running in circles now.

This is as far as could get:

My command:

#!/usr/bin/env bash

export TMPDIR=/csc/epitkane/home/dkarabay/projects/EDECA/results/work;
export NXF_OPTS='-Xms1g -Xmx4g'

nextflow run ../../src/samurai-1.2.3.1 \
        -profile singularity \
        -c ./slurm_samurai_tmp.config \
        --input ../samplesheet_bam.csv \
        --outdir SAMURAI_SHORT_VER_RUN_FROM_BAM \
        --genome hg38 \
        --fasta /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa \
        --fai /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa.fai \
        --dict /csc/epitkane/data/ref_genomes/hg38_no_alt/grch38_core+bs_controls.fa.dict \
        --analysis_type liquid_biopsy \
        --caller ichorcna \
        --binsize 500 \
        --normal_panel /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/HD_ULP_PoN_hg38_500kb_median_normAutosome_median.rds \
        --ichorcna_gc_wig /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/gc_hg38_500kb.wig \
        --ichorcna_map_wig /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/map_hg38_500kb.wig \
        --ichorcna_centromere_file /csc/epitkane/extension/projects/EDECA/src/samurai-1.2.3.1/assets/ichorcna/GRCh38.GCA_000001405.2_centromere_acen.txt \
        --run_gistic FALSE \
        --gistic_broad_analysis TRUE \
        --compute_signatures TRUE \
        -resume

My config:

process {
  executor = "slurm"
  queue = "cpu"
  queueSize = 20
  submitRateLimit = '5/1min'
  clusterOptions = '-x a0,a6'
  
  cpus   = 18
  memory = 170.GB
  time   = 240.h
  
  withName: 'GISTIC2' {
  containerOptions = "--contain --writable-tmpfs"
  env.TMPDIR = "/tmp"
  env.MCR_CACHE_ROOT = "/tmp"

 }
}

The error:

  Error using ghostscript (line 186)
  Problem converting PostScript. System returned error: -1.Failed to convert to output format; Ghostscript status: -100.
  Artifex Ghostscript 8.54: Can't find initialization file gs_init.ps.
  MATLAB:ghostscript:ProblemConvertingPostScriptSystemReturnedError

it seems that with --contain it fails to find some necessary files within the container for Gistic, but without --contain it cannot write temp files under my work directory and leads to this error:

Artifex Ghostscript 8.54: **** Could not open temporary file /csc/epitkane/home/dkarabay/projects/EDECA/results/w
ork/gs_yebpeI

For now I disabled GISTIC and skipped it.

[lbeltrame]

Which apptainer / singularity version are you using?
Due to HPC configuration issues I'm stuck on 1.1 here, so that might be the difference. Or are you using Docker?

[imdanique]

Sorry, missed that part in my previous message.

singularity version 3.7.0
CentOS Linux release 7.9.2009 (Core)

[lbeltrame]

Oh, that may explain why. But, I'll try tomorrow to check where that file is in the container image (because I'm sure it is there, or it wouldn't work for me as well...) and come up with better suggestions on how to debug this.

[imdanique]

The file is definitely there:

singularity exec /csc/epitkane/extension/projects/EDECA/results/samurai/work/singularity/quay.io-dincalcilab-gistic-2.0.23-35a282.img \
    find / -name gs_init.ps 2>/dev/null
/opt/conda/pkgs/mcr-8.3-0/share/mcr-8.3-0/v83/sys/extern/glnxa64/ghostscript/ps_files/gs_init.ps
/opt/conda/share/mcr-8.3-0/v83/sys/extern/glnxa64/ghostscript/ps_files/gs_init.ps

it seems there is something wrong with my workaround

Thanks for your time!

[lbeltrame]

You might want to set the GS_LIB environment variable to point at that folder or a parent (this is from a 2 second Google search, so it might not work at all ;)

[imdanique]

Thanks! I tried it at some point, but it didn't work.

This is the workaround I came up with that worked:

I removed this line from my bash command:
export TMPDIR=/csc/epitkane/home/dkarabay/projects/EDECA/results/work

and changed config to this:

withName: 'GISTIC2' {
   containerOptions = "--writable-tmpfs --no-home --home /tmp/home"
   env.TMPDIR = "/tmp"
   env.MCR_CACHE_ROOT = "/tmp"
   beforeScript = "mkdir -p /tmp/home /tmp/mcr_cache"
}

The whole pipeline hasn't finished yet, but at least GISTIC completed successfully :)

[lbeltrame]

Hm... I guess this warrants an addition in the docs. Unfortunately GISTIC is unmaintained since 2015 (!!!) so one has to resorts to all sorts of workarounds...