celldex expects access to a lock file in the user's home dir

[kelly-sovacool]

testing on the containerize branch: ./bin/sinclair run -profile test

  Error in `lock()`:
  ! Directory of lock file does not exist: '/home/sovacoolkl/.cache/gypsum/status/celldex/hpca/2024-02-26'
  Backtrace:
   1. global MAIN_SINGLER(so_processed, species)
   3. celldex::HumanPrimaryCellAtlasData()
   4. celldex::fetchReference("hpca", "2024-02-26", realize.assays = TRUE)
   5. gypsum::saveVersion(package, name, version, cache = cache, overwrite = overwrite)
   6. gypsum:::acquire_lock(cache, project, asset, version)
   7. filelock::lock(lockloc)
  
  Quitting from lines 175-204 [label] (seurat_preprocess.Rmd)

[kelly-sovacool]

https://rdrr.io/github/LTLA/celldex/man/HumanPrimaryCellAtlasData.html

[kelly-sovacool]

we need to pre-download all of the celldex databases, place them in a shared location in /data/CCBR_Pipeliner, and pass them in to SEURAT_PREPROCESS directly

rather than calling celldex::HumanPrimaryCellAtlasData(), we should directly call fetchReference(cache = "path/to/our/cache") -- repeat for each database.

MAIN_SINGLER <- function(so_in, species) {
  if (species == "hg38" || species == "hg19") {
    so_in$HPCA_main <- RUN_SINGLEr(so_in, celldex::HumanPrimaryCellAtlasData(), "label.main")
    so_in$HPCA <- RUN_SINGLEr(so_in, celldex::HumanPrimaryCellAtlasData(), "label.fine")
    so_in$BP_encode_main <- RUN_SINGLEr(so_in, celldex::BlueprintEncodeData(), "label.main")
    so_in$BP_encode <- RUN_SINGLEr(so_in, celldex::BlueprintEncodeData(), "label.fine")
    so_in$monaco_main <- RUN_SINGLEr(so_in, celldex::MonacoImmuneData(), "label.main")
    so_in$monaco <- RUN_SINGLEr(so_in, celldex::MonacoImmuneData(), "label.fine")
    so_in$immu_cell_exp_main <- RUN_SINGLEr(
      so_in, celldex::DatabaseImmuneCellExpressionData(),
      "label.main"
    )
    so_in$immu_cell_exp <- RUN_SINGLEr(
      so_in, celldex::DatabaseImmuneCellExpressionData(),
      "label.fine"
    )
    so_in$annot <- so_in$HPCA_main
  } else if (species == "mm10") {
    so_in$immgen_main <- RUN_SINGLEr(so_in, celldex::ImmGenData(), "label.main")
    so_in$immgen <- RUN_SINGLEr(so_in, celldex::ImmGenData(), "label.fine")
    so_in$mouseRNAseq_main <- RUN_SINGLEr(so_in, celldex::MouseRNAseqData(), "label.main")
    so_in$mouseRNAseq <- RUN_SINGLEr(so_in, celldex::MouseRNAseqData(), "label.fine")
    so_in$annot <- so_in$immgen_main
  }
  return(so_in)
}

[wong-nw]

Currently building a cache and will transfer to /data/CCBR_Pipeliner/remaining/PathNames/TBD/ to test retrieval. Will also look into how to add the cache path to the reference pull call, e.g. celldex::ImmGenData(<cachePath>)

[kelly-sovacool]

new error

Command error:
    |====================================================                  |  75%
    |                                                                            
    |======================================================================| 100%
  0%   10   20   30   40   50   60   70   80   90   100%
  [----|----|----|----|----|----|----|----|----|----|
  **************************************************|
  Using method 'umap'
  0%   10   20   30   40   50   60   70   80   90   100%
  [----|----|----|----|----|----|----|----|----|----|
  **************************************************|
  Error in `lock()`:
  ! Directory of lock file does not exist: '/home/sovacoolkl/.cache/R/ExperimentHub'
  Backtrace:
       x
    1. \-global MAIN_SINGLER(so_processed, species, celldex_cache)
    2.   +-global RUN_SINGLEr(so_in, celldex::HumanPrimaryCellAtlasData(), "label.main")
    3.   \-celldex::HumanPrimaryCellAtlasData()
    4.     \-celldex:::.create_se(...)
    5.       \-celldex:::.ExperimentHub()
    6.         \-ExperimentHub::ExperimentHub()
    7.           \-AnnotationHub::.Hub(...)
    8.             \-AnnotationHub:::.create_cache(...)
    9.               \-BiocFileCache::BiocFileCache(cache = cache, ask = ask)
   10.                 \-BiocFileCache:::.sql_create_db(bfc)
   11.                   +-base::tryCatch(...)
   12.                   | \-base (local) tryCatchList(expr, classes, parentenv, handlers)
   13.                   \-BiocFileCache:::.sql_connect_RW(.sql_dbfile(bfc))
   14.                     \-BiocFileCache:::.lock2(dbfile, exclusive = TRUE)
   15.                       \-filelock::lock(.sql_lock_path(dbfile), exclusive = exclusive)
  Error in `h()`:
  ! error in evaluating the argument 'conn' in selecting a method for function 'dbDisconnect': object 'info' not found

it's trying to use /home/sovacoolkl/.cache/R/ExperimentHub even though we set the cache...

[kelly-sovacool]

@wong-nw just like we were able to set a custom cache dir for gypsum, do you know if we can do the same for ExperimentHub?

[kelly-sovacool]

solution: add a process to fetch the celldex reference: https://github.com/nf-core/scdownstream/blob/dev/modules/local/celldex/fetchreference/main.nf

[wong-nw]

solution: add a process to fetch the celldex reference: https://github.com/nf-core/scdownstream/blob/dev/modules/local/celldex/fetchreference/main.nf

Initial fix to be built: Use the fetchreference concept from scdownstream to set working directory as cache location for local R environment import.

Followup cleanup fixes: Make modular by setting up downloads for accessibility via shared location.

[wong-nw]

@kelly-sovacool Have we completed this issue?

[kelly-sovacool]

@kelly-sovacool Have we completed this issue?

your fix for this issue was merged into #95, so once we complete #95 this issue will be automatically closed. i.e. this is still technically in progress until that PR gets merged.