NormalDB FileBasedVidMapperException error

[HuZhuocheng]

Thank you for developing this excellent tool. I am currently encountering an issue when running NormalDB.R. I am using the Docker you provided, and my command is as follows:

Rscript /opt/PureCN/NormalDB.R --out-dir $OUT_REF \ --coverage-files $refDir/normal_coverages.list
--normal-panel $ponDB/pon_db
--assay panel
--genome hg38
--force

But an error occurs:
INFO [2025-09-20 07:58:54] Loading PureCN 2.10.0...
INFO [2025-09-20 07:58:54] Creating mapping bias database.
Error: FileBasedVidMapperException : contig_info_dict.HasMember("tiledb_column_offset") && contig_info_dict["tiledb_column_offset"].IsInt64()
Execution halted

I don't know why. I initially thought it was a GATK version issue, but I ran GenomicsDBImport using GATK in the Docker with the following command:

$gatk GenomicsDBImport -R $fa -L $list
--genomicsdb-workspace-path pon_db
-V omitted (here are VCF files of ten normal samples, generated by Mutect2)

But I still got the same error.

[HuZhuocheng]

Dear author, after two days of debugging, I have found that this might be a bug. My debugging code is as follows:

R -e "options(error = quote({traceback(20); q(status=1);})); source('/opt/PureCN/NormalDB.R')" \
    --args --out-dir /opt \
    --coverage-files $refDir/normal_coverages.list \
    --normal-panel pon_db \
    --assay panel \
    --genome hg38 \
    --force

From the error message, the problem appears to lie in the calculateMappingBiasGatk4 function call process.
Directly set a breakpoint in R on the calculateMappingBiasGatk4 function involved in the error, then step through the execution to observe:

R -e "
  options(error = quote({traceback(20); q(status=1);}));
  library(PureCN); 
  debug(calculateMappingBiasGatk4); 
  source('/opt/PureCN/NormalDB.R')
" --args \
  --out-dir /opt \
  --coverage-files $refDir/normal_coverages.list \
  --normal-panel pon_db \
  --assay panel \
  --genome hg38 \
  --force

From the debugging information, the error occurs during the genomicsdb::connect() call, specifically caused by the absence or incorrect type of the tiledbcolumnoffset field in the vidmap.json metadata file.

grep -A 50 "contigs" /opt/pon_db/vidmap.json

root@5ef07cb2a44d:/opt# grep -A 50 "contigs" /opt/pon_db/vidmap.json
  "contigs": [{
    "name": "chr1",
    "length": "248956422",
    "tiledb_column_offset": "0"
  }, {
    "name": "chr2",
    "length": "242193529",
    "tiledb_column_offset": "248956422"
  }, {
    "name": "chr3",
    "length": "198295559",
    "tiledb_column_offset": "491149951"
  }, {
    "name": "chr4",
    "length": "190214555",
    "tiledb_column_offset": "689445510"
  },

From the search results analysis, while the tiledbcolumnoffset field does exist in vidmap.json, the issue stems from data type mismatch: the values of tiledbcolumnoffset and length are stored as string types (e.g., "0", "248956422", with quotation marks), while the genomicsdb R package requires these fields to be strictly integer types (Int64, without quotation marks). This data type conflict constitutes the root cause of the FileBasedVidMapperException error.

So we need to batch modify the vidmap.json file, converting all values of tiledb_column_offset and length under the contigs array from strings to integers. I used the jq tool (a lightweight JSON processing tool) to quickly achieve this, with the following steps:

apt-get update && apt-get install -y jq
cp vidmap.json vidmap.json.bak
jq '.contigs[] |= (.tiledb_column_offset |= tonumber | .length |= tonumber)' vidmap.json.bak > vidmap.json

than the normalDB begein normal.

[madelonst]

@HuZhuocheng I also ran into this issue, I used GATK 4.6.2 for creating the pon_db where the "tiledb_column_offset" is written as a string which can't be read in. This happening is for GATK versions that have their GenomicsDB dependency at >= v1.5. The root cause seems to be that the writing functionality in GenomicsDB has been changed, but the reading functionality has not been adapted accordingly.

I reported this issue with GenomicsDB: https://github.com/GenomicsDB/GenomicsDB/issues/357

[HuZhuocheng]

Yes, my friend, I encountered the same issue. However, I temporarily solved it through programming. Specifically, I used the jq tool to process JSON files. From what I remember, there should be two modifications. Taking one of them, vidmap.json, as an example, it's necessary to convert all values of tiledb_column_offset and length under the contigs array from strings to integers.

You can back up the file before processing to prevent data corruption in case the code has issues. Here's my code:

jq '.contigs[] |= (.tiledb_column_offset |= tonumber | .length |= tonumber)' vidmap.json.bak > vidmap.json

[lima1]

Thanks both! The GenomicsDB guys are usually very fast in addressing issues. I’m a few versions behind with GATK, so haven’t noticed, but will update tests and Docker image soon I hope.