Code needs to be updated for Illumina MethylationEPIC-8 v2.0 arrays

[petermchale]

Running

python \
    -m methylprep \
    process \
    -d /scratch/ucgd/lustre-labs/quinlan/data-shared/Neklason_MethylationEPIC_20241008/data \
    --all

gives encouraging output initially:

INFO:methylprep.processing.pipeline:Running pipeline in: /scratch/ucgd/lustre-labs/quinlan/data-shared/Neklason_MethylationEPIC_20241008/data
INFO:methylprep.processing.pipeline:Found 1 additional fields in sample_sheet:
Sample_ID
Reading IDATs: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:51<00:00,  3.23s/it]

but then gives an error:

INFO:methylprep.processing.pipeline:Found 1 additional fields in sample_sheet:
Sample_ID
Reading IDATs: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:51<00:00,  3.23s/it]
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/__main__.py", line 6, in <module>
    cli_app()
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/cli.py", line 678, in cli_app
    build_parser()
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/cli.py", line 82, in build_parser
    parsed_args.func(func_args)
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/cli.py", line 276, in cli_process
    run_pipeline(
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/processing/pipeline.py", line 321, in run_pipeline
    idat_datasets = parse_sample_sheet_into_idat_datasets(sample_sheet, sample_name=batch, from_s3=None, meta_only=False, bit=bit) # replaces get_raw_datasets
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/models/sigset.py", line 106, in parse_sample_sheet_into_idat_datasets
    idat_datasets[idx]['array_type'] = ArrayType.from_probe_count(n_snps_read)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/scratch/ucgd/lustre-labs/quinlan/u6018199/methylation-tools/.venv/lib/python3.11/site-packages/methylprep/models/arrays.py", line 46, in from_probe_count
    raise ValueError(f'Unknown array type: ({probe_count} probes detected)')
ValueError: Unknown array type: (1105209 probes detected)

The technology is

Illumina MethylationEPIC-8 v2.0 arrays

[petermchale]

c.f., #122

[eliopato]

Hi, indeed this package doesn't support the latest version of EPIC arrays. The maintainers don't seem to be active anymore unfortunately. To use the update mentioned in #122 , you need to uninstall this package and manually install mine, using the instructions provided here https://github.com/eliopato/methylprep?tab=readme-ov-file#installation
Note that I have only added the EPIC v2 manifest so that data can be read, but I haven't tested all methylprep functionalities. I hope it's helpful though!