Doubling suffixes in calculateDiffMeth

[J-Moravec]

In documentation, the suffix is described as

suffix A character string to append to the name of the output flat file database,
only used if save.db is true, default actions: The default suffix is a 13-character
random string appended to the fixed prefix “methylDiff”, e.g. “methylDiff_16d3047c1a254.txt.bgz”.
dbdir The directory where flat file database(s) should be stored, defaults to
getwd(), working directory for newly stored databases and to same directory for
already existing database

However, when I run calculateDiffMeth on my unite data, the fixed prefix is not methylDiff, but methylDiff_oldsufix, where oldsuffix is the suffix from the united dataset. The file from calculateDiffMeth thus looks like methylDiff_oldsuffix_suffix.txt.bgz.

This means that when running calculateDiffMeth(..., suffix = "foo") on a MethylBaseDB object stored in methylBase_bar.txt.bgz, the resulting object would be methylDiff_bar_foo.txt.bgz.

Honestly, it would be beneficial if I could specify the path directly. calculateDiffMethDSS doesn't support (according to documentation) storage in the DB format, but returns the MethDiff object, which (I assume?) can be converted to MethDiffDB using the makeMethylDB function. But makeMethylDB has no option to specify suffix.

[alexg9010]

Hi @J-Moravec ,

Thanks for bringing this up.

To give some context: The original idea of the suffix is that you have a unique random ID that allows you to trace the progress from the source tabix (usually a methylBaseDB object) to later-stage objects, e.g. after you merge all samples you have a source tabix file with a unique ID and any tabix files derived from it will maintain this ID throughout the analysis.

So, if you want to specify a custom ID or label to track your tabix files, using a suffix at the unite stage would be best. You do not need to add another suffix at later stages unless you want to record some additional information.

While specifying the output path would offer more flexibility, it could cause more trouble than it solves - especially since many users are not trained bioinformaticians or computational people. It opens the door to accidentally overwriting files, breaking naming patterns that downstream steps rely on, and generally making things harder to track and reproduce. Having a consistent, automatic naming scheme keeps things predictable, reduces errors, and helps make the whole workflow more robust and reproducible.

However, if you want, you can always retrieve the path to the underlying tabix file using getDBPath() and rename the resulting tabix file and index. Just make sure to keep the extensions (.txt.bgz and .txt.bgz.tbi) intact and recreate any object referencing the old path in the current R session.

Best,
Alex

[J-Moravec]

To give some context: The original idea of the suffix is that you have a unique random ID that allows you to trace the progress from the source tabix (usually a methylBaseDB object) to later-stage objects, e.g. after you merge all samples you have a source tabix file with a unique ID and any tabix files derived from it will maintain this ID throughout the analysis.

Yes, experimentally I have been able to find this out. But this is not mentioned anywhere in the documentation, see the above copy-pasted text.

---

The reason why I find the current rules harder than if I got an option to specify the full path myself (and why I disagree with your "it could cause more troubles than it solves)" is because the current rules are not documented and documentation is wrong.

1. According to the documentation pasted above, the name of a file should be methylDiff_suffix.txt.bgz, with [suffix] being by default some random 13 character suffix. Experimentally, I have found out that this is not true. Hence the issue.

2. There is nothing in the documentation specifying that if a file already exists, it gets _id appended to suffix, where id is integer from 1 to some value.

3. There is nothing in documentation about prefixes, yet I have some files with a random prefix preffix_methylDiff_suffix.txt.bgz and no clue where they came from.

and generally making things harder to track and reproduce

Being able to explicitly specify filepaths (or really, anything) makes things easier to track and reproduce.

---

The point here is: Documentation and implementation disagree, doubling suffixes shouldn't happen according to the documentation. Please fix code or the documentation.

[alexg9010]

Thanks for the detailed feedback — it’s much appreciated.

You’re right that the documentation doesn't clearly reflect the current behaviour around file naming. I’ll update the docs to clarify points 1 and 2 — specifically around the suffix handling and the auto-increment behaviour when files already exist.

For point 3 (the appearance of unexpected prefixes), I’ll need a bit more context or ideally a minimal reproducible example. That will help me track down where those prefixes might be coming from and whether it’s something internal or an edge case we haven’t accounted for.

Thanks again for pointing this out!

[J-Moravec]

@alexg9010 Thanks. I am reading the documentation (and readme) and some of the links could be updated as well (those pointing at google forums)

ad 3. Looks like those were created by the chunking behaviour within calculateDiffMeth, just not cleaned after terminating the run in the middle.