Data integrity/consistency issue

[deepakchandra30]

Hi guys, @juntongshi48

When generating mixed-type data using the TabDiff model, I've noticed that repeated IDs do not have consistent characteristics. Is there a way to fix this issue and ensure that repeated IDs have the same attributes for certain columns in the generated data?

Thank you

[juntongshi48]

Hi deepakchandra30,

Sorry for this very late reply!

TabDiff treats each row independently, so it does not enforce dependencies between entries. Nevertheless, I currently have 2 ways in mind to address this issue.

First, you can post-process the synthetic data by mapping each unique ID to a single entry, such as the first generation or the median of all corresponding entries.

Moreover, you can explore relational generative models like RelDiff: Relational Data Generative Modeling
with Graph-Based Diffusion Models, which explicitly model and preserve dependencies between related entities like unique IDs.

Hope this will help you!
Juntong