Predict for multiple-mutants

[Joey-Xue]

Hello Pascal,
Thanks for your nice work and great contribution to the protein VEP community. I want to try ProteinNPT for multiple-mutant prediction but couldn't find details on how the train/test was set up and code to implement it
May I wonder how the results for multiple-mutants were obtained? Did you train the model on both single&double mutants and test on double, or only train a new model for double mutants which were splited into train/test randomly?
In
For example, in the Kermut paper they provided the performance using different strategies:

In the scripts/train_subs.sh I only see how to specify the fold_variable for single mutants like random, modulo and continuous, is there any way I can adapt it for multiple mutants? Or is there anyway to retreive the predictions of proteinNPT on multiple mutants as it seems they are not available on ProteinGym.
Thanks in advance for any of your kind help,
Zhengyuan

[pascalnotin]

Hi @Joey-Xue,

Thank you for the kind words, and apologies for the delayed response. In the ProteinNPT paper, the multiple mutant results were obtained by creating a random 5-fold CV split on all assays from ProteinGym v0.1 with multiple mutants. The contiguous and modulo schemes are challenging to adapt to the multiple mutant settings, so they were only used for single mutants.
With the random CV scheme, the training data can be comprised of both single and double mutants (or triple, quadruple, etc if present), and same thing for the test data. Thus, the closest setting in the Kermut paper is "1M/2M --> 1M/2M" (although the seeds are not the same so details may vary). I don't have access to the ProteinNPT predictions in the multiple mutant setting from back then anymore, but we made random CV schemes for all assays with multiple mutants available in the ProteinGym repo at the following location (only seed assignment, not model predictions): https://marks.hms.harvard.edu/proteingym/cv_folds_multiples_substitutions.zip

Best,
Pascal