How to calculate RMSE (pK) in FEP

[yufengwhy]

How to calculate RMSE (pK) in FEP?
There are differences between paper and code
What is the "percentage" in paper ?
What is the meaning of "/ 1.379 - 9" in code?
In paper:

pK was calculated using pK = − log 10 K , where K is a percentage out of 100

In code:

pic50_exp = -float(ligand_info["exp_dg"]) / 1.379 - 9
opls4 = -float(ligand_info["pred_dg"]) / 1.379 - 9
errors.append(pic50_exp - opls4)

[BFeng14]

Hi, thank you for your interest on Actfound, the experimental results of FEP is downloaded from
https://github.com/schrodinger/public_binding_free_energy_benchmark , and their meaning is Gibbs free energy and their unit is kcal/mol.

To keep consistance with other parts of experiments which use pK for calculating RMSE, we use the Gibbs-free-energy isotherm equation (delta_G=-RTlnK) to convert the Gibbs free energy (with unit of kcal/mol) into pK, and it can be simply written in pK= - delta_G/ 1.379 in the room temperature, where K have the unit of mol.

As for the last "- 9" appeared in the code, it is only used to make the unit of K consistant with the training data (cause training data mostly have unit of nmol=10^-9mol for K). This constant term "- 9" will make no difference for Actfound, as our model is trained to predict the relative activity difference between compounds. However, losing this term will make some comparative methods which predict the absolute activity behave very bad.

The "percentage" in paper only used for bioassays with the unit of percentage "%", which only appeared in the figure 2.g-h. You can goto the "Bioactivity definition" part of our paper for details~

Thank you again for your interest on Actfound, please let me know if you have any further questions.

[yufengwhy]

@BFeng14 two questions:

1. how is 1.379 calculated ? it is 1.364 ?
2. it performs good when i use −log_10 Ki as label to train; but it performs worse when i use log_10 Ki as label to train, Could you kindly help explain the reason ?

[BFeng14]

1. Yes, it should be 1.364 in the temperature of 298.15K ( (1.9872036*10^-3*298.15) * math.log(10) ), and I think that 1.379 occured in the code should be a calculating error, thank you for your correcting, I will fix this later and report the RMSE in the unit of Kcal/mol in this git.
2. Lower Ki, Kd, IC50 means better activity, and higher −log_10 Ki also means better activity. The training data are organized as −log_10 K, so we should also use the same setting during testing.