Handling proteins with few or no MSA hits in proteinnpt

[yuzimu1]

proteinnpt requires a sufficient number of MSA sequences to generate embeddings and zeroshot auxiliary labels. What should we do if, for some proteins, no MSA is found, or only a few MSA sequences are retrieved? In those cases, can we still use this software, and if so, what are the recommended settings or workflows?

[pascalnotin]

Hi @yuzimu1 -- the MSA depth only has an impact on the quality of embeddings used for the initial protein sequence representation (as input to ProteinNPT). We use by default embeddings from MSA transformer, regardless of MSA depth, and obtain robust performance across the board (even on the proteins with "low MSA depth" from ProteinGym). The quality of these embeddings generally degrades with shallower MSAs though (see MSA Transformer paper) -- and instead of embeddings from MSA Transformer you could also use embeddings from other pLMs (we experimented with ESM and Tranception in the ProteinNPT paper). However, it is important to note that -- even if they do not explicitly use MSAs -- protein language models like ESM and Tranception also have degraded performance on protein families with shallower MSAs: since they are trained on the same databases typically used to retrieve the MSAs (eg., Uniref, Mgnify), their training data also tends be "shallower" for the same protein families, and it is not clear they are able to transfer learn across families more than MSA Transformer is able to do.