Result sensitivity to TF & ANARCI library version (silent failure) #2
Description
Hi,
I have noticed that the AbDev prediction results are sensitive to the versions of TensorFlow and ANARCI used. For example, using TensorFlow 2.13.0 gives predictions with very low correlations with the provided results, despite using the same notebook workflow. This only affects the DeepSP and not the AbDev models.
For context: using the correct TensorFlow version but wrong ANARCI gives ~0.95 Spearman correlation, while having both versions wrong drops it to ~0.3 correlation.
Proposed solution:
- Provide pinned versions of all required libraries (particularly TF and ANARCI), the provide yml in DeepSP repo works for me (TF 2.12 ANARCI 2021.02.04)
- Highlight this silent failure case in the readme & importance of correct library versions
- Long-term fix: upgrade to ANARCII which provides more consistent numbering and easy pip install