Protein language models may learn more effectively when training includes the right amount of randomness, researchers at the University of Milan report.
Their study, published October 14 in PRX Intelligence, examines transformers trained on synthetic and natural protein sequences. The team used Langevin dynamics, a technique borrowed from statistical physics, to explore different settings of the networks’ internal weights.
An intermediate range of algorithmic temperatures produced strong learning performance. Here, temperature controls the sampling process; it does not mean physically heating a computer. Models sampled in this range predicted changes in protein free energy caused by mutations better than models trained through standard optimization.
The findings add a training perspective to AI’s growing role in protein design. They also expose a complication: settings that best predicted contacts between amino acids differed from those that best supported sequence learning.
The researchers identified a practical way to choose the size of the models’ internal representations, another factor affecting performance.
The work offers a route toward better protein models, while leaving a broader question open: whether comparable gains will hold across larger architectures and other biological prediction tasks.