In recent years, computer models have helped predict the shape or role of proteins. These models aid in finding targets for drugs or creating new antibodies to fight diseases. Built on large language models, or LLMs, these protein tools give accurate results on a protein's fit for a task. Yet, until now, no one could see inside these models to know how they decide or which protein traits matter most.
A new study by MIT researchers changes that. They applied a fresh method to peek into the "black box," the hidden inner workings of these models. This insight lets scientists pick better models for jobs like speeding up new drug or vaccine searches. The researchers note that the work boosts clarity in tasks using these models and might uncover new biology facts. The study is published in PNAS.
Opening the black box
Like LLMs, protein language models study amino acid sequences instead of words to forecast protein forms, roles, or drug bindings. Past uses include spotting stable viral protein parts for vaccines against flu, HIV, and COVID-19. But predictions stayed mysterious.
The researchers used sparse autoencoders to expand the way information is represented within a neural network. Sparse autoencoders increase the original representations to a much larger set of nodes. This expansion ensures that the data is spread out very thinly across these new nodes, meaning only a small fraction of them activate for any given input. As a result, each individual node ends up focusing on a single, specific feature of the data rather than multiple overlapping ones, which makes the entire system easier for researchers to interpret and understand.
After creating sparse views of many proteins, the researchers fed them to the AI model Claude, which matched nodes to known traits like function, family, or cell location. Claude described nodes in plain terms, such as spotting proteins that move ions across cell membranes.
This interpretability helps select or adjust models for tasks and might teach biologists more about proteins as models grow stronger.