Researchers have developed a new AI tool that interprets medical images with unprecedented clarity and could speed up disease diagnosis and image interpretation.

The tool, called iStar (Inferring Super-Resolution Tissue Architecture), was developed by researchers at the Perelman School of Medicine at the University of Pennsylvania.

The researchers say the iStat tool would allow doctors and researchers to see cancer cells that might otherwise have been virtually invisible. It could also determine whether safe margins were achieved through cancer surgeries and automatically provide annotation for microscopic images, paving the way for molecular disease diagnosis at that level.

A paper on the method was published today in Nature Biotechnology.

Deciding when immunotherapy is needed

Mingyao Li, PhD, a professor of Biostatistics and Digital Pathology, said iStar can automatically detect critical anti-tumor immune formations called “tertiary lymphoid structures,” a patient’s likely survival and favorable response to immunotherapy. This means, Li said, that iStar could be a powerful tool for determining which patients would benefit most from immunotherapy.

Li and her colleagues trained a machine-learning tool called the “Hierarchical Vision Transformer” on standard tissue images. It breaks down images into different stages, and these images are used to predict gene activities, often at near-single-cell resolution.

This research was funded by the National Institutes of Health (R01GM125301 and R01HG013185).

Citation: Zhang, D., Schroeder, A., Yan, H. et al. Inferring super-resolution tissue architecture by integrating spatial transcriptomics with histology. Nat Biotechnol (2024). https://doi.org/10.1038/s41587-023-02019-9

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.