Recently, Professor Yu Zhangsheng's team, the company's founder, and their collaborators published an article titled "..." online in the Cell sub-journal *Cell Reports Medicine*.“Harnessing TME depicted by histological images to improve cancer prognosis through a deep learning system”The research findings include the development of a deep learning system that can predict tumor microenvironment information for cancer patients without spatial transcriptome data using histopathological images, thereby achieving accurate cancer prognosis and significantly expanding the use of gene spatial expression information in large-scale biomedical pathology image public databases.

Figure 1. Schematic diagram of this research work

Prognostic prediction for cancer patients remains a significant challenge in clinical practice. The tumor microenvironment is crucial for the occurrence, evolution, and metastasis of solid tumors. A growing body of research reveals the correlation between the tumor microenvironment and cancer prognosis and treatment selection. Spatial transcriptomics can characterize the tumor microenvironment from the perspective of spatial gene expression, distinguishing different prognostic subgroups of cancer patients. However, the high cost and long experimental cycles of spatial transcriptomics hinder its application in large-scale cancer patient cohorts for survival prediction. Easily accessible histopathological images in clinical settings provide rich information on tumor morphology. Developing artificial intelligence models to predict molecular-level spatial gene expression levels based on these images, thereby characterizing the tumor microenvironment, holds promise for achieving more accurate cancer prognosis.

This research aims to develop a deep learning system that uses histopathological images to predict high-dimensional spatial gene expression levels in corresponding regions. This overcomes the limitations of current spatial transcriptome data, such as high cost and limited sample size. The system will characterize the tumor microenvironment in large-scale cancer cohorts that only have pathological image data and lack spatial transcriptome data, thereby improving the accuracy of cancer patient prognosis. The deep learning system consists of two parts: the first part is a spatial transcriptome expression level prediction model based on convolutional neural networks and graph neural networks (IGI-DL); the second part is cancer survival prognosis prediction based on tumor microenvironment information depicted by spatial gene expression.

The constructed IGI-DL model integrates the advantages of convolutional neural networks and graph neural networks, making full use of pixel intensity and structural features in histopathological images to achieve more accurate prediction of gene spatial expression levels. The model performs well in three types of solid tumors: colorectal cancer, breast cancer, and cutaneous squamous cell carcinoma, with an average correlation coefficient improvement of 0.171 compared to five existing methods.

Figure 2. Predictive performance and visualization of gene spatial expression using the IGI model in colorectal cancer samples.

Furthermore, the IGI-DL model was applied to infer the spatial expression of genes from histopathological images, constructing a super-patch graph for predicting the survival prognosis of cancer patients. The results showed that using the gene spatial expression predicted by IGI-DL as node features in the super-patch graph improved the performance of the survival prognosis model in the TCGA dataset's breast cancer and colorectal cancer cohorts, with five-fold cross-validation C-indexes of 0.747 and 0.725, respectively, outperforming other survival prognosis models. This survival prognosis model also maintained accuracy in predicting the prognosis of early-stage patients (stage I and II), and the predicted risk score can serve as an independent prognostic indicator for patients at all stages and for early-stage patients. In the external test set MCO-CRC, which contains data from over a thousand patients, the survival prognosis model maintained a stable advantage and demonstrated generalization ability.

Figure 3. Performance of the survival prognosis model on internal and external test sets.

This research was supported by the National Natural Science Foundation of China, the Shanghai Science and Technology Commission Fund, and the Shanghai Jiao Tong University "Medical-Engineering Interdisciplinary Research Fund." We also thank the Network Information Center of Shanghai Jiao Tong University for providing the supercomputing platform.