PO.BCS02.04 · 生物信息与计算
整合基于图像和文本的AI改进从全切片病理图像中识别转移部位
Integrating image and text-based AI improves identification of metastatic sites from whole-slide pathology images
作者与单位 Authors & Affiliations
摘要 Abstract
中文摘要
转移性癌症占癌症相关死亡的大多数，确定肿瘤最可能的转移目的地对预后和治疗规划至关重要。尽管每个肿瘤都源自一个原发部位，但其生物学行为差异很大：有些保持局限，而另一些则遵循公认的器官特异性趋向性模式扩散至远处器官。在临床实践中，区分这些行为依赖于全切片图像（WSI）、免疫组化和临床信息。然而，转移病灶常缺乏独特的组织学特征，使得难以确定肿瘤是否已转移，以及若已转移其扩散部位——尤其在低分化活检中。我们假设肿瘤保留着微妙的形态学线索——根植于转移潜能和器官特异性趋向性——这些线索可帮助区分局限性和转移性表型，并且当存在转移时可指示可能的扩散部位。借助计算病理学和基于AI的模型，这些潜在特征可被揭示，以改善肿瘤分层和转移部位预测。我们提出一种新颖的基于图像和文本的AI模型，分析WSI内的每个区域以确定转移状态和部位。我们的模型利用由预训练病理AI模型生成的具有医学意义的文本描述——文本原型。在我们对六个临床相关部位、3,804例转移病例的研究中，每张WSI使用预训练病理AI模型转换为patch级图像特征。在识别出转移性疾病后，模型将每个patch与转移模式的简明文本描述（如淋巴结转移）进行比较。视觉-文本相似度矩阵量化每个patch与这些描述的匹配程度，将注意力引导至最能指示转移部位的区域。我们的模型实现了88%的AUC、74%的准确率和60%的宏F1值。这些发现表明，通过提供转移特异性语义线索、将注意力引导至诊断上重要的区域，改善了转移部位分类。我们相信我们的模型仅使用常规病理切片即可提示最可能的转移部位，并为AI辅助诊断提供了一种实用、可扩展的策略。
查看英文原文 English abstract
Metastatic cancer accounts for most cancer-related mortality, and determining the most likely metastatic destination of a tumor is essential for prognosis and treatment planning. Although every tumor arises from a primary site, its biological behavior varies widely: some remain localized, whereas others spread to distant organs following well-recognized patterns of organ-specific tropism. In clinical practice, distinguishing these behaviors relies on whole-slide images (WSIs), immunohistochemistry, and clinical information. However, metastatic lesions often lack distinctive histological features, making it difficult to determine whether a tumor has metastasized and, if so, its site of spread-especially in poorly differentiated biopsies. We hypothesize that tumors retain subtle morphological cues-rooted in metastatic potential and organ-specific tropism-that can help distinguish localized from metastatic phenotypes and, when metastasis is present, indicate the likely site of dissemination. Leveraging computational pathology and AI-based models, these latent features can be uncovered to improve tumor stratification and metastatic site prediction. We propose a novel image-and-text-based AI model that analyzes each region within a WSI to determine metastatic status and site. Our model utilizes medically meaningful textual descriptions-textual prototypes-generated by a pre-trained pathology AI model. In our study of 3,804 metastatic cases across six clinically relevant sites, each WSI was converted into patch-level image features using a pre-trained pathology AI model. After identifying metastatic disease, the model compares each patch with concise text descriptions of metastatic patterns (e.g., lymph node metastasis). A visual-textual similarity matrix quantifies how closely each patch matches these descriptions, guiding attention toward the regions most indicative of the metastatic site. Our model achieves an AUC of 88%, an accuracy of 74%, and a macro-F1 of 60%. These findings demonstrate improved metastatic-site classification by providing metastatic-specific semantic cues that direct attention to diagnostically important regions. We believe our model can suggest the most likely metastatic site using only routine pathology slides and offers a practical, scalable strategy for AI-assisted diagnosis.
利益披露 Disclosure
Y. Chen, None..
Z. Su, None..
M. K. Niazi, None..
E. Hasanov, None.