PO.BCS01.07 · 生物信息与计算
FLEXMIL:用于临床与转化研究的灵活多模态多示例学习框架
FLEXMIL: A flexible multimodal multiple instance learning framework for clinical and translational research
作者与单位 Authors & Affiliations
摘要 Abstract
中文摘要
背景:多示例学习 (MIL) 为在缺乏详细注释的环境中训练深度神经网络提供了强大的框架,特别是在数字病理学中,玻片层面的标签通常可获得,但区域或切片层面的注释稀缺。然而,大多数现有的MIL工具包存在维护有限、功能不足和许可限制性强的问题,阻碍了其在临床转化研究中的采用。
方法:我们提出FLEXMIL,一个灵活的端到端MIL框架,旨在统一病理学、组学和临床数据分析。FLEXMIL用Python (V3.10) 开发,以实现生存分析、分类和回归,具有稳健的交叉验证、自动数据分割和标准化报告,确保可重复性。实现了一个共注意力融合模块来整合包括临床信息、组织病理学图像和组学数据在内的多模态数据,同时也支持单模态实验。FLEXMIL生成玻片层面的预测、患者层面的汇总和基于注意力的热图,以促进生物标志物发现、可视化解释和假设生成。
结果:我们在多种转化应用案例中评估了FLEXMIL,包括生物标志物发现、肿瘤靶点表达预测、生存分析,并展示了其多功能性和稳健性能。重要的是,当FOLR1相关转录组特征与从H&E染色图像中提取的特征整合时,观察到了显著的进步,使FOLR1表达水平的预测性能得到增强。具体而言,这种多模态方法不仅将二分类的AUC从0.72提升至0.83,还将TCGA-LUAD (N=460) 中连续值预测的相关系数从0.5提升至0.78。同样的方法进一步将商业队列 (N=69) 中预测的FOLR1蛋白表达的AUC从0.78提升至0.83,并增强了TCGA-LUAD (N=334) 中的总生存期预测 (C-index从0.59增加至0.62),优于仅基于图像数据的模型。在生物标志物识别任务中,我们在85个TCGA-TNBC样本中预测了肿瘤浸润淋巴细胞 (TIL),其%TIL由两位病理学家注释。在10%的TIL阈值下,FLEXMIL达到了0.89的AUC,凸显了组织病理学特征的强预测价值。此外,FLEXMIL可以为仅图像和多模态(共注意力)模型生成注意力热图,支持多头视图和转录组特征引导的解释,以实现透明的玻片层面可解释性。
结论:FLEXMIL提供了一个灵活、可扩展且可解释的平台,连接了计算建模与临床洞察,推动了用于精准肿瘤学的整合生物标志物的开发。
查看英文原文 English abstract
Background: Multiple-instance learning (MIL) provides a powerful framework for training deep neural networks in settings lacking detailed annotations, particularly in digital pathology, where slide-level labels are routinely available, but region- or tile-level annotations are scarce. However, most existing MIL toolkits suffer from limited maintenance, functionality, and restrictive licenses, hindering adoption in clinical-translational research.
Methods: We present FLEXMIL, a flexible, end-to-end MIL framework designed to unify pathology, omics, and clinical data analysis. FLEXMIL was developed in Python (V3.10) to enable survival analysis, classification, and regression with robust cross-validation, automatic data splitting, and standardized reporting to ensure reproducibility. A co-attention fusion module was implemented to integrate multimodal data including clinical information, histopathology images and omics data, while also supporting single-modality experimentations. FLEXMIL generates slide-level predictions, patient-level summaries, and attention-based heatmaps to facilitate biomarker discovery, visual interpretation and hypothesis generation.
Results: We evaluated FLEXMIL across diverse translational use cases, including biomarker discovery, tumor target expression prediction, survival analysis and demonstrated its versatility and robust performance. Importantly, a significant advancement was observed when FOLR1-related transcriptomic signatures were integrated with features extracted from H&E-stained images, leading to enhanced predictive performance for FOLR1 expression levels. Specifically, this multimodal approach boosted not only the AUC from 0.72 to 0.83 in binary classification, but the correlation coefficient for continuous values predicted from 0.5 to 0.78 in TCGA-LUAD (N=460). Same approach further improved the AUC of the predicted FOLR1 protein expression from 0.78 to 0.83 in a commercial cohort (N=69) and enhanced overall survival prediction (C-index increased from 0.59 to 0.62) in TCGA-LUAD (N=334), outperforming models based on image data alone. In the biomarker identification task, we predicted tumor-infiltrating-lymphocytes (TILs) in 85 TCGA-TNBC samples with %TIL annotated by two pathologists. At a 10% TIL threshold, FLEXMIL achieved an AUC of 0.89, highlighting the strong predictive value of histopathology features. In addition, FLEXMIL can generate attention heatmaps for image-only and multimodal (co-attention) models, with support for multi-head views and transcriptomic signature-guided explanations to enable transparent slide-level interpretability.
Conclusions: FLEXMIL provides a flexible, scalable and interpretable platform that bridges computational modeling and clinical insight, advancing the development of integrative biomarkers for precision oncology.
利益披露 Disclosure
M. Guan, None..
Q. Zhou, None..
S. Lata, None..
D. Soong, None..
M. Lechpammer, None..
C. Thalhauser, None..
H. Si, None.