An artificial intelligence domain-specialized scalable and clinically relevant pipeline to automate standardized O-RADS stratification for imaging reports in ovarian cancer

Purpose: Despite therapeutic advances, ovarian malignancies continue to carry a disproportionate mortality burden among women worldwide. Timely and accurate assessment of adnexal lesions is critical for improving outcomes, yet the disease is often diagnosed at an advanced stage due to subtle or misinterpreted early findings. The Ovarian-Adnexal Reporting and Data System (O-RADS) provides a standardized framework for malignancy risk stratification; however, in practice, its manual application can be time-consuming and prone to inter-observer variability, creating barriers to consistency and incorporation within clinical workflows. To address this urgent clinical need, we developed an artificial intelligence (AI) pipeline that automates O-RADS classification directly from free-text pelvic ultrasound reports. Procedures/Methods: By integrating Lingshu , a multimodal Domain-Specialized large language model (LLM) for medical reports reasoning, with traditional machine learning classifiers, our system transforms unstructured radiology narratives into structured, high-fidelity malignancy risk assessments, eliminating the need for manual scoring. We also compared the performance of this framework with that of an equivalent pipeline using MedGemma. Data/Results: We analyzed 413 de-identified pelvic ultrasound reports and extracted semantic embeddings using Lingshu. These embeddings, representing clinically meaningful linguistic patterns, were used to then train machine learning classifiers via a 5-fold cross-validation. Lingshu and a logistic regression model performed the best, achieving a mean accuracy of 0.773 ± 0.031, weighted precision of 0.777 ± 0.029, recall of 0.767 ± 0.028, F1-score of 0.765 ± 0.032, and a macro-averaged AUROC of 0.929 ± 0.019. Notably, this achieved a lower AUROC of 0.923 ± 0.027. Conclusion: A foundation model such as Lingshu demonstrates remarkable semantic understanding. We show that this model can be safely and effectively leveraged to standardize O-RADS risk assessment directly from unstructured ultrasound reports, bridging the gap between AI capability and real-world radiology practice. Our approach establishes a scalable and clinically impactful pathway for integrating AI into gynecologic oncology workflows, paving the way for broader adoption of LLM-driven tools in early ovarian cancer detection and risk stratification. Importantly, this AI-driven framework is not intended to replace expert radiologic judgment but to augment it by enabling the consistent application of O-RADS criteria, supporting diagnostic confidence, and reducing variability across providers and institutions. In doing so, it has the potential to expedite early identification of high-risk adnexal lesions and ultimately improve clinical decision-making and patient outcomes.