Automating clinical and pathological staging for breast cancer patients

Background: Accurate staging is essential for treatment selection, prognosis assessment, and trial eligibility. While prior studies attempted automating pathological (p) staging, few address clinical (c) staging and no studies have compared LLM-derived staging with clinician staging during clinical visit and retrospective cancer registry staging. Methods: We developed a multi-agent framework using Gemini-2.0-Flash-001 and GPT-5 to (1) identify reports (2) extract key data, and (3) apply AJCC 8 th edition staging criteria. LLM staging was compared with clinician documentation and registry staging in breast cancer patients. Clinician-registry agreement served as the reference standard. Two independent breast oncologists reviewed discordant cases. Results: We analyzed 122 randomly selected breast cancer patients across all three Mayo Clinic sites from 2018 - 2023. LLM performance matched or exceeded human inter-rater agreement for pathological staging (95-99.2% vs. 95-98.4%). Clinical staging was more challenging, with LLM concordance of 73-77.9% for cT and 87-89.3% for cN versus 87.8% and 91.0% clinician-registry agreement. Among 89 patients with clinician-registry concordance, LLM concordance was 98.9% for pT/pN, 79.8% for cT, and 91.0% for cN. Experts favored LLM staging in 27.8% (5/18) of cT and 25.0% (2/8) of cN discordances. Error analysis revealed LLM challenges in handling non-mass enhancements (6 cases), selecting radiologic measurements (2 cases), identifying discrete masses (2 cases), and miscellaneous (3 cases). Conclusion: LLMs achieved human-level performance for pathological staging (98%), supporting human-in-the-loop deployment. For clinical staging (79.8% cT, 91.0% cN), future work must enhance multimodal reasoning and integrate physical examination data. Prospective validation is needed to assess real-world impact. With targeted refinements and oversight, automated staging can transform registry workflows while augmenting clinical decision-making. Clinical-Pathological Staging Concordances Comparison pT pN cT cN LLM vs. Registry 96.7% 99.2% 73.8% 87.8% LLM vs. Clinician 95.9% 96.7% 77.9% 89.3% Clinician vs. Registry 95.1% 98.4% 87.8% 91.0%