Leveraging large language models to classify pathology reports into ontological hierarchies

Free-text pathology reports vary in style, terminology, and uniformity, making consistent diagnostic extraction difficult for research cohort development. This issue is amplified in large archives, where evolving classifications and intentional ambiguity hinder standardized, categorical diagnoses. Structured ontologies such as OncoTree address this by enforcing codes. While OncoTree is used here as an example, the approach generalizes to any hierarchical ontology, including current WHO classifications. Using cutting-edge LLMs, we developed OncoPath, an application that classifies pathology reports within structured ontologies. We tested OncoPath by applying OncoTree hierarchy through a structured tree-walk, progressing from broad anatomic categories to specific subtypes. High-level categories are assessed first, then plausible child nodes, with each node's children displayed so the model can anticipate downstream options. Diagnostic paths are summarized, and the model selects the optimal code with a confidence tier (<50%, 50-90%, >90%). Multiple configurations of LLaMA 3.3 70B Instruct were evaluated. Surgical and hematopathology reports were processed separately, and accuracy was verified by pathologist review against the 10 most prevalent molecular codes, covering 49.6% of surgical and 84.9% of hematopathology cases. The classifier mapped diagnoses to OncoTree codes with high accuracy. Heme-based coding worked well with the full report, whereas Surg-based coding performed best with a targeted summary because reports often contained multiple parts from different sites and diagnoses. Heme reports reached 90-100% accuracy; Surg codes reached 80-100%, except HGSOC (high grade serous ovarian carcinoma) at 60%, reflecting the inherent difficulty, even for pathologists, of determining the tumor's primary site of origin. Codes were stored with original reports, enabling direct reuse in research and operations. The traversal approach provided auditability, confidence levels, and automatic re-evaluation when diagnoses changed.An LLM-assisted OncoTree traversal provides a scalable method to convert diverse pathology reports into structured codes. By standardizing reports while preserving diagnostic nuance, OncoPath enables retrospective cohort development and analyses of incidence, progression, and practice patterns. This framework can readily extend to other structured disease ontologies, establishing a generalizable approach for transforming unstructured reports into standardized, reusable data.