Weakly supervised deep learning enables spatially resolved cell type inference from H&E histopathology

Characterizing the tumor microenvironment (TME) at spatial resolution typically requires specialized molecular assays and advanced imaging technologies, but remains inaccessible in most clinical settings due to the cost and complexity of single-cell and spatial transcriptomics. We present SLIDE-EX, a weakly supervised deep learning framework that learns morphology-expression relationships from routine hematoxylin and eosin (H&E) whole-slide images and generalizes to spatially resolved cell type inference without any explicit spatial supervision. SLIDE-EX is trained using only slide-level labels derived from deconvolved bulk RNA sequencing, consisting of one abundance value per cell type per slide, without localized annotations or spatial objectives. Despite this coarse supervision, we hypothesized that the model implicitly learns local morphological features that reflect underlying cellular composition. To test this hypothesis, we applied SLIDE-EX to Visium spatial transcriptomics data from breast cancer samples with expert pathologist annotations at spot-level resolution with a 55 micrometer diameter. SLIDE-EX predictions showed strong spatial concordance with annotated tissue regions, achieving area under the curve values of 0.82 for cancer cells, 0.85 for stromal cells, and 0.79 for lymphocytes. SLIDE-EX also exceeded the performance of HoVer-Net, a model explicitly designed for nucleus-level cell detection and classification, across all three cell types, where HoVer-Net achieved area under the curve values of 0.78, 0.71, and 0.65, respectively. These results demonstrate that slide-level supervision alone is sufficient to capture spatially informative morphological signals, enabling accurate local cell type inference at resolutions substantially finer than the training signal. Beyond spatial inference, SLIDE-EX robustly predicts cell type-specific expression of thousands of genes across nine cell types in an independent cohort with 160 samples, with predicted genes enriched for canonical cellular functions. Importantly, inferred cell type-specific expression improves prediction of neoadjuvant chemotherapy response relative to direct image-based and bulk expression models across two external cohorts. Together, these findings establish SLIDE-EX as a scalable and clinically applicable framework for spatially resolved TME characterization and treatment response prediction from routine histopathology.