DIANNE: Segmentation-free localization of histology differential attributes

Pathologist-guided distinctions within histology images provide insights into tissue health, driving advances in understanding of disease mechanisms and clinical decision making. Digital pathology leveraging artificial intelligence is increasingly important to derive insights from histology and spatial omic images. To train computational models, current digital pathology methods rely on upfront manual annotations, which are time-consuming and difficult to scale. This pre-annotation process is also poorly suited for investigating novel spatial behaviors, where annotation is challenging and data requirements are unclear. To address these issues, we present DIANNE (Differential Image Annotator Environment), a digital pathology approach for rapid computation of spatial differential image attributes based on train-time Positive Class Mixup Augmentation (PCMA) of deep learning imaging features. DIANNE enables localization of differential attributes across whole slide images (H&E or antibody-based multiplex imaging, e.g. CODEX) in seconds on standard workstations, enabling interactive applications for exploratory investigation. Predictive models can be re-trained in real-time after interactive tile annotation changes, clarifying important biological attributes within and across tissue slides. We first test DIANNE on static histology images for sarcoma tumor detection. We demonstrate that classifiers can be developed with as few as 60 previously unannotated tumor slides to achieve high precision (0.86 ± 0.17) and recall (0.73 ± 0.26), as well as reasonable specificity (0.58 ± 0.3) and low false positives (0.0000 ± 0.0001). These metrics are comparable to more compute intensive pixel-based methods, e.g. [Segmenter. Strudel R et al. arXiv 2105.05633, 2021] (values 0.86 ± 0.15, 0.86 ± 0.22, 0.53 ± 0.28, 0.0062 ± 0.0129) which also required regional annotations, GPU and 6 hours of compute. Moreover, DIANNE has unique capabilities for interactive exploration of tissue slides. We show biological structure detection from human pancreatic, placenta and kidney tissue slides, including identification of staining and imaging artifacts. For kidney glomeruli detection we demonstrate tile-level recall, specificity, and false positives of 0.95, 0.95, 0.0002, respectively, based on interactive annotation of only 40 positive and 60 negative patches out of 2884 total patches. Just 9 out of 359 true glomeruli are missed, 7 of which were partially cropped. DIANNE provides multiple spatial inference workflows (interactive/static, H&E/molecular) implemented in a Jupyter widget-based toolkit. DIANNE enables efficient training of foundation model-based classifiers with only image scale annotation, rather than at the pixel-level. DIANNE's rapid, interactive system allows real-time training from static and exploratory input, providing a practical system for discovery of novel behaviors in spatial datasets.