Accurate focal plane is crucial for AI assessment of non-monolayer urine cytology specimens for bladder cancer screening and surveillance

Background: Bladder cancer requires frequent surveillance, and urine cytology is widely used to guide cystoscopic evaluation. Digital image analysis aims to provide quantitative cell-level metrics aligned with The Paris System, but many laboratories use non-monolayer preparations (e.g., SurePath) that place cells in three-dimensional arrangements, complicating automated evaluation. We assessed how focal-plane selection affects nuclear-to-cytoplasmic (NC) ratio estimation and compared several algorithmic strategies for identifying the optimal focal plane. Methods: We analyzed 300 SurePath whole-slide images scanned on a Roche Ventana DP 200 system (Johns Hopkins), evenly spanning negative, atypical, suspicious, and high-grade carcinoma cases. A published detection model identified clusters, which were reannotated by six pathologists, yielding 343 clusters and 2,435 urothelial cells. Annotators outlined nuclei and cytoplasm areas at each cell's best-focus plane. We evaluated classical focus metrics (Sum of Modified Laplacian [SML], High-Frequency Energy [HFE], Tenengrad, Brenner Gradient, Laplacian, and entropy-based OpenCV methods) and unsupervised vision-transformer approaches (feature-variance [ViT-V], attention-entropy [ViT-A], and supervised Z-stack transformer models [ViT-T, ViT-CLS] that directly predict the focal plane). Algorithms were assessed by within-1-plane accuracy relative to pathologist ground truth. NC ratios were derived from nuclear/cytoplasmic areas. U-Net segmentation generated NC ratios for a held-out test set, and Spearman correlations with ground truth were evaluated using: (1) pathologist-selected planes; (2) off-plane images; and (3) algorithm-selected planes. Results: Within-1-plane accuracy was 0.416 (ViT-A), 0.594 (Grad), 0.740 (HFE), 0.779 (OpenCV), 0.789 (ViT-V), 0.853 (Laplacian), 0.857 (Tenengrad), 0.862 (SML), and highest for Z-stack transformers ViT-T and ViT-CLS (0.874, 0.872). NC-ratio estimation at the pathologist-selected plane reached a correlation of 0.774. Correlations decreased as images moved off-plane (~0.74 at ±1 plane; ~0.69 at ±2; ~0.65 at ±3; ~0.59 at ±4; ~0.50 at ±5). Using algorithm-selected planes, correlations were 0.639 (ViT-A), 0.687 (Grad), 0.720 (ViT-V), 0.728 (HFE), 0.733 (OpenCV), 0.743 (Tenengrad), 0.744 (Laplacian), 0.746 (SML), and 0.745/0.738 (ViT-T/ViT-CLS). Conclusion: Accurate focal-plane selection is essential for reliable AI-based cytologic assessment in non-monolayer urine preparations. NC-ratio accuracy and downstream analytic validity degraded quickly off-plane, while algorithm-selected planes recovered much of this loss. Future work will evaluate impacts on cluster- and patient-level tasks and assess extended-focus fusion methods that stitch the sharpest regions across the z-stack into a single optimally focused image.