AI-augmented immersive 3D and 4D spatial analysis interface for cancer research

Background: Complex three-dimensional and time-resolved (4D) structures in cancer (cells, organoids, tissues, and drug distributions) are difficult to interrogate and discuss as a team. Our goal is to build an AI-augmented immersive interface that allows investigators to stand inside their data and explore it together in real time. We developed a multi-user VR/AR platform for volumetric microscopy that supports label-free holotomography (HT) (Tomocube HT-X1 Plus, 3D refractive-index images). The system enables interactive 3D segmentation, measurement, and annotation of subcellular and tissue-scale structures, including visualization of antibody drug conjugate (ADC) delivery. An integrated AI agent provides insight and can be augmented with domain-specific LLMs, including LG AI Research's EXAONE within a co-scientist AI agent framework. Methods: We implemented a VR/AR environment where users can manipulate, segment, and annotate volumetric datasets in real time. The interface supports gesture-based selection, dynamic adjustment of 3D rendering parameters, and synchronized multi-user viewpoints. Users can query the AI assistant for literature context, analysis guidance, or suggestions for follow-up measurements inside the scene. The platform takes HT data as well as other multi-channel datasets for real-time multi-view rendering. We adapt LG's EXAONE's LLM-based chatbot and co-scientist agent architecture, which is trained on biomedical literature, to further enhance in-session analysis. Results: We have validated the core visualization, interaction, and segmentation workflows and integrated the AI agent into the environment. Early demonstrations using 3D images of gastric cancer cell lines, organoids, and tissue sections allowed intuitive identification of nuclei, organelles, tumor glands, and stromal regions. Multiple users could concurrently explore morphology, annotate regions of interest, and derive quantitative metrics in real time. Ongoing data acquisition of gastric cancer cell lines, organoids, and tissues is coupled with manual annotation and AI-assisted segmentation to refine and benchmark performance. Conclusion: We present an AI-augmented immersive interface for interactive analysis of volumetric cell and tissue datasets. By combining multi-user VR/AR, advanced 3D analysis tools, and a conversational AI agent, this platform enables detailed exploration of subcellular architecture and tumor microenvironments directly from HT or related volumetric and molecular data. Although our initial focus is gastric cancer and ADC delivery, the framework is disease and modality-agnostic and can be extended to other tumor types and therapeutic modalities. Generative AI assistance was limited to language editing of this abstract. The scientific content, interpretation, and conclusions are the sole responsibility of the authors, who have reviewed and approved the final version.