Dyna-Q Reinforcement Learning for breast tumor malignancy classification using phantom-trained models applied to human data

Background: Obtaining large, well-annotated human datasets remains one of the major barriers to progress in biomedical AI. Ethical restrictions, patient burden, and institutional review processes limit access to clinical samples, making it difficult to train generalizable machine learning models. This issue is particularly acute in histopathology and tumor classification studies. Building on the concept of phantom-to-human transfer learning demonstrated by Das et al. (IEEE Sensors 2024), we propose a new Dyna-Q reinforcement learning framework that leverages tactile sensing to assess tumor malignancy. The model is trained exclusively on phantom data and applies a zero-shot knowledge transfer strategy to unseen human datasets, using interpretable mechanical features as the bridge between synthetic and biological domains. Methods: In this study, we utilized a Tactile Sensing System to generate images correlated with the mechanical properties (size, depth, and elasticity) of breast tumors. Tactile data were first collected from breast phantoms. Using these data, we developed a deep Dyna-Q learning model for tumor classification. A deep neural network served as the decision-making agent and was trained in a self-supervised manner to classify tumor malignancy. The training process relied exclusively on phantom-derived tactile features of breast tumors, using a total of 9,000 balanced phantom data samples. Subsequently, we collected tactile data from 40 human patients. The trained deep Dyna-Q reinforcement learning model was then tested directly on the unseen human data to classify breast tumor malignancy, with pathology results serving as the ground truth. Results: Despite the inherent domain gap between phantom models and human tissue, our approach achieved promising performance: an overall accuracy of 76.5%, sensitivity of 62.9%, and specificity of 73.5%. These findings indicate that clinically meaningful signals can be transferred across domains using tactile features, even without direct human training samples. By eliminating the need for retraining human data, this method substantially reduces reliance on scarce clinical datasets and offers a scalable solution for early-stage diagnostic modeling. Conclusions: This study highlights the feasibility of a tactile, reinforcement learning-based AI system for tumor classification in data-limited biomedical settings. By exploiting phantom data as a training substrate, the approach minimizes dependence on scarce clinical samples and provides an ethical, scalable framework for early diagnostic modeling. Beyond breast cancer, this paradigm could be extended to other organ systems where tissue-mimicking phantoms are available, offering a practical path toward cost-effective and generalizable AI-driven diagnostics.