Hierarchical Dorfman screening for robust pathway-aware feature selection identifies predictors of MEK-inhibitor response in NSCLC

Background: Gene expression predictors of targeted therapy response often exhibit strong within-pathway correlation, heavy-tailed noise, and contamination from batch effects. Such structure severely degrades performance of conventional feature selection methods (LASSO, elastic net, SIS) and existing group-regularized approaches including group LASSO, sparse group LASSO, and group SIS/AR2. Methods: We introduce Dorfman Screening, a computationally efficient hierarchical feature selection framework for pathway-structured genomic data. The method (1) groups genes into biological pathways using Hallmark annotations and dynamic tree cut clustering, (2) performs global pathway-level testing, (3) conducts within-pathway gene screening, and (4) applies a final regularized selection (Dorfman-LASSO or Dorfman-EN). Robust Dorfman variants additionally incorporate Huber-weighted regression to handle heavy-tailed errors, outliers, leverage points, and batch contamination. All tuning is data-driven through cross-validated RMSE. Results: In extensive simulations (p=1000, n=200) with varying correlation structures and severe contamination, Dorfman methods consistently outperformed LASSO/EN, SIS, SIS-LASSO, group-LASSO, sparse-group-LASSO, and group-AR2-gpLASSO. Dorfman-EN achieved the highest accuracy under strong correlations (ρ=0.8), while Dorfman-LASSO excelled at lower correlation levels. Robust variants showed marked resilience under nonlinear distortions and heavy-tailed noise, maintaining the lowest false discovery rates. Applied to Genomics of Drug Sensitivity in Cancer (GDSC) RNA-seq data for trametinib response in non-small cell lung cancer (NSCLC), Dorfman methods achieved substantially improved predictive accuracy (RMSE=2.41-2.45) compared to LASSO/EN (RMSE=2.53-2.59) and group-regularized methods (RMSE=2.63-3.66). When genes were stratified into literature-supported biomarker tiers, Dorfman selections were significantly enriched for high-confidence genes and revealed previously unreported candidates not discovered by competing methods. Conclusions: Dorfman Screening provides a scalable, robust, and biologically interpretable approach for pathway-structured genomic data, yielding improvements in both prediction performance and biomarker discovery for MEK-inhibitor response in NSCLC.