HYBRID RADIOMICS: DEEP LEARNING ARCHITECTURE FOR COMPREHENSIVE MULTI-ORGAN LESION DETECTION AND CHARACTERIZATION

Abstract

Accurate and interpretable characterization of lesions across organ systems is a core challenge in the current fields of computational pathology and radiological AI. Most existing methods are unimodal, only capable of outputting raw image predictions or handcrafted radiomic features, and very few integrate standardized uncertainty quantification. Accordingly, this study proposes the HDF-RadNet hybrid diagnostic framework, which consists of two branches paired with a cross-modal attention fusion module. The first branch is a ViT B/16 visual transformer that extracts global semantic features from volumetric medical images. The second is a Radiomics-based radiomics engine that can extract 107 handcrafted features covering first-order statistics, gray level co-occurrence matrix (GLCM), wavelet transform, and shape descriptors. For interpretability, the framework uses Grad-CAM saliency maps plus SHAP value attribution to support clinically interpretable decision outputs. The model was validated on 3072 cases spanning CT, MRI, and mammography, sourced from 4 public datasets plus an in-house institutional dataset. Across all organs, the model achieved an AUC-ROC of 0.963 (95% CI: 0.951–0.975), a sensitivity of 93.7%, a specificity of 91.2%, and an F1-score of 0.928. Its performance improvement over baseline models was statistically significant with p<0.001 by DeLong’s test. Ablation experiments verified the independent and synergistic contributions of the two branches, confirming the framework has strong clinical translation value.