Clinical Evaluation of AI-Assisted Diagnostic Tools in Multispecialty Medical Practice

Abstract

Background: In spite of the digital health development, diagnostic errors and clinician burnout have become serious issues in the multispecialty medical practice. Even though Artificial Intelligence (AI) demonstrated potential in the controlled in-silico setting, the most important gap in knowledge is its lack of clinical applicability in the in-vivo setting and the possibility of being implemented into various specialty practices at once. Goal: The present study will compare the accuracy in diagnoses, clinical efficacy, and the acceptance of a multispecialty AI-assisted diagnostic tool with the conventional clinical practice. Methods: This was a prospective observational study carried out in three different departments, namely Radiology, Cardiology, and Dermatology. The AI intervention became a part of the clinical routine as a second opinion decision support system. A multidisciplinary gold standard consensus diagnostic performance was used as a measure of diagnostic performance. Diagnostic accuracy (sensitivity and specificity) was the primary outcome, with time-to-diagnosis and the level of confidence in the clinician being the secondary outcomes, which were measured using Likert scales. Findings: 200 encounters of patients were studied. The AI + Clinician model was found to be more sensitive than clinicians alone; Radiology (95% vs 88%), Cardiology (93% vs 91%), and Dermatology (90% vs 82%). It is important to highlight that the AI-aided workflow decreased the average time-to-diagnosis by 2.2 minutes per encounter. Although there were more cases of clinician confidence in complicated cases, cases of automation bias were evident among junior residents, especially when conducting borderline dermatological tests. Conclusion: AI-based diagnostic systems have a considerable positive impact on the diagnostic sensitivity and efficiency in a multispecialty environment. Nevertheless, a specialty-based performance variance demonstrates the necessity to integrate the strategies in specialties. These results imply that although AI is already fit to support clinical work, it needs strong supervision in order to reduce over-dependence and provide high-quality human-in-the-loop decision-making.