The Transparency Problem Deep neural networks (DNNs) have shown strong diagnostic performance for melanoma, but their black-box nature is a critical barrier to clinical adoption. The EU's General Data Protection Regulation (GDPR) requires all algorithm-based decisions to be interpretable by end users. Clinicians are reluctant to use AI systems they cannot understand, particularly because DNNs can incorporate spurious correlations - making decisions based on irrelevant image features like rulers or skin marks rather than genuine diagnostic criteria.
Current XAI Limitations The two main types of explainable AI (XAI) methods are post-hoc algorithms (like Grad-CAM) that explain decisions retrospectively, and inherently interpretable algorithms designed to be understandable from the outset. Post-hoc methods dominate the field but suffer from an 'interpretability gap' - they show where the AI looked, but not why that location was relevant. This gap creates risk of confirmation bias when clinicians interpret highlights that may or may not contain the features driving the decision.
The Solution: Dermatologist-Like XAI This study introduces a multimodal XAI system that provides domain-specific, ontology-based explanations aligned with how dermatologists actually describe melanoma. Trained on 3,611 dermoscopy images annotated by 14 international dermatologists, the system outputs both a diagnosis and specific dermoscopic characteristics (from a defined ontology) with their localized regions. A three-phase reader study with 116 clinicians then measured the impact on diagnostic accuracy, confidence, and trust.
Dermoscopic Ontology A structured ontology of melanoma and nevus criteria was compiled from established dermoscopic literature (Table 1 in the paper). Melanoma criteria included thick reticular or branched lines, eccentrically located structureless areas, grey patterns, polymorphous vessels, pseudopods/radial lines not occupying the entire circumference, and black dots/globules in the periphery. Nevus criteria included only one pattern and one color, symmetrical combinations, and monomorphic vascular patterns. The ontology grounded explanations in recognized clinical concepts.
XAI Architecture A ResNet-50 backbone was used, with training guided by the dermatologist-annotated ontological features and their spatial locations. The model first predicts the presence of each individual characteristic, then infers melanoma if at least two melanoma characteristics are detected. Explanations are output as both a textual description of detected criteria and localized visual overlays (white polygons) on the lesion image. This multimodal output closes the interpretability gap by specifying not just where the AI looked, but which dermoscopic feature it detected there.
Three-Phase Study Design Phase 1 (no AI support): clinicians diagnosed 196 images and recorded which dermoscopic features they detected. Phase 2 (AI support without explanations): clinicians received the AI's diagnosis alongside their own. Phase 3 (XAI support with explanations): clinicians received both the diagnosis and the XAI explanations with localized feature maps. After each phase, participants reported diagnostic confidence and trust in the AI on Likert scales. This design isolated the incremental effect of explanations beyond AI diagnosis alone.
Diagnostic Accuracy The XAI achieved 81% balanced accuracy on the test set (95% CI: 75.6-86.3%), compared to 80% for the standard baseline ResNet-50 classifier. The attention-based state-of-the-art approach achieved 79%, and the ISIC 2020 challenge-winning ensemble achieved 81.5%. The XAI thus provides additional interpretability without compromising diagnostic performance - directly challenging the widely assumed performance-interpretability trade-off.
Lesion-Focused Attention A critical measure of reliability is whether the AI focuses on the lesion itself rather than surrounding artifacts. The XAI achieved a mean Grad-CAM attention ratio (inside vs. outside lesion) of 35.9, compared to only 4.1 for the baseline classifier (p less than 0.0001). The baseline was frequently distracted by background features - artifacts that are known to drive spurious melanoma predictions in prior studies. The XAI's superior lesion focus demonstrates it avoids learning false associations.
Alignment with Clinician Explanations Mean XAI-clinician overlap in feature selection was 0.27 (Sorensen-Dice coefficient), comparable to the inter-clinician agreement of 0.28 among human experts evaluating the same images. For lesions classified as melanoma, XAI-clinician overlap was 0.46. ROI overlap (spatial agreement in attention regions) was 0.48 for the XAI versus 0.39 for the baseline (p less than 0.0001). The XAI reasons about dermoscopy images in a way that closely mirrors how experienced dermatologists reason.
Impact on Accuracy AI support (Phase 2) significantly improved clinician accuracy from 66.2% to 72.3% balanced accuracy (p less than 0.0001). XAI support (Phase 3) produced a slight further increase to 73.2%, but this difference was not statistically significant (p = 0.34). Adding explanations did not meaningfully change the number of correct diagnoses beyond what AI alone provided. However, more experienced clinicians who engaged in regular scientific dermoscopy discussions showed the largest accuracy gains from XAI, while less experienced clinicians benefited more from plain AI.
Impact on Confidence Diagnostic confidence increased significantly with XAI support over AI-only support by a mean of 12.25% (p less than 0.0001). The XAI also showed a calibration effect: when the AI expressed high certainty, clinicians themselves were slightly more confident (mean 7.82 vs. 7.69 for uncertain predictions, p = 0.039), an effect absent in the AI-only phase where certainty was not communicated.
Impact on Trust Trust in the machine's decisions increased by a mean of 17.52% with XAI support compared to AI-only (p less than 0.0001). Clinicians were substantially more likely to trust the system when they agreed with its diagnosis (trust score 7.55 when agreeing vs. 4.8 when disagreeing, p less than 0.0001). Importantly, the XAI's communicated confidence level did not significantly affect trust scores, suggesting that explanation quality matters more than expressed certainty.
Overlap Drives Trust for Melanoma Diagnoses A key finding was that clinician trust in the XAI was correlated with the degree of overlap between the XAI's feature explanations and the clinician's own feature selections. For melanoma diagnoses where both agreed, the correlation between explanation overlap and trust was 0.23 (p less than 0.0001). This confirms the intuition: clinicians trust a system more when it identifies the same reasons they would give for a diagnosis.
No Trust-Overlap Correlation for Nevus The correlation between explanation overlap and trust was absent for nevus diagnoses. Analysis revealed this was likely because the XAI frequently cited 'melanoma simulator' as its nevus explanation - an explanation few clinicians selected. This specific ontological mismatch reduced trust alignment for benign predictions, illustrating how ontology design and calibration directly shape user experience.
Implications for AI Adoption Across all conditions, clinicians who received explanations were more likely to agree with the machine (79.5% agreement in XAI phase vs. 77.1% in AI-only phase, p = 0.009). This increased agreement occurred even for erroneous AI predictions - a subtle risk where explanations may encourage inappropriate deference to a wrong AI decision, though this effect did not reach statistical significance. These findings underscore that trust and accuracy are distinct outcomes that must both be monitored.
Trade-off Between Interpretability and Discovery By training the XAI to explain decisions using established dermatologic concepts, it sacrifices the ability to discover and explain novel patterns that clinicians do not yet recognize. The authors explicitly acknowledge this limitation: in safety-critical settings, they believe that human-verifiable explanations provide more benefit than potentially superior but uninterpretable pattern discovery. Future hybrid approaches might allow AI to propose new explanatory concepts beyond the current ontology.
Domain Specificity Requirements The approach requires a well-established clinical ontology to exist for the diagnostic task. This is available for many domains (melanoma, Gleason scoring in prostate cancer) but not all. Additionally, clinicians must be trained in the specific ontology used - the study found that clinicians familiar with its dermoscopic criteria benefited most. In settings where multiple competing descriptive frameworks exist, the choice of ontology could disadvantage some users.
Generalization and Future Work The study was conducted in an artificial setting. Domain shift - when deployment data differs systematically from training data - has not been addressed and must be studied. The authors plan to expand the dataset to include images from multiple clinics to improve generalizability. They also call for future research to disentangle the effects of the textual versus visual components of the multimodal XAI, and to study experienced versus inexperienced clinician interactions with XAI in real-world clinical workflows.