Explainable Model for Fetal Plane Classification Using Vision Transformers and Layerwise Relevance Propagation

Authors: Sinchana S., Siri Gowri H., Nisarga Kunder, Shravani Hiremath

Abstract:

For prenatal diagnosis of anomalies and monitoring them, accurate classification of fetal anatomical planes from ultrasound images is essential because it allows clinicians to evaluate fetal development and administer treatment accordingly. However, manual classification of these planes is time-consuming and heavily dependent on the operator’s expertise. In recent years, encoder-decoder-based vision transformers (ViT) have emerged as a better alternative to traditional convolutional neural networks (CNNs) in various natural language processing tasks like text prediction and image-based tasks like classification, achieving state-of-the-art performance. Despite these advancements, the use of ViTs within the medical domain, especially for MRI and ultrasound images, remains relatively less due to their intricate architecture and relative difficulty in applying explainability techniques, as ViTs’ attention-based mechanisms are more complex to interpret than those of CNNs. In clinical environments, model interpretability is essential; thus, explainability is crucial for delivering consistent and transparent decision support. This interpretability allows physicians to understand and verify model outputs, fostering trust in AI-driven diagnostic processes. Although CNN-based models frequently use explainability techniques like Grad-CAM, transformer-based models lack similar interpretability due to their distinct attention mechanisms and architectural structure. In this work, there is the introduction of an approach using layer-wise relevance propagation (LRP) to provide visual explanations for vision transformer (ViT) predictions on fetal ultrasound images. With LRP, demonstrated the feasibility of deriving meaningful and interpretable insights from ViT models in medical imaging, paving the way for reliable and explainable AI applications in healthcare.

Keywords: fetal ultrasound, vision transformer, layer-wise relevance propagation, explainability, medical image analysis, model interpretability