Comparison of VGG16 and EfficientNetB3 for Pneumonia Classification on Chest X-Ray Images Using Grad-CAM

Abstract

Background: High accuracy in chest X-ray (CXR) pneumonia classification does not by itself show which image regions drive a model's decisions, and public paediatric datasets are vulnerable to data leakage and shortcut learning.
Objective: This study compares VGG16 and EfficientNetB3 for binary Normal–Pneumonia classification and examines their outputs using Gradient-weighted Class Activation Mapping (Grad-CAM).
Method: A quantitative comparative experiment used ImageNet weights and 224 × 224-pixel inputs. A pool of 5,242 image entries was stratified per image into 3,669 training, 786 validation, and 787 internal test images. Four single-seed scenarios compared frozen backbones with backbones unfrozen under a last-20-layer rule. Wilson confidence intervals and bounded McNemar tests were derived from the confusion matrices.
Results: VGG16 fine-tuning achieved 97.84% accuracy (95% CI 96.57–98.65%), 97.26% recall, 99.51% specificity, 98.53% F1-score, and 0.9977 ROC-AUC, with 16 false negatives and one false positive. EfficientNetB3 fine-tuning achieved 95.30% accuracy and 0.9963 ROC-AUC, with 37 false negatives and no false positives; the accuracy difference remained significant under all compatible discordance patterns (p ? 0.009). Several Grad-CAM heatmaps showed strong activations at image borders.
Conclusion: VGG16 fine-tuning performed best in this preliminary experiment, but border activations and per-image splitting indicate possible shortcut learning. Patient-level splitting, the original test set, multiple seeds, and external validation are required.