As multimodal AI enters high-stakes settings like forensics and moderation, reliability, not benchmark accuracy alone, determines whether outputs can be trusted. Three properties are decisive: whether a model's confidence is calibrated, whether its failures can be anticipated, and whether its explanations reflect the evidence it uses. We audit a representative system, audio-visual deepfake detection, on LAV-DF, which provides ground-truth temporal annotations of every manipulation. Using two in-domain detectors, visual and audio, we examine explanation faithfulness, failure prediction, and calibration. The central finding: classification accuracy and reliability are decoupled. The audio detector is the stronger classifier (AUROC 0.999) yet the worst calibrated and least faithfully explained: most overconfident (Expected Calibration Error 0.247), with integrated-gradients explanations barely localizing the manipulation (1.14 times chance) despite its decisions localizing strongly (7.38 times), showing the attribution method, not the model, fails. The weaker visual detector (AUROC 0.667) is more faithfully explained and readily recalibrated. For failure prediction, confidence is the reliable signal (AUROC 0.85): abstaining on the least-confident half cuts the error rate from 22% to under 4%, whereas cross-modal explanation disagreement performs no better than chance. Benchmark accuracy is therefore a poor proxy for deployable reliability, and reliability must be audited directly.

Keywords

 Reliability auditing; model calibration; failure prediction; explanation faithfulness; deepfake detection.