Retinal layers prognosticate cognitive progression independent of relapse activity in multiple sclerosis.

Scientific progress has long depended on building interpretable models of complex systems, from Newton’s laws to the Navier–Stokes equations, which encode mechanistic understanding and enable prediction, control and design. In recent years, however, machine learning (ML) has begun to transform this paradigm. Deep neural networks can now infer relationships in high-dimensional data that are inaccessible to classical theory, often matching or surpassing human experts in tasks ranging from protein folding to turbulence control. This new reality raises a profound question: if machines can learn representations of physical systems that outperform human models, can we, in turn, learn from the learners? Addressing this question requires explainable artificial intelligence (XAI): a suite of methods that make the internal logic of ML systems interpretable to human reasoning. We use the term XAI in a broad scientific-ML sense, while distinguishing canonical XAI methods from adjacent interpretable modeling tools. In the narrower sense, XAI includes methods such as feature attribution, counterfactual explanations, data attribution and mechanistic interpretability. In this Perspective, we also discuss interpretable scientific-ML and representation-learning tools, such as symbolic regression, operator learning and autoencoders, because they can make learned models or latent representations more accessible to scientific interpretation. These tools are not all XAI methods in the strict post-hoc or mechanistic sense, but they form part of a broader explainability workflow for scientific discovery, optimization and certification. Throughout this Perspective, we distinguish between model-level explanations and system-level causal claims. XAI methods are, by themselves, informative about an AI model: they describe how inputs, learned representations, or internal components influence the predictions or decisions of the model. They do not, on their own, establish causal relationships associated with the physical system being modeled. System-level causal claims require additional assumptions and evidence, including representative data, validated physical constraints or governing models where available, robustness under distribution shift, and, ideally, targeted numerical or physical interventions. We therefore use XAI primarily as a tool for generating, organizing and testing mechanistic hypotheses; causal interpretations become credible only when the learned model is sufficiently faithful to the system, and the proposed mechanisms survive independent validation. Explanations have always been central to scientific inquiry: they connect abstract models to causal understanding and experimental validation. In modern deep learning, however, the representations learned by a model are rarely transparent. XAI thus provides an interface between machine predictions and human understanding, allowing researchers to inspect which features drive the decisions of a model, estimate their influence on model outputs and assess whether these model-level patterns are consistent with physically meaningful mechanisms. The connection between model explainability and causal reasoning is particularly relevant for science and engineering, provided the distinction above is maintained. Causality formalizes the notion of intervention and is therefore central to scientific reasoning. Recent developments in causal inference and information theory allow decomposition of causal effects into unique, redundant and synergistic components, enabling a more granular view of how different variables contribute to an outcome under specified assumptions. Such tools can help distinguish candidate mechanisms from spurious correlations in complex datasets. When combined with XAI techniques such as SHapley Additive exPlanations (SHAP) or integrated gradients, they can support a mechanistic interpretation of deep-learning models: which regions of a flow field, which frequencies in a spectrum or which molecular configurations are most influential for a predicted outcome. This model-level evidence can motivate hypotheses about physical mechanisms, but those hypotheses require validation before being treated as causal claims about the underlying system. This explainability-and-validation workflow opens new opportunities to generate and test candidate scientific laws from data-driven systems.