What is an explanation?
Let's start with an example. Concepts are expanded in the remaining sections.
An explanation example
You open a drawer, and a conversation with a friend starts.
Friend: Why did the drawer slide out?
You: Because I pulled it out? Had I not, the drawer wouldn't have slide.
The answer is an efficient cause. Aristotle proposed 4 causes: efficient (mechanism), formal (form, shape), material (properties), final (purposes).
Hume instead, understood causes through counterfactuals such as: had I not pulled, it wouldn't have slide. Hence, pulling is the cause.
Friend: I know that. But why does it slide rather than opening like a lid?
You: Oh! I misunderstood. The drawer sits on rails allowing it to slide.
The "rather than ..." is a contrast called foil, which may be implicit. Foils make answering easier.
Notice also the social process involved. For example, we tried to guess the friend's actual knowledge gap and be truthful. The friend may decide to keep asking "Why" and eventually reject or accept the causal chain (or remain sceptical).
Definitions of Explanation
Explanations can be split into an explanandum, which is a description of what is explained, and the explanans, which are the statements adduced to account for the phenomenon. These definitions are used in what follows.
Logical Process
The Studies in the logic of explanation (1948) defines scientific explanation in a few places. For example, with emphasis on the question:
To explain the phenomena in the world of our experience, to answer the question "why?" rather than only the question "what?" (...)
Or the answer:
The decisive requirement for every sound explanation remains that it subsume the explanandum under general laws [or theories].
They do require that the explanans be testable, believed to be true to a high accuracy, and logically lead to the explanandum by means of general laws.
Filling the gaps
On the opposite side, Explanations, Predictions and Laws argues that no question word (why or otherwise) is necessary for an explanation.
The paper also highlights the social aspect of communication, by suggesting that explanations are a kind of description aiming to fill a gap of understanding (or correct a misunderstanding) to an explainee. The questions then, are important but not the defining part.
This idea is the definition of explanation this post uses, even in the case of scientific explanation. However, the scientific context requires "filling the gap" to be rigorous (such as the conditions in the first definition).
Characteristics of explanations
The description (or information or answer) may use causal inference, logic inference, comparison to a reference item, subsumption into a class, metaphors, analogy and so forth. Prior beliefs or knowledge are also used to evaluate answers, and to omit what is considered obvious in the given context (or audience).
The reference item above is hinting to a foil. Explanation in AI: insights from the social sciences notes that why-questions are usually contrastive questions, phrased as why P rather than Q instead of why P. In this latter case, the foil (Q) is implicit.
Examples of answers are: "Light interferes because it is a wave." or "The sparrow chirps because it's a bird" or "It chirps because it's happy" or "The chirping is due to a vibration of its vocal strings".
In section 2.1.2, Explanation in AI: insights from the social sciences characterises an explanation as: a cognitive process, involving the generation of possible answers; a product, resulting from the cognitive process; a social process, which involves communicating the product.
Let's now expand on the cognitive and social processes of an explanation (as I see them).
The cognitive process of explanations
The cognitive process is similar to the scientific method:
- Filter aspects of the explanandum deemed relevant (using prior knowledge),
- Propose different answers,
- Weight the likelihood of each hypotheses,
- Accept until contradicted by experience or super-seeded (e.g. by a simpler explanation).
Besides using prior knowledge, the way we come up with hypotheses may involve creativity, metaphors, analogies and be aided by methods or techniques; this post won't go further into this aspect.
The steps above can also be sketched in a graph:
flowchart LR
A(("Answer why-event"))
subgraph hypotheses
B("Caused by A")
C("Caused by B")
D("Assigned to class")
end
subgraph filtered
E("Caused by B")
end
A --suggest all (inferences)--> hypotheses
hypotheses --select best--> filtered
Next the topics of causes and how are answers proposed are discussed, in the context of the cognitive process.
Causes
Aspects of causes already mentioned, but worth putting together, were:
- The cognitive process may involve inferring a cause.
- Aristotle's proposed 4 kinds of causal answers to a why-question. These explanations are not always exclusive, they can be complementary.
- Hume defined causes as counterfactuals: A is the cause of B if, had A not happened, B wouldn't have happened. This view was formalised by Pearl and Halpern.
Are all Aristotelian causes Humean causes? Efficient causes can be seen as counterfactuals, and both are common in science. The remaining 3 causes are not naturally seen as counterfactuals.
Necessary and sufficient
Talking about necessary and sufficient causes would've overloaded the example. Briefly, counterfactuals use the word happen, so it's an event rather than a condition: The spark of a lighter would be the cause of a fire, but oxygen would still be a necessary cause (or condition, or setting).
Logic Inference
Often, logic inference is used in the cognitive process (deriving a cause).
In a deduction the explanans combine to yield the explanandum. Studies in the logic of explanation (1948) argues that, in scientific explanation, explanans are testable conditions and general laws:
"Why does the phenomenon occur?" is construed as meaning "according to what general laws, and by virtue of what antecedent conditions does the phenomenon occur?"
And it should predict the explanandum, were it unknown (hence connecting prediction and explanation):
It may be said, therefore, that an explanation of a particular event is not fully adequate unless its explanans, if taken account of in time, could have served as a basis for predicting the event in question.
- Example: Light is a wave; all waves interfere; then light beams interfere.
In an induction a claim is generalised; for example: Bats are mammals; bats fly; maybe all mammals fly.
In an abduction a hypothesis is proposed which derives the explanandum. This is what is done in the science, and it is a deduction, but the difference is that it starts with a hypothesis. For example: Light shows interference patterns, waves interfere, maybe light is a wave.
The inference to a cause is very important and sometimes not obvious. It is made obvious when we do it wrong. For example, imagine that the drawer (in the example) actually slides when we touch it, then our inferred cause was wrong.
Strength of a Hypothesis
The plausibility of a hypothesis or causal claim is affected by different aspects.
-
Simplicity: if it involves a shorter chain of causes, it is preferred,
-
Generality: if it explains other cases, it is preferred,
-
Prior knowledge/beliefs
- Conditions generation and veto of hypotheses. For example, "The drawer slides because it wants." may be ignored in different basis. Another illustrative example from The structure and function of explanations:
(...) If told that herring and tuna have a disease, naive participants are more likely to extend the property to wolffish, the more similar item, than to dolphins. However, among fishing experts, who can generate an explanation for why the property might hold (e.g. tuna contract the disease by eating infected herring), similarity is less predictive of property extensions. (...)
- Aids selecting what seems causally / explanatory relevant from what is not. Consider two light beams interfering on a Sunday. The day is irrelevant (usually), we disregard a confounding factor.
I don't have much to say about product (2.), so we jump to 3.
Social Process
The answer must then be communicated, and there are expectations about it.
Gricean Maxims are rules observed in good communication. These rules can also be used as a guide for good model explanations.
- Informative (Quantity): right amount of context and details,
- Truthful (Quality, or Fidelity): Try to make it true,
- Relevance (Relation): do not state things that aren't needed (provide insight),
- Manner (clarity): express it in elegant terms.
Metaphors
The Machine and The Agent (click to open)
In the scientific and science-adjacent domains, models are conceptualised as machines:
- They have parts, each with a function, a role,
- They correspond with some aspect of the reality being modelled.
Outside of science or the technical domain, they're conceptualised as human-like agents:
- They tend to be explained in human terms,
- They are expected to be reliable, consistent, ...
So explanations are answers to why-questions; good explanations respect the Gricean maxims, and will be dependent on the audience (their preferred style, expectations, expertise).
The table below is a summary of the ideas above
| Perspective | Model is a… | Preferred Explanation style | Audience |
|---|---|---|---|
| Scientific | Machine | Mechanistic, causal, formal | Experts |
| Human-facing | Agent | Intentional, narrative | Users, stakeholders |
Many other metaphors could be proposed.
In the next post we use our knowledge to define Explainable AI.
List of sources used in this blogpost
- Studies in the logic of explanation (1948),
- Explanations, Predictions and Laws (1948),
- On the mechanization of abductive logic (1973). The first page is quite interesting.
- A Unified Approach to Interpreting Model Predictions (2017): paper proposing SHAP, that is, showing Shapley values as the best coefficients in linear combination of features, given 3 requirements (local accuracy, missingness and consistency),
- Explaining Explanations: An Overview of Interpretability of Machine Learning (2018),
- Producing radiologist-quality reports for interpretable artificial intelligence (2018): a "case study",
- The paper "Explanation in artificial intelligence: insights from the social sciences" (2019, 38 pages). Once the why-cause is found (diagnosis), it may be communicated, making rules of conversation relevant: Gricean Maxims of Communication (blog-post), or Wikipedia's.
- The definition of explanation extends previous work by Lombrozo on The structure and function of explanations (2006).
- The perils and pitfalls of explainable AI: Strategies for explaining algorithmic decision-making (2021): emphasis on socio-political aspects,
- Interpretable and Explainable Machine Learning for Materials Science and Chemistry (2022),
- Blog Posts: What is Explainable AI? (2022) and from IBM,
- A Perspective on Explainable Artificial Intelligence Methods: SHAP and LIME (2024).