Reasoning about Evidence in Causal Explanations

Phyllis Koton MIT Laboratory for Computer Science Clinical Decision Making Group 545 Technology Square, Room 371 Cambridge, MA 02139

Abstract

Causal models can provide richly-detailed knowledge bases for producing explanations about behaviors in many domains, a task often termed interpretation or diagnosis. However, producing a causal explanation from the model can be time-consuming. This paper describes a system that solves a new problem by recalling a previous, similar problem and modifying that problem’s solution to fit the current problem. Because it is unlikely that any new problem will exactly match a previous one, the system evaluates differences between the problems using a set of principles for reasoning about evidence in causal explanations. These evidence principles allow the system to reason about such concepts as alternate lines of evidence, additional supporting evidence, and inconsistent evidence. If all differences between the new situation and the remembered situation are found to be insignificant, causal repair strategies are used to adapt the previous situation’s causal explanation to the new case. Although the evidence principles and repair strategies have thus far been tested only for the domain of managing patients with heart disease, their reasoning is domainindependent and should be generally applicable.

The work reported here has been supported (in part) by National Institutes of Health grants R01 LM 04493 from the National Library of Medicine and R01 HL 33041 from the National Heart, Lung, and Blood Institute.

1 Explanation transfer

Causal models are frequently proposed for knowledge-based systems because they have a wide range of applicability, they are robust, and they contain detailed information for providing explanations of their reasoning. In practice they are not widely used because they are inefficient compared to associational rules or other types of “compiled” knowledge. This inefficiency could be reduced by using a paradigm such as Case-Based Reasoning [5] or Memory-Based Reasoning [11]. These methods use a memory of previously-solved problems to avoid complex reasoning when presented with a new problem. They recall a similar problem and adapt its solution to the new case. However, the match between a new problem and a previously solved problem usually is only partial. This presents a difficulty when producing a causal explanation. Some feature of the previously-solved problem that was used as evidence in the causal explanation may be absent from the new problem. Similarly, the new problem may exhibit features that are absent from the previously-solved problem and which must be explained. This requires that the program have a set of principles for reasoning about dependencies between pieces of evidence and the states that they support in the causal explanation, and about the relationships (such as equivalence or incompatibility) between different pieces of evidence. The program can then determine whether a new problem with a somewhat different set of features can still be explained by a previous causal explanation. I have developed and implemented such a set of principles in a new system, CASEY.

2 Overview of CASEY

CASEY integrates case-based and causal reasoning techniques with a model-based expert system for managing patients with cardiac disease, the Heart Failure program [7]. CASEY contains a self-organizing memory system [3] for storing previously-seen problems (called cases). The memory contains descriptions of patients the program has seen and generalizations derived from similarities between the patients. The patient description is comprised of features. These include both input data, such as signs and symptoms, test results, history and current therapy information, and solution data, such as the causal explanation for the patient, the diagnosis, therapy recommendation and outcome information. A typical patient description in CASEY contains about 40 input features. For example, Figure 1 shows the patient description for Newman, a new patient presented to the system.

CASEY produces a causal explanation and therapy suggestions for a new patient using a three-step process. First, CASEY searches for a similar case in its memory; second, it evaluates the significance of any differences between the new case and the retrieved case; and finally, if none of the differences rule out the match, it adapts the solution from the retrieved case to fit the particulars of the new case. If no acceptable previous case is found, CASEY uses the Heart Failure program to produce a solution for the case de novo. This paper focuses on evaluating and adapting a retrieved solution.

The new case is the case that CASEY is currently trying to solve. A retrieved case is a case that has been retrieved from memory as a result of matching against features of the new case. CASEY will often retrieve more than one case that matches the new case. CASEY’s method of choosing among retrieved cases is described in [6]. The precedent case is a retrieved case in which all differences from the current case have been judged insignificant and from which solution transfer will occur. The precedent solution is the solution associated

(defpatient "Newman"
history
(age . 65)
(sex male)
(dyspnea on-exertion)
(orthopnea absent)
(chest-pain anginal)
(anginal within-hours unstable)
(syncope/near-syncope on-exertion)
(palpitations none)
(nausea/vomiting absent)
(cough absent)
(diaphoresis absent)
(hemoptysis absent)
(fatigue absent)
(therapies none)
vital-signs
(blood-pressure 138 80)
(heart-rate . 90)
(arrhythmia-monitoring normal)
(resp . 20)
(temp . 98.4)
physical-exam
(appearance nad)
(mental-status conscious)
(jugular-pulse normal)
(pulse slow-rise)
(apex-impulse normal)
(parasternal-impulse normal)
(chest clear-to-auscultation-and-percussion)
(abdomen normal-exam)
(extremities normal-exam)
laboratory-findings
(ekg lvh normal-sinus)
(cxr calcification)
(calcification mitral aortic-valve))

Figure 1: Patient description for Newman

Feature name  Value for David  Value for Newman
age  72  65
pulse-rate  96  90
temperature  98.7  98.4
orthostatic-change  absent  unknown
angina  unstable  within-hours & unstable
mean-arterial-pressure 107  99.3
syncope  none  on exertion
auscultation  murmur of AS  unknown
pulse  normal  slow-rise
ekg  normal sinus & lv strain normal sinus & lvh
calcification  none  mitral & aortic

Figure 2: Differences between patients David and Newman.

with the precedent case. The transferred solution is a copy of the precedent solution that is repaired to fit the new case.

A retrieved case for the patient Newman is a patient named David. The cases of David and Newman have many similarities.They also have some differences, which are shown in Figure 2. CASEY must decide if these differences are serious enough to rule out the match. In this example, the first four differences are insignificant. The remaining differences are important but the precedent solution can be adapted to explain them. This will be demonstrated in section 4.

3 The causal model and causal explanation

CASEY reasons about the model of the cardiovascular system contained in the Heart Failure system. The building blocks of the Heart Failure model are measures, measure values, parameters, and states. Measures correspond to observable features. Measure values are the allowed values of the measures, for example the patient’s heart rate, and are entered by the user. Parameters are physiological parameters of the patient, such as “left atrial pressure.” States represent specific qualitative values of parameters, for example “high left atrial pressure.” The model also has information about the relationships between the entities: for example, the probability of one state causing another, and the probability of observing a particular measure given a particular state. There are just over 100 measures and about 140 states defined in the model. Some states are distinguished as “diagnosis” states. If a diagnosis state established, the name of the state is added to the patient’s diagnosis. Others are defined as “goal states”. These are states that can be treated. If a goal state is established, the therapy associated with the state is added to the list of therapy suggestions for the patient.

The Heart Failure program produces a causal explanation consisting of a set of measures, states, and directed links. The causal explanation describes the relationship between physiological states and observable features. A link between two states, or a state and a measure, indicates that one causes the other. Only abnormal findings are explained, but the program may not be able to explain all the abnormal findings.

A graphical representation of the Heart Failure program’s causal explanation for David’s symptoms is shown in Figure 3. David was diagnosed as having aortic valve disease, specif-

Figure 3: Causal explanation produced by Heart Failure program for David.

ically aortic stenosis, and unstable angina. Abnormal features in his patient description accounted for by the causal explanation are: unstable anginal chest pain, evidence of LV strain on EKG, a heart murmur, and dyspnea on exertion. David’s high heart rate, high respiration rate, and high blood pressure were not explained.

CASEY translates its input into the Heart Failure program’s representation before it begins to use the causal model. The causal explanation produced by CASEY is in the same form as one produced by the Heart Failure program. CASEY translates the solution into its own representation before storing it for future use.

4 Principles for reasoning about evidence

Most new cases will not exactly match any previous case in the memory. To allow partial matches, differences between cases must be evaluated. Two cases might have many similar features yet have one critical difference that invalidates the match. Alternatively, many differences may not affect the validity of a match. The feature involved may have no relation to the solution of the case (the patient’s name, for example) or the difference may be explainable. The module in CASEY that performs this evaluation is called the justifier because it must justify using a retrieved case as a precedent for the new case. The justifier relies on a set of principles for reasoning about evidence, termed evidence principles, that are presented below. The first evidence principle is used to determine whether a state in the retrieved casual explanation is ruled out by evidence in the new case. The next four attempt to show that the difference in question is insignificant or repairable, and the last three handle features which have special values that are easy to reason about. Other principles for determining significance are being explored.

1. Rule out. A state is ruled out from the transferred solution if there is some feature in the new case which is incompatible with that state. This is detected when the feature has zero probability for some state in the retrieved solution. For example, a heart rate of 40 beats per minute is incompatible with the state high heart rate.

2. Other evidence. This is used when a feature present in the retrieved case is missing in the new case. It tries to find a piece of evidence in the new case which supports the state that the missing feature supported.

3. Unrelated oldcase feature. This is used when a feature is present only in the retrieved case. If the feature was not used in the causal explanation, its absence has no effect on any states in the explanation, so it can be ignored.

4. Supports existing state. This principle is used when a feature is present in the new case but not in the retrieved case. It tries to attribute this feature to a state in the retrieved causal explanation.

5. Unrelated newcase feature. This is also used when a feature is present only in the new case. If the feature is abnormal, but is not evidence for any existing state and does not strongly suggest a new state, then add it to the explanation as an unexplained feature.

6. Normal. Normal values are not explained by the Heart Failure program, so a normal value in the new case does not need to be explained. (Note that if a model did reason using normal values, this rule would be eliminated).

7. No information. If there is no information given about a feature in one of the cases and it is known to be absent in the other case, then assume that it is also absent in the former case.

8. Same qualitative region. CASEY evaluates differences between features with numerical values by translating them into qualitative value regions. For example, a blood pressure of 180/100 becomes “high blood pressure.” Features whose values fall into the same qualitative region are judged not to be significantly different. The regions are determined using range information for the corresponding measure in the Heart Failure model.

CASEY can reject a match on either of two grounds: a significant difference could not be explained, or all the diagnosis states in the retrieved solution were ruled out. If all differences between the new case and the retrieved case are insignificant or repairable, then the transfer of solutions from the precedent to the current case proceeds.

CASEY makes the following inferences about the differences between patients David and Newman:

• David’s age is judged to be insignificant by the rule unrelated oldcase feature. Newman’s age is judged to be insignificant by the rule unrelated newcase feature.

• Both patients’ heart-rates are in the same qualitative region (moderately high heart rate) so the difference is considered insignificant.

• Both temperatures are in the “normal” qualitative region so the difference is considered insignificant.

• David’s mean arterial pressure is high, but Newman’s is not. However, this feature was not accounted for by the causal explanation, so it is judged insignificant by the rule unrelated oldcase feature. Newman’s mean arterial pressure is normal, so it does not have to be explained.

• Orthostatic change is absent in David, but not specified for Newman, so the rule no-info concludes that it is also absent in Newman.

• Newman’s finding of angina within hours is additional evidence for the state unstable angina.

• Syncope on exertion is additional evidence for the state limited cardiac output.

• Murmur of AS, which is absent in Newman, is evidence for the state fixed high outflow resistance in the precedent solution, but this state has other evidence supporting it in the new case.

• Newman’s pulse has slow rise. This is evidence for the existing state slow ejection.

• LV strain on David’s EKG is evidence for the state lv hypertrophy. Newman’s EKG shows LVH, which is evidence for the same state.

• Newman’s finding of aortic calcification is evidence for the existing state aortic valve disease.

• Newman’s finding of mitral calcification has only one cause, mitral valve disease, so this state is added to the causal explanation. mitral valve disease has no causes in the model, so it remains unlinked, forming a separate causal chain.

All the differences between David and Newman are insignificant or repairable, so the match is justified.

5 The repair strategies

CASEY uses repair strategies to adapt a previous solution (consisting of causal explanation, diagnosis, and therapy) to a new case. Causal explanation repair strategies add or remove nodes and links to the transferred causal explanation.^{1} CASEY can make the following repairs:

1. Remove state. When all evidence for a state has been removed, or if the only cause of a state has been removed from the transferred causal explanation, CASEY removes that state from the explanation. This also checks to see if states that this state caused have to be removed. This strategy is invoked by the rule out principle.

2. Remove evidence. When a piece of evidence that was used in the retrieved case is absent in the new case, this removes the feature and any links to it. It is invoked by the principles other evidence and unrelated oldcase feature.

3. Add evidence. This repair adds a piece of evidence to the causal explanation, and links it to the states for which it is evidence. This is invoked by other evidence and supports existing state.

4. Substitute evidence. When two numerical values have the same qualitative value, this repair replaces the old value with the new as evidence for some state. It is invoked by the same qualitative value principle.

1Diagnosis repairs add and remove diseases from the transferred diagnosis. Therapy repairs add and remove therapy suggestions. They are describe in [6].

Figure 4: CASEY’s causal explanation for Newman.

5. Add state. In one instance, it is very easy for CASEY to add a state to the causal explanation. This is when the evidence has only one possible cause. This is invoked by the principle supports existing state.^{2} The state is added to the causal explanation, and CASEY tries to link it to existing states and features in the causal explanation (using add link).

6. Add link. This adds a causal link between two states.

7. Add measure. This adds a feature which CASEY cannot link to the causal explanation. It is invoked by unrelated newcase feature.

In order to adapt the explanation transferred from David to fit the data for Newman, the following repair strategies are invoked by the justifier:

(substitute-evidence hr:90  hr:96)
(remove-evidence mean-arterial-pressure:107)
(add-evidence within-hours unstable-angina)
(add-evidence syncope-on-exertion limited-cardiac-output)
(remove-evidence murmur-of-as)
(add-evidence slow-rise slow-ejection)
(remove-evidence lv-strain)
(add-evidence lvh lv-hypertrophy)
(add-evidence aortic-calcification aortic-valve-disease)
(add-state mitral-valve-disease)
(add-evidence mitral-calcification mitral-valve-disease)

CASEY’s causal explanation for Newman is shown in Figure 4. It states Newman’s diagnosis as aortic valve disease, aortic stenosis, mitral valve disease, and unstable angina. It does not explain his high heart rate or his high respiration rate. The causal explanation for Newman produced by the Heart Failure program is identical.

The user has the option of rejecting CASEY’s solution and running the Heart Failure program on the data. In either case, the patient description and solution are saved to be used for future problem-solving.

2The fact that a feature has only one cause is discovered while we are searching for existing states that cause this feature.

6 Discussion

6.1 Related work

SWALE [2] uses a memory of previously-solved problems to create explanations. It remembers prototype explanations, which it attempts to modify to fit a new situation. SWALE’s process model is very similar to CASEY’s. However, SWALE’s explanations are very abstract, and require a large amount of real-world knowledge to evaluate and repair differences. CASEY’s explanations are very concrete. In order to evaluate or repair a difference, it needs only examine local information about states and evidence. CHEF [1] also combines casebased reasoning with a causal model. It uses causal knowledge to explain plan failures, so its causal reasoning consists of chaining rules backward from an observed failure to the cause. Also, the program does not save the explanations and repair them to fit a new situation as CASEY and SWALE do. Recent work by Resnick and Davis [8] uses a memory of previously-solved problems and a causal model to explain hardware faults. It differs from CASEY in that it requires an exact match between the descriptions of a new fault and a previous fault.

6.2 Generalizing the results

CASEY requires the following information for its reasoning:

• a set of features that can be used to describe some problem,

• a set of states that can explain these observations,

• specific information about features:

– the set of possible values for this feature,

– the set of states that use this feature as evidence,

– the advisability of ignoring a particular feature. This information is used to determine if a match is invalid because it can’t explain an important feature.^{3}

• specific information about states:

– the set of features that are evidence for this state,

– features that rule this state out,

– a list of states that cause this state,

– a list of states that this state causes.

CASEY’s reasoning does not depend on any knowledge specific to the domain. However, it accommodates certain assumptions of the Heart Failure program, the most important being the assumption that there is only limited interaction between states (for example, the Heart Failure program has no concept of two states jointly causing a third state). CASEY’s evidence principles currently are being extended to handle more complex interactions.

The criteria by which a match is ruled out can be made more or less stringent. For example, a match could be ruled out if any state in a retrieved case was ruled out. CASEY is very accommodating of differences between new and retrieved cases because the system currently does not have enough cases to allow very stringent similarity requirements. A more conservative criterion might be implemented if the systems acquires a great many more cases. This would mean that CASEY would do less modification to a retrieved causal explanation, but might also increase the number of times that it failed to find an acceptable match.

3The Heart Failure program represents this information numerically, as the percentage of time a feature is explained by the model. CASEY determines this information experientially by examining the role the feature played in similar past cases. The same information could be represented and reasoned about qualitatively.

The ability of the technique to produce a meaningful solution depends on selecting a good precedent case. Much research has been done in this area (for example [4], [10], [11], [2]). CASEY uses a novel matching algorithm specifically designed for reasoning about causal explanations. This algorithm gives extra importance to features that played a role in the causal explanation of previous similar cases [6].

6.3 Limitations of the method

CASEY’s current implementation has limitations. Some problems presented to the system have a large number of “reasonable” explanations. CASEY does not use all the quantitative information available in the Heart Failure model that would allow it to distinguish between statistically more- and less-likely solutions.

For example, the program is parsimonious about adding additional states to the causal explanation. If a new feature could be attributed to two different physiological states, one of which is already included in the transferred explanation, CASEY will use the state that is already there rather than add a new state. It is possible that a feature has a higher probability of being caused by the state not already in the explanation. The model contains information that CASEY could use to discover this circumstance.

CASEY works by modifying one particular solution, rather than by generating a solution. This makes it difficult to evaluate the likelihood of its solution being correct compared to other possible explanations for the same data. The Heart Failure program, on the other hand, calculates the probabilities of all possible causal explanations that fit the data, and chooses the one with the highest probability. Again, the model contains information that could be used to roughly determine whether or not CASEY is pursuing the most likely explanation for the data.

For certain applications (e.g. [9]), any explanation for the input features is acceptable. In the Heart Failure domain, the users require the most likely explanation. CASEY’s justifier will soon be extended to recognize when the solution it is creating is not the most likely one, in which case it can reject the match.

7 Conclusions

CASEY demonstrates that combining a memory of past cases with causal reasoning can have significant advantages over either method used alone. CASEY’s causal reasoning ability lets it produce a complete causal analysis of the new case, not simply a reference to a previous solution. CASEY can produce a causal explanation even for problems that the Heart Failure program cannot solve. This is because a problem could lack sufficient information for the Heart Failure program to calculate a solution, but still have enough information for CASEY to find a matching case. This also suggests an interesting area for future work: using CASEY to integrate several causal models developed for different domains, where no one model could account for all the features in the problem. CASEY can produce causal explanations for common problems by recalling a similar problem and adapting its solution through simple modifications. It resorts to detailed causal analysis when it lacks experience, then remembers its solution for future use.

Acknowledgments

Robert Jayes, MD kindly provided the example cases. William Long’s Heart Failure program provided an excellent testbed for this work. Thanks to Peter Szolovits, Ramesh Patil, and William Long for their supervision of this research, and to Janet Kolodner, Michael Wellman, and Paul Resnick for their helpful comments on this paper.

References

[1] Kristian Hammond. Case-based Planning: An Integrated Theory of Planning, Learning and Memory. PhD thesis, Yale University, 1986.

[2] A. M. Kass, D. B. Leake, and C. C. Owens. SWALE: A program that explains. In Explanation Patterns: Understanding Mechanically and Creatively. Lawrence Erlbaum Associates, Hillside, NJ, 1986.

[3] Janet L. Kolodner. Maintaining organization in a dymanic long-term memory. Cognitive Science, 7:243–280, 1983.

[4] Janet L. Kolodner. Reconstructive memory: A computer model. Cognitive Science, 7:281–328, 1983.

[5] Janet L. Kolodner. Experiential processes in natural problem solving. Technical Report GIT-ICS-85/22, School of Information and Computer Science, Georgia Institute of Technology, 1985.

[6] Phyllis A. Koton. Using Experience in Learning and Problem Solving. PhD thesis, Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, Cambridge, Massachusetts, 02139, 1988. Also appeared as MIT LCS TR-441, March 1989.

[7] William J. Long, Shapur Naimi, M. G. Criscitiello, and Robert Jayes. The development and use of a causal model for reasoning about heart failure. In Symposium on Computer Applications in Medical Care, pages 30–36. IEEE, November 1987.

[8] Paul Resnick and Randall Davis. Improving performance of model-based reasoning by learning from experience. Submitted to AAAI88, March 1988.

[9] Reid Simmons and Randall Davis. Generate, test, and debug: Combining associational rules and causal models. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pages 1071–1078, 1987.

[10] Robert L. Simpson. A computer model of case-based reasoning in problem solving: An investigation in the domain of dispute mediation. Technical Report GIT-ICS-85/18, Georgia Institute of Technology, 1985.

[11] Craig Stanfill and David Waltz. Towards memory-based reasoning. Communications of the ACM, 29(12):1213–1228, 1986.