%-*-mode:tex-*-

% Benjamin Kuipers
% Causal Reasoning
% format: LaTeX
% reference: ?

\documentstyle[11pt]{article}
\pagestyle{myheadings}
\head{Causal Reasoning:  \today}
\newcommand{\ul}[1]{\underline{#1}}
\renewcommand{\baselinestretch}{1.5}
\setlength{\oddsidemargin}{0in}
\setlength{\textwidth}{6.5 in}
\setlength{\unitlength}{1mm}

\title{Causal Reasoning\thanks{To appear in S. C. Shapiro (Ed.), 
{\it Encyclopedia of Artificial Intelligence}.  
New York:  Wiley-Interscience, 1986.  \\  
 This work was supported in part by the National Library of Medicine
through NIH Grants LM 03603, LM 04125, and R01 LM 04374, and by the
National Science Foundation through grants MCS-8303640 and
DCR-8417934.}}

\author{Benjamin Kuipers\thanks{MIT Laboratory for Computer Science,
Cambridge, Massachusetts 02139. \\
After July 15, Department of Computer Science, University of Texas,
Austin, Texas 78712.}}

\begin{document}
\maketitle

\section{INTRODUCTION}

Causal reasoning in artificial intelligence consists of inferences about
the behavior of systems or mechanisms, for the purpose of explaining
past observations or predicting future events.

Many different types of research in artificial intelligence fall under
this rubric.  Although the following categorization captures certain
major distinctions in the field, some work fits in more than one
category.

 \begin{itemize}
   \item {\it Causal links} may be asserted between states and events,
with inferences to determine the pattern of connecting causal
relationships in a particular situation.  The pattern is expressed as a
causal chain or causal network, and may include previously unobserved
intermediate states and events.

   \item {\it Symbolic simulation of continuous systems} infers the
possible behaviors of a continuously-changing system from its
structure and an initial state.  Rather than focusing on states and
events, the important objects are continuous state variables, their
values at particular times, and the constraints holding among them.

   \item {\it Symbolic simulation of discrete systems} also infers the
behavior of a mechanism from its structure, but the inference rules for
discrete mechanisms, typically digital electronic circuits, are based on
the definitions of electronic components rather than on the possible
behaviors of continuous functions.

   \item {\it Defining causality}, and establishing criteria for
asserting causal relationships, have been the concerns of philosophers
of science and statisticians.  Formal logicians are also concerned with
determining the expressive power a formal logic needs to describe the
objects, relationships, and inferences that make up commonsense causal
reasoning.

   \item {\it Cognitive mental models} provide human examples of 
representations capable of expressing commonsense knowledge of the
physical world.  Valuable constraints on the structure of knowledge
representations can be derived from the study of child development,
human performance on a variety of tasks, and historical, cultural, and
individual variation.
 \end{itemize}

\clearpage
\section{CAUSAL LINK MODELS}

Causal knowledge is often modeled as a network of nodes representing
states or events of various kinds, with links representing a variety of
causal relationships.  These representations have been most extensively
explored in two quite different contexts:  story understanding and
medical diagnosis.  In both contexts, the goal of causal inference is to
activate additional nodes as necessary to construct a network of states
and events linked in a causally coherent pattern.

Rieger and Grinberg (1) introduced the first extensive causal
link representation, with the goal of expressing causal knowledge of the
operation of a complex mechanism sufficient to simulate its behavior.
The representation includes nodes for actions, tendencies, states, and
state-changes, and ten link types representing different causal
relationships that might hold among these nodes.  The mechanism
description consists of local causal relations among nodes, and each
node and link is implemented as an independent computational agent.
Activation percolates through the network according to the semantics of
each type of link.  The temporal sequence of node activations is
intended to correspond to the sequence of states, actions, and
state-changes in the behavior of the mechanism.  The simulator can,
however, introduce spurious temporal ordering relations among events on
independent causal paths; no provision is made for detecting or
correcting this.  Rieger and Grinberg (1) demonstrate that their
representation can be applied to a number of non-trivial mechanisms
(e.g. the forced hot air furnace and the flush toilet), and is able to
express the structure and simulate the behavior of each mechanism.

Schank and Abelson (2) take a similar approach to causal
relationships, making them the basis for an elaborate representation of
meaning in natural language story understanding.  Their representation
includes nodes (acts and states) and five fundamental link types whose
semantics define a simple ``causal syntax'' of legal causal
relationships.  These causal relations can then be used to infer the
existence of unmentioned intermediate states required for the coherence
of a story narrative.  For example, {\it Joe burned his hand because he
forgot the stove was on}, can be seen to require additional intermediate
states and actions to be comprehended; a suitable store of real-world
knowledge allows a reasonable causal scenario to be constructed.  In
Schank and Abelson's (2) overall theory of story understanding, {\it
causal chains} provide the skeleton for the key concept of the branching
{\it script}.  Stories in which scripts are constructed on the fly can
be understood in terms of the actors' {\it plans} and {\it goals}.  The
purpose of this application of causal reasoning is to construct a
complete and coherent causal scenario, constrained by the evidence in
the text.  Neither the story nor the causal reasoning representation by
itself is sufficient to construct an unambiguous scenario, but a
well-structured story can be understood by considering both constraints
together.

The purpose of causal reasoning in medical diagnosis is similar to its
purpose in story understanding:  to determine a causally coherent set of
events that has taken place and that matches the available observations.
The medical diagnosis systems using causal reasoning typically employ a
simpler ``causal syntax'' consisting of a single type of node,
representing a partial description of the patient's state, and a single
type of causal link.

The causal relationships in the CASNET system (3) for
diagnosis of glaucoma are probabilistically weighted links between
pathophysiological states.  A disease process is modeled by a chain of
states, where the current point on that chain is the progress of the
disease at the current time.  Support for diagnostic hypotheses is
propagated from clinical evidence to pathophysiological states; among
pathophysiological states across causal links; and from chains of
pathophysiological states to disease hypotheses.  In CASNET, causal
links reflect real causal relations in the glaucoma domain, and therefore can
be activated in orderly chains with causally meaningful relations
to disease hypotheses.  However, the bulk of the inference involving
these links and their weights treats them simply as conditional
probabilities.  Thus, the fact that they are {\it causal} relations is
relevant to the construction and validation of the knowledge base, but
not to the operation of the program.

The ABEL program (4) for reasoning about acid-base and
electrolyte disturbances also uses weighted causal links between nodes
representing pathophysiological states, but the weights are interpreted
as measures of magnitudes rather than as probabilities.  This allows ABEL to
reason about interactions between multiple diseases by deciding whether
a known cause is {\it sufficient} to account for the observed magnitude
of the disorder.  If not, a second disturbance may be interacting with
the first to cause the observed problem.  ABEL reasons about
interactions at a detailed, pathophysiological level of description,
where nodes represent the deviations of electrolyte concentrations from
their normal values.  In order to construct a useful diagnosis, it
builds a model of the patient state at several levels of abstraction,
linking the pathophysiological description with a description at the
clinical level.  Although much of its reasoning is symbolic, ABEL relies
on numerical calculations to generate disease interaction hypotheses,
and to test whether a given cause is sufficient to account for an
observed effect.  This limits its applicability to domains where
adequate numerical theories exist.

In his design for Cadeuceus, the successor to Internist-I as a general
diagnostic program for internal medicine, Pople (5) develops a
pattern of interaction between causal and taxonomic links among disease
hypotheses and pathophysiological states, focusing on the search for
appropriate intermediate states.  In the causal network part of the
representation, nodes represent only states, not events or
state-changes.  There is only one kind of link, meaning {\it
may-be-caused-by}, linking a pair of states.  In searching for a
diagnosis, the causal network proposes differential diagnosis problems
by defining a small set of possible causes for an observed finding or
concluded intermediate state.  Simultaneously, an independent taxonomic
network can provide similar diagnostic problems.  Taking advantage of
interactions between these two networks, Cadeuceus analyzes the set of
proposed differential-diagnosis problems to determine the diagnostic
test that best refines the set of hypotheses at lowest cost.  Thus,
Cadeuceus constructs a network of causally related states in order to
focus its search for a good diagnosis and to test candidate diagnostic
hypotheses.

In general, causal link models of causal reasoning assume the existence
of a knowledge base of potential states, events, and causal relations.
The problem to be solved is to construct a causally coherent sub-network
best corresponding to, and hence explaining, the available observations.
The underlying ontology consists entirely of fragments of behavior:
events, actions, states, and state changes.  

Causal link models of causal reasoning have been criticized on the
grounds that they lack the ability to represent the structure of a
mechanism or to derive predictions of behavior from structure.  Without
this capability, interactions between processes are difficult to reason
about, since they interact by having simultaneous influences on
structural parameters.  Only Patil, et al (4) addresses this issue,
and then by assigning numerical strengths to causal links and performing
substantial calculations with those numbers.

\clearpage
\section{SYMBOLIC SIMULATION OF CONTINUOUS SYSTEMS}

Symbolic simulation of a continuous system, often called ``qualitative
simulation,'' is a type of causal reasoning in which qualitative
descriptions of the possible {\it behaviors} of a mechanism or system
are derived from a qualitative description of its {\it structure}.  The
structural description of a mechanism includes a set of continuous state
variables, and constraints that must be satisfied by values of those
variables at each point in time.  In the behavioral description, the
value of a variable is described qualitatively as a point or interval in
a (possibly partially ordered) {\it quantity space}, along with its
direction of change.  A single behavior of a mechanism is described as
the sequence of qualitative states of the set of state variables.
Starting with a qualitative structural description, there may not be
enough information to specify the behavior uniquely, so qualitative
simulation may predict several possible behaviors.  This level of
structural description, stated in terms of state variables and the
constraints on their values, can be considered a qualitative abstraction
of a differential equation.  The behaviors derived by qualitative
simulation are intended as qualitative abstractions of the solutions to
that differential equation (6).  Figure 1 gives an example of 
qualitative structure and behavior descriptions.

\begin{figure}
 \vspace{3in}
 \caption{The structural and behavioral descriptions of a ball rising
then falling in constant gravity.  The structural description gives the
derivative relations between $X(t)$, $V(t)$, and $A(t)$.  In the
behavioral description, both time and quantity axes are qualitative:
values can be described as being either equal to, or in the interval
between, landmark values.}
 \end{figure}

Different researchers propose different means for acquiring this
structural description.  Kuipers and Kassirer (7), working primarily
in a medical domain, assume that the structural description is given in
the knowledge base rather than derived from the {\it physical anatomy}
of the mechanism.  De Kleer (8,9) and Williams (10,11) derive structural
constraints from physical anatomy:  the components and circuit topology
of analog electrical circuits.  Forbus (12,13), studying commonsense reasoning about
physical situations, examines the physical anatomy along with what is
known about the quantities involved, to determine a set of active {\it
processes}.  The structural description is then derived from the set of
processes.  The process-based approach uses the concept of spatially and
temporally localized {\it histories} (14) to infer restrictions on
the set of possible causal interactions in the situations.  The other
approaches assume a closed set of possible interactions specified by the
structural description or circuit diagram.

When the structural description is constructed from the physical anatomy
of the device, components are described in terms of their behavior.  The
{\it no-function-in-structure} principle (9) is
an important restriction on the behavior description of components:  a
component may be described in terms of its context-independent
behavioral repertoire, but not in terms that presume any properties of
the mechanism in which it is embedded.  Thus, the component description
of a switch may not presume the existence of a current that will flow
when the switch is closed and not when it is open.  Rather, the switch
might be described as having very low resistance when closed and very
high when open, leaving the rest of its description (i.e. current and
voltages) to be filled in by the prevailing context of a particular
mechanism.

There are important similarities and differences among the different
qualitative representations and simulation algorithms.  Given a
structural description of a mechanism in terms of state variables and
constraints, descriptions of some of the variable values can be
propagated across the constraints to create a more complete description
of the mechanism's state at a given point in time.  Once the current
state is adequately described, those variables with changing values are
examined to determine if they are approaching limiting landmark values.
The next qualitatively distinct state is determined by analyzing the
possible transitions of individual variables to determine which
transitions can occur next, using either a transition ordering decision
procedure (10,13,15) or constraint-based
transition filtering rules (6,9).  A given
qualitative state of a mechanism may have zero or more successors, so
qualitative simulation creates a branching tree or directed graph of
state descriptions as the description of behavior.
Other similarities and differences include:
 \begin{itemize}
   \item Values are described qualitatively in terms of their ordinal
relations with a small set of distinguished ``landmark'' values.  The
value description also specifies the current direction of change of the
variable.  Some investigators allow zero as the only landmark (8,11),
some allow new landmarks to be discovered (6,15), and some
allow landmarks to be partially ordered (13).
   \item Forbus' (12,13) {\it influences} are open-ended relations stating
how changes to one variable affects another, ``all else being equal.''
Only the complete set of influences can be analyzed to determine how the
variables will co-vary.  For the other investigators, the structural
description consists of {\it constraints}, each of which must always be
satisfied.
   \item  De Kleer (8,9) uses an unusual style of simulation called {\it
envisionment} which constructs all the combinatorially possible
qualitative states of the system, then links them into a transition
network, capturing all possible behaviors of the mechanism.  The other
investigators simulate forward in time from an initial state.
 \end{itemize}

The qualitative simulation approach to reasoning about mechanisms does
not make use of explicit {\it causal} relations in its inference
process.  Rather, it starts with physical anatomy, extracts a structural
description, and derives a set of possible behaviors.  In some cases,
assertions of causal relationships among events are derived from the
order of events in the behavioral description and the order of
information propagation during simulation.  Clearly, qualitative
simulation does not explicate the full concept of causality in
mechanisms.  On the other hand, the use of qualitative simulation to
derive predictions of the behavior of mechanisms is a useful method for
reasoning about the physical world.

\clearpage
\section{SYMBOLIC SIMULATION OF DISCRETE SYSTEMS}

In a continuous system, symbolic simulation involves mapping a
continuous space of values into a discrete space of qualitative
descriptions.  Simulation algorithms are based on the properties of
continuously differentiable functions:  the Intermediate Value and Mean
Value Theorems.  In a digital system, on the other hand, state variables
already have discrete values.  The set of permissible state changes is
not constrained by continuity, but by the semantics of the components in
the particular circuit.  Thus, rather than having a small closed set of
qualitative state transition rules, a digital simulator must have an
open-ended catalog of device descriptions.

Although symbolic descriptions of variable values are not used to
represent open intervals of values, they are needed to handle
abstractions of values or disjunctions of possible values when
sufficient information is not available to specify the value uniquely.
In the case of automatic verification of the design of digital circuits
(16), symbolic descriptions allow the specifications for a
module to be used both as the overall behavioral description to be
justified from the modules internal structure, and as the
context-independent component description when the module is used in a
larger structure.  Using this modular, hierarchical decomposition,
Barrow (16) shows how a substantial, realistic, digital circuit can be
effectively verified by simulating its behavior and comparing the
predicted behavior with its specification.  Verification of the lower
level modules produces stored ``lemmas'' that can be referred to later
in the proof, a strategy which greatly shortens the overall reasoning
time.  The decomposition also allows different levels of the hierarchy
to operate in very different value domains:  voltage levels, logic
values, integers, etc.  The success of this approach depends strongly on
the modular decomposition:  a reasonable assumption in the case of
designed structures such as digital circuits, but less reasonable in,
say, biology.  The clarity of the hierarchical abstraction relations
also stems from the fact that both structural and behavioral
descriptions for modules can be expressed as equations relating the
values of input, output, and state variables.

When the behavior of a digital circuit fails to meet its specifications,
the problem is frequently not that the design was incorrect, but that
the actual circuit fails to satisfy the structural description in the
design.  A hypothesized fault in the circuit, once suspected, can be
tested by simulating the faulty circuit forward from a given set of test
inputs to see whether the prediction matches the observed outputs.
Thus, simulation of circuits under fault hypotheses is used as the {\it
test} portion of a {\it generate-and-test} diagnostic search.
Troubleshooting is difficult because the generation of fault hypotheses
is highly underconstrained, and there is an enormous space of
possibilities.  Davis (17) describes a hardware troubleshooting system
whose generation of fault hypotheses is controlled by a set of
assumptions about the nature of the fault, which are progressively
suspended as the search goes on so the most likely portion of the space
is considered first.  Davis also uses multiple concepts of structural
adjacency --- topological, physical, thermal, and electromagnetic, for
example --- to generate fault hypotheses that would have been invisible
from other vantage points.  For example, the set of possible bridge
faults inappropriately connecting two wires in a circuit can be
difficult to express on the circuit schematic representation, but are
nonetheless tightly constrained to occur only between physically
adjacent wires on the circuit board.  Thus a description capturing
spatial adjacency must be available to generate the appropriate set of
hypotheses.

Causal reasoning and symbolic digital simulation are key elements, but
still only a small part, of a large domain of literature on digital
circuit verification, test generation, and automated troubleshooting.
Symbolic simulation of digital circuits is feasible for the same reasons
that such circuits are useful in computers:  both time and the state
variables can be treated as having discrete values, and the state
variables can be interpreted simultaneously as lying in more than one
semantically distinct space of values.  Thus, causal reasoning about
digital circuits is a specialized application with specialized
assumptions it can draw upon.  While this line of research is likely to
produce useful and impressive results, those results are not strongly
linked with the study of causal reasoning as part of commonsense
knowledge about the everyday physical world.

\clearpage
\section{DEFINING CAUSALITY}

Some of the word described above takes as primitive the notion that one
event can cause another, and derives implications from the relationship
without defining the meaning of ``causality.''  Other research derives
behavior from structure using symbolic simulation without any essential
reference to causality at all.  The meaning of causality, and the
criteria for legitimately asserting a causal relationship given
observations of regularities in the world, have been of concern to
philosophers of science and statisticians.

In his role as philosopher of science, Simon (18) defines the causal
relation to hold between {\it sentences} describing the world, rather
than between events or states in the world.  His definition provides
criteria for asserting that one sentence causally precedes another, with
respect to a given set of laws describing the world.  The criteria are
intended to model the ordinary usage of the term ``causality,''
including the irreversibility of the causal relations, so that it can be
appropriate to say {\it The rain causes John to wear his raincoat}, but
not the contrapositive {\it John's not wearing his raincoat causes it
not to rain}.  Simon's (18) approach to causality is, of course, one
position on a major philosophical issue that extends far beyond the
scope of this article.

In the RX project, Blum (19) develops an ``automated statistician''
that examines medical evidence stored in a time-oriented database and
applies the appropriate statistical tests to determine whether a causal
relationship can be inferred.  RX follows accepted procedures in
statistical data analysis, and uses an operational definition of
causality:  ``A is said to cause B if over repeated observations (1) A
is generally followed by B, (2) the intensity of A is correlated with
the intensity of B, (3) there is no known third variable C responsible
for the correlation.'' (p. 19.)  Similarly to (4),
causal links in RX have a number of attributes including numerical ones
such as intensity and frequency.  The third test of causality, for {\it
non-spuriousness}, is the central concern of RX, and is responsible for
the bulk of its domain-specific knowledge-base of known causal links.

Knowledge of the commonsense physical world, however, is much more
complex, and includes much knowledge that must be considered ``causal,''
but that does not fit neatly into the causal link or symbolic simulation
categories of representation.  For example, we know that water poured
from a canteen might splash off a rock, wetting both the rock and one's
legs, then soak into the ground.  The main thrust of the research
described in previous sections is to create the {\it inferential power}
to understand or predict the behavior of complex mechanisms.  However,
examples such as the pouring water demonstrate that a major unsolved
problem in causal reasoning is the {\it expressive power} to state,
adequately, the simple facts and simple relationships that constitute
commonsense knowledge about the physical world.  Hayes (14) argues
that causality, as such, is not a self-contained category that can
support independent axiomatization.  Rather, it is a type of knowledge
that needs to be represented separately for different physical domains:
causal knowledge about liquids is knowledge about liquids, not about
causality.

In the ``Naive Physics Manifesto,'' Hayes (14) argues for a research
program to develop representation languages with sufficient expressive
power to handle issues such as these.  He lays out a set of domains in
which progress would be possible and useful:  scales for measurement;
shape, orientation and dimension; inside and outside; histories; energy
and effort; assemblies; support; substances and physical state; forces
and movement; and liquids.  The concept of {\it histories},
particularly, has been useful for constraining the types of causal
interactions that need to be considered in a given situation, and thus
provides a way to avoid the frame problem (cf. Forbus (12,13)).  An
object or event is considered to have four-dimensional spatio-temporal
extent, called its history.  One's knowledge of an object or event
includes knowledge of how far it is likely to be extended, and what
kinds of boundaries define its limits, in both space and time.  Two
objects can interact only if their histories intersect.  Thus, moving a
block on a tabletop cannot affect a block that is now in the other room
because the histories of the two blocks do not intersect and because the
history of a {\it move} action does not have a sufficient radius to
affect both blocks.  On the other hand, the {\it move} action could
potentially affect other blocks on the same table, so a more careful
check for collisions is warranted.

McDermott (20) proposes a semantic foundation for the concept of time,
which includes treatments of events, causal relations, continuous
change of quantities, and the persistence of facts.  In this theory,
time is organized as a branching set of {\it chronicles}, with a single
past and multiple possible futures.  A chronicle is a global history of
the entire universe, in contrast to the spatially bounded histories
belonging to individual objects in Hayes' (14) approach.  In order to
handle the frame problem, McDermott associates with each fact a {\it
persistence} or default period of time during which that fact, once known, can be
assumed to remain true, in the absence of information to the
contrary.  With these as foundation, his logic makes it possible to
assert causal relations among events in the {\it causal link} style, 
to define the set of influences on a continuously changing quantity in
the {\it qualitative simulation} style, and to plan or predict future
events.  The device of alternative future chronicles allows him to handle
some cases of the semantically difficult verb ``prevent,'' referring
to the relation between a current action and a future event which does
{\it not} take place!

In this very active research area, certain influential papers such as
Hayes' ``Ontology for Liquids'' and the fruits of the ``Commonsense
Summer'' project, circulate only in prepublication form.\footnote{Sorry!
Maybe I'll be able to fix this in proof.  B.K.} These and similar
foundational studies are essential to the eventual integration of
different and seemingly incompatible approaches to causal reasoning into
a common framework.

\clearpage
\section{COGNITIVE MENTAL MODELS}

The search for new knowledge representations in AI benefits by
considering the constraints on human representations discovered by
cognitive scientists.  In order to describe human knowledge, a
representation must be able to express the kinds of variation ---
individual, developmental, and historical --- in knowledge and
problem-solving performance that is observed in the human population.
Evidence of this expressive power suggests design directions for
knowledge representations.  

As in many other domains of knowledge, Jean Piaget (21) provides the
earliest systematic studies of causal knowledge, looking at the types of
early theories children create to explain such mechanisms as the wind
blowing, clouds moving, boats floating, water levels, shadows, bicycles,
steam-engines, trains, motorcars, and airplanes.  In (21), he
identifies a large variety of distinct types of causal explanation,
grouped into three classes:  the psychological/magical, the animistic,
and finally the rational/mechanical.  Only the third class encompasses
what we normally consider as ``causality.''  The evolution through these
classes is driven by three processes of cognitive development:  (1) the
progression from subjective to an objective view of causality as
external to the self, (2) from the view of causality as an
almost-immediate co-occurrence of events, to an ordered sequence of
intermediate steps, and (3) from causal relations seen as irreversible
changes to reversible mechanical connections.  

Causal knowledge has great influence on other kinds of reasoning.
Tversky and Kahneman (22) have shown the importance of causal
knowledge in the overall problem-solving behavior of people by
demonstrating the strong dominance of schemas of causal relations over
the other phenomena that influence estimates of relative and absolute
probabilities.

Adults frequently have incorrect models of physical causality.
McCloskey, Caramazza and Green (23) demonstrate a number of consistent
errors in adult predictions for physical situations involving
curvilinear motion.  Many of their subjects behaved as though
curvilinear motion were not the result of an imposed force, but as if it
were a property of motion like velocity that could change only gradually
under external influences.  For example, a pea blown out of a spiral
pea-shooter is predicted to follow a gradually flattening curve rather
than a straight line (because of ``conservation of angular momentum!'').
These and similar experimental examples bear a striking resemblance to
the pre-Galilean ``impetus'' theory of motion (24,25),
suggesting that the history of science may contain useful
evidence about the nature and evolution of knowledge representations.
Wise and Carey (26), however, caution that scientific theories of an
earlier age were embedded in elaborate belief systems that differ
greatly from our own, so structural analogies between the history of
science and commonsense knowledge in specific problem domains should be
treated with care.

Constraints from empirical observations of human behavior may be
combined with the computational constraints of the problem, to discover
or evaluate novel knowledge representations.  Williams, Hollan, and
Stevens (27) demonstrate this strategy by collecting and analyzing
protocols of subjects reasoning about a heat exchanger, and creating a
set of different computational models corresponding to the different
models observed in the subjects.  Similarly, Kuipers and Kassirer (7)
collect and analyze a protocol of an expert physician explaining a
physiological mechanism, and combine empirical and computational
constraints to justify a qualitative simulation representation.

\clearpage
\section{BIBLIOGRAPHY}

\begin{enumerate}

 \item 
C. Rieger and M. Grinberg.  The declarative representation and
procedural simulation of causality in physical mechanisms.
{\it Proceedings of the Fifth International Joint Conference on
Artificial Intelligence (IJCAI-77)}, 250 - 256.

 \item 
R. C. Schank and R. P. Abelson.  {\it Scripts, Plans,
Goals and Understanding}.  Hillsdale, NJ:  Lawrence Erlbaum Associates, 1977.

 \item 
S. M. Weiss, C. A. Kulikowski, S. Amarel, and A. Safir.
A model-based method for computer-aided medical decision-making.
{\it Artificial Intelligence} {\bf 11}:  145 - 172, (1978).

 \item 
R. S. Patil, P. Szolovits, and W. B. Schwartz.
Causal understanding of patient illness in medical diagnosis.
In {\it Proceedings of the Seventh International Joint Conference
on Artificial Intelligence (IJCAI-81)}, 893 - 899.

 \item 
H. E. Pople, Jr.  Heuristic methods for imposing structure on
ill structured problems:  The structuring of medical diagnostics.
In P. Szolovits (Ed.), {\it Artificial Intelligence in Medicine}.
AAAS/Westview Press, 1982, 119 - 190.

 \item
B. J. Kuipers.  The limits of qualitative simulation.
In {\it Proceedings of the Ninth International Joint Conference on
Artificial Intelligence (IJCAI-85)}.  William Kaufman, Los Altos, CA.

 \item
B. J. Kuipers and J. P. Kassirer.
Causal reasoning in medicine:  analysis of a protocol.
{\it Cognitive Science} {\bf 8}: 363 - 385, (1984).

 \item
J. de Kleer.  The origin and resolution of ambiguities in
causal arguments.  Proceedings of the Sixth International Joint
Conference on Artificial Intelligence (IJCAI-79).
William Kaufman, Los Altos, CA, 197 - 203.

 \item
J. de Kleer and J. S. Brown.  A qualitative physics
based on confluences.  {\it Artificial Intelligence} {\bf 24}: 7 - 83, (1984).

 \item
B. Williams.  The use of continuity in a qualitative physics.
In {\it Proceedings of the National Conference on Artificial Intelligence
(AAAI-84)}, 350 - 354.  (a)

 \item
B. Williams.  Qualitative analysis of MOS circuits.
{\it Artificial Intelligence} {\bf 24}: 281 - 346, (1984).

 \item
K. D. Forbus.  Qualitative reasoning about physical processes.
In {\it Proceedings of the Seventh International Joint Conference on Artificial
Intelligence (IJCAI-81)}.  William Kaufman, Los Altos, CA, 326 - 330.

 \item
K. D. Forbus.  Qualitative process theory.  
{\it Artificial Intelligence} {\bf 24}: 85 - 168, (1984).

 \item
P. J. Hayes.  The naive physics manifesto.  In D. Michie (Ed.),
{\it Expert Systems in the Micro-Electronic Age.}
Edinburgh:  Edinburgh University Press, 1979, 242 - 270.

 \item
B. J. Kuipers.  Commonsense reasoning about causality:
deriving behavior from structure.
{\it Artificial Intelligence} {\bf 24}: 169 - 204, (1984).

 \item 
H. G. Barrow.  VERIFY:  A program for proving correctness of
digital hardware designs.  {\it Artificial Intelligence} {\bf 24}:  
437 - 491, (1984).

 \item 
R. Davis.  Diagnostic reasoning based on structure and behavior.
{\it Artificial Intelligence} {\bf 24}:  347 - 410, (1984).

 \item 
H. A. Simon.  {\it Models of Discovery}.
Boston:  D. Reidel, 1977.

 \item 
R. L. Blum.  {\it Discovery and Representation of Causal Relationships 
from a Large Time-Oriented Clinical Database:  The RX Project}.  
Lecture Notes in Medical Informatics Vol. 19.  New York:  Springer-Verlag, 1982.

 \item
D. McDermott.  A temporal logic for reasoning about
processes and plans.  {\it Cognitive Science} {\bf 6}:  101 - 155, (1982).

 \item
J. Piaget.  {\it The Child's Conception of Physical Causality}.
London:  Routledge and Kegan Paul, (1930).  (Reprinted Totowa, NJ:  Littlefield,
Adams \& Co, 1966.)

 \item
A. Tversky and D. Kahneman.  Causal schemas in judgments under
uncertainty.  In M. Fishbein (Ed.), {\it Progress in Social Psychology,
vol. 1}.  Hillsdale, NJ: Erlbaum, 1980.

 \item
M. McCloskey, A. Caramazza, \& B. Green.  Curvilinear motion
in the absence of external forces:  naive beliefs about the
motion of objects.  {\it Science} {\bf 210}:  1139 - 1141, (1980).

 \item
M. McCloskey.  Naive theories of motion.
In D. Gentner and A. Stevens, (Eds.).  {\it Mental Models}.
Hillsdale, NJ:  Erlbaum, 1983, 299 - 324.

 \item
J. Clement.  A conceptual model discussed by Galileo and used intuitively
by physics students.
In D. Gentner and A. Stevens, (Eds.).  {\it Mental Models}.
Hillsdale, NJ:  Erlbaum, 1983, 325 - 340.

 \item
M. Wiser and S. Carey.  When heat and temperature were one.
In D. Gentner and A. Stevens, (Eds.).  {\it Mental Models}.
Hillsdale, NJ:  Erlbaum, 1983, 267 - 298.

 \item
M. D. Williams, J. D. Hollan, and A. L. Stevens.
Human reasoning about a simple physical system.
In D. Gentner and A. Stevens, (Eds.).  {\it Mental Models}.
Hillsdale, NJ:  Erlbaum, 1983, 131 - 154.

 \item
D. G. Bobrow (Ed.).  {\it Qualitative Reasoning about Physical Systems}.
Cambridge, Massachusetts:  Bradford Books/MIT Press, 1985.
 {\it A collection of papers, reprinting the contents of {\it Artificial
Intelligence} {\bf 24} (1984), and giving an excellent overview of the
qualitative simulation and symbolic digital simulation types of causal
reasoning.}

 \item
D. Gentner and A. Stevens, (Eds.).  {\it Mental Models}.
Hillsdale, NJ:  Erlbaum, 1983.
 {\it A collection of papers giving the best overview of cognitive
science research on causal reasoning.}

\end{enumerate}

\end{document}
