%-*- Mode: Latex; Default-character-style: (:SWISS :ROMAN :NORMAL) -*-

\documentstyle[12pt]{article}
\newcommand{\mf}[2]{#1}
\setlength{\oddsidemargin}{30pt}
\setlength{\evensidemargin}{30pt}
\setlength{\topmargin}{0pt}
\setlength{\textheight}{8in}
\setlength{\textwidth}{6in}
\renewcommand{\textfraction}{.1}
\renewcommand{\topfraction}{.8}
\renewcommand{\floatpagefraction}{.7}
\raggedbottom
\pagestyle{myheadings}
\markright{Submitted to Artificial Intelligence, version of \today}

\begin{document}

\begin{flushleft}
{\large \bf Book Review}\\[5mm]
P. L. Miller, {\em Expert Critiquing Systems: Practice-Based Medical
Consultation by Computer} (Springer-Verlag, New York, 1986); 175 pages.
\end{flushleft}
\begin{tabbing}
Reviewed by: \=Michael P. Wellman\\
\> {\em MIT Laboratory for Computer Science, Cambridge, MA $\;$ 02139,}\\
\> {\em U.~S.~A.}
\end{tabbing}

\renewcommand{\baselinestretch}{1.2}
\large
\normalsize

It is far easier to criticize ideas than to originate them.  I state this rather
obvious fact not only to express humility in my current role as reviewer, but
also to emphasize a matter of technical significance in the context of Perry
Miller's enterprise to develop what he calls {\em expert critiquing systems}.

Critiquing systems differ from the usual recommendation-oriented programs
in that they require and respond to a proposed solution provided by the
user.  In a medical context---the application environment of interest to
Miller---this means that the physician supplies the patient description plus
a proposed course of therapy and receives in exchange a critique of the
submitted plan.  No precise definition of ``critique'' is available, but the
examples presented in the book tend to include a mix of material about
the important issues in the patient description, the pros and cons of
elements of the therapy plan proposed, and often counter-suggestions that
mention alternative therapies not included in the user's proposal.

Miller, champion of the critiquing approach to medical AI, argues that
consultation systems that critique rather than dictate are likely to be more
acceptable and useful to clinicians for a variety of reasons.  Features of
clinical practice, such as variability and subjectivity, are cited as central
motivations, along with the importance of explanation, the opportunity to
inject education into consultation, and other sociological factors of the
health-care-delivery environment.

These motivations are undoubtedly valid and are perhaps sufficient in
themselves to warrant adoption of the critiquing approach.  In this
reviewer's opinion, however, these factors are overwhelmed by the basic
computational law mentioned above: generating useful critiques is easier
than producing good recommendations.

To illustrate this, consider anesthesiology, the domain of {\sc attending},
patriarch of the {\sc attending} family of critiquing systems described in
this book.  The fact that {\sc attending} is the sole AI program applied to
this domain is not only a consequence of Perry Miller's uniqueness in the
intersection of anesthesiologists and AI researchers.  Anesthesiology is a
difficult problem for knowledge-based technology because it touches on a
large fraction of the entire field of medicine.  Patients require anesthesia
not for any direct curative effects of the anesthetic agents, but rather to
enable the necessary surgical treatments for some other serious medical
problem.  Though anesthesiologists need not know as much as specialists in
each of these possible disorder categories, a broad understanding of the
medical situations and their relation to anesthesia strategies is essential.

It is precisely this breadth that we find difficult to achieve in
knowledge-based recommendation systems.  Yet a critiquing system can
provide non-negative incremental value to an anesthesiologist without
necessarily knowing enough to generate anesthesia strategies on its own.
For example, suppose our program had a rather limited knowledge base that
included only the fact that patients with a full stomach should be intubated.
Such a program would produce anesthesia management plans that were
severely incomplete.  Nevertheless, in critiquing mode the program could be
employed to verify that the proposed anesthesia plans are acceptable as
far as it knows.  Given that it knows little, the program would accept the
clinician's plan in virtually all aspects.  In this case, the program would only
assert itself in extreme lapses of judgment or perhaps in bizarre patient
situations.  Knowing nothing about extenuating circumstances, the program
must defer to the physician's judgment when conflict persists.

Of course, the trivial system described above is not worth the electricity
required to run the computer.  The situations in which we are really
interested are those where the knowledge base is extensive in some areas
but not broad enough to provide acceptable recommendation performance on
the mix of cases naturally arising in practice.  Experience with medical
applications of expert systems suggests that most fall within this region.
Rather than offer recommendations that are incomplete, insensitive to
practice variation, inadequately explained, and perhaps
dangerous---especially around the knowledge borders---programs operating
in such an area might better limit themselves to trapping obvious mistakes
and suggesting local, incremental improvements.

The feasibility of the critiquing approach depends on the availability of
users qualified to generate proposed solutions.  This condition is often met
in health care environments where knowledge-based systems are applied.
When reasonable proposed solutions are not available, critiquing is simply
not a viable alternative to recommending.

In extending knowledge-based technology to broader domains, it seems
logical that where there are solutions to criticize, critiquing programs should
naturally precede recommenders.  In one way history does bear this out, as
non-AI applications of computers to medicine sometimes perform tasks
closer to critiquing than to decision-making from scratch.  A notable
example is {\sc mediphor}~\cite{Tatro75}, a program developed at
Stanford University Medical Center in the early 1970s to detect potentially
harmful interactions among drugs ordered through the hospital pharmacy.  I
suspect that Miller would argue that {\sc mediphor} is not a true critiquing
system, primarily because of the simplicity of the ``critiques''---merely
warning messages listing the interacting drugs plus some canned information
about the interaction.

Critiques provided by the various {\sc attending} programs are a bit more
sophisticated.  Miller uses augmented transition networks for text
generation, with the traversal logic dependent on patterns detected in the
patient description and proposed therapy plan.  The mechanisms employed
are not novel or specific to critiquing in any interesting way; readers
well-versed in AI representation and reasoning techniques are bound to be
disappointed.  Miller includes all of the details, complete with appendices
full of {\sc lisp} code.  I would advise impressionable AI programming
novices to tear out and discard these pages before taking up the rest of the
book.

The efficiency with which Miller has turned out critiquing programs since
{\sc attending} was introduced in 1983 is impressive.  The complete family
(and their application domains) consists of {\sc attending} (anesthesia), {\sc
pheo-attending} (pheochromocytoma workup), {\sc ht-attending} (essential
hypertension), and {\sc vq-attending} (ventilator management).  The last
two were implemented with the inevitable ``{\sc essential}''-{\sc attending}
critiquing shell.  And the procreation continues; the book refers to several
systems under development at the time of writing.

Miller attempts to draw some broad methodological and technological
conclusions from this experience but does not fully succeed because the
programs under analysis are not generally committed to strong architectural
hypotheses about the process or content of critiquing.  The original
anesthesia program is perhaps exceptional in that it incorporated original
methods (though seriously flawed from a decision-theoretic perspective) for
the representation and analysis of ``risk.''  But generalizations appearing in
Miller's discussion of {\sc ht-attending} as critiquing based on established
practice compared to {\sc vq-attending} as critiquing organized around
therapy goals are not convincing because concepts such as ``established
practice'' and ``therapy goals'' do not enjoy special representational status
(that is, they have no semantic constraints) in his systems.

As Miller himself points out, it is far from clear how to evaluate the
performance of these systems.  The value of critical advice depends on its
salience and understandability, as well as its potential for leading to actual
improvements in the therapy plan.  Satisfaction of these criteria is more
difficult to assess than correctness of medical assertions appearing in the
critique.

Except indirectly via example, Miller does not attempt to outline a set of
basic principles for critiquing.  Designers of critiquing programs need some
basis for deciding which sorts of comments are appropriate, taking into
account legitimate variations in practice and the possibly superior
knowledge of the user.  For example, suggestions for modifications that will
clearly improve the plan should definitely be included, while warnings about
the potential risks of therapy are more questionable in situations when a
knowledgeable physician is probably well aware of them.  An expert
pestering system would see little use.  The {\sc attending} programs
appear to offer a spectrum of thresholds for speaking up, though none
explicitly assesses this tradeoff in the process of formulating a critique.

All this suggests that there remains a need for serious AI research that
takes the critiquing mode of I/O behavior as a premise rather than an
afterthought.  One exception to the general void is {\sc roundsman}, a
program developed by Rennels~\cite{Rennels87a} to critique proposed
breast cancer therapies based on a knowledge base of clinical studies on
the subject.  Starting from a critiquing vantage enabled Rennels to
concentrate on representations of the clinical literature that support
comparative rather than absolute evaluation of therapies.  Identification of
distinctions such as this one can help us understand the fundamental
differences between knowledge required for critiquing and recommending.

Thanks to Miller, we have an idea of what critiquing programs might look
like and perhaps some anecdotal evidence of how they are used and
accepted in the field.  The next steps are
\begin{enumerate}
\item to develop knowledge representations and reasoning architectures that
maximally exploit the leverage available in the proposal of a user whose
knowledge overlaps that of the program, and
\item to develop a theory of critiquing that provides a basis for choosing
the content and presentation for advice.
\end{enumerate}
Both are fruitful areas for research in knowledge-based systems that are
easily overlooked when focusing on the dominant recommendation-oriented
paradigm.

\renewcommand{\baselinestretch}{1.0}
\large
\normalsize

\bibliographystyle{ai}
\bibliography{bibcite:bibcite;bib}

\end{document}
