Management Information Systems Research Center, University of Minnesota MIS Quarterly

Expert Systems: A Question of Liability?
Author(s): Kathleen Mykytyn, Peter P. Mykytyn, Jr. and Craig W. Slinkman

Source: MIS Quarterly, Vol. 14, No. 1 (Mar., 1990), pp. 27-42
Published by: Management Information Systems Research Center, University of Minnesota
Stable URL: https://www.jstor.org/stable/249305
Accessed: 27-06-2019 08:46 UTC
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms
Management Information Systems Research Center, University of Minnesota is

collaborating with JSTOR to digitize, preserve and extend access to MIS Quarterly
This content downloaded from 103.231.241.102 on Thu, 27 Jun 2019 08:46:00 UTC
All use subject to https://about.jstor.org/terms
Expert Systems Liability
Introduction
Expert
Expert Systems:
Systems:AA
Question
Question of
of Liability?
Liability? Edward A. Feigenbaum, alluding to a Business
Week (1981) article labeling artificial intelligence
as "the second computer age," states that
By:
By: Kathleen
KathleenMykytyn
Mykytyn "... it's not just the second computer revolution,
Peter
Peter P.
P. Mykytyn,
Mykytyn,Jr.
Jr. it's the important one" (Mishkoff, 1986, pg. v).
Feigenbaum believes that artificial intelligence
Craig
Craig W.
W. Slinkman
Slinkman
will shape a world revolving around a new tech-
nology that will embody knowledge as its central
Information
InformationSystems
Systemsand
and feature. Expert systems (ES) are a segment of
Management
ManagementSciences
Sciences artificial intelligence that includes both computer
University
University of
ofTexas
TexasatatArlington
Arlington technology and knowledge, not just data.
Arlington,
Arlington, Texas
Texas76019
76019
The excitement generated by the advent of ES,
an outgrowth of artificial intelligence technology,
has led to prodigious research and substantial
financial investment in these systems. They have
taken on a high-tech mantle and have been the
Abstract subject of a great deal of "hype" as they have
proceeded to the marketplace. Expert systems
The development of expert systems has changed are still in the infant state, yet marketers are pro-
dramatically in recent years based largely on con- moting a product imbued with almost magical
cepts dealing with artificial intelligence. These ef- qualities. Such rhetoric surrounding ES may be
forts are evolving from very specific, academically creating unrealistic expectations. This may be
oriented efforts, such as medical diagnosis, to especially troublesome for corporate managers
more managerially oriented corporate issues. Un- who may know very little about the technology
fortunately, many proponents of these systems yet seek the benefits advertised in trade journals
may be overlooking possible legal ramifications or touted by information systems management
related to both the development and use of these and/or vendors and ES developers. This scenario
systems. A major issue concerns the establish- is similar to that which evolved 30 to 35 years ago
ment of liability for the decisions and recommen- when organizations first looked to data process-
dations made by expert systems. Some liabilities ing to solve problems of inefficiencies.
could include product liability and negligence. All
The technological excitement of ES must,
individuals involved with expert systems (knowl-
however, be balanced against the possible social,
edge engineers, domain experts, and users) are
political, and other organizational implications of
potentially subject to legal scrutiny. It behooves
these systems. In addition, even if enterprises are
organizations involved with these systems to in-
successful in the development and use of ES,
vestigate potential legal problems concerning
they may be overlooking another potentially crit-
them. As these systems become more pervasive,
ical corporate perspective: the legal ramifications
courts may look upon them for what they are:
arising from their development and use. One par-
systems of knowledge and experience, not simply
ticular question that may need to be addressed
passive computer software. A framework is pre-
concerns the establishment of liability for the
sented that traces the development of a product,
decisions or recommendations these systems
legal issues related to the development, and nor-
make. Although corporate management is con-
mative measures that organizations can take to
tinually faced with the legal consequences of
forestall possible legal calamities.
most endeavors, the consequences associated
with expert system development and use are
Keywords: Expert systems, legal aspects of new, leaving many organizations with few legal
computing, legal aspects of expert
guidelines. This article is aimed at improving the
systems balance between ES excitement on the one hand
ACM Categories: H.4.m, 1.2.m, K.1, K.5, K.5.1, and sound managerial and executive perfor-
K.5.2, K.5.m mance on the other.
MIS Quarterly/March 1990 27
It
It has
hasbeen
beennoted
notedthat
that
thethe
users
users
of (expert)
of (expert) the expertise
expertise in
in the
the domain
domainof
ofthe
thesystem
systembeing
being
systems
systemswill willhave
haveto to
be be
emotionally
emotionally prepared
preparedto to developed.
developed. He
He or
or she
she need
need not
notunderstand
understandre- re-
interact
interactwith witha less
a lessthan
than
perfect
perfect
program.
program. The The lated technology
technology because
because that
thatisisthe
thefunction
functionof of
users
users will
willneed
needtoto
be be
as forgiving
as forgiving of ignorance
of ignorance the knowledge
knowledge engineer.
engineer. Knowledge
Knowledgeacquisition,
acquisition,
in
in ES
ES asasthey
theyareare
forgiving
forgiving of ignorance
of ignorance in hu- in hu- the process
process of
of obtaining
obtaining thetheknowledge
knowledgefromfrom
mans
mans (Bachant
(Bachantand andMcDermott,
McDermott, 1984).1984).
In anIn ar-an ar- human
human experts,
experts, is
is based
based upon
uponaarelationship
relationshipbe-be-
ticle
ticle in
inComputers
Computers andandLaw,
Law,PaulPaul
Freeman
Freeman (1986)(1986) tween the
the knowledge
knowledge engineer
engineerandandexperts,
experts,a are-
re-
succinctly
succinctlyemphasizes
emphasizes thisthis
position,
position,
indicating
indicating lationship
lationship that
that is
is crucial
crucial for
forthis
thisprocess
processtotobebe
that
that "...
"...asasevery
every attorney
attorney knows,
knows,in human
in human af- af- successful.
fairs
fairs the
thefailure
failureofofexperts
experts often
often
leadsleads
to litiga-
to litiga-
The second component of ES is the inference
tion
tion rather
ratherthan thanforgiveness"
forgiveness" (pg. (pg.
9). Thus,
9). Thus,
the the
engine. This element, using appropriate heuristic
issues
issuessurrounding
surrounding ES ES
willwill
certainly
certainly
be ofbeinterest
of interest
search techniques, defines how the rules in the
to
to private
privateand andcorporate
corporate attorneys.
attorneys.
knowledge base are to be applied to the problem.
Corporations
Corporationshave
havea responsibility
a responsibility
to their
to their An inference engine, in essence, "runs" an ex-
customers
customersand andshareholders
shareholders to take
to take
all necessary
all necessary pert system. It decides which rules are to be uti-
precautions
precautionsconcerning
concerning thetheuse use
of any
of any
new tech-
new tech- lized, accesses the appropriate rules in the
nology
nologyandandthe
thepossible
possiblelegal
legal
problems
problems
that that
mightmight knowledge base, executes the rules, interacts
arise.
arise. Concerning
Concerning ES,ES,
these
theseproblems
problems couldcould
af- af- with the user to gain additional information about
fect
fect those
thosedirectly
directly involved
involved withwith
theirtheir
develop-
develop- the problem, and makes a decision or recommen-
ment
ment andanduse:
use:the
the
expert,
expert,thetheknowledge
knowledgeengi-engi- dation when a satisfactory solution has been
found.
neer,
neer, and
andthe
theuser.
user.
Ultimately,
Ultimately,
of course,
of course,
the final
the final
legal
legal burden
burdenmay
mayrest
rest
with
with
the the
corporation
corporation
itself.itself. The third component is the user interface. The
The
The purpose
purposeofof this
this
article
article
is toisdraw
to draw
attention
attention user interface communicates with the user, trans-
to
to some
someof ofthe
thelegal
legal
issues,
issues,
suchsuch
as liability,
as liability,
that that lating input from the user into information under-
may
may arise
ariseconcerning
concerning ES.ES.
SomeSome
current
current
comput-comput- standable by the system. The communication is
bidirectional: the intended user must be able to
er-related
er-relatedlitigation
litigationis also
is also
discussed.
discussed.
Finally,
Finally,
a a
framework
frameworkisispresented
presented that
that
traces
traces
the develop-
the develop- communicate with the system and understand
ment
ment and
andgrowth
growth of of
a product
a productwithwith
respect
respect
to to the system's responses as well as the questions
legal
legal activities
activitiesthat
that
may
may
impact
impact
it. Itit.is It
hoped
is hoped
that that the system asks. Although knowledge engineers
researchers
researchersand
andcorporate
corporate
managers
managers
will will
exam-exam- and other professionals involved with the
ine
ine the
theframework
frameworkto to
forestall
forestall
potential
potential
legal legal
harm harm development of ES usually have a great deal of
that
that could
couldresult
resultas as
ESsESs
evolve
evolve
commercially.
commercially. familiarity and experience with computers, the in-
Basic
Basic components
components andand
some
some
current
current
directions
directions tended users are often computer novices. Thus,
of
of these
thesesystems
systemsareare
identified
identified
initially
initially
to estab-
to estab- like any computer-based system, an expert sys-
lish a foundation for discussion. tem must be easy to use (Turner, 1987). A pro-
perly developed user interface will help ensure
support for the needs, preferences, and individual
Expert Systems differences of the users.
The principal components of most expert sys-

tems often referred to as knowledge-based sys-
Current directions in expert systems
tems, are a knowledge base, an inference Much has been written in the past decade about
engine, and a user interface (Mishkoff, 1986). The the development of ES. Prominent examples in-
knowledge base contains the system's factual clude MYCIN (Buchanan and Shortliffe, 1985),
knowledge as well as the heuristics of the ex- CADECEUS and INTERNIST (Pople, 1982), and
pert(s). The knowledge is often represented in the PROSPECTOR (Duda and Reboh, 1984). MYCIN
form of "if... then" rules. For example: IF the is a system designed to assist physicians in
dollar rises against foreign currencies, THEN in- diagnosing meningitis and bacteremia;
terest rates decline. The knowledge engineer, an CADUCEUS and INTERNIST are systems used
artificial intelligence professional, is skilled in the to diagnose problems dealing with internal
art of developing ES but may be unfamiliar with medicine; PROSPECTOR was designed to assist
the specific domain of the system. A domain ex- geologists in the early stages of investigating
pert is an individual who is considered to have sites for ore-grade deposits. Other examples in-
28 MIS Quarterly/March 1990
clude
clude XCON
XCON(McDermott,
(McDermott, 1982),
1982),
a system
a systemjoint-
joint- trained
trained in
in law;
law; for
for ease
ease of
ofreference,
reference,the
thefirst
firstap-
ap-
ly
ly developed
developedbybyDigial
Digial
Equipment
Equipment Corporation
Corporation propriate
propriate occurrence
occurrence ofof these
theseterms
termsisisitalicized.
italicized.
and
and Carnegie
CarnegieMellon
MellonUniversity
Universityto to
assist
assist
in the
in the The appendix
appendix contains
contains definitions
definitionsfrom
fromBlack's
Black's
configuration
configurationofofVAX
VAX computer
computer systems,
systems,and and Law Dictionary
Dictionary (Black,
(Black, 1979).
1979).When
Whenappropriate,
appropriate,
DELTA/CATS-1
DELTA/CATS-1(Harmon
(Harmon and
and
King,
King,
1985),
1985),
a sys-
a sys- explanations
explanations and/or
and/or examples
examplesin inlayman's
layman'sterms
terms
tem
tem developed
developedby
byGeneral
GeneralElectric
Electric
to help
to help are provided.
provided. They
They are
are illustrative
illustrativeof
ofthe
thepoten-
poten-
maintenance
maintenancepersonnel
personnelmaintain
maintain diesel-electric
diesel-electric tial impact
impact that
that courts
courts ofoflaw
lawcould
couldhave
haveon
onES.
ES.
locomotives.
locomotives.Although
Although these
theseexamples
examplesillustrate
illustrate
Tort and
and contract
contract theories
theoriesconcerning
concerningES ESoffer
offer
the
the attention
attentionthat
thatknowledge-based
knowledge-based systems
systems
are are
some of
of the
the most
most interesting
interestingand
andunprecented
unprecented
continuing
continuingtotoreceive,
receive,
most
most endeavors
endeavors havehave
so so
questions
questions facing
facing both
both the
the legal
legalprofession
professionand and
far
far failed
failedto
toreach
reachbeyond
beyondthethe
prototype
prototypestage
stage
or or
corporate
corporate organizations
organizations of
oftoday.
today.IfIfa adecision
decision
very
very domain-specific
domain-specific applications
applicationssuchsuch
as XCON.
as XCON.
reached
reached by
by an
an expert
expert system
systemisiswrong
wrongor orcauses
causes
Concerning
Concerningthe theoften
often
cited
cited
example
exampleof MYCIN,
of MYCIN,
injury
injury or
or harm,
harm, the
the resulting
resultingdamage
damagecould
couldlead
lead
a system
systemnot
notiningeneral
generaluse,
use,
Buchanan
Buchanan
andand
to litigation
litigation dealing
dealing with
with strict
strictliability
liabilityor
or
Shortliffe
Shortliffe(1985)
(1985)concede
concede
that
that
its its
breadth
breadth
of of
negligence.
negligence. Each
Each person
person involved
involvedwith
withthe
the
knowledge
knowledgeisisdeficient
deficientsuch
suchthat
that
thethe
system
system
is not
is not
system-the
system-the knowledge
knowledge engineer,
engineer,domain
domainex-ex-
a cost-effective
cost-effectivetool
toolfor
for
physicians
physicians
whowho
work
work
in in
pert, and
and the
the users-is
users-is subject
subjecttotolegal
legalscrutiny.
scrutiny.
MYCIN's
MYCIN'snarrow
narrowdomain.
domain. Further,
Further,
Konsynski
Konsynski
This section
section will
will discuss
discuss the
theareas
areasof
ofpossible
possible
(1988)
(1988) reports
reportsthat
thatonly
onlya handful
a handful
of over
of over
1,500
1,500
conflict
conflict with
with respect
respect to
to these
theseindividuals.
individuals.
currently
currentlyactive
activeESESprojects
projects
have
have
actually
actually
pro-pro-
duced
duced ananorganizational
organizational impact.
impact.
Current
Currentresearch
researchininknowledge-based
knowledge-basedapplica-
applica- Knowledge
Knowledge engineer
engineer
tions
tions isis also
alsotranscending
transcending thethe
boundaries
boundaries
of tradi-
of tradi-
The knowledge
knowledge engineer
engineer isisthe
thedeveloper
developerofofthe
the
tional
tional ESESto tothe
thebroader
broaderdomain
domain of of
expert
expert
support
support
expert
expert system
system and
and the
the critical
criticallink
linkto
tothe
thedomain
domain
systems
systems(ESS).
(ESS).Luconi,
Luconi,etet
al.al.
(1986)
(1986)
state
state
thatthat expert.
expert. The
The time
time involved
involvedin indeveloping
developingthe
thesys-
sys-
ESS
ESS can
can be
beused
usedtotoassist
assist
managers
managersin solving
in solving tem is considerable, often measured in man-
a much
much wider
widerclass
classofof
problems
problems
by by
pairing
pairing
ESs ESs years. This could affect the knowledge engineer
and
and their
theirAl-based
Al-basedtechnologies
technologies
with
with
thethe
inin- by causing that person to think he or she has
dividual
dividual who
whowould
wouldprovide
provide
overall
overall
direction
direction
and and
"learned" from the domain expert (Bobrow, et
specific
specific knowledge
knowledgenot
not
contained
containedin the
in the
expert
expert al., 1986). The feelings or opinions of the
system.
knowledge engineer could find their way into the
Such application of this technology, i.e., ES to system, possibly as invalid information, thus con-
ESS, should not be unexpected. Advances in the taminating the knowledge base. Any biases held
level of sophistication parallel the transition of by the knowledge engineer could also influence
earlier general problem-solving algorithms based the way the decisions are made. This problem
on domain-independent problem-solving proce- could be troublesome during the knowledge ac-
dures to domain-specific ES applications such as quisition process, especially because heuristic
XCON. As organizations strive to achieve com- knowledge is a key component of the knowledge
petitive advantage through the application of in- base. For example, a domain expert may under-
formation technology, knowledge-based systems stand the relationship of his or her experience
may well take their place alongside of or as an and intuition but could conceivably have difficul-
integral part of traditional computer-based sys- ty in conveying that information to the knowledge
tems such as DSS. Such endeavors, however, engineer; it is just as plausible that the knowledge
could open the door to many organizational dif- engineer could misunderstand that information
ficulties, not the least of which could be legal and subsequently misrepresent it.
dilemmas.
Leonard-Barton and Sviokla (1988) address
aspects of this communication problem. They in-
Legal Questions Surrounding dicate that even capturing expert knowledge
could be formidable. Domain experts have dif-
Expert Systems ficulty in retracing the analytic steps taken to
This and later sections of this article contain legal reach some particular decision. They may only
terminology that may be unfamiliar to those not be able to emphasize isolated important factors
dealing with a decision and may have difficulty minating

minating in in
defective
defective
systems.
systems.
Concurrent
Concurrent
neg- neg-
in describing the entire process, thereby possibly ligence
ligence may
maycertainly
certainly
result.result.
contaminating the knowledge base. It is evident
The
Thefailure
failureto consider
to consider
the legal
the issues
legalin issues
the in the
that communication is an essential ingredient in
development
development process
process
could could
result in
result
legal actions
in legal actions
knowledge acquisition and ES development.
involving
involving negligence,
negligence,
both intentional
both intentional
and unin- and unin-
However, communication implies understanding!
tentional.
tentional. Because
Because
a negligence
a negligence
action would
actionandwould and
Unfortunately, a misunderstanding could lead to
should
should involve
involve
the company
the company
employing
employing
the knowl.the knowl.
misrepresentation, with dysfunctional conse-
edge
edgeengineer(s),
engineer(s),
it is it
theisorganization's
the organization's
respon- respon-
quences resulting in legal actions.
sibility
sibility to to
validate
validate
the results
the results
of its product
of itsasproduct as
Since the knowledge engineer is the developer much
muchasas
possible.
possible.
of the system based on expertise supplied by the
domain expert, it would be naive to think that the Domain
Domain expert
expert
knowledge engineer could not, through misun- The
Thegoal
goalof of
ES is
EStoisprovide
to provide
supportsupport
for decisions
for decisions
derstanding or misrepresentation, invadidate the individuals
individuals make
make
or, better
or, better
yet, to yet,
actually
to have
actually have
system. Whether intentionally or unintentionally an
anexpert
expert assistant
assistantavailable
available
to maketo themake
deci- the deci-
done, this type of action could result in active or sion
sion(Hurley
(Hurleyand and
Wallace,
Wallace,
1986). How
1986).
experts
Howare experts are
concurrent negligence. viewed,
viewed, however,
however,is subject
is subject
to widetovariability.
wide variability.
In In
Philip Leith, a knowledge engineer, developed an courts
courtsofof law,
law,
for for
example,
example,
experts experts
are called are
on called on
expert system without using a domain expert to
to testify,
testify,andand
rebuttal
rebuttal
expertsexperts
are used are
to refute
used to refute
(Computers and Law, 1984). Because of the their
theirtestimony.
testimony. Experts
Experts
can find
canpoints
findofpoints
agree- of agree-
simplicity of the information used in E.L.I. (a legal ment,
ment,but
but
often
often
therethere
are more
arepoints
moreofpoints
disagree-
of disagree-
expert system), Leith (not himself trained in law) ment.
ment.The
Theresolution
resolution
of possible
of possible
conflicting
conflicting
expert expert
became an "expert." Such may not always be knowledge,
knowledge, especially
especially
that related
that related
to heuristics
to heuristics
the case, however. It must be remembered that or
orother
other rules
rules
of thumb,
of thumb,
is a difficult
is a difficult
enough task
enough task
one does not become an expert merely through in
in itself;
itself; thethe
representation
representation
of such of
knowledge
such knowledge
in in
casual contact. Such minimal knowledge, leading the
thesoftware
software of an
ofexpert
an expert
systemsystem
is even more
is even more
a person into thinking he or she knows a great challenging.
challenging. Yet Yet
the expert
the expert
system system
does just this
does just this
deal, could be quite harmful. To become an ex- by
bydrawing
drawing a conclusion
a conclusion
and presenting
and presenting
it to the it to the
user.
pert, on the other hand, takes years of training
and experience (Susskind, 1986). Given the current legal environment with respe
In another instance, an organization might be in to lawsuits involving negligence and malpractic
the process of developing an expert system for it would appear that professional experts, as con
future sale to the general business community. tributors to ES, could individually or as a grou
As an employee of the organization, the develop- be held liable for their expertise (Warsaw, 1986
ment process would be the responsibility of the Negligence would be an area of concern for thos
knowledge engineer. If the knowledge is repre- rendering expertise even though they may be re
sented in error, it could produce invalid decisons moved from the actual scene where their expe
tise was used. Such was the case when two
or recommendations. Under the principles of
respondeat superior, an employer/employee rela- structural engineers were found guilty of negli-
tionship, the organization itself could be judged gence in connection with the 1981 walkway col-
legally liable for any injury or damages caused lapse at the Kansas City Hyatt Regency Hotel.
by the faulty system. Negligence occurred even though they were not
present at the job site but were working for
fabricators. Judge James B. Deutsch said they
Much of the preceding discussion has presumed
could not "abdicate" responsibility for connec-
that only one knowledge engineer was involved
tion design to fabricators; thus it would appear
in the knowledge acquisition process. Such may
that such professionals should be on the job site
not be the case. Organizations may attempt to
(Engineering News-Record, 1985). An analogy
minimize the development time or try to validate
could be drawn to all "experts" who contribute
the expertise-gathering process by including two
expertise to ES.
or more knowledge engineers on the same pro-
ject. Unfortunately, this could lead to conflicting The issue to expert contribution should also con-
knowledge acquisition and representation, cul- cern corporations involved with the internal de-
velopment
velopmentofofES.ES.
They
They
maymay
develop
develop
veryvery say what
what is
is required;
required;there
thereare
areprecautions
precautionssoso
im-
im-
complex
complexsystems
systems involving
involving sophisticated
sophisticated
tools tools perative
perative that
that even
eventheir
theiruniversal
universaldisregard
disregardwill
will
and
and knowledge
knowledge engineers
engineers or use
or use
readily
readily available not excuse their omission.
available
ES
ES development
development tools
tools
likelike
expert
expert
system
system
shells,shells,
Averill (1984) indicates that the use of an expert
such
such as
asEXSYS
EXSYS (EXSYS,
(EXSYS, Inc.,
Inc.,
1985),
1985),
depending
depending
system in medical diagnosis could place as af-
on
on their
theirneeds.
needs.In In
either
either
case,
case,
theythey
may may
be rely-
be rely-
firmative duty the responsibility for using the
ing
ing on
onthe
thevoluntary
voluntary compliance
compliance of key
of em-
key em-
system. It could also be used in court as support
ployees
ployeesasasdomain
domain experts.
experts. However,
However,thesethese
for a diagnosis, even though the diagnosis was
employees
employeesmay may bebe hesitant
hesitant
to participate,
to participate,
fear- fear-
incorrect. Even if a physician admitted the diag-
ing
ing potential
potentiallegal
legal
repercussions.
repercussions.
These
These
inin-
nosis was incorrect, a system could be used to
dividuals
dividualsmaymayinsist
insist
on on
somesome
typetype
of insurance
of insurance
support a contention that the incorrect diagnosis
as
as protection
protectionagainst
againstpotential
potential
liability.
liability.
With With
the the
was not reached negligently. Still, the plaintiff
uncertainty
uncertaintyofof the
the
legal
legal
environment,
environment, however,
however,
would be able to attack the credibility and reliabili-
such
such protection
protection may
may notnot
be sufficient.
be sufficient.
ty of the system and thus protect his or her right
Organizations
Organizationsthatthat
develop
develop
ES for
ES for
resale
resale
may may to compensation for negligent misdiagnosis
also
also face
facepotential
potentialproblems
problemsregarding
regarding
the do-
the do- (Averill, 1984).
main
main experts
expertsthey
theyemploy
employfor for
ES development.
ES development.
Zeide and Liebowitz (1987) indicate that negli-
Where will the courts draw the line? Could the
gence may be difficult to prove in cases involving
courts hold the domain expert liable? If the do-
computer software. Another opinion examines
main expert is considered an employee of the or-
the issue from a different perspective, concluding
ganization, could the courts hold both the expert
that medical ES used only by professional medi-
and the company at fault? If the expert is acting
cal personnel may subject the professional to
as a consultant or independent contractor, where
negligence due to the bifurcated sale/service
might liability be placed?
nature inherent in the use of these systems (High
Potential problems arising from the involving of Tech Law Journal, 1986). The issues surrounding
multiple knowledge engineers in ES development the users in medical systems are certainly cloudy;
may also apply to two or more experts contribut- other professional areas may be as obscure, leav-
ing their expertise to a project. In fact, organiza- ing the courts to determine culpability.
tions may seek out the involvement of multiple
Another issue concerning users of ES pertains
experts in their effort to obtain the best and most
to the risk of misapplication and the potential
current knowledge available. The problem may
legal issues associated with it. In typical organiza-
be even more of a concern because of the heur-
tional environments wherein a manager or other
istic nature of the knowledge. Such knowledge
individual is faced with a problem, the process
is based on individual experience, feelings, in-
of gathering appropriate information leading to
tuition, or instincts of each domain expert. The
a decision may be relatively straightforward. The
potential for conflicting expertise is high, possibly
decision maker can look for help or advice from
leading to an invalid system, with the courts be-
corporate associates, external sources, pub-
ing asked to intervene.
lished records and reports, or other sources
deemed appropriate. In such a case the decison
maker may have a feeling for whom and what can
Users be trusted to impart the correct information need-
ed. In a situation where ES may be utilized, the
The evidence thus far has possibly painted the
decision maker lacks such an opportunity. Unless
user as an unwitting victim. Users must realize,
he or she has particular knowledge about the do-
however, that they too can be in a vulnerable
main experts and the knowledge engineer, the
position. By not using an available resource, by
sources drawing conclusions and recommenda-
affirmative duty, they could be negligent by omis-
tions are buried in a "black box."
sion (passive negligence) (Dombroff, 1985). In the
T.J. Hooper case (Averill, 1984), the court found Likewise, in non-ES environments decision
the owner of two tugboats negligent for failing to makers are free to discuss with and question their
equip the tugs with radio sets, even though only associates and other information sources con-
one other company so equipped its tugs. Judge tinually in order to refine the information
Learned Hand writes that courts must in the end provided-an iterative process-ultimately per-
mitting
mitting the
thedecision
decisionmaker
makerto to
make
make
a more-
a more- below the standard of care or that reasonable
informed decision. However, if the decision care could have prevented the injury and was not
maker employs an expert system, such leeway used, and that damages provide appropriate
is constrained. Although the system may contain compensation (Zeide and Liebowitz, 1987). Prov-
a friendly user interface and explanation sub- ing negligence may be difficult, and it could be
system, the decision maker is limited to operating hard for plaintiffs to be compensated for injuries
within the boundaries of the system's domain and caused by computer program failures. It would
the system's capability for providing meaningful be very difficult to pinpoint a mistake among
explanations. The explanations would obviously thousands of lines of code in a program and then
be based on the facts, judgments, intuition, and prove that the injury was reasonably foreseeable
experiences of the experts and the knowledge (foreseeability) (San Diego Law Review, 1983).
engineer's representation of that information. Public policy considerations make strict products
liability more desirable because it eliminates the
A bottom-line conclusion, therefore, is that an ex-
need to prove fault. It should be noted, however,
pert system may be quite effective for one user
but less so, or even erroneous, for other users. that if an action in a particular case is contrary
to standards of conduct established by statutory
A user's knowledge, responses to the system's
law, "negligence per se" may be in order. In such
questions, and user queries of the system leading
an instance, it would not be necessary for a plain-
to subsequent user responses could cause the
tiff to prove that a defendant's conduct consti-
system to reach incorrect conclusions and make
tuted a failure to exercise reasonable care in
faulty recommendations.
order for the defendant to be held negligent.
Finally, the question of correctly defining the ex-
Although strict products liability is often not a
pertise of a user must be dealt with. Expert sys-
recourse for purely economic damage, manufac-
tems are not designed to instruct someone in
turers of unreasonably dangerous products are
dealing with a situation. Rather, the intended user
liable for injuries caused by their products
must be experienced in the domain of the system.
regardless of reasonableness of care. The logic
The amount or degree of that experience may be
behind this is that manufacturers are considered
open to legal examination if, based on the user's
to be in a better position than the consumer to
responses to the system's questions, an incor-
test for, identify, and address a problem, and are
rect recommendation is made and implemented.
able to absorb or spread losses. The law expects,
At some point, the use of ES, right or wrong, for example, that assembly line products have
could be incumbent on the user. Given the claims uniformity in design and quality and that the
being made about their impact on mankind, their manufacturer exercises the control required to
use could be demanded. However, organizations achieve this objective. The consumer has no con-
would do well to exhaustively investigate all trol over the manufacturing process, so if pro-
possible legal ramifications first. ducts are defective and/or unreasonably
dangerous, then the court applies strict liability
to the manufacturer, even if reasonable care was
Negligence and product liability used in production and no amount of care in pro-
Negligence and product liability are legal terms duction could have prevented the injury (Zeide
included within the domain of torts, wrongful acts and Liebowitz, 1987).
subject to civil action. However, questions con-
The uncertainty associated with the economic
cerning ES abound. Does strict liability for defec-
harms doctrine would not protect developers of
tive products apply to ES? If so, how is a defective
medical or other such ES where personal injury
system defined? At what point should a negli-
occurred as a result of defective design. Should
gence standard apply? More succinctly, what
physicians assume complete responsibility for
constitutes negligence in the development of an
any tort liability arising from the use of such
expert system?
systems? Would the appropriate legal standard
Under general negligence theory, given the cir- be one of strict products liability or professional
cumstances of each case, the courts consider malpractice? Although many such systems have
reasonableness of conduct or care. Injured users not progressed beyond the prototype stage, a
of services, suing under theories of negligence, system called SPE, designed to distinguish be-
must prove that a performance of the service fell tween various causes of inflammatory conditions
in
in patients,
patients,did
did
achieve
achieve
commercial
commercial
status
status Warranties and disclaimers
(Waterman,
(Waterman,1986).
1986).
If If
personal
personal
injury
injury
resulting
resulting
A related second question deals with the issue
from
fromthetheuse
useofof
such
such
a system
a system
werewere
to occur,
to occur,
the the
of warranties and disclaimers. Most software
knowledge
knowledgeengineer,
engineer,
domain
domain
experts
experts
and the
andre-
the re-
packages today are sold with limited warranties
spective
spectivecorporate
corporate organizations
organizations
wouldwould
also be
also be
covering the diskettes only. Companies frequent-
subject
subjecttotolegal
legal
inquiry.
inquiry.
ly issue disclaimers stating that the package is
sold and licensed "as is" without any warranty
of any kind concerning performance, accuracy,
or freedom from error. Further, use of the infor-
Expert
Expertsystem-product
system-productor service?
or service?
mation or results from the package shall be totally
The
The question
questionofof
whether
whether
ESs ESs
are aare
product
a product
or ser-
or ser- at the risk of the user. An even more pronounced
vice
vice isisnot
noteasily
easily
answered.
answered.
OneOne
scholarly
scholarly
opi- opi- disclaimer common to the sale of computer soft-
nion states that the determination should be ware indicates that the developer, dealers, dis-
based on how the software is marketed. Off-the-
tributors, or anyone else involved in the creation,
shelf software that is mass-marketed should be
production, or delivery of the package shall not
treated as a product, whereas software that is be liable for damages even if the developer has
custom-designed should be reviewed as a ser- been advised ahead of time of the possibility of
vice (Computer Law Journal, 1984). The writers such damages.
of that article suggest, however, that even soft-
ware that is custom-designed could be treated STOCK TRADING SYSTEM (STS), an expert
as a product if it affects a large number of cus- system developed by Criterion Software, Inc., ap-
tomers. Another view (San Diego Law Review, pears to include even more ES-related terminol-
1983) states that software should be treated as ogy in its disclaimer. STS is a system designed
a hybrid transaction involving both a product and to assist individuals with stock market recommen-
a service. Zeide and Liebowitz (1987) also believe dations. The disclaimer for this product stipulates
that ES software is both a product and a service. in part that the company does not "make any rep-
Finally, another view is that software is a product resentations regarding the use or the results of
because the purchaser is buying a concrete set this product, nor in the validity of the formulae
of instructions that a computer executes, not just or the economic or investment principles on
a set of ideas (Michigan Law Review, 1979). which the formulae are based" (Criterion Soft-
Ducker (1971), also concludes that computer software, Inc., 1987, p. iii). In discussing "formulae"
ware is a product because it is sold, owned, and and "investment principles," an individual and
transported on magnetic tape and computer the courts might perceive those terms to mean
disks. system rules and the factual and heuristic knowl-
edge upon which such rules are based.
The resolution of the product versus service
predicament would appear to be critical for de-
Although the disclaimers are clearly written, their
velopers and users of ES. If ESs are treated as
legality is uncertain. The central issue concerns
a service, injured parties may have to resort to
the product versus service question in terms of
proving negligence among domain experts,
whether the U.C.C. is applicable. Of course, in
knowledge engineers, etc.-a difficult task. How-
cases of personal injury or harm, disclaimers may
ever, if ESs are products, the necessity of prov-
not be upheld by the courts at all. A step toward
ing negligence is eliminated, thereby permitting
resolving this issue may have already been taken,
strict liability actions under the Uniform Commer-
however, since many software limited warranties
cial Code (U.C.C.).
and disclaimers clearly refer to software "as prod-
Organizations developing and using ES must ucts." It is also conceivable that since the soft-
recognize the uncertainty surrounding the prod- ware is an "expert" system, an implied warranty
uct/service environment. They should make of fitness for a particular purpose, namely for
themselves aware of courtroom decisions and what the package is designed to do, may be ap-
possible legislative actions and take appropriate plicable. After all, expert opinion and knowledge
precautions to protect themselves. Illustrative form the base for the system, expert knowledge
normative precautionary measures will be dis- that may be relied on by the purchaser of the soft-
cussed later in this article. ware, disclaimers notwithstanding.
The
Theresponsibility
responsibilityof programmers,
of programmers, engineersa pilot.
engineers pilot. The
Thecompany
companyselling
selling
the
the
read-out
read-out
waswas
and
andusers
userswill
willbe be
the the
focus
focus
of attention
of attention
for future
for future held
held liable
liable for
fornot
notverifying
verifyingthe
the
information
information
on on
thatthat
legal
legalquestions.
questions.TheThe
universal
universal
dependence
dependence
on onread-out
read-out and
andfor
forrelying
relyingonon
the
the
FAA
FAA
forfor
accuracy.
accuracy.
computers,
computers, from
fromseemingly
seemingly
trivialtrivial
mechanical
mechanicalThis
This is
is an
an example
exampleofofstrict
strict
liability
liability
dealing
dealing
with
with
computations
computations to to
decision
decision and recommendation- unreasonably
and recommendation- unreasonablydangerous
dangerousproducts
products regardless
regardless
of of
making
makingsystems,
systems,makes
makes
systems
systems
reliability
reliability
a aconduct
conduct or orcare
care(IEEE
(IEEEExpert,
Expert, 1986).
1986).
crucial factor that must be dealt with. Lawsuits
Organizations
Organizationsinvolved
involvedininthe
the
development
development of of
could result if ESs are consulted and fail to per-
computer
computer systems
systemsand
andsoftware
softwarehave
have
beenbeen
ex-ex-
form correctly, such as providing inaccuracies or
posed
posed to
to court
courtactions
actionsfor
for many
manyyears.
years.
Several
Several
misleading answers that lead to injury (Nycum
examples
examples are
arediscussed
discussedbelow
belowinin
order
order
to to
il- il-
and Fong, 1985).
lustrate
lustrate their
theirimportance.
importance. InIn
1984,
1984,
United
UnitedStates
States
Welding
Welding sued
suedBurroughs
BurroughsCorporation
Corporation because
because
U.S.
U.S. Welding
Weldingwas wasdissatisfied
dissatisfiedwith
with
thethe
perfor-
perfor-
Computer-Related Litigation mance
mance of of aaleased
leasedcomputer
computer system,
system,including
including
the software. The court held that a claim for
Legal problems in the computer industry have
monetary losses caused by a negligent misrep-
certainly affected organizations, e.g., health and
resentation is allowed provided the parties have
other ergonomic issues surrounding video dis-
sufficient privity. In this case the court noted that
play terminals. The impact on the public will de-
pure economic harms may be recoverable in a
pend on the severity of the injuries and/or the
strict products liability claim (United States
frequency of litigation. As lawsuits gain more
Welding, 1984).
public attention, suspicion and distrust of the in-
dustry places it in a legally vulnerable position In 1982 the insurance brokerage firm of Black,
(Joyce, 1987). For example, a Texas lawsuit in- Jackson and Simmons sued IBM Corporation be-
volved a malfunctioning machine, the Therac 25, cause a computer system and software did not
which administered a lethal dose of radiation to function as represented. While not permitting an
a patient. The malfunction occurred because of action based on negligence for purely economic
a programming error in the software. The at- harms, the court did allow a claim based on war-
torney for the plaintiff filed suit against the ranty (Black, et al., 1982).
manufacturer, Atomic Energy of Canada Ltd.
In 1984 Invacare Corporation brought suit against
(AECL); the East Texas Cancer Center (ETCC)
Sperry Corporation. In this action Invacare al-
in Tyler, Texas; and the supervisinig oncologist.
leged that Sperry negligently recommended the
Similar action in Georgia has been brought
purchase of its own computer system. The court
against AECL, the Kennestone Regional On- held that such action would be a valid claim for
cology Center, and the supervising physician.
fraud if the alleged facts were proved. The court
This suit was precipitated by the news stories also held that because of all the contractual
about the Texas incident and involves injuries
agreements were not independent of one an-
from a radiation overdose received by a patient
other, no statute of limitations existed until all por-
in Marietta, Georgia. In the Georgia case, the law
tions of the contract had been completed
firm has requested, among other things, the per-
(Invacare, 1984).
sonnel file of the initial programmer of the Therac
25 (Joyce, 1987). In these particular cases where As a final example, in 1979 Quad County
legal precedent is being established, attorneys Distributing Company sued Burroughs Corpora-
take a traditional "shotgun approach" by suing tion as a result of a dispute involving their con-
everyone who is involved. During the trial, true tract. This case was significant because the court
liability is established and the innocent parties held that a computer program is covered by the
are dropped from the suit (Joyce, 1987). U.C.C. provisions pertaining to the sale of goods,
An extensive review of the literature was con-
possibly establishing a precedent (Quad County
Distributing, 1979).
ducted to determine if ES had been subjected to
lawsuits, but only one case was found. The case The examples cited illustrate the uncharted
concerns faulty information provided by the Fed- nature of the legal setting surrounding computer
eral Aviation Administration; this information wassystems and software. Given the fluctuations in
printed out as a navigational aid for pilots. In thisthe legal environment, as court decisions involv-
case, the faulty information led to the death ofing ES evolve, only time will determine what
precedents
precedents will
will
be most
be most
used in
used
litigation. As the As theStages
in litigation. of
of development-descriptive
development-descriptive
responsibilities
responsibilities of ES
ofdevelopers
ES developers
and users
andareusers are
actions
defined
definedininthethe
courts,
courts,
it will
itbewill
easier
be to
easier
develop
to develop
standards
standards andand
guidelines
guidelines
to meetto the
meetconstraints
the constraintsAt CREATION a new product is conceived. Since
set
setbybytheir
their
decisions.
decisions.
UntilUntil
that time,
thathowever,
time, however, it is undergoing testing and development, it is not
it
it would
wouldbebe
wise
wise
to become
to become
aware aware of the evolu- yet available to the public; therefore, legal activity
of the evolu-
is minimal or non-existent.
tionary
tionarynature
nature
of the
of the
law and
lawuse
andthat
useknowledge
that knowledge
to
to minimize
minimizelegal
legal
risk.risk. At the CURIOSITY stage the product is selective-
ly introduced to the public, resulting in public
fascination, excitement, and increased media ex-
posure. Such excitement and exposure might
preclude any organized endeavor to deal with the
A Framework for Product product's legal impact or potential for same.
Growth and Legal Impact At REFINEMENT the product continues to

evolve, possibly due to technological enhance-
The life cycle of a product has traditionally been
ments. The level of sophistication can produce
tied to many valid factors, but it has not been
an increase in public awareness and demand,
related to the constraints imposed on it by both
suggesting to the manufacturer societal impact
common law or statutory law. Expert systems ap-
and possible financial gain. Perceived complica-
pear to be proceeding along a similar develop-
tions and obstacles could lead to the develop-
mental path. Although they have not been subject
ment of warranties, disclaimers, or contracts.
to many legal difficulties yet, considerable
litigious potential exists over their use, misuse,During APPLICATIONS EXPANDED the product
and/or non-use (Zeide and Liebowitz, 1987). This is intensively marketed with high public accep-
potential could soon be realized, with the court tance. Legal activity is abated, brought about by
determining their development and use. The anticipated product success. However, problems
question, then, that corporate managers should may be occurring that lead to the establishment
ask is just how one should pursue the investiga- of legal precedents and/or a refinement in the
tion of ES technology and the implications of thatwarranties and disclaimers.
technology to effective implementation for cor-
As the product evolves, significant PROBLEMS
porate competitive advantage. To abandon ES
stemming from product defects or injudicious use
efforts, either as a developer or potential user of
may occur. Perceived responsibility for product
this technology, is NOT the avenue to pursue.
development and use seek definition. Such prob-
Instead, the astute managers should become
lems may undoubtedly lead to a decrease in
proactive, thus applying the concept of foresee-
public acceptance and product fascination as
ability, and take the initiative in examining the
well as real and/or perceived injury.
possible legal ramifications of expert systems.
During LITIGATION, disputes are settled in the
To assist in this endeavor, we have developed
courts. Responsibility of both product developer
a framework (see Figure 1) that traces the devel-
and user is defined in the courtroom. These
opment and the stages of growth of a hypotheti-
cases may cause the developer and user to
cal product. More specifically, the framework
assess the product's position in the marketplace,
permits the manager to examine possible legal
to hasten awareness of the product, and to re-
ramifications impacting this product along two
evaluate it to preclude further litigation.
fronts: a descriptive perspective describing what
often occurs or could occur, and a normative ap- LEGAL IMPACT may be felt by court decisions
proach specifying what should be done. We have (common law) as well as the enactment of spe-
illustrated this evolution in a manner such that cific statutes (statutory law) stemming from real
or perceived injury. In some cases the manufac-
possible litigation and/or governmental regulation
could impact the success of the product if nor- turer may elect to, or may even be forced to, re-
mative actions are not taken. At each stage it ismove the product from the market. The impact
understood that applicable actions taken in pre- may be a temporary phenomenon while correc-
ceding stages should be included during the cur- tions or modifications are made, or it may be
rent stage as well. permanent.
Stage of
Development Legal Growth Description Normative Legal Measures
Creation * Creation of new product ** Investigate

Investigaterelated
related
court decisions
court decisions
* Testing and development * Follow related courtroom decisions
* No usage by general public * Develop "worst possible case" scenarios
Curiosity * Product selectively introduced * Identify and address overenthusiastic public

* Public fascination and excitement dialogue
* Media exposure increases product awareness * Communicate realistic product expectations to
* Product problems are addressed internally public
* Determine extent of possible damages or
injuries-physical, financial or psychological
* Develop formal standards and guidelines
Refinement
Refinement * More
* More
product product
versatility versatility
through expanding
through expanding
* Strictly define and follow product use
technology * Note and correct questions, problems
* New applications developed * Review/revise guidelines and standards
* Financial and social values perceived * Examine prospect for litigation
* Increased public awareness and demand * Avoid outlandish marketing and advertising
* Problems create suspicion of product * Monitor and investigate current litigation involving
* Internal problem solution taking place product
* Warranties, disclaimers, contracts, etc. are
issued
Applications * Public acceptance * Address specific product problem areas

Expanded * Intensive marketing with confidence and * Avoid litigation through internal problem resolution
expectations expanding * Remove a dysfunctional product from the
* Fascination minimizes legal activities marketplace and negotiate compensation
* Test cases establishing precedence * Emphasize and provide continuous training
* Widespread use of product makes internal
problem solution difficult
* Warranties, disclaimers, contracts, etc. are
refined
Problems * Fascination declines * Identify and observe other organizations for

* Public no longer accepts problems overzealous product exploitations
* Product has defects or is used injudiciously * Counter unrealistic public expectations
* Real or perceived injury occurs and aggressively disputes
arise * Investigate and determine public trends in product
* Problems give rise to argument andacceptance suspicion
* Dissatisfaction with internal solutions leads to * Misuse and dubious product development
litigation techniques identified, addressed/noted
Litigation * Disputes argued in court * Identify and note misuse of product

* Increased awareness by developer and user of * Aggressively address product anomalies
product responsibilities * Communicate formally to users and stress need
* Product re-evaluation to avoid future litigation for continuous instruction in use of product
* Emphasize possible dangers of product
* Investigate occurrences of injury and the court's
decision to determine trends
Legal Impact
Common Law * Court decisions guide further production * Continuously re-evaluate and update product
* Unwritten rules/limits defined development procedures, standards and
* Product removed from marketplace guidelines
* Determine risk categories of product
Statutory Law * Laws stemming from real and/or perceived * Identify legal activity in other companies and
injury determine dubious practices if litigation is
* Specific rules mandated taking place
* Product removed from marketplace * Study court interpretations of statutory laws
involving product
Legally * Constrained product marketing * Aggressive practice of all of the above

Responsible * Developers and public mindful of legal * Re-evaluate product legal position in the
Progress responsibilities marketplace
* Vigorous research of public trends with regard to
product
Figure 1. A Framework for Product Growth and Legal Impact
LEGALLY RESPONSIBLE PROGRESS would sible threats

threats should
should be
be identified
identifiedand
andcorrected
corrected
include the acceptance and understanding of along with
with the
the investigation
investigationof
ofany
anyoccurrences
occurrences
of injury
legal responsibilities by the developer and public.injury and
and court
court actions
actions as
asaaresult
resultofofthem.
them.
Further product development and marketing is
At LEGAL
LEGAL IMPACT,
IMPACT, standards,
standards,policies,
policies,and
andpro-
pro-
constrained, guided by common or statutory law. cedures should be modified as needed based on
In short, corporate management that adheres to common or statutory laws. As a result of all legal
the above set of descriptive actions has followed actions, have any risk categories been identified
a course of action based largely on reaction. that were not previously known?
Organizations that endeavor to maintain LEGAL-
Stages of development-normative LY RESPONSIBLE PROGRESS of the product
actions will have enthusiastically and aggressively fol-
lowed the product's growth with these legal con-
At CREATION, organizations should hasten to
research those court decisions that are related siderations foremost in mind. Concern for public
well-being may make the difference between a
in some way to the product under development.
commercially safe and acceptable product or one
Any precedent-setting decisions involving such
mired in a litigious quagmire leading to possible
cases could impact the development and testing
organizational ruin.
of the proposed product, including its design and
engineering. Assessment of "worst case" In short, management that follows the normative
scenarios should be conducted. actions outlined above has taken a proactive
course of action, aggressively attempting to iden-
When the product reaches the CURIOSITY
tify and correct anomalies before they lead to
stage, organizations must be conscious of public
damaging litigation.
fascination and possible overenthusiasm, espe-
cially that driven by technology. A sense of
realism with regard to the product's progress
should be of concern along with the examination Conclusions
of possible injuries that could occur. Such injuries
could be physical, financial, or even psychologi- In reviewing the literature from both the world
cal. Standards and guidelines should be initiated of business, information systems, and the law,
that are consistent with corporate goals and pub- more questions about ES were raised than re-
lic appeal for the product. solved. In view of these findings, what direction
should management take? Perhaps senior orga-
During the REFINEMENT stage, organizations
nizational management, IS management, and
should carefully monitor the product's use by the
other functional managers may view the resulting
public. Possible outlandish claims and over-
questions and uncertainties as a false blessing
zealous marketing practices should be avoided
in disguise, possibly a time for quick financial
because they may lead to litigation.
gain. They may unknowingly continue with ES
At the APPLICATIONS EXPANDED stage, any development and use, choose to ignore the cur-
specific problems with the product should be rent environment, and simply attribute their ac-
defined. Emphasis should be placed on internal tions to a world driven by technology. Or the
problem resolution with possible removal of a astute and responsible managers can become
dysfunctional product from the market. proactive in their investigation and research ef-
forts to minimize the legal ramifications rather
As PROBLEMS arise with the product, any
than place themselves in the unfortunate posi-
misuse should be curtailed. Such problems may
tion of reacting to public outcries or unfavorable
be related to the public's expectations and prod-
courtroom decisions. Perhaps corporations
uct acceptance as well as to other organizations
should utilize attorneys as investigators or fact-
that may be competing in the marketplace. The
finders. These lawyers would be responsible for
identification of trends dealing with the product's
researching related precedent-setting decisions
use and acceptance should be carefully
observed. in order to limit the legal ramifications before they
arise. The use of such "special counsels" for
Public misuse of the product may lead to LITIGA- possible illegality or other misconduct is dis-
TION. Is such misuse caused by anomalies? Pos- cussed at length by Anderson and Stier (1987).
Although
AlthoughAnderson
Andersonand
and
Stier
Stier
consider
consider
suchsuch
inin- Bachant,
Bachant, J.
J.and
andMcDermott,
McDermott, J. J.
"R1"R1
Revisited:
Revisited:
vestigations
vestigationsofofalleged
alleged
fraud
fraud
by by
a securities
a securities Four
Four Years
Yearsininthe
theTrenches,"
Trenches,"The
TheAl Al
Magazine
Magazine
dealer
dealer and
andsafety
safetyissues
issues
surrounding
surrounding the the
Three
Three (5:3),
(5:3), Fall
Fall 1984,
1984,pp.
pp.21-32.
21-32.
Mile
Mile Island
Islandnuclear
nuclearplant
plant
as as
appropriate,
appropriate,
there
there
is is Black,
Black, H.C.
H.C.Black's
Black'sLaw
LawDictionary,
Dictionary,
West
West
no
no reason
reasonforfororganizations
organizationsnotnot
to proactively
to proactively
inin- Publishing,
Publishing,St.St.Paul,
Paul,MN,
MN, 1979.
1979.
vestigate
vestigateES
ESininsuch
sucha way.
a way. Black,
Black, Jackson
Jacksonand
andSimmons
Simmons Ins.
Ins.
Brokerage
Brokeragev. v.
IBM
IBM Corp.
Corp.Northeastern
NortheasternReporter,
Reporter, 2nd2nd
Series
Series
In
In law,
law, words
wordsarearecarefully
carefullyselected
selected
andand
usedused
to to
(440),
(440), West
WestPublishing
PublishingCo.,
Co.,
St.St.
Paul,
Paul,
MN,MN,1982,
1982,
convey
convey thoughts
thoughtsasasaccurately
accurately as possible.
as possible.
The The
p. 282.
language
languagesurrounding
surrounding ES,ES,
however,
however, opens
opens
the the
Bobrow, D., Mittal, S. and Stefik, M. "Expert
door
door to
to speculation
speculationasastotoexactly
exactlyhow how
ESs ESs
are are
Systems: Peril and Promise," Communications
perceived
perceivedby bythe
thegeneral
generalpublic.
public.
This
This
perception
perception
could lead ESs and their users to the courtroom. of the ACM (29:9), September 1986, pp.
880-894.
As the knowledge of problems increases,
Buchanan, B.G. and Shortliffe, E.H. Rule-Based
dissatisfaction and suspicion of this type of pro-
Expert Systems, Addison-Wesley, Reading,
duct will keep the legal activity level high. The
MA, 1985.
law with respect to ES will evolve over a long
Business Week. "Computers That Think Like
period of time because of the advent of newer
Human Experts," July 6, 1981, pp. 50-51.
and more refined systems and the expanding
Computer Law Journal. "Strict Products Liabili-
prevalence of ESs and their varied applications.
ty and Computer Software: Caveat Vendor,"
Because ESs are in their infancy, we do not yet (4), 1984, p. 373.
have a clear definition of what they are and how Computers and Law. "Cautionary Notes on Legal
they are to be used. The framework we have pre- Expert Systems," No. 40, May 1984, pp.
sented could be useful in identifying the stage 14-16.
of growth and then using that knowledge to pro- Criterion Software, Inc. Stock Trading System,
ceed judiciously. Further, possible injury and the Version 2.0, Incline, NV, 1987.
resulting suspicion of the product could be mini- Dombroff, M. "Computers Have Big Impact on
mized or even eliminated. There is little doubt that Expert Testimony," Legal Times (7), May 20,
ESs are making inroads in organizations, with 1985, p. 19.
corporations striving to achieve the benefits of Ducker, B. "Liability for Computer Software,"
this technology. However, enthusiasm regarding Business Lawyer (21), 1971, p. 1081.
ES must be tempered with a realistic view of the Duda, R.O. and Reboh, R. "Al and Decision-
world we live in. If prudently developed and Making: The PROSPECTOR Experience," in
marketed wisely, expert systems could be ex- Artificial Intelligence Applications for Business,
citing tools to be used in a beneficial way. W. Reitman (ed.), Ablex, Norwood, NJ, 1984.
Engineering News-Record. "Hyatt Ruling Rocks
Engineers," (215:22), November 28, 1985,
Acknowledgement p. 13.
We are indebted to Donald W. Cantwell, J.D., EXSYS, Inc., EXSYS, Albuquerque, NM, 1985.
University of Texas at Arlington, for providing his
Freeman, P. "Expert Systems and the Law,"
invaluable assistance and legal acumen. We also Computers and Law (47), March 1986, pp.
9-12.
acknowledge the very constructive comments of
the anonymous referees and the associate editor. Harmon, P. and King, D. Expert Systems, John
Wiley & Sons, New York, NY, 1985.
High Tech Law Journal. "Medical Expert
References Systems: Grappling with Issues of Liability,"
(1), Fall 1986, pp. 483-520.
Anderson, H. and Stier, E. "What You Don't Hurley, M. and Wallace, W. "Expert Systems as
Know Can Hurt You: The Case for 'Special Decision Aids for Public Managers: An Assess-
Counsel Investigations'," California Manage- ment of the Technology and Prototyping as a
ment Review (XXIX:3), Spring 1987, pp. 77-98. Design Strategy," Public Administration
Averill, K. "Computers in the Courtroom: Using Review (46), November 1986, pp. 563-571.
Computer Diagnosis as Expert Opinion," Com- IEEE Expert. "Artificial Intelligence Goes to
puter Law Journal (5), Fall 1984, pp. 217-231. Court," (1:2), Summer 1986, p. 101.
Invacare
Invacare Corp.
Corp.v.v.Sperry
SperryCorp.
Corp.Federal
Federal
Supple-
Supple- Addison-Wesley
Addison-Wesley Publishing
Publishing
Company,
Company,
Read- Read-
ment, District Court Ohio (612), West ing, MA, 1986.
Publishing Co., St. Paul, MN, 1984, p. 448. Zeide, J. and Liebowitz, J. "Using Expert
Joyce, E. "Software Bugs: A Matter of Life and Systems: The Legal Perspective," IEEE Expert
Liability," Datamation (33:10), May 15, 1987, (2:1), Spring 1987, pp. 19-21.
pp. 6-14.
Konsynski, B.R. "Special Issue: Decision Sup-
port and Knowledge-Based Systems,"' Journal
of Management Information Systems (5:1),
Summer 1988, pp. 3-5.
Leonard-Barton, D. and Sviokla, J. "Putting Ex-
pert Systems to Work," Harvard Business
Review (67:2), March-April 1988, pp. 91-98. About the Authors
Luconi, F.L., Malone, T.W. and Scott Morton,
M.S. "Expert Systems: The Next Challenge for Kathleen Mykytyn is a Ph.D. student in the
Managers," Sloan Management Review (27:4), Department of Information System and Manage-
Summer 1986, pp. 3-13. ment Sciences at the University of Texas at Arl-
McDermott, J. "R1: A Rule-Based Configurer of ington. Her research interests are concerned with
Computer Systems," Artificial Intelligence the impact and use of information systems
(19:1), 1982, pp. 39-88. technology on organizations and end users.
Michigan Law Review. "Computer Programs as
Peter P. Mykytyn, Jr. is an assistant professor
Goods Under the U.C.C.," (77:4), April 1979,
pp. 1149-1165. in the Department of Information Systems and
Management Sciences, the University of Texas
Mishkoff, H. Understanding Artificial Intelligence,
Texas Instruments, Inc., Dallas, TX, 1986. at Arlington. He received his Ph.D. in computer
information systems from Arizona State Univer-
Nycum, S. and Fong, I. "Artificial Intelligence and
sity in 1985. He has published in Information &
Certain Resulting Legal Issues," The Com-
Management, Al & Society, and Journal of
puter Lawyer (2:5), May 1985, pp. 1-10.
Systems Management. His primary research in-
Pople, H. "Heuristic Methods for Imposing Struc-
ture on Ill-Structured Problems: The Structur- terests include information technology to support
decision making, human/computer interaction ef-
ing of Medical Diagnosis," in Artificial
fects in MIS/DSS design and use, the role and
Intelligence in Medicine, P. Szolovitz (ed.),
Westview Press, Boulder, Co, 1982. effect of expert systems in managerial decision-
making activities, and end-user computing
Quad County Distributing Co. v. Burroughs Corp.
issues. He is a member of the Decision Sciences
Northeastern Reporter, 2nd Series (385), West
Institute, The Institute of Management Science,
Publishing Co., St. Paul, MN, 1979, p. 1108.
and the Association for Systems Management.
San Diego Law Review. "Computer Software and
Strict'Products Liability," (20:2), March 1983, Craig W. Slinkman is an associate professor in
pp. 439-456. the Department of Information Systems and
Susskind, R. "Expert Systems in Law: A Juris- Management Sciences at the University of Texas
prudential Approach to Artificial Intelligence at Arlington. He received his Ph.D. degree in
and Legal Reasoning," Modern Law Review quantitative analysis from the University of Min-
(49), March 1986, pp. 168-194. nesota in 1984. He has published in Communica-
Turner, G. "How an Outside Expert Views Ex- tions in Statistics, The Financial Analysts Journal,
pert Systems," Journal of Commercial Bank and The Journal of High Technology Manage-
Lending (69:4), January 1987, pp. 8-9. ment. His primary research interests include con-
United States Welding v. Burroughs Corp. ceptual data modeling methodologies for
Federal Supplement (587), West Publishing database design, application design in the
Co., St. Paul, MN, 1984, p. 49. database environment, cooperative processing
Warsaw, J. "Don't Let a Good Lawsuit Slip application design methodologies, and clinical in-
Away," American Bar Association Journal formation systems. He is a member of the Asso-
(80:4), October 1, 1986, pp. 70-71. ciation for Systems Management, and the
Waterman, D. A Guide to Expert Systems, Decision Sciences Institute.
Appendix
Appendix
Selected
Selectedlegal
legalterms,
terms,definitions,
definitions,and,
and,
where
where
appropriate,
appropriate,
explanations
explanations
and examples
and examples
are included
are included
here here
for
for elucidation.
elucidation.The Thedefinitions
definitionsareare
taken
taken
fromfrom
Black's
Black's
Law Dictionary
Law Dictionary
(Black,(Black,
1979). Examples
1979). Examples
or ex- or ex-
planation
planationin inlayman's
layman's terms
termsareare
highlighted
highlighted
withwith
asterisks
asterisks
(*).1 (*).1
1.
1. Active
ActiveNegligence-A
Negligence-Aterm
term
of of
extensive
extensive
meaning
meaning
embracing
embracing
manymany
occurrences
occurrences
that would
thatfall
would
short
fall short
of
of willful
willfulwrongdoing,
wrongdoing, ororof of
crass
crass
negligence;
negligence;
for example,
for example,
all inadvertent
all inadvertent
acts causing
acts causing
injury to
injury
others,
to others,
resulting
resultingfrom
fromfailure
failureto to
exercise
exercise
ordinary
ordinarycare;care;
likewise,
likewise,
all acts
allthe
acts
effect
the effect
of whichof are
which
misjudged
are misjudged
or
or unforeseen,
unforeseen,through
through want
wantof of
proper
proper
attention,
attention,
or reflection,
or reflection,
and hence
and hence
the term
thecovers
term the
covers
acts the acts
of
of willful
willfulwrongdoing
wrongdoingand
and
also
also
those
those
which
which
are not
are of
notthat
of that
character.
character.
* If
If aa knowledge
knowledgeengineer
engineer
and/or
and/orexpert
expert
do not
do not
take take
appropriate
appropriate
measures
measures
to validate
to validate
the output
the output
produced
producedfromfromthetheknowledge
knowledgeand/or
and/orsearch
search
techniques
techniques
contained
contained
in an expert
in an expert
system,system,
especially
especially
where
where foreseeable
foreseeableeconomic
economic
or or
physical
physical
harmharm
can result
can result
fromfrom
the output,
the output,
they could
theybecould
held be
liable
held liable
for
for any
anyinjury
injuryresulting
resultingfrom
fromthethe
use use
of the
of the
system
system
underunder
the concept
the concept
of active
of negligence.
active negligence.
2.
2. Common
CommonLaw-In
Law-Ina broad
a broad
sense,
sense,common
commonlaw law
maymay
designate
designate
all that
allpart
thatofpart
the of
positive
the positive
law, juristic
law, juristic
theory,
theory,and
andancient
ancientcustom
custom of of
anyany
state
state
or nation
or nation
which
which
is of is
general
of general
and universal
and universal
application,
application,
thus
thus marking
markingoff
offspecial
special
or or
local
local
rules
rules
or customs.
or customs.
* The
The common
commonlaw lawis is
generally
generally
derived
derivedfrom
from
principles
principles
rather
rather
than enacted
than enacted
laws; itlaws;
does it
notdoes
consist
not consist
of
of absolute,
absolute,fixed,
fixed,and
and inflexible
inflexible
rules,
rules,
but but
rather
rather
of broad
of broad
and comprehensive
and comprehensive
principles
principles
based onbased on
justice,
justice, reason,
reason,and
andcommon
common sense.
sense.
TheseThese
principles
principles
are subject
are subject
to change
to change
as newas conditions
new conditions
in in
society
society arise.
arise.
3. Concurrent Negligence-Consists of the negligence of two or more persons concurring, not
necessarily in point of time, but in point of consequence, in producing a single indivisible injury.
* If it can be shown that a mdecial expert system used by a physician (one not involved in the develop-
ment of the system) to treat a patient results in injury, both the system developers and the physician
could possibly be held liable. Both the developers and the user are acting independently when a
third party is injured.
4. Contract-An agreement between two or more persons that creates an obligation to do or not to
do a particular thing. Its essentials are competent parties, subject matter, a legal consideration,
mutuality of agreement, and mutuality of obligation.
* Under contract theories, there are many types of contracts. An implied contract, as differentiated
from an expressed contract, is inferred, i.e., a tacit understanding that an obligation (contract) does
exist between persons. This kind of obligation rests on the principle that whatever a person should
do, the law will suppose that he or she will have promised to do it.
5. Disclaimer-The repudiation or renunciation of a claim or power vested in a person or which he
had formerly alleged to be his. The refusal, or rejection of an estate or right offered to a person.
The disavowal, denial, or renunciation of an interest, right, or property imputed to a person or al-
leged to be his. Also the declaration, or the instrument, by which such disclaimer is published.
* A disclaimer can be viewed as an appendix to a warranty. It deals with what may be "implied"
in a warranty in order to release a vendor from tort liability. Thus, it is a defensive measure used
generally with the purpose of protection from unwanted claims or liability.
6. Duty-Legal or moral obligation. Obligatory conduct or service. The word "duty" is used throughout
the Restatement of Torts to denote the fact that the actor is required to conduct himself in a par-
ticular matter at the risk that if he does not do so he becomes subject to liability to another to whom
the duty is owed for any injury sustained by such other, of which the actor's conduct is a legal cause.
These definitions are excerpted with permission from West Publishing Co., St. Paul, MN.
7. Expert-One
Expert-Onewhowhoisisknowledgeable
knowledgeablein in
a specialized
a specialized
field,
field,
thatthat
knowledge
knowledge
beingbeing
obtained
obtained
from from
either either
educational
educationalor
orpersonal
personalexperience.
experience.
* Experts
Experts do
donot
nothave
havetotocite
cite
authority;
authority;
they
they
are are
considered
considered
the the
authority.
authority.
8. Foreseeability-The
Foreseeability-Theability
ability
toto
seesee
or or
know
knowin advance;
in advance;
hence,
hence,
the reasonable
the reasonable
anticipation
anticipation
that harm
that harm
or injury
injury isisaalikely
likelyresult
resultofofacts
actsor or
omissions.
omissions.
9. Injury-Any
Injury-Anywrong
wrongofofdamage
damagedone
done
to to
another,
another,
either
either
in his
in person,
his person,
rights,
rights,
reputation,
reputation,
or property.
or property.
It is
is the
the invasion
invasionofofany
anylegally
legally
protected
protected
interest
interest
of another.
of another.
10.
10. Negligence-The
Negligence-Theomission
omissiontoto
dodo
something
something
which
which
a reasonable
a reasonable
man,man,
guided
guided
by those
by those
ordinary
ordinary
considerations
considerationswhich
whichordinarily
ordinarilyregulate
regulatehuman
humanaffairs,
affairs,
would
would
do, or
do,the
or doing
the doing
of something
of something
which which
a reasonable
reasonableand
andprudent
prudentmanmanwould
would notnot
do.do.
TheThe
lawlaw
of negligence
of negligence
is founded
is founded
on reasonable
on reasonable
con- con-
duct
duct or
or reasonable
reasonablecare
careunder
underallall
circumstances
circumstances of aofparticular
a particular
case.case.
Doctrine
Doctrine
of negligence
of negligence
rests rests
on
on the
the duty
dutyofofevery
everyperson
person
totoexercise
exercise
duedue
carecare
in his
in his
conduct
conduct
toward
toward
othersothers
fromfrom
whichwhich
injuryinjury
may result.
* When dealing with others, reasonable conduct and care must be observed. Additionally, in to-
day's society, the general public depends on the special expertise of persons who engage in specializ-
ed activities, such as knowledge engineering. Anyone who undertakes to perform them, even without
compensation, is judged by the standards of conduct of those properly qualified to do so, regardless
of whether the person is actually a recognized member of the group. Thus, reasonable conduct
and care will also be measured against any existing standards of conduct.
11. Passive Negligence-Failure to do something that should have been done. It is also referred to
as nonfeasance.
12. Privity-Concept of "privity" pertains to the relationship between a party to a suit and a
who was not a party, but whose interests in the action was such that he will be bound by th
judgment as if he were a party.
* Privity implies a private relationship where the person you relied upon is a person on wh
are entitled to rely, i.e., doctor-patient, home purchaser-real estate agent, pharmacist-patron, f
advisor-investor.
13. Product Liability-Refers to the legal liability of manufacturers and sellers to compensate buyers,
users, and even bystanders for damages or injuries suffered because of defects in goods purchased.
It is a tort which makes a manufacturer liable if his product has a defective condition that makes
it unreasonably dangerous to the user or consumer. Although the ultimate responsibility for injury
or damage in a products liability case most frequently rests with the manufacturer, liability may also
be imposed upon a retailer, occasionally upon a wholesaler or a middleman, and infrequently upon
a party wholly outside the manufacturing and distributing process, such as a certifier.
14. Reliance-In tort for deceit, it is necessary for plaintiff to prove that he relied on misrepresentation
though such misrepresentation need not be the sole or even dominant reason for acting if it was
a substantial factor in the plaintiff's decision.
15. Respondeat Superior-Let the master answer. This maxim means that a master is liable in certain
cases for the wrongful acts of his servant, and a principal for those of his agent. Under this doctrine
the master is responsible for want of care on the servant's part toward those to whom master owes
duty to use care, provided failure of servant to use such care occurred in the course of his employment.
* An employer can be held liable for the acts of an employee.

16. Statutory Law-That body of law created by acts of the legislature in contrast to law generated
by judicial opinions and administrative bodies.
17. Strict Liability-A concept applied by the courts in product liability cases in which a seller is liable
for any and all defective or hazardous products which unduly threaten a consumer's personal safe-
ty. This concept applies to all members involved in the manufacturing and selling of any facet of
the
the product.
product.The The
concept
conceptof strict
of strict
liability
liability
in tort
in istort
founded
is founded
on the premise
on the premise
that whenthat
a manufac-
when a manufac-
turer
turer presents
presentshishis
goods
goods
to the
to the
public
public
for sale,
for he
sale,
represents
he represents
that they
that
arethey
suitable
are for
suitable
their intended
for their intended
use,
use, and
andtotoinvoke
invokesuch
such
doctrine
doctrine
it isit
essential
is essential
to provde
to provde
that the
that
product
the product
was defective
was defective
when placedwhen placed
in the stream of commerce.
* The concept of strict liability has been emerging since the 1950s in order to establish a chain
of responsibility for a product. There is no responsibility to show fault; simply put, the courts believe
that if the product causes injury, no matter what care has been taken in its development, someone
must be held accountable for the injury.
18. Tort-A legal wrong committed upon the person or property independent of contract. It may be
either (1) a direct invasion of some legal right of the individual; (2) the infraction of some public
duty by which special damage accrues to the individual; (3) the violation of some private obligation
by which like damage accrues to the individual.
* A tort is simply a wrongful injury to a person, a person's reputation, or a person's property.
19. Uniform Commercial Code (U.C.C.)-One of the Uniform Laws drafted by the National Conference
of Commissioners on Uniform State Laws governing commercial transactions (sales or goods, com-
mercial paper, bank deposits and collections, letters of credit, bulk transfers, warehouse receipts,
bill of lading, investment securities, and secured transactions). The U.C.C. has been adopted by
all states, except Louisiana.
* U.C.C. is the codification of common law decisions throughout the United States, i.e., a "can-
ning" of common law. It is substantially the same in all of the states with only minor variations and
alterations depending on the laws and practices of a state.
20. Warranty-A promise that a proposition of fact is true. A promise that certain facts are truly as they
are represented to be and that they will remain so, subject to specified limitations. In certain cir-
cumstances a warranty will be presumed, known as an "implied" warranty.
* If a person purchases a new car, along with specified (expressed) warranties, it is also expected
(implied) that it will run.

Management Information Systems Research Center, University of Minnesota MIS Quarterly

Enviado por

Dados do documento

Descrição original:

Título original

Direitos autorais

Formatos disponíveis

Compartilhar este documento

Compartilhar ou incorporar documento

Opções de compartilhamento

Você considera este documento útil?

Este conteúdo é inapropriado?

Direitos autorais:

Formatos disponíveis

Management Information Systems Research Center, University of Minnesota MIS Quarterly

Enviado por

Direitos autorais:

Formatos disponíveis

Expert Systems: A Question of Liability?

Author(s): Kathleen Mykytyn, Peter P. Mykytyn, Jr. and Craig W. Slinkman

Management Information Systems Research Center, University of Minnesota is

MIS Quarterly/March 1990 27

The principal components of most expert sys-

28 MIS Quarterly/March 1990

MIS Quarterly/March 1990 29

dealing with a decision and may have difficulty minating

30 MIS Quarterly/March 1990

MIS Quarterly/March 1990 31

32 MIS Quarterly/March 1990

MIS Quarterly/March 1990 33

34 MIS Quarterly/March 1990

Growth and Legal Impact At REFINEMENT the product continues to

MIS Quarterly/March 1990 35

Creation * Creation of new product ** Investigate

Curiosity * Product selectively introduced * Identify and address overenthusiastic public

Applications * Public acceptance * Address specific product problem areas

Problems * Fascination declines * Identify and observe other organizations for

Litigation * Disputes argued in court * Identify and note misuse of product

Legally * Constrained product marketing * Aggressive practice of all of the above

Figure 1. A Framework for Product Growth and Legal Impact

36 MIS Quarterly/March 1990

LEGALLY RESPONSIBLE PROGRESS would sible threats

MIS Quarterly/March 1990 37

38 MIS Quarterly/March 1990

MIS Quarterly/March 1990 39

40 MIS Quarterly/March 1990

* An employer can be held liable for the acts of an employee.

MIS Quarterly/March 1990 41

42 MIS Quarterly/March 1990

Você também pode gostar