Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!uunet!portal!cup.portal.com!Barry_A_Stevens
From: Barry_A_Stevens@cup.portal.com
Newsgroups: comp.ai
Subject: Can you sue an expert system?
Message-ID: <1788@cup.portal.com>
Date: Fri, 4-Dec-87 00:05:00 EST
Article-I.D.: cup.1788
Posted: Fri Dec  4 00:05:00 1987
Date-Received: Wed, 9-Dec-87 02:45:48 EST
Organization: The Portal System (TM)
Lines: 73
XPortal-User-Id: 1.1001.1587


I am interested in the legal aspects of using expert systems.

Consider, and please comment on, this scenario.

                     * * * * * * * * * * *

A well-respected, well-established expert systems(ES) company constructs
an expert financial advisory system. The firm employs the top ES
applications specialists in the country. The system is constructed with
help from the top domain experts in the financial services industry. It
is exhaustively tested, including verification of rules, verification of
reasoning, and further analyses to establish the system's overall value.
All results are excellent, and the system is offered for sale.

Joe Smith is looking for a financial advisory system. He reads the sales
literature, which lists names of experts whose advice was used when
building the system. It lists the credentials of the people in the
company who were the implementors. It lists names of satisfied users,
and quotes comments that praise the product. Joe wavers, weakens, and
buys the product.
 
"The product IS good,", Joe explains. "I got it up and running in less
than an hour!" Joe spends the remainder of that evening entering his own
personal financial data, answering questions asked by the ES, and
anticipating the results.
 
By now, you know the outcome. On the Friday morning before Black Monday,
the expert system tells Joe to "sell everything he has and go into the
stock market." ESs can usually explain their actions, and Joe asks for
an explanation. The ES replies "because ... it's only been going UP for
the past five years and there are NO PROBLEMS IN SIGHT."
 
Joe loses big on Monday. Since he lives in California, (where there is
one lawyer for every four households, or so it seems, and a motion
asking that a lawsuit be declared frivolous is itself declared
frivolous) he is going to sue someone. But who?
 
     The company that implemented the system?
 
     The domain experts that built their advice into the system?
 
     The knowledge engineers who turned expertise into a system?
 
     The distributor who sold an obviously defective product?
 
Will a warranty protect the parties involved? Probably not. If real
damages are involved, people will file lawsuits anyway.
 
Can the domain experts hide behind the company? Probably not. The
company will specifically want to use their names and reputations as the
source of credibility for the product. The user's reaction could be,
"There's the so-and-so who told me to go into the stock market."
 
Can the knowledge engineers be sued for faulty construction of a system?
Why not, when people who build anything else badly can be sued?
 
How about the distributor -- after all, he ultimately took money from
the customer and gave him the product.
 
                     * * * * * * * * * * *
 
I would be very interested in any of your thoughts on this subject. I'd
be happy to summarize the responses to the net.
 
Barry A. Stevens
Applied AI Systems, Inc.
PO Box 2747
Del Mar, CA 92014
619-755-7231