Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!uunet!husc6!rutgers!sri-spam!ames!elroy!mahendo!jplgodo!wlbr!scgvaxd!trwrb!aero!venera.isi.edu!smoliar From: smoliar@vaxa.isi.edu (Stephen Smoliar) Newsgroups: comp.ai Subject: Re: Can you sue an expert system? Message-ID: <4260@venera.isi.edu> Date: Mon, 7-Dec-87 11:54:08 EST Article-I.D.: venera.4260 Posted: Mon Dec 7 11:54:08 1987 Date-Received: Sun, 13-Dec-87 12:05:55 EST References: <1788@cup.portal.com> Sender: daemon@venera.isi.edu Reply-To: smoliar@vaxa.isi.edu.UUCP (Stephen Smoliar) Organization: Information Sciences Institute Lines: 51 In article <1788@cup.portal.com> Barry_A_Stevens@cup.portal.com writes: > >Consider, and please comment on, this scenario. > > * * * * * * * * * * * > >A well-respected, well-established expert systems(ES) company constructs >an expert financial advisory system. The firm employs the top ES >applications specialists in the country. The system is constructed with >help from the top domain experts in the financial services industry. It >is exhaustively tested, including verification of rules, verification of >reasoning, and further analyses to establish the system's overall value. >All results are excellent, and the system is offered for sale. > Anyone who is willing to accept these premises at face value may be more interested in investing in the bridge I have between Manhattan and Brooklyn than in expert systems. The sort of "ideal" product envisaged here is certainly beyond the grasp of current development technology and may remain so for quite some time. The most important omission from this scenario is the assumption that any sort of disclaimer has been attached to the product. I have encountered a variety of advertisements for human financial consultants; and, as a rule, there is always some disclaimer about risk present. The idea that their would be a machine-based product which would be risk-free borders on ludicrous. If a customer was hooked by such a claim, most likely the only place he would be able to complain would be to the Better Business Bureau. > >By now, you know the outcome. On the Friday morning before Black Monday, >the expert system tells Joe to "sell everything he has and go into the >stock market." ESs can usually explain their actions, and Joe asks for >an explanation. The ES replies "because ... it's only been going UP for >the past five years and there are NO PROBLEMS IN SIGHT." > Would Joe have accepted such an explanation from a human advisor? If so, he has gotten what he deserved. (I happened to be discussing an analogous case with my lawyer-neighbor. Our scenario involved medical systems and malpractice, but the theme is basically the same.) This raises another question: Assuming Joe is no dummy (and that he can afford good human advice), why would he be intersted in an machine advisor? I would argue that the area in which machines tend to have it over humans is that of quantitative risk assessment. Thus, the machine is more likely to synthesize and justify concrete quantitative predictive models than is a human expert, whose skills are fundamentally qualitative. Thus, the best Joe could hope for would be such a model. INTERPRETING the model would remain his responsibility (although that interpretation may be linked to the machines justification of the model, itself). I would conclude that this scenario is far too simplistic for the real world. I suggest that Mr. Stevens debug it a bit. Then we might be able to have a more realistic debate on the matter.