Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 9/18/84; site utcsrgv.UUCP Path: utzoo!utcsrgv!eas From: eas@utcsrgv.UUCP (Ann Struthers) Newsgroups: ont.events Subject: AI SEMINAR Message-ID: <632@utcsrgv.UUCP> Date: Mon, 7-Jan-85 10:57:55 EST Article-I.D.: utcsrgv.632 Posted: Mon Jan 7 10:57:55 1985 Date-Received: Mon, 7-Jan-85 11:44:26 EST Distribution: ont Organization: CSRI, University of Toronto Lines: 39 ARTIFICIAL INTELLIGENCE SEMINAR Tuesday, January 8, 1985 3:00 P.M. Sandford Fleming Building 1105 Dr. Jack Edwards Longwoods Research Group Ltd., Toronto "Evaluating the Intelligence in Dialogue Systems" The talk presents a set of criteria for evaluating intelligence in AI computer systems which engage in dialogue with human users. It is divided in five sections as follows: (1) A definition of dialogue, characterization of a dialogue system, and brief discussion of the concept of intelligence in dialogue systems. (2) An elaboration of eight aspects of dialogue and dialogue systems. (3) A discussion of four evaluative dimensions and an illustration of how they may be combined with the aspects above to generate criteria for designing and evaluating dialogue systems. (4) Applications of some of these criteria to three example dialogue systems: ELIZA, GUS, and GUIDON. (5) A discussion of the current state of evaluation and our evaluative methodology, and directions for future work.