Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.1 6/24/83; site utah-cs.UUCP Path: utzoo!watmath!clyde!burl!ulysses!unc!mcnc!decvax!decwrl!greipa!pesnta!amd!amdcad!lll-crg!seismo!utah-cs!shebs From: shebs@utah-cs.UUCP (Stanley Shebs) Newsgroups: net.ai Subject: Re: Any data on programmer productivity Message-ID: <3511@utah-cs.UUCP> Date: Wed, 30-Oct-85 18:08:29 EST Article-I.D.: utah-cs.3511 Posted: Wed Oct 30 18:08:29 1985 Date-Received: Sun, 3-Nov-85 05:30:39 EST References: <526@ihwpt.UUCP> <3500007@uiucdcsp> Reply-To: shebs@utah-cs.UUCP (Stanley shebs) Organization: Univ of Utah CS Dept Lines: 42 In article <3500007@uiucdcsp> forbus@uiucdcsp.CS.UIUC.EDU writes: >Your best source for raw performance data is > "Performance and Evaluation of Lisp Systems" > by Richard P. Gabriel, $25 from MIT press > >This is the book form of Gabriel's DARPA-funded study. One of the amazing >regularities is that performance of specialized lisp machines is vastly >better than any mainframe run as a STAND-ALONE machine, much less >time-shared. The only exception is running a Cray stand-alone; no >measurements are available for a time-shared Cray. Are we looking at the same study? The numbers on my crummy old xerox copies of the results don't show that at all. Admittedly they are incomplete compared to the book (which I've not yet purchased a copy of), but I don't believe the numbers have changed any. On the average, the 3600 does the best of any lisp machine. The 3600 gets 12.0 secs for the boyer benchmark, while PSL on a DEC-20 (a mainframe presumably) gets 11.96 secs. Not "vast" in my book. PSL on a 68K (HP200) gets 25.52 secs, which is half as fast as the 3600. I suppose that might be considered "vast", but then the 68K *is* a microprocessor... For other benchmarks, the 3600 (and the other lisp machines) fare worse. Running "browse", the 3600 gets 32.1 secs against the HP200's 39.84; a 20 running PSL gets 23.81 secs. Running "puzzle", the 3600 gets 14.21 secs, and the HP200 zips along at 10.85 secs! That's the best result; the worst puts the 68K PSL about 6 times slower. The mainframes range all over from substantially better to substantially worse. My reading overall is that the 3600 is on the average comparable to a VAX 780 running PSL, and faster than a 68K Unix box, but not impressively so. The true moral of the study is that no Lisp can be judged "best" or even "fastest" on the basis of benchmarks; each implementation has its strengths and weaknesses. In fact, the proper audience of this book is the implementors themselves; we now have a basis for deciding what aspects of our Lisps need work, and which implementations we ought to look at to improve those aspects. stan shebs