Xref: utzoo comp.arch:6076 comp.lang.prolog:1188 Path: utzoo!utgpu!attcan!uunet!lll-winken!lll-tis!helios.ee.lbl.gov!pasteur!agate!eos!eugene From: eugene@eos.UUCP (Eugene Miya) Newsgroups: comp.arch,comp.lang.prolog Subject: Re: Perils of comparison -- an example Message-ID: <1303@eos.UUCP> Date: 15 Aug 88 18:55:25 GMT References: <282@quintus.UUCP> <15221@shemp.CS.UCLA.EDU> <292@quintus.UUCP> Reply-To: eugene@eos.UUCP (Eugene Miya) Organization: NASA Ames Research Center, Calif. Lines: 35 In article <292@quintus.UUCP> ok@quintus.UUCP (Richard A. O'Keefe) writes: >Don't get me wrong: Naive Reverse is not a specially good benchmark. I see you came from prolog and cross posted to arch. >There is a single specific benchmark, called naive reverse, which happens >to do 496 procedure calls. To determine the kLI/s rating, you run this >benchmark N times, for some large N. If it takes T seconds, you report >(496*N)/T as the LIPS rating. I've stated this many times in comp.arch, and I'll repeat this once for the prolog community benefit. Measurement of repetition isn't equivalent to repetition of measurement on a computer. Cache, paging, and optimization conspire against oversimplistic measurements of this type. >When you are benchmarking, it is necessary to be precise You said it all. I've been trying to find out what "really constitutes a Logical Instruction" As far as I can tell, it's totally arbitrary whereas Instructions and Operations tend to correspond to discrete states (barring instruction pipelining, yes yes....). (Yes I have Gabriel's thesis and others). Your keyword about measuring prolog is "naive." This isn't a putdown, but the prolog community will have to recognize some of these problems. Another gross generalization from --eugene miya, NASA Ames Research Center, eugene@aurora.arc.nasa.gov resident cynic at the Rock of Ages Home for Retired Hackers: "Mailers?! HA!", "If my mail does not reach you, please accept my apology." {uunet,hplabs,ncar,decwrl,allegra,tektronix}!ames!aurora!eugene "Send mail, avoid follow-ups. If enough, I'll summarize."