Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 (Tek) 9/28/84 based on 9/17/84; site tekchips.UUCP Path: utzoo!watmath!clyde!cbosgd!ihnp4!houxm!mtuxo!mtunh!vax135!cornell!uw-beaver!tektronix!tekcrl!tekchips!stevev From: stevev@tekchips.UUCP (Steve Vegdahl) Newsgroups: net.lang Subject: Re: Efficiency of Languages (and complexity) Message-ID: <345@tekchips.UUCP> Date: Wed, 6-Nov-85 13:37:44 EST Article-I.D.: tekchips.345 Posted: Wed Nov 6 13:37:44 1985 Date-Received: Fri, 8-Nov-85 21:57:56 EST References: <15100004@ada-uts.UUCP> <15100007@ada-uts.UUCP> <189@opus.UUCP> <12860@rochester.UUCP> <196@opus.UUCP> Organization: Tektronix, Beaverton OR Lines: 105 This message is in response to a number of articles regarding the above- named subject. My intent is to eliminate some of the misunderstanding and miscommumincation that has been taking place. (And probably replace it with new misunderstanding ... :-)) First, the O(n log n) result for sorting applies when 1. The architecture has a single processor with a "large" amount of memory 2. Sorting is done using comparisons and 3. Probablistic algorithms are not used 1--> Arguments crying "foul, you can't use an infinite amount of hardware", do not hold water. The point of the original posting (as I read it anyway) was that a problem's complexity depends on the architecture. Comparison sorting is O(n^2) on a Turing Machine, and is O(n log n) on a von Neumann architecture. It is perfectly reasonable to assume an architecture in which each memory element has a dedicated processor. The O-notation allows one to assume that there is an infinite amount of memory (or at least an amount of memory that is proportional to problem size). Thus, on a processor-per-memory-element architecture, the assumption of n-processors for a problem of size n is not only reasonable, but manditory. 2--> There are, of course, sorting algorithms such as radix-sort (see Knuth, Vol. 3), that do not use comparisons. 3--> It has been shown (e.g., Bruce Weide's thesis, Carnegie-Mellon U. (1977?)), that probablistic techniques can be used to sort in linear time. (This must be qualified by defining what "worst-case" means for a probablistic algorithm, but I'll leave that as an exercise ...) With that as a preface, I will now respond to some of the statements that have been made regarding this subject. >NO! You cannot throw "n different processors" at the array! N is >(potentially) larger than the number of processors you have. Actually, >there is an assumption in analyzing algorithms that one does not have an >infinite number of computational elements (whatever they may be). If you >throw out that assumption, you're nowhere--because (1) you can't build, or >even emulate, the hardware implied and mostly (2) all the algorithms >you're going to find interesting will take constant time! (If you have >unlimited hardware, you just keep replicating and working in parallel.) Responding to (1): certain computation models allow an infinite number of processors. Even disallowing these as impractical, assumming N processors is no more impractical than assuming N memory elements. Responding to (2): there are professors that make careers studying the complexity of algorithms assuming an infinite number of processors. There are lots of interesting algorithms do not work in constant time even with that assumption. >OK, now I am SURE that the folks who are responding to this don't >understand O() notation. This notation is explicitly designed to express >the ASYMPTOTIC COMPLEXITY of an algorithm. If that's not what you mean, >use some other notation. The usefulness of this sort of complexity measure >is that it tells how the required processing time (or space) increases in >relation to the "size" of the problem being solved. Again, O() notation is >an asymptotic measure. This makes it quite explicit that you cannot have >an "...amount of hardware...proportional to size of the problem..." unless >you consider an infinite amount of hardware. > >The "order-of" notation is not the only measure that can be applied to an >algorithm--but it DOES have a specific meaning and a specific use, and it's >a very basic part of analysis of algorithms. Try to understand it before >you use it. The O-notation makes no such assumption. In fact, it assumes just the opposite: that is enough memory (i.e., hardware) to hold the problem. The important thing is that the be O-notation used with respect to a computation model, and that the everyone agree (for a given discussion) on what the model is. The writer of the above quote seems to insist that the von Neumann model (or something very close to it) is the only model of computation that can be used. Allowing the number of processors to increase in proportion to the number of memory elements simply trades space for time. It is still appropriate to use the O-notation. >If I remember right, even though the time complexity of sort is O(log n) >on a single processor, the median of a set can be found in O(n). >It is hard to imagine how this might be done, but... The median of an array of elements (or more generally, the kth-smallest element) can definitely be found in linear time. See Aho, Hopcroft and Ullman (1974), page 97. >There is inherent sequencing in many >problems. For example, it is extremely difficult to sort in O(log n) >time using n processors. Noone knows (and indeed I think it is >impossible) to sort in O(1) time with n log n processors. If pipelining is allowed, sorting can be done in "constant time" on a network of size O(n * log^2(n)) (see Knuth, vol. 3). By this I mean that you present n elements to the network at each time unit, and you get n sorted elements out of the network at each time unit, with a latency of log^2(n). I seem to remember reading recently that someone discovered an O(n * log(n)) sorting network that would do the same thing. Ullman, maybe? Steve Vegdahl Computer Research Lab. Tektronix, Inc. Beaverton, Oregon