Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Posting-Version: version B 2.10.2 9/18/84; site loral.UUCP Path: utzoo!watmath!clyde!burl!ulysses!allegra!bellcore!decvax!ittvax!dcdwest!sdcsvax!sdcc3!sdcc6!loral!ian From: ian@loral.UUCP (Ian Kaplan) Newsgroups: net.arch Subject: Cube designs Message-ID: <798@loral.UUCP> Date: Wed, 27-Feb-85 22:42:52 EST Article-I.D.: loral.798 Posted: Wed Feb 27 22:42:52 1985 Date-Received: Sat, 2-Mar-85 04:05:12 EST References: <7306@watrose.UUCP> Reply-To: ian@loral.UUCP (Ian Kaplan) Organization: Loral Instrumentation, San Diego Lines: 94 Keywords: Cosmic cube, parallel processing Summary: Networks are not as simple as some suggest. In article <7306@watrose.UUCP> cdshaw@watrose.UUCP (Chris Shaw) writes: > >The average communication length is 3, with no bus contention along the way. >The contention is really hidden in the point-to-point waiting for receive and >transmit. > I think that things are a little more complex than this. Contention is not entirely nonexistent. Two processors can contend for the bus which connects them. This is probably a minor issue, but it could be a problem in applications with heavy nearest neighbor communication. Other Problems with Networked Parallel Processors Before you start to plan your network connected parallel processor which will threaten Cray's dominance of the super-computer market here are a few things to consider. 1. Network routing One problem which has not been addressed in the discussion of the cube architectures is the complexity of managing communication in the cube network. I have not seem a description of the devices which handle the interprocessor busses, but I assume that they simply handle access and contention. Once a message arrives, it must be routed to another I/O channel if it is addressed to that processor. This seems to require processor intervention. After all, routing tables must be examined to determine the port through which the message should be sent. This processing will introduce some lag at each stop in the network. If a separate network processor is not provided to handle network routing, it must be done by the CPU which is executing the application code. This will be overhead that will slow down application execution. Another area where lag can be introduced is when several processors route their messages through the same intermediate processor. Presumably the network router can not handle them all at once and they will have to be queued. 2. Broadcasting Broadcasting in a network system is difficult. There must be a special message header for messages which are broadcast. When a processor gets this message it broadcasts it to all of the processors it is connected to. All cube processors are connected to an ethernet, but I wonder if multicasting is supported. Even if multicasting is supported, this divides communication between two media, one where multicasting is supported and one where it is not. This adds still more complexity to the system. 4. Software People have been building parallel processors for some time and it is now widely realized that the hard part is not constructing the machine, but programming it. Below are listed several areas which must be addressed in the software for a network system. A. Associating Logical Communication Channels with Physical Communication Channels. I have not yet received anything from iNTEL on the software for the iNTEL cube, so I am still uncertain about how they program it. According to the ACM article on the Cosmic Cube, Cal. Tech. uses communicating parallel processes. I assume that iNTEL does the same. When process A communicates with process B, it does so via a logical communication channel. No processor address is specified. When the program is loaded onto the cube there must be a mapping between the logical communication paths and the physical processor address. B. Programming Parallel Processes is Difficult A number of people have commented that programming applications composed of parallel processes is more like writing operating systems than applications as we usually regard them. A particularly good discussion of the problems encounted in programs composed of parallel processes is provided in [1]. In the Cosmic Cube article no example was given of an application which had heavy interprocess communication. By not discussing this, the author side stepped an important issue. If the iNTEL cube follows the Cal. Tech. model I think that it will be very hard to program. [1] "Exploiting Multiprocessors: Issues and Options" by James McGraw and Timothy S. Axelrod, Preprint - Lawrence Livermore National Labs. UCRL-91734 Available from - Technical Information Dept. LLNL, Livermore, CA 94550 Ian Kaplan Loral Instrumentation (619) 560-5888 x4812 USENET: {decvax, ucbvax,ihnp4}!sdcsvax!sdcc6!loral!ian ARPA: sdcc6!loral!ian@UCSD USPS: 8401 Aero Dr. San Diego, CA 92123