Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP Path: utzoo!mnetor!seismo!rutgers!ames!cit-vax!mangler From: mangler@cit-vax.Caltech.Edu (System Mangler) Newsgroups: comp.unix.questions Subject: Re: Asynchronous output Message-ID: <1389@cit-vax.Caltech.Edu> Date: Sat, 20-Dec-86 13:33:29 EST Article-I.D.: cit-vax.1389 Posted: Sat Dec 20 13:33:29 1986 Date-Received: Sat, 20-Dec-86 22:00:18 EST References: <1858@batcomputer.tn.cornell.edu> <1538@ulysses.homer.nj.att.com> Organization: California Institute of Technology Lines: 18 Summary: can you parallelize your data crunching too? In article <1858@batcomputer.tn.cornell.edu>, garry@batcomputer.UUCP writes: > I have an application which entails computing and churning out vast > quantities of data and, for speed, I'd like to have the I/O happening > in parallel with the computing. Many computations go through a "read the data, crunch it, output it" cycle in which the crunching of one block is independent of the crunching of another. If that's your case, use three processes. At any given time, one is reading, another is crunching what it just read, and one is writing. You'll get a fairly continuous flow of data, won't have to pass around large volumes of data (i.e. low overhead), and get a large share of the CPU. One curious gotcha is that throughput will be substantially worse if you run it "nice --20". Explain that, BSD scheduler wizards! (I noticed this on 4.3bsd dump, which works in precisely this fashion). Don Speck speck@vlsi.caltech.edu {seismo,rutgers,ames}!cit-vax!speck