Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!mnetor!seismo!rutgers!ames!cit-vax!mangler
From: mangler@cit-vax.Caltech.Edu (System Mangler)
Newsgroups: comp.unix.questions
Subject: Re: Asynchronous output
Message-ID: <1389@cit-vax.Caltech.Edu>
Date: Sat, 20-Dec-86 13:33:29 EST
Article-I.D.: cit-vax.1389
Posted: Sat Dec 20 13:33:29 1986
Date-Received: Sat, 20-Dec-86 22:00:18 EST
References: <1858@batcomputer.tn.cornell.edu> <1538@ulysses.homer.nj.att.com>
Organization: California Institute of Technology
Lines: 18
Summary: can you parallelize your data crunching too?

In article <1858@batcomputer.tn.cornell.edu>, garry@batcomputer.UUCP writes:
> I have an application which entails computing and churning out vast
> quantities of data and, for speed, I'd like to have the I/O happening
> in parallel with the computing.

Many computations go through a "read the data, crunch it, output it"
cycle in which the crunching of one block is independent of the
crunching of another.  If that's your case, use three processes.
At any given time, one is reading, another is crunching what it
just read, and one is writing.	You'll get a fairly continuous
flow of data, won't have to pass around large volumes of data
(i.e. low overhead), and get a large share of the CPU.

One curious gotcha is that throughput will be substantially worse
if you run it "nice --20".   Explain that, BSD scheduler wizards!
(I noticed this on 4.3bsd dump, which works in precisely this fashion).

Don Speck   speck@vlsi.caltech.edu  {seismo,rutgers,ames}!cit-vax!speck