From: utzoo!decvax!vortex!lauren
Newsgroups: net.news
Title: increasing mail/netnews efficiency
Article-I.D.: vortex.32
Posted: Sat Mar 12 09:19:02 1983
Received: Fri Mar 18 02:31:31 1983

I strongly suspect that the best place to try improve uucp "efficiency"
is in the "higher-level" file handling routines -- I recommend
AGAINST trying to improve or replace the packet driver.  Several points:

1) The packet driver, with its non-trivial buffering techniques, manages
   to allow fairly high rates of data transfer, even on multiple lines,
   without dragging most systems into the ground.  Most other protocols
   with which I've experimented have turned out to have substantially
   lower throughput and presented a greater load (sometimes much greater)
   on the system.

   The packet driver is not the problem -- it's a nice piece of work
   and performs very nicely.

2) Occasionally we hear complaints about uucp not really using the full
   duplex capabilities of the channel.  Frankly, I doubt very much
   if true "full duplex" file transfers would really improve the
   situation.  Most heavy uucp traffic "tends" to be mostly in one direction 
   (especially netnews traffic!) -- the number of cases where both
   sites have even approximately equal traffic *at a given time* are
   pretty small.  Given the possibility of load disruptions and other
   factors, plus the "unidirectional" nature of most traffic, leads me
   to suspect that an effort into more full duplex usage of uucp would
   also NOT be the best way to proceed.

3) The queue.  The place where we see most of the "clogging" clearly
   seems to be the uucp spool directory itself.  The mechanism of
   having three files for each message to be mailed provides good
   generality (i.e. mail is but a special case of a very general
   inter-machine mechanism) but is costly in time and space.  However,
   we don't really want to toss out the generality either!  My suggestion
   would be to define a new "channel" for mail/netnews which still uses
   the conventional packet driver.  At first, since there would be
   some compatibility issues, I would propose that the new channel
   only be used for netnews between cooperating sites.  As more sites
   started running the appropriate versions of uucico/uuxqt, the channel
   could be used for mail as well.  The old channel would still exist
   as a fallback in all cases.

   The easiest way to set this up would be to define the new channel
   as an "alternative" to the normal "g" channel that we now use.
   Perhaps calling this a new channel is a bit deceptive.  What I really
   want to do is establish a new mail/netnews delivery scheme that uses
   only one or two files (instead of the current three) for delivery 
   of a single message.  Sites would indicate their ability to handle
   the procedure by negotiating to use the "new channel" (perhaps "m"?)
   instead of "g".  Channel "m" would still use the ordinary packet
   driver and still transfer files in the conventional manner -- all the
   "m" indicates is that the sites have agreed to use the new mail
   delivery mechanism.
 
   There are a number of manners in which the new delivery mechanism could
   function.  One of the most obvious is to include the addressing
   information for the message in the body of the data file *of* the message.
   This information would be stripped from the message before final 
   delivery.  For example, something like:

   *TO-USER: ihnp4!vortex!lauren
   From seymour DateTime remote from foosite
   ...


   In this case, the "*TO-USER" line (or whatever we'd call it)
   contains the addressing information which would normally be 
   contained in the "X.foo" file for mail delivery.
   
   As I suggested above, netnews would probably be implemented first:

   *TO-PROGRAM: rnews
   From...

   If a site accepted a uucico connection using the "m" protocol, it 
   would be saying that its uuxqt was ready and willing to handle
   messages in this sort of format, as an alternative to the "g" format.

   This new technique involves the use of two files (the C. and D.) files
   in the sending system's spool dir, and the transfer of one file (D.) to
   the remote system.  With considerably more work, it would be possible
   to reduce the number of files in the sending spool dir to one, but I'm not
   sure that this would really be worth the effort involved.

   To avoid confusion, it might well be advisable to store the mail
   data file, under the new format, as something other than a typical
   D. file -- Perhaps M. or something similar would be suitable.

   I believe that the overall changes required to implement such a scheme
   for netnews/mail, both in uucico and uuxqt, are actually quite small.
   I also suspect that a substantial increase in overall uucp efficiency
   might result, with minimal compatibility problems.  We would still
   be using an efficient transfer mechanism, only the upper-level file
   handling/delivery mechanism would change.  For netnews, message batching
   and other techniques could still be used to gain even more benefits.
   
   Comments?

--Lauren--