Path: utzoo!attcan!utgpu!jarvis.csri.toronto.edu!mailrus!ames!pacbell!indetech!david
From: david@indetech.uucp (David Kuder)
Newsgroups: news.software.b
Subject: Re: C news meets a mailing list
Summary: some success after sufficient sleep
Message-ID: <1989Aug10.221915.15905@indetech.uucp>
Date: 10 Aug 89 22:19:15 GMT
References: <1989Aug7.060124.8135@indetech.uucp> <1989Aug8.154631.2816@utzoo.uucp>
Reply-To: david@indetech.UUCP (David Kuder)
Distribution: na
Organization: Independence Technologies, Inc. Fremont, CA
Lines: 76

In article <1989Aug8.154631.2816@utzoo.uucp> henry@utzoo.uucp (Henry Spencer) writes:
>In article <1989Aug7.060124.8135@indetech.uucp> I wrote:
>>Given my track record on this stuff I may have missed it, but is there
>>a method for gatewaying an inbound mailing list into a news group in 
>>C news?  In B I used "recnews"...
>
>If you look at the newsmail(8) manual page, you'll see our equivalents.
>You may also want to look at contrib/nntpmail/mail*, which contains stuff
>done elsewhere at U of T for such things.  We didn't include anything
>fancy as an official part of the distribution -- note the disclaimers
>in contrib/README -- partly because we didn't have an opportunity to test
>it ourselves.

I got several specific solutions from other folks.  I will get around
to picking one of them this weekend when I have time.  As for
newsmail(8) I did see those but had interpreted them as transport
mechanisms and not gatewaying programs.  In particular recenews and
recpnews both presume that they are recieving mail that has been
encoded.  I overlooked the stuff in contrib because of the NNTP in the
name.  Silly me.  I consider it with the other candidates this weekend.

In article <1989Aug8.030953.13302@heifetz.ann-arbor.mi.us>
	osm@heifetz.ann-arbor.mi.us (Owen Scott Medd) writes:
=>Speaking not being able to see straight.  Any ideas were nearly empty
=>batches might be generated?  We send out the occasional cunbatch script
=>wrapped around the magic that says 12 bit compressed and zero actual
=>data.
=
=I *think* we see this problem when a togo.[0-9] contains nothing but
=expired/cancelled articles.  The batcher -> muncher -> sender pipeline
=in sendbatches doesn't allow for the fact that a togo file might not
=contain anything of value.

Back to Henry:
>Is it causing problems?  It's a normal occurrence if you have stuff
>expiring quickly enough that the batcher sometimes can't find any of
>the articles that are supposed to go into a batch.  Normally this only
>happens when a system which does fast expiring is feeding a site
>which has communications problems, so some of the batching gets deferred
>until after a lot of stuff has expired.

And quoting Owen:
=Since we have some downstream feeds that send sendme's that date from
=weeks ago and some other feeds that don't pick up their news for up to
=a week or so at a time, I tend to get those annoying "Inbound news
=garbled" messages.  I've got a little shell script that checks the batches
=to make sure at least one article exists, but this is definitely a hack.

What he said!  Those error messages that down stream sites send back
consume my time.  The sending of the empty batches and the error messages
consume phone time even with a telebit.  An empty (or short) UUCP
batches chew up a space on the UUCP queue.  This compounds of the
problem.  If a downstream site isn't picking up its UUCP batches fast
enough that articles referenced by the out.going/site/togo file are no
longer on my finite disk then short UUCP articles are created.  Since
these batches are short the queue length isn't an accurate
representation of how much news is queued to go out.  The feedback here
could end up with a site that managed to be just slow enough
guarenteeing that it never actually got any news.

Now, I have a hack in hand.  Given that I have sites that don't poll me
often enough for my own good I have a perl program that clears news
batches from their UUCP queue.  A quick mod to it makes it just nuke
the (nearly) empty batches.

In the long run (or this weekend which ever comes first) I will figure
out how to enforce the policy I want using the mechanism Henry and
Geoff have supplied.  My guess it that a check for empty batches in the
sendbatches pipeline after the muncher runs will be liveable.  To
really insure full batches would require rethinking batchsplit and its
interaction with sendbatches.
-- 
____*_  David A. Kuder              {sun,sharkey,pacbell}!indetech!david
\  / /  Independence Technologies
 \/ /   42705 Lawrence Place        FAX: 415 438-2034
  \/    Fremont, CA 94538           Voice: 415 438-2003