Path: utzoo!yunexus!spectrix!clewis From: clewis@spectrix.UUCP (Chris Lewis (It's loose again!)) Newsgroups: news.software.b Subject: Re: News header compliance... (fix for hangup) Summary: Fix for hang.... Keywords: Headers RFC B-news Message-ID: <676@spectrix.UUCP> Date: 23 Jun 88 17:30:37 GMT Article-I.D.: spectrix.676 References: <674@spectrix.UUCP> Organization: Spectrix Microsystems Inc., Toronto, Ontario, Canada Lines: 100 In article <674@spectrix.UUCP>, clewis@spectrix.UUCP (Chris Lewis (It's loose again!)) writes: > I've been having our B News (2.11 patch 14) hang occasionally while trying > to parse batched news - goes into some sort of loop when it decides > that an article is garbled. I'm still trying to fix the hang problem > (the "#!rnews" piping business appears to hang in the write), > but I've noticed that this usually comes from an article from utcsri > (often from one of the upcoming events articles). [synopsis: when an incoming batch contains an article with a tab instead of a space after the "Date:" token, rnews will report that the article is garbled and then sometimes hang.] I seem to have resolved the hanging problem. SYNOPSIS: When the "rnews -S" invocation is parsing batches and tossing "#! rnews nnn"-prefixed chunks to separate forks of itself, if the forked rnews determines that the article is garbled, it will exit and the "rnews -S" will hang in the write to the pipe - potentially forever. Regardless of whether the batch is compressed, or how many other articles are in the batch. DISCUSSION: The "rnews -S" personality makes a decision about whether to create a temporary file or use a pipe to transmit one article to a forked copy of itself, by figuring if the article is smaller than the buffer (CPBFSZ in ifuncs.c) then pipe it, otherwise dump the file to a "/tmp/unb*" temporary, and fork itself with either the pipe or the temporary file as standard input. If the forked rnews determines that the message is garbled, it exits immediately. HOWEVER, if the article is bigger than the in-core kernel pipe buffer size (PIPEMAX in some System V implementations - see SVID, or pipe(2)), but smaller than CPBFSZ, the write will not return immediately - it has written the first pipe-buffer-full, but has to wait for the destination to read the buffer so as to send the rest of the write buffer - which never gets read since the child rnews has exitted. This only appears to be a problem when both the article and CPBFSZ are bigger than the in-core kernel pipe buffer size and the article is smaller than CPBFSZ. On Xenix 2.1.3 and NCR Tower SVR2 (and many other systems (ie: V7) - we only have documentation for the two specific ones) the pipe buffer size is 5120. And, CPBFSZ is 8192 (which I *think* is the BSD pipe buffer size). REPEAT BY: Create a batch file (with "#! rnews" header) which has an article about 6K long and garble the "Date: " header (eg: change the space to a tab). Then issue: rnews -S < batchfile It will say something like: inews: : Inbound news is garbled. Then hang. If it doesn't hang, then congrats, you're probably on a machine with a bigger pipe buffer, and you don't need to do anything. Otherwise, you might want to do the fix given below. The fix is relatively harmless even if you don't *have* to do it - there'll simply be a few more articles using temporary files rather than pipes - only a slight performance hit. FIX: in ifuncs.c, find the "#define" for "CPBFSZ" and change it to be the same or smaller than your pipe buffer size. We chose 4096. Rebuild, reinstall and viola. NOTE: This is a fragment of the pipeing code in ifuncs.c: /* parent of fork */ if (rc == asize) { /* article fits in buffer */ wc = write(piped[1], buf, rc); if (wc != rc) { fprintf(stderr, "write of %d to pipe returned %d", rc, wc); perror("rnews: write"); exit(1); } (void) close(piped[0]); (void) close(piped[1]); } We figured that part of the reason why the write didn't terminate was because the "close(piped[0])" was *after* the write not before - thus there are two people (both sides of the fork) with the input side of the pipe open - thus no broken pipe when the child exited. I tried moving it (before changing CPBFSZ - which is the size of buf in the write), and the test shown above under "REPEAT-BY" would print one "garbled" message, a blank line, and ignore the next article in the batch. -- Chris Lewis, Spectrix Microsystems Inc, Phone: (416)-474-1955 UUCP: {uunet!mnetor, utcsri!utzoo, lsuc, yunexus}!spectrix!clewis Moderator of the Ferret Mailing List (ferret-list,ferret-request@spectrix)