Path: network.ucsd.edu!ihnp4.ucsd.edu!swrinde!cs.utexas.edu!uunet!news.mentorg.com!sdl!not-for-mail From: tal@Warren.MENTORG.COM (Tom Limoncelli) Newsgroups: news.software.nntp,news.software.b,news.answers Subject: INN FAQ Part 1/4: General Information Supersedes: <inn-faq-1-764139607@Warren.MENTORG.COM> Followup-To: news.software.nntp Date: 5 Apr 1994 04:00:09 -0000 Organization: Mentor Graphics - IC Group, Warren, NJ, USA Lines: 673 Sender: tal@Warren.MENTORG.COM Approved: news-answers-request@MIT.Edu Distribution: world Expires: 04/20/94 Message-ID: <inn-faq-1-765518407@Warren.MENTORG.COM> Reply-To: Tom_Limoncelli@Warren.MENTORG.COM (Tom Limoncelli) NNTP-Posting-Host: sdl.warren.mentorg.com X-Summary-1: Part 1: Common questions about INN itself, useful to people that do not currently run INN. Also, some advice specific to certain operating systems. X-Summary-2: Part 2: Read this AFTER you've read and followed the directions in Install.ms. Help with getting innd to start. A tutorial on debugging posting/access problems. A list of error messages and what they mean. X-Summary-3: Part 3: Day-to-day operational questions. General questions asked once INN is running for a while. Some big changes you can make. Bug warnings for 1.4, 1.3, 1.2. X-Summary-4: Part 4: Norman's quick guide to getting started (assumes SunOS and other things), and misc. other things. Xref: network.ucsd.edu news.software.nntp:6307 news.software.b:6163 news.answers:20327 Posted-By: auto-faq 2.4 Archive-name: inn-faq/part1 Last Changed: $Id: FAQ-inn.1,v 1.65 1994/03/24 16:59:42 tal Exp $ Part 1 of 4 INN FAQ Part 1/4: General Information: Questions from people that don't (yet) run INN Specific notes for specific operating systems INN FAQ Part 2/4: Debugging Guide & Tutorial: INN FAQ Part 3/4: Operational and Misc. Questions INN FAQ Part 4/4: Appendix A: Norman's install guide ------------------------------ Subject: Table Of Contents for Part 1/4 ===================================================================== TABLE OF CONTENTS FOR PART 1/4: General Information ===================================================================== QUESTIONS FROM PEOPLE THAT DON'T (YET) RUN INN: Where can I get the INN software? Where can I get the latest copy of this FAQ? What is INN? What machines does it run on? INN must be really complicated since this FAQ is so long! Can I run C News with INN? Can I run NNTP with INN? Can I run the reference implementation (NNTP1.5) with INN? Can I run INN on my UUCP-only machine? Suppose I have a 286 machine? Does INN implement NOV, xthread (trn) or xuser/xmotd (tin) commands? Is an ident or authorization protocol supported? Does INN do UUCP batching like C News? Help! How do I configure this beast? How do I thank the FAQ maintainer? SPECIFIC NOTES FOR SPECIFIC OPERATING SYSTEMS: AIX tips SunOS 4.1.2 tips Ultrix mmap tip Ultrix syslog tip HP-UX tips Pyramid and system's where only root can have "cron" jobs. System V based Unixes (SVR4, Solaris 2.x, SCO ODT 3.0, A/UX, ...) Solaris 2.x special needs Mac running A/UX 3.0 NOV problems on a Pyramid Warnings to people that must set HAVE_UNIX_DOMAIN to DONT Alpha OSF tips The FAQ was written by Rich $alz <rsalz@rodan.uu.net> and is now maintained by Tom Limoncelli <tal@warren.mentorg.com>. These documents would not exist if it weren't for the people that have submitted questions and (most importantly) answers. THANK YOU ALL! ===================================================================== QUESTIONS FROM PEOPLE THAT DON'T (YET) RUN INN ===================================================================== ------------------------------ Subject: Where can I get the INN software? The official archive site is ftp.uu.net in the directory networking/news/nntp/inn. Archie current lists over 30 archive sites; three other international sites are grasp.insa-lyon.fr in pub/unix/news/inn, munnari.oz.au in pub/news/inn, and src.doc.ic.ac.uk in computing/usenet/software/transport The latest version of INN is 1.4sec. This is rev. 1.4 with a slight modification of certain shell scripts to fix a security hole. Do not run 1.4 without the patch (see part 3/4 of this FAQ). ------------------------------ Subject: Where can I get the latest copy of this FAQ? 1. This four-part FAQ is available via FTP at any FTP site that carries INN itself. 2. This FAQ is also posted to news.software.nntp, news.software.b and news.answers on the 5th and 20th of every month. 3. You can also get this FAQ by sending email to majordomo@warren.mentorg.com with "get file faq-inn-1" on the first line of the message. (the Subject: of the message will be ignored.) (Substitute faq-inn-2, faq-inn-3 or faq-inn-4 to get those parts). For example: % echo get file faq-inn-1 | mail majordomo@warren.mentorg.com % echo get file faq-inn-2 | mail majordomo@warren.mentorg.com % echo get file faq-inn-3 | mail majordomo@warren.mentorg.com % echo get file faq-inn-4 | mail majordomo@warren.mentorg.com or % mail majordomo@warren.mentorg.com Subject: ignored get file faq-inn-1 get file faq-inn-2 get file faq-inn-3 get file faq-inn-4 ^D or if you csh and are a total nerd, you can: echo 'get file new-inn.'{1,2,3,4}@ \ | tr @ '\012' | mail majordomo@warren.mentorg.com ...or get a real hobby. ------------------------------ Subject: What is INN? For a complete answer, why not read the Usenix paper that introduced INN to the world? It's available as ftp.uu.net:networking/news/nntp/inn/inn.usenix.ps.Z InterNetNews is a complete Usenet system. The cornerstone of the package is innd, an NNTP server that multiplexes all I/O. Think of it as an nntpd merged with the B News inews, or as a C News relaynews that reads multiple NNTP streams. Newsreading is handled by a separate server, nnrpd, that is spawned for each client. Both innd and nnrpd have some slight variances from the NNTP protocol (although in normal use you will never notice); see the manpages. INN separates hosts that feed you news from those that have users reading news. If you need to support a mixed environment you will have to do some extra work; the installation manual gives some hints. ------------------------------ Subject: What machines does it run on? If you have socket() and select() then INN will probably run on your machine. In addition to the common platforms found around the Internet (SunOS and Ultrix, for example), INN runs on IBM`s AIX, Apple's A/UX, NeXT, Solaris 2.x, most SVR4 platforms, BSDI, most free BSD systems on Intel (NetBSD, FreeBSD, ...) and a host of others. It might require a little bit of tweaking of some free BSD platforms that have really bad shells. ------------------------------ Subject: INN must be really complicated since this FAQ is so long! No, it's just that the FAQ is very complete. Part 2 is so long because it includes a tutorial that tries to turn a beginner into an TCP/IP protocol expert. Part 3 is long because the FAQ maintainer is constantly trying to add every question ever asked on news.software.nntp. Maybe someone should volunteer to maintain an index. A lot of the material could be integrated into the Install.ms doc. ------------------------------ Subject: Can I run C News with INN? No. INN handles all article reception, filing, forwarding, and expiration. You will get a corrupted database if you try to run INN with any other news system. For testing, you can probably shut down your old system, bring up INN, and then reverse the process. (INN uses the C News history file and DBZ database, so if you don't run C News you will have to do some fiddling around with those files.) ------------------------------ Subject: Can I run NNTP with INN? There's a confusion here. NNTP is a protocol, defined in RFC 977. There is also an implementation of the protocol, NNTP1.5, that many people call NNTP. When there was only one implementation of the protocol, that was okay, but now that there are other implementations (for example, INN) it is getting confusing. It would be as if "sendmail" were named "smtp." Please try to be clear -- do you mean the NNTP protocol, or the NNTP reference implementation currently maintained by Stan Barber? ------------------------------ Subject: Can I run the reference implementation (NNTP1.5) with INN? The quick answer is no. INN listens on the NNTP port and handles all incoming traffic. It receives articles, files them, and arranges for them to be forwarded to your peers. If a site connects that is not listed as a peer (e.g., a local workstation that does newsreading) then the INN server hands the connection off to another program that handles just the NNTP commands that newsreaders use. By default, this is nnrpd (notice the "r"), which implements the NNTP protocol for newsreaders (for example, it includes the POST command but not the IHAVE command). You can run the reference implementation server instead of nnrpd if you want. Doing this can be useful if you have clients that want to do both reading and article transfer. ------------------------------ Subject: Can I run INN on my UUCP-only machine? Sure. While not designed for this, several people are running INN on machines that do not have IP-connectivity (such as UUCP-only hosts) and are quite happy with it. You might want to give it a try, especially if you think you will be joining the Internet some day. ------------------------------ Subject: Suppose I have a 286 machine? Won't work. INN is designed to be a memory hog; a server that has been up for a few days while will have a working set size of a few to several megabytes, although not all of it will be resident. For example, the server keeps the active file and list of who gets what in memory, as well as all articles that it is receiving. Unless you can do things like "malloc(64 * 1024)" without pain, INN won't work on your machine. ------------------------------ Subject: Does INN implement NOV, xthread (trn) or xuser/xmotd (tin) commands? Newsreaders all need some way to quickly grab data from a range of articles. trn, tin, nn, and others each developed their own database format, and their own extensions to the NNTP protocol for clients to read the database. Then, Geoff Collyer invented NOV, the database to end all other databases. Rather than support a different database format for each newsreader, INN supports Geoff Collyer's news overview database, NOV. INN includes everything you need to create/maintain/expire NOV's .overview files. You only need Geoff's distribution (available via FTP on world.std.com src/news/nov.dist.tar.Z) if you want to see how he implements things, or to get his client library (useful when building some newsreaders). It is very easy to configure INN to use NOV. Read Part 3/4 of this faq: Subject: Cookbook example setting up NOV ("overchan") NB: The NOV code in INN 1.3 is buggy. Use 1.4 or higher. The xover command is used for querying the NOV database. The xover command is very smart in that if the article has been canceled, the data isn't given out. If the article is so new that it's data isn't in the NOV database, nnrpd opens the article and digs out the data. xhrd and xpat make every effort to use NOV data before they dig the data out of the actual articles, thus making them considerably faster than other implementations. The "xoverview" command does not exist. If your server supports this command you need to upgrade. What about TRN's xthread command? The xthread command has code but it is not supported; look at $inn/nnrpd/nnrpd.h. This code will probably vanish after 1.4. What about TIN's many commands? Tin commands are not supported. However, other people have added TIN support. See the next section. ------------------------------ Subject: Is an ident or authorization protocol supported? An unsupported patch for nnrpd to add ident support can be found via FTP at ftp.csie.nctu.edu.tw:/pub/news/nnrpd-identd-patch.shar.gz. It includes patches to add tin's xuser and xmotd commands as well as a list subscription system. (skhuang@csie.nctu.edu.tw) Remember: ident information is only as valid as the server you connect to. Any fool with root access (or anyone with a PC or Mac) can create a bogus ident server that will give out whatever information they want you to see. It's not an authorization or identification protocol, it's just informational. The authd protocol is added to nnrpd by <nhiro@isci.kyutech.ac.jp>. This unsupported patch can be found in ftp.geophys.hokudai.ac.jp:/pub/network/news/inn/patches/authd-patch-1.3.tar.gz The documentation is in Japanese. Good luck. In a future release, you will be able to log all POST commands with ident information if you so choose. (Maybe INN 1.5) ------------------------------ Subject: Does INN do UUCP batching like C News? Not as part of the standard distribution. The batching system right now is better than B News, but Rich has said he will be working on improving that part of INN in a future release. Christophe Wolfhugel <Christophe.Wolfhugel@grasp.insa-lyon.fr> has written a package that is very much like the C News batching system, however. You can find it on grasp.insa-lyon.fr in the pub/unix/news/inn/contrib directory. Version 3 of Christophe's package includes a shell version and a Perl version. Version 4, not yet planned will only be in Perl. The configuration file has evolved from older releases in order to support new features like "minimum batching". ------------------------------ Subject: Help! How do I configure this beast? READ AND FOLLOW THE "Install.ms" FILE. This FAQ is meant to add to what Install.ms says not replace it. Many people that thought the Install.ms doc was incomplete later re-read the "First Time Installation" portion and were amazed how much they missed (or just plain skipped) the first time. You should also purchase the O'Reilly And Associates book on Managing Usenet to give yourself a good grounding on how to run a site. ------------------------------ Subject: How do I thank the FAQ maintainer? If you find this document useful, please consider making a donation to the maintainer's favorite charity: The Personal Liberty Fund PO Box 11335 New Brunswick, NJ 08906-1335 USA The PLF is a legal and educational organization which helps people in New Jersey, USA. They have many on-going projects which range from combating youth suicide to legal advocacy. Their Anti-Violence Project runs a phone hotline for reporting gay-bashing which helps hundreds of people a year. The PLF is an all-volunteer organization and couldn't survive without donations. The PLF is recognized by the IRS as a 501(c)3 tax-exempt charity. Donations are tax deductible to the fullest extent of the law (U.S. citizens only). Please write "INN" in the memo field of the check. ====================================================================== SPECIFIC NOTES FOR SPECIFIC OPERATING SYSTEMS ====================================================================== ------------------------------ Subject: AIX tips Q: When I run news.daily, there's always a few lines of error messages at the end of the output: compress: bad file number A: AIX /usr/bin/compress has a bug when compressing files with zero length. Then it spits out this error. Solution: Ignore it or use a different compress programm and change config.data accordingly. (from Kurt Jaeger <pi@rus.uni-stuttgart.de>) Q: innwatch doesn't work well from /etc/inittab, does it? Q: Nope. Instead, you can create a "subsystem" with this command: mkssys -s innwatch -p /usr/local/news/bin/innwatch -u news -G news -S -n 15 -f 9 Note that your path to innwatch may differ, depending on where you decided to install the inn components. You also need to enter the command as one long line. This will create a subsystem named "innwatch" belonging to an SRC group named "news". The "-S" means that it uses signals for SRC to tell it when to stop and the "-n" is the SIGTERM signal, for normal shutdown, and the "-f" is the SIGKILL signal, which is sent if the process does not stop within 20 seconds. Then, modify rc.news to issue the command startsrc -s innwatch to get innwatch going. That's it! Shane Castle <swcxt@boco.co.gov, swcxt@csn.org> ------------------------------ Subject: SunOS 4.1.2 tips SunOS 4.1.2 (but not 4.1.1 or 4.1.3) broke the write system call but a patch is available. Any write could fail "half way", it is just more likely to happen when writing large files and in-core DBZ writes the history file out in one chunk. See the "Known Problems" section of the installation manual. ------------------------------ Subject: Ultrix mmap tip Ultrix has a "mmap()" function, but it doesn't do the same thing as the SunOS/BSD mmap() function. Therefore, do not configure INN to use mmap() on a Ultrix system. INN wants to find a mmap() function that is like the one on SunOS/BSD systems. ------------------------------ Subject: Ultrix syslog tip The syslog on Ultrix sucks rotten eggs and Digital refuses to fix it. (source: everyone that uses Ultrix and has ever used other systems) Luckily, you can replace it with the routine that comes with INN. However, some people have had better luck installing the syslog that can be found on gatekeeper.dec.com:/pub/DEC/jtkohl-syslog-complete.tar.Z It still works with old clients but does new-style syslogging, too. Works great for me so far. (this information from: nelson@reed.edu (Nelson Minar)). The syslog that is shipped with INN works pretty well but there have been some claims that some old clients don't like it. ------------------------------ Subject: HP-UX tips Q. My logs keep telling me there is no space for articles A. Edit innwatch.ctl to use "bdf" instead of "df". Q. I am running inn on an HP machine. INN won't start up automatically. I can start it manually. There is no problem with news or INN once it is started. A. Try adding a "sleep 10" to the bottom of /etc/rc.news, or in /etc/rc, right after /etc/rc.news is invoked. On some machines, including HP, the shell started by "#!/bin/sh" when /etc/rc is executed will exit before innd has disassociated itself from that shell. This causes innd to exit, sometimes without printing an error message. (source: pjoslin@mbvlab.wpafb.af.mil (Paul Joslin )) This problem goes away if you set HAVE_SETSID to "DO". Something to do with Posix Session Leader concepts. Ick. (source: Steve Howie <scotty@piranha.cs.uoguelph.ca>) HP-UX 8.x and 9.x users might find a problem with getting innwatch to start up. People have found that having "at" start it seems to work more reliably than other methods: ${DOINNWATCH} && { echo "${INNWATCH} &" | su ${NEWSUSER} -c 'at now + 2 min' > /dev/null } The '&' in the command line prevents innwatch from taking up an at-job slot. SysV-style crons have a per-category and overall limit on the number of jobs executing simultaneously. ------------------------------ Subject: Pyramid and system's where only root can have "cron" jobs. Your cron jobs may not work if you use: su news -c /usr/lib/news/bin/news.daily delayrm expireover Instead, you must put the entire command in quotes. Like this: su news -c "/usr/lib/news/bin/news.daily delayrm expireover" Look for "Pyramid" in part ??? of this FAQ for details. ------------------------------ Subject: System V based Unixes (SVR4, Solaris 2.x, SCO ODT 3.0, A/UX, ...) If you are running any non-BSD (i.e. System V based) Unix you MUST have the following option set: ## How should close-on-exec be done? Pick IOCTL or FCNTL. #### =()<CLX_STYLE @<CLX_STYLE>@>()= CLX_STYLE FCNTL This includes SVR4, Solaris 2.x, A/UX and SCO ODT 3.0. This is clearly stated in the Install.ms file and repeated here since so many people post to news.software.nntp after ignoring the warnings. If it isn't FCNTL, you'll get tons of overchan processes hanging around. (source: Philip Gladstone <philip@charon.cto.citicorp.com>) In SCO ODT 3.0 and MOST systems, innd will link and run if you use IOCTL but eventually will stop answering incoming calls. Don't be fooled. Just because it compiles doesn't mean it's going to work. If you start innd on an AT&T SysV Rel 4.0 machine and get syslog messages like: localhost:15 cant setsockopt(SNDBUF) Protocol error localhost:15 cant setsockopt(RCVBUF) Protocol error then you should FIRST try to change HAVE_UNIX_DOMAIN to "DONT" in config.data. If that doesn't fix the problem, you should comment out the "setsockopt()" calls and things will work. Many SVR4 for i486 binaries (sendmail, mh, vmail, innd, rnews are now on ftp.germany.eu.net in pub/comp/i486/svr4/*.SVR4.tgz But remember that some of the above need site specific changes, so their usefulness may be limited. If your SVR4 system still doesn't run correctly, check the Solaris 2.x suggestions. ------------------------------ Subject: Solaris 2.x special needs Under Solaris 2.x (where x = 0, 1, 2 or 3) you need to change the file "getfqdn.c". Find the lines that read: if (strchr(hp->h_name, '.') == NULL) { /* Try to force DNS lookup if NIS/whatever gets in the way. */ (void)strncpy(temp, buff, sizeof buff); (void)strcat(temp, "."); hp = gethostbyname(temp); } and delete them. Under Solaris 2.[012] (SunOS 5.0, 5.1, 5.2) you must add the following at the beginning of each file using gethostbyname(): #define gethostbyname __switch_gethostbyname Under Solaris 2.3 gethostbyname() might work without changes depending on your configuration. We haven't figured out when they work and when they don't. If you run into problems, try to change "gethostbyname()" to "solaris_gethostbyname()" and then use the gethostbyname() listed in the Solaris Porting FAQ. This isn't a perfect solution, because you now need a different binary for Solaris 2.[012] systems. It would be great if someone were to submit a solaris_gethostbyname() function who's binary works under all Solaris revs and gives all the semantics of BSD gethostbyname(). In particular, one that doesn't have the problems discussed in sun bugid #1126573 or #1135988. It would be amazing if this was submitted by one of the many Sun employees that flame the INN FAQ maintainer in comp.sys.sun.admin everytime he bitches about how much he hates Solaris 2.x. :-) Under all Solaris 2.* versions there is a problem with innwatch.ctl. It expects to use "df -i" to find out how many inodes are free on your disk. /usr/{sbin,5bin,bin}/df doesn't support the "-i" option. You have to use "/usr/ucb/df -i" instead, since this version of df includes the "-i" option. ------------------------------ Subject: Mac running A/UX 3.0 Tip #1: Use the INN malloc. Tip #2: If you are running INN 1.4 on a Mac running A/UX 3.0.1, Every so often, (generally when someone fires up a reader), INN goes beserk. Syslog says: innd: ME cant select Bad file number This message repeats about 20 times per second. It freezes up my computer and I need to reboot. That's a kernel bug. You do have to reboot. If you compiled inn with gcc, don't. My experience was that somehow, if INN was compiled with GCC the kernel bug is triggered, but that doesn't happen with cc. ------------------------------ Subject: NOV problems on a Pyramid Q: I just turned on the overview stuff and I don't think news.daily is properly expiring the .overview files. I'm using a Pyramid. A: Do you need quotes in your crontab entry? Look at your news.daily report -- expire using "expireover delayrm" should take a few minutes. If it takes longer than, say, 10-20 minutes, then the keywords aren't being seen by news.daily so perhaps the commandline quoting is wrong. i.e. you had: su news -c /usr/lib/news/bin/news.daily delayrm expireover You should have: su news -c "/usr/lib/news/bin/news.daily delayrm expireover" ------------------------------ Subject: Warnings to people that must set HAVE_UNIX_DOMAIN to DONT Disclaimer: First of all, if you have to set HAVE_UNIX_DOMAIN to DONT, YOU HAVE TO SET IT to DONT. It's not a choice you can make, it's a description of the operating system that you purchased. If you wrongly set this variable to DO your system isn't going to work *at* *all*. When you use the POST via NNTP, you are talking to nnrpd. nnrpd cleans up your headers, adds the missing headers that it is allowed to add, checks whatever it checks, and then submits the finalized version to innd. How does it talk to innd? If you have HAVE_UNIX_DOMAIN set to DO, nnrpd opens a Unix domain socket and sends the text. At this point it is talking to innd somewhat like ctlinnd does. innd can trust that the post isn't forged since it is coming from a program trustworthy enough to get to the socket (which isn't much). If you have HAVE_UNIX_DOMAIN set to DONT, it has no choice but to open a socket to port 119, issue the "IHAVE" command, and send the text that way (just like a remote newsreader). This means that innd (not another nnrpd) has to be at the other end of the pipe. If it opens the connection and sees a "nnrpd" you're hosed and you get "441 480 Transfer permission denied". (Better the "441 480" message than an infinate loop of nnrpd's connecting to nnrpd's!) To get innd to not hand off the connection to a nnrpd process, you must have the host's name in the hosts.nntp file. (don't forget to do "ctlinnd reload hosts.nntp") If you have your host's name in the hosts.nntp file, then any newsreader running on your nntphost must be "INN-aware" (i.e. that they issue the "mode reader" command) or they must read news via the file system instead of NNTP. If you have NNTP-based newsreaders that can't send the "mode reader" command, you can try including "server: localhost" in your inn.conf file, but then you must have a different inn.conf file for the other machines. If you can't do that, you have no other options but to recompile your newsreaders. Remember, if you change your inn.conf file, you must shutdown and restart innd. There is no "ctlinnd reload inn.conf" command. ------------------------------ Subject: Alpha OSF tips: To compile INN for the DEC Alpha, follow the instructions in the INN patch archive on ftp.pop.psu.edu:/pub/src/news/inn-patches In rc.news you need to start $INNWATCH using the following: ${DOINNWATCH} && { echo "${INNWATCH} &" | su ${NEWSUSER} -c 'at now + 2 min' > /dev/null } The '&' in the command line prevents innwatch from taking up an at-job slot. SysV-style crons have a per-category and overall limit on the number of jobs executing simultaneously. -- Tom Limoncelli -- tal@warren.mentorg.com (work) -- tal@plts.org (play) "Psst! Hey, Anthony! Y'know what I | Disclaimer: I do not like about existing?" "Uh... uh... what?" | speak for Mentor Graphics. "Possessing a physical extension." -TSA |