Synchronet NewsLink Installation/Configuration ============================================== $Id: newslink.txt,v 1.6 2012/07/18 14:52:35 deuce Exp $ Background ---------- NewsLink is an external module for Synchronet BBS Software v3.1+. It is used to link one or more Synchronet sub-boards (aka message areas) with one or more USENET (or other NNTP network) newsgroups on one or more News Servers. News Servers use the Network News Transfer Protocol (NNTP) to transfer messages to and from a News Client (aka newsreader). Popular News Clients include Microsoft Outlook Express, Netscape Communicator, Mozilla, Pine, Xnews, and many others. NewsLink emulates a News Client based on the NNTP standard. News Servers normally listen for incoming connections on TCP port 119 and usually require an authenticated user login. NewsLink is written in 100% JavaScript and the source code (if you're interested) is contained in the files newslink.js and newsutil.js in the Synchronet EXEC directory. Step-by-Step Installation/Configuration --------------------------------------- If you have installed Synchronet v3.10 or v3.11 from scratch (not an upgrade from an earlier version), then some of the following steps may have already been completed for you. 1. Go into SCFG->External Programs->Timed Events and created a timed event for NewsLink (if one doesn't already exist). Example: [NEWSLINK Timed Event] 1: Internal Code NEWSLINK 2: Start-up Directory 3: Command Line ?newslink 4: Enabled Yes 5: Execution Node 1 6: Execution Days of Month Any 7: Execution Days of Week All 8: Execution Frequency 48 times a day 9: Requires Exclusive Execution No 10: Force Users Off-line For Event No 11: Native (32-bit) Executable 12: Use Shell to Execute 13: Background Execution 14: Always Run After Init/Re-init No You may change the Execution Frequency to suit your needs. 2. Go into SCFG->Message Areas and create a new Message Group for your USENET messsage areas (assuming you're linking to a USENET News Server). Example: [USENET Group] 1: Long Name USENET Newsgroups 2: Short Name USENET 3: Internal Code Prefix USENET_ 4: Access Requirements Note: At this point you may want to login to your New Server with a regular News Client (aka newsreader), retrieve a list of available newsgroups and make a list of the newsgroups you would like linked into your BBS's message areas. 3. Now go into SCFG->Message Areas->USENET->Sub-boards and create some sub-boards to link to newsgroups on the News Server. Example: [alt.bbs.allsysop Sub-board] 1: Long Name alt.bbs.allsysop 2: Short Name alt.bbs.allsysop 3: QWK Name allsysop 4: Internal Code USENET_ALLSYSOP 5: Newsgroup Name alt.bbs.allsysop 6: Access Requirements 7: Reading Requirements 8: Posting Requirements REST NOT G 9: Operator Requirements 10: Moderated Posting User 11: Maximum Messages 5000 12: Purge by Age Disabled 13: Duplicate Checking Enabled (10000 message CRCs) 4. For each NewsLinked sub-board, set Network Options->Internet (UUCP/NNTP) to "Yes". 5. Edit your ctrl/newslink.cfg file: A. Change the "SERVER" line to specify the host name or IP address of the News Server you are linking with. If the News Server is listening on a non-standard TCP port (e.g. other than 119), you will need to add a "PORT" line specifying the TCP port number of the server. B. If the News Server requires an authenticated user login, change the "USER" and "PASS" lines to specify the username and password you use to login to the above News Server. If the "USER" line is missing or commented-out (with a semicolon), then no user authentication will be attempted (sometimes called "anonymous login"). C. For each sub-board you configured in step 3, add an "AREA" line, specifying each sub-board's "Internal Code" (from SCFG->Message Areas). Example: AREA USENET_ALLSYSOP D. By default, NewsLink will only import articles (aka messages) posted *after* the first scan of the newsgroups (i.e. it will not import *any* articles during the first run). If you want NewsLink to import *all* existing articles for *all* linked newsgroups (potentially many *thousands* of messages, depending on the News Server), add the line "FLAGS I" (case insensitive). WARNING: Using this feature may cause the initial NewsLink execution to take a long time to download all the existing articles (potentially many thousands) that may be stored on the News Server. E. Remove or comment-out (with a semicolon) the "DISABLED" line. F. If you wish to use TLS/SSL, incomment (remove the semicolon) the "TLS" line. 6. Now you may force the NEWSLINK timed event to run. In SBBSCTRL (Win32 GUI), you can do this by clicking BBS->Force Timed Event->NEWSLINK, or for the console or NT services versions of Synchronet, you can touch or create the file data/newslink.now to force the event to run. 7. Watch your events log for the NewsLink execution progress and details. Remember, if the 'I' flag was not specified (globally or per-newsgroup) in your ctrl/newslink.cfg, then no articles (aka messages) will be imported during the first run and only newly-posted articles will be imported on subsequent runs. Area Flags ---------- Each "AREA" line in the NewsLink configuration file can specify one or more "flag characters" to enable special behaviors. These same flag characters may be specified globally (for all linked areas) using the "FLAGS" line. The supported flags characters are: x do not add tearlines & taglines to exported messages n do not add "From Newsgroup" text to imported messages t do not add tearline to imported messages a convert extended-ASCII chars to ASCII on imported messages r remove "Newsgroups:" header field from imported messages b decode single-part (uuencoded or yenc-encoded) binary attachments i import all articles on first run (not just new articles) s no subject filtering (do not check against text/subject.can) m moderate imported messages (a subop will need to validate every message) To specify one or more flag characters for a specific area, you must use the following "AREA" line syntax: AREA [attachment_dir] where: is the sub-board's complete internal code (including the group's code prefix, if there is one). is the name of the newsgroup as it is represented on the News Server. If this field is missing or is less than 2 characters in length, the newsgroup name from SCFG is used. is the optional list of flag characters. [attachment_dir] is the optional destination directory for decoded binary file attachments (only used with the 'b' flag) Examples: AREA USENET_ALLSYSOP alt.bbs.allsysop nib /sysop_attachments AREA USENET_ALLSYSOP - asr Multiple Servers/Config Files ----------------------------- ctrl/newslink.cfg is just the default NewsLink configuration file name. If you specify the path to a different file on the "?newslink" command-line, NewsLink will use that configuration file instead. This allows you to have multiple NewsLink timed events configured with different configurations for different News Servers. Command-line Options -------------------- Other supported NewsLink command-line options: -d enable debug output -ri reset import pointers (import all) -ui update import pointers (import none) -re reset export pointers (export all) -ue update export pointers (export none) -ne no e-mail addresses -nm no mangling of e-mail addresses (disable anti-SPAM measures) -um un-mangle e-mail addresses when importing -ix [n] import a fixed number (n) of articles/messages (default is 500) Controlling SPAM ---------------- Most SPAM messages on public NNTP networks (e.g. USENET) are directed at multiple newsgroups. For this reason, the newslink.cfg file supports a "max_newsgroups_per_article" value which defaults to 5. If you find NewsLink is filtering valid messages with an error in the log like: !Too many newsgroups (7) try increasing this value. If you find NewsLink is allowing SPAM through that is addressed to multiple newsgroups, try reducing this value. Set max_newsgroups_per_article to 0 to disable this feature. NewsLink also uses the text/subject.can filter file to filter (ignore) articles with specific text in the subject and the ctrl/twitlist.cfg file to ignore messages from specific users or domains. Yielding -------- NewsLink will yield time-slices based on the "lines_per_yield" (default: 5) and "yield_length" (default: 1) values. If you find NewsLink is consuming a large portion of your CPU cycles, try decreasing the "lines_per_yield" value or increasing the "yield_length" value. Stopping NewsLink ----------------- NewsLink can be stopped by "touching" (creating or updating) the file: data/newslink.stop JSexec ------ With Synchronet v3.11, it is also possible to run NewsLink externally from Synchronet (or as a background timed event) by using JSexec. /* End of file */