ROOTSWEB REVIEW: Genealogical Data Cooperative Weekly News Vol. 1, No. 1, 17 June 1998 Copyright (c) 1998 RootsWeb Genealogical Data Cooperative Editors: Julia M. Case and Myra Vanderpool Gormley, CG ***************************************************************** This is the premier issue of the RootsWeb Review, a weekly e-zine with news about RootsWeb happenings, new mailing lists and Web sites, a continuing tutorial on using RootsWeb's resources, monthly statistics, news from major hosted activities such as USGenWeb, USGenWeb Archives, WorldGenWeb, and Cyndi's List, schedules for IIGS/RootsWeb IRC, and other information of interest to RootsWeb users and the genealogical community. The initial list of subscribers consists of those we know have used RootsWeb or are subscribers to "Missing Links," which has just moved to RootsWeb. To continue to receive RootsWeb Review, do nothing. However, if you would prefer not to receive future issues of RootsWeb Review, just send a message to RootsWeb-Review-L-request@rootsweb.com and put the word "unsubscribe" (omit the quotation marks) in the subject line and in the body of the message. Please remember to turn off your signature file before sending the message. ***************************************************************** CONTENTS: "Virtual Dreams: What is RootsWeb?"; "Spam Omelettes"; "We've Got Roots"; Mailing Lists and Web Pages; Geek Primer; Backstage at RootsWeb: Excerpts from the Staff Diary; and humor. ***************************************************************** VIRTUAL DREAMS: WHAT IS ROOTSWEB? RootsWeb Genealogical Data Cooperative (RootsWeb) is the oldest and largest genealogy resource on the Internet. Today it houses more than 2,650 genealogy Websites, including the USGenWeb Archives; ROOTS-L (in its 11th year) and more than 3,600 other mailing lists; and the Roots Surname List (in its 10th year). The June 1998 Roots Surname List contains 420,971 names from about 58,000 submitters. RootsWeb has two missions: (1) to make large volumes of data available to the online genealogical community at minimal cost; and (2) to provide support services to online genealogical activities such as Usenet newsgroup moderation, mailing list maintenance, and surname list generation. While all of RootsWeb's databases, mailing lists, and other activities are open to everyone, selected services (such as automatic notification when your surname appears in new material uploaded to RootsWeb) are available only to members and sponsors. Some services that are expensive to provide (such as personal Web space or Web serving of personal GEDCOM files) are available at a nominal cost to sponsors. The "cooperative" in RootsWeb's name means that we are a community-supported facility and your choice about joining has a direct impact on how fast RootsWeb can bring more resources online for the Internet genealogical community. By becoming a member, sponsor, donor, or patron, your support of RootsWeb helps to provide Web and FTP space to thousands of genealogical activities, mailing lists for thousands of groups of genealogists with shared interests, and search engines to make more than four billion bytes of genealogical data (the equivalent of two million printed pages) freely available to Internet genealogists. If you would like to help RootsWeb by becoming a member, sponsor, donor, or patron, please visit: ***************************************************************** SPAM OMELETTES by Tim Pierce RootsWeb Genealogical Data Cooperative System Obfuscator and Hack-of-All-Trades In terms of what we worry about day to day, spam is a more serious problem for us than security crackers. Computer capacity is not infinite. Even a fantastically powerful computer system can only process a certain amount of information in a day -- maybe a lot more than you or I would ever want on our desks, but finite nonetheless. When a spammer sends mail to one of our lists, that makes 50 copies that our list server has to distribute to the list subscribers -- or maybe 10 copies, or 1,000, depending on the list. Multiply that by 6,000 (which is approximately the number of lists we host on our mail servers, counting mail-mode and digest lists separately). Those numbers mean that a spammer who sends mail to every list on our systems -- which happens frequently -- is a big problem. Big enough that just sending enough spam to our machines is sufficient to crash them. So spam is actually a show-stopping problem for a big mailing list house like RootsWeb. We have had to contribute a significant chunk of staff and volunteer time to fighting the problem. About a month ago, we blocked almost 800 known spam domains from RootsWeb's list servers, to cut out some of the spam. More effectively, we installed some filters that look for certain spam-like patterns in each incoming message, and discard any message that matches (that is, bounce the message to the listowner). This has reduced the number of successful spam attacks on our systems considerably, which is a bonus both for our listowners and for our harried system administrators. By my estimates, subscribers to our lists see a lot less spam than users of other genealogical list servers. If folks receive spam that was sent to a RootsWeb list, they should forward it to abuse@rootsweb.com so I can figure out how to block any future attacks. I really need to know when spam makes it through our filters. When we get repeatedly hit by the same outfit, too, we can figure out where the spam is really coming from and try to put pressure on the Internet Service Provider. * * * 6/15/98 UPDATE: This was quite a day for the RootsWeb abuse team. famhis@doit.cc, origin@doit.cc, heraldry@doit.cc: This morning we were spammed by an outfit promoting surname histories. All of the posts came from a forged "doit.cc" domain, which I blocked as soon as I got word of what was happening. Because of serious Net lag this morning, however, that didn't happen until well after it had achieved its goal. CVBOX spam: Meanwhile, back at the ranch, this afternoon, a Portuguese spam outfit connected to rootsweb.com and spammed just about every list on our system. Three times each. Over about an hour, they pumped about 4,000 messages and 11MB into our list servers. Thankfully, they're using a well-known spamming software package that was readily caught by our filters. Even as I started to write this message, they started to do it all over again. So we had to take more drastic measures, dropping the Internet routes to their systems and blocking any mail from their (forged) domain name. That should prevent them from even being able to connect to our machines. All RootsWeb members, sponsors, donors, and patrons should congratulate yourselves for this. The funds that you've contributed to improve RootsWeb's systems were directly responsible for our being able to implement spam filters like these. If it weren't for your support, RootsWeb's list servers would be a smoking pile of slag right now. Thanks for helping to keep us running! ***************************************************************** WE'VE GOT ROOTS by Yvonne Martins Sr. Marketing Manager, Ultimate Family Tree Many of you are familiar with Ultimate Family Tree from its days as Roots. We are proud of our 17-year commitment to deliver the very best family tree software, enabling beginners to start easily, while not limiting the power needed by professional genealogists. But our commitment goes beyond our software's core features. We are proud of our corporate sponsorship of RootsWeb. We believe co-founders Dr. Brian Leverich and Karen Isaacson have done a great service for the genealogical community. Further to enhance RootsWeb, we are exploring the development of an Internet spider that will search the Web for genealogical sites and data more intelligently. To assist genealogists in their research, we recently announced the Ultimate Family Data Library Series. Our goal in creating the Ultimate Family Data Library is to ensure that the quality and reporting accuracy of our data are second to none. With dedicated research and independent testing, our data CDs enable users to search records by surname or SOUNDEX, and Ultimate Family Data Library is compatible with every brand of genealogy software on the market today. The Ultimate Family Data Library consists of a Master Index CD-ROM and 18 individual data CDs. With the Master Index, users will be able to search for names and learn which CD-ROMs contain more detailed information on their ancestors, allowing users to purchase only the CDs that they need. The Master Index will be available later this month on our data Web site located at , or, from anywhere in North America, by calling our toll free number: (888) 891-1919. We support your interest in genealogy and applaud your choice of hobby. Whether you're just getting started or are well into the process, we hope Ultimate Family Tree and the Ultimate Family Data Library will meet your needs. ***************************************************************** GEEK PRIMER: This week's entries come to you by courtesy of Tim Pierce, RootsWeb system obfuscator and hack of all trades. DAEMON. A daemon is generally any program that handles requests for data. When you go to a http: URL in your Web browser, the program that takes your request and sends you stuff is called a "http daemon." The program that handles incoming mail messages is called a "mail daemon." Just about everyone on the Net has gotten mail from a MAILER-DAEMON at least once in their lives. FSCK is short for "filesystem check." It is a command used by Unix sysadmins to scan a hard disk, looking for inconsistencies, corrupted files, and other lovely things that might arise after a disk crash. Running a filesystem check on a disk is colloquially referred to as "fscking" it. ***************************************************************** MAILING LISTS and WEB PAGES: To subscribe or unsubscribe from any RootsWeb mailing list, send an e-mail message with only the word SUBSCRIBE (or UNSUBSCRIBE) in the subject and the body of the message to [name of list]-L-Request@rootsweb.com (for mail mode) or to [name of list]-D-Request@rootsweb.com (for digest mode). If you have an automatic signature file, please be sure to turn it off before you send the request. For example, if you wish to be removed from this mailing list, send your UNSUBSCRIBE message to: RootsWeb-Review-L-Request@rootsweb.com Beginning next week a list of the previous week's new RootsWeb mailing lists and Web pages will be published in RootsWeb Review. Among the more unusual recently created RootsWeb mailing lists and Web pages are: GEN-UNSOLVED-MYSTERIES-L@rootsweb.com (David Burgess dwburgess@worldnet.att.net). This is a list for people whose family genealogies include "unsolved mysteries." Postings should include only mysterious disappearances or appearances, unsolved murders, questionable incarcerations, and other mysterious or unsolved events in an ancestor's life. Postings should NOT include "brick walls," as that would be repetitive of the content of other lists. INTERNATIONAL BLACK SHEEP SOCIETY OF GENEALOGISTS (IBSSG) . In addition to the Web page where you can read members' stories, news and notes, visit historical links and genealogical links, or join in a real time chat session, there is a Black Sheep mailing list (BLACKSHEEP-L@rootsweb.com), whose members include "those who have a dastardly, infamous individual of public knowledge and ill-repute somewhere in their family ... preferably in their direct line. This individual must have been publically pilloried in disgrace for acts of a significantly anti-social nature." ***************************************************************** BACKSTAGE AT ROOTSWEB: Excerpts from the Staff Diary. (The past month has been more than a little hectic at RootsWeb. We hope that this will give you the flavor of what it takes to keep RootsWeb up and running.) KEY: Dr. Brian Leverich (B); Karen Isaacson (K); Tim Pierce (P); Dale ("Doc") Schneider (D) 5/18/98 (B) A NEW SERVER IS STILLBORN. I love it when everything goes smoothly. ): There's a stability problem with the "homepages" box -- it doesn't crash, but gcc dies at random places during compiles moaning piteously about "Internal error 11." I've tried swapping out every component except the motherboard to isolate the problem. No joy. The motherboard is next. *sigh* . . . If I've learned anything from of a quarter of a century of hacking, it's that nature sides with the hidden flaw and even the best-designed hacks fail regularly and miserably for reasons that are only clear after the fact. 5/18/98 (B) A NEW SERVER IS REBORN. With a Pentium Pro 200, 512 MB of EDO DRAM, and 36 gigabytes of user disk space on four ultrawide fast SCSI drives, the box ought to do just peachy keen in this application. I should have the box on The Net on a lightly loaded T1 this evening. I swapped out the motherboard (genuine GF3), and "homepages" now seems to be rock solid. Also seems about twice as fast. 5/19/98 (B) While doing routine maintenance down at the NOC, I noticed both of the main cooling fans on BL-30 had failed. I'm going to kill the inbound sendmail right now. Once the box is quiescent, I'll shut it down and replace the fan. Probably about an hour's downtime total. 5/21/98 (B) I did a Los Angeles shopping run yesterday and should have all the parts on hand to build Marc's archiver box, the upgraded mail hub, and a third list server. And upgrade the main Web server. I've got bits and pieces for a couple more servers, too. 5/28/98 (B via K) Small Problems. This note is actually from Brian, but he sent it as root, so SmartList happily dumped it in the error bucket, rather than send it on. Sometimes being Important just doesn't pay. Brian says: When I logged onto WWW's console, I noticed some file system errors. Didn't look too serious, but I'm going to have a good time before I do the CPU upgrade. I'm going to take down the daemons and umount and check all the file systems before I shut down the box for the processor upgrade. 5/29/98 (B) I'm down in L.A. this afternoon and evening buying server parts and meeting with a consulting client. 5/30/98 (B) Power Outage. We lost power to our Birchwood and PMC NOCs this morning, and I've been down nursing the generator. Looks like we need a bigger generator and UPS. *sigh* 5/31/98 (B) Leverich Does Work -- World Shocked And Appalled! I actually cleaned up the room I use as a new-server nursery and discovered that we have enough parts (almost) lying around to build o One real rock-crusher of a machine (like searches) o Four solid servers like FP-1 and BL-30 o Three fast and light servers like cgi/cyndislist o And one miserable POS like the current BL-5. We have enough parts almost to double our current capacity. So I started bolting stuff together. I'll be down in L.A. tomorrow shopping for server parts and attending the SoCal Genealogical Jamboree. Monday I'll be back at building more boxen. I'm going to try to get the hardware online as fast as I can. 5/31/98 (D) BL-12 Died. At about 2:45 p.m. Sunday bl-12/users/ftp/www .rootsweb.com died. I was logged in and someone sent me a message that the webserver wasn't responding. I tried to save it but got knocked off. Tried to get back on for the past 45 minutes. Nothing. Tried pinging the box from BL-11 and nothing. Tried calling Tim (not there). B and K I know aren't there . . . So I guess until someone gets back things are going to be down. [signed] one frustrated Doc 6/01/98 (D) I'm working right now to bring the www.usgenweb.org and .net domains online on a new box. By moving things to their own box means better performance. Just some of the new and improved services from your hosts at RootsWeb. [signed] Doc, Sysadmin and guru of RootsWeb. 6/01/98 (B) HARDWARE FAILURE. One of the big SCSI hard drives in RootsWeb's main Web server is failing. I'm running backups now, and I hope I can get a current tape made before the drive fails entirely. We may be up and down quite a bit for the next 36 hours as I work through the problems. These are the times when I look forward to the day that we can afford to have RAID-5 arrays on all our servers. (RAID-5 arrays are composed of multiple redundant drives, and the failure of a single drive has no effect on the operation of the overall array.) Thanks to all the members, sponsors, donors, and patrons who have provided the resources that allowed us to have spare drives available for emergencies like this. 6/02/98 (B) Tomorrow morning I will down the old server and move the still functional user disks to the new server. I'm actually not doing the repair today (Tuesday). In view of a number of technical issues, we're totally rebuilding the server from the case up with the fastest parts we can buy. If I can get the proper parts on site by then, I will probably be doing maintenance on the old server tomorrow (Wednesday) morning. If all goes well, we'll be cutting over onto the new server on Thursday morning. Thursday will probably be a little rocky as we fight fires and fix the odd bug. I think we'll be completely back to normal and on a much faster server by this weekend. Sorry for the mess. And does anyone have any virtual java? Have been running on minimal sleep for a few days, and probably have to keep doing that for a few more days. 6/02/98 (later) (B) Turns out I'm not ready to go yet with repairs to the server. Number of things are going wrong. ): 6/02/98 (still later) (B) Karen and I are down in L.A. shopping for parts. Be back this evening. 6/03/98 (B) We had all the proper parts on hand by yesterday afternoon. However, we've been having problems with AOL rejecting mail for all its subscribers as "user unknown" and, as an added bonus, the folks from whom we buy our listservers' T1s have had a sick router in their shop and our listservers have been sporadically losing their Net connectivity. So, courtesy of AOL and our upstream, we've had really sick listservers and we've been too busy to work on the Web server. I hope to begin repair work on the old Web server this afternoon. If we're lucky, we may be able to give the appearance of fully normal operations by the end of the day (Wednesday). We will possibly cut over onto the new server on Friday, but we'll probably wait until after the weekend if the old server seems to be operating adequately. Wish me luck on the repair work ... (: What a week, B. 6/05/98 (B) A NEW SERVER IS BORN. We obviously still have stability problems on the old Web server. I'm in the process of replacing all the adapter boards, with especial prejudice concerning the SCSI disk controller. Regardless of whether I can get the old server to stay up, we'll have the new server online within the next couple of days. That should fix the problem in a permanent sense. 6/07/98 (B) A NEW SERVER IS BORN. The new server first booted on Friday night, and after working on it all day Saturday we attempted to bring it online Saturday night when the old server crashed. And we failed. Failed because it turned out we had to do an OS upgrade on the new server to handle some of its hardware. So last night we did some additional tunes on the old server, and we're still limping on it. Server-side includes are still disabled. Today I've done the OS upgrade on the new server, and everything seems to be working great. Barring crashes of the old server, we will do a planned upgrade to the new server Monday morning. After that you can expect some anomalies in our operations, because major OS upgrades always introduce some interesting bugs. Nonetheless, we expect to be returning to normal operations by Monday evening. 06/08 (D) By gosh, I think I have it working a lot better then before. Check It does need a bit of touching up on the form but it works GREAT! 06/09 (B) For those of you who don't spend every second of your life watching the MRTG graph for the Web server's port at DataCourse, we've just hit an all-time high of 247.0 KB/s outbound from that box. That's about 1.25 T1s being filled by WWW alone. And that's *after* Dale moved usgenweb.net/org to homepages. (: Longterm load average on the Web server is 7.64. So far, no signs of kernel problems or instabilities. I think we may have hit a home run with the new server. 06/13 (B) We have a new STAFF-Ler: Randy Winch . Randy did most of the hacking on the wildly successful GenConnect Project (other team members were Pam Carey, Carol Hepburn, and Nancy Trice). He's also the hacker who sort of saved our bacon by rewriting the Webtools counter into C, and who has recently implemented a graphical and user-resettable counter for the local Webmasters. At last report Randy was hiding under his bed in response to the list of chores I dumped on him but, seriously, I expect he'll be working mostly in the cgi and spider world. This is sorta who else is on STAFF-L, in alphabetical order: Julie Case and Myra Vanderpool Gormley, editors of the RootsWeb Review/RootsWeb Spokespersons Karen Isaacson, RSL Database/ROOTS-L Pages Maintainer Brian Leverich, Server Assembler/Janitor/Beggar Tim Pierce, List/Mail/News Administrator Dale Schneider, IRC/Mirrors/Search Engine/Web Administrator None of the job descriptions really covers everything we do -- there are so few of us that essentially everyone is responsible for everything in a pinch. 06/14 (P) I've added Ellen Seebacher , also known as my better half, to STAFF-L. Ellen is likely to do a lot of the list creation and netnews handling for the forseeable future. She also is crackerjack with a blue pencil. :-) 06/14 (B) Hi Ellen! I'm glad to see you here -- K & I had been hoping we could rope you into the REPTAWD. Everybody, if you don't know Ellen already, she's another Internet genealogy and netnews oldbie. Formerly newsmistress@uchicago.edu, and way cool to work with. ***************************************************************** HUMOR: A Mac user we know contributed these messages that allegedly are under consideration for Windows 2000. 1. Smash forehead on keyboard to continue. 2. Enter any 11-digit prime number to continue. 3. Press Ctrl-Alt-Del now for IQ test. 4. This will end your Windows session. Do you want to play another game? 5. Error saving file! Format drive now? (Y/Y) 6. BREAKFAST.SYS halted. Cereal port not responding. 7. File not found. Should I fake it? (Y/N) 8. Runtime Error 6D at 417A:32CF: Incompetent User. 9. User Error: Replace user. 10. Your hard drive has been scanned and all stolen software titles have been deleted. The police are on the way.