Open Source Pulls Together to Cover September 11th Attacks

Community
by John D. Biggs

Tuesday, September 11, the Internet stood still. Millions of users claimed over 80 per cent of internet bandwidth and caused sites around the world to choke on requests for information. As larger web sites like CNN.com and MSNBC.com fell under the pressure, smaller sites–some run out of bedrooms and failed dot-com offices–kept news and information flowing, proving that the Internet is a resilient beast.

As the importance of the Internet has grown, the value of smaller sites providing news and opinions has become more apparent. The broadcast model of information distribution created by radio and television no longer holds in today’s networked world, and one site in particular, Slashdot.com, proved this admirably. The site survived the onslaught of users and served up-to-the-minute information during the glut, even as other sites fell by the wayside.

Slashdot was founded in 1997 by Rob “CmdrTaco” Malda, 25, who started the site to post technical information while a student at Hope College in Holland, Michigan. Since then Slashdot has grown to serve almost a million pages daily on topics as diverse as new video games, privacy and freedom.

“We’ve always thought of Slashdot as the pub of the techie world; it’s a place where people come and can talk about all of the issues”, said co-editor Jeff Bates, 25. “Rob and I are the barkeeps–we try to minimize fighting between the people and keep the discussion interesting.”

When the editors of Slashdot activate a new post, readers often overrun and crash web sites mentioned in the text as a result of their exuberance, a phenomena called “Slashdotting”. On September 11, it seemed as if the whole Internet had been Slashdotted. Servers disappeared. News and mail could not travel over the web as nodes dropped out due to high usage.

Yet tiny Slashdot continued to serve up information on the disaster, including mirrors of images taken from mainstream sites like CNN.com and NYTimes.com. Eye-witness reports rolled in, and news that stagnated at larger web sites appeared immediately on Slashdot message boards.

“We were just getting slammed”, said programmer Cliff Woods. “Slashdot was handling three times its usual peak traffic. [We] went from keeping no more than 50-70 kilobytes of comments [about 100 typed pages] in its database to holding over two million comments.”

As more news and e-mails poured into Slashdot’s bins, Malda and Slashdot editor Jeff Bates decided to dedicate space to the unfolding tragedy.

“[Malda] was listening to Howard Stern who was covering [the crash]”, Bates said. “At that point, he decided that this was a bigger deal than anything else, and that we had to do something on this. Something like this was much larger then anything the tech world was doing.”

As the news rolled, Malda posted a quick apology for disrupting the normal tech topics and then continued to post information about the attacks.

“Normally I wouldn’t consider posting this on Slashdot”, he wrote after the first tower fell, “but I’m making an exception this time because I can’t get news through any of the conventional web sites, and I assume I’m not alone.”

As dependence on the Internet increased during the first few hours of the attacks, Slashdot maintained its web site by shutting down the automatic Perl systems that posted information and creating web pages by hand.

“Normally Slashdot serves pages to users at a maximum rate of about 20 per second, but by 10 AM Tuesday the server load was spiking as high as 40 pages per second, approximately double normal capacity”, said Robin “Roblimo” Miller, editor-in-chief of OSDN, the company that hosts the Slashdot site.

Internet traffic fell over time and Slashdot returned to normal, posting news about an increased capacity disk drive. However, the lessons learned were clear, and the goal of the original Internet, the ARPAnet, rang true: to be a network that routes around damaged nodes, allowing users unfettered access to the breaking developments, even when the media giants are struck down.

John D. Biggs is a writer and consultant in Brooklyn, New York.