I first found this story on Netcaft about 2 weeks ago and couldn’t decide if it was worth a mention here. I’ve just made a decision in that regard, mostly due to the evidence that RSS is becoming the “new email newsletter”.

Basically, an RSS reader is a program that when installed and configured, will fetch RSS documents from the websites it has been set to visit. The RSS itself is usually story titles, links and story summaries. Now RSS is just starting to get very popular, and RSS readers sometimes fetch stories on the hour, by the hour. The result of 10,000 RSS readers all trying to fetch an XML file (which is what RSS really is) from one site at the same time, could be compared to a DDOS (Distributed Denial Of Service) attack, like those attacks that have been in the news over the last few years against companies like Amazon and Yahoo, where the server is answering so many requests that it can’t handle any new ones and the site goes down.

HTMLfixIT both uses, and offers RSS feeds, for example the Sophos virus information on the front page is fetched via RSS from their site. (Instead of doing it in real time, a Perl script I wrote runs from Cron [which is sort of like Windows Scheduler for Linux] and it fetches the RSS file from Sophos every two hours if it has changed since it was last fetched. The reason I used a Perl script instead of just using PHP to pull the RSS directly from the Sophos is pretty simple, Speed. It is considerably faster if the Perl script has already fetched the file, and the PHP reader only has to read the local copy. The other reason was bandwidth, it seemed silly to fetch the results from Sophos every single time that somebody visited our site. [which happens quite a bit you know. ])

Anyway the point of the story, is that nearly all RSS readers will update their feeds on a scheduled basis, usually on the hour. To help the sites that you patronize, you should configure your reader to do that update at a random time each hour, not on the actual hour, this way you can lighten the load on the servers hosting your favourite sites. (I should clarify what I mean when I say random, what I mean is that you should just pick any time, like say every hour at *.46 so that at 46 past the hour, the RSS is updated, by having every one using different times, the load on the servers is spread out and much easier to handle.)

Regards

Franki

This article was posted on Tuesday, July 27th, 2004 at 10:57 pm