You may have noticed that I’ve been “closed for business”, so to speak, the past couple of days. Let me explain and beseech the community to help me get to the bottom of this.

The day before yesterday I found my hosting account had been suspended due to me using too many server resources. Upon pleading and promising to research what happened I got my account unsuspended and I pulled up my logs. What I noticed was… odd. I initially thought it was some kind of DoS attack, but it really didn’t look like it. I saw someone visit a page — unfortunately my logs are currently gone — then start requesting all of my monthly archive pages at a fast rate, re-requesting them when they didn’t get the data fast enough. Apache was returning a 200 success code at them, but apparently they weren’t getting their data fast enough as the kept re-requesting. That was when the account was suspended.

Yesterday at about the same time my account was suspended again. The pattern was about the same. Someone — from General Electric, apparently — visited me from Google. The requests were perfectly innocuous. About 12 seconds following the final ordinary request, the same IP initiated a flurry of requests for all of my monthly archives again. What occurred to me this time was that all of these archives were only linked via <link rel="archives" ...> tags in the header of the page. So, something was fetching/prefetching all of those links programatically at a high rate of speed.

I’ve heard of Google prefetching under certain circumstances, but only for <link rel="prefetch" ...> or <link rel="next" ...> tags. This makes me think that someone is trying to one-up Google and accelerate browsing even more, but they’re doing it in a particularly hamfisted way. I’ve seen a couple of add-ons for IE6 — Browster and something called KybIE GetEmAll — that will prefetch pages for you but I have yet to catch them pulling the links from this type of link.

That’s where you come in. Do you know of an Internet Explorer add-on or some wacky version of IE6 that performs prefetching to speed up access? If so, please post in the comments about your experiences so I can investigate. I’ve since removed those (useless) link tags from my header, so I think I’ll be okay. I’ve also removed some features — you’ll no longer get “welcomed back” to the site and have an indication of how many and which posts and comments are new — and I’m trying a static caching plugin to boot to try to reduce my overall burden on the server.

However, this is literally my last chance. My hosting provider has assured me that if I get suspended again because of this behavior, I’m gone. And I don’t have the desire or the cash to pay for a more robust hosting solution. Those of you that may recommend other hosting solutions — and I would love to hear your suggestions, believe me — please bear in mind that my requirements are 1G storage, at least 10G of traffic to allow for growth, multiple MySQL databases and a cost that’s not too much more than $5 a month… if it ain’t got all that I ain’t interested.

Thanks for your assistance and patience.

Addition: If Else has a wonderful writeup on the perils of prefetching. Well worth the read.

This entry was posted 2 years, 11 months ago and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.