I’ve noticed, over the past few months or so, that sometimes this site seems to load rather slowly. The slow periods didn’t seem to match any spikes in my own traffic, though, so I didn’t see that there was necessarily much I could do about it; moreover, as it wasn’t this site’s traffic that seemed to be causing the problem, I wasn’t under any obligation to do anything about it.
Searching my logs, rather than looking at the Google statistics, showed that I was getting huge numbers of hits for very long URLs, consisting of valid paths joined together by lots of directories named ‘&’:
That’s a screenshot of a single request in the logfile – the whole thing being about 850 characters long. ‘%26′ is an encoded ‘&’ character. Because of the way WordPress works, these things are valid URLs, and requests for them were coming in at a pretty fast rate. Before long, the request rate was faster than the page generation time – and that’s when the problem really starts to build up, because from there things snowball until nobody gets served.
All these requests were coming from a single IP address, an ordinary consumer type of address in Italy.** Moreover, the user-agent was being disguised. Each hit was coming in from the same IP address, but with a different-but-plausible-looking user-agent string, so the hits looked like a normal, ordinary browser with a real person behind it.
The problem was solved fairly easily, to be honest; and the site was soon behaving itself again. It should still be behaving itself now. But if you came here yesterday afternoon and thought the site didn’t seem to be working very well, that’s why it was. I’m going to have to keep an eye on things, to see if it starts happening again.
** which made me think to myself: “I know I’ve pissed people off … but none of them are Italian!”