+++*

Symbolic Forest

A homage to loading screens.

Blog

Performance

In which things turn to treacle

I’ve noticed, over the past few months or so, that sometimes this site seems to load rather slowly. The slow periods didn’t seem to match any spikes in my own traffic, though, so I didn’t see that there was necessarily much I could do about it; moreover, as it wasn’t this site’s traffic that seemed to be causing the problem, I wasn’t under any obligation to do anything about it.

As I’ve mentioned before, a few months back I switched to Google Analytics for my statistics-tracking. Which is all well and good; it has a lot more features than I had available previously. Its only limitation is: it uses cookies and Javascript to do its work. Because of that, it only logs visits by real people, using real browsers,* and not spiders, robots, RSS readers or nasty cracking attempts. Often, especially if you’re a marketing person, that’s exactly what you want. If you’re into the geekery, though, it can cover up what’s exactly going on, traffic-wise, at the server level.

Searching my logs, rather than looking at the Google statistics, showed that I was getting huge numbers of hits for very long URLs, consisting of valid paths joined together by lots of directories named ‘&’:

Logfile extract

That’s a screenshot of a single request in the logfile – the whole thing being about 850 characters long. ‘%26′ is an encoded ‘&’ character. Because of the way WordPress works, these things are valid URLs, and requests for them were coming in at a pretty fast rate. Before long, the request rate was faster than the page generation time – and that’s when the problem really starts to build up, because from there things snowball until nobody gets served.

All these requests were coming from a single IP address, an ordinary consumer type of address in Italy.** Moreover, the user-agent was being disguised. Each hit was coming in from the same IP address, but with a different-but-plausible-looking user-agent string, so the hits looked like a normal, ordinary browser with a real person behind it.

The problem was solved fairly easily, to be honest; and the site was soon behaving itself again. It should still be behaving itself now. But if you came here yesterday afternoon and thought the site didn’t seem to be working very well, that’s why it was. I’m going to have to keep an eye on things, to see if it starts happening again.

* and only if they have Javascript enabled, at that, although I know that covers 99% of the known world nowadays.

** which made me think to myself: “I know I’ve pissed people off … but none of them are Italian!