How Bad Behavior Works
It’s black magic.
Bad Behavior manages to block nearly all link spam without ever looking at the spam. While it might be useful to do so, for performance reasons, Bad Behavior does not analyze received spam. I’ve found that this way lies madness; spammers are constantly buying new domain names, so it’s possible to miss a lot of spam by looking at it.
Instead, Bad Behavior pioneered an HTTP fingerprinting approach. Instead of looking at the spam, we look at the spammer. Bad Behavior analyzes the HTTP headers, IP address, and other metadata regarding the request to determine if it is spammy or malicious. This approach has proved, as one user said, “shockingly effective.” After all, spammers write their bots on the cheap, and have little incentive to code very well. If they could code very well, they probably wouldn’t be spammers.
When Bad Behavior looks at a request, it determines if the request matches a profile of known malicious or spammy activity, and falls outside the bounds of a normal human browsing the web. If so, the request is blocked. But a way out is provided for any human beings with unusual configurations or viruses/Trojans on their computer who may be blocked.
From the start, Bad Behavior has had two overriding design requirements. The first is that it must be fast. Users will get annoyed by waiting around for their traffic to be screened for spamminess. (Is that a word?) Especially since Bad Behavior screens all requests in order to block email harvesters and certain malicious robots, speed is paramount. I’ve had to abandon good ideas because they would add significantly to Bad Behavior’s run time, which is typically measured in milliseconds, and can be cut to hundreds of microseconds for very high traffic sites.
The second requirement is that it must block as few legitimate users as possible, and when one is blocked, they must be able to unblock themselves through an action simple and fast enough that they can simply hit the browser’s reload button once they’ve completed the action. Bad Behavior provides a technical support key to each blocked request which allows the requester, if it’s a legitimate human being, to get immediate, self-service support to fix the problem (e.g. virus removal, change of browser preference, etc.) and go back to browsing. Out of countless millions of requests served daily, an average of 50 people use the technical support system, and virtually all of those resolve the problem themselves in under five minutes.
