Bad Behavior 2 Roadmap Update
February 2nd, 2006 by Michael Hampton
Updated with new information and — by request — a tentative timeline.
As many of you are no doubt aware, there’s a new class of automated spambots out there which Bad Behavior and other spam tools don’t yet handle. Spammers have indeed adapted their techniques to get past tools such as Bad Behavior, Spam Karma, and Akismet, and are actually succeeding. I first caught wind of this new generation a few months ago, and began working on Bad Behavior 2, my attempt to deal with the new generation of malicious spambots.
I had hoped to have a final release long before now, but various problems cropped up and prevented me from completing the project. To date I’ve only been able to release a very early alpha so that other software authors who write code that depends on Bad Behavior can begin to update their tools. The alpha, while it’s functional, not only requires WordPress 2,0, but also provides no more protection than the current release code. In fact, it provides a bit less because one of the checks in Bad Behavior 1 is not present (right now).
As I’ve heard from every such author and they have either updated their code or have plans to do so, it’s time to move this forward.
A representative from a major open source project informed me that the project would be willing to contribute financially to Bad Behavior, but wanted to ensure that it would get something in return, and have a better idea of the timeframe of development. Thus I’m updating the previously posted roadmap.
I have the basic structure of Bad Behavior laid out to allow Bad Behavior to drop in much more easily into packages such as DotClear and Geeklog, where the plugin architecture is quite different than everything else. This will also allow Bad Behavior to be ported to even more software packages. It consists of two components: a core consisting of the test suite itself, and a glue component for each host platform. I’m also planning an administrative interface that will hook into each host platform, though I am not sure if this will be ready for all platforms at the time of release. Finally you’ll be able to configure Bad Behavior and view its activity within WordPress, MediaWiki, or whatever platform, using the native administrative interface provided by the host platform. This is the largest design change in version 2. (Estimated timeframe: 8 development hours per platform.)
Bad Behavior’s API needs improvement. It started as a simple generic interface, and has already outgrown that interface. Version 2 will feature a completely redesigned API for integration into the host PHP program, offering more flexibility, and hopefully the ability for the host program to provide services to Bad Behavior, such as statistics and log viewing.
Bad Behavior needs to deal with the database more intelligently. In version 1, I kept a log of requests which had been denied, expanded it to optionally include all requests, and expanded it again to include the reasons for denial. Then I started using the information in the log to make decisions. Version 2 will feature a complete redesign of the database table, and expansion into two tables, one strictly for logging (for you to stare at), one strictly for making decisions. I expect to gain significant performance improvements thereby, as well as being able to make more intelligent decisions on which requests should be allowed and which should not be. (Estimated timeframe: Complete.)
Most legitimate users unfortunate enough to see the Bad Behavior error page have no idea what to do, even though the page does provide suggestions. It needs to be shortened, clarified and contain links to expanded information sources so that users can solve the problem on their own whenever possible. It should also customize the message based on the specific reasons for denial. Though the ideal is that Bad Behavior should never present the page to a legitimate user — only to spammers. The architecture is in place for Bad Behavior to show more informative error messages, each one including a unique key which either the user or the blog admin can look up to determine what went wrong and how to fix it. While all of the keys have been set, the documentation for each remains to be written. Bad Behavior will now serve errors such as 400 and 403, depending on the request, rather than 412. (Estimated timeframe: 12 development hours.)
Bad Behavior needs to provide better tools for site administrators to search for and eliminate any false positives that may arise. While version 1 contains whitelisting capability, it’s not easy for a site owner to determine why a particular request was blocked, due to being unable to find it in the logs. As I mentioned above, Version 2 will provide a unique key to each denied request which the site owner can use to immediately find the problem, if any, and take any necessary corrective action.
In addition, in certain circumstances when the access is suspicious but it can’t be conclusively shown that the access is malicious, Bad Behavior 2 will attempt to clear the access using as-yet unreleased methods, only providing an error message if the access can’t be confirmed as a human being through multiple techniques. (I would say more on these, but I don’t want to give the spammers a head start on me!) (Estimated timeframe: 6 development hours.)
And I’m experimenting with automated methods of detecting spam attack runs which may originate from dozens of different IP addresses and have somewhat different signatures. I may call for some assistance with this in the near future, and this isn’t likely to make it into 2.0, but it is in the works. (Estimated timeframe: 75-100 development hours.)
Finally, Bad Behavior must continue to keep up with spammers as they attempt to adapt and find new ways to post their automated garbage. To date, this has been at most a minor issue, as there is only so much the spammers can do, while maintaining their high rates of spamming (10,000 or more posts in a single run is not unusual). Bad Behavior attempts to drive up the cost of link spamming, by blocking as many of those spammy requests as possible, forcing the spammers to resort to MUCH slower manual methods, or ideally, give up and find more honest work.
This is my vision for Bad Behavior 2.
Bad Behavior is open source software, released under the GNU General Public License, which you can find copies of all over the Internet, or included with the program. You don’t have to pay a cent to download or use it. However, developing it still costs me time and money, which is why it can go so long between minor releases. Unless (until) some cash comes in, it doesn’t get updated except in cases of dire emergency. Which only happens if I ship code with a typo in it, or Microsoft changes their search engine, or something like that.
Complicating the issue is the fact that a month ago, my laptop, which was my main development platform, died a horrible death. So I’ve had to suspend development of Bad Behavior as well as some of my blogging and other work, until I can get a replacement laptop and get it up and running. As of now I’m about $250 short of where I need to be to have a laptop which is inexpensive and yet capable of serving as a development platform. The computer I’ve been borrowing for the last month or so is simply not sufficient to do much with, unfortunately, and all of my paying endeavors have suffered for it.
If you think this roadmap looks good, and want to accelerate the development of Bad Behavior, contribute financially and I’ll be able to devote more time to it, meaning version 2 comes closer to reality sooner.
And for those of you who are concerned about actually getting something back, my commitment to you is for every $25 contributed, whether from one person or multiple people, to complete one hour of development work within the seven days following the contribution, even if I have to put something else on the back burner temporarily. (Since I generally charge roughly double this amount — or more — for most paid development work, you’re getting something of a bargain.) I hope to have a feature-complete beta working on WordPress, MediaWiki and ExpressionEngine within 45 days, and a final 2.0 release within 75 days, and with your help, this may actually come to pass.
And by all means, if you think I left something out that should be in version 2, please let me know. And yes, I know a lot of you are flat broke, so even if you are unable to contribute financially, leave a comment. Say hi, or suggest changes, or something, just so that I know you’re there and you think I should continue this project.