Skip to:
Content
Pages
Categories
Search
Top
Bottom

Here come the spammers!!!

  • Avatar of foxly
    foxly
    Participant

    @foxly

    Seven spam PM’s. From five different users. In the past 24 hours.

    And they’re getting clever too …they’ve found a defect that allows them to delete the backlink to their profile from the message, making it harder to come after them.

    I was always shocked by the lack of spam control features on BuddyPress, and I’ve been amazed that nobody has been attacking BP installations …given that with under 1000 lines of code its possible to write a bot that posts billions of spam PM’s, forum posts, and splogs daily to practically any BuddyPress site.

    Well, it looks like somebody has written that bot, because the posts are definitely at a volume where its being automated.

    If we don’t get this nailed ASAP, we can probably look forward to crippling attacks against pretty much every BP site out there within a matter of weeks.

    So I have a proposal: If the core devs give me permission, I can take a few days off working on BP Album+ and write a patch (not a plugin, as it requires mods to the core) that deals with this problem.

    @apeatling @r-a-y @DJPaul @21cdb @johnjamesjacoby

    What do you think?

    ^F^

Viewing 21 replies - 26 through 46 (of 46 total)
  • Avatar of foxly
    foxly
    Participant

    @foxly

    PART 3 – STRONG -vs- WEAK METHODS

    When it comes to spam on BP sites, you’ll see all sorts of stuff posted on blogs saying “change [whatever] on your site and your spam problem will disappear”.

    Truthfully, a lot of these tricks will actually work …for a while… but eventually, the spammer makes a minor change to their bot, and they’re back in business. In fact, many of the leading blog spamming packages include sophisticated logging features to catch the errors that “uniquely configured” blogs generate and help the spammer quickly fix the “problem”.

    If we’re going to have a reliable anti-spam solution for BuddyPress, we should probably focus on “Mathematically Strong” methods, not on “Obfuscation” and “Moving Things Around”. That way, we won’t have to constantly change our spam protection methods.

    Changing Page Slugs

    Many people recommend changing the page slugs on BP installations to reduce spam. While this is certainly easy to do, you of course need to give your users *links* to those page slugs somewhere on your site so they can actually visit the pages. And if users can follow the links, so can a spam bot.

    Changing page slugs is kind of like boarding-up the front door of your house, installing a new door in the side of your house, and then attaching a piece of string from the front door to the side door of so everyone can find the new door.

    The “change your page slugs” approach seems to come from the “change your admin menu URL” technique. Changing your admin menu URL is actually a *strong* protection technique. Since there is no link to it anywhere on the site and you’re the only one that knows the URL, it’s like having two passwords on your admin login. An attacker would have to try billions of URL’s to find it.

    Not so with all the other URL’s on your site. They have to be linked off other pages so your users can find them.

    Adding Fake Form Fields

    Many people recommend adding a few extra fields to forms throughout your site (sign-up, login, post to group, etc) and “hiding” these fields using CSS. If any of the “trap” fields are filled out, in theory, you’ve just detected a bot, because a normal user would never see the fields and fill them out.

    This approach *might* defeat a very simple bot that searches every web page it can find for forms, and fills every field in every form with random spam; but it will not defeat a bot that understands CSS or is specifically targeted at BuddyPress, especially considering that BuddyPress is *open source*.

    Don’t think bots can analyze CSS? Read this: http://www.google.com/support/webmasters/bin/answer.py?answer=66353

    A bot designer can simply read through the BP source code and discover the names of the fields that should be filled in and the names of the fields that should be left empty.

    To use our “house” analogy, adding extra form fields is like installing 3 front doors on your house and rigging two of them with grenades …then hanging a big red “out of order” sign on the the two rigged doors so your friends don’t use them.

    Obviously if your friends can read the signs, so can your enemies.

    JavaScript Proof of Work

    Javascript proof of work (Wp Hashcash) defeats spammers by making visitor’s web browsers solve a math problem in JavaScript before they are allowed to post.

    Because everyone knows spam bots can’t run JavaScript.

    http://forums.digitalpoint.com/showthread.php?t=1124949
    http://www.scrapebox.com/
    http://blogcommentdemon.com/
    http://www.senuke.com
    http://www.botmasternet.com/more1/

    Except when they can. ;)

    There’s also the issue of what to do with visitors that don’t have JavaScript enabled.

    The WordPress and BuddyPress development teams have put an epic amount of work into ensuring both platforms will work reliably when JavaScript isn’t available. Requiring users to have JavaScript to post any kind of content to the site nullifies much of this work.

    Proof-of-work was a great idea back in 1997 when spammers ran hundreds of attack threads from a single server and solving the JavaScript math problems slowed it to a crawl.

    In 1997, we’d be dealing with a single spammer running 1000 attack threads against the site. Because the spammer was running 1000 threads, each of which would have to solve the JavaScript problem, they would effectively be penalized 1000 fold over a normal user. The end result is they would only be able to run a few threads before their computer slowed to a crawl and their spamming abilities would be sharply limited.

    Epic win for site.

    Unfortunately, things are different in 2010.

    Spam bots have become the tool of choice for basement SEO marketers. Instead of a few members of the “spam elite”, we’re dealing with tens of thousands of “do it yourself” spammers each running 1 attack thread using the new “automatic backlink software” they just picked up for $29.00 off some random SEO website. Instead of fighting one spammer splitting their resources across a thousand threads, we’re fighting a thousand spammers running a single thread dedicated *just to our site*.

    Skipping a ton of math, what this means, is that in order to cause a spammer a 1-second delay while their computer solves our JavaScript challenge, we have to cause each of our *legitimate users* a 1 second delay while *their* computer solves our JavaScript challenge. And, considering the 3 to 5 second database lag I see on 90% of the BP sites I visit, the challenge would need to take much longer than a second to have any merit at all …otherwise page refresh time would be the limiting factor, not the JS challenge.

    So what happens when a user visits the site using a computer that is much slower than a typical desktop …say a mobile phone or an old laptop? The challenge would take proportionally longer to complete. A challenge that requires 5 seconds to solve on a desktop PC, could take 30 seconds on an iphone …and 30 second response times would not make for an enjoyable user experience.

    Overall, proof-of-work challenges are probably not a good choice in the 2010 Internet landscape.

    Mathematically Strong Methods

    In the next post, I’ll cover the specific details of the methods I’ve proposed for the BP spam solution, and why they will defeat most spam attacks.

    ^F^

    Avatar of stwc
    stwc
    Participant

    @stwc

    Good on you, foxly. There are a lot of significant problems with BP (made more apparent by the new design we’re using here on buddypress.org, which will, hopefully, mean that they are addressed more quickly), and it’s great to see someone from the dev community step up with substantive proposals to address one of the most glaring ones.

    Avatar of Arx Poetica
    Arx Poetica
    Participant

    @arxpoetica

    Yow, I this is a very satisfying thread. I’ve been so frustrated in the past by cursory deflections from long-time WP gurus (I won’t name names, ha), when it’s been clear for a long time that WP (and now BP) has very weak defenses, and yet seems to have the type of community that could build something very robust and strong. That being said, I have also thought that BuddyPress, by it’s linked nature, provides a greater threshold for working on this type of problem. To use a buzzword, it’s “synergy.”

    I’m glad somebody has finally taken up the mantle.

    (Sorry for ranting for a moment there, but it’s true!)

    Avatar of foxly
    foxly
    Participant

    @foxly

    I think, overall, the core devs have approached BP development in the most effective order possible.

    There really isn’t much point in adding spam protection to a platform nobody uses, and nobody would be using BP if the core developers had spent the past two years hardening it with spam protection instead of adding member-centric features.

    But we’ve hit “critical mass” now …and we have to deal with the spam problem before BP is ready to move from a science experiment people use on hobby sites to a “platform” that developers can use for serious commercial endeavors. And I’m willing to throw a sizable chunk of dev time at it to help make that happen.

    Anyhow, at this point I’ve written the first draft of the proposed changes to the forum, and what I need now is everyone to read through it and post their feedback.

    Seriously.

    Because if something in the proposal doesn’t work for your application …or you can think of a better way to do it… you’d better get a post up NOW before we start writing code.

    Thanks!

    ^F^

    Avatar of Arx Poetica
    Arx Poetica
    Participant

    @arxpoetica

    Btw, I really didn’t mean to dig on *anyone* –> I think WP and BP is BRILL, to say the least, but I do get frustrated @ how spammy it all can be. THAT SAID, I should just keep my mouth shut from here on out (ha), since I’m not really contributing to the solution.

    SO.

    @foxly I still need to read through all your suggestions, but the few I did read seemed spot on.

    I do think it’s important to note: this thread becomes documentation for would-be spammers. Sad to say, as @andrea_r has noted many a time, when it’s documented, the evil-doers will read it and then find a way around. So how does your solution work in spite of the documentation herein? Or do we need to take some of this offline, into a more hushed conversation, i.e., email (gasp!) or some other less porous document? Google docs? :)

    Avatar of PJ
    PJ
    Participant

    @pjnu

    There are some good exhaustive posts about spammers on this site… but what will change from the server level? I have gotten a few spam messages too and there’s nothing I can do except for delete them. Could the site setup a recaptcha required box before sending messages? Would that be effective?

    If Buddypress can create a proactive plugin or core feature to mitigate spam that many of us are receiving, it will improve our confidence in the service (privacy and security is a common mantra in the forums). If buddypress.org finds a plugin works well, please share which one it is in a sticky post so we all see it and it won’t get lost between many posts. When there are multiple suggestions to a specific problem then it shows that there really isn’t a solution at this time. Thank you.

    Avatar of zageek
    zageek
    Participant

    @zageek

    I must say the SPAM problems are real pain, and for a long time people have been complaining about the SPAM and its been brushed off. I have laid off the complaining and tried to be proactive to fight the spammers off but its really hard to keep up with them.

    I think the admins and developers need to start a working group or something and start giving this a lot of attention because if they remain nonchalant about it, it will eventually work enough people’s nerves to chase them away to use some other solution.

    The other day I was talking to someone who played with BP a while ago and then dropped it in favour of another solution after the sploggers started coming in.

    I have created a group called SPAM Eater where members who are pissed off about SPAM can gather and vent our frustration a bit

    http://buddypress.org/community/groups/spam-eater/

    Avatar of r-a-y
    r-a-y
    Moderator

    @r-a-y

    This thread should be developer-focused and less “why aren’t you guys doing anything about it?”.

    I don’t want to start moderating this thread, but please keep comments focused on combatting spam instead of bickering about it.
    We are all aware of the problem.

    Avatar of Arx Poetica
    Arx Poetica
    Participant

    @arxpoetica

    Has there been any more movement on this front?

    interesting:


    2) How do spammers find BP communities?
    Using Google.
    Example: http://www.google.ca/search?hl=en&q=%2B”is+proudly+powered+by+WordPress+and+BuddyPress”; (front page of every BP site on the net)
    Example: http://www.google.ca/search?hl=en&q=inurl:%22/community/members/%22+%2Bbuddypress (members page of every BP site on the net)

    I have two open test BP installs (one wp_user linked to my blog domain) and changed the standard footer. Not a single spam sign-up (just hashcash installed)

    Avatar of Hugo
    Hugo
    Moderator

    @hnla

    It’s my belief that the footer link plays a bigger part in the problem than is given credit for :) at least it was the last thing I implemented (removing) and appeared to have a significant impact. One thing I think people should stop putting forward is this notion that adding custom fields to sign up makes a difference, it doesn’t! and is obviously and emphatically demonstrated by the fact that spam bot signups manage to fill all fields with garbage including user created ones but as you would expect.

    Avatar of Jeff Sayre
    Jeff Sayre
    Participant

    @jeffsayre

    The issue of spam will be one of the topics discussed at this week’s dev chat on Wednesday. There should be more to report after that chat.

    Avatar of Arx Poetica
    Arx Poetica
    Participant

    @arxpoetica

    Awesome.

    Avatar of Peter Kirn
    Peter Kirn
    Participant

    @peterkirn

    Hi Jeff, I can’t make the chat Wednesday as I’m going to be on a plane between London and Hamburg, but I wanted to add to this:

    1. wp-recaptcha — I’m working with the developer of this plug-in so that we have one fork that works everywhere, BP included. Given that this is the topic, let me try to get that basic code up. Even with simple recaptcha support, there’s a huge decrease in spam signups. It seems not to solve the smartest scripts, the ones that send PMs (at least not on our site), so I think once we get one recaptcha working, making the “failed” recaptchas more intelligent to avoid these automated bots would be great. Thanks for the ideas above — this is great fodder — so I’d encourage people to get involved on the same fork so we can put this into action sooner rather than later. Let me post a separate update within the next couple of days.

    2. Since PMs are a big problem, and this thread is getting very, very ambitious, why not at least begin testing this with a separate plugin? I’d like to at least see something that stops mass-mailings and highlights that user, as that’d be an easy way to weed people out, at least as more comprehensive solutions are developed.

    3. Reviewing core is probably worthwhile. A mistake in bp_signup_validate’s code was being exploited by hackers. I know this is part of 1.2.4, but I went ahead and applied the diff attached to this (now-closed) ticket to our current 1.2.3 install:
    http://trac.buddypress.org/ticket/2289
    – this made a big difference. I wonder if anything else follows this pattern, and how we might hunt it down.

    Grand, wide-reaching plans sound terrific, but I’d hate if that derailed some short-term fixes; seems we can have both.

    Avatar of thelandman
    thelandman
    Participant

    @thelandman

    This is why we need to intercept spammers. This is a the first thing I saw when I logged in!

    http://img337.imageshack.us/img337/3872/blahg.jpg

    Avatar of Arx Poetica
    Arx Poetica
    Participant

    @arxpoetica

    @jeffsayre & @foxly it’s been a while since I visited this thread, but I liked the momentum on it. What was the consensus at the dev chat? What’s the stat on this item? Can I help in any way? :D

    Avatar of foxly
    foxly
    Participant

    @foxly

    @arxpoetica

    Basically, nothing happened.

    I sense that there are some politics at play here, so I’m going to keep my head down. When spam becomes a problem on the sites I run, I’ll develop my own solution.

    ^F^

    Avatar of Tom(dB)
    Tom(dB)
    Participant

    @tomdebruin

    @foxly, even though your solutions require code modification you do say “i’ll develop my own solution” – will these be plugins you release, or perhaps become part of the bp-moderation plugin that is being developed by @francescolaffi?

    I really appreciated your detailed posts on the tools spammers employ. It has given me a much better understanding of the problem. Thank you.

    Avatar of foxly
    foxly
    Participant

    @foxly

    @tomdebruin

    Oh I’m sure @foxly & @francescolaffi development inc will come up with something. It’s just that not having it part of the core makes much harder to develop, because frankly, BuddyPress doesn’t currently have many useful hooks for plugins to manipulate the core. So you have to hack the system in all sorts of crazy ways and that can really slow down the site.

    And then there’s the whole issue of *where automattic is taking buddypress” and what they intend to do with it, now that they’ve taken Andy off as the lead developer.

    EDIT: what I meant to say was “now that Andy is not working *full time* on BP anymore, as automattic has split his time between other topics” – my bad :(

    ^F^

    Avatar of @mercime
    @mercime
    Keymaster

    @mercime

    @foxly: And then there’s the whole issue of *where automattic is taking buddypress” and what they intend to do with it, now that they’ve taken Andy off as the lead developer.

    say what? .

    EDIT – there you go again, editing your forum post. fortunately, i’ve got a screenshot of it.

    Foxly, I would have expected you to know that getting more actions or filters into buddypress is very easy, as long as you aren’t changing the template files between major releases.

    Mercime, Andy’s gone nowhere. He’s still project lead.

    Im closing this topic as this is not on-topic anymore, start new posts if you want to discuss more.

Viewing 21 replies - 26 through 46 (of 46 total)

The topic ‘Here come the spammers!!!’ is closed to new replies.