[foxly]

Seven spam PM’s. From five different users. In the past 24 hours.

And they’re getting clever too …they’ve found a defect that allows them to delete the backlink to their profile from the message, making it harder to come after them.

I was always shocked by the lack of spam control features on BuddyPress, and I’ve been amazed that nobody has been attacking BP installations …given that with under 1000 lines of code its possible to write a bot that posts billions of spam PM’s, forum posts, and splogs daily to practically any BuddyPress site.

Well, it looks like somebody has written that bot, because the posts are definitely at a volume where its being automated.

If we don’t get this nailed ASAP, we can probably look forward to crippling attacks against pretty much every BP site out there within a matter of weeks.

So I have a proposal: If the core devs give me permission, I can take a few days off working on BP Album+ and write a patch (not a plugin, as it requires mods to the core) that deals with this problem.

@apeatling @r-a-y @DJPaul @21cdb @johnjamesjacoby

What do you think?

^F^

[r-a-y]

@foxly – Come into the #buddypress-dev irc room on Freenode and let the team know what you have in mind!

You can also use a java web version of IRC if you don’t have a client:
http://java.freenode.net/?channel=buddypress-dev

[Paul Wong-Gibbs]

Initial reaction from Jeff & I is that you detail what you have in mind before you dash off in one direction etc

[Jeff Sayre]

Okay, per IRC dev chat, let’s use this thread for discussions on ideas to combat registration spam and other types of spam.

[foxly]

Sounds good to me. Give me a day or so to put some thought into it, then I’ll post a more structured proposal.

^F^

[Andrea Rennick]

And this would specifically deal with spam from actual users who have managed to sign up and and now using the internal messaging system to spam, correct?

And not spam signups, spam blog. Just to clarify.

[foxly]

@Andrea_r

The goal is to limit:

1) Spam PM’s
2) Spam friend requests
3) Spam comments
4) Spam group creation
5) Spam group posts

Once a spammer / troll / hostile has created a member account on the system.

The goal is NOT to stop:

6) Spam comments on blog posts from non-members.
-> Already handled by dozens of plugins

7) Spam in profile fields
-> Limited damage. Will be handled by @francescolaffi ‘s GSoC project

8 ) Spam blog creation
-> Limited damage. Will be handled by @francescolaffi ‘s GSoC project

9) Spam sign-ups
-> Impossibly hard target. The only effective countermeasure is phone verification + geo IP + proxy blacklist; as implemented by Craigslist, eBay, PayPal, Elance, and many others.

Full background on all this stuff in about an hour.

Thanks!

^F^

[modemlooper]

You rock!

[foxly]

All About BuddyPress Spam

From what I’ve seen over the past few days, the range of knowledge about spam in the BP community ranges from zero to PhD research project. So, to get this thread off to a productive start, I’m going to give everyone some background info on why spammers target our installations, how they do it, and what we can do to reduce or eliminate these kinds of attacks.

1) Why do spammers attack BP communities?

-> Spam is 100% economically motivated. Spammers do what they do because it’s very profitable. Even if only 1 out of a million messages the spammer sends actually reaches somebody, if it cost $2 to send out those million messages and the spammer makes $50 by tricking one person into giving them a credit card number, the spammer is going to throw every resource they have into sending out more messages …because they’re getting a 2500% return on their investment.

-> Given the choice between multiple sites, a spammer will pick the one that gives the largest payout.

Gmail is a “hard” target, with users that are experienced with spam. If a spammer sent a billion spam messages to accounts on Gmail, 99.9% of them would be probably be deleted by automated filters at other ISP’s along the way before even arriving at Gmail. The first thousand messages that arrived at gmail would likely be delivered but would be put in user’s spam folders; and the remaining 999,000 messages would be flat-out refused by Gmail’s servers.

Because anyone with an email account is familiar with spam, probably 999 of those 1000 users would ignore the spam message and 1 user might act on it. So if it cost $20 to send those billion messages and the spammer made $50 by tricking the one person into giving them a credit card number, they’ve only made $30 for all that work.

BP communities are usually “soft” targets that are inexperienced with spam.

Once a spammer gets into a BP community, every single message they send is delivered to a member, and most members are NOT expecting to be attacked by other users on the site.

If a user called “site_news” sends everyone a message that says: “Our site just got featured on Oprah! check out the video! http://www.youtube.com/watch/dQw4w9WgXcQ.cn” every single member is going to get that message, and probably half of them are going to click on the link. (did anyone notice what’s wrong with that “YouTube video” … )

Then, assuming there are 50,000 members on the BP site, half of them click on the link, half of those people are using Internet Explorer, and the attack site the link points to installs a backdoor on computers running IE …at $2 / install the spammer has just made $25,000!

Now, if *you* were a spammer, which site would you attack?

2) How do spammers find BP communities?

Using Google.

Example: http://www.google.ca/search?hl=en&q=%2B”is+proudly+powered+by+WordPress+and+BuddyPress” (front page of every BP site on the net)
Example: http://www.google.ca/search?hl=en&q=inurl:%22/community/members/%22+%2Bbuddypress (members page of every BP site on the net)

3) How do spammers attack websites?

-> Most spam attacks are done using robots, because sheer volume of posts is usually the winning factor. In situations where there is a “captcha wall” or other defense blocking registration to a “high value” site (hint: yours), spammers will use people in low-wage countries to break the captcha and sign up on the site. The going rate is about $2 per 1000 captchas.

http://www.decaptcher.com/client/

Once inside the site, they will then use bots to post spam to all the members on the site.

-> There are literally *thousands* of different programs available that spam websites, and they all have *different* venerabilities.

For example, this program: http://forums.digitalpoint.com/showthread.php?t=1124949

a) Will DEFEAT a “hidden fields” challenge,
b) Will DEFEAT a “javascript proof of work” challenge,
c) Will FAIL a “captcha” challenge
d) Will FAIL an “Akismet” challenge
e) Will FAIL a “Hashed Form Field ID” challenge

But this program: http://www.botmasternet.com/more1/ , wikipedia: http://en.wikipedia.org/wiki/XRumer , video of it running: http://www.youtube.com/watch?v=AL2i4SNPJmg

a) Will DEFEAT a “hidden fields” challenge,
b) Will DEFEAT a “javascript proof of work” challenge,
c) Will DEFEAT a “captcha” challenge
d) Will DEFEAT an “Akismet” challenge (uses proxy networks, never sends the same message twice)
e) Will DEFEAT a “Hashed Form Field ID” challenge
f) Will FAIL a “enter the numbers with a triangle over them” challenge (as used by PlentyOfFish.com)
g) Will FAIL a “click on the photos of cats but not the photos of dogs” challenge

4) How do we stop spammers from attacking BP communities?

-> By making it frustrating and unprofitable (but not necessarily impossible) for spammers to target us; while making these tactics invisible to normal users.

I will cover how I propose to do this in the next post.

^F^

[Hugo Ashmore]

While you’re preparing part 2 I’ll make the comment (probably unpopular) that too an extent this is an issue that BP, WP, Automatic must accept some responsibility for in that WP has always followed the course of making it as easy as possible for inexperienced people to set up a blog/blogs the principle of ‘Out Of The Box’ and ‘5 Minute Install’ all designed to promote the app/s to those users who otherwise might be put off, it’s a marketing ploy to ensure that the app gains widespread and popular use (that is being deliberately cynical to make a point) It is due to that that I say there is a duty of care that falls on the App not on the user or community. I know how to get down and dirty with htaccess files, to read logs, enable various methods to deal with an issue – as do many others here – but lets not forget most don’t! I would suggest that it’s time to pull together all the various approaches to dealing with spam in one clear stickied post, make the steps as clear as possible but emphasize that these steps are of paramount importance to follow (thinking about it that may already exist?) Until such time as Foxly or the dev team comes up with the core improvements.

For the record I have enabled most of the steps found in various threads here and elsewhere and also disabled sub blog registration and receive no more than around 6 -8 spam sign ups a day, which we can deal with quite quickly and effectively, I’m still slightly puzzled as to why some appear to have such ongoing issues though, very sympathetic but puzzled nonetheless.

[Jeff Sayre]

@foxly-

This is a very nice summary of the problem. Thank you for providing the introduction to the various attack vectors spammers currently use.

I would argue, as you know, that WP / BP also needs to combat registration spam–even though it is the hardest issue to address. There area a number of BP.org members that are looking for a solution, however imperfect, that will noticeably reduce spam signups. If a person is infected with a small viral load, the resulting illness often will not be as severe as if they had received a large dose of invading organisms. The same can be said with website spam signups. Any reduction is better than none.

But, as this is your thread and I do not want to take your thread off topic (or have others do what I just did ), I will ask that we table that discussion for another thread at another time and focus in this thread on solutions to combating spam once a spam account has successfully registered.

Once again, this is a great start to the conversation.

[5887735]

Maybe BP should require that you choose your own register slug after activating the plugin. Perhaps also require you name your required fields fields, instead of the default “name” or “base.” The less default settings BP has the harder it is for BOTS.

[xspringe]

Allowing users to report spam would be a very useful feature. There’s only so much you can do in terms of technical spam prevention, and technical spam prevention always gets cracked eventually.

If the amount of spam reports for a certain account exceeds x, then freeze the account until an admin can review the account. The admin should then have the option to do an IP based ban of the account if it appears to be a spammer. Some very basic IP based messaging/commenting/posting rate limitation would help too.

[5887735]

My BP site is fairly new. I had one PM spammer and I changed my register slug and added birth day to the required fields and so far no return spammers (about 1000 new members per month, 4,000 current). I’m sure this won’t end attacks, but hopefully it with stave off many of the BOTS.

[stwc]

Example: http://www.google.ca/search?hl=en&q=%2B”is+proudly+powered+by+WordPress+and+BuddyPress”; (front page of every BP site on the net)
Example: http://www.google.ca/search?hl=en&q=inurl:%22/community/members/%22+%2Bbuddypress (members page of every BP site on the net)

Very much behind this, but I will mention that changing those two things are the first thing I’ve done with my BP installs (along with other stuff I mentioned in the article I did for the I-guess-it’s-not-coming-back bp-tricks.org). Agree that an install routine that forces the user to customize their slugs (explaining possibly consequences if they don’t) would be a great idea.

[modemlooper]

I change my slug, have at least one new required profile filed as a drop down and added WP Super captcha plugin and have never had a spam sign up ever.

[Hugo Ashmore]

As Jeff mentioned we ought not to derail foxlies thread any further. Perhaps we ought to start that thread suggested re-hashing all the approaches tried, implemented, proven or not and maybe a mod could extract a set of definitive steps that ought to be implemented by anyone setting up a new install.

[jwordsmith]

@foxly I love this. It’s what I’ve been looking for and I am really looking forward to Part II. Wait, did that sound like a spam comment. Damn, they’ve getting to me!

[foxly]

PART 2 – DEFEATING SPAMMERS

In the last post I covered why and how spammers attack BP installations. This post will cover how I propose to counter them.

Fast Attacks -vs- Slow Attacks

There are two basic kinds of spam attacks that get run on social networks: “fast” or “flood” attacks, and “slow” attacks.

In a fast attack, the spammer signs up for an account on the site, then sends thousands of messages as quickly as possible.

Obviously, the site admin will be deluged with complaints about the spam user and quickly delete their account …but in the hours (or days) it takes the admin to respond, hundreds and hundreds of people will read the spam messages. Then the spammer signs up for another account, and repeats the process.

In a “slow” attack, the spammer signs up for *hundreds* of accounts on the system, often over a period of many months, and only sends out spam messages one at a time …often days, weeks, or months apart.

“Slow” attacks are very difficult to counter using automation …at least without annoying legitimate users.

The best way people have come up with so far is just a “report spam” button which, when clicked, reports the member to an admin so they can investigate it and if necessary delete the account. This will be implemented as part of @francescolaffi ‘s BP content moderation plugin in a couple of months.

Unfortunately, a “report spam” button doesn’t work well against “fast” attacks.

This is because:

a) There is a delay while the admin responds to manually submitted spam reports, or,
b) When a consensus scheme is used (if X users report a member their account gets suspended), there is a delay while enough votes are accumulated to flag the member as a spammer.

During that time, people are reading the spam messages and the spammer is winning.

Goals of Proposed BP Core Anti-Spam Mods

The goal of the proposed core modifications is to counter “fast” attacks by the following means:

1) To make it difficult for a spammer to create large numbers of member accounts using automated means.

2) To make it difficult for a spammer that already has a member account to use automated means to:
a) send large volumes of PM’s
b) send large numbers of friend requests
c) create large numbers of groups
d) create large numbers of group posts
e) post large numbers of comments
f) post large numbers of status updates

3) To accomplish 1) and 2) without being annoying to legitimate users.

4) To make the system configurable, so it can be adapted to the needs of the site …for example: visually impaired users, or display on mobile phones.

5) To make the system “on by default” and “secure by default”

How We Can Accomplish This

1) New User Sign-up

a) Add a captcha on the new account sign-up screen.
b) If the “user” gets the captcha wrong on the first try, require *TWO* captchas to be solved before they can proceed. (If the odds of a bot solving ONE captcha with OCR are 1 in 100, the odds of the bot solving TWO captchas with OCR are 1 in 10,000. This is a technique Gmail uses.)

…set X to be a random number on each installation between 3 and 7…

c) If the user gets X captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours). (This is what Craigslist does)
d) If the user fails X captchas *again* after being blocked, permanently ban their IP and post it to akismet.
e) If a locally banned IP tries to sign-up, don’t throw an “error page”. Completely ignore the request and don’t send anything.
f) If an akismet banned IP tries to sign up, require *TWO* captchas to be solved on the first try, and if they get X captchas wrong in a row, permanently ban their IP and repost it to akismet.
g) Add an option field to the admin menu that limits the number of accounts that can be created per IP address. By default, set it at 2.

2) Existing User Sign-In

a) Use a “normal” password box on first sign-in attempt.
b) If the member gets their password wrong on the first try, require them to solve a captcha on the second try. Offer password recovery option.
c) If the member gets their password wrong on the second try, require *TWO* captchas to be solved before they can proceed. Offer password recovery option.

…set X to be a random number on each installation between 3 and 7…

d) If the user gets X logins / captchas wrong in a row, block the visitor’s IP for a random amount of time (15 minutes to 2 hours).

3) Private Messages

a) Add a field to the user table that allows PM limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Add three option fields on the admin menu: allow “X” messages to be sent every 24 hours, averaged over the past “Y” hours with “Z” hysteresis
…when BP is installed, randomly set X, Y, and Z to allow a daily maximum of between 18 and 24 messages, averaged over between 2 and 24 hours, +/- 3 messages.
c) If the maximum is exceeded, require the member to solve a captcha before they can send another PM.
d) If they get the first captcha wrong, require them to solve two captchas before they can send another PM.

…set R to be a random number on each installation between 3 and 7…

e) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours). (This is what Craigslist does)
f) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
g) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

Consider how difficult the algorithm above makes it to send automated messages. A spammer can’t just send “12 messages a day” or “1 message an hour” and avoid triggering the system. Every BP installation will have a unique combination that will cause it to trip. Yet for a “normal” user, the system will hardly ever trip, and if it does, it takes all of 5 seconds to enter a captcha and continue. And the system can be bypassed entirely for edge cases, like paid advertisers or site news.

3) Friend Requests

a) Create a config option in BuddyPress that allows the admin to remove the member’s directory with one click. Disable the member directory by “default” on new installs. In my experience, the only people that use the member’s directory (in its default state, on a socially oriented site) are Spammers, Marketers, and Competitors. There’s a reason Facebook, MySpace, LinkedIn, and Twitter do not have “global” member directories.
b) Implement same scheme as private messages.

4) Group Creation

a) Add a field to the user table that allows Group limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Add an option field on the admin menu that sets a maximum number of groups that can be created by a user. By default, set it at 5.

5) Group Posts

a) Add a field to the user table that allows group post limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Create a “whitelist” field on the admin page that allows “trusted” media sharing URL’s like YouTube, Revver, Flickr, etc to be bypassed in spam protection.
c) Create an option that automatically “scrubs” URL’s and email addresses from group posts if they are not on the whitelist. Not just “nofollow” …complete removal. This will stop 90% of abuse dead in its tracks, because most spammers are just trying to get traffic to a site or replies to an email.
d) If the system detects a URL or email address embedded in a message, and it’s not on the whitelist, require a captcha to be solved before allowing the post.
e) If they get the first captcha wrong, require them to solve two captchas before approving the post.

…set R to be a random number on each installation between 3 and 7…

f) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
g) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
h) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

i) For posts that do not contain a URL or email address, run the post through akismet. If it passes, approve the post. If it fails, require a captcha to be solved before allowing the post.
j) If they get the first captcha wrong, require them to solve two captchas before approving the post.
k) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
l) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
m) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

6) Comments

a) Create an admin option that only allows users to comment on their *friend’s* items. Activate it by default on new BP installations.

7) Status Updates

a) Add a field to the user table that allows status update limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Create a “whitelist” field on the admin page that allows “trusted” media sharing URL’s like YouTube, Revver, Flickr, etc to be bypassed in spam protection.
c) Create an option that automatically “scrubs” URL’s and email addresses from status updates if they are not on the whitelist. Not just “nofollow” …complete removal. This will stop 90% of abuse dead in its tracks, because most spammers are just trying to get traffic to a site or replies to an email.
d) If the system detects a URL or email address embedded in a message, and it’s not on the whitelist, require a captcha to be solved before allowing the activity stream post.
e) If they get the first captcha wrong, require them to solve two captchas before approving the activity stream post.

…set R to be a random number on each installation between 3 and 7…

f) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
g) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
h) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

i) For activity stream posts that do not contain a URL or email address, run the post through akismet. If it passes, approve the post. If it fails, require a captcha to be solved before allowing the post.
j) If they get the first captcha wrong, require them to solve two captchas before approving the post.
k) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
l) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
m) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

8 ) In All Cases

a) When a member account is banned, or repeatedly triggers spam protection measures, send an alert to the site administrator.
b) Allow admin alerts to be disabled if necessary, example: DDOS attack against the site.

9) CONCLUSION

While the list of modifications above may look incredibly complicated, really, it’s not.

I’d say “worst case” it’s about a week of work to research and make these modifications. Then we can push it out into beta testing with all the other new code to give it a proper shakedown.

I’m sure there are plenty of ways the algorithms above could be improved, so please go ahead and post your feedback!

Thanks!

^F^

[thekmen]

Great posts & solutions @foxly.
I am eagerly awaiting the next release of BP Album+, but would happily wait till this major issue is sorted out.
Even on this site the spam is becoming more evident & annoying by the day.
As it is, there is no way I would roll out BuddyPress out to my larger sites, if https://buddypress.org/community/activity/ can’t keep the spam off the activity stream, what chance do us wanting to implement BuddyPress have?

[abcde666]

Privacy-features and Spam-control is much more important than the following-plugin…..

[thekmen]

also, while harder to combat but still surprised akismet didn’t kill the last post from @alstinwalker, lets not forget the link juice/indexing spammers, 5 mins with a post like that can give them the results they require.

[foxly]

@thekmen

Last I heard, BuddyPress does not run activity stream posts, or anything else, through Aksmet …it’s wide-open and that’s what’s causing the problem!

If you install the WP Akismet plugin, it runs *blog comments* through Akismet, but that’s it.

See why I’m really concerned and am putting work into this?

^F^

[luccame]

This plan is as beautiful as aggressive, it really seems that you want to completely eradicate the problem, so I like it very much! Just add bold warnings to the user explaining that her account will be suspended if login/captcha fails too many times… real people deserve to be alerted.

[abcde666]

for Gmail, their Captcha is very hard to read so I need to fill the Captcha usually 2 or 3 times.
Would not be great if the user-account gets deleted or user is not getting accepted to register if the Captcha has been entered wrong for 3 times.

Any other ideas ?

[foxly]

@Erich73

Add a “refresh” button beside the captcha that allows the user to flip through multiple captchas until they find one they like.

^F^