BuddyPress.org

Sounds good to me. Give me a day or so to put some thought into it, then I’ll post a more structured proposal.

^F^

@Andrea_r

The goal is to limit:

1) Spam PM’s
2) Spam friend requests
3) Spam comments
4) Spam group creation
5) Spam group posts

Once a spammer / troll / hostile has created a member account on the system.

The goal is NOT to stop:

6) Spam comments on blog posts from non-members.
-> Already handled by dozens of plugins

7) Spam in profile fields
-> Limited damage. Will be handled by @francescolaffi ‘s GSoC project

8 ) Spam blog creation
-> Limited damage. Will be handled by @francescolaffi ‘s GSoC project

9) Spam sign-ups
-> Impossibly hard target. The only effective countermeasure is phone verification + geo IP + proxy blacklist; as implemented by Craigslist, eBay, PayPal, Elance, and many others.

Full background on all this stuff in about an hour.

Thanks!

^F^

All About BuddyPress Spam

From what I’ve seen over the past few days, the range of knowledge about spam in the BP community ranges from zero to PhD research project. So, to get this thread off to a productive start, I’m going to give everyone some background info on why spammers target our installations, how they do it, and what we can do to reduce or eliminate these kinds of attacks.

1) Why do spammers attack BP communities?

-> Spam is 100% economically motivated. Spammers do what they do because it’s very profitable. Even if only 1 out of a million messages the spammer sends actually reaches somebody, if it cost $2 to send out those million messages and the spammer makes $50 by tricking one person into giving them a credit card number, the spammer is going to throw every resource they have into sending out more messages …because they’re getting a 2500% return on their investment.

-> Given the choice between multiple sites, a spammer will pick the one that gives the largest payout.

Gmail is a “hard” target, with users that are experienced with spam. If a spammer sent a billion spam messages to accounts on Gmail, 99.9% of them would be probably be deleted by automated filters at other ISP’s along the way before even arriving at Gmail. The first thousand messages that arrived at gmail would likely be delivered but would be put in user’s spam folders; and the remaining 999,000 messages would be flat-out refused by Gmail’s servers.

Because anyone with an email account is familiar with spam, probably 999 of those 1000 users would ignore the spam message and 1 user might act on it. So if it cost $20 to send those billion messages and the spammer made $50 by tricking the one person into giving them a credit card number, they’ve only made $30 for all that work.

BP communities are usually “soft” targets that are inexperienced with spam.

Once a spammer gets into a BP community, every single message they send is delivered to a member, and most members are NOT expecting to be attacked by other users on the site.

If a user called “site_news” sends everyone a message that says: “Our site just got featured on Oprah! check out the video! http://www.youtube.com/watch/dQw4w9WgXcQ.cn” every single member is going to get that message, and probably half of them are going to click on the link. (did anyone notice what’s wrong with that “YouTube video” … )

Then, assuming there are 50,000 members on the BP site, half of them click on the link, half of those people are using Internet Explorer, and the attack site the link points to installs a backdoor on computers running IE …at $2 / install the spammer has just made $25,000!

Now, if *you* were a spammer, which site would you attack?

2) How do spammers find BP communities?

Using Google.

Example: http://www.google.ca/search?hl=en&q=%2B”is+proudly+powered+by+WordPress+and+BuddyPress” (front page of every BP site on the net)
Example: http://www.google.ca/search?hl=en&q=inurl:%22/community/members/%22+%2Bbuddypress (members page of every BP site on the net)

3) How do spammers attack websites?

-> Most spam attacks are done using robots, because sheer volume of posts is usually the winning factor. In situations where there is a “captcha wall” or other defense blocking registration to a “high value” site (hint: yours), spammers will use people in low-wage countries to break the captcha and sign up on the site. The going rate is about $2 per 1000 captchas.

http://www.decaptcher.com/client/

Once inside the site, they will then use bots to post spam to all the members on the site.

-> There are literally *thousands* of different programs available that spam websites, and they all have *different* venerabilities.

For example, this program: http://forums.digitalpoint.com/showthread.php?t=1124949

a) Will DEFEAT a “hidden fields” challenge,
b) Will DEFEAT a “javascript proof of work” challenge,
c) Will FAIL a “captcha” challenge
d) Will FAIL an “Akismet” challenge
e) Will FAIL a “Hashed Form Field ID” challenge

But this program: http://www.botmasternet.com/more1/ , wikipedia: http://en.wikipedia.org/wiki/XRumer , video of it running: http://www.youtube.com/watch?v=AL2i4SNPJmg

a) Will DEFEAT a “hidden fields” challenge,
b) Will DEFEAT a “javascript proof of work” challenge,
c) Will DEFEAT a “captcha” challenge
d) Will DEFEAT an “Akismet” challenge (uses proxy networks, never sends the same message twice)
e) Will DEFEAT a “Hashed Form Field ID” challenge
f) Will FAIL a “enter the numbers with a triangle over them” challenge (as used by PlentyOfFish.com)
g) Will FAIL a “click on the photos of cats but not the photos of dogs” challenge

4) How do we stop spammers from attacking BP communities?

-> By making it frustrating and unprofitable (but not necessarily impossible) for spammers to target us; while making these tactics invisible to normal users.

I will cover how I propose to do this in the next post.

^F^

PART 2 – DEFEATING SPAMMERS

In the last post I covered why and how spammers attack BP installations. This post will cover how I propose to counter them.

Fast Attacks -vs- Slow Attacks

There are two basic kinds of spam attacks that get run on social networks: “fast” or “flood” attacks, and “slow” attacks.

In a fast attack, the spammer signs up for an account on the site, then sends thousands of messages as quickly as possible.

Obviously, the site admin will be deluged with complaints about the spam user and quickly delete their account …but in the hours (or days) it takes the admin to respond, hundreds and hundreds of people will read the spam messages. Then the spammer signs up for another account, and repeats the process.

In a “slow” attack, the spammer signs up for *hundreds* of accounts on the system, often over a period of many months, and only sends out spam messages one at a time …often days, weeks, or months apart.

“Slow” attacks are very difficult to counter using automation …at least without annoying legitimate users.

The best way people have come up with so far is just a “report spam” button which, when clicked, reports the member to an admin so they can investigate it and if necessary delete the account. This will be implemented as part of @francescolaffi ‘s BP content moderation plugin in a couple of months.

Unfortunately, a “report spam” button doesn’t work well against “fast” attacks.

This is because:

a) There is a delay while the admin responds to manually submitted spam reports, or,
b) When a consensus scheme is used (if X users report a member their account gets suspended), there is a delay while enough votes are accumulated to flag the member as a spammer.

During that time, people are reading the spam messages and the spammer is winning.

Goals of Proposed BP Core Anti-Spam Mods

The goal of the proposed core modifications is to counter “fast” attacks by the following means:

1) To make it difficult for a spammer to create large numbers of member accounts using automated means.

2) To make it difficult for a spammer that already has a member account to use automated means to:
a) send large volumes of PM’s
b) send large numbers of friend requests
c) create large numbers of groups
d) create large numbers of group posts
e) post large numbers of comments
f) post large numbers of status updates

3) To accomplish 1) and 2) without being annoying to legitimate users.

4) To make the system configurable, so it can be adapted to the needs of the site …for example: visually impaired users, or display on mobile phones.

5) To make the system “on by default” and “secure by default”

How We Can Accomplish This

1) New User Sign-up

a) Add a captcha on the new account sign-up screen.
b) If the “user” gets the captcha wrong on the first try, require *TWO* captchas to be solved before they can proceed. (If the odds of a bot solving ONE captcha with OCR are 1 in 100, the odds of the bot solving TWO captchas with OCR are 1 in 10,000. This is a technique Gmail uses.)

…set X to be a random number on each installation between 3 and 7…

c) If the user gets X captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours). (This is what Craigslist does)
d) If the user fails X captchas *again* after being blocked, permanently ban their IP and post it to akismet.
e) If a locally banned IP tries to sign-up, don’t throw an “error page”. Completely ignore the request and don’t send anything.
f) If an akismet banned IP tries to sign up, require *TWO* captchas to be solved on the first try, and if they get X captchas wrong in a row, permanently ban their IP and repost it to akismet.
g) Add an option field to the admin menu that limits the number of accounts that can be created per IP address. By default, set it at 2.

2) Existing User Sign-In

a) Use a “normal” password box on first sign-in attempt.
b) If the member gets their password wrong on the first try, require them to solve a captcha on the second try. Offer password recovery option.
c) If the member gets their password wrong on the second try, require *TWO* captchas to be solved before they can proceed. Offer password recovery option.

…set X to be a random number on each installation between 3 and 7…

d) If the user gets X logins / captchas wrong in a row, block the visitor’s IP for a random amount of time (15 minutes to 2 hours).

3) Private Messages

a) Add a field to the user table that allows PM limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Add three option fields on the admin menu: allow “X” messages to be sent every 24 hours, averaged over the past “Y” hours with “Z” hysteresis
…when BP is installed, randomly set X, Y, and Z to allow a daily maximum of between 18 and 24 messages, averaged over between 2 and 24 hours, +/- 3 messages.
c) If the maximum is exceeded, require the member to solve a captcha before they can send another PM.
d) If they get the first captcha wrong, require them to solve two captchas before they can send another PM.

…set R to be a random number on each installation between 3 and 7…

e) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours). (This is what Craigslist does)
f) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
g) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

Consider how difficult the algorithm above makes it to send automated messages. A spammer can’t just send “12 messages a day” or “1 message an hour” and avoid triggering the system. Every BP installation will have a unique combination that will cause it to trip. Yet for a “normal” user, the system will hardly ever trip, and if it does, it takes all of 5 seconds to enter a captcha and continue. And the system can be bypassed entirely for edge cases, like paid advertisers or site news.

3) Friend Requests

a) Create a config option in BuddyPress that allows the admin to remove the member’s directory with one click. Disable the member directory by “default” on new installs. In my experience, the only people that use the member’s directory (in its default state, on a socially oriented site) are Spammers, Marketers, and Competitors. There’s a reason Facebook, MySpace, LinkedIn, and Twitter do not have “global” member directories.
b) Implement same scheme as private messages.

4) Group Creation

a) Add a field to the user table that allows Group limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Add an option field on the admin menu that sets a maximum number of groups that can be created by a user. By default, set it at 5.

5) Group Posts

a) Add a field to the user table that allows group post limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Create a “whitelist” field on the admin page that allows “trusted” media sharing URL’s like YouTube, Revver, Flickr, etc to be bypassed in spam protection.
c) Create an option that automatically “scrubs” URL’s and email addresses from group posts if they are not on the whitelist. Not just “nofollow” …complete removal. This will stop 90% of abuse dead in its tracks, because most spammers are just trying to get traffic to a site or replies to an email.
d) If the system detects a URL or email address embedded in a message, and it’s not on the whitelist, require a captcha to be solved before allowing the post.
e) If they get the first captcha wrong, require them to solve two captchas before approving the post.

…set R to be a random number on each installation between 3 and 7…

f) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
g) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
h) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

i) For posts that do not contain a URL or email address, run the post through akismet. If it passes, approve the post. If it fails, require a captcha to be solved before allowing the post.
j) If they get the first captcha wrong, require them to solve two captchas before approving the post.
k) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
l) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
m) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

6) Comments

a) Create an admin option that only allows users to comment on their *friend’s* items. Activate it by default on new BP installations.

7) Status Updates

a) Add a field to the user table that allows status update limiting to be bypassed or set to a unique value on a user-per-user basis.
b) Create a “whitelist” field on the admin page that allows “trusted” media sharing URL’s like YouTube, Revver, Flickr, etc to be bypassed in spam protection.
c) Create an option that automatically “scrubs” URL’s and email addresses from status updates if they are not on the whitelist. Not just “nofollow” …complete removal. This will stop 90% of abuse dead in its tracks, because most spammers are just trying to get traffic to a site or replies to an email.
d) If the system detects a URL or email address embedded in a message, and it’s not on the whitelist, require a captcha to be solved before allowing the activity stream post.
e) If they get the first captcha wrong, require them to solve two captchas before approving the activity stream post.

…set R to be a random number on each installation between 3 and 7…

f) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
g) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
h) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

i) For activity stream posts that do not contain a URL or email address, run the post through akismet. If it passes, approve the post. If it fails, require a captcha to be solved before allowing the post.
j) If they get the first captcha wrong, require them to solve two captchas before approving the post.
k) If the user gets R captchas wrong in a row, block their IP for a random amount of time (15 minutes to 2 hours).
l) If the user fails R captchas *again* after being blocked, permanently ban their IP and post it to akismet.
m) If a locally banned IP tries to visit the site, don’t throw an “error page”. Completely ignore the request and don’t send anything.

8 ) In All Cases

a) When a member account is banned, or repeatedly triggers spam protection measures, send an alert to the site administrator.
b) Allow admin alerts to be disabled if necessary, example: DDOS attack against the site.

9) CONCLUSION

While the list of modifications above may look incredibly complicated, really, it’s not.

I’d say “worst case” it’s about a week of work to research and make these modifications. Then we can push it out into beta testing with all the other new code to give it a proper shakedown.

I’m sure there are plenty of ways the algorithms above could be improved, so please go ahead and post your feedback!

Thanks!

^F^

@thekmen

Last I heard, BuddyPress does not run activity stream posts, or anything else, through Aksmet …it’s wide-open and that’s what’s causing the problem!

If you install the WP Akismet plugin, it runs *blog comments* through Akismet, but that’s it.

See why I’m really concerned and am putting work into this?

^F^