BuddyPress.org

+1

I’ve got a client who needs private blogs (to hide from the scary google robots) but a public directory.

~Kyle~

meta name=’robots’ content=’noindex,nofollow’

@TedMann

I think someone more knowledgeable about things .htaccess could better answer that question. I’m really still learning about all this stuff myself.

About your other idea though… now that could be brilliantly simple! It could sure put one heck of a damper on the efforts of human sploggers who are, if their activities are any indicator, a lazy bunch. Only thing is, it wouldn’t do much for those bots who manage to squeeze through whatever “backdoor” they happen to find (or make).

Anyone want to take on a little “Avatar Required” plugin challenge here?

@TedMann

This is what I’ve added to .htaccess to block bots:

# IF THE UA STARTS WITH THESE
RewriteCond %{HTTP_USER_AGENT} ^(aesop_com_spiderman|alexibot|backweb|bandit|batchftp|bigfoot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(black.?hole|blackwidow|blowfish|botalot|buddy|builtbottough|bullseye) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cheesebot|cherrypicker|chinaclaw|collector|copier|copyrightcheck) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(cosmos|crescent|curl|custo|da|diibot|disco|dittospyder|dragonfly) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(drip|easydl|ebingbong|ecatch|eirgrabber|emailcollector|emailsiphon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(emailwolf|erocrawler|exabot|eyenetie|filehound|flashget|flunky) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(frontpage|getright|getweb|go.?zilla|go-ahead-got-it|gotit|grabnet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grafula|harvest|hloader|hmview|httplib|httrack|humanlinks|ilsebot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(infonavirobot|infotekies|intelliseek|interget|iria|jennybot|jetcar) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(joc|justview|jyxobot|kenjin|keyword|larbin|leechftp|lexibot|lftp|libweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(likse|linkscan|linkwalker|lnspiderguy|lwp|magnet|mag-net|markwatch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mata.?hari|memo|microsoft.?url|midown.?tool|miixpc|mirror|missigua) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister.?pix|moget|mozilla.?newt|nameprotect|navroad|backdoorbot|nearsite) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(net.?vampire|netants|netcraft|netmechanic|netspider|nextgensearchbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(attach|nicerspro|nimblecrawler|npbot|octopus|offline.?explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(offline.?navigator|openfind|outfoxbot|pagegrabber|papa|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcbrowser|php.?version.?tracker|pockey|propowerbot|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(psbot|pump|queryn|recorder|realdownload|reaper|reget|true_robot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(repomonkey|rma|internetseer|sitesnagger|siphon|slysearch|smartdownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(snake|snapbot|snoopy|sogou|spacebison|spankbot|spanner|sqworm|superbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(superhttp|surfbot|asterias|suzuran|szukacz|takeout|teleport) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|the.?intraformant|thenomad|tighttwatbot|titan|urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(turingos|turnitinbot|urly.?warning|vacuum|vci|voideye|whacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww-perl|widow|wisenutbot|wwwoffle|xaldon|xenu|zeus|zyborg|anonymouse) [NC,OR]
# STARTS WITH WEB
RewriteCond %{HTTP_USER_AGENT} ^web(zip|emaile|enhancer|fetch|go.?is|auto|bandit|clip|copier|master|reaper|sauger|site.?quester|whack) [NC,OR]
# ANYWHERE IN UA — GREEDY REGEX
RewriteCond %{HTTP_USER_AGENT} ^.*(craftbot|download|extract|stripper|sucker|ninja|clshttp|webspider|leacher|collector|grabber|webpictures).*$ [NC]
# ISSUE 403 / SERVE ERRORDOCUMENT
RewriteRule . – [F,L]

To help block spam registrations, add the following to .htaccess, then create a simple GOAWAY type html page and upload to your root directory:

# BEGIN ANTISPAMBLOG REGISTRATION
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .wp-signup.php*
RewriteCond %{HTTP_REFERER} !.yoursitehere.com. [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule (.*) http://yoursitehere.com/yourgoawaypage.html [R=301,L]

Add the following to .htaccess to deny access to wp-config.php to anyone who doesn’t have your ftp details:

order allow,deny
deny from all

Instead of example.com/register or example.com/sign-up, use something like example.com/unb2x-2010 for your register page. If you were a spammer, would that look like an inviting url to hack?

Hope this helps

I was having 5 or 6 sploggers sign up daily no matter what I did until about 2 weeks ago when I revamped my tactics. Since then, I have had 0 spam signups… not one. Fingers crossed Here’s what I’ve done:

– Removed references to WP/BP in footer text
– Changed the register slug to something unrecognizable that has no bearing whatsoever to the concept of signing up (so even those grossly underpaid 3rd-world human spammers can’t figure it out)
– Installed WPMU Super Captcha to let the nice humans through: https://wordpress.org/extend/plugins/super-capcha/
– Installed WP-Ban to block the not-so-nice ones: https://wordpress.org/extend/plugins/wp-ban/
– Installed Buddypress Humanity as a double-check: https://buddypress.org/community/groups/buddypress-humanity/
– Blocked lists of bad bots in .htaccess as suggested in this post: https://buddypress.org/community/groups/how-to-and-troubleshooting/forum/topic/buddypress-spam/?topic_page=2&num=15#post-60177
– Added “deny from all” in .htaccess for wp-config.php
– If someone does manage to access the register page through a direct url (without visiting any other page first), they are bumped to a GOAWAY page with the following in .htaccess. .

# BEGIN ANTISPAMBLOG REGISTRATION
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .wp-signup.php*
RewriteCond %{HTTP_REFERER} !.examplesite.com. [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule (.*) http://examplesite.com/goaway.html [R=301,L]

So far, so good. As I mentioned, not a single splogger has managed to get through in about 2 weeks. If they do, there are 2 ingredients in the above recipe that can be adjusted:
– the captcha image is fully customizable to render bot algorithms redundant (hopefully)
– the register slug can be changed as often as you change socks

On a final note, there are also some interesting tweaks to be found here: http://www.smashingmagazine.com/2010/07/01/10-useful-wordpress-security-tweaks/

Not sure of the process but even if you haven’t got bbpress running locate and remove the file. If spambots are managing to get around hidden fields that should remain empty it suggests they are not using whatever form that protection is on.

For CURL try adding this: (but check carefully things still work!)

# trap curl registration downloaders – block in allow,deny rules
SetEnvIfNoCase User-Agent “^curl” blog_spammer
Order Allow,Deny
Allow from All
Deny from env=blog_spammer

Be careful about blocking IP ranges it’s a difficult practice and one that technically you are supposed to notify about in case innocent yet important sites get blocked, you can add further rules to the deny lines above but unless there is a very persistent IP it’s probably not worth it and likely spoffed anyway.

Robots.txt file – google and you will find guides.

a few things i’ve done

removed the powered by in the footer (just changed up the wording to WP/BP)
block the crappy browser MSIE ([3456]).
block a bunch of bad bots (something like: http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html )
block a bunch of CDIR ranges (something like: http://www.wizcrafts.net/blocklists.html )

For those of you that want to directly influence the future of BuddyPress, http://trac.buddypress.org. Make it your friend. Learn it. Love it. Live it. Give it a hug everyday and patch a bug.

The Trac is where you can post code snippets, or giant mega patches of code that you think should be integrated into BuddyPress. You can see the timeline of when people have done what, and see the outstanding bugs that need squashing before we can safely release the next version. The more bugs you fix, the more code you contribute, the more you are directly involved not only in the community, but directly in the future of the platform as a whole.

As incentive to help out, if your goal is to be a developer and make a career out of BuddyPress, consider walking into a meeting with a possible client, and when they ask what your level of involvement is with WordPress or BuddyPress, and you can respond with “I make it,” your chances of securing that client are pretty good. In order to help make BuddyPress, you have to actually help us make it, and you do that via the Trac. I can say this, because that’s how I did it with both WordPress and BuddyPress, and I’m down to help you guys do it too.

There are plenty of people that are highly active in the Trac that aren’t so much so in the forums, and vice versa. Since we moved BuddyPress.org over to 1.2, both Andy and myself have been busy with our own assignments that yes, do involve BuddyPress, but also involve other neat things like the WordPress.com “Like” feature and planning some neat things for a WordCamp.org redesign.

Truth be told, if /anyone/ is concerned about where I am or what I’m doing in regards to BuddyPress or the future of the project, there are at least 10 methods to contact me directly and I am totally happy and not annoyed by anything that has to do with BuddyPress. Drop me a line, let’s chat http://en.gravatar.com/johnjamesjacoby

To answer a few of the questions/comments/statements in this topic: Private Messages are turned off because spam bots have started targeting BuddyPress installations and we were getting hit pretty hard after we upgraded the site. Raise your hand if you got a PM from someone claiming to love you enough to help you with male enhancement. Regarding my absence in the forums, I’ve really just taken on too many clients and haven’t had the time to look backwards at support AND forwards at development at the same time. It won’t always be that way, but it has been lately and I like it about as much as you all seem to too. I love being in the forums and helping people out, and I’m sad I haven’t been able too recently.

Andy is the figure head of BuddyPress and serves as the guiding light of the project similar to how Matt does for WordPress and bbPress, but there is no shortage of capable people in the BuddyPress community that could take this project by the horns and make it their own at any point. I know I’m not Andy, but if I can pretend to be to help anyone when he’s not around, ping me.

Along the lines of what @matt said, I love using @nacin as an example. He stormed into the WordPress Trac and started contributing code and patching bugs. Some were great, and some were rubbish, but he learned as he contributed and within 1 calendar year he has merited his way into being a core committer for WordPress, and contributed something insane; like 60% or more of the commits on the WP3.0 branch are his doing or somehow as a result of his hard work and commitment to the project. While there is only one @nacin, there is plenty of room for any one of you to be very @nacin like.

By the way, if there is an election and I’m voted out, I’m not leaving without a fight. You’ll have to chase me out of town with torches and pitchforks.

Now… my spambusher script gives me some very rudimentary statistics… and in the 2 days and 9 hours since that post above, the number of spam registrations has gone up 50%… but the number that I actually delete myself has gone down by about the same number

Which tells me two things:

1. People or bots are actually following the link above and tripping the spambush

2. Links to buddypress sites from buddypress.org are just ASKING to be spammed

Glad you figured it out.

FYI, the community takes time out of their day to help people and write documentation for free. Contrary to popular belief, we are not robots! We need more people like you to contribute to the codex, after all this is an open-source project. The more people that contribute, the more rich the documentation will be — open source documentation!

Also, sometimes forum posts are missed due to the amount that gets posted. A simple post to bump your thread after 24 hours will help bring attention to your issue.

Spammers target all social networks. They literally overran an elgg site I have and I’m rebuilding with WP/BP. A few still get through with the latest BP and anti-spam and custom profile fields, not a lot, two or three a day, but that’s nothing. I found about 600 in my first month in my users that never made it to the surface. As for the few that did, having a couple fill in custom profile fields is really helpful. The bots that do sneak through fill those two spots with gibberish, in my case age and location, so it’s easy to identify spam members.

Actually… Terry is correct. SPAMers are in fact hiring people from India to fill our registration forms and CAPTCHA’s by hand. They get paid next to nothing and just sit there for hours and hours a day filling out CAPTCHA’s. I’m sure the majority of SPAM comes from Bots… but it’s not all bots. And there is no way to stop a human short of banning entire countries.

I am also having this problem – I guess everybody is. I also followed the tips you mentioned and initially, it reduced splogger registrations a lot.

However, I disagree with Terry: there are not real people setting up blogs and answering captchas, these are bots. As kiwipearls mentioned, if you go and try to sign up manually, you have to fill in the required fields.

There is a leak somewhere in BuddyPress/WPMU registration and all methods to stop the oil have failed until now. BP (haha) people say it’s WPMU and the other way around, I guess. The leak has been here for months and nobody seems to want to fix it. Maybe it’s some kind of corruption since the premium site Terry mentioned has a way to fix it.

Hi Jeff, I can’t make the chat Wednesday as I’m going to be on a plane between London and Hamburg, but I wanted to add to this:

1. wp-recaptcha — I’m working with the developer of this plug-in so that we have one fork that works everywhere, BP included. Given that this is the topic, let me try to get that basic code up. Even with simple recaptcha support, there’s a huge decrease in spam signups. It seems not to solve the smartest scripts, the ones that send PMs (at least not on our site), so I think once we get one recaptcha working, making the “failed” recaptchas more intelligent to avoid these automated bots would be great. Thanks for the ideas above — this is great fodder — so I’d encourage people to get involved on the same fork so we can put this into action sooner rather than later. Let me post a separate update within the next couple of days.

2. Since PMs are a big problem, and this thread is getting very, very ambitious, why not at least begin testing this with a separate plugin? I’d like to at least see something that stops mass-mailings and highlights that user, as that’d be an easy way to weed people out, at least as more comprehensive solutions are developed.

3. Reviewing core is probably worthwhile. A mistake in bp_signup_validate’s code was being exploited by hackers. I know this is part of 1.2.4, but I went ahead and applied the diff attached to this (now-closed) ticket to our current 1.2.3 install:
https://trac.buddypress.org/ticket/2289
— this made a big difference. I wonder if anything else follows this pattern, and how we might hunt it down.

Grand, wide-reaching plans sound terrific, but I’d hate if that derailed some short-term fixes; seems we can have both.