[rcain]

ps. we also use (dreaded) captcha fields (plugin) on forms, but have noted of late there are bots out there (eg: XRummer, et al) scraping such captcha images off the web in order to seed simple AI scripts to bypass such protection. pretty clever stuff & born out by what we see in our logs.

[rcain]

@bp-help

good suggestions. thx. 2 of them r new to me, so other people may find them helpful also.

on our sites we r using::

Keith Graham’s most excellent ‘stop-spammer-registrations-plugin’ – https://wordpress.org/plugins/stop-spammer-registrations-plugin/

– has stopped over 53,000 spammers since feb this year! it uses external lookups on StopForumSpam, ProjectHoneyPot, BotScout, (Akismet, which we dont use), others – thus great collective benefit/advance warning of bad traffic. also traps brute force attacks (bad logins/registrations/comment posts, etc), etc. is simple enough to play nice with most plugins.

to try & keep as much load off the front-end of the server as possible, we also have set up:

linux iptables ( & ufw add on )- as the basis of all firewall stuff. also has our manually maintained blacklists & whitelists. various custom rule chains setup. takes a while to get your head around, but is essential.

linux fail2ban – essentially an add on to iptables, puts people in jail for bad behaviour – eg: brute force attacks against ssh, ftp, mail logins. we also have set up custom rules detecting bad activity against wp-login.php itself via fail2ban. am looking to do some more with this.

linux apache – mod-security2, libapache2-mod-evasive, libapache2-mod-antiloris, libapache2mod-spamhaus – which help protect against general bad behaviour, DDOS, blank header attacks, the infamous ‘Loris’ script (which we’ve experienced!), and bot-nets. still assessing how effective these r.

we have also had to tune apache on our VPS for resilience in the face of DDOS type attacks and heavy-handed brute force attacks.

some further good tips here: http://www.dannytsang.co.uk/index.php/apache-2-hardening-tips/ & elsewhere.

linux logwatch – reports various access stats (the good & the bad & the ugly) via email – very useful indeed for checking whther situation is under control (or not).

linux rkhunter – scans for rootkits on the server from time to time – just be sure – & particularly useful if u ever do get infected in hunting down the intruder’s code.

obviously we also have file system bolted down. (there is a good wp plugin to check permissions bolt-down, i forget what its called). we also spend a LONG time analysing logs etc.

anway, that takes care of many of the bad boys, but we r still left with the following problems to crack:

1) we have observed that many bad bots/scripts are exhibiting ‘learning behaviour’ (ie. heuristic) and r finding ways around fail2ban rules/jails, etc. in particular:

a) rotating IP addresses to match ‘ban counts’ – currently we have them wasting an IP address every 3-4 attempts, but they still seem to have an inexhaustable supply, else are spoofing extreamly well.

b) varying their retry period to match the length of jail sentence. (ie. they are not wasting their mips whilst in jail, just enough to detect when they are released,record it, and tune their future responses).

2) content scrapers, probes and bad-bots generally – these r wasting enormous resource on our servers. typically i would suggest such ‘bad traffic’ is responsible for over 50% of total server load (ie. not good at peak times on a busy site). additional problems we r facing here:

a) bad bots often spoof the agent string to pretend to be eg. google, bing, etc. the only way u can tell is by reverse lookup of ip address and try and match to one of well known range of ‘good bot’ addresses. but, despite fact that many ranges are well known, most of them are never actually published or confirmed, many are variable. i am not aware of any definitve list of ip addresses of good bots (though there is http://www.iplists.com/ whichis not bad, & http://www.webmasterworld.com/search_engine_spiders/ which is often helpful – these are very much ‘best efforts/as seen in the wild’ lists.). this problem worsens with the rise of social network agregation services, other (legitimate) content agregators, and personal content aggregating software on mobiles, tablets, etc.

idea: i am thinking of writing a script/plugin/rule to do smart lookup of ip against good bots list, & to automatically maintain that (collective) list. ideally, this is a service that someone like spamhause, or projecthoneypot should offer, since they already have the infrastructure. but, we’ll see. the script will detect traffic ‘purporting to be a SE bot, of any kind and to ban it via iptables if it isnt in the approved list/doesnt check out. the risk is in false positives and harming ones SEO. anyone any thought in this area?

b) probes & sniffers hunting out wp/bp forms, ajax ports, plugin files, forms, etc – in advance of main attack by penatration/spamming bots. typically always use swiftly rotated ip’s. many many variants out there. usually they have no luck on our sites, but that does not stop them trying in vast numbers (bot-nets, collectives? hives?) and harming out response times, etc.

idea: url obfuscation has been brought up on this forum before, particularly for eg: login, registration, admin url’s, etc. i am thinking of creating a plugin to dynamically hash encode links of choice using someething based on wp forms nonce system. not only useful for causing probes & hackers pain, but also to help thwart media thieves. obviously, scripters will soon respond by just snanning for link titles in html, so not bullet proof in any way, but they will at least be on 1-time request code, so causing them page reload every request & less sophisticted scripts will be totally wasting their own time.

anyway. these have been my thought so far. would love to hear experience/insights of others.

unfortuntely wordpress & buddypress sites in particular represent the richest of prizes for hackers, content scrapers, spammers, etc – & they r really on our case. furthermore, there is some BIG money involved, from porn to pharma to credit card fraud; that means some very smart programmers being paid excellent rates, to hack our systems, full time. add to that, the 10’s of millions of infected machines out there (often unknowingly) operating as botnet drones, trying to pernetrate our servers 24×7, steal our machine resources and steal our members personal data. it is a war of attrition.

all further experience, ideas welcome, here.

[rcain]

lol excellent answer! thanks Paul. that sort of explains why i couldnt make sense of them.

looking a bit closer, i see that:

in file: bp-groups-classes
class: BP_Group_Extension
function/method: _register

reads:

Code:

var $ display_hook = ‘ groups_custom_group_boxes ‘;

…

`if ( bp_is_group() ) {
if ( $ this->visibility == ‘public’ || ( $ this->visibility != ‘ public ‘ && $ bp->groups->current_group->user_has_access ) ) {`

…

// Hook the group home widget
if ( !bp_current_action() && bp_is_current_action( ‘ home ‘ ) )
add_action( $ this->display_hook, array( & $ this, ‘ widget_display ‘ ) );
}
}
…

– then in file bp-groups-template.php:

Code:

function bp_custom_group_boxes() {
do_action( ‘ groups_custom_group_boxes ‘ );
}

– which doesnt appeare to be called from anywhere in BP 1.6 (or its std theme)

– though does i suppose make the hook available (on group home only), were i to override the default template or call the theme/template func. bp_custom_group_boxes.

i cant think of a use for it yet either. but i’m sure i will.

thanks again

ps. code backquotes seem to be broken at the moment – thus i put spaces around $ signs etc above – hope its stil readable

[rcain]

i am about to try this: https://wordpress.org/extend/plugins/stop-spammer-registrations-plugin/
– it looks very good. anyone tried it with BP yet?

[rcain]

ditto – both the above suggestions.
i think lack of forums search + no way to find ones own posts limits the community activity here somewhat – essential for BP community to grow and develop – IMHO.

[rcain]

OK – I’ve found the answer.

please ignore my first suggestion above.

the solution is to either:

a) wordpress, admin, settings, privacy (site visibility) – allow search engines to index this site – must be set

OR

b) include the following filter in my plugin (or themes functions.php file):

`
add_filter(‘bp_is_blog_public’,’scl_bp_is_blog_public’,10,1);
function scl_bp_is_blog_public($is_blog_public) {
return true;
}
`

i chose the second method for my purposes.

note: BP seems to make very confusing/ambiguous use of the wordpress ‘blog_public’ option – in the file functions referred to in my first post above, it makes an explicit test of this option, and if not set to public then disables posts or comments from appearing as items in the activity stream.

i note that issues around this factor were raised as a ticket a year or so back and this mechanism was provided as some sort of solution (to exactly what problem i am uncertain – though luckily a filter was provided).

personally, i cant see the point of it – surely a BP option to specifically allow/disallow wordpress post and comment activities would have been far better – after all, what has WP search engine visibility got to do with BP activity streams – i have no idea! (perhaps someone could enlighten me).

meanwhile, I am considering raising a new/reopening a ticket on this (since the situation appears unchanged in the new BP 1.6 code).

be interested in others views.

hope this helps someone.

[rcain]

ps. having scanned though the buddypress source code i’ve just found a clue to what the problem maybe:

file: bp-blogs-function.php

function: bp_blogs_record_comment

– takes a parameter ‘is_approved’ – if it is false, then blog comments are NOT added to BP activity stream.

this suggests that WP comments are ONLY recorded in BP activity stream IFF comments are wide open and ‘automatically approved’/’do not require approval’. If on the other hand, comments are set as ‘requiring admin/moderator approval’ (as they most usually will be), then they will NOT EVER be posted to BP activity stream

this seems to be confirmed by the appearance of a NEW function on the forthcoming BP 1.6. The new function is:

bp_blogs_transition_activity_status

– and its purpose seems to be to enable WP comments status to appear on BP activity stream WHEN/IFF admin/moderator approves them.

please could someone confirm this is the case?

is there any workaround for BP V 1.5.4?

many thanks

[rcain]

Hi Paul,
Thank you so much for your reply. That is exactly the info i needed.
All the best with bbPress2 dev. Cant wait to give it a go when its ready for test.
Rob

[rcain]

Hi Chaps – just to let you all know, you are not alone in this quest.

We have customers running BP sites who need to ‘re-capture’ ‘lost/unconfirmed’ user registrations (attempts).

A high proportion of confirmation emails do get filtered into peoples spam buckets, bounced by firewalls and blacklists, overlooked in the inbox, whatever. Its an ongoing part of community/site admin to re-sift this list of unsuccessful registrants and correct anything impeding signup.

Its pretty apparent neither BP, nor WP ( == WPMU) address this ‘corrections process’ out of the box.

My perusal of available plugins has yielded some near fits for the job (reporting at least), but nothing yet bang-on (the ‘resend_registration_confirmation_email()’ function), which suprises me.

Looks like someone will have to lift the hood and write somethething to do it. (I have some plugins in mind to extend for that purpose – will keep you all posted )