Skip to:
Content
Pages
Categories
Search
Top
Bottom

Hide buddypress members details from search engines


  • mattrea
    Participant

    @mattrea

    8 years ago

    Hi, I have previously been asked to set my site up so that buddyprerss members’ profiles can not be found by search engines. They want theme to only be accessible by visiting the site. I have put a robots file on my site (see below) but am still getting profiles appearing in searches – please can you tell me if there is anything I need to add to the file. Thanks, Matt.

    User-agent: *
    Disallow: */activity/p/*
    Disallow: /docs/
    Disallow: *send-invites*
    Disallow: */groups/*members*
    Disallow: */groups/*media*
    Disallow: *widget-title*
    Disallow: *members/*activity*
    Disallow: *members/*notifications*
    Disallow: *members/*friends*
    Disallow: *members/*groups*
    Disallow: *members/*docs*
    Disallow: *members/*media*
    Disallow: *acpage*
    Disallow: *messages*
    Disallow: *friends*
    Disallow: *settings*
    Disallow: /*/comment-page*
    Disallow: *register*
    Disallow: *login*
    Disallow: *profile*
    Disallow: *admin*
    Disallow: *includes*
    Disallow: *content*

Viewing 3 replies - 1 through 3 (of 3 total)

  • u_Oi
    Participant

    @arutam

    7 years, 11 months ago

    Hi @mattrea

    Thanks for the code.

    I am trying to disallow the “activity” url.

    I tried any combination:

    Disallow: */activity/*
    Disallow: /activity/
    Disallow: /*/activity
    Disallow: /activity

    But When I generate the xlm file , it recover the activity url from users:

    Ex: https://www.mydomain.com/username/activity/282/

    I copied that url and paste into Search Console to check if its “Allow” or “Disallow” and give me “Disallow” , so the question is… why the xlm generator got those urls?

    Any advice?


    metalhead
    Participant

    @aaronthomas1979

    7 years, 11 months ago

    Yes, my BP brothers, there’s a plugin called BP Simple Private and I think it will solve your problems, without having to work with robots.txt

    Prior to using BP Simple Private, my site’s member pages were way too accessible.. You could type http://mysite.com/members <- and you’d see the whole members directory.

    Prior to using BP Simple Private, you could type http://mysite.com/members/aaronthomas1979, and you’d jump straight to my profile.

    With this plugin, people (or search engine bots,) will be redirected to a page of your choosing, and the member pages will not be displayed.

    If you already have the issue of member pages being displayed in search results, there’s nothing you can do to immediately fix that, but make the change (get the plugin) and wait for those bots to crawl your site again, but next time, they won’t find those pages.

    It works for me..


    u_Oi
    Participant

    @arutam

    7 years, 11 months ago

    @aaronthomas1979 thanks for the answer.

    In fact, I did not find any solution therefore I decided to upload the xml file to the root (with “activity” urls).

    Surprisingly, when I requested a search from Google WMT after uploaded the xml file, It shows me that /activity/ url are blocked. That means, GWMT check robot.txt tool was right and the XML generator (xml-sitemaps.com) has something working incorrectly.

    Solved!

Viewing 3 replies - 1 through 3 (of 3 total)
  • You must be logged in to reply to this topic.

Topic Info

Skip to toolbar