Content Invisible to Google
I have a BP implementation at http://www.freakyrivet.com. Content on my group pages seems to be invisible to Google. I’ve gone so far as to submit a specific page to Google, then search using some of its content a few days later, and nothing is returned.
So for instance, if I search for “Defying Gravity (Making People Think You Can)”, then my homepage is returned which has that content as a link from the page, along with a couple of tweets I sent out with the title. However, the page itself (http://www.freakyrivet.com/activities/defying-gravity-making-people-think-you-can/) is not returned.
The blog part of the site (i.e. WP) returns fine from Google.
Any help? Most of my content sits in these group pages, but it seems that it is all invisible to Google.
Have you allowed 6-8 weeks for Google to index the pages? It can be that long in some cases.
Hi and thanks Henry.
I got one indexed about 3 months ago with no joy. But pages have been going in there for nearly a year with no visibility.
What’s the content of your robots.txt file?
Go to your main groups page, and an individual group page.
On both do a “view source”
Is there anything in your source that has anything to do with “robots”
(like robots meta noindex)
That would be an issue – but I bet it’s just the way buddypress handles the
“meta name description”
and html page “title” for groups, and individual member’s profile pages
G’s Wb Tools always says it’s an issue for my bp sites..
(missing completely, or duplicate your other pages, or too short)
This issue was brought up and worked on a bit a few months ago here:
(since then there was some interesting discussion in a trac where I was impressed some of the devs thought about a lot of issues with this – can’t remember the trac # atm)
I also made a suggestion about this issue here:
Although there was some great work on this issue recently, I think it’s still far from what the big G wants to see.
Given the issues with each group’s lack of good (or any) meta name description, and lame page titles, it may actually be more beneficial to block their indexing with a robots.txt file – as having a bunch of issues there affects the ranking for the rest of your site from what I gather.
@baldarab – I just took a quick look at your site – where are your “Groups” ? I don’t see any links to what I know of as groups. Do you have to register and login to get a navigation to the groups?
Hi and thank you, @djsteveb. In answer to your questions:
We don’t have a main groups page, only individual ones. Looking at the source of an individual one (e.g. http://www.freakyrivet.com/activities/defying-gravity-making-people-think-you-can/), I’ve searched and there’s no ‘robots’ text anywhere in the file.
Looking at the links you’ve sent, I note that Yoast’s plugin comes up a lot – I have it on my site as well.
I see how you are using groups – so it does appear that links to them are easily discoverable from your public facing pages.
If you do a view source and look at your code for each of your groups, you will notice that the page title is exactly the same for all of your groups ( <title>Activities – Freaky Rivet</title> ) – and there is no meta name description =
Two major issues for the big G.
Which version of buddypress are you using?
This problem concerns me so much that I have kept one of my sites with an older version of buddypress (126.96.36.199 ) with the cobbled custom code I was able to get from wpmudev ( http://premium.wpmudev.org/forums/topic/bp-meta-tite-description-for-groups-and-members-pages#post-806736 ) – the cobbled code may work the newer version of bp – but things are quite confusing with how wp has changed up some of the title thing, and bp has it’s way of things and there are possible conflicts with priority or something..
The discussion on this trac ticket( https://buddypress.trac.wordpress.org/ticket/6107 ) says things have been “fixed” – I do appreciate the devs working on it a bit – but I have a feeling the fix is more of a “make it work in a semi-okay way with a couple of setups” – not much as far as giving users real control over the titles and descriptions.
I am guessing people will point to theme designers to hook this up, and they will point to “we use wordpress’s suggested way of title-ing (wp the title or whatever) – and so it’s wp/bp issue, and descriptions should be handled my an seo plugin – but given that bp’s pages are pseudo pages without any taxonomy registered, it appears that yoast and others have so desire to work with it.
This is my current understanding, and may be not 100% accurate – what I gather after posting many questions and issues relating to this in various places.
I’d pay someone to code a plugin that gives options for bp meta stuff like the suggestion I posted before, and give it to the community. If I could understand all the bp/wp code and php I’d gladly make it. It seems the BP coders are spread a bit thin, so I have no good options for fixing this issue at the moment.
Thanks again @djsteveb. Looking through all of this, it certainly seems like a big hole still left unfilled. If Google’s not seeing it, I’m almost thinking of a HORRIBLE workaround to get a process to create a new WP post of each group with a link back to the group. I’m assuming if Google’s not indexing the page content, duplication won’t be an issue for me. But who knows……
@baldarab – if you were to create duplicate content in order for it to get indexed, I would add a robots.txt that tells google and others to not crawl / index your pages. It’s bad to have pages with duplicate titles and no meta description- as is the issue with how groups are currently handled in your case and many others… however, having a second page on the site with the exact same content in order to get it indexed may cause the dreaded duplicate content penalty, and perhaps the “thin content” – lower ranking factoring.
Certainly it would be best to get the title and meta descriptions fixed with bp groups (and member pages) as the best solution, but seeing how you are currently using the groups as basically a page with an activity, votes, comments, and related ones.. with a “sign up to add your own” option, – I wonder if it would be best to just delete those “groups”…
Instead I would consider making them just a new “page” of the site.. or if you really want to get into the whole “different users sharing different projects” – kind of instructables kind of thing – with a twist, it may be better to turn on WP multi-site – and make each activity a new “site (blog)”
– doing so would give you the same features – a video, instructions, related activities (posts and sites) – ability to sign up and create your own, comments, etc…
and that would give you the standard WP search indexing options – good page titles, descriptions, etc (not the errors that bP is currently producing for those pages)
just a thought – not exactly sure how you are using the groups thing and what the goal is with it, just glancing at the setup and thinking what I may do…
Thanks for the tip r.e. robots.txt.
As to whether we change them to pages, actually once you’re a member we do use them A LOT as groups, so making them pages would be challenging. But good thought, thank you.
@baldarab – well you can add some more original text to each of the “groups” and get some “Above the fold” – that may help a bit,
some other problems..
your sitemaps (http://www.freakyrivet.com/sitemap_index.xml ) (created by yoast) – are not showing any of your “activities / groups” – however if you create a sitemap with a tool like: https://www.xml-sitemaps.com/download/freakyrivet.15149203/sitemap.html?view=1
It does show your groups pages in the sitemap created by an actual spider – so add this to another fail with BP and yoast not playing well. I am guessing same for members page since those are “pseudo pages”.
Looking at how another sitemap maker sees your pages, it obvious that you should add some code to get original page titles and meta descriptions. You have about 70 or so pages with the same page title and description.
I just tested that code from wpmudev that I posted above with a newer BP install / updated version and it is working for me. SO you could give that a go – although you will likely have to edit the code and change the name of the profile fields to match your site profile fields (for the code to pull and create info on the member’s pages)
So – your pages are a bit thin on original content, especially above the fold, you have no or duplicate page titles and meta descriptions – and your site is instructing google and others that those pages are not important as they are not in your published sitemap.
I would tell yoast to not make your sitemap and just upload your own.
I would also – not sure why your yoast settings are having it makes mpas for categories and tags – I normally put settings to noindex nofollow all tags and category pages and author pages.. well that’s a lot of free advice – anyhow – hope that info makes sense / helps..
You have been more than generous with your advice, @djsteveb. I’ll try this out (or rather ask my developer to!). I really appreciate your taking the time and sharing your experiences of this.
It does sound like a real hole that needs to be filled in elegantly by BP / Yoast / other, as we’re having to create workarounds to get there. And the trails on this topic seem to go back for years, so it’s not an unknown issue!
Again, thank you so much for your advice. Truly appreciated.
- The topic ‘Content Invisible to Google’ is closed to new replies.