Hi @independent
What does your robots.txt file look like?
The robots.txt just says
User-agent: *
Disallow: /wp-admin/
which is pretty standard. The profile pages are under the ‘/members’ directory
(Here’s a link to the site)
Are you using a sitemap at all? Google might need to be told about your member pages if their spider isn’t automatically picking them up. Alternatively, check if they are indexed but just have a very low rank. Or my last thought is your theme might have some crawler-blocking script on it. Usually Flash type stuff would be a culprit
Yup. Yoast didn’t automatically include the members profiles in the sitemap it generated so we added another sub-map specifically for that. And they all have a priority of 90%…
Link to the sitemap
Some issue with the theme’s buddy press profile pages is my best bet, as you say, but we’ve not got any flash on there…
There hasn’t been any errors returned when we submitted it on Google Web Development Tools; it just seems to have indexed pretty much every page on the map bar the profile pages…
@mugwumpman I just tried searching google for “independenthostels abernethybunkhouse” and can see that the member page for abernethybunkhouse is now being returned in the list of results.
Ok, so I’m guessing that means they have been indexed in the past, but they’re not showing anywhere near where they were showing in the past, in terms of search-rankings. But crucially, looking at the Webmaster Development Tools dashboard where we submitted our sitemap, it lists only 317 out of the 658 pages as being indexed, and those pages that are refusing to be indexed are hostel profile pages.
Sitemap Indexed / Submitted
category-sitemap.xml 2 / 2
events-category-sitemap.xml 3 / 3
members-sitemap.xml 7 / 345
natters-category-sitemap.xml 2 / 2
natters-sitemap.xml 136 / 138
page-sitemap.xml 54 / 55
post-sitemap.xml 83 / 83
post_tag-sitemap.xml 2 / 2
It doesn’t flag up any errors, it merely leaves them as unindexed.
I’ve posted a thread on Google Webmaster Tools forum in case it’s an issue with the sitemap rather than buddypress, but our search engine rankings have dropped dramatically since we installed the new Buddypress-powered site. (Hopefully we can fix this without moving away from Buddypress, but it’s such a key issue to us that we may move away from it if it can’t be resolved.)
The way I see it it’s either…
- An inherent issue with using Buddypress
- An issue with the members profile template (part of the custom theme)
- An issue with the sitemap
To resolve this, you’ll need to look at the member pages which aren’t being indexed (look at their templates, URLs, sitemap entries) and work out what is different to the pages that are being indexed. I think you mentioned there could be an issue with the sitemap, so that’s another avenue to investigate. Perhaps try installing a different XML sitemap. Personally, I’ve used XML Sitemap which has always been reliable (and support from the author is very good as you’ll be able to see from the resolved tickets I’ve opened on that support forum). Then there is Google XML Sitemaps which has 1m+ active installs, so that looks popular
