Check out buddypress/bp-groups.php/ at line 1278. There’s already a function in there to hide the activity sitewide for any group that is NOT public. I’m by no means a coder, so I’m just guessing here, but maybe you can create/assign a new status for the category groups. By snooping around in the same file, you might find other actions/hooks to keep them out of profiles too.
EDIT – Just saw @rich‘s post