I thought about doing the same thing. I setup 2 instances of bbpress. One for groups and one for ‘the community’. The groups forums were hidden. I didn’t like it too much. I wanted the public groups to be public forums but read only until users joined a group. I also wanted the private and hidden groups to have hidden forums.
See this thread: https://buddypress.org/forums/topic.php?id=696