There are different ways to do this.
Do you have a bp-custom.php file in your plugins directory?
http://codex.buddypress.org/extending-buddypress/bp-custom-php/
If so, try adding this to it:
// Redirect logged out users viewing homepage to register page.
function custom_redirect_home_to_register() {
global $bp;
// If user is not logged in and viewing front page
if (!is_user_logged_in() && bp_is_front_page() ) {
// Redirect to registration page. Change /register to your register page slug
bp_core_redirect( get_option('home') . '/register' );
}
}
add_action( 'wp', 'custom_redirect_home_to_register', 3 );
If not, create the file and include an opening php tag before adding that code. This uses basic conditionals and can be customized however you’d like.
http://codex.buddypress.org/developer-docs/conditional-template-tags/