Re: http://USER1.domain.com not working at all!
well,so your subdomain style issue is solved.
For the random non existing subdomain behavior is expected by wpmu.since you are using buddypress default theme ,so you will not be able to see wp-signup.php and it is one of the most discussed topic here in forum.just search it.
If you are trying to create forums (as you may be thinking of standalone forum in bbpress),this is not implemented in that way in buddypress.In buddypress ,the bbpress (internal integration) is used for creating forums with groups and they do not have a separate admin.You can enable forum from group admin screen for a particular group.
As far as I know and my experiences with shared hosting,you won’t be able to install wordpress mu on addon domains.If you are able to install wpmu on addon domain ,then buddypress should work without any problem.