Some plugins, like BuddyPress and Events Manager, are using own email delivery solutions. By default, this option does not block their emails, as those plugins do not use default wp_mail() function to send emails.
You will need to consult with their documentation to switch them to use default WordPress email delivery.
Test emails are allowed to be sent, regardless of this option.
I don’t think there’s a bulk emailer specific for BuddyPress, not seen one anyway, the closest I’ve seen is https://wordpress.org/plugins/mass-messaging-in-buddypress/ which sends bulk messages. Of course BuddyPress users are also WordPress users so any plugin for wordpress that emails all users would work.