Sure, that explains why it breaks after I remove Buddypress, but why the blank white screen when Buddypress is still installed?
I am narrowing the problem down to having too many sites, which is weird.
For example, I deleted about 200 sites and got it working again. So when I have 30 sites assigned to admin everything seems to work fine. But after I add the 200 sites back I begin to see the WSOD as I add new sites from admin.