Not really. The entire group tree has to be loaded for it to work, and caching will have minimal effect. However, the slowness you’re seeing on drag and drop is the result of Javascript re-calculation, so it isn’t slowing down your server.
any possible fix for this in the future?
besides, starting the process, making a coffee, then coming back.
Which i don’t mind 
The JavaScript load was reduced significantly in the most recent release. Loading the tree certainly won’t be instantaneous, but it may be faster.
I am using the buddy press plugins BP Group Hierarchy and BP Group Organizer to add user groups for my site at freeduds.com. The organization and drag and drop design are wonderful.
However, I have been adding a lot of groups—one for each county in the US states. I am concerned that there may be memory issues because adding each county group takes a very long time, with frequent memory messages and then today the file just stopped working all together, giving me the following message
Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 107955385 bytes) in/home4/freeduds/public_html/wp-content/plugins/bp-group-organizer/includes/group-organizer.php on line 309
I can still view my groups on my web site, but since the memory message happened I am no longer able to add any more groups to the admin side of buddy press.
I was wondering if you could make a suggestion as to how I could get back the ability to add groups and also increase the speed so I can add them quickly.
Any suggestions would be much appreciated.
Have a look at this link. The guy had the same problem and increased the memory size. (Think he is still having some problems) he seems to have increased the memory in the wrong file, but it might give you some idea where to go.
http://stackoverflow.com/questions/3903363/allowed-memory-size-of-268435456-bytes-exhausted
I had a look at your site and noticed when I click on a county group, it does not click through, so its affecting that as well.
PS
here is the link from the same site for all memory issues, if you dont find the answer above, maybe one of the questions on this list will help.
http://stackoverflow.com/questions/tagged/memory-limit