If it’s in the activity stream, then yes, it’s cached.
You’ll have to manually change the entries in the DB or write up a few RewriteRules in .htaccess to redirect the old “site” urls.
You could try this in .htaccess:
RedirectMatch 301 ^/site/members/(.*)$ http://example.com/members/$1
RedirectMatch 301 ^/site/groups/(.*)$ http://example.com/groups/$1
RedirectMatch 301 ^/site/blogs/(.*)$ http://example.com/blogs/$1
RedirectMatch 301 ^/site/forums/(.*)$ http://example.com/forums/$1
Change example.com to your URL.
This is if you’re still using the first method,.