I would like to overwrite the bp_login_bar template tag. But I am a noob to wordpress plugins/tags. I could just rewrite it there but for easier upgrades is there a place to overwrite that like is themes/mytheme/functions.php a good place for it? And do I use add_filter or add_action or something else?
As far as overwriting the template tags are concerned,That is not allowed.You can only overwrite pluggable functions(In case of wordpress/buddypress).
As far as I see, the bp_login_bar() is a normal function(template tag), you may have a look at the code in buddypress/bp-core/bp-core-templatetags.php,so you can not overwrite it.
Yes, if you want a customize the look of the form/output generated by it, you can use a work around.
Copy the code for bp_login_bar and put it inside a new function say bp_custom_login_bar() inside your theme’s functions.php,now you may modify the code inside the bp_custom_login_bar() to make it look as you want.
The only extra work you will have to do is, you will have to replace all the calls of bp_login_bar tag in your template by bp_custom_login_bar.