BuddyPress.org

Can you confirm that they are making it back to the bbpress tables correctly? IE if you open the post in bbpress, are they already corrupt?

Or is it only when they get displayed back to Buddypress?

Also if you make a post in bbpress, does it show up in buddypress correctly or is it corrupt there also?

Answering these questions may help to narrow down the bug, or may prove to be a wide spread issue.

Sorry I couldn’t give you a solution, but first we must understand the problem at hand.

Brad

Lemme go look at this. I was just in that area about a week ago.

(later) Works for me. Lots of u-umlaut chars going to and from bp group forums, to bbpress and back again. Showing up as proper u-umlaut chars. Andy just installed some fixes lately in this area. Try upgrading to the latest bp trunk.

The fixes include an upgrade to the buddypress-enable.php plugin that runs in bbpress. Don’t forget that.

I can’t reproduce the problem you are describing.

In bp the title of a post going from bp group forums to bbpress goes through all the same filtering as the content except the following:

wpautop

make_clickable

bp_forums_filter_encode / bp_forums_filter_decode filters. The content gets that treatment.

Are your umlaut chars showing ok in bbpress if you enter them from bp forums?

Just to narrow things down. Can you temporarily comment out line 4 in bp-forums-filters.php which reads: add_filter( ‘bp_forums_new_post_text’, ‘bp_forums_filter_encode’ );

Then try adding a new post with umlaut chars. This filter needs to be there but let’s see if it is the problem.

I put in a ticket in trac on this. It’s become and ‘official’ problem.

I’ve been looking at this problem and getting frustrated. I realized finally that the filter function actually does two things:

$content = htmlentities( $content, ENT_COMPAT, "UTF-8" );

$content = str_replace( '&', '/amp/', $content );

From http://loadaveragezero.com/app/drx/Data_Formats/Character_Encoding

[…] But there are a number of other issues to deal with. In particular, because UTF-8 is a multibyte encoding, meaning one character can be represented by more one or more bytes. This causes trouble for PHP, because the language parses and processes strings based on bytes, not characters, and makes mincemeat multibyte strings – for example, by splitting characters ‘in half’, bodging up regular expressions, and rendering email unreadable.

Karl can you just comment out the following lines please:

Line 46 $content = str_replace( '&', '/amp/', $content ) in bp-forums-filters.php

and line 52 $post_text = str_replace( '/amp/', '&', $post_text ); in buddypress-enable.php on the bbpress side.

I’d like to narrow this down to the htmlentities fn.

I’m gonna help solve this or just move to a planet where only ASCII is spoken.