[Latest BP 1.2.3, latest WordPressMU]

My users are reporting that they can post unicode content (text in other languages) in the Activity streams, but it gets stripped to ‘???????? ?????’ in forum posts. I’ve confirmed with Korean and Japanese text.

Is this one a known bug? Couldn’t find this specific issue with a search here… any ideas on how to fix it (or if other people are having the same problem)? Not a tier-1 issue for me, but it seems like it might be important for installs in non-English environments.

[@r-a-y Keymaster 14 years, 8 months ago]

It shouldn’t be because I’ve seen international languages (including Japanese) posted on testbp.org.

Does it work with the default theme?

Have you set your meta content-type tag to utf-8 in your template?

[@stwc Participant 14 years, 8 months ago]

Yep, utf-8 is set in the MU backend, and I’m showing

<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

in the head.

Switched back to default theme from my childtheme and posted non-English content: same issue. Weird.

[@r-a-y Keymaster 14 years, 8 months ago]

Ahh… didn’t read the thread title! You said it works for activity items and not forum posts! D’oh!

Try creating a new forum thread on testbp.org using a different language and see what happens.

Then report back.

[@stwc Participant 14 years, 8 months ago]

Hmm. Getting an error message on thread creation at testbp.org no matter what I do.

Tried it as a reply to an existing thread, and can confirm unicode’s borked.

Here: http://testbp.org/groups/just-trying/forum/topic/xcv/#post-113

[@r-a-y Keymaster 14 years, 8 months ago]

Hmm… can you check to see if the bbPress database is using some variation of utf8?

Also try looking through some threads on bbPress.org:

https://bbpress.org/forums/topic/internationalization-after-installation

https://bbpress.org/forums/topic/wrong-charset-cannot-specify-utf-8#post-3751

Would be great if you posted this on Trac, stwc!

[@stwc Participant 14 years, 8 months ago]

Sure, I’ll post a ticket to trac and try and have a look at the bbpress stuff.

[@stwc Participant 14 years, 8 months ago]

Tried adding the .htaccess line from that second thread. No joy.

bb-config.php is using utf-8

define( 'BBDB_CHARSET', 'utf8' );

Oh dear. In phpAdmin, I see that wp_bb_posts is using latin1_swedish_ci as it’s collation, for some unknown reason. Is that germane? I’m rapidly getting out of my depth here.

[@djpaul Keymaster 14 years, 8 months ago]

It’s a known bug. I’ve commented on your trac ticket.

[@stwc Participant 14 years, 8 months ago]

Thanks, DJPaul.

In the trac ticket you said

Please would you confirm what character set the bbPress database tables have. Is it latin1?

If that was directed to me, could you (this is embarassing) tell me how to do that? I know my way around phpMyadmin more or less, but not sure where to look.

[@adrianmirea Member 14 years, 5 months ago]

It appears that the default bbForum tables have a strange latin1 swedish Collation set, that will not work for most people.
You can use phpmyadmin or some other tool to connect to the database and manually set a different Collation, like utf8_general_ci.
To do this, you have to connect to phpmyadmin, then select the database, then select the tables one by one, click on structure, notice the swedish Collation, click on Change [ the pencil like icon ] and select utf8_general_ci from the list.
I’m sure there’s a more automated way to do this, but there are only about a dozen entries to change so I did it by hand.
It probably is a bug inside the normal installation, it should be set to utf8_general_ci by default, not swedish.

I hope it works for you, I just did this and it fixed my problems

God Bless.