{"id":73,"date":"2006-12-03T12:44:12","date_gmt":"2006-12-03T12:44:12","guid":{"rendered":"http:\/\/dalelane.co.uk\/blog\/?p=73"},"modified":"2006-12-03T12:44:12","modified_gmt":"2006-12-03T12:44:12","slug":"supporting-different-languages","status":"publish","type":"post","link":"https:\/\/dalelane.co.uk\/blog\/?p=73","title":{"rendered":"Supporting different languages"},"content":{"rendered":"

I got an email on Friday from a German guy called Bernard. He uses my wiki note-taking app<\/a> that I wrote to play with the Windows Mobile SDK (that in itself was a surprise!). <\/p>\n

He asked if I’d add support for accented characters to it, as he (unsurprisingly, being German!) wanted to use German characters in his notes. That was an easy enough fix – just add a lookup table to the wiki markup parser which replaces characters with their HTML code equivalent<\/a>.<\/p>\n

Hurrah – I could feel suitably smug for making it a little less English-centric. <\/p>\n

A guy called Alex brought me back down to earth on Saturday morning with an email pointing out that when he uses my wiki note-taking app (wow – how many people are using this??), it displays the wrong Chinese characters in ‘View’ mode to the ones he enters in ‘Edit’ mode. Chinese? Eeek… this isn’t something I knew about. <\/p>\n

A bit of research (with Alex’s help) showed that the answer was actually pretty simple. Joel Spolsky has a brilliant introduction to the topic<\/a> which helped me get my head around it. <\/p>\n

Slightly concerningly…<\/p>\n

I have an announcement to make: if you are a programmer working in 2003 and you don’t know the basics of characters, character sets, encodings, and Unicode, and I catch you, I’m going to punish you by making you peel onions for 6 months in a submarine. I swear I will.<\/p><\/blockquote>\n

A little scary. Still, he lives in the US, so I’m probably safe. <\/p>\n

The aim of the article is to:<\/p>\n

fill you in on exactly what every working programmer should know. All that stuff about “plain text == ascii == characters are 8 bits” is not only wrong, it’s hopelessly wrong, and if you’re still programming that way, you’re not much better than a medical doctor who doesn’t believe in germs. Please do not write another line of code until you finish reading this article.<\/p><\/blockquote>\n

A little harsh? Still, any smugness I might have felt after fixing Bernard’s problem was now long gone \ud83d\ude42<\/p>\n

To get back to the problem Alex was having – the wrong characters were being displayed in the WebBrowser component (an embedded Internet Explorer control) used in the wiki’s ‘View’ mode. To quote from Joel’s article some more:<\/p>\n

Almost every stupid “my website looks like gibberish” or “she can’t read my emails when I use accents” problem comes down to one naive programmer who didn’t understand the simple fact that if you don’t tell me whether a particular string is encoded using UTF-8 or ASCII or ISO 8859-1 (Latin 1) or Windows 1252 (Western European), you simply cannot display it correctly or even figure out where it ends.<\/p><\/blockquote>\n

This was the problem – the HTML that my parser was spitting out didn’t identify an encoding. The answer seems to be easy. I can either:<\/p>\n

    \n
  1. Tell users to right-click on the WebBrowser control, and use the ‘Encoding’ menu to choose something that makes it look right. This should work, but it’s a little icky. Or:<\/li>\n
  2. Add something to the header of the HTML file that the Wiki markup parser produces which identifies UTF-8 as the encoding:\n
    <html>\r\n<head>\r\n<meta http-equiv=\"Content-Type\" content=\"text\/html; charset=utf-8\"><\/code><\/pre>\n<\/li>\n<\/ol>\n
    It seems that UTF-8 is a good choice<\/a> for any web-app which uses non-European characters. <\/p>\n
    This has been interesting. Globalization\/Localization wasn’t at the front of my mind when hacking together an app for my own personal use, so I think I’ve got a reasonable excuse for not looking into this earlier. But I’m glad I can do a few simple things to make it a little better.<\/p>\n","protected":false},"excerpt":{"rendered":"
    I got an email on Friday from a German guy called Bernard. He uses my wiki note-taking app that I wrote to play with the Windows Mobile SDK (that in itself was a surprise!). He asked if I’d add support for accented characters to it, as he (unsurprisingly, being German!) wanted to use German characters […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[7],"tags":[],"class_list":["post-73","post","type-post","status-publish","format-standard","hentry","category-code"],"_links":{"self":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/73","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=73"}],"version-history":[{"count":0,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/73\/revisions"}],"wp:attachment":[{"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=73"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=73"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dalelane.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=73"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}