Unicode support in Winboard/XBoard

[cday]

Hello,
I am the main developer for the raptor interface for fics http://code.google.com/p/raptor-chess-interface/. One of the features I have added to raptor is unicode support via html characters. It has turned out to be fairly popular. There are quite a few people on fics which speak foreign languages and it is a convenient way for them to communicate. If you finger maciejg you will see the standard. It uses the html unicode characters with the x after the &# i.e. <pre>&#x394;</pre> is the greek delta. It is also nice because you can send ucicode symbols for chess pieces over fics http://en.wikipedia.org/wiki/Chess_symbols_in_Unicode for the unicode characters (e.g. ♔). It was easy to support this in Raptor because its java and java has very nice unicode support. I know winboard/xboard uses c/c++ but if you would like to see how I did it:
http://code.google.com/p/raptor-chess-i ... Utils.java

This method translates from the maciejg html unicode characters into unicode.
line 669: public static String maciejgFormatToUnicode(String inputString)

This method translate from unicode into the maciejg html unicode characters.
line 541 public static void filterOutbound(StringBuilder message)

As you can see it really wasn't that difficult to implement and is a great feature. It may be more difficult to do it in xboard/winboard since its unicode but if it is it easy it would certainly be nice to have more interfaces supporting it. Quite a few Raptor users are using it.

Carson (CDay on fics)

[H.G.Muller]

Let me see if I understand what you are saying:

Raptor translates non-ASCII characters that people type into escape sequences consisting of pure ASCII characters, before it sends them to FICS (as part of tell commands, etc.). The escape sequences you have chosen for this are the those that html uses for these foreign characters. The FICS server knows nothing of this, and just passes the ASCII message. So when the receiver of the tell would use an interface that recognizes these escape sequences, and displays them as the corresponding character, such a ser would see exactly what the other one typed. Correct?

Why don't you simply send the unicode itself? I suppose that is more universal than using html out of context. E.g. when people use telnet to log in, I suppose that telnet could be configured to emulate a unicode terminal, but I am not so sure it could understand these html escape sequences.

A for WinBoard / XBoard: at this time I have no idea how difficult it is to display the non-ASCII characters. I.e. if the text-output widgets recognize unicode, or that they need some non-standard way to encode the required non-ASCII characters (if they can do that at all). But that might only be a translation problem. But if the fonts turn out to support unicode, (of the type where the ASCII-128 set maps to single-byte codes, and is thus upward-compatible with ASCII) it would be nice if no translation was needed at all (i.e. if the unicode went directly over the ICS).

[edit] OK, I see that although FICS does correctly echo non-ASCII characters in commands, it refuses them in tell messages. So this is why you need to escape them.

[cday]

Actually fics will disconnect you if you send them. You have to convert the unicode into the supported fics character set to avoid this, thus maciejgs proposal of using the html unicode chars with the x character.

[H.G.Muller]

No, it doesn't. That is just a bug in timeseal. Tim Mann has a timeseal that does not suffer from that.

If you log in through telnet, and send "tell hgm Müller", FICS replies with a complaint about an "unprintable character in the message", but you stay logged in and get a new prompt.

But I guess that really amounts to the same thing, because that i the place we want to send them. So we indeed need to escape them.

I cannot right away tell you if I will be able to do that. At this point I have no idea if you even can type, say, Greek characters into the WinBoard ICS window input field, and what kind of character codes that would generate. Internally WinBoard uses strings of characters, not wide characters, I am pretty sure of that.

[cday]

Hopefully its not to difficult for you to implement. Its a popular Raptor feature and I am sure your user base would enjoy it if you add it in. I am currently on a crusade to get all interface authors to add it in.

Thanks,
CDay

[H.G.Muller]

Well, perhaps we can make a trade. Can you support WinBoard engines in Raptor?

[cday]

Haha, fair enough. I plan to add support for winboard/xboard engines eventually. It seems like the way to go for variants for sure. It would also be nice to support crafty.

[Eric Mullins]

It would have to be done in such a way to not break Winboard with Windows 9x.

There is a package to add support called unicows. It works for 95/98/Me, so it would solve that problem. However, asking people to download some microsoft API to be able to run a unicode only app might create more problems than are solved.

Sorry for not linking to unicows-- I'm not going to link to MS download because they change the links all the time, better to do a search for unicows.