DOS Japanese crap

[tgies]

Does anyone have any idea how to get cmd.exe in Windows XP to correctly handle programs that want to output text in a Japanese character encoding?

I'm trying to use this 6502 assembler and it's giving me messages which I can't read and can't even determine the encoding of because it's munged them so badly. It's kind of annoying.

I'm aware this is

Comments

[disastrouscode]

have you tried piping it to a text file and then opening it with a browser/advanced text editor with encoding options?

[tgies]

Yes, that's the first thing I did, but it still winds up getting munged. I've actually done a lot of work programming internationalization stuff, so I can usually identify encoding at a glance by looking at the binary or ASCII-garbage representation of the text, but this looks almost random and has a lot of features I can't identify and it's just completely ridiculous. If anyone knows what the hell DOS does to the internal representation of the encoding that screws it up so badly, that would also help me a lot.

[njyoder]

You checked if it's Unicode or Shift-JIS? I'm assuming you have all the relevant fonts installed, and if not, read daev's comment. I'm not sure if this would work, but the unix 'file' utility (also available with cygwin and possibly GNU-Win32) may be able to identify the encoding.

Do you have a link to the software? I may be able to identify it myself.

[tgies]

I tried file, yeah. Examining it more closely, it turns out it's Shift-JIS shifted three bits to the left and with the rightmost three bits replaced with 101. So yeah, it's getting totally munged somewhere along the line. Software works fine on a Japanese MS-DOS machine, so I want to blame Windows for this.

[daev]

(XP Pro)

Start > Settings > Control Panel > Regional and Language Options > Languages > Install files for East Asian Languages
Start > Settings > Control Panel > Regional and Language Options > Advanced > Language for Non-Unicode Programs > Japanese
Also make sure the various Japanese encodings are checked in the Code Page Conversion Tables, especially 20932 (JIS).

Fiddling around on that control panel will give you the ability to turn your whole system Japanese (I really think so).

Now you can start a Command Prompt and it should show up with Japanese characters, including the yen instead of the backslash in paths. If that doesn't work, I think this community has some memories related to further Japanifying your computer.

(All this is assuming your program's using something sane like ShiftJIS. If it's some wacky NEC encoding from the dark ages, all bets are off.)

[tgies]

Program is using SJIS. cmd is munging it and continues to munge it even after doing that, which I tried before making this post. Program works fine on Japanese MS-DOS machine. Confusion.

[akibare]

Right, so essentially Japanese version cmd.exe works?

Dang. I thought XP was FINALLY the windows OS where the languages just worked, but I must admit I'd not needed to use cmd.exe with Japanese (I had cygwin for another reason before I ran some command line things).

Versions of windows before XP all had things even in the GUI or web experience where the language didn't work (long story) and the answer was always "you should have bought a PC from Japan (with bona-fide Japanese-version Windows) via the resellers."

Needless to say this region encoding and specifically market segmentation business has been a lifelong peeve.

I'm sorry I haven't a suggestion but this post has been interesting information for me.

[darth_spacey]

What's the software? If it's freely available online, that might make your experience easier to replicate.

My gut reaction would be Microsoft CP 932, but that's pretty much a wild-assed guess based on the notion that the assembler was originally coded for XP (and not ported sloppily from some other OS).

You might find you have some luck with the MODE command, specifically, the MODE CON CP SELECT= form, which changes the code page that cmd.exe pretends to use. You're probably using CP 437 right now, which is guaranteed to mangle CJK and any other DBCS text.

For a list of values to try, you might start from http://msdn2.microsoft.com/en-us/library/ms776446.aspx

[tgies]

CP932 is hilariously not supported through mode or chcp.

[droid_1]

There is a thing called Japanese Windows. My son, who lives in Japan, says "English Windows can't handle Japanese path names and such."