Unicode Japanese Characters

[Erotemic]

The original matrix animation uses Japanese characters.

https://static.independent.co.uk/s3fs-public/thumbnails/image/2017/10/27/09/matrix-code.jpg?w968

It would be nice if unicode support was added so cmatrix could display Japanese characters.

[pberto]

Yes, and to be more precise the characters are "flipped" on the X axis

[abishekvashok]

Yes, and to be more precise the characters are "flipped" on the X axis

I managed to get the Japanese characters easily but could'nt flip them.

[ask6155]

Hey! I'd be happy to know which font is required for having the japanese characters show up?
currently when I run it with -c and symbols show up

[abishekvashok]

Hey! I'd be happy to know which font is required for having the japanese characters show up?

Any Japanese font complying with utf-8 works fine

[ask6155]

I have font installed and they work (tested japanese characters in browser & terminal)
But they don't show up in cmatrix.
Only weird characters.

[abishekvashok]

Strange. Will look into it

[grenzionky]

im having the same issue as @ask6155

my terminal (urxvt) can display japanese just fine

but this is what happens when i run cmatrix -c

[strayer]

I have the same issue on macOS with Alacritty and iTerm2:

[kamrannabi]

Hey guys, did anyone manage to get this working? I made sure to install a japenese font but at the moment cmatrix -c leaves a blank screen. Im using ubuntu server btw.

[abishekvashok]

Nope still researching on it

[DarkWiiPlayer]

I'm having this same issue. My idea was that it might be that it needs to use libncursesw instead, but even after changing that it still doesn't work.

What I've noticed, though, is that if I use configure, the result looks different than with CMake:

(configure)

(cmake)

[vperilla]

I have font installed and they work (tested japanese characters in browser & terminal)
But they don't show up in cmatrix.
Only weird characters.

same here on archlinux

[oxr463]

See also: https://help.accusoft.com/PrizmDoc/v12.1/HTML/Installing_Asian_Fonts_on_Ubuntu_and_Debian.html

It might be nice to include a list of fonts that are known to work with this.

[bickycheese]

Issue is from 2018, tried running cmatrix -c in Konsole on Manjaro with zsh as shell, still same issue.
And I don't even know how to begin looking for a Japanese font.. 🤷‍♂️

[polluks]

FYI https://stackoverflow.com/a/60555200/1430535

[abishekvashok]

FYI https://stackoverflow.com/a/60555200/1430535

Thanks for this pointer

[pouet]

Hello,

I have made the thing working. I don't have the time to make a correctly formatted pull request, but here is the trick.
After looking how to print wide characters with ncurses, I figured out that the header have a section with #if NCURSES_WIDECHAR and a function add_wch. After some tests to print the first hiragana character (0x3041 => https://www.key-shortcut.com/en/writing-systems/%E3%81%B2%E3%82%89%E3%81%8C%E3%81%AA-japanese) it finally prints correctly ! :)

A little workaround to print it is to replace all calls to addch to a add_wch if the constant is defined. Move all to a simple function and we got :

void print_char(int c) {
#if NCURSES_WIDECHAR
  cchar_t wc = {
      A_NORMAL
  };
  wc.chars[0] = c;

  add_wch(&wc);
#else
  addch(c);
#endif
}

The other thing is to link to ncursesw instead of ncurses.

$ gcc -DHAVE_CONFIG_H -I. -DNCURSES_WIDECHAR -g -O2 -MT cmatrix.o -MD -MP -MF .deps/cmatrix.Tpo -c -o cmatrix.o cmatrix.c
$ gcc -g -O2 -DNCURSES_WIDECHAR -o cmatrix cmatrix.o -lncursesw
$ ./cmatrix -c

Also note that all the characters in the range (12288; 12351 => 0x3000; 0x303f) are not displayed correctly (maybe a missing font, my browser doesn't print all characters correclty too). You can display all the kanji correctly with the range [0x4e00;0x4db5].

[space-pagan]

I was working on trying to fix this issue yesterday. The current japanese range is incorrect, as 0x3000-0x303f are Japanese punctuation characters, whereas the movie uses hiragana and katakana (0x3041-0x30ff), and kanji (scattered variously from 0x3400-0x4db0 and 0x4e00-0x9fa0) source

I'll work on a pr if no one else gets to it before I have time later this month.

[space-pagan]

Making progress, lads

Getting some weird overlap, which will require a more intricate digging into the code than I have bothered to do so far. The film frame in OP seems to also have some numbers, which would be impossible to include given the current method of random character generation, so I may work on revamping the system to work on any arbitrary character set while I'm at it.

[space-pagan]

Just noticed #112 which seems to implement a fix already... However, after cloning the repo linked in 112 and running it, I get something that looks like this:

Compared to my current progress of:

Given that, I think I'll keep working on it, unless @abishekvashok wants to merge the existing PR (in which case please let me know so I don't waste my time)

[PensiveCynic]

Getting some weird overlap, which will require a more intricate digging into the code than I have bothered to do so far.

this being what this app is - the overlap is actually kinda cool IMHO

[space-pagan]

@PensiveCynic I was thinking that as well. Would tie in well with #93 if I can get it working in a controlled manner

[PensiveCynic]

@space-pagan if you figure out what's causing it exactly maybe make it a cli option? ;-)

[space-pagan]

I know what was causing it - the use of japanese characters which took up more than one unicode char cell (not sure what the proper term for this is, double-wide seems to refer to the fact that their representation is two bytes of data), which caused column alignment to be off.

I just need to figure out how to control it. But yes, agreed, this can be a cli option.

[patatahooligan]

Just noticed #112 which seems to implement a fix already... However, after cloning the repo linked in 112 and running it, I get something that looks like this:

Compared to my current progress of:

Given that, I think I'll keep working on it, unless @abishekvashok wants to merge the existing PR (in which case please let me know so I don't waste my time)

For the record, #112 is technically what is done in the movie, because it uses half-width kana. By looking at your solution, I guess you are using the regular katakana. The problem with those is that they occupy the space of two characters (note that the spaces between columns in my solution is what cmatrix always does, not something I added, and it looks like your characters occupy what was supposed to be whitespace). So while I believe that your solution looks nicer as it is now, I hope in the future to create a mode that mixes katakana, latin script & numerals to create something that is much closer to the original movie style, and I think I need half-width kana for that.

Maybe it's worth it to merge both solutions (as different flags) so that the users can choose what they prefer. Or maybe we should use your solution for japanese-only output since it looks nicer and we'll only use half-width kana if/when I get around to implementing mixed-script (which needs some discussion & planning as it's not really compatible with the current way of choosing random characters to print as we use a single contiguous character range).

[patatahooligan]

@space-pagan Nope, I was wrong. I looked at your code and you also use the half-width forms. The difference in appearance is because you explicitly changed the spacing I guess. So your solution is basically a superset of mine because the only thing I did was enable wchar support and change the character range. You should open a PR to get this merged and we should close mine if you do that.

[space-pagan]

I had originally used full-width when taking the screenshot, but then switched to half-width later on. I think that adding custom charset support at the same time as fixing this would make the most sense so that people can chose between full width, half width, etc... I've just been busy with school and work and haven't had a chance to do anything since my last comment. I'll get on that soon :)

[abishekvashok]

Just to clarify @space-pagan,

but then switched to half-width later on

If the output is same, we could merge #112 of @patatahooligan and then we could add an additional flag for the full width characters -- just as suggested :D

What are your takes on this guys?

[space-pagan]

That should be fine. I don't remember what other changes I had made since I was working on fixing many things at once. If there's anything missing I'll add it in a pr after you merge.

[patatahooligan]

@abishekvashok

I don't know if you tried out both our versions so I'll briefly describe the situation to make sure we're on the same page.

#112 is a small fix that does the absolute minimum to get the japanese characters to print. It doesn't look great, probably because the character set is not well-suited for cmatrix's default behavior of leaving whitespace columns between the characters. I also haven't fixed problems beyond the printing of the characters, eg the rain drop's head is not white for some reason.

space-pagan's version contains my fix but also includes adjustments/fixes to make the output look nice, plus various other changes (code comments, docs, etc). These changes affect other modes of cmatrix beyond the japanese characters, eg try simply -ab without the c flag and see.

So the crucial question to decide what you want to do is: do you like space-pagan's other changes? If yes, merge directly from them and skip #112 as it offers nothing more. If you don't like their changes then it makes sense to merge #112 for now just to have something that works and discuss with them what you would want changed to merge their version.

That's how I see it, if you guys have differing opinions let me know.

[ark231]

Another choice, I hope.
I've made this from #112

[onceagainifindmyselfhere]

So, to be clear, how do i make this happen in termux

[JanzenJohn]

@ark231 I compiled #112 too, but cant seem to get it to work, Did you use a special font ? Mine looks like this (Font & Locale installed)

[patatahooligan]

@JanzenJohn For #112 you obviously need a font that has the desired unicode characters. I guess you could quickly check that by copying and pasting these characters in your terminal and seeing if they display properly. If they don't display in your browser either, then you probably don't have a unicode font for that as well.

ｱｲｳｴｵｶｷｸｹｺｻｼｽｾｿ

[hobbsecky]

This might be the easiest option:
https://github.com/GeertJohan/gomatrix

[ctrlcctrlv]

Fixed by #137

[sarensabertooth]

Just to clarify @space-pagan,

but then switched to half-width later on

If the output is same, we could merge #112 of @patatahooligan and then we could add an additional flag for the full width characters -- just as suggested :D

What are your takes on this guys?

@space-pagan ´s fork works for me on macOS when it comes to japanese characters. However the window does not resize and it draws on black background which renders my blurry/vibrancy background of my hyper terminal window just opaque. The official 2.0 -c option does not work on macOS.

[pabloab]

To be more precise, according to this scifi.stackexchange.com post they're not only katakana, but just normal Latin alphabet digits, letters, and symbols; the rest are Japanese characters (mostly half-width katakana, though there's at least one kanji in there as well). In total, that's around 67 characters.

Unfortunately it seems not possible to mirror characters with just Unicode, maybe could be used a specific font.

[i3bel]

when i use cmatrix -c in iterm2,i get a blank screen.
Apple M1 Pro
macos 13.0.1
what should i do?

[polluks2]

@Fufu-S Warp also fails.

[justinmayer]

@unrealapex: Please read this: Any updates?

[paul-uz]

v2.0 fails to render anything on MacOS 11.6.5 and iTerm 3.3.12.

In the default terminal, I get some boxes showing, but no font.

Do I need to actually install a Japanese font? If so, how do I do that exactly?

[pabloab]

Meanwhile, people subscribed to this issue might be interested on UniMatrix project.

[Manamama]

Me no coder, but Ms Bing (Precise) suggested this, which solved two probs with one (AI) stone :

The error message undefined reference to 'waddnwstr' indicates that the linker is unable to find the definition for the function waddnwstr1. This function is part of the wide-character version of the ncurses library, libncursesw2.

To resolve this issue, you need to link against libncursesw instead of libncurses3. You can do this by modifying the Makefile of cmatrix to include -lncursesw in the LIBS variable1. Here’s how you can do it:

Open the Makefile in a text editor. You might use nano for this:
nano Makefile
Find the line that starts with
LIBS =...

Change that line to:

LIBS = -lncursesw
Save the file and exit the text editor.

Run make and make install again.

Ref:
1.

...
make all-am
make[1]: Entering directory '/cmatrix'
gcc -DHAVE_CONFIG_H -I. -g -O2 -MT cmatrix.o -MD -MP -MF .deps/cmatrix.Tpo -c -o cmatrix.o cmatrix.c
mv -f .deps/cmatrix.Tpo .deps/cmatrix.Po
gcc -g -O2 -o cmatrix cmatrix.o -lncurses -lncurses
/usr/bin/ld: cmatrix.o: in function main': /cmatrix/cmatrix.c:850: undefined reference to waddnwstr'
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:428: cmatrix] Error 1
make[1]: Leaving directory '/cmatrix'
make: *** [Makefile:329: all] Error 2
gcc -g -O2 -o cmatrix cmatrix.o -lncurses -lncurses
/usr/bin/ld: cmatrix.o: in function main': /cmatrix/cmatrix.c:850: undefined reference to waddnwstr'
collect2: error: ld returned 1 exit status
make: *** [Makefile:428: cmatrix] Error 1

2. My box:

---

Operating System: Ubuntu 22.04.3 LTS x86_64
Kernel: 6.2.0-35-generic
Shell: /bin/bash 5.1.16
Python: 3.10.12

ldconfig -p | grep ncurses

libncursesw.so.6 (libc6,x86-64) => /lib/x86_64-linux-gnu/libncursesw.so.6
libncursesw.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libncursesw.so.5
libncurses.so.6 (libc6,x86-64) => /lib/x86_64-linux-gnu/libncurses.so.6
libncurses.so.5 (libc6,x86-64) => /lib/x86_64-linux-gnu/libncurses.so.5

[p1r473]

How do I install fonts?
I installed with apt install cmatrix
Get error setfont: ERROR setfont.c:402 kfont_load_font: Cannot find default font
cmatrix -lbac: