CJK characters don't show properly on some occasions

[zyxir]

Under such directory structure:

treemacs-bug
  └abc
     └xyz我是狗

As "xyz我是狗" is the only subdirectory of "abc", treemacs will display them improperly, like this:

before.png

This seems like an encoding problem. However, if another directory is created under "abc", like "uvw", the Chinese characters will display properly:

after.png

[Alexander-Miller]

The encoding is broken in python because treemacs uses a python script to asynchronously look through the file system for directories to fold them into one.

I have no problem with CJK characters on my end

so I guess that there's something different about your setup. Encoding issues with python aren't exactly my area of expertise, so the only help I can offer are wild guesses.
StackOverflow says it might help if you put # -*- coding: utf-8 -*- of the top of the python script, so let's try that. The script file is called "treemacs-dirs-to-collapse.py", you'll find it with the rest of your treemacs installation.

[zyxir]

@Alexander-Miller Thank you for your help! Adding that encoding comment doesn't work, but now I am able to understand this problem. Maybe I'll figure out how to fix it in the future.

For encoding settings, I have these lines in my config:

(prefer-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(when (boundp 'buffer-file-coding-system)
  (setq-default buffer-file-coding-system 'utf-8-unix))

And I encounter this problem on a Windows OS with Chinese language pack (so GBK is the default encoding). From my experience, encoding is always a pain on Windows. I have never encountered encoding problems while using Emacs on Linux: everything just works fine by default.

Anyway, do you know how to make treemacs not combine directories with only one subdirectory? I think this could be a temporary fix to this problem.

[Alexander-Miller]

For encoding settings, I have these lines in my config:

Mine doesn't look much different. Another line I also have is (set-charset-priority 'unicode).

Anyway, do you know how to make treemacs not combine directories with only one subdirectory?

treemacs-collapse-dirs determines the maximum number of directories that may be flattened. You can set it to zero to disable the feature.

[zyxir]

After some tweakering, I found that "os.listdir()" returns different things in Windows Terminal and in Emacs.

In Windows Terminal, where Chinese characters are correctly displayed:

While in Emacs, with M-x run-python, Chinese characters are broken:

Even if I set-buffer-process-coding-system to utf-8, those filenames are still broken.

But at last I have finally located the problem. I'll further investigate into it.

[zyxir]

The problem is finally solved by adding this line into my configuration:

(set-language-environment 'Chinese-GB18030)

This problem only occur on Microsoft Windows, and only when language is set to Chinese (so default language environment is GBK, which is equal to GB18030). Windows has a beta setting to enable UTF-8 globally, but that will break many other things. I hate Windows for having so many shit to deal with, as most of my problems are encountered on Windows, but I cannot simply switch to Linux as many of my work softwares are exclusive to Windows.

Anyway, thank you @Alexander-Miller for your help! I think I will not be the last one to experience this encoding problem on Windows (everything else works fine! And treemacs is the first place that I encountered this issue), so maybe you can put the solution to the FAQ section or somewhere else? Because a portion of Chinese users are too shy to ask questions or post issues due to their inconfidence in their English, so they may give up Treemacs if they find this issue insolvable.

[Alexander-Miller]

so maybe you can put the solution to the FAQ section or somewhere else?

Done: 7997d74