Command palette's camel case matching does not work for non ASCII characters

[Tyriar]

From @espresso3389 in #4679:

I'm on Windows 10 and the IME is ATOK; a Japanese thirdparty IME.
Even with MS-IME, the behavior is same.
Typing "git: プル" correctly works but doing "git プル" does not work.
The search function does not seem to work with Japanese characters.

Is there an optimization to not recheck characters if the last character is replaced? While not being shown here, Japanese IMEs can also transforms multiple Japanese characters into an even smaller number of characters, by going from English -> Kana -> Kanji.

[Tyriar]

@mysticatea ran into this issue too:

[dbaeumer]

@bpasero assigning to you since this is quick box related. Marking as candidate since we should look into fixing this for 1.0. Let me know if there is something I can do from the translation side.

[bpasero]

Looked into this and I think the issue is not about finding Japanese input in general but rather how the filtering works in the command palette. There are different matchers being used and one of them is a camel case matcher that would match for a given input of git pull on Git: Pull only because we think Git: Pull is a valid camel case word with G and P. This however is not the case when using Japanese labels because our check for camel case only considers ASCII characters.

Matching on camel case in the command palette is questionable and should probably be revisited. I quickly checked to use another matcher approach with fully fuzzy matching (similar to files) and could not reproduce the issue there. However, changing our filtering for commands like that can have quite an impact and we also need to revisit the sorting in that case.

I would not do this for GA, but @egamma and @chrisdias should speak up.

[egamma]

makes sense