Unicode issues (Android)

[mirko77]

Hi there,

I had the app failing to load a string from the database due to Unicode chars, like:

\u2019
\u2013

Is this still related to this Android bug?

I confirm it is failing on Nougat and Oreo, DevTools report:
VM288:1Uncaught SyntaxError: Invalid or unexpected token

[brody4hire]

Hi @mirko77,

The error message indicates to me that you are using Unicode characters in the SQL query string, NOT in a parameter argument value. I highly recommend that you use parameter argument values together with ? placeholders, as already documented, for reasons including maximum flexibility and reduced vulnerability to SQL injection issues.

A related issue reported in #641 is if you would use CJK or certain other Unicode characters in a column name. (I think this case is NOT covered in the existing test suite.)

Using certain Unicode characters in either SQL statements or parameter argument values is known to be broken on Android in case of default Android database implementation, already reported in #564 in case of emojis and other 4-byte UTF-8 characters. This issue is already documented but I think not clearly enough. The workaround is to use the androidDatabaseImplementation: 2 setting when opening the database, also documented.

I hope to resolve this issue in the near future on the evcore versions and eventually on this plugin version.

[mirko77]

Thanks, @brodybits .
We are actually using parameterized queries already, still the issue is present.

I have not looked into it thoroughly honestly, as we have the chance to sanitise the source data before saving to the DB, and that fixes the issue. In our case, we have to filter out Unicode characters coming from copying and pasting from Word documents into our web forms.

We are already not allowing emoji.

Thanks again for the clarification