Issue #1 resolved

Multibyte unicode characters are mangled

dp wiz avatardp wiz created an issue

Strings:

λ> putStrLn . unpack $ Data.EDN.encode (pack "ыюя")
"KNO"

λ> Data.EDN.decode "\"ыюя\"" :: Maybe Data.EDN.Value 
Just (String "KNO")

Chars:

λ> putStrLn . unpack $ Data.EDN.encode 'Й'


λ> Data.EDN.decode "\\Й" :: Maybe Data.EDN.Value 
Just (Character '\EM')™

Comments (2)

  1. dp wiz

    I've used utf8-strings function decode to guess unicode character boundary. It has some O(n) pieces, but ut8 characters aren't too big so it is okay.

  2. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.