Support `char16_t` and `char32_t`

Issue #315 resolved
Ben Longbons
created an issue

wchar_t is inherently unportable - on some platforms, it acts like char16_t, on others it acts like char32_t. Better to support both of those directly - in particular, in ffi.string and ffi.unpack and when using unicode objects as initializer for cdata objects of the appropriate type.

Note that there is probably a lot of code that does typedef uint16_t char16_t;. But I think there's already code to ignore builtin types when there's an explicit typedef - isn't that how uintN_t work?

Comments (3)

  1. Armin Rigo

    +1, good plan. I didn't know about char16_t and char32_t being standardized nowadays.

    Note some transition issue: if some user code has got the line typedef uint16_t char16_t; in the cdef(), then it means that char16_t is handled as a 16-bit integer---this is how it works right now (it will continue to work like that, that's indeed how uintN_t works). By not including this line, the user will ask instead for it to be handled as a unicode character. It's more natural but will require fixes.

  2. Log in to comment