PyPy 4.0.1 on FreeBSD 10.2 fails to get the correct locale

Issue #2233 resolved
Amber Brown
created an issue

On FBSD10.2, PyPy does not seem to read the LANG variable, or even the LC_LANG variable (which isn't proper on FBSD, but anyway). This means there's never a locale, and it falls back to the default of None -- which causes everything Unicode to explode, as the default on CPython is "US-ASCII".

$ LANG=de_DE.UTF-8 /opt/pypy401/bin/pypy -c "import sys; print sys.getfilesystemencoding()"
None

And:

[oberstet@bvr-file1 ~]$ LC_LANG=C /usr/local/bin/pypy -c "import sys; print sys.getfilesystemencoding()"
None
[oberstet@bvr-file1 ~]$ export LC_LANG=C
[oberstet@bvr-file1 ~]$ /usr/local/bin/pypy -c "import sys; print sys.getfilesystemencoding()"
None
[oberstet@bvr-file1 ~]$ export LANG=de_DE.ISO8859-1
[oberstet@bvr-file1 ~]$ export LC_LANG=de_DE.ISO8859-1
[oberstet@bvr-file1 ~]$ /usr/local/bin/pypy -c "import sys; print sys.getfilesystemencoding()"
None
[oberstet@bvr-file1 ~]$ /opt/cpy2711/bin/python -c "import sys; print sys.getfilesystemencoding()"
ISO8859-1

...and...

[oberstet@bvr-file1 ~]$ /opt/pypy401/bin/pypy -c "import sys; print sys.getfilesystemencoding()"
None
[oberstet@bvr-file1 ~]$ /opt/cpy2711/bin/python -c "import sys; print sys.getfilesystemencoding()"
US-ASCII

Comments (12)

  1. Maxim Sobolev

    I think this is also related to the breakage of stuff including "setuptools", and causes pretty much any file-related function from the os module to die with "TypeError: expected string, got NoneType object" when unicode string is passed instead of the regular str() instance. As reported here for example: https://mail.python.org/pipermail/pypy-dev/2016-January/014231.html

    The following snippet illustrates the issue:

    $ pypy
    Python 2.7.10 (5f8302b8bf9f53056e40426f10c72151564e5b19, Feb 05 2016, 06:04:41)
    [PyPy 4.0.1 with GCC 4.2.1 Compatible FreeBSD Clang 3.4.1 (tags/RELEASE_34/dot1-final 208032)] on freebsd10
    Type "help", "copyright", "credits" or "license" for more information.
    >>>> fn=unicode('/tmp/test.file')
    >>>> file(fn, 'w')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: expected string, got NoneType object
    

    The same two-liner works fine with the CPython 2.7 as well as with pypy 4.0.1 on CentOS 7, so it's clearly some kind of FreeBSD-specific issue.

    -Max

  2. Kubilay Kocak

    @Amber Brown FreeBSD 11? Can you add uname -a output here so we can see exactly what revision is producing the issue

    There were some recent (last 3 months) updates to locale on CURRENT (head) which may be implicated, but we appear to have reports for 10.x showing the issue too

    At the very least at least the issue summary needs to be tweaked (not 12, not just 11) until the root cause is isolated

  3. Maxim Sobolev

    It could be totally unrelated issue, but that smells very similar. My test system is

    $ uname -a
    FreeBSD xyz.sippysoft.com 10.3-PRERELEASE FreeBSD 10.3-PRERELEASE #1 80de3e2(master)-dirty: Tue Feb  2 12:19:57 PST 2016     sobomax@xyz.sippysoft.com:/usr/obj/usr/home/sobomax/projects/freebsd103/sys/KERN01  amd64
    

    P.S. Yes, there is no FreeBSD 12 yet, so it must be a typo in a ticket name indeed, could be "11" or "10.2"

  4. Amber Brown reporter

    <p dir="ltr">That's the same issue - sys.getfilesystemencoding() is passed to encode/decode whenever you do filesystem stuff. </p> <p dir="ltr">And yes it was fbsd11, typo/conflating with 10.2. I know 10.1 worked. </p>

  5. Log in to comment