pip install fails for non utf-8 encoding on host

When installing savreaderwriter one has to specify a unicode compatible locale like so LANG=en_US.UTF-8. With the locale not set explicitly to UTF-8 such as LANG=C <path-to-venv>/bin/pip install savreaderwriter a pip install fails with an error such as :

Copying savReaderWriter.egg-info to /app/.virtualenv/lib/python3.6/site-packages/savReaderWriter-3.4.2.egg-info
    running install_scripts
    writing list of installed files to '/tmp/pip-record-r__pxnb1/install-record.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-o7a4ddzm/savreaderwriter/setup.py", line 129, in <module>
        cmdclass=versioneer.get_cmdclass()
      File "/app/.virtualenv/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/app/.virtualenv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 616, in run
        self.record)
      File "/usr/lib/python3.6/distutils/cmd.py", line 335, in execute
        util.execute(func, args, msg, dry_run=self.dry_run)
      File "/usr/lib/python3.6/distutils/util.py", line 301, in execute
        func(*args)
      File "/usr/lib/python3.6/distutils/file_util.py", line 236, in write_file
        f.write(line + "\n")
    UnicodeEncodeError: 'ascii' codec can't encode characters in position 76-77: ordinal not in range(128)

    ----------------------------------------
Command "/app/.virtualenv/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-o7a4ddzm/savreaderwriter/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-r__pxnb1/install-record.txt --single-version-externally-managed --compile --install-headers /app/.virtualenv/include/site/python3.6/savreaderwriter" failed with error code 1 in /tmp/pip-install-o7a4ddzm/savreaderwriter/
The command '/bin/sh -c python3.6 -m venv /app/.virtualenv && /app/.virtualenv/bin/pip install pip --upgrade && /app/.virtualenv/bin/pip install -r requirements/base.pip' returned a non-zero code: 1

I believe this happens in this line https://github.com/python/cpython/blob/master/Lib/distutils/file_util.py#L229 I suspect due to filename scheiß Encoding.sav so we could: - rename the file "scheiß Encoding.sav" to something ascii - advise users to use unicode.

Another example: a Dockerfile such as https://bitbucket.org/snippets/urbanslug/EeLAzk for https://github.com/onaio/onadata needs to set LANG to be set on L35 to work.

Related (and really the root cause): https://bugs.python.org/issue9561

Comments (2)