pip install fails for non utf-8 encoding on host
When installing savreaderwriter one has to specify a unicode compatible locale like so LANG=en_US.UTF-8
.
With the locale not set explicitly to UTF-8 such as LANG=C <path-to-venv>/bin/pip install savreaderwriter
a pip install fails with an error such as :
Copying savReaderWriter.egg-info to /app/.virtualenv/lib/python3.6/site-packages/savReaderWriter-3.4.2.egg-info
running install_scripts
writing list of installed files to '/tmp/pip-record-r__pxnb1/install-record.txt'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-o7a4ddzm/savreaderwriter/setup.py", line 129, in <module>
cmdclass=versioneer.get_cmdclass()
File "/app/.virtualenv/lib/python3.6/site-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/app/.virtualenv/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/lib/python3.6/distutils/command/install.py", line 616, in run
self.record)
File "/usr/lib/python3.6/distutils/cmd.py", line 335, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib/python3.6/distutils/util.py", line 301, in execute
func(*args)
File "/usr/lib/python3.6/distutils/file_util.py", line 236, in write_file
f.write(line + "\n")
UnicodeEncodeError: 'ascii' codec can't encode characters in position 76-77: ordinal not in range(128)
----------------------------------------
Command "/app/.virtualenv/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-o7a4ddzm/savreaderwriter/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-record-r__pxnb1/install-record.txt --single-version-externally-managed --compile --install-headers /app/.virtualenv/include/site/python3.6/savreaderwriter" failed with error code 1 in /tmp/pip-install-o7a4ddzm/savreaderwriter/
The command '/bin/sh -c python3.6 -m venv /app/.virtualenv && /app/.virtualenv/bin/pip install pip --upgrade && /app/.virtualenv/bin/pip install -r requirements/base.pip' returned a non-zero code: 1
I believe this happens in this line https://github.com/python/cpython/blob/master/Lib/distutils/file_util.py#L229 I suspect due to filename scheiß Encoding.sav so we could: - rename the file "scheiß Encoding.sav" to something ascii - advise users to use unicode.
Another example: a Dockerfile such as https://bitbucket.org/snippets/urbanslug/EeLAzk for https://github.com/onaio/onadata needs to set LANG to be set on L35 to work.
Related (and really the root cause): https://bugs.python.org/issue9561
Comments (2)
-
-
PR with a potential workaround: https://bitbucket.org/fomcl/savreaderwriter/pull-requests/6/zip-non-ascii-test-file/diff
- Log in to comment
I'm also experiencing this issue.
The workaround is insufficient for me, since I'm building a Docker image that doesn't have any locales installed. The full workaround I need is:
The linked Python bug was closed in 2011 and discussion in the bug suggests logging new bugs for new files, so I created a minimal setup.py that reproduces the issue and logged https://bugs.python.org/issue35882 .