1. Tarek Ziadé
  2. distribute
  3. Issues
Issue #297 new

UnicodeDecodeError triggered by malformed URLs

jordanl
created an issue

When Distribute follows URL links that it finds on a package index (such as PyPi), it fails ungracefully when it encounters a malformed URL. See below for an example that can currently be reproduced when installing the s3cmd package.

The culprit seems to be a bad URL link that appears on http://s3tools.org/s3cmd

Line 732 of setuptools/package_index.py is trying to run a regular expression on "“http://ekspressfinans.net/”>lÃ¥n"

{{{ jordan@cicero:/tmp/s3cmd$ ve/bin/easy_install s3cmd Searching for s3cmd Reading http://pypi.python.org/simple/s3cmd/ Reading http://s3tools.org Traceback (most recent call last): File "ve/bin/easy_install", line 9, in <module> load_entry_point('distribute==0.6.27', 'console_scripts', 'easy_install')() File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/easy_install.py", line 1915, in main with_ei_usage(lambda: File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/easy_install.py", line 1896, in with_ei_usage return f() File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/easy_install.py", line 1919, in <lambda> distclass=DistributionWithoutHelpCommands, **kw File "/usr/lib/python2.7/distutils/core.py", line 152, in setup dist.run_commands() File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands self.run_command(cmd) File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command cmd_obj.run() File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/easy_install.py", line 350, in run self.easy_install(spec, not self.no_deps) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/command/easy_install.py", line 577, in easy_install self.local_index File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 492, in fetch_distribution self.find_packages(requirement) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 326, in find_packages self.scan_url(self.index_url + requirement.unsafe_name+'/') File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 669, in scan_url self.process_url(url, True) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 224, in process_url page = self.process_index(url, page) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 301, in process_index self.scan_url(new_url) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 669, in scan_url self.process_url(url, True) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 221, in process_url link = urlparse.urljoin(base, htmldecode(match.group(1))) File "/tmp/s3cmd/ve/local/lib/python2.7/site-packages/distribute-0.6.27-py2.7.egg/setuptools/package_index.py", line 732, in htmldecode return entity_sub(decode_entity, text) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128) }}}

Comments (0)

  1. Log in to comment