Issue #212 resolved

reading bytes instead of string from distutils windows .exe in extract_wininst_cfg (easy_install.py)

Peter Zsoldos
created an issue

== Summary == when installing http://pyyaml.org/download/pyyaml/PyYAML-3.10.win32-py3.2.exe, I got a TypeError: Type str doesn't support the buffer API at cfg.readfp(io.StringIO(f.read(cfglen).split(chr(0),1)[0])), because f.read(cfglen) returns bytes and not string (b'[metadata]....') === Quickfix === {{{

!python

        read_cfg = f.read(cfglen)
        sio_param = None
        try:
          sio_param = read_cfg.split(chr(0),1)[0]
        except TypeError:
          sio_param = read_cfg.split(b'\0',1)[0].decode('ascii')
        readfp_param = io.StringIO(sio_param)
        cfg.readfp(readfp_param)

}}} == Details == === Environment === Windows XP 32 bit

Python 3.2 32 bit

distribute-0.6.15-py3.2 === To reproduce === git clone git://github.com/EnigmaCurry/blogofile.git src

cd src

python setup.py develop === Original Stacktrace === {{{

!python

Best match: PyYAML 3.10 Downloading http://pyyaml.org/download/pyyaml/PyYAML-3.10.win32-py3.2.exe Processing PyYAML-3.10.win32-py3.2.exe Traceback (most recent call last): File "setup.py", line 102, in <module> """ File "c:\Program Files\python\3.2\Lib\distutils\core.py", line 149, in setup dist.run_commands() File "c:\Program Files\python\3.2\Lib\distutils\dist.py", line 919, in run_commands self.run_command(cmd) File "c:\Program Files\python\3.2\Lib\distutils\dist.py", line 938, in run_command cmd_obj.run() File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\develop.py", line 27, in run self.install_for_development() File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\develop.py", line 105, in install_for_development self.process_distribution(None, self.dist, not self.no_deps) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 668, in process_distribution [requirement], self.local_index, self.easy_install File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\pkg_resources.py", line 544, in resolve dist = best[req.key] = env.best_match(req, self, installer) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\pkg_resources.py", line 786, in best_match return self.obtain(req, installer) # try and download/install File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\pkg_resources.py", line 798, in obtain return installer(requirement) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 584, in easy_install return self.install_item(spec, dist.location, tmpdir, deps) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 614, in install_item dists = self.install_eggs(spec, download, tmpdir) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 770, in install_eggs return [self.install_exe(dist_filename, tmpdir)] File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 849, in install_exe cfg = extract_wininst_cfg(dist_filename) File "c:\Documents and Settings\zsoldosp\Dokumentumok\tzsp-blog\virtualpython32\lib\site-packages\distribute-0.6.15-py3.2.egg\setuptools\command\easy_install.py", line 1427, in extract_wininst_cfg cfg.readfp(io.StringIO(f.read(cfglen).split(chr(0),1)[0])) TypeError: Type str doesn't support the buffer API }}}

Comments (9)

  1. Jason R. Coombs

    Thanks Peter for the report. I didn't use your quickfix because it only worked on Python 3 (where the io module was available) and failed to run at all on Python 2.5 and earlier (because of the b'' syntax). The patch I put together should run on Python 2.4 and above.

  2. Anonymous

    Your patch doesn't work on Python3.2: need two fixes

    StringIO and construction "config, ="

  3. Jason R. Coombs

    Are you sure? When I run "python setup.py build" on the tip under Python 3.2, it produces the following function in build/lib/setuptools/command/easy_install.py.

    def extract_wininst_cfg(dist_filename):
        """Extract configuration data from a bdist_wininst .exe
    
        Returns a ConfigParser.RawConfigParser, or None
        """
        f = open(dist_filename,'rb')
        try:
            endrec = zipfile._EndRecData(f)
            if endrec is None:
                return None
    
            prepended = (endrec[9] - endrec[5]) - endrec[6]
            if prepended < 12:  # no wininst data here
                return None
            f.seek(prepended-12)
    
            import struct, io, configparser
            tag, cfglen, bmlen = struct.unpack("<iii",f.read(12))
            if tag not in (0x1234567A, 0x1234567B):
                return None     # not a valid tag
    
            f.seek(prepended-(12+cfglen))
            cfg = configparser.RawConfigParser({'version':'','target_version':''})
            try:
                part = f.read(cfglen)
                # part is in bytes, but we need to read up to the first null
                #  byte.
                null_byte = bytes([0]) if sys.version_info >= (2,6) else chr(0)
                config = part.split(null_byte, 1)[0]
                # Now the config is in bytes, but on Python 3, it must be
                #  unicode for the RawConfigParser, so decode it. Is this the
                #  right encoding?
                config = config.decode('ascii')
                cfg.readfp(io.StringIO(config))
            except configparser.Error:
                return None
            if not cfg.has_section('metadata') or not cfg.has_section('Setup'):
                return None
            return cfg
    
        finally:
            f.close()
    

    Can you confirm you have the latest code and that you don't have stale code in your build/ directory? If you're still experiencing the problem, please re-open and describe which branch and what technique you're using to build and perhaps include steps to reproduce the error.

  4. Log in to comment