Need a release 1.0.8 with Python 3 fixes

Create issue
Issue #69 resolved
Victor Stinner created an issue


Could you please release a new version with my fixes for Python 3? I still need it to port the OpenStack Swift project to Python 3.

I'm my pending patch for Swift, I'm using sed on pyeclib, which is really ugly!

There is an ongoing discussing on the new embedded liberasurecode tarball:

I don't know if it's a new feature or a required fix to get a release. It looks like the current code works, but liberasurecode 1.0.8 may be reinstalled whereas it's already installed. The .so doesn't end with .so.1.0.8 on my Fedora 22, there is only a "" file.

If there are open issues which should be fixed before a release, can you please try to list them there?

Thank you!

Comments (24)

  1. Tushar Gohad

    @haypo .so symlinks are only installed as part of development (-dev/-devel) packages. Check out the output of "ldd" on a liberasurecode based program and you will see, which is the SONAME, and, which it symlinks to, is the real name. Here are some basics:

    Now about the liberasurecode tarball issue, we have explained this previously: if we don't include it in the pyeclib package, given Openstack CI/Jenkins slave doesn't have liberasurecode installed, pyeclib build and thus the check job is going to fail. Hope you see it.

  2. Victor Stinner reporter

    I did some more tests.

    When liberasurecode is installed by PyEClib, it is installed in /usr/local. The problem is that the .so file is installed in /usr/local/lib, whereas .so files should go to /usr/lib64 and /usr/local/lib64. Because of that, is unable to find liberasurecode, and so calling "python install" again will reinstall liberasurecode.

    I installed manually liberasurecode in /usr using ./configure --prefix=/usr. With that, PyEClib finds the .so(.1.0.8) file and everything goes fine. I suggest this small change in

    diff --git a/ b/
    index c62400f..fd86e66 100644
    --- a/
    +++ b/
    @@ -120,7 +120,7 @@ class build(_build):
                 if (os.path.isdir(locallibsrcdir)):
                     curdir = os.getcwd()
    -                configure_cmd = ("./configure --prefix=/usr/local")
    +                configure_cmd = ("./configure --prefix=/usr")
                     retval = os.system(configure_cmd)
                     if retval != 0:

    Note: /usr/lib64/ file exists. Now I'm no more sure that was not installed before.

    Note: there is a small typo in README:

    -This library makes use of Jesasure for Reed-Solomon as implemented by the
    +This library makes use of Jerasure for Reed-Solomon as implemented by the
  3. Tushar Gohad

    @haypo typically the "libdir" (lib or lib64) is determined automagically by autoconf. The only thing we have control over is the prefix. Changing it to "/usr" won't help much. I am looking into fixing it for the general case (by determining libdir correctly). Thanks.

  4. Victor Stinner reporter

    Victor Stinner typically the "libdir" (lib or lib64) is determined automagically by autoconf.

    autoconf uses /usr/local/lib when the prefix is /usr/local, but autoconf uses /usr/lib64 when the prefix is /usr.

    The only thing we have control over is the prefix. Changing it to "/usr" won't help much. only check /lib64 on 64-bit systems, and only /lib on 32-bit systems. For these reason, installing liberasurecode with --prefix=/usr helps. .so files are installed into /usr/lib64 and so is able to detect liberasurecode and it doesn't reinstall it.

    When using --prefix=/usr/local (current default, hardcoded in, is unable to find liberasurecode in /usr/local/lib.

    For me, there are two options:

    • (my favorite option) modify to install liberasurecode with --prefix=/usr: since PyEClib is also installed in /usr, it looks consistent to me. No?
    • modify to lookup for liberasurecode in /lib64 and /lib directories

    Example of patch for the second option:

    diff --git a/ b/
    index c62400f..268b8c2 100644
    --- a/
    +++ b/
    @@ -51,24 +51,13 @@ default_python_libdir = get_python_lib()
     default_library_paths = [default_python_libdir]
    -if platform_arch[0].startswith('64') and os.path.exists('/lib64'):
    -    default_library_paths.append('/lib64')
    -    default_library_paths.append('/lib')
    -if platform_arch[0].startswith('64') and os.path.exists('/usr/lib64'):
    -    default_library_paths.append('/usr/lib64')
    -    default_library_paths.append('/usr/lib')
    -if platform_arch[0].startswith('64') and os.path.exists('/usr/local/lib64'):
    -    default_library_paths.append('/usr/local/lib64')
    -    default_library_paths.append('/usr/local/lib')
    -if platform_arch[0].startswith('64') and os.path.exists('%s/lib64'
    -                                                        % _exec_prefix):
    -    default_library_paths.append('%s/lib64' % _exec_prefix)
    -    default_library_paths.append('%s/lib' % _exec_prefix)
    +arch64 = platform_arch[0].startswith('64')
    +for prefix in ('/', '/usr', '/usr/local', _exec_prefix):
    +    libdir = os.path.join(prefix, '/lib')
    +    libdir64 = os.path.join(prefix, '/lib64')
    +    if arch64 and os.path.exists(lib64):
    +        default_library_paths.append(libdir64)
    +    default_library_paths.append(libdir)
     # utility routine
     def _read_file_as_str(name):
  5. Victor Stinner reporter

    It became difficult to me to follow multiple discussions splitted in different issues, so I created the issue #72: " install" reinstalls liberasurecode 1.0.8 on Fedora 22.

    I also opened other issues for other problems that I saw.

  6. Victor Stinner reporter

    I uninstalled liberasurecode and I ran some tests on the master branch (rev 5347db2171d55cc4587529268c5e5c8d16a4f79d): everything is fine. "sudo python install" installs liberasurecode in /usr, is able to locate installed C headers, and PyEClib is correctly installed.

    For my usage of PyEClib: the master branch is ready for a release. Now it's up to you :-)

  7. Victor Stinner reporter

    "If there are open issues which should be fixed before a release, can you please try to list them there?"


  8. Kevin Greenan repo owner

    @haypo We are testing the master branch on a few platforms (CentOS, Fedora, Mac OS X, etc.) before tagging. I talked with @tsg- last night and he mentioned having library path issues on CentOS, but I think he had a fix. We believe that the core functionality of the library is stable (i.e. erasure coding), but dealing with platform-specific stuff is always tricky and fickle.

  9. Kota Tsuyuzaki

    @haypo @tsg- will push latest 1.0.9 to PyPI tomorrow morning in US. The reason we didn't push 1.0.8 to PyPI is that it would break Swift gate on openstack gerrit because Swift has a bunch of tests using jerasure even though 1.0.8 doesn't bandle the jerasure binary.

    Recently the restriction of PyECLib was merged on Swift master so that Swift has now a constraint PyECLib==1.0.7 so it's time to push latest version into PyPI. I'm sure he was working a lot to make the bumping smoothly. Sorry for inconvenience.

  10. Tushar Gohad

    Thanks @bloodeagle40234. @haypo Kota described the situation accurately. We have 1.0.9 release ready - would you be able to give the latest master a test (with "python install --install-liberasurecode [ yes | no ]")?

  11. sdague

    This broke OpenStack because only the master case was addressed. OpenStack includes upgrade testing, and this broke the stable/kilo branches, so all OpenStack patches are blocked.

  12. Kevin Greenan repo owner

    @tsg- @bloodeagle40234 I am missing context on what @sdague is saying above. Does this mean you need to push the requirements.txt change to other branches?

  13. Kevin Greenan repo owner

    @sdague Got it! The requirements change allowed us to break much of the common C code into a separate library. This is a stop-gap that will give time for package maintainers to post the separated C library (liberasurecode).

    @tsg- Should the default behavior install the internal liberasurecode?

  14. sdague

    For a change that doesn't retain compatibility like this, it would be better to not have it be a Z bump, but be an X or Y bump in the version number. Z bumps are typically assumed to be safe and backwards compatible.

  15. Kevin Greenan repo owner

    @sdague Hrm... This may be a bug, as it should be installing the included library by default.

    @tsg- I wonder if this is an installation issue (e.g. change of install directory).

  16. Kevin Greenan repo owner

    This installs liberasurecode by default (after removing it) on both Linux 2.6.32-573.el6.x86_64 and Mac OS 11.4.2... Not sure why this is not working for the OpenStack Gate.

  17. Tushar Gohad

    @kmgreen2, yes both Kota and I missed the stable/kilo branch. This is being addressed here - which @sdague has already approved.

    Still need to look why 1.0.9 is not installing liberasurecode by default and blowing up at gate .. we did test "pip install pyeclib" after the package was uploaded and it did work as expected.

  18. Kevin Greenan repo owner

    @tsg- Cool! Yeah, the install works fine from both pip and manually installing using Not sure what is going on here...

  19. Log in to comment