dylibs on OS X, IBM Problem?

[Former user]

Trying to run 3.2.1 on OS X 10.9 I cannot fix the following Error:

OSError: dlopen(/usr/local/lib/python2.7/site-packages/savReaderWriter/spssio/macos/libicuuc48.1.dylib, 6): Library not loaded: @executable_path/../lib/libicudata48.1.dylib Referenced from: /usr/local/lib/python2.7/site-packages/savReaderWriter/spssio/macos/libicuuc48.1.dylib

[McAthy]

I managed to get it running with a lot of install_name_tool and a few extra dylibs from the SPSS application.

[Albert-Jan Roskam]

Hi,

Good to hear that you finally got it working, but too bad it did not work out of the box. The I/O modules are supposed to work without any additional libraries (that's what IBM told me). Below I pasted the output of a slightly modified version of savReaderWriter.generic.Generic._loadLibs. This is the order in which I/O libraries are loaded for each platform (I am including the other platforms for reference). The last library may not be required at all. This works for Linux, so I am surprised it does not work on OSX.

Did you try setting DYLD_LIBRARY_PATH or DYLD_FALLBACK_LIBRARY_PATH to the location where the I/O libraries live (..savReaderWriter/spssio/macos), then run the program? A previous version of the program did work this way under OS X (and Linux with LD_LIBRARY_PATH). Seems easier than install_name_tool (is that something like patchelf? sorry I am a Mac noob!)

Another option might be to upgrade the I/O libraries to v22 (this is on my to-do list too!). The Windows 64 I/O libraries that are currently included in savReaderWriter actually contain one 32-bit library. Maybe IBM made a mistake with the Mac libraries too?

Should you find a way to make this work out of the box, I would love to know about it. I was just happy I got rid of all the messing with LD_LIBRARY_PATH and equivalents.

Best wishes, Albert-Jan

-------------------------------------AIX64-------------------------------------
libicudata48.1.a
libicuuc48.1.a
libicui18n48.1.a
libzlib123spss.so
libspssdio.so.1
libspssjdio.so.1
------------------------------------HPUX_IT------------------------------------
libicudata.so.48.1
libicuuc.so.48.1
libicui18n.so.48.1
libzlib123spss.sl
libspssdio.sl.1
libspssjdio.sl.1
-------------------------------------LIN32-------------------------------------
libicudata.so.48.1
libicuuc.so.48.1
libicui18n.so.48.1
libzlib123spss.so
libspssdio.so.1
libspssjdio.so.1
-------------------------------------LIN64-------------------------------------
libicudata.so.48.1
libicuuc.so.48.1
libicui18n.so.48.1
libzlib123spss.so
libspssdio.so.1
libspssjdio.so.1
-------------------------------------MACOS-------------------------------------
libicudata48.1.dylib
libicuuc48.1.dylib
libicui18n48.1.dylib
libzlib123spss.dylib
libspssdio.dylib
libspssjdio.dylib
-------------------------------------SOL64-------------------------------------
libicudata.so.48.1
libicuuc.so.48.1
libicui18n.so.48.1
libzlib123spss.so
libspssdio.so.1
libspssjdio.so.1
-------------------------------------WIN32-------------------------------------
icudt48.dll
icuuc48.dll
icuin48.dll
zlib123spss.dll
spssio32.dll
spssjdio.dll
-------------------------------------WIN64-------------------------------------
icudt48.dll
icuuc48.dll
icuin48.dll
zlib123spss.dll
spssio64.dll
spssjdio.dll
------------------------------------ZLINUX64-----------------------------------
libicudata.so.48.1
libicuuc.so.48.1
libicui18n.so.48.1
libzlib123spss.so
libspssdio.so.1
libspssjdio.so.1

[GIDS.com]

Hi,

I am experiencing the same problems on OSX 64 bit. After appending the DYLD_LIBRARY_PATH, my python setup says it's missing libifcore.dylib.

I can't find that file anywhere on the net, it seems part of the SPSS commercial app install.

Any solutions would be welcome.

thx, Daniel

[Albert-Jan Roskam]

Hi Daniel,

Sorry for the late reply. I have been browsing around a bit and it seems tat libifcore is an Intel library for Fortran. I am 100 % sure that this was working on OSX with a much older version of savReaderWriter (I do not know what architecture was used, 32 or 64 bit). No additional libs required.

So what might go wrong: (1) the SPSS I/O libraries were upgraded to v21 and they contain some error (they do in case of Win 64). (2) savReaderWriter.generic.Generic._loadLibs somehow does not work for OSX. Try to not load "libspssjdio.dylib" as this has been a source of errors on other OSes. FYI: the aforementioned older version of savReaderWriter only loaded "libspssdio.dylib", but required that .../savReaderWriter/spssio/macos was on DYLD_LIBRARY_PATH. I hope this will make it work, though it sucks if OSX were the only OS that requires tinkering with the environment.

Albert-Jan

[GIDS.com]

Hi Albert-Jan,

Thanks for the response. I have tried excluding the libspssjdio.dylib, but no joy. Below the error message I get.

For now, I have found a different solution to my problem and written a simple parser to import .sps files with data. So no rush to fix this bug for me at least.

best regards, Daniel

/opt/anaconda/bin/python2.7 /opt/github/GDWH/spss.py NOTE. Psyco module not found. Install this module to increase reader performance NOTE. cWriterow module not found. Install this module to increase writer performance Traceback (most recent call last): File "/opt/github/GDWH/spss.py", line 3, in <module> data = spss.SavReader('/Users/Daniel/Downloads/Personen_met_indicatie_of_gebruik_ZZV__functie_en_regio.SAV', idVar = 'id') File "/opt/anaconda/lib/python2.7/site-packages/savReaderWriter/savReader.py", line 48, in init ioUtf8, ioLocale) File "/opt/anaconda/lib/python2.7/site-packages/savReaderWriter/header.py", line 20, in init super(Header, self).init(savFileName, ioUtf8, ioLocale) File "/opt/anaconda/lib/python2.7/site-packages/savReaderWriter/generic.py", line 20, in init self.wholeCaseIn = self.spssio.spssWholeCaseIn File "/opt/anaconda/lib/python2.7/ctypes/init.py", line 378, in

[Albert-Jan Roskam]

Hi Daniel,

Could you email me your code + .sav file (or a hyperlink to the Statline page where you got it from)? My email: fomcl <<at>> yahoo <<dot>> com. Thanks!

regards, Albert-Jan

[Albert-Jan Roskam]

The easiest way is to add the following two lines to ~/.bashrc and then to proceed as normal:

export DYLD_LIBRARY_PATH=/Library/Python/2.7/site-packages/savReaderWriter/spssio/macos
export LC_ALL=en_US.UTF-8

[Ömer Özak]

Hi Albert,

have you thought of including this as an I/O option for pandas? I think it would be very useful. See here. From what I see (after correcting some minor errors I had, which made me think this was not working), is that e.g. conda does not advise setting the DYLD_LIBRARY_PATH, so it would be nice to have a workaround. E.g. that symbolic links are created in the installation process. Any ideas on how to do this? The other option is to use the icu4c that is installed on the system.

[Ömer Özak]

It seems using DYLD_FALLBACK_LIBRARY_PATH instead may be a work around. Not sure how to change this so it works out of the box, especially for inclusion in other python packages like pandas. For conda there seems to exist a way to do this (see here and here), not sure how to do this more generally.

[Ömer Özak]

Update: Using DYLD_FALLBACK_LIBRARY_PATH causes issues with other packages not being able to find their own libraries. So I would not advice this route, at least on conda.