Trying to us cWriterow creates large .sav files with "-Infinity" values

[John Sivak]

I don't have a good test case, but "enabling" the cWriterow shared library created a .sav file that was about 6x larger than running without cWriterow enabled. Also pspp shows "-Infinity" for fields of Numeric type with a width > 1.

Basically it looks like cWriterow() is broken.

I built cWriterow() on a CentOS 5.10 64 bit system in a Python 2.7.5 Virtual Environment using savReaderWriter 3.3.0 and Cython 0.20.1.

[Albert-Jan Roskam]

Hi John,

Thanks for letting me know. I already had a hunch that this was broken. I just fixed cWriterow. On my machine, writing 1 M rows, 100 cols takes 77 seconds with pyWriterow, but 62 seconds with cWriterow. Not the kind of improvement I was hoping for, but still not bad. I tested it with Python 2.7 and 3.3 and Cython version 0.15.1 (via apt-get) and 0.20.1 (the latter in a virtualenv, via easy_install). This was on Debian Linux 7, x86-64. I also added a way to toggle between py- and cWriterow, by setting environment variable SAVRW_USE_CWRITEROW to 'off' and 'on'. I checked the file in PSPP and it looked ok. I do not have SPSS here. There is a difference in the way SPSS and PSPP deal with (display) null bytes, but I replaced all those by spaces.

Regards, Albert-Jan Roskam

[Albert-Jan Roskam]

fixed in commit cc268b5