Add paratext

[Ewan Higgs]

Paratext claims to be fast: http://www.wise.io/tech/paratext

[Sander Maijers]

Great to see this ticket! Just commented to say as much on here http://www.wise.io/tech/paratext .. 6/13/2016, 1:33:51 PM

[Ewan Higgs]

Thanks. I changed computers so I need to rerun all the examples. And/or write a script to run them all for me.

[Damian Eads]

Hi,

Thank you for your interest.

Are you timing cold or warm reads? The "Filesystem and Disk Cache" section of Appendix D talks about how to reduce caching confounds (https://deads.gitbooks.io/paratext-bench/content/metrics.html).

Damian

[Ewan Higgs]

I measure warm reads and I don't discount the JVM startup. I basically run the read command a bunch and take the lowest value. And that's basically my use case: running scripts on the command line, tweaking a script, rerunning it, and probably dumping some plots.

[Ewan Higgs]

Paratext isn't in pypi. This is annoying. :(

[Ewan Higgs]

I tried to add Paratext, but it just doesn't work properly. There are race conditions and it can't parse an empty file.

In any event, using a single thread, I can get it to parse the test file in about half a second on a Macbook. But obviously this can't be compared to the timings coming out of wercker. It does, however, give us a ballpark idea of how quick it is. It's certainly not the fastest, but it seems pretty quick.

time python2/csvreader-paratext.py </tmp/hello.csv
5000000

real    0m0.559s

[Ewan Higgs]

Paratext code hasn't been touched in 2 months and the issue I raised hasn't been acknowledged in 14 days. I think it's abandonware.