rinohtype much slower on PyPy3

Issue #2365 new
Brecht Machiels created an issue
pip install rinohtype
wget http://docutils.sourceforge.net/docs/user/rst/demo.txt
rm -rf demo.rtc && time rinoh demo.txt

CPython 3.3.6 takes about 16 seconds on my machine. PyPy3 takes about 34 seconds.

(I installed PyPy through pyenv, which reports pypy3.3-5.2-alpha1, but pypy -V reports "PyPy 5.2.0-alpha0")

Comments (12)

  1. Maciej Fijalkowski

    rinohtype uses += to concatenate strings, which is known to be slow. concatenating unicode might be even slower that way

  2. Brecht Machiels reporter

    Hi Fijal. We talked at EuroPython last year. IIRC you tried to introduce a change (to PyPy2, I think) that should speed up the += operations. Did that make it into the released version eventually?

    I'm assuming you are talking about the += operations in rinoh.backend.pdf.Canvas.show_glyphs() and elsewhere in the PDF backend (the cos and filter modules)?

    Should I collect strings in a list and ''.join() them? Or is there a faster option? Is there a fundamental reason += operations are slower than alternatives? I'm assuming PyPy could detect and optimize these (that's what your change would do, I assume).

  3. Maciej Fijalkowski

    We tried to introduce that change, but it seems to have some overheads in other areas that we indeed ended up not merging it.

    ''.join() is a lot faster, yes, there are also pypy-only ways to do it even faster. You can also use bytearray if this is not unicode, but I'm not sure I recommend it.

    The fundamental reason is refcounting - there are ways around it but it's not super simple how we gonna make it with no cost somewhere else.

    For what is worth, one reason why we didn't merge it was that while it did help rinohtype, it didn't help it ENOUGH. I would like to revisit that benchmark at some point, especially given improvements to unicode that are in the plan for pypy 3.5 (which will also benefit pypy 2.7)

    Can we revisit that in say 2 months?

    Cheers, fijal

  4. Brecht Machiels reporter

    I tried benchmarking again with PyPy 3.3-5.5-alpha. Performance seems about on par with 5.2a (half the speed of CPython).

  5. Brecht Machiels reporter

    Correction. PyPy 3.3-5.5a is faster than 3.3-5.2a, but still much slower than CPython.

    pip install rinohtype
    wget http://docutils.sourceforge.net/docs/user/rst/demo.txt
    rm -rf demo.rtc && time rinoh -f restructuredtext demo.txt

    CPython 3.6.0 takes 14s, PyPy 3.3-5.5a takes 26s.

  6. Brecht Machiels reporter

    I don't have a native Linux install to test on. I believe benchmarking on a VM can give odd results?

    If you have the time, you can easily perform the benchmark using these commands:

    pip install rinohtype
    wget http://docutils.sourceforge.net/docs/user/rst/demo.txt
    rm -rf demo.rtc && time rinoh -f restructuredtext demo.txt
  7. Brecht Machiels reporter

    By the way, a long time ago I created a benchmark based on rinohtype since Maciej expressed interest about including it in PyPy's speed center. See https://mail.python.org/pipermail/pypy-dev/2015-August/013769.html.

    That pypy_benchmark branch is horribly outdated by now. But if you like, I'd be happy to prepare a new benchmark for inclusion in the speed center. Let me know if this is would be useful to you, and which restrictions the benchmark should adhere to.

  8. Brecht Machiels reporter

    A longer running benchmark is to render the Sphinx documentation using the rinohtype builder.

    pip install rinohtype sphinx==1.4.9 # Sphinx 1.5 and up require Python 3.4
    git clone --branch 1.4.9 https://github.com/sphinx-doc/sphinx.git
    cd sphinx/doc
    # add 'rinoh.frontend.sphinx' to extensions in conf.py
    rm -rf _build
    time sphinx-build -b rinoh -d _build/doctrees . _build/rinoh

    On PyPy 5.5.0-alpha0 this took 8m50s on my MacBook

    On CPython 3.6.1 this took 8m13s

  9. Brecht Machiels reporter

    I managed to build PyPy3 v5.7.1 on macOS using the patch from #2538. It runs the demo.txt benchmark in 22s. However, it crashes at the very end when writing the PDF file though (not sure how long that usually takes, but should be definitely less than 5 seconds). I'll create a ticket for that.

    The Sphinx benchmark (previous comment) now takes only 6 minutes. It also crashes when writing the PDF file, but that takes only a couple of seconds, so there's a significant improvement over PyPy3 5.5.0a1 (and CPython). I'm impressed!

  10. Log in to comment