Performance regression?

[zoomorph]

I have a benchmark which essentially runs a variant of A* path finding 1000 times (using the same coordinates and everything, yes...):

    for _ in xrange(1000):
    a = astar.find_path(start, destination)

On pypy-c-jit-66172-f4f5fae1001d-linux it runs in 0.4 sec, while in pypy-c-jit-69737-b03d8d46d83d-linux it takes 2.12 sec. This method is the main performance bottleneck in my app.

I have another algorithm that iterates outward from a starting square until it finds an empty square.

This benchmark iterates over thousands of tiles each run; it takes 6 seconds on the newer build and 3 seconds on the older one.

I use NumPy 2-d arrays to store tile terrain information for my game for memory efficiency. Both of these functions access data in the array. Perhaps the new NumPy module is the cause of this impact?

[mattip]

Could you provide enough information to reproduce this: what platform, code for find_path, a simple app that loads data and runs the loop...

[zoomorph]

Hi Matti,

I can't share find_path, sorry. Here is a very basic example I've made, hope it's useful.

import time
try:
    import numpy as np
except ImportError:
    import numpypy as np

tiles = np.zeros((256, 256), np.uint8)  # also tried uint32 - same result
tiles[5, 5] = 1

def benchmark_raw():
    start_time = time.time()
    total = 0
    for _ in xrange(500):
        for x in xrange(256):
            for y in xrange(256):
                total += tiles[x, y]
    end_time = time.time()
    print "Total:", total
    print "Time taken: %.5f" % (end_time - start_time)

if __name__ == "__main__":
    benchmark_raw()

Here are my results:

~# /root/pypy-c-jit-66172-f4f5fae1001d-linux/bin/pypy -m tpserv.test.bench
Total: 500
Time taken: 0.75626
~# pypy-c-jit-69737-b03d8d46d83d-linux/bin/pypy -m tpserv.test.bench
Total: 500
Time taken: 14.07425

Cpython 2.7.3 takes 17 seconds.

My host is running 32bit ubuntu server. Both versions of pypy were taken from: http://buildbot.pypy.org/nightly/trunk/

The most recent version has pypy-numpy-6a4e4772c3f6 installed.

[Brian Kearns]

Looking into this -- there is an identifiable regression. In the mean time, try builds around 69421... there was an improvement before regression:

$ ./pypy-c-jit-69353-a55cda9fb045-linux64/bin/pypy pypy/l.py
[2.5020031929016113, 2.4976398944854736, 2.4837629795074463]

$ ./pypy-c-jit-69421-f3e717c94913-linux64/bin/pypy pypy/l.py
[0.2546401023864746, 0.23397111892700195, 0.2335798740386963]

$ ./pypy-c-jit-69468-0e5972cfb8d4-linux64/bin/pypy pypy/l.py
[0.24732208251953125, 0.23346686363220215, 0.23294711112976074]

$ ./pypy-c-jit-69524-8df8a01902f8-linux64/bin/pypy pypy/l.py
[49.25207495689392, 49.31953716278076, 49.26526212692261]

You may want to follow https://bugs.pypy.org/issue1707.

[zoomorph]

Thank you I will.

[Ronan Lamy]

PyPy's issue 1707 has been fixed and I'm now getting timings similar to build 69421.