1. Pypy
  2. Untitled project
  3. pypy
  4. Pull requests

Pull requests

#132 Merged
Repository
kostialopuhin
Branch
pickle-dumps
Repository
pypy
Branch
default

Speed up pickle.dumps - fixes issue 979

Author
  1. Konstantin Lopuhin
Reviewers
Description

As noted in https://bugs.pypy.org/issue979 pickle.dumps is much slower than pickle.dump, because of slow StringIO. It would be better to fix StringIO instead, and I will try, but for now this patch fixes pickle.dumps slowness by using StringBuilder instead. After this change pickle.dumps has the same speed as pickle.dump, I also run pickle app-level tests

This is PR #131 redone after advice from Brian Kearns. Now it is faster both on small and large objects:

class Point(object):
    def __init__(self, x, y):
        self.x = x
        self.y = y

def test():
    n_loops = 1
    N = 500
    data = [Point(x, y) for x in xrange(N) for y in xrange(N)]
    for _ in xrange(n_loops):
        len(pickle.dumps(data))

N = 500, n_loops = 1

3.17 s vs 22.9 s on default
2.13 s vs 8.07 s on default - with HIGHEST_PROTOCOL

N = 2, n_loops = 200000

4.63 s vs 5.02 on default

N = 2, n_loops = 2000000

3.53 s vs 3.64 on default - with HIGHEST_PROTOCOL 

And for interegs instead of instances:

N = 1000, n_loops = 10

1.86 vs 24.0 on default
1.45 vs 5.15 on default - with HIGHEST_PROTOCOL

N = 2, n_loops = 200000

2.34 s vs 3.80 on default
0.34 s vs 0.54 on default - with HIGHEST_PROTOCOL

Comments (0)