ARM regressions from 2.6.1 to 4.0.1

Issue #2203 new
Pete Vine created an issue

You may find it interesting on a 1GB, cortex-a5 board, PyPy 2.6.1 is actually 2-25% faster in a few benchmarks:

bm_mdp -> 1.0193x faster
crypto_pyaes -> 1.1196x faster
django    ->   1.0296x            
hexiom2 -> 1.2309x faster
meteor-contest -> 1.0180x faster
richards -> 1.2676x faster
spectral-norm -> 1.1051x faster
telco -> 1.1002x faster
twisted_iteration -> 1.0277x faster

I'd also similarly benchmarked the slow-path versions of 32-bit x86 pypies in case you're interested and there was just one abnormal result : richards 1.1505x faster

Comments (7)

  1. Pete Vine reporter

    After repeating just those few benchmarks @ 50 iterations (2.6.1 starting first), the results held in all cases save for twisted_iteration, meaning

     ### bm_mdp ###
    Avg: 131.275326 -> 134.262058: 1.0228x slower
    Significant (t=-10.173899, a=0.95)
    
    ### crypto_pyaes ###
    Avg: 0.514372 -> 0.547363: 1.0641x slower
    Not significant
    
    ### django ###
    Avg: 0.362997 -> 0.372441: 1.0260x slower
    Significant (t=-3.409582, a=0.95)
    
    ### hexiom2 ###
    Avg: 121.106531 -> 156.107206: 1.2890x slower
    Significant (t=-43.463973, a=0.95)
    
    ### meteor-contest ###
    Avg: 0.640916 -> 0.659607: 1.0292x slower
    Significant (t=-2.170065, a=0.95)
    
    ### richards ###
    Avg: 0.036859 -> 0.047081: 1.2773x slower
    Significant (t=-5.257329, a=0.95)
    
    ### spectral-norm ###
    Avg: 0.128372 -> 0.151176: 1.1776x slower
    Significant (t=-3.986840, a=0.95)
    
    ### telco ###
    Avg: 0.273557 -> 0.292710: 1.0700x slower
    Not significant
    
  2. Maciej Fijalkowski

    it seems there were accidental regressions when merging vecopt. We're in the process of reversing those, wait a few days and recheck please. Thanks for looking into those!

  3. Pete Vine reporter

    Benchmarks (50 itrs) of 4.1.0 nightly still lagging behind 2.6.1:

    hexiom2 1.1917x

    richards 1.3400x

    spectral-norm 1.1303x

    New ones:

    go 1.1843x

    spitfire 1.0775x

  4. Log in to comment