1. Sebastian Rahlf
  2. python-amazon-product-api
  3. Pull requests

Pull requests

#2 Merged at f4315b4

added retry decorator to api.call

  1. jerryji

Making api.call() more robust to battle common Amazon timeout error.

Monkey patch in work --

""" 80) ASIN: B000PSP4YO Nautica Men's N07545 Leather Square Analog Watch <urlopen error timed out>, Retrying in 3 seconds... <urlopen error timed out>, Retrying in 3 seconds... Page 9 of 56 81) ASIN: B000PDD108 Momentum Women's 1M-SP01G2 Atlas Green Dial Black Nautica Leather Watch """



Hi Rahlf,

I've made changes to the retry decorator according to your comments --

  1. retry() is now moved into api.py

  2. The retry-able exceptions are hard coded, only "timed out" from urllib2.URLError and socket.timeout for the time being

  3. I took a look at your unittests but most of them fly above my head, but I have tested the retry live many times


Update 2:

In order to make this nice functionality available for the next release, I've added it to contrib. It may still be fully integrated into the API in the future.


Comments (2)

  1. Sebastian Rahlf repo owner

    I like the idea!

    May I offer a few thoughts?

    1. I think the decorator would better live in api.py because it probably won't be needed anywhere else.
    2. Why do you pass the exception to be caught to the decorator? Wouldn't it be simpler to manage them in a constant (btw I'm a big fan of those)? Then you could do something like
                        return f(*args, **kwargs)
                    except Exception, e:
                        if e.__class__ not in self.RETRY_EXCEPTIONS:
                            raise # re-raise

    Any chance you could also provide one or two unittests?

  2. Sebastian Rahlf repo owner

    Come to think of it, it might better live in the amazonproduct.contrib package as subclass of API.

    I'm thinking of something along the lines of:

    class RetryAPI (API):
        #: Max number of tries before giving up
        TRIES = 5
        #: Delay between tries in seconds
        DELAY = 3
        #: Between each try the delay will be lengthed by this backoff multiplier
        BACKOFF = 1
        def _fetch(self, url):
            Retrieves XML response from Amazon. In case of a timeout, it will try
            :const:`~RetryAPI.TRIES`` times before raising an error.
            attempts = 0
            delay = self.DELAY
            while True:
                    attempts += 1
                    return API._fetch(self, url)
                except urllib2.URLError, e:
                    # if a timeout occurred
                    # wait for some time before trying again
                    reason = getattr(e, 'reason', None)
                    if isinstance(reason, socket.timeout) and attempts < self.TRIES:
                        delay *= self.BACKOFF
                    # otherwise reraise the original error

    BTW: Please use the Reply link below