1. adroll
  2. turtl


turtl / README

Turtl is an HTTP proxy whose purpose is to throttle connections to
specific hostnames to avoid breaking terms of usage of those API
providers (like del.icio.us, technorati and so on).

The proxy can be used either as a library embedded in another
application or it can be started stand-alone using the provided twistd

Here's an example to use turtl from inside another python application:

from twisted.internet import reactor
from turtl import proxy, config

s = """\
# defaults is pulled out of the end results
# and used to fill hostnames with missing
# parameters
defaults: &defaults
    # calls that can be made inside the interval of time
    calls: 1

    # interval is specified in seconds
    interval: 1

    # the number of concurrent calls that we are allowed to
    # have running at any given time.
    concurrency: 10

    <<: [*defaults]

    calls: 5
    interval: 1

filter-rest: True
port: 8080

urlmap, rest, port = config.loadConfigFromString(s)
f = proxy.TurtlHTTPFactory(urlmap, rest)
reactor.listenTCP(port, f)

And here's an example of a command line to start it stand alone:

twistd --logfile=/var/log/turtl.log --pidfile=/var/run/turtl.pid turtl -c /some/path/config.yaml -p turtlOne