1. Chris Adams
  2. webtoolbox

Commits

Chris Adams  committed 30a0489

clients: Spider.queue now accepts kwargs

This allows callers to pass in options for the underlying client

  • Participants
  • Parent commits 3ce3a5d
  • Branches default

Comments (0)

Files changed (1)

File webtoolbox/clients.py

View file
 
         super(Spider, self).run()
 
-    def queue(self, url):
+    def queue(self, url, **kwargs):
         """Add a URL to the queue to be retrieved"""
 
         if url not in self.url_history:
             self.url_history.add(url)
-            super(Spider, self).queue(url, request_timeout=60)
+            super(Spider, self).queue(url, **kwargs)
 
     def guess_charset(self, response):
         """