Commits

Chris Adams committed 30a0489

clients: Spider.queue now accepts kwargs

This allows callers to pass in options for the underlying client

  • Participants
  • Parent commits 3ce3a5d

Comments (0)

Files changed (1)

File webtoolbox/clients.py

 
         super(Spider, self).run()
 
-    def queue(self, url):
+    def queue(self, url, **kwargs):
         """Add a URL to the queue to be retrieved"""
 
         if url not in self.url_history:
             self.url_history.add(url)
-            super(Spider, self).queue(url, request_timeout=60)
+            super(Spider, self).queue(url, **kwargs)
 
     def guess_charset(self, response):
         """