Commits

Show all
Author Commit Message Labels Comments Date
Pablo Hoffman
added new downloader middleware: ChunkedTransferMiddleware
Pablo Hoffman
Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12
Pablo Hoffman
scrapy.utils.python: fixed bug introduced when adding support for new IPython 0.11. refs #335
Pablo Hoffman
scrapy.utils.python: fixed bug introduced when adding support for new IPython 0.11. refs #335
Pablo Hoffman
fixed backwards compability with images/media pipeline, after crawler singleton removal in r2758
Pablo Hoffman
removed support for passing more than a single spider on 'scrapy crawl' command
Daniel Graña
support s3 signing on pre and post boto v2.0
Pablo Hoffman
proper fix to what r2760 is supposed to fix
Pablo Hoffman
Some telnet console changes:
Pablo Hoffman
fixed bug in scheduler 'has_pending_requests' method which prevented spiders to close properly in some cases
Daniel Graña
Correctly handle query parameters on s3:// urls
Pablo Hoffman
Another step towards singleton removal: deprecated crawler singleton import (from scrapy.project import crawler) by a new class method that extensions can implement to receive the crawler
Pablo Hoffman
replaced DeprecationWarning by a new ScrapyDeprecationWarning category, since the default DeprecationWarning is silenced on Python 2.7+
Pablo Hoffman
added note about engine_started signal
Pablo Hoffman
scrapy.utils.sitemap: added one more case of parsing invalid sitemaps
Pablo Hoffman
scrapy.utils.sitemap: added support for parsing sitemaps with wrong namespaces, found in some bogus websites
Pablo Hoffman
moved module scrapy.core.downloader.responsetypes to scrapy.responsetypes
Pablo Hoffman
added setting to support disabling DNS cache: DNSCACHE_ENABLED
Pablo Hoffman
added MarshalDiskQueue unittests
Pablo Hoffman
removed no longer working tests from get_engine_status()
Pablo Hoffman
MarshalDiskQueue bug fix
Pablo Hoffman
scheduler: bug fix to use in-memory queues when request can't be serialized by the disk-queues
Pablo Hoffman
removed wrong blocking api usage (socket.gethostbyname()) from downloader when using CONCURRENT_REQUESTS_PER_IP
Pablo Hoffman
updated get_engine_status() after scheduler changes
Pablo Hoffman
fixed crawlspider bug introduced after scheduler refactoring
Pablo Hoffman
added marshal to formats supported by feed exports
Pablo Hoffman
Automated merge with ssh://hg.scrapy.org:2222/scrapy-0.12
Pablo Hoffman
fix start_python_console() to work with IPython >= 0.11. closes #335
Pablo Hoffman
fixed bug with scheduler.__len__() when disk queues are disabled
Pablo Hoffman
added __len__() method to scheduler
  1. Prev
  2. Next