1. Evgeniy Tatarkin
  2. pomp

Source

pomp /

Filename Size Date modified Message
docs
examples
pomp
tests
108 B
cover report fix
163 B
update: setup.py, examples; use async/await syntax; hide asynciotools under package
153 B
Added tag rel_0_2_0 for changeset 95937c212b0c
2.6 KB
fix pipe start/stop/process exception processing - release 0.2.1
1.5 KB
added license file, BSD with one author yet...
32 B
fix dist
2.3 KB
project moved to github
363 B
move middleware logic to engine, rename BaseDownloaderException to BaseCrawlException, rename BaseDownloaderMiddleware to BaseMiddleware, more generator like behaviour
868 B
ctrl-c signal support for concurency doaloder on python2.x
28 B
wheel distrib format
1.3 KB
update: setup.py, examples; use async/await syntax; hide asynciotools under package
1.1 KB
update: setup.py, examples; use async/await syntax; hide asynciotools under package
1.0 KB
update: setup.py, examples; use async/await syntax; hide asynciotools under package

Pomp

Attention!

Project moved to https://github.com/estin/pomp

Pomp is a screen scraping and web crawling framework. Pomp is inspired by and similar to Scrapy, but has a simpler implementation that lacks the hard Twisted dependency.

Features:

  • Pure python
  • Only one dependency for Python 2.x - concurrent.futures (backport of package for Python 2.x)
  • Supports one file applications; Pomps doesn't force a specific project layout or other restrictions.
  • Pomp is a meta framework like Paste: you may use it to create your own scraping framework.
  • Extensible networking: you may use any sync or async method.
  • No parsing libraries in the core; use you preferred approach.
  • Pomp instances may be distributed and are designed to work with an external queue.

Pomp makes no attempt to accomodate:

  • redirects
  • proxies
  • caching
  • database integration
  • cookies
  • authentication
  • etc.

If you want proxies, redirects, or similar, you may use the excellent requests library as the Pomp downloader.

Pomp examples

Pomp docs

codecov Latest PyPI version Number of PyPI downloads Have wheel License Documentation Status

Pomp is written and maintained by Evgeniy Tatarkin and is licensed under the BSD license.