Commits

Author Commit Message Labels Comments Date
Gregory Petukhov
Generate warning if Selector class is used
Gregory Petukhov
Add MIT LICENSE file
Gregory Petukhov
Refactor Selector module. The select method does not accept queries of different types now. Selector class is deprecated. You should use more specific classes like XpathSelector, PyquerySelector
Gregory Petukhov
Add runtime_body property which links to live(dynamic) body of response. That makes sense only for javascript-eanbled transports like Kit transport.
Gregory Petukhov
Add javascript test for Kit transport
Gregory Petukhov
Tryring to fix Queue issue on travis-ci.org service
Gregory Petukhov
Fix bug: segfault when multiple Grab instance with Kit transport are created
Gregory Petukhov
Add cookies/POST/headers/user-agent features to webkit transport
Gregory Petukhov
Cookie support in Kit transport
Gregory Petukhov
Refactor method of global request counter generation
Gregory Petukhov
Fix grab_xml_processing test
Gregory Petukhov
Remove sleep method
Gregory Petukhov
Remove grab option: `strip_xml_declaration`
Gregory Petukhov
Refactor tests
Gregory Petukhov
Refactor Kit transport
Gregory Petukhov
Start working on qtwebkit transport
Gregory Petukhov
Update the update documentation script
Gregory Petukhov
Fix documentation (#1)
Gregory Petukhov
Make class-based config
Gregory Petukhov
Change lxml import in Selector module
Gregory Petukhov
Add ability to control `refresh_cache` option of all tasks with same name
Gregory Petukhov
Add new standart script `crawl` which is used to run spiders. Add ability to specify spider settings via settings file
Gregory Petukhov
Add additional argument to constructor of queue class: spider_name
Gregory Petukhov
Add get_name method to Spider class. Add grab.util.misc.camel_case_to_underscore function to build spider name from the name of its class.
Gregory Petukhov
mongo queue: drop clear_on_init & clear_on_exit options, change method of generation of collection name if it was not defined explicitly
Gregory Petukhov
Refactor spider multicurl transport
Gregory Petukhov
Add get_function method to Item which return raw functions which was used to bild FuncField field
Gregory Petukhov
Simplify func_field decorator in Item module
Gregory Petukhov
Version 0.4.11
Gregory Petukhov
Remove obsolete repair_grab function from multicurl spider transport
  1. Prev
  2. Next