grab / usecase / spider_url.py

import setup_script
from grab.spider import Spider, Task
import logging

class CustomSpider(Spider):
    initial_urls = ['http://ezinearticles.com/?What-I-Learnt-Losing-%A360,000-My-First-Year-as-a-Full-time-Trader&id=50262']

    def task_initial(self, grab, task):
        print grab.xpath_text('//title')

logging.basicConfig(level=logging.DEBUG)
bot = CustomSpider()
bot.setup_cache(
    backend='mongo',
    database='spider_test',
)
bot.run()
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.