Source

metaTED / requirements.txt

Author Commit Message Labels Comments Date
Petar Marić
Updated requirements.txt file to be more specific about `shove` version
Petar Marić
Added parallelism to metaTED crawler, leading to substantial performance improvements
Petar Marić
Switched from `BeautifulSoup` to `lxml`, removed custom crawler code in favor of `lxml.html.parse`.
Petar Marić
Switched from `urllib2` to `requests`, removed page caching as it's never used
Petar Marić
Sorted requirements.txt
Petar Marić
Upgraded `shove` verion requirements, with 0.2.2 storage backends switch to `pickle.HIGHEST_PROTOCOL` by default
Petar Marić
Updated requirements.txt file to be more specific about versions
Petar Marić
Updated requirements and added cache and crawler code