You have a lot of data to process.
There are many people willing to help you.
task.me.up is the place to coordinate the processing in a customizable, flexible and structured way.
This application is developed by Manas Technology Solutions for InSTEDD.org.
Initially developed for RHOK Dec 2010 http://wiki.rhok.org/TaskMeUp, 1st place at NYC.
=== Elastic Search
This application requires elastic search to be running on the host machine. By default, it should be placed in localhost:9200, but this value can be changed in settings.yml.
=== Feeds Daemon
There is a daemon running which imports feeds every a specified number of seconds (which can be set in settings.yml). The daemon is controlled by the commands:
ruby lib/daemons/feeds_ctl start|stop RAILS_ENV=environment