| o| | | | |--- . . ..|--- |--- ,---.,---.|--- ,---.,---.,---.|__/ ,---.,---. | | | ||| | |---'| | | ,---|| | \ |---'| `---'`-'-'``---'`---'`---'` `---'` `---^`---'` ``---
A simple Twitter alert service with e-mail notifications. Sign up, register a set of keywords and receive notifications of tweets matching these keywords.
- Multiple keyword tracking, eg.:
- Selecting notification frequency between daily, hourly, instantly or never.
- Exporting tracked tweets on a CSV file.
Gustavo Brunoro (root at brunoro dot org)
There are three entities (or models), in the system:
- Tweet: a message collected from Twitter.
- Tracking: a set of keywords separed by space; a Tracking contains many Tweets; the same Tweet can belong to two different Trackings; a Tweet must match all keywords of a Tracking to be contained on it; the keywords on Tracking cannot be modified.
- User: has many Trackings; one Tracking can only belong to one user;
- Tracker: service that collects Tweets from the Twitter Streaming API according to the keywords present on all Trackings on the database. As of network capacity, only one connection is kept open with the Twitter API, and this connection collects the set of all keywords needed. A separate thread checks for changes on the set of keywords to track on the database on a fixed frequency (default 300 seconds). If there are any new keywords, the connection with the API is restarted for the new set. When a tweet is recieved, all Trackings are checked for a match. English stopwords and punctuation are not considered on keywords. The matching case insensitive. A Notifier may also be called if the matched Tracking notification option is set to instantly.
- Notifier: module that notifies matched Tweets to Users. It has two methods: one to notify one Tweet when it arrives (notification option set to instantly) or to notify a set of Tweets on reports (notification set to daily or hourly). On the case of the instant notification, the method is called from within the Tracker module. For the periodical notifications, the call is made from a task registered to run at the given frequency on the server to notify all Trackings defined at that option.
- Web Interface: module that allows Users to sign in, create and remove Trackings, edit their notification options and view its current Tweets and Trackings. The interface shows all Tweet timestamps on the user-defined timezone. There is an option to export all Tweets from a Tracking to a CSV file.
- Ruby on Rails: robust web framework for the Ruby language. Used to build the user interface and for sending mails through a SMTP server.
- MongoDB: scalable NoSQL database based on a collection/document abstraction.
- Twitter Streaming API: service from Twitter that allows collecting tweets filtered by keywords streamed through an open connection.
- Cron: UNIX daemon to schedule jobs. Used here for scheduling the frequent Notifiers.
The application is currently deployed using Capistrano over the Unicorn server.
The corresponding configuration files are found in
The tracker has to be started manually using the provided rake task
The tracker configuration (including Twitter API keys) have to be setup on
Cronjobs are setup manually using the tasks
rake notify:hourly e
The system is disposed by the standard file structure for Rails projects. Some relevant files:
Gemfile: lists the required gems for the project, which are installed by running
app/controllers: directory containing the controller classes.
app/helpers/application_helper.rb: defines the helper methods used throughout the application, such as date formatting.
app/mailers/tracking_notifier.rb: class that provides methods for frequent and instant notifications.
app/models: definition of the entities manipulated on the system.
app/views: all the code used to generate the html pages served on the webapp, the csv format used for exporting trackings and the mail html notifications.
config/application.rb: main configuration file for the project.
config/deploy.rb: definition of the tasks for deployment using Capistrano.
config/unicorn.rb: configuration of the Unicorn application server.
config/routes.rb: maps all the URLs served by the app.
config/tracker.yml: contains the Twitter API keys and parameters of the tracker.
config/mongoid.yml: parameters for connecting with MongoDB.
config/mailer.rb: ActionMailer SMTP settings.
db/seeds.rb: task that puts the database on the inital state.
lib/tasks/notify.rake: rake tasks for hourly and daily notification.
lib/tasks/track.rake: implementation of the Twitter tracker.