Overview

Blazing cluster app

0.1 -> create jobs to use phantomJS for rendering urls and saving them as images Unfortunately phantomJS runs in a separate child process

Basic Operation:

node cluster.js

wait for all worker to start, Redis must be running

node client.js to create jobs

Visually http://localhost:3000 you can see jobs in a queue

Feats

  • client loads up a list of sites to check out
  • each worker to process a job from the Redis Queue

Plans:

V 0.2

  • use amazon SQS and workers will listen to queue
  • worry-free queue
  • if using SQS = major rewrite to dump redis and kue, benchmark if it makes sense

v 0.3

  • once the image of website comes back transform it using nodejs gm , gm("img.png").monochrome()