Elastic JS

Idea

The need to analysis a huge amount of data or perform a time intensive computational task is not limited to enterprize and large organization. Individuals users for example university researchers, scientists, and small startups can easily find themselves in a situation where they need to do a data analysis task or computational operations that are simply beyond their in house resources. Complex data analysis and intensive computational tasks are commonly executed in data centers over a cluster of high end computation units. Of course, individuals and small startups can not offered building or renting this expensive infrastructure. That raises the needs for inexpensive computational platform that provide computational powers for individuals. In my opinion recent developments in web technologies, in particular HTML5 and Java scripts could enable the development of cost effective computational environment over the web using web-browsers and volunteers end users as unlimited computational power.

                                       Project Log

update (1) :

• First this week,
  I started to read about volunteers computational systems Like MRJS , Elastics HTML5
• Second,
  I wrote the project proposal See project proposal.

Update (2) :

• Create a simple calculation code
• Read about Apache Hadoop & Spark.
• Started talking spark course.

Update (3):

• A equation is send from server to the browser, browser return the result.
• Server send a data file with code, Browser compute word count and return it to server.
• Speculation: What data format will look like?

Update (4):

• Client load the code as script in browser.
• Now working on tasks distribution algorithm.

Update (5):

Initial:

• Javascript code (Task) send over web socket connection.
• Tasks & data uploaded by clients
• Server keep tracks of currently connected Clients.
• Server divide Data for an uploaded task and send data chunks to browsers with the code.

Clients:

• statue: Busy or Free
• Clients will either be in executor (working on Data) or publisher (The main client, who uploaded the task) * Data: Will be handled in CSV format

Update (6):

• First come First serve or Round Robin.
• Separating the data from the Code.

###Update (7): * looking at Round Robin Algorithm.

###Update (8): * Working at finalizing the implementation. * change project name to Elastic JS. * Testing * Fixing the design diagram.

###Update (9): * After finishing will upload code here for inspection if anyone interested.