HTTPS SSH

Amazon IAM Users

In this assignment, we will provide keys for each program that needs to interact with Amazon AWS services. Each key will assigned to a individual user and has its own key. The user will be restrited.

The Client

Usage

usage: client.py [-h] -s IPADDRESS:PORT -w WORKLOAD_FILE [-l LISTEN_PORT]

optional arguments:
  -h, --help         show this help message and exit
  -s IPADDRESS:PORT  the location of the front-end schedular.
  -w WORKLOAD_FILE   the local file that will store the tasks that need to be
                     submitted to the schedular.
  -l LISTEN_PORT     if specified, the program will listen to the given port
                     when getting result.

Required libraries

Important!

Don't use a proxy at the front of this program! As well as nginx, the proxy feature of this may also break up the functionality of this program.

for Extra Credits

  • function get_task_result() gets all results with in one request (section 2.1, 5 credits)
  • commandline argument -l LISTEN_PORT will open a listening port for a signal that tells the client that the server is ready to pickup results (section 2.1, more extra credits, amount not given)

The Front-End Schedular

Required libraries

API Design

HTTP 1.1 - POST /tasks/submit

Submits a task to schedular.

request :

  • count: The number of tasks submitted.
  • tasks: An array that contains all tasks. Each item is a string.
    • command: The command of the submitting task

response :

  • status: ok or failed
  • cursor: A cursor which can let client to query if tasks are done.

HTTP 1.1 - GET /tasks/result/{cursor}

Query the progress of task completion.

response :

  • status: completed or not completed
  • tasks: An array with all tasks. If it is not completed, this will be an empty array. Each item is a dictionary.
    • command: The command of the task.
    • result: The result of the task. success or failed.
    • output: The output of the task.

Design of Schedular

Their will be one process for doing task scheduling works(within a loop). A RESTful API interface will be set up when initializing the task schedular.

Receive the task submittion from client

The schedular can receive the task submittion from the client. Then it will send the messages to SQS(queue: cs553hw4tasks) based on those tasks. Then the schedular will generate a cursor back to the client. The schedular will also maintains a list of cursor. Each cursor will assigned to a set of tasks. The list will save in the DynamoDB(table: cs553hw4tasks).

Queue the task completion for a cursor

When the client queue the task completion with a cursor, the schedular searches the task list from the DynamoDB(table: cs553hw4tasks).