Amazon IAM Users
In this assignment, we will provide keys for each program that needs to interact with Amazon AWS services. Each key will assigned to a individual user and has its own key. The user will be restrited.
client.py [-h] -s IPADDRESS:PORT -w WORKLOAD_FILE [-l LISTEN_PORT]
optional arguments: -h, --help show this help message and exit -s IPADDRESS:PORT the location of the front-end schedular. -w WORKLOAD_FILE the local file that will store the tasks that need to be submitted to the schedular. -l LISTEN_PORT if specified, the program will listen to the given port when getting result.
Don't use a proxy at the front of this program! As well as nginx, the proxy feature of this may also break up the functionality of this program.
for Extra Credits
get_task_result()gets all results with in one request (section 2.1, 5 credits)
- commandline argument
-l LISTEN_PORTwill open a listening port for a signal that tells the client that the server is ready to pickup results (section 2.1, more extra credits, amount not given)
The Front-End Schedular
HTTP 1.1 -
Submits a task to schedular.
count: The number of tasks submitted.
tasks: An array that contains all tasks. Each item is a string.
command: The command of the submitting task
cursor: A cursor which can let client to query if tasks are done.
HTTP 1.1 -
Query the progress of task completion.
tasks: An array with all tasks. If it is not completed, this will be an empty array. Each item is a dictionary.
command: The command of the task.
result: The result of the task.
output: The output of the task.
Design of Schedular
Their will be one process for doing task scheduling works(within a loop). A RESTful API interface will be set up when initializing the task schedular.
Receive the task submittion from
The schedular can receive the task submittion from the client. Then it will send the messages to SQS(queue:
cs553hw4tasks) based on those tasks.
Then the schedular will generate a cursor back to the client. The schedular will also maintains a list of cursor. Each cursor will assigned to a set of tasks.
The list will save in the DynamoDB(table:
Queue the task completion for a cursor
When the client queue the task completion with a cursor, the schedular searches the task list from the DynamoDB(table: