COMP 512: Distributed Systems
Fall 2011 - McGill University
Authors: Alexandre Beaulieu
There are three separate programs included in the project.
ResourceManager.jar - Manages a single type of resource.
Server.jar - Acts as middleware between the client and resource manager.
Client.jar - Client to the complete system.
The deployment process of a distributed application is fairly big and complex.
To that end, we have written a fairly sophisticated deployment script and
execution script that will take care of setting up everything for us. The
main steps consist of checking out from source control, compiling the java
source files, generating RMI stubs and creating jar files for all components.
Once this is done, the stubs and class files are uploaded to the RMI webserver
and the jar files are uploaded to the production server (here mimi.cs.mcgill.ca)
under the folder cs512-deploy/
Our run script then fires a multiscreen session with each component running
over ssh on a different servers. The breakdown is as follows:
- Middleware on mimi
- 2x RM on willy
- 1x RM on ubuntu
- Client on ubuntu
The result of this is that after developing from our home machines, we can
issue "./run.sh" from the repository root and wait for everything to launch.
A failure in any part of the deployment results in a launch abortion.
We decided to use our own port and manage the rmiregistry ourselves. The run
script takes care of ensuring the rmiregistries are running on the production
machines. We opted to use port 13337. It is hard coded as a variable inside
the ResourceManager interface.
Resource Managers take instead a parameter specifying the name they should
bind to in the registry. This allows us to run more than one RM per host,
which makes it easier to test.
The middleware server expects to find hosts for 'Car', 'Flight' and 'Room'
as consecutive parameters. This is also hardcoded in the run script.
The middleware stores all customers and ensures that they are consistent
across all resource managers. The customer entries on the middleware contain
no information about reservations or anything else, they merely serve as a
way to keep data consistent across the resource managers.
When a query is made to the middleware, it will forward the relevant queries
to the appropriate resource manager. If a query requires more than one type
of resource, the middleware will query each resource manager individually to
gather the required information. This is the case, for example, when booking
We only allow one active transaction at any given time on the client
Transactions are distributed, with each resource manager having its own
lock manager. The middleware simply forwards all operations to the
appropriate RM and waits for a reply. A transaction provides a tick()
function that is called every time the transaction performs an operation
on an RM. The transaction manager is responsible for automatically
aborting any transaction that has not ticked within a certain timeout.
The middleware has an additional lock manager that it uses to keep track
of the customer table locks on its side. This is done to ensure
customers do not go out of sync.
In the client-RM version, the transaction manager is kept on the RM.
In the client-middleware-RM version, all transactions are kept only on
** IMPORTANT ** All our transactional code assumes that the connections
between Middleware and RM never fails.
To initiate a transaction on the client, simply execute the "start"
command. The transaction manager will generate a transaction ID and
the client will print it, indicating that the transaction has started.
Once a transaction is underway, any commands have an implicit
transaction ID parameter passed to them as first argument. The user does
not need to type it.
To commit a transaction, simply type "commit".
Similarly, to abort a transaction, type "abort".
A new transaction can then be started again using the "start" command.
Note: Transactions have a 5-second inactivity timeout. The user must be
fast while typing commands, or have them prepared beforehand for easy
execution. Alternatively, the user may also use the benchmark command
to run a series of commands at once.
A benchmark framework is included along with the client. The CLI for the
client provides a function that can be used to call individual
benchmarks by their names. The package comes with three benchmarks that
showcase what can be done. You can get help on the benchmark command
when runing the client by typing
at the command line prompt. A list of available benchmarks will be
displayed and you can then run one. The defaults provided are:
default - A simple benchmark that always fails, mostly to demonstrate
the benchmark capability without running any actual tests.
responsetime - Tests the response time for standard transactions both
for a single resource manager and for multiple resource managers.
load - Tests the response time of the system under various levels of
strain and provides an output file `load_data.csv` that can be used to
plot the outcome.
Note that benchmarks can be long running and CPU straining.
To compile the project manually, navigate to the root of the repository and issue
Assuming you have apache-ant installed, this should compile all the source files
and package the jar files. The resulting jars and class files will go to
This only compiles the source and does not prepare the server for production.
Will follow all the steps necessary to ready the application for production.
Note that SSH keys and passwords might be required if the machine running the
setup does not have the required information available.
While it is possible to run the components manually from the production servers,
it is recommended to use
$ ./run.sh [CS account username]
Instead, which will perform deployment of necessary components, setup the
remote production environment and start a nice interface with all components
running together as described in the project overview.
This script requires a patched version of GNU screen, with vertical splitting
support. Recent versions of Debian-based distributions have this patch already.