1. Robert Moore Avatar Robert Moore
  2. Untitled project
  3. gnrs

Wiki

Clone wiki

gnrs / Orbit Experiments with OMF

View History

This article has been completely replaced as of June 20, 2013. Previous versions are available in the history.

#Preparing Your Domain# This guide will assume you are running on one of the ORBIT lab domains. Ideally, you will want to run experiments on the Main Grid, while doing testing/practice on one of the Sandbox domains (sb1, sb2, . . . sb9). The guide also assumes the following software packages are available to you:

  • OMF-5.4
  • GNU Octave 3.2.4

If one or more of the utilities listed above is not available to you, please contact your administrator to request installation.

##Loading a Node Image## If it is available, you may use the GNRS testing image available on the ORBIT repository and named gnrs-1.0.ndz. If this disk image is not available to you, you can create your own.

# On the Grid, use only Gen-3 nodes
omf load -i gnrs-1.0.ndz -t inventory:topo:gen3

# On a sandbox, image everything
omf load -i gnrs-1.0.ndz -t system:topo:all

##Preparing Experiment Files## Let's put our experiment into a directory called my-exp-test in home.

# Change to our home directory
cd 
# Create experiment directory
mkdir my-exp-test
cd my-exp-test

Now let's download the files we need to prep our experiment

# The main GNRS compiled JAR file. Contains server and client code
wget https://bitbucket.org/romoore/gnrs/downloads/gnrs.jar
# Scripts for running on nodes
wget https://bitbucket.org/romoore/gnrs/raw/master/jserver/gnrsd
wget https://bitbucket.org/romoore/gnrs/raw/master/jserver/gbench
wget https://bitbucket.org/romoore/gnrs/raw/master/jserver/ggen
wget https://bitbucket.org/romoore/gnrs/raw/master/jserver/gnrsd.init
# Client trace generator
wget https://bitbucket.org/romoore/gnrs/raw/master/jserver/genTrace.sh
# Topology generation files
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/topoGen.sh
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/degreeTopoGenerator.m
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/foo.sh
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/jellyfishTopoGenerator.m
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/locTopoGenerator.m
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/locTopoProc.m
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/sizeTopoGenerator.m
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/topoGen.sh
wget https://bitbucket.org/romoore/gnrs/raw/master/src/tools/topology_generator/topoGenerator.m
# The OMF experiment files
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/as-binding.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/gnrs_config.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/gnrs_group.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/gnrs_node.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/resources.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/simple.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/statscollect.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/utils.rb
# Preprocessing script
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/prepare.sh
# GNUPlot graphing script (Optional)
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/res-graph/process.rb
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/res-graph/plot-ins.gp
wget https://bitbucket.org/romoore/gnrs/raw/master/omf/res-graph/plot-lkp.gp

Once you have all the files above, the next step is to create a client trace. For this example, we will generate a simple client trace of 10,000 GUID inserts/lookups. Depending on the message delay, it should take 1-2 minutes to execute this trace. The trace files MUST BE NAMED client_1.trace, client_2.trace, etc. This is because prepare.sh expects them to be named as such.

# Generate a trace file
chmod +x genTrace.sh
./genTrace.sh 1 10000 >client_1.trace
# Check that the file is correct
wc -l client_1.trace # should print 20,000

Now we need a topology. A topology basically determines the network configuration - which ASes are directly connected and what the delays are. To build a topology, we will rely on Octave to do some fancy work for us. For this experiment, we will generate a simple topology of 4 Autonomous Systems based on largest announced prefixes.

# The script below will generate a topology based on the largest prefixes. Requires GNU Octave
chmod +x topoGen.sh foo.sh
./topoGen.sh 1 4

After it is done running, which may take some time, you will be left with 2 files you care about:

  • prefix.data
  • topology.data

You can delete link.data and datatraceInput.mat.

# Remove unnecessary files
rm datatraceInput.mat link.data

So now that we have the topology and client traces, let's prepare our experiment archive topology.tgz that we will deploy to our web home directory.

# Bundle up experiment files for nodes
chmod +x prepare.sh
./prepare.sh # Creates topology.tgz

Now that we have all this, we're ready to prepare our web server.

##Preparing the Web Server## We will make use of ORBIT's repository web server by placing some files in our public_html directory so the OMF experiment can access them on the nodes. Let's start by creating the directory and an experiment-related subdirectory.

# Create directories
mkdir -p $HOME/public_html/gnrs/static
# Ensure the directories are accessible
chmod +rx $HOME/public_html $HOME/public_html/gnrs $HOME/public_html/gnrs/static

Now let's move our static files to our newly-created static directory. These are files that do NOT change between experiments, unlike client traces and topologies. We will also unpack our dynamic files (for the topology/clients) into the public_html/gnrs directory, and it will create a topology subdirectory.

cp gnrs.jar gnrsd gnrsd.init ggen gbench $HOME/public_html/gnrs/static/
cd $HOME/public_html/gnrs && tar -zxvf $HOME/my-exp-test/topology.tgz && cd $HOME/my-exp-test

#Running the Experiment# We've spent plenty of time preparing for our experiment. It feels like a lot of work now, but later on you can just copy the my-exp-test directory, replace (or reuse) the topology and client trace files, and run more experiments. I tend to separate my experiments by topology, number of servers/clients, and client messaging rate.

In any event, let's get to the fun part: grinding out results! We'll start with turning-on the nodes that successfully imaged. On the grid, this is almost never all the nodes you specified, but on most sandboxes you should get 100% success.

# Turn on nodes that were successfully imaged
omf tell -a on -t system:topo:imaged

Now that the nodes are turning on, let's run the experiment. There are plenty of parameters to adjust, for a complete list, see resources.rb. This time, however, we'll adjust the most common that I've used. First, here's the command I'll use:

omf exec simple.rb -- --tarUrl http://repository1:8080/~$USER/gnrs/tar/ --tmpDir \
$HOME/public_html/gnrs/tar --dataUrl http://repository1:8080/~$USER/gnrs/topology --scriptUrl \
http://repository1:8080/~$USER/gnrs/static --clientWait 60 --numServers 4 --numClients 1 --sNodes 1 \
--cNodes 1 --messageDelay 250

And a brief description of the invocation:

  • omf exec simple.rb -- : This is telling omf which script contains our experiment. The -- is separator between OMF options and experiment properties.
  • --tarUrl : This is the URL from which nodes will download their .tgz files built by the experiment. This should be the web server where your user directory is located.
  • --tmpDir : This is the local directory that serves the .tgz files for nodes to download.
  • --dataUrl : This is where the topology.tgz was unpacked and where the experiment can find the dynamic files.
  • --scriptUrl : The is the URL where the experiment will get static files (gnrs.jar, gnrsd.init, etc.).
  • --clientWait : How long to wait for the trace to complete, in seconds. The experiment is currently not able to detect the client terminating, so a guess is required. You can review the client logs to get feedback about the actual running time.
  • --numServers : How many GNRS servers to run total. This is divided across the number of server ORBIT nodes.
  • --numClients : How many GNRS clients to run total. This is divided across the number of client ORBIT nodes.
  • --sNodes : How many ORBIT nodes (hosts) to utilize for GNRS servers. The sum of this and cNodes cannot exceed the total number of imaged nodes.
  • --cNodes : How many ORBIT nodes (hosts) to utilize for GNRS clients. The sum of this and sNodes cannot exceed the total number of imaged nodes.
  • --messageDelay : How long the client should wait between messages, in microseconds. This parameter has a large effect on both dropped messages and experiment running time. It should not be lower than 100 in most cases.

Viewing the Results

If everything went well, your experiment should have gone well. Your log files and statistics are located in subdirectories from where you ran the experiment in either server_X (for server number X=1, 2, ...), or clientXRY (for client Y of AS X). For example, our experiment should have.

  • server_1 - Server for AS 1
  • server_2 - Server for AS 2
  • server_3 - Server for AS 3
  • server_4 - Server for AS 4
  • client1R0 - Client 0 for AS 1

The stats files are Comma-Separated Values (CSV) and can be loaded into most spreadsheet programs. The logs are the output from the Java client/server.

#Cleaning-up# Be a good citizen and at least shut-down your nodes. You can do so easily before your reserved time slot is over. You may wish to restart between experiments, but it's not strictly necessary.

# Turn off all nodes in the domain
omf tell -a offh

#Plotting the Results# To be completed later...

Updated