Overview

GVL Stress Test

Run a selenium based stress test with selected GVL tutorials. Consists of two main parts - gvl_test_runner.py: which can be used to execute an individual test. (Tests are standard nose tests which must also derive from GVLTestBase). run_benchmark.py - Which will run a whole suite of tests and record the results and atop logs.

Local deployment

Start by cloning GVLStressTest source, installing virtualenv, and adding Python libraries required by GVLStressTest. The steps are:

$ cd <your work dir>
$ git clone https://bitbucket.org/gvl/gvl-stress-test
$ cd gvl-stress-test
$ virtualenv .
$ source bin/activate
$ pip install -r requirements.txt
$ sh runtests.sh -c tests.workflow_deseq_basic.WorkflowDeseqBasic -s http://118.138.243.115/

Usage

Running the tests

Example:

$ source bin/activate
$ python gvl_test_runner.py -c tests.workflow_microbial_assembly.WorkflowMicrobialAssembly -s http://118.138.243.115/

Usage:

usage: gvl_test_runner.py [-h] [-s SERVER] [-u USER] [-e EMAIL] [-p PASSWORD]
                          [-l LOG] [-pt PTIMEOUT] [-jt JTIMEOUT] [-g GRID]
                          [-c TESTCLASS]

optional arguments:
  -h, --help                    show this help message and exit
  -s SERVER, --server SERVER
                                URL of galaxy server to use. The default is http://galaxy-tut.genome.edu.au/
  -u USER, --user USER          Galaxy username. Default is testbot
  -e EMAIL, --email EMAIL       Galaxy login email. Default is testbot@genome.edu.au
  -p PASSWORD, --password PWD   Password for galaxy user. Default is gvl_letmein
  -l LOG, --log LOG             Prefix for log files. Default is gvltest
  -pt PTIMEOUT                  Maximum time (in seconds) to wait for page to be
                                ready. Default is 60
  -jt JTIMEOUT                  Maximum time (in seconds) to wait for job to finish
                                execution. Default is 3600
  -g GRID, --grid GRID          Use remote selenium grid server. e.g.
                                http://115.146.86.155:5555/wd/hub. Default is: None
  -c TESTCLASS, --testclass CLS Execute given test suite. Fully qualified classname
                                should be provided. Default:
                                tests.gvl_test_base.GVLTestBase

Running the benchmark

usage: run_benchmark.py [-h] -ak AKEY -sk SKEY [-m MACHINE] [-w WORKERS]
                        [-d GALAXYDATA] [-idx GALAXYINDICES] [-img IMAGE]
                        [-z ZONE] [-gr GRID] [-b BUCKET]
                        [-s SKIPSTART SKIPSTART SKIPSTART]
                        [-e SKIPEND SKIPEND SKIPEND]

optional arguments:
  -h, --help                    show this help message and exit
  -ak AKEY, --akey AKEY         Access key to use
  -sk SKEY, --skey SKEY         Secret key to use
  -m MACHINE, --machine MCH     Instance type/flavor to use (e.g. m1.small, m1.medium
                                etc). Default is m1.medium
  -w WORKERS, --workers WORKERS Maximum number of workers to test. Default is 5
  -d GALAXYDATA                 Storage to use for galaxy data. Should be either
                                transient, or custom-size. Default is transient
  -idx GALAXYINDICES            Storage to use for galaxy index. Should be either
                                volume or gluster. Default is gluster
  -img IMAGE, --image IMAGE     AMI id to use. Default is ami-00001de9
  -z ZONE, --zone ZONE          Placement zone to use. Default is melbourne-qh2
  -gr GRID, --grid GRID         Use remote selenium grid server. e.g.
                                http://115.146.86.155:5555/wd/hub.
  -b BUCKET, --bucket BUCKET    Default cloudman bucket to use. Default is: cloudman-os
  -s USER WORKLOAD WORKER       Skip tests to desired start point. 3 values must be
                                provided. e.g. --skiptest 2 0 3 would start running
                                tests from users=2, workload=0, workers=3
  -e USER WORKLOAD WORKER       Finish tests at desired end point. 3 values must be
                                provided. e.g. --skipend 2 0 3 would start running
                                tests till users=2, workload=0, workers=3

Available tests

tests.import_tutorial_histories.ImportTutorialHistories tests.rnaseq_dge_basic_prep.RNAseqDGEBASICPrep tests.rnaseq_dge_advanced_prep.RNAseqDGEAdvancedPrep tests.variant_detection_advanced.VariantDetectionAdvanced tests.workflow_deseq_basic.WorkflowDeseqBasic tests.workflow_microbial_assembly.WorkflowMicrobialAssembly tests.workflow_variant_detection_basic.WorkflowVariantDetectionBasic