Testsuite for SCons

Welcome to the SCons testsuite. This repository is a collection of several real-life projects and artificial benchmarks using SCons ( Together with some scripts for automatically timing and profiling single builds and update runs, its purpose is to be able to compare different SCons revisions regarding time and memory consumption.

Disclaimer: This testsuite was developed during my analysis of SCons' runtime behaviour for the Wiki article "Why SCons is not slow". It's currently not in a state of "running everywhere", but especially crafted for my very own Ubuntu Linux machine. So, if you try to start the examples on your own, be prepared to run into some pitfalls. You might have to adapt the controlling scripts, or even need to patch the software packages themselves...including the installation of special packages as prerequisites.


Contains the source packages for the example projects in various formats like .zip or .tgz.
A simple Python package named sconstest, providing some basic helper and support routines.
A copy of and two wrapper scripts for profiling SCons runs.
The basic shell and Python scripts for running single tests or evaluating and comparing the results, respectively.

Configuring and installing

For being able to run any tests, you have to install the sconstest package, but need to configure some paths beforehand. At the top of the file src/sconstest/, change the three variables

Absolute path, pointing to the folder where your copy of the sconstest repository is located. Example: /home/dirk/workspace/scons_testsuite
Absolute path, pointing to the src folder of the SCons version that you want to test. Example: /home/dirk/workspace/scons_experimental/src
Relative path, the name of the folder where all results of the single runs get stored. Example: results

according to your local setup.

Then issue the command:

sudo python install


Running a test

There are three basic types of tests that you can run:

Simple measuring of the runtime via /usr/bin/time, for clean builds and updates. In the case of running the script, each run is repeated three times and the median of all values gets calculated automatically.
Runs SCons through, in order to get an impression about where exactly the larger portions of runtime are spent.
Starts a separate memory watcher in parallel to SCons. This watcher peeks into /proc/PID/status for the process and writes the currently used memory (VmSize) into a CSV file every second.

The scripts and combine all these three tests, while the latter runs each time test three times and mediates all results accordingly.

For each of the three test types, there exists a and a version. While the version loop over all defined project packages and get started from the scons_testsuite folder itself, the scripts get called from the actual source folders for each project.

For the result folder structure it's assumed that you want to compare different source changes, or revisions, for a single SCons version. That's why the results folder is somewhat hardcoded into the test framework. Within this results folder the single runs get a separate subdirectory each. For example if you call:

./ changed

this will create a results/changed folder with all the results of all project packages inside, based on the current source code of the SCons version that you specified via the sconswd variable.

Before starting any project builds or tests manually, you'll want to call:


, which creates the subfolder build for you and unpacks all source packages for the examples to it.

Evaluating results

This is actually done automatically on a call of the all_suite* scripts. However, you can also call the script manually as:

python default

, which will dive into the current results folder and create PNG and SVG images for the profiling results.

Comparing different runs

Once you collected the results of different changes, you might want to compare their results against the original state, in order to check for significant speedups.

We assume that you ran a full test of your original source state as default, and you have some new results for your changed sources in the folder changed. Then you can create a simple HTML file with the command:

python default changed

. It gets written as comparison.html to the changed folder.