cram /

Filename Size Date modified Message
90 B
17.6 KB
92 B
727 B
3.5 KB
224 B
6.3 KB
2.1 KB

Cram: It's test time

Cram is a functional testing framework for command line applications based on Mercurial's unified test format.

Here's a snippet from cram.t in Cram's own test suite:

The $PYTHON environment variable should be set when running this
test from Python.

  $ [ -n "$PYTHON" ] || PYTHON=python
  $ if [ -n "$COVERAGE" ]; then
  >   coverage erase
  >   alias cram='coverage run -a'
  > else
  >   alias cram="$PYTHON"
  > fi


  $ cram -h
  [Uu]sage: cram \[OPTIONS\] TESTS\.\.\.

    -h, --help     show this help message and exit
    -v, --verbose  Show filenames and test status
  $ cram
  [Uu]sage: cram \[OPTIONS\] TESTS\.\.\.

The format in a nutshell:

  • Cram tests use the .t file extension.
  • Lines beginning with two spaces, a dollar sign, and a space are run in the shell.
  • Lines beginning with two spaces, a greater than sign, and a space allow multi-line commands.
  • All other lines beginning with two spaces are considered command output.
  • Command output in the test is first matched literally with the actual output. If it doesn't match, it's then compiled and matched as a Perl-compatible regular expression.
  • Anything else is a comment.



You can use pip to install Cram:

$ sudo pip install cram

Or you can install Cram the old fashioned way:

$ wget
$ tar zxvf cram-0.1.tar.gz
$ cd cram-0.1.tar.gz
$ sudo python install


Cram will print a dot for each passing test. If a test fails, a unified context diff is printed showing the test's expected output and the actual output.

For example, if we run cram on its own example tests:

$ cram examples
--- examples/fail.t
+++ examples/fail.t.out
@@ -3,11 +3,11 @@
   $ echo 1
   $ echo 1
-  2
+  1
   $ echo 1

 Invalid regex:

   $ echo 1
-  +++
+  1

Cram will also write the test with its actual output to examples/fail.t.err. This makes it easy to merge output back into the test file using when writing a test for the first time. You can run diff examples/fail.t{,.err} and use patch -p0 to apply the patch.


Download the official development repository using Mercurial:

hg clone

Test Cram using Cram:

make tests

Get a test coverage report using

make coverage

Visit Bitbucket if you'd like to fork the project, watch for new changes, or report issues.