Pushed to simonpeter/zadok
ed0b4dc Add capability to test return values.
ZADOK ===== A multi-language test suite runner. 1. COPYRIGHT ------------ Copyright 2004 - 2017, Simon Peter Chappell 2. PRIMARY CONTACT ------------------ firstname.lastname@example.org email@example.com 3. LICENCE ---------- Zadok is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. Zadok is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with Zadok; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA 4. PURPOSE ---------- Zadok is a multi-language test suite runner. It is designed to run small test programs written in one of the programming or scripting languages that it understands, compare their output to what is expected and determine a pass or fail. Tests are grouped into suites for conceptual organisation purposes. 5. USAGE -------- To run Zadok, place "zadok.sh" on the PATH. The traditional way to do this, on Linux and Mac OS X, is to add a snippit like the following one to your .bash_profile file. ~~~ ZADOK_HOME=/opt/zadok export ZADOK_HOME PATH=$PATH:$ZADOK_HOME/bin ~~~ From the directory where Zadok is started, Zadok will look for any directory that ends with the text "Suite" and will consider it to be a suite of test programs. Zadok will then look inside each suite to check for the presence of a file called "suitename.txt" and directories with names that end in "Test". Zadok will then execute and test the output of each program that it recognises and it has the resources to run. 6. DESIGN --------- The design of Zadok, while no means perfect, is designed to overcome the problem of testing suites of programs written in multiple programming languages. When working with a single programming language, there are often testing libraries or utilities specifically for that language, for example JUnit for the Java language. Unfortunately, these libraries are generally unsuitable for multiple languages. To test programs written in multiple programming languages, it is necessary to find the lowest common denominator, or common factor, between programs. The common factors that fit the requirements are the standard input, standard output and command line parameters. At this time Zadok uses the standard output, which allows it to receive consistent output from the programs under test. Being able to get a copy of the output from the program enables us to determine whether it was what we expected and therefore correct. Zadok uses a plugin capability for the language runners. Previously, each known language had to be added to the body of the Zadok code specifically. With plugins, each language suffix found is compared to the available runner plugins and where a match is found, the runner is used, otherwise that language is skipped. 7. PROGRAMMING LANGUAGES ------------------------ At this time Zadok supports programs written in the following programming and scripting languages (numbers in parenthesis are the versions used by the author at this time): Bash (4.2.10), Groovy (1.7.4), C (gcc 4.6.1), Clisp (2.49), Erlang/OTP (5.8.3), Java (1.6.0_23), Ruby (1.8.7), Lua (5.1.4), Perl (5), Python (2.7.2+), AWK (3.1.8), Haskell (GHC 7.0.3), OCaml (3.12.0), REXX (Regina 3.5), Scala (2.7.3.final) and TCL (8.5). Wherever possible, the languages are installed from the official package repository. The author uses Ubuntu Linux, so a number of languages are already installed as part of the distribution. The language homepages are listed for your convenience incase you need to download and compile any of them. groovy.codehaus.org www.erlang.org www.ruby-lang.org www.lua.org www.perl.com www.python.org www.gnu.org/software/gawk/gawk.html www.haskell.org regina-rexx.sourceforge.net clisp.cons.org caml.inria.fr www.tcl.tk 8. TO DO -------- There are many functionalities that could be added: * More test suites. (under active development at this time) * More programming and scripting languages. * Run only a specified suite. * Run only tests written in a specified language. * Save the results to a file in either plain text or XML format. * It would be interesting to port Zadok to other languages. 9. THANKS --------- Many thanks are due to my friend Mike Clark for his encouragement in all matters of testing. William Juroff has given so much help and design advice that he also deserves an honorable mention.