pysh - Mercurial test shell

Author: Patrick Mezard <pmezard@gmail.com>


Execute mercurial test shell scripts under WinNT platforms.


  • Python 2.4 or higher.
  • Cygwin utilities setup in LF-only mode, visible in PATH.
  • pskill and pslist from sysinternals. Could be replaced by anything killing Windows process from PID (see builtin.utility_kill).


Run installation script:

python setup.py install

pysh.bat is copied in python/scripts. Move it somewhere in your PATH to allow mercurial run-tests.py to call it.

0.5 - 2007-12-01

  • Get rid of testhg.py, tests are run with a patched run-tests.py.
  • Support redirect list construct.
  • Add basic support for asynchronous lists and wait command.
  • Support PYSH_HGTEXT and PYSH_TEXT to decide whether mercurial and pysh standard streams are supposed to operated in binary mode or not.
  • Stop interpreting number like filenames as file descriptors
  • Fix Redirections and FileWrapper reference counting
  • Support fgrep utility
  • Fix "done", "do", "then" keywords priority rules

0.4 - 2007-07-10

  • Fix "if" keyword being used as a command argument.
  • Add -c option to pass command string to pysh
  • Upgrade to ply 2.2
  • Fix strange sed issue with missing trailing whitespace.
  • Add unit tests.
  • Add colon utility

0.3 - 2007-05-01

  • Fix many bugs related to shebangs and quoted fields parsing/expansion.

0.2 - 2007-02-21

  • Support shebang mechanism, implemented for /usr/bin/env and /bin/sh.
  • Add shift, env, kill (requires pskill), sort.
  • Fix quote within quotes escaping.

Design Notes

  • Implement whatever is necessary to run mercurial tests under Win32.
  • Implement as specified by OpenGroup:


  • Add whatever extension (shebang, ...) is necessary to run tests.

  • Lexing is done by basic hand-written state-machines. Test coverage is pretty good. Parsing is implemented via PLY (http://www.dabeaz.com/ply/), an SLR parser generator. PLY modules are pretty old by now and might be replaced when a real pull mode is implemented (for interactive mode).

  • The parsing modules were implemented to work on incremental inputs. However, they do not currently run in pull mode, mostly because it was unnecessary to run mercurial tests, and also because I understood too late how it needed to be done. Assuming that PLY can run in pull mode, there should not be any structural issues with the lexer. Sub-lexers were not thoroughly tests in incremental mode though.

  • The grammar handles most mercurial tests. When using it in another context, be ready to encounter the following issues: - Asynchronous lists implementation is minimal. It is good enoug to run

    mercurial tests, no more.

    • Several constructs (until, case) are not implemented.
    • Using reserved words in basic token contexts (command arguments) may be unsupported. Several priority rules were added for specific words already.
  • Cygwin utilities are (not very reliably, see builtin.py) detected since they already output in binary LF mode. CRLF to LF conversion is performed on other external utilities.

  • Unit tests coverage is good for the lexer, medium on the parser and partial for the interpreter. The latter relies on mercurial tests which are a pretty good test in themselves.

Mercurial Tests Coverage

  • Current crew + w32-crew.mq status is:

    # Ran 211 tests, 9 skipped, 9 failed.

ERROR: test-convert-git output changed

Converted changeset identifier don't match. I expect a cygwin git issue here. The converted repository looks good.

ERROR: test-convert-hg-svn output changed ERROR: test-convert-svn-sink output changed

These are real failures and should be fixed. The svn sink is still young and not advertized yet.

ERROR: test-flags output changed

umask and unix like permissions are not supported.

ERROR: test-git-export output changed

Permission failure makes the whole test fail.

ERROR: test-lock-badness output changed

Unix permissions are not supported. The test should be fixed to make the relevant files not accessible.

ERROR: test-merge-types output changed

Non-win32 failure.

ERROR: test-permissions output changed ERROR: test-pull-permission output changed

Unix permissions are not supported.

Skipped test-hup: unknown feature: failure

The test uses FIFO and HUP signal. But I think it really tests serve interruptibility. Fixable but need some work.