command options and redirection confusion

[Former user]

This was unexpected based on typical bash shell behavior:

>>> p = run('echo foo | gzip -9 > debug.out')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "sarge/__init__.py", line 1265, in run
    with Pipeline(cmd, **kwargs) as p:
  File "sarge/__init__.py", line 873, in __init__
    t = command_line_parser.parse(source, posix=posix)
  File "sarge/__init__.py", line 730, in parse
    result = self.parse_list()
  File "sarge/__init__.py", line 734, in parse_list
    parts = [self.parse_pipeline()]
  File "sarge/__init__.py", line 750, in parse_pipeline
    parts = [self.parse_logical()]
  File "sarge/__init__.py", line 783, in parse_logical
    part = self.parse_command()
  File "sarge/__init__.py", line 800, in parse_command
    node = self.parse_command_part()
  File "sarge/__init__.py", line 849, in parse_command_part
    'stderr, not %s' % list(d.keys()))
ValueError: semantics: can only redirect stdout and stderr, not [-9]

[Former user]

Playing around with the tokenizer I changed next_token() to only treat non-negative numbers as a token type of 'number':

diff -r 0b0055aa9315 sarge/__init__.py
--- a/sarge/__init__.py Fri Jun 07 12:20:47 2013 +0100
+++ b/sarge/__init__.py Wed Jun 12 13:06:33 2013 -0400
@@ -663,8 +663,10 @@
                 t = t[1:-1]
             elif tt == 'a':
                 try:
-                    int(t)
-                    tt = 'number'
+                    if int(t) >= 0:
+                        tt = 'number'
+                    else:
+                        tt = 'word'
                 except ValueError:
                     tt = 'word'
             elif tt == 'c':

And this fixes the above command. However, redirection is still broken for other use cases. For example:

>>> import sarge
>>> pipeline = sarge.run('seq 1 10 > results')
1
>>> pipeline.commands
[Command('seq 1')]

Whereas with normal bash (4.1.2) I get expected results:

$ seq 1 10 > results
$ cat results
1
2
3
4
5
6
7
8
9
10

Perhaps whitespace could be taken into account here? Or allow quoting of the arguments to disable treating arguments as file descriptors.

I.e. I would expect the current sarge results from a bash shell by the syntax:

# redirect fd 10 to results
$ seq 1 10>results
1
$ wc -l results
0 results

But quoting the second argument works for bash

$ seq 1 "10">results
$ wc -l results
10 results

But not for sarge:

>>> pipeline = sarge.run('seq 1 "10" > results')
1
>>> pipeline.commands
[Command('seq 1')]

[Vinay Sajip]

Thanks for the report. I also implemented an interim fix, but I think the answer is to change the tokenizer. I have been using a variant of the stdlib's tokenizer, but I fear it is too inflexible. I'm working on an approach which, if it doesn't work, may mean that a new tokenizer is needed :-(

[Vinay Sajip]

Closes #7: Fixed bugs in handling whitespace and redirections.

→ <<cset c515eb5cccc9>>