TypeError: compile() expected string without null bytes

Issue #591 wontfix
Anonymous created an issue

Executing "coverage xml" was failed. (coverage version 4.4.1)

error   15-Jul-2017 19:29:29    Traceback (most recent call last):
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/bin/coverage", line 11, in 
error   15-Jul-2017 19:29:29    sys.exit(main())
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/cmdline.py", line 756, in main
error   15-Jul-2017 19:29:29    status = CoverageScript().command_line(argv)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/cmdline.py", line 517, in command_line
error   15-Jul-2017 19:29:29    total = self.coverage.xml_report(outfile=outfile, **report_args)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/control.py", line 1070, in xml_report
error   15-Jul-2017 19:29:29    return reporter.report(morfs, outfile=outfile)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/xmlreport.py", line 76, in report
error   15-Jul-2017 19:29:29    self.report_files(self.xml_file, morfs)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/report.py", line 91, in report_files
error   15-Jul-2017 19:29:29    report_fn(fr, self.coverage._analyze(fr))
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/control.py", line 899, in _analyze
error   15-Jul-2017 19:29:29    return Analysis(self.data, it)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/results.py", line 19, in init
error   15-Jul-2017 19:29:29    self.statements = self.file_reporter.lines()
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/python.py", line 176, in lines
error   15-Jul-2017 19:29:29    return self.parser.statements
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/python.py", line 171, in parser
error   15-Jul-2017 19:29:29    self._parser.parse_source()
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/parser.py", line 238, in parse_source
error   15-Jul-2017 19:29:29    self._raw_parse()
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/parser.py", line 207, in _raw_parse
error   15-Jul-2017 19:29:29    self.raw_statements.update(self.byte_parser._find_statements())
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/parser.py", line 96, in byte_parser
error   15-Jul-2017 19:29:29    self._byte_parser = ByteParser(self.text, filename=self.filename)
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/parser.py", line 364, in init
error   15-Jul-2017 19:29:29    self.code = compile_unicode(text, filename, "exec")
error   15-Jul-2017 19:29:29    File "/opt/ActivePython-2.7/lib/python2.7/site-packages/coverage/phystokens.py", line 286, in compile_unicode
error   15-Jul-2017 19:29:29    code = compile(source, filename, mode)
error   15-Jul-2017 19:29:29    TypeError: compile() expected string without null bytes

I add print filename in site-packages/coverage/phystokens.py line 285 to indicate the filename before exception occurred. Then I use a script with the filename to identify null byte.

import sys
# input filename found by site-packages/coverage/phystokens.py
filename = sys.argv[1]
with open(filename, 'r') as f:
   content = f.readlines()

for i,l in enumerate(content):
    if b'\x00' in l:
        print i,l

Finally, remove null bytes, coverage works fine.

BTW, I found the null bytes were in the doc string, it most likely caused compile_unicode method invalid.

Comments (2)

  1. Ned Batchelder repo owner

    In my tests, I can't even run a Python program with null bytes in docstrings:

    $ hexdump -C bug591.py
    00000000  22 22 22 4e 75 6c 6c 20  62 79 74 65 3a 20 00 22  |"""Null byte: ."|
    00000010  22 22 0a 61 20 3d 20 31  0a                       |"".a = 1.|
    00000019
    $ python2.7 bug591.py
      File "bug591.py", line 3
    
        ^
    SyntaxError: EOF while scanning triple-quoted string literal
    $ python3.6 bug591.py
      File "bug591.py", line 3
    
        ^
    SyntaxError: EOF while scanning triple-quoted string literal
    
  2. Log in to comment