autodoc causing a crash with non-python __file__

Anonymous avatarAnonymous created an issue

I use a custom sys.meta_path hook for importing C-API modules. This is problematic for the autodoc extension as it attempts to scan for attribute documentation in a C file.

Doing some debugging around: self.tokens = list(tokenize.generate_tokens(self.source.readline))

I found that self.source was indeed the C code.

ISTM that a warning would be more appropriate than a crash.

Traceback (most recent call last):e                                                                                                                                                                                                                                                           
  File "/litterbox/lib/python3.3/runpy.py", line 160, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/litterbox/lib/python3.3/runpy.py", line 73, in _run_code
    exec(code, run_globals)
  File "./dev/bin/sphinx.py", line 9, in <module>
    sys.exit(main(sys.argv[1:]))
  File "./dev/bin/sphinx.py", line 6, in main
    libsphinx.build(x, statusfile = sys.stderr, warningfile = sys.stderr)
  File "./dev/libsphinx.py", line 90, in build
    s.build(force_all = True)
  File "/litterbox/lib/python3.3/site-packages/sphinx/application.py", line 200, in build
    self.builder.build_all()
  File "/litterbox/lib/python3.3/site-packages/sphinx/builders/__init__.py", line 159, in build_all
    self.build(None, summary='all source files', method='all')
  File "/litterbox/lib/python3.3/site-packages/sphinx/builders/__init__.py", line 216, in build
    purple, length):
  File "/litterbox/lib/python3.3/site-packages/sphinx/builders/__init__.py", line 120, in status_iterator
    for item in iterable:
  File "/litterbox/lib/python3.3/site-packages/sphinx/environment.py", line 613, in update_generator
    self.read_doc(docname, app=app)
  File "/litterbox/lib/python3.3/site-packages/sphinx/environment.py", line 761, in read_doc
    pub.publish()
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/core.py", line 217, in publish
    self.settings)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/readers/__init__.py", line 72, in read
    self.parse()
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/readers/__init__.py", line 78, in parse
    self.parser.parse(self.input, document)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/__init__.py", line 172, in parse
    self.statemachine.run(inputlines, document, inliner=self.inliner)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 174, in run
    input_source=document['source'])
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2941, in text
    self.section(title.lstrip(), source, style, lineno + 1, messages)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 331, in section
    self.new_subsection(title, lineno, messages)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 399, in new_subsection
    node=section_node, match_titles=True)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 286, in nested_parse
    node=node, match_titles=match_titles)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 199, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2706, in underline
    self.section(title, source, style, lineno - 1, messages)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 331, in section
    self.new_subsection(title, lineno, messages)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 399, in new_subsection
    node=section_node, match_titles=True)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 286, in nested_parse
    node=node, match_titles=match_titles)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 199, in run
    results = StateMachineWS.run(self, input_lines, input_offset)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 239, in run
    context, state, transitions)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/statemachine.py", line 460, in check_line
    return method(match, context, next_state)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2279, in explicit_markup
    nodelist, blank_finish = self.explicit_construct(match)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2291, in explicit_construct
    return method(self, expmatch)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2034, in directive
    directive_class, match, type_name, option_presets)
  File "/litterbox/lib/python3.3/site-packages/docutils-0.10-py3.3.egg/docutils/parsers/rst/states.py", line 2083, in run_directive
    result = directive_instance.run()
  File "/litterbox/lib/python3.3/site-packages/sphinx/ext/autodoc.py", line 1303, in run
    documenter.generate(more_content=self.content)
  File "/litterbox/lib/python3.3/site-packages/sphinx/ext/autodoc.py", line 700, in generate
    self.analyzer.find_attr_docs()
  File "/litterbox/lib/python3.3/site-packages/sphinx/pycode/__init__.py", line 261, in find_attr_docs
    self.parse()
  File "/litterbox/lib/python3.3/site-packages/sphinx/pycode/__init__.py", line 251, in parse
    self.tokenize()
  File "/litterbox/lib/python3.3/site-packages/sphinx/pycode/__init__.py", line 244, in tokenize
    self.tokens = list(tokenize.generate_tokens(self.source.readline))
  File "/litterbox/lib/python3.3/site-packages/sphinx/pycode/pgen2/tokenize.py", line 345, in generate_tokens
    raise TokenError("EOF in multi-line statement", (lnum, 0))
sphinx.pycode.pgen2.tokenize.TokenError: ('EOF in multi-line statement', (6132, 0))

Comments (1)

  1. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.