PicklingError when autodocumenting a class with a dynamically created parent

Thomas Kluyver avatarThomas Kluyver created an issue

In IPython, we're using a copy of the six with_metaclass function for Python 2 & 3 compatibility. This dynamically creates a NewBase class, which the real class will inherit from. In our docs, we use the autoclass directive with the :show-inheritance: option. This fails with the error given below.

The failure appears to be new in Sphinx 1.2 - I don't see it with 1.1.3.

This issue reported for the Django-treebeard library appears to be the same.

# Sphinx version: 1.2b3
# Python version: 2.7.5+
# Docutils version: 0.11 release
# Jinja2 version: 2.7.1
# Loaded extensions:
#   sphinx.ext.graphviz from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/graphviz.pyc
#   github from /home/takluyver/Code/ipython/docs/sphinxext/github.pyc
#   sphinx.ext.autodoc from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/autodoc.pyc
#   sphinx.ext.intersphinx from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/intersphinx.pyc
#   sphinx.ext.doctest from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/doctest.pyc
#   matplotlib.sphinxext.mathmpl from /usr/lib/pymodules/python2.7/matplotlib/sphinxext/mathmpl.pyc
#   matplotlib.sphinxext.plot_directive from /usr/lib/pymodules/python2.7/matplotlib/sphinxext/plot_directive.pyc
#   sphinx.ext.inheritance_diagram from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/inheritance_diagram.pyc
#   matplotlib.sphinxext.only_directives from /usr/lib/pymodules/python2.7/matplotlib/sphinxext/only_directives.pyc
#   numpydoc from /home/takluyver/Code/ipython/docs/sphinxext/numpydoc.pyc
#   IPython.sphinxext.ipython_console_highlighting from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/IPython/sphinxext/ipython_console_highlighting.pyc
#   IPython.sphinxext.ipython_directive from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/IPython/sphinxext/ipython_directive.pyc
#   sphinx.ext.oldcmarkup from /home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/ext/oldcmarkup.pyc
Traceback (most recent call last):
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/cmdline.py", line 246, in main
    app.build(force_all, filenames)
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/application.py", line 212, in build
    self.builder.build_update()
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 214, in build_update
    'out of date' % len(to_build))
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 234, in build
    purple, length):
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/builders/__init__.py", line 134, in status_iterator
    for item in iterable:
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/environment.py", line 470, in update_generator
    self.read_doc(docname, app=app)
  File "/home/takluyver/.virtualenvs/ipy-trunk/local/lib/python2.7/site-packages/sphinx/environment.py", line 689, in read_doc
    pickle.dump(doctree, f, pickle.HIGHEST_PROTOCOL)
PicklingError: Can't pickle <class 'IPython.utils.traitlets.NewBase'>: attribute lookup IPython.utils.traitlets.NewBase failed

Comments (4)

  1. Thomas Kluyver

    I've traced this to pull request #25 - it only hits when an inheritance diagram includes a class with a dynamically defined parent.

    In the inheritance diagram, class_info is now stored as a dict, keyed by the class objects themselves, whereas previously only the values from that dict were stored. That means that pickling now saves the class objects, not just the names.

    A workaround that suffices for my case is to give the dynamically defined class a name beginning with underscore, e.g. _NewBase. Then it will be skipped over as a private base class, as long as you don't set the private-bases flag in the inheritance diagram directive.

  2. Log in to comment
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.