Pull requests

#89 Open
Repository
heng heng
Branch
default
Repository
birkenfeld birkenfeld
Branch
default

Explicitly track files on which the file depends.

Bitbucket cannot automatically merge this request.

The commits that make up this pull request have been removed.

Bitbucket cannot automatically merge this request due to conflicts.

Review the conflicts on the Overview tab. You can then either decline the request or merge it manually on your local system using the following commands:

hg update default
hg pull -r default https://bitbucket.org/heng/sphinx
hg merge 224785ff3e7e
hg commit -m 'Merged in heng/sphinx (pull request #89)'
Author
  1. heng
Reviewers
Description

With any remotely non-simple module dependency, the docs are not logged as being dependent on the necessary files. This means the user needs to manually touch each file to get persuade the docs to update.

This adds a config option, dependencies, which is a dict from the target rst file to a dep list, which is then checked for changes during builds.

  • Learn about pull requests

Comments (11)

  1. Jon Waltman

    The dependency tracking is indeed problematic, thanks for working on this.

    I wouldn't want to explicitly list dependencies like this though for each autodoc directive.

    What about having a configuration value in conf.py that explicitly list dependencies for the entire project. This might be useful in other cases besides autodoc.

    Something like this:

    dependencies = ['../pkg/foo.py', ...]
    

    or programmatically

    dependencies = []
    
    import os
    for root, dirs, files in os.walk(os.path.abspath('../pkg')):
      for f in files:
          if f.endswith('.py'):
              dependencies.append(os.path.join(root, f))
    
  2. heng author

    The problem with the dependencies being in conf.py is that it moves the logic away from where the dependency is actually applicable, and so one could easily end up with orphan dependencies. I initially thought about conf.py, but it seemed neater to do what I did.

    I don't see the problem with specifying the dependencies at the point the module is needed (apart from it being autodoc specific).

    Perhaps another way to do it is to add a dictionary of modules to dependencies in conf.py, which extensions can look up as necessary...?

  3. heng author

    Another quick point, the problem with a global dependency list is that its not then clear which documents need rebuilding when a file is touched, so necessarily triggering the whole build.

  4. Jon Waltman

    I don't see the problem with specifying the dependencies at the point the module is needed (apart from it being autodoc specific).

    The dependency graph of a python module can get pretty complicated. a depends on b, c, and d. b depends on x and y ...

    the problem with a global dependency list is that its not then clear which documents need rebuilding when a file is touched, so necessarily triggering the whole build.

    I think a bigger problem is people rebuilding their docs and not seeing the changes because of the cache handling.

    1. heng author

      The dependency graph of a python module can get pretty complicated. a depends on b, c, and d. b depends on x and y ...

      Right, but that's dealt with by the user deciding (the simple solution!). They just tell autodoc what files to check for changes. Doing this globally for any given module is not a bad idea though.

      1. Jon Waltman

        While it may be the simple solution, it would be annoying to have to hand-write the dependency graph for every module and difficult to maintain. For a large project, these lists could get quite large.

        I don't see an easy way to programmtically generate the needed files for a specific module other than just getting every Python file from the tip of the package down.

        There might be a better way to do this than the approach I gave but I'm more concerned with not using an outdated cache and missing changes rather than saving build time.

        1. heng author

          If auto is what is needed, it strikes me that Snakefood is the correct way to do it. Use it if it's there, with a fallback to the current default. Shall I put together some code to test it?

          The other problem is that my files are actually cython files, but that's another level of difficulty!

          1. Jon Waltman

            Snakefood looks interesting. I'm hesitant though about conditionally using a package most people won't have installed. Also, I don't think snakefood has Python 3 support which would be required.

            Using my second approach, users could use snakefood in their conf.py to populate the dependencies dict. This of course isn't a general solution.

  5. Jon Waltman

    What about instead of a list of global dependencies for the entire project, a dict mapping documents to their dependencies.

    dependencies = {
        'foo': ['../pkg/foo.py', ...],
         ...
    }
    

    You could make it more convenient by allowing glob style patterns for the keys (maybe the values too). html_sidebars works like this.

    1. heng author

      Yeah, this was what I meant. I think it's a neat solution. Perhaps the way to incorporate snakefood is as a little extension that just populates that dictionary when it's loaded.

  6. heng author

    I've updated the pull reques to reflect all the discussions.

    Also, there seemed to be a bug in code not returning an iterable from a env-get-outdated event handler (which the docs seem to suggest is allowed).