Streamlined API docs?

[jvanasco]

First off -- I'm not sure if this is possible. I looked into the SqlA docs, and couldn't figure out a way to do this with RST. If there is a way and consensus, I would be glad to do the busywork on this idea....

The docs are getting hard for me to read and reference. They're absolutely great in content -- it's just the interface.

The two big issues I've encountered are:

a- Many classes in the docs require paginating across dozens of screens to see all the available methods. Object and Session are good examples. I think they'd benefit greatly by an "Section Table-Of-Contents". RST doesn't seem to support that though.

a- The narrative docs are amazing, possibly the best in Python. But there is no "API Reference" style documentation online, where one can quickly navigate into a specific module or method. I think it might be worth adding to the docs generation (online and local) some automatic API output , along the lines of what Twisted does with pydoctor or Pyramid does with Sphinx.

[Mike Bayer]

yeah it is "blue sky" because sphinx begins to break down when you push it this far. A lot of packages have the javadoc style "index of everything" set up, and we used to follow more of that style. But predictably, then we had an enormous matrix of functions and classes that nobody ever looked inside, because it totally lacked context.

the current design of the docs is modeled after that of Python's itself. if you go to https://docs.python.org/3/library/index.html, there is no "giant matrix of all functions", you have to go into the narrative docs you're looking for, say https://docs.python.org/3/library/difflib.html, and you see the docstrings inlined with the narrative.

now here's the thing, why not have both ? sure. The problem is sphinx falls down on that, it really gets very confusing when the same object is docstring'ed in two places. linking becomes hard, and folks might find themselves at the same docstring in one context or another.

So maybe here what we need is the navigation for API docs, but then it ultimately links you back into narrative sections. Right now there is just the index: http://docs.sqlalchemy.org/en/rel_0_9/genindex.html which is ugly. But this is kind of what you see these days. Pandas was doing this awhile back but they seem to have improved with this display: http://pandas.pydata.org/pandas-docs/stable/api.html, though clicking on those takes you to API-specific pages. Their narrative docs seem to lack inline links for methods, so they're not there yet either.

there's really a single problem to be solved here. a system in sphinx where API docs can be in narrative context and also indexed hierarchically at the same time. Sphinx is super duper extensible so it is very doable.

[jvanasco]

I think navigation for API linking to narrative would work well.

The genindex is perfect when you're looking up a specific method (i would never want to lose it!), but horrible for understanding a concept.

The pandas display is pretty great.

A good practical example would start with looking up merge ( someone has been posting on the list, it's fresh in my mind). If i jump on the index, I can see what classes have a merge method and quickly link to those objects. Great.

But is merge what I want? Maybe I want something else in the Session.

Because of the structure of the narrative docs, there is no concise list of all the methods sqlalchemy.orm.session.Session offers. So in order for me to see every possible method of sqlalchemy.orm.session.Session, I have to scroll down through lot of formatted text and try to make sense (visually) of what are the relevant docs vs inline examples vs scrolling too far and looking at the docs for the next class.

I find myself doing this a lot:

import sqlalchemy.orm.session
for i in ([i for i in dir(sqlalchemy.orm.session.Session) if i[0] != '_']): print i

then searching the docs for the method I want.

[Mike Bayer]

OK, but we already established that's the issue. Repeating it isn't going to lead to a solution :). Pandas has the issue that all the information about their API has to be in the narrative docs - when you go to their API docs, you get brief information, and no links back to where the particular object is used or how. That the narrative is separate forces them to have very succinct API docs lest they run the risk of repeating themselves. This was the problem I was hitting all over the place and is why i put the API docs inline with narrative, because the API docs are the narrative in many cases.

[Mike Bayer]

I'd love if you could look into what exactly "autosummary" does, this might be what pandas uses, and how that would work in our docs in order to build out an "index of everything" section. Though i have a feeling that's not automated enough and we'll need to write an extension. http://sphinx-doc.org/ext/autosummary.html

[jvanasco]

Will do this week.

[Mike Bayer]

there's not much going on here, agree that more navigational improvements are better though the API docs have been improved since this issue was opened (including the interactive left hand navigation)