Support RTD model for building/uploading docs

[Jason R. Coombs]

I'm struggling a bit with uploading docs for my projects. I see that devpi has support for distutils-based docs uploads, and I commend you for the work creating a PyPI-compatible interface for docs.

Unfortunately, PyPI is losing docs support with Warehouse and for that reason, I've deprecated the upload_docs command.

It might be fairly straightforward to simply continue to support the upload_docs command for devpi usage, but it has several irreconcilable differences with the way RTD builds docs:

• RTD always builds from the docs directory. setup.py is invoked from the project root. As a result, any references to files in the project dir can't support both environments without careful resolution of the cwd.
• setup.py downloads setup_requires dependencies which are often a poor approximation of the dependencies needed for building docs. There's no standard way to declare the dependencies needed for building docs. RTD has a nice model for this - one can specify a requirements.txt and can opt to install the project (a little redundant, actually).
• RTD implies that Sphinx is installed; build_sphinx requires it to be installed.

There are other issues as well, like authentication and specification of the pypi repository, which I believe has to be supplied redundantly with the upload command. I'm not sure if the password will be resolved from the keyring or from devpi. Does devpi upload --with-docs pass devpi credentials to the upload_docs command?

What I'd rather see is native support for uploading docs (built by some other process) and maybe consider if a devpi build-docs command might replicate the RTD model, which while opinionated does form something of a basis for a standard.

[Florian Schulze]

Providing other means to upload docs should be fairly trivial.

Could you elaborate on what build-docs would entail? Do we have to do anything here at all? In devpi and PyPI the docs are just a zip file with index.html at root, nothing else is needed. AFAIK RTD is triggered by github etc on push of commits and it then looks for some config like Travis for testing. Do you think we should implement a builder for that? Isn't there already something to test RTD building locally?

I think from our end we should only provide a way to upload ready made docs, everything else shouldn't be our concern.

[Jason R. Coombs]

Do you think we should implement a builder for trigger on github commits?

No.

Isn't there already something to test RTD building locally?

There is a local VM install option and docker images, but all of that is overkill and out of scope of devpi, I think.

we should only provide a way to upload ready made docs

Perhaps, but that leaves it to each project (or an auxiliary project) to implement the build and bundle steps. But now that I say it, that's definitely a preferable approach. Perhaps an upload_docs command would accept a zipfile or directory, and if a directory, it will zip it up before sending it, saving the need for tooling altogether:

python -m sphinx docs build/html
devpi upload-docs build/html

[Jason R. Coombs]

After further consideration, I realized that perhaps one of the blockers (the cwd) could actually be addressed directly in the offending plugins, which led me to rst.linker #4. After addressing that, I've found I can probably rely on rwt to address the build dependency issues using something like this:

rwt sphinx -r docs/requirements.txt . -- setup.py build_sphinx upload_docs

So maybe this ticket should be a won't fix. I'll continue exploring it and update here.

[Jason R. Coombs]

Initial investigation indicates that the upload_docs command should work. I'll probably remove the deprecation from the distutils command instead.

[Florian Schulze]

This issue has been moved to GitHub: https://github.com/devpi/devpi/issues/388