DevPI should support generic WSGI deployment

Issue #155 on hold
Timothy Allen created an issue

As far as I can tell, DevPI currently supports two deployment scenarios: running devpi-sever from the command-line interactively (with its built-in webserver) or running devpi-server from supervisord with nginx in front of it.

Unfortunately, I'm using DevPI inside a large organisation, and while I could campaign for supervisord and nginx, it would be much easier for me if I could just deploy DevPI with Apache and mod_wsgi like all our other internal websites.

Currently I have a WSGI script that calls main.XOM.create_app() to build the application object, but there's some delicate and undocumented configuration that needs to happen before calling that method. For DevPI 1.2 it wasn't too difficult to get something working, but DevPI 2.0 adds setuptools plugins into the mix.

Ideally, my WSGI script would look something like this:

VIRTENV_PATH = "/path/to/devpi-virtenv"

# Make sure all DevPI's requirements are available
import site
site.addsitedir(VIRTENV_PATH)

# Log everything to a handy location
import logging
logging.basicConfig(filename="/var/log/devpi.log")

# Build our application object
from devpi_server import wsgi
application = wsgi.create_app(
    serverdir="/var/local/devpi",
    refresh=60,
    debug=False,
)

Comments (8)

  1. Florian Schulze

    It's not possible to use mod_wsgi with devpi-server, because we spawn long running threads and probably do other incompatible things.

    This should be documented though.

  2. Daniel Holth

    I am attempting to deploy devpi in uwsgi. It seems to work so far. All I had to do to get the web interface to run was to add one line return app right after app=xom.create_app() in XOM.main()

    uwsgi ini file:

    [uwsgi]
    # the important part is that threading is enabled
    # additional config for socket= etc (listen for connections!), logging will be necessary
    plugin=python27
    threads=1
    processes=1
    wsgi-file=%d/wsgi.py
    

    wsgi.py: (main() has been modified to return the wsgi application instead of starting a server)

    # Build our application object
    import devpi_server.main
    application = devpi_server.main.main()
    
  3. Florian Schulze

    I'm all for adding small fixes like that if it helps, but in the docs I would clearly write "You are on your own!"

  4. Timothy Allen reporter

    Are long-running threads really incompatible with mod_wsgi? Although I couldn't get DevPI 2.x running, the problems I encountered didn't immediately look like "can't launch long-running threads", and DevPI 1.x works pretty well in that environment.

    Otherwise, yes, it would be good to document that devpi-server is a monolithic application that must be run as its own daemon, perhaps behind a reverse proxy.

  5. Eli Collins

    I'm not positive as to how threads normally behave under mod_wsgi -- wsgi apps are normally run under a shared python VM inside apache, and are dependant on apache's configured worker model.

    However, if using mod_wsgi's WSGIDaemonProcess directive, each application can be launched in it's own separate subprocess. I've got a couple of wsgi applications that spawn long running threads (and subprocesses), and haven't had any troubles under this model. You can even kill the subprocess, and apache/mod_wsgi restarts it. It's essentially the same behavior as the nginx proxy + supervisord combo described in the devpi docs.

    If this is ever supported officially, making a recommendation to use this directive might help prevent (potential) threading issues.

  6. Log in to comment