1. Manlio Perillo
  2. ngx_python_module

Overview

ngx_python_module README

ngx_python_module embeds the CPython interpreter in the Nginx web server.

This software contains code from the Apache mod_wsgi and is being used here with the permission of Graham Dumpleton.

Installation

ngx_python_module requires at least Python 2.5.0. Python 3.x is not supported.

ngx_python_module has been tested with Nginx 1.3.3 (but it should work with older versions, down to 0.6.35).

Download the Nginx sources from http://nginx.net/ and unpack it.

The sources of ngx_python_module can be downloaded from BitBucket.

Configuration for Nginx module

To build Nginx, change to the directory which contains the Nginx sources, and run the configuration script, making sure to add the path to the ngx_python_module sources using the --add-module option:

$ ./configure --add-module=/path/to/ngx_python_module/

Nginx will use as default prefix path the directory /usr/local/nginx.

In case of errors, check that both the Python executable and development headers are properly installed. The Python executable must be available in the system PATH.

It is possible to change the Python instance to use, by manually modifying the NGX_PYTHON variable in the config file.

In case the configuration fails, check the objs/autoconf.err file.

If the configuration was successful, you can build and install the software:

$ make

and, as root (unless a different prefix was specified in configuration):

$ make install

Configuration for Python package

With the ngx_python_module sources come a Python package named nginx.

This package contains support code for handling Nginx from Python.

To install the package, use the setup.py file:

python setup.py install

TODO: dependencies, requirements and test suite.

Directives

Python directives are defined inside the python directive block.

optimize

Syntax:

optimize n

Default:

optimize 0

Context:

python

Description:

Directive sets the optimization level of the Python compiler.

This is equivalent to the -O option of the Python Interpreter.

executable

Syntax:

executable path

Default:

executable python

Context:

python

Description:

Directive sets the value of argv[0].

This is used during interpreter initialization to find the Python run-time libraries relative to the interpreter executable.

Executable path can be relative, and it is resolved against ngx_prefix directory.

See http://docs.python.org/c-api/init.html#Py_SetProgramName.

home

Syntax:

home path

Default:
Context:

python

Description:

Directive sets the default "home" directory, that is, the location of the standard Python libraries.

The libraries are searched in {home}/lib/python{version} and {home}/lib/python{version} directories.

Home path can be relative, and it is resolved against ngx_prefix directory.

See http://docs.python.org/c-api/init.html#Py_SetPythonHome.

exception_handler

Syntax:

exception_handler entry point

Default:

traceback:print_exception

Context:

python

Description:

Directive sets the function used to print an unhandled exception to the Nginx error log.

The syntax is a simplified form of pkg_resources entry points: module:callable.

Custom Python environment setup

There are several methods to setup a custom Python environment, as an example when using virtualenv.

Assuming the virtual environment has been created with:

virtualenv --no-site-packages /usr/local/nginx/env

the recommended method is to use the home directive:

python {
    home /usr/local/nginx/env;
}

As an alternative, it is possible to set the PYTHONPATH or PYTHONHOME environment variables, using the env directive:

env PYTHONPATH /path1:/path2... ;
env PYTHONHOME /path/to/pythonhome;

See: http://docs.python.org/using/cmdline.html#envvar-PYTHONPATH and http://docs.python.org/using/cmdline.html#envvar-PYTHONHOME

It is also possible to use the executable directive:

python {
    executable /usr/local/nginx/env/bin/python;
}

NOTES

  • From the Python/C API Reference Manual:

    On most systems (in particular, on Unix and Windows, although the details are slightly different), Py_Initialize() calculates the module search path based upon its best guess for the location of the standard Python interpreter executable ...

    What this means is that if you change the module search path, the changes will not be visible by Python scripts embedded in Nginx.

    A solution is to call Py_Finalize followed by Py_Initialize when Nginx is reconfigured (via kill -HUP <pid>), but this may leak memory, so it is not used.

    Fortunately there is a far better solution: using the upgrade binary feature of Nginx.

    The procedure is described here: http://wiki.nginx.org/NginxCommandLine#Upgrading_To_a_New_Binary_On_The_Fly.

    By sending the SIGUSR2 signal to the master process, Nginx will start a new master process, that does not inherit the address space of the current master process (the exec syscall is used).

    The new master process will thus have a completely new instance of Python initialized.

    Do not forget to kill the old master process.

  • CPython sub interpreters are not used.

    It make no sense to use multiple, independent, interpreters insided a Nginx instance (use multiple Nginx instances, instead), and sub interpreters support has some issues in CPython.

  • Threads are not supported.

    The reason is to keep the code as simple as possible. Moreover Nginx does not support threads.

    Threads executed in a Python application embedded in Nginx will always be in a suspended state, since the GIL on the main thread is never released.