HTTPS SSH

ILPSLogging

Requirements

  • Python > 2.7
    • pip
    • virtualenv
  • A relational database (e.g. MySQL or PostgreSQL; tested with PostgreSQL) for storing user and project information
  • An Elasticsearch instance, used for storing and querying logged data
  • A webserver with WSGI or proxy capabilities (tested with uwsgi in combination with nginx)

Automated installation

A Fabric script (fabfile.py) is included in the repository. This script can be used for setting up the server on which ILPSLogging will be running as well as for deploying updated code. This script assumes that you are deploying the application to freshly installed Debian/Ubuntu server (tested with Ubuntu 12.04 64-bit).

The setup process consists of the following tasks:

  1. Setup of the server Install required packages (git, ntp, nginx, postgresql, elasticsearch, etc.)
  2. Setup of the project environment Create the project directories, initialize the database, install required python packeges
  3. Deploying the ILPSLogging code and restarting services Pull the latest from the git repo, restart affected services and build the docs
  4. Create an ILPSLogging admin user

Let's get started!

  1. Clone the repository to your local machine:

    $ git clone git@git.dispectu.com:dispectu/ilps-logging.git
    $ cd ilps-logging
    
  2. Open the settings.py file and inspect the FABRIC settings.

  3. Add a another dictionary to FABRIC with the key production containing the settings of your environment:

    FABRIC = {
       'staging': {},
       'production': {
          # Your settings go here
       }
    }
    
  4. Create the following files in the deploy/ directory:

    • staging_settings.conf
    • staging_nginx.conf
    • staging_supervisord.conf
    • production_kibana.js

    In most cases you can copy the contents of the 'staging' configuration files already present deploy/.

  5. From your local machine, run the Fabric tasks:

    $ fab production setup_server
    $ fab production setup_project
    $ fab production deploy
    

Manual installation

  1. Clone the repository:

    $ git clone git@git.dispectu.com:dispectu/ilps-logging.git
    $ cd ilps-logging
    
  2. Create a virtualenv, activate it and install the required Python packages:

    $ virtualenv ~/my_pyenvs/ilpslogging
    $ source ~/my_pyenvs/ilpslogging/bin/activate
    $ pip install -r requirements.txt
    
  3. Create a local settings file to override the default settings specified in settings.py. In the next steps we describe to mininal number of settings that should be changed to get the application up-and-running. Please have a look at the comments in settings.py to get an overview of all available settings:

    $ vim ilpslogging/local_settings.py
    
  4. When running the application in production, set DEBUG to False

  5. Set the SECRET_KEY for the installation (used, for example, to sign cookies). A good random key can be generated as follows:

    >>> import os
    >>> os.urandom(24)
    '\x86\xb8f\xcc\xbf\xd6f\x96\xf0\x08v\x90\xed\xad\x07\xfa\x01\xd0\\L#\x95\xf6\xdd'
    
  6. Provide the URI of the database (SQLALCHEMY_DATABASE_URI). The SQLAlchemy documentation provides inforamation on how to structure the URI for different databases.

  7. Load the application's database schema into the database (this will create all required tables):

    $ python manage.py db tables_create
    
  8. Create an admin user. This user will receive a notification when a new users registers an account. Multiple users can have an admin role (see the admin_promote and admin_revoke CLI options):

    $ python manage.py users add
    Email address: justin@dispectu.com
    Name: Justin van Wees
    Password:
    Is admin [n]: y
    Created user: <User: u'justin@dispectu.com'>
    
  9. Set the URL of the Elasticsearch server (ELASTICSEARCH_URL).

  10. Provide the settings of the SMTP server that should be used for sending notification emails:

    MAIL_SERVER = 'localhost'
    MAIL_PORT = 25
    MAIL_USERNAME = ''
    MAIL_PASSWORD = ''
    
  11. Set the absolute URLs on which the application interfaces and documentation an be reached:

    KIBANA_URL = 'http://ilpslogging.staging.dispectu.com/kibana/#/dashboard'
    JSSDK_URL = 'http://ilpslogging.staging.dispectu.com/jssdk/'
    DOCS_URL = 'http://ilpslogging.staging.dispectu.com/docs/'
    
  12. Configure the webserver (nginx):

    server {
        listen 80;
        server_name ilpslogging.staging.dispectu.com;
        client_max_body_size 100M;
        keepalive_timeout 60;
    
        gzip on;
        gzip_types application/x-javascript text/javascript;
    
        location / {
            include uwsgi_params;
            uwsgi_pass unix:/tmp/ilpslogging.staging.dispectu.com.sock;
        }
    
        location /static/ {
            root        /path/to/project_repo/ilpslogging/frontend/;
            expires     max;
        }
    
        location /docs/ {
            alias        /path/to/project_repo/docs/build/html/;
        }
    
        location /kibana/ {
            alias       /path/to/project_repo/kibana/;
        }
    
        location /jssdk/ {
            alias       /path/to/project_repo/ilpslogging/frontend/static/jssdk/;
        }
    }
    
  13. Build the JavaScript SDK:

    $ python manage.py build_jssdk
    
  14. Build the application documentation:

    $ cd docs
    $ make html
    
  15. Use uWSGI to run the application:

    $ uwsgi --socket /tmp/lpslogging.staging.dispectu.com.sock --pidfile /path/to/uwsgi.pid -p 2 -w wsgi:application