HTTPS SSH

GO-Docker-Web

Web interface to manage via the web or the API GO-Docker (https://bitbucket.org/osallou/go-docker/overview). GO-Docker is a cluster management tool with Docker to execute jobs in Docker containers with no specific privileges, mount user home directories or other local cluster shared directories, ...

Features

  • Create a job
  • List running jobs and finished jobs
  • View job details and generated files
  • View running jobs usage (cpu, ram) using cAdvisor
  • Historical usage using cAdvisor+specific stats with InfluxDB backend.
  • Prometheus endpoint
  • Marketplace: save a Job as a new recipe for sharing and replay
  • Task sharing: via a public URL allow access to task info (requirements, script, ...) as well as generated files

Requirements

  • GO-Docker
  • Optional but recommended: cAdvisor running on all nodes, configured with InfluxDB as backend

Security

Web server should run behind an HTTPS proxy.

Installation

For development: (bower is needed on host)

cd godweb/webapp
bower install
# during bower install, angular 1.3.14 must be selected if asked, and codemirror 5.0.0
cd ../../
python setup.py develop

For production:

# Makes use of precompiled/minified css/js libs in godweb/dist
# If you wish to regenerate them:
# cd godweb/app
# bower install
# grunt
python setup.py develop

Running

Development

export PYRAMID_ENV=dev # optional, default is dev
rm -rf ..path_to/godocker-prometheus-multiproc
mkdir -p ..path_to/godocker-prometheus-multiproc
export prometheus_multiproc_dir=..path_to/godocker-prometheus-multiproc
cp development.ini.example development.ini
gunicorn -c gunicorn_conf.py --paste development.ini

Production

export PYRAMID_ENV=prod
rm -rf ..path_to/godocker-prometheus-multiproc
mkdir -p ..path_to/godocker-prometheus-multiproc
export prometheus_multiproc_dir=..path_to/godocker-prometheus-multiproc
cp production.ini.example production.ini
gunicorn -c gunicorn_conf.py -D -p godweb.pid --log-config=production.ini --paste production.ini

Systemd service

Following example should be adapted to your installation directories, user running the web service should have write access to pid file. According to database location, you may want to add start dependency on mongod, redis, docker services.

[Unit]
Description=gunicorn godocker daemon

[Service]
PIDFile=/var/run/godockerweb.pid
WorkingDirectory=/opt/go-docker-web
Environment="PYRAMID_ENV=prod"
ExecStart=/usr/bin/gunicorn --name godockerweb --pid /var/run/godockerweb.pid --log-config=production.ini --paste production.ini
ExecReload=/bin/kill -s HUP $MAINPID
ExecStop=/bin/kill -s TERM $MAINPID
PrivateTmp=true

[Install]
WantedBy=multi-user.target

API

API in json (swagger compatible) in available in doc/godocker.json (online version available at http://www.genouest.org/api/godocker-api/. API is available at http://go-docker-web.readthedocs.org/en/latest/views.html#go-d-docker-web-api-reference

Communication should be done over HTTPS and use the user API Key for authentication.

Authenticate to get a token Bearer, then add token in Authorization header of API calls (Bearer token).

An explanation and example usage is available at https://auth0.com/blog/2014/01/07/angularjs-authentication-with-cookies-vs-token/

User settings

In user settings, user has access to his API Key to use the Go-Docker API.

It is also possible to put the public SSH Key for interactive jobs

System Dependencies

Debian/Ubuntu: openssl, libpython-dev, libffi-dev, libssl-dev Fedora/CenOs: openssl, python-devel, libffi-devel, openssl-devel

Badges

Build Status

codecov.io

Dependency Status

Image credits

https://upload.wikimedia.org/wikipedia/commons/1/13/Container_ship_Hanjin_Taipei.jpg http://upload.wikimedia.org/wikipedia/commons/e/e5/Nanoscience_High-Performance_Computing_Facility.jpg http://upload.wikimedia.org/wikipedia/commons/4/4b/Above_the_Clouds.jpg