KPI

KPI is the new form builder that supersede dkobo.

TODO

~~disable dkobo image~~
~~fix nginx configuration~~
~~remove link for switching back~~
send emails

Prerequisites

docker-engine and docker-compose
clone the kobo-install repository in ~/src (git clone git@bitbucket.org:unhcr/kobo-install.git) and switch to kpi branch: git fetch && git checkout kpi
clone the kobo-dockers repository in ~/src (git clone git@bitbucket.org:unhcr/kobo-dockers.git) and switch to kpi branch: git fetch && git checkout kpi

Settings

Edit settings to suit your needs on set_vars and common.tpl

set_vars settings

set KOBO_DOMAIN: unhcr.dev (in staging), unchr.org (in production)
set KOBO_SSL_KEY and KOBO_SSL_CRT: check Configure Nginx to Use SSL
set NGINX_HOST to 0.0.0.0
set DEFAULT_FILE_STORAGE: $e DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage'

diff --git a/set_vars b/set_vars
index 8cafa4f..4bfc09d 100644
--- a/set_vars
+++ b/set_vars
@@ -14,9 +14,9 @@ $e KOBO_SERVER_IP=$DOCKER0_IP
 $e KOBO_WB_SERVER_IP=${KOBO_SERVER_IP}
 $e KOBO_DB_SERVER_IP=${KOBO_SERVER_IP}
 $e KOBO_PREFIX=
-$e KOBO_DOMAIN=humanitarianresponse.info
-$e KOBO_SSL_KEY=""
-$e KOBO_SSL_CRT=""
+$e KOBO_DOMAIN=unhcr.dev
+$e KOBO_SSL_KEY="MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDXfhEq6SeJ8e0S 21wl6qcO96ed7/c4LioD7kXgKQHnO6Z3aQgySMgiKPXI1Nhl8Zkie08nsAIFfoPB bL3uPsMmsrjPLFMcz95qbVeriMZoQqaZhjO7U3e/HpBDaqOvOW3wkf
+$e KOBO_SSL_CRT="MIIDXTCCAkWgAwIBAgIJAPHrtUu6JN3RMA0GCSqGSIb3DQEBCwUAMEUxCzAJBgNV BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX aWRnaXRzIFB0eSBMdGQwHhcNMTUxMTE4MTM0NjI4WhcNMTYxMTE3MT
 $e TRACKJS_TOKEN=''
 $e GOOGLE_ANALYTICS_TOKEN=''

@@ -58,7 +58,7 @@ $e MONGO_HOST=${KOBO_DB_SERVER_IP}
 $e MONGO_PORT=27017

 # -------------- NginX -----------------
-$e NGINX_HOST=${KOBO_WB_SERVER_IP}
+$e NGINX_HOST=0.0.0.0
 $e NGINX_HTTP_PORT=80
 $e NGINX_HTTPS_PORT=443

@@ -66,7 +66,7 @@ $e NGINX_HTTPS_PORT=443
 ##################### PROJECT SETTINGS ######################

 $e ENKETO_SERVER_ADDR="https://enketo.kobotoolbox.org"
-ENKETO_VERSION=legacy  # [legacy | express]
+$e ENKETO_VERSION=express # [legacy | express]
 $e KOBO_PSQL_DB_NAME=kobotoolbox
 $e KOBO_PSQL_DB_USER=kobo
 $e KOBO_PSQL_DB_PASS=kobo
@@ -86,6 +86,7 @@ $e KOBOCAT_MEDIA_URL=http://${KOBOCAT_PUBLIC_ADDR}/media/
 #$e AWS_S3_SECRET_ACCESS_KEY='not-a-real-key'
 #$e AWS_S3_STORAGE_BUCKET_NAME='kobostatic'
 #$e DEFAULT_FILE_STORAGE='storages.backends.s3boto.S3BotoStorage'
+$e DEFAULT_FILE_STORAGE='django.core.files.storage.FileSystemStorage'

 $e ENKETO_URL=${ENKETO_SERVER_ADDR}
 # this is the protocol enketo will try to use to reach our server!
@@ -100,7 +101,7 @@ else
     $e ENKETO_OFFLINE_SURVEYS='False'
 fi
 # Prepended settings.
-$e KOBOCAT_DJANGO_DEBUG=False
+$e KOBOCAT_DJANGO_DEBUG=True
 $e KOBOCAT_DJANGO_SETTINGS_MODULE="onadata.settings.kc_environ"
 $e KOBOCAT_RAVEN_DSN=

@@ -117,7 +118,7 @@ $e EMAIL_BACKEND='django.core.mail.backends.filebased.EmailBackend'
 $e EMAIL_FILE_PATH=${SRC_DIR}/emails

 # Prepended settings.
-$e KOBOFORM_DJANGO_DEBUG=False
+$e KOBOFORM_DJANGO_DEBUG=True
 $e KOBOFORM_DJANGO_LIVE_RELOAD=False
 $e KOBOFORM_DJANGO_SITE_ID=1
 $e KOBOFORM_DJANGO_SETTINGS_MODULE="dkobo.settings"
@@ -125,9 +126,9 @@ $e KOBOFORM_RAVEN_DSN=


 # -------------- KPI-only -------------------
-$e KPI_PREFIX=/forms
+$e KPI_PREFIX=/
 $e KOBO_USE_MINIFIED_SCRIPTS=False
-$e DKOBO_PREFIX=/
+$e DKOBO_PREFIX=/dkobo
 $e KOBO_SURVEY_PREVIEW_EXPIRATION=24

 # Prepended settings.

common.tpl settings

diff --git a/common.tpl b/common.tpl
index 685cbd0..4ab2329 100644
--- a/common.tpl
+++ b/common.tpl
@@ -102,8 +102,8 @@ web:
     - ./env_nginx
   #  - ./env_secrets
   ports:
-    - "${KOBO_WB_SERVER_IP}:${NGINX_HTTP_PORT}:80"
-    - "${KOBO_WB_SERVER_IP}:${NGINX_HTTPS_PORT}:443"
+    - "${NGINX_HOST}:${NGINX_HTTP_PORT}:80"
+    - "${NGINX_HOST}:${NGINX_HTTPS_PORT}:443"
   volumes:
       - "${VOL_WB}/static:/srv/www:ro"
       # get the logs out of glusterfs!

Installation

run bash prepare.sh (inspect, change and save if needed, exit)
docker-compose up -d
grab a coffee it will take some time until all images are pulled and containers are started
docker exec into kobocat:
- docker exec -i -t koboinstall_kobocat_1 bash /srv/src/init.sh -> ONLY ON NEW INSTALLS!
- Temporary solution for postgres tables mess
  - docker exec -i -t koboinstall_kobocat_1 python /srv/src/kobocat/manage.py migrate --fake reversion
  - docker exec -i -t koboinstall_kobocat_1 python /srv/src/kobocat/manage.py migrate --fake oauth2_provider
  - docker exec -i -t koboinstall_kobocat_1 python /srv/src/kobocat/manage.py migrate --fake authtoken
  - docker exec -i -t koboinstall_kobocat_1 python /srv/src/kobocat/manage.py migrate --fake taggit
  - rerun docker exec -i -t koboinstall_kobocat_1 bash /srv/src/init.sh
- docker exec -i -t koboinstall_kobocat_1 bash /srv/src/sync_static.sh
docker exec into mongo:
- docker exec -i -t koboinstall_mongo_1 bash /srv/add_index.sh <- this will add a missing index to mongo db that will greatly improve speed

Post-installation

Add kobo user

Previously this taskhas been handled by a dkobo script (that at some point should be replicated in kpi). For the time being (2015-01-21):

Login in kpi: docker exec -i -t koboinstall_kpi_1 bash
Go in the main directory: cd /srv/src/kpi
Run python manage.py createsuperuser (kobo/kobo being the default password)
Run this command (kobo/kobo being the default password): echo "from django.contrib.auth.models import User; print 'UserExists' if User.objects.filter(username='kobo').count() > 0 else User.objects.create_superuser('kobo', 'kobo@example.com', 'kobo');" | python manage.py shell 2>&1

Remove the "leave beta"

Login in kpi: docker exec -i -t koboinstall_kpi_1 bash
Go in the main directory: cd /srv/src/kpi
Edit jsapp/js/components/drawer.es6 and remove the line 104: <DrawerLink label={t('leave beta')} href={`${dataInterface.rootUrl}/hub/switch_builder?beta=0`} fa-icon='circle-o' />
Rebuild the scripts: grunt buildall
Collect static files: ./manage.py collectstatic --noinput
logout from the docker container and restart it: docker restart koboinstall_kpi_1

Registration - email configuration

Temporary solution - check Sending email in django
set_vars

comment 2 lines from koboforms:
- #$e EMAIL_BACKEND='django.core.mail.backends.filebased.EmailBackend'
- #$e EMAIL_FILE_PATH=${SRC_DIR}/emails
add under KPI-only :
- $e EMAIL_HOST='smtp.gmail.com'
- $e EMAIL_HOST_USER='EMAIL'
- $e EMAIL_HOST_PASSWORD='PWD'
- $e EMAIL_PORT=587
- $e EMAIL_USE_TLS=True
- $e EMAIL_BACKEND='django.core.mail.backends.smtp.EmailBackend'
add to env_kpi.tpl
- EMAIL_HOST=${EMAIL_HOST}
- EMAIL_HOST_USER=${EMAIL_HOST_USER}
- EMAIL_HOST_PASSWORD=${EMAIL_HOST_PASSWORD}
- EMAIL_PORT=${EMAIL_PORT}
- EMAIL_USE_TLS=${EMAIL_USE_TLS}
Login in kpi: docker exec -i -t koboinstall_kpi_1 bash
Go to directory: cd /srv/src/kpi/kobo_playground

Edit settings.pyand apply the patch:

diff --git a/kobo_playground/settings.py b/kobo_playground/settings.py
index ad8cf39..d0a52c1 100644
--- a/kobo_playground/settings.py
+++ b/kobo_playground/settings.py
@@ -249,6 +249,19 @@ if os.environ.get('DEFAULT_FROM_EMAIL'):
     DEFAULT_FROM_EMAIL = os.environ.get('DEFAULT_FROM_EMAIL')
     SERVER_EMAIL = DEFAULT_FROM_EMAIL

+if os.environ.get('EMAIL_HOST'):
+    EMAIL_HOST = os.environ.get('EMAIL_HOST')
+
+if os.environ.get('EMAIL_HOST_USER'):
+    EMAIL_HOST_USER = os.environ.get('EMAIL_HOST_USER')
+    EMAIL_HOST_PASSWORD = os.environ.get('EMAIL_HOST_PASSWORD')
+
+if os.environ.get('EMAIL_PORT'):
+    EMAIL_PORT = os.environ.get('EMAIL_PORT')
+
+if os.environ.get('EMAIL_USE_TLS'):
+    EMAIL_USE_TLS = os.environ.get('EMAIL_USE_TLS')
+
 if os.environ.get('AWS_ACCESS_KEY_ID'):
     AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
     AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')

Logout from the docker container and restart it: docker restart koboinstall_kpi_1

Enketo

TODO

Merge ports definition in the set_vars

Prerequisites

clone the kobo-install repository in ~/src (git clone git@bitbucket.org:unhcr/enketo-express.git) and switch to docker-unhcr branch: git fetch && git checkout docker-unhcr

Settings

Set Enketo Express's API key, the linked form and data server, and any additional desired configurations in the file setup/docker/envfile.txt.

diff --git a/setup/docker/envfile.txt b/setup/docker/envfile.txt
index 8a27dbe..25caf03 100644
--- a/setup/docker/envfile.txt
+++ b/setup/docker/envfile.txt
@@ -1,9 +1,9 @@
 # Necessary variable.
-ENKETO_API_KEY=
+ENKETO_API_KEY=secret

 # Optional variables (and their default values).
 #ENKETO_OFFLINE_SURVEYS=True
-#ENKETO_FORM_DATA_SERVER_URL=kobocat
+ENKETO_FORM_DATA_SERVER_URL=.*
 #ENKETO_REDIS_MAIN_HOST=redis_main
 #ENKETO_REDIS_MAIN_PORT=6379
 #ENKETO_REDIS_MAIN_PASSWORD=

(Optional) For HTTPS, copy your SSL certificate and key files to setup/docker/secrets/ssl.crt and setup/docker/secrets/ssl.key respectively (take care not to commit these files back to any public git repository). Plain HTTP requests to Enketo Express will be automatically redirected to https://${SSL_HOSTNAME}/$request_uri; the SSL_HOSTNAME environment variable can set e.g. with export SSL_HOSTNAME=ee-staging.kobotoolbox.org before running Enketo (see step 4).
Retrieve the IP address of the docker0 bridge network interface: ifconfig | grep -A 1 'docker0' | tail -1 | cut -d ':' -f 2 | cut -d ' ' -f 1
Edit the docker-compose.yml and insert the ip in the ports definition. TODO: include in set_vars/.tpl files

Installation

Execute docker-compose up -d from the project directory and check with docker ps.

Wiki

kobo-install / kpi-enketo

KPI

TODO

Prerequisites

Settings

Installation

Post-installation

Add kobo user

Remove the "leave beta"

Registration - email configuration

Enketo

TODO

Prerequisites

Settings

Installation