HTTPS SSH

Deployment

Getting ready

You need to register on http://bitbucket.org and http://github.com and share your public key. If you don't have any, here is the command that generates it, and shows the public key (it's fine to share your public key with others, but never show your private key):

ssh-keygen
cat ~/.ssh/id_rsa.pub

System dependencies

To deploy fowler you need to install the following system packages.

Fedora 20

sudo yum install mercurial git python3 python3-setuptools \
python3-numpy python3-numexpr python3-Cython

sudo yum install yum-utils
sudo yum-builddep python3-tables python3-scipy python3-scikit-learn \
python3-PyYAML

Ubuntu 12..04

sudo add-apt-repository ppa:fkrull/deadsnakes
sudo apt-get update

sudo apt-get install python3.3 python3.3-dev git mercurial libfreetype6-dev libpng-dev

sudo apt-get build-dep python-scipy

python3.3 -m venv .env
source .env/bin/activate

python3 ez_setup.py
python3.3 get-pip.py

.env/local/bin/pip install numpy
.env/local/bin/pip install numexpr

python3 bootstrap.py

Creating virtualenv if numpy, numexpr and other packages are not available

Some packages have to be installed before running buildout. If there is no virtualenv script, install it:

python3 ez_setup.py --user
python3.3 get-pip.py --user
~/.local/bin/pip3.3 install --user virtualenv

Now you are ready to create an isolated virtual invironment:

~/.local/bin/virtualenv -p python3.3 --system-site-packages .env

Note

You can access the python interpreter within the virtual envinroment by using:

.env/bin/python

Buildout

Buildout is a project deployment tool. Use mercurial to get the configuration:

hg clone ssh://hg@bitbucket.org/dimazest/phd-buildout
cd phd-buildout

There are two modes for the project deployment. The base mode installs all the necessary tools. To choose it just copy _base.cfg to buildout.cfg:

cp _buildout.cfg buildout.cfg

However, for the development you will need much more tools, for example test runners. To deploy the development mode, your buildout.cfg should inherit from dev.cfg. So, the head of the buildout.cfg should look like:

[buildout]
extends = dev.cfg

Deployment

Now you are ready to deploy the project. It will take a while while the code repositories are cloned and dependencies are build. Run this inside a virtual machine, if you want to use vagrant:

# You might need to
# export LC_ALL="en_US.UTF-8"

# Note required setuptools version
grep setuptools versions.cfg
setuptools = 18.1  # 18.1 in my case

# And bootstrap buildout with a required version
python3.4 bootstrap-buildout.py --setuptools-version 18.1

bin/buildout

Vagrant

Vagrant is a virtual machine manager. A virtual machine is provided with all the dependencies installed. You just need to install virtualbox and vagrant.

ssh-add ~/.ssh/id_rsa

vagrant up
vagrant ssh

# Note, that /vagrant (inside the virtual machine) shares files on the host machine!
cd /vagrant
# Now you are inside the virtual machine and ready to deploy.
# python3 bootstrap.py
# bin/buildout

Running the experiments

There is the notebooks folder which is ment to store the data and the journal of the experiments.

The idea is to create a subfolder for every experiment for it, for example:

mkdir notebooks/chatlogs
cd notebooks/chatlogs
../../bin/corpora-ipython notebook