Bitbucket is a code hosting site with unlimited public and private repositories. We're also free for small teams!

Close

Xenv

Xenv stands for Execution Environment. A xenv is a specific set of applications, libraries and files that are used when running some application.

Why?

Why would anyone need to use xenv? Well, the reason for using an xenv is that it provides a simple way to layer your execution environment within a larger environment.

In order to create a consistent deployment model you need to make agreements on what the deployed systems look like. At the lowest level you have standards such as POSIX and the applications that make up the operating system. In the Linux world, you have a distribution that makes some decisions regarding things like system level package management and directory structure. You application may rely on certain applications being present along with libraries. All these systems effectively layer upon each other in order to provide different levels of support.

Often times developers utilize virtualization technology to create an execution environment for their applications. This is a very poweful model, yet it is somewhat inefficient in that you reproduce every layer of support in order to run some application. Another approach is to reuse much of the lower level resources. This is where xenv comes in.

Xenv allows you to create an application level execution environment that focuses on installing application level tools while reusing the system level resources. By providing an application level execution environment it allows each level of support to function independently. The application level has the ability to install and use the underlying system, while also allowing overriding those features as necessary.

The result is that system administrators can safely upgrade systems without being limited by an application requirements. An application has a means of using the libraries it needs without impacting the underlying system or other execution environments.

Virtualenv and Python

Xenv is written specifically with Python applications in mind. Hence it uses virtualenv in order to support Python within an execution environment. The difference between xenv and virtualenv is that a xenv is meant to wrap vritualenv in a way that is compatible with other runtimes. This allows a consistent pattern no matter the runtime environment. The xenv allows deploying and running a Java application in the same way you would run a Python application.

With all that said, virtualenv is the only system somewhat formally supported. There is generic support for any executable, but no effort has been placed in running something more specific such as Java. I'd expect this to change if others see use in this idea and adopt it directly or in concept, then I'd imagine support for a specific runtime being developed.

How to use it

Xenv provides a command line application 'xe' that is an entry point to the execution environment. Its primary purpose is to make sure the correct environment has been configured. Here is the help output:

usage: xe [-h] [--root ROOT] {create,run} ...

Utilize a specific executable environment (xenv) for running applications

positional arguments:
  {create,run}          sub command help
    create              Create a new xenv
    run                 Run a command from the xenv

optional arguments:
  -h, --help            show this help message and exit
  --root ROOT, -r ROOT  The xenv root. This will override using the XE_ROOT env var.

Creating a xenv will make a directory with a set of folders. The most important folder is the /usr directory. This is where a virtualenv is installed. Here is an example creating a new xenv and calling its Python interpreter.

$ xe create xenv
mkdir xenv (mode 511)
mkdir xenv/local (mode 511)
New python executable in xenv/usr/bin/python
Installing setuptools............done.
Installing pip...............done.
$ XE_ROOT=xenv xe run python

Packages

A package is a single file that contains a directory structure to untar on top a xenv. It may place files anywhere in the xenv and is responsible for not corrupting the environment.

The only restriction is that it may not add any hidden (.) files to the root directory.

Package Name

The package must be a tarball (.tar) or gzipped tar (.tar.gz). It must be named using the following pattern:

{package-name}-{version}.tar[.gz]

The version must comply with the semver.org specification.

File Hiearchy

The file hierarchy can be whatever the package creator likes. The only restrictions are the following.

The /usr Directory

The usr directory is where an xenv installs its core utilities and executables. For example, we install a virtualenv in /usr for using Python.

The /local Directory

If you create a /local/bin, it will be added to the PATH when running something via xe.

Installation Hooks

A xenv package does not aim to be a "real" package format and as such, it may wrap a more complex package such as an RPM. For this reason we provide a package hook in order to allow installing these sorts of packages.

When a package is installed (untarred) in the xenv, an install script can be placed at /packages/install. This file must be executable. It may use any language that is available on the host system via a shebang.

After it is installed, the install script will be moved to:

/packages/{package-name}/{version}/install

In the future xenv will also supporting a "uninstall" hook file as well that can be used to script removing packages that might not have been untarred. For example, if you wanted to make a package that installed some packages from pypi, it might be a single tarball with an install and uninstall script.

Here is what an install script might look like:

#!/usr/bin/bash

xe run pip install pyramid
xe run pip install http://urlto.somerepo.com/my-pyramidapp-4.3.5.tar.gz

And the uninstall might be:

#!/usr/bin/bash

xe run pip uninstall my-pyramidapp
xe run pip uninstall pyramid

In this way the "package" is really just a tool to deploy the application via other means.

Seriously? That is it?

Yup! The reason for making the package is that you can install it to get the 'xe' command. I think that is a helpful first step. There is nothing stopping you from using a Makefile or pavement.py to do the exact same sort of thing.

In the future I'm hoping that some sort of package management aspect can be included. I'd like to make it possible to take a xenv and install RPMs and/or tarballs. I'd also like the 'xe' command to support customized actions such as running tests for an application. This would probably require some agreement by the applications, but including a simple test directory with an appname and a set of scripts seems like a reasonable first step.

I realize this isn't that big a deal but I hope the idea makes more sense then the actual implementation. At the moment, xenv is really for my own use more than others. I have no problem with that changing if others feel like the idea has legs.

Recent activity

Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

05211cd - Added an id command to help see where the current xenv is. This is helpful when you want to verify you are using the right ...
Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

d4a15e7 - The xe command will layer on top of the current env's PATH
Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

1d501f9 - Added a python runner.
Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

391eb1d - Adding an uninstall hook
Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

0cc64a0 - Added an uninstall command
Eric Larson

Commits by Eric Larson were pushed to elarson/xenv

3aaddab - Added a really simple package format
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.