A simple python script to provision and run a twisted server on Amazon S3


A wee bit simpler than trying to use Cloud formation or Beanstalk


  • git clone

    • This grabs the code required into a local directory
  • Install pip

    • You hopefully have this already setup.
  • pip install -r requirements.txt

    • This will install Boto and Fabric to your local machine.
    • However I found that while both install via pip, on a windows machine I needed to install pycrypto via a precompiled package on my windows 7 machine.
  • Configure your .boto

    • My script assumes the use of a .boto config file that allows you to connect without asking for credentials each time.
  • Create an AWS keypair for the machine that you are going to launch in the EC2 AWS console.

    • While you could create a new one each time, I use the same one for dev machines as I was launching and terminating them often. For ease of use just dump it in the same directory as the, if this is only going to be a short lived dev server.
  • Create a security group for the class of machine you are going to launch in the EC2 AWS console.

    • A security group is essentially a set of firewall rules, just stick in the ports you are going to use in the twisted app. Remember to keep the SSH port 22 open.
  • Make sure your code is externally accessible via git, subversion, or your source control of choice.

    • In my case I was using svn to checkout of a google code repo. We do this so it is easy to just pull / checkout the code to a location and then run it.
    • Make sure that your code also has a pip requirements file so that the required python packages can be installed in one command.
  • Create a configuration script that will run the provision script

    • This is just another python file that will run the provision script with a dictionary of values, dump it in the same directory as the provision script and you wont have any path issues. Shown below is the one I used for mailthrottler.

      import provision
      config = {
          "region" : "us-east-1",
          "image_id" : "ami-51792c38",
          "instance_type" : "t1.micro",
          "security_groups" : ["mailthrottler-1"],
          "key_name" : "mailthrottler",
          "key_location" : "mailthrottler.pem",
          "sc_output_location" : "mailthrottler",
          "sc_command" : "svn checkout mailthrottler",
          "twistd_command" : "twistd -o -y mailThrottler.tac && sleep 5",
          "twistd_proof_of_life" : "tail"
    • The config keys have the following meanings

      • region = the region to launch the instance into
      • image_id = the id of the image to use (ami-51792c38 is the 2013-09 AWS 32bit Linux)
      • instance_type = is the size of the instance to launch
      • security_groups = is the name of the security group you created above (has to be in a list)
      • key_name = is the name of the keypair that you created above
      • key_location = the path to the keypair .pem file that you download when you created the keypair
      • sc_output_location = the location that the source control files will be created in
      • sc_command = the full command to get the files from source control
      • twistd_command = the command to run twistd on your code. Include a && sleep 5 at the end as for some reason the fabric doesn't give twistd time to start.
      • twistd_proof_of_life = the command to prove that twistd started. I am just tailing the twistd pid file which is created when it starts.
  • Run the above script

  • To stop the server I just terminate it. But if you have data that you want to preserve that isn't a good idea.