HTTPS SSH

SPECS Enabling Platform -- Cloud Resource Allocator

SPECS Cloud Resource Allocator is in charge with SPECS Enabling Platform hardware resources acquisition.

Table of contents

Installation

Requirements

  • Python-2.7
  • apache-libcloud-0.12.4
  • cherrypy-3.2.4
  • soappy
  • paramiko

Installation steps

These installation steps are compatible with mOS 4.0.x or OpenSUSE 13.1 (with SPECS repositories activated) environments.

Install the requirements

zypper install python python-pip mercurial
cd /opt
hg clone https://bitbucket.org/specs-team/specs-core-enabling_platform-cloud-resource-allocator
cd specs-core-enabling_platform-cloud-resource-allocator
pip install -r requirements.txt

Control the service daemon

/opt/specs-core-enabling_platform-cloud-resource-allocator/run.sh [start | stop]

Configuration steps

Customize the Resource Allocator

In /opt/specs-core-enabling_platform-cloud-resource-allocator/conf/ there are the configuration files that can be customized to fit the user needs.

seetings.conf

A JSON file format with the following information:

  • users : path to the users database file;
  • db : path to database directory;
  • log : stdout and stderr log files;
  • pidfile : pid file;
  • operations: supported operations with it's own functions (unless you are a developer do not edit operations key);
  • plugins: plugin declaration for various features to be enabled on top of the Resource Allocator;
{
    "users": "/conf/users.conf",
    "db": "/db",
    "log": {
        "stdout": "log/cloud-ra.out",
        "stderr": "log/cloud-ra.err"
    },
    "pidfile": "log/cloud-ra.pid",
    "operations": {
        "query": [
            "list_images",
            "list_roles",
            "list_providers",
            "list_apps",
            "list_sessions"
        ],
        "control": [
            "create_node",
            "list_node",
            "terminate_node",
            "run_command"
        ],
        "clusterformation": [
            "cluster_deployment",
            "cluster_terminate",
            "cluster_query"
        ]
    },
    "plugins": {
        "cluster-formation": {
            "path": "/plugins/cluster-formation"
        }
    }
}

users.conf

A JSON file format with the following information:

  • token : unique identifier;
  • name : user name;
  • acl : a list of allowed operations for the current user;
[
    { 
        "token" : "73f956c423sfd177793c63c5be30ba21",
        "name" : "username",
        "acl" : [ "list_roles", "list_images", "list_providers", "create_node", "start_node", "stop_node", "run_command" ]
    }
]

providers.conf

[
    {
        "connection": {
            "hostname": "cloud_controller_address", 
            "port": "cloud_controller_port", 
            "ssl": "0"
        }, 
        "driver": "cloud_provider_driver_name", 
        "name": "cloud_provider_name",
        "zones": [
                "cloud_zone_1"
        ],
        "resources": [
            {
                "cpu": 1, 
                "disk": 8, 
                "ecu": 1, 
                "memory": 0.5, 
                "name": "resource_size_name_1", 
                "platform": 0
            }
        ], 
        "types": [
            "resource_size_name_1" 
        ]
    },
     {
        "connection": {
            "hostname": "0.0.0.0", 
            "port": "0", 
            "ssl": "0"
        }, 
        "driver": "dummy", 
        "name": "dummy",
        "resources" : [
            {
                "name": "dummy"
            }
        ]
    }
]

NOTE: the default setup is preconfigured to support Amazon EC2 and HP Helion Eucalyptus.

images.conf

[
    {
        "description": "mOS Custom OS 64bit", 
        "name": "mOS-image-version", 
        "platform": 64, 
        "supported": [
            {
                "image": "emi-ID", 
                "kernel": "eki-ID", 
                "provider": "cloud_provider_name", 
                "ramdisk": "eri-ID"
            }
        ], 
        "version": "4.2"
    }
]
  • NOTE -- ['supported']['provider'] is linked with providers.conf->[name];

application.conf

[
    {
        "description": "Application description", 
        "maintainer": "maintainer@email.address", 
        "name": "application-name", 
        "supported": [
            "mOS-image-version"
        ], 
        "userdata": "default_userdata"
    }
]
  • NOTE: [supported] is linked with images.conf->[name];

Usage

The Resource Allocator is a RESTful service that takes as an input a JSON document that described the desired operation.

For example, by using curl it can be accessed like:

curl -i -L -H "Content-Type: application/json" -X PUT -T operation.json http://RESOURCE_ALLOCATOR_IP:RESOURCE_ALLOCATOR_PORT/cloud/[list|control]

In return, the service will reply with an JSON document:

{
  "code"   : "int",
  "message": "description message",
  "payload": "extra information"
}
  • code - value 0 means operation successfull while 1 means that an error occured;

Concepts

  • application : a metadata that describes a suite of packages (representing the underlying running platform for the application) that need to be installed on a predefined * image hosted by a cloud provider;
  • images : a list of information representing a cloud hosted image characteristics (image id, kernel id, ramdisk id, provider name etc.);
  • providers : a list of information representing different supported cloud providers;

The relation between these three actors is: application -(supported)-> image -(hosted)-> cloud provider

List operations

Controller operation list supports the following functions:

  • list_apps: a list with all supported applications;
{ 
  "token"     : "users.conf->[token]", 
  "operation" : "list_apps"
}
  • list_images - a list of supported images;
{ 
  "token"     : "users.conf->[token]", 
  "operation" : "list_images"
}
  • list_providers - a list of supported providers;
{ 
  "token"     : "users.conf->[token]", 
  "operation" : "list_providers"
}

Control operations

create_node

This function will start a predefined (see: images.conf) image hosted by a predefined provider (see: providers.conf).

{
  "token"       : "users.conf[token]",
  "operation"   : "create_node",
  "session_id"  : "s-ASDF2342",
  "cloud_provider" : { 
            "name": "providers.conf{}[name]",
            "credentials": {
                "accesskey" : "user_defined",
                "privatekey" : "user_defined"
            }
             },
  "instance" : {
        "type" : "providers.conf{}[types]",
        "count" : "int",
        "image" : "images.conf{}[name]"
        },
  "application_id" : "applications.conf{}[name]",
  "extra" : { 
        "create_ssh_key" : "0|1",
            "ssh_key_name" : "user_defined",
        "user_data" : "user_defined",
                "security_group_name" : "user_defined",
        "create_security_groups" : "0|1"
        }
}
  • [mandatory] token : user token as defined in users.conf;
  • [mandatory] operation : operation name - create_node;
  • [optional] session_id : session identifier; if not specified a random one will be generated ;
  • [optional] cloud_provider : cloud provider definition is optional in case of already defined session;
  • [mandatory] name : it must match one defined in providers.conf;
  • [mandatory] credentials : accesskey and privatekey must be provided by the user;
  • [mandatory] instance:
  • [mandatory] type: it must match a virtual machine type defined in providers.conf->[types];
  • [mandatory] count: number of instances to be started;
  • [mandatory] image: it must match a cloud image name defined in images.conf{}[name];
  • [mandatory] application_id : the name of the application to be started; it must match applications.conf{}[name];
  • extra: defines extra parameters for the virtual machines;
  • create_ssh_key: generate a new ssh passwordless authentication [0-no, 1-yes]; the * key will be returned in the JSON response;
  • ssh_key_name: the name of the ssh key to be used and/or to be created;
  • security_group_name: the name of the existent or new security group;
  • create_security_groups: create a new security group with open access [0-no, 1-yes];
  • user_data : override applications.conf->[userdata] value;

  • RESPONSE

{
    "code": "0", 
    "message": "OK. Check payload for details.", 
    "payload": {
        "extra": {
            "security_group_name": "sg-da05c7b2", 
            "ssh_key_name": "silviu-hp-1", 
            "ssh_private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEogIBAAKCAQEAo1EZYu3aNDMDMlojC1K1cVqkKjtg2yyQDld4REf36hHw0TH1UqkDKm8yndeA\nE/N4G5jyM3bUjYSBKVTt/1A0GovhEntmuK3gL+Lbb59R0j6VJTckuEGjCeO/UMEq/rPv4+ELexgP\ncZBvT1gmiwE+7R/oV+NC+MCk+v+9SkXuAQfzrI0N0wcjf431ixPtPMj+InIx9fuYfDrXgL9pxz92\nuH47TXA9cuVIWAGf/5E6NTUiyfGOOiDkBunxVb2KtyAsdvyRJbRwQ9DUSewqIVH1qwZDfhoVS3V1\nYrQ0xQUiacjiR9x7pARXfvSlTg8sO+zY/BHZ01BSrtZk2AC6KVcp1wIDAQABAoIBACpJUGJuJaaI\nYtaBwjsnUjWke4c30q9NTYt5GHnNjKJ9vLXd0qBxhJFTM27tIouH6dxzcJrktMpjubHC/NCzqpoJ\nB+DqNMGyu66KdbjkLqJac/Gr4oUW8akY4Ct/qKtEWePOHFPl+fG0rS2FLI2nZ+yr61P+UxPnhZA7\n8X4utx5qzyDT4JSPBjxogy305jUVfYU8W+ASh/D2Ie+hVWRsAZoQx59Ih5jdD02ZANf+6g6N/Mue\n2a2/jCMHxPQ+e/jOZ1ZGxQ7NDHuvWG+DQ6I857eh0fzUhQSwieq5n2JxqvK1H0l3gjkzOiTfoCZ4\ntsYxX1ALx84mSI+Zbeh1ehGIx+ECgYEA5JBxVZ0q/cV01spzs5+7nJtqGfYwNB7WHYXSPWulCo3v\nSFg28ebfqRvGHWpwiKWaSsz4CTh2nnQATrqCGdT67rOnuAqvbNanCdL/tcDFDsaAHrB6kc82KUuR\nuYlwuq69SPQcEH8J/KENxrkuU0qa+o7GCQXEQY71ds3ockSfn9ECgYEAtuuqfZAwE2P5WsJBBx6p\n3MWYsdAdlQp+MZpx/jZhFDoXcJiPrmbauqMhuIveTD/ISyXoxOsT1SRNEccBkQcpY0mdAD73/VXn\nF6X7gjBL0fW23P9ew+5WOCRq6MIt31/sPAoabaNZcF0B9tLmeI62m5aSKltAcYco6gc1Ig6GAScC\ngYBCFtoOjWGO+3drXCLUf416HR4y35LUn03+P2GpTOTuBlh4LLnGfKGTfmpsuwhLGsdgdaAK2n7C\nFny4RwmzengMaXpHpx9Oc0rddltmLJVOTmthdFocqzPYAPfkFC+6ayhkXeF3i6RRnigjnAwfHWVu\nqbXtifNTmYi0j5l04Q7e4QKBgB8+XZ6Y1D1bpCnGaEkJCcZ10cD9ZTAh3LAoSzQ6oZZM3dgTlucZ\nFl8uogHdkhjABDwhu4J3DKopSyiwLypQaNdWqYRzPPR+ibxWU4FHMowoRhW1YMBio0NvpPsgv6kZ\nv0X6C82SSyKQIFfCai9JYRNuQoUNlRJPHcPrez3X8TedAoGAH8XTrR2+uIk59Vo4fc6HykubxmRx\nFCwHLhKXJ+QUlcNjkbUJcPOxV48ZFZk/I3fC1Z5HFv/oIhpVqZdMN2g/ed8P1CU12+L+5l4/wRwU\nkrjj7YDfiF/vTea3PsJCHDiWhViiafpUBkIFW3ww3c9Wk6f3pMLzuqtwbdzKeZ8RVfA=\n-----END RSA PRIVATE KEY-----"
        }, 
        "instances": [
            {
              "id" : "i-ecc7a596", 
              "name" : "i-ecc7a596"
             },
            { 
              "id" : "i-e2c7a598",
              "name" : "i-e2c7a598"
             }
        ], 
        "session_id": "s-E87E77D2"
    }
}

IMPORTANT

Sessions are available by default per user: session_id is created when is called for the first time and [cloud_provider][credentials] (accesskey, privatekey) are saved in settings.conf[db]/users.conf[token]/session_id file; an user can add extra nodes in the current session by specifying the session identifier without the [cloud_provider] key;

{
  "token"       : "73f956c4ad78d177793c63c5be30ba21",
  "session_id"  : "s-E87E77D2",
  "operation"   : "create_node",
  "instance" : {
        "type" : "m1.small",
        "count" : "3",
        "image" : "mos-0.7-32"
        },
  "application_id" : "software_platform",
  "extra" : { 
        "create_ssh_key" : "0",
            "ssh_key_name" : "silviu-hp-1",
        "user_data" : "user_defined",
                "security_group_name": "sg-da05c7b2"
        }
}

list_nodes

This function will list the status of the specified virtual machines. It works in two ways: list based - a list of virtual machines IDs must be specified together with the cloud credentials; session based - this function will list the status of the current registered virtual machines within the session id;

{
  "token"       : "users.conf->[token]",
  "operation"   : "list_node",
  "cloud_provider" : { 
            "name": "cloud_provider_name",
            "credentials": {
                "accesskey" : "user_defined",
                "privatekey" : "user_defined"
            }
             },
  "instances"   : [ "i-ecc7a596", "i-e2c7a598" ]
}
  • session based
{
  "token"       : "users.conf->[token]",
  "operation"   : "list_node",
  "session_id"  : "s-E87E77D2"
}
  • RESPONSE (in both cases)
{
    "code": "0", 
    "message": "list_nodes:[OK]. Check payload for details.", 
    "payload": [
        {
            "dns_name": "ec2-174-129-46-61.compute-1.amazonaws.com", 
            "id": "i-ecc7a596", 
            "launch_time": "2012-09-10T21:07:00.000Z", 
            "ssh_key_name": "silviu-hp-1", 
            "status": "running"
        }, 
        {
            "dns_name": "ec2-23-23-47-167.compute-1.amazonaws.com", 
            "id": "i-e2c7a598", 
            "launch_time": "2012-09-10T21:07:00.000Z", 
            "ssh_key_name": "silviu-hp-1", 
            "status": "running"
        }
    ]
}

terminate_node

This function will terminate running virtual machines. It works in two ways: list based - a user defined list of virtual machines IDs must be specified together with the cloud providers credentials; session based - this function will terminate all the current registered virtual machines within the session id;

{
  "token"       : "users.conf->[token]",
  "operation"   : "terminate_node",
  "cloud_provider" : { 
            "name": "provider_name",
            "credentials": {
                "accesskey" : "user_defined",
                "privatekey" : "user_defined"
            }
             },
  "instances"   : [ "i-ecc7a596", "i-e2c7a598" ]
}
  • session based
{
  "token"       : "users.conf->[token]",
  "operation"   : "terminate_node",
  "session_id"  : "s-E87E77D2"
}
RESPONSE (in both cases)
{
    "code": "0", 
    "message": "terminate_node:[OK]. Check payload for details", 
    "payload": [
        {
            "id": "i-ecc7a596", 
            "status": "terminated"
        }, 
        {
            "id": "i-e2c7a598", 
            "status": "terminated"
        }
    ]
}

NOTICE

This product includes software developed at "Institute e-Austria, Timisoara", as part of the "SPECS - Secure Provisioning of Cloud Services based on SLA Management" research project (an EC FP7-ICT Grant, agreement 610795).

Developers:

Silviu Panica, silviu@solsys.ro / silviu.panica@e-uvt.ro

Copyright:

Copyright 2013-2015, Institute e-Austria, Timisoara, Romania
    http://www.ieat.ro/

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at:
    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.