Turn any machine image into a scalable cloud cluster platform

Added: 05 Jul 2012

Ready to use machine images for using CloudMan cluster platforms exist on AWS and NeCTAR clouds (see BioCloudCentral.org about starting one). However, if you have a custom machine image and would like to use CloudMan with it, that is also possible. CloudMan can be installed on any (Ubuntu) machine image and thus turn an instance of that machine image into a cluster on the cloud instance. More so, the process of installing CloudMan on a custom machine image is automated and documented.

To get started, clone the set of mi-deployment scripts from https://bitbucket.org/afgane/mi-deployment/ to your local machine:

$ hg clone https://bitbucket.org/afgane/mi-deployment/
$ cd mi-deployment

Before actually running the scripts, make sure your system environment is properly setup: mi-deployment scripts require fabric and boto, so if you do not already have those installed on your local machine, do so with:

$ pip install fabric
$ pip install boto

Next, create the following file with your cloud credentials:

$ cat ~/.boto
[Credentials]
aws_access_key_id = <your cloud access key>
aws_secret_access_key = <your cloud secret key>

Lastly, define which applications you would like to have installed on your machine image by editing mi-deployment/conf_files/apps.yaml and (un)commenting any applications whose dependencies you want or do not want installed (this is primarily cloudman but others are also supported). 

The local environment is now properly setup, so start an instance of your machine image and simply start the mi-deployment configuration script. From your local machine, run:

$ fab -f mi_fabfile.py -i <key_file> -H <instance_IP> configure_MI

After the configuration step completes (typically, 6-8 minutes if CloudMan is the only application your're adding), bundle the instance into a new machine image by running the following command:

$ fab -f mi_fabfile.py -i <key_file> -H <instance_IP> create_image

And there it is. A new machine image with all of your customizations plus all the features of the CloudMan platform is ready to be used.

Tags: customize , mi-deployment