Deploying CloudStack with Ansible

In this article, Paul Angus Cloud Architect at ShapeBlue takes a look at using Ansible to Deploy an Apache CloudStack cloud.

What is Ansible

Ansible is a deployment and configuration management tool similar in intent to Chef and Puppet. It allows (usually) DevOps teams to orchestrate the deployment and configuration of their environments without having to re-write custom scripts to make changes.

Like Chef and Puppet, Ansible is designed to be idempotent. This means that you determine the state you want a host to be in and Ansible will decide if it needs to act in order to achieve that state.

There’s already Chef and Puppet, so what’s the fuss about Ansible?

Let’s take it as a given that configuration management makes life much easier (and is quite cool), Ansible only needs an SSH connection to the hosts that you’re going to manage to get started. While Ansible requires Python 2.4 or greater on the host you’re going to manage in order to leverage the vast majority of its functionality, it is able to connect to hosts which don’t have Python installed in order to then install Python, so it’s not really a problem. This greatly simplifies the deployment procedure for hosts, avoiding the need to pre-install agents onto the clients before the configuration management can take over.

Ansible will allow you to connect as any user to a managed host (with that user’s privileges) or by using public/private keys – allowing fully automated management.

There also doesn’t need to be a central server to run everything, as long as your playbooks and inventories are in-sync you can create as many Ansible servers as you need (generally a bit of Git pushing and pulling will do the trick).

Finally – its structure and language is pretty simple and clean. I’ve found it a bit tricky to get the syntax correct for variables in some circumstances, but otherwise I’ve found it one of the easier tools to get my head around.

So let’s see something

For this example we’re going to create an Ansible server which will then deploy a CloudStack server. Both of these servers will be CentOS 6.4 virtual machines.

Installing Ansible

Installing Ansible is blessedly easy. We generally prefer to use CentOS so to install Ansible you run the following commands on the Ansible server.

  1. # rpm -ivh http://www.mirrorservice.org/sites/dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
  2. # yum install -y ansible

And that’s it.

(There is a commercial version which has more features such as callback to request configurations and a RESTful API and also support. The installation of this is different)

By default Ansible uses /etc/ansible to store your playbooks, I tend to move it, but there’s no real problem with using the default location. Create yourself a little directory structure to get started with. The documentation recommends something like this:

Playbooks

Ansible uses playbooks to specify the state in which you wish the target host to be in to be able to accomplish its role. Ansible playbooks are written in YAML format.

Modules

To get Ansible to do things you specify the hosts a playbook will act upon and then call modules and supply arguments which determine what Ansible will do to those hosts.

To keep things simple, this example is a cut-down version of a full deployment. This example creates a single management server with a local MySQL server and assumes you have your secondary storage already provisioned somewhere. For this example I’m also not going to include securing the MySQL server, configuring NTP or using Ansible to configure the networking on the hosts either. Although normally we’d use Ansible to do exactly that.

The pre-requisites to this CloudStack build are:

  • A CentOS 6.4 host to install CloudStack on
  • An IP address already assigned on the ACS management host
  • The ACS management host should have a resolvable FQDN (either through DNS or the host file on the ACS management host)
  • Internet connectivity on the ACS management host

Planning

The first step I use is to list all of the tasks I think I’ll need and group them or split them into logical blocks. So for this deployment of CloudStack I’d start with:

  • Configure selinux
  • (libselinux-python required for Ansible to work with selinux enabled hosts)
  • Install and configure MySQL
  • (Python MySQL-DB required for Ansible MySQL module)
  • Install cloud-client
  • Seed secondary storage

Ansible is built around the idea of hosts having roles, so generally you would group or manage your hosts by their roles. So now to create some roles for these tasks

I’ve created:

  • cloudstack-manager
  • mysql

First up we need to tell Ansible where to find our CloudStack management host. In the root Ansible directory there is a file called ‘hosts’ (/etc/Ansible/hosts) add a section like this:

  1. [acs-manager]
  2. xxx.xxx.xxx.xxx

where xxx.xxx.xxx.xxx is the ip address of your ACS management host.

MySQL

So let’s start with the MySQL server. We’ll need to create a task within the mysql role directory called main.yml. The ‘task’ in this case to have MySQL running and configured on the target host. The contents of the file will look like this:

  1. -name: Ensure mysql server is installed
  3. yum: name=mysql-server state=present
  5. - name: Ensure mysql python is installed
  7. yum: name=MySQL-python state=present
  10. - name: Ensure selinux python bindings are installed
  12. yum: name=libselinux-python state=present
  14. - name: Ensure cloudstack specfic my.cnf lines are present
  16. lineinfile: dest=/etc/my.cnf regexp=’$item’ insertafter=”symbolic-links=0″ line=’$item’
  18. with\_items:
  20. – skip-name-resolve
  22. – default-time-zone=’+00:00′
  24. – innodb\_rollback\_on\_timeout=1
  26. – innodb\_lock\_wait\_timeout=600
  28. – max\_connections=350
  30. – log-bin=mysql-bin
  32.  – binlog-format = ‘ROW’
  35. - name: Ensure MySQL service is started
  37. service: name=mysqld state=started
  39. - name: Ensure MySQL service is enabled at boot
  41. service: name=mysqld enabled=yes
  45. - name: Ensure root password is set
  47. mysql\_user: user=root password=$mysql\_root\_password host=localhost
  49. ignore\_errors: true
  51. - name: Ensure root has sufficient privileges
  53. mysql\_user: login\_user=root login\_password=$mysql\_root\_password user=root host=% password=$mysql\_root\_password priv=\*.\*:GRANT,ALL state=present

This needs to be saved as /etc/ansible/roles/mysql/tasks/main.yml

As explained earlier, this playbook in fact describes the state of the host rather than setting out commands to be run. For instance, we specify certain lines which must be in the my.cnf file and allow Ansible to decide whether or not it needs to add them.

Most of the modules are self-explanatory once you see them, but to run through them briefly;

The ‘yum’ module is used to specify which packages are required, the ‘service’ module controls the running of services, while the ‘mysql_user’ module controls mysql user configuration. The ‘lineinfile’ module controls the contents in a file.

We have a couple of variables which need declaring. You could do that within this playbook or its ‘parent’ playbook, or as a higher level variable. I’m going to declare them in a higher level playbook. More on this later.

That’s enough to provision a MySQL server. Now for the management server.

CloudStack Management server service

For the management server role we create a main.yml task like this:

  1. - name: Ensure selinux python bindings are installed
  3.   yum: name=libselinux-python state=present
  6. - name: Ensure the Apache Cloudstack Repo file exists as per template
  8.   template: src=cloudstack.repo.j2 dest=/etc/yum.repos.d/cloudstack.repo
  11. - name: Ensure selinux is in permissive mode
  13.   command: setenforce permissive
  16. - name: Ensure selinux is set permanently
  18.   selinux: policy=targeted state=permissive
  21. -name: Ensure CloudStack packages are installed
  23.   yum: name=cloud-client state=present
  26. - name: Ensure vhdutil is in correct location
  28.   get\_url: url=http://download.cloudstack.org/tools/vhd-util dest=/usr/share/cloudstack-common/scripts/vm/hypervisor/xenserver/vhd-util mode=0755

Save this as /etc/ansible/roles/cloudstack-management/tasks/main.yml

Now we have some new elements to deal with. The Ansible template module uses Jinja2 based templating. As we’re doing a simplified example here, the Jinja template for the cloudstack.repo won’t have any variables in it, so it would simply look like this:

  1. [cloudstack]
  2. name=cloudstack
  3. baseurl=http://download.cloudstack.org/rhel/4.2/
  4. enabled=1
  5. gpgcheck=0

This is saved in /etc/ansible/roles/cloudstack-manager/templates/cloudstack.repo.j2

That gives us the packages installed, we need to set up the database. To do this I’ve created a separate task called setupdb.yml

  1. - name: cloudstack-setup-databases
  2. command: /usr/bin/cloudstack-setup-databases cloud:{{mysql\_cloud\_password }}@localhost –deploy-as=root:{{mysql\_root\_password }}
  4. - name: Setup CloudStack manager
  5. command: /usr/bin/cloudstack-setup-management

Save this as: /etc/ansible/roles/cloudstack-management/tasks/setupdb.yml

As there isn’t (as yet) a CloudStack module, Ansible doesn’t inherently know whether or not the databases have already been provisioned, therefore this step is not currently idempotent and will overwrite any previously provisioned databases.

There are some more variables here for us to declare later.

System VM Templates:

Finally we would want to seed the system VM templates into the secondary storage. The playbook for this would look as follows:

  1. - name: Ensure secondary storage mount exists
  2.   file: path={{ tmp\_nfs\_path }} state=directory
  5. - name: Ensure  NFS storage is mounted
  6.   mount: name={{ tmp\_nfs\_path }} src={{ sec\_nfs\_ip }}:{{sec\_nfs\_path }} fstype=nfs state=mounted opts=nolock
  9. - name: Seed secondary storage
  10.   command:
  11. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2 -h kvm -F
  13.   command:
  14. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2 -h xenserver -F
  16.   command:
  17. /usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt -m {{ tmp\_nfs\_path }} -u http://download.cloud.com/templates/4.2/systemvmtemplate-4.2-vh7.ov -h vmware -F

Save this as /etc/ansible/roles/cloudstack-manager/tasks/seedstorage.yml

Again, there isn’t a CloudStack module so Ansible will always run this even if the secondary storage already has the templates in it.

Bringing it all together

Ansible can use playbooks which run other playbooks, this allows us to group these playbooks together and declare variables across all of the individual playbooks. So in the Ansible playbook directory create a file called deploy-cloudstack.yml, which would look like this:

  1. -hosts: acs-manager
  3.  vars:
  5.     mysql\_root\_password: Cl0ud5tack
  6.     mysql\_cloud\_password: Cl0ud5tack
  7.     tmp\_nfs\_path: /mnt/secondary
  8.     sec\_nfs\_ip: IP\_OF\_YOUR\_SECONDARY\_STORAGE
  9.     sec\_nfs\_path: PATH\_TO\_YOUR\_SECONDARY\_STORAGE\_MOUNT
  12.  roles:
  14.    – mysql
  15.    – cloudstack-manager
  17.  tasks:
  19.    – include: /etc/ansible/roles/cloudstack-manager/tasks/setupdb.yml
  20.    – include: /etc/ansible/roles/cloudstack-manager/tasks/seedstorage.yml

Save this as /etc/ansible/deploy-cloudstack.yml inserting the IP address and path for your secondary storage and changing the passwords if you wish to.

To run this go to the Ansible directory (cd /etc/ansible ) and run:

  1. # ansible-playbook deploy-cloudstack.yml -k

‘-k’ tells Ansible to ask you for the root password to connect to the remote host.

Now log in to the CloudStack UI on the new management server.

How is this example different from a production deployment?

In a production deployment, the Ansible playbooks would configure multiple management servers connected to master/slave replicating MySQL databases along with any other infrastructure components required and deploy and configure the hypervisor hosts. We would also have a dedicated file describing the hosts in the environment and a dedicated file containing variables which describe the environment.

The advantage of using a configuration management tool such as Ansible is that we can specify components like the MySQL database VIP once and use it multiple times when configuring the MySQL server itself and other components which need to use that information.

Acknowledgements

Thanks to Shanker Balan for introducing me to Ansible and a load of handy hints along the way.