Cloud/Manage/Ansible (original) (raw)

Provisioning CentOS images on EC2 with Ansible

What I'm going to discuss here is provisioning some CentOS images on EC2 instances with Ansible and well, do that from a CentOS machine of course.

First off, you don't need the AWS tools installed on your management system, and that saves you from having to install Java as well. We need the following components on our CentOS management system:

You've set up your AWS account, and you've obtained authorization secrets to interact with EC2. For everything we do from here onwards, we need the following variables in our shell's environment:

export EC2_ACCESS_KEY="xxxxxxxxxxxxxxxxxxxx" export EC2_SECRET_KEY="xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" export EC2_URL=https://ec2.amazonaws.com export S3_URL=https://s3.amazonaws.com:443 export AWS_ACCESS_KEY_ID=${EC2_ACCESS_KEY} export AWS_SECRET_ACCESS_KEY=${EC2_SECRET_KEY}

1. Ansible inventory

Ansible uses an inventory in which I describe the machines I want it to speak to, the groups they belong to and specific variables I want those machines to use. The inventory file defaults to /etc/ansible/hosts, but I can override that by setting $ANSIBLE_HOSTS to a different path. An inventory file can be as short as this, and believe it or not, this is actually the inventory we're going to give to Ansible in order to launch EC2 instances:

[local] 127.0.0.1

On the other hand we need to provision EC2 instances running on the other side of the world (for me at least). How do we do that? How do we know their hostnames?

There's an aws_ec2.py inventory script for Ansible which enumerates the EC2 instances we can access. If I launch this program, I see the following JSON output because I already have an instance running. (Compare the instance ID and the public hostname to what we saw earlier.)

{ "i-df75cea0": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ], "key_jp1": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ], "security_group_default": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ], "type_m1_small": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ], "us-east-1": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ], "us-east-1a": [ "ec2-54-242-141-105.compute-1.amazonaws.com" ] }

Even though we have a single machine only, it shows up in different groups. These groups will allow us to target specific groups of instances when we use Ansible to provision them. (Note: the ec2.py program caches its output in configurable paths, so it may take a minute until the list is refreshed.) If I invoke the inventory program with a specific host, I get a list of variables particular to that instance: (I'm omitting lots of output for brevity)

ec2.py --host ec2-54-242-141-105.compute-1.amazonaws.com { "ec2_architecture": "x86_64", "ec2_dns_name": "ec2-54-242-141-105.compute-1.amazonaws.com", "ec2_hypervisor": "xen", "ec2_id": "i-df75cea0", "ec2_image_id": "ami-8a8932e3", "ec2_instance_type": "m1.small", "ec2_ip_address": "54.242.141.105", "ec2_key_name": "jp1", "ec2_launch_time": "2012-11-18T10:57:20.000Z", "ec2_monitored": false, "ec2_placement": "us-east-1a", "ec2_root_device_type": "ebs", "ec2_security_group_ids": "sg-29652b41", "ec2_security_group_names": "default", "ec2_virtualization_type": "paravirtual" }

Using the ec2.py inventory script will allow Ansible to interact with instances on EC2. This happens either by installing the file as an executable /etc/ansible/hosts or by pointing $ANSIBLE_HOSTS to that executable.

Will it "ping"?

$ export ANSIBLE_HOSTS=~/ec2.py $ ansible -u root ec2-54-242-141-105.compute-1.amazonaws.com -m ping ec2-54-242-141-105.compute-1.amazonaws.com | success >> { "changed": false, "ping": "pong" }

At this point I could leave you to it, and you could successfully use Ansible to install and configure your EC2 instances. But I won't leave you to it: let's do a bit of provisioning.

2. Ansible launches a CentOS instance on EC2

Ansible's ec2 module creates an instance on EC2 and optionally waits for that instance to become ready. (Note: ready doesn't mean booted -- that can take a few minutes.) Upon creating an instance, I specify the SSH keypair I want to use (we created a key called jp1 for that), the image name, and a few other parameters which are described in the module's documentation. One parameter I want to point out is called group. This is a so-called _security group_ which, as far as I've been able to determine, specifies e.g. firewall rules from the AWS point of view. By default only port 22 (SSH) is allowed into my instance, but I'm creating Web servers so I also want (at least) port 80.

To create an EC2 security group, I used the following commands:

$ euca-add-group -d "Web Servers" webs-a $ euca-authorize -P tcp -p 80-80 -s 0.0.0.0/0 webs-a $ euca-authorize -P tcp -p 22-22 -s 0.0.0.0/0 webs-a

The specified security group later on shows up as a group in the Ansible inventory (ec2.py).

The variable ec2, obtained by registering the result of the ec2 module, contains the following values, which I use in the e-mail I fire off to the admins:

{ "instances" : [ { "public_ip" : "107.22.159.172", "id" : "i-3d1ba042" } ], "changed" : true }

Let me run Ansible on this playbook. Note that I use the simple inventory containing just localhost, because these modules run on my Ansible management machine and not remotely.

$ ansible-playbook newinstances.yml What is the shortname of this host to be?: : web31

PLAY [127.0.0.1] *********************

TASK: [Launch new EC2 instance] ********************* changed: [127.0.0.1]

TASK: [Send e-mail to admins] ********************* ok: [127.0.0.1]

PLAY RECAP ********************* 127.0.0.1 : ok=2 changed=1 unreachable=0 failed=0

To recapitulate: so far Ansible used modules locally (i.e. on our management machine) to remotely create a CentOS instance on EC2 and to send the e-mail. I could now use some of the euca- tools to look and see what is happening.

Also: have a bit of patience: it can take a few minutes for the EC2 instance to actually come alive. And also: I have mail:

Date: Sun, 18 Nov 2012 12:51:07 +0100 (CET) From: Ansible@c6.ww.example.com Subject: EC2 instance i-3d1ba042

EC2 instance i-3d1ba042 created on 107.22.159.172

3. Ansible provisions CentOS instances

After a couple minutes of patience I use Ansible to actually provision the instance I just brought up. To illustrate, I'll just install an Apache Web server and a template, so nothing special.

The hosts Ansible should act upon are specified by a group name as obtained by ec2.py, and I'm connecting as root because that's the user for which our jp1 SSH key has been injected into by EC2. After the playbook has run, I can use a Web browser to connect to the ec2-*.compute-1.amazonaws.com hostname or to its public IP address.

The instance is still waiting for us to do something with it. We tell Ansible to use a different inventory this time, i.e. the EC2 inventory, and launch the configuration playbook:

ANSIBLE_HOSTS=~/ec2.py ansible-playbook apache.yml

PLAY [security_group_webs-a] *********************

TASK: [Install | Apache] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com]

TASK: [Machine | Launch Apache service] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com]

TASK: [Machine | Disable firewall (fixme)] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com]

TASK: [Web | Install Web templates] ********************* changed: [ec2-107-22-159-172.compute-1.amazonaws.com]

PLAY RECAP ********************* ec2-107-22-159-172.compute-1.amazonaws.com : ok=4 changed=4 unreachable=0 failed=0

Provisioning is complete, and the services are running.

4. Lessons

CentOS on EC2 is cool, and using Ansible to provision CentOS instances is also. There are many ways to accomplish this I suppose, and you might do it differently.