Packer is an image creation tool, its Created by the guys over at Hashicorp and its here to make our lives easier when deploying images. Just recently I’ve been doing a lot of work on Amazon Web Services and needed a way to have AMI images quickly and neatly created so I could have them ready for use in our infrastructure as code tools. Packer is a cli tool and is relatively simple to use.
Installation
Packer is a available here for most popular architectures and is not a complex product to install, in this article as I use Linux most frequently we will show the installation of packer for Linux, but it is similar on almost every platform.
To download packer you simply need to download the zip file from the distribution site, I did that like this:
export VER="1.4.3"
wget https://releases.hashicorp.com/packer/${VER}/packer_${VER}_linux_amd64.zip
unzip packer_${VER}_linux_amd64.zip -d packer
sudo mv packer /usr/local/bin/
Once installed you can confirm it is there by calling packer and you should get output similar to the following:
$ packer
Usage: packer [--version] [--help] <command> [<args>]
Available commands are:
build build image(s) from template
console creates a console for testing variable interpolation
fix fixes templates from old versions of packer
inspect see components of a template
validate check that a template is valid
version Prints the Packer versionAfter this packer is ready to use. Packer using builders to generate images that you can deploy. There are a number of templates you can use and a list is provided on the packer site here. A common builder to us is the amazon-ebs builder which we will use for the purposes of this article, this will allow us to create Amazon Machine Images, it does this by launching an E2 instance and then creating a golden image from that instance from which other machines can be created. It then drops that in your account for you to manage. A first template configuration may look like this:
{
"variables": {
"aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",
"aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}"
},
"builders": [
{
"type": "amazon-ebs",
"access_key": "{{user `aws_access_key`}}",
"secret_key": "{{user `aws_secret_key`}}",
"region": "eu-west-1",
"source_ami": "ami-06358f49b5839867c",
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"ami_name": "packer_article {{timestamp}}"
}
],
"provisioners": [
{
"type": "shell",
"script": "demo-script"
}
]
}This is perfectly valid but its not generally a good idea to pass in your access and secret keys in this way, this is merely an example to show how you can pass in variables to the builders, the credentials should simply be set in your aws config files as normal and this would leave us with a template which looks like this:
{
"builders": [
{
"type": "amazon-ebs",
"region": "eu-west-1",
"source_ami": "ami-06358f49b5839867c",
"instance_type": "t2.micro",
"ssh_username": "ubuntu",
"ami_name": "packer_article {{timestamp}}"
}
],
"provisioners": [
{
"type": "shell",
"script": "configure-machine.sh"
}
]
}The basic parts of a template are:
- variables – define any variables here.
- builders – this is where the details for the AMI image go.
- provisioners – this are is where you can list various provisioners to configure the AMI.
Supported provisioners are:
- Ansible
- Chef
- Salt
- Shell
- Powershell
- Windows cmd
- File – this copies a local file to an VM image.
We are using a simple shell provisioner here which will launch a basic script to install nginx. This script looks like so:
#!/bin/bash
sudo yum -y update
sudo yum install -y nginxNow you have your template defined the next step is to build the template.
packer validate
packer build -color=false build.json 2>&1 | tee output.txtDuring this process you will get a bunch of output and above I have piped this to a file (which is why I used the color=false option or you will get a lot of ansi colour codes). The output should look something like this:
amazon-ebs: Prevalidating AMI Name: packer_article 1567873619
amazon-ebs: Found Image ID: ami-06358f49b5839867c
amazon-ebs: Creating temporary keypair: packer_5d73da53-1797-d184-50ef-190d3acf3f79
amazon-ebs: Creating temporary security group for this instance: packer_5d73da78-cc5c-3179-19bb-0164e574d474
amazon-ebs: Authorizing access to port 22 from [0.0.0.0/0] in the temporary security groups...
amazon-ebs: Launching a source AWS instance...
amazon-ebs: Adding tags to source instance
amazon-ebs: Adding tag: "Name": "Packer Builder"
amazon-ebs: Instance ID: i-09a8c7a2ac70cc31c
amazon-ebs: Waiting for instance (i-09a8c7a2ac70cc31c) to become ready...
amazon-ebs: Using ssh communicator to connect: 18.203.99.110
amazon-ebs: Waiting for SSH to become available...
amazon-ebs: Connected to SSH!
amazon-ebs: Provisioning with shell script: configure_machine.sh
amazon-ebs: Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
amazon-ebs: Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease
amazon-ebs: Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
amazon-ebs: Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
amazon-ebs: Get:5 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [8570 kB]
amazon-ebs: Get:6 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [497 kB]
amazon-ebs: Get:7 http://archive.ubuntu.com/ubuntu bionic/universe Translation-en [4941 kB]
amazon-ebs: Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [169 kB]
amazon-ebs: Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [6296 B]
amazon-ebs: Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [2776 B]
amazon-ebs: Get:11 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [604 kB]
amazon-ebs: Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [201 kB]
amazon-ebs: Get:13 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4688 B]
amazon-ebs: Get:14 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [2356 B]
amazon-ebs: Get:15 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [151 kB]
amazon-ebs: Get:16 http://archive.ubuntu.com/ubuntu bionic/multiverse Translation-en [108 kB]
amazon-ebs: Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [722 kB]
amazon-ebs: Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [262 kB]
amazon-ebs: Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [13.1 kB]
amazon-ebs: Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [4448 B]
amazon-ebs: Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1003 kB]
amazon-ebs: Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [308 kB]
amazon-ebs: Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [7308 B]
amazon-ebs: Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [3836 B]
amazon-ebs: Get:25 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [2512 B]
amazon-ebs: Get:26 http://archive.ubuntu.com/ubuntu bionic-backports/main Translation-en [1644 B]
amazon-ebs: Get:27 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4000 B]
amazon-ebs: Get:28 http://archive.ubuntu.com/ubuntu bionic-backports/universe Translation-en [1856 B]
amazon-ebs: Fetched 17.8 MB in 7s (2603 kB/s)
amazon-ebs: Reading package lists...
amazon-ebs: Reading package lists...
amazon-ebs: Building dependency tree...
amazon-ebs: Reading state information...
amazon-ebs: The following additional packages will be installed:
amazon-ebs: libnginx-mod-http-echo nginx-common nginx-light
amazon-ebs: Suggested packages:
amazon-ebs: fcgiwrap nginx-doc ssl-cert
amazon-ebs: The following NEW packages will be installed:
amazon-ebs: libnginx-mod-http-echo nginx nginx-common nginx-light
amazon-ebs: 0 upgraded, 4 newly installed, 0 to remove and 14 not upgraded.
amazon-ebs: Need to get 452 kB of archives.
amazon-ebs: After this operation, 1554 kB of additional disk space will be used.
amazon-ebs: Get:1 http://security.ubuntu.com/ubuntu bionic-security/main amd64 nginx-common all 1.14.0-0ubuntu1.6 [37.3 kB]
amazon-ebs: Get:2 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 libnginx-mod-http-echo amd64 1.14.0-0ubuntu1.6 [21.2 kB]
amazon-ebs: Get:3 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 nginx-light amd64 1.14.0-0ubuntu1.6 [390 kB]
amazon-ebs: Get:4 http://security.ubuntu.com/ubuntu bionic-security/main amd64 nginx all 1.14.0-0ubuntu1.6 [3596 B]
amazon-ebs: debconf: unable to initialize frontend: Dialog
amazon-ebs: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
amazon-ebs: debconf: falling back to frontend: Readline
amazon-ebs: debconf: unable to initialize frontend: Readline
amazon-ebs: debconf: (This frontend requires a controlling tty.)
amazon-ebs: debconf: falling back to frontend: Teletype
amazon-ebs: dpkg-preconfigure: unable to re-open stdin:
amazon-ebs: Fetched 452 kB in 0s (2310 kB/s)
amazon-ebs: Selecting previously unselected package nginx-common.
amazon-ebs: (Reading database ... 56638 files and directories currently installed.)
amazon-ebs: Preparing to unpack .../nginx-common_1.14.0-0ubuntu1.6_all.deb ...
amazon-ebs: Unpacking nginx-common (1.14.0-0ubuntu1.6) ...
amazon-ebs: Selecting previously unselected package libnginx-mod-http-echo.
amazon-ebs: Preparing to unpack .../libnginx-mod-http-echo_1.14.0-0ubuntu1.6_amd64.deb ...
amazon-ebs: Unpacking libnginx-mod-http-echo (1.14.0-0ubuntu1.6) ...
amazon-ebs: Selecting previously unselected package nginx-light.
amazon-ebs: Preparing to unpack .../nginx-light_1.14.0-0ubuntu1.6_amd64.deb ...
amazon-ebs: Unpacking nginx-light (1.14.0-0ubuntu1.6) ...
amazon-ebs: Selecting previously unselected package nginx.
amazon-ebs: Preparing to unpack .../nginx_1.14.0-0ubuntu1.6_all.deb ...
amazon-ebs: Unpacking nginx (1.14.0-0ubuntu1.6) ...
amazon-ebs: Processing triggers for ufw (0.36-0ubuntu0.18.04.1) ...
amazon-ebs: Processing triggers for ureadahead (0.100.0-21) ...
amazon-ebs: Setting up nginx-common (1.14.0-0ubuntu1.6) ...
amazon-ebs: debconf: unable to initialize frontend: Dialog
amazon-ebs: debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
amazon-ebs: debconf: falling back to frontend: Readline
amazon-ebs: Created symlink /etc/systemd/system/multi-user.target.wants/nginx.service → /lib/systemd/system/nginx.service.
amazon-ebs: Processing triggers for systemd (237-3ubuntu10.24) ...
amazon-ebs: Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
amazon-ebs: Setting up libnginx-mod-http-echo (1.14.0-0ubuntu1.6) ...
amazon-ebs: Setting up nginx-light (1.14.0-0ubuntu1.6) ...
amazon-ebs: Setting up nginx (1.14.0-0ubuntu1.6) ...
amazon-ebs: Processing triggers for ureadahead (0.100.0-21) ...
amazon-ebs: Processing triggers for ufw (0.36-0ubuntu0.18.04.1) ...
amazon-ebs: Stopping the source instance...
amazon-ebs: Stopping instance
amazon-ebs: Waiting for the instance to stop...
amazon-ebs: Creating AMI packer_article 1567873619 from instance i-09a8c7a2ac70cc31c
amazon-ebs: AMI: ami-09511096fd8530040
amazon-ebs: Waiting for AMI to become ready...
amazon-ebs: Terminating the source AWS instance...
amazon-ebs: Cleaning up any extra volumes...
amazon-ebs: No volumes to clean up, skipping
amazon-ebs: Deleting temporary security group...
amazon-ebs: Deleting temporary keypair...
Build 'amazon-ebs' finished.
==> Builds finished. The artifacts of successful builds are:
--> amazon-ebs: AMIs were created:
eu-west-1: ami-09511096fd8530040You can see that packer has now created a standard AMI and place it in our account, this is where packers management ends of our AMIS, it is up to us to manage them once built.
You can check out your image with the AWS console to make sure its there and get more information:
aws ec2 describe-images --image-ids ami-09511096fd8530040 --output=jsonAnd you will get a fair amount of information back
{
"Images": [
{
"Architecture": "x86_64",
"CreationDate": "2019-09-07T16:29:32.000Z",
"ImageId": "ami-09511096fd8530040",
"ImageLocation": "<myaccountId>/packer_article 1567873619",
"ImageType": "machine",
"Public": false,
"OwnerId": "<myaccountId>",
"State": "available",
"BlockDeviceMappings": [
{
"DeviceName": "/dev/sda1",
"Ebs": {
"DeleteOnTermination": true,
"SnapshotId": "snap-000b743eca850ac19",
"VolumeSize": 8,
"VolumeType": "gp2",
"Encrypted": false
}
},
{
"DeviceName": "/dev/sdb",
"VirtualName": "ephemeral0"
},
{
"DeviceName": "/dev/sdc",
"VirtualName": "ephemeral1"
}
],
"EnaSupport": true,
"Hypervisor": "xen",
"Name": "packer_article 1567873619",
"RootDeviceName": "/dev/sda1",
"RootDeviceType": "ebs",
"SriovNetSupport": "simple",
"VirtualizationType": "hvm"
}
]
}There are a lot of configuration options with packer, far too many to go into in this short article but this should get most started. For those not wanting to type all this out you can find the code here