Infrastructure as Code (IAC) Cookbook

Table of Contents

Infrastructure as Code (IAC) Cookbook

Credits

About the Authors

About the Reviewer

www.PacktPub.com

eBooks, discount offers, and more

Why Subscribe?

Customer Feedback

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Getting ready

How to do it…

How it works…

There's more…

See also

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

1. Vagrant Development Environments

Introduction

Adding an Ubuntu Xenial (16.04 LTS) Vagrant box

Getting ready

How to do it…

How it works…

There's more…

Using a disposable Ubuntu Xenial (16.04) in seconds

Getting ready

How to do it…

How it works…

Enabling VirtualBox Guest Additions in Vagrant

Getting ready

How to do it…

How it works…

There's more…

Using a disposable CentOS 7.x with VMware in seconds

Getting ready

How to do it…

How it works…

There's more…

See also

Extending the VMware VM capabilities

Getting ready

How to do it…

How it works…

There's more…

Enabling multiprovider Vagrant environments

Getting ready

How to do it…

How it works…

Customizing a Vagrant VM

Getting ready

How to do it…

Set the hostname

Disable new box version check at startup

Use a specific box version

Display an informational message to the user

Specify a minimum Vagrant version

Using Docker with Vagrant

Getting ready

How to do it…

Using NGINX Docker container through Vagrant

Exposing Docker ports in Vagrant

Sharing folders with Docker through Vagrant

There's more…

Using Docker in Vagrant for a Ghost blog behind NGINX

Getting ready

How to do it…

There's more…

A Docker Compose equivalent

Using Vagrant remotely with AWS EC2 and Docker

Getting ready

How to do it…

Simulating dynamic multiple host networking

Getting ready

How to do it…

There's more…

Speed up deployments with linked clones

Using named NAT networks

Simulating a networked three-tier architecture app with Vagrant

Getting ready

How to do it…

Tier 3 – the database

Tier 2: the application servers

The Node.js application

Tier 1: the NGINX reverse proxy

Showing your work on the LAN while working with Laravel

Getting ready

How to do it…

A sample NGINX configuration for Laravel

Simple shell provisioning

Enable provisioning

Shared folder

Public LAN Networking

There's more…

Sharing access to your Vagrant environment with the world

Getting ready

How to do it…

Provisioning

Starting Ghost engine

Sharing access

HTTP

SSH

Simulating Chef upgrades using Vagrant

Getting ready

How to do it…

Vagrant Omnibus Chef plugin

A sample Chef recipe

Vagrant and Chef integration

Testing the Chef version update

There's more…

Controlling default Vagrant VMs

Berkshelf and Vagrant

Testing with Test Kitchen

Using Ansible with Vagrant to create a Docker host

Getting ready

How to do it…

A simple Ansible Docker playbook for Vagrant

Apply Ansible from Vagrant

There's more…

Using Docker containers on CoreOS with Vagrant

Getting ready

How to do it…

There's more…

2. Provisioning IaaS with Terraform

Introduction

Configuring the Terraform AWS provider

Getting ready

How to do it…

How it works…

There's more…

Creating and using an SSH key pair to use on AWS

Getting ready

How to do it…

How it works…

There's more…

Using AWS security groups with Terraform

Getting ready

How to do it…

There's more…

Creating an Ubuntu EC2 instance with Terraform

Getting ready

How to do it…

Scaling the number of instances

There's more…

Generating meaningful outputs with Terraform

Getting ready

How to do it…

There's more…

Using contextual defaults with Terraform

Getting ready

How to do it…

There's more…

Managing S3 storage with Terraform

Getting ready

How to do it…

There's more…

Creating private Docker repositories with Terraform

Getting ready

How to do it…

Creating a PostgreSQL RDS database with Terraform

Getting ready

How to do it…

There's more…

Enabling CloudWatch Logs for Docker with Terraform

Getting ready

How to do it…

Amazon CloudWatch Logs Docker logging driver

Managing IAM users with Terraform

Getting ready

How to do it…

An IAM user for S3 access

Testing the restrictions

An IAM user for EC2 in read-only

An application user IAM – CloudWatch Logs

There's more…

3. Going Further with Terraform

Introduction

Handling different environments with Terraform

Getting ready

How to do it…

Keeping the tfstate isolated

Setting the production flag

Provisioning a CentOS 7 EC2 instance with Chef using Terraform

Getting ready

How to do it…

Creating the EC2 instance

Passing connection information

Giving Chef information

How it works…

There's more…

Using data sources, templates, and local execution

Getting ready

How to do it…

Data and templates

The local-exec Terraform provisioner

Apply a configured Ansible

Executing remote commands at bootstrap using Terraform

Getting ready

How to do it…

Using Docker with Terraform

Getting ready

How to do it…

Simulating infrastructure changes using Terraform

Getting ready

How to do it…

Planning

Quickly simulating changes

Targeting for a specific change

Teamwork – sharing Terraform infrastructure state

Getting ready

How to do it…

Sharing with Git

Sharing remotely with S3

Sharing remotely with Consul

Other state sharing options

Maintaining a clean and standardized Terraform code

Getting ready

How to do it…

Syntax validation

Style validation

One Makefile to rule them all

Getting ready

How to do it…

See also

Team workflow example

Getting ready

How to do it…

A simple Git repository

Initial infrastructure code

Terraform code validation

Infrastructure code commit

Make a pull request

Apply the changes

Managing GitHub with Terraform

Getting ready

How to do it…

Configuring GitHub

Adding users to the GitHub organization

Adding GitHub teams

Setting Git repository access rights

External monitoring integration with StatusCake

Getting ready

How to do it…

Creating an automated ping monitoring test

Creating an HTTPS test

4. Automating Complete Infrastructures with Terraform

Introduction

Provisioning a complete CoreOS infrastructure on Digital Ocean with Terraform

Getting ready

How to do it…

Handling the SSH key

Creating the CoreOS cluster members

Adding useful output

Dynamic DNS Integration

Integrating cloud-init

Integrating dynamic StatusCake monitoring

Provisioning a three-tier infrastructure on Google Compute Engine

Getting ready

How to do it…

Generating API credentials for a Google project

Creating Google Compute HTTP instances

Creating a Google Compute Firewall rule

Load balancing Google Compute instances

Creating a Google MySQL database instance

Adding some useful outputs

Provisioning a GitLab CE + CI runners on OpenStack

Getting ready

How to do it…

Configuring the OpenStack provider

Creating a key pair on OpenStack

Creating a security group on OpenStack

Creating block storage volumes on OpenStack

Creating compute instances on OpenStack

Creating an object storage container on OpenStack

Applying

Managing Heroku apps and add-ons using Terraform

Getting ready

How to do it…

Creating a Heroku application with Terraform

Adding Heroku add-ons using Terraform

Using Heroku with Terraform

Creating a scalable Docker Swarm cluster on bare metal with Packet

Getting ready

How to do it…

Creating a Packet project using Terraform

Handling Packet SSH keys using Terraform

Bootstraping a Docker Swarm manager on Packet using Terraform

Bootstraping Docker Swarm nodes on Packet using Terraform

Using the Docker Swarm cluster

5. Provisioning the Last Mile with Cloud-Init

Introduction

Using cloud-init on AWS, Digital Ocean, or OpenStack

Getting ready

How to do it…

Using cloud-init on Amazon Web Services

Using cloud-init on Digital Ocean

Using cloud-init on OpenStack

Combining cloud-init and Terraform for any IaaS

Handling files using cloud-init

Getting ready

How to do it…

Configuring the server's time zone using cloud-init

Getting ready

How to do it…

Managing users, keys, and credentials using cloud-init

Getting ready

How to do it…

Managing repositories and packages using cloud-init

Getting ready

How to do it…

Running commands during boot using cloud-init

Getting ready

How to do it…

Configuring CoreOS using cloud-init

Getting ready

How to do it…

Configuring etcd using cloud-init

Configuring fleet using cloud-init

Configuring the update strategy using cloud-init

Configuring locksmith using cloud-init

Configuring systemd units using cloud-init

Configuring flannel using cloud-init

Deploying Chef Client from start to finish using cloud-init

Getting ready

How to do it…

Deploying the Chef omnibus installer using cloud-init

Configuring Chef against a Chef Server organization using cloud-init

Applying a Chef cookbook at bootstrap using cloud-init

Deploying a remote Docker server using cloud-init

Getting ready

How to do it...

Setting the timezone on CoreOS using cloud-init

Enabling Docker TCP socket for network access

There's more...

See also

6. Fundamentals of Managing Servers with Chef and Puppet

Introduction

Getting started (notions and tools)

Running Chef

Chef plugins

Chef organizations

Chef nodes

Chef environments

Chef roles

Chef resources

Chef recipes

Chef cookbooks

Chef run list

There's more…

Installing the Chef Development kit and Puppet Collections

Getting ready

How to do it…

Chef DK contents

How it works…

There's more…

See also

Creating a free hosted server Chef account and a Puppet server

Getting ready

How to do it…

There's more…

Automatically bootstrapping a Chef client and a Puppet agent

Getting ready

How to do it…

There's more…

Installing packages

Getting ready

How to do it…

Generating an empty Apache cookbook

Uploading the cookbook

Applying the cookbook

Creating a MariaDB cookbook

Creating a PHP cookbook

There's more…

See also

Managing services

Getting ready

How to do it…

Enabling and starting Apache service

Enabling and starting the MariaDB service

There's more…

See also

Managing files, directories, and templates

Getting ready

How to do it…

Managing a simple static file

Managing dynamic files and directories from a template

There's more…

See also

Handling dependencies

Getting ready

How to do it…

There's more…

See also

More dynamic code using notifications

Getting ready

How to do it…

There's more…

See also

Centrally sharing data using a Chef data bag and Hiera with Puppet

Getting ready

How to do it…

There's more…

See also

Creating functional roles

Getting ready

How to do it…

There's more…

See also

Managing external Chef cookbooks and Puppet modules

Getting ready

How to do it…

Using the official MySQL cookbook and its dependencies with Berkshelf

Including dependencies in a role

Uploading cookbook dependencies using Berkshelf

Testing MySQL deployment

There's more…

See also

7. Testing and Writing Better Infrastructure Code with Chef and Puppet

Introduction

Linting Chef code with Foodcritic and Puppet code with puppet-lint

Getting ready

How to do it…

Cookstyle

Foodcritic

There's more…

Puppet coding style

Documentation

See also

Unit testing with ChefSpec and rspec-puppet

Getting ready

How to do it…

The Spec Helper

Testing a successful Chef run context

Testing a package installation

Testing services status

Testing another recipe from the same cookbook

Testing directory creation

Testing file creation

Testing templates creation

Stubbing data bags for searches

Testing recipes inclusion

Intercepting errors in tests

There's more…

See also

Testing infrastructure with Test Kitchen for Chef and Beaker for Puppet

Getting ready

How to do it…

Configuring Test Kitchen

Testing with Test Kitchen

How it works…

There's more…

See also

Integration testing with ServerSpec

Getting ready

How to do it…

Creating a ServerSpec helper script

Testing a package installation

Testing for service status

Testing for listening ports

Testing for files existence and content

Testing for repository existence

There's more…

See also

8. Maintaining Systems Using Chef and Puppet

Introduction

Maintaining consistent systems using scheduled convergence

Getting ready

How to do it…

Using the Chef client as a daemon

Tweaking the convergence interval time

Running the Chef client as a cron

Tweaking the Chef cron job

There's more…

See also

Creating environments

Getting ready

How to do it…

Creating a production environment

Setting an environment to a node

Bootstrapping a node with an environment

Fixing cookbook versions for an environment

Overriding attributes for an environment

Accessing the environment from a recipe

There's more...

Manual environment creation in the Puppet server

Node environment selection

Getting the environment from manifests

The dynamic way – r10k

See also

Using Chef encrypted data bags and Hiera-eyaml with Puppet

Getting ready

How to do it…

Encrypting data bags with a shared secret

Accessing an encrypted data bag in the CLI

Using an encrypted data bag from a recipe

There's more…

Preparing the Puppet server

Preparing the workstation

Securing the MySQL root password

See also

Using Chef Vault encryption

Getting ready

How to do it…

Accessing the encrypted vault from a cookbook

See also

Accessing and manipulating system information with Ohai

Getting ready

How to do it…

Accessing Ohai information from a Chef recipe

There's more…

See also

Automating application deployment (a WordPress example)

Getting ready

How to do it…

Including dependencies

Creating the application's database

Deploying an application from git or GitHub

There's more…

See also

Using a TDD workflow

Getting ready

How to do it…

Infrastructure TDD – writing tests first

Deploying Docker with Chef

Linting the code

Supporting another platform

Team working using Chef and git

Deploying to staging

Deploying to production

There's more…

See also

Planning for the worse – train to rebuild working systems

Getting ready

How to do it…

Multi-machine recovery

There's more…

9. Working with Docker

Introduction

Docker usage overview

Getting ready

How to do it…

Running Bash in an Ubuntu 16.04 container

Running Nginx in a container

Sharing data with a container

Building a container with utilities

Using a private registry

See also

Choosing the right Docker base image

Getting ready

How to do it…

Starting from an Ubuntu image

Starting from a CentOS image

Starting from a Red Hat Enterprise Linux (RHEL) image

Starting from a Fedora image

Starting from an Alpine Linux image

Starting from a Debian image

Linux distributions container image size table

Starting from a Node JS image

Starting from a Golang image

Starting from a Ruby image

Starting from a Python image

Starting from a Java image

Starting from a PHP image

See also

Optimizing the Docker image size

Getting ready

How to do it…

How it works…

Versioning Docker images with tags

Getting ready

How to do it…

Deploying a Ruby-on-Rails web application in Docker

Getting ready

How to do it…

Building and using Golang applications with Docker

Getting ready

How to do it…

Using the golang Docker image to cross-compile a Go program

Using the golang Docker image to build and ship a Go program

Using the scratch Docker image

Using the Alpine Linux alternative for a Go program

Networking with Docker

Getting ready

How to do it…

Docker networks

Connecting multiple networks for one container

Creating more dynamic containers

Getting ready

How to do it…

Auto-configuring dynamic containers