Terraform and Ansible Tutorial: Integrating Terraform-Managed Instances With Ansible Control Nodes
Infrastructure automation is really important nowadays. We tend to focus on software deployment processes the most, but infrastructure deployment strategy should be equally important. Infrastructure automation not only helps with disaster recovery but also makes the testing and development process easier. With good infrastructure automation in place, test environments can be deployed quickly and easily. In this post, you’ll learn how to integrate the two most common infrastructures, Terraform and Ansible, as code tools to create seamless infrastructure automation.
Infrastructure as Code
Let’s start with a brief explanation of the infrastructure-as-code (IaC) concept. As the name suggests, IaC is a process of managing infrastructure by describing it in the code. What this means is that, for example, instead of using GUI and clicking buttons to deploy a virtual machine, you describe the properties of the virtual machine in code (or use already written code) and execute it with an IaC tool. Sounds like more work than using a simple button? Well, at first, yes. It creates a bit more work initially, but the benefits in the long term are indisputable.
Imagine that you need to deploy ten virtual machines, three storage accounts, a load balancer, and a virtual network. Then you need to configure routing on that network and format disks. “Just one click” in the UI becomes hours of clicking and filling in forms in the UI. Now, imagine that something went wrong near the end and you need to start from scratch. With the IaC model, you can write code to describe all the infrastructure components and then leave the deployment to Terraform and Ansible. And if something breaks near the end, you can just change a few lines of code to fix the issue and run Terraform and Ansible again.
Another advantage is the ability to use a source code versioning system like Git to store your code. And that makes it possible to easily see all the changes in the infrastructure and make it available for all team members.
Terraform and Ansible
So, why do we need both Terraform and Ansible? Well, they each serve a different purpose and complement each other. Terraform is designed to provision different infrastructure components. Ansible is a configuration-management and application-deployment tool. It means that you’ll use Terraform first to create, for example, a virtual machine and then use Ansible to install necessary applications on that machine. But by default, these two are separate tools. In order to make them work together, you need to integrate Terraform-managed nodes with Ansible control nodes. Let’s see how to do that.
Ansible Control Node vs. Terraform-Managed Instances
When you work with Ansible you execute playbooks from your machine (or any other machine where Ansible is installed). Playbooks are the manifest files that describe what needs to be done on the desired machine. For example, add a new user, install the NGINX server, and configure its root folder. Then, you also need to create an inventory, which is a file with the list of hosts that Ansible should configure. Ansible reads the inventory file and connects to the desired machines via SSH in order to install and configure whatever was specified in the playbook.
Terraform works similarly. You install Terraform on your machine and write Terraform manifests. When you run the terraform apply command, it creates the infrastructure somewhere in the cloud according to your manifest. Since Terraform is mainly used to create new resources, its manifest files define, for example, the type of virtual machines and how many of them should be deployed. After Terraform is done provisioning, it saves the details of the provisioned resources (for example, IP addresses of created virtual machines) back into the state file.
So, in order to make these two work together, we need to somehow pass the information about infrastructure created by Terraform into Ansible, or vice versa. More specifically, we have two options: either use Terraform’s output as input for Ansible’s inventory or instruct Terraform to execute Ansible.
Execute Ansible from Terraform
Let’s start with the second option. For this to work, we can use Terraform’s local-exec and remote-exec provisioners. They allow you to execute any command from Terraform, either on the machine from where Terraform is run or from the instance provisioned by Terraform. What we will do here is to make Terraform create machines and then run the Ansible playbook against these instances. In order to do so, you need to add the following code to your instances provisioning section in Terraform:
provisioner "local-exec" {command = "ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook -u {var.user} -i '${self.ipv4_address},' --private-key ${var.ssh_private_key} playbook.yml"}
The key component here is the ${self.ipv4_address} variable. After provisioning the machine, Terraform knows its IP address. And we need to pass an IP address for Ansible. Therefore, we are using the built-in Terraform variable as input for Ansible.
Ansible Provisioner for Terraform
Following the same idea, you can also use this unofficial Ansible Provisioner for Terraform. It basically does the same as we just explained: executes Ansible from Terraform but in a more native way for Terraform. However, keep in mind that it does not cover all the use cases.
Using Terraform Output as Ansible Inventory
Another option is to run Terraform and Ansible separately but import the data from one to another. Terraform saves all the information about provisioned resources into a Terraform state file. We can find the IP addresses of Terraform-provisioned instances there and import them into the Ansible inventory file. By default, Terraform saves the state file locally as a JSON file, which makes it easy to parse it and extract the necessary IP addresses. You can also use projects like Terraform Inventory to do it automatically:
$ terraform-inventory -inventory terraform.tfstate
[all]
10.10.1.2
10.10.1.3
[vm.0]
10.10.1.2
[vm.1]
10.10.1.3
With the inventory file ready, you can simply execute the Ansible playbook to configure the nodes.
Managing Infrastructure at Scale
As mentioned at the beginning of this post, IaC may seem like an unnecessary complication of the simple task of deploying a virtual machine. But then again, while deploying one virtual machine may be faster from a web UI, deploying and configuring dozens of them definitely won’t be. The bigger the infrastructure, the more benefits IaC brings. Terraform and Ansible can drastically decrease the precious engineering time spent on adding new machines to the cluster. But without Terraform and Ansible integration, these could also be pretty time-consuming tasks.
Infrastructure management nowadays can be treated the same way as software provisioning. This means using software version control tools and CI/CD pipelines. You certainly don’t want to run an Ansible playbook by mistake on the wrong set of machines. Well-designed CI/CD pipelines will make sure that this won’t happen.
Summary
Infrastructure provisioning automation helps on many fronts. It lets you keep the history of the changes in the infrastructure (if using Git to store manifests). In the long term, it saves a lot of time (creating a completely new cluster consisting of many components can be as easy as copy-pasting a block of code and changing the values of a few variables). And of course, it helps you avoid the situation where a team member deployed some resources via UI months ago and doesn’t remember what configuration was used. If you want to learn more about Terraform, Ansible, and CI/CD, consider this Deploying and Automating Infrastructure at Scale course.
Deploying and Automating Infrastructure at Scale
View Course
