Introduction to Infrastructure Provisioning Using Terraform

Informal introduction to the industry and IaC

IT industry, including organizations of almost every size, is adopting cloud-based services for maintaining its application workloads. One cannot deny the fact that there are lot of operational constraints regarding the underlying infrastructure which becomes a headache for the development teams. That is where IaC has stepped in and allowed you to operate using cloud-based services with greater independence from the associated constraints.

IaC has become very important for provisioning our infrastructure-as-code in an efficient and collaborative manner. However, it may not be ideal for every organization, yet the agility IaC provides makes it ideal for companies looking to speed up the pace of infrastructure provisioning and application deployment.

Now lets talk about some of the features of IaC.

Infracturcture as Code or IaC:

“Infrastructure as code” (IaC) has become a buzz word which plays a key role in DevOps processes like infrastructure provisioning. It is a practice of addressing the scale and complexity of today’s IT infrastructures by allowing us to configure the desired state of infrastructure using code.

According to Microsoft, Infrastructure as Code refers to the governance of infrastructure (networks, virtual machines, load balancers, connection topology) in a descriptive model by leveraging the same versioning as DevOpsteam uses for source code. It is an integral DevOps practice and is used in combination with Continuous Delivery.

Infrastructure as Code refers to the governance of infrastructure (networks, virtual machines, load balancers, connection topology) in a descriptive model by leveraging the same versioning as DevOpsteam uses for source code.

Some of the infrastructure management problems solved by IaC are as:

• Inconsistent configuration
• Out of date documentation
• Configuration change tracking
• Configuration drift detection

Configuration drift normally occurs in data center environments due to inconsistent configuration items, where any modifications to the hardware or software are not recorded and tracked. Since IaC uses configuration files that maintain the desired state of infrastructure.

In simple words, we can say IaC does the following:

1. Uses code for provisioning infrastructure: It allows you to manage operations environment, build systems in the same way you do applications or other code for general release. Here, UI or CLI is not used to say launch a VM or create a VPC, rather, you decribe its desired characteristics in code and tell the tool to do that.
2. Reusability of tested infrastructure:It allows you to resuse the already existing tested code for infrastructure management. You dont need to provision a new infrastructure every time. All this is possible because of state file which is generated by the provision tool.
3. Automates operations: It allows you to use a very simple and quick to learn configuration syntax which is provided by the IaC tool. You no longer need to run hectic commands to launch and configure the system. Thus, automating most of the server side work, reduces the inefficiency and errors.
4. Team collaboration, peer reviews, version control: Since your infrastructure is in code based configuration file, it is very much possible to maintain our code in version control like any other application code base. It results in well structured and tested state file that can be collaborated by the team and at the same time reviewed by peers.

What is Terraform

Terraform or Hashicorp Terraform, is one of the most popular Iac tools used in DevOps chain created by HashiCorp. According to Hashicorp:

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Terraform requires a configuration file that describes the components needed to provision a desired infrastructure. Based upon that configuration file, Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.

Features of Terraform

The important features of terraform are as:

1. Infrastructure as code: As we already discussed that Terraform is an Iac tool, it works on a configuration file. The code that defines the infrastructure can be versioned, forked and rolled back, just like any other code.
2. Execution Plan: Execution plan is generated in planning step of Terraform and the command used for that is “terraform plan”. This helps the operator in checking what all Terraform is going to execute on applying the described configuration. For example, terraform plan might be run before committing a change to version control, to create confidence that it will behave as expected. Once operator is confident enough, he runs terraform apply command to actually make the changes.
3. Resource Graph: This feature of terraform plays an important role in building infrastructure as efficiently as possible. this allows operators to have an insight into the dependencies among the resources described in their infrastructure. Terraform builds a resource graph of all described resources and parallelizes the creation and modification of any non-dependent resources, thus efficient.
4. Hybrid cloud as one-tool-one-language: As terraform is not cloud specific, it supports almost ever provider such as IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Terraform Enterprise, DNSimple, CloudFlare).
5. Identical Infrastructure: Since Terraform provisions infrastructure using a piece of code, creating different environments like production, qa and staging with identical configuration becomes easy.

I suggest to visit https://www.terraform.io/intro/vs/index.html for more features and some of the distinguishing features between terraform and other popularly known Iac tools.

The Nginx Webserver

Let us create Nginx webserver on EC2 instance using Terraform.

I assume that you have installed terraform, if not download the Terraform binary executable for your platform and follow the steps to install. Since, I will be using AWS as provider so ensure that AWS CLI is installed and configured correctly.

Create one project in your desired location and name it whatever you like. cd into the project and follow the following steps:

1. Variable Declaration: Create a file variables.tf where you would declare some important variables as follows:

variables.tf

variables.tf

Variables or Input Variables or Terraform Variable serve as parameters for a Terraform module. Some important terms used in declaring variables:

1. variable: Variables are declared with variable block followed by a unique variable name.
2. description: It gives a brief description of the variable.
3. default: It requires a literal value. If present, the default value will be used if no value is set when calling the module or running Terraform.

Here, you declare three variables for region, access key and secret key. You need to provide your region and credentials for your AWS user against the default key. You can read more about Terraform variables here.

2. Provider: Create another file main.tf and describe the provider, that is AWS in this case.

main.tf

main.tf

In main.tf you use provider block to describe the provider you want to use. The AWS provider offers a flexible means of providing credentials for authentication. There are multiple ways of authenticating and you can learn more about those here.

The method of authenticating used in this example is known as static credentials method wherein region, access_key and secret_key are added in-line in the AWS provider block.

I need you to understand that the above variables are referenced using a special syntax which is known as interpolation syntax. These interpolations are wrapped in ${}, such as ${var.aws_access_key}. Interpolation is powerful and allows you to reference variables, attributes of resources, call functions, etc.

In order to initialize Terraform run terraform init and you can see that Terraform will download the provider plugin for AWS.

//output of Terraform init

3. AWS VPC Resources: Create a file vpc.tf, wherein you would describe the vpc resources.

In this example, AWS VPC Terraform module is used to create the vpc resources. A module is a container for multiple resources that are used together. You can also explore Terraform Module Registry to learn about various modules provided by Terraform. Put following lines of code in your vpc.tf file.

module "vpc" {
source               = "terraform-aws-modules/vpc/aws"
name                 = "vpc-main"
cidr                 = "10.0.0.0/16"
azs                  = ["${var.aws_region}a", "${var.aws_region}b"]
private_subnets      = ["10.0.1.0/24", "10.0.2.0/24"]
public_subnets       = ["10.0.101.0/24", "10.0.102.0/24"]
enable_dns_hostnames = true
enable_dns_support   = true
enable_nat_gateway   = false
enable_vpn_gateway   = falsetags = {
Terraform   = "true"
Environment = "dev"
}
}

Modules are declared using module block. The label immediately after the module keyword is a local name, which the calling module can use to refer to this instance of the module. Within the block body (between { and }) are the arguments for the module.

source: All modules require a source argument, which is a meta-argument defined by Terraform CLI. Its value is either the path to a local directory of the module’s configuration files, or a remote module source that Terraform should download and use. Other arguments mentioned above are self understood and depends upon one’s desired values.

Note: The same source address can be specified in multiple module blocks to create multiple copies of the resources defined within, possibly with different variable values.

Note: Every time a module is added, modified or deleted, terraform init comand should be re-run so that Terraform can adjust the installed modules.

4. Security group: Add the below lines of code in main.tf file

resource "aws_security_group" "allow_ports" {
name        = "allow_ssh_http"
description = "Allow inbound SSH traffic and http from any IP"
vpc_id      = "${module.vpc.vpc_id}"#ssh accessingress {
from_port   = 22
to_port     = 22
protocol    = "tcp"# Please restrict your ingress to only necessary IPs and ports.
cidr_blocks = ["0.0.0.0/0"]
}# HTTP accessingress {
from_port   = 80
to_port     = 80
protocol    = "tcp"# Please restrict your ingress to only necessary IPs and ports.
cidr_blocks = ["0.0.0.0/0"]
}egress {
from_port   = 0
to_port     = 0
protocol    = "-1"
cidr_blocks = ["0.0.0.0/0"]
}tags {
Name = "Allow SSH and HTTP"
}}

Terraform’s resource block is used to describe infrastructure objects for example instances, security groups, etc. In this example you describe an aws resource of type aws_security_group with a given name allow_ports. Resource type and local name together serve as identifier of the given resource. Within the block body (between { and }) are the configuration arguments for the resource itself.

As the name suggests, a security group is added that would allow any inbound ssh and http traffic using egress and ingress blocks. These blocks may be specified multiple times for each ingress/egress rule. Moreover, vpc idis accessed using the interpolation syntax ${module.vpc.vpc_id} using module because vpc is added as a module.

By now, the setup for vpc with two public and private subnets in two availability zones and one security group is done. You are good to create an instance now where you would instal nginx webserver in a docker container!

5. AWS EC2 instance: Before you create an instance, some important variables mentioned below need to be added to variables.tf file.

variable "aws_amis" {
default = {
us-east-1 = "ami-0f9cf087c1f27d9b1"
eu-west-2 = "ami-095ed825090131933"
}
}variable "instance_type" {
description = "Type of AWS EC2 instance."
default     = "t2.micro"
}variable "public_key_path" {
description = "Enter the path to the SSH Public Key to add to AWS."
default     = "~/.ssh/yourkey.pem"
}variable "key_name" {
description = "AWS key name"
default     = "name of keypair"
}variable "instance_count" {
default = 1
}

aws_amis: specifies the type of ami we are going to use for our instance based upon aregion.

instance_type: specifies the type of instance e.g. General Purpose Instance(T2, M3, M4), Computer Optimized(C5, C4, C3), Memory Optimized(X1, R4, R3), etc based upon the requirement.

public_key_path: specfies the location of the public key (pem file)on your machine used to login to instance.

key_name: specifies the aws key pair name.

instance_count: specifies the number of instances to create, one in this case.

Create one outputs.tf file to output some important information regarding the resources on their successful completion. Lets paste the below lines of code:

output "vpc_id" {
value = ["${module.vpc.vpc_id}"]
}output "vpc_public_subnets" {
value = ["${module.vpc.public_subnets}"]
}output "webserver_ids" {
value = ["${aws_instance.webserver.*.id}"]
}output "ip_addresses" {
value = ["${aws_instance.webserver.*.id}"]
}

Okay all set, now lets create the instance.

Add below lines of code to main.tf file:

resource "aws_instance" "webserver" {
instance_type          = "${var.instance_type}"
ami                    = "${lookup(var.aws_amis, var.aws_region)}"
count                  = "${var.instance_count}"
key_name               = "${var.key_name}"
vpc_security_group_ids = ["${aws_security_group.allow_ports.id}"]
subnet_id     = "${element(module.vpc.public_subnets,count.index)}"
user_data = "${file("scripts/init.sh")}"tags {
Name = "Webserver"
}
}

Again ec2 instance is being created with resource block followed by type of resource aws_instance and a unique local name webserver to identify the resource within other modules. Arguments used are explained as:

instance_type: Type of instance to be created.

ami: Type of ami. Notice the use of lookupwhich fetches the value from variables.tf file based on specified region.

count: Number of instances to be created.

key_name: Name of the key pair.

vpc_security_group_ids: The array of ids of the security groups to be added to the instance/instances.

subnet_id: The id of the subnet to launch the instance in. Herecount.index is passed as a parameter to the element function for multiple instances to be created in each subnet.

user_data: You need to install docker and run nginx in a container once our instance launches. This is achieved by the user_data argument. You can embedd the commands in our main.tf itself or more conveniently create one sh file and pass that to the user_data argument. Lets create one scripts/init.sh file and paste below code:

#!/bin/bash#Installing Dockersudo apt-get remove docker docker-engine docker.io
sudo apt-get update
sudo apt-get install -y \
apt-transport-https \
ca-certificates \
curl \
software-properties-commoncurl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -sudo apt-key fingerprint 0EBFCD88
sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"sudo apt-get update
sudo apt-get install docker-ce -y
sudo usermod -a -G docker $USER
sudo systemctl enable docker
sudo systemctl restart dockersudo docker run --name docker-nginx -p 80:80 nginx:latest

tags: A mapping of tags to assign to the resource.

6. Execution plan and Applying the changes: You can now generate the execution plan by running the terraform plan command and check if everything is as expected. Once confirmed, you can proceed with terraform apply command to actually provision a new or apply the changes to an existing infrastructure.

On Successful apply, you will get the ip addresses of the instance/instances created by Terraform that would be used to login to them using ssh.

You can also explore terraform show command in order to see the detailed information of the provisioned infrastructure.

Once you are logged in, run docker ps and you will see the running nginx container. Curl localhost and you should see the default nginx webpage.

7. Destroying the infrastructure: If you want to delete whole infrastructure, you can run terraform destroy command. However, you can delete a specific resource also using the target flag. For example, you can destroy the above instance by running terraform destroy -target=aws_instance.webserver command.