Infrastructure-as-Code (IaC) is the practice of managing cloud computing infrastructure such as virtual machines, containers, virtual networks, etc, via machine-readable templates and definition files. These definitions, once written, can be used multiple times to provision cloud infrastructure via IaC tools. Many companies, especially software and SaaS companies are embracing IaC because it standardizes IT configuration, minimizes manual infrastructure setup, and can boost the efficiency of the software development process.
In this article we will explore best practices for Infrastructure-as-Code (IaC) and strategies for utilizing IaC alongside your core DevOps processes.
Infrastructure-as-Code (IaC) and DevOps
Modern software companies have adopted DevOps processes. DevOps teams are responsible for the cloud infrastructure that runs the software applications. During the software development and delivery process, a DevOps team must provision cloud resources for many scenarios.
A typical software team uses separate development, test, and production computing environments. With each release cycle of the software, the DevOps team has to set up and tear down the computing infrastructure for these separate environments multiple times. Without IaC the DevOps engineers would be wasting precious hours provisioning cloud computing resources manually via CLIs or GUIs. IaC can ease this configuration burden, and has become a vital part of the DevOps practices.
Infrastructure-as-Code (IaC) Tools
The popularity of IaC has led to the development of many proprietary and open-source IaC tools. Many public cloud providers offer a dedicated IaC service intended specifically for provisioning their cloud resources. While such tools support only a single cloud provider, there are other tools offered by third parties that can provision resources across multiple cloud computing systems.
Most IaC tools utilize either YAML or JSON formatting for their definition files. However, each tool uses proprietary syntax so that the definition files created for one tool cannot be used by another.
Declarative vs Imperative IaC Definitions
There are two styles of IaC definitions.
- Declarative IaC
In declarative style, you define the desired end state of the system. The vast majority of IaC tools adopt the declarative approach, with some support for imperative use cases. While decorative IaC may have a steeper learning curve, it can abstract most of the underlying complexities. Therefore, declarative IaC is suitable for managing systems with complex cloud infrastructure.
- Imperative IaC
In contrast to the declarative style, the imperative definitions resemble procedures. They contain a series of steps that must be performed on a system to reach a final status. Imperative style is easier for beginners and could be more flexible for certain use cases. However, they may be harder to maintain for complex cloud infrastructure.
Now, let’s look at some of the popular IaC tools.
Ansible
Ansible is one of the leading open-source IaC projects. It has a modular architecture so that any cloud service provider can implement support for Ansible, and all leading cloud providers have done that. The detailed list of Ansible modules for cloud providers can be found here.
Ansible users can define one or more Ansible playbooks. An Ansible playbook is a blueprint for automation tasks and can be used for complex IT infrastructure tasks in an Ansible inventory. Ansible provides extensive support for automating IT infrastructure (such as operating systems and container/Kubernetes platforms). It also supports configuration of network devices, security settings and storage for market-leading cloud platforms and services.
Terraform
Terraform is another popular open-source IaC tool, provided by HashiCorp. Terraform supports provisioning resources on all leading cloud providers. Teams can leverage a wide array of providers from the Terraform Registry tp
Terraform users can define IT and infrastructure configuration inside a Terraform manifest file. Teams may define configuration for computing, networking, security settings related to their systems. Teams can create “modules” or reusable components to simplify the infrastructure provisioning process. Oftentimes, DevOps teams will leverage Terraform alongside Ansible or other tooling in their process.
AWS CloudFormation
AWS CloudFormation is an IaC service provided by Amazon Web Services (AWS). CloudFormation supports managing resources on AWS cloud using JSON format.
DevOps members can create CloudFormation (CF) templates defining almost any known AWS resource, including compute and storage services, as well as security and configuration settings such as IAM users/roles and networking rules.
Teams can leverage CloudFormation alongside other AWS CI/CD tools such as AWS CodeDeploy and AWS CodePipeline in their build pipeline. CloudFormation templates work exclusively in AWS, and are not suitable for other cloud platforms such as Microsoft Azure, and Google Cloud Platform (GCP).
Azure Resource Manager
Azure Resource Manager (ARM) is an IaC tool offered by Azure. It used JSON format for managing resources on Azure cloud. Azure Resource Manager enables teams to define configuration for Azure resources and resource groups.
Similar to CloudFormation, ARM templates work exclusively in Azure, and are not suitable for other cloud platforms such as Amazon Web Services (AWS), and Google Cloud Platform (GCP).
Google Cloud Deployment Manager
The Google Cloud Deployment Manager is the IaC offering from Google. It uses YAML format and enables teams to utilize Jinja or Python to declaratively define cloud resources on Google Cloud Platform (GCP).
Like CloudFormation and ARM, Deployment Manager templates work exclusively in GCP, and are not suitable for other cloud platforms such as AWS and Azure.
Puppet
Puppet is one of the oldest IaC tools. It adopts a client-server architecture which differentiates it from other agentless tools like Ansible.
With Puppet, one server is used as a master or “Puppet server” and other nodes wher configuration needs to be managed become client nodes where the Puppet Agent is deployed. Teams can use manifests to describe how the operating system, services, packages, and networking should be configured. Puppet will then apply these manifests from your Puppet server to your client nodes.
Pulumi
Pulumi is different from most of the above tools as it takes a more imperative approach, allowing you to use programming languages like JavaScript, Python, or Go to create the definition templates.
Benefits of Infrastructure-as-Code (IaC)
Eliminates Time-Consuming, Routine Work
Manually provisioning a complex cloud infrastructure setup is time-consuming. When releasing a new software version or a patch, there will be several cycles of infrastructure setup and teardown. Therefore, sticking to manual methods for infrastructure provisioning could significantly stretch software release timelines.
By using IaC tooling, DevOps teams can instantly provision the cloud infrastructure, thus improving the efficiency of the overall process. IaC helps the DevOps team to stay lean by automating the routine and labor-intensive work.
Reduce Manual Errors
Manual provisioning of cloud infrastructure is error-prone. Errors in computing infrastructure could be hard to troubleshoot and cost a lot of time. IaC can eliminate most manual errors. Once an IaC definition is created and tested, using that will always produce the exact same setup. Therefore, IaC will save the DevOps teams from exhaustive troubleshooting sessions.
Documents The Infrastructure
As much as your software needs documentation, the infrastructure also needs documentation. This is especially important when working in a large, diverse team that spans across multiple regions. The IaC definitions can act as part of the documentation for your cloud infrastructure, eliminating the requirement of maintaining some parts of your legacy documentation. Unlike separately created documentation, your documentation with IaC would always be an exact representation of the actual setup.
Infrastructure-as-Code (IaC) Best Practices
Choose The Appropriate Style
We already discussed imperative and the declarative styles of IaC definitions. When adopting IaC practices, you must decide on what style your team is going to use. While the declarative style seems to be the more dominant of the two, there could be use cases that benefit from the imperative styles of IaC. You must balance the pros and cons according to your context when selecting one of these styles.
Choose The Correct Tools
There are many IaC tools and systems out there and we already discussed some of the popular IaC tools. These tools have different strengths and weaknesses. Therefore, you must carefully analyze their capabilities according to your requirements before making a selection. As an example, if you are working with serverless applications, you may not benefit from a tool like Puppet that has extensive support for configuring servers and virtual machines.
Use Version Control
Version control is a necessary part of software development. IaC models your infrastructure configuration in software so that version control of IaC definitions is also essential. Version control becomes more significant in large teams that work with complex cloud infrastructure setups. However, it is a good practice to adopt version control regardless of the scale of your team.
Centralized version control enables the DevOps teams to collaboratively work on the IaC definitions, just like on any other software. It also keeps track of changes and provides means to rollback IaC and infrastructure changes if something goes wrong.
Avoid Manual Infrastructure Changes
Once you ingrain Infrastructure-as-Code into your software development lifecycle, you should ensure that all infrastructure is provisioned via IaC tooling only. Any manual tweaks done directly to cloud resources could lead to the actual infrastructure being different from the definitions in your IaC templates (aka drift). The next time you use the IaC templates to provision, the new infrastructure might not be identical to your current infrastructure.
Engineers who make manual modifications to resources often do not bother to document their work, making it harder to reproduce your current infrastructure in a new environment. Manual changes could result in hard-to-debug problems in your application.
Test Your Infrastructure-as-Code Templates
Testing your IaC definitions is almost as important as testing your software. However, testing IaC requires different approaches to software testing. Nowadays, software testing may involve significant automation. Teams may consider using certain template validation after making changes to IaC files. Syntax checking, and apply-and-destroy are the most common methods that you can use to test the IaC definitions.
As the most basic form of syntax checking, you must check the formatting of your IaC definition language such as YAML, JSON, etc. Text editors like Visual Studio Code can provide tools for performing these kinds of checks. After writing the files, you should verify the syntax against the tool that you are using. Most IaC tools provide this testing as a feature. As an example, if you are using AWS CloudFormation templates, the AWS CLI provides a command `aws cloudformation validate-template` to validate the syntax of a template. Other IaC tools also have similar services to verify the syntax.
Finally, you should also test your IaC definition by actually creating the infrastructure. After creating the infrastructure, you must manually verify that the cloud resources are created with the intended parameters.
Be Cautious With Sensitive Variables and Parameters
When provisioning the cloud infrastructure using IaC templates, you need to provide secure parameters such as passwords, SSH keys, or API tokens. While you can hardcode such parameters into the template like any other parameter, it would be insecure to store plain text versions of such sensitive data. The IaC tools provide different solutions to handle this problem.
AWS recommends using the AWS Secrets Manager for securely storing parameters. Terraform lets you pass the secure parameters separately as environment variables in your operating system. It also supports using separate variable definition templates to define the secure parameters so that they can be stored separately from the main definition files.
Due to the variances of the tools, it is not possible to recommend one single way for securing parameters or environment variables in IaC. You must follow the guidelines of your respective toolset to ensure that parameters are secure and not exposed to unintended parties.
Write Precise Code Rather Than Documentation
Your team should treat your IaC tooling as executable documentation for your infrastructure. Therefore, writing clean IaC definitions and configuration files can provide a lot of value over writing additional documentation for the infrastructure. Strive to write clear and concise IaC definitions so that you have up-to-date and executable documentation your team members can reference to understand your systems.
Write Modular Configuration Files
When your IaC definitions are growing in size and complexity, try to split them into modules. Most of the IaC tools support this approach by allowing you to organize your definitions into multiple files in a folder.
Modular IaC definitions encourage reuse so that you can avoid lengthy repetitions. Modularity also makes it easier for multiple team members to work collaboratively. Teams may consider creating reusable modules or components for specific networking configurations, server or serverless resources, and other such resources.
Conclusion
Infrastructure-as-Code (IaC) is an essential component for modern DevOps teams who are working with applications that span across one or more cloud providers. IaC can yield massive benefits, by saving time and automating many tedious tasks. While we have discussed some of the best practices for utilizing Infrastructure-as-Code, your team should determine what is best for managing and building IaC and configuration management processes into your overall DevOps follow and build processes.
