Terraform Isn’t Dead
Deploying applications in the cloud presents various challenges, primarily centered around the complexity of infrastructure management. Developers have come to rely on Infrastructure as Code (IaC) to streamline and automate these processes. However, a new paradigm, Infrastructure from Code (IfC), is emerging to further enhance deployment efficiency and alignment between infrastructure and application development.
Terraform has recently made headlines due to HashiCorp’s acquisition by IBM sparking discussions that speculate on its future. However, such conjectures overlook the enduring significance of Terraform and Infrastructure as Code (IaC) in future and existing projects.
It is likely that Terraform will not only continue to be utilized by teams across the industry but will also see further enhancements rather than being replaced. The focus should shift to how we can evolve to meet the changing demands of modern cloud environments, which will no doubt include enhancements to the way we leverage IaC.
In this article, I’ll discuss IfC as an enhancement to IaC. Throughout this article, I’ll use Terraform as my default IaC, but the concepts apply to other IaC solutions, such as OpenTofu, Pulumi and more. As part of the Nitric team, I’ll use our open source framework for code examples to help explain some of the IfC concepts.
IaC’s Role in Cloud Deployment
Cloud applications are built by giving applications access to functionality from a variety of distributed services. Building an application requires building blocks such as computes, storage buckets, key-value stores, queues, event messaging and scheduled tasks. These services are generally unaware of each other; the glue is the application code you write to utilize their unique capabilities.
You also need to configure the cloud services so that they run as expected. For example, key-value stores offer built-in replication support by automatically copying data across multiple storage nodes. This helps with auto-recovery from disasters; you still have your data in case of server failure.
In the early days, setting up a cloud environment involved manual configuration through the cloud’s consoles. Teams would log in to the console and manually configure services such as storage buckets, set up identity access management (IAM) policies and adjust permissions for cloud functions such as AWS Lambda to access these resources. As projects scaled, this method generally required following a growing set of documented steps on a per-project basis, which was labor-intensive and prone to human error. This caused inconsistencies across different environments and projects, basically a maintenance nightmare.
To combat these challenges, IaC tools including Pulumi, Terraform, AWS Cloud Development Kit (CDK) and Ansible have been instrumental. They enhance automation and speed, which not only expedites the deployment processes but also minimizes the risks of human errors, which is crucial for maintaining reliable environments during scaling or multi-region deployments. By defining infrastructure as code, these tools allow version control, enabling better management of changes and facilitating easy rollbacks when necessary. This marks a significant advancement over manual processes, which often lack transparency.
Pulumi allows developers to use familiar languages like JavaScript, Python and Go, making it highly accessible. Terraform’s declarative approach specifies the desired state without managing state transitions, ideal for clear, straightforward deployment strategies. AWS CDK benefits from tight integration with AWS services, using familiar programming languages to define resources, and Ansible, typically known for configuration management, also facilitates infrastructure deployment, providing a versatile tool for both application deployment and infrastructure setup.
The Need for Enhancing IaC
Collectively, these IaC tools have significantly transformed organizational operations, leading to more agile, secure and efficient infrastructures. However, despite its advantages, IaC has limitations, particularly in maintaining synchronization between the rapidly evolving application code and the static IaC scripts.
When a developer decides to replace a manually managed storage bucket with a third-party service alternative, the corresponding IaC scripts must also be manually updated, which becomes cumbersome and error-prone as projects scale. The desync that occurs between the application and its runtime can lead to serious security implications, where resources are granted far more permissions than they require or are left rogue and forgotten.
Basically, maintaining consistency across various projects and organizations becomes a major hurdle with economic impacts on the project. So what can we do about it?
Introducing Infrastructure from Code (IfC)
The major gap between applications and their cloud runtimes is their inability to stay in sync with each other.
IfC is an innovative approach that enhances the traditional IaC paradigm by keeping code and infrastructure in perfect synchronization. This method not only automates the provisioning and deployment of infrastructure but also continuously aligns it with the application code changes.
By integrating IfC into development workflows alongside IaC, organizations can achieve an unprecedented level of consistency and efficiency, as both codebases and their underlying infrastructure are managed in tandem. This seamless integration prevents discrepancies between the development and operational environments, reducing deployment failures and operational overhead. Adding IfC to complement IaC represents a significant step forward in DevOps practices, enabling faster, more reliable and more scalable application deployments.
Whenever an application changes, IfC can help provision resources and configurations that accurately reflect its runtime requirements, eliminating much of the manual work typically involved. This automation includes setting up IAM roles and permissions, helping ensure that security and compliance are seamlessly maintained.
How IfC and IaC Work Together
Synchronizing an application with its runtime resources and config is a two-step process. The IfC framework is responsible for automatically building a resource specification by inferring requirements from the application code and then mapping these requirements to extensible and flexible IaC modules that fulfill the request.
Step 1: Build a Resource Spec
First, IfC builds a resource specification of the application’s current runtime requirements. The following snippet of code is more informative than you might realize.
If you scan through it without focusing on the application logic, you can infer that this application requires:
- An API named
milkywaywith:- A compute/handler that is routed on the
/planets/:idpath.- A key-value store named
planets_store.- This handler intends to
getandsetvalues.
- This handler intends to
- A key-value store named
- A storge bucket named
planets_images.-
- This handler intends to
readandwritecontent.
- This handler intends to
-
- A compute/handler that is routed on the
The hierarchy and the intention provided by the developer allow you to craft a resource specification that effectively communicates the runtime requirements of the application. It’s important to note that developers are not manually writing detailed infrastructure configurations; they are simply coding the application to leverage necessary resources from a software development kit (SDK).
To build the runtime specification requirements, there’s no need to know or specify infrastructure-related policies such as the data replication settings of the key-value store. In fact, as with traditional IaC projects, the developer isn’t required to even know which key-value store their software will eventually run on. They only need to know it will be available, and their application will have adequate roles and permissions to access it. In the example above, this is simply setting and getting values.
The beauty of automating a resource specification with IfC is that it is not static; it is living documentation that automatically updates whenever the application needs deployment, testing or analysis. This adaptability is crucial because it helps ensure that the infrastructure always aligns with the current state of the application without manual intervention.
Resource specification as visualized by the Nitric framework.
Step 2: Fulfill the Requirements With IaC
Now that IfC has automatically created a specification that lists all the required resources, their relationship to each other and their required permissions, the IfC framework can streamline the provisioning process by leveraging IaC.
IaC providers like Terraform utilize the concept of modules. A Terraform module is a reusable, self-contained set of Terraform configurations that encapsulates a specific set of infrastructure resources. Modules simplify and organize infrastructure code and allow reusing configurations across different projects or within different parts of the same project.
The following is an example of a module required to provision a secret, and this is where teams can apply their infrastructure governance. This example implements a consistent way of tagging the secret resources "x-nitric-${var.stack_id}-name"=var.secret_name and specifies the replication policy auto{}, which is applied universally for all secrets deployed in this project.
The secret will also require roles and permissions so that the application code can access it. The following snippet shows the possible ways an application can declare and specify its intent for a secret. There are two types of intentions available to this secret: access and putting, and the resource specification will capture details that the IfC framework will use to determine which permissions to grant.
The modules shown below should not be unfamiliar to Terraform developers. In many cases, the transition to leverage Terraform modules in an IfC framework will involve using resources that have already been developed, requiring only light modifications to expose parameters when fine-grained control is required.
Using this technique of mapping resource requirements to modules for all resources allows teams to streamline the communication of requirements from application to infrastructure.
Frameworks like Nitric will continually grow their resource support, empowering developers with a toolkit of self-provisioning resources that can be visualized and even provisioned locally for an offline development emulation experience.
Enhance IaC with Automation of IfC
By automating infrastructure provisioning and management, IaC not only simplifies the DevOps workflow but significantly enhances it. Instead of DevOps teams manually reviewing application updates and painstakingly updating Terraform scripts, they can now rely on resource specifications from IfC to determine if their Terraform modules require modifications.
This shift not only reduces the potential for human error but also accelerates deployment cycles, so teams can focus more on innovation and less on infrastructure management. As organizations continue to adopt IaC tools like Terraform, Pulumi and Ansible, the role of manual intervention diminishes, paving the way for a more efficient, reliable and agile development process.
If you’d like to take this concept for a spin, Nitric is an open source IfC framework for building in your language of choice and deploying applications to your AWS, Google Cloud Platform and Microsoft Azure clouds.
