Skip to main content

Resource Management (Evolution)

It's difficult to distinguish what is an application and what is infrastructure. We are moving towards immutability and everything managed through APIs, no matter if it's network, processes, clusters, or anything else. It's all about managing resources and what has changed over time is how we manage resources.

First Wave​

We started writing scripts to manage resources that were complicated to write and difficult to maintain, given the number of permutations between the desired state and the actual state of resources. The result was creating tools like Chef and Puppet, and some others, which were swallowed by Ansible. We call these tools configuration management tools and they all have two things in common:

  • They are based on promise theory, allowing us to specify what we want instead of specifying how to do something, and their job was to convert the desired state into the actual state.
  • They were designed to work with servers where everything is mutable. We call these tools configuration management tools.

Second Wave​

Then we started using APIs and immutability and Terraform and Pulumi were born to declare what we want. We would define what we want, the desired state, and these tools would reconcile this with the actual state by talking to APIs, whether AWS, Azure, Google Cloud, or any other, and thus Infrastructure as Code was born.

There's always some binary somewhere that interprets our manifests, our declarations, and communicates with some API to transform our desired state into actual state.

Third Wave (Now)​

Let's call this wave of tools Control Plane. Its main focus is to create custom APIs. Instead of defining what we want at the same level as the resources we're trying to manage, these tools allow us to become service providers.

They achieve this by allowing us to create APIs that consumers can talk to. Now we can define what an application, a database, a cluster, or anything else is as an API endpoint with its schema.

Consumers don't need to deal with low-level details like VPC, subnets, or anything else that providers like AWS or Kubernetes give us. There's a clear internal separation of concerns with some people in a company creating services and others consuming those services.

Consumers consume by invoking APIs and providers provide by building services behind those APIs. While infrastructure as code tools are based on lessons learned from consuming services created by others, Control Plane tools are based on lessons learned from building services.

Tools in this category:

  • Open Application Model (OAM): One of the oldest and started with a focus on applications, but today has been extended to all types of resources.
    • KubeVela is the most commonly used implementation of OAM, but its popularity is declining.
  • Cluster API: Its problem is in its focus only on Kubernetes clusters, solving only part of the problem.
  • Crossplane: Like KubeVela, it allows us to create our own API endpoints, as well as processes that listen to these endpoints and perform the necessary reconciliation to convert the actual state to the desired state of resources. Because it's more flexible, we can create any type of definition with very few restrictions, if any. It started with some limitations, but has been improving along the way.

The kro - Kube Resource Orchestrator is part of this wave. It is very similar to Crossplane Compositions and may one day be a better solution, but it's still growing. It's a very new project, but with a lot of potential and therefore worth keeping an eye on. Is it time to use? No, but it's time to test and, if possible, participate in the project.