Installation
We'll install Crossplane using Helm. Official installation documentation
Let's assume that:
- You already have a cluster created
- You already have kubectl installed
- You already have helm installed version 3 or higher
Let's download the helm chart
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update
# Let's download all templates
# The command below will create a folder called crossplane in the root of where the command was applied, move to the correct location if necessary.
helm pull crossplane-stable/crossplane --untar
At this moment we are in a transition from Crossplane version v1 to v2.
When I use Helm I like to download all the files used by the chart locally and have the source code in my hands to analyze locally what has been developed by the community.
A good suggestion is to create a backup of values.yaml to keep it for future reference if needed.
cd crossplane
cp values.yaml values_bkp.yaml
To apply this values.yaml we'll execute the commands below only when everything is configured.
# It's good to create a dedicated namespace beforehand instead of passing the --create-namespace crossplane parameter to create it together in the helm command.
kubectl create namespace crossplane-system
helm install crossplane . -n crossplane-system
If you're installing for the first time and are testing, after understanding the resources come back here and fine-tune the values. But for now, baby steps, go with the default
With the above command we can see that 2 deployments were created, one for crossplane and one for rbac-manager and their respective pods and replicasets, as well as the crossplane-webhook service.
❯ kubectl get all --namespace crossplane-system
NAME READY STATUS RESTARTS AGE
pod/crossplane-5d66b859db-jdfrp 1/1 Running 0 100s
pod/crossplane-rbac-manager-649488698b-r5sbm 1/1 Running 0 100s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/crossplane-webhooks ClusterIP 172.20.169.218 <none> 9443/TCP 101s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/crossplane 1/1 1 1 101s
deployment.apps/crossplane-rbac-manager 1/1 1 1 101s
NAME DESIRED CURRENT READY AGE
replicaset.apps/crossplane-5d66b859db 1 1 1 101s
replicaset.apps/crossplane-rbac-manager-649488698b 1 1 1 101s
Best Practice
I believe Crossplane should have at least 2 pods in its replicaset and each pod should be on different nodes. If your cluster distributes nodes across more than one region, it would be good to ensure they're on different nodes and different regions, in the case of cloud.
There are several ways to solve this, but one of them is to modify the deployment template and add:
replicas: 2
topologySpreadConstraints:
- maxSkew: 1
topologyKey: "kubernetes.io/hostname"
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app.kubernetes.io/name: crossplane
- maxSkew: 1
topologyKey: "topology.kubernetes.io/region"
whenUnsatisfiable: DoNotSchedule
labelSelector:
matchLabels:
app: crossplane
Deployment Crossplane
The main Crossplane deployment is responsible for running Crossplane Core, which acts as the central orchestrator of the entire platform. This deployment runs multiple controllers within a single process, each with specific responsibilities:
Package Manager Controller
- Manages installation, update, and removal of providers and configurations
- Automatically resolves dependencies between packages
- Monitors health status of installed packages
- Controls complete component lifecycle
Composition Controller
- Processes CompositeResourceDefinitions (XRDs) and their respective Compositions
- Manages lifecycle of Composite Resources (XRs) and Claims (XRCs)
- Performs transformations and mappings between high-level resources and managed resources
- Executes continuous reconciliation loops to maintain consistency
Revision Controller
- Controls versioning of packages and compositions
- Allows safe rollbacks between versions
- Maintains revision history for auditing
Deployment crossplane-rbac-manager
The crossplane-rbac-manager is a dedicated deployment that automates RBAC permission management for all Crossplane components. It's installed and enabled by default, automatically creating and maintaining all necessary ClusterRoles.
Main Responsibilities:
-
Service Account Management
- Creates and links specific RBAC roles for provider service accounts
- Allows providers to control their managed resources with appropriate permissions
- Configures the crossplaneServiceAccount with permissions to create managed resources
-
ClusterRole Automation
- Automatically generates ClusterRoles based on CRDs installed by providers
- Maintains synchronization between new providers and their required permissions
- Removes obsolete permissions when providers are uninstalled
Default Role Hierarchy
Crossplane creates a well-defined hierarchy of roles for different access levels:
-
crossplane-admin
- Full access to all Crossplane resources
- Ability to manage providers, compositions, and managed resources
- Can link RBAC roles to other entities (requires care in production)
-
crossplane-edit
- Creation and modification of Crossplane resources
- Access for development and daily operations
- No ability to manage RBAC permissions
-
crossplane-view
- Viewing of all Crossplane resources
- Ideal for monitoring and auditing
- Read-only access to managed resources
-
crossplane-browse
- Basic navigation through Crossplane resources
- Most restrictive access for users who only need to view basic information
Namespace-Specific Roles
In addition to global ClusterRoles, the RBAC manager also creates namespace-specific roles, allowing granular access control by project or team.
If you want to understand better check
❯ kubectl get clusterrole -l=app=crossplane
NAME CREATED AT
crossplane 2023-07-30T18:21:36Z
crossplane-admin 2023-07-30T18:21:36Z
crossplane-browse 2023-07-30T18:21:36Z
crossplane-edit 2023-07-30T18:21:36Z
crossplane-rbac-manager 2023-07-30T18:21:36Z
crossplane-view 2023-07-30T18:21:36Z
crossplane:aggregate-to-admin 2023-07-30T18:21:36Z
crossplane:aggregate-to-browse 2023-07-30T18:21:36Z
crossplane:aggregate-to-edit 2023-07-30T18:21:36Z
crossplane:aggregate-to-view 2023-07-30T18:21:36Z
crossplane:allowed-provider-permissions 2023-07-30T18:21:36Z
crossplane:system:aggregate-to-crossplane 2023-07-30T18:21:36Z
Customizing the Installation
At a later time when everything is well-founded, see everything that can be customized in the installation.
https://docs.crossplane.io/v1.20/software/install/#customize-the-crossplane-helm-chart
Although it's possible to define providers, functions, and other configurations directly in the Helm chart's values.yaml during Crossplane installation, this approach is not recommended in practice. Maintaining separate manifests for each component offers greater flexibility, granular control, and facilitates versioning and resource maintenance. Additionally, this separation allows better traceability of changes and facilitates implementation of GitOps practices.
CRDs Overview
Before starting the main Crossplane container, an init container runs. The init container installs the Custom Resource Definitions, sets up Crossplane webhooks, and installs any provided providers or configurations, but we haven't provided any yet during installation. Could we? Yes, but we didn't.
These are the CRDs.
❯ kubectl api-resources | grep crossplane
NAME SHORTNAMES APIVERSION NAMESPACED KIND
compositeresourcedefinitions xrd,xrds apiextensions.crossplane.io/v1 false CompositeResourceDefinition
compositionrevisions comprev apiextensions.crossplane.io/v1 false CompositionRevision
compositions comp apiextensions.crossplane.io/v1 false Composition
environmentconfigs envcfg apiextensions.crossplane.io/v1beta1 false EnvironmentConfig
usages apiextensions.crossplane.io/v1beta1 false Usage
configurationrevisions pkg.crossplane.io/v1 false ConfigurationRevision
configurations pkg.crossplane.io/v1 false Configuration
controllerconfigs pkg.crossplane.io/v1alpha1 false ControllerConfig
deploymentruntimeconfigs pkg.crossplane.io/v1beta1 false DeploymentRuntimeConfig
functionrevisions pkg.crossplane.io/v1 false FunctionRevision
functions pkg.crossplane.io/v1 false Function
imageconfigs pkg.crossplane.io/v1beta1 false ImageConfig
locks pkg.crossplane.io/v1beta1 false Lock
providerrevisions pkg.crossplane.io/v1 false ProviderRevision
providers pkg.crossplane.io/v1 false Provider
storeconfigs secrets.crossplane.io/v1alpha1 false StoreConfig
For example we see the providers CRD and we could have defined which providers should be installed using the values.yaml in the helm chart at the point below, or we would simply do that later with a providers manifest.
provider:
# -- A list of Provider packages to install.
packages: []
### If we were to add an aws provider...
# packages:
# xpkg.upbound.io/crossplane-contrib/provider-aws: v0.39.0
configuration:
# -- A list of Configuration packages to install.
packages: []
The main CRDs installed by the init container include:
- CompositeResourceDefinitions, Compositions, Configurations and Providers
- Locks to manage package dependencies
- ControllerConfigs to apply configurations to installed providers
- StoreConfigs to connect external secret stores like HashiCorp Vault
- DeploymentRuntimeConfigs to customize how providers run (detailed in RuntimeConfig)
The crossplane pod works in a reconciliation loop, constantly checking the status of deployed resources and correcting any differences. There's a time for this that can be changed.
Crossplane monitors resources through a Kubernetes watch or through periodic polling. Some resources can be watched and polled.
Crossplane requests that the kubernetes kube-api-server notify it about any changes to objects. This notification tool is a Watch. The kube-api-server will notify of changes to crossplane resources, but not within providers. This is a crossplane function, to monitor differences between what it should do and what resources it already has in the provider.
The crossplane pod can receive some arguments, which are defined inside arg, to change its configuration. Below is an example changing some things.
args:
# Periodic polling of items that the kubernetes api is not responsible for, but crossplane is
- args:
- core
- start
- --sync-interval=60m # this is the default
- --poll-interval=1m # this is the default
- --max-reconcile-rate=10 # 10 times is the default
Now that you have crossplane installed, let's create our first provider in provider
Uninstall
To completely uninstall Crossplane you need to follow an order, as there are dependencies between resources.
Additionally, uninstalling a provider will remove all custom resources definitions it controls, so the resource will be abandoned, meaning it exists but has no control as if it had been created manually, because they won't be deleted.
-
Remove all compositions
# To list
kubectl get xrd
# kubectl delete xrd x
# kubectl delete xrd y
# ... -
Remove all managed resources
# To list
kubectl get managed
# kubectl delete x
# kubectl delete y
# ... -
Remove all providers
# To list
kubectl get providers
# kubectl delete provider x
# kubectl delete provider y
# ... -
Uninstall using helm if you installed via helm
helm uninstall crossplane --namespace crossplane-system
## Finally delete the namespace
kubectl delete namespace crossplane-system
Crossplane CLI
https://github.com/crossplane/crossplane-cli
There's a Crossplane CLI that can be used to install and configure providers, but nothing you can't do using manifests. This CLI is more for use in provider development.
Dependency Management
As we saw earlier, a package can have dependencies on other packages including configurations and other providers.
So when installing, if this can't be resolved we'll have Health below as False instead of true. To see why do a describe on the resource
❯ k get providers
NAME INSTALLED HEALTHY PACKAGE AGE
provider-aws-ec2 True True xpkg.upbound.io/upbound/provider-aws-ec2:v1 9h
provider-aws-s3 True True xpkg.upbound.io/upbound/provider-aws-s3:v1 9h
upbound-provider-family-aws True True xpkg.upbound.io/upbound/provider-family-aws:v1.20.1 9h
Updating is simple, just change the package version.