Question 24 | OPA Gatekeeper Policy Extension
Use context: kubectl config use-context infra-prod
There is an existing Open Policy Agent + Gatekeeper policy to enforce that all Namespaces need to have label security-level set. Extend the policy constraint and template so that all Namespaces also need to set label management-team. Any new Namespace creation without these two labels should be prevented.
Write the names of all existing Namespaces which violate the updated policy into /opt/course/p2/fix-namespaces.
Answer:
We look at existing OPA constraints, these are implemeted using CRDs by Gatekeeper:
➜ k get crd
NAME CREATED AT
blacklistimages.constraints.gatekeeper.sh 2020-09-14T19:29:31Z
configs.config.gatekeeper.sh 2020-09-14T19:29:04Z
constraintpodstatuses.status.gatekeeper.sh 2020-09-14T19:29:05Z
constrainttemplatepodstatuses.status.gatekeeper.sh 2020-09-14T19:29:05Z
constrainttemplates.templates.gatekeeper.sh 2020-09-14T19:29:05Z
requiredlabels.constraints.gatekeeper.sh 2020-09-14T19:29:31Z
So we can do:
➜ k get constraint
NAME AGE
blacklistimages.constraints.gatekeeper.sh/pod-trusted-images 10m
NAME AGE
requiredlabels.constraints.gatekeeper.sh/namespace-mandatory-labels 10m
# And check violations for the namespace-mandatory-label one, which we can do in the resource status:
➜ k describe requiredlabels namespace-mandatory-labels
Name: namespace-mandatory-labels
Namespace:
Labels: <none>
Annotations: <none>
API Version: constraints.gatekeeper.sh/v1beta1
Kind: RequiredLabels
...
Status:
...
Total Violations: 1
Violations:
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"security-level"}
Name: sidecar-injector
Events:
# We see one violation for Namespace "sidecar-injector". Let's get an overview over all Namespaces:
➜ k get ns --show-labels
NAME STATUS AGE LABELS
default Active 21m management-team=green,security-level=high
gatekeeper-system Active 14m admission.gatekeeper.sh/ignore=no-self-managing,control-plane=controller-manager,gatekeeper.sh/system=yes,management-team=green,security-level=high
jeffs-playground Active 14m security-level=high
kube-node-lease Active 21m management-team=green,security-level=high
kube-public Active 21m management-team=red,security-level=low
kube-system Active 21m management-team=green,security-level=high
restricted Active 14m management-team=blue,security-level=medium
security Active 14m management-team=blue,security-level=medium
sidecar-injector Active 14m <none>
When we try to create a Namespace without the required label we get an OPA error:
➜ k create ns test
Error from server ([denied by namespace-mandatory-labels] you must provide labels: {"security-level"}): error when creating "ns.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [denied by namespace-mandatory-labels] you must provide labels: {"security-level"}
Next we edit the constraint to add another required label:
k edit requiredlabels namespace-mandatory-labels
# kubectl edit requiredlabels namespace-mandatory-labels
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: RequiredLabels
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"constraints.gatekeeper.sh/v1beta1","kind":"RequiredLabels","metadata":{"annotations":{},"name":"namespace-mandatory-labels"},"spec":{"match":{"kinds":[{"apiGroups":[""],"kinds":["Namespace"]}]},"parameters":{"labels":["security-level"]}}}
creationTimestamp: "2020-09-14T19:29:53Z"
generation: 1
name: namespace-mandatory-labels
resourceVersion: "3081"
selfLink: /apis/constraints.gatekeeper.sh/v1beta1/requiredlabels/namespace-mandatory-labels
uid: 2a51a291-e07f-4bab-b33c-9b8c90e5125b
spec:
match:
kinds:
- apiGroups:
- ""
kinds:
- Namespace
parameters:
labels:
- security-level
- management-team # add
As we can see the constraint is using kind: RequiredLabels as template, which is a CRD created by Gatekeeper. Let's apply the change and see what happens (give OPA a minute to apply the changes internally):
➜ k describe requiredlabels namespace-mandatory-labels
...
Violations:
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"management-team"}
Name: jeffs-playground
After the changes we can see that now another Namespace jeffs-playground is in trouble. Because that one only specifies one required label. But what about the earlier violation of Namespace sidecar-injector?
➜ k get ns --show-labels
NAME STATUS AGE LABELS
default Active 21m management-team=green,security-level=high
gatekeeper-system Active 17m admission.gatekeeper.sh/ignore=no-self-managing,control-plane=controller-manager,gatekeeper.sh/system=yes,management-team=green,security-level=high
jeffs-playground Active 17m security-level=high
kube-node-lease Active 21m management-team=green,security-level=high
kube-public Active 21m management-team=red,security-level=low
kube-system Active 21m management-team=green,security-level=high
restricted Active 17m management-team=blue,security-level=medium
security Active 17m management-team=blue,security-level=medium
sidecar-injector Active 17m <none>
Namespace sidecar-injector should also be in trouble, but it isn't any longer. This doesn't seem right, it means we could still create Namespaces without any labels just like using k create ns test.
So we check the template:
➜ k get constrainttemplates
NAME AGE
blacklistimages 20m
requiredlabels 20m
➜ k edit constrainttemplates requiredlabels
# kubectl edit constrainttemplates requiredlabels
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
...
spec:
crd:
spec:
names:
kind: RequiredLabels
validation:
openAPIV3Schema:
properties:
labels:
items: string
type: array
targets:
- rego: |
package k8srequiredlabels
violation[{"msg": msg, "details": {"missing_labels": missing}}] {
provided := {label | input.review.object.metadata.labels[label]}
required := {label | label := input.parameters.labels[_]}
missing := required - provided
# count(missing) == 1 # WRONG
count(missing) > 0
msg := sprintf("you must provide labels: %v", [missing])
}
target: admission.k8s.gatekeeper.sh
In the rego script we need to change count(missing) == 1 to count(missing) > 0 . If we don't do this then the policy only complains if there is one missing label, but there can be multiple missing ones.
After waiting a bit we check the constraint again:
➜ k describe requiredlabels namespace-mandatory-labels
...
Total Violations: 2
Violations:
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"management-team"}
Name: jeffs-playground
Enforcement Action: deny
Kind: Namespace
Message: you must provide labels: {"security-level", "management-team"}
Name: sidecar-injector
Events: <none>
This looks better. Finally we write the Namespace names with violations into the required location:
# /opt/course/p2/fix-namespaces
sidecar-injector
jeffs-playground