Advanced Kubectl Commands
The most advanced commands in kubectl use json queries to filter output. See about jsonpath in jsonpath.
When working with a production cluster, we'll have hundreds or thousands of pods, dozens of nodes, deployments, replicasets, services, secrets and other Kubernetes objects, and we use kubectl to view these objects.
Often you'll have requirements where you need to print a general summary or print specific fields from multiple resources at the same time.
You won't want to keep running describe for each object doing a grep in a loop in a script and manipulating the output all the time with awk, cut, etc. First, it takes time to create the script, second, we have tools for this that kubectl itself supports using jsonpath.
Kubectl makes requests to kube-apiserver and gets a json response. Under the hood, kubectl itself manipulates this data and displays organized information, but under the hood it received a json.
kubectl create deployment nginx --image nginx
# Here shows a manipulated output
kubectl get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
nginx 1/1 1 1 7s
kubectl get deploy -o wide
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR
nginx 1/1 1 1 31m nginx nginx app=nginx
# But what it received was this
kubectl get deploy -o json
{
"apiVersion": "v1",
"items": [
{
"apiVersion": "apps/v1",
"kind": "Deployment",
"metadata": {
"annotations": {
"deployment.kubernetes.io/revision": "1"
},
"creationTimestamp": "2024-02-20T08:35:03Z",
"generation": 1,
"labels": {
"app": "nginx"
},
"name": "nginx",
"namespace": "default",
"resourceVersion": "1430016",
"uid": "e4ecc32a-7d98-48ff-8bce-ba8af52e27e2"
},
"spec": {
"progressDeadlineSeconds": 600,
"replicas": 1,
"revisionHistoryLimit": 10,
"selector": {
"matchLabels": {
"app": "nginx"
}
},
"strategy": {
"rollingUpdate": {
"maxSurge": "25%",
"maxUnavailable": "25%"
},
"type": "RollingUpdate"
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"app": "nginx"
}
},
"spec": {
"containers": [
{
"image": "nginx",
"imagePullPolicy": "Always",
"name": "nginx",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File"
}
],
"dnsPolicy": "ClusterFirst",
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"terminationGracePeriodSeconds": 30
}
}
},
"status": {
"availableReplicas": 1,
"conditions": [
{
"lastTransitionTime": "2024-02-20T08:35:07Z",
"lastUpdateTime": "2024-02-20T08:35:07Z",
"message": "Deployment has minimum availability.",
"reason": "MinimumReplicasAvailable",
"status": "True",
"type": "Available"
},
{
"lastTransitionTime": "2024-02-20T08:35:03Z",
"lastUpdateTime": "2024-02-20T08:35:07Z",
"message": "ReplicaSet \"nginx-7854ff8877\" has successfully progressed.",
"reason": "NewReplicaSetAvailable",
"status": "True",
"type": "Progressing"
}
],
"observedGeneration": 1,
"readyReplicas": 1,
"replicas": 1,
"updatedReplicas": 1
}
}
],
"kind": "List",
"metadata": {
"resourceVersion": ""
}
}
When we run -o yaml, the output is displayed in yaml converted from json. If we were to use an external tool for this purpose, it would be yq.
# kubectl uses libraries in go that do this conversion, but what I want to make clear here is that it received a json
kubectl get deploy -o yaml
## Same output below
## Using external tools to convert
kubectl get deploy -o json | yq -p json
apiVersion: v1
items:
- apiVersion: apps/v1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
creationTimestamp: "2024-02-20T08:35:03Z"
generation: 1
labels:
app: nginx
name: nginx
namespace: default
resourceVersion: "1430016"
uid: e4ecc32a-7d98-48ff-8bce-ba8af52e27e2
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app: nginx
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
creationTimestamp: null
labels:
app: nginx
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
status:
availableReplicas: 1
conditions:
- lastTransitionTime: "2024-02-20T08:35:07Z"
lastUpdateTime: "2024-02-20T08:35:07Z"
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: "2024-02-20T08:35:03Z"
lastUpdateTime: "2024-02-20T08:35:07Z"
message: ReplicaSet "nginx-7854ff8877" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
kind: List
metadata:
resourceVersion: ""
# compare the describe output with the yaml output
kubectl describe deploy nginx
kubectl JSONPath
When we do describe, the output is formatted for easy reading and much information is hidden. Knowing this, what if we wanted to create a specific output instead of what we already have with -o wide.
If we're going to manipulate the output, we need to know the json received, so it's good to get familiar with -o json which is present in practically all commands.
Let's generate an output using jsonpath. We want a list with all the images used.
The expression used in jsonpath needs to be in '{}' and starts with . instead of $..
kubectl get pods -A -o=jsonpath='{.items[*].spec.containers[*].image}'
nginx registry.k8s.io/coredns/coredns:v1.11.1 registry.k8s.io/coredns/coredns:v1.11.1 registry.k8s.io/etcd:3.5.10-0 docker.io/kindest/kindnetd:v20240202-8f1494ea docker.io/kindest/kindnetd:v20240202-8f1494ea docker.io/kindest/kindnetd:v20240202-8f1494ea docker.io/kindest/kindnetd:v20240202-8f1494ea registry.k8s.io/kube-apiserver:v1.29.1 registry.k8s.io/kube-controller-manager:v1.29.1 registry.k8s.io/kube-proxy:v1.29.1 registry.k8s.io/kube-proxy:v1.29.1 registry.k8s.io/kube-proxy:v1.29.1 registry.k8s.io/kube-proxy:v1.29.1 registry.k8s.io/kube-scheduler:v1.29.1 docker.io/alpine:3.13 docker.io/alpine:3.13 kubernetesui/dashboard:v2.7.0 docker.io/kindest/local-path-provisioner:v20240202-8f1494ea%
The images came all in one line. That's not good, let's improve it.
If we were to think in logic, it would be like this.
FOR EACH item
PRINT ITEM DATA + NEW LINE
END EACH
And we can transcribe to:
- FOR EACH item
{range.items[*]}- PRINT ITEM DATA + NEW LINE
{.spec.containers[*].image}{"\n"}
- PRINT ITEM DATA + NEW LINE
- END EACH
{end}
Applying...
kubectl get pods -A -o=jsonpath='{range.items[*]}{.spec.containers[*].image}{"\n"}{end}'
nginx
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/coredns/coredns:v1.11.1
registry.k8s.io/etcd:3.5.10-0
docker.io/kindest/kindnetd:v20240202-8f1494ea
docker.io/kindest/kindnetd:v20240202-8f1494ea
docker.io/kindest/kindnetd:v20240202-8f1494ea
docker.io/kindest/kindnetd:v20240202-8f1494ea
registry.k8s.io/kube-apiserver:v1.29.1
registry.k8s.io/kube-controller-manager:v1.29.1
registry.k8s.io/kube-proxy:v1.29.1
registry.k8s.io/kube-proxy:v1.29.1
registry.k8s.io/kube-proxy:v1.29.1
registry.k8s.io/kube-proxy:v1.29.1
registry.k8s.io/kube-scheduler:v1.29.1
docker.io/alpine:3.13
docker.io/alpine:3.13
kubernetesui/dashboard:v2.7.0
docker.io/kindest/local-path-provisioner:v20240202-8f1494ea
We could add more things like the container name, a tab, and then the image.
kubectl get pods -A -o=jsonpath='{range.items[*]}{.spec.containers[*].name}{"\t"}{.spec.containers[*].image}{"\n"}{end}'
nginx nginx
coredns registry.k8s.io/coredns/coredns:v1.11.1
coredns registry.k8s.io/coredns/coredns:v1.11.1
etcd registry.k8s.io/etcd:3.5.10-0
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kube-apiserver registry.k8s.io/kube-apiserver:v1.29.1
kube-controller-manager registry.k8s.io/kube-controller-manager:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-scheduler registry.k8s.io/kube-scheduler:v1.29.1
shell docker.io/alpine:3.13
shell docker.io/alpine:3.13
kubernetes-dashboard kubernetesui/dashboard:v2.7.0
local-path-provisioner docker.io/kindest/local-path-provisioner:v20240202-8f1494ea
But this doesn't align properly, right?
So we could do this.
kubectl get pods -A -o=jsonpath='{range .items[*]}{.spec.containers[*].name}{"\t"}{.spec.containers[*].image}{"\n"}{end}' | column -t
nginx nginx
coredns registry.k8s.io/coredns/coredns:v1.11.1
coredns registry.k8s.io/coredns/coredns:v1.11.1
etcd registry.k8s.io/etcd:3.5.10-0
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kube-apiserver registry.k8s.io/kube-apiserver:v1.29.1
kube-controller-manager registry.k8s.io/kube-controller-manager:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-scheduler registry.k8s.io/kube-scheduler:v1.29.1
shell docker.io/alpine:3.13
shell docker.io/alpine:3.13
kubernetes-dashboard kubernetesui/dashboard:v2.7.0
local-path-provisioner docker.io/kindest/local-path-provisioner:v20240202-8f1494ea
And let's add column names to make it more presentable.
kubectl get pods -A -o=jsonpath='{"NAME\tIMAGE\n\n"}{range .items[*]}{.spec.containers[*].name}{"\t"}{.spec.containers[*].image}{"\n"}{end}' | column -t
NAME IMAGE
nginx nginx
coredns registry.k8s.io/coredns/coredns:v1.11.1
coredns registry.k8s.io/coredns/coredns:v1.11.1
etcd registry.k8s.io/etcd:3.5.10-0
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kindnet-cni docker.io/kindest/kindnetd:v20240202-8f1494ea
kube-apiserver registry.k8s.io/kube-apiserver:v1.29.1
kube-controller-manager registry.k8s.io/kube-controller-manager:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-proxy registry.k8s.io/kube-proxy:v1.29.1
kube-scheduler registry.k8s.io/kube-scheduler:v1.29.1
shell docker.io/alpine:3.13
shell docker.io/alpine:3.13
kubernetes-dashboard kubernetesui/dashboard:v2.7.0
local-path-provisioner docker.io/kindest/local-path-provisioner:v20240202-8f1494ea
See this other example.
kubectl get nodes -o=jsonpath='{"NAME\tCPU\tARCH\n"}{range .items[*]}{.metadata.name}{"\t"}{.status.capacity.cpu}{"\t"}{.status.nodeInfo.architecture}{"\n"}{end}' | column -t
NAME CPU ARCH
kind-cluster-control-plane 16 amd64
kind-cluster-worker 16 amd64
kind-cluster-worker2 16 amd64
kind-cluster-worker3 16 amd64
The more familiar you are with jsonpath, the more you'll be able to generate specific outputs.
But there are easier ways to do this.
kubectl Custom Columns
Kubectl does these formattings we did above for you using -o=custom-columns. Under the hood, custom-columns does jsonpath, but making life easier by focusing on the expression for each one that will go to each column.
The syntax is KUBECTL COMMAND -o=custom-columns=<COLUMNNAME>:<jsonpath>,<COLUMNNAME>:<jsonpath>,<COLUMNNAME>:<jsonpath>...
The command below is the same thing as kubectl get nodes -o=jsonpath='{"NAME\tCPU\tARCH\n"}{range .items[*]}{.metadata.name}{"\t"}{.status.capacity.cpu}{"\t"}{.status.nodeInfo.architecture}{"\n"}{end}' | column -t.
kubectl get nodes -o=custom-columns=NAME:.metadata.name,CPU:.status.capacity.cpu,ARCH:.status.nodeInfo.architecture
NAME CPU ARCH
kind-cluster-control-plane 16 amd64
kind-cluster-worker 16 amd64
kind-cluster-worker2 16 amd64
kind-cluster-worker3 16 amd64
Easier than that, only squared.
Sorting
To sort, we can use --sort-by and pass which jsonpath we'll use to sort. By default, it sorts by .metadata.name, but we can change it.
kubectl get nodes -o wide --sort-by='{.status.addresses[0].address}'
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
kind-cluster-control-plane Ready control-plane 11d v1.29.1 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-worker2 Ready <none> 11d v1.29.1 172.18.0.3 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-worker Ready <none> 11d v1.29.1 172.18.0.4 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-worker3 Ready <none> 11d v1.29.1 172.18.0.5 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
sort-by only sorts in ascending order, if we wanted descending we would need some Linux resources.
kubectl get nodes -o wide --sort-by='{.status.addresses[0].address}' --no-headers| tac
kind-cluster-worker3 Ready <none> 11d v1.29.1 172.18.0.5 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-worker Ready <none> 11d v1.29.1 172.18.0.4 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-worker2 Ready <none> 11d v1.29.1 172.18.0.3 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13
kind-cluster-control-plane Ready control-plane 11d v1.29.1 172.18.0.2 <none> Debian GNU/Linux 12 (bookworm) 6.5.0-17-generic containerd://1.7.13