Skip to main content

Cluster Monitoring

Kubernetes doesn't come with an integrated monitoring solution to collect metrics from nodes and pods, but we have many open-source solutions we can use.

  • Metrics Server (formerly Heapster)
  • Prometheus
  • Elastic Stack

And paid ones like:

  • Datadog
  • Dynatrace
  • NewRelic

Let's focus on Metrics Server. We can run one metrics server per Kubernetes cluster.

One of the Kubelet's subcomponents is cAdvisor, which has the function of exposing pod metrics through a Kubelet API so that Metrics Server can collect them.

Metrics Server collects data from Kubelet using the Kubelet API and stores it in memory. Metrics Server doesn't keep a history like Prometheus and other solutions.

To deploy Metrics Server, we'll use GitHub Metrics Server or Helm Metrics Server

helm repo add metrics-server https://kubernetes-sigs.github.io/metrics-server/
helm show values metrics-server/metrics-server > values.yaml

Metrics Server applied directly to the Kind cluster sometimes has issues with certificates. Add the last line.

defaultArgs:
  - --cert-dir=/tmp
  - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
  - --kubelet-use-node-status-port
  - --metric-resolution=15s
  - --kubelet-insecure-tls # For kind clusters

And execute the command to install passing the values we just modified.

helm upgrade --install metrics-server metrics-server/metrics-server --values values.yaml

Once installed, we can use the commands.

kubectl top nodes
NAME                        CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
cka-cluster-control-plane   165m         1%     1027Mi          1%
cka-cluster-worker          41m          0%     478Mi           0%
cka-cluster-worker2         38m          0%     475Mi           0%
cka-cluster-worker3         39m          0%     393Mi           0%

kubectl top pods -A
NAMESPACE            NAME                                                CPU(cores)   MEMORY(bytes)
default              metrics-server-699ffcdc79-njfq8                     5m           20Mi
default              myapp-rs-4qj5l                                      0m           23Mi
default              myapp-rs-jpct9                                      0m           23Mi
default              myapp-rs-wvd7l                                      0m           23Mi
istio-system         istiod-6d8c55dd6c-47s57                             4m           107Mi
kube-system          coredns-76f75df574-7d75t                            3m           15Mi
kube-system          coredns-76f75df574-99kmh                            3m           58Mi
kube-system          etcd-cka-cluster-control-plane                      26m          97Mi
kube-system          kube-apiserver-cka-cluster-control-plane            68m          387Mi
kube-system          kube-controller-manager-cka-cluster-control-plane   22m          113Mi
kube-system          kube-proxy-6gk2j                                    8m           70Mi
kube-system          kube-proxy-bgpzr                                    5m           70Mi
kube-system          kube-proxy-l8xxj                                    7m           70Mi
kube-system          kube-proxy-vg8mb                                    4m           71Mi
kube-system          kube-scheduler-cka-cluster-control-plane            6m           65Mi
kube-system          weave-net-29ffl                                     2m           118Mi
kube-system          weave-net-4q8kr                                     2m           117Mi
kube-system          weave-net-8zxbw                                     1m           117Mi
kube-system          weave-net-phbz8                                     2m           117Mi
local-path-storage   local-path-provisioner-7577fdbbfb-7t9pc             1m           31Mi