Skip to main content

Kubernetes Releases

We don't need this for the exam, but it's worth understanding to comprehend the Kubernetes project as a whole.

Kubernetes is constantly evolving. What do we know so far about Kubernetes API versions?

  • When we install Kubernetes, we install a specific version of Kubernetes.
# In this case we have version 1.27.4
kubectl get nodes
NAME                       STATUS   ROLES                  AGE    VERSION
k3d-k3s-default-server-0   Ready    control-plane,master   2d8h   v1.27.4+k3s1
k3d-k3s-default-agent-1    Ready    <none>                 2d8h   v1.27.4+k3s1
k3d-k3s-default-agent-0    Ready    <none>                 2d8h   v1.27.4+k3s1

Kubernetes version is controlled through semantic versioning.

Alt text

The first major version was v1.0 in July 2015 but the first stable version was v1.13.0

Alt text

They usually release the alpha version with new features disabled and then the beta version with features enabled and create the release.

Alt text

We can find releases at [https://github.com/kubernetes/kubernetes/releases]

When we download one of these releases, what do we have?

wget https://github.com/kubernetes/kubernetes/archive/refs/tags/v1.28.5.zip
unzip v1.28.5.zip

ls -lha kubernetes-1.28.5/cmd
total 108K
drwxr-xr-x 26 david david 4.0K Dec 19 10:32 .
drwxr-xr-x 18 david david 4.0K Dec 19 10:32 ..
-rw-r--r--  1 david david  365 Dec 19 10:32 OWNERS
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 clicheck
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 cloud-controller-manager
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 dependencycheck
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 dependencyverifier
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 fieldnamedocscheck
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 gendocs
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 genkubedocs
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 genman
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 genswaggertypedocs
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 genutils
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 genyaml
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 importverifier
drwxr-xr-x  3 david david 4.0K Dec 19 10:32 kube-apiserver #<<<
drwxr-xr-x  4 david david 4.0K Dec 19 10:32 kube-controller-manager #<<<
drwxr-xr-x  3 david david 4.0K Dec 19 10:32 kube-proxy #<<<
drwxr-xr-x  3 david david 4.0K Dec 19 10:32 kube-scheduler #<<<
drwxr-xr-x  4 david david 4.0K Dec 19 10:32 kubeadm
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 kubectl
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 kubectl-convert
drwxr-xr-x  3 david david 4.0K Dec 19 10:32 kubelet #<<<
drwxr-xr-x  3 david david 4.0K Dec 19 10:32 kubemark
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 preferredimports
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 prune-junit-xml
drwxr-xr-x  2 david david 4.0K Dec 19 10:32 yamlfmt

We have all the code we need but let's pay attention to this.

Kubernetes is a project that encompasses other projects. Remember that cAdvisor is present in Kubelet? Remember that ETCD is used?

See that we need CNI, CoreDNS, ETCD, cAdvisor, etc

The kube-apiserver, controller-manager, kube-scheduler, kubelet, kube-proxy kubectl are at the SAME VERSION because they are Kubernetes projects, but other components are at different versions.

Other projects have their own repositories with their specific versions.

cat  kubernetes-1.28.5/build/dependencies.yaml
dependencies:
  # zeitgeist (https://github.com/kubernetes-sigs/zeitgeist) was inspired by
  # (and now replaces) the cmd/verifydependencies tool to verify external
  # dependencies across the repo.
  #
  # The zeitgeist dependencies.yaml file format is intended to be
  # backwards-compatible with the original tooling.
  #
  # In instances where the file format may change across versions, this meta
  # dependency check exists to ensure we're pinned to a known good version.
  #
  # ref: https://github.com/kubernetes/kubernetes/pull/98845
  - name: "zeitgeist"
    version: "v0.2.0"
    refPaths:
    - path: hack/verify-external-dependencies-version.sh
      match: sigs.k8s.io/zeitgeist@v.*

  # CNI plugins
  - name: "cni"
    version: 1.2.0
    refPaths:
    - path: cluster/gce/config-common.sh
      match: WINDOWS_CNI_VERSION=
    - path: cluster/gce/gci/configure.sh
      match: DEFAULT_CNI_VERSION=
    - path: test/e2e_node/remote/utils.go
      match: cniVersion[\t\n\f\r ]*=

  # CoreDNS
  - name: "coredns-kube-up"
    version: 1.10.1
    refPaths:
    - path: cluster/addons/dns/coredns/coredns.yaml.base
      match: registry.k8s.io/coredns
    - path: cluster/addons/dns/coredns/coredns.yaml.in
      match: registry.k8s.io/coredns
    - path: cluster/addons/dns/coredns/coredns.yaml.sed
      match: registry.k8s.io/coredns

  - name: "coredns-kubeadm"
    version: 1.10.1
    refPaths:
    - path: cmd/kubeadm/app/constants/constants.go
      match: CoreDNSVersion =

  # CRI Tools
  - name: "crictl"
    version: 1.27.0
    refPaths:
    - path: cluster/gce/windows/k8s-node-setup.psm1
      match: CRICTL_VERSION =
    - path: cluster/gce/gci/configure.sh
      match: DEFAULT_CRICTL_VERSION=

  # protoc
  - name: "protoc"
    version: 23.4
    refPaths:
    - path: hack/lib/protoc.sh
      match: PROTOC_VERSION=

  # etcd
  - name: "etcd"
    version: 3.5.9
    refPaths:
    - path: cluster/gce/manifests/etcd.manifest
      match: etcd_docker_tag|etcd_version
    - path: cluster/gce/upgrade-aliases.sh
      match: ETCD_IMAGE|ETCD_VERSION
    - path: cmd/kubeadm/app/constants/constants.go
      match: DefaultEtcdVersion =
    - path: hack/lib/etcd.sh
      match: ETCD_VERSION=
    - path: staging/src/k8s.io/sample-apiserver/artifacts/example/deployment.yaml
      match: gcr.io/etcd-development/etcd
    - path: test/e2e/framework/providers/gcp.go
      match: const etcdImage
    - path: test/utils/image/manifest.go
      match: configs\[Etcd\] = Config{list\.GcEtcdRegistry, "etcd", "\d+\.\d+.\d+(-(alpha|beta|rc).\d+)?(-\d+)?"}

  - name: "etcd-image"
    version: 3.5.9
    refPaths:
    - path: cluster/images/etcd/Makefile
      match: BUNDLED_ETCD_VERSIONS\?|LATEST_ETCD_VERSION\?
    - path: cluster/images/etcd/migrate/options.go

  - name: "node-problem-detector"
    version: 0.8.13
    refPaths:
    - path: test/e2e_node/image_list.go
      match: const defaultImage
    - path: test/kubemark//docs/kubernetes/certifications/cka/cluster-maintenance/resources/hollow-node_template.yaml
      match: registry.k8s.io/node-problem-detector/node-problem-detector
    - path: cluster/addons/node-problem-detector/npd.yaml
      match: registry.k8s.io/node-problem-detector/node-problem-detector
    # TODO(dims): Ensure newer versions get uploaded to
    # - https://console.cloud.google.com/storage/browser/gke-release/winnode/node-problem-detector
    # - https://gcsweb.k8s.io/gcs/kubernetes-release/node-problem-detector/
    # and then the following references get fixed.
    #
    # - path: cluster/gce/gci/configure.sh
    #  match: DEFAULT_NPD_VERSION=
    #- path: cluster/gce/windows/k8s-node-setup.psm1
    #  match: DEFAULT_NPD_VERSION

  # From https://github.com/etcd-io/etcd/blob/main/Makefile
  - name: "golang: etcd release version"
    version: 1.19.9 # https://github.com/etcd-io/etcd/blob/main/CHANGELOG/CHANGELOG-3.5.md
    refPaths:
    - path: cluster/images/etcd/Makefile
      match: 'GOLANG_VERSION := \d+.\d+(alpha|beta|rc)?\.?(\d+)?'

  # Golang
  - name: "golang: upstream version"
    version: 1.20.12
    refPaths:
    - path: .go-version
    - path: build/build-image/cross/VERSION
    - path: staging/publishing/rules.yaml
      match: 'default-go-version\: \d+.\d+(alpha|beta|rc)?\.?(\d+)?'
    - path: test/images/Makefile
      match: GOLANG_VERSION=\d+.\d+(alpha|beta|rc)?\.?\d+

  # Golang pre-releases are denoted as `1.y<pre-release stage>`
  # Example: go1.16rc1
  #
  # This entry is a stub of the major version to allow dependency checks to
  # pass when building Kubernetes using a pre-release of Golang.
  - name: "golang: 1.<major>"
    version: 1.20
    refPaths:
    - path: build/build-image/cross/VERSION
    - path: hack/lib/golang.sh
      match: minimum_go_version=go([0-9]+\.[0-9]+)

  - name: "registry.k8s.io/kube-cross: dependents"
    version: v1.28.0-go1.20.12-bullseye.0
    refPaths:
    - path: build/build-image/cross/VERSION

  # Base images
  - name: "registry.k8s.io/debian-base: dependents"
    version: bookworm-v1.0.0
    refPaths:
    - path: cluster/images/etcd/Makefile
      match: BASEIMAGE\?\=registry\.k8s\.io\/build-image\/debian-base:[a-zA-Z]+\-v((([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)
    - path: cluster/images/etcd/Makefile
      match: BASEIMAGE\?\=registry\.k8s\.io\/build-image\/debian-base-arm:[a-zA-Z]+\-v((([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)
    - path: cluster/images/etcd/Makefile
      match: BASEIMAGE\?\=registry\.k8s\.io\/build-image\/debian-base-arm64:[a-zA-Z]+\-v((([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)
    - path: cluster/images/etcd/Makefile
      match: BASEIMAGE\?\=registry\.k8s\.io\/build-image\/debian-base-ppc64le:[a-zA-Z]+\-v((([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)
    - path: cluster/images/etcd/Makefile
      match: BASEIMAGE\?\=registry\.k8s\.io\/build-image\/debian-base-s390x:[a-zA-Z]+\-v((([0-9]+)\.([0-9]+)\.([0-9]+)(?:-([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?)
    - path: test/conformance/image/Makefile
      match: BASE_IMAGE_VERSION\?=

  - name: "registry.k8s.io/distroless-iptables: dependents"
    version: v0.2.8
    refPaths:
    - path: build/common.sh
      match: __default_distroless_iptables_version=
    - path: test/utils/image/manifest.go
      match: configs\[DistrolessIptables\] = Config{list\.BuildImageRegistry, "distroless-iptables", "v([0-9]+)\.([0-9]+)\.([0-9]+)"}

  - name: "registry.k8s.io/go-runner: dependents"
    version: v2.3.1-go1.20.12-bullseye.0
    refPaths:
    - path: build/common.sh
      match: __default_go_runner_version=

  - name: "registry.k8s.io/pause"
    version: 3.9
    refPaths:
    - path: build/pause/Makefile
      match: TAG\s*\?=

  - name: "registry.k8s.io/pause: dependents"
    version: 3.9
    refPaths:
    - path: cluster/gce/config-common.sh
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: cluster/gce/gci/configure-helper.sh
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: cluster/gce/windows/smoke-test.sh
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: cmd/kubeadm/app/constants/constants.go
      match: PauseVersion\s+=
    - path: cmd/kubeadm/app/util/template_test.go
      match: validTmpl\s+=
    - path: cmd/kubeadm/app/util/template_test.go
      match: validTmplOut\s+=
    - path: cmd/kubeadm/app/util/template_test.go
      match: doNothing\s+=
    - path: cmd/kubelet/app/options/container_runtime.go
      match: defaultPodSandboxImageVersion\s+=
    - path: hack/testdata/pod-with-precision.json
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: staging/src/k8s.io/kubectl/testdata/set/multi-resource-yaml.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: staging/src/k8s.io/kubectl/testdata/set/namespaced-resource.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/cmd/core.sh
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/fixtures/pkg/kubectl/cmd/set/multi-resource-yaml.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/fixtures/pkg/kubectl/cmd/set/namespaced-resource.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/benchmark-controller.json
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-default.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-node-affinity.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-pod-affinity.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-pod-anti-affinity.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-preferred-pod-affinity.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-preferred-pod-anti-affinity.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-preferred-topology-spreading.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-secret-volume.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/integration/scheduler_perf/config/pod-with-topology-spreading.yaml
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/utils/runners.go
      match: registry.k8s.io\/pause:\d+\.\d+
    - path: test/utils/image/manifest.go
      match: configs\[Pause\] = Config{list\.GcRegistry, "pause", "\d+\.\d+(.\d+)?"}

  - name: "registry.k8s.io/build-image/setcap: dependents"
    version: bookworm-v1.0.0
    refPaths:
    - path: build/common.sh
      match: __default_setcap_version=

  # cadvisor
  - name: "gcr.io/cadvisor/cadvisor: dependents"
    version: "v0.47.2"
    refPaths:
    - path: test/e2e_node/resource_collector.go
      match: gcr.io\/cadvisor\/cadvisor:v\d+\.\d+\.\d+
    - path: test/e2e_node/image_list.go
      mathc: gcr.io\/cadvisor\/cadvisor:v\d+\.\d+\.\d+

  # GCB docker gcloud image
  - name: "gcb-docker-gcloud: dependents"
    version: v20221007-69e0da97ef
    refPaths:
    - path: build/pause/cloudbuild.yaml
      match: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud
    - path: cluster/images/etcd/cloudbuild.yaml
      match: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud
    - path: test/images/cloudbuild.yaml
      match: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud

Kubernetes versions deprecate some things and create others. We need to be aware of this as they may require adjustments to manifests.