Manual Scheduling

By default, all Nodes are eligible by the kube-scheduler if we don't define a nodeName in the spec.

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 8080
  #nodeName: <<<<<

If there is no Scheduler (like kube-scheduler) to define which Node to choose, the Pod would remain in Pending state.

If there is no Scheduler and a specific Node is defined for the Pod, it will start on the defined Node if it exists and has sufficient resources.

Let's start the Pod with a nodeName that doesn't exist.

kubectl get nodes
NAME                       STATUS   ROLES                  AGE   VERSION
k3d-k3d-cluster-agent-1    Ready    <none>                 69d   v1.27.4+k3s1
k3d-k3d-cluster-agent-2    Ready    <none>                 69d   v1.27.4+k3s1
k3d-k3d-cluster-agent-0    Ready    <none>                 69d   v1.27.4+k3s1
k3d-k3d-cluster-server-0   Ready    control-plane,master   69d   v1.27.4+k3s1

cat <<EOF > pod-node-name.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 8080
  nodeName: node01
EOF

# Trying to start a Pod on a Node that doesn't exist in the cluster
kubectl apply -f pod-node-name.yaml
pod/nginx created

kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
nginx   0/1     Pending   0          4s

Even with an existing Scheduler, it didn't find the selected Node because we don't have node01, so it stayed in pending and then disappeared.

Changing the nodeName of a Pod is not allowed by Kubernetes, as it's not possible to move a Pod from one Node to another. It needs to be recreated, meaning you must delete it first and then create it again.

To do this with kubectl apply, we can use kubectl replace --force

# Attempting to change by modifying the file from node01 to node02 which also doesn't exist
cat <<EOF > pod-node-name.yaml
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 8080
  nodeName: node02
EOF

kubectl apply -f ./files/pod-node-name.yaml
pod/nginx created
kubectl apply -f ./files/pod-node-name.yaml
The Pod "nginx" is invalid: spec: Forbidden: pod updates may not change fields other than `spec.containers[*].image`,`spec.initContainers[*].image`,`spec.activeDeadlineSeconds`,`spec.tolerations` (only additions to existing tolerations),`spec.terminationGracePeriodSeconds` (allow it to be set to 1 if it was previously negative)
  core.PodSpec{
        ... // 9 identical fields
        ServiceAccountName:           "default",
        AutomountServiceAccountToken: nil,
-       NodeName:                     "node01",
+       NodeName:                     "node02",
        SecurityContext:              &{},
        ImagePullSecrets:             nil,
        ... // 19 identical fields
  }

kubectl replace --force -f ./files/pod-node-name.yaml
pod "nginx" deleted
pod/nginx replaced

# The watch command keeps monitoring, since we don't have node02 it terminated and didn't keep trying to start because it's just a Pod without a ReplicaSet to control it
kubectl get pods -o wide --watch
NAME    READY   STATUS    RESTARTS   AGE   IP       NODE     NOMINATED NODE   READINESS GATES
nginx   0/1     Pending   0          12s   <none>   node02   <none>           <none>
nginx   0/1     Failed    0          49s   <none>   node02   <none>           <none>
nginx   0/1     Terminating   0          49s   <none>   node02   <none>           <none>
nginx   0/1     Terminating   0          49s   <none>   node02   <none>           <none>

Of course, if we set a Node that exists, everything would work accordingly.