[Kubernetes] 11. Node Selector, Node Affinity, Taints & Tolerations

본문 바로가기

Notice

Recent Posts

Recent Comments

Link

Tags more

Archives

Today

Total

관리 메뉴

나의 공부기록

[Kubernetes] 11. Node Selector, Node Affinity, Taints & Tolerations 본문

CS/Kubernetes

[Kubernetes] 11. Node Selector, Node Affinity, Taints & Tolerations

나의 개발자 2025. 4. 22. 11:25

Node Selector

노드에 Key:Value 형태의 label을 붙여놓고 Pod를 띄울 때, 특정 라벨갑을 충족하는 노드를 선택하는 기능

1. 노드 이름 확인

2. 노드에 label 지정

임의로 정하는 값 ➡️ gpu(Key)=false(Value)

root@master-250410:~# kubectl label node worker1-250410 gpu=false
node/worker1-250410 labeled
root@master-250410:~# kubectl label node worker2-250410 gpu=true
node/worker2-250410 labeled

3. 노드의 label 확인

root@master-250410:~# kubectl get nodes --show-labels 
NAME             STATUS   ROLES           AGE     VERSION    LABELS
master-250410    Ready    control-plane   3d23h   v1.30.11   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=master-250410,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=
worker1-250410   Ready    <none>          3d23h   v1.30.11   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,gpu=false,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker1-250410,kubernetes.io/os=linux
worker2-250410   Ready    <none>          3d23h   v1.30.11   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,gpu=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=worker2-250410,kubernetes.io/os=linux

gpu=false/true 확인 가능

4. label이 gpu=true인 node에 pod 설치

root@master-250410:~/mani# cp dep.yml selector.yml
root@master-250410:~/mani# vi selector.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: selector-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: my-con
        image: 61.254.18.30:5000/ipnginx
      nodeSelector:
        gpu: "true"

5. Pod 생성

root@master-250410:~/mani# kubectl apply -f selector.yml 
deployment.apps/selector-dep created

6. 특정 Node에만 Pod가 생성되는 것을 확인 가능

root@master-250410:~/mani# kubectl get pod -o wide
NAME                            READY   STATUS    RESTARTS      AGE     IP             NODE             NOMINATED NODE   READINESS GATES
live-pod                        1/1     Running   2 (12m ago)   3d23h   10.244.1.94    worker1-250410   <none>           <none>
selector-dep-7c7bf7459f-ccwhl   1/1     Running   0             19s     10.244.2.101   worker2-250410   <none>           <none>
selector-dep-7c7bf7459f-crhmp   1/1     Running   0             19s     10.244.2.100   worker2-250410   <none>           <none>

➕ Pod 개수를 늘려서 특정 Node에만 생성되는 것인지 정확히 검증

반영 후, node 확인

root@master-250410:~/mani# kubectl apply -f selector.yml 
deployment.apps/selector-dep configured

root@master-250410:~/mani# kubectl get pod -o wide
NAME                            READY   STATUS    RESTARTS      AGE     IP             NODE             NOMINATED NODE   READINESS GATES
live-pod                        1/1     Running   2 (12m ago)   3d23h   10.244.1.94    worker1-250410   <none>           <none>
selector-dep-7c7bf7459f-6g94m   1/1     Running   0             8s      10.244.2.103   worker2-250410   <none>           <none>
selector-dep-7c7bf7459f-ccwhl   1/1     Running   0             57s     10.244.2.101   worker2-250410   <none>           <none>
selector-dep-7c7bf7459f-crhmp   1/1     Running   0             57s     10.244.2.100   worker2-250410   <none>           <none>
selector-dep-7c7bf7459f-qg9wg   1/1     Running   0             8s      10.244.2.102   worker2-250410   <none>           <none>
selector-dep-7c7bf7459f-r7zz2   1/1     Running   0             8s      10.244.2.104   worker2-250410   <none>           <none>

Node Affinity

끌림 Node Selector와 비슷하지만, 좀 더 세부적인 조건을 걸어줄 수 있음

1. preferred affinity 정의

root@master-250410:~/mani/selector# vi affinity.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-dep
spec:
  replicas: 4
  selector:
    matchLabels:
      app: my-httpd
  template:
    metadata:
      labels:
        app: my-httpd
    spec:
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 1
              preference:
                matchExpressions:
                  - key: gpu
                    operator: In
                    values:
                      - "false"
      containers:
        - name: httpd-pod-con
          image: 61.254.18.30:5000/nginx

2. preferred affinity 생성

root@master-250410:~/mani/selector# kubectl apply -f affinity.yml 
deployment.apps/my-dep created

3. preferred affinity 결과 확인

gpu="false"인 worker1에 배치된 것을 확인 가능

root@master-250410:~/mani/selector# kubectl get pod -o wide
NAME                      READY   STATUS    RESTARTS      AGE     IP             NODE             NOMINATED NODE   READINESS GATES
live-pod                  1/1     Running   2 (45m ago)   3d23h   10.244.1.94    worker1-250410   <none>           <none>
my-dep-68bddbfd8c-87wtt   1/1     Running   0             26s     10.244.1.98    worker1-250410   <none>           <none>
my-dep-68bddbfd8c-fccb9   1/1     Running   0             26s     10.244.1.100   worker1-250410   <none>           <none>
my-dep-68bddbfd8c-n9mv2   1/1     Running   0             26s     10.244.1.97    worker1-250410   <none>           <none>
my-dep-68bddbfd8c-t6l4r   1/1     Running   0             26s     10.244.1.99    worker1-250410   <none>           <none>

4. required affinity 정의

requiredDuringSchedulingIgnoredDuringExecution: 으로 대체
➡️prefer : 되도록이면, required : 반드시

root@master-250410:~/mani/selector# vi affinity-required.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-dep
spec:
  replicas: 4
  selector:
    matchLabels:
      app: my-httpd
  template:
    metadata:
      labels:
        app: my-httpd
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: gpu
                    operator: In
                    values:
                      - "true"
      containers:
        - name: httpd-pod-con
          image: 61.254.18.30:5000/nginx

5. required affinity 생성

root@master-250410:~/mani/selector# kubectl apply -f affinity-required.yml 
deployment.apps/my-dep created

6. required affinity 생성 확인

root@master-250410:~/mani/selector# kubectl get pod -o wide
NAME                     READY   STATUS    RESTARTS      AGE   IP             NODE             NOMINATED NODE   READINESS GATES
live-pod                 1/1     Running   2 (53m ago)   4d    10.244.1.94    worker1-250410   <none>           <none>
my-dep-d78c57679-gmcpn   1/1     Running   0             4s    10.244.2.105   worker2-250410   <none>           <none>
my-dep-d78c57679-jjppt   1/1     Running   0             4s    10.244.2.106   worker2-250410   <none>           <none>
my-dep-d78c57679-jntwp   1/1     Running   0             4s    10.244.2.108   worker2-250410   <none>           <none>
my-dep-d78c57679-tgsgb   1/1     Running   0             4s    10.244.2.107   worker2-250410   <none>           <none>

실습 - 01

문제

worker1은 disktype이 ssd이며, worker2는 disktype이 hdd일 때,
간단한 deployment 파일을 작성하여 ssd-dep는 ssd가 설치된 노드에, hdd-dep는 hdd가 설치된 노드에 배치해 보세요.

nodeSelector와 nodeAffinity를 둘 다 사용해 보세요.

풀이

1. node에 label 설정

root@master-250410:~/mani/selector# kubectl label node worker1-250410 disktype=ssd
node/worker1-250410 labeled
root@master-250410:~/mani/selector# kubectl label node worker2-250410 disktype=hdd
node/worker2-250410 labeled

2. node에 label 설정 확인

3. ssd-deploy 정의

root@master-250410:~/mani/selector/exam-1# vi ssd-dep.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ssd-dep
spec:
  replicas: 5
  selector:
    matchLabels:
      app: ssd-nginx
  template:
    metadata:
      labels:
        app: ssd-nginx
    spec:
      containers:
      - name: ssd-nginx-con
        image: 61.254.18.30:5000/ipnginx
      nodeSelector:
        disktype: "ssd"

4. hdd-deploy 정의

root@master-250410:~/mani/selector/exam-1# vi hdd-dep.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hdd-dep
spec:
  replicas: 4
  selector:
    matchLabels:
      app: hdd-httpd
  template:
    metadata:
      labels:
        app: hdd-httpd
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: disktype
                    operator: In
                    values:
                      - "hdd"
      containers:
        - name: hdd-httpd-pod-con
          image: 61.254.18.30:5000/nginx

5. deploy 생성

root@master-250410:~/mani/selector/exam-1# kubectl apply -f ssd-dep.yml 
deployment.apps/ssd-dep created
root@master-250410:~/mani/selector/exam-1# kubectl apply -f hdd-dep.yml 
deployment.apps/hdd-dep created

6. pod가 생성된 node 확인

ssd-dep은 worker1에 hdd-dep은 worker2에 생성됨을 확인 가능

root@master-250410:~/mani/selector/exam-1# kubectl get pod -o wide
NAME                       READY   STATUS    RESTARTS      AGE     IP             NODE             NOMINATED NODE   READINESS GATES
hdd-dep-7596f8b94c-np9tm   1/1     Running   0             11s     10.244.2.110   worker2-250410   <none>           <none>
hdd-dep-7596f8b94c-t6c4t   1/1     Running   0             11s     10.244.2.111   worker2-250410   <none>           <none>
hdd-dep-7596f8b94c-v4vdz   1/1     Running   0             11s     10.244.2.112   worker2-250410   <none>           <none>
hdd-dep-7596f8b94c-vshln   1/1     Running   0             11s     10.244.2.109   worker2-250410   <none>           <none>
live-pod                   1/1     Running   2 (59m ago)   4d      10.244.1.94    worker1-250410   <none>           <none>
ssd-dep-595bc67b9-7w27v    1/1     Running   0             16s     10.244.1.111   worker1-250410   <none>           <none>
ssd-dep-595bc67b9-bvgbl    1/1     Running   0             16s     10.244.1.109   worker1-250410   <none>           <none>
ssd-dep-595bc67b9-mwpc9    1/1     Running   0             16s     10.244.1.110   worker1-250410   <none>           <none>
ssd-dep-595bc67b9-rhcgt    1/1     Running   0             16s     10.244.1.112   worker1-250410   <none>           <none>

👉 Node Selector와 Node Affinity가 Pod 입장에서 노드를 선택하는 개념

➕ label 해제

root@master-250410:~/mani/selector/taints_tolerations# kubectl label node worker1-250410 gpu-
node/worker1-250410 unlabeled
root@master-250410:~/mani/selector/taints_tolerations# kubectl label node worker2-250410 gpu-
node/worker2-250410 unlabeled

Taints & Tolerations

Taints : Node 입장에서 Pod를 기피하는 개념
Tolerations : taints를 통해 Pod를 배치하지 않지만, 특정 조건을 만족하는 Pod는 허용

1. Taints 설정

worker1 : team=dev라는 toleration이 없다면, Pod를 배치❌
worker2 : team=ops라는 toleration이 없다면, Pod를 배치❌

root@master-250410:~/mani/selector# kubectl taint node worker1-250410 team=dev:NoSchedule
node/worker1-250410 tainted

root@master-250410:~/mani/selector# kubectl taint node worker2-250410 team=ops:NoSchedule
node/worker2-250410 tainted

2. Deployment 정의

root@master-250410:~/mani/selector/taints_tolerations# vi dep.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: my-con
        image: 61.254.18.30:5000/nginx

3. Deployment 생성

root@master-250410:~/mani/selector/taints_tolerations# kubectl apply -f dep.yml 
deployment.apps/my-dep created

4. 결과 확인

my-dep이 "pending"상태임을 확인 가능

root@master-250410:~/mani/selector/taints_tolerations# kubectl get pod
NAME                     READY   STATUS    RESTARTS      AGE
live-pod                 1/1     Running   2 (91m ago)   4d
my-dep-b4d5bcfb7-9fhpz   0/1     Pending   0             5s
my-dep-b4d5bcfb7-mlqgv   0/1     Pending   0             5s

5. 문제 파악

root@master-250410:~/mani/selector/taints_tolerations# kubectl describe pod my-dep-b4d5bcfb7-9fhpz

👉 taint에 의해 pod를 배치할 수 있는 노드가 없는 걸 확인

👉 Deployment에 team=dev 또는 ops를 설정하지 않았기 때문에, node에 설치가 되지 않음

➡️ 해결 방법 : toleration을 통해 team=dep or ops 설정을 해줘야 함

6. toleration 설정

toleration에 해당하는 node에 Pod를 설치 허용

root@master-250410:~/mani/selector/taints_tolerations# cp dep.yml tol-dep.yml

root@master-250410:~/mani/selector/taints_tolerations# vi tol-dep.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: my-con
        image: 61.254.18.30:5000/nginx
      tolerations:
      - key: "team"
        operator: "Equal"
        value: "dev"
        effect: "NoSchedule"

7. Deployment 생성

root@master-250410:~/mani/selector/taints_tolerations# kubectl apply -f tol-dep.yml 
deployment.apps/my-dep configured

8. 결과 확인

➕ taints 해제

root@master-250410:~/mani/selector/taints_tolerations# kubectl taint node worker2-250410 team=ops:NoSchedule-
node/worker2-250410 untainted
root@master-250410:~/mani/selector/taints_tolerations# kubectl taint node worker1-250410 team=dev:NoSchedule-
node/worker1-250410 untainted

실습 - 02

문제

위의 tolerations을 수정해서 worker2에 Pod를 설치해 보세요.

풀이

1. Deployment 정의

root@master-250410:~/mani/selector/taints_tolerations# cp tol-dep.yml tol-dep-2.yml
root@master-250410:~/mani/selector/taints_tolerations# vi tol-dep-2.yml

root@master-250410:~/mani/selector/taints_tolerations# vi tol-dep-2.yml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: worker2-dep
spec:
  replicas: 2
  selector:
    matchLabels:
      app: worker2-nginx
  template:
    metadata:
      labels:
        app: worker2-nginx
    spec:
      containers:
      - name: worker2-my-con
        image: 61.254.18.30:5000/nginx
      tolerations:
      - key: "team"
        operator: "Equal"
        value: "ops"
        effect: "NoSchedule"

2. Deployment 생성

root@master-250410:~/mani/selector/taints_tolerations# kubectl apply -f tol-dep-2.yml 
deployment.apps/worker2-dep created

3. 결과 확인

worker2에 설치된 것을 확인 가능

'CS > Kubernetes' 카테고리의 다른 글

[Kubernetes] 13. Helm Chart 생성 (0)	2025.04.23
[Kubernetes] 12. Helm(헬름) & Chart(차트) (2)	2025.04.22
[Kubernetes] 10. HPA(Horizontal Pod AutoScaler) & Control Plane 3중화(HAProxy) (1)	2025.04.19
[Kubernetes] 08. Container Health Check(컨테이너 헬스체크) - livenessProbe, readinessProbe (1)	2025.04.17
[Kubernetes] 07. 자율과제 - Dynamic Provisioner (0)	2025.04.17

'CS/Kubernetes' Related Articles

more

티스토리툴바