k8s-Pod排程
2022-08-29 06:00:52
Deployment全自動排程
NodeSelector定向排程
NodeAffinity親和性
PodAffinity-Pod親和性與互斥性
汙點和容忍度
DaemonSet
Job
CronJob
Deployment升級策略
Deployment回滾
Deployment暫停和恢復
DeamonSet的更新策略
Pod的擴縮容
Deployment全自動排程
宣告Deployment後，通過篩選標籤對匹配的pod做副本控制。Deployment會建立一個RS和replicas個pod。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3 #副本數
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
刪除pod後會自動建立新的。
刪除Deployment後會關聯刪除RS和Pod,無法單獨刪除RS。
NodeSelector定向排程
從排程策略上來說，這三個pod由系統全自動完成排程，他們各自執行在哪個node節點，完全由master的scheduler經過一系列演演算法計算得出，使用者無法干預排程過程和結果。1個被排程到了node1兩個被排程到了node2。
在實際情況下，也可能需要將pod排程到指定節點上，可以通過給node打標籤，和pod的nodeselector屬性匹配，來達到定向排程的目的。
給node新增標籤：kubectl label nodes nodename key=value
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
nodeSelector:
zone: north
pod全部排程到了node01上。注意如果pod使用了nodeselector但是沒有匹配的node，則pod不會被建立。
刪除node標籤：kubectl label nodes nodename key-
建立deployment後發現pod一直在被建立中。
通過檢視其中一個pod發現錯誤資訊
給node新增上對應標籤後，pod又自動建立了。
在pod建立後，刪除node標籤，pod正常執行。
但是如果pod被刪除後，RS重新建立pod會失敗。
如果同時給多個node打上匹配的標籤，則也會排程到不同的pod上。
NodeAffinity親和性
requiredDuringSchedulingIgnoredDuringExecution
表示pod必須部署到滿足條件的節點上，如果沒有滿足條件的節點，就不停重試。其中IgnoreDuringExecution表示pod部署之後執行的時候，如果節點標籤發生了變化，不再滿足pod指定的條件，pod也會繼續執行。
kubectl label node k8s-node02 disktype=ssd
kubectl get node --show-labels
在k8s02上打上disktype=ssd標籤，然後pod選擇策略指定label中包含 disktype=ssd,三個節點都排程到了k8s-node02
apiVersion: apps/v1
kind: Deployment
metadata:
name: d3
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: disktype
operator: In # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- ssd
如果operator修改為 NotIn則3個pod都會排程到k8s-node01上。
preferredDuringSchedulingIgnoredDuringExecution
表示優先部署到滿足條件的節點上，如果沒有滿足條件的節點，就忽略這些條件，按照正常邏輯部署。
刪除掉node02上的disktype標籤後，沒有滿足affiinity的但是還是被排程到了。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d4
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: disktype
operator: NotIn # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- ssd
權重的值在1-100之間，看這個結果，越大權重越大。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d5
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 1
preference:
matchExpressions:
- key: disktype
operator: In # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- ssd1
- weight: 100
preference:
matchExpressions:
- key: disktype
operator: In # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- ssd2
如果你同時指定了 nodeSelector 和 nodeAffinity，兩者 必須都要滿足， 才能將 Pod 排程到候選節點上。
如果你指定了多個與 nodeAffinity 型別關聯的 nodeSelectorTerms， 只要其中一個 nodeSelectorTerms 滿足的話，Pod 就可以被排程到節點上。
如果你指定了多個與同一 nodeSelectorTerms 關聯的 matchExpressions， 則只有當所有 matchExpressions 都滿足時 Pod 才可以被排程到節點上。
PodAffinity-Pod親和性與互斥性
建立一個測試節點，一個pod被分到了node01。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d6
spec:
selector:
matchLabels:
security: s1 #Deployment會控制label相同的pod
replicas: 1
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
security: s1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
podAffinity
親和性，會盡力與具有指定標籤的pod排程到同一個node上。下邊例子，app:v1與security：s1親和
apiVersion: apps/v1
kind: Deployment
metadata:
name: d7
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
podAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: security
operator: In # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- s1
topologyKey: kubernetes.io/hostname
podAntiAffinity
互斥性，對具有指定標籤的pod有互斥性，拒絕放到一個node上。下邊例子，app:v2與security：s1互斥
apiVersion: apps/v1
kind: Deployment
metadata:
name: d8
spec:
selector:
matchLabels:
app: v2 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v2
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: security
operator: In # In: label的值在某個列表中 NotIn：label的值不在某個列表中 Exists：某個label存在 DoesNotExist：某個label不存在 Gt：label的值大於某個值（字串比較） Lt：label的值小於某個值（字串比較）
values:
- s1
topologyKey: kubernetes.io/hostname
汙點和容忍度
給node01新增汙點 taint0=taint00:NoSchedule 給node02新增汙點 taint2-taint22:NoSchedule
kubectl taint nodes k8s-node01 taint0=taint00:NoSchedule
kubectl taint nodes k8s-node02 taint2=taint22:NoSchedule
[root@k8s-master01 ~]# kubectl describe node k8s-node01
Name: k8s-node01
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
disktype=ssd1
kubernetes.io/arch=amd64
kubernetes.io/hostname=k8s-node01
kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"16:30:be:e9:46:bb"}
flannel.alpha.coreos.com/backend-type: vxlan
flannel.alpha.coreos.com/kube-subnet-manager: true
flannel.alpha.coreos.com/public-ip: 192.168.180.130
kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Sat, 20 Aug 2022 22:07:32 +0800
Taints: taint0=taint00:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Thu, 25 Aug 2022 15:38:42 +0800 Thu, 25 Aug 2022 15:38:42 +0800 FlannelIsUp Flannel is running on this node
MemoryPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:32 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 26 Aug 2022 10:52:04 +0800 Sat, 20 Aug 2022 22:07:42 +0800 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.180.130
Hostname: k8s-node01
Capacity:
cpu: 1
ephemeral-storage: 17394Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 995676Ki
pods: 110
Allocatable:
cpu: 1
ephemeral-storage: 16415037823
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 893276Ki
pods: 110
System Info:
Machine ID: f914368afc1643a4a6fda29f9b7d4da1
System UUID: ABE14D56-5112-0934-DFE1-70325811365B
Boot ID: c064f3ee-6080-40a7-8273-ee9b605a0adb
Kernel Version: 3.10.0-1160.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.17
Kubelet Version: v1.15.1
Kube-Proxy Version: v1.15.1
PodCIDR: 10.244.1.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kube-flannel-ds-amd64-jsn6j 100m (10%) 100m (10%) 50Mi (5%) 50Mi (5%) 5d12h
kube-system kube-proxy-l89l8 0 (0%) 0 (0%) 0 (0%) 0 (0%) 5d12h
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (10%) 100m (10%)
memory 50Mi (5%) 50Mi (5%)
ephemeral-storage 0 (0%) 0 (0%)
Events: <none>
[root@k8s-master01 ~]# kubectl describe node k8s-node02
Name: k8s-node02
Roles: <none>
Labels: beta.kubernetes.io/arch=amd64
beta.kubernetes.io/os=linux
disktype=ssd2
kubernetes.io/arch=amd64
kubernetes.io/hostname=k8s-node02
kubernetes.io/os=linux
Annotations: flannel.alpha.coreos.com/backend-data: {"VtepMAC":"b2:58:6a:57:86:15"}
flannel.alpha.coreos.com/backend-type: vxlan
flannel.alpha.coreos.com/kube-subnet-manager: true
flannel.alpha.coreos.com/public-ip: 192.168.180.131
kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
node.alpha.kubernetes.io/ttl: 0
volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp: Wed, 24 Aug 2022 20:50:34 +0800
Taints: taint2=taint22:NoSchedule
Unschedulable: false
Conditions:
Type Status LastHeartbeatTime LastTransitionTime Reason Message
---- ------ ----------------- ------------------ ------ -------
NetworkUnavailable False Thu, 25 Aug 2022 15:38:43 +0800 Thu, 25 Aug 2022 15:38:43 +0800 FlannelIsUp Flannel is running on this node
MemoryPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available
DiskPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure
PIDPressure False Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:34 +0800 KubeletHasSufficientPID kubelet has sufficient PID available
Ready True Fri, 26 Aug 2022 10:55:57 +0800 Wed, 24 Aug 2022 20:50:44 +0800 KubeletReady kubelet is posting ready status
Addresses:
InternalIP: 192.168.180.131
Hostname: k8s-node02
Capacity:
cpu: 1
ephemeral-storage: 17394Mi
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 995676Ki
pods: 110
Allocatable:
cpu: 1
ephemeral-storage: 16415037823
hugepages-1Gi: 0
hugepages-2Mi: 0
memory: 893276Ki
pods: 110
System Info:
Machine ID: f914368afc1643a4a6fda29f9b7d4da1
System UUID: CBA34D56-4220-5A92-0FEB-563F66061138
Boot ID: 376e5260-63e9-46e2-bf34-2991cecc5526
Kernel Version: 3.10.0-1160.el7.x86_64
OS Image: CentOS Linux 7 (Core)
Operating System: linux
Architecture: amd64
Container Runtime Version: docker://20.10.17
Kubelet Version: v1.15.1
Kube-Proxy Version: v1.15.1
PodCIDR: 10.244.2.0/24
Non-terminated Pods: (2 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
kube-system kube-flannel-ds-amd64-8k8ww 100m (10%) 100m (10%) 50Mi (5%) 50Mi (5%) 38h
kube-system kube-proxy-t825f 0 (0%) 0 (0%) 0 (0%) 0 (0%) 38h
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 100m (10%) 100m (10%)
memory 50Mi (5%) 50Mi (5%)
ephemeral-storage 0 (0%) 0 (0%)
Events: <none>
啟動一個普通的pod，發現這個pod一直是Pending狀態，檢視事件 nodes are available: 3 node(s) had taints that the pod didn't tolerate.兩個node節點都有汙染標記，都不能被pod容忍。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d6
spec:
selector:
matchLabels:
security: s1 #Deployment會控制label相同的pod
replicas: 1
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
security: s1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
[root@k8s-master01 home]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
d6-5cfddfb4fd-ffblh 0/1 Pending 0 2m <none> <none> <none> <none>
[root@k8s-master01 home]# kubectl describe pod d6-5cfddfb4fd-ffblh
Name: d6-5cfddfb4fd-ffblh
Namespace: default
Priority: 0
Node: <none>
Labels: pod-template-hash=5cfddfb4fd
security=s1
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/d6-5cfddfb4fd
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v1
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-6wl7b (ro)
Conditions:
Type Status
PodScheduled False
Volumes:
default-token-6wl7b:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-6wl7b
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 76s (x2 over 2m21s) default-scheduler 0/3 nodes are available: 3 node(s) had taints that the pod didn't tolerate.
建立一個可以容忍taint0=taint00:NoSchedule的pod，node01上有這個汙點，所以pod都排程到了node01
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Equal
value: taint00
effect: NoSchedule
operator:Equal
那麼key和value都需要匹配node的taint，下面這個例子，value沒有匹配則排程失敗。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Equal
value: taint001
effect: NoSchedule
operator:Exists
則表示不匹配value。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint0
operator: Exists
effect: NoSchedule
刪除node汙點
kubectl taint nodes k8s-node01 taint0=taint00:NoSchedule-
kubectl taint nodes k8s-node02 taint2=taint22:NoSchedule-
PreferNoSchedule
更改汙點型別為node汙點型別，pod的容忍型別也修改為PreferNoSchedule雖然沒有一個node可以匹配，但是還是可以成功排程。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint33
operator: Exists
effect: PreferNoSchedule
NoExecute
驅逐節點，看下面的情況，先建立三個pod，兩個被分配到了node01。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d9
spec:
selector:
matchLabels:
app: v1 #Deployment會控制label相同的pod
replicas: 3
template: # pod 的模板，Deployment通過這個模板建立pod
metadata:
labels:
app: v1
#name: poddemo1 #不能在設定pod名稱了，多個副本的情況下不能重名，會由自動生成
spec:
#restartPolicy: Always #deployment需要控制副本數量，所以重啟策略必須是Always，預設也是Always，所以可以不寫。
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
tolerations:
- key: taint33
operator: Exists
effect: NoExecute
tolerationSeconds: 60
現在兩個node都沒有設定任何的汙點，然後將node01的汙點型別修改為NoExecute，發現三個pod都被驅逐出了node01，排程到了node02下，因為node02沒有任何的汙點。tolerationSeconds是在node01上還能待的時間單位秒。
DaemonSet
DaemonSet確保全部（或者某些）節點上執行一個 Pod 的副本。 當有節點加入叢集時， 也會為他們新增一個 Pod 。 當有節點從叢集移除時，這些 Pod 也會被回收。刪除 DaemonSet 將會刪除它建立的所有 Pod。
DaemonSet支援NodeSelector,NodeAffinity來指定滿足條件的Node範圍進行排程，也支援Taints和Tolerations。
以下案例只有先啟動了一個node，建立daemonSet後，在node01建立了一個pod。
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
再次啟動node02，無需做任何操作，node02上會自動排程一個pod。
Job
The Job "j2" is invalid: spec.template.spec.restartPolicy: Unsupported value: "Always": supported values: "OnFailure", "Never"
restartPolicy不能設定為Always
單次Job
通常只啟動一個 Pod，除非該 Pod 失敗。當 Pod 成功終止時，立即視 Job 為完成狀態。
apiVersion: batch/v1
kind: Job
metadata:
name: j1
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
刪除job會關聯刪除pod
多次Job
completions指定了job會建立幾個pod，也就是會執行幾次，是並行操作。
apiVersion: batch/v1
kind: Job
metadata:
name: j2
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
completions: 2
並行Job
parallelism最大並行數。completions最小完成數。
apiVersion: batch/v1
kind: Job
metadata:
name: j3
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
completions: 2 #成功執行的Pod個數，如果不設定，預設和parallelism
parallelism: 2 #並行執行的pod個數，parallelism預設值是1，completions預設值也是1
CronJob
CronJob 建立基於時隔重複排程的 Jobs CronJob 用於執行週期性的動作，例如備份、報告生成等。 這些任務中的每一個都應該設定為週期性重複的（例如：每天/每週/每月一次）； 你可以定義任務開始執行的時間間隔。
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: cj1
spec:
schedule: '*/1 * * * *'
jobTemplate:
spec:
template:
spec:
containers:
- name: myapp01
image: ubuntu:14.04
command: ['/bin/echo','aaabbbbbbbbbbaa']
restartPolicy: Never
Pod升級策略
RollingUpdate
這是預設的更新方式。設定spec.strategy.type=RollingUpdate，表示Deployment會以捲動更新的方式逐個更新pod，同時可以設定spec.strategy.rollingUpdate下的兩個引數maxUnavailable和maxSurge來控制捲動更新過程。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1 #用於指定Deployment在更新過程中不可用狀態的Pod數量的上限，可以是百分比
maxUnavailable: 1 #用於指定Deploymnet在更新Pod的過程中Pod總數量超過預期副本數量的最大值，可以是百分比
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
現在pod映象需要被更新為192.168.180.129:9999/myharbor/myapp:v2通過命令更新。
kubectl set image deployment d1 myapp01=192.168.180.129:9999/myharbor/myapp:v2
[root@k8s-master01 home]# kubectl set image deployment d1 myapp01=192.168.180.129:9999/myharbor/myapp:v2
deployment.extensions/d1 image updated
[root@k8s-master01 home]# kubectl rollout status deployment d1
deployment "d1" successfully rolled out
[root@k8s-master01 home]# kubectl get pod
NAME READY STATUS RESTARTS AGE
d1-7c86978c57-cfvsj 1/1 Running 0 51s
d1-7c86978c57-dn748 1/1 Running 0 55s
d1-7c86978c57-mhtdp 1/1 Running 0 48s
[root@k8s-master01 home]# kubectl describe pod d1-7c86978c57-cfvsj
Name: d1-7c86978c57-cfvsj
Namespace: default
Priority: 0
Node: k8s-node02/192.168.180.131
Start Time: Sun, 28 Aug 2022 09:21:10 +0800
Labels: app=v1
pod-template-hash=7c86978c57
Annotations: <none>
Status: Running
IP: 10.244.2.168
Controlled By: ReplicaSet/d1-7c86978c57
Containers:
myapp01:
Container ID: docker://ffc089051facd31a242d199e21ca0d3d423cbcdadfbeaa49dd7b993330124f8d
Image: 192.168.180.129:9999/myharbor/myapp:v2
Image ID: docker-pullable://192.168.180.129:9999/myharbor/myapp@sha256:a97b2685e86ee13eaa0cb625e832fb195d39c0ccc8ef4bc7611aab6cac319e34
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
State: Running
Started: Sun, 28 Aug 2022 09:21:12 +0800
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-6wl7b (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
default-token-6wl7b:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-6wl7b
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 86s default-scheduler Successfully assigned default/d1-7c86978c57-cfvsj to k8s-node02
Normal Pulling 84s kubelet, k8s-node02 Pulling image "192.168.180.129:9999/myharbor/myapp:v2"
Normal Pulled 84s kubelet, k8s-node02 Successfully pulled image "192.168.180.129:9999/myharbor/myapp:v2"
Normal Created 84s kubelet, k8s-node02 Created container myapp01
Normal Started 84s kubelet, k8s-node02 Started container myapp01
kubectl describe deploy d1 檢視deployment是如何升級的pod。
9d建立了3個pod
57建立了1個pod
9d縮減到2個pod
57擴容到2個pod
9d縮減到1個pod
57擴容到3個pod
9d縮減到0個pod
最後的結果是1個deployment2個replicaSet3個pod。
Recreate
設定spec.strategy.type=Recreate，表示Deployment在更新pod時，會先殺掉所有正在執行的pod，然後建立新的pod。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
strategy:
type: Recreate
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
9d直接將pod縮減到0個然後57建立了3個。
Deployment回滾
如果在Deployment升級過程中出現意外，比如寫錯新映象的名稱而導致升級失敗，就需要回退到升級之前的舊版本，這時就需要使用到Deployment的回滾功能了。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
現在沒有myapp:v3這個映象，檢視Deployment部署過程。
kubectl rollout status deployment d1
發現新的rs在建立pod時被卡在映象拉去過程中。為了解決這個問題，我們需要回滾到之前穩定版本的Deployment。首先檢視Deployment部署記錄。
kubectl rollout history deployment d1
檢視Deployment特定的版本資訊。
kubectl rollout history deployment d1 --revision=1
[root@k8s-master01 ~]# kubectl rollout history deployment d1 --revision=1
deployment.extensions/d1 with revision #1
Pod Template:
Labels: app=v1
pod-template-hash=8b5b8699d
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v1
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts: <none>
Volumes: <none>
[root@k8s-master01 ~]# kubectl rollout history deployment d1 --revision=2
deployment.extensions/d1 with revision #2
Pod Template:
Labels: app=v1
pod-template-hash=6c659bd7dd
Containers:
myapp01:
Image: 192.168.180.129:9999/myharbor/myapp:v3
Port: 8080/TCP
Host Port: 0/TCP
Command:
nohup
java
-jar
/usr/local/test/dockerdemo.jar
&
Environment: <none>
Mounts: <none>
Volumes: <none>
現在我們回滾到上一個版本，或者回滾到指定的版本。
回滾到上個版本：kubectl rollout undo deployment
回滾到指定版本：kubectl rollout undo deployment d1 --to-revision=1
Deployment暫停和恢復
對於一次複雜的Deployment設定修改，為了避免頻繁觸發Deployment的更新操作，可以先暫停Deployment的更新操作，然後進行設定修改，再恢復Depolyment。注意，在恢復暫停的Deployment前，無法回滾該Deployment。
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
暫停命令
kubectl rollout pause deployment d1
檢視Deployment更新記錄，發現並沒有觸發新的Deployment部署操作。
恢復命令
kubectl rollout resume deploy d1
可以看到恢復後重新建立了一個新的rs，並且Depolyment也有了更新記錄
kubectl describe deploy d1 檢視更新資訊
DeamonSet的更新策略
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
updateStrategy:
type: OnDelete
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
OnDelete
在建立好新的DaemonSet設定後，新的pod並不會被自動建立，直到使用者手動刪除舊的pod才會觸發新建操作。
手動刪除其中一個pod，它才會觸發更新操作
另外一個pod還是v1版本
RollingUpdate
舊版本的pod將自動被殺掉，然後自動部署新版本的pod，整過過程與普通的Deployment捲動升級一樣是可控的。
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: ds1
spec:
selector:
matchLabels:
app: v1
updateStrategy:
type: RollingUpdate
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
回滾與Deployment操作相同。
Pod的擴縮容
k8s對pod的擴縮容操作提供了手動和自動兩種模式，對Deployment或RCC進行pod副本數量設定，即可一鍵完成。
手動擴縮容
通過Deployment將3個副本變成5個，如果副本數少於replicas則是縮容
apiVersion: apps/v1
kind: Deployment
metadata:
name: d1
spec:
selector:
matchLabels:
app: v1
replicas: 3
template:
metadata:
labels:
app: v1
spec:
containers:
- name: myapp01
image: 192.168.180.129:9999/myharbor/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: tomcatport
containerPort: 8080
command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"]
kubectl scale deployment d1 --replicas 5
HPA自動擴縮容
關於HPA的概念參考官方檔案
https://kubernetes.io/zh-cn/docs/tasks/run-application/horizontal-pod-autoscale/
apiVersion: apps/v1 kind: Deployment metadata: name: d1 spec: selector: matchLabels: app: v1 replicas: 1 template: metadata: labels: app: v1 spec: containers: - name: myapp01 image: 192.168.180.129:9999/myharbor/myapp:v1 imagePullPolicy: IfNotPresent ports: - name: tomcatport containerPort: 8080 command: ["nohup","java","-jar","/usr/local/test/dockerdemo.jar","&"] resources: requests: cpu: 50m memory: 50Mi
Metrics-Server安裝
參考這篇部落格，安裝時需要注意自己的k8s版本 https://blog.51cto.com/lingxudong/2545242
基於CPU擴容縮容
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: hpa1
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: d1
minReplicas: 1
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 20
基於記憶體擴容
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: hpa2 spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: d1 minReplicas: 1 maxReplicas: 100 metrics: - type: Resource resource: name: memory target: type: AverageValue averageUtilization: 20
