kubernetes之DaemonSet以及捲動更新

1.什麼是DaemonSet？

1.1DaemonSet是Pod控制器的又一種實現方式，用於在叢集中的全部節點上同時執行一份指定的Pod資源副本，後續加入叢集的節點也會自動建立一個相關的Pod物件，當從叢集移除節點時，此類Pod物件也將被自動回收無需建立。管理員也可以使用"節點選擇器"以及標籤指定僅在部分具有特定特徵的節點上執行指定的Pod物件。
1.2官方檔案: https://kubernetes.io/zh-cn/docs/concepts/workloads/controllers/daemonset/

2.DaemonSet的用法

在每個節點上執行叢集儲存守護行程，如glusterd或Ceph；
在每個節點上執行紀錄檔收集守護行程，如fluentd、logstash；
在每個節點上執行監控系統代理程式，如Prometheus Node Exporter；
在每個節點上執行網路外掛為Pod提供網路服務，如flannel、Calico；
在每個節點上執行kube-Proxy通過API-Server持續監控各個Service及其關聯的Pod物件，並將其建立或變動實時反應至當前節點相應的iptables、ipvs規則上；

3.DaemonSet資源規範

DaemonSet是標準的K8s資源型別，它在Spec欄位中巢狀了Selector、Template和minReadySeconds，並且功能和用法基本相同，但它不支援使用Replicas，DaemonSet並不是基於期望的副本數來控制Pod資源數量，而是基於節點數量來控制Pod數量；

apiVersion: apps/v1   # API群組及版本；
kind: DaemonSet      # 資源型別；
metadata:             # Pod後設資料；
   name:              # 資源名稱，在作用域中要唯一；
   namespace: <string>         # 名稱空間，DaemonSet隸屬名稱空間級別；
spec:
  minReadySeconds: <integer>   # Pod就緒後多少秒內任一容器無Crash方可為就緒，來控制捲動更新的速度，預設值為0，表示新建的Pod物件一旦"就緒"將立即被視作可用，隨後開始下一輪更新。如果設定了spec.minReadySeconds: 3表示新建的Pod物件至少要成功執行多久才會被視作可用，即就緒之後還要等待指定的3s才能開始下一批次的更新，在一個批次內新建的所有Pod就緒探測失敗，都會導致捲動更新被終止，因此為minReadySeconds設定一個合理的值，不僅能夠減緩更新的速度，還能夠讓Deployment提前發現一部分程式因為Bug導致的升級故障，生產環境中為Pod設定指標探測。
  selector: <Object>           # 標籤選擇器，必須匹配template欄位中Pod模板中的標籤；
  template: <Object>           # Pod模板物件；
    metadata: <Object>         # Pod名稱
    spec: <Object>             # Pod詳情
  revisionHistoryLimit: <integer>  # 捲動更新歷史記錄數量，預設為10；
  updateStrategy: <Object>               # 捲動更新策略；
    type: <string>                 # 捲動更新型別，可用值有OnDelete和Rollingupdate；
    rollingUpdate: <Object>       # 捲動更新引數，專用於RollingUpdate型別；
      maxSurge: <string>           # 更新期間可比期望的Pod數量多出的數量和比例；
      maxUnavailable: <string>     # 更新期間可比期望的Pod數量缺少的數量或比例；

4.DaemonSet部署Node_exporter範例

Github地址:https://github.com/prometheus/node_exporter
DockerHub地址:https://hub.docker.com/r/prom/node-exporter
Kubernetes1.8版本起，DaemonSet也必須使用selector來匹配Pod模板中指定的標籤，而且它也支援matchLabels和matchExpressions兩種標籤選擇器；

# 範例組態檔
root@kubernetes-master01:~# cat daemonSet-node_exporter.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共用主機網路
      hostPID: true                        # 獲取主機的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:v0.18.0
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 監聽在節點的9100埠上面，節點的9100，實際就在存取這個Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5
root@kubernetes-master01:~# kubectl apply -f daemonSet-node_exporter.yaml

4.1 檢視Pod範例；

root@kubernetes-master01:~# kubectl get pods -o wide -l app=node-exporter
NAME                  READY   STATUS    RESTARTS   AGE   IP           NODE                NOMINATED NODE   READINESS GATES
node-exporter-l7lfh   1/1     Running   0          55s   xx.0.0.xx   kubernetes-xxx    <none>            <none>
node-exporter-qljd6   1/1     Running   0          55s   xx.0.0.xx   kubernetes-xxx    <none>            <none>

4.2 describe命令檢視詳細資訊；

# DaemonSet也可以簡寫為ds；
root@kubernetes-master01:~# kubectl get daemonset
NAME            DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
node-exporter   2         2         2       2            2           <none>          3m9s

# describe檢視詳細資訊；
root@kubernetes-master01:~# kubectl describe ds node-exporter 
Name:           node-exporter
Selector:       app=node-exporter
Node-Selector:  <none>
Labels:         <none>
Annotations:    deprecated.daemonset.template.generation: 1
Desired Number of Nodes Scheduled: 2   # 期望的Pod副本數
Current Number of Nodes Scheduled: 2   # 
Number of Nodes Scheduled with Up-to-date Pods: 2
Number of Nodes Scheduled with Available Pods: 2
Number of Nodes Misscheduled: 0
Pods Status:  2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:   # 兩個Pod都是Running
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:v0.18.0  # Pod使用的映象
    Port:         9100/TCP                     # Pod的埠
    Host Port:    9100/TCP                     #  開啟了共用宿主機埠
    Liveness:     tcp-socket :node-ex-http delay=5s timeout=1s period=10s #success=1 #failure=3        # 探針檢測
    Readiness:    http-get http://:node-ex-http/metrics delay=5s timeout=1s period=10s #success=1 #failure=3   # 探針檢測
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:     # 排程給兩個節點
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  4m26s  daemonset-controller  Created pod: node-exporter-qljd6
  Normal  SuccessfulCreate  4m26s  daemonset-controller  Created pod: node-exporter-l7lfh

4.3 node_exporter預設監聽在TCP的9100埠，可以向任意一節點發起請求也是沒有問題；

root@kubernetes-master01:~# curl -s xx.0.0.xx:9100/metrics | grep memory 
# HELP node_memory_Active_anon_bytes Memory information field Active_anon_bytes.
# TYPE node_memory_Active_anon_bytes gauge
node_memory_Active_anon_bytes 3.01551616e+08
# HELP node_memory_Active_bytes Memory information field Active_bytes.
# TYPE node_memory_Active_bytes gauge
node_memory_Active_bytes 8.14239744e+08
# HELP node_memory_Active_file_bytes Memory information field Active_file_bytes.
# TYPE node_memory_Active_file_bytes gauge
node_memory_Active_file_bytes 5.12688128e+08
# HELP node_memory_AnonHugePages_bytes Memory information field AnonHugePages_bytes.
# TYPE node_memory_AnonHugePages_bytes gauge
node_memory_AnonHugePages_bytes 0

5.DaemonSet更新方式

1.DaemonSet自1.16版本開始也支援捲動更新機制，相關設定定義在spec.update-Strategy巢狀欄位中。目前它支援RollingUpdate(捲動更新)和OnDelete(刪除式更新)兩種更新策略。捲動更新為預設的更新策略。工作邏輯類似於Deployment控制，不過僅支援使用maxUnavailable屬性定義最大不可用Pod資源的副本數(預設為1)而刪除式更新的方式則是在刪除相應節點的Pod資源後重建並更新為新版本；

5.1 RollingUpdate更新範例:

# image: prom/node-exporter:v0.18.0修改為0.18.1
root@kubernetes-master01:~# cat daemonset-node_exporter.yaml 
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  minReadySeconds: 3
  revisionHistoryLimit: 20
  updateStrategy:
    type: RollingUpdate  # 更新策略
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共用主機網路
      hostPID: true                        # 獲取主機的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:v0.18.1
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 監聽在節點的9100埠上面，節點的9100，實際就在存取這個Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5

5.1.1執行卷動更新及狀態；

root@kubernetes-master01:~# kubectl apply -f daemonset-node_exporter.yaml  && kubectl rollout status daemonset node-exporter 
daemonset.apps/node-exporter configured
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 0 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 out of 2 new pods have been updated...
Waiting for daemon set "node-exporter" rollout to finish: 1 of 2 updated pods are available...
Waiting for daemon set "node-exporter" rollout to finish: 1 of 2 updated pods are available...
daemon set "node-exporter" successfully rolled out

5.1.2預設的RollingUpdate策略將採用一次更新一個Pod物件，待新建的Pod物件就緒後，再更新下一個Pod物件，直到全部完成。可以看到映象版本為v0.18.1

root@kubernetes-master01:~# kubectl describe daemonsets.apps node-exporter 
Pod Template:
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:v0.18.1  # 映象版本已經為0.18.1
Events:
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  46m    daemonset-controller  Created pod: node-exporter-qljd6
  Normal  SuccessfulCreate  46m    daemonset-controller  Created pod: node-exporter-l7lfh
  Normal  SuccessfulDelete  2m34s  daemonset-controller  Deleted pod: node-exporter-qljd6
  Normal  SuccessfulCreate  2m32s  daemonset-controller  Created pod: node-exporter-q479h
  Normal  SuccessfulDelete  2m21s  daemonset-controller  Deleted pod: node-exporter-l7lfh
  Normal  SuccessfulCreate  2m11s  daemonset-controller  Created pod: node-exporter-8c2mr

5.2 OnDelete更新範例:

# 修改映象版本為latest
root@kubernetes-master01:~# cat daemonset-node_exporter.yaml 
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
  namespace: default
spec:
  minReadySeconds: 3
  revisionHistoryLimit: 10
  updateStrategy:      # 捲動更新策略
    type: OnDelete     #  使用OnDelete
    rollingUpdate: 
      maxUnavailable: 1
  selector:
    matchLabels:
      app: node-exporter

  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true                    # 共用主機網路
      hostPID: true                        # 獲取主機的PID
      containers:
      - name: prometheus-node-exporter
        image: prom/node-exporter:latest  # 修改映象版本為latest。
        ports:
        - name: node-ex-http
          containerPort: 9100
          hostPort: 9100                  # 監聽在節點的9100埠上面，節點的9100，實際就在存取這個Pod
        livenessProbe:
          tcpSocket:
            port: node-ex-http
          initialDelaySeconds: 5

        readinessProbe:
          httpGet:
            path: '/metrics'
            port: node-ex-http
          initialDelaySeconds: 5

5.2.1 由於OnDelete並非自動完成升級，它需要管理員手動去刪除Pod，然後重新拉起新的Pod，才能完成更新。(對於升級有著先後順序的軟體，這種方法非常有用；)
5.2.2刪除Pod；

root@kubernetes-master01:~# kubectl delete pods node-exporter-8c2mr 
pod "node-exporter-8c2mr" deleted
root@kubernetes-master01:~# kubectl delete pods node-exporter-q479h
pod "node-exporter-q479h" deleted

5.2.3檢視Pod已經被拉起；

root@kubernetes-master01:~# kubectl get pods -l app=node-exporter
NAME                  READY   STATUS    RESTARTS   AGE
node-exporter-fbmkv   1/1     Running   0          99s
node-exporter-hv69b   1/1     Running   0          85s

5.2.4檢視Events和版本，已經升級為latest；

root@kubernetes-master01:~# kubectl describe ds node-exporter 
Name:           node-exporter
Selector:       app=node-exporter
Node-Selector:  <none>
Labels:         <none>
Annotations:    deprecated.daemonset.template.generation: 3
Desired Number of Nodes Scheduled: 2
Current Number of Nodes Scheduled: 2
Number of Nodes Scheduled with Up-to-date Pods: 2
Number of Nodes Scheduled with Available Pods: 2
Number of Nodes Misscheduled: 0
Pods Status:  2 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=node-exporter
  Containers:
   prometheus-node-exporter:
    Image:        prom/node-exporter:latest
    Port:         9100/TCP
    Host Port:    9100/TCP
    Liveness:     tcp-socket :node-ex-http delay=5s timeout=1s period=10s #success=1 #failure=3
    Readiness:    http-get http://:node-ex-http/metrics delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age    From                  Message
  ----    ------            ----   ----                  -------
  Normal  SuccessfulCreate  3m31s  daemonset-controller  Created pod: node-exporter-fbmkv
  Normal  SuccessfulCreate  3m17s  daemonset-controller  Created pod: node-exporter-hv69b

root@kubernetes-master01:~# kubectl get pods node-exporter-fbmkv  -o yaml
- containerID: docker://03e895265b19f82b3086c3bd7c013f7fa5edea96551a669aac164b92ea2e26b5
    image: prom/node-exporter:latest