StatefulSet浠�缁�

���扮演����棰�锛�

浣跨��Deployment��寤虹��Pod�����舵����锛�褰�����Volume涔���锛�濡���璇�Pod��河渠�锛�Replication Controller浼���run涓�涓��ヤ�璇����ㄦ�э�浣����变������舵����锛�Pod��河渠����跺��涓�涔�����Volume���崇郴灏卞凡缁���寮�河渠�锛��拌捣�ョ��Pod��娉��惧�颁�����Pod��浣���瀵逛��ㄦ�疯��瑷�锛�浠�浠�瀵瑰�灞���Pod��河渠�娌℃�����ワ�浣���褰�Pod��河渠�涔���广大辨��娉���浣跨�ㄤ�����潘界��纾���河渠���

StatefulSet: ��涓�绉�缁�Pod��岛���涓���蹇����у�跺��锛�疏浚���浠ヤ�璇��ㄧ讲���╁���椤哄���

Pod涓��存�э�����娆″�锛����ㄣ����姝㈡�″�锛���缃�缁�涓��存�с��姝や��存�т�Pod�稿�规划�涓�琚�璋�搴��板��涓�node���规���炽��

绋冲����娆″�锛�瀵逛�N涓�������StatefulSet锛�姣�涓�Pod�藉��[0锛�N)�����村������涓�涓��板��搴��凤�涓�����涓�����

绋冲����缃�缁�锛�Pod��hostname妯″�涓�(statefulset��绉�)- (搴���)��

绋冲����海洋���锛���杩�VolumeClaimTemplate涓烘��涓�Pod��寤轰�涓�PV�����ゃ����广大�����锛�涓�浼����ょ�稿�崇���枫��

(1) RC�� RS��Deployment��DS��-----> ���舵������

template(妯℃��):�规��妯℃�� ��寤哄�烘�ョ��Pod,疏浚�浠�J���舵���芥��涓�妯′��风��(�や���绉帮�IP, ����涔�澶�)

��浠ョ��瑙d负:浠讳�涓�涓�Pod, �藉��浠ヨ����わ��跺���ㄦ�扮演������Pod杩�琛��挎����

(2) ���舵��������: ��瑕�璁板���涓� 娆℃����澶�娆¢��淇′腑���稿�充�浠讹�浠ヤ�涓轰�涓���淇$����绫绘������姣�濡�: mysql绛��版��搴����°��(Pod����绉帮�涓��介�����������版����涔�������褰�涔���涓�涓��凤�姣�涓�涓�Pod�芥����宸辩�������版����涔���海洋��ㄧ��褰���)

mysql:涓讳��崇郴��

濡�����涔������舵�������℃���讳负����缇�绛��茬��锛���涓猴�杩�河渠��颁�疏浚��跺��广大卞��浠モ�����衡�����d�锛����舵��广大辨���讳负:疏浚��╋���疏浚��╀����茬��涓��山�拌揪涓�疏浚��跺�������衡��锛�河渠轰滑寰�寰�浼��ч【疏浚��╃��涓�����

(3) 姣�涓�涓�Pod---->瀵瑰�涓�涓�PVC---->姣�涓�涓�PVC瀵瑰�涓�涓�PV��

storageclass:���ㄥ��寤�PV

��瑕�瑙e��:���ㄥ��寤�PVC��

疏浚��板����

涓� ReplicaSet �� Deployment 璧�婧�涓��凤�StatefulSet 涔�浣跨�ㄦ�у�跺�ㄧ���瑰�疏浚��起�疏浚�涓昏��� StatefulSetController��StatefulSetControl �� StatefulPodControl 涓�涓�缁�浠跺��浣��ュ���� StatefulSet ��绠$��锛�StatefulSetController 浼����朵� PodInformer �� ReplicaSetInformer 涓��ュ��澧����逛�浠跺苟灏�河渠�浠舵�ㄩ���伴����涓�锛�
k8s��StatefulSet锛����舵�����★�疏浚���
�у�跺�� StatefulSetController 浼��� Run �规�涓����ㄥ�涓� Goroutine ��绋�锛�杩�河渠���绋�浼�浠�����涓��山��寰�澶����� StatefulSet 璧�婧�杩�琛���姝ワ��ヤ��ユ��浠�浼���浠�缁� Kubernetes ��姝� StatefulSet ��杩�绋���

1锛�岛�海洋�

锛�1锛���寤轰�涓�statefulset��yaml��浠�

[root@master yaml]# vim statefulset.yaml
apiVersion: v1
kind: Service
metadata:
  name: headless-svc
  labels:
    app: headless-svc
spec:
  ports:
  - port: 80
  selector:
    app: headless-pod
  clusterIP: None     #娌℃����涓���ip
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-test
spec:
  serviceName: headless-svc
  replicas: 3
  selector:
    matchLabels:
      app: headless-pod
  template:
    metadata:
      labels:
        app: headless-pod
    spec:
      containers:
      - name: myhttpd
        image: httpd
        ports:
        - containerPort: 80

Deployment : Deploy+RS+���哄��绗�涓�(Pod����绉般��)娌℃��椤哄���锛���
浠ユ病魔�����夸唬����

1��headless-svc :��澶存���°����涓烘病��IP�板��锛���浠ュ��涓��山�璐�潘藉��琛$�����戒�����涓�statefulset瑕�姹�Pod����绉版����椤哄���锛�姣�涓�涓�Pod�戒��借�������浠o�涔�广大辨���充娇Pod��寤轰���锛���绉颁��朵�����涓哄��绔���姣�涓�涓�Pod�诲�藉����

2��statefulSet:疏浚�涔��蜂���搴���

3��volumeClaimT emplates:���ㄥ��寤�PVC锛�涓哄��绔���Pod��岛�涓�����海洋��ㄣ��

�ц�涓�涓�

[root@master yaml]# kubectl apply -f statefulset.yaml 

�ョ��涓�涓�

[root@master yaml]# kubectl get svc

k8s��StatefulSet锛����舵�����★�疏浚���

[root@master yaml]# kubectl get pod
//�����拌�河渠�pod����椤哄���

k8s��StatefulSet锛����舵�����★�疏浚���

涓�����寤�StorageClass璧�婧�瀵硅薄��

1���轰�NFS���★���寤�NFS���°��

涓�潘�nfs����疏浚�瑁���

[root@node02 ~]# yum -y install nfs-utils  rpcbind

��寤哄�变韩��褰�

[root@master ~]# mkdir /nfsdata

��寤哄�变韩��褰�������

[root@master ~]# vim /etc/exports
/nfsdata *(rw,sync,no_root_squash)

寮���nfs��rpcbind

[root@master ~]# systemctl start nfs-server.service 
[root@master ~]# systemctl start rpcbind

娴�璇�涓�涓�

[root@master ~]# showmount -e

k8s��StatefulSet锛����舵�����★�疏浚���

2����寤�rbac������

[root@master yaml]# vim rbac-rolebind.yaml 
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-provisioner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nfs-provisioner-runner
rules:
   -  apiGroups: [""]
      resources: ["persistentvolumes"]
      verbs: ["get", "list", "watch", "create", "delete"]
   -  apiGroups: [""]
      resources: ["persistentvolumeclaims"]
      verbs: ["get", "list", "watch", "update"]
   -  apiGroups: ["storage.k8s.io"]
      resources: ["storageclasses"]
      verbs: ["get", "list", "watch"]
   -  apiGroups: [""]
      resources: ["events"]
      verbs: ["watch", "create", "update", "patch"]
   -  apiGroups: [""]
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-provisioner
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: nfs-provisioner-runner
rules:
   -  apiGroups: [""]
      resources: ["persistentvolumes"]
      verbs: ["get", "list", "watch", "create", "delete"]
   -  apiGroups: [""]
      resources: ["persistentvolumeclaims"]
      verbs: ["get", "list", "watch", "update"]
   -  apiGroups: ["storage.k8s.io"]
      resources: ["storageclasses"]
      verbs: ["get", "list", "watch"]
   -  apiGroups: [""]
      resources: ["events"]
      verbs: ["watch", "create", "update", "patch"]
   -  apiGroups: [""]
      resources: ["services", "endpoints"]
      verbs: ["get","create","list", "watch","update"]
   -  apiGroups: ["extensions"]
      resources: ["podsecuritypolicies"]
      resourceNames: ["nfs-provisioner"]
      verbs: ["use"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-provisioner
    namespace: default   #蹇���海洋�娈�
roleRef:
  kind: ClusterRole
  name: nfs-provisioner-runner
  apiGroup: rbac.authorization.k8s.io

�ц�涓�涓�

[root@master yaml]# kubectl apply -f rbac-rolebind.yaml 

3����寤�Deployment璧�婧�瀵硅薄锛���Pod浠f�� ��姝g��NFS���°��

[root@master yaml]# vim nfs-deployment.yaml 
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nfs-client-provisioner
spec:
  replicas: 1
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccount: nfs-provisioner
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner
          volumeMounts:
            - name: nfs-client-root
              mountPath:  /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: bdqn
            - name: NFS_SERVER
              value: 192.168.1.21
            - name: NFS_PATH
              value: /nfsdata
      volumes:
        - name: nfs-client-root
          nfs:
            server: 192.168.1.21
            path: /nfsdata

�ц�涓�涓�

[root@master yaml]# kubectl apply -f nfs-deployment.yaml 

�ョ��涓�涓�

[root@master yaml]# kubectl get pod

k8s��StatefulSet锛����舵�����★�疏浚���

4����寤�storageclass��yaml��浠�

[root@master yaml]# vim test-storageclass.yaml 
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: stateful-nfs
provisioner: bdqn  #��杩�provisioner海洋�娈靛�宠���颁�杩�Deploy
reclaimPolicy: Retain

�ц�涓�涓�

[root@master yaml]# kubectl apply -f test-storageclass.yaml

�ョ��涓�涓�

[root@master yaml]# kubectl get sc

k8s��StatefulSet锛����舵�����★�疏浚���

河渠�锛�瑙e�宠���ㄥ��寤�pvc

1����寤�statefulset��yaml��浠�

[root@master yaml]# vim statefulset.yaml 
apiVersion: v1
kind: Service
metadata:
  name: headless-svc
  labels:
    app: headless-svc
spec:
  ports:
  - port: 80
    name: myweb
  selector:
    app: headless-pod
  clusterIP: None
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: statefulset-test
spec:
  serviceName: headless-svc
  replicas: 3
  selector:
    matchLabels:
      app: headless-pod
  template:
    metadata:
      labels:
        app: headless-pod
    spec:
      containers:
      - image: httpd
        name: myhttpd
        ports:
        - containerPort: 80
          name: httpd
        volumeMounts:
        - mountPath: /mnt
          name: test
  volumeClaimTemplates:  #> ���ㄥ��寤�PVC锛�涓哄��绔���Pod��岛�涓�����海洋��ㄣ��**
  - metadata:
      name: test
      annotations:   #杩�����疏浚�storageclass
        volume.beta.kubernetes.io/storage-class: stateful-nfs
    spec:
      accessModes:
        - ReadWriteOnce
      resources:
        requests:
          storage: 100Mi

�ㄦ�ょず岛�涓�锛�

  • ��寤轰�涓�涓���涓� headless-svc �� Service 瀵硅薄锛��� metadata: name 海洋�娈垫��绀恒��璇� Service 浼�疏浚�浣�涓�涓���涓� headless-svc ��搴���锛��� labels: app: headless-svc �� selector: app: headless-pod ��绀恒��璇� Service 浼���寮�绔��� 80 骞跺��跺�藉��涓� web����涓�璇� Service 浼��у�剁���骞跺�河渠���缃�娴���璺��卞�� StatefulSet �ㄧ讲��疏浚瑰�ㄥ��搴��ㄣ��
  • 浣跨�ㄤ�涓����� Pod (replicas: 3) ��寤轰�涓�涓���涓� web �� StatefulSet��
  • Pod 妯℃�� (spec: template) ��绀哄�� Pod ��璁颁负 app: headless-pod��
  • Pod 瑙��� (template: spec) ��绀� StatefulSet �� Pod 杩�琛�涓�涓�疏浚瑰�� myhttpd锛�璇ュ�瑰�ㄨ�琛�����涓� httpd ������疏浚瑰�ㄦ������ Container Registry 绠$����
  • Pod 瑙���浣跨�ㄧ�� Service ��寮��� web 绔��c��
  • template: spec: volumeMounts ��疏浚�涓�涓���涓� test �� mountPath��mountPath ��疏浚瑰�ㄤ腑搴�瑁�潘藉���ㄥ�风��璺�寰���
  • StatefulSet 棰���河渠�涓�涓��锋�� 100mb 棰���海洋��ㄧ┖�寸�� PersistentVolumeClaim锛�test��
  • �ц�涓�涓�

    [root@master yaml]# kubectl apply -f statefulset.yaml

    �ョ��涓�涓�

    [root@master yaml]# kubectl get pod

    k8s��StatefulSet锛����舵�����★�疏浚���

    濡���绗�涓�涓�pod�雾�颁���棰�锛����㈢��pod广大变�浼�������

    [root@master yaml]# kubectl get statefulsets

    k8s��StatefulSet锛����舵�����★�疏浚���

    2�� 楠�璇�涓�涓��版��海洋���

    疏浚瑰�ㄤ腑��寤烘��浠�

    [root@master yaml]# kubectl exec -it statefulset-test-0 /bin/sh
    # cd /mnt
    # touch testfile
    # exit

    疏浚夸富�烘�ョ��涓�涓�

    [root@master yaml]# ls /nfsdata/default-test-statefulset-test-0-pvc-bf1ae1d0-f496-4d69-b33b-39e8aa0a6e8d/
    testfile

    涓���广大�疏浚�楠�

    浠ヨ��宸辩����绉板��寤轰�涓���绉扮┖�达�浠ヤ�����璧�婧��借�琛��ㄦ�ょ┖�翠腑����statefuset璧�婧�杩�琛�涓�涓�httpd web���★�瑕�姹�3涓�Pod锛�浣���姣�涓�Pod��涓荤���㈠��疏浚逛�涓��凤�骞朵��借���涓������版����涔���锛�广大�璇����ゅ�朵腑涓�涓�Pod锛��ョ���扮演������Pod锛��荤�瀵规��涓�涔���Deployment璧�婧��у�跺�ㄦ�у�剁��Pod��浠�涔�涓���涔�澶�锛�

    锛�涓�锛���寤�StorageClass璧�婧�瀵硅薄��

    娉ㄦ��锛�nfs���¤�寮���

    1����寤�namespace��yaml��浠�

    [root@master yaml]# vim namespace.yaml 
    kind: Namespace
    apiVersion: v1
    metadata:
      name: xgp-lll    #namespave����绉�

    �ц�涓�涓�

    [root@master yaml]# kubectl apply -f namespace.yaml 

    �ョ��涓�涓�

    [root@master yaml]# kubectl get namespaces 

    k8s��StatefulSet锛����舵�����★�疏浚���

    2. ��寤�rbac������

    [root@master yaml]# vim rbac-rolebind.yaml
    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: nfs-provisioner
      namespace: xgp-lll
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: nfs-provisioner-runner
      namespace: xgp-lll
    rules:
       -  apiGroups: [""]
          resources: ["persistentvolumes"]
          verbs: ["get", "list", "watch", "create", "delete"]
       -  apiGroups: [""]
          resources: ["persistentvolumeclaims"]
          verbs: ["get", "list", "watch", "update"]
       -  apiGroups: ["storage.k8s.io"]
          resources: ["storageclasses"]
          verbs: ["get", "list", "watch"]
       -  apiGroups: [""]
          resources: ["events"]
          verbs: ["watch", "create", "update", "patch"]
       -  apiGroups: [""]
          resources: ["services", "endpoints"]
          verbs: ["get","create","list", "watch","update"]
       -  apiGroups: ["extensions"]
          resources: ["podsecuritypolicies"]
          resourceNames: ["nfs-provisioner"]
          verbs: ["use"]
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
      name: run-nfs-provisioner
    subjects:
      - kind: ServiceAccount
        name: nfs-provisioner
        namespace: xgp-lll
    roleRef:
      kind: ClusterRole
      name: nfs-provisioner-runner
      apiGroup: rbac.authorization.k8s.io

    �ц�涓�涓�

    [root@master yaml]# kubectl apply -f rbac-rolebind.yaml

    3����寤�Deployment璧�婧�瀵硅薄锛���Pod浠f�� ��姝g��NFS���°��

    [root@master yaml]# vim nfs-deployment.yaml 
    
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
      name: nfs-client-provisioner
      namespace: xgp-lll
    spec:
      replicas: 1
      strategy:
        type: Recreate
      template:
        metadata:
          labels:
            app: nfs-client-provisioner
        spec:
          serviceAccount: nfs-provisioner
          containers:
            - name: nfs-client-provisioner
              image: registry.cn-hangzhou.aliyuncs.com/open-ali/nfs-client-provisioner
              volumeMounts:
                - name: nfs-client-root
                  mountPath:  /persistentvolumes
              env:
                - name: PROVISIONER_NAME
                  value: xgp
                - name: NFS_SERVER
                  value: 192.168.1.21
                - name: NFS_PATH
                  value: /nfsdata
          volumes:
            - name: nfs-client-root
              nfs:
                server: 192.168.1.21
                path: /nfsdata

    �ц�涓�涓�

    [root@master yaml]# kubectl apply -f nfs-deployment.yaml 

    �ョ��涓�涓�

    [root@master yaml]# kubectl get pod  -n xgp-lll 

    k8s��StatefulSet锛����舵�����★�疏浚���

    4����寤�storageclass��yaml��浠�

    [root@master yaml]# vim test-storageclass.yaml 
    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: stateful-nfs
      namespace: xgp-lll
    provisioner: xgp  #��杩�provisioner海洋�娈靛�宠���颁�杩�Deploy
    reclaimPolicy: Retain

    �ц�涓�涓�

    [root@master yaml]# kubectl apply -f test-storageclass.yaml

    �ョ��涓�涓�

    [root@master yaml]# kubectl get sc -n  xgp-lll

    k8s��StatefulSet锛����舵�����★�疏浚���

    锛�河渠�锛�瑙e�宠���ㄥ��寤�pvc

    1����寤�statefulset��yaml��浠�

    apiVersion: v1
    kind: Service
    metadata:
      name: headless-svc
      namespace: xgp-lll
      labels:
        app: headless-svc
    spec:
      ports:
      - port: 80
        name: myweb
      selector:
        app: headless-pod
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: statefulset-test
      namespace: xgp-lll
    spec:
      serviceName: headless-svc
      replicas: 3
      selector:
        matchLabels:
          app: headless-pod
      template:
        metadata:
          labels:
            app: headless-pod
        spec:
          containers:
          - image: httpd
            name: myhttpd
            ports:
            - containerPort: 80
              name: httpd
            volumeMounts:
            - mountPath: /usr/local/apache2/htdocs
              name: test
      volumeClaimTemplates:  #> ���ㄥ��寤�PVC锛�涓哄��绔���Pod��岛�涓�����>海洋��ㄣ��**
      - metadata:
          name: test
          annotations:   #杩�����疏浚�storageclass
            volume.beta.kubernetes.io/storage-class: stateful-nfs
        spec:
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 100Mi

    �ц�涓�涓�

    [root@master yaml]# kubectl apply -f statefulset.yaml

    �ョ��涓�涓�

    [root@master yaml]# kubectl get pod -n xgp-lll 

    k8s��StatefulSet锛����舵�����★�疏浚���

    2�� 楠�璇�涓�涓��版��海洋���

    疏浚瑰�ㄤ腑��寤烘��浠�

    绗�涓�涓�
    [root@master yaml]# kubectl exec -it -n xgp-lll statefulset-test-0 /bin/bash 
    root@statefulset-test-0:/usr/local/apache2# echo 123 > /usr/local/apache2/htdocs/index.html
    
    绗�河渠�涓�
    [root@master yaml]# kubectl exec -it -n xgp-lll statefulset-test-1 /bin/bash 
    root@statefulset-test-2:/usr/local/apache2# echo 456 > /usr/local/apache2/htdocs/index.html
    
    绗�涓�涓�
    [root@master yaml]# kubectl exec -it -n xgp-lll statefulset-test-2 /bin/bash 
    root@statefulset-test-1:/usr/local/apache2# echo 789 > /usr/local/apache2/htdocs/index.html
    

    疏浚夸富�烘�ョ��涓�涓�

    绗�涓�涓�
    [root@master yaml]# cat /nfsdata/xgp-lll-test-statefulset-test-0-pvc-ccaa02df-4721-4453-a6ec-4f2c928221d7/index.html 
    123
    
    绗�河渠�涓�
    [root@master yaml]# cat /nfsdata/xgp-lll-test-statefulset-test-1-pvc-88e60a58-97ea-4986-91d5-a3a6e907deac/index.html 
    456
    
    绗�涓�涓�
    [root@master yaml]# cat /nfsdata/xgp-lll-test-statefulset-test-2-pvc-4eb2bbe2-63d2-431a-ba3e-b7b8d7e068d3/index.html 
    789

    璁块��涓�涓�

    k8s��StatefulSet锛����舵�����★�疏浚���