使用cronjob定期对k8s集群里的etcd数据库备份

分类:笔记 日期:2024-01-22 作者:caocaofff 浏览:172

前提:确保定时任务的pod要和etcd的pod要在同一个node上面(使用nodeAffinity)。

apiVersion: batch/v1
kind: CronJob
metadata:
  name: etcd-disaster-recovery
  namespace: default
spec:
 schedule: "0 16 * * *"
 jobTemplate:
  spec:
    template:
      metadata:
       labels:
        app: etcd-disaster-recovery
      spec:
        affinity:
          nodeAffinity:
            requiredDuringSchedulingIgnoredDuringExecution:
                  nodeSelectorTerms:
                  - matchExpressions:
                    - key: kubernetes.io/role
                      operator: In
                      values:
                      - master
        containers:
        - name: etcd
          image: bitnami/etcd:3.5.1
          command:
          - sh
          - -c
          - "export ETCDCTL_API=3; \
             etcdctl --endpoints $ENDPOINT --cert=/etc/kubernetes/pki/etcd/server.crt
          --key=/etc/kubernetes/pki/etcd/server.key --cacert=/etc/kubernetes/pki/etcd/ca.crt snapshot save /snapshot/$(date +%Y%m%d_%H%M%S)_snapshot.db; \
             echo etcd backup sucess"
          env:
          - name: ENDPOINT
            value: "127.0.0.1:2379"
          volumeMounts:
            - mountPath: "/snapshot"
              name: snapshot
              #subPath: data/etcd-snapshot
            - mountPath: /etc/localtime
              name: lt-config
            - mountPath: /etc/timezone
              name: tz-config
            - mountPath: /etc/kubernetes/pki/etcd/server.crt
              name: etcd-cert
            - mountPath: /etc/kubernetes/pki/etcd/server.key
              name: etcd-key
            - mountPath: /etc/kubernetes/pki/etcd/ca.crt
              name: etcd-cacert
        restartPolicy: OnFailure
        volumes:
          - name: snapshot
            persistentVolumeClaim:
              claimName: cron-etcd-bak
          - name: lt-config
            hostPath:
              path: /etc/localtime
          - name: tz-config
            hostPath:
              path: /etc/timezone
          - hostPath:
              path: /etc/kubernetes/pki/etcd/server.crt
            name: etcd-cert
          - hostPath:
               path: /etc/kubernetes/pki/etcd/server.key
            name: etcd-key
          - hostPath:
               path: /etc/kubernetes/pki/etcd/ca.crt
            name: etcd-cacert
        hostNetwork: true

测试环境:k8s v1.23,使用nfs pvc

需要在etcd所在节点打上标签,否则无法调度:

kubectl label node master kubernetes.io/role=master

效果:

[root@master ~]# kubectl get po,job,cronjob 
NAME                                                       READY   STATUS      RESTARTS       AGE
pod/etcd-disaster-recovery-28431487-jhrsm                  0/1     Completed   0              2m13s

NAME                                        COMPLETIONS   DURATION   AGE
job.batch/etcd-disaster-recovery-28431487   1/1           9s         2m13s

NAME                                   SCHEDULE     SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/etcd-disaster-recovery   0 16 * * *   False     0        2m13s           5d1h

Dingtalk_20240122104159.jpg

报错的解决方法

1、pod报错:

Error: failed to start container "etcd": Error response from daemon:
failed to create shim task: OCI runtime create failed: runc create
failed: unable to start container process: error during container
init: error mounting "/etc/timezone" to rootfs at "/etc/timezone":
mount /etc/timezone:/etc/timezone (via /proc/self/fd/6), flags:
0x5000: not a directory: unknown: Are you trying to mount a directory
onto a file (or vice-versa)? Check if the specified host path exists
and is the expected type

在节点执行下面命令:

echo 'Asia/Shanghai' >/etc/timezone

2、pod报错:

Error: open /etc/kubernetes/pki/etcd/server.key: permission denied

那么需要把证书相关文件加上读取权限:

[root@master ~]# ll /etc/kubernetes/pki/etcd/server.key
-rw------- 1 root root 1679 Nov  9 15:58 /etc/kubernetes/pki/etcd/server.key
[root@master ~]# chmod a+r /etc/kubernetes/pki/etcd/server.key
[root@master ~]# chmod a+r /etc/kubernetes/pki/etcd/server.crt 
[root@master ~]# chmod a+r /etc/kubernetes/pki/etcd/ca.crt

CC版权: 本篇博文采用《CC 协议》,转载必须注明作者和本文链接

评论 (暂无评论)

发表评论

昵称:  
邮箱:  
网址:

验证码:captcha