RClone for Backups

There are many backup solutions out there for Kubernetes, but the one I ended up going with for my needs is Rclone. It supports MANY different back ends and is incredibly flexible. It's been called the "swiss army knife" of backup solutions.

For my needs, I've configured Rclone as a cronjob in a namespace called backup to run every night to send files from various locations to private Backblaze buckets for off-site storage. Below is an example of how I deployed Rclone in Kubernetes.

To keep things tidy I create a dedicated backup namespace.

00-namespace.yaml

apiVersion: v1
kind: Namespace
metadata:
  name: backup
  labels:
    name: backup

Now we need to define the volumes and claims we're going to backup so they can be mounted to the Rclone container. Below is an example of an NFS share on my network that I want backed up. You can define as many volumes and claims as you need.

01-storage.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: backups
spec:
  capacity:
    storage: 1Ti
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: nfs
  nfs:
    path: /path/to/share
    server: server.name.or.ip
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  namespace: backup
  name: backups
spec:
  accessModes:
    - ReadWriteMany
  volumeName: backups
  storageClassName: nfs
  resources:
    requests:
      storage: 1Ti

Be sure to change the details like you server name, path. and storage size. While I'm using NFS shares here, you could certainly use any storage available to you (like Longhorn for example).

Now that we've created the foundations, we need to define some configurations for Rclone. Rclone will look for a rclone.conf file for it's configuration and we will want to provide this as a Kubernetes Secret so we can easily maintain it. Since I'm using Backblaze I've configured Rclone for B2. You can either use the documentation as a guide or even install Rclone on your workstation and run:

rclone config

This will interactively create your config file locally and you can use that to create your Secret.

Additionally, I have created a bash script as a Kubernetes ConfigMap that will do the work of backing up my drives. I've combined the Secret and ConfigMap into a single "configs" YAML file here.

02-configs.yaml

apiVersion: v1
kind: Secret
metadata:
  name: rclone-config
  namespace: backup
stringData: 
  rclone.conf: |
    [My-Backup]
    type = b2
    account = <your-application-api-keyID>
    key = <your-applications-api-secret>
    hard_delete = true
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: rclone-scripts
  namespace: backup
data:
  rclone-sync.sh: |
    #!/bin/sh

    echo "---------------------------------------------------"
    echo "Begining backup of path 1"
    echo "---------------------------------------------------"
    rclone sync /mnt/backup/path1 <Your Instance Name>:<Your Bucket Name> --config /config/rclone/rclone.conf -v
    echo "---------------------------------------------------"
    echo "Begining backup of path 2"
    echo "---------------------------------------------------"
    rclone sync /mnt/backup/path2 <Your Instance Name>:<Your Bucket Other Name> --config /config/rclone/rclone.conf -v

You can define multiple volumes and PVCs to backup if you need to, or have a single PVC with different directories. Rclone will recursively backup all items is a defined path.

We need to put this all together as a CronJob in Kubernetes to execute automatically.

03-cronob.yaml

apiVersion: batch/v1
kind: CronJob
metadata:
  labels:
    app: rclone
  name: rclone-sync
  namespace: backup
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: rclone
              image: rclone/rclone:latest
              imagePullPolicy: Always
              env:
                - name: TZ
                  value: "America/NewYork"
              volumeMounts:
                - name: scripts
                  mountPath: /scripts
                - name: backups
                  mountPath: /mnt/backup
                - name: rclone-config
                  mountPath: /config/rclone/rclone.conf
                  subPath: rclone.conf
              resources:
                limits:
                  cpu: '4'
                  memory: 16Gi
                requests:
                  cpu: 100m
                  memory: 100Mi
              command: 
                - "/bin/sh"
                - "/scripts/rclone-sync.sh"
          restartPolicy: OnFailure
          volumes:
            - name: backups
              persistentVolumeClaim:
                claimName: backups
            - name: scripts
              configMap:
                name: rclone-scripts
            - name: rclone-config
              secret:
                secretName: rclone-config

As you can see, this will create a CronJob that runs daily at 2 AM and will mount the Secret and ConfigMap we created earlier as files on the file system. The container then runs the script we defined in the ConfigMap to do the backups.

Now we need to deploy to the Kubernetes Cluster. I like to save these in a script to easily redeploy again if I make any changes. Simply run the below script to deploy to Kubernetes.

build-rclone.sh

#!/bin/bash

kubectl apply -f 00-namespace.yaml \
              -f 01-storage.yaml \
              -f 02-configs.yaml \
              -f 03-cronjob.yaml

While there are several ways to do backups, I hope this simple example will give you some ideas on how to implement a backup strategy that works for you.