In this tutorial, we'll explain automate incremental backups with Restic on Linux.
Restic is a fast, secure, and efficient backup tool written in Go. It supports various storage backends, deduplication, encryption, and incremental backups, making it an ideal choice for automating backups on Linux systems. This tutorial will guide you through the process of setting up and automating incremental backups with Restic.
Backups with Restic on Linux
Prerequisites
- A Linux dedicated server or KVM VPS with sudo privileges.
- Basic knowledge of the Linux command line.
- A remote storage destination (e.g., S3, Backblaze B2, or a local directory).
1. Install Restic
Restic can be installed from your distribution’s package manager or by downloading the binary from the official website.
Ubuntu/Debian:
sudo apt update
sudo apt install restic
CentOS/RHEL:
sudo yum install epel-release
sudo yum install restic
2. Initialize a Backup Repository
A Restic repository is where your backups will be stored. You can create a repository in a local directory, on a remote server, or in cloud storage.
Local Directory:
restic init -r /path/to/repo
It will ask to enter the password. Keep the password secure. It you lost the password, data is irrecoverably lost.
Remote Server (via SSH):
restic -r sftp:user@host:/path/to/repo init
Cloud Storage (e.g., S3):
export AWS_ACCESS_KEY_ID="your_access_key"
export AWS_SECRET_ACCESS_KEY="your_secret_key"
restic -r s3:s3.amazonaws.com/your-bucket-name init
3. Create a Backup Script
To automate backups, create a script that defines the backup process. This script will specify the directories to back up, the repository, and any additional options.
nano /usr/local/bin/restic-backup.sh
Example Backup Script:
#!/bin/bash
# Configuration
BACKUP_REPO="/path/to/repo"
BACKUP_SOURCE="/path/to/backup"
PASSWORD="your_password"
LOG_FILE="/var/log/restic-backup.log"
# Export password for non-interactive use
export RESTIC_PASSWORD=$PASSWORD
# Run the backup
restic -r $BACKUP_REPO backup $BACKUP_SOURCE --verbose 2>> $LOG_FILE
# Remove old snapshots (e.g., keep last 7 daily snapshots)
restic -r $BACKUP_REPO forget --keep-daily 7 --prune 2>> $LOG_FILE
# Unset password
unset RESTIC_PASSWORD
Make the script executable:
chmod +x /usr/local/bin/restic-backup.sh
4. Automate the Script with Cron
To schedule the backup script, you can use cron, a time-based job scheduler in Unix-like operating systems.
Edit the crontab file:
crontab -e
Add the following line to run the backup script daily at 2 AM:
0 2 * * * /usr/local/bin/restic-backup.sh
Save and exit the crontab editor. Your backups will now run automatically at the scheduled time.
5. Verify Your Backups
After the backups have been created, you can verify the integrity of your data by running:
restic -r /path/to/repo check
Output:
enter password for repository:
repository b2affaa9 opened (version 2, compression level auto)
created new cache in /tmp/restic-check-cache-3301975772
create exclusive lock for repository
load indexes
[0:00] 100.00% 1 / 1 index files loaded
check all packs
check snapshots, trees and blobs
[0:00] 100.00% 1 / 1 snapshots
no errors were found
To restore files from a backup, use the restore command:
restic -r /path/to/repo restore latest --target /path/to/restore
Output:
enter password for repository:
repository b2affaa9 opened (version 2, compression level auto)
restoring <Snapshot 6a631479 of [/root/backup] at 2024-08-21 02:46:03.185016261 +0000 UTC by root@back> to /root/backup-restore/
Summary: Restored 9 files/dirs (0 B) in 0:00
Advanced Commands
Here are some advanced Restic commands that provide more control and flexibility for managing backups:
1. Excluding Files and Directories
You can exclude specific files or directories from your backups using the --exclude or --exclude-file options.
Exclude a Single Directory:
restic -r /path/to/repo backup /home/user --exclude /home/user/Downloads
Exclude Multiple Patterns:
restic -r /path/to/repo backup /home/user --exclude '*.tmp' --exclude '*.log'
Exclude Patterns from a File:
Create a file named exclude.txt with the following content:
*.tmp
*.log
/home/user/Downloads
Then run:
restic -r /path/to/repo backup /home/user --exclude-file=/path/to/exclude.txt
2. Tagging Snapshots
Tagging allows you to add metadata to snapshots, making it easier to organize and find specific backups.
Create a Tagged Snapshot:
restic -r /path/to/repo backup /home/user --tag daily
View Snapshots with a Specific Tag:
restic -r /path/to/repo snapshots --tag daily
3. Mounting a Backup Repository
You can mount a Restic repository as a filesystem, allowing you to browse and restore files using standard file management tools.
Mount the Repository:
restic -r /path/to/repo mount /mnt/restic
Navigate to /mnt/restic to browse your backups. To unmount, use:
fusermount -u /mnt/restic
4. Running Backups with Low Bandwidth
Restic allows you to limit the bandwidth used during backups, which is useful for running backups over slow or metered connections.
Limit Bandwidth:
restic -r sftp:user@host:/path/to/repo backup /home/user --limit-upload 500 --limit-download 1000
--limit-upload 500: Limits upload speed to 500 KB/s.
--limit-download 1000: Limits download speed to 1 MB/s.
5. Managing Snapshots with forget and prune
The forget and prune commands help you manage snapshots by automatically deleting old backups and reclaiming space.
Keep the Last 7 Daily, 4 Weekly, and 12 Monthly Snapshots:
restic -r /path/to/repo forget --keep-daily 7 --keep-weekly 4 --keep-monthly 12 --prune
Forget and Prune Specific Snapshots:
restic -r /path/to/repo forget a1b2c3d4 --prune
6. Encrypting Your Backups
Restic encrypts backups by default using the password provided during repository initialization. However, you can also use an environment variable for more secure password handling.
Set the Password Using an Environment Variable:
export RESTIC_PASSWORD="your_password"
restic -r /path/to/repo backup /home/user
unset RESTIC_PASSWORD
7. Running Restic in a Script with Error Handling
To enhance the automation script with error handling, use the following approach:
#!/bin/bash
# Configuration
BACKUP_REPO="/path/to/repo"
BACKUP_SOURCE="/path/to/backup"
PASSWORD="your_password"
LOG_FILE="/var/log/restic-backup.log"
# Export password for non-interactive use
export RESTIC_PASSWORD=$PASSWORD
# Function to handle errors
handle_error() {
echo "Backup failed! Check the log at $LOG_FILE"
mail -s "Restic Backup Failed" user@example.com < $LOG_FILE
exit 1
}
# Run the backup with error handling
restic -r $BACKUP_REPO backup $BACKUP_SOURCE --verbose 2>> $LOG_FILE || handle_error
# Remove old snapshots (e.g., keep last 7 daily snapshots)
restic -r $BACKUP_REPO forget --keep-daily 7 --prune 2>> $LOG_FILE || handle_error
# Unset password
unset RESTIC_PASSWORD
echo "Backup completed successfully!"
8. Verifying Backups with check
To ensure the integrity of your backups, regularly use the check command. This command verifies the repository for missing files or corrupted data.
Verify Repository Integrity:
restic -r /path/to/repo check
9. Restoring a Single File or Directory
You can restore specific files or directories without restoring the entire snapshot.
Restore a Specific File:
restic -r /path/to/repo restore latest --target /home/user/restore --include /home/user/Documents/report.docx
Restore a Specific Directory:
restic -r /path/to/repo restore latest --target /home/user/restore --include /home/user/Pictures
10. Monitoring Backup Status
For real-time monitoring of backup progress, you can use the --verbose or --json options.
Real-Time Progress:
restic -r /path/to/repo backup /home/user --verbose=2
Output in JSON Format:
restic -r /path/to/repo backup /home/user --json
This output can be piped into a logging system or parsed by a script for automated monitoring.
Conclusion
By following this tutorial, you've set up and automated incremental backups with Restic on your Linux system. This ensures that your data is regularly backed up with minimal storage space, thanks to Restic's deduplication and incremental backup features. Advanced commands will give you greater control over your Restic backups, enabling you to fine-tune the process according to your specific requirements.
