Linux Backup and Recovery

Linux Backup and Recovery

As Linux system administrator one of the most important tasks to be done is the system backup. Hardware/software failures will bring down your system and then you must be prepared to recover it as quickly and efficiently as possible. In order to perform system backup Linux offers several commands/services depending on the type of backup required.

tar

This command is used for creating full backups of an entire part of the system that do not change a lot. For example to backup the /usr/local system directory :

$ tar cvfz usr_local.tar.gz /usr/local

It creates 'c' a compressed 'z' tar file 'usr_local.tar.gz' that contains all /usr/local structure and data preserving files permissions and ownership. As the data is compressed and archived on the tar file the backups uses less space that the original data, and can be transferred to another machines using scp or ftp.

Note: By default selinux file attributes are not preserved on the tar file. In order to preserve selinux atributtes '--selinux' flag must be used.

* In order to recover tar file on the system the following command must be used :

$ tar tvzf usr_local.tar.gz
...
drwxr-xr-x root/root 0 2009-12-04 14:33 usr/local/share/man/man4x/
drwxr-xr-x root/root 0 2009-12-04 14:33 usr/local/libexec/
drwxr-xr-x root/root 0 2009-12-04 14:33 usr/local/include/

It shows the files that are going to be restored on the system without performing the restore at all. It is just a test 't' to be warned where the files will be restored on the system. NOTE: the file path reported is the file system path where the file will be restored: in this case is usr/local/ what means that if the tar file is restored on /tmp directory the files will be written on /tmp/usr/local.

Once sure where the files are going to be written using tar test mode the restore can be done :

$ cd /
$ tar xvfz /root/usr_local.tar.gz
...
usr/local/share/man/man4x/
usr/local/libexec/
usr/local/include/

rsync

As most of we believe rsync is the best tool that can be used to perform backups. It can be used to copy files on the local system or remotely through the network. The main difference between rsync and tar is that rsync only copies the differences between the source and destination, tar always copy all the data structure from the source when the tar is created and all the data is restored on destination.

$ rsync -av /usr/local/ /tmp/destination/

sending incremental file list
...
share/man/mann/
share/perl5/
src/
sent 514 bytes received 45 bytes 1118.00 bytes/sec
total size is 0 speedup is 0.00
Copy recursively /usr/local/* on /tmp/destination/ directory, for example /usr/local/bin is copied on /tmp/destination/bin.

$ rsync -avz /usr/local/ root@remotehost:/tmp/destination/

sending incremental file list
...
share/man/mann/
share/perl5/
src/
sent 514 bytes received 45 bytes 1118.00 bytes/sec
total size is 0 speedup is 0.00
Copy recursively /usr/local/* on /tmp/destination/ directory on remotehost using ssh or rsync credentials. In this case compression is enabled 'z' because the copy is done through the network.

$ rsync -avz --delete /usr/local/ root@remotehost:/tmp/destination/

sending incremental file list
...
share/man/mann/
share/perl5/
src/
sent 514 bytes received 45 bytes 1118.00 bytes/sec
total size is 0 speedup is 0.00
Copy recursively /usr/local/* on /tmp/destination/ directory on remotehost. In this case it also deletes '--delete' the files that are on destination but not on source: it keeps on completely sync /usr/local/ and /tmp/destination/ on the remote host.

Note : rsync keeps file permissions and ownership on the file transmission, but it does not keep selinux attributes. There is not such option as '--selinux' as tar, the selinux relabeling must be done by hand with the 'chcon' command.

* In order to restore the information, just switch the source <-> destination :

$ rsync -n -av /tmp/destination/ /usr/local/
sending incremental file list
./
file1
file2

sent 567 bytes received 54 bytes 1242.00 bytes/sec
total size is 0 speedup is 0.00 (DRY RUN)

With the dry-run option '-n' the rsync is simulated but it is not done. This is a very useful option to test what is going to be copied or deleted by the rsync command just before running it.

$ rsync -av /tmp/destination/ /usr/local/

sending incremental file list
./
file1
file2

sent 639 bytes received 86 bytes 1450.00 bytes/sec
total size is 0 speedup is 0.00

The restore has been done.

Hard link and rsync

A hard link to a file provides the ability to reference the same inode (hardware location) from multiple places within the same filesystem. If there is a hard link and there are other hard links to the original file, only the link is removed and the original file will not be modified. As an example, lets create a 100M file called fileorig :

mkdir -p /tmp/hard
$ cd /tmp/hard
$ dd if=/dev/zero of=fileorig bs=1024 count=100000
$ du -sh ../hard
98M ../hard

Lets create a file called filehlink hard linked to fileorig :

$ ln fileorig filehlink

Verify the link

$ ls -lrti
total 200000
134435 -rw-r--r--. 2 root root 102400000 Dec 4 13:52 fileorig
134435 -rw-r--r--. 2 root root 102400000 Dec 4 13:52 filehlink

It can be seen that both files has the same inode number 134435 and the same size (100M). These file are exactly the same file...

$ du -h ../hard/
98M ../hard/

BUT THE SIZE OF THE DIRECTORY IS STILL 100M !!!. This is because filehlink is just a link to the original file fileorig.

* Using hard link copies in combination with rsync provides the ability of having system full backups only using the size disk consumed by the differences applied by rsync. Lets have a look to the following script :

$ cat /backup/rsync_snapshot.sh

rm -rf tmp.3
mv tmp.2 tmp.3
mv tmp.1 tmp.2
cp -al tmp.0 tmp.1
rsync -av --delete /tmp/ ./tmp.0/

If this script is executed daily, tmp.0, tmp.1, tmp.2 and tmp.3 will appear as daily full backup of /tmp thanks to the hard link copy done by the command 'cp -al' and actualized by 'rsync' using only '2X(size tmp)+(size changes rsync)' instead of 4X(size of tmp) disk space.

The combination of rsync with hard link copies must be seriously considered as the core of a custom made backup system.

tapes

The Advanced Maryland Automatic Network Disk Archiver AMANDA, installed by amanda rpm, is a system tool to manage a network backup system using client-server architecture. This system can be used to rotate automatically full and incremental backups off all amanda-clients on amanda server.

dd

The command 'dd' can be used to clone an entire system, coping bit to bit one disk into another. Suppose that your the system has on disk (/dev/sda) and we want to clone the entire system to another disk :

1.- Shutdown the system and connect a second disk sdb equal or bigger in size that the system disk sda.

2.- Start the system into user single mode 's', adding an s on the kernel loading grub file.

3.- Clone the entire disk sda on sdb using 'dd' command :

$ dd if=/dev/sda of=/dev/sdb

It takes a while depending on the size of the system. It copies sda on sdb bit to bit, so MBR, partitions ,LVM , RAID, filesystems and data are copied on sdb.

4.- Now sdb is ready to be used in other system with the same hardware than the original. Connect disk sdb on the first SATA channel on the new system (--> it will be recognized as sda) and boot it as usual.

Note: As MBR is copied on sdb it is not necessary to install grub on it.