NETW 110 Handout Lab 10 - Backup and Restore

Transcription

1 The Importance of Scheduled Backups Most people who have worked in client-server or mainframe environment have lost files or data. Therefore, it is necessary for the IT staff to develop a thorough, yet easily maintained backup procedure since at some point, any piece of hardware will fail. The first step in designing a backup regimen is to create a written policy that specifies: who is responsible for ensuring that backups occur. what file systems and files need regular archiving. Anything that is mission critical for the organization should be backed up regularly. when backups should occur. Backups should be conducted during slow hours, because most backup programs will not properly archive a file if it is in use. The policy should specify a backup time, and all users should know the days and times. where the backup tapes and media should be stored. This should be a secure, cool, and dry place. Some backups should be stored off campus which systems need backing up. Decide which systems require a regular backup and the frequency. what type of media should be used for backing up. This can include DAT tapes, re-writable CDs and DVD s, Flash ROM, removable hard drives connected to Firewire or USB ports, even another server or servers that exist exclusively for backups how to backup As a backup policy is developed, the decision on what needs to be backed up must be made. There must be a way to differentiate between the operating system files and user files. Generally, we will need to archive an operating system (including /, /usr, /var, and /etc) only when significant changes occur, such as when we install or update a daemon or kernel. We need to backup user files in /home much more frequently. In some conditions, these directories may become very large, so we must schedule major backups for off-peak backup times, as well as a strategy that backs up only those files that have changed since the last backup. Usually Friday or Saturday nights are good times to perform full backups. Selecting a Backup Strategy Archiving requires considerable time. In large systems, a single backup can take all night. Most administrators work to achieve a balance between long backup times and long restore times. We have several options to help achieve this balance, and we should choose the best strategy for our particular situation. Below are descriptions of the various backup options. Full Also known as level 0, archival, or epochal, a full backup is a complete archive of all files, including system and user files, on a drive or partition. This method requires one set of media, but generally takes the longest. This type is a mandatory part of all other strategies. Depending on some factor, this may be done on a weekly, monthly or quarterly basis. The frequency depends on how often system files are updated. Differential Known as a level 1 backup, all files that were created or modified since the last full backup are saved during a differential backup. Differential backups require two sets of tapes. The first tape contains the full backup. The second contains a backup of all files created or changed since the full backup. The second tape will grow larger, but not as large as the full backup tape. This option saves backup time,

2 but requires more restoring time, because we use only these two tapes. This option is usually the easiest to implement. Incremental Known as a level 2 backup, an incremental backup uses several sets of tapes to archive only those files with newer created or modified dates since the last backup. A new tape is used for each backup. Each new tape contains only files created or changed on the day of the backup. This option is more complex. When restoring, we must use the archival tape, then restore each tape for each day. This option allows us to spend the least time backing up, though it requires the most time when restoring. Incremental backups pose the threat of operator error and can force administrators into searching each tape for the proper archive. Network backup Most systems permit backing up filesystems from one machine onto a backup device connected to another machine--this is referred to as a remote or network backup. Examples of programs for remote dumps include rdump and rrestore. Disk Mirroring Disk Mirroring duplicates everything on a drive to another drive. It is usually common for a system of this type to update the duplicate drive continuously with the most current information. With RAID 1, the duplicate drive is written at the same time as the original. This reduces or eliminates the recovery time in case of a disaster. If the main drive fails, the backup drive can immediately take over. Many popular repositories of shareware and free software use disk mirroring. It also reduces incoming traffic since users can visit a local mirror to download software. Types of Backup Devices and Media Comparison of Backup Media Medium Capacity Reliability Cost Speed Magnetic Tape High High Cheap Slow Writable CD s Medium Medium Cheap Fast Hard Drive High High Expensive Fast Floppy Low Low Cheap Slow DVD High High Medium Slow ZIP Disks Medium Low Medium Slow Flash ROM Medium High Expensive Fast Firewire Hard Drive High High Expensive Fast USB Hard Drive High High Expensive Medium Tapes Tapes remain the most common archiving media. At one time, 9-mm tapes were the most common tape media, but today 4mm and 8mm DDS, DLT, and DAT tape cartridges, about the size of an audiocassette, are now standard. Tape carts can hold from 20GB to 600GB CD-ROMs Re-writable CD-ROMs have become popular, although they only hold up to 600 to 700 MB of information. Highly secure or frequently hacked sites use read-only CD ROMS to store log files to this

3 media, which ensures that a hacker cannot alter the data. A CD jukebox contains multiple read-write drives, which can then store up to five stacks of 100 CDs. With a jukebox, we can back up multiple machines across a network. The difference between a jukebox and a CD tower is that the former is meant for archiving, and the latter is meant for real-time network access. DVD-ROMs Similar to CD s except each disk stores much more data (4.7G and up depending on the type of drive). Some DVD Jukebox systems can hold up to 200 disks, and can write either CDs or DVDs Other Archiving Media SyQuest drives, Iomega ZIP, and JAZ drives can also be used. Generally, these devices can store from 100 MB to 1 GB per disk, but not a good choice for unattended backups. Firewire and USB or removable hard drives are also a possibility. The remainder of this handout will examine the commands to backup and restore all or part of a filesystem. Adding a Tape Drive Tape drives and removable drives are often used for system backups and archiving of important software. Removable drives can also be used to store a Linux file system. Numerous tape drives are supported by Linux through the ftape.o and st.o tape modules, found under the /lib/modules/2.0.xx/misc directory (where xx represents your kernel s version). The ftape module is a device driver used to support floppy controller tape or removable backup devices, whereas the st.o module supports SCSI tape devices. Both modules are automatically loaded and unloaded by the kerneld system daemon when needed during tape operation. Generally speaking, although nearly all SCSI and floppy controller tape drives should work with modern Linux kernels, version 2.0 and up, but many high-speed or parallel-port tapes may not. Installing a Tape Drive SCSI tape drives are either attached directly to an installed SCSI adapter or added to the current SCSI device chain. Make sure that a free device number is used, and that the SCSI chain is properly terminated. The devices used to support SCSI tape drives are found under the /dev directory and begin with the letters st: /dev/st0 /dev/st1 /dev/st2 /dev/st3 /dev/st4 /dev/st5 /dev/st6 /dev/st7 /dev/nst0 /dev/nst1 /dev/nst2 /dev/nst3 /dev/nst4 /dev/nst5 /dev/nst6 /dev/nst7 There are two types of tape drive devices: Rewinding Rewinding devices automatically rewind after tape operation. Nonrewinding Devices with names starting with n are nonrewinding tape devices, and will not rewind after tape operation. Floppy controller tape drives are attached to the spare floppy controller port. You cannot use your floppy drive when using a floppy tape drive. The devices used to support tape drives are found under the /dev directory and have the letters ft in their filenames: /dev/nrft0 /dev/nrft2 /dev/rft0 /dev/rft2 /dev/nrft1 /dev/nrft3 /dev/rft1 /dev/rft3

4 Creating a tape drive device symbolic link 1. Choose the device names, such as /dev/st0 or /dev/rft0, that match your SCSI or floppy tape drive. 2. As the root operator, use the ln command to create symbolic links for your devices, like so: ln -s /dev/nrft0 /dev/nftape ln -s /dev/rft0 /dev/ftape This command creates easily remembered names of your tape drive. Tapes must be purchased preformatted or must be formatted by using another operating system s utilities; at the time of this writing, no Linux utilities were available to format magnetic tapes. Using the mt Command The mt (magnetic tape) command is used to control tapes in your tape drive. These operations include the following: Rewinding the tape Retensioning the tape Erasing the tape More than two dozen SCSI-specific tape operations The mt command can also be used by backup scripts, or backup utilities such as the tar or cpio commands, to archive Linux directories and files. The mt command has several different commandline options, but is usually used with the -f option to specify the tape device followed by a command: mt -f /dev/nftape command Tape drives, unlike other file systems, are not mounted or unmounted for read or write operation. Retensioning Tapes The general consensus among Linux tape users is that it is a good idea to retension new tapes before use. Tapes should be retensioned before use to take up any existing slack, and to make sure the tape conforms to your tape drive s tension strength. This will help to reduce the risk of errors during backups. To retension a tape, insert the tape into the tape drive, then use the mt command with the nonrewinding tape drive device and the retension command: mt -f /dev/nftape retension Rewinding Tapes Although some tape drives automatically rewind inserted tapes, you might occasionally need to rewind a tape yourself, especially if a backup operation has been halted by software or operator intervention. To rewind a tape, use the mt command with the nonrewinding tape drive device and the rewind command: mt -f /dev/nftape rewind Preparing Tapes for Linux Backups Before using a formatted tape for backups, you must generally use the mt command to erase the tape and prepare it for Linux. To erase a tape, use the mt command with the nonrewinding tape drive device and the erase command: mt -f /dev/nftape erase

5 The dump and restore Commands There has been much controversy about the best application choice for the Linux environment, especially the use of dump at all. See dump is ideal for archiving entire file systems, though we can also archive selected files. It was originally developed for BSD UNIX, but is now been ported to all major versions of UNIX. It is ideal for incremental backups because we can set it to use and modify the /etc/dumpdates file, which lists the date and level of the latest backup. This information allows dump to perform differential backups. dump uses the syntax: dump option... argument... filesystem-to-backup AIX and Solaris use a similar syntax, but they name their dump programs backup and ufsdump, respectively. dump Options and Arguments Option Description Argument 0-9 Specifies the dump level The 0 option creates a full backup. A 9 option creates an incremental backup. dump uses a zero-based count, which means that it begins counting at 0, not at 1. Therefore, if we specify 1, we are specifying dump level 2. If we do not specify a level, dump defaults to 9. a archive-file archive-file will be a table of contents of the archive. f dump-file specify the file (usually a device file) to write the dump to, a specifies standard output u Updates /etc/dumpdates None v after writing each volume, rewind the tape and verify. The file system must not be used during dump or the verification. s Gives size of tape. Depends upon tape length. The 8-mm default is if combined with the c option. c Specifies the use of a tape cartridge. Depends on the size of the tape. This option is often used with s. d Specifies tape density. Recent versions of dump can autodetect tape lengths and density. Depends on density. For 8-mm, 42500, or 1000 when used with the c option. As we specify dump options and arguments, note that some flavors of UNIX do not permit hyphens between the commands. restore The restore command allows us to retrieve the files we backed up using dump. Solaris names the program ufsrestore. Using restore, we can retrieve single files or entire filesystems. The syntax for restore is: restore options arguments file/directory The options for restore include:

6 Option Description -i Launches restore in interactive mode. -r Reads and restores the full tape. -t Lists the files on the archive tape. -x Restores all files and directories into the current directory. -f Specifies the device to use. If not specified, the default setting applies. -s Selects the specific backup tape (for use on jukeboxes). -s n skip to the nth dump file on the tape -d turn on debugging -h prevent hierarchical restoration of subdirectories -v verbose mode -a archive-file use an archive file to search for a file s location. Convert contents of the dump tape to the new file system format -f dump-file specify dump-file to use, - refers to standard input Use of the dump Command to Archive a filesystem. As we dump file systems, we must specify the dump level, the device where we wish to put the archive, and the file system to back up. The dump level we specify helps we create epochal, differential, or incremental backups. The following table shows the special file name corresponding to a tape drive for various systems. System File name AIX /dev/rfd0 IRIX /dev/tape HP-UX /dev/rmt/0 Linux /dev/tape Solaris /dev/rmt0 SunOS and BSD dump ->../devices/pseudo/dump@0:dump An example of the dump command on a Linux system: dump 0uf /dev/rft0 /dev/hda Specifies the entire filesystem (level 0) 2. on /dev/hda3 3. should be backed up to mag tape (f option combined with destination device) 4. at /dev/rft0 5. and that the dumpdate (u option) should be written to /etc/dumpdates Below is an example of what we could do when dumping a file system to a device we specify: dump 1fu /dev/rmt/1 /dev/hda3 This command dumps the /dev/hda3 file system to the /dev/rmt/1 tape. The backup would be differential, because a 0 creates a full, or epochal backup, and a 1 archives only those files that had changed since the epochal backup. If we wished to do an incremental backup, we would specify a 2 for the next backup, and so on. The f option allows we to specify a non-default tape device, and the u option updates the /etc/dumpdates file.

7 When dumping file systems, we can use the -a option, which allows dump to write to the tape drive without verifying its size. However, only newer tape drives support this option. Also, the -b option allows we to specify block size, which is important for certain file systems. There are limitations with the dump command and we must take precautions when it is used. In addition, we should also archive any export directories (/export, /export/home) daily on mission-critical systems. The dump command does not support authentication procedures. We can automate backups using scripts, as well as crontab. Improper use of dump can lead to various disasters. First, because dump generally requires several options and arguments, we may make procedural errors. If we reverse dump syntax as root, we could restore the wrong file system on top of another. Also, if /etc/dumpdates entries are incorrect, or if we specify the wrong dump level, we probably will not back up the proper files. dump to standard output In addition to tape drives, we can dump to standard output. We denote standard output as a stand-alone hyphen: (-). The standard output option can be attractive if we have a large hard disk on a separate computer. We can then dump the root file system onto another partition, hard disk, or partition without having to worry about restoring it from tape. For example, suppose that we wanted to dump the root file system of the system we are working on to a separate hard drive partition, /backup, which resides on /dev/hda10. We could issue the following command to do so: ufsdump 0uf - / (cd backup ; restore xf- ) This command results in an epochal backup of the root file system into standard output (f and its argument, -), an updating of the /etc/dumpdates file (u), which is then piped into the backup directory. The contents of the backup directory are not in archival form; they are live files that we can move back to the repaired partition in the case of an event. Although tape backups are the most secure because we can store them offsite, this option has become quite popular because it is quick. Using the restore Command to Restore Individual Files To restore an archive, we generally reverse the steps we took to back it up. Therefore, recovering from a differential backup is different from recovering from an incremental backup. We must always restore backups in the order they were created, as follows: Differential: To recover from a differential backup, we would first restore the full (epochal) backup set. We would then restore the set of tapes we used to archive subsequent changes. This restoration can take considerable time. Incremental: Depending upon how often we back up, this option requires a full (epochal) backup set, then as many tapes as were used to conduct subsequent backups for each day. If we used six subsequent (sets of) tapes to increment the full backup, we will need each of these tapes. This restoration option requires the most backup time. Generally, we have to change to the destination directory before using restore. For example, to restore the /etc directory, we would change to it, then use restore with its options. With no options, restore will attempt to load the contents of the entire tape to the destination drive. Therefore, when restoring individual files, the interactive mode is the most useful. Interactive mode Interactive mode allows us to navigate through an archival device and specify exactly what file to restore from it.

8 In most systems, the following command allows us to search the default tape for an individual file manually: restore -i -f <device> The -i option provides a limited range of shell-type navigation options, including ls, cd, add (selects a file to be restored) and delete. Using standard UNIX directory-related commands like cd and ls for restoring the files and directories, use wildcards * and? in the usual way. Additional commands include help, quit, and extract, which extracts files to the destination directory after we have selected them. Non-interactive commands Non-interactive commands are available. For example, -s specifies an archive's number: restore -s 4 would restore the fourth archive to the current directory. The -x option allows us to restore a file in the current directory using relative path names. From the /etc directory, the following command would restore the passwd file from a default device to /etc: restore -x /etc/passwd Using the restore command to restore a filesystem If we have lost an entire filesystem, we must usually take certain steps before using restore. They include replacing damaged equipment and formatting drives using mkfs, for example. Once we have paved the way for the filesystem, we can restore it using one of several options. Recursive option The recursive option, -r, is particularly powerful. It allows us to first change to the partition we want to restore, then run restore: restore -r archive.name To restore a lost or damaged file system, you must first create the partition and directory. Assume we want to recreate the /home directory on hdb1 mkfs /dev/hdb1 mount /dev/hdb1/home cd /home restore rf /dev/rft0 The r option specifies that the entire archive on rft0 should be restored to the current directory. If we do not specify the -f option, restore will read from the default device, obtain the /backup/dev/hda1, and place it on the current partition. mt program If we have dumped multiple archives to a tape, we will probably need to access different areas of the tape. The mt program allows us to reposition the tape so that we can access the proper archives. mt syntax is: mt -t tapename command [count] mt -f /dev/rmt2 rewind would bring the tape back to the beginning. The cpio utility cpio was the original archiving utility. It has much of dump's functionality, but does not support backing up across a network. However, it is particularly useful when archiving entire file systems.

9 Program syntax is: Flags Options i extracts all files matching one or more of the given pattern arguments from the archive. o writes a new archive to the standard output, using the list of files read from the standard input b causes 16-bit words to be swapped within each long word and bytes to be swapped within each 16-bit word of each extracted file. d forces the creation of necessary intermediate directories if they do not already exist. v provides more verbose information than usual. cpio prints the names of files as it extracts them from or adds them to archives. The command to restore all files from the tape device /dev/rmt1 that have the word accounts in them is: cpio -icdv accounts < /dev/rmt1 cpio and find cpio is often used with find. The command: find /etc -print cpio -o >/etc/rmt1 copies all files in the /etc directory to the tape device /dev/rmt1. The -o option allows we to copy the files in the specified path using input to output. cpio then takes the redirected output and writes it to the device we specify.) The -p Option The -p option allows us to archive files from one directory to another. The command to archive all contents of /etc cpio -pd /etc /backup/ The dd Program dd is mainly used to create images of file systems. dd copies an input file and sends it to an output file. The program defaults to standard output if no arguments are given. Program syntax is: dd option=value For example, the command to archive the passwd file to the /dev/rmt1 archive is dd if=/etc/passwd dd of=/dev/rmt1 The tar command We have previously examined the use of tar. Its strength is archiving directory structures. The -c option creates a new tape, whereas the -r option appends new files to existing archives. Appending is a standard feature of most archiving programs. The following command would archive the /etc directory in a non-default tape device: tar cvf /dev/rmt0 /etc The pax Program pax is an improvement over tar and cpio that is used in many flavors of UNIX, but its chief strength lies in its ability to archive files within their original directory hierarchies. It supports tar and cpio formats, and introduces a few of its own. The instruction syntax is: pax options arguments

10 An example of this command that backs up the contents of /etc to the tape device /dev/rmt1 would be: pax -w -f /dev/rmt1 /etc The gzip Utility gzip is a common utility that compresses and decompresses files, leaving the.gz extension. Program syntax is: gzip option Other Backup and Restore Options Although dump is a standard utility, other utilities may be used to backup a system. Right now, the cpio and tar solutions are definitely considered much more reliable, and they all work on multiple filesystems. Another popular solution is AMANDA - Advanced Maryland Automatic Network Disk Archiver from the University of Maryland. The AMANDA server manages a pool of backup media and rotates usage through the pool in order to ensure that all backups are retained for the administrator specified retention period. All media is preformatted with data that allows AMANDA to detect whether the proper media is available or not. In addition, AMANDA can be interfaced with robotic media changing units, making it possible to completely automate backups. AMANDA can use either tar or dump to do the actual backups. Complete information on AMANDA can be found at dump and restore Example without a Tape Drive We do not have tape drives or similar backup media connected to our system. However, it is important to experiment with the dump and restore commands to gain an understanding of how they work. This handout offers an example that will allow you to use these commands without a tape drive. The method relies on the fact that Linux and UNIX accesses devices through files. A Practice File System For our experimentation with the commands, we will work with a practice file system. Practicing backups with hard-drive partitions is not that efficient as they will almost certainly be very large, and a small mistake could have disastrous consequences. Instead we are going to work with a floppy or ZIP drive. We will use a floppy disk in this example. The first step is to format a floppy with the ext2 file system. /sbin/mke2fs /dev/fd0 mke2fs 1.34 (25-Jul-2003) Filesystem label= OS type: Linux Block size=1024 (log=0) Fragment size=1024 (log=0) 96 inodes, 720 blocks 36 blocks (5.00%) reserved for the super user First data block=1 1 block group 8192 blocks per group, 8192 fragments per group 96 inodes per group Writing inode tables: done Writing superblocks and filesystem accounting information: done

11 This filesystem will be automatically checked every 27 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. We will now mount the drive and copy some files to the floppy. mount -t ext2 /dev/fd0 /mnt/floppy cp /etc/passwd /etc/issue /etc/group /var/log/messages /mnt/floppy cd /mnt/floppy ls group issue lost+found messages passwd Performing a level 0 dump We have copied some important files to the disk. Now we will backup the entire floppy volume to the /tmp directory on the hard drive. This is an example of a full backup. Here is an example of the commands with the output. The following is an explanation of the dump command that we will use: /sbin/dump 0uf /tmp/backup /mnt/floppy Specifies the entire filesystem (level 0) 7. on /mnt/floppy 8. should be backed up to the hard drive 9. to a file named backup in /tmp directory 10. and that the dumpdate (u option) should be written to /etc/dumpdates Now we will change to the root directory on fd0 and issue the dump command. cd / /sbin/dump 0uf /tmp/backup /mnt/floppy DUMP: Date of this level 0 dump: Thu Dec 9 10:04: DUMP: Dumping /dev/fd0 (/mnt/floppy) to /tmp/backup DUMP: Excluding inode 7 (resize inode) from dump DUMP: Label: none DUMP: Writing 10 Kilobyte records DUMP: mapping (Pass I) [regular files] DUMP: mapping (Pass II) [directories] DUMP: estimated 39 blocks. DUMP: Volume 1 started with block 1 at: Thu Dec 9 10:04: DUMP: dumping (Pass III) [directories] DUMP: dumping (Pass IV) [regular files] DUMP: Closing /tmp/backup DUMP: Volume 1 completed at: Thu Dec 9 10:04: DUMP: Volume 1 20 blocks (0.02MB) DUMP: 20 blocks (0.02MB) on 1 volume(s) DUMP: finished in less than a second DUMP: Date of this level 0 dump: Thu Dec 9 10:04: DUMP: Date this dump completed: Thu Dec 9 10:04: DUMP: Average transfer rate: 0 kb/s DUMP: DUMP IS DONE The entire filesystem on fd0 is now backed up. This process created a file on the hard drive of the server /tmp/backup, which contains a level 0 dump. The presence of the backup can be verified with the command: ls -l /tmp/backup -rw-r--r-- 1 root root Dec 9 10:04 /tmp/backup

12 Since we used the u option with dump, it also created a file named /etc/dumpdates. The contents of /etc/dumpdates is: /dev/fd0 0 Thu Dec 9 10:04: Restoring the backup Since we have a archive to work with, we could use the restore command to restore files. /sbin/restore -if /tmp/backup /sbin/restore > ls.: group issue lost+found/ messages passwd /sbin/restore > add passwd /sbin/restore > extract You have not read any volumes yet. Unless you know which volume your file(s) are on you should start with the last volume and work towards the first. Specify next volume # (none if no more volumes): 1 restoring./passwd set owner/mode for '.'? [yn] y /sbin/restore > quit Alternative Rather than backup to a normal file on the hard-drive you could choose to backup files directly to a floppy drive (i.e. use /dev/fd0 rather than /tmp/backup). One problem with this alternative is that you are limited to 1.44Mb per media. This used to be a problem because the Linux version of dump did not support multiple volumes. It appears it now does.