Linux Journaling File System: ext3 Shangyou zeng Physics & Astronomy Dept., Ohio University Athens, OH, 45701

Transcription

1 Linux Journaling File System: ext3 Shangyou zeng Physics & Astronomy Dept., Ohio University Athens, OH, Abstract In Red Hat Linux 7.2, Red Hat provides the first officially supported journaling file system: ext3. The ext3 file system has incremental enhancements to the ext2 file system. It has several advantages: availability, data integrity, speed and easy transition. This paper summarizes some of those advantages, explain the detail of journaling file system, and discuss the methods that transfer ext2 file system to ext3 file system. 1.Introduction ext3 is a Journalizing file system for Linux. It was written by Dr Stephen C. Tweedie for 2.2 kernels. Peter Braam, Andreas Dilger and Andrew Morton ported the file system to 2.4 kernels with much valuable assistance from Stephen Tweedie Hurray. Now ext3 is merged into Linus kernel code from Kernel onwards. The ext3 file system is a journaling file system that is 100% compatible with all of the utilities created for creating, managing, and fine-tuning the ext2 file system, which is the default file system used by Linux systems for the last few years. 2.Advantages of Ext3 Why do we want to transfer the file system ext2 to the file system ext3? What are advantages of ext3 compared to ext2? There are four main reasons: availability, data integrity, speed, and easy transition. A. Availability After an unclean system shutdown (unexpected power failure, system crash), each ext2 file system needs to check the consistency by the e2fsck program. Only passing the consistency check, ext2 file system then can be mounted. The amount of time that the e2fsck program takes is mainly determined by the size of the file system. Today, the size of the file system is becoming larger and larger, then this takes a long time. The bigger the file system is, the longer the consistency check takes. File systems that are several hundreds of gigabytes in size may take an hour or more to check. Ext3 writes the data into disk in such a way that the file system is always consistent. Then, ext3 does not need a file system consistency check after an unclean system shutdown, except for certain rare hardware failure cases. The time to recover an ext3 file system after an unclean system shutdown does not depend on the size of the file system, but, it depends on the size of the journal used to maintain the file system consistency. The default journal size is very small, only takes about a second to recover.

2 B. Data Integrity The ext3 file system can provide stronger guarantees about data integrity in case of an unclean system shutdown. We can choose the type and level to protect the data received.essentially, you can choose either to keep the file system consistent but allow for damage to data on the file system in the case of unclean system shutdown (for a modest speed up under some but not all circumstances) or to ensure that the data is consistent with the state of the file system (which means that you will never see garbage data in recently-written files after a crash.) Ext3 has the same on-disk format as ext2 it can use the very well tested and reliable e2fsck to ensure file system integrity and recover from errors. The safe choice keeps the data consistent with the state of the file system, and it is default. C. Speed Because ext3 s journaling optimizes the hard drive head motion, ext3 is often faster than ext2, despite writing some data more than once. There are three journaling modes to optimize speed, choosing to trade off some data integrity. The first mode is data = writeback. This mode limits the data integrity guarantees, allow old data to show up in files after a crash, for a potential increase in speed under some circumstances. This mode is the default journaling mode for most journaling file systems. This mode provides more limited data integrity guarantees of the ext2 file system, and avoid the long file system check at the boot time. The second mode is data = ordered. This mode is the default mode. This mode guarantees that the data is consistent with the file system; recently-written files will never show up with garbage contents after a crash. The last mode is data=journal. This mode requires a larger journal for reasonable speed in most cases and therefore takes longer to recover in case of unclean shutdown, but is sometimes faster for certain database operations, NFS, and synchronous MTA (mail server) operations. The default mode is recommended for all general-purpose computing needs. To change the mode, add the data = something option to the mount options for that file system in the /etc/fstab file. D. Easy Transition It is easy to transfer ext2 to ext3 and gain the benefits of a robust journaling file system, without reformatting. There is no need to do a long, tedious and error-prone backupreformat-restore operation in order to experience the advantages of ext3. The tune2fs program can add a journal to an existing ext2 file system. If the file system is already

3 mounted while it is being transitioned, the journal will be visible as the file.journal in the root directory of the file system. If the file system is not mounted, the journal will be hidden and will not appear in the file system. Just run tune2fs j /dev/hda1. 3.What is Journaling? In order to minimize the file system inconsistencies and minimize system restart time after an unclean system shutdown, before the changes are actually made to the file system, journaling file system will keep track of the changes that will be made to the file system. The records of journaling file system changes are stored in a separate part of the file system, and the records are usually known as the journal or log. Once these journal records are safely written, the journaling file system applies these changes to the file system and then purges those records from the journaling record. Because journaling records are written before the file system changes are made, and because the file system keeps these file system change records until they have been safely and completely applied to the file system, journaling file systems maximize file system consistency and minimize system restart time after an unclean system shutdown. When a computer using journaling file system is rebooted, the mount program will check the journal record. If the journal records have some changes that are not marked as being done, then the changes will be applied to the file systems. The mount program then can guarantee the consistency of the file systems. In most cases, the system does not have to check the file system consistency, meaning that computers using journaling file systems are available almost instantly after rebooting them. The chances of losing data due to file system consistency problems are similarly reduced. There are many journaling file systems available for Linux. The best known is XFS. XFS is originally developed by Silicon Graphics but now released as open source. The ReiserFS is a journaling file system developed especially for Linux. JFS is a journaling file system originally developed by IBM but now released as open source. The ext3 file system is developed by Dr. Stephen Tweedie at Red Hat and a host of other contributors. 4. How does journaling work? The action of most serious database engines is a transaction. A transaction composes of a set of single operation that satisfy several properties. The most important property of transaction is the so-called ACID. ACID is the abbreviation of Atomicity, Consistency, Isolation and Durability. The most important feature related to the journaling file system is the Atomicity. This property implies that all operations belonging to a single transaction are completed without errors or cancelled, producing no changes. Combined with the Isolation, this property makes the transaction look as if they were atomic operation that can not be broken into parts. While exploiting concurrency in database, the transaction property solves the problem related to keeping consistency. In order to implement this property, database records every single operation within one transaction into a log file. In the log file, the operation name and operation argument s content before the operation s execution are logged. After every single transaction, the log buffer is

4 written into disks, which is so called commit operation. Therefore, if there is a system crash, we could trace the log back to the first commit statement, writing the argument s previous content back to its position in the disk. Journal file system uses the same technique above to log the file system operations, then the file system can be recoverable in a small period time after a system crash. There are some differences between databases and file system journaling. One major difference is that databases log users and control data, but file systems only log metadata. Metadata are the control structures inside a file system: I-nodes, free block allocation maps, I-nodes maps, etc. 5. Multiple Journaling Modes in the ext3 file system Ext3 file system is compatible with ext2 file system, and it is easy to convert ext2 file system to ext3 file system. The ext3 file system also offers several different types of journaling. A classic issue in journaling file systems is whether they only log changes to file system metadata or log changes to all file system data, including changes to files themselves. The ext3 file system supports three different journaling modes, which you can activate in the /etc/fstab entry for an ext3 file system. The three journaling modes are the following: Journal: This mode records all file system data and metadata changes. It is the slowest mode of the three ext3 journaling modes. This journaling mode minimizes the chance of losing the changes that have been made to any file in an ext3 file system. Ordered: This mode only records changes to file system metadata, but flushed file data updates to disk before making changes to associated file system metadata. This mode is the default ext3 journaling mode. Writeback: This mode only records changes to file system metadata. This mode relies on the standard file system write process to write file data changes to disk. This is the fastest ext3 journaling mode. The differences between these journaling modes are both subtle and profound. When the journal mode is used, it is necessary that ext3 file system writes every change to a file system twice, once to the journal, and then to the file system again. This can reduce the overall performance of the file system. This mode is the favorite mode of the users. This mode records both metadata and data updates in ext3 journal, which can be replayed in a system reboot, so it can minimize the chances of losing changes to the file. When the order mode is used, only file system metadata changes are logged. Then it reduces redundancy between writing to the file system and to the journal, so it is faster than the journaling mode. This mode does not record the changes to file data. In order to guarantee that files in the file system will never be out of synchronization with any

5 related changes to the file system metadata, the changes to file data must be done before associated file system metadata changes are made by the ext3 journaling daemon, which can reduce the performance of the file system. The writeback mode is faster than the other two ext3 journaling modes. This mode only records change to file system metadata and does not wait for associated changes to file data to be written before updating things like file size and directory information. Because updates to file data are not done synchronously to recorded changes to the file system metadata, files in the file system may exhibit metadata inconsistencies to which updated data was not written when the system went down. The ordered mode is the default mode of ext3 file system, but the different journaling mode can be specified by updating the file system option potion of an /etc/fstab entry. For example, if in order to specify the journal mode, an /etc/fstab entry would look like the following: /dev/hda5 /opt ext3 data=journaling Installing ext3 The ext3 file system is directly developed on the basis of its ancestor, ext2. Because ext3 file system is just the ext2 file system with journaling, it is convenient to transfer from ext2 file system to ext3 file system, and vice versa. The obvious shortcoming of ext3 file system is that it does not implement any modern file system features to increase data manipulation speed and packing. Ext3 comes as a patch of kernel of linux. So at first, we need get a linux kernel from ftp.kernel.org or from one of its mirrors. The patch is available at ftp.linux.org.uk/pub/linux/sct/fs/jfs or ftp.kernel.org/pub/linux/kernel/people/sct/ext3 or from one mirror of this site. In the above address, we need get the following files, ext a.tar.bz2: the kernel patch. e2fsprogs-1.21-wip-0601.tar.bz2: the e2fsprogs suite with ext3 support. Then we need to copy Linux kernel linux tar.bz2 and ext a.tar.bz2 files to /usr/src directory and extract them: mv linux linux-old tar -Ixvf linux tar.bz2 tar -Ixvf ext a.tar.bz2 cd linux cat../ext a/linux kdb.diff patch -sp1 cat../ext a/linux ext3.diff patch -sp1 After the kernel is compiled and installed, we need to make and install the e2fsprogs: tar -Ixvf e2fsprogs-1.21-wip-0601.tar.bz2 cd e2fsprogs-1.21./configure

6 make make check make install The next step is to make an ext3 file system in a partition. After rebooting with the new kernel, we have two options: make a new journaling file system or journal an existing one. In order to make a new ext3 file system, we just need use the mke2fs from the installed e2fsprogs, and use the -j option when running mke2fs. The command is following: mke2fs -j /dev/xxx where /dev/xxx is the device where you would create the ext3 file system. The "-j" flag tells mke2fs to create an ext3 file system with a hidden journal. You could control the size of the journal using the optional flag -Jsize=<n> (n is the preferred size of the journal in Mb). 7. Converting from Ext2 to Ext3 The conversion procedure from ext2 to ext3 is simple enough. Suppose /dev/hda10 is mounted as /test, then the procedure would be as follows: Log in as root Make sure /etc/fstab has /dev/hda10 mounted to /test as ext2, read write unmount /dev/hda10 o If /dev/hda10 can not be unmounted, then we need remount it as read only mode. The command is: mount -o remount,rw /dev/hda10 tune2fs -j /dev/hda10 Edit the file of /etc/fstab, and change ext2 to ext3 for the enrty /dev/hda10. mount /dev/hda10 /sbin/shutdown -h now mount grep /dev/hda10 o If it's not shown as ext3, reboot again, if still not shown as ext3, then need troubleshoot to fix the conversion. o If it is shown as ext3, then the conversion is finished. The tunefs command creates the journal file. The journal file is kept in a special inode on the device by default. You then need change the /etc/fstab entry from ext2 to ext3 to reflect it's a journaling file system, and then mount it again.

7 8. Conclusion The ext3 journaling file systems provide significant across the whole spectrum of Linux users. The file systems minimize delays when rebooting a Linux system after a computer crash, and improve the consistency of file systems. The ext3 file system is compatible with the ext2 file system, and it is easy to transfer ext2 file system to ext3 file system, and vice versa. This compatibility also extends the usability of all of the utilities that have already been developed for working with the ext2 file system. The ext3 file system is a true successful solution for improving the availability and consistency of Linux systems everywhere. Bibliography 1. Bill Von Hagen Rajesh Fowkar 4. Steve Litt 5. Matteo Dell'Omodarme 6. Michael K. Johnson 7. Juan I. Santos Florido 8. Moshe Bar