This article is meant to serve as a guide for migrating a live system from
ext3 to an
ext4 filesystem, including migration of files to use extents, a major feature in
ext4.
It describes the entire migration procedure, including common pitfalls
involving a migration of a live system, as opposed to doing a fresh
install.
Reasons for conversion
The explanation of advantages and disadvantages of
ext4 is beyond the scope of this article. If you are not affected by the limitations of
ext3,
and not willing to take risks, it may not be worth it. On the other
hand, on successful completion of the migration procedure your system
may perform faster, experience shortened file system checks, and have
increased reliability with no ill effects.
Trying ext4 without conversion
An interesting property worth noting is that you can mount
ext3 filesystems as
ext4 without doing the conversion first. All you need to do is to modify your
/etc/fstab to say "ext4" instead of "ext3" for all your filesystems, and reboot. This way you will be able to go back to
ext3 at any time.
This may be a nice test before attempting full conversion, but this will only enable
ext4 features that are compatible with
ext3. And that means that no major feature of
ext4, like extents, will be enabled.
Migrating to ext4
Word of warning
Do a backup before attempting this procedure. It may render your system unbootable, and may destroy your data.
Your filesystems will become incompatible with
ext3, so you need to make sure that you have a complete toolkit available with
ext4 support before doing a conversion. This includes a bootloader, e2fsprogs, mount, and a recent kernel.
Prerequisites
- e2fsprogs 1.41.6
- mount 2.16
- linux-image 2.6.30
- grub 1.96+20090808
All of these packages are available in Debian unstable or
experimental. Lower versions may work, except for e2fsprogs - this is
indeed the lowest version.
Converting a non-root filesystem to ext4
As long as you are converting a filesystem that can be unmounted, it
is fairly simple procedure. In this example we will be converting a
/dev/sdc1 partition mounted as
/home directory.
First, unmount the partition.
umount /dev/sdc1
Next, run a filesystem check on it to make sure it is in sane condition. We are still on
ext3.
fsck.ext3 -pf /dev/sdc1
Enable new features of
ext4 on the filesystem.
tune2fs -O extents,uninit_bg,dir_index /dev/sdc1
Option "extents" enables the filesystem to use extents instead of
bitmap mapping for files, "uninit_bg" reduces file system check times by
only checking used portions of the disk, and "dir_index" allows storing
the contents of large directories in a htree for faster access. Option
"dir_index" is also supported by
ext3, so you may already be using it, but it makes no harm to specify it here.
Run a filesystem check. It will find errors. It is normal. Let it fix
them. You may want to run the check twice to make sure that the
filesystem is now clean.
fsck.ext4 -yfD /dev/sdc1
The "-D" parameter will actually enable the "dir_index" option by
rebuilding directory index. It can be rebuilt (optimized) at any later
time by running the check with the parameter.
Now edit your
/etc/fstab file to say "ext4" instead of "ext3" for
/home. Other options may differ for your system.
/dev/sdc1 /home ext4 defaults 0 2
Try to mount your new
ext4 filesystem.
mount /home
If it succeeds, congratulations. If not, do not panic. You have not
lost your data. And you have a backup after all, right? Make sure you
have all the latest tools listed in prerequisites. Get them form Debian
unstable or experimental if needed. Upgrade and try again.
The /boot partition
If your
/boot is a separate partition, all is good and great. Just leave it as
ext3. Latest development
grub versions do have support for
ext4, but it still may and will fail for some given snapshot of
grub.
As
ext3 can be mounted
ext4 without conversion, you can just edit your
/etc/fstab to say "ext4" instead of "ext3" for boot partition.
/dev/sdb1 /boot ext4 defaults 0 1
Most features of
ext4 will not be used, but that makes little difference for
/boot, as it is only used early at boot time. And since this is essentially still an
ext3 partition,
grub will have no problem booting it.
If, on the other hand, you do not have a separate
/boot
partition, you should consider creating one. Otherwise you must be
really careful, and not enable features not supported by grub, or make
sure that you are using a version of grub that supports all of them.
Converting a root filesystem to ext4
Converting a root filesystem is a bit more tricky because you cannot
unmount it, as your system is running on it. Nevertheless it is still
possible to do it without using an external bootable media. You should
do this in a single-user mode.
First step is to modify your
/etc/fstab file to say "ext4"
instead of "ext3" for root partition. This is important because you will
be operating on a read-only filesystem later, and will not be able to
make the change, and this would result in your system unable to mount a
root filesystem on next boot.
Let us assume that root partition is
/dev/sda1, so your
/etc/fstab should look something like this.
/dev/sda1 / ext4 defaults 0 1
Now remount the root filesystem read-only.
mount -o remount,ro /
Then run a filesystem check on the root filesystem.
fsck.ext3 -pf /dev/sda1
It will tell you to reboot the system. That may be a good idea, so
simply boot into single-user mode and remount it read-only again. It is
fine even though we have already modified
/etc/fstab, because
ext3 can be mounted as
ext4 without conversion.
Next, enable all the
ext4 features on the root filesystem.
tune2fs -O extents,uninit_bg,dir_index /dev/sda1
And run run a filesystem check on the root filesystem again. It will find and fix errors. This is normal.
fsck.ext4 -yfD /dev/sda1
You can now reboot to your new
ext4 system, and enjoy faster filesystem check times, better performance, and all the improvements of
ext4. Well, almost; read the next section.
Migrating files to extents
It may seem that the migration from
ext3 to
ext4 is now
complete, and it is almost true. Except that any old files created
before the conversion will continue using the bitmap mapping of
ext3 instead of extents of ext4.
Files will eventually migrate to the new format as they are updated
during normal system operation, because on next write they will be saved
using extents. Unfortunately many frequently used files (like
application binaries) are often read and rarely written to. The outcome
is that the files will remain using the old format for a long time, and
you will not be able to experience full potential of
ext4.
A utility called
e4defrag, which would be able to migrate
files, is being developed. But unfortunately it is far complete, and is
not suitable for use on real data as of time of writing.
Fortunately it is possible to migrate separate files to extents by using
chattr utility, which comes with
e2fsprogs
package. It allows you to set an attribute on a file which causes the
kernel to rewrite the file using extents. It even possible to do this on
a mounted and working file system. In fact, that is the only way to do
it.
Please not that this feature is still experimental and has not been
tested thoroughly. Performing such operation may be dangerous. It may
also flood your system log with warnings and errors. You should first
test this on small number of insignificant files.
You can check if a file (or a directory) is using extents with
lsattr.
lsattr /home/user/foo/bar
If it not using extents, the output will be something like this.
------------------- /home/user/foo/bar
The dashed line here is simply a placeholder for various attributes a file can have. This means that it has no attributes.
Now set the attribute to use extents on the file.
chattr +e /home/user/foo/bar
Now list the attributes again, and you will notice that the output looks like this.
-----------------e- /home/user/foo/bar
Note the "e", which means that it is now using extents.
Check your system log and look for scary things. Not finding any is a good sign. That means you can continue.
Modifying attributes with
chattr can be done on multiple
files. Although digging trough the entire directory system is not really
feasible, so you can use some of the shell magic to accomplish the
task.
find /home -xdev -type f -print0 | xargs -0 chattr +e
find /home -xdev -type d -print0 | xargs -0 chattr +e
The first command will run "chattr +e" on all files in
/home, and the second will do the same for directories.
It is possible to run this on the root directory and convert
everything at once. But running this on one file system at a time with
"-xdev" parameter will prevent it from diving into file systems that do
not support extents. It may also be useful to run filesystem check on a
partition after conversion.
It may make sense to perform this in single-user mode to minimize the
chances of something interfering with a task. You may also want to
shutdown syslog, as it may generate a lot of warnings.