Sunday, February 22, 2009

The Obligatory RAID and Backup Post

This weekend I upgraded my back up server at home from mirrored 160GB disks to mirrored 1TB disks, so here's my obligatory "Linux RAID is insanely great" post:
Linux RAID is about as great as all other high quality RAID: It just plain works. It's not sexy, it's not going to get you borged by Google, but it will get the job done reliably with a minimum of fuss. Which is exactly what you want in your storage solution.
There are a lot of writeups about moving arrays around, so I'll skip the commentary and get to the "Mirrored disk migration HOWTO". This assumes you have two disks, sda and sdb being mirrored, and that your /boot is a separate ext3 filesystem that's being mirrored (eg: not in LVM)
  • grub-install /dev/sda
  • grub-install /dev/sdb
  • sfdisk -d /dev/sda > sda.partmap
  • sfdisk -d /dev/sdb > sdb.partmap
  • tar -cpvf boot-backup.tgz /boot
  • mdadm /dev/md0 --fail /dev/sda1 # wash rinse repeat for each md you have
  • halt
  • swap disk with nice shiny new disk
  • boot, if you get grub issues you probably need to do something like this at the grub prompt to get your menu back:
    # root (hd1,0)
    # configfile /grub/grub.conf
    then edit your boot line in the menu to make sure you boot off the other disk for this boot
  • sfdisk /dev/sda <>
  • mdadm /dev/md0 --re-add /dev/sda1 #wash, rinse, repeat for remaining partitions, watch /proc/mdstat for completion times
  • grub-install /dev/sda
  • Wash, Rinse, Repeat the above for /dev/sdb
  • At this point you can extend the last partition by blowing it away and making a new partition of larger size to use the rest of the disk and then use mdadm's --grow flag to make the md device use the new space. Alternatively, you can make a new md device with a new partition. I chose the latter because I then LVM the md device. yes, I know, that's very belt-and-suspenders of me and I could use the built in RAID features of LVM now, but hey, dm-raid was new when I did the original build of this system.
The important part to note here is that you need to make sure that you grub-install each disk before, during, and after, the migration. Otherwise you'll find yourself lacking an MBR a bootlaoder and sitting at a "PLEASE INSERT SYSTEM DISK AND PRESS ANY KEY TO CONTINUE" prompt. If you find yourself in that situation, just grab a LiveCD from your distro, boot it, and use grub-install from that like so (assuming your mirror of /boot is on sd{a,b}1):
  • mount -t ext2 /dev/sda1 /mnt
  • grub-install --root-dir=/mnt
  • umount /mnt
  • reboot
After that, it's a short hop, skip and a jump to use Time Machine to backup your Mac or something like rsnapshot, duplicity, Unison, BackupPC, or Bacula