DRBD for SSD+USB
Summary: DRBD provides a nifty way of allowing a “RAID when available” setup, which offers more flexibility then MDADM. When in “RAID” mode, DRBD performs about 10% slower then MDADM, but provides near SSD performance when DRBD is not mirroring. Importantly you don’t need to have the USB drive connected all the time, so its great if you want to grab your laptop and have it mirror to your USB disk when you get back to base.
The Project
I have an HP Folio Laptop with 128 gig Samsung MZPA128 SSD drive built in, and a 2.5″ spinning disk connected to my PC over USB3 when at home – running Ubuntu. My research on SSD drives leads me to believe that they are about as unreliable as hard drives, but with the added disadvantage that when they fail, they normally fail catastrophically, unlike hard drives where you can often recover most data. While RAID is not a valid method of backing up, I want the “best of both worlds” in terms of performance and resiliency.
Tests and Benchmarks
The baseline tests for the SSD and HDD were performed using raw LVMs formatted with EXT4 (defaults).
Performance results for SSD: bonnie++ -s 8000 -u 1000:1000
folio 8000M 798 99 190836 20 95514 10 4230 99 256948 14 6882 170
folio 8000M 812 99 189668 18 95897 10 4213 99 257911 13 13106 262
folio 8000M 789 99 177182 18 95188 10 4149 99 253408 15 6537 163
Performance results for HDD: bonnie++ -s 8000 -u 1000:1000
folio 8000M 751 97 65353 9 25092 5 3675 97 78205 7 148.6 5
folio 8000M 771 96 66600 9 25452 5 4131 96 75136 7 149.3 5
MDADM
1 configuration stuff-up and complete reinstall later I have a raid array for testing built with the following command: mdadm –create /dev/md0 –level=mirror –write-behind=1024 –raid-devices=2 /dev/sda6 -W /dev/sdc1 -b /ssdgen/mdadm.bitmap
folio 8000M 805 98 33984 5 33185 5 3967 97 220468 12 2724 60
folio 8000M 783 99 34145 4 33485 5 4009 97 235770 12 2539 60
DRBD Protocol A
folio 8000M 761 98 31097 5 30644 6 3799 97 190543 14 2118 75
folio 8000M 798 99 31910 5 31631 6 4078 99 238925 12 2037 73
DRBD Protocol A, Unplugged
folio 8000M 811 99 110606 14 81152 8 4206 100 258376 13 2791 109
Setup to Support DRBD configuration.
I configured DRBD on top of LVM to allow the greatest flexibility with resizing file systems. The configuration details I used were as follows:
root@folio:/home/davidgo# pvs PV VG Fmt Attr PSize PFree /dev/sda5 intssd lvm2 a- 119.00g 0 /dev/sdc1 usbhdd lvm2 a- 298.09g 193.98g
root@folio:/home/davidgo# vgs VG #PV #LV #SN Attr VSize VFree intssd 1 3 0 wz--n- 119.00g 0 usbhdd 1 1 0 wz--n- 298.09g 193.98g
root@folio:/home/davidgo# lvs LV VG Attr LSize Origin Snap% Move Log Copy% Convert root intssd -wi-ao 13.04g ssd_drbd intssd -wi-ao 104.11g swap intssd -wi-ao 1.86g hdd_drbd usbhdd -wi-ao 104.11g
The DRBD Configuration is
global
{
usage-count no;
}
common
{
syncer
{
# Rate in Megabytes
rate 15M;
}
net
{
max-buffers 250;
}
}
resource drbd0
{
protocol A;
startup
{
become-primary-on folio;
}
on folio
{
device /dev/drbd0;
disk /dev/mapper/intssd-ssd_drbd;
address 127.0.0.1:7790;
meta-disk "internal";
}
on drbd2
{
device /dev/drbd1;
disk /dev/mapper/usbhdd-hdd_drbd;
address 127.0.0.1:7791;
meta-disk "internal";
}
}
resource drbd1
{
protocol A;
on folio
{
device /dev/drbd1;
disk /dev/mapper/usbhdd-hdd_drbd;
address 127.0.0.1:7791;
meta-disk "internal";
}
on drbd2
{
device /dev/drbd0;
disk /dev/mapper/intssd-ssd_drbd;
address 127.0.0.1:7790;
meta-disk "internal";
}
}
In order to get “Automatic RAID when available working”, I usedle a custom script and UDEV. The UDEV command will need to be modified to match your device, but mine looks as follows:
/etc/udev/rules.d/95-drbdusb.rules:
# Generated by inspecting output of
# udevadm info -a -p $(udevadm info -q path -n /dev/sdd1)
# We can use "parents" info as well, not only the sdd block
KERNEL=="sd?1", ATTRS{product}=="SK7301", ATTRS{serial}=="000020110813",RUN+="/usr/local/bin/reconnectdrbd"
/usr/local/reconnectdrbd:
#! /bin/bash /sbin/lvchange -a n /dev/usbhdd/hdd_drbd /sbin/vgexport -a sleep 2 /sbin/vgimport -a /sbin/lvchange -a y /dev/usbhdd/hdd_drbd /sbin/drbdadm attach drbd1
Remember to “chmod 755 /usr/local/reconnectdrbd”
If you have problems getting automatic recovery working (as I did), try running it manually after connecting the drive. If that works, check the udev rule is working and the script is being called (I found adding a line “/bin/date >> /tmp/debugme.log” and inspecting this file helped prove the udev rule was not matching).
This script almost certainly contains bits which are unnecessary, but its “good enough” for what I want to do. (The same can probably be said of the DRBD configuration !!!)