Solaris,ZFS研究室Wiki - USBメモリ3本でraidzの実験
USBメモリ3本でZFSと同じUSBメモリを使って、今度はraidzでpoolを作成します。
# zpool create -f usbpool5 raidz c6t0d0 c7t0d0 c8t0d0
# zpool status usbpool5
  pool: usbpool5
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        usbpool5    ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  ONLINE       0     0     0
            c8t0d0  ONLINE       0     0     0

errors: No known data errors
# df -h /usbpool5
Filesystem             size   used  avail capacity  Mounted on
usbpool5               3.6G    25K   3.6G     1%    /usbpool5

ここから実験のために1GBのファイルを作成します。
# mkfile 1g /usbpool5/1G
# df -h /usbpool5
Filesystem             size   used  avail capacity  Mounted on
usbpool5               3.6G   982M   2.6G    27%    /usbpool5

その後、3本うち1本をいきなり抜きます。
c7t0d0が見えなくなります。
# rmformat
Looking for devices...
     1. Logical Node: /dev/rdsk/c6t0d0p0
        Physical Node: /pci@0,0/pci8086,464c@1d,7/storage@1/disk@0,0
        Connected Device: Generic  USB  SD Reader   1.00
        Device Type: Removable
        Bus: USB
        Size: 1.9 GB
        Label: <Unknown>
        Access permissions: <Unknown>
     2. Logical Node: /dev/rdsk/c8t0d0p0
        Physical Node: /pci@0,0/pci8086,464c@1d,7/storage@2/disk@0,0
        Connected Device: Kingmax  USB2.0 FlashDisk 0.00
        Device Type: Removable
        Bus: USB
        Size: 1.9 GB
        Label: <Unknown>
        Access permissions: <Unknown>

readアクセスしたらエラーがでました。ただしあくまでONLINE。
# dd if=/usbpool5/1G of=/dev/null bs=1024k count=100
100+0 records in
100+0 records out
# zpool status usbpool5
  pool: usbpool5
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        usbpool5    ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  ONLINE       3     0     0
            c8t0d0  ONLINE       0     0     0

errors: No known data errors

100MBほど書き込むとエラーが大量に出てREMOVEDとなりました。
# mkfile 100m /usbpool5/100m
# zpool status usbpool5
  pool: usbpool5
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        usbpool5    DEGRADED     0     0     0
          raidz1    DEGRADED     0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  REMOVED      0   598     0
            c8t0d0  ONLINE       0     0     0

errors: No known data errors

ここでUSBを挿しなおすと、再びc7t0d0と認識されたため、
挿した瞬間からscrub(RAIDの再構築)が自動的に行われました。
# zpool status usbpool5
  pool: usbpool5
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: resilver completed after 0h0m with 0 errors on Sun May 24 23:29:35 2009
config:

        NAME        STATE     READ WRITE CKSUM
        usbpool5    ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  ONLINE       0 1.54K     0  50.1M resilvered
            c8t0d0  ONLINE       0     0     0

errors: No known data errors

ここで注目は「50.1M resilvered」というところです。
2GBのメディアですが、全体を再構築するわけでもなければ、
以前書き込まれていた1GBのファイルを再書き込みするわけでもなく、
このメディア抜かれていた間に書き込まれた100MBのファイルの部分だけです。

もし全く新しいメディアでreplaceした場合でも、1GB+100MBの使用中部分だけの
再構築で済むのがZFSのいいところです。

最後に zpool clear をすれば元通りです。
# zpool clear usbpool5
# zpool status usbpool5
  pool: usbpool5
 state: ONLINE
 scrub: resilver completed after 0h0m with 0 errors on Sun May 24 23:29:35 2009
config:

        NAME        STATE     READ WRITE CKSUM
        usbpool5    ONLINE       0     0     0
          raidz1    ONLINE       0     0     0
            c6t0d0  ONLINE       0     0     0
            c7t0d0  ONLINE       0     0     0  50.1M resilvered
            c8t0d0  ONLINE       0     0     0

errors: No known data errors