Why

What You Do and See

Get familiar with command

$ man zpool
$ man zfs

structure and options

$ su
Password:
# cd /
# mkfile 100m disk1 disk2 disk3 disk5
# mkfile 50m disk4
# ls -l disk*
-rw------T
1 root
root
104857600 Sep 11 12:15
disk1
-rw------T
1 root
root
104857600 Sep 11 12:15
disk2
-rw------T
1 root
root
104857600 Sep 11 12:15
disk3
-rw------T
1 root
root
52428800 Sep 11 12:15
disk4
-rw------T
1 root
root
104857600 Sep 11 12:15
disk5
# zpool create myzfs /disk1 /disk2
# zpool list
NAME
SIZE
USED
AVAIL
CAP HEALTH
ALTROOT
myzfs
191M
94K
191M
0% ONLINE
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME
myzfs
/disk1
/disk2

STATE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0

Create some "virtual

devices" or vdevs as
described in the zpool
documentation. These can
also be real disk slices if you
have them available.

Create a storage pool and

check the size and usage.

Get more detailed status of

the zfs storage pool.

errors: No known data errors

# zpool destroy myzfs
# zpool list
no pools available

Destroy a zfs storage pool

Attempt to create a zfs pool
with different size vdevs

# zpool create myzfs mirror /disk1 /disk4

invalid vdev specification
use '-f' to override the following errors:
mirror contains devices of different sizes

fails. Using -f options forces

it to occur but only uses
space allowed by smallest
device.

# zpool
# zpool
NAME
ALTROOT
myzfs
# zpool
pool:
state:
scrub:

create myzfs mirror /disk1 /disk2 /disk3

list
SIZE
USED
AVAIL
CAP HEALTH
95.5M
112K
status -v
myzfs
ONLINE
none requested

95.4M

0%

ONLINE

Create a mirrored storage

pool. In this case, a 3 way
-

mirrored storage pool.

config:
NAME
myzfs
mirror
/disk1
/disk2
/disk3
errors:
# zpool
# zpool
pool:
state:
scrub:
config:

STATE
ONLINE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

No known data errors

detach myzfs /disk3
status -v
myzfs
ONLINE
none requested

Detach a device from a

NAME
myzfs
mirror
/disk1
/disk2

STATE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

# zpool attach myzfs /disk1 /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11
13:31:49 2007
config:
NAME
myzfs
mirror
/disk1
/disk2
/disk3

STATE
ONLINE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

mirrored pool.

Attach device to pool. This

creates a two-way mirror is
the pool is not already a
mirror, else it adds another
mirror, in this case making it
a 3 way mirror.

errors: No known data errors

# zpool remove myzfs /disk3
cannot remove /disk3: only inactive hot spares can be
removed
# zpool detach myzfs /disk3

Attempt to remove a device

from a pool. In this case it's
a mirror, so we must use
"zpool detach".

# zpool add myzfs spare /disk3

# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME
myzfs
mirror
/disk1
/disk2
spares
/disk3

STATE
ONLINE
ONLINE
ONLINE
ONLINE
AVAIL

Add a hot spare to a storage

pool.

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors:
# zpool
# zpool
pool:
state:
scrub:
config:

No known data errors

remove myzfs /disk3
status -v
myzfs
ONLINE
none requested

Remove a hot spare from a

NAME
myzfs
mirror
/disk1
/disk2

STATE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

# zpool offline myzfs /disk1
# zpool status -v
pool: myzfs
state: DEGRADED
status: One or more devices has been taken offline by
the administrator.
Sufficient replicas exist for the pool to
continue functioning
in a degraded state.
action: Online the device using 'zpool online' or
replace the device
with 'zpool replace'.
scrub: resilver completed with 0 errors on Tue Sep 11
13:39:25 2007
config:
NAME
myzfs
mirror
/disk1
/disk2

STATE
DEGRADED
DEGRADED
OFFLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

# zpool online myzfs /disk1
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11
13:47:14 2007
config:
NAME
myzfs
mirror
/disk1
/disk2

STATE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

# zpool replace myzfs /disk1 /disk3
# zpool status -v
pool: myzfs
state: ONLINE
scrub: resilver completed with 0 errors on Tue Sep 11
13:25:48 2007

pool.

Take the specified device

offline. No attempt to read
or write to the device will
take place until it's brought
back online. Use the -t
option to temporarily offline
a device. A reboot will bring
the device back online.

Bring the specified device

online.

Replace a disk in a pool with

another disk, for example
when a disk fails

config:
NAME
myzfs
mirror
/disk3
/disk2

STATE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

Perform a scrub of the

storage pool to verify that it
checksums correctly. On
mirror or raidz pools, ZFS

# zpool scrub myzfs

will automatically repair any

damage.
WARNING: scrubbing is I/O
intensive.
Export a pool from the

# zpool export myzfs

# zpool list
no pools available

system for importing on

another system.
Import a previously exported
storage pool. If -d is not

# zpool import -d / myzfs

# zpool list
NAME
SIZE
USED
ALTROOT
myzfs
95.5M
114K

specified, this command

AVAIL

CAP

HEALTH

95.4M

0%

ONLINE

searches /dev/dsk. As we're

-

using files in this example,

we need to specify the
directory of the files used by
the storage pool.

# zpool upgrade
This system is currently running ZFS pool version 8.
All pools are formatted using this version.
# zpool upgrade -v
This system is currently running ZFS pool version 8.
The following versions are supported:
VER DESCRIPTION
---------------------------------------------------------1
Initial ZFS version
2
Ditto blocks (replicated metadata)
3
Hot spares and double parity RAID-Z
4
zpool history
5
Compression using the gzip algorithm
6
pool properties
7
Separate intent log devices
8
Delegated administration
For more information on a particular version, including

Display pools format

version. The -v flag shows
the features supported by
the current version. Use the
-a flag to upgrade all pools
to the latest on-disk version.
Pools that are upgraded will
no longer be accessible to
any systems running older
versions.

supported
releases, see:
http://www.opensolaris.org/os/community/zfs/version/N
Where 'N' is the version number.
# zpool iostat 5
capacity
operations
pool
used avail
read write
---------- ----- ----- ----- ----myzfs
112K 95.4M
0
4
myzfs
112K 95.4M
0
0
myzfs
112K 95.4M
0
0

bandwidth
read write
----- ----26 11.4K
0
0
0
0

Get I/O statistics for the pool

Create a file system and

check it with standard df -h
command. File systems are
# zfs create myzfs/colin
# df -h
automatically mounted by
Filesystem
kbytes
used
avail capacity Mounted on
default under the /zfs
...
myzfs/colin 64M
18K
63M
1%
/myzfs/colin
location. See the
Mountpoints section of the
zfs man page for more
details.
# zfs list
NAME
USED
AVAIL REFER MOUNTPOINT
myzfs
139K 63.4M
19K /myzfs
myzfs/colin
18K 63.4M
18K /myzfs/colin
# zpool add myzfs /disk1
invalid vdev specification
use '-f' to override the following errors:
mismatched replication level: pool uses mirror and new
vdev is file
# zpool add myzfs mirror /disk1 /disk5
# zpool status -v
pool: myzfs
state: ONLINE
scrub: none requested
config:
NAME
myzfs
mirror
/disk3
/disk2
mirror
/disk1
/disk5

STATE
ONLINE
ONLINE
ONLINE
ONLINE
ONLINE
ONLINE
ONLINE

READ WRITE CKSUM

0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0

errors: No known data errors

# zfs create myzfs/colin2
# zfs list
NAME
USED AVAIL REFER
myzfs
172K
159M
21K
myzfs/colin
18K
159M
18K
myzfs/colin2
18K
159M
18K

List current zfs file systems.

Attempt to add a single vdev

to a mirrored set fails

Add a mirrored set of vdevs

Create a second file system.

MOUNTPOINT
/myzfs
/myzfs/colin
/myzfs/colin2

Note that both file system

show 159M available
because no quotas are set.

Each "could" grow to fill the

pool.
# zfs set reservation=20m myzfs/colin
# zfs list -o reservation
RESERV
none
20M
none

Reserve a specified amount

of space for a file system
ensuring that other users
don't take up all the space.

# zfs set quota=20m myzfs/colin2

# zfs list -o quota myzfs/colin myzfs/colin2
QUOTA
none
20M
# zfs set compression=on myzfs/colin2
# zfs list -o compression
COMPRESS
off
off
on

Set and view quotas

Turn on and verify

compression
Share a filesystem over NFS.

# zfs set sharenfs=on myzfs/colin2

# zfs get sharenfs myzfs/colin2
NAME PROPERTY VALUE
SOURCE
myzfs/colin2
sharenfs on
local

There is no need to modify

the /etc/dfs/dfstab as the
filesystem will be share
automatically on boot.
Share a filesystem over

# zfs set sharesmb=on myzfs/colin2

# zfs get sharesmb myzfs/colin2
NAME PROPERTY VALUE
SOURCE
myzfs/colin2
sharesmb on
local

CIFS/SMB. This will make

your ZFS filesystem
accessible to Windows
users.

# zfs snapshot myzfs/colin@test

# zfs list
NAME
USED AVAIL REFER
myzfs
20.2M
139M
21K
myzfs/colin
18K
159M
18K
myzfs/colin@test
0
18K
myzfs/colin2
18K 20.0M
18K

MOUNTPOINT
/myzfs
/myzfs/colin
/myzfs/colin2

# zfs rollback myzfs/colin@test

Create a snapshot called

test.

Rollback to a snapshot.

# zfs clone myzfs/colin@test myzfs/colin3

# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
20.2M
139M
21K /myzfs
myzfs/colin
18K
159M
18K /myzfs/colin
myzfs/colin@test
0
18K myzfs/colin2
18K 20.0M
18K /myzfs/colin2
myzfs/colin3
0
139M
18K /myzfs/colin3

A snapshot is not directly

addressable. A clone must
be made. The target dataset
can be located anywhere in
the ZFS hierarchy, and will
be created as the same type
as the original.

# zfs destroy myzfs/colin2

# zfs list
NAME
USED AVAIL

Destroy a filesystem
REFER

MOUNTPOINT

myzfs
myzfs/colin
myzfs/colin@test
myzfs/colin3

20.1M
18K
0
0

139M
159M
139M

22K
18K
18K
18K

/myzfs
/myzfs/colin
/myzfs/colin3

Attempt to destroy a
filesystem that had a child.
# zfs destroy myzfs/colin
cannot destroy 'myzfs/colin': filesystem has children
use '-r' to destroy the following datasets:
myzfs/colin@test

In this case, the snapshot

filesystem. We must either
remove the snapshot, or
make a clone and promote
the clone.

# zfs promote myzfs/colin3

# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
20.1M
139M
21K /myzfs
myzfs/colin
0
159M
18K /myzfs/colin
myzfs/colin3
18K
139M
18K /myzfs/colin3
myzfs/colin3@test
0
18K # zfs destroy myzfs/colin
# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
147K
159M
21K /myzfs
myzfs/colin3
18K
159M
18K /myzfs/colin3
myzfs/colin3@test
0
18K # zfs rename myzfs/colin3 myzfs/bob
# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
153K
159M
21K /myzfs
myzfs/bob
18K
159M
18K /myzfs/bob
myzfs/bob@test
0
18K # zfs rename myzfs/bob@test myzfs/bob@newtest
# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
146K
159M
20K /myzfs
myzfs/bob
18K
159M
18K /myzfs/bob
myzfs/bob@newtest
0
18K # zfs get all
NAME
PROPERTY
VALUE
SOURCE
myzfs
type
filesystem
myzfs
creation
Tue Sep 11 14:21 2007
myzfs
used
146K
myzfs
available
159M
myzfs
referenced
20K
[...]
# zpool destroy myzfs
cannot destroy 'myzfs': pool is not empty
use '-f' to force destruction anyway
# zfs unmount myzfs/bob
# df -h

Promte a clone filesystem to

no longer be a dependent on
it's "origin" snapshot. This
now associates makes the
snapshot a child of the
cloned filesystem. We can
then delete the original
filesystem.

Rename a filesystem, and

separately rename the
snapshot.

Display properties for the

given datasets. This can be
refined further using
options.

Can't destroy a pool with

active filesystems.
Unmount a ZFS file system

myzfs
/myzfs
# zfs mount myzfs/bob
# df -h
myzfs
/myzfs
myzfs/bob
/myzfs/bob

159M

20K

159M

1%

159M

20K

159M

1%

159M

18K

159M

1%

Mount a ZFS filesystem. This

is usually automatically
done on boot.
Create a stream

# zfs send myzfs/bob@newtest | ssh localhost zfs receive

myzfs/backup
# zfs list
NAME
USED AVAIL REFER MOUNTPOINT
myzfs
172K
159M
20K /myzfs
myzfs/backup
18K
159M
18K /myzfs/backup
myzfs/backup@newtest
0
18K myzfs/bob
18K
159M
18K /myzfs/bob
myzfs/bob@newtest
0
18K -

representation of the
snapshot and redirect it to
zfs receive. In this example
I've redirected to the
localhost for illustration
purposes. This can be used
to backup to a remote host,
or even to a local file.

# zpool history
History for 'myzfs':
2007-09-11.15:35:50 zpool create myzfs mirror /disk1
/disk2 /disk3
2007-09-11.15:36:00 zpool detach myzfs /disk3
2007-09-11.15:36:10 zpool attach myzfs /disk1 /disk3
2007-09-11.15:36:53 zpool detach myzfs /disk3
2007-09-11.15:36:59 zpool add myzfs spare /disk3
2007-09-11.15:37:09 zpool remove myzfs /disk3
2007-09-11.15:37:18 zpool offline myzfs /disk1
2007-09-11.15:37:27 zpool online myzfs /disk1
2007-09-11.15:37:37 zpool replace myzfs /disk1 /disk3
2007-09-11.15:37:47 zpool scrub myzfs
2007-09-11.15:37:57 zpool export myzfs
2007-09-11.15:38:05 zpool import -d / myzfs
2007-09-11.15:38:52 zfs create myzfs/colin
2007-09-11.15:39:27 zpool add myzfs mirror /disk1 /disk5
2007-09-11.15:39:38 zfs create myzfs/colin2
2007-09-11.15:39:50 zfs set reservation=20m myzfs/colin
2007-09-11.15:40:18 zfs set quota=20m myzfs/colin2
2007-09-11.15:40:35 zfs set compression=on myzfs/colin2
2007-09-11.15:40:48 zfs snapshot myzfs/colin@test
2007-09-11.15:40:59 zfs rollback myzfs/colin@test
2007-09-11.15:41:11 zfs clone myzfs/colin@test
myzfs/colin3
2007-09-11.15:41:25 zfs destroy myzfs/colin2
2007-09-11.15:42:12 zfs promote myzfs/colin3
2007-09-11.15:42:26 zfs rename myzfs/colin3 myzfs/bob
2007-09-11.15:42:57 zfs destroy myzfs/colin
2007-09-11.15:43:23 zfs rename myzfs/bob@test
myzfs/bob@newtest
2007-09-11.15:44:30 zfs receive myzfs/backup
# zpool destroy -f myzfs
# zpool status -v
no pools available

Display the command

history of all storage pools.
This can be limited to a
single pool by specifying its
name on the command line.
The history is only stored for
existing pools. Once you've
destroyed the pool, you'll no
longer have access to it's
history.

Use the -f option to destroy

a pool with files systems
created.