DNFS Workshop

Company: Oracle Corporation

Author: Esteban Bernal

Goal: The present document is to provide a clear example about How To

Setup NFS & DNFS for Oracle.

Requirements: 2 Linux Servers (Client & NFS server)

DNFS configuration steps:

1) Verify the Oracle OS user on all the nodes (NFS server & Client nodes) has the same OS id
and group id:

NFS server:

DB11G:oracle>hostname
dbaasm.us.oracle.com
DB11G:oracle>id
uid=501(oracle) gid=501(oinstall)
groups=501(oinstall),502(dba),503(oper),505(asmadmin)
context=user_u:system_r:unconfined_t

Client node:

DB11G:oracle>hostname
asmstorage.us.oracle.com

DB11G:oracle>id
uid=501(oracle) gid=501(oinstall) groups=501(oinstall),502(dba),503(oper)
context=user_u:system_r:unconfined_t

2) On dbaasm.us.oracle.com (NFS server), as root, create the directory for the oracle files and
set the ownership of this directory to this UID and the GID:

[root@dbaasm ~]# mkdir /oraclenfs

[root@dbaasm ~]# chown 501:501 /oraclenfs

[root@dbaasm ~]# ls -ld /oraclenfs
drwxr-xr-x 2 oracle oinstall 4096 Mar 6 09:46 /oraclenfs

3) Add it to the NFS exports /etc/exports file (NFS server):

/oraclenfs *(rw,sync,all_squash,anonuid=501,anongid=501)

4) Enable NFS service restart during the reboot (NFS server):

[root@dbaasm etc]# chkconfig --level 345 nfs on

5) Then explicitly start the NFS service (NFS server):

[root@dbaasm etc]# service nfs start

Starting NFS services: [ OK ]
Starting NFS quotas: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]

6) Verify that the /oraclenfs directory was correctly exported (NFS server):

[root@dbaasm etc]# exportfs -v

/oraclenfs <world>(rw,wdelay,root_squash,all_squash,anonuid=501,anongid=501)

7) Then mount the /oraclenfs directory on the client node (dbaasm.us.oracle.com):

7.1) Create a new directory (client node):

[root@asmstorage ~]# mkdir /nfsdir

[root@asmstorage ~]# ls -ld /nfsdir

drwxr-xr-x 2 root root 4096 Mar 6 09:28 /nfsdir

7.2) Update the /etc/fstab file to mount the /nfsdir during the reboot (client node):

dbaasm:/oraclenfs /nfsdir nfs

rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,vers=3,timeo=600 0 0

7.3) Explicitly mount the /nfsdir on dbaasm:/oraclenfs (client node):

[root@asmstorage ~]# mount /nfsdir

[root@asmstorage ~]# df -k /nfsdir
Filesystem 1K-blocks Used Available Use% Mounted on
dbaasm:/oraclenfs 20641792 15334976 4258176 79% /nfsdir
========================================
[root@asmstorage ~]# mount
.
.
.
dbaasm:/oraclenfs on /nfsdir type nfs
(rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,actimeo=0,nfsvers=3,timeo=600,ad
dr=10.141.161.150)
7.4) Confirm you can create files on that nfs directory on the Client node as Oracle OS user:

DB11G:oracle>hostname
asmstorage.us.oracle.com
DB11G:oracle>pwd
/nfsdir
DB11G:oracle>touch testfile
DB11G:oracle>ls -l testfile
-rw-r--r-- 1 oracle oinstall 0 Mar 6 2009 testfile
========================================
DB11G:oracle>hostname
dbaasm.us.oracle.com
DB11G:oracle>pwd
/oraclenfs
DB11G:oracle>ls -l
total 4
-rw-r--r-- 1 oracle oinstall 0 Mar 6 10:14 testfile

8) Configure the DNFS on the client node (asmstorage.us.oracle.com):

Note 1: Direct NFS Client can use a new configuration file oranfstab or the mount tab file
(/etc/mtab on Linux) to determine the mount point settings for NFS storage devices.

Note 2: Direct NFS searches for mount entries in the following order:
i. $ORACLE_HOME/dbs/oranfstab
ii. /etc/oranfstab
iii. /etc/mtab

8.1) Configure the oranfstab as follow (client node):

$ORACLE_HOME/dbs/oranfstab
server: dbaasm <=== NFS server Host name
path: 10.141.161.150 <--- First path to NFS server ie NFS server NIC
local: 10.141.161.151 <--- First client-side NIC
path: 10.141.161.150 <--- Second path to NFS server ie NFS server NIC (For load balance
purpose)
local: 10.141.161.151 <--- Second client-side NIC (For load balance purpose)
export: /oraclenfs mount: /nfsdir

DB11G:oracle>hostname
asmstorage.us.oracle.com

DB11G:oracle>vi $ORACLE_HOME/dbs/oranfstab

# I put the next lines and saved the file oranfstab:

server: dbaasm
path: 10.141.161.150
local: 10.141.161.151
path: 10.141.161.150
local: 10.141.161.151
export: /oraclenfs mount: /nfsdir

8.2) Enable the Direct NFS Client ODM Library:

DB11G:oracle>cd $ORACLE_HOME/lib
DB11G:oracle>mv libodm11.so libodm11.so_bak
DB11G:oracle>ln -s libnfsodm11.so libodm11.so
DB11G:oracle>ls -l libnfsodm11.so
-rw-r--r-- 1 oracle oinstall 44670 Aug 1 2007 libnfsodm11.so
DB11G:oracle>ls -l libodm11.so
lrwxrwxrwx 1 oracle oinstall 14 Mar 6 09:57 libodm11.so -> libnfsodm11.so

8.3) Restart the DB11G database to enable DNFS on this instance:

DB11G:oracle>echo $ORACLE_SID
DB11G
DB11G:oracle>sqlplus "/as sysdba"

SQL*Plus: Release 11.1.0.6.0 - Production on Fri Mar 6 09:58:51 2009

Copyright (c) 1982, 2007, Oracle. All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.1.0.6.0 - Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options

SQL> shutdown immediate

Database closed.
Database dismounted.
ORACLE instance shut down.
SQL> startup
ORACLE instance started.

Total System Global Area 1489829888 bytes

Fixed Size 1300016 bytes
Variable Size 905972176 bytes
Database Buffers 570425344 bytes
Redo Buffers 12132352 bytes
Database mounted.
Database opened.

8.4) The database instance (during the startup) should report, in the alert.log, that ODM is
enabled as follow:
========================================
.
.
Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 2.0
.
.
========================================

9) Then create a datafile/tablespace on the DNFS directory:

SQL> show parameter db_name

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------
db_name string DB11G

SQL> create tablespace dnfs_ts datafile '/nfsdir/dnfs_ts_01.dbf' size 100M;

Tablespace created.

SQL> !hostname
asmstorage.us.oracle.com

SQL> !ls -l /nfsdir/

total 102516
-rw-r----- 1 oracle oinstall 104865792 Mar 6 2009 dnfs_ts_01.dbf
========================================

DB11G:oracle>hostname
dbaasm.us.oracle.com

DB11G:oracle>cd /oraclenfs

DB11G:oracle>ls -l
total 102520
-rw-r----- 1 oracle oinstall 104865792 Mar 6 10:56 dnfs_ts_01.dbf

10) Finally, verify that DNFS is working:

SQL> select * from v$dnfs_servers;

ID SVRNAME DIRNAME MNTPORT NFSPORT WTMAX RTMAX

---------- ---------- --------------- ---------- ---------- ---------- --------
--
1 dbaasm /oraclenfs 738 2049 0 0