Oracle 10.2 RMAN Backup on NFS
You probably wouldn’t expect a technical Oracle post here.
But I can’t call it Oracle blog without relevant content so here it goes. Let’s call it a late birthday present — my first post came on the 2nd of May, 2006 so it’s now one year, one month and one week old.
Today, purely by accident, I came across Metalink Note 413098.1 Extremely Poor RMAN Backup Performance to NFS After Upgrade to 10.2 on Solaris. It seems that it also affects HP-UX at least.
What I love the best from that note is the workaround for RAC environments:
Do not write backups to NFS at 10.2. Backup to tape using Oracle Secure Backup or an alternative Media Manager or backup to a local disk drive.
Oracle Secure Backup?!?! Does it mean mean it has fewer bugs? Nice ad.
I.e. if my backup infrastructure is based on NFS - I’m screwed. The advice basically turns into “do not backup your databases - it’s slow”. Where are the other workarounds like stay on 10.1 or 9i? “Don’t use big databases” would be just as appropriate.
If I have time, I’m going to test it on Linux but if someone has interest - benchmark backups from 10.1. and 10.2 on NFS and let us know. Better yet, run it through strace and check the system calls.

I have experienced similar problem.
Actually I was not able to create a database (using DBCA), because RMAN was failing to restore and recover seed files ($ORACLE_HOME/assistants/dbca/templates/)
After the research I made on Metalink, I read about bug #5146667 and applied a patch to fix this bug.
Now RMAN backups run well for my 10.2.0.2 database on Solaris 5.10 (using NFS).
Mihajlo,
What’s your mount options? Did you applied a one-off patch to fix it rather than setting one special event to workaround?
Alex,
My database is a non-RAC database.
I have set my mount options according to Oracle Database Installation Guide - C.4 NFS Mount Options (in http://download-east.oracle.com/docs/cd/B19306_01/install.102/b15690/app_nas.htm#CHDIAGAD). I haven’t set noac option (which is mandatory only for RAC db).
I applied the patch for the bug #5146667 before I create the database.
I am using NFS to write my backups and so far I haven’t experienced any issues with that.
I don’t know if this patch can help to a RAC database.
I might be wrong, but I think the reason for the slow RMAN backups on NFS might be due to noac option specified. I have read in few white papers that this option may slow down the database performances. From the other hand it is mandatory for RAC configurations.
Alex,
We are currently doing some work with 10.2 RAC on NetApp NFS. Still playing a bit with the mount options. The docs (both NetApp and Oracle) are all over the place. It reads like people played with the options, got something to work, and published. Not much in the way of backing up the choices.
We found a lot of NFS calls when we had noac (though I can’t state for sure that was the cause). This was even when CRS was up but no db installed.
Oracle says to use noac with RAC for datafile autoextends. If the filesize changes, all nodes need to know ASAP. They also equate noac to actime=0 (in the doc referenced by Mihajlo). NetApp’s doc (http://www.netapp.com/library/tr/3527.pdf) has different settings for different mount points, but the difference is some have noac and others have actimeo=0. Huh??
Anyway, if RAC, noac, NFS was a serious problem I would have expected a lot more Google hits when searching. But as I said, I’m still playing and learning this stuff.
Alex, good to have met you in Edinburgh.
Henry
Hey Henry,
MSDBF 07 in Edinburgh was fantastic and it very nice to meet new people there.
Regarding NFS configuration - I’m wondering when Kevin is going to step in.
I’m obviously going to have to look into this and blog it.
First a question to Henry. Henry, what platform is the database server? It is very important to know because the mount option requirements are port-specific and sadly some ports require slightly different mount options for RMAN than the database itself.
Second, can you please cite where Oracle supposedly says this absurd comment about needing noac for RAC so they can get notified about file size changes? That is utterly absurd. RAC knows when the database changes with internal comms…DDL in RAC (that includes autoextend) requires GES. The only time noac was needed in my recollection was with junk like RH 3.0 to NAS…before the Linux client supported opens with the O_DIRECT flag.
And, ultimately all this goes away. Someday we’ll all be on 11g and NFS mount options are a complete thing of the past in that release when using NAS. I’ll blog about how and why as we get closer to July 11 launch.
I was always following Metalink Note 359515.1.
“noac” is required for CRS and voting disk. It does have noac stated for some platforms for datafiles and for Linux there is actimeo=0 but it’s not the same. Is it?
Let’s see. I’m running on RedHat RHEL4. So do the mount options depend on both the client and server?
The NetApp paper (#3527 Oracle 10g RAC release 2 Install with Power Linux RHEL AS4 U3 and NetApp Storage) talks about using noac or actimeo=0 on varying mount points (I don’t have the article at home and I can’t download it at the moment so I can check the details at work). Some mount points with noac and others with actimeo=0?? Nothing seems to be backed up, just stated. After reading other NetApp white papers where I had a better understanding of the material, I tend to read them fairly suspiciously (I’ll dig up some of my favorite lines when I get into work).
Oracle doc 279393.1 said noac is the same as setting sync, and actimeo=0 in RHEL3. For RHEL4, the recommendation is actimeo=0. This “Stems from Oracle behavior with autoextend, that the new filesize is not propagated to other nodes in the cluster. without this option, NFS will cache the old filesize and incorrect behavior will result. NOTE: Using actimeo=0 disables all NFS attribute caching. This is required when you have the possiblity that the file will be extended and the other nodes not notified. Currently, Oracle depends on filesystem messaging to advertise the changed size of datafiles, therefore this setting is necessary.”
OK, so for someone from the outside, who do I believe, and how do I test it? What does noac and actimeo really do for CRS? What is client dependent? server dependent? kernal? NFS version? The problem is that a simple misconfiguration can cause a whole technology stack (e.g. NFS) to be jettisoned (we are following what the vendor said, aren’t we?).
Kevin, I appreciate your input.
Henry
Thank you so much for this precious information.
Hi Alex,
We are using Ora 10.2.0.1.0 on RHEL 4. We are taking RMAN backup on NFS. It is a level 0 backup plus archivelog. I started the backup on 27/01/2011 at 21:00 Hrs. But it is still 50% done.
I have mounted the drive using mount -o hard,rw,noac,rsize=32768,wsize=32768,suid,proto=tcp,vers=3
130.131.2.64:/backup /backup/rmanBkp
Is there any solution?
@Mahesh: Before seeking for a solution, you need to find out what’s the bottleneck of your backup speed. It can be network infrastructure, CPU, low # of spindles, poor RAID choice and bunch of other reasons.
Hi Alex,
try this this way to mount
vim /etc/fstab
malinda:/home/oracle/archivelog /home/oracle/archivelog nfs rw,bg,hard,nointr,tcp,vers=3,timeo=300,rsize=32768,wsize=32768,actimeo=0 0 0
Thanks