Subject: NFS on OSX Heads up info
From: TimY (timy@darkhorse.com)
Date: Tue Sep 12 2000 - 11:32:02 EDT
I just thought some people in this group might find this info useful
and interesting. Sorry if a bit off topic. I don't want this
posting to interrupt the netatalk info.
We have been testing NFS ver2 and ver3 on Mac OSX for about a month
now. We have really been hitting it hard. And I mean really hard.
Copying several gigs from several machines at one time 24 hours a day
7 days a week for a month straight. To our suprise it never crashed
once. We also tweaked with some settings on the OSX box (see below)
I am including an email that sent to the OSX mailing list from my
boss. I think everyone will find it interesting. Any questions you
may have please feel free to email me.
ORIGINAL EMAIL TO OSX MAILING BEGINS HERE
==========================================
We have been running some test here as we plan to start using the NFS
server of OS X Server under a good size load. We used the NFS server
way back in 1.0 but found it to crash the machine under heavy load.
(a netinfo problem I belive.) Things seem to be much more stable
today. We ran test for over a week without rebooting the server or
noticing any major problems. We did find a few strange things, and
I'd be interested in what comments you all might have.
========
nbuf=
========
We started looking at what tuning avenues we could explore to make it
as fast as possible. A while ago, I tinkered around with the
0200_Tune startup script. From what I've heard, the 'nbuf=' boot
argument increases the buffers used by the basic unix read and write
commands. When you change this value, the 'wired down' memory
allocation changes. My first guess would be that we might be able to
see some nfsd performance changes with the size of this wired down
buffer. But, No. It seems that this is not used at all.
========
nfsd -n
========
The -n option allows us to start several nfsd processes. This didn't
seem to effect things to much. Our test machine has no other load so
this might prove necessary once the machine has other stuff running
on it. The man page recommends one precess for each concurrent user.
The default value of 6 gave us slightly better performance than some
other values we tried.
========
ver2 || ver3 & proto= (Client side setup, Solaris)
========
These flags on our nfs client (Solaris) proved to have a big effect.
TCP protocol connections prove to be about 4-5 times faster than UDP.
The version 2 or 3 didn't have a very big effect. Version 2 looks
like it might be just slightly faster than version 3. We are still
opting for version 3 with hopes that it will be improved in areas
outside of our testing.
========
noac, actimeo= (Client side setup, Solaris)
========
These flags have major impacts on read performance. This will take
some more testing and a better understanding on how caching might
disturb our operation. The noac option, that disables caching, took
reads down to just a dribble.
========
pages free and inactive
========
This is very strange. I'm only guessing as to what the term
'inactive' means on this OS. It looks related to cached io
transactions such as mmap(2). For example, accessing part or all off
a large temp file allocates a bunch of memory as inactive. If more
active memory is needed for a processes, this memory is recycled. If
this file temp file is deleted, the inactive memory becomes free.
We noticed that this inactive memory would fluctuate wildly while we
were testing nfsd. We then wondered how bad would nfs performance
get if most of the memory was either wired-down or active (not
available to become inactive.)
We wired down as much memory as the OS would let us. Then we opened
lots of applications that would require a bunch of active memory
without much cpu. This left us with just a little memory that could
be used for inactive. To our surprise, nfs performance went up
15-40% !!! How could less memory mean better performance. The only
thing I can think of is that ram cache algorithm was creating extra
unneeded transactions that was slower than our raw disk.
========
bottom line
========
The nbuf= and srv= boot arguments seem unrelated to the performance of nfsd
UDP sucks for NFS. Use TCP instead.
Reducing client cache properties hurts performance, but might make
for more accurate operation in a multi-client environment.
NFSD under MacOS X Server slows down with more available memory.
(Yikes! I can't belive I'm saying this! Our testing was limited so
this may not apply across the board.)
Please comment if you have any ideas on this. My understanding of
nbuf and inactive memory could be wrong. Please explain if you know
more.
========
testing setup
========
Server
Apple G3 beige tower G3 300, 66MHz bus, 256MB ram
Asante 10/100 Ethernet (DECchip 21140)
Apple LVD scsi with 9GB 10k lvd drive.
Mac OS X Server 1.2
Client
Dual PentiumPro 166 512k (clocked to 200), 320MB ram
Intel EtherExpressPro 10/100
Adaptec 2940UW2 with multiple drives
Solaris 7
Network
100Mb half-duplex shared hub.
Average results from iozone 100 (older version, single write then read):
ver2,udp,default memory: 1.0 MB/s write, 1.8 MB/s read
ver3,tcp,defualt memory: 4.2 MB/s write, 3.1 MB/s read
ver3,tcp,default memory,actimeo=0: 4.2 MB/s write, 0.8 MB/s read
ver3,tcp,low memory free: 5.8 MB/s write, 4.7 MB/s read
--
This archive was generated by hypermail 2b28 : Wed Jan 17 2001 - 14:32:09 EST