Re: NFS v4 and pNFS


In article <Hglmg.2153$Oe6.1732@xxxxxxxxxxxxxxxx>,
Rick Jones <rick.jones2@xxxxxx> writes:
|>
|> IIRC everything NFS sends is always a single datagram at the UDP
|> layer. It might be more accurate to say that when the NFS message is
|> larger than the MTU and so the IP datagram containing the UDP datagram
|> and NFS message is fragmented, if any one of the IP datagram fragments
|> for that UDP datagram are lost, the entire UDP datagram is lost and so
|> the NFS message must be resent.

OK.

|> I may have some of the math wrong (I slept too often in probstats) but
|> if we have a packet loss probability p, the probability of the packet
|> not being lost is (1-p) which means then that the probability of the
|> entire fragmented IP/UDP datagram carrying the NFS message not being
|> lost becomes (1-p)^N where N is the number of fragments. For
|> something like a 32678 byte mount, that means N is ~= 22. Now, _that_
|> can be rather nasty.

Grrk. Yes, the formula is correct, but the analysis isn't! Firstly,
p is normally VERY small - well below 10^-3 - which means that the
probability of the overall loss is still small. Secondly, that assumes
no correlation, and packet losses are often strongly correlated, which
(surprisingly) is likely to REDUCE the overall loss probability.

Yes, when things go sour, they can go VERY sour. But TCP itself isn't
reliable with a seriously unreliable transport, as its timeout-based
recovery is prone to fail horribly under certain conditions. I don't
know if that is a bug in the RFC or the Berkeley reference implementation,
or whether it is due to the timing/size constants having changed over
the years, but I have seen one particular syndrome on 3 wildly different
systems, with NOTHING in common in the hardware, transport or software
(except the design they wrote from, whether RFC or Berkeley). And I
have seen others where I suspect a similar effect.

I have less experience with NFS, but what experience I have is fairly
similar, even over UDP. Except the players are different :-)


Regards,
Nick Maclaren.
.

Relevant Pages

  • Re: Reproducible 2.6.11.9 NFS Kernel Crashing Bug!
    ... > NFS Version 3. ... The problem is that UDP is limited to a 64K datagram. ... RPC over UDP will not allow the use of multiple UDP ...
    (Linux-Kernel)
  • Re: serious networking (em) performance (ggate and NFS) problem
    ... November 2004 13:56 schrieb Robert Watson: ... > multiple NIC's, using polling can make quite a big difference, not just by ... My main problem seems to be the 3ware controller in combination with NFS. ... use UDP, TCP or nonstandard r/w-sizes. ...
    (freebsd-current)
  • Re: serious networking (em) performance (ggate and NFS) problem
    ... November 2004 13:56 schrieb Robert Watson: ... > multiple NIC's, using polling can make quite a big difference, not just by ... My main problem seems to be the 3ware controller in combination with NFS. ... use UDP, TCP or nonstandard r/w-sizes. ...
    (freebsd-stable)
  • Re: Firewall problems with NFS
    ... It seems to only allow use as an NFS client, since that worked fine when I tested it. ... U was surprised to see that TCP with tcp_adv_win_size=5 and rsize=8192 was as fast as UDP, ... 100005 1 udp 841 mountd ...
    (Fedora)
  • SuSE 10.0 NFS vs. Firewall
    ... I am attempting to get NFS working; both client and server are running ... 3/min burst 5 LOG level warning tcp-options ip-options prefix ... 3/min burst 5 state NEW udp dpt:sunrpc LOG level warning tcp-options ... 3/min burst 5 state NEW tcp dpt:sunrpc LOG level warning tcp-options ...
    (alt.os.linux.suse)