€•‰Ý      Œsphinx.addnodes”Œdocument”“”)”}”(Œ	rawsource”Œ ”Œchildren”]”(Œtranslations”ŒLanguagesNode”“”)”}”(hhh]”(h Œpending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ
attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ	refdomain”Œstd”Œreftype”Œdoc”Œ	reftarget”Œ+/translations/zh_CN/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuŒtagname”hhhubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/zh_TW/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/it_IT/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/ja_JP/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/ko_KR/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubh)”}”(hhh]”hŒPortuguese (Brazilian)”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/pt_BR/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubh)”}”(hhh]”hŒSpanish”…””}”hh–sbah}”(h]”h ]”h"]”h$]”h&]”Œ	refdomain”h)Œreftype”h+Œ	reftarget”Œ+/translations/sp_SP/filesystems/nfs/localio”Œmodname”NŒ	classname”NŒrefexplicit”ˆuh1hhhubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h
hhŒ	_document”hŒsource”NŒline”NubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒNFS LOCALIO”h]”hŒNFS LOCALIO”…””}”(hh¼h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhh·h²hh³ŒE/var/lib/git/docbuild/linux/Documentation/filesystems/nfs/localio.rst”h´Kubh¶)”}”(hhh]”(h»)”}”(hŒOverview”h]”hŒOverview”…””}”(hhÎh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhhËh²hh³hÊh´KubhŒ	paragraph”“”)”}”(hXK  The LOCALIO auxiliary RPC protocol allows the Linux NFS client and
server to reliably handshake to determine if they are on the same
host. Select "NFS client and server support for LOCALIO auxiliary
protocol" in menuconfig to enable CONFIG_NFS_LOCALIO in the kernel
config (both CONFIG_NFS_FS and CONFIG_NFSD must also be enabled).”h]”hXO  The LOCALIO auxiliary RPC protocol allows the Linux NFS client and
server to reliably handshake to determine if they are on the same
host. Select â€œNFS client and server support for LOCALIO auxiliary
protocolâ€ in menuconfig to enable CONFIG_NFS_LOCALIO in the kernel
config (both CONFIG_NFS_FS and CONFIG_NFSD must also be enabled).”…””}”(hhÞh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KhhËh²hubhÝ)”}”(hŒÐOnce an NFS client and server handshake as "local", the client will
bypass the network RPC protocol for read, write and commit operations.
Due to this XDR and RPC bypass, these operations will operate faster.”h]”hŒÔOnce an NFS client and server handshake as â€œlocalâ€, the client will
bypass the network RPC protocol for read, write and commit operations.
Due to this XDR and RPC bypass, these operations will operate faster.”…””}”(hhìh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KhhËh²hubhÝ)”}”(hŒ¢The LOCALIO auxiliary protocol's implementation, which uses the same
connection as NFS traffic, follows the pattern established by the NFS
ACL protocol extension.”h]”hŒ¤The LOCALIO auxiliary protocolâ€™s implementation, which uses the same
connection as NFS traffic, follows the pattern established by the NFS
ACL protocol extension.”…””}”(hhúh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KhhËh²hubhÝ)”}”(hXz  The LOCALIO auxiliary protocol is needed to allow robust discovery of
clients local to their servers. In a private implementation that
preceded use of this LOCALIO protocol, a fragile sockaddr network
address based match against all local network interfaces was attempted.
But unlike the LOCALIO protocol, the sockaddr-based matching didn't
handle use of iptables or containers.”h]”hX|  The LOCALIO auxiliary protocol is needed to allow robust discovery of
clients local to their servers. In a private implementation that
preceded use of this LOCALIO protocol, a fragile sockaddr network
address based match against all local network interfaces was attempted.
But unlike the LOCALIO protocol, the sockaddr-based matching didnâ€™t
handle use of iptables or containers.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KhhËh²hubhÝ)”}”(hXÝ  The robust handshake between local client and server is just the
beginning, the ultimate use case this locality makes possible is the
client is able to open files and issue reads, writes and commits
directly to the server without having to go over the network. The
requirement is to perform these loopback NFS operations as efficiently
as possible, this is particularly useful for container use cases
(e.g. kubernetes) where it is possible to run an IO job local to the
server.”h]”hXÝ  The robust handshake between local client and server is just the
beginning, the ultimate use case this locality makes possible is the
client is able to open files and issue reads, writes and commits
directly to the server without having to go over the network. The
requirement is to perform these loopback NFS operations as efficiently
as possible, this is particularly useful for container use cases
(e.g. kubernetes) where it is possible to run an IO job local to the
server.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KhhËh²hubhÝ)”}”(hŒ‰The performance advantage realized from LOCALIO's ability to bypass
using XDR and RPC for reads, writes and commits can be extreme, e.g.:”h]”hŒ‹The performance advantage realized from LOCALIOâ€™s ability to bypass
using XDR and RPC for reads, writes and commits can be extreme, e.g.:”…””}”(hj$  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K&hhËh²hubhŒdefinition_list”“”)”}”(hhh]”(hŒdefinition_list_item”“”)”}”(hXƒ  fio for 20 secs with directio, qd of 8, 16 libaio threads:
- With LOCALIO:
  4K read:    IOPS=979k,  BW=3825MiB/s (4011MB/s)(74.7GiB/20002msec)
  4K write:   IOPS=165k,  BW=646MiB/s  (678MB/s)(12.6GiB/20002msec)
  128K read:  IOPS=402k,  BW=49.1GiB/s (52.7GB/s)(982GiB/20002msec)
  128K write: IOPS=11.5k, BW=1433MiB/s (1503MB/s)(28.0GiB/20004msec)

- Without LOCALIO:
  4K read:    IOPS=79.2k, BW=309MiB/s  (324MB/s)(6188MiB/20003msec)
  4K write:   IOPS=59.8k, BW=234MiB/s  (245MB/s)(4671MiB/20002msec)
  128K read:  IOPS=33.9k, BW=4234MiB/s (4440MB/s)(82.7GiB/20004msec)
  128K write: IOPS=11.5k, BW=1434MiB/s (1504MB/s)(28.0GiB/20011msec)
”h]”(hŒterm”“”)”}”(hŒ:fio for 20 secs with directio, qd of 8, 16 libaio threads:”h]”hŒ:fio for 20 secs with directio, qd of 8, 16 libaio threads:”…””}”(hj?  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1j=  h³hÊh´K4hj9  ubhŒ
definition”“”)”}”(hhh]”hŒbullet_list”“”)”}”(hhh]”(hŒ	list_item”“”)”}”(hX  With LOCALIO:
4K read:    IOPS=979k,  BW=3825MiB/s (4011MB/s)(74.7GiB/20002msec)
4K write:   IOPS=165k,  BW=646MiB/s  (678MB/s)(12.6GiB/20002msec)
128K read:  IOPS=402k,  BW=49.1GiB/s (52.7GB/s)(982GiB/20002msec)
128K write: IOPS=11.5k, BW=1433MiB/s (1503MB/s)(28.0GiB/20004msec)
”h]”hÝ)”}”(hX  With LOCALIO:
4K read:    IOPS=979k,  BW=3825MiB/s (4011MB/s)(74.7GiB/20002msec)
4K write:   IOPS=165k,  BW=646MiB/s  (678MB/s)(12.6GiB/20002msec)
128K read:  IOPS=402k,  BW=49.1GiB/s (52.7GB/s)(982GiB/20002msec)
128K write: IOPS=11.5k, BW=1433MiB/s (1503MB/s)(28.0GiB/20004msec)”h]”hX  With LOCALIO:
4K read:    IOPS=979k,  BW=3825MiB/s (4011MB/s)(74.7GiB/20002msec)
4K write:   IOPS=165k,  BW=646MiB/s  (678MB/s)(12.6GiB/20002msec)
128K read:  IOPS=402k,  BW=49.1GiB/s (52.7GB/s)(982GiB/20002msec)
128K write: IOPS=11.5k, BW=1433MiB/s (1503MB/s)(28.0GiB/20004msec)”…””}”(hj]  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K*hjY  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjT  ubjX  )”}”(hX  Without LOCALIO:
4K read:    IOPS=79.2k, BW=309MiB/s  (324MB/s)(6188MiB/20003msec)
4K write:   IOPS=59.8k, BW=234MiB/s  (245MB/s)(4671MiB/20002msec)
128K read:  IOPS=33.9k, BW=4234MiB/s (4440MB/s)(82.7GiB/20004msec)
128K write: IOPS=11.5k, BW=1434MiB/s (1504MB/s)(28.0GiB/20011msec)
”h]”hÝ)”}”(hX  Without LOCALIO:
4K read:    IOPS=79.2k, BW=309MiB/s  (324MB/s)(6188MiB/20003msec)
4K write:   IOPS=59.8k, BW=234MiB/s  (245MB/s)(4671MiB/20002msec)
128K read:  IOPS=33.9k, BW=4234MiB/s (4440MB/s)(82.7GiB/20004msec)
128K write: IOPS=11.5k, BW=1434MiB/s (1504MB/s)(28.0GiB/20011msec)”h]”hX  Without LOCALIO:
4K read:    IOPS=79.2k, BW=309MiB/s  (324MB/s)(6188MiB/20003msec)
4K write:   IOPS=59.8k, BW=234MiB/s  (245MB/s)(4671MiB/20002msec)
128K read:  IOPS=33.9k, BW=4234MiB/s (4440MB/s)(82.7GiB/20004msec)
128K write: IOPS=11.5k, BW=1434MiB/s (1504MB/s)(28.0GiB/20011msec)”…””}”(hju  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K0hjq  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjT  ubeh}”(h]”h ]”h"]”h$]”h&]”Œbullet”Œ-”uh1jR  h³hÊh´K*hjO  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jM  hj9  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1j7  h³hÊh´K4hj4  ubj8  )”}”(hX‚  fio for 20 secs with directio, qd of 8, 1 libaio thread:
- With LOCALIO:
  4K read:    IOPS=230k,  BW=898MiB/s  (941MB/s)(17.5GiB/20001msec)
  4K write:   IOPS=22.6k, BW=88.3MiB/s (92.6MB/s)(1766MiB/20001msec)
  128K read:  IOPS=38.8k, BW=4855MiB/s (5091MB/s)(94.8GiB/20001msec)
  128K write: IOPS=11.4k, BW=1428MiB/s (1497MB/s)(27.9GiB/20001msec)

- Without LOCALIO:
  4K read:    IOPS=77.1k, BW=301MiB/s  (316MB/s)(6022MiB/20001msec)
  4K write:   IOPS=32.8k, BW=128MiB/s  (135MB/s)(2566MiB/20001msec)
  128K read:  IOPS=24.4k, BW=3050MiB/s (3198MB/s)(59.6GiB/20001msec)
  128K write: IOPS=11.4k, BW=1430MiB/s (1500MB/s)(27.9GiB/20001msec)
”h]”(j>  )”}”(hŒ8fio for 20 secs with directio, qd of 8, 1 libaio thread:”h]”hŒ8fio for 20 secs with directio, qd of 8, 1 libaio thread:”…””}”(hj¡  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1j=  h³hÊh´KAhj  ubjN  )”}”(hhh]”jS  )”}”(hhh]”(jX  )”}”(hX  With LOCALIO:
4K read:    IOPS=230k,  BW=898MiB/s  (941MB/s)(17.5GiB/20001msec)
4K write:   IOPS=22.6k, BW=88.3MiB/s (92.6MB/s)(1766MiB/20001msec)
128K read:  IOPS=38.8k, BW=4855MiB/s (5091MB/s)(94.8GiB/20001msec)
128K write: IOPS=11.4k, BW=1428MiB/s (1497MB/s)(27.9GiB/20001msec)
”h]”hÝ)”}”(hX  With LOCALIO:
4K read:    IOPS=230k,  BW=898MiB/s  (941MB/s)(17.5GiB/20001msec)
4K write:   IOPS=22.6k, BW=88.3MiB/s (92.6MB/s)(1766MiB/20001msec)
128K read:  IOPS=38.8k, BW=4855MiB/s (5091MB/s)(94.8GiB/20001msec)
128K write: IOPS=11.4k, BW=1428MiB/s (1497MB/s)(27.9GiB/20001msec)”h]”hX  With LOCALIO:
4K read:    IOPS=230k,  BW=898MiB/s  (941MB/s)(17.5GiB/20001msec)
4K write:   IOPS=22.6k, BW=88.3MiB/s (92.6MB/s)(1766MiB/20001msec)
128K read:  IOPS=38.8k, BW=4855MiB/s (5091MB/s)(94.8GiB/20001msec)
128K write: IOPS=11.4k, BW=1428MiB/s (1497MB/s)(27.9GiB/20001msec)”…””}”(hj¹  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K7hjµ  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj²  ubjX  )”}”(hX  Without LOCALIO:
4K read:    IOPS=77.1k, BW=301MiB/s  (316MB/s)(6022MiB/20001msec)
4K write:   IOPS=32.8k, BW=128MiB/s  (135MB/s)(2566MiB/20001msec)
128K read:  IOPS=24.4k, BW=3050MiB/s (3198MB/s)(59.6GiB/20001msec)
128K write: IOPS=11.4k, BW=1430MiB/s (1500MB/s)(27.9GiB/20001msec)
”h]”hÝ)”}”(hX  Without LOCALIO:
4K read:    IOPS=77.1k, BW=301MiB/s  (316MB/s)(6022MiB/20001msec)
4K write:   IOPS=32.8k, BW=128MiB/s  (135MB/s)(2566MiB/20001msec)
128K read:  IOPS=24.4k, BW=3050MiB/s (3198MB/s)(59.6GiB/20001msec)
128K write: IOPS=11.4k, BW=1430MiB/s (1500MB/s)(27.9GiB/20001msec)”h]”hX  Without LOCALIO:
4K read:    IOPS=77.1k, BW=301MiB/s  (316MB/s)(6022MiB/20001msec)
4K write:   IOPS=32.8k, BW=128MiB/s  (135MB/s)(2566MiB/20001msec)
128K read:  IOPS=24.4k, BW=3050MiB/s (3198MB/s)(59.6GiB/20001msec)
128K write: IOPS=11.4k, BW=1430MiB/s (1500MB/s)(27.9GiB/20001msec)”…””}”(hjÑ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K=hjÍ  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj²  ubeh}”(h]”h ]”h"]”h$]”h&]”j  j  uh1jR  h³hÊh´K7hj¯  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jM  hj  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1j7  h³hÊh´KAhj4  h²hubeh}”(h]”h ]”h"]”h$]”h&]”uh1j2  hhËh²hh³Nh´Nubeh}”(h]”Œoverview”ah ]”h"]”Œoverview”ah$]”h&]”uh1hµhh·h²hh³hÊh´Kubh¶)”}”(hhh]”(h»)”}”(hŒFAQ”h]”hŒFAQ”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj  h²hh³hÊh´KDubhŒenumerated_list”“”)”}”(hhh]”(jX  )”}”(hX6  What are the use cases for LOCALIO?

a. Workloads where the NFS client and server are on the same host
   realize improved IO performance. In particular, it is common when
   running containerised workloads for jobs to find themselves
   running on the same host as the knfsd server being used for
   storage.
”h]”(hÝ)”}”(hŒ#What are the use cases for LOCALIO?”h]”hŒ#What are the use cases for LOCALIO?”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KFhj  ubj  )”}”(hhh]”jX  )”}”(hX  Workloads where the NFS client and server are on the same host
realize improved IO performance. In particular, it is common when
running containerised workloads for jobs to find themselves
running on the same host as the knfsd server being used for
storage.
”h]”hÝ)”}”(hX  Workloads where the NFS client and server are on the same host
realize improved IO performance. In particular, it is common when
running containerised workloads for jobs to find themselves
running on the same host as the knfsd server being used for
storage.”h]”hX  Workloads where the NFS client and server are on the same host
realize improved IO performance. In particular, it is common when
running containerised workloads for jobs to find themselves
running on the same host as the knfsd server being used for
storage.”…””}”(hj4  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KHhj0  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj-  ubah}”(h]”h ]”h"]”h$]”h&]”Œenumtype”Œ
loweralpha”Œprefix”hŒsuffix”Œ.”uh1j  hj  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³Nh´NubjX  )”}”(hXr  What are the requirements for LOCALIO?

a. Bypass use of the network RPC protocol as much as possible. This
   includes bypassing XDR and RPC for open, read, write and commit
   operations.
b. Allow client and server to autonomously discover if they are
   running local to each other without making any assumptions about
   the local network topology.
c. Support the use of containers by being compatible with relevant
   namespaces (e.g. network, user, mount).
d. Support all versions of NFS. NFSv3 is of particular importance
   because it has wide enterprise usage and pNFS flexfiles makes use
   of it for the data path.
”h]”(hÝ)”}”(hŒ&What are the requirements for LOCALIO?”h]”hŒ&What are the requirements for LOCALIO?”…””}”(hj]  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KNhjY  ubj  )”}”(hhh]”(jX  )”}”(hŒŒBypass use of the network RPC protocol as much as possible. This
includes bypassing XDR and RPC for open, read, write and commit
operations.”h]”hÝ)”}”(hŒŒBypass use of the network RPC protocol as much as possible. This
includes bypassing XDR and RPC for open, read, write and commit
operations.”h]”hŒŒBypass use of the network RPC protocol as much as possible. This
includes bypassing XDR and RPC for open, read, write and commit
operations.”…””}”(hjr  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KPhjn  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjk  ubjX  )”}”(hŒ™Allow client and server to autonomously discover if they are
running local to each other without making any assumptions about
the local network topology.”h]”hÝ)”}”(hŒ™Allow client and server to autonomously discover if they are
running local to each other without making any assumptions about
the local network topology.”h]”hŒ™Allow client and server to autonomously discover if they are
running local to each other without making any assumptions about
the local network topology.”…””}”(hjŠ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KShj†  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjk  ubjX  )”}”(hŒgSupport the use of containers by being compatible with relevant
namespaces (e.g. network, user, mount).”h]”hÝ)”}”(hŒgSupport the use of containers by being compatible with relevant
namespaces (e.g. network, user, mount).”h]”hŒgSupport the use of containers by being compatible with relevant
namespaces (e.g. network, user, mount).”…””}”(hj¢  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KVhjž  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjk  ubjX  )”}”(hŒšSupport all versions of NFS. NFSv3 is of particular importance
because it has wide enterprise usage and pNFS flexfiles makes use
of it for the data path.
”h]”hÝ)”}”(hŒ™Support all versions of NFS. NFSv3 is of particular importance
because it has wide enterprise usage and pNFS flexfiles makes use
of it for the data path.”h]”hŒ™Support all versions of NFS. NFSv3 is of particular importance
because it has wide enterprise usage and pNFS flexfiles makes use
of it for the data path.”…””}”(hjº  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KXhj¶  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjk  ubeh}”(h]”h ]”h"]”h$]”h&]”jN  jO  jP  hjQ  jR  uh1j  hjY  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³Nh´NubjX  )”}”(hX”  Why doesnâ€™t LOCALIO just compare IP addresses or hostnames when
deciding if the NFS client and server are co-located on the same
host?

Since one of the main use cases is containerised workloads, we cannot
assume that IP addresses will be shared between the client and
server. This sets up a requirement for a handshake protocol that
needs to go over the same connection as the NFS traffic in order to
identify that the client and the server really are running on the
same host. The handshake uses a secret that is sent over the wire,
and can be verified by both parties by comparing with a value stored
in shared kernel memory if they are truly co-located.
”h]”(hÝ)”}”(hŒˆWhy doesnâ€™t LOCALIO just compare IP addresses or hostnames when
deciding if the NFS client and server are co-located on the same
host?”h]”hŒˆWhy doesnâ€™t LOCALIO just compare IP addresses or hostnames when
deciding if the NFS client and server are co-located on the same
host?”…””}”(hjÞ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K\hjÚ  ubhÝ)”}”(hX	  Since one of the main use cases is containerised workloads, we cannot
assume that IP addresses will be shared between the client and
server. This sets up a requirement for a handshake protocol that
needs to go over the same connection as the NFS traffic in order to
identify that the client and the server really are running on the
same host. The handshake uses a secret that is sent over the wire,
and can be verified by both parties by comparing with a value stored
in shared kernel memory if they are truly co-located.”h]”hX	  Since one of the main use cases is containerised workloads, we cannot
assume that IP addresses will be shared between the client and
server. This sets up a requirement for a handshake protocol that
needs to go over the same connection as the NFS traffic in order to
identify that the client and the server really are running on the
same host. The handshake uses a secret that is sent over the wire,
and can be verified by both parties by comparing with a value stored
in shared kernel memory if they are truly co-located.”…””}”(hjì  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K`hjÚ  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hX*  Does LOCALIO improve pNFS flexfiles?

Yes, LOCALIO complements pNFS flexfiles by allowing it to take
advantage of NFS client and server locality.  Policy that initiates
client IO as closely to the server where the data is stored naturally
benefits from the data path optimization LOCALIO provides.
”h]”(hÝ)”}”(hŒ$Does LOCALIO improve pNFS flexfiles?”h]”hŒ$Does LOCALIO improve pNFS flexfiles?”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kihj   ubhÝ)”}”(hX  Yes, LOCALIO complements pNFS flexfiles by allowing it to take
advantage of NFS client and server locality.  Policy that initiates
client IO as closely to the server where the data is stored naturally
benefits from the data path optimization LOCALIO provides.”h]”hX  Yes, LOCALIO complements pNFS flexfiles by allowing it to take
advantage of NFS client and server locality.  Policy that initiates
client IO as closely to the server where the data is stored naturally
benefits from the data path optimization LOCALIO provides.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kkhj   ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hXÅ  Why not develop a new pNFS layout to enable LOCALIO?

A new pNFS layout could be developed, but doing so would put the
onus on the server to somehow discover that the client is co-located
when deciding to hand out the layout.
There is value in a simpler approach (as provided by LOCALIO) that
allows the NFS client to negotiate and leverage locality without
requiring more elaborate modeling and discovery of such locality in a
more centralized manner.
”h]”(hÝ)”}”(hŒ4Why not develop a new pNFS layout to enable LOCALIO?”h]”hŒ4Why not develop a new pNFS layout to enable LOCALIO?”…””}”(hj*  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kphj&  ubhÝ)”}”(hXŽ  A new pNFS layout could be developed, but doing so would put the
onus on the server to somehow discover that the client is co-located
when deciding to hand out the layout.
There is value in a simpler approach (as provided by LOCALIO) that
allows the NFS client to negotiate and leverage locality without
requiring more elaborate modeling and discovery of such locality in a
more centralized manner.”h]”hXŽ  A new pNFS layout could be developed, but doing so would put the
onus on the server to somehow discover that the client is co-located
when deciding to hand out the layout.
There is value in a simpler approach (as provided by LOCALIO) that
allows the NFS client to negotiate and leverage locality without
requiring more elaborate modeling and discovery of such locality in a
more centralized manner.”…””}”(hj8  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Krhj&  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hX„  Why is having the client perform a server-side file OPEN, without
using RPC, beneficial?  Is the benefit pNFS specific?

Avoiding the use of XDR and RPC for file opens is beneficial to
performance regardless of whether pNFS is used. Especially when
dealing with small files its best to avoid going over the wire
whenever possible, otherwise it could reduce or even negate the
benefits of avoiding the wire for doing the small file I/O itself.
Given LOCALIO's requirements the current approach of having the
client perform a server-side file open, without using RPC, is ideal.
If in the future requirements change then we can adapt accordingly.
”h]”(hÝ)”}”(hŒwWhy is having the client perform a server-side file OPEN, without
using RPC, beneficial?  Is the benefit pNFS specific?”h]”hŒwWhy is having the client perform a server-side file OPEN, without
using RPC, beneficial?  Is the benefit pNFS specific?”…””}”(hjP  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KzhjL  ubhÝ)”}”(hX
  Avoiding the use of XDR and RPC for file opens is beneficial to
performance regardless of whether pNFS is used. Especially when
dealing with small files its best to avoid going over the wire
whenever possible, otherwise it could reduce or even negate the
benefits of avoiding the wire for doing the small file I/O itself.
Given LOCALIO's requirements the current approach of having the
client perform a server-side file open, without using RPC, is ideal.
If in the future requirements change then we can adapt accordingly.”h]”hX  Avoiding the use of XDR and RPC for file opens is beneficial to
performance regardless of whether pNFS is used. Especially when
dealing with small files its best to avoid going over the wire
whenever possible, otherwise it could reduce or even negate the
benefits of avoiding the wire for doing the small file I/O itself.
Given LOCALIOâ€™s requirements the current approach of having the
client perform a server-side file open, without using RPC, is ideal.
If in the future requirements change then we can adapt accordingly.”…””}”(hj^  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K}hjL  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hX@  Why is LOCALIO only supported with UNIX Authentication (AUTH_UNIX)?

Strong authentication is usually tied to the connection itself. It
works by establishing a context that is cached by the server, and
that acts as the key for discovering the authorisation token, which
can then be passed to rpc.mountd to complete the authentication
process. On the other hand, in the case of AUTH_UNIX, the credential
that was passed over the wire is used directly as the key in the
upcall to rpc.mountd. This simplifies the authentication process, and
so makes AUTH_UNIX easier to support.
”h]”(hÝ)”}”(hŒCWhy is LOCALIO only supported with UNIX Authentication (AUTH_UNIX)?”h]”hŒCWhy is LOCALIO only supported with UNIX Authentication (AUTH_UNIX)?”…””}”(hjv  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K†hjr  ubhÝ)”}”(hXú  Strong authentication is usually tied to the connection itself. It
works by establishing a context that is cached by the server, and
that acts as the key for discovering the authorisation token, which
can then be passed to rpc.mountd to complete the authentication
process. On the other hand, in the case of AUTH_UNIX, the credential
that was passed over the wire is used directly as the key in the
upcall to rpc.mountd. This simplifies the authentication process, and
so makes AUTH_UNIX easier to support.”h]”hXú  Strong authentication is usually tied to the connection itself. It
works by establishing a context that is cached by the server, and
that acts as the key for discovering the authorisation token, which
can then be passed to rpc.mountd to complete the authentication
process. On the other hand, in the case of AUTH_UNIX, the credential
that was passed over the wire is used directly as the key in the
upcall to rpc.mountd. This simplifies the authentication process, and
so makes AUTH_UNIX easier to support.”…””}”(hj„  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kˆhjr  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hXZ  How do export options that translate RPC user IDs behave for LOCALIO
operations (eg. root_squash, all_squash)?

Export options that translate user IDs are managed by nfsd_setuser()
which is called by nfsd_setuser_and_check_port() which is called by
__fh_verify().  So they get handled exactly the same way for LOCALIO
as they do for non-LOCALIO.
”h]”(hÝ)”}”(hŒnHow do export options that translate RPC user IDs behave for LOCALIO
operations (eg. root_squash, all_squash)?”h]”hŒnHow do export options that translate RPC user IDs behave for LOCALIO
operations (eg. root_squash, all_squash)?”…””}”(hjœ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K‘hj˜  ubhÝ)”}”(hŒéExport options that translate user IDs are managed by nfsd_setuser()
which is called by nfsd_setuser_and_check_port() which is called by
__fh_verify().  So they get handled exactly the same way for LOCALIO
as they do for non-LOCALIO.”h]”hŒéExport options that translate user IDs are managed by nfsd_setuser()
which is called by nfsd_setuser_and_check_port() which is called by
__fh_verify().  So they get handled exactly the same way for LOCALIO
as they do for non-LOCALIO.”…””}”(hjª  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K”hj˜  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´NubjX  )”}”(hŒ¾How does LOCALIO make certain that object lifetimes are managed
properly given NFSD and NFS operate in different contexts?

See the detailed "NFS Client and Server Interlock" section below.
”h]”(hÝ)”}”(hŒzHow does LOCALIO make certain that object lifetimes are managed
properly given NFSD and NFS operate in different contexts?”h]”hŒzHow does LOCALIO make certain that object lifetimes are managed
properly given NFSD and NFS operate in different contexts?”…””}”(hjÂ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K™hj¾  ubhÝ)”}”(hŒASee the detailed "NFS Client and Server Interlock" section below.”h]”hŒESee the detailed â€œNFS Client and Server Interlockâ€ section below.”…””}”(hjÐ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kœhj¾  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jW  hj  h²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”jN  Œarabic”jP  hjQ  jR  uh1j  hj  h²hh³hÊh´KFubeh}”(h]”Œfaq”ah ]”h"]”Œfaq”ah$]”h&]”uh1hµhh·h²hh³hÊh´KDubh¶)”}”(hhh]”(h»)”}”(hŒRPC”h]”hŒRPC”…””}”(hjö  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjó  h²hh³hÊh´KŸubhÝ)”}”(hXš  The LOCALIO auxiliary RPC protocol consists of a single "UUID_IS_LOCAL"
RPC method that allows the Linux NFS client to verify the local Linux
NFS server can see the nonce (single-use UUID) the client generated and
made available in nfs_common. This protocol isn't part of an IETF
standard, nor does it need to be considering it is Linux-to-Linux
auxiliary RPC protocol that amounts to an implementation detail.”h]”hX   The LOCALIO auxiliary RPC protocol consists of a single â€œUUID_IS_LOCALâ€
RPC method that allows the Linux NFS client to verify the local Linux
NFS server can see the nonce (single-use UUID) the client generated and
made available in nfs_common. This protocol isnâ€™t part of an IETF
standard, nor does it need to be considering it is Linux-to-Linux
auxiliary RPC protocol that amounts to an implementation detail.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K¡hjó  h²hubhÝ)”}”(hŒÛThe UUID_IS_LOCAL method encodes the client generated uuid_t in terms of
the fixed UUID_SIZE (16 bytes). The fixed size opaque encode and decode
XDR methods are used instead of the less efficient variable sized
methods.”h]”hŒÛThe UUID_IS_LOCAL method encodes the client generated uuid_t in terms of
the fixed UUID_SIZE (16 bytes). The fixed size opaque encode and decode
XDR methods are used instead of the less efficient variable sized
methods.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K¨hjó  h²hubhÝ)”}”(hŒÂThe RPC program number for the NFS_LOCALIO_PROGRAM is 400122 (as assigned
by IANA, see https://www.iana.org/assignments/rpc-program-numbers/ ):
Linux Kernel Organization       400122  nfslocalio”h]”(hŒWThe RPC program number for the NFS_LOCALIO_PROGRAM is 400122 (as assigned
by IANA, see ”…””}”(hj   h²hh³Nh´NubhŒ	reference”“”)”}”(hŒ5https://www.iana.org/assignments/rpc-program-numbers/”h]”hŒ5https://www.iana.org/assignments/rpc-program-numbers/”…””}”(hj*  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œrefuri”j,  uh1j(  hj   ubhŒ6 ):
Linux Kernel Organization       400122  nfslocalio”…””}”(hj   h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K­hjó  h²hubhÝ)”}”(hŒ/The LOCALIO protocol spec in rpcgen syntax is::”h]”hŒ.The LOCALIO protocol spec in rpcgen syntax is:”…””}”(hjC  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´K±hjó  h²hubhŒliteral_block”“”)”}”(hŒ÷/* raw RFC 9562 UUID */
#define UUID_SIZE 16
typedef u8 uuid_t<UUID_SIZE>;

program NFS_LOCALIO_PROGRAM {
    version LOCALIO_V1 {
        void
            NULL(void) = 0;

        void
            UUID_IS_LOCAL(uuid_t) = 1;
    } = 1;
} = 400122;”h]”hŒ÷/* raw RFC 9562 UUID */
#define UUID_SIZE 16
typedef u8 uuid_t<UUID_SIZE>;

program NFS_LOCALIO_PROGRAM {
    version LOCALIO_V1 {
        void
            NULL(void) = 0;

        void
            UUID_IS_LOCAL(uuid_t) = 1;
    } = 1;
} = 400122;”…””}”hjS  sbah}”(h]”h ]”h"]”h$]”h&]”Œ	xml:space”Œpreserve”uh1jQ  h³hÊh´K³hjó  h²hubhÝ)”}”(hŒkLOCALIO uses the same transport connection as NFS traffic. As such,
LOCALIO is not registered with rpcbind.”h]”hŒkLOCALIO uses the same transport connection as NFS traffic. As such,
LOCALIO is not registered with rpcbind.”…””}”(hjc  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KÁhjó  h²hubeh}”(h]”Œrpc”ah ]”h"]”Œrpc”ah$]”h&]”uh1hµhh·h²hh³hÊh´KŸubh¶)”}”(hhh]”(h»)”}”(hŒ&NFS Common and Client/Server Handshake”h]”hŒ&NFS Common and Client/Server Handshake”…””}”(hj|  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjy  h²hh³hÊh´KÅubhÝ)”}”(hXú  fs/nfs_common/nfslocalio.c provides interfaces that enable an NFS client
to generate a nonce (single-use UUID) and associated short-lived
nfs_uuid_t struct, register it with nfs_common for subsequent lookup and
verification by the NFS server and if matched the NFS server populates
members in the nfs_uuid_t struct. The NFS client then uses nfs_common to
transfer the nfs_uuid_t from its nfs_uuids to the nn->nfsd_serv
clients_list from the nfs_common's uuids_list.  See:
fs/nfs/localio.c:nfs_local_probe()”h]”hXü  fs/nfs_common/nfslocalio.c provides interfaces that enable an NFS client
to generate a nonce (single-use UUID) and associated short-lived
nfs_uuid_t struct, register it with nfs_common for subsequent lookup and
verification by the NFS server and if matched the NFS server populates
members in the nfs_uuid_t struct. The NFS client then uses nfs_common to
transfer the nfs_uuid_t from its nfs_uuids to the nn->nfsd_serv
clients_list from the nfs_commonâ€™s uuids_list.  See:
fs/nfs/localio.c:nfs_local_probe()”…””}”(hjŠ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KÇhjy  h²hubhÝ)”}”(hXó  nfs_common's nfs_uuids list is the basis for LOCALIO enablement, as such
it has members that point to nfsd memory for direct use by the client
(e.g. 'net' is the server's network namespace, through it the client can
access nn->nfsd_serv with proper rcu read access). It is this client
and server synchronization that enables advanced usage and lifetime of
objects to span from the host kernel's nfsd to per-container knfsd
instances that are connected to nfs client's running on the same local
host.”h]”hXÿ  nfs_commonâ€™s nfs_uuids list is the basis for LOCALIO enablement, as such
it has members that point to nfsd memory for direct use by the client
(e.g. â€˜netâ€™ is the serverâ€™s network namespace, through it the client can
access nn->nfsd_serv with proper rcu read access). It is this client
and server synchronization that enables advanced usage and lifetime of
objects to span from the host kernelâ€™s nfsd to per-container knfsd
instances that are connected to nfs clientâ€™s running on the same local
host.”…””}”(hj˜  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KÐhjy  h²hubeh}”(h]”Œ&nfs-common-and-client-server-handshake”ah ]”h"]”Œ&nfs common and client/server handshake”ah$]”h&]”uh1hµhh·h²hh³hÊh´KÅubh¶)”}”(hhh]”(h»)”}”(hŒNFS Client and Server Interlock”h]”hŒNFS Client and Server Interlock”…””}”(hj±  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj®  h²hh³hÊh´KÚubhÝ)”}”(hŒˆLOCALIO provides the nfs_uuid_t object and associated interfaces to
allow proper network namespace (net-ns) and NFSD object refcounting.”h]”hŒˆLOCALIO provides the nfs_uuid_t object and associated interfaces to
allow proper network namespace (net-ns) and NFSD object refcounting.”…””}”(hj¿  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KÜhj®  h²hubhÝ)”}”(hŒùLOCALIO required the introduction and use of NFSD's percpu nfsd_net_ref
to interlock nfsd_shutdown_net() and nfsd_open_local_fh(), to ensure
each net-ns is not destroyed while in use by nfsd_open_local_fh(), and
warrants a more detailed explanation:”h]”hŒûLOCALIO required the introduction and use of NFSDâ€™s percpu nfsd_net_ref
to interlock nfsd_shutdown_net() and nfsd_open_local_fh(), to ensure
each net-ns is not destroyed while in use by nfsd_open_local_fh(), and
warrants a more detailed explanation:”…””}”(hjÍ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kßhj®  h²hubhŒblock_quote”“”)”}”(hXç  nfsd_open_local_fh() uses nfsd_net_try_get() before opening its
nfsd_file handle and then the caller (NFS client) must drop the
reference for the nfsd_file and associated net-ns using
nfsd_file_put_local() once it has completed its IO.

This interlock working relies heavily on nfsd_open_local_fh() being
afforded the ability to safely deal with the possibility that the
NFSD's net-ns (and nfsd_net by association) may have been destroyed
by nfsd_destroy_serv() via nfsd_shutdown_net().
”h]”(hÝ)”}”(hŒënfsd_open_local_fh() uses nfsd_net_try_get() before opening its
nfsd_file handle and then the caller (NFS client) must drop the
reference for the nfsd_file and associated net-ns using
nfsd_file_put_local() once it has completed its IO.”h]”hŒënfsd_open_local_fh() uses nfsd_net_try_get() before opening its
nfsd_file handle and then the caller (NFS client) must drop the
reference for the nfsd_file and associated net-ns using
nfsd_file_put_local() once it has completed its IO.”…””}”(hjá  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KähjÝ  ubhÝ)”}”(hŒùThis interlock working relies heavily on nfsd_open_local_fh() being
afforded the ability to safely deal with the possibility that the
NFSD's net-ns (and nfsd_net by association) may have been destroyed
by nfsd_destroy_serv() via nfsd_shutdown_net().”h]”hŒûThis interlock working relies heavily on nfsd_open_local_fh() being
afforded the ability to safely deal with the possibility that the
NFSDâ€™s net-ns (and nfsd_net by association) may have been destroyed
by nfsd_destroy_serv() via nfsd_shutdown_net().”…””}”(hjï  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´KéhjÝ  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1jÛ  h³hÊh´Kähj®  h²hubhÝ)”}”(hX¶  This interlock of the NFS client and server has been verified to fix an
easy to hit crash that would occur if an NFSD instance running in a
container, with a LOCALIO client mounted, is shutdown. Upon restart of
the container and associated NFSD, the client would go on to crash due
to NULL pointer dereference that occurred due to the LOCALIO client's
attempting to nfsd_open_local_fh() without having a proper reference on
NFSD's net-ns.”h]”hXº  This interlock of the NFS client and server has been verified to fix an
easy to hit crash that would occur if an NFSD instance running in a
container, with a LOCALIO client mounted, is shutdown. Upon restart of
the container and associated NFSD, the client would go on to crash due
to NULL pointer dereference that occurred due to the LOCALIO clientâ€™s
attempting to nfsd_open_local_fh() without having a proper reference on
NFSDâ€™s net-ns.”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kîhj®  h²hubeh}”(h]”Œnfs-client-and-server-interlock”ah ]”h"]”Œnfs client and server interlock”ah$]”h&]”uh1hµhh·h²hh³hÊh´KÚubh¶)”}”(hhh]”(h»)”}”(hŒ&NFS Client issues IO instead of Server”h]”hŒ&NFS Client issues IO instead of Server”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj  h²hh³hÊh´K÷ubhÝ)”}”(hŒÈBecause LOCALIO is focused on protocol bypass to achieve improved IO
performance, alternatives to the traditional NFS wire protocol (SUNRPC
with XDR) must be provided to access the backing filesystem.”h]”hŒÈBecause LOCALIO is focused on protocol bypass to achieve improved IO
performance, alternatives to the traditional NFS wire protocol (SUNRPC
with XDR) must be provided to access the backing filesystem.”…””}”(hj*  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kùhj  h²hubhÝ)”}”(hŒüSee fs/nfs/localio.c:nfs_local_open_fh() and
fs/nfsd/localio.c:nfsd_open_local_fh() for the interface that makes
focused use of select nfs server objects to allow a client local to a
server to open a file pointer without needing to go over the network.”h]”hŒüSee fs/nfs/localio.c:nfs_local_open_fh() and
fs/nfsd/localio.c:nfsd_open_local_fh() for the interface that makes
focused use of select nfs server objects to allow a client local to a
server to open a file pointer without needing to go over the network.”…””}”(hj8  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Kýhj  h²hubhÝ)”}”(hX^  The client's fs/nfs/localio.c:nfs_local_open_fh() will call into the
server's fs/nfsd/localio.c:nfsd_open_local_fh() and carefully access
both the associated nfsd network namespace and nn->nfsd_serv in terms of
RCU. If nfsd_open_local_fh() finds that the client no longer sees valid
nfsd objects (be it struct net or nn->nfsd_serv) it returns -ENXIO
to nfs_local_open_fh() and the client will try to reestablish the
LOCALIO resources needed by calling nfs_local_probe() again. This
recovery is needed if/when an nfsd instance running in a container were
to reboot while a LOCALIO client is connected to it.”h]”hXb  The clientâ€™s fs/nfs/localio.c:nfs_local_open_fh() will call into the
serverâ€™s fs/nfsd/localio.c:nfsd_open_local_fh() and carefully access
both the associated nfsd network namespace and nn->nfsd_serv in terms of
RCU. If nfsd_open_local_fh() finds that the client no longer sees valid
nfsd objects (be it struct net or nn->nfsd_serv) it returns -ENXIO
to nfs_local_open_fh() and the client will try to reestablish the
LOCALIO resources needed by calling nfs_local_probe() again. This
recovery is needed if/when an nfsd instance running in a container were
to reboot while a LOCALIO client is connected to it.”…””}”(hjF  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mhj  h²hubhÝ)”}”(hX{  Once the client has an open nfsd_file pointer it will issue reads,
writes and commits directly to the underlying local filesystem (normally
done by the nfs server). As such, for these operations, the NFS client
is issuing IO to the underlying local filesystem that it is sharing with
the NFS server. See: fs/nfs/localio.c:nfs_local_doio() and
fs/nfs/localio.c:nfs_local_commit().”h]”hX{  Once the client has an open nfsd_file pointer it will issue reads,
writes and commits directly to the underlying local filesystem (normally
done by the nfs server). As such, for these operations, the NFS client
is issuing IO to the underlying local filesystem that it is sharing with
the NFS server. See: fs/nfs/localio.c:nfs_local_doio() and
fs/nfs/localio.c:nfs_local_commit().”…””}”(hjT  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mhj  h²hubhÝ)”}”(hXG  With normal NFS that makes use of RPC to issue IO to the server, if an
application uses O_DIRECT the NFS client will bypass the pagecache but
the NFS server will not. The NFS server's use of buffered IO affords
applications to be less precise with their alignment when issuing IO to
the NFS client. But if all applications properly align their IO, LOCALIO
can be configured to use end-to-end O_DIRECT semantics from the NFS
client to the underlying local filesystem, that it is sharing with
the NFS server, by setting the 'localio_O_DIRECT_semantics' nfs module
parameter to Y, e.g.:”h]”hXM  With normal NFS that makes use of RPC to issue IO to the server, if an
application uses O_DIRECT the NFS client will bypass the pagecache but
the NFS server will not. The NFS serverâ€™s use of buffered IO affords
applications to be less precise with their alignment when issuing IO to
the NFS client. But if all applications properly align their IO, LOCALIO
can be configured to use end-to-end O_DIRECT semantics from the NFS
client to the underlying local filesystem, that it is sharing with
the NFS server, by setting the â€˜localio_O_DIRECT_semanticsâ€™ nfs module
parameter to Y, e.g.:”…””}”(hjb  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mhj  h²hubjÜ  )”}”(hŒ?echo Y > /sys/module/nfs/parameters/localio_O_DIRECT_semantics
”h]”hÝ)”}”(hŒ>echo Y > /sys/module/nfs/parameters/localio_O_DIRECT_semantics”h]”hŒ>echo Y > /sys/module/nfs/parameters/localio_O_DIRECT_semantics”…””}”(hjt  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mhjp  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jÛ  h³hÊh´Mhj  h²hubhÝ)”}”(hŒŸOnce enabled, it will cause LOCALIO to use end-to-end O_DIRECT semantics
(but again, this may cause IO to fail if applications do not properly
align their IO).”h]”hŒŸOnce enabled, it will cause LOCALIO to use end-to-end O_DIRECT semantics
(but again, this may cause IO to fail if applications do not properly
align their IO).”…””}”(hjˆ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mhj  h²hubeh}”(h]”Œ&nfs-client-issues-io-instead-of-server”ah ]”h"]”Œ&nfs client issues io instead of server”ah$]”h&]”uh1hµhh·h²hh³hÊh´K÷ubh¶)”}”(hhh]”(h»)”}”(hŒSecurity”h]”hŒSecurity”…””}”(hj¡  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjž  h²hh³hÊh´M$ubhÝ)”}”(hŒ[LOCALIO is only supported when UNIX-style authentication (AUTH_UNIX, aka
AUTH_SYS) is used.”h]”hŒ[LOCALIO is only supported when UNIX-style authentication (AUTH_UNIX, aka
AUTH_SYS) is used.”…””}”(hj¯  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M&hjž  h²hubhÝ)”}”(hX	  Care is taken to ensure the same NFS security mechanisms are used
(authentication, etc) regardless of whether LOCALIO or regular NFS
access is used. The auth_domain established as part of the traditional
NFS client access to the NFS server is also used for LOCALIO.”h]”hX	  Care is taken to ensure the same NFS security mechanisms are used
(authentication, etc) regardless of whether LOCALIO or regular NFS
access is used. The auth_domain established as part of the traditional
NFS client access to the NFS server is also used for LOCALIO.”…””}”(hj½  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M)hjž  h²hubhÝ)”}”(hX¢  Relative to containers, LOCALIO gives the client access to the network
namespace the server has. This is required to allow the client to access
the server's per-namespace nfsd_net struct. With traditional NFS, the
client is afforded this same level of access (albeit in terms of the NFS
protocol via SUNRPC). No other namespaces (user, mount, etc) have been
altered or purposely extended from the server to the client.”h]”hX¤  Relative to containers, LOCALIO gives the client access to the network
namespace the server has. This is required to allow the client to access
the serverâ€™s per-namespace nfsd_net struct. With traditional NFS, the
client is afforded this same level of access (albeit in terms of the NFS
protocol via SUNRPC). No other namespaces (user, mount, etc) have been
altered or purposely extended from the server to the client.”…””}”(hjË  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M.hjž  h²hubeh}”(h]”Œsecurity”ah ]”h"]”Œsecurity”ah$]”h&]”uh1hµhh·h²hh³hÊh´M$ubh¶)”}”(hhh]”(h»)”}”(hŒModule Parameters”h]”hŒModule Parameters”…””}”(hjä  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhjá  h²hh³hÊh´M6ubhÝ)”}”(hŒÀ/sys/module/nfs/parameters/localio_enabled (bool)
controls if LOCALIO is enabled, defaults to Y. If client and server are
local but 'localio_enabled' is set to N then LOCALIO will not be used.”h]”hŒÄ/sys/module/nfs/parameters/localio_enabled (bool)
controls if LOCALIO is enabled, defaults to Y. If client and server are
local but â€˜localio_enabledâ€™ is set to N then LOCALIO will not be used.”…””}”(hjò  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M8hjá  h²hubhÝ)”}”(hŒÛ/sys/module/nfs/parameters/localio_O_DIRECT_semantics (bool)
controls if O_DIRECT extends down to the underlying filesystem, defaults
to N. Application IO must be logical blocksize aligned, otherwise
O_DIRECT will fail.”h]”hŒÛ/sys/module/nfs/parameters/localio_O_DIRECT_semantics (bool)
controls if O_DIRECT extends down to the underlying filesystem, defaults
to N. Application IO must be logical blocksize aligned, otherwise
O_DIRECT will fail.”…””}”(hj   h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M<hjá  h²hubhÝ)”}”(hX7  /sys/module/nfsv3/parameters/nfs3_localio_probe_throttle (uint)
controls if NFSv3 read and write IOs will trigger (re)enabling of
LOCALIO every N (nfs3_localio_probe_throttle) IOs, defaults to 0
(disabled). Must be power-of-2, admin keeps all the pieces if they
misconfigure (too low a value or non-power-of-2).”h]”hX7  /sys/module/nfsv3/parameters/nfs3_localio_probe_throttle (uint)
controls if NFSv3 read and write IOs will trigger (re)enabling of
LOCALIO every N (nfs3_localio_probe_throttle) IOs, defaults to 0
(disabled). Must be power-of-2, admin keeps all the pieces if they
misconfigure (too low a value or non-power-of-2).”…””}”(hj  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MAhjá  h²hubeh}”(h]”Œmodule-parameters”ah ]”h"]”Œmodule parameters”ah$]”h&]”uh1hµhh·h²hh³hÊh´M6ubh¶)”}”(hhh]”(h»)”}”(hŒTesting”h]”hŒTesting”…””}”(hj'  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhj$  h²hh³hÊh´MHubhÝ)”}”(hŒŠThe LOCALIO auxiliary protocol and associated NFS LOCALIO read, write
and commit access have proven stable against various test scenarios:”h]”hŒŠThe LOCALIO auxiliary protocol and associated NFS LOCALIO read, write
and commit access have proven stable against various test scenarios:”…””}”(hj5  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MJhj$  h²hubjS  )”}”(hhh]”(jX  )”}”(hŒ)Client and server both on the same host.
”h]”hÝ)”}”(hŒ(Client and server both on the same host.”h]”hŒ(Client and server both on the same host.”…””}”(hjJ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MMhjF  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hŒfAll permutations of client and server support enablement for both
local and remote client and server.
”h]”hÝ)”}”(hŒeAll permutations of client and server support enablement for both
local and remote client and server.”h]”hŒeAll permutations of client and server support enablement for both
local and remote client and server.”…””}”(hjb  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MOhj^  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hŒaTesting against NFS storage products that don't support the LOCALIO
protocol was also performed.
”h]”hÝ)”}”(hŒ`Testing against NFS storage products that don't support the LOCALIO
protocol was also performed.”h]”hŒbTesting against NFS storage products that donâ€™t support the LOCALIO
protocol was also performed.”…””}”(hjz  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MRhjv  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hŒ»Client on host, server within a container (for both v3 and v4.2).
The container testing was in terms of podman managed containers and
includes successful container stop/restart scenario.
”h]”hÝ)”}”(hŒºClient on host, server within a container (for both v3 and v4.2).
The container testing was in terms of podman managed containers and
includes successful container stop/restart scenario.”h]”hŒºClient on host, server within a container (for both v3 and v4.2).
The container testing was in terms of podman managed containers and
includes successful container stop/restart scenario.”…””}”(hj’  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MUhjŽ  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hX}  Formalizing these test scenarios in terms of existing test
infrastructure is on-going. Initial regular coverage is provided in
terms of ktest running xfstests against a LOCALIO-enabled NFS loopback
mount configuration, and includes lockdep and KASAN coverage, see:
https://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next
https://github.com/koverstreet/ktest
”h]”hÝ)”}”(hX|  Formalizing these test scenarios in terms of existing test
infrastructure is on-going. Initial regular coverage is provided in
terms of ktest running xfstests against a LOCALIO-enabled NFS loopback
mount configuration, and includes lockdep and KASAN coverage, see:
https://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next
https://github.com/koverstreet/ktest”h]”(hX	  Formalizing these test scenarios in terms of existing test
infrastructure is on-going. Initial regular coverage is provided in
terms of ktest running xfstests against a LOCALIO-enabled NFS loopback
mount configuration, and includes lockdep and KASAN coverage, see:
”…””}”(hjª  h²hh³Nh´Nubj)  )”}”(hŒNhttps://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next”h]”hŒNhttps://evilpiepirate.org/~testdashboard/ci?user=snitzer&branch=snitm-nfs-next”…””}”(hj²  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œrefuri”j´  uh1j(  hjª  ubhŒ
”…””}”(hjª  h²hh³Nh´Nubj)  )”}”(hŒ$https://github.com/koverstreet/ktest”h]”hŒ$https://github.com/koverstreet/ktest”…””}”(hjÅ  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œrefuri”jÇ  uh1j(  hjª  ubeh}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´MYhj¦  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hŒ±Various kdevops testing (in terms of "Chuck's BuildBot") has been
performed to regularly verify the LOCALIO changes haven't caused any
regressions to non-LOCALIO NFS use cases.
”h]”hÝ)”}”(hŒ°Various kdevops testing (in terms of "Chuck's BuildBot") has been
performed to regularly verify the LOCALIO changes haven't caused any
regressions to non-LOCALIO NFS use cases.”h]”hŒ¸Various kdevops testing (in terms of â€œChuckâ€™s BuildBotâ€) has been
performed to regularly verify the LOCALIO changes havenâ€™t caused any
regressions to non-LOCALIO NFS use cases.”…””}”(hjä  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´M`hjà  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´NubjX  )”}”(hŒvAll of Hammerspace's various sanity tests pass with LOCALIO enabled
(this includes numerous pNFS and flexfiles tests).”h]”hÝ)”}”(hŒvAll of Hammerspace's various sanity tests pass with LOCALIO enabled
(this includes numerous pNFS and flexfiles tests).”h]”hŒxAll of Hammerspaceâ€™s various sanity tests pass with LOCALIO enabled
(this includes numerous pNFS and flexfiles tests).”…””}”(hjü  h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÜh³hÊh´Mdhjø  ubah}”(h]”h ]”h"]”h$]”h&]”uh1jW  hjC  h²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”j  j  uh1jR  h³hÊh´MMhj$  h²hubeh}”(h]”Œtesting”ah ]”h"]”Œtesting”ah$]”h&]”uh1hµhh·h²hh³hÊh´MHubeh}”(h]”Œnfs-localio”ah ]”h"]”Œnfs localio”ah$]”h&]”uh1hµhhh²hh³hÊh´Kubeh}”(h]”h ]”h"]”h$]”h&]”Œsource”hÊuh1hŒcurrent_source”NŒcurrent_line”NŒsettings”Œdocutils.frontend”ŒValues”“”)”}”(hºNŒ	generator”NŒ	datestamp”NŒsource_link”NŒ
source_url”NŒtoc_backlinks”Œentry”Œfootnote_backlinks”KŒsectnum_xform”KŒstrip_comments”NŒstrip_elements_with_classes”NŒstrip_classes”NŒreport_level”KŒ
halt_level”KŒexit_status_level”KŒdebug”NŒwarning_stream”NŒ	traceback”ˆŒinput_encoding”Œ	utf-8-sig”Œinput_encoding_error_handler”Œstrict”Œoutput_encoding”Œutf-8”Œoutput_encoding_error_handler”jI  Œerror_encoding”Œutf-8”Œerror_encoding_error_handler”Œbackslashreplace”Œlanguage_code”Œen”Œrecord_dependencies”NŒconfig”NŒ	id_prefix”hŒauto_id_prefix”Œid”Œdump_settings”NŒdump_internals”NŒdump_transforms”NŒdump_pseudo_xml”NŒexpose_internals”NŒstrict_visitor”NŒ_disable_config”NŒ_source”hÊŒ_destination”NŒ_config_files”]”Œ7/var/lib/git/docbuild/linux/Documentation/docutils.conf”aŒfile_insertion_enabled”ˆŒraw_enabled”KŒline_length_limit”M'Œpep_references”NŒpep_base_url”Œhttps://peps.python.org/”Œpep_file_url_template”Œpep-%04d”Œrfc_references”NŒrfc_base_url”Œ&https://datatracker.ietf.org/doc/html/”Œ	tab_width”KŒtrim_footnote_reference_space”‰Œsyntax_highlight”Œlong”Œsmart_quotes”ˆŒsmartquotes_locales”]”Œcharacter_level_inline_markup”‰Œdoctitle_xform”‰Œdocinfo_xform”KŒsectsubtitle_xform”‰Œimage_loading”Œlink”Œembed_stylesheet”‰Œcloak_email_addresses”ˆŒsection_self_link”‰Œenv”NubŒreporter”NŒindirect_targets”]”Œsubstitution_defs”}”Œsubstitution_names”}”Œrefnames”}”Œrefids”}”Œnameids”}”(j#  j   j  jÿ  jð  jí  jv  js  j«  j¨  j  j  j›  j˜  jÞ  jÛ  j!  j  j  j  uŒ	nametypes”}”(j#  ‰j  ‰jð  ‰jv  ‰j«  ‰j  ‰j›  ‰jÞ  ‰j!  ‰j  ‰uh}”(j   h·jÿ  hËjí  j  js  jó  j¨  jy  j  j®  j˜  j  jÛ  jž  j  já  j  j$  uŒfootnote_refs”}”Œcitation_refs”}”Œautofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ	footnotes”]”Œ	citations”]”Œautofootnote_start”KŒsymbol_footnote_start”K Œ
id_counter”Œcollections”ŒCounter”“”}”…”R”Œparse_messages”]”Œtransform_messages”]”Œtransformer”NŒinclude_log”]”Œ
decoration”Nh²hub.