msphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget%/translations/zh_CN/networking/devmemmodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/zh_TW/networking/devmemmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/it_IT/networking/devmemmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/ja_JP/networking/devmemmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/ko_KR/networking/devmemmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hPortuguese (Brazilian)}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/pt_BR/networking/devmemmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget%/translations/sp_SP/networking/devmemmodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhh?/var/lib/git/docbuild/linux/Documentation/networking/devmem.rsthKubhsection)}(hhh](htitle)}(hDevice Memory TCPh]hDevice Memory TCP}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hIntroh]hIntro}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhK ubh paragraph)}(hDevice memory TCP (devmem TCP) enables receiving data directly into device memory (dmabuf). The feature is currently implemented for TCP sockets.h]hDevice memory TCP (devmem TCP) enables receiving data directly into device memory (dmabuf). The feature is currently implemented for TCP sockets.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hhh](h)}(h Opportunityh]h Opportunity}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hA large number of data transfers have device memory as the source and/or destination. Accelerators drastically increased the prevalence of such transfers. Some examples include:h]hA large number of data transfers have device memory as the source and/or destination. Accelerators drastically increased the prevalence of such transfers. Some examples include:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh bullet_list)}(hhh](h list_item)}(h]Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data. h]h)}(h\Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data.h]h\Distributed training, where ML accelerators, such as GPUs on different hosts, exchange data.}(hj(hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj$ubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubj#)}(hDistributed raw block storage applications transfer large amounts of data with remote SSDs. Much of this data does not require host processing. h]h)}(hDistributed raw block storage applications transfer large amounts of data with remote SSDs. Much of this data does not require host processing.h]hDistributed raw block storage applications transfer large amounts of data with remote SSDs. Much of this data does not require host processing.}(hj@hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj<ubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubeh}(h]h ]h"]h$]h&]bullet-uh1jhhhKhhhhubh)}(hTypically the Device-to-Device data transfers in the network are implemented as the following low-level operations: Device-to-Host copy, Host-to-Host network transfer, and Host-to-Device copy.h]hTypically the Device-to-Device data transfers in the network are implemented as the following low-level operations: Device-to-Host copy, Host-to-Host network transfer, and Host-to-Device copy.}(hj\hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hThe flow involving host copies is suboptimal, especially for bulk data transfers, and can put significant strains on system resources such as host memory bandwidth and PCIe bandwidth.h]hThe flow involving host copies is suboptimal, especially for bulk data transfers, and can put significant strains on system resources such as host memory bandwidth and PCIe bandwidth.}(hjjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hDevmem TCP optimizes this use case by implementing socket APIs that enable the user to receive incoming network packets directly into device memory.h]hDevmem TCP optimizes this use case by implementing socket APIs that enable the user to receive incoming network packets directly into device memory.}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK$hhhhubh)}(h:Packet payloads go directly from the NIC to device memory.h]h:Packet payloads go directly from the NIC to device memory.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK'hhhhubh)}(hPacket headers go to host memory and are processed by the TCP/IP stack normally. The NIC must support header split to achieve this.h]hPacket headers go to host memory and are processed by the TCP/IP stack normally. The NIC must support header split to achieve this.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK)hhhhubh)}(h Advantages:h]h Advantages:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK,hhhhubj)}(hhh](j#)}(hiAlleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics. h]h)}(hhAlleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics.h]hhAlleviate host memory bandwidth pressure, compared to existing network-transfer + device-copy semantics.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK.hjubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubj#)}(hAlleviate PCIe bandwidth pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to the traditional path which sends data through the root complex. h]h)}(hAlleviate PCIe bandwidth pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to the traditional path which sends data through the root complex.h]hAlleviate PCIe bandwidth pressure, by limiting data transfer to the lowest level of the PCIe tree, compared to the traditional path which sends data through the root complex.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK1hjubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubeh}(h]h ]h"]h$]h&]jZj[uh1jhhhK.hhhhubeh}(h] opportunityah ]h"] opportunityah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(h More Infoh]h More Info}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhK7ubh block_quote)}(hslides, video https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html patchset [PATCH net-next v24 00/13] Device Memory TCP https://lore.kernel.org/netdev/20240831004313.3713467-1-almasrymina@google.com/ h]hdefinition_list)}(hhh](hdefinition_list_item)}(hOslides, video https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html h](hterm)}(h slides, videoh]h slides, video}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK:hjubh definition)}(hhh]h)}(h@https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.htmlh]h reference)}(hj*h]h@https://netdevconf.org/0x17/sessions/talk/device-memory-tcp.html}(hj.hhhNhNubah}(h]h ]h"]h$]h&]refurij*uh1j,hj(ubah}(h]h ]h"]h$]h&]uh1hhhhK:hj%ubah}(h]h ]h"]h$]h&]uh1j#hjubeh}(h]h ]h"]h$]h&]uh1j hhhK:hj ubj)}(hpatchset [PATCH net-next v24 00/13] Device Memory TCP https://lore.kernel.org/netdev/20240831004313.3713467-1-almasrymina@google.com/ h](j)}(hpatchseth]hpatchset}(hjRhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK?hjNubj$)}(hhh]h)}(h|[PATCH net-next v24 00/13] Device Memory TCP https://lore.kernel.org/netdev/20240831004313.3713467-1-almasrymina@google.com/h](h-[PATCH net-next v24 00/13] Device Memory TCP }(hjchhhNhNubj-)}(hOhttps://lore.kernel.org/netdev/20240831004313.3713467-1-almasrymina@google.com/h]hOhttps://lore.kernel.org/netdev/20240831004313.3713467-1-almasrymina@google.com/}(hjkhhhNhNubah}(h]h ]h"]h$]h&]refurijmuh1j,hjcubeh}(h]h ]h"]h$]h&]uh1hhhhK=hj`ubah}(h]h ]h"]h$]h&]uh1j#hjNubeh}(h]h ]h"]h$]h&]uh1j hhhK?hj ubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhhhK9hjhhubeh}(h] more-infoah ]h"] more infoah$]h&]uh1hhhhhhhhK7ubeh}(h]introah ]h"]introah$]h&]uh1hhhhhhhhK ubh)}(hhh](h)}(h RX Interfaceh]h RX Interface}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKBubh)}(hhh](h)}(hExampleh]hExample}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKFubh)}(hs./tools/testing/selftests/drivers/net/hw/ncdevmem:do_server shows an example of setting up the RX path of this API.h]hs./tools/testing/selftests/drivers/net/hw/ncdevmem:do_server shows an example of setting up the RX path of this API.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKHhjhhubeh}(h]exampleah ]h"]h$]exampleah&]uh1hhjhhhhhKF referencedKubh)}(hhh](h)}(h NIC Setuph]h NIC Setup}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKMubh)}(hHHeader split, flow steering, & RSS are required features for devmem TCP.h]hHHeader split, flow steering, & RSS are required features for devmem TCP.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKOhjhhubh)}(hzHeader split is used to split incoming packets into a header buffer in host memory, and a payload buffer in device memory.h]hzHeader split is used to split incoming packets into a header buffer in host memory, and a payload buffer in device memory.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKQhjhhubh)}(hlFlow steering & RSS are used to ensure that only flows targeting devmem land on an RX queue bound to devmem.h]hlFlow steering & RSS are used to ensure that only flows targeting devmem land on an RX queue bound to devmem.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKThjhhubh)}(h%Enable header split & flow steering::h]h$Enable header split & flow steering:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKWhjhhubh literal_block)}(hj# enable header split ethtool -G eth1 tcp-data-split on # enable flow steering ethtool -K eth1 ntuple onh]hj# enable header split ethtool -G eth1 tcp-data-split on # enable flow steering ethtool -K eth1 ntuple on}hj,sbah}(h]h ]h"]h$]h&]hhuh1j*hhhKYhjhhubh)}(h]Configure RSS to steer all traffic away from the target RX queue (queue 15 in this example)::h]h\Configure RSS to steer all traffic away from the target RX queue (queue 15 in this example):}(hj:hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK`hjhhubj+)}(hðtool --set-rxfh-indir eth1 equal 15h]hðtool --set-rxfh-indir eth1 equal 15}hjHsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKchjhhubh)}(h]The user must bind a dmabuf to any number of RX queues on a given NIC using the netlink API::h]h\The user must bind a dmabuf to any number of RX queues on a given NIC using the netlink API:}(hjVhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKfhjhhubj+)}(hX/* Bind dmabuf to NIC RX queue 15 */ struct netdev_queue_id *queues; queues = netdev_queue_id_alloc(1); netdev_queue_id_set_type(&queues[0], NETDEV_QUEUE_TYPE_RX); netdev_queue_id_set_id(&queues[0], 15); *ys = ynl_sock_create(&ynl_netdev_family, &yerr); req = netdev_bind_rx_req_alloc(); netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */); netdev_bind_rx_req_set_fd(req, dmabuf_fd); __netdev_bind_rx_req_set_queues(req, queues, 1); rsp = netdev_bind_rx(*ys, req); dmabuf_id = rsp->id;h]hX/* Bind dmabuf to NIC RX queue 15 */ struct netdev_queue_id *queues; queues = netdev_queue_id_alloc(1); netdev_queue_id_set_type(&queues[0], NETDEV_QUEUE_TYPE_RX); netdev_queue_id_set_id(&queues[0], 15); *ys = ynl_sock_create(&ynl_netdev_family, &yerr); req = netdev_bind_rx_req_alloc(); netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */); netdev_bind_rx_req_set_fd(req, dmabuf_fd); __netdev_bind_rx_req_set_queues(req, queues, 1); rsp = netdev_bind_rx(*ys, req); dmabuf_id = rsp->id;}hjdsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKihjhhubh)}(h`The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf that has been bound.h]h`The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf that has been bound.}(hjrhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK|hjhhubh)}(hThe user can unbind the dmabuf from the netdevice by closing the netlink socket that established the binding. We do this so that the binding is automatically unbound even if the userspace process crashes.h]hThe user can unbind the dmabuf from the netdevice by closing the netlink socket that established the binding. We do this so that the binding is automatically unbound even if the userspace process crashes.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hNote that any reasonably well-behaved dmabuf from any exporter should work with devmem TCP, even if the dmabuf is not actually backed by devmem. An example of this is udmabuf, which wraps user memory (non-devmem) in a dmabuf.h]hNote that any reasonably well-behaved dmabuf from any exporter should work with devmem TCP, even if the dmabuf is not actually backed by devmem. An example of this is udmabuf, which wraps user memory (non-devmem) in a dmabuf.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubeh}(h] nic-setupah ]h"]h$] nic setupah&]uh1hhjhhhhhKMjKubh)}(hhh](h)}(h Socket Setuph]h Socket Setup}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(h>The socket must be flow steered to the dmabuf bound RX queue::h]h=The socket must be flow steered to the dmabuf bound RX queue:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj+)}(h+ethtool -N eth1 flow-type tcp4 ... queue 15h]h+ethtool -N eth1 flow-type tcp4 ... queue 15}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKhjhhubeh}(h] socket-setupah ]h"]h$] socket setupah&]uh1hhjhhhhhKjKubh)}(hhh](h)}(hReceiving datah]hReceiving data}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hThe user application must signal to the kernel that it is capable of receiving devmem data by passing the MSG_SOCK_DEVMEM flag to recvmsg::h]hThe user application must signal to the kernel that it is capable of receiving devmem data by passing the MSG_SOCK_DEVMEM flag to recvmsg:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj+)}(h)ret = recvmsg(fd, &msg, MSG_SOCK_DEVMEM);h]h)ret = recvmsg(fd, &msg, MSG_SOCK_DEVMEM);}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKhjhhubh)}(h`Applications that do not specify the MSG_SOCK_DEVMEM flag will receive an EFAULT on devmem data.h]h`Applications that do not specify the MSG_SOCK_DEVMEM flag will receive an EFAULT on devmem data.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hDevmem data is received directly into the dmabuf bound to the NIC in 'NIC Setup', and the kernel signals such to the user via the SCM_DEVMEM_* cmsgs::h]hDevmem data is received directly into the dmabuf bound to the NIC in ‘NIC Setup’, and the kernel signals such to the user via the SCM_DEVMEM_* cmsgs:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj+)}(hX<for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) { if (cm->cmsg_level != SOL_SOCKET || (cm->cmsg_type != SCM_DEVMEM_DMABUF && cm->cmsg_type != SCM_DEVMEM_LINEAR)) continue; dmabuf_cmsg = (struct dmabuf_cmsg *)CMSG_DATA(cm); if (cm->cmsg_type == SCM_DEVMEM_DMABUF) { /* Frag landed in dmabuf. * * dmabuf_cmsg->dmabuf_id is the dmabuf the * frag landed on. * * dmabuf_cmsg->frag_offset is the offset into * the dmabuf where the frag starts. * * dmabuf_cmsg->frag_size is the size of the * frag. * * dmabuf_cmsg->frag_token is a token used to * refer to this frag for later freeing. */ struct dmabuf_token token; token.token_start = dmabuf_cmsg->frag_token; token.token_count = 1; continue; } if (cm->cmsg_type == SCM_DEVMEM_LINEAR) /* Frag landed in linear buffer. * * dmabuf_cmsg->frag_size is the size of the * frag. */ continue; }h]hX<for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) { if (cm->cmsg_level != SOL_SOCKET || (cm->cmsg_type != SCM_DEVMEM_DMABUF && cm->cmsg_type != SCM_DEVMEM_LINEAR)) continue; dmabuf_cmsg = (struct dmabuf_cmsg *)CMSG_DATA(cm); if (cm->cmsg_type == SCM_DEVMEM_DMABUF) { /* Frag landed in dmabuf. * * dmabuf_cmsg->dmabuf_id is the dmabuf the * frag landed on. * * dmabuf_cmsg->frag_offset is the offset into * the dmabuf where the frag starts. * * dmabuf_cmsg->frag_size is the size of the * frag. * * dmabuf_cmsg->frag_token is a token used to * refer to this frag for later freeing. */ struct dmabuf_token token; token.token_start = dmabuf_cmsg->frag_token; token.token_count = 1; continue; } if (cm->cmsg_type == SCM_DEVMEM_LINEAR) /* Frag landed in linear buffer. * * dmabuf_cmsg->frag_size is the size of the * frag. */ continue; }}hj"sbah}(h]h ]h"]h$]h&]hhuh1j*hhhKhjhhubh)}(h!Applications may receive 2 cmsgs:h]h!Applications may receive 2 cmsgs:}(hj0hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj)}(hhh](j#)}(h\SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf indicated by dmabuf_id. h]h)}(h[SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf indicated by dmabuf_id.h]h[SCM_DEVMEM_DMABUF: this indicates the fragment landed in the dmabuf indicated by dmabuf_id.}(hjEhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjAubah}(h]h ]h"]h$]h&]uh1j"hj>hhhhhNubj#)}(hSCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer. This typically happens when the NIC is unable to split the packet at the header boundary, such that part (or all) of the payload landed in host memory. h]h)}(hSCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer. This typically happens when the NIC is unable to split the packet at the header boundary, such that part (or all) of the payload landed in host memory.h]hSCM_DEVMEM_LINEAR: this indicates the fragment landed in the linear buffer. This typically happens when the NIC is unable to split the packet at the header boundary, such that part (or all) of the payload landed in host memory.}(hj]hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjYubah}(h]h ]h"]h$]h&]uh1j"hj>hhhhhNubeh}(h]h ]h"]h$]h&]jZj[uh1jhhhKhjhhubh)}(hApplications may receive no SO_DEVMEM_* cmsgs. That indicates non-devmem, regular TCP data that landed on an RX queue not bound to a dmabuf.h]hApplications may receive no SO_DEVMEM_* cmsgs. That indicates non-devmem, regular TCP data that landed on an RX queue not bound to a dmabuf.}(hjwhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubeh}(h]receiving-dataah ]h"]receiving dataah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(h Freeing fragsh]h Freeing frags}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hFrags received via SCM_DEVMEM_DMABUF are pinned by the kernel while the user processes the frag. The user must return the frag to the kernel via SO_DEVMEM_DONTNEED::h]hFrags received via SCM_DEVMEM_DMABUF are pinned by the kernel while the user processes the frag. The user must return the frag to the kernel via SO_DEVMEM_DONTNEED:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj+)}(hdret = setsockopt(client_fd, SOL_SOCKET, SO_DEVMEM_DONTNEED, &token, sizeof(token));h]hdret = setsockopt(client_fd, SOL_SOCKET, SO_DEVMEM_DONTNEED, &token, sizeof(token));}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKhjhhubh)}(hThe user must ensure the tokens are returned to the kernel in a timely manner. Failure to do so will exhaust the limited dmabuf that is bound to the RX queue and will lead to packet drops.h]hThe user must ensure the tokens are returned to the kernel in a timely manner. Failure to do so will exhaust the limited dmabuf that is bound to the RX queue and will lead to packet drops.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hThe user must pass no more than 128 tokens, with no more than 1024 total frags among the token->token_count across all the tokens. If the user provides more than 1024 frags, the kernel will free up to 1024 frags and return early.h]hThe user must pass no more than 128 tokens, with no more than 1024 total frags among the token->token_count across all the tokens. If the user provides more than 1024 frags, the kernel will free up to 1024 frags and return early.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hThe kernel returns the number of actual frags freed. The number of frags freed can be less than the tokens provided by the user in case of:h]hThe kernel returns the number of actual frags freed. The number of frags freed can be less than the tokens provided by the user in case of:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubhenumerated_list)}(hhh](j#)}(han internal kernel leak bug.h]h)}(hjh]han internal kernel leak bug.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubj#)}(h&the user passed more than 1024 frags. h]h)}(h%the user passed more than 1024 frags.h]h%the user passed more than 1024 frags.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j"hjhhhhhNubeh}(h]h ]h"]h$]h&]enumtype loweralphaprefix(suffix)uh1jhjhhhhhKubeh}(h] freeing-fragsah ]h"] freeing fragsah$]h&]uh1hhjhhhhhKubeh}(h] rx-interfaceah ]h"] rx interfaceah$]h&]uh1hhhhhhhhKBubh)}(hhh](h)}(h TX Interfaceh]h TX Interface}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj4hhhhhKubh)}(hhh](h)}(hExampleh]hExample}(hjHhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjEhhhhhKubh)}(hs./tools/testing/selftests/drivers/net/hw/ncdevmem:do_client shows an example of setting up the TX path of this API.h]hs./tools/testing/selftests/drivers/net/hw/ncdevmem:do_client shows an example of setting up the TX path of this API.}(hjVhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjEhhubeh}(h]id1ah ]h"]h$]jah&]uh1hhj4hhhhhKjKubh)}(hhh](h)}(h NIC Setuph]h NIC Setup}(hjnhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjkhhhhhKubh)}(hEThe user must bind a TX dmabuf to a given NIC using the netlink API::h]hDThe user must bind a TX dmabuf to a given NIC using the netlink API:}(hj|hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjkhhubj+)}(hXMstruct netdev_bind_tx_req *req = NULL; struct netdev_bind_tx_rsp *rsp = NULL; struct ynl_error yerr; *ys = ynl_sock_create(&ynl_netdev_family, &yerr); req = netdev_bind_tx_req_alloc(); netdev_bind_tx_req_set_ifindex(req, ifindex); netdev_bind_tx_req_set_fd(req, dmabuf_fd); rsp = netdev_bind_tx(*ys, req); tx_dmabuf_id = rsp->id;h]hXMstruct netdev_bind_tx_req *req = NULL; struct netdev_bind_tx_rsp *rsp = NULL; struct ynl_error yerr; *ys = ynl_sock_create(&ynl_netdev_family, &yerr); req = netdev_bind_tx_req_alloc(); netdev_bind_tx_req_set_ifindex(req, ifindex); netdev_bind_tx_req_set_fd(req, dmabuf_fd); rsp = netdev_bind_tx(*ys, req); tx_dmabuf_id = rsp->id;}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhKhjkhhubh)}(h`The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf that has been bound.h]h`The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf that has been bound.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjkhhubh)}(hThe user can unbind the dmabuf from the netdevice by closing the netlink socket that established the binding. We do this so that the binding is automatically unbound even if the userspace process crashes.h]hThe user can unbind the dmabuf from the netdevice by closing the netlink socket that established the binding. We do this so that the binding is automatically unbound even if the userspace process crashes.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjkhhubh)}(hNote that any reasonably well-behaved dmabuf from any exporter should work with devmem TCP, even if the dmabuf is not actually backed by devmem. An example of this is udmabuf, which wraps user memory (non-devmem) in a dmabuf.h]hNote that any reasonably well-behaved dmabuf from any exporter should work with devmem TCP, even if the dmabuf is not actually backed by devmem. An example of this is udmabuf, which wraps user memory (non-devmem) in a dmabuf.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjkhhubeh}(h]id2ah ]h"]h$]jah&]uh1hhj4hhhhhKjKubh)}(hhh](h)}(h Socket Setuph]h Socket Setup}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hThe user application must use MSG_ZEROCOPY flag when sending devmem TCP. Devmem cannot be copied by the kernel, so the semantics of the devmem TX are similar to the semantics of MSG_ZEROCOPY::h]hThe user application must use MSG_ZEROCOPY flag when sending devmem TCP. Devmem cannot be copied by the kernel, so the semantics of the devmem TX are similar to the semantics of MSG_ZEROCOPY:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj+)}(hBsetsockopt(socket_fd, SOL_SOCKET, SO_ZEROCOPY, &opt, sizeof(opt));h]hBsetsockopt(socket_fd, SOL_SOCKET, SO_ZEROCOPY, &opt, sizeof(opt));}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhMhjhhubh)}(hIt is also recommended that the user binds the TX socket to the same interface the dma-buf has been bound to via SO_BINDTODEVICE::h]hIt is also recommended that the user binds the TX socket to the same interface the dma-buf has been bound to via SO_BINDTODEVICE:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj+)}(hOsetsockopt(socket_fd, SOL_SOCKET, SO_BINDTODEVICE, ifname, strlen(ifname) + 1);h]hOsetsockopt(socket_fd, SOL_SOCKET, SO_BINDTODEVICE, ifname, strlen(ifname) + 1);}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhM"hjhhubeh}(h]id3ah ]h"]h$]jah&]uh1hhj4hhhhhMjKubh)}(hhh](h)}(h Sending datah]h Sending data}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM&ubh)}(h5Devmem data is sent using the SCM_DEVMEM_DMABUF cmsg.h]h5Devmem data is sent using the SCM_DEVMEM_DMABUF cmsg.}(hj*hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM(hjhhubh)}(h&The user should create a msghdr where,h]h&The user should create a msghdr where,}(hj8hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM*hjhhubj)}(hhh](j#)}(hCiov_base is set to the offset into the dmabuf to start sending fromh]h)}(hjKh]hCiov_base is set to the offset into the dmabuf to start sending from}(hjMhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM,hjIubah}(h]h ]h"]h$]h&]uh1j"hjFhhhhhNubj#)}(hAiov_len is set to the number of bytes to be sent from the dmabuf h]h)}(h@iov_len is set to the number of bytes to be sent from the dmabufh]h@iov_len is set to the number of bytes to be sent from the dmabuf}(hjdhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM-hj`ubah}(h]h ]h"]h$]h&]uh1j"hjFhhhhhNubeh}(h]h ]h"]h$]h&]jZ*uh1jhhhM,hjhhubh)}(hBThe user passes the dma-buf id to send from as a u32 cmsg payload.h]hBThe user passes the dma-buf id to send from as a u32 cmsg payload.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM/hjhhubh)}(hThe example below sends 1024 bytes from offset 100 into the dmabuf, and 2048 from offset 2000 into the dmabuf. The dmabuf to send from is tx_dmabuf_id::h]hThe example below sends 1024 bytes from offset 100 into the dmabuf, and 2048 from offset 2000 into the dmabuf. The dmabuf to send from is tx_dmabuf_id:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM1hjhhubj+)}(hX(char ctrl_data[CMSG_SPACE(sizeof(__u32))]; struct msghdr msg = {}; struct cmsghdr *cmsg; struct iovec iov[2]; iov[0].iov_base = (void*)100; iov[0].iov_len = 1024; iov[1].iov_base = (void*)2000; iov[1].iov_len = 2048; msg.msg_iov = iov; msg.msg_iovlen = 2; msg.msg_control = ctrl_data; msg.msg_controllen = sizeof(ctrl_data); cmsg = CMSG_FIRSTHDR(&msg); cmsg->cmsg_level = SOL_SOCKET; cmsg->cmsg_type = SCM_DEVMEM_DMABUF; cmsg->cmsg_len = CMSG_LEN(sizeof(__u32)); *((__u32 *)CMSG_DATA(cmsg)) = tx_dmabuf_id; sendmsg(socket_fd, &msg, MSG_ZEROCOPY);h]hX(char ctrl_data[CMSG_SPACE(sizeof(__u32))]; struct msghdr msg = {}; struct cmsghdr *cmsg; struct iovec iov[2]; iov[0].iov_base = (void*)100; iov[0].iov_len = 1024; iov[1].iov_base = (void*)2000; iov[1].iov_len = 2048; msg.msg_iov = iov; msg.msg_iovlen = 2; msg.msg_control = ctrl_data; msg.msg_controllen = sizeof(ctrl_data); cmsg = CMSG_FIRSTHDR(&msg); cmsg->cmsg_level = SOL_SOCKET; cmsg->cmsg_type = SCM_DEVMEM_DMABUF; cmsg->cmsg_len = CMSG_LEN(sizeof(__u32)); *((__u32 *)CMSG_DATA(cmsg)) = tx_dmabuf_id; sendmsg(socket_fd, &msg, MSG_ZEROCOPY);}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhM4hjhhubeh}(h] sending-dataah ]h"] sending dataah$]h&]uh1hhj4hhhhhM&ubh)}(hhh](h)}(hReusing TX dmabufsh]hReusing TX dmabufs}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMOubh)}(hX)Similar to MSG_ZEROCOPY with regular memory, the user should not modify the contents of the dma-buf while a send operation is in progress. This is because the kernel does not keep a copy of the dmabuf contents. Instead, the kernel will pin and send data from the buffer available to the userspace.h]hX)Similar to MSG_ZEROCOPY with regular memory, the user should not modify the contents of the dma-buf while a send operation is in progress. This is because the kernel does not keep a copy of the dmabuf contents. Instead, the kernel will pin and send data from the buffer available to the userspace.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMQhjhhubh)}(hcJust as in MSG_ZEROCOPY, the kernel notifies the userspace of send completions using MSG_ERRQUEUE::h]hbJust as in MSG_ZEROCOPY, the kernel notifies the userspace of send completions using MSG_ERRQUEUE:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMVhjhhubj+)}(hXint64_t tstop = gettimeofday_ms() + waittime_ms; char control[CMSG_SPACE(100)] = {}; struct sock_extended_err *serr; struct msghdr msg = {}; struct cmsghdr *cm; int retries = 10; __u32 hi, lo; msg.msg_control = control; msg.msg_controllen = sizeof(control); while (gettimeofday_ms() < tstop) { if (!do_poll(fd)) continue; ret = recvmsg(fd, &msg, MSG_ERRQUEUE); for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) { serr = (void *)CMSG_DATA(cm); hi = serr->ee_data; lo = serr->ee_info; fprintf(stdout, "tx complete [%d,%d]\n", lo, hi); } }h]hXint64_t tstop = gettimeofday_ms() + waittime_ms; char control[CMSG_SPACE(100)] = {}; struct sock_extended_err *serr; struct msghdr msg = {}; struct cmsghdr *cm; int retries = 10; __u32 hi, lo; msg.msg_control = control; msg.msg_controllen = sizeof(control); while (gettimeofday_ms() < tstop) { if (!do_poll(fd)) continue; ret = recvmsg(fd, &msg, MSG_ERRQUEUE); for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) { serr = (void *)CMSG_DATA(cm); hi = serr->ee_data; lo = serr->ee_info; fprintf(stdout, "tx complete [%d,%d]\n", lo, hi); } }}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhMYhjhhubh)}(h[After the associated sendmsg has been completed, the dmabuf can be reused by the userspace.h]h[After the associated sendmsg has been completed, the dmabuf can be reused by the userspace.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMshjhhubeh}(h]reusing-tx-dmabufsah ]h"]reusing tx dmabufsah$]h&]uh1hhj4hhhhhMOubeh}(h] tx-interfaceah ]h"] tx interfaceah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hImplementation & Caveatsh]hImplementation & Caveats}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMxubh)}(hhh](h)}(hUnreadable skbsh]hUnreadable skbs}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM{ubh)}(hDevmem payloads are inaccessible to the kernel processing the packets. This results in a few quirks for payloads of devmem skbs:h]hDevmem payloads are inaccessible to the kernel processing the packets. This results in a few quirks for payloads of devmem skbs:}(hj,hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM}hjhhubj)}(hhh](j#)}(hlLoopback is not functional. Loopback relies on copying the payload, which is not possible with devmem skbs. h]h)}(hkLoopback is not functional. Loopback relies on copying the payload, which is not possible with devmem skbs.h]hkLoopback is not functional. Loopback relies on copying the payload, which is not possible with devmem skbs.}(hjAhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj=ubah}(h]h ]h"]h$]h&]uh1j"hj:hhhhhNubj#)}(h%Software checksum calculation fails. h]h)}(h$Software checksum calculation fails.h]h$Software checksum calculation fails.}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjUubah}(h]h ]h"]h$]h&]uh1j"hj:hhhhhNubj#)}(h7TCP Dump and bpf can't access devmem packet payloads. h]h)}(h5TCP Dump and bpf can't access devmem packet payloads.h]h7TCP Dump and bpf can’t access devmem packet payloads.}(hjqhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjmubah}(h]h ]h"]h$]h&]uh1j"hj:hhhhhNubeh}(h]h ]h"]h$]h&]jZj[uh1jhhhMhjhhubeh}(h]unreadable-skbsah ]h"]unreadable skbsah$]h&]uh1hhj hhhhhM{ubeh}(h]implementation-caveatsah ]h"]implementation & caveatsah$]h&]uh1hhhhhhhhMxubh)}(hhh](h)}(hTestingh]hTesting}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hyMore realistic example code can be found in the kernel source under ``tools/testing/selftests/drivers/net/hw/ncdevmem.c``h](hDMore realistic example code can be found in the kernel source under }(hjhhhNhNubhliteral)}(h5``tools/testing/selftests/drivers/net/hw/ncdevmem.c``h]h1tools/testing/selftests/drivers/net/hw/ncdevmem.c}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hnncdevmem is a devmem TCP netcat. It works very similarly to netcat, but receives data directly into a udmabuf.h]hnncdevmem is a devmem TCP netcat. It works very similarly to netcat, but receives data directly into a udmabuf.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hTo run ncdevmem, you need to run it on a server on the machine under test, and you need to run netcat on a peer to provide the TX data.h]hTo run ncdevmem, you need to run it on a server on the machine under test, and you need to run netcat on a peer to provide the TX data.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hncdevmem has a validation mode as well that expects a repeating pattern of incoming data and validates it as such. For example, you can launch ncdevmem on the server by::h]hncdevmem has a validation mode as well that expects a repeating pattern of incoming data and validates it as such. For example, you can launch ncdevmem on the server by:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj+)}(hBncdevmem -s -c -f -l -p 5201 -v 7h]hBncdevmem -s -c -f -l -p 5201 -v 7}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhMhjhhubh)}(hVOn client side, use regular netcat to send TX data to ncdevmem process on the server::h]hUOn client side, use regular netcat to send TX data to ncdevmem process on the server:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj+)}(hoyes $(echo -e \\x01\\x02\\x03\\x04\\x05\\x06) | \ tr \\n \\0 | head -c 5G | nc 5201 -p 5201h]hoyes $(echo -e \\x01\\x02\\x03\\x04\\x05\\x06) | \ tr \\n \\0 | head -c 5G | nc 5201 -p 5201}hjsbah}(h]h ]h"]h$]h&]hhuh1j*hhhMhjhhubeh}(h]testingah ]h"]testingah$]h&]uh1hhhhhhhhMubeh}(h]device-memory-tcpah ]h"]device memory tcpah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksentryfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerjQerror_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourcehnj _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}nameids}(j+j(jjjjjjj1j.exampleN nic setupN socket setupNjjj)j&jjjjjjjjjjj#j u nametypes}(j+jjjj1jjjjj)jjjjjj#uh}(j(hjhjhjjj.jjjjjjjjjj&jjj4jfjEjjkjjjjjjjj jjj ju footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}j_KsRparse_messages](hsystem_message)}(hhh]h)}(h*Duplicate implicit target name: "example".h]h.Duplicate implicit target name: “example”.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]jfalevelKtypeINFOsourcehnjlineKuh1jhjEhhhhhKubj)}(hhh]h)}(h,Duplicate implicit target name: "nic setup".h]h0Duplicate implicit target name: “nic setup”.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]jalevelKtypejsourcehnjlineKuh1jhjkhhhhhKubj)}(hhh]h)}(h/Duplicate implicit target name: "socket setup".h]h3Duplicate implicit target name: “socket setup”.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]jalevelKtypejsourcehnjlineMuh1jhjhhhhhMubetransform_messages] transformerN include_log] decorationNhhub.