€•_TŒsphinx.addnodes”Œdocument”“”)”}”(Œ rawsource”Œ”Œchildren”]”(Œ translations”Œ LanguagesNode”“”)”}”(hhh]”(hŒ pending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ refdomain”Œstd”Œreftype”Œdoc”Œ reftarget”Œ'/translations/zh_CN/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuŒtagname”hhh ubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ'/translations/zh_TW/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ'/translations/it_IT/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ'/translations/ja_JP/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ'/translations/ko_KR/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒSpanish”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ'/translations/sp_SP/networking/iou-zcrx”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h hhŒ _document”hŒsource”NŒline”NubhŒcomment”“”)”}”(hŒ SPDX-License-Identifier: GPL-2.0”h]”hŒ SPDX-License-Identifier: GPL-2.0”…””}”hh£sbah}”(h]”h ]”h"]”h$]”h&]”Œ xml:space”Œpreserve”uh1h¡hhhžhhŸŒA/var/lib/git/docbuild/linux/Documentation/networking/iou-zcrx.rst”h KubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒio_uring zero copy Rx”h]”hŒio_uring zero copy Rx”…””}”(hh»hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hh¶hžhhŸh³h Kubhµ)”}”(hhh]”(hº)”}”(hŒ Introduction”h]”hŒ Introduction”…””}”(hhÌhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hhÉhžhhŸh³h KubhŒ paragraph”“”)”}”(hX¬io_uring zero copy Rx (ZC Rx) is a feature that removes kernel-to-user copy on the network receive path, allowing packet data to be received directly into userspace memory. This feature is different to TCP_ZEROCOPY_RECEIVE in that there are no strict alignment requirements and no need to mmap()/munmap(). Compared to kernel bypass solutions such as e.g. DPDK, the packet headers are processed by the kernel TCP stack as normal.”h]”hX¬io_uring zero copy Rx (ZC Rx) is a feature that removes kernel-to-user copy on the network receive path, allowing packet data to be received directly into userspace memory. This feature is different to TCP_ZEROCOPY_RECEIVE in that there are no strict alignment requirements and no need to mmap()/munmap(). Compared to kernel bypass solutions such as e.g. DPDK, the packet headers are processed by the kernel TCP stack as normal.”…””}”(hhÜhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K hhÉhžhubeh}”(h]”Œ introduction”ah ]”h"]”Œ introduction”ah$]”h&]”uh1h´hh¶hžhhŸh³h Kubhµ)”}”(hhh]”(hº)”}”(hŒNIC HW Requirements”h]”hŒNIC HW Requirements”…””}”(hhõhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hhòhžhhŸh³h KubhÛ)”}”(hŒ“Several NIC HW features are required for io_uring ZC Rx to work. For now the kernel API does not configure the NIC and it must be done by the user.”h]”hŒ“Several NIC HW features are required for io_uring ZC Rx to work. For now the kernel API does not configure the NIC and it must be done by the user.”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h Khhòhžhubhµ)”}”(hhh]”(hº)”}”(hŒHeader/data split”h]”hŒHeader/data split”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hjhžhhŸh³h KubhÛ)”}”(hŒÛRequired to split packets at the L4 boundary into a header and a payload. Headers are received into kernel memory as normal and processed by the TCP stack as normal. Payloads are received into userspace memory directly.”h]”hŒÛRequired to split packets at the L4 boundary into a header and a payload. Headers are received into kernel memory as normal and processed by the TCP stack as normal. Payloads are received into userspace memory directly.”…””}”(hj"hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h Khjhžhubeh}”(h]”Œheader-data-split”ah ]”h"]”Œheader/data split”ah$]”h&]”uh1h´hhòhžhhŸh³h Kubhµ)”}”(hhh]”(hº)”}”(hŒ Flow steering”h]”hŒ Flow steering”…””}”(hj;hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hj8hžhhŸh³h KubhÛ)”}”(hŒýSpecific HW Rx queues are configured for this feature, but modern NICs typically distribute flows across all HW Rx queues. Flow steering is required to ensure that only desired flows are directed towards HW queues that are configured for io_uring ZC Rx.”h]”hŒýSpecific HW Rx queues are configured for this feature, but modern NICs typically distribute flows across all HW Rx queues. Flow steering is required to ensure that only desired flows are directed towards HW queues that are configured for io_uring ZC Rx.”…””}”(hjIhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K!hj8hžhubeh}”(h]”Œ flow-steering”ah ]”h"]”Œ flow steering”ah$]”h&]”uh1h´hhòhžhhŸh³h Kubhµ)”}”(hhh]”(hº)”}”(hŒRSS”h]”hŒRSS”…””}”(hjbhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hj_hžhhŸh³h K'ubhÛ)”}”(hŒ“In addition to flow steering above, RSS is required to steer all other non-zero copy flows away from queues that are configured for io_uring ZC Rx.”h]”hŒ“In addition to flow steering above, RSS is required to steer all other non-zero copy flows away from queues that are configured for io_uring ZC Rx.”…””}”(hjphžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K)hj_hžhubeh}”(h]”Œrss”ah ]”h"]”Œrss”ah$]”h&]”uh1h´hhòhžhhŸh³h K'ubeh}”(h]”Œnic-hw-requirements”ah ]”h"]”Œnic hw requirements”ah$]”h&]”uh1h´hh¶hžhhŸh³h Kubhµ)”}”(hhh]”(hº)”}”(hŒUsage”h]”hŒUsage”…””}”(hj‘hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hjŽhžhhŸh³h K-ubhµ)”}”(hhh]”(hº)”}”(hŒ Setup NIC”h]”hŒ Setup NIC”…””}”(hj¢hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hjŸhžhhŸh³h K0ubhÛ)”}”(hŒ!Must be done out of band for now.”h]”hŒ!Must be done out of band for now.”…””}”(hj°hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K2hjŸhžhubhÛ)”}”(hŒ&Ensure there are at least two queues::”h]”hŒ%Ensure there are at least two queues:”…””}”(hj¾hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K4hjŸhžhubhŒ literal_block”“”)”}”(hŒethtool -L eth0 combined 2”h]”hŒethtool -L eth0 combined 2”…””}”hjÎsbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h K6hjŸhžhubhÛ)”}”(hŒEnable header/data split::”h]”hŒEnable header/data split:”…””}”(hjÜhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K8hjŸhžhubjÍ)”}”(hŒ!ethtool -G eth0 tcp-data-split on”h]”hŒ!ethtool -G eth0 tcp-data-split on”…””}”hjêsbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h K:hjŸhžhubhÛ)”}”(hŒhjŸhžhubhÛ)”}”(hŒASet up flow steering, bearing in mind that queues are 0-indexed::”h]”hŒ@Set up flow steering, bearing in mind that queues are 0-indexed:”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K@hjŸhžhubjÍ)”}”(hŒ+ethtool -N eth0 flow-type tcp6 ... action 1”h]”hŒ+ethtool -N eth0 flow-type tcp6 ... action 1”…””}”hj"sbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h KBhjŸhžhubeh}”(h]”Œ setup-nic”ah ]”h"]”Œ setup nic”ah$]”h&]”uh1h´hjŽhžhhŸh³h K0ubhµ)”}”(hhh]”(hº)”}”(hŒSetup io_uring”h]”hŒSetup io_uring”…””}”(hj;hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hj8hžhhŸh³h KEubhÛ)”}”(hŒ…This section describes the low level io_uring kernel API. Please refer to liburing documentation for how to use the higher level API.”h]”hŒ…This section describes the low level io_uring kernel API. Please refer to liburing documentation for how to use the higher level API.”…””}”(hjIhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h KGhj8hžhubhÛ)”}”(hŒECreate an io_uring instance with the following required setup flags::”h]”hŒDCreate an io_uring instance with the following required setup flags:”…””}”(hjWhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h KJhj8hžhubjÍ)”}”(hŒHIORING_SETUP_SINGLE_ISSUER IORING_SETUP_DEFER_TASKRUN IORING_SETUP_CQE32”h]”hŒHIORING_SETUP_SINGLE_ISSUER IORING_SETUP_DEFER_TASKRUN IORING_SETUP_CQE32”…””}”hjesbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h KLhj8hžhubeh}”(h]”Œsetup-io-uring”ah ]”h"]”Œsetup io_uring”ah$]”h&]”uh1h´hjŽhžhhŸh³h KEubhµ)”}”(hhh]”(hº)”}”(hŒCreate memory area”h]”hŒCreate memory area”…””}”(hj~hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hj{hžhhŸh³h KQubhÛ)”}”(hŒ=Allocate userspace memory area for receiving zero copy data::”h]”hŒioprio |= IORING_RECV_MULTISHOT;”h]”hŒ˜struct io_uring_sqe *sqe; sqe = io_uring_get_sqe(ring); io_uring_prep_rw(IORING_OP_RECV_ZC, sqe, fd, NULL, 0, 0); sqe->ioprio |= IORING_RECV_MULTISHOT;”…””}”hjÌsbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h Kžhj­hžhubhÛ)”}”(hŒNow, submit and wait::”h]”hŒNow, submit and wait:”…””}”(hjÚhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K¤hj­hžhubjÍ)”}”(hŒ"io_uring_submit_and_wait(ring, 1);”h]”hŒ"io_uring_submit_and_wait(ring, 1);”…””}”hjèsbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h K¦hj­hžhubhÛ)”}”(hŒFinally, process completions::”h]”hŒFinally, process completions:”…””}”(hjöhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K¨hj­hžhubjÍ)”}”(hX€struct io_uring_cqe *cqe; unsigned int count = 0; unsigned int head; io_uring_for_each_cqe(ring, head, cqe) { struct io_uring_zcrx_cqe *rcqe = (struct io_uring_zcrx_cqe *)(cqe + 1); unsigned long mask = (1ULL << IORING_ZCRX_AREA_SHIFT) - 1; unsigned char *data = area_ptr + (rcqe->off & mask); /* do something with the data */ count++; } io_uring_cq_advance(ring, count);”h]”hX€struct io_uring_cqe *cqe; unsigned int count = 0; unsigned int head; io_uring_for_each_cqe(ring, head, cqe) { struct io_uring_zcrx_cqe *rcqe = (struct io_uring_zcrx_cqe *)(cqe + 1); unsigned long mask = (1ULL << IORING_ZCRX_AREA_SHIFT) - 1; unsigned char *data = area_ptr + (rcqe->off & mask); /* do something with the data */ count++; } io_uring_cq_advance(ring, count);”…””}”hjsbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h Kªhj­hžhubeh}”(h]”Œreceiving-data”ah ]”h"]”Œreceiving data”ah$]”h&]”uh1h´hjŽhžhhŸh³h Kšubhµ)”}”(hhh]”(hº)”}”(hŒRecycling buffers”h]”hŒRecycling buffers”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hjhžhhŸh³h KºubhÛ)”}”(hŒ4Return buffers back to the kernel to be used again::”h]”hŒ3Return buffers back to the kernel to be used again:”…””}”(hj+hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h K¼hjhžhubjÍ)”}”(hXHstruct io_uring_zcrx_rqe *rqe; unsigned mask = refill_ring.ring_entries - 1; rqe = &refill_ring.rqes[refill_ring.rq_tail & mask]; unsigned long area_offset = rcqe->off & ~IORING_ZCRX_AREA_MASK; rqe->off = area_offset | area_reg.rq_area_token; rqe->len = cqe->res; IO_URING_WRITE_ONCE(*refill_ring.ktail, ++refill_ring.rq_tail);”h]”hXHstruct io_uring_zcrx_rqe *rqe; unsigned mask = refill_ring.ring_entries - 1; rqe = &refill_ring.rqes[refill_ring.rq_tail & mask]; unsigned long area_offset = rcqe->off & ~IORING_ZCRX_AREA_MASK; rqe->off = area_offset | area_reg.rq_area_token; rqe->len = cqe->res; IO_URING_WRITE_ONCE(*refill_ring.ktail, ++refill_ring.rq_tail);”…””}”hj9sbah}”(h]”h ]”h"]”h$]”h&]”h±h²uh1jÌhŸh³h K¾hjhžhubeh}”(h]”Œrecycling-buffers”ah ]”h"]”Œrecycling buffers”ah$]”h&]”uh1h´hjŽhžhhŸh³h Kºubeh}”(h]”Œusage”ah ]”h"]”Œusage”ah$]”h&]”uh1h´hh¶hžhhŸh³h K-ubhµ)”}”(hhh]”(hº)”}”(hŒTesting”h]”hŒTesting”…””}”(hjZhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¹hjWhžhhŸh³h KÈubhÛ)”}”(hŒ9See ``tools/testing/selftests/drivers/net/hw/iou-zcrx.c``”h]”(hŒSee ”…””}”(hjhhžhhŸNh Nubjæ)”}”(hŒ5``tools/testing/selftests/drivers/net/hw/iou-zcrx.c``”h]”hŒ1tools/testing/selftests/drivers/net/hw/iou-zcrx.c”…””}”(hjphžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jåhjhubeh}”(h]”h ]”h"]”h$]”h&]”uh1hÚhŸh³h KÊhjWhžhubeh}”(h]”Œtesting”ah ]”h"]”Œtesting”ah$]”h&]”uh1h´hh¶hžhhŸh³h KÈubeh}”(h]”Œio-uring-zero-copy-rx”ah ]”h"]”Œio_uring zero copy rx”ah$]”h&]”uh1h´hhhžhhŸh³h Kubeh}”(h]”h ]”h"]”h$]”h&]”Œsource”h³uh1hŒcurrent_source”NŒ current_line”NŒsettings”Œdocutils.frontend”ŒValues”“”)”}”(h¹NŒ generator”NŒ datestamp”NŒ source_link”NŒ source_url”NŒ toc_backlinks”Œentry”Œfootnote_backlinks”KŒ sectnum_xform”KŒstrip_comments”NŒstrip_elements_with_classes”NŒ strip_classes”NŒ report_level”KŒ halt_level”KŒexit_status_level”KŒdebug”NŒwarning_stream”NŒ traceback”ˆŒinput_encoding”Œ utf-8-sig”Œinput_encoding_error_handler”Œstrict”Œoutput_encoding”Œutf-8”Œoutput_encoding_error_handler”j·Œerror_encoding”Œutf-8”Œerror_encoding_error_handler”Œbackslashreplace”Œ language_code”Œen”Œrecord_dependencies”NŒconfig”NŒ id_prefix”hŒauto_id_prefix”Œid”Œ dump_settings”NŒdump_internals”NŒdump_transforms”NŒdump_pseudo_xml”NŒexpose_internals”NŒstrict_visitor”NŒ_disable_config”NŒ_source”h³Œ _destination”NŒ _config_files”]”Œ7/var/lib/git/docbuild/linux/Documentation/docutils.conf”aŒfile_insertion_enabled”ˆŒ raw_enabled”KŒline_length_limit”M'Œpep_references”NŒ pep_base_url”Œhttps://peps.python.org/”Œpep_file_url_template”Œpep-%04d”Œrfc_references”NŒ rfc_base_url”Œ&https://datatracker.ietf.org/doc/html/”Œ tab_width”KŒtrim_footnote_reference_space”‰Œsyntax_highlight”Œlong”Œ smart_quotes”ˆŒsmartquotes_locales”]”Œcharacter_level_inline_markup”‰Œdoctitle_xform”‰Œ docinfo_xform”KŒsectsubtitle_xform”‰Œ image_loading”Œlink”Œembed_stylesheet”‰Œcloak_email_addresses”ˆŒsection_self_link”‰Œenv”NubŒreporter”NŒindirect_targets”]”Œsubstitution_defs”}”Œsubstitution_names”}”Œrefnames”}”Œrefids”}”Œnameids”}”(j‘jŽhïhìj‹jˆj5j2j\jYjƒj€jTjQj5j2jxjuj­jªjjjcj`jªj§jjjLjIj‰j†uŒ nametypes”}”(j‘‰hï‰j‹‰j5‰j\‰jƒ‰jT‰j5‰jx‰j­‰j‰jc‰jª‰j‰jL‰j‰‰uh}”(jŽh¶hìhÉjˆhòj2jjYj8j€j_jQjŽj2jŸjuj8jªj{jj°j`jj§jfjj­jIjj†jWuŒ footnote_refs”}”Œ citation_refs”}”Œ autofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ footnotes”]”Œ citations”]”Œautofootnote_start”KŒsymbol_footnote_start”KŒ id_counter”Œ collections”ŒCounter”“”}”…”R”Œparse_messages”]”Œtransform_messages”]”Œ transformer”NŒ include_log”]”Œ decoration”Nhžhub.