€•Ì—Œsphinx.addnodes”Œdocument”“”)”}”(Œ rawsource”Œ”Œchildren”]”(Œ translations”Œ LanguagesNode”“”)”}”(hhh]”(hŒ pending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ refdomain”Œstd”Œreftype”Œdoc”Œ reftarget”Œ/translations/zh_CN/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuŒtagname”hhh ubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/zh_TW/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/it_IT/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/ja_JP/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/ko_KR/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒPortuguese (Brazilian)”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/pt_BR/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒSpanish”…””}”hh–sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ/translations/sp_SP/bpf/ringbuf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h hhŒ _document”hŒsource”NŒline”NubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒBPF ring buffer”h]”hŒBPF ring buffer”…””}”(hh¼h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hºhh·h²hh³Œ9/var/lib/git/docbuild/linux/Documentation/bpf/ringbuf.rst”h´KubhŒ paragraph”“”)”}”(hŒPThis document describes BPF ring buffer design, API, and implementation details.”h]”hŒPThis document describes BPF ring buffer design, API, and implementation details.”…””}”(hhÍh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khh·h²hubhŒtopic”“”)”}”(hhh]”hŒ bullet_list”“”)”}”(hhh]”(hŒ list_item”“”)”}”(hhh]”hÌ)”}”(hhh]”hŒ reference”“”)”}”(hhh]”hŒ Motivation”…””}”(hhïh²hh³Nh´Nubah}”(h]”Œid1”ah ]”h"]”h$]”h&]”Œrefid”Œ motivation”uh1híhhêubah}”(h]”h ]”h"]”h$]”h&]”uh1hËhhçubah}”(h]”h ]”h"]”h$]”h&]”uh1håhhâubhæ)”}”(hhh]”hÌ)”}”(hhh]”hî)”}”(hhh]”hŒSemantics and APIs”…””}”(hjh²hh³Nh´Nubah}”(h]”Œid2”ah ]”h"]”h$]”h&]”Œrefid”Œsemantics-and-apis”uh1híhjubah}”(h]”h ]”h"]”h$]”h&]”uh1hËhj ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhhâubhæ)”}”(hhh]”hÌ)”}”(hhh]”hî)”}”(hhh]”hŒDesign and Implementation”…””}”(hj3h²hh³Nh´Nubah}”(h]”Œid3”ah ]”h"]”h$]”h&]”Œrefid”Œdesign-and-implementation”uh1híhj0ubah}”(h]”h ]”h"]”h$]”h&]”uh1hËhj-ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhhâubeh}”(h]”h ]”h"]”h$]”h&]”uh1hàhhÝh²hh³Nh´Nubah}”(h]”Œcontents”ah ]”(Œcontents”Œlocal”eh"]”Œcontents”ah$]”h&]”uh1hÛh³hÊh´K hh·h²hubh¶)”}”(hhh]”(h»)”}”(hŒ Motivation”h]”hŒ Motivation”…””}”(hjbh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”Œrefid”høuh1hºhj_h²hh³hÊh´K ubhÌ)”}”(hŒ¡There are two distinctive motivators for this work, which are not satisfied by existing perf buffer, which prompted creation of a new ring buffer implementation.”h]”hŒ¡There are two distinctive motivators for this work, which are not satisfied by existing perf buffer, which prompted creation of a new ring buffer implementation.”…””}”(hjqh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khj_h²hubhá)”}”(hhh]”(hæ)”}”(hŒEmore efficient memory utilization by sharing ring buffer across CPUs;”h]”hÌ)”}”(hj„h]”hŒEmore efficient memory utilization by sharing ring buffer across CPUs;”…””}”(hj†h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khj‚ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjh²hh³hÊh´Nubhæ)”}”(hŒ„preserving ordering of events that happen sequentially in time, even across multiple CPUs (e.g., fork/exec/exit events for a task). ”h]”hÌ)”}”(hŒƒpreserving ordering of events that happen sequentially in time, even across multiple CPUs (e.g., fork/exec/exit events for a task).”h]”hŒƒpreserving ordering of events that happen sequentially in time, even across multiple CPUs (e.g., fork/exec/exit events for a task).”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khj™ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjh²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”Œbullet”Œ-”uh1hàh³hÊh´Khj_h²hubhÌ)”}”(hX¤These two problems are independent, but perf buffer fails to satisfy both. Both are a result of a choice to have per-CPU perf ring buffer. Both can be also solved by having an MPSC implementation of ring buffer. The ordering problem could technically be solved for perf buffer with some in-kernel counting, but given the first one requires an MPSC buffer, the same solution would solve the second problem automatically.”h]”hX¤These two problems are independent, but perf buffer fails to satisfy both. Both are a result of a choice to have per-CPU perf ring buffer. Both can be also solved by having an MPSC implementation of ring buffer. The ordering problem could technically be solved for perf buffer with some in-kernel counting, but given the first one requires an MPSC buffer, the same solution would solve the second problem automatically.”…””}”(hj¹h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khj_h²hubeh}”(h]”hþah ]”h"]”Œ motivation”ah$]”h&]”uh1hµhh·h²hh³hÊh´K ubh¶)”}”(hhh]”(h»)”}”(hŒSemantics and APIs”h]”hŒSemantics and APIs”…””}”(hjÑh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”jpjuh1hºhjÎh²hh³hÊh´KubhÌ)”}”(hŒ§Single ring buffer is presented to BPF programs as an instance of BPF map of type ``BPF_MAP_TYPE_RINGBUF``. Two other alternatives considered, but ultimately rejected.”h]”(hŒRSingle ring buffer is presented to BPF programs as an instance of BPF map of type ”…””}”(hjßh²hh³Nh´NubhŒliteral”“”)”}”(hŒ``BPF_MAP_TYPE_RINGBUF``”h]”hŒBPF_MAP_TYPE_RINGBUF”…””}”(hjéh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjßubhŒ=. Two other alternatives considered, but ultimately rejected.”…””}”(hjßh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K hjÎh²hubhÌ)”}”(hXgOne way would be to, similar to ``BPF_MAP_TYPE_PERF_EVENT_ARRAY``, make ``BPF_MAP_TYPE_RINGBUF`` could represent an array of ring buffers, but not enforce "same CPU only" rule. This would be more familiar interface compatible with existing perf buffer use in BPF, but would fail if application needed more advanced logic to lookup ring buffer by arbitrary key. ``BPF_MAP_TYPE_HASH_OF_MAPS`` addresses this with current approach. Additionally, given the performance of BPF ringbuf, many use cases would just opt into a simple single ring buffer shared among all CPUs, for which current approach would be an overkill.”h]”(hŒ One way would be to, similar to ”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ!``BPF_MAP_TYPE_PERF_EVENT_ARRAY``”h]”hŒBPF_MAP_TYPE_PERF_EVENT_ARRAY”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhŒ, make ”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ``BPF_MAP_TYPE_RINGBUF``”h]”hŒBPF_MAP_TYPE_RINGBUF”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhX  could represent an array of ring buffers, but not enforce “same CPU only†rule. This would be more familiar interface compatible with existing perf buffer use in BPF, but would fail if application needed more advanced logic to lookup ring buffer by arbitrary key. ”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ``BPF_MAP_TYPE_HASH_OF_MAPS``”h]”hŒBPF_MAP_TYPE_HASH_OF_MAPS”…””}”(hj-h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhŒá addresses this with current approach. Additionally, given the performance of BPF ringbuf, many use cases would just opt into a simple single ring buffer shared among all CPUs, for which current approach would be an overkill.”…””}”(hjh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K$hjÎh²hubhÌ)”}”(hX´Another approach could introduce a new concept, alongside BPF map, to represent generic "container" object, which doesn't necessarily have key/value interface with lookup/update/delete operations. This approach would add a lot of extra infrastructure that has to be built for observability and verifier support. It would also add another concept that BPF developers would have to familiarize themselves with, new syntax in libbpf, etc. But then would really provide no additional benefits over the approach of using a map. ``BPF_MAP_TYPE_RINGBUF`` doesn't support lookup/update/delete operations, but so doesn't few other map types (e.g., queue and stack; array doesn't support delete, etc).”h]”(hXAnother approach could introduce a new concept, alongside BPF map, to represent generic “container†object, which doesn’t necessarily have key/value interface with lookup/update/delete operations. This approach would add a lot of extra infrastructure that has to be built for observability and verifier support. It would also add another concept that BPF developers would have to familiarize themselves with, new syntax in libbpf, etc. But then would really provide no additional benefits over the approach of using a map. ”…””}”(hjEh²hh³Nh´Nubjè)”}”(hŒ``BPF_MAP_TYPE_RINGBUF``”h]”hŒBPF_MAP_TYPE_RINGBUF”…””}”(hjMh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjEubhŒ– doesn’t support lookup/update/delete operations, but so doesn’t few other map types (e.g., queue and stack; array doesn’t support delete, etc).”…””}”(hjEh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K.hjÎh²hubhÌ)”}”(hXHThe approach chosen has an advantage of re-using existing BPF map infrastructure (introspection APIs in kernel, libbpf support, etc), being familiar concept (no need to teach users a new type of object in BPF program), and utilizing existing tooling (bpftool). For common scenario of using a single ring buffer for all CPUs, it's as simple and straightforward, as would be with a dedicated "container" object. On the other hand, by being a map, it can be combined with ``ARRAY_OF_MAPS`` and ``HASH_OF_MAPS`` map-in-maps to implement a wide variety of topologies, from one ring buffer for each CPU (e.g., as a replacement for perf buffer use cases), to a complicated application hashing/sharding of ring buffers (e.g., having a small pool of ring buffers with hashed task's tgid being a look up key to preserve order, but reduce contention).”h]”(hXÛThe approach chosen has an advantage of re-using existing BPF map infrastructure (introspection APIs in kernel, libbpf support, etc), being familiar concept (no need to teach users a new type of object in BPF program), and utilizing existing tooling (bpftool). For common scenario of using a single ring buffer for all CPUs, it’s as simple and straightforward, as would be with a dedicated “container†object. On the other hand, by being a map, it can be combined with ”…””}”(hjeh²hh³Nh´Nubjè)”}”(hŒ``ARRAY_OF_MAPS``”h]”hŒ ARRAY_OF_MAPS”…””}”(hjmh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjeubhŒ and ”…””}”(hjeh²hh³Nh´Nubjè)”}”(hŒ``HASH_OF_MAPS``”h]”hŒ HASH_OF_MAPS”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjeubhXO map-in-maps to implement a wide variety of topologies, from one ring buffer for each CPU (e.g., as a replacement for perf buffer use cases), to a complicated application hashing/sharding of ring buffers (e.g., having a small pool of ring buffers with hashed task’s tgid being a look up key to preserve order, but reduce contention).”…””}”(hjeh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K8hjÎh²hubhÌ)”}”(hŒ‰Key and value sizes are enforced to be zero. ``max_entries`` is used to specify the size of ring buffer and has to be a power of 2 value.”h]”(hŒ-Key and value sizes are enforced to be zero. ”…””}”(hj—h²hh³Nh´Nubjè)”}”(hŒ``max_entries``”h]”hŒ max_entries”…””}”(hjŸh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj—ubhŒM is used to specify the size of ring buffer and has to be a power of 2 value.”…””}”(hj—h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KEhjÎh²hubhÌ)”}”(hŒ|There are a bunch of similarities between perf buffer (``BPF_MAP_TYPE_PERF_EVENT_ARRAY``) and new BPF ring buffer semantics:”h]”(hŒ7There are a bunch of similarities between perf buffer (”…””}”(hj·h²hh³Nh´Nubjè)”}”(hŒ!``BPF_MAP_TYPE_PERF_EVENT_ARRAY``”h]”hŒBPF_MAP_TYPE_PERF_EVENT_ARRAY”…””}”(hj¿h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj·ubhŒ$) and new BPF ring buffer semantics:”…””}”(hj·h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KHhjÎh²hubhá)”}”(hhh]”(hæ)”}”(hŒvariable-length records;”h]”hÌ)”}”(hjÜh]”hŒvariable-length records;”…””}”(hjÞh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KKhjÚubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj×h²hh³hÊh´Nubhæ)”}”(hŒNif there is no more space left in ring buffer, reservation fails, no blocking;”h]”hÌ)”}”(hŒNif there is no more space left in ring buffer, reservation fails, no blocking;”h]”hŒNif there is no more space left in ring buffer, reservation fails, no blocking;”…””}”(hjõh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KLhjñubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj×h²hh³hÊh´Nubhæ)”}”(hŒcmemory-mappable data area for user-space applications for ease of consumption and high performance;”h]”hÌ)”}”(hŒcmemory-mappable data area for user-space applications for ease of consumption and high performance;”h]”hŒcmemory-mappable data area for user-space applications for ease of consumption and high performance;”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KNhj ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj×h²hh³hÊh´Nubhæ)”}”(hŒ*epoll notifications for new incoming data;”h]”hÌ)”}”(hj#h]”hŒ*epoll notifications for new incoming data;”…””}”(hj%h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KPhj!ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj×h²hh³hÊh´Nubhæ)”}”(hŒcbut still the ability to do busy polling for new data to achieve the lowest latency, if necessary. ”h]”hÌ)”}”(hŒbbut still the ability to do busy polling for new data to achieve the lowest latency, if necessary.”h]”hŒbbut still the ability to do busy polling for new data to achieve the lowest latency, if necessary.”…””}”(hj<h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KQhj8ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj×h²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”j·j¸uh1hàh³hÊh´KKhjÎh²hubhÌ)”}”(hŒ6BPF ringbuf provides two sets of APIs to BPF programs:”h]”hŒ6BPF ringbuf provides two sets of APIs to BPF programs:”…””}”(hjVh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KThjÎh²hubhá)”}”(hhh]”(hæ)”}”(hŒy``bpf_ringbuf_output()`` allows to *copy* data from one place to a ring buffer, similarly to ``bpf_perf_event_output()``;”h]”hÌ)”}”(hŒy``bpf_ringbuf_output()`` allows to *copy* data from one place to a ring buffer, similarly to ``bpf_perf_event_output()``;”h]”(jè)”}”(hŒ``bpf_ringbuf_output()``”h]”hŒbpf_ringbuf_output()”…””}”(hjoh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjkubhŒ allows to ”…””}”(hjkh²hh³Nh´NubhŒemphasis”“”)”}”(hŒ*copy*”h]”hŒcopy”…””}”(hjƒh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jhjkubhŒ4 data from one place to a ring buffer, similarly to ”…””}”(hjkh²hh³Nh´Nubjè)”}”(hŒ``bpf_perf_event_output()``”h]”hŒbpf_perf_event_output()”…””}”(hj•h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjkubhŒ;”…””}”(hjkh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KVhjgubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjdh²hh³hÊh´Nubhæ)”}”(hXÁ``bpf_ringbuf_reserve()``/``bpf_ringbuf_commit()``/``bpf_ringbuf_discard()`` APIs split the whole process into two steps. First, a fixed amount of space is reserved. If successful, a pointer to a data inside ring buffer data area is returned, which BPF programs can use similarly to a data inside array/hash maps. Once ready, this piece of memory is either committed or discarded. Discard is similar to commit, but makes consumer ignore the record. ”h]”hÌ)”}”(hXÀ``bpf_ringbuf_reserve()``/``bpf_ringbuf_commit()``/``bpf_ringbuf_discard()`` APIs split the whole process into two steps. First, a fixed amount of space is reserved. If successful, a pointer to a data inside ring buffer data area is returned, which BPF programs can use similarly to a data inside array/hash maps. Once ready, this piece of memory is either committed or discarded. Discard is similar to commit, but makes consumer ignore the record.”h]”(jè)”}”(hŒ``bpf_ringbuf_reserve()``”h]”hŒbpf_ringbuf_reserve()”…””}”(hj»h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj·ubhŒ/”…””}”(hj·h²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_commit()``”h]”hŒbpf_ringbuf_commit()”…””}”(hjÍh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj·ubhŒ/”…””}”hj·sbjè)”}”(hŒ``bpf_ringbuf_discard()``”h]”hŒbpf_ringbuf_discard()”…””}”(hjßh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj·ubhXt APIs split the whole process into two steps. First, a fixed amount of space is reserved. If successful, a pointer to a data inside ring buffer data area is returned, which BPF programs can use similarly to a data inside array/hash maps. Once ready, this piece of memory is either committed or discarded. Discard is similar to commit, but makes consumer ignore the record.”…””}”(hj·h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KXhj³ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjdh²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”j·j¸uh1hàh³hÊh´KVhjÎh²hubhÌ)”}”(hX<``bpf_ringbuf_output()`` has disadvantage of incurring extra memory copy, because record has to be prepared in some other place first. But it allows to submit records of the length that's not known to verifier beforehand. It also closely matches ``bpf_perf_event_output()``, so will simplify migration significantly.”h]”(jè)”}”(hŒ``bpf_ringbuf_output()``”h]”hŒbpf_ringbuf_output()”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhŒà has disadvantage of incurring extra memory copy, because record has to be prepared in some other place first. But it allows to submit records of the length that’s not known to verifier beforehand. It also closely matches ”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ``bpf_perf_event_output()``”h]”hŒbpf_perf_event_output()”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhŒ+, so will simplify migration significantly.”…””}”(hjh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K`hjÎh²hubhÌ)”}”(hX‹``bpf_ringbuf_reserve()`` avoids the extra copy of memory by providing a memory pointer directly to ring buffer memory. In a lot of cases records are larger than BPF stack space allows, so many programs have use extra per-CPU array as a temporary heap for preparing sample. bpf_ringbuf_reserve() avoid this needs completely. But in exchange, it only allows a known constant size of memory to be reserved, such that verifier can verify that BPF program can't access memory outside its reserved record space. bpf_ringbuf_output(), while slightly slower due to extra memory copy, covers some use cases that are not suitable for ``bpf_ringbuf_reserve()``.”h]”(jè)”}”(hŒ``bpf_ringbuf_reserve()``”h]”hŒbpf_ringbuf_reserve()”…””}”(hj5h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj1ubhXZ avoids the extra copy of memory by providing a memory pointer directly to ring buffer memory. In a lot of cases records are larger than BPF stack space allows, so many programs have use extra per-CPU array as a temporary heap for preparing sample. bpf_ringbuf_reserve() avoid this needs completely. But in exchange, it only allows a known constant size of memory to be reserved, such that verifier can verify that BPF program can’t access memory outside its reserved record space. bpf_ringbuf_output(), while slightly slower due to extra memory copy, covers some use cases that are not suitable for ”…””}”(hj1h²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_reserve()``”h]”hŒbpf_ringbuf_reserve()”…””}”(hjGh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj1ubhŒ.”…””}”(hj1h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KfhjÎh²hubhÌ)”}”(hX]The difference between commit and discard is very small. Discard just marks a record as discarded, and such records are supposed to be ignored by consumer code. Discard is useful for some advanced use-cases, such as ensuring all-or-nothing multi-record submission, or emulating temporary ``malloc()``/``free()`` within single BPF program invocation.”h]”(hX The difference between commit and discard is very small. Discard just marks a record as discarded, and such records are supposed to be ignored by consumer code. Discard is useful for some advanced use-cases, such as ensuring all-or-nothing multi-record submission, or emulating temporary ”…””}”(hj_h²hh³Nh´Nubjè)”}”(hŒ ``malloc()``”h]”hŒmalloc()”…””}”(hjgh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj_ubhŒ/”…””}”(hj_h²hh³Nh´Nubjè)”}”(hŒ ``free()``”h]”hŒfree()”…””}”(hjyh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj_ubhŒ& within single BPF program invocation.”…””}”(hj_h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KphjÎh²hubhÌ)”}”(hŒÇEach reserved record is tracked by verifier through existing reference-tracking logic, similar to socket ref-tracking. It is thus impossible to reserve a record, but forget to submit (or discard) it.”h]”hŒÇEach reserved record is tracked by verifier through existing reference-tracking logic, similar to socket ref-tracking. It is thus impossible to reserve a record, but forget to submit (or discard) it.”…””}”(hj‘h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KvhjÎh²hubhÌ)”}”(hŒm``bpf_ringbuf_query()`` helper allows to query various properties of ring buffer. Currently 4 are supported:”h]”(jè)”}”(hŒ``bpf_ringbuf_query()``”h]”hŒbpf_ringbuf_query()”…””}”(hj£h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjŸubhŒV helper allows to query various properties of ring buffer. Currently 4 are supported:”…””}”(hjŸh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KzhjÎh²hubhá)”}”(hhh]”(hæ)”}”(hŒG``BPF_RB_AVAIL_DATA`` returns amount of unconsumed data in ring buffer;”h]”hÌ)”}”(hjÀh]”(jè)”}”(hŒ``BPF_RB_AVAIL_DATA``”h]”hŒBPF_RB_AVAIL_DATA”…””}”(hjÅh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjÂubhŒ2 returns amount of unconsumed data in ring buffer;”…””}”(hjÂh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K}hj¾ubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj»h²hh³hÊh´Nubhæ)”}”(hŒ5``BPF_RB_RING_SIZE`` returns the size of ring buffer;”h]”hÌ)”}”(hjåh]”(jè)”}”(hŒ``BPF_RB_RING_SIZE``”h]”hŒBPF_RB_RING_SIZE”…””}”(hjêh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjçubhŒ! returns the size of ring buffer;”…””}”(hjçh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K~hjãubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj»h²hh³hÊh´Nubhæ)”}”(hŒm``BPF_RB_CONS_POS``/``BPF_RB_PROD_POS`` returns current logical position of consumer/producer, respectively. ”h]”hÌ)”}”(hŒl``BPF_RB_CONS_POS``/``BPF_RB_PROD_POS`` returns current logical position of consumer/producer, respectively.”h]”(jè)”}”(hŒ``BPF_RB_CONS_POS``”h]”hŒBPF_RB_CONS_POS”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj ubhŒ/”…””}”(hj h²hh³Nh´Nubjè)”}”(hŒ``BPF_RB_PROD_POS``”h]”hŒBPF_RB_PROD_POS”…””}”(hj"h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhj ubhŒE returns current logical position of consumer/producer, respectively.”…””}”(hj h²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Khjubah}”(h]”h ]”h"]”h$]”h&]”uh1håhj»h²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”j·j¸uh1hàh³hÊh´K}hjÎh²hubhÌ)”}”(hX!Returned values are momentarily snapshots of ring buffer state and could be off by the time helper returns, so this should be used only for debugging/reporting reasons or for implementing various heuristics, that take into account highly-changeable nature of some of those characteristics.”h]”hX!Returned values are momentarily snapshots of ring buffer state and could be off by the time helper returns, so this should be used only for debugging/reporting reasons or for implementing various heuristics, that take into account highly-changeable nature of some of those characteristics.”…””}”(hjFh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K‚hjÎh²hubhÌ)”}”(hXÊOne such heuristic might involve more fine-grained control over poll/epoll notifications about new data availability in ring buffer. Together with ``BPF_RB_NO_WAKEUP``/``BPF_RB_FORCE_WAKEUP`` flags for output/commit/discard helpers, it allows BPF program a high degree of control and, e.g., more efficient batched notifications. Default self-balancing strategy, though, should be adequate for most applications and will work reliable and efficiently already.”h]”(hŒ“One such heuristic might involve more fine-grained control over poll/epoll notifications about new data availability in ring buffer. Together with ”…””}”(hjTh²hh³Nh´Nubjè)”}”(hŒ``BPF_RB_NO_WAKEUP``”h]”hŒBPF_RB_NO_WAKEUP”…””}”(hj\h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjTubhŒ/”…””}”(hjTh²hh³Nh´Nubjè)”}”(hŒ``BPF_RB_FORCE_WAKEUP``”h]”hŒBPF_RB_FORCE_WAKEUP”…””}”(hjnh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjTubhX  flags for output/commit/discard helpers, it allows BPF program a high degree of control and, e.g., more efficient batched notifications. Default self-balancing strategy, though, should be adequate for most applications and will work reliable and efficiently already.”…””}”(hjTh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K‡hjÎh²hubeh}”(h]”j ah ]”h"]”Œsemantics and apis”ah$]”h&]”uh1hµhh·h²hh³hÊh´Kubh¶)”}”(hhh]”(h»)”}”(hŒDesign and Implementation”h]”hŒDesign and Implementation”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”jpj<uh1hºhjh²hh³hÊh´KubhÌ)”}”(hX¬This reserve/commit schema allows a natural way for multiple producers, either on different CPUs or even on the same CPU/in the same BPF program, to reserve independent records and work with them without blocking other producers. This means that if BPF program was interrupted by another BPF program sharing the same ring buffer, they will both get a record reserved (provided there is enough space left) and can work with it and submit it independently. This applies to NMI context as well, except that due to using a spinlock during reservation, in NMI context, ``bpf_ringbuf_reserve()`` might fail to get a lock, in which case reservation will fail even if ring buffer is not full.”h]”(hX4This reserve/commit schema allows a natural way for multiple producers, either on different CPUs or even on the same CPU/in the same BPF program, to reserve independent records and work with them without blocking other producers. This means that if BPF program was interrupted by another BPF program sharing the same ring buffer, they will both get a record reserved (provided there is enough space left) and can work with it and submit it independently. This applies to NMI context as well, except that due to using a spinlock during reservation, in NMI context, ”…””}”(hjžh²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_reserve()``”h]”hŒbpf_ringbuf_reserve()”…””}”(hj¦h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjžubhŒ_ might fail to get a lock, in which case reservation will fail even if ring buffer is not full.”…””}”(hjžh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K’hjh²hubhÌ)”}”(hŒÎThe ring buffer itself internally is implemented as a power-of-2 sized circular buffer, with two logical and ever-increasing counters (which might wrap around on 32-bit architectures, that's not a problem):”h]”hŒÐThe ring buffer itself internally is implemented as a power-of-2 sized circular buffer, with two logical and ever-increasing counters (which might wrap around on 32-bit architectures, that’s not a problem):”…””}”(hj¾h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´Kœhjh²hubhá)”}”(hhh]”(hæ)”}”(hŒOconsumer counter shows up to which logical position consumer consumed the data;”h]”hÌ)”}”(hŒOconsumer counter shows up to which logical position consumer consumed the data;”h]”hŒOconsumer counter shows up to which logical position consumer consumed the data;”…””}”(hjÓh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K hjÏubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjÌh²hh³hÊh´Nubhæ)”}”(hŒCproducer counter denotes amount of data reserved by all producers. ”h]”hÌ)”}”(hŒBproducer counter denotes amount of data reserved by all producers.”h]”hŒBproducer counter denotes amount of data reserved by all producers.”…””}”(hjëh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K¢hjçubah}”(h]”h ]”h"]”h$]”h&]”uh1håhjÌh²hh³hÊh´Nubeh}”(h]”h ]”h"]”h$]”h&]”j·j¸uh1hàh³hÊh´K hjh²hubhÌ)”}”(hXŸEach time a record is reserved, producer that "owns" the record will successfully advance producer counter. At that point, data is still not yet ready to be consumed, though. Each record has 8 byte header, which contains the length of reserved record, as well as two extra bits: busy bit to denote that record is still being worked on, and discard bit, which might be set at commit time if record is discarded. In the latter case, consumer is supposed to skip the record and move on to the next one. Record header also encodes record's relative offset from the beginning of ring buffer data area (in pages). This allows ``bpf_ringbuf_commit()``/``bpf_ringbuf_discard()`` to accept only the pointer to the record itself, without requiring also the pointer to ring buffer itself. Ring buffer memory location will be restored from record metadata header. This significantly simplifies verifier, as well as improving API usability.”h]”(hXrEach time a record is reserved, producer that “owns†the record will successfully advance producer counter. At that point, data is still not yet ready to be consumed, though. Each record has 8 byte header, which contains the length of reserved record, as well as two extra bits: busy bit to denote that record is still being worked on, and discard bit, which might be set at commit time if record is discarded. In the latter case, consumer is supposed to skip the record and move on to the next one. Record header also encodes record’s relative offset from the beginning of ring buffer data area (in pages). This allows ”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_commit()``”h]”hŒbpf_ringbuf_commit()”…””}”(hj h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhŒ/”…””}”(hjh²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_discard()``”h]”hŒbpf_ringbuf_discard()”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjubhX to accept only the pointer to the record itself, without requiring also the pointer to ring buffer itself. Ring buffer memory location will be restored from record metadata header. This significantly simplifies verifier, as well as improving API usability.”…””}”(hjh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K¤hjh²hubhÌ)”}”(hX£Producer counter increments are serialized under spinlock, so there is a strict ordering between reservations. Commits, on the other hand, are completely lockless and independent. All records become available to consumer in the order of reservations, but only after all previous records where already committed. It is thus possible for slow producers to temporarily hold off submitted records, that were reserved later.”h]”hX£Producer counter increments are serialized under spinlock, so there is a strict ordering between reservations. Commits, on the other hand, are completely lockless and independent. All records become available to consumer in the order of reservations, but only after all previous records where already committed. It is thus possible for slow producers to temporarily hold off submitted records, that were reserved later.”…””}”(hj7h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K²hjh²hubhÌ)”}”(hXZOne interesting implementation bit, that significantly simplifies (and thus speeds up as well) implementation of both producers and consumers is how data area is mapped twice contiguously back-to-back in the virtual memory. This allows to not take any special measures for samples that have to wrap around at the end of the circular buffer data area, because the next page after the last data page would be first data page again, and thus the sample will still appear completely contiguous in virtual memory. See comment and a simple ASCII diagram showing this visually in ``bpf_ringbuf_area_alloc()``.”h]”(hX=One interesting implementation bit, that significantly simplifies (and thus speeds up as well) implementation of both producers and consumers is how data area is mapped twice contiguously back-to-back in the virtual memory. This allows to not take any special measures for samples that have to wrap around at the end of the circular buffer data area, because the next page after the last data page would be first data page again, and thus the sample will still appear completely contiguous in virtual memory. See comment and a simple ASCII diagram showing this visually in ”…””}”(hjEh²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_area_alloc()``”h]”hŒbpf_ringbuf_area_alloc()”…””}”(hjMh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjEubhŒ.”…””}”(hjEh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´K¹hjh²hubhÌ)”}”(hXÈAnother feature that distinguishes BPF ringbuf from perf ring buffer is a self-pacing notifications of new data being availability. ``bpf_ringbuf_commit()`` implementation will send a notification of new record being available after commit only if consumer has already caught up right up to the record being committed. If not, consumer still has to catch up and thus will see new data anyways without needing an extra poll notification. Benchmarks (see tools/testing/selftests/bpf/benchs/bench_ringbufs.c) show that this allows to achieve a very high throughput without having to resort to tricks like "notify only every Nth sample", which are necessary with perf buffer. For extreme cases, when BPF program wants more manual control of notifications, commit/discard/output helpers accept ``BPF_RB_NO_WAKEUP`` and ``BPF_RB_FORCE_WAKEUP`` flags, which give full control over notifications of data availability, but require extra caution and diligence in using this API.”h]”(hŒ„Another feature that distinguishes BPF ringbuf from perf ring buffer is a self-pacing notifications of new data being availability. ”…””}”(hjeh²hh³Nh´Nubjè)”}”(hŒ``bpf_ringbuf_commit()``”h]”hŒbpf_ringbuf_commit()”…””}”(hjmh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjeubhX} implementation will send a notification of new record being available after commit only if consumer has already caught up right up to the record being committed. If not, consumer still has to catch up and thus will see new data anyways without needing an extra poll notification. Benchmarks (see tools/testing/selftests/bpf/benchs/bench_ringbufs.c) show that this allows to achieve a very high throughput without having to resort to tricks like “notify only every Nth sampleâ€, which are necessary with perf buffer. For extreme cases, when BPF program wants more manual control of notifications, commit/discard/output helpers accept ”…””}”(hjeh²hh³Nh´Nubjè)”}”(hŒ``BPF_RB_NO_WAKEUP``”h]”hŒBPF_RB_NO_WAKEUP”…””}”(hjh²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjeubhŒ and ”…””}”(hjeh²hh³Nh´Nubjè)”}”(hŒ``BPF_RB_FORCE_WAKEUP``”h]”hŒBPF_RB_FORCE_WAKEUP”…””}”(hj‘h²hh³Nh´Nubah}”(h]”h ]”h"]”h$]”h&]”uh1jçhjeubhŒƒ flags, which give full control over notifications of data availability, but require extra caution and diligence in using this API.”…””}”(hjeh²hh³Nh´Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hËh³hÊh´KÂhjh²hubeh}”(h]”jBah ]”h"]”Œdesign and implementation”ah$]”h&]”uh1hµhh·h²hh³hÊh´Kubeh}”(h]”Œbpf-ring-buffer”ah ]”h"]”Œbpf ring buffer”ah$]”h&]”uh1hµhhh²hh³hÊh´Kubeh}”(h]”h ]”h"]”h$]”h&]”Œsource”hÊuh1hŒcurrent_source”NŒ current_line”NŒsettings”Œdocutils.frontend”ŒValues”“”)”}”(hºNŒ generator”NŒ datestamp”NŒ source_link”NŒ source_url”NŒ toc_backlinks”Œentry”Œfootnote_backlinks”KŒ sectnum_xform”KŒstrip_comments”NŒstrip_elements_with_classes”NŒ strip_classes”NŒ report_level”KŒ halt_level”KŒexit_status_level”KŒdebug”NŒwarning_stream”NŒ traceback”ˆŒinput_encoding”Œ utf-8-sig”Œinput_encoding_error_handler”Œstrict”Œoutput_encoding”Œutf-8”Œoutput_encoding_error_handler”jÛŒerror_encoding”Œutf-8”Œerror_encoding_error_handler”Œbackslashreplace”Œ language_code”Œen”Œrecord_dependencies”NŒconfig”NŒ id_prefix”hŒauto_id_prefix”Œid”Œ dump_settings”NŒdump_internals”NŒdump_transforms”NŒdump_pseudo_xml”NŒexpose_internals”NŒstrict_visitor”NŒ_disable_config”NŒ_source”hÊŒ _destination”NŒ _config_files”]”Œ7/var/lib/git/docbuild/linux/Documentation/docutils.conf”aŒfile_insertion_enabled”ˆŒ raw_enabled”KŒline_length_limit”M'Œpep_references”NŒ pep_base_url”Œhttps://peps.python.org/”Œpep_file_url_template”Œpep-%04d”Œrfc_references”NŒ rfc_base_url”Œ&https://datatracker.ietf.org/doc/html/”Œ tab_width”KŒtrim_footnote_reference_space”‰Œsyntax_highlight”Œlong”Œ smart_quotes”ˆŒsmartquotes_locales”]”Œcharacter_level_inline_markup”‰Œdoctitle_xform”‰Œ docinfo_xform”KŒsectsubtitle_xform”‰Œ image_loading”Œlink”Œembed_stylesheet”‰Œcloak_email_addresses”ˆŒsection_self_link”‰Œenv”NubŒreporter”NŒindirect_targets”]”Œsubstitution_defs”}”Œsubstitution_names”}”Œrefnames”}”Œrefids”}”Œnameids”}”(jµj²j\jWjËhþjŠj j­jBuŒ nametypes”}”(jµ‰j\‰jˉjŠ‰j­‰uh}”(j²h·jWhÝhþj_j jÎjBjhøhïjjj<j3uŒ footnote_refs”}”Œ citation_refs”}”Œ autofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ footnotes”]”Œ citations”]”Œautofootnote_start”KŒsymbol_footnote_start”KŒ id_counter”Œ collections”ŒCounter”“”}”jéKs…”R”Œparse_messages”]”Œtransform_messages”]”Œ transformer”NŒ include_log”]”Œ decoration”Nh²hub.