sphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget1/translations/zh_CN/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget1/translations/zh_TW/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget1/translations/it_IT/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget1/translations/ja_JP/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget1/translations/ko_KR/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget1/translations/sp_SP/PCI/endpoint/pci-ntb-functionmodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhK/var/lib/git/docbuild/linux/Documentation/PCI/endpoint/pci-ntb-function.rsthKubhsection)}(hhh](htitle)}(hPCI NTB Functionh]hPCI NTB Function}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh field_list)}(hhh]hfield)}(hhh](h field_name)}(hAuthorh]hAuthor}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhKubh field_body)}(h'Kishon Vijay Abraham I h]h paragraph)}(h&Kishon Vijay Abraham I h](hKishon Vijay Abraham I <}(hhhhhNhNubh reference)}(h kishon@ti.comh]h kishon@ti.com}(hhhhhNhNubah}(h]h ]h"]h$]h&]refurimailto:kishon@ti.comuh1hhhubh>}(hhhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhhubah}(h]h ]h"]h$]h&]uh1hhhubeh}(h]h ]h"]h$]h&]uh1hhhhKhhhhubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hX}PCI Non-Transparent Bridges (NTB) allow two host systems to communicate with each other by exposing each host as a device to the other host. NTBs typically support the ability to generate interrupts on the remote machine, expose memory ranges as BARs, and perform DMA. They also support scratchpads, which are areas of memory within the NTB that are accessible from both machines.h]hX}PCI Non-Transparent Bridges (NTB) allow two host systems to communicate with each other by exposing each host as a device to the other host. NTBs typically support the ability to generate interrupts on the remote machine, expose memory ranges as BARs, and perform DMA. They also support scratchpads, which are areas of memory within the NTB that are accessible from both machines.}(hj!hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hPCI NTB Function allows two different systems (or hosts) to communicate with each other by configuring the endpoint instances in such a way that transactions from one system are routed to the other system.h]hPCI NTB Function allows two different systems (or hosts) to communicate with each other by configuring the endpoint instances in such a way that transactions from one system are routed to the other system.}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hXMIn the below diagram, PCI NTB function configures the SoC with multiple PCI Endpoint (EP) instances in such a way that transactions from one EP controller are routed to the other EP controller. Once PCI NTB function configures the SoC with multiple EP instances, HOST1 and HOST2 can communicate with each other using SoC as a bridge.h]hXMIn the below diagram, PCI NTB function configures the SoC with multiple PCI Endpoint (EP) instances in such a way that transactions from one EP controller are routed to the other EP controller. Once PCI NTB function configures the SoC with multiple EP instances, HOST1 and HOST2 can communicate with each other using SoC as a bridge.}(hj=hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh literal_block)}(hX4 +-------------+ +-------------+ | | | | | HOST1 | | HOST2 | | | | | +------^------+ +------^------+ | | | | +---------|-------------------------------------------------|---------+ | +------v------+ +------v------+ | | | | | | | | | EP | | EP | | | | CONTROLLER1 | | CONTROLLER2 | | | | <-----------------------------------> | | | | | | | | | | | | | | | | | SoC With Multiple EP Instances | | | | | | (Configured using NTB Function) | | | | +-------------+ +-------------+ | +---------------------------------------------------------------------+h]hX4 +-------------+ +-------------+ | | | | | HOST1 | | HOST2 | | | | | +------^------+ +------^------+ | | | | +---------|-------------------------------------------------|---------+ | +------v------+ +------v------+ | | | | | | | | | EP | | EP | | | | CONTROLLER1 | | CONTROLLER2 | | | | <-----------------------------------> | | | | | | | | | | | | | | | | | SoC With Multiple EP Instances | | | | | | (Configured using NTB Function) | | | | +-------------+ +-------------+ | +---------------------------------------------------------------------+}hjMsbah}(h]h ]h"]h$]h&]hhforcelanguagetexthighlight_args}uh1jKhhhKhhhhubh)}(hhh](h)}(h$Constructs used for Implementing NTBh]h$Constructs used for Implementing NTB}(hjchhhNhNubah}(h]h ]h"]h$]h&]uh1hhj`hhhhhK1ubh block_quote)}(h}1) Config Region 2) Self Scratchpad Registers 3) Peer Scratchpad Registers 4) Doorbell (DB) Registers 5) Memory Window (MW) h]henumerated_list)}(hhh](h list_item)}(h Config Regionh]h)}(hjh]h Config Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK3hj~ubah}(h]h ]h"]h$]h&]uh1j|hjyubj})}(hSelf Scratchpad Registersh]h)}(hjh]hSelf Scratchpad Registers}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK4hjubah}(h]h ]h"]h$]h&]uh1j|hjyubj})}(hPeer Scratchpad Registersh]h)}(hjh]hPeer Scratchpad Registers}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK5hjubah}(h]h ]h"]h$]h&]uh1j|hjyubj})}(hDoorbell (DB) Registersh]h)}(hjh]hDoorbell (DB) Registers}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK6hjubah}(h]h ]h"]h$]h&]uh1j|hjyubj})}(hMemory Window (MW) h]h)}(hMemory Window (MW)h]hMemory Window (MW)}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK7hjubah}(h]h ]h"]h$]h&]uh1j|hjyubeh}(h]h ]h"]h$]h&]enumtypearabicprefixhsuffix)uh1jwhjsubah}(h]h ]h"]h$]h&]uh1jqhhhK3hj`hhubh)}(hhh](h)}(hConfig Region:h]hConfig Region:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhK;ubh)}(hXZConfig Region is a construct that is specific to NTB implemented using NTB Endpoint Function Driver. The host and endpoint side NTB function driver will exchange information with each other using this region. Config Region has Control/Status Registers for configuring the Endpoint Controller. Host can write into this region for configuring the outbound Address Translation Unit (ATU) and to indicate the link status. Endpoint can indicate the status of commands issued by host in this region. Endpoint can also indicate the scratchpad offset and number of memory windows to the host using this region.h]hXZConfig Region is a construct that is specific to NTB implemented using NTB Endpoint Function Driver. The host and endpoint side NTB function driver will exchange information with each other using this region. Config Region has Control/Status Registers for configuring the Endpoint Controller. Host can write into this region for configuring the outbound Address Translation Unit (ATU) and to indicate the link status. Endpoint can indicate the status of commands issued by host in this region. Endpoint can also indicate the scratchpad offset and number of memory windows to the host using this region.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK=hjhhubh)}(hLThe format of Config Region is given below. All the fields here are 32 bits.h]hLThe format of Config Region is given below. All the fields here are 32 bits.}(hj"hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKFhjhhubjL)}(hX +------------------------+ | COMMAND | +------------------------+ | ARGUMENT | +------------------------+ | STATUS | +------------------------+ | TOPOLOGY | +------------------------+ | ADDRESS (LOWER 32) | +------------------------+ | ADDRESS (UPPER 32) | +------------------------+ | SIZE | +------------------------+ | NO OF MEMORY WINDOW | +------------------------+ | MEMORY WINDOW1 OFFSET | +------------------------+ | SPAD OFFSET | +------------------------+ | SPAD COUNT | +------------------------+ | DB ENTRY SIZE | +------------------------+ | DB DATA | +------------------------+ | : | +------------------------+ | : | +------------------------+ | DB DATA | +------------------------+ COMMAND: NTB function supports three commands: CMD_CONFIGURE_DOORBELL (0x1): Command to configure doorbell. Before invoking this command, the host should allocate and initialize MSI/MSI-X vectors (i.e., initialize the MSI/MSI-X Capability in the Endpoint). The endpoint on receiving this command will configure the outbound ATU such that transactions to Doorbell BAR will be routed to the MSI/MSI-X address programmed by the host. The ARGUMENT register should be populated with number of DBs to configure (in the lower 16 bits) and if MSI or MSI-X should be configured (BIT 16). CMD_CONFIGURE_MW (0x2): Command to configure memory window (MW). The host invokes this command after allocating a buffer that can be accessed by remote host. The allocated address should be programmed in the ADDRESS register (64 bit), the size should be programmed in the SIZE register and the memory window index should be programmed in the ARGUMENT register. The endpoint on receiving this command will configure the outbound ATU such that transactions to MW BAR are routed to the address provided by the host. CMD_LINK_UP (0x3): Command to indicate an NTB application is bound to the EP device on the host side. Once the endpoint receives this command from both the hosts, the endpoint will raise a LINK_UP event to both the hosts to indicate the host NTB applications can start communicating with each other. ARGUMENT: The value of this register is based on the commands issued in command register. See COMMAND section for more information. TOPOLOGY: Set to NTB_TOPO_B2B_USD for Primary interface Set to NTB_TOPO_B2B_DSD for Secondary interface ADDRESS/SIZE: Address and Size to be used while configuring the memory window. See "CMD_CONFIGURE_MW" for more info. MEMORY WINDOW1 OFFSET: Memory Window 1 and Doorbell registers are packed together in the same BAR. The initial portion of the region will have doorbell registers and the latter portion of the region is for memory window 1. This register will specify the offset of the memory window 1. NO OF MEMORY WINDOW: Specifies the number of memory windows supported by the NTB device. SPAD OFFSET: Self scratchpad region and config region are packed together in the same BAR. The initial portion of the region will have config region and the latter portion of the region is for self scratchpad. This register will specify the offset of the self scratchpad registers. SPAD COUNT: Specifies the number of scratchpad registers supported by the NTB device. DB ENTRY SIZE: Used to determine the offset within the DB BAR that should be written in order to raise doorbell. EPF NTB can use either MSI or MSI-X to ring doorbell (MSI-X support will be added later). MSI uses same address for all the interrupts and MSI-X can provide different addresses for different interrupts. The MSI/MSI-X address is provided by the host and the address it gives is based on the MSI/MSI-X implementation supported by the host. For instance, ARM platform using GIC ITS will have the same MSI-X address for all the interrupts. In order to support all the combinations and use the same mechanism for both MSI and MSI-X, EPF NTB allocates a separate region in the Outbound Address Space for each of the interrupts. This region will be mapped to the MSI/MSI-X address provided by the host. If a host provides the same address for all the interrupts, all the regions will be translated to the same address. If a host provides different addresses, the regions will be translated to different addresses. This will ensure there is no difference while raising the doorbell. DB DATA: EPF NTB supports 32 interrupts, so there are 32 DB DATA registers. This holds the MSI/MSI-X data that has to be written to MSI address for raising doorbell interrupt. This will be populated by EPF NTB while invoking CMD_CONFIGURE_DOORBELL.h]hX +------------------------+ | COMMAND | +------------------------+ | ARGUMENT | +------------------------+ | STATUS | +------------------------+ | TOPOLOGY | +------------------------+ | ADDRESS (LOWER 32) | +------------------------+ | ADDRESS (UPPER 32) | +------------------------+ | SIZE | +------------------------+ | NO OF MEMORY WINDOW | +------------------------+ | MEMORY WINDOW1 OFFSET | +------------------------+ | SPAD OFFSET | +------------------------+ | SPAD COUNT | +------------------------+ | DB ENTRY SIZE | +------------------------+ | DB DATA | +------------------------+ | : | +------------------------+ | : | +------------------------+ | DB DATA | +------------------------+ COMMAND: NTB function supports three commands: CMD_CONFIGURE_DOORBELL (0x1): Command to configure doorbell. Before invoking this command, the host should allocate and initialize MSI/MSI-X vectors (i.e., initialize the MSI/MSI-X Capability in the Endpoint). The endpoint on receiving this command will configure the outbound ATU such that transactions to Doorbell BAR will be routed to the MSI/MSI-X address programmed by the host. The ARGUMENT register should be populated with number of DBs to configure (in the lower 16 bits) and if MSI or MSI-X should be configured (BIT 16). CMD_CONFIGURE_MW (0x2): Command to configure memory window (MW). The host invokes this command after allocating a buffer that can be accessed by remote host. The allocated address should be programmed in the ADDRESS register (64 bit), the size should be programmed in the SIZE register and the memory window index should be programmed in the ARGUMENT register. The endpoint on receiving this command will configure the outbound ATU such that transactions to MW BAR are routed to the address provided by the host. CMD_LINK_UP (0x3): Command to indicate an NTB application is bound to the EP device on the host side. Once the endpoint receives this command from both the hosts, the endpoint will raise a LINK_UP event to both the hosts to indicate the host NTB applications can start communicating with each other. ARGUMENT: The value of this register is based on the commands issued in command register. See COMMAND section for more information. TOPOLOGY: Set to NTB_TOPO_B2B_USD for Primary interface Set to NTB_TOPO_B2B_DSD for Secondary interface ADDRESS/SIZE: Address and Size to be used while configuring the memory window. See "CMD_CONFIGURE_MW" for more info. MEMORY WINDOW1 OFFSET: Memory Window 1 and Doorbell registers are packed together in the same BAR. The initial portion of the region will have doorbell registers and the latter portion of the region is for memory window 1. This register will specify the offset of the memory window 1. NO OF MEMORY WINDOW: Specifies the number of memory windows supported by the NTB device. SPAD OFFSET: Self scratchpad region and config region are packed together in the same BAR. The initial portion of the region will have config region and the latter portion of the region is for self scratchpad. This register will specify the offset of the self scratchpad registers. SPAD COUNT: Specifies the number of scratchpad registers supported by the NTB device. DB ENTRY SIZE: Used to determine the offset within the DB BAR that should be written in order to raise doorbell. EPF NTB can use either MSI or MSI-X to ring doorbell (MSI-X support will be added later). MSI uses same address for all the interrupts and MSI-X can provide different addresses for different interrupts. The MSI/MSI-X address is provided by the host and the address it gives is based on the MSI/MSI-X implementation supported by the host. For instance, ARM platform using GIC ITS will have the same MSI-X address for all the interrupts. In order to support all the combinations and use the same mechanism for both MSI and MSI-X, EPF NTB allocates a separate region in the Outbound Address Space for each of the interrupts. This region will be mapped to the MSI/MSI-X address provided by the host. If a host provides the same address for all the interrupts, all the regions will be translated to the same address. If a host provides different addresses, the regions will be translated to different addresses. This will ensure there is no difference while raising the doorbell. DB DATA: EPF NTB supports 32 interrupts, so there are 32 DB DATA registers. This holds the MSI/MSI-X data that has to be written to MSI address for raising doorbell interrupt. This will be populated by EPF NTB while invoking CMD_CONFIGURE_DOORBELL.}hj0sbah}(h]h ]h"]h$]h&]hhj[j\textj^}uh1jKhhhKHhjhhubeh}(h] config-regionah ]h"]config region:ah$]h&]uh1hhj`hhhhhK;ubh)}(hhh](h)}(hScratchpad Registers:h]hScratchpad Registers:}(hjKhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjHhhhhhKubjr)}(hXEach host has its own register space allocated in the memory of NTB endpoint controller. They are both readable and writable from both sides of the bridge. They are used by applications built over NTB and can be used to pass control and status information between both sides of a device. Scratchpad registers has 2 parts 1) Self Scratchpad: Host's own register space 2) Peer Scratchpad: Remote host's register space. h](h)}(hXEach host has its own register space allocated in the memory of NTB endpoint controller. They are both readable and writable from both sides of the bridge. They are used by applications built over NTB and can be used to pass control and status information between both sides of a device.h]hXEach host has its own register space allocated in the memory of NTB endpoint controller. They are both readable and writable from both sides of the bridge. They are used by applications built over NTB and can be used to pass control and status information between both sides of a device.}(hj]hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjYubhdefinition_list)}(hhh]hdefinition_list_item)}(hScratchpad registers has 2 parts 1) Self Scratchpad: Host's own register space 2) Peer Scratchpad: Remote host's register space. h](hterm)}(h Scratchpad registers has 2 partsh]h Scratchpad registers has 2 parts}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1jvhhhKhjrubh definition)}(hhh]jx)}(hhh](j})}(h*Self Scratchpad: Host's own register spaceh]h)}(hjh]h,Self Scratchpad: Host’s own register space}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j|hjubj})}(h/Peer Scratchpad: Remote host's register space. h]h)}(h.Peer Scratchpad: Remote host's register space.h]h0Peer Scratchpad: Remote host’s register space.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j|hjubeh}(h]h ]h"]h$]h&]jjjhjjuh1jwhjubah}(h]h ]h"]h$]h&]uh1jhjrubeh}(h]h ]h"]h$]h&]uh1jphhhKhjmubah}(h]h ]h"]h$]h&]uh1jkhjYubeh}(h]h ]h"]h$]h&]uh1jqhhhKhjHhhubeh}(h]scratchpad-registersah ]h"]scratchpad registers:ah$]h&]uh1hhj`hhhhhKubh)}(hhh](h)}(hDoorbell Registers:h]hDoorbell Registers:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubjr)}(hBDoorbell Registers are used by the hosts to interrupt each other. h]h)}(hADoorbell Registers are used by the hosts to interrupt each other.h]hADoorbell Registers are used by the hosts to interrupt each other.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jqhhhKhjhhubeh}(h]doorbell-registersah ]h"]doorbell registers:ah$]h&]uh1hhj`hhhhhKubh)}(hhh](h)}(hMemory Window:h]hMemory Window:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubjr)}(hSActual transfer of data between the two hosts will happen using the memory window. h]h)}(hRActual transfer of data between the two hosts will happen using the memory window.h]hRActual transfer of data between the two hosts will happen using the memory window.}(hj)hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj%ubah}(h]h ]h"]h$]h&]uh1jqhhhKhjhhubeh}(h] memory-windowah ]h"]memory window:ah$]h&]uh1hhj`hhhhhKubeh}(h]$constructs-used-for-implementing-ntbah ]h"]$constructs used for implementing ntbah$]h&]uh1hhhhhhhhK1ubh)}(hhh](h)}(hModeling Constructs:h]hModeling Constructs:}(hjPhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjMhhhhhKubh)}(hX8There are 5 or more distinct regions (config, self scratchpad, peer scratchpad, doorbell, one or more memory windows) to be modeled to achieve NTB functionality. At least one memory window is required while more than one is permitted. All these regions should be mapped to BARs for hosts to access these regions.h]hX8There are 5 or more distinct regions (config, self scratchpad, peer scratchpad, doorbell, one or more memory windows) to be modeled to achieve NTB functionality. At least one memory window is required while more than one is permitted. All these regions should be mapped to BARs for hosts to access these regions.}(hj^hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjMhhubh)}(hZIf one 32-bit BAR is allocated for each of these regions, the scheme would look like this:h]hZIf one 32-bit BAR is allocated for each of these regions, the scheme would look like this:}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjMhhubhtable)}(hhh]htgroup)}(hhh](hcolspec)}(hhh]h}(h]h ]h"]h$]h&]colwidthKuh1jhjubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthKuh1jhjubhthead)}(hhh]hrow)}(hhh](hentry)}(hhh]h)}(hBAR NOh]hBAR NO}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hCONSTRUCTS USEDh]hCONSTRUCTS USED}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhjubhtbody)}(hhh](j)}(hhh](j)}(hhh]h)}(hBAR0h]hBAR0}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(h Config Regionh]h Config Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR1h]hBAR1}(hj"hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hSelf Scratchpadh]hSelf Scratchpad}(hj9hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj6ubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR2h]hBAR2}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjVubah}(h]h ]h"]h$]h&]uh1jhjSubj)}(hhh]h)}(hPeer Scratchpadh]hPeer Scratchpad}(hjphhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjmubah}(h]h ]h"]h$]h&]uh1jhjSubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR3h]hBAR3}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hDoorbellh]hDoorbell}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR4h]hBAR4}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hMemory Window 1h]hMemory Window 1}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR5h]hBAR5}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hMemory Window 2h]hMemory Window 2}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]colsKuh1jhj|ubah}(h]h ]h"]h$]h&]uh1jzhjMhhhhhNubh)}(hHowever if we allocate a separate BAR for each of the regions, there would not be enough BARs for all the regions in a platform that supports only 64-bit BARs.h]hHowever if we allocate a separate BAR for each of the regions, there would not be enough BARs for all the regions in a platform that supports only 64-bit BARs.}(hjBhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjMhhubh)}(hIn order to be supported by most of the platforms, the regions should be packed and mapped to BARs in a way that provides NTB functionality and also makes sure the host doesn't access any region that it is not supposed to.h]hIn order to be supported by most of the platforms, the regions should be packed and mapped to BARs in a way that provides NTB functionality and also makes sure the host doesn’t access any region that it is not supposed to.}(hjPhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjMhhubh)}(h1The following scheme is used in EPF NTB Function:h]h1The following scheme is used in EPF NTB Function:}(hj^hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjMhhubj{)}(hhh]j)}(hhh](j)}(hhh]h}(h]h ]h"]h$]h&]colwidthKuh1jhjoubj)}(hhh]h}(h]h ]h"]h$]h&]colwidthKuh1jhjoubj)}(hhh]j)}(hhh](j)}(hhh]h)}(hBAR NOh]hBAR NO}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hCONSTRUCTS USEDh]hCONSTRUCTS USED}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1jhjoubj)}(hhh](j)}(hhh](j)}(hhh]h)}(hBAR0h]hBAR0}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hConfig Region + Self Scratchpadh]hConfig Region + Self Scratchpad}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR1h]hBAR1}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hPeer Scratchpadh]hPeer Scratchpad}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR2h]hBAR2}(hj=hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj:ubah}(h]h ]h"]h$]h&]uh1jhj7ubj)}(hhh]h)}(hDoorbell + Memory Window 1h]hDoorbell + Memory Window 1}(hjThhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjQubah}(h]h ]h"]h$]h&]uh1jhj7ubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR3h]hBAR3}(hjthhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjqubah}(h]h ]h"]h$]h&]uh1jhjnubj)}(hhh]h)}(hMemory Window 2h]hMemory Window 2}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjnubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR4h]hBAR4}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hMemory Window 3h]hMemory Window 3}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh](j)}(hhh]h)}(hBAR5h]hBAR5}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjubah}(h]h ]h"]h$]h&]uh1jhjubj)}(hhh]h)}(hMemory Window 4h]hMemory Window 4}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhjoubeh}(h]h ]h"]h$]h&]colsKuh1jhjlubah}(h]h ]h"]h$]h&]uh1jzhjMhhhhhNubh)}(hNWith this scheme, for the basic NTB functionality 3 BARs should be sufficient.h]hNWith this scheme, for the basic NTB functionality 3 BARs should be sufficient.}(hj&hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjMhhubh)}(hhh](h)}(h"Modeling Config/Scratchpad Region:h]h"Modeling Config/Scratchpad Region:}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj4hhhhhMubjL)}(hX+-----------------+------->+------------------+ +-----------------+ | BAR0 | | CONFIG REGION | | BAR0 | +-----------------+----+ +------------------+<-------+-----------------+ | BAR1 | | |SCRATCHPAD REGION | | BAR1 | +-----------------+ +-->+------------------+<-------+-----------------+ | BAR2 | Local Memory | BAR2 | +-----------------+ +-----------------+ | BAR3 | | BAR3 | +-----------------+ +-----------------+ | BAR4 | | BAR4 | +-----------------+ +-----------------+ | BAR5 | | BAR5 | +-----------------+ +-----------------+ EP CONTROLLER 1 EP CONTROLLER 2h]hX+-----------------+------->+------------------+ +-----------------+ | BAR0 | | CONFIG REGION | | BAR0 | +-----------------+----+ +------------------+<-------+-----------------+ | BAR1 | | |SCRATCHPAD REGION | | BAR1 | +-----------------+ +-->+------------------+<-------+-----------------+ | BAR2 | Local Memory | BAR2 | +-----------------+ +-----------------+ | BAR3 | | BAR3 | +-----------------+ +-----------------+ | BAR4 | | BAR4 | +-----------------+ +-----------------+ | BAR5 | | BAR5 | +-----------------+ +-----------------+ EP CONTROLLER 1 EP CONTROLLER 2}hjEsbah}(h]h ]h"]h$]h&]hhj[j\textj^}uh1jKhhhMhj4hhubh)}(hXAbove diagram shows Config region + Scratchpad region for HOST1 (connected to EP controller 1) allocated in local memory. The HOST1 can access the config region and scratchpad region (self scratchpad) using BAR0 of EP controller 1. The peer host (HOST2 connected to EP controller 2) can also access this scratchpad region (peer scratchpad) using BAR1 of EP controller 2. This diagram shows the case where Config region and Scratchpad regions are allocated for HOST1, however the same is applicable for HOST2.h]hXAbove diagram shows Config region + Scratchpad region for HOST1 (connected to EP controller 1) allocated in local memory. The HOST1 can access the config region and scratchpad region (self scratchpad) using BAR0 of EP controller 1. The peer host (HOST2 connected to EP controller 2) can also access this scratchpad region (peer scratchpad) using BAR1 of EP controller 2. This diagram shows the case where Config region and Scratchpad regions are allocated for HOST1, however the same is applicable for HOST2.}(hjUhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM#hj4hhubeh}(h]!modeling-config-scratchpad-regionah ]h"]"modeling config/scratchpad region:ah$]h&]uh1hhjMhhhhhMubh)}(hhh](h)}(h"Modeling Doorbell/Memory Window 1:h]h"Modeling Doorbell/Memory Window 1:}(hjnhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjkhhhhhM,ubjL)}(hX@ +-----------------+ +----->+----------------+-----------+-----------------+ | BAR0 | | | Doorbell 1 +-----------> MSI-X ADDRESS 1 | +-----------------+ | +----------------+ +-----------------+ | BAR1 | | | Doorbell 2 +---------+ | | +-----------------+----+ +----------------+ | | | | BAR2 | | Doorbell 3 +-------+ | +-----------------+ +-----------------+----+ +----------------+ | +-> MSI-X ADDRESS 2 | | BAR3 | | | Doorbell 4 +-----+ | +-----------------+ +-----------------+ | |----------------+ | | | | | BAR4 | | | | | | +-----------------+ +-----------------+ | | MW1 +---+ | +-->+ MSI-X ADDRESS 3|| | BAR5 | | | | | | +-----------------+ +-----------------+ +----->-----------------+ | | | | EP CONTROLLER 1 | | | | +-----------------+ | | | +---->+ MSI-X ADDRESS 4 | +----------------+ | +-----------------+ EP CONTROLLER 2 | | | (OB SPACE) | | | +-------> MW1 | | | | | +-----------------+ | | | | | | | | | | +-----------------+ PCI Address Space (Managed by HOST2)h]hX@ +-----------------+ +----->+----------------+-----------+-----------------+ | BAR0 | | | Doorbell 1 +-----------> MSI-X ADDRESS 1 | +-----------------+ | +----------------+ +-----------------+ | BAR1 | | | Doorbell 2 +---------+ | | +-----------------+----+ +----------------+ | | | | BAR2 | | Doorbell 3 +-------+ | +-----------------+ +-----------------+----+ +----------------+ | +-> MSI-X ADDRESS 2 | | BAR3 | | | Doorbell 4 +-----+ | +-----------------+ +-----------------+ | |----------------+ | | | | | BAR4 | | | | | | +-----------------+ +-----------------+ | | MW1 +---+ | +-->+ MSI-X ADDRESS 3|| | BAR5 | | | | | | +-----------------+ +-----------------+ +----->-----------------+ | | | | EP CONTROLLER 1 | | | | +-----------------+ | | | +---->+ MSI-X ADDRESS 4 | +----------------+ | +-----------------+ EP CONTROLLER 2 | | | (OB SPACE) | | | +-------> MW1 | | | | | +-----------------+ | | | | | | | | | | +-----------------+ PCI Address Space (Managed by HOST2)}hj|sbah}(h]h ]h"]h$]h&]hhj[j\textj^}uh1jKhhhM.hjkhhubh)}(hX7Above diagram shows how the doorbell and memory window 1 is mapped so that HOST1 can raise doorbell interrupt on HOST2 and also how HOST1 can access buffers exposed by HOST2 using memory window1 (MW1). Here doorbell and memory window 1 regions are allocated in EP controller 2 outbound (OB) address space. Allocating and configuring BARs for doorbell and memory window1 is done during the initialization phase of NTB endpoint function driver. Mapping from EP controller 2 OB space to PCI address space is done when HOST2 sends CMD_CONFIGURE_MW/CMD_CONFIGURE_DOORBELL.h]hX7Above diagram shows how the doorbell and memory window 1 is mapped so that HOST1 can raise doorbell interrupt on HOST2 and also how HOST1 can access buffers exposed by HOST2 using memory window1 (MW1). Here doorbell and memory window 1 regions are allocated in EP controller 2 outbound (OB) address space. Allocating and configuring BARs for doorbell and memory window1 is done during the initialization phase of NTB endpoint function driver. Mapping from EP controller 2 OB space to PCI address space is done when HOST2 sends CMD_CONFIGURE_MW/CMD_CONFIGURE_DOORBELL.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMOhjkhhubeh}(h]!modeling-doorbell-memory-window-1ah ]h"]"modeling doorbell/memory window 1:ah$]h&]uh1hhjMhhhhhM,ubh)}(hhh](h)}(h!Modeling Optional Memory Windows:h]h!Modeling Optional Memory Windows:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMYubh)}(hiThis is modeled the same was as MW1 but each of the additional memory windows is mapped to separate BARs.h]hiThis is modeled the same was as MW1 but each of the additional memory windows is mapped to separate BARs.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM[hjhhubeh}(h] modeling-optional-memory-windowsah ]h"]!modeling optional memory windows:ah$]h&]uh1hhjMhhhhhMYubeh}(h]modeling-constructsah ]h"]modeling constructs:ah$]h&]uh1hhhhhhhhKubeh}(h]pci-ntb-functionah ]h"]pci ntb functionah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksjfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerjerror_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourceh _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}nameids}(jjjJjGjEjBjjjjjBj?jjjhjejjjju nametypes}(jjJjEjjjBjjhjjuh}(jhjGj`jBjjjHjjj?jjjMjej4jjkjju footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}Rparse_messages]transform_messages] transformerN include_log] decorationNhhub.