hsphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget3/translations/zh_CN/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/zh_TW/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/it_IT/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/ja_JP/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/ko_KR/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hPortuguese (Brazilian)}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/pt_BR/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/sp_SP/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhM/var/lib/git/docbuild/linux/Documentation/driver-api/cxl/linux/cxl-driver.rsthKubhsection)}(hhh](htitle)}(hCXL Driver Operationh]hCXL Driver Operation}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh paragraph)}(h7The devices described in this section are present in ::h]h4The devices described in this section are present in}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh literal_block)}(h/sys/bus/cxl/devices/ /dev/cxl/h]h/sys/bus/cxl/devices/ /dev/cxl/}hhsbah}(h]h ]h"]h$]h&]hhuh1hhhhK hhhhubh)}(h|The :code:`cxl-cli` library, maintained as part of the NDTCL project, may be used to script interactions with these devices.h](hThe }(hhhhhNhNubhliteral)}(h:code:`cxl-cli`h]hcxl-cli}(hjhhhNhNubah}(h]h ]codeah"]h$]h&]languagehuh1jhhubhi library, maintained as part of the NDTCL project, may be used to script interactions with these devices.}(hhhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hhh](h)}(hDriversh]hDrivers}(hj$hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj!hhhhhKubh)}(h1The CXL driver is split into a number of drivers.h]h1The CXL driver is split into a number of drivers.}(hj2hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj!hhubh bullet_list)}(hhh](h list_item)}(h?cxl_core - fundamental init interface and core object creationh]h)}(hjIh]h?cxl_core - fundamental init interface and core object creation}(hjKhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjGubah}(h]h ]h"]h$]h&]uh1jEhjBhhhhhNubjF)}(hEcxl_port - initializes root and provides port enumeration interface.h]h)}(hj`h]hEcxl_port - initializes root and provides port enumeration interface.}(hjbhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj^ubah}(h]h ]h"]h$]h&]uh1jEhjBhhhhhNubjF)}(hCcxl_acpi - initializes root decoders and interacts with ACPI data.h]h)}(hjwh]hCcxl_acpi - initializes root decoders and interacts with ACPI data.}(hjyhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjuubah}(h]h ]h"]h$]h&]uh1jEhjBhhhhhNubjF)}(h&cxl_p/mem - initializes memory devicesh]h)}(hjh]h&cxl_p/mem - initializes memory devices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jEhjBhhhhhNubjF)}(hDcxl_pci - uses cxl_port to enumerate the actual fabric hierarchy. h]h)}(hCcxl_pci - uses cxl_port to enumerate the actual fabric hierarchy.h]hCcxl_pci - uses cxl_port to enumerate the actual fabric hierarchy.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jEhjBhhhhhNubeh}(h]h ]h"]h$]h&]bullet*uh1j@hhhKhj!hhubeh}(h]driversah ]h"]driversah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hDriver Devicesh]hDriver Devices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hHere is an example from a single-socket system with 4 host bridges. Two host bridges have a single memory device attached, and the devices are interleaved into a single memory region. The memory region has been converted to dax. ::h]hHere is an example from a single-socket system with 4 host bridges. Two host bridges have a single memory device attached, and the devices are interleaved into a single memory region. The memory region has been converted to dax.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h# ls /sys/bus/cxl/devices/ dax_region0 decoder3.0 decoder6.0 mem0 port3 decoder0.0 decoder4.0 decoder6.1 mem1 port4 decoder1.0 decoder5.0 endpoint5 port1 region0 decoder2.0 decoder5.1 endpoint6 port2 root0h]h# ls /sys/bus/cxl/devices/ dax_region0 decoder3.0 decoder6.0 mem0 port3 decoder0.0 decoder4.0 decoder6.1 mem1 port4 decoder1.0 decoder5.0 endpoint5 port1 region0 decoder2.0 decoder5.1 endpoint6 port2 root0}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKhjhhubhfigure)}(hhh](kfigure kernel_render)}(hhh]h)}(hXdigraph foo { "root0" -> "port1"; "root0" -> "port3"; "root0" -> "decoder0.0"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; "port1" -> "decoder1.0"; "port3" -> "decoder3.0"; "endpoint5" -> "decoder5.0"; "endpoint6" -> "decoder6.0"; "decoder0.0" -> "region0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; "decoder5.0" -> "region0"; "decoder6.0" -> "region0"; "region0" -> "dax_region0"; "dax_region0" -> "dax0.0"; }h]hXdigraph foo { "root0" -> "port1"; "root0" -> "port3"; "root0" -> "decoder0.0"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; "port1" -> "decoder1.0"; "port3" -> "decoder3.0"; "endpoint5" -> "decoder5.0"; "endpoint6" -> "decoder6.0"; "decoder0.0" -> "region0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; "decoder5.0" -> "region0"; "decoder6.0" -> "region0"; "region0" -> "dax_region0"; "dax_region0" -> "dax0.0"; }}hjsbah}(h]h ]h"]h$]h&]hhuh1hhjhhubah}(h]h ]h"]h$]h&]alt9Digraph of CXL fabric describing host-bridge interleavingsrclangDOTuh1jhjubhcaption)}(hBDiagraph of CXL fabric with a host-bridge interleave memory regionh]hBDiagraph of CXL fabric with a host-bridge interleave memory region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubeh}(h]id1ah ]h"]h$]h&]altjcaptionjuh1jhjhhhhhNubh)}(hFor this section we'll explore the devices present in this configuration, but we'll explore more configurations in-depth in example configurations below.h]hFor this section we’ll explore the devices present in this configuration, but we’ll explore more configurations in-depth in example configurations below.}(hj4hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK?hjhhubh)}(hhh](h)}(h Base Devicesh]h Base Devices}(hjEhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjBhhhhhKCubh)}(hMost devices in a CXL fabric are a `port` of some kind (because each device mostly routes request from one device to the next, rather than provide a direct service).h](h#Most devices in a CXL fabric are a }(hjShhhNhNubhtitle_reference)}(h`port`h]hport}(hj]hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjSubh| of some kind (because each device mostly routes request from one device to the next, rather than provide a direct service).}(hjShhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKDhjBhhubh)}(hhh](h)}(hRooth]hRoot}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjuhhhhhKIubh)}(hThe `CXL Root` is logical object created by the `cxl_acpi` driver during :code:`cxl_acpi_probe` - if the :code:`ACPI0017` `Compute Express Link Root Object` Device Class is found.h](hThe }(hjhhhNhNubj\)}(h `CXL Root`h]hCXL Root}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh" is logical object created by the }(hjhhhNhNubj\)}(h `cxl_acpi`h]hcxl_acpi}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh driver during }(hjhhhNhNubj)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh - if the }(hjhhhNhNubj)}(h:code:`ACPI0017`h]hACPI0017}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh }(hjhhhNhNubj\)}(h"`Compute Express Link Root Object`h]h Compute Express Link Root Object}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh Device Class is found.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKJhjuhhubh)}(hThe Root contains links to:h]hThe Root contains links to:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKNhjuhhubjA)}(hhh](jF)}(hN`Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>` h]h)}(hM`Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>`h](j\)}(h`Host Bridge Ports`h]hHost Bridge Ports}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh defined by CHBS in the }(hjhhhNhNubh)}(h":doc:`CEDT<../platform/acpi/cedt>`h]hinline)}(hjh]hCEDT}(hj!hhhNhNubah}(h]h ](xrefstdstd-doceh"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]refdocdriver-api/cxl/linux/cxl-driver refdomainj,reftypedoc refexplicitrefwarn reftarget../platform/acpi/cedtuh1hhhhKPhjubeh}(h]h ]h"]h$]h&]uh1hhhhKPhjubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(h?`Downstream Ports` typically connected to `Host Bridge Ports`. h]h)}(h>`Downstream Ports` typically connected to `Host Bridge Ports`.h](j\)}(h`Downstream Ports`h]hDownstream Ports}(hjThhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjPubh typically connected to }(hjPhhhNhNubj\)}(h`Host Bridge Ports`h]hHost Bridge Ports}(hjfhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjPubh.}(hjPhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKRhjLubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(hH`Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>` h]h)}(hG`Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>`h](j\)}(h`Root Decoders`h]h Root Decoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh defined by CFMWS the }(hjhhhNhNubh)}(h":doc:`CEDT<../platform/acpi/cedt>`h]j )}(hjh]hCEDT}(hjhhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]refdocj8 refdomainjreftypedoc refexplicitrefwarnj>../platform/acpi/cedtuh1hhhhKThjubeh}(h]h ]h"]h$]h&]uh1hhhhKThjubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j@hhhKPhjuhhubh)}(hXE# ls /sys/bus/cxl/devices/root0 decoder0.0 dport0 dport5 port2 subsystem decoders_committed dport1 modalias port3 uevent devtype dport4 port1 port4 uport # cat /sys/bus/cxl/devices/root0/devtype cxl_port # cat port1/devtype cxl_port # cat decoder0.0/devtype cxl_decoder_rooth]hXE# ls /sys/bus/cxl/devices/root0 decoder0.0 dport0 dport5 port2 subsystem decoders_committed dport1 modalias port3 uevent devtype dport4 port1 port4 uport # cat /sys/bus/cxl/devices/root0/devtype cxl_port # cat port1/devtype cxl_port # cat decoder0.0/devtype cxl_decoder_root}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKXhjuhhubh)}(hThe root is first `logical port` in the CXL fabric, as presented by the Linux CXL driver. The `CXL root` is a special type of `switch port`, in that it only has downstream port connections.h](hThe root is first }(hjhhhNhNubj\)}(h`logical port`h]h logical port}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh? in the CXL fabric, as presented by the Linux CXL driver. The }(hjhhhNhNubj\)}(h `CXL root`h]hCXL root}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh is a special type of }(hjhhhNhNubj\)}(h `switch port`h]h switch port}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh2, in that it only has downstream port connections.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKfhjuhhubeh}(h]rootah ]h"]rootah$]h&]uh1hhjBhhhhhKIubh)}(hhh](h)}(hPorth]hPort}(hj-hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj*hhhhhKkubh)}(hX.A `port` object is better described as a `switch port`. It may represent a host bridge to the root or an actual switch port on a switch. A `switch port` contains one or more decoders used to route memory requests downstream ports, which may be connected to another `switch port` or an `endpoint port`.h](hA }(hj;hhhNhNubj\)}(h`port`h]hport}(hjChhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj;ubh! object is better described as a }(hj;hhhNhNubj\)}(h `switch port`h]h switch port}(hjUhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj;ubhV. It may represent a host bridge to the root or an actual switch port on a switch. A }(hj;hhhNhNubj\)}(h `switch port`h]h switch port}(hjghhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj;ubhq contains one or more decoders used to route memory requests downstream ports, which may be connected to another }(hj;hhhNhNubj\)}(h `switch port`h]h switch port}(hjyhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj;ubh or an }(hj;hhhNhNubj\)}(h`endpoint port`h]h endpoint port}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj;ubh.}(hj;hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKlhj*hhubh)}(hX2# ls /sys/bus/cxl/devices/port1 decoder1.0 dport0 driver parent_dport uport decoders_committed dport113 endpoint5 subsystem devtype dport2 modalias uevent # cat devtype cxl_port # cat decoder1.0/devtype cxl_decoder_switch # cat endpoint5/devtype cxl_porth]hX2# ls /sys/bus/cxl/devices/port1 decoder1.0 dport0 driver parent_dport uport decoders_committed dport113 endpoint5 subsystem devtype dport2 modalias uevent # cat devtype cxl_port # cat decoder1.0/devtype cxl_decoder_switch # cat endpoint5/devtype cxl_port}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKshj*hhubh)}(hCXL `Host Bridges` in the fabric are probed during :code:`cxl_acpi_probe` at the time the `CXL Root` is probed. The allows for the immediate logical connection to between the root and host bridge.h](hCXL }(hjhhhNhNubj\)}(h`Host Bridges`h]h Host Bridges}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh! in the fabric are probed during }(hjhhhNhNubj)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh at the time the }(hjhhhNhNubj\)}(h `CXL Root`h]hCXL Root}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubha is probed. The allows for the immediate logical connection to between the root and host bridge.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj*hhubjA)}(hhh](jF)}(h;The root has a downstream port connection to a host bridge h]h)}(h:The root has a downstream port connection to a host bridgeh]h:The root has a downstream port connection to a host bridge}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(h=The host bridge has an upstream port connection to the root. h]h)}(h "port1"; "root0" -> "port2"; "root0" -> "port3"; "root0" -> "port4"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; }h]hdigraph foo { "root0" -> "port1"; "root0" -> "port2"; "root0" -> "port3"; "root0" -> "port4"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; }}hjrsbah}(h]h ]h"]h$]h&]hhuh1hhjohhubah}(h]h ]h"]h$]h&]j9Digraph of CXL fabric describing host-bridge interleavingjDOTuh1jhjlubj)}(hBDiagraph of CXL fabric with a host-bridge interleave memory regionh]hBDiagraph of CXL fabric with a host-bridge interleave memory region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjlubeh}(h]id2ah ]h"]h$]h&]altjcaptionjuh1jhjMhhhhhNubeh}(h]port-relationshipsah ]h"]port relationshipsah$]h&]uh1hhjBhhhhhKubeh}(h] base-devicesah ]h"] base devicesah$]h&]uh1hhjhhhhhKCubh)}(hhh](h)}(hDecodersh]hDecoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hA `Decoder` is short for a CXL Host-Managed Device Memory (HDM) Decoder. It is a device that routes accesses through the CXL fabric to an endpoint, and at the endpoint translates a `Host Physical` to `Device Physical` Addressing.h](hA }(hjhhhNhNubj\)}(h `Decoder`h]hDecoder}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh is short for a CXL Host-Managed Device Memory (HDM) Decoder. It is a device that routes accesses through the CXL fabric to an endpoint, and at the endpoint translates a }(hjhhhNhNubj\)}(h`Host Physical`h]h Host Physical}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh to }(hjhhhNhNubj\)}(h`Device Physical`h]hDevice Physical}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh Addressing.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hThe CXL 3.1 specification heavily implies that only endpoint decoders should engage in translation of `Host Physical Address` to `Device Physical Address`. ::h](hfThe CXL 3.1 specification heavily implies that only endpoint decoders should engage in translation of }(hjhhhNhNubj\)}(h`Host Physical Address`h]hHost Physical Address}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh to }(hjhhhNhNubj\)}(h`Device Physical Address`h]hDevice Physical Address}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h8.2.4.20 CXL HDM Decoder Capability Structure IMPLEMENTATION NOTE CXL Host Bridge and Upstream Switch Port Decode Flow IMPLEMENTATION NOTE Device Decode Logich]h8.2.4.20 CXL HDM Decoder Capability Structure IMPLEMENTATION NOTE CXL Host Bridge and Upstream Switch Port Decode Flow IMPLEMENTATION NOTE Device Decode Logic}hj6sbah}(h]h ]h"]h$]h&]hhuh1hhhhKhjhhubh)}(h@These notes imply that there are two logical groups of decoders.h]h@These notes imply that there are two logical groups of decoders.}(hjDhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjA)}(hhh](jF)}(hdRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA. h]h)}(hcRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA.h]hcRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA.}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjUubah}(h]h ]h"]h$]h&]uh1jEhjRhhhhhNubjF)}(hfTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service. h]h)}(heTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service.h]heTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service.}(hjqhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjmubah}(h]h ]h"]h$]h&]uh1jEhjRhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j@hhhKhjhhubh)}(hThe CXL drivers distinguish 3 decoder types: root, switch, and endpoint. Only endpoint decoders are Translating Decoders, all others are Routing Decoders.h]hThe CXL drivers distinguish 3 decoder types: root, switch, and endpoint. Only endpoint decoders are Translating Decoders, all others are Routing Decoders.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubhnote)}(hXPLATFORM VENDORS BE AWARE Linux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric. It is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.h](h)}(hPLATFORM VENDORS BE AWAREh]hPLATFORM VENDORS BE AWARE}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hLinux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric.h]hLinux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hXIt is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.h]hXIt is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubeh}(h]h ]h"]h$]h&]uh1jhjhhhhhNubh)}(hDecoders may have one or more `Downstream Targets` if configured to interleave memory accesses. This will be presented in sysfs via the :code:`target_list` parameter.h](hDecoders may have one or more }(hjhhhNhNubj\)}(h`Downstream Targets`h]hDownstream Targets}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubhW if configured to interleave memory accesses. This will be presented in sysfs via the }(hjhhhNhNubj)}(h:code:`target_list`h]h target_list}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh parameter.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hhh](h)}(h Root Decoderh]h Root Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhKubh)}(hXA `Root Decoder` is logical construct of the physical address and interleave configurations present in the CFMWS field of the :doc:`CEDT <../platform/acpi/cedt>`. Linux presents this information as a decoder present in the `CXL Root`. We consider this a `Root Decoder`, though technically it exists on the boundary of the CXL specification and platform-specific CXL root implementations.h](hA }(hj hhhNhNubj\)}(h`Root Decoder`h]h Root Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubhn is logical construct of the physical address and interleave configurations present in the CFMWS field of the }(hj hhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hj/ h]hCEDT}(hj1 hhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhj- ubah}(h]h ]h"]h$]h&]refdocj8 refdomainj; reftypedoc refexplicitrefwarnj>../platform/acpi/cedtuh1hhhhKhj ubh>. Linux presents this information as a decoder present in the }(hj hhhNhNubj\)}(h `CXL Root`h]hCXL Root}(hjQ hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh. We consider this a }(hj hhhNhNubj\)}(h`Root Decoder`h]h Root Decoder}(hjc hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubhw, though technically it exists on the boundary of the CXL specification and platform-specific CXL root implementations.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj hhubh)}(hLinux considers these logical decoders a type of `Routing Decoder`, and is the first decoder in the CXL fabric to receive a memory access from the platform's memory controllers.h](h1Linux considers these logical decoders a type of }(hj{ hhhNhNubj\)}(h`Routing Decoder`h]hRouting Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj{ ubhq, and is the first decoder in the CXL fabric to receive a memory access from the platform’s memory controllers.}(hj{ hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj hhubh)}(h`Root Decoders` are created during :code:`cxl_acpi_probe`. One root decoder is created per CFMWS entry in the :doc:`CEDT <../platform/acpi/cedt>`.h](j\)}(h`Root Decoders`h]h Root Decoders}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh are created during }(hj hhhNhNubj)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh6. One root decoder is created per CFMWS entry in the }(hj hhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hj h]hCEDT}(hj hhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhj ubah}(h]h ]h"]h$]h&]refdocj8 refdomainj reftypedoc refexplicitrefwarnj>../platform/acpi/cedtuh1hhhhMhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hThe :code:`target_list` parameter is filled by the CFMWS target fields. Targets of a root decoder are `Host Bridges`, which means interleave done at the root decoder level is an `Inter-Host-Bridge Interleave`.h](hThe }(hj hhhNhNubj)}(h:code:`target_list`h]h target_list}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubhO parameter is filled by the CFMWS target fields. Targets of a root decoder are }(hj hhhNhNubj\)}(h`Host Bridges`h]h Host Bridges}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh>, which means interleave done at the root decoder level is an }(hj hhhNhNubj\)}(h`Inter-Host-Bridge Interleave`h]hInter-Host-Bridge Interleave}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hAOnly root decoders are capable of `Inter-Host-Bridge Interleave`.h](h"Only root decoders are capable of }(hj3 hhhNhNubj\)}(h`Inter-Host-Bridge Interleave`h]hInter-Host-Bridge Interleave}(hj; hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj3 ubh.}(hj3 hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM hj hhubh)}(hXDSuch interleaves must be configured by the platform and described in the ACPI CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL host bridge UIDs in the CHBS field of the :doc:`CEDT <../platform/acpi/cedt>` and the UID field of CXL Host Bridges defined in the :doc:`DSDT <../platform/acpi/dsdt>`.h](hSuch interleaves must be configured by the platform and described in the ACPI CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL host bridge UIDs in the CHBS field of the }(hjS hhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hj] h]hCEDT}(hj_ hhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhj[ ubah}(h]h ]h"]h$]h&]refdocj8 refdomainji reftypedoc refexplicitrefwarnj>../platform/acpi/cedtuh1hhhhM hjS ubh6 and the UID field of CXL Host Bridges defined in the }(hjS hhhNhNubh)}(h#:doc:`DSDT <../platform/acpi/dsdt>`h]j )}(hj h]hDSDT}(hj hhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhj ubah}(h]h ]h"]h$]h&]refdocj8 refdomainj reftypedoc refexplicitrefwarnj>../platform/acpi/dsdtuh1hhhhM hjS ubh.}(hjS hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM hj hhubh)}(hInterleave settings in a root decoder describe how to interleave accesses among the *immediate downstream targets*, not the entire interleave set.h](hTInterleave settings in a root decoder describe how to interleave accesses among the }(hj hhhNhNubhemphasis)}(h*immediate downstream targets*h]himmediate downstream targets}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh , not the entire interleave set.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(h9The memory range described in the root decoder is used toh]h9The memory range described in the root decoder is used to}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubhenumerated_list)}(hhh](jF)}(h>Create a memory region (:code:`region0` in this example), and h]h)}(h=Create a memory region (:code:`region0` in this example), andh](hCreate a memory region (}(hj hhhNhNubj)}(h:code:`region0`h]hregion0}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh in this example), and}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jEhj hhhhhNubjF)}(hLAssociate the region with an IO Memory Resource (:code:`kernel/resource.c`) h]h)}(hKAssociate the region with an IO Memory Resource (:code:`kernel/resource.c`)h](h1Associate the region with an IO Memory Resource (}(hj hhhNhNubj)}(h:code:`kernel/resource.c`h]hkernel/resource.c}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh)}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jEhj hhhhhNubeh}(h]h ]h"]h$]h&]enumtypearabicprefixhsuffix)uh1j hj hhhhhMubh)}(hX# ls /sys/bus/cxl/devices/decoder0.0/ cap_pmem devtype region0 cap_ram interleave_granularity size cap_type2 interleave_ways start cap_type3 locked subsystem create_ram_region modalias target_list delete_region qos_class uevent # cat /sys/bus/cxl/devices/decoder0.0/region0/resource 0xc050000000h]hX# ls /sys/bus/cxl/devices/decoder0.0/ cap_pmem devtype region0 cap_ram interleave_granularity size cap_type2 interleave_ways start cap_type3 locked subsystem create_ram_region modalias target_list delete_region qos_class uevent # cat /sys/bus/cxl/devices/decoder0.0/region0/resource 0xc050000000}hj? sbah}(h]h ]h"]h$]h&]hhuh1hhhhMhj hhubh)}(hThe IO Memory Resource is created during early boot when the CFMWS region is identified in the EFI Memory Map or E820 table (on x86).h]hThe IO Memory Resource is created during early boot when the CFMWS region is identified in the EFI Memory Map or E820 table (on x86).}(hjM hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM(hj hhubh)}(h}Root decoders are defined as a separate devtype, but are also a type of `Switch Decoder` due to having downstream targets. ::h](hHRoot decoders are defined as a separate devtype, but are also a type of }(hj[ hhhNhNubj\)}(h`Switch Decoder`h]hSwitch Decoder}(hjc hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj[ ubh" due to having downstream targets.}(hj[ hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM+hj hhubh)}(h@# cat /sys/bus/cxl/devices/decoder0.0/devtype cxl_decoder_rooth]h@# cat /sys/bus/cxl/devices/decoder0.0/devtype cxl_decoder_root}hj{ sbah}(h]h ]h"]h$]h&]hhuh1hhhhM.hj hhubeh}(h] root-decoderah ]h"] root decoderah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hSwitch Decoderh]hSwitch Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhM2ubh)}(hAny non-root, translating decoder is considered a `Switch Decoder`, and will present with the type :code:`cxl_decoder_switch`. Both `Host Bridge` and `CXL Switch` (device) decoders are of type :code:`cxl_decoder_switch`. ::h](h2Any non-root, translating decoder is considered a }(hj hhhNhNubj\)}(h`Switch Decoder`h]hSwitch Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh!, and will present with the type }(hj hhhNhNubj)}(h:code:`cxl_decoder_switch`h]hcxl_decoder_switch}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh. Both }(hj hhhNhNubj\)}(h `Host Bridge`h]h Host Bridge}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh and }(hj hhhNhNubj\)}(h `CXL Switch`h]h CXL Switch}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh (device) decoders are of type }(hj hhhNhNubj)}(h:code:`cxl_decoder_switch`h]hcxl_decoder_switch}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM3hj hhubh)}(hXM# ls /sys/bus/cxl/devices/decoder1.0/ devtype locked size target_list interleave_granularity modalias start target_type interleave_ways region subsystem uevent # cat /sys/bus/cxl/devices/decoder1.0/devtype cxl_decoder_switch # cat /sys/bus/cxl/devices/decoder1.0/region region0h]hXM# ls /sys/bus/cxl/devices/decoder1.0/ devtype locked size target_list interleave_granularity modalias start target_type interleave_ways region subsystem uevent # cat /sys/bus/cxl/devices/decoder1.0/devtype cxl_decoder_switch # cat /sys/bus/cxl/devices/decoder1.0/region region0}hj sbah}(h]h ]h"]h$]h&]hhuh1hhhhM7hj hhubh)}(hA `Switch Decoder` has associations between a region defined by a root decoder and downstream target ports. Interleaving done within a switch decoder is a multi-downstream-port interleave (or `Intra-Host-Bridge Interleave` for host bridges).h](hA }(hj hhhNhNubj\)}(h`Switch Decoder`h]hSwitch Decoder}(hj" hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh has associations between a region defined by a root decoder and downstream target ports. Interleaving done within a switch decoder is a multi-downstream-port interleave (or }(hj hhhNhNubj\)}(h`Intra-Host-Bridge Interleave`h]hIntra-Host-Bridge Interleave}(hj4 hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh for host bridges).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMBhj hhubh)}(hInterleave settings in a switch decoder describe how to interleave accesses among the *immediate downstream targets*, not the entire interleave set.h](hVInterleave settings in a switch decoder describe how to interleave accesses among the }(hjL hhhNhNubj )}(h*immediate downstream targets*h]himmediate downstream targets}(hjT hhhNhNubah}(h]h ]h"]h$]h&]uh1j hjL ubh , not the entire interleave set.}(hjL hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMGhj hhubh)}(hSwitch decoders are created during :code:`cxl_switch_port_probe` in the :code:`cxl_port` driver, and is created based on a PCI device's DVSEC registers.h](h#Switch decoders are created during }(hjl hhhNhNubj)}(h:code:`cxl_switch_port_probe`h]hcxl_switch_port_probe}(hjt hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjl ubh in the }(hjl hhhNhNubj)}(h:code:`cxl_port`h]hcxl_port}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjl ubhB driver, and is created based on a PCI device’s DVSEC registers.}(hjl hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMJhj hhubh)}(hSwitch decoder programming is validated during probe if the platform programs them during boot (See `Auto Decoders` below), or on commit if programmed at runtime (See `Runtime Programming` below).h](hdSwitch decoder programming is validated during probe if the platform programs them during boot (See }(hj hhhNhNubj\)}(h`Auto Decoders`h]h Auto Decoders}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh4 below), or on commit if programmed at runtime (See }(hj hhhNhNubj\)}(h`Runtime Programming`h]hRuntime Programming}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh below).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMNhj hhubeh}(h]switch-decoderah ]h"]switch decoderah$]h&]uh1hhjhhhhhM2ubh)}(hhh](h)}(hEndpoint Decoderh]hEndpoint Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMTubh)}(hAny decoder attached to a *terminal* point in the CXL fabric (`An Endpoint`) is considered an `Endpoint Decoder`. Endpoint decoders are of type :code:`cxl_decoder_endpoint`. ::h](hAny decoder attached to a }(hj hhhNhNubj )}(h *terminal*h]hterminal}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh point in the CXL fabric (}(hj hhhNhNubj\)}(h `An Endpoint`h]h An Endpoint}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh) is considered an }(hj hhhNhNubj\)}(h`Endpoint Decoder`h]hEndpoint Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh . Endpoint decoders are of type }(hj hhhNhNubj)}(h:code:`cxl_decoder_endpoint`h]hcxl_decoder_endpoint}(hj) hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMUhj hhubh)}(hXt# ls /sys/bus/cxl/devices/decoder5.0 devtype locked start dpa_resource modalias subsystem dpa_size mode target_type interleave_granularity region uevent interleave_ways size # cat /sys/bus/cxl/devices/decoder5.0/devtype cxl_decoder_endpoint # cat /sys/bus/cxl/devices/decoder5.0/region region0h]hXt# ls /sys/bus/cxl/devices/decoder5.0 devtype locked start dpa_resource modalias subsystem dpa_size mode target_type interleave_granularity region uevent interleave_ways size # cat /sys/bus/cxl/devices/decoder5.0/devtype cxl_decoder_endpoint # cat /sys/bus/cxl/devices/decoder5.0/region region0}hjB sbah}(h]h ]h"]h$]h&]hhuh1hhhhMYhj hhubh)}(hAn `Endpoint Decoder` has an association with a region defined by a root decoder and describes the device-local resource associated with this region.h](hAn }(hjP hhhNhNubj\)}(h`Endpoint Decoder`h]hEndpoint Decoder}(hjX hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjP ubh has an association with a region defined by a root decoder and describes the device-local resource associated with this region.}(hjP hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMfhj hhubh)}(hUnlike root and switch decoders, endpoint decoders translate `Host Physical` to `Device Physical` address ranges. The interleave settings on an endpoint therefore describe the entire *interleave set*.h](h=Unlike root and switch decoders, endpoint decoders translate }(hjp hhhNhNubj\)}(h`Host Physical`h]h Host Physical}(hjx hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjp ubh to }(hjp hhhNhNubj\)}(h`Device Physical`h]hDevice Physical}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjp ubhW address ranges. The interleave settings on an endpoint therefore describe the entire }(hjp hhhNhNubj )}(h*interleave set*h]hinterleave set}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hjp ubh.}(hjp hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMihj hhubh)}(h`Device Physical Address` regions must be committed in-order. For example, the DPA region starting at 0x80000000 cannot be committed before the DPA region starting at 0x0.h](j\)}(h`Device Physical Address`h]hDevice Physical Address}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh regions must be committed in-order. For example, the DPA region starting at 0x80000000 cannot be committed before the DPA region starting at 0x0.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMmhj hhubh)}(hAs of Linux v6.15, Linux does not support *imbalanced* interleave setups, all endpoints in an interleave set are expected to have the same interleave settings (granularity and ways must be the same).h](h*As of Linux v6.15, Linux does not support }(hj hhhNhNubj )}(h *imbalanced*h]h imbalanced}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh interleave setups, all endpoints in an interleave set are expected to have the same interleave settings (granularity and ways must be the same).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMqhj hhubh)}(hEndpoint decoders are created during :code:`cxl_endpoint_port_probe` in the :code:`cxl_port` driver, and is created based on a PCI device's DVSEC registers.h](h%Endpoint decoders are created during }(hj hhhNhNubj)}(h:code:`cxl_endpoint_port_probe`h]hcxl_endpoint_port_probe}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh in the }(hj hhhNhNubj)}(h:code:`cxl_port`h]hcxl_port}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubhB driver, and is created based on a PCI device’s DVSEC registers.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMuhj hhubeh}(h]endpoint-decoderah ]h"]endpoint decoderah$]h&]uh1hhjhhhhhMTubh)}(hhh](h)}(hDecoder Relationshipsh]hDecoder Relationships}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj,hhhhhMyubh)}(hXIn our example described above, there is one root decoder which routes memory accesses over two host bridges. Each host bridge has a decoder which routes access to their singular endpoint targets. Each endpoint has a decoder which translates HPA to DPA and services the memory request.h]hXIn our example described above, there is one root decoder which routes memory accesses over two host bridges. Each host bridge has a decoder which routes access to their singular endpoint targets. Each endpoint has a decoder which translates HPA to DPA and services the memory request.}(hj=hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMzhj,hhubh)}(hThe driver validates relationships between ports by decoder programming, so we can think of decoders being related in a similarly hierarchical fashion to ports.h]hThe driver validates relationships between ports by decoder programming, so we can think of decoders being related in a similarly hierarchical fashion to ports.}(hjKhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj,hhubj)}(hhh](j)}(hhh]h)}(hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; }h]hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; }}hj_sbah}(h]h ]h"]h$]h&]hhuh1hhj\hhubah}(h]h ]h"]h$]h&]jQDigraph of hierarchical relationship between root, switch, and endpoint decoders.jDOTuh1jhjYubj)}(h4Diagraph of CXL root, switch, and endpoint decoders.h]h4Diagraph of CXL root, switch, and endpoint decoders.}(hjuhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjYubeh}(h]id3ah ]h"]h$]h&]altjscaptionjwuh1jhj,hhhhhNubeh}(h]decoder-relationshipsah ]h"]decoder relationshipsah$]h&]uh1hhjhhhhhMyubeh}(h]decodersah ]h"]decodersah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hRegionsh]hRegions}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(h Memory Regionh]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hA `Memory Region` is a logical construct that connects a set of CXL ports in the fabric to an IO Memory Resource. It is ultimately used to expose the memory on these devices to the DAX subsystem via a `DAX Region`.h](hA }(hjhhhNhNubj\)}(h`Memory Region`h]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh is a logical construct that connects a set of CXL ports in the fabric to an IO Memory Resource. It is ultimately used to expose the memory on these devices to the DAX subsystem via a }(hjhhhNhNubj\)}(h `DAX Region`h]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hAn example RAM region: ::h]hAn example RAM region:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hX # ls /sys/bus/cxl/devices/region0/ access0 devtype modalias subsystem uuid access1 driver mode target0 commit interleave_granularity resource target1 dax_region0 interleave_ways size ueventh]hX # ls /sys/bus/cxl/devices/region0/ access0 devtype modalias subsystem uuid access1 driver mode target0 commit interleave_granularity resource target1 dax_region0 interleave_ways size uevent}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubh)}(hA memory region can be constructed during endpoint probe, if decoders were programmed by BIOS/EFI (see `Auto Decoders`), or by creating a region manually via a `Root Decoder`'s :code:`create_ram_region` or :code:`create_pmem_region` interfaces.h](hgA memory region can be constructed during endpoint probe, if decoders were programmed by BIOS/EFI (see }(hj hhhNhNubj\)}(h`Auto Decoders`h]h Auto Decoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh*), or by creating a region manually via a }(hj hhhNhNubj\)}(h`Root Decoder`h]h Root Decoder}(hj&hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hj ubh’s }(hj hhhNhNubj)}(h:code:`create_ram_region`h]hcreate_ram_region}(hj8hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh or }(hj hhhNhNubj)}(h:code:`create_pmem_region`h]hcreate_pmem_region}(hjKhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh interfaces.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe interleave settings in a `Memory Region` describe the configuration of the `Interleave Set` - and are what can be expected to be seen in the endpoint interleave settings.h](hThe interleave settings in a }(hjdhhhNhNubj\)}(h`Memory Region`h]h Memory Region}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjdubh# describe the configuration of the }(hjdhhhNhNubj\)}(h`Interleave Set`h]hInterleave Set}(hj~hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjdubhO - and are what can be expected to be seen in the endpoint interleave settings.}(hjdhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj)}(hhh](j)}(hhh]h)}(hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "region0"; "region0" -> "decoder5.0"; "region0" -> "decoder6.0"; }h]hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "region0"; "region0" -> "decoder5.0"; "region0" -> "decoder6.0"; }}hjsbah}(h]h ]h"]h$]h&]hhuh1hhjhhubah}(h]h ]h"]h$]h&]jNDigraph of CXL memory region relationships between root and endpoint decoders.jDOTuh1jhjubj)}(hRegions are created based on root decoder configurations. Endpoint decoders must be programmed with the same interleave settings as the region.h]hRegions are created based on root decoder configurations. Endpoint decoders must be programmed with the same interleave settings as the region.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubeh}(h]id4ah ]h"]h$]h&]altjcaptionjuh1jhjhhhhhNubeh}(h] memory-regionah ]h"] memory regionah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(h DAX Regionh]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hA `DAX Region` is used to convert a CXL `Memory Region` to a DAX device. A DAX device may then be accessed directly via a file descriptor interface, or converted to System RAM via the DAX kmem driver. See the DAX driver section for more details. ::h](hA }(hjhhhNhNubj\)}(h `DAX Region`h]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh is used to convert a CXL }(hjhhhNhNubj\)}(h`Memory Region`h]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh to a DAX device. A DAX device may then be accessed directly via a file descriptor interface, or converted to System RAM via the DAX kmem driver. See the DAX driver section for more details.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hp# ls /sys/bus/cxl/devices/dax_region0/ dax0.0 devtype modalias uevent dax_region driver subsystemh]hp# ls /sys/bus/cxl/devices/dax_region0/ dax0.0 devtype modalias uevent dax_region driver subsystem}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubeh}(h] dax-regionah ]h"] dax regionah$]h&]uh1hhjhhhhhMubeh}(h]regionsah ]h"]regionsah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(hMailbox Interfacesh]hMailbox Interfaces}(hj5hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj2hhhhhMubh)}(h`. The CEDT may actually define multiple CFMWS configurations to describe the same physical capacity, with the intent to allow users to decide at runtime whether to online memory as interleaved or non-interleaved. ::h](h9Root decoder interleave is defined by CFMWS field of the }(hjhhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hjh]hCEDT}(hjhhhNhNubah}(h]h ](j+stdstd-doceh"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]refdocj8 refdomainj reftypedoc refexplicitrefwarnj>../platform/acpi/cedtuh1hhhhMhjubh. The CEDT may actually define multiple CFMWS configurations to describe the same physical capacity, with the intent to allow users to decide at runtime whether to online memory as interleaved or non-interleaved.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hX Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000100000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000007 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000200000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000006 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000300000000 Window size : 0000000200000000 Interleave Members (2^n) : 01 Interleave Arithmetic : 00 First Target : 00000007 Next Target : 00000006h]hX Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000100000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000007 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000200000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000006 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000300000000 Window size : 0000000200000000 Interleave Members (2^n) : 01 Interleave Arithmetic : 00 First Target : 00000007 Next Target : 00000006}hj(sbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubh)}(hIn this example, the CFMWS defines two discrete non-interleaved 4GB regions for each host bridge, and one interleaved 8GB region that targets both. This would result in 3 root decoders presenting in the root. ::h]hIn this example, the CFMWS defines two discrete non-interleaved 4GB regions for each host bridge, and one interleaved 8GB region that targets both. This would result in 3 root decoders presenting in the root.}(hj6hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM4hjhhubh)}(hXi# ls /sys/bus/cxl/devices/root0/decoder* decoder0.0 decoder0.1 decoder0.2 # cat /sys/bus/cxl/devices/decoder0.0/target_list start size 7 0x100000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.1/target_list start size 6 0x200000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.2/target_list start size 7,6 0x300000000 0x200000000h]hXi# ls /sys/bus/cxl/devices/root0/decoder* decoder0.0 decoder0.1 decoder0.2 # cat /sys/bus/cxl/devices/decoder0.0/target_list start size 7 0x100000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.1/target_list start size 6 0x200000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.2/target_list start size 7,6 0x300000000 0x200000000}hjDsbah}(h]h ]h"]h$]h&]hhuh1hhhhM8hjhhubh)}(hThese decoders are not runtime programmable. They are used to generate a `Memory Region` to bring this memory online with runtime programmed settings at the `Switch` and `Endpoint` decoders.h](hJThese decoders are not runtime programmable. They are used to generate a }(hjRhhhNhNubj\)}(h`Memory Region`h]h Memory Region}(hjZhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjRubhE to bring this memory online with runtime programmed settings at the }(hjRhhhNhNubj\)}(h`Switch`h]hSwitch}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjRubh and }(hjRhhhNhNubj\)}(h `Endpoint`h]hEndpoint}(hj~hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjRubh decoders.}(hjRhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMJhjhhubeh}(h]at-rootah ]h"]at rootah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(hAt Host Bridge or Switchh]hAt Host Bridge or Switch}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMOubh)}(hN`Host Bridge` and `Switch` decoders are programmable via the following fields:h](j\)}(h `Host Bridge`h]h Host Bridge}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh and }(hjhhhNhNubj\)}(h`Switch`h]hSwitch}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh4 decoders are programmable via the following fields:}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMPhjhhubjA)}(hhh](jF)}(h@:code:`start` - the HPA region associated with the memory regionh]h)}(hjh](j)}(h :code:`start`h]hstart}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh3 - the HPA region associated with the memory region}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMRhjubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(h%:code:`size` - the size of the regionh]h)}(hjh](j)}(h :code:`size`h]hsize}(hj hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj ubh - the size of the region}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMShjubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(h2:code:`target_list` - the list of downstream portsh]h)}(hj.h](j)}(h:code:`target_list`h]h target_list}(hj3hhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj0ubh - the list of downstream ports}(hj0hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMThj,ubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(hJ:code:`interleave_ways` - the number downstream ports to interleave acrossh]h)}(hjTh](j)}(h:code:`interleave_ways`h]hinterleave_ways}(hjYhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjVubh3 - the number downstream ports to interleave across}(hjVhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMUhjRubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubjF)}(hC:code:`interleave_granularity` - the granularity to interleave at. h]h)}(hB:code:`interleave_granularity` - the granularity to interleave at.h](j)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhj|ubh$ - the granularity to interleave at.}(hj|hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMVhjxubah}(h]h ]h"]h$]h&]uh1jEhjhhhhhNubeh}(h]h ]h"]h$]h&]j-uh1j@hhhMRhjhhubh)}(hX$Linux expects the :code:`interleave_granularity` of switch decoders to be derived from their upstream port connections. In `Cross-Link First` interleave configurations, the :code:`interleave_granularity` of a decoder is equal to :code:`parent_interleave_granularity * parent_interleave_ways`.h](hLinux expects the }(hjhhhNhNubj)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubhK of switch decoders to be derived from their upstream port connections. In }(hjhhhNhNubj\)}(h`Cross-Link First`h]hCross-Link First}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh interleave configurations, the }(hjhhhNhNubj)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh of a decoder is equal to }(hjhhhNhNubj)}(h>:code:`parent_interleave_granularity * parent_interleave_ways`h]h6parent_interleave_granularity * parent_interleave_ways}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMXhjhhubeh}(h]at-host-bridge-or-switchah ]h"]at host bridge or switchah$]h&]uh1hhjhhhhhMOubh)}(hhh](h)}(h At Endpointh]h At Endpoint}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM^ubh)}(h`Endpoint Decoders` are programmed similar to Host Bridge and Switch decoders, with the exception that the ways and granularity are defined by the interleave set (e.g. the interleave settings defined by the associated `Memory Region`).h](j\)}(h`Endpoint Decoders`h]hEndpoint Decoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh are programmed similar to Host Bridge and Switch decoders, with the exception that the ways and granularity are defined by the interleave set (e.g. the interleave settings defined by the associated }(hjhhhNhNubj\)}(h`Memory Region`h]h Memory Region}(hj.hhhNhNubah}(h]h ]h"]h$]h&]uh1j[hjubh).}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM_hjhhubjA)}(hhh](jF)}(h@:code:`start` - the HPA region associated with the memory regionh]h)}(hjKh](j)}(h :code:`start`h]hstart}(hjPhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjMubh3 - the HPA region associated with the memory region}(hjMhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMchjIubah}(h]h ]h"]h$]h&]uh1jEhjFhhhhhNubjF)}(h%:code:`size` - the size of the regionh]h)}(hjqh](j)}(h :code:`size`h]hsize}(hjvhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjsubh - the size of the region}(hjshhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMdhjoubah}(h]h ]h"]h$]h&]uh1jEhjFhhhhhNubjF)}(hD:code:`interleave_ways` - the number endpoints in the interleave seth]h)}(hjh](j)}(h:code:`interleave_ways`h]hinterleave_ways}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh- - the number endpoints in the interleave set}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMehjubah}(h]h ]h"]h$]h&]uh1jEhjFhhhhhNubjF)}(hC:code:`interleave_granularity` - the granularity to interleave at. h]h)}(hB:code:`interleave_granularity` - the granularity to interleave at.h](j)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]jah"]h$]h&]languagehuh1jhjubh$ - the granularity to interleave at.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMfhjubah}(h]h ]h"]h$]h&]uh1jEhjFhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j@hhhMchjhhubh)}(hThese settings are used by endpoint decoders to *Translate* memory requests from HPA to DPA. This is why they must be aware of the entire interleave set.h](h0These settings are used by endpoint decoders to }(hjhhhNhNubj )}(h *Translate*h]h Translate}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j hjubh_ memory requests from HPA to DPA. This is why they must be aware of the entire interleave set.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhhjhhubh)}(hLinux does not support unbalanced interleave configurations. As a result, all endpoints in an interleave set must have the same ways and granularity.h]hLinux does not support unbalanced interleave configurations. As a result, all endpoints in an interleave set must have the same ways and granularity.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMkhjhhubeh}(h] at-endpointah ]h"] at endpointah$]h&]uh1hhjhhhhhM^ubeh}(h] interleaveah ]h"] interleaveah$]h&]uh1hhjhhhhhMubeh}(h]decoder-programmingah ]h"]decoder programmingah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hExample Configurationsh]hExample Configurations}(hj1hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj.hhhhhMoubhcompound)}(hhh]htoctree)}(hhh]h}(h]h ]h"]h$]h&]hj8entries](N9driver-api/cxl/linux/example-configurations/single-deviceN9driver-api/cxl/linux/example-configurations/hb-interleaveN?driver-api/cxl/linux/example-configurations/intra-hb-interleaveN