sphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget3/translations/zh_CN/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/zh_TW/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/it_IT/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/ja_JP/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/ko_KR/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget3/translations/sp_SP/driver-api/cxl/linux/cxl-drivermodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhM/var/lib/git/docbuild/linux/Documentation/driver-api/cxl/linux/cxl-driver.rsthKubhsection)}(hhh](htitle)}(hCXL Driver Operationh]hCXL Driver Operation}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh paragraph)}(h7The devices described in this section are present in ::h]h4The devices described in this section are present in}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh literal_block)}(h/sys/bus/cxl/devices/ /dev/cxl/h]h/sys/bus/cxl/devices/ /dev/cxl/}hhsbah}(h]h ]h"]h$]h&]hhuh1hhhhK hhhhubh)}(h|The :code:`cxl-cli` library, maintained as part of the NDTCL project, may be used to script interactions with these devices.h](hThe }(hhhhhNhNubhliteral)}(h:code:`cxl-cli`h]hcxl-cli}(hhhhhNhNubah}(h]h ]codeah"]h$]h&]languagehuh1hhhubhi library, maintained as part of the NDTCL project, may be used to script interactions with these devices.}(hhhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hhh](h)}(hDriversh]hDrivers}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhKubh)}(h1The CXL driver is split into a number of drivers.h]h1The CXL driver is split into a number of drivers.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj hhubh bullet_list)}(hhh](h list_item)}(h?cxl_core - fundamental init interface and core object creationh]h)}(hj5h]h?cxl_core - fundamental init interface and core object creation}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj3ubah}(h]h ]h"]h$]h&]uh1j1hj.hhhhhNubj2)}(hEcxl_port - initializes root and provides port enumeration interface.h]h)}(hjLh]hEcxl_port - initializes root and provides port enumeration interface.}(hjNhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjJubah}(h]h ]h"]h$]h&]uh1j1hj.hhhhhNubj2)}(hCcxl_acpi - initializes root decoders and interacts with ACPI data.h]h)}(hjch]hCcxl_acpi - initializes root decoders and interacts with ACPI data.}(hjehhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjaubah}(h]h ]h"]h$]h&]uh1j1hj.hhhhhNubj2)}(h&cxl_p/mem - initializes memory devicesh]h)}(hjzh]h&cxl_p/mem - initializes memory devices}(hj|hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjxubah}(h]h ]h"]h$]h&]uh1j1hj.hhhhhNubj2)}(hCcxl_pci - uses cxl_port to enumates the actual fabric hierarchy. h]h)}(hBcxl_pci - uses cxl_port to enumates the actual fabric hierarchy.h]hBcxl_pci - uses cxl_port to enumates the actual fabric hierarchy.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j1hj.hhhhhNubeh}(h]h ]h"]h$]h&]bullet*uh1j,hhhKhj hhubeh}(h]driversah ]h"]driversah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hDriver Devicesh]hDriver Devices}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hHere is an example from a single-socket system with 4 host bridges. Two host bridges have a single memory device attached, and the devices are interleaved into a single memory region. The memory region has been converted to dax. ::h]hHere is an example from a single-socket system with 4 host bridges. Two host bridges have a single memory device attached, and the devices are interleaved into a single memory region. The memory region has been converted to dax.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h# ls /sys/bus/cxl/devices/ dax_region0 decoder3.0 decoder6.0 mem0 port3 decoder0.0 decoder4.0 decoder6.1 mem1 port4 decoder1.0 decoder5.0 endpoint5 port1 region0 decoder2.0 decoder5.1 endpoint6 port2 root0h]h# ls /sys/bus/cxl/devices/ dax_region0 decoder3.0 decoder6.0 mem0 port3 decoder0.0 decoder4.0 decoder6.1 mem1 port4 decoder1.0 decoder5.0 endpoint5 port1 region0 decoder2.0 decoder5.1 endpoint6 port2 root0}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKhjhhubhfigure)}(hhh](kfigure kernel_render)}(hhh]h)}(hXdigraph foo { "root0" -> "port1"; "root0" -> "port3"; "root0" -> "decoder0.0"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; "port1" -> "decoder1.0"; "port3" -> "decoder3.0"; "endpoint5" -> "decoder5.0"; "endpoint6" -> "decoder6.0"; "decoder0.0" -> "region0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; "decoder5.0" -> "region0"; "decoder6.0" -> "region0"; "region0" -> "dax_region0"; "dax_region0" -> "dax0.0"; }h]hXdigraph foo { "root0" -> "port1"; "root0" -> "port3"; "root0" -> "decoder0.0"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; "port1" -> "decoder1.0"; "port3" -> "decoder3.0"; "endpoint5" -> "decoder5.0"; "endpoint6" -> "decoder6.0"; "decoder0.0" -> "region0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; "decoder5.0" -> "region0"; "decoder6.0" -> "region0"; "region0" -> "dax_region0"; "dax_region0" -> "dax0.0"; }}hjsbah}(h]h ]h"]h$]h&]hhuh1hhjhhubah}(h]h ]h"]h$]h&]alt9Digraph of CXL fabric describing host-bridge interleavingsrclangDOTuh1jhjubhcaption)}(hBDiagraph of CXL fabric with a host-bridge interleave memory regionh]hBDiagraph of CXL fabric with a host-bridge interleave memory region}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubeh}(h]id1ah ]h"]h$]h&]altjcaptionj uh1jhjhhhhhNubh)}(hFor this section we'll explore the devices present in this configuration, but we'll explore more configurations in-depth in example configurations below.h]hFor this section we’ll explore the devices present in this configuration, but we’ll explore more configurations in-depth in example configurations below.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK?hjhhubh)}(hhh](h)}(h Base Devicesh]h Base Devices}(hj1hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj.hhhhhKCubh)}(hMost devices in a CXL fabric are a `port` of some kind (because each device mostly routes request from one device to the next, rather than provide a direct service).h](h#Most devices in a CXL fabric are a }(hj?hhhNhNubhtitle_reference)}(h`port`h]hport}(hjIhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj?ubh| of some kind (because each device mostly routes request from one device to the next, rather than provide a direct service).}(hj?hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKDhj.hhubh)}(hhh](h)}(hRooth]hRoot}(hjdhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjahhhhhKIubh)}(hThe `CXL Root` is logical object created by the `cxl_acpi` driver during :code:`cxl_acpi_probe` - if the :code:`ACPI0017` `Compute Express Link Root Object` Device Class is found.h](hThe }(hjrhhhNhNubjH)}(h `CXL Root`h]hCXL Root}(hjzhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjrubh" is logical object created by the }(hjrhhhNhNubjH)}(h `cxl_acpi`h]hcxl_acpi}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjrubh driver during }(hjrhhhNhNubh)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjrubh - if the }(hjrhhhNhNubh)}(h:code:`ACPI0017`h]hACPI0017}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjrubh }(hjrhhhNhNubjH)}(h"`Compute Express Link Root Object`h]h Compute Express Link Root Object}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjrubh Device Class is found.}(hjrhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKJhjahhubh)}(hThe Root contains links to:h]hThe Root contains links to:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKNhjahhubj-)}(hhh](j2)}(hN`Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>` h]h)}(hM`Host Bridge Ports` defined by CHBS in the :doc:`CEDT<../platform/acpi/cedt>`h](jH)}(h`Host Bridge Ports`h]hHost Bridge Ports}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh defined by CHBS in the }(hjhhhNhNubh)}(h":doc:`CEDT<../platform/acpi/cedt>`h]hinline)}(hj h]hCEDT}(hj hhhNhNubah}(h]h ](xrefstdstd-doceh"]h$]h&]uh1j hjubah}(h]h ]h"]h$]h&]refdocdriver-api/cxl/linux/cxl-driver refdomainjreftypedoc refexplicitrefwarn reftarget../platform/acpi/cedtuh1hhhhKPhjubeh}(h]h ]h"]h$]h&]uh1hhhhKPhjubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(h?`Downstream Ports` typically connected to `Host Bridge Ports`. h]h)}(h>`Downstream Ports` typically connected to `Host Bridge Ports`.h](jH)}(h`Downstream Ports`h]hDownstream Ports}(hj@hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj<ubh typically connected to }(hj<hhhNhNubjH)}(h`Host Bridge Ports`h]hHost Bridge Ports}(hjRhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj<ubh.}(hj<hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKRhj8ubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(hH`Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>` h]h)}(hG`Root Decoders` defined by CFMWS the :doc:`CEDT<../platform/acpi/cedt>`h](jH)}(h`Root Decoders`h]h Root Decoders}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjtubh defined by CFMWS the }(hjthhhNhNubh)}(h":doc:`CEDT<../platform/acpi/cedt>`h]j )}(hjh]hCEDT}(hjhhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hjubah}(h]h ]h"]h$]h&]refdocj$ refdomainjreftypedoc refexplicitrefwarnj*../platform/acpi/cedtuh1hhhhKThjtubeh}(h]h ]h"]h$]h&]uh1hhhhKThjpubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j,hhhKPhjahhubh)}(hXE# ls /sys/bus/cxl/devices/root0 decoder0.0 dport0 dport5 port2 subsystem decoders_committed dport1 modalias port3 uevent devtype dport4 port1 port4 uport # cat /sys/bus/cxl/devices/root0/devtype cxl_port # cat port1/devtype cxl_port # cat decoder0.0/devtype cxl_decoder_rooth]hXE# ls /sys/bus/cxl/devices/root0 decoder0.0 dport0 dport5 port2 subsystem decoders_committed dport1 modalias port3 uevent devtype dport4 port1 port4 uport # cat /sys/bus/cxl/devices/root0/devtype cxl_port # cat port1/devtype cxl_port # cat decoder0.0/devtype cxl_decoder_root}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKXhjahhubh)}(hThe root is first `logical port` in the CXL fabric, as presented by the Linux CXL driver. The `CXL root` is a special type of `switch port`, in that it only has downstream port connections.h](hThe root is first }(hjhhhNhNubjH)}(h`logical port`h]h logical port}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh? in the CXL fabric, as presented by the Linux CXL driver. The }(hjhhhNhNubjH)}(h `CXL root`h]hCXL root}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh is a special type of }(hjhhhNhNubjH)}(h `switch port`h]h switch port}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh2, in that it only has downstream port connections.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKfhjahhubeh}(h]rootah ]h"]rootah$]h&]uh1hhj.hhhhhKIubh)}(hhh](h)}(hPorth]hPort}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKkubh)}(hX.A `port` object is better described as a `switch port`. It may represent a host bridge to the root or an actual switch port on a switch. A `switch port` contains one or more decoders used to route memory requests downstream ports, which may be connected to another `switch port` or an `endpoint port`.h](hA }(hj'hhhNhNubjH)}(h`port`h]hport}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj'ubh! object is better described as a }(hj'hhhNhNubjH)}(h `switch port`h]h switch port}(hjAhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj'ubhV. It may represent a host bridge to the root or an actual switch port on a switch. A }(hj'hhhNhNubjH)}(h `switch port`h]h switch port}(hjShhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj'ubhq contains one or more decoders used to route memory requests downstream ports, which may be connected to another }(hj'hhhNhNubjH)}(h `switch port`h]h switch port}(hjehhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj'ubh or an }(hj'hhhNhNubjH)}(h`endpoint port`h]h endpoint port}(hjwhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj'ubh.}(hj'hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKlhjhhubh)}(hX2# ls /sys/bus/cxl/devices/port1 decoder1.0 dport0 driver parent_dport uport decoders_committed dport113 endpoint5 subsystem devtype dport2 modalias uevent # cat devtype cxl_port # cat decoder1.0/devtype cxl_decoder_switch # cat endpoint5/devtype cxl_porth]hX2# ls /sys/bus/cxl/devices/port1 decoder1.0 dport0 driver parent_dport uport decoders_committed dport113 endpoint5 subsystem devtype dport2 modalias uevent # cat devtype cxl_port # cat decoder1.0/devtype cxl_decoder_switch # cat endpoint5/devtype cxl_port}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhKshjhhubh)}(hCXL `Host Bridges` in the fabric are probed during :code:`cxl_acpi_probe` at the time the `CXL Root` is probed. The allows for the immediate logical connection to between the root and host bridge.h](hCXL }(hjhhhNhNubjH)}(h`Host Bridges`h]h Host Bridges}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh! in the fabric are probed during }(hjhhhNhNubh)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh at the time the }(hjhhhNhNubjH)}(h `CXL Root`h]hCXL Root}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubha is probed. The allows for the immediate logical connection to between the root and host bridge.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hhh](j2)}(h;The root has a downstream port connection to a host bridge h]h)}(h:The root has a downstream port connection to a host bridgeh]h:The root has a downstream port connection to a host bridge}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(h=The host bridge has an upstream port connection to the root. h]h)}(h "port1"; "root0" -> "port2"; "root0" -> "port3"; "root0" -> "port4"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; }h]hdigraph foo { "root0" -> "port1"; "root0" -> "port2"; "root0" -> "port3"; "root0" -> "port4"; "port1" -> "endpoint5"; "port3" -> "endpoint6"; }}hj^sbah}(h]h ]h"]h$]h&]hhuh1hhj[hhubah}(h]h ]h"]h$]h&]j9Digraph of CXL fabric describing host-bridge interleavingjDOTuh1jhjXubj)}(hBDiagraph of CXL fabric with a host-bridge interleave memory regionh]hBDiagraph of CXL fabric with a host-bridge interleave memory region}(hjthhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjXubeh}(h]id2ah ]h"]h$]h&]altjrcaptionjvuh1jhj9hhhhhNubeh}(h]port-relationshipsah ]h"]port relationshipsah$]h&]uh1hhj.hhhhhKubeh}(h] base-devicesah ]h"] base devicesah$]h&]uh1hhjhhhhhKCubh)}(hhh](h)}(hDecodersh]hDecoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hA `Decoder` is short for a CXL Host-Managed Device Memory (HDM) Decoder. It is a device that routes accesses through the CXL fabric to an endpoint, and at the endpoint translates a `Host Physical` to `Device Physical` Addressing.h](hA }(hjhhhNhNubjH)}(h `Decoder`h]hDecoder}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh is short for a CXL Host-Managed Device Memory (HDM) Decoder. It is a device that routes accesses through the CXL fabric to an endpoint, and at the endpoint translates a }(hjhhhNhNubjH)}(h`Host Physical`h]h Host Physical}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh to }(hjhhhNhNubjH)}(h`Device Physical`h]hDevice Physical}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh Addressing.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hThe CXL 3.1 specification heavily implies that only endpoint decoders should engage in translation of `Host Physical Address` to `Device Physical Address`. ::h](hfThe CXL 3.1 specification heavily implies that only endpoint decoders should engage in translation of }(hjhhhNhNubjH)}(h`Host Physical Address`h]hHost Physical Address}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh to }(hjhhhNhNubjH)}(h`Device Physical Address`h]hDevice Physical Address}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h8.2.4.20 CXL HDM Decoder Capability Structure IMPLEMENTATION NOTE CXL Host Bridge and Upstream Switch Port Decode Flow IMPLEMENTATION NOTE Device Decode Logich]h8.2.4.20 CXL HDM Decoder Capability Structure IMPLEMENTATION NOTE CXL Host Bridge and Upstream Switch Port Decode Flow IMPLEMENTATION NOTE Device Decode Logic}hj"sbah}(h]h ]h"]h$]h&]hhuh1hhhhKhjhhubh)}(h@These notes imply that there are two logical groups of decoders.h]h@These notes imply that there are two logical groups of decoders.}(hj0hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hhh](j2)}(hdRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA. h]h)}(hcRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA.h]hcRouting Decoder - a decoder which routes accesses but does not translate addresses from HPA to DPA.}(hjEhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjAubah}(h]h ]h"]h$]h&]uh1j1hj>hhhhhNubj2)}(hfTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service. h]h)}(heTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service.h]heTranslating Decoder - a decoder which translates accesses from HPA to DPA for an endpoint to service.}(hj]hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjYubah}(h]h ]h"]h$]h&]uh1j1hj>hhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j,hhhKhjhhubh)}(hThe CXL drivers distinguish 3 decoder types: root, switch, and endpoint. Only endpoint decoders are Translating Decoders, all others are Routing Decoders.h]hThe CXL drivers distinguish 3 decoder types: root, switch, and endpoint. Only endpoint decoders are Translating Decoders, all others are Routing Decoders.}(hjwhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubhnote)}(hXPLATFORM VENDORS BE AWARE Linux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric. It is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.h](h)}(hPLATFORM VENDORS BE AWAREh]hPLATFORM VENDORS BE AWARE}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hLinux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric.h]hLinux makes a strong assumption that endpoint decoders are the only decoder in the fabric that actively translates HPA to DPA. Linux assumes routing decoders pass the HPA unchanged to the next decoder in the fabric.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hXIt is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.h]hXIt is therefore assumed that any given decoder in the fabric will have an address range that is a subset of its upstream port decoder. Any deviation from this scheme undefined per the specification. Linux prioritizes spec-defined / architectural behavior.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubeh}(h]h ]h"]h$]h&]uh1jhjhhhhhNubh)}(hDecoders may have one or more `Downstream Targets` if configured to interleave memory accesses. This will be presented in sysfs via the :code:`target_list` parameter.h](hDecoders may have one or more }(hjhhhNhNubjH)}(h`Downstream Targets`h]hDownstream Targets}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubhW if configured to interleave memory accesses. This will be presented in sysfs via the }(hjhhhNhNubh)}(h:code:`target_list`h]h target_list}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh parameter.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hhh](h)}(h Root Decoderh]h Root Decoder}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hXA `Root Decoder` is logical construct of the physical address and interleave configurations present in the CFMWS field of the :doc:`CEDT <../platform/acpi/cedt>`. Linux presents this information as a decoder present in the `CXL Root`. We consider this a `Root Decoder`, though technically it exists on the boundary of the CXL specification and platform-specific CXL root implementations.h](hA }(hjhhhNhNubjH)}(h`Root Decoder`h]h Root Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubhn is logical construct of the physical address and interleave configurations present in the CFMWS field of the }(hjhhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hj h]hCEDT}(hj hhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hj ubah}(h]h ]h"]h$]h&]refdocj$ refdomainj' reftypedoc refexplicitrefwarnj*../platform/acpi/cedtuh1hhhhKhjubh>. Linux presents this information as a decoder present in the }(hjhhhNhNubjH)}(h `CXL Root`h]hCXL Root}(hj= hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh. We consider this a }(hjhhhNhNubjH)}(h`Root Decoder`h]h Root Decoder}(hjO hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubhw, though technically it exists on the boundary of the CXL specification and platform-specific CXL root implementations.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hLinux considers these logical decoders a type of `Routing Decoder`, and is the first decoder in the CXL fabric to receive a memory access from the platform's memory controllers.h](h1Linux considers these logical decoders a type of }(hjg hhhNhNubjH)}(h`Routing Decoder`h]hRouting Decoder}(hjo hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjg ubhq, and is the first decoder in the CXL fabric to receive a memory access from the platform’s memory controllers.}(hjg hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h`Root Decoders` are created during :code:`cxl_acpi_probe`. One root decoder is created per CFMWS entry in the :doc:`CEDT <../platform/acpi/cedt>`.h](jH)}(h`Root Decoders`h]h Root Decoders}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh are created during }(hj hhhNhNubh)}(h:code:`cxl_acpi_probe`h]hcxl_acpi_probe}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh6. One root decoder is created per CFMWS entry in the }(hj hhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hj h]hCEDT}(hj hhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hj ubah}(h]h ]h"]h$]h&]refdocj$ refdomainj reftypedoc refexplicitrefwarnj*../platform/acpi/cedtuh1hhhhMhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe :code:`target_list` parameter is filled by the CFMWS target fields. Targets of a root decoder are `Host Bridges`, which means interleave done at the root decoder level is an `Inter-Host-Bridge Interleave`.h](hThe }(hj hhhNhNubh)}(h:code:`target_list`h]h target_list}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubhO parameter is filled by the CFMWS target fields. Targets of a root decoder are }(hj hhhNhNubjH)}(h`Host Bridges`h]h Host Bridges}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh>, which means interleave done at the root decoder level is an }(hj hhhNhNubjH)}(h`Inter-Host-Bridge Interleave`h]hInter-Host-Bridge Interleave}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hAOnly root decoders are capable of `Inter-Host-Bridge Interleave`.h](h"Only root decoders are capable of }(hj hhhNhNubjH)}(h`Inter-Host-Bridge Interleave`h]hInter-Host-Bridge Interleave}(hj' hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM hjhhubh)}(hXDSuch interleaves must be configured by the platform and described in the ACPI CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL host bridge UIDs in the CHBS field of the :doc:`CEDT <../platform/acpi/cedt>` and the UID field of CXL Host Bridges defined in the :doc:`DSDT <../platform/acpi/dsdt>`.h](hSuch interleaves must be configured by the platform and described in the ACPI CEDT CFMWS, as the target CXL host bridge UIDs in the CFMWS must match the CXL host bridge UIDs in the CHBS field of the }(hj? hhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hjI h]hCEDT}(hjK hhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hjG ubah}(h]h ]h"]h$]h&]refdocj$ refdomainjU reftypedoc refexplicitrefwarnj*../platform/acpi/cedtuh1hhhhM hj? ubh6 and the UID field of CXL Host Bridges defined in the }(hj? hhhNhNubh)}(h#:doc:`DSDT <../platform/acpi/dsdt>`h]j )}(hjm h]hDSDT}(hjo hhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hjk ubah}(h]h ]h"]h$]h&]refdocj$ refdomainjy reftypedoc refexplicitrefwarnj*../platform/acpi/dsdtuh1hhhhM hj? ubh.}(hj? hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM hjhhubh)}(hInterleave settings in a root decoder describe how to interleave accesses among the *immediate downstream targets*, not the entire interleave set.h](hTInterleave settings in a root decoder describe how to interleave accesses among the }(hj hhhNhNubhemphasis)}(h*immediate downstream targets*h]himmediate downstream targets}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh , not the entire interleave set.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(h9The memory range described in the root decoder is used toh]h9The memory range described in the root decoder is used to}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubhenumerated_list)}(hhh](j2)}(h>Create a memory region (:code:`region0` in this example), and h]h)}(h=Create a memory region (:code:`region0` in this example), andh](hCreate a memory region (}(hj hhhNhNubh)}(h:code:`region0`h]hregion0}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh in this example), and}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1j1hj hhhhhNubj2)}(hLAssociate the region with an IO Memory Resource (:code:`kernel/resource.c`) h]h)}(hKAssociate the region with an IO Memory Resource (:code:`kernel/resource.c`)h](h1Associate the region with an IO Memory Resource (}(hj hhhNhNubh)}(h:code:`kernel/resource.c`h]hkernel/resource.c}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh)}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1j1hj hhhhhNubeh}(h]h ]h"]h$]h&]enumtypearabicprefixhsuffix)uh1j hjhhhhhMubh)}(hX# ls /sys/bus/cxl/devices/decoder0.0/ cap_pmem devtype region0 cap_ram interleave_granularity size cap_type2 interleave_ways start cap_type3 locked subsystem create_ram_region modalias target_list delete_region qos_class uevent # cat /sys/bus/cxl/devices/decoder0.0/region0/resource 0xc050000000h]hX# ls /sys/bus/cxl/devices/decoder0.0/ cap_pmem devtype region0 cap_ram interleave_granularity size cap_type2 interleave_ways start cap_type3 locked subsystem create_ram_region modalias target_list delete_region qos_class uevent # cat /sys/bus/cxl/devices/decoder0.0/region0/resource 0xc050000000}hj+ sbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubh)}(hThe IO Memory Resource is created during early boot when the CFMWS region is identified in the EFI Memory Map or E820 table (on x86).h]hThe IO Memory Resource is created during early boot when the CFMWS region is identified in the EFI Memory Map or E820 table (on x86).}(hj9 hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM(hjhhubh)}(h}Root decoders are defined as a separate devtype, but are also a type of `Switch Decoder` due to having downstream targets. ::h](hHRoot decoders are defined as a separate devtype, but are also a type of }(hjG hhhNhNubjH)}(h`Switch Decoder`h]hSwitch Decoder}(hjO hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjG ubh" due to having downstream targets.}(hjG hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM+hjhhubh)}(h@# cat /sys/bus/cxl/devices/decoder0.0/devtype cxl_decoder_rooth]h@# cat /sys/bus/cxl/devices/decoder0.0/devtype cxl_decoder_root}hjg sbah}(h]h ]h"]h$]h&]hhuh1hhhhM.hjhhubeh}(h] root-decoderah ]h"] root decoderah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hSwitch Decoderh]hSwitch Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj} hhhhhM2ubh)}(hAny non-root, translating decoder is considered a `Switch Decoder`, and will present with the type :code:`cxl_decoder_switch`. Both `Host Bridge` and `CXL Switch` (device) decoders are of type :code:`cxl_decoder_switch`. ::h](h2Any non-root, translating decoder is considered a }(hj hhhNhNubjH)}(h`Switch Decoder`h]hSwitch Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh!, and will present with the type }(hj hhhNhNubh)}(h:code:`cxl_decoder_switch`h]hcxl_decoder_switch}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh. Both }(hj hhhNhNubjH)}(h `Host Bridge`h]h Host Bridge}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh and }(hj hhhNhNubjH)}(h `CXL Switch`h]h CXL Switch}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh (device) decoders are of type }(hj hhhNhNubh)}(h:code:`cxl_decoder_switch`h]hcxl_decoder_switch}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM3hj} hhubh)}(hXM# ls /sys/bus/cxl/devices/decoder1.0/ devtype locked size target_list interleave_granularity modalias start target_type interleave_ways region subsystem uevent # cat /sys/bus/cxl/devices/decoder1.0/devtype cxl_decoder_switch # cat /sys/bus/cxl/devices/decoder1.0/region region0h]hXM# ls /sys/bus/cxl/devices/decoder1.0/ devtype locked size target_list interleave_granularity modalias start target_type interleave_ways region subsystem uevent # cat /sys/bus/cxl/devices/decoder1.0/devtype cxl_decoder_switch # cat /sys/bus/cxl/devices/decoder1.0/region region0}hj sbah}(h]h ]h"]h$]h&]hhuh1hhhhM7hj} hhubh)}(hA `Switch Decoder` has associations between a region defined by a root decoder and downstream target ports. Interleaving done within a switch decoder is a multi-downstream-port interleave (or `Intra-Host-Bridge Interleave` for host bridges).h](hA }(hj hhhNhNubjH)}(h`Switch Decoder`h]hSwitch Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh has associations between a region defined by a root decoder and downstream target ports. Interleaving done within a switch decoder is a multi-downstream-port interleave (or }(hj hhhNhNubjH)}(h`Intra-Host-Bridge Interleave`h]hIntra-Host-Bridge Interleave}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh for host bridges).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMBhj} hhubh)}(hInterleave settings in a switch decoder describe how to interleave accesses among the *immediate downstream targets*, not the entire interleave set.h](hVInterleave settings in a switch decoder describe how to interleave accesses among the }(hj8 hhhNhNubj )}(h*immediate downstream targets*h]himmediate downstream targets}(hj@ hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj8 ubh , not the entire interleave set.}(hj8 hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMGhj} hhubh)}(hSwitch decoders are created during :code:`cxl_switch_port_probe` in the :code:`cxl_port` driver, and is created based on a PCI device's DVSEC registers.h](h#Switch decoders are created during }(hjX hhhNhNubh)}(h:code:`cxl_switch_port_probe`h]hcxl_switch_port_probe}(hj` hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjX ubh in the }(hjX hhhNhNubh)}(h:code:`cxl_port`h]hcxl_port}(hjs hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjX ubhB driver, and is created based on a PCI device’s DVSEC registers.}(hjX hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMJhj} hhubh)}(hSwitch decoder programming is validated during probe if the platform programs them during boot (See `Auto Decoders` below), or on commit if programmed at runtime (See `Runtime Programming` below).h](hdSwitch decoder programming is validated during probe if the platform programs them during boot (See }(hj hhhNhNubjH)}(h`Auto Decoders`h]h Auto Decoders}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh4 below), or on commit if programmed at runtime (See }(hj hhhNhNubjH)}(h`Runtime Programming`h]hRuntime Programming}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh below).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMNhj} hhubeh}(h]switch-decoderah ]h"]switch decoderah$]h&]uh1hhjhhhhhM2ubh)}(hhh](h)}(hEndpoint Decoderh]hEndpoint Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMTubh)}(hAny decoder attached to a *terminal* point in the CXL fabric (`An Endpoint`) is considered an `Endpoint Decoder`. Endpoint decoders are of type :code:`cxl_decoder_endpoint`. ::h](hAny decoder attached to a }(hj hhhNhNubj )}(h *terminal*h]hterminal}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh point in the CXL fabric (}(hj hhhNhNubjH)}(h `An Endpoint`h]h An Endpoint}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh) is considered an }(hj hhhNhNubjH)}(h`Endpoint Decoder`h]hEndpoint Decoder}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh . Endpoint decoders are of type }(hj hhhNhNubh)}(h:code:`cxl_decoder_endpoint`h]hcxl_decoder_endpoint}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMUhj hhubh)}(hXt# ls /sys/bus/cxl/devices/decoder5.0 devtype locked start dpa_resource modalias subsystem dpa_size mode target_type interleave_granularity region uevent interleave_ways size # cat /sys/bus/cxl/devices/decoder5.0/devtype cxl_decoder_endpoint # cat /sys/bus/cxl/devices/decoder5.0/region region0h]hXt# ls /sys/bus/cxl/devices/decoder5.0 devtype locked start dpa_resource modalias subsystem dpa_size mode target_type interleave_granularity region uevent interleave_ways size # cat /sys/bus/cxl/devices/decoder5.0/devtype cxl_decoder_endpoint # cat /sys/bus/cxl/devices/decoder5.0/region region0}hj. sbah}(h]h ]h"]h$]h&]hhuh1hhhhMYhj hhubh)}(hAn `Endpoint Decoder` has an association with a region defined by a root decoder and describes the device-local resource associated with this region.h](hAn }(hj< hhhNhNubjH)}(h`Endpoint Decoder`h]hEndpoint Decoder}(hjD hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj< ubh has an association with a region defined by a root decoder and describes the device-local resource associated with this region.}(hj< hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMfhj hhubh)}(hUnlike root and switch decoders, endpoint decoders translate `Host Physical` to `Device Physical` address ranges. The interleave settings on an endpoint therefore describe the entire *interleave set*.h](h=Unlike root and switch decoders, endpoint decoders translate }(hj\ hhhNhNubjH)}(h`Host Physical`h]h Host Physical}(hjd hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj\ ubh to }(hj\ hhhNhNubjH)}(h`Device Physical`h]hDevice Physical}(hjv hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj\ ubhW address ranges. The interleave settings on an endpoint therefore describe the entire }(hj\ hhhNhNubj )}(h*interleave set*h]hinterleave set}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj\ ubh.}(hj\ hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMihj hhubh)}(h`Device Physical Address` regions must be committed in-order. For example, the DPA region starting at 0x80000000 cannot be committed before the DPA region starting at 0x0.h](jH)}(h`Device Physical Address`h]hDevice Physical Address}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj ubh regions must be committed in-order. For example, the DPA region starting at 0x80000000 cannot be committed before the DPA region starting at 0x0.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMmhj hhubh)}(hAs of Linux v6.15, Linux does not support *imbalanced* interleave setups, all endpoints in an interleave set are expected to have the same interleave settings (granularity and ways must be the same).h](h*As of Linux v6.15, Linux does not support }(hj hhhNhNubj )}(h *imbalanced*h]h imbalanced}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj ubh interleave setups, all endpoints in an interleave set are expected to have the same interleave settings (granularity and ways must be the same).}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMqhj hhubh)}(hEndpoint decoders are created during :code:`cxl_endpoint_port_probe` in the :code:`cxl_port` driver, and is created based on a PCI device's DVSEC registers.h](h%Endpoint decoders are created during }(hj hhhNhNubh)}(h:code:`cxl_endpoint_port_probe`h]hcxl_endpoint_port_probe}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubh in the }(hj hhhNhNubh)}(h:code:`cxl_port`h]hcxl_port}(hj hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj ubhB driver, and is created based on a PCI device’s DVSEC registers.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMuhj hhubeh}(h]endpoint-decoderah ]h"]endpoint decoderah$]h&]uh1hhjhhhhhMTubh)}(hhh](h)}(hDecoder Relationshipsh]hDecoder Relationships}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMyubh)}(hXIn our example described above, there is one root decoder which routes memory accesses over two host bridges. Each host bridge has a decoder which routes access to their singular endpoint targets. Each endpoint has a decoder which translates HPA to DPA and services the memory request.h]hXIn our example described above, there is one root decoder which routes memory accesses over two host bridges. Each host bridge has a decoder which routes access to their singular endpoint targets. Each endpoint has a decoder which translates HPA to DPA and services the memory request.}(hj)hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMzhjhhubh)}(hThe driver validates relationships between ports by decoder programming, so we can think of decoders being related in a similarly hierarchical fashion to ports.h]hThe driver validates relationships between ports by decoder programming, so we can think of decoders being related in a similarly hierarchical fashion to ports.}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj)}(hhh](j)}(hhh]h)}(hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; }h]hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "decoder1.0"; "decoder0.0" -> "decoder3.0"; "decoder1.0" -> "decoder5.0"; "decoder3.0" -> "decoder6.0"; }}hjKsbah}(h]h ]h"]h$]h&]hhuh1hhjHhhubah}(h]h ]h"]h$]h&]jQDigraph of hierarchical relationship between root, switch, and endpoint decoders.jDOTuh1jhjEubj)}(h4Diagraph of CXL root, switch, and endpoint decoders.h]h4Diagraph of CXL root, switch, and endpoint decoders.}(hjahhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjEubeh}(h]id3ah ]h"]h$]h&]altj_captionjcuh1jhjhhhhhNubeh}(h]decoder-relationshipsah ]h"]decoder relationshipsah$]h&]uh1hhjhhhhhMyubeh}(h]decodersah ]h"]decodersah$]h&]uh1hhjhhhhhKubh)}(hhh](h)}(hRegionsh]hRegions}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(h Memory Regionh]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hA `Memory Region` is a logical construct that connects a set of CXL ports in the fabric to an IO Memory Resource. It is ultimately used to expose the memory on these devices to the DAX subsystem via a `DAX Region`.h](hA }(hjhhhNhNubjH)}(h`Memory Region`h]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh is a logical construct that connects a set of CXL ports in the fabric to an IO Memory Resource. It is ultimately used to expose the memory on these devices to the DAX subsystem via a }(hjhhhNhNubjH)}(h `DAX Region`h]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hAn example RAM region: ::h]hAn example RAM region:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hX # ls /sys/bus/cxl/devices/region0/ access0 devtype modalias subsystem uuid access1 driver mode target0 commit interleave_granularity resource target1 dax_region0 interleave_ways size ueventh]hX # ls /sys/bus/cxl/devices/region0/ access0 devtype modalias subsystem uuid access1 driver mode target0 commit interleave_granularity resource target1 dax_region0 interleave_ways size uevent}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubh)}(hA memory region can be constructed during endpoint probe, if decoders were programmed by BIOS/EFI (see `Auto Decoders`), or by creating a region manually via a `Root Decoder`'s :code:`create_ram_region` or :code:`create_pmem_region` interfaces.h](hgA memory region can be constructed during endpoint probe, if decoders were programmed by BIOS/EFI (see }(hjhhhNhNubjH)}(h`Auto Decoders`h]h Auto Decoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh*), or by creating a region manually via a }(hjhhhNhNubjH)}(h`Root Decoder`h]h Root Decoder}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh’s }(hjhhhNhNubh)}(h:code:`create_ram_region`h]hcreate_ram_region}(hj$hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh or }(hjhhhNhNubh)}(h:code:`create_pmem_region`h]hcreate_pmem_region}(hj7hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh interfaces.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe interleave settings in a `Memory Region` describe the configuration of the `Interleave Set` - and are what can be expected to be seen in the endpoint interleave settings.h](hThe interleave settings in a }(hjPhhhNhNubjH)}(h`Memory Region`h]h Memory Region}(hjXhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjPubh# describe the configuration of the }(hjPhhhNhNubjH)}(h`Interleave Set`h]hInterleave Set}(hjjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjPubhO - and are what can be expected to be seen in the endpoint interleave settings.}(hjPhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj)}(hhh](j)}(hhh]h)}(hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "region0"; "region0" -> "decoder5.0"; "region0" -> "decoder6.0"; }h]hdigraph foo { "root0" -> "decoder0.0"; "decoder0.0" -> "region0"; "region0" -> "decoder5.0"; "region0" -> "decoder6.0"; }}hjsbah}(h]h ]h"]h$]h&]hhuh1hhjhhubah}(h]h ]h"]h$]h&]jNDigraph of CXL memory region relationships between root and endpoint decoders.jDOTuh1jhjubj)}(hRegions are created based on root decoder configurations. Endpoint decoders must be programmed with the same interleave settings as the region.h]hRegions are created based on root decoder configurations. Endpoint decoders must be programmed with the same interleave settings as the region.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubeh}(h]id4ah ]h"]h$]h&]altjcaptionjuh1jhjhhhhhNubeh}(h] memory-regionah ]h"] memory regionah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(h DAX Regionh]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hA `DAX Region` is used to convert a CXL `Memory Region` to a DAX device. A DAX device may then be accessed directly via a file descriptor interface, or converted to System RAM via the DAX kmem driver. See the DAX driver section for more details. ::h](hA }(hjhhhNhNubjH)}(h `DAX Region`h]h DAX Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh is used to convert a CXL }(hjhhhNhNubjH)}(h`Memory Region`h]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh to a DAX device. A DAX device may then be accessed directly via a file descriptor interface, or converted to System RAM via the DAX kmem driver. See the DAX driver section for more details.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hp# ls /sys/bus/cxl/devices/dax_region0/ dax0.0 devtype modalias uevent dax_region driver subsystemh]hp# ls /sys/bus/cxl/devices/dax_region0/ dax0.0 devtype modalias uevent dax_region driver subsystem}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubeh}(h] dax-regionah ]h"] dax regionah$]h&]uh1hhjhhhhhMubeh}(h]regionsah ]h"]regionsah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(hMailbox Interfacesh]hMailbox Interfaces}(hj!hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(h`. The CEDT may actually define multiple CFMWS configurations to describe the same physical capacity, with the intent to allow users to decide at runtime whether to online memory as interleaved or non-interleaved. ::h](h9Root decoder interleave is defined by CFMWS field of the }(hjhhhNhNubh)}(h#:doc:`CEDT <../platform/acpi/cedt>`h]j )}(hjh]hCEDT}(hjhhhNhNubah}(h]h ](jstdstd-doceh"]h$]h&]uh1j hjubah}(h]h ]h"]h$]h&]refdocj$ refdomainjreftypedoc refexplicitrefwarnj*../platform/acpi/cedtuh1hhhhMhjubh. The CEDT may actually define multiple CFMWS configurations to describe the same physical capacity, with the intent to allow users to decide at runtime whether to online memory as interleaved or non-interleaved.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hX Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000100000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000007 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000200000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000006 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000300000000 Window size : 0000000200000000 Interleave Members (2^n) : 01 Interleave Arithmetic : 00 First Target : 00000007 Next Target : 00000006h]hX Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000100000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000007 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000200000000 Window size : 0000000100000000 Interleave Members (2^n) : 00 Interleave Arithmetic : 00 First Target : 00000006 Subtable Type : 01 [CXL Fixed Memory Window Structure] Window base address : 0000000300000000 Window size : 0000000200000000 Interleave Members (2^n) : 01 Interleave Arithmetic : 00 First Target : 00000007 Next Target : 00000006}hjsbah}(h]h ]h"]h$]h&]hhuh1hhhhMhjhhubh)}(hIn this example, the CFMWS defines two discrete non-interleaved 4GB regions for each host bridge, and one interleaved 8GB region that targets both. This would result in 3 root decoders presenting in the root. ::h]hIn this example, the CFMWS defines two discrete non-interleaved 4GB regions for each host bridge, and one interleaved 8GB region that targets both. This would result in 3 root decoders presenting in the root.}(hj"hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM4hjhhubh)}(hXi# ls /sys/bus/cxl/devices/root0/decoder* decoder0.0 decoder0.1 decoder0.2 # cat /sys/bus/cxl/devices/decoder0.0/target_list start size 7 0x100000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.1/target_list start size 6 0x200000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.2/target_list start size 7,6 0x300000000 0x200000000h]hXi# ls /sys/bus/cxl/devices/root0/decoder* decoder0.0 decoder0.1 decoder0.2 # cat /sys/bus/cxl/devices/decoder0.0/target_list start size 7 0x100000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.1/target_list start size 6 0x200000000 0x100000000 # cat /sys/bus/cxl/devices/decoder0.2/target_list start size 7,6 0x300000000 0x200000000}hj0sbah}(h]h ]h"]h$]h&]hhuh1hhhhM8hjhhubh)}(hThese decoders are not runtime programmable. They are used to generate a `Memory Region` to bring this memory online with runtime programmed settings at the `Switch` and `Endpoint` decoders.h](hJThese decoders are not runtime programmable. They are used to generate a }(hj>hhhNhNubjH)}(h`Memory Region`h]h Memory Region}(hjFhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj>ubhE to bring this memory online with runtime programmed settings at the }(hj>hhhNhNubjH)}(h`Switch`h]hSwitch}(hjXhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj>ubh and }(hj>hhhNhNubjH)}(h `Endpoint`h]hEndpoint}(hjjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhj>ubh decoders.}(hj>hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMJhjhhubeh}(h]at-rootah ]h"]at rootah$]h&]uh1hhjhhhhhMubh)}(hhh](h)}(hAt Host Bridge or Switchh]hAt Host Bridge or Switch}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMOubh)}(hN`Host Bridge` and `Switch` decoders are programmable via the following fields:h](jH)}(h `Host Bridge`h]h Host Bridge}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh and }(hjhhhNhNubjH)}(h`Switch`h]hSwitch}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh4 decoders are programmable via the following fields:}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMPhjhhubj-)}(hhh](j2)}(h@:code:`start` - the HPA region associated with the memory regionh]h)}(hjh](h)}(h :code:`start`h]hstart}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh3 - the HPA region associated with the memory region}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMRhjubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(h%:code:`size` - the size of the regionh]h)}(hjh](h)}(h :code:`size`h]hsize}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh - the size of the region}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMShjubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(h2:code:`target_list` - the list of downstream portsh]h)}(hjh](h)}(h:code:`target_list`h]h target_list}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh - the list of downstream ports}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMThjubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(hJ:code:`interleave_ways` - the number downstream ports to interleave acrossh]h)}(hj@h](h)}(h:code:`interleave_ways`h]hinterleave_ways}(hjEhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjBubh3 - the number downstream ports to interleave across}(hjBhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMUhj>ubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubj2)}(hC:code:`interleave_granularity` - the granularity to interleave at. h]h)}(hB:code:`interleave_granularity` - the granularity to interleave at.h](h)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjlhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjhubh$ - the granularity to interleave at.}(hjhhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMVhjdubah}(h]h ]h"]h$]h&]uh1j1hjhhhhhNubeh}(h]h ]h"]h$]h&]j-uh1j,hhhMRhjhhubh)}(hX$Linux expects the :code:`interleave_granularity` of switch decoders to be derived from their upstream port connections. In `Cross-Link First` interleave configurations, the :code:`interleave_granularity` of a decoder is equal to :code:`parent_interleave_granularity * parent_interleave_ways`.h](hLinux expects the }(hjhhhNhNubh)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubhK of switch decoders to be derived from their upstream port connections. In }(hjhhhNhNubjH)}(h`Cross-Link First`h]hCross-Link First}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh interleave configurations, the }(hjhhhNhNubh)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh of a decoder is equal to }(hjhhhNhNubh)}(h>:code:`parent_interleave_granularity * parent_interleave_ways`h]h6parent_interleave_granularity * parent_interleave_ways}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMXhjhhubeh}(h]at-host-bridge-or-switchah ]h"]at host bridge or switchah$]h&]uh1hhjhhhhhMOubh)}(hhh](h)}(h At Endpointh]h At Endpoint}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM^ubh)}(h`Endpoint Decoders` are programmed similar to Host Bridge and Switch decoders, with the exception that the ways and granularity are defined by the interleave set (e.g. the interleave settings defined by the associated `Memory Region`).h](jH)}(h`Endpoint Decoders`h]hEndpoint Decoders}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh are programmed similar to Host Bridge and Switch decoders, with the exception that the ways and granularity are defined by the interleave set (e.g. the interleave settings defined by the associated }(hjhhhNhNubjH)}(h`Memory Region`h]h Memory Region}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jGhjubh).}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM_hjhhubj-)}(hhh](j2)}(h@:code:`start` - the HPA region associated with the memory regionh]h)}(hj7h](h)}(h :code:`start`h]hstart}(hj<hhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj9ubh3 - the HPA region associated with the memory region}(hj9hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMchj5ubah}(h]h ]h"]h$]h&]uh1j1hj2hhhhhNubj2)}(h%:code:`size` - the size of the regionh]h)}(hj]h](h)}(h :code:`size`h]hsize}(hjbhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhj_ubh - the size of the region}(hj_hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMdhj[ubah}(h]h ]h"]h$]h&]uh1j1hj2hhhhhNubj2)}(hD:code:`interleave_ways` - the number endpoints in the interleave seth]h)}(hjh](h)}(h:code:`interleave_ways`h]hinterleave_ways}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh- - the number endpoints in the interleave set}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMehjubah}(h]h ]h"]h$]h&]uh1j1hj2hhhhhNubj2)}(hC:code:`interleave_granularity` - the granularity to interleave at. h]h)}(hB:code:`interleave_granularity` - the granularity to interleave at.h](h)}(h:code:`interleave_granularity`h]hinterleave_granularity}(hjhhhNhNubah}(h]h ]hah"]h$]h&]languagehuh1hhjubh$ - the granularity to interleave at.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMfhjubah}(h]h ]h"]h$]h&]uh1j1hj2hhhhhNubeh}(h]h ]h"]h$]h&]jjuh1j,hhhMchjhhubh)}(hThese settings are used by endpoint decoders to *Translate* memory requests from HPA to DPA. This is why they must be aware of the entire interleave set.h](h0These settings are used by endpoint decoders to }(hjhhhNhNubj )}(h *Translate*h]h Translate}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j hjubh_ memory requests from HPA to DPA. This is why they must be aware of the entire interleave set.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhhjhhubh)}(hLinux does not support unbalanced interleave configurations. As a result, all endpoints in an interleave set must have the same ways and granularity.h]hLinux does not support unbalanced interleave configurations. As a result, all endpoints in an interleave set must have the same ways and granularity.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMkhjhhubeh}(h] at-endpointah ]h"] at endpointah$]h&]uh1hhjhhhhhM^ubeh}(h] interleaveah ]h"] interleaveah$]h&]uh1hhjhhhhhMubeh}(h]decoder-programmingah ]h"]decoder programmingah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hExample Configurationsh]hExample Configurations}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMoubhcompound)}(hhh]htoctree)}(hhh]h}(h]h ]h"]h$]h&]hj$entries](N9driver-api/cxl/linux/example-configurations/single-deviceN9driver-api/cxl/linux/example-configurations/hb-interleaveN?driver-api/cxl/linux/example-configurations/intra-hb-interleaveN