€•¯YŒsphinx.addnodes”Œdocument”“”)”}”(Œ rawsource”Œ”Œchildren”]”(Œ translations”Œ LanguagesNode”“”)”}”(hhh]”(hŒ pending_xref”“”)”}”(hhh]”Œdocutils.nodes”ŒText”“”ŒChinese (Simplified)”…””}”Œparent”hsbaŒ attributes”}”(Œids”]”Œclasses”]”Œnames”]”Œdupnames”]”Œbackrefs”]”Œ refdomain”Œstd”Œreftype”Œdoc”Œ reftarget”Œ+/translations/zh_CN/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuŒtagname”hhh ubh)”}”(hhh]”hŒChinese (Traditional)”…””}”hh2sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/zh_TW/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒItalian”…””}”hhFsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/it_IT/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒJapanese”…””}”hhZsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ja_JP/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒKorean”…””}”hhnsbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/ko_KR/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubh)”}”(hhh]”hŒSpanish”…””}”hh‚sbah}”(h]”h ]”h"]”h$]”h&]”Œ refdomain”h)Œreftype”h+Œ reftarget”Œ+/translations/sp_SP/admin-guide/mm/numaperf”Œmodname”NŒ classname”NŒ refexplicit”ˆuh1hhh ubeh}”(h]”h ]”h"]”h$]”h&]”Œcurrent_language”ŒEnglish”uh1h hhŒ _document”hŒsource”NŒline”NubhŒsection”“”)”}”(hhh]”(hŒtitle”“”)”}”(hŒNUMA Memory Performance”h]”hŒNUMA Memory Performance”…””}”(hh¨hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¦hh£hžhhŸŒE/var/lib/git/docbuild/linux/Documentation/admin-guide/mm/numaperf.rst”h Kubh¢)”}”(hhh]”(h§)”}”(hŒ NUMA Locality”h]”hŒ NUMA Locality”…””}”(hhºhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¦hh·hžhhŸh¶h KubhŒ paragraph”“”)”}”(hXSome platforms may have multiple types of memory attached to a compute node. These disparate memory ranges may share some characteristics, such as CPU cache coherence, but may have different performance. For example, different media types and buses affect bandwidth and latency.”h]”hXSome platforms may have multiple types of memory attached to a compute node. These disparate memory ranges may share some characteristics, such as CPU cache coherence, but may have different performance. For example, different media types and buses affect bandwidth and latency.”…””}”(hhÊhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h Khh·hžhubhÉ)”}”(hXþA system supports such heterogeneous memory by grouping each memory type under different domains, or "nodes", based on locality and performance characteristics. Some memory may share the same node as a CPU, and others are provided as memory only nodes. While memory only nodes do not provide CPUs, they may still be local to one or more compute nodes relative to other nodes. The following diagram shows one such example of two compute nodes with local memory and a memory only node for each of compute node::”h]”hXA system supports such heterogeneous memory by grouping each memory type under different domains, or “nodesâ€, based on locality and performance characteristics. Some memory may share the same node as a CPU, and others are provided as memory only nodes. While memory only nodes do not provide CPUs, they may still be local to one or more compute nodes relative to other nodes. The following diagram shows one such example of two compute nodes with local memory and a memory only node for each of compute node:”…””}”(hhØhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K hh·hžhubhŒ literal_block”“”)”}”(hXe+------------------+ +------------------+ | Compute Node 0 +-----+ Compute Node 1 | | Local Node0 Mem | | Local Node1 Mem | +--------+---------+ +--------+---------+ | | +--------+---------+ +--------+---------+ | Slower Node2 Mem | | Slower Node3 Mem | +------------------+ +--------+---------+”h]”hXe+------------------+ +------------------+ | Compute Node 0 +-----+ Compute Node 1 | | Local Node0 Mem | | Local Node1 Mem | +--------+---------+ +--------+---------+ | | +--------+---------+ +--------+---------+ | Slower Node2 Mem | | Slower Node3 Mem | +------------------+ +--------+---------+”…””}”hhèsbah}”(h]”h ]”h"]”h$]”h&]”Œ xml:space”Œpreserve”uh1hæhŸh¶h Khh·hžhubhÉ)”}”(hXA "memory initiator" is a node containing one or more devices such as CPUs or separate memory I/O devices that can initiate memory requests. A "memory target" is a node containing one or more physical address ranges accessible from one or more memory initiators.”h]”hXA “memory initiator†is a node containing one or more devices such as CPUs or separate memory I/O devices that can initiate memory requests. A “memory target†is a node containing one or more physical address ranges accessible from one or more memory initiators.”…””}”(hhøhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h Khh·hžhubhÉ)”}”(hXõWhen multiple memory initiators exist, they may not all have the same performance when accessing a given memory target. Each initiator-target pair may be organized into different ranked access classes to represent this relationship. The highest performing initiator to a given target is considered to be one of that target's local initiators, and given the highest access class, 0. Any given target may have one or more local initiators, and any given initiator may have multiple local memory targets.”h]”hX÷When multiple memory initiators exist, they may not all have the same performance when accessing a given memory target. Each initiator-target pair may be organized into different ranked access classes to represent this relationship. The highest performing initiator to a given target is considered to be one of that target’s local initiators, and given the highest access class, 0. Any given target may have one or more local initiators, and any given initiator may have multiple local memory targets.”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K#hh·hžhubhÉ)”}”(hŒÔTo aid applications matching memory targets with their initiators, the kernel provides symlinks to each other. The following example lists the relationship for the access class "0" memory initiators and targets::”h]”hŒ×To aid applications matching memory targets with their initiators, the kernel provides symlinks to each other. The following example lists the relationship for the access class “0†memory initiators and targets:”…””}”(hjhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K,hh·hžhubhç)”}”(hX# symlinks -v /sys/devices/system/node/nodeX/access0/targets/ relative: /sys/devices/system/node/nodeX/access0/targets/nodeY -> ../../nodeY # symlinks -v /sys/devices/system/node/nodeY/access0/initiators/ relative: /sys/devices/system/node/nodeY/access0/initiators/nodeX -> ../../nodeX”h]”hX# symlinks -v /sys/devices/system/node/nodeX/access0/targets/ relative: /sys/devices/system/node/nodeX/access0/targets/nodeY -> ../../nodeY # symlinks -v /sys/devices/system/node/nodeY/access0/initiators/ relative: /sys/devices/system/node/nodeY/access0/initiators/nodeX -> ../../nodeX”…””}”hj"sbah}”(h]”h ]”h"]”h$]”h&]”höh÷uh1hæhŸh¶h K0hh·hžhubhÉ)”}”(hXXA memory initiator may have multiple memory targets in the same access class. The target memory's initiators in a given class indicate the nodes' access characteristics share the same performance relative to other linked initiator nodes. Each target within an initiator's access class, though, do not necessarily perform the same as each other.”h]”hX^A memory initiator may have multiple memory targets in the same access class. The target memory’s initiators in a given class indicate the nodes’ access characteristics share the same performance relative to other linked initiator nodes. Each target within an initiator’s access class, though, do not necessarily perform the same as each other.”…””}”(hj0hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K6hh·hžhubhÉ)”}”(hŒòThe access class "1" is used to allow differentiation between initiators that are CPUs and hence suitable for generic task scheduling, and IO initiators such as GPUs and NICs. Unlike access class 0, only nodes containing CPUs are considered.”h]”hŒöThe access class “1†is used to allow differentiation between initiators that are CPUs and hence suitable for generic task scheduling, and IO initiators such as GPUs and NICs. Unlike access class 0, only nodes containing CPUs are considered.”…””}”(hj>hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h KThe memory-side caches are not directly addressable by software. When software accesses a system address, the system will return it from the near memory cache if it is present. If it is not present, the system accesses the next level of memory until there is either a hit in that cache level, or it reaches far memory.”h]”hX>The memory-side caches are not directly addressable by software. When software accesses a system address, the system will return it from the near memory cache if it is present. If it is not present, the system accesses the next level of memory until there is either a hit in that cache level, or it reaches far memory.”…””}”(hj&hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h KyhjëhžhubhÉ)”}”(hX¢An application does not need to know about caching attributes in order to use the system. Software may optionally query the memory cache attributes in order to maximize the performance out of such a setup. If the system provides a way for the kernel to discover this information, for example with ACPI HMAT (Heterogeneous Memory Attribute Table), the kernel will append these attributes to the NUMA node memory target.”h]”hX¢An application does not need to know about caching attributes in order to use the system. Software may optionally query the memory cache attributes in order to maximize the performance out of such a setup. If the system provides a way for the kernel to discover this information, for example with ACPI HMAT (Heterogeneous Memory Attribute Table), the kernel will append these attributes to the NUMA node memory target.”…””}”(hj4hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h KhjëhžhubhÉ)”}”(hŒlWhen the kernel first registers a memory cache with a node, the kernel will create the following directory::”h]”hŒkWhen the kernel first registers a memory cache with a node, the kernel will create the following directory:”…””}”(hjBhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K†hjëhžhubhç)”}”(hŒ1/sys/devices/system/node/nodeX/memory_side_cache/”h]”hŒ1/sys/devices/system/node/nodeX/memory_side_cache/”…””}”hjPsbah}”(h]”h ]”h"]”h$]”h&]”höh÷uh1hæhŸh¶h K‰hjëhžhubhÉ)”}”(hŒŽIf that directory is not present, the system either does not provide a memory-side cache, or that information is not accessible to the kernel.”h]”hŒŽIf that directory is not present, the system either does not provide a memory-side cache, or that information is not accessible to the kernel.”…””}”(hj^hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K‹hjëhžhubhÉ)”}”(hŒPThe attributes for each level of cache is provided under its cache level index::”h]”hŒOThe attributes for each level of cache is provided under its cache level index:”…””}”(hjlhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h KŽhjëhžhubhç)”}”(hŒª/sys/devices/system/node/nodeX/memory_side_cache/indexA/ /sys/devices/system/node/nodeX/memory_side_cache/indexB/ /sys/devices/system/node/nodeX/memory_side_cache/indexC/”h]”hŒª/sys/devices/system/node/nodeX/memory_side_cache/indexA/ /sys/devices/system/node/nodeX/memory_side_cache/indexB/ /sys/devices/system/node/nodeX/memory_side_cache/indexC/”…””}”hjzsbah}”(h]”h ]”h"]”h$]”h&]”höh÷uh1hæhŸh¶h K‘hjëhžhubhÉ)”}”(hŒ Each cache level's directory provides its attributes. For example, the following shows a single cache level and the attributes available for software to query::”h]”hŒ¡Each cache level’s directory provides its attributes. For example, the following shows a single cache level and the attributes available for software to query:”…””}”(hjˆhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K•hjëhžhubhç)”}”(hŒº# tree /sys/devices/system/node/node0/memory_side_cache/ /sys/devices/system/node/node0/memory_side_cache/ |-- index1 | |-- indexing | |-- line_size | |-- size | `-- write_policy”h]”hŒº# tree /sys/devices/system/node/node0/memory_side_cache/ /sys/devices/system/node/node0/memory_side_cache/ |-- index1 | |-- indexing | |-- line_size | |-- size | `-- write_policy”…””}”hj–sbah}”(h]”h ]”h"]”h$]”h&]”höh÷uh1hæhŸh¶h K™hjëhžhubhÉ)”}”(hŒ{The "indexing" will be 0 if it is a direct-mapped cache, and non-zero for any other indexed based, multi-way associativity.”h]”hŒThe “indexing†will be 0 if it is a direct-mapped cache, and non-zero for any other indexed based, multi-way associativity.”…””}”(hj¤hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K¡hjëhžhubhÉ)”}”(hŒTThe "line_size" is the number of bytes accessed from the next cache level on a miss.”h]”hŒXThe “line_size†is the number of bytes accessed from the next cache level on a miss.”…””}”(hj²hžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K¤hjëhžhubhÉ)”}”(hŒ?The "size" is the number of bytes provided by this cache level.”h]”hŒCThe “size†is the number of bytes provided by this cache level.”…””}”(hjÀhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K§hjëhžhubhÉ)”}”(hŒTThe "write_policy" will be 0 for write-back, and non-zero for write-through caching.”h]”hŒXThe “write_policy†will be 0 for write-back, and non-zero for write-through caching.”…””}”(hjÎhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K©hjëhžhubeh}”(h]”Œ numa-cache”ah ]”h"]”Œ numa cache”ah$]”h&]”uh1h¡hh£hžhhŸh¶h Kdubh¢)”}”(hhh]”(h§)”}”(hŒSee Also”h]”hŒSee Also”…””}”(hjçhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”uh1h¦hjähžhhŸh¶h K­ubhÉ)”}”(hŒT[1] https://www.uefi.org/sites/default/files/resources/ACPI_6_2.pdf - Section 5.2.27”h]”(hŒ[1] ”…””}”(hjõhžhhŸNh NubhŒ reference”“”)”}”(hŒ?https://www.uefi.org/sites/default/files/resources/ACPI_6_2.pdf”h]”hŒ?https://www.uefi.org/sites/default/files/resources/ACPI_6_2.pdf”…””}”(hjÿhžhhŸNh Nubah}”(h]”h ]”h"]”h$]”h&]”Œrefuri”juh1jýhjõubhŒ - Section 5.2.27”…””}”(hjõhžhhŸNh Nubeh}”(h]”h ]”h"]”h$]”h&]”uh1hÈhŸh¶h K¯hjähžhubeh}”(h]”Œsee-also”ah ]”h"]”Œsee also”ah$]”h&]”uh1h¡hh£hžhhŸh¶h K­ubeh}”(h]”Œnuma-memory-performance”ah ]”h"]”Œnuma memory performance”ah$]”h&]”uh1h¡hhhžhhŸh¶h Kubeh}”(h]”h ]”h"]”h$]”h&]”Œsource”h¶uh1hŒcurrent_source”NŒ current_line”NŒsettings”Œdocutils.frontend”ŒValues”“”)”}”(h¦NŒ generator”NŒ datestamp”NŒ source_link”NŒ source_url”NŒ toc_backlinks”Œentry”Œfootnote_backlinks”KŒ sectnum_xform”KŒstrip_comments”NŒstrip_elements_with_classes”NŒ strip_classes”NŒ report_level”KŒ halt_level”KŒexit_status_level”KŒdebug”NŒwarning_stream”NŒ traceback”ˆŒinput_encoding”Œ utf-8-sig”Œinput_encoding_error_handler”Œstrict”Œoutput_encoding”Œutf-8”Œoutput_encoding_error_handler”jKŒerror_encoding”Œutf-8”Œerror_encoding_error_handler”Œbackslashreplace”Œ language_code”Œen”Œrecord_dependencies”NŒconfig”NŒ id_prefix”hŒauto_id_prefix”Œid”Œ dump_settings”NŒdump_internals”NŒdump_transforms”NŒdump_pseudo_xml”NŒexpose_internals”NŒstrict_visitor”NŒ_disable_config”NŒ_source”h¶Œ _destination”NŒ _config_files”]”Œ7/var/lib/git/docbuild/linux/Documentation/docutils.conf”aŒfile_insertion_enabled”ˆŒ raw_enabled”KŒline_length_limit”M'Œpep_references”NŒ pep_base_url”Œhttps://peps.python.org/”Œpep_file_url_template”Œpep-%04d”Œrfc_references”NŒ rfc_base_url”Œ&https://datatracker.ietf.org/doc/html/”Œ tab_width”KŒtrim_footnote_reference_space”‰Œsyntax_highlight”Œlong”Œ smart_quotes”ˆŒsmartquotes_locales”]”Œcharacter_level_inline_markup”‰Œdoctitle_xform”‰Œ docinfo_xform”KŒsectsubtitle_xform”‰Œ image_loading”Œlink”Œembed_stylesheet”‰Œcloak_email_addresses”ˆŒsection_self_link”‰Œenv”NubŒreporter”NŒindirect_targets”]”Œsubstitution_defs”}”Œsubstitution_names”}”Œrefnames”}”Œrefids”}”Œnameids”}”(j%j"jQjNjèjåjájÞjjuŒ nametypes”}”(j%‰jQ‰jè‰já‰j‰uh}”(j"h£jNh·jåjTjÞjëjjäuŒ footnote_refs”}”Œ citation_refs”}”Œ autofootnotes”]”Œautofootnote_refs”]”Œsymbol_footnotes”]”Œsymbol_footnote_refs”]”Œ footnotes”]”Œ citations”]”Œautofootnote_start”KŒsymbol_footnote_start”KŒ id_counter”Œ collections”ŒCounter”“”}”…”R”Œparse_messages”]”Œtransform_messages”]”Œ transformer”NŒ include_log”]”Œ decoration”Nhžhub.