sphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget6/translations/zh_CN/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/zh_TW/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/it_IT/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/ja_JP/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/ko_KR/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hPortuguese (Brazilian)}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/pt_BR/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget6/translations/sp_SP/arch/arm/cluster-pm-race-avoidancemodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhsection)}(hhh](htitle)}(h9Cluster-wide Power-up/power-down race avoidance algorithmh]h9Cluster-wide Power-up/power-down race avoidance algorithm}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhP/var/lib/git/docbuild/linux/Documentation/arch/arm/cluster-pm-race-avoidance.rsthKubh paragraph)}(hThis file documents the algorithm which is used to coordinate CPU and cluster setup and teardown operations and to manage hardware coherency controls safely.h]hThis file documents the algorithm which is used to coordinate CPU and cluster setup and teardown operations and to manage hardware coherency controls safely.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hThe section "Rationale" explains what the algorithm is for and why it is needed. "Basic model" explains general concepts using a simplified view of the system. The other sections explain the actual details of the algorithm in use.h]hThe section “Rationale” explains what the algorithm is for and why it is needed. “Basic model” explains general concepts using a simplified view of the system. The other sections explain the actual details of the algorithm in use.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hhh](h)}(h Rationaleh]h Rationale}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hIn a system containing multiple CPUs, it is desirable to have the ability to turn off individual CPUs when the system is idle, reducing power consumption and thermal dissipation.h]hIn a system containing multiple CPUs, it is desirable to have the ability to turn off individual CPUs when the system is idle, reducing power consumption and thermal dissipation.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hwIn a system containing multiple clusters of CPUs, it is also desirable to have the ability to turn off entire clusters.h]hwIn a system containing multiple clusters of CPUs, it is also desirable to have the ability to turn off entire clusters.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hXaTurning entire clusters off and on is a risky business, because it involves performing potentially destructive operations affecting a group of independently running CPUs, while the OS continues to run. This means that we need some coordination in order to ensure that critical cluster-level operations are only performed when it is truly safe to do so.h]hXaTurning entire clusters off and on is a risky business, because it involves performing potentially destructive operations affecting a group of independently running CPUs, while the OS continues to run. This means that we need some coordination in order to ensure that critical cluster-level operations are only performed when it is truly safe to do so.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hXSimple locking may not be sufficient to solve this problem, because mechanisms like Linux spinlocks may rely on coherency mechanisms which are not immediately enabled when a cluster powers up. Since enabling or disabling those mechanisms may itself be a non-atomic operation (such as writing some hardware registers and invalidating large caches), other methods of coordination are required in order to guarantee safe power-down and power-up at the cluster level.h]hXSimple locking may not be sufficient to solve this problem, because mechanisms like Linux spinlocks may rely on coherency mechanisms which are not immediately enabled when a cluster powers up. Since enabling or disabling those mechanisms may itself be a non-atomic operation (such as writing some hardware registers and invalidating large caches), other methods of coordination are required in order to guarantee safe power-down and power-up at the cluster level.}(hj$hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hThe mechanism presented in this document describes a coherent memory based protocol for performing the needed coordination. It aims to be as lightweight as possible, while providing the required safety properties.h]hThe mechanism presented in this document describes a coherent memory based protocol for performing the needed coordination. It aims to be as lightweight as possible, while providing the required safety properties.}(hj2hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK(hhhhubeh}(h] rationaleah ]h"] rationaleah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(h Basic modelh]h Basic model}(hjKhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjHhhhhhK.ubh)}(h5Each cluster and CPU is assigned a state, as follows:h]h5Each cluster and CPU is assigned a state, as follows:}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK0hjHhhubh block_quote)}(h%- DOWN - COMING_UP - UP - GOING_DOWN h]h bullet_list)}(hhh](h list_item)}(hDOWNh]h)}(hjvh]hDOWN}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK2hjtubah}(h]h ]h"]h$]h&]uh1jrhjoubjs)}(h COMING_UPh]h)}(hjh]h COMING_UP}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK3hjubah}(h]h ]h"]h$]h&]uh1jrhjoubjs)}(hUPh]h)}(hjh]hUP}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK4hjubah}(h]h ]h"]h$]h&]uh1jrhjoubjs)}(h GOING_DOWN h]h)}(h GOING_DOWNh]h GOING_DOWN}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK5hjubah}(h]h ]h"]h$]h&]uh1jrhjoubeh}(h]h ]h"]h$]h&]bullet-uh1jmhhhK2hjiubah}(h]h ]h"]h$]h&]uh1jghhhK2hjHhhubh literal_block)}(h +---------> UP ----------+ | v COMING_UP GOING_DOWN ^ | +--------- DOWN <--------+h]h +---------> UP ----------+ | v COMING_UP GOING_DOWN ^ | +--------- DOWN <--------+}hjsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1jhhhK9hjHhhubhdefinition_list)}(hhh](hdefinition_list_item)}(h~DOWN: The CPU or cluster is not coherent, and is either powered off or suspended, or is ready to be powered off or suspended. h](hterm)}(hDOWN:h]hDOWN:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKDhjubh definition)}(hhh]h)}(hwThe CPU or cluster is not coherent, and is either powered off or suspended, or is ready to be powered off or suspended.h]hwThe CPU or cluster is not coherent, and is either powered off or suspended, or is ready to be powered off or suspended.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKChjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKDhjubj)}(hCOMING_UP: The CPU or cluster has committed to moving to the UP state. It may be part way through the process of initialisation and enabling coherency. h](j)}(h COMING_UP:h]h COMING_UP:}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKIhj+ubj )}(hhh]h)}(hThe CPU or cluster has committed to moving to the UP state. It may be part way through the process of initialisation and enabling coherency.h]hThe CPU or cluster has committed to moving to the UP state. It may be part way through the process of initialisation and enabling coherency.}(hj@hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKGhj=ubah}(h]h ]h"]h$]h&]uh1j hj+ubeh}(h]h ]h"]h$]h&]uh1jhhhKIhjhhubj)}(hUP: The CPU or cluster is active and coherent at the hardware level. A CPU in this state is not necessarily being used actively by the kernel. h](j)}(hUP:h]hUP:}(hj^hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKNhjZubj )}(hhh]h)}(hThe CPU or cluster is active and coherent at the hardware level. A CPU in this state is not necessarily being used actively by the kernel.h]hThe CPU or cluster is active and coherent at the hardware level. A CPU in this state is not necessarily being used actively by the kernel.}(hjohhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKLhjlubah}(h]h ]h"]h$]h&]uh1j hjZubeh}(h]h ]h"]h$]h&]uh1jhhhKNhjhhubj)}(hGOING_DOWN: The CPU or cluster has committed to moving to the DOWN state. It may be part way through the process of teardown and coherency exit. h](j)}(h GOING_DOWN:h]h GOING_DOWN:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKThjubj )}(hhh]h)}(hThe CPU or cluster has committed to moving to the DOWN state. It may be part way through the process of teardown and coherency exit.h]hThe CPU or cluster has committed to moving to the DOWN state. It may be part way through the process of teardown and coherency exit.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKQhjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKThjhhubeh}(h]h ]h"]h$]h&]uh1jhjHhhhhhNubh)}(hEach CPU has one of these states assigned to it at any point in time. The CPU states are described in the "CPU state" section, below.h]hEach CPU has one of these states assigned to it at any point in time. The CPU states are described in the “CPU state” section, below.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKVhjHhhubh)}(hXZEach cluster is also assigned a state, but it is necessary to split the state value into two parts (the "cluster" state and "inbound" state) and to introduce additional states in order to avoid races between different CPUs in the cluster simultaneously modifying the state. The cluster- level states are described in the "Cluster state" section.h]hXfEach cluster is also assigned a state, but it is necessary to split the state value into two parts (the “cluster” state and “inbound” state) and to introduce additional states in order to avoid races between different CPUs in the cluster simultaneously modifying the state. The cluster- level states are described in the “Cluster state” section.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKYhjHhhubh)}(hTo help distinguish the CPU states from cluster states in this discussion, the state names are given a `CPU_` prefix for the CPU states, and a `CLUSTER_` or `INBOUND_` prefix for the cluster states.h](hgTo help distinguish the CPU states from cluster states in this discussion, the state names are given a }(hjhhhNhNubhtitle_reference)}(h`CPU_`h]hCPU_}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh" prefix for the CPU states, and a }(hjhhhNhNubj)}(h `CLUSTER_`h]hCLUSTER_}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh or }(hjhhhNhNubj)}(h `INBOUND_`h]hINBOUND_}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubh prefix for the cluster states.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhK_hjHhhubeh}(h] basic-modelah ]h"] basic modelah$]h&]uh1hhhhhhhhK.ubh)}(hhh](h)}(h CPU stateh]h CPU state}(hj+hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj(hhhhhKeubh)}(hIn this algorithm, each individual core in a multi-core processor is referred to as a "CPU". CPUs are assumed to be single-threaded: therefore, a CPU can only be doing one thing at a single point in time.h]hIn this algorithm, each individual core in a multi-core processor is referred to as a “CPU”. CPUs are assumed to be single-threaded: therefore, a CPU can only be doing one thing at a single point in time.}(hj9hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKghj(hhubh)}(h1This means that CPUs fit the basic model closely.h]h1This means that CPUs fit the basic model closely.}(hjGhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKkhj(hhubh)}(hFThe algorithm defines the following states for each CPU in the system:h]hFThe algorithm defines the following states for each CPU in the system:}(hjUhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKmhj(hhubjh)}(h5- CPU_DOWN - CPU_COMING_UP - CPU_UP - CPU_GOING_DOWN h]jn)}(hhh](js)}(hCPU_DOWNh]h)}(hjlh]hCPU_DOWN}(hjnhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKohjjubah}(h]h ]h"]h$]h&]uh1jrhjgubjs)}(h CPU_COMING_UPh]h)}(hjh]h CPU_COMING_UP}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKphjubah}(h]h ]h"]h$]h&]uh1jrhjgubjs)}(hCPU_UPh]h)}(hjh]hCPU_UP}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKqhjubah}(h]h ]h"]h$]h&]uh1jrhjgubjs)}(hCPU_GOING_DOWN h]h)}(hCPU_GOING_DOWNh]hCPU_GOING_DOWN}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKrhjubah}(h]h ]h"]h$]h&]uh1jrhjgubeh}(h]h ]h"]h$]h&]jjuh1jmhhhKohjcubah}(h]h ]h"]h$]h&]uh1jghhhKohj(hhubj)}(hXV cluster setup and CPU setup complete policy decision +-----------> CPU_UP ------------+ | v CPU_COMING_UP CPU_GOING_DOWN ^ | +----------- CPU_DOWN <----------+ policy decision CPU teardown complete or hardware eventh]hXV cluster setup and CPU setup complete policy decision +-----------> CPU_UP ------------+ | v CPU_COMING_UP CPU_GOING_DOWN ^ | +----------- CPU_DOWN <----------+ policy decision CPU teardown complete or hardware event}hjsbah}(h]h ]h"]h$]h&]jjuh1jhhhKvhj(hhubh)}(hWThe definitions of the four states correspond closely to the states of the basic model.h]hWThe definitions of the four states correspond closely to the states of the basic model.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj(hhubh)}(h,Transitions between states occur as follows.h]h,Transitions between states occur as follows.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj(hhubh)}(hA trigger event (spontaneous) means that the CPU can transition to the next state as a result of making local progress only, with no requirement for any external event to happen.h]hA trigger event (spontaneous) means that the CPU can transition to the next state as a result of making local progress only, with no requirement for any external event to happen.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj(hhubj)}(hhh](j)}(hXCPU_DOWN: A CPU reaches the CPU_DOWN state when it is ready for power-down. On reaching this state, the CPU will typically power itself down or suspend itself, via a WFI instruction or a firmware call. Next state: CPU_COMING_UP Conditions: none Trigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(h CPU_DOWN:h]h CPU_DOWN:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh](h)}(hA CPU reaches the CPU_DOWN state when it is ready for power-down. On reaching this state, the CPU will typically power itself down or suspend itself, via a WFI instruction or a firmware call.h]hA CPU reaches the CPU_DOWN state when it is ready for power-down. On reaching this state, the CPU will typically power itself down or suspend itself, via a WFI instruction or a firmware call.}(hj#hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj ubj)}(hhh](j)}(hNext state: CPU_COMING_UPh](j)}(h Next state:h]h Next state:}(hj8hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhj4ubj )}(hhh]h)}(h CPU_COMING_UPh]h CPU_COMING_UP}(hjIhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjFubah}(h]h ]h"]h$]h&]uh1j hj4ubeh}(h]h ]h"]h$]h&]uh1jhhhKhj1ubj)}(hConditions: none h](j)}(h Conditions:h]h Conditions:}(hjghhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjcubj )}(hhh]h)}(hnoneh]hnone}(hjxhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjuubah}(h]h ]h"]h$]h&]uh1j hjcubeh}(h]h ]h"]h$]h&]uh1jhhhKhj1ubj)}(hTrigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(hTrigger events:h]hTrigger events:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh]henumerated_list)}(hhh](js)}(hZan explicit hardware power-up operation, resulting from a policy decision on another CPU; h]h)}(hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;h]hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jrhjubjs)}(h)a hardware event, such as an interrupt. h]h)}(h'a hardware event, such as an interrupt.h]h'a hardware event, such as an interrupt.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jrhjubeh}(h]h ]h"]h$]h&]enumtype loweralphaprefixhsuffix)uh1jhjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj1ubeh}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj ubj)}(hXCPU_COMING_UP: A CPU cannot start participating in hardware coherency until the cluster is set up and coherent. If the cluster is not ready, then the CPU will wait in the CPU_COMING_UP state until the cluster has been set up. Next state: CPU_UP Conditions: The CPU's parent cluster must be in CLUSTER_UP. Trigger events: Transition of the parent cluster to CLUSTER_UP. Refer to the "Cluster state" section for a description of the CLUSTER_UP state. h](j)}(hCPU_COMING_UP:h]hCPU_COMING_UP:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh](h)}(hA CPU cannot start participating in hardware coherency until the cluster is set up and coherent. If the cluster is not ready, then the CPU will wait in the CPU_COMING_UP state until the cluster has been set up.h]hA CPU cannot start participating in hardware coherency until the cluster is set up and coherent. If the cluster is not ready, then the CPU will wait in the CPU_COMING_UP state until the cluster has been set up.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubj)}(hhh](j)}(hNext state: CPU_UPh](j)}(h Next state:h]h Next state:}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhj+ubj )}(hhh]h)}(hCPU_UPh]hCPU_UP}(hj@hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj=ubah}(h]h ]h"]h$]h&]uh1j hj+ubeh}(h]h ]h"]h$]h&]uh1jhhhKhj(ubj)}(h;Conditions: The CPU's parent cluster must be in CLUSTER_UP.h](j)}(h Conditions:h]h Conditions:}(hj^hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjZubj )}(hhh]h)}(h/The CPU's parent cluster must be in CLUSTER_UP.h]h1The CPU’s parent cluster must be in CLUSTER_UP.}(hjohhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjlubah}(h]h ]h"]h$]h&]uh1j hjZubeh}(h]h ]h"]h$]h&]uh1jhhhKhj(ubj)}(h@Trigger events: Transition of the parent cluster to CLUSTER_UP. h](j)}(hTrigger events:h]hTrigger events:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh]h)}(h/Transition of the parent cluster to CLUSTER_UP.h]h/Transition of the parent cluster to CLUSTER_UP.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj(ubeh}(h]h ]h"]h$]h&]uh1jhjubh)}(hORefer to the "Cluster state" section for a description of the CLUSTER_UP state.h]hSRefer to the “Cluster state” section for a description of the CLUSTER_UP state.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubeh}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj hhubj)}(hXCPU_UP: When a CPU reaches the CPU_UP state, it is safe for the CPU to start participating in local coherency. This is done by jumping to the kernel's CPU resume code. Note that the definition of this state is slightly different from the basic model definition: CPU_UP does not mean that the CPU is coherent yet, but it does mean that it is safe to resume the kernel. The kernel handles the rest of the resume procedure, so the remaining steps are not visible as part of the race avoidance algorithm. The CPU remains in this state until an explicit policy decision is made to shut down or suspend the CPU. Next state: CPU_GOING_DOWN Conditions: none Trigger events: explicit policy decision h](j)}(hCPU_UP:h]hCPU_UP:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh](h)}(hfWhen a CPU reaches the CPU_UP state, it is safe for the CPU to start participating in local coherency.h]hfWhen a CPU reaches the CPU_UP state, it is safe for the CPU to start participating in local coherency.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(h8This is done by jumping to the kernel's CPU resume code.h]h:This is done by jumping to the kernel’s CPU resume code.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hXMNote that the definition of this state is slightly different from the basic model definition: CPU_UP does not mean that the CPU is coherent yet, but it does mean that it is safe to resume the kernel. The kernel handles the rest of the resume procedure, so the remaining steps are not visible as part of the race avoidance algorithm.h]hXMNote that the definition of this state is slightly different from the basic model definition: CPU_UP does not mean that the CPU is coherent yet, but it does mean that it is safe to resume the kernel. The kernel handles the rest of the resume procedure, so the remaining steps are not visible as part of the race avoidance algorithm.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubh)}(hhThe CPU remains in this state until an explicit policy decision is made to shut down or suspend the CPU.h]hhThe CPU remains in this state until an explicit policy decision is made to shut down or suspend the CPU.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubj)}(hhh](j)}(hNext state: CPU_GOING_DOWNh](j)}(h Next state:h]h Next state:}(hj,hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhj(ubj )}(hhh]h)}(hCPU_GOING_DOWNh]hCPU_GOING_DOWN}(hj=hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj:ubah}(h]h ]h"]h$]h&]uh1j hj(ubeh}(h]h ]h"]h$]h&]uh1jhhhKhj%ubj)}(hConditions: noneh](j)}(h Conditions:h]h Conditions:}(hj[hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjWubj )}(hhh]h)}(hnoneh]hnone}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjiubah}(h]h ]h"]h$]h&]uh1j hjWubeh}(h]h ]h"]h$]h&]uh1jhhhKhj%ubj)}(h*Trigger events: explicit policy decision h](j)}(hTrigger events:h]hTrigger events:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh]h)}(hexplicit policy decisionh]hexplicit policy decision}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj%ubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj hhubj)}(hXCPU_GOING_DOWN: While in this state, the CPU exits coherency, including any operations required to achieve this (such as cleaning data caches). Next state: CPU_DOWN Conditions: local CPU teardown complete Trigger events: (spontaneous) h](j)}(hCPU_GOING_DOWN:h]hCPU_GOING_DOWN:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh](h)}(hWhile in this state, the CPU exits coherency, including any operations required to achieve this (such as cleaning data caches).h]hWhile in this state, the CPU exits coherency, including any operations required to achieve this (such as cleaning data caches).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubj)}(hhh](j)}(hNext state: CPU_DOWNh](j)}(h Next state:h]h Next state:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh]h)}(hCPU_DOWNh]hCPU_DOWN}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhjubj)}(h'Conditions: local CPU teardown completeh](j)}(h Conditions:h]h Conditions:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjubj )}(hhh]h)}(hlocal CPU teardown completeh]hlocal CPU teardown complete}(hj1hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj.ubah}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhjubj)}(hTrigger events: (spontaneous) h](j)}(hTrigger events:h]hTrigger events:}(hjOhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKhjKubj )}(hhh]h)}(h (spontaneous)h]h (spontaneous)}(hj`hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj]ubah}(h]h ]h"]h$]h&]uh1j hjKubeh}(h]h ]h"]h$]h&]uh1jhhhKhjubeh}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1j hjubeh}(h]h ]h"]h$]h&]uh1jhhhKhj hhubeh}(h]h ]h"]h$]h&]uh1jhj(hhhNhNubeh}(h] cpu-stateah ]h"] cpu stateah$]h&]uh1hhhhhhhhKeubh)}(hhh](h)}(h Cluster stateh]h Cluster state}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hX A cluster is a group of connected CPUs with some common resources. Because a cluster contains multiple CPUs, it can be doing multiple things at the same time. This has some implications. In particular, a CPU can start up while another CPU is tearing the cluster down.h]hX A cluster is a group of connected CPUs with some common resources. Because a cluster contains multiple CPUs, it can be doing multiple things at the same time. This has some implications. In particular, a CPU can start up while another CPU is tearing the cluster down.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hIn this discussion, the "outbound side" is the view of the cluster state as seen by a CPU tearing the cluster down. The "inbound side" is the view of the cluster state as seen by a CPU setting the CPU up.h]hIn this discussion, the “outbound side” is the view of the cluster state as seen by a CPU tearing the cluster down. The “inbound side” is the view of the cluster state as seen by a CPU setting the CPU up.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hXIn order to enable safe coordination in such situations, it is important that a CPU which is setting up the cluster can advertise its state independently of the CPU which is tearing down the cluster. For this reason, the cluster state is split into two parts:h]hXIn order to enable safe coordination in such situations, it is important that a CPU which is setting up the cluster can advertise its state independently of the CPU which is tearing down the cluster. For this reason, the cluster state is split into two parts:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjh)}(hX "cluster" state: The global state of the cluster; or the state on the outbound side: - CLUSTER_DOWN - CLUSTER_UP - CLUSTER_GOING_DOWN "inbound" state: The state of the cluster on the inbound side. - INBOUND_NOT_COMING_UP - INBOUND_COMING_UP The different pairings of these states results in six possible states for the cluster as a whole:: CLUSTER_UP +==========> INBOUND_NOT_COMING_UP -------------+ # | | CLUSTER_UP <----+ | INBOUND_COMING_UP | v ^ CLUSTER_GOING_DOWN CLUSTER_GOING_DOWN # INBOUND_COMING_UP <=== INBOUND_NOT_COMING_UP CLUSTER_DOWN | | INBOUND_COMING_UP <----+ | | ^ | +=========== CLUSTER_DOWN <------------+ INBOUND_NOT_COMING_UP Transitions -----> can only be made by the outbound CPU, and only involve changes to the "cluster" state. Transitions ===##> can only be made by the inbound CPU, and only involve changes to the "inbound" state, except where there is no further transition possible on the outbound side (i.e., the outbound CPU has put the cluster into the CLUSTER_DOWN state). The race avoidance algorithm does not provide a way to determine which exact CPUs within the cluster play these roles. This must be decided in advance by some other means. Refer to the section "Last man and first man selection" for more explanation. CLUSTER_DOWN/INBOUND_NOT_COMING_UP is the only state where the cluster can actually be powered down. The parallelism of the inbound and outbound CPUs is observed by the existence of two different paths from CLUSTER_GOING_DOWN/ INBOUND_NOT_COMING_UP (corresponding to GOING_DOWN in the basic model) to CLUSTER_DOWN/INBOUND_COMING_UP (corresponding to COMING_UP in the basic model). The second path avoids cluster teardown completely. CLUSTER_UP/INBOUND_COMING_UP is equivalent to UP in the basic model. The final transition to CLUSTER_UP/INBOUND_NOT_COMING_UP is trivial and merely resets the state machine ready for the next cycle. Details of the allowable transitions follow. The next state in each case is notated / () where the is the side on which the transition can occur; either the inbound or the outbound side. h](h)}(hT"cluster" state: The global state of the cluster; or the state on the outbound side:h]hX“cluster” state: The global state of the cluster; or the state on the outbound side:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubjh)}(h1- CLUSTER_DOWN - CLUSTER_UP - CLUSTER_GOING_DOWN h]jn)}(hhh](js)}(h CLUSTER_DOWNh]h)}(hjh]h CLUSTER_DOWN}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jrhjubjs)}(h CLUSTER_UPh]h)}(hjh]h CLUSTER_UP}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jrhjubjs)}(hCLUSTER_GOING_DOWN h]h)}(hCLUSTER_GOING_DOWNh]hCLUSTER_GOING_DOWN}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jrhjubeh}(h]h ]h"]h$]h&]jjuh1jmhhhKhjubah}(h]h ]h"]h$]h&]uh1jghhhKhjubh)}(h>"inbound" state: The state of the cluster on the inbound side.h]hB“inbound” state: The state of the cluster on the inbound side.}(hj@hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubjh)}(h-- INBOUND_NOT_COMING_UP - INBOUND_COMING_UP h]jn)}(hhh](js)}(hINBOUND_NOT_COMING_UPh]h)}(hjWh]hINBOUND_NOT_COMING_UP}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjUubah}(h]h ]h"]h$]h&]uh1jrhjRubjs)}(hINBOUND_COMING_UP h]h)}(hINBOUND_COMING_UPh]hINBOUND_COMING_UP}(hjphhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjlubah}(h]h ]h"]h$]h&]uh1jrhjRubeh}(h]h ]h"]h$]h&]jjuh1jmhhhKhjNubah}(h]h ]h"]h$]h&]uh1jghhhKhjubh)}(hbThe different pairings of these states results in six possible states for the cluster as a whole::h]haThe different pairings of these states results in six possible states for the cluster as a whole:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubj)}(hX CLUSTER_UP +==========> INBOUND_NOT_COMING_UP -------------+ # | | CLUSTER_UP <----+ | INBOUND_COMING_UP | v ^ CLUSTER_GOING_DOWN CLUSTER_GOING_DOWN # INBOUND_COMING_UP <=== INBOUND_NOT_COMING_UP CLUSTER_DOWN | | INBOUND_COMING_UP <----+ | | ^ | +=========== CLUSTER_DOWN <------------+ INBOUND_NOT_COMING_UPh]hX CLUSTER_UP +==========> INBOUND_NOT_COMING_UP -------------+ # | | CLUSTER_UP <----+ | INBOUND_COMING_UP | v ^ CLUSTER_GOING_DOWN CLUSTER_GOING_DOWN # INBOUND_COMING_UP <=== INBOUND_NOT_COMING_UP CLUSTER_DOWN | | INBOUND_COMING_UP <----+ | | ^ | +=========== CLUSTER_DOWN <------------+ INBOUND_NOT_COMING_UP}hjsbah}(h]h ]h"]h$]h&]jjuh1jhhhKhjubh)}(hiTransitions -----> can only be made by the outbound CPU, and only involve changes to the "cluster" state.h]hmTransitions -----> can only be made by the outbound CPU, and only involve changes to the “cluster” state.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hTransitions ===##> can only be made by the inbound CPU, and only involve changes to the "inbound" state, except where there is no further transition possible on the outbound side (i.e., the outbound CPU has put the cluster into the CLUSTER_DOWN state).h]hXTransitions ===##> can only be made by the inbound CPU, and only involve changes to the “inbound” state, except where there is no further transition possible on the outbound side (i.e., the outbound CPU has put the cluster into the CLUSTER_DOWN state).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjubh)}(hThe race avoidance algorithm does not provide a way to determine which exact CPUs within the cluster play these roles. This must be decided in advance by some other means. Refer to the section "Last man and first man selection" for more explanation.h]hThe race avoidance algorithm does not provide a way to determine which exact CPUs within the cluster play these roles. This must be decided in advance by some other means. Refer to the section “Last man and first man selection” for more explanation.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hdCLUSTER_DOWN/INBOUND_NOT_COMING_UP is the only state where the cluster can actually be powered down.h]hdCLUSTER_DOWN/INBOUND_NOT_COMING_UP is the only state where the cluster can actually be powered down.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hXLThe parallelism of the inbound and outbound CPUs is observed by the existence of two different paths from CLUSTER_GOING_DOWN/ INBOUND_NOT_COMING_UP (corresponding to GOING_DOWN in the basic model) to CLUSTER_DOWN/INBOUND_COMING_UP (corresponding to COMING_UP in the basic model). The second path avoids cluster teardown completely.h]hXLThe parallelism of the inbound and outbound CPUs is observed by the existence of two different paths from CLUSTER_GOING_DOWN/ INBOUND_NOT_COMING_UP (corresponding to GOING_DOWN in the basic model) to CLUSTER_DOWN/INBOUND_COMING_UP (corresponding to COMING_UP in the basic model). The second path avoids cluster teardown completely.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hCLUSTER_UP/INBOUND_COMING_UP is equivalent to UP in the basic model. The final transition to CLUSTER_UP/INBOUND_NOT_COMING_UP is trivial and merely resets the state machine ready for the next cycle.h]hCLUSTER_UP/INBOUND_COMING_UP is equivalent to UP in the basic model. The final transition to CLUSTER_UP/INBOUND_NOT_COMING_UP is trivial and merely resets the state machine ready for the next cycle.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(h,Details of the allowable transitions follow.h]h,Details of the allowable transitions follow.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM$hjubh)}(h&The next state in each case is notatedh]h&The next state in each case is notated}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM&hjubjh)}(h1/ () h]h)}(h0/ ()h]h0/ ()}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM(hj ubah}(h]h ]h"]h$]h&]uh1jghhhM(hjubh)}(hpwhere the is the side on which the transition can occur; either the inbound or the outbound side.h]hpwhere the is the side on which the transition can occur; either the inbound or the outbound side.}(hj4 hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM*hjubeh}(h]h ]h"]h$]h&]uh1jghhhKhjhhubj)}(hhh]j)}(hX0CLUSTER_DOWN/INBOUND_NOT_COMING_UP: Next state: CLUSTER_DOWN/INBOUND_COMING_UP (inbound) Conditions: none Trigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(h#CLUSTER_DOWN/INBOUND_NOT_COMING_UP:h]h#CLUSTER_DOWN/INBOUND_NOT_COMING_UP:}(hjO hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM9hjK ubj )}(hhh]j)}(hhh](j)}(h4Next state: CLUSTER_DOWN/INBOUND_COMING_UP (inbound)h](j)}(h Next state:h]h Next state:}(hjg hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM/hjc ubj )}(hhh]h)}(h(CLUSTER_DOWN/INBOUND_COMING_UP (inbound)h]h(CLUSTER_DOWN/INBOUND_COMING_UP (inbound)}(hjx hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM0hju ubah}(h]h ]h"]h$]h&]uh1j hjc ubeh}(h]h ]h"]h$]h&]uh1jhhhM/hj` ubj)}(hConditions: none h](j)}(h Conditions:h]h Conditions:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM2hj ubj )}(hhh]h)}(hnoneh]hnone}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM2hj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhM2hj` ubj)}(hTrigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(hTrigger events:h]hTrigger events:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM9hj ubj )}(hhh]j)}(hhh](js)}(hZan explicit hardware power-up operation, resulting from a policy decision on another CPU; h]h)}(hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;h]hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM5hj ubah}(h]h ]h"]h$]h&]uh1jrhj ubjs)}(h)a hardware event, such as an interrupt. h]h)}(h'a hardware event, such as an interrupt.h]h'a hardware event, such as an interrupt.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM8hj ubah}(h]h ]h"]h$]h&]uh1jrhj ubeh}(h]h ]h"]h$]h&]jjjhjjuh1jhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhM9hj` ubeh}(h]h ]h"]h$]h&]uh1jhj] ubah}(h]h ]h"]h$]h&]uh1j hjK ubeh}(h]h ]h"]h$]h&]uh1jhhhM9hjH ubah}(h]h ]h"]h$]h&]uh1jhjhhhNhNubh)}(hCLUSTER_DOWN/INBOUND_COMING_UP:h]hCLUSTER_DOWN/INBOUND_COMING_UP:}(hj3 hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM;hjhhubjh)}(hXIn this state, an inbound CPU sets up the cluster, including enabling of hardware coherency at the cluster level and any other operations (such as cache invalidation) which are required in order to achieve this. The purpose of this state is to do sufficient cluster-level setup to enable other CPUs in the cluster to enter coherency safely. Next state: CLUSTER_UP/INBOUND_COMING_UP (inbound) Conditions: cluster-level setup and hardware coherency complete Trigger events: (spontaneous) h](h)}(hIn this state, an inbound CPU sets up the cluster, including enabling of hardware coherency at the cluster level and any other operations (such as cache invalidation) which are required in order to achieve this.h]hIn this state, an inbound CPU sets up the cluster, including enabling of hardware coherency at the cluster level and any other operations (such as cache invalidation) which are required in order to achieve this.}(hjE hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM=hjA ubh)}(hThe purpose of this state is to do sufficient cluster-level setup to enable other CPUs in the cluster to enter coherency safely.h]hThe purpose of this state is to do sufficient cluster-level setup to enable other CPUs in the cluster to enter coherency safely.}(hjS hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMBhjA ubj)}(hhh](j)}(h2Next state: CLUSTER_UP/INBOUND_COMING_UP (inbound)h](j)}(h Next state:h]h Next state:}(hjh hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMFhjd ubj )}(hhh]h)}(h&CLUSTER_UP/INBOUND_COMING_UP (inbound)h]h&CLUSTER_UP/INBOUND_COMING_UP (inbound)}(hjy hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMGhjv ubah}(h]h ]h"]h$]h&]uh1j hjd ubeh}(h]h ]h"]h$]h&]uh1jhhhMFhja ubj)}(h?Conditions: cluster-level setup and hardware coherency completeh](j)}(h Conditions:h]h Conditions:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMHhj ubj )}(hhh]h)}(h3cluster-level setup and hardware coherency completeh]h3cluster-level setup and hardware coherency complete}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMIhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMHhja ubj)}(hTrigger events: (spontaneous) h](j)}(hTrigger events:h]hTrigger events:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMLhj ubj )}(hhh]h)}(h (spontaneous)h]h (spontaneous)}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMKhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMLhja ubeh}(h]h ]h"]h$]h&]uh1jhjA ubeh}(h]h ]h"]h$]h&]uh1jghhhM=hjhhubh)}(hCLUSTER_UP/INBOUND_COMING_UP:h]hCLUSTER_UP/INBOUND_COMING_UP:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMNhjhhubjh)}(hXCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency. This is a transient state, leading immediately to CLUSTER_UP/INBOUND_NOT_COMING_UP. All other CPUs on the cluster should consider treat these two states as equivalent. Next state: CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound) Conditions: none Trigger events: (spontaneous) h](h)}(hCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency.h]hCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMPhj ubh)}(hThis is a transient state, leading immediately to CLUSTER_UP/INBOUND_NOT_COMING_UP. All other CPUs on the cluster should consider treat these two states as equivalent.h]hThis is a transient state, leading immediately to CLUSTER_UP/INBOUND_NOT_COMING_UP. All other CPUs on the cluster should consider treat these two states as equivalent.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMThj ubj)}(hhh](j)}(h6Next state: CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound)h](j)}(h Next state:h]h Next state:}(hj2 hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMXhj. ubj )}(hhh]h)}(h*CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound)h]h*CLUSTER_UP/INBOUND_NOT_COMING_UP (inbound)}(hjC hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMYhj@ ubah}(h]h ]h"]h$]h&]uh1j hj. ubeh}(h]h ]h"]h$]h&]uh1jhhhMXhj+ ubj)}(hConditions: noneh](j)}(h Conditions:h]h Conditions:}(hja hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMZhj] ubj )}(hhh]h)}(hnoneh]hnone}(hjr hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM[hjo ubah}(h]h ]h"]h$]h&]uh1j hj] ubeh}(h]h ]h"]h$]h&]uh1jhhhMZhj+ ubj)}(hTrigger events: (spontaneous) h](j)}(hTrigger events:h]hTrigger events:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM^hj ubj )}(hhh]h)}(h (spontaneous)h]h (spontaneous)}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM]hj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhM^hj+ ubeh}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]uh1jghhhMPhjhhubh)}(h!CLUSTER_UP/INBOUND_NOT_COMING_UP:h]h!CLUSTER_UP/INBOUND_NOT_COMING_UP:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM`hjhhubjh)}(hXCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency. The cluster will remain in this state until a policy decision is made to power the cluster down. Next state: CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound) Conditions: none Trigger events: policy decision to power down the cluster h](h)}(hCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency.h]hCluster-level setup is complete and hardware coherency is enabled for the cluster. Other CPUs in the cluster can safely enter coherency.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMbhj ubh)}(h`The cluster will remain in this state until a policy decision is made to power the cluster down.h]h`The cluster will remain in this state until a policy decision is made to power the cluster down.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMfhj ubj)}(hhh](j)}(h?Next state: CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound)h](j)}(h Next state:h]h Next state:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMihj ubj )}(hhh]h)}(h3CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound)h]h3CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP (outbound)}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMjhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMihj ubj)}(hConditions: noneh](j)}(h Conditions:h]h Conditions:}(hj+ hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMkhj' ubj )}(hhh]h)}(hnoneh]hnone}(hj< hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMlhj9 ubah}(h]h ]h"]h$]h&]uh1j hj' ubeh}(h]h ]h"]h$]h&]uh1jhhhMkhj ubj)}(h;Trigger events: policy decision to power down the cluster h](j)}(hTrigger events:h]hTrigger events:}(hjZ hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMohjV ubj )}(hhh]h)}(h)policy decision to power down the clusterh]h)policy decision to power down the cluster}(hjk hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMnhjh ubah}(h]h ]h"]h$]h&]uh1j hjV ubeh}(h]h ]h"]h$]h&]uh1jhhhMohj ubeh}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]uh1jghhhMbhjhhubh)}(h)CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP:h]h)CLUSTER_GOING_DOWN/INBOUND_NOT_COMING_UP:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMqhjhhubjh)}(hXAn outbound CPU is tearing the cluster down. The selected CPU must wait in this state until all CPUs in the cluster are in the CPU_DOWN state. When all CPUs are in the CPU_DOWN state, the cluster can be torn down, for example by cleaning data caches and exiting cluster-level coherency. To avoid wasteful unnecessary teardown operations, the outbound should check the inbound cluster state for asynchronous transitions to INBOUND_COMING_UP. Alternatively, individual CPUs can be checked for entry into CPU_COMING_UP or CPU_UP. Next states: CLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound) Conditions: cluster torn down and ready to power off Trigger events: (spontaneous) CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound) Conditions: none Trigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](h)}(hAn outbound CPU is tearing the cluster down. The selected CPU must wait in this state until all CPUs in the cluster are in the CPU_DOWN state.h]hAn outbound CPU is tearing the cluster down. The selected CPU must wait in this state until all CPUs in the cluster are in the CPU_DOWN state.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMshj ubh)}(hWhen all CPUs are in the CPU_DOWN state, the cluster can be torn down, for example by cleaning data caches and exiting cluster-level coherency.h]hWhen all CPUs are in the CPU_DOWN state, the cluster can be torn down, for example by cleaning data caches and exiting cluster-level coherency.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMwhj ubh)}(hTo avoid wasteful unnecessary teardown operations, the outbound should check the inbound cluster state for asynchronous transitions to INBOUND_COMING_UP. Alternatively, individual CPUs can be checked for entry into CPU_COMING_UP or CPU_UP.h]hTo avoid wasteful unnecessary teardown operations, the outbound should check the inbound cluster state for asynchronous transitions to INBOUND_COMING_UP. Alternatively, individual CPUs can be checked for entry into CPU_COMING_UP or CPU_UP.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM{hj ubh)}(h Next states:h]h Next states:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubj)}(hhh](j)}(hCLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound) Conditions: cluster torn down and ready to power off Trigger events: (spontaneous) h](j)}(h-CLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound)h]h-CLUSTER_DOWN/INBOUND_NOT_COMING_UP (outbound)}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhj ubj )}(hhh]j)}(hhh](j)}(h4Conditions: cluster torn down and ready to power offh](j)}(h Conditions:h]h Conditions:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhj ubj )}(hhh]h)}(h(cluster torn down and ready to power offh]h(cluster torn down and ready to power off}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj ubj)}(hTrigger events: (spontaneous) h](j)}(hTrigger events:h]hTrigger events:}(hj) hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhj% ubj )}(hhh]h)}(h (spontaneous)h]h (spontaneous)}(hj: hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj7 ubah}(h]h ]h"]h$]h&]uh1j hj% ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj ubeh}(h]h ]h"]h$]h&]uh1jhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj ubj)}(hX CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound) Conditions: none Trigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(h.CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound)h]h.CLUSTER_GOING_DOWN/INBOUND_COMING_UP (inbound)}(hjj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhjf ubj )}(hhh]j)}(hhh](j)}(hConditions: none h](j)}(h Conditions:h]h Conditions:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhj~ ubj )}(hhh]h)}(hnoneh]hnone}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1j hj~ ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj{ ubj)}(hTrigger events: a) an explicit hardware power-up operation, resulting from a policy decision on another CPU; b) a hardware event, such as an interrupt. h](j)}(hTrigger events:h]hTrigger events:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhj ubj )}(hhh]j)}(hhh](js)}(hZan explicit hardware power-up operation, resulting from a policy decision on another CPU; h]h)}(hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;h]hYan explicit hardware power-up operation, resulting from a policy decision on another CPU;}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jrhj ubjs)}(h)a hardware event, such as an interrupt. h]h)}(h'a hardware event, such as an interrupt.h]h'a hardware event, such as an interrupt.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jrhj ubeh}(h]h ]h"]h$]h&]jjjhjjuh1jhj ubah}(h]h ]h"]h$]h&]uh1j hj ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj{ ubeh}(h]h ]h"]h$]h&]uh1jhjx ubah}(h]h ]h"]h$]h&]uh1j hjf ubeh}(h]h ]h"]h$]h&]uh1jhhhMhj ubeh}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]uh1jghhhMshjhhubh)}(h%CLUSTER_GOING_DOWN/INBOUND_COMING_UP:h]h%CLUSTER_GOING_DOWN/INBOUND_COMING_UP:}(hj%hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubjh)}(hXThe cluster is (or was) being torn down, but another CPU has come online in the meantime and is trying to set up the cluster again. If the outbound CPU observes this state, it has two choices: a) back out of teardown, restoring the cluster to the CLUSTER_UP state; b) finish tearing the cluster down and put the cluster in the CLUSTER_DOWN state; the inbound CPU will set up the cluster again from there. Choice (a) permits the removal of some latency by avoiding unnecessary teardown and setup operations in situations where the cluster is not really going to be powered down. Next states: CLUSTER_UP/INBOUND_COMING_UP (outbound) Conditions: cluster-level setup and hardware coherency complete Trigger events: (spontaneous) CLUSTER_DOWN/INBOUND_COMING_UP (outbound) Conditions: cluster torn down and ready to power off Trigger events: (spontaneous) h](h)}(hThe cluster is (or was) being torn down, but another CPU has come online in the meantime and is trying to set up the cluster again.h]hThe cluster is (or was) being torn down, but another CPU has come online in the meantime and is trying to set up the cluster again.}(hj7hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj3ubh)}(hhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hjCluster-level initialisation may involve actions such as configuring coherency controls in the bus fabric.h]hjCluster-level initialisation may involve actions such as configuring coherency controls in the bus fabric.}(hjLhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe current implementation in mcpm_head.S uses a separate mutual exclusion mechanism to do this arbitration. This mechanism is documented in detail in vlocks.txt.h]hThe current implementation in mcpm_head.S uses a separate mutual exclusion mechanism to do this arbitration. This mechanism is documented in detail in vlocks.txt.}(hjZhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubeh}(h] last-man-and-first-man-selectionah ]h"] last man and first man selectionah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hFeatures and Limitationsh]hFeatures and Limitations}(hjshhhNhNubah}(h]h ]h"]h$]h&]uh1hhjphhhhhMubh)}(hImplementation:h]hImplementation:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjphhubjh)}(hXThe current ARM-based implementation is split between arch/arm/common/mcpm_head.S (low-level inbound CPU operations) and arch/arm/common/mcpm_entry.c (everything else): __mcpm_cpu_going_down() signals the transition of a CPU to the CPU_GOING_DOWN state. __mcpm_cpu_down() signals the transition of a CPU to the CPU_DOWN state. A CPU transitions to CPU_COMING_UP and then to CPU_UP via the low-level power-up code in mcpm_head.S. This could involve CPU-specific setup code, but in the current implementation it does not. __mcpm_outbound_enter_critical() and __mcpm_outbound_leave_critical() handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN and from there to CLUSTER_DOWN or back to CLUSTER_UP (in the case of an aborted cluster power-down). These functions are more complex than the __mcpm_cpu_*() functions due to the extra inter-CPU coordination which is needed for safe transitions at the cluster level. A cluster transitions from CLUSTER_DOWN back to CLUSTER_UP via the low-level power-up code in mcpm_head.S. This typically involves platform-specific setup code, provided by the platform-specific power_up_setup function registered via mcpm_sync_init. h](h)}(hThe current ARM-based implementation is split between arch/arm/common/mcpm_head.S (low-level inbound CPU operations) and arch/arm/common/mcpm_entry.c (everything else):h]hThe current ARM-based implementation is split between arch/arm/common/mcpm_head.S (low-level inbound CPU operations) and arch/arm/common/mcpm_entry.c (everything else):}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hT__mcpm_cpu_going_down() signals the transition of a CPU to the CPU_GOING_DOWN state.h]hT__mcpm_cpu_going_down() signals the transition of a CPU to the CPU_GOING_DOWN state.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hH__mcpm_cpu_down() signals the transition of a CPU to the CPU_DOWN state.h]hH__mcpm_cpu_down() signals the transition of a CPU to the CPU_DOWN state.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hA CPU transitions to CPU_COMING_UP and then to CPU_UP via the low-level power-up code in mcpm_head.S. This could involve CPU-specific setup code, but in the current implementation it does not.h]hA CPU transitions to CPU_COMING_UP and then to CPU_UP via the low-level power-up code in mcpm_head.S. This could involve CPU-specific setup code, but in the current implementation it does not.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(h__mcpm_outbound_enter_critical() and __mcpm_outbound_leave_critical() handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN and from there to CLUSTER_DOWN or back to CLUSTER_UP (in the case of an aborted cluster power-down).h]h__mcpm_outbound_enter_critical() and __mcpm_outbound_leave_critical() handle transitions from CLUSTER_UP to CLUSTER_GOING_DOWN and from there to CLUSTER_DOWN or back to CLUSTER_UP (in the case of an aborted cluster power-down).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hThese functions are more complex than the __mcpm_cpu_*() functions due to the extra inter-CPU coordination which is needed for safe transitions at the cluster level.h]hThese functions are more complex than the __mcpm_cpu_*() functions due to the extra inter-CPU coordination which is needed for safe transitions at the cluster level.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubh)}(hA cluster transitions from CLUSTER_DOWN back to CLUSTER_UP via the low-level power-up code in mcpm_head.S. This typically involves platform-specific setup code, provided by the platform-specific power_up_setup function registered via mcpm_sync_init.h]hA cluster transitions from CLUSTER_DOWN back to CLUSTER_UP via the low-level power-up code in mcpm_head.S. This typically involves platform-specific setup code, provided by the platform-specific power_up_setup function registered via mcpm_sync_init.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjubeh}(h]h ]h"]h$]h&]uh1jghhhMhjphhubh)}(hDeep topologies:h]hDeep topologies:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjphhubjh)}(hXnAs currently described and implemented, the algorithm does not support CPU topologies involving more than two levels (i.e., clusters of clusters are not supported). The algorithm could be extended by replicating the cluster-level states for the additional topological levels, and modifying the transition rules for the intermediate (non-outermost) cluster levels. h]h)}(hXlAs currently described and implemented, the algorithm does not support CPU topologies involving more than two levels (i.e., clusters of clusters are not supported). The algorithm could be extended by replicating the cluster-level states for the additional topological levels, and modifying the transition rules for the intermediate (non-outermost) cluster levels.h]hXlAs currently described and implemented, the algorithm does not support CPU topologies involving more than two levels (i.e., clusters of clusters are not supported). The algorithm could be extended by replicating the cluster-level states for the additional topological levels, and modifying the transition rules for the intermediate (non-outermost) cluster levels.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jghhhMhjphhubeh}(h]features-and-limitationsah ]h"]features and limitationsah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hColophonh]hColophon}(hj,hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj)hhhhhMubh)}(hyOriginally created and documented by Dave Martin for Linaro Limited, in collaboration with Nicolas Pitre and Achin Gupta.h]hyOriginally created and documented by Dave Martin for Linaro Limited, in collaboration with Nicolas Pitre and Achin Gupta.}(hj:hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj)hhubh)}(hCopyright (C) 2012-2013 Linaro Limited Distributed under the terms of Version 2 of the GNU General Public License, as defined in linux/COPYING.h]hCopyright (C) 2012-2013 Linaro Limited Distributed under the terms of Version 2 of the GNU General Public License, as defined in linux/COPYING.}(hjHhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj)hhubeh}(h]colophonah ]h"]colophonah$]h&]uh1hhhhhhhhMubeh}(h]9cluster-wide-power-up-power-down-race-avoidance-algorithmah ]h"]9cluster-wide power-up/power-down race avoidance algorithmah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksentryfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerjerror_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourcehʌ _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}nameids}(jcj`jEjBj%j"jjjjjmjjj&j#j[jXu nametypes}(jcjEj%jjjmj&j[uh}(j`hjBhj"jHjj(jjjjjj#jpjXj)u footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}Rparse_messages]transform_messages] transformerN include_log] decorationNhhub.