sphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget+/translations/zh_CN/locking/rt-mutex-designmodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget+/translations/zh_TW/locking/rt-mutex-designmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget+/translations/it_IT/locking/rt-mutex-designmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget+/translations/ja_JP/locking/rt-mutex-designmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget+/translations/ko_KR/locking/rt-mutex-designmodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget+/translations/sp_SP/locking/rt-mutex-designmodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhsection)}(hhh](htitle)}(hRT-mutex implementation designh]hRT-mutex implementation design}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhE/var/lib/git/docbuild/linux/Documentation/locking/rt-mutex-design.rsthKubh paragraph)}(h!Copyright (c) 2006 Steven Rostedth]h!Copyright (c) 2006 Steven Rostedt}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(h>Licensed under the GNU Free Documentation License, Version 1.2h]h>Licensed under the GNU Free Documentation License, Version 1.2}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hXPThis document tries to describe the design of the rtmutex.c implementation. It doesn't describe the reasons why rtmutex.c exists. For that please see Documentation/locking/rt-mutex.rst. Although this document does explain problems that happen without this code, but that is in the concept to understand what the code actually is doing.h]hXRThis document tries to describe the design of the rtmutex.c implementation. It doesn’t describe the reasons why rtmutex.c exists. For that please see Documentation/locking/rt-mutex.rst. Although this document does explain problems that happen without this code, but that is in the concept to understand what the code actually is doing.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hhhhubh)}(hThe goal of this document is to help others understand the priority inheritance (PI) algorithm that is used, as well as reasons for the decisions that were made to implement PI in the manner that was done.h]hThe goal of this document is to help others understand the priority inheritance (PI) algorithm that is used, as well as reasons for the decisions that were made to implement PI in the manner that was done.}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hhh](h)}(hUnbounded Priority Inversionh]hUnbounded Priority Inversion}(hhhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhhhhhKubh)}(hXtPriority inversion is when a lower priority process executes while a higher priority process wants to run. This happens for several reasons, and most of the time it can't be helped. Anytime a high priority process wants to use a resource that a lower priority process has (a mutex for example), the high priority process must wait until the lower priority process is done with the resource. This is a priority inversion. What we want to prevent is something called unbounded priority inversion. That is when the high priority process is prevented from running by a lower priority process for an undetermined amount of time.h]hXvPriority inversion is when a lower priority process executes while a higher priority process wants to run. This happens for several reasons, and most of the time it can’t be helped. Anytime a high priority process wants to use a resource that a lower priority process has (a mutex for example), the high priority process must wait until the lower priority process is done with the resource. This is a priority inversion. What we want to prevent is something called unbounded priority inversion. That is when the high priority process is prevented from running by a lower priority process for an undetermined amount of time.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hXThe classic example of unbounded priority inversion is where you have three processes, let's call them processes A, B, and C, where A is the highest priority process, C is the lowest, and B is in between. A tries to grab a lock that C owns and must wait and lets C run to release the lock. But in the meantime, B executes, and since B is of a higher priority than C, it preempts C, but by doing so, it is in fact preempting A which is a higher priority process. Now there's no way of knowing how long A will be sleeping waiting for C to release the lock, because for all we know, B is a CPU hog and will never give C a chance to release the lock. This is called unbounded priority inversion.h]hXThe classic example of unbounded priority inversion is where you have three processes, let’s call them processes A, B, and C, where A is the highest priority process, C is the lowest, and B is in between. A tries to grab a lock that C owns and must wait and lets C run to release the lock. But in the meantime, B executes, and since B is of a higher priority than C, it preempts C, but by doing so, it is in fact preempting A which is a higher priority process. Now there’s no way of knowing how long A will be sleeping waiting for C to release the lock, because for all we know, B is a CPU hog and will never give C a chance to release the lock. This is called unbounded priority inversion.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK"hhhhubh)}(h/Here's a little ASCII art to show the problem::h]h0Here’s a little ASCII art to show the problem:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK-hhhhubh literal_block)}(h grab lock L1 (owned by C) | A ---+ C preempted by B | C +----+ B +--------> B now keeps A from running.h]h grab lock L1 (owned by C) | A ---+ C preempted by B | C +----+ B +--------> B now keeps A from running.}hj.sbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1j,hhhK/hhhhubeh}(h]unbounded-priority-inversionah ]h"]unbounded priority inversionah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hPriority Inheritance (PI)h]hPriority Inheritance (PI)}(hjIhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjFhhhhhK;ubh)}(hxThere are several ways to solve this issue, but other ways are out of scope for this document. Here we only discuss PI.h]hxThere are several ways to solve this issue, but other ways are out of scope for this document. Here we only discuss PI.}(hjWhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK=hjFhhubh)}(hPI is where a process inherits the priority of another process if the other process blocks on a lock owned by the current process. To make this easier to understand, let's use the previous example, with processes A, B, and C again.h]hPI is where a process inherits the priority of another process if the other process blocks on a lock owned by the current process. To make this easier to understand, let’s use the previous example, with processes A, B, and C again.}(hjehhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK@hjFhhubh)}(hX*This time, when A blocks on the lock owned by C, C would inherit the priority of A. So now if B becomes runnable, it would not preempt C, since C now has the high priority of A. As soon as C releases the lock, it loses its inherited priority, and A then can continue with the resource that C had.h]hX*This time, when A blocks on the lock owned by C, C would inherit the priority of A. So now if B becomes runnable, it would not preempt C, since C now has the high priority of A. As soon as C releases the lock, it loses its inherited priority, and A then can continue with the resource that C had.}(hjshhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKDhjFhhubeh}(h]priority-inheritance-piah ]h"]priority inheritance (pi)ah$]h&]uh1hhhhhhhhK;ubh)}(hhh](h)}(h Terminologyh]h Terminology}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKJubh)}(hwHere I explain some terminology that is used in this document to help describe the design that is used to implement PI.h]hwHere I explain some terminology that is used in this document to help describe the design that is used to implement PI.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKLhjhhubhdefinition_list)}(hhh](hdefinition_list_item)}(hPI chain - The PI chain is an ordered series of locks and processes that cause processes to inherit priorities from a previous process that is blocked on one of its locks. This is described in more detail later in this document. h](hterm)}(hPI chainh]hPI chain}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKShjubh definition)}(hhh]h bullet_list)}(hhh]h list_item)}(hThe PI chain is an ordered series of locks and processes that cause processes to inherit priorities from a previous process that is blocked on one of its locks. This is described in more detail later in this document. h]h)}(hThe PI chain is an ordered series of locks and processes that cause processes to inherit priorities from a previous process that is blocked on one of its locks. This is described in more detail later in this document.h]hThe PI chain is an ordered series of locks and processes that cause processes to inherit priorities from a previous process that is blocked on one of its locks. This is described in more detail later in this document.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKPhjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]bullet-uh1jhhhKPhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKShjubj)}(hmutex - In this document, to differentiate from locks that implement PI and spin locks that are used in the PI code, from now on the PI locks will be called a mutex. h](j)}(hmutexh]hmutex}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKXhjubj)}(hhh]j)}(hhh]j)}(hIn this document, to differentiate from locks that implement PI and spin locks that are used in the PI code, from now on the PI locks will be called a mutex. h]h)}(hIn this document, to differentiate from locks that implement PI and spin locks that are used in the PI code, from now on the PI locks will be called a mutex.h]hIn this document, to differentiate from locks that implement PI and spin locks that are used in the PI code, from now on the PI locks will be called a mutex.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKVhjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]jjuh1jhhhKVhj ubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKXhjhhubj)}(hX1lock - In this document from now on, I will use the term lock when referring to spin locks that are used to protect parts of the PI algorithm. These locks disable preemption for UP (when CONFIG_PREEMPT is enabled) and on SMP prevents multiple CPUs from entering critical sections simultaneously. h](j)}(hlockh]hlock}(hjAhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK_hj=ubj)}(hhh]j)}(hhh]j)}(hX"In this document from now on, I will use the term lock when referring to spin locks that are used to protect parts of the PI algorithm. These locks disable preemption for UP (when CONFIG_PREEMPT is enabled) and on SMP prevents multiple CPUs from entering critical sections simultaneously. h]h)}(hX!In this document from now on, I will use the term lock when referring to spin locks that are used to protect parts of the PI algorithm. These locks disable preemption for UP (when CONFIG_PREEMPT is enabled) and on SMP prevents multiple CPUs from entering critical sections simultaneously.h]hX!In this document from now on, I will use the term lock when referring to spin locks that are used to protect parts of the PI algorithm. These locks disable preemption for UP (when CONFIG_PREEMPT is enabled) and on SMP prevents multiple CPUs from entering critical sections simultaneously.}(hjYhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK[hjUubah}(h]h ]h"]h$]h&]uh1jhjRubah}(h]h ]h"]h$]h&]jjuh1jhhhK[hjOubah}(h]h ]h"]h$]h&]uh1jhj=ubeh}(h]h ]h"]h$]h&]uh1jhhhK_hjhhubj)}(h spin lock - Same as lock above. h](j)}(h spin lockh]h spin lock}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKbhjubj)}(hhh]j)}(hhh]j)}(hSame as lock above. h]h)}(hSame as lock above.h]hSame as lock above.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKbhjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]jjuh1jhhhKbhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKbhjhhubj)}(hXzwaiter - A waiter is a struct that is stored on the stack of a blocked process. Since the scope of the waiter is within the code for a process being blocked on the mutex, it is fine to allocate the waiter on the process's stack (local variable). This structure holds a pointer to the task, as well as the mutex that the task is blocked on. It also has rbtree node structures to place the task in the waiters rbtree of a mutex as well as the pi_waiters rbtree of a mutex owner task (described below). waiter is sometimes used in reference to the task that is waiting on a mutex. This is the same as waiter->task. h](j)}(hwaiterh]hwaiter}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKohjubj)}(hhh]j)}(hhh]j)}(hX_A waiter is a struct that is stored on the stack of a blocked process. Since the scope of the waiter is within the code for a process being blocked on the mutex, it is fine to allocate the waiter on the process's stack (local variable). This structure holds a pointer to the task, as well as the mutex that the task is blocked on. It also has rbtree node structures to place the task in the waiters rbtree of a mutex as well as the pi_waiters rbtree of a mutex owner task (described below). waiter is sometimes used in reference to the task that is waiting on a mutex. This is the same as waiter->task. h](h)}(hXA waiter is a struct that is stored on the stack of a blocked process. Since the scope of the waiter is within the code for a process being blocked on the mutex, it is fine to allocate the waiter on the process's stack (local variable). This structure holds a pointer to the task, as well as the mutex that the task is blocked on. It also has rbtree node structures to place the task in the waiters rbtree of a mutex as well as the pi_waiters rbtree of a mutex owner task (described below).h]hXA waiter is a struct that is stored on the stack of a blocked process. Since the scope of the waiter is within the code for a process being blocked on the mutex, it is fine to allocate the waiter on the process’s stack (local variable). This structure holds a pointer to the task, as well as the mutex that the task is blocked on. It also has rbtree node structures to place the task in the waiters rbtree of a mutex as well as the pi_waiters rbtree of a mutex owner task (described below).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKehjubh)}(howaiter is sometimes used in reference to the task that is waiting on a mutex. This is the same as waiter->task.h]howaiter is sometimes used in reference to the task that is waiting on a mutex. This is the same as waiter->task.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKnhjubeh}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]jjuh1jhhhKehjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKohjhhubj)}(h;waiters - A list of processes that are blocked on a mutex. h](j)}(hwaitersh]hwaiters}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKrhjubj)}(hhh]j)}(hhh]j)}(h1A list of processes that are blocked on a mutex. h]h)}(h0A list of processes that are blocked on a mutex.h]h0A list of processes that are blocked on a mutex.}(hj-hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKrhj)ubah}(h]h ]h"]h$]h&]uh1jhj&ubah}(h]h ]h"]h$]h&]jjuh1jhhhKrhj#ubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKrhjhhubj)}(hGtop waiter - The highest priority process waiting on a specific mutex. h](j)}(h top waiterh]h top waiter}(hjWhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKuhjSubj)}(hhh]j)}(hhh]j)}(h:The highest priority process waiting on a specific mutex. h]h)}(h9The highest priority process waiting on a specific mutex.h]h9The highest priority process waiting on a specific mutex.}(hjohhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKuhjkubah}(h]h ]h"]h$]h&]uh1jhjhubah}(h]h ]h"]h$]h&]jjuh1jhhhKuhjeubah}(h]h ]h"]h$]h&]uh1jhjSubeh}(h]h ]h"]h$]h&]uh1jhhhKuhjhhubj)}(hktop pi waiter - The highest priority process waiting on one of the mutexes that a specific process owns. h](j)}(h top pi waiterh]h top pi waiter}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhKyhjubj)}(hhh]j)}(hhh]j)}(hYThe highest priority process waiting on one of the mutexes that a specific process owns. h]h)}(hXThe highest priority process waiting on one of the mutexes that a specific process owns.h]hXThe highest priority process waiting on one of the mutexes that a specific process owns.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKxhjubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]jjuh1jhhhKxhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhKyhjhhubj)}(hNote: task and process are used interchangeably in this document, mostly to differentiate between two processes that are being described together. h](j)}(hNote:h]hNote:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhK~hjubj)}(hhh]h)}(htask and process are used interchangeably in this document, mostly to differentiate between two processes that are being described together.h]htask and process are used interchangeably in this document, mostly to differentiate between two processes that are being described together.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK|hjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhK~hjhhubeh}(h]h ]h"]h$]h&]uh1jhjhhhNhNubeh}(h] terminologyah ]h"] terminologyah$]h&]uh1hhhhhhhhKJubh)}(hhh](h)}(hPI chainh]hPI chain}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hThe PI chain is a list of processes and mutexes that may cause priority inheritance to take place. Multiple chains may converge, but a chain would never diverge, since a process can't be blocked on more than one mutex at a time.h]hThe PI chain is a list of processes and mutexes that may cause priority inheritance to take place. Multiple chains may converge, but a chain would never diverge, since a process can’t be blocked on more than one mutex at a time.}(hj%hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h Example::h]hExample:}(hj3hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hXProcess: A, B, C, D, E Mutexes: L1, L2, L3, L4 A owns: L1 B blocked on L1 B owns L2 C blocked on L2 C owns L3 D blocked on L3 D owns L4 E blocked on L4h]hXProcess: A, B, C, D, E Mutexes: L1, L2, L3, L4 A owns: L1 B blocked on L1 B owns L2 C blocked on L2 C owns L3 D blocked on L3 D owns L4 E blocked on L4}hjAsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hThe chain would be::h]hThe chain would be:}(hjOhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hE->L4->D->L3->C->L2->B->L1->Ah]hE->L4->D->L3->C->L2->B->L1->A}hj]sbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hTo show where two chains merge, we could add another process F and another mutex L5 where B owns L5 and F is blocked on mutex L5.h]hTo show where two chains merge, we could add another process F and another mutex L5 where B owns L5 and F is blocked on mutex L5.}(hjkhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hThe chain for F would be::h]hThe chain for F would be:}(hjyhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hF->L5->B->L1->Ah]hF->L5->B->L1->A}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(heSince a process may own more than one mutex, but never be blocked on more than one, the chains merge.h]heSince a process may own more than one mutex, but never be blocked on more than one, the chains merge.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hHere we show both chains::h]hHere we show both chains:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hwE->L4->D->L3->C->L2-+ | +->B->L1->A | F->L5-+h]hwE->L4->D->L3->C->L2-+ | +->B->L1->A | F->L5-+}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hFor PI to work, the processes at the right end of these chains (or we may also call it the Top of the chain) must be equal to or higher in priority than the processes to the left or below in the chain.h]hFor PI to work, the processes at the right end of these chains (or we may also call it the Top of the chain) must be equal to or higher in priority than the processes to the left or below in the chain.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(hAlso since a mutex may have more than one process blocked on it, we can have multiple chains merge at mutexes. If we add another process G that is blocked on mutex L2::h]hAlso since a mutex may have more than one process blocked on it, we can have multiple chains merge at mutexes. If we add another process G that is blocked on mutex L2:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hG->L2->B->L1->Ah]hG->L2->B->L1->A}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hPAnd once again, to show how this can grow I will show the merging chains again::h]hOAnd once again, to show how this can grow I will show the merging chains again:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hE->L4->D->L3->C-+ +->L2-+ | | G-+ +->B->L1->A | F->L5-+h]hE->L4->D->L3->C-+ +->L2-+ | | G-+ +->B->L1->A | F->L5-+}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hIf process G has the highest priority in the chain, then all the tasks up the chain (A and B in this example), must have their priorities increased to that of G.h]hIf process G has the highest priority in the chain, then all the tasks up the chain (A and B in this example), must have their priorities increased to that of G.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubeh}(h]pi-chainah ]h"]pi chainah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hMutex Waiters Treeh]hMutex Waiters Tree}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hEvery mutex keeps track of all the waiters that are blocked on itself. The mutex has a rbtree to store these waiters by priority. This tree is protected by a spin lock that is located in the struct of the mutex. This lock is called wait_lock.h]hEvery mutex keeps track of all the waiters that are blocked on itself. The mutex has a rbtree to store these waiters by priority. This tree is protected by a spin lock that is located in the struct of the mutex. This lock is called wait_lock.}(hj,hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubeh}(h]mutex-waiters-treeah ]h"]mutex waiters treeah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(h Task PI Treeh]h Task PI Tree}(hjEhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjBhhhhhKubh)}(hX To keep track of the PI chains, each process has its own PI rbtree. This is a tree of all top waiters of the mutexes that are owned by the process. Note that this tree only holds the top waiters and not all waiters that are blocked on mutexes owned by the process.h]hX To keep track of the PI chains, each process has its own PI rbtree. This is a tree of all top waiters of the mutexes that are owned by the process. Note that this tree only holds the top waiters and not all waiters that are blocked on mutexes owned by the process.}(hjShhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjBhhubh)}(hThe top of the task's PI tree is always the highest priority task that is waiting on a mutex that is owned by the task. So if the task has inherited a priority, it will always be the priority of the task that is at the top of this tree.h]hThe top of the task’s PI tree is always the highest priority task that is waiting on a mutex that is owned by the task. So if the task has inherited a priority, it will always be the priority of the task that is at the top of this tree.}(hjahhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjBhhubh)}(hXThis tree is stored in the task structure of a process as a rbtree called pi_waiters. It is protected by a spin lock also in the task structure, called pi_lock. This lock may also be taken in interrupt context, so when locking the pi_lock, interrupts must be disabled.h]hXThis tree is stored in the task structure of a process as a rbtree called pi_waiters. It is protected by a spin lock also in the task structure, called pi_lock. This lock may also be taken in interrupt context, so when locking the pi_lock, interrupts must be disabled.}(hjohhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjBhhubeh}(h] task-pi-treeah ]h"] task pi treeah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hDepth of the PI Chainh]hDepth of the PI Chain}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhKubh)}(hXThe maximum depth of the PI chain is not dynamic, and could actually be defined. But is very complex to figure it out, since it depends on all the nesting of mutexes. Let's look at the example where we have 3 mutexes, L1, L2, and L3, and four separate functions func1, func2, func3 and func4. The following shows a locking order of L1->L2->L3, but may not actually be directly nested that way::h]hXThe maximum depth of the PI chain is not dynamic, and could actually be defined. But is very complex to figure it out, since it depends on all the nesting of mutexes. Let’s look at the example where we have 3 mutexes, L1, L2, and L3, and four separate functions func1, func2, func3 and func4. The following shows a locking order of L1->L2->L3, but may not actually be directly nested that way:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubj-)}(hXvoid func1(void) { mutex_lock(L1); /* do anything */ mutex_unlock(L1); } void func2(void) { mutex_lock(L1); mutex_lock(L2); /* do something */ mutex_unlock(L2); mutex_unlock(L1); } void func3(void) { mutex_lock(L2); mutex_lock(L3); /* do something else */ mutex_unlock(L3); mutex_unlock(L2); } void func4(void) { mutex_lock(L3); /* do something again */ mutex_unlock(L3); }h]hXvoid func1(void) { mutex_lock(L1); /* do anything */ mutex_unlock(L1); } void func2(void) { mutex_lock(L1); mutex_lock(L2); /* do something */ mutex_unlock(L2); mutex_unlock(L1); } void func3(void) { mutex_lock(L2); mutex_lock(L3); /* do something else */ mutex_unlock(L3); mutex_unlock(L2); } void func4(void) { mutex_lock(L3); /* do something again */ mutex_unlock(L3); }}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhKhjhhubh)}(hX'Now we add 4 processes that run each of these functions separately. Processes A, B, C, and D which run functions func1, func2, func3 and func4 respectively, and such that D runs first and A last. With D being preempted in func4 in the "do something again" area, we have a locking that follows::h]hX*Now we add 4 processes that run each of these functions separately. Processes A, B, C, and D which run functions func1, func2, func3 and func4 respectively, and such that D runs first and A last. With D being preempted in func4 in the “do something again” area, we have a locking that follows:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubj-)}(hD owns L3 C blocked on L3 C owns L2 B blocked on L2 B owns L1 A blocked on L1 And thus we have the chain A->L1->B->L2->C->L3->D.h]hD owns L3 C blocked on L3 C owns L2 B blocked on L2 B owns L1 A blocked on L1 And thus we have the chain A->L1->B->L2->C->L3->D.}hjsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhMhjhhubh)}(hX#This gives us a PI depth of 4 (four processes), but looking at any of the functions individually, it seems as though they only have at most a locking depth of two. So, although the locking depth is defined at compile time, it still is very difficult to find the possibilities of that depth.h]hX#This gives us a PI depth of 4 (four processes), but looking at any of the functions individually, it seems as though they only have at most a locking depth of two. So, although the locking depth is defined at compile time, it still is very difficult to find the possibilities of that depth.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hjhhubh)}(hXNow since mutexes can be defined by user-land applications, we don't want a DOS type of application that nests large amounts of mutexes to create a large PI chain, and have the code holding spin locks while looking at a large amount of data. So to prevent this, the implementation not only implements a maximum lock depth, but also only holds at most two different locks at a time, as it walks the PI chain. More about this below.h]hXNow since mutexes can be defined by user-land applications, we don’t want a DOS type of application that nests large amounts of mutexes to create a large PI chain, and have the code holding spin locks while looking at a large amount of data. So to prevent this, the implementation not only implements a maximum lock depth, but also only holds at most two different locks at a time, as it walks the PI chain. More about this below.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM%hjhhubeh}(h]depth-of-the-pi-chainah ]h"]depth of the pi chainah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hMutex owner and flagsh]hMutex owner and flags}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM.ubh)}(hXThe mutex structure contains a pointer to the owner of the mutex. If the mutex is not owned, this owner is set to NULL. Since all architectures have the task structure on at least a two byte alignment (and if this is not true, the rtmutex.c code will be broken!), this allows for the least significant bit to be used as a flag. Bit 0 is used as the "Has Waiters" flag. It's set whenever there are waiters on a mutex.h]hXThe mutex structure contains a pointer to the owner of the mutex. If the mutex is not owned, this owner is set to NULL. Since all architectures have the task structure on at least a two byte alignment (and if this is not true, the rtmutex.c code will be broken!), this allows for the least significant bit to be used as a flag. Bit 0 is used as the “Has Waiters” flag. It’s set whenever there are waiters on a mutex.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM0hjhhubh)}(h;See Documentation/locking/rt-mutex.rst for further details.h]h;See Documentation/locking/rt-mutex.rst for further details.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM7hjhhubeh}(h]mutex-owner-and-flagsah ]h"]mutex owner and flagsah$]h&]uh1hhhhhhhhM.ubh)}(hhh](h)}(hcmpxchg Tricksh]hcmpxchg Tricks}(hj*hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj'hhhhhM:ubh)}(hSome architectures implement an atomic cmpxchg (Compare and Exchange). This is used (when applicable) to keep the fast path of grabbing and releasing mutexes short.h]hSome architectures implement an atomic cmpxchg (Compare and Exchange). This is used (when applicable) to keep the fast path of grabbing and releasing mutexes short.}(hj8hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM<hj'hhubh)}(hBcmpxchg is basically the following function performed atomically::h]hAcmpxchg is basically the following function performed atomically:}(hjFhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM@hj'hhubj-)}(hunsigned long _cmpxchg(unsigned long *A, unsigned long *B, unsigned long *C) { unsigned long T = *A; if (*A == *B) { *A = *C; } return T; } #define cmpxchg(a,b,c) _cmpxchg(&a,&b,&c)h]hunsigned long _cmpxchg(unsigned long *A, unsigned long *B, unsigned long *C) { unsigned long T = *A; if (*A == *B) { *A = *C; } return T; } #define cmpxchg(a,b,c) _cmpxchg(&a,&b,&c)}hjTsbah}(h]h ]h"]h$]h&]j<j=uh1j,hhhMBhj'hhubh)}(hThis is really nice to have, since it allows you to only update a variable if the variable is what you expect it to be. You know if it succeeded if the return value (the old value of A) is equal to B.h]hThis is really nice to have, since it allows you to only update a variable if the variable is what you expect it to be. You know if it succeeded if the return value (the old value of A) is equal to B.}(hjbhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMLhj'hhubh)}(hXThe macro rt_mutex_cmpxchg is used to try to lock and unlock mutexes. If the architecture does not support CMPXCHG, then this macro is simply set to fail every time. But if CMPXCHG is supported, then this will help out extremely to keep the fast path short.h]hXThe macro rt_mutex_cmpxchg is used to try to lock and unlock mutexes. If the architecture does not support CMPXCHG, then this macro is simply set to fail every time. But if CMPXCHG is supported, then this will help out extremely to keep the fast path short.}(hjphhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMPhj'hhubh)}(hThe use of rt_mutex_cmpxchg with the flags in the owner field help optimize the system for architectures that support it. This will also be explained later in this document.h]hThe use of rt_mutex_cmpxchg with the flags in the owner field help optimize the system for architectures that support it. This will also be explained later in this document.}(hj~hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMUhj'hhubeh}(h]cmpxchg-tricksah ]h"]cmpxchg tricksah$]h&]uh1hhhhhhhhM:ubh)}(hhh](h)}(hPriority adjustmentsh]hPriority adjustments}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM[ubh)}(hThe implementation of the PI code in rtmutex.c has several places that a process must adjust its priority. With the help of the pi_waiters of a process this is rather easy to know what needs to be adjusted.h]hThe implementation of the PI code in rtmutex.c has several places that a process must adjust its priority. With the help of the pi_waiters of a process this is rather easy to know what needs to be adjusted.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM]hjhhubh)}(hThe functions implementing the task adjustments are rt_mutex_adjust_prio and rt_mutex_setprio. rt_mutex_setprio is only used in rt_mutex_adjust_prio.h]hThe functions implementing the task adjustments are rt_mutex_adjust_prio and rt_mutex_setprio. rt_mutex_setprio is only used in rt_mutex_adjust_prio.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMahjhhubh)}(hX,rt_mutex_adjust_prio examines the priority of the task, and the highest priority process that is waiting any of mutexes owned by the task. Since the pi_waiters of a task holds an order by priority of all the top waiters of all the mutexes that the task owns, we simply need to compare the top pi waiter to its own normal/deadline priority and take the higher one. Then rt_mutex_setprio is called to adjust the priority of the task to the new priority. Note that rt_mutex_setprio is defined in kernel/sched/core.c to implement the actual change in priority.h]hX,rt_mutex_adjust_prio examines the priority of the task, and the highest priority process that is waiting any of mutexes owned by the task. Since the pi_waiters of a task holds an order by priority of all the top waiters of all the mutexes that the task owns, we simply need to compare the top pi waiter to its own normal/deadline priority and take the higher one. Then rt_mutex_setprio is called to adjust the priority of the task to the new priority. Note that rt_mutex_setprio is defined in kernel/sched/core.c to implement the actual change in priority.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMdhjhhubj)}(hhh]j)}(hNote: For the "prio" field in task_struct, the lower the number, the higher the priority. A "prio" of 5 is of higher priority than a "prio" of 10. h](j)}(hNote:h]hNote:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMphjubj)}(hhh]h)}(hFor the "prio" field in task_struct, the lower the number, the higher the priority. A "prio" of 5 is of higher priority than a "prio" of 10.h]hFor the “prio” field in task_struct, the lower the number, the higher the priority. A “prio” of 5 is of higher priority than a “prio” of 10.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMnhjubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhMphjubah}(h]h ]h"]h$]h&]uh1jhjhhhhhNubh)}(hXIt is interesting to note that rt_mutex_adjust_prio can either increase or decrease the priority of the task. In the case that a higher priority process has just blocked on a mutex owned by the task, rt_mutex_adjust_prio would increase/boost the task's priority. But if a higher priority task were for some reason to leave the mutex (timeout or signal), this same function would decrease/unboost the priority of the task. That is because the pi_waiters always contains the highest priority task that is waiting on a mutex owned by the task, so we only need to compare the priority of that top pi waiter to the normal priority of the given task.h]hXIt is interesting to note that rt_mutex_adjust_prio can either increase or decrease the priority of the task. In the case that a higher priority process has just blocked on a mutex owned by the task, rt_mutex_adjust_prio would increase/boost the task’s priority. But if a higher priority task were for some reason to leave the mutex (timeout or signal), this same function would decrease/unboost the priority of the task. That is because the pi_waiters always contains the highest priority task that is waiting on a mutex owned by the task, so we only need to compare the priority of that top pi waiter to the normal priority of the given task.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMrhjhhubeh}(h]priority-adjustmentsah ]h"]priority adjustmentsah$]h&]uh1hhhhhhhhM[ubh)}(hhh](h)}(h(High level overview of the PI chain walkh]h(High level overview of the PI chain walk}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhM~ubh)}(hLThe PI chain walk is implemented by the function rt_mutex_adjust_prio_chain.h]hLThe PI chain walk is implemented by the function rt_mutex_adjust_prio_chain.}(hj.hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe implementation has gone through several iterations, and has ended up with what we believe is the best. It walks the PI chain by only grabbing at most two locks at a time, and is very efficient.h]hThe implementation has gone through several iterations, and has ended up with what we believe is the best. It walks the PI chain by only grabbing at most two locks at a time, and is very efficient.}(hj<hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hWThe rt_mutex_adjust_prio_chain can be used either to boost or lower process priorities.h]hWThe rt_mutex_adjust_prio_chain can be used either to boost or lower process priorities.}(hjJhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXrt_mutex_adjust_prio_chain is called with a task to be checked for PI (de)boosting (the owner of a mutex that a process is blocking on), a flag to check for deadlocking, the mutex that the task owns, a pointer to a waiter that is the process's waiter struct that is blocked on the mutex (although this parameter may be NULL for deboosting), a pointer to the mutex on which the task is blocked, and a top_task as the top waiter of the mutex.h]hXrt_mutex_adjust_prio_chain is called with a task to be checked for PI (de)boosting (the owner of a mutex that a process is blocking on), a flag to check for deadlocking, the mutex that the task owns, a pointer to a waiter that is the process’s waiter struct that is blocked on the mutex (although this parameter may be NULL for deboosting), a pointer to the mutex on which the task is blocked, and a top_task as the top waiter of the mutex.}(hjXhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hoFor this explanation, I will not mention deadlock detection. This explanation will try to stay at a high level.h]hoFor this explanation, I will not mention deadlock detection. This explanation will try to stay at a high level.}(hjfhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hWhen this function is called, there are no locks held. That also means that the state of the owner and lock can change when entered into this function.h]hWhen this function is called, there are no locks held. That also means that the state of the owner and lock can change when entered into this function.}(hjthhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXBefore this function is called, the task has already had rt_mutex_adjust_prio performed on it. This means that the task is set to the priority that it should be at, but the rbtree nodes of the task's waiter have not been updated with the new priorities, and this task may not be in the proper locations in the pi_waiters and waiters trees that the task is blocked on. This function solves all that.h]hXBefore this function is called, the task has already had rt_mutex_adjust_prio performed on it. This means that the task is set to the priority that it should be at, but the rbtree nodes of the task’s waiter have not been updated with the new priorities, and this task may not be in the proper locations in the pi_waiters and waiters trees that the task is blocked on. This function solves all that.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hThe main operation of this function is summarized by Thomas Gleixner in rtmutex.c. See the 'Chain walk basics and protection scope' comment for further details.h]hThe main operation of this function is summarized by Thomas Gleixner in rtmutex.c. See the ‘Chain walk basics and protection scope’ comment for further details.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubeh}(h](high-level-overview-of-the-pi-chain-walkah ]h"](high level overview of the pi chain walkah$]h&]uh1hhhhhhhhM~ubh)}(hhh](h)}(h$Taking of a mutex (The walk through)h]h$Taking of a mutex (The walk through)}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(h[OK, now let's take a look at the detailed walk through of what happens when taking a mutex.h]h]OK, now let’s take a look at the detailed walk through of what happens when taking a mutex.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXThe first thing that is tried is the fast taking of the mutex. This is done when we have CMPXCHG enabled (otherwise the fast taking automatically fails). Only when the owner field of the mutex is NULL can the lock be taken with the CMPXCHG and nothing else needs to be done.h]hXThe first thing that is tried is the fast taking of the mutex. This is done when we have CMPXCHG enabled (otherwise the fast taking automatically fails). Only when the owner field of the mutex is NULL can the lock be taken with the CMPXCHG and nothing else needs to be done.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hRIf there is contention on the lock, we go about the slow path (rt_mutex_slowlock).h]hRIf there is contention on the lock, we go about the slow path (rt_mutex_slowlock).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hX5The slow path function is where the task's waiter structure is created on the stack. This is because the waiter structure is only needed for the scope of this function. The waiter structure holds the nodes to store the task on the waiters tree of the mutex, and if need be, the pi_waiters tree of the owner.h]hX7The slow path function is where the task’s waiter structure is created on the stack. This is because the waiter structure is only needed for the scope of this function. The waiter structure holds the nodes to store the task on the waiters tree of the mutex, and if need be, the pi_waiters tree of the owner.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hdThe wait_lock of the mutex is taken since the slow path of unlocking the mutex also takes this lock.h]hdThe wait_lock of the mutex is taken since the slow path of unlocking the mutex also takes this lock.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hWe then call try_to_take_rt_mutex. This is where the architecture that does not implement CMPXCHG would always grab the lock (if there's no contention).h]hWe then call try_to_take_rt_mutex. This is where the architecture that does not implement CMPXCHG would always grab the lock (if there’s no contention).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXtry_to_take_rt_mutex is used every time the task tries to grab a mutex in the slow path. The first thing that is done here is an atomic setting of the "Has Waiters" flag of the mutex's owner field. By setting this flag now, the current owner of the mutex being contended for can't release the mutex without going into the slow unlock path, and it would then need to grab the wait_lock, which this code currently holds. So setting the "Has Waiters" flag forces the current owner to synchronize with this code.h]hX try_to_take_rt_mutex is used every time the task tries to grab a mutex in the slow path. The first thing that is done here is an atomic setting of the “Has Waiters” flag of the mutex’s owner field. By setting this flag now, the current owner of the mutex being contended for can’t release the mutex without going into the slow unlock path, and it would then need to grab the wait_lock, which this code currently holds. So setting the “Has Waiters” flag forces the current owner to synchronize with this code.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(h,The lock is taken if the following are true:h]h,The lock is taken if the following are true:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh block_quote)}(hn1) The lock has no owner 2) The current task is the highest priority against all other waiters of the lock h]henumerated_list)}(hhh](j)}(hThe lock has no ownerh]h)}(hj4h]hThe lock has no owner}(hj6hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj2ubah}(h]h ]h"]h$]h&]uh1jhj/ubj)}(hOThe current task is the highest priority against all other waiters of the lock h]h)}(hNThe current task is the highest priority against all other waiters of the lockh]hNThe current task is the highest priority against all other waiters of the lock}(hjMhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjIubah}(h]h ]h"]h$]h&]uh1jhj/ubeh}(h]h ]h"]h$]h&]enumtypearabicprefixhsuffix)uh1j-hj)ubah}(h]h ]h"]h$]h&]uh1j'hhhMhjhhubh)}(hIf the task succeeds to acquire the lock, then the task is set as the owner of the lock, and if the lock still has waiters, the top_waiter (highest priority task waiting on the lock) is added to this task's pi_waiters tree.h]hIf the task succeeds to acquire the lock, then the task is set as the owner of the lock, and if the lock still has waiters, the top_waiter (highest priority task waiting on the lock) is added to this task’s pi_waiters tree.}(hjrhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXIf the lock is not taken by try_to_take_rt_mutex(), then the task_blocks_on_rt_mutex() function is called. This will add the task to the lock's waiter tree and propagate the pi chain of the lock as well as the lock's owner's pi_waiters tree. This is described in the next section.h]hXIf the lock is not taken by try_to_take_rt_mutex(), then the task_blocks_on_rt_mutex() function is called. This will add the task to the lock’s waiter tree and propagate the pi chain of the lock as well as the lock’s owner’s pi_waiters tree. This is described in the next section.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubeh}(h]"taking-of-a-mutex-the-walk-throughah ]h"]$taking of a mutex (the walk through)ah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hTask blocks on mutexh]hTask blocks on mutex}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubh)}(hThe accounting of a mutex and process is done with the waiter structure of the process. The "task" field is set to the process, and the "lock" field to the mutex. The rbtree node of waiter are initialized to the processes current priority.h]hThe accounting of a mutex and process is done with the waiter structure of the process. The “task” field is set to the process, and the “lock” field to the mutex. The rbtree node of waiter are initialized to the processes current priority.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXSince the wait_lock was taken at the entry of the slow lock, we can safely add the waiter to the task waiter tree. If the current process is the highest priority process currently waiting on this mutex, then we remove the previous top waiter process (if it exists) from the pi_waiters of the owner, and add the current process to that tree. Since the pi_waiter of the owner has changed, we call rt_mutex_adjust_prio on the owner to see if the owner should adjust its priority accordingly.h]hXSince the wait_lock was taken at the entry of the slow lock, we can safely add the waiter to the task waiter tree. If the current process is the highest priority process currently waiting on this mutex, then we remove the previous top waiter process (if it exists) from the pi_waiters of the owner, and add the current process to that tree. Since the pi_waiter of the owner has changed, we call rt_mutex_adjust_prio on the owner to see if the owner should adjust its priority accordingly.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hIf the owner is also blocked on a lock, and had its pi_waiters changed (or deadlock checking is on), we unlock the wait_lock of the mutex and go ahead and run rt_mutex_adjust_prio_chain on the owner, as described earlier.h]hIf the owner is also blocked on a lock, and had its pi_waiters changed (or deadlock checking is on), we unlock the wait_lock of the mutex and go ahead and run rt_mutex_adjust_prio_chain on the owner, as described earlier.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hNow all locks are released, and if the current process is still blocked on a mutex (waiter "task" field is not NULL), then we go to sleep (call schedule).h]hNow all locks are released, and if the current process is still blocked on a mutex (waiter “task” field is not NULL), then we go to sleep (call schedule).}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubeh}(h]task-blocks-on-mutexah ]h"]task blocks on mutexah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hWaking up in the looph]hWaking up in the loop}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhjhhhhhMubj)}(hhh]j)}(hThe task can then wake up for a couple of reasons: 1) The previous lock owner released the lock, and the task now is top_waiter 2) we received a signal or timeout h](j)}(h2The task can then wake up for a couple of reasons:h]h2The task can then wake up for a couple of reasons:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhMhjubj)}(hhh]j.)}(hhh](j)}(hIThe previous lock owner released the lock, and the task now is top_waiterh]h)}(hj h]hIThe previous lock owner released the lock, and the task now is top_waiter}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj ubah}(h]h ]h"]h$]h&]uh1jhj ubj)}(h we received a signal or timeout h]h)}(hwe received a signal or timeouth]hwe received a signal or timeout}(hj. hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj* ubah}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]jgjhjihjjjkuh1j-hj ubah}(h]h ]h"]h$]h&]uh1jhjubeh}(h]h ]h"]h$]h&]uh1jhhhMhjubah}(h]h ]h"]h$]h&]uh1jhjhhhNhNubh)}(hIn both cases, the task will try again to acquire the lock. If it does, then it will take itself off the waiters tree and set itself back to the TASK_RUNNING state.h]hIn both cases, the task will try again to acquire the lock. If it does, then it will take itself off the waiters tree and set itself back to the TASK_RUNNING state.}(hjZ hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hIn first case, if the lock was acquired by another task before this task could get the lock, then it will go back to sleep and wait to be woken again.h]hIn first case, if the lock was acquired by another task before this task could get the lock, then it will go back to sleep and wait to be woken again.}(hjh hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubh)}(hXThe second case is only applicable for tasks that are grabbing a mutex that can wake up before getting the lock, either due to a signal or a timeout (i.e. rt_mutex_timed_futex_lock()). When woken, it will try to take the lock again, if it succeeds, then the task will return with the lock held, otherwise it will return with -EINTR if the task was woken by a signal, or -ETIMEDOUT if it timed out.h]hXThe second case is only applicable for tasks that are grabbing a mutex that can wake up before getting the lock, either due to a signal or a timeout (i.e. rt_mutex_timed_futex_lock()). When woken, it will try to take the lock again, if it succeeds, then the task will return with the lock held, otherwise it will return with -EINTR if the task was woken by a signal, or -ETIMEDOUT if it timed out.}(hjv hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhjhhubeh}(h]waking-up-in-the-loopah ]h"]waking up in the loopah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hUnlocking the Mutexh]hUnlocking the Mutex}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhMubh)}(hXThe unlocking of a mutex also has a fast path for those architectures with CMPXCHG. Since the taking of a mutex on contention always sets the "Has Waiters" flag of the mutex's owner, we use this to know if we need to take the slow path when unlocking the mutex. If the mutex doesn't have any waiters, the owner field of the mutex would equal the current process and the mutex can be unlocked by just replacing the owner field with NULL.h]hXThe unlocking of a mutex also has a fast path for those architectures with CMPXCHG. Since the taking of a mutex on contention always sets the “Has Waiters” flag of the mutex’s owner, we use this to know if we need to take the slow path when unlocking the mutex. If the mutex doesn’t have any waiters, the owner field of the mutex would equal the current process and the mutex can be unlocked by just replacing the owner field with NULL.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hnIf the owner field has the "Has Waiters" bit set (or CMPXCHG is not available), the slow unlock path is taken.h]hrIf the owner field has the “Has Waiters” bit set (or CMPXCHG is not available), the slow unlock path is taken.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM hj hhubh)}(hThe first thing done in the slow unlock path is to take the wait_lock of the mutex. This synchronizes the locking and unlocking of the mutex.h]hThe first thing done in the slow unlock path is to take the wait_lock of the mutex. This synchronizes the locking and unlocking of the mutex.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hXA check is made to see if the mutex has waiters or not. On architectures that do not have CMPXCHG, this is the location that the owner of the mutex will determine if a waiter needs to be awoken or not. On architectures that do have CMPXCHG, that check is done in the fast path, but it is still needed in the slow path too. If a waiter of a mutex woke up because of a signal or timeout between the time the owner failed the fast path CMPXCHG check and the grabbing of the wait_lock, the mutex may not have any waiters, thus the owner still needs to make this check. If there are no waiters then the mutex owner field is set to NULL, the wait_lock is released and nothing more is needed.h]hXA check is made to see if the mutex has waiters or not. On architectures that do not have CMPXCHG, this is the location that the owner of the mutex will determine if a waiter needs to be awoken or not. On architectures that do have CMPXCHG, that check is done in the fast path, but it is still needed in the slow path too. If a waiter of a mutex woke up because of a signal or timeout between the time the owner failed the fast path CMPXCHG check and the grabbing of the wait_lock, the mutex may not have any waiters, thus the owner still needs to make this check. If there are no waiters then the mutex owner field is set to NULL, the wait_lock is released and nothing more is needed.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(h2If there are waiters, then we need to wake one up.h]h2If there are waiters, then we need to wake one up.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hX On the wake up code, the pi_lock of the current owner is taken. The top waiter of the lock is found and removed from the waiters tree of the mutex as well as the pi_waiters tree of the current owner. The "Has Waiters" bit is marked to prevent lower priority tasks from stealing the lock.h]hX$On the wake up code, the pi_lock of the current owner is taken. The top waiter of the lock is found and removed from the waiters tree of the mutex as well as the pi_waiters tree of the current owner. The “Has Waiters” bit is marked to prevent lower priority tasks from stealing the lock.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj hhubh)}(hBFinally we unlock the pi_lock of the pending owner and wake it up.h]hBFinally we unlock the pi_lock of the pending owner and wake it up.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM$hj hhubeh}(h]unlocking-the-mutexah ]h"]unlocking the mutexah$]h&]uh1hhhhhhhhMubh)}(hhh](h)}(hContacth]hContact}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhM(ubh)}(hOFor updates on this document, please email Steven Rostedt h](h;For updates on this document, please email Steven Rostedt <}(hj hhhNhNubh reference)}(hrostedt@goodmis.orgh]hrostedt@goodmis.org}(hj" hhhNhNubah}(h]h ]h"]h$]h&]refurimailto:rostedt@goodmis.orguh1j hj ubh>}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM*hj hhubeh}(h]contactah ]h"]contactah$]h&]uh1hhhhhhhhM(ubh)}(hhh](h)}(hCreditsh]hCredits}(hjG hhhNhNubah}(h]h ]h"]h$]h&]uh1hhjD hhhhhM.ubh)}(h-Author: Steven Rostedt h](hAuthor: Steven Rostedt <}(hjU hhhNhNubj! )}(hrostedt@goodmis.orgh]hrostedt@goodmis.org}(hj] hhhNhNubah}(h]h ]h"]h$]h&]refurimailto:rostedt@goodmis.orguh1j hjU ubh>}(hjU hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM0hjD hhubh)}(h2Updated: Alex Shi - 7/6/2017h](hUpdated: Alex Shi <}(hjw hhhNhNubj! )}(halex.shi@linaro.orgh]halex.shi@linaro.org}(hj hhhNhNubah}(h]h ]h"]h$]h&]refurimailto:alex.shi@linaro.orguh1j hjw ubh > - 7/6/2017}(hjw hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM2hjD hhubj)}(hhh]j)}(hSOriginal Reviewers: Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and Randy Dunlap h](j)}(hOriginal Reviewers:h]hOriginal Reviewers:}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhhhM6hj ubj)}(hhh]h)}(h>Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and Randy Dunlaph]h>Ingo Molnar, Thomas Gleixner, Thomas Duetsch, and Randy Dunlap}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM5hj ubah}(h]h ]h"]h$]h&]uh1jhj ubeh}(h]h ]h"]h$]h&]uh1jhhhM6hj ubah}(h]h ]h"]h$]h&]uh1jhjD hhhhhNubh)}(hAUpdate (7/6/2017) Reviewers: Steven Rostedt and Sebastian Siewiorh]hAUpdate (7/6/2017) Reviewers: Steven Rostedt and Sebastian Siewior}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM8hjD hhubeh}(h]creditsah ]h"]creditsah$]h&]uh1hhhhhhhhM.ubh)}(hhh](h)}(hUpdatesh]hUpdates}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhj hhhhhM;ubh)}(hKThis document was originally written for 2.6.17-rc3-mm1 was updated on 4.12h]hKThis document was originally written for 2.6.17-rc3-mm1 was updated on 4.12}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhM=hj hhubeh}(h]updatesah ]h"]updatesah$]h&]uh1hhhhhhhhM;ubeh}(h]rt-mutex-implementation-designah ]h"]rt-mutex implementation designah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksentryfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerj9 error_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourceh _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}nameids}(j j jCj@jjjjjjj?j<jjjjj$j!jjjjjjjjjjj j j j jA j> j j j j u nametypes}(j jCjjjj?jjj$jjjjjj j jA j j uh}(j hj@hjjFjjjjj<jjjBjjj!jjj'jjjjjjjjj jj j j> j j jD j j u footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}Rparse_messages]transform_messages] transformerN include_log] decorationNhhub.