psphinx.addnodesdocument)}( rawsourcechildren]( translations LanguagesNode)}(hhh](h pending_xref)}(hhh]docutils.nodesTextChinese (Simplified)}parenthsba attributes}(ids]classes]names]dupnames]backrefs] refdomainstdreftypedoc reftarget?/translations/zh_CN/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicitutagnamehhh ubh)}(hhh]hChinese (Traditional)}hh2sbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/zh_TW/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hItalian}hhFsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/it_IT/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hJapanese}hhZsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/ja_JP/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hKorean}hhnsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/ko_KR/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hPortuguese (Brazilian)}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/pt_BR/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubh)}(hhh]hSpanish}hhsbah}(h]h ]h"]h$]h&] refdomainh)reftypeh+ reftarget?/translations/sp_SP/process/debugging/userspace_debugging_guidemodnameN classnameN refexplicituh1hhh ubeh}(h]h ]h"]h$]h&]current_languageEnglishuh1h hh _documenthsourceNlineNubhcomment)}(h SPDX-License-Identifier: GPL-2.0h]h SPDX-License-Identifier: GPL-2.0}hhsbah}(h]h ]h"]h$]h&] xml:spacepreserveuh1hhhhhhY/var/lib/git/docbuild/linux/Documentation/process/debugging/userspace_debugging_guide.rsthKubhsection)}(hhh](htitle)}(hUserspace debugging adviceh]hUserspace debugging advice}(hhhhhNhNubah}(h]h ]h"]h$]h&]refidid1uh1hhhhhhhhKubh paragraph)}(hX6This document provides a brief overview of common tools to debug the Linux Kernel from userspace. For debugging advice aimed at driver developers go :doc:`here `. For general debugging advice, see :doc:`general advice document `.h](hThis document provides a brief overview of common tools to debug the Linux Kernel from userspace. For debugging advice aimed at driver developers go }(hhhhhNhNubh)}(hC:doc:`here `h]hinline)}(hhh]hhere}(hhhhhNhNubah}(h]h ](xrefstdstd-doceh"]h$]h&]uh1hhhubah}(h]h ]h"]h$]h&]refdoc+process/debugging/userspace_debugging_guide refdomainhreftypedoc refexplicitrefwarn reftarget5/process/debugging/driver_development_debugging_guideuh1hhhhKhhubh$. For general debugging advice, see }(hhhhhNhNubh)}(h9:doc:`general advice document `h]h)}(hjh]hgeneral advice document}(hjhhhNhNubah}(h]h ](hstdstd-doceh"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]refdocj refdomainj reftypedoc refexplicitrefwarnj /process/debugging/indexuh1hhhhKhhubh.}(hhhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhhhhubhtopic)}(h Contents h](h)}(hContentsh]hContents}(hjBhhhNhNubah}(h]h ]h"]h$]h&]uh1hhj>hhhKubh bullet_list)}(hhh]h list_item)}(hhh](h)}(hhh]h reference)}(hhh]hUserspace debugging advice}(hj_hhhNhNubah}(h]hah ]h"]h$]h&]refiduserspace-debugging-adviceuh1j]hjZubah}(h]h ]h"]h$]h&]uh1hhjWubjQ)}(hhh](jV)}(hhh]h)}(hhh]j^)}(hhh]h Dynamic debug}(hj}hhhNhNubah}(h]id2ah ]h"]h$]h&]refid dynamic-debuguh1j]hjzubah}(h]h ]h"]h$]h&]uh1hhjwubah}(h]h ]h"]h$]h&]uh1jUhjtubjV)}(hhh](h)}(hhh]j^)}(hhh]hFtrace}(hjhhhNhNubah}(h]id3ah ]h"]h$]h&]refidftraceuh1j]hjubah}(h]h ]h"]h$]h&]uh1hhjubjQ)}(hhh](jV)}(hhh]h)}(hhh]j^)}(hhh]hReading the ftrace log}(hjhhhNhNubah}(h]id4ah ]h"]h$]h&]refidreading-the-ftrace-loguh1j]hjubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]uh1jUhjubjV)}(hhh]h)}(hhh]j^)}(hhh]h Kernelshark}(hjhhhNhNubah}(h]id5ah ]h"]h$]h&]refid kernelsharkuh1j]hjubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]uh1jUhjubeh}(h]h ]h"]h$]h&]uh1jPhjubeh}(h]h ]h"]h$]h&]uh1jUhjtubjV)}(hhh](h)}(hhh]j^)}(hhh]hPerf & alternatives}(hjhhhNhNubah}(h]id6ah ]h"]h$]h&]refidperf-alternativesuh1j]hj ubah}(h]h ]h"]h$]h&]uh1hhjubjQ)}(hhh](jV)}(hhh]h)}(hhh]j^)}(hhh]h)Why should you do a performance analysis?}(hj-hhhNhNubah}(h]id7ah ]h"]h$]h&]refid(why-should-you-do-a-performance-analysisuh1j]hj*ubah}(h]h ]h"]h$]h&]uh1hhj'ubah}(h]h ]h"]h$]h&]uh1jUhj$ubjV)}(hhh]h)}(hhh]j^)}(hhh]h-How to do a simple analysis with linux tools?}(hjOhhhNhNubah}(h]id8ah ]h"]h$]h&]refid,how-to-do-a-simple-analysis-with-linux-toolsuh1j]hjLubah}(h]h ]h"]h$]h&]uh1hhjIubah}(h]h ]h"]h$]h&]uh1jUhj$ubjV)}(hhh]h)}(hhh]j^)}(hhh]hDiving deeper with perf}(hjqhhhNhNubah}(h]id9ah ]h"]h$]h&]refiddiving-deeper-with-perfuh1j]hjnubah}(h]h ]h"]h$]h&]uh1hhjkubah}(h]h ]h"]h$]h&]uh1jUhj$ubjV)}(hhh]h)}(hhh]j^)}(hhh]hPerfetto}(hjhhhNhNubah}(h]id10ah ]h"]h$]h&]refidperfettouh1j]hjubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]uh1jUhj$ubeh}(h]h ]h"]h$]h&]uh1jPhjubeh}(h]h ]h"]h$]h&]uh1jUhjtubjV)}(hhh]h)}(hhh]j^)}(hhh]hKernel panic analysis tools}(hjhhhNhNubah}(h]id11ah ]h"]h$]h&]refidkernel-panic-analysis-toolsuh1j]hjubah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]uh1jUhjtubeh}(h]h ]h"]h$]h&]uh1jPhjWubeh}(h]h ]h"]h$]h&]uh1jUhjRubah}(h]h ]h"]h$]h&]uh1jPhj>hhhNhNubeh}(h]contentsah ]contentsah"]contentsah$]h&]uh1j<hhhKhhhhubh)}(h4The following sections show you the available tools.h]h4The following sections show you the available tools.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhhhhubh)}(hhh](h)}(h Dynamic debugh]h Dynamic debug}(hj hhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhjhhhhhKubh)}(hRMechanism to filter what ends up in the kernel log by dis-/en-abling log messages.h]hRMechanism to filter what ends up in the kernel log by dis-/en-abling log messages.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h&Prerequisite: ``CONFIG_DYNAMIC_DEBUG``h](hPrerequisite: }(hj%hhhNhNubhliteral)}(h``CONFIG_DYNAMIC_DEBUG``h]hCONFIG_DYNAMIC_DEBUG}(hj/hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj%ubeh}(h]h ]h"]h$]h&]uh1hhhhKhjhhubh)}(h%Dynamic debug is only able to target:h]h%Dynamic debug is only able to target:}(hjChhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjQ)}(hhh](jV)}(h pr_debug()h]h)}(hjVh]h pr_debug()}(hjXhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjTubah}(h]h ]h"]h$]h&]uh1jUhjQhhhhhNubjV)}(h dev_dbg()h]h)}(hjmh]h dev_dbg()}(hjohhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjkubah}(h]h ]h"]h$]h&]uh1jUhjQhhhhhNubjV)}(hprint_hex_dump_debug()h]h)}(hjh]hprint_hex_dump_debug()}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jUhjQhhhhhNubjV)}(hprint_hex_dump_bytes() h]h)}(hprint_hex_dump_bytes()h]hprint_hex_dump_bytes()}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK hjubah}(h]h ]h"]h$]h&]uh1jUhjQhhhhhNubeh}(h]h ]h"]h$]h&]bullet-uh1jPhhhKhjhhubh)}(hTherefore the usability of this tool is, as of now, quite limited as there is no uniform rule for adding debug prints to the codebase, resulting in a variety of ways these prints are implemented.h]hTherefore the usability of this tool is, as of now, quite limited as there is no uniform rule for adding debug prints to the codebase, resulting in a variety of ways these prints are implemented.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK"hjhhubh)}(hAlso, note that most debug statements are implemented as a variation of dprintk(), which have to be activated via a parameter in respective module, dynamic debug is unable to do that step for you.h]hAlso, note that most debug statements are implemented as a variation of dprintk(), which have to be activated via a parameter in respective module, dynamic debug is unable to do that step for you.}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK&hjhhubh)}(hNHere is one example, that enables all available pr_debug()'s within the file::h]hOHere is one example, that enables all available pr_debug()’s within the file:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK*hjhhubh literal_block)}(hXY$ alias ddcmd='echo $* > /proc/dynamic_debug/control' $ ddcmd '-p; file v4l2-h264.c +p' $ grep =p /proc/dynamic_debug/control drivers/media/v4l2-core/v4l2-h264.c:372 [v4l2_h264]print_ref_list_b =p "ref_pic_list_b%u (cur_poc %u%c) %s" drivers/media/v4l2-core/v4l2-h264.c:333 [v4l2_h264]print_ref_list_p =p "ref_pic_list_p (cur_poc %u%c) %s\n"h]hXY$ alias ddcmd='echo $* > /proc/dynamic_debug/control' $ ddcmd '-p; file v4l2-h264.c +p' $ grep =p /proc/dynamic_debug/control drivers/media/v4l2-core/v4l2-h264.c:372 [v4l2_h264]print_ref_list_b =p "ref_pic_list_b%u (cur_poc %u%c) %s" drivers/media/v4l2-core/v4l2-h264.c:333 [v4l2_h264]print_ref_list_p =p "ref_pic_list_p (cur_poc %u%c) %s\n"}hjsbah}(h]h ]h"]h$]h&]hhuh1jhhhK,hjhhubh)}(h***When should you use this over Ftrace ?**h]hstrong)}(hjh]h&When should you use this over Ftrace ?}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1jhjubah}(h]h ]h"]h$]h&]uh1hhhhK4hjhhubjQ)}(hhh](jV)}(hWhen the code contains one of the valid print statements (see above) or when you have added multiple pr_debug() statements during developmenth]h)}(hWhen the code contains one of the valid print statements (see above) or when you have added multiple pr_debug() statements during developmenth]hWhen the code contains one of the valid print statements (see above) or when you have added multiple pr_debug() statements during development}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK6hjubah}(h]h ]h"]h$]h&]uh1jUhj hhhhhNubjV)}(heWhen timing is not an issue, meaning if multiple pr_debug() statements in the code won't cause delaysh]h)}(heWhen timing is not an issue, meaning if multiple pr_debug() statements in the code won't cause delaysh]hgWhen timing is not an issue, meaning if multiple pr_debug() statements in the code won’t cause delays}(hj+hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK8hj'ubah}(h]h ]h"]h$]h&]uh1jUhj hhhhhNubjV)}(hnWhen you care more about receiving specific log messages than tracing the pattern of how a function is called h]h)}(hmWhen you care more about receiving specific log messages than tracing the pattern of how a function is calledh]hmWhen you care more about receiving specific log messages than tracing the pattern of how a function is called}(hjChhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK:hj?ubah}(h]h ]h"]h$]h&]uh1jUhj hhhhhNubeh}(h]h ]h"]h$]h&]jjuh1jPhhhK6hjhhubh)}(hFFor the full documentation see :doc:`/admin-guide/dynamic-debug-howto`h](hFor the full documentation see }(hj]hhhNhNubh)}(h':doc:`/admin-guide/dynamic-debug-howto`h]h)}(hjgh]h /admin-guide/dynamic-debug-howto}(hjihhhNhNubah}(h]h ](hstdstd-doceh"]h$]h&]uh1hhjeubah}(h]h ]h"]h$]h&]refdocj refdomainjsreftypedoc refexplicitrefwarnj  /admin-guide/dynamic-debug-howtouh1hhhhK=hj]ubeh}(h]h ]h"]h$]h&]uh1hhhhK=hjhhubeh}(h]jah ]h"] dynamic debugah$]h&]uh1hhhhhhhhKubh)}(hhh](h)}(hFtraceh]hFtrace}(hjhhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhjhhhhhK@ubh)}(h'Prerequisite: ``CONFIG_DYNAMIC_FTRACE``h](hPrerequisite: }(hjhhhNhNubj.)}(h``CONFIG_DYNAMIC_FTRACE``h]hCONFIG_DYNAMIC_FTRACE}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubeh}(h]h ]h"]h$]h&]uh1hhhhKBhjhhubh)}(hThis tool uses the tracefs file system for the control files and output files. That file system will be mounted as a ``tracing`` directory, which can be found in either ``/sys/kernel/`` or ``/sys/debug/kernel/``.h](huThis tool uses the tracefs file system for the control files and output files. That file system will be mounted as a }(hjhhhNhNubj.)}(h ``tracing``h]htracing}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh) directory, which can be found in either }(hjhhhNhNubj.)}(h``/sys/kernel/``h]h /sys/kernel/}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh or }(hjhhhNhNubj.)}(h``/sys/debug/kernel/``h]h/sys/debug/kernel/}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKDhjhhubh)}(h8Some of the most important operations for debugging are:h]h8Some of the most important operations for debugging are:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKHhjhhubjQ)}(hhh](jV)}(hXRYou can perform a function trace by adding a function name to the ``set_ftrace_filter`` file (which accepts any function name found within the ``available_filter_functions`` file) or you can specifically disable certain functions by adding their names to the ``set_ftrace_notrace`` file (more info at: :ref:`trace/ftrace:dynamic ftrace`).h]h)}(hXRYou can perform a function trace by adding a function name to the ``set_ftrace_filter`` file (which accepts any function name found within the ``available_filter_functions`` file) or you can specifically disable certain functions by adding their names to the ``set_ftrace_notrace`` file (more info at: :ref:`trace/ftrace:dynamic ftrace`).h](hBYou can perform a function trace by adding a function name to the }(hjhhhNhNubj.)}(h``set_ftrace_filter``h]hset_ftrace_filter}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh8 file (which accepts any function name found within the }(hjhhhNhNubj.)}(h``available_filter_functions``h]havailable_filter_functions}(hj2hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubhV file) or you can specifically disable certain functions by adding their names to the }(hjhhhNhNubj.)}(h``set_ftrace_notrace``h]hset_ftrace_notrace}(hjDhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh file (more info at: }(hjhhhNhNubh)}(h":ref:`trace/ftrace:dynamic ftrace`h]h)}(hjXh]htrace/ftrace:dynamic ftrace}(hjZhhhNhNubah}(h]h ](hstdstd-refeh"]h$]h&]uh1hhjVubah}(h]h ]h"]h$]h&]refdocj refdomainjdreftyperef refexplicitrefwarnj trace/ftrace:dynamic ftraceuh1hhhhKJhjubh).}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKJhjubah}(h]h ]h"]h$]h&]uh1jUhjhhhhhNubjV)}(hIn order to find out where calls originate from you can activate the ``func_stack_trace`` option under ``options/func_stack_trace``.h]h)}(hIn order to find out where calls originate from you can activate the ``func_stack_trace`` option under ``options/func_stack_trace``.h](hEIn order to find out where calls originate from you can activate the }(hjhhhNhNubj.)}(h``func_stack_trace``h]hfunc_stack_trace}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh option under }(hjhhhNhNubj.)}(h``options/func_stack_trace``h]hoptions/func_stack_trace}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKOhjubah}(h]h ]h"]h$]h&]uh1jUhjhhhhhNubjV)}(hXTracing the children of a function call and showing the return values are possible by adding the desired function in the ``set_graph_function`` file (requires config ``FUNCTION_GRAPH_RETVAL``); more info at :ref:`trace/ftrace:dynamic ftrace with the function graph tracer`. h]h)}(hXTracing the children of a function call and showing the return values are possible by adding the desired function in the ``set_graph_function`` file (requires config ``FUNCTION_GRAPH_RETVAL``); more info at :ref:`trace/ftrace:dynamic ftrace with the function graph tracer`.h](hyTracing the children of a function call and showing the return values are possible by adding the desired function in the }(hjhhhNhNubj.)}(h``set_graph_function``h]hset_graph_function}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh file (requires config }(hjhhhNhNubj.)}(h``FUNCTION_GRAPH_RETVAL``h]hFUNCTION_GRAPH_RETVAL}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh); more info at }(hjhhhNhNubh)}(hA:ref:`trace/ftrace:dynamic ftrace with the function graph tracer`h]h)}(hjh]h:trace/ftrace:dynamic ftrace with the function graph tracer}(hjhhhNhNubah}(h]h ](hstdstd-refeh"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]refdocj refdomainjreftyperef refexplicitrefwarnj :trace/ftrace:dynamic ftrace with the function graph traceruh1hhhhKQhjubh.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKQhjubah}(h]h ]h"]h$]h&]uh1jUhjhhhhhNubeh}(h]h ]h"]h$]h&]jjuh1jPhhhKJhjhhubh)}(h:For the full Ftrace documentation see :doc:`/trace/ftrace`h](h&For the full Ftrace documentation see }(hj(hhhNhNubh)}(h:doc:`/trace/ftrace`h]h)}(hj2h]h /trace/ftrace}(hj4hhhNhNubah}(h]h ](hstdstd-doceh"]h$]h&]uh1hhj0ubah}(h]h ]h"]h$]h&]refdocj refdomainj>reftypedoc refexplicitrefwarnj  /trace/ftraceuh1hhhhKVhj(ubeh}(h]h ]h"]h$]h&]uh1hhhhKVhjhhubh)}(hXOr you could also trace for specific events by :ref:`using event tracing `, which can be defined as described here: :ref:`Creating a custom Ftrace tracepoint `.h](h/Or you could also trace for specific events by }(hjVhhhNhNubh)}(h@:ref:`using event tracing `h]h)}(hj`h]husing event tracing}(hjbhhhNhNubah}(h]h ](hstdstd-refeh"]h$]h&]uh1hhj^ubah}(h]h ]h"]h$]h&]refdocj refdomainjlreftyperef refexplicitrefwarnj #trace/events:2. using event tracinguh1hhhhKXhjVubh*, which can be defined as described here: }(hjVhhhNhNubh)}(hh:ref:`Creating a custom Ftrace tracepoint `h]h)}(hjh]h#Creating a custom Ftrace tracepoint}(hjhhhNhNubah}(h]h ](hstdstd-refeh"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]refdocj refdomainjreftyperef refexplicitrefwarnj ;process/debugging/driver_development_debugging_guide:ftraceuh1hhhhKXhjVubh.}(hjVhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKXhjhhubh)}(hHFor the full Ftrace event tracing documentation see :doc:`/trace/events`h](h4For the full Ftrace event tracing documentation see }(hjhhhNhNubh)}(h:doc:`/trace/events`h]h)}(hjh]h /trace/events}(hjhhhNhNubah}(h]h ](hstdstd-doceh"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]refdocj refdomainjreftypedoc refexplicitrefwarnj  /trace/eventsuh1hhhhK]hjubeh}(h]h ]h"]h$]h&]uh1hhhhK]hjhhubhtarget)}(h.. _read_ftrace_log:h]h}(h]h ]h"]h$]h&]h݌read-ftrace-loguh1jhK_hjhhhhubh)}(hhh](h)}(hReading the ftrace logh]hReading the ftrace log}(hjhhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhjhhhhhKbubh)}(hXNThe ``trace`` file can be read just like any other file (``cat``, ``tail``, ``head``, ``vim``, etc.), the size of the file is limited by the ``buffer_size_kb`` (``echo 1000 > buffer_size_kb``). The :ref:`trace/ftrace:trace_pipe` will behave similarly to the ``trace`` file, but whenever you read from the file the content is consumed.h](hThe }(hjhhhNhNubj.)}(h ``trace``h]htrace}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh, file can be read just like any other file (}(hjhhhNhNubj.)}(h``cat``h]hcat}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh, }(hjhhhNhNubj.)}(h``tail``h]htail}(hj$hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh, }(hjhhhNhNubj.)}(h``head``h]hhead}(hj6hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh, }hjsbj.)}(h``vim``h]hvim}(hjHhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh0, etc.), the size of the file is limited by the }(hjhhhNhNubj.)}(h``buffer_size_kb``h]hbuffer_size_kb}(hjZhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh (}(hjhhhNhNubj.)}(h``echo 1000 > buffer_size_kb``h]hecho 1000 > buffer_size_kb}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh). The }(hjhhhNhNubh)}(h:ref:`trace/ftrace:trace_pipe`h]h)}(hjh]htrace/ftrace:trace_pipe}(hjhhhNhNubah}(h]h ](hstdstd-refeh"]h$]h&]uh1hhj~ubah}(h]h ]h"]h$]h&]refdocj refdomainjreftyperef refexplicitrefwarnj trace/ftrace:trace_pipeuh1hhhhKdhjubh will behave similarly to the }(hjhhhNhNubj.)}(h ``trace``h]htrace}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubhC file, but whenever you read from the file the content is consumed.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKdhjhhubeh}(h](jjeh ]h"](reading the ftrace logread_ftrace_logeh$]h&]uh1hhjhhhhhKbexpect_referenced_by_name}jjsexpect_referenced_by_id}jjsubh)}(hhh](h)}(h Kernelsharkh]h Kernelshark}(hjhhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhjhhhhhKkubh)}(hA GUI interface to visualize the traces as a graph and list view from the output of the `trace-cmd `__ application.h](hXA GUI interface to visualize the traces as a graph and list view from the output of the }(hjhhhNhNubj^)}(hM`trace-cmd `__h]h trace-cmd}(hjhhhNhNubah}(h]h ]h"]h$]h&]name trace-cmdrefuri=https://git.kernel.org/pub/scm/utils/trace-cmd/trace-cmd.git/uh1j]hjubh application.}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKmhjhhubh)}(hOFor the full documentation see ``__h](hFor the full documentation see }(hjhhhNhNubj^)}(h0``__h]h*https://kernelshark.org/Documentation.html}(hjhhhNhNubah}(h]h ]h"]h$]h&]name*https://kernelshark.org/Documentation.htmljjuh1j]hjubeh}(h]h ]h"]h$]h&]uh1hhhhKqhjhhubeh}(h]jah ]h"] kernelsharkah$]h&]uh1hhjhhhhhKkubeh}(h]jah ]h"]ftraceah$]h&]uh1hhhhhhhhK@ubh)}(hhh](h)}(hPerf & alternativesh]hPerf & alternatives}(hj*hhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhj'hhhhhKtubh)}(hThe tools mentioned above provide ways to inspect kernel code, results, variable values, etc. Sometimes you have to find out first where to look and for those cases, a box of performance tracking tools can help you to frame the issue.h]hThe tools mentioned above provide ways to inspect kernel code, results, variable values, etc. Sometimes you have to find out first where to look and for those cases, a box of performance tracking tools can help you to frame the issue.}(hj8hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKvhj'hhubh)}(hhh](h)}(h)Why should you do a performance analysis?h]h)Why should you do a performance analysis?}(hjIhhhNhNubah}(h]h ]h"]h$]h&]hj6uh1hhjFhhhhhK|ubh)}(hEA performance analysis is a good first step when among other reasons:h]hEA performance analysis is a good first step when among other reasons:}(hjWhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhK~hjFhhubjQ)}(hhh](jV)}(hyou cannot define the issueh]h)}(hjjh]hyou cannot define the issue}(hjlhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhubah}(h]h ]h"]h$]h&]uh1jUhjehhhhhNubjV)}(hyou do not know where it occursh]h)}(hjh]hyou do not know where it occurs}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jUhjehhhhhNubjV)}(htthe running system should not be interrupted or it is a remote system, where you cannot install a new module/kernel h]h)}(hsthe running system should not be interrupted or it is a remote system, where you cannot install a new module/kernelh]hsthe running system should not be interrupted or it is a remote system, where you cannot install a new module/kernel}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jUhjehhhhhNubeh}(h]h ]h"]h$]h&]jjuh1jPhhhKhjFhhubeh}(h]j<ah ]h"])why should you do a performance analysis?ah$]h&]uh1hhj'hhhhhK|ubh)}(hhh](h)}(h-How to do a simple analysis with linux tools?h]h-How to do a simple analysis with linux tools?}(hjhhhNhNubah}(h]h ]h"]h$]h&]hjXuh1hhjhhhhhKubh)}(hQFor the start of a performance analysis, you can start with the usual tools like:h]hQFor the start of a performance analysis, you can start with the usual tools like:}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjhhubjQ)}(hhh](jV)}(hf``top`` / ``htop`` / ``atop`` (*get an overview of the system load, see spikes on specific processes*)h]h)}(hf``top`` / ``htop`` / ``atop`` (*get an overview of the system load, see spikes on specific processes*)h](j.)}(h``top``h]htop}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh / }(hjhhhNhNubj.)}(h``htop``h]hhtop}(hjhhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh / }hjsbj.)}(h``atop``h]hatop}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjubh (}(hjhhhNhNubhemphasis)}(hF*get an overview of the system load, see spikes on specific processes*h]hDget an overview of the system load, see spikes on specific processes}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j hjubh)}(hjhhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhjubah}(h]h ]h"]h$]h&]uh1jUhjhhhhhNubjV)}(h>``mpstat -P ALL`` (*look at the load distribution among CPUs*)h]h)}(hj= h](j.)}(h``mpstat -P ALL``h]h mpstat -P ALL}(hjB hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj? ubh (}(hj? hhhNhNubj )}(h**look at the load distribution among CPUs*h]h(look at the load distribution among CPUs}(hjT hhhNhNubah}(h]h ]h"]h$]h&]uh1j hj? ubh)}(hj? hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj; ubah}(h]h ]h"]h$]h&]uh1jUhjhhhhhNubjV)}(hN``iostat -x`` (*observe input and output devices utilization and performance*)h]h)}(hjt h](j.)}(h ``iostat -x``h]h iostat -x}(hjy hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjv ubh (}(hjv hhhNhNubj )}(h>*observe input and output devices utilization and performance*h]h LLC-loads LLC-load-misses 1.281746009 seconds time elapsed 0.508796000 seconds user 0.773209000 seconds sys 52h]hX# perf stat -d find /usr -name 'gcc*' | wc -l Performance counter stats for 'find /usr -name gcc*': 1277.81 msec task-clock # 0.997 CPUs utilized 9 context-switches # 7.043 /sec 1 cpu-migrations # 0.783 /sec 704 page-faults # 550.943 /sec 766548897 cycles # 0.600 GHz (97.15%) 798285467 instructions # 1.04 insn per cycle (97.15%) 57582731 branches # 45.064 M/sec (2.85%) 3842573 branch-misses # 6.67% of all branches (97.15%) 281616097 L1-dcache-loads # 220.390 M/sec (97.15%) 4220975 L1-dcache-load-misses # 1.50% of all L1-dcache accesses (97.15%) LLC-loads LLC-load-misses 1.281746009 seconds time elapsed 0.508796000 seconds user 0.773209000 seconds sys 52}hj sbah}(h]h ]h"]h$]h&]hhuh1jhhhKhj hhubh)}(hMThe availability of events and metrics depends on the system you are running.h]hMThe availability of events and metrics depends on the system you are running.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj hhubh)}(hUFor the full documentation see ``__h](hFor the full documentation see }(hj hhhNhNubj^)}(h6``__h]h0https://perf.wiki.kernel.org/index.php/Main_Page}(hj$ hhhNhNubah}(h]h ]h"]h$]h&]name0https://perf.wiki.kernel.org/index.php/Main_Pagejj3 uh1j]hj ubeh}(h]h ]h"]h$]h&]uh1hhhhKhj hhubeh}(h]jah ]h"]diving deeper with perfah$]h&]uh1hhj'hhhhhKubh)}(hhh](h)}(hPerfettoh]hPerfetto}(hjD hhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhjA hhhhhKubh)}(hcA set of tools to measure and analyze how well applications and systems perform. You can use it to:h]hcA set of tools to measure and analyze how well applications and systems perform. You can use it to:}(hjR hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjA hhubjQ)}(hhh](jV)}(hidentify bottlenecksh]h)}(hje h]hidentify bottlenecks}(hjg hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjc ubah}(h]h ]h"]h$]h&]uh1jUhj` hhhhhNubjV)}(h optimize codeh]h)}(hj| h]h optimize code}(hj~ hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhjz ubah}(h]h ]h"]h$]h&]uh1jUhj` hhhhhNubjV)}(h/make software run faster and more efficiently. h]h)}(h.make software run faster and more efficiently.h]h.make software run faster and more efficiently.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj ubah}(h]h ]h"]h$]h&]uh1jUhj` hhhhhNubeh}(h]h ]h"]h$]h&]j*uh1jPhhhKhjA hhubh)}(h5**What is the difference between perfetto and perf?**h]j)}(hj h]h1What is the difference between perfetto and perf?}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubah}(h]h ]h"]h$]h&]uh1hhhhKhjA hhubjQ)}(hhh](jV)}(hXperf is tool as part of and specialized for the Linux Kernel and has CLI user interface.h]h)}(hXperf is tool as part of and specialized for the Linux Kernel and has CLI user interface.h]hXperf is tool as part of and specialized for the Linux Kernel and has CLI user interface.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj ubah}(h]h ]h"]h$]h&]uh1jUhj hhhhhNubjV)}(hperfetto cross-platform performance analysis stack, has extended functionality into userspace and provides a WEB user interface. h]h)}(hperfetto cross-platform performance analysis stack, has extended functionality into userspace and provides a WEB user interface.h]hperfetto cross-platform performance analysis stack, has extended functionality into userspace and provides a WEB user interface.}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj ubah}(h]h ]h"]h$]h&]uh1jUhj hhhhhNubeh}(h]h ]h"]h$]h&]jj uh1jPhhhKhjA hhubh)}(h?For the full documentation see ``__h](hFor the full documentation see }(hj hhhNhNubj^)}(h ``__h]hhttps://perfetto.dev/docs/}(hj hhhNhNubah}(h]h ]h"]h$]h&]namehttps://perfetto.dev/docs/jj uh1j]hj ubeh}(h]h ]h"]h$]h&]uh1hhhhKhjA hhubeh}(h]jah ]h"]perfettoah$]h&]uh1hhj'hhhhhKubeh}(h]jah ]h"]perf & alternativesah$]h&]uh1hhhhhhhhKtubh)}(hhh](h)}(hKernel panic analysis toolsh]hKernel panic analysis tools}(hj/ hhhNhNubah}(h]h ]h"]h$]h&]hjuh1hhj, hhhhhKubh block_quote)}(hX To capture the crash dump please use ``Kdump`` & ``Kexec``. Below you can find some advice for analysing the data. For the full documentation see the :doc:`/admin-guide/kdump/kdump` In order to find the corresponding line in the code you can use `faddr2line `__; note that you need to enable ``CONFIG_DEBUG_INFO`` for that to work. An alternative to using ``faddr2line`` is the use of ``objdump`` (and its derivatives for the different platforms like ``aarch64-linux-gnu-objdump``). Take this line as an example: ``[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]``. We can find the corresponding line of code by executing:: aarch64-linux-gnu-objdump -dS drivers/staging/media/rkvdec/rockchip-vdec.ko | grep rkvdec_device_run\>: -A 40 0000000000000ac8 : ac8: d503201f nop acc: d503201f nop { ad0: d503233f paciasp ad4: a9bd7bfd stp x29, x30, [sp, #-48]! ad8: 910003fd mov x29, sp adc: a90153f3 stp x19, x20, [sp, #16] ae0: a9025bf5 stp x21, x22, [sp, #32] const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; ae4: f9411814 ldr x20, [x0, #560] struct rkvdec_dev *rkvdec = ctx->dev; ae8: f9418015 ldr x21, [x0, #768] if (WARN_ON(!desc)) aec: b4000654 cbz x20, bb4 ret = pm_runtime_resume_and_get(rkvdec->dev); af0: f943d2b6 ldr x22, [x21, #1952] ret = __pm_runtime_resume(dev, RPM_GET_PUT); af4: aa0003f3 mov x19, x0 af8: 52800081 mov w1, #0x4 // #4 afc: aa1603e0 mov x0, x22 b00: 94000000 bl 0 <__pm_runtime_resume> if (ret < 0) { b04: 37f80340 tbnz w0, #31, b6c dev_warn(rkvdec->dev, "Not good\n"); b08: f943d2a0 ldr x0, [x21, #1952] b0c: 90000001 adrp x1, 0 b10: 91000021 add x1, x1, #0x0 b14: 94000000 bl 0 <_dev_warn> *bad = 1; b18: d2800001 mov x1, #0x0 // #0 ... Meaning, in this line from the crash dump:: [ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec] I can take the ``0x50`` as offset, which I have to add to the base address of the corresponding function, which I find in this line:: 0000000000000ac8 : The result of ``0xac8 + 0x50 = 0xb18`` And when I search for that address within the function I get the following line:: *bad = 1; b18: d2800001 mov x1, #0x0 h](h)}(hrTo capture the crash dump please use ``Kdump`` & ``Kexec``. Below you can find some advice for analysing the data.h](h%To capture the crash dump please use }(hjC hhhNhNubj.)}(h ``Kdump``h]hKdump}(hjK hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjC ubh & }(hjC hhhNhNubj.)}(h ``Kexec``h]hKexec}(hj] hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjC ubh8. Below you can find some advice for analysing the data.}(hjC hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj? ubh)}(hBFor the full documentation see the :doc:`/admin-guide/kdump/kdump`h](h#For the full documentation see the }(hju hhhNhNubh)}(h:doc:`/admin-guide/kdump/kdump`h]h)}(hj h]h/admin-guide/kdump/kdump}(hj hhhNhNubah}(h]h ](hstdstd-doceh"]h$]h&]uh1hhj} ubah}(h]h ]h"]h$]h&]refdocj refdomainj reftypedoc refexplicitrefwarnj /admin-guide/kdump/kdumpuh1hhhhKhju ubeh}(h]h ]h"]h$]h&]uh1hhhhKhj? ubh)}(hIn order to find the corresponding line in the code you can use `faddr2line `__; note that you need to enable ``CONFIG_DEBUG_INFO`` for that to work.h](h@In order to find the corresponding line in the code you can use }(hj hhhNhNubj^)}(hS`faddr2line `__h]h faddr2line}(hj hhhNhNubah}(h]h ]h"]h$]h&]name faddr2linejBhttps://elixir.bootlin.com/linux/v6.11.6/source/scripts/faddr2lineuh1j]hj ubh; note that you need to enable }(hj hhhNhNubj.)}(h``CONFIG_DEBUG_INFO``h]hCONFIG_DEBUG_INFO}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubh for that to work.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj? ubh)}(hAn alternative to using ``faddr2line`` is the use of ``objdump`` (and its derivatives for the different platforms like ``aarch64-linux-gnu-objdump``). Take this line as an example:h](hAn alternative to using }(hj hhhNhNubj.)}(h``faddr2line``h]h faddr2line}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubh is the use of }(hj hhhNhNubj.)}(h ``objdump``h]hobjdump}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubh7 (and its derivatives for the different platforms like }(hj hhhNhNubj.)}(h``aarch64-linux-gnu-objdump``h]haarch64-linux-gnu-objdump}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubh ). Take this line as an example:}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj? ubh)}(h@``[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]``.h](j.)}(h?``[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]``h]h;[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubh.}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhKhj? ubh)}(h9We can find the corresponding line of code by executing::h]h8We can find the corresponding line of code by executing:}(hj8 hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhKhj? ubj)}(hX5aarch64-linux-gnu-objdump -dS drivers/staging/media/rkvdec/rockchip-vdec.ko | grep rkvdec_device_run\>: -A 40 0000000000000ac8 : ac8: d503201f nop acc: d503201f nop { ad0: d503233f paciasp ad4: a9bd7bfd stp x29, x30, [sp, #-48]! ad8: 910003fd mov x29, sp adc: a90153f3 stp x19, x20, [sp, #16] ae0: a9025bf5 stp x21, x22, [sp, #32] const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; ae4: f9411814 ldr x20, [x0, #560] struct rkvdec_dev *rkvdec = ctx->dev; ae8: f9418015 ldr x21, [x0, #768] if (WARN_ON(!desc)) aec: b4000654 cbz x20, bb4 ret = pm_runtime_resume_and_get(rkvdec->dev); af0: f943d2b6 ldr x22, [x21, #1952] ret = __pm_runtime_resume(dev, RPM_GET_PUT); af4: aa0003f3 mov x19, x0 af8: 52800081 mov w1, #0x4 // #4 afc: aa1603e0 mov x0, x22 b00: 94000000 bl 0 <__pm_runtime_resume> if (ret < 0) { b04: 37f80340 tbnz w0, #31, b6c dev_warn(rkvdec->dev, "Not good\n"); b08: f943d2a0 ldr x0, [x21, #1952] b0c: 90000001 adrp x1, 0 b10: 91000021 add x1, x1, #0x0 b14: 94000000 bl 0 <_dev_warn> *bad = 1; b18: d2800001 mov x1, #0x0 // #0 ...h]hX5aarch64-linux-gnu-objdump -dS drivers/staging/media/rkvdec/rockchip-vdec.ko | grep rkvdec_device_run\>: -A 40 0000000000000ac8 : ac8: d503201f nop acc: d503201f nop { ad0: d503233f paciasp ad4: a9bd7bfd stp x29, x30, [sp, #-48]! ad8: 910003fd mov x29, sp adc: a90153f3 stp x19, x20, [sp, #16] ae0: a9025bf5 stp x21, x22, [sp, #32] const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc; ae4: f9411814 ldr x20, [x0, #560] struct rkvdec_dev *rkvdec = ctx->dev; ae8: f9418015 ldr x21, [x0, #768] if (WARN_ON(!desc)) aec: b4000654 cbz x20, bb4 ret = pm_runtime_resume_and_get(rkvdec->dev); af0: f943d2b6 ldr x22, [x21, #1952] ret = __pm_runtime_resume(dev, RPM_GET_PUT); af4: aa0003f3 mov x19, x0 af8: 52800081 mov w1, #0x4 // #4 afc: aa1603e0 mov x0, x22 b00: 94000000 bl 0 <__pm_runtime_resume> if (ret < 0) { b04: 37f80340 tbnz w0, #31, b6c dev_warn(rkvdec->dev, "Not good\n"); b08: f943d2a0 ldr x0, [x21, #1952] b0c: 90000001 adrp x1, 0 b10: 91000021 add x1, x1, #0x0 b14: 94000000 bl 0 <_dev_warn> *bad = 1; b18: d2800001 mov x1, #0x0 // #0 ...}hjF sbah}(h]h ]h"]h$]h&]hhuh1jhhhKhj? ubh)}(h+Meaning, in this line from the crash dump::h]h*Meaning, in this line from the crash dump:}(hjT hhhNhNubah}(h]h ]h"]h$]h&]uh1hhhhMhj? ubj)}(h;[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]h]h;[ +0.000240] rkvdec_device_run+0x50/0x138 [rockchip_vdec]}hjb sbah}(h]h ]h"]h$]h&]hhuh1jhhhM hj? ubh)}(hI can take the ``0x50`` as offset, which I have to add to the base address of the corresponding function, which I find in this line::h](hI can take the }(hjp hhhNhNubj.)}(h``0x50``h]h0x50}(hjx hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hjp ubhm as offset, which I have to add to the base address of the corresponding function, which I find in this line:}(hjp hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhM hj? ubj)}(h%0000000000000ac8 :h]h%0000000000000ac8 :}hj sbah}(h]h ]h"]h$]h&]hhuh1jhhhMhj? ubh)}(hxThe result of ``0xac8 + 0x50 = 0xb18`` And when I search for that address within the function I get the following line::h](hThe result of }(hj hhhNhNubj.)}(h``0xac8 + 0x50 = 0xb18``h]h0xac8 + 0x50 = 0xb18}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1j-hj ubhQ And when I search for that address within the function I get the following line:}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj? ubj)}(h4*bad = 1; b18: d2800001 mov x1, #0x0h]h4*bad = 1; b18: d2800001 mov x1, #0x0}hj sbah}(h]h ]h"]h$]h&]hhuh1jhhhMhj? ubeh}(h]h ]h"]h$]h&]uh1j= hhhKhj, hhubh)}(h **Copyright** ©2024 : Collaborah](j)}(h **Copyright**h]h Copyright}(hj hhhNhNubah}(h]h ]h"]h$]h&]uh1jhj ubh ©2024 : Collabora}(hj hhhNhNubeh}(h]h ]h"]h$]h&]uh1hhhhMhj, hhubeh}(h]jah ]h"]kernel panic analysis toolsah$]h&]uh1hhhhhhhhKubeh}(h]jmah ]h"]userspace debugging adviceah$]h&]uh1hhhhhhhhKubeh}(h]h ]h"]h$]h&]sourcehuh1hcurrent_sourceN current_lineNsettingsdocutils.frontendValues)}(hN generatorN datestampN source_linkN source_urlN toc_backlinksentryfootnote_backlinksK sectnum_xformKstrip_commentsNstrip_elements_with_classesN strip_classesN report_levelK halt_levelKexit_status_levelKdebugNwarning_streamN tracebackinput_encoding utf-8-siginput_encoding_error_handlerstrictoutput_encodingutf-8output_encoding_error_handlerjerror_encodingutf-8error_encoding_error_handlerbackslashreplace language_codeenrecord_dependenciesNconfigN id_prefixhauto_id_prefixid dump_settingsNdump_internalsNdump_transformsNdump_pseudo_xmlNexpose_internalsNstrict_visitorN_disable_configN_sourcehnj _destinationN _config_files]7/var/lib/git/docbuild/linux/Documentation/docutils.confafile_insertion_enabled raw_enabledKline_length_limitM'pep_referencesN pep_base_urlhttps://peps.python.org/pep_file_url_templatepep-%04drfc_referencesN rfc_base_url&https://datatracker.ietf.org/doc/html/ tab_widthKtrim_footnote_reference_spacesyntax_highlightlong smart_quotessmartquotes_locales]character_level_inline_markupdoctitle_xform docinfo_xformKsectsubtitle_xform image_loadinglinkembed_stylesheetcloak_email_addressessection_self_linkenvNubreporterNindirect_targets]substitution_defs}substitution_names}refnames}refids}j]jasnameids}(j jmjjjjj$jjjjjjjj) jjj<j j^j> jj" jj ju nametypes}(j jjj$jjjj) jj j> j" j uh}(jmhjj>jjjjjjjjjjjj'j<jFj^jjj jjA jj, hj_jj}jjjjjjjjj6j-jXjOjzjqjjjju footnote_refs} citation_refs} autofootnotes]autofootnote_refs]symbol_footnotes]symbol_footnote_refs] footnotes] citations]autofootnote_startKsymbol_footnote_startK id_counter collectionsCounter}j-K sRparse_messages]transform_messages]hsystem_message)}(hhh]h)}(hhh]h5Hyperlink target "read-ftrace-log" is not referenced.}hjsbah}(h]h ]h"]h$]h&]uh1hhjubah}(h]h ]h"]h$]h&]levelKtypeINFOsourcehnjlineK_uh1juba transformerN include_log] decorationNhhub.