| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The bash completions of trace-cmd commands uses the -h to find the options
to finish the completions with. trace-cmd listen -h had a tab before "-l"
which caused the completions to miss it. Not to mention, it caused -l not to
be in line with the other options.
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
|
|
Since now there is an uniform command implementation I can reduce the
`main()` function to the minimum by using a lookup table. People can
now directly focus on a command implementation because there is
"nothing" in the `main()` function.
Link: http://lkml.kernel.org/r/20170426203028.13900-3-federico.vaga@vaga.pv.it
Signed-off-by: Federico Vaga <federico.vaga@vaga.pv.it>
Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
|
|
This is exposed in debugfs but there's no quick way to set it with
trace-cmd. While we're adding support for this, fix --profile so it sets
max_graph_depth back to whatever it was before.
Link: http://lkml.kernel.org/r/52d19bb3c0daf4d03db68da438c605b318ca41a8.1466021761.git.osandov@fb.com
Signed-off-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
If --ts-diff is added to trace-cmd report, it will show the delta between
events. It doesn't matter which CPUs they are on. In case of filtering, the
delta is still only between the visible events.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When using clocks such as x86-tsc, where the clock frequency is known, the
user can convert the timestamps into readable seconds, with resolution down
to nanoseconds.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When merging two or more data files, if the timestamp difference between
them are known, add the --ts-offset option to let the user specify the
offset from one data file to the next. This helps merging trace data files
from hosts and guests where the timestamp offset is a known value.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The -S option in trace-cmd report will not print events if the SOFTIRQ flag
is set. It does not affect hard interrupts.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The -I option in trace-cmd report will not print events if the HARDIRQ flag
is set. It does not affect softirqs.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When profiling large applications that have multiple children we really want to
see the profile of the app all together. Add a --by-comm option to accomplish
this for profile. This merges all the events for like comms into one big chunk
so we can easily see what a giant app is doing, such as HHVM. Thanks,
Link: http://lkml.kernel.org/r/1432229408-1479-1-git-send-email-jbacik@fb.com
Signed-off-by: Josef Bacik <jbacik@fb.com>
[ fixed up trace_hash ]
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
As for extract, -a means all instances except the top one.
Use -t to get the top one too.
Link: http://lkml.kernel.org/r/d6cdd535ae170d4ba0f580434a51286861c75c07.1429677461.git.hcochran@kernelspring.com
Signed-off-by: Howard Cochran <hcochran@kernelspring.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
This option will extract all instances that currently exist in the
system, including the top instance. This differs from the meaning of -a
for record and stream (enable all events), which would have no purpose
for extract. Such difference in meaning already exists for -s, so this
seemed reasonable.
Had to fix a bug in create_instance(), in which it ignored its parameter.
Link: http://lkml.kernel.org/r/444ea8355d8c0013ec9d35155d4ea83ec1a7bbab.1429677461.git.hcochran@kernelspring.com
Signed-off-by: Howard Cochran <hcochran@kernelspring.com>
[ Fixed compile error to use different __add_all_instances() ]
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -B option to trace-cmd extract so that it can extract listed
instances. As with start, stop, and reset, the top level instance is not
included when any -B are given. However, -t makes it also include the
top instance. As with other commands, if neither option is given, only
operate on the top instance.
Link: http://lkml.kernel.org/r/7b18a3f624fd1e864a86c0cee2079587953fa5ee.1429677461.git.hcochran@kernelspring.com
Signed-off-by: Howard Cochran <hcochran@kernelspring.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Instead of associating interrupts to tasks, when -G is used in
trace-cmd profile or trace-cmd report, the interrupts are set to
act as global events.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add a -H option that is similar to the trace-cmd record and profile -H
option, but this will let the user hook events together in any trace.dat
file, and do a profile on them. Of course, -H depends on --profile being
set otherwise it's rather meaningless.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add hooks to let users define their own matching events for profiling.
Currently this only really works for 'trace-cmd profile'. But if used for
trace-cmd record, it will start the tracing. trace-cmd report still
needs this added.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Knowing what kernel a trace was executed on can be valuable information.
Record uname in the TRACECMD_OPTION_UNAME, and have trace-cmd report
retrieve it via the --uname option.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
If the -o option is given to the profiler, than have it write the profile
output to the file specified instead of stdout. The output of the command
executing still goes to stdout.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -C <clock> to be able to change the clock source of tracing.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add trace-cmd list -C to list the trace clocks that can be used
with trace-cmd record.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -S to profile and record --profile to allow the user to override the
events and tracer. That is, with the -S option, trace-cmd will not enable
anything that's not enabled by the user herself.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add 'trace-cmd profile' that runs like trace-cmd stream does except
that it does not print the events as they arrive, but instead
analyzes the data and creates a profile output at the end.
When stated, it will enable function graph tracing if the depth
option is supported, and will set the depth to 1 (record just
the function that enters and leaves the kernel). The sched events
are enabled with stack tracing, and at the end it will show where
the task entered and exited and for how long. It will also show
where tasks have been blocked.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add 'trace-cmd stream' which streams data from the ring buffer binary
files and converts them to human readable format and outputs them to
stdout. This is mainly added to be the infrastructure of the profile
command.
Stream is the same as record except that it does not take an output
file parameter (-o), and does not write a file.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Update the man pages to trace-cmd record and report to show the
--profile option.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The new 'trace-cmd stat' feature displays the various status of the
tracing (ftrace) system. The status that it shows is:
Tracer: if one of the tracers (like function_graph) is active.
Otherwise nothing is displayed.
Events: Lists the events that are enable.
Event filters: Shows any filters that are set for any events
Function filters: Shows any filters for the function tracers
Graph functions: Shows any functions that the function graph
tracer should graph
Buffers: Shows the trace buffer size if they have been expanded.
By default, tracing buffers are in a compressed format until they are
used. If they are compressed, the buffer display will not be shown.
Trace clock: If the tracing clock is anything other than the default "local"
it will be displayed.
Trace CPU mask: If not all available CPUs are in the tracing CPU mask, then
the tracing CPU mask will be displayed.
Trace max latency: Shows the value of the trace max latency if it is other than zero.
Kprobes: Shows any kprobes that are defined for tracing.
Uprobes: Shows any uprobes that are defined for tracing.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add trace-cmd reset -B foo -d to delete the instance foo.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When the '-b' option is set after the instance option '-B' the
buffer size will affect the instance buffer and not any other
instance buffer. This is for trace-cmd start and reset.
For example:
trace-cmd reset -t -B foo -b 1
Will reset the buffer size for foo and not touch the top level
buffer size.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -B option to trace-cmd reset to reset instances. If the -B option
is used, than the top instance is ignored unless the -t option is also
used.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add trace-cmd list -B to list the buffer instances that are defined on
the system.
Also cleaned up the code a bit by adding a show_all variable that gets
cleared when any list option is set. This gets rid of the long if statement
that checks to make sure no option was set before deciding to display
everything.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Although --stat was documented in the trace-cmd-report man page, it
was missing from the trace-cmd report -h usage.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
When using -e 'regex', where a set of events are listed in running
trace-cmd list, the options '-F' '-l' or '-R' can be added to show
those events formats, filters, or triggers respectively.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the '-R' option to set triggers to events. It acts similar to
the -f option for filters.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the following options to trace-cmd show that allows the user to
see more content of the files in the debugfs system.
--tracing_on
--current_tracer
--buffer_size (for buffer_size_kb)
--buffer_total_size (for buffer_total_size_kb)
--ftrace_filter (for set_ftrace_filter)
--ftrace_notrace (for set_ftrace_notrace)
--ftrace_pid (for set_ftrace_pid)
--graph_function (for set_graph_function)
--graph_notrace (for set_graph_notrace)
--cpumask (for tracing_cpumask)
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Allow trace-cmd stop to stop instances with the -B "instance-name" option.
Since more than one instance may be specified by multiple -B options, the
-t option is also added to still be able to include the top level instance.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add missing "-M" description in the -h help text for record.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
As long runs of recording traces can cause the disk to fill up, adding
an option that lets the user limit the size of the data file can be really
useful.
Now with the '-m' option to trace-cmd record, this will limit the max size
of the per cpu data files to this number in kilobytes (rounded to pages).
Suggested-by: David Sommerseth <davids@redhat.com>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Now that -d will try to use the function-trace option if it exists, and will
not touch the /proc/sys/kernel/ftrace_enabled file, the -D option is added
to do just that. -D makes trace-cmd work like -d use to, but it will also
enable or disable the function-trace option too (if it exists).
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the trace-cmd snapshot command to operate on the snapshot file
(if it is supported by the kernel).
trace-cmd start -p function
trace-cmd snapshot -s
trace-cmd snapshot
[ dumps the content of buffer at 'trace-cmd snapshot -s' ]
trace-cmd snapshot -s
trace-cmd snapshot
[ dumps the new content of the buffer at the last -s operation ]
It also can handle per cpu snapshots (with kernel support) as well
as buffer instances.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Instead of running trace-cmd start and trace-cmd stop and then having
to cat out the trace directory directly from the debugfs directory,
it is much nicer to just have trace-cmd do it for you.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add a '-B' option that allows trace-cmd record to created and write
to buffer instances. That is:
/sys/kernel/debug/tracing/instances/<buffer>
These extra buffers is added to Linux 3.10 and are agnostic from
the main buffer (as well as other instances).
The format is:
trace-cmd record -p function -e sched -B foo -e block -B bar -e irq:softirq_entry
The above will enable the function tracer and all sched events for the
main buffer, but it will also create two buffer instances called
'foo' and 'bar'. The 'foo' buffer will record all block events where
as the 'bar' buffer will only record the irq:softirq_entry event.
Each of these buffers will be separate. The main buffer is recorded in
the trace.dat file as normal, but a new trace option is added called
TRACECMD_OPTION_BUFFER that stores the offset of where the buffers
cpu data is locate, and the buffer's name.
Next we need to get trace-cmd report to show this data, as it simply
ignores it now.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Starting with Linux kernel v3.10, the snapshot buffer should have
a way to extract it from a raw file. Adding '-s' to extract will
make trace-cmd pull from the snapshot_raw per_cpu files instead
of the normal per_cpu files.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The default output is to show the timestamp at a microsecond
precision, even though the timestamps are recorded in a nanosecond
precision. Adding this -t option allows the user to see the full
timestamp that was recorded in the file.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the -P option to compact all functions (ignoring pids) into
their own histogram.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add trace-cmd hist, to be able to see a histogram of the
trace.dat files. Currently it only handles call graphs of the
function tracer.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Added trace-cmd list -O to list the options supplied by plugins.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the '-P' option to trace-cmd list that will list the plugin
files that get loaded for trace-cmd report.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
A tracer-plugin should just be called a tracer and not a plugin, as it
gets confusing with trace-cmd plugins that are really plugins.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the options -n to match -N for individual events. The events
listed by -n will have any registered handler by a plugin ignored
for that event.
Add the option -r to match -R for individual events. The events
listed by -r will print the raw format instead of the pretty format.
*API Breakage* the new -R option was -r previously. But as -N is global
and -n is per event, it makes sense to have -R global and -r per event.
I doubt anyone uses this but me anyway :-)
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The man pages for trace-cmd-record were missing the -c option. Only the
trace-cmd record -h showed the option, and that one showed it incorrectly.
It does not trace the child processes of tasks with -P.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add to the list command a way to show the available filter functions.
Also let another optional parameter go with -f to do a search through
the functions.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add a new option --check-events to the report target which makes sure
that the event formats in the trace data file can be parsed correctly on
the machine that trace-cmd is running on. It returns EINVAL(22) for
parsing failures and 0 for success.
Cc: Michael Rubin <mrubin@google.com>
Cc: David Sharp <dhsharp@google.com>
Signed-off-by: Vaibhav Nagarnaik <vnagarnaik@google.com>
Link: http://lkml.kernel.org/r/1313091459-12049-4-git-send-email-vnagarnaik@google.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The target check-events parses the event format strings on the local
machine and returns whether they can be parsed or not. The parsing
functionality is extended by loading plugins, if available.
This patch loads the plugins before starting parsing of the event
formats.
Cc: Michael Rubin <mrubin@google.com>
Cc: David Sharp <dhsharp@google.com>
Signed-off-by: Vaibhav Nagarnaik <vnagarnaik@google.com>
Link: http://lkml.kernel.org/r/1313091459-12049-3-git-send-email-vnagarnaik@google.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add another target 'check-events' which parses all the event formats and
returns whether there are any issues with the print format strings.
With an error in the format, the return value is 22 (EINVAL) and for
success, it is 0.
Cc: Michael Rubin <mrubin@google.com>
Cc: David Sharp <dhsharp@google.com>
Signed-off-by: Vaibhav Nagarnaik <vnagarnaik@google.com>
Link: http://lkml.kernel.org/r/1311619188-4905-1-git-send-email-vnagarnaik@google.com
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add trace-cmd options that reads all the plugins and lists
the available options for trace-cmd report.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The -O field now passes options to the plugins. The format is of:
-O [plugin:]var[=val]
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Added --func-stack option to trace-cmd record, that will enable the
function stack tracing, but only if the -l option (limit functions)
is used, and succeeds in filtering functions. It is known that running
the function stack trace on all functions can livelock the machine.
The output now looks like this:
<idle>-0 [002] 64003.696572: function: schedule
<idle>-0 [002] 64003.696577: kernel_stack: <stack trace>
=> cpu_idle (ffffffff8100a38c)
=> start_secondary (ffffffff814ab828)
trace-cmd-4132 [003] 64003.696632: function: schedule
trace-cmd-4132 [003] 64003.696637: kernel_stack: <stack trace>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -T option to perform a stacktrace on each event:
kworker/3:2-2393 [003] 58549.282806: sched_switch: kworker/3:2:2393 [120] S ==> trace-cmd:2603 [120]
kworker/3:2-2393 [003] 58549.282808: kernel_stack: <stack trace>
=> schedule (ffffffff814b260e)
=> worker_thread (ffffffff8106edd7)
=> kthread (ffffffff81072b43)
=> kernel_thread_helper (ffffffff814bcbe4)
<idle>-0 [000] 58549.282866: sched_switch: swapper:0 [120] R ==> trace-cmd:2600 [120]
<idle>-0 [000] 58549.282869: kernel_stack: <stack trace>
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Since there's no good way to know when a child forks, and we
want to update the filters when it does. We must resort to ptrace.
Yes it sucks, but it works.
When performing a trace-cmd record -F -c program
The "-c" switch will cause trace-cmd to ptrace the program. If
the program forks, trace-cmd will update the filters to follow
the children of the program.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the -T option to print the filter strings created by -F
and exit.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Added the command "trace-cmd stack" to run the stack tracer.
It simply enables the stack tracer and depending on the options
it will ouput the current stack, or disable it.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The restore feature now allows you to make a trace.dat file from
several trace.dat.cpu.X files. This will let you put together files
leftover from a crashed trace-cmd record run.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add -k option to trace-cmd record that will keep it from resetting
the buffers when it is done. This option brings back the old
behavior of trace-cmd record.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the option '-r' to the trace-cmd record that will set the
trace capture threads to that real-time FIFO priority.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add a way to pipe log info to another file, especially useful in
daemon mode.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add '-r' option that prints the events by their field content and ignores
the print format of the event.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
Add the option -L to disable system plugins and only load local ones.
Add the option -N to disable all plugins.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The help for record has one backslash with the options going to the next line,
but it does not have another one for the second line the options run off to.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
|
The usage just printed all commands and their full descriptions. This is too much
for all the commands that trace-cmd does.
This patch moves the usage out of trace-cmd.c and into a new file called trace-usage.c.
It then makes a structure that holds the name of the command, a short description
and the full usage (from the original usage).
If no command, or the command passed to trace-cmd is not found, then the list
of commands and their short usages are shown.
If a command is passed in, only the usage for that command is shown.
Signed-off-by: Steven Rostedt <rostedt@goodmis.org>
|
