NAME

cpudist - On- and off-CPU task time as a histogram.

SYNOPSIS

cpudist [-h] [-O] [-T] [-m] [-P] [-L] [-p PID] [interval] [count]

DESCRIPTION

This measures the time a task spends on the CPU before being descheduled, and shows the times as a histogram. Tasks that spend a very short time on the CPU can be indicative of excessive context-switches and poor workload distribution, and possibly point to a shared source of contention that keeps tasks switching in and out as it becomes available (such as a mutex).

Similarly, the tool can also measure the time a task spends off-CPU before it is scheduled again. This can be helpful in identifying long blocking and I/O operations, or alternatively very short descheduling times due to short-lived locks or timers.

This tool uses in-kernel eBPF maps for storing timestamps and the histogram, for efficiency. Despite this, the overhead of this tool may become significant for some workloads: see the OVERHEAD section.

Since this uses BPF, only the root user can use this tool.

REQUIREMENTS

CONFIG_BPF and bcc.

OPTIONS

-h: Print usage message.
-O: Measure off-CPU time instead of on-CPU time.
-T: Include timestamps on output.
-m: Output histogram in milliseconds.
-P: Print a histogram for each PID (tgid from the kernel's perspective).
-L: Print a histogram for each TID (pid from the kernel's perspective).
-p PID: Only show this PID (filtered in kernel for efficiency).
interval: Output interval, in seconds.
count: Number of outputs.

EXAMPLES

Summarize task on-CPU time as a histogram:: # cpudist
Summarize task off-CPU time as a histogram:: # cpudist -O
Print 1 second summaries, 10 times:: # cpudist 1 10
Print 1 second summaries, using milliseconds as units for the histogram, and include timestamps on output:: # cpudist -mT 1
Trace PID 186 only, 1 second summaries:: # cpudist -P 185 1

FIELDS

usecs: Microsecond range
msecs: Millisecond range
count: How many times a task event fell into this range
distribution: An ASCII bar chart to visualize the distribution (count column)

OVERHEAD

This traces scheduler tracepoints, which can become very frequent. While eBPF has very low overhead, and this tool uses in-kernel maps for efficiency, the frequency of scheduler events for some workloads may be high enough that the overhead of this tool becomes significant. Measure in a lab environment to quantify the overhead before use.

SOURCE

This is from bcc.

: https://github.com/iovisor/bcc

Also look in the bcc distribution for a companion _example.txt file containing example usage, output, and commentary for this tool.

OS

Linux

STABILITY

Unstable - in development.

AUTHOR

Sasha Goldshtein