Creating a tracepoint provider

Creating a tracepoint provider is the first step of using liblttng-ust. The next steps are:

A tracepoint provider is a compiled object containing the event probes corresponding to your custom tracepoint definitions. A tracepoint provider contains the code to get the size of an event and to serialize it, amongst other things.

To create a tracepoint provider, start with the following tracepoint provider header template:

#undef TRACEPOINT_PROVIDER
#define TRACEPOINT_PROVIDER my_provider
#undef TRACEPOINT_INCLUDE
#define TRACEPOINT_INCLUDE "./tp.h"
#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
#define _TP_H
#include <lttng/tracepoint.h>
/*


 * TRACEPOINT_EVENT(), TRACEPOINT_EVENT_CLASS(),


 * TRACEPOINT_EVENT_INSTANCE(), TRACEPOINT_LOGLEVEL(),


 * and `TRACEPOINT_ENUM()` are used here.


 */
#endif /* _TP_H */
#include <lttng/tracepoint-event.h>

In this template, the tracepoint provider is named my_provider (TRACEPOINT_PROVIDER definition). The file needs to bear the name of the TRACEPOINT_INCLUDE definition (tp.h in this case). Between #include <lttng/tracepoint.h> and #endif go the invocations of the TRACEPOINT_EVENT(), TRACEPOINT_EVENT_CLASS(), TRACEPOINT_EVENT_INSTANCE(), TRACEPOINT_LOGLEVEL(), and TRACEPOINT_ENUM() macros.

The tracepoint provider header file needs to be included in a source file which looks like this:

#define TRACEPOINT_CREATE_PROBES
#include "tp.h"

Together, those two files (let’s call them tp.h and tp.c) form the tracepoint provider sources, ready to be compiled.

You can create multiple tracepoint providers to be used in a single application, but each one must have its own header file.

The TRACEPOINT_EVENT() usage section below shows how to use the TRACEPOINT_EVENT() macro to define the actual tracepoints in the tracepoint provider header file.

See the EXAMPLE section below for a complete example.

TRACEPOINT_EVENT() usage

The TRACEPOINT_EVENT() macro is used in a template provider header file (see the Creating a tracepoint provider section above) to define LTTng-UST tracepoints.

The TRACEPOINT_EVENT() usage template is as follows:

TRACEPOINT_EVENT(


    /* Tracepoint provider name */


    my_provider,


    /* Tracepoint/event name */


    my_tracepoint,


    /* List of tracepoint arguments (input) */


    TP_ARGS(


        ...


    ),


    /* List of fields of eventual event (output) */


    TP_FIELDS(


        ...


    )
)

The TP_ARGS() macro contains the input arguments of the tracepoint. Those arguments can be used in the argument expressions of the output fields defined in TP_FIELDS().

The format of the TP_ARGS() parameters is: C type, then argument name; repeat as needed, up to ten times. For example:

TP_ARGS(


    int, my_int,


    const char *, my_string,


    FILE *, my_file,


    double, my_float,


    struct my_data *, my_data
)

The TP_FIELDS() macro contains the output fields of the tracepoint, that is, the actual data that can be recorded in the payload of an event emitted by this tracepoint.

The TP_FIELDS() macro contains a list of ctf_*() macros NOT separated by commas. The available macros are documented in the Available ctf_*() field type macros section below.

Available ctf_*() field type macros

This section documents the available ctf_*() macros that can be inserted in the TP_FIELDS() macro of the TRACEPOINT_EVENT() macro.

Standard integer, displayed in base 10:

ctf_integer(int_type, field_name, expr)
ctf_integer_nowrite(int_type, field_name, expr)

Standard integer, displayed in base 16:

ctf_integer_hex(int_type, field_name, expr)

Integer in network byte order (big endian), displayed in base 10:

ctf_integer_network(int_type, field_name, expr)

Integer in network byte order, displayed in base 16:

ctf_integer_network_hex(int_type, field_name, expr)

Floating point number:

ctf_float(float_type, field_name, expr)
ctf_float_nowrite(float_type, field_name, expr)

Null-terminated string:

ctf_string(field_name, expr)
ctf_string_nowrite(field_name, expr)

Statically-sized array of integers (_hex versions displayed in hexadecimal, _network versions in network byte order):

ctf_array(int_type, field_name, expr, count)
ctf_array_nowrite(int_type, field_name, expr, count)
ctf_array_hex(int_type, field_name, expr, count)
ctf_array_nowrite_hex(int_type, field_name, expr, count)
ctf_array_network(int_type, field_name, expr, count)
ctf_array_network_nowrite(int_type, field_name, expr, count)
ctf_array_network_hex(int_type, field_name, expr, count)
ctf_array_network_nowrite_hex(int_type, field_name, expr, count)

Statically-sized array, printed as text; no need to be null-terminated:

ctf_array_text(char, field_name, expr, count)
ctf_array_text_nowrite(char, field_name, expr, count)

Dynamically-sized array of integers (_hex versions displayed in hexadecimal, _network versions in network byte order):

ctf_sequence(int_type, field_name, expr, len_type, len_expr)
ctf_sequence_nowrite(int_type, field_name, expr, len_type, len_expr)
ctf_sequence_hex(int_type, field_name, expr, len_type, len_expr)
ctf_sequence_nowrite_hex(int_type, field_name, expr, len_type,


                         len_expr)
ctf_sequence_network(int_type, field_name, expr, len_type, len_expr)
ctf_sequence_network_nowrite(int_type, field_name, expr, len_type,


                             len_expr)
ctf_sequence_network_hex(int_type, field_name, expr, len_type,


                         len_expr)
ctf_sequence_network_nowrite_hex(int_type, field_name, expr,


                                 len_type, len_expr)

Dynamically-sized array, displayed as text; no need to be null-terminated:

ctf_sequence_text(char, field_name, expr, len_type, len_expr)
ctf_sequence_text_nowrite(char, field_name, expr, len_type, len_expr)

Enumeration. The enumeration field must be defined before using this macro with the TRACEPOINT_ENUM() macro. See the TRACEPOINT_ENUM() usage section for more information.

ctf_enum(prov_name, enum_name, int_type, field_name, expr)
ctf_enum_nowrite(prov_name, enum_name, int_type, field_name, expr)

The parameters are:

count

enum_name

expr

field_name

float_type

int_type

len_expr

len_type

prov_name

The _nowrite versions omit themselves from the recorded trace, but are otherwise identical. Their primary purpose is to make some of the event context available to the event filters without having to commit the data to sub-buffers. See lttng-enable-event(1) to learn more about dynamic event filtering.

See the EXAMPLE section below for a complete example.

TRACEPOINT_ENUM() usage

An enumeration field is a list of mappings between an integers, or a range of integers, and strings (sometimes called labels or enumerators). Enumeration fields can be used to have a more compact trace when the possible values for a field are limited.

An enumeration field is defined with the TRACEPOINT_ENUM() macro:

TRACEPOINT_ENUM(


    /* Tracepoint provider name */


    my_provider,


    /* Enumeration name (unique in the whole tracepoint provider) */


    my_enum,


    /* Enumeration mappings */


    TP_ENUM_VALUES(


        ...


    )
)

TP_ENUM_VALUES() contains a list of enumeration mappings, NOT separated by commas. Two macros can be used in the TP_ENUM_VALUES(): ctf_enum_value() and ctf_enum_range().

ctf_enum_value() is a single value mapping:

ctf_enum_value(label, value)

This macro maps the given label string to the value value.

ctf_enum_range() is a range mapping:

ctf_enum_range(label, start, end)

This macro maps the given label string to the range of integers from start to end, inclusively. Range mappings may overlap, but the behaviour is implementation-defined: each trace reader handles overlapping ranges as it wishes.

See the EXAMPLE section below for a complete example.

TRACEPOINT_EVENT_CLASS() usage

A tracepoint class is a class of tracepoints sharing the same field types and names. A tracepoint instance is one instance of such a declared tracepoint class, with its own event name.

LTTng-UST creates one event serialization function per tracepoint class. Using TRACEPOINT_EVENT() creates one tracepoint class per tracepoint definition, whereas using TRACEPOINT_EVENT_CLASS() and TRACEPOINT_EVENT_INSTANCE() creates one tracepoint class, and one or more tracepoint instances of this class. In other words, many tracepoints can reuse the same serialization code. Reusing the same code, when possible, can reduce cache pollution, thus improve performance.

The TRACEPOINT_EVENT_CLASS() macro accepts the same parameters as the TRACEPOINT_EVENT() macro, except that instead of an event name, its second parameter is the tracepoint class name:

TRACEPOINT_EVENT_CLASS(


    /* Tracepoint provider name */


    my_provider,


    /* Tracepoint class name */


    my_tracepoint_class,


    /* List of tracepoint arguments (input) */


    TP_ARGS(


        ...


    ),


    /* List of fields of eventual event (output) */


    TP_FIELDS(


        ...


    )
)

Once the tracepoint class is defined, you can create as many tracepoint instances as needed:

TRACEPOINT_EVENT_INSTANCE(


    /* Tracepoint provider name */


    my_provider,


    /* Tracepoint class name */


    my_tracepoint_class,


    /* Tracepoint/event name */


    my_tracepoint,


    /* List of tracepoint arguments (input) */


    TP_ARGS(


        ...


    )
)

As you can see, the TRACEPOINT_EVENT_INSTANCE() does not contain the TP_FIELDS() macro, because they are defined at the TRACEPOINT_EVENT_CLASS() level.

See the EXAMPLE section below for a complete example.

TRACEPOINT_LOGLEVEL() usage

Optionally, a log level can be assigned to a defined tracepoint. Assigning different levels of severity to tracepoints can be useful: when controlling tracing sessions, you can choose to only enable events falling into a specific log level range using the --loglevel and --loglevel-only options of the lttng-enable-event(1) command.

Log levels are assigned to tracepoints that are already defined using the TRACEPOINT_LOGLEVEL() macro. The latter must be used after having used TRACEPOINT_EVENT() or TRACEPOINT_EVENT_INSTANCE() for a given tracepoint. The TRACEPOINT_LOGLEVEL() macro is used as follows:

TRACEPOINT_LOGLEVEL(


    /* Tracepoint provider name */


    my_provider,


    /* Tracepoint/event name */


    my_tracepoint,


    /* Log level */


    TRACE_INFO
)

The available log level definitions are:

TRACE_EMERG

TRACE_ALERT

TRACE_CRIT

TRACE_ERR

TRACE_WARNING

TRACE_NOTICE

TRACE_INFO

TRACE_DEBUG_SYSTEM

TRACE_DEBUG_PROGRAM

TRACE_DEBUG_PROCESS

TRACE_DEBUG_MODULE

TRACE_DEBUG_UNIT

TRACE_DEBUG_FUNCTION

TRACE_DEBUG_LINE

TRACE_DEBUG

See the EXAMPLE section below for a complete example.

Instrumenting your application

Once the tracepoint provider is created (see the Creating a tracepoint provider section above), you can instrument your application with the defined tracepoints thanks to the tracepoint() macro:

#define tracepoint(prov_name, t_name, ...)

With:

prov_name

t_name

...

Make sure to include the tracepoint provider header file anywhere you use tracepoint() for this provider.

Sometimes, arguments to the tracepoint are expensive to compute (take call stack, for example). To avoid the computation when the tracepoint is disabled, you can use the tracepoint_enabled() and do_tracepoint() macros:

#define tracepoint_enabled(prov_name, t_name)
#define do_tracepoint(prov_name, t_name, ...)

tracepoint_enabled() returns a non-zero value if the tracepoint named t_name from the provider named prov_name is enabled at run time.

do_tracepoint() is like tracepoint(), except that it doesn’t check if the tracepoint is enabled. Using tracepoint() with tracepoint_enabled() is dangerous since tracepoint() also contains the tracepoint_enabled() check, thus a race condition is possible in this situation:

if (tracepoint_enabled(my_provider, my_tracepoint)) {


    stuff = prepare_stuff();
}
tracepoint(my_provider, my_tracepoint, stuff);

If the tracepoint is enabled after the condition, then stuff is not prepared: the emitted event will either contain wrong data, or the whole application could crash (segmentation fault, for example).

Statically linking the tracepoint provider

With the static linking method, compiled tracepoint providers are copied into the target application.

Define TRACEPOINT_DEFINE definition below the TRACEPOINT_CREATE_PROBES definition in the tracepoint provider source:

#define TRACEPOINT_CREATE_PROBES
#define TRACEPOINT_DEFINE
#include "tp.h"

Create the tracepoint provider object file:

$ cc -c -I. tp.c

At this point, you can archive this tracepoint provider object file, possibly with other object files of your application or with other tracepoint provider object files, as a static library:

$ ar rc tp.a tp.o

Using a static library does have the advantage of centralising the tracepoint providers objects so they can be shared between multiple applications. This way, when the tracepoint provider is modified, the source code changes don’t have to be patched into each application’s source code tree. The applications need to be relinked after each change, but need not to be otherwise recompiled (unless the tracepoint provider’s API changes).

Then, link your application with this object file (or with the static library containing it) and with liblttng-ust and libdl (libc on a BSD system):

$ cc -o app tp.o app.o -llttng-ust -ldl

Dynamically loading the tracepoint provider

The second approach to package the tracepoint provider is to use the dynamic loader: the library and its member functions are explicitly sought, loaded at run time.

In this scenario, the tracepoint provider is compiled as a shared object.

The process to create the tracepoint provider shared object is pretty much the same as the static linking method, except that:

Regarding TRACEPOINT_DEFINE and TRACEPOINT_PROBE_DYNAMIC_LINKAGE, the recommended practice is to use a separate C source file in your application to define them, then include the tracepoint provider header files afterwards. For example, as tp-define.c:

#define TRACEPOINT_DEFINE
#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
#include "tp.h"

The tracepoint provider object file used to create the shared library is built like it is using the static linking method, but with the -fpic option:

$ cc -c -fpic -I. tp.c

It is then linked as a shared library like this:

$ cc -shared -Wl,--no-as-needed -o tp.so tp.o -llttng-ust

This tracepoint provider shared object isn’t linked with the user application: it must be loaded manually. This is why the application is built with no mention of this tracepoint provider, but still needs libdl:

$ cc -o app app.o tp-define.o -ldl

There are two ways to dynamically load the tracepoint provider shared object:

If the application does not dynamically load the tracepoint provider shared object using one of the methods above, tracing is disabled for this application, and the events are not listed in the output of lttng-list(1).

Note that it is not safe to use dlclose(3) on a tracepoint provider shared object that is being actively used for tracing, due to a lack of reference counting from LTTng-UST to the shared object.

For example, statically linking a tracepoint provider to a shared object which is to be dynamically loaded by an application (a plugin, for example) is not safe: the shared object, which contains the tracepoint provider, could be dynamically closed (dlclose(3)) at any time by the application.

To instrument a shared object, either:

Using LTTng-UST with daemons

Some extra care is needed when using liblttng-ust with daemon applications that call fork(2), clone(2), or BSD’s rfork(2) without a following exec(3) family system call. The library liblttng-ust-fork.so needs to be preloaded before starting the application with the LD_PRELOAD environment variable (see ld.so(8)).

To use liblttng-ust with a daemon application which closes file descriptors that were not opened by it, preload the liblttng-ust-fd.so library before you start the application. Typical use cases include daemons closing all file descriptors after fork(2), and buggy applications doing “double-closes”.

Context information

Context information can be prepended by the LTTng-UST tracer before each event, or before specific events.

Context fields can be added to specific channels using lttng-add-context(1).

The following context fields are supported by LTTng-UST:

cpu_id

ip

perf:thread:COUNTER

perf:thread:raw:rN:NAME

pthread_id

procname

vpid

vtid

LTTng-UST state dump

If an application that uses liblttng-ust becomes part of a tracing session, information about its currently loaded shared objects, their build IDs, and their debug link information are emitted as events by the tracer.

The following LTTng-UST state dump events exist and must be enabled to record application state dumps. Note that, during the state dump phase, LTTng-UST can also emit shared library load/unload events (see Shared library load/unload tracking below).

lttng_ust_statedump:start

lttng_ust_statedump:end

lttng_ust_statedump:bin_info

lttng_ust_statedump:build_id

lttng_ust_statedump:debug_link

Shared library load/unload tracking

The LTTng-UST state dump and the LTTng-UST helper library to instrument the dynamic linker (see liblttng-ust-dl(3)) can emit shared library load/unload tracking events.

The following shared library load/unload tracking events exist and must be enabled to track the loading and unloading of shared libraries:

lttng_ust_lib:load

lttng_ust_lib:unload

lttng_ust_lib:build_id

lttng_ust_lib:debug_link

Detect if LTTng-UST is loaded

To detect if liblttng-ust is loaded from an application:

int lttng_ust_loaded __attribute__((weak));

/* ... */
if (lttng_ust_loaded) {


    /* LTTng-UST is loaded */
} else {


    /* LTTng-UST is NOT loaded */
}
/* ... */

Tracepoint provider header file

tp.h:

#undef TRACEPOINT_PROVIDER
#define TRACEPOINT_PROVIDER my_provider
#undef TRACEPOINT_INCLUDE
#define TRACEPOINT_INCLUDE "./tp.h"
#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
#define _TP_H
#include <lttng/tracepoint.h>
#include <stdio.h>
#include "app.h"
TRACEPOINT_EVENT(


    my_provider,


    simple_event,


    TP_ARGS(


        int, my_integer_arg,


        const char *, my_string_arg


    ),


    TP_FIELDS(


        ctf_string(argc, my_string_arg)


        ctf_integer(int, argv, my_integer_arg)


    )
)
TRACEPOINT_ENUM(


    my_provider,


    my_enum,


    TP_ENUM_VALUES(


        ctf_enum_value("ZERO", 0)


        ctf_enum_value("ONE", 1)


        ctf_enum_value("TWO", 2)


        ctf_enum_range("A RANGE", 52, 125)


        ctf_enum_value("ONE THOUSAND", 1000)


    )
)
TRACEPOINT_EVENT(


    my_provider,


    big_event,


    TP_ARGS(


        int, my_integer_arg,


        const char *, my_string_arg,


        FILE *, stream,


        double, flt_arg,


        int *, array_arg


    ),


    TP_FIELDS(


        ctf_integer(int, int_field1, my_integer_arg * 2)


        ctf_integer_hex(long int, stream_pos, ftell(stream))


        ctf_float(double, float_field, flt_arg)


        ctf_string(string_field, my_string_arg)


        ctf_array(int, array_field, array_arg, 7)


        ctf_array_text(char, array_text_field, array_arg, 5)


        ctf_sequence(int, seq_field, array_arg, int,


                     my_integer_arg / 10)


        ctf_sequence_text(char, seq_text_field, array_arg,


                          int, my_integer_arg / 5)


        ctf_enum(my_provider, my_enum, int,


                 enum_field, array_arg[1])


    )
)
TRACEPOINT_LOGLEVEL(my_provider, big_event, TRACE_WARNING)
TRACEPOINT_EVENT_CLASS(


    my_provider,


    my_tracepoint_class,


    TP_ARGS(


        int, my_integer_arg,


        struct app_struct *, app_struct_arg


    ),


    TP_FIELDS(


        ctf_integer(int, a, my_integer_arg)


        ctf_integer(unsigned long, b, app_struct_arg->b)


        ctf_string(c, app_struct_arg->c)


    )
)
TRACEPOINT_EVENT_INSTANCE(


    my_provider,


    my_tracepoint_class,


    event_instance1,


    TP_ARGS(


        int, my_integer_arg,


        struct app_struct *, app_struct_arg


    )
)
TRACEPOINT_EVENT_INSTANCE(


    my_provider,


    my_tracepoint_class,


    event_instance2,


    TP_ARGS(


        int, my_integer_arg,


        struct app_struct *, app_struct_arg


    )
)
TRACEPOINT_LOGLEVEL(my_provider, event_instance2, TRACE_INFO)
TRACEPOINT_EVENT_INSTANCE(


    my_provider,


    my_tracepoint_class,


    event_instance3,


    TP_ARGS(


        int, my_integer_arg,


        struct app_struct *, app_struct_arg


    )
)
#endif /* _TP_H */
#include <lttng/tracepoint-event.h>

Tracepoint provider source file

tp.c:

#define TRACEPOINT_CREATE_PROBES
#define TRACEPOINT_DEFINE
#include "tp.h"

Application header file

app.h:

#ifndef _APP_H
#define _APP_H
struct app_struct {


    unsigned long b;


    const char *c;


    double d;
};
#endif /* _APP_H */

Application source file

app.c:

#include <stdlib.h>
#include <stdio.h>
#include "tp.h"
#include "app.h"
static int array_of_ints[] = {


    100, -35, 1, 23, 14, -6, 28, 1001, -3000,
};
int main(int argc, char* argv[])
{


    FILE *stream;


    struct app_struct app_struct;


    tracepoint(my_provider, simple_event, argc, argv[0]);


    stream = fopen("/tmp/app.txt", "w");


    if (!stream) {


        fprintf(stderr,


                "Error: Cannot open /tmp/app.txt for writing\n");


        return EXIT_FAILURE;


    }


    if (fprintf(stream, "0123456789") != 10) {


        fclose(stream);


        fprintf(stderr, "Error: Cannot write to /tmp/app.txt\n");


        return EXIT_FAILURE;


    }


    tracepoint(my_provider, big_event, 35, "hello tracepoint",


               stream, -3.14, array_of_ints);


    fclose(stream);


    app_struct.b = argc;


    app_struct.c = "[the string]";


    tracepoint(my_provider, event_instance1, 23, &app_struct);


    app_struct.b = argc * 5;


    app_struct.c = "[other string]";


    tracepoint(my_provider, event_instance2, 17, &app_struct);


    app_struct.b = 23;


    app_struct.c = "nothing";


    tracepoint(my_provider, event_instance3, -52, &app_struct);


    return EXIT_SUCCESS;
}