On this page:

Enabling Software Defined Events
Reading SDEs from within application code or performance tools
Adding SDEs to a library

Enabling Software Defined Events

To enable reading Software Defined Events (SDEs) the user needs to link against a PAPI library that was configured with the sde component enabled. As an example the following command: ./configure --with-components="sde" is sufficient to enable the component.

Reading SDEs from within application code or performance tools

If the PAPI library is configured with the sde component enabled, then SDEs can be read just like any other event supported by a PAPI component. The same API (PAPI_start()/PAPI_read()/PAPI_stop()) applies unmodified as with hardware events. The only caveat relates to types. PAPI_read() will always store the results in an array of type long long int. However, a library might export a counter that is of a different type, e.g., double. PAPI will not cast between the different types. Instead it will pack the bits of the library counter into the long long int in the result. In the header file we provide the convenience macros GET_DOUBLE_SDE() and GET_FLOAT_SDE() that unpack the double and float value respectively. In the future, we plan to provide a C++ API, which will solve the type limitations imposed by our current C API.

Adding SDEs to a library

In the following text we will provide examples of different features and explain how to use them. All these examples pertain to libraries (or other software layers) that wish to add SDEs in their code. For merely reading SDEs generated by a library, see the previous section.

SDE counters come in three flavors:

The most common is a program variable that is being registered with PAPI as an SDE. Registered counters incur zero overhead since they are program variables whose values are being modified as part of the normal execution of the library without requiring any SDE-specific API calls.
Created counters are variables that are created and managed internally by PAPI. Modifying their value requires SDE-specific API calls, which causes overhead, but PAPI is always aware of their value and therefore can deliver accurate overflow notifications.
Recorders are multi-value counters that are also managed internally by PAPI. Recorders enable libraries to record an arbitrary long sequence of data.

Header files and linking

In the PAPI repo, under src/components/sde/interface we provide the header file papi_sde_interface.h that provides all necessary declarations for adding SDEs in a library. This folder also contains the C file papi_sde_interface.c which provides weak symbols for all SDE functions. This file is meant to be copied inside 3rd party libraries to solve linking problems in the absence of libpapi in a system. In other words, consider that you are the developer of the project SomeName and you built and linked the file papi_sde_interface.c inside libSomeName.so. If application MyApp links against libSomeName.so without libpapi.so, the application will link and work fine (but the SDEs in SomeName will not count anything). If application MyApp links against both libpapi.so and libSomeName.so, then the application will link and work fine, and the SDEs in SomeName will count the events as intended.

Minimal example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Minimal/.

The following code is the minimum necessary library code for creating an SDE. Let's call this library mintest.

long long local_var;

void mintest_init(void){
    local_var =0;
    papi_handle_t *handle = papi_sde_init("Min Example Code");
    papi_sde_register_counter(handle, "Example Event", PAPI_SDE_RO|PAPI_SDE_DELTA, PAPI_SDE_long_long, &local_var);
}

This code assumes that mintest has an internal variable (local_var) that counts some event that occurs inside the library, which the developers wish to export to the outside world. To do so, the library needs to call some papi_sde functions.

First, the function papi_sde_init() must be called. The parameter is a string (const char *) that contains the name of the library. The developers can choose an arbitrary string; PAPI does not parse or process it in any way. However, this string will be appended as a prefix to all SDEs exported by this library, so the colon character : should be avoided, due to its special meaning in PAPI events. This function returns an opaque handle that must be passed to all future papi_sde calls made by this library.

After obtaining the handle, mintest calls: papi_sde_register_counter(handle, "Example Event", PAPI_SDE_RO|PAPI_SDE_DELTA, PAPI_SDE_long_long, &local_var);

handle is the opaque handle returned by papi_sde_init().
The second parameter is an arbitrary string containing the name of the SDE being registered. Once again, only the colon character : should be avoided.
The third parameter specifies the mode of the event. The mode is a bitwise-or of two flags:
1. the flag that specifies the access mode of the event counter, which can be PAPI_SDE_RO or PAPI_SDE_RW (for read-only and read-write counters, respectively);
2. the flag that specifies whether this event is a delta (PAPI_SDE_DELTA) or an instant (PAPI_SDE_INSTANT) event. For delta events, PAPI reports the difference in the value of the counter between PAPI_start() and PAPI_read() (i.e., as in the hardware events that count instructions executed). For instant events, PAPI reports the actual value of the counter when PAPI_read() is called (i.e., as in the hardware events that report power usage).
The fourth parameter specifies the counter type and has to be one of: PAPI_SDE_long_long, PAPI_SDE_int, PAPI_SDE_double, PAPI_SDE_float
The last parameter is a pointer to the counter. I.e., a pointer to the library variable that counts the occurrences of the SDE that is being registered.

Counter Group and Function-pointer example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Simple2/.

The following code demonstrates the use of counter groups and function-pointers that can produce an event counter at run-time. Let's call this library Simple.

static const char *ev_names[4] = {
    "COMPUTED_VALUE",
    "TOTAL_ITERATIONS",
    "LOW_WATERMARK_REACHED",
    "HIGH_WATERMARK_REACHED"
};

long long int compute_sde( void *param );

void simple_init(void){

    // Initialize library specific variables
    comp_value = 0.0;
    total_iter_cnt = 0;
    low_wtrmrk = 0;
    high_wtrmrk = 0;

    // Initialize PAPI SDEs
    handle = papi_sde_init("Simple");
    papi_sde_register_fp_counter(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, compute_sde, &comp_value);
    papi_sde_register_counter(handle, ev_names[1], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &total_iter_cnt);
    papi_sde_register_counter(handle, ev_names[2], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &low_wtrmrk);
    papi_sde_register_counter(handle, ev_names[3], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &high_wtrmrk);
    papi_sde_add_counter_to_group(handle, ev_names[2], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);
    papi_sde_add_counter_to_group(handle, ev_names[3], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);

    return;
}

In addition to registering counters as the example in the previous section did, this code uses two more SDE features. First, it registers a function-pointer counter by calling the function:
papi_sde_register_fp_counter(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, compute_sde, &comp_value);

The first four parameters of this call have the same semantics as in the case of papi_sde_register_counter(). However, the fifth parameter is a pointer to a callback function (instead of a pointer to a counter). This function will be called by PAPI when the application makes a call to PAPI_read(), or PAPI_stop(). Therefore, using this functionality a library can create events whose value is computed at run-time and does not necessarily correspond to a program variable. The last parameter is a user-specified pointer that is opaque to PAPI, and it will be passed by PAPI to the callback when it is called.

In addition to registering counters, this example code calls the function papi_sde_add_counter_to_group() to add two counters (that have already been registered) to the group named ANY_WATERMARK_REACHED. The parameters of this function are:

The handle returned by papi_sde_init().
The name of the counter which is being added to the group.
The name of the group.
The operation that PAPI will perform on the members of the group when the group is read. This can be one of the following:
1. PAPI_SDE_SUM
2. PAPI_SDE_MAX
3. PAPI_SDE_MIN

Counter groups are first class citizens and can be recursively combined with other counters or groups into larger groups. The following limitations apply to the way counters can be organized in groups:

Groups can be formed out of counters of type int, long long int, float, and double, but all the counters in a single group must be of the same type. If counters of different types are grouped together the behavior is undefined.
All the counters added in a group must use the same operation (PAPI_SDE_SUM, PAPI_SDE_MAX, or PAPI_SDE_MIN).

Listing SDEs through the papi_native_avail utility

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Simple2/.

Since the introduction of the sde component in PAPI, the utility papi_native_avail accepts the flag -sde followed by a path to either a library that contains SDEs or an executable that is linked against libraries that contain SDEs. When this flag and an appropriate path are specified, papi_native_avail will list all the SDEs found in the library (or all libraries linked against the executable). To enable this behavior, the library must implement the hook function:
papi_handle_t papi_sde_hook_list_events( papi_sde_fptr_struct_t *fptr_struct)
This function will be called by papi_native_avail and the parameter it takes is a pointer to a structure that contains function pointers to all SDE functions. The following code shows an example of how this function could be implemented. Note that this function is not supposed to be called by other library functions or normal applications. It is only a hook for the utility papi_native_avail to be able to discover the SDEs that are exported by a library. Therefore the pointers to the actual counters can be NULL, since they will never be dereferenced.

papi_handle_t papi_sde_hook_list_events( papi_sde_fptr_struct_t *fptr_struct){
    handle = fptr_struct->init("Simple");
    fptr_struct->register_fp_counter(handle, ev_names[0], PAPI_SDE_RO|PAPI_SDE_INSTANT, PAPI_SDE_double, compute_sde, &comp_value);
    fptr_struct->register_counter(handle, ev_names[1], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &total_iter_cnt);
    fptr_struct->register_counter(handle, ev_names[2], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &low_wtrmrk);
    fptr_struct->register_counter(handle, ev_names[3], PAPI_SDE_RO|PAPI_SDE_DELTA,   PAPI_SDE_long_long, &high_wtrmrk);
    fptr_struct->add_counter_to_group(handle, ev_names[2], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);
    fptr_struct->add_counter_to_group(handle, ev_names[3], "ANY_WATERMARK_REACHED", PAPI_SDE_SUM);

    fptr_struct->describe_counter(handle, ev_names[0], "Sum of values that are within the watermarks.");
    fptr_struct->describe_counter(handle, ev_names[1], "Total iterations executed by the library.");
    fptr_struct->describe_counter(handle, ev_names[2], "Number of times a value was below the low watermark.");
    fptr_struct->describe_counter(handle, ev_names[3], "Number of times a value was above the high watermark.");
    fptr_struct->describe_counter(handle, "ANY_WATERMARK_REACHED",  "Number of times a value was not between the two watermarks.");

    return handle;
}

Created Counter example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Created_Counter/.

Created Counters (CCs) are SDEs that are managed internally by PAPI, in contrast with Registered Counters, which are library variables whose address has been registered with PAPI. The benefit of using a CC is that PAPI is always aware of its value, so it can notify "listeners" immediately after the value exceeds a threshold set by the listener. This is enabled through the Overflow interface of PAPI and is discussed in the following section. The downside of CCs is that their value has to be updated through an SDE API call, so using CCs incurs overhead, in contrast with registered counters.

The following example shows the creation and update of a CC.

static const char *event_names[1] = {
    "epsilon_count"
};

void *cntr_handle;

void cclib_init(void){
    papi_handle_t sde_handle;

    sde_handle = papi_sde_init("Lib_With_CC");
    papi_sde_create_counter(sde_handle, event_names[0], PAPI_SDE_DELTA, &cntr_handle);

    return;
}

void cclib_do_work(void){
    int i;

    for(i=0; i<100*1000; i++){
        double r = (double)random() / (double)RAND_MAX;
        if( r < MY_EPSILON ){
            papi_sde_inc_counter(cntr_handle, 1);
        }
        // Do some usefull work here
        if( !(i%100) )
            (void)usleep(1);
    }

    return;
}

Using the Overflow mechanism to monitor a Created Counter

A user program can invoke the PAPI function PAPI_overflow() to specify a callback function and a threshold value for a particular event. When the event counter exceeds this threshold, PAPI will invoke the callback function. This mechanism is not SDE specific but has always been part of PAPI. The SDE component supports overflowing for all types of counters, but when overflowing is used with Created Counters, the callback is invoked immediately after the CC exceeds the threshold, since PAPI is always aware of the value of the CC.

No special treatment is needed to read SDEs through the overflow mechanism, which means that 3rd-party tools that are based on sampling should work out of the box.

The preferred type of overflowing for the sde component is PAPI_OVERFLOW_HARDWARE, which signifies that the component will be in charge of supporting the overflowing mechanism, instead of the PAPI framework.

Recorder example

This section uses parts of the example code that can be found in the PAPI repository under src/components/sde/tests/Recorder/.

Recorders go beyond the notion of a single event counter in that they can record an indefinite number of values. In other words, a library can use recorders to store multiple values of interest, and the user application can retrieve the whole sequence of stored values, not just the latest one. PAPI provides the necessary storage using a dynamic data structure that tries to keep the amount of allocated space at each given time low, while also minimizing the amount of time spent in allocating additional space.

The following example code calls the function:
papi_sde_create_recorder(tmp_handle, event_names[0], sizeof(long long), papi_sde_compare_long_long, &rcrd_handle)
to create a recorder. The parameters of this function are as follows:

The handle returned by papi_sde_init().
The name of the recorder.
The size of each of the elements that will be recorded.
An optional function pointer for comparing elements. This function pointer (if not NULL) will be used to automatically compute the quartiles of the recorded elements. It can be one of the following:
1. NULL (if quartiles should not be computed automatically).
2. papi_sde_compare_long_long(const void *p1, const void *p2)
3. papi_sde_compare_int(const void *p1, const void *p2)
4. papi_sde_compare_double(const void *p1, const void *p2)
5. papi_sde_compare_float(const void *p1, const void *p2)
6. A custom, user-provided function which takes two const void * parameters and returns an int less than, equal to, or greater than zero if the first argument is considered to be respectively less than, equal to, or greater than the second.
A pointer to a void * variable. This is an output parameter, and after the return of this function it will contain a handle associated with the newly created recorder.

static const char *event_names[1] = {
    "simple_recording"
};

void *rcrd_handle;

void recorder_init_(void){
    papi_handle_t tmp_handle;

    tmp_handle = papi_sde_init("Lib_With_Recorder");
    papi_sde_create_recorder(tmp_handle, event_names[0], sizeof(long long), papi_sde_compare_long_long, &rcrd_handle);

    return;
}

void recorder_do_work_(void){
    long long r = random()%123456;
    papi_sde_record(rcrd_handle, sizeof(r), &r);
    return;
}

Just as is the case with registering counters, the creation of a recorder happens only once. However, in order to record elements, the library code must call the function papi_sde_record() for every element that must be recorded. This function takes as parameters:

The recorder handle (which was written by papi_sde_create_recorder()).
The size of the element being recorded.
A pointer to the element being recorded.

It is worth noting that PAPI treats the recorded element as opaque, which means that a library can use recorders for any object that is stored in contiguous memory, from basic types to structures to whole matrices.

Reading the elements in a Recorder

PAPI_read() can only read one value per event. If the SDE that is being read is a recorder, then PAPI stores into the read value a pointer to a contiguous buffer that contains all the recorder elements. Additionally, every time a recorder is created, PAPI automatically creates an additional SDE whose value is equal to the number of elements stored in the recorder. The name of this auxiliary SDE is formed by appending the suffix :CNT to the name of the recorder. An example of reading the elements in a recorder can be found in the PAPI repo under: src/components/sde/tests/Recorder/Recorder_Driver.c.

Wiki

papi / Software_Defined_Events.md