[lttng-dev] libbabeltrace: getting CTF sequence length?

[Simon Marchi]

On 20 May 2014 23:03, Philippe Proulx <eeppeliteloop at gmail.com> wrote:
> On Tue, May 20, 2014 at 4:50 PM, Jérémie Galarneau
> <jeremie.galarneau at efficios.com> wrote:
>> On Tue, May 20, 2014 at 4:37 PM, Philippe Proulx
>> <eeppeliteloop at gmail.com> wrote:
>>> Hello,
>>>
>>> I'm using libbabeltrace in C++. I would like to know if there's an
>>> equivalent of bt_ctf_get_array_len() for sequences. I tried using
>>> bt_ctf_get_field_list() but the reported count was 10, whereas the
>>> sequence length was really 6 (as shown by the babeltrace tool),
>>> leading to a segfault when accessing the 7th element.
>>>
>>> Looking at the Python binding, I see that this is used:
>>>
>>>     container_of(field, struct definition_sequence, p);
>>>
>>> followed by bt_sequence_len(). However, struct definition_sequence,
>>> bt_sequence_len() (and bt_sequence_index()) are not available in
>>> the API.
>>>
>>> So what would be your way to read sequence items? Do you have
>>> an example using strictly what's available in /usr/include/babeltrace?
>>>
>>
>> bt_ctf_get_field_list can be used to get the number of elements, along
>> with a pointer to the elements (via the output parameters).
>>
>
> Jérémie,
>
> consider this test I just wrote:
>
>     #include <stdlib.h>
>     #include <stdio.h>
>
>     #include <babeltrace/babeltrace.h>
>     #include <babeltrace/ctf/events.h>
>     #include <babeltrace/ctf/iterator.h>
>
>     int main(int argc, char* argv[])
>     {
>         struct bt_context* bt_context;
>         struct bt_ctf_iter* bt_ctf_iter;
>         const char* trace_path = argv[1];
>         struct bt_iter_pos begin_pos;
>         struct bt_ctf_event* bt_event;
>         const struct bt_definition* fields_def;
>         const struct bt_declaration* fields_decl;
>         const struct bt_definition* field_def;
>         const struct bt_declaration* field_decl;
>         unsigned int fields_count, seq_items_count;
>         struct bt_definition const* const* fields_list;
>         struct bt_definition const* const* seq_items_list;
>         int ret;
>         unsigned int x, y;
>         unsigned int pos = 1;
>         int err = 0;
>
>         bt_context = bt_context_create();
>         bt_context_add_trace(bt_context, trace_path, "ctf",
>                              NULL, NULL, NULL);
>
>         begin_pos.type = BT_SEEK_BEGIN;
>         begin_pos.u.seek_time = 0;
>
>         bt_ctf_iter = bt_ctf_iter_create(bt_context, &begin_pos, NULL);
>
>         while (bt_event = bt_ctf_iter_read_event(bt_ctf_iter)) {
>             fields_def = bt_ctf_get_top_level_scope(bt_event, BT_EVENT_FIELDS);
>
>             if (!fields_def) {
>                 goto next_event;
>             }
>
>             fields_decl = bt_ctf_get_decl_from_def(fields_def);
>
>             if (bt_ctf_field_type(fields_decl) != CTF_TYPE_STRUCT) {
>                 goto next_event;
>             }
>
>             ret = bt_ctf_get_field_list(bt_event, fields_def, &fields_list,
>                                         &fields_count);
>
>             if (ret < 0) {
>                 goto next_event;
>             }
>
>             for (x = 0; x < fields_count; ++x) {
>                 field_def = fields_list[x];
>
>                 if (!field_def) {
>                     printf("field element error\n");
>                     goto end;
>                 }
>
>                 field_decl = bt_ctf_get_decl_from_def(field_def);
>
>                 if (bt_ctf_field_type(field_decl) == CTF_TYPE_SEQUENCE) {
>                     ret = bt_ctf_get_field_list(bt_event, field_def,
>                                                 &seq_items_list,
>                                                 &seq_items_count);
>
>                     if (ret < 0) {
>                         /* I guess this means a count of 0? because
>                          * it does happen.
>                          */
>                         goto next_event;
>                     }
>
>                     for (y = 0; y < seq_items_count; ++y) {
>                         if (!seq_items_list[y]) {
>                             printf("error using seq_items_list[y]\n");
>                             err = 1;
>                         }
>
>                         if (!bt_ctf_get_index(bt_event, field_def, y)) {
>                             printf("error using bt_ctf_get_index()\n");
>                             err = 1;
>                         }
>
>                         if (err) {
>                             printf("  event: %s\n",
> bt_ctf_event_name(bt_event));
>                             printf("  position: %u\n", pos);
>                             printf("  field: %s\n",
> bt_ctf_field_name(field_def));
>                             printf("  count: %u\n", seq_items_count);
>                             goto end;
>                         }
>                     }
>                 }
>             }
>
>     next_event:
>             if (bt_iter_next(bt_ctf_get_iter(bt_ctf_iter)) < 0) {
>                 break;
>             }
>
>             ++pos;
>         }
>
>     end:
>         bt_ctf_iter_destroy(bt_ctf_iter);
>         bt_context_put(bt_context);
>
>         return 0;
>     }
>
> Running it with this trace <http://0x3b.org/bbt/bbt.tgz>, I get this:
>
>     error using bt_ctf_get_index()
>       event: scsi_dispatch_cmd_start
>       position: 46627
>       field: cmnd
>       count: 10
>
> So... the count is 10, and there's stuff in seq_items_list[6] to
> seq_items_list[9]
> which is not NULL, yet bt_ctf_get_index() complains at element 6 (not written
> in the results here, but I know it).
>
> Why is the count 10 if only 6 elements are good? The babeltrace tool knows this
> too:
>
>     $ babeltrace -i ctf kernel 2>/dev/null | sed -n 46627p
>     [21:40:00.281269992] (+0.000003741) eeppdesk
> scsi_dispatch_cmd_start: { cpu_id = 0 }, { host_no = 10, channel = 0,
> id = 0, lun = 0, opcode = 0, cmd_len = 6, data_sglen = 0, prot_sglen =
> 0, prot_op = 0, _cmnd_length = 6, cmnd = [ [0] = 0x0, [1] = 0x0, [2] =
> 0x0, [3] = 0x0, [4] = 0x0, [5] = 0x0 ] }
>
> See, cmd_len is 6 and we see 6 elements in the cmnd field.
>
> My question is still: where do I get this 6 from?
>
> Thanks,
> Phil

Here is what I think happens:

For each sequence definition, BT allocates a field list that only
grows. If it first encounters an event with this sequence where it
contains 6 events, it will allocate an array of 6 fields. Then, when
it encounters an event where the sequence has 10 events, the buffer is
grown to 10 fields. This happens in bt_sequence_rw, in
types/sequence.c. If, later, an event with a sequence of 6 elements is
read, only the first 6 slots of the fields array will be used.
Therefore, you (or the library) should always refer to the integer
definition to know the real count of events in the sequence, and NOT
to the allocated array length.

There is even a nice comment about that in bt_sequence_rw:

        /*
         * Yes, large sequences could be _painfully slow_ to parse due
         * to memory allocation for each event read. At least, never
         * shrink the sequence. Note: the sequence GArray len should
         * never be used as indicator of the current sequence length.
         * One should always look at the sequence->len->value._unsigned
         * value for that.
         */

bt_ctf_get_field_list however, returns you the length of the array:

        *list = (struct bt_definition const* const*) def_sequence->elems->pdata;
        *count = def_sequence->elems->len;

Changing that to get the actual length of the sequence makes your
example work (see patch in [1]). That would be a bug in babeltrace,
unless the intent of the bt_ctf_get_field_list is actually to return
all the allocated fields, but I don't see any reason to do that.

[1] http://paste.ubuntu.com/7497072/