From felix.moessbauer at siemens.com Mon Dec 1 04:34:23 2025 From: felix.moessbauer at siemens.com (MOESSBAUER, Felix) Date: Mon, 1 Dec 2025 09:34:23 +0000 Subject: babeltrace | sink.ctf.fs: add support to create LTTng index Message-ID: Hi, I recently created a babeltrace pull request that adds support to create an CTF LTTng index file along the CTF files [1]. It would be great, if someone could review it. [1] https://github.com/efficios/babeltrace/pull/123 Best regards, Felix Moessbauer -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany From eeppeliteloop at gmail.com Mon Dec 1 09:38:12 2025 From: eeppeliteloop at gmail.com (Philippe Proulx) Date: Mon, 1 Dec 2025 09:38:12 -0500 Subject: babeltrace | sink.ctf.fs: add support to create LTTng index In-Reply-To: References: Message-ID: On Mon, Dec 1, 2025 at 4:34?AM MOESSBAUER, Felix via lttng-dev wrote: > > Hi, > > I recently created a babeltrace pull request that adds support to > create an CTF LTTng index file along the CTF files [1]. > > It would be great, if someone could review it. Thank you for your pull requests. I'll need to look at this in detail, but I'm not against the idea, especially if it's guarded behind a no-by-default initialization parameter. That being said: ? We won't review on GitHub: please submit your change to our Gerrit Code Review system (). I know it's a bit more initial work on your side if you're not familiar with Gerrit, but the review process will be much easier. See to learn how to upload a change. For this reason, I'll close the PRs immediately. ? You'll need to add a few tests to validate your feature. ? I'll deal with the documentation (manual pages) myself. ? Given our current workload before the end of the year, I'm not sure I'll be able to review this before 2026. Thank you for your contribution and for your patience with our review process. Philippe Proulx EfficiOS > > [1] https://github.com/efficios/babeltrace/pull/123 > > Best regards, > Felix Moessbauer > > -- > Siemens AG > Linux Expert Center > Friedrich-Ludwig-Bauer-Str. 3 > 85748 Garching, Germany > From felix.moessbauer at siemens.com Wed Dec 3 04:58:22 2025 From: felix.moessbauer at siemens.com (MOESSBAUER, Felix) Date: Wed, 3 Dec 2025 09:58:22 +0000 Subject: babeltrace | sink.ctf.fs: add support to create LTTng index In-Reply-To: References: Message-ID: <3140f317060df66442077e0be78104695c873aca.camel@siemens.com> On Mon, 2025-12-01 at 09:38 -0500, Philippe Proulx wrote: > On Mon, Dec 1, 2025 at 4:34?AM MOESSBAUER, Felix via lttng-dev > wrote: > > > > Hi, > > > > I recently created a babeltrace pull request that adds support to > > create an CTF LTTng index file along the CTF files [1]. > > > > It would be great, if someone could review it. > > Thank you for your pull requests. > > I'll need to look at this in detail, but I'm not against the idea, > especially if it's guarded behind a no-by-default > initialization parameter. That's how I implemented it. > > That being said: > > ? We won't review on GitHub: please submit your change to our Gerrit > Code Review system (). > > I know it's a bit more initial work on your side if you're not > familiar with Gerrit, but the review process will be much easier. > > See to learn > how to upload a change. It would be great to add this note to the contributing guide as well. > > For this reason, I'll close the PRs immediately. > > ? You'll need to add a few tests to validate your feature. Sure. I will add them as I now known that the feature in general is well received within the babeltrace project > > ? I'll deal with the documentation (manual pages) myself. Thanks! > > ? Given our current workload before the end of the year, I'm not sure > I'll be able to review this before 2026. No problem. For me it would be great, if you could briefly comment if the chosen interface is OK (boolean input parameter called "create- lttng-index"). By that, I can continue the integration in bt2-ftrace- to-ctf [1] and once the feature becomes officially available I don't need to change the plugin again. [1] https://github.com/siemens/bt2-ftrace-to-ctf/pull/13 Best regards, Felix -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany From felix.moessbauer at siemens.com Thu Dec 4 05:10:24 2025 From: felix.moessbauer at siemens.com (MOESSBAUER, Felix) Date: Thu, 4 Dec 2025 10:10:24 +0000 Subject: babeltrace | sink.ctf.fs escaping of struct member names Message-ID: Hi, while working on the LTTng index addition, I noticed that the traces with index are not picked up correctly in trace-compass. The reason for that is the escaping of the package.context -> cpu_id field. When creating the CTF metadata with sink.ctf.fs, all user-provided struct member names are escaped with an underscore. As of CTF 1.8.3, readers are only recommended to strip single underscores from field names, but they don't have to. And unfortunately trace-compass does not do this for the package context cpu_id field (surprisingly only when having an index...). Just manually changing the packet->context _cpu_id to cpu_id fixes the issue. In my opinion, the escaping of the struct member names in the CTF writer is not correct according to the CTF 1.8.3 spec: Section 4.2.1 states, that reserved keywords must not be used and the recommended escaping is a underscore prefix. However, the spec does not state any other names that need to be escaped and by that also not names starting with an underscore itself. IOW: The CTF writer currently unconditionally escapes all struct member names, which I assume to be incorrect I tried fixing this with the attached patch, however this breaks the test-trace-copy.sh. To me it at least partially looks like these tests just check against the implementation, but not against the spec. Anyways, I would like to clarify this topic upfront before proposing any changes. I also tried to find the reason via a git blame, however there were that many refactorings that the original idea remains hidden. Best regards, Felix diff --git a/src/plugins/ctf/fs-sink/translate-trace-ir-to-ctf-ir.cpp b/src/plugins/ctf/fs-sink/translate-trace-ir-to-ctf-ir.cpp index 12ab213d1..43bc8e098 100644 --- a/src/plugins/ctf/fs-sink/translate-trace-ir-to-ctf-ir.cpp +++ b/src/plugins/ctf/fs-sink/translate-trace-ir-to-ctf-ir.cpp @@ -159,12 +159,6 @@ static inline bool must_protect_identifier(const char *name) } } } - /* Protect an identifier which already starts with `_` */ - if (name[0] == '_') { - must_protect = true; - goto end; - } - end: return must_protect; } @@ -182,7 +176,7 @@ static inline int cur_path_stack_push(ctf::sink::TraceIrToCtfIrCtx *ctx, const c if (name) { if (ctx->ctf_version == 1) { - if (force_protect_name) { + if (force_protect_name || must_protect_identifier(name)) { g_string_assign(field_path_elem->name, "_"); } @@ -641,7 +635,7 @@ translate_structure_field_class_members(ctf::sink::TraceIrToCtfIrCtx *ctx, member = bt_field_class_structure_borrow_member_by_index_const(ir_fc, i); name = bt_field_class_structure_member_get_name(member); memb_ir_fc = bt_field_class_structure_member_borrow_field_class_const(member); - ret = cur_path_stack_push(ctx, name, true, memb_ir_fc, &struct_fc->base); + ret = cur_path_stack_push(ctx, name, false, memb_ir_fc, &struct_fc->base); if (ret) { BT_CPPLOGE_SPEC(ctx->logger, "Cannot translate structure field class member: " -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany From eeppeliteloop at gmail.com Thu Dec 4 10:15:27 2025 From: eeppeliteloop at gmail.com (Philippe Proulx) Date: Thu, 4 Dec 2025 10:15:27 -0500 Subject: babeltrace | sink.ctf.fs escaping of struct member names In-Reply-To: References: Message-ID: On Thu, Dec 4, 2025 at 5:10?AM MOESSBAUER, Felix via lttng-dev wrote: > > In my opinion, the escaping of the struct member names in the CTF > writer is not correct according to the CTF 1.8.3 spec: Section 4.2.1 > states, that reserved keywords must not be used and the recommended > escaping is a underscore prefix. However, the spec does not state any > other names that need to be escaped and by that also not names starting > with an underscore itself. > > IOW: The CTF writer currently unconditionally escapes all struct member > names, which I assume to be incorrect It's not incorrect per se. Underscore as a member name escaping mechanism has always been a nightmare in CTF 1.8. For example, given: uint8 _len; string _strings[length here]; Should we use `_len` or `len`? And what about this: struct { uint8 meow; uint8 _meow; } _mix; LTTng chooses to escape everything with `_`, except the packet header, packet context, and event record header names it knows. We did exactly the same in `sink.ctf.fs`. CTF 1.8 doesn't specify `cpu_id` by the way; it's an LTTngism. CTF 2 solves all of that! Since CTF 1.8 is pretty much deprecated at this point, I want to invest as little time as possible in anything related to this legacy format, whatever the project. Just make it work considering the known use cases. Therefore, for your specific problem, I suggest that you specifically avoid the "protection" (escaping) for `cpu_id` in the packet context field class _only for LTTng traces_. It should be good enough and make Trace Compass work. Trace Compass could also unescape packet context member names, but it will be faster to do this in Babeltrace 2 and it will match the LTTng behaviour more closely. > > I tried fixing this with the attached patch, If you need comments on a patch, even if it's an RFC, please prefer Gerrit so that we can comment specific lines and follow the discussion more easily. BR, Philippe From felix.moessbauer at siemens.com Thu Dec 4 10:47:23 2025 From: felix.moessbauer at siemens.com (MOESSBAUER, Felix) Date: Thu, 4 Dec 2025 15:47:23 +0000 Subject: babeltrace | sink.ctf.fs escaping of struct member names In-Reply-To: References:

Message-ID: <6407e79bb0b5bc49f73b200d9a5f40eef14e70af.camel@siemens.com> On Thu, 2025-12-04 at 10:15 -0500, Philippe Proulx wrote: > On Thu, Dec 4, 2025 at 5:10?AM MOESSBAUER, Felix via lttng-dev > wrote: > > > > In my opinion, the escaping of the struct member names in the CTF > > writer is not correct according to the CTF 1.8.3 spec: Section 4.2.1 > > states, that reserved keywords must not be used and the recommended > > escaping is a underscore prefix. However, the spec does not state any > > other names that need to be escaped and by that also not names starting > > with an underscore itself. > > > > IOW: The CTF writer currently unconditionally escapes all struct member > > names, which I assume to be incorrect > > It's not incorrect per se. > > Underscore as a member name escaping mechanism has always been a > nightmare in CTF 1.8. For example, given: > > uint8 _len; > string _strings[length here]; > > Should we use `_len` or `len`? And what about this: > > struct { > uint8 meow; > uint8 _meow; > } _mix; Okay... that's the reason. Thanks for the example. > > LTTng chooses to escape everything with `_`, except the packet header, > packet context, and event record header names it knows. We did exactly > the same in `sink.ctf.fs`. > > CTF 1.8 doesn't specify `cpu_id` by the way; it's an LTTngism. Well... This is a more generic question if it is really the job of the serializer to fix possible name clashes in the output. Anyways, unfortunately LTTng uses this field and does not allow escaping. > > CTF 2 solves all of that! I would love to switch to CTF2, however we need support for it in trace-compass, which AFAIK is not even implemented yet. > > Since CTF 1.8 is pretty much deprecated at this point, I want to invest > as little time as possible in anything related to this legacy format, > whatever the project. Just make it work considering the known use cases. > > Therefore, for your specific problem, I suggest that you specifically > avoid the "protection" (escaping) for `cpu_id` in the packet context > field class _only for LTTng traces_. It should be good enough and make > Trace Compass work. Trace Compass could also unescape packet context > member names, but it will be faster to do this in Babeltrace 2 and it > will match the LTTng behaviour more closely. This is tricky, as we don't really know if the trace is an LTTng trace or not. However, we can hide it behind the proposed create-lttng-index flag, as it is anyways only needed in combination with an index (I still don't know why, but ok...). > > > > > I tried fixing this with the attached patch, > > If you need comments on a patch, even if it's an RFC, please prefer > Gerrit so that we can comment specific lines and follow the discussion > more easily. Ok, got it. I'll add the new implementation to the create-lttng-index series (as it anyways depends on this). Best regards, Felix > > BR, > > Philippe -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany From eeppeliteloop at gmail.com Thu Dec 4 11:01:05 2025 From: eeppeliteloop at gmail.com (Philippe Proulx) Date: Thu, 4 Dec 2025 11:01:05 -0500 Subject: babeltrace | sink.ctf.fs escaping of struct member names In-Reply-To: <6407e79bb0b5bc49f73b200d9a5f40eef14e70af.camel@siemens.com> References:

<6407e79bb0b5bc49f73b200d9a5f40eef14e70af.camel@siemens.com> Message-ID: On Thu, Dec 4, 2025 at 10:47?AM MOESSBAUER, Felix wrote: > > Well... This is a more generic question if it is really the job of the > serializer to fix possible name clashes in the output. Anyways, > unfortunately LTTng uses this field and does not allow escaping. Although I understand the question, I will not take the time to re-evaluate a design that involves CTF 1.8. I'm currently in a "quick fixes only" mode for this format, with the sole aim of keeping existing scenarios working. > I would love to switch to CTF2, however we need support for it in > trace-compass, which AFAIK is not even implemented yet. We're currently investigating the status of CTF 2 support in Trace Compass and what amount of work is remaining. We'll let you know. > This is tricky, as we don't really know if the trace is an LTTng trace > or not. However, we can hide it behind the proposed create-lttng-index > flag, as it is anyways only needed in combination with an index (I > still don't know why, but ok...). Yes we know. Have a look at make_lttng_trace_path_rel(). `create-lttng-index` would be another hint, but the trace environment has been good enough so far. Phil From felix.moessbauer at siemens.com Thu Dec 4 11:07:43 2025 From: felix.moessbauer at siemens.com (MOESSBAUER, Felix) Date: Thu, 4 Dec 2025 16:07:43 +0000 Subject: babeltrace | sink.ctf.fs escaping of struct member names In-Reply-To: References:

<6407e79bb0b5bc49f73b200d9a5f40eef14e70af.camel@siemens.com> Message-ID: <8eb94fdf35b763ed20b6470f737c55926cd0f0ca.camel@siemens.com> On Thu, 2025-12-04 at 11:01 -0500, Philippe Proulx wrote: > On Thu, Dec 4, 2025 at 10:47?AM MOESSBAUER, Felix > wrote: > > > > Well... This is a more generic question if it is really the job of the > > serializer to fix possible name clashes in the output. Anyways, > > unfortunately LTTng uses this field and does not allow escaping. > > Although I understand the question, I will not take the time to > re-evaluate a design that involves CTF 1.8. I'm currently in a "quick > fixes only" mode for this format, with the sole aim of keeping existing > scenarios working. > > > I would love to switch to CTF2, however we need support for it in > > trace-compass, which AFAIK is not even implemented yet. > > We're currently investigating the status of CTF 2 support in Trace > Compass and what amount of work is remaining. We'll let you know. > > > This is tricky, as we don't really know if the trace is an LTTng trace > > or not. However, we can hide it behind the proposed create-lttng-index > > flag, as it is anyways only needed in combination with an index (I > > still don't know why, but ok...). > > Yes we know. Have a look at make_lttng_trace_path_rel(). I already checked this function, but this is a bit too-strict, as really only adding the index breaks the import. I also want to be able to add the index to non lttng traces (which for instance lack some env metadata) as otherwise the ftrace-to-ctf converter needs to fake the metadata. > > `create-lttng-index` would be another hint, but the trace environment > has been good enough so far. This is easy to implement and works perfectly fine. By that, traces without an lttng index have "_cpu_id" and can be imported in trace- compass. Traces with an lttng index get "cpu_id" and also now import correctly. I hope this solution is acceptable. Anyways, thanks for the quick support. Felix > > Phil -- Siemens AG Linux Expert Center Friedrich-Ludwig-Bauer-Str. 3 85748 Garching, Germany From anajafizadeh at cs.stonybrook.edu Wed Dec 10 21:40:45 2025 From: anajafizadeh at cs.stonybrook.edu (Amir Najafi Zadeh) Date: Wed, 10 Dec 2025 21:40:45 -0500 Subject: Question about LTTng Context Message-ID: Hello everyone, I hope you?re doing well. My name is Amir, and I?m a PhD student in the Computer Science department at Stony Brook University, New York. I have a question about the LTTng context that I haven?t been able to find an answer for in the docs or man pages. Does LTTng support adding the cgroup ID or cgroup path as a context field? I want to filter my trace results based on cgroups. If this isn?t supported, are there any plans to add it in future versions, such as 2.14? For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 with kernel 6.8.0-87-generic. Cheers, Amir -- *Amirhossein Najafizadeh* *PhD Student, Computer Science Department, Stony Brook University, N.Y.File systems and Storage Lab (FSL)* najafizadeh21 at gmail.com https://amirhnajafiz.github.io/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From kstewart at efficios.com Thu Dec 11 09:30:23 2025 From: kstewart at efficios.com (Kienan Stewart) Date: Thu, 11 Dec 2025 09:30:23 -0500 Subject: Question about LTTng Context In-Reply-To: References: Message-ID: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> Hi Amir, On 12/10/25 9:40 PM, Amir Najafi Zadeh via lttng-dev wrote: > Hello everyone, > > I hope you?re doing well. My name is Amir, and I?m a PhD student in the > Computer Science department at Stony Brook University, New York. > > I have a question about the LTTng context that I haven?t been able to > find an answer for in the docs or man pages. Does LTTng support adding > the cgroup ID or cgroup path as a context field? I want to filter my > trace results based on cgroups. > I think you're looking for the `cgroup_ns` context which adds the inum of the cgroup namespace as a context field. thanks, kienan > If this isn?t supported, are there any plans to add it in future > versions, such as 2.14? > > For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 with > kernel 6.8.0-87-generic. > > Cheers, > Amir > -- > *Amirhossein Najafizadeh* > *PhD Student, Computer Science Department, Stony Brook University, N.Y. > File systems and Storage Lab (FSL) > > * > najafizadeh21 at gmail.com > https://amirhnajafiz.github.io/ From mjeanson at efficios.com Thu Dec 11 09:57:43 2025 From: mjeanson at efficios.com (Michael Jeanson) Date: Thu, 11 Dec 2025 09:57:43 -0500 Subject: Question about LTTng Context In-Reply-To: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> References: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> Message-ID: <073db198-6792-4caa-81f5-ac65f5d76f1e@efficios.com> On 12/11/25 09:30, Kienan Stewart via lttng-dev wrote: > Hi Amir, > > On 12/10/25 9:40 PM, Amir Najafi Zadeh via lttng-dev wrote: >> Hello everyone, >> >> I hope you?re doing well. My name is Amir, and I?m a PhD student in >> the Computer Science department at Stony Brook University, New York. >> >> I have a question about the LTTng context that I haven?t been able to >> find an answer for in the docs or man pages. Does LTTng support adding >> the cgroup ID or cgroup path as a context field? I want to filter my >> trace results based on cgroups. >> > > I think you're looking for the `cgroup_ns` context which adds the inum > of the cgroup namespace as a context field. This will give you the ID of the cgroup namespace but we don't have contexts for cgroups themselves. It's probably something we would like to have but there is no concrete plans on implementing this at the moment. > >> If this isn?t supported, are there any plans to add it in future >> versions, such as 2.14? >> >> For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 with >> kernel 6.8.0-87-generic. >> >> Cheers, >> Amir >> -- >> *Amirhossein Najafizadeh* >> *PhD Student, Computer Science Department, Stony Brook University, N.Y. >> File systems and Storage Lab (FSL) From anajafizadeh at cs.stonybrook.edu Thu Dec 11 10:51:50 2025 From: anajafizadeh at cs.stonybrook.edu (Amir Najafi Zadeh) Date: Thu, 11 Dec 2025 10:51:50 -0500 Subject: Question about LTTng Context In-Reply-To: <073db198-6792-4caa-81f5-ac65f5d76f1e@efficios.com> References: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> <073db198-6792-4caa-81f5-ac65f5d76f1e@efficios.com> Message-ID: Michael, Kienan, Thank you both for your replies. As you mentioned the cgroup namespace, it isn?t very helpful in some cases. Containers may share the same cgroup namespace, and the container runtime often avoids giving each container a private cgroup namespace for performance and security reasons. Because of this, the cgroup namespace can?t be used as a reliable or unique label for filtering trace logs. In my case, with container-d you can inspect a container, find its PID, and then check /proc//ns/cgroup, but you?ll see that different containers often have the same cgroup namespace ID (e.g., in the following logs you can see the calico-node and kube-proxy containers sharing a cgroup ns). ``` Name: calico-node CID: b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace PID: 2243 cgroup_ns_inode: 4026531835 Name: kube-proxy CID: 52f3c12def7a680f61a925ac92dd5ebd5eba6cad17d671efbfd1d40db7dff624 PID: 1851 cgroup_ns_inode: 4026531835 >> MATCH FOUND: shares ns with container: b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace (calico-node) ``` Thanks again for your answers. I hope this feature appears in a future patch. Best, Amir On Thu, Dec 11, 2025 at 9:57?AM Michael Jeanson wrote: > On 12/11/25 09:30, Kienan Stewart via lttng-dev wrote: > > Hi Amir, > > > > On 12/10/25 9:40 PM, Amir Najafi Zadeh via lttng-dev wrote: > >> Hello everyone, > >> > >> I hope you?re doing well. My name is Amir, and I?m a PhD student in > >> the Computer Science department at Stony Brook University, New York. > >> > >> I have a question about the LTTng context that I haven?t been able to > >> find an answer for in the docs or man pages. Does LTTng support adding > >> the cgroup ID or cgroup path as a context field? I want to filter my > >> trace results based on cgroups. > >> > > > > I think you're looking for the `cgroup_ns` context which adds the inum > > of the cgroup namespace as a context field. > > This will give you the ID of the cgroup namespace but we don't have > contexts for cgroups themselves. It's probably something we would like > to have but there is no concrete plans on implementing this at the moment. > > > > >> If this isn?t supported, are there any plans to add it in future > >> versions, such as 2.14? > >> > >> For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 with > >> kernel 6.8.0-87-generic. > >> > >> Cheers, > >> Amir > >> -- > >> *Amirhossein Najafizadeh* > >> *PhD Student, Computer Science Department, Stony Brook University, N.Y. > >> File systems and Storage Lab (FSL) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mjeanson at efficios.com Thu Dec 11 11:14:26 2025 From: mjeanson at efficios.com (Michael Jeanson) Date: Thu, 11 Dec 2025 11:14:26 -0500 Subject: Question about LTTng Context In-Reply-To: References: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> <073db198-6792-4caa-81f5-ac65f5d76f1e@efficios.com> Message-ID: On 12/11/25 10:51, Amir Najafi Zadeh wrote: > Michael, Kienan, Thank you both for your replies. As you mentioned the > cgroup namespace, it isn?t very helpful in some cases. Containers may > share the same cgroup namespace, and the container runtime often avoids > giving each container a private cgroup namespace for performance and > security reasons. Because of this, the cgroup namespace can?t be used as > a reliable or unique label for filtering trace logs. > > In my case, with container-d you can inspect a container, find its PID, > and then check /proc//ns/cgroup, but you?ll see that different > containers often have the same cgroup namespace ID (e.g., in the > following logs you can see the calico-node and kube-proxy containers > sharing a cgroup ns). If you want to track containers, the pid namespace is the one used by most container runtimes but I don't know specifically for container-d. > > ``` > Name: calico-node ?CID: > b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace ?PID: > 2243 ?cgroup_ns_inode: 4026531835 > Name: kube-proxy ?CID: > 52f3c12def7a680f61a925ac92dd5ebd5eba6cad17d671efbfd1d40db7dff624 ?PID: > 1851 ?cgroup_ns_inode: 4026531835 > ? >> MATCH FOUND: shares ns with container: > b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace > (calico-node) > ``` > > Thanks again for your answers. I hope this feature appears in a future > patch. > > Best, > Amir > > On Thu, Dec 11, 2025 at 9:57?AM Michael Jeanson > wrote: > > On 12/11/25 09:30, Kienan Stewart via lttng-dev wrote: > > Hi Amir, > > > > On 12/10/25 9:40 PM, Amir Najafi Zadeh via lttng-dev wrote: > >> Hello everyone, > >> > >> I hope you?re doing well. My name is Amir, and I?m a PhD student in > >> the Computer Science department at Stony Brook University, New York. > >> > >> I have a question about the LTTng context that I haven?t been > able to > >> find an answer for in the docs or man pages. Does LTTng support > adding > >> the cgroup ID or cgroup path as a context field? I want to > filter my > >> trace results based on cgroups. > >> > > > > I think you're looking for the `cgroup_ns` context which adds the > inum > > of the cgroup namespace as a context field. > > This will give you the ID of the cgroup namespace but we don't have > contexts for cgroups themselves. It's probably something we would like > to have but there is no concrete plans on implementing this at the > moment. > > > > >> If this isn?t supported, are there any plans to add it in future > >> versions, such as 2.14? > >> > >> For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 with > >> kernel 6.8.0-87-generic. > >> > >> Cheers, > >> Amir > >> -- > >> *Amirhossein Najafizadeh* > >> *PhD Student, Computer Science Department, Stony Brook > University, N.Y. > >> File systems and Storage Lab (FSL) > From anajafizadeh at cs.stonybrook.edu Thu Dec 11 11:21:35 2025 From: anajafizadeh at cs.stonybrook.edu (Amir Najafi Zadeh) Date: Thu, 11 Dec 2025 11:21:35 -0500 Subject: Question about LTTng Context In-Reply-To: References: <87a7e3d3-3ce2-4ded-acdf-bc7f152cdcec@efficios.com> <073db198-6792-4caa-81f5-ac65f5d76f1e@efficios.com>

Message-ID: I'll try that too. Thanks for mentioning it. Cheers, Amir On Thu, 11 Dec 2025, 11:14 Michael Jeanson, wrote: > On 12/11/25 10:51, Amir Najafi Zadeh wrote: > > Michael, Kienan, Thank you both for your replies. As you mentioned the > > cgroup namespace, it isn?t very helpful in some cases. Containers may > > share the same cgroup namespace, and the container runtime often avoids > > giving each container a private cgroup namespace for performance and > > security reasons. Because of this, the cgroup namespace can?t be used as > > a reliable or unique label for filtering trace logs. > > > > In my case, with container-d you can inspect a container, find its PID, > > and then check /proc//ns/cgroup, but you?ll see that different > > containers often have the same cgroup namespace ID (e.g., in the > > following logs you can see the calico-node and kube-proxy containers > > sharing a cgroup ns). > > If you want to track containers, the pid namespace is the one used by > most container runtimes but I don't know specifically for container-d. > > > > > ``` > > Name: calico-node CID: > > b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace PID: > > 2243 cgroup_ns_inode: 4026531835 > > Name: kube-proxy CID: > > 52f3c12def7a680f61a925ac92dd5ebd5eba6cad17d671efbfd1d40db7dff624 PID: > > 1851 cgroup_ns_inode: 4026531835 > > >> MATCH FOUND: shares ns with container: > > b5624afe31b005725f4ba53c7b6fe758f3c09fabf013085231ff8b97588f6ace > > (calico-node) > > ``` > > > > Thanks again for your answers. I hope this feature appears in a future > > patch. > > > > Best, > > Amir > > > > On Thu, Dec 11, 2025 at 9:57?AM Michael Jeanson > > wrote: > > > > On 12/11/25 09:30, Kienan Stewart via lttng-dev wrote: > > > Hi Amir, > > > > > > On 12/10/25 9:40 PM, Amir Najafi Zadeh via lttng-dev wrote: > > >> Hello everyone, > > >> > > >> I hope you?re doing well. My name is Amir, and I?m a PhD student > in > > >> the Computer Science department at Stony Brook University, New > York. > > >> > > >> I have a question about the LTTng context that I haven?t been > > able to > > >> find an answer for in the docs or man pages. Does LTTng support > > adding > > >> the cgroup ID or cgroup path as a context field? I want to > > filter my > > >> trace results based on cgroups. > > >> > > > > > > I think you're looking for the `cgroup_ns` context which adds the > > inum > > > of the cgroup namespace as a context field. > > > > This will give you the ID of the cgroup namespace but we don't have > > contexts for cgroups themselves. It's probably something we would > like > > to have but there is no concrete plans on implementing this at the > > moment. > > > > > > > >> If this isn?t supported, are there any plans to add it in future > > >> versions, such as 2.14? > > >> > > >> For reference, I?m currently using LTTng 2.13 on Ubuntu 24.04 > with > > >> kernel 6.8.0-87-generic. > > >> > > >> Cheers, > > >> Amir > > >> -- > > >> *Amirhossein Najafizadeh* > > >> *PhD Student, Computer Science Department, Stony Brook > > University, N.Y. > > >> File systems and Storage Lab (FSL) > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mathieu.desnoyers at efficios.com Fri Dec 19 17:31:58 2025 From: mathieu.desnoyers at efficios.com (Mathieu Desnoyers) Date: Fri, 19 Dec 2025 17:31:58 -0500 Subject: =?UTF-8?B?W1JFTEVBU0VdIExUVG5nIDIuMTUuMC1yYzEgLSBQw6ljaMOpIE1vcnRl?= =?UTF-8?Q?l_-_Linux_kernel_and_userspace_tracer?= Message-ID: <0628c50b-8ec6-4f84-a051-52674c2e9c1f@efficios.com> Hi everyone, We're excited to announce the availability of the first release candidate of LTTng 2.15, codenamed ?P?ch? Mortel?! Only a few months after LTTng 2.14, this release delivers an impressive number of features considering the short time frame, in particular: ? CTF 2 (Common Trace Format) as the default trace format. ? Reduced memory footprint for user space tracing. ? Improved resiliency of shared LTTng-UST buffers. ? Redesigned lttng-list(1) command with memory usage reporting. ? Extended automation capabilities for the lttng-add-trigger(1) command. Read on for the details. WHAT'S NEW IN LTTng 2.15? ????????????????????????? New features and changes in LTTng 2.15: ? Choose the trace format for each recording session with the new `--trace-format` option of the lttng-create(1) command: $ lttng create my-session --trace-format=ctf-1.8 Select between CTF 1.8 [1] and CTF 2 [2]. CTF 2 now is the default trace format for all newly created recording sessions. Babeltrace 2 [3] supports CTF 2 since version 2.1 (January 2025). To keep the default set to CTF 1.8 for your application, use the new `--default-trace-format` option of lttng-sessiond(8). The LTTng project adopts CTF 2 because it has many benefits over CTF 1.8: Easier decoding: CTF 1.8 requires parsing TSDL metadata, a CTF-specific language, which increases implementation time and limits tool integration. CTF 2 replaces this with JSON Text Sequences, readable by standard JSON parsers, significantly reducing new decoding code. Cleaner trace streaming: CTF 1.8 allows partial metadata items, forcing parsers to detect item boundaries manually, increasing complexity and bugs. CTF 2 separates metadata fragments with an RS byte, enabling simple chunk identification without decoding. Better trace readability (not in LTTng yet): CTF 1.8 lacks semantic annotations, so readers rely on external domain knowledge to interpret event record data. CTF 2 introduces user-defined attributes to describe semantics, enabling automated analysis (recognizing IP addresses, for example). Broader type support (not in LTTng yet): CTF 2 adds native field classes that can improve readability, performance, and trace size, unlike CTF 1.8's more limited types. Examples include a BLOB type for binary data, richer string encodings, bit maps, optionals, and LEB128 integers. This per-recording session configuration replaces the mechanism introduced in LTTng 2.14 where setting the `LTTNG_EXPERIMENTAL_FORCE_CTF_2` environment variable to `1` at `lttng-sessiond` startup forced _all_ recording sessions to produce CTF 2 traces. ? A new channel memory reclaim operation, exclusively available for user space channels, can free memory used by eligible sub-buffers by deallocating their backing memory when no longer needed. The two ways to reclaim memory are: Automatic reclaim: Configure a per-channel, automatic reclaim policy which periodically tries to reclaim memory for sub-buffers older than a given age, or as soon as they're consumed, with the new `--auto-reclaim-memory` option of the lttng-enable-channel(1) command. This new option is only available in discard mode (without the `--overwrite` option). Immediate reclaim: The new lttng-reclaim-memory(1) command reclaims memory immediately for eligible sub-buffers of one or more channels. Only reclaim the sub-buffers older than a given age with the `--older-than` option: $ lttng reclaim-memory --older-than=2m Both memory reclaim modes are possible because channel ring buffers are now backed by sparse files [4] when the file system supports them. ? Choose the buffer _preallocation_ policy of a user space channel with the new `--buffer-preallocation` option of the lttng-enable-channel(1) command: $ lttng enable-channel my-channel --userspace \ --buffer-preallocation=on-demand LTTng can either preallocate all ring buffer memory up front (pre-2.15 behaviour; still the default) or allocate as needed. Choose preallocation to ensure predictable memory usage and avoid run-time allocation latency; choose on demand to minimize the initial footprint for low traffic, for short-lived recording sessions, or in conjunction with memory reclaims. This feature is also enabled by sparse file-backed ring buffers. ? Before this release, an application terminated while writing to a user space ring-buffer could leave a sub-buffer stuck in a partially-written state, making it unusable. A new health check mechanism, the _watchdog timer_, now periodically monitors user space ring-buffers and makes stalled sub-buffers consumable again to keep trace recording running smoothly. This mechanism is enabled by default and can be controlled with the new `--watchdog-timer` option of the lttng-enable-channel(1) command. As of LTTng 2.15, it only applies to user space channels with a per-user buffer ownership model. ? We completely revamped the human-readable output of the lttng-list(1) command. The command now has a clean tree structure. Colors help improve the readability of objects and properties when the connected terminal supports it: the `LTTNG_TERM_COLOR` and `NO_COLOR` [5] environment variables provide further terminal color control. ? Channel `channel0` ?42 event rules? ? Auto. memory reclaim policy: None ? Loss mode: Discard newest event record ? Preallocation policy: On demand ? Ring buffer configuration: 4 sub-buffers of 512.00 KiB per CPU, per Unix user ? Timer periods: ? Monitor timer: 1.00 s ? Read timer: Inactive ? Switch timer: Inactive ? Watchdog timer: 2.00 s ? Statistics: ? Discarded event records: 23,182 By default, the command: ? Adds empty lines between blocks of related information. Remove those empty lines with the new `compact` setting of the new `--style` option. ? Truncates lines, adding an ellipsis, to fit the current terminal width. Avoid truncation with the new `--no-truncate` option. If you still prefer or need the previous (?legacy?) version, set the `LTTNG_LIST_LEGACY` environment variable to `1` before you run lttng-list(1). Please note, however, that the legacy output will never show anything related to features introduced after LTTng 2.14. ? The lttng-list(1) command now shows the total memory usage of each channel: ? Memory usage: ????????????????????????????? 19.07 MiB / 40.12 MiB The new `--mem-usage` option controls the display mode of memory usage: `compact` shows the memory usage for each Unix user or process while `full` shows the memory usage for each CPU (if available): ? Memory usage: ????????????????????????????? 13.30 MiB / 40.12 MiB ? For UID 1000 (64-bit): ?????????????????? 13.30 MiB / 40.12 MiB ? CPU 0: ?????????????????????????????? 540.00 KiB / 2.51 MiB ? CPU 1: ?????????????????????????????? 360.00 KiB / 2.51 MiB ? CPU 2: ?????????????????????????????? 1.30 MiB / 2.51 MiB ? CPU 3: ?????????????????????????????? 256.00 KiB / 2.51 MiB ? CPU 4: ?????????????????????????????? 1.20 MiB / 2.51 MiB ? CPU 5: ?????????????????????????????? 1.07 MiB / 2.51 MiB ? CPU 6: ?????????????????????????????? 768.00 KiB / 2.51 MiB ? CPU 7: ?????????????????????????????? 524.00 KiB / 2.51 MiB ? CPU 8: ?????????????????????????????? 792.00 KiB / 2.51 MiB ? The lttng-status(1) command now accepts the same options as lttng-list(1): $ lttng status --mem-usage=compact --channel=my-channel lttng-status(1) forwards its options as is to lttng-list(1) for the current recording session. ? The lttng(1) commands won't emit multi-byte UTF-8 sequences when the `LTTNG_NO_UTF_8` environment variable is set to `1`, making a best effort to find alternatives. ? New trigger conditions are now available for the lttng-add-trigger(1) command: `channel-buffer-usage-ge` and `channel-buffer-usage-le`: Satisfied when the buffer usage of a given channel becomes greater/less than or equal to some threshold (size in bytes or ratio of the total channel buffer size): $ lttng add-trigger --condition=channel-buffer-usage-ge \ --session=my-session --channel=my-channel \ --domain=kernel --threshold-ratio=0.75 \ --action=notify `session-consumed-size-ge`: Satisfied when the total consumed size of the tracing data of all the channels of a recording session becomes greater than or equal to a threshold: $ lttng add-trigger --condition=session-consumed-size-ge \ --session=my-session --threshold-size=100M \ --action=snapshot-session other-session `session-rotation-starts`: `session-rotation-finishes`: Satisfied when the rotation operation of a recording session starts or finishes: $ lttng add-trigger --condition=session-rotation-finishes \ --session=my-session --action=notify ? The `liblttng-ctl` C API is updated to support the new features above: ? Set the trace format of a recording session descriptor with lttng_session_descriptor_set_trace_format() before creating the recording session. ? Immediately reclaim user space channel memory for eligible sub-buffers with lttng_reclaim_channel_memory(). Get the results with lttng_reclaim_handle_get_reclaimed_subbuffer_count() and lttng_reclaim_handle_get_pending_subbuffer_count(). Optionally wait for pending sub-buffers to be reclaimed with lttng_reclaim_handle_wait_for_completion(). When done, destroy the memory reclaim operation handle with lttng_reclaim_handle_destroy(). ? Set and get the automatic memory reclamation policy of a user space channel with lttng_channel_set_automatic_memory_reclamation_policy() and lttng_channel_get_automatic_memory_reclamation_policy(). ? Set and get the buffer preallocation policy of a user space channel with lttng_channel_set_preallocation_policy() and lttng_channel_get_preallocation_policy(). ? Set and get the watchdog timer period of a channel with lttng_channel_set_watchdog_timer_interval() and lttng_channel_get_watchdog_timer_interval(). ? Enumerate and query the CPU ID and memory usage of the data streams of a channel with the data stream info API: ? lttng_channel_get_data_stream_info_sets() ? lttng_data_stream_info_sets_get_count() ? lttng_data_stream_info_sets_get_at_index() ? lttng_data_stream_info_set_get_count() ? lttng_data_stream_info_set_get_uid() ? lttng_data_stream_info_set_get_pid() ? lttng_data_stream_info_set_get_app_bitness() ? lttng_data_stream_info_set_get_at_index() ? lttng_data_stream_info_get_cpu_id() ? lttng_data_stream_info_get_memory_usage() ? lttng_data_stream_info_sets_destroy() ? The session daemon and relay daemon now support systemd's sd_notify() protocol, enabling proper synchronization with dependent systemd units by signaling when the daemons are ready to receive commands and when they're stopping. ? The internal v2.15 communication protocol between LTTng-tools and LTTng-UST is incompatible with the LTTng 2.14 one: you cannot use LTTng-tools 2.15 with LTTng-UST 2.14 and vice versa, for example. ? The lttng(1) Zsh completion function now fully completes the `add-trigger` command. From now on, the Debian/Ubuntu packages of LTTng-tools will contain the Zsh completion functions. VERSION NAME ???????????? This release is named after P?ch? Mortel, the iconic Imperial Stout from Montr?al's Dieu du Ciel! and a long-time team favorite. Deep black and unapologetically intense, P?ch? Mortel pours with a dense, mocha-tinted head and an aroma that immediately announces freshly roasted coffee. Decadent and dangerously drinkable, it is the ideal companion for long, cold winter evenings. IMPORTANT LINKS ??????????????? LTTng tarball: ? ? ? LTTng website: LTTng 2.15 documentation: Mailing list for support and development: IRC channel: `#lttng` on `irc.oftc.net` Bug tracker: GitHub organization: Continuous integration: ? ? ? Code review: ? ? ? REFERENCES ?????????? [1]: https://diamon.org/ctf/v1.8.3/ (CTF v1.8.3 specification) [2]: https://diamon.org/ctf/ (CTF 2 specification) [3]: https://babeltrace.org/ [4]: https://en.wikipedia.org/wiki/Sparse_file [5]: https://no-color.org/ -- Mathieu Desnoyers EfficiOS Inc. https://www.efficios.com