[lttng-dev] [Ext] Re: How to use lttng-live plugin with babeltrace2 python api for live LTTng trace reading?

[Erica Bugden]

You're welcome Ruoxiang!

On 2023-08-10 22:50, LI Ruoxiang wrote:
> HiErica,
> 
> Thank you for your kind help.
> 
> It works for my case.
> 
> Best,
> 
> Ruoxiang
> 
> *From: *Erica Bugden <ebugden at efficios.com>
> *Date: *Wednesday, August 9, 2023 at 05:29
> *To: *LI Ruoxiang <ruoxiang.li at my.cityu.edu.hk>, 
> lttng-dev at lists.lttng.org <lttng-dev at lists.lttng.org>
> *Subject: *[Ext] Re: How to use lttng-live plugin with babeltrace2 
> python api for live LTTng trace reading?
> 
> *CAUTION: External email. Do not reply, click on links or open 
> attachments unless you recognize the sender and know the content is safe. *
> 
> Hello Ruoxiang!
> 
> Thank you for your question. It's true that there are no Python bindings 
> examples specific to lttng live (we've provided a brief example below). 
> Unfortunately, the python bindings documentation is currently 
> incomplete, but information about using lttng live can be pieced 
> together using the various documentation links below (see Useful Links 
> section).
> 
> Some adjustments typically needed when using lttng live and the Python 
> bindings are:
> 
> - When referring to a trace source, use a URL (e.g. 
> net://localhost/host/luna/my-session) rather than a file path (e.g. 
> /path/to/trace)
> 
> - When querying, use the source.ctf.lttng-live component class (rather 
> than the file system class: source.ctf.fs) - source.ctf.lttng-live docs 
> https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ <https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/>
> 
> Hope this helps!
> 
> Best,
> 
> Erica
> 
> ----
> 
> Useful links
> 
> - Python bindings docs (Installation, Examples) - 
> https://babeltrace.org/docs/v2.0/python/bt2/index.html 
> <https://babeltrace.org/docs/v2.0/python/bt2/index.html>
> 
> - LTTng live, General information - 
> https://lttng.org/docs/v2.13/#doc-lttng-live 
> <https://lttng.org/docs/v2.13/#doc-lttng-live> (e.g. How to express a 
> live trace source: net://localhost/host/HOSTNAME/my-session)
> 
> - source.ctf.lttng-live component - 
> https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/ <https://babeltrace.org/docs/v2.0/man7/babeltrace2-source.ctf.lttng-live.7/> (C API doc not Python, but can be used to adapt the source.ctf.fs examples by comparing)
> 
> ----
> 
> Quick Example: Trace reading with python bindings and lttng live
> 
>       Note: This is a local example where the tracing and reading with 
> babeltrace are happening on the same machine (luna).
> 
> 1. REQUIREMENTS
> 
> Make sure babeltrace is installed with the python plugins and python 
> bindings: https://babeltrace.org/docs/v2.0/python/bt2/installation.html 
> <https://babeltrace.org/docs/v2.0/python/bt2/installation.html>
> 
> 2. PROCEDURE
> 
>   - Start root session daemon: $sudo lttng-sessiond --daemonize
> 
>   - Create live session: $lttng create my-session --live
> 
>   - Enable events: $lttng enable-event --kernel 
> sched_switch,sched_process_fork
> 
>   - Start tracing: $lttng start
> 
>   - Run python script: (see below) $python3 lttng-live.py
> 
> 3. PYTHON SCRIPT
> 
> File name: lttng-live.py
> 
> File contents:
> 
>       import bt2
> 
>       import time
> 
>       msg_iter = 
> bt2.TraceCollectionMessageIterator('net://localhost/host/luna/my-session') # The hostname (i.e. machine name) is 'luna'
> 
>       while True:
> 
>           try:
> 
>               for msg in msg_iter:
> 
>                   if type(msg) is bt2._EventMessageConst:
> 
>                       print(msg.event.name)
> 
>           except bt2.TryAgain:
> 
>               print('Try again. The iterator has no events to provide 
> right now.')
> 
>           time.sleep(0.5)
> 
> Reading trace data using the bt2.TraceCollectionMessageIterator and 
> lttng-live: When using the message iterator with live, the iterator 
> never ends by default and can be polled for trace data infinitely (hence 
> the while loop in the example). When there is available data, the 
> iterator will return it. However, there will not always be data 
> available to consume. When this is the case, the iterator returns a "Try 
> again" exception which must be caught in order to continue polling.
> 
> 4. OUTPUT SNIPPET (Example)
> 
>       erica at luna:~$ python3 lttng-live-example.py
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       Try again. The iterator has no events to provide right now.
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_process_fork
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       sched_switch
> 
>       [...]
> 
> ------------------------------------------------------------------------
> 
> *From:*lttng-dev <lttng-dev-bounces at lists.lttng.org> on behalf of LI 
> Ruoxiang via lttng-dev <lttng-dev at lists.lttng.org>
> *Sent:* August 3, 2023 12:44 PM
> *To:* lttng-dev at lists.lttng.org <lttng-dev at lists.lttng.org>
> *Subject:* [lttng-dev] How to use lttng-live plugin with babeltrace2 
> python api for live LTTng trace reading?
> 
> Hi there,
> 
> 
> 
> I am currently involved in a project on designing a Python program for 
> LTTng trace data analysis online. The following figure illustrates the 
> program with a live trace data reader using babeltrace2 Python bindings 
> (the yellow box) connected to the LTTng relay daemon. The program will 
> read (such as periodically)the trace data from the relay daemon and then 
> process them while the LLTng keeps tracing. The above “read” and 
> “process” phases repeat in a loop.
> 
> 
> 
> 
> 
> 
> 
> After reading the babeltrace2 documents, examples, and some source code, 
> I found the lttng-live plugin may be an option for reading trace data 
> from LTTng relay daemon. However, I didn't find any examples for using 
> lttng-live plugin with babeltrace2 Python bindings. And I wonder if the 
> Python bindings support the mentioned live LTTng trace reading for my 
> case. Is it possible to receive any examples about the usage of 
> babeltrace2’s live reading, if any?
> 
> Thank you.
> 
> Best,
> 
> Ruoxiang Li
> 
> 
>