DEBUG3 [9661/9661]: Creating LTTng run directory: /home/lttng/.lttng (in create_lttng_rundir() at main.c:4170) DEBUG1 [9661/9661]: Client socket path /home/lttng/.lttng/client-lttng-sessiond (in main() at main.c:4522) DEBUG1 [9661/9661]: Application socket path /home/lttng/.lttng/lttng-ust-sock-5 (in main() at main.c:4523) DEBUG1 [9661/9661]: Application wait path /lttng-ust-wait-5-16063 (in main() at main.c:4524) DEBUG1 [9661/9661]: LTTng run directory path: /home/lttng/.lttng (in main() at main.c:4525) DEBUG2 [9661/9661]: UST consumer 32 bits err path: /home/lttng/.lttng/ustconsumerd32/error (in main() at main.c:4534) DEBUG2 [9661/9661]: UST consumer 32 bits cmd path: /home/lttng/.lttng/ustconsumerd32/command (in main() at main.c:4536) DEBUG2 [9661/9661]: UST consumer 64 bits err path: /home/lttng/.lttng/ustconsumerd64/error (in main() at main.c:4545) DEBUG2 [9661/9661]: UST consumer 64 bits cmd path: /home/lttng/.lttng/ustconsumerd64/command (in main() at main.c:4547) DEBUG3 [9661/9661]: Created hashtable size 4 at 0xee0040 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9661]: Created hashtable size 4 at 0xee0440 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9661]: Created hashtable size 4 at 0xee0880 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9661]: Creating consumer directory: /home/lttng/.lttng/ustconsumerd64 (in set_consumer_sockets() at main.c:4212) DEBUG2 [9661/9661]: Creating consumer directory: /home/lttng/.lttng/ustconsumerd32 (in set_consumer_sockets() at main.c:4212) DEBUG1 [9661/9661]: Signal handler set for SIGTERM, SIGPIPE and SIGINT (in set_signal_handler() at main.c:4314) DEBUG3 [9661/9661]: Session daemon client socket 7 and application socket 8 created (in init_daemon_socket() at main.c:4075) DEBUG3 [9661/9661]: Created hashtable size 4 at 0xee0cc0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9661]: Global buffer per UID registry initialized (in buffer_reg_init_uid_registry() at buffer-registry.c:99) DEBUG3 [9661/9661]: Created hashtable size 4 at 0xee1100 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9661]: Global buffer per PID registry initialized (in buffer_reg_init_pid_registry() at buffer-registry.c:220) DEBUG1 [9661/9661]: epoll set max size is 1652162 (in compat_epoll_set_max_size() at compat-epoll.c:274) DEBUG1 [9661/9661]: Command subsystem initialized (in cmd_init() at cmd.c:2847) DEBUG1 [9661/9661]: Pid 9661 written in file /home/lttng/.lttng/lttng-sessiond.pid (in utils_create_pid_file() at utils.c:280) DEBUG1 [9661/9661]: TCP inet operation timeout set to 216 sec (in lttcomm_inet_init() at inet.c:523) DEBUG1 [9661/9661]: Health check time delta in seconds set to 236 (in health_init() at health.c:272) DEBUG1 [9661/9662]: [ht-thread] startup. (in thread_ht_cleanup() at ht-cleanup.c:34) DEBUG1 [9661/9664]: [thread] Manage client started (in thread_manage_clients() at main.c:3621) DEBUG1 [9661/9665]: [thread] Dispatch UST command started (in thread_dispatch_ust_registration() at main.c:1509) DEBUG1 [9661/9665]: Futex n to 1 prepare done (in futex_nto1_prepare() at futex.c:75) DEBUG1 [9661/9665]: Woken up but nothing in the UST command queue (in thread_dispatch_ust_registration() at main.c:1532) DEBUG1 [9661/9667]: [thread] Manage application started (in thread_manage_apps() at main.c:1218) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9661/9663]: [thread] Manage health check started (in thread_manage_health() at main.c:3418) DEBUG1 [9661/9668]: [ust-thread] Manage application notify command (in ust_thread_manage_notify() at ust-thread.c:37) DEBUG1 [9661/9667]: Apps thread polling on 2 fds (in thread_manage_apps() at main.c:1248) DEBUG1 [9661/9663]: Health check ready (in thread_manage_health() at main.c:3460) DEBUG3 [9661/9668]: [ust-thread] Manage notify polling on 2 fds (in ust_thread_manage_notify() at ust-thread.c:61) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9666]: [thread] Manage application registration started (in thread_registration_apps() at main.c:1732) DEBUG1 [9661/9666]: Notifying applications of session daemon state: 1 (in notify_ust_apps() at main.c:555) DEBUG1 [9661/9666]: Got the wait shm fd 22 (in get_wait_shm() at shm.c:115) DEBUG1 [9661/9666]: Futex wait update active 1 (in futex_wait_update() at futex.c:63) DEBUG1 [9661/9666]: Accepting application registration (in thread_registration_apps() at main.c:1769) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3758) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 8 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Waiting for 1 URIs from client ... (in process_client_msg() at main.c:3073) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff34005ea0 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9661/9664]: Tracing session appses created with ID 0 by UID 16063 GID 1001 (in session_create() at session.c:224) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff3400b330 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9664]: Setting trace directory path from URI to /home/lttng/lttng-traces/appses-20131224-124354 (in add_uri_to_consumer() at cmd.c:429) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Success) (in thread_manage_clients() at main.c:3803) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3758) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 6 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Getting session appses by name (in process_client_msg() at main.c:2655) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG1 [9661/9664]: Creating UST session (in create_ust_session() at main.c:2459) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff34003d60 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff340041a0 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9664]: UST trace session create successful (in trace_ust_create_session() at trace-ust.c:235) DEBUG3 [9661/9664]: Copying tracing session consumer output in UST session (in copy_session_consumer() at main.c:2409) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff340045e0 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9664]: Copy session consumer subdir /ust (in copy_session_consumer() at main.c:2427) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9661/9664]: Spawning consumerd (in spawn_consumerd() at main.c:2086) DEBUG2 [9661/9664]: Consumer pid 9755 (in start_consumerd() at main.c:2269) DEBUG2 [9661/9664]: Spawning consumer control thread (in start_consumerd() at main.c:2272) DEBUG1 [9661/9756]: [thread] Manage consumer started (in thread_manage_consumer() at main.c:922) DEBUG1 [9755/9755]: Using 64-bit UST consumer at: /prj/x86/test/packages/tools/lttng/lttng-build-root/lib/lttng/libexec/lttng-consumerd (in spawn_consumerd() at main.c:2161) DEBUG1 [9755/9755]: epoll set max size is 1652162 (in compat_epoll_set_max_size() at compat-epoll.c:274) DEBUG3 [9755/9755]: Created hashtable size 4 at 0x719030 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9755/9755]: Created hashtable size 4 at 0x719470 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9755/9755]: Created hashtable size 4 at 0x7198b0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9755/9755]: Created hashtable size 4 at 0x719cf0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9755/9755]: Connecting to error socket /home/lttng/.lttng/ustconsumerd64/error (in main() at lttng-consumerd.c:361) DEBUG2 [9661/9756]: Receiving code from consumer err_sock (in thread_manage_consumer() at main.c:1007) DEBUG3 [9755/9759]: Created hashtable size 4 at 0x7f3b200008e0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9755/9760]: Creating command socket /home/lttng/.lttng/ustconsumerd64/command (in consumer_thread_sessiond_poll() at consumer.c:2891) DEBUG1 [9755/9759]: Updating poll fd array (in update_poll_array() at consumer.c:957) DEBUG1 [9755/9759]: polling on 1 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG3 [9755/9757]: Created hashtable size 4 at 0x7f3b180008e0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9755/9757]: Thread channel poll started (in consumer_thread_channel_poll() at consumer.c:2661) DEBUG1 [9755/9760]: Sending ready command to lttng-sessiond (in consumer_thread_sessiond_poll() at consumer.c:2904) DEBUG1 [9755/9757]: Channel main loop started (in consumer_thread_channel_poll() at consumer.c:2676) DEBUG1 [9755/9757]: Channel poll wait with 1 fd(s) (in consumer_thread_channel_poll() at consumer.c:2685) DEBUG3 [9755/9758]: Created hashtable size 4 at 0x7f3b1c0008e0 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9755/9758]: Thread metadata poll started (in consumer_thread_metadata_poll() at consumer.c:2157) DEBUG1 [9661/9756]: Consumer command socket ready (fd: 25 (in thread_manage_consumer() at main.c:1042) DEBUG1 [9755/9758]: Metadata main loop started (in consumer_thread_metadata_poll() at consumer.c:2173) DEBUG1 [9755/9758]: Metadata poll wait with 1 fd(s) (in consumer_thread_metadata_poll() at consumer.c:2182) DEBUG1 [9755/9760]: Connection on client_socket (in consumer_thread_sessiond_poll() at consumer.c:2921) DEBUG3 [9661/9664]: Consumer socket created (fd: 25) and added to output (in consumer_create_socket() at consumer.c:298) DEBUG1 [9661/9664]: Setting relayd for session appses (in cmd_setup_relayd() at cmd.c:665) DEBUG1 [9755/9760]: Metadata connection on client_socket (in set_metadata_socket() at consumer.c:2860) DEBUG1 [9661/9664]: Enabling channel for session appses (in cmd_enable_channel() at cmd.c:849) DEBUG2 [9661/9664]: Trace UST channel channel0 not found by name (in trace_ust_find_channel_by_name() at trace-ust.c:149) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff34004f20 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9664]: Created hashtable size 4 at 0x7eff340052f0 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9664]: Trace UST channel channel0 created (in trace_ust_create_channel() at trace-ust.c:302) DEBUG2 [9661/9664]: Channel channel0 is being created for UST with buffer 1 and id 0 (in channel_ust_create() at channel.c:343) DEBUG2 [9661/9664]: UST app adding channel channel0 to UST domain for session id 0 (in ust_app_create_channel_glb() at ust-app.c:3511) DEBUG2 [9661/9664]: Channel channel0 created successfully (in channel_ust_create() at channel.c:367) DEBUG2 [9661/9664]: Trace UST channel channel0 found by name (in trace_ust_find_channel_by_name() at trace-ust.c:144) DEBUG2 [9661/9664]: Trace UST event * NOT found (in trace_ust_find_event() at trace-ust.c:183) DEBUG2 [9661/9664]: Trace UST event *, loglevel (0,-1) created (in trace_ust_create_event() at trace-ust.c:374) DEBUG1 [9661/9664]: UST app creating event * for all apps for session id 0 (in ust_app_create_event_glb() at ust-app.c:3664) DEBUG1 [9661/9664]: Event UST * created in channel channel0 (in event_ust_enable_tracepoint() at event.c:467) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Success) (in thread_manage_clients() at main.c:3803) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3758) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 16 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Getting session appses by name (in process_client_msg() at main.c:2655) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG1 [9661/9664]: Starting all UST traces (in ust_app_start_trace_all() at ust-app.c:4011) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Success) (in thread_manage_clients() at main.c:3803) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9666]: UST registration received with pid:9815 ppid:7338 uid:16063 gid:1001 sock:22 name:sample (version 5.0) (in thread_registration_apps() at main.c:1877) DEBUG1 [9661/9666]: Futex n to 1 wake done (in futex_nto1_wake() at futex.c:104) DEBUG1 [9661/9666]: Accepting application registration (in thread_registration_apps() at main.c:1769) DEBUG1 [9661/9665]: Futex n to 1 wait done (in futex_nto1_wait() at futex.c:90) DEBUG1 [9661/9665]: Futex n to 1 prepare done (in futex_nto1_prepare() at futex.c:75) DEBUG1 [9661/9665]: Dispatching UST registration pid:9815 ppid:7338 uid:16063 gid:1001 sock:22 name:sample (version 5.0) (in thread_dispatch_ust_registration() at main.c:1544) DEBUG3 [9661/9665]: UST app creating application for socket 22 (in ust_app_create() at ust-app.c:2763) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18000a10 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18000e50 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG1 [9661/9665]: Woken up but nothing in the UST command queue (in thread_dispatch_ust_registration() at main.c:1532) DEBUG1 [9661/9666]: UST registration received with pid:9815 ppid:7338 uid:16063 gid:1001 sock:27 name:sample (version 5.0) (in thread_registration_apps() at main.c:1877) DEBUG1 [9661/9666]: Futex n to 1 wake done (in futex_nto1_wake() at futex.c:104) DEBUG1 [9661/9665]: Futex n to 1 wait done (in futex_nto1_wait() at futex.c:90) DEBUG1 [9661/9665]: Futex n to 1 prepare done (in futex_nto1_prepare() at futex.c:75) DEBUG1 [9661/9666]: Accepting application registration (in thread_registration_apps() at main.c:1769) DEBUG1 [9661/9665]: Dispatching UST registration pid:9815 ppid:7338 uid:16063 gid:1001 sock:27 name:sample (version 5.0) (in thread_dispatch_ust_registration() at main.c:1544) DEBUG3 [9661/9665]: UST app notify socket 27 is set (in thread_dispatch_ust_registration() at main.c:1601) DEBUG1 [9661/9665]: App registered with pid:9815 ppid:7338 uid:16063 gid:1001 sock:22 name:sample notify_sock:27 (version 5.0) (in ust_app_add() at ust-app.c:2851) DEBUG2 [9661/9665]: UST app global update for app sock 22 for session id 0 (in ust_app_global_update() at ust-app.c:4150) DEBUG2 [9661/9665]: UST app pid: 9815 session id 0 not found, creating it (in create_ust_app_session() at ust-app.c:1735) DEBUG3 [9661/9668]: UST thread notify added sock 27 to pollset (in ust_thread_manage_notify() at ust-thread.c:131) DEBUG3 [9661/9668]: [ust-thread] Manage notify polling on 3 fds (in ust_thread_manage_notify() at ust-thread.c:61) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18002340 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9665]: Shadow copy of session handle -1 (in shadow_copy_session() at ust-app.c:1494) DEBUG2 [9661/9665]: Channel channel0 not found on shadow session copy, creating it (in shadow_copy_session() at ust-app.c:1541) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff180029c0 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18002e00 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: UST app channel channel0 allocated (in alloc_ust_app_channel() at ust-app.c:844) DEBUG2 [9661/9665]: UST app shadow copy of channel channel0 started (in shadow_copy_channel() at ust-app.c:1421) DEBUG2 [9661/9665]: UST event * not found on shadow copy channel (in shadow_copy_channel() at ust-app.c:1461) DEBUG3 [9661/9665]: UST app event * allocated (in alloc_ust_app_event() at ust-app.c:900) DEBUG3 [9661/9665]: UST app shadow copy of channel channel0 done (in shadow_copy_channel() at ust-app.c:1471) DEBUG3 [9661/9665]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18003650 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: Buffer registry per UID created id: 0, ABI: 64, uid: 16063, domain: 2 (in buffer_reg_uid_create() at buffer-registry.c:144) DEBUG3 [9661/9665]: Buffer registry per UID adding to global registry with id: 0 (in buffer_reg_uid_add() at buffer-registry.c:166) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18003b30 of type 2 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: Append to metadata: "typealias integer { size = 8; align = 8; signed = false; } := uint8_t; typealias integer { size = 16; align = 8; signed = false; } := uint16_t; typealias integer { size = 32; align = 8; signed = false; } := uint32_t; typealias integer { size = 64; align = 8; signed = false; } := uint64_t; typealias integer { size = 64; align = 8; signed = false; } := unsigned long; typealias integer { size = 5; align = 1; signed = false; } := uint5_t; typealias integer { size = 27; align = 1; signed = false; } := uint27_t; trace { major = 1; minor = 8; uuid = "3d9be8ad-48c5-48b2-a16e-c2df211a4dda"; byte_order = le; packet.header := struct { uint32_t magic; uint8_t uuid[16]; uint32_t stream_id; }; }; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "env { hostname = "main-ctrl"; domain = "ust"; tracer_name = "lttng-ust"; tracer_major = 2; tracer_minor = 3; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "}; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "clock { name = monotonic; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: " uuid = "e167bc68-508b-49c7-99cb-1394766cd20b"; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: " description = "Monotonic Clock"; freq = 1000000000; /* Frequency, in Hz */ /* clock value offset from Epoch is: offset * (1/freq) */ offset = 1387885209000000008; }; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "typealias integer { size = 27; align = 1; signed = false; map = clock.monotonic.value; } := uint27_clock_monotonic_t; typealias integer { size = 32; align = 8; signed = false; map = clock.monotonic.value; } := uint32_clock_monotonic_t; typealias integer { size = 64; align = 8; signed = false; map = clock.monotonic.value; } := uint64_clock_monotonic_t; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "struct packet_context { uint64_clock_monotonic_t timestamp_begin; uint64_clock_monotonic_t timestamp_end; uint64_t content_size; uint64_t packet_size; unsigned long events_discarded; uint32_t cpu_id; }; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: Append to metadata: "struct event_header_compact { enum : uint5_t { compact = 0 ... 30, extended = 31 } id; variant { struct { uint27_clock_monotonic_t timestamp; } compact; struct { uint32_t id; uint64_clock_monotonic_t timestamp; } extended; } v; } align(8); struct event_header_large { enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id; variant { struct { uint32_clock_monotonic_t timestamp; } compact; struct { uint32_t id; uint64_clock_monotonic_t timestamp; } extended; } v; } align(8); " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9665]: UST app buffer registry per UID created successfully (in setup_buffer_reg_uid() at ust-app.c:1697) DEBUG2 [9661/9665]: UST app session created successfully with handle 1 (in create_ust_app_session() at ust-app.c:1803) DEBUG1 [9661/9665]: UST app creating channel channel0 with per UID buffers (in create_channel_per_uid() at ust-app.c:2330) DEBUG3 [9661/9665]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG2 [9661/9665]: UST app creating buffer registry channel for channel0 (in create_buffer_reg_channel() at ust-app.c:2183) DEBUG3 [9661/9665]: Buffer registry channel create with key: 0 (in buffer_reg_channel_create() at buffer-registry.c:326) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18004080 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG2 [9661/9665]: Asking UST consumer for channel (in ask_channel_creation() at ust-consumer.c:118) DEBUG3 [9661/9665]: mkdir() recursive /home/lttng/lttng-traces/appses-20131224-124354/ust/uid/16063/64-bit with mode 504 for uid 16063 and gid 1001 (in run_as_mkdir_recursive() at runas.c:310) DEBUG1 [9661/9665]: Using run_as_clone (in run_as() at runas.c:293) DEBUG3 [9661/9665]: UST registry channel finding key 0 (in ust_registry_channel_find() at ust-registry.c:489) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: Allocated channel (key 1) (in consumer_allocate_channel() at consumer.c:889) DEBUG3 [9755/9760]: Creating channel to ustctl with attr: [overwrite: 0, subbuf_size: 131072, num_subbuf: 4, switch_timer_interval: 0, read_timer_interval: 0, output: 0, type: 0 (in create_ust_channel() at ust-consumer.c:353) DEBUG3 [9755/9760]: Allocated stream channel0_0 (key 22, chan_key 1 relayd_id 18446744073709551615, session_id 0 (in consumer_allocate_stream() at consumer.c:538) DEBUG3 [9755/9760]: open() /home/lttng/lttng-traces/appses-20131224-124354/ust/uid/16063/64-bit/channel0_0 with flags 241 mode 432 for uid 16063 and gid 1001 (in run_as_open() at runas.c:338) DEBUG1 [9755/9760]: Using run_as_clone (in run_as() at runas.c:293) DEBUG1 [9755/9760]: UST consumer add stream channel0_0 (key: 22) with relayd id 0 (in create_ust_streams() at ust-consumer.c:313) DEBUG3 [9755/9760]: Allocated stream channel0_1 (key 25, chan_key 1 relayd_id 18446744073709551615, session_id 0 (in consumer_allocate_stream() at consumer.c:538) DEBUG3 [9755/9760]: open() /home/lttng/lttng-traces/appses-20131224-124354/ust/uid/16063/64-bit/channel0_1 with flags 241 mode 432 for uid 16063 and gid 1001 (in run_as_open() at runas.c:338) DEBUG1 [9755/9760]: Using run_as_clone (in run_as() at runas.c:293) DEBUG1 [9755/9760]: UST consumer add stream channel0_1 (key: 25) with relayd id 0 (in create_ust_streams() at ust-consumer.c:313) DEBUG1 [9755/9760]: UST consumer channel added (key: 1) (in add_channel() at ust-consumer.c:106) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG2 [9661/9665]: UST ask channel 1 successfully done with 2 stream(s) (in ask_channel_creation() at ust-consumer.c:187) DEBUG1 [9755/9757]: Channel event catched in thread (in consumer_thread_channel_poll() at consumer.c:2687) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9757]: Adding channel 17 to poll set (in consumer_thread_channel_poll() at consumer.c:2729) DEBUG1 [9755/9760]: UST consumer sending channel channel0 to sessiond (in send_sessiond_channel() at ust-consumer.c:409) DEBUG1 [9755/9757]: Channel poll wait with 2 fd(s) (in consumer_thread_channel_poll() at consumer.c:2685) DEBUG1 [9755/9760]: UST consumer sending stream 22 to sessiond (in send_sessiond_stream() at ust-consumer.c:381) DEBUG1 [9755/9760]: UST consumer sending stream 25 to sessiond (in send_sessiond_stream() at ust-consumer.c:381) DEBUG1 [9755/9760]: UST consumer NULL stream sent to sessiond (in send_sessiond_channel() at ust-consumer.c:464) DEBUG3 [9755/9760]: Adding consumer stream 22 (in consumer_add_data_stream() at consumer.c:564) DEBUG3 [9755/9760]: Adding consumer stream 25 (in consumer_add_data_stream() at consumer.c:564) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: consumer_data_pipe wake up (in consumer_thread_data_poll() at consumer.c:2428) DEBUG1 [9755/9759]: Updating poll fd array (in update_poll_array() at consumer.c:957) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: consumer_data_pipe wake up (in consumer_thread_data_poll() at consumer.c:2428) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG2 [9661/9665]: UST app stream 1 received succesfully (in ust_consumer_get_channel() at ust-consumer.c:302) DEBUG2 [9661/9665]: UST app stream 2 received succesfully (in ust_consumer_get_channel() at ust-consumer.c:302) DEBUG3 [9661/9665]: UST app consumer has no more stream available (in ust_consumer_get_channel() at ust-consumer.c:285) DEBUG2 [9661/9665]: UST app setup buffer registry channel for channel0 (in setup_buffer_reg_channel() at ust-app.c:2231) DEBUG2 [9661/9665]: UST app setup buffer registry stream (in setup_buffer_reg_streams() at ust-app.c:2138) DEBUG3 [9661/9665]: Buffer registry creating stream (in buffer_reg_stream_create() at buffer-registry.c:356) DEBUG3 [9661/9665]: Buffer registry creating stream (in buffer_reg_stream_create() at buffer-registry.c:356) DEBUG1 [9661/9665]: UST app sending buffer registry channel to ust sock 22 (in send_channel_uid_to_ust() at ust-app.c:2267) DEBUG2 [9661/9665]: UST app send channel to sock 22 pid 9815 (name: channel0, key: 0) (in ust_consumer_send_channel_to_ust() at ust-consumer.c:405) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG2 [9661/9665]: UST consumer send stream to app 22 (in ust_consumer_send_stream_to_ust() at ust-consumer.c:370) DEBUG2 [9661/9665]: UST consumer send stream to app 22 (in ust_consumer_send_stream_to_ust() at ust-consumer.c:370) DEBUG2 [9661/9665]: UST app event * created successfully for pid:9815 (in create_ust_event() at ust-app.c:1347) DEBUG1 [9661/9665]: Starting tracing for ust app pid 9815 (in ust_app_start_trace() at ust-app.c:3719) DEBUG3 [9661/9665]: mkdir() recursive /home/lttng/lttng-traces/appses-20131224-124354 with mode 504 for uid 16063 and gid 1001 (in run_as_mkdir_recursive() at runas.c:310) DEBUG1 [9661/9665]: Using run_as_clone (in run_as() at runas.c:293) DEBUG3 [9661/9665]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff18006630 of type 1 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: Created hashtable size 4 at 0x7eff180069e0 of type 0 (in lttng_ht_new() at hashtable.c:112) DEBUG3 [9661/9665]: UST app channel metadata allocated (in alloc_ust_app_channel() at ust-app.c:844) DEBUG2 [9661/9665]: Asking UST consumer for channel (in ask_channel_creation() at ust-consumer.c:118) DEBUG3 [9661/9665]: mkdir() recursive /home/lttng/lttng-traces/appses-20131224-124354/ust/uid/16063/64-bit with mode 504 for uid 16063 and gid 1001 (in run_as_mkdir_recursive() at runas.c:310) DEBUG1 [9661/9665]: Using run_as_clone (in run_as() at runas.c:293) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: Allocated channel (key 2) (in consumer_allocate_channel() at consumer.c:889) DEBUG3 [9755/9760]: Creating channel to ustctl with attr: [overwrite: 0, subbuf_size: 4096, num_subbuf: 2, switch_timer_interval: 0, read_timer_interval: 0, output: 0, type: 1 (in create_ust_channel() at ust-consumer.c:353) DEBUG3 [9755/9760]: Allocated stream metadata (key 34, chan_key 2 relayd_id 18446744073709551615, session_id 0 (in consumer_allocate_stream() at consumer.c:538) DEBUG3 [9755/9760]: open() /home/lttng/lttng-traces/appses-20131224-124354/ust/uid/16063/64-bit/metadata with flags 241 mode 432 for uid 16063 and gid 1001 (in run_as_open() at runas.c:338) DEBUG1 [9755/9760]: Using run_as_clone (in run_as() at runas.c:293) DEBUG1 [9755/9760]: UST consumer add stream metadata (key: 34) with relayd id 0 (in create_ust_streams() at ust-consumer.c:313) DEBUG1 [9755/9760]: Allocated metadata cache of 4096 bytes (in consumer_metadata_cache_allocate() at consumer-metadata-cache.c:158) DEBUG1 [9755/9760]: UST consumer channel added (key: 2) (in add_channel() at ust-consumer.c:106) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG2 [9661/9665]: UST ask channel 2 successfully done with 1 stream(s) (in ask_channel_creation() at ust-consumer.c:187) DEBUG2 [9661/9665]: Consumer setup metadata channel key 2 (in consumer_setup_metadata() at consumer.c:1169) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: UST consumer setup metadata key 2 (in setup_metadata() at ust-consumer.c:700) DEBUG3 [9755/9760]: Adding metadata stream 34 to hash table (in consumer_add_metadata_stream() at consumer.c:2025) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG1 [9755/9758]: Metadata event catched in thread (in consumer_thread_metadata_poll() at consumer.c:2184) DEBUG2 [9661/9665]: UST metadata with key 2 created for app pid 9815 (in create_ust_app_metadata() at ust-app.c:2707) DEBUG1 [9755/9758]: Adding metadata stream 34 to poll set (in consumer_thread_metadata_poll() at consumer.c:2232) DEBUG3 [9661/9665]: UST app deleting channel metadata (in delete_ust_app_channel() at ust-app.c:357) DEBUG1 [9755/9758]: Metadata poll wait with 2 fd(s) (in consumer_thread_metadata_poll() at consumer.c:2182) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9668]: UST app receiving notify from sock 27 (in ust_app_recv_notify() at ust-app.c:4787) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG2 [9661/9668]: UST app ustctl register channel received (in ust_app_recv_notify() at ust-app.c:4840) DEBUG3 [9661/9668]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG3 [9661/9668]: UST registry channel finding key 0 (in ust_registry_channel_find() at ust-registry.c:489) DEBUG3 [9661/9668]: Append to metadata: "stream { id = 0; event.header := struct event_header_compact; packet.context := struct packet_context; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: Append to metadata: "}; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: UST app replying to register channel key 0 with id 0, type: 1, ret: 0 (in reply_ust_register_channel() at ust-app.c:4653) DEBUG3 [9661/9668]: [ust-thread] Manage notify polling on 3 fds (in ust_thread_manage_notify() at ust-thread.c:61) DEBUG2 [9661/9665]: UST trace started for app pid 9815 (in ust_app_global_update() at ust-app.c:4241) DEBUG1 [9661/9665]: Woken up but nothing in the UST command queue (in thread_dispatch_ust_registration() at main.c:1532) DEBUG1 [9661/9667]: Apps with sock 22 added to poll set (in thread_manage_apps() at main.c:1311) DEBUG1 [9661/9667]: Apps thread polling on 3 fds (in thread_manage_apps() at main.c:1248) DEBUG3 [9661/9668]: UST app receiving notify from sock 27 (in ust_app_recv_notify() at ust-app.c:4787) DEBUG2 [9661/9668]: UST app ustctl register event received (in ust_app_recv_notify() at ust-app.c:4807) DEBUG3 [9661/9668]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG3 [9661/9668]: UST registry channel finding key 0 (in ust_registry_channel_find() at ust-registry.c:489) DEBUG3 [9661/9668]: UST registry creating event with event: sample_component:message, sig: char *, text, id: 0, chan_objd: 2, sess_objd: 1, chan_id: 0 (in ust_registry_create_event() at ust-registry.c:298) DEBUG3 [9661/9668]: Append to metadata: "event { name = "sample_component:message"; id = 0; stream_id = 0; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: Append to metadata: " loglevel = 4; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: Append to metadata: " fields := struct { " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: Append to metadata: " string _message; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: Append to metadata: " }; }; " (in lttng_metadata_printf() at ust-metadata.c:144) DEBUG3 [9661/9668]: UST registry event sample_component:message with id 0 added successfully (in add_event_ust_registry() at ust-app.c:4768) DEBUG3 [9661/9668]: [ust-thread] Manage notify polling on 3 fds (in ust_thread_manage_notify() at ust-thread.c:61) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 131072 (len 131072) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9661/9668]: UST app notify socket unregister 27 (in ust_app_notify_sock_unregister() at ust-app.c:4914) DEBUG1 [9661/9667]: PID 9815 unregistering with sock 22 (in ust_app_unregister() at ust-app.c:2902) DEBUG3 [9661/9668]: [ust-thread] Manage notify polling on 2 fds (in ust_thread_manage_notify() at ust-thread.c:61) DEBUG3 [9661/9667]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG2 [9661/9667]: Consumer push metadata to consumer socket 25 (in consumer_push_metadata() at consumer.c:1202) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: UST consumer push metadata key 2 of len 2422 (in lttng_ustconsumer_recv_cmd() at ust-consumer.c:1382) DEBUG3 [9661/9667]: Consumer pushing metadata on sock 25 of len 2422 (in consumer_push_metadata() at consumer.c:1216) DEBUG1 [9755/9760]: UST consumer push metadata key 2 of len 2422 (in lttng_ustconsumer_recv_metadata() at ust-consumer.c:1030) DEBUG1 [9755/9760]: Writing 2422 bytes from offset 0 in metadata cache (in consumer_metadata_cache_write() at consumer-metadata-cache.c:90) DEBUG1 [9755/9760]: Waiting for metadata to be flushed (in lttng_ustconsumer_recv_metadata() at ust-consumer.c:1063) DEBUG1 [9755/9758]: Metadata event catched in thread (in consumer_thread_metadata_poll() at consumer.c:2184) DEBUG1 [9755/9758]: Metadata available on fd 34 (in consumer_thread_metadata_poll() at consumer.c:2284) DEBUG1 [9755/9758]: In UST read_subbuffer (wait_fd: 34, name: metadata) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9758]: Consumer mmap write() ret 4096 (len 4096) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9758]: In UST read_subbuffer (wait_fd: 34, name: metadata) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9758]: Metadata poll wait with 2 fd(s) (in consumer_thread_metadata_poll() at consumer.c:2182) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG1 [9661/9667]: Apps thread polling on 2 fds (in thread_manage_apps() at main.c:1248) DEBUG3 [9661/9824]: Call RCU deleting app PID 9815 (in delete_ust_app_rcu() at ust-app.c:743) DEBUG3 [9661/9824]: Buffer registry per UID find id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_find() at buffer-registry.c:195) DEBUG3 [9661/9824]: No metadata to push for metadata key 2 (in ust_app_push_metadata() at ust-app.c:442) DEBUG3 [9661/9824]: UST app deleting channel channel0 (in delete_ust_app_channel() at ust-app.c:357) DEBUG2 [9661/9824]: UST app pid 9815 deleted (in delete_ust_app() at ust-app.c:728) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3758) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 17 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Getting session appses by name (in process_client_msg() at main.c:2655) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG1 [9661/9664]: Stopping all UST traces (in ust_app_stop_trace_all() at ust-app.c:4037) DEBUG2 [9661/9664]: Consumer flush channel key 1 (in consumer_flush_channel() at consumer.c:1105) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: UST consumer flush channel key 1 (in flush_channel() at ust-consumer.c:586) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG1 [9755/9759]: poll num_rdy : 2 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 24576 (len 24576) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 8192 (len 8192) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG3 [9661/9664]: No metadata to push for metadata key 2 (in ust_app_push_metadata() at ust-app.c:442) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Success) (in thread_manage_clients() at main.c:3803) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 24 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Getting session appses by name (in process_client_msg() at main.c:2655) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG3 [9661/9664]: Consumer data pending for id 0 (in consumer_is_data_pending() at consumer.c:1053) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: UST consumer data pending command for id 0 (in lttng_ustconsumer_recv_cmd() at ust-consumer.c:1164) DEBUG1 [9755/9760]: Consumer data pending command on session id 0 (in consumer_data_pending() at consumer.c:3349) DEBUG1 [9755/9760]: UST consumer checking data pending (in lttng_ustconsumer_data_pending() at ust-consumer.c:1761) DEBUG1 [9755/9760]: UST consumer checking data pending (in lttng_ustconsumer_data_pending() at ust-consumer.c:1761) DEBUG1 [9755/9760]: UST consumer checking data pending (in lttng_ustconsumer_data_pending() at ust-consumer.c:1761) DEBUG1 [9755/9760]: UST consumer metadata pending check: contiguous 2422 vs pushed 2422 (in lttng_ustconsumer_data_pending() at ust-consumer.c:1786) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG1 [9661/9664]: Consumer data is NOT pending for session id 0 (in consumer_is_data_pending() at consumer.c:1085) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Unknown error code) (in thread_manage_clients() at main.c:3803) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Nothing recv() from client... continuing (in thread_manage_clients() at main.c:3758) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9661/9664]: Wait for client response (in thread_manage_clients() at main.c:3709) DEBUG1 [9661/9664]: Receiving data from client ... (in thread_manage_clients() at main.c:3754) DEBUG1 [9661/9664]: Processing client command 9 (in process_client_msg() at main.c:2578) DEBUG1 [9661/9664]: Getting session appses by name (in process_client_msg() at main.c:2655) DEBUG2 [9661/9664]: Trying to find session by name appses (in session_find_by_name() at session.c:133) DEBUG3 [9661/9664]: No kernel session when tearing down session (in kernel_destroy_session() at kernel.c:751) DEBUG1 [9661/9664]: Destroy all UST traces (in ust_app_destroy_trace_all() at ust-app.c:4116) DEBUG2 [9661/9664]: Trace UST destroy session 0 (in trace_ust_destroy_session() at trace-ust.c:676) DEBUG3 [9661/9664]: Buffer registry per UID destroy with id: 0, ABI: 64, uid: 16063 (in buffer_reg_uid_destroy() at buffer-registry.c:624) DEBUG2 [9661/9664]: Consumer close metadata channel key 2 (in consumer_close_metadata() at consumer.c:1137) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: UST consumer close metadata key 2 (in close_metadata() at ust-consumer.c:659) DEBUG1 [9755/9760]: received command on sock (in consumer_thread_sessiond_poll() at consumer.c:2972) DEBUG1 [9755/9758]: Metadata event catched in thread (in consumer_thread_metadata_poll() at consumer.c:2184) DEBUG1 [9755/9758]: Metadata fd 34 is hup|err. (in consumer_thread_metadata_poll() at consumer.c:2257) DEBUG1 [9661/9664]: Destroying session appses (in session_destroy() at session.c:158) DEBUG1 [9755/9758]: In UST read_subbuffer (wait_fd: 34, name: metadata) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9661/9664]: Sending response (size: 16, retcode: Success) (in thread_manage_clients() at main.c:3803) DEBUG1 [9755/9758]: Consumer mmap write() ret 4096 (len 4096) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Accepting client command ... (in thread_manage_clients() at main.c:3667) DEBUG1 [9755/9758]: In UST read_subbuffer (wait_fd: 34, name: metadata) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG3 [9755/9758]: Consumer delete metadata stream 34 (in consumer_del_metadata_stream() at consumer.c:1891) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9755/9758]: Consumer delete channel key 2 (in consumer_del_channel() at consumer.c:291) DEBUG1 [9755/9758]: Destroying metadata cache (in consumer_metadata_cache_destroy() at consumer-metadata-cache.c:180) DEBUG1 [9755/9758]: Metadata poll wait with 1 fd(s) (in consumer_thread_metadata_poll() at consumer.c:2182) DEBUG2 [9661/9824]: Trace destroy UST channel channel0 (in _trace_ust_destroy_channel() at trace-ust.c:573) DEBUG3 [9661/9824]: Buffer registry session destroy (in buffer_reg_session_destroy() at buffer-registry.c:543) DEBUG3 [9661/9824]: Buffer registry channel destroy with key 0 (in buffer_reg_channel_destroy() at buffer-registry.c:498) DEBUG3 [9661/9824]: Buffer registry stream destroy with handle -1 (in buffer_reg_stream_destroy() at buffer-registry.c:446) DEBUG3 [9661/9824]: Buffer registry stream destroy with handle -1 (in buffer_reg_stream_destroy() at buffer-registry.c:446) DEBUG1 [9755/9757]: Channel event catched in thread (in consumer_thread_channel_poll() at consumer.c:2687) DEBUG1 [9755/9757]: Channel fd 17 is hup|err. (in consumer_thread_channel_poll() at consumer.c:2819) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9755/9757]: Channel poll wait with 1 fd(s) (in consumer_thread_channel_poll() at consumer.c:2685) DEBUG1 [9755/9759]: poll num_rdy : 2 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: fd 25 is hup|err|nval. Attempting flush and read. (in consumer_thread_data_poll() at consumer.c:2509) DEBUG1 [9755/9759]: Polling fd 25 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: fd 22 is hup|err|nval. Attempting flush and read. (in consumer_thread_data_poll() at consumer.c:2509) DEBUG1 [9755/9759]: Polling fd 22 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 2 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 4096 (len 4096) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Consumer mmap write() ret 4096 (len 4096) (in lttng_consumer_on_read_subbuffer_mmap() at consumer.c:1445) DEBUG1 [9755/9759]: Polling fd 25 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: Polling fd 22 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: polling on 3 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG1 [9755/9759]: poll num_rdy : 2 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9755/9759]: Normal read on fd 25 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 25, name: channel0_1) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Reserving sub buffer failed (everything is normal, it is due to concurrency) [ret: -11] (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1675) DEBUG1 [9755/9759]: Normal read on fd 22 (in consumer_thread_data_poll() at consumer.c:2486) DEBUG1 [9755/9759]: In UST read_subbuffer (wait_fd: 22, name: channel0_0) (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1617) DEBUG1 [9755/9759]: Reserving sub buffer failed (everything is normal, it is due to concurrency) [ret: -11] (in lttng_ustconsumer_read_subbuffer() at ust-consumer.c:1675) DEBUG1 [9755/9759]: Polling fd 25 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: Consumer stream destroy monitored key: 25 (in destroy_close_stream() at consumer-stream.c:230) DEBUG1 [9755/9759]: Polling fd 22 tells it has hung up. (in consumer_thread_data_poll() at consumer.c:2520) DEBUG1 [9755/9759]: Consumer stream destroy monitored key: 22 (in destroy_close_stream() at consumer-stream.c:230) DEBUG1 [9755/9759]: Consumer delete channel key 1 (in consumer_del_channel() at consumer.c:291) DEBUG1 [9755/9759]: Updating poll fd array (in update_poll_array() at consumer.c:957) DEBUG1 [9755/9759]: polling on 1 fd (in consumer_thread_data_poll() at consumer.c:2402) DEBUG2 [9661/9824]: Trace destroy UST event * (in trace_ust_destroy_event() at trace-ust.c:524) DEBUG3 [9661/9662]: [ht-thread] Polling on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:58) DEBUG1 [9755/9755]: ignoring first SIGINT (in sighandler() at lttng-consumerd.c:81) DEBUG1 [9661/9661]: SIGINT caught (in sighandler() at main.c:4269) DEBUG1 [9661/9661]: Terminating all threads (in stop_threads() at main.c:379) DEBUG1 [9661/9661]: Futex n to 1 wake done (in futex_nto1_wake() at futex.c:104) DEBUG1 [9661/9666]: Notifying applications of session daemon state: 0 (in notify_ust_apps() at main.c:555) DEBUG1 [9661/9666]: Got the wait shm fd 22 (in get_wait_shm() at shm.c:115) DEBUG1 [9661/9666]: Futex wait update active 0 (in futex_wait_update() at futex.c:63) DEBUG1 [9661/9666]: UST Registration thread cleanup complete (in thread_registration_apps() at main.c:1925) DEBUG1 [9661/9664]: Clean command context structure (in clean_command_ctx() at main.c:535) DEBUG1 [9661/9664]: Client thread dying (in thread_manage_clients() at main.c:3851) DEBUG1 [9661/9668]: Application notify communication apps thread cleanup complete (in ust_thread_manage_notify() at ust-thread.c:174) DEBUG1 [9661/9663]: Health check thread dying (in thread_manage_health() at main.c:3594) DEBUG1 [9661/9662]: [ust-thread] cleanup complete. (in thread_ht_cleanup() at ht-cleanup.c:130) DEBUG1 [9755/9760]: Incoming command on sock (in consumer_thread_sessiond_poll() at consumer.c:2954) DEBUG1 [9755/9760]: Consumer received unexpected message size 0 (expects 4472) (in lttng_ustconsumer_recv_cmd() at ust-consumer.c:1089) DEBUG1 [9755/9760]: Communication interrupted on command socket (in consumer_thread_sessiond_poll() at consumer.c:2965) DEBUG1 [9661/9667]: Application communication apps thread cleanup complete (in thread_manage_apps() at main.c:1358) DEBUG1 [9755/9760]: Consumer thread sessiond poll exiting (in consumer_thread_sessiond_poll() at consumer.c:2975) DEBUG1 [9755/9760]: UST consumer closing all metadata streams (in lttng_ustconsumer_close_metadata() at ust-consumer.c:1831) DEBUG1 [9661/9665]: Futex n to 1 wait done (in futex_nto1_wait() at futex.c:90) DEBUG1 [9661/9665]: Dispatch thread dying (in thread_dispatch_ust_registration() at main.c:1709) DEBUG1 [9755/9759]: poll num_rdy : 1 (in consumer_thread_data_poll() at consumer.c:2404) DEBUG1 [9661/9661]: Cleaning up (in cleanup() at main.c:444) hread_data_poll() at consumer.c:2428) DEBUG1 [9755/9759]: Consumer delete flagged data stream (in validate_endpoint_status_data_stream() at consumer.c:2089) DEBUG1 [9755/9759]: polling thread exiting (in consumer_thread_data_poll() at consumer.c:2547) DEBUG1 [9661/9661]: Removing /home/lttng/.lttng directory (in cleanup() at main.c:463) DEBUG1 [9755/9757]: Channel event catched in thread (in consumer_thread_channel_poll() at consumer.c:2687) DEBUG1 [9755/9757]: Channel poll thread exiting (in consumer_thread_channel_poll() at consumer.c:2843) DEBUG1 [9755/9758]: Metadata event catched in thread (in consumer_thread_metadata_poll() at consumer.c:2184) DEBUG1 [9755/9758]: Metadata thread pipe hung up (in consumer_thread_metadata_poll() at consumer.c:2202) DEBUG1 [9755/9758]: Metadata poll thread exiting (in consumer_thread_metadata_poll() at consumer.c:2312) DEBUG1 [9661/9756]: consumer thread cleanup completed (in thread_manage_consumer() at main.c:1204) PERROR [9755/9755]: sendmsg: Broken pipe (in lttcomm_send_unix_sock() at unix.c:218) DEBUG1 [9755/9755]: Consumer destroying it. Closing everything. (in lttng_consumer_destroy() at consumer.c:1245) rm: can't remove '/home/lttng/.lttng': Permission denied DEBUG1 [9661/9661]: Cleaning up all sessions (in cleanup() at main.c:477) DEBUG1 [9661/9661]: Closing all UST sockets (in cleanup() at main.c:490) DEBUG2 [9661/9661]: UST app cleaning registered apps hash table (in ust_app_clean_list() at ust-app.c:3216) DEBUG3 [9661/9661]: Buffer registry destroy all registry (in buffer_reg_destroy_registries() at buffer-registry.c:699) DEBUG1 [9661/9661]: *** assert failed :-) *** ==> Matthew, BEET driven development works! (in cleanup() at main.c:511)