[lttng-dev] 答复: 答复: lttng trace kernel on arm64

Jonathan Rajotte Julien Jonathan.rajotte-julien at efficios.com
Sat Apr 16 15:15:46 UTC 2016



On 2016-04-16 08:27 AM, Lichuanbao wrote:
>
> Hi Jonathan
>
> Glad to see your words again!
>
> >From name hope to see one in flesh some day.
>
> I also hope one day to ask when facing you
>
> You are not interested in me ,right? haha( ‵▽′)ψ
>
> >The following command "lttng-sessiond -vvv" will necessarily fail since the 
> previous command " lttng create mysession" just spawned one.
>
> >Could you try these on a rebooted board as root:
>
> >(Make sure there is no sessiond running)
>
> >lttng-sessiond -vvv > sessiond.log 2>&1
>
> >lttng create
>
> >lttng enable-event -a -k
>
> >lttng start
>
> >sleep 1
>
> >lttng stop
>
> >lttng view
>
> >If on "lttng view" there is no events or any of the commands fail make sure to include 
> their output and the sessiond.log file so we can pinpoint the problem.
>
> Yes,Step by step according to your cmd:
>
> The session.log view attachments.there are some error as flow, I 
> installed lttng-ust already:
>
> lttng-consumerd: error while loading shared libraries: 
> liblttng-ust-ctl.so.2: cannot open shared object file: No such file or 
> directory
>
> Error: Condition timed out. The consumer thread was never ready. 
> Killing it
>
> Error: Fatal error spawning consumer control thread
>
Could you do a search for this file and make sure it exist ?

Also it might be a good idea to uninstall everything (lttng-ust, 
lttng-tools, lttng-modules) make sure that you install the same version 
(stable 2.8 from git if you can) for all three.

I also requested access board on http://open-estuary.org/ to test it myself.

Cheers

> *shell log:*
>
> -bash-4.3# lttng -v create
>
> DEBUG1 - 00:03:59.845027 [2462/2462]: Auto session name set to 
> auto-19700101-000359 (in create_session() at commands/create.c:249)
>
> DEBUG3 - 00:03:59.845136 [2462/2462]: URI string: 
> file:///root/lttng-traces/auto-19700101-000359 (in uri_parse() at 
> uri.c:292)
>
> DEBUG3 - 00:03:59.845198 [2462/2462]: URI file destination: 
> /root/lttng-traces/auto-19700101-000359 (in uri_parse() at uri.c:329)
>
> DEBUG3 - 00:03:59.845223 [2462/2462]: URI dtype: 3, proto: 0, host: , 
> subdir: , ctrl: 0, data: 0 (in uri_parse() at uri.c:506)
>
> DEBUG1 - 00:03:59.845300 [2462/2462]: LSM cmd type : 8 (in 
> send_session_msg() at lttng-ctl.c:135)
>
> Session auto-19700101-000359 created.
>
> Traces will be written in /root/lttng-traces/auto-19700101-000359
>
> DEBUG1 - 00:03:59.846076 [2462/2462]: Init config session in /root (in 
> config_init() at conf.c:323)
>
> -bash-4.3# lttng -v enable-event -a -k
>
> DEBUG1 - 00:04:19.762761 [2463/2463]: Session name found: 
> auto-19700101-000359 (in _get_session_name() at utils.c:64)
>
> DEBUG1 - 00:04:19.763042 [2463/2463]: LSM cmd type : 6 (in 
> send_session_msg() at lttng-ctl.c:135)
>
>                                                      /////////////////this 
> step wait for many seconds ,but failed last
>
> Error: Events: Kernel consumer start failed (channel channel0, session 
> auto-19700101-000359)
>
> Error: Command error
>
> DEBUG1 - 00:04:49.768267 [2463/2463]: Clean exit (in clean_exit() at 
> lttng.c:149)
>
> *发件人:*Jonathan Rajotte [mailto:jonathan.r.julien at gmail.com]
> *发送时间:*2016年4月15日3:30
> *收件人:*Lichuanbao
> *抄送:*Jonathan Rajotte Julien; lttng-dev at lists.lttng.org; Zhanweitao
> *主题:*Re: [lttng-dev] 答复: lttng trace kernel on arm64
>
> Hi Lichuanbao
>
> On Tue, Apr 12, 2016 at 4:30 AM, Lichuanbao <lichuanbao at hisilicon.com 
> <mailto:lichuanbao at hisilicon.com>> wrote:
>
> Hi Jonathan ,Thank you for your kind help
>
> >You got me curious: D02 ? D03 hisilicon board ? :)
> Ha. You know it? It's D02.
>
> From name hope to see one in flesh some day.
>
>
>     >Does "lttng list -k" return a list of events ?
>     Yes.it works,list many event name
>
> Then you have a live lttng-sessiond.
>
>
>     >This line suggest that another instance was running.
>     >Make sure to kill all lttng-sessiond instance before starting one to
>     >make sure you have the actual log of the used sessiond.
>
>
>     Yes ,I tried,reboot the board,and "ps -A|grep lttng" ,here is no
>     lttng session.
>     after " lttng create mysession ",there is an error " Error: Could
>     not get lock file /var/run/lttng/lttng-sessiond.lck, another
>     instance is running."
>     I used "2.8.0-rc1" version this time. Same problem.
>
>     Logs:
>
>     -bash-4.3# ps -A|grep lttng
>     -bash-4.3# lttng create mysession
>     Spawning a session daemon
>     Session mysession created.
>     Traces will be written in /root/lttng-traces/mysession-19700101-015353
>
> The following command "lttng-sessiond -vvv" will necessarily fail 
> since the previous command " lttng create mysession" just spawned one.
>
> Could you try these on a rebooted board as root:
>
> (Make sure there is no sessiond running)
>
> lttng-sessiond -vvv > sessiond.log 2>&1
>
> lttng create
>
> lttng enable-event -a -k
>
> lttng start
>
> sleep 1
>
> lttng stop
>
> lttng view
>
> If on "lttng view" there is no events or any of the commands fail make 
> sure to include their output and the sessiond.log file so we can 
> pinpoint the problem.
>
>     -bash-4.3# lttng-sessiond -vvv
>     DEBUG1 - 01:54:06.834954 [2687/2687]: Resetting run_as worker
>     signal handlers to default (in reset_sighandler() at runas.c:536)
>     DEBUG1 - 01:54:06.835249 [2687/2687]: run_as signal handler set
>     for SIGTERM and SIGINT (in set_worker_sighandlers() at runas.c:593)
>     DEBUG1 - 01:54:06.835345 [2686/2686]: Health check time delta in
>     seconds set to 20 (in health_init() at health.c:73)
>     DEBUG1 - 01:54:06.835614 [2686/2686]: epoll set max size is
>     27037798 (in compat_epoll_set_max_size() at compat-epoll.c:288)
>     DEBUG3 - 01:54:06.835798 [2686/2686]: Creating LTTng run
>     directory: /var/run/lttng (in create_lttng_rundir() at main.c:5246)
>     DEBUG1 - 01:54:06.835812 [2686/2688]: [ht-thread] startup. (in
>     thread_ht_cleanup() at ht-cleanup.c:36)
>     DEBUG3 - 01:54:06.835979 [2686/2688]: [ht-thread] Polling. (in
>     thread_ht_cleanup() at ht-cleanup.c:66)
>     DEBUG2 - 01:54:06.835996 [2686/2686]: Kernel consumer err path:
>     /var/run/lttng/kconsumerd/error (in main() at main.c:5688)
>     DEBUG2 - 01:54:06.836145 [2686/2686]: Kernel consumer cmd path:
>     /var/run/lttng/kconsumerd/command (in main() at main.c:5690)
>     PERROR - 01:54:06.836251 [2686/2686]: fcntl lock file: Resource
>     temporarily unavailable (in utils_create_lock_file() at utils.c:541)
>     Error: Could not get lock file /var/run/lttng/lttng-sessiond.lck,
>     another instance is running.
>     DEBUG1 - 01:54:06.836379 [2686/2686]: Cleanup sessiond (in
>     sessiond_cleanup() at main.c:624)
>     DEBUG1 - 01:54:06.836456 [2686/2686]: Removing sessiond and
>     consumerd content of directory /var/run/lttng (in
>     sessiond_cleanup() at main.c:643)
>     DEBUG1 - 01:54:06.836514 [2686/2686]: Removing
>     /var/run/lttng/lttng-sessiond.pid (in sessiond_cleanup() at
>     main.c:649)
>     DEBUG1 - 01:54:06.836609 [2686/2686]: Removing
>     /var/run/lttng/agent.port (in sessiond_cleanup() at main.c:654)
>     DEBUG1 - 01:54:06.836719 [2686/2686]: Removing
>     /var/run/lttng/kconsumerd/error (in sessiond_cleanup() at main.c:661)
>     DEBUG1 - 01:54:06.836805 [2686/2686]: Removing directory
>     /var/run/lttng/kconsumerd (in sessiond_cleanup() at main.c:667)
>     DEBUG1 - 01:54:06.836893 [2686/2686]: Removing
>     /var/run/lttng/ustconsumerd32/error (in sessiond_cleanup() at
>     main.c:674)
>     DEBUG1 - 01:54:06.836991 [2686/2686]: Removing directory
>     /var/run/lttng/ustconsumerd32 (in sessiond_cleanup() at main.c:680)
>     DEBUG1 - 01:54:06.837075 [2686/2686]: Removing
>     /var/run/lttng/ustconsumerd64/error (in sessiond_cleanup() at
>     main.c:687)
>     DEBUG1 - 01:54:06.837169 [2686/2686]: Removing directory
>     /var/run/lttng/ustconsumerd64 (in sessiond_cleanup() at main.c:693)
>     DEBUG1 - 01:54:06.837253 [2686/2686]: Cleaning up all sessions (in
>     sessiond_cleanup() at main.c:696)
>     DEBUG1 - 01:54:06.837331 [2686/2686]: Cleaning up all agent apps
>     (in sessiond_cleanup() at main.c:713)
>     DEBUG1 - 01:54:06.837397 [2686/2686]: Closing all UST sockets (in
>     sessiond_cleanup() at main.c:716)
>     DEBUG2 - 01:54:06.837488 [2686/2686]: UST app cleaning registered
>     apps hash table (in ust_app_clean_list() at ust-app.c:3818)
>     DEBUG3 - 01:54:06.846332 [2686/2686]: Buffer registry destroy all
>     registry (in buffer_reg_destroy_registries() at buffer-registry.c:753)
>     DEBUG2 - 01:54:06.846414 [2686/2686]: Closing kernel fd (in
>     sessiond_cleanup() at main.c:721)
>     DEBUG1 - 01:54:06.846481 [2686/2686]: Unloading kernel modules (in
>     sessiond_cleanup() at main.c:728)
>     DEBUG1 - 01:54:06.892609 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-metadata-mmap-client (in modprobe_remove_lttng()
>     at modprobe.c:371)
>     DEBUG1 - 01:54:06.940557 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-client-mmap-overwrite (in
>     modprobe_remove_lttng() at modprobe.c:371)
>     DEBUG1 - 01:54:06.996528 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-client-mmap-discard (in modprobe_remove_lttng()
>     at modprobe.c:371)
>     DEBUG1 - 01:54:07.405263 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-metadata-client (in modprobe_remove_lttng() at
>     modprobe.c:371)
>     DEBUG1 - 01:54:07.805463 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-client-overwrite (in modprobe_remove_lttng() at
>     modprobe.c:371)
>     DEBUG1 - 01:54:07.124584 [2686/2686]: Modprobe removal successful
>     lttng-ring-buffer-client-discard (in modprobe_remove_lttng() at
>     modprobe.c:371)
>     DEBUG3 - 01:54:07.124716 [2686/2688]: [ht-thread] Returning from
>     poll on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:73)
>     DEBUG1 - 01:54:07.124831 [2686/2688]: [ht-cleanup] quit. (in
>     thread_ht_cleanup() at ht-cleanup.c:152)
>     DEBUG1 - 01:54:07.124916 [2686/2688]: [ht-cleanup] Thread
>     terminates. (in thread_ht_cleanup() at ht-cleanup.c:163)
>     DEBUG1 - 01:54:07.125084 [2686/2686]: Cleaning up options (in
>     sessiond_cleanup_options() at main.c:775)
>     DEBUG1 - 01:54:07.125161 [2686/2686]: Destroying run_as worker (in
>     run_as_destroy_worker() at runas.c:705)
>     DEBUG1 - 01:54:07.125236 [2686/2686]: Closing run_as worker socket
>     (in run_as_destroy_worker() at runas.c:711)
>     DEBUG1 - 01:54:07.125343 [2687/2687]: run_as worker exiting (ret =
>     0) (in run_as_create_worker() at runas.c:655)
>     DEBUG1 - 01:54:07.126301 [2686/2686]: lttng-runas terminated with
>     status code 0 (in run_as_destroy_worker() at runas.c:733)
>     DEBUG1 - 01:54:07.126332 [2686/2686]: *** assert failed :-) ***
>     ==> Matthew, BEET driven development works! (in
>     sessiond_cleanup_options() at main.c:807)
>     -bash-4.3# ps -A|grep lttng
>      2582 ?        00:00:00 lttng-sessiond
>      2583 ?        00:00:00 lttng-runas
>     -bash-4.3# lttng list -k
>     Kernel events:
>     -------------
>           lttng_logger (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_touch_buffer (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           block_dirty_buffer (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           block_rq_abort (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_rq_requeue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_rq_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_rq_insert (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_rq_issue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_bio_bounce (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_bio_complete (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           block_bio_backmerge (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           block_bio_frontmerge (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           block_bio_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_getrq (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_sleeprq (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_plug (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_unplug (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_split (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_bio_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           block_rq_remap (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_transaction_commit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_inode_new (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_inode_request (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_inode_evict (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_get_extent (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_ordered_extent_add (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_ordered_extent_remove (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_ordered_extent_start (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_ordered_extent_put (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs__extent_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_writepage_end_io_hook (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_sync_file (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_sync_fs (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_delayed_tree_ref (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_delayed_data_ref (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_delayed_ref_head (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_chunk_alloc (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_chunk_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_cow_block (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           btrfs_space_reservation (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_reserved_extent_alloc (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_reserved_extent_free (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_find_free_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_reserve_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_reserve_extent_cluster (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_find_cluster (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_failed_cluster_setup (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           btrfs_setup_cluster (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_alloc_extent_state (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           btrfs_free_extent_state (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           compaction_isolate_migratepages (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           compaction_isolate_freepages (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           compaction_migratepages (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_free_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_request_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_allocate_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_evict_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_drop_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_mark_inode_dirty (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_write_begin (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_ordered_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_writeback_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_journalled_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_ordered_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_writeback_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_journalled_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_readpage (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_releasepage (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_invalidatepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_discard_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_request_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_allocate_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_free_blocks (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_sync_file_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_sync_file_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_sync_fs (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_rsv_window_add (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_discard_reservation (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_alloc_new_reservation (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext3_reserved (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_forget (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_read_block_bitmap (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_direct_IO_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_direct_IO_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_unlink_enter (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_unlink_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext3_truncate_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_truncate_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_get_blocks_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_get_blocks_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext3_load_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_free_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_request_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_allocate_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_evict_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_drop_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_mark_inode_dirty (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_begin_ordered_truncate (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_write_begin (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_da_write_begin (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ordered_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_writeback_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_journalled_write_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_da_write_end (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_da_writepages (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_da_write_pages (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_da_write_pages_extent (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_da_writepages_result (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_writepage (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_readpage (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_releasepage (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_invalidatepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_journalled_invalidatepage (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_discard_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_new_inode_pa (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_new_group_pa (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_release_inode_pa (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_release_group_pa (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_discard_preallocations (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_mb_discard_preallocations (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_request_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_allocate_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_free_blocks (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_sync_file_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_sync_file_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_sync_fs (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_alloc_da_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mballoc_alloc (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mballoc_prealloc (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mballoc_discard (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mballoc_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_forget (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_da_update_reserve_space (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_da_reserve_space (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_da_release_space (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_bitmap_load (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_mb_buddy_bitmap_load (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_read_block_bitmap_load (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_load_inode_bitmap (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_direct_IO_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_direct_IO_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_fallocate_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_fallocate_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_punch_hole (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_zero_range (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_unlink_enter (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_unlink_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_truncate_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_truncate_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_convert_to_initialized_enter (loglevel: TRACE_EMERG
>     (0)) (type: tracepoint)
>           ext4_ext_convert_to_initialized_fastpath (loglevel:
>     TRACE_EMERG (0)) (type: tracepoint)
>           ext4_ext_map_blocks_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ind_map_blocks_enter (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_map_blocks_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ind_map_blocks_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_load_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_load_inode (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_journal_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_journal_start_reserved (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_ext_handle_uninitialized_extents (loglevel: TRACE_EMERG
>     (0)) (type: tracepoint)
>           ext4_get_implied_cluster_alloc_exit (loglevel: TRACE_EMERG
>     (0)) (type: tracepoint)
>           ext4_ext_put_in_cache (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_in_cache (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_find_delalloc_range (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_get_reserved_cluster_alloc (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_ext_show_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_remove_blocks (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_rm_leaf (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_ext_rm_idx (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           ext4_ext_remove_space (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_ext_remove_space_done (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           ext4_es_insert_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           ext4_es_cache_extent (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           gpio_direction (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           gpio_value (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           irq_handler_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           irq_handler_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           irq_softirq_entry (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           irq_softirq_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           irq_softirq_raise (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd_checkpoint (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd_start_commit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd_commit_locking (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_commit_flushing (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_commit_logging (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_drop_transaction (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_end_commit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd_do_submit_data (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_cleanup_journal_tail (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd_journal_write_superblock (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           jbd2_checkpoint (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd2_start_commit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd2_commit_locking (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_commit_flushing (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_commit_logging (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_drop_transaction (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_end_commit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd2_submit_inode_data (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_run_stats (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           jbd2_checkpoint_stats (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_update_log_tail (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           jbd2_write_superblock (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kmem_kmalloc (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_cache_alloc (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_kmalloc_node (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_cache_alloc_node (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kmem_kfree (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_cache_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_mm_page_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kmem_mm_page_free_batched (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kmem_mm_page_alloc (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kmem_mm_page_alloc_zone_locked (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           kmem_mm_page_pcpu_drain (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kmem_mm_page_alloc_extfrag (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           kvm_userspace_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           kvm_set_irq (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kvm_mmio (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kvm_fpu (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           kvm_age_page (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           module_load (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           module_free (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           module_get (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           module_put (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           module_request (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           napi_poll (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           net_dev_xmit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           net_dev_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           net_if_receive_skb (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           net_if_rx (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           power_cpu_idle (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           power_cpu_frequency (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           power_machine_suspend (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           power_wakeup_source_activate (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           power_wakeup_source_deactivate (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           power_clock_enable (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           power_clock_disable (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           power_clock_set_rate (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           power_domain_target (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           printk_console (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           random_mix_pool_bytes (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           random_mix_pool_bytes_nolock (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           random_credit_entropy_bits (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           random_get_random_bytes (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           random_extract_entropy (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           random_extract_entropy_user (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           rcu_utilization (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regmap_reg_write (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regmap_reg_read (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regmap_reg_read_cache (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_hw_read_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_hw_read_done (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_hw_write_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_hw_write_done (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_regcache_sync (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regmap_cache_only (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regmap_cache_bypass (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regulator_enable (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regulator_enable_delay (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regulator_enable_complete (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regulator_disable (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           regulator_disable_complete (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           regulator_set_voltage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           regulator_set_voltage_complete (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           sched_kthread_stop (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_kthread_stop_ret (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_wakeup_new (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_switch (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_migrate_task (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_process_free (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_process_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_wait_task (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_process_wait (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_process_fork (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_process_exec (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_stat_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_stat_sleep (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_stat_iowait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           sched_stat_blocked (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_stat_runtime (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sched_pi_setprio (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           scsi_dispatch_cmd_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           scsi_dispatch_cmd_error (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           scsi_dispatch_cmd_done (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           scsi_dispatch_cmd_timeout (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           scsi_eh_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           signal_generate (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           signal_deliver (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           skb_kfree (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           skb_consume (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           skb_copy_datagram_iovec (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sock_rcvqueue_full (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           sock_exceed_buf_limit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           lttng_statedump_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           lttng_statedump_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           lttng_statedump_process_state (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           lttng_statedump_file_descriptor (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           lttng_statedump_vm_map (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           lttng_statedump_network_interface (loglevel: TRACE_EMERG
>     (0)) (type: tracepoint)
>           lttng_statedump_block_device (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           lttng_statedump_interrupt (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           rpc_call_status (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           rpc_bind_status (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           rpc_connect_status (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           rpc_task_begin (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           rpc_task_run_action (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           rpc_task_complete (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           rpc_task_sleep (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           rpc_task_wakeup (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           timer_init (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           timer_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           timer_expire_entry (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_expire_exit (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           timer_cancel (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           timer_hrtimer_init (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_hrtimer_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_hrtimer_expire_entry (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           timer_hrtimer_expire_exit (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_hrtimer_cancel (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_itimer_state (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           timer_itimer_expire (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           udp_fail_queue_rcv_skb (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_kswapd_sleep (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_kswapd_wake (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_wakeup_kswapd (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_direct_reclaim_begin (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           mm_vmscan_memcg_reclaim_begin (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           mm_vmscan_memcg_softlimit_reclaim_begin (loglevel:
>     TRACE_EMERG (0)) (type: tracepoint)
>           mm_vmscan_direct_reclaim_end (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           mm_vmscan_memcg_reclaim_end (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           mm_vmscan_memcg_softlimit_reclaim_end (loglevel: TRACE_EMERG
>     (0)) (type: tracepoint)
>           mm_vmscan_shrink_slab_start (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           mm_vmscan_shrink_slab_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_lru_isolate (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_memcg_isolate (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           mm_vmscan_lru_shrink_inactive (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           workqueue_queue_work (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           workqueue_activate_work (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           workqueue_execute_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           workqueue_execute_end (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_dirty_page (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_dirty_inode_start (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_dirty_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_mark_inode_dirty (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_write_inode_start (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_write_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_nothread (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_queue (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_exec (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_start (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_written (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_wait (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_pages_written (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_bdi_register (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_nowork (loglevel: TRACE_EMERG (0)) (type: tracepoint)
>           writeback_wake_background (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_wake_thread (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_wake_forker_thread (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_bdi_unregister (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_thread_start (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_thread_stop (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_wbc_writepage (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_queue_io (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_global_dirty_state (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_bdi_dirty_ratelimit (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_balance_dirty_pages (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_sb_inodes_requeue (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_congestion_wait (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>           writeback_wait_iff_congested (loglevel: TRACE_EMERG (0))
>     (type: tracepoint)
>           writeback_single_inode (loglevel: TRACE_EMERG (0)) (type:
>     tracepoint)
>
>
>
>     -----邮件原件-----
>     发件人: lttng-dev [mailto:lttng-dev-bounces at lists.lttng.org
>     <mailto:lttng-dev-bounces at lists.lttng.org>] 代表Jonathan Rajotte
>     Julien
>     发送时间: 2016年4月11日22:14
>     收件人: lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
>     主题: Re: [lttng-dev] lttng trace kernel on arm64
>
>
>     Hi,
>
>     On 2016-04-08 02:28 AM, Lichuanbao wrote:
>     >
>     > Hi,
>     >
>     >    I want to use lttng trace kernel event,I installed lttng-tools
>     > 2.7.1 lttng-ust 2.7.2 lttng-modules 2.7.2 on my arm64 board,
>     >
>
>     You got me curious: D02 ? D03 hisilicon board ? :)
>
>     > And then I create my session like sub command as root,but there is
>     > nothing result to view.
>     >
>     >     # lttng create
>     >
>     >     # lttng enable-event --kernel --all
>     >
>     >     # lttng start
>     >
>     >     # sleep 10       # let the system generate some activity
>     >
>     >     # lttng stop
>     >
>     >     # lttng view
>     >
>     > # lttng destroy
>     >
>
>     Does "lttng list -k" return a list of events ?
>
>     > -bash-4.3# lttng view
>     >
>     > Trace directory: /root/lttng-traces/auto-20160407-211248
>     >
>     > [error] Cannot open any trace for reading.
>     >
>     > [error] opening trace "/root/lttng-traces/auto-20160407-211248" for
>     > reading.
>     >
>     > [error] none of the specified trace paths could be opened.
>     >
>     > Get thes session daemon log as follow:
>     >
>     > bash-4.3# lttng-sessiond -vvv
>     >
>     > DEBUG1 - 21:52:15.396991 [5621/5621]: run_as signal handler set for
>     > SIGTERM and SIGINT (in set_worker_sighandlers() at runas.c:600)
>     >
>     > DEBUG1 - 21:52:15.397410 [5620/5620]: Health check time delta in
>     > seconds set to 20 (in health_init() at health.c:74)
>     >
>     > DEBUG1 - 21:52:15.397661 [5620/5620]: epoll set max size is 27037798
>     > (in compat_epoll_set_max_size() at compat-epoll.c:290)
>     >
>     > DEBUG3 - 21:52:15.397838 [5620/5620]: Creating LTTng run directory:
>     > /var/run/lttng (in create_lttng_rundir() at main.c:5156)
>     >
>     > DEBUG1 - 21:52:15.397852 [5620/5622]: [ht-thread] startup. (in
>     > thread_ht_cleanup() at ht-cleanup.c:37)
>     >
>     > DEBUG2 - 21:52:15.397986 [5620/5620]: Kernel consumer err path:
>     > /var/run/lttng/kconsumerd/error (in main() at main.c:5601)
>     >
>     > DEBUG2 - 21:52:15.398070 [5620/5620]: Kernel consumer cmd path:
>     > /var/run/lttng/kconsumerd/command (in main() at main.c:5603)
>     >
>     > DEBUG3 - 21:52:15.398034 [5620/5622]: [ht-thread] Polling. (in
>     > thread_ht_cleanup() at ht-cleanup.c:67)
>     >
>     > Error: Could not get lock file /var/run/lttng/lttng-sessiond.lck,
>     > another instance is running.
>     >
>
>     This line suggest that another instance was running.
>
>     Make sure to kill all lttng-sessiond instance before starting one to
>     make sure you have the actual log of the used sessiond.
>
>     Cheers
>
>
>     > DEBUG1 - 21:52:15.398240 [5620/5620]: Cleanup sessiond (in
>     > sessiond_cleanup() at main.c:622)
>     >
>     > DEBUG1 - 21:52:15.398294 [5620/5620]: Removing sessiond and
>     consumerd
>     > content of directory /var/run/lttng (in sessiond_cleanup() at
>     main.c:641)
>     >
>     > DEBUG1 - 21:52:15.398321 [5620/5620]: Removing
>     > /var/run/lttng/lttng-sessiond.pid (in sessiond_cleanup() at
>     main.c:647)
>     >
>     > DEBUG1 - 21:52:15.398372 [5620/5620]: Removing
>     > /var/run/lttng/agent.port (in sessiond_cleanup() at main.c:652)
>     >
>     > DEBUG1 - 21:52:15.398420 [5620/5620]: Removing
>     > /var/run/lttng/kconsumerd/error (in sessiond_cleanup() at
>     main.c:659)
>     >
>     > DEBUG1 - 21:52:15.398472 [5620/5620]: Removing directory
>     > /var/run/lttng/kconsumerd (in sessiond_cleanup() at main.c:665)
>     >
>     > DEBUG1 - 21:52:15.398526 [5620/5620]: Removing
>     > /var/run/lttng/ustconsumerd32/error (in sessiond_cleanup() at
>     main.c:672)
>     >
>     > DEBUG1 - 21:52:15.398579 [5620/5620]: Removing directory
>     > /var/run/lttng/ustconsumerd32 (in sessiond_cleanup() at main.c:678)
>     >
>     > DEBUG1 - 21:52:15.398620 [5620/5620]: Removing
>     > /var/run/lttng/ustconsumerd64/error (in sessiond_cleanup() at
>     main.c:685)
>     >
>     > DEBUG1 - 21:52:15.398673 [5620/5620]: Removing directory
>     > /var/run/lttng/ustconsumerd64 (in sessiond_cleanup() at main.c:691)
>     >
>     > DEBUG1 - 21:52:15.398734 [5620/5620]: Cleaning up all sessions (in
>     > sessiond_cleanup() at main.c:694)
>     >
>     > DEBUG1 - 21:52:15.398765 [5620/5620]: Cleaning up all agent apps (in
>     > sessiond_cleanup() at main.c:711)
>     >
>     > DEBUG1 - 21:52:15.398789 [5620/5620]: Closing all UST sockets (in
>     > sessiond_cleanup() at main.c:714)
>     >
>     > DEBUG2 - 21:52:15.398813 [5620/5620]: UST app cleaning
>     registered apps
>     > hash table (in ust_app_clean_list() at ust-app.c:3702)
>     >
>     > DEBUG3 - 21:52:15.398837 [5620/5620]: Buffer registry destroy all
>     > registry (in buffer_reg_destroy_registries() at
>     buffer-registry.c:716)
>     >
>     > DEBUG2 - 21:52:15.398861 [5620/5620]: Closing kernel fd (in
>     > sessiond_cleanup() at main.c:719)
>     >
>     > DEBUG1 - 21:52:15.398884 [5620/5620]: Unloading kernel modules (in
>     > sessiond_cleanup() at main.c:726)
>     >
>     > DEBUG1 - 21:52:15.403580 [5620/5620]: Modprobe removal successful
>     > lttng-kretprobes (in modprobe_remove_lttng() at modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.407855 [5620/5620]: Modprobe removal successful
>     > lttng-kprobes (in modprobe_remove_lttng() at modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.412073 [5620/5620]: Modprobe removal successful
>     > lttng-ftrace (in modprobe_remove_lttng() at modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.416256 [5620/5620]: Modprobe removal successful
>     > lttng-types (in modprobe_remove_lttng() at modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.420593 [5620/5620]: Modprobe removal successful
>     > lttng-ring-buffer-metadata-mmap-client (in
>     modprobe_remove_lttng() at
>     > modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.425108 [5620/5620]: Modprobe removal successful
>     > lttng-ring-buffer-client-mmap-overwrite (in
>     modprobe_remove_lttng() at
>     > modprobe.c:126)
>     >
>     > DEBUG1 - 21:52:15.429745 [5620/5620]: Modprobe removal successful
>     > lttng-ring-buffer-client-mmap-discard (in modprobe_remove_lttng() at
>     > modprobe.c:126)
>     >
>     > Error: Unable to remove module lttng-ring-buffer-metadata-client
>     >
>     > DEBUG1 - 21:52:15.438779 [5620/5620]: Modprobe removal successful
>     > lttng-ring-buffer-client-overwrite (in modprobe_remove_lttng() at
>     > modprobe.c:126)
>     >
>     > Error: Unable to remove module lttng-ring-buffer-client-discard
>     >
>     > Error: Unable to remove module lttng-lib-ring-buffer
>     >
>     > Error: Unable to remove module lttng-tracer
>     >
>     > DEBUG3 - 21:52:15.452135 [5620/5622]: [ht-thread] Returning from
>     poll
>     > on 2 fds. (in thread_ht_cleanup() at ht-cleanup.c:74)
>     >
>     > DEBUG1 - 21:52:15.452186 [5620/5622]: [ht-cleanup] quit. (in
>     > thread_ht_cleanup() at ht-cleanup.c:153)
>     >
>     > DEBUG1 - 21:52:15.452225 [5620/5622]: [ht-cleanup] Thread
>     terminates.
>     > (in thread_ht_cleanup() at ht-cleanup.c:164)
>     >
>     > DEBUG1 - 21:52:15.452333 [5620/5620]: Cleaning up options (in
>     > sessiond_cleanup_options() at main.c:773)
>     >
>     > DEBUG1 - 21:52:15.453308 [5620/5620]: lttng-runas terminated with
>     > status code 0 (in run_as_destroy_worker() at runas.c:737)
>     >
>     > DEBUG1 - 21:52:15.453349 [5620/5620]: *** assert failed :-) *** ==>
>     > Matthew, BEET driven development works! (in
>     sessiond_cleanup_options()
>     > at main.c:805)
>     >
>     > can anybody help me ?
>     >
>     > thanks very much.
>     >
>     > lichuanbao
>     >
>     > ShenZhen in China
>     >
>     >
>     >
>     > _______________________________________________
>     > lttng-dev mailing list
>     > lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
>     > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>     --
>     Jonathan R. Julien
>     Efficios
>
>     _______________________________________________
>     lttng-dev mailing list
>     lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
>     https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>     _______________________________________________
>     lttng-dev mailing list
>     lttng-dev at lists.lttng.org <mailto:lttng-dev at lists.lttng.org>
>     https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
>
>
> -- 
>
> Jonathan Rajotte Julien
>

-- 
Jonathan R. Julien
Efficios

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.lttng.org/pipermail/lttng-dev/attachments/20160416/1cab0550/attachment-0001.html>


More information about the lttng-dev mailing list