<font size=2 face="sans-serif">Bump...</font>
<br>
<br><font size=2 color=#000080 face="sans-serif">Amit Margalit</font>
<br><font size=2 color=#808000 face="sans-serif">IBM XIV </font><font size=2 face="sans-serif">-
<i>Storage Reinvented</i></font>
<br><font size=2 face="sans-serif">XIV-NAS Development Team</font>
<br><font size=2 face="sans-serif">Tel. 03</font><font size=2 face="Arial">-689-7774</font>
<br><font size=2 face="Arial">Fax. 03-689-7230</font>
<br>
<br>
<br>
<br><font size=1 color=#5f5f5f face="sans-serif">From:
</font><font size=1 face="sans-serif">Amit Margalit/Israel/IBM@IBMIL</font>
<br><font size=1 color=#5f5f5f face="sans-serif">To:
</font><font size=1 face="sans-serif">lttng-dev <lttng-dev@lists.lttng.org></font>
<br><font size=1 color=#5f5f5f face="sans-serif">Date:
</font><font size=1 face="sans-serif">04/02/2014 01:57 PM</font>
<br><font size=1 color=#5f5f5f face="sans-serif">Subject:
</font><font size=1 face="sans-serif">[lttng-dev]
Problem running lttng enable-channel shortly after
lttng create</font>
<br>
<hr noshade>
<br>
<br>
<br><font size=2 face="sans-serif">Hello,</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
I keep getting the following error from lttng enable-channel:</font><font size=3>
<br>
</font><tt><font size=2><br>
libust[2984/2986]: Error: Timed out waiting for lttng-sessiond (in lttng_ust_init()
at lttng-ust-comm.c:1444)</font></tt><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
The scenario is quite simple - during boot of the system, we create a session,
enable a channel, etc., like this:</font><font size=3> <br>
</font><tt><font size=2><br>
# /usr/bin/lttng-sessiond --no-kernel -b</font></tt><font size=3> </font><tt><font size=2><br>
Warning: No tracing group detected</font></tt><font size=3> </font><tt><font size=2><br>
# /usr/bin/lttng create nextra -o /local/traces/lttng/20140402123725</font></tt><font size=3>
</font><tt><font size=2><br>
Session nextra created.</font></tt><font size=3> </font><tt><font size=2><br>
Traces will be written in /local/traces/lttng/20140402123725</font></tt><font size=3>
</font><tt><font size=2><br>
# /usr/bin/lttng enable-channel channel0 -u --tracefile-size 67108864 --tracefile-count
40 --buffers-uid --switch-timer 1000000 --num-subbuf 16 --subbuf-size 262144</font></tt><font size=3>
</font><tt><font size=2><br>
libust[2984/2986]: Error: Timed out waiting for lttng-sessiond (in lttng_ust_init()
at lttng-ust-comm.c:1444)</font></tt><font size=3> </font><tt><font size=2><br>
Warning: No tracing group detected</font></tt><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
The PID </font><tt><font size=2>2984</font></tt><font size=2 face="sans-serif">
belongs to the lttng-consumerd. At that point in time I see the following
in syslog:</font><font size=3> <br>
</font><font size=2 face="sans-serif"><br>
2014/04/02 12:38:03.370576 kernel error nextra-6663235-module-1
[ 56.376155] proc lttng-sessiond with pid 632 goes to delayed
retry in epoll_wait(): start jiffies 4294904844, stop jiffies=4294906390
(+1546), expected final stop jiffies=9223372041149680651 (+9223372036854775807),
next stop jiffies=9223372041149682197 (+9223372036854775807)</font><font size=3>
</font><font size=2 face="sans-serif"><br>
2014/04/02 12:38:03.399245 kernel error nextra-6663235-module-1
[ 56.405166] proc lttng-sessiond with pid 632 goes to delayed
retry in epoll_wait(): start jiffies 4294904844, stop jiffies=4294906397
(+1553), expected final stop jiffies=9223372041149680651 (+9223372036854775807),
next stop jiffies=9223372041149682204 (+9223372036854775807)</font><font size=3>
</font><font size=2 face="sans-serif"><br>
2014/04/02 12:38:03.405700 kernel error nextra-6663235-module-1
[ 56.411791] proc lttng-sessiond with pid 632 goes to delayed
retry in epoll_wait(): start jiffies 4294904844, stop jiffies=4294906398
(+1554), expected final stop jiffies=9223372041149680651 (+9223372036854775807),
next stop jiffies=9223372041149682205 (+9223372036854775807)</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
And finally, the command succeeds:</font><font size=3> <br>
</font><tt><font size=2><br>
UST channel channel0 enabled for session nextra</font></tt><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
My guess is that the boot sequence is loading some traced application before
the channel is enabled, and the consumerd is started, which under heavy
system load (exists during initial boot time) times out waiting for the
sessiond. If that was the case, I could try to increase the timeout time,
but that also delays my booting, which I cannot afford to do.</font><font size=3>
<br>
</font><font size=2 face="sans-serif"><br>
Suggestions?</font><font size=3> <br>
<br>
</font><font size=2 color=#000080 face="sans-serif"><br>
Amit Margalit</font><font size=3> </font><font size=2 color=#808000 face="sans-serif"><br>
IBM XIV </font><font size=2 face="sans-serif">- <i>Storage Reinvented</i></font><font size=3>
</font><font size=2 face="sans-serif"><br>
XIV-NAS Development Team</font><font size=3> </font><font size=2 face="sans-serif"><br>
Tel. 03</font><font size=2 face="Arial">-689-7774</font><font size=3> </font><font size=2 face="Arial"><br>
Fax. 03-689-7230</font><tt><font size=2>_______________________________________________<br>
lttng-dev mailing list<br>
lttng-dev@lists.lttng.org<br>
</font></tt><a href="http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev"><tt><font size=2>http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev</font></tt></a><tt><font size=2><br>
</font></tt>
<br>