[lttng-dev] LTTng unreliable and useless under heavy loaded systems?

[Francis Giraldeau]

I also had issues with too high trace throughput for the disk. Here are few
other suggestions.

- Use SSD drive
- Raise io priority of consumer daemons with ionice
- Enable only events that are absolutely required
- Reduce CPU frequency to reduce event rate
- Reduce the number of active threads

Good luck,

Francis
On 01/16/2015 09:24 PM, Mathieu Desnoyers wrote:

> If you care about not overwriting your buffers under heavy load, I
> recommend to issue "lttng stop; lttng snapshot record" in order to
> stop tracing before gathering the snapshot. This would lessen the
> chances to overwrite your important trace data due to your heavy
> load.
>
Yeah, that perfectly makes sense. I tried this and it indeed helped to not
loose a single trace anymore! (without 'lttng stop' some 'snapshots' still
didn't contain what I was interested in). For the records, this is how I
currently use LTTng:

    lttng create --snapshot victory
    lttng enable-channel channel0 -k --subbuf-size 16M --num-subbuf 2
    lttng add-context -k -t procname -c channel0
    lttng enable-event -k --all -c channel0
    for i in $(ps -mo tid -p `pgrep lttng-consumerd` | grep -v "TID" | grep
-v "-"); do sudo chrt -f -p 50 $i; done
    lttng start

    chrt -f 50 ./run_my_test-application_here # this app (programmed in
C/C++) performs a 'system("lttng stop; lttng snapshot record; lttng
start");' whenever a certain jitter is reached

    lttng stop
    lttng destroy


I guess the line:
    for i in $(ps -mo tid -p `pgrep lttng-consumerd` | grep -v "TID" | grep
-v "-"); do sudo chrt -f -p 50 $i; done
wouldn't be necessary anymore (with 'lttng stop') but I didn't try that.

Cheers,
Raphael

_______________________________________________
lttng-dev mailing list
lttng-dev at lists.lttng.org
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lttng.org/pipermail/lttng-dev/attachments/20150126/3afbe354/attachment.html>