[ltt-dev] [RFC UST] Processes model
Mathieu Desnoyers
mathieu.desnoyers at efficios.com
Mon Jan 17 17:15:01 EST 2011
* David Goulet (david.goulet at polymtl.ca) wrote:
> Hi everyone,
>
> I'm submitting a RFC about a "new daemon model" for UST.
>
> This is the *first* draft... so a lot of feedback will be appreciated
> for improvements and ideas!
>
> English might be bad sometimes, please feel free to submit correction
> about that!
>
Hi David,
Thanks for putting our thoughts into a clear document. Some comments below,
> Thanks to all
> --
> David Goulet
> LTTng project, DORSAL Lab.
>
> PGP/GPG : 1024D/16BD8563
> BE3C 672B 9331 9796 291A 14C6 4AF7 C14B 16BD 8563
> RFC - New processes model for UST
>
> Author: David Goulet <david.goulet at polymtl.ca>
>
> Acknowledgement for helping creating that model:
> - Mathieu Desnoyers
> - Yannick Brosseau
>
> Version:
> - v0.1: 17/01/2011
> Initial proposal
>
> Terminology
> -----------------
>
> ustd - Main daemon that act as a trace session registry
>
> ust-consumerd - Daemon that reads UST buffers for a speficic application
specific
>
> tracing session - A trace linked to set of specific tracepoints
to a set of
>
> tracing buffers - Buffers containing tracing data
>
> tracing data - Data created by tracing an application
>
> inprocess library - UST library linked with the application
>
> Model
> -----------------
>
> This RFC propose a complete new UST daemon model. This re-engineering was
proposes a brand new
> mostly driven by the needs of better security in terms of access rights,
the need for
> tracing session and networking such as streaming and remote control over
> different traces.
>
> The new model follows the basic principles of having a trace registry (ustd),
> consumers for each tracing buffers (ust-consumerd) and having these buffers
> shared between the application and one consumer (shared memory).
(through system V shared memory)
>
> From now on, each component mention above is presented with their associated
mentioned
> roles and existence meaning.
"existence meaning" -> unclear
>
> USTD:
>
> The ustd daemon act as a trace registry i.e by keeping reference to all active
acts
i.e.
> session and, by active, it means a session in any state other then destroyed.
than
> Each entity we are keeping track of, here traces, will have a unique identifier
> (ID) assign to it.
>
> An ID SHOULD be a unique hash of the session name, trace path name,
> date/time, PID and/or UID
>
> The trace roles of ustd:
>
> Trace interaction - Create, Destroy, Pause, Stop, Start, Set options
>
> Registry - keep track of all tracing session information (basics):
basics ?
> * shared memory location
> * UID and GID
> * application PID
> * trace ID (unique identifier)
> * session name
>
> Basic access control - allow or deny trace interaction based on the user
> UID/GID.
How ?
>
> Consumer creation - spawns ust-consumerd.
No ? Isn't ust-consumerd spawned separately, and then connects to ustd ?
>
> Buffers creation - creates shared memory for the tracing buffers.
Yes.
>
> In order for ustd to be stateless, it should save all the registry information
> to disk. If ustd is killed or crashes, it will be able to restart and get the
> tracing state back without loosing track of all sessions.
I don't think it is needed. ustd runs as root, and if we have a bug in the
tracer, well, we have a bug in the tracer, that's it.
The in-kernel LTTng does not save state to disk. If the tracer crashes in the
kernel, we try not to kill the whole box, but the user cannot expect to be able
to trace until the box reboots.
>
> UST-CONSUMERD:
>
> This daemon basically consume the tracing buffers and write that data to disk
> for future analysis using LTTv or/and TMF (Tracing Monitoring Frameworks). Upon
> creation, that daemon UID is set to the application user.
The user creates the daemon (not ustd).
>
> SHARED MEMORY:
>
> This is the memory area where the tracing buffers of the application will be
> held and given access in write mode for the inprocess library of the
> application and in readonly mode for the ust-consumerd.
>
> This memory is ONLY used for the tracing buffers. No communication between
> components is done by that memory.
>
> PREREQUISITES:
>
> The ustd daemon MUST always be running as "root" or an equivalent user having
> the same privilege as root.
>
> This daemon MUST be up and running at all time in order to trace a tracable
> application. (Future works will propose multiple registry coexistence).
>
> A "tracing" group SHOULD be created. Whoever is in that group is ABLE to access
> the tracing data of ANY buffers and is ABLE to operate a tracing session for
operate -> control (operate is not really defined)
> ANY application.
>
> WARNING: This group name might interfer with other linux apps using the same
interfere
> group name. Carefull care should be put at install time for that.
-Carefull
>
> The new "lttngtrace" command line tool MUST be use to interact with the ustd
> registry daemon for every trace action needed by the user.
lttngtrace is a bit verbose. Maybe "ostrace" (as in OS Trace) ? Other
suggestions ?
>
> The next section illustrates different use cases using that new model.
>
> Use Cases
> -----------------
>
> Every case considers these :
Each case considers these :
>
> * user A - UID: A; GID: A, tracing
> * user B - UID: B; GID: B, tracing
> * user C - UID: C; GID: C
>
> This first scenario shows how a certain user A will start a trace for the
> specific already running application app_1:
>
> Single user:
>
> +--------+ +------+ +-------+
> | user A |---- new ---->| ustd |<------>| app_1 |
> +--------+ +------+ +-------+
> | | | write
> fork create v
> +-----------------+ | | +--------+
> | ust-consumerd A |<----+ +--------->| shared |
> +-----------------+ | memory |
> ^ +--------+
> | |
> +-------------- read -------------+
>
Why should ustd fork the ust-consumerd ?
Also, we will need to represent the pipe communication between ust-consumerd and
ustd. It is needed to control the buffer access (get/put subbuffer operations).
If we use a named pipe for this, the access rights will probably need to be
u+rw and g+rw (write access for group tracing too, to send commands).
>
> The user A ask for a tracing session (new) to ustd using the PID of app_1. At
> that point, the buffers (shared memory) that will contains the traced data is
will contain
> created by the ustd registry daemon and gives to app_1 the location of that
> memory with write access (write). Important to understand here that this memory
> segment will be set with UID of the application (app_1) and with the tracing
> group GID (tracing).
>
> Then, the ust-consumerd is spawned (fork) having these security steps taken:
>
> * setuid(<user A>) and seteuid()
> * setgid(<tracing>) and setegid()
> * attach to shared memory in readonly (shmat)
>
> ust-consumerd A - UID: user A (rw-), GID: tracing (r--)
>
> Note here that the GID is set to "tracing" because user A is in that group. So,
> for example, the user B, who is also in the group tracing, will be ABLE to read
> the tracing data of that application since ust-consumerd has the read access.
>
> Two users tracing the same application:
>
> +--------+ +------+ +-------+
> | user A |-------- new ------->| ustd |<------------>| app_1 |--+
> +--------+ ^ +------+ +-------+ |
> | | | |
> +--------+ | | | |
> | user B |------+ fork create | write
> +--------+ | | |
> | | |
> +-----------------+ | | +-------------+ |
> | ust-consumerd A |<---+ +-------->| shared mem. |<-+
> +-----------------+ | | +-------------+ |
> ^ | | | |
> +-------------O----O---- read ------+ |
> | | |
> +-----------------+ | | +-------------+ |
> | ust-consumerd B |<---+ +-------->| shared mem. |<-+
> +-----------------+ +-------------+
> ^ |
> +------------- read ----------------+
>
> This is essentially the same scheme as before but the previous figure shows the
> memory and daemon separation for two different users tracing the same
> application (app_1). This applies for two or more users. Again, as mention
> earlier, access rights for ust-consumerd:
>
> ust-consumerd A - UID: user A (rw-), GID: tracing (r--)
> ust-consumerd B - UID: user B (rw-), GID: tracing (r--)
>
> The next case shows a single user NOT in the tracing group and shows the
> process scheme associate to that particular case which is different then
associated ... which differs from
> before since ustd will NOT register that session.
>
> Single user NOT in tracing group:
>
> +--------+ +------+
> | user C |---- new ---->| ustd |
> +--------+ +------+
> | +-------+
> spawn +------------------->| app_1 |
> | v +-------+
> | +-----------------+ |
> +-->| ust-consumerd C |-create+ | write
> +-----------------+ | |
> ^ v v
> | +--------+
> | | shared |
> +----- read -----| memory |
> +--------+
>
Again, we need to see the pipe control communication -- it would be directly
between the app and ust-consumerd here ? It seems hard to manage: it requires a
n-n pipe mapping between each traced application and each ust-consumerd, which
is something I think would be better not to do.
> Access rights will look like this:
>
> ust-consumerd C - UID: user C (rw-), GID: C (r--)
>
> For someone NOT in the tracing group, he CAN NOT use the global registry for
> creating a new session in order to prevent any user of being able to trace any
> application.
>
> So ustd will simply denied lttngtrace the ability to create a session and thus
deny
> lttngtrace will have to spawn and inform ust-consumerd to create the shared
> memory and give it to the application.
Thanks,
Mathieu
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
More information about the lttng-dev
mailing list