[ltt-dev] [RFC UST] Processes model

[David Goulet]

First, let me say! This is a very impressive reply!

Thanks a lot for taking that time!

Comments below (in order to make things going!)

On 11-01-18 05:24 AM, Nils Carlson wrote:
>>
>> ustd - Main daemon that act as a trace session registry
> I would name this ust-sessiond, mostly because ustd has already been used
> and it might be good to not reuse the name.
>

Agree. Since it will only manage session, it makes perfect sense.

>> An ID SHOULD be a unique hash of the session name, trace path name,
>> date/time, PID and/or UID
>>
> What is the ID used for?

Since an application can be trace multiple time, the PID and trace name 
are the only things that makes them "unique". So, the idea of the ID is 
to be able for the user to interact with is trace without these 
information. It's a bit inspired by "screen" when you want to attach to 
a session :

There are several suitable screens on:
	14864.pts-10.raoul	(11-01-18 11:46:21 AM)	(Attached)
	14820.pts-2.raoul	(11-01-18 11:46:18 AM)	(Attached)
Type "screen [-d] -r [pid.]tty.host" to resume one of them.

That would be the kind og output that ust-sessiond sent you when you 
list the active session by you client control (ustctl).

>
>> The trace roles of ustd:
>>
>> Trace interaction - Create, Destroy, Pause, Stop, Start, Set options
>>
>> Registry - keep track of all tracing session information (basics):
>> * shared memory location
>> * UID and GID
>> * application PID
>> * trace ID (unique identifier)
>> * session name
>>
>
> This is a bit of a tricky one... I've been thinking a bit about this too.
> If we look at a traceable application we could say that we have a hierachy
> like so:
>
> +--------+
> | app_1 |
> +---+----+
> |
> +-------------+-------------+
> +---+------+ +---+------+
> | trace_1 | | trace_2 |
> +---+------+ +---+------+
> | |
> +-------------+ +....
> | | |
> +---+------+ +---+------+ +---+------+
> | buffer_1 | | buffer_2 | | buffer_2 |
> +---+------+ +---+------+ +---+------+....
>
>
> (Hope this picture is legible...)
>
> Now, the the consumer obviosuly has to be aware of the lowest levels of
> the hierarchy
> but does the session daemon need to be aware? I'm not sure...

Well, no. You are right on that. The ust-sessiond daemon doesn't need to 
be aware of all this. Later on, you propose that we only use 1 
ust-consumerd in order to scale so that it acts like a "trace registry" 
being aware of which application is connected to which buffers and so 
on. From that point of view, it makes sense.

>
>
>> Basic access control - allow or deny trace interaction based on the user
>> UID/GID.
>>
>> Consumer creation - spawns ust-consumerd.
>>
> Don't agree with this... At boot ust-consumerd should start.

Yep, at this point, after talking with some people and all the replies, 
ust-consumerd should NOT be spawned by ust-sessiond. Either by the 
control tool or at boot as you said.

> As I seet it we will always start two daemons at boot, ust-sessiond and
> ust-consumerd
>
> ust-consumerd is the default disk consumer daemon, ust-sessiond
> coordinates traces.
>
>
> Sequence of events
>
> Figure 1 ust-sessiond started:
>
> +--------------+
> | ust-sessiond |
> +---+----------+
>
>
> Figure 2 ust-consumerd started, ust-consumerd connect to ust-sessiond
> and registers
> itself as a consumer of type "disk":
>
>
> +--------------+
> | ust-sessiond |
> +---+----------+
> ^
> |
> register consumer
> |
> +---+-----------+
> | ust-consumerd |
> +---+-----------+
>
>
>
> Figure 3 application starts, connects to ust-sessiond and registers itself.
>
> +--------------+ +--------+
> | ust-sessiond |<--register--| app_1 |
> +---+----------+ +---+----+
> |
> |
> |
> |
> +---+-----------+
> | ust-consumerd |
> +---+-----------+
>
> Figure 4 ustctl starts a trace, telling the sessiond what
> the name of the trace should be, which application pid and what consumer it
> wants.

Confuse here by "what consumer it wants"... how the application will do 
that and knows?

>
> Figure 7 trace still starting, app allocates buffers in shared memory
> and passes
> each to the consumerd. The consumerd now maps each of them in turn and
> checks which
> cpu the belong to and passes them to the corresponding cpu thread where
> they are added
> to the epoll set.
>
> +---------------+ +--------+
> | ust-consumerd |<-map buffers-| app_1 |
> +---------------+ +---+----+
>
>
> 3. If we want to implement access control we can do it by passing
> credentials across the socket (see man unix). These can then be used
> by both the consumerd to open files in the right places with the right
> credentials. DBUS does this. We don't want to re-implement DBUS though.
>

I would argue here that this is a bit trickier then that. Let's see if I 
understand correctly what you are saying:

application allocates the buffers and set uid and gid to the app user. 
(Ex: uid:mysql and gid:mysql).

ust-consumerd has to be run as a privilege user to access applications 
buffers OR the buffers are set on read for everyone... which WE DON'T 
want. Also, if ust-consumerd want to write to disk using the app 
credentials, again privilege user is needed.

However, this brings up two problems:

1) I will suppose that the data on disk is set with the credentials of 
the user that requested tracing (Ex: nils) in order to be able to read 
it. If not, mysql:mysql will be used (using the app credentials) and 
"nils" will not be able to read the data.

2) This kind of architecture makes EVERY user on the machine being able 
to trace EVERY application... This is a behavior that we want to avoid 
with that re-engineering. Either ust-sessiond control the access based 
on the UID/GID (bad..) or the file system does it for us (good)

So this is why ust-sessiond was allocating the shared memory in order to 
set the "tracing" group so we could have a bit of control over that.

We could easily change that to ust-consumerd doing that but... again... 
some kind of privileges are needed to set the tracing group.

I hope I put it clearly enough here :S ... the whole point of having a 
consumerd per apps was the access rights on the buffers and trace data. 
Having a central consumerd, I agree however it makes that daemon run as 
a privilege user to manage the right credentials.

> 4. ust-sessiond doesn't have to work as a router... Routing is painful
> and I don't want to have to implement it. Connecting the consumerd with
> the app is much simpler. I thought about this, hard. I even had a prototype
> routing messages, and I promise, you don't want to do this.

I agree and it was never the initial intention. ust-sessiond only keeps 
track of session.

>> The new "lttngtrace" command line tool MUST be use to interact with
>> the ustd
>> registry daemon for every trace action needed by the user.
>
> We currently have a library (libustcmd) for this? ustctl wraps
> libustcmd. I have
> a small plan of renaming the library libustctl because it sounds better.
>
> The library has to stay as it will in not too distant a future be used
> to implement
> RPC calls to ust using tcf or some such.

The plan is to get a "strace" tool alike for tracing ust and/or lttng. 
libustctl is here to stay and lttngtrace will interface with it and 
lttctl and lttv text dump also.

However, we should discuss this within another thread. Mathieu and I 
will put some development effort in that so we have to get on the same 
page with you at least for UST!

Thanks again!
David

-- 
David Goulet
LTTng project, DORSAL Lab.

PGP/GPG : 1024D/16BD8563
BE3C 672B 9331 9796 291A  14C6 4AF7 C14B 16BD 8563