[lttng-dev] [lttng-tools] Removal of root_regression tests

[Marcel Hamer]

Hello Jonathan,

On Mon, Jun 13, 2022 at 11:21:49AM -0400, Jonathan Rajotte-Julien wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
> 
> Hi Marcel,
> 
> ----- Original Message -----
> > From: "Marcel Hamer via lttng-dev" <lttng-dev at lists.lttng.org>
> > To: "lttng-dev" <lttng-dev at lists.lttng.org>
> > Sent: Monday, 13 June, 2022 07:49:39
> > Subject: [lttng-dev] [lttng-tools] Removal of root_regression tests
> 
> > Hello,
> >
> > Since version v2.12.9 of lttng-tools the root_regression file has been emptied
> > to make the tests part of the 'make check' sequence instead.
> >
> > We were always actively using that test file as part of our regression testing.
> > In our case we are working in a cross-compilation environment, where the run.sh
> > script was used on target for testing and as such not at compile time. It is not
> > easy to run a make check sequence on a target.
> 
> I would suggest that you take a look at how OpenEmbedded does it with ptest AFAIK it match your requirements:
> 
> https://github.com/openembedded/openembedded-core/blob/c7e2901eacf3dcbd0c5bb91d2cc1d467b4a9aaf7/meta/recipes-kernel/lttng/lttng-tools_2.13.7.bb#L75
> 

That is a very good suggestion. I guess we were a bit too focused on our existing
solution of using run.sh. We will look into this.

> >
> > It is now also a bit unclear which tests actually require root access and which
> > tests do not. I understood this was the reason the file was called
> > 'root_regression'?
> 
> Yes when the tests suites primarily used `prove` via run.sh.
> 
> We have been slowly moving away from it for a good time and now mostly use the Automake test harness as much as possible.
> 
> The worse that will happen if you run a test that required root as a non-root user is that `skip` tap output will be emitted.
> 
> >
> > Some questions that get raised because of this:
> >
> > - Is there now an alternative way to run regressions on target in case of a
> >  cross-compilation environment?
> 
> AFAIU, this is out of scope of the lttng project. Still, I would recommend that you see how yocto/oe do it with ptest.
> 
> > - Would there be a possibility to fill the 'root_regression' file again and
> >  possibly revert this change?
> 
> Feel free to do it out-of-tree. I doubt that we are the only project that WindRiver handles that uses
> the automake test harness and that do not provide a easy way to run on-target for cross-compilation testing.

Yes, you are right and that is a fair point. We will look into the ptest
solution.

> 
> A quick grep with "isroot" should get you 95% there.
> 
> > - How are tests now identified that require root access?
> 
> All tests that require root access test for it at runtime
> 
> Something along:
> 
> regression/tools/streaming/test_high_throughput_limits:
> 
>  if [ "$(id -u)" == "0" ]; then
>         isroot=1
>  else
>          isroot=0
>  fi
> 
>  skip $isroot "Root access is needed to set bandwidth limits. Skipping all tests." $NUM_TESTS ||
>  {
>  ...
>     Tests are done here.
>  }
> 
> Cheers

Thanks for the tip on how to identify test cases that require root privileges.

Kind regards,

Marcel