[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20211216135057.GM7074@brightrain.aerifal.cx>
Date: Thu, 16 Dec 2021 08:50:57 -0500
From: Rich Felker <dalias@...c.org>
To: musl@...ts.openwall.com
Subject: Re: Current state and future of musl development
infrastructure
On Thu, Dec 16, 2021 at 12:40:05PM +0100, Natanael Copa wrote:
> On Thu, 16 Dec 2021 00:02:20 -0500
> Rich Felker <dalias@...c.org> wrote:
>
> > On Wed, Dec 15, 2021 at 04:52:05PM -0600, Zach van Rijn wrote:
> > > On Wed, 2021-12-15 at 22:22 +0100, jvoisin wrote:
> > > > Hello everyone,
>
> ....
>
> > > As for CI.
> > >
> > > It would probably be helpful to be able to run libc-test and
> > > perhaps other applications on a number of real hardware platforms
> > > in some automated manner.
> >
> > Yes. Having this automated would be really nice. Even better if it
> > could be done not just for the official upstream repo but for proposed
> > patches/patchsets or third-party git branches (on some sort of
> > request).
>
> I think it would be helpful to have the tests bundled in musl git
> repository and in release tarballs. It is common practice for
> downstream packagers to run `make && make check`, to run unit tests to
> have some basic smoke testing (that does not require root permissions).
I'm aware this is common practice, and I disagree with it. It *kinda*
makes sense for a project where the interfaces to be tested are
*defined* by the project itself. It does not make sense when they are
mostly or entirely defined by third parties (the relevant standards).
> Depending on some external test suite (libc-test) is suboptimal because
> they may not catch recent additions,
This would apply even if tests were integrated unless we had a
glibc-like policy that you're not allowed to fix bugs without a
matching test. In my book this is an extremely bad policy that's kept
bugs around for a long time (years?) beyond the date it was known how
to fix them just because nobody could write a good test.
> and it may be difficult to keep track of what is expected to pass
> or fail.
This could possibly be within the scope of CI?
> I think it would make sense to have both a lighter/faster integrated
> set of unit tests, that are bundled with musl itself and have a more
> extensive (external) integration tests that is run with the CI.
Aside from conceptual reasons not to do this, from a practical
standpoint I think it would mean duplicated work or gratuitous
competition, taking away effort from maintaining a strong cross-libc
test package that's incentivized not to do things in non-portable ways
or poke through abstraction layers. For example I'd love to have more
tests of things that are "hard to test" in musl, like hostname
resolution, but when they happen I want them to happen with namespaces
virtualizing resolv.conf and hosts file and port 53 (which is
future-proof and works with other libc's too), not by poking at
internal symbols in musl to attach a test harness to it (which would
not be future-proof and would create pressure to preserve
musl-internal interface boundaries).
> > I'd really love if the CI were optimized to do incremental builds and
> > reuse artifacts as much as possible for rapid results, but doing that
> > right probably requires fixing missing dependency tracking in the
> > build.
>
> Just an idea for incremental builds: Do out-of-tree builds. let the CI
> store a tarball of the out-of-tree build in some key/value storage,
> using the commit hash as the id. Subsequent builds tries fetch the old
> heads community id from storage and extract it to the out-of-tree build
> location.
I'm not clear why you'd need to re-run CI for same commit hash. The
artifacts probably need to be cached by ccache type logic or
something, not commit hash, if you're going to do this.
> It might also be a good idea to try get it work first and optimize
> afterwards :)
I was expressing this more from a standpoint of shopping for good
existing CI tools rather than building one -- in that context it's
good to know a set of properties/features that might be helpful to
developers & maintaner.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
