[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAH4OOv4ba45gN67mgTuebxCh332nW-NeQxo5pQR+WWieDomQdw@mail.gmail.com>
Date: Wed, 12 Jan 2022 19:46:26 -0800
From: Farid Zakaria <fmzakari@...c.edu>
To: Rich Felker <dalias@...c.org>
Cc: musl@...ts.openwall.com, "Scogland, Tom" <scogland1@...l.gov>, me@...menstoppels.nl,
Carlos Maltzahn <carlosm@...c.edu>
Subject: Re: is RUNPATH being incorrectly inherited inherited?
Hi Rich,
Thank you for taking the time to read my email and responding.
I was worried it was a bit too pedantic in explanation and I had had
some trivial error in the setup.
(I am only beginning to gain deeper knowledge on linking)
On Wed, Jan 12, 2022 at 7:29 PM Rich Felker <dalias@...c.org> wrote:
>
> On Wed, Jan 12, 2022 at 05:29:57PM -0800, Farid Zakaria wrote:
> > Hello,
> >
> > I'm observing a strange behavior and it seems to contradict what
> > ld.so[1] says should be the case for RUNPATH, namely I am observing
> > that musl's dynamic linker seems to use the DT_RUNPATH of the
> > executable when searching for entries of its children.
>
> I thought this was documented somewhere but maybe it's not. musl does
> not have separate RPATH/RUNPATH behaviors. The clearly wrong part of
> the legacy RPATH behavior (order relative to LD_LIBRARY_PATH) is not
> followed, and the rationale stated at the time this was implemented in
> musl was that the only justification for the old behavior there was
> compatibility with existing binaries, which was not an issue for musl.
>
> On the other hand, the glibc RUNPATH behavior of ignoring the
> dependent DSO's (or main app's) RUNPATH when loading indirect
> dependencies completely breaks the ability to set a RUNPATH in the
> application to use a *set* of unmodified library binaries in a
> particular application-specific path. I'm surprised you found an
> instance where that's what you want, since as far as I could tell at
> the time it was an unwanted change.
>
> > > Using the directories specified in the DT_RUNPATH dynamic section
> > > attribute of the binary if present. Such directories are searched
> > > only to find those objects required by DT_NEEDED (direct
> > > dependencies) entries and do not apply to those objects' children,
> > > which must themselves have their own DT_RUNPATH entries. This is
> > > unlike DT_RPATH, which is applied to searches for all children in
> > > the dependency tree.
> >
> > I've uploaded a Makefile[2] that you can use to follow along.
> >
> > First let's build the binary. It's designed to depend on two shared
> > libraries _libx.so_ and _liby.so_; _liby.so_ also depends on _libx.so_
> >
> > Note: that I make the top-level _libx.so_ an absolute path using patchelf.
> >
> > ```
> > # build it
> > $ make CC=musl-gcc
> >
> > # let's inspect it to see what it looks like
> > # here we can see the needed, including the one absolute
> > $ patchelf --print-needed exe
> > liby.so
> > /home/fzakaria/code//c/libx.so
> > libc.so
> >
> > # here is the RUNPATH
> > $ patchelf --print-rpath exe
> > $ORIGIN/b:$ORIGIN/c
> >
> > # an alternate query
> > $ objdump -x ./exe | grep RUNPATH
> > RUNPATH $ORIGIN/b:$ORIGIN/
> >
> > # here is a dependent library
> > $ patchelf --print-needed b/liby.so
> > libx.so
> > libc.so
> >
> > # liby.so has no RUNPATH
> > objdump -x b/liby.so| grep RUNPATH
> > ```
> >
> > I would expect this *to not work* since there is no way for liby.so to
> > discover libx.so, especially given the previous correspondence on the
> > mailing lists on how musl does not utilize soname for the cache.
> >
> > However you can see that the linker is trying to resolve _libx.so_
> > using the RUNPATH when it is loading _liby.so_
> >
> > ```
> > $ strace -e openat,stat,open ./exe
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
> > directory)
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > 24
> > +++ exited with 0 +++
> > ```
> >
> > Harmen's libtree[3] already fails to find the library.
> > ```
> > $ libtree exe
> > exe
> > ├── libx.so [direct]
> > └── .//b/liby.so [runpath]
> > └── libx.so not found
> > ┊ Paths considered
> > ```
> >
> > To me this means that the ld.so here is not respecting the description
> > of not propagating the search paths to the children.
>
> Yes, that's intentional. Is this something you want not to happen?
This specific case, I only brought up after having read the manpage
for linux-ld/ld.so
on my quest to better understand dynamic linkers. It does seem that
other tools are making similar assumptions
however (libtree) when trying to replicate the search pattern employed
by a linker.
The rationale you mention about the ability to set a single top-level
RUNPATH is desirable however
I do find the delta between what's documented for ld.so to have been confusing.
I am as part of some research evaluating other alternatives to how to
specify link-paths and part of that
research direction involves more explicitly specifying whether a path
should/should not be inherited; in addition to a
few other ideas.
It's clear you are aware of this discrepancy and it was done so knowingly.
At least I know I was not mis-understanding the results of my setup :)
> > FWIW, as per the other correspondence, if I remove part of the
> > RUNPATH, then the binary fails to load even though _libx.so_ was
> > already present.
>
> This is expected given the current lack of SONAME processing because,
> while it's present, it's not tracked as being "what you should get by
> resolving the name libx.so". If we do adopt SONAME processing here it
> would be found, and would necessarily preclude any path search from
> happening since it would already be present. Presumably that's what
> you would want to happen, right?
Yes! This is more of an interesting quirk I find different between
glibc whose functionality
I think would make a positive addition. Happy to continue discussing
this issue on the other
mailing list emails :)
> >
> > ```
> > $ patchelf --print-rpath exe
> > $ORIGIN/b
> >
> > $ /exe
> > Error loading shared library libx.so: No such file or directory
> > (needed by /home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so)
> > ```
> >
> > Thank you.
> >
> > [1] https://man7.org/linux/man-pages/man8/ld.so.8.html
> > [2] https://gist.github.com/fzakaria/375f2c27f2db000369ec49a684ef8095
> > [3] https://github.com/haampie/libtree
>
> Thanks for the example and for finding something that needs better
> documentation (even if just, for now, on the wiki under
> https://wiki.musl-libc.org/functional-differences-from-glibc.html)
>
> Rich
On Wed, Jan 12, 2022 at 7:29 PM Rich Felker <dalias@...c.org> wrote:
>
> On Wed, Jan 12, 2022 at 05:29:57PM -0800, Farid Zakaria wrote:
> > Hello,
> >
> > I'm observing a strange behavior and it seems to contradict what
> > ld.so[1] says should be the case for RUNPATH, namely I am observing
> > that musl's dynamic linker seems to use the DT_RUNPATH of the
> > executable when searching for entries of its children.
>
> I thought this was documented somewhere but maybe it's not. musl does
> not have separate RPATH/RUNPATH behaviors. The clearly wrong part of
> the legacy RPATH behavior (order relative to LD_LIBRARY_PATH) is not
> followed, and the rationale stated at the time this was implemented in
> musl was that the only justification for the old behavior there was
> compatibility with existing binaries, which was not an issue for musl.
>
> On the other hand, the glibc RUNPATH behavior of ignoring the
> dependent DSO's (or main app's) RUNPATH when loading indirect
> dependencies completely breaks the ability to set a RUNPATH in the
> application to use a *set* of unmodified library binaries in a
> particular application-specific path. I'm surprised you found an
> instance where that's what you want, since as far as I could tell at
> the time it was an unwanted change.
>
> > > Using the directories specified in the DT_RUNPATH dynamic section
> > > attribute of the binary if present. Such directories are searched
> > > only to find those objects required by DT_NEEDED (direct
> > > dependencies) entries and do not apply to those objects' children,
> > > which must themselves have their own DT_RUNPATH entries. This is
> > > unlike DT_RPATH, which is applied to searches for all children in
> > > the dependency tree.
> >
> > I've uploaded a Makefile[2] that you can use to follow along.
> >
> > First let's build the binary. It's designed to depend on two shared
> > libraries _libx.so_ and _liby.so_; _liby.so_ also depends on _libx.so_
> >
> > Note: that I make the top-level _libx.so_ an absolute path using patchelf.
> >
> > ```
> > # build it
> > $ make CC=musl-gcc
> >
> > # let's inspect it to see what it looks like
> > # here we can see the needed, including the one absolute
> > $ patchelf --print-needed exe
> > liby.so
> > /home/fzakaria/code//c/libx.so
> > libc.so
> >
> > # here is the RUNPATH
> > $ patchelf --print-rpath exe
> > $ORIGIN/b:$ORIGIN/c
> >
> > # an alternate query
> > $ objdump -x ./exe | grep RUNPATH
> > RUNPATH $ORIGIN/b:$ORIGIN/
> >
> > # here is a dependent library
> > $ patchelf --print-needed b/liby.so
> > libx.so
> > libc.so
> >
> > # liby.so has no RUNPATH
> > objdump -x b/liby.so| grep RUNPATH
> > ```
> >
> > I would expect this *to not work* since there is no way for liby.so to
> > discover libx.so, especially given the previous correspondence on the
> > mailing lists on how musl does not utilize soname for the cache.
> >
> > However you can see that the linker is trying to resolve _libx.so_
> > using the RUNPATH when it is loading _liby.so_
> >
> > ```
> > $ strace -e openat,stat,open ./exe
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or
> > directory)
> > open("/home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/c/libx.so",
> > O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 3
> > 24
> > +++ exited with 0 +++
> > ```
> >
> > Harmen's libtree[3] already fails to find the library.
> > ```
> > $ libtree exe
> > exe
> > ├── libx.so [direct]
> > └── .//b/liby.so [runpath]
> > └── libx.so not found
> > ┊ Paths considered
> > ```
> >
> > To me this means that the ld.so here is not respecting the description
> > of not propagating the search paths to the children.
>
> Yes, that's intentional. Is this something you want not to happen?
>
> > FWIW, as per the other correspondence, if I remove part of the
> > RUNPATH, then the binary fails to load even though _libx.so_ was
> > already present.
>
> This is expected given the current lack of SONAME processing because,
> while it's present, it's not tracked as being "what you should get by
> resolving the name libx.so". If we do adopt SONAME processing here it
> would be found, and would necessarily preclude any path search from
> happening since it would already be present. Presumably that's what
> you would want to happen, right?
>
> >
> > ```
> > $ patchelf --print-rpath exe
> > $ORIGIN/b
> >
> > $ /exe
> > Error loading shared library libx.so: No such file or directory
> > (needed by /home/fzakaria/code/playground/cpp/musl-experiment/experiment-2/b/liby.so)
> > ```
> >
> > Thank you.
> >
> > [1] https://man7.org/linux/man-pages/man8/ld.so.8.html
> > [2] https://gist.github.com/fzakaria/375f2c27f2db000369ec49a684ef8095
> > [3] https://github.com/haampie/libtree
>
> Thanks for the example and for finding something that needs better
> documentation (even if just, for now, on the wiki under
> https://wiki.musl-libc.org/functional-differences-from-glibc.html)
>
> Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
