[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20121029161425.GL254@brightrain.aerifal.cx>
Date: Mon, 29 Oct 2012 12:14:25 -0400
From: Rich Felker <dalias@...ifal.cx>
To: musl@...ts.openwall.com
Subject: Re: interesting discussion about static linking on luajit ML
On Mon, Oct 29, 2012 at 10:50:03AM +0100, Szabolcs Nagy wrote:
> * John Spencer <maillist-musl@...fooze.de> [2012-10-29 07:14:35 +0100]:
> > especially the statement here about memory usage seems to be a
> > common misconception:
> >
> > http://www.freelists.org/post/luajit/Creating-a-statically-linked-executable-for-a-LuaJITC-program,22
> >
>
> well obviously if the statically linked binary is smaller
> then the dynamically linked one, then he cannot save
> space the way he wants.. i'm not sure where he got that
> size only matters on disk
>
> but he is right about that usually dynamically linked programs
> are smaller than statically linked ones
> (you should not compare to glibc binaries as they are huge no matter what)
>
> > rich, you recently said something about this topic on irc, iirc it
> > was along the lines of:
> > you'd need to link substantially huge parts of libc.a to use nearly
> > as much memory,
> > as is typically wasted with relocation overhead in dynamically
> > linked apps, do i remember correctly ?
If you use less than the "dynamic linking overhead" on the libc
comparison page worth of code from the library, then static linking
will use less memory at runtime than dynamic linking. Right now that
threshold is at 20k but I hope to get it back down to 12k soon.
> the point is that with dynamically linked binary you must
> have the *entire* lib available in memory
This is actually not true. Only the pages which are accessed must be
resident in memory. If commonly-used and rarely-used functions are
ordered randomly in the library, then this is likely to cause nearly
the whole library to be paged in, except in cases where individual
unused functions are larger than the page size. If the order of
objects linked into the library is optimized to put all the
rarely-used stuff together, then you can get reasonable memory usage
with shared libraries. This is actually a direction of optimization we
should explore in musl -- I think it should be fairly easy to make the
build scripts group either a known set of commonly-used object files
first, or a known set of rarely-used object files last.
> (so even if you have that shared between different processes
> you need multiple processes using the lib to gain anything)
> (and of course not everything is shared: there are writable
> data segments (.data, .bss) which cannot be shared between
> processes once they are modified (although they are small in
> case of musl), and pic code is a bit bigger as well)
The sice increase of PIC is nontrivial; I'd say it's around 10-15% on
average for i386. On x86_64 and ARM it should be less, but I have not
measured. The worst case is that small functions can double or even
triple in size.
> with static linking you only link those object files that
> are actually used by the binary
> (eg. in case of musl the static lib is about 340k and about
> 80k is just the math+complex part, if you don't use any of
> that then it won't get linked in so you don't have to keep
> that around in memory)
Don't forget that iconv.o is 71k. So that's already a total of 150k
that's unlikely to be used in most programs. Once you add in all the
legacy cruft, I'd say easily over 50% of the library is unlikely to be
used.
> (this does not help when you link with xlib because the
> simplest x client pulls in a very large part of xlib
> and when you run x you surely run several x clients)
Yes, that's a design flaw.
> (btw even static linked programs can have in memory sharing:
> when you run the same process image several times.
Yes. This is especially worth noting for shells.
> i think a serious script language should have a static
> version optionally available, there are situations where
> it makes sense and the size difference they are complaining
> about rarely matters
This is even more true for languages like lua whose primary use case
is embedded in other programs. The people responsible for the
interpreter/jit implementation have no idea what usage cases the
embedding application will be dealing with and whether it might need
static linking.
Rich
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
