[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20111128010729.GA21667@openwall.com>
Date: Mon, 28 Nov 2011 05:07:29 +0400
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: .bss size in -jumbo
magnum, Jim -
.bss in 1.7.8-jumbo-8 grew to 140 MB in a 64-bit build with OpenMP:
$ size ../run/john
text data bss dec hex filename
873185 104696 140226432 141204313 86a9b59 ../run/john
The main culprits are:
$ size *.o | sort -rnk 3 | head
8318 1664 55837824 55847806 3542b7e NETNTLMv2_fmt_plug.o
6431 1088 27918480 27925999 1aa1def NETLMv2_fmt_plug.o
6230 1088 27525334 27532652 1a41d6c mskrb5_fmt_plug.o
8438 1472 10420400 10430310 9f2766 MSCHAPv2_fmt_plug.o
7500 1664 10420379 10429543 9f2467 NETNTLM_fmt_plug.o
6038 1184 2883721 2890943 2c1cbf NETLM_fmt_plug.o
4267 992 1573005 1578264 181518 NETSPLITLM_fmt_plug.o
14169 1472 713100 728741 b1ea5 mscash1_fmt_plug.o
3239 800 589872 593911 90ff7 crc32_fmt_plug.o
4332 704 270656 275692 434ec XSHA512_fmt.o
I think at least top 5 of these (or maybe 7 or 9) should be changed to
allocate their memory dynamically, like 1.7.9 does in MD5_fmt.c and
BSDI_fmt.c. (1.7.9's DES_fmt.c and c3_fmt.c also use dynamic memory
allocation, but in format-specific ways. BF_fmt.c's saved_key[] is
relatively small (the number of buffered keys is low because this hash
is so slow), and BF_std.c puts its larger data structures on thread
stacks (just one instance per thread).)
Since there are no format destructors - I mean, we have init(), but
not the opposite - the entire 140 MB will end up getting allocated by a
full --test run anyway, but at least that address space won't be
allocated during normal cracking runs. Also, systems that can't afford
140 MB will be able to run the tests individually (not all tests in one
invocation). So this will be an improvement.
Of course, on typical systems this 140 MB is just address space and not
even virtual memory unless/until someone runs the full --test. So this
cleanup won't matter too much. Yet it'd be nice to make it if we can do
that without performance impact and without much effort (e.g., limiting
this to top 5 only). Also, this may allow for scalability to much
higher thread counts.
Format destructors may be added as a next step.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
