[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150509021624.GB27423@openwall.com>
Date: Sat, 9 May 2015 05:16:24 +0300
From: Solar Designer <solar@...nwall.com>
To: john-dev@...ts.openwall.com
Subject: Re: Adding OpenMP support to SunMD5
On Sat, May 09, 2015 at 09:38:36AM +0800, Lei Zhang wrote:
> On May 9, 2015, at 5:22 AM, Solar Designer <solar@...nwall.com> wrote:
> >> Currently I manually defined _DEBUG for ease of experimenting. I'm not sure about the point of allocating those array dynamically. Is there some optimization involved?
> >
> > Not wasting address space. With static allocation, they'd be in .bss
> > even when the SunMD5 format isn't in use.
>
> BTW, I don't quite understand this point. How does "wasting address space" impact program performance?
It usually does not, except program startup (e.g., Kai's fuzzing might
become slower).
Also, you wanted these things in thread-local storage, which may be a
scarce resource:
http://en.wikipedia.org/wiki/Thread-local_storage
"Many systems impose restrictions on the size of the thread-local memory
block, in fact often rather tight limits."
http://stackoverflow.com/questions/1460483/on-which-platforms-is-thread-local-storage-limited-and-how-much-is-available
"There is an implementation specific limit to the number of thread
specific storage objects that can be created, and this limit may be
small."
"[...] if you need portability, your best bet is to minimize TLS use -
put a single pointer in TLS, and put everything you need in a data
structure hung off that pointer."
So you'd be back to dynamic allocation.
> For a 64-bit build, isn't the address space big enough anyway?
Not all builds are 64-bit.
We actually tried to deal with .bss growth back in 2011:
http://www.openwall.com/lists/john-dev/2011/11/28/1
At the time, our .bss was 140 MB. We've since got it back under
control. Now it's just 9 MB. To me, this is still pretty high, but no
longer ridiculous.
140 MB was a significant fraction of the 32-bit address space,
unnecessarily limiting e.g. what wordlist size can be loaded to memory.
Also, when someone runs --test for all formats at once, all this address
space gets written to, becoming actual memory usage. We now have done()
methods (we didn't back in 2011), which can (and do) free the dynamic
allocations, so --test no longer consumes as much memory.
You actually saw the scrypt format consume several gigabytes during
--test on MIC. Now imagine this allocation was static. A few other
formats could fail self-test because of it.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
