Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Fri, 18 Sep 2015 20:04:58 +0300
From: Solar Designer <>
Subject: Re: calloc()/mmap() vs. fork()

On Fri, Sep 18, 2015 at 06:49:22PM +0200, magnum wrote:
> On 2015-09-17 20:53, Solar Designer wrote:
> >I've just tried changing our mem_calloc() to make it actually use
> >calloc(), and using it for the hash tables and bitmaps in loader.c.
> This was news to be, although git blame shows I should have been aware 
> of it. I regard it a bug and have now committed such a change.

Now we'll need to look out for fork() performance regressions, if we
start using mem_calloc() for anything large prior to fork().

> >The result is surprising: fork() became slow.  Like, it literally takes
> >multiple seconds just to spawn the 7 child processes.  So e.g. if I
> >press a key a few seconds after all 8 processes are finally cracking, I
> >see them display very different timestamps (up to 10 seconds apart).
> >Perhaps our partial and out-of-order filling of the hash tables and
> >bitmaps results in page tables that are costly to fork().
> Was this with that multi-thread patch in use?

I actually meant calloc() as a possible alternative to that patch,
speeding up that place via avoiding memset(), so I think I did test
that.  But I am not 100% sure now.

> >We might want to revisit this once we implement keeping the loaded
> >hashes in a shared memory region.
> After reverting that problematic patch, I tried using calloc() in loader 
> with no problems (nor any clear benefit) but haven't committed that.

OK.  You have tried it with --fork, right?


Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.