[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-ID: <20150726171711.0886dba8@vostro>
Date: Sun, 26 Jul 2015 17:17:11 +0300
From: Timo Teras <timo.teras@....fi>
To: "musl@...ts.openwall.com" <musl@...ts.openwall.com>
Subject: Multithreaded program VSZ increasing due to bad free() strategy
Hi,
It seems that the current free() strategy of "allocate all free
neighbors" has results in huge VSZ sizes (while RSS size keeps small).
The problem is that when the free() is made release most of the heap,
and simultaneous malloc() occurs on another thread it is likely to
extend heap. This is because free() has all memory allocated, and is
executing madvise() (or several of them) which is slow.
The result is that heap keeps getting extended all the time increasing
VSZ, and the slowing down the program (madvise is slower, the larger
area it's telling about). Though, due to madvise calls RSS stays low,
and it's not actually using any of that memory.
It would probably help a bit, to just move madvise() out from the for
loop as one free() can result in multiple madvise() calls if other
thread simultaneously free()'s neighboring areas and retried test of
(self->psize & next->csize & C_INUSE) while locks held fails. It would
probably make sense to move madvise() just out of loop when the final
block size is known.
But the strategy really needs to be better. When a multithreaded
reactive program is mainly allocating temporary space and frees it
soon after, it almost always end up in the race that free() holds
locked most of the heap in preparation to merge it to one big
continuous chunk. Causing the unwanted consequences.
Ideas?
Thanks,
Timo
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
