[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20240319185713.GA6407@openwall.com>
Date: Tue, 19 Mar 2024 19:57:13 +0100
From: Solar Designer <solar@...nwall.com>
To: yescrypt@...ts.openwall.com
Subject: Re: yescrypt && mmap(,,,,MAP_HUGETLB) && oom-kill
Hello Matthias,
On Tue, Mar 19, 2024 at 02:25:35PM +0100, Matthias Apitz wrote:
> 11:24:13.136472 mmap(NULL, 1075838976, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB, -1, 0) = -1 ENOMEM (Cannot allocate memory) <0.000053>
> 11:24:13.136579 mmap(NULL, 1073766464, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fadab8ad000 <0.000016>
> 11:24:14.607035 +++ killed by SIGKILL +++
> What does this mean exactly and how this can be avoided to crash our
> servers in production?
It appears that you use yescrypt settings such that 1+ GiB of memory is
needed per hash computation. That's extreme, but if it works for your
use case in terms of latency and throughput, it's great. You also need
to take care not to run out of available RAM.
One way to do that can be to limit the number of concurrent hash
computations to what your RAM size permits. For example, you can have a
fixed number of password hashing processes or threads, and have the rest
of your program delegate such requests to those. Or you can have an
atomic counter (or a non-atomic one guarded by a mutex) and a (second)
mutex in your multi-threaded program, with arbitrary number of threads.
Or, easiest of all, if you don't need more than one concurrent hash
computation (can have a 2nd+ attempt wait for completion of the running
one), you only need one mutex.
I hope this helps.
Alexander
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
