[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20140805120815.GD22308@port70.net>
Date: Tue, 5 Aug 2014 14:08:16 +0200
From: Szabolcs Nagy <nsz@...t70.net>
To: musl@...ts.openwall.com
Subject: Re: openmp/pthreads and fork...
* Isaac Dunham <ibid.ag@...il.com> [2014-08-04 22:56:51 -0700]:
> OpenBLAS (optionally) uses pthreads or OpenMP, and OpenMP uses pthreads.
> OpenBLAS implements the BLAS (Basic Linear Algebra Subprograms) API/ABI,
> like ATLAS, MKL, and so on.
> Some programs that use BLAS will fork(); python is one of these.
> With OpenBLAS, this had caused "random" segfaults due to use of threads
> by the library both before and after the fork.
> The OpenBLAS issue is here:
> https://github.com/xianyi/OpenBLAS/issues/294
>
> They worked around this using pthread_atfork() to cleanup before the fork().
> I see some claims that calling any pthread functions in the child would be
> UB, so I'm wondering about a few things:
> -Is the last-mentioned claim correct?
the fix does make fork work with openmp in some cases:
if the application calls fork when no openmp code is
running then both the child and parent have the omp lib
in consistent state
if the application calls fork without synchronizing it
with openmp calls then the application is not allowed to
do much in the child anyway (it cannot call openmp code
so inconsistent state there is not an issue)
and the atfork handler is correct with some caveats:
- gomp_we_are_forked var should be volatile sig_atomic_t
in theory (but in practice their static int might work too)
- if fork happens in an async signal handler that interrupted
pthread_atfork, then the pthread_atfork global state may be
inconsistent so fork can crash in libc
(libc could probably guard against this but no libc does that
afaik, howver other libcs dont have an async-signal-safe fork
so this would be ub anyway)
> -What is musl's behavior?
> -How correct, and how likely to work with musl, is the fix for libgomp
> mentioned here:
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60035
> (and for that matter, the equivalent workaround referred to in OpenBLAS
> issue 294)?
the above workaround should work with musl (except when fork
is used form a sig handler)
with glibc there are other caveats:
- glibc dlclose removes the shared library, so if openmp was dlopened
and then dlclosed (may happen with python), then the next fork will
crash (because of the dangling atfork handler)
(the workaround is to use __register_atfork #ifdef __GLIBC__ which
takes care of the dlclose issue)
- pthread_atfork is only available in -lpthread, so a library that
is itself not multi-threaded but tries to work around fork issues
cannot use it without pulling in all the pthreads (__register_atfork
does not have this issue)
> -Is there a safe (non-crashing) way to use/write libraries that might or
> might not be built with threading--whether POSIX-specified or just
> "working with musl" ?
threading is not the issue, forking is (more specifically doing
library calls in the child after fork in a multi-threaded process)
similar issue came up recently in libressl
(see my notes starting at the "But.." section in
http://port70.net/~nsz/47_arc4random.html
)
my conclusion was that a library cannot reasonably do anything
to guard against unsafe fork usage in the application so it
shouldn't try to
pthread_atfork is broken, makes fork non-async-signal-safe and
cannot be used for the originally intended purposes
the application may not be aware of underlying threads in various
libraries, so fork should be avoided: use posix_spawn or
fork+exec (without intervening library calls), other fork usage
is just broken (unless the application has full control over
its library dependencies)
Powered by blists - more mailing lists
Confused about mailing lists and their use?
Read about mailing lists on Wikipedia
and check out these
guidelines on proper formatting of your messages.
