musl - Re: Resuming work on new semaphore

Follow @Openwall on Twitter for new release announcements and other news

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LNX.2.11.1504232031340.2677@monopod.intra.ispras.ru>
Date: Thu, 23 Apr 2015 21:24:36 +0300 (MSK)
From: Alexander Monakov <amonakov@...ras.ru>
To: musl@...ts.openwall.com
Subject: Re: Resuming work on new semaphore

> The latter saves the result of a_cas to prevent an extra load, but I
> don't think it makes any significant difference and it might be seen
> as uglier.

I think we should use the result of a_cas here: it's part of sem_post "fast
path", and doing it is not too difficult.  I'm using a slightly different
version below.

> However neither of those address the overflow issue, which I've tried
> to address here:
> 
> #define VAL0_MAX ((SEM_VALUE_MAX+1)/2)

Signed integer overflow here -- using corrected version below.

> Does this all sound correct?

I'm afraid not. We must always do futex-wake when incrementing val[1].
Otherwise wake loss is possible:

1. Semaphore initialized to VAL0_MAX
2. Thread A enters sem_post, observes saturated val[0]
3. Thread B downs val[0] to 0 by calling sem_wait VAL0_MAX times
4. Thread B calls sem_wait again and enters futex_wait
5. Thread A ups val[1].
.. At this point thread A must futex-wake val[1].

My version:

#define VAL0_MAX (SEM_VALUE_MAX/2+1)
#define VAL1_MAX (SEM_VALUE_MAX/2)

int sem_post(sem_t *sem)
{
	int old, val = sem->__val[0];
	val -= val == VAL0_MAX;
	while (old = val, (val = a_cas(sem->__val, val, val+1)) != old)
		if (val == VAL0_MAX)
			goto wake;
	if (val < 0) {
wake:;
		int priv = sem->__val[2];
		do
			if ((val = sem->__val[1]) == VAL1_MAX) {
				errno = EOVERFLOW;
				return -1;
			}
		while (val != a_cas(sem->__val+1, val, val+1));
		__wake(sem->__val+1, 1, priv);
	}
	return 0;
}

After sufficiently many waiters have been killed, val[1] can reach VAL1_MAX
without val[0] also reaching VAL0_MAX, in which case sem_post can report
EOVERFLOW prematurely.  From previous emails it seems it's not a big concern.

It is also possible that EOVERFLOW will be reported prematurely in race
windows when a waiter returning from futex-wait with EWOULDBLOCK has not
decremented val[1] of a recently saturated semaphore yet. Example:

1. Semaphore initialized to SEM_VALUE_MAX
2. Thread A downs val[0] to 0 by calling sem_wait VAL0_MAX times. val[1]
remains at VAL1_MAX.
3. Thread B calls sem_wait and enters futex wait
4. Thread A calls sem_post, observes val[0]<0 && val[1] == VAL1_MAX

It's possible to make the window smaller by reordering futex-wait loop, but it
will remain.  At the moment I don't have a good way out.

Thanks.
Alexander

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.