diff options
| author | Prateek Sood <prsood@codeaurora.org> | 2017-07-05 23:24:16 +0530 |
|---|---|---|
| committer | Gerrit - the friendly Code Review server <code-review@localhost> | 2017-07-27 04:58:44 -0700 |
| commit | c2878c131a985a117e993d97fe464326b77d2850 (patch) | |
| tree | 019d77aa5bf222c2a6119b98caf73451a932a84f /kernel | |
| parent | 0a57938d144e24ea7ef6b56ae6b2d4d732060f00 (diff) | |
rwsem: fix missed wakeup due to reordering of load
If a spinner is present, there is a chance that the load of
rwsem_has_spinner() in rwsem_wake() can be reordered with
respect to decrement of rwsem count in __up_write() leading
to wakeup being missed.
spinning writer up_write caller
--------------- -----------------------
[S] osq_unlock() [L] osq
spin_lock(wait_lock)
sem->count=0xFFFFFFFF00000001
+0xFFFFFFFF00000000
count=sem->count
MB
sem->count=0xFFFFFFFE00000001
-0xFFFFFFFF00000001
RMB
spin_trylock(wait_lock)
return
rwsem_try_write_lock(count)
spin_unlock(wait_lock)
schedule()
Reordering of atomic_long_sub_return_release() in __up_write()
and rwsem_has_spinner() in rwsem_wake() can cause missing of
wakeup in up_write() context. In spinning writer, sem->count
and local variable count is 0XFFFFFFFE00000001. It would result
in rwsem_try_write_lock() failing to acquire rwsem and spinning
writer going to sleep in rwsem_down_write_failed().
The smp_rmb() will make sure that the spinner state is
consulted after sem->count is updated in up_write context.
Change-Id: I96de9a65adedb35d1ee2c6c36dc7759c9b8f5d4d
Signed-off-by: Prateek Sood <prsood@codeaurora.org>
Diffstat (limited to 'kernel')
| -rw-r--r-- | kernel/locking/rwsem-xadd.c | 35 |
1 files changed, 35 insertions, 0 deletions
diff --git a/kernel/locking/rwsem-xadd.c b/kernel/locking/rwsem-xadd.c index a4d4de05b2d1..75c950ede9c7 100644 --- a/kernel/locking/rwsem-xadd.c +++ b/kernel/locking/rwsem-xadd.c @@ -511,6 +511,41 @@ struct rw_semaphore *rwsem_wake(struct rw_semaphore *sem) unsigned long flags; /* + * If a spinner is present, there is a chance that the load of + * rwsem_has_spinner() in rwsem_wake() can be reordered with + * respect to decrement of rwsem count in __up_write() leading + * to wakeup being missed. + * + * spinning writer up_write caller + * --------------- ----------------------- + * [S] osq_unlock() [L] osq + * spin_lock(wait_lock) + * sem->count=0xFFFFFFFF00000001 + * +0xFFFFFFFF00000000 + * count=sem->count + * MB + * sem->count=0xFFFFFFFE00000001 + * -0xFFFFFFFF00000001 + * RMB + * spin_trylock(wait_lock) + * return + * rwsem_try_write_lock(count) + * spin_unlock(wait_lock) + * schedule() + * + * Reordering of atomic_long_sub_return_release() in __up_write() + * and rwsem_has_spinner() in rwsem_wake() can cause missing of + * wakeup in up_write() context. In spinning writer, sem->count + * and local variable count is 0XFFFFFFFE00000001. It would result + * in rwsem_try_write_lock() failing to acquire rwsem and spinning + * writer going to sleep in rwsem_down_write_failed(). + * + * The smp_rmb() here is to make sure that the spinner state is + * consulted after sem->count is updated in up_write context. + */ + smp_rmb(); + + /* * If a spinner is present, it is not necessary to do the wakeup. * Try to do wakeup only if the trylock succeeds to minimize * spinlock contention which may introduce too much delay in the |