From: Davidlohr Bueso <dave@stgolabs.net>
To: Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>
Cc: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
Davidlohr Bueso <dave@stgolabs.net>,
linux-kernel@vger.kernel.org, Davidlohr Bueso <dbueso@suse.de>
Subject: [PATCH -tip 3/3] locking/osq: Relax atomic semantics
Date: Mon, 14 Sep 2015 00:37:24 -0700 [thread overview]
Message-ID: <1442216244-4409-3-git-send-email-dave@stgolabs.net> (raw)
In-Reply-To: <1442216244-4409-1-git-send-email-dave@stgolabs.net>
... by using acquire/release for ops around the lock->tail. As such,
weakly ordered archs can benefit from more relaxed use of barriers
when issuing atomics.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
---
kernel/locking/osq_lock.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c
index dc85ee2..d092a0c 100644
--- a/kernel/locking/osq_lock.c
+++ b/kernel/locking/osq_lock.c
@@ -50,7 +50,7 @@ osq_wait_next(struct optimistic_spin_queue *lock,
for (;;) {
if (atomic_read(&lock->tail) == curr &&
- atomic_cmpxchg(&lock->tail, curr, old) == curr) {
+ atomic_cmpxchg_acquire(&lock->tail, curr, old) == curr) {
/*
* We were the last queued, we moved @lock back. @prev
* will now observe @lock and will complete its
@@ -92,7 +92,11 @@ bool osq_lock(struct optimistic_spin_queue *lock)
node->next = NULL;
node->cpu = curr;
- old = atomic_xchg(&lock->tail, curr);
+ /*
+ * ACQUIRE semantics, pairs with corresponding RELEASE
+ * in unlock() uncontended, or fastpath.
+ */
+ old = atomic_xchg_acquire(&lock->tail, curr);
if (old == OSQ_UNLOCKED_VAL)
return true;
@@ -184,7 +188,8 @@ void osq_unlock(struct optimistic_spin_queue *lock)
/*
* Fast path for the uncontended case.
*/
- if (likely(atomic_cmpxchg(&lock->tail, curr, OSQ_UNLOCKED_VAL) == curr))
+ if (likely(atomic_cmpxchg_release(&lock->tail, curr,
+ OSQ_UNLOCKED_VAL) == curr))
return;
/*
--
2.1.4
next prev parent reply other threads:[~2015-09-14 7:37 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-14 7:37 [PATCH -tip 1/3] locking/qrwlock: Rename ->lock to ->wait_lock Davidlohr Bueso
2015-09-14 7:37 ` [PATCH -tip 2/3] sched/wake_q: Relax to acquire semantics Davidlohr Bueso
2015-09-14 12:32 ` Peter Zijlstra
2015-09-14 21:08 ` Davidlohr Bueso
2015-09-15 9:49 ` Peter Zijlstra
2015-09-15 9:55 ` Peter Zijlstra
2015-09-15 12:41 ` Paul E. McKenney
2015-09-15 12:48 ` Peter Zijlstra
2015-09-15 14:09 ` Paul E. McKenney
2015-09-15 14:14 ` Peter Zijlstra
2015-09-15 15:34 ` Paul E. McKenney
2015-09-15 16:30 ` Peter Zijlstra
2015-09-15 17:09 ` Paul E. McKenney
2015-09-18 21:41 ` Paul E. McKenney
2015-09-21 9:22 ` Martin Schwidefsky
2015-09-22 10:27 ` Martin Schwidefsky
2015-09-22 12:23 ` Boqun Feng
2015-09-22 12:51 ` Martin Schwidefsky
2015-09-22 13:29 ` Boqun Feng
2015-09-22 14:33 ` Martin Schwidefsky
2015-09-22 15:28 ` Paul E. McKenney
2015-09-23 6:43 ` Martin Schwidefsky
2015-09-25 21:30 ` Paul E. McKenney
2015-09-15 19:49 ` Davidlohr Bueso
2015-09-16 9:01 ` Peter Zijlstra
2015-09-14 7:37 ` Davidlohr Bueso [this message]
2015-09-18 8:50 ` [tip:locking/core] locking/osq: Relax atomic semantics tip-bot for Davidlohr Bueso
2015-09-18 8:50 ` [tip:locking/core] locking/qrwlock: Rename ->lock to ->wait_lock tip-bot for Davidlohr Bueso
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1442216244-4409-3-git-send-email-dave@stgolabs.net \
--to=dave@stgolabs.net \
--cc=dbueso@suse.de \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.