From: tip-bot for Davidlohr Bueso <tipbot@zytor.com>
To: linux-tip-commits@vger.kernel.org
Cc: torvalds@linux-foundation.org, linux-kernel@vger.kernel.org,
Waiman.Long@hpe.com, hpa@zytor.com, dave@stgolabs.net,
mingo@kernel.org, akpm@linux-foundation.org, dbueso@suse.de,
paulmck@linux.vnet.ibm.com, tglx@linutronix.de,
peterz@infradead.org
Subject: [tip:locking/core] locking/osq: Relax atomic semantics
Date: Fri, 18 Sep 2015 01:50:35 -0700 [thread overview]
Message-ID: <tip-c55a6ffa6285e29f874ed403979472631ec70bff@git.kernel.org> (raw)
In-Reply-To: <1442216244-4409-3-git-send-email-dave@stgolabs.net>
Commit-ID: c55a6ffa6285e29f874ed403979472631ec70bff
Gitweb: http://git.kernel.org/tip/c55a6ffa6285e29f874ed403979472631ec70bff
Author: Davidlohr Bueso <dave@stgolabs.net>
AuthorDate: Mon, 14 Sep 2015 00:37:24 -0700
Committer: Ingo Molnar <mingo@kernel.org>
CommitDate: Fri, 18 Sep 2015 09:27:29 +0200
locking/osq: Relax atomic semantics
... by using acquire/release for ops around the lock->tail. As such,
weakly ordered archs can benefit from more relaxed use of barriers
when issuing atomics.
Signed-off-by: Davidlohr Bueso <dbueso@suse.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Thomas Gleixner <tglx@linutronix.de>
Cc: Waiman Long <Waiman.Long@hpe.com>
Link: http://lkml.kernel.org/r/1442216244-4409-3-git-send-email-dave@stgolabs.net
Signed-off-by: Ingo Molnar <mingo@kernel.org>
---
kernel/locking/osq_lock.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c
index dc85ee2..d092a0c 100644
--- a/kernel/locking/osq_lock.c
+++ b/kernel/locking/osq_lock.c
@@ -50,7 +50,7 @@ osq_wait_next(struct optimistic_spin_queue *lock,
for (;;) {
if (atomic_read(&lock->tail) == curr &&
- atomic_cmpxchg(&lock->tail, curr, old) == curr) {
+ atomic_cmpxchg_acquire(&lock->tail, curr, old) == curr) {
/*
* We were the last queued, we moved @lock back. @prev
* will now observe @lock and will complete its
@@ -92,7 +92,11 @@ bool osq_lock(struct optimistic_spin_queue *lock)
node->next = NULL;
node->cpu = curr;
- old = atomic_xchg(&lock->tail, curr);
+ /*
+ * ACQUIRE semantics, pairs with corresponding RELEASE
+ * in unlock() uncontended, or fastpath.
+ */
+ old = atomic_xchg_acquire(&lock->tail, curr);
if (old == OSQ_UNLOCKED_VAL)
return true;
@@ -184,7 +188,8 @@ void osq_unlock(struct optimistic_spin_queue *lock)
/*
* Fast path for the uncontended case.
*/
- if (likely(atomic_cmpxchg(&lock->tail, curr, OSQ_UNLOCKED_VAL) == curr))
+ if (likely(atomic_cmpxchg_release(&lock->tail, curr,
+ OSQ_UNLOCKED_VAL) == curr))
return;
/*
next prev parent reply other threads:[~2015-09-18 8:51 UTC|newest]
Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-14 7:37 [PATCH -tip 1/3] locking/qrwlock: Rename ->lock to ->wait_lock Davidlohr Bueso
2015-09-14 7:37 ` [PATCH -tip 2/3] sched/wake_q: Relax to acquire semantics Davidlohr Bueso
2015-09-14 12:32 ` Peter Zijlstra
2015-09-14 21:08 ` Davidlohr Bueso
2015-09-15 9:49 ` Peter Zijlstra
2015-09-15 9:55 ` Peter Zijlstra
2015-09-15 12:41 ` Paul E. McKenney
2015-09-15 12:48 ` Peter Zijlstra
2015-09-15 14:09 ` Paul E. McKenney
2015-09-15 14:14 ` Peter Zijlstra
2015-09-15 15:34 ` Paul E. McKenney
2015-09-15 16:30 ` Peter Zijlstra
2015-09-15 17:09 ` Paul E. McKenney
2015-09-18 21:41 ` Paul E. McKenney
2015-09-21 9:22 ` Martin Schwidefsky
2015-09-22 10:27 ` Martin Schwidefsky
2015-09-22 12:23 ` Boqun Feng
2015-09-22 12:51 ` Martin Schwidefsky
2015-09-22 13:29 ` Boqun Feng
2015-09-22 14:33 ` Martin Schwidefsky
2015-09-22 15:28 ` Paul E. McKenney
2015-09-23 6:43 ` Martin Schwidefsky
2015-09-25 21:30 ` Paul E. McKenney
2015-09-15 19:49 ` Davidlohr Bueso
2015-09-16 9:01 ` Peter Zijlstra
2015-09-14 7:37 ` [PATCH -tip 3/3] locking/osq: Relax atomic semantics Davidlohr Bueso
2015-09-18 8:50 ` tip-bot for Davidlohr Bueso [this message]
2015-09-18 8:50 ` [tip:locking/core] locking/qrwlock: Rename ->lock to ->wait_lock tip-bot for Davidlohr Bueso
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=tip-c55a6ffa6285e29f874ed403979472631ec70bff@git.kernel.org \
--to=tipbot@zytor.com \
--cc=Waiman.Long@hpe.com \
--cc=akpm@linux-foundation.org \
--cc=dave@stgolabs.net \
--cc=dbueso@suse.de \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-tip-commits@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.vnet.ibm.com \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.