From: Waiman Long <waiman.long@hp.com>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Ingo Molnar <mingo@redhat.com>,
Thomas Gleixner <tglx@linutronix.de>,
"H. Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, linux-kernel@vger.kernel.org,
Scott J Norton <scott.norton@hp.com>,
Douglas Hatch <doug.hatch@hp.com>,
Davidlohr Bueso <dave@stgolabs.net>
Subject: Re: [PATCH v2 1/6] locking/pvqspinlock: Unconditional PV kick with _Q_SLOW_VAL
Date: Wed, 15 Jul 2015 20:18:23 -0400 [thread overview]
Message-ID: <55A6F84F.5050502@hp.com> (raw)
In-Reply-To: <20150715091012.GG2859@worktop.programming.kicks-ass.net>
On 07/15/2015 05:10 AM, Peter Zijlstra wrote:
> On Tue, Jul 14, 2015 at 10:13:32PM -0400, Waiman Long wrote:
>> The smp_store_release() is not a full barrier. In order to avoid missed
>> wakeup, we may need to add memory barrier around locked and cpu state
>> variables adding to complexity. As the chance of spurious wakeup is very
>> low, it is easier and safer to just do an unconditional kick at unlock
>> time.
> I have the below patch. We need that rmb in there anyhow for the hash to
> work.
>
> ---
> Subject: locking/pvqspinlock: Order pv_unhash after cmpxchg on unlock slowpath
> From: Will Deacon<will.deacon@arm.com>
> Date: Mon, 13 Jul 2015 16:58:30 +0100
>
> When we unlock in __pv_queued_spin_unlock, a failed cmpxchg on the lock
> value indicates that we need to take the slow-path and unhash the
> corresponding node blocked on the lock.
>
> Since a failed cmpxchg does not provide any memory-ordering guarantees,
> it is possible that the node data could be read before the cmpxchg on
> weakly-ordered architectures and therefore return a stale value, leading
> to hash corruption and/or a BUG().
>
> This patch adds an smb_rmb() following the failed cmpxchg operation, so
> that the unhashing is ordered after the lock has been checked.
>
> Cc: Paul McKenney<paulmck@linux.vnet.ibm.com>
> Cc: Waiman Long<Waiman.Long@hp.com>
> Cc: Steve Capper<Steve.Capper@arm.com>
> Reported-by: Peter Zijlstra<peterz@infradead.org>
> Signed-off-by: Will Deacon<will.deacon@arm.com>
> [peterz: More comments]
> Signed-off-by: Peter Zijlstra (Intel)<peterz@infradead.org>
> Link: http://lkml.kernel.org/r/20150713155830.GL2632@arm.com
> ---
> kernel/locking/qspinlock_paravirt.h | 23 ++++++++++++++++++-----
> 1 file changed, 18 insertions(+), 5 deletions(-)
>
> --- a/kernel/locking/qspinlock_paravirt.h
> +++ b/kernel/locking/qspinlock_paravirt.h
> @@ -244,13 +244,17 @@ static void pv_wait_head(struct qspinloc
> if (!lp) { /* ONCE */
> lp = pv_hash(lock, pn);
> /*
> - * lp must be set before setting _Q_SLOW_VAL
> + * We must hash before setting _Q_SLOW_VAL, such that
> + * when we observe _Q_SLOW_VAL in __pv_queued_spin_unlock()
> + * we'll be sure to be able to observe our hash entry.
> *
> - * [S] lp = lock [RmW] l = l->locked = 0
> - * MB MB
> - * [S] l->locked = _Q_SLOW_VAL [L] lp
> + * [S] pn->state
> + * [S]<hash> [Rmw] l->locked == _Q_SLOW_VAL
> + * MB RMB
> + * [RmW] l->locked = _Q_SLOW_VAL [L]<unhash>
> + * [L] pn->state
> *
> - * Matches the cmpxchg() in __pv_queued_spin_unlock().
> + * Matches the smp_rmb() in __pv_queued_spin_unlock().
> */
> if (!cmpxchg(&l->locked, _Q_LOCKED_VAL, _Q_SLOW_VAL)) {
> /*
> @@ -306,6 +310,15 @@ __visible void __pv_queued_spin_unlock(s
> }
>
> /*
> + * A failed cmpxchg doesn't provide any memory-ordering guarantees,
> + * so we need a barrier to order the read of the node data in
> + * pv_unhash *after* we've read the lock being _Q_SLOW_VAL.
> + *
> + * Matches the cmpxchg() in pv_wait_head() setting _Q_SLOW_VAL.
> + */
> + smp_rmb();
According to memory_barriers.txt, cmpxchg() is a full memory barrier. It
didn't say a failed cmpxchg will lose its memory guarantee. So is the
documentation right? Or is that true for some architectures? I think it
is not true for x86.
Cheers,
Longman
next prev parent reply other threads:[~2015-07-16 0:18 UTC|newest]
Thread overview: 25+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-15 2:13 [PATCH 0/6 v2] locking/qspinlock: Enhance pvqspinlock performance Waiman Long
2015-07-15 2:13 ` [PATCH v2 1/6] locking/pvqspinlock: Unconditional PV kick with _Q_SLOW_VAL Waiman Long
2015-07-15 9:10 ` Peter Zijlstra
2015-07-16 0:18 ` Waiman Long [this message]
2015-07-16 5:42 ` Peter Zijlstra
2015-07-16 14:07 ` Waiman Long
2015-07-16 15:04 ` Waiman Long
2015-07-16 15:10 ` Will Deacon
2015-08-03 16:59 ` [tip:locking/core] locking/Documentation: Clarify failed cmpxchg( ) memory ordering semantics tip-bot for Will Deacon
2015-08-03 17:36 ` Davidlohr Bueso
2015-07-15 2:13 ` [PATCH v2 2/6] locking/pvqspinlock: Add pending bit support Waiman Long
2015-07-15 2:13 ` [PATCH v2 3/6] locking/pvqspinlock: Collect slowpath lock statistics Waiman Long
2015-07-15 2:13 ` [PATCH v2 4/6] locking/pvqspinlock: Allow vCPUs kick-ahead Waiman Long
2015-07-15 9:39 ` Peter Zijlstra
2015-07-16 2:01 ` Waiman Long
2015-07-16 5:46 ` Peter Zijlstra
2015-07-16 14:51 ` Waiman Long
2015-07-15 2:13 ` [PATCH v2 5/6] locking/pvqspinlock: Opportunistically defer kicking to unlock time Waiman Long
2015-07-15 6:14 ` Raghavendra K T
2015-07-15 10:03 ` Peter Zijlstra
2015-07-16 2:18 ` Waiman Long
2015-07-16 5:49 ` Peter Zijlstra
2015-07-15 2:13 ` [PATCH v2 6/6] locking/pvqspinlock: Queue node adaptive spinning Waiman Long
2015-07-15 10:01 ` Peter Zijlstra
2015-07-16 2:13 ` Waiman Long
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55A6F84F.5050502@hp.com \
--to=waiman.long@hp.com \
--cc=dave@stgolabs.net \
--cc=doug.hatch@hp.com \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=scott.norton@hp.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.