From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754636AbbIRVl3 (ORCPT ); Fri, 18 Sep 2015 17:41:29 -0400 Received: from e36.co.us.ibm.com ([32.97.110.154]:36229 "EHLO e36.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751704AbbIRVl1 (ORCPT ); Fri, 18 Sep 2015 17:41:27 -0400 X-Helo: d03dlp01.boulder.ibm.com X-MailFrom: paulmck@linux.vnet.ibm.com X-RcptTo: linux-kernel@vger.kernel.org Date: Fri, 18 Sep 2015 14:41:20 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: Davidlohr Bueso , Ingo Molnar , Thomas Gleixner , linux-kernel@vger.kernel.org, Davidlohr Bueso , heiko.carstens@de.ibm.com, schwidefsky@de.ibm.com Subject: Re: [PATCH -tip 2/3] sched/wake_q: Relax to acquire semantics Message-ID: <20150918214120.GA4405@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20150914210806.GG19736@linux-q0g1.site> <20150915094949.GA16853@twins.programming.kicks-ass.net> <20150915095512.GA18673@twins.programming.kicks-ass.net> <20150915124142.GF4029@linux.vnet.ibm.com> <20150915124800.GB16853@twins.programming.kicks-ass.net> <20150915140922.GG4029@linux.vnet.ibm.com> <20150915141439.GE16853@twins.programming.kicks-ass.net> <20150915153448.GI4029@linux.vnet.ibm.com> <20150915163028.GG16853@twins.programming.kicks-ass.net> <20150915170941.GL4029@linux.vnet.ibm.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20150915170941.GL4029@linux.vnet.ibm.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-MML: disable X-Content-Scanned: Fidelis XPS MAILER x-cbid: 15091821-0021-0000-0000-000012F5E30F Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 15, 2015 at 10:09:41AM -0700, Paul E. McKenney wrote: > On Tue, Sep 15, 2015 at 06:30:28PM +0200, Peter Zijlstra wrote: > > On Tue, Sep 15, 2015 at 08:34:48AM -0700, Paul E. McKenney wrote: > > > On Tue, Sep 15, 2015 at 04:14:39PM +0200, Peter Zijlstra wrote: > > > > On Tue, Sep 15, 2015 at 07:09:22AM -0700, Paul E. McKenney wrote: > > > > > On Tue, Sep 15, 2015 at 02:48:00PM +0200, Peter Zijlstra wrote: > > > > > > On Tue, Sep 15, 2015 at 05:41:42AM -0700, Paul E. McKenney wrote: > > > > > > > > Never mind, the PPC people will implement this with lwsync and that is > > > > > > > > very much not transitive IIRC. > > > > > > > > > > > > > > I am probably lost on context, but... > > > > > > > > > > > > > > It turns out that lwsync is transitive in special cases. One of them > > > > > > > is a series of release-acquire pairs, which can extend indefinitely. > > > > > > > > > > > > > > Does that help in this case? > > > > > > > > > > > > Probably not, but good to know. I still don't think we want to rely on > > > > > > ACQUIRE/RELEASE being transitive in general though. > > > > > > > > > > OK, I will bite... Why not? > > > > > > > > It would mean us reviewing all archs (again) and documenting it I > > > > suppose. Which is of course entirely possible. > > > > > > > > That said, I don't think the case at hand requires it, so lets postpone > > > > this for now ;-) > > > > > > True enough, but in my experience smp_store_release() and > > > smp_load_acquire() are a -lot- easier to use than other barriers, > > > and transitivity will help promote their use. So... > > > > > > All the TSO architectures (x86, s390, SPARC, HPPA, ...) support transitive > > > smp_store_release()/smp_load_acquire() via their native ordering in > > > combination with barrier() macros. x86 with CONFIG_X86_PPRO_FENCE=y, > > > which is not TSO, uses an mfence instruction. Power supports this via > > > lwsync's partial cumulativity. ARM64 supports it in SMP via the new ldar > > > and stlr instructions (in non-SMP, it uses barrier(), which suffices > > > in that case). IA64 supports this via total ordering of all release > > > instructions in theory and by the actual full-barrier implementation > > > in practice (and the fact that gcc emits st.rel and ld.acq instructions > > > for volatile stores and loads). All other architectures use smp_mb(), > > > which is transitive. > > > > > > Did I miss anything? > > > > I think that about covers it.. the only odd duckling might be s390 which > > is documented as TSO but recently grew smp_mb__{before,after}_atomic(), > > which seems to confuse matters. > > Fair point, adding Martin and Heiko on CC for their thoughts. > > It looks like this applies to recent mainframes that have new atomic > instructions, which, yes, might need something to make them work with > fully transitive smp_load_acquire() and smp_store_release(). > > Martin, Heiko, the question is whether or not the current s390 > smp_store_release() and smp_load_acquire() can be transitive. > For example, if all the Xi variables below are initially zero, > is it possible for all the r0, r1, r2, ... rN variables to > have the value 1 at the end of the test. Right... This time actually adding Martin and Heiko on CC... Thanx, Paul > CPU 0 > r0 = smp_load_acquire(&X0); > smp_store_release(&X1, 1); > > CPU 1 > r1 = smp_load_acquire(&X1); > smp_store_release(&X2, 1); > > CPU 2 > r2 = smp_load_acquire(&X2); > smp_store_release(&X3, 1); > > ... > > CPU N > rN = smp_load_acquire(&XN); > smp_store_release(&X0, 1); > > If smp_store_release() and smp_load_acquire() are transitive, the > answer would be "no". > > A similar litmus test involving atomics would be as follows, again > with all Xi initially zero: > > CPU 0 > atomic_inc(&X0); > smp_store_release(&X1, 1); > > CPU 1 > r1 = smp_load_acquire(&X1); > smp_store_release(&X2, 1); > > CPU 2 > r2 = smp_load_acquire(&X2); > smp_store_release(&X3, 1); > > ... > > CPU N > rN = smp_load_acquire(&XN); > r0 = atomic_read(&X0); > > Here, the question is whether r0 can be zero, but r1, r2, ... rN all > being 1 at the end of the test. > > Thoughts? > > Thanx, Paul