From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_SANE_1 autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6F3A2C4743C for ; Mon, 21 Jun 2021 18:20:20 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2FC11606A5 for ; Mon, 21 Jun 2021 18:20:20 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2FC11606A5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2VVpmznHf3MY/izsKu0f0VjWRb1MrGZzMhZ4fBFeR7A=; b=THF5P8s6MFGNsx sII/tv66ek4WwMpQmsejcZdUOd3XDuHBDfwxXrYV8mrEW3S+2LKi4rCTjKNFxIEpYBzx6tj+NiqiS SqgCHV6THXbf1uXlevkJ+XaTbd4Jb3ZB2TOcDkeAftuAGbrRiHWNNOu+NpNMsmMuER5zMI8hmbfu+ QIhCoUrvCa4E0Pzqmks03rOnYkihSrY/Xcq37e3UygdQ3ONE1mfZd+2P5PiM2wKlP5YVvXZgTW6WR jw2DRZ8XetWb4FNOPIobDiSxslurjydVRqafLuQYl2UJcps20tOkxBjBUvGQtI1ltci1/D5mjL1A4 413vuoSyQjz42qi2Q6Zw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvOV6-004XAn-2C; Mon, 21 Jun 2021 18:18:24 +0000 Received: from mail.kernel.org ([198.145.29.99]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lvOQO-004V2K-17 for linux-arm-kernel@lists.infradead.org; Mon, 21 Jun 2021 18:13:37 +0000 Received: by mail.kernel.org (Postfix) with ESMTPSA id E04216100B; Mon, 21 Jun 2021 18:13:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1624299211; bh=0VGM/xjNPP00I7nXJITC0QAQKHqBhB6TnT6BLsxbI6E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=biMM4p7o5SYSWkznwR/oTsI8nPwB7FiRobBLwj/M/RX/7DnVle0lEySGRqQDaYdBV PLyUFGrLbcD1AMHWCjWxpkS0gOE1nO2zmFQuXZx//t9Ct/7iTtYTHLy7tzh5rsNyKY 2dvqSVrVMmNsJPxSCsZP5gzaDgQPcNJGAjh3/21V4AQJwpISTjwxIisqWnqaATj19q vOIheHO7WF2oTtq6eIozVtm4nujAaE86yA2KPkU5Rg8I7flY+0Xfi9FWE+GaPq8E7D i9v8JHrzLwNONC/6mbdpjUZOBbY78qbwdMa0pv6sIluLDHWhs4EDIv9rXAtRUiRrgF rLkar5YbXl3yw== Date: Mon, 21 Jun 2021 19:13:26 +0100 From: Will Deacon To: Frank Li Cc: Catalin Marinas , Zhi Li , Shenwei Wang , Han Xu , Nitin Garg , Jason Liu , "linux-arm-kernel@lists.infradead.org" Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers in default I/O accessors Message-ID: <20210621181326.GD29713@willie-the-truck> References: <20210617092744.GB6314@arm.com> <20210617172528.GA24813@willie-the-truck> <20210617174131.GC24813@willie-the-truck> <20210617214012.GA25403@willie-the-truck> <20210621162641.GA29595@willie-the-truck> <20210621165941.GB29595@willie-the-truck> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210621_111332_167361_A24320E2 X-CRM114-Status: GOOD ( 32.63 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Mon, Jun 21, 2021 at 05:56:43PM +0000, Frank Li wrote: > > > > -----Original Message----- > > From: Will Deacon > > Sent: Monday, June 21, 2021 12:00 PM > > To: Frank Li > > Cc: Catalin Marinas ; Zhi Li ; > > Shenwei Wang ; Han Xu ; Nitin Garg > > ; Jason Liu ; linux-arm- > > kernel@lists.infradead.org > > Subject: Re: [EXT] Re: The problem about arm64: io: Relax implicit barriers > > in default I/O accessors > > > > Caution: EXT Email > > > > On Mon, Jun 21, 2021 at 05:26:41PM +0100, Will Deacon wrote: > > > On Mon, Jun 21, 2021 at 04:11:57PM +0000, Frank Li wrote: > > > > > Oh, interesting. Maybe this is a case where OSH vs SY actually makes > > a > > > > > difference. I'm not quite sure what it means for the coherency of > > normal, > > > > > non-cacheable accesses (which are outer-shareable) so that probably > > needs a > > > > > bit more thought. > > > > > > > > > > Can you confirm that the issue *does* still occur if you use dmb(osh) > > > > > instead of dmb(oshst), please? > > > > > > > > After get ARM support > > https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fservices. > > arm.com%2Fsupport%2Fs%2Fcase%2F5003t00001RuJHw&data=04%7C01%7Cfrank.li% > > 40nxp.com%7Ca319ac5213a14aa6bb2508d934d5facc%7C686ea1d3bc2b4c6fa92cd99c5c30 > > 1635%7C0%7C0%7C637598915908588560%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM > > DAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6%2F%2FK > > ScsCmnUgNPnzcvyjRrOLjLVPrHtbVgI3J959U%2BQ%3D&reserved=0, > > > > This issue have some progress. > > > > > > > > Our system configure SYSBARDISABLE = 0x0, So ARM core barrier propagate > > to CCI-400 > > > > > > > > Our DMA and USB is located below downstream of CCI-400. So USB or DMA > > is located > > > > in system shared domain. Only use dmb(st), CCI-400 wait for previous > > transaction > > > > Complete. When dma(osh), the response is sent when snoop responses are > > received for > > > > all earlier transactions. CCI-400 don't wait for previous write finish. > > > > > > Thanks for following up. I'll cook a patch to fix this... > > > > ... and in doing so, I realised I still have a question about this. > > > > If a CPU is writing to a zero-initialised non-cacheable buffer in memory > > and does something like: > > > > buffer[0] = 1; > > dma_wmb(); // DMB OSHST > > buffer[64] = 1; > > > > would a non-coherent device reading this be able to see buffer[64] == 1 > > but buffer[0] = 0? In other words, do we need to upgrade the dmb_* barriers > > as well as the I/O accessors, or are they still ordered by the bus fabric > > because all of the accesses are going to the DDR? > > I think re-order is possible. According to my understanding, > If cci ack dmb(oshst), the follow order is not guaranteed if no address overlap > for normal memory. Hmm, so that's a bit rubbish because it means that load-acquire/store-release to non-cacheable memory will *not* create order for non-coherent devices, as the memory type is outer-shareable :/ So rewriting the above as: buffer[0] = 1; smp_store_release(&buffer[64], 1); wouldn't be ordered either. Can you confirm that it is the case, please? Will _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel