From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3360714D432; Thu, 11 Apr 2024 13:29:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712842144; cv=none; b=bf6RsdA8sSUx4ADsfPzEHnhWHfJUrml+3xdQGdT0a9jkv5bCYD48ObUhoukAefF+DLpoaueUJAOgTCuhHHYDM/KwmrReV8qRQ4aS70sDIhth1DZcVo9RJiEe6ySwdre/K0Y4MQzhRcuz2WGww/md12BB8k2kphlp6tD6MWJMZdY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712842144; c=relaxed/simple; bh=gUdPGNpqUc9YmdjxtvqHwxuSQo6lbsOe/wMJ91M4o0o=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=DJkm3g2Os+vZ3BMaGz4CXZSo4RRVmXXj9EB2rUm90bPvMEAmk2Q2AIM2E4Dpmvk1AJkKLBjuAchpO1k4CiUpFmdgsx4hKg4JejtdS4A43k5kK921Cw3Z8JDLfxaQZk9iDODh+VKrlQj5GgrJ1swoRh73S2CcLh+waK1Nqoxqovc= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fKoSzUBS; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fKoSzUBS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DB70C2BBFC; Thu, 11 Apr 2024 13:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712841795; bh=gUdPGNpqUc9YmdjxtvqHwxuSQo6lbsOe/wMJ91M4o0o=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=fKoSzUBSJR3KbMhdCG1CHI6hpF0gXlRuAvgMgnJu3Btuu7SakSzaUvgpYTcIqkIEc jgmJGBdpkyTo0lA+qAr0005WzjFF+rKzBEeaWW2AJ1BxyLdtuuDmzyM+CHRqXSfxEs H6rZNWX8bJ7fmqEkP4X4qFKPD24zqJfBSDiSWnTE+jSaqUrzTf5mGMl/ZZg4s1fYdp ucoWdQToXGS8Xvysk5dNy3OEIQzLISY9ijErrqIZQ6csx46lZ6JO76X+mukuZXZdQH LCzebE+5MA3IfDrazx2LsqtBN2CHGRafShYpWzoI7CTN5Xsx+GrJsNV0g5/lUIAOxk gLYmU8Zrcw2Gw== Date: Thu, 11 Apr 2024 14:28:54 +0100 From: Will Deacon To: Hector Martin Cc: Catalin Marinas , Marc Zyngier , Mark Rutland , Zayd Qumsieh , Justin Lu , Ryan Houdek , Mark Brown , Ard Biesheuvel , Mateusz Guzik , Anshuman Khandual , Oliver Upton , Miguel Luis , Joey Gouly , Christoph Paasch , Kees Cook , Sami Tolvanen , Baoquan He , Joel Granados , Dawei Li , Andrew Morton , Florent Revest , David Hildenbrand , Stefan Roesch , Andy Chiu , Josh Triplett , Oleg Nesterov , Helge Deller , Zev Weiss , Ondrej Mosnacek , Miguel Ojeda , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Asahi Linux Subject: Re: [PATCH 0/4] arm64: Support the TSO memory model Message-ID: <20240411132853.GA26481@willie-the-truck> References: <20240411-tso-v1-0-754f11abfbff@marcan.st> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240411-tso-v1-0-754f11abfbff@marcan.st> User-Agent: Mutt/1.10.1 (2018-07-13) Hi Hector, On Thu, Apr 11, 2024 at 09:51:19AM +0900, Hector Martin wrote: > x86 CPUs implement a stricter memory modern than ARM64 (TSO). For this > reason, x86 emulation on baseline ARM64 systems requires very expensive > memory model emulation. Having hardware that supports this natively is > therefore very attractive. Such hardware, in fact, exists. This series > adds support for userspace to identify when TSO is available and > toggle it on, if supported. I'm probably going to make myself hugely unpopular here, but I have a strong objection to this patch series as it stands. I firmly believe that providing a prctl() to query and toggle the memory model to/from TSO is going to lead to subtle fragmentation of arm64 Linux userspace. It's not difficult to envisage this TSO switch being abused for native arm64 applications: * A program no longer crashes when TSO is enabled, so the developer just toggles TSO to meet a deadline. * Some legacy x86 sources are being ported to arm64 but concurrency is hard so the developer just enables TSO to (mostly) avoid thinking about it. * Some binaries in a distribution exhibit instability which goes away in TSO mode, so a taskset-like program is used to run them with TSO enabled. In all these cases, we end up with native arm64 applications that will either fail to load or will crash in subtle ways on CPUs without the TSO feature. Assuming that the application cannot be fixed, a better approach would be to recompile using stronger instructions (e.g. LDAR/STLR) so that at least the resulting binary is portable. Now, it's true that some existing CPUs are TSO by design (this is a perfectly valid implementation of the arm64 memory model), but I think there's a big difference between quietly providing more ordering guarantees than software may be relying on and providing a mechanism to discover, request and ultimately rely upon the stronger behaviour. An alternative option is to go down the SPARC RMO route and just enable TSO statically (although presumably in the firmware) for Apple silicon. I'm assuming that has a performance impact for native code? Will P.S. I briefly pondered the idea of the kernel toggling the bit in the ELF loader when e.g. it sees an x86 machine type but I suspect that doesn't really help with existing emulators and you'd still need a way to tell the emulator whether or not it was enabled. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EA916C001CC for ; Thu, 11 Apr 2024 13:29:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=i1LGD+WAjYhbCzYg2Zr5li5u73SZizia+AXyoeFY/VU=; b=iKC8q3Kt6IfUtm jLPIHiFEIODJvdLkaFam8NQIxktAEJn871F4RnOM/uavXcAbzIBN8W+abooEGfSQdcIFY3jq2OtBo VFtUt2pUet29WnzBzLl0AdxVUACoiQs2H3offKvjX1CR/xdsakS2ikiuTzmMxFhgDayNrRqzgCpXO PEg3FDEQng2tgMSwbghQMGQEwfyM1kFhv4qSYYdjhb2kGMnKWJmBDuUdLlSgoBwKGcGxryMkT1FiZ 3VtrzLT9agqXaFFrISpZ8XmrnCoA6mc7C2w15BbmSlG9REylK8y1jTFGOidOY3rLh/b41vxu/Em8N 5GM/EkxZG+aEKGArtvXA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruuUG-0000000C9MC-37Y5; Thu, 11 Apr 2024 13:29:08 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1ruuUD-0000000C9Ke-1MJ1 for linux-arm-kernel@lists.infradead.org; Thu, 11 Apr 2024 13:29:06 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2973E62063; Thu, 11 Apr 2024 13:29:04 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DB70C2BBFC; Thu, 11 Apr 2024 13:23:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712841795; bh=gUdPGNpqUc9YmdjxtvqHwxuSQo6lbsOe/wMJ91M4o0o=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=fKoSzUBSJR3KbMhdCG1CHI6hpF0gXlRuAvgMgnJu3Btuu7SakSzaUvgpYTcIqkIEc jgmJGBdpkyTo0lA+qAr0005WzjFF+rKzBEeaWW2AJ1BxyLdtuuDmzyM+CHRqXSfxEs H6rZNWX8bJ7fmqEkP4X4qFKPD24zqJfBSDiSWnTE+jSaqUrzTf5mGMl/ZZg4s1fYdp ucoWdQToXGS8Xvysk5dNy3OEIQzLISY9ijErrqIZQ6csx46lZ6JO76X+mukuZXZdQH LCzebE+5MA3IfDrazx2LsqtBN2CHGRafShYpWzoI7CTN5Xsx+GrJsNV0g5/lUIAOxk gLYmU8Zrcw2Gw== Date: Thu, 11 Apr 2024 14:28:54 +0100 From: Will Deacon To: Hector Martin Cc: Catalin Marinas , Marc Zyngier , Mark Rutland , Zayd Qumsieh , Justin Lu , Ryan Houdek , Mark Brown , Ard Biesheuvel , Mateusz Guzik , Anshuman Khandual , Oliver Upton , Miguel Luis , Joey Gouly , Christoph Paasch , Kees Cook , Sami Tolvanen , Baoquan He , Joel Granados , Dawei Li , Andrew Morton , Florent Revest , David Hildenbrand , Stefan Roesch , Andy Chiu , Josh Triplett , Oleg Nesterov , Helge Deller , Zev Weiss , Ondrej Mosnacek , Miguel Ojeda , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Asahi Linux Subject: Re: [PATCH 0/4] arm64: Support the TSO memory model Message-ID: <20240411132853.GA26481@willie-the-truck> References: <20240411-tso-v1-0-754f11abfbff@marcan.st> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20240411-tso-v1-0-754f11abfbff@marcan.st> User-Agent: Mutt/1.10.1 (2018-07-13) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240411_062905_501707_32C36B24 X-CRM114-Status: GOOD ( 19.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Hector, On Thu, Apr 11, 2024 at 09:51:19AM +0900, Hector Martin wrote: > x86 CPUs implement a stricter memory modern than ARM64 (TSO). For this > reason, x86 emulation on baseline ARM64 systems requires very expensive > memory model emulation. Having hardware that supports this natively is > therefore very attractive. Such hardware, in fact, exists. This series > adds support for userspace to identify when TSO is available and > toggle it on, if supported. I'm probably going to make myself hugely unpopular here, but I have a strong objection to this patch series as it stands. I firmly believe that providing a prctl() to query and toggle the memory model to/from TSO is going to lead to subtle fragmentation of arm64 Linux userspace. It's not difficult to envisage this TSO switch being abused for native arm64 applications: * A program no longer crashes when TSO is enabled, so the developer just toggles TSO to meet a deadline. * Some legacy x86 sources are being ported to arm64 but concurrency is hard so the developer just enables TSO to (mostly) avoid thinking about it. * Some binaries in a distribution exhibit instability which goes away in TSO mode, so a taskset-like program is used to run them with TSO enabled. In all these cases, we end up with native arm64 applications that will either fail to load or will crash in subtle ways on CPUs without the TSO feature. Assuming that the application cannot be fixed, a better approach would be to recompile using stronger instructions (e.g. LDAR/STLR) so that at least the resulting binary is portable. Now, it's true that some existing CPUs are TSO by design (this is a perfectly valid implementation of the arm64 memory model), but I think there's a big difference between quietly providing more ordering guarantees than software may be relying on and providing a mechanism to discover, request and ultimately rely upon the stronger behaviour. An alternative option is to go down the SPARC RMO route and just enable TSO statically (although presumably in the firmware) for Apple silicon. I'm assuming that has a performance impact for native code? Will P.S. I briefly pondered the idea of the kernel toggling the bit in the ELF loader when e.g. it sees an x86 machine type but I suspect that doesn't really help with existing emulators and you'd still need a way to tell the emulator whether or not it was enabled. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel