From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF033C433B4 for ; Thu, 1 Apr 2021 15:39:53 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92CF661376 for ; Thu, 1 Apr 2021 15:39:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92CF661376 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc:To:From: Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender :Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References:List-Owner; bh=ktKPDQ4MCklulNw6IS+jcsO9eMdH+aTnOfrRB7UBenE=; b=UZ5wm31e67DSQbcLjx6ToF3p3S ph3Y79P05IZYzHmcZwjn/uVLaoONY52zGEi+SDoursR6CsEZ26fXbyE4XeVuCGqRtYOHby8m8uyYt Y3P3UQDOomM7vjJRZEhK3t+cdcTNHEEm53o/CDzMhFpXPVPMPMTf8TNd0ULcmIobrb348VzR8SIy/ XMuKd0r1N5OKg0FEhjNjTof+2XWBGvakD3MNSQkljWh8dl4W6QWDgwhoSWwlYY/5idFjJU/B/8R9o Ktwq2jfv/kXNNvxM5Sby9NFOGqhKvY8w2Ds7cZW+gR8121CMan0V6SP+hLgemf1kBZAhXCcxtxdyQ I6dy2xqw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lRzOX-00A2fK-Dk; Thu, 01 Apr 2021 15:38:05 +0000 Received: from userp2120.oracle.com ([156.151.31.85]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lRzJy-00A22x-Kx for linux-arm-kernel@lists.infradead.org; Thu, 01 Apr 2021 15:33:47 +0000 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 131FOvvj070236; Thu, 1 Apr 2021 15:32:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=hiWxUtZ4UrasjC/KrUrvadeOLXqzQeinrezAY9cI9k8=; b=RVp+L3Pl1GJzyn56WKXlIEUemMFn9pcxTX/KCkCaueUNO37IOkev8aKXJZqK+pWNJ7iu DIWEcuT34YdaMNuW2t6ESaTUEWY50Z+MyjRAzcpj7+Bhok7lm4RHtl9DCH7CjW5lhbnI zvlT8vztGX0nOr8VSohJtfoHu/G06K9KAy0DGpBn7i0V2u3wdJuy3gLxpxrT6zGtWON9 Mr/TAQBiR3nmIIMrqPCY4E97+ClI5538yTAFyPJWlT23lOuvhbEuBoUTQ8E/D0QoZrfw cQNaG8MPxxenthQo7bEs8mfJpDWt2bsCS6MELVE2BpURRBu+ONEfR4TU0AfJbY2SyKT3 vQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 37n2a029c1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 01 Apr 2021 15:32:30 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 131FLCAR100856; Thu, 1 Apr 2021 15:32:28 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 37n2abdcx2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 01 Apr 2021 15:32:28 +0000 Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 131FW9rT031849; Thu, 1 Apr 2021 15:32:10 GMT Received: from neelam.us.oracle.com (/10.152.128.16) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2021 08:32:09 -0700 From: Alex Kogan To: linux@armlinux.org.uk, peterz@infradead.org, mingo@redhat.com, will.deacon@arm.com, arnd@arndb.de, longman@redhat.com, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de, hpa@zytor.com, x86@kernel.org, guohanjun@huawei.com, jglauber@marvell.com Cc: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, alex.kogan@oracle.com, dave.dice@oracle.com Subject: [PATCH v14 0/6] Add NUMA-awareness to qspinlock Date: Thu, 1 Apr 2021 11:31:50 -0400 Message-Id: <20210401153156.1165900-1-alex.kogan@oracle.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Proofpoint-IMR: 1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9941 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 bulkscore=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103310000 definitions=main-2104010104 X-Proofpoint-GUID: hT3XKPPqpBvko5QewSulvlWXsfxwBalB X-Proofpoint-ORIG-GUID: hT3XKPPqpBvko5QewSulvlWXsfxwBalB X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9941 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 bulkscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103310000 definitions=main-2104010104 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210401_163323_994089_644DC110 X-CRM114-Status: GOOD ( 28.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Q2hhbmdlIGZyb20gdjEzOgotLS0tLS0tLS0tLS0tLS0tCgpGaXggcmVncmVzc2lvbiBpbiBzdHJl c3MtbmcuClJlcG9ydGVkLWJ5OiBrZXJuZWwgdGVzdCByb2JvdCA8b2xpdmVyLnNhbmdAaW50ZWwu Y29tPgoKVGhlIGZpeCBpcyB0byBtb3ZlIGNvbW1vbi1jYXNlIHN0b3JlcyBpbnRvIHRoZSAncGFy dGlhbF9vcmRlcicgZmllbGQKb2YgdGhlIHF1ZXVlIG5vZGUgZnJvbSBjbmFfd2FpdF9oZWFkX29y X2xvY2soKSAod2hlcmUgdGhvc2Ugc3RvcmVzCmNhbiBjYXVzZSBhIGNhY2hlIG1pc3MgZHVyaW5n IGxvY2sgaGFuZG92ZXIpIHRvIGNuYV9pbml0X25vZGUoKS4KClN1bW1hcnkKLS0tLS0tLQoKTG9j ayB0aHJvdWdocHV0IGNhbiBiZSBpbmNyZWFzZWQgYnkgaGFuZGluZyBhIGxvY2sgdG8gYSB3YWl0 ZXIgb24gdGhlCnNhbWUgTlVNQSBub2RlIGFzIHRoZSBsb2NrIGhvbGRlciwgcHJvdmlkZWQgY2Fy ZSBpcyB0YWtlbiB0byBhdm9pZApzdGFydmF0aW9uIG9mIHdhaXRlcnMgb24gb3RoZXIgTlVNQSBu b2Rlcy4gVGhpcyBwYXRjaCBpbnRyb2R1Y2VzIENOQQooY29tcGFjdCBOVU1BLWF3YXJlIGxvY2sp IGFzIHRoZSBzbG93IHBhdGggZm9yIHFzcGlubG9jay4gSXQgaXMKZW5hYmxlZCB0aHJvdWdoIGEg Y29uZmlndXJhdGlvbiBvcHRpb24gKE5VTUFfQVdBUkVfU1BJTkxPQ0tTKS4KCkNOQSBpcyBhIE5V TUEtYXdhcmUgdmVyc2lvbiBvZiB0aGUgTUNTIGxvY2suIFNwaW5uaW5nIHRocmVhZHMgYXJlCm9y Z2FuaXplZCBpbiB0d28gcXVldWVzLCBhIHByaW1hcnkgcXVldWUgZm9yIHRocmVhZHMgcnVubmlu ZyBvbiB0aGUgc2FtZQpub2RlIGFzIHRoZSBjdXJyZW50IGxvY2sgaG9sZGVyLCBhbmQgYSBzZWNv bmRhcnkgcXVldWUgZm9yIHRocmVhZHMKcnVubmluZyBvbiBvdGhlciBub2Rlcy4gVGhyZWFkcyBz dG9yZSB0aGUgSUQgb2YgdGhlIG5vZGUgb24gd2hpY2gKdGhleSBhcmUgcnVubmluZyBpbiB0aGVp ciBxdWV1ZSBub2Rlcy4gQWZ0ZXIgYWNxdWlyaW5nIHRoZSBNQ1MgbG9jayBhbmQKYmVmb3JlIGFj cXVpcmluZyB0aGUgc3BpbmxvY2ssIHRoZSBNQ1MgbG9jayBob2xkZXIgY2hlY2tzIHdoZXRoZXIg dGhlIG5leHQKd2FpdGVyIGluIHRoZSBwcmltYXJ5IHF1ZXVlIChpZiBleGlzdHMpIGlzIHJ1bm5p bmcgb24gdGhlIHNhbWUgTlVNQSBub2RlLgpJZiBpdCBpcyBub3QsIHRoYXQgd2FpdGVyIGlzIGRl dGFjaGVkIGZyb20gdGhlIG1haW4gcXVldWUgYW5kIG1vdmVkIGludG8KdGhlIHRhaWwgb2YgdGhl IHNlY29uZGFyeSBxdWV1ZS4gVGhpcyB3YXksIHdlIGdyYWR1YWxseSBmaWx0ZXIgdGhlIHByaW1h cnkKcXVldWUsIGxlYXZpbmcgb25seSB3YWl0ZXJzIHJ1bm5pbmcgb24gdGhlIHNhbWUgcHJlZmVy cmVkIE5VTUEgbm9kZS4gTm90ZQp0aGF0IGNlcnRhaW4gcHJpb3J0aXplZCB3YWl0ZXJzIChlLmcu LCBpbiBpcnEgYW5kIG5taSBjb250ZXh0cykgYXJlCmV4Y2x1ZGVkIGZyb20gYmVpbmcgbW92ZWQg dG8gdGhlIHNlY29uZGFyeSBxdWV1ZS4gV2UgY2hhbmdlIHRoZSBOVU1BIG5vZGUKcHJlZmVyZW5j ZSBhZnRlciBhIHdhaXRlciBhdCB0aGUgaGVhZCBvZiB0aGUgc2Vjb25kYXJ5IHF1ZXVlIHNwaW5z IGZvciBhCmNlcnRhaW4gYW1vdW50IG9mIHRpbWUuIFdlIGRvIHRoYXQgYnkgZmx1c2hpbmcgdGhl IHNlY29uZGFyeSBxdWV1ZSBpbnRvCnRoZSBoZWFkIG9mIHRoZSBwcmltYXJ5IHF1ZXVlLCBlZmZl Y3RpdmVseSBjaGFuZ2luZyB0aGUgcHJlZmVyZW5jZSB0byB0aGUKTlVNQSBub2RlIG9mIHRoZSB3 YWl0ZXIgYXQgdGhlIGhlYWQgb2YgdGhlIHNlY29uZGFyeSBxdWV1ZSBhdCB0aGUgdGltZSBvZgp0 aGUgZmx1c2guCgpNb3JlIGRldGFpbHMgYXJlIGF2YWlsYWJsZSBhdCBodHRwczovL2FyeGl2Lm9y Zy9hYnMvMTgxMC4wNTYwMC4KCldlIGhhdmUgZG9uZSBzb21lIHBlcmZvcm1hbmNlIGV2YWx1YXRp b24gd2l0aCB0aGUgbG9ja3RvcnR1cmUgbW9kdWxlCmFzIHdlbGwgYXMgd2l0aCBzZXZlcmFsIGJl bmNobWFya3MgZnJvbSB0aGUgd2lsbC1pdC1zY2FsZSByZXBvLgpUaGUgZm9sbG93aW5nIGxvY2t0 b3J0dXJlIHJlc3VsdHMgYXJlIGZyb20gYW4gT3JhY2xlIFg1LTQgc2VydmVyCihmb3VyIEludGVs IFhlb24gRTctODg5NSB2MyBAIDIuNjBHSHogc29ja2V0cyB3aXRoIDE4IGh5cGVydGhyZWFkZWQK Y29yZXMgZWFjaCkuIEVhY2ggbnVtYmVyIHJlcHJlc2VudHMgYW4gYXZlcmFnZSAob3ZlciAyNSBy dW5zKSBvZiB0aGUKdG90YWwgbnVtYmVyIG9mIG9wcyAoeDEwXjcpIHJlcG9ydGVkIGF0IHRoZSBl bmQgb2YgZWFjaCBydW4uIFRoZSAKc3RhbmRhcmQgZGV2aWF0aW9uIGlzIGFsc28gcmVwb3J0ZWQg aW4gKCksIGFuZCBpbiBnZW5lcmFsIGlzIGFib3V0IDMlCmZyb20gdGhlIGF2ZXJhZ2UuIFRoZSAn c3RvY2snIGtlcm5lbCBpcyB2NS4xMi4wLXJjNSwKY29tbWl0IDk5ZDRlOGI0ZTYwOWIsIGNvbXBp bGVkIGluIHRoZSBkZWZhdWx0IGNvbmZpZ3VyYXRpb24uIAonQ05BJyBpcyB0aGUgbW9kaWZpZWQg a2VybmVsIHdpdGggTlVNQV9BV0FSRV9TUElOTE9DS1Mgc2V0OwpUaGUgc3BlZWR1cCBpcyBjYWxj dWxhdGVkIGJ5IGRpdmlkaW5nIHRoZSByZXN1bHQgb2YgJ0NOQScgYnkgdGhlIHJlc3VsdAogYWNo aWV2ZWQgd2l0aCAnc3RvY2snLgoKI3RociAgCSBzdG9jayAgICAgQ05BIC8gc3BlZWR1cCAgICAg CiAgMSAgMi42NzUgKDAuMDg4KSAyLjcwNyAoMC4xMDApIC8gMS4wMTIKICAyICAyLjc4MSAoMC4x NzIpIDIuNzkwICgwLjE0OCkgLyAxLjAwMwogIDQgIDQuMzE0ICgwLjExNSkgNC40MDMgKDAuMjA1 KSAvIDEuMDIxCiAgOCAgNS4wODUgKDAuMTQwKSA3LjI5MiAoMC4yMTApIC8gMS40MzQKIDE2ICA1 Ljg0NiAoMC4xMTQpIDkuMzQwICgwLjE4NikgLyAxLjU5OAogMzIgIDYuMjgyICgwLjEyNykgMTAu MDc0ICgwLjE5NykgLyAxLjYwMwogMzYgIDYuMzgyICgwLjE0NCkgMTAuMjQzICgwLjE5OCkgLyAx LjYwNQogNzIgIDYuMTc1ICgwLjA4NikgMTAuNDgyICgwLjE3OSkgLyAxLjY5OAoxMDggIDYuMDM3 ICgwLjA3MykgMTAuMzU0ICgwLjE2NCkgLyAxLjcxNQoxNDIgIDUuNzk2ICgwLjA3MikgMTAuMjc1 ICgwLjE5MykgLyAxLjc3MwoKVGhlIGZvbGxvd2luZyB0YWJsZXMgY29udGFpbiB0aHJvdWdocHV0 IHJlc3VsdHMgKG9wcy91cykgZnJvbSB0aGUgc2FtZQpzZXR1cCBmb3Igd2lsbC1pdC1zY2FsZS9v cGVuMV90aHJlYWRzOiAKCiN0aHIgIAkgc3RvY2sgICAgIENOQSAvIHNwZWVkdXAKICAxICAwLjUx NSAoMC4wMDIpIDAuNTE0ICgwLjAwMSkgLyAwLjk5OQogIDIgIDAuNzc0ICgwLjAxOCkgMC43ODAg KDAuMDE5KSAvIDEuMDA5CiAgNCAgMS40MDggKDAuMDMwKSAxLjQyMSAoMC4wMzQpIC8gMS4wMDkK ICA4ICAxLjc4NSAoMC4wODIpIDEuNzI0ICgwLjExNSkgLyAwLjk2NQogMTYgIDEuODc4ICgwLjEx OCkgMS43NzUgKDAuMTE0KSAvIDAuOTQ1CiAzMiAgMC45MTQgKDAuMDcyKSAxLjc1NSAoMC4xMDUp IC8gMS45MjAKIDM2ICAwLjg3NyAoMC4wNjQpIDEuNzM4ICgwLjEwMCkgLyAxLjk4MQogNzIgIDAu ODEwICgwLjA0NykgMS42NTggKDAuMDg2KSAvIDIuMDQ3CjEwOCAgMC44MzUgKDAuMDM2KSAxLjc1 NiAoMC4wOTEpIC8gMi4xMDMKMTQyICAwLjgwOSAoMC4wMzgpIDEuNzY2ICgwLjA2OSkgLyAyLjE4 NAoKYW5kIHdpbGwtaXQtc2NhbGUvbG9jazJfdGhyZWFkczoKCiN0aHIgIAkgc3RvY2sgICAgIENO QSAvIHNwZWVkdXAKICAxICAxLjYwNSAoMC4wMDMpIDEuNjA2ICgwLjAwNSkgLyAxLjAwMAogIDIg IDIuODE0ICgwLjA3NykgMi43OTMgKDAuMDY4KSAvIDAuOTkyCiAgNCAgNS4zMTggKDAuMzUyKSA1 LjQ2MyAoMC4zNTcpIC8gMS4wMjcKICA4ICA0LjI0MSAoMC4yNjUpIDQuMDI1ICgwLjM2NikgLyAw Ljk0OQogMTYgIDQuMTI1ICgwLjEyNCkgMy45MTQgKDAuMTU5KSAvIDAuOTQ5CiAzMiAgMi40NTgg KDAuMTEzKSAzLjk1MCAoMC4xMzEpIC8gMS42MDcKIDM2ICAyLjQ1NyAoMC4wNzUpIDMuOTI0ICgw LjA4MCkgLyAxLjU5NwogNzIgIDEuODM2ICgwLjEwMikgMy45MDkgKDAuMTA0KSAvIDIuMTI5CjEw OCAgMS44NjggKDAuMDk4KSAzLjg3MSAoMC4xMDYpIC8gMi4wNzIKMTQyICAxLjc4MCAoMC4xMjQp IDMuODk0ICgwLjA4OSkgLyAyLjE4OAoKT3VyIGV2YWx1YXRpb24gc2hvd3MgdGhhdCBDTkEgYWxz byBpbXByb3ZlcyBwZXJmb3JtYW5jZSBvZiB1c2VyIAphcHBsaWNhdGlvbnMgdGhhdCBoYXZlIGhv dCBwdGhyZWFkIG11dGV4ZXMuIFRob3NlIG11dGV4ZXMgYXJlIApibG9ja2luZywgYW5kIHdhaXRp bmcgdGhyZWFkcyBwYXJrIGFuZCB1bnBhcmsgdmlhIHRoZSBmdXRleCAKbWVjaGFuaXNtIGluIHRo ZSBrZXJuZWwuIEdpdmVuIHRoYXQga2VybmVsIGZ1dGV4IGNoYWlucywgd2hpY2gKYXJlIGhhc2hl ZCBieSB0aGUgbXV0ZXggYWRkcmVzcywgYXJlIGVhY2ggcHJvdGVjdGVkIGJ5IGEgCmNoYWluLXNw ZWNpZmljIHNwaW4gbG9jaywgdGhlIGNvbnRlbnRpb24gb24gYSB1c2VyLW1vZGUgbXV0ZXggCnRy YW5zbGF0ZXMgaW50byBjb250ZW50aW9uIG9uIGEga2VybmVsIGxldmVsIHNwaW5sb2NrLiAKCkhl cmUgYXJlIHRoZSB0aHJvdWdocHV0IHJlc3VsdHMgKG9wcy91cykgZm9yIHRoZSBsZXZlbGRiIOKA mHJlYWRyYW5kb23igJkKYmVuY2htYXJrOgoKI3RociAgCSBzdG9jayAgICAgICAgIENOQSAvIHNw ZWVkdXAKICAxICAwLjQ5MyAoMC4wMzQpIDAuNTIwICgwLjAyNikgLyAxLjA1MwogIDIgIDAuNzk2 ICgwLjA0NikgMC44MzggKDAuMDMzKSAvIDEuMDUzCiAgNCAgMS4xNzIgKDAuMDc2KSAxLjIxMSAo MC4wNDMpIC8gMS4wMzMKICA4ICAxLjIxNSAoMC4wOTYpIDEuMjA3ICgwLjExNikgLyAwLjk5Mwog MTYgIDAuOTg2ICgwLjEzNikgMS4wNzAgKDAuMTI5KSAvIDEuMDg1CiAzMiAgMC43NTYgKDAuMDM2 KSAxLjE0OCAoMC4wMjEpIC8gMS41MTkKIDM2ICAwLjcwMSAoMC4wMzIpIDEuMTQ3ICgwLjAxNykg LyAxLjYzNgogNzIgIDAuNjI1ICgwLjAxNikgMS4xNDkgKDAuMDI0KSAvIDEuODM5CjEwOCAgMC42 MTIgKDAuMDE1KSAxLjE0NiAoMC4wMTQpIC8gMS44NzIKMTQyICAwLjYwNiAoMC4wMTMpIDEuMTMx ICgwLjAyNSkgLyAxLjg2NwoKRnVydGhlciBjb21tZW50cyBhcmUgd2VsY29tZSBhbmQgYXBwcmVj aWF0ZWQuCgpBbGV4IEtvZ2FuICg2KToKICBsb2NraW5nL3FzcGlubG9jazogUmVuYW1lIG1jcyBs b2NrL3VubG9jayBtYWNyb3MgYW5kIG1ha2UgdGhlbSBtb3JlCiAgICBnZW5lcmljCiAgbG9ja2lu Zy9xc3BpbmxvY2s6IFJlZmFjdG9yIHRoZSBxc3BpbmxvY2sgc2xvdyBwYXRoCiAgbG9ja2luZy9x c3BpbmxvY2s6IEludHJvZHVjZSBDTkEgaW50byB0aGUgc2xvdyBwYXRoIG9mIHFzcGlubG9jawog IGxvY2tpbmcvcXNwaW5sb2NrOiBJbnRyb2R1Y2Ugc3RhcnZhdGlvbiBhdm9pZGFuY2UgaW50byBD TkEKICBsb2NraW5nL3FzcGlubG9jazogQXZvaWQgbW92aW5nIGNlcnRhaW4gdGhyZWFkcyBiZXR3 ZWVuIHdhaXRpbmcgcXVldWVzCiAgICBpbiBDTkEKICBsb2NraW5nL3FzcGlubG9jazogSW50cm9k dWNlIHRoZSBzaHVmZmxlIHJlZHVjdGlvbiBvcHRpbWl6YXRpb24gaW50bwogICAgQ05BCgogLi4u L2FkbWluLWd1aWRlL2tlcm5lbC1wYXJhbWV0ZXJzLnR4dCAgICAgICAgIHwgIDE5ICsKIGFyY2gv YXJtL2luY2x1ZGUvYXNtL21jc19zcGlubG9jay5oICAgICAgICAgICB8ICAgNiArLQogYXJjaC94 ODYvS2NvbmZpZyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIHwgIDIwICsKIGFyY2gveDg2 L2luY2x1ZGUvYXNtL3FzcGlubG9jay5oICAgICAgICAgICAgICB8ICAgNCArCiBhcmNoL3g4Ni9r ZXJuZWwvYWx0ZXJuYXRpdmUuYyAgICAgICAgICAgICAgICAgfCAgIDQgKwogaW5jbHVkZS9hc20t Z2VuZXJpYy9tY3Nfc3BpbmxvY2suaCAgICAgICAgICAgIHwgICA0ICstCiBrZXJuZWwvbG9ja2lu Zy9tY3Nfc3BpbmxvY2suaCAgICAgICAgICAgICAgICAgfCAgMjAgKy0KIGtlcm5lbC9sb2NraW5n L3FzcGlubG9jay5jICAgICAgICAgICAgICAgICAgICB8ICA4MiArKystCiBrZXJuZWwvbG9ja2lu Zy9xc3BpbmxvY2tfY25hLmggICAgICAgICAgICAgICAgfCA0NTUgKysrKysrKysrKysrKysrKysr CiBrZXJuZWwvbG9ja2luZy9xc3BpbmxvY2tfcGFyYXZpcnQuaCAgICAgICAgICAgfCAgIDIgKy0K IDEwIGZpbGVzIGNoYW5nZWQsIDU5MyBpbnNlcnRpb25zKCspLCAyMyBkZWxldGlvbnMoLSkKIGNy ZWF0ZSBtb2RlIDEwMDY0NCBrZXJuZWwvbG9ja2luZy9xc3BpbmxvY2tfY25hLmgKCi0tIAoyLjI0 LjMgKEFwcGxlIEdpdC0xMjgpCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWlsaW5nIGxpc3QKbGludXgtYXJtLWtlcm5l bEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0cy5pbmZyYWRlYWQub3JnL21haWxtYW4v bGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNPARSEABLE_RELAY, URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB45FC43461 for ; Thu, 1 Apr 2021 18:01:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AA6E1611C9 for ; Thu, 1 Apr 2021 18:01:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235705AbhDASBy (ORCPT ); Thu, 1 Apr 2021 14:01:54 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:50336 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235204AbhDARq7 (ORCPT ); Thu, 1 Apr 2021 13:46:59 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 131FOvvj070236; Thu, 1 Apr 2021 15:32:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : mime-version : content-type : content-transfer-encoding; s=corp-2020-01-29; bh=hiWxUtZ4UrasjC/KrUrvadeOLXqzQeinrezAY9cI9k8=; b=RVp+L3Pl1GJzyn56WKXlIEUemMFn9pcxTX/KCkCaueUNO37IOkev8aKXJZqK+pWNJ7iu DIWEcuT34YdaMNuW2t6ESaTUEWY50Z+MyjRAzcpj7+Bhok7lm4RHtl9DCH7CjW5lhbnI zvlT8vztGX0nOr8VSohJtfoHu/G06K9KAy0DGpBn7i0V2u3wdJuy3gLxpxrT6zGtWON9 Mr/TAQBiR3nmIIMrqPCY4E97+ClI5538yTAFyPJWlT23lOuvhbEuBoUTQ8E/D0QoZrfw cQNaG8MPxxenthQo7bEs8mfJpDWt2bsCS6MELVE2BpURRBu+ONEfR4TU0AfJbY2SyKT3 vQ== Received: from aserp3020.oracle.com (aserp3020.oracle.com [141.146.126.70]) by userp2120.oracle.com with ESMTP id 37n2a029c1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 01 Apr 2021 15:32:30 +0000 Received: from pps.filterd (aserp3020.oracle.com [127.0.0.1]) by aserp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 131FLCAR100856; Thu, 1 Apr 2021 15:32:28 GMT Received: from userv0122.oracle.com (userv0122.oracle.com [156.151.31.75]) by aserp3020.oracle.com with ESMTP id 37n2abdcx2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 01 Apr 2021 15:32:28 +0000 Received: from abhmp0017.oracle.com (abhmp0017.oracle.com [141.146.116.23]) by userv0122.oracle.com (8.14.4/8.14.4) with ESMTP id 131FW9rT031849; Thu, 1 Apr 2021 15:32:10 GMT Received: from neelam.us.oracle.com (/10.152.128.16) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Thu, 01 Apr 2021 08:32:09 -0700 From: Alex Kogan To: linux@armlinux.org.uk, peterz@infradead.org, mingo@redhat.com, will.deacon@arm.com, arnd@arndb.de, longman@redhat.com, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de, hpa@zytor.com, x86@kernel.org, guohanjun@huawei.com, jglauber@marvell.com Cc: steven.sistare@oracle.com, daniel.m.jordan@oracle.com, alex.kogan@oracle.com, dave.dice@oracle.com Subject: [PATCH v14 0/6] Add NUMA-awareness to qspinlock Date: Thu, 1 Apr 2021 11:31:50 -0400 Message-Id: <20210401153156.1165900-1-alex.kogan@oracle.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Proofpoint-IMR: 1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9941 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 bulkscore=0 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103310000 definitions=main-2104010104 X-Proofpoint-GUID: hT3XKPPqpBvko5QewSulvlWXsfxwBalB X-Proofpoint-ORIG-GUID: hT3XKPPqpBvko5QewSulvlWXsfxwBalB X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9941 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 impostorscore=0 adultscore=0 clxscore=1015 mlxlogscore=999 phishscore=0 bulkscore=0 priorityscore=1501 spamscore=0 malwarescore=0 mlxscore=0 lowpriorityscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2103310000 definitions=main-2104010104 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Change from v13: ---------------- Fix regression in stress-ng. Reported-by: kernel test robot The fix is to move common-case stores into the 'partial_order' field of the queue node from cna_wait_head_or_lock() (where those stores can cause a cache miss during lock handover) to cna_init_node(). Summary ------- Lock throughput can be increased by handing a lock to a waiter on the same NUMA node as the lock holder, provided care is taken to avoid starvation of waiters on other NUMA nodes. This patch introduces CNA (compact NUMA-aware lock) as the slow path for qspinlock. It is enabled through a configuration option (NUMA_AWARE_SPINLOCKS). CNA is a NUMA-aware version of the MCS lock. Spinning threads are organized in two queues, a primary queue for threads running on the same node as the current lock holder, and a secondary queue for threads running on other nodes. Threads store the ID of the node on which they are running in their queue nodes. After acquiring the MCS lock and before acquiring the spinlock, the MCS lock holder checks whether the next waiter in the primary queue (if exists) is running on the same NUMA node. If it is not, that waiter is detached from the main queue and moved into the tail of the secondary queue. This way, we gradually filter the primary queue, leaving only waiters running on the same preferred NUMA node. Note that certain priortized waiters (e.g., in irq and nmi contexts) are excluded from being moved to the secondary queue. We change the NUMA node preference after a waiter at the head of the secondary queue spins for a certain amount of time. We do that by flushing the secondary queue into the head of the primary queue, effectively changing the preference to the NUMA node of the waiter at the head of the secondary queue at the time of the flush. More details are available at https://arxiv.org/abs/1810.05600. We have done some performance evaluation with the locktorture module as well as with several benchmarks from the will-it-scale repo. The following locktorture results are from an Oracle X5-4 server (four Intel Xeon E7-8895 v3 @ 2.60GHz sockets with 18 hyperthreaded cores each). Each number represents an average (over 25 runs) of the total number of ops (x10^7) reported at the end of each run. The standard deviation is also reported in (), and in general is about 3% from the average. The 'stock' kernel is v5.12.0-rc5, commit 99d4e8b4e609b, compiled in the default configuration. 'CNA' is the modified kernel with NUMA_AWARE_SPINLOCKS set; The speedup is calculated by dividing the result of 'CNA' by the result achieved with 'stock'. #thr stock CNA / speedup 1 2.675 (0.088) 2.707 (0.100) / 1.012 2 2.781 (0.172) 2.790 (0.148) / 1.003 4 4.314 (0.115) 4.403 (0.205) / 1.021 8 5.085 (0.140) 7.292 (0.210) / 1.434 16 5.846 (0.114) 9.340 (0.186) / 1.598 32 6.282 (0.127) 10.074 (0.197) / 1.603 36 6.382 (0.144) 10.243 (0.198) / 1.605 72 6.175 (0.086) 10.482 (0.179) / 1.698 108 6.037 (0.073) 10.354 (0.164) / 1.715 142 5.796 (0.072) 10.275 (0.193) / 1.773 The following tables contain throughput results (ops/us) from the same setup for will-it-scale/open1_threads: #thr stock CNA / speedup 1 0.515 (0.002) 0.514 (0.001) / 0.999 2 0.774 (0.018) 0.780 (0.019) / 1.009 4 1.408 (0.030) 1.421 (0.034) / 1.009 8 1.785 (0.082) 1.724 (0.115) / 0.965 16 1.878 (0.118) 1.775 (0.114) / 0.945 32 0.914 (0.072) 1.755 (0.105) / 1.920 36 0.877 (0.064) 1.738 (0.100) / 1.981 72 0.810 (0.047) 1.658 (0.086) / 2.047 108 0.835 (0.036) 1.756 (0.091) / 2.103 142 0.809 (0.038) 1.766 (0.069) / 2.184 and will-it-scale/lock2_threads: #thr stock CNA / speedup 1 1.605 (0.003) 1.606 (0.005) / 1.000 2 2.814 (0.077) 2.793 (0.068) / 0.992 4 5.318 (0.352) 5.463 (0.357) / 1.027 8 4.241 (0.265) 4.025 (0.366) / 0.949 16 4.125 (0.124) 3.914 (0.159) / 0.949 32 2.458 (0.113) 3.950 (0.131) / 1.607 36 2.457 (0.075) 3.924 (0.080) / 1.597 72 1.836 (0.102) 3.909 (0.104) / 2.129 108 1.868 (0.098) 3.871 (0.106) / 2.072 142 1.780 (0.124) 3.894 (0.089) / 2.188 Our evaluation shows that CNA also improves performance of user applications that have hot pthread mutexes. Those mutexes are blocking, and waiting threads park and unpark via the futex mechanism in the kernel. Given that kernel futex chains, which are hashed by the mutex address, are each protected by a chain-specific spin lock, the contention on a user-mode mutex translates into contention on a kernel level spinlock. Here are the throughput results (ops/us) for the leveldb ‘readrandom’ benchmark: #thr stock CNA / speedup 1 0.493 (0.034) 0.520 (0.026) / 1.053 2 0.796 (0.046) 0.838 (0.033) / 1.053 4 1.172 (0.076) 1.211 (0.043) / 1.033 8 1.215 (0.096) 1.207 (0.116) / 0.993 16 0.986 (0.136) 1.070 (0.129) / 1.085 32 0.756 (0.036) 1.148 (0.021) / 1.519 36 0.701 (0.032) 1.147 (0.017) / 1.636 72 0.625 (0.016) 1.149 (0.024) / 1.839 108 0.612 (0.015) 1.146 (0.014) / 1.872 142 0.606 (0.013) 1.131 (0.025) / 1.867 Further comments are welcome and appreciated. Alex Kogan (6): locking/qspinlock: Rename mcs lock/unlock macros and make them more generic locking/qspinlock: Refactor the qspinlock slow path locking/qspinlock: Introduce CNA into the slow path of qspinlock locking/qspinlock: Introduce starvation avoidance into CNA locking/qspinlock: Avoid moving certain threads between waiting queues in CNA locking/qspinlock: Introduce the shuffle reduction optimization into CNA .../admin-guide/kernel-parameters.txt | 19 + arch/arm/include/asm/mcs_spinlock.h | 6 +- arch/x86/Kconfig | 20 + arch/x86/include/asm/qspinlock.h | 4 + arch/x86/kernel/alternative.c | 4 + include/asm-generic/mcs_spinlock.h | 4 +- kernel/locking/mcs_spinlock.h | 20 +- kernel/locking/qspinlock.c | 82 +++- kernel/locking/qspinlock_cna.h | 455 ++++++++++++++++++ kernel/locking/qspinlock_paravirt.h | 2 +- 10 files changed, 593 insertions(+), 23 deletions(-) create mode 100644 kernel/locking/qspinlock_cna.h -- 2.24.3 (Apple Git-128)