From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7EE4BC433ED for ; Wed, 14 Apr 2021 02:30:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 57DDA60E09 for ; Wed, 14 Apr 2021 02:30:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232596AbhDNCac (ORCPT ); Tue, 13 Apr 2021 22:30:32 -0400 Received: from aserp2120.oracle.com ([141.146.126.78]:45184 "EHLO aserp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229834AbhDNCa0 (ORCPT ); Tue, 13 Apr 2021 22:30:26 -0400 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13E2PNLc024819; Wed, 14 Apr 2021 02:29:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=corp-2020-01-29; bh=KkLLCrZWDiKYSnLJLqsQyCX146lmHXS5CDO9gPAfauE=; b=BLki9NRwAEb5LBoAFKfS/alCw09RheMbN87a9Jh3xQr1+9/yhGOSrUVTdviYP7SP9Gto 2yOc6OVJzooeJcDKdR4t2QvJ1m5ZJca8FcmIiG6FY+c3wgtOppHzmy978xnIISOrQTWh OjyFqw+fvfHodR/9evFhUAiEQQJKMHTH+g+TAhfSG6KZco6CTQAyIbzpvsBZgF2aofdw y34We7IENQ+XDRFJhyEcmSJ1Y/UyF+8n65PpgS1SEt8BGSmlLbPkMjmmtjYnGv63rvnR CF/Q7lbOO7NxQF4kmNueriZsjy3YXkAIq7iPcBhQRI0rCnJXkPrK5S25qrN4Du71wnot RQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 37u3ymguxh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Apr 2021 02:29:29 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13E2PWM9055502; Wed, 14 Apr 2021 02:29:28 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3020.oracle.com with ESMTP id 37unst8n1n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Apr 2021 02:29:28 +0000 Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 13E2T9N1027598; Wed, 14 Apr 2021 02:29:09 GMT Received: from [10.39.235.234] (/10.39.235.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 14 Apr 2021 02:29:08 +0000 Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: [External] : Re: [PATCH v14 3/6] locking/qspinlock: Introduce CNA into the slow path of qspinlock From: Alex Kogan In-Reply-To: Date: Tue, 13 Apr 2021 22:29:07 -0400 Cc: linux@armlinux.org.uk, Ingo Molnar , Will Deacon , Arnd Bergmann , longman@redhat.com, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de, hpa@zytor.com, x86@kernel.org, guohanjun@huawei.com, jglauber@marvell.com, steven.sistare@oracle.com, daniel.m.jordan@oracle.com, dave.dice@oracle.com Content-Transfer-Encoding: quoted-printable Message-Id: References: <20210401153156.1165900-1-alex.kogan@oracle.com> <20210401153156.1165900-4-alex.kogan@oracle.com> To: Peter Zijlstra X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Proofpoint-IMR: 1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9953 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 suspectscore=0 bulkscore=0 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104140016 X-Proofpoint-GUID: ABqW3SS4571hF5OwUz4Z393S9cZYSXFB X-Proofpoint-ORIG-GUID: ABqW3SS4571hF5OwUz4Z393S9cZYSXFB X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9953 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 spamscore=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 bulkscore=0 phishscore=0 suspectscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104140016 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Peter, thanks for all the comments and suggestions! > On Apr 13, 2021, at 7:30 AM, Peter Zijlstra = wrote: >=20 > On Thu, Apr 01, 2021 at 11:31:53AM -0400, Alex Kogan wrote: >=20 >> +/* >> + * cna_splice_tail -- splice the next node from the primary queue = onto >> + * the secondary queue. >> + */ >> +static void cna_splice_next(struct mcs_spinlock *node, >> + struct mcs_spinlock *next, >> + struct mcs_spinlock *nnext) >=20 > You forgot to update the comment when you changed the name on this > thing. Good catch, thanks. >=20 >> +/* >> + * cna_order_queue - check whether the next waiter in the main queue = is on >> + * the same NUMA node as the lock holder; if not, and it has a = waiter behind >> + * it in the main queue, move the former onto the secondary queue. >> + */ >> +static void cna_order_queue(struct mcs_spinlock *node) >> +{ >> + struct mcs_spinlock *next =3D READ_ONCE(node->next); >> + struct cna_node *cn =3D (struct cna_node *)node; >> + int numa_node, next_numa_node; >> + >> + if (!next) { >> + cn->partial_order =3D LOCAL_WAITER_NOT_FOUND; >> + return; >> + } >> + >> + numa_node =3D cn->numa_node; >> + next_numa_node =3D ((struct cna_node *)next)->numa_node; >> + >> + if (next_numa_node !=3D numa_node) { >> + struct mcs_spinlock *nnext =3D READ_ONCE(next->next); >> + >> + if (nnext) { >> + cna_splice_next(node, next, nnext); >> + next =3D nnext; >> + } >> + /* >> + * Inherit NUMA node id of primary queue, to maintain = the >> + * preference even if the next waiter is on a different = node. >> + */ >> + ((struct cna_node *)next)->numa_node =3D numa_node; >> + } >> +} >=20 > So the obvious change since last time I looked a this is that it now > only looks 1 entry ahead. Which makes sense I suppose. This is in response to the critique that the worst-case time complexity = of cna_order_queue() was O(n). With this change, the complexity is = constant. >=20 > I'm not really a fan of the 'partial_order' name combined with that > silly enum { LOCAL_WAITER_FOUND, LOCAL_WAITER_NOT_FOUND }. That's just > really bad naming all around. The enum is about having a waiter while > the variable is about partial order, that doesn't match at all. Fair enough. > If you rename the variable to 'has_waiter' and simply use 0,1 values, > things would be ever so more readable. But I don't think that makes > sense, see below. >=20 > I'm also not sure about that whole numa_node thing, why would you > over-write the numa node, why at this point ? With this move-one-by-one approach, I want to keep the NUMA-node=20 preference of the lock holder even if the next-next waiter is on a = different NUMA-node. Otherwise, we will end up switching preference often and the entire scheme would not perform well. In particular, we might easily end up with threads from the preferred node waiting in the secondary = queue. >=20 >> + >> +/* Abuse the pv_wait_head_or_lock() hook to get some work done */ >> +static __always_inline u32 cna_wait_head_or_lock(struct qspinlock = *lock, >> + struct mcs_spinlock = *node) >> +{ >> + /* >> + * Try and put the time otherwise spent spin waiting on >> + * _Q_LOCKED_PENDING_MASK to use by sorting our lists. >> + */ >> + cna_order_queue(node); >> + >> + return 0; /* we lied; we didn't wait, go do so now */ >=20 > So here we inspect one entry ahead and then quit. I can't rmember, but > did we try something like: >=20 > /* > * Try and put the time otherwise spent spin waiting on > * _Q_LOCKED_PENDING_MASK to use by sorting our lists. > * Move one entry at a go until either the list is fully > * sorted or we ran out of spin condition. > */ > while (READ_ONCE(lock->val) & _Q_LOCKED_PENDING_MASK && > node->partial_order) > cna_order_queue(node); >=20 > return 0; >=20 > This will keep moving @next to the remote list until such a time that > we're forced to continue or @next is local. We have not tried that. This is actually an interesting idea, with its = pros and cons. That is, we are likely to filter out =E2=80=9Cnon-preferred=E2=80=9D = waiters into the secondary queue faster, but also we are more likely to run into a situation where the = lock becomes available at the time we are running the cna_order_queue() logic, thus = prolonging the handover time. With this change, however, there is no need to call cna_order_queue() = again=20 in cna_lock_handoff(), which is another advantage I guess. All in all, I am fine switching to this alternative if you like it more. BTW, we can return node->partial_order instead of 0, to skip the call to atomic_cond_read_acquire() in the general code. >=20 >> +} >> + >> +static inline void cna_lock_handoff(struct mcs_spinlock *node, >> + struct mcs_spinlock *next) >> +{ >> + struct cna_node *cn =3D (struct cna_node *)node; >> + u32 val =3D 1; >> + >> + u32 partial_order =3D cn->partial_order; >> + >> + if (partial_order =3D=3D LOCAL_WAITER_NOT_FOUND) >> + cna_order_queue(node); >> + >=20 > AFAICT this is where playing silly games with ->numa_node belong; but > right along with that goes a comment that describes why any of that > makes sense. >=20 > I mean, if you leave your node, for any reason, why bother coming back > to it, why not accept it is a sign of the gods you're overdue for a > node-change? I think there is some misunderstanding here =E2=80=94 let me try to = clarify. In the current logic, we first call cna_order_queue() from = cna_wait_head_or_lock(). If we find an appropriate =E2=80=9Clocal=E2=80=9D waiter (either real or = fake), we set it as next and make sure the numa_node of that node is the same as ours. If not (i.e., when we do not have any next waiter), we set partial_order = to LOCAL_WAITER_NOT_FOUND (pardon the names) and go spin on the lock = (calling atomic_cond_read_acquire() in the generic code). During this spin time, new waiters can join the queue. Hence, we recheck the state of the queue by calling cna_order_queue() = again from cna_lock_handoff(). As just mentioned, if we change the logic as you suggest, there is really no reason to call cna_order_queue() again. >=20 > Was the efficacy of this scheme tested? >=20 >> + /* >> + * We have a local waiter, either real or fake one; >> + * reload @next in case it was changed by cna_order_queue(). >> + */ >> + next =3D node->next; >> + if (node->locked > 1) >> + val =3D node->locked; /* preseve secondary queue */ >=20 > IIRC we used to do: >=20 > val |=3D node->locked; >=20 > Which is simpler for not having branches. Why change a good thing? With |=3D we might set val to encoded tail+1 (rather than encoded tail). This still would work, as decode_tail(val) does not care about LSB, but = seems fragile to me. We talked about that before:=20 = https://lore.kernel.org/linux-arm-kernel/E32F90E2-F00B-423B-A851-336004EF6= 593@oracle.com If you think this is a non-issue, I will be happy to adopt your = suggestion. Regards, =E2=80=94 Alex From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-3.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31E3CC433B4 for ; Wed, 14 Apr 2021 02:32:00 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 80EF160FF0 for ; Wed, 14 Apr 2021 02:31:59 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 80EF160FF0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=oracle.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:References:Message-Id:Cc:Date:In-Reply-To:From: Subject:Mime-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=dHBil7DPDaMDm5+h8X63l4imxd6yHtnh7wlJHYr0G7o=; b=V2mOD/o6UpYgMEeCqGZjb/oCR mDC+p5gxMe66tV+QogUurVQQ4Ba9H3az4R+/a9RNC+5nw57hgndmuE7MfvYJ/Ob7Ie5teJSJzDaWY UCw/LJc8jLkRrwwZNVB58s73v88BvjVKhA7QXdGkIyK0kbzFoirhvrMdSwq+DdQgm3/nZRgoriv6k aykJzp6I4usjlIBooLXDyoI876eOwL+S8/DiJFNQscl4RaGbK8JUKg8MBy7M2Ernkl67NnpaMFjpw EnYfIqS9dWvDDS2Nt5ZSwD521fuJq+f9k/gTEcyGu66fDzBmZzSR9kTY3I8ooS6NfPK5TCAGueiT7 Rj9yZ6z2g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lWVI9-00BBEv-QH; Wed, 14 Apr 2021 02:30:10 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWVI3-00BBDy-0m for linux-arm-kernel@desiato.infradead.org; Wed, 14 Apr 2021 02:30:03 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=To:References:Message-Id: Content-Transfer-Encoding:Cc:Date:In-Reply-To:From:Subject:Mime-Version: Content-Type:Sender:Reply-To:Content-ID:Content-Description; bh=KkLLCrZWDiKYSnLJLqsQyCX146lmHXS5CDO9gPAfauE=; b=LzXYnJANuhsfVUCmKO1EAalDbx 4EduPrfIlOPRJzWd/ENGk4louWfxd4WFBUZV220lj+TNjizokUkZJVlTjl7yLrhvLDESzRXzqIYCB wMAGh0EP8K0aEyfADj17trVtsy9kp+nJEJ1H9PfPrKMryXs/Um1GTFtboRy93t1gJV+/441YZMT7L XTxHDPDG+735D++CPiGDFCm0nIK80Kttvhj4NeBcrRc3KbVmEH3MwNUCOWO819e5cZ+XP2RJKJtr5 PpWMvOGmcN63PDPI3gVeO2q8FsV0s+fg+C/J8tk71U2vP/keqoESNNBckqJGbz87waL8fgKcpYUIP LhFW6AqQ==; Received: from aserp2120.oracle.com ([141.146.126.78]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lWVHz-007RWb-D4 for linux-arm-kernel@lists.infradead.org; Wed, 14 Apr 2021 02:30:01 +0000 Received: from pps.filterd (aserp2120.oracle.com [127.0.0.1]) by aserp2120.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13E2PNLc024819; Wed, 14 Apr 2021 02:29:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=content-type : mime-version : subject : from : in-reply-to : date : cc : content-transfer-encoding : message-id : references : to; s=corp-2020-01-29; bh=KkLLCrZWDiKYSnLJLqsQyCX146lmHXS5CDO9gPAfauE=; b=BLki9NRwAEb5LBoAFKfS/alCw09RheMbN87a9Jh3xQr1+9/yhGOSrUVTdviYP7SP9Gto 2yOc6OVJzooeJcDKdR4t2QvJ1m5ZJca8FcmIiG6FY+c3wgtOppHzmy978xnIISOrQTWh OjyFqw+fvfHodR/9evFhUAiEQQJKMHTH+g+TAhfSG6KZco6CTQAyIbzpvsBZgF2aofdw y34We7IENQ+XDRFJhyEcmSJ1Y/UyF+8n65PpgS1SEt8BGSmlLbPkMjmmtjYnGv63rvnR CF/Q7lbOO7NxQF4kmNueriZsjy3YXkAIq7iPcBhQRI0rCnJXkPrK5S25qrN4Du71wnot RQ== Received: from userp3020.oracle.com (userp3020.oracle.com [156.151.31.79]) by aserp2120.oracle.com with ESMTP id 37u3ymguxh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Apr 2021 02:29:29 +0000 Received: from pps.filterd (userp3020.oracle.com [127.0.0.1]) by userp3020.oracle.com (8.16.0.42/8.16.0.42) with SMTP id 13E2PWM9055502; Wed, 14 Apr 2021 02:29:28 GMT Received: from userv0121.oracle.com (userv0121.oracle.com [156.151.31.72]) by userp3020.oracle.com with ESMTP id 37unst8n1n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Apr 2021 02:29:28 +0000 Received: from abhmp0002.oracle.com (abhmp0002.oracle.com [141.146.116.8]) by userv0121.oracle.com (8.14.4/8.13.8) with ESMTP id 13E2T9N1027598; Wed, 14 Apr 2021 02:29:09 GMT Received: from [10.39.235.234] (/10.39.235.234) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 14 Apr 2021 02:29:08 +0000 Mime-Version: 1.0 (Mac OS X Mail 13.4 \(3608.120.23.2.4\)) Subject: Re: [External] : Re: [PATCH v14 3/6] locking/qspinlock: Introduce CNA into the slow path of qspinlock From: Alex Kogan In-Reply-To: Date: Tue, 13 Apr 2021 22:29:07 -0400 Cc: linux@armlinux.org.uk, Ingo Molnar , Will Deacon , Arnd Bergmann , longman@redhat.com, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, tglx@linutronix.de, bp@alien8.de, hpa@zytor.com, x86@kernel.org, guohanjun@huawei.com, jglauber@marvell.com, steven.sistare@oracle.com, daniel.m.jordan@oracle.com, dave.dice@oracle.com Message-Id: References: <20210401153156.1165900-1-alex.kogan@oracle.com> <20210401153156.1165900-4-alex.kogan@oracle.com> To: Peter Zijlstra X-Mailer: Apple Mail (2.3608.120.23.2.4) X-Proofpoint-IMR: 1 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9953 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 adultscore=0 malwarescore=0 suspectscore=0 bulkscore=0 mlxscore=0 spamscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104140016 X-Proofpoint-GUID: ABqW3SS4571hF5OwUz4Z393S9cZYSXFB X-Proofpoint-ORIG-GUID: ABqW3SS4571hF5OwUz4Z393S9cZYSXFB X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=9953 signatures=668683 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 mlxlogscore=999 spamscore=0 impostorscore=0 priorityscore=1501 lowpriorityscore=0 adultscore=0 bulkscore=0 phishscore=0 suspectscore=0 malwarescore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2104060000 definitions=main-2104140016 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210413_192959_541124_B68F298D X-CRM114-Status: GOOD ( 49.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org UGV0ZXIsIHRoYW5rcyBmb3IgYWxsIHRoZSBjb21tZW50cyBhbmQgc3VnZ2VzdGlvbnMhCgo+IE9u IEFwciAxMywgMjAyMSwgYXQgNzozMCBBTSwgUGV0ZXIgWmlqbHN0cmEgPHBldGVyekBpbmZyYWRl YWQub3JnPiB3cm90ZToKPiAKPiBPbiBUaHUsIEFwciAwMSwgMjAyMSBhdCAxMTozMTo1M0FNIC0w NDAwLCBBbGV4IEtvZ2FuIHdyb3RlOgo+IAo+PiArLyoKPj4gKyAqIGNuYV9zcGxpY2VfdGFpbCAt LSBzcGxpY2UgdGhlIG5leHQgbm9kZSBmcm9tIHRoZSBwcmltYXJ5IHF1ZXVlIG9udG8KPj4gKyAq IHRoZSBzZWNvbmRhcnkgcXVldWUuCj4+ICsgKi8KPj4gK3N0YXRpYyB2b2lkIGNuYV9zcGxpY2Vf bmV4dChzdHJ1Y3QgbWNzX3NwaW5sb2NrICpub2RlLAo+PiArCQkJICAgIHN0cnVjdCBtY3Nfc3Bp bmxvY2sgKm5leHQsCj4+ICsJCQkgICAgc3RydWN0IG1jc19zcGlubG9jayAqbm5leHQpCj4gCj4g WW91IGZvcmdvdCB0byB1cGRhdGUgdGhlIGNvbW1lbnQgd2hlbiB5b3UgY2hhbmdlZCB0aGUgbmFt ZSBvbiB0aGlzCj4gdGhpbmcuCkdvb2QgY2F0Y2gsIHRoYW5rcy4KCj4gCj4+ICsvKgo+PiArICog Y25hX29yZGVyX3F1ZXVlIC0gY2hlY2sgd2hldGhlciB0aGUgbmV4dCB3YWl0ZXIgaW4gdGhlIG1h aW4gcXVldWUgaXMgb24KPj4gKyAqIHRoZSBzYW1lIE5VTUEgbm9kZSBhcyB0aGUgbG9jayBob2xk ZXI7IGlmIG5vdCwgYW5kIGl0IGhhcyBhIHdhaXRlciBiZWhpbmQKPj4gKyAqIGl0IGluIHRoZSBt YWluIHF1ZXVlLCBtb3ZlIHRoZSBmb3JtZXIgb250byB0aGUgc2Vjb25kYXJ5IHF1ZXVlLgo+PiAr ICovCj4+ICtzdGF0aWMgdm9pZCBjbmFfb3JkZXJfcXVldWUoc3RydWN0IG1jc19zcGlubG9jayAq bm9kZSkKPj4gK3sKPj4gKwlzdHJ1Y3QgbWNzX3NwaW5sb2NrICpuZXh0ID0gUkVBRF9PTkNFKG5v ZGUtPm5leHQpOwo+PiArCXN0cnVjdCBjbmFfbm9kZSAqY24gPSAoc3RydWN0IGNuYV9ub2RlICop bm9kZTsKPj4gKwlpbnQgbnVtYV9ub2RlLCBuZXh0X251bWFfbm9kZTsKPj4gKwo+PiArCWlmICgh bmV4dCkgewo+PiArCQljbi0+cGFydGlhbF9vcmRlciA9IExPQ0FMX1dBSVRFUl9OT1RfRk9VTkQ7 Cj4+ICsJCXJldHVybjsKPj4gKwl9Cj4+ICsKPj4gKwludW1hX25vZGUgPSBjbi0+bnVtYV9ub2Rl Owo+PiArCW5leHRfbnVtYV9ub2RlID0gKChzdHJ1Y3QgY25hX25vZGUgKiluZXh0KS0+bnVtYV9u b2RlOwo+PiArCj4+ICsJaWYgKG5leHRfbnVtYV9ub2RlICE9IG51bWFfbm9kZSkgewo+PiArCQlz dHJ1Y3QgbWNzX3NwaW5sb2NrICpubmV4dCA9IFJFQURfT05DRShuZXh0LT5uZXh0KTsKPj4gKwo+ PiArCQlpZiAobm5leHQpIHsKPj4gKwkJCWNuYV9zcGxpY2VfbmV4dChub2RlLCBuZXh0LCBubmV4 dCk7Cj4+ICsJCQluZXh0ID0gbm5leHQ7Cj4+ICsJCX0KPj4gKwkJLyoKPj4gKwkJICogSW5oZXJp dCBOVU1BIG5vZGUgaWQgb2YgcHJpbWFyeSBxdWV1ZSwgdG8gbWFpbnRhaW4gdGhlCj4+ICsJCSAq IHByZWZlcmVuY2UgZXZlbiBpZiB0aGUgbmV4dCB3YWl0ZXIgaXMgb24gYSBkaWZmZXJlbnQgbm9k ZS4KPj4gKwkJICovCj4+ICsJCSgoc3RydWN0IGNuYV9ub2RlICopbmV4dCktPm51bWFfbm9kZSA9 IG51bWFfbm9kZTsKPj4gKwl9Cj4+ICt9Cj4gCj4gU28gdGhlIG9idmlvdXMgY2hhbmdlIHNpbmNl IGxhc3QgdGltZSBJIGxvb2tlZCBhIHRoaXMgaXMgdGhhdCBpdCBub3cKPiBvbmx5IGxvb2tzIDEg ZW50cnkgYWhlYWQuIFdoaWNoIG1ha2VzIHNlbnNlIEkgc3VwcG9zZS4KVGhpcyBpcyBpbiByZXNw b25zZSB0byB0aGUgY3JpdGlxdWUgdGhhdCB0aGUgd29yc3QtY2FzZSB0aW1lIGNvbXBsZXhpdHkg b2YKY25hX29yZGVyX3F1ZXVlKCkgd2FzIE8obikuIFdpdGggdGhpcyBjaGFuZ2UsIHRoZSBjb21w bGV4aXR5IGlzIGNvbnN0YW50LgoKPiAKPiBJJ20gbm90IHJlYWxseSBhIGZhbiBvZiB0aGUgJ3Bh cnRpYWxfb3JkZXInIG5hbWUgY29tYmluZWQgd2l0aCB0aGF0Cj4gc2lsbHkgZW51bSB7IExPQ0FM X1dBSVRFUl9GT1VORCwgTE9DQUxfV0FJVEVSX05PVF9GT1VORCB9LiBUaGF0J3MganVzdAo+IHJl YWxseSBiYWQgbmFtaW5nIGFsbCBhcm91bmQuIFRoZSBlbnVtIGlzIGFib3V0IGhhdmluZyBhIHdh aXRlciB3aGlsZQo+IHRoZSB2YXJpYWJsZSBpcyBhYm91dCBwYXJ0aWFsIG9yZGVyLCB0aGF0IGRv ZXNuJ3QgbWF0Y2ggYXQgYWxsLgpGYWlyIGVub3VnaC4KCj4gSWYgeW91IHJlbmFtZSB0aGUgdmFy aWFibGUgdG8gJ2hhc193YWl0ZXInIGFuZCBzaW1wbHkgdXNlIDAsMSB2YWx1ZXMsCj4gdGhpbmdz IHdvdWxkIGJlIGV2ZXIgc28gbW9yZSByZWFkYWJsZS4gQnV0IEkgZG9uJ3QgdGhpbmsgdGhhdCBt YWtlcwo+IHNlbnNlLCBzZWUgYmVsb3cuCj4gCj4gSSdtIGFsc28gbm90IHN1cmUgYWJvdXQgdGhh dCB3aG9sZSBudW1hX25vZGUgdGhpbmcsIHdoeSB3b3VsZCB5b3UKPiBvdmVyLXdyaXRlIHRoZSBu dW1hIG5vZGUsIHdoeSBhdCB0aGlzIHBvaW50ID8KV2l0aCB0aGlzIG1vdmUtb25lLWJ5LW9uZSBh cHByb2FjaCwgSSB3YW50IHRvIGtlZXAgdGhlIE5VTUEtbm9kZSAKcHJlZmVyZW5jZSBvZiB0aGUg bG9jayBob2xkZXIgZXZlbiBpZiB0aGUgbmV4dC1uZXh0IHdhaXRlciBpcyBvbiBhIGRpZmZlcmVu dApOVU1BLW5vZGUuIE90aGVyd2lzZSwgd2Ugd2lsbCBlbmQgdXAgc3dpdGNoaW5nIHByZWZlcmVu Y2Ugb2Z0ZW4gYW5kCnRoZSBlbnRpcmUgc2NoZW1lIHdvdWxkIG5vdCBwZXJmb3JtIHdlbGwuIElu IHBhcnRpY3VsYXIsIHdlIG1pZ2h0IGVhc2lseQplbmQgdXAgd2l0aCB0aHJlYWRzIGZyb20gdGhl IHByZWZlcnJlZCBub2RlIHdhaXRpbmcgaW4gdGhlIHNlY29uZGFyeSBxdWV1ZS4KCj4gCj4+ICsK Pj4gKy8qIEFidXNlIHRoZSBwdl93YWl0X2hlYWRfb3JfbG9jaygpIGhvb2sgdG8gZ2V0IHNvbWUg d29yayBkb25lICovCj4+ICtzdGF0aWMgX19hbHdheXNfaW5saW5lIHUzMiBjbmFfd2FpdF9oZWFk X29yX2xvY2soc3RydWN0IHFzcGlubG9jayAqbG9jaywKPj4gKwkJCQkJCSBzdHJ1Y3QgbWNzX3Nw aW5sb2NrICpub2RlKQo+PiArewo+PiArCS8qCj4+ICsJICogVHJ5IGFuZCBwdXQgdGhlIHRpbWUg b3RoZXJ3aXNlIHNwZW50IHNwaW4gd2FpdGluZyBvbgo+PiArCSAqIF9RX0xPQ0tFRF9QRU5ESU5H X01BU0sgdG8gdXNlIGJ5IHNvcnRpbmcgb3VyIGxpc3RzLgo+PiArCSAqLwo+PiArCWNuYV9vcmRl cl9xdWV1ZShub2RlKTsKPj4gKwo+PiArCXJldHVybiAwOyAvKiB3ZSBsaWVkOyB3ZSBkaWRuJ3Qg d2FpdCwgZ28gZG8gc28gbm93ICovCj4gCj4gU28gaGVyZSB3ZSBpbnNwZWN0IG9uZSBlbnRyeSBh aGVhZCBhbmQgdGhlbiBxdWl0LiBJIGNhbid0IHJtZW1iZXIsIGJ1dAo+IGRpZCB3ZSB0cnkgc29t ZXRoaW5nIGxpa2U6Cj4gCj4gCS8qCj4gCSAqIFRyeSBhbmQgcHV0IHRoZSB0aW1lIG90aGVyd2lz ZSBzcGVudCBzcGluIHdhaXRpbmcgb24KPiAJICogX1FfTE9DS0VEX1BFTkRJTkdfTUFTSyB0byB1 c2UgYnkgc29ydGluZyBvdXIgbGlzdHMuCj4gCSAqIE1vdmUgb25lIGVudHJ5IGF0IGEgZ28gdW50 aWwgZWl0aGVyIHRoZSBsaXN0IGlzIGZ1bGx5Cj4gCSAqIHNvcnRlZCBvciB3ZSByYW4gb3V0IG9m IHNwaW4gY29uZGl0aW9uLgo+IAkgKi8KPiAJd2hpbGUgKFJFQURfT05DRShsb2NrLT52YWwpICYg X1FfTE9DS0VEX1BFTkRJTkdfTUFTSyAmJgo+IAkgICAgICAgbm9kZS0+cGFydGlhbF9vcmRlcikK PiAJCWNuYV9vcmRlcl9xdWV1ZShub2RlKTsKPiAKPiAJcmV0dXJuIDA7Cj4gCj4gVGhpcyB3aWxs IGtlZXAgbW92aW5nIEBuZXh0IHRvIHRoZSByZW1vdGUgbGlzdCB1bnRpbCBzdWNoIGEgdGltZSB0 aGF0Cj4gd2UncmUgZm9yY2VkIHRvIGNvbnRpbnVlIG9yIEBuZXh0IGlzIGxvY2FsLgpXZSBoYXZl IG5vdCB0cmllZCB0aGF0LiBUaGlzIGlzIGFjdHVhbGx5IGFuIGludGVyZXN0aW5nIGlkZWEsIHdp dGggaXRzIHByb3MgYW5kIGNvbnMuClRoYXQgaXMsIHdlIGFyZSBsaWtlbHkgdG8gZmlsdGVyIG91 dCDigJxub24tcHJlZmVycmVk4oCdIHdhaXRlcnMgaW50byB0aGUgc2Vjb25kYXJ5IHF1ZXVlCmZh c3RlciwgYnV0IGFsc28gd2UgYXJlIG1vcmUgbGlrZWx5IHRvIHJ1biBpbnRvIGEgc2l0dWF0aW9u IHdoZXJlIHRoZSBsb2NrIGJlY29tZXMKYXZhaWxhYmxlIGF0IHRoZSB0aW1lIHdlIGFyZSBydW5u aW5nIHRoZSBjbmFfb3JkZXJfcXVldWUoKSBsb2dpYywgdGh1cyBwcm9sb25naW5nCnRoZSBoYW5k b3ZlciB0aW1lLgoKV2l0aCB0aGlzIGNoYW5nZSwgaG93ZXZlciwgdGhlcmUgaXMgbm8gbmVlZCB0 byBjYWxsIGNuYV9vcmRlcl9xdWV1ZSgpIGFnYWluIAppbiBjbmFfbG9ja19oYW5kb2ZmKCksIHdo aWNoIGlzIGFub3RoZXIgYWR2YW50YWdlIEkgZ3Vlc3MuCgpBbGwgaW4gYWxsLCBJIGFtIGZpbmUg c3dpdGNoaW5nIHRvIHRoaXMgYWx0ZXJuYXRpdmUgaWYgeW91IGxpa2UgaXQgbW9yZS4KCkJUVywg d2UgY2FuIHJldHVybiBub2RlLT5wYXJ0aWFsX29yZGVyIGluc3RlYWQgb2YgMCwgdG8gc2tpcCB0 aGUgY2FsbCB0bwphdG9taWNfY29uZF9yZWFkX2FjcXVpcmUoKSBpbiB0aGUgZ2VuZXJhbCBjb2Rl LgoKPiAKPj4gK30KPj4gKwo+PiArc3RhdGljIGlubGluZSB2b2lkIGNuYV9sb2NrX2hhbmRvZmYo c3RydWN0IG1jc19zcGlubG9jayAqbm9kZSwKPj4gKwkJCQkgc3RydWN0IG1jc19zcGlubG9jayAq bmV4dCkKPj4gK3sKPj4gKwlzdHJ1Y3QgY25hX25vZGUgKmNuID0gKHN0cnVjdCBjbmFfbm9kZSAq KW5vZGU7Cj4+ICsJdTMyIHZhbCA9IDE7Cj4+ICsKPj4gKwl1MzIgcGFydGlhbF9vcmRlciA9IGNu LT5wYXJ0aWFsX29yZGVyOwo+PiArCj4+ICsJaWYgKHBhcnRpYWxfb3JkZXIgPT0gTE9DQUxfV0FJ VEVSX05PVF9GT1VORCkKPj4gKwkJY25hX29yZGVyX3F1ZXVlKG5vZGUpOwo+PiArCj4gCj4gQUZB SUNUIHRoaXMgaXMgd2hlcmUgcGxheWluZyBzaWxseSBnYW1lcyB3aXRoIC0+bnVtYV9ub2RlIGJl bG9uZzsgYnV0Cj4gcmlnaHQgYWxvbmcgd2l0aCB0aGF0IGdvZXMgYSBjb21tZW50IHRoYXQgZGVz Y3JpYmVzIHdoeSBhbnkgb2YgdGhhdAo+IG1ha2VzIHNlbnNlLgo+IAo+IEkgbWVhbiwgaWYgeW91 IGxlYXZlIHlvdXIgbm9kZSwgZm9yIGFueSByZWFzb24sIHdoeSBib3RoZXIgY29taW5nIGJhY2sK PiB0byBpdCwgd2h5IG5vdCBhY2NlcHQgaXQgaXMgYSBzaWduIG9mIHRoZSBnb2RzIHlvdSdyZSBv dmVyZHVlIGZvciBhCj4gbm9kZS1jaGFuZ2U/CkkgdGhpbmsgdGhlcmUgaXMgc29tZSBtaXN1bmRl cnN0YW5kaW5nIGhlcmUg4oCUIGxldCBtZSB0cnkgdG8gY2xhcmlmeS4KSW4gdGhlIGN1cnJlbnQg bG9naWMsIHdlIGZpcnN0IGNhbGwgY25hX29yZGVyX3F1ZXVlKCkgZnJvbSBjbmFfd2FpdF9oZWFk X29yX2xvY2soKS4KSWYgd2UgZmluZCBhbiBhcHByb3ByaWF0ZSDigJxsb2NhbOKAnSB3YWl0ZXIg KGVpdGhlciByZWFsIG9yIGZha2UpLCB3ZSBzZXQgaXQgYXMKbmV4dCBhbmQgbWFrZSBzdXJlIHRo ZSBudW1hX25vZGUgb2YgdGhhdCBub2RlIGlzIHRoZSBzYW1lIGFzIG91cnMuCklmIG5vdCAoaS5l Liwgd2hlbiB3ZSBkbyBub3QgaGF2ZSBhbnkgbmV4dCB3YWl0ZXIpLCB3ZSBzZXQgcGFydGlhbF9v cmRlciB0bwpMT0NBTF9XQUlURVJfTk9UX0ZPVU5EIChwYXJkb24gdGhlIG5hbWVzKSBhbmQgZ28g c3BpbiBvbiB0aGUgbG9jayAoY2FsbGluZwphdG9taWNfY29uZF9yZWFkX2FjcXVpcmUoKSBpbiB0 aGUgZ2VuZXJpYyBjb2RlKS4KRHVyaW5nIHRoaXMgc3BpbiB0aW1lLCBuZXcgd2FpdGVycyBjYW4g am9pbiB0aGUgcXVldWUuCkhlbmNlLCB3ZSByZWNoZWNrIHRoZSBzdGF0ZSBvZiB0aGUgcXVldWUg YnkgY2FsbGluZyBjbmFfb3JkZXJfcXVldWUoKSBhZ2Fpbgpmcm9tIGNuYV9sb2NrX2hhbmRvZmYo KS4KCkFzIGp1c3QgbWVudGlvbmVkLCBpZiB3ZSBjaGFuZ2UgdGhlIGxvZ2ljIGFzIHlvdSBzdWdn ZXN0LCB0aGVyZSBpcwpyZWFsbHkgbm8gcmVhc29uIHRvIGNhbGwgY25hX29yZGVyX3F1ZXVlKCkg YWdhaW4uCgo+IAo+IFdhcyB0aGUgZWZmaWNhY3kgb2YgdGhpcyBzY2hlbWUgdGVzdGVkPwo+IAo+ PiArCS8qCj4+ICsJICogV2UgaGF2ZSBhIGxvY2FsIHdhaXRlciwgZWl0aGVyIHJlYWwgb3IgZmFr ZSBvbmU7Cj4+ICsJICogcmVsb2FkIEBuZXh0IGluIGNhc2UgaXQgd2FzIGNoYW5nZWQgYnkgY25h X29yZGVyX3F1ZXVlKCkuCj4+ICsJICovCj4+ICsJbmV4dCA9IG5vZGUtPm5leHQ7Cj4+ICsJaWYg KG5vZGUtPmxvY2tlZCA+IDEpCj4+ICsJCXZhbCA9IG5vZGUtPmxvY2tlZDsJLyogcHJlc2V2ZSBz ZWNvbmRhcnkgcXVldWUgKi8KPiAKPiBJSVJDIHdlIHVzZWQgdG8gZG86Cj4gCj4gCXZhbCB8PSBu b2RlLT5sb2NrZWQ7Cj4gCj4gV2hpY2ggaXMgc2ltcGxlciBmb3Igbm90IGhhdmluZyBicmFuY2hl cy4gV2h5IGNoYW5nZSBhIGdvb2QgdGhpbmc/CldpdGggfD0gd2UgbWlnaHQgc2V0IHZhbCB0byBl bmNvZGVkIHRhaWwrMSAocmF0aGVyIHRoYW4gZW5jb2RlZCB0YWlsKS4KVGhpcyBzdGlsbCB3b3Vs ZCB3b3JrLCBhcyBkZWNvZGVfdGFpbCh2YWwpIGRvZXMgbm90IGNhcmUgYWJvdXQgTFNCLCBidXQg c2VlbXMgZnJhZ2lsZSB0byBtZS4KV2UgdGFsa2VkIGFib3V0IHRoYXQgYmVmb3JlOiAKaHR0cHM6 Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtYXJtLWtlcm5lbC9FMzJGOTBFMi1GMDBCLTQyM0ItQTg1 MS0zMzYwMDRFRjY1OTNAb3JhY2xlLmNvbQoKSWYgeW91IHRoaW5rIHRoaXMgaXMgYSBub24taXNz dWUsIEkgd2lsbCBiZSBoYXBweSB0byBhZG9wdCB5b3VyIHN1Z2dlc3Rpb24uCgpSZWdhcmRzLAri gJQgQWxleAoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CmxpbnV4LWFybS1rZXJuZWwgbWFpbGluZyBsaXN0CmxpbnV4LWFybS1rZXJuZWxAbGlzdHMuaW5m cmFkZWFkLm9yZwpodHRwOi8vbGlzdHMuaW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xp bnV4LWFybS1rZXJuZWwK