From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B02F3C4743C for ; Wed, 23 Jun 2021 13:38:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EEA2A61075 for ; Wed, 23 Jun 2021 13:38:46 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EEA2A61075 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=basnieuwenhuizen.nl Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7E1B76E8E3; Wed, 23 Jun 2021 13:38:46 +0000 (UTC) Received: from mail-io1-xd31.google.com (mail-io1-xd31.google.com [IPv6:2607:f8b0:4864:20::d31]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1C5D66E8E3 for ; Wed, 23 Jun 2021 13:38:45 +0000 (UTC) Received: by mail-io1-xd31.google.com with SMTP id h2so3411544iob.11 for ; Wed, 23 Jun 2021 06:38:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=basnieuwenhuizen.nl; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=pbUycUN2rLw/NpVnHTIINetmdItB8ttstk1+aiKBgsI=; b=dBBJPAqrXkKWSND+v1EL2tus27yTDJPdFsHwNGt83YekplISakyk6r63tgTzokXwjl ahb3TlHilV23gBBNK99qmug87Vg8QzXVaLnn1BVMd3ReQKn/g1sGT/xH0tJmsUm00Qow aY9PQUWyJnyZFMlymGL9uSIwyK7CfvlvWQgQuWdtyNhm8YFYCsogTGNi33nNdG1c0qzI q7s1mH61NP6unfToNoXDJ/tQqGR7dJtSiLB4mlK6jYzM56zhsZwALAFkY/VulDVMnMQR Xfs1Xsjtsls+2M2+hnROYU0StIWPfQk10F8XIE8HB6VfycxUEcL1pF5CPs2jav24s0to 2kqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=pbUycUN2rLw/NpVnHTIINetmdItB8ttstk1+aiKBgsI=; b=Acgi2eyrkKEQAQ1C5g3o9takBfcKLNyeWtdBjIDUSfYp0XYpzr6c67QJKL+e6TSEvt PPvtfVn47w2pu/EBUqQm4n1HoJLkNDwZ3PuEhldtfC5z4hlqzommO6zy9ByyBDkYa0Jv FIGufUhVjC5a1JOY6yec2vLn3cC81sexmETvmE/6Wbbl/c/nr08y9azi1yF898aIK8KC AuBw9V8aFbGwp7pR2Poj0v9QoMxOvM0l0+wVI/pQPgUlLncQHrofv/BnU9vCXjnmp8Xo lzaN10FQEuQdX68Mdkg0/vPiBpUmFIBSq85UFgTjKgvKJ1RMycWRuj4wC0u9iI8t6V24 Z7lA== X-Gm-Message-State: AOAM531pLjTHsmIijPQXJ5e6TDF3DPX1Aqq2OKsqtZTJeU0rXQSi2H55 O3qrLpyRVSoDL+4xWxtH/W0EAtbadN3St2tdA4gHQw== X-Google-Smtp-Source: ABdhPJxyvGrbQ2FKDblt/MdAvighHagEciQVyyHFv/qmVSXPRKwn5bAoCENV10JBsnt+9ZJ5zgZgRhsNGgy0+22w9Z4= X-Received: by 2002:a02:ca4a:: with SMTP id i10mr8988384jal.141.1624455524349; Wed, 23 Jun 2021 06:38:44 -0700 (PDT) MIME-Version: 1.0 References: <20210622165511.3169559-1-daniel.vetter@ffwll.ch> <20210622165511.3169559-16-daniel.vetter@ffwll.ch> <3bf45006-4256-763d-601b-3a25a7057820@amd.com> In-Reply-To: <3bf45006-4256-763d-601b-3a25a7057820@amd.com> From: Bas Nieuwenhuizen Date: Wed, 23 Jun 2021 15:38:39 +0200 Message-ID: To: =?UTF-8?Q?Christian_K=C3=B6nig?= Subject: Re: [Intel-gfx] [PATCH 15/15] RFC: drm/amdgpu: Implement a proper implicit fencing uapi X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , Daniel Stone , Daniel Vetter , Intel Graphics Development , Kevin Wang , DRI Development , Sumit Semwal , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Luben Tuikov , "Kristian H . Kristensen" , Chen Li , Daniel Vetter , Alex Deucher , mesa-dev , Dave Airlie , =?UTF-8?Q?Michel_D=C3=A4nzer?= , Dennis Li , Deepak R Varma Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gV2VkLCBKdW4gMjMsIDIwMjEgYXQgMjo1OSBQTSBDaHJpc3RpYW4gS8O2bmlnCjxjaHJpc3Rp YW4ua29lbmlnQGFtZC5jb20+IHdyb3RlOgo+Cj4gQW0gMjMuMDYuMjEgdW0gMTQ6MTggc2Nocmll YiBEYW5pZWwgVmV0dGVyOgo+ID4gT24gV2VkLCBKdW4gMjMsIDIwMjEgYXQgMTE6NDUgQU0gQmFz IE5pZXV3ZW5odWl6ZW4KPiA+IDxiYXNAYmFzbmlldXdlbmh1aXplbi5ubD4gd3JvdGU6Cj4gPj4g T24gVHVlLCBKdW4gMjIsIDIwMjEgYXQgNjo1NSBQTSBEYW5pZWwgVmV0dGVyIDxkYW5pZWwudmV0 dGVyQGZmd2xsLmNoPiB3cm90ZToKPiA+Pj4gV0FSTklORzogQWJzb2x1dGVseSB1bnRlc3RlZCBi ZXlvbmQgImdjYyBpc24ndCBkeWluZyBpbiBhZ29ueSIuCj4gPj4+Cj4gPj4+IEltcGxpY2l0IGZl bmNpbmcgZG9uZSBwcm9wZXJseSBuZWVkcyB0byB0cmVhdCB0aGUgaW1wbGljaXQgZmVuY2luZwo+ ID4+PiBzbG90cyBsaWtlIGEgZnVubnkga2luZCBvZiBJUEMgbWFpbGJveC4gSW4gb3RoZXIgd29y ZHMgaXQgbmVlZHMgdG8gYmUKPiA+Pj4gZXhwbGljaXRseS4gVGhpcyBpcyB0aGUgb25seSB3YXkg aXQgd2lsbCBtZXNoIHdlbGwgd2l0aCBleHBsaWNpdAo+ID4+PiBmZW5jaW5nIHVzZXJzcGFjZSBs aWtlIHZrLCBhbmQgaXQncyBhbHNvIHRoZSBiYXJlIG1pbmltdW0gcmVxdWlyZWQgdG8KPiA+Pj4g YmUgYWJsZSB0byBtYW5hZ2UgYW55dGhpbmcgZWxzZSB0aGF0IHdhbnRzIHRvIHVzZSB0aGUgc2Ft ZSBidWZmZXIgb24KPiA+Pj4gbXVsdGlwbGUgZW5naW5lcyBpbiBwYXJhbGxlbCwgYW5kIHN0aWxs IGJlIGFibGUgdG8gc2hhcmUgaXQgdGhyb3VnaAo+ID4+PiBpbXBsaWNpdCBzeW5jLgo+ID4+Pgo+ ID4+PiBhbWRncHUgY29tcGxldGVseSBsYWNrcyBzdWNoIGFuIHVhcGkuIEZpeCB0aGlzLgo+ID4+ Pgo+ID4+PiBMdWNraWx5IHRoZSBjb25jZXB0IG9mIGlnbm9yaW5nIGltcGxpY2l0IGZlbmNlcyBl eGlzdHMgYWxyZWFkeSwgYW5kCj4gPj4+IHRha2VzIGNhcmUgb2YgYWxsIHRoZSBjb21wbGV4aXRp ZXMgb2YgbWFraW5nIHN1cmUgdGhhdCBub24tb3B0aW9uYWwKPiA+Pj4gZmVuY2VzIChsaWtlIGJv IG1vdmVzKSBhcmUgbm90IGlnbm9yZWQuIFRoaXMgc3VwcG9ydCB3YXMgYWRkZWQgaW4KPiA+Pj4K PiA+Pj4gY29tbWl0IDE3N2FlMDliNWQ2OTlhNWViZDFjYWZjZWU3ODg4OWRiOTY4YWJmNTQKPiA+ Pj4gQXV0aG9yOiBBbmRyZXMgUm9kcmlndWV6IDxhbmRyZXN4N0BnbWFpbC5jb20+Cj4gPj4+IERh dGU6ICAgRnJpIFNlcCAxNSAyMDo0NDowNiAyMDE3IC0wNDAwCj4gPj4+Cj4gPj4+ICAgICAgZHJt L2FtZGdwdTogaW50cm9kdWNlIEFNREdQVV9HRU1fQ1JFQVRFX0VYUExJQ0lUX1NZTkMgdjIKPiA+ Pj4KPiA+Pj4gVW5mb3J0dWFudGVseSBpdCdzIHRoZSB3cm9uZyBzZW1hbnRpY3MsIGJlY2F1c2Ug aXQncyBhIGJvIGZsYWcgYW5kCj4gPj4+IGRpc2FibGVzIGltcGxpY2l0IHN5bmMgb24gYW4gYWxs b2NhdGVkIGJ1ZmZlciBjb21wbGV0ZWx5Lgo+ID4+Pgo+ID4+PiBXZSBfZG9fIHdhbnQgaW1wbGlj aXQgc3luYywgYnV0IGNvbnRyb2wgaXQgZXhwbGljaXRseS4gRm9yIHRoaXMgd2UKPiA+Pj4gbmVl ZCBhIGZsYWcgb24gdGhlIGRybV9maWxlLCBzbyB0aGF0IGEgZ2l2ZW4gdXNlcnNwYWNlIChsaWtl IHZ1bGthbikKPiA+Pj4gY2FuIG1hbmFnZSB0aGUgaW1wbGljaXQgc3luYyBzbG90cyBleHBsaWNp dGx5LiBUaGUgb3RoZXIgc2lkZSBvZiB0aGUKPiA+Pj4gcGlwZWxpbmUgKGNvbXBvc2l0b3IsIG90 aGVyIHByb2Nlc3Mgb3IganVzdCBkaWZmZXJlbnQgc3RhZ2UgaW4gYSBtZWRpYQo+ID4+PiBwaXBl bGluZSBpbiB0aGUgc2FtZSBwcm9jZXNzKSBjYW4gdGhlbiBlaXRoZXIgZG8gdGhlIHNhbWUsIG9y IGZ1bGx5Cj4gPj4+IHBhcnRpY2lwYXRlIGluIHRoZSBpbXBsaWNpdCBzeW5jIGFzIGltcGxlbWVu dGVkIGJ5IHRoZSBrZXJuZWwgYnkKPiA+Pj4gZGVmYXVsdC4KPiA+Pj4KPiA+Pj4gQnkgYnVpbGRp bmcgb24gdGhlIGV4aXN0aW5nIGZsYWcgZm9yIGJ1ZmZlcnMgd2UgYXZvaWQgYW55IGlzc3VlcyB3 aXRoCj4gPj4+IG9wZW5pbmcgdXAgYWRkaXRpb25hbCBzZWN1cml0eSBjb25jZXJucyAtIGFueXRo aW5nIHRoaXMgbmV3IGZsYWcgaGVyZQo+ID4+PiBhbGxvd3MgaXMgYWxyZWFkeS4KPiA+Pj4KPiA+ Pj4gQWxsIGRyaXZlcnMgd2hpY2ggc3VwcG9ydHMgdGhpcyBjb25jZXB0IG9mIGEgdXNlcnNwYWNl LXNwZWNpZmljCj4gPj4+IG9wdC1vdXQgb2YgaW1wbGljaXQgc3luYyBoYXZlIGEgZmxhZyBpbiB0 aGVpciBDUyBpb2N0bCwgYnV0IGluIHJlYWxpdHkKPiA+Pj4gdGhhdCB0dXJuZWQgb3V0IHRvIGJl IGEgYml0IHRvbyBpbmZsZXhpYmxlLiBTZWUgdGhlIGRpc2N1c3Npb24gYmVsb3csCj4gPj4+IGxl dCdzIHRyeSB0byBkbyBhIGJpdCBiZXR0ZXIgZm9yIGFtZGdwdS4KPiA+Pj4KPiA+Pj4gVGhpcyBh bG9uZSBvbmx5IGFsbG93cyB1cyB0byBjb21wbGV0ZWx5IGF2b2lkIGFueSBzdGFsbHMgZHVlIHRv Cj4gPj4+IGltcGxpY2l0IHN5bmMsIGl0IGRvZXMgbm90IHlldCBhbGxvdyB1cyB0byB1c2UgaW1w bGljaXQgc3luYyBhcyBhCj4gPj4+IHN0cmFuZ2UgZm9ybSBvZiBJUEMgZm9yIHN5bmNfZmlsZS4K PiA+Pj4KPiA+Pj4gRm9yIHRoYXQgd2UgbmVlZCB0d28gbW9yZSBwaWVjZXM6Cj4gPj4+Cj4gPj4+ IC0gYSB3YXkgdG8gZ2V0IHRoZSBjdXJyZW50IGltcGxpY2l0IHN5bmMgZmVuY2VzIG91dCBvZiBh IGJ1ZmZlci4gQ291bGQKPiA+Pj4gICAgYmUgZG9uZSBpbiBhIGRyaXZlciBpb2N0bCwgYnV0IGV2 ZXJ5b25lIG5lZWRzIHRoaXMsIGFuZCBnZW5lcmFsbHkgYQo+ID4+PiAgICBkbWEtYnVmIGlzIGlu dm9sdmVkIGFueXdheSB0byBlc3RhYmxpc2ggdGhlIHNoYXJpbmcuIFNvIGFuIGlvY3RsIG9uCj4g Pj4+ICAgIHRoZSBkbWEtYnVmIG1ha2VzIGEgdG9uIG1vcmUgc2Vuc2U6Cj4gPj4+Cj4gPj4+ICAg IGh0dHBzOi8vbmFtMTEuc2FmZWxpbmtzLnByb3RlY3Rpb24ub3V0bG9vay5jb20vP3VybD1odHRw cyUzQSUyRiUyRmxvcmUua2VybmVsLm9yZyUyRmRyaS1kZXZlbCUyRjIwMjEwNTIwMTkwMDA3LjUz NDA0Ni00LWphc29uJTQwamxla3N0cmFuZC5uZXQlMkYmYW1wO2RhdGE9MDQlN0MwMSU3Q2Nocmlz dGlhbi5rb2VuaWclNDBhbWQuY29tJTdDZjAyNjA1NWY1MjNkNGU0ZGY5NWIwOGQ5MzY0MTBlMzkl N0MzZGQ4OTYxZmU0ODg0ZTYwOGUxMWE4MmQ5OTRlMTgzZCU3QzAlN0MwJTdDNjM3NjAwNDc1MzUx MDg1NTM2JTdDVW5rbm93biU3Q1RXRnBiR1pzYjNkOGV5SldJam9pTUM0d0xqQXdNREFpTENKUUlq b2lWMmx1TXpJaUxDSkJUaUk2SWsxaGFXd2lMQ0pYVkNJNk1uMCUzRCU3QzEwMDAmYW1wO3NkYXRh PWdVbk04JTJGdWx4JTJCJTJCREx4QnlPJTJGMFYzY0xxdCUyRmMydW5Xaml6RXBwdFFxTThnJTNE JmFtcDtyZXNlcnZlZD0wCj4gPj4+Cj4gPj4+ICAgIEN1cnJlbnQgZHJpdmVycyBpbiB1cHN0cmVh bSBzb2x2ZXMgdGhpcyBieSBoYXZpbmcgdGhlIG9wdC1vdXQgZmxhZwo+ID4+PiAgICBvbiB0aGVp ciBDUyBpb2N0bC4gVGhpcyBoYXMgdGhlIGRvd25zaWRlIHRoYXQgdmVyeSBvZnRlbiB0aGUgQ1MK PiA+Pj4gICAgd2hpY2ggbXVzdCBhY3R1YWxseSBzdGFsbCBmb3IgdGhlIGltcGxpY2l0IGZlbmNl IGlzIHJ1biBhIHdoaWxlCj4gPj4+ICAgIGFmdGVyIHRoZSBpbXBsaWNpdCBmZW5jZSBwb2ludCB3 YXMgbG9naWNhbGx5IHNhbXBsZWQgcGVyIHRoZSBhcGkKPiA+Pj4gICAgc3BlYyAodmsgcGFzc2Vz IGFuIGV4cGxpY2l0IHN5bmNvYmogYXJvdW5kIGZvciB0aGF0IGFmYWl1aSksIGFuZCBzbwo+ID4+ PiAgICByZXN1bHRzIGluIG92ZXJzeW5jLiBDb252ZXJ0aW5nIHRoZSBpbXBsaWNpdCBzeW5jIGZl bmNlcyBpbnRvIGEKPiA+Pj4gICAgc25hcC1zaG90IHN5bmNfZmlsZSBpcyBhY3R1YWxseSBhY2N1 cmF0ZS4KPiA+Pj4KPiA+Pj4gLSBTaW1pbGxhciB3ZSBuZWVkIHRvIGJlIGFibGUgdG8gc2V0IHRo ZSBleGNsdXNpdmUgaW1wbGljaXQgZmVuY2UuCj4gPj4+ICAgIEN1cnJlbnQgZHJpdmVycyBhZ2Fp biBkbyB0aGlzIHdpdGggYSBDUyBpb2N0bCBmbGFnLCB3aXRoIGFnYWluIHRoZQo+ID4+PiAgICBz YW1lIHByb2JsZW1zIHRoYXQgdGhlIHRpbWUgdGhlIENTIGhhcHBlbnMgYWRkaXRpb25hbCBkZXBl bmRlbmNpZXMKPiA+Pj4gICAgaGF2ZSBiZWVuIGFkZGVkLiBBbiBleHBsaWNpdCBpb2N0bCB0byBv bmx5IGluc2VydCBhIHN5bmNfZmlsZSAod2hpbGUKPiA+Pj4gICAgcmVzcGVjdGluZyB0aGUgcnVs ZXMgZm9yIGhvdyBleGNsdXNpdmUgYW5kIHNoYXJlZCBmZW5jZSBzbG90cyBtdXN0Cj4gPj4+ICAg IGJlIHVwZGF0ZSBpbiBzdHJ1Y3QgZG1hX3Jlc3YpIGlzIG11Y2ggYmV0dGVyLiBUaGlzIGlzIHBy b3Bvc2VkIGhlcmU6Cj4gPj4+Cj4gPj4+ICAgIGh0dHBzOi8vbmFtMTEuc2FmZWxpbmtzLnByb3Rl Y3Rpb24ub3V0bG9vay5jb20vP3VybD1odHRwcyUzQSUyRiUyRmxvcmUua2VybmVsLm9yZyUyRmRy aS1kZXZlbCUyRjIwMjEwNTIwMTkwMDA3LjUzNDA0Ni01LWphc29uJTQwamxla3N0cmFuZC5uZXQl MkYmYW1wO2RhdGE9MDQlN0MwMSU3Q2NocmlzdGlhbi5rb2VuaWclNDBhbWQuY29tJTdDZjAyNjA1 NWY1MjNkNGU0ZGY5NWIwOGQ5MzY0MTBlMzklN0MzZGQ4OTYxZmU0ODg0ZTYwOGUxMWE4MmQ5OTRl MTgzZCU3QzAlN0MwJTdDNjM3NjAwNDc1MzUxMDg1NTM2JTdDVW5rbm93biU3Q1RXRnBiR1pzYjNk OGV5SldJam9pTUM0d0xqQXdNREFpTENKUUlqb2lWMmx1TXpJaUxDSkJUaUk2SWsxaGFXd2lMQ0pY VkNJNk1uMCUzRCU3QzEwMDAmYW1wO3NkYXRhPXdGR055ZUwxWVNwa2ViZjFMMUREYjJldWloZjFm dm1SOUc4Y2Z5d3JwVlUlM0QmYW1wO3Jlc2VydmVkPTAKPiA+Pj4KPiA+Pj4gVGhlc2UgdGhyZWUg cGllY2VzIHRvZ2V0aGVyIGFsbG93IHVzZXJzcGFjZSB0byBmdWxseSBjb250cm9sIGltcGxpY2l0 Cj4gPj4+IGZlbmNpbmcgYW5kIHJlbW92ZSBhbGwgdW5lY2Vzc2FyeSBzdGFsbCBwb2ludHMgZHVl IHRvIHRoZW0uCj4gPj4+Cj4gPj4+IFdlbGwsIGFzIG11Y2ggYXMgdGhlIGltcGxpY2l0IGZlbmNp bmcgbW9kZWwgZnVuZGFtZW50YWxseSBhbGxvd3M6Cj4gPj4+IFRoZXJlIGlzIG9ubHkgb25lIHNl dCBvZiBmZW5jZXMsIHlvdSBjYW4gb25seSBjaG9vc2UgdG8gc3luYyBhZ2FpbnN0Cj4gPj4+IG9u bHkgd3JpdGVycyAoZXhjbHVzaXZlIHNsb3QpLCBvciBldmVyeW9uZS4gSGVuY2Ugc3ViYWxsb2Nh dGluZwo+ID4+PiBtdWx0aXBsZSBidWZmZXJzIG9yIGFueXRoaW5nIGVsc2UgbGlrZSB0aGlzIGlz IGZ1bmRhbWVudGFsbHkgbm90Cj4gPj4+IHBvc3NpYmxlLCBhbmQgY2FuIG9ubHkgYmUgZml4ZWQg YnkgYSBwcm9wZXIgZXhwbGljaXQgZmVuY2luZyBtb2RlbC4KPiA+Pj4KPiA+Pj4gQXNpZGUgZnJv bSB0aGF0IGNhdmVhdCB0aGlzIG1vZGVsIGdldHMgaW1wbGljaXQgZmVuY2luZyBhcyBjbG9zZWx5 IHRvCj4gPj4+IGV4cGxpY2l0IGZlbmNpbmcgc2VtYW50aWNzIGFzIHBvc3NpYmxlOgo+ID4+Pgo+ ID4+PiBPbiB0aGUgYWN0dWFsIGltcGxlbWVudGF0aW9uIEkgb3B0ZWQgZm9yIGEgc2ltcGxlIHNl dHBhcmFtIGlvY3RsLCBubwo+ID4+PiBsb2NraW5nIChqdXN0IGF0b21pYyByZWFkcy93cml0ZXMp IGZvciBzaW1wbGljaXR5LiBUaGVyZSBpcyBhIG5pY2UKPiA+Pj4gZmxhZyBwYXJhbWV0ZXIgaW4g dGhlIFZNIGlvY3RsIHdoaWNoIHdlIGNvdWxkIHVzZSwgZXhjZXB0Ogo+ID4+PiAtIGl0J3Mgbm90 IGNoZWNrZWQsIHNvIHVzZXJzcGFjZSBsaWtlbHkgcGFzc2VzIGdhcmJhZ2UKPiA+Pj4gLSB0aGVy ZSdzIGFscmVhZHkgYSBjb21tZW50IHRoYXQgdXNlcnNwYWNlIF9kb2VzXyBwYXNzIGdhcmJhZ2Ug aW4gdGhlCj4gPj4+ICAgIHByaW9yaXR5IGZpZWxkCj4gPj4+IFNvIHllYWggdW5mb3J0dW5hdGVs eSB0aGlzIGZsYWcgcGFyYW1ldGVyIGZvciBzZXR0aW5nIHZtIGZsYWdzIGlzCj4gPj4+IHVzZWxl c3MsIGFuZCB3ZSBuZWVkIHRvIGhhY2sgdXAgYSBuZXcgb25lLgo+ID4+Pgo+ID4+PiB2MjogRXhw bGFpbiB3aHkgYSBuZXcgU0VUUEFSQU0gKEphc29uKQo+ID4+Pgo+ID4+PiB2MzogQmFzIG5vdGlj ZWQgSSBmb3Jnb3QgdG8gaG9vayB1cCB0aGUgZGVwZW5kZW5jeS1zaWRlIHNob3J0Y3V0LiBXZQo+ ID4+PiBuZWVkIGJvdGgsIG9yIHRoaXMgZG9lc24ndCBkbyBtdWNoLgo+ID4+Pgo+ID4+PiB2NDog UmViYXNlIG92ZXIgdGhlIGFtZGdwdSBwYXRjaCB0byBhbHdheXMgc2V0IHRoZSBpbXBsaWNpdCBz eW5jCj4gPj4+IGZlbmNlcy4KPiA+PiBTbyBJIHRoaW5rIHRoZXJlIGlzIHN0aWxsIGEgY2FzZSBt aXNzaW5nIGluIHRoaXMgaW1wbGVtZW50YXRpb24uCj4gPj4gQ29uc2lkZXIgdGhlc2UgMyBjYXNl cwo+ID4+Cj4gPj4gKGZvcm1hdDogYS0+YjogYiB3YWl0cyBvbiBhLiBZZXMsIEkga25vdyBhcnJv d3MgYXJlIGhhcmQpCj4gPj4KPiA+PiBleHBsaWNpdC0+ZXhwbGljaXQ6IFRoaXMgZG9lc24ndCB3 YWl0IG5vdywgd2hpY2ggaXMgZ29vZAo+ID4+IEltcGxpY2l0LT5leHBsaWNpdDogVGhpcyBkb2Vz bid0IHdhaXQgbm93LCB3aGljaCBpcyBnb29kCj4gPj4gZXhwbGljaXQtPmltcGxpY2l0IDogVGhp cyBzdGlsbCB3YWl0cyBhcyB0aGUgZXhwbGljaXQgc3VibWlzc2lvbiBzdGlsbAo+ID4+IGFkZHMg c2hhcmVkIGZlbmNlcyBhbmQgbW9zdCB0aGluZ3MgdGhhdCBzZXQgYW4gZXhjbHVzaXZlIGZlbmNl IGZvcgo+ID4+IGltcGxpY2l0IHN5bmMgd2lsbCBoZW5jZSB3YWl0IG9uIGl0Lgo+ID4+Cj4gPj4g VGhpcyBpcyBwcm9iYWJseSBnb29kIGVub3VnaCBmb3Igd2hhdCByYWR2IG5lZWRzIG5vdyBidXQg YWxzbyBzb3VuZHMKPiA+PiBsaWtlIGEgcmlzayB3cnQgYmFraW5nIGluIG5ldyB1YXBpIGJlaGF2 aW9yIHRoYXQgd2UgZG9uJ3Qgd2FudCB0byBiZQo+ID4+IHRoZSBlbmQgcmVzdWx0Lgo+ID4+Cj4g Pj4gV2l0aGluIEFNREdQVSB0aGlzIGlzIHByb2JhYmx5IHNvbHZhYmxlIGluIHR3byB3YXlzOgo+ ID4+Cj4gPj4gMSkgRG93bmdyYWRlIEFNREdQVV9TWU5DX05FX09XTkVSIHRvIEFNREdQVV9TWU5D X0VYUExJQ0lUIGZvciBzaGFyZWQgZmVuY2VzLgo+ID4gSSdtIG5vdCBzdXJlIHRoYXQgd29ya3Mu IEkgdGhpbmsgdGhlIHJpZ2h0IGZpeCBpcyB0aGF0IHJhZGVvbnNpIGFsc28KPiA+IHN3aXRjaGVz IHRvIHRoaXMgbW9kZWwsIHdpdGggbWF5YmUgYSBwZXItYm8gQ1MgZmxhZyB0byBzZXQgaW5kaWNh dGUKPiA+IHdyaXRlIGFjY2VzcywgdG8gY3V0IGRvd24gb24gdGhlIG51bWJlciBvZiBpb2N0bHMg dGhhdCBhcmUgbmVlZGVkCj4gPiBvdGhlcndpc2Ugb24gc2hhcmVkIGJ1ZmZlcnMuIFRoaXMgcGVy LWJvIGZsYWcgd291bGQgZXNzZW50aWFsbHkgc2VsZWN0Cj4gPiBiZXR3ZWVuIFNZTkNfTkVfT1dO RVIgYW5kIFNZTkNfRVhQTElDSVQgb24gYSBwZXItYnVmZmVyIGJhc2lzLgo+Cj4gWWVhaCwgYnV0 IEknbSBzdGlsbCBub3QgZW50aXJlbHkgc3VyZSB3aHkgdGhhdCBhcHByb2FjaCBpc24ndCBzdWZm aWNpZW50Pwo+Cj4gUHJvYmxlbSB3aXRoIHRoZSBwZXIgY29udGV4dCBvciBwZXIgdm0gZmxhZyBp cyB0aGF0IHlvdSB0aGVuIGRvbid0IGdldAo+IGFueSBpbXBsaWNpdCBzeW5jaHJvbml6YXRpb24g YW55IG1vcmUgd2hlbiBhbm90aGVyIHByb2Nlc3Mgc3RhcnRzIHVzaW5nCj4gdGhlIGJ1ZmZlci4K ClRoYXQgaXMgZXhhY3RseSB3aGF0IEkgd2FudCBmb3IgVnVsa2FuIDopCj4KPiA+IFRoZSBjdXJy ZW50IGFtZGdwdSB1YXBpIGp1c3QgZG9lc24ndCBhbGxvdyBhbnkgb3RoZXIgbW9kZWwgd2l0aG91 dCBhbgo+ID4gZXhwbGljaXQgb3B0LWluLiBTbyBjdXJyZW50IGltcGxpY2l0IHN5bmMgdXNlcnNw YWNlIGp1c3QgaGFzIHRvCj4gPiBvdmVyc3luYywgdGhlcmUncyBub3QgbXVjaCBjaG9pY2UuCj4g Pgo+ID4+IDIpIEhhdmUgYW4gRVhQTElDSVQgZmVuY2Ugb3duZXIgdGhhdCBpcyB1c2VkIGZvciBl eHBsaWNpdCBzdWJtaXNzaW9ucwo+ID4+IHRoYXQgaXMgaWdub3JlZCBieSBBTURHUFVfU1lOQ19O RV9PV05FUi4KPiA+Pgo+ID4+IEJ1dCB0aGlzIGRvZXNuJ3Qgc29sdmUgY3Jvc3MtZHJpdmVyIGlu dGVyYWN0aW9ucyBoZXJlLgo+ID4gWWVhaCBjcm9zcy1kcml2ZXIgaXMgc3RpbGwgZW50aXJlbHkg dW5zb2x2ZWQsIGJlY2F1c2UKPiA+IGFtZGdwdV9ib19leHBsaWNpdF9zeW5jKCkgb24gdGhlIGJv IGRpZG4ndCBzb2x2ZSB0aGF0IGVpdGhlci4KPgo+IEh1aT8gWW91IGhhdmUgbG9zdCBtZS4gV2h5 IGlzIHRoYXQgc3RpbGwgdW5zb2x2ZWQ/CgpUaGUgcGFydCB3ZSdyZSB0cnlpbmcgdG8gc29sdmUg d2l0aCB0aGlzIHBhdGNoIGlzIFZ1bGthbiBzaG91bGQgbm90CnBhcnRpY2lwYXRlIGluIGFueSBp bXBsaWNpdCBzeW5jIGF0IGFsbCB3cnQgc3VibWlzc2lvbnMgKGFuZCB0aGVuCmhhbmRsZSB0aGUg aW1wbGljaXQgc3luYyBmb3IgV1NJIGV4cGxpY2l0bHkgdXNpbmcgdGhlIGZlbmNlCmltcG9ydC9l eHBvcnQgc3R1ZmYgdGhhdCBKYXNvbiB3cm90ZSkuIEFzIGxvbmcgd2UgYWRkIHNoYXJlZCBmZW5j ZXMgdG8KdGhlIGRtYV9yZXN2IHdlIHBhcnRpY2lwYXRlIGluIGltcGxpY2l0IHN5bmMgKGF0IHRo ZSBsZXZlbCBvZiBhbgppbXBsaWNpdCBzeW5jIHJlYWQpIHN0aWxsLCBhdCBsZWFzdCBmcm9tIHRo ZSBwZXJzcGVjdGl2ZSBvZiBsYXRlciBqb2JzCndhaXRpbmcgb24gdGhlc2UgZmVuY2VzLgoKPgo+ IFJlZ2FyZHMsCj4gQ2hyaXN0aWFuLgo+Cj4gPiAtRGFuaWVsCj4gPgo+ID4+PiBDYzogbWVzYS1k ZXZAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCj4gPj4+IENjOiBCYXMgTmlldXdlbmh1aXplbiA8YmFz QGJhc25pZXV3ZW5odWl6ZW4ubmw+Cj4gPj4+IENjOiBEYXZlIEFpcmxpZSA8YWlybGllZEBnbWFp bC5jb20+Cj4gPj4+IENjOiBSb2IgQ2xhcmsgPHJvYmRjbGFya0BjaHJvbWl1bS5vcmc+Cj4gPj4+ IENjOiBLcmlzdGlhbiBILiBLcmlzdGVuc2VuIDxob2Vnc2JlcmdAZ29vZ2xlLmNvbT4KPiA+Pj4g Q2M6IE1pY2hlbCBEw6RuemVyIDxtaWNoZWxAZGFlbnplci5uZXQ+Cj4gPj4+IENjOiBEYW5pZWwg U3RvbmUgPGRhbmllbHNAY29sbGFib3JhLmNvbT4KPiA+Pj4gQ2M6IFN1bWl0IFNlbXdhbCA8c3Vt aXQuc2Vtd2FsQGxpbmFyby5vcmc+Cj4gPj4+IENjOiAiQ2hyaXN0aWFuIEvDtm5pZyIgPGNocmlz dGlhbi5rb2VuaWdAYW1kLmNvbT4KPiA+Pj4gQ2M6IEFsZXggRGV1Y2hlciA8YWxleGFuZGVyLmRl dWNoZXJAYW1kLmNvbT4KPiA+Pj4gQ2M6IERhbmllbCBWZXR0ZXIgPGRhbmllbC52ZXR0ZXJAZmZ3 bGwuY2g+Cj4gPj4+IENjOiBEZWVwYWsgUiBWYXJtYSA8bWgxMmd4MjgyNUBnbWFpbC5jb20+Cj4g Pj4+IENjOiBDaGVuIExpIDxjaGVubGlAdW5pb250ZWNoLmNvbT4KPiA+Pj4gQ2M6IEtldmluIFdh bmcgPGtldmluMS53YW5nQGFtZC5jb20+Cj4gPj4+IENjOiBEZW5uaXMgTGkgPERlbm5pcy5MaUBh bWQuY29tPgo+ID4+PiBDYzogTHViZW4gVHVpa292IDxsdWJlbi50dWlrb3ZAYW1kLmNvbT4KPiA+ Pj4gQ2M6IGxpbmFyby1tbS1zaWdAbGlzdHMubGluYXJvLm9yZwo+ID4+PiBTaWduZWQtb2ZmLWJ5 OiBEYW5pZWwgVmV0dGVyIDxkYW5pZWwudmV0dGVyQGludGVsLmNvbT4KPiA+Pj4gLS0tCj4gPj4+ ICAgZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2NzLmMgIHwgIDcgKysrKystLQo+ ID4+PiAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kcnYuYyB8IDIxICsrKysr KysrKysrKysrKysrKysrKwo+ID4+PiAgIGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdw dV92bS5oICB8ICA2ICsrKysrKwo+ID4+PiAgIGluY2x1ZGUvdWFwaS9kcm0vYW1kZ3B1X2RybS5o ICAgICAgICAgICB8IDEwICsrKysrKysrKysKPiA+Pj4gICA0IGZpbGVzIGNoYW5nZWQsIDQyIGlu c2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCj4gPj4+Cj4gPj4+IGRpZmYgLS1naXQgYS9kcml2 ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfY3MuYyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQv YW1kZ3B1L2FtZGdwdV9jcy5jCj4gPj4+IGluZGV4IDY1ZGYzNGMxNzI2NC4uYzUzODZkMTNlYjRh IDEwMDY0NAo+ID4+PiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfY3Mu Ywo+ID4+PiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfY3MuYwo+ID4+ PiBAQCAtNDk4LDYgKzQ5OCw3IEBAIHN0YXRpYyBpbnQgYW1kZ3B1X2NzX3BhcnNlcl9ib3Moc3Ry dWN0IGFtZGdwdV9jc19wYXJzZXIgKnAsCj4gPj4+ICAgICAgICAgIHN0cnVjdCBhbWRncHVfYm8g KmdkczsKPiA+Pj4gICAgICAgICAgc3RydWN0IGFtZGdwdV9ibyAqZ3dzOwo+ID4+PiAgICAgICAg ICBzdHJ1Y3QgYW1kZ3B1X2JvICpvYTsKPiA+Pj4gKyAgICAgICBib29sIG5vX2ltcGxpY2l0X3N5 bmMgPSBSRUFEX09OQ0UoZnByaXYtPnZtLm5vX2ltcGxpY2l0X3N5bmMpOwo+ID4+PiAgICAgICAg ICBpbnQgcjsKPiA+Pj4KPiA+Pj4gICAgICAgICAgSU5JVF9MSVNUX0hFQUQoJnAtPnZhbGlkYXRl ZCk7Cj4gPj4+IEBAIC01NzcsNyArNTc4LDggQEAgc3RhdGljIGludCBhbWRncHVfY3NfcGFyc2Vy X2JvcyhzdHJ1Y3QgYW1kZ3B1X2NzX3BhcnNlciAqcCwKPiA+Pj4KPiA+Pj4gICAgICAgICAgICAg ICAgICBlLT5ib192YSA9IGFtZGdwdV92bV9ib19maW5kKHZtLCBibyk7Cj4gPj4+Cj4gPj4+IC0g ICAgICAgICAgICAgICBpZiAoYm8tPnRiby5iYXNlLmRtYV9idWYgJiYgIWFtZGdwdV9ib19leHBs aWNpdF9zeW5jKGJvKSkgewo+ID4+PiArICAgICAgICAgICAgICAgaWYgKGJvLT50Ym8uYmFzZS5k bWFfYnVmICYmCj4gPj4+ICsgICAgICAgICAgICAgICAgICAgIShub19pbXBsaWNpdF9zeW5jIHx8 IGFtZGdwdV9ib19leHBsaWNpdF9zeW5jKGJvKSkpIHsKPiA+Pj4gICAgICAgICAgICAgICAgICAg ICAgICAgIGUtPmNoYWluID0gZG1hX2ZlbmNlX2NoYWluX2FsbG9jKCk7Cj4gPj4+ICAgICAgICAg ICAgICAgICAgICAgICAgICBpZiAoIWUtPmNoYWluKSB7Cj4gPj4+ICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIHIgPSAtRU5PTUVNOwo+ID4+PiBAQCAtNjQ5LDYgKzY1MSw3IEBAIHN0 YXRpYyBpbnQgYW1kZ3B1X2NzX3N5bmNfcmluZ3Moc3RydWN0IGFtZGdwdV9jc19wYXJzZXIgKnAp Cj4gPj4+ICAgewo+ID4+PiAgICAgICAgICBzdHJ1Y3QgYW1kZ3B1X2Zwcml2ICpmcHJpdiA9IHAt PmZpbHAtPmRyaXZlcl9wcml2Owo+ID4+PiAgICAgICAgICBzdHJ1Y3QgYW1kZ3B1X2JvX2xpc3Rf ZW50cnkgKmU7Cj4gPj4+ICsgICAgICAgYm9vbCBub19pbXBsaWNpdF9zeW5jID0gUkVBRF9PTkNF KGZwcml2LT52bS5ub19pbXBsaWNpdF9zeW5jKTsKPiA+Pj4gICAgICAgICAgaW50IHI7Cj4gPj4+ Cj4gPj4+ICAgICAgICAgIGxpc3RfZm9yX2VhY2hfZW50cnkoZSwgJnAtPnZhbGlkYXRlZCwgdHYu aGVhZCkgewo+ID4+PiBAQCAtNjU2LDcgKzY1OSw3IEBAIHN0YXRpYyBpbnQgYW1kZ3B1X2NzX3N5 bmNfcmluZ3Moc3RydWN0IGFtZGdwdV9jc19wYXJzZXIgKnApCj4gPj4+ICAgICAgICAgICAgICAg ICAgc3RydWN0IGRtYV9yZXN2ICpyZXN2ID0gYm8tPnRiby5iYXNlLnJlc3Y7Cj4gPj4+ICAgICAg ICAgICAgICAgICAgZW51bSBhbWRncHVfc3luY19tb2RlIHN5bmNfbW9kZTsKPiA+Pj4KPiA+Pj4g LSAgICAgICAgICAgICAgIHN5bmNfbW9kZSA9IGFtZGdwdV9ib19leHBsaWNpdF9zeW5jKGJvKSA/ Cj4gPj4+ICsgICAgICAgICAgICAgICBzeW5jX21vZGUgPSBub19pbXBsaWNpdF9zeW5jIHx8IGFt ZGdwdV9ib19leHBsaWNpdF9zeW5jKGJvKSA/Cj4gPj4+ICAgICAgICAgICAgICAgICAgICAgICAg ICBBTURHUFVfU1lOQ19FWFBMSUNJVCA6IEFNREdQVV9TWU5DX05FX09XTkVSOwo+ID4+PiAgICAg ICAgICAgICAgICAgIHIgPSBhbWRncHVfc3luY19yZXN2KHAtPmFkZXYsICZwLT5qb2ItPnN5bmMs IHJlc3YsIHN5bmNfbW9kZSwKPiA+Pj4gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAgICAmZnByaXYtPnZtKTsKPiA+Pj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQv YW1kZ3B1L2FtZGdwdV9kcnYuYyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9k cnYuYwo+ID4+PiBpbmRleCBjMDgwYmExNWFlNzcuLmY5ODI2MjZiNTMyOCAxMDA2NDQKPiA+Pj4g LS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2Rydi5jCj4gPj4+ICsrKyBi L2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9kcnYuYwo+ID4+PiBAQCAtMTcyNCw2 ICsxNzI0LDI2IEBAIGludCBhbWRncHVfZmlsZV90b19mcHJpdihzdHJ1Y3QgZmlsZSAqZmlscCwg c3RydWN0IGFtZGdwdV9mcHJpdiAqKmZwcml2KQo+ID4+PiAgICAgICAgICByZXR1cm4gMDsKPiA+ Pj4gICB9Cj4gPj4+Cj4gPj4+ICtpbnQgYW1kZ3B1X3NldHBhcmFtX2lvY3RsKHN0cnVjdCBkcm1f ZGV2aWNlICpkZXYsIHZvaWQgKmRhdGEsCj4gPj4+ICsgICAgICAgICAgICAgICAgICAgICAgICAg c3RydWN0IGRybV9maWxlICpmaWxwKQo+ID4+PiArewo+ID4+PiArICAgICAgIHN0cnVjdCBkcm1f YW1kZ3B1X3NldHBhcmFtICpzZXRwYXJhbSA9IGRhdGE7Cj4gPj4+ICsgICAgICAgc3RydWN0IGFt ZGdwdV9mcHJpdiAqZnByaXYgPSBmaWxwLT5kcml2ZXJfcHJpdjsKPiA+Pj4gKwo+ID4+PiArICAg ICAgIHN3aXRjaCAoc2V0cGFyYW0tPnBhcmFtKSB7Cj4gPj4+ICsgICAgICAgY2FzZSBBTURHUFVf U0VUUEFSQU1fTk9fSU1QTElDSVRfU1lOQzoKPiA+Pj4gKyAgICAgICAgICAgICAgIGlmIChzZXRw YXJhbS0+dmFsdWUpCj4gPj4+ICsgICAgICAgICAgICAgICAgICAgICAgIFdSSVRFX09OQ0UoZnBy aXYtPnZtLm5vX2ltcGxpY2l0X3N5bmMsIHRydWUpOwo+ID4+PiArICAgICAgICAgICAgICAgZWxz ZQo+ID4+PiArICAgICAgICAgICAgICAgICAgICAgICBXUklURV9PTkNFKGZwcml2LT52bS5ub19p bXBsaWNpdF9zeW5jLCBmYWxzZSk7Cj4gPj4+ICsgICAgICAgICAgICAgICBicmVhazsKPiA+Pj4g KyAgICAgICBkZWZhdWx0Ogo+ID4+PiArICAgICAgICAgICAgICAgcmV0dXJuIC1FSU5WQUw7Cj4g Pj4+ICsgICAgICAgfQo+ID4+PiArCj4gPj4+ICsgICAgICAgcmV0dXJuIDA7Cj4gPj4+ICt9Cj4g Pj4+ICsKPiA+Pj4gICBjb25zdCBzdHJ1Y3QgZHJtX2lvY3RsX2Rlc2MgYW1kZ3B1X2lvY3Rsc19r bXNbXSA9IHsKPiA+Pj4gICAgICAgICAgRFJNX0lPQ1RMX0RFRl9EUlYoQU1ER1BVX0dFTV9DUkVB VEUsIGFtZGdwdV9nZW1fY3JlYXRlX2lvY3RsLCBEUk1fQVVUSHxEUk1fUkVOREVSX0FMTE9XKSwK PiA+Pj4gICAgICAgICAgRFJNX0lPQ1RMX0RFRl9EUlYoQU1ER1BVX0NUWCwgYW1kZ3B1X2N0eF9p b2N0bCwgRFJNX0FVVEh8RFJNX1JFTkRFUl9BTExPVyksCj4gPj4+IEBAIC0xNzQyLDYgKzE3NjIs NyBAQCBjb25zdCBzdHJ1Y3QgZHJtX2lvY3RsX2Rlc2MgYW1kZ3B1X2lvY3Rsc19rbXNbXSA9IHsK PiA+Pj4gICAgICAgICAgRFJNX0lPQ1RMX0RFRl9EUlYoQU1ER1BVX0dFTV9WQSwgYW1kZ3B1X2dl bV92YV9pb2N0bCwgRFJNX0FVVEh8RFJNX1JFTkRFUl9BTExPVyksCj4gPj4+ICAgICAgICAgIERS TV9JT0NUTF9ERUZfRFJWKEFNREdQVV9HRU1fT1AsIGFtZGdwdV9nZW1fb3BfaW9jdGwsIERSTV9B VVRIfERSTV9SRU5ERVJfQUxMT1cpLAo+ID4+PiAgICAgICAgICBEUk1fSU9DVExfREVGX0RSVihB TURHUFVfR0VNX1VTRVJQVFIsIGFtZGdwdV9nZW1fdXNlcnB0cl9pb2N0bCwgRFJNX0FVVEh8RFJN X1JFTkRFUl9BTExPVyksCj4gPj4+ICsgICAgICAgRFJNX0lPQ1RMX0RFRl9EUlYoQU1ER1BVX1NF VFBBUkFNLCBhbWRncHVfc2V0cGFyYW1faW9jdGwsIERSTV9BVVRIfERSTV9SRU5ERVJfQUxMT1cp LAo+ID4+PiAgIH07Cj4gPj4+Cj4gPj4+ICAgc3RhdGljIGNvbnN0IHN0cnVjdCBkcm1fZHJpdmVy IGFtZGdwdV9rbXNfZHJpdmVyID0gewo+ID4+PiBkaWZmIC0tZ2l0IGEvZHJpdmVycy9ncHUvZHJt L2FtZC9hbWRncHUvYW1kZ3B1X3ZtLmggYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRn cHVfdm0uaAo+ID4+PiBpbmRleCBkZGI4NWE4NWNiYmEuLjBlOGM0NDBjNjMwMyAxMDA2NDQKPiA+ Pj4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X3ZtLmgKPiA+Pj4gKysr IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X3ZtLmgKPiA+Pj4gQEAgLTMyMSw2 ICszMjEsMTIgQEAgc3RydWN0IGFtZGdwdV92bSB7Cj4gPj4+ICAgICAgICAgIGJvb2wgICAgICAg ICAgICAgICAgICAgIGJ1bGtfbW92ZWFibGU7Cj4gPj4+ICAgICAgICAgIC8qIEZsYWcgdG8gaW5k aWNhdGUgaWYgVk0gaXMgdXNlZCBmb3IgY29tcHV0ZSAqLwo+ID4+PiAgICAgICAgICBib29sICAg ICAgICAgICAgICAgICAgICBpc19jb21wdXRlX2NvbnRleHQ7Cj4gPj4+ICsgICAgICAgLyoKPiA+ Pj4gKyAgICAgICAgKiBGbGFnIHRvIGluZGljYXRlIHdoZXRoZXIgaW1wbGljaXQgc3luYyBzaG91 bGQgYWx3YXlzIGJlIHNraXBwZWQgb24KPiA+Pj4gKyAgICAgICAgKiB0aGlzIGNvbnRleHQuIFdl IGRvIG5vdCBjYXJlIGFib3V0IHJhY2VzIGF0IGFsbCwgdXNlcnNwYWNlIGlzIGFsbG93ZWQKPiA+ Pj4gKyAgICAgICAgKiB0byBzaG9vdCBpdHNlbGYgd2l0aCBpbXBsaWNpdCBzeW5jIHRvIGl0cyBm dWxsZXN0IGxpa2luZy4KPiA+Pj4gKyAgICAgICAgKi8KPiA+Pj4gKyAgICAgICBib29sIG5vX2lt cGxpY2l0X3N5bmM7Cj4gPj4+ICAgfTsKPiA+Pj4KPiA+Pj4gICBzdHJ1Y3QgYW1kZ3B1X3ZtX21h bmFnZXIgewo+ID4+PiBkaWZmIC0tZ2l0IGEvaW5jbHVkZS91YXBpL2RybS9hbWRncHVfZHJtLmgg Yi9pbmNsdWRlL3VhcGkvZHJtL2FtZGdwdV9kcm0uaAo+ID4+PiBpbmRleCAwY2JkMTU0MGFlYWMu LjllYWUyNDVjMTRkNiAxMDA2NDQKPiA+Pj4gLS0tIGEvaW5jbHVkZS91YXBpL2RybS9hbWRncHVf ZHJtLmgKPiA+Pj4gKysrIGIvaW5jbHVkZS91YXBpL2RybS9hbWRncHVfZHJtLmgKPiA+Pj4gQEAg LTU0LDYgKzU0LDcgQEAgZXh0ZXJuICJDIiB7Cj4gPj4+ICAgI2RlZmluZSBEUk1fQU1ER1BVX1ZN ICAgICAgICAgICAgICAgICAgMHgxMwo+ID4+PiAgICNkZWZpbmUgRFJNX0FNREdQVV9GRU5DRV9U T19IQU5ETEUgICAgIDB4MTQKPiA+Pj4gICAjZGVmaW5lIERSTV9BTURHUFVfU0NIRUQgICAgICAg ICAgICAgICAweDE1Cj4gPj4+ICsjZGVmaW5lIERSTV9BTURHUFVfU0VUUEFSQU0gICAgICAgICAg ICAweDE2Cj4gPj4+Cj4gPj4+ICAgI2RlZmluZSBEUk1fSU9DVExfQU1ER1BVX0dFTV9DUkVBVEUg ICAgRFJNX0lPV1IoRFJNX0NPTU1BTkRfQkFTRSArIERSTV9BTURHUFVfR0VNX0NSRUFURSwgdW5p b24gZHJtX2FtZGdwdV9nZW1fY3JlYXRlKQo+ID4+PiAgICNkZWZpbmUgRFJNX0lPQ1RMX0FNREdQ VV9HRU1fTU1BUCAgICAgIERSTV9JT1dSKERSTV9DT01NQU5EX0JBU0UgKyBEUk1fQU1ER1BVX0dF TV9NTUFQLCB1bmlvbiBkcm1fYW1kZ3B1X2dlbV9tbWFwKQo+ID4+PiBAQCAtNzEsNiArNzIsNyBA QCBleHRlcm4gIkMiIHsKPiA+Pj4gICAjZGVmaW5lIERSTV9JT0NUTF9BTURHUFVfVk0gICAgICAg ICAgICBEUk1fSU9XUihEUk1fQ09NTUFORF9CQVNFICsgRFJNX0FNREdQVV9WTSwgdW5pb24gZHJt X2FtZGdwdV92bSkKPiA+Pj4gICAjZGVmaW5lIERSTV9JT0NUTF9BTURHUFVfRkVOQ0VfVE9fSEFO RExFIERSTV9JT1dSKERSTV9DT01NQU5EX0JBU0UgKyBEUk1fQU1ER1BVX0ZFTkNFX1RPX0hBTkRM RSwgdW5pb24gZHJtX2FtZGdwdV9mZW5jZV90b19oYW5kbGUpCj4gPj4+ICAgI2RlZmluZSBEUk1f SU9DVExfQU1ER1BVX1NDSEVEICAgICAgICAgRFJNX0lPVyhEUk1fQ09NTUFORF9CQVNFICsgRFJN X0FNREdQVV9TQ0hFRCwgdW5pb24gZHJtX2FtZGdwdV9zY2hlZCkKPiA+Pj4gKyNkZWZpbmUgRFJN X0lPQ1RMX0FNREdQVV9TRVRQQVJBTSAgICAgIERSTV9JT1coRFJNX0NPTU1BTkRfQkFTRSArIERS TV9BTURHUFVfU0VUUEFSQU0sIHN0cnVjdCBkcm1fYW1kZ3B1X3NldHBhcmFtKQo+ID4+Pgo+ID4+ PiAgIC8qKgo+ID4+PiAgICAqIERPQzogbWVtb3J5IGRvbWFpbnMKPiA+Pj4gQEAgLTMwNiw2ICsz MDgsMTQgQEAgdW5pb24gZHJtX2FtZGdwdV9zY2hlZCB7Cj4gPj4+ICAgICAgICAgIHN0cnVjdCBk cm1fYW1kZ3B1X3NjaGVkX2luIGluOwo+ID4+PiAgIH07Cj4gPj4+Cj4gPj4+ICsjZGVmaW5lIEFN REdQVV9TRVRQQVJBTV9OT19JTVBMSUNJVF9TWU5DICAgICAgIDEKPiA+Pj4gKwo+ID4+PiArc3Ry dWN0IGRybV9hbWRncHVfc2V0cGFyYW0gewo+ID4+PiArICAgICAgIC8qIEFNREdQVV9TRVRQQVJB TV8qICovCj4gPj4+ICsgICAgICAgX191MzIgICBwYXJhbTsKPiA+Pj4gKyAgICAgICBfX3UzMiAg IHZhbHVlOwo+ID4+PiArfTsKPiA+Pj4gKwo+ID4+PiAgIC8qCj4gPj4+ICAgICogVGhpcyBpcyBu b3QgYSByZWxpYWJsZSBBUEkgYW5kIHlvdSBzaG91bGQgZXhwZWN0IGl0IHRvIGZhaWwgZm9yIGFu eQo+ID4+PiAgICAqIG51bWJlciBvZiByZWFzb25zIGFuZCBoYXZlIGZhbGxiYWNrIHBhdGggdGhh dCBkbyBub3QgdXNlIHVzZXJwdHIgdG8KPiA+Pj4gLS0KPiA+Pj4gMi4zMi4wLnJjMgo+ID4+Pgo+ ID4KPiA+Cj4KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18K SW50ZWwtZ2Z4IG1haWxpbmcgbGlzdApJbnRlbC1nZnhAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCmh0 dHBzOi8vbGlzdHMuZnJlZWRlc2t0b3Aub3JnL21haWxtYW4vbGlzdGluZm8vaW50ZWwtZ2Z4Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD549C48BE5 for ; Wed, 23 Jun 2021 13:38:48 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 148D361075 for ; Wed, 23 Jun 2021 13:38:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 148D361075 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=basnieuwenhuizen.nl Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C4FCB6E8E4; Wed, 23 Jun 2021 13:38:46 +0000 (UTC) Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0390189FEA for ; Wed, 23 Jun 2021 13:38:44 +0000 (UTC) Received: by mail-io1-xd36.google.com with SMTP id r12so3436151ioa.7 for ; Wed, 23 Jun 2021 06:38:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=basnieuwenhuizen.nl; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=pbUycUN2rLw/NpVnHTIINetmdItB8ttstk1+aiKBgsI=; b=dBBJPAqrXkKWSND+v1EL2tus27yTDJPdFsHwNGt83YekplISakyk6r63tgTzokXwjl ahb3TlHilV23gBBNK99qmug87Vg8QzXVaLnn1BVMd3ReQKn/g1sGT/xH0tJmsUm00Qow aY9PQUWyJnyZFMlymGL9uSIwyK7CfvlvWQgQuWdtyNhm8YFYCsogTGNi33nNdG1c0qzI q7s1mH61NP6unfToNoXDJ/tQqGR7dJtSiLB4mlK6jYzM56zhsZwALAFkY/VulDVMnMQR Xfs1Xsjtsls+2M2+hnROYU0StIWPfQk10F8XIE8HB6VfycxUEcL1pF5CPs2jav24s0to 2kqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=pbUycUN2rLw/NpVnHTIINetmdItB8ttstk1+aiKBgsI=; b=Y3gqxvy7L06eRWXxO3aDOOFx5/5ZqoCVS6IBiD07B8jQtvV4yb5u8a54V5w2XhJxrB 6kVejNbpvchEkUOBn8DfIh1x5VI/rxGQvRT3xPsuf5PSeCH+reHr8MZG928bBmXxTZKq MfJ/1tYP/Tcwe6bWV/2uAI/3Jcr6EgpJldGxqcBjn6drz94hnqdb4+Wlh7stBsM8XP/e gdrpEeojLHDKWNtOHekJaUfzkdyeGdvd1vmODq88M4BfIuyzcen13LDlJyKPnZBn6j4I WQ3eSn6J0vOMBeejcy9Pa0q5++EPEwPbEY19urHxAq+JP/3VcbgF09iwpiZ8m8ry2g0+ Qn8Q== X-Gm-Message-State: AOAM533Tcl7MIIE1oLDSiMtRAIuU9hf/tp/VqoLofqcondtVH+EC9wYE 0qPomeGX6hWIfwv7mdWE1yLASbBCqMZKt4AYMVpTUA== X-Google-Smtp-Source: ABdhPJxyvGrbQ2FKDblt/MdAvighHagEciQVyyHFv/qmVSXPRKwn5bAoCENV10JBsnt+9ZJ5zgZgRhsNGgy0+22w9Z4= X-Received: by 2002:a02:ca4a:: with SMTP id i10mr8988384jal.141.1624455524349; Wed, 23 Jun 2021 06:38:44 -0700 (PDT) MIME-Version: 1.0 References: <20210622165511.3169559-1-daniel.vetter@ffwll.ch> <20210622165511.3169559-16-daniel.vetter@ffwll.ch> <3bf45006-4256-763d-601b-3a25a7057820@amd.com> In-Reply-To: <3bf45006-4256-763d-601b-3a25a7057820@amd.com> From: Bas Nieuwenhuizen Date: Wed, 23 Jun 2021 15:38:39 +0200 Message-ID: Subject: Re: [PATCH 15/15] RFC: drm/amdgpu: Implement a proper implicit fencing uapi To: =?UTF-8?Q?Christian_K=C3=B6nig?= Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , Daniel Stone , Daniel Vetter , Intel Graphics Development , Kevin Wang , DRI Development , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Luben Tuikov , "Kristian H . Kristensen" , Chen Li , Daniel Vetter , Alex Deucher , mesa-dev , =?UTF-8?Q?Michel_D=C3=A4nzer?= , Dennis Li , Deepak R Varma Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, Jun 23, 2021 at 2:59 PM Christian K=C3=B6nig wrote: > > Am 23.06.21 um 14:18 schrieb Daniel Vetter: > > On Wed, Jun 23, 2021 at 11:45 AM Bas Nieuwenhuizen > > wrote: > >> On Tue, Jun 22, 2021 at 6:55 PM Daniel Vetter = wrote: > >>> WARNING: Absolutely untested beyond "gcc isn't dying in agony". > >>> > >>> Implicit fencing done properly needs to treat the implicit fencing > >>> slots like a funny kind of IPC mailbox. In other words it needs to be > >>> explicitly. This is the only way it will mesh well with explicit > >>> fencing userspace like vk, and it's also the bare minimum required to > >>> be able to manage anything else that wants to use the same buffer on > >>> multiple engines in parallel, and still be able to share it through > >>> implicit sync. > >>> > >>> amdgpu completely lacks such an uapi. Fix this. > >>> > >>> Luckily the concept of ignoring implicit fences exists already, and > >>> takes care of all the complexities of making sure that non-optional > >>> fences (like bo moves) are not ignored. This support was added in > >>> > >>> commit 177ae09b5d699a5ebd1cafcee78889db968abf54 > >>> Author: Andres Rodriguez > >>> Date: Fri Sep 15 20:44:06 2017 -0400 > >>> > >>> drm/amdgpu: introduce AMDGPU_GEM_CREATE_EXPLICIT_SYNC v2 > >>> > >>> Unfortuantely it's the wrong semantics, because it's a bo flag and > >>> disables implicit sync on an allocated buffer completely. > >>> > >>> We _do_ want implicit sync, but control it explicitly. For this we > >>> need a flag on the drm_file, so that a given userspace (like vulkan) > >>> can manage the implicit sync slots explicitly. The other side of the > >>> pipeline (compositor, other process or just different stage in a medi= a > >>> pipeline in the same process) can then either do the same, or fully > >>> participate in the implicit sync as implemented by the kernel by > >>> default. > >>> > >>> By building on the existing flag for buffers we avoid any issues with > >>> opening up additional security concerns - anything this new flag here > >>> allows is already. > >>> > >>> All drivers which supports this concept of a userspace-specific > >>> opt-out of implicit sync have a flag in their CS ioctl, but in realit= y > >>> that turned out to be a bit too inflexible. See the discussion below, > >>> let's try to do a bit better for amdgpu. > >>> > >>> This alone only allows us to completely avoid any stalls due to > >>> implicit sync, it does not yet allow us to use implicit sync as a > >>> strange form of IPC for sync_file. > >>> > >>> For that we need two more pieces: > >>> > >>> - a way to get the current implicit sync fences out of a buffer. Coul= d > >>> be done in a driver ioctl, but everyone needs this, and generally = a > >>> dma-buf is involved anyway to establish the sharing. So an ioctl o= n > >>> the dma-buf makes a ton more sense: > >>> > >>> https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%= 2Flore.kernel.org%2Fdri-devel%2F20210520190007.534046-4-jason%40jlekstrand.= net%2F&data=3D04%7C01%7Cchristian.koenig%40amd.com%7Cf026055f523d4e4df9= 5b08d936410e39%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637600475351085= 536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1= haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DgUnM8%2Fulx%2B%2BDLxByO%2F0V3cLqt%2F= c2unWjizEpptQqM8g%3D&reserved=3D0 > >>> > >>> Current drivers in upstream solves this by having the opt-out flag > >>> on their CS ioctl. This has the downside that very often the CS > >>> which must actually stall for the implicit fence is run a while > >>> after the implicit fence point was logically sampled per the api > >>> spec (vk passes an explicit syncobj around for that afaiui), and s= o > >>> results in oversync. Converting the implicit sync fences into a > >>> snap-shot sync_file is actually accurate. > >>> > >>> - Simillar we need to be able to set the exclusive implicit fence. > >>> Current drivers again do this with a CS ioctl flag, with again the > >>> same problems that the time the CS happens additional dependencies > >>> have been added. An explicit ioctl to only insert a sync_file (whi= le > >>> respecting the rules for how exclusive and shared fence slots must > >>> be update in struct dma_resv) is much better. This is proposed her= e: > >>> > >>> https://nam11.safelinks.protection.outlook.com/?url=3Dhttps%3A%2F%= 2Flore.kernel.org%2Fdri-devel%2F20210520190007.534046-5-jason%40jlekstrand.= net%2F&data=3D04%7C01%7Cchristian.koenig%40amd.com%7Cf026055f523d4e4df9= 5b08d936410e39%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637600475351085= 536%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1= haWwiLCJXVCI6Mn0%3D%7C1000&sdata=3DwFGNyeL1YSpkebf1L1DDb2euihf1fvmR9G8c= fywrpVU%3D&reserved=3D0 > >>> > >>> These three pieces together allow userspace to fully control implicit > >>> fencing and remove all unecessary stall points due to them. > >>> > >>> Well, as much as the implicit fencing model fundamentally allows: > >>> There is only one set of fences, you can only choose to sync against > >>> only writers (exclusive slot), or everyone. Hence suballocating > >>> multiple buffers or anything else like this is fundamentally not > >>> possible, and can only be fixed by a proper explicit fencing model. > >>> > >>> Aside from that caveat this model gets implicit fencing as closely to > >>> explicit fencing semantics as possible: > >>> > >>> On the actual implementation I opted for a simple setparam ioctl, no > >>> locking (just atomic reads/writes) for simplicity. There is a nice > >>> flag parameter in the VM ioctl which we could use, except: > >>> - it's not checked, so userspace likely passes garbage > >>> - there's already a comment that userspace _does_ pass garbage in the > >>> priority field > >>> So yeah unfortunately this flag parameter for setting vm flags is > >>> useless, and we need to hack up a new one. > >>> > >>> v2: Explain why a new SETPARAM (Jason) > >>> > >>> v3: Bas noticed I forgot to hook up the dependency-side shortcut. We > >>> need both, or this doesn't do much. > >>> > >>> v4: Rebase over the amdgpu patch to always set the implicit sync > >>> fences. > >> So I think there is still a case missing in this implementation. > >> Consider these 3 cases > >> > >> (format: a->b: b waits on a. Yes, I know arrows are hard) > >> > >> explicit->explicit: This doesn't wait now, which is good > >> Implicit->explicit: This doesn't wait now, which is good > >> explicit->implicit : This still waits as the explicit submission still > >> adds shared fences and most things that set an exclusive fence for > >> implicit sync will hence wait on it. > >> > >> This is probably good enough for what radv needs now but also sounds > >> like a risk wrt baking in new uapi behavior that we don't want to be > >> the end result. > >> > >> Within AMDGPU this is probably solvable in two ways: > >> > >> 1) Downgrade AMDGPU_SYNC_NE_OWNER to AMDGPU_SYNC_EXPLICIT for shared f= ences. > > I'm not sure that works. I think the right fix is that radeonsi also > > switches to this model, with maybe a per-bo CS flag to set indicate > > write access, to cut down on the number of ioctls that are needed > > otherwise on shared buffers. This per-bo flag would essentially select > > between SYNC_NE_OWNER and SYNC_EXPLICIT on a per-buffer basis. > > Yeah, but I'm still not entirely sure why that approach isn't sufficient? > > Problem with the per context or per vm flag is that you then don't get > any implicit synchronization any more when another process starts using > the buffer. That is exactly what I want for Vulkan :) > > > The current amdgpu uapi just doesn't allow any other model without an > > explicit opt-in. So current implicit sync userspace just has to > > oversync, there's not much choice. > > > >> 2) Have an EXPLICIT fence owner that is used for explicit submissions > >> that is ignored by AMDGPU_SYNC_NE_OWNER. > >> > >> But this doesn't solve cross-driver interactions here. > > Yeah cross-driver is still entirely unsolved, because > > amdgpu_bo_explicit_sync() on the bo didn't solve that either. > > Hui? You have lost me. Why is that still unsolved? The part we're trying to solve with this patch is Vulkan should not participate in any implicit sync at all wrt submissions (and then handle the implicit sync for WSI explicitly using the fence import/export stuff that Jason wrote). As long we add shared fences to the dma_resv we participate in implicit sync (at the level of an implicit sync read) still, at least from the perspective of later jobs waiting on these fences. > > Regards, > Christian. > > > -Daniel > > > >>> Cc: mesa-dev@lists.freedesktop.org > >>> Cc: Bas Nieuwenhuizen > >>> Cc: Dave Airlie > >>> Cc: Rob Clark > >>> Cc: Kristian H. Kristensen > >>> Cc: Michel D=C3=A4nzer > >>> Cc: Daniel Stone > >>> Cc: Sumit Semwal > >>> Cc: "Christian K=C3=B6nig" > >>> Cc: Alex Deucher > >>> Cc: Daniel Vetter > >>> Cc: Deepak R Varma > >>> Cc: Chen Li > >>> Cc: Kevin Wang > >>> Cc: Dennis Li > >>> Cc: Luben Tuikov > >>> Cc: linaro-mm-sig@lists.linaro.org > >>> Signed-off-by: Daniel Vetter > >>> --- > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 7 +++++-- > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 21 +++++++++++++++++++++ > >>> drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 6 ++++++ > >>> include/uapi/drm/amdgpu_drm.h | 10 ++++++++++ > >>> 4 files changed, 42 insertions(+), 2 deletions(-) > >>> > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm= /amd/amdgpu/amdgpu_cs.c > >>> index 65df34c17264..c5386d13eb4a 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > >>> @@ -498,6 +498,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_= parser *p, > >>> struct amdgpu_bo *gds; > >>> struct amdgpu_bo *gws; > >>> struct amdgpu_bo *oa; > >>> + bool no_implicit_sync =3D READ_ONCE(fpriv->vm.no_implicit_syn= c); > >>> int r; > >>> > >>> INIT_LIST_HEAD(&p->validated); > >>> @@ -577,7 +578,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_= parser *p, > >>> > >>> e->bo_va =3D amdgpu_vm_bo_find(vm, bo); > >>> > >>> - if (bo->tbo.base.dma_buf && !amdgpu_bo_explicit_sync(= bo)) { > >>> + if (bo->tbo.base.dma_buf && > >>> + !(no_implicit_sync || amdgpu_bo_explicit_sync(bo)= )) { > >>> e->chain =3D dma_fence_chain_alloc(); > >>> if (!e->chain) { > >>> r =3D -ENOMEM; > >>> @@ -649,6 +651,7 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_= parser *p) > >>> { > >>> struct amdgpu_fpriv *fpriv =3D p->filp->driver_priv; > >>> struct amdgpu_bo_list_entry *e; > >>> + bool no_implicit_sync =3D READ_ONCE(fpriv->vm.no_implicit_syn= c); > >>> int r; > >>> > >>> list_for_each_entry(e, &p->validated, tv.head) { > >>> @@ -656,7 +659,7 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_= parser *p) > >>> struct dma_resv *resv =3D bo->tbo.base.resv; > >>> enum amdgpu_sync_mode sync_mode; > >>> > >>> - sync_mode =3D amdgpu_bo_explicit_sync(bo) ? > >>> + sync_mode =3D no_implicit_sync || amdgpu_bo_explicit_= sync(bo) ? > >>> AMDGPU_SYNC_EXPLICIT : AMDGPU_SYNC_NE_OWNER; > >>> r =3D amdgpu_sync_resv(p->adev, &p->job->sync, resv,= sync_mode, > >>> &fpriv->vm); > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/dr= m/amd/amdgpu/amdgpu_drv.c > >>> index c080ba15ae77..f982626b5328 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c > >>> @@ -1724,6 +1724,26 @@ int amdgpu_file_to_fpriv(struct file *filp, st= ruct amdgpu_fpriv **fpriv) > >>> return 0; > >>> } > >>> > >>> +int amdgpu_setparam_ioctl(struct drm_device *dev, void *data, > >>> + struct drm_file *filp) > >>> +{ > >>> + struct drm_amdgpu_setparam *setparam =3D data; > >>> + struct amdgpu_fpriv *fpriv =3D filp->driver_priv; > >>> + > >>> + switch (setparam->param) { > >>> + case AMDGPU_SETPARAM_NO_IMPLICIT_SYNC: > >>> + if (setparam->value) > >>> + WRITE_ONCE(fpriv->vm.no_implicit_sync, true); > >>> + else > >>> + WRITE_ONCE(fpriv->vm.no_implicit_sync, false)= ; > >>> + break; > >>> + default: > >>> + return -EINVAL; > >>> + } > >>> + > >>> + return 0; > >>> +} > >>> + > >>> const struct drm_ioctl_desc amdgpu_ioctls_kms[] =3D { > >>> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_CREATE, amdgpu_gem_create_ioctl= , DRM_AUTH|DRM_RENDER_ALLOW), > >>> DRM_IOCTL_DEF_DRV(AMDGPU_CTX, amdgpu_ctx_ioctl, DRM_AUTH|DRM= _RENDER_ALLOW), > >>> @@ -1742,6 +1762,7 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[]= =3D { > >>> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_VA, amdgpu_gem_va_ioctl, DRM_AU= TH|DRM_RENDER_ALLOW), > >>> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_OP, amdgpu_gem_op_ioctl, DRM_AU= TH|DRM_RENDER_ALLOW), > >>> DRM_IOCTL_DEF_DRV(AMDGPU_GEM_USERPTR, amdgpu_gem_userptr_ioc= tl, DRM_AUTH|DRM_RENDER_ALLOW), > >>> + DRM_IOCTL_DEF_DRV(AMDGPU_SETPARAM, amdgpu_setparam_ioctl, DRM= _AUTH|DRM_RENDER_ALLOW), > >>> }; > >>> > >>> static const struct drm_driver amdgpu_kms_driver =3D { > >>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm= /amd/amdgpu/amdgpu_vm.h > >>> index ddb85a85cbba..0e8c440c6303 100644 > >>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h > >>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h > >>> @@ -321,6 +321,12 @@ struct amdgpu_vm { > >>> bool bulk_moveable; > >>> /* Flag to indicate if VM is used for compute */ > >>> bool is_compute_context; > >>> + /* > >>> + * Flag to indicate whether implicit sync should always be sk= ipped on > >>> + * this context. We do not care about races at all, userspace= is allowed > >>> + * to shoot itself with implicit sync to its fullest liking. > >>> + */ > >>> + bool no_implicit_sync; > >>> }; > >>> > >>> struct amdgpu_vm_manager { > >>> diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_= drm.h > >>> index 0cbd1540aeac..9eae245c14d6 100644 > >>> --- a/include/uapi/drm/amdgpu_drm.h > >>> +++ b/include/uapi/drm/amdgpu_drm.h > >>> @@ -54,6 +54,7 @@ extern "C" { > >>> #define DRM_AMDGPU_VM 0x13 > >>> #define DRM_AMDGPU_FENCE_TO_HANDLE 0x14 > >>> #define DRM_AMDGPU_SCHED 0x15 > >>> +#define DRM_AMDGPU_SETPARAM 0x16 > >>> > >>> #define DRM_IOCTL_AMDGPU_GEM_CREATE DRM_IOWR(DRM_COMMAND_BASE + = DRM_AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create) > >>> #define DRM_IOCTL_AMDGPU_GEM_MMAP DRM_IOWR(DRM_COMMAND_BASE + = DRM_AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap) > >>> @@ -71,6 +72,7 @@ extern "C" { > >>> #define DRM_IOCTL_AMDGPU_VM DRM_IOWR(DRM_COMMAND_BASE + = DRM_AMDGPU_VM, union drm_amdgpu_vm) > >>> #define DRM_IOCTL_AMDGPU_FENCE_TO_HANDLE DRM_IOWR(DRM_COMMAND_BASE = + DRM_AMDGPU_FENCE_TO_HANDLE, union drm_amdgpu_fence_to_handle) > >>> #define DRM_IOCTL_AMDGPU_SCHED DRM_IOW(DRM_COMMAND_BASE + D= RM_AMDGPU_SCHED, union drm_amdgpu_sched) > >>> +#define DRM_IOCTL_AMDGPU_SETPARAM DRM_IOW(DRM_COMMAND_BASE + DR= M_AMDGPU_SETPARAM, struct drm_amdgpu_setparam) > >>> > >>> /** > >>> * DOC: memory domains > >>> @@ -306,6 +308,14 @@ union drm_amdgpu_sched { > >>> struct drm_amdgpu_sched_in in; > >>> }; > >>> > >>> +#define AMDGPU_SETPARAM_NO_IMPLICIT_SYNC 1 > >>> + > >>> +struct drm_amdgpu_setparam { > >>> + /* AMDGPU_SETPARAM_* */ > >>> + __u32 param; > >>> + __u32 value; > >>> +}; > >>> + > >>> /* > >>> * This is not a reliable API and you should expect it to fail for = any > >>> * number of reasons and have fallback path that do not use userptr= to > >>> -- > >>> 2.32.0.rc2 > >>> > > > > >