From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBA26C49EA5 for ; Wed, 23 Jun 2021 12:18:57 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5BCD861002 for ; Wed, 23 Jun 2021 12:18:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5BCD861002 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=intel-gfx-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 97BC76E8CF; Wed, 23 Jun 2021 12:18:50 +0000 (UTC) Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com [IPv6:2607:f8b0:4864:20::231]) by gabe.freedesktop.org (Postfix) with ESMTPS id C238F6E8CF for ; Wed, 23 Jun 2021 12:18:48 +0000 (UTC) Received: by mail-oi1-x231.google.com with SMTP id 84so3129347oie.2 for ; Wed, 23 Jun 2021 05:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=DhZ7INqfjMkrIqqscqZr1FgEv7Thc3LBimm6s6d3jWQ=; b=Tpxb+E8Xh/onR1scIQsva+bFRIrA1PacGN/9EtTOTlr4VOl+hpcdeBctINqa5cuTfL 00yGrH9/eRZsi4JGrEO4mmvcOynx/jERBJ282uGidC5H1orZtiGlGI39vBUazGTB1Zqo 7ti8gIS8xas7gRR1dL9cgcTmv22hePZUNtomA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=DhZ7INqfjMkrIqqscqZr1FgEv7Thc3LBimm6s6d3jWQ=; b=ZF2vSwImtbMqe0XqHTr1olnNR2gK3nkeCkt1V3lbETLDw6xUo0xWRyhqsakvSQV0hd phrAFWY9KMbJPbDINZ2UdseLRa6Jy4CSM2JPDM6czqBXe4KykcR6/HSx0jt/wMTya7CN Vir2TwBu8mv+dAj+gPmxdesp5QxzfwrN4N+R+W23TgxBR669GL0z41AwoVPsXB5K58za s/gjb6LX/Gaddclc6FCduaC6wWoVgYWxTsflg9rpT2AjrOFPaNxNhghJghAxSh6Zs2CS Q1wak47364/hyquQZxUmAD4feD9yht1nGwXVY0R3PFun6oeciDSrHtv1zxq5VV0p1DZn Stdw== X-Gm-Message-State: AOAM533r7SAmTSpsdWmk6/Gzq/nPhlyLitLEgpPYom2J6K5xRATusboe hyl4wd6/e21VTHWRyV3e+HWTxAZkyVqTsT5IAX4XjQ== X-Google-Smtp-Source: ABdhPJyIkt4ftwgh1jZOld/prk5IZrQQ2XEpd2Y37zmuecslQf4c0cxS3DRuv+GAr2BBxh/u/cRfm8sa6sV+li/1BOQ= X-Received: by 2002:a54:4889:: with SMTP id r9mr2930612oic.101.1624450728097; Wed, 23 Jun 2021 05:18:48 -0700 (PDT) MIME-Version: 1.0 References: <20210622165511.3169559-1-daniel.vetter@ffwll.ch> <20210622165511.3169559-16-daniel.vetter@ffwll.ch> In-Reply-To: From: Daniel Vetter Date: Wed, 23 Jun 2021 14:18:35 +0200 Message-ID: To: Bas Nieuwenhuizen Subject: Re: [Intel-gfx] [PATCH 15/15] RFC: drm/amdgpu: Implement a proper implicit fencing uapi X-BeenThere: intel-gfx@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Intel graphics driver community testing & development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , Daniel Stone , =?UTF-8?Q?Christian_K=C3=B6nig?= , Intel Graphics Development , Kevin Wang , DRI Development , Sumit Semwal , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Luben Tuikov , "Kristian H . Kristensen" , Chen Li , Daniel Vetter , Alex Deucher , mesa-dev , Dave Airlie , =?UTF-8?Q?Michel_D=C3=A4nzer?= , Dennis Li , Deepak R Varma Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: intel-gfx-bounces@lists.freedesktop.org Sender: "Intel-gfx" T24gV2VkLCBKdW4gMjMsIDIwMjEgYXQgMTE6NDUgQU0gQmFzIE5pZXV3ZW5odWl6ZW4KPGJhc0Bi YXNuaWV1d2VuaHVpemVuLm5sPiB3cm90ZToKPgo+IE9uIFR1ZSwgSnVuIDIyLCAyMDIxIGF0IDY6 NTUgUE0gRGFuaWVsIFZldHRlciA8ZGFuaWVsLnZldHRlckBmZndsbC5jaD4gd3JvdGU6Cj4gPgo+ ID4gV0FSTklORzogQWJzb2x1dGVseSB1bnRlc3RlZCBiZXlvbmQgImdjYyBpc24ndCBkeWluZyBp biBhZ29ueSIuCj4gPgo+ID4gSW1wbGljaXQgZmVuY2luZyBkb25lIHByb3Blcmx5IG5lZWRzIHRv IHRyZWF0IHRoZSBpbXBsaWNpdCBmZW5jaW5nCj4gPiBzbG90cyBsaWtlIGEgZnVubnkga2luZCBv ZiBJUEMgbWFpbGJveC4gSW4gb3RoZXIgd29yZHMgaXQgbmVlZHMgdG8gYmUKPiA+IGV4cGxpY2l0 bHkuIFRoaXMgaXMgdGhlIG9ubHkgd2F5IGl0IHdpbGwgbWVzaCB3ZWxsIHdpdGggZXhwbGljaXQK PiA+IGZlbmNpbmcgdXNlcnNwYWNlIGxpa2UgdmssIGFuZCBpdCdzIGFsc28gdGhlIGJhcmUgbWlu aW11bSByZXF1aXJlZCB0bwo+ID4gYmUgYWJsZSB0byBtYW5hZ2UgYW55dGhpbmcgZWxzZSB0aGF0 IHdhbnRzIHRvIHVzZSB0aGUgc2FtZSBidWZmZXIgb24KPiA+IG11bHRpcGxlIGVuZ2luZXMgaW4g cGFyYWxsZWwsIGFuZCBzdGlsbCBiZSBhYmxlIHRvIHNoYXJlIGl0IHRocm91Z2gKPiA+IGltcGxp Y2l0IHN5bmMuCj4gPgo+ID4gYW1kZ3B1IGNvbXBsZXRlbHkgbGFja3Mgc3VjaCBhbiB1YXBpLiBG aXggdGhpcy4KPiA+Cj4gPiBMdWNraWx5IHRoZSBjb25jZXB0IG9mIGlnbm9yaW5nIGltcGxpY2l0 IGZlbmNlcyBleGlzdHMgYWxyZWFkeSwgYW5kCj4gPiB0YWtlcyBjYXJlIG9mIGFsbCB0aGUgY29t cGxleGl0aWVzIG9mIG1ha2luZyBzdXJlIHRoYXQgbm9uLW9wdGlvbmFsCj4gPiBmZW5jZXMgKGxp a2UgYm8gbW92ZXMpIGFyZSBub3QgaWdub3JlZC4gVGhpcyBzdXBwb3J0IHdhcyBhZGRlZCBpbgo+ ID4KPiA+IGNvbW1pdCAxNzdhZTA5YjVkNjk5YTVlYmQxY2FmY2VlNzg4ODlkYjk2OGFiZjU0Cj4g PiBBdXRob3I6IEFuZHJlcyBSb2RyaWd1ZXogPGFuZHJlc3g3QGdtYWlsLmNvbT4KPiA+IERhdGU6 ICAgRnJpIFNlcCAxNSAyMDo0NDowNiAyMDE3IC0wNDAwCj4gPgo+ID4gICAgIGRybS9hbWRncHU6 IGludHJvZHVjZSBBTURHUFVfR0VNX0NSRUFURV9FWFBMSUNJVF9TWU5DIHYyCj4gPgo+ID4gVW5m b3J0dWFudGVseSBpdCdzIHRoZSB3cm9uZyBzZW1hbnRpY3MsIGJlY2F1c2UgaXQncyBhIGJvIGZs YWcgYW5kCj4gPiBkaXNhYmxlcyBpbXBsaWNpdCBzeW5jIG9uIGFuIGFsbG9jYXRlZCBidWZmZXIg Y29tcGxldGVseS4KPiA+Cj4gPiBXZSBfZG9fIHdhbnQgaW1wbGljaXQgc3luYywgYnV0IGNvbnRy b2wgaXQgZXhwbGljaXRseS4gRm9yIHRoaXMgd2UKPiA+IG5lZWQgYSBmbGFnIG9uIHRoZSBkcm1f ZmlsZSwgc28gdGhhdCBhIGdpdmVuIHVzZXJzcGFjZSAobGlrZSB2dWxrYW4pCj4gPiBjYW4gbWFu YWdlIHRoZSBpbXBsaWNpdCBzeW5jIHNsb3RzIGV4cGxpY2l0bHkuIFRoZSBvdGhlciBzaWRlIG9m IHRoZQo+ID4gcGlwZWxpbmUgKGNvbXBvc2l0b3IsIG90aGVyIHByb2Nlc3Mgb3IganVzdCBkaWZm ZXJlbnQgc3RhZ2UgaW4gYSBtZWRpYQo+ID4gcGlwZWxpbmUgaW4gdGhlIHNhbWUgcHJvY2Vzcykg Y2FuIHRoZW4gZWl0aGVyIGRvIHRoZSBzYW1lLCBvciBmdWxseQo+ID4gcGFydGljaXBhdGUgaW4g dGhlIGltcGxpY2l0IHN5bmMgYXMgaW1wbGVtZW50ZWQgYnkgdGhlIGtlcm5lbCBieQo+ID4gZGVm YXVsdC4KPiA+Cj4gPiBCeSBidWlsZGluZyBvbiB0aGUgZXhpc3RpbmcgZmxhZyBmb3IgYnVmZmVy cyB3ZSBhdm9pZCBhbnkgaXNzdWVzIHdpdGgKPiA+IG9wZW5pbmcgdXAgYWRkaXRpb25hbCBzZWN1 cml0eSBjb25jZXJucyAtIGFueXRoaW5nIHRoaXMgbmV3IGZsYWcgaGVyZQo+ID4gYWxsb3dzIGlz IGFscmVhZHkuCj4gPgo+ID4gQWxsIGRyaXZlcnMgd2hpY2ggc3VwcG9ydHMgdGhpcyBjb25jZXB0 IG9mIGEgdXNlcnNwYWNlLXNwZWNpZmljCj4gPiBvcHQtb3V0IG9mIGltcGxpY2l0IHN5bmMgaGF2 ZSBhIGZsYWcgaW4gdGhlaXIgQ1MgaW9jdGwsIGJ1dCBpbiByZWFsaXR5Cj4gPiB0aGF0IHR1cm5l ZCBvdXQgdG8gYmUgYSBiaXQgdG9vIGluZmxleGlibGUuIFNlZSB0aGUgZGlzY3Vzc2lvbiBiZWxv dywKPiA+IGxldCdzIHRyeSB0byBkbyBhIGJpdCBiZXR0ZXIgZm9yIGFtZGdwdS4KPiA+Cj4gPiBU aGlzIGFsb25lIG9ubHkgYWxsb3dzIHVzIHRvIGNvbXBsZXRlbHkgYXZvaWQgYW55IHN0YWxscyBk dWUgdG8KPiA+IGltcGxpY2l0IHN5bmMsIGl0IGRvZXMgbm90IHlldCBhbGxvdyB1cyB0byB1c2Ug aW1wbGljaXQgc3luYyBhcyBhCj4gPiBzdHJhbmdlIGZvcm0gb2YgSVBDIGZvciBzeW5jX2ZpbGUu Cj4gPgo+ID4gRm9yIHRoYXQgd2UgbmVlZCB0d28gbW9yZSBwaWVjZXM6Cj4gPgo+ID4gLSBhIHdh eSB0byBnZXQgdGhlIGN1cnJlbnQgaW1wbGljaXQgc3luYyBmZW5jZXMgb3V0IG9mIGEgYnVmZmVy LiBDb3VsZAo+ID4gICBiZSBkb25lIGluIGEgZHJpdmVyIGlvY3RsLCBidXQgZXZlcnlvbmUgbmVl ZHMgdGhpcywgYW5kIGdlbmVyYWxseSBhCj4gPiAgIGRtYS1idWYgaXMgaW52b2x2ZWQgYW55d2F5 IHRvIGVzdGFibGlzaCB0aGUgc2hhcmluZy4gU28gYW4gaW9jdGwgb24KPiA+ICAgdGhlIGRtYS1i dWYgbWFrZXMgYSB0b24gbW9yZSBzZW5zZToKPiA+Cj4gPiAgIGh0dHBzOi8vbG9yZS5rZXJuZWwu b3JnL2RyaS1kZXZlbC8yMDIxMDUyMDE5MDAwNy41MzQwNDYtNC1qYXNvbkBqbGVrc3RyYW5kLm5l dC8KPiA+Cj4gPiAgIEN1cnJlbnQgZHJpdmVycyBpbiB1cHN0cmVhbSBzb2x2ZXMgdGhpcyBieSBo YXZpbmcgdGhlIG9wdC1vdXQgZmxhZwo+ID4gICBvbiB0aGVpciBDUyBpb2N0bC4gVGhpcyBoYXMg dGhlIGRvd25zaWRlIHRoYXQgdmVyeSBvZnRlbiB0aGUgQ1MKPiA+ICAgd2hpY2ggbXVzdCBhY3R1 YWxseSBzdGFsbCBmb3IgdGhlIGltcGxpY2l0IGZlbmNlIGlzIHJ1biBhIHdoaWxlCj4gPiAgIGFm dGVyIHRoZSBpbXBsaWNpdCBmZW5jZSBwb2ludCB3YXMgbG9naWNhbGx5IHNhbXBsZWQgcGVyIHRo ZSBhcGkKPiA+ICAgc3BlYyAodmsgcGFzc2VzIGFuIGV4cGxpY2l0IHN5bmNvYmogYXJvdW5kIGZv ciB0aGF0IGFmYWl1aSksIGFuZCBzbwo+ID4gICByZXN1bHRzIGluIG92ZXJzeW5jLiBDb252ZXJ0 aW5nIHRoZSBpbXBsaWNpdCBzeW5jIGZlbmNlcyBpbnRvIGEKPiA+ICAgc25hcC1zaG90IHN5bmNf ZmlsZSBpcyBhY3R1YWxseSBhY2N1cmF0ZS4KPiA+Cj4gPiAtIFNpbWlsbGFyIHdlIG5lZWQgdG8g YmUgYWJsZSB0byBzZXQgdGhlIGV4Y2x1c2l2ZSBpbXBsaWNpdCBmZW5jZS4KPiA+ICAgQ3VycmVu dCBkcml2ZXJzIGFnYWluIGRvIHRoaXMgd2l0aCBhIENTIGlvY3RsIGZsYWcsIHdpdGggYWdhaW4g dGhlCj4gPiAgIHNhbWUgcHJvYmxlbXMgdGhhdCB0aGUgdGltZSB0aGUgQ1MgaGFwcGVucyBhZGRp dGlvbmFsIGRlcGVuZGVuY2llcwo+ID4gICBoYXZlIGJlZW4gYWRkZWQuIEFuIGV4cGxpY2l0IGlv Y3RsIHRvIG9ubHkgaW5zZXJ0IGEgc3luY19maWxlICh3aGlsZQo+ID4gICByZXNwZWN0aW5nIHRo ZSBydWxlcyBmb3IgaG93IGV4Y2x1c2l2ZSBhbmQgc2hhcmVkIGZlbmNlIHNsb3RzIG11c3QKPiA+ ICAgYmUgdXBkYXRlIGluIHN0cnVjdCBkbWFfcmVzdikgaXMgbXVjaCBiZXR0ZXIuIFRoaXMgaXMg cHJvcG9zZWQgaGVyZToKPiA+Cj4gPiAgIGh0dHBzOi8vbG9yZS5rZXJuZWwub3JnL2RyaS1kZXZl bC8yMDIxMDUyMDE5MDAwNy41MzQwNDYtNS1qYXNvbkBqbGVrc3RyYW5kLm5ldC8KPiA+Cj4gPiBU aGVzZSB0aHJlZSBwaWVjZXMgdG9nZXRoZXIgYWxsb3cgdXNlcnNwYWNlIHRvIGZ1bGx5IGNvbnRy b2wgaW1wbGljaXQKPiA+IGZlbmNpbmcgYW5kIHJlbW92ZSBhbGwgdW5lY2Vzc2FyeSBzdGFsbCBw b2ludHMgZHVlIHRvIHRoZW0uCj4gPgo+ID4gV2VsbCwgYXMgbXVjaCBhcyB0aGUgaW1wbGljaXQg ZmVuY2luZyBtb2RlbCBmdW5kYW1lbnRhbGx5IGFsbG93czoKPiA+IFRoZXJlIGlzIG9ubHkgb25l IHNldCBvZiBmZW5jZXMsIHlvdSBjYW4gb25seSBjaG9vc2UgdG8gc3luYyBhZ2FpbnN0Cj4gPiBv bmx5IHdyaXRlcnMgKGV4Y2x1c2l2ZSBzbG90KSwgb3IgZXZlcnlvbmUuIEhlbmNlIHN1YmFsbG9j YXRpbmcKPiA+IG11bHRpcGxlIGJ1ZmZlcnMgb3IgYW55dGhpbmcgZWxzZSBsaWtlIHRoaXMgaXMg ZnVuZGFtZW50YWxseSBub3QKPiA+IHBvc3NpYmxlLCBhbmQgY2FuIG9ubHkgYmUgZml4ZWQgYnkg YSBwcm9wZXIgZXhwbGljaXQgZmVuY2luZyBtb2RlbC4KPiA+Cj4gPiBBc2lkZSBmcm9tIHRoYXQg Y2F2ZWF0IHRoaXMgbW9kZWwgZ2V0cyBpbXBsaWNpdCBmZW5jaW5nIGFzIGNsb3NlbHkgdG8KPiA+ IGV4cGxpY2l0IGZlbmNpbmcgc2VtYW50aWNzIGFzIHBvc3NpYmxlOgo+ID4KPiA+IE9uIHRoZSBh Y3R1YWwgaW1wbGVtZW50YXRpb24gSSBvcHRlZCBmb3IgYSBzaW1wbGUgc2V0cGFyYW0gaW9jdGws IG5vCj4gPiBsb2NraW5nIChqdXN0IGF0b21pYyByZWFkcy93cml0ZXMpIGZvciBzaW1wbGljaXR5 LiBUaGVyZSBpcyBhIG5pY2UKPiA+IGZsYWcgcGFyYW1ldGVyIGluIHRoZSBWTSBpb2N0bCB3aGlj aCB3ZSBjb3VsZCB1c2UsIGV4Y2VwdDoKPiA+IC0gaXQncyBub3QgY2hlY2tlZCwgc28gdXNlcnNw YWNlIGxpa2VseSBwYXNzZXMgZ2FyYmFnZQo+ID4gLSB0aGVyZSdzIGFscmVhZHkgYSBjb21tZW50 IHRoYXQgdXNlcnNwYWNlIF9kb2VzXyBwYXNzIGdhcmJhZ2UgaW4gdGhlCj4gPiAgIHByaW9yaXR5 IGZpZWxkCj4gPiBTbyB5ZWFoIHVuZm9ydHVuYXRlbHkgdGhpcyBmbGFnIHBhcmFtZXRlciBmb3Ig c2V0dGluZyB2bSBmbGFncyBpcwo+ID4gdXNlbGVzcywgYW5kIHdlIG5lZWQgdG8gaGFjayB1cCBh IG5ldyBvbmUuCj4gPgo+ID4gdjI6IEV4cGxhaW4gd2h5IGEgbmV3IFNFVFBBUkFNIChKYXNvbikK PiA+Cj4gPiB2MzogQmFzIG5vdGljZWQgSSBmb3Jnb3QgdG8gaG9vayB1cCB0aGUgZGVwZW5kZW5j eS1zaWRlIHNob3J0Y3V0LiBXZQo+ID4gbmVlZCBib3RoLCBvciB0aGlzIGRvZXNuJ3QgZG8gbXVj aC4KPiA+Cj4gPiB2NDogUmViYXNlIG92ZXIgdGhlIGFtZGdwdSBwYXRjaCB0byBhbHdheXMgc2V0 IHRoZSBpbXBsaWNpdCBzeW5jCj4gPiBmZW5jZXMuCj4KPiBTbyBJIHRoaW5rIHRoZXJlIGlzIHN0 aWxsIGEgY2FzZSBtaXNzaW5nIGluIHRoaXMgaW1wbGVtZW50YXRpb24uCj4gQ29uc2lkZXIgdGhl c2UgMyBjYXNlcwo+Cj4gKGZvcm1hdDogYS0+YjogYiB3YWl0cyBvbiBhLiBZZXMsIEkga25vdyBh cnJvd3MgYXJlIGhhcmQpCj4KPiBleHBsaWNpdC0+ZXhwbGljaXQ6IFRoaXMgZG9lc24ndCB3YWl0 IG5vdywgd2hpY2ggaXMgZ29vZAo+IEltcGxpY2l0LT5leHBsaWNpdDogVGhpcyBkb2Vzbid0IHdh aXQgbm93LCB3aGljaCBpcyBnb29kCj4gZXhwbGljaXQtPmltcGxpY2l0IDogVGhpcyBzdGlsbCB3 YWl0cyBhcyB0aGUgZXhwbGljaXQgc3VibWlzc2lvbiBzdGlsbAo+IGFkZHMgc2hhcmVkIGZlbmNl cyBhbmQgbW9zdCB0aGluZ3MgdGhhdCBzZXQgYW4gZXhjbHVzaXZlIGZlbmNlIGZvcgo+IGltcGxp Y2l0IHN5bmMgd2lsbCBoZW5jZSB3YWl0IG9uIGl0Lgo+Cj4gVGhpcyBpcyBwcm9iYWJseSBnb29k IGVub3VnaCBmb3Igd2hhdCByYWR2IG5lZWRzIG5vdyBidXQgYWxzbyBzb3VuZHMKPiBsaWtlIGEg cmlzayB3cnQgYmFraW5nIGluIG5ldyB1YXBpIGJlaGF2aW9yIHRoYXQgd2UgZG9uJ3Qgd2FudCB0 byBiZQo+IHRoZSBlbmQgcmVzdWx0Lgo+Cj4gV2l0aGluIEFNREdQVSB0aGlzIGlzIHByb2JhYmx5 IHNvbHZhYmxlIGluIHR3byB3YXlzOgo+Cj4gMSkgRG93bmdyYWRlIEFNREdQVV9TWU5DX05FX09X TkVSIHRvIEFNREdQVV9TWU5DX0VYUExJQ0lUIGZvciBzaGFyZWQgZmVuY2VzLgoKSSdtIG5vdCBz dXJlIHRoYXQgd29ya3MuIEkgdGhpbmsgdGhlIHJpZ2h0IGZpeCBpcyB0aGF0IHJhZGVvbnNpIGFs c28Kc3dpdGNoZXMgdG8gdGhpcyBtb2RlbCwgd2l0aCBtYXliZSBhIHBlci1ibyBDUyBmbGFnIHRv IHNldCBpbmRpY2F0ZQp3cml0ZSBhY2Nlc3MsIHRvIGN1dCBkb3duIG9uIHRoZSBudW1iZXIgb2Yg aW9jdGxzIHRoYXQgYXJlIG5lZWRlZApvdGhlcndpc2Ugb24gc2hhcmVkIGJ1ZmZlcnMuIFRoaXMg cGVyLWJvIGZsYWcgd291bGQgZXNzZW50aWFsbHkgc2VsZWN0CmJldHdlZW4gU1lOQ19ORV9PV05F UiBhbmQgU1lOQ19FWFBMSUNJVCBvbiBhIHBlci1idWZmZXIgYmFzaXMuCgpUaGUgY3VycmVudCBh bWRncHUgdWFwaSBqdXN0IGRvZXNuJ3QgYWxsb3cgYW55IG90aGVyIG1vZGVsIHdpdGhvdXQgYW4K ZXhwbGljaXQgb3B0LWluLiBTbyBjdXJyZW50IGltcGxpY2l0IHN5bmMgdXNlcnNwYWNlIGp1c3Qg aGFzIHRvCm92ZXJzeW5jLCB0aGVyZSdzIG5vdCBtdWNoIGNob2ljZS4KCj4gMikgSGF2ZSBhbiBF WFBMSUNJVCBmZW5jZSBvd25lciB0aGF0IGlzIHVzZWQgZm9yIGV4cGxpY2l0IHN1Ym1pc3Npb25z Cj4gdGhhdCBpcyBpZ25vcmVkIGJ5IEFNREdQVV9TWU5DX05FX09XTkVSLgo+Cj4gQnV0IHRoaXMg ZG9lc24ndCBzb2x2ZSBjcm9zcy1kcml2ZXIgaW50ZXJhY3Rpb25zIGhlcmUuCgpZZWFoIGNyb3Nz LWRyaXZlciBpcyBzdGlsbCBlbnRpcmVseSB1bnNvbHZlZCwgYmVjYXVzZQphbWRncHVfYm9fZXhw bGljaXRfc3luYygpIG9uIHRoZSBibyBkaWRuJ3Qgc29sdmUgdGhhdCBlaXRoZXIuCi1EYW5pZWwK Cj4KPiA+Cj4gPiBDYzogbWVzYS1kZXZAbGlzdHMuZnJlZWRlc2t0b3Aub3JnCj4gPiBDYzogQmFz IE5pZXV3ZW5odWl6ZW4gPGJhc0BiYXNuaWV1d2VuaHVpemVuLm5sPgo+ID4gQ2M6IERhdmUgQWly bGllIDxhaXJsaWVkQGdtYWlsLmNvbT4KPiA+IENjOiBSb2IgQ2xhcmsgPHJvYmRjbGFya0BjaHJv bWl1bS5vcmc+Cj4gPiBDYzogS3Jpc3RpYW4gSC4gS3Jpc3RlbnNlbiA8aG9lZ3NiZXJnQGdvb2ds ZS5jb20+Cj4gPiBDYzogTWljaGVsIETDpG56ZXIgPG1pY2hlbEBkYWVuemVyLm5ldD4KPiA+IENj OiBEYW5pZWwgU3RvbmUgPGRhbmllbHNAY29sbGFib3JhLmNvbT4KPiA+IENjOiBTdW1pdCBTZW13 YWwgPHN1bWl0LnNlbXdhbEBsaW5hcm8ub3JnPgo+ID4gQ2M6ICJDaHJpc3RpYW4gS8O2bmlnIiA8 Y2hyaXN0aWFuLmtvZW5pZ0BhbWQuY29tPgo+ID4gQ2M6IEFsZXggRGV1Y2hlciA8YWxleGFuZGVy LmRldWNoZXJAYW1kLmNvbT4KPiA+IENjOiBEYW5pZWwgVmV0dGVyIDxkYW5pZWwudmV0dGVyQGZm d2xsLmNoPgo+ID4gQ2M6IERlZXBhayBSIFZhcm1hIDxtaDEyZ3gyODI1QGdtYWlsLmNvbT4KPiA+ IENjOiBDaGVuIExpIDxjaGVubGlAdW5pb250ZWNoLmNvbT4KPiA+IENjOiBLZXZpbiBXYW5nIDxr ZXZpbjEud2FuZ0BhbWQuY29tPgo+ID4gQ2M6IERlbm5pcyBMaSA8RGVubmlzLkxpQGFtZC5jb20+ Cj4gPiBDYzogTHViZW4gVHVpa292IDxsdWJlbi50dWlrb3ZAYW1kLmNvbT4KPiA+IENjOiBsaW5h cm8tbW0tc2lnQGxpc3RzLmxpbmFyby5vcmcKPiA+IFNpZ25lZC1vZmYtYnk6IERhbmllbCBWZXR0 ZXIgPGRhbmllbC52ZXR0ZXJAaW50ZWwuY29tPgo+ID4gLS0tCj4gPiAgZHJpdmVycy9ncHUvZHJt L2FtZC9hbWRncHUvYW1kZ3B1X2NzLmMgIHwgIDcgKysrKystLQo+ID4gIGRyaXZlcnMvZ3B1L2Ry bS9hbWQvYW1kZ3B1L2FtZGdwdV9kcnYuYyB8IDIxICsrKysrKysrKysrKysrKysrKysrKwo+ID4g IGRyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV92bS5oICB8ICA2ICsrKysrKwo+ID4g IGluY2x1ZGUvdWFwaS9kcm0vYW1kZ3B1X2RybS5oICAgICAgICAgICB8IDEwICsrKysrKysrKysK PiA+ICA0IGZpbGVzIGNoYW5nZWQsIDQyIGluc2VydGlvbnMoKyksIDIgZGVsZXRpb25zKC0pCj4g Pgo+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1L2FtZGdwdV9jcy5j IGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X2NzLmMKPiA+IGluZGV4IDY1ZGYz NGMxNzI2NC4uYzUzODZkMTNlYjRhIDEwMDY0NAo+ID4gLS0tIGEvZHJpdmVycy9ncHUvZHJtL2Ft ZC9hbWRncHUvYW1kZ3B1X2NzLmMKPiA+ICsrKyBiL2RyaXZlcnMvZ3B1L2RybS9hbWQvYW1kZ3B1 L2FtZGdwdV9jcy5jCj4gPiBAQCAtNDk4LDYgKzQ5OCw3IEBAIHN0YXRpYyBpbnQgYW1kZ3B1X2Nz X3BhcnNlcl9ib3Moc3RydWN0IGFtZGdwdV9jc19wYXJzZXIgKnAsCj4gPiAgICAgICAgIHN0cnVj dCBhbWRncHVfYm8gKmdkczsKPiA+ICAgICAgICAgc3RydWN0IGFtZGdwdV9ibyAqZ3dzOwo+ID4g ICAgICAgICBzdHJ1Y3QgYW1kZ3B1X2JvICpvYTsKPiA+ICsgICAgICAgYm9vbCBub19pbXBsaWNp dF9zeW5jID0gUkVBRF9PTkNFKGZwcml2LT52bS5ub19pbXBsaWNpdF9zeW5jKTsKPiA+ICAgICAg ICAgaW50IHI7Cj4gPgo+ID4gICAgICAgICBJTklUX0xJU1RfSEVBRCgmcC0+dmFsaWRhdGVkKTsK PiA+IEBAIC01NzcsNyArNTc4LDggQEAgc3RhdGljIGludCBhbWRncHVfY3NfcGFyc2VyX2Jvcyhz dHJ1Y3QgYW1kZ3B1X2NzX3BhcnNlciAqcCwKPiA+Cj4gPiAgICAgICAgICAgICAgICAgZS0+Ym9f dmEgPSBhbWRncHVfdm1fYm9fZmluZCh2bSwgYm8pOwo+ID4KPiA+IC0gICAgICAgICAgICAgICBp ZiAoYm8tPnRiby5iYXNlLmRtYV9idWYgJiYgIWFtZGdwdV9ib19leHBsaWNpdF9zeW5jKGJvKSkg ewo+ID4gKyAgICAgICAgICAgICAgIGlmIChiby0+dGJvLmJhc2UuZG1hX2J1ZiAmJgo+ID4gKyAg ICAgICAgICAgICAgICAgICAhKG5vX2ltcGxpY2l0X3N5bmMgfHwgYW1kZ3B1X2JvX2V4cGxpY2l0 X3N5bmMoYm8pKSkgewo+ID4gICAgICAgICAgICAgICAgICAgICAgICAgZS0+Y2hhaW4gPSBkbWFf ZmVuY2VfY2hhaW5fYWxsb2MoKTsKPiA+ICAgICAgICAgICAgICAgICAgICAgICAgIGlmICghZS0+ Y2hhaW4pIHsKPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgciA9IC1FTk9NRU07 Cj4gPiBAQCAtNjQ5LDYgKzY1MSw3IEBAIHN0YXRpYyBpbnQgYW1kZ3B1X2NzX3N5bmNfcmluZ3Mo c3RydWN0IGFtZGdwdV9jc19wYXJzZXIgKnApCj4gPiAgewo+ID4gICAgICAgICBzdHJ1Y3QgYW1k Z3B1X2Zwcml2ICpmcHJpdiA9IHAtPmZpbHAtPmRyaXZlcl9wcml2Owo+ID4gICAgICAgICBzdHJ1 Y3QgYW1kZ3B1X2JvX2xpc3RfZW50cnkgKmU7Cj4gPiArICAgICAgIGJvb2wgbm9faW1wbGljaXRf c3luYyA9IFJFQURfT05DRShmcHJpdi0+dm0ubm9faW1wbGljaXRfc3luYyk7Cj4gPiAgICAgICAg IGludCByOwo+ID4KPiA+ICAgICAgICAgbGlzdF9mb3JfZWFjaF9lbnRyeShlLCAmcC0+dmFsaWRh dGVkLCB0di5oZWFkKSB7Cj4gPiBAQCAtNjU2LDcgKzY1OSw3IEBAIHN0YXRpYyBpbnQgYW1kZ3B1 X2NzX3N5bmNfcmluZ3Moc3RydWN0IGFtZGdwdV9jc19wYXJzZXIgKnApCj4gPiAgICAgICAgICAg ICAgICAgc3RydWN0IGRtYV9yZXN2ICpyZXN2ID0gYm8tPnRiby5iYXNlLnJlc3Y7Cj4gPiAgICAg ICAgICAgICAgICAgZW51bSBhbWRncHVfc3luY19tb2RlIHN5bmNfbW9kZTsKPiA+Cj4gPiAtICAg ICAgICAgICAgICAgc3luY19tb2RlID0gYW1kZ3B1X2JvX2V4cGxpY2l0X3N5bmMoYm8pID8KPiA+ ICsgICAgICAgICAgICAgICBzeW5jX21vZGUgPSBub19pbXBsaWNpdF9zeW5jIHx8IGFtZGdwdV9i b19leHBsaWNpdF9zeW5jKGJvKSA/Cj4gPiAgICAgICAgICAgICAgICAgICAgICAgICBBTURHUFVf U1lOQ19FWFBMSUNJVCA6IEFNREdQVV9TWU5DX05FX09XTkVSOwo+ID4gICAgICAgICAgICAgICAg IHIgPSBhbWRncHVfc3luY19yZXN2KHAtPmFkZXYsICZwLT5qb2ItPnN5bmMsIHJlc3YsIHN5bmNf bW9kZSwKPiA+ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAmZnByaXYtPnZt KTsKPiA+IGRpZmYgLS1naXQgYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZHJ2 LmMgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfZHJ2LmMKPiA+IGluZGV4IGMw ODBiYTE1YWU3Ny4uZjk4MjYyNmI1MzI4IDEwMDY0NAo+ID4gLS0tIGEvZHJpdmVycy9ncHUvZHJt L2FtZC9hbWRncHUvYW1kZ3B1X2Rydi5jCj4gPiArKysgYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2Ft ZGdwdS9hbWRncHVfZHJ2LmMKPiA+IEBAIC0xNzI0LDYgKzE3MjQsMjYgQEAgaW50IGFtZGdwdV9m aWxlX3RvX2Zwcml2KHN0cnVjdCBmaWxlICpmaWxwLCBzdHJ1Y3QgYW1kZ3B1X2Zwcml2ICoqZnBy aXYpCj4gPiAgICAgICAgIHJldHVybiAwOwo+ID4gIH0KPiA+Cj4gPiAraW50IGFtZGdwdV9zZXRw YXJhbV9pb2N0bChzdHJ1Y3QgZHJtX2RldmljZSAqZGV2LCB2b2lkICpkYXRhLAo+ID4gKyAgICAg ICAgICAgICAgICAgICAgICAgICBzdHJ1Y3QgZHJtX2ZpbGUgKmZpbHApCj4gPiArewo+ID4gKyAg ICAgICBzdHJ1Y3QgZHJtX2FtZGdwdV9zZXRwYXJhbSAqc2V0cGFyYW0gPSBkYXRhOwo+ID4gKyAg ICAgICBzdHJ1Y3QgYW1kZ3B1X2Zwcml2ICpmcHJpdiA9IGZpbHAtPmRyaXZlcl9wcml2Owo+ID4g Kwo+ID4gKyAgICAgICBzd2l0Y2ggKHNldHBhcmFtLT5wYXJhbSkgewo+ID4gKyAgICAgICBjYXNl IEFNREdQVV9TRVRQQVJBTV9OT19JTVBMSUNJVF9TWU5DOgo+ID4gKyAgICAgICAgICAgICAgIGlm IChzZXRwYXJhbS0+dmFsdWUpCj4gPiArICAgICAgICAgICAgICAgICAgICAgICBXUklURV9PTkNF KGZwcml2LT52bS5ub19pbXBsaWNpdF9zeW5jLCB0cnVlKTsKPiA+ICsgICAgICAgICAgICAgICBl bHNlCj4gPiArICAgICAgICAgICAgICAgICAgICAgICBXUklURV9PTkNFKGZwcml2LT52bS5ub19p bXBsaWNpdF9zeW5jLCBmYWxzZSk7Cj4gPiArICAgICAgICAgICAgICAgYnJlYWs7Cj4gPiArICAg ICAgIGRlZmF1bHQ6Cj4gPiArICAgICAgICAgICAgICAgcmV0dXJuIC1FSU5WQUw7Cj4gPiArICAg ICAgIH0KPiA+ICsKPiA+ICsgICAgICAgcmV0dXJuIDA7Cj4gPiArfQo+ID4gKwo+ID4gIGNvbnN0 IHN0cnVjdCBkcm1faW9jdGxfZGVzYyBhbWRncHVfaW9jdGxzX2ttc1tdID0gewo+ID4gICAgICAg ICBEUk1fSU9DVExfREVGX0RSVihBTURHUFVfR0VNX0NSRUFURSwgYW1kZ3B1X2dlbV9jcmVhdGVf aW9jdGwsIERSTV9BVVRIfERSTV9SRU5ERVJfQUxMT1cpLAo+ID4gICAgICAgICBEUk1fSU9DVExf REVGX0RSVihBTURHUFVfQ1RYLCBhbWRncHVfY3R4X2lvY3RsLCBEUk1fQVVUSHxEUk1fUkVOREVS X0FMTE9XKSwKPiA+IEBAIC0xNzQyLDYgKzE3NjIsNyBAQCBjb25zdCBzdHJ1Y3QgZHJtX2lvY3Rs X2Rlc2MgYW1kZ3B1X2lvY3Rsc19rbXNbXSA9IHsKPiA+ICAgICAgICAgRFJNX0lPQ1RMX0RFRl9E UlYoQU1ER1BVX0dFTV9WQSwgYW1kZ3B1X2dlbV92YV9pb2N0bCwgRFJNX0FVVEh8RFJNX1JFTkRF Ul9BTExPVyksCj4gPiAgICAgICAgIERSTV9JT0NUTF9ERUZfRFJWKEFNREdQVV9HRU1fT1AsIGFt ZGdwdV9nZW1fb3BfaW9jdGwsIERSTV9BVVRIfERSTV9SRU5ERVJfQUxMT1cpLAo+ID4gICAgICAg ICBEUk1fSU9DVExfREVGX0RSVihBTURHUFVfR0VNX1VTRVJQVFIsIGFtZGdwdV9nZW1fdXNlcnB0 cl9pb2N0bCwgRFJNX0FVVEh8RFJNX1JFTkRFUl9BTExPVyksCj4gPiArICAgICAgIERSTV9JT0NU TF9ERUZfRFJWKEFNREdQVV9TRVRQQVJBTSwgYW1kZ3B1X3NldHBhcmFtX2lvY3RsLCBEUk1fQVVU SHxEUk1fUkVOREVSX0FMTE9XKSwKPiA+ICB9Owo+ID4KPiA+ICBzdGF0aWMgY29uc3Qgc3RydWN0 IGRybV9kcml2ZXIgYW1kZ3B1X2ttc19kcml2ZXIgPSB7Cj4gPiBkaWZmIC0tZ2l0IGEvZHJpdmVy cy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X3ZtLmggYi9kcml2ZXJzL2dwdS9kcm0vYW1kL2Ft ZGdwdS9hbWRncHVfdm0uaAo+ID4gaW5kZXggZGRiODVhODVjYmJhLi4wZThjNDQwYzYzMDMgMTAw NjQ0Cj4gPiAtLS0gYS9kcml2ZXJzL2dwdS9kcm0vYW1kL2FtZGdwdS9hbWRncHVfdm0uaAo+ID4g KysrIGIvZHJpdmVycy9ncHUvZHJtL2FtZC9hbWRncHUvYW1kZ3B1X3ZtLmgKPiA+IEBAIC0zMjEs NiArMzIxLDEyIEBAIHN0cnVjdCBhbWRncHVfdm0gewo+ID4gICAgICAgICBib29sICAgICAgICAg ICAgICAgICAgICBidWxrX21vdmVhYmxlOwo+ID4gICAgICAgICAvKiBGbGFnIHRvIGluZGljYXRl IGlmIFZNIGlzIHVzZWQgZm9yIGNvbXB1dGUgKi8KPiA+ICAgICAgICAgYm9vbCAgICAgICAgICAg ICAgICAgICAgaXNfY29tcHV0ZV9jb250ZXh0Owo+ID4gKyAgICAgICAvKgo+ID4gKyAgICAgICAg KiBGbGFnIHRvIGluZGljYXRlIHdoZXRoZXIgaW1wbGljaXQgc3luYyBzaG91bGQgYWx3YXlzIGJl IHNraXBwZWQgb24KPiA+ICsgICAgICAgICogdGhpcyBjb250ZXh0LiBXZSBkbyBub3QgY2FyZSBh Ym91dCByYWNlcyBhdCBhbGwsIHVzZXJzcGFjZSBpcyBhbGxvd2VkCj4gPiArICAgICAgICAqIHRv IHNob290IGl0c2VsZiB3aXRoIGltcGxpY2l0IHN5bmMgdG8gaXRzIGZ1bGxlc3QgbGlraW5nLgo+ ID4gKyAgICAgICAgKi8KPiA+ICsgICAgICAgYm9vbCBub19pbXBsaWNpdF9zeW5jOwo+ID4gIH07 Cj4gPgo+ID4gIHN0cnVjdCBhbWRncHVfdm1fbWFuYWdlciB7Cj4gPiBkaWZmIC0tZ2l0IGEvaW5j bHVkZS91YXBpL2RybS9hbWRncHVfZHJtLmggYi9pbmNsdWRlL3VhcGkvZHJtL2FtZGdwdV9kcm0u aAo+ID4gaW5kZXggMGNiZDE1NDBhZWFjLi45ZWFlMjQ1YzE0ZDYgMTAwNjQ0Cj4gPiAtLS0gYS9p bmNsdWRlL3VhcGkvZHJtL2FtZGdwdV9kcm0uaAo+ID4gKysrIGIvaW5jbHVkZS91YXBpL2RybS9h bWRncHVfZHJtLmgKPiA+IEBAIC01NCw2ICs1NCw3IEBAIGV4dGVybiAiQyIgewo+ID4gICNkZWZp bmUgRFJNX0FNREdQVV9WTSAgICAgICAgICAgICAgICAgIDB4MTMKPiA+ICAjZGVmaW5lIERSTV9B TURHUFVfRkVOQ0VfVE9fSEFORExFICAgICAweDE0Cj4gPiAgI2RlZmluZSBEUk1fQU1ER1BVX1ND SEVEICAgICAgICAgICAgICAgMHgxNQo+ID4gKyNkZWZpbmUgRFJNX0FNREdQVV9TRVRQQVJBTSAg ICAgICAgICAgIDB4MTYKPiA+Cj4gPiAgI2RlZmluZSBEUk1fSU9DVExfQU1ER1BVX0dFTV9DUkVB VEUgICAgRFJNX0lPV1IoRFJNX0NPTU1BTkRfQkFTRSArIERSTV9BTURHUFVfR0VNX0NSRUFURSwg dW5pb24gZHJtX2FtZGdwdV9nZW1fY3JlYXRlKQo+ID4gICNkZWZpbmUgRFJNX0lPQ1RMX0FNREdQ VV9HRU1fTU1BUCAgICAgIERSTV9JT1dSKERSTV9DT01NQU5EX0JBU0UgKyBEUk1fQU1ER1BVX0dF TV9NTUFQLCB1bmlvbiBkcm1fYW1kZ3B1X2dlbV9tbWFwKQo+ID4gQEAgLTcxLDYgKzcyLDcgQEAg ZXh0ZXJuICJDIiB7Cj4gPiAgI2RlZmluZSBEUk1fSU9DVExfQU1ER1BVX1ZNICAgICAgICAgICAg RFJNX0lPV1IoRFJNX0NPTU1BTkRfQkFTRSArIERSTV9BTURHUFVfVk0sIHVuaW9uIGRybV9hbWRn cHVfdm0pCj4gPiAgI2RlZmluZSBEUk1fSU9DVExfQU1ER1BVX0ZFTkNFX1RPX0hBTkRMRSBEUk1f SU9XUihEUk1fQ09NTUFORF9CQVNFICsgRFJNX0FNREdQVV9GRU5DRV9UT19IQU5ETEUsIHVuaW9u IGRybV9hbWRncHVfZmVuY2VfdG9faGFuZGxlKQo+ID4gICNkZWZpbmUgRFJNX0lPQ1RMX0FNREdQ VV9TQ0hFRCAgICAgICAgIERSTV9JT1coRFJNX0NPTU1BTkRfQkFTRSArIERSTV9BTURHUFVfU0NI RUQsIHVuaW9uIGRybV9hbWRncHVfc2NoZWQpCj4gPiArI2RlZmluZSBEUk1fSU9DVExfQU1ER1BV X1NFVFBBUkFNICAgICAgRFJNX0lPVyhEUk1fQ09NTUFORF9CQVNFICsgRFJNX0FNREdQVV9TRVRQ QVJBTSwgc3RydWN0IGRybV9hbWRncHVfc2V0cGFyYW0pCj4gPgo+ID4gIC8qKgo+ID4gICAqIERP QzogbWVtb3J5IGRvbWFpbnMKPiA+IEBAIC0zMDYsNiArMzA4LDE0IEBAIHVuaW9uIGRybV9hbWRn cHVfc2NoZWQgewo+ID4gICAgICAgICBzdHJ1Y3QgZHJtX2FtZGdwdV9zY2hlZF9pbiBpbjsKPiA+ ICB9Owo+ID4KPiA+ICsjZGVmaW5lIEFNREdQVV9TRVRQQVJBTV9OT19JTVBMSUNJVF9TWU5DICAg ICAgIDEKPiA+ICsKPiA+ICtzdHJ1Y3QgZHJtX2FtZGdwdV9zZXRwYXJhbSB7Cj4gPiArICAgICAg IC8qIEFNREdQVV9TRVRQQVJBTV8qICovCj4gPiArICAgICAgIF9fdTMyICAgcGFyYW07Cj4gPiAr ICAgICAgIF9fdTMyICAgdmFsdWU7Cj4gPiArfTsKPiA+ICsKPiA+ICAvKgo+ID4gICAqIFRoaXMg aXMgbm90IGEgcmVsaWFibGUgQVBJIGFuZCB5b3Ugc2hvdWxkIGV4cGVjdCBpdCB0byBmYWlsIGZv ciBhbnkKPiA+ICAgKiBudW1iZXIgb2YgcmVhc29ucyBhbmQgaGF2ZSBmYWxsYmFjayBwYXRoIHRo YXQgZG8gbm90IHVzZSB1c2VycHRyIHRvCj4gPiAtLQo+ID4gMi4zMi4wLnJjMgo+ID4KCgoKLS0g CkRhbmllbCBWZXR0ZXIKU29mdHdhcmUgRW5naW5lZXIsIEludGVsIENvcnBvcmF0aW9uCmh0dHA6 Ly9ibG9nLmZmd2xsLmNoCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fCkludGVsLWdmeCBtYWlsaW5nIGxpc3QKSW50ZWwtZ2Z4QGxpc3RzLmZyZWVkZXNrdG9w Lm9yZwpodHRwczovL2xpc3RzLmZyZWVkZXNrdG9wLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2ludGVs LWdmeAo= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4F8F0C49EA4 for ; Wed, 23 Jun 2021 12:18:55 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1E4AF6100A for ; Wed, 23 Jun 2021 12:18:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1E4AF6100A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ffwll.ch Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 489E46E8CD; Wed, 23 Jun 2021 12:18:50 +0000 (UTC) Received: from mail-oi1-x231.google.com (mail-oi1-x231.google.com [IPv6:2607:f8b0:4864:20::231]) by gabe.freedesktop.org (Postfix) with ESMTPS id C10756E8CD for ; Wed, 23 Jun 2021 12:18:48 +0000 (UTC) Received: by mail-oi1-x231.google.com with SMTP id 14so3057418oir.11 for ; Wed, 23 Jun 2021 05:18:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=DhZ7INqfjMkrIqqscqZr1FgEv7Thc3LBimm6s6d3jWQ=; b=Tpxb+E8Xh/onR1scIQsva+bFRIrA1PacGN/9EtTOTlr4VOl+hpcdeBctINqa5cuTfL 00yGrH9/eRZsi4JGrEO4mmvcOynx/jERBJ282uGidC5H1orZtiGlGI39vBUazGTB1Zqo 7ti8gIS8xas7gRR1dL9cgcTmv22hePZUNtomA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=DhZ7INqfjMkrIqqscqZr1FgEv7Thc3LBimm6s6d3jWQ=; b=JZaRnm02Bxj1bNbm9VvzQwtEc2IhfxlVDDnEx+oejLbWLhoeMZVn5aC1EqaY+KS7NZ m3cl1MG9Qmv3G8miTxUcRu3qX9VcJE1QeQ6gf0E1GuF0uDiB6eG5ZcPaZtp+n9PUzMg9 vqX4VGQWYQPCaB60PYKs8lbcfnQWlLkLlUksmr+kPa5THJMZ9y8sOx+jpiLD5POqIFRY kr319ObVc1cAwIitnpeSta3xLyy/sYcSfYgkvFnIhyiNn+qv/Z7Wb/NfUPfy79iA3TSF 7OPZPzJNZyzmEEG7Z8PmSUGm4TYYe/hlWKxrjxWV8HIFMwZEcO3ZNdKL83br8Uot2DNg ibow== X-Gm-Message-State: AOAM532b+MKbUsipEGvf6GzROnA2tAqQ/sDpxFkCrYP0xGvl/iE3wRwb TFxKTFkKOK86s6pymWchnSGNmePGrzFCKiSYWvGu2A== X-Google-Smtp-Source: ABdhPJyIkt4ftwgh1jZOld/prk5IZrQQ2XEpd2Y37zmuecslQf4c0cxS3DRuv+GAr2BBxh/u/cRfm8sa6sV+li/1BOQ= X-Received: by 2002:a54:4889:: with SMTP id r9mr2930612oic.101.1624450728097; Wed, 23 Jun 2021 05:18:48 -0700 (PDT) MIME-Version: 1.0 References: <20210622165511.3169559-1-daniel.vetter@ffwll.ch> <20210622165511.3169559-16-daniel.vetter@ffwll.ch> In-Reply-To: From: Daniel Vetter Date: Wed, 23 Jun 2021 14:18:35 +0200 Message-ID: Subject: Re: [PATCH 15/15] RFC: drm/amdgpu: Implement a proper implicit fencing uapi To: Bas Nieuwenhuizen Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Rob Clark , Daniel Stone , =?UTF-8?Q?Christian_K=C3=B6nig?= , Intel Graphics Development , Kevin Wang , DRI Development , "moderated list:DMA BUFFER SHARING FRAMEWORK" , Luben Tuikov , "Kristian H . Kristensen" , Chen Li , Daniel Vetter , Alex Deucher , mesa-dev , =?UTF-8?Q?Michel_D=C3=A4nzer?= , Dennis Li , Deepak R Varma Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Wed, Jun 23, 2021 at 11:45 AM Bas Nieuwenhuizen wrote: > > On Tue, Jun 22, 2021 at 6:55 PM Daniel Vetter wr= ote: > > > > WARNING: Absolutely untested beyond "gcc isn't dying in agony". > > > > Implicit fencing done properly needs to treat the implicit fencing > > slots like a funny kind of IPC mailbox. In other words it needs to be > > explicitly. This is the only way it will mesh well with explicit > > fencing userspace like vk, and it's also the bare minimum required to > > be able to manage anything else that wants to use the same buffer on > > multiple engines in parallel, and still be able to share it through > > implicit sync. > > > > amdgpu completely lacks such an uapi. Fix this. > > > > Luckily the concept of ignoring implicit fences exists already, and > > takes care of all the complexities of making sure that non-optional > > fences (like bo moves) are not ignored. This support was added in > > > > commit 177ae09b5d699a5ebd1cafcee78889db968abf54 > > Author: Andres Rodriguez > > Date: Fri Sep 15 20:44:06 2017 -0400 > > > > drm/amdgpu: introduce AMDGPU_GEM_CREATE_EXPLICIT_SYNC v2 > > > > Unfortuantely it's the wrong semantics, because it's a bo flag and > > disables implicit sync on an allocated buffer completely. > > > > We _do_ want implicit sync, but control it explicitly. For this we > > need a flag on the drm_file, so that a given userspace (like vulkan) > > can manage the implicit sync slots explicitly. The other side of the > > pipeline (compositor, other process or just different stage in a media > > pipeline in the same process) can then either do the same, or fully > > participate in the implicit sync as implemented by the kernel by > > default. > > > > By building on the existing flag for buffers we avoid any issues with > > opening up additional security concerns - anything this new flag here > > allows is already. > > > > All drivers which supports this concept of a userspace-specific > > opt-out of implicit sync have a flag in their CS ioctl, but in reality > > that turned out to be a bit too inflexible. See the discussion below, > > let's try to do a bit better for amdgpu. > > > > This alone only allows us to completely avoid any stalls due to > > implicit sync, it does not yet allow us to use implicit sync as a > > strange form of IPC for sync_file. > > > > For that we need two more pieces: > > > > - a way to get the current implicit sync fences out of a buffer. Could > > be done in a driver ioctl, but everyone needs this, and generally a > > dma-buf is involved anyway to establish the sharing. So an ioctl on > > the dma-buf makes a ton more sense: > > > > https://lore.kernel.org/dri-devel/20210520190007.534046-4-jason@jleks= trand.net/ > > > > Current drivers in upstream solves this by having the opt-out flag > > on their CS ioctl. This has the downside that very often the CS > > which must actually stall for the implicit fence is run a while > > after the implicit fence point was logically sampled per the api > > spec (vk passes an explicit syncobj around for that afaiui), and so > > results in oversync. Converting the implicit sync fences into a > > snap-shot sync_file is actually accurate. > > > > - Simillar we need to be able to set the exclusive implicit fence. > > Current drivers again do this with a CS ioctl flag, with again the > > same problems that the time the CS happens additional dependencies > > have been added. An explicit ioctl to only insert a sync_file (while > > respecting the rules for how exclusive and shared fence slots must > > be update in struct dma_resv) is much better. This is proposed here: > > > > https://lore.kernel.org/dri-devel/20210520190007.534046-5-jason@jleks= trand.net/ > > > > These three pieces together allow userspace to fully control implicit > > fencing and remove all unecessary stall points due to them. > > > > Well, as much as the implicit fencing model fundamentally allows: > > There is only one set of fences, you can only choose to sync against > > only writers (exclusive slot), or everyone. Hence suballocating > > multiple buffers or anything else like this is fundamentally not > > possible, and can only be fixed by a proper explicit fencing model. > > > > Aside from that caveat this model gets implicit fencing as closely to > > explicit fencing semantics as possible: > > > > On the actual implementation I opted for a simple setparam ioctl, no > > locking (just atomic reads/writes) for simplicity. There is a nice > > flag parameter in the VM ioctl which we could use, except: > > - it's not checked, so userspace likely passes garbage > > - there's already a comment that userspace _does_ pass garbage in the > > priority field > > So yeah unfortunately this flag parameter for setting vm flags is > > useless, and we need to hack up a new one. > > > > v2: Explain why a new SETPARAM (Jason) > > > > v3: Bas noticed I forgot to hook up the dependency-side shortcut. We > > need both, or this doesn't do much. > > > > v4: Rebase over the amdgpu patch to always set the implicit sync > > fences. > > So I think there is still a case missing in this implementation. > Consider these 3 cases > > (format: a->b: b waits on a. Yes, I know arrows are hard) > > explicit->explicit: This doesn't wait now, which is good > Implicit->explicit: This doesn't wait now, which is good > explicit->implicit : This still waits as the explicit submission still > adds shared fences and most things that set an exclusive fence for > implicit sync will hence wait on it. > > This is probably good enough for what radv needs now but also sounds > like a risk wrt baking in new uapi behavior that we don't want to be > the end result. > > Within AMDGPU this is probably solvable in two ways: > > 1) Downgrade AMDGPU_SYNC_NE_OWNER to AMDGPU_SYNC_EXPLICIT for shared fenc= es. I'm not sure that works. I think the right fix is that radeonsi also switches to this model, with maybe a per-bo CS flag to set indicate write access, to cut down on the number of ioctls that are needed otherwise on shared buffers. This per-bo flag would essentially select between SYNC_NE_OWNER and SYNC_EXPLICIT on a per-buffer basis. The current amdgpu uapi just doesn't allow any other model without an explicit opt-in. So current implicit sync userspace just has to oversync, there's not much choice. > 2) Have an EXPLICIT fence owner that is used for explicit submissions > that is ignored by AMDGPU_SYNC_NE_OWNER. > > But this doesn't solve cross-driver interactions here. Yeah cross-driver is still entirely unsolved, because amdgpu_bo_explicit_sync() on the bo didn't solve that either. -Daniel > > > > > Cc: mesa-dev@lists.freedesktop.org > > Cc: Bas Nieuwenhuizen > > Cc: Dave Airlie > > Cc: Rob Clark > > Cc: Kristian H. Kristensen > > Cc: Michel D=C3=A4nzer > > Cc: Daniel Stone > > Cc: Sumit Semwal > > Cc: "Christian K=C3=B6nig" > > Cc: Alex Deucher > > Cc: Daniel Vetter > > Cc: Deepak R Varma > > Cc: Chen Li > > Cc: Kevin Wang > > Cc: Dennis Li > > Cc: Luben Tuikov > > Cc: linaro-mm-sig@lists.linaro.org > > Signed-off-by: Daniel Vetter > > --- > > drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 7 +++++-- > > drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 21 +++++++++++++++++++++ > > drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 6 ++++++ > > include/uapi/drm/amdgpu_drm.h | 10 ++++++++++ > > 4 files changed, 42 insertions(+), 2 deletions(-) > > > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/a= md/amdgpu/amdgpu_cs.c > > index 65df34c17264..c5386d13eb4a 100644 > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c > > @@ -498,6 +498,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_pa= rser *p, > > struct amdgpu_bo *gds; > > struct amdgpu_bo *gws; > > struct amdgpu_bo *oa; > > + bool no_implicit_sync =3D READ_ONCE(fpriv->vm.no_implicit_sync)= ; > > int r; > > > > INIT_LIST_HEAD(&p->validated); > > @@ -577,7 +578,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_pa= rser *p, > > > > e->bo_va =3D amdgpu_vm_bo_find(vm, bo); > > > > - if (bo->tbo.base.dma_buf && !amdgpu_bo_explicit_sync(bo= )) { > > + if (bo->tbo.base.dma_buf && > > + !(no_implicit_sync || amdgpu_bo_explicit_sync(bo)))= { > > e->chain =3D dma_fence_chain_alloc(); > > if (!e->chain) { > > r =3D -ENOMEM; > > @@ -649,6 +651,7 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_pa= rser *p) > > { > > struct amdgpu_fpriv *fpriv =3D p->filp->driver_priv; > > struct amdgpu_bo_list_entry *e; > > + bool no_implicit_sync =3D READ_ONCE(fpriv->vm.no_implicit_sync)= ; > > int r; > > > > list_for_each_entry(e, &p->validated, tv.head) { > > @@ -656,7 +659,7 @@ static int amdgpu_cs_sync_rings(struct amdgpu_cs_pa= rser *p) > > struct dma_resv *resv =3D bo->tbo.base.resv; > > enum amdgpu_sync_mode sync_mode; > > > > - sync_mode =3D amdgpu_bo_explicit_sync(bo) ? > > + sync_mode =3D no_implicit_sync || amdgpu_bo_explicit_sy= nc(bo) ? > > AMDGPU_SYNC_EXPLICIT : AMDGPU_SYNC_NE_OWNER; > > r =3D amdgpu_sync_resv(p->adev, &p->job->sync, resv, sy= nc_mode, > > &fpriv->vm); > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/= amd/amdgpu/amdgpu_drv.c > > index c080ba15ae77..f982626b5328 100644 > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c > > @@ -1724,6 +1724,26 @@ int amdgpu_file_to_fpriv(struct file *filp, stru= ct amdgpu_fpriv **fpriv) > > return 0; > > } > > > > +int amdgpu_setparam_ioctl(struct drm_device *dev, void *data, > > + struct drm_file *filp) > > +{ > > + struct drm_amdgpu_setparam *setparam =3D data; > > + struct amdgpu_fpriv *fpriv =3D filp->driver_priv; > > + > > + switch (setparam->param) { > > + case AMDGPU_SETPARAM_NO_IMPLICIT_SYNC: > > + if (setparam->value) > > + WRITE_ONCE(fpriv->vm.no_implicit_sync, true); > > + else > > + WRITE_ONCE(fpriv->vm.no_implicit_sync, false); > > + break; > > + default: > > + return -EINVAL; > > + } > > + > > + return 0; > > +} > > + > > const struct drm_ioctl_desc amdgpu_ioctls_kms[] =3D { > > DRM_IOCTL_DEF_DRV(AMDGPU_GEM_CREATE, amdgpu_gem_create_ioctl, D= RM_AUTH|DRM_RENDER_ALLOW), > > DRM_IOCTL_DEF_DRV(AMDGPU_CTX, amdgpu_ctx_ioctl, DRM_AUTH|DRM_RE= NDER_ALLOW), > > @@ -1742,6 +1762,7 @@ const struct drm_ioctl_desc amdgpu_ioctls_kms[] = =3D { > > DRM_IOCTL_DEF_DRV(AMDGPU_GEM_VA, amdgpu_gem_va_ioctl, DRM_AUTH|= DRM_RENDER_ALLOW), > > DRM_IOCTL_DEF_DRV(AMDGPU_GEM_OP, amdgpu_gem_op_ioctl, DRM_AUTH|= DRM_RENDER_ALLOW), > > DRM_IOCTL_DEF_DRV(AMDGPU_GEM_USERPTR, amdgpu_gem_userptr_ioctl,= DRM_AUTH|DRM_RENDER_ALLOW), > > + DRM_IOCTL_DEF_DRV(AMDGPU_SETPARAM, amdgpu_setparam_ioctl, DRM_A= UTH|DRM_RENDER_ALLOW), > > }; > > > > static const struct drm_driver amdgpu_kms_driver =3D { > > diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/a= md/amdgpu/amdgpu_vm.h > > index ddb85a85cbba..0e8c440c6303 100644 > > --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h > > +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h > > @@ -321,6 +321,12 @@ struct amdgpu_vm { > > bool bulk_moveable; > > /* Flag to indicate if VM is used for compute */ > > bool is_compute_context; > > + /* > > + * Flag to indicate whether implicit sync should always be skip= ped on > > + * this context. We do not care about races at all, userspace i= s allowed > > + * to shoot itself with implicit sync to its fullest liking. > > + */ > > + bool no_implicit_sync; > > }; > > > > struct amdgpu_vm_manager { > > diff --git a/include/uapi/drm/amdgpu_drm.h b/include/uapi/drm/amdgpu_dr= m.h > > index 0cbd1540aeac..9eae245c14d6 100644 > > --- a/include/uapi/drm/amdgpu_drm.h > > +++ b/include/uapi/drm/amdgpu_drm.h > > @@ -54,6 +54,7 @@ extern "C" { > > #define DRM_AMDGPU_VM 0x13 > > #define DRM_AMDGPU_FENCE_TO_HANDLE 0x14 > > #define DRM_AMDGPU_SCHED 0x15 > > +#define DRM_AMDGPU_SETPARAM 0x16 > > > > #define DRM_IOCTL_AMDGPU_GEM_CREATE DRM_IOWR(DRM_COMMAND_BASE + DRM= _AMDGPU_GEM_CREATE, union drm_amdgpu_gem_create) > > #define DRM_IOCTL_AMDGPU_GEM_MMAP DRM_IOWR(DRM_COMMAND_BASE + DRM= _AMDGPU_GEM_MMAP, union drm_amdgpu_gem_mmap) > > @@ -71,6 +72,7 @@ extern "C" { > > #define DRM_IOCTL_AMDGPU_VM DRM_IOWR(DRM_COMMAND_BASE + DRM= _AMDGPU_VM, union drm_amdgpu_vm) > > #define DRM_IOCTL_AMDGPU_FENCE_TO_HANDLE DRM_IOWR(DRM_COMMAND_BASE + D= RM_AMDGPU_FENCE_TO_HANDLE, union drm_amdgpu_fence_to_handle) > > #define DRM_IOCTL_AMDGPU_SCHED DRM_IOW(DRM_COMMAND_BASE + DRM_= AMDGPU_SCHED, union drm_amdgpu_sched) > > +#define DRM_IOCTL_AMDGPU_SETPARAM DRM_IOW(DRM_COMMAND_BASE + DRM_= AMDGPU_SETPARAM, struct drm_amdgpu_setparam) > > > > /** > > * DOC: memory domains > > @@ -306,6 +308,14 @@ union drm_amdgpu_sched { > > struct drm_amdgpu_sched_in in; > > }; > > > > +#define AMDGPU_SETPARAM_NO_IMPLICIT_SYNC 1 > > + > > +struct drm_amdgpu_setparam { > > + /* AMDGPU_SETPARAM_* */ > > + __u32 param; > > + __u32 value; > > +}; > > + > > /* > > * This is not a reliable API and you should expect it to fail for any > > * number of reasons and have fallback path that do not use userptr to > > -- > > 2.32.0.rc2 > > --=20 Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch