From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E782EC433EF for ; Mon, 28 Mar 2022 04:57:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id 6FF3E4169E; Mon, 28 Mar 2022 04:57:05 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id F9AErst9FOhf; Mon, 28 Mar 2022 04:57:02 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp4.osuosl.org (Postfix) with ESMTPS id 0D8A6416A3; Mon, 28 Mar 2022 04:57:01 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D2714C001D; Mon, 28 Mar 2022 04:57:01 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id CDE5CC0012 for ; Mon, 28 Mar 2022 04:57:00 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id B515A81A18 for ; Mon, 28 Mar 2022 04:57:00 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp1.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8nCeBKkHXsCl for ; Mon, 28 Mar 2022 04:56:57 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by smtp1.osuosl.org (Postfix) with ESMTPS id 9ED1081A16 for ; Mon, 28 Mar 2022 04:56:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648443416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2gkNQpu4kANGFFL3VkEjiCuQFdxFNSWesUdsMKBws3Q=; b=c0ejT8/5yJNtNY8QLDuKiQkGTn5xp0pubPylpyyVn7FuymOOubWTGfA6ry71k6nzYoynzp mfENF0H5/VWswEq9v76+jlbWKLw7140z8STmHuwiO5IJs0BqpERVGwDWu6DtuKUVCyV71l 73VS85x8qy2fcgaC/O0zbblxQpM/SA4= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-541-A_uL4alUMe2UNqZU2ml1Dg-1; Mon, 28 Mar 2022 00:56:54 -0400 X-MC-Unique: A_uL4alUMe2UNqZU2ml1Dg-1 Received: by mail-lj1-f200.google.com with SMTP id b2-20020a2e8942000000b002497c9833c2so5414102ljk.8 for ; Sun, 27 Mar 2022 21:56:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=2gkNQpu4kANGFFL3VkEjiCuQFdxFNSWesUdsMKBws3Q=; b=t51zVpg2/hfMveS5zuRGqIVA7XILOKmoR9ggMWQ7tui3emVM4OlboP86Tpw8j9m+hr 89AFHwuwlLxXivVvO+60fStwcwISo4aU4ZnkcngO/qBmMn2zFh9xJE2jtV68l2Bn3U00 yU6+Dmecj+ddsuPFeej/iP2TYQi6gZmi65+72pH0dEL9TvoXC/rGoA4DRhQzQd1mPvAl usT50Rg/uEEHXrm5E2yPx4crl2B8IFR3BcVlkfUU6wqfEo3ihvW4K0yz2lgl5eLGFHRw VCwylD0W2AKd3yUHP/JUZIilXOTZLCFxSpGcUSUwoMUyXEC0NXExLAPnJVzI3xgQMzOD zsMw== X-Gm-Message-State: AOAM532EN+L79WoEXtNsDkS0Ad3N/A6olowwn1ZCljhu0BPTG8cuB9XQ eBS0BzIYdnt4WAN2ql3KIMKjh79EBdkO+FbZjG5k4AdE1TG4Cz1AvLYPi0OxqcmIq7mDmTaSwcY scDiUYII1+aeR1acBvlSlH563LzM2pDqsHADykw/1gAzw6bYi/+gcrNqWkw== X-Received: by 2002:a05:6512:1395:b0:446:d382:79a5 with SMTP id p21-20020a056512139500b00446d38279a5mr17926720lfa.210.1648443412758; Sun, 27 Mar 2022 21:56:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxt/bzJ9zwsNh8eSBVNz+omWW+KwYlD+ICzTaTEWGL/Qf6hUvuQA+7bxaQNlLmtrAeX/T1GVfdn1/f74Wo+kio= X-Received: by 2002:a05:6512:1395:b0:446:d382:79a5 with SMTP id p21-20020a056512139500b00446d38279a5mr17926694lfa.210.1648443412391; Sun, 27 Mar 2022 21:56:52 -0700 (PDT) MIME-Version: 1.0 References: <20220325050947-mutt-send-email-mst@kernel.org> <20220325060659-mutt-send-email-mst@kernel.org> In-Reply-To: <20220325060659-mutt-send-email-mst@kernel.org> From: Jason Wang Date: Mon, 28 Mar 2022 12:56:41 +0800 Message-ID: Subject: Re: To: "Michael S. Tsirkin" Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Cc: "Paul E. McKenney" , Peter Zijlstra , Marc Zyngier , Keir Fraser , linux-kernel , virtualization , Thomas Gleixner X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gRnJpLCBNYXIgMjUsIDIwMjIgYXQgNjoxMCBQTSBNaWNoYWVsIFMuIFRzaXJraW4gPG1zdEBy ZWRoYXQuY29tPiB3cm90ZToKPgo+IE9uIEZyaSwgTWFyIDI1LCAyMDIyIGF0IDA1OjIwOjE5UE0g KzA4MDAsIEphc29uIFdhbmcgd3JvdGU6Cj4gPiBPbiBGcmksIE1hciAyNSwgMjAyMiBhdCA1OjEw IFBNIE1pY2hhZWwgUy4gVHNpcmtpbiA8bXN0QHJlZGhhdC5jb20+IHdyb3RlOgo+ID4gPgo+ID4g PiBPbiBGcmksIE1hciAyNSwgMjAyMiBhdCAwMzo1MjowMFBNICswODAwLCBKYXNvbiBXYW5nIHdy b3RlOgo+ID4gPiA+IE9uIEZyaSwgTWFyIDI1LCAyMDIyIGF0IDI6MzEgUE0gTWljaGFlbCBTLiBU c2lya2luIDxtc3RAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4gPgo+ID4gPiA+ID4gQmNjOgo+ ID4gPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSCAzLzNdIHZpcnRpbzogaGFyZGVuIHZyaW5nIElS UQo+ID4gPiA+ID4gTWVzc2FnZS1JRDogPDIwMjIwMzI1MDIxNDIyLW11dHQtc2VuZC1lbWFpbC1t c3RAa2VybmVsLm9yZz4KPiA+ID4gPiA+IFJlcGx5LVRvOgo+ID4gPiA+ID4gSW4tUmVwbHktVG86 IDxmNzA0NjMwMy03ZDdkLWUzOWYtM2M3MS0zNjg4MTI2Y2M4MTJAcmVkaGF0LmNvbT4KPiA+ID4g PiA+Cj4gPiA+ID4gPiBPbiBGcmksIE1hciAyNSwgMjAyMiBhdCAxMTowNDowOEFNICswODAwLCBK YXNvbiBXYW5nIHdyb3RlOgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiDlnKggMjAyMi8zLzI0IOS4 i+WNiDc6MDMsIE1pY2hhZWwgUy4gVHNpcmtpbiDlhpnpgZM6Cj4gPiA+ID4gPiA+ID4gT24gVGh1 LCBNYXIgMjQsIDIwMjIgYXQgMDQ6NDA6MDRQTSArMDgwMCwgSmFzb24gV2FuZyB3cm90ZToKPiA+ ID4gPiA+ID4gPiA+IFRoaXMgaXMgYSByZXdvcmsgb24gdGhlIHByZXZpb3VzIElSUSBoYXJkZW5p bmcgdGhhdCBpcyBkb25lIGZvcgo+ID4gPiA+ID4gPiA+ID4gdmlydGlvLXBjaSB3aGVyZSBzZXZl cmFsIGRyYXdiYWNrcyB3ZXJlIGZvdW5kIGFuZCB3ZXJlIHJldmVydGVkOgo+ID4gPiA+ID4gPiA+ ID4KPiA+ID4gPiA+ID4gPiA+IDEpIHRyeSB0byB1c2UgSVJRRl9OT19BVVRPRU4gd2hpY2ggaXMg bm90IGZyaWVuZGx5IHRvIGFmZmluaXR5IG1hbmFnZWQgSVJRCj4gPiA+ID4gPiA+ID4gPiAgICAg dGhhdCBpcyB1c2VkIGJ5IHNvbWUgZGV2aWNlIHN1Y2ggYXMgdmlydGlvLWJsawo+ID4gPiA+ID4g PiA+ID4gMikgZG9uZSBvbmx5IGZvciBQQ0kgdHJhbnNwb3J0Cj4gPiA+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+ID4gSW4gdGhpcyBwYXRjaCwgd2UgdHJpZXMgdG8gYm9ycm93IHRoZSBpZGVhIGZy b20gdGhlIElOVFggSVJRIGhhcmRlbmluZwo+ID4gPiA+ID4gPiA+ID4gaW4gdGhlIHJldmVydGVk IGNvbW1pdCAwODBjZDdjM2FjODcgKCJ2aXJ0aW8tcGNpOiBoYXJkZW4gSU5UWCBpbnRlcnJ1cHRz IikKPiA+ID4gPiA+ID4gPiA+IGJ5IGludHJvZHVjaW5nIGEgZ2xvYmFsIGlycV9zb2Z0X2VuYWJs ZWQgdmFyaWFibGUgZm9yIGVhY2gKPiA+ID4gPiA+ID4gPiA+IHZpcnRpb19kZXZpY2UuIFRoZW4g d2UgY2FuIHRvIHRvZ2dsZSBpdCBkdXJpbmcKPiA+ID4gPiA+ID4gPiA+IHZpcnRpb19yZXNldF9k ZXZpY2UoKS92aXJ0aW9fZGV2aWNlX3JlYWR5KCkuIEEgc3luY2hvcm5pemVfcmN1KCkgaXMKPiA+ ID4gPiA+ID4gPiA+IHVzZWQgaW4gdmlydGlvX3Jlc2V0X2RldmljZSgpIHRvIHN5bmNocm9uaXpl IHdpdGggdGhlIElSUSBoYW5kbGVycy4gSW4KPiA+ID4gPiA+ID4gPiA+IHRoZSBmdXR1cmUsIHdl IG1heSBwcm92aWRlIGNvbmZpZ19vcHMgZm9yIHRoZSB0cmFuc3BvcnQgdGhhdCBkb2Vzbid0Cj4g PiA+ID4gPiA+ID4gPiB1c2UgSVJRLiBXaXRoIHRoaXMsIHZyaW5nX2ludGVycnVwdCgpIGNhbiBy ZXR1cm4gY2hlY2sgYW5kIGVhcmx5IGlmCj4gPiA+ID4gPiA+ID4gPiBpcnFfc29mdF9lbmFibGVk IGlzIGZhbHNlLiBUaGlzIGxlYWQgdG8gc21wX2xvYWRfYWNxdWlyZSgpIHRvIGJlIHVzZWQKPiA+ ID4gPiA+ID4gPiA+IGJ1dCB0aGUgY29zdCBzaG91bGQgYmUgYWNjZXB0YWJsZS4KPiA+ID4gPiA+ ID4gPiBNYXliZSBpdCBzaG91bGQgYmUgYnV0IGlzIGl0PyBDYW4ndCB3ZSB1c2Ugc3luY2hyb25p emVfaXJxIGluc3RlYWQ/Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEV2ZW4g aWYgd2UgYWxsb3cgdGhlIHRyYW5zcG9ydCBkcml2ZXIgdG8gc3luY2hvcm5pemUgdGhyb3VnaAo+ ID4gPiA+ID4gPiBzeW5jaHJvbml6ZV9pcnEoKSB3ZSBzdGlsbCBuZWVkIGEgY2hlY2sgaW4gdGhl IHZyaW5nX2ludGVycnVwdCgpLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBXZSBkbyBzb21ldGhp bmcgbGlrZSB0aGUgZm9sbG93aW5nIHByZXZpb3VzbHk6Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ ICAgICAgICAgaWYgKCFSRUFEX09OQ0UodnBfZGV2LT5pbnR4X3NvZnRfZW5hYmxlZCkpCj4gPiA+ ID4gPiA+ICAgICAgICAgICAgICAgICByZXR1cm4gSVJRX05PTkU7Cj4gPiA+ID4gPiA+Cj4gPiA+ ID4gPiA+IEJ1dCBpdCBsb29rcyBsaWtlIGEgYnVnIHNpbmNlIHNwZWN1bGF0aXZlIHJlYWQgY2Fu IGJlIGRvbmUgYmVmb3JlIHRoZSBjaGVjawo+ID4gPiA+ID4gPiB3aGVyZSB0aGUgaW50ZXJydXB0 IGhhbmRsZXIgY2FuJ3Qgc2VlIHRoZSB1bmNvbW1pdHRlZCBzZXR1cCB3aGljaCBpcyBkb25lIGJ5 Cj4gPiA+ID4gPiA+IHRoZSBkcml2ZXIuCj4gPiA+ID4gPgo+ID4gPiA+ID4gSSBkb24ndCB0aGlu ayBzbyAtIGlmIHlvdSBzeW5jIGFmdGVyIHNldHRpbmcgdGhlIHZhbHVlIHRoZW4KPiA+ID4gPiA+ IHlvdSBhcmUgZ3VhcmFudGVlZCB0aGF0IGFueSBoYW5kbGVyIHJ1bm5pbmcgYWZ0ZXJ3YXJkcwo+ ID4gPiA+ID4gd2lsbCBzZWUgdGhlIG5ldyB2YWx1ZS4KPiA+ID4gPgo+ID4gPiA+IFRoZSBwcm9i bGVtIGlzIG5vdCBkaXNhYmxlZCBidXQgdGhlIGVuYWJsZS4KPiA+ID4KPiA+ID4gU28gYSBtaXNi ZWhhdmluZyBkZXZpY2UgY2FuIGxvc2UgaW50ZXJydXB0cz8gVGhhdCdzIG5vdCBhIHByb2JsZW0g YXQgYWxsCj4gPiA+IGltby4KPiA+Cj4gPiBJdCdzIHRoZSBpbnRlcnJ1cHQgcmFpc2VkIGJlZm9y ZSBzZXR0aW5nIGlycV9zb2Z0X2VuYWJsZWQgdG8gdHJ1ZToKPiA+Cj4gPiBDUFUgMCBwcm9iZSkg ZHJpdmVyIHNwZWNpZmljIHNldHVwIChub3QgY29tbWl0ZWQpCj4gPiBDUFUgMSBJUlEgaGFuZGxl cikgcmVhZCB0aGUgdW5pbml0aWFsaXplZCB2YXJpYWJsZQo+ID4gQ1BVIDAgcHJvYmUpIHNldCBp cnFfc29mdF9lbmFibGVkIHRvIHRydWUKPiA+IENQVSAxIElSUSBoYW5kbGVyKSByZWFkIGlycV9z b2Z0X2VuYWJsZSBhcyB0cnVlCj4gPiBDUFUgMSBJUlEgaGFuZGxlcikgdXNlIHRoZSB1bmluaXRp YWxpemVkIHZhcmlhYmxlCj4gPgo+ID4gVGhhbmtzCj4KPiBZZWEsIGl0IGh1cnRzIGlmIHlvdSBk byBpdC4gIFNvIGRvIG5vdCBkbyBpdCB0aGVuIDspLgo+Cj4gaXJxX3NvZnRfZW5hYmxlZCAoSSB0 aGluayBkcml2ZXJfb2sgb3Igc3RhdHVzIGlzIGEgYmV0dGVyIG5hbWUpCgpJIGNhbiBjaGFuZ2Ug aXQgdG8gZHJpdmVyX29rLgoKPiBzaG91bGQgYmUgaW5pdGlhbGl6ZWQgdG8gZmFsc2UgKmJlZm9y ZSogaXJxIGlzIHJlcXVlc3RlZC4KPgo+IEFuZCByZXF1ZXN0aW5nIGlycSBjb21taXRzIGFsbCBt ZW1vcnkgb3RoZXJ3aXNlIGFsbCBkcml2ZXJzIHdvdWxkIGJlCj4gYnJva2VuLAoKU28gSSB0aGlu ayB3ZSBtaWdodCB0YWxrIGRpZmZlcmVudCBpc3N1ZXM6CgoxKSBXaGV0aGVyIHJlcXVlc3RfaXJx KCkgY29tbWl0cyB0aGUgcHJldmlvdXMgc2V0dXBzLCBJIHRoaW5rIHRoZQphbnN3ZXIgaXMgeWVz LCBzaW5jZSB0aGUgc3Bpbl91bmxvY2sgb2YgZGVzYy0+bG9jayAocmVsZWFzZSkgY2FuCmd1YXJh bnRlZSB0aGlzIHRob3VnaCB0aGVyZSBzZWVtcyBubyBkb2N1bWVudGF0aW9uIGFyb3VuZApyZXF1 ZXN0X2lycSgpIHRvIHNheSB0aGlzLgoKQW5kIEkgY2FuIHNlZSBhdCBsZWFzdCBkcml2ZXJzL3Zp ZGVvL2ZiZGV2L29tYXAyL29tYXBmYi9kc3MvZGlzcGMuYyBpcwp1c2luZyBzbXBfd21iKCkgYmVm b3JlIHRoZSByZXF1ZXN0X2lycSgpLgoKQW5kIGV2ZW4gaWYgd3JpdGUgaXMgb3JkZXJlZCB3ZSBz dGlsbCBuZWVkIHJlYWQgdG8gYmUgb3JkZXJlZCB0byBiZQpwYWlyZWQgd2l0aCB0aGF0LgoKPiBp ZiBpdCBkb2Vzbid0IGl0IGp1c3QgbmVlZHMgdG8gYmUgZml4ZWQsIG5vdCB3b3JrZWQgYXJvdW5k IGluCj4gdmlydGlvLgoKMikgdmlydGlvIGRyaXZlcnMgbWlnaHQgZG8gYSBsb3Qgb2Ygc2V0dXBz IGJldHdlZW4gcmVxdWVzdF9pcnEoKSBhbmQKdmlydGlvX2RldmljZV9yZWFkeSgpOgoKcmVxdWVz dF9pcnEoKQpkcml2ZXIgc3BlY2lmaWMgc2V0dXBzCnZpcnRpb19kZXZpY2VfcmVhZHkoKQoKQ1BV IDAgcHJvYmUpIHJlcXVlc3RfaXJxKCkKQ1BVIDEgSVJRIGhhbmRsZXIpIHJlYWQgdGhlIHVuaW5p dGlhbGl6ZWQgdmFyaWFibGUKQ1BVIDAgcHJvYmUpIGRyaXZlciBzcGVjaWZpYyBzZXR1cHMKQ1BV IDAgcHJvYmUpIHNtcF9zdG9yZV9yZWxlYXNlKGludHJfc29mdF9lbmFibGVkLCB0cnVlKSwgY29t bWl0IHRoZSBzZXR1cHMKQ1BVIDEgSVJRIGhhbmRsZXIpIHJlYWQgaXJxX3NvZnRfZW5hYmxlIGFz IHRydWUKQ1BVIDEgSVJRIGhhbmRsZXIpIHVzZSB0aGUgdW5pbml0aWFsaXplZCB2YXJpYWJsZQoK VGhhbmtzCgo+Cj4KPiA+ID4KPiA+ID4gPiBXZSB1c2Ugc21wX3N0b3JlX3JlbGFzZSgpCj4gPiA+ ID4gdG8gbWFrZSBzdXJlIHRoZSBkcml2ZXIgY29tbWl0cyB0aGUgc2V0dXAgYmVmb3JlIGVuYWJs aW5nIHRoZSBpcnEuIEl0Cj4gPiA+ID4gbWVhbnMgdGhlIHJlYWQgbmVlZHMgdG8gYmUgb3JkZXJl ZCBhcyB3ZWxsIGluIHZyaW5nX2ludGVycnVwdCgpLgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ ID4gQWx0aG91Z2ggSSBjb3VsZG4ndCBmaW5kIGFueXRoaW5nIGFib3V0IHRoaXMgaW4gbWVtb3J5 LWJhcnJpZXJzLnR4dAo+ID4gPiA+ID4gd2hpY2ggc3VycHJpc2VzIG1lLgo+ID4gPiA+ID4KPiA+ ID4gPiA+IENDIFBhdWwgdG8gaGVscCBtYWtlIHN1cmUgSSdtIHJpZ2h0Lgo+ID4gPiA+ID4KPiA+ ID4gPiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+IFRvIGF2b2lk IGJyZWFraW5nIGxlZ2FjeSBkZXZpY2Ugd2hpY2ggY2FuIHNlbmQgSVJRIGJlZm9yZSBEUklWRVJf T0ssIGEKPiA+ID4gPiA+ID4gPiA+IG1vZHVsZSBwYXJhbWV0ZXIgaXMgaW50cm9kdWNlZCB0byBl bmFibGUgdGhlIGhhcmRlbmluZyBzbyBmdW5jdGlvbgo+ID4gPiA+ID4gPiA+ID4gaGFyZGVuaW5n IGlzIGRpc2FibGVkIGJ5IGRlZmF1bHQuCj4gPiA+ID4gPiA+ID4gV2hpY2ggZGV2aWNlcyBhcmUg dGhlc2U/IEhvdyBjb21lIHRoZXkgc2VuZCBhbiBpbnRlcnJ1cHQgYmVmb3JlIHRoZXJlCj4gPiA+ ID4gPiA+ID4gYXJlIGFueSBidWZmZXJzIGluIGFueSBxdWV1ZXM/Cj4gPiA+ID4gPiA+Cj4gPiA+ ID4gPiA+Cj4gPiA+ID4gPiA+IEkgY29waWVkIHRoaXMgZnJvbSB0aGUgY29tbWl0IGxvZyBmb3Ig MjJiNzA1MGEwMjRkNwo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiAiCj4gPiA+ID4gPiA+Cj4gPiA+ ID4gPiA+ICAgICBUaGlzIGNoYW5nZSB3aWxsIGFsc28gYmVuZWZpdCBvbGQgaHlwZXJ2aXNvcnMg KGJlZm9yZSAyMDA5KQo+ID4gPiA+ID4gPiAgICAgdGhhdCBzZW5kIGludGVycnVwdHMgd2l0aG91 dCBjaGVja2luZyBEUklWRVJfT0s6IHByZXZpb3VzbHksCj4gPiA+ID4gPiA+ICAgICB0aGUgY2Fs bGJhY2sgY291bGQgcmFjZSB3aXRoIGRyaXZlci1zcGVjaWZpYyBpbml0aWFsaXphdGlvbi4KPiA+ ID4gPiA+ID4gIgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBJZiB0aGlzIGlzIG9ubHkgZm9yIGNv bmZpZyBpbnRlcnJ1cHQsIEkgY2FuIHJlbW92ZSB0aGUgYWJvdmUgbG9nLgo+ID4gPiA+ID4KPiA+ ID4gPiA+Cj4gPiA+ID4gPiBUaGlzIGlzIG9ubHkgZm9yIGNvbmZpZyBpbnRlcnJ1cHQuCj4gPiA+ ID4KPiA+ID4gPiBPay4KPiA+ID4gPgo+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPiA+ID4gTm90ZSB0aGF0IHRoZSBoYXJkZW5pbmcgaXMgb25seSBkb25lIGZv ciB2cmluZyBpbnRlcnJ1cHQgc2luY2UgdGhlCj4gPiA+ID4gPiA+ID4gPiBjb25maWcgaW50ZXJy dXB0IGhhcmRlbmluZyBpcyBhbHJlYWR5IGRvbmUgaW4gY29tbWl0IDIyYjcwNTBhMDI0ZDcKPiA+ ID4gPiA+ID4gPiA+ICgidmlydGlvOiBkZWZlciBjb25maWcgY2hhbmdlZCBub3RpZmljYXRpb25z IikuIEJ1dCB0aGUgbWV0aG9kIHRoYXQgaXMKPiA+ID4gPiA+ID4gPiA+IHVzZWQgYnkgY29uZmln IGludGVycnVwdCBjYW4ndCBiZSByZXVzZWQgYnkgdGhlIHZyaW5nIGludGVycnVwdAo+ID4gPiA+ ID4gPiA+ID4gaGFuZGxlciBiZWNhdXNlIGl0IHVzZXMgc3BpbmxvY2sgdG8gZG8gdGhlIHN5bmNo cm9uaXphdGlvbiB3aGljaCBpcwo+ID4gPiA+ID4gPiA+ID4gZXhwZW5zaXZlLgo+ID4gPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IEphc29uIFdhbmcgPGphc293YW5n QHJlZGhhdC5jb20+Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+IC0tLQo+ID4gPiA+ID4g PiA+ID4gICBkcml2ZXJzL3ZpcnRpby92aXJ0aW8uYyAgICAgICB8IDE5ICsrKysrKysrKysrKysr KysrKysKPiA+ID4gPiA+ID4gPiA+ICAgZHJpdmVycy92aXJ0aW8vdmlydGlvX3JpbmcuYyAgfCAg OSArKysrKysrKy0KPiA+ID4gPiA+ID4gPiA+ICAgaW5jbHVkZS9saW51eC92aXJ0aW8uaCAgICAg ICAgfCAgNCArKysrCj4gPiA+ID4gPiA+ID4gPiAgIGluY2x1ZGUvbGludXgvdmlydGlvX2NvbmZp Zy5oIHwgMjUgKysrKysrKysrKysrKysrKysrKysrKysrKwo+ID4gPiA+ID4gPiA+ID4gICA0IGZp bGVzIGNoYW5nZWQsIDU2IGluc2VydGlvbnMoKyksIDEgZGVsZXRpb24oLSkKPiA+ID4gPiA+ID4g PiA+Cj4gPiA+ID4gPiA+ID4gPiBkaWZmIC0tZ2l0IGEvZHJpdmVycy92aXJ0aW8vdmlydGlvLmMg Yi9kcml2ZXJzL3ZpcnRpby92aXJ0aW8uYwo+ID4gPiA+ID4gPiA+ID4gaW5kZXggOGRkZTQ0ZWEw NDRhLi44NWUzMzFlZmE5Y2MgMTAwNjQ0Cj4gPiA+ID4gPiA+ID4gPiAtLS0gYS9kcml2ZXJzL3Zp cnRpby92aXJ0aW8uYwo+ID4gPiA+ID4gPiA+ID4gKysrIGIvZHJpdmVycy92aXJ0aW8vdmlydGlv LmMKPiA+ID4gPiA+ID4gPiA+IEBAIC03LDYgKzcsMTIgQEAKPiA+ID4gPiA+ID4gPiA+ICAgI2lu Y2x1ZGUgPGxpbnV4L29mLmg+Cj4gPiA+ID4gPiA+ID4gPiAgICNpbmNsdWRlIDx1YXBpL2xpbnV4 L3ZpcnRpb19pZHMuaD4KPiA+ID4gPiA+ID4gPiA+ICtzdGF0aWMgYm9vbCBpcnFfaGFyZGVuaW5n ID0gZmFsc2U7Cj4gPiA+ID4gPiA+ID4gPiArCj4gPiA+ID4gPiA+ID4gPiArbW9kdWxlX3BhcmFt KGlycV9oYXJkZW5pbmcsIGJvb2wsIDA0NDQpOwo+ID4gPiA+ID4gPiA+ID4gK01PRFVMRV9QQVJN X0RFU0MoaXJxX2hhcmRlbmluZywKPiA+ID4gPiA+ID4gPiA+ICsgICAgICAgICAgIkRpc2FsYmUg SVJRIHNvZnR3YXJlIHByb2Nlc3Npbmcgd2hlbiBpdCBpcyBub3QgZXhwZWN0ZWQiKTsKPiA+ID4g PiA+ID4gPiA+ICsKPiA+ID4gPiA+ID4gPiA+ICAgLyogVW5pcXVlIG51bWJlcmluZyBmb3Igdmly dGlvIGRldmljZXMuICovCj4gPiA+ID4gPiA+ID4gPiAgIHN0YXRpYyBERUZJTkVfSURBKHZpcnRp b19pbmRleF9pZGEpOwo+ID4gPiA+ID4gPiA+ID4gQEAgLTIyMCw2ICsyMjYsMTUgQEAgc3RhdGlj IGludCB2aXJ0aW9fZmVhdHVyZXNfb2soc3RydWN0IHZpcnRpb19kZXZpY2UgKmRldikKPiA+ID4g PiA+ID4gPiA+ICAgICogKi8KPiA+ID4gPiA+ID4gPiA+ICAgdm9pZCB2aXJ0aW9fcmVzZXRfZGV2 aWNlKHN0cnVjdCB2aXJ0aW9fZGV2aWNlICpkZXYpCj4gPiA+ID4gPiA+ID4gPiAgIHsKPiA+ID4g PiA+ID4gPiA+ICsgLyoKPiA+ID4gPiA+ID4gPiA+ICsgICogVGhlIGJlbG93IHN5bmNocm9uaXpl X3JjdSgpIGd1YXJhbnRlZXMgdGhhdCBhbnkKPiA+ID4gPiA+ID4gPiA+ICsgICogaW50ZXJydXB0 IGZvciB0aGlzIGxpbmUgYXJyaXZpbmcgYWZ0ZXIKPiA+ID4gPiA+ID4gPiA+ICsgICogc3luY2hy b25pemVfcmN1KCkgaGFzIGNvbXBsZXRlZCBpcyBndWFyYW50ZWVkIHRvIHNlZQo+ID4gPiA+ID4g PiA+ID4gKyAgKiBpcnFfc29mdF9lbmFibGVkID09IGZhbHNlLgo+ID4gPiA+ID4gPiA+IE5ld3Mg dG8gbWUgSSBkaWQgbm90IGtub3cgc3luY2hyb25pemVfcmN1IGhhcyBhbnl0aGluZyB0byBkbwo+ ID4gPiA+ID4gPiA+IHdpdGggaW50ZXJydXB0cy4gRGlkIG5vdCB5b3UgaW50ZW5kIHRvIHVzZSBz eW5jaHJvbml6ZV9pcnE/Cj4gPiA+ID4gPiA+ID4gSSBhbSBub3QgZXZlbiAxMDAlIHN1cmUgc3lu Y2hyb25pemVfcmN1IGlzIGJ5IGRlc2lnbiBhIG1lbW9yeSBiYXJyaWVyCj4gPiA+ID4gPiA+ID4g dGhvdWdoIGl0J3MgbW9zdCBsaWtlbHkgaXMgLi4uCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+Cj4g PiA+ID4gPiA+IEFjY29yZGluZyB0byB0aGUgY29tbWVudCBhYm92ZSB0cmVlIFJDVSB2ZXJzaW9u IG9mIHN5bmNocm9uaXplX3JjdSgpOgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiAiIiIKPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gICogUkNVIHJlYWQtc2lkZSBjcml0aWNhbCBzZWN0aW9ucyBhcmUg ZGVsaW1pdGVkIGJ5IHJjdV9yZWFkX2xvY2soKQo+ID4gPiA+ID4gPiAgKiBhbmQgcmN1X3JlYWRf dW5sb2NrKCksIGFuZCBtYXkgYmUgbmVzdGVkLiAgSW4gYWRkaXRpb24sIGJ1dCBvbmx5IGluCj4g PiA+ID4gPiA+ICAqIHY1LjAgYW5kIGxhdGVyLCByZWdpb25zIG9mIGNvZGUgYWNyb3NzIHdoaWNo IGludGVycnVwdHMsIHByZWVtcHRpb24sCj4gPiA+ID4gPiA+ICAqIG9yIHNvZnRpcnFzIGhhdmUg YmVlbiBkaXNhYmxlZCBhbHNvIHNlcnZlIGFzIFJDVSByZWFkLXNpZGUgY3JpdGljYWwKPiA+ID4g PiA+ID4gICogc2VjdGlvbnMuICBUaGlzIGluY2x1ZGVzIGhhcmR3YXJlIGludGVycnVwdCBoYW5k bGVycywgc29mdGlycSBoYW5kbGVycywKPiA+ID4gPiA+ID4gICogYW5kIE5NSSBoYW5kbGVycy4K PiA+ID4gPiA+ID4gIiIiCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IFNvIGludGVycnVwdCBoYW5k bGVycyBhcmUgdHJlYXRlZCBhcyByZWFkLXNpZGUgY3JpdGljYWwgc2VjdGlvbnMuCj4gPiA+ID4g PiA+Cj4gPiA+ID4gPiA+IEFuZCBpdCBoYXMgdGhlIGNvbW1lbnQgZm9yIGV4cGxhaW4gdGhlIGJh cnJpZXI6Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ICIiIgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4g PiAgKiBOb3RlIHRoYXQgdGhpcyBndWFyYW50ZWUgaW1wbGllcyBmdXJ0aGVyIG1lbW9yeS1vcmRl cmluZyBndWFyYW50ZWVzLgo+ID4gPiA+ID4gPiAgKiBPbiBzeXN0ZW1zIHdpdGggbW9yZSB0aGFu IG9uZSBDUFUsIHdoZW4gc3luY2hyb25pemVfcmN1KCkgcmV0dXJucywKPiA+ID4gPiA+ID4gICog ZWFjaCBDUFUgaXMgZ3VhcmFudGVlZCB0byBoYXZlIGV4ZWN1dGVkIGEgZnVsbCBtZW1vcnkgYmFy cmllciBzaW5jZQo+ID4gPiA+ID4gPiAgKiB0aGUgZW5kIG9mIGl0cyBsYXN0IFJDVSByZWFkLXNp ZGUgY3JpdGljYWwgc2VjdGlvbiB3aG9zZSBiZWdpbm5pbmcKPiA+ID4gPiA+ID4gICogcHJlY2Vk ZWQgdGhlIGNhbGwgdG8gc3luY2hyb25pemVfcmN1KCkuICBJbiBhZGRpdGlvbiwgZWFjaCBDUFUg aGF2aW5nCj4gPiA+ID4gPiA+ICIiIgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBTbyBvbiBTTVAg aXQgcHJvdmlkZXMgYSBmdWxsIGJhcnJpZXIuIEFuZCBmb3IgVVAvdGlueSBSQ1Ugd2UgZG9uJ3Qg bmVlZCB0aGUKPiA+ID4gPiA+ID4gYmFycmllciwgaWYgdGhlIGludGVycnVwdCBjb21lIGFmdGVy IFdSSVRFX09OQ0UoKSBpdCB3aWxsIHNlZSB0aGUKPiA+ID4gPiA+ID4gaXJxX3NvZnRfZW5hYmxl ZCBhcyBmYWxzZS4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+Cj4gPiA+ID4gPiBZb3UgYXJlIHJpZ2h0 LiBTbyB0aGVuCj4gPiA+ID4gPiAxLiBJIGRvIG5vdCB0aGluayB3ZSBuZWVkIGxvYWRfYWNxdWly ZSAtIHdoeSBpcyBpdCBuZWVkZWQ/IEp1c3QKPiA+ID4gPiA+ICAgIFJFQURfT05DRSBzaG91bGQg ZG8uCj4gPiA+ID4KPiA+ID4gPiBTZWUgYWJvdmUuCj4gPiA+ID4KPiA+ID4gPiA+IDIuIGlzbid0 IHN5bmNocm9uaXplX2lycSBhbHNvIGRvaW5nIHRoZSBzYW1lIHRoaW5nPwo+ID4gPiA+Cj4gPiA+ ID4KPiA+ID4gPiBZZXMsIGJ1dCBpdCByZXF1aXJlcyBhIGNvbmZpZyBvcHMgc2luY2UgdGhlIElS USBrbm93bGVkZ2UgaXMgdHJhbnNwb3J0IHNwZWNpZmljLgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4g PiA+ID4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gKyAgKi8KPiA+ID4gPiA+ID4gPiA+ ICsgV1JJVEVfT05DRShkZXYtPmlycV9zb2Z0X2VuYWJsZWQsIGZhbHNlKTsKPiA+ID4gPiA+ID4g PiA+ICsgc3luY2hyb25pemVfcmN1KCk7Cj4gPiA+ID4gPiA+ID4gPiArCj4gPiA+ID4gPiA+ID4g PiAgICAgICAgICAgZGV2LT5jb25maWctPnJlc2V0KGRldik7Cj4gPiA+ID4gPiA+ID4gPiAgIH0K PiA+ID4gPiA+ID4gPiA+ICAgRVhQT1JUX1NZTUJPTF9HUEwodmlydGlvX3Jlc2V0X2RldmljZSk7 Cj4gPiA+ID4gPiA+ID4gUGxlYXNlIGFkZCBjb21tZW50IGV4cGxhaW5pbmcgd2hlcmUgaXQgd2ls bCBiZSBlbmFibGVkLgo+ID4gPiA+ID4gPiA+IEFsc28sIHdlICpyZWFsbHkqIGRvbid0IG5lZWQg dG8gc3luY2ggaWYgaXQgd2FzIGFscmVhZHkgZGlzYWJsZWQsCj4gPiA+ID4gPiA+ID4gbGV0J3Mg bm90IGFkZCB1c2VsZXNzIG92ZXJoZWFkIHRvIHRoZSBib290IHNlcXVlbmNlLgo+ID4gPiA+ID4g Pgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBPay4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiBAQCAtNDI3LDYgKzQ0Miwx MCBAQCBpbnQgcmVnaXN0ZXJfdmlydGlvX2RldmljZShzdHJ1Y3QgdmlydGlvX2RldmljZSAqZGV2 KQo+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgIHNwaW5fbG9ja19pbml0KCZkZXYtPmNvbmZpZ19s b2NrKTsKPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICBkZXYtPmNvbmZpZ19lbmFibGVkID0gZmFs c2U7Cj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgZGV2LT5jb25maWdfY2hhbmdlX3BlbmRpbmcg PSBmYWxzZTsKPiA+ID4gPiA+ID4gPiA+ICsgZGV2LT5pcnFfc29mdF9jaGVjayA9IGlycV9oYXJk ZW5pbmc7Cj4gPiA+ID4gPiA+ID4gPiArCj4gPiA+ID4gPiA+ID4gPiArIGlmIChkZXYtPmlycV9z b2Z0X2NoZWNrKQo+ID4gPiA+ID4gPiA+ID4gKyAgICAgICAgIGRldl9pbmZvKCZkZXYtPmRldiwg IklSUSBoYXJkZW5pbmcgaXMgZW5hYmxlZFxuIik7Cj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAg LyogV2UgYWx3YXlzIHN0YXJ0IGJ5IHJlc2V0dGluZyB0aGUgZGV2aWNlLCBpbiBjYXNlIGEgcHJl dmlvdXMKPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgKiBkcml2ZXIgbWVzc2VkIGl0IHVwLiAg VGhpcyBhbHNvIHRlc3RzIHRoYXQgY29kZSBwYXRoIGEgbGl0dGxlLiAqLwo+ID4gPiA+ID4gPiA+ IG9uZSBvZiB0aGUgcG9pbnRzIG9mIGhhcmRlbmluZyBpcyBpdCdzIGFsc28gaGVscGZ1bCBmb3Ig YnVnZ3kKPiA+ID4gPiA+ID4gPiBkZXZpY2VzLiB0aGlzIGZsYWcgZGVmZWF0cyB0aGUgcHVycG9z ZS4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gRG8geW91IG1lYW46Cj4gPiA+ ID4gPiA+Cj4gPiA+ID4gPiA+IDEpIHdlIG5lZWQgc29tZXRoaW5nIGxpa2UgY29uZmlnX2VuYWJs ZT8gVGhpcyBzZWVtcyBub3QgZWFzeSB0byBiZQo+ID4gPiA+ID4gPiBpbXBsZW1lbnRlZCB3aXRo b3V0IG9idmlvdXMgb3ZlcmhlYWQsIG1haW5seSB0aGUgc3luY2hyb25pemUgd2l0aCB0aGUKPiA+ ID4gPiA+ID4gaW50ZXJydXB0IGhhbmRsZXJzCj4gPiA+ID4gPgo+ID4gPiA+ID4gQnV0IHN5bmNo cm9uaXplIGlzIG9ubHkgb24gdGVhci1kb3duIHBhdGguIFRoYXQgaXMgbm90IGNyaXRpY2FsIGZv ciBhbnkKPiA+ID4gPiA+IHVzZXJzIGF0IHRoZSBtb21lbnQsIGV2ZW4gbGVzcyB0aGFuIHByb2Jl Lgo+ID4gPiA+Cj4gPiA+ID4gSSBtZWFudCBpZiB3ZSBoYXZlIHZxLT5pcnFfcGVuZGluZywgd2Ug bmVlZCB0byBjYWxsIHZyaW5nX2ludGVycnVwdCgpCj4gPiA+ID4gaW4gdGhlIHZpcnRpb19kZXZp Y2VfcmVhZHkoKSBhbmQgc3luY2hyb25pemUgdGhlIElSUSBoYW5kbGVycyB3aXRoCj4gPiA+ID4g c3BpbmxvY2sgb3Igb3RoZXJzLgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gPiAyKSBlbmFi bGUgdGhpcyBieSBkZWZhdWx0LCBzbyBJIGRvbid0IG9iamVjdCwgYnV0IHRoaXMgbWF5IGhhdmUg c29tZSByaXNrCj4gPiA+ID4gPiA+IGZvciBvbGQgaHlwZXJ2aXNvcnMKPiA+ID4gPiA+Cj4gPiA+ ID4gPgo+ID4gPiA+ID4gVGhlIHJpc2sgaWYgdGhlcmUncyBhIGRyaXZlciBhZGRpbmcgYnVmZmVy cyB3aXRob3V0IHNldHRpbmcgRFJJVkVSX09LLgo+ID4gPiA+Cj4gPiA+ID4gUHJvYmFibHkgbm90 LCB3ZSBoYXZlIGRldmljZXMgdGhhdCBhY2NlcHQgcmFuZG9tIGlucHV0cyBmcm9tIG91dHNpZGUs Cj4gPiA+ID4gbmV0LCBjb25zb2xlLCBpbnB1dCBldGMuIEkndmUgZG9uZSBhIHJvdW5kIG9mIGF1 ZGl0cyBvZiB0aGUgUWVtdQo+ID4gPiA+IGNvZGVzLiBUaGV5IGxvb2sgYWxsIGZpbmUgc2luY2Ug ZGF5MC4KPiA+ID4gPgo+ID4gPiA+ID4gU28gd2l0aCB0aGlzIGFwcHJvYWNoLCBob3cgYWJvdXQg d2UgcmVuYW1lIHRoZSBmbGFnICJkcml2ZXJfb2siPwo+ID4gPiA+ID4gQW5kIHRoZW4gYWRkX2J1 ZiBjYW4gYWN0dWFsbHkgdGVzdCBpdCBhbmQgQlVHX09OIGlmIG5vdCB0aGVyZSAgKGF0IGxlYXN0 Cj4gPiA+ID4gPiBpbiB0aGUgZGVidWcgYnVpbGQpLgo+ID4gPiA+Cj4gPiA+ID4gVGhpcyBsb29r cyBsaWtlIGEgaGFyZGVuaW5nIG9mIHRoZSBkcml2ZXIgaW4gdGhlIGNvcmUgaW5zdGVhZCBvZiB0 aGUKPiA+ID4gPiBkZXZpY2UuIEkgdGhpbmsgaXQgY2FuIGJlIGRvbmUgYnV0IGluIGEgc2VwYXJh dGUgc2VyaWVzLgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gQW5kIGdvaW5nIGRvd24gZnJv bSB0aGVyZSwgaG93IGFib3V0IHdlIGNhY2hlIHN0YXR1cyBpbiB0aGUKPiA+ID4gPiA+IGRldmlj ZT8gVGhlbiB3ZSBkb24ndCBuZWVkIHRvIGtlZXAgcmUtcmVhZGluZyBpdCBldmVyeSB0aW1lLAo+ ID4gPiA+ID4gc3BlZWRpbmcgYm9vdCB1cCBhIHRpbnkgYml0Lgo+ID4gPiA+Cj4gPiA+ID4gSSBk b24ndCBmdWxseSB1bmRlcnN0YW5kIGhlcmUsIGFjdHVhbGx5IHNwZWMgcmVxdWlyZXMgc3RhdHVz IHRvIGJlCj4gPiA+ID4gcmVhZCBiYWNrIGZvciB2YWxpZGF0aW9uIGluIG1hbnkgY2FzZXMuCj4g PiA+ID4KPiA+ID4gPiBUaGFua3MKPiA+ID4gPgo+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvdmlydGlvL3ZpcnRp b19yaW5nLmMgYi9kcml2ZXJzL3ZpcnRpby92aXJ0aW9fcmluZy5jCj4gPiA+ID4gPiA+ID4gPiBp bmRleCA5NjJmMTQ3N2IxZmEuLjAxNzBmOGM3ODRkOCAxMDA2NDQKPiA+ID4gPiA+ID4gPiA+IC0t LSBhL2RyaXZlcnMvdmlydGlvL3ZpcnRpb19yaW5nLmMKPiA+ID4gPiA+ID4gPiA+ICsrKyBiL2Ry aXZlcnMvdmlydGlvL3ZpcnRpb19yaW5nLmMKPiA+ID4gPiA+ID4gPiA+IEBAIC0yMTQ0LDEwICsy MTQ0LDE3IEBAIHN0YXRpYyBpbmxpbmUgYm9vbCBtb3JlX3VzZWQoY29uc3Qgc3RydWN0IHZyaW5n X3ZpcnRxdWV1ZSAqdnEpCj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgcmV0dXJuIHZxLT5wYWNr ZWRfcmluZyA/IG1vcmVfdXNlZF9wYWNrZWQodnEpIDogbW9yZV91c2VkX3NwbGl0KHZxKTsKPiA+ ID4gPiA+ID4gPiA+ICAgfQo+ID4gPiA+ID4gPiA+ID4gLWlycXJldHVybl90IHZyaW5nX2ludGVy cnVwdChpbnQgaXJxLCB2b2lkICpfdnEpCj4gPiA+ID4gPiA+ID4gPiAraXJxcmV0dXJuX3QgdnJp bmdfaW50ZXJydXB0KGludCBpcnEsIHZvaWQgKnYpCj4gPiA+ID4gPiA+ID4gPiAgIHsKPiA+ID4g PiA+ID4gPiA+ICsgc3RydWN0IHZpcnRxdWV1ZSAqX3ZxID0gdjsKPiA+ID4gPiA+ID4gPiA+ICsg c3RydWN0IHZpcnRpb19kZXZpY2UgKnZkZXYgPSBfdnEtPnZkZXY7Cj4gPiA+ID4gPiA+ID4gPiAg ICAgICAgICAgc3RydWN0IHZyaW5nX3ZpcnRxdWV1ZSAqdnEgPSB0b192dnEoX3ZxKTsKPiA+ID4g PiA+ID4gPiA+ICsgaWYgKCF2aXJ0aW9faXJxX3NvZnRfZW5hYmxlZCh2ZGV2KSkgewo+ID4gPiA+ ID4gPiA+ID4gKyAgICAgICAgIGRldl93YXJuX29uY2UoJnZkZXYtPmRldiwgInZpcnRpbyB2cmlu ZyBJUlEgcmFpc2VkIGJlZm9yZSBEUklWRVJfT0siKTsKPiA+ID4gPiA+ID4gPiA+ICsgICAgICAg ICByZXR1cm4gSVJRX05PTkU7Cj4gPiA+ID4gPiA+ID4gPiArIH0KPiA+ID4gPiA+ID4gPiA+ICsK PiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICBpZiAoIW1vcmVfdXNlZCh2cSkpIHsKPiA+ID4gPiA+ ID4gPiA+ICAgICAgICAgICAgICAgICAgIHByX2RlYnVnKCJ2aXJ0cXVldWUgaW50ZXJydXB0IHdp dGggbm8gd29yayBmb3IgJXBcbiIsIHZxKTsKPiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICAgICAg ICAgIHJldHVybiBJUlFfTk9ORTsKPiA+ID4gPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9pbmNsdWRl L2xpbnV4L3ZpcnRpby5oIGIvaW5jbHVkZS9saW51eC92aXJ0aW8uaAo+ID4gPiA+ID4gPiA+ID4g aW5kZXggNTQ2NGYzOTg5MTJhLi45NTdkNmFkNjA0YWMgMTAwNjQ0Cj4gPiA+ID4gPiA+ID4gPiAt LS0gYS9pbmNsdWRlL2xpbnV4L3ZpcnRpby5oCj4gPiA+ID4gPiA+ID4gPiArKysgYi9pbmNsdWRl L2xpbnV4L3ZpcnRpby5oCj4gPiA+ID4gPiA+ID4gPiBAQCAtOTUsNiArOTUsOCBAQCBkbWFfYWRk cl90IHZpcnRxdWV1ZV9nZXRfdXNlZF9hZGRyKHN0cnVjdCB2aXJ0cXVldWUgKnZxKTsKPiA+ID4g PiA+ID4gPiA+ICAgICogQGZhaWxlZDogc2F2ZWQgdmFsdWUgZm9yIFZJUlRJT19DT05GSUdfU19G QUlMRUQgYml0IChmb3IgcmVzdG9yZSkKPiA+ID4gPiA+ID4gPiA+ICAgICogQGNvbmZpZ19lbmFi bGVkOiBjb25maWd1cmF0aW9uIGNoYW5nZSByZXBvcnRpbmcgZW5hYmxlZAo+ID4gPiA+ID4gPiA+ ID4gICAgKiBAY29uZmlnX2NoYW5nZV9wZW5kaW5nOiBjb25maWd1cmF0aW9uIGNoYW5nZSByZXBv cnRlZCB3aGlsZSBkaXNhYmxlZAo+ID4gPiA+ID4gPiA+ID4gKyAqIEBpcnFfc29mdF9jaGVjazog d2hldGhlciBvciBub3QgdG8gY2hlY2sgQGlycV9zb2Z0X2VuYWJsZWQKPiA+ID4gPiA+ID4gPiA+ ICsgKiBAaXJxX3NvZnRfZW5hYmxlZDogY2FsbGJhY2tzIGVuYWJsZWQKPiA+ID4gPiA+ID4gPiA+ ICAgICogQGNvbmZpZ19sb2NrOiBwcm90ZWN0cyBjb25maWd1cmF0aW9uIGNoYW5nZSByZXBvcnRp bmcKPiA+ID4gPiA+ID4gPiA+ICAgICogQGRldjogdW5kZXJseWluZyBkZXZpY2UuCj4gPiA+ID4g PiA+ID4gPiAgICAqIEBpZDogdGhlIGRldmljZSB0eXBlIGlkZW50aWZpY2F0aW9uICh1c2VkIHRv IG1hdGNoIGl0IHdpdGggYSBkcml2ZXIpLgo+ID4gPiA+ID4gPiA+ID4gQEAgLTEwOSw2ICsxMTEs OCBAQCBzdHJ1Y3QgdmlydGlvX2RldmljZSB7Cj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgYm9v bCBmYWlsZWQ7Cj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgYm9vbCBjb25maWdfZW5hYmxlZDsK PiA+ID4gPiA+ID4gPiA+ICAgICAgICAgICBib29sIGNvbmZpZ19jaGFuZ2VfcGVuZGluZzsKPiA+ ID4gPiA+ID4gPiA+ICsgYm9vbCBpcnFfc29mdF9jaGVjazsKPiA+ID4gPiA+ID4gPiA+ICsgYm9v bCBpcnFfc29mdF9lbmFibGVkOwo+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgIHNwaW5sb2NrX3Qg Y29uZmlnX2xvY2s7Cj4gPiA+ID4gPiA+ID4gPiAgICAgICAgICAgc3BpbmxvY2tfdCB2cXNfbGlz dF9sb2NrOyAvKiBQcm90ZWN0cyBWUXMgbGlzdCBhY2Nlc3MgKi8KPiA+ID4gPiA+ID4gPiA+ICAg ICAgICAgICBzdHJ1Y3QgZGV2aWNlIGRldjsKPiA+ID4gPiA+ID4gPiA+IGRpZmYgLS1naXQgYS9p bmNsdWRlL2xpbnV4L3ZpcnRpb19jb25maWcuaCBiL2luY2x1ZGUvbGludXgvdmlydGlvX2NvbmZp Zy5oCj4gPiA+ID4gPiA+ID4gPiBpbmRleCBkYWZkYzdmNDhjMDEuLjljMWI2MWYyZTUyNSAxMDA2 NDQKPiA+ID4gPiA+ID4gPiA+IC0tLSBhL2luY2x1ZGUvbGludXgvdmlydGlvX2NvbmZpZy5oCj4g PiA+ID4gPiA+ID4gPiArKysgYi9pbmNsdWRlL2xpbnV4L3ZpcnRpb19jb25maWcuaAo+ID4gPiA+ ID4gPiA+ID4gQEAgLTE3NCw2ICsxNzQsMjQgQEAgc3RhdGljIGlubGluZSBib29sIHZpcnRpb19o YXNfZmVhdHVyZShjb25zdCBzdHJ1Y3QgdmlydGlvX2RldmljZSAqdmRldiwKPiA+ID4gPiA+ID4g PiA+ICAgICAgICAgICByZXR1cm4gX192aXJ0aW9fdGVzdF9iaXQodmRldiwgZmJpdCk7Cj4gPiA+ ID4gPiA+ID4gPiAgIH0KPiA+ID4gPiA+ID4gPiA+ICsvKgo+ID4gPiA+ID4gPiA+ID4gKyAqIHZp cnRpb19pcnFfc29mdF9lbmFibGVkOiB3aGV0aGVyIHdlIGNhbiBleGVjdXRlIGNhbGxiYWNrcwo+ ID4gPiA+ID4gPiA+ID4gKyAqIEB2ZGV2OiB0aGUgZGV2aWNlCj4gPiA+ID4gPiA+ID4gPiArICov Cj4gPiA+ID4gPiA+ID4gPiArc3RhdGljIGlubGluZSBib29sIHZpcnRpb19pcnFfc29mdF9lbmFi bGVkKGNvbnN0IHN0cnVjdCB2aXJ0aW9fZGV2aWNlICp2ZGV2KQo+ID4gPiA+ID4gPiA+ID4gK3sK PiA+ID4gPiA+ID4gPiA+ICsgaWYgKCF2ZGV2LT5pcnFfc29mdF9jaGVjaykKPiA+ID4gPiA+ID4g PiA+ICsgICAgICAgICByZXR1cm4gdHJ1ZTsKPiA+ID4gPiA+ID4gPiA+ICsKPiA+ID4gPiA+ID4g PiA+ICsgLyoKPiA+ID4gPiA+ID4gPiA+ICsgICogUmVhZCBpcnFfc29mdF9lbmFibGVkIGJlZm9y ZSByZWFkaW5nIG90aGVyIGRldmljZSBzcGVjaWZpYwo+ID4gPiA+ID4gPiA+ID4gKyAgKiBkYXRh LiBQYXJpZWQgd2l0aCBzbXBfc3RvcmVfcmVsYXNlKCkgaW4KPiA+ID4gPiA+ID4gPiBwYWlyZWQK PiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gV2lsbCBmaXguCj4gPiA+ID4gPiA+ Cj4gPiA+ID4gPiA+IFRoYW5rcwo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ Cj4gPiA+ID4gPiA+ID4gPiArICAqIHZpcnRpb19kZXZpY2VfcmVhZHkoKSBhbmQgV1JJVEVfT05D RSgpL3N5bmNocm9uaXplX3JjdSgpIGluCj4gPiA+ID4gPiA+ID4gPiArICAqIHZpcnRpb19yZXNl dF9kZXZpY2UoKS4KPiA+ID4gPiA+ID4gPiA+ICsgICovCj4gPiA+ID4gPiA+ID4gPiArIHJldHVy biBzbXBfbG9hZF9hY3F1aXJlKCZ2ZGV2LT5pcnFfc29mdF9lbmFibGVkKTsKPiA+ID4gPiA+ID4g PiA+ICt9Cj4gPiA+ID4gPiA+ID4gPiArCj4gPiA+ID4gPiA+ID4gPiAgIC8qKgo+ID4gPiA+ID4g PiA+ID4gICAgKiB2aXJ0aW9faGFzX2RtYV9xdWlyayAtIGRldGVybWluZSB3aGV0aGVyIHRoaXMg ZGV2aWNlIGhhcyB0aGUgRE1BIHF1aXJrCj4gPiA+ID4gPiA+ID4gPiAgICAqIEB2ZGV2OiB0aGUg ZGV2aWNlCj4gPiA+ID4gPiA+ID4gPiBAQCAtMjM2LDYgKzI1NCwxMyBAQCB2b2lkIHZpcnRpb19k ZXZpY2VfcmVhZHkoc3RydWN0IHZpcnRpb19kZXZpY2UgKmRldikKPiA+ID4gPiA+ID4gPiA+ICAg ICAgICAgICBpZiAoZGV2LT5jb25maWctPmVuYWJsZV9jYnMpCj4gPiA+ID4gPiA+ID4gPiAgICAg ICAgICAgICAgICAgICAgIGRldi0+Y29uZmlnLT5lbmFibGVfY2JzKGRldik7Cj4gPiA+ID4gPiA+ ID4gPiArIC8qCj4gPiA+ID4gPiA+ID4gPiArICAqIENvbW1pdCB0aGUgZHJpdmVyIHNldHVwIGJl Zm9yZSBlbmFibGluZyB0aGUgdmlydHF1ZXVlCj4gPiA+ID4gPiA+ID4gPiArICAqIGNhbGxiYWNr cy4gUGFyaWVkIHdpdGggc21wX2xvYWRfYWN1cWlyZSgpIGluCj4gPiA+ID4gPiA+ID4gPiArICAq IHZpcnRpb19pcnFfc29mdF9lbmFibGVkKCkKPiA+ID4gPiA+ID4gPiA+ICsgICovCj4gPiA+ID4g PiA+ID4gPiArIHNtcF9zdG9yZV9yZWxlYXNlKCZkZXYtPmlycV9zb2Z0X2VuYWJsZWQsIHRydWUp Owo+ID4gPiA+ID4gPiA+ID4gKwo+ID4gPiA+ID4gPiA+ID4gICAgICAgICAgIEJVR19PTihzdGF0 dXMgJiBWSVJUSU9fQ09ORklHX1NfRFJJVkVSX09LKTsKPiA+ID4gPiA+ID4gPiA+ICAgICAgICAg ICBkZXYtPmNvbmZpZy0+c2V0X3N0YXR1cyhkZXYsIHN0YXR1cyB8IFZJUlRJT19DT05GSUdfU19E UklWRVJfT0spOwo+ID4gPiA+ID4gPiA+ID4gICB9Cj4gPiA+ID4gPiA+ID4gPiAtLQo+ID4gPiA+ ID4gPiA+ID4gMi4yNS4xCj4gPiA+ID4gPgo+ID4gPgo+CgpfX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fXwpWaXJ0dWFsaXphdGlvbiBtYWlsaW5nIGxpc3QKVmly dHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRhdGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51 eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGluZm8vdmlydHVhbGl6YXRpb24= From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05E0AC433F5 for ; Mon, 28 Mar 2022 04:57:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238089AbiC1E6m (ORCPT ); Mon, 28 Mar 2022 00:58:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234431AbiC1E6j (ORCPT ); Mon, 28 Mar 2022 00:58:39 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 5E029FD17 for ; Sun, 27 Mar 2022 21:56:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1648443416; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2gkNQpu4kANGFFL3VkEjiCuQFdxFNSWesUdsMKBws3Q=; b=c0ejT8/5yJNtNY8QLDuKiQkGTn5xp0pubPylpyyVn7FuymOOubWTGfA6ry71k6nzYoynzp mfENF0H5/VWswEq9v76+jlbWKLw7140z8STmHuwiO5IJs0BqpERVGwDWu6DtuKUVCyV71l 73VS85x8qy2fcgaC/O0zbblxQpM/SA4= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-583-Th-gYyUMMae0Cns7yLBAbw-1; Mon, 28 Mar 2022 00:56:55 -0400 X-MC-Unique: Th-gYyUMMae0Cns7yLBAbw-1 Received: by mail-lj1-f200.google.com with SMTP id o25-20020a2e7319000000b0024aa653c564so2589617ljc.23 for ; Sun, 27 Mar 2022 21:56:54 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=2gkNQpu4kANGFFL3VkEjiCuQFdxFNSWesUdsMKBws3Q=; b=UawUbsqvFy92fxTdOyDAg5x3eIZPwh9oCe1oQ1bIDuHPsSOiOh8lAcvQiFV3Ay9G0q 109u18cjl+IJ/ZMxibGtfTFqU1O6fap5jvOXNvk8nahJ7nBdSOvgID05ewhRX/9cEW6e 1pKTNGBoe94+E+kMuQm5/EpNZzWc0ASxDHvybFiIHf896CZORULU1zqhl7VbWwdwqh34 MrdzGo5+NQBdl17u3mKBhHXRWcw96ugaBb9Krkj3i0qoOjnYvbTGFEbL2KHzjvECfYom PtsNjESsVaZPj9vHC5gJJD5tFSfA3dIFriWpbgPU/XNSr50a7HtBtsEgBzirGQ+UHcpt 54iw== X-Gm-Message-State: AOAM531+vusloc05AarfyRS8CwPdl84CyZYuxhmBdqtuos0aq6sXeKbz gFpDSZZYlARXYX+t4WZb9WwJi510uipICASTzs43O7M8LdWYyOO88J4BR5zLnu8MdarLF+jYdvr SGBVnzjc2S/gqUZyYv8cdGDWoPL+XBCQLYn5O8hDM X-Received: by 2002:a05:6512:1395:b0:446:d382:79a5 with SMTP id p21-20020a056512139500b00446d38279a5mr17926718lfa.210.1648443412757; Sun, 27 Mar 2022 21:56:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxt/bzJ9zwsNh8eSBVNz+omWW+KwYlD+ICzTaTEWGL/Qf6hUvuQA+7bxaQNlLmtrAeX/T1GVfdn1/f74Wo+kio= X-Received: by 2002:a05:6512:1395:b0:446:d382:79a5 with SMTP id p21-20020a056512139500b00446d38279a5mr17926694lfa.210.1648443412391; Sun, 27 Mar 2022 21:56:52 -0700 (PDT) MIME-Version: 1.0 References: <20220325050947-mutt-send-email-mst@kernel.org> <20220325060659-mutt-send-email-mst@kernel.org> In-Reply-To: <20220325060659-mutt-send-email-mst@kernel.org> From: Jason Wang Date: Mon, 28 Mar 2022 12:56:41 +0800 Message-ID: Subject: Re: To: "Michael S. Tsirkin" Cc: virtualization , linux-kernel , Marc Zyngier , Thomas Gleixner , Peter Zijlstra , Stefano Garzarella , Keir Fraser , "Paul E. McKenney" Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Mar 25, 2022 at 6:10 PM Michael S. Tsirkin wrote: > > On Fri, Mar 25, 2022 at 05:20:19PM +0800, Jason Wang wrote: > > On Fri, Mar 25, 2022 at 5:10 PM Michael S. Tsirkin wro= te: > > > > > > On Fri, Mar 25, 2022 at 03:52:00PM +0800, Jason Wang wrote: > > > > On Fri, Mar 25, 2022 at 2:31 PM Michael S. Tsirkin = wrote: > > > > > > > > > > Bcc: > > > > > Subject: Re: [PATCH 3/3] virtio: harden vring IRQ > > > > > Message-ID: <20220325021422-mutt-send-email-mst@kernel.org> > > > > > Reply-To: > > > > > In-Reply-To: > > > > > > > > > > On Fri, Mar 25, 2022 at 11:04:08AM +0800, Jason Wang wrote: > > > > > > > > > > > > =E5=9C=A8 2022/3/24 =E4=B8=8B=E5=8D=887:03, Michael S. Tsirkin = =E5=86=99=E9=81=93: > > > > > > > On Thu, Mar 24, 2022 at 04:40:04PM +0800, Jason Wang wrote: > > > > > > > > This is a rework on the previous IRQ hardening that is done= for > > > > > > > > virtio-pci where several drawbacks were found and were reve= rted: > > > > > > > > > > > > > > > > 1) try to use IRQF_NO_AUTOEN which is not friendly to affin= ity managed IRQ > > > > > > > > that is used by some device such as virtio-blk > > > > > > > > 2) done only for PCI transport > > > > > > > > > > > > > > > > In this patch, we tries to borrow the idea from the INTX IR= Q hardening > > > > > > > > in the reverted commit 080cd7c3ac87 ("virtio-pci: harden IN= TX interrupts") > > > > > > > > by introducing a global irq_soft_enabled variable for each > > > > > > > > virtio_device. Then we can to toggle it during > > > > > > > > virtio_reset_device()/virtio_device_ready(). A synchornize_= rcu() is > > > > > > > > used in virtio_reset_device() to synchronize with the IRQ h= andlers. In > > > > > > > > the future, we may provide config_ops for the transport tha= t doesn't > > > > > > > > use IRQ. With this, vring_interrupt() can return check and = early if > > > > > > > > irq_soft_enabled is false. This lead to smp_load_acquire() = to be used > > > > > > > > but the cost should be acceptable. > > > > > > > Maybe it should be but is it? Can't we use synchronize_irq in= stead? > > > > > > > > > > > > > > > > > > Even if we allow the transport driver to synchornize through > > > > > > synchronize_irq() we still need a check in the vring_interrupt(= ). > > > > > > > > > > > > We do something like the following previously: > > > > > > > > > > > > if (!READ_ONCE(vp_dev->intx_soft_enabled)) > > > > > > return IRQ_NONE; > > > > > > > > > > > > But it looks like a bug since speculative read can be done befo= re the check > > > > > > where the interrupt handler can't see the uncommitted setup whi= ch is done by > > > > > > the driver. > > > > > > > > > > I don't think so - if you sync after setting the value then > > > > > you are guaranteed that any handler running afterwards > > > > > will see the new value. > > > > > > > > The problem is not disabled but the enable. > > > > > > So a misbehaving device can lose interrupts? That's not a problem at = all > > > imo. > > > > It's the interrupt raised before setting irq_soft_enabled to true: > > > > CPU 0 probe) driver specific setup (not commited) > > CPU 1 IRQ handler) read the uninitialized variable > > CPU 0 probe) set irq_soft_enabled to true > > CPU 1 IRQ handler) read irq_soft_enable as true > > CPU 1 IRQ handler) use the uninitialized variable > > > > Thanks > > Yea, it hurts if you do it. So do not do it then ;). > > irq_soft_enabled (I think driver_ok or status is a better name) I can change it to driver_ok. > should be initialized to false *before* irq is requested. > > And requesting irq commits all memory otherwise all drivers would be > broken, So I think we might talk different issues: 1) Whether request_irq() commits the previous setups, I think the answer is yes, since the spin_unlock of desc->lock (release) can guarantee this though there seems no documentation around request_irq() to say this. And I can see at least drivers/video/fbdev/omap2/omapfb/dss/dispc.c is using smp_wmb() before the request_irq(). And even if write is ordered we still need read to be ordered to be paired with that. > if it doesn't it just needs to be fixed, not worked around in > virtio. 2) virtio drivers might do a lot of setups between request_irq() and virtio_device_ready(): request_irq() driver specific setups virtio_device_ready() CPU 0 probe) request_irq() CPU 1 IRQ handler) read the uninitialized variable CPU 0 probe) driver specific setups CPU 0 probe) smp_store_release(intr_soft_enabled, true), commit the setups CPU 1 IRQ handler) read irq_soft_enable as true CPU 1 IRQ handler) use the uninitialized variable Thanks > > > > > > > > > We use smp_store_relase() > > > > to make sure the driver commits the setup before enabling the irq. = It > > > > means the read needs to be ordered as well in vring_interrupt(). > > > > > > > > > > > > > > Although I couldn't find anything about this in memory-barriers.t= xt > > > > > which surprises me. > > > > > > > > > > CC Paul to help make sure I'm right. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > To avoid breaking legacy device which can send IRQ before D= RIVER_OK, a > > > > > > > > module parameter is introduced to enable the hardening so f= unction > > > > > > > > hardening is disabled by default. > > > > > > > Which devices are these? How come they send an interrupt befo= re there > > > > > > > are any buffers in any queues? > > > > > > > > > > > > > > > > > > I copied this from the commit log for 22b7050a024d7 > > > > > > > > > > > > " > > > > > > > > > > > > This change will also benefit old hypervisors (before 2009) > > > > > > that send interrupts without checking DRIVER_OK: previously= , > > > > > > the callback could race with driver-specific initialization= . > > > > > > " > > > > > > > > > > > > If this is only for config interrupt, I can remove the above lo= g. > > > > > > > > > > > > > > > This is only for config interrupt. > > > > > > > > Ok. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Note that the hardening is only done for vring interrupt si= nce the > > > > > > > > config interrupt hardening is already done in commit 22b705= 0a024d7 > > > > > > > > ("virtio: defer config changed notifications"). But the met= hod that is > > > > > > > > used by config interrupt can't be reused by the vring inter= rupt > > > > > > > > handler because it uses spinlock to do the synchronization = which is > > > > > > > > expensive. > > > > > > > > > > > > > > > > Signed-off-by: Jason Wang > > > > > > > > > > > > > > > --- > > > > > > > > drivers/virtio/virtio.c | 19 +++++++++++++++++++ > > > > > > > > drivers/virtio/virtio_ring.c | 9 ++++++++- > > > > > > > > include/linux/virtio.h | 4 ++++ > > > > > > > > include/linux/virtio_config.h | 25 ++++++++++++++++++++++= +++ > > > > > > > > 4 files changed, 56 insertions(+), 1 deletion(-) > > > > > > > > > > > > > > > > diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virti= o.c > > > > > > > > index 8dde44ea044a..85e331efa9cc 100644 > > > > > > > > --- a/drivers/virtio/virtio.c > > > > > > > > +++ b/drivers/virtio/virtio.c > > > > > > > > @@ -7,6 +7,12 @@ > > > > > > > > #include > > > > > > > > #include > > > > > > > > +static bool irq_hardening =3D false; > > > > > > > > + > > > > > > > > +module_param(irq_hardening, bool, 0444); > > > > > > > > +MODULE_PARM_DESC(irq_hardening, > > > > > > > > + "Disalbe IRQ software processing when it is not = expected"); > > > > > > > > + > > > > > > > > /* Unique numbering for virtio devices. */ > > > > > > > > static DEFINE_IDA(virtio_index_ida); > > > > > > > > @@ -220,6 +226,15 @@ static int virtio_features_ok(struct v= irtio_device *dev) > > > > > > > > * */ > > > > > > > > void virtio_reset_device(struct virtio_device *dev) > > > > > > > > { > > > > > > > > + /* > > > > > > > > + * The below synchronize_rcu() guarantees that any > > > > > > > > + * interrupt for this line arriving after > > > > > > > > + * synchronize_rcu() has completed is guaranteed to see > > > > > > > > + * irq_soft_enabled =3D=3D false. > > > > > > > News to me I did not know synchronize_rcu has anything to do > > > > > > > with interrupts. Did not you intend to use synchronize_irq? > > > > > > > I am not even 100% sure synchronize_rcu is by design a memory= barrier > > > > > > > though it's most likely is ... > > > > > > > > > > > > > > > > > > According to the comment above tree RCU version of synchronize_= rcu(): > > > > > > > > > > > > """ > > > > > > > > > > > > * RCU read-side critical sections are delimited by rcu_read_lo= ck() > > > > > > * and rcu_read_unlock(), and may be nested. In addition, but = only in > > > > > > * v5.0 and later, regions of code across which interrupts, pre= emption, > > > > > > * or softirqs have been disabled also serve as RCU read-side c= ritical > > > > > > * sections. This includes hardware interrupt handlers, softir= q handlers, > > > > > > * and NMI handlers. > > > > > > """ > > > > > > > > > > > > So interrupt handlers are treated as read-side critical section= s. > > > > > > > > > > > > And it has the comment for explain the barrier: > > > > > > > > > > > > """ > > > > > > > > > > > > * Note that this guarantee implies further memory-ordering gua= rantees. > > > > > > * On systems with more than one CPU, when synchronize_rcu() re= turns, > > > > > > * each CPU is guaranteed to have executed a full memory barrie= r since > > > > > > * the end of its last RCU read-side critical section whose beg= inning > > > > > > * preceded the call to synchronize_rcu(). In addition, each C= PU having > > > > > > """ > > > > > > > > > > > > So on SMP it provides a full barrier. And for UP/tiny RCU we do= n't need the > > > > > > barrier, if the interrupt come after WRITE_ONCE() it will see t= he > > > > > > irq_soft_enabled as false. > > > > > > > > > > > > > > > > You are right. So then > > > > > 1. I do not think we need load_acquire - why is it needed? Just > > > > > READ_ONCE should do. > > > > > > > > See above. > > > > > > > > > 2. isn't synchronize_irq also doing the same thing? > > > > > > > > > > > > Yes, but it requires a config ops since the IRQ knowledge is transp= ort specific. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > + */ > > > > > > > > + WRITE_ONCE(dev->irq_soft_enabled, false); > > > > > > > > + synchronize_rcu(); > > > > > > > > + > > > > > > > > dev->config->reset(dev); > > > > > > > > } > > > > > > > > EXPORT_SYMBOL_GPL(virtio_reset_device); > > > > > > > Please add comment explaining where it will be enabled. > > > > > > > Also, we *really* don't need to synch if it was already disab= led, > > > > > > > let's not add useless overhead to the boot sequence. > > > > > > > > > > > > > > > > > > Ok. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > @@ -427,6 +442,10 @@ int register_virtio_device(struct virt= io_device *dev) > > > > > > > > spin_lock_init(&dev->config_lock); > > > > > > > > dev->config_enabled =3D false; > > > > > > > > dev->config_change_pending =3D false; > > > > > > > > + dev->irq_soft_check =3D irq_hardening; > > > > > > > > + > > > > > > > > + if (dev->irq_soft_check) > > > > > > > > + dev_info(&dev->dev, "IRQ hardening is enabled\n")= ; > > > > > > > > /* We always start by resetting the device, in ca= se a previous > > > > > > > > * driver messed it up. This also tests that cod= e path a little. */ > > > > > > > one of the points of hardening is it's also helpful for buggy > > > > > > > devices. this flag defeats the purpose. > > > > > > > > > > > > > > > > > > Do you mean: > > > > > > > > > > > > 1) we need something like config_enable? This seems not easy to= be > > > > > > implemented without obvious overhead, mainly the synchronize wi= th the > > > > > > interrupt handlers > > > > > > > > > > But synchronize is only on tear-down path. That is not critical f= or any > > > > > users at the moment, even less than probe. > > > > > > > > I meant if we have vq->irq_pending, we need to call vring_interrupt= () > > > > in the virtio_device_ready() and synchronize the IRQ handlers with > > > > spinlock or others. > > > > > > > > > > > > > > > 2) enable this by default, so I don't object, but this may have= some risk > > > > > > for old hypervisors > > > > > > > > > > > > > > > The risk if there's a driver adding buffers without setting DRIVE= R_OK. > > > > > > > > Probably not, we have devices that accept random inputs from outsid= e, > > > > net, console, input etc. I've done a round of audits of the Qemu > > > > codes. They look all fine since day0. > > > > > > > > > So with this approach, how about we rename the flag "driver_ok"? > > > > > And then add_buf can actually test it and BUG_ON if not there (a= t least > > > > > in the debug build). > > > > > > > > This looks like a hardening of the driver in the core instead of th= e > > > > device. I think it can be done but in a separate series. > > > > > > > > > > > > > > And going down from there, how about we cache status in the > > > > > device? Then we don't need to keep re-reading it every time, > > > > > speeding boot up a tiny bit. > > > > > > > > I don't fully understand here, actually spec requires status to be > > > > read back for validation in many cases. > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/= virtio_ring.c > > > > > > > > index 962f1477b1fa..0170f8c784d8 100644 > > > > > > > > --- a/drivers/virtio/virtio_ring.c > > > > > > > > +++ b/drivers/virtio/virtio_ring.c > > > > > > > > @@ -2144,10 +2144,17 @@ static inline bool more_used(const = struct vring_virtqueue *vq) > > > > > > > > return vq->packed_ring ? more_used_packed(vq) : m= ore_used_split(vq); > > > > > > > > } > > > > > > > > -irqreturn_t vring_interrupt(int irq, void *_vq) > > > > > > > > +irqreturn_t vring_interrupt(int irq, void *v) > > > > > > > > { > > > > > > > > + struct virtqueue *_vq =3D v; > > > > > > > > + struct virtio_device *vdev =3D _vq->vdev; > > > > > > > > struct vring_virtqueue *vq =3D to_vvq(_vq); > > > > > > > > + if (!virtio_irq_soft_enabled(vdev)) { > > > > > > > > + dev_warn_once(&vdev->dev, "virtio vring IRQ raise= d before DRIVER_OK"); > > > > > > > > + return IRQ_NONE; > > > > > > > > + } > > > > > > > > + > > > > > > > > if (!more_used(vq)) { > > > > > > > > pr_debug("virtqueue interrupt with no wor= k for %p\n", vq); > > > > > > > > return IRQ_NONE; > > > > > > > > diff --git a/include/linux/virtio.h b/include/linux/virtio.= h > > > > > > > > index 5464f398912a..957d6ad604ac 100644 > > > > > > > > --- a/include/linux/virtio.h > > > > > > > > +++ b/include/linux/virtio.h > > > > > > > > @@ -95,6 +95,8 @@ dma_addr_t virtqueue_get_used_addr(struct= virtqueue *vq); > > > > > > > > * @failed: saved value for VIRTIO_CONFIG_S_FAILED bit (f= or restore) > > > > > > > > * @config_enabled: configuration change reporting enable= d > > > > > > > > * @config_change_pending: configuration change reported = while disabled > > > > > > > > + * @irq_soft_check: whether or not to check @irq_soft_enab= led > > > > > > > > + * @irq_soft_enabled: callbacks enabled > > > > > > > > * @config_lock: protects configuration change reporting > > > > > > > > * @dev: underlying device. > > > > > > > > * @id: the device type identification (used to match it = with a driver). > > > > > > > > @@ -109,6 +111,8 @@ struct virtio_device { > > > > > > > > bool failed; > > > > > > > > bool config_enabled; > > > > > > > > bool config_change_pending; > > > > > > > > + bool irq_soft_check; > > > > > > > > + bool irq_soft_enabled; > > > > > > > > spinlock_t config_lock; > > > > > > > > spinlock_t vqs_list_lock; /* Protects VQs list ac= cess */ > > > > > > > > struct device dev; > > > > > > > > diff --git a/include/linux/virtio_config.h b/include/linux/= virtio_config.h > > > > > > > > index dafdc7f48c01..9c1b61f2e525 100644 > > > > > > > > --- a/include/linux/virtio_config.h > > > > > > > > +++ b/include/linux/virtio_config.h > > > > > > > > @@ -174,6 +174,24 @@ static inline bool virtio_has_feature(= const struct virtio_device *vdev, > > > > > > > > return __virtio_test_bit(vdev, fbit); > > > > > > > > } > > > > > > > > +/* > > > > > > > > + * virtio_irq_soft_enabled: whether we can execute callbac= ks > > > > > > > > + * @vdev: the device > > > > > > > > + */ > > > > > > > > +static inline bool virtio_irq_soft_enabled(const struct vi= rtio_device *vdev) > > > > > > > > +{ > > > > > > > > + if (!vdev->irq_soft_check) > > > > > > > > + return true; > > > > > > > > + > > > > > > > > + /* > > > > > > > > + * Read irq_soft_enabled before reading other device spec= ific > > > > > > > > + * data. Paried with smp_store_relase() in > > > > > > > paired > > > > > > > > > > > > > > > > > > Will fix. > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > + * virtio_device_ready() and WRITE_ONCE()/synchronize_rcu= () in > > > > > > > > + * virtio_reset_device(). > > > > > > > > + */ > > > > > > > > + return smp_load_acquire(&vdev->irq_soft_enabled); > > > > > > > > +} > > > > > > > > + > > > > > > > > /** > > > > > > > > * virtio_has_dma_quirk - determine whether this device h= as the DMA quirk > > > > > > > > * @vdev: the device > > > > > > > > @@ -236,6 +254,13 @@ void virtio_device_ready(struct virtio= _device *dev) > > > > > > > > if (dev->config->enable_cbs) > > > > > > > > dev->config->enable_cbs(dev); > > > > > > > > + /* > > > > > > > > + * Commit the driver setup before enabling the virtqueue > > > > > > > > + * callbacks. Paried with smp_load_acuqire() in > > > > > > > > + * virtio_irq_soft_enabled() > > > > > > > > + */ > > > > > > > > + smp_store_release(&dev->irq_soft_enabled, true); > > > > > > > > + > > > > > > > > BUG_ON(status & VIRTIO_CONFIG_S_DRIVER_OK); > > > > > > > > dev->config->set_status(dev, status | VIRTIO_CONF= IG_S_DRIVER_OK); > > > > > > > > } > > > > > > > > -- > > > > > > > > 2.25.1 > > > > > > > > >