From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 848C3C433B4 for ; Thu, 13 May 2021 08:15:18 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0C2AE61106 for ; Thu, 13 May 2021 08:15:17 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0C2AE61106 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=daynix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id C17CD83D76; Thu, 13 May 2021 08:15:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cK82a094tSg0; Thu, 13 May 2021 08:15:16 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTP id B97B583C27; Thu, 13 May 2021 08:15:15 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 98A6AC000D; Thu, 13 May 2021 08:15:15 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 257ABC0001 for ; Thu, 13 May 2021 08:15:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1949A83D3B for ; Thu, 13 May 2021 08:15:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id z3ZBTTxpdTpV for ; Thu, 13 May 2021 08:15:12 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from mail-oo1-xc34.google.com (mail-oo1-xc34.google.com [IPv6:2607:f8b0:4864:20::c34]) by smtp1.osuosl.org (Postfix) with ESMTPS id C8EDA83C27 for ; Thu, 13 May 2021 08:15:12 +0000 (UTC) Received: by mail-oo1-xc34.google.com with SMTP id s1-20020a4ac1010000b02901cfd9170ce2so5485133oop.12 for ; Thu, 13 May 2021 01:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aasFNrGPhd1j95YGs4IN8C+Ann70ZEOsBSeAoS3Dquc=; b=FgRd4S5mQI4uPqrArD7NNLtUWWEQExXgzxXE7I09ngYJdo9Avo10R6yRLDgZ5s2/6l synxTCie/7qazENQLROJCEiesS8U1zE1WZihOvrm9o0X8N2SR5uEexuBpT4MxTur5lVZ +aGycbkGr1hqtWAzSxSHixnaylr/972mzSJEClMt6VBxBPMeqBy6l0RVh1FisJImna88 dJat/3XnyWzQyKdV/8ojviJXIj0Si14zVisj2X4hGdqNhOiBcj+GrLkowWwnneKEH9ft SrZBHZPkVNRts2TsKYYIDdhLmS6ubKYfiM4uHfikhNEfaPJB18d48k0F9YziAX9CYDUw rhRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aasFNrGPhd1j95YGs4IN8C+Ann70ZEOsBSeAoS3Dquc=; b=lV028zh3FuxqcvoJPwVLdNnFtgLT75P4U1vesSDnrBxCagMkKcJHmOvXgsCL0335vz KzukntBA1bowdXWjyOoKa7qnluvmEkMVfdXJi7wNnsWCmSqK4ZrofqBp4RtEkSXZMmcn s848FO/OkhCRAFFQMmTnsp7VE63wOgaV2pIgon9BfpDh4KWL5uFXkt4lbFPslyXrby1e tVxrVxo5JyESs/932ZWMQ9BpP6/yqlugCe0l5zNy+UUnZWmL7DFFTDnOIzSUbnfY472c vm/zUF5jCCCgdB7FumosZuGk7oeJxbbKhncNp68lXfNdrgu1huJHBAmwaWppSOpJcEW0 F8BQ== X-Gm-Message-State: AOAM5313kxZmE/JwupuYeQkags6zLTnPm4DdsoMiLmk0WYMdolxEfgLf XNZfiFhsrwCNwFML36buAz2hsJJw3NIGjDF4UcAl0w== X-Google-Smtp-Source: ABdhPJyStCuG4H38CtGEsMNsEIxdfX6dofY62H2IsjGc96VsdMKUxff2uYB4KWMZPftWGnS3aRaxspRbGYC7HEhnpl4= X-Received: by 2002:a4a:98a4:: with SMTP id a33mr31069654ooj.21.1620893711726; Thu, 13 May 2021 01:15:11 -0700 (PDT) MIME-Version: 1.0 References: <20210511044253.469034-1-yuri.benditovich@daynix.com> <20210511044253.469034-5-yuri.benditovich@daynix.com> <89759261-3a72-df6c-7a81-b7a48abfad44@redhat.com> In-Reply-To: From: Yuri Benditovich Date: Thu, 13 May 2021 11:14:59 +0300 Message-ID: Subject: Re: [PATCH 4/4] tun: indicate support for USO feature To: Jason Wang Cc: Willem de Bruijn , mst , netdev , linux-kernel , virtualization , Yan Vugenfirer , Jakub Kicinski , davem X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" T24gVGh1LCBNYXkgMTMsIDIwMjEgYXQgMTA6MDUgQU0gSmFzb24gV2FuZyA8amFzb3dhbmdAcmVk aGF0LmNvbT4gd3JvdGU6Cj4KPiBPbiBUaHUsIE1heSAxMywgMjAyMSBhdCAxMjozNiBQTSBZdXJp IEJlbmRpdG92aWNoCj4gPHl1cmkuYmVuZGl0b3ZpY2hAZGF5bml4LmNvbT4gd3JvdGU6Cj4gPgo+ ID4gT24gVGh1LCBNYXkgMTMsIDIwMjEgYXQgNTowNyBBTSBKYXNvbiBXYW5nIDxqYXNvd2FuZ0By ZWRoYXQuY29tPiB3cm90ZToKPiA+ID4KPiA+ID4gT24gV2VkLCBNYXkgMTIsIDIwMjEgYXQgNjoz NyBQTSBZdXJpIEJlbmRpdG92aWNoCj4gPiA+IDx5dXJpLmJlbmRpdG92aWNoQGRheW5peC5jb20+ IHdyb3RlOgo+ID4gPiA+Cj4gPiA+ID4gT24gV2VkLCBNYXkgMTIsIDIwMjEgYXQgMTI6MTAgUE0g SmFzb24gV2FuZyA8amFzb3dhbmdAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4gPgo+ID4gPiA+ ID4gT24gV2VkLCBNYXkgMTIsIDIwMjEgYXQgNDozMiBQTSBZdXJpIEJlbmRpdG92aWNoCj4gPiA+ ID4gPiA8eXVyaS5iZW5kaXRvdmljaEBkYXluaXguY29tPiB3cm90ZToKPiA+ID4gPiA+ID4KPiA+ ID4gPiA+ID4gT24gV2VkLCBNYXkgMTIsIDIwMjEgYXQgMTA6NTAgQU0gSmFzb24gV2FuZyA8amFz b3dhbmdAcmVkaGF0LmNvbT4gd3JvdGU6Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBPbiBX ZWQsIE1heSAxMiwgMjAyMSBhdCAxOjI0IFBNIFl1cmkgQmVuZGl0b3ZpY2gKPiA+ID4gPiA+ID4g PiA8eXVyaS5iZW5kaXRvdmljaEBkYXluaXguY29tPiB3cm90ZToKPiA+ID4gPiA+ID4gPiA+Cj4g PiA+ID4gPiA+ID4gPiBPbiBXZWQsIE1heSAxMiwgMjAyMSBhdCA0OjMzIEFNIEphc29uIFdhbmcg PGphc293YW5nQHJlZGhhdC5jb20+IHdyb3RlOgo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4g PiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiDlnKggMjAyMS81LzExIOS4i+WNiDQ6MzMsIFl1cmkg QmVuZGl0b3ZpY2gg5YaZ6YGTOgo+ID4gPiA+ID4gPiA+ID4gPiA+IE9uIFR1ZSwgTWF5IDExLCAy MDIxIGF0IDk6NTAgQU0gSmFzb24gV2FuZyA8amFzb3dhbmdAcmVkaGF0LmNvbT4gd3JvdGU6Cj4g PiA+ID4gPiA+ID4gPiA+ID4+Cj4gPiA+ID4gPiA+ID4gPiA+ID4+IOWcqCAyMDIxLzUvMTEg5LiL 5Y2IMTI6NDIsIFl1cmkgQmVuZGl0b3ZpY2gg5YaZ6YGTOgo+ID4gPiA+ID4gPiA+ID4gPiA+Pj4g U2lnbmVkLW9mZi1ieTogWXVyaSBCZW5kaXRvdmljaCA8eXVyaS5iZW5kaXRvdmljaEBkYXluaXgu Y29tPgo+ID4gPiA+ID4gPiA+ID4gPiA+Pj4gLS0tCj4gPiA+ID4gPiA+ID4gPiA+ID4+PiAgICBk cml2ZXJzL25ldC90dW4uYyB8IDIgKy0KPiA+ID4gPiA+ID4gPiA+ID4gPj4+ICAgIDEgZmlsZSBj aGFuZ2VkLCAxIGluc2VydGlvbigrKSwgMSBkZWxldGlvbigtKQo+ID4gPiA+ID4gPiA+ID4gPiA+ Pj4KPiA+ID4gPiA+ID4gPiA+ID4gPj4+IGRpZmYgLS1naXQgYS9kcml2ZXJzL25ldC90dW4uYyBi L2RyaXZlcnMvbmV0L3R1bi5jCj4gPiA+ID4gPiA+ID4gPiA+ID4+PiBpbmRleCA4NGY4MzI4MDYz MTMuLmEzNTA1NGY5ZDk0MSAxMDA2NDQKPiA+ID4gPiA+ID4gPiA+ID4gPj4+IC0tLSBhL2RyaXZl cnMvbmV0L3R1bi5jCj4gPiA+ID4gPiA+ID4gPiA+ID4+PiArKysgYi9kcml2ZXJzL25ldC90dW4u Ywo+ID4gPiA+ID4gPiA+ID4gPiA+Pj4gQEAgLTI4MTIsNyArMjgxMiw3IEBAIHN0YXRpYyBpbnQg c2V0X29mZmxvYWQoc3RydWN0IHR1bl9zdHJ1Y3QgKnR1biwgdW5zaWduZWQgbG9uZyBhcmcpCj4g PiA+ID4gPiA+ID4gPiA+ID4+PiAgICAgICAgICAgICAgICAgICAgICAgIGFyZyAmPSB+KFRVTl9G X1RTTzR8VFVOX0ZfVFNPNik7Cj4gPiA+ID4gPiA+ID4gPiA+ID4+PiAgICAgICAgICAgICAgICB9 Cj4gPiA+ID4gPiA+ID4gPiA+ID4+Pgo+ID4gPiA+ID4gPiA+ID4gPiA+Pj4gLSAgICAgICAgICAg ICBhcmcgJj0gflRVTl9GX1VGTzsKPiA+ID4gPiA+ID4gPiA+ID4gPj4+ICsgICAgICAgICAgICAg YXJnICY9IH4oVFVOX0ZfVUZPfFRVTl9GX1VTTyk7Cj4gPiA+ID4gPiA+ID4gPiA+ID4+Cj4gPiA+ ID4gPiA+ID4gPiA+ID4+IEl0IGxvb2tzIHRvIG1lIGtlcm5lbCBkb2Vzbid0IHVzZSAiVVNPIiwg c28gVFVOX0ZfVURQX0dTT19MNCBpcyBhIGJldHRlcgo+ID4gPiA+ID4gPiA+ID4gPiA+PiBuYW1l IGZvciB0aGlzCj4gPiA+ID4gPiA+ID4gPiA+ID4gTm8gcHJvYmxlbSwgSSBjYW4gY2hhbmdlIGl0 IGluIHYyCj4gPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiAgIGFuZCBJIGd1 ZXNzIHdlIHNob3VsZCB0b2dnbGUgTkVUSUZfRl9VRFBfR1NPX2w0IGhlcmU/Cj4gPiA+ID4gPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gPiBObywgd2UgZG8gbm90LCBiZWNhdXNlIHRoaXMg aW5kaWNhdGVzIG9ubHkgdGhlIGZhY3QgdGhhdCB0aGUgZ3Vlc3QgY2FuCj4gPiA+ID4gPiA+ID4g PiA+ID4gc2VuZCBsYXJnZSBVRFAgcGFja2V0cyBhbmQgaGF2ZSB0aGVtIHNwbGl0dGVkIHRvIFVE UCBzZWdtZW50cy4KPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ ID4gPiA+ID4gQWN0dWFsbHkgdGhlIHJldmVyc2UuIFRoZSBzZXRfb2ZmbG9hZCgpIGNvbnRyb2xz IHRoZSB0dW50YXAgVFggcGF0aAo+ID4gPiA+ID4gPiA+ID4gPiAoZ3Vlc3QgUlggcGF0aCkuCj4g PiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gVGhlIHNldF9vZmZsb2FkcyBkb2VzIDIgdGhp bmdzOgo+ID4gPiA+ID4gPiA+ID4gMS4gQXQgdGhlIGluaXRpYWxpemF0aW9uIHRpbWUgcWVtdSBw cm9iZXMgc2V0X29mZmxvYWQoc29tZXRoaW5nKSB0bwo+ID4gPiA+ID4gPiA+ID4gY2hlY2sgd2hp Y2ggZmVhdHVyZXMgYXJlIHN1cHBvcnRlZCBieSBUQVAvVFVOLgo+ID4gPiA+ID4gPiA+Cj4gPiA+ ID4gPiA+ID4gTm90ZSB0aGF0IHRoZSBwcm9iaW5nIGlzIHVzZWQgZm9yIGd1ZXN0IFJYIGZlYXR1 cmVzIG5vdCBob3N0IFJYLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBJdCBsb29rcyBsaWtlIHRo ZSBoaWRkZW4gYXNzdW1wdGlvbiAodGlsbCBub3cpIGlzIHRoYXQgaWYgc29tZSBmZWF0dXJlCj4g PiA+ID4gPiA+IGlzIHByZXNlbnQgLSBpdCBleGlzdHMgc2ltdWx0YW5lb3VzbHkgZm9yIGhvc3Qg YW5kIGd1ZXN0Lgo+ID4gPiA+ID4gPiBTZWUgUUVNVSBnZXRfZmVhdHVyZXM6IGlmIHRoZSBUQVAv VFVOIGRvZXMgbm90IGhhdmUgVUZPIGJvdGggSE9TVCBhbmQKPiA+ID4gPiA+ID4gR1VFU1QgRkVB VFVSRVMgYXJlIGNsZWFyZWQuCj4gPiA+ID4gPgo+ID4gPiA+ID4gS2luZCBvZiwgYWN0dWFsbHkg dGhlIGFzc3VtcHRpb24gaXM6IGlmIGEgZ3Vlc3QgZmVhdHVyZQo+ID4gPiA+ID4gKFZJUlRJT19O RVRfRl9HVUVTVF9YWFgpIGlzIHN1cHBvcnQsIHRoZSBjb3JyZXNwb25kaW5nIGhvc3QgZmVhdHVy ZQo+ID4gPiA+ID4gKFZJUlRJT19ORVRfRl9IT1NUX1hYWCkgaXMgYWxzbyBzdXBwb3J0ZWQuCj4g PiA+ID4KPiA+ID4gPiBTbyBub3RoaW5nIHRlbGxzIHVzIHRoYXQgdGhlIFRVTlNFVE9GRkxPQUQg aXMgZ29pbmcgdG8gc2V0IEdVRVNUIG9mZmxvYWRzLgo+ID4gPiA+IEZyb20gaWZfdHVuLmgKPiA+ ID4gPiAjZGVmaW5lIFRVTl9GX0NTVU0gICAgICAweDAxICAgIC8qIFlvdSBjYW4gaGFuZCBtZSB1 bmNoZWNrc3VtbWVkIHBhY2tldHMuICovCj4gPiA+ID4gI2RlZmluZSBUVU5fRl9UU080ICAgICAg MHgwMiAgICAvKiBJIGNhbiBoYW5kbGUgVFNPIGZvciBJUHY0IHBhY2tldHMgKi8KPiA+ID4gPiAj ZGVmaW5lIFRVTl9GX1RTTzYgICAgICAweDA0ICAgIC8qIEkgY2FuIGhhbmRsZSBUU08gZm9yIElQ djYgcGFja2V0cyAqLwo+ID4gPiA+ICNkZWZpbmUgVFVOX0ZfVFNPX0VDTiAgIDB4MDggICAgLyog SSBjYW4gaGFuZGxlIFRTTyB3aXRoIEVDTiBiaXRzLiAqLwo+ID4gPiA+ICNkZWZpbmUgVFVOX0Zf VUZPICAgICAgIDB4MTAgICAgLyogSSBjYW4gaGFuZGxlIFVGTyBwYWNrZXRzICovCj4gPiA+Cj4g PiA+IFllcywgdGhhdCdzIHdoeSBJIHJlcGxpZWQgaW4gYW5vdGhlciB0aHJlYWQgdG8gc2F5IHRo YXQgdGhlcmUncyBubyB3YXkKPiA+ID4gdG8gcmVmdXNlIEdTTyBwYWNrZXRzIGZyb20gdXNlcnNw YWNlLCBldmVuIGlmIFRVTl9GX1hYWCBpcyBub3Qgc2V0IHZpYQo+ID4gPiB0dW5fc2V0X29mZmxv YWQoKS4KPiA+ID4KPiA+ID4gRS5nIHlvdSBjYW4gZGlzYWJsZSBzZW5kaW5nIEdTTyBwYWNrZXRz IHRvIGd1ZXN0cyBidXQgeW91IGNhbid0IHJlamVjdAo+ID4gPiBHU08gcGFja2V0cyBmcm9tIGd1 ZXN0L3VzZXJzcGFjZS4KPiA+Cj4gPiBXZSBhZ3JlZSBoZXJlLgo+ID4gU29ycnkgZm9yIGJlaW5n IHVuY2xlYXIuIEkgbWVhbnQgZm9sbG93aW5nOgo+ID4gQWNjb3JkaW5nIHRvIHRoZSBjb21tZW50 IHRoZSBUVU5fRl9DU1VNIGlzIGEgX2hvc3RfIGNhcGFiaWxpdHkuCj4gPiBBY2NvcmRpbmcgdG8g dGhlIGNvbW1lbnQgdGhlIFRVTl9GX1VGTyBpcyBhIF9ndWVzdF8gY2FwYWJpbGl0eS4KPiA+Cj4g PiBCdXQgc3VycHJpc2luZ2x5IHdoZW4gVFVOIHJlY2VpdmVzIFRVTl9GX1VGTyBpdCBkb2VzIG5v dCBwcm9wYWdhdGUgaXQKPiA+IGFueXdoZXJlLCB0aGVyZSBpcyBubyBjb3JyZXNwb25kaW5nIE5F VElGIGZsYWcuCj4KPiAoSXQgbG9va3MgbGlrZSBJIGRyb3AgdGhlIGNvbW11bml0eSBhbmQgb3Ro ZXIgY2NzIGFjY2lkZW50YWxseSwgYWRkaW5nCj4gdGhlbSBiYWNrIGFuZCBzb3JyeSkKSSB0aG91 Z2h0IHlvdSBkaWQgaXQgaW50ZW50aW9uYWxseSB0byBhdm9pZCB0aGUgZmxhbWUKPgo+IEFjdHVh bGx5LCB0aGVyZSBpcyBvbmUsIE5FVElGX0ZfR1NPX1VEUC4KPgo+IEtlcm5lbCB1c2VkIHRvIGhh dmUgTkVUSUZfRl9VRk8sIGJ1dCBpdCB3YXMgcmVtb3ZlZCBkdWUgdG8gYnVncyBhbmQKPiB0aGUg bGFjayBvZiByZWFsIGhhcmR3YXJlIHN1cHBvcnQuIFRoZW4gd2UgZm91bmQgaXQgYnJlYWtzIHVB QkksIHNvCj4gV2lsbGVtIHRyaWVzIHRvIG1ha2UgaXQgYXBwZWFyIGZvciB1c2Vyc3BhY2UgYWdh aW4sIGFuZCB0aGVuIGl0IHdhcwo+IHJlbmFtZWQgdG8gTkVUSUZfRl9HU09fVURQLgo+Cj4gQnV0 IEkgdGhpbmsgaXQncyBhIGJ1ZyB0aGF0IHdlIGRvbid0IHByb3BvcmF0ZSBUVU5fRl9VRk8gdG8g TkVUSUYKPiBmbGFnLCB0aGlzIGlzIGEgbXVzdCBmb3IgdGhlIGRyaXZlciB0aGF0IGRvZXNuJ3Qg c3VwcG9ydAo+IFZJUlRJT19ORVRfRl9HVUVTVF9VRk8uIEkganVzdCB0cnkgdG8gZGlzYWJsZSBh bGwgb2ZmbG9hZHMgYW5kCj4gbXJnX3J4YnVmLCB0aGVuIG5ldHBlcmYgVURQX1NUUkVBTSBmcm9t IGhvc3QgdG8gZ3Vlc3QgZ2l2ZXMgbWUgYmFkCj4gbGVuZ3RoIHBhY2tldCBpbiB0aGUgZ3Vlc3Qu Cj4KPiBXaWxsZW0sIEkgdGhpbmsgd2UgcHJvYmFibHkgbmVlZCB0byBmaXggdGhpcy4KPgo+Cj4g PiBTbyBpbiBmYWN0IFRVTl9GX1VGTyBpcyBwcm9jZXNzZWQgYnkgdGhlIFRVTi9UQVAgZXhhY3Rs eSBhcyBhIGhvc3QgY2FwYWJpbGl0eS4KPiA+Cj4gPiA+Cj4gPiA+ID4KPiA+ID4gPiBTbywgbGV0 J3Mgd3JpdGUKPiA+ID4gPgo+ID4gPiA+ICNkZWZpbmUgVFVOX0ZfVURQX0w0VFggICAgICAgMHgy MCAgICAvKiBZb3UgY2FuIHNlbmQgbWUgbGFyZ2UgVURQIHBhY2tldHMgKi8KPiA+ID4KPiA+ID4g U28gaWYgd2Ugc3RpY2sgdG8gdGhlIGFzc3VtcHRpb24gImlmIGEgZ3Vlc3QgZmVhdHVyZSBpcyBz dXBwb3J0ZWQsIHRoZQo+ID4gPiBjb3JyZXNwb25kaW5nIGhvc3QgZmVhdHVyZSBpcyBzdXBwb3J0 ZWQiLiBUaGVyZSdzIG5vIG5lZWQgZm9yIHRoaXMuCj4gPiA+IEFuZCBJIHRoaW5rIGl0J3MgdGhl IG1vc3QgY2xlYW4gd2F5Lgo+ID4KPiA+IE15IHBlcnNvbmFsIG9waW5pb24gaXMgdGhhdCBpdCBp cyBleHRyZW1lbHkgd3JvbmcgdG8gZXh0ZW5kIHN1Y2ggYW4KPiA+IHVub2J2aW91cyBhc3N1bXB0 aW9uIHRvIGVhY2ggbmV3IGZlYXR1cmUuCj4KPiBUaGlzIHJlc3VsdHMgaW4gaW5jb25zaXN0ZW5j eSB3aXRoIG90aGVyIEdTTy9DU1VNIGZsYWdzLiBBbmQgd2lsbAo+IGNvbXBsaWNhdGUgdGhlIHVB UEkgKHR3byBmbGFncywgb25lIGZvciBSWCBhbm90aGVyIGZvciBUWCkuCj4KPiBDb25zaWRlcmlu ZyB0aGUgY3VycmVudCBjb2RlIHdvcmtzIGZvciBtYW55IHllYXJzLCBpdCdzIG5vdCB3b3J0aAo+ IGJvdGhlcmluZyBJIHRoaW5rLgo+Cj4gPgo+ID4gPgo+ID4gPiA+ICNkZWZpbmUgVFVOX0ZfVURQ NF9MNFJYICAgICAweDQwICAgLyogSSBjYW4gY29hbGVzY2UgVURQdjQgc2VnbWVudHMgKi8KPiA+ ID4gPiAjZGVmaW5lIFRVTl9GX1VEUDZfTDRSWCAgICAgMHg4MCAgLyogSSBjYW4gY29hbGVzY2Ug VURQdjYgc2VnbWVudHMgKi8KPiA+ID4KPiA+ID4gQW55IHZhbHVlIHRvIGNvYWxlc2NlIFVEUCBz ZWdtZW50cyBoZXJlPyBJdCdzIGJldHRlciB0byBkbyBpdCBpbiB0aGUKPiA+ID4gVFggc291cmNl IChndWVzdCkuCj4gPgo+ID4gQ29hbGVzY2luZyBpcyBhIGNvbnNlbnQgb2YgdGhlIGd1ZXN0IHRv IHJlY2VpdmUgcGFja2V0cyBiaWdnZXIgdGhhbiBNVFUuCj4gPiBPdGhlcndpc2UgKGlmIHRoZSBn dWVzdCBkb2VzIG5vdCBhZ3JlZSkgdGhlIGhvc3QgbXVzdCBzZWdtZW50L2ZyYWdtZW50Cj4gPiBw YWNrZXRzIGJlZm9yZSB0cmFuc21pdHRpbmcgdGhlbSB0byB0aGUgZ3Vlc3QuCj4KPiBUaGlzIGxv b2tzIGxpa2UgYSBkaWZmZXJlbnQgZmVhdHVyZSB3aGljaCBpcyBub3QgbmVjZXNzYXJpbHkga25v d24gYnkgZ3Vlc3RzPwo+Cj4gS2VybmVsIHN1cHBvcnRzIEdSTyB3aGljaCBjYW4gY29hbGVzY2Ug cGFja2V0cy4gKEl0IHdhcyBub3Qgc3VwcG9ydGVkCj4gYnkgVEFQIHlldCB0aG91Z2gpLgpJZiBJ IHVuZGVyc3RhbmQgdGhpbmdzIGNvcnJlY3RseSB0aGlzIGlzIGV4YWN0bHkgdGhpcyBmZWF0dXJl OgpUaGUgZ3Vlc3QgdHJhbnNtaXRzIGEgbGFyZ2UgVURQIHBhY2tldCB3aXRoIHRoZSBHU08gdmFs dWUgdGhhdCBtZWFucwp0aGF0IHRoZSBob3N0IHNob3VsZCBzZWdtZW50IGl0IF9pZiBuZWVkZWRf LgpJZiB0aGUgZGVzdGluYXRpb24gKGZvciBleGFtcGxlIGFub3RoZXIgZ3Vlc3QpIGlzIG5vdCBh YmxlIHRvIHJlY2VpdmUKdGhlIG9yaWdpbmFsIGxhcmdlIHBhY2tldCBpdCBpcyBzZWdtZW50ZWQg YW5kIHRoZSBzZWdtZW50cyBwdXNoZWQgdG8KdGhhdCBndWVzdC4KSWYgdGhlIGRlc3RpbmF0aW9u IGNhbiByZWNlaXZlIHRoZSBvcmlnaW5hbCBsYXJnZSBwYWNrZXQgKGN1cnJlbnRseQpub3QpIGl0 IGlzIGp1c3QgcHVzaGVkIHRvIGl0IGFzIGlmIGl0IHdhcyBzZWdtZW50ZWQgYW5kIHRoZW4KY29h bGVzY2VkLgpBcyBhbiBleGFtcGxlIG9mIHRoZSBzYW1lIHdpdGggVENQOgpXaXRoIHRoZSBjdXJy ZW50IGtlcm5lbCBXaW5kb3dzIGd1ZXN0IHJlY2VpdmVzIGNvYWxlc2NlZCBwYWNrZXRzIChmb3IK ZXhhbXBsZSB3aGVuIHNlZ21lbnRlZCBwYWNrZXRzIGNvbWUgdmlhIHBoeXNpY2FsIGFkYXB0ZXIg d2l0aApjb2FsZXNjaW5nIGNhcGFiaWxpdHkpIGR1ZSB0byB0aGUgZmFjdCB0aGF0IGl0IGR5bmFt aWNhbGx5IGVuYWJsZXMKVklSVElPX05FVF9GX0dVRVNUX1RTTyB2aWEgVklSVElPX05FVF9DVFJM X0dVRVNUX09GRkxPQURTIHdoaWNoCmZpbmFsbHkgc2V0cyBORVRJRl9GX1RTTy4KPgo+Cj4gPiBJ dCBpcyBub3QgcmVsYXRlZCB0byBndWVzdCBUWC4KPiA+Cj4gPiBGb3IgZXhhbXBsZSwgV2luZG93 cyBndWVzdCBpcyBub3QgYWJsZSB0byBoYW5kbGUgbGFyZ2UgVURQIHBhY2tldHMKPiA+ICh0aGlz IGlzIG5vdCBzdXBwb3J0ZWQgYnkgdGhlIHN0YWNrIHlldCkuCj4KPiBJbiB0aGlzIGNhc2UsIHRo ZSBjb3JyZXNwb25kaW5nIGd1ZXN0IG9yIGhvc3QgZmVhdHVyZXMgd2lsbCBiZQo+IGRpc2FibGVk LCBhbmQgdGhlIGtlcm5lbCB3b24ndCBzZW5kIHRob3NlIGtpbmRzIG9mIEdTTyBwYWNrZXRzIHRv Cj4gZ3Vlc3RzLgo+Cj4gPgo+ID4gPgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiAyLiBMYXRlciBpdCBjb25maWd1cmVzIHRoZSBndWVz dCBSWCBwYXRoIGFjY29yZGluZyB0byBndWVzdCdzIG5lZWRzL2NhcGFiaWxpdGllcwo+ID4gPiA+ ID4gPiA+ID4gVHlwaWNhbCBpbml0aWFsaXphdGlvbiBzZXF1ZW5jZSBpcyAoaW4gY2FzZSB0aGUg UUVNVSBzdXBwb3J0cyBVU08gZmVhdHVyZSk6Cj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBJ dCBhbHNvIGRlcGVuZHMgb24gd2hldGhlciB0aGUgYmFja2VuZChUQVApIGhhcyB0aGUgc3VwcG9y dCBmb3IgZ3Vlc3QgUlguCj4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+IEluIHRoZSBjb2RlIG9mIFRB UCBhbmQgVFVOIEkgZG8gbm90IHNlZSBhbnkgImlmIHRoZSBiYWNrZW5kIGhhcyB0aGUKPiA+ID4g PiA+ID4gc3VwcG9ydCBmb3IgZ3Vlc3QgUlgiLgo+ID4gPiA+ID4KPiA+ID4gPiA+IFllcywgdGhl IGRldGVjdGlvbiBpcyBpbXBsaWVkIGFzIHlvdSBkZXNjcmliZWQgYWJvdmUuCj4gPiA+ID4gPgo+ ID4gPiA+ID4gPiBUaGlzIGlzIGp1c3QgdGhlIElPQ1RMIGFuZCBzZXQgb2YgVFVOX0ZfKiBiaXRz LiBUaGVpciBtZWFuaW5nIGlzCj4gPiA+ID4gPiA+IGRlZmluZWQgaW4gdGhlIGNvbW1lbnRzLgo+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiBUQVAvVFVOIHNldCBvZmZs b2FkIDExIChwcm9iZSBmb3IgVUZPIHN1cHBvcnQpCj4gPiA+ID4gPiA+ID4gPiBUQVAvVFVOIHNl dCBvZmZsb2FkIDIxIChwcm9iZSBmb3IgVVNPIHN1cHBvcnQpCj4gPiA+ID4gPiA+ID4gPiBUQVAv VFVOIHNldCBvZmZsb2FkIDAKPiA+ID4gPiA+ID4gPiA+IC4uLgo+ID4gPiA+ID4gPiA+ID4gVEFQ L1RVTiBzZXQgb2ZmbG9hZCA3IChjb25maWd1cmF0aW9uIG9mIG9mZmxvYWRzIGFjY29yZGluZyB0 byBHVUVTVCBmZWF0dXJlcykKPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiBUaGlzIHNl cmllcyBvZiBwYXRjaGVzIGlzIGZvciBWSVJUSU9fTkVUX0ZfSE9TVF9VU08gb25seSwgdmlydGlv LW5ldAo+ID4gPiA+ID4gPiA+ID4gZmVhdHVyZXMgbGlrZSBWSVJUSU9fTkVUX0ZfR1VFU1RfVVNP Xyg0LzYvd2hhdGV2ZXIpIGFyZSBub3QgZGVmaW5lZCBpbgo+ID4gPiA+ID4gPiA+ID4gdGhlIHNw ZWMgeWV0Lgo+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+IEknbSBh IGxpdHRsZSBiaXQgY29uZnVzZWQgaGVyZS4gQ29uc2lkZXIgeW91IHdhbnQgdG8gaW1wbGVtZW50 IGd1ZXN0Cj4gPiA+ID4gPiA+ID4gVFggc28gdGhlcmUncyBubyBuZWVkIGZvciBhbnkgbW9kaWZp Y2F0aW9uIG9uIHRoZSBzZXRfb2ZmbG9hZCgpLgo+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBJIGRv IG5vdCB0aGluayBzby4gUGxlYXNlIGNvcnJlY3QgbWUgaWYgSSdtIG1pc3Rha2VuOgo+ID4gPiA+ ID4gPiBRRU1VIG5lZWRzIHRvIGluZGljYXRlIHRoZSBIT1NUX1VTTyBmZWF0dXJlIChvciBub3Qg aW5kaWNhdGUpLgo+ID4gPiA+ID4gPiBIb3cgY2FuIFFFTVUga25vdyB0aGUga2VybmVsIGlzIGFi bGUgdG8gc3VwcG9ydCBWSVJUSU9fTkVUX0hEUl9HU09fVURQX0w0Pwo+ID4gPiA+ID4KPiA+ID4g PiA+IE9rLCBJIGZpbmFsbHkgZ2V0IHlvdSBpZGVhLiBUaGFua3MgZm9yIHRoZSBwYXRpZW5jZS4K PiA+ID4gPiA+Cj4gPiA+ID4gPiBCdXQgc3RpbGwgb25lIGlzc3VlOiBBc3N1bWUgd2UgaW1wbGVt ZW50IFZJUlRJT19ORVRfRl9IT1NUX1VTTy4gSG93Cj4gPiA+ID4gPiBjb3VsZCB3ZSBhZGQgVklS VElPX05FVF9GX0dVRVNUX1VTTyBpbiB0aGUgZnV0dXJlPyBBZGRpbmcgYW5vdGhlciBUVU4KPiA+ ID4gPiA+IGZsYWcgZm9yIHNldF9vZmZsb2FkKCk/IFNlZW1zIHVubmVjZXNzYXJ5IGFuZCBpbmNv bnNpc3RlbmN5IHdpdGgKPiA+ID4gPiA+IGN1cnJlbnQgVFVOIGZsYWdzLgo+ID4gPiA+ID4KPiA+ ID4gPiA+ID4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+IEkgdGhpbmsgd2UgbmVlZCB0byBp bXBsZW1lbnQgYm90aCBkaXJlY3Rpb25zIGF0IG9uZSB0aW1lIGFzIHdoYXQgaGFzCj4gPiA+ID4g PiA+ID4gYmVlbiBwYXJ0aWFsbHkgZG9uZSBpbiB0aGlzIHNlcmllczoKPiA+ID4gPiA+ID4gPgo+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiBZb3UgYWN0dWFsbHkgc3VnZ2VzdCB0aGF0IHdlIG5lZWQg dG8gc3RhcnQgZnJvbSBMaW51eCB2aXJ0aW8tbmV0Cj4gPiA+ID4gPiA+IGRyaXZlciBhbmQgaW1w bGVtZW50IG9uIGl0IGJvdGggVFggYW5kIFJYLgo+ID4gPiA+ID4gPiBPdXIgbWFpbiBhcmVhIGlz IHZpcnRpby13aW4gZHJpdmVycyBhbmQgYWxsIHRoZSByZXN0IHdlIGRvIHdoZW4gd2UgY2FuLgo+ ID4gPiA+ID4gPiBDdXJyZW50bHkgd2UgaGF2ZSAyIFdJUCB0YXNrcyByZWxhdGVkIHRvIExpbnV4 ICh2aXJ0aW8tbmV0IFJTUyBhbmQKPiA+ID4gPiA+ID4gbGlidmlydCBSU1MvZUJQRikgYW5kICht eSBmZWVsaW5nKSB3ZSBoYXJkbHkgY2FuIHN0YXJ0IHdpdGggYWRkaXRpb25hbAo+ID4gPiA+ID4g PiBvbmUuCj4gPiA+ID4gPgo+ID4gPiA+ID4gSSBjYW4gaGVscCBmb3IgdGhlIGxpbnV4IGRyaXZl ciBpZiB5b3Ugd2lzaC4KPiA+ID4gPgo+ID4gPiA+IEkgdW5kZXJzdGFuZC4gUHJvYmFibHkgSSd2 ZSBtYWRlIGEgbWlzdGFrZSBmcm9tIHRoZSBiZWdpbm5pbmc6Cj4gPiA+ID4gQXQgZmlyc3Qgc3Rh Z2UgSSd2ZSBwcmVwYXJlZCB0aGUgc3BlYyBjaGFuZ2Ugb2Ygd2hhdCB3ZSBuZWVkIGluIGhvcGUK PiA+ID4gPiB0aGF0IHRoaXMgd2lsbCBiZSBmYXN0Lgo+ID4gPiA+IFByb2JhYmx5IHRoZSBiZXR0 ZXIgd2F5IHdhcyB0byBwcmVwYXJlIFJGQyBwYXRjaGVzIGZpcnN0IHRoZW4gc3RhcnQKPiA+ID4g PiBjaGFuZ2luZyB0aGUgc3BlYy4KPiA+ID4gPgo+ID4gPiA+IFNvIHRoZSBxdWVzdGlvbiBpcyB3 aGF0IHRvIGRvIG5vdzoKPiA+ID4gPiBBKQo+ID4gPiA+IEZpbmFsaXplIHBhdGNoZXMgZm9yIGd1 ZXN0IFRYIGFuZCByZXNwZWN0aXZlIFFFTVUgcGF0Y2hlcwo+ID4gPiA+IFByZXBhcmUgUkZDIHBh dGNoZXMgZm9yIGd1ZXN0IFJYLCBnZXQgYWNrIG9uIHRoZW0KPiA+ID4gPiBDaGFuZ2UgdGhlIHNw ZWMKPiA+ID4gPiBGaW5hbGl6ZSBwYXRjaGVzIGZvciBndWVzdCBSWCBhY2NvcmRpbmcgdG8gdGhl IHNwZWMKPiA+ID4gPgo+ID4gPiA+IEIpCj4gPiA+ID4gUmVqZWN0IHRoZSBwYXRjaGVzIGZvciBn dWVzdCBUWAo+ID4gPiA+IFByZXBhcmUgUkZDIHBhdGNoZXMgZm9yIGV2ZXJ5dGhpbmcsIGdldCBh Y2sgb24gdGhlbQo+ID4gPiA+IENoYW5nZSB0aGUgc3BlYwo+ID4gPiA+IEZpbmFsaXplIHBhdGNo ZXMgZm9yIGV2ZXJ5dGhpbmcgYWNjb3JkaW5nIHRvIHRoZSBzcGVjCj4gPiA+ID4KPiA+ID4gPiBJ J20gZm9yIEEpIG9mIGNvdXJzZSA6KQo+ID4gPgo+ID4gPiBJJ20gZm9yIEIgOikKPiA+ID4KPiA+ ID4gVGhlIHJlYXNvbnMgYXJlOgo+ID4gPgo+ID4gPiAxKSBrZWVwIHRoZSBhc3N1bXB0aW9uIG9m IHR1bl9zZXRfb2ZmbG9hZCgpIHRvIHNpbXBseSB0aGUgbG9naWMgYW5kCj4gPiA+IGNvbXBhdGli aWxpdHkKPiA+ID4gMikgaXQncyBoYXJkIG9yIHRyaWNreSB0byB0b3VjaCBndWVzdCBUWCBwYXRo IG9ubHkgKGUuZyB0aGUKPiA+ID4gdmlydGlvX25ldF9oZHJfZnJvbV9za2IoKSBpcyBjYWxsZWQg aW4gYm90aCBSWCBhbmQgVFgpCj4gPgo+ID4gSSBzdXNwZWN0IHRoZXJlIGlzIF9zb21lXyBtaXN1 bmRlcnN0YW5kaW5nIGhlcmUuCj4gPiBJIGRpZCBub3QgdG91Y2ggdmlydGlvX25ldF9oZHJfZnJv bV9za2IgYXQgYWxsLgo+ID4KPgo+IFR5cG8sIGFjdHVhbGx5IEkgbWVhbnQgdmlydGlvX25ldF9o ZHJfdG9fc2tiKCkuCk9LLgoyKSB0dW5fZ2V0X3VzZXIoKSB3aGljaCBpcyBndWVzdCBUWCAtIHRo aXMgaXMgY292ZXJlZAozKSB0YXBfZ2V0X3VzZXIoKSB3aGljaCBpcyBndWVzdCBUWCAtIHRoaXMg aXMgY292ZXJlZAo0KSB7dH1wYWNrZXRfc2VuZCgpIHdoaWNoIGlzIHVzZXJzcGFjZSBUWCAtIHRo aXMgaXMgT0ssIHRoZSB1c2Vyc3BhY2UKZG9lcyBub3QgaGF2ZSB0aGlzIGZlYXR1cmUsIGl0IHdp bGwgbmV2ZXIgdXNlIFVTTwoKMSkgcmVjZWl2ZV9idWYoKSB3aGljaCBpcyBMaW51eCBndWVzdCBS WCAtIHRoaXMgaXMgaW50ZXJlc3RpbmcKRG8geW91IG1lYW4gdGhhdCB3aXRoIG15IHBhdGNoZXMg aWYgV2luZG93cyBWTSBzZW5kcyBhIHBhY2tldCB3aXRoIFVTTwotIHRoZSBMaW51eCBWTSB3aWxs IG5vdCByZWNlaXZlIGl0IGNvcnJlY3RseSBzZWdtZW50ZWQ/CldoZW4gSSBzZW5kIHBhY2tldHMg d2l0aCBVU08gdmlhIFRVTiBJIHJlY2VpdmUgdGhlbSBzZWdtZW50ZWQgb24KYW5vdGhlciBUVU4g KDIgV2luZG93cyBhZGFwdGVycykuCgoKPgo+IFRoYW5rcwo+Cj4gPiA+Cj4gPiA+IFRoYW5rcwo+ ID4gPgo+ID4gPiA+Cj4gPiA+ID4gPgo+ID4gPiA+ID4gVGhhbmtzCj4gPiA+ID4gPgo+ID4gPiA+ ID4gPiBUaGlzIGlzIGEgcmVhc29uIHdoeSBJJ3ZlIGFkZGVkIHRvIHRoZSB2aXJ0aW8gc3BlYyBv bmx5IEhPU1RfVVNPIGFuZAo+ID4gPiA+ID4gPiBub3QgR1VFU1RfVVNPNC82Lgo+ID4gPiA+ID4g PiBVRFAgUlNDICh3aGljaCBpcyBhY3R1YWxseSBndWVzdCByeCBVU08pIGlzIG5vdCBhdmFpbGFi bGUgb24gV2luZG93cwo+ID4gPiA+ID4gPiBhdCB0aGUgbW9tZW50Lgo+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+IDEpIHNldF9vZmZsb2FkKCkgaXMgZm9yIGd1ZXN0IFJYLgo+ID4gPiA+ID4gPiA+ IDIpIHZpcnRpb19uZXRfaGRyX3RvX3NrYigpIGlzIGZvciBib3RoIGd1ZXN0IFRYIGFuZCBndWVz dCBSWC4KPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+IEZvciB0ZXN0aW5nLCB5b3UgY2FuIHJ1 biBWTTJWTSBvbiB0aGUgc2FtZSBob3N0LCBhbmQgeW91IHdpbGwgZ2V0Cj4gPiA+ID4gPiA+ID4g ZXZlcnl0aGluZyB0ZXN0ZWQuCj4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiBUaGFua3MKPiA+ ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPiBXaGVuIFZJUlRJ T19ORVRfRl9HVUVTVF9YWFggd2FzIG5vdCBuZWdvdGlhdGVkLCB0aGUgY29ycmVzcG9uZGluZyBu ZXRkZXYKPiA+ID4gPiA+ID4gPiA+ID4gZmVhdHVyZXMgbmVlZHMgdG8gYmUgZGlzYWJsZWQuIFdo ZW4gaG9zdCB0cmllcyB0byBzZW5kIHRob3NlIHBhY2tldHMgdG8KPiA+ID4gPiA+ID4gPiA+ID4g Z3Vlc3QsIGl0IG5lZWRzIHRvIGRvIHNvZnR3YXJlIHNlZ21lbnRhdGlvbi4KPiA+ID4gPiA+ID4g PiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gU2VlIHZpcnRpb19uZXRfYXBwbHlfZ3Vlc3Rfb2ZmbG9h ZHMoKS4KPiA+ID4gPiA+ID4gPiA+ID4KPiA+ID4gPiA+ID4gPiA+ID4gVGhlcmUncyBjdXJyZW50 bHkgbm8gd2F5IChvciBub3QgbmVlZCkgdG8gcHJldmVudCB0dW50YXAgZnJvbSByZWNlaXZpbmcK PiA+ID4gPiA+ID4gPiA+ID4gR1NPIHBhY2tldHMuCj4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4g PiA+ID4gPiA+IFRoYW5rcwo+ID4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+ID4gPgo+ID4g PiA+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+ID4gPiA+ID4+IEFuZCBob3cgYWJvdXQgbWFjdnRh cD8KPiA+ID4gPiA+ID4gPiA+ID4gPiBXZSB3aWxsIGNoZWNrIGhvdyB0byBkbyB0aGF0IGZvciBt YWN2dGFwLiBXZSB3aWxsIHNlbmQgYSBzZXBhcmF0ZQo+ID4gPiA+ID4gPiA+ID4gPiA+IHBhdGNo IGZvciBtYWN2dGFwIG9yIGFzayBmb3IgYWR2aWNlLgo+ID4gPiA+ID4gPiA+ID4gPiA+Cj4gPiA+ ID4gPiA+ID4gPiA+ID4+IFRoYW5rcwo+ID4gPiA+ID4gPiA+ID4gPiA+Pgo+ID4gPiA+ID4gPiA+ ID4gPiA+Pgo+ID4gPiA+ID4gPiA+ID4gPiA+Pj4gICAgICAgIH0KPiA+ID4gPiA+ID4gPiA+ID4g Pj4+Cj4gPiA+ID4gPiA+ID4gPiA+ID4+PiAgICAgICAgLyogVGhpcyBnaXZlcyB0aGUgdXNlciBh IHdheSB0byB0ZXN0IGZvciBuZXcgZmVhdHVyZXMgaW4gZnV0dXJlIGJ5Cj4gPiA+ID4gPiA+ID4g PiA+Cj4gPiA+ID4gPiA+ID4gPgo+ID4gPiA+ID4gPiA+Cj4gPiA+ID4gPiA+Cj4gPiA+ID4gPgo+ ID4gPiA+Cj4gPiA+Cj4gPgo+Cl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fX19fX19fClZpcnR1YWxpemF0aW9uIG1haWxpbmcgbGlzdApWaXJ0dWFsaXphdGlvbkBsaXN0 cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcv bWFpbG1hbi9saXN0aW5mby92aXJ0dWFsaXphdGlvbg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E757C43460 for ; Thu, 13 May 2021 08:15:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0369D61106 for ; Thu, 13 May 2021 08:15:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231791AbhEMIQ1 (ORCPT ); Thu, 13 May 2021 04:16:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231655AbhEMIQX (ORCPT ); Thu, 13 May 2021 04:16:23 -0400 Received: from mail-oo1-xc33.google.com (mail-oo1-xc33.google.com [IPv6:2607:f8b0:4864:20::c33]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A94FC061574 for ; Thu, 13 May 2021 01:15:12 -0700 (PDT) Received: by mail-oo1-xc33.google.com with SMTP id v13-20020a4aa40d0000b02902052145a469so3863542ool.3 for ; Thu, 13 May 2021 01:15:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=daynix-com.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=aasFNrGPhd1j95YGs4IN8C+Ann70ZEOsBSeAoS3Dquc=; b=FgRd4S5mQI4uPqrArD7NNLtUWWEQExXgzxXE7I09ngYJdo9Avo10R6yRLDgZ5s2/6l synxTCie/7qazENQLROJCEiesS8U1zE1WZihOvrm9o0X8N2SR5uEexuBpT4MxTur5lVZ +aGycbkGr1hqtWAzSxSHixnaylr/972mzSJEClMt6VBxBPMeqBy6l0RVh1FisJImna88 dJat/3XnyWzQyKdV/8ojviJXIj0Si14zVisj2X4hGdqNhOiBcj+GrLkowWwnneKEH9ft SrZBHZPkVNRts2TsKYYIDdhLmS6ubKYfiM4uHfikhNEfaPJB18d48k0F9YziAX9CYDUw rhRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=aasFNrGPhd1j95YGs4IN8C+Ann70ZEOsBSeAoS3Dquc=; b=QHlNJq5v5Khv1lDVEhwhBlhsZQuQ/rN0HNgiCJZSn1+m9TmBAxeXzmSUJszSyUIcNs pKxBVD4TAOlJRjAzuZT4gui4ag2RKoHL8X1IFoecUcg/zQNSZCWlE0tP3GE1qlNb3YZH 1Qh2jlVhEbLVVpL6cWFikBnoL1oPPA2qCx65Hj4S7XPvxq9xa5hdobX34OH6dMh3RGxZ OymdS0LeuxXWsb63mZ5g0owv7VvDnLKb9BnYRFHd/NMMYbILlGh6mxA/CZTO5cbVgmRc t8DkwsqN/QfIZMM1CKzBpwKBwMKmAhOMqRXF9MdEDPd803pK5AcDV0g8DVe/qAiF+CK1 brKA== X-Gm-Message-State: AOAM5311Gs5zxUGX+R37zYqEZc1gNNsK4JWEKzjdyTs1tv3qcOaCBjFQ V3cLxrFsCd6yjjEIkk1thTaETJIWym4gc0g9o8vYLQ== X-Google-Smtp-Source: ABdhPJyStCuG4H38CtGEsMNsEIxdfX6dofY62H2IsjGc96VsdMKUxff2uYB4KWMZPftWGnS3aRaxspRbGYC7HEhnpl4= X-Received: by 2002:a4a:98a4:: with SMTP id a33mr31069654ooj.21.1620893711726; Thu, 13 May 2021 01:15:11 -0700 (PDT) MIME-Version: 1.0 References: <20210511044253.469034-1-yuri.benditovich@daynix.com> <20210511044253.469034-5-yuri.benditovich@daynix.com> <89759261-3a72-df6c-7a81-b7a48abfad44@redhat.com> In-Reply-To: From: Yuri Benditovich Date: Thu, 13 May 2021 11:14:59 +0300 Message-ID: Subject: Re: [PATCH 4/4] tun: indicate support for USO feature To: Jason Wang Cc: Yan Vugenfirer , davem , Jakub Kicinski , mst , netdev , linux-kernel , virtualization , Willem de Bruijn Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, May 13, 2021 at 10:05 AM Jason Wang wrote: > > On Thu, May 13, 2021 at 12:36 PM Yuri Benditovich > wrote: > > > > On Thu, May 13, 2021 at 5:07 AM Jason Wang wrote: > > > > > > On Wed, May 12, 2021 at 6:37 PM Yuri Benditovich > > > wrote: > > > > > > > > On Wed, May 12, 2021 at 12:10 PM Jason Wang w= rote: > > > > > > > > > > On Wed, May 12, 2021 at 4:32 PM Yuri Benditovich > > > > > wrote: > > > > > > > > > > > > On Wed, May 12, 2021 at 10:50 AM Jason Wang wrote: > > > > > > > > > > > > > > On Wed, May 12, 2021 at 1:24 PM Yuri Benditovich > > > > > > > wrote: > > > > > > > > > > > > > > > > On Wed, May 12, 2021 at 4:33 AM Jason Wang wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > =E5=9C=A8 2021/5/11 =E4=B8=8B=E5=8D=884:33, Yuri Benditov= ich =E5=86=99=E9=81=93: > > > > > > > > > > On Tue, May 11, 2021 at 9:50 AM Jason Wang wrote: > > > > > > > > > >> > > > > > > > > > >> =E5=9C=A8 2021/5/11 =E4=B8=8B=E5=8D=8812:42, Yuri Bend= itovich =E5=86=99=E9=81=93: > > > > > > > > > >>> Signed-off-by: Yuri Benditovich > > > > > > > > > >>> --- > > > > > > > > > >>> drivers/net/tun.c | 2 +- > > > > > > > > > >>> 1 file changed, 1 insertion(+), 1 deletion(-) > > > > > > > > > >>> > > > > > > > > > >>> diff --git a/drivers/net/tun.c b/drivers/net/tun.c > > > > > > > > > >>> index 84f832806313..a35054f9d941 100644 > > > > > > > > > >>> --- a/drivers/net/tun.c > > > > > > > > > >>> +++ b/drivers/net/tun.c > > > > > > > > > >>> @@ -2812,7 +2812,7 @@ static int set_offload(struct t= un_struct *tun, unsigned long arg) > > > > > > > > > >>> arg &=3D ~(TUN_F_TSO4|TUN_F_TS= O6); > > > > > > > > > >>> } > > > > > > > > > >>> > > > > > > > > > >>> - arg &=3D ~TUN_F_UFO; > > > > > > > > > >>> + arg &=3D ~(TUN_F_UFO|TUN_F_USO); > > > > > > > > > >> > > > > > > > > > >> It looks to me kernel doesn't use "USO", so TUN_F_UDP_= GSO_L4 is a better > > > > > > > > > >> name for this > > > > > > > > > > No problem, I can change it in v2 > > > > > > > > > > > > > > > > > > > > and I guess we should toggle NETIF_F_UDP_GSO_l4 here? > > > > > > > > > > > > > > > > > > > > No, we do not, because this indicates only the fact tha= t the guest can > > > > > > > > > > send large UDP packets and have them splitted to UDP se= gments. > > > > > > > > > > > > > > > > > > > > > > > > > > > Actually the reverse. The set_offload() controls the tunt= ap TX path > > > > > > > > > (guest RX path). > > > > > > > > > > > > > > > > The set_offloads does 2 things: > > > > > > > > 1. At the initialization time qemu probes set_offload(somet= hing) to > > > > > > > > check which features are supported by TAP/TUN. > > > > > > > > > > > > > > Note that the probing is used for guest RX features not host = RX. > > > > > > > > > > > > It looks like the hidden assumption (till now) is that if some = feature > > > > > > is present - it exists simultaneously for host and guest. > > > > > > See QEMU get_features: if the TAP/TUN does not have UFO both HO= ST and > > > > > > GUEST FEATURES are cleared. > > > > > > > > > > Kind of, actually the assumption is: if a guest feature > > > > > (VIRTIO_NET_F_GUEST_XXX) is support, the corresponding host featu= re > > > > > (VIRTIO_NET_F_HOST_XXX) is also supported. > > > > > > > > So nothing tells us that the TUNSETOFFLOAD is going to set GUEST of= floads. > > > > From if_tun.h > > > > #define TUN_F_CSUM 0x01 /* You can hand me unchecksummed pa= ckets. */ > > > > #define TUN_F_TSO4 0x02 /* I can handle TSO for IPv4 packet= s */ > > > > #define TUN_F_TSO6 0x04 /* I can handle TSO for IPv6 packet= s */ > > > > #define TUN_F_TSO_ECN 0x08 /* I can handle TSO with ECN bits. = */ > > > > #define TUN_F_UFO 0x10 /* I can handle UFO packets */ > > > > > > Yes, that's why I replied in another thread to say that there's no wa= y > > > to refuse GSO packets from userspace, even if TUN_F_XXX is not set vi= a > > > tun_set_offload(). > > > > > > E.g you can disable sending GSO packets to guests but you can't rejec= t > > > GSO packets from guest/userspace. > > > > We agree here. > > Sorry for being unclear. I meant following: > > According to the comment the TUN_F_CSUM is a _host_ capability. > > According to the comment the TUN_F_UFO is a _guest_ capability. > > > > But surprisingly when TUN receives TUN_F_UFO it does not propagate it > > anywhere, there is no corresponding NETIF flag. > > (It looks like I drop the community and other ccs accidentally, adding > them back and sorry) I thought you did it intentionally to avoid the flame > > Actually, there is one, NETIF_F_GSO_UDP. > > Kernel used to have NETIF_F_UFO, but it was removed due to bugs and > the lack of real hardware support. Then we found it breaks uABI, so > Willem tries to make it appear for userspace again, and then it was > renamed to NETIF_F_GSO_UDP. > > But I think it's a bug that we don't proporate TUN_F_UFO to NETIF > flag, this is a must for the driver that doesn't support > VIRTIO_NET_F_GUEST_UFO. I just try to disable all offloads and > mrg_rxbuf, then netperf UDP_STREAM from host to guest gives me bad > length packet in the guest. > > Willem, I think we probably need to fix this. > > > > So in fact TUN_F_UFO is processed by the TUN/TAP exactly as a host capa= bility. > > > > > > > > > > > > > So, let's write > > > > > > > > #define TUN_F_UDP_L4TX 0x20 /* You can send me large UDP p= ackets */ > > > > > > So if we stick to the assumption "if a guest feature is supported, th= e > > > corresponding host feature is supported". There's no need for this. > > > And I think it's the most clean way. > > > > My personal opinion is that it is extremely wrong to extend such an > > unobvious assumption to each new feature. > > This results in inconsistency with other GSO/CSUM flags. And will > complicate the uAPI (two flags, one for RX another for TX). > > Considering the current code works for many years, it's not worth > bothering I think. > > > > > > > > > > #define TUN_F_UDP4_L4RX 0x40 /* I can coalesce UDPv4 segments= */ > > > > #define TUN_F_UDP6_L4RX 0x80 /* I can coalesce UDPv6 segments = */ > > > > > > Any value to coalesce UDP segments here? It's better to do it in the > > > TX source (guest). > > > > Coalescing is a consent of the guest to receive packets bigger than MTU= . > > Otherwise (if the guest does not agree) the host must segment/fragment > > packets before transmitting them to the guest. > > This looks like a different feature which is not necessarily known by gue= sts? > > Kernel supports GRO which can coalesce packets. (It was not supported > by TAP yet though). If I understand things correctly this is exactly this feature: The guest transmits a large UDP packet with the GSO value that means that the host should segment it _if needed_. If the destination (for example another guest) is not able to receive the original large packet it is segmented and the segments pushed to that guest. If the destination can receive the original large packet (currently not) it is just pushed to it as if it was segmented and then coalesced. As an example of the same with TCP: With the current kernel Windows guest receives coalesced packets (for example when segmented packets come via physical adapter with coalescing capability) due to the fact that it dynamically enables VIRTIO_NET_F_GUEST_TSO via VIRTIO_NET_CTRL_GUEST_OFFLOADS which finally sets NETIF_F_TSO. > > > > It is not related to guest TX. > > > > For example, Windows guest is not able to handle large UDP packets > > (this is not supported by the stack yet). > > In this case, the corresponding guest or host features will be > disabled, and the kernel won't send those kinds of GSO packets to > guests. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. Later it configures the guest RX path according to guest= 's needs/capabilities > > > > > > > > Typical initialization sequence is (in case the QEMU suppor= ts USO feature): > > > > > > > > > > > > > > It also depends on whether the backend(TAP) has the support f= or guest RX. > > > > > > > > > > > > In the code of TAP and TUN I do not see any "if the backend has= the > > > > > > support for guest RX". > > > > > > > > > > Yes, the detection is implied as you described above. > > > > > > > > > > > This is just the IOCTL and set of TUN_F_* bits. Their meaning i= s > > > > > > defined in the comments. > > > > > > > > > > > > > > > > > > > > > TAP/TUN set offload 11 (probe for UFO support) > > > > > > > > TAP/TUN set offload 21 (probe for USO support) > > > > > > > > TAP/TUN set offload 0 > > > > > > > > ... > > > > > > > > TAP/TUN set offload 7 (configuration of offloads according = to GUEST features) > > > > > > > > > > > > > > > > This series of patches is for VIRTIO_NET_F_HOST_USO only, v= irtio-net > > > > > > > > features like VIRTIO_NET_F_GUEST_USO_(4/6/whatever) are not= defined in > > > > > > > > the spec yet. > > > > > > > > > > > > > > > > > > > > > > I'm a little bit confused here. Consider you want to implemen= t guest > > > > > > > TX so there's no need for any modification on the set_offload= (). > > > > > > > > > > > > I do not think so. Please correct me if I'm mistaken: > > > > > > QEMU needs to indicate the HOST_USO feature (or not indicate). > > > > > > How can QEMU know the kernel is able to support VIRTIO_NET_HDR_= GSO_UDP_L4? > > > > > > > > > > Ok, I finally get you idea. Thanks for the patience. > > > > > > > > > > But still one issue: Assume we implement VIRTIO_NET_F_HOST_USO. H= ow > > > > > could we add VIRTIO_NET_F_GUEST_USO in the future? Adding another= TUN > > > > > flag for set_offload()? Seems unnecessary and inconsistency with > > > > > current TUN flags. > > > > > > > > > > > > > > > > > > > > > > > > > I think we need to implement both directions at one time as w= hat has > > > > > > > been partially done in this series: > > > > > > > > > > > > > > > > > > > You actually suggest that we need to start from Linux virtio-ne= t > > > > > > driver and implement on it both TX and RX. > > > > > > Our main area is virtio-win drivers and all the rest we do when= we can. > > > > > > Currently we have 2 WIP tasks related to Linux (virtio-net RSS = and > > > > > > libvirt RSS/eBPF) and (my feeling) we hardly can start with add= itional > > > > > > one. > > > > > > > > > > I can help for the linux driver if you wish. > > > > > > > > I understand. Probably I've made a mistake from the beginning: > > > > At first stage I've prepared the spec change of what we need in hop= e > > > > that this will be fast. > > > > Probably the better way was to prepare RFC patches first then start > > > > changing the spec. > > > > > > > > So the question is what to do now: > > > > A) > > > > Finalize patches for guest TX and respective QEMU patches > > > > Prepare RFC patches for guest RX, get ack on them > > > > Change the spec > > > > Finalize patches for guest RX according to the spec > > > > > > > > B) > > > > Reject the patches for guest TX > > > > Prepare RFC patches for everything, get ack on them > > > > Change the spec > > > > Finalize patches for everything according to the spec > > > > > > > > I'm for A) of course :) > > > > > > I'm for B :) > > > > > > The reasons are: > > > > > > 1) keep the assumption of tun_set_offload() to simply the logic and > > > compatibility > > > 2) it's hard or tricky to touch guest TX path only (e.g the > > > virtio_net_hdr_from_skb() is called in both RX and TX) > > > > I suspect there is _some_ misunderstanding here. > > I did not touch virtio_net_hdr_from_skb at all. > > > > Typo, actually I meant virtio_net_hdr_to_skb(). OK. 2) tun_get_user() which is guest TX - this is covered 3) tap_get_user() which is guest TX - this is covered 4) {t}packet_send() which is userspace TX - this is OK, the userspace does not have this feature, it will never use USO 1) receive_buf() which is Linux guest RX - this is interesting Do you mean that with my patches if Windows VM sends a packet with USO - the Linux VM will not receive it correctly segmented? When I send packets with USO via TUN I receive them segmented on another TUN (2 Windows adapters). > > Thanks > > > > > > > Thanks > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > This is a reason why I've added to the virtio spec only HOST_US= O and > > > > > > not GUEST_USO4/6. > > > > > > UDP RSC (which is actually guest rx USO) is not available on Wi= ndows > > > > > > at the moment. > > > > > > > > > > > > > 1) set_offload() is for guest RX. > > > > > > > 2) virtio_net_hdr_to_skb() is for both guest TX and guest RX. > > > > > > > > > > > > > > For testing, you can run VM2VM on the same host, and you will= get > > > > > > > everything tested. > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > When VIRTIO_NET_F_GUEST_XXX was not negotiated, the corre= sponding netdev > > > > > > > > > features needs to be disabled. When host tries to send th= ose packets to > > > > > > > > > guest, it needs to do software segmentation. > > > > > > > > > > > > > > > > > > See virtio_net_apply_guest_offloads(). > > > > > > > > > > > > > > > > > > There's currently no way (or not need) to prevent tuntap = from receiving > > > > > > > > > GSO packets. > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >> And how about macvtap? > > > > > > > > > > We will check how to do that for macvtap. We will send = a separate > > > > > > > > > > patch for macvtap or ask for advice. > > > > > > > > > > > > > > > > > > > >> Thanks > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >>> } > > > > > > > > > >>> > > > > > > > > > >>> /* This gives the user a way to test for new f= eatures in future by > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >