From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC4D8C48BE6 for ; Wed, 16 Jun 2021 09:26:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9565661019 for ; Wed, 16 Jun 2021 09:26:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232256AbhFPJ24 (ORCPT ); Wed, 16 Jun 2021 05:28:56 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:55984 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232222AbhFPJ2z (ORCPT ); Wed, 16 Jun 2021 05:28:55 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623835609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RW3o2HEy7SyzLUmjGnLmHDNB/DJgysCOrmXEWJSfZLs=; b=fJLqiFcANiFcR63LQMTUBf5kZscpdGWIdxna07wP0M+gNVz77//a2MSaYuLnD65UURAAPe evRtGJGE7Z10p+B+Hsf2n1ONb12mUp4FeKo6XtQxAdcJLJb2HlT7YQfb9+qc5uiMSB3+Wu UDhk4VZwjP9GQtRp+rN1iYliZ1lhjtw= Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-360-OT1rwqHOOEibA1-oWy80pg-1; Wed, 16 Jun 2021 05:26:48 -0400 X-MC-Unique: OT1rwqHOOEibA1-oWy80pg-1 Received: by mail-pg1-f199.google.com with SMTP id k193-20020a633dca0000b029021ff326b222so1119295pga.9 for ; Wed, 16 Jun 2021 02:26:48 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=RW3o2HEy7SyzLUmjGnLmHDNB/DJgysCOrmXEWJSfZLs=; b=KEoR3j/TUbJyPYolsP/JENH3SKkgvVsE8W4Q4u+j4I5bNFNANR3gee3u1k0dEP7W8+ EOtTcg8C7p1qLJHUAsgmKUDIw2crtSxgwiW4+4YvlAGIVYBDmcSQh8Z/Pqa+1IYvFw3H GWMWRg8WPinnTTZQcXmYqQEjuHXwgXUU7PyQ/QO6gxnI4bh50LQN8N8/v7uNhmabFNqH G+kONzfkaPtOupbySbCsu0jItd7jBoIqE+hkC7SNU0iEXJKI6XV3wMxoeVrLjwKS0nxH KyWbe8+7c+TX5zLwVmltIv9/UZ9O1CpxTUTf7dY99pJ6Y1UNWs2B2IxqOhGWTDpAHBo8 Fd7A== X-Gm-Message-State: AOAM533MOtNSbhvZVym0gMTTP47pQekpLn64xxrKxEvGceTnnvOW01R2 h3Wsmswsj1Jm0Jly9Pwiop5EGNjeZnsvQ4GSKnQpfIBgs6hSwTlV//QBCFS7BL6i0FdePvZpKwx Rak79+WF4B1kA X-Received: by 2002:a17:902:9006:b029:107:394a:387 with SMTP id a6-20020a1709029006b0290107394a0387mr7997054plp.35.1623835605708; Wed, 16 Jun 2021 02:26:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxEJ42y2sE35k74Q+Nctex4+yH8HLQmbZfHrBUlaSUqNZemHWxeVundizfyeLItBIHst18R3w== X-Received: by 2002:a17:902:9006:b029:107:394a:387 with SMTP id a6-20020a1709029006b0290107394a0387mr7997030plp.35.1623835605279; Wed, 16 Jun 2021 02:26:45 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id bg16sm1701165pjb.16.2021.06.16.02.26.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Jun 2021 02:26:44 -0700 (PDT) Subject: Re: [PATCH net-next v5 12/15] virtio-net: support AF_XDP zc tx To: Xuan Zhuo , netdev@vger.kernel.org Cc: "David S. Miller" , Jakub Kicinski , "Michael S. Tsirkin" , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , Magnus Karlsson , Jonathan Lemon , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , virtualization@lists.linux-foundation.org, bpf@vger.kernel.org, "dust . li" References: <20210610082209.91487-1-xuanzhuo@linux.alibaba.com> <20210610082209.91487-13-xuanzhuo@linux.alibaba.com> From: Jason Wang Message-ID: <99116ba9-9c17-a519-471d-98ae96d049d9@redhat.com> Date: Wed, 16 Jun 2021 17:26:33 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210610082209.91487-13-xuanzhuo@linux.alibaba.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Precedence: bulk List-ID: X-Mailing-List: bpf@vger.kernel.org 在 2021/6/10 下午4:22, Xuan Zhuo 写道: > AF_XDP(xdp socket, xsk) is a high-performance packet receiving and > sending technology. > > This patch implements the binding and unbinding operations of xsk and > the virtio-net queue for xsk zero copy xmit. > > The xsk zero copy xmit depends on tx napi. Because the actual sending > of data is done in the process of tx napi. If tx napi does not > work, then the data of the xsk tx queue will not be sent. > So if tx napi is not true, an error will be reported when bind xsk. > > If xsk is active, it will prevent ethtool from modifying tx napi. > > When reclaiming ptr, a new type of ptr is added, which is distinguished > based on the last two digits of ptr: > 00: skb > 01: xdp frame > 10: xsk xmit ptr > > All sent xsk packets share the virtio-net header of xsk_hdr. If xsk > needs to support csum and other functions later, consider assigning xsk > hdr separately for each sent packet. > > Different from other physical network cards, you can reinitialize the > channel when you bind xsk. And vrtio does not support independent reset > channel, you can only reset the entire device. I think it is not > appropriate for us to directly reset the entire setting. So the > situation becomes a bit more complicated. We have to consider how > to deal with the buffer referenced in vq after xsk is unbind. > > I added the ring size struct virtnet_xsk_ctx when xsk been bind. Each xsk > buffer added to vq corresponds to a ctx. This ctx is used to record the > page where the xsk buffer is located, and add a page reference. When the > buffer is recycling, reduce the reference to page. When xsk has been > unbind, and all related xsk buffers have been recycled, release all ctx. > > Signed-off-by: Xuan Zhuo > Reviewed-by: Dust Li > --- > drivers/net/virtio/Makefile | 1 + > drivers/net/virtio/virtio_net.c | 20 +- > drivers/net/virtio/virtio_net.h | 37 +++- > drivers/net/virtio/xsk.c | 346 ++++++++++++++++++++++++++++++++ > drivers/net/virtio/xsk.h | 99 +++++++++ > 5 files changed, 497 insertions(+), 6 deletions(-) > create mode 100644 drivers/net/virtio/xsk.c > create mode 100644 drivers/net/virtio/xsk.h > > diff --git a/drivers/net/virtio/Makefile b/drivers/net/virtio/Makefile > index ccc80f40f33a..db79d2e7925f 100644 > --- a/drivers/net/virtio/Makefile > +++ b/drivers/net/virtio/Makefile > @@ -4,3 +4,4 @@ > # > > obj-$(CONFIG_VIRTIO_NET) += virtio_net.o > +obj-$(CONFIG_VIRTIO_NET) += xsk.o > diff --git a/drivers/net/virtio/virtio_net.c b/drivers/net/virtio/virtio_net.c > index 395ec1f18331..40d7751f1c5f 100644 > --- a/drivers/net/virtio/virtio_net.c > +++ b/drivers/net/virtio/virtio_net.c > @@ -1423,6 +1423,7 @@ static int virtnet_poll_tx(struct napi_struct *napi, int budget) > > txq = netdev_get_tx_queue(vi->dev, index); > __netif_tx_lock(txq, raw_smp_processor_id()); > + work_done += virtnet_poll_xsk(sq, budget); > free_old_xmit(sq, true); > __netif_tx_unlock(txq); > > @@ -2133,8 +2134,16 @@ static int virtnet_set_coalesce(struct net_device *dev, > if (napi_weight ^ vi->sq[0].napi.weight) { > if (dev->flags & IFF_UP) > return -EBUSY; > - for (i = 0; i < vi->max_queue_pairs; i++) > + > + for (i = 0; i < vi->max_queue_pairs; i++) { > + /* xsk xmit depend on the tx napi. So if xsk is active, > + * prevent modifications to tx napi. > + */ > + if (rtnl_dereference(vi->sq[i].xsk.pool)) > + continue; So this can result tx NAPI is used by some queues buy not the others. I think such inconsistency breaks the semantic of set_coalesce() which assumes the operation is done at the device not some specific queues. How about just fail here? > + > vi->sq[i].napi.weight = napi_weight; > + } > } > > return 0; > @@ -2407,6 +2416,8 @@ static int virtnet_xdp(struct net_device *dev, struct netdev_bpf *xdp) > switch (xdp->command) { > case XDP_SETUP_PROG: > return virtnet_xdp_set(dev, xdp->prog, xdp->extack); > + case XDP_SETUP_XSK_POOL: > + return virtnet_xsk_pool_setup(dev, xdp); > default: > return -EINVAL; > } > @@ -2466,6 +2477,7 @@ static const struct net_device_ops virtnet_netdev = { > .ndo_vlan_rx_kill_vid = virtnet_vlan_rx_kill_vid, > .ndo_bpf = virtnet_xdp, > .ndo_xdp_xmit = virtnet_xdp_xmit, > + .ndo_xsk_wakeup = virtnet_xsk_wakeup, > .ndo_features_check = passthru_features_check, > .ndo_get_phys_port_name = virtnet_get_phys_port_name, > .ndo_set_features = virtnet_set_features, > @@ -2569,10 +2581,12 @@ static void free_unused_bufs(struct virtnet_info *vi) > for (i = 0; i < vi->max_queue_pairs; i++) { > struct virtqueue *vq = vi->sq[i].vq; > while ((buf = virtqueue_detach_unused_buf(vq)) != NULL) { > - if (!is_xdp_frame(buf)) > + if (is_skb_ptr(buf)) > dev_kfree_skb(buf); > - else > + else if (is_xdp_frame(buf)) > xdp_return_frame(ptr_to_xdp(buf)); > + else > + virtnet_xsk_ctx_tx_put(ptr_to_xsk(buf)); > } > } > > diff --git a/drivers/net/virtio/virtio_net.h b/drivers/net/virtio/virtio_net.h > index 931cc81f92fb..e3da829887dc 100644 > --- a/drivers/net/virtio/virtio_net.h > +++ b/drivers/net/virtio/virtio_net.h > @@ -135,6 +135,16 @@ struct send_queue { > struct virtnet_sq_stats stats; > > struct napi_struct napi; > + > + struct { > + struct xsk_buff_pool __rcu *pool; > + > + /* xsk wait for tx inter or softirq */ > + bool need_wakeup; > + > + /* ctx used to record the page added to vq */ > + struct virtnet_xsk_ctx_head *ctx_head; > + } xsk; > }; > > /* Internal representation of a receive virtqueue */ > @@ -188,6 +198,13 @@ static inline void virtqueue_napi_schedule(struct napi_struct *napi, > } > } > > +#include "xsk.h" > + > +static inline bool is_skb_ptr(void *ptr) > +{ > + return !((unsigned long)ptr & (VIRTIO_XDP_FLAG | VIRTIO_XSK_FLAG)); > +} > + > static inline bool is_xdp_frame(void *ptr) > { > return (unsigned long)ptr & VIRTIO_XDP_FLAG; > @@ -206,25 +223,39 @@ static inline struct xdp_frame *ptr_to_xdp(void *ptr) > static inline void __free_old_xmit(struct send_queue *sq, bool in_napi, > struct virtnet_sq_stats *stats) > { > + unsigned int xsknum = 0; > unsigned int len; > void *ptr; > > while ((ptr = virtqueue_get_buf(sq->vq, &len)) != NULL) { > - if (!is_xdp_frame(ptr)) { > + if (is_skb_ptr(ptr)) { > struct sk_buff *skb = ptr; > > pr_debug("Sent skb %p\n", skb); > > stats->bytes += skb->len; > napi_consume_skb(skb, in_napi); > - } else { > + } else if (is_xdp_frame(ptr)) { > struct xdp_frame *frame = ptr_to_xdp(ptr); > > stats->bytes += frame->len; > xdp_return_frame(frame); > + } else { > + struct virtnet_xsk_ctx_tx *ctx; > + > + ctx = ptr_to_xsk(ptr); > + > + /* Maybe this ptr was sent by the last xsk. */ > + if (ctx->ctx.head->active) > + ++xsknum; > + > + stats->bytes += ctx->len; > + virtnet_xsk_ctx_tx_put(ctx); > } > stats->packets++; > } > -} > > + if (xsknum) > + virtnet_xsk_complete(sq, xsknum); > +} > #endif > diff --git a/drivers/net/virtio/xsk.c b/drivers/net/virtio/xsk.c > new file mode 100644 > index 000000000000..f98b68576709 > --- /dev/null > +++ b/drivers/net/virtio/xsk.c > @@ -0,0 +1,346 @@ > +// SPDX-License-Identifier: GPL-2.0-or-later > +/* > + * virtio-net xsk > + */ > + > +#include "virtio_net.h" > + > +static struct virtio_net_hdr_mrg_rxbuf xsk_hdr; > + > +static struct virtnet_xsk_ctx *virtnet_xsk_ctx_get(struct virtnet_xsk_ctx_head *head) > +{ > + struct virtnet_xsk_ctx *ctx; > + > + ctx = head->ctx; > + head->ctx = ctx->next; > + > + ++head->ref; > + > + return ctx; > +} > + > +#define virtnet_xsk_ctx_tx_get(head) ((struct virtnet_xsk_ctx_tx *)virtnet_xsk_ctx_get(head)) > + > +static void virtnet_xsk_check_queue(struct send_queue *sq) > +{ > + struct virtnet_info *vi = sq->vq->vdev->priv; > + struct net_device *dev = vi->dev; > + int qnum = sq - vi->sq; > + > + /* If it is a raw buffer queue, it does not check whether the status > + * of the queue is stopped when sending. So there is no need to check > + * the situation of the raw buffer queue. > + */ > + if (is_xdp_raw_buffer_queue(vi, qnum)) > + return; > + > + /* If this sq is not the exclusive queue of the current cpu, > + * then it may be called by start_xmit, so check it running out > + * of space. > + * > + * Stop the queue to avoid getting packets that we are > + * then unable to transmit. Then wait the tx interrupt. > + */ > + if (sq->vq->num_free < 2 + MAX_SKB_FRAGS) > + netif_stop_subqueue(dev, qnum); > +} > + > +void virtnet_xsk_complete(struct send_queue *sq, u32 num) > +{ > + struct xsk_buff_pool *pool; > + > + rcu_read_lock(); > + pool = rcu_dereference(sq->xsk.pool); > + if (!pool) { > + rcu_read_unlock(); > + return; > + } > + xsk_tx_completed(pool, num); > + rcu_read_unlock(); > + > + if (sq->xsk.need_wakeup) { > + sq->xsk.need_wakeup = false; > + virtqueue_napi_schedule(&sq->napi, sq->vq); > + } > +} > + > +static int virtnet_xsk_xmit(struct send_queue *sq, struct xsk_buff_pool *pool, > + struct xdp_desc *desc) > +{ > + struct virtnet_xsk_ctx_tx *ctx; > + struct virtnet_info *vi; > + u32 offset, n, len; > + struct page *page; > + void *data; > + > + vi = sq->vq->vdev->priv; > + > + data = xsk_buff_raw_get_data(pool, desc->addr); > + offset = offset_in_page(data); > + > + ctx = virtnet_xsk_ctx_tx_get(sq->xsk.ctx_head); > + > + /* xsk unaligned mode, desc may use two pages */ > + if (desc->len > PAGE_SIZE - offset) > + n = 3; > + else > + n = 2; > + > + sg_init_table(sq->sg, n); > + sg_set_buf(sq->sg, &xsk_hdr, vi->hdr_len); > + > + /* handle for xsk first page */ > + len = min_t(int, desc->len, PAGE_SIZE - offset); > + page = vmalloc_to_page(data); > + sg_set_page(sq->sg + 1, page, len, offset); > + > + /* ctx is used to record and reference this page to prevent xsk from > + * being released before this xmit is recycled > + */ I'm a little bit surprised that this is done manually per device instead of doing it in xsk core. > + ctx->ctx.page = page; > + get_page(page); > + > + /* xsk unaligned mode, handle for the second page */ > + if (len < desc->len) { > + page = vmalloc_to_page(data + len); > + len = min_t(int, desc->len - len, PAGE_SIZE); > + sg_set_page(sq->sg + 2, page, len, 0); > + > + ctx->ctx.page_unaligned = page; > + get_page(page); > + } else { > + ctx->ctx.page_unaligned = NULL; > + } > + > + return virtqueue_add_outbuf(sq->vq, sq->sg, n, > + xsk_to_ptr(ctx), GFP_ATOMIC); > +} > + > +static int virtnet_xsk_xmit_batch(struct send_queue *sq, > + struct xsk_buff_pool *pool, > + unsigned int budget, > + bool in_napi, int *done, > + struct virtnet_sq_stats *stats) > +{ > + struct xdp_desc desc; > + int err, packet = 0; > + int ret = -EAGAIN; > + > + while (budget-- > 0) { > + if (sq->vq->num_free < 2 + MAX_SKB_FRAGS) { AF_XDP doesn't use skb, so I don't see why MAX_SKB_FRAGS is used. Looking at virtnet_xsk_xmit(), it looks to me 3 is more suitable here. Or did AF_XDP core can handle queue full gracefully then we don't even need to worry about this? > + ret = -EBUSY; -ENOSPC looks better. > + break; > + } > + > + if (!xsk_tx_peek_desc(pool, &desc)) { > + /* done */ > + ret = 0; > + break; > + } > + > + err = virtnet_xsk_xmit(sq, pool, &desc); > + if (unlikely(err)) { If we always reserve sufficient slots, this should be an unexpected error, do we need log this as what has been done in start_xmit()?         /* This should not happen! */         if (unlikely(err)) {                 dev->stats.tx_fifo_errors++;                 if (net_ratelimit())                         dev_warn(&dev->dev,                                  "Unexpected TXQ (%d) queue failure: %d\n",                                  qnum, err); > + ret = -EBUSY; > + break; > + } > + > + ++packet; > + } > + > + if (packet) { > + if (virtqueue_kick_prepare(sq->vq) && virtqueue_notify(sq->vq)) > + ++stats->kicks; > + > + *done += packet; > + stats->xdp_tx += packet; > + > + xsk_tx_release(pool); > + } > + > + return ret; > +} > + > +static int virtnet_xsk_run(struct send_queue *sq, struct xsk_buff_pool *pool, > + int budget, bool in_napi) > +{ > + struct virtnet_sq_stats stats = {}; > + int done = 0; > + int err; > + > + sq->xsk.need_wakeup = false; > + __free_old_xmit(sq, in_napi, &stats); > + > + /* return err: > + * -EAGAIN: done == budget > + * -EBUSY: done < budget > + * 0 : done < budget > + */ It's better to move the comment to the implementation of virtnet_xsk_xmit_batch(). > +xmit: > + err = virtnet_xsk_xmit_batch(sq, pool, budget - done, in_napi, > + &done, &stats); > + if (err == -EBUSY) { > + __free_old_xmit(sq, in_napi, &stats); > + > + /* If the space is enough, let napi run again. */ > + if (sq->vq->num_free >= 2 + MAX_SKB_FRAGS) The comment does not match the code. > + goto xmit; > + else > + sq->xsk.need_wakeup = true; > + } > + > + virtnet_xsk_check_queue(sq); > + > + u64_stats_update_begin(&sq->stats.syncp); > + sq->stats.packets += stats.packets; > + sq->stats.bytes += stats.bytes; > + sq->stats.kicks += stats.kicks; > + sq->stats.xdp_tx += stats.xdp_tx; > + u64_stats_update_end(&sq->stats.syncp); > + > + return done; > +} > + > +int virtnet_poll_xsk(struct send_queue *sq, int budget) > +{ > + struct xsk_buff_pool *pool; > + int work_done = 0; > + > + rcu_read_lock(); > + pool = rcu_dereference(sq->xsk.pool); > + if (pool) > + work_done = virtnet_xsk_run(sq, pool, budget, true); > + rcu_read_unlock(); > + return work_done; > +} > + > +int virtnet_xsk_wakeup(struct net_device *dev, u32 qid, u32 flag) > +{ > + struct virtnet_info *vi = netdev_priv(dev); > + struct xsk_buff_pool *pool; > + struct send_queue *sq; > + > + if (!netif_running(dev)) > + return -ENETDOWN; > + > + if (qid >= vi->curr_queue_pairs) > + return -EINVAL; I wonder how we can hit this check. Note that we prevent the user from modifying queue pairs when XDP is enabled:         /* For now we don't support modifying channels while XDP is loaded          * also when XDP is loaded all RX queues have XDP programs so we only          * need to check a single RX queue.          */         if (vi->rq[0].xdp_prog)                 return -EINVAL; > + > + sq = &vi->sq[qid]; > + > + rcu_read_lock(); Can we simply use rcu_read_lock_bh() here? > + pool = rcu_dereference(sq->xsk.pool); > + if (pool) { > + local_bh_disable(); > + virtqueue_napi_schedule(&sq->napi, sq->vq); > + local_bh_enable(); > + } > + rcu_read_unlock(); > + return 0; > +} > + > +static struct virtnet_xsk_ctx_head *virtnet_xsk_ctx_alloc(struct xsk_buff_pool *pool, > + struct virtqueue *vq) > +{ > + struct virtnet_xsk_ctx_head *head; > + u32 size, n, ring_size, ctx_sz; > + struct virtnet_xsk_ctx *ctx; > + void *p; > + > + ctx_sz = sizeof(struct virtnet_xsk_ctx_tx); > + > + ring_size = virtqueue_get_vring_size(vq); > + size = sizeof(*head) + ctx_sz * ring_size; > + > + head = kmalloc(size, GFP_ATOMIC); > + if (!head) > + return NULL; > + > + memset(head, 0, sizeof(*head)); > + > + head->active = true; > + head->frame_size = xsk_pool_get_rx_frame_size(pool); > + > + p = head + 1; > + for (n = 0; n < ring_size; ++n) { > + ctx = p; > + ctx->head = head; > + ctx->next = head->ctx; > + head->ctx = ctx; > + > + p += ctx_sz; > + } > + > + return head; > +} > + > +static int virtnet_xsk_pool_enable(struct net_device *dev, > + struct xsk_buff_pool *pool, > + u16 qid) > +{ > + struct virtnet_info *vi = netdev_priv(dev); > + struct send_queue *sq; > + > + if (qid >= vi->curr_queue_pairs) > + return -EINVAL; > + > + sq = &vi->sq[qid]; > + > + /* xsk zerocopy depend on the tx napi. > + * > + * All data is actually consumed and sent out from the xsk tx queue > + * under the tx napi mechanism. > + */ > + if (!sq->napi.weight) > + return -EPERM; > + > + memset(&sq->xsk, 0, sizeof(sq->xsk)); > + > + sq->xsk.ctx_head = virtnet_xsk_ctx_alloc(pool, sq->vq); > + if (!sq->xsk.ctx_head) > + return -ENOMEM; > + > + /* Here is already protected by rtnl_lock, so rcu_assign_pointer is > + * safe. > + */ > + rcu_assign_pointer(sq->xsk.pool, pool); > + > + return 0; > +} > + > +static int virtnet_xsk_pool_disable(struct net_device *dev, u16 qid) > +{ > + struct virtnet_info *vi = netdev_priv(dev); > + struct send_queue *sq; > + > + if (qid >= vi->curr_queue_pairs) > + return -EINVAL; > + > + sq = &vi->sq[qid]; > + > + /* Here is already protected by rtnl_lock, so rcu_assign_pointer is > + * safe. > + */ > + rcu_assign_pointer(sq->xsk.pool, NULL); > + > + /* Sync with the XSK wakeup and with NAPI. */ > + synchronize_net(); > + > + if (READ_ONCE(sq->xsk.ctx_head->ref)) > + WRITE_ONCE(sq->xsk.ctx_head->active, false); > + else > + kfree(sq->xsk.ctx_head); > + > + sq->xsk.ctx_head = NULL; > + > + return 0; > +} > + > +int virtnet_xsk_pool_setup(struct net_device *dev, struct netdev_bpf *xdp) > +{ > + if (xdp->xsk.pool) > + return virtnet_xsk_pool_enable(dev, xdp->xsk.pool, > + xdp->xsk.queue_id); > + else > + return virtnet_xsk_pool_disable(dev, xdp->xsk.queue_id); > +} > + > diff --git a/drivers/net/virtio/xsk.h b/drivers/net/virtio/xsk.h > new file mode 100644 > index 000000000000..54948e0b07fc > --- /dev/null > +++ b/drivers/net/virtio/xsk.h > @@ -0,0 +1,99 @@ > +/* SPDX-License-Identifier: GPL-2.0-or-later */ > + > +#ifndef __XSK_H__ > +#define __XSK_H__ > + > +#define VIRTIO_XSK_FLAG BIT(1) > + > +/* When xsk disable, under normal circumstances, the network card must reclaim > + * all the memory that has been sent and the memory added to the rq queue by > + * destroying the queue. > + * > + * But virtio's queue does not support separate setting to been disable. This is a call for us to implement per queue enable/disable. Virtio-mmio has such facility but virtio-pci only allow to disable a queue (not enable). > "Reset" > + * is not very suitable. > + * > + * The method here is that each sent chunk or chunk added to the rq queue is > + * described by an independent structure struct virtnet_xsk_ctx. > + * > + * We will use get_page(page) to refer to the page where these chunks are > + * located. And these pages will be recorded in struct virtnet_xsk_ctx. So these > + * chunks in vq are safe. When recycling, put the these page. > + * > + * These structures point to struct virtnet_xsk_ctx_head, and ref records how > + * many chunks have not been reclaimed. If active == 0, it means that xsk has > + * been disabled. > + * > + * In this way, even if xsk has been unbundled with rq/sq, or a new xsk and > + * rq/sq are bound, and a new virtnet_xsk_ctx_head is created. It will not > + * affect the old virtnet_xsk_ctx to be recycled. And free all head and ctx when > + * ref is 0. This looks complicated and it will increase the footprint. Consider the performance penalty and the complexity, I would suggest to use reset instead. Then we don't need to introduce such context. Thanks > + */ > +struct virtnet_xsk_ctx; > +struct virtnet_xsk_ctx_head { > + struct virtnet_xsk_ctx *ctx; > + > + /* how many ctx has been add to vq */ > + u64 ref; > + > + unsigned int frame_size; > + > + /* the xsk status */ > + bool active; > +}; > + > +struct virtnet_xsk_ctx { > + struct virtnet_xsk_ctx_head *head; > + struct virtnet_xsk_ctx *next; > + > + struct page *page; > + > + /* xsk unaligned mode will use two page in one desc */ > + struct page *page_unaligned; > +}; > + > +struct virtnet_xsk_ctx_tx { > + /* this *MUST* be the first */ > + struct virtnet_xsk_ctx ctx; > + > + /* xsk tx xmit use this record the len of packet */ > + u32 len; > +}; > + > +static inline void *xsk_to_ptr(struct virtnet_xsk_ctx_tx *ctx) > +{ > + return (void *)((unsigned long)ctx | VIRTIO_XSK_FLAG); > +} > + > +static inline struct virtnet_xsk_ctx_tx *ptr_to_xsk(void *ptr) > +{ > + unsigned long p; > + > + p = (unsigned long)ptr; > + return (struct virtnet_xsk_ctx_tx *)(p & ~VIRTIO_XSK_FLAG); > +} > + > +static inline void virtnet_xsk_ctx_put(struct virtnet_xsk_ctx *ctx) > +{ > + put_page(ctx->page); > + if (ctx->page_unaligned) > + put_page(ctx->page_unaligned); > + > + --ctx->head->ref; > + > + if (ctx->head->active) { > + ctx->next = ctx->head->ctx; > + ctx->head->ctx = ctx; > + } else { > + if (!ctx->head->ref) > + kfree(ctx->head); > + } > +} > + > +#define virtnet_xsk_ctx_tx_put(ctx) \ > + virtnet_xsk_ctx_put((struct virtnet_xsk_ctx *)ctx) > + > +int virtnet_xsk_wakeup(struct net_device *dev, u32 qid, u32 flag); > +int virtnet_poll_xsk(struct send_queue *sq, int budget); > +void virtnet_xsk_complete(struct send_queue *sq, u32 num); > +int virtnet_xsk_pool_setup(struct net_device *dev, struct netdev_bpf *xdp); > +#endif From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.2 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ABEEC48BE5 for ; Wed, 16 Jun 2021 09:26:56 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5320F61040 for ; Wed, 16 Jun 2021 09:26:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5320F61040 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=virtualization-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 1D2DB83B0E; Wed, 16 Jun 2021 09:26:56 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id RLCQIJgK-XUM; Wed, 16 Jun 2021 09:26:54 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id EF8AC83A49; Wed, 16 Jun 2021 09:26:53 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id ABF62C000D; Wed, 16 Jun 2021 09:26:53 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id D4970C000B for ; Wed, 16 Jun 2021 09:26:52 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id AB1CC4067A for ; Wed, 16 Jun 2021 09:26:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=redhat.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FgF4I862impR for ; Wed, 16 Jun 2021 09:26:50 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [216.205.24.124]) by smtp4.osuosl.org (Postfix) with ESMTPS id A44DE4065B for ; Wed, 16 Jun 2021 09:26:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623835609; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=RW3o2HEy7SyzLUmjGnLmHDNB/DJgysCOrmXEWJSfZLs=; b=fJLqiFcANiFcR63LQMTUBf5kZscpdGWIdxna07wP0M+gNVz77//a2MSaYuLnD65UURAAPe evRtGJGE7Z10p+B+Hsf2n1ONb12mUp4FeKo6XtQxAdcJLJb2HlT7YQfb9+qc5uiMSB3+Wu UDhk4VZwjP9GQtRp+rN1iYliZ1lhjtw= Received: from mail-pj1-f72.google.com (mail-pj1-f72.google.com [209.85.216.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-223-J78Lsy2gOFiD36Otvd5Njw-1; Wed, 16 Jun 2021 05:26:47 -0400 X-MC-Unique: J78Lsy2gOFiD36Otvd5Njw-1 Received: by mail-pj1-f72.google.com with SMTP id jw3-20020a17090b4643b029016606f04954so1446658pjb.9 for ; Wed, 16 Jun 2021 02:26:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding :content-language; bh=RW3o2HEy7SyzLUmjGnLmHDNB/DJgysCOrmXEWJSfZLs=; b=tC9cnld9093gpYJCxOK1Ct2e9XWqFqeeAaR0w7mHgmLRpYjm/hwItAqFTtTb2dg0RM RO8lCe7BVu86EsXJ2ICzisn8U1CYmBS3vH2BFkehVWVchoP3J5MIR7/A/D0hSMsw//WE KsGJdEJym9+7KrPx5LZLb/HjWigI7vOoVsANZG6KCt3gth4ekbqSpIpvddh586p6myV/ 9fIT3g7CHkPhZTjHHkWXmgfHSYQfVOoXdmGa6R0zlYGPFhljQmMOTId0Vbx0rApyjtXf XuJ8IY90tC3Z4ZGlAQO8dLZqFbCNhgQi9AKu+QY0QLTZXLxyaAJIKjFnPU7JMLEdSF3D FdMA== X-Gm-Message-State: AOAM531l6hoRhnY0i3LkWshLxcgo9GoU33TewNnfP7ppM71MspSIxQYg DTt/+yPsULObUDpGLvAK4Gm9fEbkVxKThKlG5sJA6Ed9b4Wloz+vzpwSuP856fGvvr7f6vtOSbH 2vpSQd/4uykf148yVei7xtkBIZawsLwxlA3pL4tRGAA== X-Received: by 2002:a17:902:9006:b029:107:394a:387 with SMTP id a6-20020a1709029006b0290107394a0387mr7997049plp.35.1623835605707; Wed, 16 Jun 2021 02:26:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxEJ42y2sE35k74Q+Nctex4+yH8HLQmbZfHrBUlaSUqNZemHWxeVundizfyeLItBIHst18R3w== X-Received: by 2002:a17:902:9006:b029:107:394a:387 with SMTP id a6-20020a1709029006b0290107394a0387mr7997030plp.35.1623835605279; Wed, 16 Jun 2021 02:26:45 -0700 (PDT) Received: from wangxiaodeMacBook-Air.local ([209.132.188.80]) by smtp.gmail.com with ESMTPSA id bg16sm1701165pjb.16.2021.06.16.02.26.38 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 16 Jun 2021 02:26:44 -0700 (PDT) Subject: Re: [PATCH net-next v5 12/15] virtio-net: support AF_XDP zc tx To: Xuan Zhuo , netdev@vger.kernel.org References: <20210610082209.91487-1-xuanzhuo@linux.alibaba.com> <20210610082209.91487-13-xuanzhuo@linux.alibaba.com> From: Jason Wang Message-ID: <99116ba9-9c17-a519-471d-98ae96d049d9@redhat.com> Date: Wed, 16 Jun 2021 17:26:33 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 MIME-Version: 1.0 In-Reply-To: <20210610082209.91487-13-xuanzhuo@linux.alibaba.com> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=jasowang@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US Cc: Song Liu , Martin KaFai Lau , Jesper Dangaard Brouer , Daniel Borkmann , "Michael S. Tsirkin" , Yonghong Song , John Fastabend , Alexei Starovoitov , Andrii Nakryiko , =?UTF-8?B?QmrDtnJuIFTDtnBlbA==?= , "dust . li" , Jonathan Lemon , KP Singh , Jakub Kicinski , bpf@vger.kernel.org, virtualization@lists.linux-foundation.org, "David S. Miller" , Magnus Karlsson X-BeenThere: virtualization@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Linux virtualization List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: virtualization-bounces@lists.linux-foundation.org Sender: "Virtualization" CuWcqCAyMDIxLzYvMTAg5LiL5Y2INDoyMiwgWHVhbiBaaHVvIOWGmemBkzoKPiBBRl9YRFAoeGRw IHNvY2tldCwgeHNrKSBpcyBhIGhpZ2gtcGVyZm9ybWFuY2UgcGFja2V0IHJlY2VpdmluZyBhbmQK PiBzZW5kaW5nIHRlY2hub2xvZ3kuCj4KPiBUaGlzIHBhdGNoIGltcGxlbWVudHMgdGhlIGJpbmRp bmcgYW5kIHVuYmluZGluZyBvcGVyYXRpb25zIG9mIHhzayBhbmQKPiB0aGUgdmlydGlvLW5ldCBx dWV1ZSBmb3IgeHNrIHplcm8gY29weSB4bWl0Lgo+Cj4gVGhlIHhzayB6ZXJvIGNvcHkgeG1pdCBk ZXBlbmRzIG9uIHR4IG5hcGkuIEJlY2F1c2UgdGhlIGFjdHVhbCBzZW5kaW5nCj4gb2YgZGF0YSBp cyBkb25lIGluIHRoZSBwcm9jZXNzIG9mIHR4IG5hcGkuIElmIHR4IG5hcGkgZG9lcyBub3QKPiB3 b3JrLCB0aGVuIHRoZSBkYXRhIG9mIHRoZSB4c2sgdHggcXVldWUgd2lsbCBub3QgYmUgc2VudC4K PiBTbyBpZiB0eCBuYXBpIGlzIG5vdCB0cnVlLCBhbiBlcnJvciB3aWxsIGJlIHJlcG9ydGVkIHdo ZW4gYmluZCB4c2suCj4KPiBJZiB4c2sgaXMgYWN0aXZlLCBpdCB3aWxsIHByZXZlbnQgZXRodG9v bCBmcm9tIG1vZGlmeWluZyB0eCBuYXBpLgo+Cj4gV2hlbiByZWNsYWltaW5nIHB0ciwgYSBuZXcg dHlwZSBvZiBwdHIgaXMgYWRkZWQsIHdoaWNoIGlzIGRpc3Rpbmd1aXNoZWQKPiBiYXNlZCBvbiB0 aGUgbGFzdCB0d28gZGlnaXRzIG9mIHB0cjoKPiAwMDogc2tiCj4gMDE6IHhkcCBmcmFtZQo+IDEw OiB4c2sgeG1pdCBwdHIKPgo+IEFsbCBzZW50IHhzayBwYWNrZXRzIHNoYXJlIHRoZSB2aXJ0aW8t bmV0IGhlYWRlciBvZiB4c2tfaGRyLiBJZiB4c2sKPiBuZWVkcyB0byBzdXBwb3J0IGNzdW0gYW5k IG90aGVyIGZ1bmN0aW9ucyBsYXRlciwgY29uc2lkZXIgYXNzaWduaW5nIHhzawo+IGhkciBzZXBh cmF0ZWx5IGZvciBlYWNoIHNlbnQgcGFja2V0Lgo+Cj4gRGlmZmVyZW50IGZyb20gb3RoZXIgcGh5 c2ljYWwgbmV0d29yayBjYXJkcywgeW91IGNhbiByZWluaXRpYWxpemUgdGhlCj4gY2hhbm5lbCB3 aGVuIHlvdSBiaW5kIHhzay4gQW5kIHZydGlvIGRvZXMgbm90IHN1cHBvcnQgaW5kZXBlbmRlbnQg cmVzZXQKPiBjaGFubmVsLCB5b3UgY2FuIG9ubHkgcmVzZXQgdGhlIGVudGlyZSBkZXZpY2UuIEkg dGhpbmsgaXQgaXMgbm90Cj4gYXBwcm9wcmlhdGUgZm9yIHVzIHRvIGRpcmVjdGx5IHJlc2V0IHRo ZSBlbnRpcmUgc2V0dGluZy4gU28gdGhlCj4gc2l0dWF0aW9uIGJlY29tZXMgYSBiaXQgbW9yZSBj b21wbGljYXRlZC4gV2UgaGF2ZSB0byBjb25zaWRlciBob3cKPiB0byBkZWFsIHdpdGggdGhlIGJ1 ZmZlciByZWZlcmVuY2VkIGluIHZxIGFmdGVyIHhzayBpcyB1bmJpbmQuCj4KPiBJIGFkZGVkIHRo ZSByaW5nIHNpemUgc3RydWN0IHZpcnRuZXRfeHNrX2N0eCB3aGVuIHhzayBiZWVuIGJpbmQuIEVh Y2ggeHNrCj4gYnVmZmVyIGFkZGVkIHRvIHZxIGNvcnJlc3BvbmRzIHRvIGEgY3R4LiBUaGlzIGN0 eCBpcyB1c2VkIHRvIHJlY29yZCB0aGUKPiBwYWdlIHdoZXJlIHRoZSB4c2sgYnVmZmVyIGlzIGxv Y2F0ZWQsIGFuZCBhZGQgYSBwYWdlIHJlZmVyZW5jZS4gV2hlbiB0aGUKPiBidWZmZXIgaXMgcmVj eWNsaW5nLCByZWR1Y2UgdGhlIHJlZmVyZW5jZSB0byBwYWdlLiBXaGVuIHhzayBoYXMgYmVlbgo+ IHVuYmluZCwgYW5kIGFsbCByZWxhdGVkIHhzayBidWZmZXJzIGhhdmUgYmVlbiByZWN5Y2xlZCwg cmVsZWFzZSBhbGwgY3R4Lgo+Cj4gU2lnbmVkLW9mZi1ieTogWHVhbiBaaHVvIDx4dWFuemh1b0Bs aW51eC5hbGliYWJhLmNvbT4KPiBSZXZpZXdlZC1ieTogRHVzdCBMaSA8ZHVzdC5saUBsaW51eC5h bGliYWJhLmNvbT4KPiAtLS0KPiAgIGRyaXZlcnMvbmV0L3ZpcnRpby9NYWtlZmlsZSAgICAgfCAg IDEgKwo+ICAgZHJpdmVycy9uZXQvdmlydGlvL3ZpcnRpb19uZXQuYyB8ICAyMCArLQo+ICAgZHJp dmVycy9uZXQvdmlydGlvL3ZpcnRpb19uZXQuaCB8ICAzNyArKystCj4gICBkcml2ZXJzL25ldC92 aXJ0aW8veHNrLmMgICAgICAgIHwgMzQ2ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr Cj4gICBkcml2ZXJzL25ldC92aXJ0aW8veHNrLmggICAgICAgIHwgIDk5ICsrKysrKysrKwo+ICAg NSBmaWxlcyBjaGFuZ2VkLCA0OTcgaW5zZXJ0aW9ucygrKSwgNiBkZWxldGlvbnMoLSkKPiAgIGNy ZWF0ZSBtb2RlIDEwMDY0NCBkcml2ZXJzL25ldC92aXJ0aW8veHNrLmMKPiAgIGNyZWF0ZSBtb2Rl IDEwMDY0NCBkcml2ZXJzL25ldC92aXJ0aW8veHNrLmgKPgo+IGRpZmYgLS1naXQgYS9kcml2ZXJz L25ldC92aXJ0aW8vTWFrZWZpbGUgYi9kcml2ZXJzL25ldC92aXJ0aW8vTWFrZWZpbGUKPiBpbmRl eCBjY2M4MGY0MGYzM2EuLmRiNzlkMmU3OTI1ZiAxMDA2NDQKPiAtLS0gYS9kcml2ZXJzL25ldC92 aXJ0aW8vTWFrZWZpbGUKPiArKysgYi9kcml2ZXJzL25ldC92aXJ0aW8vTWFrZWZpbGUKPiBAQCAt NCwzICs0LDQgQEAKPiAgICMKPiAgIAo+ICAgb2JqLSQoQ09ORklHX1ZJUlRJT19ORVQpICs9IHZp cnRpb19uZXQubwo+ICtvYmotJChDT05GSUdfVklSVElPX05FVCkgKz0geHNrLm8KPiBkaWZmIC0t Z2l0IGEvZHJpdmVycy9uZXQvdmlydGlvL3ZpcnRpb19uZXQuYyBiL2RyaXZlcnMvbmV0L3ZpcnRp by92aXJ0aW9fbmV0LmMKPiBpbmRleCAzOTVlYzFmMTgzMzEuLjQwZDc3NTFmMWM1ZiAxMDA2NDQK PiAtLS0gYS9kcml2ZXJzL25ldC92aXJ0aW8vdmlydGlvX25ldC5jCj4gKysrIGIvZHJpdmVycy9u ZXQvdmlydGlvL3ZpcnRpb19uZXQuYwo+IEBAIC0xNDIzLDYgKzE0MjMsNyBAQCBzdGF0aWMgaW50 IHZpcnRuZXRfcG9sbF90eChzdHJ1Y3QgbmFwaV9zdHJ1Y3QgKm5hcGksIGludCBidWRnZXQpCj4g ICAKPiAgIAl0eHEgPSBuZXRkZXZfZ2V0X3R4X3F1ZXVlKHZpLT5kZXYsIGluZGV4KTsKPiAgIAlf X25ldGlmX3R4X2xvY2sodHhxLCByYXdfc21wX3Byb2Nlc3Nvcl9pZCgpKTsKPiArCXdvcmtfZG9u ZSArPSB2aXJ0bmV0X3BvbGxfeHNrKHNxLCBidWRnZXQpOwo+ICAgCWZyZWVfb2xkX3htaXQoc3Es IHRydWUpOwo+ICAgCV9fbmV0aWZfdHhfdW5sb2NrKHR4cSk7Cj4gICAKPiBAQCAtMjEzMyw4ICsy MTM0LDE2IEBAIHN0YXRpYyBpbnQgdmlydG5ldF9zZXRfY29hbGVzY2Uoc3RydWN0IG5ldF9kZXZp Y2UgKmRldiwKPiAgIAlpZiAobmFwaV93ZWlnaHQgXiB2aS0+c3FbMF0ubmFwaS53ZWlnaHQpIHsK PiAgIAkJaWYgKGRldi0+ZmxhZ3MgJiBJRkZfVVApCj4gICAJCQlyZXR1cm4gLUVCVVNZOwo+IC0J CWZvciAoaSA9IDA7IGkgPCB2aS0+bWF4X3F1ZXVlX3BhaXJzOyBpKyspCj4gKwo+ICsJCWZvciAo aSA9IDA7IGkgPCB2aS0+bWF4X3F1ZXVlX3BhaXJzOyBpKyspIHsKPiArCQkJLyogeHNrIHhtaXQg ZGVwZW5kIG9uIHRoZSB0eCBuYXBpLiBTbyBpZiB4c2sgaXMgYWN0aXZlLAo+ICsJCQkgKiBwcmV2 ZW50IG1vZGlmaWNhdGlvbnMgdG8gdHggbmFwaS4KPiArCQkJICovCj4gKwkJCWlmIChydG5sX2Rl cmVmZXJlbmNlKHZpLT5zcVtpXS54c2sucG9vbCkpCj4gKwkJCQljb250aW51ZTsKCgpTbyB0aGlz IGNhbiByZXN1bHQgdHggTkFQSSBpcyB1c2VkIGJ5IHNvbWUgcXVldWVzIGJ1eSBub3QgdGhlIG90 aGVycy4gSSAKdGhpbmsgc3VjaCBpbmNvbnNpc3RlbmN5IGJyZWFrcyB0aGUgc2VtYW50aWMgb2Yg c2V0X2NvYWxlc2NlKCkgd2hpY2ggCmFzc3VtZXMgdGhlIG9wZXJhdGlvbiBpcyBkb25lIGF0IHRo ZSBkZXZpY2Ugbm90IHNvbWUgc3BlY2lmaWMgcXVldWVzLgoKSG93IGFib3V0IGp1c3QgZmFpbCBo ZXJlPwoKCj4gKwo+ICAgCQkJdmktPnNxW2ldLm5hcGkud2VpZ2h0ID0gbmFwaV93ZWlnaHQ7Cj4g KwkJfQo+ICAgCX0KPiAgIAo+ICAgCXJldHVybiAwOwo+IEBAIC0yNDA3LDYgKzI0MTYsOCBAQCBz dGF0aWMgaW50IHZpcnRuZXRfeGRwKHN0cnVjdCBuZXRfZGV2aWNlICpkZXYsIHN0cnVjdCBuZXRk ZXZfYnBmICp4ZHApCj4gICAJc3dpdGNoICh4ZHAtPmNvbW1hbmQpIHsKPiAgIAljYXNlIFhEUF9T RVRVUF9QUk9HOgo+ICAgCQlyZXR1cm4gdmlydG5ldF94ZHBfc2V0KGRldiwgeGRwLT5wcm9nLCB4 ZHAtPmV4dGFjayk7Cj4gKwljYXNlIFhEUF9TRVRVUF9YU0tfUE9PTDoKPiArCQlyZXR1cm4gdmly dG5ldF94c2tfcG9vbF9zZXR1cChkZXYsIHhkcCk7Cj4gICAJZGVmYXVsdDoKPiAgIAkJcmV0dXJu IC1FSU5WQUw7Cj4gICAJfQo+IEBAIC0yNDY2LDYgKzI0NzcsNyBAQCBzdGF0aWMgY29uc3Qgc3Ry dWN0IG5ldF9kZXZpY2Vfb3BzIHZpcnRuZXRfbmV0ZGV2ID0gewo+ICAgCS5uZG9fdmxhbl9yeF9r aWxsX3ZpZCA9IHZpcnRuZXRfdmxhbl9yeF9raWxsX3ZpZCwKPiAgIAkubmRvX2JwZgkJPSB2aXJ0 bmV0X3hkcCwKPiAgIAkubmRvX3hkcF94bWl0CQk9IHZpcnRuZXRfeGRwX3htaXQsCj4gKwkubmRv X3hza193YWtldXAgICAgICAgICA9IHZpcnRuZXRfeHNrX3dha2V1cCwKPiAgIAkubmRvX2ZlYXR1 cmVzX2NoZWNrCT0gcGFzc3RocnVfZmVhdHVyZXNfY2hlY2ssCj4gICAJLm5kb19nZXRfcGh5c19w b3J0X25hbWUJPSB2aXJ0bmV0X2dldF9waHlzX3BvcnRfbmFtZSwKPiAgIAkubmRvX3NldF9mZWF0 dXJlcwk9IHZpcnRuZXRfc2V0X2ZlYXR1cmVzLAo+IEBAIC0yNTY5LDEwICsyNTgxLDEyIEBAIHN0 YXRpYyB2b2lkIGZyZWVfdW51c2VkX2J1ZnMoc3RydWN0IHZpcnRuZXRfaW5mbyAqdmkpCj4gICAJ Zm9yIChpID0gMDsgaSA8IHZpLT5tYXhfcXVldWVfcGFpcnM7IGkrKykgewo+ICAgCQlzdHJ1Y3Qg dmlydHF1ZXVlICp2cSA9IHZpLT5zcVtpXS52cTsKPiAgIAkJd2hpbGUgKChidWYgPSB2aXJ0cXVl dWVfZGV0YWNoX3VudXNlZF9idWYodnEpKSAhPSBOVUxMKSB7Cj4gLQkJCWlmICghaXNfeGRwX2Zy YW1lKGJ1ZikpCj4gKwkJCWlmIChpc19za2JfcHRyKGJ1ZikpCj4gICAJCQkJZGV2X2tmcmVlX3Nr YihidWYpOwo+IC0JCQllbHNlCj4gKwkJCWVsc2UgaWYgKGlzX3hkcF9mcmFtZShidWYpKQo+ICAg CQkJCXhkcF9yZXR1cm5fZnJhbWUocHRyX3RvX3hkcChidWYpKTsKPiArCQkJZWxzZQo+ICsJCQkJ dmlydG5ldF94c2tfY3R4X3R4X3B1dChwdHJfdG9feHNrKGJ1ZikpOwo+ICAgCQl9Cj4gICAJfQo+ ICAgCj4gZGlmZiAtLWdpdCBhL2RyaXZlcnMvbmV0L3ZpcnRpby92aXJ0aW9fbmV0LmggYi9kcml2 ZXJzL25ldC92aXJ0aW8vdmlydGlvX25ldC5oCj4gaW5kZXggOTMxY2M4MWY5MmZiLi5lM2RhODI5 ODg3ZGMgMTAwNjQ0Cj4gLS0tIGEvZHJpdmVycy9uZXQvdmlydGlvL3ZpcnRpb19uZXQuaAo+ICsr KyBiL2RyaXZlcnMvbmV0L3ZpcnRpby92aXJ0aW9fbmV0LmgKPiBAQCAtMTM1LDYgKzEzNSwxNiBA QCBzdHJ1Y3Qgc2VuZF9xdWV1ZSB7Cj4gICAJc3RydWN0IHZpcnRuZXRfc3Ffc3RhdHMgc3RhdHM7 Cj4gICAKPiAgIAlzdHJ1Y3QgbmFwaV9zdHJ1Y3QgbmFwaTsKPiArCj4gKwlzdHJ1Y3Qgewo+ICsJ CXN0cnVjdCB4c2tfYnVmZl9wb29sIF9fcmN1ICpwb29sOwo+ICsKPiArCQkvKiB4c2sgd2FpdCBm b3IgdHggaW50ZXIgb3Igc29mdGlycSAqLwo+ICsJCWJvb2wgbmVlZF93YWtldXA7Cj4gKwo+ICsJ CS8qIGN0eCB1c2VkIHRvIHJlY29yZCB0aGUgcGFnZSBhZGRlZCB0byB2cSAqLwo+ICsJCXN0cnVj dCB2aXJ0bmV0X3hza19jdHhfaGVhZCAqY3R4X2hlYWQ7Cj4gKwl9IHhzazsKPiAgIH07Cj4gICAK PiAgIC8qIEludGVybmFsIHJlcHJlc2VudGF0aW9uIG9mIGEgcmVjZWl2ZSB2aXJ0cXVldWUgKi8K PiBAQCAtMTg4LDYgKzE5OCwxMyBAQCBzdGF0aWMgaW5saW5lIHZvaWQgdmlydHF1ZXVlX25hcGlf c2NoZWR1bGUoc3RydWN0IG5hcGlfc3RydWN0ICpuYXBpLAo+ICAgCX0KPiAgIH0KPiAgIAo+ICsj aW5jbHVkZSAieHNrLmgiCj4gKwo+ICtzdGF0aWMgaW5saW5lIGJvb2wgaXNfc2tiX3B0cih2b2lk ICpwdHIpCj4gK3sKPiArCXJldHVybiAhKCh1bnNpZ25lZCBsb25nKXB0ciAmIChWSVJUSU9fWERQ X0ZMQUcgfCBWSVJUSU9fWFNLX0ZMQUcpKTsKPiArfQo+ICsKPiAgIHN0YXRpYyBpbmxpbmUgYm9v bCBpc194ZHBfZnJhbWUodm9pZCAqcHRyKQo+ICAgewo+ICAgCXJldHVybiAodW5zaWduZWQgbG9u ZylwdHIgJiBWSVJUSU9fWERQX0ZMQUc7Cj4gQEAgLTIwNiwyNSArMjIzLDM5IEBAIHN0YXRpYyBp bmxpbmUgc3RydWN0IHhkcF9mcmFtZSAqcHRyX3RvX3hkcCh2b2lkICpwdHIpCj4gICBzdGF0aWMg aW5saW5lIHZvaWQgX19mcmVlX29sZF94bWl0KHN0cnVjdCBzZW5kX3F1ZXVlICpzcSwgYm9vbCBp bl9uYXBpLAo+ICAgCQkJCSAgIHN0cnVjdCB2aXJ0bmV0X3NxX3N0YXRzICpzdGF0cykKPiAgIHsK PiArCXVuc2lnbmVkIGludCB4c2tudW0gPSAwOwo+ICAgCXVuc2lnbmVkIGludCBsZW47Cj4gICAJ dm9pZCAqcHRyOwo+ICAgCj4gICAJd2hpbGUgKChwdHIgPSB2aXJ0cXVldWVfZ2V0X2J1ZihzcS0+ dnEsICZsZW4pKSAhPSBOVUxMKSB7Cj4gLQkJaWYgKCFpc194ZHBfZnJhbWUocHRyKSkgewo+ICsJ CWlmIChpc19za2JfcHRyKHB0cikpIHsKPiAgIAkJCXN0cnVjdCBza19idWZmICpza2IgPSBwdHI7 Cj4gICAKPiAgIAkJCXByX2RlYnVnKCJTZW50IHNrYiAlcFxuIiwgc2tiKTsKPiAgIAo+ICAgCQkJ c3RhdHMtPmJ5dGVzICs9IHNrYi0+bGVuOwo+ICAgCQkJbmFwaV9jb25zdW1lX3NrYihza2IsIGlu X25hcGkpOwo+IC0JCX0gZWxzZSB7Cj4gKwkJfSBlbHNlIGlmIChpc194ZHBfZnJhbWUocHRyKSkg ewo+ICAgCQkJc3RydWN0IHhkcF9mcmFtZSAqZnJhbWUgPSBwdHJfdG9feGRwKHB0cik7Cj4gICAK PiAgIAkJCXN0YXRzLT5ieXRlcyArPSBmcmFtZS0+bGVuOwo+ICAgCQkJeGRwX3JldHVybl9mcmFt ZShmcmFtZSk7Cj4gKwkJfSBlbHNlIHsKPiArCQkJc3RydWN0IHZpcnRuZXRfeHNrX2N0eF90eCAq Y3R4Owo+ICsKPiArCQkJY3R4ID0gcHRyX3RvX3hzayhwdHIpOwo+ICsKPiArCQkJLyogTWF5YmUg dGhpcyBwdHIgd2FzIHNlbnQgYnkgdGhlIGxhc3QgeHNrLiAqLwo+ICsJCQlpZiAoY3R4LT5jdHgu aGVhZC0+YWN0aXZlKQo+ICsJCQkJKyt4c2tudW07Cj4gKwo+ICsJCQlzdGF0cy0+Ynl0ZXMgKz0g Y3R4LT5sZW47Cj4gKwkJCXZpcnRuZXRfeHNrX2N0eF90eF9wdXQoY3R4KTsKPiAgIAkJfQo+ICAg CQlzdGF0cy0+cGFja2V0cysrOwo+ICAgCX0KPiAtfQo+ICAgCj4gKwlpZiAoeHNrbnVtKQo+ICsJ CXZpcnRuZXRfeHNrX2NvbXBsZXRlKHNxLCB4c2tudW0pOwo+ICt9Cj4gICAjZW5kaWYKPiBkaWZm IC0tZ2l0IGEvZHJpdmVycy9uZXQvdmlydGlvL3hzay5jIGIvZHJpdmVycy9uZXQvdmlydGlvL3hz ay5jCj4gbmV3IGZpbGUgbW9kZSAxMDA2NDQKPiBpbmRleCAwMDAwMDAwMDAwMDAuLmY5OGI2ODU3 NjcwOQo+IC0tLSAvZGV2L251bGwKPiArKysgYi9kcml2ZXJzL25ldC92aXJ0aW8veHNrLmMKPiBA QCAtMCwwICsxLDM0NiBAQAo+ICsvLyBTUERYLUxpY2Vuc2UtSWRlbnRpZmllcjogR1BMLTIuMC1v ci1sYXRlcgo+ICsvKgo+ICsgKiB2aXJ0aW8tbmV0IHhzawo+ICsgKi8KPiArCj4gKyNpbmNsdWRl ICJ2aXJ0aW9fbmV0LmgiCj4gKwo+ICtzdGF0aWMgc3RydWN0IHZpcnRpb19uZXRfaGRyX21yZ19y eGJ1ZiB4c2tfaGRyOwo+ICsKPiArc3RhdGljIHN0cnVjdCB2aXJ0bmV0X3hza19jdHggKnZpcnRu ZXRfeHNrX2N0eF9nZXQoc3RydWN0IHZpcnRuZXRfeHNrX2N0eF9oZWFkICpoZWFkKQo+ICt7Cj4g KwlzdHJ1Y3QgdmlydG5ldF94c2tfY3R4ICpjdHg7Cj4gKwo+ICsJY3R4ID0gaGVhZC0+Y3R4Owo+ ICsJaGVhZC0+Y3R4ID0gY3R4LT5uZXh0Owo+ICsKPiArCSsraGVhZC0+cmVmOwo+ICsKPiArCXJl dHVybiBjdHg7Cj4gK30KPiArCj4gKyNkZWZpbmUgdmlydG5ldF94c2tfY3R4X3R4X2dldChoZWFk KSAoKHN0cnVjdCB2aXJ0bmV0X3hza19jdHhfdHggKil2aXJ0bmV0X3hza19jdHhfZ2V0KGhlYWQp KQo+ICsKPiArc3RhdGljIHZvaWQgdmlydG5ldF94c2tfY2hlY2tfcXVldWUoc3RydWN0IHNlbmRf cXVldWUgKnNxKQo+ICt7Cj4gKwlzdHJ1Y3QgdmlydG5ldF9pbmZvICp2aSA9IHNxLT52cS0+dmRl di0+cHJpdjsKPiArCXN0cnVjdCBuZXRfZGV2aWNlICpkZXYgPSB2aS0+ZGV2Owo+ICsJaW50IHFu dW0gPSBzcSAtIHZpLT5zcTsKPiArCj4gKwkvKiBJZiBpdCBpcyBhIHJhdyBidWZmZXIgcXVldWUs IGl0IGRvZXMgbm90IGNoZWNrIHdoZXRoZXIgdGhlIHN0YXR1cwo+ICsJICogb2YgdGhlIHF1ZXVl IGlzIHN0b3BwZWQgd2hlbiBzZW5kaW5nLiBTbyB0aGVyZSBpcyBubyBuZWVkIHRvIGNoZWNrCj4g KwkgKiB0aGUgc2l0dWF0aW9uIG9mIHRoZSByYXcgYnVmZmVyIHF1ZXVlLgo+ICsJICovCj4gKwlp ZiAoaXNfeGRwX3Jhd19idWZmZXJfcXVldWUodmksIHFudW0pKQo+ICsJCXJldHVybjsKPiArCj4g KwkvKiBJZiB0aGlzIHNxIGlzIG5vdCB0aGUgZXhjbHVzaXZlIHF1ZXVlIG9mIHRoZSBjdXJyZW50 IGNwdSwKPiArCSAqIHRoZW4gaXQgbWF5IGJlIGNhbGxlZCBieSBzdGFydF94bWl0LCBzbyBjaGVj ayBpdCBydW5uaW5nIG91dAo+ICsJICogb2Ygc3BhY2UuCj4gKwkgKgo+ICsJICogU3RvcCB0aGUg cXVldWUgdG8gYXZvaWQgZ2V0dGluZyBwYWNrZXRzIHRoYXQgd2UgYXJlCj4gKwkgKiB0aGVuIHVu YWJsZSB0byB0cmFuc21pdC4gVGhlbiB3YWl0IHRoZSB0eCBpbnRlcnJ1cHQuCj4gKwkgKi8KPiAr CWlmIChzcS0+dnEtPm51bV9mcmVlIDwgMiArIE1BWF9TS0JfRlJBR1MpCj4gKwkJbmV0aWZfc3Rv cF9zdWJxdWV1ZShkZXYsIHFudW0pOwo+ICt9Cj4gKwo+ICt2b2lkIHZpcnRuZXRfeHNrX2NvbXBs ZXRlKHN0cnVjdCBzZW5kX3F1ZXVlICpzcSwgdTMyIG51bSkKPiArewo+ICsJc3RydWN0IHhza19i dWZmX3Bvb2wgKnBvb2w7Cj4gKwo+ICsJcmN1X3JlYWRfbG9jaygpOwo+ICsJcG9vbCA9IHJjdV9k ZXJlZmVyZW5jZShzcS0+eHNrLnBvb2wpOwo+ICsJaWYgKCFwb29sKSB7Cj4gKwkJcmN1X3JlYWRf dW5sb2NrKCk7Cj4gKwkJcmV0dXJuOwo+ICsJfQo+ICsJeHNrX3R4X2NvbXBsZXRlZChwb29sLCBu dW0pOwo+ICsJcmN1X3JlYWRfdW5sb2NrKCk7Cj4gKwo+ICsJaWYgKHNxLT54c2submVlZF93YWtl dXApIHsKPiArCQlzcS0+eHNrLm5lZWRfd2FrZXVwID0gZmFsc2U7Cj4gKwkJdmlydHF1ZXVlX25h cGlfc2NoZWR1bGUoJnNxLT5uYXBpLCBzcS0+dnEpOwo+ICsJfQo+ICt9Cj4gKwo+ICtzdGF0aWMg aW50IHZpcnRuZXRfeHNrX3htaXQoc3RydWN0IHNlbmRfcXVldWUgKnNxLCBzdHJ1Y3QgeHNrX2J1 ZmZfcG9vbCAqcG9vbCwKPiArCQkJICAgIHN0cnVjdCB4ZHBfZGVzYyAqZGVzYykKPiArewo+ICsJ c3RydWN0IHZpcnRuZXRfeHNrX2N0eF90eCAqY3R4Owo+ICsJc3RydWN0IHZpcnRuZXRfaW5mbyAq dmk7Cj4gKwl1MzIgb2Zmc2V0LCBuLCBsZW47Cj4gKwlzdHJ1Y3QgcGFnZSAqcGFnZTsKPiArCXZv aWQgKmRhdGE7Cj4gKwo+ICsJdmkgPSBzcS0+dnEtPnZkZXYtPnByaXY7Cj4gKwo+ICsJZGF0YSA9 IHhza19idWZmX3Jhd19nZXRfZGF0YShwb29sLCBkZXNjLT5hZGRyKTsKPiArCW9mZnNldCA9IG9m ZnNldF9pbl9wYWdlKGRhdGEpOwo+ICsKPiArCWN0eCA9IHZpcnRuZXRfeHNrX2N0eF90eF9nZXQo c3EtPnhzay5jdHhfaGVhZCk7Cj4gKwo+ICsJLyogeHNrIHVuYWxpZ25lZCBtb2RlLCBkZXNjIG1h eSB1c2UgdHdvIHBhZ2VzICovCj4gKwlpZiAoZGVzYy0+bGVuID4gUEFHRV9TSVpFIC0gb2Zmc2V0 KQo+ICsJCW4gPSAzOwo+ICsJZWxzZQo+ICsJCW4gPSAyOwo+ICsKPiArCXNnX2luaXRfdGFibGUo c3EtPnNnLCBuKTsKPiArCXNnX3NldF9idWYoc3EtPnNnLCAmeHNrX2hkciwgdmktPmhkcl9sZW4p Owo+ICsKPiArCS8qIGhhbmRsZSBmb3IgeHNrIGZpcnN0IHBhZ2UgKi8KPiArCWxlbiA9IG1pbl90 KGludCwgZGVzYy0+bGVuLCBQQUdFX1NJWkUgLSBvZmZzZXQpOwo+ICsJcGFnZSA9IHZtYWxsb2Nf dG9fcGFnZShkYXRhKTsKPiArCXNnX3NldF9wYWdlKHNxLT5zZyArIDEsIHBhZ2UsIGxlbiwgb2Zm c2V0KTsKPiArCj4gKwkvKiBjdHggaXMgdXNlZCB0byByZWNvcmQgYW5kIHJlZmVyZW5jZSB0aGlz IHBhZ2UgdG8gcHJldmVudCB4c2sgZnJvbQo+ICsJICogYmVpbmcgcmVsZWFzZWQgYmVmb3JlIHRo aXMgeG1pdCBpcyByZWN5Y2xlZAo+ICsJICovCgoKSSdtIGEgbGl0dGxlIGJpdCBzdXJwcmlzZWQg dGhhdCB0aGlzIGlzIGRvbmUgbWFudWFsbHkgcGVyIGRldmljZSBpbnN0ZWFkIApvZiBkb2luZyBp dCBpbiB4c2sgY29yZS4KCgo+ICsJY3R4LT5jdHgucGFnZSA9IHBhZ2U7Cj4gKwlnZXRfcGFnZShw YWdlKTsKPiArCj4gKwkvKiB4c2sgdW5hbGlnbmVkIG1vZGUsIGhhbmRsZSBmb3IgdGhlIHNlY29u ZCBwYWdlICovCj4gKwlpZiAobGVuIDwgZGVzYy0+bGVuKSB7Cj4gKwkJcGFnZSA9IHZtYWxsb2Nf dG9fcGFnZShkYXRhICsgbGVuKTsKPiArCQlsZW4gPSBtaW5fdChpbnQsIGRlc2MtPmxlbiAtIGxl biwgUEFHRV9TSVpFKTsKPiArCQlzZ19zZXRfcGFnZShzcS0+c2cgKyAyLCBwYWdlLCBsZW4sIDAp Owo+ICsKPiArCQljdHgtPmN0eC5wYWdlX3VuYWxpZ25lZCA9IHBhZ2U7Cj4gKwkJZ2V0X3BhZ2Uo cGFnZSk7Cj4gKwl9IGVsc2Ugewo+ICsJCWN0eC0+Y3R4LnBhZ2VfdW5hbGlnbmVkID0gTlVMTDsK PiArCX0KPiArCj4gKwlyZXR1cm4gdmlydHF1ZXVlX2FkZF9vdXRidWYoc3EtPnZxLCBzcS0+c2cs IG4sCj4gKwkJCQkgICB4c2tfdG9fcHRyKGN0eCksIEdGUF9BVE9NSUMpOwo+ICt9Cj4gKwo+ICtz dGF0aWMgaW50IHZpcnRuZXRfeHNrX3htaXRfYmF0Y2goc3RydWN0IHNlbmRfcXVldWUgKnNxLAo+ ICsJCQkJICBzdHJ1Y3QgeHNrX2J1ZmZfcG9vbCAqcG9vbCwKPiArCQkJCSAgdW5zaWduZWQgaW50 IGJ1ZGdldCwKPiArCQkJCSAgYm9vbCBpbl9uYXBpLCBpbnQgKmRvbmUsCj4gKwkJCQkgIHN0cnVj dCB2aXJ0bmV0X3NxX3N0YXRzICpzdGF0cykKPiArewo+ICsJc3RydWN0IHhkcF9kZXNjIGRlc2M7 Cj4gKwlpbnQgZXJyLCBwYWNrZXQgPSAwOwo+ICsJaW50IHJldCA9IC1FQUdBSU47Cj4gKwo+ICsJ d2hpbGUgKGJ1ZGdldC0tID4gMCkgewo+ICsJCWlmIChzcS0+dnEtPm51bV9mcmVlIDwgMiArIE1B WF9TS0JfRlJBR1MpIHsKCgpBRl9YRFAgZG9lc24ndCB1c2Ugc2tiLCBzbyBJIGRvbid0IHNlZSB3 aHkgTUFYX1NLQl9GUkFHUyBpcyB1c2VkLgoKTG9va2luZyBhdCB2aXJ0bmV0X3hza194bWl0KCks IGl0IGxvb2tzIHRvIG1lIDMgaXMgbW9yZSBzdWl0YWJsZSBoZXJlLiAKT3IgZGlkIEFGX1hEUCBj b3JlIGNhbiBoYW5kbGUgcXVldWUgZnVsbCBncmFjZWZ1bGx5IHRoZW4gd2UgZG9uJ3QgZXZlbiAK bmVlZCB0byB3b3JyeSBhYm91dCB0aGlzPwoKCj4gKwkJCXJldCA9IC1FQlVTWTsKCgotRU5PU1BD IGxvb2tzIGJldHRlci4KCgo+ICsJCQlicmVhazsKPiArCQl9Cj4gKwo+ICsJCWlmICgheHNrX3R4 X3BlZWtfZGVzYyhwb29sLCAmZGVzYykpIHsKPiArCQkJLyogZG9uZSAqLwo+ICsJCQlyZXQgPSAw Owo+ICsJCQlicmVhazsKPiArCQl9Cj4gKwo+ICsJCWVyciA9IHZpcnRuZXRfeHNrX3htaXQoc3Es IHBvb2wsICZkZXNjKTsKPiArCQlpZiAodW5saWtlbHkoZXJyKSkgewoKCklmIHdlIGFsd2F5cyBy ZXNlcnZlIHN1ZmZpY2llbnQgc2xvdHMsIHRoaXMgc2hvdWxkIGJlIGFuIHVuZXhwZWN0ZWQgCmVy cm9yLCBkbyB3ZSBuZWVkIGxvZyB0aGlzIGFzIHdoYXQgaGFzIGJlZW4gZG9uZSBpbiBzdGFydF94 bWl0KCk/CgogwqDCoMKgwqDCoMKgwqAgLyogVGhpcyBzaG91bGQgbm90IGhhcHBlbiEgKi8KIMKg wqDCoMKgwqDCoMKgIGlmICh1bmxpa2VseShlcnIpKSB7CiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgZGV2LT5zdGF0cy50eF9maWZvX2Vycm9ycysrOwogwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgIGlmIChuZXRfcmF0ZWxpbWl0KCkpCiDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGRldl93YXJuKCZkZXYtPmRldiwKIMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgIlVu ZXhwZWN0ZWQgVFhRICglZCkgcXVldWUgZmFpbHVyZTogJWRcbiIsCiDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHFudW0sIGVy cik7CgoKPiArCQkJcmV0ID0gLUVCVVNZOwo+ICsJCQlicmVhazsKPiArCQl9Cj4gKwo+ICsJCSsr cGFja2V0Owo+ICsJfQo+ICsKPiArCWlmIChwYWNrZXQpIHsKPiArCQlpZiAodmlydHF1ZXVlX2tp Y2tfcHJlcGFyZShzcS0+dnEpICYmIHZpcnRxdWV1ZV9ub3RpZnkoc3EtPnZxKSkKPiArCQkJKytz dGF0cy0+a2lja3M7Cj4gKwo+ICsJCSpkb25lICs9IHBhY2tldDsKPiArCQlzdGF0cy0+eGRwX3R4 ICs9IHBhY2tldDsKPiArCj4gKwkJeHNrX3R4X3JlbGVhc2UocG9vbCk7Cj4gKwl9Cj4gKwo+ICsJ cmV0dXJuIHJldDsKPiArfQo+ICsKPiArc3RhdGljIGludCB2aXJ0bmV0X3hza19ydW4oc3RydWN0 IHNlbmRfcXVldWUgKnNxLCBzdHJ1Y3QgeHNrX2J1ZmZfcG9vbCAqcG9vbCwKPiArCQkJICAgaW50 IGJ1ZGdldCwgYm9vbCBpbl9uYXBpKQo+ICt7Cj4gKwlzdHJ1Y3QgdmlydG5ldF9zcV9zdGF0cyBz dGF0cyA9IHt9Owo+ICsJaW50IGRvbmUgPSAwOwo+ICsJaW50IGVycjsKPiArCj4gKwlzcS0+eHNr Lm5lZWRfd2FrZXVwID0gZmFsc2U7Cj4gKwlfX2ZyZWVfb2xkX3htaXQoc3EsIGluX25hcGksICZz dGF0cyk7Cj4gKwo+ICsJLyogcmV0dXJuIGVycjoKPiArCSAqIC1FQUdBSU46IGRvbmUgPT0gYnVk Z2V0Cj4gKwkgKiAtRUJVU1k6ICBkb25lIDwgYnVkZ2V0Cj4gKwkgKiAgMCAgICA6ICBkb25lIDwg YnVkZ2V0Cj4gKwkgKi8KCgpJdCdzIGJldHRlciB0byBtb3ZlIHRoZSBjb21tZW50IHRvIHRoZSBp bXBsZW1lbnRhdGlvbiBvZiAKdmlydG5ldF94c2tfeG1pdF9iYXRjaCgpLgoKCj4gK3htaXQ6Cj4g KwllcnIgPSB2aXJ0bmV0X3hza194bWl0X2JhdGNoKHNxLCBwb29sLCBidWRnZXQgLSBkb25lLCBp bl9uYXBpLAo+ICsJCQkJICAgICAmZG9uZSwgJnN0YXRzKTsKPiArCWlmIChlcnIgPT0gLUVCVVNZ KSB7Cj4gKwkJX19mcmVlX29sZF94bWl0KHNxLCBpbl9uYXBpLCAmc3RhdHMpOwo+ICsKPiArCQkv KiBJZiB0aGUgc3BhY2UgaXMgZW5vdWdoLCBsZXQgbmFwaSBydW4gYWdhaW4uICovCj4gKwkJaWYg KHNxLT52cS0+bnVtX2ZyZWUgPj0gMiArIE1BWF9TS0JfRlJBR1MpCgoKVGhlIGNvbW1lbnQgZG9l cyBub3QgbWF0Y2ggdGhlIGNvZGUuCgoKPiArCQkJZ290byB4bWl0Owo+ICsJCWVsc2UKPiArCQkJ c3EtPnhzay5uZWVkX3dha2V1cCA9IHRydWU7Cj4gKwl9Cj4gKwo+ICsJdmlydG5ldF94c2tfY2hl Y2tfcXVldWUoc3EpOwo+ICsKPiArCXU2NF9zdGF0c191cGRhdGVfYmVnaW4oJnNxLT5zdGF0cy5z eW5jcCk7Cj4gKwlzcS0+c3RhdHMucGFja2V0cyArPSBzdGF0cy5wYWNrZXRzOwo+ICsJc3EtPnN0 YXRzLmJ5dGVzICs9IHN0YXRzLmJ5dGVzOwo+ICsJc3EtPnN0YXRzLmtpY2tzICs9IHN0YXRzLmtp Y2tzOwo+ICsJc3EtPnN0YXRzLnhkcF90eCArPSBzdGF0cy54ZHBfdHg7Cj4gKwl1NjRfc3RhdHNf dXBkYXRlX2VuZCgmc3EtPnN0YXRzLnN5bmNwKTsKPiArCj4gKwlyZXR1cm4gZG9uZTsKPiArfQo+ ICsKPiAraW50IHZpcnRuZXRfcG9sbF94c2soc3RydWN0IHNlbmRfcXVldWUgKnNxLCBpbnQgYnVk Z2V0KQo+ICt7Cj4gKwlzdHJ1Y3QgeHNrX2J1ZmZfcG9vbCAqcG9vbDsKPiArCWludCB3b3JrX2Rv bmUgPSAwOwo+ICsKPiArCXJjdV9yZWFkX2xvY2soKTsKPiArCXBvb2wgPSByY3VfZGVyZWZlcmVu Y2Uoc3EtPnhzay5wb29sKTsKPiArCWlmIChwb29sKQo+ICsJCXdvcmtfZG9uZSA9IHZpcnRuZXRf eHNrX3J1bihzcSwgcG9vbCwgYnVkZ2V0LCB0cnVlKTsKPiArCXJjdV9yZWFkX3VubG9jaygpOwo+ ICsJcmV0dXJuIHdvcmtfZG9uZTsKPiArfQo+ICsKPiAraW50IHZpcnRuZXRfeHNrX3dha2V1cChz dHJ1Y3QgbmV0X2RldmljZSAqZGV2LCB1MzIgcWlkLCB1MzIgZmxhZykKPiArewo+ICsJc3RydWN0 IHZpcnRuZXRfaW5mbyAqdmkgPSBuZXRkZXZfcHJpdihkZXYpOwo+ICsJc3RydWN0IHhza19idWZm X3Bvb2wgKnBvb2w7Cj4gKwlzdHJ1Y3Qgc2VuZF9xdWV1ZSAqc3E7Cj4gKwo+ICsJaWYgKCFuZXRp Zl9ydW5uaW5nKGRldikpCj4gKwkJcmV0dXJuIC1FTkVURE9XTjsKPiArCj4gKwlpZiAocWlkID49 IHZpLT5jdXJyX3F1ZXVlX3BhaXJzKQo+ICsJCXJldHVybiAtRUlOVkFMOwoKCkkgd29uZGVyIGhv dyB3ZSBjYW4gaGl0IHRoaXMgY2hlY2suIE5vdGUgdGhhdCB3ZSBwcmV2ZW50IHRoZSB1c2VyIGZy b20gCm1vZGlmeWluZyBxdWV1ZSBwYWlycyB3aGVuIFhEUCBpcyBlbmFibGVkOgoKIMKgwqDCoMKg wqDCoMKgIC8qIEZvciBub3cgd2UgZG9uJ3Qgc3VwcG9ydCBtb2RpZnlpbmcgY2hhbm5lbHMgd2hp bGUgWERQIGlzIGxvYWRlZAogwqDCoMKgwqDCoMKgwqDCoCAqIGFsc28gd2hlbiBYRFAgaXMgbG9h ZGVkIGFsbCBSWCBxdWV1ZXMgaGF2ZSBYRFAgcHJvZ3JhbXMgc28gCndlIG9ubHkKIMKgwqDCoMKg wqDCoMKgwqAgKiBuZWVkIHRvIGNoZWNrIGEgc2luZ2xlIFJYIHF1ZXVlLgogwqDCoMKgwqDCoMKg wqDCoCAqLwogwqDCoMKgwqDCoMKgwqAgaWYgKHZpLT5ycVswXS54ZHBfcHJvZykKIMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCByZXR1cm4gLUVJTlZBTDsKCj4gKwo+ICsJc3EgPSAmdmkt PnNxW3FpZF07Cj4gKwo+ICsJcmN1X3JlYWRfbG9jaygpOwoKCkNhbiB3ZSBzaW1wbHkgdXNlIHJj dV9yZWFkX2xvY2tfYmgoKSBoZXJlPwoKCj4gKwlwb29sID0gcmN1X2RlcmVmZXJlbmNlKHNxLT54 c2sucG9vbCk7Cj4gKwlpZiAocG9vbCkgewo+ICsJCWxvY2FsX2JoX2Rpc2FibGUoKTsKPiArCQl2 aXJ0cXVldWVfbmFwaV9zY2hlZHVsZSgmc3EtPm5hcGksIHNxLT52cSk7Cj4gKwkJbG9jYWxfYmhf ZW5hYmxlKCk7Cj4gKwl9Cj4gKwlyY3VfcmVhZF91bmxvY2soKTsKPiArCXJldHVybiAwOwo+ICt9 Cj4gKwo+ICtzdGF0aWMgc3RydWN0IHZpcnRuZXRfeHNrX2N0eF9oZWFkICp2aXJ0bmV0X3hza19j dHhfYWxsb2Moc3RydWN0IHhza19idWZmX3Bvb2wgKnBvb2wsCj4gKwkJCQkJCQkgIHN0cnVjdCB2 aXJ0cXVldWUgKnZxKQo+ICt7Cj4gKwlzdHJ1Y3QgdmlydG5ldF94c2tfY3R4X2hlYWQgKmhlYWQ7 Cj4gKwl1MzIgc2l6ZSwgbiwgcmluZ19zaXplLCBjdHhfc3o7Cj4gKwlzdHJ1Y3QgdmlydG5ldF94 c2tfY3R4ICpjdHg7Cj4gKwl2b2lkICpwOwo+ICsKPiArCWN0eF9zeiA9IHNpemVvZihzdHJ1Y3Qg dmlydG5ldF94c2tfY3R4X3R4KTsKPiArCj4gKwlyaW5nX3NpemUgPSB2aXJ0cXVldWVfZ2V0X3Zy aW5nX3NpemUodnEpOwo+ICsJc2l6ZSA9IHNpemVvZigqaGVhZCkgKyBjdHhfc3ogKiByaW5nX3Np emU7Cj4gKwo+ICsJaGVhZCA9IGttYWxsb2Moc2l6ZSwgR0ZQX0FUT01JQyk7Cj4gKwlpZiAoIWhl YWQpCj4gKwkJcmV0dXJuIE5VTEw7Cj4gKwo+ICsJbWVtc2V0KGhlYWQsIDAsIHNpemVvZigqaGVh ZCkpOwo+ICsKPiArCWhlYWQtPmFjdGl2ZSA9IHRydWU7Cj4gKwloZWFkLT5mcmFtZV9zaXplID0g eHNrX3Bvb2xfZ2V0X3J4X2ZyYW1lX3NpemUocG9vbCk7Cj4gKwo+ICsJcCA9IGhlYWQgKyAxOwo+ ICsJZm9yIChuID0gMDsgbiA8IHJpbmdfc2l6ZTsgKytuKSB7Cj4gKwkJY3R4ID0gcDsKPiArCQlj dHgtPmhlYWQgPSBoZWFkOwo+ICsJCWN0eC0+bmV4dCA9IGhlYWQtPmN0eDsKPiArCQloZWFkLT5j dHggPSBjdHg7Cj4gKwo+ICsJCXAgKz0gY3R4X3N6Owo+ICsJfQo+ICsKPiArCXJldHVybiBoZWFk Owo+ICt9Cj4gKwo+ICtzdGF0aWMgaW50IHZpcnRuZXRfeHNrX3Bvb2xfZW5hYmxlKHN0cnVjdCBu ZXRfZGV2aWNlICpkZXYsCj4gKwkJCQkgICBzdHJ1Y3QgeHNrX2J1ZmZfcG9vbCAqcG9vbCwKPiAr CQkJCSAgIHUxNiBxaWQpCj4gK3sKPiArCXN0cnVjdCB2aXJ0bmV0X2luZm8gKnZpID0gbmV0ZGV2 X3ByaXYoZGV2KTsKPiArCXN0cnVjdCBzZW5kX3F1ZXVlICpzcTsKPiArCj4gKwlpZiAocWlkID49 IHZpLT5jdXJyX3F1ZXVlX3BhaXJzKQo+ICsJCXJldHVybiAtRUlOVkFMOwo+ICsKPiArCXNxID0g JnZpLT5zcVtxaWRdOwo+ICsKPiArCS8qIHhzayB6ZXJvY29weSBkZXBlbmQgb24gdGhlIHR4IG5h cGkuCj4gKwkgKgo+ICsJICogQWxsIGRhdGEgaXMgYWN0dWFsbHkgY29uc3VtZWQgYW5kIHNlbnQg b3V0IGZyb20gdGhlIHhzayB0eCBxdWV1ZQo+ICsJICogdW5kZXIgdGhlIHR4IG5hcGkgbWVjaGFu aXNtLgo+ICsJICovCj4gKwlpZiAoIXNxLT5uYXBpLndlaWdodCkKPiArCQlyZXR1cm4gLUVQRVJN Owo+ICsKPiArCW1lbXNldCgmc3EtPnhzaywgMCwgc2l6ZW9mKHNxLT54c2spKTsKPiArCj4gKwlz cS0+eHNrLmN0eF9oZWFkID0gdmlydG5ldF94c2tfY3R4X2FsbG9jKHBvb2wsIHNxLT52cSk7Cj4g KwlpZiAoIXNxLT54c2suY3R4X2hlYWQpCj4gKwkJcmV0dXJuIC1FTk9NRU07Cj4gKwo+ICsJLyog SGVyZSBpcyBhbHJlYWR5IHByb3RlY3RlZCBieSBydG5sX2xvY2ssIHNvIHJjdV9hc3NpZ25fcG9p bnRlciBpcwo+ICsJICogc2FmZS4KPiArCSAqLwo+ICsJcmN1X2Fzc2lnbl9wb2ludGVyKHNxLT54 c2sucG9vbCwgcG9vbCk7Cj4gKwo+ICsJcmV0dXJuIDA7Cj4gK30KPiArCj4gK3N0YXRpYyBpbnQg dmlydG5ldF94c2tfcG9vbF9kaXNhYmxlKHN0cnVjdCBuZXRfZGV2aWNlICpkZXYsIHUxNiBxaWQp Cj4gK3sKPiArCXN0cnVjdCB2aXJ0bmV0X2luZm8gKnZpID0gbmV0ZGV2X3ByaXYoZGV2KTsKPiAr CXN0cnVjdCBzZW5kX3F1ZXVlICpzcTsKPiArCj4gKwlpZiAocWlkID49IHZpLT5jdXJyX3F1ZXVl X3BhaXJzKQo+ICsJCXJldHVybiAtRUlOVkFMOwo+ICsKPiArCXNxID0gJnZpLT5zcVtxaWRdOwo+ ICsKPiArCS8qIEhlcmUgaXMgYWxyZWFkeSBwcm90ZWN0ZWQgYnkgcnRubF9sb2NrLCBzbyByY3Vf YXNzaWduX3BvaW50ZXIgaXMKPiArCSAqIHNhZmUuCj4gKwkgKi8KPiArCXJjdV9hc3NpZ25fcG9p bnRlcihzcS0+eHNrLnBvb2wsIE5VTEwpOwo+ICsKPiArCS8qIFN5bmMgd2l0aCB0aGUgWFNLIHdh a2V1cCBhbmQgd2l0aCBOQVBJLiAqLwo+ICsJc3luY2hyb25pemVfbmV0KCk7Cj4gKwo+ICsJaWYg KFJFQURfT05DRShzcS0+eHNrLmN0eF9oZWFkLT5yZWYpKQo+ICsJCVdSSVRFX09OQ0Uoc3EtPnhz ay5jdHhfaGVhZC0+YWN0aXZlLCBmYWxzZSk7Cj4gKwllbHNlCj4gKwkJa2ZyZWUoc3EtPnhzay5j dHhfaGVhZCk7Cj4gKwo+ICsJc3EtPnhzay5jdHhfaGVhZCA9IE5VTEw7Cj4gKwo+ICsJcmV0dXJu IDA7Cj4gK30KPiArCj4gK2ludCB2aXJ0bmV0X3hza19wb29sX3NldHVwKHN0cnVjdCBuZXRfZGV2 aWNlICpkZXYsIHN0cnVjdCBuZXRkZXZfYnBmICp4ZHApCj4gK3sKPiArCWlmICh4ZHAtPnhzay5w b29sKQo+ICsJCXJldHVybiB2aXJ0bmV0X3hza19wb29sX2VuYWJsZShkZXYsIHhkcC0+eHNrLnBv b2wsCj4gKwkJCQkJICAgICAgIHhkcC0+eHNrLnF1ZXVlX2lkKTsKPiArCWVsc2UKPiArCQlyZXR1 cm4gdmlydG5ldF94c2tfcG9vbF9kaXNhYmxlKGRldiwgeGRwLT54c2sucXVldWVfaWQpOwo+ICt9 Cj4gKwo+IGRpZmYgLS1naXQgYS9kcml2ZXJzL25ldC92aXJ0aW8veHNrLmggYi9kcml2ZXJzL25l dC92aXJ0aW8veHNrLmgKPiBuZXcgZmlsZSBtb2RlIDEwMDY0NAo+IGluZGV4IDAwMDAwMDAwMDAw MC4uNTQ5NDhlMGIwN2ZjCj4gLS0tIC9kZXYvbnVsbAo+ICsrKyBiL2RyaXZlcnMvbmV0L3ZpcnRp by94c2suaAo+IEBAIC0wLDAgKzEsOTkgQEAKPiArLyogU1BEWC1MaWNlbnNlLUlkZW50aWZpZXI6 IEdQTC0yLjAtb3ItbGF0ZXIgKi8KPiArCj4gKyNpZm5kZWYgX19YU0tfSF9fCj4gKyNkZWZpbmUg X19YU0tfSF9fCj4gKwo+ICsjZGVmaW5lIFZJUlRJT19YU0tfRkxBRwlCSVQoMSkKPiArCj4gKy8q IFdoZW4geHNrIGRpc2FibGUsIHVuZGVyIG5vcm1hbCBjaXJjdW1zdGFuY2VzLCB0aGUgbmV0d29y ayBjYXJkIG11c3QgcmVjbGFpbQo+ICsgKiBhbGwgdGhlIG1lbW9yeSB0aGF0IGhhcyBiZWVuIHNl bnQgYW5kIHRoZSBtZW1vcnkgYWRkZWQgdG8gdGhlIHJxIHF1ZXVlIGJ5Cj4gKyAqIGRlc3Ryb3lp bmcgdGhlIHF1ZXVlLgo+ICsgKgo+ICsgKiBCdXQgdmlydGlvJ3MgcXVldWUgZG9lcyBub3Qgc3Vw cG9ydCBzZXBhcmF0ZSBzZXR0aW5nIHRvIGJlZW4gZGlzYWJsZS4KCgpUaGlzIGlzIGEgY2FsbCBm b3IgdXMgdG8gaW1wbGVtZW50IHBlciBxdWV1ZSBlbmFibGUvZGlzYWJsZS4gVmlydGlvLW1taW8g CmhhcyBzdWNoIGZhY2lsaXR5IGJ1dCB2aXJ0aW8tcGNpIG9ubHkgYWxsb3cgdG8gZGlzYWJsZSBh IHF1ZXVlIChub3QgZW5hYmxlKS4KCgo+ICJSZXNldCIKPiArICogaXMgbm90IHZlcnkgc3VpdGFi bGUuCj4gKyAqCj4gKyAqIFRoZSBtZXRob2QgaGVyZSBpcyB0aGF0IGVhY2ggc2VudCBjaHVuayBv ciBjaHVuayBhZGRlZCB0byB0aGUgcnEgcXVldWUgaXMKPiArICogZGVzY3JpYmVkIGJ5IGFuIGlu ZGVwZW5kZW50IHN0cnVjdHVyZSBzdHJ1Y3QgdmlydG5ldF94c2tfY3R4Lgo+ICsgKgo+ICsgKiBX ZSB3aWxsIHVzZSBnZXRfcGFnZShwYWdlKSB0byByZWZlciB0byB0aGUgcGFnZSB3aGVyZSB0aGVz ZSBjaHVua3MgYXJlCj4gKyAqIGxvY2F0ZWQuIEFuZCB0aGVzZSBwYWdlcyB3aWxsIGJlIHJlY29y ZGVkIGluIHN0cnVjdCB2aXJ0bmV0X3hza19jdHguIFNvIHRoZXNlCj4gKyAqIGNodW5rcyBpbiB2 cSBhcmUgc2FmZS4gV2hlbiByZWN5Y2xpbmcsIHB1dCB0aGUgdGhlc2UgcGFnZS4KPiArICoKPiAr ICogVGhlc2Ugc3RydWN0dXJlcyBwb2ludCB0byBzdHJ1Y3QgdmlydG5ldF94c2tfY3R4X2hlYWQs IGFuZCByZWYgcmVjb3JkcyBob3cKPiArICogbWFueSBjaHVua3MgaGF2ZSBub3QgYmVlbiByZWNs YWltZWQuIElmIGFjdGl2ZSA9PSAwLCBpdCBtZWFucyB0aGF0IHhzayBoYXMKPiArICogYmVlbiBk aXNhYmxlZC4KPiArICoKPiArICogSW4gdGhpcyB3YXksIGV2ZW4gaWYgeHNrIGhhcyBiZWVuIHVu YnVuZGxlZCB3aXRoIHJxL3NxLCBvciBhIG5ldyB4c2sgYW5kCj4gKyAqIHJxL3NxICBhcmUgYm91 bmQsIGFuZCBhIG5ldyB2aXJ0bmV0X3hza19jdHhfaGVhZCBpcyBjcmVhdGVkLiBJdCB3aWxsIG5v dAo+ICsgKiBhZmZlY3QgdGhlIG9sZCB2aXJ0bmV0X3hza19jdHggdG8gYmUgcmVjeWNsZWQuIEFu ZCBmcmVlIGFsbCBoZWFkIGFuZCBjdHggd2hlbgo+ICsgKiByZWYgaXMgMC4KCgpUaGlzIGxvb2tz IGNvbXBsaWNhdGVkIGFuZCBpdCB3aWxsIGluY3JlYXNlIHRoZSBmb290cHJpbnQuIENvbnNpZGVy IHRoZSAKcGVyZm9ybWFuY2UgcGVuYWx0eSBhbmQgdGhlIGNvbXBsZXhpdHksIEkgd291bGQgc3Vn Z2VzdCB0byB1c2UgcmVzZXQgCmluc3RlYWQuCgpUaGVuIHdlIGRvbid0IG5lZWQgdG8gaW50cm9k dWNlIHN1Y2ggY29udGV4dC4KClRoYW5rcwoKCj4gKyAqLwo+ICtzdHJ1Y3QgdmlydG5ldF94c2tf Y3R4Owo+ICtzdHJ1Y3QgdmlydG5ldF94c2tfY3R4X2hlYWQgewo+ICsJc3RydWN0IHZpcnRuZXRf eHNrX2N0eCAqY3R4Owo+ICsKPiArCS8qIGhvdyBtYW55IGN0eCBoYXMgYmVlbiBhZGQgdG8gdnEg Ki8KPiArCXU2NCByZWY7Cj4gKwo+ICsJdW5zaWduZWQgaW50IGZyYW1lX3NpemU7Cj4gKwo+ICsJ LyogdGhlIHhzayBzdGF0dXMgKi8KPiArCWJvb2wgYWN0aXZlOwo+ICt9Owo+ICsKPiArc3RydWN0 IHZpcnRuZXRfeHNrX2N0eCB7Cj4gKwlzdHJ1Y3QgdmlydG5ldF94c2tfY3R4X2hlYWQgKmhlYWQ7 Cj4gKwlzdHJ1Y3QgdmlydG5ldF94c2tfY3R4ICpuZXh0Owo+ICsKPiArCXN0cnVjdCBwYWdlICpw YWdlOwo+ICsKPiArCS8qIHhzayB1bmFsaWduZWQgbW9kZSB3aWxsIHVzZSB0d28gcGFnZSBpbiBv bmUgZGVzYyAqLwo+ICsJc3RydWN0IHBhZ2UgKnBhZ2VfdW5hbGlnbmVkOwo+ICt9Owo+ICsKPiAr c3RydWN0IHZpcnRuZXRfeHNrX2N0eF90eCB7Cj4gKwkvKiB0aGlzICpNVVNUKiBiZSB0aGUgZmly c3QgKi8KPiArCXN0cnVjdCB2aXJ0bmV0X3hza19jdHggY3R4Owo+ICsKPiArCS8qIHhzayB0eCB4 bWl0IHVzZSB0aGlzIHJlY29yZCB0aGUgbGVuIG9mIHBhY2tldCAqLwo+ICsJdTMyIGxlbjsKPiAr fTsKPiArCj4gK3N0YXRpYyBpbmxpbmUgdm9pZCAqeHNrX3RvX3B0cihzdHJ1Y3QgdmlydG5ldF94 c2tfY3R4X3R4ICpjdHgpCj4gK3sKPiArCXJldHVybiAodm9pZCAqKSgodW5zaWduZWQgbG9uZylj dHggfCBWSVJUSU9fWFNLX0ZMQUcpOwo+ICt9Cj4gKwo+ICtzdGF0aWMgaW5saW5lIHN0cnVjdCB2 aXJ0bmV0X3hza19jdHhfdHggKnB0cl90b194c2sodm9pZCAqcHRyKQo+ICt7Cj4gKwl1bnNpZ25l ZCBsb25nIHA7Cj4gKwo+ICsJcCA9ICh1bnNpZ25lZCBsb25nKXB0cjsKPiArCXJldHVybiAoc3Ry dWN0IHZpcnRuZXRfeHNrX2N0eF90eCAqKShwICYgflZJUlRJT19YU0tfRkxBRyk7Cj4gK30KPiAr Cj4gK3N0YXRpYyBpbmxpbmUgdm9pZCB2aXJ0bmV0X3hza19jdHhfcHV0KHN0cnVjdCB2aXJ0bmV0 X3hza19jdHggKmN0eCkKPiArewo+ICsJcHV0X3BhZ2UoY3R4LT5wYWdlKTsKPiArCWlmIChjdHgt PnBhZ2VfdW5hbGlnbmVkKQo+ICsJCXB1dF9wYWdlKGN0eC0+cGFnZV91bmFsaWduZWQpOwo+ICsK PiArCS0tY3R4LT5oZWFkLT5yZWY7Cj4gKwo+ICsJaWYgKGN0eC0+aGVhZC0+YWN0aXZlKSB7Cj4g KwkJY3R4LT5uZXh0ID0gY3R4LT5oZWFkLT5jdHg7Cj4gKwkJY3R4LT5oZWFkLT5jdHggPSBjdHg7 Cj4gKwl9IGVsc2Ugewo+ICsJCWlmICghY3R4LT5oZWFkLT5yZWYpCj4gKwkJCWtmcmVlKGN0eC0+ aGVhZCk7Cj4gKwl9Cj4gK30KPiArCj4gKyNkZWZpbmUgdmlydG5ldF94c2tfY3R4X3R4X3B1dChj dHgpIFwKPiArCXZpcnRuZXRfeHNrX2N0eF9wdXQoKHN0cnVjdCB2aXJ0bmV0X3hza19jdHggKilj dHgpCj4gKwo+ICtpbnQgdmlydG5ldF94c2tfd2FrZXVwKHN0cnVjdCBuZXRfZGV2aWNlICpkZXYs IHUzMiBxaWQsIHUzMiBmbGFnKTsKPiAraW50IHZpcnRuZXRfcG9sbF94c2soc3RydWN0IHNlbmRf cXVldWUgKnNxLCBpbnQgYnVkZ2V0KTsKPiArdm9pZCB2aXJ0bmV0X3hza19jb21wbGV0ZShzdHJ1 Y3Qgc2VuZF9xdWV1ZSAqc3EsIHUzMiBudW0pOwo+ICtpbnQgdmlydG5ldF94c2tfcG9vbF9zZXR1 cChzdHJ1Y3QgbmV0X2RldmljZSAqZGV2LCBzdHJ1Y3QgbmV0ZGV2X2JwZiAqeGRwKTsKPiArI2Vu ZGlmCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpWaXJ0 dWFsaXphdGlvbiBtYWlsaW5nIGxpc3QKVmlydHVhbGl6YXRpb25AbGlzdHMubGludXgtZm91bmRh dGlvbi5vcmcKaHR0cHM6Ly9saXN0cy5saW51eGZvdW5kYXRpb24ub3JnL21haWxtYW4vbGlzdGlu Zm8vdmlydHVhbGl6YXRpb24=