From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754079AbbIGPkZ (ORCPT ); Mon, 7 Sep 2015 11:40:25 -0400 Received: from smtp02.citrix.com ([66.165.176.63]:31214 "EHLO SMTP02.CITRIX.COM" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752632AbbIGPip (ORCPT ); Mon, 7 Sep 2015 11:38:45 -0400 X-IronPort-AV: E=Sophos;i="5.17,485,1437436800"; d="scan'208";a="301753528" From: Julien Grall To: CC: , , , , "Julien Grall" , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Konrad Rzeszutek Wilk , "Boris Ostrovsky" , David Vrabel Subject: [PATCH v4 15/20] block/xen-blkfront: Make it running on 64KB page granularity Date: Mon, 7 Sep 2015 16:33:53 +0100 Message-ID: <1441640038-23615-16-git-send-email-julien.grall@citrix.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> References: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-DLP: MIA1 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The PV block protocol is using 4KB page granularity. The goal of this patch is to allow a Linux using 64KB page granularity using block device on a non-modified Xen. The block API is using segment which should at least be the size of a Linux page. Therefore, the driver will have to break the page in chunk of 4K before giving the page to the backend. When breaking a 64KB segment in 4KB chunks, it is possible that some chunks are empty. As the PV protocol always require to have data in the chunk, we have to count the number of Xen page which will be in use and avoid sending empty chunks. Note that, a pre-defined number of grants are reserved before preparing the request. This pre-defined number is based on the number and the maximum size of the segments. If each segment contains a very small amount of data, the driver may reserve too many grants (16 grants is reserved per segment with 64KB page granularity). Furthermore, in the case of persistent grants we allocate one Linux page per grant although only the first 4KB of the page will be effectively in use. This could be improved by sharing the page with multiple grants. Signed-off-by: Julien Grall Acked-by: Roger Pau Monné --- Cc: Konrad Rzeszutek Wilk Cc: Boris Ostrovsky Cc: David Vrabel Improvement such as support 64KB grant is not taken into consideration in this patch because we have the requirement to run a Linux using 64KB page on a non-modified Xen. Changes in v4: - Rebase after d50babbe300eedf33ea5b00a12c5df3a05bd96c7 " xen-blkfront: introduce blkfront_gather_backend_features()" - Fix typoes - Add Roger's acked-by Changes in v3: - Use DIV_ROUND_UP in INDIRECT_GREFS - Split lines over 80 characters whenever it's possible - s/mfn/gfn/ based on the new naming - The grant callback doesn't allow anymore to change the len (wasn't used here). - gnttab_foreach_grant has been renamed to gnttab_foreach_grant_in_range - Use gnttab_count_grant to get the number of grants in a sg - Do some renaming to use the correct variable every time Changes in v2: - Use gnttab_foreach_grant to split a Linux page into grant --- drivers/block/xen-blkfront.c | 324 ++++++++++++++++++++++++++++--------------- 1 file changed, 213 insertions(+), 111 deletions(-) diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index 4232cbd..f2cdc73 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -78,6 +78,7 @@ struct blk_shadow { struct grant **grants_used; struct grant **indirect_grants; struct scatterlist *sg; + unsigned int num_sg; }; struct split_bio { @@ -107,8 +108,12 @@ static unsigned int xen_blkif_max_ring_order; module_param_named(max_ring_page_order, xen_blkif_max_ring_order, int, S_IRUGO); MODULE_PARM_DESC(max_ring_page_order, "Maximum order of pages to be used for the shared ring"); -#define BLK_RING_SIZE(info) __CONST_RING_SIZE(blkif, PAGE_SIZE * (info)->nr_ring_pages) -#define BLK_MAX_RING_SIZE __CONST_RING_SIZE(blkif, PAGE_SIZE * XENBUS_MAX_RING_PAGES) +#define BLK_RING_SIZE(info) \ + __CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * (info)->nr_ring_pages) + +#define BLK_MAX_RING_SIZE \ + __CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * XENBUS_MAX_RING_PAGES) + /* * ring-ref%i i=(-1UL) would take 11 characters + 'ring-ref' is 8, so 19 * characters are enough. Define to 20 to keep consist with backend. @@ -147,6 +152,7 @@ struct blkfront_info unsigned int discard_granularity; unsigned int discard_alignment; unsigned int feature_persistent:1; + /* Number of 4KB segments handled */ unsigned int max_indirect_segments; int is_ready; struct blk_mq_tag_set tag_set; @@ -175,10 +181,23 @@ static DEFINE_SPINLOCK(minor_lock); #define DEV_NAME "xvd" /* name in /dev */ -#define SEGS_PER_INDIRECT_FRAME \ - (PAGE_SIZE/sizeof(struct blkif_request_segment)) -#define INDIRECT_GREFS(_segs) \ - ((_segs + SEGS_PER_INDIRECT_FRAME - 1)/SEGS_PER_INDIRECT_FRAME) +/* + * Grants are always the same size as a Xen page (i.e 4KB). + * A physical segment is always the same size as a Linux page. + * Number of grants per physical segment + */ +#define GRANTS_PER_PSEG (PAGE_SIZE / XEN_PAGE_SIZE) + +#define GRANTS_PER_INDIRECT_FRAME \ + (XEN_PAGE_SIZE / sizeof(struct blkif_request_segment)) + +#define PSEGS_PER_INDIRECT_FRAME \ + (GRANTS_INDIRECT_FRAME / GRANTS_PSEGS) + +#define INDIRECT_GREFS(_grants) \ + DIV_ROUND_UP(_grants, GRANTS_PER_INDIRECT_FRAME) + +#define GREFS(_psegs) ((_psegs) * GRANTS_PER_PSEG) static int blkfront_setup_indirect(struct blkfront_info *info); static int blkfront_gather_backend_features(struct blkfront_info *info); @@ -466,14 +485,100 @@ static int blkif_queue_discard_req(struct request *req) return 0; } +struct setup_rw_req { + unsigned int grant_idx; + struct blkif_request_segment *segments; + struct blkfront_info *info; + struct blkif_request *ring_req; + grant_ref_t gref_head; + unsigned int id; + /* Only used when persistent grant is used and it's a read request */ + bool need_copy; + unsigned int bvec_off; + char *bvec_data; +}; + +static void blkif_setup_rw_req_grant(unsigned long gfn, unsigned int offset, + unsigned int len, void *data) +{ + struct setup_rw_req *setup = data; + int n, ref; + struct grant *gnt_list_entry; + unsigned int fsect, lsect; + /* Convenient aliases */ + unsigned int grant_idx = setup->grant_idx; + struct blkif_request *ring_req = setup->ring_req; + struct blkfront_info *info = setup->info; + struct blk_shadow *shadow = &info->shadow[setup->id]; + + if ((ring_req->operation == BLKIF_OP_INDIRECT) && + (grant_idx % GRANTS_PER_INDIRECT_FRAME == 0)) { + if (setup->segments) + kunmap_atomic(setup->segments); + + n = grant_idx / GRANTS_PER_INDIRECT_FRAME; + gnt_list_entry = get_indirect_grant(&setup->gref_head, info); + shadow->indirect_grants[n] = gnt_list_entry; + setup->segments = kmap_atomic(gnt_list_entry->page); + ring_req->u.indirect.indirect_grefs[n] = gnt_list_entry->gref; + } + + gnt_list_entry = get_grant(&setup->gref_head, gfn, info); + ref = gnt_list_entry->gref; + shadow->grants_used[grant_idx] = gnt_list_entry; + + if (setup->need_copy) { + void *shared_data; + + shared_data = kmap_atomic(gnt_list_entry->page); + /* + * this does not wipe data stored outside the + * range sg->offset..sg->offset+sg->length. + * Therefore, blkback *could* see data from + * previous requests. This is OK as long as + * persistent grants are shared with just one + * domain. It may need refactoring if this + * changes + */ + memcpy(shared_data + offset, + setup->bvec_data + setup->bvec_off, + len); + + kunmap_atomic(shared_data); + setup->bvec_off += len; + } + + fsect = offset >> 9; + lsect = fsect + (len >> 9) - 1; + if (ring_req->operation != BLKIF_OP_INDIRECT) { + ring_req->u.rw.seg[grant_idx] = + (struct blkif_request_segment) { + .gref = ref, + .first_sect = fsect, + .last_sect = lsect }; + } else { + setup->segments[grant_idx % GRANTS_PER_INDIRECT_FRAME] = + (struct blkif_request_segment) { + .gref = ref, + .first_sect = fsect, + .last_sect = lsect }; + } + + (setup->grant_idx)++; +} + static int blkif_queue_rw_req(struct request *req) { struct blkfront_info *info = req->rq_disk->private_data; struct blkif_request *ring_req; unsigned long id; - unsigned int fsect, lsect; - int i, ref, n; - struct blkif_request_segment *segments = NULL; + int i; + struct setup_rw_req setup = { + .grant_idx = 0, + .segments = NULL, + .info = info, + .need_copy = rq_data_dir(req) && info->feature_persistent, + }; /* * Used to store if we are able to queue the request by just using @@ -481,25 +586,23 @@ static int blkif_queue_rw_req(struct request *req) * as there are not sufficiently many free. */ bool new_persistent_gnts; - grant_ref_t gref_head; - struct grant *gnt_list_entry = NULL; struct scatterlist *sg; - int nseg, max_grefs; + int num_sg, max_grefs, num_grant; - max_grefs = req->nr_phys_segments; + max_grefs = req->nr_phys_segments * GRANTS_PER_PSEG; if (max_grefs > BLKIF_MAX_SEGMENTS_PER_REQUEST) /* * If we are using indirect segments we need to account * for the indirect grefs used in the request. */ - max_grefs += INDIRECT_GREFS(req->nr_phys_segments); + max_grefs += INDIRECT_GREFS(max_grefs); /* Check if we have enough grants to allocate a requests */ if (info->persistent_gnts_c < max_grefs) { new_persistent_gnts = 1; if (gnttab_alloc_grant_references( max_grefs - info->persistent_gnts_c, - &gref_head) < 0) { + &setup.gref_head) < 0) { gnttab_request_free_callback( &info->callback, blkif_restart_queue_callback, @@ -516,12 +619,19 @@ static int blkif_queue_rw_req(struct request *req) info->shadow[id].request = req; BUG_ON(info->max_indirect_segments == 0 && - req->nr_phys_segments > BLKIF_MAX_SEGMENTS_PER_REQUEST); + GREFS(req->nr_phys_segments) > BLKIF_MAX_SEGMENTS_PER_REQUEST); BUG_ON(info->max_indirect_segments && - req->nr_phys_segments > info->max_indirect_segments); - nseg = blk_rq_map_sg(req->q, req, info->shadow[id].sg); + GREFS(req->nr_phys_segments) > info->max_indirect_segments); + + num_sg = blk_rq_map_sg(req->q, req, info->shadow[id].sg); + num_grant = 0; + /* Calculate the number of grant used */ + for_each_sg(info->shadow[id].sg, sg, num_sg, i) + num_grant += gnttab_count_grant(sg->offset, sg->length); + ring_req->u.rw.id = id; - if (nseg > BLKIF_MAX_SEGMENTS_PER_REQUEST) { + info->shadow[id].num_sg = num_sg; + if (num_grant > BLKIF_MAX_SEGMENTS_PER_REQUEST) { /* * The indirect operation can only be a BLKIF_OP_READ or * BLKIF_OP_WRITE @@ -532,7 +642,7 @@ static int blkif_queue_rw_req(struct request *req) BLKIF_OP_WRITE : BLKIF_OP_READ; ring_req->u.indirect.sector_number = (blkif_sector_t)blk_rq_pos(req); ring_req->u.indirect.handle = info->handle; - ring_req->u.indirect.nr_segments = nseg; + ring_req->u.indirect.nr_segments = num_grant; } else { ring_req->u.rw.sector_number = (blkif_sector_t)blk_rq_pos(req); ring_req->u.rw.handle = info->handle; @@ -560,73 +670,30 @@ static int blkif_queue_rw_req(struct request *req) ring_req->operation = 0; } } - ring_req->u.rw.nr_segments = nseg; - } - for_each_sg(info->shadow[id].sg, sg, nseg, i) { - fsect = sg->offset >> 9; - lsect = fsect + (sg->length >> 9) - 1; - - if ((ring_req->operation == BLKIF_OP_INDIRECT) && - (i % SEGS_PER_INDIRECT_FRAME == 0)) { - if (segments) - kunmap_atomic(segments); - - n = i / SEGS_PER_INDIRECT_FRAME; - gnt_list_entry = get_indirect_grant(&gref_head, info); - info->shadow[id].indirect_grants[n] = gnt_list_entry; - segments = kmap_atomic(gnt_list_entry->page); - ring_req->u.indirect.indirect_grefs[n] = gnt_list_entry->gref; - } - - gnt_list_entry = get_grant(&gref_head, - xen_page_to_gfn(sg_page(sg)), - info); - ref = gnt_list_entry->gref; - - info->shadow[id].grants_used[i] = gnt_list_entry; - - if (rq_data_dir(req) && info->feature_persistent) { - char *bvec_data; - void *shared_data; + ring_req->u.rw.nr_segments = num_grant; + } - BUG_ON(sg->offset + sg->length > PAGE_SIZE); + setup.ring_req = ring_req; + setup.id = id; + for_each_sg(info->shadow[id].sg, sg, num_sg, i) { + BUG_ON(sg->offset + sg->length > PAGE_SIZE); - shared_data = kmap_atomic(gnt_list_entry->page); - bvec_data = kmap_atomic(sg_page(sg)); + if (setup.need_copy) { + setup.bvec_off = sg->offset; + setup.bvec_data = kmap_atomic(sg_page(sg)); + } - /* - * this does not wipe data stored outside the - * range sg->offset..sg->offset+sg->length. - * Therefore, blkback *could* see data from - * previous requests. This is OK as long as - * persistent grants are shared with just one - * domain. It may need refactoring if this - * changes - */ - memcpy(shared_data + sg->offset, - bvec_data + sg->offset, - sg->length); + gnttab_foreach_grant_in_range(sg_page(sg), + sg->offset, + sg->length, + blkif_setup_rw_req_grant, + &setup); - kunmap_atomic(bvec_data); - kunmap_atomic(shared_data); - } - if (ring_req->operation != BLKIF_OP_INDIRECT) { - ring_req->u.rw.seg[i] = - (struct blkif_request_segment) { - .gref = ref, - .first_sect = fsect, - .last_sect = lsect }; - } else { - n = i % SEGS_PER_INDIRECT_FRAME; - segments[n] = - (struct blkif_request_segment) { - .gref = ref, - .first_sect = fsect, - .last_sect = lsect }; - } + if (setup.need_copy) + kunmap_atomic(setup.bvec_data); } - if (segments) - kunmap_atomic(segments); + if (setup.segments) + kunmap_atomic(setup.segments); info->ring.req_prod_pvt++; @@ -634,7 +701,7 @@ static int blkif_queue_rw_req(struct request *req) info->shadow[id].req = *ring_req; if (new_persistent_gnts) - gnttab_free_grant_references(gref_head); + gnttab_free_grant_references(setup.gref_head); return 0; } @@ -751,14 +818,14 @@ static int xlvbd_init_blk_queue(struct gendisk *gd, u16 sector_size, /* Hard sector size and max sectors impersonate the equiv. hardware. */ blk_queue_logical_block_size(rq, sector_size); blk_queue_physical_block_size(rq, physical_sector_size); - blk_queue_max_hw_sectors(rq, (segments * PAGE_SIZE) / 512); + blk_queue_max_hw_sectors(rq, (segments * XEN_PAGE_SIZE) / 512); /* Each segment in a request is up to an aligned page in size. */ blk_queue_segment_boundary(rq, PAGE_SIZE - 1); blk_queue_max_segment_size(rq, PAGE_SIZE); /* Ensure a merged request will fit in a single I/O ring slot. */ - blk_queue_max_segments(rq, segments); + blk_queue_max_segments(rq, segments / GRANTS_PER_PSEG); /* Make sure buffer addresses are sector-aligned. */ blk_queue_dma_alignment(rq, 511); @@ -1117,32 +1184,65 @@ free_shadow: } +struct copy_from_grant { + const struct blk_shadow *s; + unsigned int grant_idx; + unsigned int bvec_offset; + char *bvec_data; +}; + +static void blkif_copy_from_grant(unsigned long gfn, unsigned int offset, + unsigned int len, void *data) +{ + struct copy_from_grant *info = data; + char *shared_data; + /* Convenient aliases */ + const struct blk_shadow *s = info->s; + + shared_data = kmap_atomic(s->grants_used[info->grant_idx]->page); + + memcpy(info->bvec_data + info->bvec_offset, + shared_data + offset, len); + + info->bvec_offset += len; + info->grant_idx++; + + kunmap_atomic(shared_data); +} + static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, struct blkif_response *bret) { int i = 0; struct scatterlist *sg; - char *bvec_data; - void *shared_data; - int nseg; + int num_sg, num_grant; + struct copy_from_grant data = { + .s = s, + .grant_idx = 0, + }; - nseg = s->req.operation == BLKIF_OP_INDIRECT ? + num_grant = s->req.operation == BLKIF_OP_INDIRECT ? s->req.u.indirect.nr_segments : s->req.u.rw.nr_segments; + num_sg = s->num_sg; if (bret->operation == BLKIF_OP_READ && info->feature_persistent) { - for_each_sg(s->sg, sg, nseg, i) { + for_each_sg(s->sg, sg, num_sg, i) { BUG_ON(sg->offset + sg->length > PAGE_SIZE); - shared_data = kmap_atomic(s->grants_used[i]->page); - bvec_data = kmap_atomic(sg_page(sg)); - memcpy(bvec_data + sg->offset, - shared_data + sg->offset, - sg->length); - kunmap_atomic(bvec_data); - kunmap_atomic(shared_data); + + data.bvec_offset = sg->offset; + data.bvec_data = kmap_atomic(sg_page(sg)); + + gnttab_foreach_grant_in_range(sg_page(sg), + sg->offset, + sg->length, + blkif_copy_from_grant, + &data); + + kunmap_atomic(data.bvec_data); } } /* Add the persistent grant into the list of free grants */ - for (i = 0; i < nseg; i++) { + for (i = 0; i < num_grant; i++) { if (gnttab_query_foreign_access(s->grants_used[i]->gref)) { /* * If the grant is still mapped by the backend (the @@ -1168,7 +1268,7 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, } } if (s->req.operation == BLKIF_OP_INDIRECT) { - for (i = 0; i < INDIRECT_GREFS(nseg); i++) { + for (i = 0; i < INDIRECT_GREFS(num_grant); i++) { if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) { if (!info->feature_persistent) pr_alert_ratelimited("backed has not unmapped grant: %u\n", @@ -1312,7 +1412,7 @@ static int setup_blkring(struct xenbus_device *dev, { struct blkif_sring *sring; int err, i; - unsigned long ring_size = info->nr_ring_pages * PAGE_SIZE; + unsigned long ring_size = info->nr_ring_pages * XEN_PAGE_SIZE; grant_ref_t gref[XENBUS_MAX_RING_PAGES]; for (i = 0; i < info->nr_ring_pages; i++) @@ -1643,8 +1743,8 @@ static int blkif_recover(struct blkfront_info *info) atomic_set(&split_bio->pending, pending); split_bio->bio = bio; for (i = 0; i < pending; i++) { - offset = (i * segs * PAGE_SIZE) >> 9; - size = min((unsigned int)(segs * PAGE_SIZE) >> 9, + offset = (i * segs * XEN_PAGE_SIZE) >> 9; + size = min((unsigned int)(segs * XEN_PAGE_SIZE) >> 9, (unsigned int)bio_sectors(bio) - offset); cloned_bio = bio_clone(bio, GFP_NOIO); BUG_ON(cloned_bio == NULL); @@ -1755,15 +1855,17 @@ static void blkfront_setup_discard(struct blkfront_info *info) static int blkfront_setup_indirect(struct blkfront_info *info) { - unsigned int segs; + unsigned int psegs, grants; int err, i; if (info->max_indirect_segments == 0) - segs = BLKIF_MAX_SEGMENTS_PER_REQUEST; + grants = BLKIF_MAX_SEGMENTS_PER_REQUEST; else - segs = info->max_indirect_segments; + grants = info->max_indirect_segments; + psegs = grants / GRANTS_PER_PSEG; - err = fill_grant_buffer(info, (segs + INDIRECT_GREFS(segs)) * BLK_RING_SIZE(info)); + err = fill_grant_buffer(info, + (grants + INDIRECT_GREFS(grants)) * BLK_RING_SIZE(info)); if (err) goto out_of_memory; @@ -1773,7 +1875,7 @@ static int blkfront_setup_indirect(struct blkfront_info *info) * grants, we need to allocate a set of pages that can be * used for mapping indirect grefs */ - int num = INDIRECT_GREFS(segs) * BLK_RING_SIZE(info); + int num = INDIRECT_GREFS(grants) * BLK_RING_SIZE(info); BUG_ON(!list_empty(&info->indirect_pages)); for (i = 0; i < num; i++) { @@ -1786,20 +1888,20 @@ static int blkfront_setup_indirect(struct blkfront_info *info) for (i = 0; i < BLK_RING_SIZE(info); i++) { info->shadow[i].grants_used = kzalloc( - sizeof(info->shadow[i].grants_used[0]) * segs, + sizeof(info->shadow[i].grants_used[0]) * grants, GFP_NOIO); - info->shadow[i].sg = kzalloc(sizeof(info->shadow[i].sg[0]) * segs, GFP_NOIO); + info->shadow[i].sg = kzalloc(sizeof(info->shadow[i].sg[0]) * psegs, GFP_NOIO); if (info->max_indirect_segments) info->shadow[i].indirect_grants = kzalloc( sizeof(info->shadow[i].indirect_grants[0]) * - INDIRECT_GREFS(segs), + INDIRECT_GREFS(grants), GFP_NOIO); if ((info->shadow[i].grants_used == NULL) || (info->shadow[i].sg == NULL) || (info->max_indirect_segments && (info->shadow[i].indirect_grants == NULL))) goto out_of_memory; - sg_init_table(info->shadow[i].sg, segs); + sg_init_table(info->shadow[i].sg, psegs); } -- 2.1.4 From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Grall Subject: [PATCH v4 15/20] block/xen-blkfront: Make it running on 64KB page granularity Date: Mon, 7 Sep 2015 16:33:53 +0100 Message-ID: <1441640038-23615-16-git-send-email-julien.grall@citrix.com> References: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Return-path: Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1ZYyVL-00025k-EB for xen-devel@lists.xenproject.org; Mon, 07 Sep 2015 15:38:47 +0000 In-Reply-To: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xenproject.org Cc: ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, linux-kernel@vger.kernel.org, Julien Grall , David Vrabel , Boris Ostrovsky , linux-arm-kernel@lists.infradead.org, =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= List-Id: xen-devel@lists.xenproject.org VGhlIFBWIGJsb2NrIHByb3RvY29sIGlzIHVzaW5nIDRLQiBwYWdlIGdyYW51bGFyaXR5LiBUaGUg Z29hbCBvZiB0aGlzCnBhdGNoIGlzIHRvIGFsbG93IGEgTGludXggdXNpbmcgNjRLQiBwYWdlIGdy YW51bGFyaXR5IHVzaW5nIGJsb2NrCmRldmljZSBvbiBhIG5vbi1tb2RpZmllZCBYZW4uCgpUaGUg YmxvY2sgQVBJIGlzIHVzaW5nIHNlZ21lbnQgd2hpY2ggc2hvdWxkIGF0IGxlYXN0IGJlIHRoZSBz aXplIG9mIGEKTGludXggcGFnZS4gVGhlcmVmb3JlLCB0aGUgZHJpdmVyIHdpbGwgaGF2ZSB0byBi cmVhayB0aGUgcGFnZSBpbiBjaHVuawpvZiA0SyBiZWZvcmUgZ2l2aW5nIHRoZSBwYWdlIHRvIHRo ZSBiYWNrZW5kLgoKV2hlbiBicmVha2luZyBhIDY0S0Igc2VnbWVudCBpbiA0S0IgY2h1bmtzLCBp dCBpcyBwb3NzaWJsZSB0aGF0IHNvbWUKY2h1bmtzIGFyZSBlbXB0eS4gQXMgdGhlIFBWIHByb3Rv Y29sIGFsd2F5cyByZXF1aXJlIHRvIGhhdmUgZGF0YSBpbiB0aGUKY2h1bmssIHdlIGhhdmUgdG8g Y291bnQgdGhlIG51bWJlciBvZiBYZW4gcGFnZSB3aGljaCB3aWxsIGJlIGluIHVzZSBhbmQKYXZv aWQgc2VuZGluZyBlbXB0eSBjaHVua3MuCgpOb3RlIHRoYXQsIGEgcHJlLWRlZmluZWQgbnVtYmVy IG9mIGdyYW50cyBhcmUgcmVzZXJ2ZWQgYmVmb3JlIHByZXBhcmluZwp0aGUgcmVxdWVzdC4gVGhp cyBwcmUtZGVmaW5lZCBudW1iZXIgaXMgYmFzZWQgb24gdGhlIG51bWJlciBhbmQgdGhlCm1heGlt dW0gc2l6ZSBvZiB0aGUgc2VnbWVudHMuIElmIGVhY2ggc2VnbWVudCBjb250YWlucyBhIHZlcnkg c21hbGwKYW1vdW50IG9mIGRhdGEsIHRoZSBkcml2ZXIgbWF5IHJlc2VydmUgdG9vIG1hbnkgZ3Jh bnRzICgxNiBncmFudHMgaXMKcmVzZXJ2ZWQgcGVyIHNlZ21lbnQgd2l0aCA2NEtCIHBhZ2UgZ3Jh bnVsYXJpdHkpLgoKRnVydGhlcm1vcmUsIGluIHRoZSBjYXNlIG9mIHBlcnNpc3RlbnQgZ3JhbnRz IHdlIGFsbG9jYXRlIG9uZSBMaW51eCBwYWdlCnBlciBncmFudCBhbHRob3VnaCBvbmx5IHRoZSBm aXJzdCA0S0Igb2YgdGhlIHBhZ2Ugd2lsbCBiZSBlZmZlY3RpdmVseQppbiB1c2UuIFRoaXMgY291 bGQgYmUgaW1wcm92ZWQgYnkgc2hhcmluZyB0aGUgcGFnZSB3aXRoIG11bHRpcGxlIGdyYW50cy4K ClNpZ25lZC1vZmYtYnk6IEp1bGllbiBHcmFsbCA8anVsaWVuLmdyYWxsQGNpdHJpeC5jb20+CkFj a2VkLWJ5OiBSb2dlciBQYXUgTW9ubsOpIDxyb2dlci5wYXVAY2l0cml4LmNvbT4KCi0tLQpDYzog S29ucmFkIFJ6ZXN6dXRlayBXaWxrIDxrb25yYWQud2lsa0BvcmFjbGUuY29tPgpDYzogQm9yaXMg T3N0cm92c2t5IDxib3Jpcy5vc3Ryb3Zza3lAb3JhY2xlLmNvbT4KQ2M6IERhdmlkIFZyYWJlbCA8 ZGF2aWQudnJhYmVsQGNpdHJpeC5jb20+CgpJbXByb3ZlbWVudCBzdWNoIGFzIHN1cHBvcnQgNjRL QiBncmFudCBpcyBub3QgdGFrZW4gaW50byBjb25zaWRlcmF0aW9uIGluCnRoaXMgcGF0Y2ggYmVj YXVzZSB3ZSBoYXZlIHRoZSByZXF1aXJlbWVudCB0byBydW4gYSBMaW51eCB1c2luZyA2NEtCIHBh Z2UKb24gYSBub24tbW9kaWZpZWQgWGVuLgoKICAgIENoYW5nZXMgaW4gdjQ6CiAgICAgICAgLSBS ZWJhc2UgYWZ0ZXIgZDUwYmFiYmUzMDBlZWRmMzNlYTViMDBhMTJjNWRmM2EwNWJkOTZjNyAiCiAg ICAgICAgeGVuLWJsa2Zyb250OiBpbnRyb2R1Y2UgYmxrZnJvbnRfZ2F0aGVyX2JhY2tlbmRfZmVh dHVyZXMoKSIKICAgICAgICAtIEZpeCB0eXBvZXMKICAgICAgICAtIEFkZCBSb2dlcidzIGFja2Vk LWJ5CgogICAgQ2hhbmdlcyBpbiB2MzoKICAgICAgICAtIFVzZSBESVZfUk9VTkRfVVAgaW4gSU5E SVJFQ1RfR1JFRlMKICAgICAgICAtIFNwbGl0IGxpbmVzIG92ZXIgODAgY2hhcmFjdGVycyB3aGVu ZXZlciBpdCdzIHBvc3NpYmxlCiAgICAgICAgLSBzL21mbi9nZm4vIGJhc2VkIG9uIHRoZSBuZXcg bmFtaW5nCiAgICAgICAgLSBUaGUgZ3JhbnQgY2FsbGJhY2sgZG9lc24ndCBhbGxvdyBhbnltb3Jl IHRvIGNoYW5nZSB0aGUgbGVuCiAgICAgICAgKHdhc24ndCB1c2VkIGhlcmUpLgogICAgICAgIC0g Z250dGFiX2ZvcmVhY2hfZ3JhbnQgaGFzIGJlZW4gcmVuYW1lZCB0byBnbnR0YWJfZm9yZWFjaF9n cmFudF9pbl9yYW5nZQogICAgICAgIC0gVXNlIGdudHRhYl9jb3VudF9ncmFudCB0byBnZXQgdGhl IG51bWJlciBvZiBncmFudHMgaW4gYSBzZwogICAgICAgIC0gRG8gc29tZSByZW5hbWluZyB0byB1 c2UgdGhlIGNvcnJlY3QgdmFyaWFibGUgZXZlcnkgdGltZQoKICAgIENoYW5nZXMgaW4gdjI6CiAg ICAgICAgLSBVc2UgZ250dGFiX2ZvcmVhY2hfZ3JhbnQgdG8gc3BsaXQgYSBMaW51eCBwYWdlIGlu dG8gZ3JhbnQKLS0tCiBkcml2ZXJzL2Jsb2NrL3hlbi1ibGtmcm9udC5jIHwgMzI0ICsrKysrKysr KysrKysrKysrKysrKysrKysrKystLS0tLS0tLS0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAyMTMg aW5zZXJ0aW9ucygrKSwgMTExIGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2RyaXZlcnMvYmxv Y2sveGVuLWJsa2Zyb250LmMgYi9kcml2ZXJzL2Jsb2NrL3hlbi1ibGtmcm9udC5jCmluZGV4IDQy MzJjYmQuLmYyY2RjNzMgMTAwNjQ0Ci0tLSBhL2RyaXZlcnMvYmxvY2sveGVuLWJsa2Zyb250LmMK KysrIGIvZHJpdmVycy9ibG9jay94ZW4tYmxrZnJvbnQuYwpAQCAtNzgsNiArNzgsNyBAQCBzdHJ1 Y3QgYmxrX3NoYWRvdyB7CiAJc3RydWN0IGdyYW50ICoqZ3JhbnRzX3VzZWQ7CiAJc3RydWN0IGdy YW50ICoqaW5kaXJlY3RfZ3JhbnRzOwogCXN0cnVjdCBzY2F0dGVybGlzdCAqc2c7CisJdW5zaWdu ZWQgaW50IG51bV9zZzsKIH07CiAKIHN0cnVjdCBzcGxpdF9iaW8gewpAQCAtMTA3LDggKzEwOCwx MiBAQCBzdGF0aWMgdW5zaWduZWQgaW50IHhlbl9ibGtpZl9tYXhfcmluZ19vcmRlcjsKIG1vZHVs ZV9wYXJhbV9uYW1lZChtYXhfcmluZ19wYWdlX29yZGVyLCB4ZW5fYmxraWZfbWF4X3Jpbmdfb3Jk ZXIsIGludCwgU19JUlVHTyk7CiBNT0RVTEVfUEFSTV9ERVNDKG1heF9yaW5nX3BhZ2Vfb3JkZXIs ICJNYXhpbXVtIG9yZGVyIG9mIHBhZ2VzIHRvIGJlIHVzZWQgZm9yIHRoZSBzaGFyZWQgcmluZyIp OwogCi0jZGVmaW5lIEJMS19SSU5HX1NJWkUoaW5mbykgX19DT05TVF9SSU5HX1NJWkUoYmxraWYs IFBBR0VfU0laRSAqIChpbmZvKS0+bnJfcmluZ19wYWdlcykKLSNkZWZpbmUgQkxLX01BWF9SSU5H X1NJWkUgX19DT05TVF9SSU5HX1NJWkUoYmxraWYsIFBBR0VfU0laRSAqIFhFTkJVU19NQVhfUklO R19QQUdFUykKKyNkZWZpbmUgQkxLX1JJTkdfU0laRShpbmZvKQlcCisJX19DT05TVF9SSU5HX1NJ WkUoYmxraWYsIFhFTl9QQUdFX1NJWkUgKiAoaW5mbyktPm5yX3JpbmdfcGFnZXMpCisKKyNkZWZp bmUgQkxLX01BWF9SSU5HX1NJWkUJXAorCV9fQ09OU1RfUklOR19TSVpFKGJsa2lmLCBYRU5fUEFH RV9TSVpFICogWEVOQlVTX01BWF9SSU5HX1BBR0VTKQorCiAvKgogICogcmluZy1yZWYlaSBpPSgt MVVMKSB3b3VsZCB0YWtlIDExIGNoYXJhY3RlcnMgKyAncmluZy1yZWYnIGlzIDgsIHNvIDE5CiAg KiBjaGFyYWN0ZXJzIGFyZSBlbm91Z2guIERlZmluZSB0byAyMCB0byBrZWVwIGNvbnNpc3Qgd2l0 aCBiYWNrZW5kLgpAQCAtMTQ3LDYgKzE1Miw3IEBAIHN0cnVjdCBibGtmcm9udF9pbmZvCiAJdW5z aWduZWQgaW50IGRpc2NhcmRfZ3JhbnVsYXJpdHk7CiAJdW5zaWduZWQgaW50IGRpc2NhcmRfYWxp Z25tZW50OwogCXVuc2lnbmVkIGludCBmZWF0dXJlX3BlcnNpc3RlbnQ6MTsKKwkvKiBOdW1iZXIg b2YgNEtCIHNlZ21lbnRzIGhhbmRsZWQgKi8KIAl1bnNpZ25lZCBpbnQgbWF4X2luZGlyZWN0X3Nl Z21lbnRzOwogCWludCBpc19yZWFkeTsKIAlzdHJ1Y3QgYmxrX21xX3RhZ19zZXQgdGFnX3NldDsK QEAgLTE3NSwxMCArMTgxLDIzIEBAIHN0YXRpYyBERUZJTkVfU1BJTkxPQ0sobWlub3JfbG9jayk7 CiAKICNkZWZpbmUgREVWX05BTUUJInh2ZCIJLyogbmFtZSBpbiAvZGV2ICovCiAKLSNkZWZpbmUg U0VHU19QRVJfSU5ESVJFQ1RfRlJBTUUgXAotCShQQUdFX1NJWkUvc2l6ZW9mKHN0cnVjdCBibGtp Zl9yZXF1ZXN0X3NlZ21lbnQpKQotI2RlZmluZSBJTkRJUkVDVF9HUkVGUyhfc2VncykgXAotCSgo X3NlZ3MgKyBTRUdTX1BFUl9JTkRJUkVDVF9GUkFNRSAtIDEpL1NFR1NfUEVSX0lORElSRUNUX0ZS QU1FKQorLyoKKyAqIEdyYW50cyBhcmUgYWx3YXlzIHRoZSBzYW1lIHNpemUgYXMgYSBYZW4gcGFn ZSAoaS5lIDRLQikuCisgKiBBIHBoeXNpY2FsIHNlZ21lbnQgaXMgYWx3YXlzIHRoZSBzYW1lIHNp emUgYXMgYSBMaW51eCBwYWdlLgorICogTnVtYmVyIG9mIGdyYW50cyBwZXIgcGh5c2ljYWwgc2Vn bWVudAorICovCisjZGVmaW5lIEdSQU5UU19QRVJfUFNFRwkoUEFHRV9TSVpFIC8gWEVOX1BBR0Vf U0laRSkKKworI2RlZmluZSBHUkFOVFNfUEVSX0lORElSRUNUX0ZSQU1FIFwKKwkoWEVOX1BBR0Vf U0laRSAvIHNpemVvZihzdHJ1Y3QgYmxraWZfcmVxdWVzdF9zZWdtZW50KSkKKworI2RlZmluZSBQ U0VHU19QRVJfSU5ESVJFQ1RfRlJBTUUJXAorCShHUkFOVFNfSU5ESVJFQ1RfRlJBTUUgLyBHUkFO VFNfUFNFR1MpCisKKyNkZWZpbmUgSU5ESVJFQ1RfR1JFRlMoX2dyYW50cykJCVwKKwlESVZfUk9V TkRfVVAoX2dyYW50cywgR1JBTlRTX1BFUl9JTkRJUkVDVF9GUkFNRSkKKworI2RlZmluZSBHUkVG UyhfcHNlZ3MpCSgoX3BzZWdzKSAqIEdSQU5UU19QRVJfUFNFRykKIAogc3RhdGljIGludCBibGtm cm9udF9zZXR1cF9pbmRpcmVjdChzdHJ1Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbyk7CiBzdGF0aWMg aW50IGJsa2Zyb250X2dhdGhlcl9iYWNrZW5kX2ZlYXR1cmVzKHN0cnVjdCBibGtmcm9udF9pbmZv ICppbmZvKTsKQEAgLTQ2NiwxNCArNDg1LDEwMCBAQCBzdGF0aWMgaW50IGJsa2lmX3F1ZXVlX2Rp c2NhcmRfcmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEpCiAJcmV0dXJuIDA7CiB9CiAKK3N0cnVjdCBz ZXR1cF9yd19yZXEgeworCXVuc2lnbmVkIGludCBncmFudF9pZHg7CisJc3RydWN0IGJsa2lmX3Jl cXVlc3Rfc2VnbWVudCAqc2VnbWVudHM7CisJc3RydWN0IGJsa2Zyb250X2luZm8gKmluZm87CisJ c3RydWN0IGJsa2lmX3JlcXVlc3QgKnJpbmdfcmVxOworCWdyYW50X3JlZl90IGdyZWZfaGVhZDsK Kwl1bnNpZ25lZCBpbnQgaWQ7CisJLyogT25seSB1c2VkIHdoZW4gcGVyc2lzdGVudCBncmFudCBp cyB1c2VkIGFuZCBpdCdzIGEgcmVhZCByZXF1ZXN0ICovCisJYm9vbCBuZWVkX2NvcHk7CisJdW5z aWduZWQgaW50IGJ2ZWNfb2ZmOworCWNoYXIgKmJ2ZWNfZGF0YTsKK307CisKK3N0YXRpYyB2b2lk IGJsa2lmX3NldHVwX3J3X3JlcV9ncmFudCh1bnNpZ25lZCBsb25nIGdmbiwgdW5zaWduZWQgaW50 IG9mZnNldCwKKwkJCQkgICAgIHVuc2lnbmVkIGludCBsZW4sIHZvaWQgKmRhdGEpCit7CisJc3Ry dWN0IHNldHVwX3J3X3JlcSAqc2V0dXAgPSBkYXRhOworCWludCBuLCByZWY7CisJc3RydWN0IGdy YW50ICpnbnRfbGlzdF9lbnRyeTsKKwl1bnNpZ25lZCBpbnQgZnNlY3QsIGxzZWN0OworCS8qIENv bnZlbmllbnQgYWxpYXNlcyAqLworCXVuc2lnbmVkIGludCBncmFudF9pZHggPSBzZXR1cC0+Z3Jh bnRfaWR4OworCXN0cnVjdCBibGtpZl9yZXF1ZXN0ICpyaW5nX3JlcSA9IHNldHVwLT5yaW5nX3Jl cTsKKwlzdHJ1Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbyA9IHNldHVwLT5pbmZvOworCXN0cnVjdCBi bGtfc2hhZG93ICpzaGFkb3cgPSAmaW5mby0+c2hhZG93W3NldHVwLT5pZF07CisKKwlpZiAoKHJp bmdfcmVxLT5vcGVyYXRpb24gPT0gQkxLSUZfT1BfSU5ESVJFQ1QpICYmCisJICAgIChncmFudF9p ZHggJSBHUkFOVFNfUEVSX0lORElSRUNUX0ZSQU1FID09IDApKSB7CisJCWlmIChzZXR1cC0+c2Vn bWVudHMpCisJCQlrdW5tYXBfYXRvbWljKHNldHVwLT5zZWdtZW50cyk7CisKKwkJbiA9IGdyYW50 X2lkeCAvIEdSQU5UU19QRVJfSU5ESVJFQ1RfRlJBTUU7CisJCWdudF9saXN0X2VudHJ5ID0gZ2V0 X2luZGlyZWN0X2dyYW50KCZzZXR1cC0+Z3JlZl9oZWFkLCBpbmZvKTsKKwkJc2hhZG93LT5pbmRp cmVjdF9ncmFudHNbbl0gPSBnbnRfbGlzdF9lbnRyeTsKKwkJc2V0dXAtPnNlZ21lbnRzID0ga21h cF9hdG9taWMoZ250X2xpc3RfZW50cnktPnBhZ2UpOworCQlyaW5nX3JlcS0+dS5pbmRpcmVjdC5p bmRpcmVjdF9ncmVmc1tuXSA9IGdudF9saXN0X2VudHJ5LT5ncmVmOworCX0KKworCWdudF9saXN0 X2VudHJ5ID0gZ2V0X2dyYW50KCZzZXR1cC0+Z3JlZl9oZWFkLCBnZm4sIGluZm8pOworCXJlZiA9 IGdudF9saXN0X2VudHJ5LT5ncmVmOworCXNoYWRvdy0+Z3JhbnRzX3VzZWRbZ3JhbnRfaWR4XSA9 IGdudF9saXN0X2VudHJ5OworCisJaWYgKHNldHVwLT5uZWVkX2NvcHkpIHsKKwkJdm9pZCAqc2hh cmVkX2RhdGE7CisKKwkJc2hhcmVkX2RhdGEgPSBrbWFwX2F0b21pYyhnbnRfbGlzdF9lbnRyeS0+ cGFnZSk7CisJCS8qCisJCSAqIHRoaXMgZG9lcyBub3Qgd2lwZSBkYXRhIHN0b3JlZCBvdXRzaWRl IHRoZQorCQkgKiByYW5nZSBzZy0+b2Zmc2V0Li5zZy0+b2Zmc2V0K3NnLT5sZW5ndGguCisJCSAq IFRoZXJlZm9yZSwgYmxrYmFjayAqY291bGQqIHNlZSBkYXRhIGZyb20KKwkJICogcHJldmlvdXMg cmVxdWVzdHMuIFRoaXMgaXMgT0sgYXMgbG9uZyBhcworCQkgKiBwZXJzaXN0ZW50IGdyYW50cyBh cmUgc2hhcmVkIHdpdGgganVzdCBvbmUKKwkJICogZG9tYWluLiBJdCBtYXkgbmVlZCByZWZhY3Rv cmluZyBpZiB0aGlzCisJCSAqIGNoYW5nZXMKKwkJICovCisJCW1lbWNweShzaGFyZWRfZGF0YSAr IG9mZnNldCwKKwkJICAgICAgIHNldHVwLT5idmVjX2RhdGEgKyBzZXR1cC0+YnZlY19vZmYsCisJ CSAgICAgICBsZW4pOworCisJCWt1bm1hcF9hdG9taWMoc2hhcmVkX2RhdGEpOworCQlzZXR1cC0+ YnZlY19vZmYgKz0gbGVuOworCX0KKworCWZzZWN0ID0gb2Zmc2V0ID4+IDk7CisJbHNlY3QgPSBm c2VjdCArIChsZW4gPj4gOSkgLSAxOworCWlmIChyaW5nX3JlcS0+b3BlcmF0aW9uICE9IEJMS0lG X09QX0lORElSRUNUKSB7CisJCXJpbmdfcmVxLT51LnJ3LnNlZ1tncmFudF9pZHhdID0KKwkJCShz dHJ1Y3QgYmxraWZfcmVxdWVzdF9zZWdtZW50KSB7CisJCQkJLmdyZWYgICAgICAgPSByZWYsCisJ CQkJLmZpcnN0X3NlY3QgPSBmc2VjdCwKKwkJCQkubGFzdF9zZWN0ICA9IGxzZWN0IH07CisJfSBl bHNlIHsKKwkJc2V0dXAtPnNlZ21lbnRzW2dyYW50X2lkeCAlIEdSQU5UU19QRVJfSU5ESVJFQ1Rf RlJBTUVdID0KKwkJCShzdHJ1Y3QgYmxraWZfcmVxdWVzdF9zZWdtZW50KSB7CisJCQkJLmdyZWYg ICAgICAgPSByZWYsCisJCQkJLmZpcnN0X3NlY3QgPSBmc2VjdCwKKwkJCQkubGFzdF9zZWN0ICA9 IGxzZWN0IH07CisJfQorCisJKHNldHVwLT5ncmFudF9pZHgpKys7Cit9CisKIHN0YXRpYyBpbnQg YmxraWZfcXVldWVfcndfcmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEpCiB7CiAJc3RydWN0IGJsa2Zy b250X2luZm8gKmluZm8gPSByZXEtPnJxX2Rpc2stPnByaXZhdGVfZGF0YTsKIAlzdHJ1Y3QgYmxr aWZfcmVxdWVzdCAqcmluZ19yZXE7CiAJdW5zaWduZWQgbG9uZyBpZDsKLQl1bnNpZ25lZCBpbnQg ZnNlY3QsIGxzZWN0OwotCWludCBpLCByZWYsIG47Ci0Jc3RydWN0IGJsa2lmX3JlcXVlc3Rfc2Vn bWVudCAqc2VnbWVudHMgPSBOVUxMOworCWludCBpOworCXN0cnVjdCBzZXR1cF9yd19yZXEgc2V0 dXAgPSB7CisJCS5ncmFudF9pZHggPSAwLAorCQkuc2VnbWVudHMgPSBOVUxMLAorCQkuaW5mbyA9 IGluZm8sCisJCS5uZWVkX2NvcHkgPSBycV9kYXRhX2RpcihyZXEpICYmIGluZm8tPmZlYXR1cmVf cGVyc2lzdGVudCwKKwl9OwogCiAJLyoKIAkgKiBVc2VkIHRvIHN0b3JlIGlmIHdlIGFyZSBhYmxl IHRvIHF1ZXVlIHRoZSByZXF1ZXN0IGJ5IGp1c3QgdXNpbmcKQEAgLTQ4MSwyNSArNTg2LDIzIEBA IHN0YXRpYyBpbnQgYmxraWZfcXVldWVfcndfcmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEpCiAJICog YXMgdGhlcmUgYXJlIG5vdCBzdWZmaWNpZW50bHkgbWFueSBmcmVlLgogCSAqLwogCWJvb2wgbmV3 X3BlcnNpc3RlbnRfZ250czsKLQlncmFudF9yZWZfdCBncmVmX2hlYWQ7Ci0Jc3RydWN0IGdyYW50 ICpnbnRfbGlzdF9lbnRyeSA9IE5VTEw7CiAJc3RydWN0IHNjYXR0ZXJsaXN0ICpzZzsKLQlpbnQg bnNlZywgbWF4X2dyZWZzOworCWludCBudW1fc2csIG1heF9ncmVmcywgbnVtX2dyYW50OwogCi0J bWF4X2dyZWZzID0gcmVxLT5ucl9waHlzX3NlZ21lbnRzOworCW1heF9ncmVmcyA9IHJlcS0+bnJf cGh5c19zZWdtZW50cyAqIEdSQU5UU19QRVJfUFNFRzsKIAlpZiAobWF4X2dyZWZzID4gQkxLSUZf TUFYX1NFR01FTlRTX1BFUl9SRVFVRVNUKQogCQkvKgogCQkgKiBJZiB3ZSBhcmUgdXNpbmcgaW5k aXJlY3Qgc2VnbWVudHMgd2UgbmVlZCB0byBhY2NvdW50CiAJCSAqIGZvciB0aGUgaW5kaXJlY3Qg Z3JlZnMgdXNlZCBpbiB0aGUgcmVxdWVzdC4KIAkJICovCi0JCW1heF9ncmVmcyArPSBJTkRJUkVD VF9HUkVGUyhyZXEtPm5yX3BoeXNfc2VnbWVudHMpOworCQltYXhfZ3JlZnMgKz0gSU5ESVJFQ1Rf R1JFRlMobWF4X2dyZWZzKTsKIAogCS8qIENoZWNrIGlmIHdlIGhhdmUgZW5vdWdoIGdyYW50cyB0 byBhbGxvY2F0ZSBhIHJlcXVlc3RzICovCiAJaWYgKGluZm8tPnBlcnNpc3RlbnRfZ250c19jIDwg bWF4X2dyZWZzKSB7CiAJCW5ld19wZXJzaXN0ZW50X2dudHMgPSAxOwogCQlpZiAoZ250dGFiX2Fs bG9jX2dyYW50X3JlZmVyZW5jZXMoCiAJCSAgICBtYXhfZ3JlZnMgLSBpbmZvLT5wZXJzaXN0ZW50 X2dudHNfYywKLQkJICAgICZncmVmX2hlYWQpIDwgMCkgeworCQkgICAgJnNldHVwLmdyZWZfaGVh ZCkgPCAwKSB7CiAJCQlnbnR0YWJfcmVxdWVzdF9mcmVlX2NhbGxiYWNrKAogCQkJCSZpbmZvLT5j YWxsYmFjaywKIAkJCQlibGtpZl9yZXN0YXJ0X3F1ZXVlX2NhbGxiYWNrLApAQCAtNTE2LDEyICs2 MTksMTkgQEAgc3RhdGljIGludCBibGtpZl9xdWV1ZV9yd19yZXEoc3RydWN0IHJlcXVlc3QgKnJl cSkKIAlpbmZvLT5zaGFkb3dbaWRdLnJlcXVlc3QgPSByZXE7CiAKIAlCVUdfT04oaW5mby0+bWF4 X2luZGlyZWN0X3NlZ21lbnRzID09IDAgJiYKLQkgICAgICAgcmVxLT5ucl9waHlzX3NlZ21lbnRz ID4gQkxLSUZfTUFYX1NFR01FTlRTX1BFUl9SRVFVRVNUKTsKKwkgICAgICAgR1JFRlMocmVxLT5u cl9waHlzX3NlZ21lbnRzKSA+IEJMS0lGX01BWF9TRUdNRU5UU19QRVJfUkVRVUVTVCk7CiAJQlVH X09OKGluZm8tPm1heF9pbmRpcmVjdF9zZWdtZW50cyAmJgotCSAgICAgICByZXEtPm5yX3BoeXNf c2VnbWVudHMgPiBpbmZvLT5tYXhfaW5kaXJlY3Rfc2VnbWVudHMpOwotCW5zZWcgPSBibGtfcnFf bWFwX3NnKHJlcS0+cSwgcmVxLCBpbmZvLT5zaGFkb3dbaWRdLnNnKTsKKwkgICAgICAgR1JFRlMo cmVxLT5ucl9waHlzX3NlZ21lbnRzKSA+IGluZm8tPm1heF9pbmRpcmVjdF9zZWdtZW50cyk7CisK KwludW1fc2cgPSBibGtfcnFfbWFwX3NnKHJlcS0+cSwgcmVxLCBpbmZvLT5zaGFkb3dbaWRdLnNn KTsKKwludW1fZ3JhbnQgPSAwOworCS8qIENhbGN1bGF0ZSB0aGUgbnVtYmVyIG9mIGdyYW50IHVz ZWQgKi8KKwlmb3JfZWFjaF9zZyhpbmZvLT5zaGFkb3dbaWRdLnNnLCBzZywgbnVtX3NnLCBpKQor CSAgICAgICBudW1fZ3JhbnQgKz0gZ250dGFiX2NvdW50X2dyYW50KHNnLT5vZmZzZXQsIHNnLT5s ZW5ndGgpOworCiAJcmluZ19yZXEtPnUucncuaWQgPSBpZDsKLQlpZiAobnNlZyA+IEJMS0lGX01B WF9TRUdNRU5UU19QRVJfUkVRVUVTVCkgeworCWluZm8tPnNoYWRvd1tpZF0ubnVtX3NnID0gbnVt X3NnOworCWlmIChudW1fZ3JhbnQgPiBCTEtJRl9NQVhfU0VHTUVOVFNfUEVSX1JFUVVFU1QpIHsK IAkJLyoKIAkJICogVGhlIGluZGlyZWN0IG9wZXJhdGlvbiBjYW4gb25seSBiZSBhIEJMS0lGX09Q X1JFQUQgb3IKIAkJICogQkxLSUZfT1BfV1JJVEUKQEAgLTUzMiw3ICs2NDIsNyBAQCBzdGF0aWMg aW50IGJsa2lmX3F1ZXVlX3J3X3JlcShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQogCQkJQkxLSUZfT1Bf V1JJVEUgOiBCTEtJRl9PUF9SRUFEOwogCQlyaW5nX3JlcS0+dS5pbmRpcmVjdC5zZWN0b3JfbnVt YmVyID0gKGJsa2lmX3NlY3Rvcl90KWJsa19ycV9wb3MocmVxKTsKIAkJcmluZ19yZXEtPnUuaW5k aXJlY3QuaGFuZGxlID0gaW5mby0+aGFuZGxlOwotCQlyaW5nX3JlcS0+dS5pbmRpcmVjdC5ucl9z ZWdtZW50cyA9IG5zZWc7CisJCXJpbmdfcmVxLT51LmluZGlyZWN0Lm5yX3NlZ21lbnRzID0gbnVt X2dyYW50OwogCX0gZWxzZSB7CiAJCXJpbmdfcmVxLT51LnJ3LnNlY3Rvcl9udW1iZXIgPSAoYmxr aWZfc2VjdG9yX3QpYmxrX3JxX3BvcyhyZXEpOwogCQlyaW5nX3JlcS0+dS5ydy5oYW5kbGUgPSBp bmZvLT5oYW5kbGU7CkBAIC01NjAsNzMgKzY3MCwzMCBAQCBzdGF0aWMgaW50IGJsa2lmX3F1ZXVl X3J3X3JlcShzdHJ1Y3QgcmVxdWVzdCAqcmVxKQogCQkJCXJpbmdfcmVxLT5vcGVyYXRpb24gPSAw OwogCQkJfQogCQl9Ci0JCXJpbmdfcmVxLT51LnJ3Lm5yX3NlZ21lbnRzID0gbnNlZzsKLQl9Ci0J Zm9yX2VhY2hfc2coaW5mby0+c2hhZG93W2lkXS5zZywgc2csIG5zZWcsIGkpIHsKLQkJZnNlY3Qg PSBzZy0+b2Zmc2V0ID4+IDk7Ci0JCWxzZWN0ID0gZnNlY3QgKyAoc2ctPmxlbmd0aCA+PiA5KSAt IDE7Ci0KLQkJaWYgKChyaW5nX3JlcS0+b3BlcmF0aW9uID09IEJMS0lGX09QX0lORElSRUNUKSAm JgotCQkgICAgKGkgJSBTRUdTX1BFUl9JTkRJUkVDVF9GUkFNRSA9PSAwKSkgewotCQkJaWYgKHNl Z21lbnRzKQotCQkJCWt1bm1hcF9hdG9taWMoc2VnbWVudHMpOwotCi0JCQluID0gaSAvIFNFR1Nf UEVSX0lORElSRUNUX0ZSQU1FOwotCQkJZ250X2xpc3RfZW50cnkgPSBnZXRfaW5kaXJlY3RfZ3Jh bnQoJmdyZWZfaGVhZCwgaW5mbyk7Ci0JCQlpbmZvLT5zaGFkb3dbaWRdLmluZGlyZWN0X2dyYW50 c1tuXSA9IGdudF9saXN0X2VudHJ5OwotCQkJc2VnbWVudHMgPSBrbWFwX2F0b21pYyhnbnRfbGlz dF9lbnRyeS0+cGFnZSk7Ci0JCQlyaW5nX3JlcS0+dS5pbmRpcmVjdC5pbmRpcmVjdF9ncmVmc1tu XSA9IGdudF9saXN0X2VudHJ5LT5ncmVmOwotCQl9Ci0KLQkJZ250X2xpc3RfZW50cnkgPSBnZXRf Z3JhbnQoJmdyZWZfaGVhZCwKLQkJCQkJICAgeGVuX3BhZ2VfdG9fZ2ZuKHNnX3BhZ2Uoc2cpKSwK LQkJCQkJICAgaW5mbyk7Ci0JCXJlZiA9IGdudF9saXN0X2VudHJ5LT5ncmVmOwotCi0JCWluZm8t PnNoYWRvd1tpZF0uZ3JhbnRzX3VzZWRbaV0gPSBnbnRfbGlzdF9lbnRyeTsKLQotCQlpZiAocnFf ZGF0YV9kaXIocmVxKSAmJiBpbmZvLT5mZWF0dXJlX3BlcnNpc3RlbnQpIHsKLQkJCWNoYXIgKmJ2 ZWNfZGF0YTsKLQkJCXZvaWQgKnNoYXJlZF9kYXRhOworCQlyaW5nX3JlcS0+dS5ydy5ucl9zZWdt ZW50cyA9IG51bV9ncmFudDsKKwl9CiAKLQkJCUJVR19PTihzZy0+b2Zmc2V0ICsgc2ctPmxlbmd0 aCA+IFBBR0VfU0laRSk7CisJc2V0dXAucmluZ19yZXEgPSByaW5nX3JlcTsKKwlzZXR1cC5pZCA9 IGlkOworCWZvcl9lYWNoX3NnKGluZm8tPnNoYWRvd1tpZF0uc2csIHNnLCBudW1fc2csIGkpIHsK KwkJQlVHX09OKHNnLT5vZmZzZXQgKyBzZy0+bGVuZ3RoID4gUEFHRV9TSVpFKTsKIAotCQkJc2hh cmVkX2RhdGEgPSBrbWFwX2F0b21pYyhnbnRfbGlzdF9lbnRyeS0+cGFnZSk7Ci0JCQlidmVjX2Rh dGEgPSBrbWFwX2F0b21pYyhzZ19wYWdlKHNnKSk7CisJCWlmIChzZXR1cC5uZWVkX2NvcHkpIHsK KwkJCXNldHVwLmJ2ZWNfb2ZmID0gc2ctPm9mZnNldDsKKwkJCXNldHVwLmJ2ZWNfZGF0YSA9IGtt YXBfYXRvbWljKHNnX3BhZ2Uoc2cpKTsKKwkJfQogCi0JCQkvKgotCQkJICogdGhpcyBkb2VzIG5v dCB3aXBlIGRhdGEgc3RvcmVkIG91dHNpZGUgdGhlCi0JCQkgKiByYW5nZSBzZy0+b2Zmc2V0Li5z Zy0+b2Zmc2V0K3NnLT5sZW5ndGguCi0JCQkgKiBUaGVyZWZvcmUsIGJsa2JhY2sgKmNvdWxkKiBz ZWUgZGF0YSBmcm9tCi0JCQkgKiBwcmV2aW91cyByZXF1ZXN0cy4gVGhpcyBpcyBPSyBhcyBsb25n IGFzCi0JCQkgKiBwZXJzaXN0ZW50IGdyYW50cyBhcmUgc2hhcmVkIHdpdGgganVzdCBvbmUKLQkJ CSAqIGRvbWFpbi4gSXQgbWF5IG5lZWQgcmVmYWN0b3JpbmcgaWYgdGhpcwotCQkJICogY2hhbmdl cwotCQkJICovCi0JCQltZW1jcHkoc2hhcmVkX2RhdGEgKyBzZy0+b2Zmc2V0LAotCQkJICAgICAg IGJ2ZWNfZGF0YSAgICsgc2ctPm9mZnNldCwKLQkJCSAgICAgICBzZy0+bGVuZ3RoKTsKKwkJZ250 dGFiX2ZvcmVhY2hfZ3JhbnRfaW5fcmFuZ2Uoc2dfcGFnZShzZyksCisJCQkJCSAgICAgIHNnLT5v ZmZzZXQsCisJCQkJCSAgICAgIHNnLT5sZW5ndGgsCisJCQkJCSAgICAgIGJsa2lmX3NldHVwX3J3 X3JlcV9ncmFudCwKKwkJCQkJICAgICAgJnNldHVwKTsKIAotCQkJa3VubWFwX2F0b21pYyhidmVj X2RhdGEpOwotCQkJa3VubWFwX2F0b21pYyhzaGFyZWRfZGF0YSk7Ci0JCX0KLQkJaWYgKHJpbmdf cmVxLT5vcGVyYXRpb24gIT0gQkxLSUZfT1BfSU5ESVJFQ1QpIHsKLQkJCXJpbmdfcmVxLT51LnJ3 LnNlZ1tpXSA9Ci0JCQkJCShzdHJ1Y3QgYmxraWZfcmVxdWVzdF9zZWdtZW50KSB7Ci0JCQkJCQku Z3JlZiAgICAgICA9IHJlZiwKLQkJCQkJCS5maXJzdF9zZWN0ID0gZnNlY3QsCi0JCQkJCQkubGFz dF9zZWN0ICA9IGxzZWN0IH07Ci0JCX0gZWxzZSB7Ci0JCQluID0gaSAlIFNFR1NfUEVSX0lORElS RUNUX0ZSQU1FOwotCQkJc2VnbWVudHNbbl0gPQotCQkJCShzdHJ1Y3QgYmxraWZfcmVxdWVzdF9z ZWdtZW50KSB7Ci0JCQkJCQkuZ3JlZiAgICAgICA9IHJlZiwKLQkJCQkJCS5maXJzdF9zZWN0ID0g ZnNlY3QsCi0JCQkJCQkubGFzdF9zZWN0ICA9IGxzZWN0IH07Ci0JCX0KKwkJaWYgKHNldHVwLm5l ZWRfY29weSkKKwkJCWt1bm1hcF9hdG9taWMoc2V0dXAuYnZlY19kYXRhKTsKIAl9Ci0JaWYgKHNl Z21lbnRzKQotCQlrdW5tYXBfYXRvbWljKHNlZ21lbnRzKTsKKwlpZiAoc2V0dXAuc2VnbWVudHMp CisJCWt1bm1hcF9hdG9taWMoc2V0dXAuc2VnbWVudHMpOwogCiAJaW5mby0+cmluZy5yZXFfcHJv ZF9wdnQrKzsKIApAQCAtNjM0LDcgKzcwMSw3IEBAIHN0YXRpYyBpbnQgYmxraWZfcXVldWVfcndf cmVxKHN0cnVjdCByZXF1ZXN0ICpyZXEpCiAJaW5mby0+c2hhZG93W2lkXS5yZXEgPSAqcmluZ19y ZXE7CiAKIAlpZiAobmV3X3BlcnNpc3RlbnRfZ250cykKLQkJZ250dGFiX2ZyZWVfZ3JhbnRfcmVm ZXJlbmNlcyhncmVmX2hlYWQpOworCQlnbnR0YWJfZnJlZV9ncmFudF9yZWZlcmVuY2VzKHNldHVw LmdyZWZfaGVhZCk7CiAKIAlyZXR1cm4gMDsKIH0KQEAgLTc1MSwxNCArODE4LDE0IEBAIHN0YXRp YyBpbnQgeGx2YmRfaW5pdF9ibGtfcXVldWUoc3RydWN0IGdlbmRpc2sgKmdkLCB1MTYgc2VjdG9y X3NpemUsCiAJLyogSGFyZCBzZWN0b3Igc2l6ZSBhbmQgbWF4IHNlY3RvcnMgaW1wZXJzb25hdGUg dGhlIGVxdWl2LiBoYXJkd2FyZS4gKi8KIAlibGtfcXVldWVfbG9naWNhbF9ibG9ja19zaXplKHJx LCBzZWN0b3Jfc2l6ZSk7CiAJYmxrX3F1ZXVlX3BoeXNpY2FsX2Jsb2NrX3NpemUocnEsIHBoeXNp Y2FsX3NlY3Rvcl9zaXplKTsKLQlibGtfcXVldWVfbWF4X2h3X3NlY3RvcnMocnEsIChzZWdtZW50 cyAqIFBBR0VfU0laRSkgLyA1MTIpOworCWJsa19xdWV1ZV9tYXhfaHdfc2VjdG9ycyhycSwgKHNl Z21lbnRzICogWEVOX1BBR0VfU0laRSkgLyA1MTIpOwogCiAJLyogRWFjaCBzZWdtZW50IGluIGEg cmVxdWVzdCBpcyB1cCB0byBhbiBhbGlnbmVkIHBhZ2UgaW4gc2l6ZS4gKi8KIAlibGtfcXVldWVf c2VnbWVudF9ib3VuZGFyeShycSwgUEFHRV9TSVpFIC0gMSk7CiAJYmxrX3F1ZXVlX21heF9zZWdt ZW50X3NpemUocnEsIFBBR0VfU0laRSk7CiAKIAkvKiBFbnN1cmUgYSBtZXJnZWQgcmVxdWVzdCB3 aWxsIGZpdCBpbiBhIHNpbmdsZSBJL08gcmluZyBzbG90LiAqLwotCWJsa19xdWV1ZV9tYXhfc2Vn bWVudHMocnEsIHNlZ21lbnRzKTsKKwlibGtfcXVldWVfbWF4X3NlZ21lbnRzKHJxLCBzZWdtZW50 cyAvIEdSQU5UU19QRVJfUFNFRyk7CiAKIAkvKiBNYWtlIHN1cmUgYnVmZmVyIGFkZHJlc3NlcyBh cmUgc2VjdG9yLWFsaWduZWQuICovCiAJYmxrX3F1ZXVlX2RtYV9hbGlnbm1lbnQocnEsIDUxMSk7 CkBAIC0xMTE3LDMyICsxMTg0LDY1IEBAIGZyZWVfc2hhZG93OgogCiB9CiAKK3N0cnVjdCBjb3B5 X2Zyb21fZ3JhbnQgeworCWNvbnN0IHN0cnVjdCBibGtfc2hhZG93ICpzOworCXVuc2lnbmVkIGlu dCBncmFudF9pZHg7CisJdW5zaWduZWQgaW50IGJ2ZWNfb2Zmc2V0OworCWNoYXIgKmJ2ZWNfZGF0 YTsKK307CisKK3N0YXRpYyB2b2lkIGJsa2lmX2NvcHlfZnJvbV9ncmFudCh1bnNpZ25lZCBsb25n IGdmbiwgdW5zaWduZWQgaW50IG9mZnNldCwKKwkJCQkgIHVuc2lnbmVkIGludCBsZW4sIHZvaWQg KmRhdGEpCit7CisJc3RydWN0IGNvcHlfZnJvbV9ncmFudCAqaW5mbyA9IGRhdGE7CisJY2hhciAq c2hhcmVkX2RhdGE7CisJLyogQ29udmVuaWVudCBhbGlhc2VzICovCisJY29uc3Qgc3RydWN0IGJs a19zaGFkb3cgKnMgPSBpbmZvLT5zOworCisJc2hhcmVkX2RhdGEgPSBrbWFwX2F0b21pYyhzLT5n cmFudHNfdXNlZFtpbmZvLT5ncmFudF9pZHhdLT5wYWdlKTsKKworCW1lbWNweShpbmZvLT5idmVj X2RhdGEgKyBpbmZvLT5idmVjX29mZnNldCwKKwkgICAgICAgc2hhcmVkX2RhdGEgKyBvZmZzZXQs IGxlbik7CisKKwlpbmZvLT5idmVjX29mZnNldCArPSBsZW47CisJaW5mby0+Z3JhbnRfaWR4Kys7 CisKKwlrdW5tYXBfYXRvbWljKHNoYXJlZF9kYXRhKTsKK30KKwogc3RhdGljIHZvaWQgYmxraWZf Y29tcGxldGlvbihzdHJ1Y3QgYmxrX3NoYWRvdyAqcywgc3RydWN0IGJsa2Zyb250X2luZm8gKmlu Zm8sCiAJCQkgICAgIHN0cnVjdCBibGtpZl9yZXNwb25zZSAqYnJldCkKIHsKIAlpbnQgaSA9IDA7 CiAJc3RydWN0IHNjYXR0ZXJsaXN0ICpzZzsKLQljaGFyICpidmVjX2RhdGE7Ci0Jdm9pZCAqc2hh cmVkX2RhdGE7Ci0JaW50IG5zZWc7CisJaW50IG51bV9zZywgbnVtX2dyYW50OworCXN0cnVjdCBj b3B5X2Zyb21fZ3JhbnQgZGF0YSA9IHsKKwkJLnMgPSBzLAorCQkuZ3JhbnRfaWR4ID0gMCwKKwl9 OwogCi0JbnNlZyA9IHMtPnJlcS5vcGVyYXRpb24gPT0gQkxLSUZfT1BfSU5ESVJFQ1QgPworCW51 bV9ncmFudCA9IHMtPnJlcS5vcGVyYXRpb24gPT0gQkxLSUZfT1BfSU5ESVJFQ1QgPwogCQlzLT5y ZXEudS5pbmRpcmVjdC5ucl9zZWdtZW50cyA6IHMtPnJlcS51LnJ3Lm5yX3NlZ21lbnRzOworCW51 bV9zZyA9IHMtPm51bV9zZzsKIAogCWlmIChicmV0LT5vcGVyYXRpb24gPT0gQkxLSUZfT1BfUkVB RCAmJiBpbmZvLT5mZWF0dXJlX3BlcnNpc3RlbnQpIHsKLQkJZm9yX2VhY2hfc2cocy0+c2csIHNn LCBuc2VnLCBpKSB7CisJCWZvcl9lYWNoX3NnKHMtPnNnLCBzZywgbnVtX3NnLCBpKSB7CiAJCQlC VUdfT04oc2ctPm9mZnNldCArIHNnLT5sZW5ndGggPiBQQUdFX1NJWkUpOwotCQkJc2hhcmVkX2Rh dGEgPSBrbWFwX2F0b21pYyhzLT5ncmFudHNfdXNlZFtpXS0+cGFnZSk7Ci0JCQlidmVjX2RhdGEg PSBrbWFwX2F0b21pYyhzZ19wYWdlKHNnKSk7Ci0JCQltZW1jcHkoYnZlY19kYXRhICAgKyBzZy0+ b2Zmc2V0LAotCQkJICAgICAgIHNoYXJlZF9kYXRhICsgc2ctPm9mZnNldCwKLQkJCSAgICAgICBz Zy0+bGVuZ3RoKTsKLQkJCWt1bm1hcF9hdG9taWMoYnZlY19kYXRhKTsKLQkJCWt1bm1hcF9hdG9t aWMoc2hhcmVkX2RhdGEpOworCisJCQlkYXRhLmJ2ZWNfb2Zmc2V0ID0gc2ctPm9mZnNldDsKKwkJ CWRhdGEuYnZlY19kYXRhID0ga21hcF9hdG9taWMoc2dfcGFnZShzZykpOworCisJCQlnbnR0YWJf Zm9yZWFjaF9ncmFudF9pbl9yYW5nZShzZ19wYWdlKHNnKSwKKwkJCQkJCSAgICAgIHNnLT5vZmZz ZXQsCisJCQkJCQkgICAgICBzZy0+bGVuZ3RoLAorCQkJCQkJICAgICAgYmxraWZfY29weV9mcm9t X2dyYW50LAorCQkJCQkJICAgICAgJmRhdGEpOworCisJCQlrdW5tYXBfYXRvbWljKGRhdGEuYnZl Y19kYXRhKTsKIAkJfQogCX0KIAkvKiBBZGQgdGhlIHBlcnNpc3RlbnQgZ3JhbnQgaW50byB0aGUg bGlzdCBvZiBmcmVlIGdyYW50cyAqLwotCWZvciAoaSA9IDA7IGkgPCBuc2VnOyBpKyspIHsKKwlm b3IgKGkgPSAwOyBpIDwgbnVtX2dyYW50OyBpKyspIHsKIAkJaWYgKGdudHRhYl9xdWVyeV9mb3Jl aWduX2FjY2VzcyhzLT5ncmFudHNfdXNlZFtpXS0+Z3JlZikpIHsKIAkJCS8qCiAJCQkgKiBJZiB0 aGUgZ3JhbnQgaXMgc3RpbGwgbWFwcGVkIGJ5IHRoZSBiYWNrZW5kICh0aGUKQEAgLTExNjgsNyAr MTI2OCw3IEBAIHN0YXRpYyB2b2lkIGJsa2lmX2NvbXBsZXRpb24oc3RydWN0IGJsa19zaGFkb3cg KnMsIHN0cnVjdCBibGtmcm9udF9pbmZvICppbmZvLAogCQl9CiAJfQogCWlmIChzLT5yZXEub3Bl cmF0aW9uID09IEJMS0lGX09QX0lORElSRUNUKSB7Ci0JCWZvciAoaSA9IDA7IGkgPCBJTkRJUkVD VF9HUkVGUyhuc2VnKTsgaSsrKSB7CisJCWZvciAoaSA9IDA7IGkgPCBJTkRJUkVDVF9HUkVGUyhu dW1fZ3JhbnQpOyBpKyspIHsKIAkJCWlmIChnbnR0YWJfcXVlcnlfZm9yZWlnbl9hY2Nlc3Mocy0+ aW5kaXJlY3RfZ3JhbnRzW2ldLT5ncmVmKSkgewogCQkJCWlmICghaW5mby0+ZmVhdHVyZV9wZXJz aXN0ZW50KQogCQkJCQlwcl9hbGVydF9yYXRlbGltaXRlZCgiYmFja2VkIGhhcyBub3QgdW5tYXBw ZWQgZ3JhbnQ6ICV1XG4iLApAQCAtMTMxMiw3ICsxNDEyLDcgQEAgc3RhdGljIGludCBzZXR1cF9i bGtyaW5nKHN0cnVjdCB4ZW5idXNfZGV2aWNlICpkZXYsCiB7CiAJc3RydWN0IGJsa2lmX3NyaW5n ICpzcmluZzsKIAlpbnQgZXJyLCBpOwotCXVuc2lnbmVkIGxvbmcgcmluZ19zaXplID0gaW5mby0+ bnJfcmluZ19wYWdlcyAqIFBBR0VfU0laRTsKKwl1bnNpZ25lZCBsb25nIHJpbmdfc2l6ZSA9IGlu Zm8tPm5yX3JpbmdfcGFnZXMgKiBYRU5fUEFHRV9TSVpFOwogCWdyYW50X3JlZl90IGdyZWZbWEVO QlVTX01BWF9SSU5HX1BBR0VTXTsKIAogCWZvciAoaSA9IDA7IGkgPCBpbmZvLT5ucl9yaW5nX3Bh Z2VzOyBpKyspCkBAIC0xNjQzLDggKzE3NDMsOCBAQCBzdGF0aWMgaW50IGJsa2lmX3JlY292ZXIo c3RydWN0IGJsa2Zyb250X2luZm8gKmluZm8pCiAJCQlhdG9taWNfc2V0KCZzcGxpdF9iaW8tPnBl bmRpbmcsIHBlbmRpbmcpOwogCQkJc3BsaXRfYmlvLT5iaW8gPSBiaW87CiAJCQlmb3IgKGkgPSAw OyBpIDwgcGVuZGluZzsgaSsrKSB7Ci0JCQkJb2Zmc2V0ID0gKGkgKiBzZWdzICogUEFHRV9TSVpF KSA+PiA5OwotCQkJCXNpemUgPSBtaW4oKHVuc2lnbmVkIGludCkoc2VncyAqIFBBR0VfU0laRSkg Pj4gOSwKKwkJCQlvZmZzZXQgPSAoaSAqIHNlZ3MgKiBYRU5fUEFHRV9TSVpFKSA+PiA5OworCQkJ CXNpemUgPSBtaW4oKHVuc2lnbmVkIGludCkoc2VncyAqIFhFTl9QQUdFX1NJWkUpID4+IDksCiAJ CQkJCSAgICh1bnNpZ25lZCBpbnQpYmlvX3NlY3RvcnMoYmlvKSAtIG9mZnNldCk7CiAJCQkJY2xv bmVkX2JpbyA9IGJpb19jbG9uZShiaW8sIEdGUF9OT0lPKTsKIAkJCQlCVUdfT04oY2xvbmVkX2Jp byA9PSBOVUxMKTsKQEAgLTE3NTUsMTUgKzE4NTUsMTcgQEAgc3RhdGljIHZvaWQgYmxrZnJvbnRf c2V0dXBfZGlzY2FyZChzdHJ1Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbykKIAogc3RhdGljIGludCBi bGtmcm9udF9zZXR1cF9pbmRpcmVjdChzdHJ1Y3QgYmxrZnJvbnRfaW5mbyAqaW5mbykKIHsKLQl1 bnNpZ25lZCBpbnQgc2VnczsKKwl1bnNpZ25lZCBpbnQgcHNlZ3MsIGdyYW50czsKIAlpbnQgZXJy LCBpOwogCiAJaWYgKGluZm8tPm1heF9pbmRpcmVjdF9zZWdtZW50cyA9PSAwKQotCQlzZWdzID0g QkxLSUZfTUFYX1NFR01FTlRTX1BFUl9SRVFVRVNUOworCQlncmFudHMgPSBCTEtJRl9NQVhfU0VH TUVOVFNfUEVSX1JFUVVFU1Q7CiAJZWxzZQotCQlzZWdzID0gaW5mby0+bWF4X2luZGlyZWN0X3Nl Z21lbnRzOworCQlncmFudHMgPSBpbmZvLT5tYXhfaW5kaXJlY3Rfc2VnbWVudHM7CisJcHNlZ3Mg PSBncmFudHMgLyBHUkFOVFNfUEVSX1BTRUc7CiAKLQllcnIgPSBmaWxsX2dyYW50X2J1ZmZlcihp bmZvLCAoc2VncyArIElORElSRUNUX0dSRUZTKHNlZ3MpKSAqIEJMS19SSU5HX1NJWkUoaW5mbykp OworCWVyciA9IGZpbGxfZ3JhbnRfYnVmZmVyKGluZm8sCisJCQkJKGdyYW50cyArIElORElSRUNU X0dSRUZTKGdyYW50cykpICogQkxLX1JJTkdfU0laRShpbmZvKSk7CiAJaWYgKGVycikKIAkJZ290 byBvdXRfb2ZfbWVtb3J5OwogCkBAIC0xNzczLDcgKzE4NzUsNyBAQCBzdGF0aWMgaW50IGJsa2Zy b250X3NldHVwX2luZGlyZWN0KHN0cnVjdCBibGtmcm9udF9pbmZvICppbmZvKQogCQkgKiBncmFu dHMsIHdlIG5lZWQgdG8gYWxsb2NhdGUgYSBzZXQgb2YgcGFnZXMgdGhhdCBjYW4gYmUKIAkJICog dXNlZCBmb3IgbWFwcGluZyBpbmRpcmVjdCBncmVmcwogCQkgKi8KLQkJaW50IG51bSA9IElORElS RUNUX0dSRUZTKHNlZ3MpICogQkxLX1JJTkdfU0laRShpbmZvKTsKKwkJaW50IG51bSA9IElORElS RUNUX0dSRUZTKGdyYW50cykgKiBCTEtfUklOR19TSVpFKGluZm8pOwogCiAJCUJVR19PTighbGlz dF9lbXB0eSgmaW5mby0+aW5kaXJlY3RfcGFnZXMpKTsKIAkJZm9yIChpID0gMDsgaSA8IG51bTsg aSsrKSB7CkBAIC0xNzg2LDIwICsxODg4LDIwIEBAIHN0YXRpYyBpbnQgYmxrZnJvbnRfc2V0dXBf aW5kaXJlY3Qoc3RydWN0IGJsa2Zyb250X2luZm8gKmluZm8pCiAKIAlmb3IgKGkgPSAwOyBpIDwg QkxLX1JJTkdfU0laRShpbmZvKTsgaSsrKSB7CiAJCWluZm8tPnNoYWRvd1tpXS5ncmFudHNfdXNl ZCA9IGt6YWxsb2MoCi0JCQlzaXplb2YoaW5mby0+c2hhZG93W2ldLmdyYW50c191c2VkWzBdKSAq IHNlZ3MsCisJCQlzaXplb2YoaW5mby0+c2hhZG93W2ldLmdyYW50c191c2VkWzBdKSAqIGdyYW50 cywKIAkJCUdGUF9OT0lPKTsKLQkJaW5mby0+c2hhZG93W2ldLnNnID0ga3phbGxvYyhzaXplb2Yo aW5mby0+c2hhZG93W2ldLnNnWzBdKSAqIHNlZ3MsIEdGUF9OT0lPKTsKKwkJaW5mby0+c2hhZG93 W2ldLnNnID0ga3phbGxvYyhzaXplb2YoaW5mby0+c2hhZG93W2ldLnNnWzBdKSAqIHBzZWdzLCBH RlBfTk9JTyk7CiAJCWlmIChpbmZvLT5tYXhfaW5kaXJlY3Rfc2VnbWVudHMpCiAJCQlpbmZvLT5z aGFkb3dbaV0uaW5kaXJlY3RfZ3JhbnRzID0ga3phbGxvYygKIAkJCQlzaXplb2YoaW5mby0+c2hh ZG93W2ldLmluZGlyZWN0X2dyYW50c1swXSkgKgotCQkJCUlORElSRUNUX0dSRUZTKHNlZ3MpLAor CQkJCUlORElSRUNUX0dSRUZTKGdyYW50cyksCiAJCQkJR0ZQX05PSU8pOwogCQlpZiAoKGluZm8t PnNoYWRvd1tpXS5ncmFudHNfdXNlZCA9PSBOVUxMKSB8fAogCQkJKGluZm8tPnNoYWRvd1tpXS5z ZyA9PSBOVUxMKSB8fAogCQkgICAgIChpbmZvLT5tYXhfaW5kaXJlY3Rfc2VnbWVudHMgJiYKIAkJ ICAgICAoaW5mby0+c2hhZG93W2ldLmluZGlyZWN0X2dyYW50cyA9PSBOVUxMKSkpCiAJCQlnb3Rv IG91dF9vZl9tZW1vcnk7Ci0JCXNnX2luaXRfdGFibGUoaW5mby0+c2hhZG93W2ldLnNnLCBzZWdz KTsKKwkJc2dfaW5pdF90YWJsZShpbmZvLT5zaGFkb3dbaV0uc2csIHBzZWdzKTsKIAl9CiAKIAot LSAKMi4xLjQKCgpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f XwpYZW4tZGV2ZWwgbWFpbGluZyBsaXN0Clhlbi1kZXZlbEBsaXN0cy54ZW4ub3JnCmh0dHA6Ly9s aXN0cy54ZW4ub3JnL3hlbi1kZXZlbAo= From mboxrd@z Thu Jan 1 00:00:00 1970 From: julien.grall@citrix.com (Julien Grall) Date: Mon, 7 Sep 2015 16:33:53 +0100 Subject: [PATCH v4 15/20] block/xen-blkfront: Make it running on 64KB page granularity In-Reply-To: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> References: <1441640038-23615-1-git-send-email-julien.grall@citrix.com> Message-ID: <1441640038-23615-16-git-send-email-julien.grall@citrix.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org The PV block protocol is using 4KB page granularity. The goal of this patch is to allow a Linux using 64KB page granularity using block device on a non-modified Xen. The block API is using segment which should at least be the size of a Linux page. Therefore, the driver will have to break the page in chunk of 4K before giving the page to the backend. When breaking a 64KB segment in 4KB chunks, it is possible that some chunks are empty. As the PV protocol always require to have data in the chunk, we have to count the number of Xen page which will be in use and avoid sending empty chunks. Note that, a pre-defined number of grants are reserved before preparing the request. This pre-defined number is based on the number and the maximum size of the segments. If each segment contains a very small amount of data, the driver may reserve too many grants (16 grants is reserved per segment with 64KB page granularity). Furthermore, in the case of persistent grants we allocate one Linux page per grant although only the first 4KB of the page will be effectively in use. This could be improved by sharing the page with multiple grants. Signed-off-by: Julien Grall Acked-by: Roger Pau Monn? --- Cc: Konrad Rzeszutek Wilk Cc: Boris Ostrovsky Cc: David Vrabel Improvement such as support 64KB grant is not taken into consideration in this patch because we have the requirement to run a Linux using 64KB page on a non-modified Xen. Changes in v4: - Rebase after d50babbe300eedf33ea5b00a12c5df3a05bd96c7 " xen-blkfront: introduce blkfront_gather_backend_features()" - Fix typoes - Add Roger's acked-by Changes in v3: - Use DIV_ROUND_UP in INDIRECT_GREFS - Split lines over 80 characters whenever it's possible - s/mfn/gfn/ based on the new naming - The grant callback doesn't allow anymore to change the len (wasn't used here). - gnttab_foreach_grant has been renamed to gnttab_foreach_grant_in_range - Use gnttab_count_grant to get the number of grants in a sg - Do some renaming to use the correct variable every time Changes in v2: - Use gnttab_foreach_grant to split a Linux page into grant --- drivers/block/xen-blkfront.c | 324 ++++++++++++++++++++++++++++--------------- 1 file changed, 213 insertions(+), 111 deletions(-) diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index 4232cbd..f2cdc73 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -78,6 +78,7 @@ struct blk_shadow { struct grant **grants_used; struct grant **indirect_grants; struct scatterlist *sg; + unsigned int num_sg; }; struct split_bio { @@ -107,8 +108,12 @@ static unsigned int xen_blkif_max_ring_order; module_param_named(max_ring_page_order, xen_blkif_max_ring_order, int, S_IRUGO); MODULE_PARM_DESC(max_ring_page_order, "Maximum order of pages to be used for the shared ring"); -#define BLK_RING_SIZE(info) __CONST_RING_SIZE(blkif, PAGE_SIZE * (info)->nr_ring_pages) -#define BLK_MAX_RING_SIZE __CONST_RING_SIZE(blkif, PAGE_SIZE * XENBUS_MAX_RING_PAGES) +#define BLK_RING_SIZE(info) \ + __CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * (info)->nr_ring_pages) + +#define BLK_MAX_RING_SIZE \ + __CONST_RING_SIZE(blkif, XEN_PAGE_SIZE * XENBUS_MAX_RING_PAGES) + /* * ring-ref%i i=(-1UL) would take 11 characters + 'ring-ref' is 8, so 19 * characters are enough. Define to 20 to keep consist with backend. @@ -147,6 +152,7 @@ struct blkfront_info unsigned int discard_granularity; unsigned int discard_alignment; unsigned int feature_persistent:1; + /* Number of 4KB segments handled */ unsigned int max_indirect_segments; int is_ready; struct blk_mq_tag_set tag_set; @@ -175,10 +181,23 @@ static DEFINE_SPINLOCK(minor_lock); #define DEV_NAME "xvd" /* name in /dev */ -#define SEGS_PER_INDIRECT_FRAME \ - (PAGE_SIZE/sizeof(struct blkif_request_segment)) -#define INDIRECT_GREFS(_segs) \ - ((_segs + SEGS_PER_INDIRECT_FRAME - 1)/SEGS_PER_INDIRECT_FRAME) +/* + * Grants are always the same size as a Xen page (i.e 4KB). + * A physical segment is always the same size as a Linux page. + * Number of grants per physical segment + */ +#define GRANTS_PER_PSEG (PAGE_SIZE / XEN_PAGE_SIZE) + +#define GRANTS_PER_INDIRECT_FRAME \ + (XEN_PAGE_SIZE / sizeof(struct blkif_request_segment)) + +#define PSEGS_PER_INDIRECT_FRAME \ + (GRANTS_INDIRECT_FRAME / GRANTS_PSEGS) + +#define INDIRECT_GREFS(_grants) \ + DIV_ROUND_UP(_grants, GRANTS_PER_INDIRECT_FRAME) + +#define GREFS(_psegs) ((_psegs) * GRANTS_PER_PSEG) static int blkfront_setup_indirect(struct blkfront_info *info); static int blkfront_gather_backend_features(struct blkfront_info *info); @@ -466,14 +485,100 @@ static int blkif_queue_discard_req(struct request *req) return 0; } +struct setup_rw_req { + unsigned int grant_idx; + struct blkif_request_segment *segments; + struct blkfront_info *info; + struct blkif_request *ring_req; + grant_ref_t gref_head; + unsigned int id; + /* Only used when persistent grant is used and it's a read request */ + bool need_copy; + unsigned int bvec_off; + char *bvec_data; +}; + +static void blkif_setup_rw_req_grant(unsigned long gfn, unsigned int offset, + unsigned int len, void *data) +{ + struct setup_rw_req *setup = data; + int n, ref; + struct grant *gnt_list_entry; + unsigned int fsect, lsect; + /* Convenient aliases */ + unsigned int grant_idx = setup->grant_idx; + struct blkif_request *ring_req = setup->ring_req; + struct blkfront_info *info = setup->info; + struct blk_shadow *shadow = &info->shadow[setup->id]; + + if ((ring_req->operation == BLKIF_OP_INDIRECT) && + (grant_idx % GRANTS_PER_INDIRECT_FRAME == 0)) { + if (setup->segments) + kunmap_atomic(setup->segments); + + n = grant_idx / GRANTS_PER_INDIRECT_FRAME; + gnt_list_entry = get_indirect_grant(&setup->gref_head, info); + shadow->indirect_grants[n] = gnt_list_entry; + setup->segments = kmap_atomic(gnt_list_entry->page); + ring_req->u.indirect.indirect_grefs[n] = gnt_list_entry->gref; + } + + gnt_list_entry = get_grant(&setup->gref_head, gfn, info); + ref = gnt_list_entry->gref; + shadow->grants_used[grant_idx] = gnt_list_entry; + + if (setup->need_copy) { + void *shared_data; + + shared_data = kmap_atomic(gnt_list_entry->page); + /* + * this does not wipe data stored outside the + * range sg->offset..sg->offset+sg->length. + * Therefore, blkback *could* see data from + * previous requests. This is OK as long as + * persistent grants are shared with just one + * domain. It may need refactoring if this + * changes + */ + memcpy(shared_data + offset, + setup->bvec_data + setup->bvec_off, + len); + + kunmap_atomic(shared_data); + setup->bvec_off += len; + } + + fsect = offset >> 9; + lsect = fsect + (len >> 9) - 1; + if (ring_req->operation != BLKIF_OP_INDIRECT) { + ring_req->u.rw.seg[grant_idx] = + (struct blkif_request_segment) { + .gref = ref, + .first_sect = fsect, + .last_sect = lsect }; + } else { + setup->segments[grant_idx % GRANTS_PER_INDIRECT_FRAME] = + (struct blkif_request_segment) { + .gref = ref, + .first_sect = fsect, + .last_sect = lsect }; + } + + (setup->grant_idx)++; +} + static int blkif_queue_rw_req(struct request *req) { struct blkfront_info *info = req->rq_disk->private_data; struct blkif_request *ring_req; unsigned long id; - unsigned int fsect, lsect; - int i, ref, n; - struct blkif_request_segment *segments = NULL; + int i; + struct setup_rw_req setup = { + .grant_idx = 0, + .segments = NULL, + .info = info, + .need_copy = rq_data_dir(req) && info->feature_persistent, + }; /* * Used to store if we are able to queue the request by just using @@ -481,25 +586,23 @@ static int blkif_queue_rw_req(struct request *req) * as there are not sufficiently many free. */ bool new_persistent_gnts; - grant_ref_t gref_head; - struct grant *gnt_list_entry = NULL; struct scatterlist *sg; - int nseg, max_grefs; + int num_sg, max_grefs, num_grant; - max_grefs = req->nr_phys_segments; + max_grefs = req->nr_phys_segments * GRANTS_PER_PSEG; if (max_grefs > BLKIF_MAX_SEGMENTS_PER_REQUEST) /* * If we are using indirect segments we need to account * for the indirect grefs used in the request. */ - max_grefs += INDIRECT_GREFS(req->nr_phys_segments); + max_grefs += INDIRECT_GREFS(max_grefs); /* Check if we have enough grants to allocate a requests */ if (info->persistent_gnts_c < max_grefs) { new_persistent_gnts = 1; if (gnttab_alloc_grant_references( max_grefs - info->persistent_gnts_c, - &gref_head) < 0) { + &setup.gref_head) < 0) { gnttab_request_free_callback( &info->callback, blkif_restart_queue_callback, @@ -516,12 +619,19 @@ static int blkif_queue_rw_req(struct request *req) info->shadow[id].request = req; BUG_ON(info->max_indirect_segments == 0 && - req->nr_phys_segments > BLKIF_MAX_SEGMENTS_PER_REQUEST); + GREFS(req->nr_phys_segments) > BLKIF_MAX_SEGMENTS_PER_REQUEST); BUG_ON(info->max_indirect_segments && - req->nr_phys_segments > info->max_indirect_segments); - nseg = blk_rq_map_sg(req->q, req, info->shadow[id].sg); + GREFS(req->nr_phys_segments) > info->max_indirect_segments); + + num_sg = blk_rq_map_sg(req->q, req, info->shadow[id].sg); + num_grant = 0; + /* Calculate the number of grant used */ + for_each_sg(info->shadow[id].sg, sg, num_sg, i) + num_grant += gnttab_count_grant(sg->offset, sg->length); + ring_req->u.rw.id = id; - if (nseg > BLKIF_MAX_SEGMENTS_PER_REQUEST) { + info->shadow[id].num_sg = num_sg; + if (num_grant > BLKIF_MAX_SEGMENTS_PER_REQUEST) { /* * The indirect operation can only be a BLKIF_OP_READ or * BLKIF_OP_WRITE @@ -532,7 +642,7 @@ static int blkif_queue_rw_req(struct request *req) BLKIF_OP_WRITE : BLKIF_OP_READ; ring_req->u.indirect.sector_number = (blkif_sector_t)blk_rq_pos(req); ring_req->u.indirect.handle = info->handle; - ring_req->u.indirect.nr_segments = nseg; + ring_req->u.indirect.nr_segments = num_grant; } else { ring_req->u.rw.sector_number = (blkif_sector_t)blk_rq_pos(req); ring_req->u.rw.handle = info->handle; @@ -560,73 +670,30 @@ static int blkif_queue_rw_req(struct request *req) ring_req->operation = 0; } } - ring_req->u.rw.nr_segments = nseg; - } - for_each_sg(info->shadow[id].sg, sg, nseg, i) { - fsect = sg->offset >> 9; - lsect = fsect + (sg->length >> 9) - 1; - - if ((ring_req->operation == BLKIF_OP_INDIRECT) && - (i % SEGS_PER_INDIRECT_FRAME == 0)) { - if (segments) - kunmap_atomic(segments); - - n = i / SEGS_PER_INDIRECT_FRAME; - gnt_list_entry = get_indirect_grant(&gref_head, info); - info->shadow[id].indirect_grants[n] = gnt_list_entry; - segments = kmap_atomic(gnt_list_entry->page); - ring_req->u.indirect.indirect_grefs[n] = gnt_list_entry->gref; - } - - gnt_list_entry = get_grant(&gref_head, - xen_page_to_gfn(sg_page(sg)), - info); - ref = gnt_list_entry->gref; - - info->shadow[id].grants_used[i] = gnt_list_entry; - - if (rq_data_dir(req) && info->feature_persistent) { - char *bvec_data; - void *shared_data; + ring_req->u.rw.nr_segments = num_grant; + } - BUG_ON(sg->offset + sg->length > PAGE_SIZE); + setup.ring_req = ring_req; + setup.id = id; + for_each_sg(info->shadow[id].sg, sg, num_sg, i) { + BUG_ON(sg->offset + sg->length > PAGE_SIZE); - shared_data = kmap_atomic(gnt_list_entry->page); - bvec_data = kmap_atomic(sg_page(sg)); + if (setup.need_copy) { + setup.bvec_off = sg->offset; + setup.bvec_data = kmap_atomic(sg_page(sg)); + } - /* - * this does not wipe data stored outside the - * range sg->offset..sg->offset+sg->length. - * Therefore, blkback *could* see data from - * previous requests. This is OK as long as - * persistent grants are shared with just one - * domain. It may need refactoring if this - * changes - */ - memcpy(shared_data + sg->offset, - bvec_data + sg->offset, - sg->length); + gnttab_foreach_grant_in_range(sg_page(sg), + sg->offset, + sg->length, + blkif_setup_rw_req_grant, + &setup); - kunmap_atomic(bvec_data); - kunmap_atomic(shared_data); - } - if (ring_req->operation != BLKIF_OP_INDIRECT) { - ring_req->u.rw.seg[i] = - (struct blkif_request_segment) { - .gref = ref, - .first_sect = fsect, - .last_sect = lsect }; - } else { - n = i % SEGS_PER_INDIRECT_FRAME; - segments[n] = - (struct blkif_request_segment) { - .gref = ref, - .first_sect = fsect, - .last_sect = lsect }; - } + if (setup.need_copy) + kunmap_atomic(setup.bvec_data); } - if (segments) - kunmap_atomic(segments); + if (setup.segments) + kunmap_atomic(setup.segments); info->ring.req_prod_pvt++; @@ -634,7 +701,7 @@ static int blkif_queue_rw_req(struct request *req) info->shadow[id].req = *ring_req; if (new_persistent_gnts) - gnttab_free_grant_references(gref_head); + gnttab_free_grant_references(setup.gref_head); return 0; } @@ -751,14 +818,14 @@ static int xlvbd_init_blk_queue(struct gendisk *gd, u16 sector_size, /* Hard sector size and max sectors impersonate the equiv. hardware. */ blk_queue_logical_block_size(rq, sector_size); blk_queue_physical_block_size(rq, physical_sector_size); - blk_queue_max_hw_sectors(rq, (segments * PAGE_SIZE) / 512); + blk_queue_max_hw_sectors(rq, (segments * XEN_PAGE_SIZE) / 512); /* Each segment in a request is up to an aligned page in size. */ blk_queue_segment_boundary(rq, PAGE_SIZE - 1); blk_queue_max_segment_size(rq, PAGE_SIZE); /* Ensure a merged request will fit in a single I/O ring slot. */ - blk_queue_max_segments(rq, segments); + blk_queue_max_segments(rq, segments / GRANTS_PER_PSEG); /* Make sure buffer addresses are sector-aligned. */ blk_queue_dma_alignment(rq, 511); @@ -1117,32 +1184,65 @@ free_shadow: } +struct copy_from_grant { + const struct blk_shadow *s; + unsigned int grant_idx; + unsigned int bvec_offset; + char *bvec_data; +}; + +static void blkif_copy_from_grant(unsigned long gfn, unsigned int offset, + unsigned int len, void *data) +{ + struct copy_from_grant *info = data; + char *shared_data; + /* Convenient aliases */ + const struct blk_shadow *s = info->s; + + shared_data = kmap_atomic(s->grants_used[info->grant_idx]->page); + + memcpy(info->bvec_data + info->bvec_offset, + shared_data + offset, len); + + info->bvec_offset += len; + info->grant_idx++; + + kunmap_atomic(shared_data); +} + static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, struct blkif_response *bret) { int i = 0; struct scatterlist *sg; - char *bvec_data; - void *shared_data; - int nseg; + int num_sg, num_grant; + struct copy_from_grant data = { + .s = s, + .grant_idx = 0, + }; - nseg = s->req.operation == BLKIF_OP_INDIRECT ? + num_grant = s->req.operation == BLKIF_OP_INDIRECT ? s->req.u.indirect.nr_segments : s->req.u.rw.nr_segments; + num_sg = s->num_sg; if (bret->operation == BLKIF_OP_READ && info->feature_persistent) { - for_each_sg(s->sg, sg, nseg, i) { + for_each_sg(s->sg, sg, num_sg, i) { BUG_ON(sg->offset + sg->length > PAGE_SIZE); - shared_data = kmap_atomic(s->grants_used[i]->page); - bvec_data = kmap_atomic(sg_page(sg)); - memcpy(bvec_data + sg->offset, - shared_data + sg->offset, - sg->length); - kunmap_atomic(bvec_data); - kunmap_atomic(shared_data); + + data.bvec_offset = sg->offset; + data.bvec_data = kmap_atomic(sg_page(sg)); + + gnttab_foreach_grant_in_range(sg_page(sg), + sg->offset, + sg->length, + blkif_copy_from_grant, + &data); + + kunmap_atomic(data.bvec_data); } } /* Add the persistent grant into the list of free grants */ - for (i = 0; i < nseg; i++) { + for (i = 0; i < num_grant; i++) { if (gnttab_query_foreign_access(s->grants_used[i]->gref)) { /* * If the grant is still mapped by the backend (the @@ -1168,7 +1268,7 @@ static void blkif_completion(struct blk_shadow *s, struct blkfront_info *info, } } if (s->req.operation == BLKIF_OP_INDIRECT) { - for (i = 0; i < INDIRECT_GREFS(nseg); i++) { + for (i = 0; i < INDIRECT_GREFS(num_grant); i++) { if (gnttab_query_foreign_access(s->indirect_grants[i]->gref)) { if (!info->feature_persistent) pr_alert_ratelimited("backed has not unmapped grant: %u\n", @@ -1312,7 +1412,7 @@ static int setup_blkring(struct xenbus_device *dev, { struct blkif_sring *sring; int err, i; - unsigned long ring_size = info->nr_ring_pages * PAGE_SIZE; + unsigned long ring_size = info->nr_ring_pages * XEN_PAGE_SIZE; grant_ref_t gref[XENBUS_MAX_RING_PAGES]; for (i = 0; i < info->nr_ring_pages; i++) @@ -1643,8 +1743,8 @@ static int blkif_recover(struct blkfront_info *info) atomic_set(&split_bio->pending, pending); split_bio->bio = bio; for (i = 0; i < pending; i++) { - offset = (i * segs * PAGE_SIZE) >> 9; - size = min((unsigned int)(segs * PAGE_SIZE) >> 9, + offset = (i * segs * XEN_PAGE_SIZE) >> 9; + size = min((unsigned int)(segs * XEN_PAGE_SIZE) >> 9, (unsigned int)bio_sectors(bio) - offset); cloned_bio = bio_clone(bio, GFP_NOIO); BUG_ON(cloned_bio == NULL); @@ -1755,15 +1855,17 @@ static void blkfront_setup_discard(struct blkfront_info *info) static int blkfront_setup_indirect(struct blkfront_info *info) { - unsigned int segs; + unsigned int psegs, grants; int err, i; if (info->max_indirect_segments == 0) - segs = BLKIF_MAX_SEGMENTS_PER_REQUEST; + grants = BLKIF_MAX_SEGMENTS_PER_REQUEST; else - segs = info->max_indirect_segments; + grants = info->max_indirect_segments; + psegs = grants / GRANTS_PER_PSEG; - err = fill_grant_buffer(info, (segs + INDIRECT_GREFS(segs)) * BLK_RING_SIZE(info)); + err = fill_grant_buffer(info, + (grants + INDIRECT_GREFS(grants)) * BLK_RING_SIZE(info)); if (err) goto out_of_memory; @@ -1773,7 +1875,7 @@ static int blkfront_setup_indirect(struct blkfront_info *info) * grants, we need to allocate a set of pages that can be * used for mapping indirect grefs */ - int num = INDIRECT_GREFS(segs) * BLK_RING_SIZE(info); + int num = INDIRECT_GREFS(grants) * BLK_RING_SIZE(info); BUG_ON(!list_empty(&info->indirect_pages)); for (i = 0; i < num; i++) { @@ -1786,20 +1888,20 @@ static int blkfront_setup_indirect(struct blkfront_info *info) for (i = 0; i < BLK_RING_SIZE(info); i++) { info->shadow[i].grants_used = kzalloc( - sizeof(info->shadow[i].grants_used[0]) * segs, + sizeof(info->shadow[i].grants_used[0]) * grants, GFP_NOIO); - info->shadow[i].sg = kzalloc(sizeof(info->shadow[i].sg[0]) * segs, GFP_NOIO); + info->shadow[i].sg = kzalloc(sizeof(info->shadow[i].sg[0]) * psegs, GFP_NOIO); if (info->max_indirect_segments) info->shadow[i].indirect_grants = kzalloc( sizeof(info->shadow[i].indirect_grants[0]) * - INDIRECT_GREFS(segs), + INDIRECT_GREFS(grants), GFP_NOIO); if ((info->shadow[i].grants_used == NULL) || (info->shadow[i].sg == NULL) || (info->max_indirect_segments && (info->shadow[i].indirect_grants == NULL))) goto out_of_memory; - sg_init_table(info->shadow[i].sg, segs); + sg_init_table(info->shadow[i].sg, psegs); } -- 2.1.4