From: juncheng bai <baijuncheng@unitedstack.com>
To: Ilya Dryomov <idryomov@gmail.com>
Cc: idryomov@redhat.com, Alex Elder <elder@linaro.org>,
Josh Durgin <josh.durgin@inktank.com>,
Guangliang Zhao <lucienchao@gmail.com>,
jeff@garzik.org, yehuda@hq.newdream.net,
Sage Weil <sage@newdream.net>,
elder@inktank.com,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Ceph Development <ceph-devel@vger.kernel.org>
Subject: Re: [PATCH RFC] storage:rbd: make the size of request is equal to the, size of the object
Date: Wed, 17 Jun 2015 17:47:51 +0800 [thread overview]
Message-ID: <55814247.4090301@unitedstack.com> (raw)
In-Reply-To: <CAOi1vP_-N=_2H+39LWjNUv=Ym7jE97FzDiN6eidQPEB0Q2=omQ@mail.gmail.com>
On 2015/6/17 16:24, Ilya Dryomov wrote:
> On Wed, Jun 17, 2015 at 6:04 AM, juncheng bai
> <baijuncheng@unitedstack.com> wrote:
>> Hi.
>> Yeah, you are right, use the default max_segments, the request size can
>> be the object size, because the bi_phys_segments of bio could be recount,
>> there's just a possibility.
>>
>> I want to fully understand the bi_phys_segments, hope you can give me some
>> information, thanks.
>>
>> The test information as shown below:
>> The systemtap script:
>> global greq=0;
>> probe kernel.function("bio_attempt_back_merge")
>> {
>> greq=pointer_arg(2);
>> }
>>
>> probe kernel.function("bio_attempt_back_merge").return
>> {
>> printf("after req addr:%p req segments:%d req offset:%lu req
>> length:%lu\n",
>> greq,
>> @cast(greq, "request")->nr_phys_segments,
>> @cast(greq, "request")->__sector * 512,
>> @cast(greq, "request")->__data_len);
>> }
>>
>> probe kernel.function("blk_mq_start_request")
>> {
>> printf("req addr:%p nr_phys_segments:%d, offset:%lu len:%lu\n",
>> pointer_arg(1),
>> @cast(pointer_arg(1), "request")->nr_phys_segments,
>> @cast(pointer_arg(1), "request")->__sector * 512,
>> @cast(pointer_arg(1), "request")->__data_len);
>> }
>>
>> Test command:
>> dd if=/dev/zero of=/dev/rbd0 bs=4M count=2 oflag=direct seek=100
>>
>> Cast one:
>> blk_queue_max_segments(q, 256);
>>
>> The output of stap:
>> after req addr:0xffff880ff60a08c0 req segments:73 req offset:419430400 req
>> length:2097152
>> after req addr:0xffff880ff60a08c0 req segments:73 req offset:419430400 req
>> length:2097152
>> after req addr:0xffff880ff60a0a80 req segments:186 req offset:421527552 req
>> length:1048576
>> req addr:0xffff880ff60a08c0 nr_phys_segments:73, offset:419430400
>> len:2097152
>> req addr:0xffff880ff60a0a80 nr_phys_segments:186, offset:421527552
>> len:1048576
>> req addr:0xffff880ff60a0c40 nr_phys_segments:232, offset:422576128
>> len:1048576
>>
>> after req addr:0xffff880ff60a0c40 req segments:73 req offset:423624704 req
>> length:2097152
>> after req addr:0xffff880ff60a0c40 req segments:73 req offset:423624704 req
>> length:2097152
>> after req addr:0xffff880ff60a0e00 req segments:186 req offset:425721856 req
>> length:1048576
>> req addr:0xffff880ff60a0c40 nr_phys_segments:73, offset:423624704
>> len:2097152
>> req addr:0xffff880ff60a0e00 nr_phys_segments:186, offset:425721856
>> len:1048576
>> req addr:0xffff880ff60a0fc0 nr_phys_segments:232, offset:426770432
>> len:1048576
>>
>> Case two:
>> blk_queue_max_segments(q, segment_size / PAGE_SIZE);
>>
>> The output of stap:
>> after req addr:0xffff88101c9a0000 req segments:478 req offset:419430400 req
>> length:4194304
>> req addr:0xffff88101c9a0000 nr_phys_segments:478, offset:419430400
>> len:4194304
>>
>> after req addr:0xffff88101c9a0000 req segments:478 req offset:423624704 req
>> length:4194304
>> req addr:0xffff88101c9a0000 nr_phys_segments:478, offset:423624704
>> len:4194304
>>
>> 1.Based on the setting of max_sectors and max_segments, decides the
>> size of a request.
>> 2.We have already set max_sectors to an object's size, so we should try
>> to ensure that a request to the size as possible as merge bio.
>
> Yeah, I also tried to explain this in the commit description [1].
> Initially I had BIO_MAX_PAGES in there, and realistically I still think
> it's enough for most cases, but discussion with you made me consider
> readv/writev case and so I changed it in my patch to max_hw_sectors
> (i.e. segment_size / SECTOR_SIZE) - this ensures that max_segments will
> never be a limiting factor even in theory.
>
> [1] https://github.com/ceph/ceph-client/commit/2d8006795564fbc0fa68d75758f605fe9f7a108e
>
> Thanks,
>
> Ilya
>
Yeah, I agree with you.
Thanks.
----
juncheng bai
prev parent reply other threads:[~2015-06-17 9:48 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-15 11:18 [PATCH RFC] storage:rbd: make the size of request is equal to the, size of the object juncheng bai
2015-06-15 13:03 ` Ilya Dryomov
2015-06-15 13:23 ` juncheng bai
2015-06-15 14:27 ` Ilya Dryomov
2015-06-16 3:28 ` juncheng bai
2015-06-16 8:37 ` Ilya Dryomov
2015-06-16 11:57 ` juncheng bai
2015-06-16 13:30 ` Ilya Dryomov
2015-06-16 14:14 ` juncheng bai
2015-06-16 15:51 ` Ilya Dryomov
2015-06-17 3:04 ` juncheng bai
2015-06-17 8:24 ` Ilya Dryomov
2015-06-17 9:47 ` juncheng bai [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55814247.4090301@unitedstack.com \
--to=baijuncheng@unitedstack.com \
--cc=ceph-devel@vger.kernel.org \
--cc=elder@inktank.com \
--cc=elder@linaro.org \
--cc=idryomov@gmail.com \
--cc=idryomov@redhat.com \
--cc=jeff@garzik.org \
--cc=josh.durgin@inktank.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lucienchao@gmail.com \
--cc=sage@newdream.net \
--cc=yehuda@hq.newdream.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.