From: Mike Marshall <hubcap@omnibond.com>
To: Matthew Wilcox <willy@infradead.org>,
Mike Marshall <hubcap@omnibond.com>
Cc: linux-fsdevel <linux-fsdevel@vger.kernel.org>
Subject: Re: [RFC PATCH] implement orangefs_readahead
Date: Mon, 1 Feb 2021 22:32:58 -0500 [thread overview]
Message-ID: <CAOg9mSSd5ccoi1keeiRfkV+esekcQLxer9_1iZ-r9bQDjZLfBg@mail.gmail.com> (raw)
In-Reply-To: <20210201130800.GP308988@casper.infradead.org>
>> This is not the way to do it. You need to actually kick
>> off readahead in this routine so that you get pipelining
>> (ie the app is working on pages 0-15 at the same time
>> the server is getting you pages 16-31).
Orangefs isn't very good at reading or writing a few
pages at a time. Its optimal block size is four megabytes.
I'm trying to do IOs big enough to make Orangefs
start flowing like it needs to and then have pages
on hand to fill with the data. Perhaps I can figure
how to use Dave Howell's code to control the
readahead window and make adjustments to
how many pages Orangefs reads per IO and
end up with something that is closer to how
readahead is intended to be used.
This patch is a big performance improvement over
the code that's upstream even though I'm
not using readahead as intended.
>> I don't see much support in orangefs for doing async
>> operations; everything seems to be modelled on
>> "submit an I/O and wait for it to complete".
Yep... when we were polishing up the kernel module to
attempt to go upstream, the code in there for async was
left behind... I might be able to make sense of it now,
Ida know... You've helped me to see this place where
it is needed.
>> adding async
>> support to orangefs is a little bigger task than I'm willing to put
>> significant effort into right now.
The effort and help that you're providing is much
appreciated and just what I need, thanks!
-Mike
On Mon, Feb 1, 2021 at 8:08 AM Matthew Wilcox <willy@infradead.org> wrote:
>
> On Sun, Jan 31, 2021 at 05:25:02PM -0500, Mike Marshall wrote:
> > I wish I knew how to specify _nr_pages in the readahead_control
> > structure so that all the extra pages I need could be obtained
> > in readahead_page instead of part there and the rest in my
> > open-coded stuff in orangefs_readpage. But it looks to me as
> > if values in the readahead_control structure are set heuristically
> > outside of my control over in ondemand_readahead?
>
> That's right (for now). I pointed you at some code from Dave Howells
> that will allow orangefs to enlarge the readahead window beyond that
> determined by the core code's algorithms.
>
> > [root@vm3 linux]# git diff master..readahead
> > diff --git a/fs/orangefs/inode.c b/fs/orangefs/inode.c
> > index 48f0547d4850..682a968cb82a 100644
> > --- a/fs/orangefs/inode.c
> > +++ b/fs/orangefs/inode.c
> > @@ -244,6 +244,25 @@ static int orangefs_writepages(struct
> > address_space *mapping,
> >
> > static int orangefs_launder_page(struct page *);
> >
> > +/*
> > + * Prefill the page cache with some pages that we're probably
> > + * about to need...
> > + */
> > +static void orangefs_readahead(struct readahead_control *rac)
> > +{
> > + pgoff_t index = readahead_index(rac);
> > + struct page *page;
> > +
> > + while ((page = readahead_page(rac))) {
> > + prefetchw(&page->flags);
> > + put_page(page);
> > + unlock_page(page);
> > + index++;
> > + }
> > +
> > + return;
> > +}
>
> This is not the way to do it. You need to actually kick off readahead in
> this routine so that you get pipelining (ie the app is working on pages
> 0-15 at the same time the server is getting you pages 16-31). I don't
> see much support in orangefs for doing async operations; everything
> seems to be modelled on "submit an I/O and wait for it to complete".
>
> I'm happy to help out with pagecache interactions, but adding async
> support to orangefs is a little bigger task than I'm willing to put
> significant effort into right now.
next prev parent reply other threads:[~2021-02-02 3:34 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-31 22:25 [RFC PATCH] implement orangefs_readahead Mike Marshall
2021-02-01 13:08 ` Matthew Wilcox
2021-02-02 3:32 ` Mike Marshall [this message]
2021-03-13 15:31 ` [RFC PATCH v2] " Mike Marshall
2021-03-17 3:04 ` Mike Marshall
2021-03-24 11:10 ` David Howells
2021-03-27 2:55 ` Mike Marshall
2021-03-27 3:50 ` Matthew Wilcox
2021-03-27 8:31 ` David Howells
2021-03-27 13:56 ` Matthew Wilcox
2021-03-27 15:40 ` Mike Marshall
2021-03-27 15:56 ` Matthew Wilcox
2021-03-28 3:04 ` Mike Marshall
2021-03-29 1:51 ` Mike Marshall
2021-03-29 9:37 ` David Howells
2021-03-29 23:25 ` Mike Marshall
[not found] ` <3726695.1617284551@warthog.procyon.org.uk >
2021-04-13 15:08 ` David Howells
2021-04-16 14:36 ` Mike Marshall
2021-04-16 15:14 ` Matthew Wilcox
2021-04-25 1:51 ` Mike Marshall
2021-04-16 15:41 ` David Howells
2021-04-25 1:43 ` Mike Marshall
2021-04-25 7:49 ` David Howells
2021-04-26 14:53 ` Mike Marshall
2021-04-26 19:01 ` Mike Marshall
2021-04-26 20:01 ` David Howells
2021-04-26 8:37 ` David Howells
2021-04-01 13:42 ` David Howells
2021-04-08 20:39 ` Mike Marshall
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAOg9mSSd5ccoi1keeiRfkV+esekcQLxer9_1iZ-r9bQDjZLfBg@mail.gmail.com \
--to=hubcap@omnibond.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=willy@infradead.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).