From: Rick Macklem <rick.macklem@gmail.com>
To: Cedric Blancher <cedric.blancher@gmail.com>
Cc: Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
freebsd-hackers@freebsd.org
Subject: Re: Increase FreeBSD NFSv4 nfsd buffer size? Re: Increase default NFSv4 server size "max_block_size" to 4MB
Date: Thu, 14 May 2026 16:49:04 -0700 [thread overview]
Message-ID: <CAM5tNy6ZbrCw46fS0mKdHpQDbk6axusk_pfoSQ8+-6vj7FiLaQ@mail.gmail.com> (raw)
In-Reply-To: <CAM5tNy4zgRBFAmwWCApAYvTHuhVtnkQRdhG1xpjekUnhdtsA2Q@mail.gmail.com>
On Thu, May 14, 2026 at 3:39 PM Rick Macklem <rick.macklem@gmail.com> wrote:
>
> On Wed, May 13, 2026 at 4:38 PM Rick Macklem <rick.macklem@gmail.com> wrote:
> >
> > On Wed, May 13, 2026 at 1:37 AM Cedric Blancher
> > <cedric.blancher@gmail.com> wrote:
> > >
> > > On Wed, 13 May 2026 at 01:02, Rick Macklem <rick.macklem@gmail.com> wrote:
> > > >
> > > > On Tue, May 12, 2026 at 6:18 AM Cedric Blancher
> > > > <cedric.blancher@gmail.com> wrote:
> > > > >
> > > > > On Tue, 17 Mar 2026 at 00:35, Rick Macklem <rick.macklem@gmail.com> wrote:
> > > > > >
> > > > > > On Mon, Mar 16, 2026 at 5:41 AM Chuck Lever <cel@kernel.org> wrote:
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Mar 16, 2026, at 3:51 AM, Cedric Blancher wrote:
> > > > > > > > As debated a while ago, can the default NFSv4 server size for
> > > > > > > > "max_block_size" be increased to 4MB, please?
> > > > > > >
> > > > > > > There is an administrative setting to raise this limit for
> > > > > > > recent versions of the kernel. Can you report your experience
> > > > > > > when you raise the limit? Hiccups, performance issues, etc? I
> > > > > > > would kind of like this exercise to be data-driven.
> > > > > > >
> > > > > > > What is still unknown to me is which NFS client implementations
> > > > > > > can support 4MB or 8MB. Without client support, an increase in
> > > > > > > the default in NFSD doesn't mean anything. Rick, Anna, Roland?
> > > > > > Although it has not seen much testing, it is possible to do a > 1Mbyte NFSv4
> > > > > > mount in FreeBSD.
> > > > > > For a 2Mbyte mount, (the only size > 1Mbyte I've tried) the settings would be..
> > > > > > In /boot/loader.conf
> > > > > > kern.maxphys=2097152
> > > > > > vfs.maxbcachebuf=2097152
> > > > > >
> > > > > > and in /etc/sysctl.conf
> > > > > > kern.ipc.maxsockbuf=9455616
> > > > > >
> > > > > > Then a mount will use 2Mbytes if the server supports it.
> > > > >
> > > > > How can I verify that the FreeBSD NFSv4 nfsd now uses 2M for NFS buffers?
> > > > The default is 128K. (You can see what it is via "sysctl vfs.nfsd.srvmaxio".)
> > > >
> > > > With "main",
> > >
> > > Which FreeBSD version is "main" exactly, i.e. which 16.0 snapshot
> > > date? Is this supported for 15.0?
> The commit just hit "main" as b92b9da33006. So any snapshot after
> to-day should have it. (It will be MFC'd to stable/15 in 2 weeks and will
> be in 15.2.)
>
> To enable it, put this line in /etc/rc.conf:
> nfs_server_maxio=4194304
> and this line in /etc/sysctl.conf:
> kern.ipc.maxsockbuf=18892800 (or larger)
Oh, and I should also point out that, for the low end hardware I have,
I have never seen a performance improvement for a maxio setting
above 256Kbytes. The story may be different for faster hardware,
but I wouldn't just assume that bumping it up will get you better
performance. (There is, of course, also a difference between getting
better performance for a single client vs fair distribution of server
resources across many clients.)
rick
>
> rick
>
> > >
> > > > you can increase that up to 4M by putting
> > > > nfs_server_maxio=N (where N can be up to 4194304 for "main" and 1048576 for 15)
> > >
> > > What line do I have to put into /etc/sysctl.conf and /etc/rc.conf?
> > Oops, my bad. It isn't committed yet. I forgot that I have to convince the
> > FreeBSD collective to increase UIO_MAXIOV or convince the ZFS folk
> > to not check for that limit in ZFS.
> >
> > I will email when it gets into "main".
> >
> > It can currently be set to 1Mbyte via
> > nfs_server_maxio=1048576
> > in /etc/rc.conf and
> > kern.ipc.maxsockbuf=5242880 (the exact minimum setting is printed
> > out if you boot
> > without this set large
> > enough in
> > /etc/sysctl.conf.)
> > --> it must be at
> > least a little more than
> > 4 *
> > nfs_server_maxio so that 4 reads
> > or writes fit
> > in the socket's buffers.
> >
> > rick
> >
> > >
> > > > - adding an entry in /etc/sysctl.conf for a larger value for kern.ipc.maxsockbuf
> > > > (It will tell you the recommended minimum setting if you boot after
> > > > putting nfs_server_maxio=N in /etc/rc.conf.)
> > >
> > > How can I verify that the settings work, on the NFS server side?
> > Look in a wireshark packet trace for maxread and maxwrite in a
> > GETATTR reply.
> >
> > > Can
> > > we programmatically probe which values are supported?
> > The sysctl is vfs.nfsd.srvmaxio and, when you try and set this
> > to too large a value, it will fail with EINVAL.
> > (You can do this from C via the sysctl(3) library call. See "man 3 sysctl".)
> >
> > rick
> >
> > >
> > > Ced
> > > --
> > > Cedric Blancher <cedric.blancher@gmail.com>
> > > [https://plus.google.com/u/0/+CedricBlancher/]
> > > Institute Pasteur
> > >
next prev parent reply other threads:[~2026-05-14 23:49 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-16 7:51 Increase default NFSv4 server size "max_block_size" to 4MB Cedric Blancher
2026-03-16 12:39 ` Chuck Lever
2026-03-16 14:40 ` Roland Mainz
2026-03-16 23:35 ` Rick Macklem
2026-05-12 13:18 ` Increase FreeBSD NFSv4 nfsd buffer size? " Cedric Blancher
2026-05-12 23:02 ` Rick Macklem
2026-05-13 8:37 ` Cedric Blancher
2026-05-13 23:38 ` Rick Macklem
2026-05-14 22:39 ` Rick Macklem
2026-05-14 22:52 ` vermaden
2026-05-14 23:49 ` Rick Macklem [this message]
2026-03-17 13:51 ` Anna Schumaker
2026-03-30 18:46 ` Mike Snitzer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAM5tNy6ZbrCw46fS0mKdHpQDbk6axusk_pfoSQ8+-6vj7FiLaQ@mail.gmail.com \
--to=rick.macklem@gmail.com \
--cc=cedric.blancher@gmail.com \
--cc=freebsd-hackers@freebsd.org \
--cc=linux-nfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).