From: Enzo Matsumiya <ematsumiya@suse.de>
To: linux-cifs@vger.kernel.org
Cc: smfrench@gmail.com, pc@manguebit.com, ronniesahlberg@gmail.com,
sprasad@microsoft.com, tom@talpey.com, bharathsm@microsoft.com,
henrique.carvalho@suse.com
Subject: [PATCH 1/8] smb: client: compress: fix buffer overrun in lz77_compress()
Date: Mon, 13 Apr 2026 16:07:06 -0300 [thread overview]
Message-ID: <20260413190713.283939-1-ematsumiya@suse.de> (raw)
@dst buffer is allocated with same size as @src, which, for good
compression cases, works fine.
However, when compression goes bad (e.g. random bytes payloads), the
compressed size can increase significantly, and even by stopping the
main loop at 7/8 of @slen, writing leftover literals could write past
the end of @dst because of LZ77 metadata.
To fix this, add lz77_compressed_alloc_size() helper to compute the
correct allocation size for @dst, accounting for metadata and worst
cast scenario (all literals).
While this is overprovisioning memory, it's not only correct, but also
allows lz77_compress() main loop to run without ever checking @dst
limits (i.e. a perf improvement).
Signed-off-by: Enzo Matsumiya <ematsumiya@suse.de>
---
fs/smb/client/compress.c | 6 +-----
fs/smb/client/compress/lz77.c | 14 ++++----------
fs/smb/client/compress/lz77.h | 28 ++++++++++++++++++++++++++++
3 files changed, 33 insertions(+), 15 deletions(-)
diff --git a/fs/smb/client/compress.c b/fs/smb/client/compress.c
index 3d1e73f5d9af..be9023f841e6 100644
--- a/fs/smb/client/compress.c
+++ b/fs/smb/client/compress.c
@@ -329,11 +329,7 @@ int smb_compress(struct TCP_Server_Info *server, struct smb_rqst *rq, compress_s
goto err_free;
}
- /*
- * This is just overprovisioning, as the algorithm will error out if @dst reaches 7/8
- * of @slen.
- */
- dlen = slen;
+ dlen = lz77_compressed_alloc_size(slen);
dst = kvzalloc(dlen, GFP_KERNEL);
if (!dst) {
ret = -ENOMEM;
diff --git a/fs/smb/client/compress/lz77.c b/fs/smb/client/compress/lz77.c
index 96e8a8057a77..16c7d8f3ef17 100644
--- a/fs/smb/client/compress/lz77.c
+++ b/fs/smb/client/compress/lz77.c
@@ -137,6 +137,10 @@ noinline int lz77_compress(const void *src, u32 slen, void *dst, u32 *dlen)
long flag = 0;
u64 *htable;
+ /* This is probably a bug, so throw a warning. */
+ if (WARN_ON_ONCE(*dlen < lz77_compressed_alloc_size(slen)))
+ return -EINVAL;
+
srcp = src;
end = src + slen;
dstp = dst;
@@ -180,15 +184,6 @@ noinline int lz77_compress(const void *src, u32 slen, void *dst, u32 *dlen)
continue;
}
- /*
- * Bail out if @dstp reached >= 7/8 of @slen -- already compressed badly, not worth
- * going further.
- */
- if (unlikely(dstp - dst >= slen - (slen >> 3))) {
- *dlen = slen;
- goto out;
- }
-
dstp = lz77_write_match(dstp, &nib, dist, len);
srcp += len;
@@ -225,7 +220,6 @@ noinline int lz77_compress(const void *src, u32 slen, void *dst, u32 *dlen)
lz77_write32(flag_pos, flag);
*dlen = dstp - dst;
-out:
kvfree(htable);
if (*dlen < slen)
diff --git a/fs/smb/client/compress/lz77.h b/fs/smb/client/compress/lz77.h
index cdcb191b48a2..2603eab9e071 100644
--- a/fs/smb/client/compress/lz77.h
+++ b/fs/smb/client/compress/lz77.h
@@ -11,5 +11,33 @@
#include <linux/kernel.h>
+/**
+ * lz77_compressed_alloc_size() - Compute compressed buffer size.
+ * @size: uncompressed (src) size
+ *
+ * Compute allocation size for the compressed buffer based on uncompressed size.
+ * Accounts for metadata and overprovision for the worst case scenario.
+ *
+ * LZ77 metadata is a 4-byte flag that is written:
+ * - on dst begin (pos 0)
+ * - every 32 literals or matches
+ * - on end-of-stream (possibly, if last write was another flag)
+ *
+ * Worst case scenario is an all-literal compression, which means:
+ * metadata bytes = 4 + ((@size / 32) * 4) + 4, or, simplified, (@size >> 3) + 8
+ *
+ * The worst case scenario rarely happens, but such overprovisioning also allows lz77_compress()
+ * main loop to run without ever bound checking dst, which is a huge perf improvement, while also
+ * being safe when compression goes bad.
+ *
+ * Return: required (*) allocation size for compressed buffer.
+ *
+ * (*) checked once in the beginning of lz77_compress()
+ */
+static __always_inline u32 lz77_compressed_alloc_size(const u32 size)
+{
+ return size + (size >> 3) + 8;
+}
+
int lz77_compress(const void *src, u32 slen, void *dst, u32 *dlen);
#endif /* _SMB_COMPRESS_LZ77_H */
--
2.53.0
next reply other threads:[~2026-04-13 19:07 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-04-13 19:07 Enzo Matsumiya [this message]
2026-04-13 19:07 ` [PATCH 2/8] smb: client: compress: fix bad encoding on last LZ77 flag Enzo Matsumiya
2026-04-19 1:35 ` Steve French
2026-04-13 19:07 ` [PATCH 3/8] smb: client: compress: fix counting in LZ77 match finding Enzo Matsumiya
2026-04-13 19:07 ` [PATCH 4/8] smb: client: compress: increase LZ77_MATCH_MAX_DIST Enzo Matsumiya
2026-04-13 19:07 ` [PATCH 5/8] smb: client: compress: LZ77 optimizations Enzo Matsumiya
2026-04-13 19:07 ` [PATCH 6/8] smb: client: compress: add code docs to lz77.c Enzo Matsumiya
2026-04-13 19:07 ` [PATCH 7/8] smb: client: compress: add compress/common.h Enzo Matsumiya
2026-04-20 21:24 ` Nathan Chancellor
2026-04-20 21:31 ` Steve French
2026-04-13 19:07 ` [PATCH 8/8] smb: common: add SMB3_COMPRESS_MAX_ALGS Enzo Matsumiya
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260413190713.283939-1-ematsumiya@suse.de \
--to=ematsumiya@suse.de \
--cc=bharathsm@microsoft.com \
--cc=henrique.carvalho@suse.com \
--cc=linux-cifs@vger.kernel.org \
--cc=pc@manguebit.com \
--cc=ronniesahlberg@gmail.com \
--cc=smfrench@gmail.com \
--cc=sprasad@microsoft.com \
--cc=tom@talpey.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).