* [PATCH 2/4] http_response: clamp body read size
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
2022-12-22 9:02 ` [PATCH 3/4] standardize on 16384-byte reads Eric Wong
2022-12-22 9:02 ` [PATCH 4/4] test/test_ssl: workaround multi-line exception messages Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
We don't want to overread in case a broken HTTP backend
sends us excessive data.
---
lib/yahns/proxy_http_response.rb | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 7df2834..d4a3dda 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -146,6 +146,12 @@ def proxy_res_headers(res, req_res)
have_body
end
+ def read_len(len)
+ max = 0x2000
+ max = len if len && len < max
+ max
+ end
+
def proxy_read_body(tip, kcar, req_res)
chunk = ''.dup if kcar.chunked?
len = kcar.body_bytes_left
@@ -153,7 +159,7 @@ def proxy_read_body(tip, kcar, req_res)
alive = req_res.alive
wbuf = req_res.resbuf
- case tmp = tip.shift || req_res.kgio_tryread(0x2000, rbuf)
+ case tmp = tip.shift || req_res.kgio_tryread(read_len(len), rbuf)
when String
if len
kcar.body_bytes_left -= tmp.size # progress for body_eof? => true
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/4] standardize on 16384-byte reads
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
2022-12-22 9:02 ` [PATCH 2/4] http_response: clamp body read size Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
2022-12-22 9:02 ` [PATCH 4/4] test/test_ssl: workaround multi-line exception messages Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
This is the size used by Ruby internally for IO.copy_stream.
8192 seems too small nowadays with the higher cost of syscalls.
---
lib/yahns/http_client.rb | 2 +-
lib/yahns/proxy_http_response.rb | 4 ++--
lib/yahns/req_res.rb | 6 +++---
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/lib/yahns/http_client.rb b/lib/yahns/http_client.rb
index b6b6035..826eb8d 100644
--- a/lib/yahns/http_client.rb
+++ b/lib/yahns/http_client.rb
@@ -315,7 +315,7 @@ def app_hijacked?(env, res)
end
def do_pread(io, count, offset)
- count = 0x4000 if count > 0x4000
+ count = 16384 if count > 16384
buf = Thread.current[:yahns_sfbuf] ||= ''.dup
if io.respond_to?(:pread)
io.pread(count, offset, buf)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index d4a3dda..db9c4b7 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -147,7 +147,7 @@ def proxy_res_headers(res, req_res)
end
def read_len(len)
- max = 0x2000
+ max = 16384
max = len if len && len < max
max
end
@@ -206,7 +206,7 @@ def proxy_read_trailers(kcar, req_res)
wbuf = req_res.resbuf
until kcar.trailers(tlr, chunk)
- case rv = req_res.kgio_tryread(0x2000, rbuf)
+ case rv = req_res.kgio_tryread(16384, rbuf)
when String
chunk << rv
when :wait_readable
diff --git a/lib/yahns/req_res.rb b/lib/yahns/req_res.rb
index 4ad8e5c..283fea8 100644
--- a/lib/yahns/req_res.rb
+++ b/lib/yahns/req_res.rb
@@ -29,7 +29,7 @@ def yahns_step # yahns event loop entry point
case resbuf = @resbuf # where are we at the response?
when nil # common case, catch the response header in a single read
- case rv = kgio_tryread(0x2000, buf)
+ case rv = kgio_tryread(16384, buf)
when String
if res = req.headers(@hdr = [], rv)
return c.proxy_response_start(res, rv, req, self)
@@ -48,7 +48,7 @@ def yahns_step # yahns event loop entry point
when String # continue reading trickled response headers from upstream
- case rv = kgio_tryread(0x2000, buf)
+ case rv = kgio_tryread(16384, buf)
when String then res = req.headers(@hdr, resbuf << rv) and break
when :wait_readable then return rv
when nil
@@ -114,7 +114,7 @@ def send_req_body(req) # @rrstate == [ (str|vec), rack.input, chunked? ]
# we should not be waiting on a slow network resource when reading
# input. However, some weird configs may disable this on LANs
# and we may wait indefinitely on input.read here...
- while input.read(0x2000, rbuf)
+ while input.read(16384, rbuf)
if chunked
buf[0] = "#{rbuf.size.to_s(16)}\r\n".freeze
buf[1] = rbuf
^ permalink raw reply related [flat|nested] 5+ messages in thread