* [PATCH 0/4] a few forgotten patches from last year
@ 2022-12-22 9:02 Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
Dreading what breakage Rack 3.0 and Ruby 3.2 brings :<
Eric Wong (4):
test_buffer_tmpdir: drop fragile assertions
http_response: clamp body read size
standardize on 16384-byte reads
test/test_ssl: workaround multi-line exception messages
lib/yahns/http_client.rb | 2 +-
lib/yahns/proxy_http_response.rb | 10 ++++++++--
lib/yahns/req_res.rb | 6 +++---
test/test_buffer_tmpdir.rb | 2 --
test/test_ssl.rb | 2 +-
5 files changed, 13 insertions(+), 9 deletions(-)
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
2022-12-22 9:02 ` [PATCH 2/4] http_response: clamp body read size Eric Wong
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
When tweaking buffer sizes, another IN_CREATE event can
happen soon after the delete.
---
test/test_buffer_tmpdir.rb | 2 --
1 file changed, 2 deletions(-)
diff --git a/test/test_buffer_tmpdir.rb b/test/test_buffer_tmpdir.rb
index bac8017..39029ed 100644
--- a/test/test_buffer_tmpdir.rb
+++ b/test/test_buffer_tmpdir.rb
@@ -67,7 +67,6 @@ def test_output_buffer_tmpdir
event = @ino.take
assert_equal [:DELETE], event.events
assert_equal name, event.name
- refute File.exist?("#@tmpdir/#{name}")
end
ensure
c.close if c
@@ -101,7 +100,6 @@ def input_buffer(btype)
event = ino.take
assert_equal [:DELETE], event.events
assert_equal name, event.name
- refute File.exist?("#{tmpdir}/#{name}")
end
ensure
c.close if c
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 2/4] http_response: clamp body read size
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
2022-12-22 9:02 ` [PATCH 3/4] standardize on 16384-byte reads Eric Wong
2022-12-22 9:02 ` [PATCH 4/4] test/test_ssl: workaround multi-line exception messages Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
We don't want to overread in case a broken HTTP backend
sends us excessive data.
---
lib/yahns/proxy_http_response.rb | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 7df2834..d4a3dda 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -146,6 +146,12 @@ def proxy_res_headers(res, req_res)
have_body
end
+ def read_len(len)
+ max = 0x2000
+ max = len if len && len < max
+ max
+ end
+
def proxy_read_body(tip, kcar, req_res)
chunk = ''.dup if kcar.chunked?
len = kcar.body_bytes_left
@@ -153,7 +159,7 @@ def proxy_read_body(tip, kcar, req_res)
alive = req_res.alive
wbuf = req_res.resbuf
- case tmp = tip.shift || req_res.kgio_tryread(0x2000, rbuf)
+ case tmp = tip.shift || req_res.kgio_tryread(read_len(len), rbuf)
when String
if len
kcar.body_bytes_left -= tmp.size # progress for body_eof? => true
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 3/4] standardize on 16384-byte reads
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
2022-12-22 9:02 ` [PATCH 2/4] http_response: clamp body read size Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
2022-12-22 9:02 ` [PATCH 4/4] test/test_ssl: workaround multi-line exception messages Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
This is the size used by Ruby internally for IO.copy_stream.
8192 seems too small nowadays with the higher cost of syscalls.
---
lib/yahns/http_client.rb | 2 +-
lib/yahns/proxy_http_response.rb | 4 ++--
lib/yahns/req_res.rb | 6 +++---
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/lib/yahns/http_client.rb b/lib/yahns/http_client.rb
index b6b6035..826eb8d 100644
--- a/lib/yahns/http_client.rb
+++ b/lib/yahns/http_client.rb
@@ -315,7 +315,7 @@ def app_hijacked?(env, res)
end
def do_pread(io, count, offset)
- count = 0x4000 if count > 0x4000
+ count = 16384 if count > 16384
buf = Thread.current[:yahns_sfbuf] ||= ''.dup
if io.respond_to?(:pread)
io.pread(count, offset, buf)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index d4a3dda..db9c4b7 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -147,7 +147,7 @@ def proxy_res_headers(res, req_res)
end
def read_len(len)
- max = 0x2000
+ max = 16384
max = len if len && len < max
max
end
@@ -206,7 +206,7 @@ def proxy_read_trailers(kcar, req_res)
wbuf = req_res.resbuf
until kcar.trailers(tlr, chunk)
- case rv = req_res.kgio_tryread(0x2000, rbuf)
+ case rv = req_res.kgio_tryread(16384, rbuf)
when String
chunk << rv
when :wait_readable
diff --git a/lib/yahns/req_res.rb b/lib/yahns/req_res.rb
index 4ad8e5c..283fea8 100644
--- a/lib/yahns/req_res.rb
+++ b/lib/yahns/req_res.rb
@@ -29,7 +29,7 @@ def yahns_step # yahns event loop entry point
case resbuf = @resbuf # where are we at the response?
when nil # common case, catch the response header in a single read
- case rv = kgio_tryread(0x2000, buf)
+ case rv = kgio_tryread(16384, buf)
when String
if res = req.headers(@hdr = [], rv)
return c.proxy_response_start(res, rv, req, self)
@@ -48,7 +48,7 @@ def yahns_step # yahns event loop entry point
when String # continue reading trickled response headers from upstream
- case rv = kgio_tryread(0x2000, buf)
+ case rv = kgio_tryread(16384, buf)
when String then res = req.headers(@hdr, resbuf << rv) and break
when :wait_readable then return rv
when nil
@@ -114,7 +114,7 @@ def send_req_body(req) # @rrstate == [ (str|vec), rack.input, chunked? ]
# we should not be waiting on a slow network resource when reading
# input. However, some weird configs may disable this on LANs
# and we may wait indefinitely on input.read here...
- while input.read(0x2000, rbuf)
+ while input.read(16384, rbuf)
if chunked
buf[0] = "#{rbuf.size.to_s(16)}\r\n".freeze
buf[1] = rbuf
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH 4/4] test/test_ssl: workaround multi-line exception messages
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
` (2 preceding siblings ...)
2022-12-22 9:02 ` [PATCH 3/4] standardize on 16384-byte reads Eric Wong
@ 2022-12-22 9:02 ` Eric Wong
3 siblings, 0 replies; 5+ messages in thread
From: Eric Wong @ 2022-12-22 9:02 UTC (permalink / raw)
To: yahns-public
Needed for Ruby 3.1, and likely 3.2, as well...
---
test/test_ssl.rb | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/test/test_ssl.rb b/test/test_ssl.rb
index 7909094..9442ec8 100644
--- a/test/test_ssl.rb
+++ b/test/test_ssl.rb
@@ -178,7 +178,7 @@ def test_ssl_hijack
s.any_old_invalid_test_method
s.puts "FAIL"
rescue => e
- s.puts "#{e.class}: #{e.message}"
+ s.puts("#{e.class}: #{e.message}".split("\n")[0])
end
when nil
s.close
^ permalink raw reply related [flat|nested] 5+ messages in thread
end of thread, other threads:[~2022-12-22 9:02 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-12-22 9:02 [PATCH 0/4] a few forgotten patches from last year Eric Wong
2022-12-22 9:02 ` [PATCH 1/4] test_buffer_tmpdir: drop fragile assertions Eric Wong
2022-12-22 9:02 ` [PATCH 2/4] http_response: clamp body read size Eric Wong
2022-12-22 9:02 ` [PATCH 3/4] standardize on 16384-byte reads Eric Wong
2022-12-22 9:02 ` [PATCH 4/4] test/test_ssl: workaround multi-line exception messages Eric Wong
Code repositories for project(s) associated with this public inbox
http://yhbt.net/yahns.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).