From: Eric Wong <e@80x24.org>
To: yahns-public@yhbt.net
Subject: [PATCH] proxy_http_response: workaround non-terminated backends
Date: Wed, 6 Apr 2016 06:25:56 +0000 [thread overview]
Message-ID: <20160406062556.10988-1-e@80x24.org> (raw)
Without this, we could only support persistent connections if
the backend gives a valid Content-Length or set
"Transfer-Encoding: chunked" in the response header.
Being good netizens, we want to use persistent connections as
much as possible if a remote client supports it; so perform
chunking ourselves when our remote clients are HTTP/1.1 and
able to decode chunked responses.
This is necessary to support some non-Rack HTTP/1.0-only
backend servers which rely on connection termination to
terminate responses.
Tested manually with a Perl PSGI application running under
"plackup". Unlike Rack, the PSGI spec does not specify whether
the PSGI application or PSGI server should handle response
termination: git clone https://github.com/plack/psgi-specs.git
---
lib/yahns/proxy_http_response.rb | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 0a7e722..0e72d3a 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -68,6 +68,7 @@ def proxy_response_start(res, tip, kcar, req_res)
env['REQUEST_METHOD'] != 'HEAD'.freeze
flags = MSG_DONTWAIT
alive = @hs.next? && self.class.persistent_connections
+ term = false
response_headers = env['yahns.proxy_pass.response_headers']
res = "HTTP/1.1 #{msg ? %Q(#{code} #{msg}) : status}\r\n".dup
@@ -76,7 +77,10 @@ def proxy_response_start(res, tip, kcar, req_res)
when /\A(?:Connection|Keep-Alive)\z/i
next # do not let some upstream headers leak through
when %r{\AContent-Length\z}i
+ term = true
flags |= MSG_MORE if have_body && value.to_i > 0
+ when %r{\ATransfer-Encoding\z}i
+ term = true if value =~ /\bchunked\b/i
end
# response header mapping
@@ -92,6 +96,13 @@ def proxy_response_start(res, tip, kcar, req_res)
# For now, do not add a Date: header, assume upstream already did it
# but do not care if they did not
+
+ # chunk the response ourselves if the client supports it,
+ # but the backend does not terminate properly
+ if alive && ! term && (env['HTTP_VERSION'] == 'HTTP/1.1'.freeze)
+ res << "Transfer-Encoding: chunked\r\n".freeze
+ alive = true
+ end
res << (alive ? "Connection: keep-alive\r\n\r\n".freeze
: "Connection: close\r\n\r\n".freeze)
@@ -154,6 +165,7 @@ def proxy_response_start(res, tip, kcar, req_res)
case tmp = tip.shift || req_res.kgio_tryread(0x2000, rbuf)
when String
+ tmp = chunk_out(tmp) if alive
wbuf = proxy_write(wbuf, tmp, alive)
when nil
req_res.shutdown
@@ -174,7 +186,8 @@ def proxy_response_start(res, tip, kcar, req_res)
def proxy_response_finish(kcar, wbuf, req_res)
rbuf = Thread.current[:yahns_rbuf]
- if len = kcar.body_bytes_left
+ alive = wbuf.wbuf_persist
+ if len = kcar.body_bytes_left # known Content-Length
case tmp = req_res.kgio_tryread(0x2000, rbuf)
when String
@@ -186,7 +199,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
return :wait_readable # self remains in :ignore, wait on upstream
end while len != 0
- elsif kcar.chunked? # nasty chunked body
+ elsif kcar.chunked? # nasty chunked response body
buf = ''.dup
unless req_res.proxy_trailers
@@ -221,8 +234,10 @@ def proxy_response_finish(kcar, wbuf, req_res)
case tmp = req_res.kgio_tryread(0x2000, rbuf)
when String
+ tmp = chunk_out(tmp) if alive
wbuf.wbuf_write(self, tmp)
when nil
+ wbuf.wbuf_write(self, "0\r\n\r\n".freeze) if alive
req_res.shutdown
break
when :wait_readable
@@ -232,7 +247,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
end
busy = wbuf.busy and return proxy_busy_mod_blocked(wbuf, busy)
- proxy_busy_mod_done(wbuf.wbuf_persist) # returns nil
+ proxy_busy_mod_done(alive) # returns nil
end
def proxy_wait_next(qflags)
--
EW
next reply other threads:[~2016-04-06 6:25 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-04-06 6:25 Eric Wong [this message]
2016-04-16 8:17 ` [PATCH] proxy_http_response: workaround non-terminated backends Eric Wong
2016-04-22 0:13 ` [PATCH] proxy_pass: honor wbuf_persist when ending response Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/yahns/README
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160406062556.10988-1-e@80x24.org \
--to=e@80x24.org \
--cc=yahns-public@yhbt.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/yahns.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).