From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-3.3 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, RP_MATCHES_RCVD,URIBL_BLOCKED shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: yahns-public@yhbt.net Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id F25352044E for ; Wed, 6 Apr 2016 06:25:56 +0000 (UTC) From: Eric Wong To: yahns-public@yhbt.net Subject: [PATCH] proxy_http_response: workaround non-terminated backends Date: Wed, 6 Apr 2016 06:25:56 +0000 Message-Id: <20160406062556.10988-1-e@80x24.org> List-Id: Without this, we could only support persistent connections if the backend gives a valid Content-Length or set "Transfer-Encoding: chunked" in the response header. Being good netizens, we want to use persistent connections as much as possible if a remote client supports it; so perform chunking ourselves when our remote clients are HTTP/1.1 and able to decode chunked responses. This is necessary to support some non-Rack HTTP/1.0-only backend servers which rely on connection termination to terminate responses. Tested manually with a Perl PSGI application running under "plackup". Unlike Rack, the PSGI spec does not specify whether the PSGI application or PSGI server should handle response termination: git clone https://github.com/plack/psgi-specs.git --- lib/yahns/proxy_http_response.rb | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb index 0a7e722..0e72d3a 100644 --- a/lib/yahns/proxy_http_response.rb +++ b/lib/yahns/proxy_http_response.rb @@ -68,6 +68,7 @@ def proxy_response_start(res, tip, kcar, req_res) env['REQUEST_METHOD'] != 'HEAD'.freeze flags = MSG_DONTWAIT alive = @hs.next? && self.class.persistent_connections + term = false response_headers = env['yahns.proxy_pass.response_headers'] res = "HTTP/1.1 #{msg ? %Q(#{code} #{msg}) : status}\r\n".dup @@ -76,7 +77,10 @@ def proxy_response_start(res, tip, kcar, req_res) when /\A(?:Connection|Keep-Alive)\z/i next # do not let some upstream headers leak through when %r{\AContent-Length\z}i + term = true flags |= MSG_MORE if have_body && value.to_i > 0 + when %r{\ATransfer-Encoding\z}i + term = true if value =~ /\bchunked\b/i end # response header mapping @@ -92,6 +96,13 @@ def proxy_response_start(res, tip, kcar, req_res) # For now, do not add a Date: header, assume upstream already did it # but do not care if they did not + + # chunk the response ourselves if the client supports it, + # but the backend does not terminate properly + if alive && ! term && (env['HTTP_VERSION'] == 'HTTP/1.1'.freeze) + res << "Transfer-Encoding: chunked\r\n".freeze + alive = true + end res << (alive ? "Connection: keep-alive\r\n\r\n".freeze : "Connection: close\r\n\r\n".freeze) @@ -154,6 +165,7 @@ def proxy_response_start(res, tip, kcar, req_res) case tmp = tip.shift || req_res.kgio_tryread(0x2000, rbuf) when String + tmp = chunk_out(tmp) if alive wbuf = proxy_write(wbuf, tmp, alive) when nil req_res.shutdown @@ -174,7 +186,8 @@ def proxy_response_start(res, tip, kcar, req_res) def proxy_response_finish(kcar, wbuf, req_res) rbuf = Thread.current[:yahns_rbuf] - if len = kcar.body_bytes_left + alive = wbuf.wbuf_persist + if len = kcar.body_bytes_left # known Content-Length case tmp = req_res.kgio_tryread(0x2000, rbuf) when String @@ -186,7 +199,7 @@ def proxy_response_finish(kcar, wbuf, req_res) return :wait_readable # self remains in :ignore, wait on upstream end while len != 0 - elsif kcar.chunked? # nasty chunked body + elsif kcar.chunked? # nasty chunked response body buf = ''.dup unless req_res.proxy_trailers @@ -221,8 +234,10 @@ def proxy_response_finish(kcar, wbuf, req_res) case tmp = req_res.kgio_tryread(0x2000, rbuf) when String + tmp = chunk_out(tmp) if alive wbuf.wbuf_write(self, tmp) when nil + wbuf.wbuf_write(self, "0\r\n\r\n".freeze) if alive req_res.shutdown break when :wait_readable @@ -232,7 +247,7 @@ def proxy_response_finish(kcar, wbuf, req_res) end busy = wbuf.busy and return proxy_busy_mod_blocked(wbuf, busy) - proxy_busy_mod_done(wbuf.wbuf_persist) # returns nil + proxy_busy_mod_done(alive) # returns nil end def proxy_wait_next(qflags) -- EW