* [PATCH 1/5] test_proxy_pass: test for auto chunking on 1.0 backends
@ 2016-04-27 0:27 5% ` Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2016-04-27 0:27 UTC (permalink / raw)
To: yahns-public
These are followups to the following two commits:
* commit d16326723d
("proxy_http_response: fix non-terminated fast responses, too")
* commit 8c9f33a539
("proxy_http_response: workaround non-terminated backends")
---
test/test_proxy_pass.rb | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/test/test_proxy_pass.rb b/test/test_proxy_pass.rb
index 448e480..c938976 100644
--- a/test/test_proxy_pass.rb
+++ b/test/test_proxy_pass.rb
@@ -586,6 +586,21 @@ def check_eof_body(host, port)
assert_match %r{\AHTTP/1\.1 200 OK\r\n}, res
assert_match %r{\r\n\r\neof-body-slow\z}, res
s.close
+
+ # we auto-chunk on 1.1 requests and 1.0 backends
+ %w(eof-body-slow eof-body-fast).each do |x|
+ s = TCPSocket.new(host, port)
+ s.write("GET /#{x} HTTP/1.1\r\nHost: example.com\r\n\r\n")
+ res = ''.dup
+ res << s.readpartial(512) until res =~ /0\r\n\r\n\z/
+ s.close
+ head, body = res.split("\r\n\r\n", 2)
+ head = head.split("\r\n")
+ assert_equal 'HTTP/1.1 200 OK', head[0]
+ assert head.include?('Connection: keep-alive')
+ assert head.include?('Transfer-Encoding: chunked')
+ assert_match %r{\Ad\r\n#{x}\r\n0\r\n\r\n\z}, body
+ end
end
end
--
EW
^ permalink raw reply related [relevance 5%]
* Re: [PATCH] proxy_http_response: workaround non-terminated backends
2016-04-06 6:25 5% [PATCH] proxy_http_response: workaround non-terminated backends Eric Wong
@ 2016-04-16 8:17 7% ` Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2016-04-16 8:17 UTC (permalink / raw)
To: yahns-public
Eric Wong <e@80x24.org> wrote:
> @@ -232,7 +247,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
> end
>
> busy = wbuf.busy and return proxy_busy_mod_blocked(wbuf, busy)
> - proxy_busy_mod_done(wbuf.wbuf_persist) # returns nil
> + proxy_busy_mod_done(alive) # returns nil
> end
>
> def proxy_wait_next(qflags)
Hm... that seems wrong, The following fixup commit may be necessary.
Letting it run on our site for a few days before writing a commit
message and committing. (Not really in a state of mind to be hacking
this time of year, but I noticed some high FD usage...)
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -247,7 +247,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
end
busy = wbuf.busy and return proxy_busy_mod_blocked(wbuf, busy)
- proxy_busy_mod_done(alive) # returns nil
+ proxy_busy_mod_done(alive && wbuf.wbuf_persist) # returns nil
end
def proxy_wait_next(qflags)
^ permalink raw reply [relevance 7%]
* [ANN] yahns 1.12.3 -_- sleepy app server for Ruby
@ 2016-04-08 20:01 4% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2016-04-08 20:01 UTC (permalink / raw)
To: ruby-talk, yahns-public
A Free Software, multi-threaded, non-blocking network
application server designed for low _idle_ power consumption.
It is primarily optimized for applications with occasional users
which see little or no traffic. yahns currently hosts Rack/HTTP
applications, but may eventually support other application
types. Unlike some existing servers, yahns is extremely
sensitive to fatal bugs in the applications it hosts.
Changes:
yahns 1.12.3 - more fixes and doc updates
This release only contains two changes since v1.12.2:
* proxy_http_response: workaround non-terminated backends
The first is a fix for the undocumented and unstable "proxy_pass"
feature:
https://yhbt.net/yahns-public/20160406062556.10988-1-e@80x24.org/t/
However, I'm considering supporting proxy_pass as a stable API:
https://yhbt.net/yahns-public/20160220081619.GA10850@dcvr.yhbt.net/t/
* doc: recommend "verify_mode: OpenSSL::SSL::VERIFY_NONE"
Only a documentation change prompted by the discovery that
some browsers/platforms will try to prompt users for client
certs:
https://yhbt.net/yahns-public/20160316003434.GA14791@dcvr.yhbt.net/t/
Please note the disclaimer:
yahns is extremely sensitive to fatal bugs in the apps it hosts. There
is no (and never will be) any built-in "watchdog"-type feature to kill
stuck processes/threads. Each yahns process may be handling thousands
of clients; unexpectedly killing the process will abort _all_ of those
connections. Lives may be lost!
yahns hackers are not responsible for your application/library bugs.
Use an application server which is tolerant of buggy applications
if you cannot be bothered to fix all your fatal bugs.
* git clone git://yhbt.net/yahns
* http://yahns.yhbt.net/README
* http://yahns.yhbt.net/NEWS.atom.xml
* we only accept plain-text email yahns-public@yhbt.net
* and archive all the mail we receive: http://yhbt.net/yahns-public/
* nntp://news.public-inbox.org/inbox.comp.lang.ruby.yahns
^ permalink raw reply [relevance 4%]
* [PATCH] proxy_http_response: workaround non-terminated backends
@ 2016-04-06 6:25 5% Eric Wong
2016-04-16 8:17 7% ` Eric Wong
0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2016-04-06 6:25 UTC (permalink / raw)
To: yahns-public
Without this, we could only support persistent connections if
the backend gives a valid Content-Length or set
"Transfer-Encoding: chunked" in the response header.
Being good netizens, we want to use persistent connections as
much as possible if a remote client supports it; so perform
chunking ourselves when our remote clients are HTTP/1.1 and
able to decode chunked responses.
This is necessary to support some non-Rack HTTP/1.0-only
backend servers which rely on connection termination to
terminate responses.
Tested manually with a Perl PSGI application running under
"plackup". Unlike Rack, the PSGI spec does not specify whether
the PSGI application or PSGI server should handle response
termination: git clone https://github.com/plack/psgi-specs.git
---
lib/yahns/proxy_http_response.rb | 21 ++++++++++++++++++---
1 file changed, 18 insertions(+), 3 deletions(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 0a7e722..0e72d3a 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -68,6 +68,7 @@ def proxy_response_start(res, tip, kcar, req_res)
env['REQUEST_METHOD'] != 'HEAD'.freeze
flags = MSG_DONTWAIT
alive = @hs.next? && self.class.persistent_connections
+ term = false
response_headers = env['yahns.proxy_pass.response_headers']
res = "HTTP/1.1 #{msg ? %Q(#{code} #{msg}) : status}\r\n".dup
@@ -76,7 +77,10 @@ def proxy_response_start(res, tip, kcar, req_res)
when /\A(?:Connection|Keep-Alive)\z/i
next # do not let some upstream headers leak through
when %r{\AContent-Length\z}i
+ term = true
flags |= MSG_MORE if have_body && value.to_i > 0
+ when %r{\ATransfer-Encoding\z}i
+ term = true if value =~ /\bchunked\b/i
end
# response header mapping
@@ -92,6 +96,13 @@ def proxy_response_start(res, tip, kcar, req_res)
# For now, do not add a Date: header, assume upstream already did it
# but do not care if they did not
+
+ # chunk the response ourselves if the client supports it,
+ # but the backend does not terminate properly
+ if alive && ! term && (env['HTTP_VERSION'] == 'HTTP/1.1'.freeze)
+ res << "Transfer-Encoding: chunked\r\n".freeze
+ alive = true
+ end
res << (alive ? "Connection: keep-alive\r\n\r\n".freeze
: "Connection: close\r\n\r\n".freeze)
@@ -154,6 +165,7 @@ def proxy_response_start(res, tip, kcar, req_res)
case tmp = tip.shift || req_res.kgio_tryread(0x2000, rbuf)
when String
+ tmp = chunk_out(tmp) if alive
wbuf = proxy_write(wbuf, tmp, alive)
when nil
req_res.shutdown
@@ -174,7 +186,8 @@ def proxy_response_start(res, tip, kcar, req_res)
def proxy_response_finish(kcar, wbuf, req_res)
rbuf = Thread.current[:yahns_rbuf]
- if len = kcar.body_bytes_left
+ alive = wbuf.wbuf_persist
+ if len = kcar.body_bytes_left # known Content-Length
case tmp = req_res.kgio_tryread(0x2000, rbuf)
when String
@@ -186,7 +199,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
return :wait_readable # self remains in :ignore, wait on upstream
end while len != 0
- elsif kcar.chunked? # nasty chunked body
+ elsif kcar.chunked? # nasty chunked response body
buf = ''.dup
unless req_res.proxy_trailers
@@ -221,8 +234,10 @@ def proxy_response_finish(kcar, wbuf, req_res)
case tmp = req_res.kgio_tryread(0x2000, rbuf)
when String
+ tmp = chunk_out(tmp) if alive
wbuf.wbuf_write(self, tmp)
when nil
+ wbuf.wbuf_write(self, "0\r\n\r\n".freeze) if alive
req_res.shutdown
break
when :wait_readable
@@ -232,7 +247,7 @@ def proxy_response_finish(kcar, wbuf, req_res)
end
busy = wbuf.busy and return proxy_busy_mod_blocked(wbuf, busy)
- proxy_busy_mod_done(wbuf.wbuf_persist) # returns nil
+ proxy_busy_mod_done(alive) # returns nil
end
def proxy_wait_next(qflags)
--
EW
^ permalink raw reply related [relevance 5%]
Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2016-04-06 6:25 5% [PATCH] proxy_http_response: workaround non-terminated backends Eric Wong
2016-04-16 8:17 7% ` Eric Wong
2016-04-08 20:01 4% [ANN] yahns 1.12.3 -_- sleepy app server for Ruby Eric Wong
2016-04-27 0:26 [PATCH 0/5] proxy_pass resource cleanup fixes Eric Wong
2016-04-27 0:27 5% ` [PATCH 1/5] test_proxy_pass: test for auto chunking on 1.0 backends Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/yahns.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).