[PATCH 0/2] revenge of the body-less responses

[PATCH 0/2] revenge of the body-less responses

[Eric Wong]

It's wrong to set "Transfer-Encoding: chunked" on any response
without an entity body.  This confused curl -T- when I triggered
a rare 204 response...  Fortunately, yahns is not used in a
capacity where it's failure would start World War III.

Eric Wong (2):
      response: do not set chunked header on bodyless responses
      proxy_pass: do not chunk on bodyless upstream responses

 lib/yahns/http_response.rb       |  4 ++--
 lib/yahns/proxy_http_response.rb |  2 +-
 test/test_auto_chunk.rb          | 22 +++++++++++++++++++++-
 test/test_proxy_pass.rb          | 33 +++++++++++++++++++++++++++++++++
 4 files changed, 57 insertions(+), 4 deletions(-)

[PATCH 1/2] response: do not set chunked header on bodyless responses

[Eric Wong]

Setting "Transfer-Encoding: chunked" on responses will confuse
clients which see a 204 response and do not expect a body.
This follows Rack::Chunked behavior, as yahns should function
without Rack::Chunked middleware.

This regression appeared in yahns v1.13.0 (2016-08-05)
---
 lib/yahns/http_response.rb |  4 ++--
 test/test_auto_chunk.rb    | 22 +++++++++++++++++++++-
 2 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/lib/yahns/http_response.rb b/lib/yahns/http_response.rb
index a31ab70..bde4b3f 100644
--- a/lib/yahns/http_response.rb
+++ b/lib/yahns/http_response.rb
@@ -165,12 +165,12 @@ def http_response_write(res, opt)
     end
     count ||= clen
 
-    if !term && chunk_ok
+    if !term && chunk_ok && !hdr_only
       term = true
       body = Yahns::ChunkBody.new(body, opt)
       buf << "Transfer-Encoding: chunked\r\n".freeze
     end
-    alive &&= term
+    alive &&= (term || hdr_only)
     buf << (alive ? "Connection: keep-alive\r\n\r\n".freeze
                   : "Connection: close\r\n\r\n".freeze)
     case rv = kgio_syssend(buf, flags)
diff --git a/test/test_auto_chunk.rb b/test/test_auto_chunk.rb
index a97fe26..e1847f6 100644
--- a/test/test_auto_chunk.rb
+++ b/test/test_auto_chunk.rb
@@ -18,7 +18,12 @@ def test_auto_head
         app = Rack::Builder.new do
           use Rack::ContentType, "text/plain"
           run(lambda do |env|
-            [ 200, {}, %w(a b c) ]
+            case env['PATH_INFO']
+            when '/204'
+              [ 204, {}, [] ]
+            else
+              [ 200, {}, %w(a b c) ]
+            end
           end)
         end
         app(:rack, app) { listen "#{host}:#{port}" }
@@ -50,6 +55,21 @@ def test_auto_head
       assert_equal 200, res.code.to_i
       assert_equal 'abc', res.body
     end
+
+    s = TCPSocket.new(host, port)
+    s.write("GET /204 HTTP/1.1\r\nHost: example.com\r\n\r\n")
+    buf = s.readpartial(1024)
+    assert_match %r{\r\n\r\n\z}, buf
+    refute_match %r{^Transfer-Encoding}i, buf
+    assert_match %r{^Connection: keep-alive\r\n}, buf
+    assert_nil IO.select([s], nil, nil, 1), 'connection persists..'
+
+    # maek sure another on the same connection works
+    s.write("GET / HTTP/1.1\r\nHost: example.com\r\n\r\n")
+    buf = s.readpartial(1024)
+    assert_match %r{\AHTTP/1\.1 200}, buf
+    assert_match(%r{^Transfer-Encoding: chunked\r\n}, buf)
+    s.close
   ensure
     quit_wait(pid)
   end
-- 
EW

[PATCH 2/2] proxy_pass: do not chunk on bodyless upstream responses

[Eric Wong]

As with the previous commit
("response: do not set chunked header on bodyless responses"),
blindly setting "Transfer-Encoding: chunked" is wrong and
confuses "curl -T" on 204 responses, at least.
---
 lib/yahns/proxy_http_response.rb |  2 +-
 test/test_proxy_pass.rb          | 33 +++++++++++++++++++++++++++++++++
 2 files changed, 34 insertions(+), 1 deletion(-)

diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 74f5ce5..1776291 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -120,7 +120,7 @@ def proxy_res_headers(res, req_res)
 
     # chunk the response ourselves if the client supports it,
     # but the backend does not terminate properly
-    if alive && ! term
+    if alive && ! term && have_body
       if env['HTTP_VERSION'] == 'HTTP/1.1'.freeze
         res << "Transfer-Encoding: chunked\r\n".freeze
       else # we can't persist HTTP/1.0 and HTTP/0.9 w/o Content-Length
diff --git a/test/test_proxy_pass.rb b/test/test_proxy_pass.rb
index 4c4b53a..22f1802 100644
--- a/test/test_proxy_pass.rb
+++ b/test/test_proxy_pass.rb
@@ -154,6 +154,9 @@ def chunky.each
          env['rack.hijack'].call.close
          # should not be seen:
          [ 123, [ %w(Content-Type text/html), %w(Content-Length 0) ], [] ]
+        when '/204'
+          buf = env['rack.input'].read # drain
+          [ 204, {}, [] ]
         else
           buf = env['rack.input'].read
           [ 201, {
@@ -271,6 +274,7 @@ def test_proxy_pass
       end
     end
 
+    check_204_on_put(host, port)
     check_forbidden_put(host, port)
     check_eof_body(host, port)
     check_pipelining(host, port)
@@ -639,4 +643,33 @@ def check_forbidden_put(host, port)
   ensure
     to_close.each(&:close)
   end
+
+  def check_204_on_put(host, port)
+    s = TCPSocket.new(host, port)
+    s.write("PUT /204 HTTP/1.1\r\nHost: example.com\r\n" \
+            "Content-Length: 11\r\n" \
+            "Content-Type: application/octet-stream\r\n" \
+            "\r\nhello worldPUT")
+    buf = s.readpartial(1024)
+    assert_match %r{\AHTTP/1\.1 204}, buf
+    assert_match %r{\r\n\r\n\z}, buf
+    refute_match %r{^Transfer-Encoding}i, buf
+    refute_match %r{^Content-Length}i, buf
+    assert_match %r{^Connection: keep-alive\r\n}, buf
+    assert_nil IO.select([s], nil, nil, 1), 'connection persists..'
+
+    # make sure another on the same connection works
+    s.write(" / HTTP/1.1\r\nHost: example.com\r\n" \
+            "Content-Length: 11\r\n" \
+            "Content-Type: application/octet-stream\r\n" \
+            "\r\nhello world")
+    buf = s.readpartial(1024)
+    assert_match %r{\r\n\r\nhello world\z}, buf
+    assert_match %r{\AHTTP/1\.1 201}, buf
+    assert_match(%r{^Content-Length: 11\r\n}, buf)
+    assert_match %r{^Connection: keep-alive\r\n}, buf
+    assert_nil IO.select([s], nil, nil, 1), 'connection persists..'
+  ensure
+    s.close if s
+  end
 end
-- 
EW