HEAD responses contain body

HEAD responses contain body

[Jonathan Rudenberg]

RFC 2616 section 9.4[1] states:

> The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.

A HEAD request against this simple Rack app running on unicorn-4.6.2:

    require 'rack'

    run lambda { |env| [200, {}, []] }

Looks like this on the wire:

    HEAD / HTTP/1.1
    User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
    Host: localhost:8080
    Accept: */*

    HTTP/1.1 200 OK
    Date: Thu, 13 Jun 2013 16:04:55 GMT
    Status: 200 OK
    Connection: close
    Transfer-Encoding: chunked

    0

    HTTP/1.1 500 Internal Server Error

As you can see, not only is there a zero-length chunked encoding body, but for some unknown reason there is a 500 response with no body as well.

Please cc any responses directly to me, as I do not subscribe to this list.

Cheers,

Jonathan

[1] https://tools.ietf.org/html/rfc2616#section-9.4
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Eric Wong]

Jonathan Rudenberg <jonathan@titanous.com> wrote:
> RFC 2616 section 9.4[1] states:
> 
> > The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
> 
> A HEAD request against this simple Rack app running on unicorn-4.6.2:
> 
>     require 'rack'
> 

+     use Rack::Head

>     run lambda { |env| [200, {}, []] }

The Rack::Head middleware should be used to correctly strip HEAD
responses of their bodies (frameworks such as Rails/Sinatra should
already add Rack::Head to the middleware stack for you)

> Looks like this on the wire:
> 
>     HEAD / HTTP/1.1
>     User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
>     Host: localhost:8080
>     Accept: */*
> 
>     HTTP/1.1 200 OK
>     Date: Thu, 13 Jun 2013 16:04:55 GMT
>     Status: 200 OK
>     Connection: close
>     Transfer-Encoding: chunked
> 
>     0
> 
>     HTTP/1.1 500 Internal Server Error

> As you can see, not only is there a zero-length chunked encoding body,
> but for some unknown reason there is a 500 response with no body as
> well.

Try using "-d" on the command-line to enable debugging to see what the
error is (and check the logs/stderr output).

Also, what RACK_ENV (or -E/--env) are you using?  It could be the
incorrect HEAD response tripping Rack::Lint under development mode.

> Please cc any responses directly to me, as I do not subscribe to this list.

Done :>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Jonathan Rudenberg]

On Jun 13, 2013, at 2:22 PM, Eric Wong <normalperson@yhbt.net> wrote:

> Jonathan Rudenberg <jonathan@titanous.com> wrote:
>> RFC 2616 section 9.4[1] states:
>> 
>>> The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
>> 
>> A HEAD request against this simple Rack app running on unicorn-4.6.2:
>> 
>>    require 'rack'
>> 
> 
> +     use Rack::Head
> 
>>    run lambda { |env| [200, {}, []] }
> 
> The Rack::Head middleware should be used to correctly strip HEAD
> responses of their bodies (frameworks such as Rails/Sinatra should
> already add Rack::Head to the middleware stack for you)

This does not change the result, as the Rack::Head implementation looks like this:

    def call(env)
      status, headers, body = @app.call(env)

      if env["REQUEST_METHOD"] == "HEAD"
        body.close if body.respond_to? :close
        [status, headers, []]
      else
        [status, headers, body]
      end
    end

>> Looks like this on the wire:
>> 
>>    HEAD / HTTP/1.1
>>    User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8x zlib/1.2.5
>>    Host: localhost:8080
>>    Accept: */*
>> 
>>    HTTP/1.1 200 OK
>>    Date: Thu, 13 Jun 2013 16:04:55 GMT
>>    Status: 200 OK
>>    Connection: close
>>    Transfer-Encoding: chunked
>> 
>>    0
>> 
>>    HTTP/1.1 500 Internal Server Error
> 
>> As you can see, not only is there a zero-length chunked encoding body,
>> but for some unknown reason there is a 500 response with no body as
>> well.
> 
> Try using "-d" on the command-line to enable debugging to see what the
> error is (and check the logs/stderr output).

    Exception `Errno::ENOTCONN' at /Users/titanous/.gem/ruby/1.9.3/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:565 - Socket is not connected

> Also, what RACK_ENV (or -E/--env) are you using?  It could be the
> incorrect HEAD response tripping Rack::Lint under development mode.

None, specified, I'm booting unicorn with no configuration or flags specified.

> 
>> Please cc any responses directly to me, as I do not subscribe to this list.
> 
> Done :>

_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Eric Wong]

Jonathan Rudenberg <jonathan@titanous.com> wrote:
> On Jun 13, 2013, at 2:22 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Jonathan Rudenberg <jonathan@titanous.com> wrote:
> >> RFC 2616 section 9.4[1] states:
> >> 
> >>> The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
> >> 
> >> A HEAD request against this simple Rack app running on unicorn-4.6.2:
> >> 
> >>    require 'rack'
> >> 
> > 
> > +     use Rack::Head
> > 
> >>    run lambda { |env| [200, {}, []] }
> > 
> > The Rack::Head middleware should be used to correctly strip HEAD
> > responses of their bodies (frameworks such as Rails/Sinatra should
> > already add Rack::Head to the middleware stack for you)
> 
> This does not change the result, as the Rack::Head implementation looks like this:
> 
>     def call(env)
>       status, headers, body = @app.call(env)
> 
>       if env["REQUEST_METHOD"] == "HEAD"
>         body.close if body.respond_to? :close
>         [status, headers, []]
>       else
>         [status, headers, body]
>       end
>     end

OK, I think you were hitting another problem because you were lacking
Rack::ContentType

Try the following:
-----------------------8<---------------------
require 'rack'
use Rack::ContentLength # less ambiguous than Rack::Chunked adding '0'
use Rack::Head
use Rack::ContentType
run lambda { |env| [200, {}, []] }
-----------------------8<---------------------

I added the Rack::ContentLength (it's already in the default middleware
stack) since I believe Rack::Chunked adding the '0' is a violation of
rfc2616... I'll need to read more closely to be sure.

> >>    HTTP/1.1 500 Internal Server Error
> > 
> >> As you can see, not only is there a zero-length chunked encoding body,
> >> but for some unknown reason there is a 500 response with no body as
> >> well.
> > 
> > Try using "-d" on the command-line to enable debugging to see what the
> > error is (and check the logs/stderr output).
> 
>     Exception `Errno::ENOTCONN' at /Users/titanous/.gem/ruby/1.9.3/gems/unicorn-4.6.2/lib/unicorn/http_server.rb:565 - Socket is not connected

Ugh, that's an unfortunate side effect of the client closing the
connection, first :/

> > Also, what RACK_ENV (or -E/--env) are you using?  It could be the
> > incorrect HEAD response tripping Rack::Lint under development mode.
> 
> None, specified, I'm booting unicorn with no configuration or flags specified.

That defaults the RACK_ENV to to "development", so you got
Rack::ContentLength, Rack::Chunked, Rack::CommonLogger,
Rack::ShowExceptions and Rack::Lint
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Jonathan Rudenberg]

On Jun 13, 2013, at 3:21 PM, Eric Wong <normalperson@yhbt.net> wrote:

> Jonathan Rudenberg <jonathan@titanous.com> wrote:
>> On Jun 13, 2013, at 2:22 PM, Eric Wong <normalperson@yhbt.net> wrote:
>>> Jonathan Rudenberg <jonathan@titanous.com> wrote:
>>>> RFC 2616 section 9.4[1] states:
>>>> 
>>>>> The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
>>>> 
>>>> A HEAD request against this simple Rack app running on unicorn-4.6.2:
>>>> 
>>>>   require 'rack'
>>>> 
>>> 
>>> +     use Rack::Head
>>> 
>>>>   run lambda { |env| [200, {}, []] }
>>> 
>>> The Rack::Head middleware should be used to correctly strip HEAD
>>> responses of their bodies (frameworks such as Rails/Sinatra should
>>> already add Rack::Head to the middleware stack for you)
>> 
>> This does not change the result, as the Rack::Head implementation looks like this:
>> 
>>    def call(env)
>>      status, headers, body = @app.call(env)
>> 
>>      if env["REQUEST_METHOD"] == "HEAD"
>>        body.close if body.respond_to? :close
>>        [status, headers, []]
>>      else
>>        [status, headers, body]
>>      end
>>    end
> 
> OK, I think you were hitting another problem because you were lacking
> Rack::ContentType
> 
> Try the following:
> -----------------------8<---------------------
> require 'rack'
> use Rack::ContentLength # less ambiguous than Rack::Chunked adding '0'
> use Rack::Head
> use Rack::ContentType
> run lambda { |env| [200, {}, []] }
> -----------------------8<---------------------

Thanks, this stack works.

> I added the Rack::ContentLength (it's already in the default middleware
> stack) since I believe Rack::Chunked adding the '0' is a violation of
> rfc2616... I'll need to read more closely to be sure.

Hmm, so this is a bug in Rack::Chunked? My reading of the spec says that the '0' is incorrect.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Jonathan Rudenberg]

On Jun 13, 2013, at 3:28 PM, Jonathan Rudenberg <jonathan@titanous.com> wrote:

> 
> On Jun 13, 2013, at 3:21 PM, Eric Wong <normalperson@yhbt.net> wrote:
> 
>> Jonathan Rudenberg <jonathan@titanous.com> wrote:
>>> On Jun 13, 2013, at 2:22 PM, Eric Wong <normalperson@yhbt.net> wrote:
>>>> Jonathan Rudenberg <jonathan@titanous.com> wrote:
>>>>> RFC 2616 section 9.4[1] states:
>>>>> 
>>>>>> The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.
>>>>> 
>>>>> A HEAD request against this simple Rack app running on unicorn-4.6.2:
>>>>> 
>>>>>  require 'rack'
>>>>> 
>>>> 
>>>> +     use Rack::Head
>>>> 
>>>>>  run lambda { |env| [200, {}, []] }
>>>> 
>>>> The Rack::Head middleware should be used to correctly strip HEAD
>>>> responses of their bodies (frameworks such as Rails/Sinatra should
>>>> already add Rack::Head to the middleware stack for you)
>>> 
>>> This does not change the result, as the Rack::Head implementation looks like this:
>>> 
>>>   def call(env)
>>>     status, headers, body = @app.call(env)
>>> 
>>>     if env["REQUEST_METHOD"] == "HEAD"
>>>       body.close if body.respond_to? :close
>>>       [status, headers, []]
>>>     else
>>>       [status, headers, body]
>>>     end
>>>   end
>> 
>> OK, I think you were hitting another problem because you were lacking
>> Rack::ContentType
>> 
>> Try the following:
>> -----------------------8<---------------------
>> require 'rack'
>> use Rack::ContentLength # less ambiguous than Rack::Chunked adding '0'
>> use Rack::Head
>> use Rack::ContentType
>> run lambda { |env| [200, {}, []] }
>> -----------------------8<---------------------
> 
> Thanks, this stack works.
> 
>> I added the Rack::ContentLength (it's already in the default middleware
>> stack) since I believe Rack::Chunked adding the '0' is a violation of
>> rfc2616... I'll need to read more closely to be sure.
> 
> Hmm, so this is a bug in Rack::Chunked? My reading of the spec says that the '0' is incorrect.

Actually, the solution is that Rack::Head needs to come before Rack::Chunked. Perhaps Rack's default development stack should include this?

_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Eric Wong]

Jonathan Rudenberg <jonathan@titanous.com> wrote:
> On Jun 13, 2013, at 3:21 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Try the following:

<snip>

> Thanks, this stack works.

Good to know!

> > I added the Rack::ContentLength (it's already in the default middleware
> > stack) since I believe Rack::Chunked adding the '0' is a violation of
> > rfc2616... I'll need to read more closely to be sure.
> 
> Hmm, so this is a bug in Rack::Chunked? My reading of the spec says
> that the '0' is incorrect.

I think so, too.  Can you report this to the Rack folks?  (Or I can do it)

I've reproduced the issue with "rackup -s thin", too.
"rackup -s webrick" sets ContentLength, but that could be because
webrick was meant to work without Rack and adds its own headers.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: HEAD responses contain body

[Jonathan Rudenberg]

On Jun 13, 2013, at 3:45 PM, Eric Wong <normalperson@yhbt.net> wrote:

> I think so, too.  Can you report this to the Rack folks?  (Or I can do it)

Done: https://github.com/rack/rack/issues/574
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying