Rainbows! Rack HTTP server user/dev discussion
 help / color / mirror / code / Atom feed
From: Claudio Poli <claudio-3HQ/CcOImoi171pxa8y+qA@public.gmane.org>
To: Rainbows! list <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: Transfer encoding chunked disables HAProxy gzip compression
Date: Sat, 3 Aug 2013 00:34:17 +0200	[thread overview]
Message-ID: <FE4BFDCF-538B-4D2D-8991-FE0047DF52AA@gmail.com> (raw)
In-Reply-To: <20130802205810.GA9486-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>

Ok, in staging everything seems to work correctly, we had to handle a case where one of our internal services couldn't communicate through compressed responses.

Willy told me the HAProxy compression issue for chunked request might be solved in Sept.


On 02/ago/2013, at 22:58, Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org> wrote:

> Claudio Poli <claudio-3HQ/CcOImoi171pxa8y+qA@public.gmane.org> wrote:
>> Hello, story time.
>> I was investigating why suddenly our service become slow in response
>> times, currenrly the app is served through Rainbows! behind HAProxy.
>> It turns out that the problem is HAProxy, which we use as
>> frontend/load balancer *and* for gzip compression of responses from
>> backends.
> <snip>
> OK, hopefully these problems get fixed in HAProxy soon
>> So, I tried to study rainbows and unicorn internals a bit (-N switch,
>> -E 'development, deploy, none') to attempt and disable chunked
>> middleware and replace with a Rack::ContentLength, so if and when the
>> body can be enumerated my responses would have a fixed size length to
>> feed the client.
>> It doesn't work: I tried to place them in config.ru, in rails'
>> application.rb, tried the -N switch, messed with -E but in staging
>> every response that comes out are of type http/1.1 chunked requests.
> This could be from Rails chunked responses, especially if you're using
> -N already.  You may have to dig into Rails internals a bit...
> When in doubt, try dropping Rails from the equation and using a bare
> bones "hello world" type Rack app (e.g. Rack::Lobster)
> Rack::ContentLength won't add a Content-Length header if the response
> was already chunked anywhere before.
>> So I went ahead and added Rack::Deflater to live with the chunked
>> response types and enable gzip compression as well, even if the cpu
>> spent time compressing moved from one load balancer to the app server.
> This is probably the better way to go anyways, especially if you add
> more Rainbows! workers and maintain few HAproxy processes.
>> I would like to know more about this behavior and based on which
>> factors Rainbows! or Unicorn set and pick the transfer-encoding or
>> content-lenght to use.
> It's down to middleware/Rack stack ordering (LIFO), and reading code of
> these middlewares.  Since Rack::Chunked runs first in the default
> Rack::Server configs, Chunked can insert its headers before
> ContentLength for HTTP 1.1 requests.
> Rack::Chunked won't force an existing Content-Length response to
> be chunked, it'll only make things chunked that don't have either
> Content-Length/Transfer-Encoding set.
> _______________________________________________
> Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
> http://rubyforge.org/mailman/listinfo/rainbows-talk
> Do not quote signatures (like this one) or top post when replying

Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
Do not quote signatures (like this one) or top post when replying

  parent reply	other threads:[~2013-08-02 23:04 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-08-02 19:26 Claudio Poli
     [not found] ` <CD8781D3-288B-4B61-85ED-16B8B15A90E9-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-08-02 20:03   ` Claudio Poli
     [not found]     ` <9C3CB455-F4F3-4DF4-BF67-1801547FE1B7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-08-02 20:46       ` Eric Wong
     [not found]         ` <20130802204647.GA7479-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-02 21:00           ` Eric Wong
     [not found]             ` <20130802210004.GB9486-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-02 21:35               ` Claudio Poli
2013-08-02 21:49               ` Claudio Poli
2013-08-02 20:58   ` Eric Wong
     [not found]     ` <20130802205810.GA9486-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-02 22:34       ` Claudio Poli [this message]
     [not found]         ` <FE4BFDCF-538B-4D2D-8991-FE0047DF52AA-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-08-02 23:11           ` Eric Wong
     [not found]             ` <20130802231115.GA24708-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-03  0:20               ` Claudio Poli
     [not found]                 ` <E85CF5AB-7FF6-4BE8-BAAC-5416E14D4A14-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-08-03  1:06                   ` Eric Wong
     [not found]                     ` <20130803010627.GA4521-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2013-08-03  1:53                       ` Claudio Poli
     [not found]                         ` <9580D763-16FA-4DB5-8CA7-0648DD6788CD-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>
2013-08-03  2:30                           ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

  List information: https://yhbt.net/rainbows/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=FE4BFDCF-538B-4D2D-8991-FE0047DF52AA@gmail.com \
    --to=claudio-3hq/ccoimoi171pxa8y+qa@public.gmane.org \
    --cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
    --subject='Re: Transfer encoding chunked disables HAProxy gzip compression' \


* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Code repositories for project(s) associated with this inbox:


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).