* Unicorn configuration to increase max header size
@ 2014-11-20 21:54 Jim Zhan
2014-11-20 21:59 ` Eric Wong
0 siblings, 1 reply; 6+ messages in thread
From: Jim Zhan @ 2014-11-20 21:54 UTC (permalink / raw)
To: unicorn-public
Hi,
We are using Unicorn as the http server for one of our ruby applications
and we recently encountered an issue that some browsers won't limit the
cookie size so we will get requests with http header greater than 8k and
users are receiving "400-bad request". Is there a way to increase the
maximum allowed header size? I searched online but didn't find a lot of
useful information on it.
Thanks,
Jim Zhan
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unicorn configuration to increase max header size
2014-11-20 21:54 Unicorn configuration to increase max header size Jim Zhan
@ 2014-11-20 21:59 ` Eric Wong
2014-11-20 23:35 ` Jim Zhan
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2014-11-20 21:59 UTC (permalink / raw)
To: Jim Zhan; +Cc: unicorn-public
Jim Zhan <cjzhan2000@gmail.com> wrote:
> We are using Unicorn as the http server for one of our ruby applications
> and we recently encountered an issue that some browsers won't limit the
> cookie size so we will get requests with http header greater than 8k and
> users are receiving "400-bad request". Is there a way to increase the
> maximum allowed header size? I searched online but didn't find a lot of
> useful information on it.
This is subject to change in the next major release, but you can
change it in unicorn 4.x using:
Unicorn::HttpRequest.max_header_len = <number>
However, the default is already 112K, so I'm wondering if the 8K is
the result of your nginx configuration or similar.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unicorn configuration to increase max header size
2014-11-20 21:59 ` Eric Wong
@ 2014-11-20 23:35 ` Jim Zhan
2014-11-21 0:31 ` Eric Wong
0 siblings, 1 reply; 6+ messages in thread
From: Jim Zhan @ 2014-11-20 23:35 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Hi Eric,
Thank you for the quick reply. I checked out hosts and we are using Unicorn
3.4.1. Unfortunately there is only one setting for client_body_buffer_size.
So how does this parameter work? Will it only put a limitation on the body
itself or it applied proportionally to header and body (e.g., header 8k,
body 104k, etc).
We did experiments using curl by sending header exceeding 8k manually and I
am getting 404. So it's unicorn itself, not nginx that has the 8k header
size limitation.
The command we used for the experiment:
curl -v -H "$(./http-header-pumper.bat 8000)" <service_url>
The script we used to generate header:
#!/bin/bash
printf "x-header-pump: "
for ((i=0; i<$1; i++))
do
let "n = $i % 10"
if [ $n = 0 ]; then
printf "_"
else
printf "%d" $n
fi
done
Thank you and I am looking forward to hearing from you soon on the issue!
Rgds,
Jim Zhan
On Thu, Nov 20, 2014 at 1:59 PM, Eric Wong <e@80x24.org> wrote:
> Jim Zhan <cjzhan2000@gmail.com> wrote:
> > We are using Unicorn as the http server for one of our ruby applications
> > and we recently encountered an issue that some browsers won't limit the
> > cookie size so we will get requests with http header greater than 8k and
> > users are receiving "400-bad request". Is there a way to increase the
> > maximum allowed header size? I searched online but didn't find a lot of
> > useful information on it.
>
> This is subject to change in the next major release, but you can
> change it in unicorn 4.x using:
>
> Unicorn::HttpRequest.max_header_len = <number>
>
> However, the default is already 112K, so I'm wondering if the 8K is
> the result of your nginx configuration or similar.
>
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Unicorn configuration to increase max header size
2014-11-20 23:35 ` Jim Zhan
@ 2014-11-21 0:31 ` Eric Wong
2014-11-21 1:33 ` Jim Zhan
0 siblings, 1 reply; 6+ messages in thread
From: Eric Wong @ 2014-11-21 0:31 UTC (permalink / raw)
To: Jim Zhan; +Cc: unicorn-public
Jim Zhan <cjzhan2000@gmail.com> wrote:
> Hi Eric,
>
> Thank you for the quick reply. I checked out hosts and we are using Unicorn
> 3.4.1. Unfortunately there is only one setting for client_body_buffer_size.
> So how does this parameter work? Will it only put a limitation on the body
> itself or it applied proportionally to header and body (e.g., header 8k,
> body 104k, etc).
client_body_buffer_size in unicorn is only for request bodies (uploads),
and not relevant to header sizes.
> We did experiments using curl by sending header exceeding 8k manually and I
> am getting 404. So it's unicorn itself, not nginx that has the 8k header
> size limitation.
I suspect you're hitting the nginx large_client_header_buffers default
limit of 8K:
http://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers
I checked the unicorn source, and ext/unicorn_http/global_variables.h
defines the maximum field value as 80K, 10 times more than what you're
seeing:
DEF_MAX_LENGTH(FIELD_VALUE, 80 * 1024);
This value was inherited from Mongrel many years ago and never changed.
> The command we used for the experiment:
> curl -v -H "$(./http-header-pumper.bat 8000)" <service_url>
I just tried your script with the following config.ru to hit unicorn
directly (no nginx), and I got the expected lobster response.
$ unicorn -E none config.ru
----------- config.ru -----------
require 'rack/lobster'
use Rack::ContentLength
run Rack::Lobster.new
--------------------------------
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unicorn configuration to increase max header size
2014-11-21 0:31 ` Eric Wong
@ 2014-11-21 1:33 ` Jim Zhan
2014-11-22 3:15 ` Jim Zhan
0 siblings, 1 reply; 6+ messages in thread
From: Jim Zhan @ 2014-11-21 1:33 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Thank you. Will check out tomorrow on our production hosts based on your
comments and directions.
On Thu, Nov 20, 2014 at 4:31 PM, Eric Wong <e@80x24.org> wrote:
> Jim Zhan <cjzhan2000@gmail.com> wrote:
> > Hi Eric,
> >
> > Thank you for the quick reply. I checked out hosts and we are using
> Unicorn
> > 3.4.1. Unfortunately there is only one setting for
> client_body_buffer_size.
> > So how does this parameter work? Will it only put a limitation on the
> body
> > itself or it applied proportionally to header and body (e.g., header 8k,
> > body 104k, etc).
>
> client_body_buffer_size in unicorn is only for request bodies (uploads),
> and not relevant to header sizes.
>
> > We did experiments using curl by sending header exceeding 8k manually
> and I
> > am getting 404. So it's unicorn itself, not nginx that has the 8k header
> > size limitation.
>
> I suspect you're hitting the nginx large_client_header_buffers default
> limit of 8K:
>
>
> http://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers
>
> I checked the unicorn source, and ext/unicorn_http/global_variables.h
> defines the maximum field value as 80K, 10 times more than what you're
> seeing:
>
> DEF_MAX_LENGTH(FIELD_VALUE, 80 * 1024);
>
> This value was inherited from Mongrel many years ago and never changed.
>
> > The command we used for the experiment:
> > curl -v -H "$(./http-header-pumper.bat 8000)" <service_url>
>
> I just tried your script with the following config.ru to hit unicorn
> directly (no nginx), and I got the expected lobster response.
>
> $ unicorn -E none config.ru
> ----------- config.ru -----------
> require 'rack/lobster'
> use Rack::ContentLength
> run Rack::Lobster.new
> --------------------------------
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Unicorn configuration to increase max header size
2014-11-21 1:33 ` Jim Zhan
@ 2014-11-22 3:15 ` Jim Zhan
0 siblings, 0 replies; 6+ messages in thread
From: Jim Zhan @ 2014-11-22 3:15 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Tried out and indeed it is because of the header size limitation of load
balancer, not the rails app itself. Thank you for your answer!
On Thu, Nov 20, 2014 at 5:33 PM, Jim Zhan <cjzhan2000@gmail.com> wrote:
> Thank you. Will check out tomorrow on our production hosts based on your
> comments and directions.
>
>
> On Thu, Nov 20, 2014 at 4:31 PM, Eric Wong <e@80x24.org> wrote:
>
>> Jim Zhan <cjzhan2000@gmail.com> wrote:
>> > Hi Eric,
>> >
>> > Thank you for the quick reply. I checked out hosts and we are using
>> Unicorn
>> > 3.4.1. Unfortunately there is only one setting for
>> client_body_buffer_size.
>> > So how does this parameter work? Will it only put a limitation on the
>> body
>> > itself or it applied proportionally to header and body (e.g., header 8k,
>> > body 104k, etc).
>>
>> client_body_buffer_size in unicorn is only for request bodies (uploads),
>> and not relevant to header sizes.
>>
>> > We did experiments using curl by sending header exceeding 8k manually
>> and I
>> > am getting 404. So it's unicorn itself, not nginx that has the 8k header
>> > size limitation.
>>
>> I suspect you're hitting the nginx large_client_header_buffers default
>> limit of 8K:
>>
>>
>> http://nginx.org/en/docs/http/ngx_http_core_module.html#large_client_header_buffers
>>
>> I checked the unicorn source, and ext/unicorn_http/global_variables.h
>> defines the maximum field value as 80K, 10 times more than what you're
>> seeing:
>>
>> DEF_MAX_LENGTH(FIELD_VALUE, 80 * 1024);
>>
>> This value was inherited from Mongrel many years ago and never changed.
>>
>> > The command we used for the experiment:
>> > curl -v -H "$(./http-header-pumper.bat 8000)" <service_url>
>>
>> I just tried your script with the following config.ru to hit unicorn
>> directly (no nginx), and I got the expected lobster response.
>>
>> $ unicorn -E none config.ru
>> ----------- config.ru -----------
>> require 'rack/lobster'
>> use Rack::ContentLength
>> run Rack::Lobster.new
>> --------------------------------
>>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-11-22 3:15 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-20 21:54 Unicorn configuration to increase max header size Jim Zhan
2014-11-20 21:59 ` Eric Wong
2014-11-20 23:35 ` Jim Zhan
2014-11-21 0:31 ` Eric Wong
2014-11-21 1:33 ` Jim Zhan
2014-11-22 3:15 ` Jim Zhan
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).