[PATCH] Add some tolerance (RFC2616 sec. 19.3)

[PATCH] Add some tolerance (RFC2616 sec. 19.3)

[Mishael A Sibiryakov]

Hi all.

We're implementing client certificate authentication with nginx and
unicorn. 

Nginx configured in the following way:

proxy_set_header X-SSL-Client-Cert $ssl_client_cert;

When client submits certificate and nginx passes it to the unicorn,
unicorn responds with 400 (Bad Request). This caused because nginx
doesn't use "\r\n" they using just "\n" and multilne headers is failed
to parse (I've added test).

Accorording to RFC2616 section 19.3:
https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3

"The line terminator for message-header fields is the sequence CRLF.
However, we recommend that applications, when parsing such headers,
recognize a single LF as a line terminator and ignore the leading CR."

CRLF changed to ("\r\n" | "\n")

Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf1
76de05720f6be758f8b41b1f


PS: Googling "nginx unicorn ssl_client_cert" shows the problem. 

---
 ext/unicorn_http/unicorn_http_common.rl |  2 +-
 test/unit/test_http_parser.rb           | 16 ++++++++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/ext/unicorn_http/unicorn_http_common.rl
b/ext/unicorn_http/unicorn_http_common.rl
index cc1d455..507e570 100644
--- a/ext/unicorn_http/unicorn_http_common.rl
+++ b/ext/unicorn_http/unicorn_http_common.rl
@@ -4,7 +4,7 @@
 
 #### HTTP PROTOCOL GRAMMAR
 # line endings
-  CRLF = "\r\n";
+  CRLF = ("\r\n" | "\n");
 
 # character types
   CTL = (cntrl | 127);
diff --git a/test/unit/test_http_parser.rb
b/test/unit/test_http_parser.rb
index c72f7f2..4b1a16e 100644
--- a/test/unit/test_http_parser.rb
+++ b/test/unit/test_http_parser.rb
@@ -230,6 +230,22 @@ def test_nasty_pound_header
     assert_equal expect, req['HTTP_X_SSL_BULLSHIT']
   end
 
+  def test_multiline_header_0d0a
+    parser = HttpParser.new
+    parser.buf << "GET / HTTP/1.0\r\nX-Multiline-Header: foo
bar\r\n\tcha cha\r\n\tzha zha\r\n\r\n"
+    req = parser.env
+    assert_equal req, parser.parse
+    assert_equal 'foo bar cha cha zha zha',
req['HTTP_X_MULTILINE_HEADER']
+  end
+
+  def test_multiline_header_0a
+    parser = HttpParser.new
+    parser.buf << "GET / HTTP/1.0\nX-Multiline-Header: foo bar\n\tcha
cha\n\tzha zha\n\n"
+    req = parser.env
+    assert_equal req, parser.parse
+    assert_equal 'foo bar cha cha zha zha',
req['HTTP_X_MULTILINE_HEADER']
+  end
+
   def test_continuation_eats_leading_spaces
     parser = HttpParser.new
     header = "GET / HTTP/1.1\r\n" \
-- 
2.10.1

Re: [PATCH] Add some tolerance (RFC2616 sec. 19.3)

[Eric Wong]

Mishael A Sibiryakov <death@junki.org> wrote:
> Hi all.
> 
> We're implementing client certificate authentication with nginx and
> unicorn. 
> 
> Nginx configured in the following way:
> 
> proxy_set_header X-SSL-Client-Cert $ssl_client_cert;
> 
> When client submits certificate and nginx passes it to the unicorn,
> unicorn responds with 400 (Bad Request). This caused because nginx
> doesn't use "\r\n" they using just "\n" and multilne headers is failed
> to parse (I've added test).
> 
> Accorording to RFC2616 section 19.3:
> https://www.w3.org/Protocols/rfc2616/rfc2616-sec19.html#sec19.3
> 
> "The line terminator for message-header fields is the sequence CRLF.
> However, we recommend that applications, when parsing such headers,
> recognize a single LF as a line terminator and ignore the leading CR."
> 
> CRLF changed to ("\r\n" | "\n")

Thanks for that useful explanation.  Aside from the unnecessary,
"Hi all,", that is an informative commit message which justifies
the usefulness of that patch.

> Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf1
> 76de05720f6be758f8b41b1f

Unfortunately, the commit message in your git repo is lacking.
I've used the text at the top of your email.

> PS: Googling "nginx unicorn ssl_client_cert" shows the problem. 
> 
> ---
>  ext/unicorn_http/unicorn_http_common.rl |  2 +-
>  test/unit/test_http_parser.rb           | 16 ++++++++++++++++
>  2 files changed, 17 insertions(+), 1 deletion(-)

Eeep, Evolution does some strange things with formatting
whitespaces.  It looks like instructions for making it nicer are
in the Linux kernel:

https://bogomips.org/mirrors/linux.git/plain/Documentation/email-clients.txt?h=v4.8

<snip>

> +  def test_multiline_header_0d0a
> +    parser = HttpParser.new
> +    parser.buf << "GET / HTTP/1.0\r\nX-Multiline-Header: foo
> bar\r\n\tcha cha\r\n\tzha zha\r\n\r\n"

I expect code to be wrapped at 80 lines or less.  Fixed locally.
(I need big fonts, even 80 is a compromise, I really prefer 64)

Anyways, pushed to the "rfc2616-sec19.3" branch.

I've also uploaded a prerelease 5.1.0.4.gd5fbb to RubyGems
for folks without Ragel.

	gem install --pre unicorn -v 5.1.0.4.gd5fbb

Anything else?  Expect a 5.2.0 release in a few days or so.
Thanks.

Re: [PATCH] Add some tolerance (RFC2616 sec. 19.3)

[Mishael A Sibiryakov]

On Thu, 2016-10-20 at 17:55 +0000, Eric Wong wrote:

> Thanks for that useful explanation.  Aside from the unnecessary,
> "Hi all,", that is an informative commit message which justifies
> the usefulness of that patch.
> 
> > 
> > Github commit https://github.com/uno4ki/unicorn/commit/ed127b66e162aaf1
> > 76de05720f6be758f8b41b1f
> 
> Unfortunately, the commit message in your git repo is lacking.
> I've used the text at the top of your email.

It's just a temporary fork and I'm too lazy ))

> Eeep, Evolution does some strange things with formatting
> whitespaces.  It looks like instructions for making it nicer are
> in the Linux kernel:
> 
> https://bogomips.org/mirrors/linux.git/plain/Documentation/email-clients.txt?h=v4.8

Whoops, my fault. 

> > +  def test_multiline_header_0d0a
> > +    parser = HttpParser.new
> > +    parser.buf << "GET / HTTP/1.0\r\nX-Multiline-Header: foo
> > bar\r\n\tcha cha\r\n\tzha zha\r\n\r\n"
> 
> I expect code to be wrapped at 80 lines or less.  Fixed locally.
> (I need big fonts, even 80 is a compromise, I really prefer 64)

Line 221 with "ssl bullshit" guided me to this.

> Anyways, pushed to the "rfc2616-sec19.3" branch.
> 
> I've also uploaded a prerelease 5.1.0.4.gd5fbb to RubyGems
> for folks without Ragel.
> 
> 	gem install --pre unicorn -v 5.1.0.4.gd5fbb
> 
> Anything else?  Expect a 5.2.0 release in a few days or so.
> Thanks.

Thanks for the quick acceptance.

Re: [PATCH] Add some tolerance (RFC2616 sec. 19.3)

[Eric Wong]

Mishael A Sibiryakov <death@junki.org> wrote:
> It's just a temporary fork and I'm too lazy ))

<snip>

> Whoops, my fault. 

No worries :)

> > > +  def test_multiline_header_0d0a
> > > +    parser = HttpParser.new
> > > +    parser.buf << "GET / HTTP/1.0\r\nX-Multiline-Header: foo
> > > bar\r\n\tcha cha\r\n\tzha zha\r\n\r\n"
> > 
> > I expect code to be wrapped at 80 lines or less.  Fixed locally.
> > (I need big fonts, even 80 is a compromise, I really prefer 64)
> 
> Line 221 with "ssl bullshit" guided me to this.

Ah, that's an old test from Mongrel, probably not worth fixing;
just trying to keep new code cleaner.

Re: [PATCH] Add some tolerance (RFC2616 sec. 19.3)

[Mishael A Sibiryakov]

On Thu, 2016-10-20 at 20:50 +0000, Eric Wong wrote:

> > > > +  def test_multiline_header_0d0a
> > > > +    parser = HttpParser.new
> > > > +    parser.buf << "GET / HTTP/1.0\r\nX-Multiline-Header: foo
> > > > bar\r\n\tcha cha\r\n\tzha zha\r\n\r\n"
> > > 
> > > I expect code to be wrapped at 80 lines or less.  Fixed locally.
> > > (I need big fonts, even 80 is a compromise, I really prefer 64)
> > 
> > Line 221 with "ssl bullshit" guided me to this.
> 
> Ah, that's an old test from Mongrel, probably not worth fixing;
> just trying to keep new code cleaner.

It's a good desire :) I'll comply next time.