* [ANN] unicorn 0.93.2 - more compatible with Rails
@ 2009-10-07 8:58 7% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2009-10-07 8:58 UTC (permalink / raw)
To: mongrel-unicorn
Unicorn is a HTTP server for Rack applications designed to take
advantage of features in Unix/Unix-like kernels and only serve fast
clients on low-latency, high-bandwidth connections. Slow clients should
only be served by placing a reverse proxy capable of fully-buffering
both the the request and response in between Unicorn and slow clients.
* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git
Thanks to Chris Wanstrath for reporting issues with large
POST bodies and for helping me test.
Changes:
Avoid truncated POST bodies with URL-encoded forms in Rails
by switching TeeInput to use read-in-full semantics (only) when
a Content-Length: header exists. Chunked request bodies
continue to exhibit readpartial semantics to support
simultaneous bidirectional chunking.
The lack of return value checking in Rails to protect against a
short ios.read(length) is entirely reasonable even if not
pedantically correct. Most ios.read(length) implementations
return the full amount requested except right before EOF.
A ticket has been opened here to track the issue:
https://rails.lighthouseapp.com/projects/8994/tickets/3343
Also there are some minor documentation improvements.
Eric Wong (8):
Fix NEWS generation on single-paragraph tag messages
Include GPLv2 in docs
doc: make it clear contributors retain copyrights
TODO: removed Rainbows! (see rainbows.rubyforge.org)
Document the START_CTX hash contents
more-compatible TeeInput#read for POSTs with Content-Length
tests for read-in-full vs readpartial semantics
unicorn 0.93.2
--
Eric Wong
^ permalink raw reply [relevance 7%]
* Re: POST Body Truncated
@ 2009-10-07 3:04 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2009-10-07 3:04 UTC (permalink / raw)
To: Chris Wanstrath; +Cc: mongrel-unicorn
Chris Wanstrath <chris@ozmm.org> wrote:
> On Tue, Oct 6, 2009 at 3:52 PM, Eric Wong <normalperson@yhbt.net> wrote:
>
> > OK, here's a workaround that should work for now. I have to hit the
> > road in a few minutes but will be back on a computer in a few hours.
> >
> > This only affects older Rails (and I'm supposed to still be supporting
> > 1.2.x!) and its interaction with a wrapped CGI.stdinput somewhere is
> > going bad...
>
> This works!
>
> I suppose we should upgrade to a newer Rails :)
>
> As usual, thanks a million.
Here's a real patch with lots of documentation I just pushed out, still
working on automated test cases. Can you let me know how it works?
Thanks.
>From 438c99aec2d74489fa89b3a6c60d1fb41bb2f7e6 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Tue, 6 Oct 2009 19:45:05 -0700
Subject: [PATCH] more-compatible TeeInput#read for POSTs with Content-Length
There are existing applications and libraries that don't check
the return value of env['rack.input'].read(length) (like Rails
:x). Those applications became broken under the IO#readpartial
semantics of TeeInput#read when handling larger request bodies.
We'll preserve the IO#readpartial semantics _only_ when handling
chunked requests (as long as Rack allows it, it's useful for
real-time processing of audio/video streaming uploads,
especially with Rainbows! and mobile clients) but use
read-in-full semantics for TeeInput#read on requests with a
known Content-Length.
---
lib/unicorn/tee_input.rb | 43 +++++++++++++++++++++++++++++++++++++++++--
1 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb
index 96a053a..188e2ea 100644
--- a/lib/unicorn/tee_input.rb
+++ b/lib/unicorn/tee_input.rb
@@ -41,6 +41,26 @@ module Unicorn
@size = tmp_size
end
+ # call-seq:
+ # ios = env['rack.input']
+ # ios.read([length [, buffer ]]) => string, buffer, or nil
+ #
+ # Reads at most length bytes from the I/O stream, or to the end of
+ # file if length is omitted or is nil. length must be a non-negative
+ # integer or nil. If the optional buffer argument is present, it
+ # must reference a String, which will receive the data.
+ #
+ # At end of file, it returns nil or "" depend on length.
+ # ios.read() and ios.read(nil) returns "".
+ # ios.read(length [, buffer]) returns nil.
+ #
+ # If the Content-Length of the HTTP request is known (as is the common
+ # case for POST requests), then ios.read(length [, buffer]) will block
+ # until the specified length is read (or it is the last chunk).
+ # Otherwise, for uncommon "Transfer-Encoding: chunked" requests,
+ # ios.read(length [, buffer]) will return immediately if there is
+ # any data and only block when nothing is available (providing
+ # IO#readpartial semantics).
def read(*args)
socket or return @tmp.read(*args)
@@ -55,9 +75,9 @@ module Unicorn
rv = args.shift || @buf2.dup
diff = tmp_size - @tmp.pos
if 0 == diff
- tee(length, rv)
+ ensure_length(tee(length, rv), length)
else
- @tmp.read(diff > length ? length : diff, rv)
+ ensure_length(@tmp.read(diff > length ? length : diff, rv), length)
end
end
end
@@ -130,5 +150,24 @@ module Unicorn
StringIO === @tmp ? @tmp.size : @tmp.stat.size
end
+ # tee()s into +buf+ until it is of +length+ bytes (or until
+ # we've reached the Content-Length of the request body).
+ # Returns +buf+ (the exact object, not a duplicate)
+ # To continue supporting applications that need near-real-time
+ # streaming input bodies, this is a no-op for
+ # "Transfer-Encoding: chunked" requests.
+ def ensure_length(buf, length)
+ # @size is nil for chunked bodies, so we can't ensure length for those
+ # since they could be streaming bidirectionally and we don't want to
+ # block the caller in that case.
+ return buf if buf.nil? || @size.nil?
+
+ while buf.size < length && @size != @tmp.pos
+ buf << tee(length - buf.size, @buf2)
+ end
+
+ buf
+ end
+
end
end
--
Eric Wong
^ permalink raw reply related [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2009-10-06 21:13 POST Body Truncated Chris Wanstrath
2009-10-06 21:22 ` Eric Wong
2009-10-06 21:26 ` Chris Wanstrath
2009-10-06 21:30 ` Eric Wong
2009-10-06 21:46 ` Chris Wanstrath
2009-10-06 21:50 ` Eric Wong
2009-10-06 21:58 ` Chris Wanstrath
2009-10-06 22:32 ` Eric Wong
2009-10-06 22:52 ` Eric Wong
2009-10-06 22:57 ` Chris Wanstrath
2009-10-07 3:04 6% ` Eric Wong
2009-10-07 8:58 7% [ANN] unicorn 0.93.2 - more compatible with Rails Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).