* ATTN "sendfile" RubyGem users
@ 2011-03-10 23:29 7% Eric Wong
0 siblings, 0 replies; 1+ results
From: Eric Wong @ 2011-03-10 23:29 UTC (permalink / raw)
To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw
For anybody[1] using the sendfile 1.0.0 RubyGem with Rainbows!,
the next release of Rainbows! will require an upgrade to sendfile 1.1.0
if you want to continue using sendfile(2) without IO.copy_stream[2].
If you're currently running sendfile 1.0.0 with Rainbows 3.1.0, it's
entirely safe to start using sendfile 1.1.0 right away.
I just pushed this out to git://bogomips.org/rainbows.git :
>From cd8a874d18fe01e11bb57b91186b6c9f712a4b3f Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson-rMlxZR9MS24@public.gmane.org>
Date: Thu, 10 Mar 2011 15:06:10 -0800
Subject: [PATCH] switch from IO#sendfile_nonblock to IO#trysendfile
IO#trysendfile does not raise exceptions for common EAGAIN
errors, making it far less expensive to use with the following
concurrency models:
* Coolio
* CoolioFiberSpawn
* Revactor
* FiberSpawn
* FiberPool
This requires the new sendfile 1.1.0 RubyGem and removes support
for the sendfile 1.0.0. All sendfile users must upgrade or be
left without sendfile(2) support. IO#sendfile behaves the same
if you're using a multi-threaded concurrency option, but we
don't detect nor use it unless IO#trysendfile exists.
---
TODO | 2 +-
lib/rainbows/coolio/client.rb | 25 +++++++++++++------------
lib/rainbows/epoll/client.rb | 11 ++++++-----
lib/rainbows/fiber/body.rb | 19 ++++++++++---------
lib/rainbows/response.rb | 10 +++++-----
lib/rainbows/revactor/client/methods.rb | 17 +++++++++--------
lib/rainbows/stream_file.rb | 2 +-
lib/rainbows/writer_thread_pool/client.rb | 2 +-
lib/rainbows/writer_thread_spawn/client.rb | 2 +-
9 files changed, 47 insertions(+), 43 deletions(-)
diff --git a/TODO b/TODO
index a49e0c5..0929aa3 100644
--- a/TODO
+++ b/TODO
@@ -11,7 +11,7 @@ care about.
* allow _OPTIONAL_ splice(2) with DevFdResponse under Linux
(splice is very broken under some older kernels)
-* use IO#sendfile_nonblock for EventMachine/NeverBlock
+* use IO#trysendfile for EventMachine/NeverBlock
* Open file cache Rack app/middleware (idea from nginx), since sendfile
(and IO.copy_stream) allows pread(2)-style offsets
diff --git a/lib/rainbows/coolio/client.rb b/lib/rainbows/coolio/client.rb
index 9853321..2de421a 100644
--- a/lib/rainbows/coolio/client.rb
+++ b/lib/rainbows/coolio/client.rb
@@ -125,11 +125,8 @@ class Rainbows::Coolio::Client < Coolio::IO
when true then return # #next! will clear this bit
when nil # fall through
else
- begin
- return stream_file_chunk(@deferred)
- rescue EOFError # expected at file EOF
- close_deferred # fall through
- end
+ return if stream_file_chunk(@deferred)
+ close_deferred # EOF, fall through
end
case @state
@@ -179,7 +176,7 @@ class Rainbows::Coolio::Client < Coolio::IO
KATO.delete(self)
end
- if IO.method_defined?(:sendfile_nonblock)
+ if IO.method_defined?(:trysendfile)
def defer_file(status, headers, body, alive, io, st)
if r = sendfile_range(status, headers)
status, headers, range = r
@@ -192,11 +189,15 @@ class Rainbows::Coolio::Client < Coolio::IO
end
def stream_file_chunk(sf) # +sf+ is a Rainbows::StreamFile object
- sf.offset += (n = @_io.sendfile_nonblock(sf, sf.offset, sf.count))
- 0 == (sf.count -= n) and raise EOFError
- enable_write_watcher
- rescue Errno::EAGAIN
- enable_write_watcher
+ case n = @_io.trysendfile(sf, sf.offset, sf.count)
+ when Integer
+ sf.offset += n
+ return if 0 == (sf.count -= n)
+ when :wait_writable
+ return enable_write_watcher
+ else
+ return
+ end while true
end
else
def defer_file(status, headers, body, alive, io, st)
@@ -205,7 +206,7 @@ class Rainbows::Coolio::Client < Coolio::IO
end
def stream_file_chunk(body)
- write(body.to_io.sysread(0x4000))
+ buf = body.to_io.read(0x4000) and write(buf)
end
end
diff --git a/lib/rainbows/epoll/client.rb b/lib/rainbows/epoll/client.rb
index a243d5d..b7b0c9e 100644
--- a/lib/rainbows/epoll/client.rb
+++ b/lib/rainbows/epoll/client.rb
@@ -183,15 +183,16 @@ module Rainbows::Epoll::Client
# returns +nil+ on EOF, :wait_writable if the client blocks
def stream_file(sf) # +sf+ is a Rainbows::StreamFile object
- begin
- sf.offset += (n = sendfile_nonblock(sf, sf.offset, sf.count))
+ case n = trysendfile(sf, sf.offset, sf.count)
+ when Integer
+ sf.offset += n
0 == (sf.count -= n) and return sf.close
- rescue Errno::EAGAIN
- return :wait_writable
+ else
+ return n # :wait_writable or nil
+ end while true
rescue
sf.close
raise
- end while true
end
def defer_file_stream(offset, count, io, body)
diff --git a/lib/rainbows/fiber/body.rb b/lib/rainbows/fiber/body.rb
index 872b1df..5b2c74b 100644
--- a/lib/rainbows/fiber/body.rb
+++ b/lib/rainbows/fiber/body.rb
@@ -5,19 +5,20 @@
# this is meant to be included _after_ Rainbows::Response::Body
module Rainbows::Fiber::Body # :nodoc:
- # the sendfile 1.0.0+ gem includes IO#sendfile_nonblock
- if IO.method_defined?(:sendfile_nonblock)
+ # the sendfile 1.1.0+ gem includes IO#trysendfile
+ if IO.method_defined?(:trysendfile)
def write_body_file(body, range)
sock, n, body = to_io, nil, body_to_io(body)
offset, count = range ? range : [ 0, body.stat.size ]
- begin
- offset += (n = sock.sendfile_nonblock(body, offset, count))
- rescue Errno::EAGAIN
+ case n = sock.trysendfile(body, offset, count)
+ when Integer
+ offset += n
+ return if 0 == (count -= n)
+ when :wait_writable
kgio_wait_writable
- retry
- rescue EOFError
- break
- end while (count -= n) > 0
+ else # nil
+ return
+ end while true
ensure
close_if_private(body)
end
diff --git a/lib/rainbows/response.rb b/lib/rainbows/response.rb
index 2c517f8..576ff8d 100644
--- a/lib/rainbows/response.rb
+++ b/lib/rainbows/response.rb
@@ -67,7 +67,7 @@ module Rainbows::Response
end
# generic response writer, used for most dynamically-generated responses
- # and also when IO.copy_stream and/or IO#sendfile_nonblock is unavailable
+ # and also when IO.copy_stream and/or IO#trysendfile is unavailable
def write_response(status, headers, body, alive)
write_headers(status, headers, alive)
write_body_each(body)
@@ -77,7 +77,7 @@ module Rainbows::Response
end
include Each
- if IO.method_defined?(:sendfile_nonblock)
+ if IO.method_defined?(:trysendfile)
module Sendfile
def write_body_file(body, range)
io = body_to_io(body)
@@ -90,7 +90,7 @@ module Rainbows::Response
end
if IO.respond_to?(:copy_stream)
- unless IO.method_defined?(:sendfile_nonblock)
+ unless IO.method_defined?(:trysendfile)
module CopyStream
def write_body_file(body, range)
range ? IO.copy_stream(body, self, range[1], range[0]) :
@@ -111,7 +111,7 @@ module Rainbows::Response
alias write_body_stream write_body_each
end # ! IO.respond_to?(:copy_stream)
- if IO.method_defined?(:sendfile_nonblock) || IO.respond_to?(:copy_stream)
+ if IO.method_defined?(:trysendfile) || IO.respond_to?(:copy_stream)
HTTP_RANGE = 'HTTP_RANGE'
Content_Range = 'Content-Range'.freeze
@@ -181,5 +181,5 @@ module Rainbows::Response
end
end
include ToPath
- end # IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+ end # IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
end
diff --git a/lib/rainbows/revactor/client/methods.rb b/lib/rainbows/revactor/client/methods.rb
index e9b39a3..b2e1847 100644
--- a/lib/rainbows/revactor/client/methods.rb
+++ b/lib/rainbows/revactor/client/methods.rb
@@ -1,7 +1,7 @@
# -*- encoding: binary -*-
# :enddoc:
module Rainbows::Revactor::Client::Methods
- if IO.method_defined?(:sendfile_nonblock)
+ if IO.method_defined?(:trysendfile)
def write_body_file(body, range)
body, client = body_to_io(body), @client
sock = @client.instance_variable_get(:@_io)
@@ -9,9 +9,11 @@ module Rainbows::Revactor::Client::Methods
write_complete = T[:"#{pfx}_write_complete", client]
closed = T[:"#{pfx}_closed", client]
offset, count = range ? range : [ 0, body.stat.size ]
- begin
- offset += (n = sock.sendfile_nonblock(body, offset, count))
- rescue Errno::EAGAIN
+ case n = sock.trysendfile(body, offset, count)
+ when Integer
+ offset += n
+ return if 0 == (count -= n)
+ when :wait_writable
# The @_write_buffer is empty at this point, trigger the
# on_readable method which in turn triggers on_write_complete
# even though nothing was written
@@ -21,10 +23,9 @@ module Rainbows::Revactor::Client::Methods
filter.when(write_complete) {}
filter.when(closed) { raise Errno::EPIPE }
end
- retry
- rescue EOFError
- break
- end while (count -= n) > 0
+ else # nil
+ return
+ end while true
ensure
close_if_private(body)
end
diff --git a/lib/rainbows/stream_file.rb b/lib/rainbows/stream_file.rb
index 11c84d4..4a77a2f 100644
--- a/lib/rainbows/stream_file.rb
+++ b/lib/rainbows/stream_file.rb
@@ -1,7 +1,7 @@
# -*- encoding: binary -*-
# :enddoc:
-# Used to keep track of file offsets in IO#sendfile_nonblock + evented
+# Used to keep track of file offsets in IO#trysendfile + evented
# models. We always maintain our own file offsets in userspace because
# because sendfile() implementations offer pread()-like idempotency for
# concurrency (multiple clients can read the same underlying file handle).
diff --git a/lib/rainbows/writer_thread_pool/client.rb b/lib/rainbows/writer_thread_pool/client.rb
index 526a623..f02826e 100644
--- a/lib/rainbows/writer_thread_pool/client.rb
+++ b/lib/rainbows/writer_thread_pool/client.rb
@@ -18,7 +18,7 @@ class Rainbows::WriterThreadPool::Client < Struct.new(:to_io, :q)
}
end
- if IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+ if IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
def write_response(status, headers, body, alive)
if body.respond_to?(:close)
write_response_close(status, headers, body, alive)
diff --git a/lib/rainbows/writer_thread_spawn/client.rb b/lib/rainbows/writer_thread_spawn/client.rb
index b4166fa..3106253 100644
--- a/lib/rainbows/writer_thread_spawn/client.rb
+++ b/lib/rainbows/writer_thread_spawn/client.rb
@@ -21,7 +21,7 @@ class Rainbows::WriterThreadSpawn::Client < Struct.new(:to_io, :q, :thr)
}
end
- if IO.respond_to?(:copy_stream) || IO.method_defined?(:sendfile_nonblock)
+ if IO.respond_to?(:copy_stream) || IO.method_defined?(:trysendfile)
def write_response(status, headers, body, alive)
self.q ||= queue_writer
if body.respond_to?(:close)
--
Eric Wong
[1] - I'm still fairly certain nobody runs Rainbows! at all in
production, so this message is addressed to imaginary users :D
[2] - Use Ruby 1.9.3dev for IO.copy_stream, 1.9.2 doesn't handle
client disconnects properly
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 7%]
Results 1-1 of 1 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2011-03-10 23:29 7% ATTN "sendfile" RubyGem users Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/rainbows.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).