* Re: [PATCH] epollexclusive: avoid Ruby object allocation for buffer
2023-02-15 7:58 14% [PATCH] epollexclusive: avoid Ruby object allocation for buffer Eric Wong
@ 2023-03-01 23:12 0% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2023-03-01 23:12 UTC (permalink / raw)
To: unicorn-public
Eric Wong <bofh@yhbt.net> wrote:
> Leaving a dangling Ruby object around is suboptimal since it
> adds to GC pressure.
>
> `__attribute__((__cleanup__(foo)))' is an old gcc extension to
> provide automatic cleanup functionality by running a pre-declared
> function at the end of scope.
Unfortunately, this can't be usable due to exceptions.
Even if we avoid throwing exceptions ourselves; Thread#raise can
be called by a Rack app which spawns a background thread
> n = (long)rb_thread_call_without_gvl(do_wait, &epw, RUBY_UBF_IO, NULL);
Thread#raise can hit rb_thread_call_wihtout_gvl
> - if (n < 0) {
> - if (errno != EINTR) rb_sys_fail("epoll_wait");
> - n = 0;
> - }
> + if (n < 0 && errno != EINTR) rb_sys_fail("epoll_wait");
> /* Linux delivers events in order received */
> for (i = 0; i < n; i++) {
> struct epoll_event *ev = &epw.events[i];
> @@ -109,7 +113,6 @@ get_readers(VALUE epio, VALUE ready, VALUE readers, VALUE timeout_msec)
> if (RTEST(obj))
> rb_ary_push(ready, obj);
> }
While the `for' loop can be moved ahead of the rb_sys_fail call
and limited in scope, Thread#raise can't be predicted.
I suppose rb_ensure works, but that's a PITA.
^ permalink raw reply [relevance 0%]
* [PATCH] epollexclusive: avoid Ruby object allocation for buffer
@ 2023-02-15 7:58 14% Eric Wong
2023-03-01 23:12 0% ` Eric Wong
0 siblings, 1 reply; 43+ results
From: Eric Wong @ 2023-02-15 7:58 UTC (permalink / raw)
To: unicorn-public
Leaving a dangling Ruby object around is suboptimal since it
adds to GC pressure.
`__attribute__((__cleanup__(foo)))' is an old gcc extension to
provide automatic cleanup functionality by running a pre-declared
function at the end of scope.
gcc introduced this two decades ago and clang's had it for over
a decade. Even tinycc supports it since 2019, and I expect it
to be easy to support in any C compiler since they're already
required to do cleanup for on-stack allocations.
So no, it's not standardized in C, but it makes life significantly
easier for C hackers.
---
ext/unicorn_http/epollexclusive.h | 19 +++++++++++--------
1 file changed, 11 insertions(+), 8 deletions(-)
diff --git a/ext/unicorn_http/epollexclusive.h b/ext/unicorn_http/epollexclusive.h
index 677e1fe..67a9ba6 100644
--- a/ext/unicorn_http/epollexclusive.h
+++ b/ext/unicorn_http/epollexclusive.h
@@ -78,6 +78,14 @@ static void *do_wait(void *ptr) /* runs w/o GVL */
epw->maxevents, epw->timeout_msec);
}
+/* cleanup is supported since 2003 (gcc), 2009 (clang), tinycc: 2019 */
+#define AUTO_XFREE __attribute__((__cleanup__(cleanup_xfree)))
+static void cleanup_xfree(void *any)
+{
+ void **p = any;
+ xfree(*p);
+}
+
/* :nodoc: */
/* readers must not change between prepare_readers and get_readers */
static VALUE
@@ -85,22 +93,18 @@ get_readers(VALUE epio, VALUE ready, VALUE readers, VALUE timeout_msec)
{
struct ep_wait epw;
long i, n;
- VALUE buf;
+ AUTO_XFREE void *buf = NULL;
Check_Type(ready, T_ARRAY);
Check_Type(readers, T_ARRAY);
epw.maxevents = RARRAY_LENINT(readers);
- buf = rb_str_buf_new(sizeof(struct epoll_event) * epw.maxevents);
- epw.events = (struct epoll_event *)RSTRING_PTR(buf);
+ epw.events = buf = ALLOC_N(struct epoll_event, epw.maxevents);
epio = rb_io_get_io(epio);
GetOpenFile(epio, epw.fptr);
epw.timeout_msec = NUM2INT(timeout_msec);
n = (long)rb_thread_call_without_gvl(do_wait, &epw, RUBY_UBF_IO, NULL);
- if (n < 0) {
- if (errno != EINTR) rb_sys_fail("epoll_wait");
- n = 0;
- }
+ if (n < 0 && errno != EINTR) rb_sys_fail("epoll_wait");
/* Linux delivers events in order received */
for (i = 0; i < n; i++) {
struct epoll_event *ev = &epw.events[i];
@@ -109,7 +113,6 @@ get_readers(VALUE epio, VALUE ready, VALUE readers, VALUE timeout_msec)
if (RTEST(obj))
rb_ary_push(ready, obj);
}
- rb_str_resize(buf, 0);
return Qfalse;
}
#endif /* USE_EPOLL */
^ permalink raw reply related [relevance 14%]
* [ANN] unicorn 5.4.0 - Rack HTTP server for fast clients and Unix
@ 2017-12-23 23:42 4% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2017-12-23 23:42 UTC (permalink / raw)
To: ruby-talk, unicorn-public; +Cc: James P Robinson Jr, Sam Saffron
unicorn is an HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels. Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between unicorn and slow clients.
* https://bogomips.org/unicorn/
* public list: unicorn-public@bogomips.org
* mail archives: https://bogomips.org/unicorn-public/
* git clone git://bogomips.org/unicorn.git
* https://bogomips.org/unicorn/NEWS.atom.xml
* nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
Changes:
Rack hijack support improves as the app code can capture and use
the Rack `env' privately without copying it (to avoid clobbering
by another client). Thanks to Sam Saffron for reporting and
testing this new feature:
https://bogomips.org/unicorn-public/CAAtdryPG3nLuyo0jxfYW1YHu1Q+ZpkLkd4KdWC8vA46B5haZxw@mail.gmail.com/T/
We also now support $DEBUG being set by the Rack app (instead of
relying on the "-d" CLI switch). Thanks to James P Robinson Jr
for reporting this bug:
https://bogomips.org/unicorn-public/D6324CB4.7BC3E%25james.robinson3@cigna.com/T/
(Coincidentally, this fix will be irrelevant for Ruby 2.5
which requires 'pp' by default)
There's a few minor test cleanups and documentation updates, too.
All commits since v5.3.1 (2017-10-03):
reduce method calls with String#start_with?
require 'pp' if $DEBUG is set by Rack app
avoid reusing env on hijack
tests: cleanup some unused variable warnings
ISSUES: add a note about Debian BTS interopability
Roughly all mailing discussions since the last release:
https://bogomips.org/unicorn-public/?q=d:20171004..20171223
^ permalink raw reply [relevance 4%]
* [PATCH] tests: cleanup some unused variable warnings
@ 2017-12-22 3:17 4% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2017-12-22 3:17 UTC (permalink / raw)
To: unicorn-public
Add a new "check-warnings" target to the GNUmakefile to make
checking for this easier. Warnings aren't fatal, and newer
versions of Ruby tend to increase warnings.
---
GNUmakefile | 5 +++++
test/unit/test_droplet.rb | 2 +-
test/unit/test_request.rb | 20 ++++++++++----------
3 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/GNUmakefile b/GNUmakefile
index 51045d4..2505e1f 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -249,5 +249,10 @@ endif
$(PLACEHOLDERS):
echo olddoc_placeholder > $@
+check-warnings:
+ @(for i in $$(git ls-files '*.rb' bin | grep -v '^setup\.rb$$'); \
+ do $(RUBY) --disable-gems -d -W2 -c \
+ $$i; done) | grep -v '^Syntax OK$$' || :
+
.PHONY: .FORCE-GIT-VERSION-FILE doc $(T) $(slow_tests) man
.PHONY: test-install
diff --git a/test/unit/test_droplet.rb b/test/unit/test_droplet.rb
index 73cf38c..81ad82b 100644
--- a/test/unit/test_droplet.rb
+++ b/test/unit/test_droplet.rb
@@ -4,7 +4,7 @@
class TestDroplet < Test::Unit::TestCase
def test_create_many_droplets
now = Time.now.to_i
- tmp = (0..1024).map do |i|
+ (0..1024).each do |i|
droplet = Unicorn::Worker.new(i)
assert droplet.respond_to?(:tick)
assert_equal 0, droplet.tick
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index f0ccaf7..6cb0268 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -34,7 +34,7 @@ def test_options
assert_equal '', env['REQUEST_PATH']
assert_equal '', env['PATH_INFO']
assert_equal '*', env['REQUEST_URI']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_absolute_uri_with_query
@@ -44,7 +44,7 @@ def test_absolute_uri_with_query
assert_equal '/x', env['REQUEST_PATH']
assert_equal '/x', env['PATH_INFO']
assert_equal 'y=z', env['QUERY_STRING']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_absolute_uri_with_fragment
@@ -55,7 +55,7 @@ def test_absolute_uri_with_fragment
assert_equal '/x', env['PATH_INFO']
assert_equal '', env['QUERY_STRING']
assert_equal 'frag', env['FRAGMENT']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_absolute_uri_with_query_and_fragment
@@ -66,7 +66,7 @@ def test_absolute_uri_with_query_and_fragment
assert_equal '/x', env['PATH_INFO']
assert_equal 'a=b', env['QUERY_STRING']
assert_equal 'frag', env['FRAGMENT']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_absolute_uri_unsupported_schemes
@@ -83,7 +83,7 @@ def test_x_forwarded_proto_https
"Host: foo\r\n\r\n")
env = @request.read(client)
assert_equal "https", env['rack.url_scheme']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_x_forwarded_proto_http
@@ -92,7 +92,7 @@ def test_x_forwarded_proto_http
"Host: foo\r\n\r\n")
env = @request.read(client)
assert_equal "http", env['rack.url_scheme']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_x_forwarded_proto_invalid
@@ -101,7 +101,7 @@ def test_x_forwarded_proto_invalid
"Host: foo\r\n\r\n")
env = @request.read(client)
assert_equal "http", env['rack.url_scheme']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_rack_lint_get
@@ -109,7 +109,7 @@ def test_rack_lint_get
env = @request.read(client)
assert_equal "http", env['rack.url_scheme']
assert_equal '127.0.0.1', env['REMOTE_ADDR']
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_no_content_stringio
@@ -143,7 +143,7 @@ def test_rack_lint_put
"abcde")
env = @request.read(client)
assert ! env.include?(:http_body)
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
def test_rack_lint_big_put
@@ -177,6 +177,6 @@ def client.kgio_read!(*args)
}
assert_nil env['rack.input'].read(bs)
env['rack.input'].rewind
- res = @lint.call(env)
+ assert_kind_of Array, @lint.call(env)
end
end
--
EW
^ permalink raw reply related [relevance 4%]
* [PATCH] http_request: freeze constant strings passed IO#write
@ 2017-02-13 18:27 5% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2017-02-13 18:27 UTC (permalink / raw)
To: unicorn-public
This ensures we won't have duplicate objects in Ruby 2.0-2.4.
For Ruby 2.5.0dev+, this avoids any duplicate cleanup
introduced as of r57471: https://bugs.ruby-lang.org/issues/13085
---
lib/unicorn/http_request.rb | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 0c1f9bb..c176083 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -24,10 +24,7 @@ class Unicorn::HttpParser
NULL_IO = StringIO.new("")
# :stopdoc:
- # A frozen format for this is about 15% faster
- # Drop these frozen strings when Ruby 2.2 becomes more prevalent,
- # 2.2+ optimizes hash assignments when used with literal string keys
- HTTP_RESPONSE_START = [ 'HTTP', '/1.1 ']
+ HTTP_RESPONSE_START = [ 'HTTP'.freeze, '/1.1 '.freeze ]
@@input_class = Unicorn::TeeInput
@@check_client_connection = false
--
EW
^ permalink raw reply related [relevance 5%]
* Re: Support for HTTP/1.0
@ 2016-11-11 7:07 5% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2016-11-11 7:07 UTC (permalink / raw)
To: Joe McIlvain; +Cc: unicorn-public
Eric Wong <e@80x24.org> wrote:
> Is this nginx->haproxy->unicorn or haproxy->nginx->unicorn?
>
> Are persistent connections from nginx->unicorn enabled?
> (I suggest keeping it disabled, the default as far as I recall)
>
> What else can you share about your nginx/haproxy version and
> configuration?
(out-of-order)
Ping? With my nginx (1.6.2 on Debian jessie) -> unicorn config on
I can confirm nginx is sending 1.0 requests to backends, which
ought to prevent Rack::Chunked from chunking, at least...
Haven't looked at haproxy in a while... :x
> Joe McIlvain <joe.eli.mac@gmail.com> wrote:
> > Sure enough, looking through the Unicorn source I see that the
> > "HTTP/1.1" protocol is hard-coded in the response writing logic:
> > https://github.com/defunkt/unicorn/blob/a72d2e7fbd13a6bfe64b79ae361c17ea568d4867/lib/unicorn/http_response.rb#L30
>
> Right, it's certainly faster to avoid having an extra hash
> lookup to get the correct string in the response.
There's also several places where we assume "HTTP/1.1"
that would involve deeper changes than merely taking
the version string: the wacky[1] check_client_connection
+ response_start_sent logic
So more work would be necessary to respond with HTTP/1.0...
But I've been thinking about ways to cleanup and micro-optimize
that a bit more, anyways...
[1] https://bogomips.org/unicorn-public/?q=s:%22combating+nginx+499%22
(weirdest. feature. ever.)
^ permalink raw reply [relevance 5%]
* [PATCH 0/2] socket inheritance behavior/bug => feature!
@ 2015-10-27 3:33 5% Eric Wong
2015-10-27 3:33 5% ` [PATCH 1/2] sd_listen_fds emulation cleanup Eric Wong
0 siblings, 1 reply; 43+ results
From: Eric Wong @ 2015-10-27 3:33 UTC (permalink / raw)
To: unicorn-public
Given long enough, some bugs need to become documented behavior
(see [PATCH 2/2]).
Anyways, I guess we'll release 5.0.0 final in a day or two. I was
intending to release 5.0.0 today until I noticed how un-DRY having
listeners configured in both systemd and unicorn config files would
be. And then I noticed we have an existing bug as documented in 2/2
which would allow the user to be DRY after all.
Eric Wong (2):
sd_listen_fds emulation cleanup
inheriting sockets from UNICORN_FD does not close them
Documentation/unicorn.1.txt | 4 +---
lib/unicorn/http_server.rb | 4 ++--
test/exec/test_exec.rb | 50 ++++++++++++++++++++++++++++++++++-----------
3 files changed, 41 insertions(+), 17 deletions(-)
Fwiw, in case anybody is uncomfortable with the mere mention of the
word "systemd"; unicorn will never have any dependencies on systemd.
It will merely use systemd if available.
^ permalink raw reply [relevance 5%]
* [PATCH 1/2] sd_listen_fds emulation cleanup
2015-10-27 3:33 5% [PATCH 0/2] socket inheritance behavior/bug => feature! Eric Wong
@ 2015-10-27 3:33 5% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2015-10-27 3:33 UTC (permalink / raw)
To: unicorn-public
Re-enable and expand on the test case while we're at it for new
Rubies. The bug is now fixed in Ruby 2.3.0dev as of r51576. We
shall assume anybody running a pre-release 2.3.0 at this point is
running a fairly recent snapshot, so we won't bother doing a
finer-grained check in the test for an exact revision number.
---
lib/unicorn/http_server.rb | 4 ++--
test/exec/test_exec.rb | 40 +++++++++++++++++++++++-----------------
2 files changed, 25 insertions(+), 19 deletions(-)
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 3dbfd3e..c1a2e60 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -772,12 +772,12 @@ def inherit_listeners!
sd_pid, sd_fds = ENV.values_at('LISTEN_PID', 'LISTEN_FDS')
if sd_pid && sd_pid.to_i == $$
# 3 = SD_LISTEN_FDS_START
- inherited.concat((3...(3 + sd_fds.to_i)).map { |fd| Socket.for_fd(fd) })
+ inherited.concat((3...(3 + sd_fds.to_i)).to_a)
end
# to ease debugging, we will not unset LISTEN_PID and LISTEN_FDS
inherited.map! do |fd|
- io = String === fd ? Socket.for_fd(fd.to_i) : fd
+ io = Socket.for_fd(fd.to_i)
io.autoclose = false
io = server_cast(io)
set_server_sockopt(io, listener_opts[sock_name(io)])
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index 33d768a..af6f151 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -96,31 +96,37 @@ def teardown
end
end
- # FIXME https://bugs.ruby-lang.org/issues/11336
- # [ruby-core:69895] [Bug #11336]
- def disabled_test_sd_listen_fds_emulation
+ def test_sd_listen_fds_emulation
File.open("config.ru", "wb") { |fp| fp.write(HI) }
sock = TCPServer.new(@addr, @port)
- sock.setsockopt(:SOL_SOCKET, :SO_KEEPALIVE, 0)
- pid = xfork do
- redirect_test_io do
- # pretend to be systemd
- ENV['LISTEN_PID'] = "#$$"
- ENV['LISTEN_FDS'] = '1'
+ [ %W(-l #@addr:#@port), nil ].each do |l|
+ sock.setsockopt(:SOL_SOCKET, :SO_KEEPALIVE, 0)
+
+ pid = xfork do
+ redirect_test_io do
+ # pretend to be systemd
+ ENV['LISTEN_PID'] = "#$$"
+ ENV['LISTEN_FDS'] = '1'
- # 3 = SD_LISTEN_FDS_START
- exec($unicorn_bin, "-l", "#@addr:#@port", 3 => sock)
+ # 3 = SD_LISTEN_FDS_START
+ args = [ $unicorn_bin ]
+ args.concat(l) if l
+ args << { 3 => sock }
+ exec(*args)
+ end
end
+ res = hit(["http://#@addr:#@port/"])
+ assert_equal [ "HI\n" ], res
+ assert_shutdown(pid)
+ assert_equal 1, sock.getsockopt(:SOL_SOCKET, :SO_KEEPALIVE).int,
+ 'unicorn should always set SO_KEEPALIVE on inherited sockets'
end
- res = hit(["http://#{@addr}:#{@port}/"])
- assert_equal [ "HI\n"], res
- assert_shutdown(pid)
- assert_equal 1, sock.getsockopt(:SOL_SOCKET, :SO_KEEPALIVE).int,
- "unicorn should always set SO_KEEPALIVE on inherited sockets"
ensure
sock.close if sock
- end
+ # disabled test on old Rubies: https://bugs.ruby-lang.org/issues/11336
+ # [ruby-core:69895] [Bug #11336] fixed by r51576
+ end if RUBY_VERSION.to_f >= 2.3
def test_working_directory_rel_path_config_file
other = Tempfile.new('unicorn.wd')
--
EW
^ permalink raw reply related [relevance 5%]
* Re: TeeInput leaks file handles/space
2015-04-24 3:08 0% ` Eric Wong
@ 2015-04-24 11:56 0% ` Mulvaney, Mike
0 siblings, 0 replies; 43+ results
From: Mulvaney, Mike @ 2015-04-24 11:56 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public@bogomips.org
That sounds great. Thanks for fixing this so fast!
-Mike
> On Apr 23, 2015, at 11:08 PM, Eric Wong <e@80x24.org> wrote:
>
> "Mulvaney, Mike" <MMulvaney@bna.com> wrote:
>> I think it would be reasonable to fix this for Rack 1.6+. It won't
>> cause any problems for Rack 1.5 users, right? The environment
>> variable gets set and then ignored, so the app would behave exactly
>> the same way. If they want to use the new cleanup code, they can
>> upgrade rack.
>
> Right, this only affects 1.6 users. 1.5 users will need to upgrade for
> this.
^ permalink raw reply [relevance 0%]
* Re: TeeInput leaks file handles/space
2015-04-22 19:24 4% ` Mulvaney, Mike
@ 2015-04-24 3:08 0% ` Eric Wong
2015-04-24 11:56 0% ` Mulvaney, Mike
0 siblings, 1 reply; 43+ results
From: Eric Wong @ 2015-04-24 3:08 UTC (permalink / raw)
To: Mulvaney, Mike; +Cc: unicorn-public@bogomips.org
"Mulvaney, Mike" <MMulvaney@bna.com> wrote:
> I think it would be reasonable to fix this for Rack 1.6+. It won't
> cause any problems for Rack 1.5 users, right? The environment
> variable gets set and then ignored, so the app would behave exactly
> the same way. If they want to use the new cleanup code, they can
> upgrade rack.
Right, this only affects 1.6 users. 1.5 users will need to upgrade for
this.
^ permalink raw reply [relevance 0%]
* [PATCH 2/2] support TempfileReaper in deployment and development envs
2015-04-24 3:02 4% ` [PATCH 1/2] tee_input: support for Rack::TempfileReaper middleware Eric Wong
@ 2015-04-24 3:02 5% ` Eric Wong
1 sibling, 0 replies; 43+ results
From: Eric Wong @ 2015-04-24 3:02 UTC (permalink / raw)
To: unicorn-public; +Cc: Mike Mulvaney, Eric Wong
rack 1.6 added a TempfileReaper middleware to cleanup temporary
files. Enable it by default for users running rack 1.6 or later
to avoid leaving temporary files around.
---
lib/unicorn.rb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 467245d..9fdcb8e 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -65,6 +65,7 @@ module Unicorn
use Rack::CommonLogger, $stderr
use Rack::ShowExceptions
use Rack::Lint
+ use Rack::TempfileReaper if Rack.const_defined?(:TempfileReaper)
run inner_app
end.to_app
when "deployment"
@@ -72,6 +73,7 @@ module Unicorn
use Rack::ContentLength
use Rack::Chunked
use Rack::CommonLogger, $stderr
+ use Rack::TempfileReaper if Rack.const_defined?(:TempfileReaper)
run inner_app
end.to_app
else
--
EW
^ permalink raw reply related [relevance 5%]
* [PATCH 1/2] tee_input: support for Rack::TempfileReaper middleware
@ 2015-04-24 3:02 4% ` Eric Wong
2015-04-24 3:02 5% ` [PATCH 2/2] support TempfileReaper in deployment and development envs Eric Wong
1 sibling, 0 replies; 43+ results
From: Eric Wong @ 2015-04-24 3:02 UTC (permalink / raw)
To: unicorn-public; +Cc: Mike Mulvaney, Eric Wong
Rack::TempfileReaper was added in rack 1.6 to cleanup temporary
files. Make Unicorn::TmpIO ducktype-compatible so
Rack::TempfileReaper may be used to free up space used by temporary
buffer files.
Ref: <CY1PR0301MB078011EB5A22B733EB222A45A4EE0@CY1PR0301MB0780.namprd03.prod.outlook.com>
Reported-by: Mike Mulvaney <MMulvaney@bna.com>
---
lib/unicorn/tee_input.rb | 9 ++++++++-
lib/unicorn/tmpio.rb | 3 +++
test/unit/test_tee_input.rb | 10 ++++++++++
3 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb
index 637c583..3c6d18a 100644
--- a/lib/unicorn/tee_input.rb
+++ b/lib/unicorn/tee_input.rb
@@ -28,13 +28,20 @@ class Unicorn::TeeInput < Unicorn::StreamInput
@@client_body_buffer_size
end
+ # for Rack::TempfileReaper in rack 1.6+
+ def new_tmpio # :nodoc:
+ tmpio = Unicorn::TmpIO.new
+ (@parser.env['rack.tempfiles'] ||= []) << tmpio
+ tmpio
+ end
+
# Initializes a new TeeInput object. You normally do not have to call
# this unless you are writing an HTTP server.
def initialize(socket, request)
@len = request.content_length
super
@tmp = @len && @len <= @@client_body_buffer_size ?
- StringIO.new("") : Unicorn::TmpIO.new
+ StringIO.new("") : new_tmpio
end
# :call-seq:
diff --git a/lib/unicorn/tmpio.rb b/lib/unicorn/tmpio.rb
index c97979a..db88ed3 100644
--- a/lib/unicorn/tmpio.rb
+++ b/lib/unicorn/tmpio.rb
@@ -21,4 +21,7 @@ class Unicorn::TmpIO < File
fp.sync = true
fp
end
+
+ # pretend we're Tempfile for Rack::TempfileReaper
+ alias close! close
end
diff --git a/test/unit/test_tee_input.rb b/test/unit/test_tee_input.rb
index 0c2c941..4647e66 100644
--- a/test/unit/test_tee_input.rb
+++ b/test/unit/test_tee_input.rb
@@ -29,6 +29,13 @@ class TestTeeInput < Test::Unit::TestCase
end while true
end
+ def check_tempfiles
+ tmp = @parser.env["rack.tempfiles"]
+ assert_instance_of Array, tmp
+ assert_operator tmp.size, :>=, 1
+ assert_instance_of Unicorn::TmpIO, tmp[0]
+ end
+
def test_gets_long
r = init_request("hello", 5 + (4096 * 4 * 3) + "#$/foo#$/".size)
ti = TeeInput.new(@rd, r)
@@ -106,6 +113,7 @@ class TestTeeInput < Test::Unit::TestCase
assert_kind_of File, ti.tmp
assert_equal 0, ti.tmp.pos
assert_equal Unicorn::Const::MAX_BODY + 1, ti.size
+ check_tempfiles
end
def test_read_in_full_if_content_length
@@ -148,6 +156,7 @@ class TestTeeInput < Test::Unit::TestCase
assert_nil ti.read(1)
pid, status = Process.waitpid2(pid)
assert status.success?
+ check_tempfiles
end
def test_chunked
@@ -183,6 +192,7 @@ class TestTeeInput < Test::Unit::TestCase
status = nil
pid, status = Process.waitpid2(pid)
assert status.success?
+ check_tempfiles
end
def test_chunked_ping_pong
--
EW
^ permalink raw reply related [relevance 4%]
* RE: TeeInput leaks file handles/space
@ 2015-04-22 19:24 4% ` Mulvaney, Mike
2015-04-24 3:08 0% ` Eric Wong
0 siblings, 1 reply; 43+ results
From: Mulvaney, Mike @ 2015-04-22 19:24 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public@bogomips.org
Oh yeah, I like that even better.
The app I'm currently working on is using Rails 4.1 and Rack 1.5.x. I don't have any problem with upgrading rack, I just haven't done it yet.
I think it would be reasonable to fix this for Rack 1.6+. It won't cause any problems for Rack 1.5 users, right? The environment variable gets set and then ignored, so the app would behave exactly the same way. If they want to use the new cleanup code, they can upgrade rack.
-Mike
-----Original Message-----
From: Eric Wong [mailto:e@80x24.org]
Sent: Wednesday, April 22, 2015 3:16 PM
To: Mulvaney, Mike
Cc: unicorn-public@bogomips.org
Subject: Re: TeeInput leaks file handles/space
"Mulvaney, Mike" <MMulvaney@bna.com> wrote:
> That looks reasonable to me -- this way you would only have one file
> still open per process at a maximum, right? I think that's a good
> solution.
Right.
Below is a barely-tested alternative patch for Rack::TempfileReaper, for Rack 1.6+ users only. I'm not sure how prevalent 1.6+ was only released in December 2014...
It's more standardized, but maybe Rack 1.6 isn't prevalent enough, yet.
What do you think?
(Sorry, in a rush, no commit message, yet)
diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb index 637c583..7f6baa2 100644
--- a/lib/unicorn/tee_input.rb
+++ b/lib/unicorn/tee_input.rb
@@ -28,13 +28,20 @@ class Unicorn::TeeInput < Unicorn::StreamInput
@@client_body_buffer_size
end
+ # for Rack::TempfileReaper in rack 1.6+ def new_tmpio
+ tmpio = Unicorn::TmpIO.new
+ (@parser.env['rack.tempfiles'] ||= []) << tmpio
+ tmpio
+ end
+
# Initializes a new TeeInput object. You normally do not have to call
# this unless you are writing an HTTP server.
def initialize(socket, request)
@len = request.content_length
super
@tmp = @len && @len <= @@client_body_buffer_size ?
- StringIO.new("") : Unicorn::TmpIO.new
+ StringIO.new("") : new_tmpio
end
# :call-seq:
diff --git a/lib/unicorn/tmpio.rb b/lib/unicorn/tmpio.rb index c97979a..db88ed3 100644
--- a/lib/unicorn/tmpio.rb
+++ b/lib/unicorn/tmpio.rb
@@ -21,4 +21,7 @@ class Unicorn::TmpIO < File
fp.sync = true
fp
end
+
+ # pretend we're Tempfile for Rack::TempfileReaper alias close! close
end
^ permalink raw reply [relevance 4%]
* Re: Having issue with Unicorn
@ 2014-10-24 18:02 3% ` Imdad
0 siblings, 0 replies; 43+ results
From: Imdad @ 2014-10-24 18:02 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn-public
Thanks Eric, here is my deploy.rb and config/unicorn.rb
NOTE: /releases/6 and /releases/28 both have same error message
config/unicorn.rb
==============
# Set your full path to application.
app_dir = File.expand_path('../../', __FILE__)
shared_dir = File.expand_path('../../../shared/', __FILE__)
# Set unicorn options
worker_processes 2
preload_app true
timeout 30
# Fill path to your app
working_directory app_dir
# Set up socket location
listen "#{shared_dir}/sockets/unicorn.sock", :backlog => 64
# Loging
stderr_path "#{shared_dir}/log/unicorn.stderr.log"
stdout_path "#{shared_dir}/log/unicorn.stdout.log"
# Set master PID location
pid "#{shared_dir}/pids/unicorn.pid"
before_fork do |server, worker|
defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!
old_pid = "#{server.config[:pid]}.oldbin"
if File.exists?(old_pid) && server.pid != old_pid
begin
sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
Process.kill(sig, File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# someone else did our job for us
end
end
end
after_fork do |server, worker|
defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
end
before_exec do |server|
ENV['BUNDLE_GEMFILE'] = "#{app_dir}/Gemfile"
end
deploy.rb
==========
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rbenv' # for rbenv support. (http://rbenv.org)
# require 'mina/rvm' # for rvm support. (http://rvm.io)
# TODO: Look into this later
#require 'mina_sidekiq/tasks'
require 'mina/unicorn'
# Basic settings:
# domain - The hostname to SSH to.
# deploy_to - Path to deploy into.
# repository - Git repo to clone from. (needed by mina/git)
# branch - Branch name to deploy. (needed by mina/git)
set_default :rbenv_path, "/root/.rbenv" #{}"/root/.rbenv"
#set_default :bundle_path, '/root/.rbenv/shims/bundle'
#set_default :bundle_bin, 'bundle exec'
set :domain, '104.131.74.69'
set :deploy_to, '/var/www/hailisys'
set :repository, 'https://gitlab.com/hailisys/hailisys.git'
set :branch, 'master'
set :user, 'root'
set :forward_agent, true
# MOIN: Fix Password issue
set :term_mode, nil
# MOIN: Could be staging, production
set :rails_env, 'production'
set :port, '22'
set :unicorn_pid, "#{deploy_to}/shared/pids/unicorn.pid"
# For system-wide RVM install.
# set :rvm_path, '/usr/local/rvm/bin/rvm'
# Manually create these paths in shared/ (eg: shared/config/database.yml)
in your server.
# They will be linked in the 'deploy:link_shared_paths' step.
set :shared_paths, ['config/database.yml', 'log']
# Optional settings:
# set :user, 'foobar' # Username in the server to SSH to.
# set :port, '30000' # SSH port number.
# set :forward_agent, true # SSH forward_agent.
# This task is the environment that is loaded for most commands, such as
# `mina deploy` or `mina rake`.
task :environment do
# If you're using rbenv, use this to load the rbenv environment.
# Be sure to commit your .rbenv-version to your repository.
invoke :'rbenv:load'
# For those using RVM, use this to load an RVM version@gemset.
# invoke :'rvm:use[ruby-1.9.3-p125@default]'
end
# Put any custom mkdir's in here for when `mina setup` is ran.
# For Rails apps, we'll make some of the shared paths that are shared
between
# all releases.
task :setup => :environment do
queue! %[mkdir -p "#{deploy_to}/shared/log"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]
queue! %[mkdir -p "#{deploy_to}/shared/config"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]
queue! %[mkdir -p "#{deploy_to}/shared/pids"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/pids"]
queue! %[mkdir -p "#{deploy_to}/shared/sockets"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/sockets"]
queue! %[touch "#{deploy_to}/shared/config/database.yml"]
queue %[echo "-----> Be sure to edit 'shared/config/database.yml'."]
# sidekiq needs a place to store its pid file and log file
queue! %[mkdir -p "#{deploy_to}/shared/pids/"]
queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/pids"]
end
desc "Deploys the current version to the server."
task :deploy => :environment do
deploy do
# stop accepting new workers
# TODO: Look into this later
# invoke :'sidekiq:quiet'
# Put things that will set up an empty directory into a fully set-up
# instance of your project.
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
to :launch do
# Passenger
# queue "mkdir -p #{deploy_to}/#{current_path}/tmp/"
# queue "touch #{deploy_to}/#{current_path}/tmp/restart.txt"
# TODO: Look into this later
# invoke :'sidekiq:restart'
invoke :'unicorn:restart'
#invoke :'unicorn:start'
#queue "touch #{deploy_to}/tmp/restart.txt"
end
end
end
# For help in making your deploy script, see the Mina documentation:
#
# - http://nadarei.co/mina
# - http://nadarei.co/mina/tasks
# - http://nadarei.co/mina/settings
# - http://nadarei.co/mina/helpers
Please suggest what could be wrong.
Cheers!
Imdad Ali Khan
Mob (0) 9818484057
http://www.linkedin.com/in/imdad
On 24 October 2014 23:15, Eric Wong <e@80x24.org> wrote:
> Imdad <khanimdad@gmail.com> wrote:
> > I have rails4, nginx, unicorn and mona for app deployed
> > http://104.131.74.69/
> >
> > But after deploy hitting on above link results in error, and the log says
> >
> > root@Hailisys:~# tail -n 0 -f
> > /var/www/hailisys/shared/log/unicorn.stderr.log
> > I, [2014-10-24T17:28:53.162084 #15819] INFO -- : executing
> > ["/var/www/hailisys/releases/6/vendor/bundle/ruby/2.1.0/bin/unicorn",
> "-c",
> > "/var/www/hailisys/current/config/unicorn.rb", "-D", "-E", "production",
> > {11=>#<Kgio::UNIXServer:fd 11>}] (in /var/www/hailisys/releases/28)
>
> You're working inside /releases/28 here...
>
> >
> /var/www/hailisys/current/vendor/bundle/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:475:in
> > `exec': No such file or directory -
> > /var/www/hailisys/releases/6/vendor/bundle/ruby/2.1.0/bin/unicorn
> > (Errno::ENOENT)
>
> However, it's trying to run out of /releases/6, which I assume is
> old enough to be removed by now.
>
> You should be able to set working_directory in your config/unicorn.rb:
>
> working_directory "/var/www/hailisys/current"
>
> Then SIGHUP to reload the config before sending SIGUSR2 to it.
>
> Hopefully that works. Normally you shouldn't need to set
> working_directory if you're working out of "/current";
> but I'm not sure what your deploy environment looks like.
>
^ permalink raw reply [relevance 3%]
* unicorn 5 roadmap
@ 2014-05-25 3:52 5% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2014-05-25 3:52 UTC (permalink / raw)
To: unicorn-public; +Cc: yahns-public
unicorn is over 5 years old, things are still kicking, but
has not changed much since it was first announced in 2009:
http://mid.gmane.org/20090211230457.GB22926@dcvr.yhbt.net [1]
So 5 will be the next major version, but there's nothing earth
shattering and probably nothing visible to the majority of users.
Almost all just internal housekeeping:
* (finally) remove Unicorn::HttpParser#reset
* switch to minitest5 (or test-unit2?)
* drop some things that were intended for Rainbows!
- keepalive_requests, not necessary with the MT/one-shot-based event
loops like yahns, this is only intended as a DoS mitigation measure for
single-threaded event loops (nginx) or pure-MT-based servers
- xftrust (long deprecated)
* cleanup tests, port shell script integration tests to shunit2?
(or making the tests pure Ruby is probably fine, I trust the stability
of the Ruby language more today than I did in 2009 with the painful
1.8->1.9 transition)
* extract terminal-friendly coverage output from yahns into a gem
and use it for unicorn.
Tangentially related things to do:
* reduce memory usage in Ruby so users may run more unicorns
* improve reverse proxy support in yahns so it may be an
alternative to nginx for unicorn users
What else?
[1] Supporting the project with only email is probably the only
reason I haven't given up after all these years.
^ permalink raw reply [relevance 5%]
* [PATCH] support for Rack hijack in request and response
@ 2013-01-22 11:49 3% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2013-01-22 11:49 UTC (permalink / raw)
To: mongrel-unicorn
Rack 1.5.0 (protocol version [1,2]) adds support for
hijacking the client socket (removing it from the control
of unicorn (or any other Rack webserver)).
Tested with rack 1.5.0.
---
I've pushed this to the "hijack" branch of git://bogomips.org/unicorn
As this is a development branch, I intend to rebase and cleanup history
as needed before it hits master.
I also believe the Rack::Lint change mentioned in my t0005 comment
below is a bug (patch/pull posted to rack-devel ML:
20130122113749.GA31589@dcvr.yhbt.net )
lib/unicorn/http_request.rb | 21 +++++++++++++++++++
lib/unicorn/http_response.rb | 40 +++++++++++++++++++++++++++++--------
lib/unicorn/http_server.rb | 6 ++++--
t/hijack.ru | 37 ++++++++++++++++++++++++++++++++++
t/t0005-working_directory_app.rb.sh | 5 ++++-
t/t0200-rack-hijack.sh | 27 +++++++++++++++++++++++++
6 files changed, 125 insertions(+), 11 deletions(-)
create mode 100644 t/hijack.ru
create mode 100755 t/t0200-rack-hijack.sh
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 79ead2e..3bc64ed 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -91,6 +91,27 @@ class Unicorn::HttpParser
e[RACK_INPUT] = 0 == content_length ?
NULL_IO : @@input_class.new(socket, self)
+ hijack_setup(e, socket)
e.merge!(DEFAULTS)
end
+
+ # Rack 1.5.0 (protocol version 1.2) adds hijack request support
+ if ((Rack::VERSION[0] << 8) | Rack::VERSION[1]) >= 0x0102
+ DEFAULTS["rack.hijack?"] = true
+
+ # FIXME: asking for clarification about this in
+ # http://mid.gmane.org/20130122100802.GA28585@dcvr.yhbt.net
+ DEFAULTS["rack.version"] = [1, 2]
+
+ RACK_HIJACK = "rack.hijack".freeze
+ RACK_HIJACK_IO = "rack.hijack_io".freeze
+
+ def hijack_setup(e, socket)
+ e[RACK_HIJACK] = proc { e[RACK_HIJACK_IO] ||= socket }
+ end
+ else
+ # old Rack, do nothing.
+ def hijack_setup(e, _)
+ end
+ end
end
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index 579d957..083951c 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -25,6 +25,7 @@ module Unicorn::HttpResponse
def http_response_write(socket, status, headers, body,
response_start_sent=false)
status = CODES[status.to_i] || status
+ hijack = nil
http_response_start = response_start_sent ? '' : 'HTTP/1.1 '
if headers
@@ -33,19 +34,42 @@ module Unicorn::HttpResponse
"Status: #{status}\r\n" \
"Connection: close\r\n"
headers.each do |key, value|
- next if %r{\A(?:Date\z|Connection\z)}i =~ key
- if value =~ /\n/
- # avoiding blank, key-only cookies with /\n+/
- buf << value.split(/\n+/).map! { |v| "#{key}: #{v}\r\n" }.join
+ case key
+ when %r{\A(?:Date\z|Connection\z)}i
+ next
+ when "rack.hijack"
+ # this was an illegal key in Rack < 1.5, so it should be
+ # OK to silently discard it for those older versions
+ hijack = hijack_prepare(value)
else
- buf << "#{key}: #{value}\r\n"
+ if value =~ /\n/
+ # avoiding blank, key-only cookies with /\n+/
+ buf << value.split(/\n+/).map! { |v| "#{key}: #{v}\r\n" }.join
+ else
+ buf << "#{key}: #{value}\r\n"
+ end
end
end
socket.write(buf << CRLF)
end
- body.each { |chunk| socket.write(chunk) }
- ensure
- body.respond_to?(:close) and body.close
+ if hijack
+ body = nil # ensure we do not close body
+ hijack.call(socket)
+ else
+ body.each { |chunk| socket.write(chunk) }
+ end
+ ensure
+ body.respond_to?(:close) and body.close
+ end
+
+ # Rack 1.5.0 (protocol version 1.2) adds response hijacking support
+ if ((Rack::VERSION[0] << 8) | Rack::VERSION[1]) >= 0x0102
+ def hijack_prepare(value)
+ value
+ end
+ else
+ def hijack_prepare(_)
+ end
end
end
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index aa98aeb..2d8e4e1 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -559,8 +559,10 @@ class Unicorn::HttpServer
@request.headers? or headers = nil
http_response_write(client, status, headers, body,
@request.response_start_sent)
- client.shutdown # in case of fork() in Rack app
- client.close # flush and uncork socket immediately, no keepalive
+ unless client.closed? # rack.hijack may've close this for us
+ client.shutdown # in case of fork() in Rack app
+ client.close # flush and uncork socket immediately, no keepalive
+ end
rescue => e
handle_error(client, e)
end
diff --git a/t/hijack.ru b/t/hijack.ru
new file mode 100644
index 0000000..105e0d7
--- /dev/null
+++ b/t/hijack.ru
@@ -0,0 +1,37 @@
+use Rack::Lint
+use Rack::ContentLength
+use Rack::ContentType, "text/plain"
+class DieIfUsed
+ def each
+ abort "body.each called after response hijack\n"
+ end
+
+ def close
+ abort "body.close called after response hijack\n"
+ end
+end
+run lambda { |env|
+ case env["PATH_INFO"]
+ when "/hijack_req"
+ if env["rack.hijack?"]
+ io = env["rack.hijack"].call
+ if io.respond_to?(:read_nonblock) &&
+ env["rack.hijack_io"].respond_to?(:read_nonblock)
+ return [ 200, {}, [ "hijack.OK\n" ] ]
+ end
+ end
+ [ 500, {}, [ "hijack BAD\n" ] ]
+ when "/hijack_res"
+ r = "response.hijacked"
+ [ 200,
+ {
+ "Content-Length" => r.bytesize.to_s,
+ "rack.hijack" => proc do |io|
+ io.write(r)
+ io.close
+ end
+ },
+ DieIfUsed.new
+ ]
+ end
+}
diff --git a/t/t0005-working_directory_app.rb.sh b/t/t0005-working_directory_app.rb.sh
index 37c6fa7..0fbab4f 100755
--- a/t/t0005-working_directory_app.rb.sh
+++ b/t/t0005-working_directory_app.rb.sh
@@ -11,7 +11,10 @@ t_begin "setup and start" && {
cat > $t_pfx.app/fooapp.rb <<\EOF
class Fooapp
def self.call(env)
- [ 200, [%w(Content-Type text/plain), %w(Content-Length 2)], %w(HI) ]
+ # Rack::Lint in 1.5.0 requires headers to be a hash
+ h = [%w(Content-Type text/plain), %w(Content-Length 2)]
+ h = Rack::Utils::HeaderHash.new(h)
+ [ 200, h, %w(HI) ]
end
end
EOF
diff --git a/t/t0200-rack-hijack.sh b/t/t0200-rack-hijack.sh
new file mode 100755
index 0000000..23a9ee4
--- /dev/null
+++ b/t/t0200-rack-hijack.sh
@@ -0,0 +1,27 @@
+#!/bin/sh
+. ./test-lib.sh
+t_plan 5 "rack.hijack tests (Rack 1.5+ (Rack::VERSION >= [ 1,2]))"
+
+t_begin "setup and start" && {
+ unicorn_setup
+ unicorn -D -c $unicorn_config hijack.ru
+ unicorn_wait_start
+}
+
+t_begin "check request hijack" && {
+ test "xhijack.OK" = x"$(curl -sSfv http://$listen/hijack_req)"
+}
+
+t_begin "check response hijack" && {
+ test "xresponse.hijacked" = x"$(curl -sSfv http://$listen/hijack_res)"
+}
+
+t_begin "killing succeeds" && {
+ kill $unicorn_pid
+}
+
+t_begin "check stderr" && {
+ check_stderr
+}
+
+t_done
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 3%]
* Re: Combating nginx 499 HTTP responses during flash traffic scenario
@ 2012-12-04 3:00 3% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2012-12-04 3:00 UTC (permalink / raw)
To: unicorn list
Eric Wong <normalperson@yhbt.net> wrote:
> I fixed up some minor line-wrapping, signed-off, and added your
> quote above to the commit message. Pushed as
> commit 5c700fc2cf398848ddcf71a2aa3f0f2a6563e87b
> to git://bogomips.org/unicorn.git
One more fix/cleanup to maintain compatibility with Rainbows!
>From 69e6a793d34ff71da7c8ca59962d627e2fb508d8 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Tue, 4 Dec 2012 02:35:26 +0000
Subject: [PATCH] fix const error responses for Rainbows!
Rainbows! relies on the ERROR_XXX_RESPONSE constants of unicorn
4.x. Changing the constants in unicorn 4.x will break existing
versions of Rainbows!, so remove the dependency on the constants
and generate the error response dynamically.
Unlike Mongrel, unicorn is unlikely to see malicious traffic and
thus unlikely to benefit from making error messages constant.
For unicorn 5.x, we will drop these constants entirely.
(Rainbows! most likely cannot support check_client_connection
consistently across all concurrency models since some of them
pessimistically buffer all writes in userspace. However, the
extra concurrency of Rainbows! makes it less likely to be
overloaded than unicorn, so this feature is likely less useful
for Rainbows!)
---
lib/unicorn/const.rb | 10 ++++++----
lib/unicorn/http_response.rb | 4 ++++
lib/unicorn/http_server.rb | 15 +++++++--------
3 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index 60a63b1..02e29c7 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -29,10 +29,12 @@ module Unicorn::Const
# :stopdoc:
# common errors we'll send back
- ERROR_400_RESPONSE = "400 Bad Request\r\n\r\n"
- ERROR_414_RESPONSE = "414 Request-URI Too Long\r\n\r\n"
- ERROR_413_RESPONSE = "413 Request Entity Too Large\r\n\r\n"
- ERROR_500_RESPONSE = "500 Internal Server Error\r\n\r\n"
+ # (N.B. these are not used by unicorn, but we won't drop them until
+ # unicorn 5.x to avoid breaking Rainbows!).
+ ERROR_400_RESPONSE = "HTTP/1.1 400 Bad Request\r\n\r\n"
+ ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
+ ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
+ ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
EXPECT_100_RESPONSE_SUFFIXED = "100 Continue\r\n\r\nHTTP/1.1 "
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index 61563cd..579d957 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -17,6 +17,10 @@ module Unicorn::HttpResponse
}
CRLF = "\r\n"
+ def err_response(code, response_start_sent)
+ "#{response_start_sent ? '' : 'HTTP/1.1 '}#{CODES[code]}\r\n\r\n"
+ end
+
# writes the rack_response to socket as an HTTP response
def http_response_write(socket, status, headers, body,
response_start_sent=false)
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index ef1ea58..aa98aeb 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -519,22 +519,21 @@ class Unicorn::HttpServer
# if the socket is already closed or broken. We'll always ensure
# the socket is closed at the end of this function
def handle_error(client, e)
- msg = case e
+ code = case e
when EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL,Errno::EBADF,
Errno::ENOTCONN
- Unicorn::Const::ERROR_500_RESPONSE
+ 500
when Unicorn::RequestURITooLongError
- Unicorn::Const::ERROR_414_RESPONSE
+ 414
when Unicorn::RequestEntityTooLargeError
- Unicorn::Const::ERROR_413_RESPONSE
+ 413
when Unicorn::HttpParserError # try to tell the client they're bad
- Unicorn::Const::ERROR_400_RESPONSE
+ 400
else
Unicorn.log_error(@logger, "app error", e)
- Unicorn::Const::ERROR_500_RESPONSE
+ 500
end
- msg = "HTTP/1.1 #{msg}" unless @request.response_start_sent
- client.kgio_trywrite(msg)
+ client.kgio_trywrite(err_response(code, @request.response_start_sent))
client.close
rescue
end
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 3%]
* release unicorn v4.2.0 soon?
@ 2012-01-23 9:20 4% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2012-01-23 9:20 UTC (permalink / raw)
To: mongrel-unicorn
I just realized we haven't had a release in a while. There's nothing
major that needs fixing, but I figure we should release 4.2.0 to
get some changes that have been brewing for a while.
The SSL support is still unaudited, but since Unicorn is only good on
LAN, it should be at least useful enough to detect network corruption.
TCP checksums are weak, so SSL can add an additional layer of
checksumming to the data stream if you're intent on end-to-end data
integrity. (Rack apps can also support the Content-MD5 header/trailer
without SSL too)
Here are the changes since v4.1.1 in git://bogomips.org/unicorn.git
Brian P O'Rourke (1):
Detect daemonization via configuration.
Eric Wong (18):
add GPLv3 option to the license
http_server: a few more things eligible for GC in worker
http_server: update comment on tick == 0
Links: add a link to the UnXF middleware
add preliminary SSL support
t: ensure SSL certificates exist on fresh test
configurator: limit timeout to 30 days
tests: just use the sha1sum implemented in Ruby
tests: try to set a shorter path for Unix domain sockets
bump dependencies
socket_helper: remove out-of-date comment for TCP_NODELAY
socket_helper: set SO_KEEPALIVE on TCP sockets
socket_helper: fix grammerr fail
quiet possible IOError from SIGUSR1 (reopen logs)
cleanup exception handling on SIGUSR1
http: test case for "Connection: TE"
update tests for rack 1.4.0
Rakefile: swap freshmeat.net URL for freecode.com
Jeremy Evans (2):
t0011: fix test under OpenBSD
test_helper: ensure test client connects to valid address
Yuichi Tateno (1):
OobGC: force GC.start
In the future, feel free to ping me if you want an official release, I'm
forgetful sometimes and sometimes get completely immersed in other
projects. "master" on unicorn.git is usually in a releasable state
anyways.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 4%]
* Re: Fix hang when running tests on OpenBSD by skipping two tests
@ 2011-11-16 1:18 5% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2011-11-16 1:18 UTC (permalink / raw)
To: unicorn list
Jeremy Evans <jeremyevans0@gmail.com> wrote:
> On Tue, Nov 15, 2011 at 4:17 AM, Eric Wong <normalperson@yhbt.net> wrote:
> > I usually prefer to work on each problem, one-at-a-time. However,
> > GNU make already has a handy -k/--keep-going flag to ignore failures.
>
> Thanks, I didn't know about that, and it is much easier than patching
> make files.
>
> I think I've fixed all the issues that caused test failures on
> OpenBSD. All changes are in the test code itself. Hope this helps.
OK, I think I've pushed relevant fixes up to master of unicorn.git
(commit fbcf6aa641e5827da48a3b6776c9897de123b405)
Eric Wong (3):
configurator: limit timeout to 30 days
tests: just use the sha1sum implemented in Ruby
tests: try to set a shorter path for Unix domain sockets
Jeremy Evans (2):
t0011: fix test under OpenBSD
test_helper: ensure test client connects to valid address
> Sorry if gmail mangles these diffs.
No worries, patch(1) is very lenient.
Just wondering, do most folks have/lack decent SMTP setups nowadays?
(especially on servers they don't usually work from)
When working without a VCS repo (live fixes on servers :x),
I'll sometimes just send a patch out like this:
diff -u a b | mail -s diff-a-b a@example.com
This is a big reason I prefer no-subscription-required mailing lists.
If it's to a public mailing list, I'll always email myself first to
cleanup the Received: trail and also to rewrite my email address
so it doesn't have @evil-corporation.com in it :)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 5%]
* Re: Unicorn 3.3.1 "Too many open files" error in kgio_tryaccept
2011-01-14 4:20 5% ` ghazel
@ 2011-01-14 9:17 0% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2011-01-14 9:17 UTC (permalink / raw)
To: unicorn list
ghazel@gmail.com wrote:
> On Thu, Jan 13, 2011 at 6:47 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > ghazel@gmail.com wrote:
> >> On Thu, Jan 13, 2011 at 3:06 PM, Eric Wong <normalperson@yhbt.net> wrote:
> >> > ghazel@gmail.com wrote:
> >> > How does lsof output look for your workers?
> >>
> >> Hm. The workers seem to be at 80-90 file descriptors each. I did catch
> >> one at 787 (!) with mostly handles to a geoip database from the geoip
> >> gem, but they got collected pretty quickly. Perhaps that's the cause!
> >
> > OK, that's a fairly likely cause of EMFILE.
> >
> > A tip for geoip users:
> >
> > Install the io-extra gem to get IO.pread. This allows you to reuse
> > the same file descriptor with geoip automatically between any number
> > of threads/processes without reopening it.
>
> Fascinating. Google tells be you've been down this road before. My
> code previously was doing:
> def get_city(ip)
> GeoIP.new("/path/to/geo.dat").city(ip)
> end
>
> Which seems to create one fd per call (and leave it for the GC to
> cleanup). What's the new proposed interface if io-extra is installed?
> Keep a global GeoIP object somewhere? My fix was to stuff it in to
> Thread.current, but obviously that has one fd per thread.
Yes, with io-extra, just define a constant somewhere and use it in any
thread/process at any time. No need to deal with locks or after_fork
hooks in Unicorn at all, either, pread() is just that awesome :)
GEO_DB = GeoIP.new("/path/to/geo.dat")
def get_city(ip)
GEO_DB.city(ip)
end
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: Unicorn 3.3.1 "Too many open files" error in kgio_tryaccept
@ 2011-01-14 4:20 5% ` ghazel
2011-01-14 9:17 0% ` Eric Wong
0 siblings, 1 reply; 43+ results
From: ghazel @ 2011-01-14 4:20 UTC (permalink / raw)
To: unicorn list
On Thu, Jan 13, 2011 at 6:47 PM, Eric Wong <normalperson@yhbt.net> wrote:
> ghazel@gmail.com wrote:
>> On Thu, Jan 13, 2011 at 3:06 PM, Eric Wong <normalperson@yhbt.net> wrote:
>> > ghazel@gmail.com wrote:
>> > How does lsof output look for your workers?
>>
>> Hm. The workers seem to be at 80-90 file descriptors each. I did catch
>> one at 787 (!) with mostly handles to a geoip database from the geoip
>> gem, but they got collected pretty quickly. Perhaps that's the cause!
>
> OK, that's a fairly likely cause of EMFILE.
>
> A tip for geoip users:
>
> Install the io-extra gem to get IO.pread. This allows you to reuse
> the same file descriptor with geoip automatically between any number
> of threads/processes without reopening it.
Fascinating. Google tells be you've been down this road before. My
code previously was doing:
def get_city(ip)
GeoIP.new("/path/to/geo.dat").city(ip)
end
Which seems to create one fd per call (and leave it for the GC to
cleanup). What's the new proposed interface if io-extra is installed?
Keep a global GeoIP object somewhere? My fix was to stuff it in to
Thread.current, but obviously that has one fd per thread.
-Greg
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 5%]
* Re: Unicorn 3.3.1 "Too many open files" error in kgio_tryaccept
@ 2011-01-13 23:06 5% ` Eric Wong
0 siblings, 1 reply; 43+ results
From: Eric Wong @ 2011-01-13 23:06 UTC (permalink / raw)
To: unicorn list
ghazel@gmail.com wrote:
> Strange error which took down my server today:
>
> Unhandled listen loop exception #<Errno::EMFILE: Too many open files - accept>.
> ...
>
> My unicorn.stderr.log is 7.6GB, and I rotate it every day...
>
> Before all that started I got:
>
> Read error: #<IOError: closed stream>
> /usr/local/ruby-enterprise-1.8.7-2010.01/lib/ruby/gems/1.8/gems/unicorn-3.3.1/lib/unicorn/http_response.rb:37:in
>
> But the servers seemed to be basically operating at that point.
Weird. I have many questions:
Are you processing uploads at all?
Is nginx in front of Unicorn?
Do you open a lot of files in your application and never close them?[1]
How does lsof output look for your workers?
Can you check any exception capturing/logging middleware you have
for any other errors that may be swallowed? (especially
while reading env["rack.input"])
I changed the socket close ordering for 3.3.1 to workaround an esoteric
"bug", but I don't see how this could happen right now...
Does this happen with 3.3.0?
Or 1.1.6? hiswhich has the same "bugfix" as 3.3.1.
Also, is anybody else having this problem?
[1] - GC should be able to cleanup open files on MRI/REE anyways
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 5%]
* Re: [ANN] unicorn 1.0.1 and 1.1.2 - fix rollbacks
2010-07-15 18:07 5% ` Jamie Wilkinson
@ 2010-07-16 0:47 0% ` Lawrence Pit
0 siblings, 0 replies; 43+ results
From: Lawrence Pit @ 2010-07-16 0:47 UTC (permalink / raw)
To: unicorn list
>> It appears unicorn rolls back by itself when it can't start the new master. Nice!
>>
>> Jamie also mentions to use the shared vendor/bundler_gems path. Though I do use that in all my projects I'm not so sure it's wise to do when you can't afford any downtime. I'll have to wait for unicorn v1.1.3 before I can test whether rolling back will indeed continue with unicorn v1.1.2 instead of v1.1.3 ;o
>>
>
> Hi Lawrence, why does the shared vendor/bundler_gems cause you downtime? From not re-bundling during rollback?
>
> FWIW I made that recommendation because I ran into issues with unicorns not restarting correctly after running `cap deploy:cleanup`, since the `bundle exec` launches a binary with a path like /app/releases/201007125236/bin/unicorn, which goes missing after N deploys. I haven't tested if this applies to more recent versions
>
> Shared bundles are also significantly faster -- `bundle check || bundle install` is ~1s for me vs. several minutes to totally rebundle
>
Hi Jamie, I see what you mean. I haven't tested this yet, so I don't
know for sure if it could cause downtime. I also always use bundle check
|| bundle install to get that time benefit.
My worry was indeed that it might pick a newer unicorn to restart
workers for the old master. I guess that can/should be prevented by
having an on_rollback in my capistrano bundler task.
What is the exact command you use to start unicorn?
What I see is this, I start with:
$ su -l deployer -c "cd /app/current ; bundle exec unicorn_rails -D -c
/app/current/config/unicorn.rb"
My after_fork and before_exec methods output the ENV vars to the unicorn
log. I see for example:
AFTER_FORK: VERSION 1
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714054705/vendor/bundler_gems
GEM_PATH=/app/releases/20100714054705/vendor/bundler_gems:/app/releases/20100714054705/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile
Note that /app/releases/20100714054705/vendor/bundler_gems is a symlink
to /app/releases/shared/vendor/bundler_gems
When I deploy a new version, symlink /app/current to the new release
directory, and send USR2 :
executing
["/app/releases/20100714054705/vendor/bundler_gems/bin/unicorn_rails",
"-D", "-E", "staging", "-c", "/app/current/config/unicorn.rb"] (in
/app/releases/20100714055624)
BEFORE_EXEC: VERSION 1
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714054705/vendor/bundler_gems
GEM_PATH=/app/releases/20100714054705/vendor/bundler_gems:/app/releases/20100714054705/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile
I, [2010-07-15T23:31:47.765995 #23084] INFO -- : inherited
addr=/tmp/.unicorn_sock fd=3
I, [2010-07-15T23:31:47.766646 #23084] INFO -- : Refreshing Gem list
/app/releases/20100714055624/.bundle/environment.rb:175: warning:
already initialized constant ENV_LOADED
/app/releases/20100714055624/.bundle/environment.rb:176: warning:
already initialized constant LOCKED_BY
/app/releases/20100714055624/.bundle/environment.rb:177: warning:
already initialized constant FINGERPRINT
/app/releases/20100714055624/.bundle/environment.rb:178: warning:
already initialized constant HOME
/app/releases/20100714055624/.bundle/environment.rb:179: warning:
already initialized constant AUTOREQUIRES
/app/releases/20100714055624/.bundle/environment.rb:181: warning:
already initialized constant SPECS
AFTER_FORK: VERSION 2
PATH=/app/releases/20100714054705/vendor/bundler_gems/bin:....
GEM_HOME=/app/releases/20100714055624/vendor/bundler_gems
GEM_PATH=/app/releases/20100714055624/vendor/bundler_gems:/app/releases/20100714055624/vendor/bundler_gems/bundler/gems
BUNDLE_GEMFILE=/app/releases/20100714054705/Gemfile
What you see here is that the new worker does have correct GEM_HOME and
GEM_PATH values, but the PATH and BUNDLE_GEMFILE values are pointing to
the old dir.
Are those bundler warnings something to worry about?
The BUNDLE_GEMFILE value is worrying I think. I haven't tested this, but
I'm pretty sure when Bundler.setup is called within your app it will
actually setup using the old Gemfile then. So that would mean you need
to reset it somehow? I don't see how at the moment... should unicorn
provide a method similar to +working_directory+ to help ensure the
application always uses the "current" Gemfile (eg
"/app/current/Gemfile") ? Sound kind of strange unicorn should provide
such a method, is there another way?
What about that PATH value? What happens if 10 deployments later the
dir /app/releases/20100714054705 is pruned? I tried. After removing that
dir I could still send a HUP, but when I send a USR2 I get this:
/app/releases/20100714054705/vendor/bundler_gems/gems/unicorn-1.1.2/lib/unicorn.rb:573:in
`exec': No such file or directory -
app/releases/20100714054705/vendor/bundler_gems/bin/unicorn_rails
(Errno::ENOENT)
from
/app/releases/20100714054705/vendor/bundler_gems/gems/unicorn-1.1.2/lib/unicorn.rb:573:in
`reexec'
> since the `bundle exec` launches a binary with a path like
/app/releases/201007125236/bin/unicorn,
> which goes missing after N deploys
That's what I'm seeing. How did using shared vendor/bundler_gems help
you? Because I'm starting with bundle exec using shared
vendor/bundler_gems as well.
Cheers,
Lawrence
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 0%]
* Re: [ANN] unicorn 1.0.1 and 1.1.2 - fix rollbacks
@ 2010-07-15 18:07 5% ` Jamie Wilkinson
2010-07-16 0:47 0% ` Lawrence Pit
0 siblings, 1 reply; 43+ results
From: Jamie Wilkinson @ 2010-07-15 18:07 UTC (permalink / raw)
To: unicorn list
On Jul 14, 2010, at 12:00 AM, Lawrence Pit wrote:
> It appears unicorn rolls back by itself when it can't start the new master. Nice!
>
> Jamie also mentions to use the shared vendor/bundler_gems path. Though I do use that in all my projects I'm not so sure it's wise to do when you can't afford any downtime. I'll have to wait for unicorn v1.1.3 before I can test whether rolling back will indeed continue with unicorn v1.1.2 instead of v1.1.3 ;o
Hi Lawrence, why does the shared vendor/bundler_gems cause you downtime? From not re-bundling during rollback?
FWIW I made that recommendation because I ran into issues with unicorns not restarting correctly after running `cap deploy:cleanup`, since the `bundle exec` launches a binary with a path like /app/releases/201007125236/bin/unicorn, which goes missing after N deploys. I haven't tested if this applies to more recent versions
Shared bundles are also significantly faster -- `bundle check || bundle install` is ~1s for me vs. several minutes to totally rebundle
-jamie
http://jamiedubs.com
http://fffff.at
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 5%]
* anything left before 1.0?
@ 2010-06-16 0:09 7% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2010-06-16 0:09 UTC (permalink / raw)
To: mongrel-unicorn
Hi all,
Since some folks have been running Unicorn in production for a while,
we might as well call the next release 1.0.
I have a couple of minor fixes in unicorn.git on top of 0.991.0.
Mainly, there are small compatibility fixes for the future (MRI 1.9.2
and Rubinius) and the past (MRI 1.8.6(!)).
There are also some documentation updates for Rack 1.2, but we should be
compatible with everything from Rack 0.9 (possibly earlier) to the
latest Rack 1.2.
Shortlog since 0.991.0:
Eric Wong (8):
doc: cleanup rdoc escaping in title, hopefully
cleanup: use modules were applicable
t0005: Ruby 1.9.2 $LOAD_PATH fix
http: fix rb_str_set_len() define for 1.8.6
app/exec_cgi: rack.input may not respond to #size
local.mk.sample: add 1.8.6-p114 to "full-test"
tee_input: update documentation for Rack 1.2
FAQ: help folks help themselves for processes dying in a loop
Let us know if there's anything else missing, pipe up within the next 24
hours or so...
You can always email me privately if you don't want your address on the
public mailing list. Don't send HTML mail, though.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 7%]
* Re: Issues since 0.991.0
@ 2010-06-12 18:37 5% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2010-06-12 18:37 UTC (permalink / raw)
To: unicorn list
"Jose Avila(Tachu)" <tachu@crowdstar.com> wrote:
> Hi Guys we currently have a farm of servers that run 0.990.0 and new ones that started running 0.991.0 Since 0.991.0 We get this error
>
> unicorn_rails worker[24] -c config/unicorn.rb -E production: symbol lookup error: /usr/lib64/ruby/gems/1.8/gems/unicorn-
> 0.991.0/lib/unicorn_http.so: undefined symbol: rb_str_set_len
>
> Any idea as to why that is happening and how we can fix it?
Hi Jose, I was too aggressive with a cleanup for Rubinius compatibility
that I broke things for 1.8.6 (and possibly before).
I've pushed out the following fix and also a 0.991.0.4.g0cb0 prerelease
to RubyGems.org. You can install it with "gem install --pre unicorn"
Let me know how it works for you, thanks for the report!
>From 0cb02efd0e2e2c80667863fd404d1fad7c63bb1f Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Sat, 12 Jun 2010 18:32:12 +0000
Subject: [PATCH] http: fix rb_str_set_len() define for 1.8.6
---
ext/unicorn_http/ext_help.h | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/ext/unicorn_http/ext_help.h b/ext/unicorn_http/ext_help.h
index cc157cb..3aa24a8 100644
--- a/ext/unicorn_http/ext_help.h
+++ b/ext/unicorn_http/ext_help.h
@@ -22,6 +22,7 @@ static void rb_18_str_set_len(VALUE str, long len)
RSTRING(str)->len = len;
RSTRING(str)->ptr[len] = '\0';
}
+# define rb_str_set_len(str,len) rb_18_str_set_len(str,len)
#endif /* !defined(HAVE_RB_STR_SET_LEN) */
/* not all Ruby implementations support frozen objects (Rubinius does not) */
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 5%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-08 19:20 4% ` Eric Wong
2010-06-08 19:25 6% ` Hongli Lai
@ 2010-06-11 20:32 6% ` Eric Wong
1 sibling, 0 replies; 43+ results
From: Eric Wong @ 2010-06-11 20:32 UTC (permalink / raw)
To: unicorn list; +Cc: Hongli Lai
Eric Wong <normalperson@yhbt.net> wrote:
> Hongli Lai <hongli@phusion.nl> wrote:
> > Hi Eric.
> >
> > It would appear that recent Rails 3 changes have broken unicorn_rails,
> > just like they broke Phusion Passenger. Here's a patch which fixes the
> > problem.
> > http://gist.github.com/429944
>
> Thanks Hongli, so this only affects people who remove the
> config.ru that Rails 3 creates for them? Yikes...
And for the completely bizzare: Rails 3 is ultra aggressive
when searching for config.ru, it walks up the directory tree,
even going all the way up to "/" if it can't find config.ru.
I had a $HOME/config.ru leftover from a test on one of my boxes, and
usually have my working directory in $HOME/unicorn. This was was
causing the unicorn_rails test for this (t0301) to fail because that
test relies on Rails /not/ being able to find config.ru.
So yes, don't leave random artifacts lying around and love strace :)
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 6%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-08 19:25 6% ` Hongli Lai
@ 2010-06-08 20:55 6% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2010-06-08 20:55 UTC (permalink / raw)
To: unicorn list; +Cc: Hongli Lai
Hongli Lai <hongli@phusion.nl> wrote:
> On Tue, Jun 8, 2010 at 9:20 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Thanks Hongli, so this only affects people who remove the
> > config.ru that Rails 3 creates for them? Yikes...
>
> No. The problem even occurs if you already have config.ru. But the
> thing is, Rails 3 has deprecated ActionController::Dispatcher a few
> weeks ago and replaced it with a stub. Rails::Rack::Static changed its
> interface and must be constructed differently. The only way to obtain
> a valid Rack endpoint for the app seems to be parsing
> config/application.rb
Actually, I made "unicorn_rails" completely bypass the "rails_builder"
method if there's a config.ru, so it should never hit the
ActionController::Dispatcher stuff.
> > Let me know if the edited patch below looks alright to you.
>
> Yes it looks fine. A bit overcomplicated regexp compared to using
> 'strip' but whatever works. :)
I just kept the regexp as-is, works for me.
I just managed to push this to git://git.bogomips.org/unicorn.git before
my Internet connection died on me earlier today. I've beefed up the
tests a bit but will probably do more later.
Eric Wong (4):
t0300: Rails 3 test actually uses unicorn_rails
tests: libify common rails3 setup code
unicorn_rails: fix requires for Ruby 1.9.2
tests: add Rails 3 test for the missing config.ru case
Hongli Lai (Phusion) (1):
Fix unicorn_rails compatibility with the latest Rails 3 code
Thanks again!
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 6%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-08 19:20 4% ` Eric Wong
@ 2010-06-08 19:25 6% ` Hongli Lai
2010-06-08 20:55 6% ` Eric Wong
2010-06-11 20:32 6% ` Eric Wong
1 sibling, 1 reply; 43+ results
From: Hongli Lai @ 2010-06-08 19:25 UTC (permalink / raw)
To: Eric Wong; +Cc: unicorn list
On Tue, Jun 8, 2010 at 9:20 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Thanks Hongli, so this only affects people who remove the
> config.ru that Rails 3 creates for them? Yikes...
No. The problem even occurs if you already have config.ru. But the
thing is, Rails 3 has deprecated ActionController::Dispatcher a few
weeks ago and replaced it with a stub. Rails::Rack::Static changed its
interface and must be constructed differently. The only way to obtain
a valid Rack endpoint for the app seems to be parsing
config/application.rb
> A few comments:
>
> + if ::Rails::VERSION::MAJOR >= 3 && ::File.exist?('config/application.rb')
> + ::File.read('config/application.rb') =~ /^module (.+)$/
>
> Maybe a more flexible regexp like this: /^module\s+([\w:]+)\s*$/ (or
> maybe even starting with: "^\s*") would be more reliable for folks who
> leave extra spaces around.
I think it's easier to just call 'strip'. :)
> Unfortunately, Ruby is not Python :)
>
> @@ -148,9 +167,9 @@ def rails_builder(daemonize)
> else
> use Rails::Rack::LogTailer unless daemonize
> use Rails::Rack::Debugger if $DEBUG
> + use Rails::Rack::Static, map_path
> map(map_path) do
> - use Rails::Rack::Static
> - run ActionController::Dispatcher.new
> + run rails_dispatcher
>
> Changing the call to use Rails::Rack::Static there is wrong. map_path
> is the URI prefix (RAILS_RELATIVE_URL_ROOT) and not the static file
> path we serve from. It appears the deprecation in Rails 3 broke some
> things and ActionDispatch::Static is configured slightly differently.
>
> Let me know if the edited patch below looks alright to you.
Yes it looks fine. A bit overcomplicated regexp compared to using
'strip' but whatever works. :)
With kind regards,
Hongli Lai
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: info@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 6%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-08 12:24 7% Hongli Lai
@ 2010-06-08 19:20 4% ` Eric Wong
2010-06-08 19:25 6% ` Hongli Lai
2010-06-11 20:32 6% ` Eric Wong
0 siblings, 2 replies; 43+ results
From: Eric Wong @ 2010-06-08 19:20 UTC (permalink / raw)
To: unicorn list; +Cc: Hongli Lai
Hongli Lai <hongli@phusion.nl> wrote:
> Hi Eric.
>
> It would appear that recent Rails 3 changes have broken unicorn_rails,
> just like they broke Phusion Passenger. Here's a patch which fixes the
> problem.
> http://gist.github.com/429944
Thanks Hongli, so this only affects people who remove the
config.ru that Rails 3 creates for them? Yikes...
A few comments:
+ if ::Rails::VERSION::MAJOR >= 3 && ::File.exist?('config/application.rb')
+ ::File.read('config/application.rb') =~ /^module (.+)$/
Maybe a more flexible regexp like this: /^module\s+([\w:]+)\s*$/ (or
maybe even starting with: "^\s*") would be more reliable for folks who
leave extra spaces around.
Unfortunately, Ruby is not Python :)
@@ -148,9 +167,9 @@ def rails_builder(daemonize)
else
use Rails::Rack::LogTailer unless daemonize
use Rails::Rack::Debugger if $DEBUG
+ use Rails::Rack::Static, map_path
map(map_path) do
- use Rails::Rack::Static
- run ActionController::Dispatcher.new
+ run rails_dispatcher
Changing the call to use Rails::Rack::Static there is wrong. map_path
is the URI prefix (RAILS_RELATIVE_URL_ROOT) and not the static file
path we serve from. It appears the deprecation in Rails 3 broke some
things and ActionDispatch::Static is configured slightly differently.
Let me know if the edited patch below looks alright to you.
I'll also push out a few Rails 3 tests that exercise the missing
config.ru cases.
>From 222ae0a353eda446a480e5c4b473a218304f9594 Mon Sep 17 00:00:00 2001
From: Hongli Lai (Phusion) <hongli@phusion.nl>
Date: Tue, 8 Jun 2010 13:22:25 +0200
Subject: [PATCH] Fix unicorn_rails compatibility with the latest Rails 3 code
This allows us to properly detect Rails 3 installations
in cases where config.ru is not present.
[ew: expanded commit message
fixed static file serving,
more flexible regexp for matching module ]
ref: mid.gmane.org/AANLkTiksBxIo_PFWoiPTWi1entXZRb7D2uE-Rl7H3lbw@mail.gmail.com
Acked-by: Eric Wong <normalperson@yhbt.net>
---
bin/unicorn_rails | 26 ++++++++++++++++++++++++--
1 files changed, 24 insertions(+), 2 deletions(-)
diff --git a/bin/unicorn_rails b/bin/unicorn_rails
index 45a9b11..ed235ba 100755
--- a/bin/unicorn_rails
+++ b/bin/unicorn_rails
@@ -109,6 +109,24 @@ end
ru = ARGV[0] || (File.exist?('config.ru') ? 'config.ru' : nil)
+def rails_dispatcher
+ if ::Rails::VERSION::MAJOR >= 3 && ::File.exist?('config/application.rb')
+ if ::File.read('config/application.rb') =~ /^module\s+([\w:]+)\s*$/
+ app_module = Object.const_get($1)
+ begin
+ result = app_module::Application
+ rescue NameError
+ end
+ end
+ end
+
+ if result.nil? && defined?(ActionController::Dispatcher)
+ result = ActionController::Dispatcher.new
+ end
+
+ result || abort("Unable to locate the application dispatcher class")
+end
+
def rails_builder(daemonize)
# this lambda won't run until after forking if preload_app is false
lambda do ||
@@ -149,8 +167,12 @@ def rails_builder(daemonize)
use Rails::Rack::LogTailer unless daemonize
use Rails::Rack::Debugger if $DEBUG
map(map_path) do
- use Rails::Rack::Static
- run ActionController::Dispatcher.new
+ if defined?(ActionDispatch::Static)
+ use ActionDispatch::Static, "#{Rails.root}/public"
+ else
+ use Rails::Rack::Static
+ end
+ run rails_dispatcher
end
end
end.to_app
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 4%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
@ 2010-06-08 12:24 7% Hongli Lai
2010-06-08 19:20 4% ` Eric Wong
0 siblings, 1 reply; 43+ results
From: Hongli Lai @ 2010-06-08 12:24 UTC (permalink / raw)
To: mongrel-unicorn
Hi Eric.
It would appear that recent Rails 3 changes have broken unicorn_rails,
just like they broke Phusion Passenger. Here's a patch which fixes the
problem.
http://gist.github.com/429944
With kind regards,
Hongli Lai
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: info@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 7%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-04 1:58 4% unicorn_rails cleanup (possible fix for Rails3) pushed Eric Wong
2010-06-04 2:13 7% ` Michael Guterl
@ 2010-06-08 2:49 7% ` Eric Wong
1 sibling, 0 replies; 43+ results
From: Eric Wong @ 2010-06-08 2:49 UTC (permalink / raw)
To: mongrel-unicorn
Eric Wong <normalperson@yhbt.net> wrote:
> Hi all,
>
> I've pushed the following patch out go git://git.bogomips.org/unicorn
> along with a few other Rails-related test updates. This is more of a
> shotgun fix (but less code is better :) since I haven't been able to
> reproduce the brokeness people have been seeing with "unicorn_rails"
> and Rails 3 betas.
<snip>
> You can grab a git gem here, too:
>
> http://unicorn.bogomips.org/files/unicorn-0.99.0.16.g59a625.gem
Anybody get a chance to try this?
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 7%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-04 2:13 7% ` Michael Guterl
@ 2010-06-04 2:48 7% ` Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2010-06-04 2:48 UTC (permalink / raw)
To: unicorn list
Michael Guterl <mguterl@gmail.com> wrote:
> On Thu, Jun 3, 2010 at 9:58 PM, Eric Wong <normalperson@yhbt.net> wrote:
> <snip>
> > If I were to do it all over again, I would've just made everybody write
> > config.ru files for Rails. But yes, programming is like sex, make one
> > mistake and support it for life :)
>
> That is probably one of the funniest things I've read in awhile.
> Thanks for the unexpected source of humor! :)
You're welcome :> It's not my line, but I heard it many years ago.
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 7%]
* Re: unicorn_rails cleanup (possible fix for Rails3) pushed
2010-06-04 1:58 4% unicorn_rails cleanup (possible fix for Rails3) pushed Eric Wong
@ 2010-06-04 2:13 7% ` Michael Guterl
2010-06-04 2:48 7% ` Eric Wong
2010-06-08 2:49 7% ` Eric Wong
1 sibling, 1 reply; 43+ results
From: Michael Guterl @ 2010-06-04 2:13 UTC (permalink / raw)
To: unicorn list
On Thu, Jun 3, 2010 at 9:58 PM, Eric Wong <normalperson@yhbt.net> wrote:
<snip>
> If I were to do it all over again, I would've just made everybody write
> config.ru files for Rails. But yes, programming is like sex, make one
> mistake and support it for life :)
That is probably one of the funniest things I've read in awhile.
Thanks for the unexpected source of humor! :)
Best,
Michael Guterl
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [relevance 7%]
* unicorn_rails cleanup (possible fix for Rails3) pushed
@ 2010-06-04 1:58 4% Eric Wong
2010-06-04 2:13 7% ` Michael Guterl
2010-06-08 2:49 7% ` Eric Wong
0 siblings, 2 replies; 43+ results
From: Eric Wong @ 2010-06-04 1:58 UTC (permalink / raw)
To: mongrel-unicorn
Hi all,
I've pushed the following patch out go git://git.bogomips.org/unicorn
along with a few other Rails-related test updates. This is more of a
shotgun fix (but less code is better :) since I haven't been able to
reproduce the brokeness people have been seeing with "unicorn_rails"
and Rails 3 betas.
Even though "unicorn" works perfectly well for Rails3, some folks will
inevitably run "unicorn_rails" because of the "rails" in its name.
If I were to do it all over again, I would've just made everybody write
config.ru files for Rails. But yes, programming is like sex, make one
mistake and support it for life :)
You can grab a git gem here, too:
http://unicorn.bogomips.org/files/unicorn-0.99.0.16.g59a625.gem
>From 4b44e21957e4cb8ec6ace5604fbe096dfd8959d2 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Thu, 3 Jun 2010 22:52:11 +0000
Subject: [PATCH] unicorn_rails: avoid duplicating config.ru logic
This should allow "unicorn_rails" to be used seamlessly
with Rails 3 projects which package config.ru for you.
---
bin/unicorn_rails | 50 ++++++++++++++------------------------------------
1 files changed, 14 insertions(+), 36 deletions(-)
diff --git a/bin/unicorn_rails b/bin/unicorn_rails
index 72ab288..45a9b11 100755
--- a/bin/unicorn_rails
+++ b/bin/unicorn_rails
@@ -109,53 +109,30 @@ end
ru = ARGV[0] || (File.exist?('config.ru') ? 'config.ru' : nil)
-if ru && ru =~ /\.ru\z/
- # parse embedded command-line options in config.ru comments
- /^#\\(.*)/ =~ File.read(ru) and opts.parse!($1.split(/\s+/))
-end
-
-def rails_builder(ru, daemonize)
+def rails_builder(daemonize)
# this lambda won't run until after forking if preload_app is false
lambda do ||
# Load Rails and (possibly) the private version of Rack it bundles.
begin
require 'config/boot'
+ require 'config/environment'
rescue LoadError => err
abort "#$0 must be run inside RAILS_ROOT: #{err.inspect}"
end
- inner_app = case ru
- when nil
- require 'config/environment'
-
- defined?(::Rails::VERSION::STRING) or
- abort "Rails::VERSION::STRING not defined by config/{boot,environment}"
- # it seems Rails >=2.2 support Rack, but only >=2.3 requires it
- old_rails = case ::Rails::VERSION::MAJOR
- when 0, 1 then true
- when 2 then Rails::VERSION::MINOR < 3 ? true : false
- else
- false
- end
-
- if old_rails
- require 'unicorn/app/old_rails'
- Unicorn::App::OldRails.new
- else
- ActionController::Dispatcher.new
- end
- when /\.ru$/
- raw = File.read(ru)
- raw.sub!(/^__END__\n.*/, '')
- eval("Rack::Builder.new {(#{raw}\n)}.to_app", TOPLEVEL_BINDING, ru)
+ defined?(::Rails::VERSION::STRING) or
+ abort "Rails::VERSION::STRING not defined by config/{boot,environment}"
+ # it seems Rails >=2.2 support Rack, but only >=2.3 requires it
+ old_rails = case ::Rails::VERSION::MAJOR
+ when 0, 1 then true
+ when 2 then Rails::VERSION::MINOR < 3 ? true : false
else
- require ru
- Object.const_get(File.basename(ru, '.rb').capitalize)
+ false
end
Rack::Builder.new do
map_path = ENV['RAILS_RELATIVE_URL_ROOT'] || '/'
- if inner_app.class.to_s == "Unicorn::App::OldRails"
+ if old_rails
if map_path != '/'
# patches + tests welcome, but I really cbf to deal with this
# since all apps I've ever dealt with just use "/" ...
@@ -163,23 +140,24 @@ def rails_builder(ru, daemonize)
end
$stderr.puts "LogTailer not available for Rails < 2.3" unless daemonize
$stderr.puts "Debugger not available" if $DEBUG
+ require 'unicorn/app/old_rails'
map(map_path) do
use Unicorn::App::OldRails::Static
- run inner_app
+ run Unicorn::App::OldRails.new
end
else
use Rails::Rack::LogTailer unless daemonize
use Rails::Rack::Debugger if $DEBUG
map(map_path) do
use Rails::Rack::Static
- run inner_app
+ run ActionController::Dispatcher.new
end
end
end.to_app
end
end
-app = rails_builder(ru, daemonize)
+app = ru ? Unicorn.builder(ru, opts) : rails_builder(daemonize)
options[:listeners] << "#{host}:#{port}" if set_listener
if $DEBUG
--
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply related [relevance 4%]
* Re: unicorn, rails, and symlinked deployment
2009-11-17 22:20 0% ` Eric Wong
@ 2009-11-18 15:34 0% ` Michael Guterl
0 siblings, 0 replies; 43+ results
From: Michael Guterl @ 2009-11-18 15:34 UTC (permalink / raw)
To: unicorn list
On Tue, Nov 17, 2009 at 5:20 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Michael Guterl <mguterl@gmail.com> wrote:
>> First let me say thanks for Unicorn, it has helped us fill a gap that
>> Passenger could not fill.
>>
>> Like many using Rails, we use capistrano for deployment. At the end
>> of each deployment we use the standard capistrano deploy:cleanup task
>> to remove old releases. Everything is fine until we cleanup the
>> release directory from which unicorn_rails was originally launched.
>> When this happens we get an error in our unicorn error log.
>>
>> reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
>> error reloading
>> config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb:
>> Errno::ENOENT No such file or directory -
>> /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
>>
>> I'm sure I'm not the only who has experienced this. Does anyone have
>> any recommendations for handling this situation?
>
> Hi Michael,
>
> Unicorn was definitely implemented with Cap in mind. Which version of
> Unicorn are you running? 0.94.0 got better symlink detection for
> non-GNU/Linux systems, so if you start in
> /home/deploy/public_html/rm/current, then it should detect it and use
> /home/deploy/public_html/rm/current/config/unicorn.rb.
>
I am using unicorn v0.94.0 on Ubuntu 8.04.
> If you don't start in your "current" symlink dir, 0.94.0 also got
> support for "working_directory" in your config file.
>
I set this option in my config, however, it did not help.
After some experimentation I realized the problem is in the script I
use for managing unicorn via /etc/init.d --
http://gist.github.com/237953
Essentially start does this:
cd ~/public_html/rm/current
unicorn_rails -D -E production -c config/unicorn.rb
I determined the -c config/unicorn.rb was the problem. If I change it to:
unicorn_rails -D -E production -c ~/public_html/rm/current/config/unicorn.rb
everything works fine.
My updated init script is located at http://gist.github.com/237958 and
it seems to work fine.
Best regards,
Michael Guterl
_______________________________________________
mongrel-unicorn mailing list
mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
^ permalink raw reply [relevance 0%]
* Re: unicorn, rails, and symlinked deployment
2009-11-17 17:50 7% unicorn, rails, and symlinked deployment Michael Guterl
@ 2009-11-17 22:08 0% ` Michael Guterl
2009-11-17 22:20 0% ` Eric Wong
1 sibling, 0 replies; 43+ results
From: Michael Guterl @ 2009-11-17 22:08 UTC (permalink / raw)
To: mongrel-unicorn
On Tue, Nov 17, 2009 at 12:50 PM, Michael Guterl <mguterl@gmail.com> wrote:
> First let me say thanks for Unicorn, it has helped us fill a gap that
> Passenger could not fill.
>
> Like many using Rails, we use capistrano for deployment. At the end
> of each deployment we use the standard capistrano deploy:cleanup task
> to remove old releases. Everything is fine until we cleanup the
> release directory from which unicorn_rails was originally launched.
> When this happens we get an error in our unicorn error log.
>
> reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
> error reloading
> config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb:
> Errno::ENOENT No such file or directory -
> /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
>
> I'm sure I'm not the only who has experienced this. Does anyone have
> any recommendations for handling this situation?
>
I should also point out that in my unicorn logs when I start the app
it references the path in it's symbolic link form:
I, [2009-11-17T17:06:10.215485 #30857] INFO -- : unlinking existing
socket=/home/deploy/public_html/rm/current/tmp/sockets/unicorn.sock
I, [2009-11-17T17:06:10.227485 #30857] INFO -- : listening on
addr=/home/deploy/public_html/rm/current/tmp/sockets/unicorn.sock fd=3
I, [2009-11-17T17:06:10.227485 #30857] INFO -- : Refreshing Gem list
But when I send HUP to restart, it references the actual path, not the symlink.
reloading config_file=/home/deploy/public_html/rm/releases/20091117215841/config/unicorn.rb
Refreshing Gem list
done reloading config_file=/home/deploy/public_html/rm/releases/20091117215841/config/unicorn.rb
Best regards,
Michael Guterl
_______________________________________________
mongrel-unicorn mailing list
mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
^ permalink raw reply [relevance 0%]
* Re: unicorn, rails, and symlinked deployment
2009-11-17 17:50 7% unicorn, rails, and symlinked deployment Michael Guterl
2009-11-17 22:08 0% ` Michael Guterl
@ 2009-11-17 22:20 0% ` Eric Wong
2009-11-18 15:34 0% ` Michael Guterl
1 sibling, 1 reply; 43+ results
From: Eric Wong @ 2009-11-17 22:20 UTC (permalink / raw)
To: unicorn list
Michael Guterl <mguterl@gmail.com> wrote:
> First let me say thanks for Unicorn, it has helped us fill a gap that
> Passenger could not fill.
>
> Like many using Rails, we use capistrano for deployment. At the end
> of each deployment we use the standard capistrano deploy:cleanup task
> to remove old releases. Everything is fine until we cleanup the
> release directory from which unicorn_rails was originally launched.
> When this happens we get an error in our unicorn error log.
>
> reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
> error reloading
> config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb:
> Errno::ENOENT No such file or directory -
> /home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
>
> I'm sure I'm not the only who has experienced this. Does anyone have
> any recommendations for handling this situation?
Hi Michael,
Unicorn was definitely implemented with Cap in mind. Which version of
Unicorn are you running? 0.94.0 got better symlink detection for
non-GNU/Linux systems, so if you start in
/home/deploy/public_html/rm/current, then it should detect it and use
/home/deploy/public_html/rm/current/config/unicorn.rb.
If you don't start in your "current" symlink dir, 0.94.0 also got
support for "working_directory" in your config file.
--
Eric Wong
^ permalink raw reply [relevance 0%]
* unicorn, rails, and symlinked deployment
@ 2009-11-17 17:50 7% Michael Guterl
2009-11-17 22:08 0% ` Michael Guterl
2009-11-17 22:20 0% ` Eric Wong
0 siblings, 2 replies; 43+ results
From: Michael Guterl @ 2009-11-17 17:50 UTC (permalink / raw)
To: mongrel-unicorn
First let me say thanks for Unicorn, it has helped us fill a gap that
Passenger could not fill.
Like many using Rails, we use capistrano for deployment. At the end
of each deployment we use the standard capistrano deploy:cleanup task
to remove old releases. Everything is fine until we cleanup the
release directory from which unicorn_rails was originally launched.
When this happens we get an error in our unicorn error log.
reloading config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
error reloading
config_file=/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb:
Errno::ENOENT No such file or directory -
/home/deploy/public_html/rm/releases/20091116213921/config/unicorn.rb
I'm sure I'm not the only who has experienced this. Does anyone have
any recommendations for handling this situation?
Best regards,
Michael Guterl
^ permalink raw reply [relevance 7%]
* [ANN] unicorn 0.94.0 - small fixes and new features
@ 2009-11-05 10:06 5% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2009-11-05 10:06 UTC (permalink / raw)
To: mongrel-unicorn
Unicorn is a HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels. Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.
* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git
Changes:
The HTTP parser is fix for oddly-aligned reads of trailers (this
technically affects headers, too, but is highly unlikely due to
our non-support of slow clients). This allows our HTTP parser
to better support very slow clients when used by other servers
(like Rainbows!). Fortunately this bug does not appear to lead
to any invalid memory accesses (and potential arbitrary code
execution).
FreeBSD (and possibly other *BSDs) support is improved and and
all the test cases pass under FreeBSD 7.2. Various flavors of
GNU/Linux remains our primary platform for development and
production.
New features added include the "working_directory" directive in
the configurator . Even without specifying a
"working_directory", symlink-aware detection of the current path
no longer depends on /bin/sh so it should work out-of-the-box on
FreeBSD and Solaris and not just systems where /bin/sh is dash,
ksh93 or bash.
User-switching support is finally supported but only intended
for use in the after_fork hook of worker processes. Putting it
in the after_fork hook allows allows users to set things like
CPU affinity[1] on a per-worker basis before dropping
privileges. The master process retains all privileges it
started with.
The ENV["RACK_ENV"] (process-wide) environment variable is now
both read and set for `unicorn' in the same way RAILS_ENV is
used by `unicorn_rails'. This allows the Merb launcher to read
ENV["RACK_ENV"] in config.ru. Other web servers already set
this and there may be applications or libraries that already
rely on this de facto standard.
Eric Wong (26):
cleanup: avoid redundant error checks for fstat
test_helper: connect(2) may fail with EINVAL
GNUmakefile: fix non-portable tar(1) usage
tests: provide a pure Ruby setsid(8) equivalent
more portable symlink awareness for START_CTX[:cwd]
test_signals: avoid portability issues with fchmod(2)
cleanup error handling and make it less noisy
Do not override Dir.chdir in config files
configurator: add "working_directory" directive
configurator: working_directory is expanded
configurator: set ENV["PWD"] with working_directory, too
configurator: working_directory affects pid, std{err,out}_paths
configurator: update documentation for working_directory
TODO: remove working_directory bit, done
Util.reopen_logs: remove needless Range
worker: user/group switching for after_fork hooks
Fix autoload of Etc in Worker for Ruby 1.9
bin/unicorn: allow RACK_ENV to be passed from parent
tests for RACK_ENV preservation
http: allow headers/trailers to be written byte-wise
http: extra test for bytewise chunked bodies
tee_input: do not clobber trailer buffer on partial uploads
test_exec: ensure master is killed after test
Util::tmpio returns a TmpIO that responds to #size
TODO: remove user-switching bit, done
unicorn 0.94.0
Wayne Larsen (1):
bin/unicorn: set ENV["RACK_ENV"] on startup
[1] - Unicorn does not support CPU affinity directly, but it is
possible to load code that allows it inside after_fork hooks,
or even just call sched_tool(8).
--
Eric Wong
^ permalink raw reply [relevance 5%]
* unicorn.git news for 2009.11.01
@ 2009-11-01 20:57 7% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2009-11-01 20:57 UTC (permalink / raw)
To: mongrel-unicorn
I've pushed out a handful of portability fixes to allow all the tests to
pass under FreeBSD 7.2 (1.8.7-p174 and 1.9.1-p243). Hopefully these
fixes extend to other *BSDs out there, too. Please let us know if there
are any other issues I've missed and we can work on them together.
All of the issues were bugs with the test code rather than Unicorn
itself, so folks running 0.93.5 in production on FreeBSD 7.2 should be
fine already.
There are also some small cleanups and one new feature.
Eric Wong (9):
cleanup: avoid redundant error checks for fstat
test_helper: connect(2) may fail with EINVAL
GNUmakefile: fix non-portable tar(1) usage
tests: provide a pure Ruby setsid(8) equivalent
more portable symlink awareness for START_CTX[:cwd]
test_signals: avoid portability issues with fchmod(2)
cleanup error handling and make it less noisy
Do not override Dir.chdir in config files
configurator: add "working_directory" directive
* git://git.bogomips.org/unicorn.git
* http://git.bogomips.org/cgit/unicorn.git
* http://unicorn.bogomips.org/HACKING.html
--
Eric Wong
^ permalink raw reply [relevance 7%]
* draft release notes (so far) for upcoming 0.93.0
@ 2009-10-01 8:13 6% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2009-10-01 8:13 UTC (permalink / raw)
To: mongrel-unicorn
The one minor bugfix is only for users who set RAILS_RELATIVE_URL_ROOT
in a config file. Users of the "--path" switch or those who set the
environment variable in the shell were unaffected by this bug. Note
that we still don't have relative URL root support for Rails < 2.3, and
are unlikely to bother with it unless there is visible demand for it. I
didn't even know people used/cared for relative URL roots before today
(it was a Rails 2.3.4 user).
New features (so far) includes support for :tries and :delay when
specifying a "listen" in an after_fork hook. This was inspired by Chris
Wanstrath's example of binding per-worker listen sockets in a loop while
migrating (or upgrading) Unicorn. Setting a negative value for :tries
means we'll retry the listen indefinitely until the socket becomes
available.
So you can do something like this in an after_fork hook:
after_fork do |server,worker|
addr = "127.0.0.1:#{9293 + worker.nr}"
server.listen(addr, :tries => -1, :delay => 5)
end
There's also the usual round of added documentation, packaging fixes,
code cleanups and minor performance improvements that are viewable
in the git log....
Shortlog since v0.92.0 (so far) below:
Eric Wong (45):
build: hardcode the canonical git URL
build: manifest dropped manpages
build: smaller ChangeLog
doc/LATEST: remove trailing newline
http: don't force -fPIC if it can't be used
.gitignore on *.rbc files Rubinius generates
README/gemspec: a better description, hopefully
GNUmakefile: add missing .manifest dep on test installs
Add HACKING document
configurator: fix user switch example in RDoc
local.mk.sample: time and perms enforcement
unicorn_rails: show "RAILS_ENV" in help message
gemspec: compatibility with older Rubygems
Split out KNOWN_ISSUES document
KNOWN_ISSUES: add notes about the "isolate" gem
gemspec: fix test_files regexp match
gemspec: remove tests that fork from test_files
test_signals: ensure we can parse pids in response
GNUmakefile: cleanup test/manifest generation
util: remove APPEND_FLAGS constant
http_request: simplify and remove handle_body method
http_response: simplify and remove const dependencies
local.mk.sample: fix .js times
TUNING: notes about benchmarking a high :backlog
HttpServer#listen accepts :tries and :delay parameters
"make install" avoids installing multiple .so objects
Use Configurator#expand_addr in HttpServer#listen
configurator: move initialization stuff to #initialize
Remove "Z" constant for binary strings
cgi_wrapper: don't warn about stdoutput usage
cgi_wrapper: simplify status handling in response
cgi_wrapper: use Array#concat instead of +=
server: correctly unset reexec_pid on child death
configurator: update and modernize examples
configurator: add colons in front of listen() options
configurator: remove DEFAULT_LOGGER constant
gemspec: clarify commented-out licenses section
Add makefile targets for non-release installs
cleanup: use question mark op for 1-byte comparisons
RDoc for Unicorn::HttpServer::Worker
small cleanup to pid file handling + documentation
rails: RAILS_RELATIVE_URL_ROOT may be set in Unicorn config
unicorn_rails: undeprecate --path switch
manpages: document environment variables
README: remove reference to different versions
--
Eric Wong
^ permalink raw reply [relevance 6%]
* [ANN] unicorn 0.92.0
@ 2009-09-18 22:16 5% Eric Wong
0 siblings, 0 replies; 43+ results
From: Eric Wong @ 2009-09-18 22:16 UTC (permalink / raw)
To: mongrel-unicorn
Unicorn is a Rack HTTP server for Unix and fast clients
Small fixes and documentation are the focus of this release.
James Golick reported and helped me track down a bug that caused
SIGHUP to drop the default listener (0.0.0.0:8080) if and only
if listeners were completely unspecified in both the
command-line and Unicorn config file. The Unicorn config file
remains the recommended option for specifying listeners as it
allows fine-tuning of the :backlog, :rcvbuf, :sndbuf,
:tcp_nopush, and :tcp_nodelay options.
There are some documentation (and resulting website)
improvements. setup.rb users will notice the new section 1
manpages for `unicorn` and `unicorn_rails`, Rubygems users
will have to install manpages manually or use the website.
Edit: That's not entirely true, I screwed up the package but
you can get them from http://unicorn.bogomips.org/unicorn.1
and http://unicorn.bogomips.org/unicorn_rails.1
The HTTP parser got a 3rd-party code review which resulted in
some cleanups and one insignificant bugfix as a result.
Additionally, the HTTP parser compiles, runs and passes unit
tests under Rubinius. The pure-Ruby parts still do not work yet
and we currently lack the resources/interest to pursue this
further but help will be gladly accepted.
The website now has an Atom feed for new release announcements.
Those unfamiliar with Atom or HTTP may finger unicorn@bogomips.org
for the latest announcements.
Eric Wong (53):
README: update with current version
http: cleanup and avoid potential signedness warning
http: clarify the setting of the actual header in the hash
http: switch to macros for bitflag handling
http: refactor keepalive tracking to functions
http: use explicit elses for readability
http: remove needless goto
http: extra assertion when advancing p manually
http: verbose assertions
http: NIL_P(var) instead of var == Qnil
http: rb_gc_mark already ignores immediates
http: ignore Host: continuation lines with absolute URIs
doc/SIGNALS: fix the no-longer-true bit about socket options
"encoding: binary" comments for all sources (1.9)
http_response: don't "rescue nil" for body.close
CONTRIBUTORS: fix capitalization for why
http: support Rubies without the OBJ_FROZEN macro
http: define OFFT2NUM macro on Rubies without it
http: no-op rb_str_modify() for Rubies without it
http: compile with -fPIC
http: use rb_str_{update,flush} if available
http: create a new string buffer on empty values
Update documentation for Rubinius support status
http: cleanup assertion for memoized header strings
http: add #endif comment labels where appropriate
Add .mailmap file for "git shortlog" and other tools
Update Manifest with mailmap
Fix comment about speculative accept()
SIGNALS: use "Unicorn" when referring to the web server
Add new Documentation section for manpages
test_exec: add extra tests for HUP and preload_app
socket_helper: (FreeBSD) don't freeze the accept filter constant
Avoid freezing objects that don't benefit from it
SIGHUP no longer drops lone, default listener
doc: generate ChangeLog and NEWS file for RDoc
Remove Echoe and roll our own packaging/release...
unicorn_rails: close parentheses in help message
launchers: deprecate ambiguous -P/--p* switches
man1/unicorn: avoid unnecessary emphasis
Add unicorn_rails(1) manpage
Documentation: don't force --rsyncable flag with gzip(1)
Simplify and standardize manpages build/install
GNUmakefile: package .tgz includes all generated files
doc: begin integration of HTML manpages into RDoc
Update TODO
html: add Atom feeds
doc: latest news is available through finger
NEWS.atom: file timestamp matches latest entry
pandoc needs the standalone switch for manpages
man1/unicorn: split out RACK ENVIRONMENT section
man1/unicorn_rails: fix unescaped underscore
NEWS.atom.xml only lists the first 10 entries
unicorn 0.92.0
* site: http://unicorn.bogomips.org/
* git: git://git.bogomips.org/unicorn.git
* cgit: http://git.bogomips.org/cgit/unicorn.git/
* list: mongrel-unicorn@rubyforge.org
* nntp: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
* finger: unicorn@bogomips.org
--
Eric Wong
^ permalink raw reply [relevance 5%]
Results 1-43 of 43 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2009-09-18 22:16 5% [ANN] unicorn 0.92.0 Eric Wong
2009-10-01 8:13 6% draft release notes (so far) for upcoming 0.93.0 Eric Wong
2009-11-01 20:57 7% unicorn.git news for 2009.11.01 Eric Wong
2009-11-05 10:06 5% [ANN] unicorn 0.94.0 - small fixes and new features Eric Wong
2009-11-17 17:50 7% unicorn, rails, and symlinked deployment Michael Guterl
2009-11-17 22:08 0% ` Michael Guterl
2009-11-17 22:20 0% ` Eric Wong
2009-11-18 15:34 0% ` Michael Guterl
2010-06-04 1:58 4% unicorn_rails cleanup (possible fix for Rails3) pushed Eric Wong
2010-06-04 2:13 7% ` Michael Guterl
2010-06-04 2:48 7% ` Eric Wong
2010-06-08 2:49 7% ` Eric Wong
2010-06-08 12:24 7% Hongli Lai
2010-06-08 19:20 4% ` Eric Wong
2010-06-08 19:25 6% ` Hongli Lai
2010-06-08 20:55 6% ` Eric Wong
2010-06-11 20:32 6% ` Eric Wong
2010-06-12 9:30 Issues since 0.991.0 Jose Avila(Tachu)
2010-06-12 18:37 5% ` Eric Wong
2010-06-16 0:09 7% anything left before 1.0? Eric Wong
2010-07-13 20:19 [ANN] unicorn 1.0.1 and 1.1.2 - fix rollbacks Eric Wong
2010-07-14 0:13 ` Lawrence Pit
2010-07-14 0:38 ` Eric Wong
2010-07-14 2:14 ` Lawrence Pit
2010-07-14 2:34 ` Eric Wong
2010-07-14 7:00 ` Lawrence Pit
2010-07-15 18:07 5% ` Jamie Wilkinson
2010-07-16 0:47 0% ` Lawrence Pit
2011-01-13 21:20 Unicorn 3.3.1 "Too many open files" error in kgio_tryaccept ghazel
2011-01-13 23:06 5% ` Eric Wong
2011-01-14 2:17 ` ghazel
2011-01-14 2:47 ` Eric Wong
2011-01-14 4:20 5% ` ghazel
2011-01-14 9:17 0% ` Eric Wong
2011-11-14 19:33 Fix hang when running tests on OpenBSD by skipping two tests Jeremy Evans
2011-11-14 20:54 ` Eric Wong
2011-11-14 23:46 ` Jeremy Evans
2011-11-15 3:17 ` Eric Wong
2011-11-15 20:03 ` Jeremy Evans
2011-11-16 1:18 5% ` Eric Wong
2012-01-23 9:20 4% release unicorn v4.2.0 soon? Eric Wong
2012-10-30 20:40 Combating nginx 499 HTTP responses during flash traffic scenario Tom Burns
2012-10-30 21:37 ` Eric Wong
2012-11-02 17:59 ` Tom Burns
2012-11-02 19:38 ` Eric Wong
2012-11-03 22:45 ` Tom Burns
2012-11-05 11:48 ` Eric Wong
2012-11-06 3:16 ` Tom Burns
2012-11-06 21:23 ` Eric Wong
2012-11-29 15:52 ` Tom Burns
2012-11-29 20:41 ` Eric Wong
2012-12-04 3:00 3% ` Eric Wong
2013-01-22 11:49 3% [PATCH] support for Rack hijack in request and response Eric Wong
2014-05-25 3:52 5% unicorn 5 roadmap Eric Wong
2014-10-24 17:33 Having issue with Unicorn Imdad
2014-10-24 17:45 ` Eric Wong
2014-10-24 18:02 3% ` Imdad
2015-04-22 16:42 TeeInput leaks file handles/space Mulvaney, Mike
2015-04-22 18:38 ` Eric Wong
2015-04-22 19:10 ` Mulvaney, Mike
2015-04-22 19:16 ` Eric Wong
2015-04-22 19:24 4% ` Mulvaney, Mike
2015-04-24 3:08 0% ` Eric Wong
2015-04-24 11:56 0% ` Mulvaney, Mike
2015-04-24 3:02 [PATCH 0/2] Rack::TempfileReaper support Eric Wong
2015-04-24 3:02 4% ` [PATCH 1/2] tee_input: support for Rack::TempfileReaper middleware Eric Wong
2015-04-24 3:02 5% ` [PATCH 2/2] support TempfileReaper in deployment and development envs Eric Wong
2015-10-27 3:33 5% [PATCH 0/2] socket inheritance behavior/bug => feature! Eric Wong
2015-10-27 3:33 5% ` [PATCH 1/2] sd_listen_fds emulation cleanup Eric Wong
2016-11-01 17:30 Support for HTTP/1.0 Joe McIlvain
2016-11-01 18:11 ` Eric Wong
2016-11-11 7:07 5% ` Eric Wong
2017-02-13 18:27 5% [PATCH] http_request: freeze constant strings passed IO#write Eric Wong
2017-12-22 3:17 4% [PATCH] tests: cleanup some unused variable warnings Eric Wong
2017-12-23 23:42 4% [ANN] unicorn 5.4.0 - Rack HTTP server for fast clients and Unix Eric Wong
2023-02-15 7:58 14% [PATCH] epollexclusive: avoid Ruby object allocation for buffer Eric Wong
2023-03-01 23:12 0% ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).