From 2023d725a57b400d8f19a6aad2e7d119ee27d7f3 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 30 Aug 2010 23:25:59 -0700 Subject: TUNING: more on socket buffer sizes Large buffers can hurt as well as help. And the difference in real apps that do a lot of things other than I/O often makes it not worth it. (cherry picked from commit f9a7a19a361fd674bab4e2df7e0897015528bba7) --- TUNING | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/TUNING b/TUNING index d96529a..9a54a01 100644 --- a/TUNING +++ b/TUNING @@ -22,7 +22,13 @@ See Unicorn::Configurator for details on the config file format. listeners under Linux 2.6 because auto-tuning is enabled. UNIX domain sockets do not have auto-tuning buffer sizes; so increasing those will allow syscalls and task switches to be saved for larger requests - and responses. + and responses. If your app only generates small responses or expects + small requests, you may shrink the buffer sizes to save memory, too. + +* Having socket buffers too large can also be detrimental or have + little effect. Huge buffers can put more pressure on the allocator + and may also thrash CPU caches, cancelling out performance gains + one would normally expect. * Setting "preload_app true" can allow copy-on-write-friendly GC to be used to save memory. It will probably not work out of the box with -- cgit v1.2.3-24-ge0c7 From 0bf83bb4246aefe172e2c5adcb8b28bda9dae283 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 15 Sep 2010 14:42:54 -0700 Subject: doc: update Sandbox document for Bundler Thanks to Lawrence Pit, Jamie Wilkinson, and Eirik Dentz Sinclair. ref: mid.gmane.org/4C8986DA.7090603@gmail.com ref: mid.gmane.org/5F1A02DB-CBDA-4302-9E26-8050C2D72433@efficiency20.com (cherry picked from commit 1a75966a5d1a1f6307ed3386e2f91a28bbb72ca0) --- Sandbox | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/Sandbox b/Sandbox index d2f7590..d101106 100644 --- a/Sandbox +++ b/Sandbox @@ -24,6 +24,9 @@ this: Then use HUP to reload, and then continue with the USR2+QUIT upgrade sequence. +Environment variable pollution when exec-ing a new process (with USR2) +is the primary issue with sandboxing tools such as Bundler and Isolate. + == Bundler === Running @@ -42,6 +45,12 @@ This is no longer be an issue as of bundler 0.9.17 ref: http://mid.gmane.org/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com +=== Other ENV pollution issues + +You may need to set or reset BUNDLE_GEMFILE, GEM_HOME, GEM_PATH and PATH +environment variables in the before_exec hook as illustrated by +http://gist.github.com/534668 + == Isolate === Running -- cgit v1.2.3-24-ge0c7 From 206d8c77bc0ac15a5c4e7a56aeff2e9fc280e6a3 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 15 Sep 2010 14:57:27 -0700 Subject: doc: update HACKING for documentation contributions We switched to RDoc 2.5.x long ago and this should clarify some documentation preferences I have. (cherry picked from commit 505a9e72d320fe3ae521ceb0f381c1c0f5ae4389) --- HACKING | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/HACKING b/HACKING index 6a05d4b..781c4ca 100644 --- a/HACKING +++ b/HACKING @@ -16,8 +16,8 @@ Tests are good, but slow tests make development slow, so we make tests faster (in parallel) with GNU make (instead of Rake) and avoiding RubyGems. -Users of GNU-based systems (such as GNU/Linux) usually have GNU make installed -as "make" instead of "gmake". +Users of GNU-based systems (such as GNU/Linux) usually have GNU make +installed as "make" instead of "gmake". Since we don't load RubyGems by default, loading Rack properly requires setting up RUBYLIB to point to where Rack is located. Not loading @@ -57,11 +57,18 @@ programming experience will come in handy (or be learned) here. === Documentation -We use RDoc 2.4.x with Darkfish for documentation as much as possible, +We use RDoc 2.5.x with Darkfish for documentation as much as possible, if you're on Ruby 1.8 you want to install the latest "rdoc" gem. Due to the lack of RDoc-to-manpage converters we know about, we're writing manpages in Markdown and converting to troff/HTML with Pandoc. +Please wrap documentation at 72 characters-per-line or less (long URLs +are exempt) so it is comfortably readable from terminals. + +When referencing mailing list posts, use +"http://mid.gmane.org/$MESSAGE_ID" if possible since the Message-ID +remains searchable even if Gmane becomes unavailable. + === Ruby/C Compatibility We target Ruby 1.8.6+, 1.9 and will target Rubinius as it becomes -- cgit v1.2.3-24-ge0c7 From 8612ccdd2f7f0c0b182ecc9616a76e97b879f6c0 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 4 Oct 2010 04:17:31 +0000 Subject: avoid unlinking actively listening sockets While we've always unlinked dead sockets from nuked/leftover processes, blindly unlinking them can cause unnecessary failures when an active process is already listening on them. We now make a simple connect(2) check to ensure the socket is not in use before unlinking it. Thanks to Jordan Ritter for the detailed bug report leading to this fix. ref: http://mid.gmane.org/8D95A44B-A098-43BE-B532-7D74BD957F31@darkridge.com (cherry picked from commit 1a2363b17b1d06be6b35d347ebcaed6a0c940200) --- lib/unicorn/socket_helper.rb | 10 ++++-- t/t0011-active-unix-socket.sh | 79 +++++++++++++++++++++++++++++++++++++++++ test/unit/test_socket_helper.rb | 9 ++++- 3 files changed, 95 insertions(+), 3 deletions(-) create mode 100644 t/t0011-active-unix-socket.sh diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb index 9a155e1..1d03eab 100644 --- a/lib/unicorn/socket_helper.rb +++ b/lib/unicorn/socket_helper.rb @@ -111,8 +111,14 @@ module Unicorn sock = if address[0] == ?/ if File.exist?(address) if File.socket?(address) - logger.info "unlinking existing socket=#{address}" - File.unlink(address) + begin + UNIXSocket.new(address).close + # fall through, try to bind(2) and fail with EADDRINUSE + # (or succeed from a small race condition we can't sanely avoid). + rescue Errno::ECONNREFUSED + logger.info "unlinking existing socket=#{address}" + File.unlink(address) + end else raise ArgumentError, "socket=#{address} specified but it is not a socket!" diff --git a/t/t0011-active-unix-socket.sh b/t/t0011-active-unix-socket.sh new file mode 100644 index 0000000..6f9ac53 --- /dev/null +++ b/t/t0011-active-unix-socket.sh @@ -0,0 +1,79 @@ +#!/bin/sh +. ./test-lib.sh +t_plan 11 "existing UNIX domain socket check" + +read_pid_unix () { + x=$(printf 'GET / HTTP/1.0\r\n\r\n' | \ + socat - UNIX:$unix_socket | \ + tail -1) + test -n "$x" + y="$(expr "$x" : '\([0-9]\+\)')" + test x"$x" = x"$y" + test -n "$y" + echo "$y" +} + +t_begin "setup and start" && { + rtmpfiles unix_socket unix_config + rm -f $unix_socket + unicorn_setup + grep -v ^listen < $unicorn_config > $unix_config + echo "listen '$unix_socket'" >> $unix_config + unicorn -D -c $unix_config pid.ru + unicorn_wait_start + orig_master_pid=$unicorn_pid +} + +t_begin "get pid of worker" && { + worker_pid=$(read_pid_unix) + t_info "worker_pid=$worker_pid" +} + +t_begin "fails to start with existing pid file" && { + rm -f $ok + unicorn -D -c $unix_config pid.ru || echo ok > $ok + test x"$(cat $ok)" = xok +} + +t_begin "worker pid unchanged" && { + test x"$(read_pid_unix)" = x$worker_pid + > $r_err +} + +t_begin "fails to start with listening UNIX domain socket bound" && { + rm $ok $pid + unicorn -D -c $unix_config pid.ru || echo ok > $ok + test x"$(cat $ok)" = xok + > $r_err +} + +t_begin "worker pid unchanged (again)" && { + test x"$(read_pid_unix)" = x$worker_pid +} + +t_begin "nuking the existing Unicorn succeeds" && { + kill -9 $unicorn_pid $worker_pid + while kill -0 $unicorn_pid + do + sleep 1 + done + check_stderr +} + +t_begin "succeeds in starting with leftover UNIX domain socket bound" && { + test -S $unix_socket + unicorn -D -c $unix_config pid.ru + unicorn_wait_start +} + +t_begin "worker pid changed" && { + test x"$(read_pid_unix)" != x$worker_pid +} + +t_begin "killing succeeds" && { + kill $unicorn_pid +} + +t_begin "no errors" && check_stderr + +t_done diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb index bbce359..c6d0d42 100644 --- a/test/unit/test_socket_helper.rb +++ b/test/unit/test_socket_helper.rb @@ -101,7 +101,14 @@ class TestSocketHelper < Test::Unit::TestCase def test_bind_listen_unix_rebind test_bind_listen_unix - new_listener = bind_listen(@unix_listener_path) + new_listener = nil + assert_raises(Errno::EADDRINUSE) do + new_listener = bind_listen(@unix_listener_path) + end + assert_nothing_raised do + File.unlink(@unix_listener_path) + new_listener = bind_listen(@unix_listener_path) + end assert UNIXServer === new_listener assert new_listener.fileno != @unix_listener.fileno assert_equal sock_name(new_listener), sock_name(@unix_listener) -- cgit v1.2.3-24-ge0c7 From 28fc5a6b069a544389603e817f608a3d09eb0440 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 30 Aug 2010 08:11:44 +0000 Subject: update Rails 3 tests to use Rails 3 final Rails 3 is out, and requires no code changes on our end to work (as far as our tests show :) (cherry picked from commit da272fc48ffaa808456fe94dd7a3e01bc9799832) --- Rakefile | 2 +- t/rails3-app/Gemfile | 2 +- t/test-rails3.sh | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Rakefile b/Rakefile index 833288c..c140b89 100644 --- a/Rakefile +++ b/Rakefile @@ -207,7 +207,7 @@ task :isolate do status.success? or abort status.inspect # pure Ruby gems can be shared across all Rubies - %w(3.0.0.rc2).each do |rails_ver| + %w(3.0.0).each do |rails_ver| opts[:path] = "tmp/isolate/rails-#{rails_ver}" pid = fork { Isolate.now!(opts) { gem 'rails', rails_ver } } _, status = Process.waitpid2(pid) diff --git a/t/rails3-app/Gemfile b/t/rails3-app/Gemfile index 9fbe5ee..d996f34 100644 --- a/t/rails3-app/Gemfile +++ b/t/rails3-app/Gemfile @@ -1,6 +1,6 @@ source 'http://rubygems.org' -# gem 'rails', '3.0.0.rc2' +# gem 'rails', '3.0.0' # Bundle edge Rails instead: # gem 'rails', :git => 'git://github.com/rails/rails.git' diff --git a/t/test-rails3.sh b/t/test-rails3.sh index 5661e29..b398f03 100644 --- a/t/test-rails3.sh +++ b/t/test-rails3.sh @@ -1,5 +1,5 @@ . ./test-lib.sh -RAILS_VERSION=${RAILS_VERSION-3.0.0.rc2} +RAILS_VERSION=${RAILS_VERSION-3.0.0} case $RUBY_VERSION in 1.8.7|1.9.2) ;; *) -- cgit v1.2.3-24-ge0c7 From 61516865943f1b18370357cd555811807339c1f7 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 4 Oct 2010 20:27:28 +0000 Subject: unicorn 1.1.4 - small bug fix and doc updates We no longer unlinking actively listening sockets upon startup (but continue to unlink dead ones). This bug could trigger downtime and nginx failures if a user makes an error and attempts to start Unicorn while it is already running. Thanks to Jordan Ritter for the detailed bug report leading to this fix. ref: http://mid.gmane.org/8D95A44B-A098-43BE-B532-7D74BD957F31@darkridge.com There are also minor documentation and test updates pulled in from master. This is hopefully the last bugfix release of the 1.1.x series. --- GIT-VERSION-GEN | 2 +- lib/unicorn/const.rb | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/GIT-VERSION-GEN b/GIT-VERSION-GEN index 40adf6d..755e132 100755 --- a/GIT-VERSION-GEN +++ b/GIT-VERSION-GEN @@ -1,7 +1,7 @@ #!/bin/sh GVF=GIT-VERSION-FILE -DEF_VER=v1.1.3.GIT +DEF_VER=v1.1.4.GIT LF=' ' diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb index 5743678..a166780 100644 --- a/lib/unicorn/const.rb +++ b/lib/unicorn/const.rb @@ -8,8 +8,8 @@ module Unicorn # Symbols did not really improve things much compared to constants. module Const - # The current version of Unicorn, currently 1.1.3 - UNICORN_VERSION="1.1.3" + # The current version of Unicorn, currently 1.1.4 + UNICORN_VERSION="1.1.4" DEFAULT_HOST = "0.0.0.0" # default TCP listen host address DEFAULT_PORT = 8080 # default TCP listen port -- cgit v1.2.3-24-ge0c7