diff options
124 files changed, 6008 insertions, 2607 deletions
@@ -1,11 +1,13 @@ README +TUNING +PHILOSOPHY DESIGN -CHANGELOG CONTRIBUTORS LICENSE SIGNALS TODO -bin +bin/unicorn +bin/unicorn_rails lib ext/**/*.c ext/**/*.rl @@ -10,3 +10,4 @@ ext/unicorn/http11/Makefile log/ pkg/ +/vendor @@ -1,19 +1,14 @@ +v0.7.0 - rack.version is 1.0 +v0.6.0 - cleanups + optimizations, signals to {in,de}crement processes +v0.5.4 - fix data corruption with some small uploads (not curl) +v0.5.3 - fix 100% CPU usage when idle, small cleanups +v0.5.2 - force Status: header for compat, small cleanups +v0.5.1 - exit correctly on INT/TERM, QUIT is still recommended, however +v0.5.0 - {after,before}_fork API change, small tweaks/fixes +v0.4.2 - fix Rails ARStore, FD leak prevention, descriptive proctitles +v0.4.1 - Rails support, per-listener backlog and {snd,rcv}buf +v0.2.3 - Unlink Tempfiles after use (they were closed, just not unlinked) +v0.2.2 - small bug fixes, fix Rack multi-value headers (Set-Cookie:) +v0.2.1 - Fix broken Manifest that cause unicorn_rails to not be bundled +v0.2.0 - unicorn_rails launcher script. v0.1.0 - Unicorn - UNIX-only fork of Mongrel free of threading - -v2.0. (WIP) Rack support. - -v1.1.4. Fix camping handler. Correct treatment of @throttle parameter. - -v1.1.3. Fix security flaw of DirHandler; reported on mailing list. - -v1.1.2. Fix worker termination bug; fix JRuby 1.0.3 load order issue; fix require issue on systems without Rubygems. - -v1.1.1. Fix mongrel_rails restart bug; fix bug with Rack status codes. - -v1.1. Pure Ruby URIClassifier. More modular architecture. JRuby support. Move C URIClassifier into mongrel_experimental project. - -v1.0.4. Backport fixes for versioning inconsistency, mongrel_rails bug, and DirHandler bug. - -v1.0.3. Fix user-switching bug; make people upgrade to the latest from the RC. - -v1.0.2. Signed gem; many minor bugfixes and patches. diff --git a/CONTRIBUTORS b/CONTRIBUTORS index ac47dfb..5a6fa4d 100644 --- a/CONTRIBUTORS +++ b/CONTRIBUTORS @@ -1,20 +1,29 @@ -Unicorn would not be possible without Zed and all the contributors to Mongrel. +Unicorn developers: +* Eric Wong +* ... (help wanted) -Eric Wong -Ezra Zygmuntowicz -Zed A. Shaw -Luis Lavena -Wilson Bilkovich -Why the Lucky Stiff -Dan Kubb -MenTaLguY -Filipe Lautert -Rick Olson -Wayne E. Seguin -Kirk Haines -Bradley Taylor -Matt Pelletier -Ry Dahl -Nick Sieger -Evan Weaver -Marc-André Cournoyer +We would like to thank following folks for helping make Unicorn possible: + +* Ezra Zygmuntowicz - for helping Eric decide on a sane configuration + format and reasonable defaults. +* Christian Neukirchen - for Rack, which let us put more focus on the server + and drastically cut down on the amount of code we have to maintain. +* Zed A. Shaw - for Mongrel, without which Unicorn would not be possible + +The original Mongrel contributors: + +* Luis Lavena +* Wilson Bilkovich +* Why the Lucky Stiff +* Dan Kubb +* MenTaLguY +* Filipe Lautert +* Rick Olson +* Wayne E. Seguin +* Kirk Haines +* Bradley Taylor +* Matt Pelletier +* Ry Dahl +* Nick Sieger +* Evan Weaver +* Marc-André Cournoyer @@ -32,14 +32,16 @@ Rack application itself is called only within the worker process (but can be loaded within the master). A copy-on-write friendly garbage collector like Ruby Enterprise Edition can be used to minimize memory - usage along with the "preload_app true" directive. + usage along with the "preload_app true" directive (see + Unicorn::Configurator). * The number of worker processes should be scaled to the number of CPUs, memory or even spindles you have. If you have an existing - Mongrel cluster, using the same amount of processes should work. - Let a full-HTTP-request-buffering reverse proxy like nginx manage - concurrency to thousands of slow clients for you. Unicorn scaling - should only be concerned about limits of your backend system(s). + Mongrel cluster on a single-threaded app, using the same amount of + processes should work. Let a full-HTTP-request-buffering reverse + proxy like nginx manage concurrency to thousands of slow clients for + you. Unicorn scaling should only be concerned about limits of your + backend system(s). * Load balancing between worker processes is done by the OS kernel. All workers share a common set of listener sockets and does @@ -55,8 +57,8 @@ * Blocking I/O is used for clients. This allows a simpler code path to be followed within the Ruby interpreter and fewer syscalls. - Applications that use threads should continue to work if Unicorn - is serving LAN or localhost clients. + Applications that use threads continue to work if Unicorn + is only serving LAN or localhost clients. * Timeout implementation is done via fchmod(2) in each worker on a shared file descriptor to update st_ctime on the inode. @@ -66,8 +68,9 @@ pwrite(2)/pread(2) are supported by base MRI, nor are they as portable on UNIX systems as fchmod(2). -* SIGKILL is used to terminate the timed-out workers as reliably - as possible on a UNIX system. +* SIGKILL is used to terminate the timed-out workers from misbehaving apps + as reliably as possible on a UNIX system. The default timeout is a + generous 60 seconds (same default as in Mongrel). * The poor performance of select() on large FD sets is avoided as few file descriptors are used in each worker. @@ -78,3 +81,9 @@ * If the master process dies unexpectedly for any reason, workers will notice within :timeout/2 seconds and follow the master to its death. + +* There is never any explicit real-time dependency or communication + between the worker processes nor to the master process. + Synchronization is handled entirely by the OS kernel and shared + resources are never accessed by the worker when it is servicing + a client. diff --git a/GNUmakefile b/GNUmakefile index 63a5bd0..1145143 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -1,6 +1,8 @@ # use GNU Make to run tests in parallel, and without depending on Rubygems all:: test ruby = ruby +ragel = ragel +RLFLAGS = -G2 -include local.mk ruby_bin := $(shell which $(ruby)) ifeq ($(DLEXT),) # "so" for Linux @@ -13,58 +15,75 @@ endif # dunno how to implement this as concisely in Ruby, and hell, I love awk awk_slow := awk '/def test_/{print FILENAME"--"$$2".n"}' 2>/dev/null -slow_tests := test/unit/test_server.rb test/exec/test_exec.rb +rails_vers := $(subst test/rails/app-,,$(wildcard test/rails/app-*)) +slow_tests := test/unit/test_server.rb test/exec/test_exec.rb \ + test/unit/test_signals.rb test/unit/test_upload.rb log_suffix = .$(RUBY_VERSION).log -T := $(filter-out $(slow_tests),$(wildcard test/*/test*.rb)) +T_r := $(wildcard test/rails/test*.rb) +T := $(filter-out $(slow_tests) $(T_r), $(wildcard test/*/test*.rb)) T_n := $(shell $(awk_slow) $(slow_tests)) T_log := $(subst .rb,$(log_suffix),$(T)) T_n_log := $(subst .n,$(log_suffix),$(T_n)) +T_r_log := $(subst .r,$(log_suffix),$(T_r)) test_prefix = $(CURDIR)/test/install-$(RUBY_VERSION) -http11_deps := $(addprefix ext/unicorn/http11/, \ - ext_help.h http11.c http11_parser.c http11_parser.h \ - http11_parser.rl http11_parser_common.rl) -inst_deps := $(wildcard bin/*) $(wildcard lib/*.rb) \ - $(wildcard lib/*/*.rb) $(http11_deps) - -ext/unicorn/http11/http11_parser.c: ext/unicorn/http11/http11_parser.rl - cd $(@D) && ragel $(<F) -C -G2 -o $(@F) -ext/unicorn/http11/Makefile: ext/unicorn/http11/extconf.rb - cd $(@D) && $(ruby) $(<F) -ext/unicorn/http11/http11.$(DLEXT): $(http11_deps) ext/unicorn/http11/Makefile +ext := ext/unicorn/http11 +c_files := $(addprefix $(ext)/,ext_help.h http11.c http11_parser.h) +rl_files := $(addprefix $(ext)/,http11_parser.rl http11_parser_common.rl) +rb_files := $(shell grep '^\(bin\|lib\)' Manifest) +inst_deps := $(c_files) $(rb_files) + +ragel: $(ext)/http11_parser.h +$(ext)/http11_parser.h: $(rl_files) + cd $(@D) && $(ragel) http11_parser.rl -C $(RLFLAGS) -o $(@F) + $(ruby) -i -p -e '$$_.gsub!(%r{[ \t]*$$},"")' $@ +$(ext)/Makefile: $(ext)/extconf.rb $(c_files) + cd $(@D) && $(ruby) extconf.rb +$(ext)/http11.$(DLEXT): $(ext)/Makefile $(MAKE) -C $(@D) -lib/unicorn/http11.$(DLEXT): ext/unicorn/http11/http11.$(DLEXT) +lib/unicorn/http11.$(DLEXT): $(ext)/http11.$(DLEXT) @mkdir -p lib install -m644 $< $@ http11: lib/unicorn/http11.$(DLEXT) $(test_prefix)/.stamp: $(inst_deps) - $(MAKE) clean-http11 - $(MAKE) install-test - > $@ - -install-test: mkdir -p $(test_prefix)/.ccache - tar c bin ext lib GNUmakefile | (cd $(test_prefix) && tar x) + tar c bin ext lib GNUmakefile Manifest | (cd $(test_prefix) && tar x) + $(MAKE) -C $(test_prefix) clean $(MAKE) -C $(test_prefix) http11 shebang + > $@ + +bins := $(wildcard bin/*) # this is only intended to be run within $(test_prefix) -shebang: bin/unicorn - $(ruby) -i -p -e '$$_.gsub!(%r{^#!.*$$},"#!$(ruby_bin)")' $< +shebang: $(bins) + $(ruby) -i -p -e '$$_.gsub!(%r{^#!.*$$},"#!$(ruby_bin)")' $^ t_log := $(T_log) $(T_n_log) test: $(T) $(T_n) @cat $(t_log) | $(ruby) test/aggregate.rb @$(RM) $(t_log) -slow-tests: $(slow_tests) -$(slow_tests): +test-exec: $(wildcard test/exec/test_*.rb) +test-unit: $(wildcard test/unit/test_*.rb) +$(slow_tests): $(test_prefix)/.stamp @$(MAKE) $(shell $(awk_slow) $@) TEST_OPTS = -v -run_test = @echo '*** $(arg) ***'; \ - setsid $(ruby) $(arg) $(TEST_OPTS) >$(t) 2>&1 || \ - (cat >&2 < $(t); exit 1) +TEST_OPTS = -v +ifndef V + quiet_pre = @echo '* $(arg)$(extra)'; + quiet_post = >$(t) 2>&1 +else + # we can't rely on -o pipefail outside of bash 3+, + # so we use a stamp file to indicate success and + # have rm fail if the stamp didn't get created + stamp = $@$(log_suffix).ok + quiet_pre = @echo $(ruby) $(arg) $(TEST_OPTS); ! test -f $(stamp) && ( + quiet_post = && > $(stamp) )>&2 | tee $(t); rm $(stamp) 2>/dev/null +endif +run_test = $(quiet_pre) setsid $(ruby) -w $(arg) $(TEST_OPTS) $(quiet_post) || \ + (sed "s,^,$(extra): ," >&2 < $(t); exit 1) %.n: arg = $(subst .n,,$(subst --, -n ,$@)) %.n: t = $(subst .n,$(log_suffix),$@) @@ -80,23 +99,24 @@ $(T): export RUBYLIB := $(test_prefix)/lib:$(RUBYLIB) $(T): $(test_prefix)/.stamp $(run_test) -install: bin/unicorn +install: $(bins) $(prep_setup_rb) - git diff --quiet $< + $(RM) -r .install-tmp + mkdir .install-tmp + cp -p $^ .install-tmp $(ruby) setup.rb all - git checkout $< + $(RM) $^ + mv $(addprefix .install-tmp/,$(^F)) bin/ + $(RM) -r .install-tmp $(prep_setup_rb) -clean-http11: - -$(MAKE) -C ext/unicorn/http11 clean - $(RM) ext/unicorn/http11/Makefile lib/unicorn/http11.$(DLEXT) - setup_rb_files := .config InstalledFiles -prep_setup_rb := @-$(RM) $(setup_rb_files);$(MAKE) -C ext/unicorn/http11 clean +prep_setup_rb := @-$(RM) $(setup_rb_files);$(MAKE) -C $(ext) clean -clean: clean-http11 - $(RM) $(setup_rb_files) - $(RM) $(t_log) +clean: + -$(MAKE) -C $(ext) clean + $(RM) $(ext)/Makefile lib/unicorn/http11.$(DLEXT) + $(RM) $(setup_rb_files) $(t_log) $(RM) -r $(test_prefix) Manifest: @@ -108,4 +128,23 @@ Manifest: doc: .document rdoc -Na -m README -t "$(shell sed -ne '1s/^= //p' README)" +rails_git_url = git://github.com/rails/rails.git +rails_git := vendor/rails.git +$(rails_git)/info/cloned-stamp: + git clone --mirror -q $(rails_git_url) $(rails_git) + > $@ + +rails_tests := $(addsuffix .r,$(addprefix $(T_r).,$(rails_vers))) +test-rails: $(rails_tests) +$(T_r).%.r: t = $(addsuffix $(log_suffix),$@) +$(T_r).%.r: rv = $(subst .r,,$(subst $(T_r).,,$@)) +$(T_r).%.r: extra = ' 'v$(rv) +$(T_r).%.r: arg = $(T_r) +$(T_r).%.r: export PATH := $(test_prefix)/bin:$(PATH) +$(T_r).%.r: export RUBYLIB := $(test_prefix)/lib:$(RUBYLIB) +$(T_r).%.r: export UNICORN_RAILS_TEST_VERSION = $(rv) +$(T_r).%.r: export RAILS_GIT_REPO = $(CURDIR)/$(rails_git) +$(T_r).%.r: $(test_prefix)/.stamp $(rails_git)/info/cloned-stamp + $(run_test) + .PHONY: doc $(T) $(slow_tests) Manifest @@ -6,36 +6,127 @@ DESIGN GNUmakefile LICENSE Manifest +PHILOSOPHY README Rakefile SIGNALS TODO +TUNING bin/unicorn +bin/unicorn_rails +examples/init.sh ext/unicorn/http11/ext_help.h ext/unicorn/http11/extconf.rb ext/unicorn/http11/http11.c -ext/unicorn/http11/http11_parser.c ext/unicorn/http11/http11_parser.h ext/unicorn/http11/http11_parser.rl ext/unicorn/http11/http11_parser_common.rl lib/unicorn.rb +lib/unicorn/app/exec_cgi.rb +lib/unicorn/app/old_rails.rb +lib/unicorn/app/old_rails/static.rb +lib/unicorn/cgi_wrapper.rb lib/unicorn/configurator.rb lib/unicorn/const.rb lib/unicorn/http_request.rb lib/unicorn/http_response.rb -lib/unicorn/socket.rb +lib/unicorn/launcher.rb +lib/unicorn/socket_helper.rb lib/unicorn/util.rb +local.mk.sample setup.rb test/aggregate.rb -test/benchmark/previous.rb -test/benchmark/simple.rb -test/benchmark/utils.rb +test/benchmark/README +test/benchmark/big_request.rb +test/benchmark/dd.ru +test/benchmark/request.rb +test/benchmark/response.rb test/exec/README test/exec/test_exec.rb +test/rails/app-1.2.3/.gitignore +test/rails/app-1.2.3/Rakefile +test/rails/app-1.2.3/app/controllers/application.rb +test/rails/app-1.2.3/app/controllers/foo_controller.rb +test/rails/app-1.2.3/app/helpers/application_helper.rb +test/rails/app-1.2.3/config/boot.rb +test/rails/app-1.2.3/config/database.yml +test/rails/app-1.2.3/config/environment.rb +test/rails/app-1.2.3/config/environments/development.rb +test/rails/app-1.2.3/config/environments/production.rb +test/rails/app-1.2.3/config/routes.rb +test/rails/app-1.2.3/db/.gitignore +test/rails/app-1.2.3/log/.gitignore +test/rails/app-1.2.3/public/404.html +test/rails/app-1.2.3/public/500.html +test/rails/app-2.0.2/.gitignore +test/rails/app-2.0.2/Rakefile +test/rails/app-2.0.2/app/controllers/application.rb +test/rails/app-2.0.2/app/controllers/foo_controller.rb +test/rails/app-2.0.2/app/helpers/application_helper.rb +test/rails/app-2.0.2/config/boot.rb +test/rails/app-2.0.2/config/database.yml +test/rails/app-2.0.2/config/environment.rb +test/rails/app-2.0.2/config/environments/development.rb +test/rails/app-2.0.2/config/environments/production.rb +test/rails/app-2.0.2/config/routes.rb +test/rails/app-2.0.2/db/.gitignore +test/rails/app-2.0.2/log/.gitignore +test/rails/app-2.0.2/public/404.html +test/rails/app-2.0.2/public/500.html +test/rails/app-2.1.2/.gitignore +test/rails/app-2.1.2/Rakefile +test/rails/app-2.1.2/app/controllers/application.rb +test/rails/app-2.1.2/app/controllers/foo_controller.rb +test/rails/app-2.1.2/app/helpers/application_helper.rb +test/rails/app-2.1.2/config/boot.rb +test/rails/app-2.1.2/config/database.yml +test/rails/app-2.1.2/config/environment.rb +test/rails/app-2.1.2/config/environments/development.rb +test/rails/app-2.1.2/config/environments/production.rb +test/rails/app-2.1.2/config/routes.rb +test/rails/app-2.1.2/db/.gitignore +test/rails/app-2.1.2/log/.gitignore +test/rails/app-2.1.2/public/404.html +test/rails/app-2.1.2/public/500.html +test/rails/app-2.2.2/.gitignore +test/rails/app-2.2.2/Rakefile +test/rails/app-2.2.2/app/controllers/application.rb +test/rails/app-2.2.2/app/controllers/foo_controller.rb +test/rails/app-2.2.2/app/helpers/application_helper.rb +test/rails/app-2.2.2/config/boot.rb +test/rails/app-2.2.2/config/database.yml +test/rails/app-2.2.2/config/environment.rb +test/rails/app-2.2.2/config/environments/development.rb +test/rails/app-2.2.2/config/environments/production.rb +test/rails/app-2.2.2/config/routes.rb +test/rails/app-2.2.2/db/.gitignore +test/rails/app-2.2.2/log/.gitignore +test/rails/app-2.2.2/public/404.html +test/rails/app-2.2.2/public/500.html +test/rails/app-2.3.2.1/.gitignore +test/rails/app-2.3.2.1/Rakefile +test/rails/app-2.3.2.1/app/controllers/application_controller.rb +test/rails/app-2.3.2.1/app/controllers/foo_controller.rb +test/rails/app-2.3.2.1/app/helpers/application_helper.rb +test/rails/app-2.3.2.1/config/boot.rb +test/rails/app-2.3.2.1/config/database.yml +test/rails/app-2.3.2.1/config/environment.rb +test/rails/app-2.3.2.1/config/environments/development.rb +test/rails/app-2.3.2.1/config/environments/production.rb +test/rails/app-2.3.2.1/config/routes.rb +test/rails/app-2.3.2.1/db/.gitignore +test/rails/app-2.3.2.1/log/.gitignore +test/rails/app-2.3.2.1/public/404.html +test/rails/app-2.3.2.1/public/500.html +test/rails/test_rails.rb test/test_helper.rb test/tools/trickletest.rb test/unit/test_configurator.rb test/unit/test_http_parser.rb +test/unit/test_request.rb test/unit/test_response.rb test/unit/test_server.rb +test/unit/test_signals.rb +test/unit/test_socket_helper.rb test/unit/test_upload.rb +test/unit/test_util.rb diff --git a/PHILOSOPHY b/PHILOSOPHY new file mode 100644 index 0000000..ce7763a --- /dev/null +++ b/PHILOSOPHY @@ -0,0 +1,139 @@ += The Philosophy Behind Unicorn + +Being a server that only runs on Unix-like platforms, Unicorn is +strongly tied to the Unix philosophy of doing one thing and (hopefully) +doing it well. Despite using HTTP, Unicorn is strictly a _backend_ +application server for running Rack-based Ruby applications. + +== Avoid Complexity + +Instead of attempting to be efficient at serving slow clients, Unicorn +relies on a buffering reverse proxy to efficiently deal with slow +clients. + +Unicorn uses an old-fashioned preforking worker model with blocking I/O. +Our processing model is the antithesis of more modern (and theoretically +more efficient) server processing models using threads or non-blocking +I/O with events. + +=== Threads and Events Are Hard + +...to many developers. Reasons for this is beyond the scope of this +document. Unicorn avoids concurrency within each worker process so you +have fewer things to worry about when developing your application. Of +course Unicorn can use multiple worker processes to utilize multiple +CPUs or spindles. Applications can still use threads internally, however. + +== Slow Clients Are Problematic + +Most benchmarks we've seen don't tell you this, and Unicorn doesn't +care about slow clients... but <i>you</i> should. + +A "slow client" can be any client outside of your datacenter. Network +traffic within a local network is always faster than traffic that +crosses outside of it. The laws of physics do not allow otherwise. + +Persistent connections were introduced in HTTP/1.1 reduce latency from +connection establishment and TCP slow start. They also waste server +resources when clients are idle. + +Persistent connections mean one of the Unicorn worker processes +(depending on your application, it can be very memory hungry) would +spend a significant amount of its time idle keeping the connection alive +<i>and not doing anything else</i>. Being single-threaded and using +blocking I/O, a worker cannot serve other clients while keeping a +connection alive. Thus Unicorn does not implement persistent +connections. + +If your application responses are larger than the socket buffer or if +you're handling large requests (uploads), worker processes will also be +bottlenecked by the speed of the *client* connection. You should +not allow Unicorn to serve clients outside of your local network. + +== Application Concurrency != Network Concurrency + +Performance is asymmetric across the different subsystems of the machine +and parts of the network. CPUs and main memory can process gigabytes of +data in a second; clients on the Internet are usually only capable of a +tiny fraction of that. Unicorn deployments should avoid dealing with +slow clients directly and instead rely on a reverse proxy to shield it +from the effects of slow I/O. + +== Improved Performance Through Reverse Proxying + +By acting as a buffer to shield Unicorn from slow I/O, a reverse proxy +will inevitably incur overhead in the form of extra data copies. +However, as I/O within a local network is fast (and faster still +with local sockets), this overhead is neglible for the vast majority +of HTTP requests and responses. + +The ideal reverse proxy complements the weaknesses of Unicorn. +A reverse proxy for Unicorn should meet the following requirements: + + 1. It should fully buffer all HTTP requests (and large responses). + Each request should be "corked" in the reverse proxy and sent + as fast as possible to the backend Unicorn processes. This is + the most important feature to look for when choosing a + reverse proxy for Unicorn. + + 2. It should spend minimal time in userspace. Network (and disk) I/O + are system-level tasks and usually managed by the kernel. + This may change if userspace TCP stacks become more popular in the + future; but the reverse proxy should not waste time with + application-level logic. These concerns should be separated + + 3. It should avoid context switches and CPU scheduling overhead. + In many (most?) cases, network devices and their interrupts are + only be handled by one CPU at a time. It should avoid contention + within the system by serializing all network I/O into one (or few) + userspace procceses. Network I/O is not a CPU-intensive task and + it is not helpful to use multiple CPU cores (at least not for GigE). + + 4. It should efficiently manage persistent connections (and + pipelining) to slow clients. If you care to serve slow clients + outside your network, then these features of HTTP/1.1 will help. + + 5. It should (optionally) serve static files. If you have static + files on your site (especially large ones), they are far more + efficiently served with as few data copies as possible (e.g. with + sendfile() to completely avoid copying the data to userspace). + +nginx is the only (Free) solution we know of that meets the above +requirements. + +Indeed, the author of Unicorn has deployed nginx as a reverse-proxy not +only for Ruby applications, but also for production applications running +Apache/mod_perl, Apache/mod_php and Apache Tomcat. In every single +case, performance improved because application servers were able to use +backend resources more efficiently and spend less time waiting on slow +I/O. + +== Worse Is Better + +Requirements and scope for applications change frequently and +drastically. Thus languages like Ruby and frameworks like Rails were +built to give developers fewer things to worry about in the face of +rapid change. + +On the other hand, stable protocols which host your applications (HTTP +and TCP) only change rarely. This is why we recommend you NOT tie your +rapidly-changing application logic directly into the processes that deal +with the stable outside world. Instead, use HTTP as a common RPC +protocol to communicate between your frontend and backend. + +In short: separate your concerns. + +Of course a theoretical "perfect" solution would combine the pieces +and _maybe_ give you better performance at the end of the day, but +that is not the Unix way. + +== Just Worse in Some Cases + +Unicorn is not suited for all applications. Unicorn is optimized for +applications that are CPU/memory/disk intensive and spend little time +waiting on external resources (e.g. a database server or external API). + +Unicorn is highly inefficient for Comet/reverse-HTTP/push applications +where the HTTP connection spends a large amount of time idle. +Nevertheless, the ease of troubleshooting, debugging, and management of +Unicorn may still outweigh the drawbacks for these applications. @@ -1,73 +1,148 @@ -= Unicorn: UNIX + LAN/localhost-only fork of Mongrel - -Only run this behind a full-HTTP-request-buffering reverse proxy if -you're serving slow clients. That said, nginx is the only reverse -proxy we know of that meets this requirement. += Unicorn: Unix + LAN/localhost-only fork of Mongrel == Features -* process management: Unicorn will reap and restart workers that - die because of broken apps and there is no need to manage - multiple processes yourself. +* Designed for Rack, Unix, fast clients, and ease-of-debugging. We + cut out all things that are better-supported by nginx or Rack. + +* Mostly written in Ruby, only the HTTP parser (stolen and trimmed + down from Mongrel) is written in C. Unicorn is compatible with + both Ruby 1.8 and 1.9. A pure-Ruby (but still Unix-only) version + is planned. + +* Process management: Unicorn will reap and restart workers that + die from broken apps. There is no need to manage multiple processes + or ports yourself. Unicorn can spawn and manage any fixed number of + worker processes you choose to scale to your backend. + +* Load balancing is done entirely by the operating system kernel. + Requests never pile up behind a busy worker process. -* does not care if your application is thread-safe or not, workers +* Does not care if your application is thread-safe or not, workers all run within their own isolated address space and only serve one - client at a time... + client at a time. -* able to listen on multiple interfaces, including UNIX sockets, - each worker process can also bind to a private port via the - after_fork hook for easy debugging. +* Supports all Rack applications, along with pre-Rack versions of + Ruby on Rails via a Rack wrapper. -* supports all Rack applications +* Builtin reopening of all log files in your application via + USR1 signal. This allows logrotate to rotate files atomically and + quickly via rename instead of the racy and slow copytruncate method. + Unicorn also takes steps to ensure multi-line log entries from one + request all stay within the same file. * nginx-style binary re-execution without losing connections. - You can upgrade unicorn, your entire application, libraries - and even your Ruby interpreter as long as unicorn is + You can upgrade Unicorn, your entire application, libraries + and even your Ruby interpreter as long as Unicorn is installed in the same path. * before_fork and after_fork hooks in case your application - has special needs when dealing with forked processes. + has special needs when dealing with forked processes. These + should not be needed when the "preload_app" directive is + false (the default). -* builtin log rotation via USR1 signal +* Can be used with copy-on-write-friendly memory management + to save memory (by setting "preload_app" to true). -* Ruby 1.9-compatible (at least the test cases all pass :>) +* Able to listen on multiple interfaces including UNIX sockets, + each worker process can also bind to a private port via the + after_fork hook for easy debugging. == License Unicorn is copyright 2009 Eric Wong and contributors. -It is based on Mongrel: +It is based on Mongrel and carries the same license: Mongrel is copyright 2007 Zed A. Shaw and contributors. It is licensed -under the Ruby license and the GPL2. See the include LICENSE file for +under the Ruby license and the GPL2. See the included LICENSE file for details. +Unicorn is 100% Free Software. + == Install The library consists of a C extension so you'll need a C compiler or at least a friend who can build it for you. -Finally, the source includes a setup.rb for those who hate RubyGems. +You may download the tarball from the Mongrel project page on Rubyforge +and run setup.rb after unpacking it: -You can get the source via git via the following locations: +http://rubyforge.org/frs/?group_id=1306 - git://git.bogomips.org/unicorn.git +You may also install it via Rubygems on Rubyforge: + + gem install unicorn + +You can get the latest source via git from the following locations +(these versions may not be stable): + git://git.bogomips.org/unicorn.git http://git.bogomips.org/unicorn.git + git://repo.or.cz/unicorn.git (mirror) + http://repo.or.cz/r/unicorn.git (mirror) + +If you have web browser software for the World Wide Web +(on the Information Superhighway), you may browse the code from +your web browser and download the latest snapshot tarballs here: + +* http://git.bogomips.org/cgit/unicorn.git (cgit) +* http://repo.or.cz/w/unicorn.git (gitweb) == Usage +=== non-Rails Rack applications + +In APP_ROOT, run: + + unicorn + +=== for Rails applications (should work for all 1.2 or later versions) + +In RAILS_ROOT, run: + + unicorn_rails + +Unicorn will bind to all interfaces TCP port 8080 by default. +You may use the +-l/--listen+ switch to bind to a different +address:port or a UNIX socket. + +=== Configuration File(s) + Unicorn will look for the config.ru file used by rackup in APP_ROOT. -Optionally, it can use a config file specified by the --config-file/-c -command-line switch. -Unicorn should be capable of running all Rack applications. Since this -is a preforking webserver, you do not have to worry about thread-safety -of your application or libraries. However, your Rack application may use -threads internally (and should even be able to continue running threads -after the request is complete). +For deployments, it can use a config file for Unicorn-specific options +specified by the +--config-file/-c+ command-line switch. See +Unicorn::Configurator for the syntax of the Unicorn-specific options. +The default settings are designed for maximum out-of-the-box +compatibility with existing applications. -== Contact +Most command-line options for other Rack applications (above) are also +supported. Run `unicorn -h` or `unicorn_rails -h` to see command-line +options. -Newsgroup and mailing list coming, or it'll be a part of the Mongrel project... +== Disclaimer + +Like the creatures themselves, production deployments of Unicorn are +rare or even non-existent. There is NO WARRANTY whatsoever if anything +goes wrong, but let us know and we'll try our best to fix it. + +Unicorn is designed to only serve fast clients. See the PHILOSOPHY +and DESIGN documents for more details regarding this. + +Rainbows are NOT included. + +== Known Issues + +* WONTFIX: code reloading with Sinatra 0.3.2 (and likely older + versions) apps is broken. The workaround is to force production + mode to disable code reloading in your Sinatra application: + set :env, :production + Since this is no longer an issue with Sinatra 0.9.x apps and only + affected non-production instances, this will not be fixed on our end. + Also remember we're capable of replacing the running binary without + dropping any connections regardless of framework :) + +== Contact Email Eric Wong at normalperson@yhbt.net for now. +Newsgroup and mailing list maybe coming... @@ -16,6 +16,7 @@ Echoe.new("unicorn") do |p| p.ignore_pattern = /^(pkg|site|projects|doc|log)|CVS|\.log/ p.need_tar_gz = false p.need_tgz = true + p.dependencies = [ 'rack' ] p.extension_pattern = ["ext/**/extconf.rb"] @@ -6,19 +6,28 @@ processes are documented here as well. === Master Process - * HUP - reload config file and gracefully restart all workers +* HUP - reload config file and gracefully restart all workers + If "preload_app" is false (the default), the application code + will be reloaded when workers are restarted as well. - * INT/TERM - quick shutdown, kills all workers immediately +* INT/TERM - quick shutdown, kills all workers immediately - * QUIT - graceful shutdown, waits for workers to finish their - current request before finishing. +* QUIT - graceful shutdown, waits for workers to finish their + current request before finishing. - * USR1 - reopen all logs owned by the master and all workers - See Unicorn::Util.reopen_logs for what is considered a log. +* USR1 - reopen all logs owned by the master and all workers + See Unicorn::Util.reopen_logs for what is considered a log. - * USR2 - reexecute the running binary. A separate QUIT - should be sent to the original process once the child is verified to - be up and running. +* USR2 - reexecute the running binary. A separate QUIT + should be sent to the original process once the child is verified to + be up and running. + +* WINCH - gracefully stops workers but keep the master running. + This will only work for daemonized processes. + +* TTIN - increment the number of worker processes by one + +* TTOU - decrement the number of worker processes by one === Worker Processes @@ -26,9 +35,64 @@ Sending signals directly to the worker processes should not normally be needed. If the master process is running, any exited worker will be automatically respawned. - * INT/TERM - quick shutdown, immediately exit +* INT/TERM - Quick shutdown, immediately exit. + Unless WINCH has been sent to the master (or the master is killed), + the master process will respawn a worker to replace this one. + +* QUIT - Gracefully exit after finishing the current request. + Unless WINCH has been sent to the master (or the master is killed), + the master process will respawn a worker to replace this one. + +* USR1 - Reopen all logs owned by the worker process. + See Unicorn::Util.reopen_logs for what is considered a log. + Log files are not reopened until it is done processing + the current request, so multiple log lines for one request + (as done by Rails) will not be split across multiple logs. + +=== Procedure to replace a running unicorn executable + +You may replace a running instance of unicorn with a new one without +losing any incoming connections. Doing so will reload all of your +application code, Unicorn config, Ruby executable, and all libraries. +The only things that will not change (due to OS limitations) are: + +1. The listener backlog size of already-bound sockets + +2. The path to the unicorn executable script. If you want to change to + a different installation of Ruby, you can modify the shebang + line to point to your alternative interpreter. + +The procedure is exactly like that of nginx: + +1. Send USR2 to the master process + +2. Check your process manager or pid files to see if a new master spawned + successfully. If you're using a pid file, the old process will have + ".oldbin" appended to its path. You should have two master instances + of unicorn running now, both of which will have workers servicing + requests. Your process tree should look something like this: + + unicorn master (old) + \_ unicorn worker[0] + \_ unicorn worker[1] + \_ unicorn worker[2] + \_ unicorn worker[3] + \_ unicorn master + \_ unicorn worker[0] + \_ unicorn worker[1] + \_ unicorn worker[2] + \_ unicorn worker[3] + +3. You can now send WINCH to the old master process so only the new workers + serve requests. If your unicorn process is bound to an interactive + terminal, you can skip this step. Step 5 will be more difficult but + you can also skip it if your process is not daemonized. + +4. You should now ensure that everything is running correctly with the + new workers as the old workers die off. - * QUIT - gracefully exit after finishing the current request +5. If everything seems ok, then send QUIT to the old master. You're done! - * USR1 - reopen all logs owned by the worker process - See Unicorn::Util.reopen_logs for what is considered a log. + If something is broken, then send HUP to the old master to reload + the config and restart its workers. Then send QUIT to the new master + process. @@ -1,5 +1,17 @@ -tests for timeouts -Performance -QA behaviour on 1.9 -X Make sure Echoe doesn't activate itself in packaged gems -Optimize Rack's dispatcher http://github.com/chneukirchen/rack/blob/cf040ea68a6a60a11f484a2145d2e62c34e7487e/lib/rack/urlmap.rb +== 1.0.0 + + * integration tests with nginx including bad client handling + + * manpages (why do so few Ruby executables come with proper manpages?) + +== 1.1.0 + + * Transfer-Encoding: chunked request handling. Testcase: + + curl -T- http://host:port/path < file_from_stdin + + * code cleanups (launchers) + + * Pure Ruby HTTP parser + + * Rubinius support? @@ -0,0 +1,63 @@ += Tuning Unicorn + +Unicorn performance is generally as good as a (mostly) Ruby web server +can provide. Most often the performance bottleneck is in the web +application running on Unicorn rather than Unicorn itself. + +== Unicorn Configuration + +See Unicorn::Configurator for details on the config file format. + +* Setting a very low value for the :backlog parameter in "listen" + directives can allow failover to happen more quickly if your + cluster is configured for it. + +* :rcvbuf and :sndbuf parameters generally do not need to be set for TCP + listeners under Linux 2.6 because auto-tuning is enabled. UNIX domain + sockets do not have auto-tuning buffer sizes; so increasing those will + allow syscalls and task switches to be saved for larger requests + and responses. + +* Setting "preload_app true" can allow copy-on-write-friendly GC to + be used to save memory. It will probably not work out of the box with + applications that open sockets or perform random I/O on files. + Databases like TokyoCabinet use concurrency-safe pread()/pwrite() + functions for safe sharing of database file descriptors across + processes. + +* On POSIX-compliant filesystems, it is safe for multiple threads or + processes to append to one log file as long as all the processes are + have them unbuffered (File#sync = true) or they are + record(line)-buffered in userspace. + +* worker_processes should be scaled to the number of processes your + backend system(s) can support. DO NOT scale it to the number of + external network clients your application expects to be serving. + Unicorn is NOT for serving slow clients, that is the job of nginx. + +== Kernel Parameters (Linux sysctl) + +WARNING: Do not change system parameters unless you know what you're doing! + +* net.core.rmem_max and net.core.wmem_max can increase the allowed + size of :rcvbuf and :sndbuf respectively. This is mostly only useful + for UNIX domain sockets which do not have auto-tuning buffer sizes. + +* For load testing/benchmarking with UNIX domain sockets, you should + consider increasing net.core.somaxconn or else nginx will start + failing to connect under heavy load. + +* If you're running out of local ports, consider lowering + net.ipv4.tcp_fin_timeout to 20-30 (default: 60 seconds). Also + consider widening the usable port range by changing + net.ipv4.ip_local_port_range. + +* Setting net.ipv4.tcp_timestamps=1 will also allow setting + net.ipv4.tcp_tw_reuse=1 and net.ipv4.tcp_tw_recycle=1, which along + with the above settings can slow down port exhaustion. Not all + networks are compatible with these settings, check with your friendly + network administrator before changing these. + +* Increasing the MTU size can reduce framing overhead for larger + transfers. One often-overlooked detail is that the loopback + device (usually "lo") can have its MTU increased, too. diff --git a/bin/unicorn b/bin/unicorn index ebf57c3..a34d9bc 100755 --- a/bin/unicorn +++ b/bin/unicorn @@ -1,14 +1,13 @@ #!/home/ew/bin/ruby -$stdin.sync = $stdout.sync = $stderr.sync = true -require 'unicorn' # require this first to populate Unicorn::DEFAULT_START_CTX +require 'unicorn/launcher' require 'optparse' env = "development" daemonize = false listeners = [] options = { :listeners => listeners } -host = Unicorn::Const::DEFAULT_HOST -port = Unicorn::Const::DEFAULT_PORT +host, port = Unicorn::Const::DEFAULT_HOST, Unicorn::Const::DEFAULT_PORT +set_listener = false opts = OptionParser.new("", 24, ' ') do |opts| opts.banner = "Usage: #{File.basename($0)} " \ @@ -47,11 +46,13 @@ opts = OptionParser.new("", 24, ' ') do |opts| opts.on("-o", "--host HOST", "listen on HOST (default: #{Unicorn::Const::DEFAULT_HOST})") do |h| host = h + set_listener = true end opts.on("-p", "--port PORT", "use PORT (default: #{Unicorn::Const::DEFAULT_PORT})") do |p| port = p.to_i + set_listener = true end opts.on("-E", "--env ENVIRONMENT", @@ -104,86 +105,58 @@ opts = OptionParser.new("", 24, ' ') do |opts| opts.parse! ARGV end -require 'pp' if $DEBUG - -# require Rack as late as possible in case $LOAD_PATH is modified -# in config.ru or command-line -require 'rack' - config = ARGV[0] || "config.ru" abort "configuration file #{config} not found" unless File.exist?(config) -inner_app = case config -when /\.ru$/ - raw = File.open(config, "rb") { |fp| fp.sysread(fp.stat.size) } +if config =~ /\.ru$/ # parse embedded command-line options in config.ru comments - if raw[/^#\\(.*)/] + if File.open(config, "rb") { |fp| fp.sysread(fp.stat.size) } =~ /^#\\(.*)/ opts.parse! $1.split(/\s+/) end - lambda { || eval("Rack::Builder.new {(#{raw}\n)}.to_app", nil, config) } -else - lambda do || +end + +require 'pp' if $DEBUG + +app = lambda do || + # require Rack as late as possible in case $LOAD_PATH is modified + # in config.ru or command-line + require 'rack' + inner_app = case config + when /\.ru$/ + raw = File.open(config, "rb") { |fp| fp.sysread(fp.stat.size) } + eval("Rack::Builder.new {(#{raw}\n)}.to_app", nil, config) + else require config Object.const_get(File.basename(config, '.rb').capitalize) end -end - -app = case env -when "development" - lambda do || + pp({ :inner_app => inner_app }) if $DEBUG + case env + when "development" Rack::Builder.new do use Rack::CommonLogger, $stderr use Rack::ShowExceptions use Rack::Lint - run inner_app.call + run inner_app end.to_app - end -when "deployment" - lambda do || + when "deployment" Rack::Builder.new do use Rack::CommonLogger, $stderr - run inner_app.call + run inner_app end.to_app + else + inner_app end -else - inner_app end -if listeners.empty? - listener = "#{host}:#{port}" - listeners << listener -end +listeners << "#{host}:#{port}" if set_listener if $DEBUG pp({ :unicorn_options => options, :app => app, - :inner_app => inner_app, :daemonize => daemonize, }) end -# only daemonize if we're not inheriting file descriptors from our parent -if daemonize - - $stdin.reopen("/dev/null") - unless ENV['UNICORN_FD'] - exit if fork - Process.setsid - exit if fork - end - - # We don't do a lot of standard daemonization stuff: - # * $stderr/$stderr can/will be redirected separately - # * umask is whatever was set by the parent process at startup - # and can be set in config.ru and config_file, so making it - # 0000 and potentially exposing sensitive log data can be bad - # policy. - # * Don't bother to chdir here since Unicorn is designed to - # run inside APP_ROOT. Unicorn will also re-chdir() to - # the directory it was started in when being re-executed - # to pickup code changes if the original deployment directory - # is a symlink or otherwise got replaced. -end - +Unicorn::Launcher.daemonize! if daemonize Unicorn.run(app, options) diff --git a/bin/unicorn_rails b/bin/unicorn_rails new file mode 100755 index 0000000..b3fda7b --- /dev/null +++ b/bin/unicorn_rails @@ -0,0 +1,202 @@ +#!/home/ew/bin/ruby +require 'unicorn/launcher' +require 'optparse' +require 'fileutils' + +rails_pid = "#{Unicorn::HttpServer::START_CTX[:cwd]}/tmp/pids/unicorn.pid" +cmd = File.basename($0) +daemonize = false +listeners = [] +options = { :listeners => listeners } +host, port = Unicorn::Const::DEFAULT_HOST, Unicorn::Const::DEFAULT_PORT +set_listener = false +ENV['RAILS_ENV'] ||= "development" +map_path = ENV['RAILS_RELATIVE_URL_ROOT'] + +opts = OptionParser.new("", 24, ' ') do |opts| + opts.banner = "Usage: #{cmd} " \ + "[ruby options] [#{cmd} options] [rackup config file]" + opts.separator "Ruby options:" + + lineno = 1 + opts.on("-e", "--eval LINE", "evaluate a LINE of code") do |line| + eval line, TOPLEVEL_BINDING, "-e", lineno + lineno += 1 + end + + opts.on("-d", "--debug", "set debugging flags (set $DEBUG to true)") do + $DEBUG = true + end + + opts.on("-w", "--warn", "turn warnings on for your script") do + $-w = true + end + + opts.on("-I", "--include PATH", + "specify $LOAD_PATH (may be used more than once)") do |path| + $LOAD_PATH.unshift(*path.split(/:/)) + end + + opts.on("-r", "--require LIBRARY", + "require the library, before executing your script") do |library| + require library + end + + opts.separator "#{cmd} options:" + + # some of these switches exist for rackup command-line compatibility, + + opts.on("-o", "--host HOST", + "listen on HOST (default: #{Unicorn::Const::DEFAULT_HOST})") do |h| + host = h + set_listener = true + end + + opts.on("-p", "--port PORT", "use PORT (default: #{port})") do |p| + port = p.to_i + set_listener = true + end + + opts.on("-E", "--env ENVIRONMENT", + "use ENVIRONMENT for defaults (default: development)") do |e| + ENV['RAILS_ENV'] = e + end + + opts.on("-D", "--daemonize", "run daemonized in the background") do |d| + daemonize = d ? true : false + end + + # Unicorn-specific stuff + opts.on("-l", "--listen {HOST:PORT|PATH}", + "listen on HOST:PORT or PATH", + "this may be specified multiple times", + "(default: #{Unicorn::Const::DEFAULT_LISTEN})") do |address| + listeners << address + end + + opts.on("-c", "--config-file FILE", "Unicorn-specific config file") do |f| + options[:config_file] = File.expand_path(f) + end + + opts.on("-P", "--path PATH", "Runs Rails app mounted at a specific path.", + "(default: /") do |v| + ENV['RAILS_RELATIVE_URL_ROOT'] = map_path = v + end + + # I'm avoiding Unicorn-specific config options on the command-line. + # IMNSHO, config options on the command-line are redundant given + # config files and make things unnecessarily complicated with multiple + # places to look for a config option. + + opts.separator "Common options:" + + opts.on_tail("-h", "--help", "Show this message") do + puts opts + exit + end + + opts.on_tail("-v", "--version", "Show version") do + puts " v#{Unicorn::Const::UNICORN_VERSION}" + exit + end + + opts.parse! ARGV +end + +config = ARGV[0] || (File.exist?('config.ru') ? 'config.ru' : nil) + +if config && config =~ /\.ru$/ + # parse embedded command-line options in config.ru comments + if File.open(config, "rb") { |fp| fp.sysread(fp.stat.size) } =~ /^#\\(.*)/ + opts.parse! $1.split(/\s+/) + end +end + +require 'pp' if $DEBUG + +# this won't run until after forking if preload_app is false +app = lambda do || + # Load Rails and the private version of Rack it bundles. + begin + require 'config/boot' + rescue LoadError => err + abort "#$0 must be run inside RAILS_ROOT: #{err.inspect}" + end + defined?(::RAILS_ROOT) or abort "RAILS_ROOT not defined by config/boot" + defined?(::RAILS_ENV) or abort "RAILS_ENV not defined by config/boot" + defined?(::Rails::VERSION::STRING) or + abort "Rails::VERSION::STRING not defined by config/boot" + + inner_app = case config + when nil + require 'config/environment' + + # it seems Rails >=2.2 support Rack, but only >=2.3 requires it + old_rails = case ::Rails::VERSION::MAJOR + when 0, 1 then true + when 2 then Rails::VERSION::MINOR < 3 ? true : false + else + false + end + + if old_rails + require 'rack' + require 'unicorn/app/old_rails' + Unicorn::App::OldRails.new + else + ActionController::Dispatcher.new + end + when /\.ru$/ + raw = File.open(config, "rb") { |fp| fp.sysread(fp.stat.size) } + eval("Rack::Builder.new {(#{raw}\n)}.to_app", nil, config) + else + require config + Object.const_get(File.basename(config, '.rb').capitalize) + end + + map_path ||= '/' + Rack::Builder.new do + if inner_app.class.to_s == "Unicorn::App::OldRails" + if map_path != '/' + # patches + tests welcome, but I really cbf to deal with this + # since all apps I've ever dealt with just use "/" ... + $stderr.puts "relative URL roots may not work for older Rails" + end + $stderr.puts "LogTailer not available for Rails < 2.3" unless daemonize + $stderr.puts "Debugger not available" if $DEBUG + map(map_path) do + require 'unicorn/app/old_rails/static' + use Unicorn::App::OldRails::Static + run inner_app + end + else + use Rails::Rack::LogTailer unless daemonize + use Rails::Rack::Debugger if $DEBUG + map(map_path) do + use Rails::Rack::Static + run inner_app + end + end + end.to_app +end + +listeners << "#{host}:#{port}" if set_listener + +if $DEBUG + pp({ + :unicorn_options => options, + :app => app, + :daemonize => daemonize, + }) +end + +# ensure Rails standard tmp paths exist +%w(cache pids sessions sockets).each do |dir| + FileUtils.mkdir_p("tmp/#{dir}") +end + +if daemonize + options[:pid] = rails_pid + Unicorn::Launcher.daemonize! +end +Unicorn.run(app, options) diff --git a/examples/init.sh b/examples/init.sh new file mode 100644 index 0000000..866a644 --- /dev/null +++ b/examples/init.sh @@ -0,0 +1,54 @@ +#!/bin/sh +set -u +set -e +# Example init script, this can be used with nginx, too, +# since nginx and unicorn accept the same signals + +# Feel free to change any of the following variables for your app: +APP_ROOT=/home/x/my_app/current +PID=$APP_ROOT/tmp/pids/unicorn.pid +CMD="/usr/bin/unicorn -D -c $APP_ROOT/config/unicorn.rb" +INIT_CONF=$APP_ROOT/config/init.conf + +test -f "$INIT_CONF" && . $INIT_CONF + +old_pid="$PID.oldbin" + +cd $APP_ROOT || exit 1 + +sig () { + test -s "$PID" && kill -$1 `cat $PID` +} + +oldsig () { + test -s $old_pid && kill -$1 `cat $old_pid` +} + +case $1 in +start) + sig 0 && echo >&2 "Already running" && exit 0 + $CMD + ;; +stop) + sig QUIT && exit 0 + echo >&2 "Not running" + ;; +force-stop) + sig TERM && exit 0 + echo >&2 "Not running" + ;; +restart|reload) + sig HUP && echo reloaded OK && exit 0 + echo >&2 "Couldn't reload, starting '$CMD' instead" + $CMD + ;; +upgrade) + sig USR2 && sleep 2 && sig 0 && oldsig QUIT && exit 0 + echo >&2 "Couldn't upgrade, starting '$CMD' instead" + $CMD + ;; +*) + echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop>" + exit 1 + ;; +esac diff --git a/ext/unicorn/http11/ext_help.h b/ext/unicorn/http11/ext_help.h index 08c0e1e..17f7b01 100644 --- a/ext/unicorn/http11/ext_help.h +++ b/ext/unicorn/http11/ext_help.h @@ -1,9 +1,6 @@ #ifndef ext_help_h #define ext_help_h -#define RAISE_NOT_NULL(T) if(T == NULL) rb_raise(rb_eArgError, "NULL found for " # T " when shouldn't be."); -#define DATA_GET(from,type,name) Data_Get_Struct(from,type,name); RAISE_NOT_NULL(name); -#define REQUIRE_TYPE(V, T) if(TYPE(V) != T) rb_raise(rb_eTypeError, "Wrong argument type for " # V " required " # T); #define ARRAY_SIZE(x) (sizeof(x)/sizeof(x[0])) #ifdef DEBUG diff --git a/ext/unicorn/http11/http11.c b/ext/unicorn/http11/http11.c index d5c364a..cd7a8f7 100644 --- a/ext/unicorn/http11/http11.c +++ b/ext/unicorn/http11/http11.c @@ -1,4 +1,5 @@ /** + * Copyright (c) 2009 Eric Wong (all bugs are Eric's fault) * Copyright (c) 2005 Zed A. Shaw * You can redistribute it and/or modify it under the same terms as Ruby. */ @@ -8,6 +9,16 @@ #include <string.h> #include "http11_parser.h" +static http_parser *data_get(VALUE self) +{ + http_parser *http; + + Data_Get_Struct(self, http_parser, http); + if (!http) + rb_raise(rb_eArgError, "NULL found for http when shouldn't be."); + return http; +} + #ifndef RSTRING_PTR #define RSTRING_PTR(s) (RSTRING(s)->ptr) #endif @@ -18,40 +29,50 @@ static VALUE mUnicorn; static VALUE cHttpParser; static VALUE eHttpParserError; +static VALUE sym_http_body; #define HTTP_PREFIX "HTTP_" #define HTTP_PREFIX_LEN (sizeof(HTTP_PREFIX) - 1) +static VALUE global_rack_url_scheme; static VALUE global_request_method; static VALUE global_request_uri; static VALUE global_fragment; static VALUE global_query_string; static VALUE global_http_version; -static VALUE global_content_length; -static VALUE global_http_content_length; static VALUE global_request_path; -static VALUE global_content_type; -static VALUE global_http_content_type; -static VALUE global_http_body; -static VALUE global_gateway_interface; -static VALUE global_gateway_interface_value; +static VALUE global_path_info; static VALUE global_server_name; static VALUE global_server_port; static VALUE global_server_protocol; static VALUE global_server_protocol_value; static VALUE global_http_host; +static VALUE global_http_x_forwarded_proto; static VALUE global_port_80; +static VALUE global_port_443; static VALUE global_localhost; +static VALUE global_http; /** Defines common length and error messages for input length validation. */ -#define DEF_MAX_LENGTH(N,length) const size_t MAX_##N##_LENGTH = length; const char *MAX_##N##_LENGTH_ERR = "HTTP element " # N " is longer than the " # length " allowed length." +#define DEF_MAX_LENGTH(N, length) \ + static const size_t MAX_##N##_LENGTH = length; \ + static const char * const MAX_##N##_LENGTH_ERR = \ + "HTTP element " # N " is longer than the " # length " allowed length." -/** Validates the max length of given input and throws an HttpParserError exception if over. */ -#define VALIDATE_MAX_LENGTH(len, N) if(len > MAX_##N##_LENGTH) { rb_raise(eHttpParserError, MAX_##N##_LENGTH_ERR); } +/** + * Validates the max length of given input and throws an HttpParserError + * exception if over. + */ +#define VALIDATE_MAX_LENGTH(len, N) do { \ + if (len > MAX_##N##_LENGTH) \ + rb_raise(eHttpParserError, MAX_##N##_LENGTH_ERR); \ +} while (0) /** Defines global strings in the init method. */ -#define DEF_GLOBAL(N, val) global_##N = rb_obj_freeze(rb_str_new2(val)); rb_global_variable(&global_##N) - +#define DEF_GLOBAL(N, val) do { \ + global_##N = rb_obj_freeze(rb_str_new(val, sizeof(val) - 1)); \ + rb_global_variable(&global_##N); \ +} while (0) /* Defines the maximum allowed lengths for various input elements.*/ DEF_MAX_LENGTH(FIELD_NAME, 256); @@ -109,26 +130,13 @@ static struct common_field common_http_fields[] = { f("USER_AGENT"), f("VIA"), f("X_FORWARDED_FOR"), /* common for proxies */ + f("X_FORWARDED_PROTO"), /* common for proxies */ f("X_REAL_IP"), /* common for proxies */ f("WARNING") # undef f }; -/* - * qsort(3) and bsearch(3) improve average performance slightly, but may - * not be worth it for lack of portability to certain platforms... - */ -#if defined(HAVE_QSORT_BSEARCH) -/* sort by length, then by name if there's a tie */ -static int common_field_cmp(const void *a, const void *b) -{ - struct common_field *cfa = (struct common_field *)a; - struct common_field *cfb = (struct common_field *)b; - signed long diff = cfa->len - cfb->len; - return diff ? diff : memcmp(cfa->name, cfb->name, cfa->len); -} -#endif /* HAVE_QSORT_BSEARCH */ - +/* this function is not performance-critical */ static void init_common_fields(void) { int i; @@ -137,32 +145,21 @@ static void init_common_fields(void) memcpy(tmp, HTTP_PREFIX, HTTP_PREFIX_LEN); for(i = 0; i < ARRAY_SIZE(common_http_fields); cf++, i++) { - memcpy(tmp + HTTP_PREFIX_LEN, cf->name, cf->len + 1); - cf->value = rb_obj_freeze(rb_str_new(tmp, HTTP_PREFIX_LEN + cf->len)); + /* Rack doesn't like certain headers prefixed with "HTTP_" */ + if (!strcmp("CONTENT_LENGTH", cf->name) || + !strcmp("CONTENT_TYPE", cf->name)) { + cf->value = rb_str_new(cf->name, cf->len); + } else { + memcpy(tmp + HTTP_PREFIX_LEN, cf->name, cf->len + 1); + cf->value = rb_str_new(tmp, HTTP_PREFIX_LEN + cf->len); + } + cf->value = rb_obj_freeze(cf->value); rb_global_variable(&cf->value); } - -#if defined(HAVE_QSORT_BSEARCH) - qsort(common_http_fields, - ARRAY_SIZE(common_http_fields), - sizeof(struct common_field), - common_field_cmp); -#endif /* HAVE_QSORT_BSEARCH */ } static VALUE find_common_field_value(const char *field, size_t flen) { -#if defined(HAVE_QSORT_BSEARCH) - struct common_field key; - struct common_field *found; - key.name = field; - key.len = (signed long)flen; - found = (struct common_field *)bsearch(&key, common_http_fields, - ARRAY_SIZE(common_http_fields), - sizeof(struct common_field), - common_field_cmp); - return found ? found->value : Qnil; -#else /* !HAVE_QSORT_BSEARCH */ int i; struct common_field *cf = common_http_fields; for(i = 0; i < ARRAY_SIZE(common_http_fields); i++, cf++) { @@ -170,21 +167,17 @@ static VALUE find_common_field_value(const char *field, size_t flen) return cf->value; } return Qnil; -#endif /* !HAVE_QSORT_BSEARCH */ } static void http_field(void *data, const char *field, size_t flen, const char *value, size_t vlen) { VALUE req = (VALUE)data; - VALUE v = Qnil; VALUE f = Qnil; VALIDATE_MAX_LENGTH(flen, FIELD_NAME); VALIDATE_MAX_LENGTH(vlen, FIELD_VALUE); - v = rb_str_new(value, vlen); - f = find_common_field_value(field, flen); if (f == Qnil) { @@ -203,9 +196,11 @@ static void http_field(void *data, const char *field, memcpy(RSTRING_PTR(f) + HTTP_PREFIX_LEN, field, flen); assert(*(RSTRING_PTR(f) + RSTRING_LEN(f)) == '\0'); /* paranoia */ /* fprintf(stderr, "UNKNOWN HEADER <%s>\n", RSTRING_PTR(f)); */ + } else if (f == global_http_host && rb_hash_aref(req, f) != Qnil) { + return; } - rb_hash_aset(req, f, v); + rb_hash_aset(req, f, rb_str_new(value, vlen)); } static void request_method(void *data, const char *at, size_t length) @@ -217,6 +212,16 @@ static void request_method(void *data, const char *at, size_t length) rb_hash_aset(req, global_request_method, val); } +static void scheme(void *data, const char *at, size_t length) +{ + rb_hash_aset((VALUE)data, global_rack_url_scheme, rb_str_new(at, length)); +} + +static void host(void *data, const char *at, size_t length) +{ + rb_hash_aset((VALUE)data, global_http_host, rb_str_new(at, length)); +} + static void request_uri(void *data, const char *at, size_t length) { VALUE req = (VALUE)data; @@ -226,6 +231,13 @@ static void request_uri(void *data, const char *at, size_t length) val = rb_str_new(at, length); rb_hash_aset(req, global_request_uri, val); + + /* "OPTIONS * HTTP/1.1\r\n" is a valid request */ + if (length == 1 && *at == '*') { + val = rb_str_new(NULL, 0); + rb_hash_aset(req, global_request_path, val); + rb_hash_aset(req, global_path_info, val); + } } static void fragment(void *data, const char *at, size_t length) @@ -248,6 +260,10 @@ static void request_path(void *data, const char *at, size_t length) val = rb_str_new(at, length); rb_hash_aset(req, global_request_path, val); + + /* rack says PATH_INFO must start with "/" or be empty */ + if (!(length == 1 && *at == '*')) + rb_hash_aset(req, global_path_info, val); } static void query_string(void *data, const char *at, size_t length) @@ -268,46 +284,49 @@ static void http_version(void *data, const char *at, size_t length) rb_hash_aset(req, global_http_version, val); } -/** Finalizes the request header to have a bunch of stuff that's - needed. */ - +/** Finalizes the request header to have a bunch of stuff that's needed. */ static void header_done(void *data, const char *at, size_t length) { VALUE req = (VALUE)data; - VALUE temp = Qnil; - VALUE ctype = Qnil; - VALUE clen = Qnil; - char *colon = NULL; - - clen = rb_hash_aref(req, global_http_content_length); - if(clen != Qnil) { - rb_hash_aset(req, global_content_length, clen); + VALUE server_name = global_localhost; + VALUE server_port = global_port_80; + VALUE temp; + + /* rack requires QUERY_STRING */ + if (rb_hash_aref(req, global_query_string) == Qnil) + rb_hash_aset(req, global_query_string, rb_str_new(NULL, 0)); + + /* set rack.url_scheme to "https" or "http", no others are allowed by Rack */ + if ((temp = rb_hash_aref(req, global_rack_url_scheme)) == Qnil) { + if ((temp = rb_hash_aref(req, global_http_x_forwarded_proto)) != Qnil && + RSTRING_LEN(temp) == 5 && + !memcmp("https", RSTRING_PTR(temp), 5)) + server_port = global_port_443; + else + temp = global_http; + rb_hash_aset(req, global_rack_url_scheme, temp); + } else if (RSTRING_LEN(temp) == 5 && !memcmp("https", RSTRING_PTR(temp), 5)) { + server_port = global_port_443; } - ctype = rb_hash_aref(req, global_http_content_type); - if(ctype != Qnil) { - rb_hash_aset(req, global_content_type, ctype); - } + /* parse and set the SERVER_NAME and SERVER_PORT variables */ + if ((temp = rb_hash_aref(req, global_http_host)) != Qnil) { + char *colon = memchr(RSTRING_PTR(temp), ':', RSTRING_LEN(temp)); + if (colon) { + long port_start = colon - RSTRING_PTR(temp) + 1; - rb_hash_aset(req, global_gateway_interface, global_gateway_interface_value); - if((temp = rb_hash_aref(req, global_http_host)) != Qnil) { - colon = memchr(RSTRING_PTR(temp), ':', RSTRING_LEN(temp)); - if(colon != NULL) { - rb_hash_aset(req, global_server_name, rb_str_substr(temp, 0, colon - RSTRING_PTR(temp))); - rb_hash_aset(req, global_server_port, - rb_str_substr(temp, colon - RSTRING_PTR(temp)+1, - RSTRING_LEN(temp))); + server_name = rb_str_substr(temp, 0, colon - RSTRING_PTR(temp)); + if ((RSTRING_LEN(temp) - port_start) > 0) + server_port = rb_str_substr(temp, port_start, RSTRING_LEN(temp)); } else { - rb_hash_aset(req, global_server_name, temp); - rb_hash_aset(req, global_server_port, global_port_80); + server_name = temp; } - } else { - rb_hash_aset(req, global_server_name, global_localhost); - rb_hash_aset(req, global_server_port, global_port_80); } + rb_hash_aset(req, global_server_name, server_name); + rb_hash_aset(req, global_server_port, server_port); /* grab the initial body and stuff it into the hash */ - rb_hash_aset(req, global_http_body, rb_str_new(at, length)); + rb_hash_aset(req, sym_http_body, rb_str_new(at, length)); rb_hash_aset(req, global_server_protocol, global_server_protocol_value); } @@ -325,14 +344,6 @@ static VALUE HttpParser_alloc(VALUE klass) VALUE obj; http_parser *hp = ALLOC_N(http_parser, 1); TRACE(); - hp->http_field = http_field; - hp->request_method = request_method; - hp->request_uri = request_uri; - hp->fragment = fragment; - hp->request_path = request_path; - hp->query_string = query_string; - hp->http_version = http_version; - hp->header_done = header_done; http_parser_init(hp); obj = Data_Wrap_Struct(klass, NULL, HttpParser_free, hp); @@ -349,9 +360,7 @@ static VALUE HttpParser_alloc(VALUE klass) */ static VALUE HttpParser_init(VALUE self) { - http_parser *http = NULL; - DATA_GET(self, http_parser, http); - http_parser_init(http); + http_parser_init(data_get(self)); return self; } @@ -366,9 +375,7 @@ static VALUE HttpParser_init(VALUE self) */ static VALUE HttpParser_reset(VALUE self) { - http_parser *http = NULL; - DATA_GET(self, http_parser, http); - http_parser_init(http); + http_parser_init(data_get(self)); return Qnil; } @@ -376,140 +383,58 @@ static VALUE HttpParser_reset(VALUE self) /** * call-seq: - * parser.finish -> true/false - * - * Finishes a parser early which could put in a "good" or bad state. - * You should call reset after finish it or bad things will happen. - */ -static VALUE HttpParser_finish(VALUE self) -{ - http_parser *http = NULL; - DATA_GET(self, http_parser, http); - http_parser_finish(http); - - return http_parser_is_finished(http) ? Qtrue : Qfalse; -} - - -/** - * call-seq: - * parser.execute(req_hash, data, start) -> Integer + * parser.execute(req_hash, data) -> true/false * - * Takes a Hash and a String of data, parses the String of data filling in the Hash - * returning an Integer to indicate how much of the data has been read. No matter - * what the return value, you should call HttpParser#finished? and HttpParser#error? - * to figure out if it's done parsing or there was an error. + * Takes a Hash and a String of data, parses the String of data filling + * in the Hash returning a boolean to indicate whether or not parsing + * is finished. * - * This function now throws an exception when there is a parsing error. This makes - * the logic for working with the parser much easier. You can still test for an - * error, but now you need to wrap the parser with an exception handling block. - * - * The third argument allows for parsing a partial request and then continuing - * the parsing from that position. It needs all of the original data as well - * so you have to append to the data buffer as you read. - */ -static VALUE HttpParser_execute(VALUE self, VALUE req_hash, - VALUE data, VALUE start) -{ - http_parser *http = NULL; - int from = 0; - char *dptr = NULL; - long dlen = 0; - - DATA_GET(self, http_parser, http); - - from = FIX2INT(start); - dptr = RSTRING_PTR(data); - dlen = RSTRING_LEN(data); - - if(from >= dlen) { - rb_raise(eHttpParserError, "Requested start is after data buffer end."); - } else { - http->data = (void *)req_hash; - http_parser_execute(http, dptr, dlen, from); - - VALIDATE_MAX_LENGTH(http_parser_nread(http), HEADER); - - if(http_parser_has_error(http)) { - rb_raise(eHttpParserError, "Invalid HTTP format, parsing fails."); - } else { - return INT2FIX(http_parser_nread(http)); - } - } -} - - - -/** - * call-seq: - * parser.error? -> true/false - * - * Tells you whether the parser is in an error state. + * This function now throws an exception when there is a parsing error. + * This makes the logic for working with the parser much easier. You + * will need to wrap the parser with an exception handling block. */ -static VALUE HttpParser_has_error(VALUE self) -{ - http_parser *http = NULL; - DATA_GET(self, http_parser, http); - - return http_parser_has_error(http) ? Qtrue : Qfalse; -} - -/** - * call-seq: - * parser.finished? -> true/false - * - * Tells you whether the parser is finished or not and in a good state. - */ -static VALUE HttpParser_is_finished(VALUE self) +static VALUE HttpParser_execute(VALUE self, VALUE req_hash, VALUE data) { - http_parser *http = NULL; - DATA_GET(self, http_parser, http); + http_parser *http = data_get(self); + char *dptr = RSTRING_PTR(data); + long dlen = RSTRING_LEN(data); - return http_parser_is_finished(http) ? Qtrue : Qfalse; -} + if (http->nread < dlen) { + http->data = (void *)req_hash; + http_parser_execute(http, dptr, dlen); + VALIDATE_MAX_LENGTH(http->nread, HEADER); -/** - * call-seq: - * parser.nread -> Integer - * - * Returns the amount of data processed so far during this processing cycle. It is - * set to 0 on initialize or reset calls and is incremented each time execute is called. - */ -static VALUE HttpParser_nread(VALUE self) -{ - http_parser *http = NULL; - DATA_GET(self, http_parser, http); + if (!http_parser_has_error(http)) + return http_parser_is_finished(http) ? Qtrue : Qfalse; - return INT2FIX(http->nread); + rb_raise(eHttpParserError, "Invalid HTTP format, parsing fails."); + } + rb_raise(eHttpParserError, "Requested start is after data buffer end."); } -void Init_http11() +void Init_http11(void) { - mUnicorn = rb_define_module("Unicorn"); + DEF_GLOBAL(rack_url_scheme, "rack.url_scheme"); DEF_GLOBAL(request_method, "REQUEST_METHOD"); DEF_GLOBAL(request_uri, "REQUEST_URI"); DEF_GLOBAL(fragment, "FRAGMENT"); DEF_GLOBAL(query_string, "QUERY_STRING"); DEF_GLOBAL(http_version, "HTTP_VERSION"); DEF_GLOBAL(request_path, "REQUEST_PATH"); - DEF_GLOBAL(content_length, "CONTENT_LENGTH"); - DEF_GLOBAL(http_content_length, "HTTP_CONTENT_LENGTH"); - DEF_GLOBAL(http_body, "HTTP_BODY"); - DEF_GLOBAL(content_type, "CONTENT_TYPE"); - DEF_GLOBAL(http_content_type, "HTTP_CONTENT_TYPE"); - DEF_GLOBAL(gateway_interface, "GATEWAY_INTERFACE"); - DEF_GLOBAL(gateway_interface_value, "CGI/1.2"); + DEF_GLOBAL(path_info, "PATH_INFO"); DEF_GLOBAL(server_name, "SERVER_NAME"); DEF_GLOBAL(server_port, "SERVER_PORT"); DEF_GLOBAL(server_protocol, "SERVER_PROTOCOL"); DEF_GLOBAL(server_protocol_value, "HTTP/1.1"); - DEF_GLOBAL(http_host, "HTTP_HOST"); + DEF_GLOBAL(http_x_forwarded_proto, "HTTP_X_FORWARDED_PROTO"); DEF_GLOBAL(port_80, "80"); + DEF_GLOBAL(port_443, "443"); DEF_GLOBAL(localhost, "localhost"); + DEF_GLOBAL(http, "http"); eHttpParserError = rb_define_class_under(mUnicorn, "HttpParserError", rb_eIOError); @@ -517,10 +442,9 @@ void Init_http11() rb_define_alloc_func(cHttpParser, HttpParser_alloc); rb_define_method(cHttpParser, "initialize", HttpParser_init,0); rb_define_method(cHttpParser, "reset", HttpParser_reset,0); - rb_define_method(cHttpParser, "finish", HttpParser_finish,0); - rb_define_method(cHttpParser, "execute", HttpParser_execute,3); - rb_define_method(cHttpParser, "error?", HttpParser_has_error,0); - rb_define_method(cHttpParser, "finished?", HttpParser_is_finished,0); - rb_define_method(cHttpParser, "nread", HttpParser_nread,0); + rb_define_method(cHttpParser, "execute", HttpParser_execute,2); + sym_http_body = ID2SYM(rb_intern("http_body")); init_common_fields(); + global_http_host = find_common_field_value("HOST", 4); + assert(global_http_host != Qnil); } diff --git a/ext/unicorn/http11/http11_parser.c b/ext/unicorn/http11/http11_parser.c deleted file mode 100644 index d33eed0..0000000 --- a/ext/unicorn/http11/http11_parser.c +++ /dev/null @@ -1,1220 +0,0 @@ -#line 1 "http11_parser.rl" -/** - * Copyright (c) 2005 Zed A. Shaw - * You can redistribute it and/or modify it under the same terms as Ruby. - */ -#include "http11_parser.h" -#include <stdio.h> -#include <assert.h> -#include <stdlib.h> -#include <ctype.h> -#include <string.h> - -/* - * capitalizes all lower-case ASCII characters, - * converts dashes to underscores. - */ -static void snake_upcase_char(char *c) -{ - if (*c >= 'a' && *c <= 'z') - *c &= ~0x20; - else if (*c == '-') - *c = '_'; -} - -#define LEN(AT, FPC) (FPC - buffer - parser->AT) -#define MARK(M,FPC) (parser->M = (FPC) - buffer) -#define PTR_TO(F) (buffer + parser->F) - -/** Machine **/ - -#line 87 "http11_parser.rl" - - -/** Data **/ - -#line 37 "http11_parser.c" -static const int http_parser_start = 1; -static const int http_parser_first_final = 57; -static const int http_parser_error = 0; - -static const int http_parser_en_main = 1; - -#line 91 "http11_parser.rl" - -int http_parser_init(http_parser *parser) { - int cs = 0; - -#line 49 "http11_parser.c" - { - cs = http_parser_start; - } -#line 95 "http11_parser.rl" - parser->cs = cs; - parser->body_start = 0; - parser->content_len = 0; - parser->mark = 0; - parser->nread = 0; - parser->field_len = 0; - parser->field_start = 0; - - return(1); -} - - -/** exec **/ -size_t http_parser_execute(http_parser *parser, const char *buffer, size_t len, size_t off) { - const char *p, *pe; - int cs = parser->cs; - - assert(off <= len && "offset past end of buffer"); - - p = buffer+off; - pe = buffer+len; - - assert(*pe == '\0' && "pointer does not end on NUL"); - assert(pe - p == len - off && "pointers aren't same distance"); - - -#line 80 "http11_parser.c" - { - if ( p == pe ) - goto _test_eof; - switch ( cs ) - { -case 1: - switch( (*p) ) { - case 36: goto tr0; - case 95: goto tr0; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto tr0; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto tr0; - } else - goto tr0; - goto st0; -st0: -cs = 0; - goto _out; -tr0: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st2; -st2: - if ( ++p == pe ) - goto _test_eof2; -case 2: -#line 111 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st38; - case 95: goto st38; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st38; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st38; - } else - goto st38; - goto st0; -tr2: -#line 49 "http11_parser.rl" - { - if(parser->request_method != NULL) - parser->request_method(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st3; -st3: - if ( ++p == pe ) - goto _test_eof3; -case 3: -#line 137 "http11_parser.c" - switch( (*p) ) { - case 42: goto tr4; - case 43: goto tr5; - case 47: goto tr6; - case 58: goto tr7; - } - if ( (*p) < 65 ) { - if ( 45 <= (*p) && (*p) <= 57 ) - goto tr5; - } else if ( (*p) > 90 ) { - if ( 97 <= (*p) && (*p) <= 122 ) - goto tr5; - } else - goto tr5; - goto st0; -tr4: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st4; -st4: - if ( ++p == pe ) - goto _test_eof4; -case 4: -#line 161 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr8; - case 35: goto tr9; - } - goto st0; -tr8: -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -tr31: -#line 34 "http11_parser.rl" - {MARK(mark, p); } -#line 57 "http11_parser.rl" - { - if(parser->fragment != NULL) - parser->fragment(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -tr34: -#line 57 "http11_parser.rl" - { - if(parser->fragment != NULL) - parser->fragment(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -tr42: -#line 73 "http11_parser.rl" - { - if(parser->request_path != NULL) - parser->request_path(parser->data, PTR_TO(mark), LEN(mark,p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -tr53: -#line 62 "http11_parser.rl" - {MARK(query_start, p); } -#line 63 "http11_parser.rl" - { - if(parser->query_string != NULL) - parser->query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -tr57: -#line 63 "http11_parser.rl" - { - if(parser->query_string != NULL) - parser->query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st5; -st5: - if ( ++p == pe ) - goto _test_eof5; -case 5: -#line 232 "http11_parser.c" - if ( (*p) == 72 ) - goto tr10; - goto st0; -tr10: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st6; -st6: - if ( ++p == pe ) - goto _test_eof6; -case 6: -#line 244 "http11_parser.c" - if ( (*p) == 84 ) - goto st7; - goto st0; -st7: - if ( ++p == pe ) - goto _test_eof7; -case 7: - if ( (*p) == 84 ) - goto st8; - goto st0; -st8: - if ( ++p == pe ) - goto _test_eof8; -case 8: - if ( (*p) == 80 ) - goto st9; - goto st0; -st9: - if ( ++p == pe ) - goto _test_eof9; -case 9: - if ( (*p) == 47 ) - goto st10; - goto st0; -st10: - if ( ++p == pe ) - goto _test_eof10; -case 10: - if ( 48 <= (*p) && (*p) <= 57 ) - goto st11; - goto st0; -st11: - if ( ++p == pe ) - goto _test_eof11; -case 11: - if ( (*p) == 46 ) - goto st12; - if ( 48 <= (*p) && (*p) <= 57 ) - goto st11; - goto st0; -st12: - if ( ++p == pe ) - goto _test_eof12; -case 12: - if ( 48 <= (*p) && (*p) <= 57 ) - goto st13; - goto st0; -st13: - if ( ++p == pe ) - goto _test_eof13; -case 13: - if ( (*p) == 13 ) - goto tr18; - if ( 48 <= (*p) && (*p) <= 57 ) - goto st13; - goto st0; -tr18: -#line 68 "http11_parser.rl" - { - if(parser->http_version != NULL) - parser->http_version(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st14; -tr26: -#line 43 "http11_parser.rl" - { MARK(mark, p); } -#line 44 "http11_parser.rl" - { - if(parser->http_field != NULL) { - parser->http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, p)); - } - } - goto st14; -tr29: -#line 44 "http11_parser.rl" - { - if(parser->http_field != NULL) { - parser->http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, p)); - } - } - goto st14; -st14: - if ( ++p == pe ) - goto _test_eof14; -case 14: -#line 330 "http11_parser.c" - if ( (*p) == 10 ) - goto st15; - goto st0; -st15: - if ( ++p == pe ) - goto _test_eof15; -case 15: - switch( (*p) ) { - case 13: goto st16; - case 33: goto tr21; - case 124: goto tr21; - case 126: goto tr21; - } - if ( (*p) < 45 ) { - if ( (*p) > 39 ) { - if ( 42 <= (*p) && (*p) <= 43 ) - goto tr21; - } else if ( (*p) >= 35 ) - goto tr21; - } else if ( (*p) > 46 ) { - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto tr21; - } else if ( (*p) > 90 ) { - if ( 94 <= (*p) && (*p) <= 122 ) - goto tr21; - } else - goto tr21; - } else - goto tr21; - goto st0; -st16: - if ( ++p == pe ) - goto _test_eof16; -case 16: - if ( (*p) == 10 ) - goto tr22; - goto st0; -tr22: -#line 78 "http11_parser.rl" - { - parser->body_start = p - buffer + 1; - if(parser->header_done != NULL) - parser->header_done(parser->data, p + 1, pe - p - 1); - {p++; cs = 57; goto _out;} - } - goto st57; -st57: - if ( ++p == pe ) - goto _test_eof57; -case 57: -#line 382 "http11_parser.c" - goto st0; -tr21: -#line 37 "http11_parser.rl" - { MARK(field_start, p); } -#line 38 "http11_parser.rl" - { snake_upcase_char((char *)p); } - goto st17; -tr23: -#line 38 "http11_parser.rl" - { snake_upcase_char((char *)p); } - goto st17; -st17: - if ( ++p == pe ) - goto _test_eof17; -case 17: -#line 398 "http11_parser.c" - switch( (*p) ) { - case 33: goto tr23; - case 58: goto tr24; - case 124: goto tr23; - case 126: goto tr23; - } - if ( (*p) < 45 ) { - if ( (*p) > 39 ) { - if ( 42 <= (*p) && (*p) <= 43 ) - goto tr23; - } else if ( (*p) >= 35 ) - goto tr23; - } else if ( (*p) > 46 ) { - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto tr23; - } else if ( (*p) > 90 ) { - if ( 94 <= (*p) && (*p) <= 122 ) - goto tr23; - } else - goto tr23; - } else - goto tr23; - goto st0; -tr24: -#line 39 "http11_parser.rl" - { - parser->field_len = LEN(field_start, p); - } - goto st18; -tr27: -#line 43 "http11_parser.rl" - { MARK(mark, p); } - goto st18; -st18: - if ( ++p == pe ) - goto _test_eof18; -case 18: -#line 437 "http11_parser.c" - switch( (*p) ) { - case 13: goto tr26; - case 32: goto tr27; - } - goto tr25; -tr25: -#line 43 "http11_parser.rl" - { MARK(mark, p); } - goto st19; -st19: - if ( ++p == pe ) - goto _test_eof19; -case 19: -#line 451 "http11_parser.c" - if ( (*p) == 13 ) - goto tr29; - goto st19; -tr9: -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st20; -tr43: -#line 73 "http11_parser.rl" - { - if(parser->request_path != NULL) - parser->request_path(parser->data, PTR_TO(mark), LEN(mark,p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st20; -tr54: -#line 62 "http11_parser.rl" - {MARK(query_start, p); } -#line 63 "http11_parser.rl" - { - if(parser->query_string != NULL) - parser->query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st20; -tr58: -#line 63 "http11_parser.rl" - { - if(parser->query_string != NULL) - parser->query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); - } -#line 53 "http11_parser.rl" - { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); - } - goto st20; -st20: - if ( ++p == pe ) - goto _test_eof20; -case 20: -#line 504 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr31; - case 35: goto st0; - case 37: goto tr32; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto tr30; -tr30: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st21; -st21: - if ( ++p == pe ) - goto _test_eof21; -case 21: -#line 522 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr34; - case 35: goto st0; - case 37: goto st22; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto st21; -tr32: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st22; -st22: - if ( ++p == pe ) - goto _test_eof22; -case 22: -#line 540 "http11_parser.c" - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st23; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st23; - } else - goto st23; - goto st0; -st23: - if ( ++p == pe ) - goto _test_eof23; -case 23: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st21; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st21; - } else - goto st21; - goto st0; -tr5: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st24; -st24: - if ( ++p == pe ) - goto _test_eof24; -case 24: -#line 571 "http11_parser.c" - switch( (*p) ) { - case 43: goto st24; - case 58: goto st25; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st24; - } else if ( (*p) > 57 ) { - if ( (*p) > 90 ) { - if ( 97 <= (*p) && (*p) <= 122 ) - goto st24; - } else if ( (*p) >= 65 ) - goto st24; - } else - goto st24; - goto st0; -tr7: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st25; -st25: - if ( ++p == pe ) - goto _test_eof25; -case 25: -#line 596 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr8; - case 35: goto tr9; - case 37: goto st26; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto st25; -st26: - if ( ++p == pe ) - goto _test_eof26; -case 26: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st27; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st27; - } else - goto st27; - goto st0; -st27: - if ( ++p == pe ) - goto _test_eof27; -case 27: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st25; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st25; - } else - goto st25; - goto st0; -tr6: -#line 34 "http11_parser.rl" - {MARK(mark, p); } - goto st28; -st28: - if ( ++p == pe ) - goto _test_eof28; -case 28: -#line 640 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr42; - case 35: goto tr43; - case 37: goto st29; - case 59: goto tr45; - case 63: goto tr46; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto st28; -st29: - if ( ++p == pe ) - goto _test_eof29; -case 29: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st30; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st30; - } else - goto st30; - goto st0; -st30: - if ( ++p == pe ) - goto _test_eof30; -case 30: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st28; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st28; - } else - goto st28; - goto st0; -tr45: -#line 73 "http11_parser.rl" - { - if(parser->request_path != NULL) - parser->request_path(parser->data, PTR_TO(mark), LEN(mark,p)); - } - goto st31; -st31: - if ( ++p == pe ) - goto _test_eof31; -case 31: -#line 689 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr8; - case 35: goto tr9; - case 37: goto st32; - case 63: goto st34; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto st31; -st32: - if ( ++p == pe ) - goto _test_eof32; -case 32: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st33; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st33; - } else - goto st33; - goto st0; -st33: - if ( ++p == pe ) - goto _test_eof33; -case 33: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st31; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st31; - } else - goto st31; - goto st0; -tr46: -#line 73 "http11_parser.rl" - { - if(parser->request_path != NULL) - parser->request_path(parser->data, PTR_TO(mark), LEN(mark,p)); - } - goto st34; -st34: - if ( ++p == pe ) - goto _test_eof34; -case 34: -#line 737 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr53; - case 35: goto tr54; - case 37: goto tr55; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto tr52; -tr52: -#line 62 "http11_parser.rl" - {MARK(query_start, p); } - goto st35; -st35: - if ( ++p == pe ) - goto _test_eof35; -case 35: -#line 755 "http11_parser.c" - switch( (*p) ) { - case 32: goto tr57; - case 35: goto tr58; - case 37: goto st36; - case 127: goto st0; - } - if ( 0 <= (*p) && (*p) <= 31 ) - goto st0; - goto st35; -tr55: -#line 62 "http11_parser.rl" - {MARK(query_start, p); } - goto st36; -st36: - if ( ++p == pe ) - goto _test_eof36; -case 36: -#line 773 "http11_parser.c" - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st37; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st37; - } else - goto st37; - goto st0; -st37: - if ( ++p == pe ) - goto _test_eof37; -case 37: - if ( (*p) < 65 ) { - if ( 48 <= (*p) && (*p) <= 57 ) - goto st35; - } else if ( (*p) > 70 ) { - if ( 97 <= (*p) && (*p) <= 102 ) - goto st35; - } else - goto st35; - goto st0; -st38: - if ( ++p == pe ) - goto _test_eof38; -case 38: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st39; - case 95: goto st39; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st39; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st39; - } else - goto st39; - goto st0; -st39: - if ( ++p == pe ) - goto _test_eof39; -case 39: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st40; - case 95: goto st40; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st40; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st40; - } else - goto st40; - goto st0; -st40: - if ( ++p == pe ) - goto _test_eof40; -case 40: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st41; - case 95: goto st41; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st41; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st41; - } else - goto st41; - goto st0; -st41: - if ( ++p == pe ) - goto _test_eof41; -case 41: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st42; - case 95: goto st42; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st42; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st42; - } else - goto st42; - goto st0; -st42: - if ( ++p == pe ) - goto _test_eof42; -case 42: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st43; - case 95: goto st43; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st43; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st43; - } else - goto st43; - goto st0; -st43: - if ( ++p == pe ) - goto _test_eof43; -case 43: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st44; - case 95: goto st44; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st44; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st44; - } else - goto st44; - goto st0; -st44: - if ( ++p == pe ) - goto _test_eof44; -case 44: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st45; - case 95: goto st45; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st45; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st45; - } else - goto st45; - goto st0; -st45: - if ( ++p == pe ) - goto _test_eof45; -case 45: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st46; - case 95: goto st46; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st46; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st46; - } else - goto st46; - goto st0; -st46: - if ( ++p == pe ) - goto _test_eof46; -case 46: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st47; - case 95: goto st47; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st47; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st47; - } else - goto st47; - goto st0; -st47: - if ( ++p == pe ) - goto _test_eof47; -case 47: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st48; - case 95: goto st48; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st48; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st48; - } else - goto st48; - goto st0; -st48: - if ( ++p == pe ) - goto _test_eof48; -case 48: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st49; - case 95: goto st49; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st49; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st49; - } else - goto st49; - goto st0; -st49: - if ( ++p == pe ) - goto _test_eof49; -case 49: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st50; - case 95: goto st50; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st50; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st50; - } else - goto st50; - goto st0; -st50: - if ( ++p == pe ) - goto _test_eof50; -case 50: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st51; - case 95: goto st51; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st51; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st51; - } else - goto st51; - goto st0; -st51: - if ( ++p == pe ) - goto _test_eof51; -case 51: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st52; - case 95: goto st52; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st52; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st52; - } else - goto st52; - goto st0; -st52: - if ( ++p == pe ) - goto _test_eof52; -case 52: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st53; - case 95: goto st53; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st53; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st53; - } else - goto st53; - goto st0; -st53: - if ( ++p == pe ) - goto _test_eof53; -case 53: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st54; - case 95: goto st54; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st54; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st54; - } else - goto st54; - goto st0; -st54: - if ( ++p == pe ) - goto _test_eof54; -case 54: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st55; - case 95: goto st55; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st55; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st55; - } else - goto st55; - goto st0; -st55: - if ( ++p == pe ) - goto _test_eof55; -case 55: - switch( (*p) ) { - case 32: goto tr2; - case 36: goto st56; - case 95: goto st56; - } - if ( (*p) < 48 ) { - if ( 45 <= (*p) && (*p) <= 46 ) - goto st56; - } else if ( (*p) > 57 ) { - if ( 65 <= (*p) && (*p) <= 90 ) - goto st56; - } else - goto st56; - goto st0; -st56: - if ( ++p == pe ) - goto _test_eof56; -case 56: - if ( (*p) == 32 ) - goto tr2; - goto st0; - } - _test_eof2: cs = 2; goto _test_eof; - _test_eof3: cs = 3; goto _test_eof; - _test_eof4: cs = 4; goto _test_eof; - _test_eof5: cs = 5; goto _test_eof; - _test_eof6: cs = 6; goto _test_eof; - _test_eof7: cs = 7; goto _test_eof; - _test_eof8: cs = 8; goto _test_eof; - _test_eof9: cs = 9; goto _test_eof; - _test_eof10: cs = 10; goto _test_eof; - _test_eof11: cs = 11; goto _test_eof; - _test_eof12: cs = 12; goto _test_eof; - _test_eof13: cs = 13; goto _test_eof; - _test_eof14: cs = 14; goto _test_eof; - _test_eof15: cs = 15; goto _test_eof; - _test_eof16: cs = 16; goto _test_eof; - _test_eof57: cs = 57; goto _test_eof; - _test_eof17: cs = 17; goto _test_eof; - _test_eof18: cs = 18; goto _test_eof; - _test_eof19: cs = 19; goto _test_eof; - _test_eof20: cs = 20; goto _test_eof; - _test_eof21: cs = 21; goto _test_eof; - _test_eof22: cs = 22; goto _test_eof; - _test_eof23: cs = 23; goto _test_eof; - _test_eof24: cs = 24; goto _test_eof; - _test_eof25: cs = 25; goto _test_eof; - _test_eof26: cs = 26; goto _test_eof; - _test_eof27: cs = 27; goto _test_eof; - _test_eof28: cs = 28; goto _test_eof; - _test_eof29: cs = 29; goto _test_eof; - _test_eof30: cs = 30; goto _test_eof; - _test_eof31: cs = 31; goto _test_eof; - _test_eof32: cs = 32; goto _test_eof; - _test_eof33: cs = 33; goto _test_eof; - _test_eof34: cs = 34; goto _test_eof; - _test_eof35: cs = 35; goto _test_eof; - _test_eof36: cs = 36; goto _test_eof; - _test_eof37: cs = 37; goto _test_eof; - _test_eof38: cs = 38; goto _test_eof; - _test_eof39: cs = 39; goto _test_eof; - _test_eof40: cs = 40; goto _test_eof; - _test_eof41: cs = 41; goto _test_eof; - _test_eof42: cs = 42; goto _test_eof; - _test_eof43: cs = 43; goto _test_eof; - _test_eof44: cs = 44; goto _test_eof; - _test_eof45: cs = 45; goto _test_eof; - _test_eof46: cs = 46; goto _test_eof; - _test_eof47: cs = 47; goto _test_eof; - _test_eof48: cs = 48; goto _test_eof; - _test_eof49: cs = 49; goto _test_eof; - _test_eof50: cs = 50; goto _test_eof; - _test_eof51: cs = 51; goto _test_eof; - _test_eof52: cs = 52; goto _test_eof; - _test_eof53: cs = 53; goto _test_eof; - _test_eof54: cs = 54; goto _test_eof; - _test_eof55: cs = 55; goto _test_eof; - _test_eof56: cs = 56; goto _test_eof; - - _test_eof: {} - _out: {} - } -#line 121 "http11_parser.rl" - - if (!http_parser_has_error(parser)) - parser->cs = cs; - parser->nread += p - (buffer + off); - - assert(p <= pe && "buffer overflow after parsing execute"); - assert(parser->nread <= len && "nread longer than length"); - assert(parser->body_start <= len && "body starts after buffer end"); - assert(parser->mark < len && "mark is after buffer end"); - assert(parser->field_len <= len && "field has length longer than whole buffer"); - assert(parser->field_start < len && "field starts after buffer end"); - - return(parser->nread); -} - -int http_parser_finish(http_parser *parser) -{ - if (http_parser_has_error(parser) ) { - return -1; - } else if (http_parser_is_finished(parser) ) { - return 1; - } else { - return 0; - } -} - -int http_parser_has_error(http_parser *parser) { - return parser->cs == http_parser_error; -} - -int http_parser_is_finished(http_parser *parser) { - return parser->cs == http_parser_first_final; -} diff --git a/ext/unicorn/http11/http11_parser.h b/ext/unicorn/http11/http11_parser.h index c96b3a0..8d95c59 100644 --- a/ext/unicorn/http11/http11_parser.h +++ b/ext/unicorn/http11/http11_parser.h @@ -1,20 +1,29 @@ + +#line 1 "http11_parser.rl" /** * Copyright (c) 2005 Zed A. Shaw * You can redistribute it and/or modify it under the same terms as Ruby. */ - #ifndef http11_parser_h #define http11_parser_h #include <sys/types.h> -typedef void (*element_cb)(void *data, const char *at, size_t length); -typedef void (*field_cb)(void *data, const char *field, size_t flen, const char *value, size_t vlen); +static void http_field(void *data, const char *field, + size_t flen, const char *value, size_t vlen); +static void request_method(void *data, const char *at, size_t length); +static void scheme(void *data, const char *at, size_t length); +static void host(void *data, const char *at, size_t length); +static void request_uri(void *data, const char *at, size_t length); +static void fragment(void *data, const char *at, size_t length); +static void request_path(void *data, const char *at, size_t length); +static void query_string(void *data, const char *at, size_t length); +static void http_version(void *data, const char *at, size_t length); +static void header_done(void *data, const char *at, size_t length); -typedef struct http_parser { +typedef struct http_parser { int cs; size_t body_start; - int content_len; size_t nread; size_t mark; size_t field_start; @@ -22,24 +31,1259 @@ typedef struct http_parser { size_t query_start; void *data; - - field_cb http_field; - element_cb request_method; - element_cb request_uri; - element_cb fragment; - element_cb request_path; - element_cb query_string; - element_cb http_version; - element_cb header_done; - } http_parser; -int http_parser_init(http_parser *parser); -int http_parser_finish(http_parser *parser); -size_t http_parser_execute(http_parser *parser, const char *data, size_t len, size_t off); -int http_parser_has_error(http_parser *parser); -int http_parser_is_finished(http_parser *parser); +static int http_parser_has_error(http_parser *parser); +static int http_parser_is_finished(http_parser *parser); + +/* + * capitalizes all lower-case ASCII characters, + * converts dashes to underscores. + */ +static void snake_upcase_char(char *c) +{ + if (*c >= 'a' && *c <= 'z') + *c &= ~0x20; + else if (*c == '-') + *c = '_'; +} + +static void downcase_char(char *c) +{ + if (*c >= 'A' && *c <= 'Z') + *c |= 0x20; +} + +#define LEN(AT, FPC) (FPC - buffer - parser->AT) +#define MARK(M,FPC) (parser->M = (FPC) - buffer) +#define PTR_TO(F) (buffer + parser->F) + +/** Machine **/ + + +#line 109 "http11_parser.rl" + + +/** Data **/ + +#line 70 "http11_parser.h" +static const int http_parser_start = 1; +static const int http_parser_first_final = 63; +static const int http_parser_error = 0; + +static const int http_parser_en_main = 1; + + +#line 113 "http11_parser.rl" + +static void http_parser_init(http_parser *parser) { + int cs = 0; + memset(parser, 0, sizeof(*parser)); + +#line 84 "http11_parser.h" + { + cs = http_parser_start; + } + +#line 118 "http11_parser.rl" + parser->cs = cs; +} + +/** exec **/ +static void http_parser_execute( + http_parser *parser, const char *buffer, size_t len) +{ + const char *p, *pe; + int cs = parser->cs; + size_t off = parser->nread; + + assert(off <= len && "offset past end of buffer"); + + p = buffer+off; + pe = buffer+len; + + assert(*pe == '\0' && "pointer does not end on NUL"); + assert(pe - p == len - off && "pointers aren't same distance"); + + +#line 110 "http11_parser.h" + { + if ( p == pe ) + goto _test_eof; + switch ( cs ) + { +case 1: + switch( (*p) ) { + case 36: goto tr0; + case 95: goto tr0; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto tr0; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto tr0; + } else + goto tr0; + goto st0; +st0: +cs = 0; + goto _out; +tr0: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st2; +st2: + if ( ++p == pe ) + goto _test_eof2; +case 2: +#line 141 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st44; + case 95: goto st44; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st44; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st44; + } else + goto st44; + goto st0; +tr2: +#line 77 "http11_parser.rl" + { + request_method(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st3; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: +#line 166 "http11_parser.h" + switch( (*p) ) { + case 42: goto tr4; + case 47: goto tr5; + case 72: goto tr6; + case 104: goto tr6; + } + goto st0; +tr4: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st4; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: +#line 182 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr7; + case 35: goto tr8; + } + goto st0; +tr7: +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +tr30: +#line 64 "http11_parser.rl" + {MARK(mark, p); } +#line 85 "http11_parser.rl" + { + fragment(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +tr33: +#line 85 "http11_parser.rl" + { + fragment(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +tr37: +#line 98 "http11_parser.rl" + { + request_path(parser->data, PTR_TO(mark), LEN(mark,p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +tr48: +#line 89 "http11_parser.rl" + {MARK(query_start, p); } +#line 90 "http11_parser.rl" + { + query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +tr52: +#line 90 "http11_parser.rl" + { + query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st5; +st5: + if ( ++p == pe ) + goto _test_eof5; +case 5: +#line 244 "http11_parser.h" + if ( (*p) == 72 ) + goto tr9; + goto st0; +tr9: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st6; +st6: + if ( ++p == pe ) + goto _test_eof6; +case 6: +#line 256 "http11_parser.h" + if ( (*p) == 84 ) + goto st7; + goto st0; +st7: + if ( ++p == pe ) + goto _test_eof7; +case 7: + if ( (*p) == 84 ) + goto st8; + goto st0; +st8: + if ( ++p == pe ) + goto _test_eof8; +case 8: + if ( (*p) == 80 ) + goto st9; + goto st0; +st9: + if ( ++p == pe ) + goto _test_eof9; +case 9: + if ( (*p) == 47 ) + goto st10; + goto st0; +st10: + if ( ++p == pe ) + goto _test_eof10; +case 10: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st11; + goto st0; +st11: + if ( ++p == pe ) + goto _test_eof11; +case 11: + if ( (*p) == 46 ) + goto st12; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st11; + goto st0; +st12: + if ( ++p == pe ) + goto _test_eof12; +case 12: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st13; + goto st0; +st13: + if ( ++p == pe ) + goto _test_eof13; +case 13: + if ( (*p) == 13 ) + goto tr17; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st13; + goto st0; +tr17: +#line 94 "http11_parser.rl" + { + http_version(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st14; +tr25: +#line 73 "http11_parser.rl" + { MARK(mark, p); } +#line 74 "http11_parser.rl" + { + http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, p)); + } + goto st14; +tr28: +#line 74 "http11_parser.rl" + { + http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, p)); + } + goto st14; +st14: + if ( ++p == pe ) + goto _test_eof14; +case 14: +#line 337 "http11_parser.h" + if ( (*p) == 10 ) + goto st15; + goto st0; +st15: + if ( ++p == pe ) + goto _test_eof15; +case 15: + switch( (*p) ) { + case 13: goto st16; + case 33: goto tr20; + case 124: goto tr20; + case 126: goto tr20; + } + if ( (*p) < 45 ) { + if ( (*p) > 39 ) { + if ( 42 <= (*p) && (*p) <= 43 ) + goto tr20; + } else if ( (*p) >= 35 ) + goto tr20; + } else if ( (*p) > 46 ) { + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr20; + } else if ( (*p) > 90 ) { + if ( 94 <= (*p) && (*p) <= 122 ) + goto tr20; + } else + goto tr20; + } else + goto tr20; + goto st0; +st16: + if ( ++p == pe ) + goto _test_eof16; +case 16: + if ( (*p) == 10 ) + goto tr21; + goto st0; +tr21: +#line 102 "http11_parser.rl" + { + parser->body_start = p - buffer + 1; + header_done(parser->data, p + 1, pe - p - 1); + {p++; cs = 63; goto _out;} + } + goto st63; +st63: + if ( ++p == pe ) + goto _test_eof63; +case 63: +#line 388 "http11_parser.h" + goto st0; +tr20: +#line 66 "http11_parser.rl" + { MARK(field_start, p); } +#line 67 "http11_parser.rl" + { snake_upcase_char((char *)p); } + goto st17; +tr22: +#line 67 "http11_parser.rl" + { snake_upcase_char((char *)p); } + goto st17; +st17: + if ( ++p == pe ) + goto _test_eof17; +case 17: +#line 404 "http11_parser.h" + switch( (*p) ) { + case 33: goto tr22; + case 58: goto tr23; + case 124: goto tr22; + case 126: goto tr22; + } + if ( (*p) < 45 ) { + if ( (*p) > 39 ) { + if ( 42 <= (*p) && (*p) <= 43 ) + goto tr22; + } else if ( (*p) >= 35 ) + goto tr22; + } else if ( (*p) > 46 ) { + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr22; + } else if ( (*p) > 90 ) { + if ( 94 <= (*p) && (*p) <= 122 ) + goto tr22; + } else + goto tr22; + } else + goto tr22; + goto st0; +tr23: +#line 69 "http11_parser.rl" + { + parser->field_len = LEN(field_start, p); + } + goto st18; +tr26: +#line 73 "http11_parser.rl" + { MARK(mark, p); } + goto st18; +st18: + if ( ++p == pe ) + goto _test_eof18; +case 18: +#line 443 "http11_parser.h" + switch( (*p) ) { + case 13: goto tr25; + case 32: goto tr26; + } + goto tr24; +tr24: +#line 73 "http11_parser.rl" + { MARK(mark, p); } + goto st19; +st19: + if ( ++p == pe ) + goto _test_eof19; +case 19: +#line 457 "http11_parser.h" + if ( (*p) == 13 ) + goto tr28; + goto st19; +tr8: +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st20; +tr38: +#line 98 "http11_parser.rl" + { + request_path(parser->data, PTR_TO(mark), LEN(mark,p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st20; +tr49: +#line 89 "http11_parser.rl" + {MARK(query_start, p); } +#line 90 "http11_parser.rl" + { + query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st20; +tr53: +#line 90 "http11_parser.rl" + { + query_string(parser->data, PTR_TO(query_start), LEN(query_start, p)); + } +#line 82 "http11_parser.rl" + { + request_uri(parser->data, PTR_TO(mark), LEN(mark, p)); + } + goto st20; +st20: + if ( ++p == pe ) + goto _test_eof20; +case 20: +#line 503 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr30; + case 35: goto st0; + case 37: goto tr31; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto tr29; +tr29: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st21; +st21: + if ( ++p == pe ) + goto _test_eof21; +case 21: +#line 521 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr33; + case 35: goto st0; + case 37: goto st22; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto st21; +tr31: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st22; +st22: + if ( ++p == pe ) + goto _test_eof22; +case 22: +#line 539 "http11_parser.h" + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st23; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st23; + } else + goto st23; + goto st0; +st23: + if ( ++p == pe ) + goto _test_eof23; +case 23: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st21; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st21; + } else + goto st21; + goto st0; +tr5: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st24; +tr65: +#line 81 "http11_parser.rl" + { host(parser->data, PTR_TO(mark), LEN(mark, p)); } +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st24; +st24: + if ( ++p == pe ) + goto _test_eof24; +case 24: +#line 576 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr37; + case 35: goto tr38; + case 37: goto st25; + case 59: goto tr40; + case 63: goto tr41; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto st24; +st25: + if ( ++p == pe ) + goto _test_eof25; +case 25: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st26; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st26; + } else + goto st26; + goto st0; +st26: + if ( ++p == pe ) + goto _test_eof26; +case 26: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st24; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st24; + } else + goto st24; + goto st0; +tr40: +#line 98 "http11_parser.rl" + { + request_path(parser->data, PTR_TO(mark), LEN(mark,p)); + } + goto st27; +st27: + if ( ++p == pe ) + goto _test_eof27; +case 27: +#line 624 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr7; + case 35: goto tr8; + case 37: goto st28; + case 63: goto st30; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto st27; +st28: + if ( ++p == pe ) + goto _test_eof28; +case 28: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st29; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st29; + } else + goto st29; + goto st0; +st29: + if ( ++p == pe ) + goto _test_eof29; +case 29: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st27; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st27; + } else + goto st27; + goto st0; +tr41: +#line 98 "http11_parser.rl" + { + request_path(parser->data, PTR_TO(mark), LEN(mark,p)); + } + goto st30; +st30: + if ( ++p == pe ) + goto _test_eof30; +case 30: +#line 671 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr48; + case 35: goto tr49; + case 37: goto tr50; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto tr47; +tr47: +#line 89 "http11_parser.rl" + {MARK(query_start, p); } + goto st31; +st31: + if ( ++p == pe ) + goto _test_eof31; +case 31: +#line 689 "http11_parser.h" + switch( (*p) ) { + case 32: goto tr52; + case 35: goto tr53; + case 37: goto st32; + case 127: goto st0; + } + if ( 0 <= (*p) && (*p) <= 31 ) + goto st0; + goto st31; +tr50: +#line 89 "http11_parser.rl" + {MARK(query_start, p); } + goto st32; +st32: + if ( ++p == pe ) + goto _test_eof32; +case 32: +#line 707 "http11_parser.h" + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st33; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st33; + } else + goto st33; + goto st0; +st33: + if ( ++p == pe ) + goto _test_eof33; +case 33: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st31; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st31; + } else + goto st31; + goto st0; +tr6: +#line 64 "http11_parser.rl" + {MARK(mark, p); } +#line 68 "http11_parser.rl" + { downcase_char((char *)p); } + goto st34; +st34: + if ( ++p == pe ) + goto _test_eof34; +case 34: +#line 740 "http11_parser.h" + switch( (*p) ) { + case 84: goto tr56; + case 116: goto tr56; + } + goto st0; +tr56: +#line 68 "http11_parser.rl" + { downcase_char((char *)p); } + goto st35; +st35: + if ( ++p == pe ) + goto _test_eof35; +case 35: +#line 754 "http11_parser.h" + switch( (*p) ) { + case 84: goto tr57; + case 116: goto tr57; + } + goto st0; +tr57: +#line 68 "http11_parser.rl" + { downcase_char((char *)p); } + goto st36; +st36: + if ( ++p == pe ) + goto _test_eof36; +case 36: +#line 768 "http11_parser.h" + switch( (*p) ) { + case 80: goto tr58; + case 112: goto tr58; + } + goto st0; +tr58: +#line 68 "http11_parser.rl" + { downcase_char((char *)p); } + goto st37; +st37: + if ( ++p == pe ) + goto _test_eof37; +case 37: +#line 782 "http11_parser.h" + switch( (*p) ) { + case 58: goto tr59; + case 83: goto tr60; + case 115: goto tr60; + } + goto st0; +tr59: +#line 80 "http11_parser.rl" + { scheme(parser->data, PTR_TO(mark), LEN(mark, p)); } + goto st38; +st38: + if ( ++p == pe ) + goto _test_eof38; +case 38: +#line 797 "http11_parser.h" + if ( (*p) == 47 ) + goto st39; + goto st0; +st39: + if ( ++p == pe ) + goto _test_eof39; +case 39: + if ( (*p) == 47 ) + goto st40; + goto st0; +st40: + if ( ++p == pe ) + goto _test_eof40; +case 40: + if ( (*p) == 95 ) + goto tr63; + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto tr63; + } else if ( (*p) > 57 ) { + if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr63; + } else if ( (*p) >= 65 ) + goto tr63; + } else + goto tr63; + goto st0; +tr63: +#line 64 "http11_parser.rl" + {MARK(mark, p); } + goto st41; +st41: + if ( ++p == pe ) + goto _test_eof41; +case 41: +#line 834 "http11_parser.h" + switch( (*p) ) { + case 47: goto tr65; + case 58: goto st42; + case 95: goto st41; + } + if ( (*p) < 65 ) { + if ( 45 <= (*p) && (*p) <= 57 ) + goto st41; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st41; + } else + goto st41; + goto st0; +st42: + if ( ++p == pe ) + goto _test_eof42; +case 42: + if ( (*p) == 47 ) + goto tr65; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st42; + goto st0; +tr60: +#line 68 "http11_parser.rl" + { downcase_char((char *)p); } + goto st43; +st43: + if ( ++p == pe ) + goto _test_eof43; +case 43: +#line 866 "http11_parser.h" + if ( (*p) == 58 ) + goto tr59; + goto st0; +st44: + if ( ++p == pe ) + goto _test_eof44; +case 44: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st45; + case 95: goto st45; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st45; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st45; + } else + goto st45; + goto st0; +st45: + if ( ++p == pe ) + goto _test_eof45; +case 45: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st46; + case 95: goto st46; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st46; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st46; + } else + goto st46; + goto st0; +st46: + if ( ++p == pe ) + goto _test_eof46; +case 46: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st47; + case 95: goto st47; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st47; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st47; + } else + goto st47; + goto st0; +st47: + if ( ++p == pe ) + goto _test_eof47; +case 47: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st48; + case 95: goto st48; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st48; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st48; + } else + goto st48; + goto st0; +st48: + if ( ++p == pe ) + goto _test_eof48; +case 48: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st49; + case 95: goto st49; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st49; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st49; + } else + goto st49; + goto st0; +st49: + if ( ++p == pe ) + goto _test_eof49; +case 49: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st50; + case 95: goto st50; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st50; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st50; + } else + goto st50; + goto st0; +st50: + if ( ++p == pe ) + goto _test_eof50; +case 50: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st51; + case 95: goto st51; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st51; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st51; + } else + goto st51; + goto st0; +st51: + if ( ++p == pe ) + goto _test_eof51; +case 51: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st52; + case 95: goto st52; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st52; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st52; + } else + goto st52; + goto st0; +st52: + if ( ++p == pe ) + goto _test_eof52; +case 52: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st53; + case 95: goto st53; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st53; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st53; + } else + goto st53; + goto st0; +st53: + if ( ++p == pe ) + goto _test_eof53; +case 53: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st54; + case 95: goto st54; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st54; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st54; + } else + goto st54; + goto st0; +st54: + if ( ++p == pe ) + goto _test_eof54; +case 54: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st55; + case 95: goto st55; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st55; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st55; + } else + goto st55; + goto st0; +st55: + if ( ++p == pe ) + goto _test_eof55; +case 55: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st56; + case 95: goto st56; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st56; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st56; + } else + goto st56; + goto st0; +st56: + if ( ++p == pe ) + goto _test_eof56; +case 56: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st57; + case 95: goto st57; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st57; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st57; + } else + goto st57; + goto st0; +st57: + if ( ++p == pe ) + goto _test_eof57; +case 57: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st58; + case 95: goto st58; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st58; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st58; + } else + goto st58; + goto st0; +st58: + if ( ++p == pe ) + goto _test_eof58; +case 58: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st59; + case 95: goto st59; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st59; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st59; + } else + goto st59; + goto st0; +st59: + if ( ++p == pe ) + goto _test_eof59; +case 59: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st60; + case 95: goto st60; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st60; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st60; + } else + goto st60; + goto st0; +st60: + if ( ++p == pe ) + goto _test_eof60; +case 60: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st61; + case 95: goto st61; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st61; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st61; + } else + goto st61; + goto st0; +st61: + if ( ++p == pe ) + goto _test_eof61; +case 61: + switch( (*p) ) { + case 32: goto tr2; + case 36: goto st62; + case 95: goto st62; + } + if ( (*p) < 48 ) { + if ( 45 <= (*p) && (*p) <= 46 ) + goto st62; + } else if ( (*p) > 57 ) { + if ( 65 <= (*p) && (*p) <= 90 ) + goto st62; + } else + goto st62; + goto st0; +st62: + if ( ++p == pe ) + goto _test_eof62; +case 62: + if ( (*p) == 32 ) + goto tr2; + goto st0; + } + _test_eof2: cs = 2; goto _test_eof; + _test_eof3: cs = 3; goto _test_eof; + _test_eof4: cs = 4; goto _test_eof; + _test_eof5: cs = 5; goto _test_eof; + _test_eof6: cs = 6; goto _test_eof; + _test_eof7: cs = 7; goto _test_eof; + _test_eof8: cs = 8; goto _test_eof; + _test_eof9: cs = 9; goto _test_eof; + _test_eof10: cs = 10; goto _test_eof; + _test_eof11: cs = 11; goto _test_eof; + _test_eof12: cs = 12; goto _test_eof; + _test_eof13: cs = 13; goto _test_eof; + _test_eof14: cs = 14; goto _test_eof; + _test_eof15: cs = 15; goto _test_eof; + _test_eof16: cs = 16; goto _test_eof; + _test_eof63: cs = 63; goto _test_eof; + _test_eof17: cs = 17; goto _test_eof; + _test_eof18: cs = 18; goto _test_eof; + _test_eof19: cs = 19; goto _test_eof; + _test_eof20: cs = 20; goto _test_eof; + _test_eof21: cs = 21; goto _test_eof; + _test_eof22: cs = 22; goto _test_eof; + _test_eof23: cs = 23; goto _test_eof; + _test_eof24: cs = 24; goto _test_eof; + _test_eof25: cs = 25; goto _test_eof; + _test_eof26: cs = 26; goto _test_eof; + _test_eof27: cs = 27; goto _test_eof; + _test_eof28: cs = 28; goto _test_eof; + _test_eof29: cs = 29; goto _test_eof; + _test_eof30: cs = 30; goto _test_eof; + _test_eof31: cs = 31; goto _test_eof; + _test_eof32: cs = 32; goto _test_eof; + _test_eof33: cs = 33; goto _test_eof; + _test_eof34: cs = 34; goto _test_eof; + _test_eof35: cs = 35; goto _test_eof; + _test_eof36: cs = 36; goto _test_eof; + _test_eof37: cs = 37; goto _test_eof; + _test_eof38: cs = 38; goto _test_eof; + _test_eof39: cs = 39; goto _test_eof; + _test_eof40: cs = 40; goto _test_eof; + _test_eof41: cs = 41; goto _test_eof; + _test_eof42: cs = 42; goto _test_eof; + _test_eof43: cs = 43; goto _test_eof; + _test_eof44: cs = 44; goto _test_eof; + _test_eof45: cs = 45; goto _test_eof; + _test_eof46: cs = 46; goto _test_eof; + _test_eof47: cs = 47; goto _test_eof; + _test_eof48: cs = 48; goto _test_eof; + _test_eof49: cs = 49; goto _test_eof; + _test_eof50: cs = 50; goto _test_eof; + _test_eof51: cs = 51; goto _test_eof; + _test_eof52: cs = 52; goto _test_eof; + _test_eof53: cs = 53; goto _test_eof; + _test_eof54: cs = 54; goto _test_eof; + _test_eof55: cs = 55; goto _test_eof; + _test_eof56: cs = 56; goto _test_eof; + _test_eof57: cs = 57; goto _test_eof; + _test_eof58: cs = 58; goto _test_eof; + _test_eof59: cs = 59; goto _test_eof; + _test_eof60: cs = 60; goto _test_eof; + _test_eof61: cs = 61; goto _test_eof; + _test_eof62: cs = 62; goto _test_eof; + + _test_eof: {} + _out: {} + } + +#line 138 "http11_parser.rl" + + if (!http_parser_has_error(parser)) + parser->cs = cs; + parser->nread += p - (buffer + off); + + assert(p <= pe && "buffer overflow after parsing execute"); + assert(parser->nread <= len && "nread longer than length"); + assert(parser->body_start <= len && "body starts after buffer end"); + assert(parser->mark < len && "mark is after buffer end"); + assert(parser->field_len <= len && "field has length longer than whole buffer"); + assert(parser->field_start < len && "field starts after buffer end"); +} -#define http_parser_nread(parser) (parser)->nread +static int http_parser_has_error(http_parser *parser) { + return parser->cs == http_parser_error; +} -#endif +static int http_parser_is_finished(http_parser *parser) { + return parser->cs == http_parser_first_final; +} +#endif /* http11_parser_h */ diff --git a/ext/unicorn/http11/http11_parser.rl b/ext/unicorn/http11/http11_parser.rl index c3c4b1f..9894276 100644 --- a/ext/unicorn/http11/http11_parser.rl +++ b/ext/unicorn/http11/http11_parser.rl @@ -2,12 +2,37 @@ * Copyright (c) 2005 Zed A. Shaw * You can redistribute it and/or modify it under the same terms as Ruby. */ -#include "http11_parser.h" -#include <stdio.h> -#include <assert.h> -#include <stdlib.h> -#include <ctype.h> -#include <string.h> +#ifndef http11_parser_h +#define http11_parser_h + +#include <sys/types.h> + +static void http_field(void *data, const char *field, + size_t flen, const char *value, size_t vlen); +static void request_method(void *data, const char *at, size_t length); +static void scheme(void *data, const char *at, size_t length); +static void host(void *data, const char *at, size_t length); +static void request_uri(void *data, const char *at, size_t length); +static void fragment(void *data, const char *at, size_t length); +static void request_path(void *data, const char *at, size_t length); +static void query_string(void *data, const char *at, size_t length); +static void http_version(void *data, const char *at, size_t length); +static void header_done(void *data, const char *at, size_t length); + +typedef struct http_parser { + int cs; + size_t body_start; + size_t nread; + size_t mark; + size_t field_start; + size_t field_len; + size_t query_start; + + void *data; +} http_parser; + +static int http_parser_has_error(http_parser *parser); +static int http_parser_is_finished(http_parser *parser); /* * capitalizes all lower-case ASCII characters, @@ -15,10 +40,16 @@ */ static void snake_upcase_char(char *c) { - if (*c >= 'a' && *c <= 'z') - *c &= ~0x20; - else if (*c == '-') - *c = '_'; + if (*c >= 'a' && *c <= 'z') + *c &= ~0x20; + else if (*c == '-') + *c = '_'; +} + +static void downcase_char(char *c) +{ + if (*c >= 'A' && *c <= 'Z') + *c |= 0x20; } #define LEN(AT, FPC) (FPC - buffer - parser->AT) @@ -28,86 +59,72 @@ static void snake_upcase_char(char *c) /** Machine **/ %%{ - machine http_parser; action mark {MARK(mark, fpc); } - action start_field { MARK(field_start, fpc); } action snake_upcase_field { snake_upcase_char((char *)fpc); } - action write_field { + action downcase_char { downcase_char((char *)fpc); } + action write_field { parser->field_len = LEN(field_start, fpc); } action start_value { MARK(mark, fpc); } - action write_value { - if(parser->http_field != NULL) { - parser->http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, fpc)); - } + action write_value { + http_field(parser->data, PTR_TO(field_start), parser->field_len, PTR_TO(mark), LEN(mark, fpc)); } - action request_method { - if(parser->request_method != NULL) - parser->request_method(parser->data, PTR_TO(mark), LEN(mark, fpc)); + action request_method { + request_method(parser->data, PTR_TO(mark), LEN(mark, fpc)); } - action request_uri { - if(parser->request_uri != NULL) - parser->request_uri(parser->data, PTR_TO(mark), LEN(mark, fpc)); + action scheme { scheme(parser->data, PTR_TO(mark), LEN(mark, fpc)); } + action host { host(parser->data, PTR_TO(mark), LEN(mark, fpc)); } + action request_uri { + request_uri(parser->data, PTR_TO(mark), LEN(mark, fpc)); } - action fragment { - if(parser->fragment != NULL) - parser->fragment(parser->data, PTR_TO(mark), LEN(mark, fpc)); + action fragment { + fragment(parser->data, PTR_TO(mark), LEN(mark, fpc)); } action start_query {MARK(query_start, fpc); } - action query_string { - if(parser->query_string != NULL) - parser->query_string(parser->data, PTR_TO(query_start), LEN(query_start, fpc)); + action query_string { + query_string(parser->data, PTR_TO(query_start), LEN(query_start, fpc)); } - action http_version { - if(parser->http_version != NULL) - parser->http_version(parser->data, PTR_TO(mark), LEN(mark, fpc)); + action http_version { + http_version(parser->data, PTR_TO(mark), LEN(mark, fpc)); } action request_path { - if(parser->request_path != NULL) - parser->request_path(parser->data, PTR_TO(mark), LEN(mark,fpc)); + request_path(parser->data, PTR_TO(mark), LEN(mark,fpc)); } - action done { - parser->body_start = fpc - buffer + 1; - if(parser->header_done != NULL) - parser->header_done(parser->data, fpc + 1, pe - fpc - 1); + action done { + parser->body_start = fpc - buffer + 1; + header_done(parser->data, fpc + 1, pe - fpc - 1); fbreak; } include http_parser_common "http11_parser_common.rl"; - }%% /** Data **/ %% write data; -int http_parser_init(http_parser *parser) { +static void http_parser_init(http_parser *parser) { int cs = 0; + memset(parser, 0, sizeof(*parser)); %% write init; parser->cs = cs; - parser->body_start = 0; - parser->content_len = 0; - parser->mark = 0; - parser->nread = 0; - parser->field_len = 0; - parser->field_start = 0; - - return(1); } - /** exec **/ -size_t http_parser_execute(http_parser *parser, const char *buffer, size_t len, size_t off) { +static void http_parser_execute( + http_parser *parser, const char *buffer, size_t len) +{ const char *p, *pe; int cs = parser->cs; + size_t off = parser->nread; assert(off <= len && "offset past end of buffer"); @@ -129,25 +146,13 @@ size_t http_parser_execute(http_parser *parser, const char *buffer, size_t len, assert(parser->mark < len && "mark is after buffer end"); assert(parser->field_len <= len && "field has length longer than whole buffer"); assert(parser->field_start < len && "field starts after buffer end"); - - return(parser->nread); -} - -int http_parser_finish(http_parser *parser) -{ - if (http_parser_has_error(parser) ) { - return -1; - } else if (http_parser_is_finished(parser) ) { - return 1; - } else { - return 0; - } } -int http_parser_has_error(http_parser *parser) { +static int http_parser_has_error(http_parser *parser) { return parser->cs == http_parser_error; } -int http_parser_is_finished(http_parser *parser) { +static int http_parser_is_finished(http_parser *parser) { return parser->cs == http_parser_first_final; } +#endif /* http11_parser_h */ diff --git a/ext/unicorn/http11/http11_parser_common.rl b/ext/unicorn/http11/http11_parser_common.rl index ee970b1..ae01a55 100644 --- a/ext/unicorn/http11/http11_parser_common.rl +++ b/ext/unicorn/http11/http11_parser_common.rl @@ -24,8 +24,9 @@ token = (ascii -- (CTL | tspecials)); # URI schemes and absolute paths - scheme = ( alpha | digit | "+" | "-" | "." )* ; - absolute_uri = (scheme ":" (uchar | reserved )*); + scheme = ( "http"i ("s"i)? ) $downcase_char >mark %scheme; + hostname = (alnum | "-" | "." | "_")+; + host_with_port = (hostname (":" digit*)?) >mark %host; path = ( pchar+ ( "/" pchar* )* ) ; query = ( uchar | reserved )* %query_string ; @@ -33,8 +34,10 @@ params = ( param ( ";" param )* ) ; rel_path = ( path? %request_path (";" params)? ) ("?" %start_query query)?; absolute_path = ( "/"+ rel_path ); + path_uri = absolute_path > mark %request_uri; + Absolute_URI = (scheme "://" host_with_port path_uri); - Request_URI = ( "*" | absolute_uri | absolute_path ) >mark %request_uri; + Request_URI = ((absolute_path | "*") >mark %request_uri) | Absolute_URI; Fragment = ( uchar | reserved )* >mark %fragment; Method = ( upper | digit | safe ){1,20} >mark %request_method; diff --git a/lib/unicorn.rb b/lib/unicorn.rb index d442f63..4a4e2e1 100644 --- a/lib/unicorn.rb +++ b/lib/unicorn.rb @@ -1,6 +1,6 @@ -require 'logger' +require 'fcntl' -require 'unicorn/socket' +require 'unicorn/socket_helper' require 'unicorn/const' require 'unicorn/http_request' require 'unicorn/http_response' @@ -23,18 +23,32 @@ module Unicorn # forked worker children. class HttpServer attr_reader :logger - include Process include ::Unicorn::SocketHelper - DEFAULT_START_CTX = { + # prevents IO objects in here from being GC-ed + IO_PURGATORY = [] + + # all bound listener sockets + LISTENERS = [] + + # This hash maps PIDs to Workers + WORKERS = {} + + # See: http://cr.yp.to/docs/selfpipe.html + SELF_PIPE = [] + + # signal queue used for self-piping + SIG_QUEUE = [] + + # We populate this at startup so we can figure out how to reexecute + # and upgrade the currently running instance of Unicorn + START_CTX = { :argv => ARGV.map { |arg| arg.dup }, # don't rely on Dir.pwd here since it's not symlink-aware, and # symlink dirs are the default with Capistrano... :cwd => `/bin/sh -c pwd`.chomp("\n"), :zero => $0.dup, - :environ => {}.merge!(ENV), - :umask => File.umask, - }.freeze + } Worker = Struct.new(:nr, :tempfile) unless defined?(Worker) class Worker @@ -46,22 +60,17 @@ module Unicorn # Creates a working server on host:port (strange things happen if # port isn't a Number). Use HttpServer::run to start the server and - # HttpServer.workers.join to join the thread that's processing + # HttpServer.run.join to join the thread that's processing # incoming requests on the socket. def initialize(app, options = {}) - start_ctx = options.delete(:start_ctx) - @start_ctx = DEFAULT_START_CTX.dup - @start_ctx.merge!(start_ctx) if start_ctx @app = app - @mode = :idle - @master_pid = $$ - @workers = Hash.new - @io_purgatory = [] # prevents IO objects in here from being GC-ed - @request = @rd_sig = @wr_sig = nil + @pid = nil @reexec_pid = 0 + @init_listeners = options[:listeners] ? options[:listeners].dup : [] @config = Configurator.new(options.merge(:use_defaults => true)) + @listener_opts = {} @config.commit!(self, :skip => [:listeners, :pid]) - @listeners = [] + @request = HttpRequest.new(@logger) end # Runs the thing. Returns self so you can run join on it @@ -72,44 +81,55 @@ module Unicorn # before they become UNIXServer or TCPServer inherited = ENV['UNICORN_FD'].to_s.split(/,/).map do |fd| io = Socket.for_fd(fd.to_i) - set_server_sockopt(io) - @io_purgatory << io - logger.info "inherited: #{io} fd=#{fd} addr=#{sock_name(io)}" + set_server_sockopt(io, @listener_opts[sock_name(io)]) + IO_PURGATORY << io + logger.info "inherited addr=#{sock_name(io)} fd=#{fd}" server_cast(io) end config_listeners = @config[:listeners].dup - @listeners.replace(inherited) + LISTENERS.replace(inherited) # we start out with generic Socket objects that get cast to either # TCPServer or UNIXServer objects; but since the Socket objects # share the same OS-level file descriptor as the higher-level *Server # objects; we need to prevent Socket objects from being garbage-collected config_listeners -= listener_names + if config_listeners.empty? && LISTENERS.empty? + config_listeners << Unicorn::Const::DEFAULT_LISTEN + end config_listeners.each { |addr| listen(addr) } - raise ArgumentError, "no listeners" if @listeners.empty? + raise ArgumentError, "no listeners" if LISTENERS.empty? self.pid = @config[:pid] build_app! if @preload_app - $stderr.reopen(File.open(@stderr_path, "a")) if @stderr_path - $stdout.reopen(File.open(@stdout_path, "a")) if @stdout_path - $stderr.sync = $stdout.sync = true - spawn_missing_workers + maintain_worker_count self end # replaces current listener set with +listeners+. This will # close the socket if it will not exist in the new listener set def listeners=(listeners) - cur_names = listener_names + cur_names, dead_names = [], [] + listener_names.each do |name| + if "/" == name[0..0] + # mark unlinked sockets as dead so we can rebind them + (File.socket?(name) ? cur_names : dead_names) << name + else + cur_names << name + end + end set_names = listener_names(listeners) - dead_names = cur_names - set_names + dead_names += cur_names - set_names + dead_names.uniq! - @listeners.delete_if do |io| + LISTENERS.delete_if do |io| if dead_names.include?(sock_name(io)) - @io_purgatory.delete_if { |pio| pio.fileno == io.fileno } - destroy_safely(io) - true + IO_PURGATORY.delete_if do |pio| + pio.fileno == io.fileno && (pio.close rescue nil).nil? # true + end + (io.close rescue nil).nil? # true else + set_server_sockopt(io, @listener_opts[sock_name(io)]) false end end @@ -117,6 +137,9 @@ module Unicorn (set_names - cur_names).each { |addr| listen(addr) } end + def stdout_path=(path); redirect_io($stdout, path); end + def stderr_path=(path); redirect_io($stderr, path); end + # sets the path for the PID file of the master process def pid=(path) if path @@ -125,26 +148,25 @@ module Unicorn raise ArgumentError, "Already running on PID:#{x} " \ "(or pid=#{path} is stale)" end - File.open(path, 'wb') { |fp| fp.syswrite("#{$$}\n") } end - unlink_pid_safe(@pid) if @pid && @pid != path + unlink_pid_safe(@pid) if @pid + File.open(path, 'wb') { |fp| fp.syswrite("#$$\n") } if path @pid = path end # add a given address to the +listeners+ set, idempotently # Allows workers to add a private, per-process listener via the # @after_fork hook. Very useful for debugging and testing. - def listen(address) + def listen(address, opt = {}.merge(@listener_opts[address] || {})) return if String === address && listener_names.include?(address) - if io = bind_listen(address, @backlog) - if Socket == io.class - @io_purgatory << io + if io = bind_listen(address, opt) + unless TCPServer === io || UNIXServer === io + IO_PURGATORY << io io = server_cast(io) end - logger.info "#{io} listening on PID:#{$$} " \ - "fd=#{io.fileno} addr=#{sock_name(io)}" - @listeners << io + logger.info "listening on addr=#{sock_name(io)} fd=#{io.fileno}" + LISTENERS << io else logger.error "adding listener failed addr=#{address} (in use)" raise Errno::EADDRINUSE, address @@ -158,55 +180,55 @@ module Unicorn def join # this pipe is used to wake us up from select(2) in #join when signals # are trapped. See trap_deferred - @rd_sig, @wr_sig = IO.pipe unless (@rd_sig && @wr_sig) - @rd_sig.nonblock = @wr_sig.nonblock = true + init_self_pipe! + respawn = true - reset_master - $0 = "unicorn master" - logger.info "master process ready" # test relies on this message + QUEUE_SIGS.each { |sig| trap_deferred(sig) } + trap(:CHLD) { |sig_nr| awaken_master } + proc_name 'master' + logger.info "master process ready" # test_exec.rb relies on this message begin loop do reap_all_workers - case @mode - when :idle + case SIG_QUEUE.shift + when nil murder_lazy_workers - spawn_missing_workers - when 'QUIT' # graceful shutdown + maintain_worker_count if respawn + master_sleep + when :QUIT # graceful shutdown break - when 'TERM', 'INT' # immediate shutdown + when :TERM, :INT # immediate shutdown stop(false) break - when 'USR1' # user-defined (probably something like log reopening) - kill_each_worker('USR1') + when :USR1 # rotate logs + logger.info "master reopening logs..." Unicorn::Util.reopen_logs - reset_master - when 'USR2' # exec binary, stay alive in case something went wrong + logger.info "master done reopening logs" + kill_each_worker(:USR1) + when :USR2 # exec binary, stay alive in case something went wrong reexec - reset_master - when 'HUP' + when :WINCH + if Process.ppid == 1 || Process.getpgrp != $$ + respawn = false + logger.info "gracefully stopping all workers" + kill_each_worker(:QUIT) + else + logger.info "SIGWINCH ignored because we're not daemonized" + end + when :TTIN + @worker_processes += 1 + when :TTOU + @worker_processes -= 1 if @worker_processes > 0 + when :HUP + respawn = true if @config.config_file load_config! - reset_master redo # immediate reaping since we may have QUIT workers else # exec binary and exit if there's no config file logger.info "config_file not present, reexecuting binary" reexec break end - else - logger.error "master process in unknown mode: #{@mode}, resetting" - reset_master - end - reap_all_workers - - ready = begin - IO.select([@rd_sig], nil, nil, 1) or next - rescue Errno::EINTR # next - end - ready[0] && ready[0][0] or next - begin # just consume the pipe when we're awakened, @mode is set - loop { @rd_sig.sysread(Const::CHUNK_SIZE) } - rescue Errno::EAGAIN, Errno::EINTR # next end end rescue Errno::EINTR @@ -214,25 +236,24 @@ module Unicorn rescue Object => e logger.error "Unhandled master loop exception #{e.inspect}." logger.error e.backtrace.join("\n") - reset_master retry end stop # gracefully shutdown all workers on our way out - logger.info "master PID:#{$$} join complete" + logger.info "master complete" unlink_pid_safe(@pid) if @pid end # Terminates all workers, but does not exit master process def stop(graceful = true) - kill_each_worker(graceful ? 'QUIT' : 'TERM') + kill_each_worker(graceful ? :QUIT : :TERM) timeleft = @timeout step = 0.2 reap_all_workers - until @workers.empty? + until WORKERS.empty? sleep(step) reap_all_workers (timeleft -= step) > 0 and next - kill_each_worker('KILL') + kill_each_worker(:KILL) end ensure self.listeners = [] @@ -241,55 +262,64 @@ module Unicorn private # list of signals we care about and trap in master. - TRAP_SIGS = %w(QUIT INT TERM USR1 USR2 HUP).map { |x| x.freeze }.freeze + QUEUE_SIGS = [ :WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, + :TTIN, :TTOU ].freeze # defer a signal for later processing in #join (master process) def trap_deferred(signal) trap(signal) do |sig_nr| - # we only handle/defer one signal at a time and ignore all others - # until we're ready again. Queueing signals can lead to more bugs, - # and simplicity is the most important thing - TRAP_SIGS.each { |sig| trap(sig, 'IGNORE') } - if Symbol === @mode - @mode = signal - begin - @wr_sig.syswrite('.') # wakeup master process from IO.select - rescue Errno::EAGAIN - rescue Errno::EINTR - retry - end + if SIG_QUEUE.size < 5 + SIG_QUEUE << signal + awaken_master + else + logger.error "ignoring SIG#{signal}, queue=#{SIG_QUEUE.inspect}" end end end + # wait for a signal hander to wake us up and then consume the pipe + # Wake up every second anyways to run murder_lazy_workers + def master_sleep + begin + ready = IO.select([SELF_PIPE.first], nil, nil, 1) or return + ready.first && ready.first.first or return + loop { SELF_PIPE.first.read_nonblock(Const::CHUNK_SIZE) } + rescue Errno::EAGAIN, Errno::EINTR + end + end - def reset_master - @mode = :idle - TRAP_SIGS.each { |sig| trap_deferred(sig) } + def awaken_master + begin + SELF_PIPE.last.write_nonblock('.') # wakeup master process from select + rescue Errno::EAGAIN, Errno::EINTR + # pipe is full, master should wake up anyways + retry + end end # reaps all unreaped workers def reap_all_workers begin loop do - pid = waitpid(-1, WNOHANG) or break + pid, status = Process.waitpid2(-1, Process::WNOHANG) + pid or break if @reexec_pid == pid - logger.error "reaped exec()-ed PID:#{pid} status=#{$?.exitstatus}" + logger.error "reaped #{status.inspect} exec()-ed" @reexec_pid = 0 self.pid = @pid.chomp('.oldbin') if @pid + proc_name 'master' else - worker = @workers.delete(pid) + worker = WORKERS.delete(pid) worker.tempfile.close rescue nil - logger.info "reaped PID:#{pid} " \ - "worker=#{worker.nr rescue 'unknown'} " \ - "status=#{$?.exitstatus}" + logger.info "reaped #{status.inspect} " \ + "worker=#{worker.nr rescue 'unknown'}" end end rescue Errno::ECHILD end end - # reexecutes the @start_ctx with a new binary + # reexecutes the START_CTX with a new binary def reexec if @reexec_pid > 0 begin @@ -317,19 +347,26 @@ module Unicorn end @reexec_pid = fork do - @rd_sig.close if @rd_sig - @wr_sig.close if @wr_sig - @workers.values.each { |other| other.tempfile.close rescue nil } - - ENV.replace(@start_ctx[:environ]) - ENV['UNICORN_FD'] = @listeners.map { |sock| sock.fileno }.join(',') - File.umask(@start_ctx[:umask]) - Dir.chdir(@start_ctx[:cwd]) - cmd = [ @start_ctx[:zero] ] + @start_ctx[:argv] + listener_fds = LISTENERS.map { |sock| sock.fileno } + ENV['UNICORN_FD'] = listener_fds.join(',') + Dir.chdir(START_CTX[:cwd]) + cmd = [ START_CTX[:zero] ] + START_CTX[:argv] + + # avoid leaking FDs we don't know about, but let before_exec + # unset FD_CLOEXEC, if anything else in the app eventually + # relies on FD inheritence. + (3..1024).each do |io| + next if listener_fds.include?(io) + io = IO.for_fd(io) rescue nil + io or next + IO_PURGATORY << io + io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) + end logger.info "executing #{cmd.inspect} (in #{Dir.pwd})" - @before_exec.call(self) if @before_exec + @before_exec.call(self) exec(*cmd) end + proc_name 'master (old)' end # forcibly terminate all workers that haven't checked in in @timeout @@ -339,48 +376,62 @@ module Unicorn # is stale for >@timeout seconds, then we'll kill the corresponding # worker. def murder_lazy_workers - now = Time.now - @workers.each_pair do |pid, worker| - (now - worker.tempfile.ctime) <= @timeout and next + WORKERS.each_pair do |pid, worker| + stat = worker.tempfile.stat + stat.mode == 0100000 and next + Time.now - stat.ctime <= @timeout and next logger.error "worker=#{worker.nr} PID:#{pid} is too old, killing" - kill_worker('KILL', pid) # take no prisoners for @timeout violations + kill_worker(:KILL, pid) # take no prisoners for @timeout violations worker.tempfile.close rescue nil end end def spawn_missing_workers - return if @workers.size == @worker_processes (0...@worker_processes).each do |worker_nr| - @workers.values.include?(worker_nr) and next - tempfile = Tempfile.new('') # as short as possible to save dir space + WORKERS.values.include?(worker_nr) and next + begin + Dir.chdir(START_CTX[:cwd]) + rescue Errno::ENOENT => err + logger.fatal "#{err.inspect} (#{START_CTX[:cwd]})" + SIG_QUEUE << :QUIT # forcibly emulate SIGQUIT + return + end + tempfile = Tempfile.new(nil) # as short as possible to save dir space tempfile.unlink # don't allow other processes to find or see it - tempfile.sync = true worker = Worker.new(worker_nr, tempfile) - @before_fork.call(self, worker.nr) + @before_fork.call(self, worker) pid = fork { worker_loop(worker) } - @workers[pid] = worker + WORKERS[pid] = worker end end + def maintain_worker_count + (off = WORKERS.size - @worker_processes) == 0 and return + off < 0 and return spawn_missing_workers + WORKERS.each_pair { |pid,w| + w.nr >= @worker_processes and kill_worker(:QUIT, pid) rescue nil + } + end + # once a client is accepted, it is processed in its entirety here # in 3 easy steps: read request, call app, write app response def process_client(client) - env = @request.read(client) or return - app_response = @app.call(env) - HttpResponse.write(client, app_response) + HttpResponse.write(client, @app.call(@request.read(client))) + # if we get any error, try to write something back to the client + # assuming we haven't closed the socket, but don't get hung up + # if the socket is already closed or broken. We'll always ensure + # the socket is closed at the end of this function rescue EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL,Errno::EBADF - client.closed? or client.close rescue nil + client.write_nonblock(Const::ERROR_500_RESPONSE) rescue nil + client.close rescue nil + rescue HttpParserError # try to tell the client they're bad + client.write_nonblock(Const::ERROR_400_RESPONSE) rescue nil + client.close rescue nil rescue Object => e + client.write_nonblock(Const::ERROR_500_RESPONSE) rescue nil + client.close rescue nil logger.error "Read error: #{e.inspect}" logger.error e.backtrace.join("\n") - ensure - begin - client.closed? or client.close - rescue Object => e - logger.error "Client error: #{e.inspect}" - logger.error e.backtrace.join("\n") - end - @request.reset end # gets rid of stuff the worker has no business keeping track of @@ -388,119 +439,107 @@ module Unicorn # traps for USR1, USR2, and HUP may be set in the @after_fork Proc # by the user. def init_worker_process(worker) - build_app! unless @preload_app - TRAP_SIGS.each { |sig| trap(sig, 'IGNORE') } - trap('CHLD', 'DEFAULT') - trap('USR1') do - @logger.info "worker=#{worker.nr} rotating logs..." - Unicorn::Util.reopen_logs - @logger.info "worker=#{worker.nr} done rotating logs" - end + QUEUE_SIGS.each { |sig| trap(sig, 'IGNORE') } + trap(:CHLD, 'DEFAULT') + SIG_QUEUE.clear + proc_name "worker[#{worker.nr}]" + START_CTX.clear + init_self_pipe! + WORKERS.values.each { |other| other.tempfile.close! rescue nil } + WORKERS.clear + LISTENERS.each { |sock| sock.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) } + worker.tempfile.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) + @after_fork.call(self, worker) # can drop perms + @timeout /= 2.0 # halve it for select() + build_app! unless @config[:preload_app] + end - $0 = "unicorn worker[#{worker.nr}]" - @rd_sig.close if @rd_sig - @wr_sig.close if @wr_sig - @workers.values.each { |other| other.tempfile.close rescue nil } - @workers.clear - @start_ctx.clear - @mode = @start_ctx = @workers = @rd_sig = @wr_sig = nil - @listeners.each { |sock| set_cloexec(sock) } - ENV.delete('UNICORN_FD') - @after_fork.call(self, worker.nr) if @after_fork - @request = HttpRequest.new(logger) + def reopen_worker_logs(worker_nr) + @logger.info "worker=#{worker_nr} reopening logs..." + Unicorn::Util.reopen_logs + @logger.info "worker=#{worker_nr} done reopening logs" + init_self_pipe! end # runs inside each forked worker, this sits around and waits # for connections and doesn't die until the parent dies (or is # given a INT, QUIT, or TERM signal) def worker_loop(worker) + master_pid = Process.ppid # slightly racy, but less memory usage init_worker_process(worker) - nr = 0 - tempfile = worker.tempfile - alive = true - ready = @listeners - client = nil - %w(TERM INT).each { |sig| trap(sig) { exit(0) } } # instant shutdown - trap('QUIT') do - alive = false - @listeners.each { |sock| sock.close rescue nil } # break IO.select - end + nr = 0 # this becomes negative if we need to reopen logs + alive = worker.tempfile # tempfile is our lifeline to the master process + ready = LISTENERS + t = ti = 0 + + # closing anything we IO.select on will raise EBADF + trap(:USR1) { nr = -65536; SELF_PIPE.first.close rescue nil } + trap(:QUIT) { alive = nil; LISTENERS.each { |s| s.close rescue nil } } + [:TERM, :INT].each { |sig| trap(sig) { exit!(0) } } # instant shutdown + @logger.info "worker=#{worker.nr} ready" - while alive && @master_pid == ppid - # we're a goner in @timeout seconds anyways if tempfile.chmod + begin + nr < 0 and reopen_worker_logs(worker.nr) + nr = 0 + + # we're a goner in @timeout seconds anyways if alive.chmod # breaks, so don't trap the exception. Using fchmod() since # futimes() is not available in base Ruby and I very strongly # prefer temporary files to be unlinked for security, # performance and reliability reasons, so utime is out. No-op # changes with chmod doesn't update ctime on all filesystems; so - # we increment our counter each and every time. - tempfile.chmod(nr += 1) + # we change our counter each and every time (after process_client + # and before IO.select). + t == (ti = Time.now.to_i) or alive.chmod(t = ti) - begin - accepted = false - ready.each do |sock| - begin - client = begin - sock.accept_nonblock - rescue Errno::EAGAIN - next - end - accepted = client.sync = true - client.nonblock = false - set_client_sockopt(client) if TCPSocket === client - process_client(client) - rescue Errno::ECONNABORTED - # client closed the socket even before accept - if client && !client.closed? - client.close rescue nil - end - end - tempfile.chmod(nr += 1) - end - client = nil - - # make the following bet: if we accepted clients this round, - # we're probably reasonably busy, so avoid calling select(2) - # and try to do a blind non-blocking accept(2) on everything - # before we sleep again in select - if accepted - ready = @listeners - else - begin - tempfile.chmod(nr += 1) - # timeout used so we can detect parent death: - ret = IO.select(@listeners, nil, nil, @timeout/2.0) or next - ready = ret[0] - rescue Errno::EINTR - ready = @listeners - rescue Errno::EBADF => e - exit(alive ? 1 : 0) - end - end - rescue SystemExit => e - exit(e.status) - rescue Object => e - if alive - logger.error "Unhandled listen loop exception #{e.inspect}." - logger.error e.backtrace.join("\n") + ready.each do |sock| + begin + process_client(sock.accept_nonblock) + nr += 1 + t == (ti = Time.now.to_i) or alive.chmod(t = ti) + rescue Errno::EAGAIN, Errno::ECONNABORTED end + break if nr < 0 end - end + + # make the following bet: if we accepted clients this round, + # we're probably reasonably busy, so avoid calling select() + # and do a speculative accept_nonblock on every listener + # before we sleep again in select(). + redo unless nr == 0 # (nr < 0) => reopen logs + + master_pid == Process.ppid or return + alive.chmod(t = 0) + begin + # timeout used so we can detect parent death: + ret = IO.select(LISTENERS, nil, SELF_PIPE, @timeout) or redo + ready = ret.first + rescue Errno::EINTR + ready = LISTENERS + rescue Errno::EBADF + nr < 0 or return + end + rescue Object => e + if alive + logger.error "Unhandled listen loop exception #{e.inspect}." + logger.error e.backtrace.join("\n") + end + end while alive end # delivers a signal to a worker and fails gracefully if the worker # is no longer running. def kill_worker(signal, pid) begin - kill(signal, pid) + Process.kill(signal, pid) rescue Errno::ESRCH - worker = @workers.delete(pid) and worker.tempfile.close rescue nil + worker = WORKERS.delete(pid) and worker.tempfile.close rescue nil end end # delivers a signal to each worker def kill_each_worker(signal) - @workers.keys.each { |pid| kill_worker(signal, pid) } + WORKERS.keys.each { |pid| kill_worker(signal, pid) } end # unlinks a PID file at given +path+ if it contains the current PID @@ -514,7 +553,7 @@ module Unicorn def valid_pid?(path) if File.exist?(path) && (pid = File.read(path).to_i) > 1 begin - kill(0, pid) + Process.kill(0, pid) return pid rescue Errno::ESRCH end @@ -525,9 +564,11 @@ module Unicorn def load_config! begin logger.info "reloading config_file=#{@config.config_file}" + @config[:listeners].replace(@init_listeners) @config.reload @config.commit!(self) - kill_each_worker('QUIT') + kill_each_worker(:QUIT) + Unicorn::Util.reopen_logs logger.info "done reloading config_file=#{@config.config_file}" rescue Object => e logger.error "error reloading config_file=#{@config.config_file}: " \ @@ -536,7 +577,7 @@ module Unicorn end # returns an array of string names for the given listener array - def listener_names(listeners = @listeners) + def listener_names(listeners = LISTENERS) listeners.map { |io| sock_name(io) } end @@ -544,5 +585,21 @@ module Unicorn @app = @app.call if @app.respond_to?(:arity) && @app.arity == 0 end + def proc_name(tag) + $0 = ([ File.basename(START_CTX[:zero]), tag ] + + START_CTX[:argv]).join(' ') + end + + def redirect_io(io, path) + File.open(path, 'a') { |fp| io.reopen(fp) } if path + io.sync = true + end + + def init_self_pipe! + SELF_PIPE.each { |io| io.close rescue nil } + SELF_PIPE.replace(IO.pipe) + SELF_PIPE.each { |io| io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) } + end + end end diff --git a/lib/unicorn/app/exec_cgi.rb b/lib/unicorn/app/exec_cgi.rb new file mode 100644 index 0000000..8f81d78 --- /dev/null +++ b/lib/unicorn/app/exec_cgi.rb @@ -0,0 +1,156 @@ +require 'unicorn' +require 'rack' + +module Unicorn::App + + # This class is highly experimental (even more so than the rest of Unicorn) + # and has never run anything other than cgit. + class ExecCgi + + CHUNK_SIZE = 16384 + PASS_VARS = %w( + CONTENT_LENGTH + CONTENT_TYPE + GATEWAY_INTERFACE + AUTH_TYPE + PATH_INFO + PATH_TRANSLATED + QUERY_STRING + REMOTE_ADDR + REMOTE_HOST + REMOTE_IDENT + REMOTE_USER + REQUEST_METHOD + SERVER_NAME + SERVER_PORT + SERVER_PROTOCOL + SERVER_SOFTWARE + ).map { |x| x.freeze }.freeze # frozen strings are faster for Hash lookups + + # Intializes the app, example of usage in a config.ru + # map "/cgit" do + # run Unicorn::App::ExecCgi.new("/path/to/cgit.cgi") + # end + def initialize(*args) + @args = args.dup + first = @args[0] or + raise ArgumentError, "need path to executable" + first[0..0] == "/" or @args[0] = ::File.expand_path(first) + File.executable?(@args[0]) or + raise ArgumentError, "#{@args[0]} is not executable" + end + + # Calls the app + def call(env) + out, err = Tempfile.new(''), Tempfile.new('') + out.unlink + err.unlink + inp = force_file_input(env) + inp.sync = out.sync = err.sync = true + pid = fork { run_child(inp, out, err, env) } + inp.close + pid, status = Process.waitpid2(pid) + write_errors(env, err, status) if err.stat.size > 0 + err.close + + return parse_output!(out) if status.success? + out.close + [ 500, { 'Content-Length' => '0', 'Content-Type' => 'text/plain' }, [] ] + end + + private + + def run_child(inp, out, err, env) + PASS_VARS.each do |key| + val = env[key] or next + ENV[key] = val + end + ENV['SCRIPT_NAME'] = @args[0] + ENV['GATEWAY_INTERFACE'] = 'CGI/1.1' + env.keys.grep(/^HTTP_/) { |key| ENV[key] = env[key] } + + a = IO.new(0).reopen(inp) + b = IO.new(1).reopen(out) + c = IO.new(2).reopen(err) + exec(*@args) + end + + # Extracts headers from CGI out, will change the offset of out. + # This returns a standard Rack-compatible return value: + # [ 200, HeadersHash, body ] + def parse_output!(out) + size = out.stat.size + out.sysseek(0) + head = out.sysread(CHUNK_SIZE) + offset = 2 + head, body = head.split(/\n\n/, 2) + if body.nil? + head, body = head.split(/\r\n\r\n/, 2) + offset = 4 + end + offset += head.length + out.instance_variable_set('@unicorn_app_exec_cgi_offset', offset) + size -= offset + + # Allows +out+ to be used as a Rack body. + def out.each + sysseek(@unicorn_app_exec_cgi_offset) + + # don't use a preallocated buffer for sysread since we can't + # guarantee an actual socket is consuming the yielded string + # (or if somebody is pushing to an array for eventual concatenation + begin + yield(sysread(CHUNK_SIZE)) + rescue EOFError + return + end while true + end + + prev = nil + headers = Rack::Utils::HeaderHash.new + head.split(/\r?\n/).each do |line| + case line + when /^([A-Za-z0-9-]+):\s*(.*)$/ then headers[prev = $1] = $2 + when /^[ \t]/ then headers[prev] << "\n#{line}" if prev + end + end + headers['Content-Length'] = size.to_s + [ 200, headers, out ] + end + + # ensures rack.input is a file handle that we can redirect stdin to + def force_file_input(env) + inp = env['rack.input'] + if inp.respond_to?(:fileno) && Integer === inp.fileno + inp + elsif inp.size == 0 # inp could be a StringIO or StringIO-like object + ::File.open('/dev/null') + else + tmp = Tempfile.new('') + tmp.unlink + tmp.binmode + + # Rack::Lint::InputWrapper doesn't allow sysread :( + buf = '' + while inp.read(CHUNK_SIZE, buf) + tmp.syswrite(buf) + end + tmp.sysseek(0) + tmp + end + end + + # rack.errors this may not be an IO object, so we couldn't + # just redirect the CGI executable to that earlier. + def write_errors(env, err, status) + err.seek(0) + dst = env['rack.errors'] + pid = status.pid + dst.write("#{pid}: #{@args.inspect} status=#{status} stderr:\n") + err.each_line { |line| dst.write("#{pid}: #{line}") } + dst.flush + end + + end + +end diff --git a/lib/unicorn/app/old_rails.rb b/lib/unicorn/app/old_rails.rb new file mode 100644 index 0000000..9b3a3b1 --- /dev/null +++ b/lib/unicorn/app/old_rails.rb @@ -0,0 +1,29 @@ +# This code is based on the original Rails handler in Mongrel +# Copyright (c) 2005 Zed A. Shaw +# Copyright (c) 2009 Eric Wong +# You can redistribute it and/or modify it under the same terms as Ruby. +# Additional work donated by contributors. See CONTRIBUTORS for more info. +require 'unicorn/cgi_wrapper' +require 'dispatcher' + +module Unicorn; module App; end; end + +# Implements a handler that can run Rails. +class Unicorn::App::OldRails + + def call(env) + cgi = Unicorn::CGIWrapper.new(env) + begin + Dispatcher.dispatch(cgi, + ActionController::CgiRequest::DEFAULT_SESSION_OPTIONS, + cgi.body) + rescue Object => e + err = env['rack.errors'] + err.write("#{e} #{e.message}\n") + e.backtrace.each { |line| err.write("#{line}\n") } + end + cgi.out # finalize the response + cgi.rack_response + end + +end diff --git a/lib/unicorn/app/old_rails/static.rb b/lib/unicorn/app/old_rails/static.rb new file mode 100644 index 0000000..17c007c --- /dev/null +++ b/lib/unicorn/app/old_rails/static.rb @@ -0,0 +1,60 @@ +# This code is based on the original Rails handler in Mongrel +# Copyright (c) 2005 Zed A. Shaw +# Copyright (c) 2009 Eric Wong +# You can redistribute it and/or modify it under the same terms as Ruby. + +require 'rack/file' + +# Static file handler for Rails < 2.3. This handler is only provided +# as a convenience for developers. Performance-minded deployments should +# use nginx (or similar) for serving static files. +# +# This supports page caching directly and will try to resolve a +# request in the following order: +# +# * If the requested exact PATH_INFO exists as a file then serve it. +# * If it exists at PATH_INFO+rest_operator+".html" exists +# then serve that. +# +# This means that if you are using page caching it will actually work +# with Unicorn and you should see a decent speed boost (but not as +# fast as if you use a static server like nginx). +class Unicorn::App::OldRails::Static + FILE_METHODS = { 'GET' => true, 'HEAD' => true }.freeze + REQUEST_METHOD = 'REQUEST_METHOD'.freeze + REQUEST_URI = 'REQUEST_URI'.freeze + PATH_INFO = 'PATH_INFO'.freeze + + def initialize(app) + @app = app + @root = "#{::RAILS_ROOT}/public" + @file_server = ::Rack::File.new(@root) + end + + def call(env) + # short circuit this ASAP if serving non-file methods + FILE_METHODS.include?(env[REQUEST_METHOD]) or return @app.call(env) + + # first try the path as-is + path_info = env[PATH_INFO].chomp("/") + if File.file?("#@root/#{::Rack::Utils.unescape(path_info)}") + # File exists as-is so serve it up + env[PATH_INFO] = path_info + return @file_server.call(env) + end + + # then try the cached version: + + # grab the semi-colon REST operator used by old versions of Rails + # this is the reason we didn't just copy the new Rails::Rack::Static + env[REQUEST_URI] =~ /^#{Regexp.escape(path_info)}(;[^\?]+)/ + path_info << "#$1#{ActionController::Base.page_cache_extension}" + + if File.file?("#@root/#{::Rack::Utils.unescape(path_info)}") + env[PATH_INFO] = path_info + return @file_server.call(env) + end + + @app.call(env) # call OldRails + end +end if defined?(Unicorn::App::OldRails) diff --git a/lib/unicorn/cgi_wrapper.rb b/lib/unicorn/cgi_wrapper.rb new file mode 100644 index 0000000..bc622ea --- /dev/null +++ b/lib/unicorn/cgi_wrapper.rb @@ -0,0 +1,149 @@ +# This code is based on the original CGIWrapper from Mongrel +# Copyright (c) 2005 Zed A. Shaw +# Copyright (c) 2009 Eric Wong +# You can redistribute it and/or modify it under the same terms as Ruby. +# +# Additional work donated by contributors. See CONTRIBUTORS for more info. + +require 'cgi' + +module Unicorn; end + +# The beginning of a complete wrapper around Unicorn's internal HTTP +# processing system but maintaining the original Ruby CGI module. Use +# this only as a crutch to get existing CGI based systems working. It +# should handle everything, but please notify us if you see special +# warnings. This work is still very alpha so we need testers to help +# work out the various corner cases. +class Unicorn::CGIWrapper < ::CGI + undef_method :env_table + attr_reader :env_table + attr_reader :body + + # these are stripped out of any keys passed to CGIWrapper.header function + NPH = 'nph'.freeze # Completely ignored, Unicorn outputs the date regardless + CONNECTION = 'connection'.freeze # Completely ignored. Why is CGI doing this? + CHARSET = 'charset'.freeze # this gets appended to Content-Type + COOKIE = 'cookie'.freeze # maps (Hash,Array,String) to "Set-Cookie" headers + STATUS = 'status'.freeze # stored as @status + Status = 'Status'.freeze # code + human-readable text, Rails sets this + + # some of these are common strings, but this is the only module + # using them and the reason they're not in Unicorn::Const + SET_COOKIE = 'Set-Cookie'.freeze + CONTENT_TYPE = 'Content-Type'.freeze + CONTENT_LENGTH = 'Content-Length'.freeze # this is NOT Const::CONTENT_LENGTH + RACK_INPUT = 'rack.input'.freeze + RACK_ERRORS = 'rack.errors'.freeze + + # this maps CGI header names to HTTP header names + HEADER_MAP = { + 'status' => Status, + 'type' => CONTENT_TYPE, + 'server' => 'Server'.freeze, + 'language' => 'Content-Language'.freeze, + 'expires' => 'Expires'.freeze, + 'length' => CONTENT_LENGTH, + }.freeze + + # Takes an a Rackable environment, plus any additional CGI.new + # arguments These are used internally to create a wrapper around the + # real CGI while maintaining Rack/Unicorn's view of the world. This + # this will NOT deal well with large responses that take up a lot of + # memory, but neither does the CGI nor the original CGIWrapper from + # Mongrel... + def initialize(rack_env, *args) + @env_table = rack_env + @status = nil + @head = {} + @headv = Hash.new { |hash,key| hash[key] = [] } + @body = StringIO.new + super(*args) + end + + # finalizes the response in a way Rack applications would expect + def rack_response + # @head[CONTENT_LENGTH] ||= @body.size + @headv[SET_COOKIE] += @output_cookies if @output_cookies + @headv.each_pair do |key,value| + @head[key] ||= value.join("\n") unless value.empty? + end + + # Capitalized "Status:", with human-readable status code (e.g. "200 OK") + parseable_status = @head.delete(Status) + @status ||= parseable_status.split(/ /)[0].to_i rescue 500 + + [ @status || 500, @head, [ @body.string ] ] + end + + # The header is typically called to send back the header. In our case we + # collect it into a hash for later usage. This can be called multiple + # times to set different cookies. + def header(options = "text/html") + # if they pass in a string then just write the Content-Type + if String === options + @head[CONTENT_TYPE] ||= options + else + HEADER_MAP.each_pair do |from, to| + from = options.delete(from) or next + @head[to] = from.to_s + end + + @head[CONTENT_TYPE] ||= "text/html" + if charset = options.delete(CHARSET) + @head[CONTENT_TYPE] << "; charset=#{charset}" + end + + # lots of ways to set cookies + if cookie = options.delete(COOKIE) + set_cookies = @headv[SET_COOKIE] + case cookie + when Array + cookie.each { |c| set_cookies << c.to_s } + when Hash + cookie.each_value { |c| set_cookies << c.to_s } + else + set_cookies << cookie.to_s + end + end + @status ||= options.delete(STATUS) # all lower-case + + # drop the keys we don't want anymore + options.delete(NPH) + options.delete(CONNECTION) + + # finally, set the rest of the headers as-is, allowing duplicates + options.each_pair { |k,v| @headv[k] << v } + end + + # doing this fakes out the cgi library to think the headers are empty + # we then do the real headers in the out function call later + "" + end + + # The dumb thing is people can call header or this or both and in + # any order. So, we just reuse header and then finalize the + # HttpResponse the right way. This will have no effect if called + # the second time if the first "outputted" anything. + def out(options = "text/html") + header(options) + @body.size == 0 or return + @body << yield if block_given? + end + + # Used to wrap the normal stdinput variable used inside CGI. + def stdinput + @env_table[RACK_INPUT] + end + + # The stdoutput should be completely bypassed but we'll drop a + # warning just in case + def stdoutput + err = @env_table[RACK_ERRORS] + err.puts "WARNING: Your program is doing something not expected." + err.puts "Please tell Eric that stdoutput was used and what software " \ + "you are running. Thanks." + @body + end + +end diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb index dd9ae3b..a432f64 100644 --- a/lib/unicorn/configurator.rb +++ b/lib/unicorn/configurator.rb @@ -1,5 +1,4 @@ -require 'unicorn/socket' -require 'unicorn/const' +require 'socket' require 'logger' module Unicorn @@ -8,42 +7,40 @@ module Unicorn # # Example (when used with the unicorn config file): # worker_processes 4 - # listeners %w(0.0.0.0:9292 /tmp/my_app.sock) + # listen '/tmp/my_app.sock', :backlog => 1 + # listen '0.0.0.0:9292' # timeout 10 # pid "/tmp/my_app.pid" - # after_fork do |server,worker_nr| - # server.listen("127.0.0.1:#{9293 + worker_nr}") rescue nil + # after_fork do |server,worker| + # server.listen("127.0.0.1:#{9293 + worker.nr}") rescue nil # end class Configurator - include ::Unicorn::SocketHelper - # The default logger writes its output to $stderr DEFAULT_LOGGER = Logger.new($stderr) unless defined?(DEFAULT_LOGGER) # Default settings for Unicorn DEFAULTS = { :timeout => 60, - :listeners => [ Const::DEFAULT_LISTEN ], + :listeners => [], :logger => DEFAULT_LOGGER, :worker_processes => 1, - :after_fork => lambda { |server, worker_nr| - server.logger.info("worker=#{worker_nr} spawned pid=#{$$}") + :after_fork => lambda { |server, worker| + server.logger.info("worker=#{worker.nr} spawned pid=#{$$}") # per-process listener ports for debugging/admin: # "rescue nil" statement is needed because USR2 will # cause the master process to reexecute itself and the # per-worker ports can be taken, necessitating another # HUP after QUIT-ing the original master: - # server.listen("127.0.0.1:#{8081 + worker_nr}") rescue nil + # server.listen("127.0.0.1:#{8081 + worker.nr}") rescue nil }, - :before_fork => lambda { |server, worker_nr| - server.logger.info("worker=#{worker_nr} spawning...") + :before_fork => lambda { |server, worker| + server.logger.info("worker=#{worker.nr} spawning...") }, :before_exec => lambda { |server| server.logger.info("forked child re-executing...") }, :pid => nil, - :backlog => 1024, :preload_app => false, :stderr_path => nil, :stdout_path => nil, @@ -83,23 +80,6 @@ module Unicorn @set[key] end - # Changes the listen() syscall backlog to +nr+ for yet-to-be-created - # sockets. Due to limitations of the OS, this cannot affect - # existing listener sockets in any way, sockets must be completely - # closed and rebound (inherited sockets preserve their existing - # backlog setting). Some operating systems allow negative values - # here to specify the maximum allowable value. See the listen(2) - # syscall documentation of your OS for the exact semantics of this. - # - # If you are running unicorn on multiple machines, lowering this number - # can help your load balancer detect when a machine is overloaded - # and give requests to a different machine. - def backlog(nr) - Integer === nr or raise ArgumentError, - "not an integer: backlog=#{nr.inspect}" - @set[:backlog] = nr - end - # sets object to the +new+ Logger-like object. The new logger-like # object must respond to the following methods: # +debug+, +info+, +warn+, +error+, +fatal+, +close+ @@ -116,23 +96,37 @@ module Unicorn # the worker after forking. The following is an example hook which adds # a per-process listener to every worker: # - # after_fork do |server,worker_nr| + # after_fork do |server,worker| # # per-process listener ports for debugging/admin: # # "rescue nil" statement is needed because USR2 will # # cause the master process to reexecute itself and the # # per-worker ports can be taken, necessitating another # # HUP after QUIT-ing the original master: - # server.listen("127.0.0.1:#{9293 + worker_nr}") rescue nil + # server.listen("127.0.0.1:#{9293 + worker.nr}") rescue nil + # + # # drop permissions to "www-data" in the worker + # # generally there's no reason to start Unicorn as a priviledged user + # # as it is not recommended to expose Unicorn to public clients. + # uid, gid = Process.euid, Process.egid + # user, group = 'www-data', 'www-data' + # target_uid = Etc.getpwnam(user).uid + # target_gid = Etc.getgrnam(group).gid + # worker.tempfile.chown(target_uid, target_gid) + # if uid != target_uid || gid != target_gid + # Process.initgroups(user, target_gid) + # Process::GID.change_privilege(target_gid) + # Process::UID.change_privilege(target_uid) + # end # end - def after_fork(&block) - set_hook(:after_fork, block) + def after_fork(*args, &block) + set_hook(:after_fork, block_given? ? block : args[0]) end # sets before_fork got be a given Proc object. This Proc # object will be called by the master process before forking # each worker. - def before_fork(&block) - set_hook(:before_fork, block) + def before_fork(*args, &block) + set_hook(:before_fork, block_given? ? block : args[0]) end # sets the before_exec hook to a given Proc object. This @@ -141,20 +135,22 @@ module Unicorn # for freeing certain OS resources that you do NOT wish to # share with the reexeced child process. # There is no corresponding after_exec hook (for obvious reasons). - def before_exec(&block) - set_hook(:before_exec, block, 1) + def before_exec(*args, &block) + set_hook(:before_exec, block_given? ? block : args[0], 1) end # sets the timeout of worker processes to +seconds+. Workers # handling the request/app.call/response cycle taking longer than # this time period will be forcibly killed (via SIGKILL). This # timeout is enforced by the master process itself and not subject - # to the scheduling limitations by the worker process. + # to the scheduling limitations by the worker process. Due the + # low-complexity, low-overhead implementation, timeouts of less + # than 3.0 seconds can be considered inaccurate and unsafe. def timeout(seconds) Numeric === seconds or raise ArgumentError, "not numeric: timeout=#{seconds.inspect}" - seconds > 0 or raise ArgumentError, - "not positive: timeout=#{seconds.inspect}" + seconds >= 3 or raise ArgumentError, + "too low: timeout=#{seconds.inspect}" @set[:timeout] = seconds end @@ -171,13 +167,59 @@ module Unicorn # sets listeners to the given +addresses+, replacing or augmenting the # current set. This is for the global listener pool shared by all # worker processes. For per-worker listeners, see the after_fork example - def listeners(addresses) + # This is for internal API use only, do not use it in your Unicorn + # config file. Use listen instead. + def listeners(addresses) # :nodoc: Array === addresses or addresses = Array(addresses) + addresses.map! { |addr| expand_addr(addr) } @set[:listeners] = addresses end - # adds an +address+ to the existing listener set - def listen(address) + # adds an +address+ to the existing listener set. + # + # The following options may be specified (but are generally not needed): + # + # +backlog+: this is the backlog of the listen() syscall. + # + # Some operating systems allow negative values here to specify the + # maximum allowable value. In most cases, this number is only + # recommendation and there are other OS-specific tunables and + # variables that can affect this number. See the listen(2) + # syscall documentation of your OS for the exact semantics of + # this. + # + # If you are running unicorn on multiple machines, lowering this number + # can help your load balancer detect when a machine is overloaded + # and give requests to a different machine. + # + # Default: 1024 + # + # +rcvbuf+, +sndbuf+: maximum send and receive buffer sizes of sockets + # + # These correspond to the SO_RCVBUF and SO_SNDBUF settings which + # can be set via the setsockopt(2) syscall. Some kernels + # (e.g. Linux 2.4+) have intelligent auto-tuning mechanisms and + # there is no need (and it is sometimes detrimental) to specify them. + # + # See the socket API documentation of your operating system + # to determine the exact semantics of these settings and + # other operating system-specific knobs where they can be + # specified. + # + # Defaults: operating system defaults + def listen(address, opt = { :backlog => 1024 }) + address = expand_addr(address) + if String === address + Hash === @set[:listener_opts] or + @set[:listener_opts] = Hash.new { |hash,key| hash[key] = {} } + [ :backlog, :sndbuf, :rcvbuf ].each do |key| + value = opt[key] or next + Integer === value or + raise ArgumentError, "not an integer: #{key}=#{value.inspect}" + end + @set[:listener_opts][address].merge!(opt) + end + @set[:listeners] = [] unless Array === @set[:listeners] @set[:listeners] << address end @@ -194,6 +236,10 @@ module Unicorn # properly close/reopen sockets. Files opened for logging do not # have to be reopened as (unbuffered-in-userspace) files opened with # the File::APPEND flag are written to atomically on UNIX. + # + # In addition to reloading the unicorn-specific config settings, + # SIGHUP will reload application code in the working + # directory/symlink when workers are gracefully restarted. def preload_app(bool) case bool when TrueClass, FalseClass @@ -249,5 +295,28 @@ module Unicorn @set[var] = my_proc end + # expands "unix:path/to/foo" to a socket relative to the current path + # expands pathnames of sockets if relative to "~" or "~username" + # expands "*:port and ":port" to "0.0.0.0:port" + def expand_addr(address) #:nodoc + return "0.0.0.0:#{address}" if Integer === address + return address unless String === address + + case address + when %r{\Aunix:(.*)\z} + File.expand_path($1) + when %r{\A~} + File.expand_path(address) + when %r{\A(?:\*:)?(\d+)\z} + "0.0.0.0:#$1" + when %r{\A(.*):(\d+)\z} + # canonicalize the name + packed = Socket.pack_sockaddr_in($2.to_i, $1) + Socket.unpack_sockaddr_in(packed).reverse!.join(':') + else + address + end + end + end end diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb index 46398e5..241c52e 100644 --- a/lib/unicorn/const.rb +++ b/lib/unicorn/const.rb @@ -1,89 +1,19 @@ -module Unicorn +require 'rack/utils' - # Every standard HTTP code mapped to the appropriate message. These are - # used so frequently that they are placed directly in Unicorn for easy - # access rather than Unicorn::Const itself. - HTTP_STATUS_CODES = { - 100 => 'Continue', - 101 => 'Switching Protocols', - 200 => 'OK', - 201 => 'Created', - 202 => 'Accepted', - 203 => 'Non-Authoritative Information', - 204 => 'No Content', - 205 => 'Reset Content', - 206 => 'Partial Content', - 300 => 'Multiple Choices', - 301 => 'Moved Permanently', - 302 => 'Moved Temporarily', - 303 => 'See Other', - 304 => 'Not Modified', - 305 => 'Use Proxy', - 400 => 'Bad Request', - 401 => 'Unauthorized', - 402 => 'Payment Required', - 403 => 'Forbidden', - 404 => 'Not Found', - 405 => 'Method Not Allowed', - 406 => 'Not Acceptable', - 407 => 'Proxy Authentication Required', - 408 => 'Request Time-out', - 409 => 'Conflict', - 410 => 'Gone', - 411 => 'Length Required', - 412 => 'Precondition Failed', - 413 => 'Request Entity Too Large', - 414 => 'Request-URI Too Large', - 415 => 'Unsupported Media Type', - 500 => 'Internal Server Error', - 501 => 'Not Implemented', - 502 => 'Bad Gateway', - 503 => 'Service Unavailable', - 504 => 'Gateway Time-out', - 505 => 'HTTP Version not supported' - } +module Unicorn # Frequently used constants when constructing requests or responses. Many times # the constant just refers to a string with the same contents. Using these constants # gave about a 3% to 10% performance improvement over using the strings directly. # Symbols did not really improve things much compared to constants. - # - # While Unicorn does try to emulate the CGI/1.2 protocol, it does not use the REMOTE_IDENT, - # REMOTE_USER, or REMOTE_HOST parameters since those are either a security problem or - # too taxing on performance. module Const - DATE="Date".freeze - - # This is the part of the path after the SCRIPT_NAME. - PATH_INFO="PATH_INFO".freeze - - # Request body - HTTP_BODY="HTTP_BODY".freeze - - # This is the initial part that your handler is identified as by URIClassifier. - SCRIPT_NAME="SCRIPT_NAME".freeze - - # The original URI requested by the client. Passed to URIClassifier to build PATH_INFO and SCRIPT_NAME. - REQUEST_URI='REQUEST_URI'.freeze - REQUEST_PATH='REQUEST_PATH'.freeze - - UNICORN_VERSION="0.1.0".freeze - - UNICORN_TMP_BASE="unicorn".freeze + UNICORN_VERSION="0.7.0".freeze DEFAULT_HOST = "0.0.0.0".freeze # default TCP listen host address DEFAULT_PORT = "8080".freeze # default TCP listen port DEFAULT_LISTEN = "#{DEFAULT_HOST}:#{DEFAULT_PORT}".freeze - # The standard empty 404 response for bad requests. Use Error4040Handler for custom stuff. - ERROR_404_RESPONSE="HTTP/1.1 404 Not Found\r\nConnection: close\r\nServer: Unicorn #{UNICORN_VERSION}\r\n\r\nNOT FOUND".freeze - - CONTENT_LENGTH="CONTENT_LENGTH".freeze - - # A common header for indicating the server is too busy. Not used yet. - ERROR_503_RESPONSE="HTTP/1.1 503 Service Unavailable\r\n\r\nBUSY".freeze - # The basic max request size we'll try to read. CHUNK_SIZE=(16 * 1024) @@ -94,23 +24,15 @@ module Unicorn # Maximum request body size before it is moved out of memory and into a tempfile for reading. MAX_BODY=MAX_HEADER + # common errors we'll send back + ERROR_400_RESPONSE = "HTTP/1.1 400 Bad Request\r\n\r\n".freeze + ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n".freeze + # A frozen format for this is about 15% faster - CONTENT_TYPE = "Content-Type".freeze - LAST_MODIFIED = "Last-Modified".freeze - ETAG = "ETag".freeze - REQUEST_METHOD="REQUEST_METHOD".freeze - GET="GET".freeze - HEAD="HEAD".freeze - # ETag is based on the apache standard of hex mtime-size-inode (inode is 0 on win32) - ETAG_FORMAT="\"%x-%x-%x\"".freeze - LINE_END="\r\n".freeze + CONTENT_LENGTH="CONTENT_LENGTH".freeze REMOTE_ADDR="REMOTE_ADDR".freeze HTTP_X_FORWARDED_FOR="HTTP_X_FORWARDED_FOR".freeze - HTTP_IF_MODIFIED_SINCE="HTTP_IF_MODIFIED_SINCE".freeze - HTTP_IF_NONE_MATCH="HTTP_IF_NONE_MATCH".freeze - REDIRECT = "HTTP/1.1 302 Found\r\nLocation: %s\r\nConnection: close\r\n\r\n".freeze - HOST = "HOST".freeze - CONNECTION = "Connection".freeze + RACK_INPUT="rack.input".freeze end end diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb index ce0e408..368305f 100644 --- a/lib/unicorn/http_request.rb +++ b/lib/unicorn/http_request.rb @@ -1,5 +1,4 @@ require 'tempfile' -require 'uri' require 'stringio' # compiled extension @@ -13,165 +12,140 @@ module Unicorn # class HttpRequest + # default parameters we merge into the request env for Rack handlers + DEFAULTS = { + "rack.errors" => $stderr, + "rack.multiprocess" => true, + "rack.multithread" => false, + "rack.run_once" => false, + "rack.version" => [1, 0].freeze, + "SCRIPT_NAME" => "".freeze, + + # this is not in the Rack spec, but some apps may rely on it + "SERVER_SOFTWARE" => "Unicorn #{Const::UNICORN_VERSION}".freeze + } + + # Optimize for the common case where there's no request body + # (GET/HEAD) requests. + NULL_IO = StringIO.new + LOCALHOST = '127.0.0.1'.freeze + + # Being explicitly single-threaded, we have certain advantages in + # not having to worry about variables being clobbered :) + BUFFER = ' ' * Const::CHUNK_SIZE # initial size, may grow + PARSER = HttpParser.new + PARAMS = Hash.new + def initialize(logger) @logger = logger - @body = nil - @buffer = ' ' * Const::CHUNK_SIZE # initial size, may grow - @parser = HttpParser.new - @params = Hash.new - end - - def reset - @parser.reset - @params.clear - @body.close rescue nil - @body = nil end - # # Does the majority of the IO processing. It has been written in - # Ruby using about 7 different IO processing strategies and no - # matter how it's done the performance just does not improve. It is - # currently carefully constructed to make sure that it gets the best - # possible performance, but anyone who thinks they can make it - # faster is more than welcome to take a crack at it. + # Ruby using about 8 different IO processing strategies. + # + # It is currently carefully constructed to make sure that it gets + # the best possible performance for the common case: GET requests + # that are fully complete after a single read(2) + # + # Anyone who thinks they can make it faster is more than welcome to + # take a crack at it. # # returns an environment hash suitable for Rack if successful # This does minimal exception trapping and it is up to the caller # to handle any socket errors (e.g. user aborted upload). def read(socket) - data = String.new(read_socket(socket)) - nparsed = 0 - - # Assumption: nparsed will always be less since data will get - # filled with more after each parsing. If it doesn't get more - # then there was a problem with the read operation on the client - # socket. Effect is to stop processing when the socket can't - # fill the buffer for further parsing. - while nparsed < data.length - nparsed = @parser.execute(@params, data, nparsed) - - if @parser.finished? - # From http://www.ietf.org/rfc/rfc3875: - # "Script authors should be aware that the REMOTE_ADDR and - # REMOTE_HOST meta-variables (see sections 4.1.8 and 4.1.9) - # may not identify the ultimate source of the request. They - # identify the client for the immediate request to the server; - # that client may be a proxy, gateway, or other intermediary - # acting on behalf of the actual source client." - @params[Const::REMOTE_ADDR] = socket.unicorn_peeraddr - - handle_body(socket) and return rack_env # success! - return nil # fail - else - # Parser is not done, queue up more data to read and continue - # parsing - data << read_socket(socket) - if data.length >= Const::MAX_HEADER - raise HttpParserError.new("HEADER is longer than allowed, " \ - "aborting client early.") - end - end + # reset the parser + unless NULL_IO == (input = PARAMS[Const::RACK_INPUT]) # unlikely + input.close rescue nil + input.close! rescue nil end - nil # XXX bug? + PARAMS.clear + PARSER.reset + + # From http://www.ietf.org/rfc/rfc3875: + # "Script authors should be aware that the REMOTE_ADDR and + # REMOTE_HOST meta-variables (see sections 4.1.8 and 4.1.9) + # may not identify the ultimate source of the request. They + # identify the client for the immediate request to the server; + # that client may be a proxy, gateway, or other intermediary + # acting on behalf of the actual source client." + PARAMS[Const::REMOTE_ADDR] = + TCPSocket === socket ? socket.peeraddr.last : LOCALHOST + + # short circuit the common case with small GET requests first + PARSER.execute(PARAMS, socket.readpartial(Const::CHUNK_SIZE, BUFFER)) and + return handle_body(socket) + + data = BUFFER.dup # socket.readpartial will clobber BUFFER + + # Parser is not done, queue up more data to read and continue parsing + # an Exception thrown from the PARSER will throw us out of the loop + begin + data << socket.readpartial(Const::CHUNK_SIZE, BUFFER) + PARSER.execute(PARAMS, data) and return handle_body(socket) + end while true rescue HttpParserError => e @logger.error "HTTP parse error, malformed request " \ - "(#{@params[Const::HTTP_X_FORWARDED_FOR] || - socket.unicorn_peeraddr}): #{e.inspect}" + "(#{PARAMS[Const::HTTP_X_FORWARDED_FOR] || + PARAMS[Const::REMOTE_ADDR]}): #{e.inspect}" @logger.error "REQUEST DATA: #{data.inspect}\n---\n" \ - "PARAMS: #{@params.inspect}\n---\n" - socket.closed? or socket.close rescue nil - nil + "PARAMS: #{PARAMS.inspect}\n---\n" + raise e end private # Handles dealing with the rest of the request - # returns true if successful, false if not + # returns a Rack environment if successful, raises an exception if not def handle_body(socket) - http_body = @params[Const::HTTP_BODY] - content_length = @params[Const::CONTENT_LENGTH].to_i - remain = content_length - http_body.length + http_body = PARAMS.delete(:http_body) + content_length = PARAMS[Const::CONTENT_LENGTH].to_i - # must read more data to complete body - if remain < Const::MAX_BODY - # small body, just use that - @body = StringIO.new(http_body) - else # huge body, put it in a tempfile - @body = Tempfile.new(Const::UNICORN_TMP_BASE) - @body.binmode - @body.sync = true - @body.syswrite(http_body) + if content_length == 0 # short circuit the common case + PARAMS[Const::RACK_INPUT] = NULL_IO.closed? ? NULL_IO.reopen : NULL_IO + return PARAMS.update(DEFAULTS) end + # must read more data to complete body + remain = content_length - http_body.length + + body = PARAMS[Const::RACK_INPUT] = (remain < Const::MAX_BODY) ? + StringIO.new : Tempfile.new('unicorn') + + body.binmode + body.write(http_body) + # Some clients (like FF1.0) report 0 for body and then send a body. # This will probably truncate them but at least the request goes through # usually. - if remain > 0 - read_body(socket, remain) or return false # fail! - end - @body.rewind - @body.sysseek(0) if @body.respond_to?(:sysseek) + read_body(socket, remain, body) if remain > 0 + body.rewind # in case read_body overread because the client tried to pipeline # another request, we'll truncate it. Again, we don't do pipelining # or keepalive - @body.truncate(content_length) - true + body.truncate(content_length) + PARAMS.update(DEFAULTS) end - # Returns an environment which is rackable: - # http://rack.rubyforge.org/doc/files/SPEC.html - # Based on Rack's old Mongrel handler. - def rack_env - # It might be a dumbass full host request header - @params[Const::REQUEST_PATH] ||= - URI.parse(@params[Const::REQUEST_URI]).path - raise "No REQUEST PATH" unless @params[Const::REQUEST_PATH] - - @params["QUERY_STRING"] ||= '' - @params.delete "HTTP_CONTENT_TYPE" - @params.delete "HTTP_CONTENT_LENGTH" - @params.update({ "rack.version" => [0,1], - "rack.input" => @body, - "rack.errors" => $stderr, - "rack.multithread" => false, - "rack.multiprocess" => true, - "rack.run_once" => false, - "rack.url_scheme" => "http", - Const::PATH_INFO => @params[Const::REQUEST_PATH], - Const::SCRIPT_NAME => "", - }) - end - - # Does the heavy lifting of properly reading the larger body requests in - # small chunks. It expects @body to be an IO object, socket to be valid, - # It also expects any initial part of the body that has been read to be in - # the @body already. It will return true if successful and false if not. - def read_body(socket, remain) - while remain > 0 - # writes always write the requested amount on a POSIX filesystem - remain -= @body.syswrite(read_socket(socket)) - end - true # success! + # Does the heavy lifting of properly reading the larger body + # requests in small chunks. It expects PARAMS['rack.input'] to be + # an IO object, socket to be valid, It also expects any initial part + # of the body that has been read to be in the PARAMS['rack.input'] + # already. It will return true if successful and false if not. + def read_body(socket, remain, body) + begin + # write always writes the requested amount on a POSIX filesystem + remain -= body.write(socket.readpartial(Const::CHUNK_SIZE, BUFFER)) + end while remain > 0 rescue Object => e - logger.error "Error reading HTTP body: #{e.inspect}" - socket.closed? or socket.close rescue nil + @logger.error "Error reading HTTP body: #{e.inspect}" # Any errors means we should delete the file, including if the file # is dumped. Truncate it ASAP to help avoid page flushes to disk. - @body.truncate(0) rescue nil + body.truncate(0) rescue nil reset - false - end - - # read(2) on "slow" devices like sockets can be interrupted by signals - def read_socket(socket) - begin - socket.sysread(Const::CHUNK_SIZE, @buffer) - rescue Errno::EINTR - retry - end + raise e end end diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb index 7bbb940..15df3f6 100644 --- a/lib/unicorn/http_response.rb +++ b/lib/unicorn/http_response.rb @@ -21,52 +21,50 @@ module Unicorn class HttpResponse - # headers we allow duplicates for - ALLOWED_DUPLICATES = { - 'Set-Cookie' => true, - 'Set-Cookie2' => true, - 'Warning' => true, - 'WWW-Authenticate' => true, - }.freeze + # Every standard HTTP code mapped to the appropriate message. + CODES = Rack::Utils::HTTP_STATUS_CODES.inject({}) { |hash,(code,msg)| + hash[code] = "#{code} #{msg}" + hash + } + + # Rack does not set/require a Date: header. We always override the + # Connection: and Date: headers no matter what (if anything) our + # Rack application sent us. + SKIP = { 'connection' => true, 'date' => true, 'status' => true }.freeze + EMPTY = ''.freeze # :nodoc + OUT = [] # :nodoc # writes the rack_response to socket as an HTTP response def self.write(socket, rack_response) status, headers, body = rack_response + status = CODES[status.to_i] + OUT.clear - # Rack does not set/require Date, but don't worry about Content-Length - # since Rack applications that conform to Rack::Lint enforce that - out = [ "#{Const::DATE}: #{Time.now.httpdate}" ] - sent = { Const::CONNECTION => true, Const::DATE => true } - + # Don't bother enforcing duplicate supression, it's a Hash most of + # the time anyways so just hope our app knows what it's doing headers.each do |key, value| - if ! sent[key] || ALLOWED_DUPLICATES[key] - sent[key] = true - out << "#{key}: #{value}" + next if SKIP.include?(key.downcase) + if value =~ /\n/ + value.split(/\n/).each { |v| OUT << "#{key}: #{v}\r\n" } + else + OUT << "#{key}: #{value}\r\n" end end - socket_write(socket, - "HTTP/1.1 #{status} #{HTTP_STATUS_CODES[status]}\r\n" \ + # Rack should enforce Content-Length or chunked transfer encoding, + # so don't worry or care about them. + # Date is required by HTTP/1.1 as long as our clock can be trusted. + # Some broken clients require a "Status" header so we accomodate them + socket.write("HTTP/1.1 #{status}\r\n" \ + "Date: #{Time.now.httpdate}\r\n" \ + "Status: #{status}\r\n" \ "Connection: close\r\n" \ - "#{out.join("\r\n")}\r\n\r\n") - body.each { |chunk| socket_write(socket, chunk) } + "#{OUT.join(EMPTY)}\r\n") + body.each { |chunk| socket.write(chunk) } + socket.close # flushes and uncorks the socket immediately + ensure + body.respond_to?(:close) and body.close rescue nil end - private - - # write(2) can return short on slow devices like sockets as well - # as fail with EINTR if a signal was caught. - def self.socket_write(socket, buffer) - loop do - begin - written = socket.syswrite(buffer) - return written if written == buffer.length - buffer = buffer[written..-1] - rescue Errno::EINTR - retry - end - end - end - end end diff --git a/lib/unicorn/launcher.rb b/lib/unicorn/launcher.rb new file mode 100644 index 0000000..8c96059 --- /dev/null +++ b/lib/unicorn/launcher.rb @@ -0,0 +1,33 @@ +$stdin.sync = $stdout.sync = $stderr.sync = true +require 'unicorn' + +class Unicorn::Launcher + + # We don't do a lot of standard daemonization stuff: + # * umask is whatever was set by the parent process at startup + # and can be set in config.ru and config_file, so making it + # 0000 and potentially exposing sensitive log data can be bad + # policy. + # * don't bother to chdir("/") here since unicorn is designed to + # run inside APP_ROOT. Unicorn will also re-chdir() to + # the directory it was started in when being re-executed + # to pickup code changes if the original deployment directory + # is a symlink or otherwise got replaced. + def self.daemonize! + $stdin.reopen("/dev/null") + + # We only start a new process group if we're not being reexecuted + # and inheriting file descriptors from our parent + unless ENV['UNICORN_FD'] + exit if fork + Process.setsid + exit if fork + + # $stderr/$stderr can/will be redirected separately in the Unicorn config + $stdout.reopen("/dev/null", "a") + $stderr.reopen("/dev/null", "a") + end + $stdin.sync = $stdout.sync = $stderr.sync = true + end + +end diff --git a/lib/unicorn/socket.rb b/lib/unicorn/socket.rb deleted file mode 100644 index 4913261..0000000 --- a/lib/unicorn/socket.rb +++ /dev/null @@ -1,142 +0,0 @@ -require 'fcntl' -require 'socket' -require 'io/nonblock' - -# non-portable Socket code goes here: -class Socket - module Constants - # configure platform-specific options (only tested on Linux 2.6 so far) - case RUBY_PLATFORM - when /linux/ - # from /usr/include/linux/tcp.h - TCP_DEFER_ACCEPT = 9 unless defined?(TCP_DEFER_ACCEPT) - TCP_CORK = 3 unless defined?(TCP_CORK) - when /freebsd(([1-4]\..{1,2})|5\.[0-4])/ - when /freebsd/ - # Use the HTTP accept filter if available. - # The struct made by pack() is defined in /usr/include/sys/socket.h - # as accept_filter_arg - unless `/sbin/sysctl -nq net.inet.accf.http`.empty? - unless defined?(SO_ACCEPTFILTER_HTTPREADY) - SO_ACCEPTFILTER_HTTPREADY = ['httpready',nil].pack('a16a240').freeze - end - - end - end - end -end - -class UNIXSocket - UNICORN_PEERADDR = '127.0.0.1'.freeze - def unicorn_peeraddr - UNICORN_PEERADDR - end -end - -class TCPSocket - def unicorn_peeraddr - peeraddr.last - end -end - -module Unicorn - module SocketHelper - include Socket::Constants - - def set_client_sockopt(sock) - sock.setsockopt(IPPROTO_TCP, TCP_NODELAY, 1) if defined?(TCP_NODELAY) - sock.setsockopt(SOL_TCP, TCP_CORK, 1) if defined?(TCP_CORK) - end - - def set_cloexec(io) - io.fcntl(Fcntl::F_SETFD, Fcntl::FD_CLOEXEC) if defined?(Fcntl::FD_CLOEXEC) - end - - def set_server_sockopt(sock) - if defined?(TCP_DEFER_ACCEPT) - sock.setsockopt(SOL_TCP, TCP_DEFER_ACCEPT, 1) rescue nil - end - if defined?(SO_ACCEPTFILTER_HTTPREADY) - sock.setsockopt(SOL_SOCKET, SO_ACCEPTFILTER, - SO_ACCEPTFILTER_HTTPREADY) rescue nil - end - end - - def destroy_safely(io) - if io.respond_to?(:path) && File.stat(io.path).ino == io.stat.ino - File.unlink(io.path) rescue nil - end - io.close rescue nil - end - - # creates a new server, socket. address may be a HOST:PORT or - # an absolute path to a UNIX socket. address can even be a Socket - # object in which case it is immediately returned - def bind_listen(address = '0.0.0.0:8080', backlog = 1024) - return address unless String === address - - domain, bind_addr = if address[0..0] == "/" - if File.exist?(address) - if File.socket?(address) - if self.respond_to?(:logger) - logger.info "unlinking existing socket=#{address}" - end - File.unlink(address) - else - raise ArgumentError, - "socket=#{address} specified but it is not a socket!" - end - end - [ AF_UNIX, Socket.pack_sockaddr_un(address) ] - elsif address =~ /^(\d+\.\d+\.\d+\.\d+):(\d+)$/ - [ AF_INET, Socket.pack_sockaddr_in($2.to_i, $1) ] - else - raise ArgumentError, "Don't know how to bind: #{address}" - end - - sock = Socket.new(domain, SOCK_STREAM, 0) - sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1) if defined?(SO_REUSEADDR) - begin - sock.bind(bind_addr) - rescue Errno::EADDRINUSE - sock.close rescue nil - return nil - end - sock.listen(backlog) - set_server_sockopt(sock) if domain == AF_INET - sock - end - - # Returns the configuration name of a socket as a string. sock may - # be a string value, in which case it is returned as-is - # Warning: TCP sockets may not always return the name given to it. - def sock_name(sock) - case sock - when String then sock - when UNIXServer - Socket.unpack_sockaddr_un(sock.getsockname) - when TCPServer - Socket.unpack_sockaddr_in(sock.getsockname).reverse!.join(':') - when Socket - begin - Socket.unpack_sockaddr_in(sock.getsockname).reverse!.join(':') - rescue ArgumentError - Socket.unpack_sockaddr_un(sock.getsockname) - end - else - raise ArgumentError, "Unhandled class #{sock.class}: #{sock.inspect}" - end - end - - # casts a given Socket to be a TCPServer or UNIXServer - def server_cast(sock) - begin - Socket.unpack_sockaddr_in(sock.getsockname) - TCPServer.for_fd(sock.fileno) - rescue ArgumentError - UNIXServer.for_fd(sock.fileno) - end - end - - end # module SocketHelper -end # module Unicorn diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb new file mode 100644 index 0000000..850ad85 --- /dev/null +++ b/lib/unicorn/socket_helper.rb @@ -0,0 +1,90 @@ +require 'socket' + +module Unicorn + module SocketHelper + include Socket::Constants + + def set_server_sockopt(sock, opt) + opt ||= {} + if opt[:rcvbuf] || opt[:sndbuf] + log_buffer_sizes(sock, "before: ") + sock.setsockopt(SOL_SOCKET, SO_RCVBUF, opt[:rcvbuf]) if opt[:rcvbuf] + sock.setsockopt(SOL_SOCKET, SO_SNDBUF, opt[:sndbuf]) if opt[:sndbuf] + log_buffer_sizes(sock, " after: ") + end + sock.listen(opt[:backlog] || 1024) + end + + def log_buffer_sizes(sock, pfx = '') + respond_to?(:logger) or return + rcvbuf = sock.getsockopt(SOL_SOCKET, SO_RCVBUF).unpack('i') + sndbuf = sock.getsockopt(SOL_SOCKET, SO_SNDBUF).unpack('i') + logger.info "#{pfx}#{sock_name(sock)} rcvbuf=#{rcvbuf} sndbuf=#{sndbuf}" + end + + # creates a new server, socket. address may be a HOST:PORT or + # an absolute path to a UNIX socket. address can even be a Socket + # object in which case it is immediately returned + def bind_listen(address = '0.0.0.0:8080', opt = { :backlog => 1024 }) + return address unless String === address + + sock = if address[0..0] == "/" + if File.exist?(address) + if File.socket?(address) + if self.respond_to?(:logger) + logger.info "unlinking existing socket=#{address}" + end + File.unlink(address) + else + raise ArgumentError, + "socket=#{address} specified but it is not a socket!" + end + end + old_umask = File.umask(0) + begin + UNIXServer.new(address) + ensure + File.umask(old_umask) + end + elsif address =~ /^(\d+\.\d+\.\d+\.\d+):(\d+)$/ + TCPServer.new($1, $2.to_i) + else + raise ArgumentError, "Don't know how to bind: #{address}" + end + set_server_sockopt(sock, opt) + sock + end + + # Returns the configuration name of a socket as a string. sock may + # be a string value, in which case it is returned as-is + # Warning: TCP sockets may not always return the name given to it. + def sock_name(sock) + case sock + when String then sock + when UNIXServer + Socket.unpack_sockaddr_un(sock.getsockname) + when TCPServer + Socket.unpack_sockaddr_in(sock.getsockname).reverse!.join(':') + when Socket + begin + Socket.unpack_sockaddr_in(sock.getsockname).reverse!.join(':') + rescue ArgumentError + Socket.unpack_sockaddr_un(sock.getsockname) + end + else + raise ArgumentError, "Unhandled class #{sock.class}: #{sock.inspect}" + end + end + + # casts a given Socket to be a TCPServer or UNIXServer + def server_cast(sock) + begin + Socket.unpack_sockaddr_in(sock.getsockname) + TCPServer.for_fd(sock.fileno) + rescue ArgumentError + UNIXServer.for_fd(sock.fileno) + end + end + + end # module SocketHelper +end # module Unicorn diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb index 0400fd0..2d3f827 100644 --- a/lib/unicorn/util.rb +++ b/lib/unicorn/util.rb @@ -4,6 +4,8 @@ module Unicorn class Util class << self + APPEND_FLAGS = File::WRONLY | File::APPEND + # this reopens logs that have been rotated (using logrotate(8) or # similar). It is recommended that you install # A +File+ object is considered for reopening if it is: @@ -17,17 +19,20 @@ module Unicorn ObjectSpace.each_object(File) do |fp| next if fp.closed? next unless (fp.sync && fp.path[0..0] == "/") - - flags = fp.fcntl(Fcntl::F_GETFL) - open_flags = File::WRONLY | File::APPEND - next unless (flags & open_flags) == open_flags + next unless (fp.fcntl(Fcntl::F_GETFL) & APPEND_FLAGS) == APPEND_FLAGS begin a, b = fp.stat, File.stat(fp.path) next if a.ino == b.ino && a.dev == b.dev rescue Errno::ENOENT end - fp.reopen(fp.path, "a") + + open_arg = 'a' + if fp.respond_to?(:external_encoding) && enc = fp.external_encoding + open_arg << ":#{enc.to_s}" + enc = fp.internal_encoding and open_arg << ":#{enc.to_s}" + end + fp.reopen(fp.path, open_arg) fp.sync = true nr += 1 end # each_object diff --git a/local.mk.sample b/local.mk.sample new file mode 100644 index 0000000..84bcf44 --- /dev/null +++ b/local.mk.sample @@ -0,0 +1,44 @@ +# this is the local.mk file used by Eric Wong on his dev boxes. +# GNUmakefile will source local.mk in the top-level source tree +# if it is present. +# +# This is depends on a bunch of GNU-isms from bash, sed, touch. + +DLEXT := so +rack_ver := 1.0.0 + +# Avoid loading rubygems to speed up tests because gmake is +# fork+exec heavy with Ruby. +ifeq ($(r19),) + ruby := $(HOME)/bin/ruby + RUBYLIB := $(HOME)/lib/ruby/gems/1.8/gems/rack-$(rack_ver)/lib +else + export PATH := $(HOME)/ruby-1.9/bin:$(PATH) + ruby := $(HOME)/ruby-1.9/bin/ruby --disable-gems + RUBYLIB := $(HOME)/ruby-1.9/lib/ruby/gems/1.9.1/gems/rack-$(rack_ver)/lib +endif + +# pipefail is THE reason to use bash (v3+) +SHELL := /bin/bash -e -o pipefail + +full-test: test-18 test-19 +test-18: + $(MAKE) test test-rails 2>&1 | sed -u -e 's!^!1.8 !' +test-19: + $(MAKE) test test-rails r19=1 2>&1 | sed -u -e 's!^!1.9 !' + +# publishes docs to http://unicorn.bogomips.org +publish_doc: + -git set-file-times + $(MAKE) doc + $(MAKE) doc_gz + rsync -av --delete doc/ dcvr:/srv/unicorn/ + git ls-files | xargs touch + +# Create gzip variants of the same timestamp as the original so nginx +# "gzip_static on" can serve the gzipped versions directly. +doc_gz: suf := html js css +doc_gz: globs := $(addprefix doc/*.,$(suf)) $(addprefix doc/*/*.,$(suf)) +doc_gz: docs := $(wildcard $(globs)) +doc_gz: + for i in $(docs); do gzip < $$i > $$i.gz; touch -r $$i $$i.gz; done diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb index 712037c..014b270 100644 --- a/test/exec/test_exec.rb +++ b/test/exec/test_exec.rb @@ -1,43 +1,34 @@ # Copyright (c) 2009 Eric Wong -STDIN.sync = STDOUT.sync = STDERR.sync = true require 'test/test_helper' -require 'pathname' -require 'tempfile' -require 'fileutils' do_test = true -DEFAULT_TRIES = 1000 -DEFAULT_RES = 0.2 - $unicorn_bin = ENV['UNICORN_TEST_BIN'] || "unicorn" redirect_test_io do do_test = system($unicorn_bin, '-v') end unless do_test - STDERR.puts "#{$unicorn_bin} not found in PATH=#{ENV['PATH']}, " \ - "skipping this test" + warn "#{$unicorn_bin} not found in PATH=#{ENV['PATH']}, " \ + "skipping this test" end -begin - require 'rack' -rescue LoadError - STDERR.puts "Unable to load Rack, skipping this test" +unless try_require('rack') + warn "Unable to load Rack, skipping this test" do_test = false end class ExecTest < Test::Unit::TestCase - trap('QUIT', 'IGNORE') + trap(:QUIT, 'IGNORE') HI = <<-EOS use Rack::ContentLength -run proc { |env| [ 200, { 'Content-Type' => 'text/plain' }, "HI\\n" ] } +run proc { |env| [ 200, { 'Content-Type' => 'text/plain' }, [ "HI\\n" ] ] } EOS HELLO = <<-EOS class Hello def call(env) - [ 200, { 'Content-Type' => 'text/plain' }, "HI\\n" ] + [ 200, { 'Content-Type' => 'text/plain' }, [ "HI\\n" ] ] end end EOS @@ -47,10 +38,9 @@ end HEAVY_CFG = <<-EOS worker_processes 4 timeout 30 -backlog 128 logger Logger.new('#{COMMON_TMP.path}') -before_fork do |server, worker_nr| - server.logger.info "before_fork: worker=\#{worker_nr}" +before_fork do |server, worker| + server.logger.info "before_fork: worker=\#{worker.nr}" end EOS @@ -82,6 +72,22 @@ end end end + def test_exit_signals + %w(INT TERM QUIT).each do |sig| + File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } + pid = xfork { redirect_test_io { exec($unicorn_bin, "-l#@addr:#@port") } } + wait_master_ready("test_stderr.#{pid}.log") + status = nil + assert_nothing_raised do + Process.kill(sig, pid) + pid, status = Process.waitpid2(pid) + end + reaped = File.readlines("test_stderr.#{pid}.log").grep(/reaped/) + assert_equal 1, reaped.size + assert status.exited? + end + end + def test_basic File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } pid = fork do @@ -92,6 +98,28 @@ end assert_shutdown(pid) end + def test_ttin_ttou + File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } + pid = fork { redirect_test_io { exec($unicorn_bin, "-l#@addr:#@port") } } + log = "test_stderr.#{pid}.log" + wait_master_ready(log) + [ 2, 3].each { |i| + assert_nothing_raised { Process.kill(:TTIN, pid) } + wait_workers_ready(log, i) + } + File.truncate(log, 0) + reaped = nil + [ 2, 1, 0].each { |i| + assert_nothing_raised { Process.kill(:TTOU, pid) } + DEFAULT_TRIES.times { + sleep DEFAULT_RES + reaped = File.readlines(log).grep(/reaped.*\s*worker=#{i}$/) + break if reaped.size == 1 + } + assert_equal 1, reaped.size + } + end + def test_help redirect_test_io do assert(system($unicorn_bin, "-h"), "help text returns true") @@ -113,7 +141,7 @@ end pid_file = "#{@tmpdir}/test.pid" old_file = "#{pid_file}.oldbin" ucfg = Tempfile.new('unicorn_test_config') - ucfg.syswrite("listeners %w(#{@addr}:#{@port})\n") + ucfg.syswrite("listen %(#@addr:#@port)\n") ucfg.syswrite("pid %(#{pid_file})\n") ucfg.syswrite("logger Logger.new(%(#{@tmpdir}/log))\n") pid = xfork do @@ -126,14 +154,16 @@ end wait_for_file(pid_file) Process.waitpid(pid) - Process.kill('USR2', File.read(pid_file).to_i) + Process.kill(:USR2, File.read(pid_file).to_i) wait_for_file(old_file) wait_for_file(pid_file) - Process.kill('QUIT', File.read(old_file).to_i) + old_pid = File.read(old_file).to_i + Process.kill(:QUIT, old_pid) + wait_for_death(old_pid) ucfg.syswrite("timeout %(#{pid_file})\n") # introduce a bug current_pid = File.read(pid_file).to_i - Process.kill('USR2', current_pid) + Process.kill(:USR2, current_pid) # wait for pid_file to restore itself tries = DEFAULT_TRIES @@ -156,9 +186,11 @@ end # fix the bug ucfg.sysseek(0) ucfg.truncate(0) - ucfg.syswrite("listeners %w(#{@addr}:#{@port} #{@addr}:#{port2})\n") + ucfg.syswrite("listen %(#@addr:#@port)\n") + ucfg.syswrite("listen %(#@addr:#{port2})\n") ucfg.syswrite("pid %(#{pid_file})\n") - Process.kill('USR2', current_pid) + assert_nothing_raised { Process.kill(:USR2, current_pid) } + wait_for_file(old_file) wait_for_file(pid_file) new_pid = File.read(pid_file).to_i @@ -170,8 +202,8 @@ end assert_equal String, results[1].class assert_nothing_raised do - Process.kill('QUIT', current_pid) - Process.kill('QUIT', new_pid) + Process.kill(:QUIT, current_pid) + Process.kill(:QUIT, new_pid) end end @@ -192,14 +224,16 @@ end wait_for_file(pid_file) Process.waitpid(pid) - Process.kill('USR2', File.read(pid_file).to_i) + Process.kill(:USR2, File.read(pid_file).to_i) wait_for_file(old_file) wait_for_file(pid_file) - Process.kill('QUIT', File.read(old_file).to_i) + old_pid = File.read(old_file).to_i + Process.kill(:QUIT, old_pid) + wait_for_death(old_pid) File.unlink("config.ru") # break reloading current_pid = File.read(pid_file).to_i - Process.kill('USR2', current_pid) + Process.kill(:USR2, current_pid) # wait for pid_file to restore itself tries = DEFAULT_TRIES @@ -210,17 +244,17 @@ end rescue Errno::ENOENT (sleep(DEFAULT_RES) and (tries -= 1) > 0) and retry end - assert_equal current_pid, File.read(pid_file).to_i tries = DEFAULT_TRIES while File.exist?(old_file) (sleep(DEFAULT_RES) and (tries -= 1) > 0) or break end assert ! File.exist?(old_file), "oldbin=#{old_file} gone" + assert_equal current_pid, File.read(pid_file).to_i # fix the bug File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } - Process.kill('USR2', current_pid) + assert_nothing_raised { Process.kill(:USR2, current_pid) } wait_for_file(old_file) wait_for_file(pid_file) new_pid = File.read(pid_file).to_i @@ -230,25 +264,86 @@ end assert_equal String, results[0].class assert_nothing_raised do - Process.kill('QUIT', current_pid) - Process.kill('QUIT', new_pid) + Process.kill(:QUIT, current_pid) + Process.kill(:QUIT, new_pid) end end - def test_unicorn_config_listeners_overrides_cli - port2 = unused_port(@addr) + def test_unicorn_config_listener_swap + port_cli = unused_port File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } - # listeners = [ ... ] => should _override_ command-line options ucfg = Tempfile.new('unicorn_test_config') - ucfg.syswrite("listeners %w(#{@addr}:#{@port})\n") + ucfg.syswrite("listen '#@addr:#@port'\n") pid = xfork do redirect_test_io do - exec($unicorn_bin, "-c#{ucfg.path}", "-l#{@addr}:#{port2}") + exec($unicorn_bin, "-c#{ucfg.path}", "-l#@addr:#{port_cli}") end end + results = retry_hit(["http://#@addr:#{port_cli}/"]) + assert_equal String, results[0].class + results = retry_hit(["http://#@addr:#@port/"]) + assert_equal String, results[0].class + + port2 = unused_port(@addr) + ucfg.sysseek(0) + ucfg.truncate(0) + ucfg.syswrite("listen '#@addr:#{port2}'\n") + Process.kill(:HUP, pid) + + results = retry_hit(["http://#@addr:#{port2}/"]) + assert_equal String, results[0].class + results = retry_hit(["http://#@addr:#{port_cli}/"]) + assert_equal String, results[0].class + assert_nothing_raised do + reuse = TCPServer.new(@addr, @port) + reuse.close + end + assert_shutdown(pid) + end + + def test_unicorn_config_listen_with_options + File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } + ucfg = Tempfile.new('unicorn_test_config') + ucfg.syswrite("listen '#{@addr}:#{@port}', :backlog => 512,\n") + ucfg.syswrite(" :rcvbuf => 4096,\n") + ucfg.syswrite(" :sndbuf => 4096\n") + pid = xfork do + redirect_test_io { exec($unicorn_bin, "-c#{ucfg.path}") } + end + results = retry_hit(["http://#{@addr}:#{@port}/"]) + assert_equal String, results[0].class + assert_shutdown(pid) + end + + def test_unicorn_config_per_worker_listen + port2 = unused_port + pid_spit = 'use Rack::ContentLength;' \ + 'run proc { |e| [ 200, {"Content-Type"=>"text/plain"}, ["#$$\\n"] ] }' + File.open("config.ru", "wb") { |fp| fp.syswrite(pid_spit) } + tmp = Tempfile.new('test.socket') + File.unlink(tmp.path) + ucfg = Tempfile.new('unicorn_test_config') + ucfg.syswrite("listen '#@addr:#@port'\n") + ucfg.syswrite("before_fork { |s,w|\n") + ucfg.syswrite(" s.listen('#{tmp.path}', :backlog => 5, :sndbuf => 8192)\n") + ucfg.syswrite(" s.listen('#@addr:#{port2}', :rcvbuf => 8192)\n") + ucfg.syswrite("\n}\n") + pid = xfork do + redirect_test_io { exec($unicorn_bin, "-c#{ucfg.path}") } + end results = retry_hit(["http://#{@addr}:#{@port}/"]) - assert_raises(Errno::ECONNREFUSED) { TCPSocket.new(@addr, port2) } assert_equal String, results[0].class + worker_pid = results[0].to_i + assert_not_equal pid, worker_pid + s = UNIXSocket.new(tmp.path) + s.syswrite("GET / HTTP/1.0\r\n\r\n") + results = '' + loop { results << s.sysread(4096) } rescue nil + assert_nothing_raised { s.close } + assert_equal worker_pid, results.split(/\r\n/).last.to_i + results = hit(["http://#@addr:#{port2}/"]) + assert_equal String, results[0].class + assert_equal worker_pid, results[0].to_i assert_shutdown(pid) end @@ -283,34 +378,36 @@ end results = retry_hit(["http://#{@addr}:#{@port}/"]) assert_equal String, results[0].class wait_master_ready(COMMON_TMP.path) + wait_workers_ready(COMMON_TMP.path, 4) bf = File.readlines(COMMON_TMP.path).grep(/\bbefore_fork: worker=/) assert_equal 4, bf.size rotate = Tempfile.new('unicorn_rotate') assert_nothing_raised do File.rename(COMMON_TMP.path, rotate.path) - Process.kill('USR1', pid) + Process.kill(:USR1, pid) end wait_for_file(COMMON_TMP.path) assert File.exist?(COMMON_TMP.path), "#{COMMON_TMP.path} exists" # USR1 should've been passed to all workers tries = DEFAULT_TRIES log = File.readlines(rotate.path) - while (tries -= 1) > 0 && log.grep(/rotating logs\.\.\./).size < 4 + while (tries -= 1) > 0 && + log.grep(/reopening logs\.\.\./).size < 5 sleep DEFAULT_RES log = File.readlines(rotate.path) end - assert_equal 4, log.grep(/rotating logs\.\.\./).size - assert_equal 0, log.grep(/done rotating logs/).size + assert_equal 5, log.grep(/reopening logs\.\.\./).size + assert_equal 0, log.grep(/done reopening logs/).size tries = DEFAULT_TRIES log = File.readlines(COMMON_TMP.path) - while (tries -= 1) > 0 && log.grep(/done rotating logs/).size < 4 + while (tries -= 1) > 0 && log.grep(/done reopening logs/).size < 5 sleep DEFAULT_RES log = File.readlines(COMMON_TMP.path) end - assert_equal 4, log.grep(/done rotating logs/).size - assert_equal 0, log.grep(/rotating logs\.\.\./).size - assert_nothing_raised { Process.kill('QUIT', pid) } + assert_equal 5, log.grep(/done reopening logs/).size + assert_equal 0, log.grep(/reopening logs\.\.\./).size + assert_nothing_raised { Process.kill(:QUIT, pid) } status = nil assert_nothing_raised { pid, status = Process.waitpid2(pid) } assert status.success?, "exited successfully" @@ -380,6 +477,39 @@ end reexec_basic_test(pid, pid_file) end + def test_socket_unlinked_restore + results = nil + sock = Tempfile.new('unicorn_test_sock') + sock_path = sock.path + @sockets << sock_path + sock.close! + ucfg = Tempfile.new('unicorn_test_config') + ucfg.syswrite("listen \"#{sock_path}\"\n") + + File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } + pid = xfork { redirect_test_io { exec($unicorn_bin, "-c#{ucfg.path}") } } + wait_for_file(sock_path) + assert File.socket?(sock_path) + assert_nothing_raised do + sock = UNIXSocket.new(sock_path) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + results = sock.sysread(4096) + end + assert_equal String, results.class + assert_nothing_raised do + File.unlink(sock_path) + Process.kill(:HUP, pid) + end + wait_for_file(sock_path) + assert File.socket?(sock_path) + assert_nothing_raised do + sock = UNIXSocket.new(sock_path) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + results = sock.sysread(4096) + end + assert_equal String, results.class + end + def test_unicorn_config_file pid_file = "#{@tmpdir}/test.pid" sock = Tempfile.new('unicorn_test_sock') @@ -415,7 +545,7 @@ end assert_equal String, results.class # try reloading the config - sock = Tempfile.new('unicorn_test_sock') + sock = Tempfile.new('new_test_sock') new_sock_path = sock.path @sockets << new_sock_path sock.close! @@ -425,11 +555,12 @@ end assert_nothing_raised do ucfg = File.open(ucfg.path, "wb") + ucfg.syswrite("listen \"#{sock_path}\"\n") ucfg.syswrite("listen \"#{new_sock_path}\"\n") ucfg.syswrite("pid \"#{pid_file}\"\n") ucfg.syswrite("logger Logger.new('#{new_log.path}')\n") ucfg.close - Process.kill('HUP', pid) + Process.kill(:HUP, pid) end wait_for_file(new_sock_path) @@ -472,99 +603,72 @@ end reexec_usr2_quit_test(new_pid, pid_file) end - private + def test_reexec_fd_leak + unless RUBY_PLATFORM =~ /linux/ # Solaris may work, too, but I forget... + warn "FD leak test only works on Linux at the moment" + return + end + pid_file = "#{@tmpdir}/test.pid" + log = Tempfile.new('unicorn_test_log') + log.sync = true + ucfg = Tempfile.new('unicorn_test_config') + ucfg.syswrite("pid \"#{pid_file}\"\n") + ucfg.syswrite("logger Logger.new('#{log.path}')\n") + ucfg.syswrite("stderr_path '#{log.path}'\n") + ucfg.syswrite("stdout_path '#{log.path}'\n") + ucfg.close - # sometimes the server may not come up right away - def retry_hit(uris = []) - tries = DEFAULT_TRIES - begin - hit(uris) - rescue Errno::ECONNREFUSED => err - if (tries -= 1) > 0 - sleep DEFAULT_RES - retry - end - raise err + File.open("config.ru", "wb") { |fp| fp.syswrite(HI) } + pid = xfork do + redirect_test_io do + exec($unicorn_bin, "-D", "-l#{@addr}:#{@port}", "-c#{ucfg.path}") end end - def assert_shutdown(pid) - wait_master_ready("#{@tmpdir}/test_stderr.#{pid}.log") - assert_nothing_raised { Process.kill('QUIT', pid) } - status = nil - assert_nothing_raised { pid, status = Process.waitpid2(pid) } - assert status.success?, "exited successfully" - end + wait_master_ready(log.path) + File.truncate(log.path, 0) + wait_for_file(pid_file) + orig_pid = pid = File.read(pid_file).to_i + orig_fds = `ls -l /proc/#{pid}/fd`.split(/\n/) + assert $?.success? + expect_size = orig_fds.size - def wait_master_ready(master_log) - tries = DEFAULT_TRIES - while (tries -= 1) > 0 - begin - File.readlines(master_log).grep(/master process ready/)[0] and return - rescue Errno::ENOENT - end - sleep DEFAULT_RES - end - raise "master process never became ready" + assert_nothing_raised do + Process.kill(:USR2, pid) + wait_for_file("#{pid_file}.oldbin") + Process.kill(:QUIT, pid) end + wait_for_death(pid) + + wait_master_ready(log.path) + File.truncate(log.path, 0) + wait_for_file(pid_file) + pid = File.read(pid_file).to_i + assert_not_equal orig_pid, pid + curr_fds = `ls -l /proc/#{pid}/fd`.split(/\n/) + assert $?.success? - def reexec_usr2_quit_test(pid, pid_file) - assert File.exist?(pid_file), "pid file OK" - assert ! File.exist?("#{pid_file}.oldbin"), "oldbin pid file" - assert_nothing_raised { Process.kill('USR2', pid) } - assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } + # we could've inherited descriptors the first time around + assert expect_size >= curr_fds.size, curr_fds.inspect + expect_size = curr_fds.size + + assert_nothing_raised do + Process.kill(:USR2, pid) wait_for_file("#{pid_file}.oldbin") - wait_for_file(pid_file) - - # kill old master process - assert_not_equal pid, File.read(pid_file).to_i - assert_equal pid, File.read("#{pid_file}.oldbin").to_i - assert_nothing_raised { Process.kill('QUIT', pid) } - assert_not_equal pid, File.read(pid_file).to_i - assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } - wait_for_file(pid_file) - assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } - assert_nothing_raised { Process.kill('QUIT', File.read(pid_file).to_i) } - end - - def reexec_basic_test(pid, pid_file) - results = retry_hit(["http://#{@addr}:#{@port}/"]) - assert_equal String, results[0].class - assert_nothing_raised { Process.kill(0, pid) } - master_log = "#{@tmpdir}/test_stderr.#{pid}.log" - wait_master_ready(master_log) - File.truncate(master_log, 0) - nr = 50 - kill_point = 2 - assert_nothing_raised do - nr.times do |i| - hit(["http://#{@addr}:#{@port}/#{i}"]) - i == kill_point and Process.kill('HUP', pid) - end - end - wait_master_ready(master_log) - assert File.exist?(pid_file), "pid=#{pid_file} exists" - new_pid = File.read(pid_file).to_i - assert_not_equal pid, new_pid - assert_nothing_raised { Process.kill(0, new_pid) } - assert_nothing_raised { Process.kill('QUIT', new_pid) } + Process.kill(:QUIT, pid) end + wait_for_death(pid) - def wait_for_file(path) - tries = DEFAULT_TRIES - while (tries -= 1) > 0 && ! File.exist?(path) - sleep DEFAULT_RES - end - assert File.exist?(path), "path=#{path} exists" - end + wait_master_ready(log.path) + File.truncate(log.path, 0) + wait_for_file(pid_file) + pid = File.read(pid_file).to_i + curr_fds = `ls -l /proc/#{pid}/fd`.split(/\n/) + assert $?.success? + assert_equal expect_size, curr_fds.size, curr_fds.inspect - def xfork(&block) - fork do - ObjectSpace.each_object(Tempfile) do |tmp| - ObjectSpace.undefine_finalizer(tmp) - end - yield - end - end + Process.kill(:QUIT, pid) + wait_for_death(pid) + end end if do_test diff --git a/test/rails/app-1.2.3/.gitignore b/test/rails/app-1.2.3/.gitignore new file mode 100644 index 0000000..f451f91 --- /dev/null +++ b/test/rails/app-1.2.3/.gitignore @@ -0,0 +1,2 @@ +/tmp +/vendor diff --git a/test/rails/app-1.2.3/Rakefile b/test/rails/app-1.2.3/Rakefile new file mode 100644 index 0000000..fbebfca --- /dev/null +++ b/test/rails/app-1.2.3/Rakefile @@ -0,0 +1,7 @@ +require(File.join(File.dirname(__FILE__), 'config', 'boot')) + +require 'rake' +require 'rake/testtask' +require 'rake/rdoctask' + +require 'tasks/rails' diff --git a/test/rails/app-1.2.3/app/controllers/application.rb b/test/rails/app-1.2.3/app/controllers/application.rb new file mode 100644 index 0000000..ae8cac0 --- /dev/null +++ b/test/rails/app-1.2.3/app/controllers/application.rb @@ -0,0 +1,4 @@ +class ApplicationController < ActionController::Base + # Pick a unique cookie name to distinguish our session data from others' + session :session_key => "_unicorn_rails_test.#{rand}" +end diff --git a/test/rails/app-1.2.3/app/controllers/foo_controller.rb b/test/rails/app-1.2.3/app/controllers/foo_controller.rb new file mode 100644 index 0000000..8d877d1 --- /dev/null +++ b/test/rails/app-1.2.3/app/controllers/foo_controller.rb @@ -0,0 +1,34 @@ +require 'digest/sha1' +class FooController < ApplicationController + def index + render :text => "FOO\n" + end + + def xcookie + cookies["foo"] = "cookie #$$" + render :text => "" + end + + def xnotice + flash[:notice] = "session #$$" + render :text => "" + end + + def xpost + if request.post? + digest = Digest::SHA1.new + out = "params: #{params.inspect}\n" + if file = params[:file] + loop do + buf = file.read(4096) or break + digest.update(buf) + end + out << "sha1: #{digest.to_s}\n" + end + headers['content-type'] = 'text/plain' + render :text => out + else + render :status => 403, :text => "need post\n" + end + end +end diff --git a/test/rails/app-1.2.3/app/helpers/application_helper.rb b/test/rails/app-1.2.3/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/test/rails/app-1.2.3/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/test/rails/app-1.2.3/config/boot.rb b/test/rails/app-1.2.3/config/boot.rb new file mode 100644 index 0000000..71c7d7c --- /dev/null +++ b/test/rails/app-1.2.3/config/boot.rb @@ -0,0 +1,9 @@ +unless defined?(RAILS_ROOT) + root_path = File.join(File.dirname(__FILE__), '..') + RAILS_ROOT = root_path +end + +unless defined?(Rails::Initializer) + require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer" + Rails::Initializer.run(:set_load_path) +end diff --git a/test/rails/app-1.2.3/config/database.yml b/test/rails/app-1.2.3/config/database.yml new file mode 100644 index 0000000..9f77843 --- /dev/null +++ b/test/rails/app-1.2.3/config/database.yml @@ -0,0 +1,12 @@ +development: + adapter: sqlite3 + database: db/development.sqlite3 + timeout: 5000 +test: + adapter: sqlite3 + database: db/test.sqlite3 + timeout: 5000 +production: + adapter: sqlite3 + database: db/production.sqlite3 + timeout: 5000 diff --git a/test/rails/app-1.2.3/config/environment.rb b/test/rails/app-1.2.3/config/environment.rb new file mode 100644 index 0000000..2ef6b4a --- /dev/null +++ b/test/rails/app-1.2.3/config/environment.rb @@ -0,0 +1,11 @@ +unless defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION = ENV['UNICORN_RAILS_VERSION'] # || '1.2.3' +end + +# Bootstrap the Rails environment, frameworks, and default configuration +require File.join(File.dirname(__FILE__), 'boot') + +Rails::Initializer.run do |config| + config.frameworks -= [ :action_web_service, :action_mailer ] + config.action_controller.session_store = :active_record_store +end diff --git a/test/rails/app-1.2.3/config/environments/development.rb b/test/rails/app-1.2.3/config/environments/development.rb new file mode 100644 index 0000000..032fb46 --- /dev/null +++ b/test/rails/app-1.2.3/config/environments/development.rb @@ -0,0 +1,7 @@ +config.cache_classes = false +config.whiny_nils = true +config.breakpoint_server = true +config.action_controller.consider_all_requests_local = true +config.action_controller.perform_caching = false +config.action_view.cache_template_extensions = false +config.action_view.debug_rjs = true diff --git a/test/rails/app-1.2.3/config/environments/production.rb b/test/rails/app-1.2.3/config/environments/production.rb new file mode 100644 index 0000000..c4059e3 --- /dev/null +++ b/test/rails/app-1.2.3/config/environments/production.rb @@ -0,0 +1,3 @@ +config.cache_classes = true +config.action_controller.consider_all_requests_local = false +config.action_controller.perform_caching = true diff --git a/test/rails/app-1.2.3/config/routes.rb b/test/rails/app-1.2.3/config/routes.rb new file mode 100644 index 0000000..774028f --- /dev/null +++ b/test/rails/app-1.2.3/config/routes.rb @@ -0,0 +1,4 @@ +ActionController::Routing::Routes.draw do |map| + map.connect ':controller/:action/:id.:format' + map.connect ':controller/:action/:id' +end diff --git a/test/rails/app-1.2.3/db/.gitignore b/test/rails/app-1.2.3/db/.gitignore new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/rails/app-1.2.3/db/.gitignore diff --git a/test/rails/app-1.2.3/log/.gitignore b/test/rails/app-1.2.3/log/.gitignore new file mode 100644 index 0000000..397b4a7 --- /dev/null +++ b/test/rails/app-1.2.3/log/.gitignore @@ -0,0 +1 @@ +*.log diff --git a/test/rails/app-1.2.3/public/404.html b/test/rails/app-1.2.3/public/404.html new file mode 100644 index 0000000..44d986c --- /dev/null +++ b/test/rails/app-1.2.3/public/404.html @@ -0,0 +1 @@ +404 Not Found diff --git a/test/rails/app-1.2.3/public/500.html b/test/rails/app-1.2.3/public/500.html new file mode 100644 index 0000000..e534a49 --- /dev/null +++ b/test/rails/app-1.2.3/public/500.html @@ -0,0 +1 @@ +500 Internal Server Error diff --git a/test/rails/app-2.0.2/.gitignore b/test/rails/app-2.0.2/.gitignore new file mode 100644 index 0000000..f451f91 --- /dev/null +++ b/test/rails/app-2.0.2/.gitignore @@ -0,0 +1,2 @@ +/tmp +/vendor diff --git a/test/rails/app-2.0.2/Rakefile b/test/rails/app-2.0.2/Rakefile new file mode 100644 index 0000000..fbebfca --- /dev/null +++ b/test/rails/app-2.0.2/Rakefile @@ -0,0 +1,7 @@ +require(File.join(File.dirname(__FILE__), 'config', 'boot')) + +require 'rake' +require 'rake/testtask' +require 'rake/rdoctask' + +require 'tasks/rails' diff --git a/test/rails/app-2.0.2/app/controllers/application.rb b/test/rails/app-2.0.2/app/controllers/application.rb new file mode 100644 index 0000000..09705d1 --- /dev/null +++ b/test/rails/app-2.0.2/app/controllers/application.rb @@ -0,0 +1,2 @@ +class ApplicationController < ActionController::Base +end diff --git a/test/rails/app-2.0.2/app/controllers/foo_controller.rb b/test/rails/app-2.0.2/app/controllers/foo_controller.rb new file mode 100644 index 0000000..8d877d1 --- /dev/null +++ b/test/rails/app-2.0.2/app/controllers/foo_controller.rb @@ -0,0 +1,34 @@ +require 'digest/sha1' +class FooController < ApplicationController + def index + render :text => "FOO\n" + end + + def xcookie + cookies["foo"] = "cookie #$$" + render :text => "" + end + + def xnotice + flash[:notice] = "session #$$" + render :text => "" + end + + def xpost + if request.post? + digest = Digest::SHA1.new + out = "params: #{params.inspect}\n" + if file = params[:file] + loop do + buf = file.read(4096) or break + digest.update(buf) + end + out << "sha1: #{digest.to_s}\n" + end + headers['content-type'] = 'text/plain' + render :text => out + else + render :status => 403, :text => "need post\n" + end + end +end diff --git a/test/rails/app-2.0.2/app/helpers/application_helper.rb b/test/rails/app-2.0.2/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/test/rails/app-2.0.2/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/test/rails/app-2.0.2/config/boot.rb b/test/rails/app-2.0.2/config/boot.rb new file mode 100644 index 0000000..71c7d7c --- /dev/null +++ b/test/rails/app-2.0.2/config/boot.rb @@ -0,0 +1,9 @@ +unless defined?(RAILS_ROOT) + root_path = File.join(File.dirname(__FILE__), '..') + RAILS_ROOT = root_path +end + +unless defined?(Rails::Initializer) + require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer" + Rails::Initializer.run(:set_load_path) +end diff --git a/test/rails/app-2.0.2/config/database.yml b/test/rails/app-2.0.2/config/database.yml new file mode 100644 index 0000000..9f77843 --- /dev/null +++ b/test/rails/app-2.0.2/config/database.yml @@ -0,0 +1,12 @@ +development: + adapter: sqlite3 + database: db/development.sqlite3 + timeout: 5000 +test: + adapter: sqlite3 + database: db/test.sqlite3 + timeout: 5000 +production: + adapter: sqlite3 + database: db/production.sqlite3 + timeout: 5000 diff --git a/test/rails/app-2.0.2/config/environment.rb b/test/rails/app-2.0.2/config/environment.rb new file mode 100644 index 0000000..7c720f6 --- /dev/null +++ b/test/rails/app-2.0.2/config/environment.rb @@ -0,0 +1,15 @@ +unless defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION = ENV['UNICORN_RAILS_VERSION'] +end + +# Bootstrap the Rails environment, frameworks, and default configuration +require File.join(File.dirname(__FILE__), 'boot') + +Rails::Initializer.run do |config| + config.frameworks -= [ :action_web_service, :action_mailer ] + config.action_controller.session_store = :active_record_store + config.action_controller.session = { + :session_key => "_unicorn_rails_test.#{rand}", + :secret => "#{rand}#{rand}#{rand}#{rand}", + } +end diff --git a/test/rails/app-2.0.2/config/environments/development.rb b/test/rails/app-2.0.2/config/environments/development.rb new file mode 100644 index 0000000..6a613c1 --- /dev/null +++ b/test/rails/app-2.0.2/config/environments/development.rb @@ -0,0 +1,6 @@ +config.cache_classes = false +config.whiny_nils = true +config.action_controller.consider_all_requests_local = true +config.action_controller.perform_caching = false +config.action_view.cache_template_extensions = false +config.action_view.debug_rjs = true diff --git a/test/rails/app-2.0.2/config/environments/production.rb b/test/rails/app-2.0.2/config/environments/production.rb new file mode 100644 index 0000000..c4059e3 --- /dev/null +++ b/test/rails/app-2.0.2/config/environments/production.rb @@ -0,0 +1,3 @@ +config.cache_classes = true +config.action_controller.consider_all_requests_local = false +config.action_controller.perform_caching = true diff --git a/test/rails/app-2.0.2/config/routes.rb b/test/rails/app-2.0.2/config/routes.rb new file mode 100644 index 0000000..774028f --- /dev/null +++ b/test/rails/app-2.0.2/config/routes.rb @@ -0,0 +1,4 @@ +ActionController::Routing::Routes.draw do |map| + map.connect ':controller/:action/:id.:format' + map.connect ':controller/:action/:id' +end diff --git a/test/rails/app-2.0.2/db/.gitignore b/test/rails/app-2.0.2/db/.gitignore new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/rails/app-2.0.2/db/.gitignore diff --git a/test/rails/app-2.0.2/log/.gitignore b/test/rails/app-2.0.2/log/.gitignore new file mode 100644 index 0000000..397b4a7 --- /dev/null +++ b/test/rails/app-2.0.2/log/.gitignore @@ -0,0 +1 @@ +*.log diff --git a/test/rails/app-2.0.2/public/404.html b/test/rails/app-2.0.2/public/404.html new file mode 100644 index 0000000..44d986c --- /dev/null +++ b/test/rails/app-2.0.2/public/404.html @@ -0,0 +1 @@ +404 Not Found diff --git a/test/rails/app-2.0.2/public/500.html b/test/rails/app-2.0.2/public/500.html new file mode 100644 index 0000000..e534a49 --- /dev/null +++ b/test/rails/app-2.0.2/public/500.html @@ -0,0 +1 @@ +500 Internal Server Error diff --git a/test/rails/app-2.1.2/.gitignore b/test/rails/app-2.1.2/.gitignore new file mode 100644 index 0000000..f451f91 --- /dev/null +++ b/test/rails/app-2.1.2/.gitignore @@ -0,0 +1,2 @@ +/tmp +/vendor diff --git a/test/rails/app-2.1.2/Rakefile b/test/rails/app-2.1.2/Rakefile new file mode 100644 index 0000000..fbebfca --- /dev/null +++ b/test/rails/app-2.1.2/Rakefile @@ -0,0 +1,7 @@ +require(File.join(File.dirname(__FILE__), 'config', 'boot')) + +require 'rake' +require 'rake/testtask' +require 'rake/rdoctask' + +require 'tasks/rails' diff --git a/test/rails/app-2.1.2/app/controllers/application.rb b/test/rails/app-2.1.2/app/controllers/application.rb new file mode 100644 index 0000000..09705d1 --- /dev/null +++ b/test/rails/app-2.1.2/app/controllers/application.rb @@ -0,0 +1,2 @@ +class ApplicationController < ActionController::Base +end diff --git a/test/rails/app-2.1.2/app/controllers/foo_controller.rb b/test/rails/app-2.1.2/app/controllers/foo_controller.rb new file mode 100644 index 0000000..8d877d1 --- /dev/null +++ b/test/rails/app-2.1.2/app/controllers/foo_controller.rb @@ -0,0 +1,34 @@ +require 'digest/sha1' +class FooController < ApplicationController + def index + render :text => "FOO\n" + end + + def xcookie + cookies["foo"] = "cookie #$$" + render :text => "" + end + + def xnotice + flash[:notice] = "session #$$" + render :text => "" + end + + def xpost + if request.post? + digest = Digest::SHA1.new + out = "params: #{params.inspect}\n" + if file = params[:file] + loop do + buf = file.read(4096) or break + digest.update(buf) + end + out << "sha1: #{digest.to_s}\n" + end + headers['content-type'] = 'text/plain' + render :text => out + else + render :status => 403, :text => "need post\n" + end + end +end diff --git a/test/rails/app-2.1.2/app/helpers/application_helper.rb b/test/rails/app-2.1.2/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/test/rails/app-2.1.2/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/test/rails/app-2.1.2/config/boot.rb b/test/rails/app-2.1.2/config/boot.rb new file mode 100644 index 0000000..0a51688 --- /dev/null +++ b/test/rails/app-2.1.2/config/boot.rb @@ -0,0 +1,109 @@ +# Don't change this file! +# Configure your app in config/environment.rb and config/environments/*.rb + +RAILS_ROOT = "#{File.dirname(__FILE__)}/.." unless defined?(RAILS_ROOT) + +module Rails + class << self + def boot! + unless booted? + preinitialize + pick_boot.run + end + end + + def booted? + defined? Rails::Initializer + end + + def pick_boot + (vendor_rails? ? VendorBoot : GemBoot).new + end + + def vendor_rails? + File.exist?("#{RAILS_ROOT}/vendor/rails") + end + + def preinitialize + load(preinitializer_path) if File.exist?(preinitializer_path) + end + + def preinitializer_path + "#{RAILS_ROOT}/config/preinitializer.rb" + end + end + + class Boot + def run + load_initializer + Rails::Initializer.run(:set_load_path) + end + end + + class VendorBoot < Boot + def load_initializer + require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer" + Rails::Initializer.run(:install_gem_spec_stubs) + end + end + + class GemBoot < Boot + def load_initializer + self.class.load_rubygems + load_rails_gem + require 'initializer' + end + + def load_rails_gem + if version = self.class.gem_version + gem 'rails', version + else + gem 'rails' + end + rescue Gem::LoadError => load_error + $stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v=#{version} rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.) + exit 1 + end + + class << self + def rubygems_version + Gem::RubyGemsVersion rescue nil + end + + def gem_version + if defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION + elsif ENV.include?('RAILS_GEM_VERSION') + ENV['RAILS_GEM_VERSION'] + else + parse_gem_version(read_environment_rb) + end + end + + def load_rubygems + require 'rubygems' + min_version = '1.3.1' + unless rubygems_version >= min_version + $stderr.puts %Q(Rails requires RubyGems >= #{min_version} (you have #{rubygems_version}). Please `gem update --system` and try again.) + exit 1 + end + + rescue LoadError + $stderr.puts %Q(Rails requires RubyGems >= #{min_version}. Please install RubyGems and try again: http://rubygems.rubyforge.org) + exit 1 + end + + def parse_gem_version(text) + $1 if text =~ /^[^#]*RAILS_GEM_VERSION\s*=\s*["']([!~<>=]*\s*[\d.]+)["']/ + end + + private + def read_environment_rb + File.read("#{RAILS_ROOT}/config/environment.rb") + end + end + end +end + +# All that for this: +Rails.boot! diff --git a/test/rails/app-2.1.2/config/database.yml b/test/rails/app-2.1.2/config/database.yml new file mode 100644 index 0000000..9f77843 --- /dev/null +++ b/test/rails/app-2.1.2/config/database.yml @@ -0,0 +1,12 @@ +development: + adapter: sqlite3 + database: db/development.sqlite3 + timeout: 5000 +test: + adapter: sqlite3 + database: db/test.sqlite3 + timeout: 5000 +production: + adapter: sqlite3 + database: db/production.sqlite3 + timeout: 5000 diff --git a/test/rails/app-2.1.2/config/environment.rb b/test/rails/app-2.1.2/config/environment.rb new file mode 100644 index 0000000..7c720f6 --- /dev/null +++ b/test/rails/app-2.1.2/config/environment.rb @@ -0,0 +1,15 @@ +unless defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION = ENV['UNICORN_RAILS_VERSION'] +end + +# Bootstrap the Rails environment, frameworks, and default configuration +require File.join(File.dirname(__FILE__), 'boot') + +Rails::Initializer.run do |config| + config.frameworks -= [ :action_web_service, :action_mailer ] + config.action_controller.session_store = :active_record_store + config.action_controller.session = { + :session_key => "_unicorn_rails_test.#{rand}", + :secret => "#{rand}#{rand}#{rand}#{rand}", + } +end diff --git a/test/rails/app-2.1.2/config/environments/development.rb b/test/rails/app-2.1.2/config/environments/development.rb new file mode 100644 index 0000000..7f49032 --- /dev/null +++ b/test/rails/app-2.1.2/config/environments/development.rb @@ -0,0 +1,5 @@ +config.cache_classes = false +config.whiny_nils = true +config.action_controller.consider_all_requests_local = true +config.action_controller.perform_caching = false +config.action_view.debug_rjs = true diff --git a/test/rails/app-2.1.2/config/environments/production.rb b/test/rails/app-2.1.2/config/environments/production.rb new file mode 100644 index 0000000..c4059e3 --- /dev/null +++ b/test/rails/app-2.1.2/config/environments/production.rb @@ -0,0 +1,3 @@ +config.cache_classes = true +config.action_controller.consider_all_requests_local = false +config.action_controller.perform_caching = true diff --git a/test/rails/app-2.1.2/config/routes.rb b/test/rails/app-2.1.2/config/routes.rb new file mode 100644 index 0000000..774028f --- /dev/null +++ b/test/rails/app-2.1.2/config/routes.rb @@ -0,0 +1,4 @@ +ActionController::Routing::Routes.draw do |map| + map.connect ':controller/:action/:id.:format' + map.connect ':controller/:action/:id' +end diff --git a/test/rails/app-2.1.2/db/.gitignore b/test/rails/app-2.1.2/db/.gitignore new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/rails/app-2.1.2/db/.gitignore diff --git a/test/rails/app-2.1.2/log/.gitignore b/test/rails/app-2.1.2/log/.gitignore new file mode 100644 index 0000000..397b4a7 --- /dev/null +++ b/test/rails/app-2.1.2/log/.gitignore @@ -0,0 +1 @@ +*.log diff --git a/test/rails/app-2.1.2/public/404.html b/test/rails/app-2.1.2/public/404.html new file mode 100644 index 0000000..44d986c --- /dev/null +++ b/test/rails/app-2.1.2/public/404.html @@ -0,0 +1 @@ +404 Not Found diff --git a/test/rails/app-2.1.2/public/500.html b/test/rails/app-2.1.2/public/500.html new file mode 100644 index 0000000..e534a49 --- /dev/null +++ b/test/rails/app-2.1.2/public/500.html @@ -0,0 +1 @@ +500 Internal Server Error diff --git a/test/rails/app-2.2.2/.gitignore b/test/rails/app-2.2.2/.gitignore new file mode 100644 index 0000000..f451f91 --- /dev/null +++ b/test/rails/app-2.2.2/.gitignore @@ -0,0 +1,2 @@ +/tmp +/vendor diff --git a/test/rails/app-2.2.2/Rakefile b/test/rails/app-2.2.2/Rakefile new file mode 100644 index 0000000..fbebfca --- /dev/null +++ b/test/rails/app-2.2.2/Rakefile @@ -0,0 +1,7 @@ +require(File.join(File.dirname(__FILE__), 'config', 'boot')) + +require 'rake' +require 'rake/testtask' +require 'rake/rdoctask' + +require 'tasks/rails' diff --git a/test/rails/app-2.2.2/app/controllers/application.rb b/test/rails/app-2.2.2/app/controllers/application.rb new file mode 100644 index 0000000..09705d1 --- /dev/null +++ b/test/rails/app-2.2.2/app/controllers/application.rb @@ -0,0 +1,2 @@ +class ApplicationController < ActionController::Base +end diff --git a/test/rails/app-2.2.2/app/controllers/foo_controller.rb b/test/rails/app-2.2.2/app/controllers/foo_controller.rb new file mode 100644 index 0000000..8d877d1 --- /dev/null +++ b/test/rails/app-2.2.2/app/controllers/foo_controller.rb @@ -0,0 +1,34 @@ +require 'digest/sha1' +class FooController < ApplicationController + def index + render :text => "FOO\n" + end + + def xcookie + cookies["foo"] = "cookie #$$" + render :text => "" + end + + def xnotice + flash[:notice] = "session #$$" + render :text => "" + end + + def xpost + if request.post? + digest = Digest::SHA1.new + out = "params: #{params.inspect}\n" + if file = params[:file] + loop do + buf = file.read(4096) or break + digest.update(buf) + end + out << "sha1: #{digest.to_s}\n" + end + headers['content-type'] = 'text/plain' + render :text => out + else + render :status => 403, :text => "need post\n" + end + end +end diff --git a/test/rails/app-2.2.2/app/helpers/application_helper.rb b/test/rails/app-2.2.2/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/test/rails/app-2.2.2/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/test/rails/app-2.2.2/config/boot.rb b/test/rails/app-2.2.2/config/boot.rb new file mode 100644 index 0000000..0a51688 --- /dev/null +++ b/test/rails/app-2.2.2/config/boot.rb @@ -0,0 +1,109 @@ +# Don't change this file! +# Configure your app in config/environment.rb and config/environments/*.rb + +RAILS_ROOT = "#{File.dirname(__FILE__)}/.." unless defined?(RAILS_ROOT) + +module Rails + class << self + def boot! + unless booted? + preinitialize + pick_boot.run + end + end + + def booted? + defined? Rails::Initializer + end + + def pick_boot + (vendor_rails? ? VendorBoot : GemBoot).new + end + + def vendor_rails? + File.exist?("#{RAILS_ROOT}/vendor/rails") + end + + def preinitialize + load(preinitializer_path) if File.exist?(preinitializer_path) + end + + def preinitializer_path + "#{RAILS_ROOT}/config/preinitializer.rb" + end + end + + class Boot + def run + load_initializer + Rails::Initializer.run(:set_load_path) + end + end + + class VendorBoot < Boot + def load_initializer + require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer" + Rails::Initializer.run(:install_gem_spec_stubs) + end + end + + class GemBoot < Boot + def load_initializer + self.class.load_rubygems + load_rails_gem + require 'initializer' + end + + def load_rails_gem + if version = self.class.gem_version + gem 'rails', version + else + gem 'rails' + end + rescue Gem::LoadError => load_error + $stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v=#{version} rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.) + exit 1 + end + + class << self + def rubygems_version + Gem::RubyGemsVersion rescue nil + end + + def gem_version + if defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION + elsif ENV.include?('RAILS_GEM_VERSION') + ENV['RAILS_GEM_VERSION'] + else + parse_gem_version(read_environment_rb) + end + end + + def load_rubygems + require 'rubygems' + min_version = '1.3.1' + unless rubygems_version >= min_version + $stderr.puts %Q(Rails requires RubyGems >= #{min_version} (you have #{rubygems_version}). Please `gem update --system` and try again.) + exit 1 + end + + rescue LoadError + $stderr.puts %Q(Rails requires RubyGems >= #{min_version}. Please install RubyGems and try again: http://rubygems.rubyforge.org) + exit 1 + end + + def parse_gem_version(text) + $1 if text =~ /^[^#]*RAILS_GEM_VERSION\s*=\s*["']([!~<>=]*\s*[\d.]+)["']/ + end + + private + def read_environment_rb + File.read("#{RAILS_ROOT}/config/environment.rb") + end + end + end +end + +# All that for this: +Rails.boot! diff --git a/test/rails/app-2.2.2/config/database.yml b/test/rails/app-2.2.2/config/database.yml new file mode 100644 index 0000000..9f77843 --- /dev/null +++ b/test/rails/app-2.2.2/config/database.yml @@ -0,0 +1,12 @@ +development: + adapter: sqlite3 + database: db/development.sqlite3 + timeout: 5000 +test: + adapter: sqlite3 + database: db/test.sqlite3 + timeout: 5000 +production: + adapter: sqlite3 + database: db/production.sqlite3 + timeout: 5000 diff --git a/test/rails/app-2.2.2/config/environment.rb b/test/rails/app-2.2.2/config/environment.rb new file mode 100644 index 0000000..7c720f6 --- /dev/null +++ b/test/rails/app-2.2.2/config/environment.rb @@ -0,0 +1,15 @@ +unless defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION = ENV['UNICORN_RAILS_VERSION'] +end + +# Bootstrap the Rails environment, frameworks, and default configuration +require File.join(File.dirname(__FILE__), 'boot') + +Rails::Initializer.run do |config| + config.frameworks -= [ :action_web_service, :action_mailer ] + config.action_controller.session_store = :active_record_store + config.action_controller.session = { + :session_key => "_unicorn_rails_test.#{rand}", + :secret => "#{rand}#{rand}#{rand}#{rand}", + } +end diff --git a/test/rails/app-2.2.2/config/environments/development.rb b/test/rails/app-2.2.2/config/environments/development.rb new file mode 100644 index 0000000..7f49032 --- /dev/null +++ b/test/rails/app-2.2.2/config/environments/development.rb @@ -0,0 +1,5 @@ +config.cache_classes = false +config.whiny_nils = true +config.action_controller.consider_all_requests_local = true +config.action_controller.perform_caching = false +config.action_view.debug_rjs = true diff --git a/test/rails/app-2.2.2/config/environments/production.rb b/test/rails/app-2.2.2/config/environments/production.rb new file mode 100644 index 0000000..c4059e3 --- /dev/null +++ b/test/rails/app-2.2.2/config/environments/production.rb @@ -0,0 +1,3 @@ +config.cache_classes = true +config.action_controller.consider_all_requests_local = false +config.action_controller.perform_caching = true diff --git a/test/rails/app-2.2.2/config/routes.rb b/test/rails/app-2.2.2/config/routes.rb new file mode 100644 index 0000000..774028f --- /dev/null +++ b/test/rails/app-2.2.2/config/routes.rb @@ -0,0 +1,4 @@ +ActionController::Routing::Routes.draw do |map| + map.connect ':controller/:action/:id.:format' + map.connect ':controller/:action/:id' +end diff --git a/test/rails/app-2.2.2/db/.gitignore b/test/rails/app-2.2.2/db/.gitignore new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/rails/app-2.2.2/db/.gitignore diff --git a/test/rails/app-2.2.2/log/.gitignore b/test/rails/app-2.2.2/log/.gitignore new file mode 100644 index 0000000..397b4a7 --- /dev/null +++ b/test/rails/app-2.2.2/log/.gitignore @@ -0,0 +1 @@ +*.log diff --git a/test/rails/app-2.2.2/public/404.html b/test/rails/app-2.2.2/public/404.html new file mode 100644 index 0000000..44d986c --- /dev/null +++ b/test/rails/app-2.2.2/public/404.html @@ -0,0 +1 @@ +404 Not Found diff --git a/test/rails/app-2.2.2/public/500.html b/test/rails/app-2.2.2/public/500.html new file mode 100644 index 0000000..e534a49 --- /dev/null +++ b/test/rails/app-2.2.2/public/500.html @@ -0,0 +1 @@ +500 Internal Server Error diff --git a/test/rails/app-2.3.2.1/.gitignore b/test/rails/app-2.3.2.1/.gitignore new file mode 100644 index 0000000..f451f91 --- /dev/null +++ b/test/rails/app-2.3.2.1/.gitignore @@ -0,0 +1,2 @@ +/tmp +/vendor diff --git a/test/rails/app-2.3.2.1/Rakefile b/test/rails/app-2.3.2.1/Rakefile new file mode 100644 index 0000000..fbebfca --- /dev/null +++ b/test/rails/app-2.3.2.1/Rakefile @@ -0,0 +1,7 @@ +require(File.join(File.dirname(__FILE__), 'config', 'boot')) + +require 'rake' +require 'rake/testtask' +require 'rake/rdoctask' + +require 'tasks/rails' diff --git a/test/rails/app-2.3.2.1/app/controllers/application_controller.rb b/test/rails/app-2.3.2.1/app/controllers/application_controller.rb new file mode 100644 index 0000000..6160f52 --- /dev/null +++ b/test/rails/app-2.3.2.1/app/controllers/application_controller.rb @@ -0,0 +1,3 @@ +class ApplicationController < ActionController::Base + helper :all +end diff --git a/test/rails/app-2.3.2.1/app/controllers/foo_controller.rb b/test/rails/app-2.3.2.1/app/controllers/foo_controller.rb new file mode 100644 index 0000000..261669c --- /dev/null +++ b/test/rails/app-2.3.2.1/app/controllers/foo_controller.rb @@ -0,0 +1,34 @@ +require 'digest/sha1' +class FooController < ApplicationController + def index + render :text => "FOO\n" + end + + def xcookie + cookies["foo"] = "cookie-#$$-#{session[:gotta_use_the_session_in_2_3]}" + render :text => "" + end + + def xnotice + flash[:notice] = "session #$$" + render :text => "" + end + + def xpost + if request.post? + digest = Digest::SHA1.new + out = "params: #{params.inspect}\n" + if file = params[:file] + loop do + buf = file.read(4096) or break + digest.update(buf) + end + out << "sha1: #{digest.to_s}\n" + end + headers['content-type'] = 'text/plain' + render :text => out + else + render :status => 403, :text => "need post\n" + end + end +end diff --git a/test/rails/app-2.3.2.1/app/helpers/application_helper.rb b/test/rails/app-2.3.2.1/app/helpers/application_helper.rb new file mode 100644 index 0000000..de6be79 --- /dev/null +++ b/test/rails/app-2.3.2.1/app/helpers/application_helper.rb @@ -0,0 +1,2 @@ +module ApplicationHelper +end diff --git a/test/rails/app-2.3.2.1/config/boot.rb b/test/rails/app-2.3.2.1/config/boot.rb new file mode 100644 index 0000000..d22e6b0 --- /dev/null +++ b/test/rails/app-2.3.2.1/config/boot.rb @@ -0,0 +1,107 @@ +RAILS_ROOT = "#{File.dirname(__FILE__)}/.." unless defined?(RAILS_ROOT) + +module Rails + class << self + def boot! + unless booted? + preinitialize + pick_boot.run + end + end + + def booted? + defined? Rails::Initializer + end + + def pick_boot + (vendor_rails? ? VendorBoot : GemBoot).new + end + + def vendor_rails? + File.exist?("#{RAILS_ROOT}/vendor/rails") + end + + def preinitialize + load(preinitializer_path) if File.exist?(preinitializer_path) + end + + def preinitializer_path + "#{RAILS_ROOT}/config/preinitializer.rb" + end + end + + class Boot + def run + load_initializer + Rails::Initializer.run(:set_load_path) + end + end + + class VendorBoot < Boot + def load_initializer + require "#{RAILS_ROOT}/vendor/rails/railties/lib/initializer" + Rails::Initializer.run(:install_gem_spec_stubs) + Rails::GemDependency.add_frozen_gem_path + end + end + + class GemBoot < Boot + def load_initializer + self.class.load_rubygems + load_rails_gem + require 'initializer' + end + + def load_rails_gem + if version = self.class.gem_version + gem 'rails', version + else + gem 'rails' + end + rescue Gem::LoadError => load_error + $stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v=#{version} rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.) + exit 1 + end + + class << self + def rubygems_version + Gem::RubyGemsVersion rescue nil + end + + def gem_version + if defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION + elsif ENV.include?('RAILS_GEM_VERSION') + ENV['RAILS_GEM_VERSION'] + else + parse_gem_version(read_environment_rb) + end + end + + def load_rubygems + require 'rubygems' + min_version = '1.3.1' + unless rubygems_version >= min_version + $stderr.puts %Q(Rails requires RubyGems >= #{min_version} (you have #{rubygems_version}). Please `gem update --system` and try again.) + exit 1 + end + + rescue LoadError + $stderr.puts %Q(Rails requires RubyGems >= #{min_version}. Please install RubyGems and try again: http://rubygems.rubyforge.org) + exit 1 + end + + def parse_gem_version(text) + $1 if text =~ /^[^#]*RAILS_GEM_VERSION\s*=\s*["']([!~<>=]*\s*[\d.]+)["']/ + end + + private + def read_environment_rb + File.read("#{RAILS_ROOT}/config/environment.rb") + end + end + end +end + +# All that for this: +Rails.boot! diff --git a/test/rails/app-2.3.2.1/config/database.yml b/test/rails/app-2.3.2.1/config/database.yml new file mode 100644 index 0000000..9f77843 --- /dev/null +++ b/test/rails/app-2.3.2.1/config/database.yml @@ -0,0 +1,12 @@ +development: + adapter: sqlite3 + database: db/development.sqlite3 + timeout: 5000 +test: + adapter: sqlite3 + database: db/test.sqlite3 + timeout: 5000 +production: + adapter: sqlite3 + database: db/production.sqlite3 + timeout: 5000 diff --git a/test/rails/app-2.3.2.1/config/environment.rb b/test/rails/app-2.3.2.1/config/environment.rb new file mode 100644 index 0000000..17abdb7 --- /dev/null +++ b/test/rails/app-2.3.2.1/config/environment.rb @@ -0,0 +1,15 @@ +unless defined? RAILS_GEM_VERSION + RAILS_GEM_VERSION = ENV['UNICORN_RAILS_VERSION'] +end + +# Bootstrap the Rails environment, frameworks, and default configuration +require File.join(File.dirname(__FILE__), 'boot') + +Rails::Initializer.run do |config| + config.frameworks -= [ :active_resource, :action_mailer ] + config.action_controller.session_store = :active_record_store + config.action_controller.session = { + :session_key => "_unicorn_rails_test.#{rand}", + :secret => "#{rand}#{rand}#{rand}#{rand}", + } +end diff --git a/test/rails/app-2.3.2.1/config/environments/development.rb b/test/rails/app-2.3.2.1/config/environments/development.rb new file mode 100644 index 0000000..55376c5 --- /dev/null +++ b/test/rails/app-2.3.2.1/config/environments/development.rb @@ -0,0 +1,5 @@ +config.cache_classes = false +config.whiny_nils = true +config.action_controller.consider_all_requests_local = true +config.action_view.debug_rjs = true +config.action_controller.perform_caching = false diff --git a/test/rails/app-2.3.2.1/config/environments/production.rb b/test/rails/app-2.3.2.1/config/environments/production.rb new file mode 100644 index 0000000..474257d --- /dev/null +++ b/test/rails/app-2.3.2.1/config/environments/production.rb @@ -0,0 +1,4 @@ +config.cache_classes = true +config.action_controller.consider_all_requests_local = false +config.action_controller.perform_caching = true +config.action_view.cache_template_loading = true diff --git a/test/rails/app-2.3.2.1/config/routes.rb b/test/rails/app-2.3.2.1/config/routes.rb new file mode 100644 index 0000000..4248853 --- /dev/null +++ b/test/rails/app-2.3.2.1/config/routes.rb @@ -0,0 +1,4 @@ +ActionController::Routing::Routes.draw do |map| + map.connect ':controller/:action/:id' + map.connect ':controller/:action/:id.:format' +end diff --git a/test/rails/app-2.3.2.1/db/.gitignore b/test/rails/app-2.3.2.1/db/.gitignore new file mode 100644 index 0000000..e69de29 --- /dev/null +++ b/test/rails/app-2.3.2.1/db/.gitignore diff --git a/test/rails/app-2.3.2.1/log/.gitignore b/test/rails/app-2.3.2.1/log/.gitignore new file mode 100644 index 0000000..397b4a7 --- /dev/null +++ b/test/rails/app-2.3.2.1/log/.gitignore @@ -0,0 +1 @@ +*.log diff --git a/test/rails/app-2.3.2.1/public/404.html b/test/rails/app-2.3.2.1/public/404.html new file mode 100644 index 0000000..44d986c --- /dev/null +++ b/test/rails/app-2.3.2.1/public/404.html @@ -0,0 +1 @@ +404 Not Found diff --git a/test/rails/app-2.3.2.1/public/500.html b/test/rails/app-2.3.2.1/public/500.html new file mode 100644 index 0000000..e534a49 --- /dev/null +++ b/test/rails/app-2.3.2.1/public/500.html @@ -0,0 +1 @@ +500 Internal Server Error diff --git a/test/rails/test_rails.rb b/test/rails/test_rails.rb new file mode 100644 index 0000000..c7add20 --- /dev/null +++ b/test/rails/test_rails.rb @@ -0,0 +1,247 @@ +# Copyright (c) 2009 Eric Wong +require 'test/test_helper' + +# don't call exit(0) since it may be run under rake (but gmake is recommended) +do_test = true + +$unicorn_rails_bin = ENV['UNICORN_RAILS_TEST_BIN'] || "unicorn_rails" +redirect_test_io { do_test = system($unicorn_rails_bin, '-v') } + +unless do_test + warn "#$unicorn_rails_bin not found in PATH=#{ENV['PATH']}, " \ + "skipping this test" +end + +unless which('git') + warn "git not found in PATH=#{ENV['PATH']}, skipping this test" + do_test = false +end + +if RAILS_GIT_REPO = ENV['RAILS_GIT_REPO'] + unless File.directory?(RAILS_GIT_REPO) + warn "#{RAILS_GIT_REPO} not found, create it with:\n" \ + "\tgit clone --mirror git://github.com/rails/rails #{RAILS_GIT_REPO}" \ + "skipping this test for now" + do_test = false + end +else + warn "RAILS_GIT_REPO not defined, don't know where to git clone from" + do_test = false +end + +unless UNICORN_RAILS_TEST_VERSION = ENV['UNICORN_RAILS_TEST_VERSION'] + warn 'UNICORN_RAILS_TEST_VERSION not defined in environment, ' \ + 'skipping this test' + do_test = false +end + +RAILS_ROOT = "#{File.dirname(__FILE__)}/app-#{UNICORN_RAILS_TEST_VERSION}" +unless File.directory?(RAILS_ROOT) + warn "unsupported UNICORN_RAILS_TEST_VERSION=#{UNICORN_RAILS_TEST_VERSION}" + do_test = false +end + +ROR_V = UNICORN_RAILS_TEST_VERSION.split(/\./).map { |x| x.to_i } +RB_V = RUBY_VERSION.split(/\./).map { |x| x.to_i } +if RB_V[0] >= 1 && RB_V[1] >= 9 + unless ROR_V[0] >= 2 && ROR_V[1] >= 3 + warn "skipping Ruby >=1.9 test with Rails <2.3" + do_test = false + end +end + +class RailsTest < Test::Unit::TestCase + trap(:QUIT, 'IGNORE') + + COMMON_TMP = Tempfile.new('unicorn_tmp') unless defined?(COMMON_TMP) + + HEAVY_CFG = <<-EOS +worker_processes 2 +timeout 30 +logger Logger.new('#{COMMON_TMP.path}') + EOS + + def setup + @pwd = Dir.pwd + @tmpfile = Tempfile.new('unicorn_rails_test') + @tmpdir = @tmpfile.path + @tmpfile.close! + assert_nothing_raised do + FileUtils.cp_r(RAILS_ROOT, @tmpdir, :preserve => true) + end + Dir.chdir(@tmpdir) + system('git', 'clone', '-nsq', RAILS_GIT_REPO, 'vendor/rails') + Dir.chdir("#@tmpdir/vendor/rails") do + system('git', 'reset', '-q', '--hard', "v#{UNICORN_RAILS_TEST_VERSION}") + end + + assert(system('rake', 'db:sessions:create')) + assert(system('rake', 'db:migrate')) + + @addr = ENV['UNICORN_TEST_ADDR'] || '127.0.0.1' + @port = unused_port(@addr) + @start_pid = $$ + @pid = nil + end + + def test_launcher + tmp_dirs = %w(cache pids sessions sockets) + tmp_dirs.each { |dir| assert(! File.exist?("tmp/#{dir}")) } + redirect_test_io { @pid = fork { exec 'unicorn_rails', "-l#@addr:#@port" } } + wait_master_ready("test_stderr.#$$.log") + + # temp dirs exist + tmp_dirs.each { |dir| assert(File.directory?("tmp/#{dir}")) } + + # basic GET + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/foo")) + assert_equal "FOO\n", res.body + assert_match %r{^text/html\b}, res['Content-Type'] + assert_equal "4", res['Content-Length'] + assert_equal "200 OK", res['Status'] + + # can we set cookies? + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/foo/xcookie")) + assert_equal "200", res.code + assert_equal "200 OK", res['Status'] + cookies = res.get_fields('Set-Cookie') + assert_equal 2, cookies.size + assert_equal 1, cookies.grep(/\A_unicorn_rails_test\./).size + assert_equal 1, cookies.grep(/\Afoo=cookie/).size + + # how about just a session? + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/foo/xnotice")) + assert_equal "200", res.code + assert_equal "200 OK", res['Status'] + cookies = res.get_fields('Set-Cookie') + assert_equal 1, cookies.size + assert_equal 1, cookies.grep(/\A_unicorn_rails_test\./).size + + # posting forms? + uri = URI.parse("http://#@addr:#@port/foo/xpost") + wait_master_ready("test_stderr.#$$.log") + res = Net::HTTP.post_form(uri, {"a" => "b", "c"=>"d"}) + assert_equal "200", res.code + params = res.body.split(/\n/).grep(/^params:/) + assert_equal 1, params.size + params = eval(params[0].gsub!(/\Aparams:/, '')) + assert_equal Hash, params.class + assert_equal 'b', params['a'] + assert_equal 'd', params['c'] + assert_equal "200 OK", res['Status'] + + # try uploading a big file + tmp = Tempfile.new('random') + sha1 = Digest::SHA1.new + assert_nothing_raised do + File.open("/dev/urandom", "rb") do |fp| + 256.times do + buf = fp.sysread(4096) + sha1.update(buf) + tmp.syswrite(buf) + end + end + end + resp = `curl -isSfN -Ffile=@#{tmp.path} http://#@addr:#@port/foo/xpost` + assert $?.success? + resp = resp.split(/\r?\n/) + grepped = resp.grep(/^sha1: (.{40})/) + assert_equal 1, grepped.size + assert_equal(sha1.hexdigest, /^sha1: (.{40})/.match(grepped.first)[1]) + + grepped = resp.grep(/^Content-Type:\s+(.+)/i) + assert_equal 1, grepped.size + assert_match %r{^text/plain}, grepped.first.split(/\s*:\s*/)[1] + + assert_equal 1, resp.grep(/^Status:/i).size + + # make sure we can get 403 responses, too + uri = URI.parse("http://#@addr:#@port/foo/xpost") + wait_master_ready("test_stderr.#$$.log") + res = Net::HTTP.get_response(uri) + assert_equal "403", res.code + assert_equal "403 Forbidden", res['Status'] + + # non existent controller + uri = URI.parse("http://#@addr:#@port/asdf") + res = Net::HTTP.get_response(uri) + assert_equal "404", res.code + assert_equal "404 Not Found", res['Status'] + + # static files + + # ensure file we're about to serve is not there yet + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/pid.txt")) + assert_equal "404 Not Found", res['Status'] + assert_equal '404', res.code + + # can we serve text files based on suffix? + File.open("public/pid.txt", "wb") { |fp| fp.syswrite("#$$\n") } + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/pid.txt")) + assert_equal '200', res.code + assert_equal "200 OK", res['Status'] + assert_match %r{^text/plain}, res['Content-Type'] + assert_equal "#$$\n", res.body + + # can we serve HTML files based on suffix? + assert File.exist?("public/500.html") + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/500.html")) + assert_equal '200', res.code + assert_equal '200 OK', res['Status'] + assert_match %r{^text/html}, res['Content-Type'] + five_hundred_body = res.body + + # lets try pretending 500 is a controller that got cached + assert ! File.exist?("public/500") + assert_equal five_hundred_body, File.read("public/500.html") + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/500")) + assert_equal '200', res.code + assert_equal '200 OK', res['Status'] + assert_match %r{^text/html}, res['Content-Type'] + assert_equal five_hundred_body, res.body + end + + def test_alt_url_root + # cbf to actually work on this since I never use this feature (ewong) + return unless ROR_V[0] >= 2 && ROR_V[1] >= 3 + redirect_test_io do + @pid = fork { exec 'unicorn_rails', "-l#@addr:#@port", '-P/poo' } + end + wait_master_ready("test_stderr.#$$.log") + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/poo/foo")) + # p res + # p res.body + # system 'cat', 'log/development.log' + assert_equal "200", res.code + assert_equal '200 OK', res['Status'] + assert_equal "FOO\n", res.body + assert_match %r{^text/html\b}, res['Content-Type'] + assert_equal "4", res['Content-Length'] + + res = Net::HTTP.get_response(URI.parse("http://#@addr:#@port/foo")) + assert_equal "404", res.code + assert_equal '404 Not Found', res['Status'] + end + + def teardown + return if @start_pid != $$ + + if @pid + Process.kill(:QUIT, @pid) + pid2, status = Process.waitpid2(@pid) + assert status.success? + end + + Dir.chdir(@pwd) + FileUtils.rmtree(@tmpdir) + loop do + Process.kill('-QUIT', 0) + begin + Process.waitpid(-1, Process::WNOHANG) or break + rescue Errno::ECHILD + break + end + end + end + +end if do_test diff --git a/test/test_helper.rb b/test/test_helper.rb index f809af3..55aa70c 100644 --- a/test/test_helper.rb +++ b/test/test_helper.rb @@ -4,9 +4,16 @@ # Additional work donated by contributors. See http://mongrel.rubyforge.org/attributions.html # for more information. +STDIN.sync = STDOUT.sync = STDERR.sync = true # buffering makes debugging hard + +# Some tests watch a log file or a pid file to spring up to check state +# Can't rely on inotify on non-Linux and logging to a pipe makes things +# more complicated +DEFAULT_TRIES = 1000 +DEFAULT_RES = 0.2 HERE = File.dirname(__FILE__) unless defined?(HERE) -%w(lib ext bin test).each do |dir| +%w(lib ext).each do |dir| $LOAD_PATH.unshift "#{HERE}/../#{dir}" end @@ -15,8 +22,10 @@ require 'net/http' require 'digest/sha1' require 'uri' require 'stringio' +require 'pathname' +require 'tempfile' +require 'fileutils' require 'unicorn' -require 'tmpdir' if ENV['DEBUG'] require 'ruby-debug' @@ -26,8 +35,9 @@ end def redirect_test_io orig_err = STDERR.dup orig_out = STDOUT.dup - STDERR.reopen("test_stderr.#{$$}.log") - STDOUT.reopen("test_stdout.#{$$}.log") + STDERR.reopen("test_stderr.#{$$}.log", "a") + STDOUT.reopen("test_stdout.#{$$}.log", "a") + STDERR.sync = STDOUT.sync = true at_exit do File.unlink("test_stderr.#{$$}.log") rescue nil @@ -41,7 +51,18 @@ def redirect_test_io STDOUT.reopen(orig_out) end end - + +# which(1) exit codes cannot be trusted on some systems +# We use UNIX shell utilities in some tests because we don't trust +# ourselves to write Ruby 100% correctly :) +def which(bin) + ex = ENV['PATH'].split(/:/).detect do |x| + x << "/#{bin}" + File.executable?(x) + end or warn "`#{bin}' not found in PATH=#{ENV['PATH']}" + ex +end + # Either takes a string to do a get request against, or a tuple of [URI, HTTP] where # HTTP is some kind of Net::HTTP request object (POST, HEAD, etc.) def hit(uris) @@ -101,3 +122,141 @@ def unused_port(addr = '127.0.0.1') sock.close rescue nil port end + +def try_require(lib) + begin + require lib + true + rescue LoadError + false + end +end + +# sometimes the server may not come up right away +def retry_hit(uris = []) + tries = DEFAULT_TRIES + begin + hit(uris) + rescue Errno::ECONNREFUSED => err + if (tries -= 1) > 0 + sleep DEFAULT_RES + retry + end + raise err + end +end + +def assert_shutdown(pid) + wait_master_ready("test_stderr.#{pid}.log") + assert_nothing_raised { Process.kill(:QUIT, pid) } + status = nil + assert_nothing_raised { pid, status = Process.waitpid2(pid) } + assert status.success?, "exited successfully" +end + +def wait_workers_ready(path, nr_workers) + tries = DEFAULT_TRIES + lines = [] + while (tries -= 1) > 0 + begin + lines = File.readlines(path).grep(/worker=\d+ ready/) + lines.size == nr_workers and return + rescue Errno::ENOENT + end + sleep DEFAULT_RES + end + raise "#{nr_workers} workers never became ready:" \ + "\n\t#{lines.join("\n\t")}\n" +end + +def wait_master_ready(master_log) + tries = DEFAULT_TRIES + while (tries -= 1) > 0 + begin + File.readlines(master_log).grep(/master process ready/)[0] and return + rescue Errno::ENOENT + end + sleep DEFAULT_RES + end + raise "master process never became ready" +end + +def reexec_usr2_quit_test(pid, pid_file) + assert File.exist?(pid_file), "pid file OK" + assert ! File.exist?("#{pid_file}.oldbin"), "oldbin pid file" + assert_nothing_raised { Process.kill(:USR2, pid) } + assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } + wait_for_file("#{pid_file}.oldbin") + wait_for_file(pid_file) + + old_pid = File.read("#{pid_file}.oldbin").to_i + new_pid = File.read(pid_file).to_i + + # kill old master process + assert_not_equal pid, new_pid + assert_equal pid, old_pid + assert_nothing_raised { Process.kill(:QUIT, old_pid) } + assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } + wait_for_death(old_pid) + assert_equal new_pid, File.read(pid_file).to_i + assert_nothing_raised { retry_hit(["http://#{@addr}:#{@port}/"]) } + assert_nothing_raised { Process.kill(:QUIT, new_pid) } +end + +def reexec_basic_test(pid, pid_file) + results = retry_hit(["http://#{@addr}:#{@port}/"]) + assert_equal String, results[0].class + assert_nothing_raised { Process.kill(0, pid) } + master_log = "#{@tmpdir}/test_stderr.#{pid}.log" + wait_master_ready(master_log) + File.truncate(master_log, 0) + nr = 50 + kill_point = 2 + assert_nothing_raised do + nr.times do |i| + hit(["http://#{@addr}:#{@port}/#{i}"]) + i == kill_point and Process.kill(:HUP, pid) + end + end + wait_master_ready(master_log) + assert File.exist?(pid_file), "pid=#{pid_file} exists" + new_pid = File.read(pid_file).to_i + assert_not_equal pid, new_pid + assert_nothing_raised { Process.kill(0, new_pid) } + assert_nothing_raised { Process.kill(:QUIT, new_pid) } +end + +def wait_for_file(path) + tries = DEFAULT_TRIES + while (tries -= 1) > 0 && ! File.exist?(path) + sleep DEFAULT_RES + end + assert File.exist?(path), "path=#{path} exists #{caller.inspect}" +end + +def xfork(&block) + fork do + ObjectSpace.each_object(Tempfile) do |tmp| + ObjectSpace.undefine_finalizer(tmp) + end + yield + end +end + +# can't waitpid on detached processes +def wait_for_death(pid) + tries = DEFAULT_TRIES + while (tries -= 1) > 0 + begin + Process.kill(0, pid) + begin + Process.waitpid(pid, Process::WNOHANG) + rescue Errno::ECHILD + end + sleep(DEFAULT_RES) + rescue Errno::ESRCH + return + end + end + raise "PID:#{pid} never died!" +end diff --git a/test/tools/trickletest.rb b/test/tools/trickletest.rb deleted file mode 100644 index e19ed71..0000000 --- a/test/tools/trickletest.rb +++ /dev/null @@ -1,45 +0,0 @@ -require 'socket' -require 'stringio' - -def do_test(st, chunk) - s = TCPSocket.new('127.0.0.1',ARGV[0].to_i); - req = StringIO.new(st) - nout = 0 - randstop = rand(st.length / 10) - STDERR.puts "stopping after: #{randstop}" - - begin - while data = req.read(chunk) - nout += s.write(data) - s.flush - sleep 0.1 - if nout > randstop - STDERR.puts "BANG! after #{nout} bytes." - break - end - end - rescue Object => e - STDERR.puts "ERROR: #{e}" - ensure - s.close - end -end - -content = "-" * (1024 * 240) -st = "GET / HTTP/1.1\r\nHost: www.zedshaw.com\r\nContent-Type: text/plain\r\nContent-Length: #{content.length}\r\n\r\n#{content}" - -puts "length: #{content.length}" - -threads = [] -ARGV[1].to_i.times do - t = Thread.new do - size = 100 - puts ">>>> #{size} sized chunks" - do_test(st, size) - end - - t.abort_on_exception = true - threads << t -end - -threads.each {|t| t.join} diff --git a/test/unit/test_configurator.rb b/test/unit/test_configurator.rb index 8de0b13..98f2db6 100644 --- a/test/unit/test_configurator.rb +++ b/test/unit/test_configurator.rb @@ -4,10 +4,34 @@ require 'unicorn/configurator' class TestConfigurator < Test::Unit::TestCase - def test_config_defaults + def test_config_init assert_nothing_raised { Unicorn::Configurator.new {} } end + def test_expand_addr + meth = Unicorn::Configurator.new.method(:expand_addr) + + assert_equal "/var/run/unicorn.sock", meth.call("/var/run/unicorn.sock") + assert_equal "#{Dir.pwd}/foo/bar.sock", meth.call("unix:foo/bar.sock") + + path = meth.call("~/foo/bar.sock") + assert_equal "/", path[0..0] + assert_match %r{/foo/bar\.sock\z}, path + + path = meth.call("~root/foo/bar.sock") + assert_equal "/", path[0..0] + assert_match %r{/foo/bar\.sock\z}, path + + assert_equal "1.2.3.4:2007", meth.call('1.2.3.4:2007') + assert_equal "0.0.0.0:2007", meth.call('0.0.0.0:2007') + assert_equal "0.0.0.0:2007", meth.call(':2007') + assert_equal "0.0.0.0:2007", meth.call('*:2007') + assert_equal "0.0.0.0:2007", meth.call('2007') + assert_equal "0.0.0.0:2007", meth.call(2007) + assert_match %r{\A\d+\.\d+\.\d+\.\d+:2007\z}, meth.call('1:2007') + assert_match %r{\A\d+\.\d+\.\d+\.\d+:2007\z}, meth.call('2:2007') + end + def test_config_invalid tmp = Tempfile.new('unicorn_config') tmp.syswrite(%q(asdfasdf "hello-world")) @@ -45,4 +69,43 @@ class TestConfigurator < Test::Unit::TestCase assert_nil @logger end + def test_listen_options + tmp = Tempfile.new('unicorn_config') + expect = { :sndbuf => 1, :rcvbuf => 2, :backlog => 10 }.freeze + listener = "127.0.0.1:12345" + tmp.syswrite("listen '#{listener}', #{expect.inspect}\n") + cfg = nil + assert_nothing_raised do + cfg = Unicorn::Configurator.new(:config_file => tmp.path) + end + assert_nothing_raised { cfg.commit!(self) } + assert(listener_opts = instance_variable_get("@listener_opts")) + assert_equal expect, listener_opts[listener] + end + + def test_listen_option_bad + tmp = Tempfile.new('unicorn_config') + expect = { :sndbuf => "five" } + listener = "127.0.0.1:12345" + tmp.syswrite("listen '#{listener}', #{expect.inspect}\n") + assert_raises(ArgumentError) do + Unicorn::Configurator.new(:config_file => tmp.path) + end + end + + def test_after_fork_proc + [ proc { |a,b| }, Proc.new { |a,b| }, lambda { |a,b| } ].each do |my_proc| + Unicorn::Configurator.new(:after_fork => my_proc).commit!(self) + assert_equal my_proc, @after_fork + end + end + + def test_after_fork_wrong_arity + [ proc { |a| }, Proc.new { }, lambda { |a,b,c| } ].each do |my_proc| + assert_raises(ArgumentError) do + Unicorn::Configurator.new(:after_fork => my_proc) + end + end + end + end diff --git a/test/unit/test_http_parser.rb b/test/unit/test_http_parser.rb index ca1cd01..a158ebb 100644 --- a/test/unit/test_http_parser.rb +++ b/test/unit/test_http_parser.rb @@ -14,46 +14,82 @@ class HttpParserTest < Test::Unit::TestCase parser = HttpParser.new req = {} http = "GET / HTTP/1.1\r\n\r\n" - nread = parser.execute(req, http, 0) - - assert nread == http.length, "Failed to parse the full HTTP request" - assert parser.finished?, "Parser didn't finish" - assert !parser.error?, "Parser had error" - assert nread == parser.nread, "Number read returned from execute does not match" + assert parser.execute(req, http) assert_equal 'HTTP/1.1', req['SERVER_PROTOCOL'] assert_equal '/', req['REQUEST_PATH'] assert_equal 'HTTP/1.1', req['HTTP_VERSION'] assert_equal '/', req['REQUEST_URI'] - assert_equal 'CGI/1.2', req['GATEWAY_INTERFACE'] - assert_equal 'GET', req['REQUEST_METHOD'] + assert_equal 'GET', req['REQUEST_METHOD'] assert_nil req['FRAGMENT'] - assert_nil req['QUERY_STRING'] - + assert_equal '', req['QUERY_STRING'] + parser.reset - assert parser.nread == 0, "Number read after reset should be 0" + req.clear + + assert ! parser.execute(req, "G") + assert req.empty? + + # try parsing again to ensure we were reset correctly + http = "GET /hello-world HTTP/1.1\r\n\r\n" + assert parser.execute(req, http) + + assert_equal 'HTTP/1.1', req['SERVER_PROTOCOL'] + assert_equal '/hello-world', req['REQUEST_PATH'] + assert_equal 'HTTP/1.1', req['HTTP_VERSION'] + assert_equal '/hello-world', req['REQUEST_URI'] + assert_equal 'GET', req['REQUEST_METHOD'] + assert_nil req['FRAGMENT'] + assert_equal '', req['QUERY_STRING'] + end + + def test_parse_server_host_default_port + parser = HttpParser.new + req = {} + assert parser.execute(req, "GET / HTTP/1.1\r\nHost: foo\r\n\r\n") + assert_equal 'foo', req['SERVER_NAME'] + assert_equal '80', req['SERVER_PORT'] + end + + def test_parse_server_host_alt_port + parser = HttpParser.new + req = {} + assert parser.execute(req, "GET / HTTP/1.1\r\nHost: foo:999\r\n\r\n") + assert_equal 'foo', req['SERVER_NAME'] + assert_equal '999', req['SERVER_PORT'] + end + + def test_parse_server_host_empty_port + parser = HttpParser.new + req = {} + assert parser.execute(req, "GET / HTTP/1.1\r\nHost: foo:\r\n\r\n") + assert_equal 'foo', req['SERVER_NAME'] + assert_equal '80', req['SERVER_PORT'] end - + + def test_parse_server_host_xfp_https + parser = HttpParser.new + req = {} + assert parser.execute(req, "GET / HTTP/1.1\r\nHost: foo:\r\n" \ + "X-Forwarded-Proto: https\r\n\r\n") + assert_equal 'foo', req['SERVER_NAME'] + assert_equal '443', req['SERVER_PORT'] + end + def test_parse_strange_headers parser = HttpParser.new req = {} should_be_good = "GET / HTTP/1.1\r\naaaaaaaaaaaaa:++++++++++\r\n\r\n" - nread = parser.execute(req, should_be_good, 0) - assert_equal should_be_good.length, nread - assert parser.finished? - assert !parser.error? + assert parser.execute(req, should_be_good) - # ref: http://thread.gmane.org/gmane.comp.lang.ruby.Unicorn.devel/37/focus=45 + # ref: http://thread.gmane.org/gmane.comp.lang.ruby.mongrel.devel/37/focus=45 # (note we got 'pen' mixed up with 'pound' in that thread, # but the gist of it is still relevant: these nasty headers are irrelevant # # nasty_pound_header = "GET / HTTP/1.1\r\nX-SSL-Bullshit: -----BEGIN CERTIFICATE-----\r\n\tMIIFbTCCBFWgAwIBAgICH4cwDQYJKoZIhvcNAQEFBQAwcDELMAkGA1UEBhMCVUsx\r\n\tETAPBgNVBAoTCGVTY2llbmNlMRIwEAYDVQQLEwlBdXRob3JpdHkxCzAJBgNVBAMT\r\n\tAkNBMS0wKwYJKoZIhvcNAQkBFh5jYS1vcGVyYXRvckBncmlkLXN1cHBvcnQuYWMu\r\n\tdWswHhcNMDYwNzI3MTQxMzI4WhcNMDcwNzI3MTQxMzI4WjBbMQswCQYDVQQGEwJV\r\n\tSzERMA8GA1UEChMIZVNjaWVuY2UxEzARBgNVBAsTCk1hbmNoZXN0ZXIxCzAJBgNV\r\n\tBAcTmrsogriqMWLAk1DMRcwFQYDVQQDEw5taWNoYWVsIHBhcmQYJKoZIhvcNAQEB\r\n\tBQADggEPADCCAQoCggEBANPEQBgl1IaKdSS1TbhF3hEXSl72G9J+WC/1R64fAcEF\r\n\tW51rEyFYiIeZGx/BVzwXbeBoNUK41OK65sxGuflMo5gLflbwJtHBRIEKAfVVp3YR\r\n\tgW7cMA/s/XKgL1GEC7rQw8lIZT8RApukCGqOVHSi/F1SiFlPDxuDfmdiNzL31+sL\r\n\t0iwHDdNkGjy5pyBSB8Y79dsSJtCW/iaLB0/n8Sj7HgvvZJ7x0fr+RQjYOUUfrePP\r\n\tu2MSpFyf+9BbC/aXgaZuiCvSR+8Snv3xApQY+fULK/xY8h8Ua51iXoQ5jrgu2SqR\r\n\twgA7BUi3G8LFzMBl8FRCDYGUDy7M6QaHXx1ZWIPWNKsCAwEAAaOCAiQwggIgMAwG\r\n\tA1UdEwEB/wQCMAAwEQYJYIZIAYb4QgEBBAQDAgWgMA4GA1UdDwEB/wQEAwID6DAs\r\n\tBglghkgBhvhCAQ0EHxYdVUsgZS1TY2llbmNlIFVzZXIgQ2VydGlmaWNhdGUwHQYD\r\n\tVR0OBBYEFDTt/sf9PeMaZDHkUIldrDYMNTBZMIGaBgNVHSMEgZIwgY+AFAI4qxGj\r\n\tloCLDdMVKwiljjDastqooXSkcjBwMQswCQYDVQQGEwJVSzERMA8GA1UEChMIZVNj\r\n\taWVuY2UxEjAQBgNVBAsTCUF1dGhvcml0eTELMAkGA1UEAxMCQ0ExLTArBgkqhkiG\r\n\t9w0BCQEWHmNhLW9wZXJhdG9yQGdyaWQtc3VwcG9ydC5hYy51a4IBADApBgNVHRIE\r\n\tIjAggR5jYS1vcGVyYXRvckBncmlkLXN1cHBvcnQuYWMudWswGQYDVR0gBBIwEDAO\r\n\tBgwrBgEEAdkvAQEBAQYwPQYJYIZIAYb4QgEEBDAWLmh0dHA6Ly9jYS5ncmlkLXN1\r\n\tcHBvcnQuYWMudmT4sopwqlBWsvcHViL2NybC9jYWNybC5jcmwwPQYJYIZIAYb4QgEDBDAWLmh0\r\n\tdHA6Ly9jYS5ncmlkLXN1cHBvcnQuYWMudWsvcHViL2NybC9jYWNybC5jcmwwPwYD\r\n\tVR0fBDgwNjA0oDKgMIYuaHR0cDovL2NhLmdyaWQt5hYy51ay9wdWIv\r\n\tY3JsL2NhY3JsLmNybDANBgkqhkiG9w0BAQUFAAOCAQEAS/U4iiooBENGW/Hwmmd3\r\n\tXCy6Zrt08YjKCzGNjorT98g8uGsqYjSxv/hmi0qlnlHs+k/3Iobc3LjS5AMYr5L8\r\n\tUO7OSkgFFlLHQyC9JzPfmLCAugvzEbyv4Olnsr8hbxF1MbKZoQxUZtMVu29wjfXk\r\n\thTeApBv7eaKCWpSp7MCbvgzm74izKhu3vlDk9w6qVrxePfGgpKPqfHiOoGhFnbTK\r\n\twTC6o2xq5y0qZ03JonF7OJspEd3I5zKY3E+ov7/ZhW6DqT8UFvsAdjvQbXyhV8Eu\r\n\tYhixw1aKEPzNjNowuIseVogKOLXxWI5vAi5HgXdS0/ES5gDGsABo4fqovUKlgop3\r\n\tRA==\r\n\t-----END CERTIFICATE-----\r\n\r\n" # parser = HttpParser.new # req = {} - # nread = parser.execute(req, nasty_pound_header, 0) - # assert_equal nasty_pound_header.length, nread - # assert parser.finished? - # assert !parser.error? + # assert parser.execute(req, nasty_pound_header, 0) end def test_parse_ie6_urls @@ -67,10 +103,7 @@ class HttpParserTest < Test::Unit::TestCase parser = HttpParser.new req = {} sorta_safe = %(GET #{path} HTTP/1.1\r\n\r\n) - nread = parser.execute(req, sorta_safe, 0) - assert_equal sorta_safe.length, nread - assert parser.finished? - assert !parser.error? + assert parser.execute(req, sorta_safe) end end @@ -79,28 +112,149 @@ class HttpParserTest < Test::Unit::TestCase req = {} bad_http = "GET / SsUTF/1.1" - error = false - begin - nread = parser.execute(req, bad_http, 0) - rescue => details - error = true - end + assert_raises(HttpParserError) { parser.execute(req, bad_http) } + parser.reset + assert(parser.execute({}, "GET / HTTP/1.0\r\n\r\n")) + end + + def test_piecemeal + parser = HttpParser.new + req = {} + http = "GET" + assert ! parser.execute(req, http) + assert_raises(HttpParserError) { parser.execute(req, http) } + assert ! parser.execute(req, http << " / HTTP/1.0") + assert_equal '/', req['REQUEST_PATH'] + assert_equal '/', req['REQUEST_URI'] + assert_equal 'GET', req['REQUEST_METHOD'] + assert ! parser.execute(req, http << "\r\n") + assert_equal 'HTTP/1.0', req['HTTP_VERSION'] + assert ! parser.execute(req, http << "\r") + assert parser.execute(req, http << "\n") + assert_equal 'HTTP/1.1', req['SERVER_PROTOCOL'] + assert_nil req['FRAGMENT'] + assert_equal '', req['QUERY_STRING'] + end + + # not common, but underscores do appear in practice + def test_absolute_uri_underscores + parser = HttpParser.new + req = {} + http = "GET http://under_score.example.com/foo?q=bar HTTP/1.0\r\n\r\n" + assert parser.execute(req, http) + assert_equal 'http', req['rack.url_scheme'] + assert_equal '/foo?q=bar', req['REQUEST_URI'] + assert_equal '/foo', req['REQUEST_PATH'] + assert_equal 'q=bar', req['QUERY_STRING'] + + assert_equal 'under_score.example.com', req['HTTP_HOST'] + assert_equal 'under_score.example.com', req['SERVER_NAME'] + assert_equal '80', req['SERVER_PORT'] + end + + def test_absolute_uri + parser = HttpParser.new + req = {} + http = "GET http://example.com/foo?q=bar HTTP/1.0\r\n\r\n" + assert parser.execute(req, http) + assert_equal 'http', req['rack.url_scheme'] + assert_equal '/foo?q=bar', req['REQUEST_URI'] + assert_equal '/foo', req['REQUEST_PATH'] + assert_equal 'q=bar', req['QUERY_STRING'] + + assert_equal 'example.com', req['HTTP_HOST'] + assert_equal 'example.com', req['SERVER_NAME'] + assert_equal '80', req['SERVER_PORT'] + end + + # X-Forwarded-Proto is not in rfc2616, absolute URIs are, however... + def test_absolute_uri_https + parser = HttpParser.new + req = {} + http = "GET https://example.com/foo?q=bar HTTP/1.1\r\n" \ + "X-Forwarded-Proto: http\r\n\r\n" + assert parser.execute(req, http) + assert_equal 'https', req['rack.url_scheme'] + assert_equal '/foo?q=bar', req['REQUEST_URI'] + assert_equal '/foo', req['REQUEST_PATH'] + assert_equal 'q=bar', req['QUERY_STRING'] + + assert_equal 'example.com', req['HTTP_HOST'] + assert_equal 'example.com', req['SERVER_NAME'] + assert_equal '443', req['SERVER_PORT'] + end + + # Host: header should be ignored for absolute URIs + def test_absolute_uri_with_port + parser = HttpParser.new + req = {} + http = "GET http://example.com:8080/foo?q=bar HTTP/1.2\r\n" \ + "Host: bad.example.com\r\n\r\n" + assert parser.execute(req, http) + assert_equal 'http', req['rack.url_scheme'] + assert_equal '/foo?q=bar', req['REQUEST_URI'] + assert_equal '/foo', req['REQUEST_PATH'] + assert_equal 'q=bar', req['QUERY_STRING'] + + assert_equal 'example.com:8080', req['HTTP_HOST'] + assert_equal 'example.com', req['SERVER_NAME'] + assert_equal '8080', req['SERVER_PORT'] + end + + def test_absolute_uri_with_empty_port + parser = HttpParser.new + req = {} + http = "GET https://example.com:/foo?q=bar HTTP/1.1\r\n" \ + "Host: bad.example.com\r\n\r\n" + assert parser.execute(req, http) + assert_equal 'https', req['rack.url_scheme'] + assert_equal '/foo?q=bar', req['REQUEST_URI'] + assert_equal '/foo', req['REQUEST_PATH'] + assert_equal 'q=bar', req['QUERY_STRING'] + + assert_equal 'example.com:', req['HTTP_HOST'] + assert_equal 'example.com', req['SERVER_NAME'] + assert_equal '443', req['SERVER_PORT'] + end + + def test_put_body_oneshot + parser = HttpParser.new + req = {} + http = "PUT / HTTP/1.0\r\nContent-Length: 5\r\n\r\nabcde" + assert parser.execute(req, http) + assert_equal '/', req['REQUEST_PATH'] + assert_equal '/', req['REQUEST_URI'] + assert_equal 'PUT', req['REQUEST_METHOD'] + assert_equal 'HTTP/1.0', req['HTTP_VERSION'] + assert_equal 'HTTP/1.1', req['SERVER_PROTOCOL'] + assert_equal "abcde", req[:http_body] + end - assert error, "failed to throw exception" - assert !parser.finished?, "Parser shouldn't be finished" - assert parser.error?, "Parser SHOULD have error" + def test_put_body_later + parser = HttpParser.new + req = {} + http = "PUT /l HTTP/1.0\r\nContent-Length: 5\r\n\r\n" + assert parser.execute(req, http) + assert_equal '/l', req['REQUEST_PATH'] + assert_equal '/l', req['REQUEST_URI'] + assert_equal 'PUT', req['REQUEST_METHOD'] + assert_equal 'HTTP/1.0', req['HTTP_VERSION'] + assert_equal 'HTTP/1.1', req['SERVER_PROTOCOL'] + assert_equal "", req[:http_body] end def test_fragment_in_uri parser = HttpParser.new req = {} get = "GET /forums/1/topics/2375?page=1#posts-17408 HTTP/1.1\r\n\r\n" + ok = false assert_nothing_raised do - parser.execute(req, get, 0) + ok = parser.execute(req, get) end - assert parser.finished? + assert ok assert_equal '/forums/1/topics/2375?page=1', req['REQUEST_URI'] assert_equal 'posts-17408', req['FRAGMENT'] + assert_equal 'page=1', req['QUERY_STRING'] end # lame random garbage maker @@ -125,7 +279,7 @@ class HttpParserTest < Test::Unit::TestCase 10.times do |c| get = "GET /#{rand_data(10,120)} HTTP/1.1\r\nX-#{rand_data(1024, 1024+(c*1024))}: Test\r\n\r\n" assert_raises Unicorn::HttpParserError do - parser.execute({}, get, 0) + parser.execute({}, get) parser.reset end end @@ -134,7 +288,7 @@ class HttpParserTest < Test::Unit::TestCase 10.times do |c| get = "GET /#{rand_data(10,120)} HTTP/1.1\r\nX-Test: #{rand_data(1024, 1024+(c*1024), false)}\r\n\r\n" assert_raises Unicorn::HttpParserError do - parser.execute({}, get, 0) + parser.execute({}, get) parser.reset end end @@ -143,7 +297,7 @@ class HttpParserTest < Test::Unit::TestCase get = "GET /#{rand_data(10,120)} HTTP/1.1\r\n" get << "X-Test: test\r\n" * (80 * 1024) assert_raises Unicorn::HttpParserError do - parser.execute({}, get, 0) + parser.execute({}, get) parser.reset end @@ -151,7 +305,7 @@ class HttpParserTest < Test::Unit::TestCase 10.times do |c| get = "GET #{rand_data(1024, 1024+(c*1024), false)} #{rand_data(1024, 1024+(c*1024), false)}\r\n\r\n" assert_raises Unicorn::HttpParserError do - parser.execute({}, get, 0) + parser.execute({}, get) parser.reset end end diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb new file mode 100644 index 0000000..0bfff7d --- /dev/null +++ b/test/unit/test_request.rb @@ -0,0 +1,159 @@ +# Copyright (c) 2009 Eric Wong +# You can redistribute it and/or modify it under the same terms as Ruby. + +require 'test/test_helper' +begin + require 'rack' + require 'rack/lint' +rescue LoadError + warn "Unable to load rack, skipping test" + exit 0 +end + +include Unicorn + +class RequestTest < Test::Unit::TestCase + + class MockRequest < StringIO + alias_method :readpartial, :sysread + end + + def setup + @request = HttpRequest.new(Logger.new($stderr)) + @app = lambda do |env| + [ 200, { 'Content-Length' => '0', 'Content-Type' => 'text/plain' }, [] ] + end + @lint = Rack::Lint.new(@app) + end + + def test_options + client = MockRequest.new("OPTIONS * HTTP/1.1\r\n" \ + "Host: foo\r\n\r\n") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert_equal '', env['REQUEST_PATH'] + assert_equal '', env['PATH_INFO'] + assert_equal '*', env['REQUEST_URI'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_absolute_uri_with_query + client = MockRequest.new("GET http://e:3/x?y=z HTTP/1.1\r\n" \ + "Host: foo\r\n\r\n") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert_equal '/x', env['REQUEST_PATH'] + assert_equal '/x', env['PATH_INFO'] + assert_equal 'y=z', env['QUERY_STRING'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_absolute_uri_with_fragment + client = MockRequest.new("GET http://e:3/x#frag HTTP/1.1\r\n" \ + "Host: foo\r\n\r\n") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert_equal '/x', env['REQUEST_PATH'] + assert_equal '/x', env['PATH_INFO'] + assert_equal '', env['QUERY_STRING'] + assert_equal 'frag', env['FRAGMENT'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_absolute_uri_with_query_and_fragment + client = MockRequest.new("GET http://e:3/x?a=b#frag HTTP/1.1\r\n" \ + "Host: foo\r\n\r\n") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert_equal '/x', env['REQUEST_PATH'] + assert_equal '/x', env['PATH_INFO'] + assert_equal 'a=b', env['QUERY_STRING'] + assert_equal 'frag', env['FRAGMENT'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_absolute_uri_unsupported_schemes + %w(ssh+http://e/ ftp://e/x http+ssh://e/x).each do |abs_uri| + client = MockRequest.new("GET #{abs_uri} HTTP/1.1\r\n" \ + "Host: foo\r\n\r\n") + assert_raises(HttpParserError) { @request.read(client) } + end + end + + def test_x_forwarded_proto_https + res = env = nil + client = MockRequest.new("GET / HTTP/1.1\r\n" \ + "X-Forwarded-Proto: https\r\n" \ + "Host: foo\r\n\r\n") + assert_nothing_raised { env = @request.read(client) } + assert_equal "https", env['rack.url_scheme'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_x_forwarded_proto_http + res = env = nil + client = MockRequest.new("GET / HTTP/1.1\r\n" \ + "X-Forwarded-Proto: http\r\n" \ + "Host: foo\r\n\r\n") + assert_nothing_raised { env = @request.read(client) } + assert_equal "http", env['rack.url_scheme'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_x_forwarded_proto_invalid + res = env = nil + client = MockRequest.new("GET / HTTP/1.1\r\n" \ + "X-Forwarded-Proto: ftp\r\n" \ + "Host: foo\r\n\r\n") + assert_nothing_raised { env = @request.read(client) } + assert_equal "http", env['rack.url_scheme'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_rack_lint_get + client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert_equal "http", env['rack.url_scheme'] + assert_equal '127.0.0.1', env['REMOTE_ADDR'] + assert_nothing_raised { res = @lint.call(env) } + end + + def test_rack_lint_put + client = MockRequest.new( + "PUT / HTTP/1.1\r\n" \ + "Host: foo\r\n" \ + "Content-Length: 5\r\n" \ + "\r\n" \ + "abcde") + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert ! env.include?(:http_body) + assert_nothing_raised { res = @lint.call(env) } + end + + def test_rack_lint_big_put + count = 100 + bs = 0x10000 + buf = (' ' * bs).freeze + length = bs * count + client = Tempfile.new('big_put') + client.syswrite( + "PUT / HTTP/1.1\r\n" \ + "Host: foo\r\n" \ + "Content-Length: #{length}\r\n" \ + "\r\n") + count.times { assert_equal bs, client.syswrite(buf) } + assert_equal 0, client.sysseek(0) + res = env = nil + assert_nothing_raised { env = @request.read(client) } + assert ! env.include?(:http_body) + assert_equal length, env['rack.input'].size + count.times { assert_equal buf, env['rack.input'].read(bs) } + assert_nil env['rack.input'].read(bs) + assert_nothing_raised { env['rack.input'].rewind } + assert_nothing_raised { res = @lint.call(env) } + end + +end + diff --git a/test/unit/test_response.rb b/test/unit/test_response.rb index c30a141..66c2b54 100644 --- a/test/unit/test_response.rb +++ b/test/unit/test_response.rb @@ -13,16 +13,26 @@ class ResponseTest < Test::Unit::TestCase def test_response_headers out = StringIO.new HttpResponse.write(out,[200, {"X-Whatever" => "stuff"}, ["cool"]]) + assert out.closed? assert out.length > 0, "output didn't have data" end + def test_response_string_status + out = StringIO.new + HttpResponse.write(out,['200', {}, []]) + assert out.closed? + assert out.length > 0, "output didn't have data" + assert_equal 1, out.string.split(/\r\n/).grep(/^Status: 200 OK/).size + end + def test_response_OFS_set old_ofs = $, $, = "\f\v" out = StringIO.new - HttpResponse.write(out,[200, {"X-Whatever" => "stuff"}, ["cool"]]) - resp = out.read + HttpResponse.write(out,[200, {"X-k" => "cd","X-y" => "z"}, ["cool"]]) + assert out.closed? + resp = out.string assert ! resp.include?("\f\v"), "output didn't use $, ($OFS)" ensure $, = old_ofs @@ -31,6 +41,7 @@ class ResponseTest < Test::Unit::TestCase def test_response_200 io = StringIO.new HttpResponse.write(io, [200, {}, []]) + assert io.closed? assert io.length > 0, "output didn't have data" end @@ -38,8 +49,49 @@ class ResponseTest < Test::Unit::TestCase code = 400 io = StringIO.new HttpResponse.write(io, [code, {}, []]) - io.rewind - assert_match(/.* #{HTTP_STATUS_CODES[code]}$/, io.readline.chomp, "wrong default reason phrase") + assert io.closed? + lines = io.string.split(/\r\n/) + assert_match(/.* Bad Request$/, lines.first, + "wrong default reason phrase") end -end + def test_rack_multivalue_headers + out = StringIO.new + HttpResponse.write(out,[200, {"X-Whatever" => "stuff\nbleh"}, []]) + assert out.closed? + assert_match(/^X-Whatever: stuff\r\nX-Whatever: bleh\r\n/, out.string) + end + + # Even though Rack explicitly forbids "Status" in the header hash, + # some broken clients still rely on it + def test_status_header_added + out = StringIO.new + HttpResponse.write(out,[200, {"X-Whatever" => "stuff"}, []]) + assert out.closed? + assert_equal 1, out.string.split(/\r\n/).grep(/^Status: 200 OK/i).size + end + + # we always favor the code returned by the application, since "Status" + # in the header hash is not allowed by Rack (but not every app is + # fully Rack-compliant). + def test_status_header_ignores_app_hash + out = StringIO.new + header_hash = {"X-Whatever" => "stuff", 'StaTus' => "666" } + HttpResponse.write(out,[200, header_hash, []]) + assert out.closed? + assert_equal 1, out.string.split(/\r\n/).grep(/^Status: 200 OK/i).size + assert_equal 1, out.string.split(/\r\n/).grep(/^Status:/i).size + end + + def test_body_closed + expect_body = %w(1 2 3 4).join("\n") + body = StringIO.new(expect_body) + body.rewind + out = StringIO.new + HttpResponse.write(out,[200, {}, body]) + assert out.closed? + assert body.closed? + assert_match(expect_body, out.string.split(/\r\n/).last) + end + +end diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb index d19064c..742b240 100644 --- a/test/unit/test_server.rb +++ b/test/unit/test_server.rb @@ -25,8 +25,8 @@ class WebServerTest < Test::Unit::TestCase @tester = TestHandler.new redirect_test_io do @server = HttpServer.new(@tester, :listeners => [ "127.0.0.1:#{@port}" ] ) + @server.start end - @server.start end def teardown @@ -35,6 +35,60 @@ class WebServerTest < Test::Unit::TestCase end end + def test_preload_app_config + teardown + tmp = Tempfile.new('test_preload_app_config') + ObjectSpace.undefine_finalizer(tmp) + app = lambda { || + tmp.sysseek(0) + tmp.truncate(0) + tmp.syswrite($$) + lambda { |env| [ 200, { 'Content-Type' => 'text/plain' }, [ "#$$\n" ] ] } + } + redirect_test_io do + @server = HttpServer.new(app, :listeners => [ "127.0.0.1:#@port"] ) + @server.start + end + results = hit(["http://localhost:#@port/"]) + worker_pid = results[0].to_i + tmp.sysseek(0) + loader_pid = tmp.sysread(4096).to_i + assert_equal worker_pid, loader_pid + teardown + + redirect_test_io do + @server = HttpServer.new(app, :listeners => [ "127.0.0.1:#@port"], + :preload_app => true) + @server.start + end + results = hit(["http://localhost:#@port/"]) + worker_pid = results[0].to_i + tmp.sysseek(0) + loader_pid = tmp.sysread(4096).to_i + assert_equal $$, loader_pid + assert worker_pid != loader_pid + ensure + tmp.close! + end + + def test_broken_app + teardown + app = lambda { |env| raise RuntimeError, "hello" } + # [200, {}, []] } + redirect_test_io do + @server = HttpServer.new(app, :listeners => [ "127.0.0.1:#@port"] ) + @server.start + end + sock = nil + assert_nothing_raised do + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + end + + assert_match %r{\AHTTP/1.[01] 500\b}, sock.sysread(4096) + assert_nothing_raised { sock.close } + end + def test_simple_server results = hit(["http://localhost:#{@port}/test"]) assert_equal 'hello!\n', results[0], "Handler didn't really run" @@ -77,6 +131,16 @@ class WebServerTest < Test::Unit::TestCase end end + def test_bad_client_400 + sock = nil + assert_nothing_raised do + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\nHost: foo\rbar\r\n\r\n") + end + assert_match %r{\AHTTP/1.[01] 400\b}, sock.sysread(4096) + assert_nothing_raised { sock.close } + end + def test_header_is_too_long redirect_test_io do long = "GET /test HTTP/1.1\r\n" + ("X-Big: stuff\r\n" * 15000) + "\r\n" diff --git a/test/unit/test_signals.rb b/test/unit/test_signals.rb new file mode 100644 index 0000000..ef66ed6 --- /dev/null +++ b/test/unit/test_signals.rb @@ -0,0 +1,191 @@ +# Copyright (c) 2009 Eric Wong +# You can redistribute it and/or modify it under the same terms as Ruby. +# +# Ensure we stay sane in the face of signals being sent to us + +require 'test/test_helper' + +include Unicorn + +class Dd + def initialize(bs, count) + @count = count + @buf = ' ' * bs + end + + def each(&block) + @count.times { yield @buf } + end +end + +class SignalsTest < Test::Unit::TestCase + + def setup + @bs = 1 * 1024 * 1024 + @count = 100 + @port = unused_port + tmp = @tmp = Tempfile.new('unicorn.sock') + File.unlink(@tmp.path) + n = 0 + tmp.chmod(0) + @server_opts = { + :listeners => [ "127.0.0.1:#@port", @tmp.path ], + :after_fork => lambda { |server,worker| + trap(:HUP) { tmp.chmod(n += 1) } + }, + } + @server = nil + end + + def test_worker_dies_on_dead_master + pid = fork { + app = lambda { |env| [ 200, {'X-Pid' => "#$$" }, [] ] } + opts = @server_opts.merge(:timeout => 3) + redirect_test_io { HttpServer.new(app, opts).start.join } + } + child = sock = buf = t0 = nil + assert_nothing_raised do + wait_workers_ready("test_stderr.#{pid}.log", 1) + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + buf = sock.readpartial(4096) + sock.close + buf =~ /\bX-Pid: (\d+)\b/ or raise Exception + child = $1.to_i + wait_master_ready("test_stderr.#{pid}.log") + Process.kill(:KILL, pid) + Process.waitpid(pid) + t0 = Time.now + end + assert child + assert t0 + assert_raises(Errno::ESRCH) { loop { Process.kill(0, child); sleep 0.2 } } + assert((Time.now - t0) < 60) + end + + def test_sleepy_kill + rd, wr = IO.pipe + pid = fork { + rd.close + app = lambda { |env| wr.syswrite('.'); sleep; [ 200, {}, [] ] } + redirect_test_io { HttpServer.new(app, @server_opts).start.join } + } + sock = buf = nil + wr.close + assert_nothing_raised do + wait_workers_ready("test_stderr.#{pid}.log", 1) + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + buf = rd.readpartial(1) + wait_master_ready("test_stderr.#{pid}.log") + Process.kill(:INT, pid) + Process.waitpid(pid) + end + assert_equal '.', buf + buf = nil + assert_raises(EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL, + Errno::EBADF) do + buf = sock.sysread(4096) + end + assert_nil buf + ensure + end + + def test_timeout_slow_response + pid = fork { + app = lambda { |env| sleep } + opts = @server_opts.merge(:timeout => 3) + redirect_test_io { HttpServer.new(app, opts).start.join } + } + t0 = Time.now + sock = nil + assert_nothing_raised do + wait_workers_ready("test_stderr.#{pid}.log", 1) + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + end + + buf = nil + assert_raises(EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL, + Errno::EBADF) do + buf = sock.sysread(4096) + end + diff = Time.now - t0 + assert_nil buf + assert diff > 1.0, "diff was #{diff.inspect}" + assert diff < 60.0 + ensure + Process.kill(:QUIT, pid) rescue nil + end + + def test_response_write + app = lambda { |env| + [ 200, { 'Content-Type' => 'text/plain', 'X-Pid' => Process.pid.to_s }, + Dd.new(@bs, @count) ] + } + redirect_test_io { @server = HttpServer.new(app, @server_opts).start } + sock = nil + assert_nothing_raised do + wait_workers_ready("test_stderr.#{$$}.log", 1) + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + end + buf = '' + header_len = pid = nil + assert_nothing_raised do + buf = sock.sysread(16384, buf) + pid = buf[/\r\nX-Pid: (\d+)\r\n/, 1].to_i + header_len = buf[/\A(.+?\r\n\r\n)/m, 1].size + end + read = buf.size + mode_before = @tmp.stat.mode + assert_raises(EOFError,Errno::ECONNRESET,Errno::EPIPE,Errno::EINVAL, + Errno::EBADF) do + loop do + 3.times { Process.kill(:HUP, pid) } + sock.sysread(16384, buf) + read += buf.size + 3.times { Process.kill(:HUP, pid) } + end + end + + redirect_test_io { @server.stop(true) } + # can't check for == since pending signals get merged + assert mode_before < @tmp.stat.mode + assert_equal(read - header_len, @bs * @count) + assert_nothing_raised { sock.close } + end + + def test_request_read + app = lambda { |env| + [ 200, {'Content-Type'=>'text/plain', 'X-Pid'=>Process.pid.to_s}, [] ] + } + redirect_test_io { @server = HttpServer.new(app, @server_opts).start } + pid = nil + + assert_nothing_raised do + wait_workers_ready("test_stderr.#{$$}.log", 1) + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("GET / HTTP/1.0\r\n\r\n") + pid = sock.sysread(4096)[/\r\nX-Pid: (\d+)\r\n/, 1].to_i + sock.close + end + + sock = TCPSocket.new('127.0.0.1', @port) + sock.syswrite("PUT / HTTP/1.0\r\n") + sock.syswrite("Content-Length: #{@bs * @count}\r\n\r\n") + 1000.times { Process.kill(:HUP, pid) } + mode_before = @tmp.stat.mode + killer = fork { loop { Process.kill(:HUP, pid); sleep(0.0001) } } + buf = ' ' * @bs + @count.times { sock.syswrite(buf) } + Process.kill(:TERM, killer) + Process.waitpid2(killer) + redirect_test_io { @server.stop(true) } + # can't check for == since pending signals get merged + assert mode_before < @tmp.stat.mode + assert_equal pid, sock.sysread(4096)[/\r\nX-Pid: (\d+)\r\n/, 1].to_i + sock.close + end + +end diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb new file mode 100644 index 0000000..75d9f7b --- /dev/null +++ b/test/unit/test_socket_helper.rb @@ -0,0 +1,131 @@ +require 'test/test_helper' +require 'tempfile' + +class TestSocketHelper < Test::Unit::TestCase + include Unicorn::SocketHelper + attr_reader :logger + GET_SLASH = "GET / HTTP/1.0\r\n\r\n".freeze + + def setup + @log_tmp = Tempfile.new 'logger' + @logger = Logger.new(@log_tmp.path) + @test_addr = ENV['UNICORN_TEST_ADDR'] || '127.0.0.1' + GC.disable + end + + def teardown + GC.enable + end + + def test_bind_listen_tcp + port = unused_port @test_addr + @tcp_listener_name = "#@test_addr:#{port}" + @tcp_listener = bind_listen(@tcp_listener_name) + assert TCPServer === @tcp_listener + assert_equal @tcp_listener_name, sock_name(@tcp_listener) + end + + def test_bind_listen_options + port = unused_port @test_addr + tcp_listener_name = "#@test_addr:#{port}" + tmp = Tempfile.new 'unix.sock' + unix_listener_name = tmp.path + File.unlink(tmp.path) + [ { :backlog => 5 }, { :sndbuf => 4096 }, { :rcvbuf => 4096 }, + { :backlog => 16, :rcvbuf => 4096, :sndbuf => 4096 } + ].each do |opts| + assert_nothing_raised do + tcp_listener = bind_listen(tcp_listener_name, opts) + assert TCPServer === tcp_listener + tcp_listener.close + unix_listener = bind_listen(unix_listener_name, opts) + assert UNIXServer === unix_listener + unix_listener.close + end + end + #system('cat', @log_tmp.path) + end + + def test_bind_listen_unix + old_umask = File.umask(0777) + tmp = Tempfile.new 'unix.sock' + @unix_listener_path = tmp.path + File.unlink(@unix_listener_path) + @unix_listener = bind_listen(@unix_listener_path) + assert UNIXServer === @unix_listener + assert_equal @unix_listener_path, sock_name(@unix_listener) + assert File.readable?(@unix_listener_path), "not readable" + assert File.writable?(@unix_listener_path), "not writable" + assert_equal 0777, File.umask + ensure + File.umask(old_umask) + end + + def test_bind_listen_unix_idempotent + test_bind_listen_unix + a = bind_listen(@unix_listener) + assert_equal a.fileno, @unix_listener.fileno + unix_server = server_cast(@unix_listener) + assert UNIXServer === unix_server + a = bind_listen(unix_server) + assert_equal a.fileno, unix_server.fileno + assert_equal a.fileno, @unix_listener.fileno + end + + def test_bind_listen_tcp_idempotent + test_bind_listen_tcp + a = bind_listen(@tcp_listener) + assert_equal a.fileno, @tcp_listener.fileno + tcp_server = server_cast(@tcp_listener) + assert TCPServer === tcp_server + a = bind_listen(tcp_server) + assert_equal a.fileno, tcp_server.fileno + assert_equal a.fileno, @tcp_listener.fileno + end + + def test_bind_listen_unix_rebind + test_bind_listen_unix + new_listener = bind_listen(@unix_listener_path) + assert UNIXServer === new_listener + assert new_listener.fileno != @unix_listener.fileno + assert_equal sock_name(new_listener), sock_name(@unix_listener) + assert_equal @unix_listener_path, sock_name(new_listener) + pid = fork do + client = server_cast(new_listener).accept + client.syswrite('abcde') + exit 0 + end + s = UNIXSocket.new(@unix_listener_path) + IO.select([s]) + assert_equal 'abcde', s.sysread(5) + pid, status = Process.waitpid2(pid) + assert status.success? + end + + def test_server_cast + assert_nothing_raised do + test_bind_listen_unix + test_bind_listen_tcp + end + unix_listener_socket = Socket.for_fd(@unix_listener.fileno) + assert Socket === unix_listener_socket + @unix_server = server_cast(unix_listener_socket) + assert_equal @unix_listener.fileno, @unix_server.fileno + assert UNIXServer === @unix_server + assert File.socket?(@unix_server.path) + assert_equal @unix_listener_path, sock_name(@unix_server) + + tcp_listener_socket = Socket.for_fd(@tcp_listener.fileno) + assert Socket === tcp_listener_socket + @tcp_server = server_cast(tcp_listener_socket) + assert_equal @tcp_listener.fileno, @tcp_server.fileno + assert TCPServer === @tcp_server + assert_equal @tcp_listener_name, sock_name(@tcp_server) + end + + def test_sock_name + test_server_cast + sock_name(@unix_server) + end + +end diff --git a/test/unit/test_upload.rb b/test/unit/test_upload.rb index edc94da..9ef3ed7 100644 --- a/test/unit/test_upload.rb +++ b/test/unit/test_upload.rb @@ -18,12 +18,29 @@ class UploadTest < Test::Unit::TestCase @sha1 = Digest::SHA1.new @sha1_app = lambda do |env| input = env['rack.input'] - resp = { :pos => input.pos, :size => input.stat.size } + resp = { :pos => input.pos, :size => input.size, :class => input.class } + + # sysread + @sha1.reset begin loop { @sha1.update(input.sysread(@bs)) } rescue EOFError end resp[:sha1] = @sha1.hexdigest + + # read + input.sysseek(0) if input.respond_to?(:sysseek) + input.rewind + @sha1.reset + loop { + buf = input.read(@bs) or break + @sha1.update(buf) + } + + if resp[:sha1] == @sha1.hexdigest + resp[:sysread_read_byte_match] = true + end + [ 200, @hdr.merge({'X-Resp' => resp.inspect}), [] ] end end @@ -50,6 +67,61 @@ class UploadTest < Test::Unit::TestCase assert_equal @sha1.hexdigest, resp[:sha1] end + def test_put_trickle_small + @count, @bs = 2, 128 + start_server(@sha1_app) + assert_equal 256, length + sock = TCPSocket.new(@addr, @port) + hdr = "PUT / HTTP/1.0\r\nContent-Length: #{length}\r\n\r\n" + @count.times do + buf = @random.sysread(@bs) + @sha1.update(buf) + hdr << buf + sock.syswrite(hdr) + hdr = '' + sleep 0.6 + end + read = sock.read.split(/\r\n/) + assert_equal "HTTP/1.1 200 OK", read[0] + resp = eval(read.grep(/^X-Resp: /).first.sub!(/X-Resp: /, '')) + assert_equal length, resp[:size] + assert_equal 0, resp[:pos] + assert_equal @sha1.hexdigest, resp[:sha1] + assert_equal StringIO, resp[:class] + end + + def test_tempfile_unlinked + spew_path = lambda do |env| + if orig = env['HTTP_X_OLD_PATH'] + assert orig != env['rack.input'].path + end + assert_equal length, env['rack.input'].size + [ 200, @hdr.merge('X-Tempfile-Path' => env['rack.input'].path), [] ] + end + start_server(spew_path) + sock = TCPSocket.new(@addr, @port) + sock.syswrite("PUT / HTTP/1.0\r\nContent-Length: #{length}\r\n\r\n") + @count.times { sock.syswrite(' ' * @bs) } + path = sock.read[/^X-Tempfile-Path: (\S+)/, 1] + sock.close + + # send another request to ensure we hit the next request + sock = TCPSocket.new(@addr, @port) + sock.syswrite("PUT / HTTP/1.0\r\nX-Old-Path: #{path}\r\n" \ + "Content-Length: #{length}\r\n\r\n") + @count.times { sock.syswrite(' ' * @bs) } + path2 = sock.read[/^X-Tempfile-Path: (\S+)/, 1] + sock.close + assert path != path2 + + # make sure the next request comes in so the unlink got processed + sock = TCPSocket.new(@addr, @port) + sock.syswrite("GET ?lasdf\r\n\r\n\r\n\r\n") + sock.sysread(4096) rescue nil + sock.close + + assert ! File.exist?(path) + end def test_put_keepalive_truncates_small_overwrite start_server(@sha1_app) @@ -135,6 +207,52 @@ class UploadTest < Test::Unit::TestCase assert_equal resp[:size], new_tmp.stat.size end + # Despite reading numerous articles and inspecting the 1.9.1-p0 C + # source, Eric Wong will never trust that we're always handling + # encoding-aware IO objects correctly. Thus this test uses shell + # utilities that should always operate on files/sockets on a + # byte-level. + def test_uncomfortable_with_onenine_encodings + # POSIX doesn't require all of these to be present on a system + which('curl') or return + which('sha1sum') or return + which('dd') or return + + start_server(@sha1_app) + + tmp = Tempfile.new('dd_dest') + assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}", + "bs=#{@bs}", "count=#{@count}"), + "dd #@random to #{tmp}") + sha1_re = %r!\b([a-f0-9]{40})\b! + sha1_out = `sha1sum #{tmp.path}` + assert $?.success?, 'sha1sum ran OK' + + assert_match(sha1_re, sha1_out) + sha1 = sha1_re.match(sha1_out)[1] + resp = `curl -isSfN -T#{tmp.path} http://#@addr:#@port/` + assert $?.success?, 'curl ran OK' + assert_match(%r!\b#{sha1}\b!, resp) + assert_match(/Tempfile/, resp) + assert_match(/sysread_read_byte_match/, resp) + + # small StringIO path + assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}", + "bs=1024", "count=1"), + "dd #@random to #{tmp}") + sha1_re = %r!\b([a-f0-9]{40})\b! + sha1_out = `sha1sum #{tmp.path}` + assert $?.success?, 'sha1sum ran OK' + + assert_match(sha1_re, sha1_out) + sha1 = sha1_re.match(sha1_out)[1] + resp = `curl -isSfN -T#{tmp.path} http://#@addr:#@port/` + assert $?.success?, 'curl ran OK' + assert_match(%r!\b#{sha1}\b!, resp) + assert_match(/StringIO/, resp) + assert_match(/sysread_read_byte_match/, resp) + end + private def length diff --git a/test/unit/test_util.rb b/test/unit/test_util.rb new file mode 100644 index 0000000..1616eac --- /dev/null +++ b/test/unit/test_util.rb @@ -0,0 +1,87 @@ +require 'test/test_helper' +require 'tempfile' + +class TestUtil < Test::Unit::TestCase + + EXPECT_FLAGS = File::WRONLY | File::APPEND + def test_reopen_logs_noop + tmp = Tempfile.new(nil) + tmp.reopen(tmp.path, 'a') + tmp.sync = true + ext = tmp.external_encoding rescue nil + int = tmp.internal_encoding rescue nil + before = tmp.stat.inspect + Unicorn::Util.reopen_logs + assert_equal before, File.stat(tmp.path).inspect + assert_equal ext, (tmp.external_encoding rescue nil) + assert_equal int, (tmp.internal_encoding rescue nil) + end + + def test_reopen_logs_renamed + tmp = Tempfile.new(nil) + tmp_path = tmp.path.freeze + tmp.reopen(tmp_path, 'a') + tmp.sync = true + ext = tmp.external_encoding rescue nil + int = tmp.internal_encoding rescue nil + before = tmp.stat.inspect + to = Tempfile.new(nil) + File.rename(tmp_path, to.path) + assert ! File.exist?(tmp_path) + Unicorn::Util.reopen_logs + assert_equal tmp_path, tmp.path + assert File.exist?(tmp_path) + assert before != File.stat(tmp_path).inspect + assert_equal tmp.stat.inspect, File.stat(tmp_path).inspect + assert_equal ext, (tmp.external_encoding rescue nil) + assert_equal int, (tmp.internal_encoding rescue nil) + assert_equal(EXPECT_FLAGS, EXPECT_FLAGS & tmp.fcntl(Fcntl::F_GETFL)) + assert tmp.sync + end + + def test_reopen_logs_renamed_with_encoding + tmp = Tempfile.new(nil) + tmp_path = tmp.path.dup.freeze + Encoding.list.each { |encoding| + tmp.reopen(tmp_path, "a:#{encoding.to_s}") + tmp.sync = true + assert_equal encoding, tmp.external_encoding + assert_nil tmp.internal_encoding + File.unlink(tmp_path) + assert ! File.exist?(tmp_path) + Unicorn::Util.reopen_logs + assert_equal tmp_path, tmp.path + assert File.exist?(tmp_path) + assert_equal tmp.stat.inspect, File.stat(tmp_path).inspect + assert_equal encoding, tmp.external_encoding + assert_nil tmp.internal_encoding + assert_equal(EXPECT_FLAGS, EXPECT_FLAGS & tmp.fcntl(Fcntl::F_GETFL)) + assert tmp.sync + } + end if STDIN.respond_to?(:external_encoding) + + def test_reopen_logs_renamed_with_internal_encoding + tmp = Tempfile.new(nil) + tmp_path = tmp.path.dup.freeze + Encoding.list.each { |ext| + Encoding.list.each { |int| + next if ext == int + tmp.reopen(tmp_path, "a:#{ext.to_s}:#{int.to_s}") + tmp.sync = true + assert_equal ext, tmp.external_encoding + assert_equal int, tmp.internal_encoding + File.unlink(tmp_path) + assert ! File.exist?(tmp_path) + Unicorn::Util.reopen_logs + assert_equal tmp_path, tmp.path + assert File.exist?(tmp_path) + assert_equal tmp.stat.inspect, File.stat(tmp_path).inspect + assert_equal ext, tmp.external_encoding + assert_equal int, tmp.internal_encoding + assert_equal(EXPECT_FLAGS, EXPECT_FLAGS & tmp.fcntl(Fcntl::F_GETFL)) + assert tmp.sync + } + } + end if STDIN.respond_to?(:external_encoding) + +end |