about summary refs log tree commit homepage
path: root/test
diff options
context:
space:
mode:
authorEric Wong <normalperson@yhbt.net>2009-03-10 18:44:34 -0700
committerEric Wong <normalperson@yhbt.net>2009-03-22 19:04:27 -0700
commit98ea5cca50b907e20d6357f425d7789bac1d1a47 (patch)
tree73e3823c0cd4f23f806ac392e66e12c3d77fc61c /test
parentca4265ea4d8bdeb9c569a50a05ee45e31f4b4269 (diff)
downloadunicorn-98ea5cca50b907e20d6357f425d7789bac1d1a47.tar.gz
dd.ru is a rackup file is intended as a dd(1)-like test for I/O
performance.

There are also individual request, response, and big_request
benchmarks for micro benchmarking some parts of Unicorn.

The rest of the benchmarks are gone: I am not interested in
performance comparisons (and pissing matches) with other web
servers (or their fanboys/girls).

I will _NEVER_ publically publish benchmarks comparing Unicorn
against other web servers.  I will only compare Unicorn against
other versions of Unicorn, possibly on different platforms.

Neutral third-parties are invited to publish their own
benchmarks (along with detailed procedures, version numbers and
other details) comparing Unicorn to other servers.

Signed-off-by: Eric Wong <normalperson@yhbt.net>
Diffstat (limited to 'test')
-rw-r--r--test/benchmark/README55
-rw-r--r--test/benchmark/big_request.rb35
-rw-r--r--test/benchmark/dd.ru18
-rw-r--r--test/benchmark/previous.rb11
-rw-r--r--test/benchmark/request.rb47
-rw-r--r--test/benchmark/response.rb29
-rw-r--r--test/benchmark/simple.rb11
-rw-r--r--test/benchmark/utils.rb82
8 files changed, 184 insertions, 104 deletions
diff --git a/test/benchmark/README b/test/benchmark/README
new file mode 100644
index 0000000..b63b8a3
--- /dev/null
+++ b/test/benchmark/README
@@ -0,0 +1,55 @@
+= Performance
+
+Unicorn is pretty fast, and we want it to get faster.  Unicorn strives
+to get HTTP requests to your application and write HTTP responses back
+as quickly as possible.  Unicorn does not do any background processing
+while your app runs, so your app will get all the CPU time provided to
+it by your OS kernel.
+
+A gentle reminder: Unicorn is NOT for serving clients over slow network
+connections.  Use nginx (or something similar) to complement Unicorn if
+you have slow clients.
+
+== dd.ru
+
+This is a pure I/O benchmark.  In the context of Unicorn, this is the
+only one that matters.  It is a standard rackup-compatible .ru file and
+may be used with other Rack-compatible servers.
+
+  unicorn -E none dd.ru
+
+You can change the size and number of chunks in the response with
+the "bs" and "count" environment variables.   The following command
+will cause dd.ru to return 4 chunks of 16384 bytes each, leading to
+65536 byte response:
+
+  bs=16384 count=4 unicorn -E none dd.ru
+
+Or if you want to add logging (small performance impact):
+
+  unicorn -E deployment dd.ru
+
+Eric runs then runs clients on a LAN it in several different ways:
+
+  client@host1 -> unicorn@host1(tcp)
+  client@host2 -> unicorn@host1(tcp)
+  client@host3 -> nginx@host1 -> unicorn@host1(tcp)
+  client@host3 -> nginx@host1 -> unicorn@host1(unix)
+  client@host3 -> nginx@host2 -> unicorn@host1(tcp)
+
+The benchmark client is usually httperf.
+
+Another gentle reminder: performance with slow networks/clients
+is NOT our problem.  That is the job of nginx (or similar).
+
+== request.rb, response.rb, big_request.rb
+
+These are micro-benchmarks designed to test internal components
+of Unicorn.  It assumes the internal Unicorn API is mostly stable.
+
+== Contributors
+
+This directory is maintained independently in the "benchmark" branch
+based against v0.1.0.  Only changes to this directory (test/benchmarks)
+are committed to this branch although the master branch may merge this
+branch occassionaly.
diff --git a/test/benchmark/big_request.rb b/test/benchmark/big_request.rb
new file mode 100644
index 0000000..5f2111b
--- /dev/null
+++ b/test/benchmark/big_request.rb
@@ -0,0 +1,35 @@
+require 'benchmark'
+require 'tempfile'
+require 'unicorn'
+nr = ENV['nr'] ? ENV['nr'].to_i : 100
+bs = ENV['bs'] ? ENV['bs'].to_i : (1024 * 1024)
+count = ENV['count'] ? ENV['count'].to_i : 4
+length = bs * count
+slice = (' ' * bs).freeze
+
+big = Tempfile.new('')
+def big.unicorn_peeraddr; '127.0.0.1'; end
+big.syswrite(
+"PUT /hello/world/puturl?abcd=efg&hi#anchor HTTP/1.0\r\n" \
+"Host: localhost\r\n" \
+"Accept: */*\r\n" \
+"Content-Length: #{length}\r\n" \
+"User-Agent: test-user-agent 0.1.0 (Mozilla compatible) 5.0 asdfadfasda\r\n" \
+"\r\n")
+count.times { big.syswrite(slice) }
+big.sysseek(0)
+big.fsync
+
+include Unicorn
+request = HttpRequest.new(Logger.new($stderr))
+
+Benchmark.bmbm do |x|
+  x.report("big") do
+    for i in 1..nr
+      request.read(big)
+      request.reset
+      big.sysseek(0)
+    end
+  end
+end
+
diff --git a/test/benchmark/dd.ru b/test/benchmark/dd.ru
new file mode 100644
index 0000000..7e9a3fa
--- /dev/null
+++ b/test/benchmark/dd.ru
@@ -0,0 +1,18 @@
+# This benchmark is the simplest test of the I/O facilities in
+# unicorn.  It is meant to return a fixed-sized blob to test
+# the performance of things in Unicorn, _NOT_ the app.
+#
+# Adjusting this benchmark is done via the "bs" (byte size) and "count"
+# environment variables.  "count" designates the count of elements of
+# "bs" length in the Rack response body.  The defaults are bs=4096, count=1
+# to return one 4096-byte chunk.
+bs = ENV['bs'] ? ENV['bs'].to_i : 4096
+count = ENV['count'] ? ENV['count'].to_i : 1
+slice = (' ' * bs).freeze
+body = (1..count).map { slice }.freeze
+hdr = {
+  'Content-Length' => bs * count,
+  'Content-Type' => 'text/plain'.freeze
+}.freeze
+response = [ 200, hdr, body ].freeze
+run(lambda { |env| response })
diff --git a/test/benchmark/previous.rb b/test/benchmark/previous.rb
deleted file mode 100644
index 8b6182a..0000000
--- a/test/benchmark/previous.rb
+++ /dev/null
@@ -1,11 +0,0 @@
-# Benchmark to compare Mongrel performance against
-# previous Mongrel version (the one installed as a gem).
-#
-# Run with:
-#
-#  ruby previous.rb [num of request]
-#
-
-require File.dirname(__FILE__) + '/utils'
-
-benchmark "print", %w(current gem), 1000, [1, 10, 100]
diff --git a/test/benchmark/request.rb b/test/benchmark/request.rb
new file mode 100644
index 0000000..67266cb
--- /dev/null
+++ b/test/benchmark/request.rb
@@ -0,0 +1,47 @@
+require 'benchmark'
+require 'unicorn'
+nr = ENV['nr'] ? ENV['nr'].to_i : 100000
+
+class TestClient
+  def initialize(response)
+    @response = (response.join("\r\n") << "\r\n\r\n").freeze
+  end
+  def sysread(len, buf)
+    buf.replace(@response)
+  end
+
+  def unicorn_peeraddr
+    '127.0.0.1'
+  end
+end
+
+small = TestClient.new([
+  'GET / HTTP/1.0',
+  'Host: localhost',
+  'Accept: */*',
+  'User-Agent: test-user-agent 0.1.0'
+])
+
+medium = TestClient.new([
+  'GET /hello/world/geturl?abcd=efg&hi#anchor HTTP/1.0',
+  'Host: localhost',
+  'Accept: */*',
+  'User-Agent: test-user-agent 0.1.0 (Mozilla compatible) 5.0 asdfadfasda'
+])
+
+include Unicorn
+request = HttpRequest.new(Logger.new($stderr))
+Benchmark.bmbm do |x|
+  x.report("small") do
+    for i in 1..nr
+      request.read(small)
+      request.reset
+    end
+  end
+  x.report("medium") do
+    for i in 1..nr
+      request.read(medium)
+      request.reset
+    end
+  end
+end
diff --git a/test/benchmark/response.rb b/test/benchmark/response.rb
new file mode 100644
index 0000000..95291f1
--- /dev/null
+++ b/test/benchmark/response.rb
@@ -0,0 +1,29 @@
+require 'benchmark'
+require 'unicorn'
+
+class NullWriter
+  def syswrite(buf); buf.size; end
+  def close; end
+end
+
+include Unicorn
+
+socket = NullWriter.new
+bs = ENV['bs'] ? ENV['bs'].to_i : 4096
+count = ENV['count'] ? ENV['count'].to_i : 1
+slice = (' ' * bs).freeze
+body = (1..count).map { slice }.freeze
+hdr = {
+  'Content-Length' => bs * count,
+  'Content-Type' => 'text/plain'.freeze
+}.freeze
+response = [ 200, hdr, body ].freeze
+
+nr = ENV['nr'] ? ENV['nr'].to_i : 100000
+Benchmark.bmbm do |x|
+  x.report do
+    for i in 1..nr
+      HttpResponse.write(socket.dup, response)
+    end
+  end
+end
diff --git a/test/benchmark/simple.rb b/test/benchmark/simple.rb
deleted file mode 100644
index 906f74c..0000000
--- a/test/benchmark/simple.rb
+++ /dev/null
@@ -1,11 +0,0 @@
-#
-# Simple benchmark to compare Mongrel performance against
-# other webservers supported by Rack.
-#
-
-require File.dirname(__FILE__) + '/utils'
-
-libs = %w(current gem WEBrick EMongrel Thin)
-libs = ARGV if ARGV.any?
-
-benchmark "print", libs, 1000, [1, 10, 100]
diff --git a/test/benchmark/utils.rb b/test/benchmark/utils.rb
deleted file mode 100644
index feb22c1..0000000
--- a/test/benchmark/utils.rb
+++ /dev/null
@@ -1,82 +0,0 @@
-
-require 'rubygems'
-require 'rack'
-require 'rack/lobster'
-
-def run(handler_name, n=1000, c=1)
-  port = 7000
-  
-  server = fork do
-    [STDOUT, STDERR].each { |o| o.reopen "/dev/null" }
-      
-    case handler_name
-    when 'EMongrel'
-      require 'swiftcore/evented_mongrel'
-      handler_name = 'Mongrel'
-    
-    when 'Thin'
-      require 'thin'
-      hander_name = 'Thin'
-    
-    when 'gem' # Load the current Mongrel gem
-      require 'mongrel'
-      handler_name = 'Mongrel'
-    
-    when 'current' # Load the current Mongrel version under /lib
-      require File.dirname(__FILE__) + '/../lib/mongrel'
-      handler_name = 'Mongrel'
-      
-    end
-    
-    app = Rack::Lobster.new
-    
-    handler = Rack::Handler.const_get(handler_name)
-    handler.run app, :Host => '0.0.0.0', :Port => port
-  end
-
-  sleep 2
-
-  out = `nice -n20 ab -c #{c} -n #{n} http://127.0.0.1:#{port}/ 2> /dev/null`
-
-  Process.kill('SIGKILL', server)
-  Process.wait
-  
-  if requests = out.match(/^Requests.+?(\d+\.\d+)/)
-    requests[1].to_i
-  else
-    0
-  end
-end
-
-def benchmark(type, servers, request, concurrency_levels)
-  send "#{type}_benchmark", servers, request, concurrency_levels
-end
-
-def graph_benchmark(servers, request, concurrency_levels)
-  require '/usr/local/lib/ruby/gems/1.8/gems/gruff-0.2.9/lib/gruff'
-  g = Gruff::Area.new
-  g.title = "Server benchmark"
-  
-  servers.each do |server|
-    g.data(server, concurrency_levels.collect { |c| print '.'; run(server, request, c) })
-  end
-  puts
-  
-  g.x_axis_label = 'Concurrency'
-  g.y_axis_label = 'Requests / sec'
-  g.labels = {}
-  concurrency_levels.each_with_index { |c, i| g.labels[i] = c.to_s }
-  
-  g.write('bench.png')
-  `open bench.png`
-end
-
-def print_benchmark(servers, request, concurrency_levels)
-  puts 'server     request   concurrency   req/s'
-  puts '=' * 42
-  concurrency_levels.each do |c|
-    servers.each do |server|
-      puts "#{server.ljust(8)}   #{request}      #{c.to_s.ljust(4)}          #{run(server, request, c)}"
-    end
-  end
-end \ No newline at end of file