[PATCH 0/4] a few more minor cleanups pushed out

[PATCH 0/4] a few more minor cleanups pushed out

[Eric Wong]

Not much going on other than killing old stuff and minor cleanups.

I'll probably give systemd a try soon and see if the
"unicorn_forever" script makes sense to bundle:
http://bogomips.org/unicorn-public/m/20130724031151.GA14534@dcvr.yhbt.net.txt

Since CoW-friendliness is important to unicorn users, I think we'll
always have to manage our own process forking.

For now, I think using the UNICORN_FD environment variable
(comma-delimited list of integer file descriptors) and matching "listen"
directives is enough to get started.  Perhaps supporting something like:

        listen :inherit

...can help DRY-up configurations so users won't have to specify
redundant listen directives.  Ruby startup times are still horrific; so
I don't think anybody wants to use socket activation with on-demand
fork+exec of unicorn processes.

The following changes since commit f203eaae7ea84de9e974ea5dac2df97d664d8e61:

  http_response: remove Status: header (2014-08-17 19:26:17 +0000)

are available in the git repository at:

  git://bogomips.org/unicorn master

for you to fetch changes up to 4b2782a926d8f131b1e7382be35e3abb77bf4be5:

  http: reduce parser from 72 to 56 bytes on 64-bit (2014-09-17 03:15:25 +0000)

----------------------------------------------------------------
Eric Wong (4):
      remove RubyForge and Freecode references
      remove mongrel.rubyforge.org references
      http: remove the keepalive requests limit
      http: reduce parser from 72 to 56 bytes on 64-bit

 GNUmakefile                      |  7 ----
 Rakefile                         | 44 ----------------------
 ext/unicorn_http/unicorn_http.rl | 55 ++++++---------------------
 test/test_helper.rb              |  4 +-
 test/unit/test_http_parser.rb    |  4 +-
 test/unit/test_http_parser_ng.rb | 80 ++++------------------------------------
 test/unit/test_response.rb       |  4 +-
 test/unit/test_server.rb         |  4 +-
 unicorn.gemspec                  |  1 -
 9 files changed, 26 insertions(+), 177 deletions(-)

[PATCH 1/4] remove RubyForge and Freecode references

[Eric Wong]

Both sites are gone.
---
 GNUmakefile     |  7 -------
 Rakefile        | 44 --------------------------------------------
 unicorn.gemspec |  1 -
 3 files changed, 52 deletions(-)

diff --git a/GNUmakefile b/GNUmakefile
index 50819fc..6916b6e 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -245,15 +245,8 @@ $(pkgtgz): .manifest fix-perms
 package: $(pkgtgz) $(pkggem)
 
 release: verify package $(release_notes) $(release_changes)
-	# make tgz release on RubyForge
-	rubyforge add_release -f -n $(release_notes) -a $(release_changes) \
-	  $(rfproject) $(rfpackage) $(VERSION) $(pkgtgz)
 	# push gem to Gemcutter
 	gem push $(pkggem)
-	# in case of gem downloads from RubyForge releases page
-	-rubyforge add_file \
-	  $(rfproject) $(rfpackage) $(VERSION) $(pkggem)
-	$(RAKE) fm_update VERSION=$(VERSION)
 else
 gem install-gem: GIT-VERSION-FILE
 	$(MAKE) $@ VERSION=$(GIT_VERSION)
diff --git a/Rakefile b/Rakefile
index 01ff5d0..37569ce 100644
--- a/Rakefile
+++ b/Rakefile
@@ -1,47 +1,3 @@
-# -*- encoding: binary -*-
-autoload :Gem, 'rubygems'
-require 'wrongdoc'
-
-cgit_url = Wrongdoc.config[:cgit_url]
-git_url = Wrongdoc.config[:git_url]
-
-desc "post to FM"
-task :fm_update do
-  require 'tempfile'
-  require 'net/http'
-  require 'net/netrc'
-  require 'json'
-  version = ENV['VERSION'] or abort "VERSION= needed"
-  uri = URI.parse('https://freecode.com/projects/unicorn/releases.json')
-  rc = Net::Netrc.locate('unicorn-fm') or abort "~/.netrc not found"
-  api_token = rc.password
-  _, subject, body = `git cat-file tag v#{version}`.split(/\n\n/, 3)
-  tmp = Tempfile.new('fm-changelog')
-  tmp.puts subject
-  tmp.puts
-  tmp.puts body
-  tmp.flush
-  system(ENV["VISUAL"], tmp.path) or abort "#{ENV["VISUAL"]} failed: #$?"
-  changelog = File.read(tmp.path).strip
-
-  req = {
-    "auth_code" => api_token,
-    "release" => {
-      "tag_list" => "Experimental",
-      "version" => version,
-      "changelog" => changelog,
-    },
-  }.to_json
-
-  if ! changelog.strip.empty? && version =~ %r{\A[\d\.]+\d+\z}
-    Net::HTTP.start(uri.host, uri.port, :use_ssl => true) do |http|
-      p http.post(uri.path, req, {'Content-Type'=>'application/json'})
-    end
-  else
-    warn "not updating freshmeat for v#{version}"
-  end
-end
-
 # optional rake-compiler support in case somebody needs to cross compile
 begin
   mk = "ext/unicorn_http/Makefile"
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 9456db2..8623d44 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -26,7 +26,6 @@ Gem::Specification.new do |s|
   s.files = manifest
   s.homepage = Wrongdoc.config[:rdoc_url]
   s.rdoc_options = rdoc_options
-  s.rubyforge_project = %q{mongrel}
   s.test_files = test_files
 
   # for people that are absolutely stuck on Rails 2.3.2 and can't
-- 
EW

[PATCH 2/4] remove mongrel.rubyforge.org references

[Eric Wong]

mongrel.rubyforge.org has been dead longer than rubyforge.org!
---
 test/test_helper.rb           | 4 ++--
 test/unit/test_http_parser.rb | 4 ++--
 test/unit/test_response.rb    | 4 ++--
 test/unit/test_server.rb      | 4 ++--
 4 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/test/test_helper.rb b/test/test_helper.rb
index c65f2f3..c4fe07a 100644
--- a/test/test_helper.rb
+++ b/test/test_helper.rb
@@ -1,10 +1,10 @@
 # -*- encoding: binary -*-
 
-# Copyright (c) 2005 Zed A. Shaw 
+# Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
 # the GPLv2+ (GPLv3+ preferred)
 #
-# Additional work donated by contributors.  See http://mongrel.rubyforge.org/attributions.html 
+# Additional work donated by contributors.  See git history
 # for more information.
 
 STDIN.sync = STDOUT.sync = STDERR.sync = true # buffering makes debugging hard
diff --git a/test/unit/test_http_parser.rb b/test/unit/test_http_parser.rb
index 8d5b251..2251dcf 100644
--- a/test/unit/test_http_parser.rb
+++ b/test/unit/test_http_parser.rb
@@ -1,10 +1,10 @@
 # -*- encoding: binary -*-
 
-# Copyright (c) 2005 Zed A. Shaw 
+# Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
 # the GPLv2+ (GPLv3+ preferred)
 #
-# Additional work donated by contributors.  See http://mongrel.rubyforge.org/attributions.html
+# Additional work donated by contributors.  See git history
 # for more information.
 
 require 'test/test_helper'
diff --git a/test/unit/test_response.rb b/test/unit/test_response.rb
index fcddc5e..bdca9f5 100644
--- a/test/unit/test_response.rb
+++ b/test/unit/test_response.rb
@@ -1,10 +1,10 @@
 # -*- encoding: binary -*-
 
-# Copyright (c) 2005 Zed A. Shaw 
+# Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
 # the GPLv2+ (GPLv3+ preferred)
 #
-# Additional work donated by contributors.  See http://mongrel.rubyforge.org/attributions.html 
+# Additional work donated by contributors.  See git history
 # for more information.
 
 require 'test/test_helper'
diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index e5b335f..9c92bab 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -1,10 +1,10 @@
 # -*- encoding: binary -*-
 
-# Copyright (c) 2005 Zed A. Shaw 
+# Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
 # the GPLv2+ (GPLv3+ preferred)
 #
-# Additional work donated by contributors.  See http://mongrel.rubyforge.org/attributions.html
+# Additional work donated by contributors.  See git history
 # for more information.
 
 require 'test/test_helper'
-- 
EW

[PATCH 3/4] http: remove the keepalive requests limit

[Eric Wong]

This was a hack for some event loops such as those found in nginx
and some Rainbows! concurrency models.  Using epoll/kqueue with
one-shot notification (which yahns does) avoids all fairness
problems.
---
 ext/unicorn_http/unicorn_http.rl | 37 ++------------------
 test/unit/test_http_parser_ng.rb | 74 +---------------------------------------
 2 files changed, 3 insertions(+), 108 deletions(-)

diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index ecdadb0..9372d39 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -29,29 +29,6 @@ void init_unicorn_httpdate(void);
 /* all of these flags need to be set for keepalive to be supported */
 #define UH_FL_KEEPALIVE (UH_FL_KAVERSION | UH_FL_REQEOF | UH_FL_HASHEADER)
 
-static unsigned long keepalive_requests = 100; /* same as nginx */
-
-/*
- * Returns the maximum number of keepalive requests a client may make
- * before the parser refuses to continue.
- */
-static VALUE ka_req(VALUE self)
-{
-  return ULONG2NUM(keepalive_requests);
-}
-
-/*
- * Sets the maximum number of keepalive requests a client may make.
- * A special value of +nil+ causes this to be the maximum value
- * possible (this is architecture-dependent).
- */
-static VALUE set_ka_req(VALUE self, VALUE val)
-{
-  keepalive_requests = NIL_P(val) ? ULONG_MAX : NUM2ULONG(val);
-
-  return ka_req(self);
-}
-
 static size_t MAX_HEADER_LEN = 1024 * (80 + 32); /* same as Mongrel */
 
 /* this is only intended for use with Rainbows! */
@@ -64,7 +41,6 @@ static VALUE set_maxhdrlen(VALUE self, VALUE len)
 struct http_parser {
   int cs; /* Ragel internal state */
   unsigned int flags;
-  unsigned long nr_requests;
   size_t mark;
   size_t offset;
   union { /* these 2 fields don't nest */
@@ -580,7 +556,6 @@ static VALUE HttpParser_init(VALUE self)
   http_parser_init(hp);
   hp->buf = rb_str_new(NULL, 0);
   hp->env = rb_hash_new();
-  hp->nr_requests = keepalive_requests;
 
   return self;
 }
@@ -814,15 +789,13 @@ static VALUE HttpParser_keepalive(VALUE self)
  *    parser.next? => true or false
  *
  * Exactly like HttpParser#keepalive?, except it will reset the internal
- * parser state on next parse if it returns true.  It will also respect
- * the maximum *keepalive_requests* value and return false if that is
- * reached.
+ * parser state on next parse if it returns true.
  */
 static VALUE HttpParser_next(VALUE self)
 {
   struct http_parser *hp = data_get(self);
 
-  if ((HP_FL_ALL(hp, KEEPALIVE)) && (hp->nr_requests-- != 0)) {
+  if (HP_FL_ALL(hp, KEEPALIVE)) {
     HP_FL_SET(hp, TO_CLEAR);
     return Qtrue;
   }
@@ -984,12 +957,6 @@ void Init_unicorn_http(void)
    */
   rb_define_const(cHttpParser, "LENGTH_MAX", OFFT2NUM(UH_OFF_T_MAX));
 
-  /* default value for keepalive_requests */
-  rb_define_const(cHttpParser, "KEEPALIVE_REQUESTS_DEFAULT",
-                  ULONG2NUM(keepalive_requests));
-
-  rb_define_singleton_method(cHttpParser, "keepalive_requests", ka_req, 0);
-  rb_define_singleton_method(cHttpParser, "keepalive_requests=", set_ka_req, 1);
   rb_define_singleton_method(cHttpParser, "max_header_len=", set_maxhdrlen, 1);
 
   init_common_fields();
diff --git a/test/unit/test_http_parser_ng.rb b/test/unit/test_http_parser_ng.rb
index ab335ac..9167845 100644
--- a/test/unit/test_http_parser_ng.rb
+++ b/test/unit/test_http_parser_ng.rb
@@ -8,7 +8,6 @@ include Unicorn
 class HttpParserNgTest < Test::Unit::TestCase
 
   def setup
-    HttpParser.keepalive_requests = HttpParser::KEEPALIVE_REQUESTS_DEFAULT
     @parser = HttpParser.new
   end
 
@@ -29,25 +28,6 @@ class HttpParserNgTest < Test::Unit::TestCase
     assert_equal false, @parser.response_start_sent
   end
 
-  def test_keepalive_requests_default_constant
-    assert_kind_of Integer, HttpParser::KEEPALIVE_REQUESTS_DEFAULT
-    assert HttpParser::KEEPALIVE_REQUESTS_DEFAULT >= 0
-  end
-
-  def test_keepalive_requests_setting
-    HttpParser.keepalive_requests = 0
-    assert_equal 0, HttpParser.keepalive_requests
-    HttpParser.keepalive_requests = nil
-    assert HttpParser.keepalive_requests >= 0xffffffff
-    HttpParser.keepalive_requests = 1
-    assert_equal 1, HttpParser.keepalive_requests
-    HttpParser.keepalive_requests = 666
-    assert_equal 666, HttpParser.keepalive_requests
-
-    assert_raises(TypeError) { HttpParser.keepalive_requests = "666" }
-    assert_raises(TypeError) { HttpParser.keepalive_requests = [] }
-  end
-
   def test_connection_TE
     @parser.buf << "GET / HTTP/1.1\r\nHost: example.com\r\nConnection: TE\r\n"
     @parser.buf << "TE: trailers\r\n\r\n"
@@ -71,41 +51,11 @@ class HttpParserNgTest < Test::Unit::TestCase
       "REQUEST_METHOD" => "GET",
       "QUERY_STRING" => ""
     }.freeze
-    HttpParser::KEEPALIVE_REQUESTS_DEFAULT.times do |nr|
-      @parser.buf << req
-      assert_equal expect, @parser.parse
-      assert @parser.next?
-    end
-    @parser.buf << req
-    assert_equal expect, @parser.parse
-    assert ! @parser.next?
-  end
-
-  def test_fewer_keepalive_requests_with_next?
-    HttpParser.keepalive_requests = 5
-    @parser = HttpParser.new
-    req = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n".freeze
-    expect = {
-      "SERVER_NAME" => "example.com",
-      "HTTP_HOST" => "example.com",
-      "rack.url_scheme" => "http",
-      "REQUEST_PATH" => "/",
-      "SERVER_PROTOCOL" => "HTTP/1.1",
-      "PATH_INFO" => "/",
-      "HTTP_VERSION" => "HTTP/1.1",
-      "REQUEST_URI" => "/",
-      "SERVER_PORT" => "80",
-      "REQUEST_METHOD" => "GET",
-      "QUERY_STRING" => ""
-    }.freeze
-    5.times do |nr|
+    100.times do |nr|
       @parser.buf << req
       assert_equal expect, @parser.parse
       assert @parser.next?
     end
-    @parser.buf << req
-    assert_equal expect, @parser.parse
-    assert ! @parser.next?
   end
 
   def test_default_keepalive_is_off
@@ -664,28 +614,6 @@ class HttpParserNgTest < Test::Unit::TestCase
     assert_equal "", @parser.buf
   end
 
-  def test_keepalive_requests_disabled
-    req = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n".freeze
-    expect = {
-      "SERVER_NAME" => "example.com",
-      "HTTP_HOST" => "example.com",
-      "rack.url_scheme" => "http",
-      "REQUEST_PATH" => "/",
-      "SERVER_PROTOCOL" => "HTTP/1.1",
-      "PATH_INFO" => "/",
-      "HTTP_VERSION" => "HTTP/1.1",
-      "REQUEST_URI" => "/",
-      "SERVER_PORT" => "80",
-      "REQUEST_METHOD" => "GET",
-      "QUERY_STRING" => ""
-    }.freeze
-    HttpParser.keepalive_requests = 0
-    @parser = HttpParser.new
-    @parser.buf << req
-    assert_equal expect, @parser.parse
-    assert ! @parser.next?
-  end
-
   def test_chunk_only
     tmp = ""
     assert_equal @parser, @parser.dechunk!
-- 
EW

[PATCH 4/4] http: reduce parser from 72 to 56 bytes on 64-bit

[Eric Wong]

This allows the parser struct to fit in one cache line on
x86-64 systems where cache lines are 64 bytes.

Using 32-bit integer lengths is safe here because these are only for
tracking offsets within the HTTP header buffer.  We can safely limit
HTTP headers and in-memory buffers to be less than 4GB without
anybody complaining.

HTTP bodies continue to use off_t (usually 64-bit, even on 32-bit
systems) sizes and support as much as the OS/hardware can handle.
---
 ext/unicorn_http/unicorn_http.rl | 18 +++++++++---------
 test/unit/test_http_parser_ng.rb |  6 ++++++
 2 files changed, 15 insertions(+), 9 deletions(-)

diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 9372d39..3294357 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -29,27 +29,27 @@ void init_unicorn_httpdate(void);
 /* all of these flags need to be set for keepalive to be supported */
 #define UH_FL_KEEPALIVE (UH_FL_KAVERSION | UH_FL_REQEOF | UH_FL_HASHEADER)
 
-static size_t MAX_HEADER_LEN = 1024 * (80 + 32); /* same as Mongrel */
+static unsigned int MAX_HEADER_LEN = 1024 * (80 + 32); /* same as Mongrel */
 
 /* this is only intended for use with Rainbows! */
 static VALUE set_maxhdrlen(VALUE self, VALUE len)
 {
-  return SIZET2NUM(MAX_HEADER_LEN = NUM2SIZET(len));
+  return UINT2NUM(MAX_HEADER_LEN = NUM2UINT(len));
 }
 
 /* keep this small for Rainbows! since every client has one */
 struct http_parser {
   int cs; /* Ragel internal state */
   unsigned int flags;
-  size_t mark;
-  size_t offset;
+  unsigned int mark;
+  unsigned int offset;
   union { /* these 2 fields don't nest */
-    size_t field;
-    size_t query;
+    unsigned int field;
+    unsigned int query;
   } start;
   union {
-    size_t field_len; /* only used during header processing */
-    size_t dest_offset; /* only used during body processing */
+    unsigned int field_len; /* only used during header processing */
+    unsigned int dest_offset; /* only used during body processing */
   } s;
   VALUE buf;
   VALUE env;
@@ -74,7 +74,7 @@ static void parser_raise(VALUE klass, const char *msg)
 }
 
 #define REMAINING (unsigned long)(pe - p)
-#define LEN(AT, FPC) (FPC - buffer - hp->AT)
+#define LEN(AT, FPC) (FPC - buffer - (unsigned long)hp->AT)
 #define MARK(M,FPC) (hp->M = (FPC) - buffer)
 #define PTR_TO(F) (buffer + hp->F)
 #define STR_NEW(M,FPC) rb_str_new(PTR_TO(M), LEN(M, FPC))
diff --git a/test/unit/test_http_parser_ng.rb b/test/unit/test_http_parser_ng.rb
index 9167845..d5c8d2e 100644
--- a/test/unit/test_http_parser_ng.rb
+++ b/test/unit/test_http_parser_ng.rb
@@ -11,6 +11,12 @@ class HttpParserNgTest < Test::Unit::TestCase
     @parser = HttpParser.new
   end
 
+  def test_parser_max_len
+    assert_raises(RangeError) do
+      HttpParser.max_header_len = 0xffffffff + 1
+    end
+  end
+
   def test_next_clear
     r = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"
     @parser.buf << r
-- 
EW