* [ANN] unicorn 5.0.0.pre1 - incompatible changes!
@ 2015-06-15 22:56 5% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2015-06-15 22:56 UTC (permalink / raw)
To: unicorn-public
This release finally drops Ruby 1.8 support and requires Ruby 1.9.3
or later. The horrible "Status:" header in our HTTP response is
finally gone, saving at least 16 precious bytes in every single HTTP
response.
Under Ruby 2.1 and later, the monotonic clock is used for timeout
handling for better accuracy.
Several experimental, unused and undocumented features are removed.
There's also tiny, minor performance and memory improvements from
dropping 1.8 compatibility, but probably nothing noticeable on a
typical real-life (bloated) app.
The biggest performance improvement we made was to our website by
switching to olddoc. Depending on connection speed, latency, and
renderer performance, it typically loads two to four times faster.
Finally, for the billionth time: unicorn must never be exposed
to slow clients, as it will never ever use new-fangled things
like non-blocking socket I/O, threads, epoll or kqueue. unicorn
must be used with a fully-buffering reverse proxy such as nginx
for slow clients.
I'll tag 5.0.0 final in a week or so if all goes well
= gem install --pre unicorn
= git clone git://bogomips.org/unicorn.git
= http://unicorn.bogomips.org/
* ISSUES: update with mailing list subscription
* GIT-VERSION-GEN: start 5.0.0 development
* http: remove xftrust options
* FAQ: add entry for Rails autoflush_log
* dev: remove isolate dependency
* unicorn.gemspec: depend on test-unit 3.0
* http_response: remove Status: header
* remove RubyForge and Freecode references
* remove mongrel.rubyforge.org references
* http: remove the keepalive requests limit
* http: reduce parser from 72 to 56 bytes on 64-bit
* examples: add run_once to before_fork hook example
* worker: remove old tmp accessor
* http_server: save 450+ bytes of memory on x86-64
* t/t0002-parser-error.sh: relax test for rack 1.6.0
* remove SSL support
* tmpio: drop the "size" method
* switch docs + website to olddoc
* README: clarify/reduce references to unicorn_rails
* gemspec: fixup olddoc migration
* use the monotonic clock under Ruby 2.1+
* http: -Wshorten-64-to-32 warnings on clang
* remove old inetd+git examples and exec_cgi
* http: standalone require + reduction in binary size
* GNUmakefile: fix clean gem build + reduce build cruft
* socket_helper: reduce constant lookups and caching
* remove 1.8, <= 1.9.1 fallback for missing IO#autoclose=
* favor IO#close_on_exec= over fcntl in 1.9+
* use require_relative to reduce syscalls at startup
* doc: update support status for Ruby versions
* fix uninstalled testing and reduce require paths
* test_socket_helper: do not depend on SO_REUSEPORT
* favor "a.b(&:c)" form over "a.b { |x| x.c }"
* ISSUES: add section for bugs in other projects
* http_server: favor ivars over constants
* explain 11 byte magic number for self-pipe
* const: drop constants used by Rainbows!
* reduce and localize constant string use
* Links: mark Rainbows! as historical, reference yahns
* save about 200 bytes of memory on x86-64
* http: remove deprecated reset method
* http: remove experimental dechunk! method
* socket_helper: update comments
* doc: document UNICORN_FD in manpage
* doc: document Etc.nprocessors for worker_processes
* favor more string literals for cold call sites
* tee_input: support for Rack::TempfileReaper middleware
* support TempfileReaper in deployment and development envs
* favor kgio_wait_readable for single FD over select
* Merge tag 'v4.9.0'
* http_request: support rack.hijack by default
* avoid extra allocation for hijack proc creation
* FAQ: add note about ECONNRESET errors from bodies
* process SIGWINCH unless stdin is a TTY
* ISSUES: discourage HTML mail strongly, welcome nyms
* http: use rb_hash_clear in Ruby 2.0+
* http_response: avoid special-casing for Rack < 1.5
* www: install NEWS.atom.xml properly
* http_server: remove a few more accessors and constants
* http_response: simplify regular expression
* move the socket into Rack env for hijacking
* http: move response_start_sent into the C ext
* FAQ: reorder bit on Rack 1.1.x and Rails 2.3.x
* ensure body is closed during hijack
--
EW
^ permalink raw reply [relevance 5%]
* [PATCH] http: -Wshorten-64-to-32 warnings on clang
@ 2015-01-28 18:44 4% Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2015-01-28 18:44 UTC (permalink / raw)
To: unicorn-public
Tested on x86_64, clang version 3.5-1ubuntu1 (trunk) (LLVM 3.5)
These warnings were introduced on
commit 4b2782a926d8f131b1e7382be35e3abb77bf4be5
("http: reduce parser from 72 to 56 bytes on 64-bit")
and did not affect any releases.
These length checks should not be necessary in reality because
HTTP header sizes never come close to 4GB in size.
Fixup a minor coding style (inherited from Mongrel) violation
while we're at it (tabs => spaces).
---
ext/unicorn_http/unicorn_http.rl | 22 +++++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)
diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 3294357..de83652 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -69,13 +69,25 @@ static void parser_raise(VALUE klass, const char *msg)
VALUE exc = rb_exc_new2(klass, msg);
VALUE bt = rb_ary_new();
- rb_funcall(exc, id_set_backtrace, 1, bt);
- rb_exc_raise(exc);
+ rb_funcall(exc, id_set_backtrace, 1, bt);
+ rb_exc_raise(exc);
+}
+
+static inline unsigned int ulong2uint(unsigned long n)
+{
+ unsigned int i = (unsigned int)n;
+
+ if (sizeof(unsigned int) != sizeof(unsigned long)) {
+ if ((unsigned long)i != n) {
+ rb_raise(rb_eRangeError, "too large to be 32-bit uint: %lu", n);
+ }
+ }
+ return i;
}
#define REMAINING (unsigned long)(pe - p)
-#define LEN(AT, FPC) (FPC - buffer - (unsigned long)hp->AT)
-#define MARK(M,FPC) (hp->M = (FPC) - buffer)
+#define LEN(AT, FPC) (ulong2uint(FPC - buffer) - hp->AT)
+#define MARK(M,FPC) (hp->M = ulong2uint((FPC) - buffer))
#define PTR_TO(F) (buffer + hp->F)
#define STR_NEW(M,FPC) rb_str_new(PTR_TO(M), LEN(M, FPC))
#define STRIPPED_STR_NEW(M,FPC) stripped_str_new(PTR_TO(M), LEN(M, FPC))
@@ -411,7 +423,7 @@ http_parser_execute(struct http_parser *hp, char *buffer, size_t len)
post_exec: /* "_out:" also goes here */
if (hp->cs != http_parser_error)
hp->cs = cs;
- hp->offset = p - buffer;
+ hp->offset = ulong2uint(p - buffer);
assert(p <= pe && "buffer overflow after parsing execute");
assert(hp->offset <= len && "offset longer than length");
--
EW
^ permalink raw reply related [relevance 4%]
* [PATCH 4/4] http: reduce parser from 72 to 56 bytes on 64-bit
2014-09-22 1:05 5% [PATCH 0/4] a few more minor cleanups pushed out Eric Wong
@ 2014-09-22 1:05 7% ` Eric Wong
0 siblings, 0 replies; 4+ results
From: Eric Wong @ 2014-09-22 1:05 UTC (permalink / raw)
To: unicorn-public; +Cc: Eric Wong
This allows the parser struct to fit in one cache line on
x86-64 systems where cache lines are 64 bytes.
Using 32-bit integer lengths is safe here because these are only for
tracking offsets within the HTTP header buffer. We can safely limit
HTTP headers and in-memory buffers to be less than 4GB without
anybody complaining.
HTTP bodies continue to use off_t (usually 64-bit, even on 32-bit
systems) sizes and support as much as the OS/hardware can handle.
---
ext/unicorn_http/unicorn_http.rl | 18 +++++++++---------
test/unit/test_http_parser_ng.rb | 6 ++++++
2 files changed, 15 insertions(+), 9 deletions(-)
diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 9372d39..3294357 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -29,27 +29,27 @@ void init_unicorn_httpdate(void);
/* all of these flags need to be set for keepalive to be supported */
#define UH_FL_KEEPALIVE (UH_FL_KAVERSION | UH_FL_REQEOF | UH_FL_HASHEADER)
-static size_t MAX_HEADER_LEN = 1024 * (80 + 32); /* same as Mongrel */
+static unsigned int MAX_HEADER_LEN = 1024 * (80 + 32); /* same as Mongrel */
/* this is only intended for use with Rainbows! */
static VALUE set_maxhdrlen(VALUE self, VALUE len)
{
- return SIZET2NUM(MAX_HEADER_LEN = NUM2SIZET(len));
+ return UINT2NUM(MAX_HEADER_LEN = NUM2UINT(len));
}
/* keep this small for Rainbows! since every client has one */
struct http_parser {
int cs; /* Ragel internal state */
unsigned int flags;
- size_t mark;
- size_t offset;
+ unsigned int mark;
+ unsigned int offset;
union { /* these 2 fields don't nest */
- size_t field;
- size_t query;
+ unsigned int field;
+ unsigned int query;
} start;
union {
- size_t field_len; /* only used during header processing */
- size_t dest_offset; /* only used during body processing */
+ unsigned int field_len; /* only used during header processing */
+ unsigned int dest_offset; /* only used during body processing */
} s;
VALUE buf;
VALUE env;
@@ -74,7 +74,7 @@ static void parser_raise(VALUE klass, const char *msg)
}
#define REMAINING (unsigned long)(pe - p)
-#define LEN(AT, FPC) (FPC - buffer - hp->AT)
+#define LEN(AT, FPC) (FPC - buffer - (unsigned long)hp->AT)
#define MARK(M,FPC) (hp->M = (FPC) - buffer)
#define PTR_TO(F) (buffer + hp->F)
#define STR_NEW(M,FPC) rb_str_new(PTR_TO(M), LEN(M, FPC))
diff --git a/test/unit/test_http_parser_ng.rb b/test/unit/test_http_parser_ng.rb
index 9167845..d5c8d2e 100644
--- a/test/unit/test_http_parser_ng.rb
+++ b/test/unit/test_http_parser_ng.rb
@@ -11,6 +11,12 @@ class HttpParserNgTest < Test::Unit::TestCase
@parser = HttpParser.new
end
+ def test_parser_max_len
+ assert_raises(RangeError) do
+ HttpParser.max_header_len = 0xffffffff + 1
+ end
+ end
+
def test_next_clear
r = "GET / HTTP/1.1\r\nHost: example.com\r\n\r\n"
@parser.buf << r
--
EW
^ permalink raw reply related [relevance 7%]
* [PATCH 0/4] a few more minor cleanups pushed out
@ 2014-09-22 1:05 5% Eric Wong
2014-09-22 1:05 7% ` [PATCH 4/4] http: reduce parser from 72 to 56 bytes on 64-bit Eric Wong
0 siblings, 1 reply; 4+ results
From: Eric Wong @ 2014-09-22 1:05 UTC (permalink / raw)
To: unicorn-public
Not much going on other than killing old stuff and minor cleanups.
I'll probably give systemd a try soon and see if the
"unicorn_forever" script makes sense to bundle:
http://bogomips.org/unicorn-public/m/20130724031151.GA14534@dcvr.yhbt.net.txt
Since CoW-friendliness is important to unicorn users, I think we'll
always have to manage our own process forking.
For now, I think using the UNICORN_FD environment variable
(comma-delimited list of integer file descriptors) and matching "listen"
directives is enough to get started. Perhaps supporting something like:
listen :inherit
...can help DRY-up configurations so users won't have to specify
redundant listen directives. Ruby startup times are still horrific; so
I don't think anybody wants to use socket activation with on-demand
fork+exec of unicorn processes.
The following changes since commit f203eaae7ea84de9e974ea5dac2df97d664d8e61:
http_response: remove Status: header (2014-08-17 19:26:17 +0000)
are available in the git repository at:
git://bogomips.org/unicorn master
for you to fetch changes up to 4b2782a926d8f131b1e7382be35e3abb77bf4be5:
http: reduce parser from 72 to 56 bytes on 64-bit (2014-09-17 03:15:25 +0000)
----------------------------------------------------------------
Eric Wong (4):
remove RubyForge and Freecode references
remove mongrel.rubyforge.org references
http: remove the keepalive requests limit
http: reduce parser from 72 to 56 bytes on 64-bit
GNUmakefile | 7 ----
Rakefile | 44 ----------------------
ext/unicorn_http/unicorn_http.rl | 55 ++++++---------------------
test/test_helper.rb | 4 +-
test/unit/test_http_parser.rb | 4 +-
test/unit/test_http_parser_ng.rb | 80 ++++------------------------------------
test/unit/test_response.rb | 4 +-
test/unit/test_server.rb | 4 +-
unicorn.gemspec | 1 -
9 files changed, 26 insertions(+), 177 deletions(-)
^ permalink raw reply [relevance 5%]
Results 1-4 of 4 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2014-09-22 1:05 5% [PATCH 0/4] a few more minor cleanups pushed out Eric Wong
2014-09-22 1:05 7% ` [PATCH 4/4] http: reduce parser from 72 to 56 bytes on 64-bit Eric Wong
2015-01-28 18:44 4% [PATCH] http: -Wshorten-64-to-32 warnings on clang Eric Wong
2015-06-15 22:56 5% [ANN] unicorn 5.0.0.pre1 - incompatible changes! Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/unicorn.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).