unicorn Ruby/Rack server user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH 0/4] a small pile of patches
@ 2024-03-23 19:45  4% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2024-03-23 19:45 UTC (permalink / raw)
  To: unicorn-public

[-- Attachment #1: Type: text/plain, Size: 4446 bytes --]

Some stuff to future-proof against future Ruby incompatibilities.
More coming....

I've also pushed out preliminary work (started in 2021) to the
`pico' branch to switch the HTTP parser from Ragel to
picohttpparser.  It will simplify the build + maintenance,
especially when distros carry different Ragel versions (or don't
package it all, as some hackers can't afford bandwidth and disk
for a C++ toolchain).

Other notes: New releases will probably be hosted on yhbt.net if
the Rubygems.org MFA threshold is reached.  Caring about the
identity of hackers is totally misguided when we already show
our code (and even document it!).  If you can't audit the code
yourself, get an actual professional to do it and don't bother
amateurs like me.

Eric Wong (4):
  t/integration: disable proxies when running curl(1)
  tests: port back-out-of-upgrade to Perl 5
  doc: various updates and disclaimers
  treewide: future-proof frozen_string_literal changes

 HACKING                             |  13 +++-
 README                              |   9 +++
 Rakefile                            |   1 +
 TODO                                |   4 +-
 bin/unicorn                         |   1 +
 bin/unicorn_rails                   |   1 +
 examples/big_app_gc.rb              |   1 +
 examples/echo.ru                    |   1 +
 examples/logger_mp_safe.rb          |   1 +
 examples/unicorn.conf.minimal.rb    |   1 +
 examples/unicorn.conf.rb            |   1 +
 ext/unicorn_http/extconf.rb         |   1 +
 lib/unicorn.rb                      |   1 +
 lib/unicorn/app/old_rails.rb        |   1 +
 lib/unicorn/app/old_rails/static.rb |   1 +
 lib/unicorn/cgi_wrapper.rb          |   1 +
 lib/unicorn/configurator.rb         |   1 +
 lib/unicorn/const.rb                |   1 +
 lib/unicorn/http_request.rb         |   1 +
 lib/unicorn/http_response.rb        |   1 +
 lib/unicorn/http_server.rb          |   1 +
 lib/unicorn/launcher.rb             |   1 +
 lib/unicorn/oob_gc.rb               |   1 +
 lib/unicorn/preread_input.rb        |   1 +
 lib/unicorn/select_waiter.rb        |   1 +
 lib/unicorn/socket_helper.rb        |   1 +
 lib/unicorn/stream_input.rb         |   1 +
 lib/unicorn/tee_input.rb            |   1 +
 lib/unicorn/tmpio.rb                |   1 +
 lib/unicorn/util.rb                 |   1 +
 lib/unicorn/worker.rb               |   1 +
 setup.rb                            |   1 +
 t/back-out-of-upgrade.t             |  44 +++++++++++
 t/broken-app.ru                     |   1 +
 t/client_body_buffer_size.ru        |   1 +
 t/detach.ru                         |   1 +
 t/env.ru                            |   1 +
 t/fails-rack-lint.ru                |   1 +
 t/heartbeat-timeout.ru              |   1 +
 t/integration.ru                    |   1 +
 t/integration.t                     |   1 +
 t/lib.perl                          |  67 ++++++++++++++---
 t/listener_names.ru                 |   1 +
 t/oob_gc.ru                         |   1 +
 t/oob_gc_path.ru                    |   1 +
 t/pid.ru                            |   1 +
 t/preread_input.ru                  |   1 +
 t/reopen-logs.ru                    |   1 +
 t/t0008-back_out_of_upgrade.sh      | 110 ----------------------------
 t/t0013.ru                          |   1 +
 t/t0014.ru                          |   1 +
 t/t0301.ru                          |   1 +
 test/aggregate.rb                   |   1 +
 test/benchmark/dd.ru                |   1 +
 test/benchmark/ddstream.ru          |   1 +
 test/benchmark/readinput.ru         |   1 +
 test/benchmark/stack.ru             |   1 +
 test/exec/test_exec.rb              |   1 +
 test/test_helper.rb                 |   1 +
 test/unit/test_ccc.rb               |   1 +
 test/unit/test_configurator.rb      |   1 +
 test/unit/test_droplet.rb           |   1 +
 test/unit/test_http_parser.rb       |   1 +
 test/unit/test_http_parser_ng.rb    |   1 +
 test/unit/test_request.rb           |   1 +
 test/unit/test_server.rb            |   1 +
 test/unit/test_signals.rb           |   1 +
 test/unit/test_socket_helper.rb     |   1 +
 test/unit/test_stream_input.rb      |   1 +
 test/unit/test_tee_input.rb         |   1 +
 test/unit/test_util.rb              |   1 +
 test/unit/test_waiter.rb            |   1 +
 unicorn.gemspec                     |   1 +
 73 files changed, 188 insertions(+), 126 deletions(-)
 create mode 100644 t/back-out-of-upgrade.t
 delete mode 100755 t/t0008-back_out_of_upgrade.sh

[-- Attachment #2: 0001-t-integration-disable-proxies-when-running-curl-1.patch --]
[-- Type: text/x-diff, Size: 737 bytes --]

From f3acce5dce62ac4b0288d3c0ddf0a6db2cbd9e7f Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Tue, 9 Jan 2024 21:35:08 +0000
Subject: [PATCH 1/4] t/integration: disable proxies when running curl(1)

This was also done in t/test-lib.sh, but using '*' is more
encompassing.
---
 t/integration.t | 1 +
 1 file changed, 1 insertion(+)

diff --git a/t/integration.t b/t/integration.t
index 7310ff2..d17ace0 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -27,6 +27,7 @@ listen "$u1"
 EOM
 my $ar = unicorn(qw(-E none t/integration.ru -c), $u_conf, { 3 => $srv });
 my $curl = which('curl');
+local $ENV{NO_PROXY} = '*'; # for curl
 my $fifo = "$tmpdir/fifo";
 POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
 my %PUT = (

[-- Attachment #3: 0002-tests-port-back-out-of-upgrade-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 8889 bytes --]

From 724fb631c76f09964ec289ee8e144886ba15d380 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 6 Nov 2023 05:45:29 +0000
Subject: [PATCH 2/4] tests: port back-out-of-upgrade to Perl 5

Another place where we can be faster without adding more
dependencies on Ruby maintaining stable behavior.
---
 t/back-out-of-upgrade.t        |  44 +++++++++++++
 t/lib.perl                     |  67 +++++++++++++++++---
 t/t0008-back_out_of_upgrade.sh | 110 ---------------------------------
 3 files changed, 102 insertions(+), 119 deletions(-)
 create mode 100644 t/back-out-of-upgrade.t
 delete mode 100755 t/t0008-back_out_of_upgrade.sh

diff --git a/t/back-out-of-upgrade.t b/t/back-out-of-upgrade.t
new file mode 100644
index 0000000..cf3b09f
--- /dev/null
+++ b/t/back-out-of-upgrade.t
@@ -0,0 +1,44 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# test backing out of USR2 upgrade
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+my $srv = tcp_server();
+mkfifo_die $fifo;
+write_file '>', $u_conf, <<EOM;
+preload_app true
+stderr_path "$err_log"
+pid "$pid_file"
+after_fork { |s,w| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
+EOM
+my $ar = unicorn(qw(-E none t/pid.ru -c), $u_conf, { 3 => $srv });
+
+like(my $wpid_orig_1 = slurp($fifo), qr/\Apid=\d+\z/a, 'got worker pid');
+
+ok $ar->do_kill('USR2'), 'USR2 to start upgrade';
+ok $ar->do_kill('WINCH'), 'drop old worker';
+
+like(my $wpid_new = slurp($fifo), qr/\Apid=\d+\z/a, 'got pid from new master');
+chomp(my $new_pid = slurp($pid_file));
+isnt $new_pid, $ar->{pid}, 'PID file changed';
+chomp(my $pid_oldbin = slurp("$pid_file.oldbin"));
+is $pid_oldbin, $ar->{pid}, '.oldbin PID valid';
+
+ok $ar->do_kill('HUP'), 'HUP old master';
+like(my $wpid_orig_2 = slurp($fifo), qr/\Apid=\d+\z/a, 'got worker new pid');
+ok kill('QUIT', $new_pid), 'abort old master';
+kill_until_dead $new_pid;
+
+my ($st, $hdr, $req_pid) = do_req $srv, 'GET /';
+chomp $req_pid;
+is $wpid_orig_2, "pid=$req_pid", 'new worker on old worker serves';
+
+ok !-f "$pid_file.oldbin", '.oldbin PID file gone';
+chomp(my $old_pid = slurp($pid_file));
+is $old_pid, $ar->{pid}, 'PID file restored';
+
+my @log = grep !/ERROR -- : reaped .*? exec\(\)-ed/, slurp($err_log);
+check_stderr @log;
+undef $tmpdir;
+done_testing;
diff --git a/t/lib.perl b/t/lib.perl
index 9254b23..b20a2c6 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -6,30 +6,58 @@ use v5.14;
 use parent qw(Exporter);
 use autodie;
 use Test::More;
+use Socket qw(SOMAXCONN);
 use Time::HiRes qw(sleep time);
 use IO::Socket::INET;
+use IO::Socket::UNIX;
+use Carp qw(croak);
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh, $err_log, $u_sock, $u_conf, $daemon_pid,
-	$pid_file);
+	$pid_file, $wtest_sock, $fifo);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn
 	$tmpdir $errfh $err_log $u_sock $u_conf $daemon_pid $pid_file
+	$wtest_sock $fifo
 	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr
-	do_req stop_daemon sleep time);
+	do_req stop_daemon sleep time mkfifo_die kill_until_dead write_file);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
+
+$wtest_sock = "$tmpdir/wtest.sock";
 $err_log = "$tmpdir/err.log";
 $pid_file = "$tmpdir/pid";
+$fifo = "$tmpdir/fifo";
 $u_sock = "$tmpdir/u.sock";
 $u_conf = "$tmpdir/u.conf.rb";
 open($errfh, '>>', $err_log);
 
+if (my $t = $ENV{TAIL}) {
+	my @tail = $t =~ /tail/ ? split(/\s+/, $t) : (qw(tail -F));
+	push @tail, $err_log;
+	my $pid = fork;
+	if ($pid == 0) {
+		open STDOUT, '>&', \*STDERR;
+		exec @tail;
+		die "exec(@tail): $!";
+	}
+	say "# @tail";
+	sleep 0.2;
+	UnicornTest::AutoReap->new($pid);
+}
+
+sub kill_until_dead ($;%) {
+	my ($pid, %opt) = @_;
+	my $tries = $opt{tries} // 1000;
+	my $sig = $opt{sig} // 0;
+	while (CORE::kill($sig, $pid) && --$tries) { sleep(0.01) }
+	$tries or croak "PID: $pid died after signal ($sig)";
+}
+
 sub stop_daemon (;$) {
 	my ($is_END) = @_;
 	kill('TERM', $daemon_pid);
-	my $tries = 1000;
-	while (CORE::kill(0, $daemon_pid) && --$tries) { sleep(0.01) }
+	kill_until_dead $daemon_pid;
 	if ($is_END && CORE::kill(0, $daemon_pid)) { # after done_testing
 		CORE::kill('KILL', $daemon_pid);
 		die "daemon_pid=$daemon_pid did not die";
@@ -44,8 +72,9 @@ END {
 	stop_daemon(1) if defined $daemon_pid;
 };
 
-sub check_stderr () {
-	my @log = slurp($err_log);
+sub check_stderr (@) {
+	my @log = @_;
+	slurp($err_log) if !@log;
 	diag("@log") if $ENV{V};
 	my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
 	@err = grep(!/failed to set accept_filter=/, @err);
@@ -63,6 +92,16 @@ sub slurp_hdr {
 	($status, \@hdr);
 }
 
+sub unix_server (;$@) {
+	my $l = shift // $u_sock;
+	IO::Socket::UNIX->new(Listen => SOMAXCONN, Local => $l, Blocking => 0,
+				Type => SOCK_STREAM, @_);
+}
+
+sub unix_connect ($) {
+	IO::Socket::UNIX->new(Peer => $_[0], Type => SOCK_STREAM);
+}
+
 sub tcp_server {
 	my %opt = (
 		ReuseAddr => 1,
@@ -95,8 +134,7 @@ sub tcp_host_port {
 
 sub unix_start ($@) {
 	my ($dst, @req) = @_;
-	my $s = IO::Socket::UNIX->new(Peer => $dst, Type => SOCK_STREAM) or
-		BAIL_OUT "unix connect $dst: $!";
+	my $s = unix_connect($dst) or BAIL_OUT "unix connect $dst: $!";
 	$s->autoflush(1);
 	print $s @req, "\r\n\r\n" if @req;
 	$s;
@@ -201,7 +239,7 @@ sub unicorn {
 	state $ver = $ENV{TEST_RUBY_VERSION} // `$ruby -e 'print RUBY_VERSION'`;
 	state $eng = $ENV{TEST_RUBY_ENGINE} // `$ruby -e 'print RUBY_ENGINE'`;
 	state $ext = File::Spec->rel2abs("test/$eng-$ver/ext/unicorn_http");
-	state $exe = File::Spec->rel2abs('bin/unicorn');
+	state $exe = File::Spec->rel2abs("test/$eng-$ver/bin/unicorn");
 	my $pid = spawn(\%env, $ruby, '-I', $lib, '-I', $ext, $exe, @args);
 	UnicornTest::AutoReap->new($pid);
 }
@@ -219,6 +257,17 @@ sub do_req ($@) {
 	($status, $hdr, $bdy);
 }
 
+sub mkfifo_die ($;$) {
+	POSIX::mkfifo($_[0], $_[1] // 0600) or croak "mkfifo: $!";
+}
+
+sub write_file ($$@) { # mode, filename, LIST (for print)
+	open(my $fh, shift, shift);
+	print $fh @_;
+	# return $fh for futher writes if user wants it:
+	defined(wantarray) && !wantarray ? $fh : close $fh;
+}
+
 # automatically kill + reap children when this goes out-of-scope
 package UnicornTest::AutoReap;
 use v5.14;
diff --git a/t/t0008-back_out_of_upgrade.sh b/t/t0008-back_out_of_upgrade.sh
deleted file mode 100755
index 96d4057..0000000
--- a/t/t0008-back_out_of_upgrade.sh
+++ /dev/null
@@ -1,110 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 13 "backout of USR2 upgrade"
-
-worker_wait_start () {
-	test xSTART = x"$(cat $fifo)"
-	unicorn_pid=$(cat $pid)
-}
-
-t_begin "setup and start" && {
-	unicorn_setup
-	rm -f $pid.oldbin
-
-cat >> $unicorn_config <<EOF
-after_fork do |server, worker|
-  # test script will block while reading from $fifo,
-  # so notify the script on the first worker we spawn
-  # by opening the FIFO
-  if worker.nr == 0
-    File.open("$fifo", "wb") { |fp| fp.syswrite "START" }
-  end
-end
-EOF
-	unicorn -D -c $unicorn_config pid.ru
-	worker_wait_start
-	orig_master_pid=$unicorn_pid
-}
-
-t_begin "read original worker pid" && {
-	orig_worker_pid=$(curl -sSf http://$listen/)
-	test -n "$orig_worker_pid" && kill -0 $orig_worker_pid
-}
-
-t_begin "upgrade to new master" && {
-	kill -USR2 $orig_master_pid
-}
-
-t_begin "kill old worker" && {
-	kill -WINCH $orig_master_pid
-}
-
-t_begin "wait for new worker to start" && {
-	worker_wait_start
-	test $unicorn_pid -ne $orig_master_pid
-	new_master_pid=$unicorn_pid
-}
-
-t_begin "old master pid is stashed in $pid.oldbin" && {
-	test -s "$pid.oldbin"
-	test $orig_master_pid -eq $(cat $pid.oldbin)
-}
-
-t_begin "ensure old worker is no longer running" && {
-	i=0
-	while kill -0 $orig_worker_pid 2>/dev/null
-	do
-		i=$(( $i + 1 ))
-		test $i -lt 600 || die "timed out"
-		sleep 1
-	done
-}
-
-t_begin "capture pid of new worker" && {
-	new_worker_pid=$(curl -sSf http://$listen/)
-}
-
-t_begin "reload old master process" && {
-	kill -HUP $orig_master_pid
-	worker_wait_start
-}
-
-t_begin "gracefully kill new master and ensure it dies" && {
-	kill -QUIT $new_master_pid
-	i=0
-	while kill -0 $new_worker_pid 2>/dev/null
-	do
-		i=$(( $i + 1 ))
-		test $i -lt 600 || die "timed out"
-		sleep 1
-	done
-}
-
-t_begin "ensure $pid.oldbin does not exist" && {
-	i=0
-	while test -s $pid.oldbin
-	do
-		i=$(( $i + 1 ))
-		test $i -lt 600 || die "timed out"
-		sleep 1
-	done
-	while ! test -s $pid
-	do
-		i=$(( $i + 1 ))
-		test $i -lt 600 || die "timed out"
-		sleep 1
-	done
-}
-
-t_begin "ensure $pid is correct" && {
-	cur_master_pid=$(cat $pid)
-	test $orig_master_pid -eq $cur_master_pid
-}
-
-t_begin "killing succeeds" && {
-	kill $orig_master_pid
-}
-
-dbgcat r_err
-
-t_done

[-- Attachment #4: 0003-doc-various-updates-and-disclaimers.patch --]
[-- Type: text/x-diff, Size: 3343 bytes --]

From 69d15a7a51a096b6acf00ccf23e1b988076d3b5f Mon Sep 17 00:00:00 2001
From: Eric Wong <bofh@yhbt.net>
Date: Mon, 1 Jan 2024 10:43:13 +0000
Subject: [PATCH 3/4] doc: various updates and disclaimers

Covering my ass from draconian legislation.
---
 HACKING | 13 +++++++++----
 README  |  9 +++++++++
 TODO    |  4 +---
 3 files changed, 19 insertions(+), 7 deletions(-)

diff --git a/HACKING b/HACKING
index 5aca83e..777e75e 100644
--- a/HACKING
+++ b/HACKING
@@ -6,6 +6,8 @@ Like Mongrel, we use Ruby where it makes sense, and Ragel with C where
 it helps performance.  All of the code that actually runs your Rack
 application is written Ruby, Ragel or C.
 
+Ragel may be dropped in favor of a picohttpparser-based one in the future.
+
 As far as tests and documentation goes, we're not afraid to embrace Unix
 and use traditional Unix tools where they make sense and get the job
 done.
@@ -16,6 +18,9 @@ Tests are good, but slow tests make development slow, so we make tests
 faster (in parallel) with GNU make (instead of Rake) and avoiding
 RubyGems.
 
+New tests are written in Perl 5 and use TAP <https://testanything.org/>
+to ensure stability and immunity from Ruby incompatibilities.
+
 Users of GNU-based systems (such as GNU/Linux) usually have GNU make
 installed as "make" instead of "gmake".
 
@@ -69,10 +74,10 @@ supported by the versions of Ruby we target.
 
 === Ragel Compatibility
 
-We target the latest released version of Ragel and will update our code
-to keep up with new releases.  Packaged tarballs and gems include the
-generated source code so they will remain usable if compatibility is
-broken.
+We target the latest released version of Ragel in Debian and will update
+our code to keep up with new releases.  Packaged tarballs and gems
+include the generated source code so they will remain usable if
+compatibility is broken.
 
 == Contributing
 
diff --git a/README b/README
index 84c0fdf..b60ed00 100644
--- a/README
+++ b/README
@@ -122,6 +122,7 @@ supported.  Run `unicorn -h` to see command-line options.
 
 There is NO WARRANTY whatsoever if anything goes wrong, but
 {let us know}[link:ISSUES.html] and maybe someone can fix it.
+No commercial support will ever be provided by the amateur maintainer.
 
 unicorn is designed to only serve fast clients either on the local host
 or a fast LAN.  See the PHILOSOPHY and DESIGN documents for more details
@@ -132,6 +133,14 @@ damage done to the entire Ruby ecosystem.  Its unintentional popularity
 set Ruby back decades in parallelism, concurrency and robustness since
 it prolongs and proliferates the existence of poorly-written code.
 
+unicorn hackers are NOT responsible for your supply chain security:
+read and understand it yourself or get someone you trust to audit it.
+Malicious commits and releases will be made if under duress.  The only
+defense you'll ever have is from reviewing the source code.
+
+No user or contributor will ever be expected to sacrifice their own
+security by running JavaScript or revealing any personal information.
+
 == Contact
 
 All feedback (bug reports, user/development dicussion, patches, pull
diff --git a/TODO b/TODO
index ebbccdc..a3b18fd 100644
--- a/TODO
+++ b/TODO
@@ -1,3 +1 @@
-* Documentation improvements
-
-* improve test suite
+* improve test suite (port to Perl 5 for stability and maintainability)

[-- Attachment #5: 0004-treewide-future-proof-frozen_string_literal-changes.patch --]
[-- Type: text/x-diff, Size: 24100 bytes --]

From ccf2443901c18ffb26b2785f52d921005e862167 Mon Sep 17 00:00:00 2001
From: Eric Wong <bofh@yhbt.net>
Date: Thu, 8 Feb 2024 12:16:31 +0000
Subject: [PATCH 4/4] treewide: future-proof frozen_string_literal changes

Once again Ruby seems ready to introduce more incompatibilities
and force busywork upon maintainers[1].  In order to avoid
incompatibilities in the future, I used a Perl script[2] to
prepend `frozen_string_literal: false' to every Ruby file.

Somebody interested will have to go through every Ruby source
file and enable frozen_string_literal once they've thoroughly
verified it's safe to do so.

[1] https://bugs.ruby-lang.org/issues/20205
[2] https://yhbt.net/add-fsl.git/74d7689/s/?b=add-fsl.perl
---
 Rakefile                            | 1 +
 bin/unicorn                         | 1 +
 bin/unicorn_rails                   | 1 +
 examples/big_app_gc.rb              | 1 +
 examples/echo.ru                    | 1 +
 examples/logger_mp_safe.rb          | 1 +
 examples/unicorn.conf.minimal.rb    | 1 +
 examples/unicorn.conf.rb            | 1 +
 ext/unicorn_http/extconf.rb         | 1 +
 lib/unicorn.rb                      | 1 +
 lib/unicorn/app/old_rails.rb        | 1 +
 lib/unicorn/app/old_rails/static.rb | 1 +
 lib/unicorn/cgi_wrapper.rb          | 1 +
 lib/unicorn/configurator.rb         | 1 +
 lib/unicorn/const.rb                | 1 +
 lib/unicorn/http_request.rb         | 1 +
 lib/unicorn/http_response.rb        | 1 +
 lib/unicorn/http_server.rb          | 1 +
 lib/unicorn/launcher.rb             | 1 +
 lib/unicorn/oob_gc.rb               | 1 +
 lib/unicorn/preread_input.rb        | 1 +
 lib/unicorn/select_waiter.rb        | 1 +
 lib/unicorn/socket_helper.rb        | 1 +
 lib/unicorn/stream_input.rb         | 1 +
 lib/unicorn/tee_input.rb            | 1 +
 lib/unicorn/tmpio.rb                | 1 +
 lib/unicorn/util.rb                 | 1 +
 lib/unicorn/worker.rb               | 1 +
 setup.rb                            | 1 +
 t/broken-app.ru                     | 1 +
 t/client_body_buffer_size.ru        | 1 +
 t/detach.ru                         | 1 +
 t/env.ru                            | 1 +
 t/fails-rack-lint.ru                | 1 +
 t/heartbeat-timeout.ru              | 1 +
 t/integration.ru                    | 1 +
 t/listener_names.ru                 | 1 +
 t/oob_gc.ru                         | 1 +
 t/oob_gc_path.ru                    | 1 +
 t/pid.ru                            | 1 +
 t/preread_input.ru                  | 1 +
 t/reopen-logs.ru                    | 1 +
 t/t0013.ru                          | 1 +
 t/t0014.ru                          | 1 +
 t/t0301.ru                          | 1 +
 test/aggregate.rb                   | 1 +
 test/benchmark/dd.ru                | 1 +
 test/benchmark/ddstream.ru          | 1 +
 test/benchmark/readinput.ru         | 1 +
 test/benchmark/stack.ru             | 1 +
 test/exec/test_exec.rb              | 1 +
 test/test_helper.rb                 | 1 +
 test/unit/test_ccc.rb               | 1 +
 test/unit/test_configurator.rb      | 1 +
 test/unit/test_droplet.rb           | 1 +
 test/unit/test_http_parser.rb       | 1 +
 test/unit/test_http_parser_ng.rb    | 1 +
 test/unit/test_request.rb           | 1 +
 test/unit/test_server.rb            | 1 +
 test/unit/test_signals.rb           | 1 +
 test/unit/test_socket_helper.rb     | 1 +
 test/unit/test_stream_input.rb      | 1 +
 test/unit/test_tee_input.rb         | 1 +
 test/unit/test_util.rb              | 1 +
 test/unit/test_waiter.rb            | 1 +
 unicorn.gemspec                     | 1 +
 66 files changed, 66 insertions(+)

diff --git a/Rakefile b/Rakefile
index 37569ce..fe1588b 100644
--- a/Rakefile
+++ b/Rakefile
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # optional rake-compiler support in case somebody needs to cross compile
 begin
   mk = "ext/unicorn_http/Makefile"
diff --git a/bin/unicorn b/bin/unicorn
index 00c8464..af8353c 100755
--- a/bin/unicorn
+++ b/bin/unicorn
@@ -1,5 +1,6 @@
 #!/this/will/be/overwritten/or/wrapped/anyways/do/not/worry/ruby
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require 'unicorn/launcher'
 require 'optparse'
 
diff --git a/bin/unicorn_rails b/bin/unicorn_rails
index 354c1df..374fd8e 100755
--- a/bin/unicorn_rails
+++ b/bin/unicorn_rails
@@ -1,5 +1,6 @@
 #!/this/will/be/overwritten/or/wrapped/anyways/do/not/worry/ruby
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require 'unicorn/launcher'
 require 'optparse'
 require 'fileutils'
diff --git a/examples/big_app_gc.rb b/examples/big_app_gc.rb
index c1bae10..0baea26 100644
--- a/examples/big_app_gc.rb
+++ b/examples/big_app_gc.rb
@@ -1,2 +1,3 @@
+# frozen_string_literal: false
 # see {Unicorn::OobGC}[https://yhbt.net/unicorn/Unicorn/OobGC.html]
 # Unicorn::OobGC was broken in Unicorn v3.3.1 - v3.6.1 and fixed in v3.6.2
diff --git a/examples/echo.ru b/examples/echo.ru
index e982180..453a5e6 100644
--- a/examples/echo.ru
+++ b/examples/echo.ru
@@ -1,4 +1,5 @@
 #\-E none
+# frozen_string_literal: false
 #
 # Example application that echoes read data back to the HTTP client.
 # This emulates the old echo protocol people used to run.
diff --git a/examples/logger_mp_safe.rb b/examples/logger_mp_safe.rb
index 05ad3fa..f2c0500 100644
--- a/examples/logger_mp_safe.rb
+++ b/examples/logger_mp_safe.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # Multi-Processing-safe monkey patch for Logger
 #
 # This monkey patch fixes the case where "preload_app true" is used and
diff --git a/examples/unicorn.conf.minimal.rb b/examples/unicorn.conf.minimal.rb
index 46fd634..4f96ede 100644
--- a/examples/unicorn.conf.minimal.rb
+++ b/examples/unicorn.conf.minimal.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # Minimal sample configuration file for Unicorn (not Rack) when used
 # with daemonization (unicorn -D) started in your working directory.
 #
diff --git a/examples/unicorn.conf.rb b/examples/unicorn.conf.rb
index d90bdc4..5bae830 100644
--- a/examples/unicorn.conf.rb
+++ b/examples/unicorn.conf.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # Sample verbose configuration file for Unicorn (not Rack)
 #
 # This configuration file documents many features of Unicorn
diff --git a/ext/unicorn_http/extconf.rb b/ext/unicorn_http/extconf.rb
index 11099cd..de896fe 100644
--- a/ext/unicorn_http/extconf.rb
+++ b/ext/unicorn_http/extconf.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require 'mkmf'
 
 have_func("rb_hash_clear", "ruby.h") or abort 'Ruby 2.0+ required'
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 564cb30..fb91679 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require 'etc'
 require 'stringio'
 require 'raindrops'
diff --git a/lib/unicorn/app/old_rails.rb b/lib/unicorn/app/old_rails.rb
index 1e8c41a..54b3e69 100644
--- a/lib/unicorn/app/old_rails.rb
+++ b/lib/unicorn/app/old_rails.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # :enddoc:
 # This code is based on the original Rails handler in Mongrel
diff --git a/lib/unicorn/app/old_rails/static.rb b/lib/unicorn/app/old_rails/static.rb
index 2257270..cf34e02 100644
--- a/lib/unicorn/app/old_rails/static.rb
+++ b/lib/unicorn/app/old_rails/static.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # :enddoc:
 # This code is based on the original Rails handler in Mongrel
 # Copyright (c) 2005 Zed A. Shaw
diff --git a/lib/unicorn/cgi_wrapper.rb b/lib/unicorn/cgi_wrapper.rb
index d9b7fe5..fb43605 100644
--- a/lib/unicorn/cgi_wrapper.rb
+++ b/lib/unicorn/cgi_wrapper.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # :enddoc:
 # This code is based on the original CGIWrapper from Mongrel
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index b21a01d..3c81596 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require 'logger'
 
 # Implements a simple DSL for configuring a unicorn server.
diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index 33ab4ac..8032863 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 module Unicorn::Const # :nodoc:
   # default TCP listen host address (0.0.0.0, all interfaces)
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index ab3bd6e..a48dab7 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # :enddoc:
 # no stable API here
 require 'unicorn_http'
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index 0ed0ae3..3634165 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # :enddoc:
 # Writes a Rack response to your client using the HTTP/1.1 specification.
 # You use it by simply doing:
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index ed5bbf1..08fbe40 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # This is the process manager of Unicorn. This manages worker
 # processes which in turn handle the I/O and application process.
diff --git a/lib/unicorn/launcher.rb b/lib/unicorn/launcher.rb
index 78e8f39..bd3324e 100644
--- a/lib/unicorn/launcher.rb
+++ b/lib/unicorn/launcher.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # :enddoc:
 $stdout.sync = $stderr.sync = true
diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb
index db9f2cb..efd9177 100644
--- a/lib/unicorn/oob_gc.rb
+++ b/lib/unicorn/oob_gc.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Strongly consider https://github.com/tmm1/gctools if using Ruby 2.1+
 # It is built on new APIs in Ruby 2.1, so it is more intelligent than
diff --git a/lib/unicorn/preread_input.rb b/lib/unicorn/preread_input.rb
index 12eb3e8..c62cc09 100644
--- a/lib/unicorn/preread_input.rb
+++ b/lib/unicorn/preread_input.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 module Unicorn
 # This middleware is used to ensure input is buffered to memory
diff --git a/lib/unicorn/select_waiter.rb b/lib/unicorn/select_waiter.rb
index cb84aab..d11ea57 100644
--- a/lib/unicorn/select_waiter.rb
+++ b/lib/unicorn/select_waiter.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # fallback for non-Linux and Linux <4.5 systems w/o EPOLLEXCLUSIVE
 class Unicorn::SelectWaiter # :nodoc:
   def get_readers(ready, readers, timeout) # :nodoc:
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index 06ec2b2..986932f 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # :enddoc:
 require 'socket'
 
diff --git a/lib/unicorn/stream_input.rb b/lib/unicorn/stream_input.rb
index 9246f73..23a9976 100644
--- a/lib/unicorn/stream_input.rb
+++ b/lib/unicorn/stream_input.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # When processing uploads, unicorn may expose a StreamInput object under
 # "rack.input" of the Rack environment when
diff --git a/lib/unicorn/tee_input.rb b/lib/unicorn/tee_input.rb
index 2ccc2d9..b3c6535 100644
--- a/lib/unicorn/tee_input.rb
+++ b/lib/unicorn/tee_input.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Acts like tee(1) on an input input to provide a input-like stream
 # while providing rewindable semantics through a File/StringIO backing
diff --git a/lib/unicorn/tmpio.rb b/lib/unicorn/tmpio.rb
index 0bbf6ec..deecd80 100644
--- a/lib/unicorn/tmpio.rb
+++ b/lib/unicorn/tmpio.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # :stopdoc:
 require 'tmpdir'
 
diff --git a/lib/unicorn/util.rb b/lib/unicorn/util.rb
index b826de4..f28d929 100644
--- a/lib/unicorn/util.rb
+++ b/lib/unicorn/util.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require 'fcntl'
 module Unicorn::Util # :nodoc:
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 4af31be..d2445d5 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 require "raindrops"
 
 # This class and its members can be considered a stable interface
diff --git a/setup.rb b/setup.rb
index cf1abd9..96cf75a 100644
--- a/setup.rb
+++ b/setup.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 #
 # setup.rb
 #
diff --git a/t/broken-app.ru b/t/broken-app.ru
index d05d7ab..5966bff 100644
--- a/t/broken-app.ru
+++ b/t/broken-app.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # we do not want Rack::Lint or anything to protect us
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
diff --git a/t/client_body_buffer_size.ru b/t/client_body_buffer_size.ru
index 44161a5..1a0fb16 100644
--- a/t/client_body_buffer_size.ru
+++ b/t/client_body_buffer_size.ru
@@ -1,4 +1,5 @@
 #\ -E none
+# frozen_string_literal: false
 app = lambda do |env|
   input = env['rack.input']
   case env["PATH_INFO"]
diff --git a/t/detach.ru b/t/detach.ru
index bbd998e..8d35951 100644
--- a/t/detach.ru
+++ b/t/detach.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentType, "text/plain"
 fifo_path = ENV["TEST_FIFO"] or abort "TEST_FIFO not set"
 run lambda { |env|
diff --git a/t/env.ru b/t/env.ru
index 388412e..86c3cfa 100644
--- a/t/env.ru
+++ b/t/env.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
 run lambda { |env| [ 200, {}, [ env.inspect << "\n" ] ] }
diff --git a/t/fails-rack-lint.ru b/t/fails-rack-lint.ru
index 82bfb5f..8b8b5ec 100644
--- a/t/fails-rack-lint.ru
+++ b/t/fails-rack-lint.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # This rack app returns an invalid status code, which will cause
 # Rack::Lint to throw an exception if it is present.  This
 # is used to check whether Rack::Lint is in the stack or not.
diff --git a/t/heartbeat-timeout.ru b/t/heartbeat-timeout.ru
index 3eeb5d6..ccc6a8e 100644
--- a/t/heartbeat-timeout.ru
+++ b/t/heartbeat-timeout.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentLength
 headers = { 'content-type' => 'text/plain' }
 run lambda { |env|
diff --git a/t/integration.ru b/t/integration.ru
index 888833a..6df481c 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -1,4 +1,5 @@
 #!ruby
+# frozen_string_literal: false
 # Copyright (C) unicorn hackers <unicorn-public@80x24.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 
diff --git a/t/listener_names.ru b/t/listener_names.ru
index edb4e6a..f52c59b 100644
--- a/t/listener_names.ru
+++ b/t/listener_names.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
 names = Unicorn.listener_names.inspect # rely on preload_app=true
diff --git a/t/oob_gc.ru b/t/oob_gc.ru
index 224cb06..2ae58a8 100644
--- a/t/oob_gc.ru
+++ b/t/oob_gc.ru
@@ -1,4 +1,5 @@
 #\-E none
+# frozen_string_literal: false
 require 'unicorn/oob_gc'
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
diff --git a/t/oob_gc_path.ru b/t/oob_gc_path.ru
index 7f40601..5358222 100644
--- a/t/oob_gc_path.ru
+++ b/t/oob_gc_path.ru
@@ -1,4 +1,5 @@
 #\-E none
+# frozen_string_literal: false
 require 'unicorn/oob_gc'
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
diff --git a/t/pid.ru b/t/pid.ru
index f5fd31f..b49b137 100644
--- a/t/pid.ru
+++ b/t/pid.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
 run lambda { |env| [ 200, {}, [ "#$$\n" ] ] }
diff --git a/t/preread_input.ru b/t/preread_input.ru
index 18af221..5f68fe9 100644
--- a/t/preread_input.ru
+++ b/t/preread_input.ru
@@ -1,4 +1,5 @@
 #\-E none
+# frozen_string_literal: false
 require 'digest/md5'
 require 'unicorn/preread_input'
 use Unicorn::PrereadInput
diff --git a/t/reopen-logs.ru b/t/reopen-logs.ru
index c39e8f6..488da85 100644
--- a/t/reopen-logs.ru
+++ b/t/reopen-logs.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, "text/plain"
 run lambda { |env|
diff --git a/t/t0013.ru b/t/t0013.ru
index 48a3a34..e425093 100644
--- a/t/t0013.ru
+++ b/t/t0013.ru
@@ -1,4 +1,5 @@
 #\ -E none
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, 'text/plain'
 app = lambda do |env|
diff --git a/t/t0014.ru b/t/t0014.ru
index b0bd2b7..686d214 100644
--- a/t/t0014.ru
+++ b/t/t0014.ru
@@ -1,4 +1,5 @@
 #\ -E none
+# frozen_string_literal: false
 use Rack::ContentLength
 use Rack::ContentType, 'text/plain'
 app = lambda do |env|
diff --git a/t/t0301.ru b/t/t0301.ru
index ce68213..54929b1 100644
--- a/t/t0301.ru
+++ b/t/t0301.ru
@@ -1,4 +1,5 @@
 #\-N --debug
+# frozen_string_literal: false
 run(lambda do |env|
   case env['PATH_INFO']
   when '/vars'
diff --git a/test/aggregate.rb b/test/aggregate.rb
index 5eebbe5..0f32b2f 100755
--- a/test/aggregate.rb
+++ b/test/aggregate.rb
@@ -1,5 +1,6 @@
 #!/usr/bin/ruby -n
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 BEGIN { $tests = $assertions = $failures = $errors = 0 }
 
diff --git a/test/benchmark/dd.ru b/test/benchmark/dd.ru
index 111fa2e..5bd2739 100644
--- a/test/benchmark/dd.ru
+++ b/test/benchmark/dd.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # This benchmark is the simplest test of the I/O facilities in
 # unicorn.  It is meant to return a fixed-sized blob to test
 # the performance of things in Unicorn, _NOT_ the app.
diff --git a/test/benchmark/ddstream.ru b/test/benchmark/ddstream.ru
index b14c973..fd40ced 100644
--- a/test/benchmark/ddstream.ru
+++ b/test/benchmark/ddstream.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # This app is intended to test large HTTP responses with or without
 # a fully-buffering reverse proxy such as nginx. Without a fully-buffering
 # reverse proxy, unicorn will be unresponsive when client count exceeds
diff --git a/test/benchmark/readinput.ru b/test/benchmark/readinput.ru
index c91bec3..95c0226 100644
--- a/test/benchmark/readinput.ru
+++ b/test/benchmark/readinput.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 # This app is intended to test large HTTP requests with or without
 # a fully-buffering reverse proxy such as nginx. Without a fully-buffering
 # reverse proxy, unicorn will be unresponsive when client count exceeds
diff --git a/test/benchmark/stack.ru b/test/benchmark/stack.ru
index fc9193f..17a565b 100644
--- a/test/benchmark/stack.ru
+++ b/test/benchmark/stack.ru
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 run(lambda { |env|
   body = "#{caller.size}\n"
   h = {
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index 8494452..807f724 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 # Don't add to this file, new tests are in Perl 5. See t/README
 FLOCK_PATH = File.expand_path(__FILE__)
 require './test/test_helper'
diff --git a/test/test_helper.rb b/test/test_helper.rb
index d86f83b..0bf3c90 100644
--- a/test/test_helper.rb
+++ b/test/test_helper.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
diff --git a/test/unit/test_ccc.rb b/test/unit/test_ccc.rb
index f518230..a0a2bff 100644
--- a/test/unit/test_ccc.rb
+++ b/test/unit/test_ccc.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 require 'socket'
 require 'unicorn'
 require 'io/wait'
diff --git a/test/unit/test_configurator.rb b/test/unit/test_configurator.rb
index 1298f0e..1a89aca 100644
--- a/test/unit/test_configurator.rb
+++ b/test/unit/test_configurator.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require 'test/unit'
 require 'tempfile'
diff --git a/test/unit/test_droplet.rb b/test/unit/test_droplet.rb
index 81ad82b..4b2d2d0 100644
--- a/test/unit/test_droplet.rb
+++ b/test/unit/test_droplet.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 require 'test/unit'
 require 'unicorn'
 
diff --git a/test/unit/test_http_parser.rb b/test/unit/test_http_parser.rb
index 697af44..adcc84f 100644
--- a/test/unit/test_http_parser.rb
+++ b/test/unit/test_http_parser.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
diff --git a/test/unit/test_http_parser_ng.rb b/test/unit/test_http_parser_ng.rb
index 425d5ad..fd47246 100644
--- a/test/unit/test_http_parser_ng.rb
+++ b/test/unit/test_http_parser_ng.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require './test/test_helper'
 require 'digest/md5'
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index 53ae944..9d1b350 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Copyright (c) 2009 Eric Wong
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index 7ffa48f..5a2252f 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Copyright (c) 2005 Zed A. Shaw
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
diff --git a/test/unit/test_signals.rb b/test/unit/test_signals.rb
index 6c48754..49ff3c7 100644
--- a/test/unit/test_signals.rb
+++ b/test/unit/test_signals.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 # Copyright (c) 2009 Eric Wong
 # You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb
index a446f06..4363474 100644
--- a/test/unit/test_socket_helper.rb
+++ b/test/unit/test_socket_helper.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require './test/test_helper'
 require 'tempfile'
diff --git a/test/unit/test_stream_input.rb b/test/unit/test_stream_input.rb
index 7986ca7..7ee98e4 100644
--- a/test/unit/test_stream_input.rb
+++ b/test/unit/test_stream_input.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require 'test/unit'
 require 'digest/sha1'
diff --git a/test/unit/test_tee_input.rb b/test/unit/test_tee_input.rb
index 607ce87..8f05c77 100644
--- a/test/unit/test_tee_input.rb
+++ b/test/unit/test_tee_input.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require 'test/unit'
 require 'digest/sha1'
diff --git a/test/unit/test_util.rb b/test/unit/test_util.rb
index bc7b233..ce53b86 100644
--- a/test/unit/test_util.rb
+++ b/test/unit/test_util.rb
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 
 require './test/test_helper'
 require 'tempfile'
diff --git a/test/unit/test_waiter.rb b/test/unit/test_waiter.rb
index 0995de2..a20994b 100644
--- a/test/unit/test_waiter.rb
+++ b/test/unit/test_waiter.rb
@@ -1,3 +1,4 @@
+# frozen_string_literal: false
 require 'test/unit'
 require 'unicorn'
 require 'unicorn/select_waiter'
diff --git a/unicorn.gemspec b/unicorn.gemspec
index e7e3ef7..36700a8 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -1,4 +1,5 @@
 # -*- encoding: binary -*-
+# frozen_string_literal: false
 manifest = File.exist?('.manifest') ?
   IO.readlines('.manifest').map!(&:chomp!) : `git ls-files`.split("\n")
 

^ permalink raw reply related	[relevance 4%]

* [PATCH 00..11/11] more tests to Perl 5..
@ 2023-09-10 20:08  1% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2023-09-10 20:08 UTC (permalink / raw)
  To: unicorn-public

[-- Attachment #1: Type: text/plain, Size: 3020 bytes --]

Hopefully this is less maintenance down the line since Ruby
introduces incompatibilities at a higher rate than Perl.
I don't fully trust Perl, either, but far more Ruby code gets
broken by new releases.

More to come at some point...

Note: attached patches are generated with --irreversible-delete
to save bandwidth.

Eric Wong (11):
  tests: port some bad config tests to Perl 5
  tests: port working_directory tests to Perl 5
  tests: port t/heartbeat-timeout to Perl 5
  tests: port reopen logs test over to Perl 5
  tests: rewrite SIGWINCH && SIGTTIN test in Perl 5
  tests: introduce `do_req' helper sub
  tests: use more common variable names between tests
  tests: use Time::HiRes `sleep' and `time' everywhere
  tests: fold SO_KEEPALIVE check to Perl 5 integration
  tests: move broken app test to Perl 5 integration test
  tests: fold early shutdown() tests into t/integration.t

 t/active-unix-socket.t                    |  4 +-
 t/client_body_buffer_size.t               |  6 +-
 t/heartbeat-timeout.ru                    |  2 +-
 t/heartbeat-timeout.t                     | 62 +++++++++++++++
 t/integration.ru                          |  1 +
 t/integration.t                           | 82 +++++++++++++-------
 t/lib.perl                                | 51 ++++++++++--
 t/reload-bad-config.t                     | 54 +++++++++++++
 t/{t0006.ru => reopen-logs.ru}            |  0
 t/reopen-logs.t                           | 39 ++++++++++
 t/t0001-reload-bad-config.sh              | 53 -------------
 t/t0002-config-conflict.sh                | 49 ------------
 t/t0003-working_directory.sh              | 51 ------------
 t/t0004-heartbeat-timeout.sh              | 69 -----------------
 t/t0004-working_directory_broken.sh       | 24 ------
 t/t0005-working_directory_app.rb.sh       | 40 ----------
 t/t0006-reopen-logs.sh                    | 83 --------------------
 t/t0007-working_directory_no_embed_cli.sh | 44 -----------
 t/t0009-winch_ttin.sh                     | 59 --------------
 t/winch_ttin.t                            | 67 ++++++++++++++++
 t/working_directory.t                     | 94 +++++++++++++++++++++++
 test/exec/test_exec.rb                    | 23 +-----
 test/unit/test_server.rb                  | 67 ----------------
 23 files changed, 424 insertions(+), 600 deletions(-)
 create mode 100644 t/heartbeat-timeout.t
 create mode 100644 t/reload-bad-config.t
 rename t/{t0006.ru => reopen-logs.ru} (100%)
 create mode 100644 t/reopen-logs.t
 delete mode 100755 t/t0001-reload-bad-config.sh
 delete mode 100755 t/t0002-config-conflict.sh
 delete mode 100755 t/t0003-working_directory.sh
 delete mode 100755 t/t0004-heartbeat-timeout.sh
 delete mode 100755 t/t0004-working_directory_broken.sh
 delete mode 100755 t/t0005-working_directory_app.rb.sh
 delete mode 100755 t/t0006-reopen-logs.sh
 delete mode 100755 t/t0007-working_directory_no_embed_cli.sh
 delete mode 100755 t/t0009-winch_ttin.sh
 create mode 100644 t/winch_ttin.t
 create mode 100644 t/working_directory.t

[-- Attachment #2: 0001-tests-port-some-bad-config-tests-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 3987 bytes --]

From f43c28ea10ca8d520b55f2fbb20710dd66fc4fb5 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Thu, 7 Sep 2023 22:55:09 +0000
Subject: [PATCH 01/11] tests: port some bad config tests to Perl 5

We can fold some tests into one test to save on Perl startup
time (but Ruby startup time is a lost cause).
---
 t/lib.perl                   | 12 ++++----
 t/reload-bad-config.t        | 58 ++++++++++++++++++++++++++++++++++++
 t/t0001-reload-bad-config.sh | 53 --------------------------------
 t/t0002-config-conflict.sh   | 49 ------------------------------
 4 files changed, 65 insertions(+), 107 deletions(-)
 create mode 100644 t/reload-bad-config.t
 delete mode 100755 t/t0001-reload-bad-config.sh
 delete mode 100755 t/t0002-config-conflict.sh

diff --git a/t/lib.perl b/t/lib.perl
index fe3404ba..7de9e426 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -9,17 +9,19 @@ use Test::More;
 use IO::Socket::INET;
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
-our ($tmpdir, $errfh);
-our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn $tmpdir $errfh
+our ($tmpdir, $errfh, $err_log);
+our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn
+	$tmpdir $errfh $err_log
 	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
-open($errfh, '>>', "$tmpdir/err.log");
-END { diag slurp("$tmpdir/err.log") if $tmpdir };
+$err_log = "$tmpdir/err.log";
+open($errfh, '>>', $err_log);
+END { diag slurp($err_log) if $tmpdir };
 
 sub check_stderr () {
-	my @log = slurp("$tmpdir/err.log");
+	my @log = slurp($err_log);
 	diag("@log") if $ENV{V};
 	my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
 	@err = grep(!/failed to set accept_filter=/, @err);
diff --git a/t/reload-bad-config.t b/t/reload-bad-config.t
new file mode 100644
index 00000000..c7055c7e
--- /dev/null
+++ b/t/reload-bad-config.t
@@ -0,0 +1,58 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+my $srv = tcp_server();
+my $host_port = tcp_host_port($srv);
+my $ru = "$tmpdir/config.ru";
+my $u_conf = "$tmpdir/u.conf.rb";
+
+open my $fh, '>', $ru;
+print $fh <<'EOM';
+use Rack::ContentLength
+use Rack::ContentType, 'text/plain'
+config = ru = "hello world\n" # check for config variable conflicts, too
+run lambda { |env| [ 200, {}, [ ru.to_s ] ] }
+EOM
+close $fh;
+
+open $fh, '>', $u_conf;
+print $fh <<EOM;
+preload_app true
+stderr_path "$err_log"
+EOM
+close $fh;
+
+my $ar = unicorn(qw(-E none -c), $u_conf, $ru, { 3 => $srv });
+my $c = tcp_start($srv, 'GET / HTTP/1.0');
+my ($status, $hdr) = slurp_hdr($c);
+my $bdy = do { local $/; <$c> };
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid at start');
+is($bdy, "hello world\n", 'body matches expected');
+
+open $fh, '>>', $ru;
+say $fh '....this better be a syntax error in any version of ruby...';
+close $fh;
+
+$ar->do_kill('HUP'); # reload
+my @l;
+for (1..1000) {
+	@l = grep(/(?:done|error) reloading/, slurp($err_log)) and
+		last;
+	select undef, undef, undef, 0.011;
+}
+diag slurp($err_log) if $ENV{V};
+ok(grep(/error reloading/, @l), 'got error reloading');
+open $fh, '>', $err_log;
+close $fh;
+
+$c = tcp_start($srv, 'GET / HTTP/1.0');
+($status, $hdr) = slurp_hdr($c);
+$bdy = do { local $/; <$c> };
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid afte reload');
+is($bdy, "hello world\n", 'body matches expected after reload');
+
+check_stderr;
+undef $tmpdir; # quiet t/lib.perl END{}
+done_testing;
diff --git a/t/t0001-reload-bad-config.sh b/t/t0001-reload-bad-config.sh
deleted file mode 100755
index 55bb3555..00000000
diff --git a/t/t0002-config-conflict.sh b/t/t0002-config-conflict.sh
deleted file mode 100755
index d7b2181a..00000000

[-- Attachment #3: 0002-tests-port-working_directory-tests-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 4809 bytes --]

From d4514174ee7eadea89003f380acacf32d52acd9d Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Thu, 7 Sep 2023 23:18:16 +0000
Subject: [PATCH 02/11] tests: port working_directory tests to Perl 5

We can fold a bunch of them into one test to save startup
time, inodes, and FS activity.
---
 t/t0003-working_directory.sh              |  51 ---------
 t/t0004-working_directory_broken.sh       |  24 -----
 t/t0005-working_directory_app.rb.sh       |  40 -------
 t/t0007-working_directory_no_embed_cli.sh |  44 --------
 t/working_directory.t                     | 122 ++++++++++++++++++++++
 5 files changed, 122 insertions(+), 159 deletions(-)
 delete mode 100755 t/t0003-working_directory.sh
 delete mode 100755 t/t0004-working_directory_broken.sh
 delete mode 100755 t/t0005-working_directory_app.rb.sh
 delete mode 100755 t/t0007-working_directory_no_embed_cli.sh
 create mode 100644 t/working_directory.t

diff --git a/t/t0003-working_directory.sh b/t/t0003-working_directory.sh
deleted file mode 100755
index 79988d8b..00000000
diff --git a/t/t0004-working_directory_broken.sh b/t/t0004-working_directory_broken.sh
deleted file mode 100755
index ca9d3825..00000000
diff --git a/t/t0005-working_directory_app.rb.sh b/t/t0005-working_directory_app.rb.sh
deleted file mode 100755
index 0fbab4fc..00000000
diff --git a/t/t0007-working_directory_no_embed_cli.sh b/t/t0007-working_directory_no_embed_cli.sh
deleted file mode 100755
index 77d67072..00000000
diff --git a/t/working_directory.t b/t/working_directory.t
new file mode 100644
index 00000000..e7ff43a5
--- /dev/null
+++ b/t/working_directory.t
@@ -0,0 +1,122 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+mkdir "$tmpdir/alt";
+my $u_sock = "$tmpdir/u.sock";
+my $ru = "$tmpdir/alt/config.ru";
+my $u_conf = "$tmpdir/u.conf.rb";
+open my $fh, '>', $u_conf;
+print $fh <<EOM;
+pid "$tmpdir/pid"
+preload_app true
+stderr_path "$err_log"
+working_directory "$tmpdir/alt" # the whole point of this test
+before_fork { |_,_| \$master_ppid = Process.ppid }
+EOM
+close $fh;
+
+my $common_ru = <<'EOM';
+use Rack::ContentLength
+use Rack::ContentType, 'text/plain'
+run lambda { |env| [ 200, {}, [ "#{$master_ppid}\n" ] ] }
+EOM
+
+open $fh, '>', $ru;
+print $fh <<EOM;
+#\\--daemonize --listen $u_sock
+$common_ru
+EOM
+close $fh;
+
+my $pid;
+my $stop_daemon = sub {
+	my ($is_END) = @_;
+	kill('TERM', $pid);
+	my $tries = 1000;
+	while (CORE::kill(0, $pid) && --$tries) {
+		select undef, undef, undef, 0.01;
+	}
+	if ($is_END && CORE::kill(0, $pid)) {
+		CORE::kill('KILL', $pid);
+		die "daemonized PID=$pid did not die";
+	} else {
+		ok(!CORE::kill(0, $pid), 'daemonized unicorn gone');
+		undef $pid;
+	}
+};
+
+END { $stop_daemon->(1) if defined $pid };
+
+unicorn('-c', $u_conf)->join; # will daemonize
+chomp($pid = slurp("$tmpdir/pid"));
+
+my $c = unix_start($u_sock, 'GET / HTTP/1.0');
+my ($status, $hdr) = slurp_hdr($c);
+chomp(my $bdy = do { local $/; <$c> });
+is($bdy, 1, 'got expected $master_ppid');
+
+$stop_daemon->();
+check_stderr;
+
+if ('test without CLI switches in config.ru') {
+	truncate $err_log, 0;
+	open $fh, '>', $ru;
+	print $fh $common_ru;
+	close $fh;
+
+	unicorn('-D', '-l', $u_sock, '-c', $u_conf)->join; # will daemonize
+	chomp($pid = slurp("$tmpdir/pid"));
+
+	$c = unix_start($u_sock, 'GET / HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	chomp($bdy = do { local $/; <$c> });
+	is($bdy, 1, 'got expected $master_ppid');
+
+	$stop_daemon->();
+	check_stderr;
+}
+
+if ('ensures broken working_directory (missing config.ru) is OK') {
+	truncate $err_log, 0;
+	unlink $ru;
+
+	my $auto_reap = unicorn('-c', $u_conf);
+	$auto_reap->join;
+	isnt($?, 0, 'exited with error due to missing config.ru');
+
+	like(slurp($err_log), qr/rackup file \Q(config.ru)\E not readable/,
+		'noted unreadability of config.ru in stderr');
+}
+
+if ('fooapp.rb (not config.ru) works with working_directory') {
+	truncate $err_log, 0;
+	my $fooapp = "$tmpdir/alt/fooapp.rb";
+	open $fh, '>', $fooapp;
+	print $fh <<EOM;
+class Fooapp
+  def self.call(env)
+    b = "dir=#{Dir.pwd}"
+    h = { 'content-type' => 'text/plain', 'content-length' => b.bytesize.to_s }
+    [ 200, h, [ b ] ]
+  end
+end
+EOM
+	close $fh;
+	my $srv = tcp_server;
+	my $auto_reap = unicorn(qw(-c), $u_conf, qw(-I. fooapp.rb),
+				{ -C => '/', 3 => $srv });
+	$c = tcp_start($srv, 'GET / HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	chomp($bdy = do { local $/; <$c> });
+	is($bdy, "dir=$tmpdir/alt",
+		'fooapp.rb (w/o config.ru) w/ working_directory');
+	close $c;
+	$auto_reap->join('TERM');
+	is($?, 0, 'fooapp.rb process exited');
+	check_stderr;
+}
+
+undef $tmpdir;
+done_testing;

[-- Attachment #4: 0003-tests-port-t-heartbeat-timeout-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 3478 bytes --]

From d67284a692683bca59effd9c0670bd5dd47e4fa3 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Thu, 7 Sep 2023 23:53:58 +0000
Subject: [PATCH 03/11] tests: port t/heartbeat-timeout to Perl 5

I absolutely detest and regret adding this feature,
but I'm hell bent on supporting it until the end of days
because we don't break compatibility.
---
 t/heartbeat-timeout.ru       |  2 +-
 t/heartbeat-timeout.t        | 69 ++++++++++++++++++++++++++++++++++++
 t/t0004-heartbeat-timeout.sh | 69 ------------------------------------
 3 files changed, 70 insertions(+), 70 deletions(-)
 create mode 100644 t/heartbeat-timeout.t
 delete mode 100755 t/t0004-heartbeat-timeout.sh

diff --git a/t/heartbeat-timeout.ru b/t/heartbeat-timeout.ru
index 20a79380..3eeb5d64 100644
--- a/t/heartbeat-timeout.ru
+++ b/t/heartbeat-timeout.ru
@@ -7,6 +7,6 @@
     sleep # in case STOP signal is not received in time
     [ 500, headers, [ "Should never get here\n" ] ]
   else
-    [ 200, headers, [ "#$$\n" ] ]
+    [ 200, headers, [ "#$$" ] ]
   end
 }
diff --git a/t/heartbeat-timeout.t b/t/heartbeat-timeout.t
new file mode 100644
index 00000000..1fcf21a2
--- /dev/null
+++ b/t/heartbeat-timeout.t
@@ -0,0 +1,69 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
+mkdir "$tmpdir/alt";
+my $srv = tcp_server();
+my $u_conf = "$tmpdir/u.conf.rb";
+open my $fh, '>', $u_conf;
+print $fh <<EOM;
+pid "$tmpdir/pid"
+preload_app true
+stderr_path "$err_log"
+timeout 3 # WORST FEATURE EVER
+EOM
+close $fh;
+
+my $ar = unicorn(qw(-E none t/heartbeat-timeout.ru -c), $u_conf, { 3 => $srv });
+
+my $c = tcp_start($srv, 'GET /pid HTTP/1.0');
+my ($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'PID request succeeds');
+my $wpid = do { local $/; <$c> };
+like($wpid, qr/\A[0-9]+\z/, 'worker is running');
+
+my $t0 = clock_gettime(CLOCK_MONOTONIC);
+$c = tcp_start($srv, 'GET /block-forever HTTP/1.0');
+vec(my $rvec = '', fileno($c), 1) = 1;
+is(select($rvec, undef, undef, 6), 1, 'got readiness');
+$c->blocking(0);
+is(sysread($c, my $buf, 128), 0, 'got EOF response');
+my $elapsed = clock_gettime(CLOCK_MONOTONIC) - $t0;
+ok($elapsed > 3, 'timeout took >3s');
+
+my @timeout_err = slurp($err_log);
+truncate($err_log, 0);
+is(grep(/timeout \(\d+s > 3s\), killing/, @timeout_err), 1,
+    'noted timeout error') or diag explain(\@timeout_err);
+
+# did it respawn?
+$c = tcp_start($srv, 'GET /pid HTTP/1.0');
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'PID request succeeds');
+my $new_pid = do { local $/; <$c> };
+isnt($new_pid, $wpid, 'spawned new worker');
+
+diag 'SIGSTOP for 4 seconds...';
+$ar->do_kill('STOP');
+sleep 4;
+$ar->do_kill('CONT');
+for my $i (1..2) {
+	$c = tcp_start($srv, 'GET /pid HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 200\b!,
+		"PID request succeeds #$i after STOP+CONT");
+	my $spid = do { local $/; <$c> };
+	is($new_pid, $spid, "worker pid unchanged after STOP+CONT #$i");
+	if ($i == 1) {
+		diag 'sleeping 2s to ensure timeout is not delayed';
+		sleep 2;
+	}
+}
+
+$ar->join('TERM');
+check_stderr;
+undef $tmpdir;
+
+done_testing;
diff --git a/t/t0004-heartbeat-timeout.sh b/t/t0004-heartbeat-timeout.sh
deleted file mode 100755
index 29652837..00000000

[-- Attachment #5: 0004-tests-port-reopen-logs-test-over-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 2317 bytes --]

From 1607ac966f604ec4cf383025c4c3ee296f638fff Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 07:13:11 +0000
Subject: [PATCH 04/11] tests: port reopen logs test over to Perl 5

Being able to do subsecond sleeps is one welcome advantage
over POSIX (not GNU) sleep(1) in portable Bourne sh.
---
 t/{t0006.ru => reopen-logs.ru} |  0
 t/reopen-logs.t                | 43 ++++++++++++++++++
 t/t0006-reopen-logs.sh         | 83 ----------------------------------
 3 files changed, 43 insertions(+), 83 deletions(-)
 rename t/{t0006.ru => reopen-logs.ru} (100%)
 create mode 100644 t/reopen-logs.t
 delete mode 100755 t/t0006-reopen-logs.sh

diff --git a/t/t0006.ru b/t/reopen-logs.ru
similarity index 100%
rename from t/t0006.ru
rename to t/reopen-logs.ru
diff --git a/t/reopen-logs.t b/t/reopen-logs.t
new file mode 100644
index 00000000..e1bf524c
--- /dev/null
+++ b/t/reopen-logs.t
@@ -0,0 +1,43 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+my $srv = tcp_server();
+my $u_conf = "$tmpdir/u.conf.rb";
+my $out_log = "$tmpdir/out.log";
+open my $fh, '>', $u_conf;
+print $fh <<EOM;
+stderr_path "$err_log"
+stdout_path "$out_log"
+EOM
+close $fh;
+
+my $auto_reap = unicorn('-c', $u_conf, 't/reopen-logs.ru', { 3 => $srv } );
+my $c = tcp_start($srv, 'GET / HTTP/1.0');
+my ($status, $hdr) = slurp_hdr($c);
+my $bdy = do { local $/; <$c> };
+is($bdy, "true\n", 'logs opened');
+
+rename($err_log, "$err_log.rot");
+rename($out_log, "$out_log.rot");
+
+$auto_reap->do_kill('USR1');
+
+my $tries = 1000;
+while (!-f $err_log && --$tries) { select undef, undef, undef, 0.01 };
+while (!-f $out_log && --$tries) { select undef, undef, undef, 0.01 };
+
+ok(-f $out_log, 'stdout_path recreated after USR1');
+ok(-f $err_log, 'stderr_path recreated after USR1');
+
+$c = tcp_start($srv, 'GET / HTTP/1.0');
+($status, $hdr) = slurp_hdr($c);
+$bdy = do { local $/; <$c> };
+is($bdy, "true\n", 'logs reopened with sync==true');
+
+$auto_reap->join('QUIT');
+is($?, 0, 'no error on exit');
+check_stderr;
+undef $tmpdir;
+done_testing;
diff --git a/t/t0006-reopen-logs.sh b/t/t0006-reopen-logs.sh
deleted file mode 100755
index a6e7a17c..00000000

[-- Attachment #6: 0005-tests-rewrite-SIGWINCH-SIGTTIN-test-in-Perl-5.patch --]
[-- Type: text/x-diff, Size: 2916 bytes --]

From 86aea575c331a3b5242db1c14a848928a37ff9e3 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 08:27:04 +0000
Subject: [PATCH 05/11] tests: rewrite SIGWINCH && SIGTTIN test in Perl 5

No need to deal with full second sleeps, here.
---
 t/t0009-winch_ttin.sh | 59 -----------------------------------
 t/winch_ttin.t        | 72 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 59 deletions(-)
 delete mode 100755 t/t0009-winch_ttin.sh
 create mode 100644 t/winch_ttin.t

diff --git a/t/t0009-winch_ttin.sh b/t/t0009-winch_ttin.sh
deleted file mode 100755
index 6e56e30c..00000000
diff --git a/t/winch_ttin.t b/t/winch_ttin.t
new file mode 100644
index 00000000..1a198778
--- /dev/null
+++ b/t/winch_ttin.t
@@ -0,0 +1,72 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+use POSIX qw(mkfifo);
+my $u_conf = "$tmpdir/u.conf.rb";
+my $u_sock = "$tmpdir/u.sock";
+my $fifo = "$tmpdir/fifo";
+mkfifo($fifo, 0666) or die "mkfifo($fifo): $!";
+
+open my $fh, '>', $u_conf;
+print $fh <<EOM;
+pid "$tmpdir/pid"
+listen "$u_sock"
+stderr_path "$err_log"
+after_fork do |server, worker|
+  # test script will block while reading from $fifo,
+  File.open("$fifo", "wb") { |fp| fp.syswrite worker.nr.to_s }
+end
+EOM
+close $fh;
+
+unicorn('-D', '-c', $u_conf, 't/integration.ru')->join;
+is($?, 0, 'daemonized properly');
+open $fh, '<', "$tmpdir/pid";
+chomp(my $pid = <$fh>);
+ok(kill(0, $pid), 'daemonized PID works');
+my $quit = sub { kill('QUIT', $pid) if $pid; $pid = undef };
+END { $quit->() };
+
+open $fh, '<', $fifo;
+my $worker_nr = <$fh>;
+close $fh;
+is($worker_nr, '0', 'initial worker spawned');
+
+my $c = unix_start($u_sock, 'GET /pid HTTP/1.0');
+my ($status, $hdr) = slurp_hdr($c);
+like($status, qr/ 200\b/, 'got 200 response');
+my $worker_pid = do { local $/; <$c> };
+like($worker_pid, qr/\A[0-9]+\n\z/s, 'PID in response');
+chomp $worker_pid;
+ok(kill(0, $worker_pid), 'worker_pid is valid');
+
+ok(kill('WINCH', $pid), 'SIGWINCH can be sent');
+
+my $tries = 1000;
+while (CORE::kill(0, $worker_pid) && --$tries) {
+	select undef, undef, undef, 0.01;
+}
+ok(!CORE::kill(0, $worker_pid), 'worker not running');
+
+ok(kill('TTIN', $pid), 'SIGTTIN to restart worker');
+
+open $fh, '<', $fifo;
+$worker_nr = <$fh>;
+close $fh;
+is($worker_nr, '0', 'worker restarted');
+
+$c = unix_start($u_sock, 'GET /pid HTTP/1.0');
+($status, $hdr) = slurp_hdr($c);
+like($status, qr/ 200\b/, 'got 200 response');
+chomp(my $new_worker_pid = do { local $/; <$c> });
+like($new_worker_pid, qr/\A[0-9]+\z/, 'got new worker PID');
+ok(kill(0, $new_worker_pid), 'got a valid worker PID');
+isnt($worker_pid, $new_worker_pid, 'worker PID changed');
+
+$quit->();
+
+check_stderr;
+undef $tmpdir;
+done_testing;

[-- Attachment #7: 0006-tests-introduce-do_req-helper-sub.patch --]
[-- Type: text/x-diff, Size: 11556 bytes --]

From 29885f0d95aaa8e1d1f6cf3b791d9f08338a511e Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 09:15:16 +0000
Subject: [PATCH 06/11] tests: introduce `do_req' helper sub

While early tests required fine-grained control in trickling
requests, many of our later tests can use a short one-liner
w/o having to spawn curl.
---
 t/heartbeat-timeout.t | 12 +++---------
 t/integration.t       | 33 +++++++++++++--------------------
 t/lib.perl            | 16 +++++++++++++++-
 t/reload-bad-config.t |  8 ++------
 t/reopen-logs.t       |  8 ++------
 t/winch_ttin.t        | 11 ++++-------
 t/working_directory.t | 17 +++++------------
 7 files changed, 44 insertions(+), 61 deletions(-)

diff --git a/t/heartbeat-timeout.t b/t/heartbeat-timeout.t
index 1fcf21a2..ce1f7e16 100644
--- a/t/heartbeat-timeout.t
+++ b/t/heartbeat-timeout.t
@@ -18,10 +18,8 @@ close $fh;
 
 my $ar = unicorn(qw(-E none t/heartbeat-timeout.ru -c), $u_conf, { 3 => $srv });
 
-my $c = tcp_start($srv, 'GET /pid HTTP/1.0');
-my ($status, $hdr) = slurp_hdr($c);
+my ($status, $hdr, $wpid) = do_req($srv, 'GET /pid HTTP/1.0');
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'PID request succeeds');
-my $wpid = do { local $/; <$c> };
 like($wpid, qr/\A[0-9]+\z/, 'worker is running');
 
 my $t0 = clock_gettime(CLOCK_MONOTONIC);
@@ -39,10 +37,8 @@ is(grep(/timeout \(\d+s > 3s\), killing/, @timeout_err), 1,
     'noted timeout error') or diag explain(\@timeout_err);
 
 # did it respawn?
-$c = tcp_start($srv, 'GET /pid HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
+($status, $hdr, my $new_pid) = do_req($srv, 'GET /pid HTTP/1.0');
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'PID request succeeds');
-my $new_pid = do { local $/; <$c> };
 isnt($new_pid, $wpid, 'spawned new worker');
 
 diag 'SIGSTOP for 4 seconds...';
@@ -50,11 +46,9 @@ $ar->do_kill('STOP');
 sleep 4;
 $ar->do_kill('CONT');
 for my $i (1..2) {
-	$c = tcp_start($srv, 'GET /pid HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr, my $spid) = do_req($srv, 'GET /pid HTTP/1.0');
 	like($status, qr!\AHTTP/1\.[01] 200\b!,
 		"PID request succeeds #$i after STOP+CONT");
-	my $spid = do { local $/; <$c> };
 	is($new_pid, $spid, "worker pid unchanged after STOP+CONT #$i");
 	if ($i == 1) {
 		diag 'sleeping 2s to ensure timeout is not delayed';
diff --git a/t/integration.t b/t/integration.t
index bb2ab51b..13b07467 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -62,11 +62,10 @@ EOM
 	},
 );
 
-my ($c, $status, $hdr);
+my ($c, $status, $hdr, $bdy);
 
 # response header tests
-$c = tcp_start($srv, 'GET /rack-2-newline-headers HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
+($status, $hdr) = do_req($srv, 'GET /rack-2-newline-headers HTTP/1.0');
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
 my $orig_200_status = $status;
 is_deeply([ grep(/^X-R2: /, @$hdr) ],
@@ -84,16 +83,16 @@ SKIP: { # Date header check
 };
 
 
-$c = tcp_start($srv, 'GET /rack-3-array-headers HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
+($status, $hdr) = do_req($srv, 'GET /rack-3-array-headers HTTP/1.0');
 is_deeply([ grep(/^x-r3: /, @$hdr) ],
 	[ 'x-r3: a', 'x-r3: b', 'x-r3: c' ],
 	'rack 3 array headers supported') or diag(explain($hdr));
 
 SKIP: {
 	eval { require JSON::PP } or skip "JSON::PP missing: $@", 1;
-	my $c = tcp_start($srv, 'GET /env_dump');
-	my $json = do { local $/; readline($c) };
+	($status, $hdr, my $json) = do_req $srv, 'GET /env_dump';
+	is($status, undef, 'no status for HTTP/0.9');
+	is($hdr, undef, 'no header for HTTP/0.9');
 	unlike($json, qr/^Connection: /smi, 'no connection header for 0.9');
 	unlike($json, qr!\AHTTP/!s, 'no HTTP/1.x prefix for 0.9');
 	my $env = JSON::PP->new->decode($json);
@@ -102,8 +101,7 @@ SKIP: {
 }
 
 # cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
-$c = tcp_start($srv, 'GET /nil-header-value HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
+($status, $hdr) = do_req($srv, 'GET /nil-header-value HTTP/1.0');
 is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
 	'nil header value accepted for broken apps') or diag(explain($hdr));
 
@@ -128,12 +126,10 @@ my $ck_early_hints = sub {
 $ck_early_hints->('ccc off'); # we'll retest later
 
 if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
-	$c = tcp_start($srv, 'POST /tweak-status-code HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr) = do_req $srv, 'POST /tweak-status-code HTTP/1.0';
 	like($status, qr!\AHTTP/1\.[01] 200 HI\b!, 'status tweaked');
 
-	$c = tcp_start($srv, 'POST /restore-status-code HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr) = do_req $srv, 'POST /restore-status-code HTTP/1.0';
 	is($status, $orig_200_status, 'original status restored');
 }
 
@@ -145,12 +141,11 @@ SKIP: {
 }
 
 if ('bad requests') {
-	$c = tcp_start($srv, 'GET /env_dump HTTP/1/1');
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr) = do_req $srv, 'GET /env_dump HTTP/1/1';
 	like($status, qr!\AHTTP/1\.[01] 400 \b!, 'got 400 on bad request');
 
 	$c = tcp_start($srv);
-	print $c 'GET /';;
+	print $c 'GET /';
 	my $buf = join('', (0..9), 'ab');
 	for (0..1023) { print $c $buf }
 	print $c " HTTP/1.0\r\n\r\n";
@@ -308,12 +303,10 @@ EOM
 	$wpid =~ s/\Apid=// or die;
 	ok(CORE::kill(0, $wpid), 'worker PID retrieved');
 
-	$c = tcp_start($srv, $req);
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr) = do_req($srv, $req);
 	like($status, qr!\AHTTP/1\.[01] 200\b!, 'minimal request succeeds');
 
-	$c = tcp_start($srv, 'GET /xxxxxx HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
+	($status, $hdr) = do_req($srv, 'GET /xxxxxx HTTP/1.0');
 	like($status, qr!\AHTTP/1\.[01] 413\b!, 'big request fails');
 }
 
diff --git a/t/lib.perl b/t/lib.perl
index 7de9e426..13e390d6 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -12,7 +12,8 @@ use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh, $err_log);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn
 	$tmpdir $errfh $err_log
-	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr);
+	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr
+	do_req);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
@@ -182,6 +183,19 @@ sub unicorn {
 	UnicornTest::AutoReap->new($pid);
 }
 
+sub do_req ($@) {
+	my ($dst, @req) = @_;
+	my $c = ref($dst) ? tcp_start($dst, @req) : unix_start($dst, @req);
+	return $c if !wantarray;
+	my ($status, $hdr);
+	# read headers iff HTTP/1.x request, HTTP/0.9 remains supported
+	my ($first) = (join('', @req) =~ m!\A([^\r\n]+)!);
+	($status, $hdr) = slurp_hdr($c) if $first =~ m{\s*HTTP/\S+$};
+	my $bdy = do { local $/; <$c> };
+	close $c;
+	($status, $hdr, $bdy);
+}
+
 # automatically kill + reap children when this goes out-of-scope
 package UnicornTest::AutoReap;
 use v5.14;
diff --git a/t/reload-bad-config.t b/t/reload-bad-config.t
index c7055c7e..543421da 100644
--- a/t/reload-bad-config.t
+++ b/t/reload-bad-config.t
@@ -25,9 +25,7 @@ EOM
 close $fh;
 
 my $ar = unicorn(qw(-E none -c), $u_conf, $ru, { 3 => $srv });
-my $c = tcp_start($srv, 'GET / HTTP/1.0');
-my ($status, $hdr) = slurp_hdr($c);
-my $bdy = do { local $/; <$c> };
+my ($status, $hdr, $bdy) = do_req($srv, 'GET / HTTP/1.0');
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid at start');
 is($bdy, "hello world\n", 'body matches expected');
 
@@ -47,9 +45,7 @@ ok(grep(/error reloading/, @l), 'got error reloading');
 open $fh, '>', $err_log;
 close $fh;
 
-$c = tcp_start($srv, 'GET / HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
-$bdy = do { local $/; <$c> };
+($status, $hdr, $bdy) = do_req($srv, 'GET / HTTP/1.0');
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid afte reload');
 is($bdy, "hello world\n", 'body matches expected after reload');
 
diff --git a/t/reopen-logs.t b/t/reopen-logs.t
index e1bf524c..8a58c1b9 100644
--- a/t/reopen-logs.t
+++ b/t/reopen-logs.t
@@ -14,9 +14,7 @@ EOM
 close $fh;
 
 my $auto_reap = unicorn('-c', $u_conf, 't/reopen-logs.ru', { 3 => $srv } );
-my $c = tcp_start($srv, 'GET / HTTP/1.0');
-my ($status, $hdr) = slurp_hdr($c);
-my $bdy = do { local $/; <$c> };
+my ($status, $hdr, $bdy) = do_req($srv, 'GET / HTTP/1.0');
 is($bdy, "true\n", 'logs opened');
 
 rename($err_log, "$err_log.rot");
@@ -31,9 +29,7 @@ while (!-f $out_log && --$tries) { select undef, undef, undef, 0.01 };
 ok(-f $out_log, 'stdout_path recreated after USR1');
 ok(-f $err_log, 'stderr_path recreated after USR1');
 
-$c = tcp_start($srv, 'GET / HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
-$bdy = do { local $/; <$c> };
+($status, $hdr, $bdy) = do_req($srv, 'GET / HTTP/1.0');
 is($bdy, "true\n", 'logs reopened with sync==true');
 
 $auto_reap->join('QUIT');
diff --git a/t/winch_ttin.t b/t/winch_ttin.t
index 1a198778..509b118f 100644
--- a/t/winch_ttin.t
+++ b/t/winch_ttin.t
@@ -34,10 +34,8 @@ my $worker_nr = <$fh>;
 close $fh;
 is($worker_nr, '0', 'initial worker spawned');
 
-my $c = unix_start($u_sock, 'GET /pid HTTP/1.0');
-my ($status, $hdr) = slurp_hdr($c);
+my ($status, $hdr, $worker_pid) = do_req($u_sock, 'GET /pid HTTP/1.0');
 like($status, qr/ 200\b/, 'got 200 response');
-my $worker_pid = do { local $/; <$c> };
 like($worker_pid, qr/\A[0-9]+\n\z/s, 'PID in response');
 chomp $worker_pid;
 ok(kill(0, $worker_pid), 'worker_pid is valid');
@@ -57,11 +55,10 @@ $worker_nr = <$fh>;
 close $fh;
 is($worker_nr, '0', 'worker restarted');
 
-$c = unix_start($u_sock, 'GET /pid HTTP/1.0');
-($status, $hdr) = slurp_hdr($c);
+($status, $hdr, my $new_worker_pid) = do_req($u_sock, 'GET /pid HTTP/1.0');
 like($status, qr/ 200\b/, 'got 200 response');
-chomp(my $new_worker_pid = do { local $/; <$c> });
-like($new_worker_pid, qr/\A[0-9]+\z/, 'got new worker PID');
+like($new_worker_pid, qr/\A[0-9]+\n\z/, 'got new worker PID');
+chomp $new_worker_pid;
 ok(kill(0, $new_worker_pid), 'got a valid worker PID');
 isnt($worker_pid, $new_worker_pid, 'worker PID changed');
 
diff --git a/t/working_directory.t b/t/working_directory.t
index e7ff43a5..6c974720 100644
--- a/t/working_directory.t
+++ b/t/working_directory.t
@@ -52,10 +52,8 @@ END { $stop_daemon->(1) if defined $pid };
 unicorn('-c', $u_conf)->join; # will daemonize
 chomp($pid = slurp("$tmpdir/pid"));
 
-my $c = unix_start($u_sock, 'GET / HTTP/1.0');
-my ($status, $hdr) = slurp_hdr($c);
-chomp(my $bdy = do { local $/; <$c> });
-is($bdy, 1, 'got expected $master_ppid');
+my ($status, $hdr, $bdy) = do_req($u_sock, 'GET / HTTP/1.0');
+is($bdy, "1\n", 'got expected $master_ppid');
 
 $stop_daemon->();
 check_stderr;
@@ -69,10 +67,8 @@ if ('test without CLI switches in config.ru') {
 	unicorn('-D', '-l', $u_sock, '-c', $u_conf)->join; # will daemonize
 	chomp($pid = slurp("$tmpdir/pid"));
 
-	$c = unix_start($u_sock, 'GET / HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
-	chomp($bdy = do { local $/; <$c> });
-	is($bdy, 1, 'got expected $master_ppid');
+	($status, $hdr, $bdy) = do_req($u_sock, 'GET / HTTP/1.0');
+	is($bdy, "1\n", 'got expected $master_ppid');
 
 	$stop_daemon->();
 	check_stderr;
@@ -107,12 +103,9 @@ EOM
 	my $srv = tcp_server;
 	my $auto_reap = unicorn(qw(-c), $u_conf, qw(-I. fooapp.rb),
 				{ -C => '/', 3 => $srv });
-	$c = tcp_start($srv, 'GET / HTTP/1.0');
-	($status, $hdr) = slurp_hdr($c);
-	chomp($bdy = do { local $/; <$c> });
+	($status, $hdr, $bdy) = do_req($srv, 'GET / HTTP/1.0');
 	is($bdy, "dir=$tmpdir/alt",
 		'fooapp.rb (w/o config.ru) w/ working_directory');
-	close $c;
 	$auto_reap->join('TERM');
 	is($?, 0, 'fooapp.rb process exited');
 	check_stderr;

[-- Attachment #8: 0007-tests-use-more-common-variable-names-between-tests.patch --]
[-- Type: text/x-diff, Size: 6507 bytes --]

From 948f78403172657590d690b9255467b9ccb968cd Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 09:31:44 +0000
Subject: [PATCH 07/11] tests: use more common variable names between tests

Stuff like $u_conf, $daemon_pid, $pid_file, etc. will
reduce cognitive overhead.
---
 t/active-unix-socket.t      |  2 +-
 t/client_body_buffer_size.t |  6 ++----
 t/heartbeat-timeout.t       |  3 +--
 t/integration.t             |  5 ++---
 t/lib.perl                  | 31 +++++++++++++++++++++++++++----
 t/working_directory.t       | 31 +++++--------------------------
 6 files changed, 38 insertions(+), 40 deletions(-)

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index 4dcc8dc6..32cb0c2e 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -15,7 +15,7 @@ my $u2 = "$tmpdir/u2.sock";
 	print $fh <<EOM;
 pid "$tmpdir/u.pid"
 listen "$u1"
-stderr_path "$tmpdir/err.log"
+stderr_path "$err_log"
 EOM
 	close $fh;
 
diff --git a/t/client_body_buffer_size.t b/t/client_body_buffer_size.t
index 3067f284..d4799012 100644
--- a/t/client_body_buffer_size.t
+++ b/t/client_body_buffer_size.t
@@ -4,16 +4,14 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
-my $uconf = "$tmpdir/u.conf.rb";
-
-open my $conf_fh, '>', $uconf;
+open my $conf_fh, '>', $u_conf;
 $conf_fh->autoflush(1);
 print $conf_fh <<EOM;
 client_body_buffer_size 0
 EOM
 my $srv = tcp_server();
 my $host_port = tcp_host_port($srv);
-my @uarg = (qw(-E none t/client_body_buffer_size.ru -c), $uconf);
+my @uarg = (qw(-E none t/client_body_buffer_size.ru -c), $u_conf);
 my $ar = unicorn(@uarg, { 3 => $srv });
 my ($c, $status, $hdr);
 my $mem_class = 'StringIO';
diff --git a/t/heartbeat-timeout.t b/t/heartbeat-timeout.t
index ce1f7e16..694867a4 100644
--- a/t/heartbeat-timeout.t
+++ b/t/heartbeat-timeout.t
@@ -6,7 +6,6 @@ use autodie;
 use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
 mkdir "$tmpdir/alt";
 my $srv = tcp_server();
-my $u_conf = "$tmpdir/u.conf.rb";
 open my $fh, '>', $u_conf;
 print $fh <<EOM;
 pid "$tmpdir/pid"
@@ -23,7 +22,7 @@ like($status, qr!\AHTTP/1\.[01] 200\b!, 'PID request succeeds');
 like($wpid, qr/\A[0-9]+\z/, 'worker is running');
 
 my $t0 = clock_gettime(CLOCK_MONOTONIC);
-$c = tcp_start($srv, 'GET /block-forever HTTP/1.0');
+my $c = tcp_start($srv, 'GET /block-forever HTTP/1.0');
 vec(my $rvec = '', fileno($c), 1) = 1;
 is(select($rvec, undef, undef, 6), 1, 'got readiness');
 $c->blocking(0);
diff --git a/t/integration.t b/t/integration.t
index 13b07467..eb40ffc7 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -10,15 +10,14 @@ use autodie;
 our $srv = tcp_server();
 our $host_port = tcp_host_port($srv);
 my $t0 = time;
-my $conf = "$tmpdir/u.conf.rb";
-open my $conf_fh, '>', $conf;
+open my $conf_fh, '>', $u_conf;
 $conf_fh->autoflush(1);
 my $u1 = "$tmpdir/u1";
 print $conf_fh <<EOM;
 early_hints true
 listen "$u1"
 EOM
-my $ar = unicorn(qw(-E none t/integration.ru -c), $conf, { 3 => $srv });
+my $ar = unicorn(qw(-E none t/integration.ru -c), $u_conf, { 3 => $srv });
 my $curl = which('curl');
 my $fifo = "$tmpdir/fifo";
 POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
diff --git a/t/lib.perl b/t/lib.perl
index 13e390d6..244972bc 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -6,20 +6,43 @@ use v5.14;
 use parent qw(Exporter);
 use autodie;
 use Test::More;
+use Time::HiRes qw(sleep);
 use IO::Socket::INET;
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
-our ($tmpdir, $errfh, $err_log);
+our ($tmpdir, $errfh, $err_log, $u_sock, $u_conf, $daemon_pid,
+	$pid_file);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn
-	$tmpdir $errfh $err_log
+	$tmpdir $errfh $err_log $u_sock $u_conf $daemon_pid $pid_file
 	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr
-	do_req);
+	do_req stop_daemon);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
 $err_log = "$tmpdir/err.log";
+$pid_file = "$tmpdir/pid";
+$u_sock = "$tmpdir/u.sock";
+$u_conf = "$tmpdir/u.conf.rb";
 open($errfh, '>>', $err_log);
-END { diag slurp($err_log) if $tmpdir };
+
+sub stop_daemon (;$) {
+	my ($is_END) = @_;
+	kill('TERM', $daemon_pid);
+	my $tries = 1000;
+	while (CORE::kill(0, $daemon_pid) && --$tries) { sleep(0.01) }
+	if ($is_END && CORE::kill(0, $daemon_pid)) { # after done_testing
+		CORE::kill('KILL', $daemon_pid);
+		die "daemon_pid=$daemon_pid did not die";
+	} else {
+		ok(!CORE::kill(0, $daemon_pid), 'daemonized unicorn gone');
+		undef $daemon_pid;
+	}
+};
+
+END {
+	diag slurp($err_log) if $tmpdir;
+	stop_daemon(1) if defined $daemon_pid;
+};
 
 sub check_stderr () {
 	my @log = slurp($err_log);
diff --git a/t/working_directory.t b/t/working_directory.t
index 6c974720..f9254eb8 100644
--- a/t/working_directory.t
+++ b/t/working_directory.t
@@ -4,12 +4,10 @@
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
 mkdir "$tmpdir/alt";
-my $u_sock = "$tmpdir/u.sock";
 my $ru = "$tmpdir/alt/config.ru";
-my $u_conf = "$tmpdir/u.conf.rb";
 open my $fh, '>', $u_conf;
 print $fh <<EOM;
-pid "$tmpdir/pid"
+pid "$pid_file"
 preload_app true
 stderr_path "$err_log"
 working_directory "$tmpdir/alt" # the whole point of this test
@@ -30,32 +28,13 @@ $common_ru
 EOM
 close $fh;
 
-my $pid;
-my $stop_daemon = sub {
-	my ($is_END) = @_;
-	kill('TERM', $pid);
-	my $tries = 1000;
-	while (CORE::kill(0, $pid) && --$tries) {
-		select undef, undef, undef, 0.01;
-	}
-	if ($is_END && CORE::kill(0, $pid)) {
-		CORE::kill('KILL', $pid);
-		die "daemonized PID=$pid did not die";
-	} else {
-		ok(!CORE::kill(0, $pid), 'daemonized unicorn gone');
-		undef $pid;
-	}
-};
-
-END { $stop_daemon->(1) if defined $pid };
-
 unicorn('-c', $u_conf)->join; # will daemonize
-chomp($pid = slurp("$tmpdir/pid"));
+chomp($daemon_pid = slurp($pid_file));
 
 my ($status, $hdr, $bdy) = do_req($u_sock, 'GET / HTTP/1.0');
 is($bdy, "1\n", 'got expected $master_ppid');
 
-$stop_daemon->();
+stop_daemon;
 check_stderr;
 
 if ('test without CLI switches in config.ru') {
@@ -65,12 +44,12 @@ if ('test without CLI switches in config.ru') {
 	close $fh;
 
 	unicorn('-D', '-l', $u_sock, '-c', $u_conf)->join; # will daemonize
-	chomp($pid = slurp("$tmpdir/pid"));
+	chomp($daemon_pid = slurp($pid_file));
 
 	($status, $hdr, $bdy) = do_req($u_sock, 'GET / HTTP/1.0');
 	is($bdy, "1\n", 'got expected $master_ppid');
 
-	$stop_daemon->();
+	stop_daemon;
 	check_stderr;
 }
 

[-- Attachment #9: 0008-tests-use-Time-HiRes-sleep-and-time-everywhere.patch --]
[-- Type: text/x-diff, Size: 4106 bytes --]

From dd9f2efeebf20cfa1def0ce92cb4e35a8b5c1580 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 09:35:09 +0000
Subject: [PATCH 08/11] tests: use Time::HiRes `sleep' and `time' everywhere

The time(2) syscall use by CORE::time is inaccurate[1].
It's also easier to read `sleep 0.01' rather than the
longer `select' equivalent.

[1] a6463151bd1db5b9 (httpdate: favor gettimeofday(2) over time(2) for correctness, 2023-06-01)
---
 t/active-unix-socket.t | 2 +-
 t/integration.t        | 5 +++--
 t/lib.perl             | 4 ++--
 t/reload-bad-config.t  | 2 +-
 t/reopen-logs.t        | 4 ++--
 t/winch_ttin.t         | 4 +---
 6 files changed, 10 insertions(+), 11 deletions(-)

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index 32cb0c2e..ff731b5f 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -86,7 +86,7 @@ is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
 		'fail to connect to u1');
 	for (1..50) { # wait for init process to reap worker
 		kill(0, $worker_pid) or last;
-		select(undef, undef, undef, 0.011);
+		sleep 0.011;
 	}
 	ok(!kill(0, $worker_pid), 'worker gone after parent dies');
 }
diff --git a/t/integration.t b/t/integration.t
index eb40ffc7..80485e44 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -77,8 +77,9 @@ SKIP: { # Date header check
 	eval { require HTTP::Date } or skip "HTTP::Date missing: $@", 1;
 	$d[0] =~ s/^Date: //i or die 'BUG: did not strip date: prefix';
 	my $t = HTTP::Date::str2time($d[0]);
-	ok($t >= $t0 && $t > 0 && $t <= time, 'valid date') or
-		diag(explain([$t, $!, \@d]));
+	my $now = time;
+	ok($t >= ($t0 - 1) && $t > 0 && $t <= ($now + 1), 'valid date') or
+		diag(explain(["t=$t t0=$t0 now=$now", $!, \@d]));
 };
 
 
diff --git a/t/lib.perl b/t/lib.perl
index 244972bc..9254b23b 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -6,7 +6,7 @@ use v5.14;
 use parent qw(Exporter);
 use autodie;
 use Test::More;
-use Time::HiRes qw(sleep);
+use Time::HiRes qw(sleep time);
 use IO::Socket::INET;
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
@@ -15,7 +15,7 @@ our ($tmpdir, $errfh, $err_log, $u_sock, $u_conf, $daemon_pid,
 our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn
 	$tmpdir $errfh $err_log $u_sock $u_conf $daemon_pid $pid_file
 	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr
-	do_req stop_daemon);
+	do_req stop_daemon sleep time);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
diff --git a/t/reload-bad-config.t b/t/reload-bad-config.t
index 543421da..c023b88c 100644
--- a/t/reload-bad-config.t
+++ b/t/reload-bad-config.t
@@ -38,7 +38,7 @@ my @l;
 for (1..1000) {
 	@l = grep(/(?:done|error) reloading/, slurp($err_log)) and
 		last;
-	select undef, undef, undef, 0.011;
+	sleep 0.011;
 }
 diag slurp($err_log) if $ENV{V};
 ok(grep(/error reloading/, @l), 'got error reloading');
diff --git a/t/reopen-logs.t b/t/reopen-logs.t
index 8a58c1b9..76a4dbdf 100644
--- a/t/reopen-logs.t
+++ b/t/reopen-logs.t
@@ -23,8 +23,8 @@ rename($out_log, "$out_log.rot");
 $auto_reap->do_kill('USR1');
 
 my $tries = 1000;
-while (!-f $err_log && --$tries) { select undef, undef, undef, 0.01 };
-while (!-f $out_log && --$tries) { select undef, undef, undef, 0.01 };
+while (!-f $err_log && --$tries) { sleep 0.01 };
+while (!-f $out_log && --$tries) { sleep 0.01 };
 
 ok(-f $out_log, 'stdout_path recreated after USR1');
 ok(-f $err_log, 'stderr_path recreated after USR1');
diff --git a/t/winch_ttin.t b/t/winch_ttin.t
index 509b118f..c5079599 100644
--- a/t/winch_ttin.t
+++ b/t/winch_ttin.t
@@ -43,9 +43,7 @@ ok(kill(0, $worker_pid), 'worker_pid is valid');
 ok(kill('WINCH', $pid), 'SIGWINCH can be sent');
 
 my $tries = 1000;
-while (CORE::kill(0, $worker_pid) && --$tries) {
-	select undef, undef, undef, 0.01;
-}
+while (CORE::kill(0, $worker_pid) && --$tries) { sleep 0.01 }
 ok(!CORE::kill(0, $worker_pid), 'worker not running');
 
 ok(kill('TTIN', $pid), 'SIGTTIN to restart worker');

[-- Attachment #10: 0009-tests-fold-SO_KEEPALIVE-check-to-Perl-5-integration.patch --]
[-- Type: text/x-diff, Size: 2675 bytes --]

From b588ccbbf73547487f54fd1a9d5396d6848e8661 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 19:21:05 +0000
Subject: [PATCH 09/11] tests: fold SO_KEEPALIVE check to Perl 5 integration

No need to startup more processes than necessary.
---
 t/integration.t        | 13 +++++++++++++
 test/exec/test_exec.rb | 23 +----------------------
 2 files changed, 14 insertions(+), 22 deletions(-)

diff --git a/t/integration.t b/t/integration.t
index 80485e44..bea221ce 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -7,8 +7,16 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
+use Socket qw(SOL_SOCKET SO_KEEPALIVE);
 our $srv = tcp_server();
 our $host_port = tcp_host_port($srv);
+
+if ('ensure Perl does not set SO_KEEPALIVE by default') {
+	my $val = getsockopt($srv, SOL_SOCKET, SO_KEEPALIVE);
+	unpack('i', $val) == 0 or
+		setsockopt($srv, SOL_SOCKET, SO_KEEPALIVE, pack('i', 0));
+	$val = getsockopt($srv, SOL_SOCKET, SO_KEEPALIVE);
+}
 my $t0 = time;
 open my $conf_fh, '>', $u_conf;
 $conf_fh->autoflush(1);
@@ -71,6 +79,11 @@ is_deeply([ grep(/^X-R2: /, @$hdr) ],
 	[ 'X-R2: a', 'X-R2: b', 'X-R2: c' ],
 	'rack 2 LF-delimited headers supported') or diag(explain($hdr));
 
+{
+	my $val = getsockopt($srv, SOL_SOCKET, SO_KEEPALIVE);
+	is(unpack('i', $val), 1, 'SO_KEEPALIVE set on inherited socket');
+}
+
 SKIP: { # Date header check
 	my @d = grep(/^Date: /i, @$hdr);
 	is(scalar(@d), 1, 'got one date header') or diag(explain(\@d));
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index 55f828e7..84944520 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -1,6 +1,5 @@
 # -*- encoding: binary -*-
-
-# Copyright (c) 2009 Eric Wong
+# Don't add to this file, new tests are in Perl 5. See t/README
 FLOCK_PATH = File.expand_path(__FILE__)
 require './test/test_helper'
 
@@ -97,26 +96,6 @@ def teardown
     end
   end
 
-  def test_inherit_listener_unspecified
-    File.open("config.ru", "wb") { |fp| fp.write(HI) }
-    sock = TCPServer.new(@addr, @port)
-    sock.setsockopt(:SOL_SOCKET, :SO_KEEPALIVE, 0)
-
-    pid = xfork do
-      redirect_test_io do
-        ENV['UNICORN_FD'] = sock.fileno.to_s
-        exec($unicorn_bin, sock.fileno => sock.fileno)
-      end
-    end
-    res = hit(["http://#@addr:#@port/"])
-    assert_equal [ "HI\n" ], res
-    assert_shutdown(pid)
-    assert sock.getsockopt(:SOL_SOCKET, :SO_KEEPALIVE).bool,
-                'unicorn should always set SO_KEEPALIVE on inherited sockets'
-  ensure
-    sock.close if sock
-  end
-
   def test_working_directory_rel_path_config_file
     other = Tempfile.new('unicorn.wd')
     File.unlink(other.path)

[-- Attachment #11: 0010-tests-move-broken-app-test-to-Perl-5-integration-tes.patch --]
[-- Type: text/x-diff, Size: 2376 bytes --]

From 7160f1b519aece0fe645d22a7d8fb954a43ad6fb Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 19:37:32 +0000
Subject: [PATCH 10/11] tests: move broken app test to Perl 5 integration test

Less Ruby means fewer incompatibilities to worry about with
every new version.
---
 t/integration.ru         |  1 +
 t/integration.t          |  6 ++++++
 test/unit/test_server.rb | 14 --------------
 3 files changed, 7 insertions(+), 14 deletions(-)

diff --git a/t/integration.ru b/t/integration.ru
index 086126ab..888833a9 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -98,6 +98,7 @@ def rack_input_tests(env)
     when '/pid'; [ 200, {}, [ "#$$\n" ] ]
     when '/early_hints_rack2'; early_hints(env, "r\n2")
     when '/early_hints_rack3'; early_hints(env, %w(r 3))
+    when '/broken_app'; raise RuntimeError, 'hello'
     else '/'; [ 200, {}, [ env_dump(env) ] ]
     end # case PATH_INFO (GET)
   when 'POST'
diff --git a/t/integration.t b/t/integration.t
index bea221ce..ba17dd9e 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -118,6 +118,12 @@ SKIP: {
 is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
 	'nil header value accepted for broken apps') or diag(explain($hdr));
 
+check_stderr;
+($status, $hdr, $bdy) = do_req($srv, 'GET /broken_app HTTP/1.0');
+like($status, qr!\AHTTP/1\.[0-1] 500\b!, 'got 500 error on broken endpoint');
+is($bdy, undef, 'no response body after exception');
+truncate($errfh, 0);
+
 my $ck_early_hints = sub {
 	my ($note) = @_;
 	$c = unix_start($u1, 'GET /early_hints_rack2 HTTP/1.0');
diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index 0a710d12..2af12eac 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -127,20 +127,6 @@ def test_after_reply
     sock.close
   end
 
-  def test_broken_app
-    teardown
-    app = lambda { |env| raise RuntimeError, "hello" }
-    # [200, {}, []] }
-    redirect_test_io do
-      @server = HttpServer.new(app, :listeners => [ "127.0.0.1:#@port"] )
-      @server.start
-    end
-    sock = tcp_socket('127.0.0.1', @port)
-    sock.syswrite("GET / HTTP/1.0\r\n\r\n")
-    assert_match %r{\AHTTP/1.[01] 500\b}, sock.sysread(4096)
-    assert_nil sock.close
-  end
-
   def test_simple_server
     results = hit(["http://localhost:#{@port}/test"])
     assert_equal 'hello!\n', results[0], "Handler didn't really run"

[-- Attachment #12: 0011-tests-fold-early-shutdown-tests-into-t-integration.t.patch --]
[-- Type: text/x-diff, Size: 4527 bytes --]

From 05028146b5e69c566663fdab9f8b92c6145a791a Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Sun, 10 Sep 2023 19:52:03 +0000
Subject: [PATCH 11/11] tests: fold early shutdown() tests into t/integration.t

This means fewer redundant tests and more chances to notice
Ruby incompatibilities.
---
 t/integration.t          | 22 +++++++++++++++--
 test/unit/test_server.rb | 53 ----------------------------------------
 2 files changed, 20 insertions(+), 55 deletions(-)

diff --git a/t/integration.t b/t/integration.t
index ba17dd9e..7310ff29 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -7,7 +7,7 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
-use Socket qw(SOL_SOCKET SO_KEEPALIVE);
+use Socket qw(SOL_SOCKET SO_KEEPALIVE SHUT_WR);
 our $srv = tcp_server();
 our $host_port = tcp_host_port($srv);
 
@@ -209,6 +209,7 @@ SKIP: {
 		defined($opt{overwrite}) and
 			print { $c } ('x' x $opt{overwrite});
 		$c->flush or die $!;
+		shutdown($c, SHUT_WR);
 		($status, $hdr) = slurp_hdr($c);
 		is(readline($c), $blob_hash, "$sub $path");
 	};
@@ -225,6 +226,8 @@ SKIP: {
 	# ensure small overwrites don't get checksummed
 	$ck_hash->('identity', '/rack_input', -s => $blob_size,
 			overwrite => 1); # one extra byte
+	unlike(slurp($err_log), qr/ClientShutdown/,
+		'no overreads after client SHUT_WR');
 
 	# excessive overwrite truncated
 	$c = tcp_start($srv);
@@ -238,8 +241,23 @@ SKIP: {
 		$! = 0;
 		while (print $c $buf and time < $end) { ++$n }
 		ok($!, 'overwrite truncated') or diag "n=$n err=$! ".time;
+		undef $c;
+	}
+
+	# client shutdown early
+	$c = tcp_start($srv);
+	$c->autoflush(0);
+	print $c "PUT /rack_input HTTP/1.0\r\nContent-Length: 16384\r\n\r\n";
+	if (1) {
+		local $SIG{PIPE} = 'IGNORE';
+		print $c 'too short body';
+		shutdown($c, SHUT_WR);
+		vec(my $rvec = '', fileno($c), 1) = 1;
+		select($rvec, undef, undef, 10) or BAIL_OUT "timed out";
+		my $buf = <$c>;
+		is($buf, undef, 'server aborted after client SHUT_WR');
+		undef $c;
 	}
-	undef $c;
 
 	$curl // skip 'no curl found in PATH', 1;
 
diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index 2af12eac..7ffa48f0 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -132,59 +132,6 @@ def test_simple_server
     assert_equal 'hello!\n', results[0], "Handler didn't really run"
   end
 
-  def test_client_shutdown_writes
-    bs = 15609315 * rand
-    sock = tcp_socket('127.0.0.1', @port)
-    sock.syswrite("PUT /hello HTTP/1.1\r\n")
-    sock.syswrite("Host: example.com\r\n")
-    sock.syswrite("Transfer-Encoding: chunked\r\n")
-    sock.syswrite("Trailer: X-Foo\r\n")
-    sock.syswrite("\r\n")
-    sock.syswrite("%x\r\n" % [ bs ])
-    sock.syswrite("F" * bs)
-    sock.syswrite("\r\n0\r\nX-")
-    "Foo: bar\r\n\r\n".each_byte do |x|
-      sock.syswrite x.chr
-      sleep 0.05
-    end
-    # we wrote the entire request before shutting down, server should
-    # continue to process our request and never hit EOFError on our sock
-    sock.shutdown(Socket::SHUT_WR)
-    buf = sock.read
-    assert_match %r{\bhello!\\n\b}, buf.split(/\r\n\r\n/, 2).last
-    next_client = Net::HTTP.get(URI.parse("http://127.0.0.1:#@port/"))
-    assert_equal 'hello!\n', next_client
-    lines = File.readlines("test_stderr.#$$.log")
-    assert lines.grep(/^Unicorn::ClientShutdown: /).empty?
-    assert_nil sock.close
-  end
-
-  def test_client_shutdown_write_truncates
-    bs = 15609315 * rand
-    sock = tcp_socket('127.0.0.1', @port)
-    sock.syswrite("PUT /hello HTTP/1.1\r\n")
-    sock.syswrite("Host: example.com\r\n")
-    sock.syswrite("Transfer-Encoding: chunked\r\n")
-    sock.syswrite("Trailer: X-Foo\r\n")
-    sock.syswrite("\r\n")
-    sock.syswrite("%x\r\n" % [ bs ])
-    sock.syswrite("F" * (bs / 2.0))
-
-    # shutdown prematurely, this will force the server to abort
-    # processing on us even during app dispatch
-    sock.shutdown(Socket::SHUT_WR)
-    IO.select([sock], nil, nil, 60) or raise "Timed out"
-    buf = sock.read
-    assert_equal "", buf
-    next_client = Net::HTTP.get(URI.parse("http://127.0.0.1:#@port/"))
-    assert_equal 'hello!\n', next_client
-    lines = File.readlines("test_stderr.#$$.log")
-    lines = lines.grep(/^Unicorn::ClientShutdown: bytes_read=\d+/)
-    assert_equal 1, lines.size
-    assert_match %r{\AUnicorn::ClientShutdown: bytes_read=\d+ true$}, lines[0]
-    assert_nil sock.close
-  end
-
   def test_client_malformed_body
     bs = 15653984
     sock = tcp_socket('127.0.0.1', @port)

^ permalink raw reply related	[relevance 1%]

* [RFC 0-3/3] depend on Ruby 2.5+, eliminate kgio
@ 2023-09-05  9:44  1% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2023-09-05  9:44 UTC (permalink / raw)
  To: unicorn-public

[-- Attachment #1: Type: text/plain, Size: 1658 bytes --]

Explanation (and bulk of the work is in patch 2/3).

I wrote patch 1/3 over 4 years ago since it was simple and
didn't rely on "newer" Ruby 2.3 features.  Patch 2/3 completes
the change and actually depends on 2.5+; while patch 3/3 updates
the gemspec, docs and dependencies for Ruby 2.5+.

I haven't actually used Ruby 2.5 in a while, but I'm still on
Ruby 2.7 since that's what I have in Debian bullseye
(oldstable).

Patch 2/3 could use an extra set of eyes since it's fairly big,
but passes all tests.

Note: I can't benchmark anything since I have limited (shared)
hardware

Eric Wong (3):
  remove kgio from all read(2) and write(2) wrappers
  kill off remaining kgio uses
  update dependency to Ruby 2.5+

 HACKING                         |  2 +-
 README                          |  2 +-
 lib/unicorn.rb                  |  3 +-
 lib/unicorn/http_request.rb     | 18 ++++-----
 lib/unicorn/http_server.rb      | 38 +++++++----------
 lib/unicorn/oob_gc.rb           |  4 +-
 lib/unicorn/socket_helper.rb    | 50 +++--------------------
 lib/unicorn/stream_input.rb     | 20 +++++----
 lib/unicorn/worker.rb           | 10 ++---
 lib/unicorn/write_splat.rb      |  7 ----
 t/README                        |  2 +-
 t/oob_gc.ru                     |  3 --
 t/oob_gc_path.ru                |  3 --
 test/unit/test_request.rb       | 47 ++++++++-------------
 test/unit/test_socket_helper.rb | 72 +++++++--------------------------
 test/unit/test_stream_input.rb  |  2 +-
 test/unit/test_tee_input.rb     |  2 +-
 unicorn.gemspec                 |  5 +--
 18 files changed, 87 insertions(+), 203 deletions(-)
 delete mode 100644 lib/unicorn/write_splat.rb

[-- Attachment #2: 0001-remove-kgio-from-all-read-2-and-write-2-wrappers.patch --]
[-- Type: text/x-diff, Size: 5330 bytes --]

From 36ba7f971c571031101c0b718724bdcb06dd7e03 Mon Sep 17 00:00:00 2001
From: Eric Wong <e@80x24.org>
Date: Sun, 26 May 2019 22:15:44 +0000
Subject: [RFC 1/3] remove kgio from all read(2) and write(2) wrappers

It's fairly easy given unicorn was designed with synchronous I/O
in mind.  The overhead of backtraces from EOFError on
readpartial should be rare given our requirement to only accept
requests from fast, reliable clients on LAN (e.g. nginx or
yet-another-horribly-named-server).
---
 lib/unicorn/http_request.rb |  4 ++--
 lib/unicorn/http_server.rb  |  8 +++++---
 lib/unicorn/stream_input.rb | 20 ++++++++++++--------
 lib/unicorn/worker.rb       |  4 ++--
 4 files changed, 21 insertions(+), 15 deletions(-)

diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index e3ad592..8bca60a 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -74,11 +74,11 @@ def read(socket)
     e['REMOTE_ADDR'] = socket.kgio_addr
 
     # short circuit the common case with small GET requests first
-    socket.kgio_read!(16384, buf)
+    socket.readpartial(16384, buf)
     if parse.nil?
       # Parser is not done, queue up more data to read and continue parsing
       # an Exception thrown from the parser will throw us out of the loop
-      false until add_parse(socket.kgio_read!(16384))
+      false until add_parse(socket.readpartial(16384))
     end
 
     check_client_connection(socket) if @@check_client_connection
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index f1b4a54..2f1eb1b 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -389,12 +389,13 @@ def master_sleep(sec)
     # the Ruby itself and not require a separate malloc (on 32-bit MRI 1.9+).
     # Most reads are only one byte here and uncommon, so it's not worth a
     # persistent buffer, either:
-    @self_pipe[0].kgio_tryread(11)
+    @self_pipe[0].read_nonblock(11, exception: false)
   end
 
   def awaken_master
     return if $$ != @master_pid
-    @self_pipe[1].kgio_trywrite('.') # wakeup master process from select
+    # wakeup master process from select
+    @self_pipe[1].write_nonblock('.', exception: false)
   end
 
   # reaps all unreaped workers
@@ -565,7 +566,8 @@ def handle_error(client, e)
       500
     end
     if code
-      client.kgio_trywrite(err_response(code, @request.response_start_sent))
+      code = err_response(code, @request.response_start_sent)
+      client.write_nonblock(code, exception: false)
     end
     client.close
   rescue
diff --git a/lib/unicorn/stream_input.rb b/lib/unicorn/stream_input.rb
index 41d28a0..9246f73 100644
--- a/lib/unicorn/stream_input.rb
+++ b/lib/unicorn/stream_input.rb
@@ -49,8 +49,7 @@ def read(length = nil, rv = '')
         to_read = length - @rbuf.size
         rv.replace(@rbuf.slice!(0, @rbuf.size))
         until to_read == 0 || eof? || (rv.size > 0 && @chunked)
-          @socket.kgio_read(to_read, @buf) or eof!
-          filter_body(@rbuf, @buf)
+          filter_body(@rbuf, @socket.readpartial(to_read, @buf))
           rv << @rbuf
           to_read -= @rbuf.size
         end
@@ -61,6 +60,8 @@ def read(length = nil, rv = '')
       read_all(rv)
     end
     rv
+  rescue EOFError
+    return eof!
   end
 
   # :call-seq:
@@ -83,9 +84,10 @@ def gets
     begin
       @rbuf.sub!(re, '') and return $1
       return @rbuf.empty? ? nil : @rbuf.slice!(0, @rbuf.size) if eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
-      filter_body(once = '', @buf)
+      filter_body(once = '', @socket.readpartial(@@io_chunk_size, @buf))
       @rbuf << once
+    rescue EOFError
+      return eof!
     end while true
   end
 
@@ -107,14 +109,15 @@ def each
   def eof?
     if @parser.body_eof?
       while @chunked && ! @parser.parse
-        once = @socket.kgio_read(@@io_chunk_size) or eof!
-        @buf << once
+        @buf << @socket.readpartial(@@io_chunk_size)
       end
       @socket = nil
       true
     else
       false
     end
+  rescue EOFError
+    return eof!
   end
 
   def filter_body(dst, src)
@@ -127,10 +130,11 @@ def read_all(dst)
     dst.replace(@rbuf)
     @socket or return
     until eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
-      filter_body(@rbuf, @buf)
+      filter_body(@rbuf, @socket.readpartial(@@io_chunk_size, @buf))
       dst << @rbuf
     end
+  rescue EOFError
+    return eof!
   ensure
     @rbuf.clear
   end
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 5ddf379..ad5814c 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -65,7 +65,7 @@ def soft_kill(sig) # :nodoc:
     end
     # writing and reading 4 bytes on a pipe is atomic on all POSIX platforms
     # Do not care in the odd case the buffer is full, here.
-    @master.kgio_trywrite([signum].pack('l'))
+    @master.write_nonblock([signum].pack('l'), exception: false)
   rescue Errno::EPIPE
     # worker will be reaped soon
   end
@@ -73,7 +73,7 @@ def soft_kill(sig) # :nodoc:
   # this only runs when the Rack app.call is not running
   # act like a listener
   def kgio_tryaccept # :nodoc:
-    case buf = @to_io.kgio_tryread(4)
+    case buf = @to_io.read_nonblock(4, exception: false)
     when String
       # unpack the buffer and trigger the signal handler
       signum = buf.unpack('l')

[-- Attachment #3: 0002-kill-off-remaining-kgio-uses.patch --]
[-- Type: text/x-diff, Size: 25476 bytes --]

From 03ec6e69fc6219a40aa8db368abe53017cd164e3 Mon Sep 17 00:00:00 2001
From: Eric Wong <bofh@yhbt.net>
Date: Tue, 5 Sep 2023 06:43:20 +0000
Subject: [RFC 2/3] kill off remaining kgio uses

kgio is an extra download and shared object which costs users
bandwidth, disk space, startup time and memory.  Ruby 2.3+
provides `Socket#accept_nonblock(exception: false)' support
in addition to `exception: false' support in IO#*_nonblock
methods from Ruby 2.1.

We no longer distinguish between TCPServer and UNIXServer as
separate classes internally; instead favoring the `Socket' class
of Ruby for both.  This allows us to use `Socket#accept_nonblock'
and get a populated `Addrinfo' object off accept4(2)/accept(2)
without resorting to a getpeername(2) syscall (kgio avoided
getpeername(2) in the same way).

The downside is there's more Ruby-level argument passing and
stack usage on our end with HttpRequest#read_headers (formerly
HttpRequest#read).  I chose this tradeoff since advancements in
Ruby itself can theoretically mitigate the cost of argument
passing, while syscalls are a high fixed cost given modern CPU
vulnerability mitigations.

Note: no benchmarks have been run since I don't have a suitable
system.
---
 lib/unicorn.rb                  |  3 +-
 lib/unicorn/http_request.rb     | 14 +++----
 lib/unicorn/http_server.rb      | 30 +++++---------
 lib/unicorn/oob_gc.rb           |  4 +-
 lib/unicorn/socket_helper.rb    | 50 +++--------------------
 lib/unicorn/worker.rb           |  6 +--
 t/oob_gc.ru                     |  3 --
 t/oob_gc_path.ru                |  3 --
 test/unit/test_request.rb       | 47 ++++++++-------------
 test/unit/test_socket_helper.rb | 72 +++++++--------------------------
 test/unit/test_stream_input.rb  |  2 +-
 test/unit/test_tee_input.rb     |  2 +-
 unicorn.gemspec                 |  1 -
 13 files changed, 61 insertions(+), 176 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index b817b77..564cb30 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -1,7 +1,6 @@
 # -*- encoding: binary -*-
 require 'etc'
 require 'stringio'
-require 'kgio'
 require 'raindrops'
 require 'io/wait'
 
@@ -112,7 +111,7 @@ def self.log_error(logger, prefix, exc)
   F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/
 
   def self.pipe # :nodoc:
-    Kgio::Pipe.new.each do |io|
+    IO.pipe.each do |io|
       # shrink pipes to minimize impact on /proc/sys/fs/pipe-user-pages-soft
       # limits.
       if defined?(F_SETPIPE_SZ)
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 8bca60a..ab3bd6e 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -61,7 +61,7 @@ def self.check_client_connection=(bool)
   # returns an environment hash suitable for Rack if successful
   # This does minimal exception trapping and it is up to the caller
   # to handle any socket errors (e.g. user aborted upload).
-  def read(socket)
+  def read_headers(socket, ai)
     e = env
 
     # From https://www.ietf.org/rfc/rfc3875:
@@ -71,7 +71,7 @@ def read(socket)
     #  identify the client for the immediate request to the server;
     #  that client may be a proxy, gateway, or other intermediary
     #  acting on behalf of the actual source client."
-    e['REMOTE_ADDR'] = socket.kgio_addr
+    e['REMOTE_ADDR'] = ai.unix? ? '127.0.0.1' : ai.ip_address
 
     # short circuit the common case with small GET requests first
     socket.readpartial(16384, buf)
@@ -81,7 +81,7 @@ def read(socket)
       false until add_parse(socket.readpartial(16384))
     end
 
-    check_client_connection(socket) if @@check_client_connection
+    check_client_connection(socket, ai) if @@check_client_connection
 
     e['rack.input'] = 0 == content_length ?
                       NULL_IO : @@input_class.new(socket, self)
@@ -107,8 +107,8 @@ def hijacked?
   if Raindrops.const_defined?(:TCP_Info)
     TCPI = Raindrops::TCP_Info.allocate
 
-    def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket
+    def check_client_connection(socket, ai) # :nodoc:
+      if ai.ip?
         # Raindrops::TCP_Info#get!, #state (reads struct tcp_info#tcpi_state)
         raise Errno::EPIPE, "client closed connection".freeze,
               EMPTY_ARRAY if closed_state?(TCPI.get!(socket).state)
@@ -152,8 +152,8 @@ def closed_state?(state) # :nodoc:
     # Ruby 2.2+ can show struct tcp_info as a string Socket::Option#inspect.
     # Not that efficient, but probably still better than doing unnecessary
     # work after a client gives up.
-    def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket && @@tcpi_inspect_ok
+    def check_client_connection(socket, ai) # :nodoc:
+      if @@tcpi_inspect_ok && ai.ip?
         opt = socket.getsockopt(Socket::IPPROTO_TCP, Socket::TCP_INFO).inspect
         if opt =~ /\bstate=(\S+)/
           raise Errno::EPIPE, "client closed connection".freeze,
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 2f1eb1b..ed5bbf1 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -111,9 +111,7 @@ def initialize(app, options = {})
 
     @worker_data = if worker_data = ENV['UNICORN_WORKER']
       worker_data = worker_data.split(',').map!(&:to_i)
-      worker_data[1] = worker_data.slice!(1..2).map do |i|
-        Kgio::Pipe.for_fd(i)
-      end
+      worker_data[1] = worker_data.slice!(1..2).map { |i| IO.for_fd(i) }
       worker_data
     end
   end
@@ -243,10 +241,6 @@ def listen(address, opt = {}.merge(listener_opts[address] || {}))
     tries = opt[:tries] || 5
     begin
       io = bind_listen(address, opt)
-      unless Kgio::TCPServer === io || Kgio::UNIXServer === io
-        io.autoclose = false
-        io = server_cast(io)
-      end
       logger.info "listening on addr=#{sock_name(io)} fd=#{io.fileno}"
       LISTENERS << io
       io
@@ -594,9 +588,9 @@ def e100_response_write(client, env)
 
   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
-  def process_client(client)
+  def process_client(client, ai)
     @request = Unicorn::HttpRequest.new
-    env = @request.read(client)
+    env = @request.read_headers(client, ai)
 
     if early_hints
       env["rack.early_hints"] = lambda do |headers|
@@ -708,10 +702,9 @@ def worker_loop(worker)
       reopen = reopen_worker_logs(worker.nr) if reopen
       worker.tick = time_now.to_i
       while sock = ready.shift
-        # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
-        # but that will return false
-        if client = sock.kgio_tryaccept
-          process_client(client)
+        client_ai = sock.accept_nonblock(exception: false)
+        if client_ai != :wait_readable
+          process_client(*client_ai)
           worker.tick = time_now.to_i
         end
         break if reopen
@@ -809,7 +802,6 @@ def redirect_io(io, path)
 
   def inherit_listeners!
     # inherit sockets from parents, they need to be plain Socket objects
-    # before they become Kgio::UNIXServer or Kgio::TCPServer
     inherited = ENV['UNICORN_FD'].to_s.split(',')
     immortal = []
 
@@ -825,8 +817,6 @@ def inherit_listeners!
     inherited.map! do |fd|
       io = Socket.for_fd(fd.to_i)
       @immortal << io if immortal.include?(fd)
-      io.autoclose = false
-      io = server_cast(io)
       set_server_sockopt(io, listener_opts[sock_name(io)])
       logger.info "inherited addr=#{sock_name(io)} fd=#{io.fileno}"
       io
@@ -835,11 +825,9 @@ def inherit_listeners!
     config_listeners = config[:listeners].dup
     LISTENERS.replace(inherited)
 
-    # we start out with generic Socket objects that get cast to either
-    # Kgio::TCPServer or Kgio::UNIXServer objects; but since the Socket
-    # objects share the same OS-level file descriptor as the higher-level
-    # *Server objects; we need to prevent Socket objects from being
-    # garbage-collected
+    # we only use generic Socket objects for aggregate Socket#accept_nonblock
+    # return value [ Socket, Addrinfo ].  This allows us to avoid having to
+    # make getpeername(2) syscalls later on to fill in env['REMOTE_ADDR']
     config_listeners -= listener_names
     if config_listeners.empty? && LISTENERS.empty?
       config_listeners << Unicorn::Const::DEFAULT_LISTEN
diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb
index 91a8e51..db9f2cb 100644
--- a/lib/unicorn/oob_gc.rb
+++ b/lib/unicorn/oob_gc.rb
@@ -65,8 +65,8 @@ def self.new(app, interval = 5, path = %r{\A/})
   end
 
   #:stopdoc:
-  def process_client(client)
-    super(client) # Unicorn::HttpServer#process_client
+  def process_client(*args)
+    super(*args) # Unicorn::HttpServer#process_client
     env = instance_variable_get(:@request).env
     if OOBGC_PATH =~ env['PATH_INFO'] && ((@@nr -= 1) <= 0)
       @@nr = OOBGC_INTERVAL
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index c2ba75e..06ec2b2 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -3,32 +3,6 @@
 require 'socket'
 
 module Unicorn
-
-  # Instead of using a generic Kgio::Socket for everything,
-  # tag TCP sockets so we can use TCP_INFO under Linux without
-  # incurring extra syscalls for Unix domain sockets.
-  # TODO: remove these when we remove kgio
-  TCPClient = Class.new(Kgio::Socket) # :nodoc:
-  class TCPSrv < Kgio::TCPServer # :nodoc:
-    def kgio_tryaccept # :nodoc:
-      super(TCPClient)
-    end
-  end
-
-  if IO.instance_method(:write).arity == 1 # Ruby <= 2.4
-    require 'unicorn/write_splat'
-    UNIXClient = Class.new(Kgio::Socket) # :nodoc:
-    class UNIXSrv < Kgio::UNIXServer # :nodoc:
-      include Unicorn::WriteSplat
-      def kgio_tryaccept # :nodoc:
-        super(UNIXClient)
-      end
-    end
-    TCPClient.__send__(:include, Unicorn::WriteSplat)
-  else # Ruby 2.5+
-    UNIXSrv = Kgio::UNIXServer
-  end
-
   module SocketHelper
 
     # internal interface
@@ -105,7 +79,7 @@ def set_tcp_sockopt(sock, opt)
     def set_server_sockopt(sock, opt)
       opt = DEFAULTS.merge(opt || {})
 
-      TCPSocket === sock and set_tcp_sockopt(sock, opt)
+      set_tcp_sockopt(sock, opt) if sock.local_address.ip?
 
       rcvbuf, sndbuf = opt.values_at(:rcvbuf, :sndbuf)
       if rcvbuf || sndbuf
@@ -149,7 +123,9 @@ def bind_listen(address = '0.0.0.0:8080', opt = {})
         end
         old_umask = File.umask(opt[:umask] || 0)
         begin
-          UNIXSrv.new(address)
+          s = Socket.new(:UNIX, :STREAM)
+          s.bind(Socket.sockaddr_un(address))
+          s
         ensure
           File.umask(old_umask)
         end
@@ -177,8 +153,7 @@ def new_tcp_server(addr, port, opt)
         sock.setsockopt(:SOL_SOCKET, :SO_REUSEPORT, 1)
       end
       sock.bind(Socket.pack_sockaddr_in(port, addr))
-      sock.autoclose = false
-      TCPSrv.for_fd(sock.fileno)
+      sock
     end
 
     # returns rfc2732-style (e.g. "[::1]:666") addresses for IPv6
@@ -194,10 +169,6 @@ def tcp_name(sock)
     def sock_name(sock)
       case sock
       when String then sock
-      when UNIXServer
-        Socket.unpack_sockaddr_un(sock.getsockname)
-      when TCPServer
-        tcp_name(sock)
       when Socket
         begin
           tcp_name(sock)
@@ -210,16 +181,5 @@ def sock_name(sock)
     end
 
     module_function :sock_name
-
-    # casts a given Socket to be a TCPServer or UNIXServer
-    def server_cast(sock)
-      begin
-        Socket.unpack_sockaddr_in(sock.getsockname)
-        TCPSrv.for_fd(sock.fileno)
-      rescue ArgumentError
-        UNIXSrv.for_fd(sock.fileno)
-      end
-    end
-
   end # module SocketHelper
 end # module Unicorn
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index ad5814c..4af31be 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -71,8 +71,8 @@ def soft_kill(sig) # :nodoc:
   end
 
   # this only runs when the Rack app.call is not running
-  # act like a listener
-  def kgio_tryaccept # :nodoc:
+  # act like Socket#accept_nonblock(exception: false)
+  def accept_nonblock(*_unused) # :nodoc:
     case buf = @to_io.read_nonblock(4, exception: false)
     when String
       # unpack the buffer and trigger the signal handler
@@ -82,7 +82,7 @@ def kgio_tryaccept # :nodoc:
     when nil # EOF: master died, but we are at a safe place to exit
       fake_sig(:QUIT)
     when :wait_readable # keep waiting
-      return false
+      return :wait_readable
     end while true # loop, as multiple signals may be sent
   end
 
diff --git a/t/oob_gc.ru b/t/oob_gc.ru
index c253540..224cb06 100644
--- a/t/oob_gc.ru
+++ b/t/oob_gc.ru
@@ -7,9 +7,6 @@
 
 # Mock GC.start
 def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
-    x.closed? or abort "not closed #{x}"
-  end
   $gc_started = true
 end
 run lambda { |env|
diff --git a/t/oob_gc_path.ru b/t/oob_gc_path.ru
index af8e3b9..7f40601 100644
--- a/t/oob_gc_path.ru
+++ b/t/oob_gc_path.ru
@@ -7,9 +7,6 @@
 
 # Mock GC.start
 def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
-    x.closed? or abort "not closed #{x}"
-  end
   $gc_started = true
 end
 run lambda { |env|
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index 7f22b24..53ae944 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -10,14 +10,9 @@
 
 class RequestTest < Test::Unit::TestCase
 
-  class MockRequest < StringIO
-    alias_method :readpartial, :sysread
-    alias_method :kgio_read!, :sysread
-    alias_method :read_nonblock, :sysread
-    def kgio_addr
-      '127.0.0.1'
-    end
-  end
+  MockRequest = Class.new(StringIO)
+
+  AI = Addrinfo.new(Socket.sockaddr_un('/unicorn/sucks'))
 
   def setup
     @request = HttpRequest.new
@@ -30,7 +25,7 @@ def setup
   def test_options
     client = MockRequest.new("OPTIONS * HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal '', env['REQUEST_PATH']
     assert_equal '', env['PATH_INFO']
     assert_equal '*', env['REQUEST_URI']
@@ -40,7 +35,7 @@ def test_options
   def test_absolute_uri_with_query
     client = MockRequest.new("GET http://e:3/x?y=z HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'y=z', env['QUERY_STRING']
@@ -50,7 +45,7 @@ def test_absolute_uri_with_query
   def test_absolute_uri_with_fragment
     client = MockRequest.new("GET http://e:3/x#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal '', env['QUERY_STRING']
@@ -61,7 +56,7 @@ def test_absolute_uri_with_fragment
   def test_absolute_uri_with_query_and_fragment
     client = MockRequest.new("GET http://e:3/x?a=b#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'a=b', env['QUERY_STRING']
@@ -73,7 +68,7 @@ def test_absolute_uri_unsupported_schemes
     %w(ssh+http://e/ ftp://e/x http+ssh://e/x).each do |abs_uri|
       client = MockRequest.new("GET #{abs_uri} HTTP/1.1\r\n" \
                                "Host: foo\r\n\r\n")
-      assert_raises(HttpParserError) { @request.read(client) }
+      assert_raises(HttpParserError) { @request.read_headers(client, AI) }
     end
   end
 
@@ -81,7 +76,7 @@ def test_x_forwarded_proto_https
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: https\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal "https", env['rack.url_scheme']
     assert_kind_of Array, @lint.call(env)
   end
@@ -90,7 +85,7 @@ def test_x_forwarded_proto_http
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: http\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal "http", env['rack.url_scheme']
     assert_kind_of Array, @lint.call(env)
   end
@@ -99,14 +94,14 @@ def test_x_forwarded_proto_invalid
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: ftp\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal "http", env['rack.url_scheme']
     assert_kind_of Array, @lint.call(env)
   end
 
   def test_rack_lint_get
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal "http", env['rack.url_scheme']
     assert_equal '127.0.0.1', env['REMOTE_ADDR']
     assert_kind_of Array, @lint.call(env)
@@ -114,7 +109,7 @@ def test_rack_lint_get
 
   def test_no_content_stringio
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal StringIO, env['rack.input'].class
   end
 
@@ -122,7 +117,7 @@ def test_zero_content_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 0\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal StringIO, env['rack.input'].class
   end
 
@@ -130,7 +125,7 @@ def test_real_content_not_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert_equal Unicorn::TeeInput, env['rack.input'].class
   end
 
@@ -141,7 +136,7 @@ def test_rack_lint_put
       "Content-Length: 5\r\n" \
       "\r\n" \
       "abcde")
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert ! env.include?(:http_body)
     assert_kind_of Array, @lint.call(env)
   end
@@ -152,14 +147,6 @@ def test_rack_lint_big_put
     buf = (' ' * bs).freeze
     length = bs * count
     client = Tempfile.new('big_put')
-    def client.kgio_addr; '127.0.0.1'; end
-    def client.kgio_read(*args)
-      readpartial(*args)
-    rescue EOFError
-    end
-    def client.kgio_read!(*args)
-      readpartial(*args)
-    end
     client.syswrite(
       "PUT / HTTP/1.1\r\n" \
       "Host: foo\r\n" \
@@ -167,7 +154,7 @@ def client.kgio_read!(*args)
       "\r\n")
     count.times { assert_equal bs, client.syswrite(buf) }
     assert_equal 0, client.sysseek(0)
-    env = @request.read(client)
+    env = @request.read_headers(client, AI)
     assert ! env.include?(:http_body)
     assert_equal length, env['rack.input'].size
     count.times {
diff --git a/test/unit/test_socket_helper.rb b/test/unit/test_socket_helper.rb
index 62d5a3a..a446f06 100644
--- a/test/unit/test_socket_helper.rb
+++ b/test/unit/test_socket_helper.rb
@@ -24,7 +24,8 @@ def test_bind_listen_tcp
     port = unused_port @test_addr
     @tcp_listener_name = "#@test_addr:#{port}"
     @tcp_listener = bind_listen(@tcp_listener_name)
-    assert TCPServer === @tcp_listener
+    assert Socket === @tcp_listener
+    assert @tcp_listener.local_address.ip?
     assert_equal @tcp_listener_name, sock_name(@tcp_listener)
   end
 
@@ -38,10 +39,10 @@ def test_bind_listen_options
       { :backlog => 16, :rcvbuf => 4096, :sndbuf => 4096 }
     ].each do |opts|
       tcp_listener = bind_listen(tcp_listener_name, opts)
-      assert TCPServer === tcp_listener
+      assert tcp_listener.local_address.ip?
       tcp_listener.close
       unix_listener = bind_listen(unix_listener_name, opts)
-      assert UNIXServer === unix_listener
+      assert unix_listener.local_address.unix?
       unix_listener.close
     end
   end
@@ -52,11 +53,13 @@ def test_bind_listen_unix
     @unix_listener_path = tmp.path
     File.unlink(@unix_listener_path)
     @unix_listener = bind_listen(@unix_listener_path)
-    assert UNIXServer === @unix_listener
+    assert Socket === @unix_listener
+    assert @unix_listener.local_address.unix?
     assert_equal @unix_listener_path, sock_name(@unix_listener)
     assert File.readable?(@unix_listener_path), "not readable"
     assert File.writable?(@unix_listener_path), "not writable"
     assert_equal 0777, File.umask
+    assert_equal @unix_listener, bind_listen(@unix_listener)
   ensure
     File.umask(old_umask)
   end
@@ -67,7 +70,6 @@ def test_bind_listen_unix_umask
     @unix_listener_path = tmp.path
     File.unlink(@unix_listener_path)
     @unix_listener = bind_listen(@unix_listener_path, :umask => 077)
-    assert UNIXServer === @unix_listener
     assert_equal @unix_listener_path, sock_name(@unix_listener)
     assert_equal 0140700, File.stat(@unix_listener_path).mode
     assert_equal 0777, File.umask
@@ -75,28 +77,6 @@ def test_bind_listen_unix_umask
     File.umask(old_umask)
   end
 
-  def test_bind_listen_unix_idempotent
-    test_bind_listen_unix
-    a = bind_listen(@unix_listener)
-    assert_equal a.fileno, @unix_listener.fileno
-    unix_server = server_cast(@unix_listener)
-    assert UNIXServer === unix_server
-    a = bind_listen(unix_server)
-    assert_equal a.fileno, unix_server.fileno
-    assert_equal a.fileno, @unix_listener.fileno
-  end
-
-  def test_bind_listen_tcp_idempotent
-    test_bind_listen_tcp
-    a = bind_listen(@tcp_listener)
-    assert_equal a.fileno, @tcp_listener.fileno
-    tcp_server = server_cast(@tcp_listener)
-    assert TCPServer === tcp_server
-    a = bind_listen(tcp_server)
-    assert_equal a.fileno, tcp_server.fileno
-    assert_equal a.fileno, @tcp_listener.fileno
-  end
-
   def test_bind_listen_unix_rebind
     test_bind_listen_unix
     new_listener = nil
@@ -107,14 +87,18 @@ def test_bind_listen_unix_rebind
     File.unlink(@unix_listener_path)
     new_listener = bind_listen(@unix_listener_path)
 
-    assert UNIXServer === new_listener
     assert new_listener.fileno != @unix_listener.fileno
     assert_equal sock_name(new_listener), sock_name(@unix_listener)
     assert_equal @unix_listener_path, sock_name(new_listener)
     pid = fork do
-      client = server_cast(new_listener).accept
-      client.syswrite('abcde')
-      exit 0
+      begin
+        client, _ = new_listener.accept
+        client.syswrite('abcde')
+        exit 0
+      rescue => e
+        warn "#{e.message} (#{e.class})"
+        exit 1
+      end
     end
     s = unix_socket(@unix_listener_path)
     IO.select([s])
@@ -123,32 +107,6 @@ def test_bind_listen_unix_rebind
     assert status.success?
   end
 
-  def test_server_cast
-    test_bind_listen_unix
-    test_bind_listen_tcp
-    unix_listener_socket = Socket.for_fd(@unix_listener.fileno)
-    assert Socket === unix_listener_socket
-    @unix_server = server_cast(unix_listener_socket)
-    assert_equal @unix_listener.fileno, @unix_server.fileno
-    assert UNIXServer === @unix_server
-    assert_equal(@unix_server.path, @unix_listener.path,
-                 "##{@unix_server.path} != #{@unix_listener.path}")
-    assert File.socket?(@unix_server.path)
-    assert_equal @unix_listener_path, sock_name(@unix_server)
-
-    tcp_listener_socket = Socket.for_fd(@tcp_listener.fileno)
-    assert Socket === tcp_listener_socket
-    @tcp_server = server_cast(tcp_listener_socket)
-    assert_equal @tcp_listener.fileno, @tcp_server.fileno
-    assert TCPServer === @tcp_server
-    assert_equal @tcp_listener_name, sock_name(@tcp_server)
-  end
-
-  def test_sock_name
-    test_server_cast
-    sock_name(@unix_server)
-  end
-
   def test_tcp_defer_accept_default
     return unless defined?(TCP_DEFER_ACCEPT)
     port = unused_port @test_addr
diff --git a/test/unit/test_stream_input.rb b/test/unit/test_stream_input.rb
index 2a14135..7986ca7 100644
--- a/test/unit/test_stream_input.rb
+++ b/test/unit/test_stream_input.rb
@@ -9,7 +9,7 @@ def setup
     @rs = "\n"
     $/ == "\n" or abort %q{test broken if \$/ != "\\n"}
     @env = {}
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
     @rd.sync = @wr.sync = true
     @start_pid = $$
   end
diff --git a/test/unit/test_tee_input.rb b/test/unit/test_tee_input.rb
index 6f5bc8a..607ce87 100644
--- a/test/unit/test_tee_input.rb
+++ b/test/unit/test_tee_input.rb
@@ -10,7 +10,7 @@ class TeeInput < Unicorn::TeeInput
 
 class TestTeeInput < Test::Unit::TestCase
   def setup
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
     @rd.sync = @wr.sync = true
     @start_pid = $$
     @rs = "\n"
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 7bb1154..85183d9 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -36,7 +36,6 @@
   # won't have descriptive text, only the numeric status.
   s.add_development_dependency(%q<rack>)
 
-  s.add_dependency(%q<kgio>, '~> 2.6')
   s.add_dependency(%q<raindrops>, '~> 0.7')
 
   s.add_development_dependency('test-unit', '~> 3.0')

[-- Attachment #4: 0003-update-dependency-to-Ruby-2.5.patch --]
[-- Type: text/x-diff, Size: 3257 bytes --]

From c67ebf96edc8ca691dfc556d4813fed242fe77ca Mon Sep 17 00:00:00 2001
From: Eric Wong <bofh@yhbt.net>
Date: Tue, 5 Sep 2023 09:14:11 +0000
Subject: [RFC 3/3] update dependency to Ruby 2.5+

We actually need Ruby 2.3+ for `accept_nonblock(exception: false)';
and (AFAIK) we can't easily use a subclass of `Socket' while using
Socket#accept_nonblock to inject WriteSplat support for
`IO#write(*multi_args)'

So just depend on Ruby 2.5+ since all my Ruby is already on the
already-ancient Ruby 2.7+ anyways.
---
 HACKING                    | 2 +-
 README                     | 2 +-
 lib/unicorn/write_splat.rb | 7 -------
 t/README                   | 2 +-
 unicorn.gemspec            | 4 ++--
 5 files changed, 5 insertions(+), 12 deletions(-)
 delete mode 100644 lib/unicorn/write_splat.rb

diff --git a/HACKING b/HACKING
index 020209e..5aca83e 100644
--- a/HACKING
+++ b/HACKING
@@ -60,7 +60,7 @@ becomes unavailable.
 
 === Ruby/C Compatibility
 
-We target C Ruby 2.0 and later.  We need the Ruby
+We target C Ruby 2.5 and later.  We need the Ruby
 implementation to support fork, exec, pipe, UNIX signals, access to
 integer file descriptors and ability to use unlinked files.
 
diff --git a/README b/README
index 5411003..7e29daf 100644
--- a/README
+++ b/README
@@ -12,7 +12,7 @@ both the the request and response in between unicorn and slow clients.
   cut out everything that is better supported by the operating system,
   {nginx}[https://nginx.org/] or {Rack}[https://rack.github.io/].
 
-* Compatible with Ruby 2.0.0 and later.
+* Compatible with Ruby 2.5 and later.
 
 * Process management: unicorn will reap and restart workers that
   die from broken apps.  There is no need to manage multiple processes
diff --git a/lib/unicorn/write_splat.rb b/lib/unicorn/write_splat.rb
deleted file mode 100644
index 7e6e363..0000000
--- a/lib/unicorn/write_splat.rb
+++ /dev/null
@@ -1,7 +0,0 @@
-# -*- encoding: binary -*-
-# compatibility module for Ruby <= 2.4, remove when we go Ruby 2.5+
-module Unicorn::WriteSplat # :nodoc:
-  def write(*arg) # :nodoc:
-    super(arg.join(''))
-  end
-end
diff --git a/t/README b/t/README
index d09c715..7bd093d 100644
--- a/t/README
+++ b/t/README
@@ -14,7 +14,7 @@ Old tests are in Bourne shell and slowly being ported to Perl 5.
 
 == Requirements
 
-* {Ruby 2.0.0+}[https://www.ruby-lang.org/en/]
+* {Ruby 2.5.0+}[https://www.ruby-lang.org/en/]
 * {Perl 5.14+}[https://www.perl.org/] # your distro should have it
 * {GNU make}[https://www.gnu.org/software/make/]
 * {curl}[https://curl.haxx.se/]
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 85183d9..e7e3ef7 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -25,11 +25,11 @@
   s.homepage = 'https://yhbt.net/unicorn/'
   s.test_files = test_files
 
-  # 2.0.0 is the minimum supported version. We don't specify
+  # 2.5.0 is the minimum supported version. We don't specify
   # a maximum version to make it easier to test pre-releases,
   # but we do warn users if they install unicorn on an untested
   # version in extconf.rb
-  s.required_ruby_version = ">= 2.0.0"
+  s.required_ruby_version = ">= 2.5.0"
 
   # We do not have a hard dependency on rack, it's possible to load
   # things which respond to #call.  HTTP status lines in responses

^ permalink raw reply related	[relevance 1%]

* [PATCH 00-23/23] start porting tests to Perl5
@ 2023-06-05 10:32  3% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2023-06-05 10:32 UTC (permalink / raw)
  To: unicorn-public

[-- Attachment #1: Type: text/plain, Size: 4179 bytes --]

Still a lot more work to do, but at least socat is no longer a
test dependency.  Perl5 is installed on far more systems than
socat.

Ruby introduces breaking changes every year and I can't trust
tests to work as they were originally intended, anymore.
Perl 5 doesn't have perfect backwards compatibility, either; but
it's the least bad of any widely-installed scripting language.

Note: that 23/23 introduces a subtle bugfix which changes
behavior for systemd users

Patches are attached to reduce load on SMTP servers.
Some more patches to come as I deal with Ruby 3.x deprecation
warnings :<

Eric Wong (23):
  switch unit/test_response.rb to Perl 5 integration test
  support rack 3 multi-value headers
  port t0018-write-on-close.sh to Perl 5
  port t0000-http-basic.sh to Perl 5
  port t0002-parser-error.sh to Perl 5
  t/integration.t: use start_req to simplify test slighly
  port t0011-active-unix-socket.sh to Perl 5
  port t0100-rack-input-tests.sh to Perl 5
  tests: use autodie to simplify error checking
  port t0019-max_header_len.sh to Perl 5
  test_exec: drop sd_listen_fds emulation test
  test_exec: drop test_basic and test_config_ru_alt_path
  tests: check_stderr consistently in Perl 5 tests
  tests: consistent tcp_start and unix_start across Perl 5 tests
  port t9000-preread-input.sh to Perl 5
  port t/t0116-client_body_buffer_size.sh to Perl 5
  tests: get rid of sha1sum.rb and rsha1() sh function
  early_hints supports Rack 3 array headers
  test_server: drop early_hints test
  t/integration.t: switch PUT tests to MD5, reuse buffers
  tests: move test_upload.rb tests to t/integration.t
  drop redundant IO#close_on_exec=false calls
  LISTEN_FDS-inherited sockets are immortal across SIGHUP

 GNUmakefile                                |   7 +-
 lib/unicorn/http_server.rb                 |  12 +-
 t/README                                   |  21 +-
 t/active-unix-socket.t                     | 113 +++++++
 t/bin/content-md5-put                      |  36 ---
 t/bin/sha1sum.rb                           |  17 --
 t/{t0116.ru => client_body_buffer_size.ru} |   2 -
 t/client_body_buffer_size.t                |  82 ++++++
 t/integration.ru                           | 114 +++++++
 t/integration.t                            | 326 +++++++++++++++++++++
 t/lib.perl                                 | 217 ++++++++++++++
 t/preread_input.ru                         |  21 +-
 t/rack-input-tests.ru                      |  21 --
 t/t0000-http-basic.sh                      |  50 ----
 t/t0002-parser-error.sh                    |  94 ------
 t/t0011-active-unix-socket.sh              |  79 -----
 t/t0018-write-on-close.sh                  |  23 --
 t/t0019-max_header_len.sh                  |  49 ----
 t/t0100-rack-input-tests.sh                | 124 --------
 t/t0116-client_body_buffer_size.sh         |  80 -----
 t/t9000-preread-input.sh                   |  48 ---
 t/test-lib.sh                              |   4 -
 t/write-on-close.ru                        |  11 -
 test/exec/test_exec.rb                     |  57 ----
 test/unit/test_response.rb                 | 111 -------
 test/unit/test_server.rb                   |  31 --
 test/unit/test_upload.rb                   | 301 -------------------
 27 files changed, 891 insertions(+), 1160 deletions(-)
 create mode 100644 t/active-unix-socket.t
 delete mode 100755 t/bin/content-md5-put
 delete mode 100755 t/bin/sha1sum.rb
 rename t/{t0116.ru => client_body_buffer_size.ru} (82%)
 create mode 100644 t/client_body_buffer_size.t
 create mode 100644 t/integration.ru
 create mode 100644 t/integration.t
 create mode 100644 t/lib.perl
 delete mode 100644 t/rack-input-tests.ru
 delete mode 100755 t/t0000-http-basic.sh
 delete mode 100755 t/t0002-parser-error.sh
 delete mode 100755 t/t0011-active-unix-socket.sh
 delete mode 100755 t/t0018-write-on-close.sh
 delete mode 100755 t/t0019-max_header_len.sh
 delete mode 100755 t/t0100-rack-input-tests.sh
 delete mode 100755 t/t0116-client_body_buffer_size.sh
 delete mode 100755 t/t9000-preread-input.sh
 delete mode 100644 t/write-on-close.ru
 delete mode 100644 test/unit/test_response.rb
 delete mode 100644 test/unit/test_upload.rb

[-- Attachment #2: 0001-switch-unit-test_response.rb-to-Perl-5-integration-t.patch --]
[-- Type: text/x-diff, Size: 15667 bytes --]

From 086e397abc0126556af24df77a976671294df2ee Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:30 +0000
Subject: [PATCH 01/23] switch unit/test_response.rb to Perl 5 integration test

http_response_write may benefit from API changes for Rack 3
support.

Since there's no benefit I can see from using a unit test,
switch to an integration test to avoid having to maintain the
unit test if our internal http_response_write method changes.

Of course, I can't trust tests written in Ruby since I've had to
put up with a constant stream of incompatibilities over the past
two decades :<   Perl is more widely installed than socat[1], and
nearly all the Perl I wrote 20 years ago still works
unmodified today.

[1] the rarest dependency of the Bourne shell integration tests
---
 GNUmakefile                |   5 +-
 t/README                   |  24 +++--
 t/integration.ru           |  38 ++++++++
 t/integration.t            |  64 +++++++++++++
 t/lib.perl                 | 189 +++++++++++++++++++++++++++++++++++++
 test/unit/test_response.rb | 111 ----------------------
 6 files changed, 313 insertions(+), 118 deletions(-)
 create mode 100644 t/integration.ru
 create mode 100644 t/integration.t
 create mode 100644 t/lib.perl
 delete mode 100644 test/unit/test_response.rb

diff --git a/GNUmakefile b/GNUmakefile
index 0e08ef0..5cca189 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -86,7 +86,7 @@ $(tmp_bin)/%: bin/% | $(tmp_bin)
 bins: $(tmp_bins)
 
 t_log := $(T_log) $(T_n_log)
-test: $(T) $(T_n)
+test: $(T) $(T_n) test-prove
 	@cat $(t_log) | $(MRI) test/aggregate.rb
 	@$(RM) $(t_log)
 
@@ -141,6 +141,9 @@ t/random_blob:
 
 test-integration: $(T_sh)
 
+test-prove:
+	prove -vw
+
 check: test-require test test-integration
 test-all: check
 
diff --git a/t/README b/t/README
index 14de559..8a5243e 100644
--- a/t/README
+++ b/t/README
@@ -5,16 +5,24 @@ TCP ports or Unix domain sockets.  They're all designed to run
 concurrently with other tests to minimize test time, but tests may be
 run independently as well.
 
-We write our tests in Bourne shell because that's what we're
-comfortable writing integration tests with.
+New tests are written in Perl 5 because we need a stable language
+to test real-world behavior and Ruby introduces incompatibilities
+at a far faster rate than Perl 5.  Perl is Ruby's older cousin, so
+it should be easy-to-learn for Rubyists.
+
+Old tests are in Bourne shell, but the socat(1) dependency was probably
+too rare compared to Perl 5.
 
 == Requirements
 
-* {Ruby 2.0.0+}[https://www.ruby-lang.org/en/] (duh!)
+* {Ruby 2.0.0+}[https://www.ruby-lang.org/en/]
+* {Perl 5.14+}[https://www.perl.org/] # your distro should have it
 * {GNU make}[https://www.gnu.org/software/make/]
+
+The following requirements will eventually be dropped.
+
 * {socat}[http://www.dest-unreach.org/socat/]
 * {curl}[https://curl.haxx.se/]
-* standard UNIX shell utilities (Bourne sh, awk, sed, grep, ...)
 
 We do not use bashisms or any non-portable, non-POSIX constructs
 in our shell code.  We use the "pipefail" option if available and
@@ -26,9 +34,13 @@ with {dash}[http://gondor.apana.org.au/~herbert/dash/] and
 
 To run the entire test suite with 8 tests running at once:
 
-  make -j8
+  make -j8 && prove -vw
+
+To run one individual test (Perl5):
+
+  prove -vw t/integration.t
 
-To run one individual test:
+To run one individual test (shell):
 
   make t0000-simple-http.sh
 
diff --git a/t/integration.ru b/t/integration.ru
new file mode 100644
index 0000000..6ef873c
--- /dev/null
+++ b/t/integration.ru
@@ -0,0 +1,38 @@
+#!ruby
+# Copyright (C) unicorn hackers <unicorn-public@80x24.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+
+# this goes for t/integration.t  We'll try to put as many tests
+# in here as possible to avoid startup overhead of Ruby.
+
+$orig_rack_200 = nil
+def tweak_status_code
+  $orig_rack_200 = Rack::Utils::HTTP_STATUS_CODES[200]
+  Rack::Utils::HTTP_STATUS_CODES[200] = "HI"
+  [ 200, {}, [] ]
+end
+
+def restore_status_code
+  $orig_rack_200 or return [ 500, {}, [] ]
+  Rack::Utils::HTTP_STATUS_CODES[200] = $orig_rack_200
+  [ 200, {}, [] ]
+end
+
+run(lambda do |env|
+  case env['REQUEST_METHOD']
+  when 'GET'
+    case env['PATH_INFO']
+    when '/rack-2-newline-headers'; [ 200, { 'X-R2' => "a\nb\nc" }, [] ]
+    when '/nil-header-value'; [ 200, { 'X-Nil' => nil }, [] ]
+    when '/unknown-status-pass-through'; [ '666 I AM THE BEAST', {}, [] ]
+    end # case PATH_INFO (GET)
+  when 'POST'
+    case env['PATH_INFO']
+    when '/tweak-status-code'; tweak_status_code
+    when '/restore-status-code'; restore_status_code
+    end # case PATH_INFO (POST)
+    # ...
+  when 'PUT'
+    # ...
+  end # case REQUEST_METHOD
+end) # run
diff --git a/t/integration.t b/t/integration.t
new file mode 100644
index 0000000..5569155
--- /dev/null
+++ b/t/integration.t
@@ -0,0 +1,64 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+
+use v5.14; BEGIN { require './t/lib.perl' };
+my $srv = tcp_server();
+my $t0 = time;
+my $ar = unicorn(qw(-E none t/integration.ru), { 3 => $srv });
+
+sub slurp_hdr {
+	my ($c) = @_;
+	local $/ = "\r\n\r\n"; # affects both readline+chomp
+	chomp(my $hdr = readline($c));
+	my ($status, @hdr) = split(/\r\n/, $hdr);
+	diag explain([ $status, \@hdr ]) if $ENV{V};
+	($status, \@hdr);
+}
+
+my ($c, $status, $hdr);
+
+# response header tests
+$c = tcp_connect($srv);
+print $c "GET /rack-2-newline-headers HTTP/1.0\r\n\r\n" or die $!;
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+my $orig_200_status = $status;
+is_deeply([ grep(/^X-R2: /, @$hdr) ],
+	[ 'X-R2: a', 'X-R2: b', 'X-R2: c' ],
+	'rack 2 LF-delimited headers supported') or diag(explain($hdr));
+
+SKIP: { # Date header check
+	my @d = grep(/^Date: /i, @$hdr);
+	is(scalar(@d), 1, 'got one date header') or diag(explain(\@d));
+	eval { require HTTP::Date } or skip "HTTP::Date missing: $@", 1;
+	$d[0] =~ s/^Date: //i or die 'BUG: did not strip date: prefix';
+	my $t = HTTP::Date::str2time($d[0]);
+	ok($t >= $t0 && $t > 0 && $t <= time, 'valid date') or
+		diag(explain([$t, $!, \@d]));
+};
+
+# cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
+$c = tcp_connect($srv);
+print $c "GET /nil-header-value HTTP/1.0\r\n\r\n" or die $!;
+($status, $hdr) = slurp_hdr($c);
+is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
+	'nil header value accepted for broken apps') or diag(explain($hdr));
+
+if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
+	$c = tcp_connect($srv);
+	print $c "POST /tweak-status-code HTTP/1.0\r\n\r\n" or die $!;
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 200 HI\b!, 'status tweaked');
+
+	$c = tcp_connect($srv);
+	print $c "POST /restore-status-code HTTP/1.0\r\n\r\n" or die $!;
+	($status, $hdr) = slurp_hdr($c);
+	is($status, $orig_200_status, 'original status restored');
+}
+
+
+# ... more stuff here
+undef $ar;
+diag slurp("$tmpdir/err.log") if $ENV{V};
+done_testing;
diff --git a/t/lib.perl b/t/lib.perl
new file mode 100644
index 0000000..dd9c6b7
--- /dev/null
+++ b/t/lib.perl
@@ -0,0 +1,189 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@80x24.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+package UnicornTest;
+use v5.14;
+use parent qw(Exporter);
+use Test::More;
+use IO::Socket::INET;
+use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
+use File::Temp 0.19 (); # 0.19 for ->newdir
+our ($tmpdir, $errfh);
+our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
+	SEEK_SET);
+
+my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
+$tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
+open($errfh, '>>', "$tmpdir/err.log") or die "open: $!";
+
+sub tcp_server {
+	my %opt = (
+		ReuseAddr => 1,
+		Proto => 'tcp',
+		Type => SOCK_STREAM,
+		Listen => SOMAXCONN,
+		Blocking => 0,
+		@_,
+	);
+	eval {
+		die 'IPv4-only' if $ENV{TEST_IPV4_ONLY};
+		require IO::Socket::INET6;
+		IO::Socket::INET6->new(%opt, LocalAddr => '[::1]')
+	} || eval {
+		die 'IPv6-only' if $ENV{TEST_IPV6_ONLY};
+		IO::Socket::INET->new(%opt, LocalAddr => '127.0.0.1')
+	} || BAIL_OUT "failed to create TCP server: $! ($@)";
+}
+
+sub tcp_host_port {
+	my ($s) = @_;
+	my ($h, $p) = ($s->sockhost, $s->sockport);
+	my $ipv4 = $s->sockdomain == AF_INET;
+	if (wantarray) {
+		$ipv4 ? ($h, $p) : ("[$h]", $p);
+	} else {
+		$ipv4 ? "$h:$p" : "[$h]:$p";
+	}
+}
+
+sub tcp_connect {
+	my ($dest, %opt) = @_;
+	my $addr = tcp_host_port($dest);
+	my $s = ref($dest)->new(
+		Proto => 'tcp',
+		Type => SOCK_STREAM,
+		PeerAddr => $addr,
+		%opt,
+	) or BAIL_OUT "failed to connect to $addr: $!";
+	$s->autoflush(1);
+	$s;
+}
+
+sub slurp {
+	open my $fh, '<', $_[0] or die "open($_[0]): $!";
+	local $/;
+	<$fh>;
+}
+
+sub spawn {
+	my $env = ref($_[0]) eq 'HASH' ? shift : undef;
+	my $opt = ref($_[-1]) eq 'HASH' ? pop : {};
+	my @cmd = @_;
+	my $old = POSIX::SigSet->new;
+	my $set = POSIX::SigSet->new;
+	$set->fillset or die "sigfillset: $!";
+	sigprocmask(SIG_SETMASK, $set, $old) or die "SIG_SETMASK: $!";
+	pipe(my ($r, $w)) or die "pipe: $!";
+	my $pid = fork // die "fork: $!";
+	if ($pid == 0) {
+		close $r;
+		$SIG{__DIE__} = sub {
+			warn(@_);
+			syswrite($w, my $num = $! + 0);
+			_exit(1);
+		};
+
+		# pretend to be systemd (cf. sd_listen_fds(3))
+		my $cfd;
+		for ($cfd = 0; ($cfd < 3) || defined($opt->{$cfd}); $cfd++) {
+			my $io = $opt->{$cfd} // next;
+			my $pfd = fileno($io) // die "fileno($io): $!";
+			if ($pfd == $cfd) {
+				fcntl($io, F_SETFD, 0) // die "F_SETFD: $!";
+			} else {
+				dup2($pfd, $cfd) // die "dup2($pfd, $cfd): $!";
+			}
+		}
+		if (($cfd - 3) > 0) {
+			$env->{LISTEN_PID} = $$;
+			$env->{LISTEN_FDS} = $cfd - 3;
+		}
+
+		if (defined(my $pgid = $opt->{pgid})) {
+			setpgid(0, $pgid) // die "setpgid(0, $pgid): $!";
+		}
+		$SIG{$_} = 'DEFAULT' for grep(!/^__/, keys %SIG);
+		if (defined(my $cd = $opt->{-C})) {
+			chdir $cd // die "chdir($cd): $!";
+		}
+		$old->delset(POSIX::SIGCHLD) or die "sigdelset CHLD: $!";
+		sigprocmask(SIG_SETMASK, $old) or die "SIG_SETMASK: ~CHLD: $!";
+		@ENV{keys %$env} = values(%$env) if $env;
+		exec { $cmd[0] } @cmd;
+		die "exec @cmd: $!";
+	}
+	close $w;
+	sigprocmask(SIG_SETMASK, $old) or die "SIG_SETMASK(old): $!";
+	if (my $cerrnum = do { local $/, <$r> }) {
+		$! = $cerrnum;
+		die "@cmd PID=$pid died: $!";
+	}
+	$pid;
+}
+
+sub which {
+	my ($file) = @_;
+	return $file if index($file, '/') >= 0;
+	for my $p (split(/:/, $ENV{PATH})) {
+		$p .= "/$file";
+		return $p if -x $p;
+	}
+	undef;
+}
+
+# returns an AutoReap object
+sub unicorn {
+	my %env;
+	if (ref($_[0]) eq 'HASH') {
+		my $e = shift;
+		%env = %$e;
+	}
+	my @args = @_;
+	push(@args, {}) if ref($args[-1]) ne 'HASH';
+	$args[-1]->{2} //= $errfh; # stderr default
+
+	state $ruby = which($ENV{RUBY} // 'ruby');
+	state $lib = File::Spec->rel2abs('lib');
+	state $ver = $ENV{TEST_RUBY_VERSION} // `$ruby -e 'print RUBY_VERSION'`;
+	state $eng = $ENV{TEST_RUBY_ENGINE} // `$ruby -e 'print RUBY_ENGINE'`;
+	state $ext = File::Spec->rel2abs("test/$eng-$ver/ext/unicorn_http");
+	state $exe = File::Spec->rel2abs('bin/unicorn');
+	my $pid = spawn(\%env, $ruby, '-I', $lib, '-I', $ext, $exe, @args);
+	UnicornTest::AutoReap->new($pid);
+}
+
+# automatically kill + reap children when this goes out-of-scope
+package UnicornTest::AutoReap;
+use v5.14;
+
+sub new {
+	my (undef, $pid) = @_;
+	bless { pid => $pid, owner => $$ }, __PACKAGE__
+}
+
+sub kill {
+	my ($self, $sig) = @_;
+	CORE::kill($sig // 'TERM', $self->{pid});
+}
+
+sub join {
+	my ($self, $sig) = @_;
+	my $pid = delete $self->{pid} or return;
+	CORE::kill($sig, $pid) if defined $sig;
+	my $ret = waitpid($pid, 0) // die "waitpid($pid): $!";
+	$ret == $pid or die "BUG: waitpid($pid) != $ret";
+}
+
+sub DESTROY {
+	my ($self) = @_;
+	return if $self->{owner} != $$;
+	$self->join('TERM');
+}
+
+package main; # inject ourselves into the t/*.t script
+UnicornTest->import;
+Test::More->import;
+# try to ensure ->DESTROY fires:
+$SIG{TERM} = sub { exit(15 + 128) };
+$SIG{INT} = sub { exit(2 + 128) };
+1;
diff --git a/test/unit/test_response.rb b/test/unit/test_response.rb
deleted file mode 100644
index fbe433f..0000000
--- a/test/unit/test_response.rb
+++ /dev/null
@@ -1,111 +0,0 @@
-# -*- encoding: binary -*-
-
-# Copyright (c) 2005 Zed A. Shaw
-# You can redistribute it and/or modify it under the same terms as Ruby 1.8 or
-# the GPLv2+ (GPLv3+ preferred)
-#
-# Additional work donated by contributors.  See git history
-# for more information.
-
-require './test/test_helper'
-require 'time'
-
-include Unicorn
-
-class ResponseTest < Test::Unit::TestCase
-  include Unicorn::HttpResponse
-
-  def test_httpdate
-    before = Time.now.to_i - 1
-    str = httpdate
-    assert_kind_of(String, str)
-    middle = Time.parse(str).to_i
-    after = Time.now.to_i
-    assert before <= middle
-    assert middle <= after
-  end
-
-  def test_response_headers
-    out = StringIO.new
-    http_response_write(out, 200, {"X-Whatever" => "stuff"}, ["cool"])
-    assert ! out.closed?
-
-    assert out.length > 0, "output didn't have data"
-  end
-
-  # ref: <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
-  def test_response_header_broken_nil
-    out = StringIO.new
-    http_response_write(out, 200, {"Nil" => nil}, %w(hysterical raisin))
-    assert ! out.closed?
-
-    assert_match %r{^Nil: \r\n}sm, out.string, 'nil accepted'
-  end
-
-  def test_response_string_status
-    out = StringIO.new
-    http_response_write(out,'200', {}, [])
-    assert ! out.closed?
-    assert out.length > 0, "output didn't have data"
-  end
-
-  def test_response_200
-    io = StringIO.new
-    http_response_write(io, 200, {}, [])
-    assert ! io.closed?
-    assert io.length > 0, "output didn't have data"
-  end
-
-  def test_response_with_default_reason
-    code = 400
-    io = StringIO.new
-    http_response_write(io, code, {}, [])
-    assert ! io.closed?
-    lines = io.string.split(/\r\n/)
-    assert_match(/.* Bad Request$/, lines.first,
-                 "wrong default reason phrase")
-  end
-
-  def test_rack_multivalue_headers
-    out = StringIO.new
-    http_response_write(out,200, {"X-Whatever" => "stuff\nbleh"}, [])
-    assert ! out.closed?
-    assert_match(/^X-Whatever: stuff\r\nX-Whatever: bleh\r\n/, out.string)
-  end
-
-  # Even though Rack explicitly forbids "Status" in the header hash,
-  # some broken clients still rely on it
-  def test_status_header_added
-    out = StringIO.new
-    http_response_write(out,200, {"X-Whatever" => "stuff"}, [])
-    assert ! out.closed?
-  end
-
-  def test_unknown_status_pass_through
-    out = StringIO.new
-    http_response_write(out,"666 I AM THE BEAST", {}, [] )
-    assert ! out.closed?
-    headers = out.string.split(/\r\n\r\n/).first.split(/\r\n/)
-    assert %r{\AHTTP/\d\.\d 666 I AM THE BEAST\z}.match(headers[0])
-  end
-
-  def test_modified_rack_http_status_codes_late
-    r, w = IO.pipe
-    pid = fork do
-      r.close
-      # Users may want to globally override the status text associated
-      # with an HTTP status code in their app.
-      Rack::Utils::HTTP_STATUS_CODES[200] = "HI"
-      http_response_write(w, 200, {}, [])
-      w.close
-    end
-    w.close
-    assert_equal "HTTP/1.1 200 HI\r\n", r.gets
-    r.read # just drain the pipe
-    pid, status = Process.waitpid2(pid)
-    assert status.success?, status.inspect
-  ensure
-    r.close
-    w.close unless w.closed?
-  end
-end

[-- Attachment #3: 0002-support-rack-3-multi-value-headers.patch --]
[-- Type: text/x-diff, Size: 1710 bytes --]

From ea0559c700fa029044464de4bd572662c10b7273 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:31 +0000
Subject: [PATCH 02/23] support rack 3 multi-value headers

The first step in adding Rack 3 support.  Rack supports
multi-value headers via array rather than newlines.

Tested-by: Martin Posthumus <martin.posthumus@gmail.com>
Link: https://yhbt.net/unicorn-public/7c851d8a-bc57-7df8-3240-2f5ab831c47c@gmail.com/
---
 t/integration.ru | 1 +
 t/integration.t  | 9 +++++++++
 2 files changed, 10 insertions(+)

diff --git a/t/integration.ru b/t/integration.ru
index 6ef873c..5183217 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -23,6 +23,7 @@ def restore_status_code
   when 'GET'
     case env['PATH_INFO']
     when '/rack-2-newline-headers'; [ 200, { 'X-R2' => "a\nb\nc" }, [] ]
+    when '/rack-3-array-headers'; [ 200, { 'x-r3' => %w(a b c) }, [] ]
     when '/nil-header-value'; [ 200, { 'X-Nil' => nil }, [] ]
     when '/unknown-status-pass-through'; [ '666 I AM THE BEAST', {}, [] ]
     end # case PATH_INFO (GET)
diff --git a/t/integration.t b/t/integration.t
index 5569155..e876c71 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -38,6 +38,15 @@ SKIP: { # Date header check
 		diag(explain([$t, $!, \@d]));
 };
 
+
+$c = tcp_connect($srv);
+print $c "GET /rack-3-array-headers HTTP/1.0\r\n\r\n" or die $!;
+($status, $hdr) = slurp_hdr($c);
+is_deeply([ grep(/^x-r3: /, @$hdr) ],
+	[ 'x-r3: a', 'x-r3: b', 'x-r3: c' ],
+	'rack 3 array headers supported') or diag(explain($hdr));
+
+
 # cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
 $c = tcp_connect($srv);
 print $c "GET /nil-header-value HTTP/1.0\r\n\r\n" or die $!;

[-- Attachment #4: 0003-port-t0018-write-on-close.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 4091 bytes --]

From 295a6c616f8840bc04617a377c04c3422aeebddc Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:32 +0000
Subject: [PATCH 03/23] port t0018-write-on-close.sh to Perl 5

This doesn't require restarting, so it's a perfect candidate.
---
 t/integration.ru          | 15 +++++++++++++++
 t/integration.t           | 14 +++++++++++++-
 t/lib.perl                |  2 +-
 t/t0018-write-on-close.sh | 23 -----------------------
 t/write-on-close.ru       | 11 -----------
 5 files changed, 29 insertions(+), 36 deletions(-)
 delete mode 100755 t/t0018-write-on-close.sh
 delete mode 100644 t/write-on-close.ru

diff --git a/t/integration.ru b/t/integration.ru
index 5183217..12f5d48 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -18,6 +18,20 @@ def restore_status_code
   [ 200, {}, [] ]
 end
 
+class WriteOnClose
+  def each(&block)
+    @callback = block
+  end
+
+  def close
+    @callback.call "7\r\nGoodbye\r\n0\r\n\r\n"
+  end
+end
+
+def write_on_close
+  [ 200, { 'transfer-encoding' => 'chunked' }, WriteOnClose.new ]
+end
+
 run(lambda do |env|
   case env['REQUEST_METHOD']
   when 'GET'
@@ -26,6 +40,7 @@ def restore_status_code
     when '/rack-3-array-headers'; [ 200, { 'x-r3' => %w(a b c) }, [] ]
     when '/nil-header-value'; [ 200, { 'X-Nil' => nil }, [] ]
     when '/unknown-status-pass-through'; [ '666 I AM THE BEAST', {}, [] ]
+    when '/write_on_close'; write_on_close
     end # case PATH_INFO (GET)
   when 'POST'
     case env['PATH_INFO']
diff --git a/t/integration.t b/t/integration.t
index e876c71..3ab5c90 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -4,6 +4,7 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 my $srv = tcp_server();
+my $host_port = tcp_host_port($srv);
 my $t0 = time;
 my $ar = unicorn(qw(-E none t/integration.ru), { 3 => $srv });
 
@@ -66,8 +67,19 @@ if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
 	is($status, $orig_200_status, 'original status restored');
 }
 
+SKIP: {
+	eval { require HTTP::Tiny } or skip "HTTP::Tiny missing: $@", 1;
+	my $ht = HTTP::Tiny->new;
+	my $res = $ht->get("http://$host_port/write_on_close");
+	is($res->{content}, 'Goodbye', 'write-on-close body read');
+}
 
 # ... more stuff here
 undef $ar;
-diag slurp("$tmpdir/err.log") if $ENV{V};
+my @log = slurp("$tmpdir/err.log");
+diag("@log") if $ENV{V};
+my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
+is_deeply(\@err, [], 'no unexpected errors in stderr');
+is_deeply([grep(/SIGKILL/, @log)], [], 'no SIGKILL in stderr');
+
 done_testing;
diff --git a/t/lib.perl b/t/lib.perl
index dd9c6b7..12deaf8 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -10,7 +10,7 @@ use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
-	SEEK_SET);
+	SEEK_SET tcp_host_port);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
diff --git a/t/t0018-write-on-close.sh b/t/t0018-write-on-close.sh
deleted file mode 100755
index 3afefea..0000000
--- a/t/t0018-write-on-close.sh
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 4 "write-on-close tests for funky response-bodies"
-
-t_begin "setup and start" && {
-	unicorn_setup
-	unicorn -D -c $unicorn_config write-on-close.ru
-	unicorn_wait_start
-}
-
-t_begin "write-on-close response body succeeds" && {
-	test xGoodbye = x"$(curl -sSf http://$listen/)"
-}
-
-t_begin "killing succeeds" && {
-	kill $unicorn_pid
-}
-
-t_begin "check stderr" && {
-	check_stderr
-}
-
-t_done
diff --git a/t/write-on-close.ru b/t/write-on-close.ru
deleted file mode 100644
index 725c4d6..0000000
--- a/t/write-on-close.ru
+++ /dev/null
@@ -1,11 +0,0 @@
-class WriteOnClose
-  def each(&block)
-    @callback = block
-  end
-
-  def close
-    @callback.call "7\r\nGoodbye\r\n0\r\n\r\n"
-  end
-end
-use Rack::ContentType, "text/plain"
-run(lambda { |_| [ 200, { 'transfer-encoding' => 'chunked' }, WriteOnClose.new ] })

[-- Attachment #5: 0004-port-t0000-http-basic.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 3372 bytes --]

From 1bb4362cee167ac7aeec910d3f52419e391f1e61 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:33 +0000
Subject: [PATCH 04/23] port t0000-http-basic.sh to Perl 5

One more socat dependency down...
---
 t/integration.ru      | 16 ++++++++++++++
 t/integration.t       | 11 ++++++++++
 t/t0000-http-basic.sh | 50 -------------------------------------------
 3 files changed, 27 insertions(+), 50 deletions(-)
 delete mode 100755 t/t0000-http-basic.sh

diff --git a/t/integration.ru b/t/integration.ru
index 12f5d48..c0bef99 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -32,6 +32,21 @@ def write_on_close
   [ 200, { 'transfer-encoding' => 'chunked' }, WriteOnClose.new ]
 end
 
+def env_dump(env)
+  require 'json'
+  h = {}
+  env.each do |k,v|
+    case v
+    when String, Integer, true, false; h[k] = v
+    else
+      case k
+      when 'rack.version', 'rack.after_reply'; h[k] = v
+      end
+    end
+  end
+  h.to_json
+end
+
 run(lambda do |env|
   case env['REQUEST_METHOD']
   when 'GET'
@@ -40,6 +55,7 @@ def write_on_close
     when '/rack-3-array-headers'; [ 200, { 'x-r3' => %w(a b c) }, [] ]
     when '/nil-header-value'; [ 200, { 'X-Nil' => nil }, [] ]
     when '/unknown-status-pass-through'; [ '666 I AM THE BEAST', {}, [] ]
+    when '/env_dump'; [ 200, {}, [ env_dump(env) ] ]
     when '/write_on_close'; write_on_close
     end # case PATH_INFO (GET)
   when 'POST'
diff --git a/t/integration.t b/t/integration.t
index 3ab5c90..ee22e7e 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -47,6 +47,17 @@ is_deeply([ grep(/^x-r3: /, @$hdr) ],
 	[ 'x-r3: a', 'x-r3: b', 'x-r3: c' ],
 	'rack 3 array headers supported') or diag(explain($hdr));
 
+SKIP: {
+	eval { require JSON::PP } or skip "JSON::PP missing: $@", 1;
+	$c = tcp_connect($srv);
+	print $c "GET /env_dump\r\n" or die $!;
+	my $json = do { local $/; readline($c) };
+	unlike($json, qr/^Connection: /smi, 'no connection header for 0.9');
+	unlike($json, qr!\AHTTP/!s, 'no HTTP/1.x prefix for 0.9');
+	my $env = JSON::PP->new->decode($json);
+	is(ref($env), 'HASH', 'JSON decoded body to hashref');
+	is($env->{SERVER_PROTOCOL}, 'HTTP/0.9', 'SERVER_PROTOCOL is 0.9');
+}
 
 # cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
 $c = tcp_connect($srv);
diff --git a/t/t0000-http-basic.sh b/t/t0000-http-basic.sh
deleted file mode 100755
index 8ab58ac..0000000
--- a/t/t0000-http-basic.sh
+++ /dev/null
@@ -1,50 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 8 "simple HTTP connection tests"
-
-t_begin "setup and start" && {
-	unicorn_setup
-	unicorn -D -c $unicorn_config env.ru
-	unicorn_wait_start
-}
-
-t_begin "single request" && {
-	curl -sSfv http://$listen/
-}
-
-t_begin "check stderr has no errors" && {
-	check_stderr
-}
-
-t_begin "HTTP/0.9 request should not return headers" && {
-	(
-		printf 'GET /\r\n'
-		cat $fifo > $tmp &
-		wait
-		echo ok > $ok
-	) | socat - TCP:$listen > $fifo
-}
-
-t_begin "env.inspect should've put everything on one line" && {
-	test 1 -eq $(count_lines < $tmp)
-}
-
-t_begin "no headers in output" && {
-	if grep ^Connection: $tmp
-	then
-		die "Connection header found in $tmp"
-	elif grep ^HTTP/ $tmp
-	then
-		die "HTTP/ found in $tmp"
-	fi
-}
-
-t_begin "killing succeeds" && {
-	kill $unicorn_pid
-}
-
-t_begin "check stderr has no errors" && {
-	check_stderr
-}
-
-t_done

[-- Attachment #6: 0005-port-t0002-parser-error.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 4875 bytes --]

From 2eb7b1662c291ab535ee5dabf5d96194ca6483d4 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:34 +0000
Subject: [PATCH 05/23] port t0002-parser-error.sh to Perl 5

Another socat dependency down...
---
 t/integration.t         | 33 +++++++++++++++
 t/lib.perl              |  9 +++-
 t/t0002-parser-error.sh | 94 -----------------------------------------
 3 files changed, 41 insertions(+), 95 deletions(-)
 delete mode 100755 t/t0002-parser-error.sh

diff --git a/t/integration.t b/t/integration.t
index ee22e7e..503b7eb 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -85,6 +85,39 @@ SKIP: {
 	is($res->{content}, 'Goodbye', 'write-on-close body read');
 }
 
+if ('bad requests') {
+	$c = start_req($srv, 'GET /env_dump HTTP/1/1');
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 400 \b!, 'got 400 on bad request');
+
+	$c = tcp_connect($srv);
+	print $c 'GET /' or die $!;
+	my $buf = join('', (0..9), 'ab');
+	for (0..1023) { print $c $buf or die $! }
+	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 414 \b!,
+		'414 on REQUEST_PATH > (12 * 1024)');
+
+	$c = tcp_connect($srv);
+	print $c 'GET /hello-world?a' or die $!;
+	$buf = join('', (0..9));
+	for (0..1023) { print $c $buf or die $! }
+	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 414 \b!,
+		'414 on QUERY_STRING > (10 * 1024)');
+
+	$c = tcp_connect($srv);
+	print $c 'GET /hello-world#a' or die $!;
+	$buf = join('', (0..9), 'a'..'f');
+	for (0..63) { print $c $buf or die $! }
+	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 414 \b!, '414 on FRAGMENT > (1024)');
+}
+
+
 # ... more stuff here
 undef $ar;
 my @log = slurp("$tmpdir/err.log");
diff --git a/t/lib.perl b/t/lib.perl
index 12deaf8..7d712b5 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -10,7 +10,7 @@ use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
-	SEEK_SET tcp_host_port);
+	SEEK_SET tcp_host_port start_req);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
@@ -59,6 +59,13 @@ sub tcp_connect {
 	$s;
 }
 
+sub start_req {
+	my ($srv, @req) = @_;
+	my $c = tcp_connect($srv);
+	print $c @req, "\r\n\r\n" or die "print: $!";
+	$c;
+}
+
 sub slurp {
 	open my $fh, '<', $_[0] or die "open($_[0]): $!";
 	local $/;
diff --git a/t/t0002-parser-error.sh b/t/t0002-parser-error.sh
deleted file mode 100755
index 9dc1cd2..0000000
--- a/t/t0002-parser-error.sh
+++ /dev/null
@@ -1,94 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 11 "parser error test"
-
-t_begin "setup and startup" && {
-	unicorn_setup
-	unicorn -D env.ru -c $unicorn_config
-	unicorn_wait_start
-}
-
-t_begin "send a bad request" && {
-	(
-		printf 'GET / HTTP/1/1\r\nHost: example.com\r\n\r\n'
-		cat $fifo > $tmp &
-		wait
-		echo ok > $ok
-	) | socat - TCP:$listen > $fifo
-	test xok = x$(cat $ok)
-}
-
-dbgcat tmp
-
-t_begin "response should be a 400" && {
-	grep -F 'HTTP/1.1 400 Bad Request' $tmp
-}
-
-t_begin "send a huge Request URI (REQUEST_PATH > (12 * 1024))" && {
-	rm -f $tmp
-	cat $fifo > $tmp &
-	(
-		set -e
-		trap 'echo ok > $ok' EXIT
-		printf 'GET /'
-		for i in $(awk </dev/null 'BEGIN{for(i=0;i<1024;i++) print i}')
-		do
-			printf '0123456789ab'
-		done
-		printf ' HTTP/1.1\r\nHost: example.com\r\n\r\n'
-	) | socat - TCP:$listen > $fifo || :
-	test xok = x$(cat $ok)
-	wait
-}
-
-t_begin "response should be a 414 (REQUEST_PATH)" && {
-	grep -F 'HTTP/1.1 414 ' $tmp
-}
-
-t_begin "send a huge Request URI (QUERY_STRING > (10 * 1024))" && {
-	rm -f $tmp
-	cat $fifo > $tmp &
-	(
-		set -e
-		trap 'echo ok > $ok' EXIT
-		printf 'GET /hello-world?a'
-		for i in $(awk </dev/null 'BEGIN{for(i=0;i<1024;i++) print i}')
-		do
-			printf '0123456789'
-		done
-		printf ' HTTP/1.1\r\nHost: example.com\r\n\r\n'
-	) | socat - TCP:$listen > $fifo || :
-	test xok = x$(cat $ok)
-	wait
-}
-
-t_begin "response should be a 414 (QUERY_STRING)" && {
-	grep -F 'HTTP/1.1 414 ' $tmp
-}
-
-t_begin "send a huge Request URI (FRAGMENT > 1024)" && {
-	rm -f $tmp
-	cat $fifo > $tmp &
-	(
-		set -e
-		trap 'echo ok > $ok' EXIT
-		printf 'GET /hello-world#a'
-		for i in $(awk </dev/null 'BEGIN{for(i=0;i<64;i++) print i}')
-		do
-			printf '0123456789abcdef'
-		done
-		printf ' HTTP/1.1\r\nHost: example.com\r\n\r\n'
-	) | socat - TCP:$listen > $fifo || :
-	test xok = x$(cat $ok)
-	wait
-}
-
-t_begin "response should be a 414 (FRAGMENT)" && {
-	grep -F 'HTTP/1.1 414 ' $tmp
-}
-
-t_begin "server stderr should be clean" && check_stderr
-
-t_begin "term signal sent" && kill $unicorn_pid
-
-t_done

[-- Attachment #7: 0006-t-integration.t-use-start_req-to-simplify-test-sligh.patch --]
[-- Type: text/x-diff, Size: 2556 bytes --]

From 0bb06cc0c8c4f5b76514858067bbb2871dda0d6e Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:35 +0000
Subject: [PATCH 06/23] t/integration.t: use start_req to simplify test slighly

Less code is usually better.
---
 t/integration.t | 18 ++++++------------
 1 file changed, 6 insertions(+), 12 deletions(-)

diff --git a/t/integration.t b/t/integration.t
index 503b7eb..b7ba1fb 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -20,8 +20,7 @@ sub slurp_hdr {
 my ($c, $status, $hdr);
 
 # response header tests
-$c = tcp_connect($srv);
-print $c "GET /rack-2-newline-headers HTTP/1.0\r\n\r\n" or die $!;
+$c = start_req($srv, 'GET /rack-2-newline-headers HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
 my $orig_200_status = $status;
@@ -40,8 +39,7 @@ SKIP: { # Date header check
 };
 
 
-$c = tcp_connect($srv);
-print $c "GET /rack-3-array-headers HTTP/1.0\r\n\r\n" or die $!;
+$c = start_req($srv, 'GET /rack-3-array-headers HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 is_deeply([ grep(/^x-r3: /, @$hdr) ],
 	[ 'x-r3: a', 'x-r3: b', 'x-r3: c' ],
@@ -49,8 +47,7 @@ is_deeply([ grep(/^x-r3: /, @$hdr) ],
 
 SKIP: {
 	eval { require JSON::PP } or skip "JSON::PP missing: $@", 1;
-	$c = tcp_connect($srv);
-	print $c "GET /env_dump\r\n" or die $!;
+	my $c = start_req($srv, 'GET /env_dump');
 	my $json = do { local $/; readline($c) };
 	unlike($json, qr/^Connection: /smi, 'no connection header for 0.9');
 	unlike($json, qr!\AHTTP/!s, 'no HTTP/1.x prefix for 0.9');
@@ -60,20 +57,17 @@ SKIP: {
 }
 
 # cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
-$c = tcp_connect($srv);
-print $c "GET /nil-header-value HTTP/1.0\r\n\r\n" or die $!;
+$c = start_req($srv, 'GET /nil-header-value HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
 	'nil header value accepted for broken apps') or diag(explain($hdr));
 
 if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
-	$c = tcp_connect($srv);
-	print $c "POST /tweak-status-code HTTP/1.0\r\n\r\n" or die $!;
+	$c = start_req($srv, 'POST /tweak-status-code HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 200 HI\b!, 'status tweaked');
 
-	$c = tcp_connect($srv);
-	print $c "POST /restore-status-code HTTP/1.0\r\n\r\n" or die $!;
+	$c = start_req($srv, 'POST /restore-status-code HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
 	is($status, $orig_200_status, 'original status restored');
 }

[-- Attachment #8: 0007-port-t0011-active-unix-socket.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 6945 bytes --]

From 10c83beaca58df8b92d8228e798559069cd89beb Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:36 +0000
Subject: [PATCH 07/23] port t0011-active-unix-socket.sh to Perl 5

Another socat dependency down...  I've also started turning
FD_CLOEXEC off on a pipe as a mechanism to detect daemonized
process death in tests.
---
 t/active-unix-socket.t        | 117 ++++++++++++++++++++++++++++++++++
 t/integration.ru              |   1 +
 t/t0011-active-unix-socket.sh |  79 -----------------------
 3 files changed, 118 insertions(+), 79 deletions(-)
 create mode 100644 t/active-unix-socket.t
 delete mode 100755 t/t0011-active-unix-socket.sh

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
new file mode 100644
index 0000000..6b5c218
--- /dev/null
+++ b/t/active-unix-socket.t
@@ -0,0 +1,117 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+
+use v5.14; BEGIN { require './t/lib.perl' };
+use IO::Socket::UNIX;
+my %to_kill;
+END { kill('TERM', values(%to_kill)) if keys %to_kill }
+my $u1 = "$tmpdir/u1.sock";
+my $u2 = "$tmpdir/u2.sock";
+my $unix_req = sub {
+	my $s = IO::Socket::UNIX->new(Peer => shift, Type => SOCK_STREAM);
+	print $s @_, "\r\n\r\n" or die $!;
+	$s;
+};
+{
+	use autodie;
+	open my $fh, '>', "$tmpdir/u1.conf.rb";
+	print $fh <<EOM;
+pid "$tmpdir/u.pid"
+listen "$u1"
+stderr_path "$tmpdir/err1.log"
+EOM
+	close $fh;
+
+	open $fh, '>', "$tmpdir/u2.conf.rb";
+	print $fh <<EOM;
+pid "$tmpdir/u.pid"
+listen "$u2"
+stderr_path "$tmpdir/err2.log"
+EOM
+	close $fh;
+
+	open $fh, '>', "$tmpdir/u3.conf.rb";
+	print $fh <<EOM;
+pid "$tmpdir/u3.pid"
+listen "$u1"
+stderr_path "$tmpdir/err3.log"
+EOM
+	close $fh;
+}
+
+my @uarg = qw(-D -E none t/integration.ru);
+
+# this pipe will be used to notify us when all daemons die:
+pipe(my ($p0, $p1)) or die "pipe: $!";
+fcntl($p1, POSIX::F_SETFD, 0) or die "fcntl: $!"; # clear FD_CLOEXEC
+
+# start the first instance
+unicorn('-c', "$tmpdir/u1.conf.rb", @uarg)->join;
+is($?, 0, 'daemonized 1st process');
+chomp($to_kill{u1} = slurp("$tmpdir/u.pid"));
+like($to_kill{u1}, qr/\A\d+\z/s, 'read pid file');
+
+chomp(my $worker_pid = readline($unix_req->($u1, 'GET /pid')));
+like($worker_pid, qr/\A\d+\z/s, 'captured worker pid');
+ok(kill(0, $worker_pid), 'worker is kill-able');
+
+
+# 2nd process conflicts on PID
+unicorn('-c', "$tmpdir/u2.conf.rb", @uarg)->join;
+isnt($?, 0, 'conflicting PID file fails to start');
+
+chomp(my $pidf = slurp("$tmpdir/u.pid"));
+is($pidf, $to_kill{u1}, 'pid file contents unchanged after start failure');
+
+chomp(my $pid2 = readline($unix_req->($u1, 'GET /pid')));
+is($worker_pid, $pid2, 'worker PID unchanged');
+
+
+# 3rd process conflicts on socket
+unicorn('-c', "$tmpdir/u3.conf.rb", @uarg)->join;
+isnt($?, 0, 'conflicting UNIX socket fails to start');
+
+chomp($pid2 = readline($unix_req->($u1, 'GET /pid')));
+is($worker_pid, $pid2, 'worker PID still unchanged');
+
+chomp($pidf = slurp("$tmpdir/u.pid"));
+is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
+
+{ # teardown initial process via SIGKILL
+	ok(kill('KILL', delete $to_kill{u1}), 'SIGKILL initial daemon');
+	close $p1;
+	vec(my $rvec = '', fileno($p0), 1) = 1;
+	is(select($rvec, undef, undef, 5), 1, 'timeout for pipe HUP');
+	is(my $undef = <$p0>, undef, 'process closed pipe writer at exit');
+	ok(-f "$tmpdir/u.pid", 'pid file stayed after SIGKILL');
+	ok(-S $u1, 'socket stayed after SIGKILL');
+	is(IO::Socket::UNIX->new(Peer => $u1, Type => SOCK_STREAM), undef,
+		'fail to connect to u1');
+	ok(!kill(0, $worker_pid), 'worker gone after parent dies');
+}
+
+# restart the first instance
+{
+	pipe(($p0, $p1)) or die "pipe: $!";
+	fcntl($p1, POSIX::F_SETFD, 0) or die "fcntl: $!"; # clear FD_CLOEXEC
+	unicorn('-c', "$tmpdir/u1.conf.rb", @uarg)->join;
+	is($?, 0, 'daemonized 1st process');
+	chomp($to_kill{u1} = slurp("$tmpdir/u.pid"));
+	like($to_kill{u1}, qr/\A\d+\z/s, 'read pid file');
+
+	chomp($pid2 = readline($unix_req->($u1, 'GET /pid')));
+	like($pid2, qr/\A\d+\z/, 'worker running');
+
+	ok(kill('TERM', delete $to_kill{u1}), 'SIGTERM restarted daemon');
+	close $p1;
+	vec(my $rvec = '', fileno($p0), 1) = 1;
+	is(select($rvec, undef, undef, 5), 1, 'timeout for pipe HUP');
+	is(my $undef = <$p0>, undef, 'process closed pipe writer at exit');
+	ok(!-f "$tmpdir/u.pid", 'pid file gone after SIGTERM');
+	ok(-S $u1, 'socket stays after SIGTERM');
+}
+
+my @log = slurp("$tmpdir/err.log");
+diag("@log") if $ENV{V};
+done_testing;
diff --git a/t/integration.ru b/t/integration.ru
index c0bef99..21f5449 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -57,6 +57,7 @@ def env_dump(env)
     when '/unknown-status-pass-through'; [ '666 I AM THE BEAST', {}, [] ]
     when '/env_dump'; [ 200, {}, [ env_dump(env) ] ]
     when '/write_on_close'; write_on_close
+    when '/pid'; [ 200, {}, [ "#$$\n" ] ]
     end # case PATH_INFO (GET)
   when 'POST'
     case env['PATH_INFO']
diff --git a/t/t0011-active-unix-socket.sh b/t/t0011-active-unix-socket.sh
deleted file mode 100755
index fae0b6c..0000000
--- a/t/t0011-active-unix-socket.sh
+++ /dev/null
@@ -1,79 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 11 "existing UNIX domain socket check"
-
-read_pid_unix () {
-	x=$(printf 'GET / HTTP/1.0\r\n\r\n' | \
-	    socat - UNIX:$unix_socket | \
-	    tail -1)
-	test -n "$x"
-	y="$(expr "$x" : '\([0-9][0-9]*\)')"
-	test x"$x" = x"$y"
-	test -n "$y"
-	echo "$y"
-}
-
-t_begin "setup and start" && {
-	rtmpfiles unix_socket unix_config
-	rm -f $unix_socket
-	unicorn_setup
-	grep -v ^listen < $unicorn_config > $unix_config
-	echo "listen '$unix_socket'" >> $unix_config
-	unicorn -D -c $unix_config pid.ru
-	unicorn_wait_start
-	orig_master_pid=$unicorn_pid
-}
-
-t_begin "get pid of worker" && {
-	worker_pid=$(read_pid_unix)
-	t_info "worker_pid=$worker_pid"
-}
-
-t_begin "fails to start with existing pid file" && {
-	rm -f $ok
-	unicorn -D -c $unix_config pid.ru || echo ok > $ok
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "worker pid unchanged" && {
-	test x"$(read_pid_unix)" = x$worker_pid
-	> $r_err
-}
-
-t_begin "fails to start with listening UNIX domain socket bound" && {
-	rm $ok $pid
-	unicorn -D -c $unix_config pid.ru || echo ok > $ok
-	test x"$(cat $ok)" = xok
-	> $r_err
-}
-
-t_begin "worker pid unchanged (again)" && {
-	test x"$(read_pid_unix)" = x$worker_pid
-}
-
-t_begin "nuking the existing Unicorn succeeds" && {
-	kill -9 $unicorn_pid
-	while kill -0 $unicorn_pid
-	do
-		sleep 1
-	done
-	check_stderr
-}
-
-t_begin "succeeds in starting with leftover UNIX domain socket bound" && {
-	test -S $unix_socket
-	unicorn -D -c $unix_config pid.ru
-	unicorn_wait_start
-}
-
-t_begin "worker pid changed" && {
-	test x"$(read_pid_unix)" != x$worker_pid
-}
-
-t_begin "killing succeeds" && {
-	kill $unicorn_pid
-}
-
-t_begin "no errors" && check_stderr
-
-t_done

[-- Attachment #9: 0008-port-t0100-rack-input-tests.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 11722 bytes --]

From b4ed148186295f2d5c8448eab7f2b201615d1e4e Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:37 +0000
Subject: [PATCH 08/23] port t0100-rack-input-tests.sh to Perl 5

Yet another socat dependency gone \o/
---
 t/bin/content-md5-put       |  36 -----------
 t/integration.ru            |  27 +++++++-
 t/integration.t             |  97 +++++++++++++++++++++++++++-
 t/lib.perl                  |   3 +-
 t/rack-input-tests.ru       |  21 ------
 t/t0100-rack-input-tests.sh | 124 ------------------------------------
 6 files changed, 124 insertions(+), 184 deletions(-)
 delete mode 100755 t/bin/content-md5-put
 delete mode 100644 t/rack-input-tests.ru
 delete mode 100755 t/t0100-rack-input-tests.sh

diff --git a/t/bin/content-md5-put b/t/bin/content-md5-put
deleted file mode 100755
index 01da0bb..0000000
--- a/t/bin/content-md5-put
+++ /dev/null
@@ -1,36 +0,0 @@
-#!/usr/bin/env ruby
-# -*- encoding: binary -*-
-# simple chunked HTTP PUT request generator (and just that),
-# it reads stdin and writes to stdout so socat can write to a
-# UNIX or TCP socket (or to another filter or file) along with
-# a Content-MD5 trailer.
-require 'digest/md5'
-$stdout.sync = $stderr.sync = true
-$stdout.binmode
-$stdin.binmode
-
-bs = ENV['bs'] ? ENV['bs'].to_i : 4096
-
-if ARGV.grep("--no-headers").empty?
-  $stdout.write(
-      "PUT / HTTP/1.1\r\n" \
-      "Host: example.com\r\n" \
-      "Transfer-Encoding: chunked\r\n" \
-      "Trailer: Content-MD5\r\n" \
-      "\r\n"
-    )
-end
-
-digest = Digest::MD5.new
-if buf = $stdin.readpartial(bs)
-  begin
-    digest.update(buf)
-    $stdout.write("%x\r\n" % [ buf.size ])
-    $stdout.write(buf)
-    $stdout.write("\r\n")
-  end while $stdin.read(bs, buf)
-end
-
-digest = [ digest.digest ].pack('m').strip
-$stdout.write("0\r\n")
-$stdout.write("Content-MD5: #{digest}\r\n\r\n")
diff --git a/t/integration.ru b/t/integration.ru
index 21f5449..98528f6 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -47,6 +47,29 @@ def env_dump(env)
   h.to_json
 end
 
+def rack_input_tests(env)
+  return [ 100, {}, [] ] if /\A100-continue\z/i =~ env['HTTP_EXPECT']
+  cap = 16384
+  require 'digest/sha1'
+  digest = Digest::SHA1.new
+  input = env['rack.input']
+  case env['PATH_INFO']
+  when '/rack_input/size_first'; input.size
+  when '/rack_input/rewind_first'; input.rewind
+  when '/rack_input'; # OK
+  else
+    abort "bad path: #{env['PATH_INFO']}"
+  end
+  if buf = input.read(rand(cap))
+    begin
+      raise "#{buf.size} > #{cap}" if buf.size > cap
+      digest.update(buf)
+    end while input.read(rand(cap), buf)
+  end
+  [ 200, {'content-length' => '40', 'content-type' => 'text/plain'},
+    [ digest.hexdigest ] ]
+end
+
 run(lambda do |env|
   case env['REQUEST_METHOD']
   when 'GET'
@@ -66,6 +89,8 @@ def env_dump(env)
     end # case PATH_INFO (POST)
     # ...
   when 'PUT'
-    # ...
+    case env['PATH_INFO']
+    when %r{\A/rack_input}; rack_input_tests(env)
+    end
   end # case REQUEST_METHOD
 end) # run
diff --git a/t/integration.t b/t/integration.t
index b7ba1fb..8cef561 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -1,13 +1,16 @@
 #!perl -w
 # Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# this is the main integration test for things which don't require
+# restarting or signals
 
 use v5.14; BEGIN { require './t/lib.perl' };
 my $srv = tcp_server();
 my $host_port = tcp_host_port($srv);
 my $t0 = time;
 my $ar = unicorn(qw(-E none t/integration.ru), { 3 => $srv });
-
+my $curl = which('curl');
+END { diag slurp("$tmpdir/err.log") if $tmpdir };
 sub slurp_hdr {
 	my ($c) = @_;
 	local $/ = "\r\n\r\n"; # affects both readline+chomp
@@ -17,6 +20,48 @@ sub slurp_hdr {
 	($status, \@hdr);
 }
 
+my %PUT = (
+	chunked_md5 => sub {
+		my ($in, $out, $path, %opt) = @_;
+		my $bs = $opt{bs} // 16384;
+		require Digest::MD5;
+		my $dig = Digest::MD5->new;
+		print $out <<EOM;
+PUT $path HTTP/1.1\r
+Transfer-Encoding: chunked\r
+Trailer: Content-MD5\r
+\r
+EOM
+		my ($buf, $r);
+		while (1) {
+			$r = read($in, $buf, $bs) // die "read: $!";
+			last if $r == 0;
+			printf $out "%x\r\n", length($buf);
+			print $out $buf, "\r\n";
+			$dig->add($buf);
+		}
+		print $out "0\r\nContent-MD5: ", $dig->b64digest, "\r\n\r\n";
+	},
+	identity => sub {
+		my ($in, $out, $path, %opt) = @_;
+		my $bs = $opt{bs} // 16384;
+		my $clen = $opt{-s} // -s $in;
+		print $out <<EOM;
+PUT $path HTTP/1.0\r
+Content-Length: $clen\r
+\r
+EOM
+		my ($buf, $r, $len);
+		while ($clen) {
+			$len = $clen > $bs ? $bs : $clen;
+			$r = read($in, $buf, $len) // die "read: $!";
+			die 'premature EOF' if $r == 0;
+			print $out $buf;
+			$clen -= $r;
+		}
+	},
+);
+
 my ($c, $status, $hdr);
 
 # response header tests
@@ -111,6 +156,55 @@ if ('bad requests') {
 	like($status, qr!\AHTTP/1\.[01] 414 \b!, '414 on FRAGMENT > (1024)');
 }
 
+# input tests
+my ($blob_size, $blob_hash);
+SKIP: {
+	open(my $rh, '<', 't/random_blob') or
+		skip "t/random_blob not generated $!", 1;
+	$blob_size = -s $rh;
+	require Digest::SHA;
+	$blob_hash = Digest::SHA->new(1)->addfile($rh)->hexdigest;
+
+	my $ck_hash = sub {
+		my ($sub, $path, %opt) = @_;
+		seek($rh, 0, SEEK_SET) // die "seek: $!";
+		$c = tcp_connect($srv);
+		$c->autoflush(0);
+		$PUT{$sub}->($rh, $c, $path, %opt);
+		$c->flush or die "flush: $!";
+		($status, $hdr) = slurp_hdr($c);
+		is(readline($c), $blob_hash, "$sub $path");
+	};
+	$ck_hash->('identity', '/rack_input', -s => $blob_size);
+	$ck_hash->('chunked_md5', '/rack_input');
+	$ck_hash->('identity', '/rack_input/size_first', -s => $blob_size);
+	$ck_hash->('identity', '/rack_input/rewind_first', -s => $blob_size);
+	$ck_hash->('chunked_md5', '/rack_input/size_first');
+	$ck_hash->('chunked_md5', '/rack_input/rewind_first');
+
+
+	$curl // skip 'no curl found in PATH', 1;
+
+	my ($copt, $cout);
+	my $url = "http://$host_port/rack_input";
+	my $do_curl = sub {
+		my (@arg) = @_;
+		pipe(my $cout, $copt->{1}) or die "pipe: $!";
+		open $copt->{2}, '>', "$tmpdir/curl.err" or die $!;
+		my $cpid = spawn($curl, '-sSf', @arg, $url, $copt);
+		close(delete $copt->{1}) or die "close: $!";
+		is(readline($cout), $blob_hash, "curl @arg response");
+		is(waitpid($cpid, 0), $cpid, "curl @arg exited");
+		is($?, 0, "no error from curl @arg");
+		is(slurp("$tmpdir/curl.err"), '', "no stderr from curl @arg");
+	};
+
+	$do_curl->(qw(-T t/random_blob));
+
+	seek($rh, 0, SEEK_SET) // die "seek: $!";
+	$copt->{0} = $rh;
+	$do_curl->('-T-');
+}
 
 # ... more stuff here
 undef $ar;
@@ -120,4 +214,5 @@ my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
 is_deeply(\@err, [], 'no unexpected errors in stderr');
 is_deeply([grep(/SIGKILL/, @log)], [], 'no SIGKILL in stderr');
 
+undef $tmpdir;
 done_testing;
diff --git a/t/lib.perl b/t/lib.perl
index 7d712b5..ae9f197 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -10,7 +10,7 @@ use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
-	SEEK_SET tcp_host_port start_req);
+	SEEK_SET tcp_host_port start_req which spawn);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
@@ -193,4 +193,5 @@ Test::More->import;
 # try to ensure ->DESTROY fires:
 $SIG{TERM} = sub { exit(15 + 128) };
 $SIG{INT} = sub { exit(2 + 128) };
+$SIG{PIPE} = sub { exit(13 + 128) };
 1;
diff --git a/t/rack-input-tests.ru b/t/rack-input-tests.ru
deleted file mode 100644
index 5459e85..0000000
--- a/t/rack-input-tests.ru
+++ /dev/null
@@ -1,21 +0,0 @@
-# SHA1 checksum generator
-require 'digest/sha1'
-use Rack::ContentLength
-cap = 16384
-app = lambda do |env|
-  /\A100-continue\z/i =~ env['HTTP_EXPECT'] and
-    return [ 100, {}, [] ]
-  digest = Digest::SHA1.new
-  input = env['rack.input']
-  input.size if env["PATH_INFO"] == "/size_first"
-  input.rewind if env["PATH_INFO"] == "/rewind_first"
-  if buf = input.read(rand(cap))
-    begin
-      raise "#{buf.size} > #{cap}" if buf.size > cap
-      digest.update(buf)
-    end while input.read(rand(cap), buf)
-  end
-
-  [ 200, {'content-type' => 'text/plain'}, [ digest.hexdigest << "\n" ] ]
-end
-run app
diff --git a/t/t0100-rack-input-tests.sh b/t/t0100-rack-input-tests.sh
deleted file mode 100755
index ee7a437..0000000
--- a/t/t0100-rack-input-tests.sh
+++ /dev/null
@@ -1,124 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-test -r random_blob || die "random_blob required, run with 'make $0'"
-
-t_plan 10 "rack.input read tests"
-
-t_begin "setup and startup" && {
-	rtmpfiles curl_out curl_err
-	unicorn_setup
-	unicorn -E none -D rack-input-tests.ru -c $unicorn_config
-	blob_sha1=$(rsha1 < random_blob)
-	blob_size=$(count_bytes < random_blob)
-	t_info "blob_sha1=$blob_sha1"
-	unicorn_wait_start
-}
-
-t_begin "corked identity request" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'PUT / HTTP/1.0\r\n'
-		printf 'Content-Length: %d\r\n\r\n' $blob_size
-		cat random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "corked chunked request" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		content-md5-put < random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "corked identity request (input#size first)" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'PUT /size_first HTTP/1.0\r\n'
-		printf 'Content-Length: %d\r\n\r\n' $blob_size
-		cat random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "corked identity request (input#rewind first)" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'PUT /rewind_first HTTP/1.0\r\n'
-		printf 'Content-Length: %d\r\n\r\n' $blob_size
-		cat random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "corked chunked request (input#size first)" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'PUT /size_first HTTP/1.1\r\n'
-		printf 'Host: example.com\r\n'
-		printf 'Transfer-Encoding: chunked\r\n'
-		printf 'Trailer: Content-MD5\r\n'
-		printf '\r\n'
-		content-md5-put --no-headers < random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "corked chunked request (input#rewind first)" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'PUT /rewind_first HTTP/1.1\r\n'
-		printf 'Host: example.com\r\n'
-		printf 'Transfer-Encoding: chunked\r\n'
-		printf 'Trailer: Content-MD5\r\n'
-		printf '\r\n'
-		content-md5-put --no-headers < random_blob
-		wait
-		echo ok > $ok
-	) | ( sleep 1 && socat - TCP4:$listen > $fifo )
-	test 1 -eq $(grep $blob_sha1 $tmp |count_lines)
-	test x"$(cat $ok)" = xok
-}
-
-t_begin "regular request" && {
-	curl -sSf -T random_blob http://$listen/ > $curl_out 2> $curl_err
-        test x$blob_sha1 = x$(cat $curl_out)
-        test ! -s $curl_err
-}
-
-t_begin "chunked request" && {
-	curl -sSf -T- < random_blob http://$listen/ > $curl_out 2> $curl_err
-        test x$blob_sha1 = x$(cat $curl_out)
-        test ! -s $curl_err
-}
-
-dbgcat r_err
-
-t_begin "shutdown" && {
-	kill $unicorn_pid
-}
-
-t_done

[-- Attachment #10: 0009-tests-use-autodie-to-simplify-error-checking.patch --]
[-- Type: text/x-diff, Size: 8495 bytes --]

From 3a1d015a3859b639d8e4463e9436a49f4f0f720e Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:38 +0000
Subject: [PATCH 09/23] tests: use autodie to simplify error checking

autodie is bundled with Perl 5.10+ and simplifies error
checking in most cases.  Some subroutines aren't perfectly
translatable and their call sites had to be tweaked, but
most of them are.
---
 t/active-unix-socket.t | 13 +++++++------
 t/integration.t        | 37 +++++++++++++++++++------------------
 t/lib.perl             | 30 +++++++++++++++---------------
 3 files changed, 41 insertions(+), 39 deletions(-)

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index 6b5c218..1241904 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -4,17 +4,18 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use IO::Socket::UNIX;
+use autodie;
+no autodie 'kill';
 my %to_kill;
 END { kill('TERM', values(%to_kill)) if keys %to_kill }
 my $u1 = "$tmpdir/u1.sock";
 my $u2 = "$tmpdir/u2.sock";
 my $unix_req = sub {
 	my $s = IO::Socket::UNIX->new(Peer => shift, Type => SOCK_STREAM);
-	print $s @_, "\r\n\r\n" or die $!;
+	print $s @_, "\r\n\r\n";
 	$s;
 };
 {
-	use autodie;
 	open my $fh, '>', "$tmpdir/u1.conf.rb";
 	print $fh <<EOM;
 pid "$tmpdir/u.pid"
@@ -43,8 +44,8 @@ EOM
 my @uarg = qw(-D -E none t/integration.ru);
 
 # this pipe will be used to notify us when all daemons die:
-pipe(my ($p0, $p1)) or die "pipe: $!";
-fcntl($p1, POSIX::F_SETFD, 0) or die "fcntl: $!"; # clear FD_CLOEXEC
+pipe(my $p0, my $p1);
+fcntl($p1, POSIX::F_SETFD, 0);
 
 # start the first instance
 unicorn('-c', "$tmpdir/u1.conf.rb", @uarg)->join;
@@ -93,8 +94,8 @@ is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
 
 # restart the first instance
 {
-	pipe(($p0, $p1)) or die "pipe: $!";
-	fcntl($p1, POSIX::F_SETFD, 0) or die "fcntl: $!"; # clear FD_CLOEXEC
+	pipe($p0, $p1);
+	fcntl($p1, POSIX::F_SETFD, 0);
 	unicorn('-c', "$tmpdir/u1.conf.rb", @uarg)->join;
 	is($?, 0, 'daemonized 1st process');
 	chomp($to_kill{u1} = slurp("$tmpdir/u.pid"));
diff --git a/t/integration.t b/t/integration.t
index 8cef561..af17d51 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -5,6 +5,7 @@
 # restarting or signals
 
 use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
 my $srv = tcp_server();
 my $host_port = tcp_host_port($srv);
 my $t0 = time;
@@ -34,7 +35,7 @@ Trailer: Content-MD5\r
 EOM
 		my ($buf, $r);
 		while (1) {
-			$r = read($in, $buf, $bs) // die "read: $!";
+			$r = read($in, $buf, $bs);
 			last if $r == 0;
 			printf $out "%x\r\n", length($buf);
 			print $out $buf, "\r\n";
@@ -54,7 +55,7 @@ EOM
 		my ($buf, $r, $len);
 		while ($clen) {
 			$len = $clen > $bs ? $bs : $clen;
-			$r = read($in, $buf, $len) // die "read: $!";
+			$r = read($in, $buf, $len);
 			die 'premature EOF' if $r == 0;
 			print $out $buf;
 			$clen -= $r;
@@ -130,28 +131,28 @@ if ('bad requests') {
 	like($status, qr!\AHTTP/1\.[01] 400 \b!, 'got 400 on bad request');
 
 	$c = tcp_connect($srv);
-	print $c 'GET /' or die $!;
+	print $c 'GET /';
 	my $buf = join('', (0..9), 'ab');
-	for (0..1023) { print $c $buf or die $! }
-	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	for (0..1023) { print $c $buf }
+	print $c " HTTP/1.0\r\n\r\n";
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 414 \b!,
 		'414 on REQUEST_PATH > (12 * 1024)');
 
 	$c = tcp_connect($srv);
-	print $c 'GET /hello-world?a' or die $!;
+	print $c 'GET /hello-world?a';
 	$buf = join('', (0..9));
-	for (0..1023) { print $c $buf or die $! }
-	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	for (0..1023) { print $c $buf }
+	print $c " HTTP/1.0\r\n\r\n";
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 414 \b!,
 		'414 on QUERY_STRING > (10 * 1024)');
 
 	$c = tcp_connect($srv);
-	print $c 'GET /hello-world#a' or die $!;
+	print $c 'GET /hello-world#a';
 	$buf = join('', (0..9), 'a'..'f');
-	for (0..63) { print $c $buf or die $! }
-	print $c " HTTP/1.0\r\n\r\n" or die $!;
+	for (0..63) { print $c $buf }
+	print $c " HTTP/1.0\r\n\r\n";
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 414 \b!, '414 on FRAGMENT > (1024)');
 }
@@ -159,7 +160,7 @@ if ('bad requests') {
 # input tests
 my ($blob_size, $blob_hash);
 SKIP: {
-	open(my $rh, '<', 't/random_blob') or
+	CORE::open(my $rh, '<', 't/random_blob') or
 		skip "t/random_blob not generated $!", 1;
 	$blob_size = -s $rh;
 	require Digest::SHA;
@@ -167,11 +168,11 @@ SKIP: {
 
 	my $ck_hash = sub {
 		my ($sub, $path, %opt) = @_;
-		seek($rh, 0, SEEK_SET) // die "seek: $!";
+		seek($rh, 0, SEEK_SET);
 		$c = tcp_connect($srv);
 		$c->autoflush(0);
 		$PUT{$sub}->($rh, $c, $path, %opt);
-		$c->flush or die "flush: $!";
+		$c->flush or die $!;
 		($status, $hdr) = slurp_hdr($c);
 		is(readline($c), $blob_hash, "$sub $path");
 	};
@@ -189,10 +190,10 @@ SKIP: {
 	my $url = "http://$host_port/rack_input";
 	my $do_curl = sub {
 		my (@arg) = @_;
-		pipe(my $cout, $copt->{1}) or die "pipe: $!";
-		open $copt->{2}, '>', "$tmpdir/curl.err" or die $!;
+		pipe(my $cout, $copt->{1});
+		open $copt->{2}, '>', "$tmpdir/curl.err";
 		my $cpid = spawn($curl, '-sSf', @arg, $url, $copt);
-		close(delete $copt->{1}) or die "close: $!";
+		close(delete $copt->{1});
 		is(readline($cout), $blob_hash, "curl @arg response");
 		is(waitpid($cpid, 0), $cpid, "curl @arg exited");
 		is($?, 0, "no error from curl @arg");
@@ -201,7 +202,7 @@ SKIP: {
 
 	$do_curl->(qw(-T t/random_blob));
 
-	seek($rh, 0, SEEK_SET) // die "seek: $!";
+	seek($rh, 0, SEEK_SET);
 	$copt->{0} = $rh;
 	$do_curl->('-T-');
 }
diff --git a/t/lib.perl b/t/lib.perl
index ae9f197..49632cf 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -4,6 +4,7 @@
 package UnicornTest;
 use v5.14;
 use parent qw(Exporter);
+use autodie;
 use Test::More;
 use IO::Socket::INET;
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
@@ -14,7 +15,7 @@ our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
-open($errfh, '>>', "$tmpdir/err.log") or die "open: $!";
+open($errfh, '>>', "$tmpdir/err.log");
 
 sub tcp_server {
 	my %opt = (
@@ -62,14 +63,14 @@ sub tcp_connect {
 sub start_req {
 	my ($srv, @req) = @_;
 	my $c = tcp_connect($srv);
-	print $c @req, "\r\n\r\n" or die "print: $!";
+	print $c @req, "\r\n\r\n";
 	$c;
 }
 
 sub slurp {
-	open my $fh, '<', $_[0] or die "open($_[0]): $!";
+	open my $fh, '<', $_[0];
 	local $/;
-	<$fh>;
+	readline($fh);
 }
 
 sub spawn {
@@ -80,8 +81,8 @@ sub spawn {
 	my $set = POSIX::SigSet->new;
 	$set->fillset or die "sigfillset: $!";
 	sigprocmask(SIG_SETMASK, $set, $old) or die "SIG_SETMASK: $!";
-	pipe(my ($r, $w)) or die "pipe: $!";
-	my $pid = fork // die "fork: $!";
+	pipe(my $r, my $w);
+	my $pid = fork;
 	if ($pid == 0) {
 		close $r;
 		$SIG{__DIE__} = sub {
@@ -94,9 +95,9 @@ sub spawn {
 		my $cfd;
 		for ($cfd = 0; ($cfd < 3) || defined($opt->{$cfd}); $cfd++) {
 			my $io = $opt->{$cfd} // next;
-			my $pfd = fileno($io) // die "fileno($io): $!";
+			my $pfd = fileno($io);
 			if ($pfd == $cfd) {
-				fcntl($io, F_SETFD, 0) // die "F_SETFD: $!";
+				fcntl($io, F_SETFD, 0);
 			} else {
 				dup2($pfd, $cfd) // die "dup2($pfd, $cfd): $!";
 			}
@@ -110,9 +111,7 @@ sub spawn {
 			setpgid(0, $pgid) // die "setpgid(0, $pgid): $!";
 		}
 		$SIG{$_} = 'DEFAULT' for grep(!/^__/, keys %SIG);
-		if (defined(my $cd = $opt->{-C})) {
-			chdir $cd // die "chdir($cd): $!";
-		}
+		if (defined(my $cd = $opt->{-C})) { chdir $cd }
 		$old->delset(POSIX::SIGCHLD) or die "sigdelset CHLD: $!";
 		sigprocmask(SIG_SETMASK, $old) or die "SIG_SETMASK: ~CHLD: $!";
 		@ENV{keys %$env} = values(%$env) if $env;
@@ -162,22 +161,23 @@ sub unicorn {
 # automatically kill + reap children when this goes out-of-scope
 package UnicornTest::AutoReap;
 use v5.14;
+use autodie;
 
 sub new {
 	my (undef, $pid) = @_;
 	bless { pid => $pid, owner => $$ }, __PACKAGE__
 }
 
-sub kill {
+sub do_kill {
 	my ($self, $sig) = @_;
-	CORE::kill($sig // 'TERM', $self->{pid});
+	kill($sig // 'TERM', $self->{pid});
 }
 
 sub join {
 	my ($self, $sig) = @_;
 	my $pid = delete $self->{pid} or return;
-	CORE::kill($sig, $pid) if defined $sig;
-	my $ret = waitpid($pid, 0) // die "waitpid($pid): $!";
+	kill($sig, $pid) if defined $sig;
+	my $ret = waitpid($pid, 0);
 	$ret == $pid or die "BUG: waitpid($pid) != $ret";
 }
 

[-- Attachment #11: 0010-port-t0019-max_header_len.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 5571 bytes --]

From 43c7d73b8b9e6995b5a986b10a8623395e89a538 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:39 +0000
Subject: [PATCH 10/23] port t0019-max_header_len.sh to Perl 5

This was the final socat requirement for integration tests.
I think curl will remain an optional dependency for tests
since it's probably the most widely-installed HTTP client.
---
 GNUmakefile               |  2 +-
 t/README                  |  7 +-----
 t/integration.ru          |  1 +
 t/integration.t           | 43 +++++++++++++++++++++++++++++++---
 t/t0019-max_header_len.sh | 49 ---------------------------------------
 5 files changed, 43 insertions(+), 59 deletions(-)
 delete mode 100755 t/t0019-max_header_len.sh

diff --git a/GNUmakefile b/GNUmakefile
index 5cca189..eab9082 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -125,7 +125,7 @@ $(T_sh): dep $(test_prereq) t/random_blob t/trash/.gitignore
 t/trash/.gitignore : | t/trash
 	echo '*' >$@
 
-dependencies := socat curl
+dependencies := curl
 deps := $(addprefix t/.dep+,$(dependencies))
 $(deps): dep_bin = $(lastword $(subst +, ,$@))
 $(deps):
diff --git a/t/README b/t/README
index 8a5243e..d09c715 100644
--- a/t/README
+++ b/t/README
@@ -10,18 +10,13 @@ to test real-world behavior and Ruby introduces incompatibilities
 at a far faster rate than Perl 5.  Perl is Ruby's older cousin, so
 it should be easy-to-learn for Rubyists.
 
-Old tests are in Bourne shell, but the socat(1) dependency was probably
-too rare compared to Perl 5.
+Old tests are in Bourne shell and slowly being ported to Perl 5.
 
 == Requirements
 
 * {Ruby 2.0.0+}[https://www.ruby-lang.org/en/]
 * {Perl 5.14+}[https://www.perl.org/] # your distro should have it
 * {GNU make}[https://www.gnu.org/software/make/]
-
-The following requirements will eventually be dropped.
-
-* {socat}[http://www.dest-unreach.org/socat/]
 * {curl}[https://curl.haxx.se/]
 
 We do not use bashisms or any non-portable, non-POSIX constructs
diff --git a/t/integration.ru b/t/integration.ru
index 98528f6..edc408c 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -81,6 +81,7 @@ def rack_input_tests(env)
     when '/env_dump'; [ 200, {}, [ env_dump(env) ] ]
     when '/write_on_close'; write_on_close
     when '/pid'; [ 200, {}, [ "#$$\n" ] ]
+    else '/'; [ 200, {}, [ env_dump(env) ] ]
     end # case PATH_INFO (GET)
   when 'POST'
     case env['PATH_INFO']
diff --git a/t/integration.t b/t/integration.t
index af17d51..c687655 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -1,15 +1,19 @@
 #!perl -w
 # Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
-# this is the main integration test for things which don't require
-# restarting or signals
+
+# This is the main integration test for fast-ish things to minimize
+# Ruby startup time penalties.
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
 my $srv = tcp_server();
 my $host_port = tcp_host_port($srv);
 my $t0 = time;
-my $ar = unicorn(qw(-E none t/integration.ru), { 3 => $srv });
+my $conf = "$tmpdir/u.conf.rb";
+open my $conf_fh, '>', $conf;
+$conf_fh->autoflush(1);
+my $ar = unicorn(qw(-E none t/integration.ru -c), $conf, { 3 => $srv });
 my $curl = which('curl');
 END { diag slurp("$tmpdir/err.log") if $tmpdir };
 sub slurp_hdr {
@@ -207,7 +211,40 @@ SKIP: {
 	$do_curl->('-T-');
 }
 
+
 # ... more stuff here
+
+# SIGHUP-able stuff goes here
+
+if ('max_header_len internal API') {
+	undef $c;
+	my $req = 'GET / HTTP/1.0';
+	my $len = length($req."\r\n\r\n");
+	my $fifo = "$tmpdir/fifo";
+	POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
+	print $conf_fh <<EOM;
+Unicorn::HttpParser.max_header_len = $len
+listen "$host_port" # TODO: remove this requirement for SIGHUP
+after_fork { |_,_| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
+EOM
+	$ar->do_kill('HUP');
+	open my $fifo_fh, '<', $fifo;
+	my $wpid = readline($fifo_fh);
+	like($wpid, qr/\Apid=\d+\z/a , 'new worker ready');
+	close $fifo_fh;
+	$wpid =~ s/\Apid=// or die;
+	ok(CORE::kill(0, $wpid), 'worker PID retrieved');
+
+	$c = start_req($srv, $req);
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 200\b!, 'minimal request succeeds');
+
+	$c = start_req($srv, 'GET /xxxxxx HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 413\b!, 'big request fails');
+}
+
+
 undef $ar;
 my @log = slurp("$tmpdir/err.log");
 diag("@log") if $ENV{V};
diff --git a/t/t0019-max_header_len.sh b/t/t0019-max_header_len.sh
deleted file mode 100755
index 6a355b4..0000000
--- a/t/t0019-max_header_len.sh
+++ /dev/null
@@ -1,49 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 5 "max_header_len setting (only intended for Rainbows!)"
-
-t_begin "setup and start" && {
-	unicorn_setup
-	req='GET / HTTP/1.0\r\n\r\n'
-	len=$(printf "$req" | count_bytes)
-	echo Unicorn::HttpParser.max_header_len = $len >> $unicorn_config
-	unicorn -D -c $unicorn_config env.ru
-	unicorn_wait_start
-}
-
-t_begin "minimal request succeeds" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf "$req"
-		wait
-		echo ok > $ok
-	) | socat - TCP:$listen > $fifo
-	test xok = x$(cat $ok)
-
-	fgrep "HTTP/1.1 200 OK" $tmp
-}
-
-t_begin "big request fails" && {
-	rm -f $tmp
-	(
-		cat $fifo > $tmp &
-		printf 'GET /xxxxxx HTTP/1.0\r\n\r\n'
-		wait
-		echo ok > $ok
-	) | socat - TCP:$listen > $fifo
-	test xok = x$(cat $ok)
-	fgrep "HTTP/1.1 413" $tmp
-}
-
-dbgcat tmp
-
-t_begin "killing succeeds" && {
-	kill $unicorn_pid
-}
-
-t_begin "check stderr" && {
-	check_stderr
-}
-
-t_done

[-- Attachment #12: 0011-test_exec-drop-sd_listen_fds-emulation-test.patch --]
[-- Type: text/x-diff, Size: 1751 bytes --]

From 5d828a4ef7683345bcf2ff659442fed0a6fb7a97 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:40 +0000
Subject: [PATCH 11/23] test_exec: drop sd_listen_fds emulation test

The Perl 5 tests already rely on this implicitly, and there was
never a point when Perl 5 couldn't emulate systemd behavior.
---
 test/exec/test_exec.rb | 33 ---------------------------------
 1 file changed, 33 deletions(-)

diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index 2929b2e..1d3a0fd 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -97,39 +97,6 @@ def teardown
     end
   end
 
-  def test_sd_listen_fds_emulation
-    # [ruby-core:69895] [Bug #11336] fixed by r51576
-    return if RUBY_VERSION.to_f < 2.3
-
-    File.open("config.ru", "wb") { |fp| fp.write(HI) }
-    sock = TCPServer.new(@addr, @port)
-
-    [ %W(-l #@addr:#@port), nil ].each do |l|
-      sock.setsockopt(:SOL_SOCKET, :SO_KEEPALIVE, 0)
-
-      pid = xfork do
-        redirect_test_io do
-          # pretend to be systemd
-          ENV['LISTEN_PID'] = "#$$"
-          ENV['LISTEN_FDS'] = '1'
-
-          # 3 = SD_LISTEN_FDS_START
-          args = [ $unicorn_bin ]
-          args.concat(l) if l
-          args << { 3 => sock }
-          exec(*args)
-        end
-      end
-      res = hit(["http://#@addr:#@port/"])
-      assert_equal [ "HI\n" ], res
-      assert_shutdown(pid)
-      assert sock.getsockopt(:SOL_SOCKET, :SO_KEEPALIVE).bool,
-                  'unicorn should always set SO_KEEPALIVE on inherited sockets'
-    end
-  ensure
-    sock.close if sock
-  end
-
   def test_inherit_listener_unspecified
     File.open("config.ru", "wb") { |fp| fp.write(HI) }
     sock = TCPServer.new(@addr, @port)

[-- Attachment #13: 0012-test_exec-drop-test_basic-and-test_config_ru_alt_pat.patch --]
[-- Type: text/x-diff, Size: 1667 bytes --]

From 548593c6b3d52a4bebd52542ad9c423ed2b7252d Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:41 +0000
Subject: [PATCH 12/23] test_exec: drop test_basic and test_config_ru_alt_path

We already have coverage for these basic things elsewhere.
---
 test/exec/test_exec.rb | 24 ------------------------
 1 file changed, 24 deletions(-)

diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index 1d3a0fd..55f828e 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -265,16 +265,6 @@ def test_exit_signals
     end
   end
 
-  def test_basic
-    File.open("config.ru", "wb") { |fp| fp.syswrite(HI) }
-    pid = fork do
-      redirect_test_io { exec($unicorn_bin, "-l", "#{@addr}:#{@port}") }
-    end
-    results = retry_hit(["http://#{@addr}:#{@port}/"])
-    assert_equal String, results[0].class
-    assert_shutdown(pid)
-  end
-
   def test_rack_env_unset
     File.open("config.ru", "wb") { |fp| fp.syswrite(SHOW_RACK_ENV) }
     pid = fork { redirect_test_io { exec($unicorn_bin, "-l#@addr:#@port") } }
@@ -638,20 +628,6 @@ def test_read_embedded_cli_switches
     assert_shutdown(pid)
   end
 
-  def test_config_ru_alt_path
-    config_path = "#{@tmpdir}/foo.ru"
-    File.open(config_path, "wb") { |fp| fp.syswrite(HI) }
-    pid = fork do
-      redirect_test_io do
-        Dir.chdir("/")
-        exec($unicorn_bin, "-l#{@addr}:#{@port}", config_path)
-      end
-    end
-    results = retry_hit(["http://#{@addr}:#{@port}/"])
-    assert_equal String, results[0].class
-    assert_shutdown(pid)
-  end
-
   def test_load_module
     libdir = "#{@tmpdir}/lib"
     FileUtils.mkpath([ libdir ])

[-- Attachment #14: 0013-tests-check_stderr-consistently-in-Perl-5-tests.patch --]
[-- Type: text/x-diff, Size: 2415 bytes --]

From cd7ee67fc8ebadec9bdd913d49ed3f214596ea47 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:42 +0000
Subject: [PATCH 13/23] tests: check_stderr consistently in Perl 5 tests

The Bourne shell tests did, so lets not let stuff sneak past us.
---
 t/active-unix-socket.t |  5 ++---
 t/integration.t        |  7 ++-----
 t/lib.perl             | 10 +++++++++-
 3 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index 1241904..c132dc2 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -20,7 +20,7 @@ my $unix_req = sub {
 	print $fh <<EOM;
 pid "$tmpdir/u.pid"
 listen "$u1"
-stderr_path "$tmpdir/err1.log"
+stderr_path "$tmpdir/err.log"
 EOM
 	close $fh;
 
@@ -113,6 +113,5 @@ is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
 	ok(-S $u1, 'socket stays after SIGTERM');
 }
 
-my @log = slurp("$tmpdir/err.log");
-diag("@log") if $ENV{V};
+check_stderr;
 done_testing;
diff --git a/t/integration.t b/t/integration.t
index c687655..939dc24 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -246,11 +246,8 @@ EOM
 
 
 undef $ar;
-my @log = slurp("$tmpdir/err.log");
-diag("@log") if $ENV{V};
-my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
-is_deeply(\@err, [], 'no unexpected errors in stderr');
-is_deeply([grep(/SIGKILL/, @log)], [], 'no SIGKILL in stderr');
+
+check_stderr;
 
 undef $tmpdir;
 done_testing;
diff --git a/t/lib.perl b/t/lib.perl
index 49632cf..315ef2d 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -11,12 +11,20 @@ use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
-	SEEK_SET tcp_host_port start_req which spawn);
+	SEEK_SET tcp_host_port start_req which spawn check_stderr);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
 open($errfh, '>>', "$tmpdir/err.log");
 
+sub check_stderr () {
+	my @log = slurp("$tmpdir/err.log");
+	diag("@log") if $ENV{V};
+	my @err = grep(!/NameError.*Unicorn::Waiter/, grep(/error/i, @log));
+	is_deeply(\@err, [], 'no unexpected errors in stderr');
+	is_deeply([grep(/SIGKILL/, @log)], [], 'no SIGKILL in stderr');
+}
+
 sub tcp_server {
 	my %opt = (
 		ReuseAddr => 1,

[-- Attachment #15: 0014-tests-consistent-tcp_start-and-unix_start-across-Per.patch --]
[-- Type: text/x-diff, Size: 8017 bytes --]

From 0dcd8bd569813a175ad43837db3ab07019a95b99 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:43 +0000
Subject: [PATCH 14/23] tests: consistent tcp_start and unix_start across Perl
 5 tests

I'll be using Unix sockets more in tests since there's no
risk of system-wide conflicts with TCP port allocation.
Furthermore, curl supports `--unix-socket' nowadays; so
there's little reason to rely on TCP sockets and the conflicts
they bring in tests.
---
 t/active-unix-socket.t | 13 ++++---------
 t/integration.t        | 28 ++++++++++++++--------------
 t/lib.perl             | 30 ++++++++++++++++--------------
 3 files changed, 34 insertions(+), 37 deletions(-)

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index c132dc2..8723137 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -10,11 +10,6 @@ my %to_kill;
 END { kill('TERM', values(%to_kill)) if keys %to_kill }
 my $u1 = "$tmpdir/u1.sock";
 my $u2 = "$tmpdir/u2.sock";
-my $unix_req = sub {
-	my $s = IO::Socket::UNIX->new(Peer => shift, Type => SOCK_STREAM);
-	print $s @_, "\r\n\r\n";
-	$s;
-};
 {
 	open my $fh, '>', "$tmpdir/u1.conf.rb";
 	print $fh <<EOM;
@@ -53,7 +48,7 @@ is($?, 0, 'daemonized 1st process');
 chomp($to_kill{u1} = slurp("$tmpdir/u.pid"));
 like($to_kill{u1}, qr/\A\d+\z/s, 'read pid file');
 
-chomp(my $worker_pid = readline($unix_req->($u1, 'GET /pid')));
+chomp(my $worker_pid = readline(unix_start($u1, 'GET /pid')));
 like($worker_pid, qr/\A\d+\z/s, 'captured worker pid');
 ok(kill(0, $worker_pid), 'worker is kill-able');
 
@@ -65,7 +60,7 @@ isnt($?, 0, 'conflicting PID file fails to start');
 chomp(my $pidf = slurp("$tmpdir/u.pid"));
 is($pidf, $to_kill{u1}, 'pid file contents unchanged after start failure');
 
-chomp(my $pid2 = readline($unix_req->($u1, 'GET /pid')));
+chomp(my $pid2 = readline(unix_start($u1, 'GET /pid')));
 is($worker_pid, $pid2, 'worker PID unchanged');
 
 
@@ -73,7 +68,7 @@ is($worker_pid, $pid2, 'worker PID unchanged');
 unicorn('-c', "$tmpdir/u3.conf.rb", @uarg)->join;
 isnt($?, 0, 'conflicting UNIX socket fails to start');
 
-chomp($pid2 = readline($unix_req->($u1, 'GET /pid')));
+chomp($pid2 = readline(unix_start($u1, 'GET /pid')));
 is($worker_pid, $pid2, 'worker PID still unchanged');
 
 chomp($pidf = slurp("$tmpdir/u.pid"));
@@ -101,7 +96,7 @@ is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
 	chomp($to_kill{u1} = slurp("$tmpdir/u.pid"));
 	like($to_kill{u1}, qr/\A\d+\z/s, 'read pid file');
 
-	chomp($pid2 = readline($unix_req->($u1, 'GET /pid')));
+	chomp($pid2 = readline(unix_start($u1, 'GET /pid')));
 	like($pid2, qr/\A\d+\z/, 'worker running');
 
 	ok(kill('TERM', delete $to_kill{u1}), 'SIGTERM restarted daemon');
diff --git a/t/integration.t b/t/integration.t
index 939dc24..b33e3c3 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -70,7 +70,7 @@ EOM
 my ($c, $status, $hdr);
 
 # response header tests
-$c = start_req($srv, 'GET /rack-2-newline-headers HTTP/1.0');
+$c = tcp_start($srv, 'GET /rack-2-newline-headers HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
 my $orig_200_status = $status;
@@ -89,7 +89,7 @@ SKIP: { # Date header check
 };
 
 
-$c = start_req($srv, 'GET /rack-3-array-headers HTTP/1.0');
+$c = tcp_start($srv, 'GET /rack-3-array-headers HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 is_deeply([ grep(/^x-r3: /, @$hdr) ],
 	[ 'x-r3: a', 'x-r3: b', 'x-r3: c' ],
@@ -97,7 +97,7 @@ is_deeply([ grep(/^x-r3: /, @$hdr) ],
 
 SKIP: {
 	eval { require JSON::PP } or skip "JSON::PP missing: $@", 1;
-	my $c = start_req($srv, 'GET /env_dump');
+	my $c = tcp_start($srv, 'GET /env_dump');
 	my $json = do { local $/; readline($c) };
 	unlike($json, qr/^Connection: /smi, 'no connection header for 0.9');
 	unlike($json, qr!\AHTTP/!s, 'no HTTP/1.x prefix for 0.9');
@@ -107,17 +107,17 @@ SKIP: {
 }
 
 # cf. <CAO47=rJa=zRcLn_Xm4v2cHPr6c0UswaFC_omYFEH+baSxHOWKQ@mail.gmail.com>
-$c = start_req($srv, 'GET /nil-header-value HTTP/1.0');
+$c = tcp_start($srv, 'GET /nil-header-value HTTP/1.0');
 ($status, $hdr) = slurp_hdr($c);
 is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
 	'nil header value accepted for broken apps') or diag(explain($hdr));
 
 if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
-	$c = start_req($srv, 'POST /tweak-status-code HTTP/1.0');
+	$c = tcp_start($srv, 'POST /tweak-status-code HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 200 HI\b!, 'status tweaked');
 
-	$c = start_req($srv, 'POST /restore-status-code HTTP/1.0');
+	$c = tcp_start($srv, 'POST /restore-status-code HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
 	is($status, $orig_200_status, 'original status restored');
 }
@@ -130,12 +130,12 @@ SKIP: {
 }
 
 if ('bad requests') {
-	$c = start_req($srv, 'GET /env_dump HTTP/1/1');
+	$c = tcp_start($srv, 'GET /env_dump HTTP/1/1');
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 400 \b!, 'got 400 on bad request');
 
-	$c = tcp_connect($srv);
-	print $c 'GET /';
+	$c = tcp_start($srv);
+	print $c 'GET /';;
 	my $buf = join('', (0..9), 'ab');
 	for (0..1023) { print $c $buf }
 	print $c " HTTP/1.0\r\n\r\n";
@@ -143,7 +143,7 @@ if ('bad requests') {
 	like($status, qr!\AHTTP/1\.[01] 414 \b!,
 		'414 on REQUEST_PATH > (12 * 1024)');
 
-	$c = tcp_connect($srv);
+	$c = tcp_start($srv);
 	print $c 'GET /hello-world?a';
 	$buf = join('', (0..9));
 	for (0..1023) { print $c $buf }
@@ -152,7 +152,7 @@ if ('bad requests') {
 	like($status, qr!\AHTTP/1\.[01] 414 \b!,
 		'414 on QUERY_STRING > (10 * 1024)');
 
-	$c = tcp_connect($srv);
+	$c = tcp_start($srv);
 	print $c 'GET /hello-world#a';
 	$buf = join('', (0..9), 'a'..'f');
 	for (0..63) { print $c $buf }
@@ -173,7 +173,7 @@ SKIP: {
 	my $ck_hash = sub {
 		my ($sub, $path, %opt) = @_;
 		seek($rh, 0, SEEK_SET);
-		$c = tcp_connect($srv);
+		$c = tcp_start($srv);
 		$c->autoflush(0);
 		$PUT{$sub}->($rh, $c, $path, %opt);
 		$c->flush or die $!;
@@ -235,11 +235,11 @@ EOM
 	$wpid =~ s/\Apid=// or die;
 	ok(CORE::kill(0, $wpid), 'worker PID retrieved');
 
-	$c = start_req($srv, $req);
+	$c = tcp_start($srv, $req);
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 200\b!, 'minimal request succeeds');
 
-	$c = start_req($srv, 'GET /xxxxxx HTTP/1.0');
+	$c = tcp_start($srv, 'GET /xxxxxx HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
 	like($status, qr!\AHTTP/1\.[01] 413\b!, 'big request fails');
 }
diff --git a/t/lib.perl b/t/lib.perl
index 315ef2d..1d6e78d 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -10,8 +10,8 @@ use IO::Socket::INET;
 use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
-our @EXPORT = qw(unicorn slurp tcp_server tcp_connect unicorn $tmpdir $errfh
-	SEEK_SET tcp_host_port start_req which spawn check_stderr);
+our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn $tmpdir $errfh
+	SEEK_SET tcp_host_port which spawn check_stderr unix_start);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
@@ -55,26 +55,28 @@ sub tcp_host_port {
 	}
 }
 
-sub tcp_connect {
-	my ($dest, %opt) = @_;
-	my $addr = tcp_host_port($dest);
-	my $s = ref($dest)->new(
+sub unix_start ($@) {
+	my ($dst, @req) = @_;
+	my $s = IO::Socket::UNIX->new(Peer => $dst, Type => SOCK_STREAM) or
+		BAIL_OUT "unix connect $dst: $!";
+	$s->autoflush(1);
+	print $s @req, "\r\n\r\n" if @req;
+	$s;
+}
+
+sub tcp_start ($@) {
+	my ($dst, @req) = @_;
+	my $addr = tcp_host_port($dst);
+	my $s = ref($dst)->new(
 		Proto => 'tcp',
 		Type => SOCK_STREAM,
 		PeerAddr => $addr,
-		%opt,
 	) or BAIL_OUT "failed to connect to $addr: $!";
 	$s->autoflush(1);
+	print $s @req, "\r\n\r\n" if @req;
 	$s;
 }
 
-sub start_req {
-	my ($srv, @req) = @_;
-	my $c = tcp_connect($srv);
-	print $c @req, "\r\n\r\n";
-	$c;
-}
-
 sub slurp {
 	open my $fh, '<', $_[0];
 	local $/;

[-- Attachment #16: 0015-port-t9000-preread-input.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 3856 bytes --]

From 1b8840d8d13491eecd2fa92e06f73c65eadd33ba Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:44 +0000
Subject: [PATCH 15/23] port t9000-preread-input.sh to Perl 5

Stuffing it into t/integration.t for now so we can save on
startup costs.
---
 t/integration.t          | 32 ++++++++++++++++++++++++---
 t/lib.perl               |  2 +-
 t/preread_input.ru       |  4 +---
 t/t9000-preread-input.sh | 48 ----------------------------------------
 4 files changed, 31 insertions(+), 55 deletions(-)
 delete mode 100755 t/t9000-preread-input.sh

diff --git a/t/integration.t b/t/integration.t
index b33e3c3..f5afd5d 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -7,8 +7,8 @@
 
 use v5.14; BEGIN { require './t/lib.perl' };
 use autodie;
-my $srv = tcp_server();
-my $host_port = tcp_host_port($srv);
+our $srv = tcp_server();
+our $host_port = tcp_host_port($srv);
 my $t0 = time;
 my $conf = "$tmpdir/u.conf.rb";
 open my $conf_fh, '>', $conf;
@@ -209,8 +209,34 @@ SKIP: {
 	seek($rh, 0, SEEK_SET);
 	$copt->{0} = $rh;
 	$do_curl->('-T-');
-}
 
+	diag 'testing Unicorn::PrereadInput...';
+	local $srv = tcp_server();
+	local $host_port = tcp_host_port($srv);
+	check_stderr;
+	truncate($errfh, 0);
+
+	my $pri = unicorn(qw(-E none t/preread_input.ru), { 3 => $srv });
+	$url = "http://$host_port/";
+
+	$do_curl->(qw(-T t/random_blob));
+	seek($rh, 0, SEEK_SET);
+	$copt->{0} = $rh;
+	$do_curl->('-T-');
+
+	my @pr_err = slurp("$tmpdir/err.log");
+	is(scalar(grep(/app dispatch:/, @pr_err)), 2, 'app dispatched twice');
+
+	# abort a chunked request by blocking curl on a FIFO:
+	$c = tcp_start($srv, "PUT / HTTP/1.1\r\nTransfer-Encoding: chunked");
+	close $c;
+	@pr_err = slurp("$tmpdir/err.log");
+	is(scalar(grep(/app dispatch:/, @pr_err)), 2,
+			'app did not dispatch on aborted request');
+	undef $pri;
+	check_stderr;
+	diag 'Unicorn::PrereadInput middleware tests done';
+}
 
 # ... more stuff here
 
diff --git a/t/lib.perl b/t/lib.perl
index 1d6e78d..b6148cf 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -79,7 +79,7 @@ sub tcp_start ($@) {
 
 sub slurp {
 	open my $fh, '<', $_[0];
-	local $/;
+	local $/ if !wantarray;
 	readline($fh);
 }
 
diff --git a/t/preread_input.ru b/t/preread_input.ru
index 79685c4..f0a1748 100644
--- a/t/preread_input.ru
+++ b/t/preread_input.ru
@@ -1,8 +1,6 @@
 #\-E none
 require 'digest/sha1'
 require 'unicorn/preread_input'
-use Rack::ContentLength
-use Rack::ContentType, "text/plain"
 use Unicorn::PrereadInput
 nr = 0
 run lambda { |env|
@@ -13,5 +11,5 @@
     dig.update(buf)
   end
 
-  [ 200, {}, [ "#{dig.hexdigest}\n" ] ]
+  [ 200, {}, [ dig.hexdigest ] ]
 }
diff --git a/t/t9000-preread-input.sh b/t/t9000-preread-input.sh
deleted file mode 100755
index d6c73ab..0000000
--- a/t/t9000-preread-input.sh
+++ /dev/null
@@ -1,48 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 9 "PrereadInput middleware tests"
-
-t_begin "setup and start" && {
-	random_blob_sha1=$(rsha1 < random_blob)
-	unicorn_setup
-	unicorn  -D -c $unicorn_config preread_input.ru
-	unicorn_wait_start
-}
-
-t_begin "single identity request" && {
-	curl -sSf -T random_blob http://$listen/ > $tmp
-}
-
-t_begin "sha1 matches" && {
-	test x"$(cat $tmp)" = x"$random_blob_sha1"
-}
-
-t_begin "single chunked request" && {
-	curl -sSf -T- < random_blob http://$listen/ > $tmp
-}
-
-t_begin "sha1 matches" && {
-	test x"$(cat $tmp)" = x"$random_blob_sha1"
-}
-
-t_begin "app only dispatched twice" && {
-	test 2 -eq "$(grep 'app dispatch:' < $r_err | count_lines )"
-}
-
-t_begin "aborted chunked request" && {
-	rm -f $tmp
-	curl -sSf -T- < $fifo http://$listen/ > $tmp &
-	curl_pid=$!
-	kill -9 $curl_pid
-	wait
-}
-
-t_begin "app only dispatched twice" && {
-	test 2 -eq "$(grep 'app dispatch:' < $r_err | count_lines )"
-}
-
-t_begin "killing succeeds" && {
-	kill -QUIT $unicorn_pid
-}
-
-t_done

[-- Attachment #17: 0016-port-t-t0116-client_body_buffer_size.sh-to-Perl-5.patch --]
[-- Type: text/x-diff, Size: 8861 bytes --]

From e9593301044f305d4a0e074f77eea35015ca0ec4 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:45 +0000
Subject: [PATCH 16/23] port t/t0116-client_body_buffer_size.sh to Perl 5

While I'm fine with depending on curl for certain things,
there's no need for it here since unicorn has had lazy
rack.input for over a decade, at this point.
---
 t/active-unix-socket.t                     |  1 +
 t/{t0116.ru => client_body_buffer_size.ru} |  2 -
 t/client_body_buffer_size.t                | 83 ++++++++++++++++++++++
 t/integration.t                            | 10 ---
 t/lib.perl                                 | 12 +++-
 t/t0116-client_body_buffer_size.sh         | 80 ---------------------
 6 files changed, 95 insertions(+), 93 deletions(-)
 rename t/{t0116.ru => client_body_buffer_size.ru} (82%)
 create mode 100644 t/client_body_buffer_size.t
 delete mode 100755 t/t0116-client_body_buffer_size.sh

diff --git a/t/active-unix-socket.t b/t/active-unix-socket.t
index 8723137..4e11837 100644
--- a/t/active-unix-socket.t
+++ b/t/active-unix-socket.t
@@ -109,4 +109,5 @@ is($pidf, $to_kill{u1}, 'pid file contents unchanged after 2nd start failure');
 }
 
 check_stderr;
+undef $tmpdir;
 done_testing;
diff --git a/t/t0116.ru b/t/client_body_buffer_size.ru
similarity index 82%
rename from t/t0116.ru
rename to t/client_body_buffer_size.ru
index fab5fce..44161a5 100644
--- a/t/t0116.ru
+++ b/t/client_body_buffer_size.ru
@@ -1,6 +1,4 @@
 #\ -E none
-use Rack::ContentLength
-use Rack::ContentType, 'text/plain'
 app = lambda do |env|
   input = env['rack.input']
   case env["PATH_INFO"]
diff --git a/t/client_body_buffer_size.t b/t/client_body_buffer_size.t
new file mode 100644
index 0000000..b1a99f3
--- /dev/null
+++ b/t/client_body_buffer_size.t
@@ -0,0 +1,83 @@
+#!perl -w
+# Copyright (C) unicorn hackers <unicorn-public@yhbt.net>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+
+use v5.14; BEGIN { require './t/lib.perl' };
+use autodie;
+my $uconf = "$tmpdir/u.conf.rb";
+
+open my $conf_fh, '>', $uconf;
+$conf_fh->autoflush(1);
+print $conf_fh <<EOM;
+client_body_buffer_size 0
+EOM
+my $srv = tcp_server();
+my $host_port = tcp_host_port($srv);
+my @uarg = (qw(-E none t/client_body_buffer_size.ru -c), $uconf);
+my $ar = unicorn(@uarg, { 3 => $srv });
+my ($c, $status, $hdr);
+my $mem_class = 'StringIO';
+my $fs_class = 'Unicorn::TmpIO';
+
+$c = tcp_start($srv, "PUT /input_class HTTP/1.0\r\nContent-Length: 0");
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $mem_class, 'zero-byte file is StringIO');
+
+$c = tcp_start($srv, "PUT /tmp_class HTTP/1.0\r\nContent-Length: 1");
+print $c '.';
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $fs_class, '1 byte file is filesystem-backed');
+
+
+my $fifo = "$tmpdir/fifo";
+POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
+seek($conf_fh, 0, SEEK_SET);
+truncate($conf_fh, 0);
+print $conf_fh <<EOM;
+listen "$host_port" # TODO: remove this requirement for SIGHUP
+after_fork { |_,_| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
+EOM
+$ar->do_kill('HUP');
+open my $fifo_fh, '<', $fifo;
+like(my $wpid = readline($fifo_fh), qr/\Apid=\d+\z/a ,
+	'reloaded w/ default client_body_buffer_size');
+
+
+$c = tcp_start($srv, "PUT /tmp_class HTTP/1.0\r\nContent-Length: 1");
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $mem_class, 'class for a 1 byte file is memory-backed');
+
+
+my $one_meg = 1024 ** 2;
+$c = tcp_start($srv, "PUT /tmp_class HTTP/1.0\r\nContent-Length: $one_meg");
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $fs_class, '1 megabyte file is FS-backed');
+
+# reload with bigger client_body_buffer_size
+say $conf_fh "client_body_buffer_size $one_meg";
+$ar->do_kill('HUP');
+open $fifo_fh, '<', $fifo;
+like($wpid = readline($fifo_fh), qr/\Apid=\d+\z/a ,
+	'reloaded w/ bigger client_body_buffer_size');
+
+
+$c = tcp_start($srv, "PUT /tmp_class HTTP/1.0\r\nContent-Length: $one_meg");
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $mem_class, '1 megabyte file is now memory-backed');
+
+my $too_big = $one_meg + 1;
+$c = tcp_start($srv, "PUT /tmp_class HTTP/1.0\r\nContent-Length: $too_big");
+($status, $hdr) = slurp_hdr($c);
+like($status, qr!\AHTTP/1\.[01] 200\b!, 'status line valid');
+is(readline($c), $fs_class, '1 megabyte + 1 byte file is FS-backed');
+
+
+undef $ar;
+check_stderr;
+undef $tmpdir;
+done_testing;
diff --git a/t/integration.t b/t/integration.t
index f5afd5d..855c260 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -15,16 +15,6 @@ open my $conf_fh, '>', $conf;
 $conf_fh->autoflush(1);
 my $ar = unicorn(qw(-E none t/integration.ru -c), $conf, { 3 => $srv });
 my $curl = which('curl');
-END { diag slurp("$tmpdir/err.log") if $tmpdir };
-sub slurp_hdr {
-	my ($c) = @_;
-	local $/ = "\r\n\r\n"; # affects both readline+chomp
-	chomp(my $hdr = readline($c));
-	my ($status, @hdr) = split(/\r\n/, $hdr);
-	diag explain([ $status, \@hdr ]) if $ENV{V};
-	($status, \@hdr);
-}
-
 my %PUT = (
 	chunked_md5 => sub {
 		my ($in, $out, $path, %opt) = @_;
diff --git a/t/lib.perl b/t/lib.perl
index b6148cf..2685c3b 100644
--- a/t/lib.perl
+++ b/t/lib.perl
@@ -11,11 +11,12 @@ use POSIX qw(dup2 _exit setpgid :signal_h SEEK_SET F_SETFD);
 use File::Temp 0.19 (); # 0.19 for ->newdir
 our ($tmpdir, $errfh);
 our @EXPORT = qw(unicorn slurp tcp_server tcp_start unicorn $tmpdir $errfh
-	SEEK_SET tcp_host_port which spawn check_stderr unix_start);
+	SEEK_SET tcp_host_port which spawn check_stderr unix_start slurp_hdr);
 
 my ($base) = ($0 =~ m!\b([^/]+)\.[^\.]+\z!);
 $tmpdir = File::Temp->newdir("unicorn-$base-XXXX", TMPDIR => 1);
 open($errfh, '>>', "$tmpdir/err.log");
+END { diag slurp("$tmpdir/err.log") if $tmpdir };
 
 sub check_stderr () {
 	my @log = slurp("$tmpdir/err.log");
@@ -25,6 +26,15 @@ sub check_stderr () {
 	is_deeply([grep(/SIGKILL/, @log)], [], 'no SIGKILL in stderr');
 }
 
+sub slurp_hdr {
+	my ($c) = @_;
+	local $/ = "\r\n\r\n"; # affects both readline+chomp
+	chomp(my $hdr = readline($c));
+	my ($status, @hdr) = split(/\r\n/, $hdr);
+	diag explain([ $status, \@hdr ]) if $ENV{V};
+	($status, \@hdr);
+}
+
 sub tcp_server {
 	my %opt = (
 		ReuseAddr => 1,
diff --git a/t/t0116-client_body_buffer_size.sh b/t/t0116-client_body_buffer_size.sh
deleted file mode 100755
index c9e17c7..0000000
--- a/t/t0116-client_body_buffer_size.sh
+++ /dev/null
@@ -1,80 +0,0 @@
-#!/bin/sh
-. ./test-lib.sh
-t_plan 12 "client_body_buffer_size settings"
-
-t_begin "setup and start" && {
-	unicorn_setup
-	rtmpfiles unicorn_config_tmp one_meg
-	dd if=/dev/zero bs=1M count=1 of=$one_meg
-	cat >> $unicorn_config <<EOF
-after_fork do |server, worker|
-  File.open("$fifo", "wb") { |fp| fp.syswrite "START" }
-end
-EOF
-	cat $unicorn_config > $unicorn_config_tmp
-	echo client_body_buffer_size 0 >> $unicorn_config
-	unicorn -D -c $unicorn_config t0116.ru
-	unicorn_wait_start
-	fs_class=Unicorn::TmpIO
-	mem_class=StringIO
-
-	test x"$(cat $fifo)" = xSTART
-}
-
-t_begin "class for a zero-byte file should be StringIO" && {
-	> $tmp
-	test xStringIO = x"$(curl -T $tmp -sSf http://$listen/input_class)"
-}
-
-t_begin "class for a 1 byte file should be filesystem-backed" && {
-	echo > $tmp
-	test x$fs_class = x"$(curl -T $tmp -sSf http://$listen/tmp_class)"
-}
-
-t_begin "reload with default client_body_buffer_size" && {
-	mv $unicorn_config_tmp $unicorn_config
-	kill -HUP $unicorn_pid
-	test x"$(cat $fifo)" = xSTART
-}
-
-t_begin "class for a 1 byte file should be memory-backed" && {
-	echo > $tmp
-	test x$mem_class = x"$(curl -T $tmp -sSf http://$listen/tmp_class)"
-}
-
-t_begin "class for a random blob file should be filesystem-backed" && {
-	resp="$(curl -T random_blob -sSf http://$listen/tmp_class)"
-	test x$fs_class = x"$resp"
-}
-
-t_begin "one megabyte file should be filesystem-backed" && {
-	resp="$(curl -T $one_meg -sSf http://$listen/tmp_class)"
-	test x$fs_class = x"$resp"
-}
-
-t_begin "reload with a big client_body_buffer_size" && {
-	echo "client_body_buffer_size(1024 * 1024)" >> $unicorn_config
-	kill -HUP $unicorn_pid
-	test x"$(cat $fifo)" = xSTART
-}
-
-t_begin "one megabyte file should be memory-backed" && {
-	resp="$(curl -T $one_meg -sSf http://$listen/tmp_class)"
-	test x$mem_class = x"$resp"
-}
-
-t_begin "one megabyte + 1 byte file should be filesystem-backed" && {
-	echo >> $one_meg
-	resp="$(curl -T $one_meg -sSf http://$listen/tmp_class)"
-	test x$fs_class = x"$resp"
-}
-
-t_begin "killing succeeds" && {
-	kill $unicorn_pid
-}
-
-t_begin "check stderr" && {
-	check_stderr
-}
-
-t_done

[-- Attachment #18: 0017-tests-get-rid-of-sha1sum.rb-and-rsha1-sh-function.patch --]
[-- Type: text/x-diff, Size: 1255 bytes --]

From b47912160f2336dde3901e588cc23fb2c2f8d9dc Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:46 +0000
Subject: [PATCH 17/23] tests: get rid of sha1sum.rb and rsha1() sh function

These are no longer needed since Perl has long included
Digest::SHA
---
 t/bin/sha1sum.rb | 17 -----------------
 t/test-lib.sh    |  4 ----
 2 files changed, 21 deletions(-)
 delete mode 100755 t/bin/sha1sum.rb

diff --git a/t/bin/sha1sum.rb b/t/bin/sha1sum.rb
deleted file mode 100755
index 53d68ce..0000000
--- a/t/bin/sha1sum.rb
+++ /dev/null
@@ -1,17 +0,0 @@
-#!/usr/bin/env ruby
-# -*- encoding: binary -*-
-# Reads from stdin and outputs the SHA1 hex digest of the input
-
-require 'digest/sha1'
-$stdout.sync = $stderr.sync = true
-$stdout.binmode
-$stdin.binmode
-bs = 16384
-digest = Digest::SHA1.new
-if buf = $stdin.read(bs)
-  begin
-    digest.update(buf)
-  end while $stdin.read(bs, buf)
-end
-
-$stdout.syswrite("#{digest.hexdigest}\n")
diff --git a/t/test-lib.sh b/t/test-lib.sh
index e70d0c6..8613144 100644
--- a/t/test-lib.sh
+++ b/t/test-lib.sh
@@ -123,7 +123,3 @@ unicorn_wait_start () {
 	# no need to play tricks with FIFOs since we got "ready_pipe" now
 	unicorn_pid=$(cat $pid)
 }
-
-rsha1 () {
-	sha1sum.rb
-}

[-- Attachment #19: 0018-early_hints-supports-Rack-3-array-headers.patch --]
[-- Type: text/x-diff, Size: 4606 bytes --]

From 6ad9f4b54ee16ffecea7e16b710552b45db33a16 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:47 +0000
Subject: [PATCH 18/23] early_hints supports Rack 3 array headers

We can hoist out append_headers into a new method and use it in
both e103_response_write and http_response_write.

t/integration.t now tests early_hints with both possible
values of check_client_connection.
---
 t/integration.ru |  7 +++++++
 t/integration.t  | 47 ++++++++++++++++++++++++++++++++++++++++++-----
 2 files changed, 49 insertions(+), 5 deletions(-)

diff --git a/t/integration.ru b/t/integration.ru
index edc408c..dab384d 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -5,6 +5,11 @@
 # this goes for t/integration.t  We'll try to put as many tests
 # in here as possible to avoid startup overhead of Ruby.
 
+def early_hints(env, val)
+  env['rack.early_hints'].call('link' => val) # val may be ary or string
+  [ 200, {}, [ val.class.to_s ] ]
+end
+
 $orig_rack_200 = nil
 def tweak_status_code
   $orig_rack_200 = Rack::Utils::HTTP_STATUS_CODES[200]
@@ -81,6 +86,8 @@ def rack_input_tests(env)
     when '/env_dump'; [ 200, {}, [ env_dump(env) ] ]
     when '/write_on_close'; write_on_close
     when '/pid'; [ 200, {}, [ "#$$\n" ] ]
+    when '/early_hints_rack2'; early_hints(env, "r\n2")
+    when '/early_hints_rack3'; early_hints(env, %w(r 3))
     else '/'; [ 200, {}, [ env_dump(env) ] ]
     end # case PATH_INFO (GET)
   when 'POST'
diff --git a/t/integration.t b/t/integration.t
index 855c260..8433497 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -13,8 +13,16 @@ my $t0 = time;
 my $conf = "$tmpdir/u.conf.rb";
 open my $conf_fh, '>', $conf;
 $conf_fh->autoflush(1);
+my $u1 = "$tmpdir/u1";
+print $conf_fh <<EOM;
+early_hints true
+listen "$u1"
+listen "$host_port" # TODO: remove this requirement for SIGHUP
+EOM
 my $ar = unicorn(qw(-E none t/integration.ru -c), $conf, { 3 => $srv });
 my $curl = which('curl');
+my $fifo = "$tmpdir/fifo";
+POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
 my %PUT = (
 	chunked_md5 => sub {
 		my ($in, $out, $path, %opt) = @_;
@@ -102,6 +110,26 @@ $c = tcp_start($srv, 'GET /nil-header-value HTTP/1.0');
 is_deeply([grep(/^X-Nil:/, @$hdr)], ['X-Nil: '],
 	'nil header value accepted for broken apps') or diag(explain($hdr));
 
+my $ck_early_hints = sub {
+	my ($note) = @_;
+	$c = unix_start($u1, 'GET /early_hints_rack2 HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 103\b!, 'got 103 for rack 2 value');
+	is_deeply(['link: r', 'link: 2'], $hdr, 'rack 2 hints match '.$note);
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 200\b!, 'got 200 afterwards');
+	is(readline($c), 'String', 'early hints used a String for rack 2');
+
+	$c = unix_start($u1, 'GET /early_hints_rack3 HTTP/1.0');
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 103\b!, 'got 103 for rack 3');
+	is_deeply(['link: r', 'link: 3'], $hdr, 'rack 3 hints match '.$note);
+	($status, $hdr) = slurp_hdr($c);
+	like($status, qr!\AHTTP/1\.[01] 200\b!, 'got 200 afterwards');
+	is(readline($c), 'Array', 'early hints used a String for rack 3');
+};
+$ck_early_hints->('ccc off'); # we'll retest later
+
 if ('TODO: ensure Rack::Utils::HTTP_STATUS_CODES is available') {
 	$c = tcp_start($srv, 'POST /tweak-status-code HTTP/1.0');
 	($status, $hdr) = slurp_hdr($c);
@@ -154,6 +182,7 @@ if ('bad requests') {
 # input tests
 my ($blob_size, $blob_hash);
 SKIP: {
+	skip 'SKIP_EXPENSIVE on', 1 if $ENV{SKIP_EXPENSIVE};
 	CORE::open(my $rh, '<', 't/random_blob') or
 		skip "t/random_blob not generated $!", 1;
 	$blob_size = -s $rh;
@@ -232,16 +261,24 @@ SKIP: {
 
 # SIGHUP-able stuff goes here
 
+if ('check_client_connection') {
+	print $conf_fh <<EOM; # appending to existing
+check_client_connection true
+after_fork { |_,_| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
+EOM
+	$ar->do_kill('HUP');
+	open my $fifo_fh, '<', $fifo;
+	my $wpid = readline($fifo_fh);
+	like($wpid, qr/\Apid=\d+\z/a , 'new worker ready');
+	$ck_early_hints->('ccc on');
+}
+
 if ('max_header_len internal API') {
 	undef $c;
 	my $req = 'GET / HTTP/1.0';
 	my $len = length($req."\r\n\r\n");
-	my $fifo = "$tmpdir/fifo";
-	POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
-	print $conf_fh <<EOM;
+	print $conf_fh <<EOM; # appending to existing
 Unicorn::HttpParser.max_header_len = $len
-listen "$host_port" # TODO: remove this requirement for SIGHUP
-after_fork { |_,_| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
 EOM
 	$ar->do_kill('HUP');
 	open my $fifo_fh, '<', $fifo;

[-- Attachment #20: 0019-test_server-drop-early_hints-test.patch --]
[-- Type: text/x-diff, Size: 1737 bytes --]

From 3e6bc9fb589fd88469349a38a77704c3333623e0 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:48 +0000
Subject: [PATCH 19/23] test_server: drop early_hints test

t/integration.t already is more complete in that it tests
both Rack 2 and 3 along with both possible values of
check_client_connection.
---
 test/unit/test_server.rb | 31 -------------------------------
 1 file changed, 31 deletions(-)

diff --git a/test/unit/test_server.rb b/test/unit/test_server.rb
index fe98fcc..0a710d1 100644
--- a/test/unit/test_server.rb
+++ b/test/unit/test_server.rb
@@ -23,17 +23,6 @@ def call(env)
   end
 end
 
-class TestEarlyHintsHandler
-  def call(env)
-    while env['rack.input'].read(4096)
-    end
-    env['rack.early_hints'].call(
-      "Link" => "</style.css>; rel=preload; as=style\n</script.js>; rel=preload"
-    )
-    [200, { 'content-type' => 'text/plain' }, ['hello!\n']]
-  end
-end
-
 class TestRackAfterReply
   def initialize
     @called = false
@@ -112,26 +101,6 @@ def test_preload_app_config
     tmp.close!
   end
 
-  def test_early_hints
-    teardown
-    redirect_test_io do
-      @server = HttpServer.new(TestEarlyHintsHandler.new,
-                               :listeners => [ "127.0.0.1:#@port"],
-                               :early_hints => true)
-      @server.start
-    end
-
-    sock = tcp_socket('127.0.0.1', @port)
-    sock.syswrite("GET / HTTP/1.0\r\n\r\n")
-
-    responses = sock.read(4096)
-    assert_match %r{\AHTTP/1.[01] 103\b}, responses
-    assert_match %r{^Link: </style\.css>}, responses
-    assert_match %r{^Link: </script\.js>}, responses
-
-    assert_match %r{^HTTP/1.[01] 200\b}, responses
-  end
-
   def test_after_reply
     teardown
 

[-- Attachment #21: 0020-t-integration.t-switch-PUT-tests-to-MD5-reuse-buffer.patch --]
[-- Type: text/x-diff, Size: 3740 bytes --]

From cb826915cdd1881cbcfc1fb4e645d26244dfda71 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:49 +0000
Subject: [PATCH 20/23] t/integration.t: switch PUT tests to MD5, reuse buffers

MD5 is faster, and these tests aren't meant to be secure,
they're just for checking for data corruption.
Furthermore, Content-MD5 is a supported HTTP trailer and
we can verify that here to obsolete other tests.

Furthermore, we can reuse buffers on env['rack.input'].read
calls to avoid malloc(3) and GC overhead.

Combined, these give roughly a 3% speedup for t/integration.t
on my system.
---
 t/integration.ru   | 20 +++++++++++++++-----
 t/integration.t    |  5 ++---
 t/preread_input.ru | 17 ++++++++++++-----
 3 files changed, 29 insertions(+), 13 deletions(-)

diff --git a/t/integration.ru b/t/integration.ru
index dab384d..086126a 100644
--- a/t/integration.ru
+++ b/t/integration.ru
@@ -55,8 +55,8 @@ def env_dump(env)
 def rack_input_tests(env)
   return [ 100, {}, [] ] if /\A100-continue\z/i =~ env['HTTP_EXPECT']
   cap = 16384
-  require 'digest/sha1'
-  digest = Digest::SHA1.new
+  require 'digest/md5'
+  dig = Digest::MD5.new
   input = env['rack.input']
   case env['PATH_INFO']
   when '/rack_input/size_first'; input.size
@@ -68,11 +68,21 @@ def rack_input_tests(env)
   if buf = input.read(rand(cap))
     begin
       raise "#{buf.size} > #{cap}" if buf.size > cap
-      digest.update(buf)
+      dig.update(buf)
     end while input.read(rand(cap), buf)
+    buf.clear # remove this call if Ruby ever gets escape analysis
   end
-  [ 200, {'content-length' => '40', 'content-type' => 'text/plain'},
-    [ digest.hexdigest ] ]
+  h = { 'content-type' => 'text/plain' }
+  if env['HTTP_TRAILER'] =~ /\bContent-MD5\b/i
+    cmd5_b64 = env['HTTP_CONTENT_MD5'] or return [500, {}, ['No Content-MD5']]
+    cmd5_bin = cmd5_b64.unpack('m')[0]
+    if cmd5_bin != dig.digest
+      h['content-length'] = cmd5_b64.size.to_s
+      return [ 500, h, [ cmd5_b64 ] ]
+    end
+  end
+  h['content-length'] = '32'
+  [ 200, h, [ dig.hexdigest ] ]
 end
 
 run(lambda do |env|
diff --git a/t/integration.t b/t/integration.t
index 8433497..38a9675 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -27,7 +27,6 @@ my %PUT = (
 	chunked_md5 => sub {
 		my ($in, $out, $path, %opt) = @_;
 		my $bs = $opt{bs} // 16384;
-		require Digest::MD5;
 		my $dig = Digest::MD5->new;
 		print $out <<EOM;
 PUT $path HTTP/1.1\r
@@ -186,8 +185,8 @@ SKIP: {
 	CORE::open(my $rh, '<', 't/random_blob') or
 		skip "t/random_blob not generated $!", 1;
 	$blob_size = -s $rh;
-	require Digest::SHA;
-	$blob_hash = Digest::SHA->new(1)->addfile($rh)->hexdigest;
+	require Digest::MD5;
+	$blob_hash = Digest::MD5->new->addfile($rh)->hexdigest;
 
 	my $ck_hash = sub {
 		my ($sub, $path, %opt) = @_;
diff --git a/t/preread_input.ru b/t/preread_input.ru
index f0a1748..18af221 100644
--- a/t/preread_input.ru
+++ b/t/preread_input.ru
@@ -1,15 +1,22 @@
 #\-E none
-require 'digest/sha1'
+require 'digest/md5'
 require 'unicorn/preread_input'
 use Unicorn::PrereadInput
 nr = 0
 run lambda { |env|
   $stderr.write "app dispatch: #{nr += 1}\n"
   input = env["rack.input"]
-  dig = Digest::SHA1.new
-  while buf = input.read(16384)
-    dig.update(buf)
+  dig = Digest::MD5.new
+  if buf = input.read(16384)
+    begin
+      dig.update(buf)
+    end while input.read(16384, buf)
+    buf.clear # remove this call if Ruby ever gets escape analysis
+  end
+  if env['HTTP_TRAILER'] =~ /\bContent-MD5\b/i
+    cmd5_b64 = env['HTTP_CONTENT_MD5'] or return [500, {}, ['No Content-MD5']]
+    cmd5_bin = cmd5_b64.unpack('m')[0]
+    return [500, {}, [ cmd5_b64 ] ] if cmd5_bin != dig.digest
   end
-
   [ 200, {}, [ dig.hexdigest ] ]
 }

[-- Attachment #22: 0021-tests-move-test_upload.rb-tests-to-t-integration.t.patch --]
[-- Type: text/x-diff, Size: 12280 bytes --]

From 181e4b5b6339fc5e9c3ad7d3690b736f6bd038aa Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:50 +0000
Subject: [PATCH 21/23] tests: move test_upload.rb tests to t/integration.t

The overread tests are ported over, and checksumming alone
is enough to guard against data corruption.

Randomizing the size of `read' calls on the client side will
shake out any boundary bugs on the server side.
---
 t/integration.t          |  32 ++++-
 test/unit/test_upload.rb | 301 ---------------------------------------
 2 files changed, 27 insertions(+), 306 deletions(-)
 delete mode 100644 test/unit/test_upload.rb

diff --git a/t/integration.t b/t/integration.t
index 38a9675..a568758 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -26,7 +26,6 @@ POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
 my %PUT = (
 	chunked_md5 => sub {
 		my ($in, $out, $path, %opt) = @_;
-		my $bs = $opt{bs} // 16384;
 		my $dig = Digest::MD5->new;
 		print $out <<EOM;
 PUT $path HTTP/1.1\r
@@ -36,7 +35,7 @@ Trailer: Content-MD5\r
 EOM
 		my ($buf, $r);
 		while (1) {
-			$r = read($in, $buf, $bs);
+			$r = read($in, $buf, 999 + int(rand(0xffff)));
 			last if $r == 0;
 			printf $out "%x\r\n", length($buf);
 			print $out $buf, "\r\n";
@@ -46,15 +45,15 @@ EOM
 	},
 	identity => sub {
 		my ($in, $out, $path, %opt) = @_;
-		my $bs = $opt{bs} // 16384;
 		my $clen = $opt{-s} // -s $in;
 		print $out <<EOM;
 PUT $path HTTP/1.0\r
 Content-Length: $clen\r
 \r
 EOM
-		my ($buf, $r, $len);
+		my ($buf, $r, $len, $bs);
 		while ($clen) {
+			$bs = 999 + int(rand(0xffff));
 			$len = $clen > $bs ? $bs : $clen;
 			$r = read($in, $buf, $len);
 			die 'premature EOF' if $r == 0;
@@ -192,8 +191,10 @@ SKIP: {
 		my ($sub, $path, %opt) = @_;
 		seek($rh, 0, SEEK_SET);
 		$c = tcp_start($srv);
-		$c->autoflush(0);
+		$c->autoflush($opt{sync} // 0);
 		$PUT{$sub}->($rh, $c, $path, %opt);
+		defined($opt{overwrite}) and
+			print { $c } ('x' x $opt{overwrite});
 		$c->flush or die $!;
 		($status, $hdr) = slurp_hdr($c);
 		is(readline($c), $blob_hash, "$sub $path");
@@ -205,6 +206,27 @@ SKIP: {
 	$ck_hash->('chunked_md5', '/rack_input/size_first');
 	$ck_hash->('chunked_md5', '/rack_input/rewind_first');
 
+	$ck_hash->('identity', '/rack_input', -s => $blob_size, sync => 1);
+	$ck_hash->('chunked_md5', '/rack_input', sync => 1);
+
+	# ensure small overwrites don't get checksummed
+	$ck_hash->('identity', '/rack_input', -s => $blob_size,
+			overwrite => 1); # one extra byte
+
+	# excessive overwrite truncated
+	$c = tcp_start($srv);
+	$c->autoflush(0);
+	print $c "PUT /rack_input HTTP/1.0\r\nContent-Length: 1\r\n\r\n";
+	if (1) {
+		local $SIG{PIPE} = 'IGNORE';
+		my $buf = "\0" x 8192;
+		my $n = 0;
+		my $end = time + 5;
+		$! = 0;
+		while (print $c $buf and time < $end) { ++$n }
+		ok($!, 'overwrite truncated') or diag "n=$n err=$! ".time;
+	}
+	undef $c;
 
 	$curl // skip 'no curl found in PATH', 1;
 
diff --git a/test/unit/test_upload.rb b/test/unit/test_upload.rb
deleted file mode 100644
index 76e6c1c..0000000
--- a/test/unit/test_upload.rb
+++ /dev/null
@@ -1,301 +0,0 @@
-# -*- encoding: binary -*-
-
-# Copyright (c) 2009 Eric Wong
-require './test/test_helper'
-require 'digest/md5'
-
-include Unicorn
-
-class UploadTest < Test::Unit::TestCase
-
-  def setup
-    @addr = ENV['UNICORN_TEST_ADDR'] || '127.0.0.1'
-    @port = unused_port
-    @hdr = {'Content-Type' => 'text/plain', 'Content-Length' => '0'}
-    @bs = 4096
-    @count = 256
-    @server = nil
-
-    # we want random binary data to test 1.9 encoding-aware IO craziness
-    @random = File.open('/dev/urandom','rb')
-    @sha1 = Digest::SHA1.new
-    @sha1_app = lambda do |env|
-      input = env['rack.input']
-      resp = {}
-
-      @sha1.reset
-      while buf = input.read(@bs)
-        @sha1.update(buf)
-      end
-      resp[:sha1] = @sha1.hexdigest
-
-      # rewind and read again
-      input.rewind
-      @sha1.reset
-      while buf = input.read(@bs)
-        @sha1.update(buf)
-      end
-
-      if resp[:sha1] == @sha1.hexdigest
-        resp[:sysread_read_byte_match] = true
-      end
-
-      if expect_size = env['HTTP_X_EXPECT_SIZE']
-        if expect_size.to_i == input.size
-          resp[:expect_size_match] = true
-        end
-      end
-      resp[:size] = input.size
-      resp[:content_md5] = env['HTTP_CONTENT_MD5']
-
-      [ 200, @hdr.merge({'X-Resp' => resp.inspect}), [] ]
-    end
-  end
-
-  def teardown
-    redirect_test_io { @server.stop(false) } if @server
-    @random.close
-    reset_sig_handlers
-  end
-
-  def test_put
-    start_server(@sha1_app)
-    sock = tcp_socket(@addr, @port)
-    sock.syswrite("PUT / HTTP/1.0\r\nContent-Length: #{length}\r\n\r\n")
-    @count.times do |i|
-      buf = @random.sysread(@bs)
-      @sha1.update(buf)
-      sock.syswrite(buf)
-    end
-    read = sock.read.split(/\r\n/)
-    assert_equal "HTTP/1.1 200 OK", read[0]
-    resp = eval(read.grep(/^X-Resp: /).first.sub!(/X-Resp: /, ''))
-    assert_equal length, resp[:size]
-    assert_equal @sha1.hexdigest, resp[:sha1]
-  end
-
-  def test_put_content_md5
-    md5 = Digest::MD5.new
-    start_server(@sha1_app)
-    sock = tcp_socket(@addr, @port)
-    sock.syswrite("PUT / HTTP/1.0\r\nTransfer-Encoding: chunked\r\n" \
-                  "Trailer: Content-MD5\r\n\r\n")
-    @count.times do |i|
-      buf = @random.sysread(@bs)
-      @sha1.update(buf)
-      md5.update(buf)
-      sock.syswrite("#{'%x' % buf.size}\r\n")
-      sock.syswrite(buf << "\r\n")
-    end
-    sock.syswrite("0\r\n")
-
-    content_md5 = [ md5.digest! ].pack('m').strip.freeze
-    sock.syswrite("Content-MD5: #{content_md5}\r\n\r\n")
-    read = sock.read.split(/\r\n/)
-    assert_equal "HTTP/1.1 200 OK", read[0]
-    resp = eval(read.grep(/^X-Resp: /).first.sub!(/X-Resp: /, ''))
-    assert_equal length, resp[:size]
-    assert_equal @sha1.hexdigest, resp[:sha1]
-    assert_equal content_md5, resp[:content_md5]
-  end
-
-  def test_put_trickle_small
-    @count, @bs = 2, 128
-    start_server(@sha1_app)
-    assert_equal 256, length
-    sock = tcp_socket(@addr, @port)
-    hdr = "PUT / HTTP/1.0\r\nContent-Length: #{length}\r\n\r\n"
-    @count.times do
-      buf = @random.sysread(@bs)
-      @sha1.update(buf)
-      hdr << buf
-      sock.syswrite(hdr)
-      hdr = ''
-      sleep 0.6
-    end
-    read = sock.read.split(/\r\n/)
-    assert_equal "HTTP/1.1 200 OK", read[0]
-    resp = eval(read.grep(/^X-Resp: /).first.sub!(/X-Resp: /, ''))
-    assert_equal length, resp[:size]
-    assert_equal @sha1.hexdigest, resp[:sha1]
-  end
-
-  def test_put_keepalive_truncates_small_overwrite
-    start_server(@sha1_app)
-    sock = tcp_socket(@addr, @port)
-    to_upload = length + 1
-    sock.syswrite("PUT / HTTP/1.0\r\nContent-Length: #{to_upload}\r\n\r\n")
-    @count.times do
-      buf = @random.sysread(@bs)
-      @sha1.update(buf)
-      sock.syswrite(buf)
-    end
-    sock.syswrite('12345') # write 4 bytes more than we expected
-    @sha1.update('1')
-
-    buf = sock.readpartial(4096)
-    while buf !~ /\r\n\r\n/
-      buf << sock.readpartial(4096)
-    end
-    read = buf.split(/\r\n/)
-    assert_equal "HTTP/1.1 200 OK", read[0]
-    resp = eval(read.grep(/^X-Resp: /).first.sub!(/X-Resp: /, ''))
-    assert_equal to_upload, resp[:size]
-    assert_equal @sha1.hexdigest, resp[:sha1]
-  end
-
-  def test_put_excessive_overwrite_closed
-    tmp = Tempfile.new('overwrite_check')
-    tmp.sync = true
-    start_server(lambda { |env|
-      nr = 0
-      while buf = env['rack.input'].read(65536)
-        nr += buf.size
-      end
-      tmp.write(nr.to_s)
-      [ 200, @hdr, [] ]
-    })
-    sock = tcp_socket(@addr, @port)
-    buf = ' ' * @bs
-    sock.syswrite("PUT / HTTP/1.0\r\nContent-Length: #{length}\r\n\r\n")
-
-    @count.times { sock.syswrite(buf) }
-    assert_raise(Errno::ECONNRESET, Errno::EPIPE) do
-      ::Unicorn::Const::CHUNK_SIZE.times { sock.syswrite(buf) }
-    end
-    sock.gets
-    tmp.rewind
-    assert_equal length, tmp.read.to_i
-  end
-
-  # Despite reading numerous articles and inspecting the 1.9.1-p0 C
-  # source, Eric Wong will never trust that we're always handling
-  # encoding-aware IO objects correctly.  Thus this test uses shell
-  # utilities that should always operate on files/sockets on a
-  # byte-level.
-  def test_uncomfortable_with_onenine_encodings
-    # POSIX doesn't require all of these to be present on a system
-    which('curl') or return
-    which('sha1sum') or return
-    which('dd') or return
-
-    start_server(@sha1_app)
-
-    tmp = Tempfile.new('dd_dest')
-    assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}",
-                        "bs=#{@bs}", "count=#{@count}"),
-           "dd #@random to #{tmp}")
-    sha1_re = %r!\b([a-f0-9]{40})\b!
-    sha1_out = `sha1sum #{tmp.path}`
-    assert $?.success?, 'sha1sum ran OK'
-
-    assert_match(sha1_re, sha1_out)
-    sha1 = sha1_re.match(sha1_out)[1]
-    resp = `curl -isSfN -T#{tmp.path} http://#@addr:#@port/`
-    assert $?.success?, 'curl ran OK'
-    assert_match(%r!\b#{sha1}\b!, resp)
-    assert_match(/sysread_read_byte_match/, resp)
-
-    # small StringIO path
-    assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}",
-                        "bs=1024", "count=1"),
-           "dd #@random to #{tmp}")
-    sha1_re = %r!\b([a-f0-9]{40})\b!
-    sha1_out = `sha1sum #{tmp.path}`
-    assert $?.success?, 'sha1sum ran OK'
-
-    assert_match(sha1_re, sha1_out)
-    sha1 = sha1_re.match(sha1_out)[1]
-    resp = `curl -isSfN -T#{tmp.path} http://#@addr:#@port/`
-    assert $?.success?, 'curl ran OK'
-    assert_match(%r!\b#{sha1}\b!, resp)
-    assert_match(/sysread_read_byte_match/, resp)
-  end
-
-  def test_chunked_upload_via_curl
-    # POSIX doesn't require all of these to be present on a system
-    which('curl') or return
-    which('sha1sum') or return
-    which('dd') or return
-
-    start_server(@sha1_app)
-
-    tmp = Tempfile.new('dd_dest')
-    assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}",
-                        "bs=#{@bs}", "count=#{@count}"),
-           "dd #@random to #{tmp}")
-    sha1_re = %r!\b([a-f0-9]{40})\b!
-    sha1_out = `sha1sum #{tmp.path}`
-    assert $?.success?, 'sha1sum ran OK'
-
-    assert_match(sha1_re, sha1_out)
-    sha1 = sha1_re.match(sha1_out)[1]
-    cmd = "curl -H 'X-Expect-Size: #{tmp.size}' --tcp-nodelay \
-           -isSf --no-buffer -T- " \
-          "http://#@addr:#@port/"
-    resp = Tempfile.new('resp')
-    resp.sync = true
-
-    rd, wr = IO.pipe.each do |io|
-      io.sync = io.close_on_exec = true
-    end
-    pid = spawn(*cmd, { 0 => rd, 1 => resp })
-    rd.close
-
-    tmp.rewind
-    @count.times { |i|
-      wr.write(tmp.read(@bs))
-      sleep(rand / 10) if 0 == i % 8
-    }
-    wr.close
-    pid, status = Process.waitpid2(pid)
-
-    resp.rewind
-    resp = resp.read
-    assert status.success?, 'curl ran OK'
-    assert_match(%r!\b#{sha1}\b!, resp)
-    assert_match(/sysread_read_byte_match/, resp)
-    assert_match(/expect_size_match/, resp)
-  end
-
-  def test_curl_chunked_small
-    # POSIX doesn't require all of these to be present on a system
-    which('curl') or return
-    which('sha1sum') or return
-    which('dd') or return
-
-    start_server(@sha1_app)
-
-    tmp = Tempfile.new('dd_dest')
-    # small StringIO path
-    assert(system("dd", "if=#{@random.path}", "of=#{tmp.path}",
-                        "bs=1024", "count=1"),
-           "dd #@random to #{tmp}")
-    sha1_re = %r!\b([a-f0-9]{40})\b!
-    sha1_out = `sha1sum #{tmp.path}`
-    assert $?.success?, 'sha1sum ran OK'
-
-    assert_match(sha1_re, sha1_out)
-    sha1 = sha1_re.match(sha1_out)[1]
-    resp = `curl -H 'X-Expect-Size: #{tmp.size}' --tcp-nodelay \
-            -isSf --no-buffer -T- http://#@addr:#@port/ < #{tmp.path}`
-    assert $?.success?, 'curl ran OK'
-    assert_match(%r!\b#{sha1}\b!, resp)
-    assert_match(/sysread_read_byte_match/, resp)
-    assert_match(/expect_size_match/, resp)
-  end
-
-  private
-
-  def length
-    @bs * @count
-  end
-
-  def start_server(app)
-    redirect_test_io do
-      @server = HttpServer.new(app, :listeners => [ "#{@addr}:#{@port}" ] )
-      @server.start
-    end
-  end
-
-end

[-- Attachment #23: 0022-drop-redundant-IO-close_on_exec-false-calls.patch --]
[-- Type: text/x-diff, Size: 891 bytes --]

From 841b9e756beb1aa00d0f89097a808adcbbf45397 Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:51 +0000
Subject: [PATCH 22/23] drop redundant IO#close_on_exec=false calls

Passing the `{ FD => IO }' mapping to #spawn or #exec already
ensures Ruby will clear FD_CLOEXEC on these FDs before execve(2).
---
 lib/unicorn/http_server.rb | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 348e745..dd92b38 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -472,10 +472,7 @@ def worker_spawn(worker)
 
   def listener_sockets
     listener_fds = {}
-    LISTENERS.each do |sock|
-      sock.close_on_exec = false
-      listener_fds[sock.fileno] = sock
-    end
+    LISTENERS.each { |sock| listener_fds[sock.fileno] = sock }
     listener_fds
   end
 

[-- Attachment #24: 0023-LISTEN_FDS-inherited-sockets-are-immortal-across-SIG.patch --]
[-- Type: text/x-diff, Size: 3229 bytes --]

From 6ff8785c9277c5978e6dc01cb1b3da25d6bae2db Mon Sep 17 00:00:00 2001
From: Eric Wong <BOFH@YHBT.net>
Date: Mon, 5 Jun 2023 10:12:52 +0000
Subject: [PATCH 23/23] LISTEN_FDS-inherited sockets are immortal across SIGHUP

When using systemd-style socket activation, consider the
inherited socket immortal and do not drop it on SIGHUP.
This means configs w/o any `listen' directives at all can
continue to work after SIGHUP.

I only noticed this while writing some tests in Perl 5 and
the test suite is two lines shorter to test this feature :>
---
 lib/unicorn/http_server.rb  | 7 ++++++-
 t/client_body_buffer_size.t | 1 -
 t/integration.t             | 1 -
 3 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index dd92b38..f1b4a54 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -77,6 +77,7 @@ def initialize(app, options = {})
     options[:use_defaults] = true
     self.config = Unicorn::Configurator.new(options)
     self.listener_opts = {}
+    @immortal = [] # immortal inherited sockets from systemd
 
     # We use @self_pipe differently in the master and worker processes:
     #
@@ -158,6 +159,7 @@ def listeners=(listeners)
     end
     set_names = listener_names(listeners)
     dead_names.concat(cur_names - set_names).uniq!
+    dead_names -= @immortal.map { |io| sock_name(io) }
 
     LISTENERS.delete_if do |io|
       if dead_names.include?(sock_name(io))
@@ -807,17 +809,20 @@ def inherit_listeners!
     # inherit sockets from parents, they need to be plain Socket objects
     # before they become Kgio::UNIXServer or Kgio::TCPServer
     inherited = ENV['UNICORN_FD'].to_s.split(',')
+    immortal = []
 
     # emulate sd_listen_fds() for systemd
     sd_pid, sd_fds = ENV.values_at('LISTEN_PID', 'LISTEN_FDS')
     if sd_pid.to_i == $$ # n.b. $$ can never be zero
       # 3 = SD_LISTEN_FDS_START
-      inherited.concat((3...(3 + sd_fds.to_i)).to_a)
+      immortal = (3...(3 + sd_fds.to_i)).to_a
+      inherited.concat(immortal)
     end
     # to ease debugging, we will not unset LISTEN_PID and LISTEN_FDS
 
     inherited.map! do |fd|
       io = Socket.for_fd(fd.to_i)
+      @immortal << io if immortal.include?(fd)
       io.autoclose = false
       io = server_cast(io)
       set_server_sockopt(io, listener_opts[sock_name(io)])
diff --git a/t/client_body_buffer_size.t b/t/client_body_buffer_size.t
index b1a99f3..3067f28 100644
--- a/t/client_body_buffer_size.t
+++ b/t/client_body_buffer_size.t
@@ -36,7 +36,6 @@ POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!";
 seek($conf_fh, 0, SEEK_SET);
 truncate($conf_fh, 0);
 print $conf_fh <<EOM;
-listen "$host_port" # TODO: remove this requirement for SIGHUP
 after_fork { |_,_| File.open('$fifo', 'w') { |fp| fp.write "pid=#\$\$" } }
 EOM
 $ar->do_kill('HUP');
diff --git a/t/integration.t b/t/integration.t
index a568758..bb2ab51 100644
--- a/t/integration.t
+++ b/t/integration.t
@@ -17,7 +17,6 @@ my $u1 = "$tmpdir/u1";
 print $conf_fh <<EOM;
 early_hints true
 listen "$u1"
-listen "$host_port" # TODO: remove this requirement for SIGHUP
 EOM
 my $ar = unicorn(qw(-E none t/integration.ru -c), $conf, { 3 => $srv });
 my $curl = which('curl');

^ permalink raw reply related	[relevance 3%]

* [PATCH] Get rid of Kgio
@ 2022-07-08 13:17  2% Jean Boussier
  0 siblings, 0 replies; 31+ results
From: Jean Boussier @ 2022-07-08 13:17 UTC (permalink / raw)
  To: unicorn-public

[-- Attachment #1: Type: text/plain, Size: 17184 bytes --]

As discussed kgio is no longer absolutely necessary.

We can use Ruby 2+ non blocking IO capabilities instead.

Also available at https://github.com/casperisfine/unicorn.git (remove-kgio)

---
  lib/unicorn.rb                 |  3 +--
  lib/unicorn/http_request.rb    | 15 +++++++++-----
  lib/unicorn/http_server.rb     | 33 +++++++++++++++++++++--------
  lib/unicorn/socket_helper.rb   | 20 ++++--------------
  lib/unicorn/stream_input.rb    | 16 ++++++++------
  lib/unicorn/worker.rb          | 38 ++++++++++++++++++++++------------
  t/oob_gc.ru                    |  2 +-
  t/oob_gc_path.ru               |  2 +-
  test/unit/test_request.rb      | 16 +++-----------
  test/unit/test_stream_input.rb |  7 ++-----
  test/unit/test_tee_input.rb    |  2 +-
  unicorn.gemspec                |  1 -
  12 files changed, 82 insertions(+), 73 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 1a50631..170719c 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -1,7 +1,6 @@
  # -*- encoding: binary -*-
  require 'etc'
  require 'stringio'
-require 'kgio'
  require 'raindrops'
  require 'io/wait'

@@ -113,7 +112,7 @@ def self.log_error(logger, prefix, exc)
    F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/

    def self.pipe # :nodoc:
-    Kgio::Pipe.new.each do |io|
+    IO.pipe.each do |io|
        # shrink pipes to minimize impact on 
/proc/sys/fs/pipe-user-pages-soft
        # limits.
        if defined?(F_SETPIPE_SZ)
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index e3ad592..7f7324b 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -71,14 +71,19 @@ def read(socket)
      #  identify the client for the immediate request to the server;
      #  that client may be a proxy, gateway, or other intermediary
      #  acting on behalf of the actual source client."
-    e['REMOTE_ADDR'] = socket.kgio_addr
+    address = socket.remote_address
+    e['REMOTE_ADDR'] = if address.unix?
+      "127.0.0.1"
+    else
+      address.ip_address
+    end

      # short circuit the common case with small GET requests first
-    socket.kgio_read!(16384, buf)
+    socket.readpartial(16384, buf)
      if parse.nil?
        # Parser is not done, queue up more data to read and continue 
parsing
        # an Exception thrown from the parser will throw us out of the loop
-      false until add_parse(socket.kgio_read!(16384))
+      false until add_parse(socket.readpartial(16384))
      end

      check_client_connection(socket) if @@check_client_connection
@@ -108,7 +113,7 @@ def hijacked?
      TCPI = Raindrops::TCP_Info.allocate

      def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket
+      if TCPSocket === socket
          # Raindrops::TCP_Info#get!, #state (reads struct 
tcp_info#tcpi_state)
          raise Errno::EPIPE, "client closed connection".freeze,
                EMPTY_ARRAY if closed_state?(TCPI.get!(socket).state)
@@ -153,7 +158,7 @@ def closed_state?(state) # :nodoc:
      # Not that efficient, but probably still better than doing unnecessary
      # work after a client gives up.
      def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket && @@tcpi_inspect_ok
+      if TCPSocket === socket && @@tcpi_inspect_ok
          opt = socket.getsockopt(Socket::IPPROTO_TCP, 
Socket::TCP_INFO).inspect
          if opt =~ /\bstate=(\S+)/
            raise Errno::EPIPE, "client closed connection".freeze,
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 21f2a05..98cd119 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -111,7 +111,7 @@ def initialize(app, options = {})
      @worker_data = if worker_data = ENV['UNICORN_WORKER']
        worker_data = worker_data.split(',').map!(&:to_i)
        worker_data[1] = worker_data.slice!(1..2).map do |i|
-        Kgio::Pipe.for_fd(i)
+        IO.for_fd(i)
        end
        worker_data
      end
@@ -240,7 +240,7 @@ def listen(address, opt = 
{}.merge(listener_opts[address] || {}))
      tries = opt[:tries] || 5
      begin
        io = bind_listen(address, opt)
-      unless Kgio::TCPServer === io || Kgio::UNIXServer === io
+      unless TCPServer === io || UNIXServer === io
          io.autoclose = false
          io = server_cast(io)
        end
@@ -386,12 +386,18 @@ def master_sleep(sec)
      # the Ruby itself and not require a separate malloc (on 32-bit MRI 
1.9+).
      # Most reads are only one byte here and uncommon, so it's not worth a
      # persistent buffer, either:
-    @self_pipe[0].kgio_tryread(11)
+    begin
+      @self_pipe[0].read_nonblock(11)
+    rescue Errno::EAGAIN
+    end
    end

    def awaken_master
      return if $$ != @master_pid
-    @self_pipe[1].kgio_trywrite('.') # wakeup master process from select
+    begin
+      @self_pipe[1].write_nonblock('.') # wakeup master process from select
+    rescue Errno::EAGAIN
+    end
    end

    # reaps all unreaped workers
@@ -581,7 +587,10 @@ def handle_error(client, e)
        500
      end
      if code
-      client.kgio_trywrite(err_response(code, 
@request.response_start_sent))
+      begin
+        client.write_nonblock(err_response(code, 
@request.response_start_sent))
+      rescue Errno::EAGAIN
+      end
      end
      client.close
    rescue
@@ -733,9 +742,15 @@ def worker_loop(worker)
        reopen = reopen_worker_logs(worker.nr) if reopen
        worker.tick = time_now.to_i
        while sock = ready.shift
-        # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
+        # Unicorn::Worker#accept_nonblock is not like accept(2) at all,
          # but that will return false
-        if client = sock.kgio_tryaccept
+        client = begin
+          sock.accept_nonblock
+        rescue Errno::EAGAIN
+          false
+        end
+
+        if client
            process_client(client)
            worker.tick = time_now.to_i
          end
@@ -834,7 +849,7 @@ def redirect_io(io, path)

    def inherit_listeners!
      # inherit sockets from parents, they need to be plain Socket objects
-    # before they become Kgio::UNIXServer or Kgio::TCPServer
+    # before they become UNIXServer or TCPServer
      inherited = ENV['UNICORN_FD'].to_s.split(',')

      # emulate sd_listen_fds() for systemd
@@ -858,7 +873,7 @@ def inherit_listeners!
      LISTENERS.replace(inherited)

      # we start out with generic Socket objects that get cast to either
-    # Kgio::TCPServer or Kgio::UNIXServer objects; but since the Socket
+    # TCPServer or UNIXServer objects; but since the Socket
      # objects share the same OS-level file descriptor as the higher-level
      # *Server objects; we need to prevent Socket objects from being
      # garbage-collected
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index 8a6f6ee..9d2ba52 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -3,18 +3,6 @@
  require 'socket'

  module Unicorn
-
-  # Instead of using a generic Kgio::Socket for everything,
-  # tag TCP sockets so we can use TCP_INFO under Linux without
-  # incurring extra syscalls for Unix domain sockets.
-  # TODO: remove these when we remove kgio
-  TCPClient = Class.new(Kgio::Socket) # :nodoc:
-  class TCPSrv < Kgio::TCPServer # :nodoc:
-    def kgio_tryaccept # :nodoc:
-      super(TCPClient)
-    end
-  end
-
    module SocketHelper

      # internal interface
@@ -135,7 +123,7 @@ def bind_listen(address = '0.0.0.0:8080', opt = {})
          end
          old_umask = File.umask(opt[:umask] || 0)
          begin
-          Kgio::UNIXServer.new(address)
+          UNIXServer.new(address)
          ensure
            File.umask(old_umask)
          end
@@ -164,7 +152,7 @@ def new_tcp_server(addr, port, opt)
        end
        sock.bind(Socket.pack_sockaddr_in(port, addr))
        sock.autoclose = false
-      TCPSrv.for_fd(sock.fileno)
+      TCPServer.for_fd(sock.fileno)
      end

      # returns rfc2732-style (e.g. "[::1]:666") addresses for IPv6
@@ -201,9 +189,9 @@ def sock_name(sock)
      def server_cast(sock)
        begin
          Socket.unpack_sockaddr_in(sock.getsockname)
-        TCPSrv.for_fd(sock.fileno)
+        TCPServer.for_fd(sock.fileno)
        rescue ArgumentError
-        Kgio::UNIXServer.for_fd(sock.fileno)
+        UNIXServer.for_fd(sock.fileno)
        end
      end

diff --git a/lib/unicorn/stream_input.rb b/lib/unicorn/stream_input.rb
index 41d28a0..3241ff4 100644
--- a/lib/unicorn/stream_input.rb
+++ b/lib/unicorn/stream_input.rb
@@ -1,3 +1,4 @@
+
  # -*- encoding: binary -*-

  # When processing uploads, unicorn may expose a StreamInput object under
@@ -49,7 +50,11 @@ def read(length = nil, rv = '')
          to_read = length - @rbuf.size
          rv.replace(@rbuf.slice!(0, @rbuf.size))
          until to_read == 0 || eof? || (rv.size > 0 && @chunked)
-          @socket.kgio_read(to_read, @buf) or eof!
+          begin
+            @socket.readpartial(to_read, @buf)
+          rescue EOFError
+            eof!
+          end
            filter_body(@rbuf, @buf)
            rv << @rbuf
            to_read -= @rbuf.size
@@ -72,8 +77,7 @@ def read(length = nil, rv = '')
    # Returns nil if called at the end of file.
    # This takes zero arguments for strict Rack::Lint compatibility,
    # unlike IO#gets.
-  def gets
-    sep = $/
+  def gets(sep = $/)
      if sep.nil?
        read_all(rv = '')
        return rv.empty? ? nil : rv
@@ -83,7 +87,7 @@ def gets
      begin
        @rbuf.sub!(re, '') and return $1
        return @rbuf.empty? ? nil : @rbuf.slice!(0, @rbuf.size) if eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
+      @socket.readpartial(@@io_chunk_size, @buf) or eof!
        filter_body(once = '', @buf)
        @rbuf << once
      end while true
@@ -107,7 +111,7 @@ def each
    def eof?
      if @parser.body_eof?
        while @chunked && ! @parser.parse
-        once = @socket.kgio_read(@@io_chunk_size) or eof!
+        once = @socket.readpartial(@@io_chunk_size) or eof!
          @buf << once
        end
        @socket = nil
@@ -127,7 +131,7 @@ def read_all(dst)
      dst.replace(@rbuf)
      @socket or return
      until eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
+      @socket.readpartial(@@io_chunk_size, @buf) or eof!
        filter_body(@rbuf, @buf)
        dst << @rbuf
      end
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 5ddf379..fe741c0 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -63,27 +63,39 @@ def soft_kill(sig) # :nodoc:
        signum = Signal.list[sig.to_s] or
            raise ArgumentError, "BUG: bad signal: #{sig.inspect}"
      end
+
      # writing and reading 4 bytes on a pipe is atomic on all POSIX 
platforms
      # Do not care in the odd case the buffer is full, here.
-    @master.kgio_trywrite([signum].pack('l'))
+    begin
+      @master.write_nonblock([signum].pack('l'))
+    rescue Errno::EAGAIN
+    end
    rescue Errno::EPIPE
      # worker will be reaped soon
    end

    # this only runs when the Rack app.call is not running
    # act like a listener
-  def kgio_tryaccept # :nodoc:
-    case buf = @to_io.kgio_tryread(4)
-    when String
-      # unpack the buffer and trigger the signal handler
-      signum = buf.unpack('l')
-      fake_sig(signum[0])
-      # keep looping, more signals may be queued
-    when nil # EOF: master died, but we are at a safe place to exit
-      fake_sig(:QUIT)
-    when :wait_readable # keep waiting
-      return false
-    end while true # loop, as multiple signals may be sent
+  def accept_nonblock # :nodoc:
+    loop do
+      buf = begin
+        @to_io.read_nonblock(4)
+      rescue Errno::EAGAIN # keep waiting
+        return false
+      rescue EOFError # master died, but we are at a safe place to exit
+        fake_sig(:QUIT)
+      end
+
+      case buf
+      when String
+        # unpack the buffer and trigger the signal handler
+        signum = buf.unpack('l')
+        fake_sig(signum[0])
+        # keep looping, more signals may be queued
+      else
+        raise TypeError, "Unexpected read_nonblock returns: #{buf.inspect}"
+      end
+    end # loop, as multiple signals may be sent
    end

    # worker objects may be compared to just plain Integers
diff --git a/t/oob_gc.ru b/t/oob_gc.ru
index c253540..43c0f68 100644
--- a/t/oob_gc.ru
+++ b/t/oob_gc.ru
@@ -7,7 +7,7 @@

  # Mock GC.start
  def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
+  ObjectSpace.each_object(Socket) do |x|
      x.closed? or abort "not closed #{x}"
    end
    $gc_started = true
diff --git a/t/oob_gc_path.ru b/t/oob_gc_path.ru
index af8e3b9..e772261 100644
--- a/t/oob_gc_path.ru
+++ b/t/oob_gc_path.ru
@@ -7,7 +7,7 @@

  # Mock GC.start
  def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
+  ObjectSpace.each_object(Socket) do |x|
      x.closed? or abort "not closed #{x}"
    end
    $gc_started = true
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index 6cb0268..a951057 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -11,11 +11,8 @@
  class RequestTest < Test::Unit::TestCase

    class MockRequest < StringIO
-    alias_method :readpartial, :sysread
-    alias_method :kgio_read!, :sysread
-    alias_method :read_nonblock, :sysread
-    def kgio_addr
-      '127.0.0.1'
+    def remote_address
+      Addrinfo.tcp('127.0.0.1', 55608)
      end
    end

@@ -152,14 +149,7 @@ def test_rack_lint_big_put
      buf = (' ' * bs).freeze
      length = bs * count
      client = Tempfile.new('big_put')
-    def client.kgio_addr; '127.0.0.1'; end
-    def client.kgio_read(*args)
-      readpartial(*args)
-    rescue EOFError
-    end
-    def client.kgio_read!(*args)
-      readpartial(*args)
-    end
+    def client.remote_address; Addrinfo.tcp('127.0.0.1', 55608); end
      client.syswrite(
        "PUT / HTTP/1.1\r\n" \
        "Host: foo\r\n" \
diff --git a/test/unit/test_stream_input.rb b/test/unit/test_stream_input.rb
index 1a07ec3..445b415 100644
--- a/test/unit/test_stream_input.rb
+++ b/test/unit/test_stream_input.rb
@@ -6,16 +6,14 @@

  class TestStreamInput < Test::Unit::TestCase
    def setup
-    @rs = $/
      @env = {}
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
      @rd.sync = @wr.sync = true
      @start_pid = $$
    end

    def teardown
      return if $$ != @start_pid
-    $/ = @rs
      @rd.close rescue nil
      @wr.close rescue nil
      Process.waitall
@@ -54,10 +52,9 @@ def test_gets_multiline
    end

    def test_gets_empty_rs
-    $/ = nil
      r = init_request("a\nb\n\n")
      si = Unicorn::StreamInput.new(@rd, r)
-    assert_equal "a\nb\n\n", si.gets
+    assert_equal "a\nb\n\n", si.gets(nil)
      assert_nil si.gets
    end

diff --git a/test/unit/test_tee_input.rb b/test/unit/test_tee_input.rb
index 4647e66..21b90f1 100644
--- a/test/unit/test_tee_input.rb
+++ b/test/unit/test_tee_input.rb
@@ -12,7 +12,7 @@ class TestTeeInput < Test::Unit::TestCase

    def setup
      @rs = $/
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
      @rd.sync = @wr.sync = true
      @start_pid = $$
    end
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 7bb1154..85183d9 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -36,7 +36,6 @@
    # won't have descriptive text, only the numeric status.
    s.add_development_dependency(%q<rack>)

-  s.add_dependency(%q<kgio>, '~> 2.6')
    s.add_dependency(%q<raindrops>, '~> 0.7')

    s.add_development_dependency('test-unit', '~> 3.0')
-- 
2.35.1

[-- Attachment #2: 0001-Get-rid-of-Kgio.patch --]
[-- Type: text/plain, Size: 15534 bytes --]

From d0412e7305d7b6b4abc64996119e5722709bb6b0 Mon Sep 17 00:00:00 2001
From: Jean Boussier <jean.boussier@gmail.com>
Date: Fri, 8 Jul 2022 12:58:25 +0200
Subject: [PATCH] Get rid of Kgio

As discussed kgio is no longer absolutely necessary.

We can use Ruby 2+ non blocking IO capabilities instead.
---
 lib/unicorn.rb                 |  3 +--
 lib/unicorn/http_request.rb    | 15 +++++++++-----
 lib/unicorn/http_server.rb     | 33 +++++++++++++++++++++--------
 lib/unicorn/socket_helper.rb   | 20 ++++--------------
 lib/unicorn/stream_input.rb    | 16 ++++++++------
 lib/unicorn/worker.rb          | 38 ++++++++++++++++++++++------------
 t/oob_gc.ru                    |  2 +-
 t/oob_gc_path.ru               |  2 +-
 test/unit/test_request.rb      | 16 +++-----------
 test/unit/test_stream_input.rb |  7 ++-----
 test/unit/test_tee_input.rb    |  2 +-
 unicorn.gemspec                |  1 -
 12 files changed, 82 insertions(+), 73 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 1a50631..170719c 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -1,7 +1,6 @@
 # -*- encoding: binary -*-
 require 'etc'
 require 'stringio'
-require 'kgio'
 require 'raindrops'
 require 'io/wait'
 
@@ -113,7 +112,7 @@ def self.log_error(logger, prefix, exc)
   F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/
 
   def self.pipe # :nodoc:
-    Kgio::Pipe.new.each do |io|
+    IO.pipe.each do |io|
       # shrink pipes to minimize impact on /proc/sys/fs/pipe-user-pages-soft
       # limits.
       if defined?(F_SETPIPE_SZ)
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index e3ad592..7f7324b 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -71,14 +71,19 @@ def read(socket)
     #  identify the client for the immediate request to the server;
     #  that client may be a proxy, gateway, or other intermediary
     #  acting on behalf of the actual source client."
-    e['REMOTE_ADDR'] = socket.kgio_addr
+    address = socket.remote_address
+    e['REMOTE_ADDR'] = if address.unix?
+      "127.0.0.1"
+    else
+      address.ip_address
+    end
 
     # short circuit the common case with small GET requests first
-    socket.kgio_read!(16384, buf)
+    socket.readpartial(16384, buf)
     if parse.nil?
       # Parser is not done, queue up more data to read and continue parsing
       # an Exception thrown from the parser will throw us out of the loop
-      false until add_parse(socket.kgio_read!(16384))
+      false until add_parse(socket.readpartial(16384))
     end
 
     check_client_connection(socket) if @@check_client_connection
@@ -108,7 +113,7 @@ def hijacked?
     TCPI = Raindrops::TCP_Info.allocate
 
     def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket
+      if TCPSocket === socket
         # Raindrops::TCP_Info#get!, #state (reads struct tcp_info#tcpi_state)
         raise Errno::EPIPE, "client closed connection".freeze,
               EMPTY_ARRAY if closed_state?(TCPI.get!(socket).state)
@@ -153,7 +158,7 @@ def closed_state?(state) # :nodoc:
     # Not that efficient, but probably still better than doing unnecessary
     # work after a client gives up.
     def check_client_connection(socket) # :nodoc:
-      if Unicorn::TCPClient === socket && @@tcpi_inspect_ok
+      if TCPSocket === socket && @@tcpi_inspect_ok
         opt = socket.getsockopt(Socket::IPPROTO_TCP, Socket::TCP_INFO).inspect
         if opt =~ /\bstate=(\S+)/
           raise Errno::EPIPE, "client closed connection".freeze,
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 21f2a05..98cd119 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -111,7 +111,7 @@ def initialize(app, options = {})
     @worker_data = if worker_data = ENV['UNICORN_WORKER']
       worker_data = worker_data.split(',').map!(&:to_i)
       worker_data[1] = worker_data.slice!(1..2).map do |i|
-        Kgio::Pipe.for_fd(i)
+        IO.for_fd(i)
       end
       worker_data
     end
@@ -240,7 +240,7 @@ def listen(address, opt = {}.merge(listener_opts[address] || {}))
     tries = opt[:tries] || 5
     begin
       io = bind_listen(address, opt)
-      unless Kgio::TCPServer === io || Kgio::UNIXServer === io
+      unless TCPServer === io || UNIXServer === io
         io.autoclose = false
         io = server_cast(io)
       end
@@ -386,12 +386,18 @@ def master_sleep(sec)
     # the Ruby itself and not require a separate malloc (on 32-bit MRI 1.9+).
     # Most reads are only one byte here and uncommon, so it's not worth a
     # persistent buffer, either:
-    @self_pipe[0].kgio_tryread(11)
+    begin
+      @self_pipe[0].read_nonblock(11)
+    rescue Errno::EAGAIN
+    end
   end
 
   def awaken_master
     return if $$ != @master_pid
-    @self_pipe[1].kgio_trywrite('.') # wakeup master process from select
+    begin
+      @self_pipe[1].write_nonblock('.') # wakeup master process from select
+    rescue Errno::EAGAIN
+    end
   end
 
   # reaps all unreaped workers
@@ -581,7 +587,10 @@ def handle_error(client, e)
       500
     end
     if code
-      client.kgio_trywrite(err_response(code, @request.response_start_sent))
+      begin
+        client.write_nonblock(err_response(code, @request.response_start_sent))
+      rescue Errno::EAGAIN
+      end
     end
     client.close
   rescue
@@ -733,9 +742,15 @@ def worker_loop(worker)
       reopen = reopen_worker_logs(worker.nr) if reopen
       worker.tick = time_now.to_i
       while sock = ready.shift
-        # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
+        # Unicorn::Worker#accept_nonblock is not like accept(2) at all,
         # but that will return false
-        if client = sock.kgio_tryaccept
+        client = begin
+          sock.accept_nonblock
+        rescue Errno::EAGAIN
+          false
+        end
+
+        if client
           process_client(client)
           worker.tick = time_now.to_i
         end
@@ -834,7 +849,7 @@ def redirect_io(io, path)
 
   def inherit_listeners!
     # inherit sockets from parents, they need to be plain Socket objects
-    # before they become Kgio::UNIXServer or Kgio::TCPServer
+    # before they become UNIXServer or TCPServer
     inherited = ENV['UNICORN_FD'].to_s.split(',')
 
     # emulate sd_listen_fds() for systemd
@@ -858,7 +873,7 @@ def inherit_listeners!
     LISTENERS.replace(inherited)
 
     # we start out with generic Socket objects that get cast to either
-    # Kgio::TCPServer or Kgio::UNIXServer objects; but since the Socket
+    # TCPServer or UNIXServer objects; but since the Socket
     # objects share the same OS-level file descriptor as the higher-level
     # *Server objects; we need to prevent Socket objects from being
     # garbage-collected
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index 8a6f6ee..9d2ba52 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -3,18 +3,6 @@
 require 'socket'
 
 module Unicorn
-
-  # Instead of using a generic Kgio::Socket for everything,
-  # tag TCP sockets so we can use TCP_INFO under Linux without
-  # incurring extra syscalls for Unix domain sockets.
-  # TODO: remove these when we remove kgio
-  TCPClient = Class.new(Kgio::Socket) # :nodoc:
-  class TCPSrv < Kgio::TCPServer # :nodoc:
-    def kgio_tryaccept # :nodoc:
-      super(TCPClient)
-    end
-  end
-
   module SocketHelper
 
     # internal interface
@@ -135,7 +123,7 @@ def bind_listen(address = '0.0.0.0:8080', opt = {})
         end
         old_umask = File.umask(opt[:umask] || 0)
         begin
-          Kgio::UNIXServer.new(address)
+          UNIXServer.new(address)
         ensure
           File.umask(old_umask)
         end
@@ -164,7 +152,7 @@ def new_tcp_server(addr, port, opt)
       end
       sock.bind(Socket.pack_sockaddr_in(port, addr))
       sock.autoclose = false
-      TCPSrv.for_fd(sock.fileno)
+      TCPServer.for_fd(sock.fileno)
     end
 
     # returns rfc2732-style (e.g. "[::1]:666") addresses for IPv6
@@ -201,9 +189,9 @@ def sock_name(sock)
     def server_cast(sock)
       begin
         Socket.unpack_sockaddr_in(sock.getsockname)
-        TCPSrv.for_fd(sock.fileno)
+        TCPServer.for_fd(sock.fileno)
       rescue ArgumentError
-        Kgio::UNIXServer.for_fd(sock.fileno)
+        UNIXServer.for_fd(sock.fileno)
       end
     end
 
diff --git a/lib/unicorn/stream_input.rb b/lib/unicorn/stream_input.rb
index 41d28a0..3241ff4 100644
--- a/lib/unicorn/stream_input.rb
+++ b/lib/unicorn/stream_input.rb
@@ -1,3 +1,4 @@
+
 # -*- encoding: binary -*-
 
 # When processing uploads, unicorn may expose a StreamInput object under
@@ -49,7 +50,11 @@ def read(length = nil, rv = '')
         to_read = length - @rbuf.size
         rv.replace(@rbuf.slice!(0, @rbuf.size))
         until to_read == 0 || eof? || (rv.size > 0 && @chunked)
-          @socket.kgio_read(to_read, @buf) or eof!
+          begin
+            @socket.readpartial(to_read, @buf)
+          rescue EOFError
+            eof!
+          end
           filter_body(@rbuf, @buf)
           rv << @rbuf
           to_read -= @rbuf.size
@@ -72,8 +77,7 @@ def read(length = nil, rv = '')
   # Returns nil if called at the end of file.
   # This takes zero arguments for strict Rack::Lint compatibility,
   # unlike IO#gets.
-  def gets
-    sep = $/
+  def gets(sep = $/)
     if sep.nil?
       read_all(rv = '')
       return rv.empty? ? nil : rv
@@ -83,7 +87,7 @@ def gets
     begin
       @rbuf.sub!(re, '') and return $1
       return @rbuf.empty? ? nil : @rbuf.slice!(0, @rbuf.size) if eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
+      @socket.readpartial(@@io_chunk_size, @buf) or eof!
       filter_body(once = '', @buf)
       @rbuf << once
     end while true
@@ -107,7 +111,7 @@ def each
   def eof?
     if @parser.body_eof?
       while @chunked && ! @parser.parse
-        once = @socket.kgio_read(@@io_chunk_size) or eof!
+        once = @socket.readpartial(@@io_chunk_size) or eof!
         @buf << once
       end
       @socket = nil
@@ -127,7 +131,7 @@ def read_all(dst)
     dst.replace(@rbuf)
     @socket or return
     until eof?
-      @socket.kgio_read(@@io_chunk_size, @buf) or eof!
+      @socket.readpartial(@@io_chunk_size, @buf) or eof!
       filter_body(@rbuf, @buf)
       dst << @rbuf
     end
diff --git a/lib/unicorn/worker.rb b/lib/unicorn/worker.rb
index 5ddf379..fe741c0 100644
--- a/lib/unicorn/worker.rb
+++ b/lib/unicorn/worker.rb
@@ -63,27 +63,39 @@ def soft_kill(sig) # :nodoc:
       signum = Signal.list[sig.to_s] or
           raise ArgumentError, "BUG: bad signal: #{sig.inspect}"
     end
+
     # writing and reading 4 bytes on a pipe is atomic on all POSIX platforms
     # Do not care in the odd case the buffer is full, here.
-    @master.kgio_trywrite([signum].pack('l'))
+    begin
+      @master.write_nonblock([signum].pack('l'))
+    rescue Errno::EAGAIN
+    end
   rescue Errno::EPIPE
     # worker will be reaped soon
   end
 
   # this only runs when the Rack app.call is not running
   # act like a listener
-  def kgio_tryaccept # :nodoc:
-    case buf = @to_io.kgio_tryread(4)
-    when String
-      # unpack the buffer and trigger the signal handler
-      signum = buf.unpack('l')
-      fake_sig(signum[0])
-      # keep looping, more signals may be queued
-    when nil # EOF: master died, but we are at a safe place to exit
-      fake_sig(:QUIT)
-    when :wait_readable # keep waiting
-      return false
-    end while true # loop, as multiple signals may be sent
+  def accept_nonblock # :nodoc:
+    loop do
+      buf = begin
+        @to_io.read_nonblock(4)
+      rescue Errno::EAGAIN # keep waiting
+        return false
+      rescue EOFError # master died, but we are at a safe place to exit
+        fake_sig(:QUIT)
+      end
+
+      case buf
+      when String
+        # unpack the buffer and trigger the signal handler
+        signum = buf.unpack('l')
+        fake_sig(signum[0])
+        # keep looping, more signals may be queued
+      else
+        raise TypeError, "Unexpected read_nonblock returns: #{buf.inspect}"
+      end
+    end # loop, as multiple signals may be sent
   end
 
   # worker objects may be compared to just plain Integers
diff --git a/t/oob_gc.ru b/t/oob_gc.ru
index c253540..43c0f68 100644
--- a/t/oob_gc.ru
+++ b/t/oob_gc.ru
@@ -7,7 +7,7 @@
 
 # Mock GC.start
 def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
+  ObjectSpace.each_object(Socket) do |x|
     x.closed? or abort "not closed #{x}"
   end
   $gc_started = true
diff --git a/t/oob_gc_path.ru b/t/oob_gc_path.ru
index af8e3b9..e772261 100644
--- a/t/oob_gc_path.ru
+++ b/t/oob_gc_path.ru
@@ -7,7 +7,7 @@
 
 # Mock GC.start
 def GC.start
-  ObjectSpace.each_object(Kgio::Socket) do |x|
+  ObjectSpace.each_object(Socket) do |x|
     x.closed? or abort "not closed #{x}"
   end
   $gc_started = true
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index 6cb0268..a951057 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -11,11 +11,8 @@
 class RequestTest < Test::Unit::TestCase
 
   class MockRequest < StringIO
-    alias_method :readpartial, :sysread
-    alias_method :kgio_read!, :sysread
-    alias_method :read_nonblock, :sysread
-    def kgio_addr
-      '127.0.0.1'
+    def remote_address
+      Addrinfo.tcp('127.0.0.1', 55608)
     end
   end
 
@@ -152,14 +149,7 @@ def test_rack_lint_big_put
     buf = (' ' * bs).freeze
     length = bs * count
     client = Tempfile.new('big_put')
-    def client.kgio_addr; '127.0.0.1'; end
-    def client.kgio_read(*args)
-      readpartial(*args)
-    rescue EOFError
-    end
-    def client.kgio_read!(*args)
-      readpartial(*args)
-    end
+    def client.remote_address; Addrinfo.tcp('127.0.0.1', 55608); end
     client.syswrite(
       "PUT / HTTP/1.1\r\n" \
       "Host: foo\r\n" \
diff --git a/test/unit/test_stream_input.rb b/test/unit/test_stream_input.rb
index 1a07ec3..445b415 100644
--- a/test/unit/test_stream_input.rb
+++ b/test/unit/test_stream_input.rb
@@ -6,16 +6,14 @@
 
 class TestStreamInput < Test::Unit::TestCase
   def setup
-    @rs = $/
     @env = {}
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
     @rd.sync = @wr.sync = true
     @start_pid = $$
   end
 
   def teardown
     return if $$ != @start_pid
-    $/ = @rs
     @rd.close rescue nil
     @wr.close rescue nil
     Process.waitall
@@ -54,10 +52,9 @@ def test_gets_multiline
   end
 
   def test_gets_empty_rs
-    $/ = nil
     r = init_request("a\nb\n\n")
     si = Unicorn::StreamInput.new(@rd, r)
-    assert_equal "a\nb\n\n", si.gets
+    assert_equal "a\nb\n\n", si.gets(nil)
     assert_nil si.gets
   end
 
diff --git a/test/unit/test_tee_input.rb b/test/unit/test_tee_input.rb
index 4647e66..21b90f1 100644
--- a/test/unit/test_tee_input.rb
+++ b/test/unit/test_tee_input.rb
@@ -12,7 +12,7 @@ class TestTeeInput < Test::Unit::TestCase
 
   def setup
     @rs = $/
-    @rd, @wr = Kgio::UNIXSocket.pair
+    @rd, @wr = UNIXSocket.pair
     @rd.sync = @wr.sync = true
     @start_pid = $$
   end
diff --git a/unicorn.gemspec b/unicorn.gemspec
index 7bb1154..85183d9 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -36,7 +36,6 @@
   # won't have descriptive text, only the numeric status.
   s.add_development_dependency(%q<rack>)
 
-  s.add_dependency(%q<kgio>, '~> 2.6')
   s.add_dependency(%q<raindrops>, '~> 0.7')
 
   s.add_development_dependency('test-unit', '~> 3.0')
-- 
2.35.1


^ permalink raw reply related	[relevance 2%]

* Re: [PATCH] Master promotion with SIGURG (CoW optimization)
  @ 2022-07-06  7:40  4%   ` Jean Boussier
  0 siblings, 0 replies; 31+ results
From: Jean Boussier @ 2022-07-06  7:40 UTC (permalink / raw)
  Cc: unicorn-public

> OK, any numbers from Puma users which can be used to project
> improvements for unicorn (and other servers)?

There are some user reports here: https://github.com/puma/puma/issues/2258
but they are mixed in reports for two other new featurs.

Some reports are up to 20-30% savings, but I'd expect unicorn to benefit
even more from it, given that typical puma users spawn less processes
than with unicorn.

> > They also include automatic reforking after a predetermined amount
> > of requests (1k by default).
>
> 1k seems like a lot of requests (more on this below)

Agreed. Shared pages start being invalidated much faster than that.

> > - master: on `SIGURG`
> >   - Forward `SIGURG` to a chosen worker.
>
> OK, I guess SIGURG is safe to use since nobody else relies on it (right?).

That's my understanding, an alternative could be to re-use USR2 and have a
config flag to define wether it is a rexec or refork.

> Right.  PID file races and corner cases were painful to deal
> with in the earliest days of this project, and I don't look
> forward to having to deal with them ever again...
>
> It looked like the world was moving towards socket-activation
> with dedicated process managers and away from fragile PID files;
> so being self-daemonization dependent seems like a step back.

Agreed. We're currently running unicorn as PID 1 inside containers
and I'm not exactly looking forward to have to minitor a PID file.

Another avenue I explored was to keep the existing master and
refork from one of the worker like puma, but re-assign the parent
to the orignal master using PR_SET_CHILD_SUBREAPER.

Here's my notes of how it works:

### Requirements:

- Need `PR_SET_CHILD_SUBREAPER`, so Linux 3.4+ only (May 2012)
- Need `File.mkfifo`, so Ruby 2.3 (Dec 2015), but can be shimed for older
  rubies.

### Flow:

- master: set `PR_SET_CHILD_SUBREAPER` during boot.
- master: create a temporary directory (`$TMP`).
- master: spawn initial workers the old way.

- master: on `SIGCHLD` (a child died):
  - Fake signal oldest worker (`SIGURG`).
  - Write the new worker number in the fake signal pipe (at the same time).

- worker: on `SIGURG` (should spawn a sibling):
  - Note: worker fake signals are processed after the current request is
    completed.
  - Run `before_fork` callback (MAYBE: need a special `before_refork`
    callback?)
  - create a pipe.
  - daemonize (fork twice so that the new process is attributed to the
    nearest `CHILD_SUBREAPER`, aka the original master).
  - wait for the first child to exit.
  - write into the pipe to signal the parent is dead.
  - Run `after_fork` callback (MAYBE: need a special `after_refork` callback?)

- new sibling after fork:
  - wait for the parent to exit (though the temporary pipe).
  - create a named pipe (`File.mkfifo`) at `$TMP/$WORKER_NUM.pipe`.
  - create a pidfile at `$TMP/WORKER_NUM.pid`.
  - Open the named pipe in `IO::RDONLY | IO::NONBLOCK` (otherwise it would
    block until the master open in write mode).
    - NB: Need to convert it to a `Kgio::Pipe` with
      `Kgio::Pipe.for_fd(f.to_i)`, and keep `f` need `autoclose = false`.
  - Create the `Unicorn::Worker` instance with that pipe and worker number.
  - Send `SIGURG` to the parent process (should be the master).
  - Wait for `SIGCONT` in the named pipe.
  - enter the worker loop.

- master: on `SIGURG`:
  - for each outstanding refork order:
    - Look for `$TMP/$WORKER_NUM.pid` and `$TMP/$WORKER_NUM.pipe`
    - Read the pidfile
    - Open the pipe with `IO::RDONLY | IO::NONBLOCK`
      - NB: Need to convert it to a `Kgio::Pipe` with
        `Kgio::Pipe.for_fd(f.to_i)`, and keep `f` need `autoclose = false`.
    - Delete pidfile and named pipe.
    - Register that new worker normally.
    - Write `SIGCONT` into the pipe.

I can share the patch if you are interested, but the extra complexity
and Linux only features made me prefer the master-promotion approach.

> > However it work well enough for demonstration.
> >
> > So I figured I'd ask wether the feature is desired upstream
> > before investing more effort into it.
>
> It really depends on:
>
> 1) what memory savings and/or speedups can be achieved
>
> 2) what support can we expect from you (and other users) for
>    this feature and potential regressions going forward?
>
> I don't have the free time nor mental capacity to deal with
> fallout from major new features such as this, myself.

Yeah, this is just a RFC, I wouldn't submit a final patch without
first deploying it on our infra with good results. I'm just on the
fence between trying to get this upstream vs maintaining our own
server that solely work like this, hence somewhat simpler and that
can make more assumptions.

> The hook interface would be yet another documentation+support
> burden I'm worried about (and also more unicorn-specific stuff
> to lock people in :<).

Understandable. I suppose it can be done with a monitoring process.
Check `smaps_rollup` and send `SIGURG` when deemed necessary.

More moving pieces but keep unicorn simpler.

> A completely different idea which should achieve the same goal,
> completely independent of unicorn:
>
>   App startup/loading can be made to trigger warmup paths which
>   hit common endpoints.  IOW, app loading could run a small
>   warmup suite which simulates common requests to heat up caches
>   and trigger JIT.

Yeah, I don't really believe in this for a few reasons:

  - That would slow boot time.
  - On large apps there's just too many codepath for this to be realistic.
  - Many endpoints require database and other network access you probably
    don't want in the master.
  - Endpoints may have side effects.

> Ultimately (long-term for ruby-core), it would be better to make
> JIT (and perhaps even VM cache) results persistent and shareable
> across different processes, somehow...  That would help all Ruby
> tools, even short-lived CLI tools.  Portable locking would be a
> pain, I suppose, but proprietary OSes are always a pain :P

Based on my limited understanding of both JIT and VM caches, I don't think
that's really possible.

The VM itself could definitely do better CoW wise, and I have some proposals
on that front (https://bugs.ruby-lang.org/issues/18885) but that will take
time and will never be perfect.

That's why I think periodically promoting a new master has potential.

> There's some line-wrapping issues caused by your MUA;

Urk. Ok, trying another client now, and I'll resend the patch.

> Perhaps documenting this as experimental and subject to removal
> at any time would make the addition of a major new feature an
> easier pill to swallow; but I still think this is better outside
> of app servers.

Up to you, if you don't feel like maintaining such a feature I would
perfectly understand.

---
 SIGNALS                        |   4 ++
 lib/unicorn.rb                 |   2 +-
 lib/unicorn/http_server.rb     | 115 +++++++++++++++++++++++++--------
 lib/unicorn/promoted_worker.rb |  40 ++++++++++++
 4 files changed, 134 insertions(+), 27 deletions(-)
 create mode 100644 lib/unicorn/promoted_worker.rb

diff --git a/SIGNALS b/SIGNALS
index 7321f2b..f5716b9 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -39,6 +39,10 @@ https://yhbt.net/unicorn/examples/init.sh
 * WINCH - gracefully stops workers but keep the master running.
   This will only work for daemonized processes.

+* URG - promote one of the existing workers as a new master, and gracefully
+  stop workers.
+  This will only work for daemonized processes.
+
 * TTIN - increment the number of worker processes by one

 * TTOU - decrement the number of worker processes by one
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 1a50631..832f78d 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -133,6 +133,6 @@ def self.pipe # :nodoc:
 # :enddoc:

 %w(const socket_helper stream_input tee_input http_request configurator
-   tmpio util http_response worker http_server).each do |s|
+   tmpio util http_response worker promoted_worker http_server).each do |s|
   require_relative "unicorn/#{s}"
 end
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 21f2a05..dd8f021 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -50,6 +50,7 @@ class Unicorn::HttpServer
   #   Unicorn::HttpServer::START_CTX[0] = "/home/bofh/2.3.0/bin/unicorn"
   START_CTX = {
     :argv => ARGV.map(&:dup),
+    :generation => 0,
     0 => $0.dup,
   }
   # We favor ENV['PWD'] since it is (usually) symlink aware for Capistrano
@@ -106,7 +107,7 @@ def initialize(app, options = {})
     @orig_app = app
     # list of signals we care about and trap in master.
     @queue_sigs = [
-      :WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, :TTIN, :TTOU ]
+      :WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, :TTIN, :TTOU, :URG ]

     @worker_data = if worker_data = ENV['UNICORN_WORKER']
       worker_data = worker_data.split(',').map!(&:to_i)
@@ -119,7 +120,24 @@ def initialize(app, options = {})

   # Runs the thing.  Returns self so you can run join on it
   def start
+    @new_master = false
     inherit_listeners!
+    promote
+
+    # write pid early for Mongrel compatibility if we're not inheriting sockets
+    # This is needed for compatibility some Monit setups at least.
+    # This unfortunately has the side effect of clobbering valid PID if
+    # we upgrade and the upgrade breaks during preload_app==true && build_app!
+    self.pid = config[:pid]
+
+    build_app! if preload_app
+    bind_new_listeners!
+
+    spawn_missing_workers
+    self
+  end
+
+  def promote
     # this pipe is used to wake us up from select(2) in #join when signals
     # are trapped.  See trap_deferred.
     @self_pipe.replace(Unicorn.pipe)
@@ -130,18 +148,6 @@ def start
     # Note that signals don't actually get handled until the #join method
     @queue_sigs.each { |sig| trap(sig) { @sig_queue << sig; awaken_master } }
     trap(:CHLD) { awaken_master }
-
-    # write pid early for Mongrel compatibility if we're not inheriting sockets
-    # This is needed for compatibility some Monit setups at least.
-    # This unfortunately has the side effect of clobbering valid PID if
-    # we upgrade and the upgrade breaks during preload_app==true && build_app!
-    self.pid = config[:pid]
-
-    build_app! if preload_app
-    bind_new_listeners!
-
-    spawn_missing_workers
-    self
   end

   # replaces current listener set with +listeners+.  This will
@@ -178,16 +184,16 @@ def logger=(obj)
     Unicorn::HttpRequest::DEFAULTS["rack.logger"] = @logger = obj
   end

-  def clobber_pid(path)
+  def clobber_pid(path, content = $$)
     unlink_pid_safe(@pid) if @pid
     if path
       fp = begin
-        tmp = "#{File.dirname(path)}/#{rand}.#$$"
+        tmp = "#{File.dirname(path)}/#{rand}.#{content}"
         File.open(tmp, File::RDWR|File::CREAT|File::EXCL, 0644)
       rescue Errno::EEXIST
         retry
       end
-      fp.syswrite("#$$\n")
+      fp.syswrite("#{content}\n")
       File.rename(fp.path, path)
       fp.close
     end
@@ -279,6 +285,11 @@ def join
       end
       @ready_pipe = @ready_pipe.close rescue nil
     end
+
+    if @promoted
+      Process.kill(:WINCH, Process.ppid)
+    end
+
     begin
       reap_all_workers
       case @sig_queue.shift
@@ -292,6 +303,13 @@ def join
           @logger.debug("waiting #{sleep_time}s after suspend/hibernation")
         end
         maintain_worker_count if respawn
+
+        if @new_master && @new_master.ready? && @workers.empty?
+          # TODO: we should handle the new master dying like with reexec.
+          clobber_pid(pid, @new_master.pid)
+          break
+        end
+
         master_sleep(sleep_time)
       when :QUIT # graceful shutdown
         break
@@ -305,10 +323,20 @@ def join
         soft_kill_each_worker(:USR1)
       when :USR2 # exec binary, stay alive in case something went wrong
         reexec
+      when :URG
+        if $stdin.tty?
+          logger.info "SIGURG ignored because we're not daemonized"
+        else
+          promote_new_master
+        end
       when :WINCH
         if $stdin.tty?
           logger.info "SIGWINCH ignored because we're not daemonized"
         else
+          if @new_master
+            @new_master.ready!
+          end
+
           respawn = false
           logger.info "gracefully stopping all workers"
           soft_kill_each_worker(:QUIT)
@@ -408,11 +436,30 @@ def reap_all_workers
         worker = @workers.delete(wpid) and worker.close rescue nil
         @after_worker_exit.call(self, worker, status)
       end
+
+      if @new_master && @new_master.ready?
+        @new_master.scale(@workers.size)
+      end
     rescue Errno::ECHILD
       break
     end while true
   end

+  def promote_new_master
+    # Promoting the oldest worker
+    # TODO: handle `@new_master` being dead.
+    if @new_master
+      logger.error "can't promote because worker=#{@new_master.nr} is
being promoted"
+    elsif pair = @workers.first
+      @new_master = Unicorn::PromotedWorker.new(*pair, worker_processes)
+      @workers.delete(@new_master.pid)
+      logger.info "master promoting worker=#{@new_master.worker.nr}"
+      @new_master.promote
+    else
+      logger.error "can't promote because there is no existing workers"
+    end
+  end
+
   # reexecutes the START_CTX with a new binary
   def reexec
     if @reexec_pid > 0
@@ -516,10 +563,11 @@ def murder_lazy_workers
   end

   def after_fork_internal
+    self.worker_processes = 0
     @self_pipe.each(&:close).clear # this is master-only, now
     @ready_pipe.close if @ready_pipe
     Unicorn::Configurator::RACKUP.clear
-    @ready_pipe = @init_listeners = @before_exec = @before_fork = nil
+    @ready_pipe = nil

     # The OpenSSL PRNG is seeded with only the pid, and apps with frequently
     # dying workers can recycle pids
@@ -545,6 +593,13 @@ def spawn_missing_workers
       unless pid
         after_fork_internal
         worker_loop(worker)
+
+        if @promoted
+          worker.tick = 0
+          promote
+          join
+        end
+
         exit
       end

@@ -678,19 +733,22 @@ def init_worker_process(worker)
     trap(:CHLD, 'DEFAULT')
     @sig_queue.clear
     proc_name "worker[#{worker.nr}]"
-    START_CTX.clear
     @workers.clear

     after_fork.call(self, worker) # can drop perms and create listeners
     LISTENERS.each { |sock| sock.close_on_exec = true }

     worker.user(*user) if user.kind_of?(Array) && ! worker.switched
-    @config = nil
     build_app! unless preload_app
-    @after_fork = @listener_opts = @orig_app = nil
+    @listener_opts = @orig_app = nil
     readers = LISTENERS.dup
     readers << worker
     trap(:QUIT) { nuke_listeners!(readers) }
+    @promoted = false
+    trap(:URG) do
+      @promoted = true
+      START_CTX[:generation] += 1
+    end
     readers
   end

@@ -706,11 +764,11 @@ def reopen_worker_logs(worker_nr)

   def prep_readers(readers)
     wtr = Unicorn::Waiter.prep_readers(readers)
-    @timeout *= 500 # to milliseconds for epoll, but halved
+    @select_timeout = @timeout * 500 # to milliseconds for epoll, but halved
     wtr
   rescue
     require_relative 'select_waiter'
-    @timeout /= 2.0 # halved for IO.select
+    @select_timeout = @timeout / 2.0 # halved for IO.select
     Unicorn::SelectWaiter.new
   end

@@ -720,7 +778,7 @@ def prep_readers(readers)
   def worker_loop(worker)
     readers = init_worker_process(worker)
     waiter = prep_readers(readers)
-    reopen = false
+    promote = reopen = false

     # this only works immediately if the master sent us the signal
     # (which is the normal case)
@@ -739,12 +797,17 @@ def worker_loop(worker)
           process_client(client)
           worker.tick = time_now.to_i
         end
+        if @promoted
+          worker.tick = time_now.to_i
+          return
+        end
+
         break if reopen
       end

       # timeout so we can .tick and keep parent from SIGKILL-ing us
       worker.tick = time_now.to_i
-      waiter.get_readers(ready, readers, @timeout)
+      waiter.get_readers(ready, readers, @select_timeout)
     rescue => e
       redo if reopen && readers[0]
       Unicorn.log_error(@logger, "listen loop error", e) if readers[0]
@@ -823,8 +886,8 @@ def build_app!
   end

   def proc_name(tag)
-    $0 = ([ File.basename(START_CTX[0]), tag
-          ]).concat(START_CTX[:argv]).join(' ')
+    $0 = ([ File.basename(START_CTX[0]), tag, "(gen:
#{START_CTX[:generation]})",
+          ]).concat(START_CTX[:argv]).compact.join(' ')
   end

   def redirect_io(io, path)
diff --git a/lib/unicorn/promoted_worker.rb b/lib/unicorn/promoted_worker.rb
new file mode 100644
index 0000000..2595182
--- /dev/null
+++ b/lib/unicorn/promoted_worker.rb
@@ -0,0 +1,40 @@
+# -*- encoding: binary -*-
+
+class Unicorn::PromotedWorker
+  attr_reader :pid, :worker
+
+  def initialize(pid, worker, expected_worker_processes)
+    @pid = pid
+    @worker = worker
+    @worker_processes = 0
+    @expected_worker_processes = expected_worker_processes
+    @ready = false
+  end
+
+  def ready?
+    @ready
+  end
+
+  def ready!
+    @ready = true
+  end
+
+  def promote
+    @worker.soft_kill(:URG)
+  end
+
+  def scale(old_master_worker_processes)
+    diff = @expected_worker_processes -
+      old_master_worker_processes -
+      @worker_processes
+
+    if diff > 0
+      diff.times { kill(:TTIN) }
+      @worker_processes += diff
+    end
+  end
+
+  def kill(sig)
+    Process.kill(sig, @pid)
+  end
+end
-- 
2.35.1

^ permalink raw reply related	[relevance 4%]

* [PATCH] Master promotion with SIGURG (CoW optimization)
@ 2022-07-05 20:05  3% Jean Boussier
    0 siblings, 1 reply; 31+ results
From: Jean Boussier @ 2022-07-05 20:05 UTC (permalink / raw)
  To: unicorn-public

Unicorn rely on Copy on Write to limit applications' memory usage,
however Ruby Copy on Write performance isn't exactly perfect.
As code get executed various internal performance caches get warmed
which cause shared pages to be invalidated.

While there's certainly some improvements to be made to Ruby itself
on that front, it will likely get worse in the near future if YJIT
become popular, as it will generate executable code in the workers
hence won't be shared.

One way effective CoW performance could be improved would be to
periodically promote one worker as the new master. Since that worker
processed some requests, VM caches, JITed code, etc should be somewhat
warm already, hence shared pages should be dirtied less frequently after
each promotion.

Puma 5.0.0 introduced a similar experimental feature called
`fork_worker` or `refork`.

It's a bit more limited though, as instead of promoting a new
master and exiting, they only ask worker #0 to fork itself to
replace its siblings. So once all workers are replaces, there
is 3 levels of processes: cluster -> first_worker -> other_workers.

They also include automatic reforking after a predetermined amount
of requests (1k by default).

The happy path works pretty much like this:

- Assume daemonized mode.

- master: on `SIGURG`
   - Forward `SIGURG` to a chosen worker.

- worker: on `SIGURG` (become a master)
   - do the prep, register traps, open self-pipe, etc
   - don't spawn any worker, set number of wokers to 0.
   - send `SIGWHINCH` to master.

- old master: on `SIGWHINCH`
   - Gracefully shutdown workers, like during a reexec.

- old master: when a worker is reaped.
   - Send `SIGTTIN` to the new master
   - If it was the last worker:
     - replace pidfile
     - exit

This patch is not exactly production quality yet:

   - I need to write some tests
   - There's a race condition that can cause the promoted master
     master to have one less worker than required. Need to be addressed.
   - The pidfile replacement should be improved.
   - Multiple corner cases like a `QUIT` during promotion are not handled.

However it work well enough for demonstration.

So I figured I'd ask wether the feature is desired upstream
before investing more effort into it.

For this to be used in production without too much integration effort
I think automatic promotion based on some criteria like number of
request or process lifetime would be needed.

Ideally a hook interface to programatically trigger promotion would
be very valuable as I'd likely want to trigger promotion
based on memory metrics read from `/proc/<pid>/smaps_rollup`.
---
  SIGNALS                        |   4 ++
  lib/unicorn.rb                 |   2 +-
  lib/unicorn/http_server.rb     | 115 +++++++++++++++++++++++++--------
  lib/unicorn/promoted_worker.rb |  40 ++++++++++++
  4 files changed, 134 insertions(+), 27 deletions(-)
  create mode 100644 lib/unicorn/promoted_worker.rb

diff --git a/SIGNALS b/SIGNALS
index 7321f2b..f5716b9 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -39,6 +39,10 @@ https://yhbt.net/unicorn/examples/init.sh
  * WINCH - gracefully stops workers but keep the master running.
    This will only work for daemonized processes.

+* URG - promote one of the existing workers as a new master, and gracefully
+  stop workers.
+  This will only work for daemonized processes.
+
  * TTIN - increment the number of worker processes by one

  * TTOU - decrement the number of worker processes by one
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 1a50631..832f78d 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -133,6 +133,6 @@ def self.pipe # :nodoc:
  # :enddoc:

  %w(const socket_helper stream_input tee_input http_request configurator
-   tmpio util http_response worker http_server).each do |s|
+   tmpio util http_response worker promoted_worker http_server).each do |s|
    require_relative "unicorn/#{s}"
  end
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 21f2a05..dd8f021 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -50,6 +50,7 @@ class Unicorn::HttpServer
    #   Unicorn::HttpServer::START_CTX[0] = "/home/bofh/2.3.0/bin/unicorn"
    START_CTX = {
      :argv => ARGV.map(&:dup),
+    :generation => 0,
      0 => $0.dup,
    }
    # We favor ENV['PWD'] since it is (usually) symlink aware for Capistrano
@@ -106,7 +107,7 @@ def initialize(app, options = {})
      @orig_app = app
      # list of signals we care about and trap in master.
      @queue_sigs = [
-      :WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, :TTIN, :TTOU ]
+      :WINCH, :QUIT, :INT, :TERM, :USR1, :USR2, :HUP, :TTIN, :TTOU, :URG ]

      @worker_data = if worker_data = ENV['UNICORN_WORKER']
        worker_data = worker_data.split(',').map!(&:to_i)
@@ -119,7 +120,24 @@ def initialize(app, options = {})

    # Runs the thing.  Returns self so you can run join on it
    def start
+    @new_master = false
      inherit_listeners!
+    promote
+
+    # write pid early for Mongrel compatibility if we're not inheriting 
sockets
+    # This is needed for compatibility some Monit setups at least.
+    # This unfortunately has the side effect of clobbering valid PID if
+    # we upgrade and the upgrade breaks during preload_app==true && 
build_app!
+    self.pid = config[:pid]
+
+    build_app! if preload_app
+    bind_new_listeners!
+
+    spawn_missing_workers
+    self
+  end
+
+  def promote
      # this pipe is used to wake us up from select(2) in #join when signals
      # are trapped.  See trap_deferred.
      @self_pipe.replace(Unicorn.pipe)
@@ -130,18 +148,6 @@ def start
      # Note that signals don't actually get handled until the #join method
      @queue_sigs.each { |sig| trap(sig) { @sig_queue << sig; 
awaken_master } }
      trap(:CHLD) { awaken_master }
-
-    # write pid early for Mongrel compatibility if we're not inheriting 
sockets
-    # This is needed for compatibility some Monit setups at least.
-    # This unfortunately has the side effect of clobbering valid PID if
-    # we upgrade and the upgrade breaks during preload_app==true && 
build_app!
-    self.pid = config[:pid]
-
-    build_app! if preload_app
-    bind_new_listeners!
-
-    spawn_missing_workers
-    self
    end

    # replaces current listener set with +listeners+.  This will
@@ -178,16 +184,16 @@ def logger=(obj)
      Unicorn::HttpRequest::DEFAULTS["rack.logger"] = @logger = obj
    end

-  def clobber_pid(path)
+  def clobber_pid(path, content = $$)
      unlink_pid_safe(@pid) if @pid
      if path
        fp = begin
-        tmp = "#{File.dirname(path)}/#{rand}.#$$"
+        tmp = "#{File.dirname(path)}/#{rand}.#{content}"
          File.open(tmp, File::RDWR|File::CREAT|File::EXCL, 0644)
        rescue Errno::EEXIST
          retry
        end
-      fp.syswrite("#$$\n")
+      fp.syswrite("#{content}\n")
        File.rename(fp.path, path)
        fp.close
      end
@@ -279,6 +285,11 @@ def join
        end
        @ready_pipe = @ready_pipe.close rescue nil
      end
+
+    if @promoted
+      Process.kill(:WINCH, Process.ppid)
+    end
+
      begin
        reap_all_workers
        case @sig_queue.shift
@@ -292,6 +303,13 @@ def join
            @logger.debug("waiting #{sleep_time}s after 
suspend/hibernation")
          end
          maintain_worker_count if respawn
+
+        if @new_master && @new_master.ready? && @workers.empty?
+          # TODO: we should handle the new master dying like with reexec.
+          clobber_pid(pid, @new_master.pid)
+          break
+        end
+
          master_sleep(sleep_time)
        when :QUIT # graceful shutdown
          break
@@ -305,10 +323,20 @@ def join
          soft_kill_each_worker(:USR1)
        when :USR2 # exec binary, stay alive in case something went wrong
          reexec
+      when :URG
+        if $stdin.tty?
+          logger.info "SIGURG ignored because we're not daemonized"
+        else
+          promote_new_master
+        end
        when :WINCH
          if $stdin.tty?
            logger.info "SIGWINCH ignored because we're not daemonized"
          else
+          if @new_master
+            @new_master.ready!
+          end
+
            respawn = false
            logger.info "gracefully stopping all workers"
            soft_kill_each_worker(:QUIT)
@@ -408,11 +436,30 @@ def reap_all_workers
          worker = @workers.delete(wpid) and worker.close rescue nil
          @after_worker_exit.call(self, worker, status)
        end
+
+      if @new_master && @new_master.ready?
+        @new_master.scale(@workers.size)
+      end
      rescue Errno::ECHILD
        break
      end while true
    end

+  def promote_new_master
+    # Promoting the oldest worker
+    # TODO: handle `@new_master` being dead.
+    if @new_master
+      logger.error "can't promote because worker=#{@new_master.nr} is 
being promoted"
+    elsif pair = @workers.first
+      @new_master = Unicorn::PromotedWorker.new(*pair, worker_processes)
+      @workers.delete(@new_master.pid)
+      logger.info "master promoting worker=#{@new_master.worker.nr}"
+      @new_master.promote
+    else
+      logger.error "can't promote because there is no existing workers"
+    end
+  end
+
    # reexecutes the START_CTX with a new binary
    def reexec
      if @reexec_pid > 0
@@ -516,10 +563,11 @@ def murder_lazy_workers
    end

    def after_fork_internal
+    self.worker_processes = 0
      @self_pipe.each(&:close).clear # this is master-only, now
      @ready_pipe.close if @ready_pipe
      Unicorn::Configurator::RACKUP.clear
-    @ready_pipe = @init_listeners = @before_exec = @before_fork = nil
+    @ready_pipe = nil

      # The OpenSSL PRNG is seeded with only the pid, and apps with 
frequently
      # dying workers can recycle pids
@@ -545,6 +593,13 @@ def spawn_missing_workers
        unless pid
          after_fork_internal
          worker_loop(worker)
+
+        if @promoted
+          worker.tick = 0
+          promote
+          join
+        end
+
          exit
        end

@@ -678,19 +733,22 @@ def init_worker_process(worker)
      trap(:CHLD, 'DEFAULT')
      @sig_queue.clear
      proc_name "worker[#{worker.nr}]"
-    START_CTX.clear
      @workers.clear

      after_fork.call(self, worker) # can drop perms and create listeners
      LISTENERS.each { |sock| sock.close_on_exec = true }

      worker.user(*user) if user.kind_of?(Array) && ! worker.switched
-    @config = nil
      build_app! unless preload_app
-    @after_fork = @listener_opts = @orig_app = nil
+    @listener_opts = @orig_app = nil
      readers = LISTENERS.dup
      readers << worker
      trap(:QUIT) { nuke_listeners!(readers) }
+    @promoted = false
+    trap(:URG) do
+      @promoted = true
+      START_CTX[:generation] += 1
+    end
      readers
    end

@@ -706,11 +764,11 @@ def reopen_worker_logs(worker_nr)

    def prep_readers(readers)
      wtr = Unicorn::Waiter.prep_readers(readers)
-    @timeout *= 500 # to milliseconds for epoll, but halved
+    @select_timeout = @timeout * 500 # to milliseconds for epoll, but 
halved
      wtr
    rescue
      require_relative 'select_waiter'
-    @timeout /= 2.0 # halved for IO.select
+    @select_timeout = @timeout / 2.0 # halved for IO.select
      Unicorn::SelectWaiter.new
    end

@@ -720,7 +778,7 @@ def prep_readers(readers)
    def worker_loop(worker)
      readers = init_worker_process(worker)
      waiter = prep_readers(readers)
-    reopen = false
+    promote = reopen = false

      # this only works immediately if the master sent us the signal
      # (which is the normal case)
@@ -739,12 +797,17 @@ def worker_loop(worker)
            process_client(client)
            worker.tick = time_now.to_i
          end
+        if @promoted
+          worker.tick = time_now.to_i
+          return
+        end
+
          break if reopen
        end

        # timeout so we can .tick and keep parent from SIGKILL-ing us
        worker.tick = time_now.to_i
-      waiter.get_readers(ready, readers, @timeout)
+      waiter.get_readers(ready, readers, @select_timeout)
      rescue => e
        redo if reopen && readers[0]
        Unicorn.log_error(@logger, "listen loop error", e) if readers[0]
@@ -823,8 +886,8 @@ def build_app!
    end

    def proc_name(tag)
-    $0 = ([ File.basename(START_CTX[0]), tag
-          ]).concat(START_CTX[:argv]).join(' ')
+    $0 = ([ File.basename(START_CTX[0]), tag, "(gen: 
#{START_CTX[:generation]})",
+          ]).concat(START_CTX[:argv]).compact.join(' ')
    end

    def redirect_io(io, path)
diff --git a/lib/unicorn/promoted_worker.rb b/lib/unicorn/promoted_worker.rb
new file mode 100644
index 0000000..2595182
--- /dev/null
+++ b/lib/unicorn/promoted_worker.rb
@@ -0,0 +1,40 @@
+# -*- encoding: binary -*-
+
+class Unicorn::PromotedWorker
+  attr_reader :pid, :worker
+
+  def initialize(pid, worker, expected_worker_processes)
+    @pid = pid
+    @worker = worker
+    @worker_processes = 0
+    @expected_worker_processes = expected_worker_processes
+    @ready = false
+  end
+
+  def ready?
+    @ready
+  end
+
+  def ready!
+    @ready = true
+  end
+
+  def promote
+    @worker.soft_kill(:URG)
+  end
+
+  def scale(old_master_worker_processes)
+    diff = @expected_worker_processes -
+      old_master_worker_processes -
+      @worker_processes
+
+    if diff > 0
+      diff.times { kill(:TTIN) }
+      @worker_processes += diff
+    end
+  end
+
+  def kill(sig)
+    Process.kill(sig, @pid)
+  end
+end
-- 
2.35.1


^ permalink raw reply related	[relevance 3%]

* [PATCH 3/3] doc: v3 .onion updates, nntp => nntps, minor wording changes
  @ 2021-12-25 17:41  3% ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2021-12-25 17:41 UTC (permalink / raw)
  To: unicorn-public

Tor v2 .onion support is deprecated, so we're stuck with longer
(but more secure :P) v3 .onion names.  Use NNTPS where available
instead of NNTP to reduce the likelyhood of MiTM censorship and
injection.
---
 .olddoc.yml  | 11 +++++++----
 CONTRIBUTORS |  8 ++++++--
 ISSUES       | 38 ++++++++++++++++++--------------------
 README       |  8 ++++----
 4 files changed, 35 insertions(+), 30 deletions(-)

diff --git a/.olddoc.yml b/.olddoc.yml
index 0609bdbf..10ddc073 100644
--- a/.olddoc.yml
+++ b/.olddoc.yml
@@ -3,7 +3,7 @@ cgit_url: https://yhbt.net/unicorn.git
 rdoc_url: https://yhbt.net/unicorn/
 ml_url:
 - https://yhbt.net/unicorn-public/
-- http://ou63pmih66umazou.onion/unicorn-public/
+- http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/unicorn-public/
 merge_html:
   unicorn_1: Documentation/unicorn.1.html
   unicorn_rails_1: Documentation/unicorn_rails.1.html
@@ -13,10 +13,13 @@ noindex:
 - TODO
 - unicorn_rails_1
 public_email: unicorn-public@yhbt.net
+imap_url:
+- imaps://yhbt.net/inbox.comp.lang.ruby.unicorn.0
+- imap://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.lang.ruby.unicorn.0
 nntp_url:
-- nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-- nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn
+- nntps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
+- nntp://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.lang.ruby.unicorn
 - nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 source_code:
 - git clone https://yhbt.net/unicorn.git
-- torsocks git clone http://ou63pmih66umazou.onion/unicorn.git
+- torsocks git clone http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/unicorn.git
diff --git a/CONTRIBUTORS b/CONTRIBUTORS
index bda399bc..9991fdca 100644
--- a/CONTRIBUTORS
+++ b/CONTRIBUTORS
@@ -1,5 +1,9 @@
-Unicorn developers (let us know if we forgot you):
-* Eric Wong (BDFL, BOFH)
+Unicorn developers (let us know if we forgot you, ...or if you no longer wish
+to be associated with the doofus running this disaster :P):
+* Eric Wong (Bozo Doofus For Life, Bastard Operator From Hell)
+
+There's numerous contributors over email the years, all of our mail
+is archived @ https://yhbt.net/unicorn-public/
 * Suraj N. Kurapati
 * Andrey Stikheev
 * Wayne Larsen
diff --git a/ISSUES b/ISSUES
index 4513ad5b..015de37c 100644
--- a/ISSUES
+++ b/ISSUES
@@ -20,6 +20,10 @@ can interoperate with any bug tracker which can Cc: us plain-text to
 mailto:unicorn-public@yhbt.net   This includes the Debian BTS
 at https://bugs.debian.org/unicorn and possibly others.
 
+unicorn is a server; it does not depend on graphics/audio.  Nobody
+communicating with us will ever be expected to go through the trouble
+of setting up graphics nor audio support.
+
 If your issue is of a sensitive nature or you're just shy in public,
 use anonymity tools such as Tor or Mixmaster; and rely on the public
 mail archives for responses.  Be sure to scrub sensitive log messages
@@ -73,27 +77,21 @@ document distributed with git) on guidelines for patch submission.
 
 == Contact Info
 
-* public: mailto:unicorn-public@yhbt.net
-* nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
-* nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-* imaps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn.0
-* https://yhbt.net/unicorn-public/
-* http://ou63pmih66umazou.onion/unicorn-public/
-
-Mailing list subscription is optional, so Cc: all participants.
+Mail is publicly-archived, SMTP subscription is discouraged to avoid
+servers being a single-point-of-failure, so Cc: all participants.
 
-You can follow along via NNTP or IMAP (read-only):
+The HTTP(S) archives have links to per-thread Atom feeds and downloadable
+mboxes.  Read-only IMAP(S) folders and NNTP(S) newsgroups are also available.
 
-	nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-	nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
-	imaps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn.0
-	imap://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn.0
-
-Or Atom feeds:
+* https://yhbt.net/unicorn-public/
+* http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/unicorn-public/
+* imaps://yhbt.net/inbox.comp.lang.ruby.unicorn.0
+* imap://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.lang.ruby.unicorn.0
+* nntps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
+* nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 
-	https://yhbt.net/unicorn-public/new.atom
-	http://ou63pmih66umazou.onion/unicorn-public/new.atom
+Full Atom feeds:
+* https://yhbt.net/unicorn-public/new.atom
+* http://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/unicorn-public/new.atom
 
-	The HTML archives at https://yhbt.net/unicorn-public/
-	also has links to per-thread Atom feeds and downloadable
-	mboxes.
+We only accept plain-text mail: mailto:unicorn-public@yhbt.net
diff --git a/README b/README
index 5fdd1e8c..22c27e84 100644
--- a/README
+++ b/README
@@ -137,13 +137,13 @@ information on the {mailing list}[mailto:unicorn-public@yhbt.net].
 The mailing list is archived at https://yhbt.net/unicorn-public/
 
 Read-only NNTP access is available at:
-nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn and
+nntps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn and
 nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 
 Read-only IMAP access is also avaialble at:
-imaps://news.public-inbox.org/inbox.comp.lang.ruby.unicorn.0 and
-imap://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn.0
-AUTH=ANONYMOUS mechanism is supported, as is any username+password
+imaps://yhbt.net/inbox.comp.lang.ruby.unicorn.0 and
+imap://7fh6tueqddpjyxjmgtdiueylzoqt6pt7hec3pukyptlmohoowvhde4yd.onion/inbox.comp.lang.ruby.unicorn.0
+The AUTH=ANONYMOUS mechanism is supported, as is any username+password
 combination.
 
 For the latest on unicorn releases, you may also finger us at

^ permalink raw reply related	[relevance 3%]

* [PATCH] doc: s/bogomips.org/yhbt.net/g
  @ 2020-01-14  7:46  1% ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2020-01-14  7:46 UTC (permalink / raw)
  To: unicorn-public

bogomips.org is due to expire, soon, and I'm not willing to pay
extortionist fees to Ethos Capital/PIR/ICANN to keep a .org.  So
it's at yhbt.net, for now, but it will change again to
whatever's affordable...  Identity is overrated.

Tor users can use .onions and kick ICANN to the curb:

	torsocks w3m http://unicorn.ou63pmih66umazou.onion/
	torsocks git clone http://ou63pmih66umazou.onion/unicorn.git/
	torsocks w3m http://ou63pmih66umazou.onion/unicorn-public/

While we're at it, `s/news.gmane.org/news.gmane.io/g', too.
(but I suspect that'll need to be resynched since our mail
"List-Id:" header is changing).
---
 .olddoc.yml                      | 19 ++++++++++++-------
 Documentation/unicorn.1          |  8 ++++----
 Documentation/unicorn_rails.1    |  8 ++++----
 FAQ                              |  2 +-
 GNUmakefile                      |  4 ++--
 HACKING                          |  2 +-
 ISSUES                           | 24 ++++++++++++------------
 KNOWN_ISSUES                     |  4 ++--
 Links                            | 10 +++++-----
 README                           | 12 ++++++------
 SIGNALS                          |  2 +-
 Sandbox                          |  4 ++--
 archive/slrnpull.conf            |  2 +-
 examples/big_app_gc.rb           |  2 +-
 examples/logrotate.conf          |  4 ++--
 examples/nginx.conf              |  2 +-
 examples/unicorn.conf.minimal.rb |  4 ++--
 examples/unicorn.conf.rb         |  4 ++--
 ext/unicorn_http/unicorn_http.rl |  2 +-
 lib/unicorn.rb                   |  2 +-
 lib/unicorn/configurator.rb      |  6 +++---
 lib/unicorn/http_server.rb       |  2 +-
 lib/unicorn/oob_gc.rb            |  4 ++--
 unicorn.gemspec                  |  4 ++--
 24 files changed, 71 insertions(+), 66 deletions(-)

diff --git a/.olddoc.yml b/.olddoc.yml
index d2d340f..0609bdb 100644
--- a/.olddoc.yml
+++ b/.olddoc.yml
@@ -1,8 +1,9 @@
 ---
-cgit_url: https://bogomips.org/unicorn.git
-git_url: https://bogomips.org/unicorn.git
-rdoc_url: https://bogomips.org/unicorn/
-ml_url: https://bogomips.org/unicorn-public/
+cgit_url: https://yhbt.net/unicorn.git
+rdoc_url: https://yhbt.net/unicorn/
+ml_url:
+- https://yhbt.net/unicorn-public/
+- http://ou63pmih66umazou.onion/unicorn-public/
 merge_html:
   unicorn_1: Documentation/unicorn.1.html
   unicorn_rails_1: Documentation/unicorn_rails.1.html
@@ -11,7 +12,11 @@ noindex:
 - LATEST
 - TODO
 - unicorn_rails_1
-public_email: unicorn-public@bogomips.org
+public_email: unicorn-public@yhbt.net
 nntp_url:
-  - nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-  - nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
+- nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
+- nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.unicorn
+- nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
+source_code:
+- git clone https://yhbt.net/unicorn.git
+- torsocks git clone http://ou63pmih66umazou.onion/unicorn.git
diff --git a/Documentation/unicorn.1 b/Documentation/unicorn.1
index 3f8cb96..d76d40f 100644
--- a/Documentation/unicorn.1
+++ b/Documentation/unicorn.1
@@ -154,7 +154,7 @@ TTIN \- increment the number of worker processes by one
 .IP \[bu] 2
 TTOU \- decrement the number of worker processes by one
 .PP
-See the SIGNALS (https://bogomips.org/unicorn/SIGNALS.html) document for
+See the SIGNALS (https://yhbt.net/unicorn/SIGNALS.html) document for
 full description of all signals used by Unicorn.
 .SH RACK ENVIRONMENT
 .PP
@@ -204,11 +204,11 @@ the unicorn config file.
 \f[I]Rack::Builder\f[] ri/RDoc
 .IP \[bu] 2
 \f[I]Unicorn::Configurator\f[] ri/RDoc
-.UR https://bogomips.org/unicorn/Unicorn/Configurator.html
+.UR https://yhbt.net/unicorn/Unicorn/Configurator.html
 .UE
 .IP \[bu] 2
 unicorn RDoc
-.UR https://bogomips.org/unicorn/
+.UR https://yhbt.net/unicorn/
 .UE
 .IP \[bu] 2
 Rack RDoc
@@ -219,4 +219,4 @@ Rackup HowTo
 .UR https://github.com/rack/rack/wiki/(tutorial)-rackup-howto
 .UE
 .SH AUTHORS
-The Unicorn Community <unicorn-public@bogomips.org>.
+The Unicorn Community <unicorn-public@yhbt.net>.
diff --git a/Documentation/unicorn_rails.1 b/Documentation/unicorn_rails.1
index 71c80be..fec0a2a 100644
--- a/Documentation/unicorn_rails.1
+++ b/Documentation/unicorn_rails.1
@@ -180,7 +180,7 @@ TTIN \- increment the number of worker processes by one
 .IP \[bu] 2
 TTOU \- decrement the number of worker processes by one
 .PP
-See the SIGNALS (https://bogomips.org/unicorn/SIGNALS.html) document for
+See the SIGNALS (https://yhbt.net/unicorn/SIGNALS.html) document for
 full description of all signals used by Unicorn.
 .SH SEE ALSO
 .IP \[bu] 2
@@ -189,11 +189,11 @@ unicorn(1)
 \f[I]Rack::Builder\f[] ri/RDoc
 .IP \[bu] 2
 \f[I]Unicorn::Configurator\f[] ri/RDoc
-.UR https://bogomips.org/unicorn/Unicorn/Configurator.html
+.UR https://yhbt.net/unicorn/Unicorn/Configurator.html
 .UE
 .IP \[bu] 2
 unicorn RDoc
-.UR https://bogomips.org/unicorn/
+.UR https://yhbt.net/unicorn/
 .UE
 .IP \[bu] 2
 Rack RDoc
@@ -204,4 +204,4 @@ Rackup HowTo
 .UR https://github.com/rack/rack/wiki/(tutorial)-rackup-howto
 .UE
 .SH AUTHORS
-The Unicorn Community <unicorn-public@bogomips.org>.
+The Unicorn Community <unicorn-public@yhbt.net>.
diff --git a/FAQ b/FAQ
index 4ae2034..018ca92 100644
--- a/FAQ
+++ b/FAQ
@@ -7,7 +7,7 @@ drained entirely by the application.  This may happen when request
 bodies are gzipped, as unicorn reads request body data lazily to avoid
 overhead from bad requests.
 
-Ref: https://bogomips.org/unicorn-public/FC91211E-FD32-432C-92FC-0318714C2170@zendesk.com/
+Ref: https://yhbt.net/unicorn-public/FC91211E-FD32-432C-92FC-0318714C2170@zendesk.com/
 
 === Why aren't my Rails log files rotated when I use SIGUSR1?
 
diff --git a/GNUmakefile b/GNUmakefile
index 94c46ee..eac3473 100644
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -193,13 +193,13 @@ doc: .document $(ext)/unicorn_http.c man html .olddoc.yml $(PLACEHOLDERS)
 	install -m644 $(man1_paths) doc/
 	tar cf - $$(git ls-files examples/) | (cd doc && tar xf -)
 
-# publishes docs to https://bogomips.org/unicorn/
+# publishes docs to https://yhbt.net/unicorn/
 publish_doc:
 	-git set-file-times
 	$(MAKE) doc
 	$(MAKE) doc_gz
 	chmod 644 $$(find doc -type f)
-	$(RSYNC) -av doc/ bogomips.org:/srv/bogomips/unicorn/
+	$(RSYNC) -av doc/ yhbt.net:/srv/yhbt/unicorn/
 	git ls-files | xargs touch
 
 # Create gzip variants of the same timestamp as the original so nginx
diff --git a/HACKING b/HACKING
index be1bb85..976bf92 100644
--- a/HACKING
+++ b/HACKING
@@ -57,7 +57,7 @@ Please wrap documentation at 72 characters-per-line or less (long URLs
 are exempt) so it is comfortably readable from terminals.
 
 When referencing mailing list posts, use
-<tt>https://bogomips.org/unicorn-public/$MESSAGE_ID/</tt> if possible
+<tt>https://yhbt.net/unicorn-public/$MESSAGE_ID/</tt> if possible
 since the Message-ID remains searchable even if a particular site
 becomes unavailable.
 
diff --git a/ISSUES b/ISSUES
index 473da2f..d11dc56 100644
--- a/ISSUES
+++ b/ISSUES
@@ -1,9 +1,9 @@
 = Issues
 
-mailto:unicorn-public@bogomips.org is the best place to report bugs,
+mailto:unicorn-public@yhbt.net is the best place to report bugs,
 submit patches and/or obtain support after you have searched the
-{email archives}[https://bogomips.org/unicorn-public/] and
-{documentation}[https://bogomips.org/unicorn/].
+{email archives}[https://yhbt.net/unicorn-public/] and
+{documentation}[https://yhbt.net/unicorn/].
 
 * No subscription will ever be required to email us
 * Cc: all participants in a thread or commit, as subscription is optional
@@ -12,12 +12,12 @@ submit patches and/or obtain support after you have searched the
 * Do not send HTML mail or images,
   they hurt reader privacy and will be flagged as spam
 * Anonymous and pseudonymous messages will ALWAYS be welcome
-* The email submission port (587) is enabled on the bogomips.org MX:
-  https://bogomips.org/unicorn-public/20141004232241.GA23908@dcvr.yhbt.net/t/
+* The email submission port (587) is enabled on the yhbt.net MX:
+  https://yhbt.net/unicorn-public/20141004232241.GA23908@dcvr.yhbt.net/t/
 
 We will never have a centralized or formal bug tracker.  Instead we
 can interoperate with any bug tracker which can Cc: us plain-text to
-mailto:unicorn-public@bogomips.org   This includes the Debian BTS
+mailto:unicorn-public@yhbt.net   This includes the Debian BTS
 at https://bugs.debian.org/unicorn and possibly others.
 
 If your issue is of a sensitive nature or you're just shy in public,
@@ -73,10 +73,10 @@ document distributed with git) on guidelines for patch submission.
 
 == Contact Info
 
-* public: mailto:unicorn-public@bogomips.org
-* nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
+* public: mailto:unicorn-public@yhbt.net
+* nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 * nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-* https://bogomips.org/unicorn-public/
+* https://yhbt.net/unicorn-public/
 * http://ou63pmih66umazou.onion/unicorn-public/
 
 Mailing list subscription is optional, so Cc: all participants.
@@ -84,13 +84,13 @@ Mailing list subscription is optional, so Cc: all participants.
 You can follow along via NNTP (read-only):
 
 	nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
-	nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
+	nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 
 Or Atom feeds:
 
-	https://bogomips.org/unicorn-public/new.atom
+	https://yhbt.net/unicorn-public/new.atom
 	http://ou63pmih66umazou.onion/unicorn-public/new.atom
 
-	The HTML archives at https://bogomips.org/unicorn-public/
+	The HTML archives at https://yhbt.net/unicorn-public/
 	also has links to per-thread Atom feeds and downloadable
 	mboxes.
diff --git a/KNOWN_ISSUES b/KNOWN_ISSUES
index ebd4822..0017f20 100644
--- a/KNOWN_ISSUES
+++ b/KNOWN_ISSUES
@@ -9,7 +9,7 @@ acceptable solution.  Those issues are documented here.
   handlers.
 
 * Issues with FreeBSD jails can be worked around as documented by Tatsuya Ono:
-  https://bogomips.org/unicorn-public/CAHBuKRj09FdxAgzsefJWotexw-7JYZGJMtgUp_dhjPz9VbKD6Q@mail.gmail.com/
+  https://yhbt.net/unicorn-public/CAHBuKRj09FdxAgzsefJWotexw-7JYZGJMtgUp_dhjPz9VbKD6Q@mail.gmail.com/
 
 * PRNGs (pseudo-random number generators) loaded before forking
   (e.g. "preload_app true") may need to have their internal state
@@ -60,7 +60,7 @@ acceptable solution.  Those issues are documented here.
   application to use Rails 2.3.2 and you have no other choice, then
   you may edit your unicorn gemspec and remove the Rack dependency.
 
-  ref: https://bogomips.org/unicorn-public/20091014221552.GA30624@dcvr.yhbt.net/
+  ref: https://yhbt.net/unicorn-public/20091014221552.GA30624@dcvr.yhbt.net/
   Note: the workaround described in the article above only made
   the issue more subtle and we didn't notice them immediately.
 
diff --git a/Links b/Links
index 10551a6..f81142d 100644
--- a/Links
+++ b/Links
@@ -2,7 +2,7 @@
 
 If you're interested in unicorn, you may be interested in some of the projects
 listed below.  If you have any links to add/change/remove, please tell us at
-mailto:unicorn-public@bogomips.org!
+mailto:unicorn-public@yhbt.net!
 
 == Disclaimer
 
@@ -23,10 +23,10 @@ or services behind them.
 * {golden_brindle}[https://github.com/simonoff/golden_brindle] - tool to
   manage multiple unicorn instances/applications on a single server
 
-* {raindrops}[https://bogomips.org/raindrops/] - real-time stats for
+* {raindrops}[https://yhbt.net/raindrops/] - real-time stats for
   preforking Rack servers
 
-* {UnXF}[https://bogomips.org/unxf/]  Un-X-Forward* the Rack environment,
+* {UnXF}[https://yhbt.net/unxf/]  Un-X-Forward* the Rack environment,
   useful since unicorn is designed to be deployed behind a reverse proxy.
 
 === unicorn is written to work with
@@ -52,7 +52,7 @@ or services behind them.
 * {Mongrel}[https://rubygems.org/gems/mongrel] - the awesome webserver
   unicorn is based on.  A historical archive of the mongrel dev list
   featuring early discussions of unicorn is available at:
-  https://bogomips.org/mongrel-devel/
+  https://yhbt.net/mongrel-devel/
 
-* {david}[https://bogomips.org/david.git] - a tool to explain why you need
+* {david}[https://yhbt.net/david.git] - a tool to explain why you need
   nginx in front of unicorn
diff --git a/README b/README
index 89467fc..0e95f48 100644
--- a/README
+++ b/README
@@ -80,12 +80,12 @@ You may install it via RubyGems on RubyGems.org:
 You can get the latest source via git from the following locations
 (these versions may not be stable):
 
-  https://bogomips.org/unicorn.git
+  https://yhbt.net/unicorn.git
   https://repo.or.cz/unicorn.git (mirror)
 
 You may browse the code from the web:
 
-* https://bogomips.org/unicorn.git
+* https://yhbt.net/unicorn.git
 * https://repo.or.cz/w/unicorn.git (gitweb)
 
 See the HACKING guide on how to contribute and build prerelease gems
@@ -133,13 +133,13 @@ and libraries which run on top of it.
 
 All feedback (bug reports, user/development dicussion, patches, pull
 requests) go to the mailing list/newsgroup.  See the ISSUES document for
-information on the {mailing list}[mailto:unicorn-public@bogomips.org].
+information on the {mailing list}[mailto:unicorn-public@yhbt.net].
 
-The mailing list is archived at https://bogomips.org/unicorn-public/
+The mailing list is archived at https://yhbt.net/unicorn-public/
 Read-only NNTP access is available at:
 nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn and
-nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
+nntp://news.gmane.io/gmane.comp.lang.ruby.unicorn.general
 
 For the latest on unicorn releases, you may also finger us at
-unicorn@bogomips.org or check our NEWS page (and subscribe to our Atom
+unicorn@yhbt.net or check our NEWS page (and subscribe to our Atom
 feed).
diff --git a/SIGNALS b/SIGNALS
index 1af851d..7321f2b 100644
--- a/SIGNALS
+++ b/SIGNALS
@@ -8,7 +8,7 @@ should be possible to easily share process management scripts between
 Unicorn and nginx.
 
 One example init script is distributed with unicorn:
-https://bogomips.org/unicorn/examples/init.sh
+https://yhbt.net/unicorn/examples/init.sh
 
 === Master Process
 
diff --git a/Sandbox b/Sandbox
index d0f915e..651e5cd 100644
--- a/Sandbox
+++ b/Sandbox
@@ -34,7 +34,7 @@ is the primary issue with sandboxing tools such as Bundler and Isolate.
 If you're bundling unicorn, use "bundle exec unicorn" (or "bundle exec
 unicorn_rails") to start unicorn with the correct environment variables
 
-ref: https://bogomips.org/unicorn-public/9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us/
+ref: https://yhbt.net/unicorn-public/9ECF07C4-5216-47BE-961D-AFC0F0C82060@internetfamo.us/
 
 Otherwise (if you choose to not sandbox your unicorn installation), we
 expect the tips for Isolate (below) apply, too.
@@ -44,7 +44,7 @@ expect the tips for Isolate (below) apply, too.
 This is no longer be an issue as of bundler 0.9.17
 
 ref:
-https://bogomips.org/unicorn-public/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com/
+https://yhbt.net/unicorn-public/8FC34B23-5994-41CC-B5AF-7198EF06909E@tramchase.com/
 
 === BUNDLE_GEMFILE for Capistrano users
 
diff --git a/archive/slrnpull.conf b/archive/slrnpull.conf
index fcfcafe..fd04f97 100644
--- a/archive/slrnpull.conf
+++ b/archive/slrnpull.conf
@@ -1,4 +1,4 @@
 # group_name                         max        expire     headers_only
 gmane.comp.lang.ruby.unicorn.general 1000000000 1000000000 0
 
-# usage: slrnpull -d $PWD -h news.gmane.org --no-post
+# usage: slrnpull -d $PWD -h news.gmane.io --no-post
diff --git a/examples/big_app_gc.rb b/examples/big_app_gc.rb
index 9d05719..c1bae10 100644
--- a/examples/big_app_gc.rb
+++ b/examples/big_app_gc.rb
@@ -1,2 +1,2 @@
-# see {Unicorn::OobGC}[https://bogomips.org/unicorn/Unicorn/OobGC.html]
+# see {Unicorn::OobGC}[https://yhbt.net/unicorn/Unicorn/OobGC.html]
 # Unicorn::OobGC was broken in Unicorn v3.3.1 - v3.6.1 and fixed in v3.6.2
diff --git a/examples/logrotate.conf b/examples/logrotate.conf
index 77a01b5..c3aa40d 100644
--- a/examples/logrotate.conf
+++ b/examples/logrotate.conf
@@ -5,7 +5,7 @@
 #    https://linux.die.net/man/8/logrotate
 #
 # public logrotate-related discussion in our archives:
-#    https://bogomips.org/unicorn-public/?q=logrotate
+#    https://yhbt.net/unicorn-public/?q=logrotate
 
 # Modify the following glob to match the logfiles your app writes to:
 /var/log/unicorn_app/*.log {
@@ -33,7 +33,7 @@
 		systemctl kill -s SIGUSR1 unicorn@2.service
 
 		# Examples for other process management systems appreciated
-		# Mail us at unicorn-public@bogomips.org
+		# Mail us at unicorn-public@yhbt.net
 		# (see above for archives)
 
 		# If you use a pid file and assuming your pid file
diff --git a/examples/nginx.conf b/examples/nginx.conf
index b6b69c1..c5026f9 100644
--- a/examples/nginx.conf
+++ b/examples/nginx.conf
@@ -113,7 +113,7 @@ http {
     # try_files directive appeared in in nginx 0.7.27 and has stabilized
     # over time.  Older versions of nginx (e.g. 0.6.x) requires
     # "if (!-f $request_filename)" which was less efficient:
-    # https://bogomips.org/unicorn.git/tree/examples/nginx.conf?id=v3.3.1#n127
+    # https://yhbt.net/unicorn.git/tree/examples/nginx.conf?id=v3.3.1#n127
     try_files $uri/index.html $uri.html $uri @app;
 
     location @app {
diff --git a/examples/unicorn.conf.minimal.rb b/examples/unicorn.conf.minimal.rb
index 2d1bf0a..46fd634 100644
--- a/examples/unicorn.conf.minimal.rb
+++ b/examples/unicorn.conf.minimal.rb
@@ -1,9 +1,9 @@
 # Minimal sample configuration file for Unicorn (not Rack) when used
 # with daemonization (unicorn -D) started in your working directory.
 #
-# See https://bogomips.org/unicorn/Unicorn/Configurator.html for complete
+# See https://yhbt.net/unicorn/Unicorn/Configurator.html for complete
 # documentation.
-# See also https://bogomips.org/unicorn/examples/unicorn.conf.rb for
+# See also https://yhbt.net/unicorn/examples/unicorn.conf.rb for
 # a more verbose configuration using more features.
 
 listen 2007 # by default Unicorn listens on port 8080
diff --git a/examples/unicorn.conf.rb b/examples/unicorn.conf.rb
index d2897ef..d90bdc4 100644
--- a/examples/unicorn.conf.rb
+++ b/examples/unicorn.conf.rb
@@ -2,10 +2,10 @@
 #
 # This configuration file documents many features of Unicorn
 # that may not be needed for some applications. See
-# https://bogomips.org/unicorn/examples/unicorn.conf.minimal.rb
+# https://yhbt.net/unicorn/examples/unicorn.conf.minimal.rb
 # for a much simpler configuration file.
 #
-# See https://bogomips.org/unicorn/Unicorn/Configurator.html for complete
+# See https://yhbt.net/unicorn/Unicorn/Configurator.html for complete
 # documentation.
 
 # Use at least one worker per core if you're on a dedicated server,
diff --git a/ext/unicorn_http/unicorn_http.rl b/ext/unicorn_http/unicorn_http.rl
index 8ef23bc..dfe3a63 100644
--- a/ext/unicorn_http/unicorn_http.rl
+++ b/ext/unicorn_http/unicorn_http.rl
@@ -487,7 +487,7 @@ static void set_url_scheme(VALUE env, VALUE *server_port)
      * and X-Forwarded-Proto handling from this parser?  We've had it
      * forever and nobody has said anything against it, either.
      * Anyways, please send comments to our public mailing list:
-     * unicorn-public@bogomips.org (no HTML mail, no subscription necessary)
+     * unicorn-public@yhbt.net (no HTML mail, no subscription necessary)
      */
     scheme = rb_hash_aref(env, g_http_x_forwarded_ssl);
     if (!NIL_P(scheme) && STR_CSTR_EQ(scheme, "on")) {
diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index dd5dff4..d5991fe 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -96,7 +96,7 @@ def self.builder(ru, op)
 
   # returns an array of strings representing TCP listen socket addresses
   # and Unix domain socket paths.  This is useful for use with
-  # Raindrops::Middleware under Linux: https://bogomips.org/raindrops/
+  # Raindrops::Middleware under Linux: https://yhbt.net/raindrops/
   def self.listener_names
     Unicorn::HttpServer::LISTENERS.map do |io|
       Unicorn::SocketHelper.sock_name(io)
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index e8b76f5..c3a4f2d 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -3,11 +3,11 @@
 
 # Implements a simple DSL for configuring a unicorn server.
 #
-# See https://bogomips.org/unicorn/examples/unicorn.conf.rb and
-# https://bogomips.org/unicorn/examples/unicorn.conf.minimal.rb
+# See https://yhbt.net/unicorn/examples/unicorn.conf.rb and
+# https://yhbt.net/unicorn/examples/unicorn.conf.minimal.rb
 # example configuration files.  An example config file for use with
 # nginx is also available at
-# https://bogomips.org/unicorn/examples/nginx.conf
+# https://yhbt.net/unicorn/examples/nginx.conf
 #
 # See the link:/TUNING.html document for more information on tuning unicorn.
 class Unicorn::Configurator
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 5334fa0..a52931a 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -6,7 +6,7 @@
 # forked worker children.
 #
 # Users do not need to know the internals of this class, but reading the
-# {source}[https://bogomips.org/unicorn.git/tree/lib/unicorn/http_server.rb]
+# {source}[https://yhbt.net/unicorn.git/tree/lib/unicorn/http_server.rb]
 # is education for programmers wishing to learn how unicorn works.
 # See Unicorn::Configurator for information on how to configure unicorn.
 class Unicorn::HttpServer
diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb
index c4741a0..3b2f488 100644
--- a/lib/unicorn/oob_gc.rb
+++ b/lib/unicorn/oob_gc.rb
@@ -43,8 +43,8 @@
 #     use Unicorn::OobGC, 2, %r{\A/(?:expensive/foo|more_expensive/foo)}
 #
 # Feedback from users of early implementations of this module:
-# * https://bogomips.org/unicorn-public/0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com/
-# * https://bogomips.org/unicorn-public/AANLkTilUbgdyDv9W1bi-s_W6kq9sOhWfmuYkKLoKGOLj@mail.gmail.com/
+# * https://yhbt.net/unicorn-public/0BFC98E9-072B-47EE-9A70-05478C20141B@lukemelia.com/
+# * https://yhbt.net/unicorn-public/AANLkTilUbgdyDv9W1bi-s_W6kq9sOhWfmuYkKLoKGOLj@mail.gmail.com/
 
 module Unicorn::OobGC
 
diff --git a/unicorn.gemspec b/unicorn.gemspec
index ceea831..a189f8d 100644
--- a/unicorn.gemspec
+++ b/unicorn.gemspec
@@ -15,14 +15,14 @@
   s.authors = ['unicorn hackers']
   s.summary = 'Rack HTTP server for fast clients and Unix'
   s.description = File.read('README').split("\n\n")[1]
-  s.email = %q{unicorn-public@bogomips.org}
+  s.email = %q{unicorn-public@yhbt.net}
   s.executables = %w(unicorn unicorn_rails)
   s.extensions = %w(ext/unicorn_http/extconf.rb)
   s.extra_rdoc_files = IO.readlines('.document').map!(&:chomp!).keep_if do |f|
     File.exist?(f)
   end
   s.files = manifest
-  s.homepage = 'https://bogomips.org/unicorn/'
+  s.homepage = 'https://yhbt.net/unicorn/'
   s.test_files = test_files
 
   # technically we need ">= 1.9.3", too, but avoid the array here since

^ permalink raw reply related	[relevance 1%]

* [PATCH] shrink pipes under Linux
@ 2018-08-20 20:30  4% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2018-08-20 20:30 UTC (permalink / raw)
  To: unicorn-public; +Cc: Eric Wong

We have never had any need for pipes with the default 64K
capacity on Linux.  Our pipes are only used for tiny writes
in signal handlers and to perform parent shutdown detection.

With the current /proc/sys/fs/pipe-user-pages-soft
default, only 1024 pipes can be created by an unprivileged
user before Linux clamps down the pipe size to 4K (a single page)
for newly-created pipes[1].

So avoid penalizing OTHER pipe users who could benefit from the
increased capacity and use only a single page for ourselves.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/pipe.c?h=v4.18#n642
---
 lib/unicorn.rb              | 18 ++++++++++++++++--
 lib/unicorn/http_request.rb |  1 -
 lib/unicorn/launcher.rb     |  2 +-
 test/unit/test_util.rb      | 23 +++++++++++++++++++++++
 4 files changed, 40 insertions(+), 4 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index 85e4df1..b6dae36 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -2,6 +2,7 @@
 require 'etc'
 require 'stringio'
 require 'kgio'
+require 'raindrops'
 require 'io/wait'
 
 begin
@@ -113,9 +114,22 @@ def self.log_error(logger, prefix, exc)
     exc.backtrace.each { |line| logger.error(line) }
   end
 
-  # remove this when we only support Ruby >= 2.0
+  F_SETPIPE_SZ = 1031 if RUBY_PLATFORM =~ /linux/
+
   def self.pipe # :nodoc:
-    Kgio::Pipe.new.each { |io| io.close_on_exec = true }
+    Kgio::Pipe.new.each do |io|
+      io.close_on_exec = true  # remove this when we only support Ruby >= 2.0
+
+      # shrink pipes to minimize impact on /proc/sys/fs/pipe-user-pages-soft
+      # limits.
+      if defined?(F_SETPIPE_SZ)
+        begin
+          io.fcntl(F_SETPIPE_SZ, Raindrops::PAGE_SIZE)
+        rescue Errno::EINVAL
+          # old kernel
+        end
+      end
+    end
   end
   # :startdoc:
 end
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index d713b19..8bb884b 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -2,7 +2,6 @@
 # :enddoc:
 # no stable API here
 require 'unicorn_http'
-require 'raindrops'
 
 # TODO: remove redundant names
 Unicorn.const_set(:HttpRequest, Unicorn::HttpParser)
diff --git a/lib/unicorn/launcher.rb b/lib/unicorn/launcher.rb
index 5eafe5b..78e8f39 100644
--- a/lib/unicorn/launcher.rb
+++ b/lib/unicorn/launcher.rb
@@ -31,7 +31,7 @@ def self.daemonize!(options)
       #  \_ parent  - exits immediately ASAP
       #      \_ unicorn master - writes to pipe when ready
 
-      rd, wr = IO.pipe
+      rd, wr = Unicorn.pipe
       grandparent = $$
       if fork
         wr.close # grandparent does not write
diff --git a/test/unit/test_util.rb b/test/unit/test_util.rb
index dc6302e..586fc6c 100644
--- a/test/unit/test_util.rb
+++ b/test/unit/test_util.rb
@@ -102,4 +102,27 @@ def test_reopen_logs_renamed_with_internal_encoding
     }
     tmp.close!
   end
+
+  def test_pipe
+    r, w = Unicorn.pipe
+    assert r
+    assert w
+
+    return if RUBY_PLATFORM =~ /linux/
+
+    f_getpipe_sz = 1032
+    IO.pipe do |a, b|
+      a_sz = a.fcntl(f_getpipe_sz)
+      b_sz = b.fcntl(f_getpipe_sz)
+      assert_kind_of Integer, a_sz
+      r_sz = r.fcntl(f_getpipe_sz)
+      assert_equal Raindrops::PAGE_SIZE, r_sz
+      assert_operator a_sz, :>=, r_sz
+    end
+  rescue Errno::EINVAL
+    # Linux <= 2.6.34
+  ensure
+    w.close
+    r.close
+  end
 end
-- 
EW


^ permalink raw reply related	[relevance 4%]

* [ANN] raindrops 0.18.0 - real-time stats for preforking Rack servers
@ 2017-03-23  2:48  4% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2017-03-23  2:48 UTC (permalink / raw)
  To: ruby-talk, raindrops-public; +Cc: unicorn-public, Simon Eskildsen, Jeremy Evans

raindrops is a real-time stats toolkit to show statistics for Rack HTTP
servers.  It is designed for preforking servers such as unicorn, but
should support any Rack HTTP server on platforms supporting POSIX shared
memory.  It may also be used as a generic scoreboard for sharing atomic
counters across multiple processes.

* https://bogomips.org/raindrops/
* No subscription necessary, no HTML mail:
  raindrops-public@bogomips.org
* mail archives: https://bogomips.org/raindrops-public/
  http://ou63pmih66umazou.onion/raindrops-public/
  nntp://news.public-inbox.org/inbox.comp.lang.ruby.raindrops
  nntp://ou63pmih66umazou.onion/inbox.comp.lang.ruby.raindrops
* git clone git://bogomips.org/raindrops.git
* https://bogomips.org/raindrops/NEWS.atom.xml
* Demo site: https://raindrops-demo.bogomips.org:8443/

Changes:

    raindrops 0.18.0
    
    The most notable feature of this release is the addition of
    FreeBSD and OpenBSD TCP_INFO support.  This includes the
    Raindrops::TCP for portably mapping TCP state names to
    platform-dependent numeric values:
    
      https://bogomips.org/raindrops/Raindrops.html#TCP
    
    Thanks to Jeremy Evans and Simon Eskildsen on the
    unicorn-public@bogomips.org mailing list for inspiring
    these changes to raindrops.
    
    There's also a few internal cleanups, and documentation
    improvements, including some fixes to the largely-forgotten
    Raindrops::Aggreage::PMQ class:
    
      https://bogomips.org/raindrops/Raindrops/Aggregate/PMQ.html
    
    20 changes since 0.17.0:
    
          test_inet_diag_socket: fix Fixnum deprecation warning
          TODO: add item for IPv6 breakage
          ext: fix documentation for C ext-defined classes
          TCP_Info: custom documentation for #get!
          TypedData C-API conversion
          test_watcher: disable test correctly when aggregate is missing
          tcp_info: support this struct under FreeBSD
          define Raindrops::TCP hash for TCP states
          linux_inet_diag: reduce stack usage and simplify
          avoid reading errno repeatedly
          aggregate/pmq: avoid false sharing of lock buffers
          aggregate/pmq: remove io-extra requirement
          aggregate/pmq: avoid File#stat allocation
          Merge remote-tracking branch 'origin/freebsd'
          Merge remote-tracking branch 'origin/aggregate-pmq'
          doc: remove private email support address
          doc: update location of TCP_INFO-related stuff
          build: avoid olddoc for building the RubyGem
          doc: document Raindrops::TCP hash
          aggregate/pmq: update version numbers for Ruby and Linux

^ permalink raw reply	[relevance 4%]

* [PATCH] doc: remove private email support address
@ 2017-03-20 20:32  5% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2017-03-20 20:32 UTC (permalink / raw)
  To: unicorn-public

Email was never private, and won't further burden myself or
any future maintainers with trying to maintain someone elses'
privacy.

Offering private support is also unfair to readers on public
lists who may get a watered down or improperly translated
summary (if at all).

Instead, encourage the use of anonymity tools and scrubbing of
sensitive information when the sender deems necessary.
---
 No, I don't do GPG, as I have no real identity :P

 .olddoc.yml |  1 -
 ISSUES      | 11 ++++++-----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/.olddoc.yml b/.olddoc.yml
index ee2d306..cacc0ab 100644
--- a/.olddoc.yml
+++ b/.olddoc.yml
@@ -12,7 +12,6 @@ noindex:
 - TODO
 - unicorn_rails_1
 public_email: unicorn-public@bogomips.org
-private_email: unicorn@bogomips.org
 nntp_url:
   - nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
   - nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
diff --git a/ISSUES b/ISSUES
index 291441a..e06f9cd 100644
--- a/ISSUES
+++ b/ISSUES
@@ -9,14 +9,16 @@ submit patches and/or obtain support after you have searched the
 * Cc: all participants in a thread or commit, as subscription is optional
 * Do not {top post}[http://catb.org/jargon/html/T/top-post.html] in replies
 * Quote as little as possible of the message you're replying to
-* Do not send HTML mail or images, it will be flagged as spam
-* Anonymous and pseudonymous messages will always be welcome.
+* Do not send HTML mail or images,
+  they hurt reader privacy and will be flagged as spam
+* Anonymous and pseudonymous messages will ALWAYS be welcome
 * The email submission port (587) is enabled on the bogomips.org MX:
   https://bogomips.org/unicorn-public/20141004232241.GA23908@dcvr.yhbt.net/t/
 
 If your issue is of a sensitive nature or you're just shy in public,
-then feel free to email us privately at mailto:unicorn@bogomips.org
-instead and your issue will be handled discreetly.
+use anonymity tools such as Tor or Mixmaster; and rely on the public
+mail archives for responses.  Be sure to scrub sensitive log messages
+and such.
 
 If you don't get a response within a few days, we may have forgotten
 about it so feel free to ask again.
@@ -64,7 +66,6 @@ document distributed with git) on guidelines for patch submission.
 == Contact Info
 
 * public: mailto:unicorn-public@bogomips.org
-* private: mailto:unicorn@bogomips.org
 * nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
 * nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn
 * https://bogomips.org/unicorn-public/
-- 
EW


^ permalink raw reply related	[relevance 5%]

* [PATCH 2/3] revert signature change to HttpServer#process_client
  @ 2017-03-08  6:02  3% ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2017-03-08  6:02 UTC (permalink / raw)
  To: unicorn-public; +Cc: Simon Eskildsen, Aman Gupta

We can force kgio_tryaccept to return an internal class
for TCP objects by subclassing Kgio::TCPServer.

This avoids breakage in any unfortunate projects which depend on
our undocumented internal APIs, such as gctools
<https://github.com/tmm1/gctools>

Cc: Aman Gupta <aman@tmm1.net>
---
 lib/unicorn/http_request.rb  | 10 +++++-----
 lib/unicorn/http_server.rb   |  6 +++---
 lib/unicorn/oob_gc.rb        |  4 ++--
 lib/unicorn/socket_helper.rb | 16 ++++++++++++++--
 test/unit/test_request.rb    | 28 ++++++++++++++--------------
 5 files changed, 38 insertions(+), 26 deletions(-)

diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 9acde50..68bde16 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -61,7 +61,7 @@ def self.check_client_connection=(bool)
   # returns an environment hash suitable for Rack if successful
   # This does minimal exception trapping and it is up to the caller
   # to handle any socket errors (e.g. user aborted upload).
-  def read(socket, listener)
+  def read(socket)
     clear
     e = env
 
@@ -82,7 +82,7 @@ def read(socket, listener)
       false until add_parse(socket.kgio_read!(16384))
     end
 
-    check_client_connection(socket, listener) if @@check_client_connection
+    check_client_connection(socket) if @@check_client_connection
 
     e['rack.input'] = 0 == content_length ?
                       NULL_IO : @@input_class.new(socket, self)
@@ -105,8 +105,8 @@ def hijacked?
   end
 
   if defined?(Raindrops::TCP_Info)
-    def check_client_connection(socket, listener) # :nodoc:
-      if Kgio::TCPServer === listener
+    def check_client_connection(socket) # :nodoc:
+      if Unicorn::TCPClient === socket
         @@tcp_info ||= Raindrops::TCP_Info.new(socket)
         @@tcp_info.get!(socket)
         raise Errno::EPIPE, "client closed connection".freeze,
@@ -127,7 +127,7 @@ def closed_state?(state) # :nodoc:
       end
     end
   else
-    def check_client_connection(socket, listener) # :nodoc:
+    def check_client_connection(socket) # :nodoc:
       write_http_header(socket)
     end
   end
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 2aa1072..c2086cb 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -558,8 +558,8 @@ def e100_response_write(client, env)
 
   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
-  def process_client(client, listener)
-    status, headers, body = @app.call(env = @request.read(client, listener))
+  def process_client(client)
+    status, headers, body = @app.call(env = @request.read(client))
 
     begin
       return if @request.hijacked?
@@ -655,7 +655,7 @@ def worker_loop(worker)
         # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
         # but that will return false
         if client = sock.kgio_tryaccept
-          process_client(client, sock)
+          process_client(client)
           nr += 1
           worker.tick = time_now.to_i
         end
diff --git a/lib/unicorn/oob_gc.rb b/lib/unicorn/oob_gc.rb
index 74a1d51..5572e59 100644
--- a/lib/unicorn/oob_gc.rb
+++ b/lib/unicorn/oob_gc.rb
@@ -67,8 +67,8 @@ def self.new(app, interval = 5, path = %r{\A/})
 
   #:stopdoc:
   PATH_INFO = "PATH_INFO"
-  def process_client(client, listener)
-    super(client, listener) # Unicorn::HttpServer#process_client
+  def process_client(client)
+    super(client) # Unicorn::HttpServer#process_client
     if OOBGC_PATH =~ OOBGC_ENV[PATH_INFO] && ((@@nr -= 1) <= 0)
       @@nr = OOBGC_INTERVAL
       OOBGC_ENV.clear
diff --git a/lib/unicorn/socket_helper.rb b/lib/unicorn/socket_helper.rb
index df8315e..5371413 100644
--- a/lib/unicorn/socket_helper.rb
+++ b/lib/unicorn/socket_helper.rb
@@ -3,6 +3,18 @@
 require 'socket'
 
 module Unicorn
+
+  # Instead of using a generic Kgio::Socket for everything,
+  # tag TCP sockets so we can use TCP_INFO under Linux without
+  # incurring extra syscalls for Unix domain sockets.
+  # TODO: remove these when we remove kgio
+  TCPClient = Class.new(Kgio::Socket) # :nodoc:
+  class TCPSrv < Kgio::TCPServer # :nodoc:
+    def kgio_tryaccept # :nodoc:
+      super(TCPClient)
+    end
+  end
+
   module SocketHelper
 
     # internal interface
@@ -148,7 +160,7 @@ def new_tcp_server(addr, port, opt)
       end
       sock.bind(Socket.pack_sockaddr_in(port, addr))
       sock.autoclose = false
-      Kgio::TCPServer.for_fd(sock.fileno)
+      TCPSrv.for_fd(sock.fileno)
     end
 
     # returns rfc2732-style (e.g. "[::1]:666") addresses for IPv6
@@ -185,7 +197,7 @@ def sock_name(sock)
     def server_cast(sock)
       begin
         Socket.unpack_sockaddr_in(sock.getsockname)
-        Kgio::TCPServer.for_fd(sock.fileno)
+        TCPSrv.for_fd(sock.fileno)
       rescue ArgumentError
         Kgio::UNIXServer.for_fd(sock.fileno)
       end
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index dbe8af7..f0ccaf7 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -30,7 +30,7 @@ def setup
   def test_options
     client = MockRequest.new("OPTIONS * HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal '', env['REQUEST_PATH']
     assert_equal '', env['PATH_INFO']
     assert_equal '*', env['REQUEST_URI']
@@ -40,7 +40,7 @@ def test_options
   def test_absolute_uri_with_query
     client = MockRequest.new("GET http://e:3/x?y=z HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'y=z', env['QUERY_STRING']
@@ -50,7 +50,7 @@ def test_absolute_uri_with_query
   def test_absolute_uri_with_fragment
     client = MockRequest.new("GET http://e:3/x#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal '', env['QUERY_STRING']
@@ -61,7 +61,7 @@ def test_absolute_uri_with_fragment
   def test_absolute_uri_with_query_and_fragment
     client = MockRequest.new("GET http://e:3/x?a=b#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'a=b', env['QUERY_STRING']
@@ -73,7 +73,7 @@ def test_absolute_uri_unsupported_schemes
     %w(ssh+http://e/ ftp://e/x http+ssh://e/x).each do |abs_uri|
       client = MockRequest.new("GET #{abs_uri} HTTP/1.1\r\n" \
                                "Host: foo\r\n\r\n")
-      assert_raises(HttpParserError) { @request.read(client, nil) }
+      assert_raises(HttpParserError) { @request.read(client) }
     end
   end
 
@@ -81,7 +81,7 @@ def test_x_forwarded_proto_https
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: https\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal "https", env['rack.url_scheme']
     res = @lint.call(env)
   end
@@ -90,7 +90,7 @@ def test_x_forwarded_proto_http
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: http\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal "http", env['rack.url_scheme']
     res = @lint.call(env)
   end
@@ -99,14 +99,14 @@ def test_x_forwarded_proto_invalid
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: ftp\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal "http", env['rack.url_scheme']
     res = @lint.call(env)
   end
 
   def test_rack_lint_get
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal "http", env['rack.url_scheme']
     assert_equal '127.0.0.1', env['REMOTE_ADDR']
     res = @lint.call(env)
@@ -114,7 +114,7 @@ def test_rack_lint_get
 
   def test_no_content_stringio
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal StringIO, env['rack.input'].class
   end
 
@@ -122,7 +122,7 @@ def test_zero_content_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 0\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal StringIO, env['rack.input'].class
   end
 
@@ -130,7 +130,7 @@ def test_real_content_not_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert_equal Unicorn::TeeInput, env['rack.input'].class
   end
 
@@ -141,7 +141,7 @@ def test_rack_lint_put
       "Content-Length: 5\r\n" \
       "\r\n" \
       "abcde")
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert ! env.include?(:http_body)
     res = @lint.call(env)
   end
@@ -167,7 +167,7 @@ def client.kgio_read!(*args)
       "\r\n")
     count.times { assert_equal bs, client.syswrite(buf) }
     assert_equal 0, client.sysseek(0)
-    env = @request.read(client, nil)
+    env = @request.read(client)
     assert ! env.include?(:http_body)
     assert_equal length, env['rack.input'].size
     count.times {
-- 
EW


^ permalink raw reply related	[relevance 3%]

* Re: [PATCH] check_client_connection: use tcp state on linux
  @ 2017-03-06 21:32  3%           ` Simon Eskildsen
  0 siblings, 0 replies; 31+ results
From: Simon Eskildsen @ 2017-03-06 21:32 UTC (permalink / raw)
  To: Eric Wong; +Cc: unicorn-public

Here's another update Eric!

* Use a frozen empty array and a class variable for TCP_Info to avoid
garbage. As far as I can tell, this shouldn't result in any garbage on
any requests (other than on the first request).
* Pass listener socket to #read to only check the client connection on
a TCP server.
* Short circuit CLOSE_WAIT after ESTABLISHED since in my testing it's
the most common state after ESTABLISHED, it makes the numbers
un-ordered, though. But comment should make it OK.
* Definition of of `check_client_connection` based on whether
Raindrops::TCP_Info is defined, instead of the class variable
approach.
* Changed the unit tests to pass a `nil` listener.

Tested on our staging environment, and still works like a dream.

I should note that I got the idea between this patch into Puma as well!

https://github.com/puma/puma/pull/1227


---
 lib/unicorn/http_request.rb | 44 ++++++++++++++++++++++++++++++++++++++------
 lib/unicorn/http_server.rb  |  6 +++---
 test/unit/test_request.rb   | 28 ++++++++++++++--------------
 3 files changed, 55 insertions(+), 23 deletions(-)

diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 0c1f9bb..21a99ef 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -2,6 +2,7 @@
 # :enddoc:
 # no stable API here
 require 'unicorn_http'
+require 'raindrops'

 # TODO: remove redundant names
 Unicorn.const_set(:HttpRequest, Unicorn::HttpParser)
@@ -28,6 +29,7 @@ class Unicorn::HttpParser
   # Drop these frozen strings when Ruby 2.2 becomes more prevalent,
   # 2.2+ optimizes hash assignments when used with literal string keys
   HTTP_RESPONSE_START = [ 'HTTP', '/1.1 ']
+  EMPTY_ARRAY = [].freeze
   @@input_class = Unicorn::TeeInput
   @@check_client_connection = false

@@ -62,7 +64,7 @@ def self.check_client_connection=(bool)
   # returns an environment hash suitable for Rack if successful
   # This does minimal exception trapping and it is up to the caller
   # to handle any socket errors (e.g. user aborted upload).
-  def read(socket)
+  def read(socket, listener)
     clear
     e = env

@@ -83,11 +85,7 @@ def read(socket)
       false until add_parse(socket.kgio_read!(16384))
     end

-    # detect if the socket is valid by writing a partial response:
-    if @@check_client_connection && headers?
-      self.response_start_sent = true
-      HTTP_RESPONSE_START.each { |c| socket.write(c) }
-    end
+    check_client_connection(socket, listener) if @@check_client_connection

     e['rack.input'] = 0 == content_length ?
                       NULL_IO : @@input_class.new(socket, self)
@@ -108,4 +106,38 @@ def call
   def hijacked?
     env.include?('rack.hijack_io'.freeze)
   end
+
+  if defined?(Raindrops::TCP_Info)
+    def check_client_connection(socket, listener) # :nodoc:
+      if Kgio::TCPServer === listener
+        @@tcp_info ||= Raindrops::TCP_Info.new(socket)
+        @@tcp_info.get!(socket)
+        raise Errno::EPIPE, "client closed connection".freeze,
EMPTY_ARRAY if closed_state?(@@tcp_info.state)
+      else
+        write_http_header(socket)
+      end
+    end
+
+    def closed_state?(state) # :nodoc:
+      case state
+      when 1 # ESTABLISHED
+        false
+      when 8, 6, 7, 9, 11 # CLOSE_WAIT, TIME_WAIT, CLOSE, LAST_ACK, CLOSING
+        true
+      else
+        false
+      end
+    end
+  else
+    def check_client_connection(socket, listener) # :nodoc:
+      write_http_header(socket)
+    end
+  end
+
+  def write_http_header(socket) # :nodoc:
+    if headers?
+      self.response_start_sent = true
+      HTTP_RESPONSE_START.each { |c| socket.write(c) }
+    end
+  end
 end
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 35bd100..4190641 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -558,8 +558,8 @@ def e100_response_write(client, env)

   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
-  def process_client(client)
-    status, headers, body = @app.call(env = @request.read(client))
+  def process_client(client, listener)
+    status, headers, body = @app.call(env = @request.read(client, listener))

     begin
       return if @request.hijacked?
@@ -655,7 +655,7 @@ def worker_loop(worker)
         # Unicorn::Worker#kgio_tryaccept is not like accept(2) at all,
         # but that will return false
         if client = sock.kgio_tryaccept
-          process_client(client)
+          process_client(client, sock)
           nr += 1
           worker.tick = time_now.to_i
         end
diff --git a/test/unit/test_request.rb b/test/unit/test_request.rb
index f0ccaf7..dbe8af7 100644
--- a/test/unit/test_request.rb
+++ b/test/unit/test_request.rb
@@ -30,7 +30,7 @@ def setup
   def test_options
     client = MockRequest.new("OPTIONS * HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal '', env['REQUEST_PATH']
     assert_equal '', env['PATH_INFO']
     assert_equal '*', env['REQUEST_URI']
@@ -40,7 +40,7 @@ def test_options
   def test_absolute_uri_with_query
     client = MockRequest.new("GET http://e:3/x?y=z HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'y=z', env['QUERY_STRING']
@@ -50,7 +50,7 @@ def test_absolute_uri_with_query
   def test_absolute_uri_with_fragment
     client = MockRequest.new("GET http://e:3/x#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal '', env['QUERY_STRING']
@@ -61,7 +61,7 @@ def test_absolute_uri_with_fragment
   def test_absolute_uri_with_query_and_fragment
     client = MockRequest.new("GET http://e:3/x?a=b#frag HTTP/1.1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal '/x', env['REQUEST_PATH']
     assert_equal '/x', env['PATH_INFO']
     assert_equal 'a=b', env['QUERY_STRING']
@@ -73,7 +73,7 @@ def test_absolute_uri_unsupported_schemes
     %w(ssh+http://e/ ftp://e/x http+ssh://e/x).each do |abs_uri|
       client = MockRequest.new("GET #{abs_uri} HTTP/1.1\r\n" \
                                "Host: foo\r\n\r\n")
-      assert_raises(HttpParserError) { @request.read(client) }
+      assert_raises(HttpParserError) { @request.read(client, nil) }
     end
   end

@@ -81,7 +81,7 @@ def test_x_forwarded_proto_https
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: https\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal "https", env['rack.url_scheme']
     res = @lint.call(env)
   end
@@ -90,7 +90,7 @@ def test_x_forwarded_proto_http
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: http\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal "http", env['rack.url_scheme']
     res = @lint.call(env)
   end
@@ -99,14 +99,14 @@ def test_x_forwarded_proto_invalid
     client = MockRequest.new("GET / HTTP/1.1\r\n" \
                              "X-Forwarded-Proto: ftp\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal "http", env['rack.url_scheme']
     res = @lint.call(env)
   end

   def test_rack_lint_get
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal "http", env['rack.url_scheme']
     assert_equal '127.0.0.1', env['REMOTE_ADDR']
     res = @lint.call(env)
@@ -114,7 +114,7 @@ def test_rack_lint_get

   def test_no_content_stringio
     client = MockRequest.new("GET / HTTP/1.1\r\nHost: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal StringIO, env['rack.input'].class
   end

@@ -122,7 +122,7 @@ def test_zero_content_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 0\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal StringIO, env['rack.input'].class
   end

@@ -130,7 +130,7 @@ def test_real_content_not_stringio
     client = MockRequest.new("PUT / HTTP/1.1\r\n" \
                              "Content-Length: 1\r\n" \
                              "Host: foo\r\n\r\n")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert_equal Unicorn::TeeInput, env['rack.input'].class
   end

@@ -141,7 +141,7 @@ def test_rack_lint_put
       "Content-Length: 5\r\n" \
       "\r\n" \
       "abcde")
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert ! env.include?(:http_body)
     res = @lint.call(env)
   end
@@ -167,7 +167,7 @@ def client.kgio_read!(*args)
       "\r\n")
     count.times { assert_equal bs, client.syswrite(buf) }
     assert_equal 0, client.sysseek(0)
-    env = @request.read(client)
+    env = @request.read(client, nil)
     assert ! env.include?(:http_body)
     assert_equal length, env['rack.input'].size
     count.times {
-- 
2.11.0

On Tue, Feb 28, 2017 at 10:18 PM, Eric Wong <e@80x24.org> wrote:
>> Simon Eskildsen <simon.eskildsen@shopify.com> wrote:
>> > +      tcp_info = Raindrops::TCP_Info.new(socket)
>> > +      raise Errno::EPIPE, "client closed connection".freeze, [] if
>> > closed_state?(tcp_info.state)
>
> Also, I guess if you're using this frequently, it might make
> sense to keep the tcp_info object around and recycle it
> using the Raindrops::TCP_Info#get! method.
>
> #get! has always been supported in raindrops, but I just noticed
> RDoc wasn't picking it up properly :x
>
> Anyways I've fixed the documentation over on the raindrops list
>
>   https://bogomips.org/raindrops-public/20170301025541.26183-1-e@80x24.org/
>   ("[PATCH] ext: fix documentation for C ext-defined classes")
>
>   https://bogomips.org/raindrops-public/20170301025546.26233-1-e@80x24.org/
>   ("[PATCH] TCP_Info: custom documentation for #get!")
>
> ... and updated the RDoc on https://bogomips.org/raindrops/
>
> Heck, I wonder if it even makes sense to reuse a frozen empty
> Array when raising the exception...

^ permalink raw reply related	[relevance 3%]

* Re: [PATCH] check_client_connection: use tcp state on linux
  2017-02-27 11:44  3%     ` Simon Eskildsen
@ 2017-02-28 21:12  0%       ` Eric Wong
    0 siblings, 1 reply; 31+ results
From: Eric Wong @ 2017-02-28 21:12 UTC (permalink / raw)
  To: Simon Eskildsen; +Cc: unicorn-public

Simon Eskildsen <simon.eskildsen@shopify.com> wrote:

<snip>
> I would assume you would see TIME_WAIT and CLOSE. LAST_ACK_CLOSING it
> seems pretty unlikely to hit, but not impossible. As with CLOSING,
> I've included LAST_ACK_CLOSING for completeness.

Did you mean "LAST_ACK, and CLOSING"? (not joined by underscore)

Anyways, thanks for testing and adding

> <e@80x24.org> wrote:
> > Yep, we need to account for the UNIX socket case.  I forget if
> > kgio even makes them different...
> 
> I read the implementation and verified by dumping the class when
> testing on some test boxes. You are right—it's a simple Kgio::Socket
> object, not differentiating between Kgio::TCPSocket and
> Kgio::UnixSocket at the class level. Kgio only does this if they're
> explicitly passed to override the class returned from #try_accept.
> Unicorn doesn't do this.
> 
> I've tried to find a way to determine the socket domain (INET vs.
> UNIX) on the socket object, but neither Ruby's Socket class nor Kgio
> seems to expose this. I'm not entirely sure what the simplest way to
> do this check would be. We could have the accept loop pass the correct
> class to #try_accept based on the listening socket that came back from
> #accept. If we passed the listening socket to #read after accept, we'd
> know.. but I don't like that the request knows about the listener
> either. Alternatively, we could expose the socket domain in Kgio, but
> that'll be problematic in the near-ish future as you've mentioned
> wanting to move away from Kgio as Ruby's IO library is at parity as
> per Ruby 2.4.
> 
> What do you suggest pursuing here to check whether the client socket
> is a TCP socket?

I think passing the listening socket is the best way to go about
detecting whether a socket is INET or UNIX, for now.

You're right about kgio, I'd rather not make more changes to
kgio but we will still need it to for Ruby <2.2.x users.

And #read is an overloaded name, feel free to change it :)

> Below is a patch addressing the other concerns. I had to include
> require raindrops so the `defined?` check would do the right thing, as
> the only other file that requires Raindrops is the worker one which is
> loaded after http_request. I can change the load-order or require
> raindrops in lib/unicorn.rb if you prefer.

The require is fine.  However we do not need a class variable,
below...

>  # TODO: remove redundant names
>  Unicorn.const_set(:HttpRequest, Unicorn::HttpParser)
> @@ -29,6 +30,7 @@ class Unicorn::HttpParser
>    # 2.2+ optimizes hash assignments when used with literal string keys
>    HTTP_RESPONSE_START = [ 'HTTP', '/1.1 ']
>    @@input_class = Unicorn::TeeInput
> +  @@raindrops_tcp_info_defined = defined?(Raindrops::TCP_Info)

I prefer we avoid adding this cvar, instead...

>    @@check_client_connection = false
> 
>    def self.input_class
> @@ -83,11 +85,7 @@ def read(socket)
>        false until add_parse(socket.kgio_read!(16384))
>      end
> 
> -    # detect if the socket is valid by writing a partial response:
> -    if @@check_client_connection && headers?
> -      self.response_start_sent = true
> -      HTTP_RESPONSE_START.each { |c| socket.write(c) }
> -    end
> +    check_client_connection(socket) if @@check_client_connection
> 
>      e['rack.input'] = 0 == content_length ?
>                        NULL_IO : @@input_class.new(socket, self)
> @@ -108,4 +106,27 @@ def call
>    def hijacked?
>      env.include?('rack.hijack_io'.freeze)
>    end

... we can have different methods defined:

   if defined?(Raindrops::TCP_Info) # Linux, maybe FreeBSD
     def check_client_connection(client, listener) # :nodoc:
     ...
     end
   else # portable version
     def check_client_connection(client, listener) # :nodoc:
     ...
     end
   end

And eliminate the class variable entirely.

> +
> +  private

I prefer to avoid marking methods as 'private' to ease any
ad-hoc unit testing which may come up.  Instead, rely on :nodoc:
directives to discourage people from depending on it.

Thanks.

> +  def check_client_connection(socket)
> +    if @@raindrops_tcp_info_defined
> +      tcp_info = Raindrops::TCP_Info.new(socket)
> +      raise Errno::EPIPE, "client closed connection".freeze, [] if
> closed_state?(tcp_info.state)
> +    elsif headers?
> +      self.response_start_sent = true
> +      HTTP_RESPONSE_START.each { |c| socket.write(c) }
> +    end
> +  end
> +
> +  def closed_state?(state)
> +    case state
> +    when 1 # ESTABLISHED
> +      false
> +    when 6, 7, 8, 9, 11 # TIME_WAIT, CLOSE, CLOSE_WAIT, LAST_ACK, CLOSING
> +      true
> +    else
> +      false
> +    end
> +  end
>  end

closed_state? looks good to me, good call on short-circuiting
the common case of ESTABLISHED!

^ permalink raw reply	[relevance 0%]

* Re: [PATCH] check_client_connection: use tcp state on linux
  @ 2017-02-27 11:44  3%     ` Simon Eskildsen
  2017-02-28 21:12  0%       ` Eric Wong
  0 siblings, 1 reply; 31+ results
From: Simon Eskildsen @ 2017-02-27 11:44 UTC (permalink / raw)
  To: Eric Wong; +Cc: unicorn-public

> I prefer we use a hash or case statement.  Both allow more
> optimization in the YARV VM of CRuby (opt_aref and
> opt_case_dispatch in insns.def).  case _might_ be a little
> faster if there's no constant lookup overhead, but
> a microbench or dumping the bytecode will be necessary
> to be sure :)
>
> A hash or a case can also help portability-wise in case
> we hit a system where these numbers are non-sequential;
> or if we forgot something.

Good point. I double checked all the states on Linux and found that we
were missing TCP_CLOSING [1] [2]. This is a state where the other side
is closed, and you have buffered data on your side. It doesn't seem
like this would ever happen in Unicorn, but I think we should include
it for completeness. This also means the range becomes non-sequential.
I looked at Illumus (solaris-derived) [3] and BSD [4] and for the TCP
states we're interested in it also appears to have a non-continues
range.

My co-worker, Kir Shatrov, benchmarked a bunch of approaches to the
state check and found that case is a good solution [5].  Due to the
realness of non-sequential states in common operating systems, I think
case is the way to go here as you suggested. I've made sure to
short-circuit the common-case of TCP_ESTABLISHED. I've only seen
CLOSE_WAIT in testing, but in the wild-life of large production scale
I would assume you would see TIME_WAIT and CLOSE. LAST_ACK_CLOSING it
seems pretty unlikely to hit, but not impossible. As with CLOSING,
I've included LAST_ACK_CLOSING for completeness.

[1] https://github.com/torvalds/linux/blob/5924bbecd0267d87c24110cbe2041b5075173a25/include/net/tcp_states.h#L27
[2] https://github.com/torvalds/linux/blob/ca78d3173cff3503bcd15723b049757f75762d15/net/ipv4/tcp.c#L228
[3] https://github.com/freebsd/freebsd/blob/386ddae58459341ec567604707805814a2128a57/sys/netinet/tcp_fsm.h
[4] https://github.com/illumos/illumos-gate/blob/f7877f5d39900cfd8b20dd673e5ccc1ef7cc7447/usr/src/uts/common/netinet/tcp_fsm.h
[5] https://gist.github.com/kirs/11ba4ce84c08188c9f7eba9c639616a5

> Yep, we need to account for the UNIX socket case.  I forget if
> kgio even makes them different...

I read the implementation and verified by dumping the class when
testing on some test boxes. You are right—it's a simple Kgio::Socket
object, not differentiating between Kgio::TCPSocket and
Kgio::UnixSocket at the class level. Kgio only does this if they're
explicitly passed to override the class returned from #try_accept.
Unicorn doesn't do this.

I've tried to find a way to determine the socket domain (INET vs.
UNIX) on the socket object, but neither Ruby's Socket class nor Kgio
seems to expose this. I'm not entirely sure what the simplest way to
do this check would be. We could have the accept loop pass the correct
class to #try_accept based on the listening socket that came back from
#accept. If we passed the listening socket to #read after accept, we'd
know.. but I don't like that the request knows about the listener
either. Alternatively, we could expose the socket domain in Kgio, but
that'll be problematic in the near-ish future as you've mentioned
wanting to move away from Kgio as Ruby's IO library is at parity as
per Ruby 2.4.

What do you suggest pursuing here to check whether the client socket
is a TCP socket?

Below is a patch addressing the other concerns. I had to include
require raindrops so the `defined?` check would do the right thing, as
the only other file that requires Raindrops is the worker one which is
loaded after http_request. I can change the load-order or require
raindrops in lib/unicorn.rb if you prefer.

Missing is the socket type check. Thanks for your feedback!

---
 lib/unicorn/http_request.rb | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index 0c1f9bb..eedccac 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -2,6 +2,7 @@
 # :enddoc:
 # no stable API here
 require 'unicorn_http'
+require 'raindrops'

 # TODO: remove redundant names
 Unicorn.const_set(:HttpRequest, Unicorn::HttpParser)
@@ -29,6 +30,7 @@ class Unicorn::HttpParser
   # 2.2+ optimizes hash assignments when used with literal string keys
   HTTP_RESPONSE_START = [ 'HTTP', '/1.1 ']
   @@input_class = Unicorn::TeeInput
+  @@raindrops_tcp_info_defined = defined?(Raindrops::TCP_Info)
   @@check_client_connection = false

   def self.input_class
@@ -83,11 +85,7 @@ def read(socket)
       false until add_parse(socket.kgio_read!(16384))
     end

-    # detect if the socket is valid by writing a partial response:
-    if @@check_client_connection && headers?
-      self.response_start_sent = true
-      HTTP_RESPONSE_START.each { |c| socket.write(c) }
-    end
+    check_client_connection(socket) if @@check_client_connection

     e['rack.input'] = 0 == content_length ?
                       NULL_IO : @@input_class.new(socket, self)
@@ -108,4 +106,27 @@ def call
   def hijacked?
     env.include?('rack.hijack_io'.freeze)
   end
+
+  private
+
+  def check_client_connection(socket)
+    if @@raindrops_tcp_info_defined
+      tcp_info = Raindrops::TCP_Info.new(socket)
+      raise Errno::EPIPE, "client closed connection".freeze, [] if
closed_state?(tcp_info.state)
+    elsif headers?
+      self.response_start_sent = true
+      HTTP_RESPONSE_START.each { |c| socket.write(c) }
+    end
+  end
+
+  def closed_state?(state)
+    case state
+    when 1 # ESTABLISHED
+      false
+    when 6, 7, 8, 9, 11 # TIME_WAIT, CLOSE, CLOSE_WAIT, LAST_ACK, CLOSING
+      true
+    else
+      false
+    end
+  end
 end
-- 
2.11.0

^ permalink raw reply related	[relevance 3%]

* [ANN] unicorn 5.2.0 - Rack HTTP server for fast clients and *nix
       [not found]     <20161031-unicorn-5.2.0-released@bogomips.org>
@ 2016-10-31 20:04  5% ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2016-10-31 20:04 UTC (permalink / raw)
  To: ruby-talk, unicorn-public; +Cc: Mishael A Sibiryakov

unicorn is an HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels.  Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between unicorn and slow clients.

* https://bogomips.org/unicorn/
* public list: unicorn-public@bogomips.org
* mail archives: https://bogomips.org/unicorn-public/
* git clone git://bogomips.org/unicorn.git
* https://bogomips.org/unicorn/NEWS.atom.xml
* nntp://news.public-inbox.org/inbox.comp.lang.ruby.unicorn

Changes:

    Most notably, this release allows us to support requests with
    lines delimited by LF-only, as opposed to the standard CRLF
    pair and allowed by RFC 2616 sec 19.3.

    Thanks to Mishael A Sibiryakov for the explanation and change:

      https://bogomips.org/unicorn-public/1476954332.1736.156.camel@junki.org/

    Thanks to Let's Encrypt, the website also moves to HTTPS
    <https://bogomips.org/unicorn/> to improve reader privacy.  The
    "unicorn.bogomips.org" subdomain will be retired soon to reduce
    subjectAltName bloat and speed up certificate renewals.

    There's also the usual round of documentation and example
    updates, too.

    Eric Wong (7):
          examples/init.sh: update to reduce upgrade raciness
          doc: systemd should only kill master in example
          examples/logrotate.conf: update example for systemd
          doc: update gmane URLs to point to our own archives
          relocate website to https://bogomips.org/unicorn/
          TODO: remove Rack 2.x item
          build: "install-gem" target avoids network

    Mishael A Sibiryakov (1):
          Add some tolerance (RFC2616 sec. 19.3)

^ permalink raw reply	[relevance 5%]

* Re: [DRE-maint] unicorn: native systemd service
  @ 2015-07-08 13:08  4%             ` Christos Trochalakis
  0 siblings, 0 replies; 31+ results
From: Christos Trochalakis @ 2015-07-08 13:08 UTC (permalink / raw)
  To: Eric Wong; +Cc: Dmitry Smirnov, Hleb Valoshka, unicorn, unicorn-public

>Christos Trochalakis <yatiohi@ideopolis.gr> wrote:
>> On Sat, Jun 27, 2015 at 03:05:01AM +0000, Eric Wong wrote:
>> >Is it possible to make systemd fire up two unicorn masters?
>> >That would be a nice feature to have with socket activation.
>>
>> That would be great! I am a bit surprised that specifying multiple
>> services ('Service=' option) is not supported.

Apparently I was wrong. I missed the 'Sockets=' option for Service units.

You can specify a list of socket units to be inherited (if active) when
the service is started. Note that this is not the reverse of Sockets'
'Service=' option: the service is not auto-started when there is an
incoming connnection on the relevant socket, you have to use 'Service='
for socket activation.

Here is a test config for 2 preforked unicorn masters using service
templates (I am using the latest unicorn with the systemd patch):

==> /srv/uni/config.ru <==
puts "initializing for 5sec"
sleep 5

app = proc do |env|
  [
    200,
    { 'Content-Type' => 'text/plain' },
    ["ppid:#{Process.ppid}\n"]
  ]
end

run app

==> /srv/uni/unicorn.conf.rb <==
worker_processes 2
working_directory "/srv/uni"

# Keep in sync with uni.socket file
listen 9000
listen 9001

==> /etc/systemd/system/uni@.service <==
[Unit]
Description=Unicorn Server %i
Wants=uni.socket
After=uni.socket

[Service]
ExecStart=/usr/local/bin/unicorn -c /srv/uni/unicorn.conf.rb -d
Sockets=uni.socket
KillSignal=SIGQUIT

[Install]
WantedBy=multi-user.target

==> /etc/systemd/system/uni.socket <==
[Unit]
Description=Unicorn Sockets

[Socket]
ListenStream=0.0.0.0:9000
ListenStream=0.0.0.0:9001
Service=uni@1.service

[Install]
WantedBy=sockets.target

==> end <==

systemctl daemon-reload
systemctl enable uni.socket
systemctl start uni.socket

systemctl enable uni@1 uni@2 # Make them start at boot
systemctl start uni@1 uni@2

Now you can start and stop uni@1 and uni@2 with zero latency and without losing
a connection. So we have a happy ending after all :)

While searching systemd's mailing list, I found a thread discussing about a
'Distribute=' option to sockets a few years back[0]. There is also a relevant
item in systemd's TODO list.

[0] http://thread.gmane.org/gmane.comp.sysutils.systemd.devel/14242/focus=14255

^ permalink raw reply	[relevance 4%]

* Re: Having issue with Unicorn
  @ 2014-10-24 18:02  6%   ` Imdad
  0 siblings, 0 replies; 31+ results
From: Imdad @ 2014-10-24 18:02 UTC (permalink / raw)
  To: Eric Wong; +Cc: unicorn-public

Thanks Eric, here is my deploy.rb and config/unicorn.rb
NOTE: /releases/6 and /releases/28 both have same error message

config/unicorn.rb
==============
# Set your full path to application.
app_dir = File.expand_path('../../', __FILE__)
shared_dir = File.expand_path('../../../shared/', __FILE__)

# Set unicorn options
worker_processes 2
preload_app true
timeout 30

# Fill path to your app
working_directory app_dir

# Set up socket location
listen "#{shared_dir}/sockets/unicorn.sock", :backlog => 64

# Loging
stderr_path "#{shared_dir}/log/unicorn.stderr.log"
stdout_path "#{shared_dir}/log/unicorn.stdout.log"

# Set master PID location
pid "#{shared_dir}/pids/unicorn.pid"

before_fork do |server, worker|
  defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!
  old_pid = "#{server.config[:pid]}.oldbin"
  if File.exists?(old_pid) && server.pid != old_pid
    begin
      sig = (worker.nr + 1) >= server.worker_processes ? :QUIT : :TTOU
      Process.kill(sig, File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
      # someone else did our job for us
    end
  end
end

after_fork do |server, worker|
  defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
end

before_exec do |server|
  ENV['BUNDLE_GEMFILE'] = "#{app_dir}/Gemfile"
end

deploy.rb
==========
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rbenv'  # for rbenv support. (http://rbenv.org)
# require 'mina/rvm'    # for rvm support. (http://rvm.io)
# TODO: Look into this later
#require 'mina_sidekiq/tasks'
require 'mina/unicorn'


# Basic settings:
#   domain       - The hostname to SSH to.
#   deploy_to    - Path to deploy into.
#   repository   - Git repo to clone from. (needed by mina/git)
#   branch       - Branch name to deploy. (needed by mina/git)
set_default :rbenv_path, "/root/.rbenv" #{}"/root/.rbenv"
#set_default :bundle_path, '/root/.rbenv/shims/bundle'
#set_default :bundle_bin, 'bundle exec'

set :domain, '104.131.74.69'

set :deploy_to, '/var/www/hailisys'

set :repository, 'https://gitlab.com/hailisys/hailisys.git'
set :branch, 'master'
set :user, 'root'
set :forward_agent, true
# MOIN: Fix Password issue
set :term_mode, nil
# MOIN: Could be staging, production
set :rails_env, 'production'
set :port, '22'
set :unicorn_pid, "#{deploy_to}/shared/pids/unicorn.pid"

# For system-wide RVM install.
#   set :rvm_path, '/usr/local/rvm/bin/rvm'

# Manually create these paths in shared/ (eg: shared/config/database.yml)
in your server.
# They will be linked in the 'deploy:link_shared_paths' step.
set :shared_paths, ['config/database.yml', 'log']

# Optional settings:
#   set :user, 'foobar'    # Username in the server to SSH to.
#   set :port, '30000'     # SSH port number.
#   set :forward_agent, true     # SSH forward_agent.

# This task is the environment that is loaded for most commands, such as
# `mina deploy` or `mina rake`.
task :environment do
  # If you're using rbenv, use this to load the rbenv environment.
  # Be sure to commit your .rbenv-version to your repository.
  invoke :'rbenv:load'

  # For those using RVM, use this to load an RVM version@gemset.
  # invoke :'rvm:use[ruby-1.9.3-p125@default]'
end

# Put any custom mkdir's in here for when `mina setup` is ran.
# For Rails apps, we'll make some of the shared paths that are shared
between
# all releases.

task :setup => :environment do
  queue! %[mkdir -p "#{deploy_to}/shared/log"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/log"]

  queue! %[mkdir -p "#{deploy_to}/shared/config"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/config"]

  queue! %[mkdir -p "#{deploy_to}/shared/pids"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/pids"]

  queue! %[mkdir -p "#{deploy_to}/shared/sockets"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/sockets"]

  queue! %[touch "#{deploy_to}/shared/config/database.yml"]
  queue  %[echo "-----> Be sure to edit 'shared/config/database.yml'."]

  # sidekiq needs a place to store its pid file and log file
  queue! %[mkdir -p "#{deploy_to}/shared/pids/"]
  queue! %[chmod g+rx,u+rwx "#{deploy_to}/shared/pids"]
end

desc "Deploys the current version to the server."
task :deploy => :environment do
  deploy do

    # stop accepting new workers
    # TODO: Look into this later
    # invoke :'sidekiq:quiet'

    # Put things that will set up an empty directory into a fully set-up
    # instance of your project.
    invoke :'git:clone'
    invoke :'deploy:link_shared_paths'
    invoke :'bundle:install'
    invoke :'rails:db_migrate'
    invoke :'rails:assets_precompile'
    invoke :'deploy:cleanup'

    to :launch do
      # Passenger
      # queue "mkdir -p #{deploy_to}/#{current_path}/tmp/"
      # queue "touch #{deploy_to}/#{current_path}/tmp/restart.txt"

      # TODO: Look into this later
      # invoke :'sidekiq:restart'
      invoke :'unicorn:restart'
      #invoke :'unicorn:start'
      #queue "touch #{deploy_to}/tmp/restart.txt"

    end
  end
end

# For help in making your deploy script, see the Mina documentation:
#
#  - http://nadarei.co/mina
#  - http://nadarei.co/mina/tasks
#  - http://nadarei.co/mina/settings
#  - http://nadarei.co/mina/helpers

Please suggest what could be wrong.


Cheers!
Imdad Ali Khan
Mob (0) 9818484057
http://www.linkedin.com/in/imdad

On 24 October 2014 23:15, Eric Wong <e@80x24.org> wrote:

> Imdad <khanimdad@gmail.com> wrote:
> > I have rails4, nginx, unicorn and mona for app deployed
> > http://104.131.74.69/
> >
> > But after deploy hitting on above link results in error, and the log says
> >
> > root@Hailisys:~# tail -n 0 -f
> > /var/www/hailisys/shared/log/unicorn.stderr.log
> > I, [2014-10-24T17:28:53.162084 #15819]  INFO -- : executing
> > ["/var/www/hailisys/releases/6/vendor/bundle/ruby/2.1.0/bin/unicorn",
> "-c",
> > "/var/www/hailisys/current/config/unicorn.rb", "-D", "-E", "production",
> > {11=>#<Kgio::UNIXServer:fd 11>}] (in /var/www/hailisys/releases/28)
>
> You're working inside /releases/28 here...
>
> >
> /var/www/hailisys/current/vendor/bundle/ruby/2.1.0/gems/unicorn-4.8.3/lib/unicorn/http_server.rb:475:in
> > `exec': No such file or directory -
> > /var/www/hailisys/releases/6/vendor/bundle/ruby/2.1.0/bin/unicorn
> > (Errno::ENOENT)
>
> However, it's trying to run out of /releases/6, which I assume is
> old enough to be removed by now.
>
> You should be able to set working_directory in your config/unicorn.rb:
>
>         working_directory "/var/www/hailisys/current"
>
> Then SIGHUP to reload the config before sending SIGUSR2 to it.
>
> Hopefully that works.  Normally you shouldn't need to set
> working_directory if you're working out of "/current";
> but I'm not sure what your deploy environment looks like.
>


^ permalink raw reply	[relevance 6%]

* Re: Combating nginx 499 HTTP responses during flash traffic scenario
  2012-11-03 22:45  0%             ` Tom Burns
@ 2012-11-05 11:48  0%               ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2012-11-05 11:48 UTC (permalink / raw)
  To: unicorn list

Tom Burns <tom.burns@jadedpixel.com> wrote:
> On Fri, Nov 2, 2012 at 3:38 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Everything looks good for stock configurations.  The ERROR_XXX_RESPONSE
> > constants should have "HTTP/1.1 " removed if used with this option, too.
> > That requires keeping track of whether or not we've written out
> > "HTTP/1.1 ", yet.
> 
> The patch below tracks if we've sent the start of the response.  It
> also passes that to http_response_write as a parameter with default
> value false.  By doing this, the test suite passes if you change the
> default value for check_client_connection to true, which makes me feel
> a lot better about the patch.

Cool.

> > > The only thing it's missing from your TODO is enforcing
> > > tcp_nodelay/tcp_nopush as I wasn't sure where was the best place to do
> > > the enforcement.
> >
> > It should probably be done inside Unicorn::Configurator#commit!
> 
> So, I still didn't so this, because in commit! we don't know if it's a
> TCP or UNIX socket, and I don't think it'd be nice to force tcp
> configs if the user's using a UNIX socket.  If you disagree I can add
> the check there, it just seemed weird when I looked at it.

Erm, just check if :tcp_* is set in the :listener_opts hash and raise if
there's incompatible settings.  We shouldn't be changing settings behind
users' backs.

> >From d7e249202b0fd5c24f2b98c700e02bf992c6576b Mon Sep 17 00:00:00 2001
> From: Tom Burns <tom.burns@jadedpixel.com>
> Date: Tue, 30 Oct 2012 16:22:21 -0400
> Subject: [PATCH] Begin writing HTTP request headers early to detect
>  disconnected clients

This definitely needs a thorough commit message which explains why this
is a necessary addition, how it works, when it works, and what benefits
it brings.

We can wait until you can report on how this change improves your
situation before fleshing this out.  Real-world results are always
good to have :>

> --- a/lib/unicorn/http_request.rb
> +++ b/lib/unicorn/http_request.rb
> @@ -70,6 +82,15 @@ class Unicorn::HttpParser
>        # an Exception thrown from the parser will throw us out of the loop
>        false until add_parse(socket.kgio_read!(16384))
>      end
> +
> +    # detect if the socket is valid by writing a partial response:
> +    if @@check_client_connection && headers?
> +      @response_start_sent = true
> +      Unicorn::Const::HTTP_RESPONSE_START.each { |c| socket.write(c) }
> +    else
> +      @response_start_sent = false

This else portion is redundant given it's set to false when you reset
the parser (see my final comment below)

> --- a/lib/unicorn/http_server.rb
> +++ b/lib/unicorn/http_server.rb
> @@ -524,23 +533,33 @@ class Unicorn::HttpServer
>        Unicorn.log_error(@logger, "app error", e)
>        Unicorn::Const::ERROR_500_RESPONSE
>      end
> +    msg = 'HTTP/1.1 ' + msg unless @request.response_start_sent

String interpolation would be easier to read, here (and slightly
smaller bytecode if it matters :>).

>      client.kgio_trywrite(msg)
>      client.close
>      rescue
>    end
> 
> +  def expect_100_response
> +    if @request.response_start_sent
> +      Unicorn::Const::EXPECT_100_RESPONSE_SUFFIXED
> +    else
> +      Unicorn::Const::EXPECT_100_RESPONSE
> +    end
> +  end
> +
>    # once a client is accepted, it is processed in its entirety here
>    # in 3 easy steps: read request, call app, write app response
>    def process_client(client)
> +    @request.response_start_sent = false

Better to reset this ivar in the HttpParser#reset method
(it's in the C extension, I can change it later).

Random thought:  HttpResponse generation gains even more coupling
with the current Http{Request,Parser} state.  Organizing code is hard :x
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[relevance 0%]

* Re: Combating nginx 499 HTTP responses during flash traffic scenario
  2012-11-02 19:38  0%           ` Eric Wong
@ 2012-11-03 22:45  0%             ` Tom Burns
  2012-11-05 11:48  0%               ` Eric Wong
  0 siblings, 1 reply; 31+ results
From: Tom Burns @ 2012-11-03 22:45 UTC (permalink / raw)
  To: unicorn list

On Fri, Nov 2, 2012 at 3:38 PM, Eric Wong <normalperson@yhbt.net> wrote:
> Everything looks good for stock configurations.  The ERROR_XXX_RESPONSE
> constants should have "HTTP/1.1 " removed if used with this option, too.
> That requires keeping track of whether or not we've written out
> "HTTP/1.1 ", yet.

The patch below tracks if we've sent the start of the response.  It
also passes that to http_response_write as a parameter with default
value false.  By doing this, the test suite passes if you change the
default value for check_client_connection to true, which makes me feel
a lot better about the patch.

> > The only thing it's missing from your TODO is enforcing
> > tcp_nodelay/tcp_nopush as I wasn't sure where was the best place to do
> > the enforcement.
>
> It should probably be done inside Unicorn::Configurator#commit!

So, I still didn't so this, because in commit! we don't know if it's a
TCP or UNIX socket, and I don't think it'd be nice to force tcp
configs if the user's using a UNIX socket.  If you disagree I can add
the check there, it just seemed weird when I looked at it.

>
> You'll probably find it more reliable and easier to use "git send-email".

Sadly with 2-factor authentication I think this is a no go.

Let me know what you think!

Cheers,
Tom

>From d7e249202b0fd5c24f2b98c700e02bf992c6576b Mon Sep 17 00:00:00 2001
From: Tom Burns <tom.burns@jadedpixel.com>
Date: Tue, 30 Oct 2012 16:22:21 -0400
Subject: [PATCH] Begin writing HTTP request headers early to detect
 disconnected clients

---
 examples/unicorn.conf.rb       |    6 ++++++
 lib/unicorn/configurator.rb    |    7 +++++++
 lib/unicorn/const.rb           |   12 ++++++++----
 lib/unicorn/http_request.rb    |   21 +++++++++++++++++++++
 lib/unicorn/http_response.rb   |    5 +++--
 lib/unicorn/http_server.rb     |   23 +++++++++++++++++++++--
 test/exec/test_exec.rb         |    7 ++++++-
 test/unit/test_configurator.rb |   12 ++++++++++++
 8 files changed, 84 insertions(+), 9 deletions(-)

diff --git a/examples/unicorn.conf.rb b/examples/unicorn.conf.rb
index 0238043..1f4c9c0 100644
--- a/examples/unicorn.conf.rb
+++ b/examples/unicorn.conf.rb
@@ -46,6 +46,12 @@ preload_app true
 GC.respond_to?(:copy_on_write_friendly=) and
   GC.copy_on_write_friendly = true

+# Enable this flag to have unicorn test client connections by writing the
+# beginning of the HTTP headers before calling the application.  This
+# prevents calling the application for connections that have disconnected
+# while queued.
+check_client_connection false
+
 before_fork do |server, worker|
   # the following is highly recomended for Rails + "preload_app true"
   # as there's no need for the master process to hold a connection
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index 89cbf5c..ca84a88 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -45,6 +45,7 @@ class Unicorn::Configurator
       },
     :pid => nil,
     :preload_app => false,
+    :check_client_connection => false,
     :rewindable_input => true, # for Rack 2.x: (Rack::VERSION[0] <= 1),
     :client_body_buffer_size => Unicorn::Const::MAX_BODY,
     :trust_x_forwarded => true,
@@ -454,6 +455,12 @@ class Unicorn::Configurator
     set_int(:client_body_buffer_size, bytes, 0)
   end

+  # When enabled, unicorn will check the client connection by writing
+  # the beginning of the HTTP headers before calling the application.
+  def check_client_connection(bool)
+    set_bool(:check_client_connection, bool)
+  end
+
   # Allow redirecting $stderr to a given path.  Unlike doing this from
   # the shell, this allows the unicorn process to know the path its
   # writing to and rotate the file if it is used for logging.  The
diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index b3d8d71..f0c4c12 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -29,12 +29,16 @@ module Unicorn::Const

   # :stopdoc:
   # common errors we'll send back
-  ERROR_400_RESPONSE = "HTTP/1.1 400 Bad Request\r\n\r\n"
-  ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
-  ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
-  ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
+  ERROR_400_RESPONSE = "400 Bad Request\r\n\r\n"
+  ERROR_414_RESPONSE = "414 Request-URI Too Long\r\n\r\n"
+  ERROR_413_RESPONSE = "413 Request Entity Too Large\r\n\r\n"
+  ERROR_500_RESPONSE = "500 Internal Server Error\r\n\r\n"
+
   EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
+  EXPECT_100_RESPONSE_SUFFIXED = "100 Continue\r\n\r\nHTTP/1.1 "

+  HTTP_RESPONSE_START = ['HTTP', '/1.1 ']
   HTTP_EXPECT = "HTTP_EXPECT"
+
   # :startdoc:
 end
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index a0435d6..9aceb0e 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -22,11 +22,14 @@ class Unicorn::HttpParser

   NULL_IO = StringIO.new("")

+  attr_accessor :response_start_sent
+
   # :stopdoc:
   # A frozen format for this is about 15% faster
   REMOTE_ADDR = 'REMOTE_ADDR'.freeze
   RACK_INPUT = 'rack.input'.freeze
   @@input_class = Unicorn::TeeInput
+  @@check_client_connection = false

   def self.input_class
     @@input_class
@@ -35,6 +38,15 @@ class Unicorn::HttpParser
   def self.input_class=(klass)
     @@input_class = klass
   end
+
+  def self.check_client_connection
+    @@check_client_connection
+  end
+
+  def self.check_client_connection=(bool)
+    @@check_client_connection = bool
+  end
+
   # :startdoc:

   # Does the majority of the IO processing.  It has been written in
@@ -70,6 +82,15 @@ class Unicorn::HttpParser
       # an Exception thrown from the parser will throw us out of the loop
       false until add_parse(socket.kgio_read!(16384))
     end
+
+    # detect if the socket is valid by writing a partial response:
+    if @@check_client_connection && headers?
+      @response_start_sent = true
+      Unicorn::Const::HTTP_RESPONSE_START.each { |c| socket.write(c) }
+    else
+      @response_start_sent = false
+    end
+
     e[RACK_INPUT] = 0 == content_length ?
                     NULL_IO : @@input_class.new(socket, self)
     e.merge!(DEFAULTS)
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index b781e20..9c2bacc 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -18,11 +18,12 @@ module Unicorn::HttpResponse
   CRLF = "\r\n"

   # writes the rack_response to socket as an HTTP response
-  def http_response_write(socket, status, headers, body)
+  def http_response_write(socket, status, headers, body,
response_start_sent=false)
     status = CODES[status.to_i] || status

+    http_response_start = response_start_sent ? '' : 'HTTP/1.1 '
     if headers
-      buf = "HTTP/1.1 #{status}\r\n" \
+      buf = "#{http_response_start}#{status}\r\n" \
             "Date: #{httpdate}\r\n" \
             "Status: #{status}\r\n" \
             "Connection: close\r\n"
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 13df55a..abe27db 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -17,6 +17,7 @@ class Unicorn::HttpServer
                 :listener_opts, :preload_app,
                 :reexec_pid, :orig_app, :init_listeners,
                 :master_pid, :config, :ready_pipe, :user
+
   attr_reader :pid, :logger
   include Unicorn::SocketHelper
   include Unicorn::HttpResponse
@@ -355,6 +356,14 @@ class Unicorn::HttpServer
     Unicorn::HttpParser.trust_x_forwarded = bool
   end

+  def check_client_connection
+    Unicorn::HttpRequest.check_client_connection
+  end
+
+  def check_client_connection=(bool)
+    Unicorn::HttpRequest.check_client_connection = bool
+  end
+
   private

   # wait for a signal hander to wake us up and then consume the pipe
@@ -524,23 +533,33 @@ class Unicorn::HttpServer
       Unicorn.log_error(@logger, "app error", e)
       Unicorn::Const::ERROR_500_RESPONSE
     end
+    msg = 'HTTP/1.1 ' + msg unless @request.response_start_sent
     client.kgio_trywrite(msg)
     client.close
     rescue
   end

+  def expect_100_response
+    if @request.response_start_sent
+      Unicorn::Const::EXPECT_100_RESPONSE_SUFFIXED
+    else
+      Unicorn::Const::EXPECT_100_RESPONSE
+    end
+  end
+
   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
   def process_client(client)
+    @request.response_start_sent = false
     status, headers, body = @app.call(env = @request.read(client))

     if 100 == status.to_i
-      client.write(Unicorn::Const::EXPECT_100_RESPONSE)
+      client.write(expect_100_response)
       env.delete(Unicorn::Const::HTTP_EXPECT)
       status, headers, body = @app.call(env)
     end
     @request.headers? or headers = nil
-    http_response_write(client, status, headers, body)
+    http_response_write(client, status, headers, body,
@request.response_start_sent)
     client.shutdown # in case of fork() in Rack app
     client.close # flush and uncork socket immediately, no keepalive
   rescue => e
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index b30a3d6..1cee2b7 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -871,13 +871,14 @@ EOF
     wait_for_death(pid)
   end

-  def hup_test_common(preload)
+  def hup_test_common(preload, check_client=false)
     File.open("config.ru", "wb") { |fp| fp.syswrite(HI.gsub("HI", '#$$')) }
     pid_file = Tempfile.new('pid')
     ucfg = Tempfile.new('unicorn_test_config')
     ucfg.syswrite("listen '#@addr:#@port'\n")
     ucfg.syswrite("pid '#{pid_file.path}'\n")
     ucfg.syswrite("preload_app true\n") if preload
+    ucfg.syswrite("check_client_connection true\n") if check_client
     ucfg.syswrite("stderr_path 'test_stderr.#$$.log'\n")
     ucfg.syswrite("stdout_path 'test_stdout.#$$.log'\n")
     pid = xfork {
@@ -942,6 +943,10 @@ EOF
     hup_test_common(false)
   end

+  def test_check_client_hup
+    hup_test_common(false, true)
+  end
+
   def test_default_listen_hup_holds_listener
     default_listen_lock do
       res, pid_path = default_listen_setup
diff --git a/test/unit/test_configurator.rb b/test/unit/test_configurator.rb
index c19c427..fc4170e 100644
--- a/test/unit/test_configurator.rb
+++ b/test/unit/test_configurator.rb
@@ -139,6 +139,18 @@ class TestConfigurator < Test::Unit::TestCase
     end
   end

+  def test_check_client_connection
+    tmp = Tempfile.new('unicorn_config')
+    test_struct = TestStruct.new
+    tmp.syswrite("check_client_connection true\n")
+
+    assert_nothing_raised do
+      Unicorn::Configurator.new(:config_file => tmp.path).commit!(test_struct)
+    end
+
+    assert test_struct.check_client_connection
+  end
+
   def test_after_fork_proc
     test_struct = TestStruct.new
     [ proc { |a,b| }, Proc.new { |a,b| }, lambda { |a,b| } ].each do |my_proc|
-- 
1.7.7.5 (Apple Git-26)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply related	[relevance 0%]

* Re: Combating nginx 499 HTTP responses during flash traffic scenario
  2012-11-02 17:59  2%         ` Tom Burns
@ 2012-11-02 19:38  0%           ` Eric Wong
  2012-11-03 22:45  0%             ` Tom Burns
  0 siblings, 1 reply; 31+ results
From: Eric Wong @ 2012-11-02 19:38 UTC (permalink / raw)
  To: unicorn list

Tom Burns <tom.burns@jadedpixel.com> wrote:
> On Tue, Oct 30, 2012 at 5:37 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Tom Burns <tom.burns@jadedpixel.com> wrote:
> > > We'd prefer to not have to fork unicorn for this change.  How do you
> > > feel about merging this or a derivative thereof?  I can develop this
> > > further if you can send me what you'd want.
> >
> > Sure thing!
> 
> Below is a patch for this functionality.
> 
> We're going to be testing this further next week before rolling it out
> in production.

Thank you for the patch and I'm looking forward to hearing about how it
works for you guys.  I'd love to hear feedback from anybody else who is
trying this, too.

> It's still pre-writing the entire HTTP/1.1 header start, but as just
> two strings.  It could be shortened to just pre-writing HT but I
> thought writing that full word looked cleaner.
> 
> Please let me know what you think.

Everything looks good for stock configurations.  The ERROR_XXX_RESPONSE
constants should have "HTTP/1.1 " removed if used with this option, too.
That requires keeping track of whether or not we've written out
"HTTP/1.1 ", yet.

(I don't think there's much benefit in using constants for rarely-used
 error responses, either.  Most of those were inherited from Mongrel
 which had many client-triggered errors to deal with)

> The only thing it's missing from your TODO is enforcing
> tcp_nodelay/tcp_nopush as I wasn't sure where was the best place to do
> the enforcement.

It should probably be done inside Unicorn::Configurator#commit!

> Never done this inline with GMail, sorry in advance if formatting gets
> destroyed :)

"git apply" runs cleanly on this message :)

You'll probably find it more reliable and easier to use "git send-email".
The git-send-email manpage has an example for using it with
"git format-patch" + GMail SMTP.  Using "git format-patch" also lets
you preserve your original commit message/authorship information.
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply	[relevance 0%]

* Re: Combating nginx 499 HTTP responses during flash traffic scenario
  @ 2012-11-02 17:59  2%         ` Tom Burns
  2012-11-02 19:38  0%           ` Eric Wong
  0 siblings, 1 reply; 31+ results
From: Tom Burns @ 2012-11-02 17:59 UTC (permalink / raw)
  To: unicorn list

On Tue, Oct 30, 2012 at 5:37 PM, Eric Wong <normalperson@yhbt.net> wrote:
>
> Tom Burns <tom.burns@jadedpixel.com> wrote:
> > We'd prefer to not have to fork unicorn for this change.  How do you
> > feel about merging this or a derivative thereof?  I can develop this
> > further if you can send me what you'd want.
>
> Sure thing!

Below is a patch for this functionality.

We're going to be testing this further next week before rolling it out
in production.

It's still pre-writing the entire HTTP/1.1 header start, but as just
two strings.  It could be shortened to just pre-writing HT but I
thought writing that full word looked cleaner.

Please let me know what you think.

The only thing it's missing from your TODO is enforcing
tcp_nodelay/tcp_nopush as I wasn't sure where was the best place to do
the enforcement.

Never done this inline with GMail, sorry in advance if formatting gets
destroyed :)

Cheers,
Tom

---
 examples/unicorn.conf.rb       |    6 ++++++
 lib/unicorn/configurator.rb    |    7 +++++++
 lib/unicorn/const.rb           |    4 ++++
 lib/unicorn/http_request.rb    |   16 ++++++++++++++++
 lib/unicorn/http_response.rb   |    6 +++++-
 lib/unicorn/http_server.rb     |   19 ++++++++++++++++++-
 test/exec/test_exec.rb         |    7 ++++++-
 test/unit/test_configurator.rb |   12 ++++++++++++
 8 files changed, 74 insertions(+), 3 deletions(-)

diff --git a/examples/unicorn.conf.rb b/examples/unicorn.conf.rb
index 0238043..1f4c9c0 100644
--- a/examples/unicorn.conf.rb
+++ b/examples/unicorn.conf.rb
@@ -46,6 +46,12 @@ preload_app true
 GC.respond_to?(:copy_on_write_friendly=) and
   GC.copy_on_write_friendly = true

+# Enable this flag to have unicorn test client connections by writing the
+# beginning of the HTTP headers before calling the application.  This
+# prevents calling the application for connections that have disconnected
+# while queued.
+check_client_connection false
+
 before_fork do |server, worker|
   # the following is highly recomended for Rails + "preload_app true"
   # as there's no need for the master process to hold a connection
diff --git a/lib/unicorn/configurator.rb b/lib/unicorn/configurator.rb
index 89cbf5c..ca84a88 100644
--- a/lib/unicorn/configurator.rb
+++ b/lib/unicorn/configurator.rb
@@ -45,6 +45,7 @@ class Unicorn::Configurator
       },
     :pid => nil,
     :preload_app => false,
+    :check_client_connection => false,
     :rewindable_input => true, # for Rack 2.x: (Rack::VERSION[0] <= 1),
     :client_body_buffer_size => Unicorn::Const::MAX_BODY,
     :trust_x_forwarded => true,
@@ -454,6 +455,12 @@ class Unicorn::Configurator
     set_int(:client_body_buffer_size, bytes, 0)
   end

+  # When enabled, unicorn will check the client connection by writing
+  # the beginning of the HTTP headers before calling the application.
+  def check_client_connection(bool)
+    set_bool(:check_client_connection, bool)
+  end
+
   # Allow redirecting $stderr to a given path.  Unlike doing this from
   # the shell, this allows the unicorn process to know the path its
   # writing to and rotate the file if it is used for logging.  The
diff --git a/lib/unicorn/const.rb b/lib/unicorn/const.rb
index b3d8d71..56598d9 100644
--- a/lib/unicorn/const.rb
+++ b/lib/unicorn/const.rb
@@ -33,8 +33,12 @@ module Unicorn::Const
   ERROR_414_RESPONSE = "HTTP/1.1 414 Request-URI Too Long\r\n\r\n"
   ERROR_413_RESPONSE = "HTTP/1.1 413 Request Entity Too Large\r\n\r\n"
   ERROR_500_RESPONSE = "HTTP/1.1 500 Internal Server Error\r\n\r\n"
+
   EXPECT_100_RESPONSE = "HTTP/1.1 100 Continue\r\n\r\n"
+  EXPECT_100_RESPONSE_SUFFIXED = "100 Continue\r\n\r\nHTTP/1.1 "

+  HTTP_RESPONSE_START = ['HTTP', '/1.1 ']
   HTTP_EXPECT = "HTTP_EXPECT"
+
   # :startdoc:
 end
diff --git a/lib/unicorn/http_request.rb b/lib/unicorn/http_request.rb
index a0435d6..095ca7c 100644
--- a/lib/unicorn/http_request.rb
+++ b/lib/unicorn/http_request.rb
@@ -27,6 +27,7 @@ class Unicorn::HttpParser
   REMOTE_ADDR = 'REMOTE_ADDR'.freeze
   RACK_INPUT = 'rack.input'.freeze
   @@input_class = Unicorn::TeeInput
+  @@check_client_connection = false

   def self.input_class
     @@input_class
@@ -35,6 +36,15 @@ class Unicorn::HttpParser
   def self.input_class=(klass)
     @@input_class = klass
   end
+
+  def self.check_client_connection
+    @@check_client_connection
+  end
+
+  def self.check_client_connection=(bool)
+    @@check_client_connection = bool
+  end
+
   # :startdoc:

   # Does the majority of the IO processing.  It has been written in
@@ -70,6 +80,12 @@ class Unicorn::HttpParser
       # an Exception thrown from the parser will throw us out of the loop
       false until add_parse(socket.kgio_read!(16384))
     end
+
+    # detect if the socket is valid by writing a partial response:
+    if @@check_client_connection && headers?
+      Unicorn::Const::HTTP_RESPONSE_START.each { |c| socket.write(c) }
+    end
+
     e[RACK_INPUT] = 0 == content_length ?
                     NULL_IO : @@input_class.new(socket, self)
     e.merge!(DEFAULTS)
diff --git a/lib/unicorn/http_response.rb b/lib/unicorn/http_response.rb
index b781e20..10a92d1 100644
--- a/lib/unicorn/http_response.rb
+++ b/lib/unicorn/http_response.rb
@@ -17,12 +17,16 @@ module Unicorn::HttpResponse
   }
   CRLF = "\r\n"

+  def http_response_start
+    Unicorn::HttpParser.check_client_connection ? '' : 'HTTP/1.1 '
+  end
+
   # writes the rack_response to socket as an HTTP response
   def http_response_write(socket, status, headers, body)
     status = CODES[status.to_i] || status

     if headers
-      buf = "HTTP/1.1 #{status}\r\n" \
+      buf = "#{http_response_start}#{status}\r\n" \
             "Date: #{httpdate}\r\n" \
             "Status: #{status}\r\n" \
             "Connection: close\r\n"
diff --git a/lib/unicorn/http_server.rb b/lib/unicorn/http_server.rb
index 13df55a..3e061af 100644
--- a/lib/unicorn/http_server.rb
+++ b/lib/unicorn/http_server.rb
@@ -17,6 +17,7 @@ class Unicorn::HttpServer
                 :listener_opts, :preload_app,
                 :reexec_pid, :orig_app, :init_listeners,
                 :master_pid, :config, :ready_pipe, :user
+
   attr_reader :pid, :logger
   include Unicorn::SocketHelper
   include Unicorn::HttpResponse
@@ -355,6 +356,14 @@ class Unicorn::HttpServer
     Unicorn::HttpParser.trust_x_forwarded = bool
   end

+  def check_client_connection
+    Unicorn::HttpRequest.check_client_connection
+  end
+
+  def check_client_connection=(bool)
+    Unicorn::HttpRequest.check_client_connection = bool
+  end
+
   private

   # wait for a signal hander to wake us up and then consume the pipe
@@ -529,13 +538,21 @@ class Unicorn::HttpServer
     rescue
   end

+  def expect_100_response
+    if Unicorn::HttpRequest.check_client_connection
+      Unicorn::Const::EXPECT_100_RESPONSE_SUFFIXED
+    else
+      Unicorn::Const::EXPECT_100_RESPONSE
+    end
+  end
+
   # once a client is accepted, it is processed in its entirety here
   # in 3 easy steps: read request, call app, write app response
   def process_client(client)
     status, headers, body = @app.call(env = @request.read(client))

     if 100 == status.to_i
-      client.write(Unicorn::Const::EXPECT_100_RESPONSE)
+      client.write(expect_100_response)
       env.delete(Unicorn::Const::HTTP_EXPECT)
       status, headers, body = @app.call(env)
     end
diff --git a/test/exec/test_exec.rb b/test/exec/test_exec.rb
index b30a3d6..1cee2b7 100644
--- a/test/exec/test_exec.rb
+++ b/test/exec/test_exec.rb
@@ -871,13 +871,14 @@ EOF
     wait_for_death(pid)
   end

-  def hup_test_common(preload)
+  def hup_test_common(preload, check_client=false)
     File.open("config.ru", "wb") { |fp| fp.syswrite(HI.gsub("HI", '#$$')) }
     pid_file = Tempfile.new('pid')
     ucfg = Tempfile.new('unicorn_test_config')
     ucfg.syswrite("listen '#@addr:#@port'\n")
     ucfg.syswrite("pid '#{pid_file.path}'\n")
     ucfg.syswrite("preload_app true\n") if preload
+    ucfg.syswrite("check_client_connection true\n") if check_client
     ucfg.syswrite("stderr_path 'test_stderr.#$$.log'\n")
     ucfg.syswrite("stdout_path 'test_stdout.#$$.log'\n")
     pid = xfork {
@@ -942,6 +943,10 @@ EOF
     hup_test_common(false)
   end

+  def test_check_client_hup
+    hup_test_common(false, true)
+  end
+
   def test_default_listen_hup_holds_listener
     default_listen_lock do
       res, pid_path = default_listen_setup
diff --git a/test/unit/test_configurator.rb b/test/unit/test_configurator.rb
index c19c427..fc4170e 100644
--- a/test/unit/test_configurator.rb
+++ b/test/unit/test_configurator.rb
@@ -139,6 +139,18 @@ class TestConfigurator < Test::Unit::TestCase
     end
   end

+  def test_check_client_connection
+    tmp = Tempfile.new('unicorn_config')
+    test_struct = TestStruct.new
+    tmp.syswrite("check_client_connection true\n")
+
+    assert_nothing_raised do
+      Unicorn::Configurator.new(:config_file => tmp.path).commit!(test_struct)
+    end
+
+    assert test_struct.check_client_connection
+  end
+
   def test_after_fork_proc
     test_struct = TestStruct.new
     [ proc { |a,b| }, Proc.new { |a,b| }, lambda { |a,b| } ].each do |my_proc|
-- 
1.7.7.5 (Apple Git-26)
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply related	[relevance 2%]

* Re: Suggestion for reload action (USR2)
@ 2010-01-20  9:20  6% Iñaki Baz Castillo
  0 siblings, 0 replies; 31+ results
From: Iñaki Baz Castillo @ 2010-01-20  9:20 UTC (permalink / raw)
  To: mongrel-unicorn

El Miércoles, 20 de Enero de 2010, Iñaki Baz Castillo escribió:

> This would involve just three changes:
> 
> - Add an option "--reload" in OptionParser in bin/unicorn.
> - Store such option somewhere so it can be readed later by the Rack
> application or other library.
> - Add "--reload" to the list of original arguments when receiving USR2.

I attach a patch (for master branch) that would do the job (except the 
fact that I don't know how to store the "reload" option):


-----------------------------------------
diff --git a/bin/unicorn b/bin/unicorn
index 5af021d..43f5434 100755
--- a/bin/unicorn
+++ b/bin/unicorn
@@ -88,6 +88,10 @@ opts = OptionParser.new("", 24, '  ') do |opts|
     options[:config_file] = f
   end

+  opts.on("--reload", "HIDDEN") do |f|
+    # TODO: Store it somewhere
+  end
+
   # I'm avoiding Unicorn-specific config options on the command-line.
   # IMNSHO, config options on the command-line are redundant given
   # config files and make things unnecessarily complicated with multiple
@@ -96,7 +100,7 @@ opts = OptionParser.new("", 24, '  ') do |opts|
   opts.separator "Common options:"

   opts.on_tail("-h", "--help", "Show this message") do
-    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '')
+    puts opts.to_s.gsub(/^.*DEPRECATED.*$/s, '').gsub(/^.*HIDDEN.*$/s, '')
     exit
   end

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index e3e4315..49ad79b 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -519,7 +519,7 @@ module Unicorn
         end
         logger.info "executing #{cmd.inspect} (in #{Dir.pwd})"
         before_exec.call(self)
-        exec(*cmd)
+        exec(*cmd << "--reload")
       end
       proc_name 'master (old)'
     end
-----------------------------------------


The patch is available here:
  http://oversip.net/public/unicorn_reload.patch

(unfortunatelly I cannot attach files in this mailist as the mail is 
rejected).


Regards.


-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

^ permalink raw reply related	[relevance 6%]

* [ANN] unicorn 0.94.0 - small fixes and new features
@ 2009-11-05 10:06  5% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-11-05 10:06 UTC (permalink / raw)
  To: mongrel-unicorn

Unicorn is a HTTP server for Rack applications designed to only serve
fast clients on low-latency, high-bandwidth connections and take
advantage of features in Unix/Unix-like kernels.  Slow clients should
only be served by placing a reverse proxy capable of fully buffering
both the the request and response in between Unicorn and slow clients.

* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git

Changes:

The HTTP parser is fix for oddly-aligned reads of trailers (this
technically affects headers, too, but is highly unlikely due to
our non-support of slow clients).  This allows our HTTP parser
to better support very slow clients when used by other servers
(like Rainbows!).  Fortunately this bug does not appear to lead
to any invalid memory accesses (and potential arbitrary code
execution).

FreeBSD (and possibly other *BSDs) support is improved and and
all the test cases pass under FreeBSD 7.2.  Various flavors of
GNU/Linux remains our primary platform for development and
production.

New features added include the "working_directory" directive in
the configurator .  Even without specifying a
"working_directory", symlink-aware detection of the current path
no longer depends on /bin/sh so it should work out-of-the-box on
FreeBSD and Solaris and not just systems where /bin/sh is dash,
ksh93 or bash.

User-switching support is finally supported but only intended
for use in the after_fork hook of worker processes.  Putting it
in the after_fork hook allows allows users to set things like
CPU affinity[1] on a per-worker basis before dropping
privileges.  The master process retains all privileges it
started with.

The ENV["RACK_ENV"] (process-wide) environment variable is now
both read and set for `unicorn' in the same way RAILS_ENV is
used by `unicorn_rails'.  This allows the Merb launcher to read
ENV["RACK_ENV"] in config.ru.  Other web servers already set
this and there may be applications or libraries that already
rely on this de facto standard.

Eric Wong (26):
      cleanup: avoid redundant error checks for fstat
      test_helper: connect(2) may fail with EINVAL
      GNUmakefile: fix non-portable tar(1) usage
      tests: provide a pure Ruby setsid(8) equivalent
      more portable symlink awareness for START_CTX[:cwd]
      test_signals: avoid portability issues with fchmod(2)
      cleanup error handling and make it less noisy
      Do not override Dir.chdir in config files
      configurator: add "working_directory" directive
      configurator: working_directory is expanded
      configurator: set ENV["PWD"] with working_directory, too
      configurator: working_directory affects pid, std{err,out}_paths
      configurator: update documentation for working_directory
      TODO: remove working_directory bit, done
      Util.reopen_logs: remove needless Range
      worker: user/group switching for after_fork hooks
      Fix autoload of Etc in Worker for Ruby 1.9
      bin/unicorn: allow RACK_ENV to be passed from parent
      tests for RACK_ENV preservation
      http: allow headers/trailers to be written byte-wise
      http: extra test for bytewise chunked bodies
      tee_input: do not clobber trailer buffer on partial uploads
      test_exec: ensure master is killed after test
      Util::tmpio returns a TmpIO that responds to #size
      TODO: remove user-switching bit, done
      unicorn 0.94.0

Wayne Larsen (1):
      bin/unicorn: set ENV["RACK_ENV"] on startup

[1] - Unicorn does not support CPU affinity directly, but it is
      possible to load code that allows it inside after_fork hooks,
      or even just call sched_tool(8).

-- 
Eric Wong

^ permalink raw reply	[relevance 5%]

* Re: alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD
  2009-10-08 23:57  7% alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD Jeremy Evans
@ 2009-10-09  3:22  0% ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-10-09  3:22 UTC (permalink / raw)
  To: Jeremy Evans; +Cc: mongrel-unicorn

Jeremy Evans <jeremyevans0@gmail.com> wrote:
> On OpenBSD:
> 
> $ ruby -e "File.new('TODO').chmod(Time.now.to_i)"
> -e:1:in `chmod': Invalid argument - TODO (Errno::EINVAL)
>         from -e:1
> 
> This is explained in the man page:
> 
>      int
>      fchmod(int fd, mode_t mode);
> 
>     ...
> 
>      [EINVAL]      mode contains bits other than the file type and those de-
>                    scribed above.
> 
> I think 04777 is the highest allowed mode on OpenBSD.  Time.now.to_i
> is obviously higher than that.

Yikes.  I was worried about the portability of this.  I actually redid
this in the Revactor model of Rainbows! to flip between 0 and 1, I'll do
that in Unicorn, too.

> Here's a diff that should fix the problem.  At the very least it
> allows the workers to start without crashing:
> 
> diff --git a/lib/unicorn.rb b/lib/unicorn.rb
> index ddec8e9..092f500 100644
> --- a/lib/unicorn.rb
> +++ b/lib/unicorn.rb
> @@ -579,13 +579,13 @@ module Unicorn
>          # changes with chmod doesn't update ctime on all filesystems; so
>          # we change our counter each and every time (after process_client
>          # and before IO.select).
> -        t == (ti = Time.now.to_i) or alive.chmod(t = ti)
> +        t == (ti = Time.now.to_i) or (t = ti;
> alive.chmod(alive.stat.mode ^ 0100))

I think the Time.now.to_i bit is overkill (and probably detrimental to
performance on non-x86_64, non-VDSO-enabled, non-GNU/Linux machines).

> There are definitely other ways that will work, as long as the mode is
> kept between 0 and 04777.

Here's what I committed and pushed out, it should work for all *nix
now since it only alternates between 0 and 1, but be sure to let
me know if it doesn't for some reason.  Thanks!

>From 24a1b4c6b5fcb948e4fdee04e286c044d6d45f98 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Thu, 8 Oct 2009 19:56:29 -0700
Subject: [PATCH] fchmod heartbeat flips between 0/1 for compatibility

This removes the Time.now.to_i comparison that was used to avoid
multiple, no-op fchmod() syscalls[1] within the same second.

This should allow us to run on OpenBSD where it can raise EINVAL
when Time.now.to_i is passed to it.

Reported-by: Jeremy Evans <jeremyevans0@gmail.com>

[1] - gettimeofday() from Time.now is not a real syscall on
VDSO-enabled x86_64 GNU/Linux systems where Unicorn is primarily
developed.
---
 lib/unicorn.rb |    8 ++++----
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index d63567f..13c203a 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -575,13 +575,13 @@ module Unicorn
       nr = 0 # this becomes negative if we need to reopen logs
       alive = worker.tmp # tmp is our lifeline to the master process
       ready = LISTENERS
-      t = ti = 0
 
       # closing anything we IO.select on will raise EBADF
       trap(:USR1) { nr = -65536; SELF_PIPE.first.close rescue nil }
       trap(:QUIT) { alive = nil; LISTENERS.each { |s| s.close rescue nil } }
       [:TERM, :INT].each { |sig| trap(sig) { exit!(0) } } # instant shutdown
       logger.info "worker=#{worker.nr} ready"
+      m = 0
 
       begin
         nr < 0 and reopen_worker_logs(worker.nr)
@@ -595,13 +595,13 @@ module Unicorn
         # changes with chmod doesn't update ctime on all filesystems; so
         # we change our counter each and every time (after process_client
         # and before IO.select).
-        t == (ti = Time.now.to_i) or alive.chmod(t = ti)
+        alive.chmod(m = 0 == m ? 1 : 0)
 
         ready.each do |sock|
           begin
             process_client(sock.accept_nonblock)
             nr += 1
-            t == (ti = Time.now.to_i) or alive.chmod(t = ti)
+            alive.chmod(m = 0 == m ? 1 : 0)
           rescue Errno::EAGAIN, Errno::ECONNABORTED
           end
           break if nr < 0
@@ -614,7 +614,7 @@ module Unicorn
         redo unless nr == 0 # (nr < 0) => reopen logs
 
         ppid == Process.ppid or return
-        alive.chmod(t = 0)
+        alive.chmod(m = 0 == m ? 1 : 0)
         begin
           # timeout used so we can detect parent death:
           ret = IO.select(LISTENERS, nil, SELF_PIPE, timeout) or redo
-- 
Eric Wong

^ permalink raw reply related	[relevance 0%]

* alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD
@ 2009-10-08 23:57  7% Jeremy Evans
  2009-10-09  3:22  0% ` Eric Wong
  0 siblings, 1 reply; 31+ results
From: Jeremy Evans @ 2009-10-08 23:57 UTC (permalink / raw)
  To: mongrel-unicorn

On OpenBSD:

$ ruby -e "File.new('TODO').chmod(Time.now.to_i)"
-e:1:in `chmod': Invalid argument - TODO (Errno::EINVAL)
        from -e:1

This is explained in the man page:

     int
     fchmod(int fd, mode_t mode);

    ...

     [EINVAL]      mode contains bits other than the file type and those de-
                   scribed above.

I think 04777 is the highest allowed mode on OpenBSD.  Time.now.to_i
is obviously higher than that.

Here's a diff that should fix the problem.  At the very least it
allows the workers to start without crashing:

diff --git a/lib/unicorn.rb b/lib/unicorn.rb
index ddec8e9..092f500 100644
--- a/lib/unicorn.rb
+++ b/lib/unicorn.rb
@@ -579,13 +579,13 @@ module Unicorn
         # changes with chmod doesn't update ctime on all filesystems; so
         # we change our counter each and every time (after process_client
         # and before IO.select).
-        t == (ti = Time.now.to_i) or alive.chmod(t = ti)
+        t == (ti = Time.now.to_i) or (t = ti;
alive.chmod(alive.stat.mode ^ 0100))

         ready.each do |sock|
           begin
             process_client(sock.accept_nonblock)
             nr += 1
-            t == (ti = Time.now.to_i) or alive.chmod(t = ti)
+            t == (ti = Time.now.to_i) or (t = ti;
alive.chmod(alive.stat.mode ^ 0100))
           rescue Errno::EAGAIN, Errno::ECONNABORTED
           end
           break if nr < 0

There are definitely other ways that will work, as long as the mode is
kept between 0 and 04777.

Jeremy

^ permalink raw reply related	[relevance 7%]

* [ANN] unicorn 0.93.2 - more compatible with Rails
@ 2009-10-07  8:58  5% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-10-07  8:58 UTC (permalink / raw)
  To: mongrel-unicorn

Unicorn is a HTTP server for Rack applications designed to take
advantage of features in Unix/Unix-like kernels and only serve fast
clients on low-latency, high-bandwidth connections.  Slow clients should
only be served by placing a reverse proxy capable of fully-buffering
both the the request and response in between Unicorn and slow clients.

* http://unicorn.bogomips.org/
* mongrel-unicorn@rubyforge.org
* git://git.bogomips.org/unicorn.git

Thanks to Chris Wanstrath for reporting issues with large
POST bodies and for helping me test.

Changes:

Avoid truncated POST bodies with URL-encoded forms in Rails
by switching TeeInput to use read-in-full semantics (only) when
a Content-Length: header exists.  Chunked request bodies
continue to exhibit readpartial semantics to support
simultaneous bidirectional chunking.

The lack of return value checking in Rails to protect against a
short ios.read(length) is entirely reasonable even if not
pedantically correct.  Most ios.read(length) implementations
return the full amount requested except right before EOF.

A ticket has been opened here to track the issue:
  https://rails.lighthouseapp.com/projects/8994/tickets/3343

Also there are some minor documentation improvements.

Eric Wong (8):
      Fix NEWS generation on single-paragraph tag messages
      Include GPLv2 in docs
      doc: make it clear contributors retain copyrights
      TODO: removed Rainbows! (see rainbows.rubyforge.org)
      Document the START_CTX hash contents
      more-compatible TeeInput#read for POSTs with Content-Length
      tests for read-in-full vs readpartial semantics
      unicorn 0.93.2
-- 
Eric Wong

^ permalink raw reply	[relevance 5%]

* Re: TERM vs QUIT
  @ 2009-10-02  1:53  5%   ` Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-10-02  1:53 UTC (permalink / raw)
  To: Chris Wanstrath; +Cc: mongrel-unicorn

Chris Wanstrath <chris@ozmm.org> wrote:
> On Thu, Oct 1, 2009 at 4:52 PM, Chris Wanstrath <chris@ozmm.org> wrote:
> 
> > According to http://en.wikipedia.org/wiki/SIGTERM, SIGTERM should be
> > used for graceful termination.
> >
> > According to http://www.cons.org/cracauer/sigint.html, SIGQUIT should
> > be used for immediate termination.
> >
> > Is there any reason Unicorn seems to flip these two and use TERM for
> > immediate termination and QUIT for graceful termination?
> 
> Looks like nginx flips them, too, and Unicorn's signal handling is
> based on nginx's.
> 
> We will defer to the Russians and make our stuff consistent with nginx!

Yup, I based the current user interface around the one in nginx to
capitalize on some of my target users' existing familiarity with nginx
and also to (hopefully) lower the barrier of entry for folks new to
both.

Of course I'm not happy with some of the choices nginx made. Using "pid"
instead of "pid_file" in the configuration language really bothers me,
and I completely agree TERM is more logical for graceful shutdowns. But
in the end the nginx UI was "good enough" and consistency won me over.

So things like the init script in examples/init.sh can be adapted to
work for both unicorn and nginx without changing anything other than
path names and some (hopefully few) command-line arguments.


Making user interface choices for for other people isn't something I'm
comfortable with, so I just try to take advantage of what people
already know/expect as much as possible.  I'm lazy in that way :)


For a few weeks (or months now?) I've been debating/avoiding the
"--kill" switch item I have in the TODO (it was requested by a user in
Real-Life(TM)).  I'm still not sure where/how/if it would fit, and
maybe it just doesn't fit.  /me goes on to ponder it a bit more ...

-- 
Eric Wong

^ permalink raw reply	[relevance 5%]

* [ANN] unicorn 0.92.0
@ 2009-09-18 22:16  3% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-09-18 22:16 UTC (permalink / raw)
  To: mongrel-unicorn

Unicorn is a Rack HTTP server for Unix and fast clients

Small fixes and documentation are the focus of this release.

James Golick reported and helped me track down a bug that caused
SIGHUP to drop the default listener (0.0.0.0:8080) if and only
if listeners were completely unspecified in both the
command-line and Unicorn config file.  The Unicorn config file
remains the recommended option for specifying listeners as it
allows fine-tuning of the :backlog, :rcvbuf, :sndbuf,
:tcp_nopush, and :tcp_nodelay options.

There are some documentation (and resulting website)
improvements.  setup.rb users will notice the new section 1
manpages for `unicorn` and `unicorn_rails`, Rubygems users
will have to install manpages manually or use the website.

  Edit: That's not entirely true, I screwed up the package but
  you can get them from http://unicorn.bogomips.org/unicorn.1
  and http://unicorn.bogomips.org/unicorn_rails.1

The HTTP parser got a 3rd-party code review which resulted in
some cleanups and one insignificant bugfix as a result.

Additionally, the HTTP parser compiles, runs and passes unit
tests under Rubinius.  The pure-Ruby parts still do not work yet
and we currently lack the resources/interest to pursue this
further but help will be gladly accepted.

The website now has an Atom feed for new release announcements.
Those unfamiliar with Atom or HTTP may finger unicorn@bogomips.org
for the latest announcements.

Eric Wong (53):
      README: update with current version
      http: cleanup and avoid potential signedness warning
      http: clarify the setting of the actual header in the hash
      http: switch to macros for bitflag handling
      http: refactor keepalive tracking to functions
      http: use explicit elses for readability
      http: remove needless goto
      http: extra assertion when advancing p manually
      http: verbose assertions
      http: NIL_P(var) instead of var == Qnil
      http: rb_gc_mark already ignores immediates
      http: ignore Host: continuation lines with absolute URIs
      doc/SIGNALS: fix the no-longer-true bit about socket options
      "encoding: binary" comments for all sources (1.9)
      http_response: don't "rescue nil" for body.close
      CONTRIBUTORS: fix capitalization for why
      http: support Rubies without the OBJ_FROZEN macro
      http: define OFFT2NUM macro on Rubies without it
      http: no-op rb_str_modify() for Rubies without it
      http: compile with -fPIC
      http: use rb_str_{update,flush} if available
      http: create a new string buffer on empty values
      Update documentation for Rubinius support status
      http: cleanup assertion for memoized header strings
      http: add #endif comment labels where appropriate
      Add .mailmap file for "git shortlog" and other tools
      Update Manifest with mailmap
      Fix comment about speculative accept()
      SIGNALS: use "Unicorn" when referring to the web server
      Add new Documentation section for manpages
      test_exec: add extra tests for HUP and preload_app
      socket_helper: (FreeBSD) don't freeze the accept filter constant
      Avoid freezing objects that don't benefit from it
      SIGHUP no longer drops lone, default listener
      doc: generate ChangeLog and NEWS file for RDoc
      Remove Echoe and roll our own packaging/release...
      unicorn_rails: close parentheses in help message
      launchers: deprecate ambiguous -P/--p* switches
      man1/unicorn: avoid unnecessary emphasis
      Add unicorn_rails(1) manpage
      Documentation: don't force --rsyncable flag with gzip(1)
      Simplify and standardize manpages build/install
      GNUmakefile: package .tgz includes all generated files
      doc: begin integration of HTML manpages into RDoc
      Update TODO
      html: add Atom feeds
      doc: latest news is available through finger
      NEWS.atom: file timestamp matches latest entry
      pandoc needs the standalone switch for manpages
      man1/unicorn: split out RACK ENVIRONMENT section
      man1/unicorn_rails: fix unescaped underscore
      NEWS.atom.xml only lists the first 10 entries
      unicorn 0.92.0

* site: http://unicorn.bogomips.org/
* git: git://git.bogomips.org/unicorn.git
* cgit: http://git.bogomips.org/cgit/unicorn.git/
* list: mongrel-unicorn@rubyforge.org
* nntp: nntp://news.gmane.org/gmane.comp.lang.ruby.unicorn.general
* finger: unicorn@bogomips.org

-- 
Eric Wong

^ permalink raw reply	[relevance 3%]

* [ANN] unicorn 0.9.0 (experimental release)
@ 2009-07-01 22:58  4% Eric Wong
  0 siblings, 0 replies; 31+ results
From: Eric Wong @ 2009-07-01 22:58 UTC (permalink / raw)
  To: mongrel-unicorn, ruby-talk, rack-devel; +Cc: mongrel-development

Unicorn is a Rack HTTP server for Unix, fast clients and nothing else[1]

We now have support for "Transfer-Encoding: chunked" bodies in
requests.  Not only that, Rack applications reading input bodies
get that data streamed off to the client socket on an as-needed
basis.  This allows the application to do things like upload
progress notification and even tunneling of arbitrary stream
protocols via bidirectional chunked encoding.

See Unicorn::App::Inetd and examples/git.ru (including the
comments) for an example of tunneling the git:// protocol over
HTTP.

This release also gives applications the ability to respond
positively to "Expect: 100-continue" headers before being rerun
without closing the socket connection.  See Unicorn::App::Inetd
for an example of how this is used.

This release is NOT recommended for production use.

Eric Wong (43):
      http_request: no need to reset the request
      http_request: StringIO is binary for empty bodies (1.9)
      http_request: fix typo for 1.9
      Transfer-Encoding: chunked streaming input support
      Unicorn::App::Inetd: reinventing Unix, poorly :)
      README: update with mailing list info
      local.mk.sample: publish_doc gzips all html, js, css
      Put copyright text in new files, include GPL2 text
      examples/cat-chunk-proxy: link to proposed curl(1) patch
      Update TODO
      Avoid duplicating the "Z" constant
      Optimize body-less GET/HEAD requests (again)
      tee_input: Don't expose the @rd object as a return value
      exec_cgi: small cleanups
      README: another note about older Sinatra
      tee_input: avoid defining a @rd.size method
      Make TeeInput easier to use
      test_upload: add tests for chunked encoding
      GNUmakefile: more stringent error checking in tests
      test_upload: fix ECONNRESET with 1.9
      GNUmakefile: allow TRACER= to be specified for tests
      test_rails: workaround long-standing 1.9 bug
      tee_input: avoid rereading fresh data
      "Fix" tests that break with stream_input=false
      inetd: fix broken constant references
      configurator: provide stream_input (true|false) option
      chunked_reader: simpler interface
      http_request: force BUFFER to be Encoding::BINARY
      ACK clients on "Expect: 100-continue" header
      Only send "100 Continue" when no body has been sent
      http_request: tighter Transfer-Encoding: "chunked" check
      Add trailer_parser for parsing trailers
      chunked_reader: Add test for chunk parse failure
      TeeInput: use only one IO for tempfile
      trailer_parser: set keys with "HTTP_" prefix
      TrailerParser integration into ChunkedReader
      Unbind listeners as before stopping workers
      Retry listen() on EADDRINUSE 5 times ever 500ms
      Re-add support for non-portable socket options
      Move "Expect: 100-continue" handling to the app
      tee_input: avoid ignoring initial body blob
      Force streaming input onto apps by default
      unicorn 0.9.0

* site: http://unicorn.bogomips.org/
* git: git://git.bogomips.org/unicorn.git
* http+git: http://git.bogomips.org:8080/unicorn.git [1]
* cgit: http://git.bogomips.org/cgit/unicorn.git/
* list: mongrel-unicorn@rubyforge.org

[1] - Actually, most of the new features in this release should in fact
work on non-Unix OSes so they can be adapted for other servers with
wider audiences.

[2] - This is the git:// protocol tunneled inside a bidirectional
"Transfer-Encoding: chunked" request/response using this latest
release of Unicorn.

-- 
Eric Wong

^ permalink raw reply	[relevance 4%]

Results 1-31 of 31 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2009-07-01 22:58  4% [ANN] unicorn 0.9.0 (experimental release) Eric Wong
2009-09-18 22:16  3% [ANN] unicorn 0.92.0 Eric Wong
2009-10-01 23:52     TERM vs QUIT Chris Wanstrath
2009-10-01 23:56     ` Chris Wanstrath
2009-10-02  1:53  5%   ` Eric Wong
2009-10-07  8:58  5% [ANN] unicorn 0.93.2 - more compatible with Rails Eric Wong
2009-10-08 23:57  7% alive.chmod(Time.now.to_i) raises Errno::EINVAL on OpenBSD Jeremy Evans
2009-10-09  3:22  0% ` Eric Wong
2009-11-05 10:06  5% [ANN] unicorn 0.94.0 - small fixes and new features Eric Wong
2010-01-20  9:20  6% Suggestion for reload action (USR2) Iñaki Baz Castillo
2012-10-29 17:44     Combating nginx 499 HTTP responses during flash traffic scenario Tom Burns
2012-10-29 18:45     ` Eric Wong
2012-10-29 21:53       ` Eric Wong
2012-10-30 20:40         ` Tom Burns
2012-10-30 21:37           ` Eric Wong
2012-11-02 17:59  2%         ` Tom Burns
2012-11-02 19:38  0%           ` Eric Wong
2012-11-03 22:45  0%             ` Tom Burns
2012-11-05 11:48  0%               ` Eric Wong
2014-10-24 17:33     Having issue with Unicorn Imdad
2014-10-24 17:45     ` Eric Wong
2014-10-24 18:02  6%   ` Imdad
     [not found]     <3137844.nsIDTWmvl4@debstor>
     [not found]     ` <20150625083118.GA22140@luke.ws.skroutz.gr>
2015-06-25 23:26       ` [DRE-maint] unicorn: native systemd service Eric Wong
2015-06-26 11:41         ` Christos Trochalakis
2015-06-27  3:05           ` Eric Wong
2015-06-30  9:20             ` Christos Trochalakis
2015-06-30 17:30               ` Eric Wong
2015-07-08 13:08  4%             ` Christos Trochalakis
     [not found]     <20161031-unicorn-5.2.0-released@bogomips.org>
2016-10-31 20:04  5% ` [ANN] unicorn 5.2.0 - Rack HTTP server for fast clients and *nix Eric Wong
2017-02-25 14:03     [PATCH] check_client_connection: use tcp state on linux Simon Eskildsen
2017-02-25 16:19     ` Simon Eskildsen
2017-02-25 23:12       ` Eric Wong
2017-02-27 11:44  3%     ` Simon Eskildsen
2017-02-28 21:12  0%       ` Eric Wong
2017-03-01  3:18             ` Eric Wong
2017-03-06 21:32  3%           ` Simon Eskildsen
2017-03-08  6:02     [PATCH 0/3] TCP_INFO check_client_connection followups Eric Wong
2017-03-08  6:02  3% ` [PATCH 2/3] revert signature change to HttpServer#process_client Eric Wong
2017-03-20 20:32  5% [PATCH] doc: remove private email support address Eric Wong
2017-03-23  2:48  4% [ANN] raindrops 0.18.0 - real-time stats for preforking Rack servers Eric Wong
2018-08-20 20:30  4% [PATCH] shrink pipes under Linux Eric Wong
2020-01-14  7:46     [PATCH] doc: s/bogomips.org/yhbt.net/g Eric Wong
2020-01-14  7:46  1% ` Eric Wong
2021-12-25 17:41     [PATCH 0/3] Ruby 3.1 + doc/URL updates Eric Wong
2021-12-25 17:41  3% ` [PATCH 3/3] doc: v3 .onion updates, nntp => nntps, minor wording changes Eric Wong
2022-07-05 20:05  3% [PATCH] Master promotion with SIGURG (CoW optimization) Jean Boussier
2022-07-06  2:33     ` Eric Wong
2022-07-06  7:40  4%   ` Jean Boussier
2022-07-08 13:17  2% [PATCH] Get rid of Kgio Jean Boussier
2023-06-05 10:32  3% [PATCH 00-23/23] start porting tests to Perl5 Eric Wong
2023-09-05  9:44  1% [RFC 0-3/3] depend on Ruby 2.5+, eliminate kgio Eric Wong
2023-09-10 20:08  1% [PATCH 00..11/11] more tests to Perl 5 Eric Wong
2024-03-23 19:45  4% [PATCH 0/4] a small pile of patches Eric Wong

Code repositories for project(s) associated with this public inbox

	https://yhbt.net/unicorn.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).