There's no TCP_NOPUSH in Solaris 10

There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

The definition of kgio_autopush_read and kgio_autopush_write (in
read_write.c) never happened in Solaris 10: it's not linux and there's
no TCP_NOPUSH anywhere in its system header. It caused error when
running unicorn (and of course the test):

ld.so.1: ruby: fatal: relocation error: file
/home/edho/app/ruby19/lib/ruby/gems/1.9.1/gems/kgio-2.7.3/lib/kgio_ext.so:
symbol kgio_autopush_read: referenced symbol not found
ld.so.1: ruby: fatal: relocation error: file
/home/edho/app/ruby19/lib/ruby/gems/1.9.1/gems/kgio-2.7.3/lib/kgio_ext.so:
symbol kgio_autopush_write: referenced symbol not found

Forcing definition of them enables unicorn to work (at least seems to
be working) but fails test (tried both Linux and BSD version):

  1) Failure:
test_wait_writable_ruby_default(TestUnixServerReadClientWrite)
[/home/edho/git/kgio/test/lib_read_write.rb:289]:
Exception raised:
<Interrupt>.

(it took long time so I stopped it)

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> The definition of kgio_autopush_read and kgio_autopush_write (in
> read_write.c) never happened in Solaris 10: it's not linux and there's
> no TCP_NOPUSH anywhere in its system header. It caused error when
> running unicorn (and of course the test):

I don't care for non-Free platforms, but maybe some of the other BSDs
got broken, too, so I pushed out this fix to "master" on
git://bogomips.org/kgio.git

>From 85ae255f73b9b81ae3d17e6420dbb95a29dbe8b7 Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Mon, 19 Mar 2012 05:53:03 +0000
Subject: [PATCH] fix for non-Linux, non-TCP_NOPUSH platforms

We don't need to care for TCP_NOPUSH in read_write.c, it's
entirely in autopush.c and no-op on platforms without
TCP_CORK/TCP_NOPUSH.  TCP_CORK/TCP_NOPUSH are non-POSIX, so
it's entirely possible some Free systems will lack them.

Reported-by: Edho Arief <edho@myconan.net>
---
 ext/kgio/read_write.c |    3 ---
 1 file changed, 3 deletions(-)

diff --git a/ext/kgio/read_write.c b/ext/kgio/read_write.c
index 9c2440e..51d2d16 100644
--- a/ext/kgio/read_write.c
+++ b/ext/kgio/read_write.c
@@ -20,11 +20,8 @@ static inline void kgio_autopush_write(VALUE io) { }
 
 #else
 static const int peek_flags = MSG_PEEK;
-#  include <netinet/tcp.h>
-#  if defined(TCP_NOPUSH)
 static inline void kgio_autopush_read(VALUE io) { kgio_autopush_recv(io); }
 static inline void kgio_autopush_write(VALUE io) { kgio_autopush_send(io); }
-#  endif
 #endif
 
 NORETURN(static void raise_empty_bt(VALUE, const char *));

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> Forcing definition of them enables unicorn to work (at least seems to
> be working) but fails test (tried both Linux and BSD version):

Wait, this fails on Linux and *BSD, too?  Which Ruby version and exactly
which OSes did you use?

>   1) Failure:
> test_wait_writable_ruby_default(TestUnixServerReadClientWrite)
> [/home/edho/git/kgio/test/lib_read_write.rb:289]:
> Exception raised:
> <Interrupt>.
> 
> (it took long time so I stopped it)

I've removed my idiotic assert_nothing_raised usage in all tests to make
it easier to track down.  Also pushed to bogomips.org/kgio.git

>From 74b9f78e11b915439555290dc3bdd4331303561c Mon Sep 17 00:00:00 2001
From: Eric Wong <normalperson@yhbt.net>
Date: Mon, 19 Mar 2012 06:05:06 +0000
Subject: [PATCH] test/*: remove assert_nothing_raised

It makes test failures hard to track down, tests will
already fail if exceptions are thrown and we'll get
nice backtraces.
---
 test/lib_read_write.rb            |   19 +++++++------------
 test/test_accept_class.rb         |   10 +++++-----
 test/test_autopush.rb             |   22 +++++++++-------------
 test/test_connect_fd_leak.rb      |   10 ++++------
 test/test_poll.rb                 |    4 +---
 test/test_singleton_read_write.rb |    2 +-
 test/test_tryopen.rb              |    2 +-
 7 files changed, 28 insertions(+), 41 deletions(-)

diff --git a/test/lib_read_write.rb b/test/lib_read_write.rb
index d7f50c2..dfcba20 100644
--- a/test/lib_read_write.rb
+++ b/test/lib_read_write.rb
@@ -9,10 +9,8 @@ module LibReadWriteTest
   RANDOM_BLOB = File.open("/dev/urandom") { |fp| fp.read(10 * 1024 * 1024) }
 
   def teardown
-    assert_nothing_raised do
-      @rd.close if defined?(@rd) && ! @rd.closed?
-      @wr.close if defined?(@wr) && ! @wr.closed?
-    end
+    @rd.close if defined?(@rd) && ! @rd.closed?
+    @wr.close if defined?(@wr) && ! @wr.closed?
   end
 
   def test_write_empty
@@ -264,10 +262,8 @@ module LibReadWriteTest
     foo = nil
     t0 = Time.now
     thr = Thread.new { sleep 1; @wr.write "HELLO" }
-    assert_nothing_raised do
-      foo = @rd.kgio_read(5)
-      elapsed = Time.now - t0
-    end
+    foo = @rd.kgio_read(5)
+    elapsed = Time.now - t0
     assert elapsed >= 1.0, "elapsed: #{elapsed}"
     assert_equal "HELLO", foo
     thr.join
@@ -286,10 +282,9 @@ module LibReadWriteTest
     foo = nil
     t0 = Time.now
     thr = Thread.new { sleep 1; @rd.readpartial(nr) }
-    assert_nothing_raised do
-      foo = @wr.kgio_write("HELLO")
-      elapsed = Time.now - t0
-    end
+    foo = @wr.kgio_write("HELLO")
+    elapsed = Time.now - t0
+
     assert_nil foo
     if @wr.stat.pipe?
       assert elapsed >= 1.0, "elapsed: #{elapsed}"
diff --git a/test/test_accept_class.rb b/test/test_accept_class.rb
index cf59a2f..0e1d172 100644
--- a/test/test_accept_class.rb
+++ b/test/test_accept_class.rb
@@ -12,12 +12,12 @@ class TestAcceptClass < Test::Unit::TestCase
   end
 
   def teardown
-    assert_nothing_raised { Kgio.accept_class = nil }
+    Kgio.accept_class = nil
     assert_equal Kgio::Socket, Kgio.accept_class
   end
 
   def test_tcp_socket
-    assert_nothing_raised { Kgio.accept_class = Kgio::TCPSocket }
+    Kgio.accept_class = Kgio::TCPSocket
     assert_equal Kgio::TCPSocket, Kgio.accept_class
   end
 
@@ -31,21 +31,21 @@ class TestAcceptClass < Test::Unit::TestCase
     @srv = Kgio::TCPServer.new(@host, 0)
     @port = @srv.addr[1]
 
-    assert_nothing_raised { Kgio.accept_class = Kgio::TCPSocket }
+    Kgio.accept_class = Kgio::TCPSocket
     client = TCPSocket.new(@host, @port)
     assert_instance_of Kgio::TCPSocket, @srv.kgio_accept
     client = TCPSocket.new(@host, @port)
     IO.select([@srv])
     assert_instance_of Kgio::TCPSocket, @srv.kgio_tryaccept
 
-    assert_nothing_raised { Kgio.accept_class = nil }
+    Kgio.accept_class = nil
     client = TCPSocket.new(@host, @port)
     assert_instance_of Kgio::Socket, @srv.kgio_accept
     client = TCPSocket.new(@host, @port)
     IO.select([@srv])
     assert_instance_of Kgio::Socket, @srv.kgio_tryaccept
 
-    assert_nothing_raised { Kgio.accept_class = Kgio::UNIXSocket }
+    Kgio.accept_class = Kgio::UNIXSocket
     client = TCPSocket.new(@host, @port)
     assert_instance_of Kgio::UNIXSocket, @srv.kgio_accept
     client = TCPSocket.new(@host, @port)
diff --git a/test/test_autopush.rb b/test/test_autopush.rb
index 57fbefa..6c6e05f 100644
--- a/test/test_autopush.rb
+++ b/test/test_autopush.rb
@@ -18,12 +18,10 @@ class TestAutopush < Test::Unit::TestCase
 
     @host = ENV["TEST_HOST"] || '127.0.0.1'
     @srv = Kgio::TCPServer.new(@host, 0)
-    assert_nothing_raised {
+    RUBY_PLATFORM =~ /linux/ and
       @srv.setsockopt(Socket::IPPROTO_TCP, TCP_CORK, 1)
-    } if RUBY_PLATFORM =~ /linux/
-    assert_nothing_raised {
+    RUBY_PLATFORM =~ /freebsd/ and
       @srv.setsockopt(Socket::IPPROTO_TCP, TCP_NOPUSH, 1)
-    } if RUBY_PLATFORM =~ /freebsd/
     @port = @srv.addr[1]
   end
 
@@ -35,7 +33,7 @@ class TestAutopush < Test::Unit::TestCase
     assert ! s.kgio_autopush?
     s.kgio_autopush = true
     assert s.kgio_autopush?
-    assert_nothing_raised { s.kgio_write 'asdf' }
+    s.kgio_write 'asdf'
     assert_equal :wait_readable, s.kgio_tryread(1)
     assert s.kgio_autopush?
     val = s.getsockopt(Socket::IPPROTO_TCP, opt).unpack('i')[0]
@@ -64,12 +62,10 @@ class TestAutopush < Test::Unit::TestCase
       lines = io.readlines
       assert lines.grep(/TCP_CORK/).empty?, lines.inspect
     else
-      assert_nothing_raised do
-        @wr = @srv.kgio_accept
-        t0 = Time.now
-        @wr.kgio_write "HI\n"
-        rc = @wr.kgio_tryread 666
-      end
+      @wr = @srv.kgio_accept
+      t0 = Time.now
+      @wr.kgio_write "HI\n"
+      rc = @wr.kgio_tryread 666
     end
     assert_equal "HI\n", @rd.kgio_read(3)
     diff = Time.now - t0
@@ -151,8 +147,8 @@ class TestAutopush < Test::Unit::TestCase
       lines = io.readlines
       assert_equal 2, lines.grep(/TCP_CORK/).size, lines.inspect
     end
-    assert_nothing_raised { @wr.close }
-    assert_nothing_raised { @rd.close }
+    @wr.close
+    @rd.close
 
     @wr = Kgio::TCPSocket.new(@host, @port)
     if defined?(Strace)
diff --git a/test/test_connect_fd_leak.rb b/test/test_connect_fd_leak.rb
index f6a8543..1dfc4cd 100644
--- a/test/test_connect_fd_leak.rb
+++ b/test/test_connect_fd_leak.rb
@@ -9,11 +9,9 @@ class TestConnectFDLeak < Test::Unit::TestCase
     nr = 0
     path = "/non/existent/path"
     assert(! File.exist?(path), "#{path} should not exist")
-    assert_nothing_raised do
-      begin
-        sock = Kgio::UNIXSocket.new(path)
-      rescue Errno::ENOENT
-      end while (nr += 1) < 10000
-    end
+    begin
+      sock = Kgio::UNIXSocket.new(path)
+    rescue Errno::ENOENT
+    end while (nr += 1) < 10000
   end
 end
diff --git a/test/test_poll.rb b/test/test_poll.rb
index df59354..40092cf 100644
--- a/test/test_poll.rb
+++ b/test/test_poll.rb
@@ -122,9 +122,7 @@ class TestPoll < Test::Unit::TestCase
       exit!(0)
     end
 
-    assert_nothing_raised do
-      empty += 1 until Kgio.poll(set.dup, 100)
-    end
+    empty += 1 until Kgio.poll(set.dup, 100)
     _, status = Process.waitpid2(pid)
     assert status.success?, status.inspect
     assert usr1 > 0, "usr1: #{usr1}"
diff --git a/test/test_singleton_read_write.rb b/test/test_singleton_read_write.rb
index 5abbf00..86d30a2 100644
--- a/test/test_singleton_read_write.rb
+++ b/test/test_singleton_read_write.rb
@@ -6,7 +6,7 @@ class TestSingletonReadWrite < Test::Unit::TestCase
 
   def test_unix_socketpair
     a, b = UNIXSocket.pair
-    assert_nothing_raised { Kgio.trywrite(a, "HELLO") }
+    Kgio.trywrite(a, "HELLO")
     buf = ""
     assert_equal "HELLO", Kgio.tryread(b, 5, buf)
     assert_equal "HELLO", buf
diff --git a/test/test_tryopen.rb b/test/test_tryopen.rb
index 5a8efb2..80b5de3 100644
--- a/test/test_tryopen.rb
+++ b/test/test_tryopen.rb
@@ -15,7 +15,7 @@ class TestTryopen < Test::Unit::TestCase
     assert_equal File.read(__FILE__), tmp.read
     assert_equal __FILE__, tmp.path
     assert_equal __FILE__, tmp.to_path
-    assert_nothing_raised { tmp.close }
+    tmp.close
   end
 
   def test_tryopen_ENOENT

Re: There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

2012/3/19 Eric Wong <normalperson@yhbt.net>:
> Edho Arief <edho@myconan.net> wrote:
>> Forcing definition of them enables unicorn to work (at least seems to
>> be working) but fails test (tried both Linux and BSD version):
>
> Wait, this fails on Linux and *BSD, too?  Which Ruby version and exactly
> which OSes did you use?
>

I mean neither BSD version:

static inline void kgio_autopush_read(VALUE io) { kgio_autopush_recv(io); }
static inline void kgio_autopush_write(VALUE io) { kgio_autopush_send(io); }

nor Linux:

static inline void kgio_autopush_read(VALUE io) { }
static inline void kgio_autopush_write(VALUE io) { }

passed the test on Solaris 10.

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> 2012/3/19 Eric Wong <normalperson@yhbt.net>:
> > Edho Arief <edho@myconan.net> wrote:
> >> Forcing definition of them enables unicorn to work (at least seems to
> >> be working) but fails test (tried both Linux and BSD version):
> >
> > Wait, this fails on Linux and *BSD, too?  Which Ruby version and exactly
> > which OSes did you use?
> 
> I mean neither BSD version:
> nor Linux:

OK, can you try with latest git?  Also which Ruby version are you using?
Did any of this work in previous version of kgio?

Re: There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

2012/3/19 Eric Wong <normalperson@yhbt.net>:
>
> OK, can you try with latest git?  Also which Ruby version are you using?
> Did any of this work in previous version of kgio?

I've never tested on other version since it was just my random weekend
experiment.

ruby 1.9.3p125 (2012-02-16 revision 34643) [i386-solaris2.10]

ruby -I lib:tmp/ext/ruby-1.9.3/ext/kgio
test/test_unix_server_read_client_write.rb
Run options:

# Running tests:

..............................^C

Finished tests in 489.996062s, 0.0000 tests/s, 0.0000 assertions/s.

0 tests, 0 assertions, 0 failures, 0 errors, 0 skips
Interrupted
make: *** [test/test_unix_server_read_client_write.rb] Error 1

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> 2012/3/19 Eric Wong <normalperson@yhbt.net>:
> >
> > OK, can you try with latest git?  Also which Ruby version are you using?
> > Did any of this work in previous version of kgio?
> 
> I've never tested on other version since it was just my random weekend
> experiment.
> 
> ruby 1.9.3p125 (2012-02-16 revision 34643) [i386-solaris2.10]

If you have time, maybe truss/dtruss can tell you what's wrong?  Do ruby
tests themselves pass?  I'm not sure if anybody on the Ruby core team
tests/maintains your platform on a regular basis.

Re: There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

2012/3/20 Eric Wong <normalperson@yhbt.net>:
> Edho Arief <edho@myconan.net> wrote:
>> 2012/3/19 Eric Wong <normalperson@yhbt.net>:
>> >
>> > OK, can you try with latest git?  Also which Ruby version are you using?
>> > Did any of this work in previous version of kgio?
>>
>> I've never tested on other version since it was just my random weekend
>> experiment.
>>
>> ruby 1.9.3p125 (2012-02-16 revision 34643) [i386-solaris2.10]
>
> If you have time, maybe truss/dtruss can tell you what's wrong?  Do ruby
> tests themselves pass?  I'm not sure if anybody on the Ruby core team
> tests/maintains your platform on a regular basis.

Failed one test for ruby: http://pastie.org/3634027

Here is the truss for kgio make test (relevant part, I hope):

ruby -I lib:tmp/ext/ruby-1.9.3/ext/kgio
test/test_unix_server_read_client_write.rb
write(1, " r u b y   - I   l i b :".., 84)      = 84
lwp_sigmask(SIG_SETMASK, 0x60004007, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFFF) = 0x60004007 [0x00000000]
vfork()                                         = 13190
lwp_sigmask(SIG_SETMASK, 0x60004007, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
schedctl()                                      = 0xFEF4F000
lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
Run options:

# Running tests:

........................waitid(P_ALL, 0, 0x080460E0, WEXITED|WTRAPPED)
(sleeping...)
......

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> 2012/3/20 Eric Wong <normalperson@yhbt.net>:
> > Edho Arief <edho@myconan.net> wrote:
> >> 2012/3/19 Eric Wong <normalperson@yhbt.net>:
> >> >
> >> > OK, can you try with latest git?  Also which Ruby version are you using?
> >> > Did any of this work in previous version of kgio?
> >>
> >> I've never tested on other version since it was just my random weekend
> >> experiment.
> >>
> >> ruby 1.9.3p125 (2012-02-16 revision 34643) [i386-solaris2.10]
> >
> > If you have time, maybe truss/dtruss can tell you what's wrong?  Do ruby
> > tests themselves pass?  I'm not sure if anybody on the Ruby core team
> > tests/maintains your platform on a regular basis.
> 
> Failed one test for ruby: http://pastie.org/3634027

Use "make check", that runs "make test-all" as well

> Here is the truss for kgio make test (relevant part, I hope):
> 
> ruby -I lib:tmp/ext/ruby-1.9.3/ext/kgio
> test/test_unix_server_read_client_write.rb
> write(1, " r u b y   - I   l i b :".., 84)      = 84
> lwp_sigmask(SIG_SETMASK, 0x60004007, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
> lwp_sigmask(SIG_SETMASK, 0xFFBFFEFF, 0x0000FFFF) = 0x60004007 [0x00000000]
> vfork()                                         = 13190
> lwp_sigmask(SIG_SETMASK, 0x60004007, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
> schedctl()                                      = 0xFEF4F000
> lwp_sigmask(SIG_SETMASK, 0x00000000, 0x00000000) = 0xFFBFFEFF [0x0000FFFF]
> Run options:
> 
> # Running tests:
> 
> ........................waitid(P_ALL, 0, 0x080460E0, WEXITED|WTRAPPED)
> (sleeping...)

You probably need to follow the fork() and trace children ("strace -f"
for Linux, maybe "truss -f"?  My memory escapes me).

Just wondering, does this patch help?

--- a/test/lib_read_write.rb
+++ b/test/lib_read_write.rb
@@ -281,7 +281,7 @@ module LibReadWriteTest
     elapsed = 0
     foo = nil
     t0 = Time.now
-    thr = Thread.new { sleep 1; @rd.readpartial(nr) }
+    thr = Thread.new { sleep 1; @rd.read(nr) }
     foo = @wr.kgio_write("HELLO")
     elapsed = Time.now - t0
 

Re: There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

On 2012-03-22 3:15, Eric Wong wrote:
>> Failed one test for ruby: http://pastie.org/3634027
>
> Use "make check", that runs "make test-all" as well
>

Exactly same result.

> You probably need to follow the fork() and trace children ("strace -f"
> for Linux, maybe "truss -f"?  My memory escapes me).
>
> Just wondering, does this patch help?
>

Yes it does: http://pastie.org/3645240

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> On 2012-03-22 3:15, Eric Wong wrote:
> >> Failed one test for ruby: http://pastie.org/3634027
> >
> > Use "make check", that runs "make test-all" as well
> 
> Exactly same result.

Err, maybe "make test-all" or somehow skip the segfaulting test to
go to the socket tests

> > You probably need to follow the fork() and trace children ("strace -f"
> > for Linux, maybe "truss -f"?  My memory escapes me).
> >
> > Just wondering, does this patch help?

OK, pushed this out (I thought I responded to this email earlier :x).

> Yes it does: http://pastie.org/3645240

  1) Error:
test_unix_socket_new_invalid(TestKgioUnixConnect):
Errno::ENOENT: No such file or directory - /tmp/kgio_unix20120322-20281-fg6a03
    test/test_unix_connect.rb:27:in `unlink'
    test/test_unix_connect.rb:27:in `teardown'

I'm not sure how this could happen... Race condition in the filesystem?
Is /tmp a tmpfs directory?  Can you try setting TMPDIR to point to a
different filesystem and see if you can reliably reproduce this?

Re: There's no TCP_NOPUSH in Solaris 10

[Edho Arief]

2012/3/22 Eric Wong <normalperson@yhbt.net>:
>
>  1) Error:
> test_unix_socket_new_invalid(TestKgioUnixConnect):
> Errno::ENOENT: No such file or directory - /tmp/kgio_unix20120322-20281-fg6a03
>    test/test_unix_connect.rb:27:in `unlink'
>    test/test_unix_connect.rb:27:in `teardown'
>
> I'm not sure how this could happen... Race condition in the filesystem?
> Is /tmp a tmpfs directory?  Can you try setting TMPDIR to point to a
> different filesystem and see if you can reliably reproduce this?

/tmp is tmpfs, yes. Changing to other mount point (ufs) didn't help.
It occurred 4 times out of 4 test runs (ie. every test).

Re: There's no TCP_NOPUSH in Solaris 10

[Eric Wong]

Edho Arief <edho@myconan.net> wrote:
> 2012/3/22 Eric Wong <normalperson@yhbt.net>:
> >
> >  1) Error:
> > test_unix_socket_new_invalid(TestKgioUnixConnect):
> > Errno::ENOENT: No such file or directory - /tmp/kgio_unix20120322-20281-fg6a03
> >    test/test_unix_connect.rb:27:in `unlink'
> >    test/test_unix_connect.rb:27:in `teardown'
> >
> > I'm not sure how this could happen... Race condition in the filesystem?
> > Is /tmp a tmpfs directory?  Can you try setting TMPDIR to point to a
> > different filesystem and see if you can reliably reproduce this?
> 
> /tmp is tmpfs, yes. Changing to other mount point (ufs) didn't help.
> It occurred 4 times out of 4 test runs (ie. every test).

Weird.  Do you think you can figure out a fix/workaround for this?  It's
not a concern unless it affects a Free Software platform people use, but
I will accept a patch if you can fix the issue.