From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-1.9 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, URIBL_BLOCKED shortcircuit=no autolearn=unavailable version=3.3.2 X-Original-To: yahns-public@yhbt.net Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 5C4CE1F627; Wed, 16 Jul 2014 19:57:39 +0000 (UTC) From: Eric Wong To: yahns-public@yhbt.net Cc: e@80x24.org, Eric Wong Subject: [PATCH 2/2] wbuf: avoid corrupted large responses with zero-copy sendfile Date: Wed, 16 Jul 2014 19:57:20 +0000 Message-Id: <1405540640-10669-3-git-send-email-e@80x24.org> X-Mailer: git-send-email 2.0.0.423.g7962671 In-Reply-To: <1405540640-10669-1-git-send-email-e@80x24.org> References: <1405540640-10669-1-git-send-email-e@80x24.org> List-Id: From: Eric Wong This bug is noticeable on a amd64 FreeBSD 9.2 VM, and possible under Linux, too. This happens as a zero-copy sendfile implementation means pages queued for transmission by the sendfile system call should not be modified at any point after the sendfile syscall is made. To prevent modification, we replace the temporary file with a new one. This has a similar effect as truncate and can still prevent a dirty flush in cases when a client consumes the response fast enough. This reverts the misguided ade89b5142bedbcf07f38aa062bfdbfcb8bc48d3 commit ("wbuf: hack to avoid response corruption on FreeBSD") Note: this bug was finally fixed because I finally noticed this flaw in a different (non-Ruby, non-HTTP) server of mine. --- lib/yahns/wbuf.rb | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-) diff --git a/lib/yahns/wbuf.rb b/lib/yahns/wbuf.rb index 21bccce..991557e 100644 --- a/lib/yahns/wbuf.rb +++ b/lib/yahns/wbuf.rb @@ -30,16 +30,9 @@ require_relative 'wbuf_common' class Yahns::Wbuf # :nodoc: include Yahns::WbufCommon - # TODO: Figure out why this hack is needed to pass output buffering tests. - # It could be a bug in our code, Ruby, the sendfile gem, or FreeBSD itself. - # Tested on FreeBSD fbsd 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898 - # We are able to use bypass mode on Linux to reduce buffering in some - # cases. Without bypass mode, we must always finish writing the entire - # response completely before sending more data to the client. - bypass_ok = RUBY_PLATFORM =~ /linux/ - def initialize(body, persist, tmpdir) @tmpio = Yahns::TmpIO.new(tmpdir) + @tmpdir = tmpdir @sf_offset = @sf_count = 0 @wbuf_persist = persist # whether or not we keep the connection alive @body = body @@ -71,16 +64,12 @@ class Yahns::Wbuf # :nodoc: # we're all caught up, try to prevent dirty data from getting flushed # to disk if we can help it. - @tmpio.truncate(@sf_offset = 0) - @tmpio.rewind + @tmpio.close + @sf_offset = 0 + @tmpio = Yahns::TmpIO.new(@tmpdir) @bypass = true nil - end if bypass_ok - - def wbuf_write(client, buf) - @sf_count += @tmpio.write(buf) - :wait_writable - end unless bypass_ok + end # called by last wbuf_flush def wbuf_close(client) -- EW