* [WIP] proxy_pass: experiment to disable buffer sharing
@ 2015-04-14 21:31 Eric Wong
2015-04-17 2:21 ` Eric Wong
0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2015-04-14 21:31 UTC (permalink / raw)
To: yahns-public; +Cc: Eric Wong
I suspect we're inadvertantly sharing buffers somewhere and
leading to segfaults down-the-line. The core dumps I've gotten
aren't even close to any of the new proxy code, so it looks like
the bug is due to mis-sharing of a thread-local buffer which
we didn't detached.
Will run this for a few days to see if there's still segfaults
with the proxy code. Multithreading is tricky...
Not going to push this, but I'm fairly convinced the issue
somewhere related and can be fixed without thrashing allocations.
---
lib/yahns/proxy_http_response.rb | 8 ++++----
lib/yahns/proxy_pass.rb | 12 ++----------
2 files changed, 6 insertions(+), 14 deletions(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index af8d8cc..632dfe6 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -88,7 +88,7 @@ module Yahns::HttpResponse # :nodoc:
break # keep buffering as much as possible
end while true
- rbuf = Thread.current[:yahns_rbuf]
+ rbuf = ''
tip = tip.empty? ? [] : [ tip ]
if have_body
@@ -123,7 +123,7 @@ module Yahns::HttpResponse # :nodoc:
buf = tmp
req_res.proxy_trailers = [ buf, tlr = [] ]
- rbuf = Thread.current[:yahns_rbuf] = ''
+ rbuf = ''
until kcar.trailers(tlr, buf)
case rv = req_res.kgio_tryread(0x2000, rbuf)
when String
@@ -165,7 +165,7 @@ module Yahns::HttpResponse # :nodoc:
end
def proxy_response_finish(kcar, wbuf, req_res)
- rbuf = Thread.current[:yahns_rbuf]
+ rbuf = ''
if len = kcar.body_bytes_left
case tmp = req_res.kgio_tryread(0x2000, rbuf)
@@ -193,7 +193,7 @@ module Yahns::HttpResponse # :nodoc:
return :wait_readable # self remains in :ignore, wait on upstream
end until kcar.body_eof?
req_res.proxy_trailers = [ tmp, [] ] # onto trailers!
- rbuf = Thread.current[:yahns_rbuf] = ''
+ rbuf = ''
end
buf, tlr = *req_res.proxy_trailers
diff --git a/lib/yahns/proxy_pass.rb b/lib/yahns/proxy_pass.rb
index e86e9c9..f6021d4 100644
--- a/lib/yahns/proxy_pass.rb
+++ b/lib/yahns/proxy_pass.rb
@@ -21,17 +21,11 @@ class Yahns::ProxyPass # :nodoc:
Thread.current[:yahns_queue].queue_add(self, Yahns::Queue::QEV_WR)
end
- # we must reinitialize the thread-local rbuf if it may get beyond the
- # current thread
- def detach_rbuf!
- Thread.current[:yahns_rbuf] = ''
- end
-
def yahns_step # yahns event loop entry point
c = @yahns_client
case req = @rrstate
when Kcar::Parser # reading response...
- buf = Thread.current[:yahns_rbuf]
+ buf = ''
case resbuf = @resbuf # where are we at the response?
when nil # common case, catch the response header in a single read
@@ -41,8 +35,8 @@ class Yahns::ProxyPass # :nodoc:
if res = req.headers(@hdr = [], rv)
return c.proxy_response_start(res, rv, req, self)
else # ugh, big headers or tricked response
- buf = detach_rbuf!
@resbuf = rv
+ buf = ''
end
# continue looping in middle "case @resbuf" loop
when :wait_readable
@@ -109,7 +103,6 @@ class Yahns::ProxyPass # :nodoc:
when Array
vec = rv # partial write, retry in case loop
when :wait_writable
- detach_rbuf!
req[0] = vec
return :wait_writable
when nil
@@ -128,7 +121,6 @@ class Yahns::ProxyPass # :nodoc:
when String
buf = rv # partial write, retry in case loop
when :wait_writable
- detach_rbuf!
req[0] = buf
return :wait_writable
when nil
--
EW
^ permalink raw reply related [flat|nested] 2+ messages in thread
* Re: [WIP] proxy_pass: experiment to disable buffer sharing
2015-04-14 21:31 [WIP] proxy_pass: experiment to disable buffer sharing Eric Wong
@ 2015-04-17 2:21 ` Eric Wong
0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2015-04-17 2:21 UTC (permalink / raw)
To: yahns-public
Eric Wong <e@80x24.org> wrote:
> Will run this for a few days to see if there's still segfaults
> with the proxy code. Multithreading is tricky...
Nope. I spotted a definite problem elsewhere, and it's definitely
tricky with the way the full rack.request hijack API works.
I think switching to response-only hijacking will solve the problem,
though.
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2015-04-17 2:21 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-14 21:31 [WIP] proxy_pass: experiment to disable buffer sharing Eric Wong
2015-04-17 2:21 ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/yahns.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).