* [PATCH] proxy_http_response: reduce memory pressure from larger headers
@ 2016-12-31 4:46 5% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2016-12-31 4:46 UTC (permalink / raw)
To: yahns-public
We try to cleanup after ourselves if possible so future mallocs
can find memory more easily. We do not need to nil the object
slot, just String#clear since we're exiting the function soon,
anyways.
Followup to commit bb774680aae0a827f887761b18da304aa94111cc
("use String#clear for short-lived buffers we create")
---
lib/yahns/proxy_http_response.rb | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/lib/yahns/proxy_http_response.rb b/lib/yahns/proxy_http_response.rb
index 1776291..7df2834 100644
--- a/lib/yahns/proxy_http_response.rb
+++ b/lib/yahns/proxy_http_response.rb
@@ -132,7 +132,9 @@ def proxy_res_headers(res, req_res)
# send the headers
case rv = kgio_syssend(res, flags)
- when nil then break # all done, likely
+ when nil # all done, likely
+ res.clear
+ break
when String # partial write, highly unlikely
flags = MSG_DONTWAIT
res = rv # hope the skb grows
--
EW
^ permalink raw reply related [relevance 5%]
* [ANN] yahns 1.12.0 -_- sleepy app server for Ruby
@ 2016-02-14 22:37 4% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2016-02-14 22:37 UTC (permalink / raw)
To: ruby-talk, yahns-public
A Free Software, multi-threaded, non-blocking network
application server designed for low _idle_ power consumption.
It is primarily optimized for applications with occasional users
which see little or no traffic. yahns currently hosts Rack/HTTP
applications, but may eventually support other application
types. Unlike some existing servers, yahns is extremely
sensitive to fatal bugs in the applications it hosts.
Changes:
yahns 1.12.0 - TLS fixes and more!
Most notably, serving static files over HTTPS did not work
before this release with the "sendfile" gem installed. The
yahns_config(5) manpage is also updated with an example for
using OpenSSL::SSL::SSLContext objects. Users of
Rack::Request#scheme and env['rack.url_scheme'] should see
"https" properly set for HTTPS connections.
There's also a bunch of internal tweaks like taking advantage of
the file-level frozen_string_literal: directive in 2.3 and
explicitly clearing short-lived string buffers
TLS support is still in its early stages, but I'm experimenting
with Let's Encrypt (via getssl[1]) and hosting https://YHBT.net/
on it.
For now, I suggest using a separate yahns instance (with a
different master process) to avoid any potential data leaks
between HTTPS and HTTP instances. In the future, it may be
possible to isolate HTTPS from HTTP at the worker process level.
Supporting GnuTLS (alongside OpenSSL) may be in our future, too.
To paraphrase the warning in http://www.postfix.org/TLS_README.html
(which was written before Heartbleed):
WARNING
By turning on TLS support in yahns, you not only get the
ability to encrypt traffic and to authenticate remote
clients. You also turn on thousands and thousands of
lines of OpenSSL library code. Assuming that OpenSSL is
written as carefully as Eric's own code, every 1000 lines
introduce one additional bug into yahns.
I'm not nearly as careful with yahns as Wietse is with postfix,
either.
20 changes since v1.11.0:
README: updates for kqueue
add .gitattributes for Ruby method detection
nodoc internals
enable frozen_string_literal for Ruby 2.3+
copyright updates for 2016
extras/exec_cgi: fix frozen string error on slow responses
avoid StringIO#binmode for the next few years
use String#clear for short-lived buffers we create
gemspec: make rack a development dependency
build: install-gem forced to "--local" domain
acceptor: all subclasses of TCPServer use TCP_INFO
properly emulate sendfile for OpenSSL sockets
avoid race conditions in OpenSSL::SSL::SSLContext#setup
set HTTPS and rack.url_scheme in Rack env as appropriate
proxy_pass: pass X-Forwarded-Proto through
doc: switch to perlpod (from pandoc-flavored Markdown)
doc: trim down documentation slightly
doc: document ssl_ctx for "listen" directive
doc: various doc and linkification improvements
http_context: reduce constant lookup + bytecode
[1] git clone https://github.com/srvrco/getssl.git
Please note the disclaimer:
yahns is extremely sensitive to fatal bugs in the apps it hosts. There
is no (and never will be) any built-in "watchdog"-type feature to kill
stuck processes/threads. Each yahns process may be handling thousands
of clients; unexpectedly killing the process will abort _all_ of those
connections. Lives may be lost!
yahns hackers are not responsible for your application/library bugs.
Use an application server which is tolerant of buggy applications
if you cannot be bothered to fix all your fatal bugs.
* git clone git://yhbt.net/yahns
* http://yahns.yhbt.net/README
* http://yahns.yhbt.net/NEWS.atom.xml
* we only accept plain-text email yahns-public@yhbt.net
* and archive all the mail we receive: http://yhbt.net/yahns-public/
* nntp://news.public-inbox.org/inbox.comp.lang.ruby.yahns
^ permalink raw reply [relevance 4%]
* [PATCH] use String#clear for short-lived buffers we create
@ 2016-01-04 12:07 7% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2016-01-04 12:07 UTC (permalink / raw)
To: yahns-public
This should reduce memory pressure slightly as we can
have finer-grained control of memory usage for buffers which
can be several kilobytes large.
It is not safe to do this for output buffers we get from the
application, as they may reuse that memory themselves.
---
lib/yahns/http_response.rb | 6 ++++--
lib/yahns/max_body/wrapper.rb | 2 ++
lib/yahns/tee_input.rb | 1 +
3 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/lib/yahns/http_response.rb b/lib/yahns/http_response.rb
index 292fab7..4b36db2 100644
--- a/lib/yahns/http_response.rb
+++ b/lib/yahns/http_response.rb
@@ -162,11 +162,13 @@ def http_response_write(status, headers, body)
: "Connection: close\r\n\r\n".freeze)
case rv = kgio_syssend(buf, flags)
when nil # all done, likely
+ buf.clear
+ buf = nil # recycle any memory we used ASAP
break
when String
flags = MSG_DONTWAIT
- buf = rv # hope the skb grows
- when :wait_writable, :wait_readable
+ buf = rv # unlikely, hope the skb grows
+ when :wait_writable, :wait_readable # unlikely
if k.output_buffering
alive = hijack ? hijack : alive
rv = response_header_blocked(rv, buf, body, alive, offset, count)
diff --git a/lib/yahns/max_body/wrapper.rb b/lib/yahns/max_body/wrapper.rb
index b75c3b6..b6ca1a3 100644
--- a/lib/yahns/max_body/wrapper.rb
+++ b/lib/yahns/max_body/wrapper.rb
@@ -48,6 +48,7 @@ def gets
if tmp = checked_read(16384)
@rbuf << tmp
+ tmp.clear
elsif @rbuf.empty? # EOF
return nil
else # EOF, return whatever is left
@@ -69,6 +70,7 @@ def read_all
while checked_read(16384, tmp)
rv << tmp
end
+ tmp.clear
rv
end
end
diff --git a/lib/yahns/tee_input.rb b/lib/yahns/tee_input.rb
index 55b1604..93ec148 100644
--- a/lib/yahns/tee_input.rb
+++ b/lib/yahns/tee_input.rb
@@ -100,6 +100,7 @@ def consume!
junk = ''.dup
rsize = __rsize
nil while read(rsize, junk)
+ junk.clear
end
def tee(buffer)
--
EW
^ permalink raw reply related [relevance 7%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2016-01-04 12:07 7% [PATCH] use String#clear for short-lived buffers we create Eric Wong
2016-02-14 22:37 4% [ANN] yahns 1.12.0 -_- sleepy app server for Ruby Eric Wong
2016-12-31 4:46 5% [PATCH] proxy_http_response: reduce memory pressure from larger headers Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/yahns.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).