| Date | Commit message (Collapse) |
|---|
|
This release fixes warnings for users combining Rack::Deflater
with HTTPS support:
https://yhbt.net/yahns-public/20180714005630.11812-1-e@80x24.org/
yahns-rackup(1) users may not specify "-O listen=inherit"
when spawning from systemd. The "listen" directive may
be omitted entirely from the yahns-config(5) files.
A few other cleanups and nothing interesting, otherwise.
20 yawn-worthy changes since v1.15.0 (2017-03-23):
config: more descriptive variable name
proxy_pass: comment explaining what rack.hijack calls
USR2 upgrades may use Process.spawn for vfork
avoid Thread#[] and Thread#[]= across threads
gemspec: declare Ruby 2.0+ dependency, here
test_bin: SO_KEEPALIVE value only needs to be true
rackup_handler: remove unnecessary branch
test_bin: use RbConfig.ruby for non-standard names
allow omitting specifying socket name when inheriting listen socket
server: fix incomplete comment about SSLContext#setup
test_rack_env: additional test for Rack environment
test: allow setting TAIL env to watch error logs
fix some unused variables
tests: thread-safety fixes
test/server_helper: describe reason for termination
test/test_ssl: set SSLContext#security_level=0
doc: https:// URLs instead of git://
http_client: clear backtrace on "wrong version number" in OpenSSL
openssl_client: do not attempt writes after SystemCallError
use IO#pread if available in Ruby 2.5
*ZZZZZZ*
|
|
In the future, this will allow sharing open files across
different clients when serving static files. For now, it
saves us one syscall.
|
|
Whenever @ssl.write_nonblock fails due to ECONNRESET/EPIPE in
Rack::Deflater#each; Rack::Deflater#each will still attempt to
write again in the "ensure" statement via Zlib::GzipWriter#close.
This causes SSL_write to complain of "bad length" or
"bad write retry" errors. Now, re-raise any SystemCallError
we caught from previous write_nonblock calls to prevent
calls to SSL_write which would trigger such an exception.
|
|
Errors which are the fault of the client should not generate
backtraces which can lead to dinky servers running out-of-space.
|
|
Might as well eat our own dogfood w.r.t. yahns TLS support
|
|
This is apparently needed to pass tests with a newer version of
OpenSSL found in Debian 9
|
|
A test failure was causing SIGQUIT to be delivered before
the forked process had a chance to hit trap(:QUIT).
|
|
We can't require 'proxy_pass' in both a parent and forked child,
so require it up front (as kcar will become a hard dependency
in place of unicorn).
Then, rely on GTL (global test lock) to synchronize around fork
since the VM may not always be able to protect that.
However, there's no need to synchronize around
spawn/system/`backtick`, as the VM should always be using those
in a thread-safe way (via vfork).
|
|
Current (tested r60757) ruby trunk warns in a few more places than
2.4.x did, so clean them up.
|
|
Setting TAIL=1 will automatically use the portable "tail -f".
This can be helpful in diagnosing failures during development.
GNU tail users may set TAIL="tail -F" (or "gtail -F")
to use the "-F" ("--follow=name") option to track changes
across SIGUSR1 log reopening testing.
|
|
Since there'll be some changes to accomodate the new parser,
ensure we prepare the Rack environment correctly.
|
|
Oops. Occasionally my brain experiences packet loss :x
|
|
Since the common case is still to run a single app inside yahns,
we can simplify setup a bit for systemd (and like) users by
allowing them to omit the "listen" directive when they are
running a single app in yahns.
|
|
It's possible to have "ruby" executables by other names
(e.g. "ruby24"), so use a supported API for finding our
executable.
This feature was added in Ruby 1.9.2, so it's safe to use
as we've always been 1.9.3+ (and nowadays 2.0+)
|
|
No point in optimizing for the single listener case in
setup code.
|
|
On FreeBSD 10.3 (and presumably other *BSD TCP stacks, the value
of SO_KEEPALIVE returned by getsockopt is 8, even when set to
'1' via setsockopt. Relax the test to only ensure the boolean
value is interpreted as "true".
Verified independently of Ruby using the following:
--------8<---------
#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
static int err(const char *msg)
{
perror(msg);
return 1;
}
int main(void)
{
int sv[2];
int set = 1;
int got;
socklen_t len = (socklen_t) sizeof(int);
int rc;
rc = socketpair(PF_LOCAL, SOCK_STREAM, 0, sv);
if (rc)
return err("socketpair failed");
rc = setsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &set, len);
if (rc)
return err("setsockopt failed");
rc = getsockopt(sv[0], SOL_SOCKET, SO_KEEPALIVE, &got, &len);
if (rc)
return err("getsockopt failed");
printf("got: %d\n", got);
return 0;
}
|
|
We've been 2.0+ for a few years, actually.
|
|
Support for it may be removed in future versions of Ruby(*),
and we actually do not need to waste time looping when
a instance variable will do.
(*) https://bugs.ruby-lang.org/issues/13245
|
|
Remove false statements about close-on-exec being cleared in the
parent process (verified via strace) and refactor the code to
take full advantage of options for Process.spawn and Process.exec.
This avoids unnecessary CoW traffic for the common case where
before_exec is unset.
|
|
"call" is a generic name and may not obvious to somebody
new to the code.
|
|
That object is not a Rack app, so lets avoid confusing
ourselves.
|
|
Minor changes to reduce allocations and simplify our code
and dependencies. Nothing particularly interesting unless
you're the type of person who appreciates brake pedals
in with holes drilled in them for weight reduction.
11 changes since 1.14.1 (2016-12-14):
stream_input: avoid allocation for common #read case
proxy_pass: add a note about the instability of this
tee_input: simplify conditional for writing to temporary file
proxy_http_response: reduce memory pressure from larger headers
http_response: make response headers eligible for GC, sooner
update more referenced URLs to be HTTPS
chunk_body: nodoc this internal class
Revert "use olddoc 1.1.0 for generating NEWS + NEWS.atom.xml"
gemspec: stop advertising "private" email address
doc: design_notes: we do not use EPOLLEXCLUSIVE
README: update with disclaimer about subscription
Ovt punatrf pbzvat...
|
|
Also, give the Atom feed a little more visibility.
|
|
And clarify that we only have one thread by default.
Since EPOLLEXCLUSIVE seems to have gotten some more press, I
guess we should emphasize our design does not rely on it.
|
|
Anonymity is a job for the users sending the mail; lets not hold
ourselves unnecessarily accountable for anonymizing or hiding
messages.
|
|
This reverts commit bcf0f0efb3173b18957ddba2af6592219a2d72a3.
The dependency on olddoc makes it slightly more difficult for
users to package their own and distribute RubyGems. We will
also not host HTML RDoc files since we have no internal API
to support.
|
|
Although we do not publish RDoc-generated documentation on our
website, other places may, an people use ri(1) locally.
|
|
HTTPS provides some security against spying and MitM attacks,
so refer users to HTTPS sites, instead.
|
|
Iterating through the response body can be time consuming and
allocate memory, so nil out the headers object when we're done
with it to give Ruby a chance to recover the memory.
|
|
We try to cleanup after ourselves if possible so future mallocs
can find memory more easily. We do not need to nil the object
slot, just String#clear since we're exiting the function soon,
anyways.
Followup to commit bb774680aae0a827f887761b18da304aa94111cc
("use String#clear for short-lived buffers we create")
|
|
It's rare to have an empty string returned by `read` since it
requires a length argument of zero (careless) or nil (danger!
OOM!). Lets not try to optimize away a method dispatch in this
case by checking .size (optimized instruction in YARV).
Ruby IO#write already treats writing an empty string as a noop,
so no syscall is saved.
n.b. `gets` can return an empty string, too, but that's also
dangerous w.r.t. memory usage on untrusted input.
|
|
In case somebody stumbles upon it...
|
|
When entering StreamInput#read, it is common for @rbuf to
be drained and empty, meaning a simple String#clear can
be used in place of a more complex String#slice! String#replace
combo on an empty @rbuf.
While we're at it, we'll avoid an extra instructions for calling
`@rbuf.size` by storing the size in a local.
|
|
This release avoids confusing HTTP/1.1 clients with a
"Transfer-Encoding: chunked" header on bodyless responses.
The header was causing "curl -T" to wait indefinitely for
a response body after the server sent a 204.
This regression was introduced with autochunk introduced
with yahns 1.13(*).
rack.hijack was also broken for ancient "HTTP/0.9" requests,
and now fixed.
4 changes since 1.14.0:
queue_*: fix outdated comments
http_response: support rack.hijack on HTTP/0.9 responses
response: do not set chunked header on bodyless responses
proxy_pass: do not chunk on bodyless upstream responses
(*) https://yhbt.net/yahns-public/20160805-yahns-1.13.0-released@lucky13/
|
|
As with the previous commit
("response: do not set chunked header on bodyless responses"),
blindly setting "Transfer-Encoding: chunked" is wrong and
confuses "curl -T" on 204 responses, at least.
|
|
Setting "Transfer-Encoding: chunked" on responses will confuse
clients which see a 204 response and do not expect a body.
This follows Rack::Chunked behavior, as yahns should function
without Rack::Chunked middleware.
This regression appeared in yahns v1.13.0 (2016-08-05)
|
|
We still need to iterate through all response headers to support
response-only Rack hijacking. Previously, we only supported
full hijacking on so-called "HTTP/0.9" clients.
n.b. This diff will be easier to read with the
-b/--ignore-space-change option of git-diff(1) or GNU diff(1)
|
|
These comments have been outdated since 2013 with commit
cd84e2ccbdf2 ("ensure we stop all threads at exit")
|
|
There's minor feature removals for undocumented and
undefined features and behavior which are unlikely to
affect anybody unless they serve HTTPS.
Our website is now self-hosted with HTTPS support (HTTP remains
supported for legacy systems):
https://yhbt.net/yahns/README
See git history at git://yhbt.net/yahns.git for full details.
openssl_client: avoid undefined SSL_write behavior
move website to https://yhbt.net/yahns/
stream_file: remove #to_io support from responses
response: only stream "file" responses on known length
response: fixup compile error
req_res: do not send 502 on catchall error if response buffered
|
|
If we already started writing a response, we cannot be sending a
502 to inform a user a connection has failed. This should
prevent errors from the OpenSSL layer about mismatched buffers
due to the combination of slow clients and upstreams prematurely
aborting.
|
|
Introduced in commit 08cea89483e90c79daa2c8efe70da6bdeba71147
('response: only stream "file" responses on known length')
|
|
This means we need to check Content-Length and use it
properly (unless Content-Range is set).
|
|
This is not a part of any Rack specification, and can lead to
interesting bugs with wrapped classes for gzip and TLS. AFAIK,
.to_io was meant to support IO#wait_*able and IO.select methods,
not to actually perform read/writes on the return value.
|
|
HTTPS adds some level of privacy protection and helps marketing
(because we care soooo much about good marketing! :P).
Performance-wise, this reduces subjectAltName bloat when
negotiating connections and will also speed up occasional
certificate renewals when/if we drop the old name.
Also, not occupying the document root of a domain will make it
easier to add alternative site locations in the future, because
centralization sucks and I don't like the idea of anybody paying
ICANN or similar entities for domain names.
|
|
Sometimes apps may trigger zero-byte chunks in the response
body for whatever reason. We should maintain consistent
behavior with the rest of kgio; and Ruby OpenSSL::SSL::SSLSocket
should maintain consistent behavior with the core IO class:
https://bugs.ruby-lang.org/issues/12660
|
|
And probably a billion new regressions!
yahns now allows users to skip the Rack::Head, Rack::Chunked and
Rack::ContentLength middlewares to ease migrating from/to other
real-world Rack HTTP servers. Most notably, our chunked
encoding implementation is a bit faster than Rack::Chunked by
taking advantage of the writev(2) syscall:
https://yhbt.net/yahns-public/20160803031906.14553-4-e@80x24.org/
There's also rack 2.x fixes in the test case and extras/ section
(these incompatibilities did not affect existing users unless
they use the wonky extras/ section).
There's also some graceful shutdown fixes, the process title is
now changed to display the number of live FDs.
Of course, there's the usual round of documentation improvements
which are systemd and OpenSSL setup-related this time around.
However, the majority of changes (proxy_*, wbuf_lite), affect
currently-unadvertised functionality which is subject to removal
or incompatible config changes. However, they are used to serve
our mailing list archives at:
https://yhbt.net/yahns-public/
49 changes since yahns 1.12.5:
proxy_pass: simplify writing request bodies upstream
proxy_pass: hoist out proxy_res_headers method
proxy_pass: simplify proxy_http_response
proxy_pass: split out body and trailer reading in response
proxy_pass: trim down proxy_response_finish, too
proxy_pass: split out req_res into a separate file
proxy_pass: fix resumes after complete buffering is unblocked
proxy_pass: X-Forwarded-For appends to existing list
proxy_pass: pass entire object to proxy_http_response
proxy_pass: support "proxy_buffering: false"
proxy_pass: remove unnecessary rescue
req_res: store proxy_pass object here, instead
proxy_pass: redo "proxy_buffering: false"
wbuf: remove needless "busy" parameter
Merge branch 'maint'
extras/try_gzip_static: do not show backtrace on syscall errors
wbuf: remove tmpdir parameter
wbuf_lite: fix write retries for OpenSSL sockets
test_proxy_pass_no_buffering: fix racy test
queue_*: check for closed IO objects
cleanup graceful shutdown handling
proxy_pass: more descriptive error messages
proxy_pass: fix HTTP/1.0 backends on EOF w/o buffering
wbuf_common: reset offset counter when done
extras/try_gzip_static: resolve symlinks
test_ssl: remove unnecessary priv_key DH parameter
openssl_client: wrap shutdown for graceful termination
proxy_pass: keep trailer buffer on blocked client writes
proxy_pass: avoid TOCTTOU race when unbuffering, too
proxy_pass: avoid accessing logger in env after hijacking
proxy_pass: avoid stuck responses in "proxy_buffering: false"
extras: include status messages in responses
update init and add systemd examples
test_proxy_pass_no_buffering: exclude rb/ru files, too
wbuf_lite: use StringIO instead of TmpIO
wbuf_lite: truncate StringIO when done
wbuf_lite: prevent clobbering responses
wbuf_lite: unify EOF error handling
wbuf_lite: reset sf_offset/sf_count consistently
wbuf_lite: clear @busy flag when re-arming
http_response: drop bodies for non-compliant responses
fix rack 2.x compatibility bugs
doc: add session cache usage to OpenSSL example
test: skip some buffering tests on non-default values
response: drop clients after HTTP responses of unknown length
response: reduce stack overhead for parameter passing
response: support auto-chunking for HTTP/1.1
Revert "document Rack::Chunked/ContentLength semi-requirements"
extras/exec_cgi: fix for HTTPoxy vulnerability
|
|
Bad clients may set the Proxy: header in the response and
cause any CGI programs we execute to use the value of that
header as the HTTP proxy. This affects folks calling code
which respects the HTTP_PROXY environment variable in CGI
programs.
ref: https://httpoxy.org/
|
|
Actually, I guess I misread, rack (starting at) 1.0 stopped
requiring Content-Length/Chunked headers but I never noticed.
Oh well.
This reverts commit 4968041a7e1ff90b920704f50fccb9e7968d0d99.
|
|
We might as well do it since puma and thin both do(*),
and we can still do writev for now to get some speedups
by avoiding Rack::Chunked overhead.
timing runs of "curl --no-buffer http://127.0.0.1:9292/ >/dev/null"
results in a best case drop from ~260ms to ~205ms on one VM
by disabling Rack::Chunked in the below config.ru
$ ruby -I lib bin/yahns-rackup -E none config.ru
==> config.ru <==
class Body
STR = ' ' * 1024 * 16
def each
10000.times { yield STR }
end
end
use Rack::Chunked if ENV['RACK_CHUNKED']
run(lambda do |env|
[ 200, [ %w(Content-Type text/plain) ], Body.new ]
end)
(*) they can do Content-Length, but I don't think it's
worth the effort at the server level.
|
|
This makes it easier to add more parameters to
http_response_write and simplifies current callers.
|