| Date | Commit message (Collapse) |
|---|
|
We should warn gracefully when we hit IPv7+ or whatever...
|
|
While no user is likely to have enough listeners to trigger an
overflow, just use ruby_xrealloc2 to be safe since it's already
provided by Ruby (and AFAIK reallocarray(3) isn't standardized).
|
|
Given the history of Ruby removing public C APIs, get ahead
of potential incompatibilities by switching to an externally
maintained unordered hash table.
khashl is a newer, more memory-efficient evolution of the
khash hash table adopted by the git SCM and this will be
my first (and likely not last) time using it in a public
codebase.
|
|
OBJ_FREEZE before calling rb_hash_aset was actually preventing
deduplication since Ruby 2.6. This introduces a performance
regression for Ruby 2.5 users, but I expect the majority of
users are on newer versions (I'm on 2.7, which is still ancient).
|
|
I just debugged an issue with our system, I was witnessing the
number of file descriptor in our process grow at an alarming rate
which I mapped to our use of raindrops to report utilisation.
For various reasons we don’t call raindrops from a Rack middleware
but have one process that monitor the socket continuously, and
share that data with the workers.
Since we call tcp_listener_stats every seconds in a process
that doesn't do much else, GC very rarely triggers if at all
which cause `InetDiagSocket` instances to accumulate very
quickly.
Each of those instances holds a file descriptor.
Looking at the raindrops implementation it seems to assume
the GC will take care of regularly closing these sockets, but
I think it’s a bit too bold of an assumption.
[ew: don't close user-passed sockets on exception]
Acked-by: Eric Wong <e@80x24.org>
|
|
alloca makes stack usage unpredictable and life difficult for
static analysis tools and compilers. The 46 bytes of
INET6_ADDRSTRLEN is fine to keep on stack, but page size can be
several MB large in some architectures (but typically 4K on
common architectures).
Thus we handle page size-ed allocations via `rb_str_tmp_new'.
`rb_str_tmp_new' has been in public Ruby headers since the 1.9
days and used by the core `zlib', `digest', and `zlib'
extensions, so it should be safe to use (and `rb_str_resize' is
used in many more C extensions).
|
|
Calling `#to_io' is only necessary when we're handling an argument
from user code where the user could pass a non-IO object.
`#to_io' calls are a waste of time when we create the IO object
ourselves (in `Raindrops::InetDiagSock.new').
This allows us to define the `my_fileno' macro for Ruby 3.1+
users to call the new `rb_io_descriptor' function directly
without an extra C stack frame. This also allows us to get
rid of nesting CPP directives inside C functions which (IMHO)
improves readability.
Furthermore, any necessary #to_io calls using `rb_convert_type'
can be replaced with `rb_io_get_io' to decrease code size.
`rb_io_get_io' has been in ruby/io.h since Ruby 1.9.2 and
there's no expectation that it'd be deprecated since it only
deals with opaque `VALUE' types.
|
|
SO_REUSEPORT was introduced in 2013, see https://lwn.net/Articles/542629/.
However, the current logic for calculating socket queue backlog stats
is from 2011, predating the advent of SO_REUSEPORT.
The current strategy was thus written before the notion that the mapping of
INET_ADDR:socket could potentially be 1:N instead of always 1:1.
This causes raindrops to provide invalid socket backlog values when
SO_REUSEPORT is used, which Unicorn supports since 2013. The current behaviour
will set the queue metric to the queue depth of the last inet_diag_msg
it processes matching the INET_ADDR. In practice, this will result in
the backlog being off by a factor N, assuming relatively even
distribution of requests to sockets, which the kernel should guarantee
through consistent hashing.
The fix here is to accumulate the socket queue depth as we iterate
over the the socket diagnostics, rather than reset it each time.
I have a provided a test, but it only checks the queues, not the
accept metrics, as those are not affected by this bug, and it is
not possible to know which of the listeners will be dispatched the
request by the kernel, and thus which should call accept.
|
|
With GC.compact in Ruby 3.x, Ruby-defined constants need to be
explicitly marked to prevent movement.
Link: https://yhbt.net/kgio-public/CAAvYYt5Z5f2rMuXO5DMpR1-6uRvu_gXKDvqcyoZ+oNcLiTH39g@mail.gmail.com/T/
|
|
The expiration for bogomips.org is coming up and I'm not
keen on paying or supporting extortionists.
Not wanting to be beholden to ICANN or any powerful
organizations, .onion sites are available to Tor users:
http://raindrops.ou63pmih66umazou.onion/
http://ou63pmih66umazou.onion/raindrops.git/
http://ou63pmih66umazou.onion/raindrops-public/
(the demo is not yet available via .onion, yet, could be a bit)
|
|
Drop vestigial Ruby 1.8 bits, and start using
rb_thread_call_without_gvl on modern Rubies.
|
|
getnameinfo is overkill for NI_NUMERICHOST + NI_NUMERICSERV usage,
and has a more complex and error-prone API than using inet_ntop
and snprintf.
|
|
Defining the "Raindrops" class explicitly helps RDoc find
subclasses for documentation, and ought to reduce the binary
size slightly due to the removal of rb_intern calls.
Furthermore, use "Socket" to ensure the base class for
Raindrops::InetDiagSocket is documented properly in RDoc.
|
|
This helps the compiler detect bugs and
quiets down a -Wsuggest-attribute=format warning
|
|
With invalid addresses specified which give no currently-bound
address, we must avoid leaving placeholders ('true' objects)
in our results.
Clean up some shadowing "cur" while we're at it.
|
|
POSIX and glibc 2.2+ declare the hostlen and servlen
args of getnameinfo(3) to be socklen_t, not size_t,
so favor socklen_t for those calculations.
While we're at it, nlmsg_len is u32, too, so cast it
as such to avoid the warning.
Tested on clang version 3.5-1ubuntu1 on x86-64
|
|
A reviewer may wonder why fprintf is chosen instead of rb_warn,
so make it clear we're outside of the GVL when spewing the
warning message and cannot use most rb_* functions.
|
|
Existing subscribers on librelist will need to resubscribe since
there's no published subscriber lists anywhere.
The public-inbox + mlmmj setup on bogomips.org allows posting
without subscription and offers downloadable archives via git.
The lack of rsyncable archives on librelist nowadays and
subscription-required nature of librelist are points against it.
Repliers should Cc: all recipients (using the reply-all
function of their mail client) since many readers are not
subscribed.
This project has never accepted or encouraged HTML email, but
librelist accepted it. The bogomips.org mail server is
configured to treat HTML mail as spam, so do not send HTML mail
if you expect a response.
Users who wish to subscribe may send a message to:
raindrops-public+subscribe@bogomips.org
Similarly, they may unsubscribe via:
raindrops-public+unsubscribe@bogomips.org
HTTP archives are available via:
http://bogomips.org/raindrops-public/
ssoma users may also use: git://bogomips.org/raindrops-public
(see README change)
Old messages to the librelist addresses will continue to
get routed to the new mailing list.
ref: http://public-inbox.org/
|
|
Clarify the code so memory leak alarms in reviewers eyes do not go off.
|
|
Do not define or use rb_thread_blocking_region if
rb_thread_io_blocking_region is available.
rb_thread_blocking_region is gone entirely in Ruby trunk.
|
|
Scoped ipv6 addresses are defined in rfc4007.
Ruby doesn't support them yet and it's unknown whether it will
(see http://bugs.ruby-lang.org/issues/8464).
So we just remove scope ids.
Tested with MRI and Rubinius.
|
|
RARRAY_PTR is expensive with GCs in Ruby 2.1.0dev and Rubinius, so
use rb_ary_entry for non-performance critical paths. Eventually,
RARRAY_AREF/RARRAY_ASET may be common, but for now, using
rb_ary_entry should require the least cognitive overhead for a
developer.
|
|
Using an extra 4 bytes for the listener_stats should not
significantly increase space usage, and it has the side benefit
of making our code slightly smaller.
$ ~/linux/scripts/bloat-o-meter before.so after.so
add/remove: 0/0 grow/shrink: 1/2 up/down: 14/-32 (-18)
function old new delta
tcp_stats 392 406 +14
st_to_hash 195 187 -8
diag 763 739 -24
|
|
This initialization was unnecessary and avoids the following
warning with -Wmissing-braces on gcc 4.7.2-5 on Debian testing:
linux_inet_diag.c: In function ‘stats_for’:
linux_inet_diag.c:192:8: warning: missing braces around initializer [-Wmissing-braces]
linux_inet_diag.c:192:8: warning: (near initialization for ‘sa.ss’) [-Wmissing-braces]
|
|
This is exported (visibly) under Ruby 1.9.3 but not
in headers, so it was causing warnings.
|
|
A non-profit TLD makes more sense for a Free Software project.
|
|
No need to cast when C has handy unions. Maybe the compiler
will be able to make better optimization choices here, but
at least it'll be less noisy.
|
|
Seems to breaks under newer libc headers (on Debian sid).
|
|
|
|
Very few programs can take advantage of inheriting FDs
across exec() boundaries, and inet_diag sockets have no
reason to be used in this way.
|
|
It can detect cross-thread close() calls
|
|
Oops :x This was totally broken with the all-listener
filter.
|
|
|
|
getaddrinfo() needs to get a list of available interfaces
from the kernel with every single call (since ipv6 could've
been modprobed), so it's a waste of syscalls.
|
|
No need to waste resources on creating/destroying
a socket.
|
|
I thought my compiler would be smarter :<
|
|
The way we dump, we dump it all, it seems.
|
|
This means we can read multiple addresses at once,
even IPv6 ones.
|
|
It's slow, but at least it works.
|
|
No reason to have an extra method. This also speeds up
the multi-listener case for tcp_listener_stats since it
avoids expensive sendmsg() syscalls.
|
|
|
|
netlink is fast and predictable in response times, so permitting
interrupts would just complicate things and lead to errors.
|
|
Fewer places to check for errors, we think.
|
|
Be stricter about invalid inputs.
|
|
This is a work-in-progress and will probably be modified
before the next release.
|
|
64-bit counters are unnecessarily large for tracking
active or queued connections until we have IP_ROFLSCALE
support :>
|
|
Oops :x
|
|
Too hard to maintain.
|
|
We can't have negative values
|
|
We don't care for this address.
|