* [ANN] sleepy_penguin 3.5.0 - Linux I/O events (and more) for Ruby
@ 2017-03-22 7:27 6% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2017-03-22 7:27 UTC (permalink / raw)
To: ruby-talk, sleepy-penguin
sleepy_penguin provides access to newer, Linux-only system calls to wait
on events from traditionally non-I/O sources. Bindings to the eventfd,
timerfd, inotify, and epoll interfaces are provided. Experimental support
for kqueue on FreeBSD (and likely OpenBSD/NetBSD) is also provided.
* homepage: https://bogomips.org/sleepy_penguin/
* git clone git://bogomips.org/sleepy_penguin.git
* Atom feed: https://bogomips.org/sleepy_penguin/NEWS.atom.xml
* mailing list: sleepy-penguin@bogomips.org
* mail archives: https://bogomips.org/sleepy-penguin/
nntp://news.public-inbox.org/inbox.comp.lang.ruby.sleepy-penguin
https://bogomips.org/sleepy-penguin/new.atom
Changes:
sleepy_penguin 3.5.0
Most notably, kevent and epoll_wait wrappers are now
nestable, so you can write your own event loops inside
somebody elses event loop (not that it's a good idea,
but hey, that's reality, sometimes).
https://bogomips.org/sleepy_penguin/SleepyPenguin/Kqueue.html#method-i-kevent
https://bogomips.org/sleepy_penguin/SleepyPenguin/Epoll.html#method-i-wait
For Linux users, there is now copy_file_range(2) support
for copying regular files:
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#method-c-copy_file_range
There is also a new sendfile wrapper which emulates Linux
sendfile behavior regardless of platform. It will use the
native sendfile(2) syscall on FreeBSD and Linux, at least.
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#method-c-linux_sendfile
Wrappers for the splice(2) and tee(2) syscalls also exist for
Linux users (vmsplice(2) is omitted):
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#method-c-splice
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#method-c-tee
Along with some related constants:
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#F_GETPIPE_SZ
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#F_SETPIPE_SZ
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#F_MORE
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#F_MOVE
https://bogomips.org/sleepy_penguin/SleepyPenguin.html#F_NONBLOCK
In other words, this release merges the useful parts of the old
"io_splice" RubyGem: https://bogomips.org/ruby_io_splice/
Linux 4.5+ epoll users also get EPOLLEXCLUSIVE along existing
constants:
https://bogomips.org/sleepy_penguin/SleepyPenguin/Epoll.html#EXCLUSIVE
Ruby 1.8 and 1.9 support are both gone, Ruby 2.0.0+ is required
for keyword args, now(*).
31 changes since 3.4.1:
README: fix wording: are => is
TODO: add memfd item
epoll: allow :CLOEXEC instead of the long constant
note the epoll/io.rb file is only for Ruby 1.8
support the splice(2) and tee(2) syscalls
implement copy_file_range support for Linux 4.5+
doc: various URL updates (https)
unify rb_gc() handling for out-of-FD conditions
splice: clarification regarding tee() flags
pkg.mk: various updates from other projects
copy_file_range: use correct syscall numbers on x86/x86-64
new API for splice and tee
doc: remove references to IO#pipe_size accessor
remove PIPE_BUF constant definition
copy_file_range: move wrapper to Ruby for keyword arg handling
gemspec: use SPDX-compatible license specifier
implement linux_sendfile support
portability fixes for systems w/o splice, copy_file_range
tests: switch to test-unit
free buffer if pthread_setspecific fails
allow nestable TLS buffers within the same thread
drop unused Ruby 1.8 support code
epoll: add newline to Kernel#warn messages for atomicity
favor require_relative over require
epoll: add EPOLLEXCLUSIVE constant and documentation
kqueue: remove IO#autoclose= and 1.8-related checks
sp_copy: remove dummy 1.8 code for non-native threaded Ruby
build: remove build-time olddoc dependency
copy_file_range: add documentation
doc: tests and examples for changing pipe size in Linux
doc: avoid incorrect links to Epoll::IO
(*) Even ruby 2.1 is unsupported by ruby-core upstream;
but I could be prodded to re-add 1.9.3 support if
somebody really wants it...
^ permalink raw reply [relevance 6%]
* [PATCH 2/1] portability fixes for systems w/o splice, copy_file_range
@ 2017-03-11 9:37 7% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2017-03-11 9:37 UTC (permalink / raw)
To: sleepy-penguin
We need to support FreeBSD, at least.
---
ext/sleepy_penguin/cfr.c | 14 +++-----------
ext/sleepy_penguin/init.c | 12 ++++++++++++
ext/sleepy_penguin/sleepy_penguin.h | 11 +++++++++++
test/test_splice.rb | 2 +-
test/test_splice_eintr.rb | 2 +-
5 files changed, 28 insertions(+), 13 deletions(-)
diff --git a/ext/sleepy_penguin/cfr.c b/ext/sleepy_penguin/cfr.c
index acece4f..860c5f5 100644
--- a/ext/sleepy_penguin/cfr.c
+++ b/ext/sleepy_penguin/cfr.c
@@ -2,17 +2,6 @@
#include "sp_copy.h"
#include <unistd.h>
-#ifndef HAVE_COPY_FILE_RANGE
-# include <sys/syscall.h>
-# if !defined(__NR_copy_file_range)
-# if defined(__x86_64__)
-# define __NR_copy_file_range 326
-# elif defined(__i386__)
-# define __NR_copy_file_range 377
-# endif /* supported arches */
-# endif /* __NR_copy_file_range */
-#endif
-
#ifdef __NR_copy_file_range
static ssize_t my_cfr(int fd_in, off_t *off_in, int fd_out, off_t *off_out,
size_t len, unsigned int flags)
@@ -26,6 +15,8 @@ static ssize_t my_cfr(int fd_in, off_t *off_in, int fd_out, off_t *off_out,
my_cfr((fd_in),(off_in),(fd_out),(off_out),(len),(flags))
#endif
+#if defined(HAVE_COPY_FILE_RANGE) || \
+ (defined(__linux__) && defined(__NR_copy_file_range))
static void *nogvl_cfr(void *ptr)
{
struct copy_args *a = ptr;
@@ -68,3 +59,4 @@ void sleepy_penguin_init_cfr(void)
rb_define_singleton_method(mod, "__cfr", rb_sp_cfr, 6);
}
+#endif /* !HAVE_COPY_FILE_RANGE */
diff --git a/ext/sleepy_penguin/init.c b/ext/sleepy_penguin/init.c
index 8295073..01bb52f 100644
--- a/ext/sleepy_penguin/init.c
+++ b/ext/sleepy_penguin/init.c
@@ -52,8 +52,20 @@ void sleepy_penguin_init_signalfd(void);
# define sleepy_penguin_init_signalfd() for(;0;)
#endif
+#ifdef HAVE_SPLICE
void sleepy_penguin_init_splice(void);
+#else
+# define sleepy_penguin_init_splice() for(;0;)
+#endif
+
+#if defined(HAVE_COPY_FILE_RANGE) || \
+ (defined(__linux__) && defined(__NR_copy_file_range))
void sleepy_penguin_init_cfr(void);
+#else
+# define sleepy_penguin_init_cfr() for (;0;)
+#endif
+
+/* everyone */
void sleepy_penguin_init_sendfile(void);
static size_t l1_cache_line_size_detect(void)
diff --git a/ext/sleepy_penguin/sleepy_penguin.h b/ext/sleepy_penguin/sleepy_penguin.h
index 8aa514a..99ad0b7 100644
--- a/ext/sleepy_penguin/sleepy_penguin.h
+++ b/ext/sleepy_penguin/sleepy_penguin.h
@@ -92,4 +92,15 @@ void *rb_sp_gettlsbuf(size_t *size);
int rb_sp_gc_for_fd(int err);
+#ifndef HAVE_COPY_FILE_RANGE
+# include <sys/syscall.h>
+# if !defined(__NR_copy_file_range) && defined(__linux__)
+# if defined(__x86_64__)
+# define __NR_copy_file_range 326
+# elif defined(__i386__)
+# define __NR_copy_file_range 377
+# endif /* supported arches */
+# endif /* __NR_copy_file_range */
+#endif
+
#endif /* SLEEPY_PENGUIN_H */
diff --git a/test/test_splice.rb b/test/test_splice.rb
index 3650ee3..266f52b 100644
--- a/test/test_splice.rb
+++ b/test/test_splice.rb
@@ -249,4 +249,4 @@ class TestSplice < Test::Unit::TestCase
assert Integer === SleepyPenguin.const_get("F_#{x.upcase}")
}
end
-end
+end if SleepyPenguin.respond_to?(:splice)
diff --git a/test/test_splice_eintr.rb b/test/test_splice_eintr.rb
index 41b6dd0..fb2dbbc 100644
--- a/test/test_splice_eintr.rb
+++ b/test/test_splice_eintr.rb
@@ -31,4 +31,4 @@ class Test_Splice_EINTR < Test::Unit::TestCase
assert_equal 2, nr
assert_equal 1, @usr1
end
-end if defined?(RUBY_ENGINE)
+end if SleepyPenguin.respond_to?(:splice)
--
EW
^ permalink raw reply related [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2017-03-11 1:00 [PATCH] implement linux_sendfile support Eric Wong
2017-03-11 9:37 7% ` [PATCH 2/1] portability fixes for systems w/o splice, copy_file_range Eric Wong
2017-03-22 7:27 6% [ANN] sleepy_penguin 3.5.0 - Linux I/O events (and more) for Ruby Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/sleepy_penguin.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).