From: Thorsten von Eicken <tve-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org>
To: "Rainbows! list" <rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org>
Subject: Re: HUP restart is delayed by 2 minutes
Date: Wed, 29 Jan 2014 12:11:38 -0800 [thread overview]
Message-ID: <52E9607A.60107@rightscale.com> (raw)
In-Reply-To: <20140129083147.GA26652-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
On 1/29/2014 12:31 AM, Eric Wong wrote:
> Thorsten von Eicken <tve-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org> wrote:
>> On 1/28/2014 11:02 PM, Eric Wong wrote:
>>> Thorsten von Eicken <tve-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org> wrote:
>>>> I'm having trouble getting the HUP restart to work properly (I do have
>>>> preload_app=false). What happens is that on HUP the master reloads the
>>>> config file, but it then always takes 2 minutes plus 1-2 seconds for the
>>>> workers to be reaped. The servers is totally idle, so this is not due to
>>>> some open connections. Here's how it looks in the log file with 2 worker
>>>> processes:
>>> Which concurrency option are you using and which version of Rainbows!,
>>> unicorn and Ruby is this?
>> root@vscale:/home/vscale-admin/current# bundle show rainbows
>> /var/lib/gems/1.9.1/gems/rainbows-4.5.0
>> root@vscale:/home/vscale-admin/current# bundle show unicorn
>> /var/lib/gems/1.9.1/gems/unicorn-4.7.0
>> root@vscale:/home/vscale-admin/current# bundle exec ruby -v
>> ruby 1.9.3p0 (2011-10-30 revision 33570) [x86_64-linux]
>> root@vscale:/home/vscale-admin/current# bundle exec rainbows --version
>> Rainbows! v4.5.0
> This looks like a symptom of unicorn 4.8.0 interacting badly
> with rainbows 4.5.0, not unicorn 4.7.0. Can you give
> rainbows 4.6.0 + unicorn 4.8.0 a try? Unfortunately, I broke
> rainbows 4.5.0 and earlier with unicorn 4.8.0 :x
>
> I'll be pushing out unicorn 4.8.1 soon with a minor, unrelated bugfix.
>
I am now running with Rainbows 4.6.0 and Unicorn 4.8.1 and still see the
same, but not always (the first restart only took 3-4 seconds to
happen). I'm printing the versions in the config file:
Rainbows version: 4.6.0
Unicorn version: 4.8.1
I, [2014-01-29T11:55:43.687022 #18919] INFO -- : done reloading
config_file=./config/rainbows.rb
I did an strace of a worker process and not much is happening:
$ strace -p 19255
Process 19255 attached - interrupt to quit
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 805429204}, NULL, 8) = 0
(Timeout)
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 999995000}, NULL, 8) = 0
(Timeout)
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 999996000}, NULL, 8) = 0
(Timeout)
...
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 999997000}, NULL, 8) = 0
(Timeout)
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 999996000}, NULL, 8) = 0
(Timeout)
ppoll([{fd=9, events=POLLIN|POLLPRI}], 1, {0, 999997000}, NULL, 8) = 0
(Timeout)
exit_group(2) = ?
Process 19255 detached
Here's an strace of the master process (the workers at that time were
19781 and 19783). I marked the spot where everything just sits waiting
for 2 minutes and there clearly is a select call with a 2 minute timeout.
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = -1 EINTR (Interrupted system call)
futex(0x7f0ac29b67a0, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
futex(0x7f0ac29b67a0, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(6, F_GETFL) = 0x801 (flags O_WRONLY|O_NONBLOCK)
write(6, ".", 1) = 1
select(6, [5], NULL, NULL, {77, 985736}) = 1 (in [5], left {77, 985735})
fcntl(5, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK)
read(5, ".", 11) = 1
wait4(-1, 0x7fff006feabc, WNOHANG, NULL) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "I, [2014-01-29T12:03:19.875207 #"..., 93) = 93
open("./config/rainbows.rb", O_RDONLY) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=1642, ...}) = 0
fstat(8, {st_mode=S_IFREG|0644, st_size=1642, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe8c8) = -1 ENOTTY
(Inappropriate ioctl for device)
fstat(8, {st_mode=S_IFREG|0644, st_size=1642, ...}) = 0
lseek(8, 0, SEEK_CUR) = 0
read(8, "# See http://unicorn.bogomips.or"..., 1642) = 1642
read(8, "", 8192) = 0
close(8) = 0
getcwd("/home/vscale-admin/git", 200) = 23
write(2, "./config/rainbows.rb:4: warning:"..., 71) = 71
write(1, "APP_PATH is /home/vscale-admin/c"..., 38) = 38
write(1, "\n", 1) = 1
write(2, "./config/rainbows.rb:8: warning:"..., 71) = 71
write(2, "./config/rainbows.rb:9: warning:"..., 71) = 71
stat("/home/vscale-admin/current/./config/rainbows.rb",
{st_mode=S_IFREG|0644, st_size=1642, ...}) = 0
geteuid() = 0
getegid() = 0
getuid() = 0
getgid() = 0
access("/home/vscale-admin/current/./config/rainbows.rb", R_OK) = 0
chdir("/home/vscale-admin/current") = 0
write(1, "Rainbows version: 4.6.0", 23) = 23
write(1, "\n", 1) = 1
write(1, "Unicorn version: 4.8.1", 22) = 22
write(1, "\n", 1) = 1
stat("./config.ru", {st_mode=S_IFREG|0644, st_size=633, ...}) = 0
geteuid() = 0
getegid() = 0
getuid() = 0
getgid() = 0
access("./config.ru", R_OK) = 0
open("./config.ru", O_RDONLY) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=633, ...}) = 0
fstat(8, {st_mode=S_IFREG|0644, st_size=633, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe8c8) = -1 ENOTTY
(Inappropriate ioctl for device)
fstat(8, {st_mode=S_IFREG|0644, st_size=633, ...}) = 0
lseek(8, 0, SEEK_CUR) = 0
read(8, "require 'rack/mount'\nrequire Fil"..., 633) = 633
brk(0x24c6000) = 0x24c6000
read(8, "", 8192) = 0
brk(0x24c4000) = 0x24c4000
close(8) = 0
stat("/var/run/admin.pid", {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
geteuid() = 0
getegid() = 0
getuid() = 0
getgid() = 0
access("/var/run/admin.pid", W_OK) = 0
stat("/var/log/admin.log", {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
geteuid() = 0
getegid() = 0
getuid() = 0
getgid() = 0
access("/var/log/admin.log", W_OK) = 0
stat("/var/log/admin.log", {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
geteuid() = 0
getegid() = 0
getuid() = 0
getgid() = 0
access("/var/log/admin.log", W_OK) = 0
open("/var/run/admin.pid", O_RDONLY) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
fstat(8, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe3a8) = -1 ENOTTY
(Inappropriate ioctl for device)
fstat(8, {st_mode=S_IFREG|0644, st_size=6, ...}) = 0
lseek(8, 0, SEEK_CUR) = 0
read(8, "18919\n", 6) = 6
read(8, "", 8192) = 0
close(8) = 0
kill(18919, SIG_0) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
setsockopt(7, SOL_SOCKET, SO_KEEPALIVE, [1], 4) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
setsockopt(7, SOL_TCP, TCP_NODELAY, [1], 4) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
setsockopt(7, SOL_TCP, TCP_CORK, [1], 4) = 0
getsockname(7, {sa_family=AF_INET, sin_port=htons(8001),
sin_addr=inet_addr("127.0.0.1")}, [16]) = 0
setsockopt(7, SOL_TCP, TCP_DEFER_ACCEPT, [60], 4) = 0
listen(7, 1024) = 0
open("/var/log/admin.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
fstat(8, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe388) = -1 ENOTTY
(Inappropriate ioctl for device)
dup2(8, 2) = 2
fstat(2, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
close(8) = 0
open("/var/log/admin.log", O_WRONLY|O_CREAT|O_APPEND, 0666) = 8
fstat(8, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
fstat(8, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe388) = -1 ENOTTY
(Inappropriate ioctl for device)
dup2(8, 1) = 1
fstat(1, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
close(8) = 0
fcntl(9, F_GETFL) = 0x1 (flags O_WRONLY)
fcntl(9, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
write(9, "\3\0\0\0", 4) = 4
fcntl(10, F_GETFL) = 0x1 (flags O_WRONLY)
fcntl(10, F_SETFL, O_WRONLY|O_NONBLOCK) = 0
write(10, "\3\0\0\0", 4) = 4
fstat(2, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
fcntl(2, F_GETFL) = 0x8401 (flags
O_WRONLY|O_APPEND|O_LARGEFILE)
fstat(1, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
fcntl(1, F_GETFL) = 0x8401 (flags
O_WRONLY|O_APPEND|O_LARGEFILE)
fstat(2, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
stat("/var/log/admin.log", {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
fstat(1, {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
stat("/var/log/admin.log", {st_mode=S_IFREG|0644, st_size=606916, ...}) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "I, [2014-01-29T12:03:19.887123 #"..., 98) = 98
wait4(-1, 0x7fff006feabc, WNOHANG, NULL) = 0
select(6, [5], NULL, NULL, {120, 0}
# ======================== LONG DELAY HERE
====================================
) = 0 (Timeout)
wait4(-1, 0x7fff006feabc, WNOHANG, NULL) = 0
select(6, [5], NULL, NULL, {120, 0}) = ? ERESTARTNOHAND (To be restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
write(4, "!", 1) = 1
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 1
rt_sigreturn(0x2) = -1 EINTR (Interrupted system call)
futex(0x1902708, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = 0
futex(0x1902708, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(6, F_GETFL) = 0x801 (flags O_WRONLY|O_NONBLOCK)
write(6, ".", 1) = 1
select(6, [5], NULL, NULL, {116, 267701}) = 1 (in [5], left {116, 267700})
fcntl(5, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK)
read(5, ".", 11) = 1
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 2}], WNOHANG, NULL) = 19781
close(9) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "E, [2014-01-29T12:05:23.674893 #"..., 103) = 103
wait4(-1, 0x7fff006feabc, WNOHANG, NULL) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=4*1024}) = 0
pipe([8, 9]) = 0
fstat(8, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fstat(9, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fcntl(8, F_GETFL) = 0 (flags O_RDONLY)
fstat(8, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe338) = -1 ENOTTY
(Inappropriate ioctl for device)
fcntl(9, F_GETFL) = 0x1 (flags O_WRONLY)
fstat(9, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(9, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe338) = -1 ENOTTY
(Inappropriate ioctl for device)
fcntl(8, F_SETFD, FD_CLOEXEC) = 0
fcntl(9, F_SETFD, FD_CLOEXEC) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "I, [2014-01-29T12:05:23.676466 #"..., 71) = 71
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGPIPE, {0x7f0ac26607a0, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, {SIG_IGN, [], SA_RESTORER|SA_SIGINFO, 0x7f0ac221b4a0},
8) = 0
write(4, "!", 1) = 1
futex(0x7f0ac2bd89d0, FUTEX_WAIT, 19784, NULL) = ? ERESTARTSYS (To be
restarted)
--- SIGCHLD (Child exited) @ 0 (0) ---
write(4, "!", 1) = 1
rt_sigreturn(0x2) = -1 EINTR (Interrupted system call)
futex(0x7f0ac2bd89d0, FUTEX_WAIT, 19784, NULL) = 0
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f0ac2bcf9d0) = 19798
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clone(child_stack=0x7f0ac2bd7ff0,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0x7f0ac2bd89d0, tls=0x7f0ac2bd8700,
child_tidptr=0x7f0ac2bd89d0) = 19799
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, {0x7f0ac26607a0, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
close(8) = 0
select(6, [5], NULL, NULL, {120, 0}) = ? ERESTARTNOHAND (To be restarted)
--- SIGVTALRM (Virtual timer expired) @ 0 (0) ---
rt_sigreturn(0x1a) = -1 EINTR (Interrupted system call)
futex(0x1902708, FUTEX_WAIT_PRIVATE, 2, NULL) = 0
futex(0x1902708, FUTEX_WAKE_PRIVATE, 1) = 0
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
fcntl(6, F_GETFL) = 0x801 (flags O_WRONLY|O_NONBLOCK)
write(6, ".", 1) = 1
select(6, [5], NULL, NULL, {119, 999369}) = 1 (in [5], left {119, 999368})
fcntl(5, F_GETFL) = 0x800 (flags O_RDONLY|O_NONBLOCK)
read(5, ".", 11) = 1
wait4(-1, [{WIFEXITED(s) && WEXITSTATUS(s) == 2}], WNOHANG, NULL) = 19783
close(10) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "E, [2014-01-29T12:05:23.681011 #"..., 103) = 103
wait4(-1, 0x7fff006feabc, WNOHANG, NULL) = 0
getrlimit(RLIMIT_NOFILE, {rlim_cur=1024, rlim_max=4*1024}) = 0
pipe([8, 10]) = 0
fstat(8, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fstat(10, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
fcntl(8, F_GETFL) = 0 (flags O_RDONLY)
fstat(8, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(8, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe338) = -1 ENOTTY
(Inappropriate ioctl for device)
fcntl(10, F_GETFL) = 0x1 (flags O_WRONLY)
fstat(10, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
ioctl(10, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff006fe338) = -1 ENOTTY
(Inappropriate ioctl for device)
fcntl(8, F_SETFD, FD_CLOEXEC) = 0
fcntl(10, F_SETFD, FD_CLOEXEC) = 0
stat("/etc/localtime", {st_mode=S_IFREG|0644, st_size=2819, ...}) = 0
write(2, "I, [2014-01-29T12:05:23.682344 #"..., 71) = 71
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
rt_sigaction(SIGPIPE, {0x7f0ac26607a0, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, {SIG_IGN, [], SA_RESTORER|SA_SIGINFO, 0x7f0ac221b4a0},
8) = 0
write(4, "!", 1) = 1
futex(0x7f0ac2bd89d0, FUTEX_WAIT, 19799, NULL) = 0
clone(child_stack=0,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f0ac2bcf9d0) = 19800
rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
clone(child_stack=0x7f0ac2bd7ff0,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID,
parent_tidptr=0x7f0ac2bd89d0, tls=0x7f0ac2bd8700,
child_tidptr=0x7f0ac2bd89d0) = 19801
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
rt_sigaction(SIGPIPE, {SIG_IGN, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, {0x7f0ac26607a0, [], SA_RESTORER|SA_SIGINFO,
0x7f0ac221b4a0}, 8) = 0
rt_sigprocmask(SIG_SETMASK, ~[SEGV VTALRM RTMIN RT_1], NULL, 8) = 0
close(8) = 0
select(6, [5], NULL, NULL, {120, 0}
^C <unfinished ...>
Process 18919 detached
My config file is:
# See http://unicorn.bogomips.org/Unicorn/Configurator.html for complete
# documentation.
APP_PATH = File.expand_path(File.join(File.dirname(__FILE__),
'../../current'))
puts "APP_PATH is #{APP_PATH}"
if [ 'staging', 'production' ].include?(ENV['RACK_ENV'])
PID_FILE = '/var/run/admin.pid'
LOG_FILE = '/var/log/admin.log'
else
PID_FILE = File.join(APP_PATH, 'rainbows.pid')
LOG_FILE = APP_PATH + "/log/admin.log"
end
Rainbows! do
use :EventMachine # concurrency model to use
worker_connections 5
keepalive_timeout 2 # zero disables keepalives entirely
client_max_body_size 5*1024*1024 # 5 megabytes
Rainbows.module_eval do
EventMachine.kqueue = false if RUBY_PLATFORM =~ /darwin/i
end
end
# Help ensure your application will always spawn in the symlinked
# "current" directory that Capistrano sets up.
working_directory APP_PATH
worker_processes 2 # make this configurable later
stderr_path LOG_FILE
stdout_path LOG_FILE
# Requests can take a looonng time
timeout 120
# we use a shorter backlog for quicker failover when busy
listen "127.0.0.1:8001", :tcp_nopush => true
# PID file location
pid PID_FILE
# Don't preload app: minimal savings and breaks HUP process control signal
preload_app false
GC.respond_to?(:copy_on_write_friendly=) and GC.copy_on_write_friendly =
true
puts "Rainbows version: #{Rainbows::Const::RAINBOWS_VERSION}"
puts "Unicorn version: #{Rainbows::Const::UNICORN_VERSION}"
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
next prev parent reply other threads:[~2014-01-29 20:11 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-01-29 6:03 HUP restart is delayed by 2 minutes Thorsten von Eicken
[not found] ` <52E89997.6020504-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org>
2014-01-29 7:02 ` Eric Wong
[not found] ` <20140129070259.GA5938-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2014-01-29 7:26 ` Thorsten von Eicken
[not found] ` <52E8AD1B.8090009-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org>
2014-01-29 7:32 ` Eric Wong
2014-01-29 8:31 ` Eric Wong
[not found] ` <20140129083147.GA26652-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2014-01-29 20:11 ` Thorsten von Eicken [this message]
[not found] ` <52E9607A.60107-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org>
2014-01-29 22:03 ` Eric Wong
[not found] ` <20140129220352.GA24393-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2014-01-29 22:41 ` Thorsten von Eicken
2014-03-17 21:23 ` Thorsten von Eicken
[not found] ` <532767E2.70509-nkzjiHSqi6rZJqsBc5GL+g@public.gmane.org>
2014-03-17 21:27 ` Eric Wong
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
List information: https://yhbt.net/rainbows/
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=52E9607A.60107@rightscale.com \
--to=tve-nkzjihsqi6rzjqsbc5gl+g@public.gmane.org \
--cc=rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://yhbt.net/rainbows.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).