* background threads
@ 2012-06-18 18:32 Alexandre Riveira
[not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
0 siblings, 1 reply; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 18:32 UTC (permalink / raw)
To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw
Hello !
I'm trying to put in background threads in Rainbows. I created a
config/initializers/background_thread.rb, where I
Thread.new do
# my background job
end
With that, I have problems to "reload" in rainbows
Thank you all for your help.
Alexandre Riveira
Brazil
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
@ 2012-06-18 19:08 ` Alexandre Riveira
[not found] ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 19:26 ` Alexandre Riveira
1 sibling, 1 reply; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 19:08 UTC (permalink / raw)
To: Eric Wong; +Cc: Rainbows! list
Em 18-06-2012 21:55, Eric Wong escreveu:
What kind of problems?
The application consumes 100% cpu after reload
Is preload_app true or false? (default is false)
preload_app is true
I changed my code as below and got a result in small servers
Thread.new(log) do |rotina_log|
loop do
log.info("rotina automatica") {"waiting process before end"}
sleep(5)
break unless File.exists? Dir.pwd + '/tmp/pids/rainbows.pid.oldbin'
end
loop do
#rotina_log.info("rotina automatica") { "waiting 60 secs #{Process.pid}" }
sleep(60)
if $0.include? "old"
rotina_log.info("rotina automatica") { "exit..... #{Process.pid}" }
Thread.exit
end
begin
#rotina_log.info("rotina automatica") { "exec #{Process.pid}" }
empresas = Empresa::Base.find_all_by_cancelado(false)
#sleep(180)
empresas.invoke(:rotinas_automaticas, :usuario_id => Auth::Configuracao.instance.usuario_sistema_id)
#rotina_log.info("rotina automatica") { "exec ok #{Process.pid}" }
rescue Exception => e
rotina_log.fatal("rotina automatica") { e.to_s + e.backtrace.inspect }
end
end
end
Tanks,
Alexandre Riveira
Brazil
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-06-18 19:08 ` Alexandre Riveira
@ 2012-06-18 19:26 ` Alexandre Riveira
1 sibling, 0 replies; 7+ messages in thread
From: Alexandre Riveira @ 2012-06-18 19:26 UTC (permalink / raw)
To: Eric Wong; +Cc: Rainbows! list
Em 18-06-2012 21:55, Eric Wong escreveu:
> So for background threads, I prefer to start it in the app stack
> on the first request (use a mutex to protect against double-starts).
> For rails, you can probably do it in a before_filter.
Eric, como você garante a aplicação não criar uma thread em cada worker
Tanks
Alexandre Riveira
brazil
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 21:55 ` Eric Wong
[not found] ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
[not found] ` <4FDF81A5.6080605@objectdata.com.br>
0 siblings, 2 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 21:55 UTC (permalink / raw)
To: Rainbows! list; +Cc: Alexandre Riveira
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Hello !
>
> I'm trying to put in background threads in Rainbows. I created a
> config/initializers/background_thread.rb, where I
>
> Thread.new do
> # my background job
> end
>
> With that, I have problems to "reload" in rainbows
What kind of problems?
Is preload_app true or false? (default is false)
preload_app being true will cause that background thread to get
started in the master process and not the workers. You can
workaround it by starting th background thread in an after_fork
hook in the Rainbows!/unicorn config file:
http://unicorn.bogomips.org/Unicorn/Configurator.html#method-i-after_fork
> Thank you all for your help.
I'm often lazy when it comes to setting up hooks myself :x
So for background threads, I prefer to start it in the app stack
on the first request (use a mutex to protect against double-starts).
For rails, you can probably do it in a before_filter.
Raindrops::Watcher is a Rack app (no other framework, just rack)
which starts a background thread in the #call method:
git clone git://bogomips.org/raindrops raindrops
cat raindrops/lib/watcher.rb
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:21 ` Eric Wong
0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:21 UTC (permalink / raw)
To: Rainbows! list
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 21:55, Eric Wong escreveu:
>
> What kind of problems?
>
> The application consumes 100% cpu after reload
>
>
> Is preload_app true or false? (default is false)
> preload_app is true
With preload_app true, the application worker won't be able to
communicate with the background thread after reload.
I'm not sure if your setup relies on threads communicating
directly with the application.
> I changed my code as below and got a result in small servers
> Thread.new(log) do |rotina_log|
> loop do
> log.info("rotina automatica") {"waiting process before end"}
> sleep(5)
> break unless File.exists? Dir.pwd + '/tmp/pids/rainbows.pid.oldbin'
> end
>
> loop do
> #rotina_log.info("rotina automatica") { "waiting 60 secs #{Process.pid}" }
> sleep(60)
> if $0.include? "old"
> rotina_log.info("rotina automatica") { "exit..... #{Process.pid}" }
> Thread.exit
> end
> begin
> #rotina_log.info("rotina automatica") { "exec #{Process.pid}" }
> empresas = Empresa::Base.find_all_by_cancelado(false)
> #sleep(180)
> empresas.invoke(:rotinas_automaticas, :usuario_id => Auth::Configuracao.instance.usuario_sistema_id)
> #rotina_log.info("rotina automatica") { "exec ok #{Process.pid}" }
> rescue Exception => e
> rotina_log.fatal("rotina automatica") { e.to_s + e.backtrace.inspect }
> end
> end
> end
Reading the code above (but not knowing your original code), I'm not
seeing any communcation between the threads. It could probably be done
better as a standalone daemon or even a cronjob. There's more setup
overhead, sure, but it should also be easier to debug/troubleshoot.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <4FDF81A5.6080605-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:42 ` Eric Wong
[not found] ` <4FDF85C2.1030900@objectdata.com.br>
0 siblings, 1 reply; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:42 UTC (permalink / raw)
To: rainbows-talk-GrnCvJ7WPxnNLxjTenLetw; +Cc: Alexandre Riveira
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 21:55, Eric Wong escreveu:
> >So for background threads, I prefer to start it in the app stack
> >on the first request (use a mutex to protect against double-starts).
> >For rails, you can probably do it in a before_filter.
>
> Eric, how do you ensure the application does not create a thread for
> each worker
In the Raindrops::Watcher case, I don't. Watcher works as a single
process and doesn't need/care about multiple background threads.
If you only want one background thread for all workers, it probably is
best to just use a separate daemon independent of the Rainbows! server.
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: background threads
[not found] ` <4FDF85C2.1030900-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
@ 2012-06-18 22:53 ` Eric Wong
0 siblings, 0 replies; 7+ messages in thread
From: Eric Wong @ 2012-06-18 22:53 UTC (permalink / raw)
To: Alexandre Riveira; +Cc: rainbows-talk
Alexandre Riveira <alexandre-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org> wrote:
> Em 18-06-2012 22:42, Eric Wong escreveu:
> >If you only want one background thread for all workers, it
> >probably is best to just use a separate daemon independent of the
> >Rainbows! server.
> I want only one worker to run a background thread. I would like to
> ensure that only one worker thread will be in the background. Sure
> ... using rainbows :)
It's a bad idea, I think...
You can use process-shared locks Ruby provides File#flock.
It's also possible to get fcntl()-based locking to work with pack().
Maybe there's extensions to share locks with multiple processes
(e.g. POSIX semaphores, database-based locks in Redis/MySQL/Postgres).
...Or even use a really ugly method of using TCPServer to bind to a
pre-determined port (and bailing on Errno::EADDRINUSE) (no need to
accept connections on the TCPServer, just keep the object around
so it doesn't get close(2)-ed on GC.
But really, I think this type of background thread is a bad idea
for Rainbows! (or any HTTP server).
_______________________________________________
Rainbows! mailing list - rainbows-talk-GrnCvJ7WPxnNLxjTenLetw@public.gmane.org
http://rubyforge.org/mailman/listinfo/rainbows-talk
Do not quote signatures (like this one) or top post when replying
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2012-06-18 22:53 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-06-18 18:32 background threads Alexandre Riveira
[not found] ` <4FDF7420.7030300-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 21:55 ` Eric Wong
[not found] ` <20120618215550.GA31162-yBiyF41qdooeIZ0/mPfg9Q@public.gmane.org>
2012-06-18 19:08 ` Alexandre Riveira
[not found] ` <4FDF7CA0.6030103-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:21 ` Eric Wong
2012-06-18 19:26 ` Alexandre Riveira
[not found] ` <4FDF81A5.6080605@objectdata.com.br>
[not found] ` <4FDF81A5.6080605-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:42 ` Eric Wong
[not found] ` <4FDF85C2.1030900@objectdata.com.br>
[not found] ` <4FDF85C2.1030900-VwDbj2YsoUp0ZRtCdD4y8VAUjnlXr6A1@public.gmane.org>
2012-06-18 22:53 ` Eric Wong
Code repositories for project(s) associated with this public inbox
https://yhbt.net/rainbows.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).