Confused classes

Confused classes

[Warren Konkel]

I switched from mod_passenger to unicorn on a fairly high traffic site
and ran into a strange problem that forced me to move back to
mod_passenger... it seems as if classes would sometimes get mixed up
with each other.  If I had two Rails models:

  class Foo < ActiveRecord::Base;  end
  class Bar < ActiveRecord::Base;  end

Normally Foo.inspect and Bar.inspect would return:

  Foo(field1: integer, field2: integer)
  Bar(field3: integer, field4: integer)

When things were "broken" within a process, sometimes I would see:

   Foo(field3: integer, field4: integer)    <--- note field3/field4
actually belong to Bar, not Foo

And because of that, wacky errors would appear in my logs like:

   Foo.find_by_field1(12345) --> not a method
   Foo.create(:field1 => 12345)  --> column not found

I also noticed the problem with field serializing in ActiveRecord... given:

  class Boz < ActiveRecord::Base
    serialize :some_data
  end

When processes were working correctly,  Boz.find(1).some_data would
return an actual object (like a Hash).  When things were broken, the
raw serialized string from the database would be returned... almost as
if the Boz class "forgot" that it's supposed to deserialize
"some_data".

Could it be that class attributes are somehow being co-mingled when
unicorn is starting up under high concurrency?  Perhaps a mutex is
missing somewhere?
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: Confused classes

[Iñaki Baz Castillo]

El Sábado, 6 de Febrero de 2010, Warren Konkel escribió:
> I switched from mod_passenger to unicorn on a fairly high traffic site
> and ran into a strange problem that forced me to move back to
> mod_passenger... it seems as if classes would sometimes get mixed up
> with each other.  If I had two Rails models:
> 
>   class Foo < ActiveRecord::Base;  end
>   class Bar < ActiveRecord::Base;  end
> 
> Normally Foo.inspect and Bar.inspect would return:
> 
>   Foo(field1: integer, field2: integer)
>   Bar(field3: integer, field4: integer)
> 
> When things were "broken" within a process, sometimes I would see:
> 
>    Foo(field3: integer, field4: integer)    <--- note field3/field4
> actually belong to Bar, not Foo
> 
> And because of that, wacky errors would appear in my logs like:
> 
>    Foo.find_by_field1(12345) --> not a method
>    Foo.create(:field1 => 12345)  --> column not found
> 
> I also noticed the problem with field serializing in ActiveRecord... given:
> 
>   class Boz < ActiveRecord::Base
>     serialize :some_data
>   end
> 
> When processes were working correctly,  Boz.find(1).some_data would
> return an actual object (like a Hash).  When things were broken, the
> raw serialized string from the database would be returned... almost as
> if the Boz class "forgot" that it's supposed to deserialize
> "some_data".
> 
> Could it be that class attributes are somehow being co-mingled when
> unicorn is starting up under high concurrency?  Perhaps a mutex is
> missing somewhere?

IMHO all your Unicorn workers are sharing the same DB connection (the same 
ActiveRecord instances) so the problem arises.

Take a look to the configuration here:
  http://unicorn.bogomips.org/examples/unicorn.conf.rb
You can see there how the ActiveRecord is disconnected at the beggining and 
started for each worker later.


-- 
Iñaki Baz Castillo <ibc@aliax.net>
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying

Re: Confused classes

[Eric Wong]

Iñaki Baz Castillo <ibc@aliax.net> wrote:
> El Sábado, 6 de Febrero de 2010, Warren Konkel escribió:
> > Could it be that class attributes are somehow being co-mingled when
> > unicorn is starting up under high concurrency?  Perhaps a mutex is
> > missing somewhere?
> 
> IMHO all your Unicorn workers are sharing the same DB connection (the same 
> ActiveRecord instances) so the problem arises.
> 
> Take a look to the configuration here:
>   http://unicorn.bogomips.org/examples/unicorn.conf.rb
> You can see there how the ActiveRecord is disconnected at the beggining and 
> started for each worker later.

Yes, you have to reconnect any connected TCP sockets since they have no
defined atomicity semantics when they're shared across processes/threads.

Ruby Mutexes aren't useful at all here, they're only useful with threads
in the same process.  If you need to protect TCP client sockets from
multiple processes, you'll need SysV/POSIX semaphores or file locks
(flock/fcntl), but you're really better off using multiple TCP client
sockets in the first place.

-- 
Eric Wong
_______________________________________________
Unicorn mailing list - mongrel-unicorn@rubyforge.org
http://rubyforge.org/mailman/listinfo/mongrel-unicorn
Do not quote signatures (like this one) or top post when replying