Re: [ruby-core:99184] [Ruby master Bug#17023] How to prevent String memory to be relocated in ruby-ffi

Re: [ruby-core:99184] [Ruby master Bug#17023] How to prevent String memory to be relocated in ruby-ffi

[Eric Wong]

tenderlove@ruby-lang.org wrote:
> Right, that makes sense.  I really need to document this (and
> I apologize for not doing so already), but
> `rb_gc_register_address` will pin your objects.  When you know
> you're done with the reference, you can release it with
> `rb_gc_unregister_address`.  Of course if you don't call the
> unregister function, the reference will stay alive forever.

Btw, does rb_gc_register_mark_object pin?  A quick glance at
gc.c tells me it doesn't, and I'll need to revert commit
2a6cb76d5010cb763ef5a2c305728465d15eb7c9 in unicorn:
https://yhbt.net/unicorn-public/20181226050857.6413-1-e@80x24.org/

Anyways, it takes me too long to compile Ruby so I'm back to
running whatever my distro ships.  I haven't been able to
test GC.compact at all.

> https://bugs.ruby-lang.org/issues/17023#change-86563

Re: [ruby-core:99185] [Ruby master Bug#17023] How to prevent String memory to be relocated in ruby-ffi

[Aaron Patterson]

> On Jul 15, 2020, at 4:35 PM, Eric Wong <normalperson@yhbt.net> wrote:
> 
> tenderlove@ruby-lang.org wrote:
>> Right, that makes sense.  I really need to document this (and
>> I apologize for not doing so already), but
>> `rb_gc_register_address` will pin your objects.  When you know
>> you're done with the reference, you can release it with
>> `rb_gc_unregister_address`.  Of course if you don't call the
>> unregister function, the reference will stay alive forever.
> 
> Btw, does rb_gc_register_mark_object pin?  A quick glance at
> gc.c tells me it doesn't, and I'll need to revert commit
> 2a6cb76d5010cb763ef5a2c305728465d15eb7c9 in unicorn:
> https://yhbt.net/unicorn-public/20181226050857.6413-1-e@80x24.org/

Yes, it does pin.  I’m not super proud of this code, but here is where objects passed to rb_gc_register_mark_object get pinned:

  https://github.com/ruby/ruby/blob/c2a6295ec04a191c689d22254ac1ad5d665e27ad/vm.c#L2307-L2320

I don’t know why the mark object array is an array of arrays (I assume so as not to waste space in the array buffer?).  Maybe this could be a more friendly data structure.

I created a pinned list in compile.c so that objects allocated and used at compile time don’t move (they become free to move once iseq assembly is finished). It seems that might be a more generally useful thing, but so far I’ve only seen two places that need this feature.

Re: [ruby-core:99185] [Ruby master Bug#17023] How to prevent String memory to be relocated in ruby-ffi

[Eric Wong]

Aaron Patterson <aaron.patterson@gmail.com> wrote:
> > On Jul 15, 2020, at 4:35 PM, Eric Wong <normalperson@yhbt.net> wrote:
> > Btw, does rb_gc_register_mark_object pin?  A quick glance at
> > gc.c tells me it doesn't, and I'll need to revert commit
> > 2a6cb76d5010cb763ef5a2c305728465d15eb7c9 in unicorn:
> > https://yhbt.net/unicorn-public/20181226050857.6413-1-e@80x24.org/
> 
> Yes, it does pin.  I’m not super proud of this code, but here
> is where objects passed to rb_gc_register_mark_object get
> pinned:
> 
>   https://github.com/ruby/ruby/blob/c2a6295ec04a191c689d22254ac1ad5d665e27ad/vm.c#L2307-L2320

Ah, missed that, thanks.  So all is well in unicorn (for now).

> I don’t know why the mark object array is an array of arrays
> (I assume so as not to waste space in the array buffer?).
> Maybe this could be a more friendly data structure.

It seems so, since rb_ary_push can double the internal buffer.
Anyways, ko1 made commit
a617afc643e17cea5c1960a60adcd5ffb25fdfe2 r46714
Maybe it can be flattened nowadays and rely on GC.compact
to reclaim unused space.

/me goes back to hacking in Perl 5...