Re: Segfaults on http_close?

[Eric Wong <e@80x24.org>]

Eric Wong <e@80x24.org> wrote:
> Xiao Yu <xyu@automattic.com> wrote:
> > Howdy, we are running a 96 node cmogstored cluster and have noticed
> > that when the cluster is busy with lots of writes we occasionally get
> > segfaults in cmogstored. This has happened 7 times in the past week
> > each time on a random and different cmogstored node. Looking at the
> > abrt backtrace of the core dump shows something similar to the
> > following in each instance:
> 
> Thanks for the bug report, sorry this caused you trouble
> and I wonder if this is the same issue Arkadi was hitting
> last year...

Hi Xiao and Arkadi: Can either of you try the 1-line patch
below to disable pthread_attr_setstacksize?

I took another look at the code and couldn't find any other
culprits... (though I admit I'm not mentally as sharp given
pandemic-induced stress and anxiety :<).

Given the mysterious nature of this problem and my inability to
reproduce it; I wonder if there's stack corruption with certain
compilers/glibc happening and blowing past the 4K guard page...

@Arkadi: Xiao recently brought up this (or similar) issue again:
  https://yhbt.net/cmogstored-public/20210111212621.GA12555@dcvr/T/

diff --git a/thrpool.c b/thrpool.c
index bc67ea0..bd71f95 100644
--- a/thrpool.c
+++ b/thrpool.c
@@ -141,7 +141,7 @@ thrpool_add(struct mog_thrpool *tp, unsigned size, unsigned long *nr_eagain)
 
 	CHECK(int, 0, pthread_attr_init(&attr));
 
-	if (stacksize > 0)
+	if (0 && stacksize > 0)
 		CHECK(int, 0, pthread_attr_setstacksize(&attr, stacksize));
 
 	thr = &tp->threads[tp->n_threads].thr;



In retrospect, running a small stack is unnecessary on 64-bit
systems due to practically unlimited virtual address space and
lazy allocation.  It may still make sense for 32-bit (some
embedded systems), though they can set RLIMIT_STACK before
launching.