LKML Archive mirror
 help / color / mirror / Atom feed
From: Ben Hutchings <ben@decadent.org.uk>
To: Tejun Heo <tj@kernel.org>, Sasha Levin <sasha.levin@oracle.com>
Cc: Jan Kara <jack@suse.cz>, Shaohua Li <shli@fb.com>,
	LKML <linux-kernel@vger.kernel.org>,
	stable@vger.kernel.org, Daniel Bilik <daniel.bilik@neosystem.cz>,
	Thomas Gleixner <tglx@linutronix.de>
Subject: Re: Crashes with 874bbfe600a6 in 3.18.25
Date: Sat, 23 Jan 2016 02:20:23 +0000	[thread overview]
Message-ID: <1453515623.3734.156.camel@decadent.org.uk> (raw)
In-Reply-To: <20160122160903.GH32380@htj.duckdns.org>

[-- Attachment #1: Type: text/plain, Size: 2196 bytes --]

On Fri, 2016-01-22 at 11:09 -0500, Tejun Heo wrote:
> (cc'ing Thomas)
> 
> On Thu, Jan 21, 2016 at 08:10:20PM -0500, Sasha Levin wrote:
> > On 01/21/2016 04:52 AM, Jan Kara wrote:
> > > On Wed 20-01-16 13:39:01, Shaohua Li wrote:
> > > > On Wed, Jan 20, 2016 at 10:19:26PM +0100, Jan Kara wrote:
> > > > > Hello,
> > > > > 
> > > > > a friend of mine started seeing crashes with 3.18.25 kernel - once
> > > > > appropriate load is put on the machine it crashes within minutes. He
> > > > > tracked down that reverting commit 874bbfe600a6 (this is the commit ID from
> > > > > Linus' tree, in stable tree the commit ID is 1e7af294dd03) "workqueue: make
> > > > > sure delayed work run in local cpu" makes the kernel stable again. I'm
> > > > > attaching screenshot of the crash - sadly the initial part is missing but
> > > > > it seems that we crashed when processing timers on otherwise idle CPU. This
> > > > > is a production machine so experimentation is not easy but if we really
> > > > > need more information it may be possible to reproduce the issue again and
> > > > > gather it.
> > > > > 
> > > > > Anyone has idea what is going on? I was looking into the code for a while
> > > > > but so far I have no good explanation.  It would be good to understand the
> > > > > cause instead of just blindly reverting the commit from stable tree...
> > > > 
> > > > Tejun fixed a bug in timer: 22b886dd10180939. is it included in 3.18.25?
> > > 
> > > That doesn't seem to be included in 3.18-stable although it was CCed to stable.
> > > Sasha?
> > 
> > Looks like it requires more than trivial backport (I think). Tejun?
> 
> The timer migration has changed quite a bit.  Given that we've never
> seen vmstat work crashing in 3.18 era, I wonder whether the right
> thing to do here is reverting 874bbfe600a6 from 3.18 stable?

It's not just 3.18 that has this; 874bbfe600a6 was backported to all
stable branches from 3.10 onward.  Only the 4.2-ckt branch has
22b886dd10180939.

Ben.

-- 
Ben Hutchings
Life is what happens to you while you're busy making other plans.
                                                               - John Lennon

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

  reply	other threads:[~2016-01-23  2:20 UTC|newest]

Thread overview: 57+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-01-20 21:19 Crashes with 874bbfe600a6 in 3.18.25 Jan Kara
2016-01-20 21:39 ` Shaohua Li
2016-01-21  9:52   ` Jan Kara
2016-01-21 13:29     ` Sasha Levin
2016-01-22  1:10     ` Sasha Levin
2016-01-22 16:09       ` Tejun Heo
2016-01-23  2:20         ` Ben Hutchings [this message]
2016-01-23 16:11           ` Thomas Gleixner
2016-01-26  9:34             ` Jan Kara
2016-01-26  9:49               ` Thomas Gleixner
2016-01-26 11:14               ` Petr Mladek
2016-01-26 13:09                 ` Thomas Gleixner
2016-02-03  9:35                   ` Jiri Slaby
2016-02-03 10:41                     ` Thomas Gleixner
2016-02-03 12:28                     ` Michal Hocko
2016-02-03 16:24                       ` Tejun Heo
2016-02-03 16:48                         ` Michal Hocko
2016-02-03 16:59                           ` Tejun Heo
2016-02-04  6:37                             ` Michal Hocko
2016-02-04  7:40                               ` Michal Hocko
2016-02-03 17:01                         ` Mike Galbraith
2016-02-03 17:06                           ` Tejun Heo
2016-02-03 17:13                             ` Mike Galbraith
2016-02-03 17:15                               ` Tejun Heo
2016-02-04  2:00                             ` Mike Galbraith
2016-02-05 16:49                               ` Tejun Heo
2016-02-05 20:47                                 ` Mike Galbraith
2016-02-05 20:54                                   ` Tejun Heo
2016-02-05 20:59                                     ` Mike Galbraith
2016-02-05 21:06                                       ` Tejun Heo
2016-02-06 13:07                                         ` Henrique de Moraes Holschuh
2016-02-07  5:19                                           ` Mike Galbraith
2016-02-07  5:59                                             ` Mike Galbraith
2016-02-09 15:31                                         ` Mike Galbraith
2016-02-09 16:39                                           ` Linus Torvalds
2016-02-09 16:50                                             ` Tejun Heo
2016-02-09 17:04                                               ` Mike Galbraith
2016-02-09 17:54                                                 ` Tejun Heo
2016-02-09 17:56                                                   ` Mike Galbraith
2016-02-09 18:02                                                     ` Mike Galbraith
2016-02-09 18:27                                                       ` Tejun Heo
2016-02-09 17:04                                               ` Linus Torvalds
2016-02-09 17:51                                                 ` Tejun Heo
2016-02-09 18:06                                                   ` Linus Torvalds
2016-02-04 10:04                             ` Mike Galbraith
2016-02-04 10:46                               ` Thomas Gleixner
2016-02-04 11:07                                 ` Mike Galbraith
2016-02-04 11:20                                 ` Jan Kara
2016-02-04 16:39                                   ` Daniel Bilik
2016-02-05  2:40                                     ` Mike Galbraith
2016-02-05  8:11                                       ` Daniel Bilik
2016-02-05  8:33                                         ` Mike Galbraith
2016-02-03 18:46                         ` Thomas Gleixner
2016-02-03 19:01                           ` Tejun Heo
2016-02-03 19:05                             ` Thomas Gleixner
2016-02-03 19:15                               ` Tejun Heo
2016-02-05  5:44                         ` Mike Galbraith

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1453515623.3734.156.camel@decadent.org.uk \
    --to=ben@decadent.org.uk \
    --cc=daniel.bilik@neosystem.cz \
    --cc=jack@suse.cz \
    --cc=linux-kernel@vger.kernel.org \
    --cc=sasha.levin@oracle.com \
    --cc=shli@fb.com \
    --cc=stable@vger.kernel.org \
    --cc=tglx@linutronix.de \
    --cc=tj@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).