From: Mike Galbraith <umgwanakikbuti@gmail.com>
To: Tejun Heo <tj@kernel.org>
Cc: Michal Hocko <mhocko@kernel.org>, Jiri Slaby <jslaby@suse.cz>,
Thomas Gleixner <tglx@linutronix.de>,
Petr Mladek <pmladek@suse.com>, Jan Kara <jack@suse.cz>,
Ben Hutchings <ben@decadent.org.uk>,
Sasha Levin <sasha.levin@oracle.com>, Shaohua Li <shli@fb.com>,
LKML <linux-kernel@vger.kernel.org>,
stable@vger.kernel.org, Daniel Bilik <daniel.bilik@neosystem.cz>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: Crashes with 874bbfe600a6 in 3.18.25
Date: Tue, 09 Feb 2016 16:31:25 +0100 [thread overview]
Message-ID: <1455031885.3807.74.camel@gmail.com> (raw)
In-Reply-To: <20160205210606.GH4401@htj.duckdns.org>
On Fri, 2016-02-05 at 16:06 -0500, Tejun Heo wrote:
> On Fri, Feb 05, 2016 at 09:59:49PM +0100, Mike Galbraith wrote:
> > On Fri, 2016-02-05 at 15:54 -0500, Tejun Heo wrote:
> >
> > > What are you suggesting?
> >
> > That 874bbfe6 should die.
>
> Yeah, it's gonna be killed. The commit is there because the behavior
> change broke things. We don't want to guarantee it but have been and
> can't change it right away just because we don't like it when things
> may break from it. The plan is to implement a debug option to force
> workqueue to always execute these work items on a foreign cpu to weed
> out breakages.
A niggling question remaining is when is it gonna be killed?
1. Meanwhile, 874bbfe6 was sent to 2.6.31+, meaning that every stable
tree where it landed which did not ALSO receive 22b886dd has become
destabilized. We have two 3.12-stability reports, one the hotplug
explosion that you provided a workaround for, one the corruption, and
one corruption report for 3.18. Both breakage types would be sort of
fixed up by getting 22b886dd and your hotplug workaround (which does
_not_ guarantee survival) were applied everywhere, however...
2. We also have a report for the 3.18 corruption victim that adding
22b886dd did NOT restore the stable status quo, rather it replaced the
corruption that 874bbfe6 caused with a performance regression.
3. 874bbfe6 + 22b886dd also inflicts a NO_HZ_FULL regression.
Admittedly not a huge deal, but another regression nonetheless.
The only evidence I've seen that anything at all was the broken by the
changes that triggered the inception of 874bbfe6 in the first place was
the b0rked vmstat thing that Linus had already fixed with 176bed1d. So
where is the breakage you mention that makes keeping 874bbfe6 the
prudent thing to do vs just reverting 874bbfe6 immediately, perhaps
22b886dd as well given it is fallout thereof, and getting that sent off
to stable?
It looks for all the world as if the sole excuse for either to exist is
to prevent any other stupid mistakes like the vmstat thing from being
exposed for what they are by actively hiding them, when in fact, that
hiding doesn't survive a hotplug event (as we saw in the crash analysis
I showed you). Surely there's a better reason to keep that commit than
hiding bugs that can only remain hidden until they meet hotplug. What
is it?
-Mike
next prev parent reply other threads:[~2016-02-09 15:31 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-01-20 21:19 Crashes with 874bbfe600a6 in 3.18.25 Jan Kara
2016-01-20 21:39 ` Shaohua Li
2016-01-21 9:52 ` Jan Kara
2016-01-21 13:29 ` Sasha Levin
2016-01-22 1:10 ` Sasha Levin
2016-01-22 16:09 ` Tejun Heo
2016-01-23 2:20 ` Ben Hutchings
2016-01-23 16:11 ` Thomas Gleixner
2016-01-26 9:34 ` Jan Kara
2016-01-26 9:49 ` Thomas Gleixner
2016-01-26 11:14 ` Petr Mladek
2016-01-26 13:09 ` Thomas Gleixner
2016-02-03 9:35 ` Jiri Slaby
2016-02-03 10:41 ` Thomas Gleixner
2016-02-03 12:28 ` Michal Hocko
2016-02-03 16:24 ` Tejun Heo
2016-02-03 16:48 ` Michal Hocko
2016-02-03 16:59 ` Tejun Heo
2016-02-04 6:37 ` Michal Hocko
2016-02-04 7:40 ` Michal Hocko
2016-02-03 17:01 ` Mike Galbraith
2016-02-03 17:06 ` Tejun Heo
2016-02-03 17:13 ` Mike Galbraith
2016-02-03 17:15 ` Tejun Heo
2016-02-04 2:00 ` Mike Galbraith
2016-02-05 16:49 ` Tejun Heo
2016-02-05 20:47 ` Mike Galbraith
2016-02-05 20:54 ` Tejun Heo
2016-02-05 20:59 ` Mike Galbraith
2016-02-05 21:06 ` Tejun Heo
2016-02-06 13:07 ` Henrique de Moraes Holschuh
2016-02-07 5:19 ` Mike Galbraith
2016-02-07 5:59 ` Mike Galbraith
2016-02-09 15:31 ` Mike Galbraith [this message]
2016-02-09 16:39 ` Linus Torvalds
2016-02-09 16:50 ` Tejun Heo
2016-02-09 17:04 ` Mike Galbraith
2016-02-09 17:54 ` Tejun Heo
2016-02-09 17:56 ` Mike Galbraith
2016-02-09 18:02 ` Mike Galbraith
2016-02-09 18:27 ` Tejun Heo
2016-02-09 17:04 ` Linus Torvalds
2016-02-09 17:51 ` Tejun Heo
2016-02-09 18:06 ` Linus Torvalds
2016-02-04 10:04 ` Mike Galbraith
2016-02-04 10:46 ` Thomas Gleixner
2016-02-04 11:07 ` Mike Galbraith
2016-02-04 11:20 ` Jan Kara
2016-02-04 16:39 ` Daniel Bilik
2016-02-05 2:40 ` Mike Galbraith
2016-02-05 8:11 ` Daniel Bilik
2016-02-05 8:33 ` Mike Galbraith
2016-02-03 18:46 ` Thomas Gleixner
2016-02-03 19:01 ` Tejun Heo
2016-02-03 19:05 ` Thomas Gleixner
2016-02-03 19:15 ` Tejun Heo
2016-02-05 5:44 ` Mike Galbraith
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1455031885.3807.74.camel@gmail.com \
--to=umgwanakikbuti@gmail.com \
--cc=ben@decadent.org.uk \
--cc=daniel.bilik@neosystem.cz \
--cc=gregkh@linuxfoundation.org \
--cc=jack@suse.cz \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=mhocko@kernel.org \
--cc=pmladek@suse.com \
--cc=sasha.levin@oracle.com \
--cc=shli@fb.com \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
--cc=tj@kernel.org \
--cc=torvalds@linux-foundation.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).