From: Sasha Levin <sasha.levin@oracle.com>
To: Greg KH <greg@kroah.com>, NeilBrown <neilb@suse.com>
Cc: "ksummit-discuss@lists.linuxfoundation.org"
<ksummit-discuss@lists.linuxfoundation.org>
Subject: Re: [Ksummit-discuss] [CORE TOPIC] Issues with stable process
Date: Mon, 13 Jul 2015 20:51:59 -0400 [thread overview]
Message-ID: <55A45D2F.5050705@oracle.com> (raw)
In-Reply-To: <20150713205125.GA26074@kroah.com>
On 07/13/2015 04:51 PM, Greg KH wrote:
> On Mon, Jul 13, 2015 at 10:52:10AM +1000, NeilBrown wrote:
>>
>> I've been bitten by this a couple of times too. At least two fairly
>> serious md bugs *never* got into a release from Linus, but did get into
>> -stable and at least one into a vendor kernel.
>
> How did that happen? Did I not wait for a -rc release? Did I miss
> something else? What broke down that caused this?
This is a good point in general: it seems that a few folks have reported issues
where stable bit them in the arse, and if it happens to kernel folks it happens
to regular users too.
It'll be interesting to hear those incidents and figure out what went wrong if
you end up getting bitten by one, the stable folks do really want to know when
things don't work out right.
>> On Sun, 12 Jul 2015 09:32:11 -0400 Sasha Levin <sasha.levin@oracle.com>
>> wrote:
>>
>>
>>>> So it boils down to: "How soon to apply fixes to -stable?", and the trade-off
>>>> between applying fixes early, but risking to break something unknown and new,
>>>> vs. applying fixes late (after more validation), causing more breakage from a
>>>> known issue.
>>>
>>> That's just one solution, but there are a few more (which is why it's worth discussing
>>> it :) ).
>>>
>>> Consider also:
>>>
>>> - Aligning the stable release process with the kernel where we'd do a few release
>>> candidates for the stable kernel before releasing it.
>>>
>>> - Tightening what is allowed to go in as -rc patches, requiring some time in -next
>>> before it even gets into Linus's hands. Even for "serious" things (does it matter if
>>> a fix for a privesc gets in -rc2 or -rc6, beyond that it would be pulled to stable
>>> earlier?)
>>>
>>> - Differentiate the type of patches going into "regular" -stable, and LTS?
>>>
>>
>> My proposal would be to change the default timing.
>> Currently patches tagged for 'stable' go into the next -stable release
>> after they get into Linus's tree. You can ask for an exception
>> (sooner, later, different patch) and Greg (or any other stable
>> maintainer) tries to be accommodating. But you have to remember to ask.
>>
>> I would rather that the default was that patches don't go into -stable
>> until they have
>> - been in a full release from Linus and
>> - been in a Linus's tree for at least 2 weeks.
>> (or 1 week times the age of the target in releases.
>> So a fix in 4.4 get to 4.3-stable after a week, 4.2-stable
>> after 2 weeks etc .... maybe I'm going over-board here).
>>
>> Many fixes are important but simply aren't that urgent so the two or
>> more weeks is no great cost.
>
> Really? Based on the traffic I get, I have people asking me why a patch
> is in Linus's tree is not yet in a -stable release about once a week or
> so.
>
> A year or so ago we made the decision to wait for a patch to show up in
> a -rc release before adding it to -stable because people felt I was
> being "too fast". So we did that, and now people want to wait even
> longer? I don't buy it, and feel that will only delay the problem
> another week.
>
> If you look closely, you will note that for the past 3 months or so,
> I've already been waiting an extra -rc release as it is, just due to
> spare time issues on my side (travel, outreachy review, etc.). And even
> with that delay (which people do keep bugging me about, see the ALSA
> email this weekend for an example of such a thing), we get bugs
> introduced into -stable releases.
>
> So I don't think that a manditory 2 week waiting period is going to help
> out much here, sorry.
>
> If it gets into Linus's tree you had better think it is correct,
> especially if you tag it for stable. Yes, bugs happen, that's part of
> life, but let's not slow down everyone just because we get 1-2 bugs
> introduced into -stable every 6 months or so.
Looking back at the git log and my notes, it would have prevented 6 bugs
that I had to revert/apply a different fix to. I guess it depends if
there's a magic number here that will make everyone change their mind.
What's your opinion about introducing RC pre-releases for stable kernel?
if someone cares about getting a particular fix he can use the RC, if
he cares more about having a tested kernel he'll wait another week or
two for the release.
Thanks,
Sasha
next prev parent reply other threads:[~2015-07-14 0:52 UTC|newest]
Thread overview: 83+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-11 16:12 [Ksummit-discuss] [CORE TOPIC] Issues with stable process Sasha Levin
2015-07-12 10:02 ` Geert Uytterhoeven
2015-07-12 13:32 ` Sasha Levin
2015-07-13 0:52 ` NeilBrown
2015-07-13 3:32 ` Andy Lutomirski
2015-07-13 4:27 ` Sasha Levin
2015-07-13 5:10 ` NeilBrown
2015-07-13 22:55 ` Rafael J. Wysocki
2015-07-13 18:21 ` Steven Rostedt
2015-07-13 18:51 ` Mark Brown
2015-07-15 14:52 ` Olof Johansson
2015-07-15 15:59 ` Guenter Roeck
2015-07-15 16:03 ` Greg KH
2015-07-15 16:15 ` Sasha Levin
2015-07-15 16:40 ` Greg KH
2015-07-15 19:34 ` Josh Boyer
2015-07-15 21:21 ` Sasha Levin
2015-07-15 22:34 ` Greg KH
2015-07-15 22:40 ` Sasha Levin
2015-07-16 3:36 ` Greg KH
2015-07-17 0:52 ` Rafael J. Wysocki
2015-07-16 9:06 ` Zefan Li
2015-07-16 18:14 ` Olof Johansson
2015-07-14 0:42 ` Sasha Levin
2015-07-14 1:02 ` Steven Rostedt
2015-07-14 2:00 ` Sasha Levin
2015-07-14 2:28 ` Jonathan Corbet
2015-07-14 3:48 ` Stephen Rothwell
2015-07-14 7:14 ` Geert Uytterhoeven
2015-07-14 11:03 ` James Bottomley
2015-07-14 13:29 ` Steven Rostedt
2015-07-14 20:17 ` James Bottomley
2015-07-14 20:45 ` Mark Brown
2015-07-14 22:12 ` Steven Rostedt
2015-07-14 22:36 ` Andy Lutomirski
2015-09-01 8:44 ` Jani Nikula
2015-09-01 20:52 ` Greg KH
2015-09-01 21:00 ` Sasha Levin
2015-09-01 21:08 ` Jiri Kosina
2015-09-01 22:47 ` Sasha Levin
2015-09-02 10:10 ` Luis Henriques
2015-07-16 0:53 ` Rafael J. Wysocki
2015-07-16 11:50 ` Mark Brown
2015-07-14 3:42 ` Stephen Rothwell
2015-07-14 7:03 ` Geert Uytterhoeven
2015-07-14 10:46 ` Mark Brown
2015-07-14 13:57 ` Sasha Levin
2015-07-14 15:25 ` Mark Brown
2015-07-14 15:32 ` Sasha Levin
2015-07-14 15:38 ` Steven Rostedt
2015-07-14 15:53 ` Sasha Levin
2015-07-14 16:02 ` Steven Rostedt
2015-07-14 19:30 ` Sasha Levin
2015-07-14 19:38 ` Steven Rostedt
2015-07-15 1:49 ` NeilBrown
2015-07-15 2:09 ` Sasha Levin
2015-07-15 2:28 ` NeilBrown
2015-07-15 10:13 ` James Bottomley
2015-07-15 23:24 ` NeilBrown
2015-07-16 1:05 ` Andy Lutomirski
2015-07-16 1:43 ` Linus Torvalds
2015-07-16 1:25 ` Steven Rostedt
2015-07-16 9:19 ` James Bottomley
2015-07-16 12:33 ` Jonathan Cameron
2015-08-03 8:32 ` Fengguang Wu
2015-07-14 15:56 ` Mark Brown
2015-07-14 19:01 ` Sasha Levin
2015-07-14 19:18 ` Geert Uytterhoeven
2015-07-14 19:31 ` Sasha Levin
2015-07-15 9:26 ` Jan Kara
2015-07-16 12:53 ` Mark Brown
2015-07-13 9:22 ` Jan Kara
2015-07-13 20:51 ` Greg KH
2015-07-14 0:51 ` Sasha Levin [this message]
2015-07-14 2:46 ` NeilBrown
2015-07-15 19:41 ` Steven Rostedt
2015-07-15 20:14 ` James Bottomley
2015-07-12 15:01 ` Masami Hiramatsu
2015-07-13 10:15 ` Zefan Li
2015-07-13 16:12 ` Sasha Levin
2015-07-14 10:08 ` Zefan Li
2015-07-14 14:00 ` Sasha Levin
2015-07-15 0:01 ` Greg Kroah-Hartman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=55A45D2F.5050705@oracle.com \
--to=sasha.levin@oracle.com \
--cc=greg@kroah.com \
--cc=ksummit-discuss@lists.linuxfoundation.org \
--cc=neilb@suse.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.