From: Linus Torvalds <torvalds@linux-foundation.org>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: ksummit <ksummit-discuss@lists.linuxfoundation.org>
Subject: Re: [Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches
Date: Thu, 6 Sep 2018 20:43:09 -0700 [thread overview]
Message-ID: <CA+55aFx9hKVphfU3+aGONbg_wOx9+Ov6HF3Xkq7ZL_pW5m2CLQ@mail.gmail.com> (raw)
In-Reply-To: <20180906224541.27a9c8fe@vmware.local.home>
On Thu, Sep 6, 2018 at 7:45 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> Really, the only testing coverage that a patch gets in linux-next is by
> the bots that are run on them. I will agree that the number of bots and
> automated tests are getting better.
I do have to agree with this.
The bots have gone from finding build issues due to configurations to
actually being pretty damn good for many things.
So linux-next test coverage has clearly improved. It finds real issues
(ok, build problems are real issues too, but I think we all agree they
are trivial in comparison), and thanks to having a lot of debug stuff
enabled, testing these days often finds stuff that is core kernel and
triggerable with KASAN or lockdep etc.
It's been very helpful.
But the automated tests are still very limited, particularly when it
comes to hardware issues. Most bot runs tend to be in virtual machines
and/or on a fairly small set of hardware, and usually for a fairly
limited set of loads.
It still does find things, but to just take that list of five stable
regressions that was posted, none of them look like they'd necessarily
have been found by one of the automated boot bots.
And while I think more real people run development kernels with real
loads, I don't think the coverage is *that* good there either. For
example, I tend find one or two major bugs (as in "doesn't boot") kind
of issues that were never found in linux-next pretty much every single
merge window (I consider it a good merge window if I didn't have to
bisect anything).
And _my_ hardware and usage really is pretty damn basic, which should
tell you about how little some of the automated stuff catches.
(Side note: I think it's improving even on the hardware side. I think
the i915 people must be running a _lot_ more testing before pushing to
me, because while GPU issues used to be one of the areas that was one
of the common causes, and it really hasn't been that lately).
And I suspect most people who run development kernels actually end up
running fairly similar hardware (ie "fairly modern workstation" kind
of hardware).
The thing that starts seeing more actual users tends to be the distros
that have "test" versions. At least with Fedora has had a
bleeding-edge rawhide that often has quite recent kernels, and it has
often found things early.
And then there is stable and actual distro users.
And that really is when you find the _much_ wider hardware, and the
odder cases. Sure, the early testing has hopefully found the _core_
problems, but sometimes there are core problems that are simply
triggered by specific hardware patterns or software uses.
> Another issue about having fixes sit in linux-next for some time after
> -rc5, is that by that time, linux-next is filled with new development
> code waiting for the next merge window. A subtle fix for a bug that
> wasn't caught by linux-next in the first place (how else would that bug
> still be around by rc5?) is highly likely not to catch a bug with the
> fix to that subtle bug.
Also note that going into my tree does mean that now linux-next covers
it. So it's not like a patch being accepted should ever make for
_less_ coverage.
If the bug isn't found in the development tree for a week, and ithe
stable people take it, I think we can just all agree that the
automation in linux-next simply didn't find it.
So the argument that we should delay bug-fixes in order for them to
get more coverage in linux-next seems entirely mis-guided.
Now, if the argument is that people send me stuff that doesn't even
_pretend_ to be a bug-fix, and that this is a problem, then I agree
whole-heartedly with that being a problem. I do occasionally complain
loudly about _that_ problem. It doesn't affect the stable kernels,
perhaps, but it affects the general stability of the development
process.
Linus
next prev parent reply other threads:[~2018-09-07 3:43 UTC|newest]
Thread overview: 138+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-04 20:16 [Ksummit-discuss] [MAINTAINERS SUMMIT] Bug-introducing patches Sasha Levin
2018-09-04 20:53 ` Daniel Vetter
2018-09-05 14:17 ` Steven Rostedt
2018-09-07 0:51 ` Sasha Levin
2018-09-07 1:09 ` Steven Rostedt
2018-09-07 20:12 ` Greg KH
2018-09-07 21:12 ` Greg KH
2018-09-07 1:09 ` Linus Torvalds
2018-09-07 1:49 ` Sasha Levin
2018-09-07 2:31 ` Linus Torvalds
2018-09-07 2:45 ` Steven Rostedt
2018-09-07 3:43 ` Linus Torvalds [this message]
2018-09-07 8:52 ` Daniel Vetter
2018-09-07 8:40 ` Geert Uytterhoeven
2018-09-07 9:07 ` Daniel Vetter
2018-09-07 9:28 ` Geert Uytterhoeven
2018-09-07 17:05 ` Olof Johansson
2018-09-07 14:54 ` Sasha Levin
2018-09-07 15:52 ` Linus Torvalds
2018-09-07 16:17 ` Linus Torvalds
2018-09-07 21:39 ` Mauro Carvalho Chehab
2018-09-09 12:50 ` Stephen Rothwell
2018-09-10 20:05 ` Tony Lindgren
2018-09-10 19:43 ` Sasha Levin
2018-09-10 20:45 ` Steven Rostedt
2018-09-10 21:20 ` Guenter Roeck
2018-09-10 21:46 ` Steven Rostedt
2018-09-10 23:03 ` Eduardo Valentin
2018-09-10 23:13 ` Steven Rostedt
2018-09-11 15:42 ` Steven Rostedt
2018-09-11 17:40 ` Tony Lindgren
2018-09-11 17:47 ` James Bottomley
2018-09-11 18:12 ` Eduardo Valentin
2018-09-11 18:17 ` Geert Uytterhoeven
2018-09-12 15:15 ` Eduardo Valentin
2018-09-11 18:19 ` James Bottomley
2018-09-12 15:17 ` Eduardo Valentin
2018-09-11 18:39 ` Steven Rostedt
2018-09-11 20:09 ` James Bottomley
2018-09-11 20:31 ` Steven Rostedt
2018-09-11 22:53 ` James Bottomley
2018-09-11 23:04 ` Sasha Levin
2018-09-11 23:11 ` James Bottomley
2018-09-11 23:20 ` Sasha Levin
2018-09-12 15:41 ` Eduardo Valentin
2018-09-11 23:22 ` Tony Lindgren
2018-09-11 23:29 ` James Bottomley
2018-09-12 11:55 ` Geert Uytterhoeven
2018-09-12 12:03 ` Laurent Pinchart
2018-09-12 12:29 ` Thomas Gleixner
2018-09-12 12:53 ` Laurent Pinchart
2018-09-12 13:10 ` Alexandre Belloni
2018-09-12 13:30 ` Thomas Gleixner
2018-09-12 23:16 ` Laurent Pinchart
2018-09-12 14:11 ` Thomas Gleixner
2018-09-19 8:26 ` Laurent Pinchart
2018-09-20 9:02 ` Rafael J. Wysocki
2018-09-20 10:10 ` Laurent Pinchart
2018-09-20 11:00 ` Daniel Vetter
2018-09-20 11:08 ` Laurent Pinchart
2018-09-20 11:49 ` Daniel Vetter
2018-09-12 12:36 ` James Bottomley
2018-09-12 13:38 ` Guenter Roeck
2018-09-12 13:59 ` Tony Lindgren
2018-09-12 10:04 ` Mark Brown
2018-09-12 20:24 ` Steven Rostedt
2018-09-12 20:29 ` Sasha Levin
2018-09-13 0:19 ` Stephen Rothwell
2018-09-13 11:39 ` Mark Brown
2018-09-19 6:27 ` Stephen Rothwell
2018-09-19 17:24 ` Mark Brown
2018-09-19 21:42 ` Stephen Rothwell
2018-09-11 0:49 ` Stephen Rothwell
2018-09-11 1:01 ` Al Viro
2018-09-11 0:47 ` Stephen Rothwell
2018-09-11 17:35 ` Linus Torvalds
2018-09-11 0:43 ` Stephen Rothwell
2018-09-11 16:49 ` Guenter Roeck
2018-09-11 17:47 ` Guenter Roeck
2018-09-11 11:18 ` Mark Brown
2018-09-11 17:02 ` Guenter Roeck
2018-09-11 17:12 ` Jani Nikula
2018-09-11 17:31 ` Mark Brown
2018-09-11 17:41 ` Daniel Vetter
2018-09-11 18:54 ` Mark Brown
2018-09-11 18:03 ` Geert Uytterhoeven
2018-09-11 17:22 ` James Bottomley
2018-09-11 17:56 ` Mark Brown
2018-09-11 18:00 ` James Bottomley
2018-09-11 18:16 ` Mark Brown
2018-09-11 18:07 ` Geert Uytterhoeven
2018-09-12 9:09 ` Dan Carpenter
2018-09-11 17:26 ` Mark Brown
2018-09-11 18:45 ` Steven Rostedt
2018-09-11 18:57 ` Daniel Vetter
2018-09-11 20:15 ` Thomas Gleixner
2018-09-12 9:03 ` Dan Carpenter
2018-09-10 23:01 ` Eduardo Valentin
2018-09-10 23:12 ` Steven Rostedt
2018-09-10 23:32 ` Eduardo Valentin
2018-09-10 23:38 ` Guenter Roeck
2018-09-10 23:38 ` Sasha Levin
2018-09-07 2:33 ` Steven Rostedt
2018-09-07 2:52 ` Guenter Roeck
2018-09-07 14:37 ` Laura Abbott
2018-09-07 15:06 ` Sasha Levin
2018-09-07 15:54 ` Laura Abbott
2018-09-07 16:09 ` Sasha Levin
2018-09-07 20:23 ` Greg KH
2018-09-07 21:13 ` Sasha Levin
2018-09-07 22:27 ` Linus Torvalds
2018-09-07 22:43 ` Guenter Roeck
2018-09-07 22:53 ` Linus Torvalds
2018-09-07 22:57 ` Sasha Levin
2018-09-07 23:52 ` Guenter Roeck
2018-09-08 16:33 ` Greg Kroah-Hartman
2018-09-08 18:35 ` Guenter Roeck
2018-09-10 13:47 ` Mark Brown
2018-09-09 4:36 ` Sasha Levin
2018-09-10 16:20 ` Dan Rue
2018-09-07 21:32 ` Dan Carpenter
2018-09-07 21:43 ` Sasha Levin
2018-09-08 13:20 ` Dan Carpenter
2018-09-10 8:23 ` Jan Kara
2018-09-10 7:53 ` Jan Kara
2018-09-07 3:38 ` Al Viro
2018-09-07 4:27 ` Theodore Y. Ts'o
2018-09-07 5:45 ` Stephen Rothwell
2018-09-07 9:13 ` Daniel Vetter
2018-09-07 11:32 ` Mark Brown
2018-09-07 21:06 ` Mauro Carvalho Chehab
2018-09-08 9:44 ` Laurent Pinchart
2018-09-08 11:48 ` Mauro Carvalho Chehab
2018-09-09 14:26 ` Laurent Pinchart
2018-09-10 22:14 ` Eduardo Valentin
2018-09-07 14:56 ` Sasha Levin
2018-09-07 15:07 ` Jens Axboe
2018-09-07 20:58 ` Mauro Carvalho Chehab
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CA+55aFx9hKVphfU3+aGONbg_wOx9+Ov6HF3Xkq7ZL_pW5m2CLQ@mail.gmail.com \
--to=torvalds@linux-foundation.org \
--cc=ksummit-discuss@lists.linuxfoundation.org \
--cc=rostedt@goodmis.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).