From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753835AbcAVQJJ (ORCPT ); Fri, 22 Jan 2016 11:09:09 -0500 Received: from mail-qg0-f44.google.com ([209.85.192.44]:32778 "EHLO mail-qg0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751388AbcAVQJH (ORCPT ); Fri, 22 Jan 2016 11:09:07 -0500 Date: Fri, 22 Jan 2016 11:09:03 -0500 From: Tejun Heo To: Sasha Levin Cc: Jan Kara , Shaohua Li , LKML , stable@vger.kernel.org, Daniel Bilik , Thomas Gleixner Subject: Re: Crashes with 874bbfe600a6 in 3.18.25 Message-ID: <20160122160903.GH32380@htj.duckdns.org> References: <20160120211926.GJ10810@quack.suse.cz> <20160120213901.GA755895@devbig084.prn1.facebook.com> <20160121095234.GN10810@quack.suse.cz> <56A1817C.10300@oracle.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <56A1817C.10300@oracle.com> User-Agent: Mutt/1.5.24 (2015-08-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org (cc'ing Thomas) On Thu, Jan 21, 2016 at 08:10:20PM -0500, Sasha Levin wrote: > On 01/21/2016 04:52 AM, Jan Kara wrote: > > On Wed 20-01-16 13:39:01, Shaohua Li wrote: > >> On Wed, Jan 20, 2016 at 10:19:26PM +0100, Jan Kara wrote: > >>> Hello, > >>> > >>> a friend of mine started seeing crashes with 3.18.25 kernel - once > >>> appropriate load is put on the machine it crashes within minutes. He > >>> tracked down that reverting commit 874bbfe600a6 (this is the commit ID from > >>> Linus' tree, in stable tree the commit ID is 1e7af294dd03) "workqueue: make > >>> sure delayed work run in local cpu" makes the kernel stable again. I'm > >>> attaching screenshot of the crash - sadly the initial part is missing but > >>> it seems that we crashed when processing timers on otherwise idle CPU. This > >>> is a production machine so experimentation is not easy but if we really > >>> need more information it may be possible to reproduce the issue again and > >>> gather it. > >>> > >>> Anyone has idea what is going on? I was looking into the code for a while > >>> but so far I have no good explanation. It would be good to understand the > >>> cause instead of just blindly reverting the commit from stable tree... > >> > >> Tejun fixed a bug in timer: 22b886dd10180939. is it included in 3.18.25? > > > > That doesn't seem to be included in 3.18-stable although it was CCed to stable. > > Sasha? > > Looks like it requires more than trivial backport (I think). Tejun? The timer migration has changed quite a bit. Given that we've never seen vmstat work crashing in 3.18 era, I wonder whether the right thing to do here is reverting 874bbfe600a6 from 3.18 stable? Thanks. -- tejun