From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755607AbcAWCUo (ORCPT ); Fri, 22 Jan 2016 21:20:44 -0500 Received: from shadbolt.e.decadent.org.uk ([88.96.1.126]:33938 "EHLO shadbolt.e.decadent.org.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755275AbcAWCUn (ORCPT ); Fri, 22 Jan 2016 21:20:43 -0500 Message-ID: <1453515623.3734.156.camel@decadent.org.uk> Subject: Re: Crashes with 874bbfe600a6 in 3.18.25 From: Ben Hutchings To: Tejun Heo , Sasha Levin Cc: Jan Kara , Shaohua Li , LKML , stable@vger.kernel.org, Daniel Bilik , Thomas Gleixner Date: Sat, 23 Jan 2016 02:20:23 +0000 In-Reply-To: <20160122160903.GH32380@htj.duckdns.org> References: <20160120211926.GJ10810@quack.suse.cz> <20160120213901.GA755895@devbig084.prn1.facebook.com> <20160121095234.GN10810@quack.suse.cz> <56A1817C.10300@oracle.com> <20160122160903.GH32380@htj.duckdns.org> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-jx18G651xTw74INMIslm" X-Mailer: Evolution 3.18.3-1 Mime-Version: 1.0 X-SA-Exim-Connect-IP: 192.168.4.247 X-SA-Exim-Mail-From: ben@decadent.org.uk X-SA-Exim-Scanned: No (on shadbolt.decadent.org.uk); SAEximRunCond expanded to false Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org --=-jx18G651xTw74INMIslm Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, 2016-01-22 at 11:09 -0500, Tejun Heo wrote: > (cc'ing Thomas) >=20 > On Thu, Jan 21, 2016 at 08:10:20PM -0500, Sasha Levin wrote: > > On 01/21/2016 04:52 AM, Jan Kara wrote: > > > On Wed 20-01-16 13:39:01, Shaohua Li wrote: > > > > On Wed, Jan 20, 2016 at 10:19:26PM +0100, Jan Kara wrote: > > > > > Hello, > > > > >=20 > > > > > a friend of mine started seeing crashes with 3.18.25 kernel - onc= e > > > > > appropriate load is put on the machine it crashes within minutes.= He > > > > > tracked down that reverting commit 874bbfe600a6 (this is the comm= it ID from > > > > > Linus' tree, in stable tree the commit ID is 1e7af294dd03) "workq= ueue: make > > > > > sure delayed work run in local cpu" makes the kernel stable again= . I'm > > > > > attaching screenshot of the crash - sadly the initial part is mis= sing but > > > > > it seems that we crashed when processing timers on otherwise idle= CPU. This > > > > > is a production machine so experimentation is not easy but if we = really > > > > > need more information it may be possible to reproduce the issue a= gain and > > > > > gather it. > > > > >=20 > > > > > Anyone has idea what is going on? I was looking into the code for= a while > > > > > but so far I have no good explanation.=C2=A0=C2=A0It would be goo= d to understand the > > > > > cause instead of just blindly reverting the commit from stable tr= ee... > > > >=20 > > > > Tejun fixed a bug in timer: 22b886dd10180939. is it included in 3.1= 8.25? > > >=20 > > > That doesn't seem to be included in 3.18-stable although it was CCed = to stable. > > > Sasha? > >=20 > > Looks like it requires more than trivial backport (I think). Tejun? >=20 > The timer migration has changed quite a bit.=C2=A0=C2=A0Given that we've = never > seen vmstat work crashing in 3.18 era, I wonder whether the right > thing to do here is reverting 874bbfe600a6 from 3.18 stable? It's not just 3.18 that has this; 874bbfe600a6 was backported to all stable branches from 3.10 onward. =C2=A0Only the 4.2-ckt branch has 22b886dd10180939. Ben. --=20 Ben Hutchings Life is what happens to you while you're busy making other plans. - John Lenno= n --=-jx18G651xTw74INMIslm Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQIVAwUAVqLjZ+e/yOyVhhEJAQrUHg/+MXJHgUyD8B4zA8/pCMphTL87n2CpKRaQ /xtInBYzTg2P+bxOLSPVFJQsm9ZGOw7XVvUfC766W7QoqaDQH1uKljkSx+/+KY6m tJqq7WD6b4v2Rm7RI9rxm0h5Dcvzf3jbHP9lqwYmZdCEOvWJtY8n+PhIJfUNr37V /FTCaFLJMZjj9TuSkjoEQ16wXi4V0a/UCrvOpIRjBq2GwGmFX6ImMN6U3Gp1mpKe COevTK4EIf831s6gpRh6DxlA68160UlBQT8cf3AMiIS6IsgKqtCFhCnx4ehHKpgY 2PaECQLYEtonUdpjBiEZcsuiTxVPWxnTG9QNxGsKAc6jh9VxsWCOktQENR5yd82Y gHujujGLqRf+6GuUuaxxjlFU5H2MG0+knG+E5GRx3rR3d9JlbjtXsrou71R9Yxtj 9Ww0xs3JybESDeCUKXUZWcfBUC/G312E6GGFRI44sGHJA13SQfYHGnyskP/ELYjR fSYo1vpA1Qa29YtZh1qv5uO0E7ySvcpoBxdimrRZ5I8Yb4SdTvEBlZUokfeBQcZF pAKDlgDzHrDBWui4Yu2UWUxCaIULdUkfVy8jmVQCd8SkTLyofyreF7/9NdAhIubx bX9BZAcZWibjv41+KQzxsEg00ZLZpnHmVJSwmUv6VwGXozN1onD2vntvotigmXj2 HTLErrzsoxM= =YxWN -----END PGP SIGNATURE----- --=-jx18G651xTw74INMIslm--