* [BUG] ODEBUG: assert_init not available (active state 0)
@ 2016-02-03 10:36 Chris Bainbridge
2016-02-05 2:52 ` Zheng, Lv
0 siblings, 1 reply; 9+ messages in thread
From: Chris Bainbridge @ 2016-02-03 10:36 UTC (permalink / raw)
To: lv.zheng; +Cc: robert.moore, rafael.j.wysocki, linux-kernel
v4.5-rc2.
Macbook 2012 (IVB).
ODEBUG reports an error or two at shutdown. Apart from the ODEBUG
error(s), this bug also seems to cause some graphical corruption when
typing in Firefox text fields.
efaed9be998b5ae0afb7458e057e5f4402b43fa0 is the first bad commit
commit efaed9be998b5ae0afb7458e057e5f4402b43fa0
Author: Lv Zheng <lv.zheng@intel.com>
Date: Tue Dec 29 14:03:08 2015 +0800
ACPICA: Events: Enhance acpi_ev_execute_reg_method() to ensure no _REG evaluations can happen during OS early boot stages
ACPICA commit 31178590dde82368fdb0f6b0e466b6c0add96c57
We can ensure no early _REG evaluations by ensuring the following rules in
acpi_ev_execute_reg_method():
1. If an address space handler is installed during early stage,
_REG(CONNECT) evaluations are blocked. This is achieved using
acpi_gbl_reg_methods_enabled which is renamed from
acpi_gbl_reg_methods_executed.
2. If _REG(CONNECT) has never been evalauted for the region object,
_REG(DISCONNECT) evaluations are blocked. This is achieved by a new
region object flag: AOPOBJ_REG_CONNECTED.
Note that, after applying this patch, we can ensure _REG(DISCONNECT) is
always paired to _REG(CONNECT). Lv Zheng
Link: https://github.com/acpica/acpica/commit/31178590
Signed-off-by: Lv Zheng <lv.zheng@intel.com>
Signed-off-by: Bob Moore <robert.moore@intel.com>
Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
:040000 040000 357c50e20ee06ad557c77a2a03d09719dae9dc52 3e0610826dc9cb86454baf06de9392aa31e0aa91 M drivers
[ 34.512758] ------------[ cut here ]------------
[ 34.512765] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263 debug_print_object+0x85/0xa0()
[ 34.512770] ODEBUG: assert_init not available (active state 0) object type: timer_list hint: stub_timer+0x0/0x20
[ 34.512772] Modules linked in:
[ 34.512774] CPU: 0 PID: 4975 Comm: systemd Not tainted 4.4.0-rc7+ #353
[ 34.512776] Hardware name: Apple Inc. MacBookPro10,2/Mac-AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
[ 34.512779] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829 ffff880227a1bdf8
[ 34.512782] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90 ffffffff822514c0
[ 34.512785] ffffffff81f9b4c2 ffffffff8327e288 00007f29190ba700 ffff880227a1be48
[ 34.512786] Call Trace:
[ 34.512790] [<ffffffff814ec829>] dump_stack+0x4b/0x72
[ 34.512793] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
[ 34.512795] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
[ 34.512798] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
[ 34.512800] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
[ 34.512802] [<ffffffff81139830>] ? trace_event_raw_event_tick_stop+0x100/0x100
[ 34.512805] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
[ 34.512807] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
[ 34.512810] [<ffffffff8113afdf>] del_timer+0x1f/0x70
[ 34.512813] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
[ 34.512815] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
[ 34.512819] [<ffffffff812569cf>] sys_sync+0x7f/0x90
[ 34.512824] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
[ 34.512825] ---[ end trace 8fe52cbaccc47e66 ]---
[ 34.512830] ------------[ cut here ]------------
[ 34.512833] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263 debug_print_object+0x85/0xa0()
[ 34.512835] ODEBUG: assert_init not available (active state 0) object type: timer_list hint: stub_timer+0x0/0x20
[ 34.512836] Modules linked in:
[ 34.512838] CPU: 0 PID: 4975 Comm: systemd Tainted: G W 4.4.0-rc7+ #353
[ 34.512839] Hardware name: Apple Inc. MacBookPro10,2/Mac-AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
[ 34.512842] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829 ffff880227a1bdf8
[ 34.512845] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90 ffffffff822514c0
[ 34.512847] ffffffff81f9b4c2 ffffffff8331f388 00007f29190ba700 ffff880227a1be48
[ 34.512848] Call Trace:
[ 34.512850] [<ffffffff814ec829>] dump_stack+0x4b/0x72
[ 34.512853] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
[ 34.512855] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
[ 34.512857] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
[ 34.512859] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
[ 34.512861] [<ffffffff81139830>] ? trace_event_raw_event_tick_stop+0x100/0x100
[ 34.512864] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
[ 34.512865] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
[ 34.512868] [<ffffffff8113afdf>] del_timer+0x1f/0x70
[ 34.512869] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
[ 34.512871] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
[ 34.512873] [<ffffffff812569cf>] sys_sync+0x7f/0x90
[ 34.512876] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
[ 34.512877] ---[ end trace 8fe52cbaccc47e67 ]---
^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [BUG] ODEBUG: assert_init not available (active state 0)
2016-02-03 10:36 [BUG] ODEBUG: assert_init not available (active state 0) Chris Bainbridge
@ 2016-02-05 2:52 ` Zheng, Lv
2016-02-14 10:59 ` Chris Bainbridge
0 siblings, 1 reply; 9+ messages in thread
From: Zheng, Lv @ 2016-02-05 2:52 UTC (permalink / raw)
To: Chris Bainbridge
Cc: Moore, Robert, Wysocki, Rafael J, linux-kernel@vger.kernel.org
Hi,
The below change is spec compliant.
I think you should first test again after the RefOrName stuffs reverted.
Then, if you still can see the issues, the story is:
===
We are enabling correct initialization order and table parsing facilities.
These patches are part of them.
===
So you could wait for just several days to test it again.
And report here if you can still see issues.
Thanks and best regards
-Lv
> From: Chris Bainbridge [mailto:chris.bainbridge@gmail.com]
> Sent: Wednesday, February 3, 2016 6:37 PM
>
> v4.5-rc2.
> Macbook 2012 (IVB).
>
> ODEBUG reports an error or two at shutdown. Apart from the ODEBUG
> error(s), this bug also seems to cause some graphical corruption when
> typing in Firefox text fields.
>
>
> efaed9be998b5ae0afb7458e057e5f4402b43fa0 is the first bad commit
> commit efaed9be998b5ae0afb7458e057e5f4402b43fa0
> Author: Lv Zheng <lv.zheng@intel.com>
> Date: Tue Dec 29 14:03:08 2015 +0800
>
> ACPICA: Events: Enhance acpi_ev_execute_reg_method() to ensure no _REG
> evaluations can happen during OS early boot stages
>
> ACPICA commit 31178590dde82368fdb0f6b0e466b6c0add96c57
>
> We can ensure no early _REG evaluations by ensuring the following rules in
> acpi_ev_execute_reg_method():
> 1. If an address space handler is installed during early stage,
> _REG(CONNECT) evaluations are blocked. This is achieved using
> acpi_gbl_reg_methods_enabled which is renamed from
> acpi_gbl_reg_methods_executed.
> 2. If _REG(CONNECT) has never been evalauted for the region object,
> _REG(DISCONNECT) evaluations are blocked. This is achieved by a new
> region object flag: AOPOBJ_REG_CONNECTED.
> Note that, after applying this patch, we can ensure _REG(DISCONNECT) is
> always paired to _REG(CONNECT). Lv Zheng
>
> Link: https://github.com/acpica/acpica/commit/31178590
> Signed-off-by: Lv Zheng <lv.zheng@intel.com>
> Signed-off-by: Bob Moore <robert.moore@intel.com>
> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
>
> :040000 040000 357c50e20ee06ad557c77a2a03d09719dae9dc52
> 3e0610826dc9cb86454baf06de9392aa31e0aa91 M drivers
>
>
> [ 34.512758] ------------[ cut here ]------------
> [ 34.512765] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263
> debug_print_object+0x85/0xa0()
> [ 34.512770] ODEBUG: assert_init not available (active state 0) object type:
> timer_list hint: stub_timer+0x0/0x20
> [ 34.512772] Modules linked in:
> [ 34.512774] CPU: 0 PID: 4975 Comm: systemd Not tainted 4.4.0-rc7+ #353
> [ 34.512776] Hardware name: Apple Inc. MacBookPro10,2/Mac-
> AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
> [ 34.512779] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829
> ffff880227a1bdf8
> [ 34.512782] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90
> ffffffff822514c0
> [ 34.512785] ffffffff81f9b4c2 ffffffff8327e288 00007f29190ba700
> ffff880227a1be48
> [ 34.512786] Call Trace:
> [ 34.512790] [<ffffffff814ec829>] dump_stack+0x4b/0x72
> [ 34.512793] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
> [ 34.512795] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
> [ 34.512798] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
> [ 34.512800] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
> [ 34.512802] [<ffffffff81139830>] ?
> trace_event_raw_event_tick_stop+0x100/0x100
> [ 34.512805] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
> [ 34.512807] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
> [ 34.512810] [<ffffffff8113afdf>] del_timer+0x1f/0x70
> [ 34.512813] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
> [ 34.512815] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
> [ 34.512819] [<ffffffff812569cf>] sys_sync+0x7f/0x90
> [ 34.512824] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
> [ 34.512825] ---[ end trace 8fe52cbaccc47e66 ]---
> [ 34.512830] ------------[ cut here ]------------
> [ 34.512833] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263
> debug_print_object+0x85/0xa0()
> [ 34.512835] ODEBUG: assert_init not available (active state 0) object type:
> timer_list hint: stub_timer+0x0/0x20
> [ 34.512836] Modules linked in:
> [ 34.512838] CPU: 0 PID: 4975 Comm: systemd Tainted: G W 4.4.0-rc7+
> #353
> [ 34.512839] Hardware name: Apple Inc. MacBookPro10,2/Mac-
> AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
> [ 34.512842] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829
> ffff880227a1bdf8
> [ 34.512845] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90
> ffffffff822514c0
> [ 34.512847] ffffffff81f9b4c2 ffffffff8331f388 00007f29190ba700
> ffff880227a1be48
> [ 34.512848] Call Trace:
> [ 34.512850] [<ffffffff814ec829>] dump_stack+0x4b/0x72
> [ 34.512853] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
> [ 34.512855] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
> [ 34.512857] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
> [ 34.512859] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
> [ 34.512861] [<ffffffff81139830>] ?
> trace_event_raw_event_tick_stop+0x100/0x100
> [ 34.512864] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
> [ 34.512865] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
> [ 34.512868] [<ffffffff8113afdf>] del_timer+0x1f/0x70
> [ 34.512869] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
> [ 34.512871] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
> [ 34.512873] [<ffffffff812569cf>] sys_sync+0x7f/0x90
> [ 34.512876] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
> [ 34.512877] ---[ end trace 8fe52cbaccc47e67 ]---
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG] ODEBUG: assert_init not available (active state 0)
2016-02-05 2:52 ` Zheng, Lv
@ 2016-02-14 10:59 ` Chris Bainbridge
2016-02-23 2:18 ` Zheng, Lv
0 siblings, 1 reply; 9+ messages in thread
From: Chris Bainbridge @ 2016-02-14 10:59 UTC (permalink / raw)
To: Zheng, Lv; +Cc: Moore, Robert, Wysocki, Rafael J, linux-kernel@vger.kernel.org
On 5 February 2016 at 02:52, Zheng, Lv <lv.zheng@intel.com> wrote:
>
> So you could wait for just several days to test it again.
> And report here if you can still see issues.
I didn't notice any revert last week, is there something specific that
I should test? I just noticed that there are graphical glitches in
Firefox on Youtube as well.
^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: [BUG] ODEBUG: assert_init not available (active state 0)
2016-02-14 10:59 ` Chris Bainbridge
@ 2016-02-23 2:18 ` Zheng, Lv
2016-02-27 19:08 ` Chris Bainbridge
0 siblings, 1 reply; 9+ messages in thread
From: Zheng, Lv @ 2016-02-23 2:18 UTC (permalink / raw)
To: Chris Bainbridge
Cc: Moore, Robert, Wysocki, Rafael J, linux-kernel@vger.kernel.org
Hi,
> From: Chris Bainbridge [mailto:chris.bainbridge@gmail.com]
> Subject: Re: [BUG] ODEBUG: assert_init not available (active state 0)
>
> On 5 February 2016 at 02:52, Zheng, Lv <lv.zheng@intel.com> wrote:
> >
> > So you could wait for just several days to test it again.
> > And report here if you can still see issues.
>
> I didn't notice any revert last week, is there something specific that
> I should test? I just noticed that there are graphical glitches in
> Firefox on Youtube as well.
[Lv Zheng]
In fact, I don't understand what your report is.
What did you mean by referring the following commit:
> ACPICA: Events: Enhance acpi_ev_execute_reg_method() to ensure no _REG
> evaluations can happen during OS early boot stages
Was this a bisection result for an issue?
And what was the issue?
If the issue is the following warning messages:
> [ 34.512758] ------------[ cut here ]------------
> [ 34.512765] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263
> debug_print_object+0x85/0xa0()
> [ 34.512770] ODEBUG: assert_init not available (active state 0) object type:
> timer_list hint: stub_timer+0x0/0x20
> [ 34.512772] Modules linked in:
> [ 34.512774] CPU: 0 PID: 4975 Comm: systemd Not tainted 4.4.0-rc7+ #353
> [ 34.512776] Hardware name: Apple Inc. MacBookPro10,2/Mac-
> AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
> [ 34.512779] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829
> ffff880227a1bdf8
> [ 34.512782] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90
> ffffffff822514c0
> [ 34.512785] ffffffff81f9b4c2 ffffffff8327e288 00007f29190ba700
> ffff880227a1be48
> [ 34.512786] Call Trace:
> [ 34.512790] [<ffffffff814ec829>] dump_stack+0x4b/0x72
> [ 34.512793] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
> [ 34.512795] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
> [ 34.512798] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
> [ 34.512800] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
> [ 34.512802] [<ffffffff81139830>] ?
> trace_event_raw_event_tick_stop+0x100/0x100
> [ 34.512805] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
> [ 34.512807] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
> [ 34.512810] [<ffffffff8113afdf>] del_timer+0x1f/0x70
> [ 34.512813] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
> [ 34.512815] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
> [ 34.512819] [<ffffffff812569cf>] sys_sync+0x7f/0x90
> [ 34.512824] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
> [ 34.512825] ---[ end trace 8fe52cbaccc47e66 ]---
Our investigation result shows this is caused by a multiple deletion on the same Linux kernel timer.
And the commit you reported shouldn't be the culprit.
So it looks to me like a wrong bisection result if it was a bisection result.
While the following commit may trigger such breakage:
> ACPICA: Parser: Fix for SuperName method invocation
So I think you need to test again after reverting this patch.
The patch is actually reverted in ACPICA 20160212 release.
You can use the following branch where ACPICA 20160212 release is applied:
https://git.kernel.org/cgit/linux/kernel/git/rafael/linux-pm.git/?h=acpica-test
test it again to confirm if such breakage still can be seen.
If the issue is some graphical glitches.
As I said, the commit is just a part of whole series that try to make Linux ACPICA initialization sequences correctly ordered.
It, along with other patches try to make Linux working as spec compliant (also proven by Windows behavior).
So you really need to apply more patches on top of acpica-test branch to confirm again.
For example, patches from this series:
http://www.spinics.net/lists/linux-acpi/msg63550.html
Especially this one:
http://www.spinics.net/lists/linux-acpi/msg63558.html
There are simply so many issues fixed by the series.
The series contains many reversions reverting old wrong fixes, and fixes those issues using different approaches that can be spec compliant.
The wrong fixes are proven to be wrong, but they've been there for so many years, making Linux working on those old platforms.
Thus without fixing them all together, we may see regressions.
And it is likely that there are still cases not covered by this series.
Thus we may need your test support to learn the new cases.
Probably also need graphic developers' help to root cause your issue.
Let me just file a kernel Bugzilla bug, so that we can communicate there to do more investigation.
https://bugzilla.kernel.org/show_bug.cgi?id=112911
Thanks and best regards
-Lv
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [BUG] ODEBUG: assert_init not available (active state 0)
2016-02-23 2:18 ` Zheng, Lv
@ 2016-02-27 19:08 ` Chris Bainbridge
2016-03-05 15:11 ` [PATCH] ACPICA: Events: Execute some _REG methods in early boot Chris Bainbridge
0 siblings, 1 reply; 9+ messages in thread
From: Chris Bainbridge @ 2016-02-27 19:08 UTC (permalink / raw)
To: Zheng, Lv; +Cc: Moore, Robert, Wysocki, Rafael J, linux-kernel@vger.kernel.org
On Tue, Feb 23, 2016 at 02:18:22AM +0000, Zheng, Lv wrote:
>
> In fact, I don't understand what your report is.
> What did you mean by referring the following commit:
> > ACPICA: Events: Enhance acpi_ev_execute_reg_method() to ensure no _REG
> > evaluations can happen during OS early boot stages
> Was this a bisection result for an issue?
> And what was the issue?
Yes it was a bisection result for the ODEBUG errors.
> If the issue is the following warning messages:
> > [ 34.512758] ------------[ cut here ]------------
> > [ 34.512765] WARNING: CPU: 0 PID: 4975 at lib/debugobjects.c:263
> > debug_print_object+0x85/0xa0()
> > [ 34.512770] ODEBUG: assert_init not available (active state 0) object type:
> > timer_list hint: stub_timer+0x0/0x20
> > [ 34.512772] Modules linked in:
> > [ 34.512774] CPU: 0 PID: 4975 Comm: systemd Not tainted 4.4.0-rc7+ #353
> > [ 34.512776] Hardware name: Apple Inc. MacBookPro10,2/Mac-
> > AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
> > [ 34.512779] ffffffff81f9b41c ffff880227a1bdb0 ffffffff814ec829
> > ffff880227a1bdf8
> > [ 34.512782] ffff880227a1bde8 ffffffff810cd831 ffff880227a1be90
> > ffffffff822514c0
> > [ 34.512785] ffffffff81f9b4c2 ffffffff8327e288 00007f29190ba700
> > ffff880227a1be48
> > [ 34.512786] Call Trace:
> > [ 34.512790] [<ffffffff814ec829>] dump_stack+0x4b/0x72
> > [ 34.512793] [<ffffffff810cd831>] warn_slowpath_common+0x81/0xc0
> > [ 34.512795] [<ffffffff810cd8b7>] warn_slowpath_fmt+0x47/0x50
> > [ 34.512798] [<ffffffff811398c2>] ? do_init_timer+0x52/0x60
> > [ 34.512800] [<ffffffff8150a015>] debug_print_object+0x85/0xa0
> > [ 34.512802] [<ffffffff81139830>] ?
> > trace_event_raw_event_tick_stop+0x100/0x100
> > [ 34.512805] [<ffffffff8150ac38>] debug_object_assert_init+0xf8/0x130
> > [ 34.512807] [<ffffffff8111a5bd>] ? trace_hardirqs_on+0xd/0x10
> > [ 34.512810] [<ffffffff8113afdf>] del_timer+0x1f/0x70
> > [ 34.512813] [<ffffffff811c2331>] laptop_sync_completion+0x61/0x100
> > [ 34.512815] [<ffffffff811c22d0>] ? laptop_io_completion+0x30/0x30
> > [ 34.512819] [<ffffffff812569cf>] sys_sync+0x7f/0x90
> > [ 34.512824] [<ffffffff81b88e17>] entry_SYSCALL_64_fastpath+0x12/0x6f
> > [ 34.512825] ---[ end trace 8fe52cbaccc47e66 ]---
>
> Our investigation result shows this is caused by a multiple deletion on the same Linux kernel timer.
> And the commit you reported shouldn't be the culprit.
> So it looks to me like a wrong bisection result if it was a bisection result.
I've checked again and the bisect result is correct. It could be that
the issue was always there, but this error is new on my system at least.
> While the following commit may trigger such breakage:
> > ACPICA: Parser: Fix for SuperName method invocation
> So I think you need to test again after reverting this patch.
> The patch is actually reverted in ACPICA 20160212 release.
> You can use the following branch where ACPICA 20160212 release is applied:
> https://git.kernel.org/cgit/linux/kernel/git/rafael/linux-pm.git/?h=acpica-test
> test it again to confirm if such breakage still can be seen.
That branch also gives the ODEBUG error.
> If the issue is some graphical glitches.
> As I said, the commit is just a part of whole series that try to make Linux ACPICA initialization sequences correctly ordered.
> It, along with other patches try to make Linux working as spec compliant (also proven by Windows behavior).
> So you really need to apply more patches on top of acpica-test branch to confirm again.
> For example, patches from this series:
> http://www.spinics.net/lists/linux-acpi/msg63550.html
Patch series does not apply cleanly (3 4 5 6 12 14 fail)
> Especially this one:
> http://www.spinics.net/lists/linux-acpi/msg63558.html
No change, ODEBUG error still occurs.
To sum up test results:
v4.4-rc7-48-g849c25719ac6
- ok
v4.4-rc7-49-gefaed9be998b
- odebug error
acpia-test
- odebug error
http://www.spinics.net/lists/linux-acpi/msg63550.html
- patch series does not apply cleanly (3 4 5 6 12 14 fail)
http://www.spinics.net/lists/linux-acpi/msg63558.html (patch7)
- odebug error
^ permalink raw reply [flat|nested] 9+ messages in thread
* [PATCH] ACPICA: Events: Execute some _REG methods in early boot
2016-02-27 19:08 ` Chris Bainbridge
@ 2016-03-05 15:11 ` Chris Bainbridge
2016-03-07 6:36 ` Zheng, Lv
0 siblings, 1 reply; 9+ messages in thread
From: Chris Bainbridge @ 2016-03-05 15:11 UTC (permalink / raw)
To: lv.zheng
Cc: Chris Bainbridge, robert.moore, rafael.j.wysocki, linux-acpi,
linux-kernel
The regression caused _REG methods to not be run in early boot for all
space IDs, but a removed comment stated that _REG methods should be
executed for IDs like embedded_controller. Before the regression this
was the case:
[ 0.230469] Executing Method \_SB.PCI0.LPCB.EC._REG
[ 0.230531] Initializing Region \GNVS
[ 0.230607] Initializing Region \_SB.PCI0.LPCB.EC.ECOR
[ 0.231043] Initializing Region \_SB.PCI0.IGPU.IGDM
After the regression the initialisation is not done and ODEBUG warnings
are shown at boot and/or shutdown:
[ 6.676570] WARNING: CPU: 0 PID: 3317 at lib/debugobjects.c:263 debug_print_object+0x85/0xa0()
[ 6.676576] ODEBUG: assert_init not available (active state 0) object type: timer_list hint: stub_timer+0x0/0x20
[ 6.676578] Modules linked in:
[ 6.676582] CPU: 0 PID: 3317 Comm: ccpd Not tainted 4.5.0-rc6 #509
[ 6.676584] Hardware name: Apple Inc. MacBookPro10,2/Mac-AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
[ 6.676586] 0000000000000000 ffff880256543db0 ffffffff814802b5 ffff880256543df8
[ 6.676590] ffffffff81f40dbd ffff880256543de8 ffffffff810bd0ec ffff880256543e90
[ 6.676594] ffffffff822532c0 ffffffff81f40e63 ffffffff83138c88 0000000001fdf040
[ 6.676598] Call Trace:
[ 6.676603] [<ffffffff814802b5>] dump_stack+0x67/0x92
[ 6.676608] [<ffffffff810bd0ec>] warn_slowpath_common+0x7c/0xb0
[ 6.676611] [<ffffffff810bd167>] warn_slowpath_fmt+0x47/0x50
[ 6.676614] [<ffffffff8149d7a5>] debug_print_object+0x85/0xa0
[ 6.676616] [<ffffffff8111f440>] ? cascade+0x70/0x70
[ 6.676620] [<ffffffff8149e398>] debug_object_assert_init+0xf8/0x130
[ 6.676624] [<ffffffff811038cd>] ? trace_hardirqs_on+0xd/0x10
[ 6.676626] [<ffffffff8111fcff>] del_timer+0x1f/0x70
[ 6.676631] [<ffffffff811852f1>] laptop_sync_completion+0x61/0x100
[ 6.676633] [<ffffffff81185290>] ? laptop_io_completion+0x30/0x30
[ 6.676637] [<ffffffff81212c9f>] sys_sync+0x7f/0x90
[ 6.676641] [<ffffffff81ad0d97>] entry_SYSCALL_64_fastpath+0x12/0x6f
Fixes: efaed9be998b ("ACPICA: Events: Enhance acpi_ev_execute_reg_method() to ensure no _REG evaluations can happen during OS early boot stages")
Link: https://lkml.org/lkml/2016/2/3/273
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=112911
Signed-off-by: Chris Bainbridge <chris.bainbridge@gmail.com>
---
drivers/acpi/acpica/evregion.c | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/drivers/acpi/acpica/evregion.c b/drivers/acpi/acpica/evregion.c
index 47092b4d633c..d0b02ef0effa 100644
--- a/drivers/acpi/acpica/evregion.c
+++ b/drivers/acpi/acpica/evregion.c
@@ -590,6 +590,7 @@ acpi_ev_execute_reg_method(union acpi_operand_object *region_obj, u32 function)
union acpi_operand_object *args[3];
union acpi_operand_object *region_obj2;
acpi_status status;
+ bool sp;
ACPI_FUNCTION_TRACE(ev_execute_reg_method);
@@ -598,9 +599,28 @@ acpi_ev_execute_reg_method(union acpi_operand_object *region_obj, u32 function)
return_ACPI_STATUS(AE_NOT_EXIST);
}
+ /*
+ * For the default space_IDs, (the IDs for which there are default region handlers
+ * installed) Only execute the _REG methods if the global initialization _REG
+ * methods have already been run (via acpi_initialize_objects). In other words,
+ * we will defer the execution of the _REG methods for these space_IDs until
+ * execution of acpi_initialize_objects. This is done because we need the handlers
+ * for the default spaces (mem/io/pci/table) to be installed before we can run
+ * any control methods (or _REG methods). There is known BIOS code that depends
+ * on this.
+ *
+ * For all other space_IDs, we can safely execute the _REG methods immediately.
+ * This means that for IDs like embedded_controller, this function should be called
+ * only after acpi_enable_subsystem has been called.
+ */
+
+ sp = (region_obj->region.space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY ||
+ region_obj->region.space_id == ACPI_ADR_SPACE_SYSTEM_IO ||
+ region_obj->region.space_id == ACPI_ADR_SPACE_PCI_CONFIG ||
+ region_obj->region.space_id == ACPI_ADR_SPACE_DATA_TABLE);
if (region_obj2->extra.method_REG == NULL ||
region_obj->region.handler == NULL ||
- !acpi_gbl_reg_methods_enabled) {
+ (sp && !acpi_gbl_reg_methods_enabled)) {
return_ACPI_STATUS(AE_OK);
}
--
2.1.4
^ permalink raw reply related [flat|nested] 9+ messages in thread
* RE: [PATCH] ACPICA: Events: Execute some _REG methods in early boot
2016-03-05 15:11 ` [PATCH] ACPICA: Events: Execute some _REG methods in early boot Chris Bainbridge
@ 2016-03-07 6:36 ` Zheng, Lv
2016-03-07 10:19 ` Chris Bainbridge
0 siblings, 1 reply; 9+ messages in thread
From: Zheng, Lv @ 2016-03-07 6:36 UTC (permalink / raw)
To: Chris Bainbridge
Cc: Moore, Robert, Wysocki, Rafael J, linux-acpi@vger.kernel.org,
linux-kernel@vger.kernel.org
Hi,
First of all, why don't you respond on the kernel bugzilla?
Posting a fix here directly without communication doesn't look like a constructive help but more like a destructive attack to me.
> From: linux-acpi-owner@vger.kernel.org [mailto:linux-acpi-
> Subject: [PATCH] ACPICA: Events: Execute some _REG methods in early boot
>
> The regression caused _REG methods to not be run in early boot for all
> space IDs, but a removed comment stated that _REG methods should be
> executed for IDs like embedded_controller. Before the regression this
> was the case:
>
> [ 0.230469] Executing Method \_SB.PCI0.LPCB.EC._REG
> [ 0.230531] Initializing Region \GNVS
> [ 0.230607] Initializing Region \_SB.PCI0.LPCB.EC.ECOR
> [ 0.231043] Initializing Region \_SB.PCI0.IGPU.IGDM
[Lv Zheng]
As I said in the previous reply, this is the known issue and can be fixed by applying the whole series.
Especially this commit:
https://patchwork.kernel.org/patch/8241421/
That's why I asked you to test again by applying the whole series.
And that's why after having not seen your response for so long time, we prepared a test branch and was waiting for your response.
You need to post acpidump there to have the issue root caused so that more accurate fix can be generated.
>
> After the regression the initialisation is not done and ODEBUG warnings
> are shown at boot and/or shutdown:
>
> [ 6.676570] WARNING: CPU: 0 PID: 3317 at lib/debugobjects.c:263
> debug_print_object+0x85/0xa0()
> [ 6.676576] ODEBUG: assert_init not available (active state 0) object type:
> timer_list hint: stub_timer+0x0/0x20
> [ 6.676578] Modules linked in:
> [ 6.676582] CPU: 0 PID: 3317 Comm: ccpd Not tainted 4.5.0-rc6 #509
> [ 6.676584] Hardware name: Apple Inc. MacBookPro10,2/Mac-
> AFD8A9D944EA4843, BIOS MBP102.88Z.0106.B0A.1509130955 09/13/2015
> [ 6.676586] 0000000000000000 ffff880256543db0 ffffffff814802b5
> ffff880256543df8
> [ 6.676590] ffffffff81f40dbd ffff880256543de8 ffffffff810bd0ec
> ffff880256543e90
> [ 6.676594] ffffffff822532c0 ffffffff81f40e63 ffffffff83138c88
> 0000000001fdf040
> [ 6.676598] Call Trace:
> [ 6.676603] [<ffffffff814802b5>] dump_stack+0x67/0x92
> [ 6.676608] [<ffffffff810bd0ec>] warn_slowpath_common+0x7c/0xb0
> [ 6.676611] [<ffffffff810bd167>] warn_slowpath_fmt+0x47/0x50
> [ 6.676614] [<ffffffff8149d7a5>] debug_print_object+0x85/0xa0
> [ 6.676616] [<ffffffff8111f440>] ? cascade+0x70/0x70
> [ 6.676620] [<ffffffff8149e398>] debug_object_assert_init+0xf8/0x130
> [ 6.676624] [<ffffffff811038cd>] ? trace_hardirqs_on+0xd/0x10
> [ 6.676626] [<ffffffff8111fcff>] del_timer+0x1f/0x70
> [ 6.676631] [<ffffffff811852f1>] laptop_sync_completion+0x61/0x100
> [ 6.676633] [<ffffffff81185290>] ? laptop_io_completion+0x30/0x30
> [ 6.676637] [<ffffffff81212c9f>] sys_sync+0x7f/0x90
> [ 6.676641] [<ffffffff81ad0d97>] entry_SYSCALL_64_fastpath+0x12/0x6f
>
> Fixes: efaed9be998b ("ACPICA: Events: Enhance acpi_ev_execute_reg_method()
> to ensure no _REG evaluations can happen during OS early boot stages")
> Link: https://lkml.org/lkml/2016/2/3/273
> Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=112911
> Signed-off-by: Chris Bainbridge <chris.bainbridge@gmail.com>
> ---
> drivers/acpi/acpica/evregion.c | 22 +++++++++++++++++++++-
> 1 file changed, 21 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/acpi/acpica/evregion.c b/drivers/acpi/acpica/evregion.c
> index 47092b4d633c..d0b02ef0effa 100644
> --- a/drivers/acpi/acpica/evregion.c
> +++ b/drivers/acpi/acpica/evregion.c
> @@ -590,6 +590,7 @@ acpi_ev_execute_reg_method(union
> acpi_operand_object *region_obj, u32 function)
> union acpi_operand_object *args[3];
> union acpi_operand_object *region_obj2;
> acpi_status status;
> + bool sp;
>
> ACPI_FUNCTION_TRACE(ev_execute_reg_method);
>
> @@ -598,9 +599,28 @@ acpi_ev_execute_reg_method(union
> acpi_operand_object *region_obj, u32 function)
> return_ACPI_STATUS(AE_NOT_EXIST);
> }
>
> + /*
> + * For the default space_IDs, (the IDs for which there are default region
> handlers
> + * installed) Only execute the _REG methods if the global initialization
> _REG
> + * methods have already been run (via acpi_initialize_objects). In other
> words,
> + * we will defer the execution of the _REG methods for these
> space_IDs until
> + * execution of acpi_initialize_objects. This is done because we need
> the handlers
> + * for the default spaces (mem/io/pci/table) to be installed before we
> can run
> + * any control methods (or _REG methods). There is known BIOS code
> that depends
> + * on this.
> + *
> + * For all other space_IDs, we can safely execute the _REG methods
> immediately.
> + * This means that for IDs like embedded_controller, this function
> should be called
> + * only after acpi_enable_subsystem has been called.
> + */
> +
> + sp = (region_obj->region.space_id ==
> ACPI_ADR_SPACE_SYSTEM_MEMORY ||
> + region_obj->region.space_id == ACPI_ADR_SPACE_SYSTEM_IO ||
> + region_obj->region.space_id == ACPI_ADR_SPACE_PCI_CONFIG ||
> + region_obj->region.space_id == ACPI_ADR_SPACE_DATA_TABLE);
> if (region_obj2->extra.method_REG == NULL ||
> region_obj->region.handler == NULL ||
> - !acpi_gbl_reg_methods_enabled) {
> + (sp && !acpi_gbl_reg_methods_enabled)) {
> return_ACPI_STATUS(AE_OK);
> }
>
[Lv Zheng]
The above fix looks hackish to me.
IMO, if you want to stop regressions that are triggered by this commit.
A simpler and proper way would be to move acpi_gbl_reg_method_enabled = TRUE to the end of acpi_load_tables().
So that when the order of table loading and ECDT probing is corrected, the correct logic can still apply.
I don't have ECDT platforms in hand to confirm.
Can you help to just give it a try?
Thanks and best regards
-Lv
> --
> 2.1.4
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH] ACPICA: Events: Execute some _REG methods in early boot
2016-03-07 6:36 ` Zheng, Lv
@ 2016-03-07 10:19 ` Chris Bainbridge
2016-03-08 0:54 ` Zheng, Lv
0 siblings, 1 reply; 9+ messages in thread
From: Chris Bainbridge @ 2016-03-07 10:19 UTC (permalink / raw)
To: Zheng, Lv
Cc: Moore, Robert, Wysocki, Rafael J, linux-acpi@vger.kernel.org,
linux-kernel@vger.kernel.org
On Mon, Mar 07, 2016 at 06:36:05AM +0000, Zheng, Lv wrote:
> Hi,
>
> First of all, why don't you respond on the kernel bugzilla?
> Posting a fix here directly without communication doesn't look like a constructive help but more like a destructive attack to me.
Sorry if it appears to be an attack. It wasn't meant that way. I was
under the impression that email is the preferred means of communication
for kernel development. I'm not sure whether other developers actively
monitor bugzilla reports (some do, but bug trackers are often graveyards
for bug reports and it's easy for communications to be missed).
> As I said in the previous reply, this is the known issue and can be fixed by applying the whole series.
> Especially this commit:
> https://patchwork.kernel.org/patch/8241421/
> That's why I asked you to test again by applying the whole series.
> And that's why after having not seen your response for so long time, we prepared a test branch and was waiting for your response.
I already replied 9 days ago: https://lkml.org/lkml/2016/2/27/164
The suggested patch did not fix the issue and the patch series did not
apply cleanly.
(btw I'm not paid for this work so I tend to handle it in batches when I
have spare time, which is why you may see replies delayed to weekends etc.)
> You need to post acpidump there to have the issue root caused so that more accurate fix can be generated.
I already sent an acpidump for this system to you and Robert (email 3rd Feb).
> The above fix looks hackish to me.
> IMO, if you want to stop regressions that are triggered by this commit.
> A simpler and proper way would be to move acpi_gbl_reg_method_enabled = TRUE to the end of acpi_load_tables().
> So that when the order of table loading and ECDT probing is corrected, the correct logic can still apply.
>
> I don't have ECDT platforms in hand to confirm.
> Can you help to just give it a try?
Yes the fix may be hackish, but it served the purpose of correctly
identifying the bug. Your suggestion works fine, for reference the
tested patch was:
diff --git a/drivers/acpi/acpica/tbxfload.c b/drivers/acpi/acpica/tbxfload.c
index 278666e39563..9136d7250022 100644
--- a/drivers/acpi/acpica/tbxfload.c
+++ b/drivers/acpi/acpica/tbxfload.c
@@ -83,6 +83,7 @@ acpi_status __init acpi_load_tables(void)
"While loading namespace from ACPI tables"));
}
+ acpi_gbl_reg_methods_enabled = TRUE;
return_ACPI_STATUS(status);
}
diff --git a/drivers/acpi/acpica/utxfinit.c b/drivers/acpi/acpica/utxfinit.c
index 721b87cce908..2c0491038068 100644
--- a/drivers/acpi/acpica/utxfinit.c
+++ b/drivers/acpi/acpica/utxfinit.c
@@ -267,7 +267,6 @@ acpi_status __init acpi_initialize_objects(u32 flags)
* initialized, even if they contain executable AML (see the call to
* acpi_ns_initialize_objects below).
*/
- acpi_gbl_reg_methods_enabled = TRUE;
if (!(flags & ACPI_NO_ADDRESS_SPACE_INIT)) {
ACPI_DEBUG_PRINT((ACPI_DB_EXEC,
"[Init] Executing _REG OpRegion methods\n"));
With this patch applied the ODEBUG errors do not occur.
Thanks.
^ permalink raw reply related [flat|nested] 9+ messages in thread
* RE: [PATCH] ACPICA: Events: Execute some _REG methods in early boot
2016-03-07 10:19 ` Chris Bainbridge
@ 2016-03-08 0:54 ` Zheng, Lv
0 siblings, 0 replies; 9+ messages in thread
From: Zheng, Lv @ 2016-03-08 0:54 UTC (permalink / raw)
To: Chris Bainbridge
Cc: Moore, Robert, Wysocki, Rafael J, linux-acpi@vger.kernel.org,
linux-kernel@vger.kernel.org
Hi, Chris
> From: linux-acpi-owner@vger.kernel.org [mailto:linux-acpi-
> Subject: Re: [PATCH] ACPICA: Events: Execute some _REG methods in early
> boot
>
> On Mon, Mar 07, 2016 at 06:36:05AM +0000, Zheng, Lv wrote:
> > Hi,
> >
> > First of all, why don't you respond on the kernel bugzilla?
> > Posting a fix here directly without communication doesn't look like a
> constructive help but more like a destructive attack to me.
>
> Sorry if it appears to be an attack. It wasn't meant that way. I was
> under the impression that email is the preferred means of communication
> for kernel development. I'm not sure whether other developers actively
> monitor bugzilla reports (some do, but bug trackers are often graveyards
> for bug reports and it's easy for communications to be missed).
>
[Lv Zheng]
Apologies. It's my fault.
I didn't see your last reply.
> > As I said in the previous reply, this is the known issue and can be fixed by
> applying the whole series.
> > Especially this commit:
> > https://patchwork.kernel.org/patch/8241421/
> > That's why I asked you to test again by applying the whole series.
> > And that's why after having not seen your response for so long time, we
> prepared a test branch and was waiting for your response.
>
> I already replied 9 days ago: https://lkml.org/lkml/2016/2/27/164
> The suggested patch did not fix the issue and the patch series did not
> apply cleanly.
>
> (btw I'm not paid for this work so I tend to handle it in batches when I
> have spare time, which is why you may see replies delayed to weekends etc.)
>
[Lv Zheng]
The problem is the reply didn't arrive the proper mailbox here.
> > You need to post acpidump there to have the issue root caused so that more
> accurate fix can be generated.
>
> I already sent an acpidump for this system to you and Robert (email 3rd Feb).
>
> > The above fix looks hackish to me.
> > IMO, if you want to stop regressions that are triggered by this commit.
> > A simpler and proper way would be to move acpi_gbl_reg_method_enabled =
> TRUE to the end of acpi_load_tables().
> > So that when the order of table loading and ECDT probing is corrected, the
> correct logic can still apply.
> >
> > I don't have ECDT platforms in hand to confirm.
> > Can you help to just give it a try?
>
> Yes the fix may be hackish, but it served the purpose of correctly
> identifying the bug. Your suggestion works fine, for reference the
> tested patch was:
>
>
> diff --git a/drivers/acpi/acpica/tbxfload.c b/drivers/acpi/acpica/tbxfload.c
> index 278666e39563..9136d7250022 100644
> --- a/drivers/acpi/acpica/tbxfload.c
> +++ b/drivers/acpi/acpica/tbxfload.c
> @@ -83,6 +83,7 @@ acpi_status __init acpi_load_tables(void)
> "While loading namespace from ACPI tables"));
> }
>
> + acpi_gbl_reg_methods_enabled = TRUE;
> return_ACPI_STATUS(status);
> }
>
> diff --git a/drivers/acpi/acpica/utxfinit.c b/drivers/acpi/acpica/utxfinit.c
> index 721b87cce908..2c0491038068 100644
> --- a/drivers/acpi/acpica/utxfinit.c
> +++ b/drivers/acpi/acpica/utxfinit.c
> @@ -267,7 +267,6 @@ acpi_status __init acpi_initialize_objects(u32 flags)
> * initialized, even if they contain executable AML (see the call to
> * acpi_ns_initialize_objects below).
> */
> - acpi_gbl_reg_methods_enabled = TRUE;
> if (!(flags & ACPI_NO_ADDRESS_SPACE_INIT)) {
> ACPI_DEBUG_PRINT((ACPI_DB_EXEC,
> "[Init] Executing _REG OpRegion
> methods\n"));
>
>
> With this patch applied the ODEBUG errors do not occur.
[Lv Zheng]
Great!
This line doesn't differ too much in ACPICA upstream, while it matters much in Linux code base.
I should rebase the other patches along with this on top of ACPICA Feb release.
But as I don't have real ECDT platforms, I can just test the fixes by faking ECDT using initrd table override mechanism.
I think it's better to have you confirm the whole series again.
I saw you started to reply on the Bugzilla entry, I'll ping you there after rebasing the patchset.
>
> Thanks.
[Lv Zheng]
Thanks and best regards
-Lv
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2016-03-08 0:55 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-02-03 10:36 [BUG] ODEBUG: assert_init not available (active state 0) Chris Bainbridge
2016-02-05 2:52 ` Zheng, Lv
2016-02-14 10:59 ` Chris Bainbridge
2016-02-23 2:18 ` Zheng, Lv
2016-02-27 19:08 ` Chris Bainbridge
2016-03-05 15:11 ` [PATCH] ACPICA: Events: Execute some _REG methods in early boot Chris Bainbridge
2016-03-07 6:36 ` Zheng, Lv
2016-03-07 10:19 ` Chris Bainbridge
2016-03-08 0:54 ` Zheng, Lv
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).