x86: ipipe: Tracing of preemptirq events triggers a kernel PANIC

All the mail mirrored from lore.kernel.org
 help / color / mirror / Atom feed

* x86: ipipe: Tracing of preemptirq events triggers a kernel PANIC
@ 2021-06-22  9:41 Bezdeka, Florian
  2021-06-22 14:54 ` Bezdeka, Florian
  0 siblings, 1 reply; 2+ messages in thread
From: Bezdeka, Florian @ 2021-06-22  9:41 UTC (permalink / raw
  To: xenomai@xenomai.org

Hi,

I'm able to easily reproduce a kernel PANIC when trying to enable the
following trace events:

- preemptirq:irq_enable
- preemptirq:irq_disable

or as cmdline:

trace-cmd start -e "preemptirq:irq_enable"
trace-cmd start -e "preemptirq:irq_disable"

The PANIC can be reproduced with 4.19 and 5.4 and is ipipe specific. I
was not able to reproduce with 5.10 and dovetail. (More details below
the panic dump.) The PANIC happens just after enabling the trace
events.

The (truncated) panic dump:

[16598.599331] PANIC: double fault, error_code: 0x0
[16598.626938] CPU: 10 PID: 257657 Comm: containerd Not tainted 4.19.192-cip50-xenomai-1 #1
[16598.675349] Hardware name: Siemens <snip>
[16598.728440] I-pipe domain: Linux
[16598.747737] RIP: 0010:page_fault+0x0/0x58
[16598.771704] Code: 74 14 fa f6 84 24 88 00 00 00 03 0f 84 da f4 ff ff e9 45 f4 ff ff e9 4f 03 00 00 66 66 2e 0f 1f 84 00 00 00 00 00
 0f 1f 40 00  3b 02 00 00 4c 8b b4 24 90 00 00 00 48 89 e7 be 0e 00 00 00 48
[16598.884072] RSP: 0018:fffffe00001ba000 EFLAGS: 00010293
[16598.915323] RAX: 00000000b6800b17 RBX: 0000000000000000 RCX: ffffffffb6800b17
[16598.958016] RDX: 0000000000000000 RSI: ffffffffb6801625 RDI: fffffe00001baf38
[16599.000707] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
[16599.043399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[16599.086090] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[16599.128783] FS:  00007f42017fa700(0000) GS:ffff9aa3df880000(0000) knlGS:0000000000000000
[16599.177193] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[16599.211573] CR2: fffffe00001b9ff8 CR3: 000000084964e005 CR4: 00000000001606e0
[16599.254257] Call Trace:
[16599.268871]  
[16599.287645] RIP: 0010:trace_hardirqs_off_thunk+0x0/0x39
[16599.318892] Code: 52 41 53 48 8b 7d 08 e8 e7 cd 11 00 eb 38 55 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8 9b d2
 11 00 eb 1c <55> 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8
[16599.431262] RSP: 0018:fffffe00001ba038 EFLAGS: 00010093
[16599.431267]  error_entry+0x82/0x100
[16599.483367] RIP: 0010:trace_hardirqs_off_thunk+0x0/0x39
[16599.514617] Code: 52 41 53 48 8b 7d 08 e8 e7 cd 11 00 eb 38 55 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8 9b d2
 11 00 eb 1c <55> 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8


Tested versions:

4.19 with CONFIG_IRQ_PIPLINE enabled: PANIC
4.19 with CONFIG_IRQ_PIPLINE disabled: PANIC
5.4  with CONFIG_IRQ_PIPLINE enabled: PANIC
5.4  with CONFIG_IRQ_PIPLINE disabled: OK


The problem sounds very similar to [1]. These patches were not
backported to 4.19 because they were considered to intrusive. So I
guess 5.4 should be fine, but the CONFIG_IRQ_PIPLINE code /wrt page
faults re-introduces something similar to the CR2 corruption in [1].

[1] https://www.lkml.org/lkml/2020/10/5/535


Best regards,
Florian


-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: x86: ipipe: Tracing of preemptirq events triggers a kernel PANIC
  2021-06-22  9:41 x86: ipipe: Tracing of preemptirq events triggers a kernel PANIC Bezdeka, Florian
@ 2021-06-22 14:54 ` Bezdeka, Florian
  0 siblings, 0 replies; 2+ messages in thread
From: Bezdeka, Florian @ 2021-06-22 14:54 UTC (permalink / raw
  To: xenomai@xenomai.org

On Tue, 2021-06-22 at 09:41 +0000, Bezdeka, Florian via Xenomai wrote:
> Hi,
> 
> I'm able to easily reproduce a kernel PANIC when trying to enable the
> following trace events:
> 
> - preemptirq:irq_enable
> - preemptirq:irq_disable
> 
> or as cmdline:
> 
> trace-cmd start -e "preemptirq:irq_enable"
> trace-cmd start -e "preemptirq:irq_disable"
> 
> The PANIC can be reproduced with 4.19 and 5.4 and is ipipe specific. I
> was not able to reproduce with 5.10 and dovetail. (More details below
> the panic dump.) The PANIC happens just after enabling the trace
> events.
> 
> The (truncated) panic dump:
> 
> [16598.599331] PANIC: double fault, error_code: 0x0
> [16598.626938] CPU: 10 PID: 257657 Comm: containerd Not tainted 4.19.192-cip50-xenomai-1 #1
> [16598.675349] Hardware name: Siemens <snip>
> [16598.728440] I-pipe domain: Linux
> [16598.747737] RIP: 0010:page_fault+0x0/0x58
> [16598.771704] Code: 74 14 fa f6 84 24 88 00 00 00 03 0f 84 da f4 ff ff e9 45 f4 ff ff e9 4f 03 00 00 66 66 2e 0f 1f 84 00 00 00 00 00
>  0f 1f 40 00  3b 02 00 00 4c 8b b4 24 90 00 00 00 48 89 e7 be 0e 00 00 00 48
> [16598.884072] RSP: 0018:fffffe00001ba000 EFLAGS: 00010293
> [16598.915323] RAX: 00000000b6800b17 RBX: 0000000000000000 RCX: ffffffffb6800b17
> [16598.958016] RDX: 0000000000000000 RSI: ffffffffb6801625 RDI: fffffe00001baf38
> [16599.000707] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
> [16599.043399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> [16599.086090] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> [16599.128783] FS:  00007f42017fa700(0000) GS:ffff9aa3df880000(0000) knlGS:0000000000000000
> [16599.177193] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [16599.211573] CR2: fffffe00001b9ff8 CR3: 000000084964e005 CR4: 00000000001606e0
> [16599.254257] Call Trace:
> [16599.268871]  
> [16599.287645] RIP: 0010:trace_hardirqs_off_thunk+0x0/0x39
> [16599.318892] Code: 52 41 53 48 8b 7d 08 e8 e7 cd 11 00 eb 38 55 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8 9b d2
>  11 00 eb 1c <55> 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8
> [16599.431262] RSP: 0018:fffffe00001ba038 EFLAGS: 00010093
> [16599.431267]  error_entry+0x82/0x100
> [16599.483367] RIP: 0010:trace_hardirqs_off_thunk+0x0/0x39
> [16599.514617] Code: 52 41 53 48 8b 7d 08 e8 e7 cd 11 00 eb 38 55 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8 9b d2
>  11 00 eb 1c <55> 48 89 e5 57 56 52 51 50 41 50 41 51 41 52 41 53 48 8b 7d 08 e8
> 
> 
> Tested versions:
> 
> 4.19 with CONFIG_IRQ_PIPLINE enabled: PANIC
> 4.19 with CONFIG_IRQ_PIPLINE disabled: PANIC
> 5.4  with CONFIG_IRQ_PIPLINE enabled: PANIC
> 5.4  with CONFIG_IRQ_PIPLINE disabled: OK

I have to correct myself: It's CONFIG_IPIPE of course. Sorry.

> 
> 
> The problem sounds very similar to [1]. These patches were not
> backported to 4.19 because they were considered to intrusive. So I
> guess 5.4 should be fine, but the CONFIG_IRQ_PIPLINE code /wrt page
> faults re-introduces something similar to the CR2 corruption in [1].

Again: /s/CONFIG_IRQ_PIPLINE/CONFIG_IPIPE

> 
> [1] https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.lkml.org%2Flkml%2F2020%2F10%2F5%2F535&amp;data=04%7C01%7Cflorian.bezdeka%40siemens.com%7Cbdb6e83a444a4ffe3a1d08d93561f6a1%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637599517125278706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=DpJ9Z7yozA50QBRFcrTlF5uvcN2vU8bER5kAFe6rQcU%3D&amp;reserved=0
> 
> 
> Best regards,
> Florian
> 
> 
> -- 
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux


^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2021-06-22 14:54 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-06-22  9:41 x86: ipipe: Tracing of preemptirq events triggers a kernel PANIC Bezdeka, Florian
2021-06-22 14:54 ` Bezdeka, Florian

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.