From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Christoffer Dall <christofferdall@gmail.com>
Cc: David Vrabel <david.vrabel@citrix.com>,
Wei Liu <wei.liu2@citrix.com>,
Ian Campbell <ian.campbell@citrix.com>,
Stefano Stabellini <stefano.stabellini@eu.citrix.com>,
xen-devel@lists.xen.org
Subject: Re: Dom0 crash with apache bench (ab)
Date: Wed, 30 Sep 2015 11:12:13 -0400 [thread overview]
Message-ID: <20150930151213.GC30549@localhost.localdomain> (raw)
In-Reply-To: <CAEDV+gKuPKFguFbRG+s3_XsZ=M44zRejue2dN_a0CenZhb8FGA@mail.gmail.com>
On Mon, Sep 28, 2015 at 10:53:33PM +0200, Christoffer Dall wrote:
> On Mon, Sep 14, 2015 at 5:20 PM, Ian Campbell <ian.campbell@citrix.com>
> wrote:
>
> > On Mon, 2015-09-14 at 14:40 +0200, Christoffer Dall wrote:
> > > On Fri, Jul 31, 2015 at 03:17:56PM +0200, Christoffer Dall wrote:
> > > > On Fri, Jul 31, 2015 at 12:28 PM, David Vrabel <
> > david.vrabel@citrix.com
> > > > >
> > > > wrote:
> > > >
> > > > > On 31/07/15 11:24, Stefano Stabellini wrote:
> > > > > > This is a Linux Dom0 crash on x86 (Dell PowerEdge R320, Xeon E5
> > > > > > -2450),
> > > > > > CC'ing relevant people. As you can see from the links below the
> > > > > > crash
> > > > > > is:
> > > > > >
> > > > > > [ 253.619326] Call Trace:
> > > > > > [ 253.619330] <IRQ>
> > > > > > [ 253.619332] [<ffffffff815d7c25>] ? skb_copy_ubufs+0xa5/0x230
> > > > > > [ 253.619347] [<ffffffff815e8525>]
> > > > > > __netif_receive_skb_core+0x6f5/0x940
> > > > > > [ 253.619353] [<ffffffff815e8788>] __netif_receive_skb+0x18/0x60
> > > > > > [ 253.619360] [<ffffffff815e87f8>]
> > > > > > netif_receive_skb_internal+0x28/0x90
> > > > > > [ 253.619366] [<ffffffff815e91f5>] napi_gro_frags+0x125/0x1a0
> > > > > > [ 253.619378] [<ffffffffa01b1173>]
> > > > > > mlx4_en_process_rx_cq+0x753/0xb50
> > > > > [mlx4_en]
> > > > > > [ 253.619387] [<ffffffffa01b1657>] mlx4_en_poll_rx_cq+0x97/0x160
> > > > > [mlx4_en]
> > > > >
> > > > > What makes you think this is Xen specific? I suggest raising this
> > > > > the
> > > > > the mlx4 maintainers.
> > > > >
> > > > >
> > > > Linux native and KVM guests (same hw, same kernel version+config) run
> > > > just
> > > > fine under the same workload.
> > > >
> > > Ping?
> > >
> > > From the fact that bare-metal and KVM works fine with this hardware I
> > > still think it's reasonable to assume that it's a Xen issue and not a
> > > mlx4 issue.
> > >
> > > Is this completely flawed?
> >
> > My (somewhat educated) guess is that this is to do with the difference
> > between (pseudo-)physical addresses and machine (AKA real-physical)
> > addresses when running under Xen.
> >
> > The way this often shows up is in drivers which do not make correct use of
> > the kernels DMA APIs but which happen to work on native x86 because
> > physical==bus address on x86.
> >
> > Sometimes booting natively with 'iommu=soft swiotlb=force' can expose these
> > sorts of issues.
> >
>
> Indeed it does, on both v4.0 and v4.3-rc2.
Yeeey!
>
>
> >
> > You are running 64-bit so I don't think the recent "config: Enable
> > NEED_DMA_MAP_STATE by default when SWIOTLB is selected" is likely to be
> > relevant (it's already unconditionally on for 64-bit).
> >
> > The trace appears to be on rx from a physical nic, there shouldn't be any
> > magic Xen stuff (granted pages etc) getting themselves into that path at
> > all. If it were tx then maybe it might be an issue with foreign pages. In
> > any case I think you are able to repro with just dom0, i.e. never having
> > started a domU, is that right?
> >
> >
> Yes, I can reproduce on Dom0.
>
> I will send this to the Mellanox people.
Thank you :-) Thought please do keep us (or at least me) CC, this is an
interesting bug.
>
> Thanks,
> -Christoffer
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
prev parent reply other threads:[~2015-09-30 15:12 UTC|newest]
Thread overview: 13+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-28 13:09 Dom0 crash with apache bench (ab) Christoffer Dall
2015-07-28 14:50 ` Konrad Rzeszutek Wilk
2015-07-28 14:55 ` Ian Campbell
2015-07-28 15:00 ` Christoffer Dall
2015-07-31 10:24 ` Stefano Stabellini
2015-07-31 10:28 ` David Vrabel
2015-07-31 13:17 ` Christoffer Dall
2015-09-14 12:40 ` Christoffer Dall
2015-09-14 15:11 ` Konrad Rzeszutek Wilk
2015-09-14 15:20 ` Ian Campbell
2015-09-14 16:16 ` Christoffer Dall
2015-09-28 20:53 ` Christoffer Dall
2015-09-30 15:12 ` Konrad Rzeszutek Wilk [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150930151213.GC30549@localhost.localdomain \
--to=konrad.wilk@oracle.com \
--cc=christofferdall@gmail.com \
--cc=david.vrabel@citrix.com \
--cc=ian.campbell@citrix.com \
--cc=stefano.stabellini@eu.citrix.com \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.