From: Allen Hubbe <allenbh@gmail.com>
To: Anubhav Guleria <anubhav.nitsri.it@gmail.com>
Cc: linux-ntb <linux-ntb@googlegroups.com>
Subject: Re: Using NTRDMA with Intel Skylake in b2b configuration
Date: Mon, 25 May 2020 15:17:48 -0400 [thread overview]
Message-ID: <CAJ80sav6-3OEYp3wTUQ-nc541-Zzb3dNCxnXaLY4x7AjLn=tug@mail.gmail.com> (raw)
In-Reply-To: <ca1ae048-bb3d-4aec-8d89-17178734f097@googlegroups.com>
Hi Anubhav,
On Sun, May 24, 2020 at 5:37 PM Anubhav Guleria
<anubhav.nitsri.it@gmail.com> wrote:
>
>
> Hi,
> I am trying to setup ntrdma(https://github.com/ntrdma/ntrdma-ext) with Intel Skylake b2b configuration and Linux 4.14.178.
> Following is how my memory windows are setup:
> 1) MW[0] (BAR23): dma_alloc_coherent
> 2) MW[1] (BAR45): memremap
It would be interesting to know the size of the windows, and any
module parameters used with ntb_hw_intel. The last memory window
needs to be at least as large as the installed dram of the system.
> Further, what I understand that in case of MR registering there are 2 steps[ntrdma_reg_user_mr()]:
>
> 1) Call ib_umem_get() to Pin and DMA Map userspace Memory.
> 2) Then, prepare a memory buffer in a NTB window, by ntc_mr_buf_map_dma(..).
>
> The doubt that I have is that is it necessary to have the dma_addr from step 1 to be within the limits of NTB window, in this case MW[0] ?
> And if so, is there some specific configuration that needs to be made?
The dma addresses of the memory needs to fall within the translation
range of the window. If the memory is not within the window, then the
peer will not be able to access it. There is no bounce buffer for
memory not in the range.
Step 2 is to communicate the offset in the memory window to the peer,
which is the same as the dma address because the translation range is
zero-based. Then the peer will access the memory directly.
>
> Also, is above configuration of MWs correct?
>
> Thanks in advance.
Welcome,
Allen
prev parent reply other threads:[~2020-05-25 19:18 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-05-24 21:37 Using NTRDMA with Intel Skylake in b2b configuration Anubhav Guleria
2020-05-25 19:17 ` Allen Hubbe [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJ80sav6-3OEYp3wTUQ-nc541-Zzb3dNCxnXaLY4x7AjLn=tug@mail.gmail.com' \
--to=allenbh@gmail.com \
--cc=anubhav.nitsri.it@gmail.com \
--cc=linux-ntb@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).