From: Tejun Heo <tj@kernel.org>
To: Parav Pandit <pandit.parav@gmail.com>
Cc: cgroups@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
lizefan@huawei.com, Johannes Weiner <hannes@cmpxchg.org>,
Doug Ledford <dledford@redhat.com>,
Jonathan Corbet <corbet@lwn.net>,
james.l.morris@oracle.com, serge@hallyn.com,
Haggai Eran <haggaie@mellanox.com>,
Or Gerlitz <ogerlitz@mellanox.com>,
Matan Barak <matanb@mellanox.com>,
raindel@mellanox.com, akpm@linux-foundation.org,
linux-security-module@vger.kernel.org
Subject: Re: [PATCH 0/7] devcg: device cgroup extension for rdma resource
Date: Thu, 10 Sep 2015 12:49:46 -0400 [thread overview]
Message-ID: <20150910164946.GH8114@mtj.duckdns.org> (raw)
In-Reply-To: <CAG53R5VnYJ9+VEKtbnFO1HntSp=O=ZGiknucbQ-QLuEk_UP44w@mail.gmail.com>
Hello, Parav.
On Wed, Sep 09, 2015 at 09:27:40AM +0530, Parav Pandit wrote:
> This is one old white paper, but most of the reasoning still holds true on RDMA.
> http://h10032.www1.hp.com/ctg/Manual/c00257031.pdf
Just read it. Much appreciated.
...
> These resources include are- QP (queue pair) to transfer data, CQ
> (Completion queue) to indicate completion of data transfer operation,
> MR (memory region) to represent user application memory as source or
> destination for data transfer.
> Common resources are QP, SRQ (shared received queue), CQ, MR, AH
> (Address handle), FLOW, PD (protection domain), user context etc.
It's kinda bothering that all these are disparate resources. I
suppose that each restriction comes from the underlying hardware and
there's no accepted higher level abstraction for these things?
> >> This patch-set allows limiting rdma resources to set of processes.
> >> It extend device cgroup controller for limiting rdma device limits.
> >
> > I don't think this belongs to devcg. If these make sense as a set of
> > resources to be controlled via cgroup, the right way prolly would be a
> > separate controller.
> >
>
> In past there has been similar comment to have dedicated cgroup
> controller for RDMA instead of merging with device cgroup.
> I am ok with both the approach, however I prefer to utilize device
> controller instead of spinning of new controller for new devices
> category.
> I anticipate more such need would arise and for new device category,
> it might not be worth to have new cgroup controller.
> RapidIO though very less popular and upcoming PCIe are on horizon to
> offer similar benefits as that of RDMA and in future having one
> controller for each of them again would not be right approach.
>
> I certainly seek your and others inputs in this email thread here whether
> (a) to continue to extend device cgroup (which support character,
> block devices white list) and now RDMA devices
> or
> (b) to spin of new controller, if so what are the compelling reasons
> that it can provide compare to extension.
I'm doubtful that these things are gonna be mainstream w/o building up
higher level abstractions on top and if we ever get there we won't be
talking about MR or CQ or whatever. Also, whatever next-gen is
unlikely to have enough commonalities when the proposed resource knobs
are this low level, so let's please keep it separate, so that if/when
this goes out of fashion for one reason or another, the controller can
silently wither away too.
> Current scope of the patch is limited to RDMA resources as first
> patch, but for fact I am sure that there are more functionality in
> pipe to support via this cgroup by me and others.
> So keeping atleast these two aspects in mind, I need input on
> direction of dedicated controller or new one.
>
> In future, I anticipate that we might have sub directory to device
> cgroup for individual device class to control.
> such as,
> <sys/fs/cgroup/devices/
> /char
> /block
> /rdma
> /pcie
> /child_cgroup..1..N
> Each controllers cgroup access files would remain within their own
> scope. We are not there yet from base infrastructure but something to
> be done as it matures and users start using it.
I don't think that jives with the rest of cgroup and what generic
block or pcie attributes are directly exposed to applications and need
to be hierarchically controlled via cgroup?
Thanks.
--
tejun
next prev parent reply other threads:[~2015-09-10 16:49 UTC|newest]
Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-09-07 20:38 [PATCH 0/7] devcg: device cgroup extension for rdma resource Parav Pandit
2015-09-07 20:38 ` Parav Pandit
2015-09-07 20:38 ` [PATCH 1/7] devcg: Added user option to rdma resource tracking Parav Pandit
2015-09-07 20:38 ` Parav Pandit
2015-09-07 20:38 ` [PATCH 2/7] devcg: Added rdma resource tracking module Parav Pandit
2015-09-07 20:38 ` Parav Pandit
2015-09-07 20:38 ` [PATCH 3/7] devcg: Added infrastructure for rdma device cgroup Parav Pandit
2015-09-08 5:31 ` Haggai Eran
2015-09-08 5:31 ` Haggai Eran
2015-09-08 7:02 ` Parav Pandit
2015-09-08 7:02 ` Parav Pandit
2015-09-07 20:38 ` [PATCH 4/7] devcg: Added rdma resource tracker object per task Parav Pandit
2015-09-08 5:48 ` Haggai Eran
2015-09-08 5:48 ` Haggai Eran
2015-09-08 7:04 ` Parav Pandit
2015-09-08 8:24 ` Haggai Eran
2015-09-08 8:24 ` Haggai Eran
2015-09-08 8:26 ` Parav Pandit
2015-09-07 20:38 ` [PATCH 5/7] devcg: device cgroup's extension for RDMA resource Parav Pandit
2015-09-07 20:38 ` Parav Pandit
2015-09-08 8:22 ` Haggai Eran
2015-09-08 8:22 ` Haggai Eran
2015-09-08 10:18 ` Parav Pandit
2015-09-08 13:50 ` Haggai Eran
2015-09-08 13:50 ` Haggai Eran
2015-09-08 14:13 ` Parav Pandit
2015-09-08 8:36 ` Haggai Eran
2015-09-08 8:36 ` Haggai Eran
2015-09-08 10:50 ` Parav Pandit
2015-09-08 10:50 ` Parav Pandit
2015-09-08 14:10 ` Haggai Eran
2015-09-08 14:10 ` Haggai Eran
2015-09-07 20:38 ` [PATCH 6/7] devcg: Added support to use RDMA device cgroup Parav Pandit
2015-09-08 8:40 ` Haggai Eran
2015-09-08 8:40 ` Haggai Eran
2015-09-08 10:22 ` Parav Pandit
2015-09-08 13:40 ` Haggai Eran
2015-09-08 13:40 ` Haggai Eran
2015-09-07 20:38 ` [PATCH 7/7] devcg: Added Documentation of " Parav Pandit
2015-09-07 20:38 ` Parav Pandit
2015-09-07 20:55 ` [PATCH 0/7] devcg: device cgroup extension for rdma resource Parav Pandit
2015-09-08 12:45 ` Haggai Eran
2015-09-08 12:45 ` Haggai Eran
2015-09-08 15:23 ` Tejun Heo
2015-09-08 15:23 ` Tejun Heo
2015-09-09 3:57 ` Parav Pandit
2015-09-10 16:49 ` Tejun Heo [this message]
2015-09-10 17:46 ` Parav Pandit
2015-09-10 17:46 ` Parav Pandit
2015-09-10 20:22 ` Tejun Heo
2015-09-11 3:39 ` Parav Pandit
2015-09-11 4:04 ` Tejun Heo
2015-09-11 4:04 ` Tejun Heo
2015-09-11 4:24 ` Doug Ledford
2015-09-11 4:24 ` Doug Ledford
2015-09-11 14:52 ` Tejun Heo
2015-09-11 14:52 ` Tejun Heo
2015-09-11 16:26 ` Parav Pandit
2015-09-11 16:34 ` Tejun Heo
2015-09-11 16:34 ` Tejun Heo
2015-09-11 16:39 ` Parav Pandit
2015-09-11 16:39 ` Parav Pandit
2015-09-11 19:25 ` Tejun Heo
2015-09-14 10:18 ` Parav Pandit
2015-09-14 10:18 ` Parav Pandit
2015-09-11 16:47 ` Parav Pandit
2015-09-11 16:47 ` Parav Pandit
2015-09-11 19:05 ` Tejun Heo
2015-09-11 19:05 ` Tejun Heo
2015-09-11 19:22 ` Hefty, Sean
2015-09-11 19:43 ` Jason Gunthorpe
2015-09-11 19:43 ` Jason Gunthorpe
2015-09-11 20:06 ` Hefty, Sean
2015-09-14 11:09 ` Parav Pandit
2015-09-14 14:04 ` Parav Pandit
2015-09-14 15:21 ` Tejun Heo
2015-09-14 15:21 ` Tejun Heo
2015-09-14 17:28 ` Jason Gunthorpe
2015-09-14 17:28 ` Jason Gunthorpe
2015-09-14 18:54 ` Parav Pandit
2015-09-14 18:54 ` Parav Pandit
2015-09-14 20:18 ` Jason Gunthorpe
2015-09-15 3:08 ` Parav Pandit
2015-09-15 3:45 ` Jason Gunthorpe
2015-09-15 3:45 ` Jason Gunthorpe
2015-09-16 4:41 ` Parav Pandit
2015-09-16 4:41 ` Parav Pandit
2015-09-20 10:35 ` Haggai Eran
2015-09-20 10:35 ` Haggai Eran
2015-10-28 8:14 ` Parav Pandit
2015-10-28 8:14 ` Parav Pandit
2015-09-14 10:15 ` Parav Pandit
2015-09-11 4:43 ` Parav Pandit
2015-09-11 15:03 ` Tejun Heo
2015-09-10 17:48 ` Hefty, Sean
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150910164946.GH8114@mtj.duckdns.org \
--to=tj@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=cgroups@vger.kernel.org \
--cc=corbet@lwn.net \
--cc=dledford@redhat.com \
--cc=haggaie@mellanox.com \
--cc=hannes@cmpxchg.org \
--cc=james.l.morris@oracle.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=linux-security-module@vger.kernel.org \
--cc=lizefan@huawei.com \
--cc=matanb@mellanox.com \
--cc=ogerlitz@mellanox.com \
--cc=pandit.parav@gmail.com \
--cc=raindel@mellanox.com \
--cc=serge@hallyn.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.