From: Paolo Abeni <pabeni@redhat.com>
To: netdev@vger.kernel.org
Cc: Jakub Kicinski <kuba@kernel.org>, Jiri Pirko <jiri@resnulli.us>,
Madhu Chittim <madhu.chittim@intel.com>,
Sridhar Samudrala <sridhar.samudrala@intel.com>,
Simon Horman <horms@kernel.org>,
John Fastabend <john.fastabend@gmail.com>,
Sunil Kovvuri Goutham <sgoutham@marvell.com>,
Jamal Hadi Salim <jhs@mojatatu.com>
Subject: [RFC PATCH] net: introduce HW Rate Limiting Driver API
Date: Wed, 8 May 2024 22:20:51 +0200 [thread overview]
Message-ID: <3d1e2d945904a0fb55258559eb7322d7e11066b6.1715199358.git.pabeni@redhat.com> (raw)
This is the first incarnation in a formal (pre-RFC) patch of the
HW TX Rate Limiting Driver API proposal shared here[1].
The goal is to outline the proposed APIs before pushing the actual
implementation.
The network devices gain a new ops struct to directly manipulate the
H/W shapers implemented by the NIC.
The shapers can be attached to a pre-defined set of 'domains' - port,
vf, etc. - and the overall shapers configuration pushed to the H/W is
maintained by the kernel.
Each shaper is identified by an unique integer id based on the domain
and additional domain-specific information - e.g. for the VF domain, the
virtual function number/identifier.
[1] https://lore.kernel.org/netdev/20240405102313.GA310894@kernel.org/
Co-developed-by: Simon Horman <horms@kernel.org>
Signed-off-by: Simon Horman <horms@kernel.org>
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
---
include/linux/netdevice.h | 15 +++
include/net/net_shaper.h | 206 ++++++++++++++++++++++++++++++++++++++
net/Kconfig | 3 +
3 files changed, 224 insertions(+)
create mode 100644 include/net/net_shaper.h
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index cf261fb89d73..39f66af014be 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -79,6 +79,8 @@ struct xdp_buff;
struct xdp_frame;
struct xdp_metadata_ops;
struct xdp_md;
+struct net_shaper_ops;
+struct net_shaper_data;
typedef u32 xdp_features_t;
@@ -1596,6 +1598,13 @@ struct net_device_ops {
int (*ndo_hwtstamp_set)(struct net_device *dev,
struct kernel_hwtstamp_config *kernel_config,
struct netlink_ext_ack *extack);
+
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /** @net_shaper_ops: Device shaping offload operations
+ * see include/net/net_shapers.h
+ */
+ const struct net_shaper_ops *net_shaper_ops;
+#endif
};
/**
@@ -2403,6 +2412,12 @@ struct net_device {
/** @page_pools: page pools created for this netdevice */
struct hlist_head page_pools;
#endif
+#if IS_ENABLED(CONFIG_NET_SHAPER)
+ /** @net_shaper_data: data tracking the current shaper status
+ * see include/net/net_shapers.h
+ */
+ struct net_shaper_data *net_shaper_data;
+#endif
};
#define to_net_dev(d) container_of(d, struct net_device, dev)
diff --git a/include/net/net_shaper.h b/include/net/net_shaper.h
new file mode 100644
index 000000000000..a4fbadd99870
--- /dev/null
+++ b/include/net/net_shaper.h
@@ -0,0 +1,206 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+
+#ifndef _NET_SHAPER_H_
+#define _NET_SHAPER_H_
+
+#include <linux/types.h>
+#include <linux/netdevice.h>
+#include <linux/netlink.h>
+
+/**
+ * enum net_shaper_metric - the metric of the shaper
+ * @NET_SHAPER_METRIC_PPS: Shaper operates on a packets per second basis
+ * @NET_SHAPER_METRIC_BPS: Shaper operates on a bits per second basis
+ */
+enum net_shaper_metric {
+ NET_SHAPER_METRIC_PPS,
+ NET_SHAPER_METRIC_BPS
+};
+
+/**
+ * struct net_shaper_info - represents a shaping node on the NIC H/W
+ * @metric: Specify if the bw limits refers to PPS or BPS
+ * @bw_min: Minimum guaranteed rate for this shaper
+ * @bw_max: Maximum peak bw allowed for this shaper
+ * @burst: Maximum burst for the peek rate of this shaper
+ * @priority: Scheduling priority for this shaper
+ * @weight: Scheduling weight for this shaper
+ */
+struct net_shaper_info {
+ enum net_shaper_metric metric;
+ u64 bw_min; /* minimum guaranteed bandwidth, according to metric */
+ u64 bw_max; /* maximum allowed bandwidth */
+ u32 burst; /* maximum burst in bytes for bw_max */
+ u32 priority; /* scheduling strict priority */
+ u32 weight; /* scheduling WRR weight*/
+};
+
+/**
+ * enum net_shaper_scope - the different scopes where a shaper could be attached
+ * @NET_SHAPER_SCOPE_PORT: The root shaper for the whole H/W.
+ * @NET_SHAPER_SCOPE_NETDEV: The main shaper for the given network device.
+ * @NET_SHAPER_SCOPE_VF: The shaper is attached to the given virtual
+ * function.
+ * @NET_SHAPER_SCOPE_QUEUE_GROUP: The shaper groups multiple queues under the
+ * same device.
+ * @NET_SHAPER_SCOPE_QUEUE: The shaper is attached to the given device queue.
+ *
+ * NET_SHAPER_SCOPE_PORT and NET_SHAPER_SCOPE_VF are only available on
+ * PF devices, usually inside the host/hypervisor.
+ * NET_SHAPER_SCOPE_NETDEV, NET_SHAPER_SCOPE_QUEUE_GROUP and
+ * NET_SHAPER_SCOPE_QUEUE are available on both PFs and VFs devices.
+ */
+enum net_shaper_scope {
+ NET_SHAPER_SCOPE_PORT,
+ NET_SHAPER_SCOPE_NETDEV,
+ NET_SHAPER_SCOPE_VF,
+ NET_SHAPER_SCOPE_QUEUE_GROUP,
+ NET_SHAPER_SCOPE_QUEUE,
+};
+
+/**
+ * struct net_shaper_ops - Operations on device H/W shapers
+ * @add: Creates a new shaper in the specified scope.
+ * @set: Modify the existing shaper.
+ * @delete: Delete the specified shaper.
+ * @move: Move an existing shaper under a different parent.
+ *
+ * The initial shaping configuration ad device initialization is empty/
+ * a no-op/does not constraint the b/w in any way.
+ * The network core keeps track of the applied user-configuration in
+ * per device storage.
+ *
+ * Each shaper is uniquely identified within the device with an 'handle',
+ * dependent on the shaper scope and other data, see @shaper_make_handle()
+ */
+struct net_shaper_ops {
+ /** add - Add a shaper inside the shaper hierarchy
+ * @dev: netdevice to operate on
+ * @handle: the shaper indetifier
+ * @shaper: configuration of shaper
+ * @extack: Netlink extended ACK for reporting errors.
+ *
+ * Return:
+ * * 0 on success
+ * * %-EOPNOTSUPP - Operation is not supported by hardware, driver,
+ * or core for any reason. @extack should be set to
+ * text describing the reason.
+ * * Other negative error values on failure.
+ *
+ * Examples or reasons this operation may fail include:
+ * * H/W resources limits.
+ * * Can’t respect the requested bw limits.
+ */
+ int (*add)(struct net_device *dev, u32 handle,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack);
+
+ /** set - Update the specified shaper, if it exists
+ * @dev: Netdevice to operate on.
+ * @handle: the shaper identifier
+ * @shaper: Configuration of shaper.
+ * @extack: Netlink extended ACK for reporting errors.
+ *
+ * Return:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Operation is not supported by hardware, driver,
+ * or core for any reason. @extack should be set to
+ * text describing the reason.
+ * * Other negative error values on failure.
+ */
+ int (*set)(struct net_device *dev, u32 handle,
+ const struct net_shaper_info *shaper,
+ struct netlink_ext_ack *extack);
+
+ /** delete - Removes a shaper from the NIC
+ * @dev: netdevice to operate on.
+ * @handle: the shaper identifier
+ * @extack: Netlink extended ACK for reporting errors.
+ *
+ * Return:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Operation is not supported by hardware, driver,
+ * or core for any reason. @extack should be set to
+ * text describing the reason.
+ * * Other negative error value on failure.
+ */
+ int (*delete)(struct net_device *dev, u32 handle,
+ struct netlink_ext_ack *extack);
+
+ /** Move - change the parent id of the specified shaper
+ * @dev: netdevice to operate on.
+ * @handle: unique identifier for the shaper
+ * @new_parent_id: identifier of the new parent for this shaper
+ * @extack: Netlink extended ACK for reporting errors.
+ *
+ * Move the specified shaper in the hierarchy replacing its
+ * current parent shaper with @new_parent_id
+ *
+ * Return:
+ * * %0 - Success
+ * * %-EOPNOTSUPP - Operation is not supported by hardware, driver,
+ * or core for any reason. @extack should be set to
+ * text describing the reason.
+ * * Other negative error values on failure.
+ */
+ int (*move)(struct net_device *dev, u32 handle,
+ u32 new_parent_handle, struct netlink_ext_ack *extack);
+};
+
+/**
+ * net_shaper_make_handle - creates an unique shaper identifier
+ * @scope: the shaper scope
+ * @vf: virtual function number
+ * @id: queue group or queue id
+ *
+ * Return: an unique identifier for the shaper
+ *
+ * Combines the specified arguments to create an unique identifier for
+ * the shaper.
+ * The virtual function number is only used within @NET_SHAPER_SCOPE_VF,
+ * @NET_SHAPER_SCOPE_QUEUE_GROUP and @NET_SHAPER_SCOPE_QUEUE.
+ * The @id number is only used for @NET_SHAPER_SCOPE_QUEUE_GROUP and
+ * @NET_SHAPER_SCOPE_QUEUE, and must be, respectively, the queue group
+ * identifier or the queue number.
+ */
+u32 net_shaper_make_handle(enum net_shaper_scope scope, int vf, int id);
+
+/*
+ * Examples:
+ * - set shaping on a given queue
+ * struct shaper_info info = { }; // fill this
+ * u32 handle = shaper_make_handle(NET_SHAPER_SCOPE_QUEUE, 0, queue_id);
+ * dev->shaper_ops->add(dev, handle, &info, NULL);
+ *
+ * - create a queue group with a queue group shaping limits.
+ * Assuming the following topology already exists:
+ * < netdev shaper >
+ * / \
+ * <queue 0 shaper> . . . <queue N shaper>
+ *
+ * struct shaper_info ginfo = { }; // fill this
+ * u32 ghandle = shaper_make_handle(NET_SHAPER_SCOPE_QUEUE_GROUP, 0, 0);
+ * dev->shaper_ops->add(dev, ghandle, &ginfo);
+ *
+ * // now topology is:
+ * // < netdev shaper >
+ * // / | \
+ * // / | < newly created shaper >
+ * // / |
+ * // <queue 0 shaper> . . . <queue N shaper>
+ *
+ * // move a shapers for queues 3..n out of such queue group
+ * for (i = 0; i <= 2; ++i) {
+ * u32 qhandle = net_shaper_make_handle(NET_SHAPER_SCOPE_QUEUE, 0, i);
+ * dev->netshaper_ops->move(dev, qhandle, ghandle, NULL);
+ * }
+ *
+ * // now the topology is:
+ * // < netdev shaper >
+ * // / \
+ * // < newly created shaper> <queue 3 shaper> .. <queue n shaper>
+ * // / \
+ * // <queue 0 shaper> . . . <queue 2 shaper>
+ */
+#endif
+
diff --git a/net/Kconfig b/net/Kconfig
index f0a8692496ff..29c6fec54711 100644
--- a/net/Kconfig
+++ b/net/Kconfig
@@ -66,6 +66,9 @@ config SKB_DECRYPTED
config SKB_EXTENSIONS
bool
+config NET_SHAPER
+ bool
+
menu "Networking options"
source "net/packet/Kconfig"
--
2.43.2
next reply other threads:[~2024-05-08 20:21 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-08 20:20 Paolo Abeni [this message]
2024-05-08 21:47 ` [RFC PATCH] net: introduce HW Rate Limiting Driver API Andrew Lunn
2024-05-09 14:19 ` Paolo Abeni
2024-05-09 15:00 ` Andrew Lunn
2024-05-09 15:43 ` Paolo Abeni
2024-05-09 16:17 ` Andrew Lunn
2024-05-10 11:05 ` Paolo Abeni
2024-05-15 9:51 ` Simon Horman
2024-05-15 14:19 ` Andrew Lunn
2024-05-15 14:56 ` Simon Horman
2024-05-09 15:09 ` Andrew Lunn
2024-05-10 7:10 ` Naveen Mamindlapalli
2024-05-10 7:58 ` Paolo Abeni
2024-05-15 14:41 ` Andrew Lunn
2024-05-15 14:50 ` Simon Horman
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=3d1e2d945904a0fb55258559eb7322d7e11066b6.1715199358.git.pabeni@redhat.com \
--to=pabeni@redhat.com \
--cc=horms@kernel.org \
--cc=jhs@mojatatu.com \
--cc=jiri@resnulli.us \
--cc=john.fastabend@gmail.com \
--cc=kuba@kernel.org \
--cc=madhu.chittim@intel.com \
--cc=netdev@vger.kernel.org \
--cc=sgoutham@marvell.com \
--cc=sridhar.samudrala@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).