From: Johannes Berg <johannes@sipsolutions.net>
To: linux-wireless <linux-wireless@vger.kernel.org>
Subject: mac80211 and multiple RX queues (with RSS hashing)
Date: Fri, 12 Jun 2015 10:31:54 +0200 [thread overview]
Message-ID: <1434097914.2679.37.camel@sipsolutions.net> (raw)
Hi,
We're currently investigating how to best add RSS hashing to the
wireless stack.
Let's say, for the sake of discussion, that we'll add a function to
mac80211 called
ieee80211_rx_napi_mq(struct hw *, struct napi_struct *, struct sk_buff
*);
and (depending on the outcome of the design) the driver would have to
set the skb_queue_mapping to the queue this packet was received on.
For context, there are a few requirements for callers of this function -
here are the ones I'm planning:
* It would only support (non-null) data frames, management frames must
not be
passed to the new API, but must be passed to the regular
ieee80211_rx()
function.
* I'm not going to support ieee80211_rx_irqsafe() with this API, it
would be
quite pointless (or a lot of code to have per-queue tasklets?)
* Won't support software crypto (PN checking can't really work in
parallel)
* Won't support monitor mode, and probably a few other similar things
(TBD)
* Won't support defragmentation (fragmented must anyway either be
reassembled
for hashing or somehow not hashed perhaps)
* Won't support client powersave in mac80211 - uh ... just say no to
that!
* For not, I'm not planning to support mesh on this, maybe not even
IBSS.
* Duplicate detection must be handled by hardware/firmware or similar
(possibly could be done with HW assist in driver, but clearly that
cannot be
handled generically in mac80211)
* RX aggregation reordering done by hardware/firmware or similar
(same here)
* require a NAPI struct? (not really sure yet how the stack treats
parallel RX)
Of course these don't really seem fairly natural so far, by the nature
of using multiple queues.
There are a few other areas that are of more concern:
a) Statistics
Obviously, we can no longer have single counters. Using atomic
counters would
kill much of the benefit of having multiple queues to start with, so
in some
way we need to have multiple counters.
b) handling AP/GO powersaving clients
With RSS, we can end up with various races - right now we say TX
status and
RX must be serialized by the driver, but clearly that can no longer
be
guaranteed with multiple RX queues.
[also need to check if there are *other* things that require
serialization]
c) aggregation session timeout/reorder timer handling
There's a single field/timer (for each of this) per session,
obviously it's
not a great idea to hit these from multiple CPUs.
Let's take these one by one:
a) This is probably the biggest one. We have a LOT of statistics that we
keep, and they all rely on RX being serialized. For example, per-station
RX packet and byte counters. Making all of these atomic would be
correct, but would obviously kill much of the performance benefit of
RSS.
As a consequence, I see two possible solutions here:
a1) Just make this the driver's concern, change sta_set_sinfo() to not
provide any (with a few exceptions) mac80211 statistics when multi-queue
RX is used. This could mean the same kind of statistics code is in each
driver, if the driver supports the statistics at all - or it could mean
that we cause a lot of divergence with statistics between drivers.
a2) Alternatively, drivers could tell mac80211 before-hand how many
queues they'll use, pass a queue identifier to mac80211 for each packet
(e.g. in skb's queue_mapping) and have mac80211 gather per-queue
statistics that get combined when read. This means allocating separate
statistics/queue arrays for stations etc. in mac80211, and then using
u64_stats_update_begin() etc. to get a consistent reading like we do
with per-cpu netdev stats already (since my fairly recent patch.)
Clearly this cannot support the "average" values like "average signal
strength" and the "last packet signal strength" might not always be
really the very last packet (which doesn't really matter though) so
those would still have to be excluded and generated by the driver, but
it could mean a bit more code sharing (if more than our driver ends up
using this facility) and more consistency. The downside might be that if
drivers want to do statistics in the firmware or so, they'd waste the
extra cycles on the host. I don't think we're planning to do that for
now though.
b) I think this one is pretty simple - just require the driver to set
AP_LINK_PS and if necessary call ieee80211_sta_ps_transition(). However,
it might require adding more logic for U-APSD support depending on the
hardware design.
c) I'm not really sure about this. I think it really needs hardware
assist.
Does anyone else have any thoughts?
johannes
next reply other threads:[~2015-06-12 8:31 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-06-12 8:31 Johannes Berg [this message]
2015-06-16 13:28 ` mac80211 and multiple RX queues (with RSS hashing) Johannes Berg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1434097914.2679.37.camel@sipsolutions.net \
--to=johannes@sipsolutions.net \
--cc=linux-wireless@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.