lartc.vger.kernel.org archive mirror
 help / color / mirror / Atom feed

From: "Ethy H. Brito" <ethy.brito@inexo.com.br>
To: lartc@vger.kernel.org
Subject: Traffic shaping at 10~300mbps at a 10Gbps link
Date: Mon, 07 Jun 2021 16:38:53 +0000	[thread overview]
Message-ID: <20210607133853.045a96d5@babalu> (raw)


Hi

I am having a hard time trying to shape 3000 users at ceil speeds from 10 to 300mbps in a 7/7Gbps link using HTB+SFQ+TC(filter by IP hashkey mask) for a few days now tweaking HTB and SFQ parameters with no luck so far.

Everything seems right, up 4Gbps overall download speed with shaping on.
I have no significant packets delay, no dropped packets and no high CPU average loads (not more than 20% - htop info)

But when the speed comes to about 4.5Gbps download (upload is about 500mbps), chaos kicks in.
CPU load goes sky high (all 24x2.4Ghz physical cores above 90% - 48x2.4Ghz if count that virtualization is on) and as a consequence packets are dropped (as reported by tc -s class sh ...), RTT goes above 200ms and a lots of ungry users. This goes from about 7PM to 11 PM every day.

If I turn shaping off, everything return to normality immediately and peaks of not more than 5Gbps (1 second average) are observed and a CPU load of about 5%. So I infer the uplink is not crowded.

I use one root HTB qdisc and one root (1:) HTB class.
Then about 20~30 same level (1:xx) inner classes to (sort of) separate the users by region 
And under these inner classes, goes the almost 3000 leaves (1:xxxx). 
I have one class with about 900 users and this quantity decreases by the other inner classes having some of them with just one user.

Is the way I'm using HTB+SFQ+TC suitable for this job?

Since the script that creates the shaping environment is too long I do not post it here.

What can I inform you guys to help me solve this?
Fragments of code, stats, some measurements? What?

Thanks.

Regards

Ethy

next             reply	other threads:[~2021-06-07 16:38 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-06-07 16:38 Ethy H. Brito [this message]
2021-06-09 15:04 ` Traffic shaping at 10~300mbps at a 10Gbps link Ethy H. Brito
2021-06-09 19:11 ` Adam Niescierowicz
2021-06-09 19:30 ` Ethy H. Brito
2021-06-09 19:57 ` Adam Niescierowicz
2021-06-09 22:13 ` Ethy H. Brito
2021-06-12 11:24 ` Anatoly Muliarski
2021-06-14 16:13 ` Ethy H. Brito
2021-06-16 11:43 ` cronolog+lartc
2021-06-16 12:08 ` Erik Auerswald
2021-07-07  2:10 ` L A Walsh

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210607133853.045a96d5@babalu \
    --to=ethy.brito@inexo.com.br \
    --cc=lartc@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).