From: Marcelo Tosatti <mtosatti@redhat.com>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: linux-kernel@vger.kernel.org,
Frederic Weisbecker <frederic@kernel.org>,
Peter Xu <peterx@redhat.com>,
Nitesh Narayan Lal <nitesh@redhat.com>,
Alex Belits <abelits@marvell.com>
Subject: [PATCH] hrtimer: avoid retrigger_next_event IPI
Date: Wed, 7 Apr 2021 10:53:01 -0300 [thread overview]
Message-ID: <20210407135301.GA16985@fuller.cnet> (raw)
Setting the realtime clock triggers an IPI to all CPUs to reprogram
hrtimers.
However, only base, boottime and tai clocks have their offsets updated
(and therefore potentially require a reprogram).
If the CPU is a nohz_full one, check if it only has
monotonic active timers, and in that case update the
realtime base offsets, skipping the IPI.
This reduces interruptions to nohz_full CPUs.
Signed-off-by: Marcelo Tosatti <mtosatti@redhat.com>
diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c
index 743c852e10f2..b42b1a434b22 100644
--- a/kernel/time/hrtimer.c
+++ b/kernel/time/hrtimer.c
@@ -853,6 +853,28 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
tick_program_event(expires, 1);
}
+#define CLOCK_SET_BASES ((1U << HRTIMER_BASE_REALTIME)| \
+ (1U << HRTIMER_BASE_REALTIME_SOFT)| \
+ (1U << HRTIMER_BASE_BOOTTIME)| \
+ (1U << HRTIMER_BASE_BOOTTIME_SOFT)| \
+ (1U << HRTIMER_BASE_TAI)| \
+ (1U << HRTIMER_BASE_TAI_SOFT))
+
+static bool need_reprogram_timer(struct hrtimer_cpu_base *cpu_base)
+{
+ unsigned int active = 0;
+
+ if (!cpu_base->softirq_activated)
+ active = cpu_base->active_bases & HRTIMER_ACTIVE_SOFT;
+
+ active = active | (cpu_base->active_bases & HRTIMER_ACTIVE_HARD);
+
+ if ((active & CLOCK_SET_BASES) == 0)
+ return false;
+
+ return true;
+}
+
/*
* Clock realtime was set
*
@@ -867,9 +889,41 @@ static void hrtimer_reprogram(struct hrtimer *timer, bool reprogram)
void clock_was_set(void)
{
#ifdef CONFIG_HIGH_RES_TIMERS
- /* Retrigger the CPU local events everywhere */
- on_each_cpu(retrigger_next_event, NULL, 1);
+ cpumask_var_t mask;
+ int cpu;
+
+ if (!tick_nohz_full_enabled()) {
+ /* Retrigger the CPU local events everywhere */
+ on_each_cpu(retrigger_next_event, NULL, 1);
+ goto set_timerfd;
+ }
+
+ if (!zalloc_cpumask_var(&mask, GFP_KERNEL)) {
+ on_each_cpu(retrigger_next_event, NULL, 1);
+ goto set_timerfd;
+ }
+
+ /* Avoid interrupting nohz_full CPUs if possible */
+ preempt_disable();
+ for_each_online_cpu(cpu) {
+ if (tick_nohz_full_cpu(cpu)) {
+ unsigned long flags;
+ struct hrtimer_cpu_base *cpu_base = &per_cpu(hrtimer_bases, cpu);
+
+ raw_spin_lock_irqsave(&cpu_base->lock, flags);
+ if (need_reprogram_timer(cpu_base))
+ cpumask_set_cpu(cpu, mask);
+ else
+ hrtimer_update_base(cpu_base);
+ raw_spin_unlock_irqrestore(&cpu_base->lock, flags);
+ }
+ }
+
+ smp_call_function_many(mask, retrigger_next_event, NULL, 1);
+ preempt_enable();
+ free_cpumask_var(mask);
#endif
+set_timerfd:
timerfd_clock_was_set();
}
next reply other threads:[~2021-04-07 15:29 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-07 13:53 Marcelo Tosatti [this message]
2021-04-07 19:28 ` [PATCH] hrtimer: avoid retrigger_next_event IPI kernel test robot
2021-04-07 19:28 ` kernel test robot
2021-04-07 22:14 ` Frederic Weisbecker
2021-04-08 12:27 ` Marcelo Tosatti
2021-04-09 14:15 ` Thomas Gleixner
2021-04-09 16:51 ` Marcelo Tosatti
2021-04-10 7:53 ` Thomas Gleixner
2021-04-13 17:04 ` [PATCH v2] " Marcelo Tosatti
2021-04-14 17:19 ` Thomas Gleixner
2021-04-15 15:39 ` [PATCH v3] " Marcelo Tosatti
2021-04-15 18:59 ` Thomas Gleixner
2021-04-15 20:40 ` [PATCH v4] " Marcelo Tosatti
2021-04-16 16:00 ` [PATCH v5] " Marcelo Tosatti
2021-04-16 17:13 ` Peter Xu
2021-04-17 16:24 ` Thomas Gleixner
2021-04-17 16:51 ` Thomas Gleixner
2021-04-19 18:56 ` Marcelo Tosatti
2021-04-19 19:39 ` [PATCH v6] " Marcelo Tosatti
2021-04-19 20:52 ` Thomas Gleixner
-- strict thread matches above, loose matches on Subject: below --
2021-04-07 18:20 [PATCH] " kernel test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210407135301.GA16985@fuller.cnet \
--to=mtosatti@redhat.com \
--cc=abelits@marvell.com \
--cc=frederic@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=nitesh@redhat.com \
--cc=peterx@redhat.com \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.