From: Alexandre Ghiti <alex@ghiti.fr>
To: Jisheng Zhang <jszhang@kernel.org>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>, Will Deacon <will@kernel.org>,
"Aneesh Kumar K . V" <aneesh.kumar@linux.ibm.com>,
Andrew Morton <akpm@linux-foundation.org>,
Nick Piggin <npiggin@gmail.com>,
Peter Zijlstra <peterz@infradead.org>
Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
linux-arch@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH 3/4] riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU
Date: Sun, 31 Dec 2023 07:32:47 +0100 [thread overview]
Message-ID: <e90f3b0f-95be-4ead-85cf-cca4378755f3@ghiti.fr> (raw)
In-Reply-To: <20231219175046.2496-4-jszhang@kernel.org>
On 19/12/2023 18:50, Jisheng Zhang wrote:
> In order to implement fast gup we need to ensure that the page
> table walker is protected from page table pages being freed from
> under it.
>
> riscv situation is more complicated than other architectures: some
> riscv platforms may use IPI to perform TLB shootdown, for example,
> those platforms which support AIA, usually the riscv_ipi_for_rfence is
> true on these platforms; some riscv platforms may rely on the SBI to
> perform TLB shootdown, usually the riscv_ipi_for_rfence is false on
> these platforms. To keep software pagetable walkers safe in this case
> we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the
> comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in
> include/asm-generic/tlb.h for more details.
>
> This patch enables MMU_GATHER_RCU_TABLE_FREE, then use
>
> *tlb_remove_page_ptdesc() for those platforms which use IPI to perform
> TLB shootdown;
>
> *tlb_remove_ptdesc() for those platforms which use SBI to perform TLB
> shootdown;
Can you elaborate a bit more on what those functions do differently and
why we need to differentiate IPI vs SBI TLB shootdown? I don't
understand this.
Thanks,
Alex
> Both case mean that disabling interrupts will block the free and
> protect the fast gup page walker.
>
> Signed-off-by: Jisheng Zhang <jszhang@kernel.org>
> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/pgalloc.h | 23 ++++++++++++++++++-----
> arch/riscv/include/asm/tlb.h | 18 ++++++++++++++++++
> 3 files changed, 37 insertions(+), 5 deletions(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index 24c1799e2ec4..d3555173d9f4 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -147,6 +147,7 @@ config RISCV
> select IRQ_FORCED_THREADING
> select KASAN_VMALLOC if KASAN
> select LOCK_MM_AND_FIND_VMA
> + select MMU_GATHER_RCU_TABLE_FREE if SMP && MMU
> select MODULES_USE_ELF_RELA if MODULES
> select MODULE_SECTIONS if MODULES
> select OF
> diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h
> index 3c5e3bd15f46..deaf971253a2 100644
> --- a/arch/riscv/include/asm/pgalloc.h
> +++ b/arch/riscv/include/asm/pgalloc.h
> @@ -102,7 +102,10 @@ static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pud,
> struct ptdesc *ptdesc = virt_to_ptdesc(pud);
>
> pagetable_pud_dtor(ptdesc);
> - tlb_remove_page_ptdesc(tlb, ptdesc);
> + if (riscv_use_ipi_for_rfence())
> + tlb_remove_page_ptdesc(tlb, ptdesc);
> + else
> + tlb_remove_ptdesc(tlb, ptdesc);
> }
> }
>
> @@ -136,8 +139,12 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d)
> static inline void __p4d_free_tlb(struct mmu_gather *tlb, p4d_t *p4d,
> unsigned long addr)
> {
> - if (pgtable_l5_enabled)
> - tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d));
> + if (pgtable_l5_enabled) {
> + if (riscv_use_ipi_for_rfence())
> + tlb_remove_page_ptdesc(tlb, virt_to_ptdesc(p4d));
> + else
> + tlb_remove_ptdesc(tlb, virt_to_ptdesc(p4d));
> + }
> }
> #endif /* __PAGETABLE_PMD_FOLDED */
>
> @@ -169,7 +176,10 @@ static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmd,
> struct ptdesc *ptdesc = virt_to_ptdesc(pmd);
>
> pagetable_pmd_dtor(ptdesc);
> - tlb_remove_page_ptdesc(tlb, ptdesc);
> + if (riscv_use_ipi_for_rfence())
> + tlb_remove_page_ptdesc(tlb, ptdesc);
> + else
> + tlb_remove_ptdesc(tlb, ptdesc);
> }
>
> #endif /* __PAGETABLE_PMD_FOLDED */
> @@ -180,7 +190,10 @@ static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
> struct ptdesc *ptdesc = page_ptdesc(pte);
>
> pagetable_pte_dtor(ptdesc);
> - tlb_remove_page_ptdesc(tlb, ptdesc);
> + if (riscv_use_ipi_for_rfence())
> + tlb_remove_page_ptdesc(tlb, ptdesc);
> + else
> + tlb_remove_ptdesc(tlb, ptdesc);
> }
> #endif /* CONFIG_MMU */
>
> diff --git a/arch/riscv/include/asm/tlb.h b/arch/riscv/include/asm/tlb.h
> index 1eb5682b2af6..a0b8b853503f 100644
> --- a/arch/riscv/include/asm/tlb.h
> +++ b/arch/riscv/include/asm/tlb.h
> @@ -10,6 +10,24 @@ struct mmu_gather;
>
> static void tlb_flush(struct mmu_gather *tlb);
>
> +#ifdef CONFIG_MMU
> +#include <linux/swap.h>
> +
> +/*
> + * While riscv platforms with riscv_ipi_for_rfence as true require an IPI to
> + * perform TLB shootdown, some platforms with riscv_ipi_for_rfence as false use
> + * SBI to perform TLB shootdown. To keep software pagetable walkers safe in this
> + * case we switch to RCU based table free (MMU_GATHER_RCU_TABLE_FREE). See the
> + * comment below 'ifdef CONFIG_MMU_GATHER_RCU_TABLE_FREE' in include/asm-generic/tlb.h
> + * for more details.
> + */
> +static inline void __tlb_remove_table(void *table)
> +{
> + free_page_and_swap_cache(table);
> +}
> +
> +#endif /* CONFIG_MMU */
> +
> #define tlb_flush tlb_flush
> #include <asm-generic/tlb.h>
>
next prev parent reply other threads:[~2023-12-31 6:32 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-19 17:50 [PATCH 0/4] riscv: support fast gup Jisheng Zhang
2023-12-19 17:50 ` [PATCH 1/4] riscv: tlb: fix __p*d_free_tlb() Jisheng Zhang
2023-12-31 6:21 ` Alexandre Ghiti
2024-02-07 7:28 ` Alexandre Ghiti
2024-01-04 10:55 ` Alexandre Ghiti
2024-01-10 14:52 ` Palmer Dabbelt
2023-12-19 17:50 ` [PATCH 2/4] riscv: tlb: convert __p*d_free_tlb() to inline functions Jisheng Zhang
2023-12-20 2:59 ` [External] " yunhui cui
2023-12-20 12:57 ` Jisheng Zhang
2023-12-31 6:24 ` Alexandre Ghiti
2023-12-19 17:50 ` [PATCH 3/4] riscv: enable MMU_GATHER_RCU_TABLE_FREE for SMP && MMU Jisheng Zhang
2023-12-31 6:32 ` Alexandre Ghiti [this message]
2024-01-02 3:23 ` Jisheng Zhang
2024-01-04 10:45 ` Alexandre Ghiti
2023-12-19 17:50 ` [PATCH 4/4] riscv: enable HAVE_FAST_GUP if MMU Jisheng Zhang
2023-12-31 6:37 ` Alexandre Ghiti
2024-01-02 3:25 ` Jisheng Zhang
2024-01-04 10:46 ` Alexandre Ghiti
2024-01-25 21:30 ` [PATCH 0/4] riscv: support fast gup patchwork-bot+linux-riscv
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e90f3b0f-95be-4ead-85cf-cca4378755f3@ghiti.fr \
--to=alex@ghiti.fr \
--cc=akpm@linux-foundation.org \
--cc=aneesh.kumar@linux.ibm.com \
--cc=aou@eecs.berkeley.edu \
--cc=jszhang@kernel.org \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux-riscv@lists.infradead.org \
--cc=npiggin@gmail.com \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=peterz@infradead.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).