From: "Chaney, Ben" <bchaney@akamai.com>
To: "ardb@kernel.org" <ardb@kernel.org>,
"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
"linux-efi@vger.kernel.org" <linux-efi@vger.kernel.org>,
"stable@vger.kernel.org" <stable@vger.kernel.org>
Cc: "bp@alien8.de" <bp@alien8.de>,
"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"mingo@redhat.com" <mingo@redhat.com>,
"tglx@linutronix.de" <tglx@linutronix.de>,
"Tottenham, Max" <mtottenh@akamai.com>,
"Hunt, Joshua" <johunt@akamai.com>,
"Galaxy, Michael" <mgalaxy@akamai.com>
Subject: Regression in 6.1.81: Missing memory in pmem device
Date: Wed, 15 May 2024 17:32:27 +0000 [thread overview]
Message-ID: <FA5F6719-8824-4B04-803E-82990E65E627@akamai.com> (raw)
Hello,
I encountered an issue when upgrading to 6.1.89 from 6.1.77. This upgrade caused a breakage in emulated persistent memory. Significant amounts of memory are missing from a pmem device:
fdisk -l /dev/pmem*
Disk /dev/pmem0: 355.9 GiB, 382117871616 bytes, 746323968 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk /dev/pmem1: 25.38 GiB, 27246198784 bytes, 53215232 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
The memmap parameter that created these pmem devices is “memmap=364416M!28672M,367488M!419840M”, which should cause a much larger amount of memory to be allocated to /dev/pmem1. The amount of missing memory and the device it is missing from is randomized on each reboot. There is some amount of memory missing in almost all cases, but not 100% of the time. Notably, the memory that is missing from these devices is not reclaimed by the system for general use. This system in question has 768GB of memory split evenly across two NUMA nodes.
When the error occurs, there are also the following error messages showing up in dmesg:
[ 5.318317] nd_pmem namespace1.0: [mem 0x5c2042c000-0x5ff7ffffff flags 0x200] misaligned, unable to map
[ 5.335073] nd_pmem: probe of namespace1.0 failed with error -95
Bisection implicates 2dfaeac3f38e4e550d215204eedd97a061fdc118 as the patch that first caused the issue. I believe the cause of the issue is that the EFI stub is randomizing the location of the decompressed kernel without accounting for the memory map, and it is clobbering some of the memory that has been reserved for pmem.
Thank you,
Ben Chaney
next reply other threads:[~2024-05-15 18:05 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-15 17:32 Chaney, Ben [this message]
2024-05-15 17:42 ` Regression in 6.1.81: Missing memory in pmem device Ard Biesheuvel
2024-05-15 18:30 ` Kees Cook
2024-05-16 14:59 ` Chaney, Ben
2024-05-16 17:22 ` Ard Biesheuvel
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=FA5F6719-8824-4B04-803E-82990E65E627@akamai.com \
--to=bchaney@akamai.com \
--cc=ardb@kernel.org \
--cc=bp@alien8.de \
--cc=dave.hansen@linux.intel.com \
--cc=gregkh@linuxfoundation.org \
--cc=johunt@akamai.com \
--cc=linux-efi@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mgalaxy@akamai.com \
--cc=mingo@redhat.com \
--cc=mtottenh@akamai.com \
--cc=stable@vger.kernel.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).