Userspace pages in UC mode

From: matlackdavid@gmail.com (David Matlack)
To: kernelnewbies@lists.kernelnewbies.org
Subject: Userspace pages in UC mode
Date: Fri, 11 Sep 2015 11:10:15 -0700	[thread overview]
Message-ID: <CANoiuGL5Hs4_0kN_soq5VEa2Noh_UwV_tYWuJ_AVH2+ATRK3DA@mail.gmail.com> (raw)
In-Reply-To: <CANtw=au_mvON0A0q1smbP74H_KTh8mzc+an81e__zcZUT7mVcQ@mail.gmail.com>

On Fri, Sep 11, 2015 at 7:59 AM, Sabela Ramos Garea
<sabelaraga@gmail.com> wrote:
> Sorry, little mistake copypasting and cleaning. The pages and vma
> structs should look like that:
>
> struct page *pages --> struct page *pages[MAX_PAGES];
> struct vma_area_struct *vma --> struct vma_area_struct *vma[MAX_PAGES];
>
> Where MAX_PAGES is defined to 5.
>
> Sabela.
>
> 2015-09-11 16:07 GMT+02:00 Sabela Ramos Garea <sabelaraga@gmail.com>:
>> Dear all,
>>
>> For research purposes I need some userspace memory pages to be in
>> uncacheable mode. I am using two different Intel architectures (Sandy
>> Bridge and Haswell) and two different kernels (2.6.32-358 and
>> 3.19.0-28).
>>
>> The non-temporal stores from Intel assembly are not a valid solution
>> so I am programming a kernel module that gets a set of pages from user
>> space reserved with posix_memalign (get_user_pages) and then sets them
>> as uncacheable (I have tried set_pages_uc and set_pages_array_uc).
>> When I use one page, the access times are not very coherent and with
>> more than one page the module crashes (in both architectures and both
>> kernels).

set_pages_uc might just set the kernel virtual address mappings for
those pages as uncacheable via the Page Attribute Table, but you are
accessing from userspace virtual addresses. If you have the userspace
virtual address of the pages you can manually twiddle the PAT bits of
the page tables in your module to make the pages UC.

>>
>> I wonder if I am using the correct approach or if I have to use kernel
>> space pages in order to work with uncacheable memory. Or if I have to
>> remap the memory. Just in case it makes it clearer, I am attaching the
>> relevant lines of a kernel module function that should set the pages
>> as uncacheable. (This function is the .write of a misc device; count
>> is treated as the number of pages).
>>
>> Best and Thanks,
>>
>> Sabela.
>>
>> struct page *pages; //defined outside in order to be able to set them
>> to WB in the release function.
>> int numpages;
>>
>> static ssize_t setup_memory(struct file *filp, const char __user *buf,
>> size_t count, loff_t * ppos)
>> {
>>         int res;
>>         struct vm_area_struct *vmas;
>>
>>         numpages = count/4096;
>>
>>         down_read(&current->mm->mmap_sem);
>>         res = get_user_pages(current, current->mm,
>>                                 (unsigned long) buf,
>>                                 numpages, /* Number of pages */
>>                                 0, /* Do want to write into it */
>>                                 1, /* do force */
>>                                 &pages,
>>                                 &vmas);
>>         up_read(&current->mm->mmap_sem);
>>
>>         numpages=res;
>>
>>         if (res > 0) {
>>                 set_pages_uc(pages, numpages); /* Uncached */
>>                 printk("Write: %d pages set as uncacheable\n",numpages);
>>         }
>>         else{
>>                 pr_err("Couldn't get pages to set them as UC :(\n");
>>                 return -EAGAIN;
>>         }
>> }
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies