From: George Dunlap <George.Dunlap@eu.citrix.com>
To: Jan Beulich <JBeulich@suse.com>
Cc: Wei Liu <wei.liu2@citrix.com>,
Ian Campbell <ian.campbell@citrix.com>,
Stefano Stabellini <stefano.stabellini@eu.citrix.com>,
Andrew Cooper <andrew.cooper3@citrix.com>,
Ian Jackson <ian.jackson@eu.citrix.com>,
"xen-devel@lists.xen.org" <xen-devel@lists.xen.org>,
Tiejun Chen <tiejun.chen@intel.com>, Keir Fraser <keir@xen.org>
Subject: Re: [v7][PATCH 06/16] hvmloader/pci: skip reserved ranges
Date: Wed, 15 Jul 2015 16:19:54 +0100 [thread overview]
Message-ID: <CAFLBxZZXWTL8JmBrR3x=knCxavtH+ExAFa7z9vx5O_eThFSiOA@mail.gmail.com> (raw)
In-Reply-To: <55A683A5020000780009162B@mail.emea.novell.com>
On Wed, Jul 15, 2015 at 3:00 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 15.07.15 at 15:40, <dunlapg@umich.edu> wrote:
>> On Mon, Jul 13, 2015 at 2:12 PM, Jan Beulich <JBeulich@suse.com> wrote:
>>> Therefore I'll not make any further comments on the rest of the
>>> patch, but instead outline an allocation model that I think would
>>> fit our needs: Subject to the constraints mentioned above, set up
>>> a bitmap (maximum size 64k [2Gb = 2^^19 pages needing 2^^19
>>> bits], i.e. reasonably small a memory block). Each bit represents a
>>> page usable for MMIO: First of all you remove the range from
>>> PCI_MEM_END upwards. Then remove all RDM pages. Now do a
>>> first pass over all devices, allocating (in the bitmap) space for only
>>> the 32-bit MMIO BARs, starting with the biggest one(s), by finding
>>> a best fit (i.e. preferably a range not usable by any bigger BAR)
>>> from top down. For example, if you have available
>>>
>>> [f0000000,f8000000)
>>> [f9000000,f9001000)
>>> [fa000000,fa003000)
>>> [fa010000,fa012000)
>>>
>>> and you're looking for a single page slot, you should end up
>>> picking fa002000.
>>>
>>> After this pass you should be able to do RAM relocation in a
>>> single attempt just like we do today (you may still grow the MMIO
>>> window if you know you need to and can fit some of the 64-bit
>>> BARs in there, subject to said constraints; this is in an attempt
>>> to help OSes not comfortable with 64-bit resources).
>>>
>>> In a 2nd pass you'd then assign 64-bit resources: If you can fit
>>> them below 4G (you still have the bitmap left of what you've got
>>> available), put them there. Allocation strategy could be the same
>>> as above (biggest first), perhaps allowing for some factoring out
>>> of logic, but here smallest first probably could work equally well.
>>> The main thought to decide between the two is whether it is
>>> better to fit as many (small) or as big (in total) as possible a set
>>> under 4G. I'd generally expect the former (as many as possible,
>>> leaving only a few huge ones to go above 4G) to be the better
>>> approach, but that's more a gut feeling than based on hard data.
>>
>> I agree that it would be more sensible for hvmloader to make a "plan"
>> first, and then do the memory reallocation (if it's possible) at one
>> time, then go through and actually update the device BARs according to
>> the "plan".
>>
>> However, I don't really see how having a bitmap really helps in this
>> case. I would think having a list of free ranges (perhaps aligned by
>> powers of two?), sorted small->large, makes the most sense.
>
> I view bitmap vs list as just two different representations, and I
> picked the bitmap approach as being more compact storage wise
> in case there are many regions to deal with. I'd be fine with a list
> approach too, provided lookup times don't become prohibitive.
Sure, you can obviously translate one into the other. The main reason
I dislike the idea of a bitmap is having to write code to determine
where the next free region is, and how big that region is, rather than
just going down the next on the list and reading range.start and
range.len.
Also, in your suggestion each bit is a page (4k); so assuming a 64-bit
pointer, a 64-bit starting point, and a 64-bit length (juts to make
things simple), a single "range" takes up enough bitmap to reserve
(64+64+64)*4k = 768k. So if we make the bitmap big enough for 2GiB,
then the break-even point for storage is 2,730 ranges. It's even
higher if we have an array instead of a linked list.
I'm pretty sure that having such a large number of ranges will be
vanishingly rare; I'd expect the number of ranges so the "range"
representation will not only be easier to code and read, but will in
the common case (I believe) be far more compact.
-George
next prev parent reply other threads:[~2015-07-15 15:19 UTC|newest]
Thread overview: 119+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-07-09 5:33 [v7][PATCH 00/16] Fix RMRR Tiejun Chen
2015-07-09 5:33 ` [v7][PATCH 01/16] xen: introduce XENMEM_reserved_device_memory_map Tiejun Chen
2015-07-09 5:33 ` [v7][PATCH 02/16] xen/vtd: create RMRR mapping Tiejun Chen
2015-07-09 5:33 ` [v7][PATCH 03/16] xen/passthrough: extend hypercall to support rdm reservation policy Tiejun Chen
2015-07-10 13:26 ` George Dunlap
2015-07-10 15:01 ` Jan Beulich
2015-07-10 15:07 ` George Dunlap
2015-07-13 6:37 ` Chen, Tiejun
2015-07-13 5:57 ` Chen, Tiejun
2015-07-13 6:47 ` Chen, Tiejun
2015-07-13 8:57 ` Jan Beulich
2015-07-14 10:46 ` George Dunlap
2015-07-14 10:53 ` Chen, Tiejun
2015-07-14 11:30 ` George Dunlap
2015-07-14 11:45 ` Jan Beulich
2015-07-14 13:25 ` George Dunlap
2015-07-09 5:33 ` [v7][PATCH 04/16] xen: enable XENMEM_memory_map in hvm Tiejun Chen
2015-07-09 5:33 ` [v7][PATCH 05/16] hvmloader: get guest memory map into memory_map[] Tiejun Chen
2015-07-10 13:49 ` George Dunlap
2015-07-13 7:03 ` Chen, Tiejun
2015-07-09 5:33 ` [v7][PATCH 06/16] hvmloader/pci: skip reserved ranges Tiejun Chen
2015-07-13 13:12 ` Jan Beulich
2015-07-14 6:39 ` Chen, Tiejun
2015-07-14 9:27 ` Jan Beulich
2015-07-14 10:54 ` Chen, Tiejun
2015-07-14 11:50 ` Jan Beulich
2015-07-15 0:55 ` Chen, Tiejun
2015-07-15 4:27 ` Chen, Tiejun
2015-07-15 8:34 ` Jan Beulich
2015-07-15 8:59 ` Chen, Tiejun
2015-07-15 9:10 ` Chen, Tiejun
2015-07-15 9:27 ` Jan Beulich
2015-07-15 10:34 ` Chen, Tiejun
2015-07-15 11:25 ` Jan Beulich
2015-07-15 11:34 ` Chen, Tiejun
2015-07-15 13:56 ` George Dunlap
2015-07-15 16:14 ` George Dunlap
2015-07-16 2:05 ` Chen, Tiejun
2015-07-16 9:40 ` George Dunlap
2015-07-16 10:01 ` Chen, Tiejun
2015-07-15 11:05 ` George Dunlap
2015-07-15 11:20 ` Chen, Tiejun
2015-07-15 12:43 ` George Dunlap
2015-07-15 13:23 ` Chen, Tiejun
2015-07-15 11:24 ` Jan Beulich
2015-07-15 11:38 ` George Dunlap
2015-07-15 11:27 ` Jan Beulich
2015-07-15 11:40 ` Chen, Tiejun
2015-07-15 8:32 ` Jan Beulich
2015-07-15 9:04 ` Chen, Tiejun
2015-07-15 12:57 ` Wei Liu
2015-07-15 13:40 ` George Dunlap
2015-07-15 14:00 ` Jan Beulich
2015-07-15 15:19 ` George Dunlap [this message]
2015-07-09 5:33 ` [v7][PATCH 07/16] hvmloader/e820: construct guest e820 table Tiejun Chen
2015-07-13 13:35 ` Jan Beulich
2015-07-14 5:22 ` Chen, Tiejun
2015-07-14 9:32 ` Jan Beulich
2015-07-14 10:22 ` Chen, Tiejun
2015-07-14 10:48 ` Jan Beulich
2015-07-15 16:00 ` George Dunlap
2015-07-16 1:58 ` Chen, Tiejun
2015-07-16 9:41 ` George Dunlap
2015-07-09 5:33 ` [v7][PATCH 08/16] tools/libxc: Expose new hypercall xc_reserved_device_memory_map Tiejun Chen
2015-07-09 5:34 ` [v7][PATCH 09/16] tools: extend xc_assign_device() to support rdm reservation policy Tiejun Chen
2015-07-09 5:34 ` [v7][PATCH 10/16] tools: introduce some new parameters to set rdm policy Tiejun Chen
2015-07-09 9:20 ` Wei Liu
2015-07-09 9:44 ` Chen, Tiejun
2015-07-09 10:37 ` Ian Jackson
2015-07-09 10:53 ` Chen, Tiejun
2015-07-09 18:02 ` Ian Jackson
2015-07-10 0:46 ` Chen, Tiejun
2015-07-09 5:34 ` [v7][PATCH 11/16] tools/libxl: detect and avoid conflicts with RDM Tiejun Chen
2015-07-09 9:11 ` Wei Liu
2015-07-09 9:41 ` Chen, Tiejun
2015-07-09 18:14 ` Ian Jackson
2015-07-10 3:19 ` Chen, Tiejun
2015-07-10 10:14 ` Ian Jackson
2015-07-13 9:19 ` Chen, Tiejun
2015-07-09 5:34 ` [v7][PATCH 12/16] tools: introduce a new parameter to set a predefined rdm boundary Tiejun Chen
2015-07-09 18:14 ` Ian Jackson
2015-07-09 5:34 ` [v7][PATCH 13/16] libxl: construct e820 map with RDM information for HVM guest Tiejun Chen
2015-07-09 18:17 ` Ian Jackson
2015-07-10 5:40 ` Chen, Tiejun
2015-07-10 9:18 ` Ian Campbell
2015-07-13 9:47 ` Chen, Tiejun
2015-07-13 10:15 ` Ian Campbell
2015-07-14 5:44 ` Chen, Tiejun
2015-07-14 7:42 ` Ian Campbell
2015-07-14 8:03 ` Chen, Tiejun
2015-07-10 10:15 ` Ian Jackson
2015-07-09 5:34 ` [v7][PATCH 14/16] xen/vtd: enable USB device assignment Tiejun Chen
2015-07-09 5:34 ` [v7][PATCH 15/16] xen/vtd: prevent from assign the device with shared rmrr Tiejun Chen
2015-07-13 13:41 ` Jan Beulich
2015-07-14 1:42 ` Chen, Tiejun
2015-07-14 9:19 ` Jan Beulich
2015-07-09 5:34 ` [v7][PATCH 16/16] tools: parse to enable new rdm policy parameters Tiejun Chen
2015-07-09 18:23 ` Ian Jackson
2015-07-10 6:05 ` Chen, Tiejun
2015-07-10 10:23 ` Ian Jackson
2015-07-13 9:31 ` Chen, Tiejun
2015-07-13 9:40 ` Ian Campbell
2015-07-13 9:55 ` Chen, Tiejun
2015-07-13 10:17 ` Ian Campbell
2015-07-13 17:08 ` Ian Jackson
2015-07-14 1:29 ` Chen, Tiejun
2015-07-10 14:50 ` [v7][PATCH 00/16] Fix RMRR George Dunlap
2015-07-10 14:56 ` Jan Beulich
2015-07-16 7:55 ` Jan Beulich
2015-07-16 8:03 ` Chen, Tiejun
2015-07-16 8:08 ` Jan Beulich
2015-07-16 8:13 ` Chen, Tiejun
2015-07-16 8:26 ` Jan Beulich
2015-07-16 9:27 ` George Dunlap
2015-07-16 9:44 ` Jan Beulich
2015-07-16 9:59 ` George Dunlap
2015-07-16 8:30 ` Ian Campbell
2015-07-16 8:46 ` Wei Liu
2015-07-16 9:45 ` Lars Kurth
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFLBxZZXWTL8JmBrR3x=knCxavtH+ExAFa7z9vx5O_eThFSiOA@mail.gmail.com' \
--to=george.dunlap@eu.citrix.com \
--cc=JBeulich@suse.com \
--cc=andrew.cooper3@citrix.com \
--cc=ian.campbell@citrix.com \
--cc=ian.jackson@eu.citrix.com \
--cc=keir@xen.org \
--cc=stefano.stabellini@eu.citrix.com \
--cc=tiejun.chen@intel.com \
--cc=wei.liu2@citrix.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.