From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dagaen Golomb Subject: Re: [PATCH] Modified RTDS scheduler to use an event-driven model instead of polling. Date: Wed, 17 Jun 2015 02:03:59 -0400 Message-ID: References: <1433764019-2194-1-git-send-email-dgolomb@seas.upenn.edu> <1433854393.2403.141.camel@citrix.com> <1433975412.2482.116.camel@citrix.com> <1434500436.2420.154.camel@citrix.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============5530625142775654799==" Return-path: In-Reply-To: List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Meng Xu Cc: Wei Liu , Sisu Xi , George Dunlap , Dario Faggioli , "xen-devel@lists.xen.org" , Meng Xu , Jan Beulich , Chong Li List-Id: xen-devel@lists.xenproject.org --===============5530625142775654799== Content-Type: multipart/alternative; boundary=001a113ce90cff19ae0518b07462 --001a113ce90cff19ae0518b07462 Content-Type: text/plain; charset=UTF-8 Thanks for the reply, budget enforcement in the scheduler timer makes sense. I think I have an idea of what he wants done now. ~Dagaen On Jun 17, 2015 1:45 AM, "Meng Xu" wrote: > Hi Dagaen, > > I just comment on the summary of scheduler design you proposed at the > end of the email. I'm looking forward to Dario's more insightful > reply. > > > > > > > > > >> I simply > > >> don't see how it can > > >> be done without heavy interaction and information sharing between them > > >> which really > > >> defeats the purpose. > > >> > > > No, it doesn't. > > > > Ok this last line is the zinger. > > > > Almost this entire email was built on the premise that you would NOT > like the > > idea of the replenishment handler having basically the same decision > logic > > as the scheduler and then tickling the pCPU to pick up the new vCPU. > Actually, > > with it done this way, I would have a hard time calling the > > tickle-invoked method > > the "scheduler." It would be more like the mover, with the > > replenishment function > > being essentially the scheduler. In this case, I'm not too sure > > actually how much > > different this would be from what we have now. It feels like, from > > what you want, > > that we could get the same behavior by modifying rt_schedule to do > > replenishments > > first, then check if a reschedule is needed (these two parts would be in > this > > proposed replenishment timer routine) and the perform any move if > necessary. I > > know this isn't exactly what you want, but that sounds close right? > > > > But the scheduler will have to decide which to move in, so that logic > > is done twice. > > Also, if these are done back-to-back and require the locks, isn't it > > really the same > > as having one long hot path? If you want maintainability, couldn't we > just do > > replenishment then schedule and move (if necessary) in one timer (the > > one we have > > now) and move them to functions. It seems this can be done with one > > timer neatly. > > > > So here's my proposal, lets see if it fits what you want: > > > I will leave this to Dario to answer if it fits what he wants. :-P > > > > > > 1.) The scheduler does not do any timing, > > > Not really. The rt_schedule does the budget enforcement. When a > current VCPU runs out of budget, the rt_schedule will be invoked by a > timer (you can refer to the schedule function in xen/sched/schedule.c > to have a look how the timer is armed/disarmed.). When the rt_schedule > is invoked, it needs to: > a) update the budget of the current running VCPU and move it from runq > to depleted queue; > b) pick the current highest VCPU from runq and return it as the snext VCPU. > > So scheduling logic is still involved in the rt_schedule function. > > > > > 2.) replenishments are scheduled via timer at each [next] > > replenishment period. Each > > replenishment resorts the replenished vCPUs, and if any of the first > > #CPUs in the > > runq change, we tickle a pCPU for each change > > > This is right. > > > > > > > In this case, we can use one timer. > > > We actually have two: one for budget enforcement and the other for > budget replenishment. > > > > > > We could use the current one as a > > replenishment > > timer, and make it so rt_schedule isn't the default invoked method. > > > > Does this sound similar to what you are suggesting? > > > I don't think so, but I will leave it for Dario's for his opinion. > > In Dario's suggestion, you just simply remove the update_budget > function out of rt_schedule. This is because budget enforcement, which > is the work of rt_schedule, does not naturelly involves budget > replenishment. > > In order to achieve budget replenishment, we need another timer to > invoke another function (budget_replenish function), that is dedicated > to that. > > > > > I have to ask > > because I thought > > you wouldn't like the idea, > > > I guess Dario won't like this idea. :-P (I'm kidding, but it should be > the case.) > > > > > > and its not really *that* far off from > > what we have now, Its > > a little restructuring so that replenishments occur before any > > scheduling activity and > > the handler checks if switching is needed (basically acting as the > > scheduler) and then > > calls tickle. Sounds like what you had in mind? > > > Thanks and best regards, > > Meng > -- > > > ----------- > Meng Xu > PhD Student in Computer and Information Science > University of Pennsylvania > http://www.cis.upenn.edu/~mengxu/ > --001a113ce90cff19ae0518b07462 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Thanks for the reply, budget enforcement in the scheduler ti= mer makes sense. I think I have an idea of what he wants done now.

~Dagaen

On Jun 17, 2015 1:45 AM, "Meng Xu" <= ;xumengpanda@gmail.com> wro= te:
Hi Dagaen,

I just comment on the summary of scheduler design you proposed at the
end of the email. I'm looking forward to Dario's more insightful reply.
>
>
> >
> >> I simply
> >> don't see how it can
> >> be done without heavy interaction and information sharing bet= ween them
> >> which really
> >> defeats the purpose.
> >>
> > No, it doesn't.
>
> Ok this last line is the zinger.
>
> Almost this entire email was built on the premise that you would NOT l= ike the
> idea of the replenishment handler having basically the same decision l= ogic
> as the scheduler and then tickling the pCPU to pick up the new vCPU. A= ctually,
> with it done this way, I would have a hard time calling the
> tickle-invoked method
> the "scheduler." It would be more like the mover, with the > replenishment function
> being essentially the scheduler. In this case, I'm not too sure > actually how much
> different this would be from what we have now. It feels like, from
> what you want,
> that we could get the same behavior by modifying rt_schedule to do
> replenishments
> first, then check if a reschedule is needed (these two parts would be = in this
> proposed replenishment timer routine) and the perform any move if nece= ssary. I
> know this isn't exactly what you want, but that sounds close right= ?
>
> But the scheduler will have to decide which to move in, so that logic<= br> > is done twice.
> Also, if these are done back-to-back and require the locks, isn't = it
> really the same
> as having one long hot path? If you want maintainability, couldn't= we just do
> replenishment then schedule and move (if necessary) in one timer (the<= br> > one we have
> now) and move them to functions. It seems this can be done with one > timer neatly.
>
> So here's my proposal, lets see if it fits what you want:


I will leave this to Dario to answer if it fits what he wants. :-P


>
> 1.) The scheduler does not do any timing,


Not really. The rt_schedule does the budget enforcement. When a
current VCPU runs out of budget, the rt_schedule will be invoked by a
timer (you can refer to the schedule function in xen/sched/schedule.c
to have a look how the timer is armed/disarmed.). When the rt_schedule
is invoked, it needs to:
a) update the budget of the current running VCPU and move it from runq
to depleted queue;
b) pick the current highest VCPU from runq and return it as the snext VCPU.=

So scheduling logic is still=C2=A0 involved in the rt_schedule function.
>
> 2.) replenishments are scheduled via timer at each [next]
> replenishment period. Each
> replenishment resorts the replenished vCPUs, and if any of the first > #CPUs in the
> runq change, we tickle a pCPU for each change


This is right.

>
>
> In this case, we can use one timer.


We actually have two: one for budget enforcement and the other for
budget replenishment.


>
> We could use the current one as a
> replenishment
> timer, and make it so rt_schedule isn't the default invoked method= .
>
> Does this sound similar to what you are suggesting?


I don't think so, but I will leave it for Dario's for his opinion.<= br>
In Dario's suggestion, you just simply remove the update_budget
function out of rt_schedule. This is because budget enforcement, which
is the work of rt_schedule, does not naturelly involves budget
replenishment.

In order to achieve budget replenishment, we need another timer to
invoke another function (budget_replenish function), that is dedicated
to that.

>
> I have to ask
> because I thought
> you wouldn't like the idea,


I guess Dario won't like this idea. :-P (I'm kidding, but it should= be
the case.)


>
> and its not really *that* far off from
> what we have now, Its
> a little restructuring so that replenishments occur before any
> scheduling activity and
> the handler checks if switching is needed (basically acting as the
> scheduler) and then
> calls tickle. Sounds like what you had in mind?


Thanks and best regards,

Meng
--


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/
--001a113ce90cff19ae0518b07462-- --===============5530625142775654799== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel --===============5530625142775654799==--