Thanks for the reply, budget enforcement in the scheduler timer makes
sense. I think I have an idea of what he wants done now.

~Dagaen
On Jun 17, 2015 1:45 AM, "Meng Xu" <xumengpanda@gmail.com> wrote:

> Hi Dagaen,
>
> I just comment on the summary of scheduler design you proposed at the
> end of the email. I'm looking forward to Dario's more insightful
> reply.
> >
> >
> > >
> > >> I simply
> > >> don't see how it can
> > >> be done without heavy interaction and information sharing between them
> > >> which really
> > >> defeats the purpose.
> > >>
> > > No, it doesn't.
> >
> > Ok this last line is the zinger.
> >
> > Almost this entire email was built on the premise that you would NOT
> like the
> > idea of the replenishment handler having basically the same decision
> logic
> > as the scheduler and then tickling the pCPU to pick up the new vCPU.
> Actually,
> > with it done this way, I would have a hard time calling the
> > tickle-invoked method
> > the "scheduler." It would be more like the mover, with the
> > replenishment function
> > being essentially the scheduler. In this case, I'm not too sure
> > actually how much
> > different this would be from what we have now. It feels like, from
> > what you want,
> > that we could get the same behavior by modifying rt_schedule to do
> > replenishments
> > first, then check if a reschedule is needed (these two parts would be in
> this
> > proposed replenishment timer routine) and the perform any move if
> necessary. I
> > know this isn't exactly what you want, but that sounds close right?
> >
> > But the scheduler will have to decide which to move in, so that logic
> > is done twice.
> > Also, if these are done back-to-back and require the locks, isn't it
> > really the same
> > as having one long hot path? If you want maintainability, couldn't we
> just do
> > replenishment then schedule and move (if necessary) in one timer (the
> > one we have
> > now) and move them to functions. It seems this can be done with one
> > timer neatly.
> >
> > So here's my proposal, lets see if it fits what you want:
>
>
> I will leave this to Dario to answer if it fits what he wants. :-P
>
>
> >
> > 1.) The scheduler does not do any timing,
>
>
> Not really. The rt_schedule does the budget enforcement. When a
> current VCPU runs out of budget, the rt_schedule will be invoked by a
> timer (you can refer to the schedule function in xen/sched/schedule.c
> to have a look how the timer is armed/disarmed.). When the rt_schedule
> is invoked, it needs to:
> a) update the budget of the current running VCPU and move it from runq
> to depleted queue;
> b) pick the current highest VCPU from runq and return it as the snext VCPU.
>
> So scheduling logic is still  involved in the rt_schedule function.
>
> >
> > 2.) replenishments are scheduled via timer at each [next]
> > replenishment period. Each
> > replenishment resorts the replenished vCPUs, and if any of the first
> > #CPUs in the
> > runq change, we tickle a pCPU for each change
>
>
> This is right.
>
> >
> >
> > In this case, we can use one timer.
>
>
> We actually have two: one for budget enforcement and the other for
> budget replenishment.
>
>
> >
> > We could use the current one as a
> > replenishment
> > timer, and make it so rt_schedule isn't the default invoked method.
> >
> > Does this sound similar to what you are suggesting?
>
>
> I don't think so, but I will leave it for Dario's for his opinion.
>
> In Dario's suggestion, you just simply remove the update_budget
> function out of rt_schedule. This is because budget enforcement, which
> is the work of rt_schedule, does not naturelly involves budget
> replenishment.
>
> In order to achieve budget replenishment, we need another timer to
> invoke another function (budget_replenish function), that is dedicated
> to that.
>
> >
> > I have to ask
> > because I thought
> > you wouldn't like the idea,
>
>
> I guess Dario won't like this idea. :-P (I'm kidding, but it should be
> the case.)
>
>
> >
> > and its not really *that* far off from
> > what we have now, Its
> > a little restructuring so that replenishments occur before any
> > scheduling activity and
> > the handler checks if switching is needed (basically acting as the
> > scheduler) and then
> > calls tickle. Sounds like what you had in mind?
>
>
> Thanks and best regards,
>
> Meng
> --
>
>
> -----------
> Meng Xu
> PhD Student in Computer and Information Science
> University of Pennsylvania
> http://www.cis.upenn.edu/~mengxu/
>