sched: sched_slice() fixlet

Mike's change: 0a582440f "sched: fix sched_slice())" broke group
scheduling by forgetting to reload cfs_rq on each loop.

This patch fixes aim7 regression and specjbb2005 regression becomes
less than 1.5% on 8-core stokley.

Signed-off-by: Lin Ming <[email protected]>
Signed-off-by: Peter Zijlstra <[email protected]>
Tested-by: Jayson King <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/kernel/sched_fair.c b/kernel/sched_fair.c
index 16b419bb8b0aa25da8118e2885427e2f907e0c3f..5cc1c162044fc4f40b15c351cd170ce2c58703f7 100644 (file)
--- a/kernel/sched_fair.c
+++ b/kernel/sched_fair.c
@@ -429,7 +429,10 @@ static u64 sched_slice(struct cfs_rq *cfs_rq, struct sched_entity *se)
        u64 slice = __sched_period(cfs_rq->nr_running + !se->on_rq);
 
        for_each_sched_entity(se) {
-               struct load_weight *load = &cfs_rq->load;
+               struct load_weight *load;
+
+               cfs_rq = cfs_rq_of(se);
+               load = &cfs_rq->load;
 
                if (unlikely(!se->on_rq)) {
                        struct load_weight lw = cfs_rq->load;