sched/core: Fix remote wakeups
authorPeter Zijlstra <[email protected]>
Mon, 23 May 2016 09:19:07 +0000 (11:19 +0200)
committerIngo Molnar <[email protected]>
Wed, 25 May 2016 06:35:18 +0000 (08:35 +0200)
commitb7e7ade34e6188bee2e3b0d42b51d25137d9e2a5
tree7cb4526b1902a93c7197a3c11150e616d565f0d5
parent2e636d5e66c35dfcbaf617aa8fa963f6847478fe
sched/core: Fix remote wakeups

Commit:

  b5179ac70de8 ("sched/fair: Prepare to fix fairness problems on migration")

... introduced a bug: Mike Galbraith found that it introduced a
performance regression, while Paul E. McKenney reported lost
wakeups and bisected it to this commit.

The reason is that I mis-read ttwu_queue() such that I assumed any
wakeup that got a remote queue must have had the task migrated.

Since this is not so; we need to transfer this information between
queueing the wakeup and actually doing the wakeup. Use a new
task_struct::sched_flag for this, we already write to
sched_contributes_to_load in the wakeup path so this is a hot and
modified cacheline.

Reported-by: Paul E. McKenney <[email protected]>
Reported-by: Mike Galbraith <[email protected]>
Tested-by: Mike Galbraith <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: Andrew Hunter <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Ben Segall <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: Fenghua Yu <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Matt Fleming <[email protected]>
Cc: Morten Rasmussen <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Paul Turner <[email protected]>
Cc: Pavan Kondeti <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Quentin Casasnovas <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Fixes: b5179ac70de8 ("sched/fair: Prepare to fix fairness problems on migration")
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
include/linux/sched.h
kernel/sched/core.c