sched/core: Better document the try_to_wake_up() barriers

Explain how the control dependency and smp_rmb() end up providing
ACQUIRE semantics and pair with smp_store_release() in
finish_lock_switch().

Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Mike Galbraith <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/kernel/sched/core.c b/kernel/sched/core.c
index eee4ee655db2854a7f85090346ca1987b2c36f5f..b64f163d512cbc4adb1477d7d493fd78b3571be2 100644 (file)
--- a/kernel/sched/core.c
+++ b/kernel/sched/core.c
@@ -1953,7 +1953,13 @@ try_to_wake_up(struct task_struct *p, unsigned int state, int wake_flags)
        while (p->on_cpu)
                cpu_relax();
        /*
-        * Pairs with the smp_wmb() in finish_lock_switch().
+        * Combined with the control dependency above, we have an effective
+        * smp_load_acquire() without the need for full barriers.
+        *
+        * Pairs with the smp_store_release() in finish_lock_switch().
+        *
+        * This ensures that tasks getting woken will be fully ordered against
+        * their previous state and preserve Program Order.
         */
        smp_rmb();
 

diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h
index efd3bfc7e34722883e2f08ca82f91cffde812963..b242775bf670e116233862c590915e06132485ca 100644 (file)
--- a/kernel/sched/sched.h
+++ b/kernel/sched/sched.h
@@ -1073,6 +1073,9 @@ static inline void finish_lock_switch(struct rq *rq, struct task_struct *prev)
         * We must ensure this doesn't happen until the switch is completely
         * finished.
         *
+        * In particular, the load of prev->state in finish_task_switch() must
+        * happen before this.
+        *
         * Pairs with the control dependency and rmb in try_to_wake_up().
         */
        smp_store_release(&prev->on_cpu, 0);