mm: numa: guarantee that tlb_flush_pending updates are visible before page table...

According to documentation on barriers, stores issued before a LOCK can
complete after the lock implying that it's possible tlb_flush_pending
can be visible after a page table update.  As per revised documentation,
this patch adds a smp_mb__before_spinlock to guarantee the correct
ordering.

Signed-off-by: Mel Gorman <[email protected]>
Acked-by: Paul E. McKenney <[email protected]>
Reviewed-by: Rik van Riel <[email protected]>
Cc: <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index e5c49c30460f12ae42a8873ad0e1c733f9d64411..ad0616f2fe2c38d825d899c00f4e79e696c291ac 100644 (file)
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -482,7 +482,12 @@ static inline bool mm_tlb_flush_pending(struct mm_struct *mm)
 static inline void set_tlb_flush_pending(struct mm_struct *mm)
 {
        mm->tlb_flush_pending = true;
-       barrier();
+
+       /*
+        * Guarantee that the tlb_flush_pending store does not leak into the
+        * critical section updating the page tables
+        */
+       smp_mb__before_spinlock();
 }
 /* Clearing is done after a TLB flush, which also provides a barrier. */
 static inline void clear_tlb_flush_pending(struct mm_struct *mm)