mm/swap_slots.c: don't disable preemption while taking the per-CPU cache

get_cpu_var() disables preemption and returns the per-CPU version of the
variable.  Disabling preemption is useful to ensure atomic access to the
variable within the critical section.

In this case however, after the per-CPU version of the variable is
obtained the ->free_lock is acquired.  For that reason it seems the raw
accessor could be used.  It only seems that ->slots_ret should be
retested (because with disabled preemption this variable can not be set
to NULL otherwise).

This popped up during PREEMPT-RT testing because it tries to take
spinlocks in a preempt disabled section.  In RT, spinlocks can sleep.

Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Sebastian Andrzej Siewior <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Cc: Tim Chen <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Ying Huang <[email protected]>
Cc: Hugh Dickins <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/mm/swap_slots.c b/mm/swap_slots.c
index 90c1032a8ac30ef15af2766d28ccbe0ba3fa2217..13a174006b91234c395d186e6bc9e77319470be0 100644 (file)
--- a/mm/swap_slots.c
+++ b/mm/swap_slots.c
@@ -273,11 +273,11 @@ int free_swap_slot(swp_entry_t entry)
 {
        struct swap_slots_cache *cache;
 
-       cache = &get_cpu_var(swp_slots);
+       cache = raw_cpu_ptr(&swp_slots);
        if (use_swap_slot_cache && cache->slots_ret) {
                spin_lock_irq(&cache->free_lock);
                /* Swap slots cache may be deactivated before acquiring lock */
-               if (!use_swap_slot_cache) {
+               if (!use_swap_slot_cache || !cache->slots_ret) {
                        spin_unlock_irq(&cache->free_lock);
                        goto direct_free;
                }
@@ -297,7 +297,6 @@ int free_swap_slot(swp_entry_t entry)
 direct_free:
                swapcache_free_entries(&entry, 1);
        }
-       put_cpu_var(swp_slots);
 
        return 0;
 }