mm, vmscan: make zone_reclaimable_pages more precise

zone_reclaimable_pages() is used in should_reclaim_retry() which uses it
to calculate the target for the watermark check.  This means that
precise numbers are important for the correct decision.
zone_reclaimable_pages uses zone_page_state which can contain stale data
with per-cpu diffs not synced yet (the last vmstat_update might have run
1s in the past).

Use zone_page_state_snapshot() in zone_reclaimable_pages() instead.
None of the current callers is in a hot path where getting the precise
value (which involves per-cpu iteration) would cause an unreasonable
overhead.

Signed-off-by: Michal Hocko <[email protected]>
Signed-off-by: Tetsuo Handa <[email protected]>
Suggested-by: David Rientjes <[email protected]>
Acked-by: David Rientjes <[email protected]>
Acked-by: Hillf Danton <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/mm/vmscan.c b/mm/vmscan.c
index db5722a07d4fbd5712b68dadfce68be7c0e48071..039f08d369a56451c2328c6783d77c689ca1bbee 100644 (file)
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -195,21 +195,21 @@ static unsigned long zone_reclaimable_pages(struct zone *zone)
 {
        unsigned long nr;
 
-       nr = zone_page_state(zone, NR_ACTIVE_FILE) +
-            zone_page_state(zone, NR_INACTIVE_FILE) +
-            zone_page_state(zone, NR_ISOLATED_FILE);
+       nr = zone_page_state_snapshot(zone, NR_ACTIVE_FILE) +
+            zone_page_state_snapshot(zone, NR_INACTIVE_FILE) +
+            zone_page_state_snapshot(zone, NR_ISOLATED_FILE);
 
        if (get_nr_swap_pages() > 0)
-               nr += zone_page_state(zone, NR_ACTIVE_ANON) +
-                     zone_page_state(zone, NR_INACTIVE_ANON) +
-                     zone_page_state(zone, NR_ISOLATED_ANON);
+               nr += zone_page_state_snapshot(zone, NR_ACTIVE_ANON) +
+                     zone_page_state_snapshot(zone, NR_INACTIVE_ANON) +
+                     zone_page_state_snapshot(zone, NR_ISOLATED_ANON);
 
        return nr;
 }
 
 bool zone_reclaimable(struct zone *zone)
 {
-       return zone_page_state(zone, NR_PAGES_SCANNED) <
+       return zone_page_state_snapshot(zone, NR_PAGES_SCANNED) <
                zone_reclaimable_pages(zone) * 6;
 }