mm/apply_to_range: call pte function with lazy updates

Make the pte-level function in apply_to_range be called in lazy mmu mode,
so that any pagetable modifications can be batched.

Signed-off-by: Jeremy Fitzhardinge <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Nick Piggin <[email protected]>
Cc: Venkatesh Pallipadi <[email protected]>
Cc: Hugh Dickins <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/mm/memory.c b/mm/memory.c
index 99e8d5c7b3126a455d664942d0f90621ce6c8825..b5af358b8b22a1a0c9856d2c8f0da6c134acd74b 100644 (file)
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -1645,6 +1645,8 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
 
        BUG_ON(pmd_huge(*pmd));
 
+       arch_enter_lazy_mmu_mode();
+
        token = pmd_pgtable(*pmd);
 
        do {
@@ -1653,6 +1655,8 @@ static int apply_to_pte_range(struct mm_struct *mm, pmd_t *pmd,
                        break;
        } while (pte++, addr += PAGE_SIZE, addr != end);
 
+       arch_leave_lazy_mmu_mode();
+
        if (mm != &init_mm)
                pte_unmap_unlock(pte-1, ptl);
        return err;