userfaultfd: call userfaultfd_unmap_prep only if __split_vma succeeds

A __split_vma is not a worthy event to report, and it's definitely not a
unmap so it would be incorrect to report unmap for the whole region to
the userfaultfd manager if a __split_vma fails.

So only call userfaultfd_unmap_prep after the __vma_splitting is over
and do_munmap cannot fail anymore.

Also add unlikely because it's better to optimize for the vast majority
of apps that aren't using userfaultfd in a non cooperative way.  Ideally
we should also find a way to eliminate the branch entirely if
CONFIG_USERFAULTFD=n, but it would complicate things so stick to
unlikely for now.

Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Andrea Arcangeli <[email protected]>
Cc: "Dr. David Alan Gilbert" <[email protected]>
Cc: Alexey Perevalov <[email protected]>
Cc: Maxime Coquelin <[email protected]>
Cc: Mike Kravetz <[email protected]>
Cc: Mike Rapoport <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/mm/mmap.c b/mm/mmap.c
index 9800e29763f40939ff557515faf2cfc14a66ac24..52f6c6b18f40dcc25d0d3d610b413300f3597470 100644 (file)
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -2639,13 +2639,6 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len,
        if (vma->vm_start >= end)
                return 0;
 
-       if (uf) {
-               int error = userfaultfd_unmap_prep(vma, start, end, uf);
-
-               if (error)
-                       return error;
-       }
-
        /*
         * If we need to split any vma, do it now to save pain later.
         *
@@ -2679,6 +2672,21 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len,
        }
        vma = prev ? prev->vm_next : mm->mmap;
 
+       if (unlikely(uf)) {
+               /*
+                * If userfaultfd_unmap_prep returns an error the vmas
+                * will remain splitted, but userland will get a
+                * highly unexpected error anyway. This is no
+                * different than the case where the first of the two
+                * __split_vma fails, but we don't undo the first
+                * split, despite we could. This is unlikely enough
+                * failure that it's not worth optimizing it for.
+                */
+               int error = userfaultfd_unmap_prep(vma, start, end, uf);
+               if (error)
+                       return error;
+       }
+
        /*
         * unlock any mlock()ed ranges before detaching vmas
         */