x86/asm: Enable fast 32-bit put_user_64() for copy_to_user()

For fixed sized copies, copy_to_user() will utilize
__put_user_size() fastpaths. However, it is missing the
translation for 64-bit copies on x86/32.

Testing on a Pinetrail Atom, the 64 bit put_user() fastpath
is substantially faster than the generic copy_to_user()
fallback.

Signed-off-by: Chris Wilson <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>

diff --git a/arch/x86/include/asm/uaccess_32.h b/arch/x86/include/asm/uaccess_32.h
index 3c03a5de64d30c01c1408953bf75c4970f54bcb1..0ed5504c606081642912ae8ca31d3dc438f45200 100644 (file)
--- a/arch/x86/include/asm/uaccess_32.h
+++ b/arch/x86/include/asm/uaccess_32.h
@@ -59,6 +59,10 @@ __copy_to_user_inatomic(void __user *to, const void *from, unsigned long n)
                        __put_user_size(*(u32 *)from, (u32 __user *)to,
                                        4, ret, 4);
                        return ret;
+               case 8:
+                       __put_user_size(*(u64 *)from, (u64 __user *)to,
+                                       8, ret, 8);
+                       return ret;
                }
        }
        return __copy_to_user_ll(to, from, n);