x86, rwsem: Stay on fast path when count > 0 in __up_write()

When count > 0 there is no need to take the call_rwsem_wake path.  If
we did take that path, it would just return without doing anything due
to the active count not being zero.

Signed-off-by: Michel Lespinasse <[email protected]>
LKML-Reference: <201007202219[email protected]>
Acked-by: David Howells <[email protected]>
Cc: Mike Waychison <[email protected]>
Cc: Suleiman Souhlal <[email protected]>
Cc: Ying Han <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: H. Peter Anvin <[email protected]>

diff --git a/arch/x86/include/asm/rwsem.h b/arch/x86/include/asm/rwsem.h
index 606ede126972e568b992268da3df6f64b7e98480..5bf5e04e497fa83bf81f58d765c8e40d4d9f67ea 100644 (file)
--- a/arch/x86/include/asm/rwsem.h
+++ b/arch/x86/include/asm/rwsem.h
@@ -216,9 +216,8 @@ static inline void __up_write(struct rw_semaphore *sem)
        rwsem_count_t tmp;
        asm volatile("# beginning __up_write\n\t"
                     LOCK_PREFIX "  xadd      %1,(%2)\n\t"
-                    /* tries to transition
-                       0xffff0001 -> 0x00000000 */
-                    "  jz       1f\n"
+                    /* subtracts 0xffff0001, returns the old value */
+                    "  jns        1f\n\t"
                     "  call call_rwsem_wake\n"
                     "1:\n\t"
                     "# ending __up_write\n"