idr: make idr_remove_all() do removal -before- free_layer()

Fix a problem in the IDR system, where an idr_remove_all() hands a data
element to call_rcu() (via free_layer()) before making that data element
inaccessible to new readers.  This is very bad, and results in readers
still having a reference to this data element at the end of the grace
period.

Tests on large machines that concurrently map and unmap user-space memory
within the same multithreaded process result in crashes within about five
minutes.  Applying this patch increases the kernel's longevity to the
three-to-eight-hour range.

There appear to be other similar problems in idr_get_empty_slot() and
sub_remove(), but I fixed the easy one in idr_remove_all() first.  It is
therefore no surprise that failures still occur.

Located-by: Milton Miller II <[email protected]>
Tested-by: Milton Miller II <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
Cc: Manfred Spraul <[email protected]>
Cc: Ingo Molnar <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/lib/idr.c b/lib/idr.c
index c11c5765cdefd2b5115e4f37e94905ab7364adad..dab4bca86f5d1981027c0be5104a69ce7d05fbe9 100644 (file)
--- a/lib/idr.c
+++ b/lib/idr.c
@@ -449,6 +449,7 @@ void idr_remove_all(struct idr *idp)
 
        n = idp->layers * IDR_BITS;
        p = idp->top;
+       rcu_assign_pointer(idp->top, NULL);
        max = 1 << n;
 
        id = 0;
@@ -467,7 +468,6 @@ void idr_remove_all(struct idr *idp)
                        p = *--paa;
                }
        }
-       rcu_assign_pointer(idp->top, NULL);
        idp->layers = 0;
 }
 EXPORT_SYMBOL(idr_remove_all);