aio: add missing smp_rmb() in read_events_ring

We ran into a case on ppc64 running mariadb where io_getevents would
return zeroed out I/O events.  After adding instrumentation, it became
clear that there was some missing synchronization between reading the
tail pointer and the events themselves.  This small patch fixes the
problem in testing.

Thanks to Zach for helping to look into this, and suggesting the fix.

Signed-off-by: Jeff Moyer <[email protected]>
Signed-off-by: Benjamin LaHaise <[email protected]>
Cc: [email protected]

diff --git a/fs/aio.c b/fs/aio.c
index 97bc62cbe2dab816cd33b20b86e6baa73d2ae19f..5f2e9c6c328e89bd9f6bb276f79f6f8eb68a0d06 100644 (file)
--- a/fs/aio.c
+++ b/fs/aio.c
@@ -1111,6 +1111,12 @@ static long aio_read_events_ring(struct kioctx *ctx,
        tail = ring->tail;
        kunmap_atomic(ring);
 
+       /*
+        * Ensure that once we've read the current tail pointer, that
+        * we also see the events that were stored up to the tail.
+        */
+       smp_rmb();
+
        pr_debug("h%u t%u m%u\n", head, tail, ctx->nr_events);
 
        if (head == tail)