Memoryless nodes: Fix interleave behavior for memoryless nodes

MPOL_INTERLEAVE currently simply loops over all nodes.  Allocations on
memoryless nodes will be redirected to nodes with memory.  This results in an
imbalance because the neighboring nodes to memoryless nodes will get
significantly more interleave hits that the rest of the nodes on the system.

We can avoid this imbalance by clearing the nodes in the interleave node set
that have no memory.  If we use the node map of the memory nodes instead of
the online nodes then we have only the nodes we want.

Signed-off-by: Christoph Lameter <[email protected]>
Signed-off-by: Nishanth Aravamudan <[email protected]>
Tested-by: Lee Schermerhorn <[email protected]>
Acked-by: Bob Picco <[email protected]>
Cc: KAMEZAWA Hiroyuki <[email protected]>
Cc: Mel Gorman <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index ccbdb22147bb9ca16b6c4728e006c884a951a40e..5daf63bd97e7f36e6620641cc86fda3b5eeaab35 100644 (file)
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -183,7 +183,9 @@ static struct mempolicy *mpol_new(int mode, nodemask_t *nodes)
        switch (mode) {
        case MPOL_INTERLEAVE:
                policy->v.nodes = *nodes;
-               if (nodes_weight(*nodes) == 0) {
+               nodes_and(policy->v.nodes, policy->v.nodes,
+                                       node_states[N_HIGH_MEMORY]);
+               if (nodes_weight(policy->v.nodes) == 0) {
                        kmem_cache_free(policy_cache, policy);
                        return ERR_PTR(-EINVAL);
                }