ACPI: Be in TS_POLLING state during mwait based C-state entry

ACPI deep C-state entry had a long standing bug/missing feature, wherein we were sending
resched IPIs when an idle CPU is in mwait based deep C-state. Only mwait based C1 was using
the write to the monitored address to wake up mwait'ing CPU.

This patch changes the code to retain TS_POLLING bit if we are entering an mwait based
deep C-state.

The patch has been verified to reduce the number of resched IPIs in general and also
improves the performance/power on workloads with low system utilization (i.e., when mwait based
deep C-states are being used).

Fixes "netperf ~50% regression with 2.6.33-rc1, bisect to 1b9508f"
http://marc.info/?l=linux-kernel&m=126441481427331&w=4

Reported-by: Lin Ming <[email protected]>
Tested-by: Alex Shi <[email protected]>
Signed-off-by: Venkatesh Pallipadi <[email protected]>
Signed-off-by: Len Brown <[email protected]>

diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c
index e88e8ae04fdbe5a9ecd8fe44c8128e65ab7f96bf..cc978a8c00b797d6c5e9d4300a857a7fa612d6ea 100644 (file)
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -880,12 +880,14 @@ static int acpi_idle_enter_simple(struct cpuidle_device *dev,
                return(acpi_idle_enter_c1(dev, state));
 
        local_irq_disable();
-       current_thread_info()->status &= ~TS_POLLING;
-       /*
-        * TS_POLLING-cleared state must be visible before we test
-        * NEED_RESCHED:
-        */
-       smp_mb();
+       if (cx->entry_method != ACPI_CSTATE_FFH) {
+               current_thread_info()->status &= ~TS_POLLING;
+               /*
+                * TS_POLLING-cleared state must be visible before we test
+                * NEED_RESCHED:
+                */
+               smp_mb();
+       }
 
        if (unlikely(need_resched())) {
                current_thread_info()->status |= TS_POLLING;
@@ -965,12 +967,14 @@ static int acpi_idle_enter_bm(struct cpuidle_device *dev,
        }
 
        local_irq_disable();
-       current_thread_info()->status &= ~TS_POLLING;
-       /*
-        * TS_POLLING-cleared state must be visible before we test
-        * NEED_RESCHED:
-        */
-       smp_mb();
+       if (cx->entry_method != ACPI_CSTATE_FFH) {
+               current_thread_info()->status &= ~TS_POLLING;
+               /*
+                * TS_POLLING-cleared state must be visible before we test
+                * NEED_RESCHED:
+                */
+               smp_mb();
+       }
 
        if (unlikely(need_resched())) {
                current_thread_info()->status |= TS_POLLING;