cpufreq: cpu-boost: Fix deadlock in wake_up of sync threads

If wake_up() is called on the current task on a CPU, the call will wait
until the current task is switched out before it wakes it up again and
returns.

The sync notifier for a CPU always runs on that CPU.

These two together can result in a deadlock if the sync notifier on CPU A
tries to wake up the sync thread of CPU A as it goes to sleep (is the
current task). A previous commit fixed this by adding a check to the sync
notifier to not wake up the sync thread of CPU A if it's the current task.

But this is still not sufficient to prevent deadlocks.

Sync thread of CPU A could be the current task on CPU B and sync thread of
CPU B could be the current task on CPU A.  At this point, if sync notifier
of CPU A and B try to wake up the sync threads of CPU A and B, it will
result in CPU A waiting for the current task in CPU B to get switched out
and CPU B waiting for the current task in CPU A to get switched out.  This
will result in a deadlock.

Prevent this scenario from happening by pinning the sync threads of each
CPU to run on that CPU. By doing this, we guarantee that sync notifiers
will only try to wake up sync threads running on that CPU. The fix added by
"cpufreq: cpu-boost: Resolve deadlock when waking up sync thread" ensures a
deadlock doesn't happen when a sync notifier tries to wake up a sync thread
running on that CPU.

Change-Id: I864e545529722a23886dd5a82f66089155d2d193
Signed-off-by: Saravana Kannan <skannan@codeaurora.org>

1 files changed, 23 insertions, 11 deletions

diff --git a/drivers/cpufreq/cpu-boost.c b/drivers/cpufreq/cpu-boost.c
index 445b1798aa4a..179cb179b019 100644
--- a/drivers/cpufreq/cpu-boost.c
+++ b/drivers/cpufreq/cpu-boost.c
@@ -63,6 +63,12 @@ static u64 last_input_time;
  * The CPUFREQ_ADJUST notifier is used to override the current policy min to
  * make sure policy min >= boost_min. The cpufreq framework then does the job
  * of enforcing the new policy.
+ *
+ * The sync kthread needs to run on the CPU in question to avoid deadlocks in
+ * the wake up code. Achieve this by binding the thread to the respective
+ * CPU. But a CPU going offline unbinds threads from that CPU. So, set it up
+ * again each time the CPU comes back up. We can use CPUFREQ_START to figure
+ * out a CPU is coming online instead of registering for hotplug notifiers.
  */
 static int boost_adjust_notify(struct notifier_block *nb, unsigned long val,
 				void *data)
@@ -74,22 +80,27 @@ static int boost_adjust_notify(struct notifier_block *nb, unsigned long val,
 	unsigned int ib_min = s->input_boost_min;
 	unsigned int min;
 
-	if (val != CPUFREQ_ADJUST)
-		return NOTIFY_OK;
+	switch (val) {
+	case CPUFREQ_ADJUST:
+		if (!b_min && !ib_min)
+			break;
 
-	if (!b_min && !ib_min)
-		return NOTIFY_OK;
+		min = max(b_min, ib_min);
 
-	min = max(b_min, ib_min);
+		pr_debug("CPU%u policy min before boost: %u kHz\n",
+			 cpu, policy->min);
+		pr_debug("CPU%u boost min: %u kHz\n", cpu, min);
 
-	pr_debug("CPU%u policy min before boost: %u kHz\n",
-		 cpu, policy->min);
-	pr_debug("CPU%u boost min: %u kHz\n", cpu, min);
+		cpufreq_verify_within_limits(policy, min, UINT_MAX);
 
-	cpufreq_verify_within_limits(policy, min, UINT_MAX);
+		pr_debug("CPU%u policy min after boost: %u kHz\n",
+			 cpu, policy->min);
+		break;
 
-	pr_debug("CPU%u policy min after boost: %u kHz\n",
-		 cpu, policy->min);
+	case CPUFREQ_START:
+		set_cpus_allowed(s->thread, *cpumask_of(cpu));
+		break;
+	}
 
 	return NOTIFY_OK;
 }
@@ -344,6 +355,7 @@ static int cpu_boost_init(void)
 		INIT_DELAYED_WORK(&s->input_boost_rem, do_input_boost_rem);
 		s->thread = kthread_run(boost_mig_sync_thread, (void *)cpu,
 					"boost_sync/%d", cpu);
+		set_cpus_allowed(s->thread, *cpumask_of(cpu));
 	}
 	atomic_notifier_chain_register(&migration_notifier_head,
 					&boost_migration_nb);