From 00d37423ceaf815799a1d3e4fd5c958b8b3c66a6 Mon Sep 17 00:00:00 2001 From: Rohit Vaswani Date: Tue, 4 Nov 2014 22:21:35 -0800 Subject: [PATCH] arm64: spinlock: Drop the prfm from arch_spin_trylock As per ARM the prefetch for store (prfm pstl1strm) in the arch_spin_trylock routine can lead to some false positives, decreasing the lock performance. On the Cortex-A57 / Cortex-A53, if the memory type is Shareable, then any linefill started by a PST (prefetch for store)/PLDW instruction also causes the data to be invalidated in other cores, so that the line is ready for writing. This will also clear the exclusive marker associated with that cache line (clearing the exclusive monitors). So, in the scenario where we could have multiple cores trying to acquire exclusive access to a cacheline, the removal of prefetch would help with potentially increasing the chances of one of the cores making progress. Example: struct { spinlock_t lock; atomic_t count; } x; We have 2 cores trying to run the below code spin_lock(&x.lock); atomic_inc(&x.count); spin_unlock(&x.lock); lock and count are part of a struct so they fall into the same cacheline. The lock function uses the trylock mechanism as part of debug spinlock. 1. Core1 has acquired the spinlock and is performing the ldxr, add, strx loop in atomic_inc. 2. Core2 are trying to acquire the spinlock. Core1 | Core2 ldxr | | | | prfm pstl1strm add | | | | ldaxr stxr (fails) | | | | Now, the prfm always clears the exclusive marker for the core1 ldxr, so the stxr always fails. This prevents core1 from making progress and releasing the spinlock that core2 is waiting for. This could potentially go on forever and we end up breaking this pattern if the timing changes or if an interrupt triggers. This could happen with more cores trying to acquire the spinlock, cause more prefetches and make the problem worse. By removing the prfm, we allow the stxr @core1 to suceed and atomic_inc completes, allowing core1 to unlock the spinlock and let core2 proceed. Change-Id: I742ab9b6d98aded628e4dcf4a6ec68b8e2a4ec3e Signed-off-by: Rohit Vaswani Signed-off-by: Prasad Sodagudi --- arch/arm64/include/asm/spinlock.h | 1 - 1 file changed, 1 deletion(-) diff --git a/arch/arm64/include/asm/spinlock.h b/arch/arm64/include/asm/spinlock.h index cae331d553f8..55082cc2933b 100644 --- a/arch/arm64/include/asm/spinlock.h +++ b/arch/arm64/include/asm/spinlock.h @@ -127,7 +127,6 @@ static inline int arch_spin_trylock(arch_spinlock_t *lock) asm volatile(ARM64_LSE_ATOMIC_INSN( /* LL/SC */ - " prfm pstl1strm, %2\n" "1: ldaxr %w0, %2\n" " eor %w1, %w0, %w0, ror #16\n" " cbnz %w1, 2f\n"