arm64: spinlock: Drop the prfm from arch_spin_trylock

As per ARM the prefetch for store (prfm pstl1strm) in the arch_spin_trylock
routine can lead to some false positives, decreasing the lock performance.
On the Cortex-A57 / Cortex-A53, if the memory type is Shareable, then any
linefill started by a PST (prefetch for store)/PLDW instruction also causes
the data to be invalidated in other cores, so that the line is ready for
writing. This will also clear the exclusive marker associated with that
cache line (clearing the exclusive monitors).
So, in the scenario where we could have multiple cores trying to acquire
exclusive access to a cacheline, the removal of prefetch would help with
potentially increasing the chances of one of the cores making progress.

Example:
struct {
spinlock_t lock;
atomic_t count;
} x;

We have 2 cores trying to run the below code

spin_lock(&x.lock);
atomic_inc(&x.count);
spin_unlock(&x.lock);

lock and count are part of a struct so they fall into the same cacheline.
The lock function uses the trylock mechanism as part of debug spinlock.
1. Core1 has acquired the spinlock and is performing the ldxr, add, strx
loop in atomic_inc.
2. Core2 are trying to acquire the spinlock.

Core1		|    Core2
ldxr		|	|
|		|  prfm pstl1strm
add		|	|
|		|     ldaxr
stxr (fails)	|	|
|		|

Now, the prfm always clears the exclusive marker for the core1 ldxr,
so the stxr always fails. This prevents core1 from making progress and
releasing the spinlock that core2 is waiting for.
This could potentially go on forever and we end up breaking this pattern
if the timing changes or if an interrupt triggers.
This could happen with more cores trying to acquire the spinlock, cause
more prefetches and make the problem worse.
By removing the prfm, we allow the stxr @core1 to suceed and atomic_inc
completes, allowing core1 to unlock the spinlock and let core2 proceed.

Change-Id: I742ab9b6d98aded628e4dcf4a6ec68b8e2a4ec3e
Signed-off-by: Rohit Vaswani <rvaswani@codeaurora.org>
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
tirimbino
Rohit Vaswani 10 years ago committed by Prasad Sodagudi
parent a8313924c3
commit 00d37423ce
  1. 1
      arch/arm64/include/asm/spinlock.h

@ -127,7 +127,6 @@ static inline int arch_spin_trylock(arch_spinlock_t *lock)
asm volatile(ARM64_LSE_ATOMIC_INSN(
/* LL/SC */
" prfm pstl1strm, %2\n"
"1: ldaxr %w0, %2\n"
" eor %w1, %w0, %w0, ror #16\n"
" cbnz %w1, 2f\n"

Loading…
Cancel
Save