AArch64 atomics: Might they be harming your performance

We explore the performance of atomic CAS and LL-SC instructions on contemporary high-performance AArch64 CPUs, namely the A64FX, ThunderX2, and Gravitons 2 and 3. We observe that these instructions can lead to fundamentally different performance profiles. On the A64FX and ThunerX2, CAS instructions exhibit a quadratic increase in average time per successful CAS operation as the number of threads increases instead of the expected linear scaling. LL-SC can give performance improvements across all the micro-architectures we tested, showing potential speedups of more than 20x on A64FX.