Mark Rutland 6d2779ecae locking/atomic: scripts: fix fallback ifdeffery ... Since commit: 9257959a6e ("locking/atomic: scripts: restructure fallback ifdeffery") The ordering fallbacks for atomic*_read_acquire() and atomic*_set_release() erroneously fall back to the implictly relaxed atomic*_read() and atomic*_set() variants respectively, without any additional barriers. This loses the ACQUIRE and RELEASE ordering semantics, which can result in a wide variety of problems, even on strongly-ordered architectures where the implementation of atomic*_read() and/or atomic*_set() allows the compiler to reorder those relative to other accesses. In practice this has been observed to break bit spinlocks on arm64, resulting in dentry cache corruption. The fallback logic was intended to allow ACQUIRE/RELEASE/RELAXED ops to be defined in terms of FULL ops, but where an op had RELAXED ordering by default, this unintentionally permitted the ACQUIRE/RELEASE ops to be defined in terms of the implicitly RELAXED default. This patch corrects the logic to avoid falling back to implicitly RELAXED ops, resulting in the same behaviour as prior to commit 9257959a6e. I've verified the resulting assembly on arm64 by generating outlined wrappers of the atomics. Prior to this patch the compiler generates sequences using relaxed load (LDR) and store (STR) instructions, e.g. | <outlined_atomic64_read_acquire>: | ldr x0, [x0] | ret | | <outlined_atomic64_set_release>: | str x1, [x0] | ret With this patch applied the compiler generates sequences using the intended load-acquire (LDAR) and store-release (STLR) instructions, e.g. | <outlined_atomic64_read_acquire>: | ldar x0, [x0] | ret | | <outlined_atomic64_set_release>: | stlr x1, [x0] | ret To make sure that there were no other victims of the ifdeffery rewrite, I generated outlined copies of all of the {atomic,atomic64,atomic_long} atomic operations before and after commit 9257959a6e. A diff of the generated assembly on arm64 shows that only the read_acquire() and set_release() operations were changed, and only lost their intended ordering: | [mark@lakrids:~/src/linux]% diff -u \ | <(aarch64-linux-gnu-objdump -d before-9257959a6e5b4fca.o) | <(aarch64-linux-gnu-objdump -d after-9257959a6e5b4fca.o) | --- /proc/self/fd/11 2023-09-19 16:51:51.114779415 +0100 | +++ /proc/self/fd/16 2023-09-19 16:51:51.114779415 +0100 | @@ -1,5 +1,5 @@ | | -before-9257959a6e5b4fca.o: file format elf64-littleaarch64 | +after-9257959a6e5b4fca.o: file format elf64-littleaarch64 | | | Disassembly of section .text: | @@ -9,7 +9,7 @@ | 4: d65f03c0 ret | | 0000000000000008 <outlined_atomic_read_acquire>: | - 8: 88dffc00 ldar w0, [x0] | + 8: b9400000 ldr w0, [x0] | c: d65f03c0 ret | | 0000000000000010 <outlined_atomic_set>: | @@ -17,7 +17,7 @@ | 14: d65f03c0 ret | | 0000000000000018 <outlined_atomic_set_release>: | - 18: 889ffc01 stlr w1, [x0] | + 18: b9000001 str w1, [x0] | 1c: d65f03c0 ret | | 0000000000000020 <outlined_atomic_add>: | @@ -1230,7 +1230,7 @@ | 1070: d65f03c0 ret | | 0000000000001074 <outlined_atomic64_read_acquire>: | - 1074: c8dffc00 ldar x0, [x0] | + 1074: f9400000 ldr x0, [x0] | 1078: d65f03c0 ret | | 000000000000107c <outlined_atomic64_set>: | @@ -1238,7 +1238,7 @@ | 1080: d65f03c0 ret | | 0000000000001084 <outlined_atomic64_set_release>: | - 1084: c89ffc01 stlr x1, [x0] | + 1084: f9000001 str x1, [x0] | 1088: d65f03c0 ret | | 000000000000108c <outlined_atomic64_add>: | @@ -2427,7 +2427,7 @@ | 207c: d65f03c0 ret | | 0000000000002080 <outlined_atomic_long_read_acquire>: | - 2080: c8dffc00 ldar x0, [x0] | + 2080: f9400000 ldr x0, [x0] | 2084: d65f03c0 ret | | 0000000000002088 <outlined_atomic_long_set>: | @@ -2435,7 +2435,7 @@ | 208c: d65f03c0 ret | | 0000000000002090 <outlined_atomic_long_set_release>: | - 2090: c89ffc01 stlr x1, [x0] | + 2090: f9000001 str x1, [x0] | 2094: d65f03c0 ret | | 0000000000002098 <outlined_atomic_long_add>: I've build tested this with a variety of configs for alpha, arm, arm64, csky, i386, m68k, microblaze, mips, nios2, openrisc, powerpc, riscv, s390, sh, sparc, x86_64, and xtensa, for which I've seen no issues. I was unable to build test for ia64 and parisc due to existing build breakage in v6.6-rc2. Fixes: 9257959a6e ("locking/atomic: scripts: restructure fallback ifdeffery") Reported-by: Ming Lei <ming.lei@redhat.com> Reported-by: Darrick J. Wong <djwong@kernel.org> Signed-off-by: Mark Rutland <mark.rutland@arm.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> Tested-by: Baokun Li <libaokun1@huawei.com> Link: https://lkml.kernel.org/r/20230919171430.2697727-1-mark.rutland@arm.com		2023-09-20 09:39:03 +02:00
..
atomic	locking/atomic: scripts: fix fallback ifdeffery	2023-09-20 09:39:03 +02:00
basic	kbuild: implement CONFIG_TRIM_UNUSED_KSYMS without recursion	2023-06-22 21:21:06 +09:00
clang-tools	gen_compile_commands: add assembly files to compilation database	2023-07-23 22:36:07 +09:00
coccinelle	Revert "debugfs, coccinelle: check for obsolete DEFINE_SIMPLE_ATTRIBUTE() usage"	2023-07-29 11:05:31 -04:00
dtc	dt: dt-check-compatible: Find struct of_device_id instances with compiler annotations	2023-08-17 13:07:39 -05:00
dummy-tools	kbuild: dummy-tools: make MPROFILE_KERNEL checks work on BE	2023-09-01 16:38:04 +09:00
gcc-plugins	gcc-plugins: Rename last_stmt() for GCC 14+	2023-08-10 23:10:09 -07:00
gdb	scripts/gdb/vmalloc: add vmallocinfo support	2023-08-21 13:46:23 -07:00
genksyms
kconfig	kconfig: fix possible buffer overflow	2023-09-06 02:00:02 +09:00
ksymoops
mod	parisc architecture fixes and enhancements for kernel v6.6-rc2:	2023-09-13 11:35:53 -07:00
package	kbuild: fix kernel-devel RPM package and linux-headers Deb package	2023-09-15 02:39:24 +09:00
selinux
tracing
.gitignore	rust: support running Rust documentation tests as KUnit ones	2023-07-19 09:32:53 -06:00
as-version.sh
asn1_compiler.c
bloat-o-meter	scripts/bloat-o-meter: count weak symbol sizes	2023-08-21 13:46:25 -07:00
bootgraph.pl
bpf_doc.py	bpf, docs: Fix invalid escape sequence warnings in bpf_doc.py	2023-08-31 13:56:31 +02:00
cc-can-link.sh
cc-version.sh
check_extable.sh
check-git
check-sysctl-docs	sysctl: Remove register_sysctl_table	2023-05-23 21:43:26 -07:00
checkdeclares.pl
checkincludes.pl
checkkconfigsymbols.py
checkpatch.pl	- An extensive rework of kexec and crash Kconfig from Eric DeVolder	2023-08-29 14:53:51 -07:00
checkstack.pl
checksyscalls.sh
checkversion.pl
cleanfile
cleanpatch
coccicheck
config
const_structs.checkpatch
decode_stacktrace.sh
decodecode
depmod.sh	kbuild: move depmod rule to scripts/Makefile.modinst	2023-08-29 22:38:23 +09:00
dev-needs.sh
diffconfig
documentation-file-ref-check
export_report.pl
extract_xc3028.pl
extract-ikconfig
extract-module-sig.pl
extract-sys-certs.pl
extract-vmlinux
faddr2line
file-size.sh
find-unused-docs.sh
gcc-x86_32-has-stack-protector.sh
gcc-x86_64-has-stack-protector.sh
gen-randstruct-seed.sh
generate_initcall_order.pl
generate_rust_analyzer.py	scripts: generate_rust_analyzer: provide cfgs for core and alloc	2023-08-20 22:54:32 +02:00
generate_rust_target.rs
get_abi.pl
get_dvb_firmware
get_feat.pl
get_maintainer.pl
gfp-translate	scripts: fix the gfp flags header path in gfp-translate	2023-06-19 13:19:32 -07:00
head-object-list.txt	powerpc/64: Rename entry_64.S to prom_entry_64.S	2023-06-15 14:04:19 +10:00
headerdep.pl
headers_install.sh	x86: Remove the arch_calc_vm_prot_bits() macro from the UAPI	2023-09-06 23:50:46 +02:00
insert-sys-cert.c
install.sh
jobserver-exec
kallsyms.c	scripts/kallsyms: Fix build failure by setting errno before calling getline()	2023-07-29 15:57:32 +09:00
Kbuild.include
Kconfig.include
kernel-doc	scripts: kernel-doc: fix macro handling in enums	2023-08-18 11:14:08 -06:00
ld-version.sh
leaking_addresses.pl
Lindent
link-vmlinux.sh	Kbuild updates for v6.4	2023-04-30 11:32:53 -07:00
Makefile	rust: support running Rust documentation tests as KUnit ones	2023-07-19 09:32:53 -06:00
Makefile.asm-generic
Makefile.build	kbuild: rust: avoid creating temporary files	2023-07-24 03:15:31 +09:00
Makefile.clang	Kbuild updates for v6.5	2023-07-01 09:24:31 -07:00
Makefile.clean	kbuild: make clean rule robust against too long argument error	2023-06-25 23:12:20 +09:00
Makefile.compiler	Kbuild updates for v6.5	2023-07-01 09:24:31 -07:00
Makefile.debug
Makefile.defconf
Makefile.dtbinst	kbuild: Support flat DTBs install	2023-06-21 07:51:08 -06:00
Makefile.extrawarn	extrawarn: move -Wrestrict into W=1 warnings	2023-08-20 14:16:59 +09:00
Makefile.gcc-plugins
Makefile.headersinst
Makefile.host	kbuild: rust: avoid creating temporary files	2023-07-24 03:15:31 +09:00
Makefile.kasan
Makefile.kcov
Makefile.kcsan
Makefile.kmsan
Makefile.lib	kbuild: Disallow DTB overlays to built from .dts named source files	2023-05-22 10:34:37 +09:00
Makefile.modfinal	btf, scripts: rust: drop is_rust_module.sh	2023-08-10 22:28:04 +02:00
Makefile.modinst	kbuild: avoid long argument lists in make modules_install	2023-09-15 02:39:24 +09:00
Makefile.modpost	linux/export.h: make <linux/export.h> independent of CONFIG_MODULES	2023-07-25 00:59:32 +09:00
Makefile.package	kbuild: deb-pkg: support DEB_BUILD_OPTIONS=parallel=N in debian/rules	2023-08-29 22:29:35 +09:00
Makefile.randstruct
Makefile.ubsan	ubsan: Tighten UBSAN_BOUNDS on GCC	2023-05-16 13:57:14 -07:00
Makefile.userprogs
Makefile.vmlinux	kbuild: Fix CFI failures with GCOV	2023-06-25 23:12:20 +09:00
Makefile.vmlinux_o
makelst
markup_oops.pl
min-tool-version.sh	rust: bindgen: upgrade to 0.65.1	2023-08-15 00:37:22 +02:00
misc-check
mkcompile_h
mksysmap	scripts/mksysmap: Ignore prefixed KCFI symbols	2023-06-27 08:35:43 +09:00
mkuboot.sh
module.lds.S
modules-check.sh
nsdeps
objdiff
objdump-func
orc_hash.sh	x86/unwind/orc: Add ELF section with ORC version identifier	2023-06-16 17:17:42 +02:00
pahole-flags.sh	bpf: Add --skip_encoding_btf_inconsistent_proto, --btf_gen_optimized to pahole flags for v1.25	2023-05-12 11:47:05 -07:00
pahole-version.sh
parse-maintainers.pl
patch-kernel
profile2linkerlist.pl
prune-kernel
recordmcount.c
recordmcount.h
recordmcount.pl
relocs_check.sh
remove-stale-files	kbuild: rpm-pkg: rename binkernel.spec to kernel.spec	2023-07-25 00:59:33 +09:00
rust_is_available_bindgen_libclang.h
rust_is_available_test.py	kbuild: rust_is_available: add test suite	2023-08-10 01:18:34 +02:00
rust_is_available.sh	kbuild: rust_is_available: check that output looks as expected	2023-08-10 01:18:34 +02:00
rustdoc_test_builder.rs	rust: support running Rust documentation tests as KUnit ones	2023-07-19 09:32:53 -06:00
rustdoc_test_gen.rs	rust: support running Rust documentation tests as KUnit ones	2023-07-19 09:32:53 -06:00
setlocalversion	scripts/setlocalversion: also consider annotated tags of the form vx.y.z-${file_localversion}	2023-08-08 01:08:54 +09:00
show_delta
sign-file.c
sorttable.c
sorttable.h
spdxcheck-test.sh
spdxcheck.py
spdxexclude
spelling.txt	scripts/spelling.txt: remove 'thead' as a typo	2023-07-27 13:07:04 -07:00
sphinx-pre-install
split-man.pl
stackdelta
stackusage
subarch.include
syscallhdr.sh
syscallnr.sh
syscalltbl.sh
tags.sh	Char/Misc and other driver subsystem updates for 6.5-rc1	2023-07-03 12:46:47 -07:00
test_fortify.sh
tools-support-relr.sh
unifdef.c
ver_linux
xen-hypercalls.sh
xz_wrap.sh