Linus Torvalds e987af4546 percpu: changes for v6.6 ... percpu * A couple cleanups by Baoquan He and Bibo Mao. The only behavior change is to start printing messages if we're under the warn limit for failed atomic allocations. percpu_counter * Shakeel introduced percpu counters into mm_struct which caused percpu allocations be on the hot path [1]. Originally I spent some time trying to improve the percpu allocator, but instead preferred what Mateusz Guzik proposed grouping at the allocation site, percpu_counter_init_many(). This allows a single percpu allocation to be shared by the counters. I like this approach because it creates a shared lifetime by the allocations. Additionally, I believe many inits have higher level synchronization requirements, like percpu_counter does against HOTPLUG_CPU. Therefore we can group these optimizations together. [1] https://lore.kernel.org/linux-mm/20221024052841.3291983-1-shakeelb@google.com/ -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEE3hZPHJdcVwe+yTTtiDc0yuoFPR0FAmTv2IUACgkQiDc0yuoF PR0+gg//U430Y9jRSKQtbh3dEPaAeWGcTfSTnVHbQGfBj3A4ePJyWl/Tgzri31AC rzr8SRs0yX8b82TbECWsV67i/GrntLJyz4yQ52S/RRqVwnQqSn/wicEdCY00lJBt Tye8zApOnYBouaYqIOxm/M7ofvKzJ3gWOVeF/zBwM6hwvNaXXtY5r86fSDxoEbhY HOFnCDmg5Spf0U50j1G7nV5KfAb7BNA3/HFyzfzH+w+OWi4IGbThsfrg1qvjyFot KlEK/kF8Af2xj2A2se4XFsLc2D/Tj+29juYVQqIPBJzVPrZ2uerKSszK5Zcr+Use kMiG7tRWKE+2vkOM1RQ5Y5NCVEBhlXlienz1gf/C7247SEGs6OIyqvyDAgPTRx6p oR2/vx9hMtaSMf4aHWd+fYS5gNZ05iMvOIbRZnI1wZkQglQVkJvXhzuLaJ+dIGSP ypv6XOepik7vDjZ3p3xJXd0TAn4NSkn3jWRetrymdtMFanF99qw1VqjmkLecSil0 Gr0UhRL1oiMde6niVJrOpdOGLwt/M4N99Y5rksw6NCnktRJ99coFGj7LglZGMsu+ YkOyjD8MVJXTkBtBNGeqHTKe6nyVkHFq9ad5EmWjPkefP5JziH8i18k7JlF1dLA5 c8peq3ES659D5f0mU2jilD9PsCsBfSn6Of4ruMZa2Zr1XDD8snI= =vcA1 -----END PGP SIGNATURE----- Merge tag 'percpu-for-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu Pull percpu updates from Dennis Zhou: "One bigger change to percpu_counter's api allowing for init and destroy of multiple counters via percpu_counter_init_many() and percpu_counter_destroy_many(). This is used to help begin remediating a performance regression with percpu rss stats. Additionally, it seems larger core count machines are feeling the burden of the single threaded allocation of percpu. Mateusz is thinking about it and I will spend some time on it too. percpu: - A couple cleanups by Baoquan He and Bibo Mao. The only behavior change is to start printing messages if we're under the warn limit for failed atomic allocations. percpu_counter: - Shakeel introduced percpu counters into mm_struct which caused percpu allocations be on the hot path [1]. Originally I spent some time trying to improve the percpu allocator, but instead preferred what Mateusz Guzik proposed grouping at the allocation site, percpu_counter_init_many(). This allows a single percpu allocation to be shared by the counters. I like this approach because it creates a shared lifetime by the allocations. Additionally, I believe many inits have higher level synchronization requirements, like percpu_counter does against HOTPLUG_CPU. Therefore we can group these optimizations together" Link: https://lore.kernel.org/linux-mm/20221024052841.3291983-1-shakeelb@google.com/ [1] * tag 'percpu-for-6.6' of git://git.kernel.org/pub/scm/linux/kernel/git/dennis/percpu: kernel/fork: group allocation/free of per-cpu counters for mm struct pcpcntr: add group allocation/free mm/percpu.c: print error message too if atomic alloc failed mm/percpu.c: optimize the code in pcpu_setup_first_chunk() a little bit mm/percpu.c: remove redundant check mm/percpu: Remove some local variables in pcpu_populate_pte		2023-09-01 15:44:45 -07:00
..
842
crypto
dim
fonts
kunit	kunit: fix struct kunit_attr header	2023-08-21 08:07:56 -06:00
livepatch
lz4
lzo
math
pldmfw
raid6
reed_solomon
test_fortify
vdso
xz
zlib_deflate
zlib_dfltcc
zlib_inflate
zstd
.gitignore
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c
assoc_array.c
atomic64_test.c
atomic64.c
audit.c
base64.c
bcd.c
bch.c
bitfield_kunit.c
bitmap.c
bitrev.c
bootconfig-data.S
bootconfig.c
bsearch.c
btree.c
bucket_locks.c
bug.c
build_OID_registry
buildid.c
bust_spinlocks.c
check_signature.c
checksum_kunit.c	kunit: Fix checksum tests on big endian CPUs	2023-08-25 10:14:34 +01:00
checksum.c
clz_ctz.c	lib/clz_ctz.c: Fix __clzdi2() and __ctzdi2() for 32-bit kernels	2023-08-25 13:22:10 -07:00
clz_tab.c
cmdline_kunit.c
cmdline.c
cmpdi2.c
compat_audit.c
cpu_rmap.c
cpumask_kunit.c
cpumask.c
crc4.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
crc32test.c
crc64-rocksoft.c
crc64.c
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c
ctype.c
debug_info.c
debug_locks.c
debugobjects.c
dec_and_lock.c
decompress_bunzip2.c
decompress_inflate.c
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
decompress_unzstd.c
decompress.c
devmem_is_allowed.c
devres.c
dhry_1.c
dhry_2.c
dhry_run.c
dhry.h
digsig.c
dump_stack.c
dynamic_debug.c
dynamic_queue_limits.c
earlycpio.c
errname.c
error-inject.c
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c
find_bit.c
flex_proportions.c
fortify_kunit.c
gen_crc32table.c
gen_crc64table.c
genalloc.c	Devicetree include cleanups for v6.6:	2023-08-30 17:04:28 -07:00
generic-radix-tree.c
glob.c
globtest.c
group_cpus.c
hashtable_test.c
hexdump.c
hweight.c
idr.c
inflate.c
interval_tree_test.c
interval_tree.c
iomap_copy.c
iomap.c
iommu-helper.c
iov_iter.c	sound updates for 6.6-rc1	2023-08-30 13:45:05 -07:00
irq_poll.c
irq_regs.c
is_signed_type_kunit.c
is_single_threaded.c
kasprintf.c
Kconfig
Kconfig.debug	RISC-V Patches for the 6.6 Merge Window, Part 1	2023-09-01 08:09:48 -07:00
Kconfig.kasan
Kconfig.kcsan
Kconfig.kfence
Kconfig.kgdb
Kconfig.kmsan
Kconfig.ubsan
kfifo.c
klist.c
kobject_uevent.c
kobject.c	kobject: Remove redundant checks for whether ktype is NULL	2023-08-19 19:37:53 +02:00
kstrtox.c	kstrtox: consistently use _tolower()	2023-08-21 13:46:25 -07:00
kstrtox.h
libcrc32c.c
linear_ranges.c
list_debug.c
list_sort.c
list-test.c
llist.c
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c
logic_iomem.c
logic_pio.c	minmax: add in_range() macro	2023-08-24 16:20:18 -07:00
lru_cache.c
lshrdi3.c
Makefile	This update includes the following changes:	2023-08-29 11:23:29 -07:00
maple_tree.c	maple_tree: clean up mas_wr_append()	2023-08-24 16:20:32 -07:00
memcat_p.c
memcpy_kunit.c
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c
nmi_backtrace.c
notifier-error-inject.c
notifier-error-inject.h
objagg.c
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c
overflow_kunit.c
packing.c
parman.c
parser.c
pci_iomap.c
percpu_counter.c	pcpcntr: add group allocation/free	2023-08-25 08:06:53 -07:00
percpu_test.c
percpu-refcount.c
plist.c
pm-notifier-error-inject.c
polynomial.c
radix-tree.c	radix tree: remove unused variable	2023-08-21 13:07:22 -07:00
radix-tree.h
random32.c
ratelimit.c
rbtree_test.c
rbtree.c
rcuref.c
ref_tracker.c
refcount.c
rhashtable.c
sbitmap.c
scatterlist.c
seq_buf.c
sg_pool.c
sg_split.c
siphash_kunit.c
siphash.c
slub_kunit.c
smp_processor_id.c
sort.c
stackdepot.c
stackinit_kunit.c
stmp_device.c
strcat_kunit.c
string_helpers.c
string.c
strncpy_from_user.c
strnlen_user.c
strscpy_kunit.c
syscall.c
test_bitmap.c
test_bitops.c
test_bits.c
test_blackhole_dev.c
test_bpf.c
test_debug_virtual.c
test_dynamic_debug.c
test_firmware.c
test_fprobe.c
test_fpu.c
test_free_pages.c
test_hash.c
test_hexdump.c
test_hmm_uapi.h
test_hmm.c
test_ida.c
test_kmod.c
test_kprobes.c
test_linear_ranges.c
test_list_sort.c
test_lockup.c
test_maple_tree.c
test_memcat_p.c
test_meminit.c
test_min_heap.c
test_module.c
test_objagg.c
test_parman.c
test_printf.c	lib/vsprintf: declare no_hash_pointers in sprintf.h	2023-08-21 13:46:24 -07:00
test_ref_tracker.c
test_rhashtable.c
test_scanf.c
test_sort.c
test_static_key_base.c
test_static_keys.c
test_string.c
test_sysctl.c
test_ubsan.c
test_user_copy.c
test_uuid.c
test_vmalloc.c
test_xarray.c
test-kstrtox.c
test-string_helpers.c
textsearch.c
timerqueue.c
trace_readwrite.c
ts_bm.c
ts_fsm.c
ts_kmp.c
ubsan.c
ubsan.h
ucmpdi2.c
ucs2_string.c
usercopy.c
uuid.c
vsprintf.c	lib/vsprintf: split out sprintf() and friends	2023-08-21 13:46:24 -07:00
win_minmax.c
xarray.c
xxhash.c