Willem de Bruijn 9ef5d4748d bpf: Adjust free target to avoid global starvation of LRU map ... [ Upstream commit d4adf1c9ee ] BPF_MAP_TYPE_LRU_HASH can recycle most recent elements well before the map is full, due to percpu reservations and force shrink before neighbor stealing. Once a CPU is unable to borrow from the global map, it will once steal one elem from a neighbor and after that each time flush this one element to the global list and immediately recycle it. Batch value LOCAL_FREE_TARGET (128) will exhaust a 10K element map with 79 CPUs. CPU 79 will observe this behavior even while its neighbors hold 78 * 127 + 1 * 15 == 9921 free elements (99%). CPUs need not be active concurrently. The issue can appear with affinity migration, e.g., irqbalance. Each CPU can reserve and then hold onto its 128 elements indefinitely. Avoid global list exhaustion by limiting aggregate percpu caches to half of map size, by adjusting LOCAL_FREE_TARGET based on cpu count. This change has no effect on sufficiently large tables. Similar to LOCAL_NR_SCANS and lru->nr_scans, introduce a map variable lru->free_target. The extra field fits in a hole in struct bpf_lru. The cacheline is already warm where read in the hot path. The field is only accessed with the lru lock held. Tested-by: Anton Protopopov <a.s.protopopov@gmail.com> Signed-off-by: Willem de Bruijn <willemb@google.com> Acked-by: Stanislav Fomichev <sdf@fomichev.me> Link: https://lore.kernel.org/r/20250618215803.3587312-1-willemdebruijn.kernel@gmail.com Signed-off-by: Alexei Starovoitov <ast@kernel.org> Signed-off-by: Sasha Levin <sashal@kernel.org>		2025-07-17 18:37:22 +02:00
..
libbpf	docs/bpf: Add missing BPF program types to docs	2024-09-12 10:56:41 -07:00
standardization
bpf_design_QA.rst
bpf_devel_QA.rst	bpf: Add namespace to BPF internal symbols	2025-05-02 07:59:04 +02:00
bpf_iterators.rst
bpf_licensing.rst
bpf_prog_run.rst
btf.rst
clang-notes.rst
classic_vs_extended.rst
cpumasks.rst
drgn.rst
faq.rst
fs_kfuncs.rst
graph_ds_impl.rst
helpers.rst
index.rst
kfuncs.rst
linux-notes.rst
llvm_reloc.rst
map_array.rst
map_bloom_filter.rst
map_cgroup_storage.rst
map_cgrp_storage.rst
map_cpumap.rst
map_devmap.rst
map_hash.rst	bpf: Adjust free target to avoid global starvation of LRU map	2025-07-17 18:37:22 +02:00
map_lpm_trie.rst
map_lru_hash_update.dot	bpf: Adjust free target to avoid global starvation of LRU map	2025-07-17 18:37:22 +02:00
map_of_maps.rst
map_queue_stack.rst
map_sk_storage.rst
map_sockmap.rst
map_xskmap.rst
maps.rst
other.rst
prog_cgroup_sockopt.rst
prog_cgroup_sysctl.rst
prog_flow_dissector.rst
prog_lsm.rst
prog_sk_lookup.rst
programs.rst
redirect.rst
ringbuf.rst
s390.rst
syscall_api.rst
test_debug.rst
verifier.rst