Skip to content

selftests/bpf: restructure the Makefile as a layered build - #13046

Closed
kernel-patches-daemon-bpf[bot] wants to merge 3 commits into
bpf-next_basefrom
series/1133647=>bpf-next
Closed

selftests/bpf: restructure the Makefile as a layered build#13046
kernel-patches-daemon-bpf[bot] wants to merge 3 commits into
bpf-next_basefrom
series/1133647=>bpf-next

Conversation

@kernel-patches-daemon-bpf

@kernel-patches-daemon-bpf kernel-patches-daemon-bpf Bot commented Jul 30, 2026

Copy link
Copy Markdown

Pull request for series with
subject: selftests/bpf: restructure the Makefile as a layered build
version: 4
url: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 30, 2026

Copy link
Copy Markdown
Author

Upstream branch: 2659f94
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 30, 2026

Copy link
Copy Markdown
Author

Upstream branch: 863f3dd
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot force-pushed the series/1133647=>bpf-next branch from 6cfc214 to 89c7fd9 Compare July 30, 2026 22:46
@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 30, 2026

Copy link
Copy Markdown
Author

Upstream branch: f0e80de
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot force-pushed the series/1133647=>bpf-next branch from 89c7fd9 to efee5e3 Compare July 30, 2026 23:46
@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 31, 2026

Copy link
Copy Markdown
Author

Upstream branch: f0e80de
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot force-pushed the series/1133647=>bpf-next branch from efee5e3 to 82e0845 Compare July 31, 2026 19:53
@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 31, 2026

Copy link
Copy Markdown
Author

Upstream branch: f0e80de
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot force-pushed the series/1133647=>bpf-next branch from 82e0845 to da1e88e Compare July 31, 2026 21:40
@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 31, 2026

Copy link
Copy Markdown
Author

Upstream branch: f0e80de
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot force-pushed the series/1133647=>bpf-next branch from da1e88e to 2e199c8 Compare July 31, 2026 21:56
@kernel-patches-review-bot

kernel-patches-review-bot Bot commented Jul 31, 2026

Copy link
Copy Markdown 
commit 2c16f90bdbfdce775dede80c5b300114774bd7b4
Author: Mykola Lysenko <nickolay.lysenko@gmail.com>

selftests/bpf: move shared build definitions into Makefile.buildvars

This patch refactors the BPF selftests Makefile by moving toolchain, path,
flag, probe and knob definitions into a separate Makefile.buildvars file.

> diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile
> index 2ed953b84409..1843679fc0e2 100644
> --- a/tools/testing/selftests/bpf/Makefile
> +++ b/tools/testing/selftests/bpf/Makefile

[ ... ]

> @@ -213,21 +149,8 @@ ifeq ($(feature-llvm),1)
>  endif
>  endif
>
> -SCRATCH_DIR := $(OUTPUT)/tools
> -BUILD_DIR := $(SCRATCH_DIR)/build
> -INCLUDE_DIR := $(SCRATCH_DIR)/include
> -BPFOBJ := $(BUILD_DIR)/libbpf/libbpf.a
> -ifneq ($(CROSS_COMPILE),)
> -HOST_BUILD_DIR		:= $(BUILD_DIR)/host
> -HOST_SCRATCH_DIR	:= $(OUTPUT)/host-tools
> -HOST_INCLUDE_DIR	:= $(HOST_SCRATCH_DIR)/include
> -else
> -HOST_BUILD_DIR		:= $(BUILD_DIR)
> -HOST_SCRATCH_DIR	:= $(SCRATCH_DIR)
> -HOST_INCLUDE_DIR	:= $(INCLUDE_DIR)
> -endif
> -HOST_BPFOBJ := $(HOST_BUILD_DIR)/libbpf/libbpf.a
> -RESOLVE_BTFIDS := $(HOST_BUILD_DIR)/resolve_btfids/resolve_btfids
> +include Makefile.buildvars
> +
>  VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux)				\
>  		     $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux)	\
>  		     ../../../../vmlinux				\

The srctree fallback block moved into Makefile.buildvars, but that file is
included after ../../../build/Makefile.feature has already been parsed. Does
this break the feature-llvm probe for standalone builds?

In the pre-image, the ordering was:

  include ../../../scripts/Makefile.include
  <srctree fallback at old line 53-58>
  include ../lib.mk
  $(eval include ../../../build/Makefile.feature)  # old line 183/186

After the patch:

  include ../../../scripts/Makefile.include
  include ../lib.mk
  $(eval include ../../../build/Makefile.feature)  # new line 119/122
  include Makefile.buildvars                        # srctree defined here

Makefile.feature line 2 does:

  feature_dir := $(srctree)/tools/build/feature

This is an immediate assignment, so it captures srctree when
Makefile.feature is parsed. In a standalone 'cd tools/testing/selftests/bpf
&& make' build, srctree is not in the environment and nothing in the earlier
include chain defines it - none of tools/build/Build.include,
tools/scripts/Makefile.arch, tools/scripts/Makefile.include or
tools/testing/selftests/lib.mk contain a srctree assignment.

Consequently feature_dir becomes '/tools/build/feature' instead of
'<tree>/tools/build/feature', and the probe at Makefile.feature:11 runs:

  $(MAKE) ... -C /tools/build/feature ...

The '-C /tools/build/feature' fails because the directory does not exist,
stderr is discarded, and the probe silently yields feature-llvm=0.

This flips the conditional at Makefile:127-150 off, so LLVM_CFLAGS never
gains -DHAVE_LLVM_SUPPORT and LLVM_LDLIBS/LLVM_LDFLAGS stay empty.
jit_disasm_helpers.c then compiles its '#else' stub whose get_jited_program_text()
returns -EOPNOTSUPP, and test_loader.c:1442-1449 turns every __jited expectation
into test__skip() with "tests are built w/o LLVM development libs". The six
progs/ files that use __jited stop running, and test_progs loses
-DHAVE_LLVM_SUPPORT and the LLVM libraries it previously linked.

This contradicts the commit message's "Build artifacts are byte-identical
before and after this change" claim for the standard build invocation
documented in Documentation/bpf/bpf_devel_QA.rst:459.

The srctree fallback belongs with BPF_GCC in the "needed before lib.mk"
group that the commit message already identifies, for the same reason -
it is consumed before Makefile.buildvars is included. The same applies to
PKG_CONFIG, which is used in Makefile.feature at lines 178/183 but is now
only defined in Makefile.buildvars.

> diff --git a/tools/testing/selftests/bpf/Makefile.buildvars b/tools/testing/selftests/bpf/Makefile.buildvars
> new file mode 100644
> index 000000000000..ef12018fa254
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/Makefile.buildvars
> @@ -0,0 +1,165 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Shared toolchain, path and flag definitions for the BPF selftests
> +# build, included by both Makefile and Makefile.runner. Definitions
> +# follow the order they had in the Makefile before the split.
> +
> +CXX ?= $(CROSS_COMPILE)g++
> +OBJCOPY ?= $(CROSS_COMPILE)objcopy
> +
> +CURDIR := $(abspath .)
> +
> +TOOLSDIR := $(abspath ../../..)
> +LIBDIR := $(TOOLSDIR)/lib
> +BPFDIR := $(LIBDIR)/bpf
> +TOOLSINCDIR := $(TOOLSDIR)/include
> +TOOLSARCHINCDIR := $(TOOLSDIR)/arch/$(SRCARCH)/include
> +BPFTOOLDIR := $(TOOLSDIR)/bpf/bpftool
> +APIDIR := $(TOOLSINCDIR)/uapi
> +ifneq ($(O),)
> +GENDIR := $(O)/include/generated
> +else
> +GENDIR := $(abspath ../../../../include/generated)
> +endif
> +GENHDR := $(GENDIR)/autoconf.h
> +PKG_CONFIG ?= $(CROSS_COMPILE)pkg-config

Same ordering issue: PKG_CONFIG was defined before ../lib.mk in the
pre-image, but now it is only defined here in Makefile.buildvars, which is
reached after Makefile.feature has already been parsed.

Makefile.feature uses PKG_CONFIG at parse time:

  ifneq ($(PKG_CONFIG),)
    $(foreach package,$(FEATURE_PKG_CONFIG),$(call feature_pkg_config,$(package)))

with immediate assignments at lines 178-179:

  FEATURE_CHECK_CFLAGS-$(1)  := $(shell $(PKG_CONFIG) --cflags $(1) 2>/dev/null)
  FEATURE_CHECK_LDFLAGS-$(1) := $(shell $(PKG_CONFIG) --libs $(1) 2>/dev/null)

Nothing parsed before Makefile.buildvars defines PKG_CONFIG, so the ifneq
guard becomes false and the pkg-config-derived FEATURE_CHECK_*FLAGS block is
skipped.

> +ifeq ($(srctree),)
> +srctree := $(patsubst %/,%,$(dir $(CURDIR)))
> +srctree := $(patsubst %/,%,$(dir $(srctree)))
> +srctree := $(patsubst %/,%,$(dir $(srctree)))
> +srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif

[ ... ]

> +ifneq ($(CROSS_COMPILE),)
> +CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
> +endif
> +
> +# Immediate on purpose: expanding get_sys_includes forks several shells.
> +CLANG_SYS_INCLUDES := $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))

The commit message states "Definitions keep the order they had in the
Makefile; the only exception is called out in a comment (CFLAGS is prepended
rather than appended, as this file is included after lib.mk)."

However, CLANG_SYS_INCLUDES also changed from recursive assignment ('=') to
immediate assignment (':='):

  pre  (Makefile:463): CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
  post (buildvars:139): CLANG_SYS_INCLUDES := $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))

The inline comment explains the fork-count motivation, but this is a second
evaluation-time change beyond the CFLAGS exception documented in the
changelog. It happens to be benign at this commit since CLANG and
CLANG_TARGET_ARCH are already final, but any later target-specific or
conditional override of CLANG would silently stop affecting CLANG_SYS_INCLUDES.

Would it be worth mentioning this change in the commit message too?

AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

In-Reply-To-Subject: selftests/bpf: move shared build definitions into Makefile.buildvars
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/30582411360

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Jul 31, 2026

Copy link
Copy Markdown
Author

Forwarding comment 5147922708 via email
In-Reply-To: 20260724042600.175440-3-nickolay.lysenko@gmail.com
Patch: https://patchwork.kernel.org/project/netdevbpf/patch/20260724042600.175440-3-nickolay.lysenko@gmail.com/

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 1, 2026

Copy link
Copy Markdown
Author

Upstream branch: 682b1c1
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 1, 2026

Copy link
Copy Markdown
Author

Upstream branch: 28e911d
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 2, 2026

Copy link
Copy Markdown
Author

Upstream branch: 8f876c7
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 3, 2026

Copy link
Copy Markdown
Author

Upstream branch: 0b10b94
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 3, 2026

Copy link
Copy Markdown
Author

Upstream branch: 6078126
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 3, 2026

Copy link
Copy Markdown
Author

Upstream branch: 457d4ec
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

prozak added 2 commits August 4, 2026 02:47
@prozak Kernel Patches Daemon
selftests/bpf: extract BPF skeleton generation into a helper script
bb5d0d1 
The four skeleton generation recipes (regular, light, signed light and
linked) are near-identical five-step pipelines - link via "bpftool gen
object" three times, compare the second and third results as a
regression test for bpftool's determinism, generate the skeleton (and
possibly subskeleton), remove intermediates - duplicated with small
variations inside DEFINE_TEST_RUNNER_RULES, where every line pays the
double-expansion escaping tax.

Move the pipeline into gen_bpf_skel.sh; the differences between the
four variants become the --lskel, --sign and --subskel flags. Signing
takes the key and certificate from $PRIVATE_KEY and $VERIFICATION_CERT
in the environment, like the bpftool binary comes from $BPFTOOL. The
distinct linked/llinked intermediate infixes are kept but derived
inside the script: tests listed in LSKELS_EXTRA generate both a
.skel.h and a .lskel.h from the same .bpf.o, and distinct intermediate
names are what keeps parallel builds from racing. Build-log messages
stay in the recipes with the usual $(call msg,...) helpers; behavior
is unchanged.

Generated skeletons are byte-identical to the previous recipes'
output.

The script is a prerequisite of every skeleton rule, so editing it
regenerates the headers.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
@prozak Kernel Patches Daemon
selftests/bpf: move shared build definitions into Makefile.buildvars
117cc30 
Move the toolchain, path, flag, probe and knob definitions that are not
rules into Makefile.buildvars: output-tree layout, libbpf/bpftool binary
locations, BPF_CFLAGS/COMMON_CFLAGS assembly, libelf/libpcap probes,
endianness and clang feature detection, the vmlinux BTF search list,
signing key paths, permissive-mode knob and the libarena skeleton names.

This is preparation for building each test runner instance in its own
sub-make: the definitions become includable by more than one makefile.
No rules or recipes are changed. One deliberate nuance: the include
sits after ../lib.mk, so CFLAGS is now assembled by *prepending*
COMMON_CFLAGS (keeping the include-search order identical to before,
where the definitions preceded lib.mk's additions).

Build artifacts are byte-identical before and after this change.

Definitions keep the order they had in the Makefile; the only
exception is called out in a comment (CFLAGS is prepended rather than
appended, as this file is included after lib.mk). Knobs the runner
never reads (SKIP_*, submake_extras, VMLINUX_BTF, TEST_KMOD_TARGETS)
stay in the main Makefile.

BPF_GCC, TEST_KMODS and the VMLINUX_BTF block stay in the Makefile at
their current positions - the first two are needed before lib.mk is
included, where Makefile.buildvars cannot yet be, and none of them is
shared with the coming runner sub-makes except through the environment
(a later patch exports what the runners need).

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
@kernel-patches-daemon-bpf

kernel-patches-daemon-bpf Bot commented Aug 4, 2026

Copy link
Copy Markdown
Author

Upstream branch: 7f333f8
series: https://patchwork.kernel.org/project/netdevbpf/list/?series=1133647
version: 4

@prozak Kernel Patches Daemon
selftests/bpf: build each test runner instance in its own sub-make
18702fe 
Replace the DEFINE_TEST_RUNNER/DEFINE_TEST_RUNNER_RULES double-expansion
machinery with Makefile.runner: one sub-make invocation per test runner
instance (test_progs, test_progs-no_alu32, test_progs-cpuv4,
test_progs-bpf_gcc, test_maps), each in its own single-flavor namespace
written in plain make - no $$-escaping, no per-flavor eval guards, no
accumulating vpath directives.

The main Makefile keeps everything that exists once - tool sub-builds,
vmlinux.h, signing keys, tests.h generation, shared helper objects,
standalone binaries and the kselftest lib.mk contract - and delegates to
the runner through explicit per-instance rules. Light-skeleton demand
lists are derived from the tests' own '#include "*.lskel.h"' lines; the
bench object list is derived from $(wildcard benchs/bench_*.c). The ~20
inline $(if $(PERMISSIVE),...) fragments collapse into one skip_on_fail
helper in Makefile.buildvars.

This also fixes a latent parallel-build race (objects including libbpf
internal headers now order against the bpftool sub-build that installs
them) and makes the signing key generation rule properly grouped (the
two-target form ran genkey twice concurrently under -j).

Validation is as for the whole series: BPF objects, skeleton headers,
userspace objects and 21/22 binaries byte-identical with the previous
Makefile (bench differs only in object link order); emit_tests and the
installed tree identical; BPF CI green on x86_64 gcc/llvm, aarch64 and
s390x including GCC-BPF, ASAN and veristat jobs; all 76 benchmarks
produce identical outcomes.

The shared helper objects the main Makefile pre-builds depend on a
superset of the prerequisites the runner's uniform object rule uses,
so the unflavored test_progs and test_maps sub-makes always see them
as up to date and never race to recompile them in the shared output
directory.

In permissive mode the runner link rule keeps the current Makefile's
semantics: test objects existing at parse time are normal
prerequisites (editing a test source relinks the runner), the full
set stays order-only to drive the build attempts, and the recipe
links the wildcard survivors.

BPF_GCC and TEST_KMODS are exported next to the LLVM probe results:
the runner sub-makes read both from the environment.

The BPF compiler differences between flavors (clang vs bpf-gcc,
-mcpu level, extra defines and system includes) are plain parameters
on the sub-make invocation, so the runner has a single BPF
compilation rule with no conditionals.

Four stale target-specific lines are dropped on the way:
'$(OUTPUT)/test_l4lb_noinline.o: BPF_CFLAGS += -fno-inline' (and the
test_xdp_noinline sibling) attach to names nothing builds - the BPF
objects have been %.bpf.o for years - and the
$(OUTPUT)/flow_dissector_load.o / $(OUTPUT)/cgroup_getset_retval_hooks.o
header dependencies likewise name objects no rule produces.

Suggested-by: Eduard Zingerman <eddyz87@gmail.com>
Assisted-by: Claude:claude-fable-5 shellcheck
Signed-off-by: Mykola Lysenko <nickolay.lysenko@gmail.com>
@kernel-patches-daemon-bpf
kernel-patches-daemon-bpf Bot deleted the series/1133647=>bpf-next branch August 6, 2026 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bpf-next under-review V4-ci-pass V4

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@prozak