From 5e14d9edac8afcf86f4378e691f3771b00ed8d36 Mon Sep 17 00:00:00 2001 From: Thomas Walker Lynch Date: Fri, 17 Oct 2025 04:34:27 +0000 Subject: [PATCH] release directory structure simplification for simplified multiple architecture approach --- developer/tool/release | 26 +++--- document/build_environmnt.org | 158 +++++++++++++++++----------------- release/aarch64/.githolder | 0 release/armv7l/.githolder | 0 release/i686/.githolder | 0 release/ppc64le/.githolder | 0 release/riscv64/.githolder | 0 release/s390x/.githolder | 0 release/x86_64/.githolder | 0 9 files changed, 90 insertions(+), 94 deletions(-) delete mode 100644 release/aarch64/.githolder delete mode 100644 release/armv7l/.githolder delete mode 100644 release/i686/.githolder delete mode 100644 release/ppc64le/.githolder delete mode 100644 release/riscv64/.githolder delete mode 100644 release/s390x/.githolder delete mode 100644 release/x86_64/.githolder diff --git a/developer/tool/release b/developer/tool/release index 582c84c..afffc94 100755 --- a/developer/tool/release +++ b/developer/tool/release @@ -1,10 +1,12 @@ #!/usr/bin/env bash script_afp=$(realpath "${BASH_SOURCE[0]}") -# before running this, make library is built and is in the scratchpad directory - -# input guards +# This is a bespoke script for copying files into ../release. +# Files in ../release can be used by the tester role. +# On a release branch, files in the ../release constitute the public release. +# conventional preliminaries +# env_must_be="developer/tool/env" if [ "$ENV" != "$env_must_be" ]; then echo "$(script_fp):: error: must be run in the $env_must_be environment" @@ -13,21 +15,17 @@ script_afp=$(realpath "${BASH_SOURCE[0]}") cd "$REPO_HOME"/developer || exit 1 - if [ ! -d scratchpad ]; then - echo "$(script_fp):: no scratchpad directory" - exit 1 - fi - #set -e #set -x - release_dir=$(release_dir) - mkdir -p ${release_dir} +release_dir="$REPO_HOME/release" + +# examples + + # yes machine directory, but no release/kmod directory, on the skeleton + mkdir -p ${release_dir}/kmod && install -m 0600 scratchpad/*.ko "${release_dir}/kmod/" + install -m 0700 scratchpad/hello "${release_dir}/machine/" - install_file scratchpad/libN.a ${release_dir} "ug+r" || true - install_file cc/*.lib.c ${release_dir} "ug+r" || true - install_file cc/*.lib.c ${release_dir} "ug+r" || true -#set +x echo "$(script_fn) done." diff --git a/document/build_environmnt.org b/document/build_environmnt.org index 4fb50c0..81d96a1 100644 --- a/document/build_environmnt.org +++ b/document/build_environmnt.org @@ -2,42 +2,41 @@ #+OPTIONS: toc:nil * 0. Scope -A concise contract for building user-space artifacts (static libraries + CLI executables) and Linux kernel modules (out-of-tree, *.ko) in one repo without target collisions. The design keeps authored sources read-only, writes artifacts to scratchpad, and exposes a small, stable public interface via an orchestrator Makefile. +A concise contract for building user-space artifacts (static libraries + CLI executables) and Linux kernel modules (out-of-tree, *.ko) in one repo without target collisions. Authored sources remain read-only; all synthesis happens in ~scratchpad/~. A small, stable public interface is exposed via an orchestrator Makefile. * 1. Core Principles -1.1 Minimize complexity (YAGNI). Prefer one happy path; add branches only when mandated. -1.2 Variable-driven config: the project declares *what* to build; domain makefiles define *how*. -1.3 Automatic discovery: detect work by suffix/patterns—~*.lib.c~, ~*.cli.c~, ~*.mod.c~—never hard-code basenames. -1.4 Authored sources are read-only to scripts; all synthesis happens in ~scratchpad/~. -1.5 Build domains are independent and reusable; orchestration is done by separate ~make -f~ calls. +1.1 Minimize complexity (YAGNI): one happy path; branch only when mandated. +1.2 Variable-driven config: the project declares *what*; domain makefiles define *how*. +1.3 Automatic discovery by suffix: ~*.lib.c~, ~*.cli.c~, ~*.mod.c~; never hard-code basenames. +1.4 Authored trees are read-only to scripts; writes go to ~scratchpad/~. +1.5 Build domains are independent and reusable; orchestration is via separate ~make -f~ calls. * 2. Repo Layout, Authority, and Skeleton -2.1 =REPO_HOME= is exported by the environment and is the **authoritative project root**. Never recompute or override it in Makefiles. -2.2 Invoke the build from =$REPO_HOME/developer=. -2.3 Domain makefiles are vendored under: - =$(REPO_HOME)/tool_shared/third_party/RT-project-share/release/make/= -2.4 Directory semantics: - 2.4.1 =developer/cc/= **authored** C sources (read-only to scripts). - 2.4.2 =scratchpad/= **SCRATCHPAD** (intermediates, depfiles, kbuild outputs). Git-ignored. - 2.4.3 =machine/= released executables (version-controlled). Prefer arch suffix in names. -2.5 **Harmony skeleton invariants (pre-created directories):** The Harmony skeleton lays down the standard tree *before* any build runs (e.g., =developer/=, =developer/cc/=, =scratchpad/=, =machine/=, =release/make/=, =tool_shared/...=). Domain makefiles and recipes may assume these exist; =mkdir -p= in build rules is unnecessary unless a domain explicitly introduces new subtrees. +2.1 =REPO_HOME= is exported by the environment and is the authoritative project root. +2.2 Build is invoked from =$REPO_HOME/developer=. +2.3 Domain makefiles are vendored under: + =$(REPO_HOME)/tool_shared/third_party/RT-project-share/release/make/= +2.4 Directory semantics: + 2.4.1 =developer/cc/= **authored** C sources (read-only to scripts). + 2.4.2 =developer/scratchpad/= **SCRATCHPAD** (intermediates, depfiles, kbuild outputs, CLI binaries). + 2.4.3 =release/= publish area (e.g., =release/machine= for executables, =release/module= for *.ko). +2.5 Harmony skeleton invariants: the skeleton pre-creates the standard tree; recipes may assume directories exist. * 3. Build Domains & Files -3.1 Orchestrator (project-local): =developer/tool/makefile= - - Public targets: ~all~, ~lib_cli~, ~kmod~, ~clean~ (~usage~ optional; ~all~ is first). +3.1 Orchestrator (project-local): =developer/tool/makefile= + - Public targets: ~all~, ~lib_cli~, ~kmod~, ~clean~ (~usage~ optional; ~all~ is first). - Includes =environment_RT_1.mk= and invokes domain makefiles via ~make -f~. -3.2 Domain: user-space build → =target_lib_cli.mk= (shared; under RT-project-share) - - Builds static lib from ~*.lib.c~ and CLI executables from ~*.cli.c~. - - Public targets: ~library~, ~cli~, ~clean~. ~cli~ depends on ~library~. -3.3 Domain: kernel modules → =targets_kmod.mk= (shared; under RT-project-share) - - Builds out-of-tree modules (.ko) from ~*.mod.c~ using Kbuild. +3.2 Domain: user-space → =target_lib_cli.mk= + - Builds static lib from ~*.lib.c~ and CLI executables from ~*.cli.c~. + - Public targets: ~library~, ~cli~, ~clean~. ~cli~ depends on ~library~. +3.3 Domain: kernel modules → =target_kmod.mk= + - Builds out-of-tree modules (~.ko~) from ~*.mod.c~ using Kbuild. - Public targets: ~kmod~, ~clean~. * 4. Public-Target Orchestration (strict) -4.1 The **orchestrator** owns public targets and their order. Domain makefiles expose callable, explicit targets but are not public entrypoints. -4.2 Example (shape): +4.1 The orchestrator owns public targets and their order. Domains expose callable targets but are not public entrypoints. +4.2 Shape: #+BEGIN_SRC makefile -# developer/tool/makefile — public surface RT_INCOMMON := $(REPO_HOME)/tool_shared/third_party/RT-project-share/release include $(RT_INCOMMON)/make/environment_RT_1.mk @@ -48,49 +47,44 @@ lib_cli: @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk cli kmod: - @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk kmod + @$(MAKE) -f $(RT_INCOMMON)/make/target_kmod.mk kmod clean: @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk clean - @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk clean + @$(MAKE) -f $(RT_INCOMMON)/make/target_kmod.mk clean #+END_SRC -4.3 Result: no target collisions (each domain runs in its own ~make -f~). ~all~ remains the single happy path. +4.3 Result: no target collisions. ~all~ remains the single happy path. * 5. Discovery & Suffix Contract -5.1 Suffixes define intent; scripts don’t embed filenames: - 5.1.1 ~*.lib.c~ → compiled into a static archive. - 5.1.2 ~*.cli.c~ → linked into CLI executables. - 5.1.3 ~*.mod.c~ → compiled by Kbuild into ~.ko~. -5.2 Discovery happens in the domain makefiles; the orchestrator can optionally export a filtered list (e.g., subset kmods), but domains must work without it. +5.1 Suffixes define intent: + 5.1.1 ~*.lib.c~ → compiled into a static archive. + 5.1.2 ~*.cli.c~ → linked into CLI executables. + 5.1.3 ~*.mod.c~ → compiled via Kbuild into ~.ko~. +5.2 Discovery is implemented in the domain makefiles; the orchestrator can optionally export a filtered list but domains must work without it. -* 6. Authored vs Write Areas (contract) -6.1 **Authored** (=developer/cc/= and other documented trees): tools do not write, delete, or generate within. -6.2 **SCRATCHPAD** (=scratchpad/=): all intermediates (.o, .d, temp .c for Kbuild) and module outputs (.ko). ~.gitignore~ contains this. -6.3 **machine/**: released executables (checked in). If shared across hosts, embed arch in filename (e.g., ~prog-$(shell uname -m)~). “Machine-like” *temporary* outputs go to ~scratchpad/~, not ~machine/~. +* 6. Authored vs Write Areas +6.1 **Authored** (=developer/cc/= and other documented trees): tools never write into authored trees. +6.2 **SCRATCHPAD** (=developer/scratchpad/=): all intermediates, depfiles, CLI binaries, and kbuild outputs (~.ko~). +6.3 **release/**: explicit promotion target. Nothing writes here during build; only an explicit release step copies from ~developer/scratchpad/~. * 7. Dependencies -7.1 User-space domain enables per-object depfiles: ~CFLAGS += -MMD -MP~; domain includes ~$(OBJECT:%.o=%.d)~. -7.2 Depfiles live beside objects in ~scratchpad/~ and are removed by ~clean~. -7.3 No separate “make dependency” step is required; changes to headers or structure are picked up automatically. -7.4 Kbuild manages its own deps internally when invoked for kmods. +7.1 User-space: ~CFLAGS += -MMD -MP~ with per-object depfiles included. +7.2 Depfiles live beside objects in ~developer/scratchpad/~ and are removed by ~clean~. +7.3 No separate “make dependency” step. +7.4 Kbuild manages kernel deps internally. * 8. Kbuild Integration (kmod) -8.1 Kbuild interface directory: =/lib/modules/$(uname -r)/build=. -8.2 All Kbuild artifacts are directed to project SCRATCHPAD via ~O=$(REPO_HOME)/$(SCRATCHPAD)~; sources remain authored. -8.3 Pattern (shape): +8.1 Kbuild interface: =/lib/modules/$(uname -r)/build= (kernel headers/devel tree). +8.2 All Kbuild artifacts are directed to the project SCRATCHPAD: + ~KBUILD_OUTPUT_DIR := $(CURDIR)/$(SCRATCHPAD)~. +8.3 Shape: #+BEGIN_SRC makefile -# make/targets_kmod.mk ifndef REPO_HOME $(error REPO_HOME is not set; build must be done in a project environment) endif - -# If not provided by caller, discover from authored sources (recommended default) -ifndef KBUILD_BASE_List KBUILD_BASE_List := $(basename $(notdir $(wildcard $(REPO_HOME)/developer/cc/*.mod.c))) -endif - -KDIR := /lib/modules/$(shell uname -r)/build -KBUILD_OUTPUT_DIR := $(REPO_HOME)/$(SCRATCHPAD) +KMOD_BUILD_DPath := /lib/modules/$(shell uname -r)/build +KBUILD_OUTPUT_DIR := $(CURDIR)/$(SCRATCHPAD) KERNEL_OBJS_M := $(addsuffix .o,$(KBUILD_BASE_List)) KMOD_TARGETS := $(addsuffix .ko,$(addprefix $(KBUILD_OUTPUT_DIR)/,$(KBUILD_BASE_List))) @@ -102,45 +96,49 @@ $(KBUILD_OUTPUT_DIR)/%.c: $(REPO_HOME)/developer/cc/%.mod.c cp $< $@ $(KBUILD_OUTPUT_DIR)/%.ko: $(KBUILD_OUTPUT_DIR)/%.c - $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m=$*.o + $(MAKE) -C $(KMOD_BUILD_DPath) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m=$*.o clean: - $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m="$(KERNEL_OBJS_M)" clean + $(MAKE) -C $(KMOD_BUILD_DPath) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m="$(KERNEL_OBJS_M)" clean #+END_SRC * 9. User-Space Domain (lib + CLI) -9.1 Domain name: =target_lib_cli.mk=. -9.2 Pattern (shape): +9.1 Domain name: =target_lib_cli.mk=. +9.2 Shape: #+BEGIN_SRC makefile -# target_lib_cli.mk (shape) # Scan *.lib.c / *.cli.c; derive OBJECT_LIB / OBJECT_EXEC under $(SCRATCHPAD) # -include $(OBJECT_LIB:.o=.d) $(OBJECT_EXEC:.o=.d) -# library: builds $(LIBFILE) from OBJECT_LIB (empty archive is OK) +# library: builds $(LIBFILE) from OBJECT_LIB (empty archive OK) # cli: depends on $(LIBFILE); links each $(EXECDIR)/% from %.cli.o + $(LIBFILE) +# With EXECDIR := $(SCRATCHPAD), CLIs land in developer/scratchpad/ #+END_SRC * 10. Variable Conventions -10.1 Only variables intended for export to sub-processes are ALL_CAPS (environment-style). -10.2 =REPO_HOME= is authoritative (exported by env); do not assign to it in Makefiles. -10.3 Use =SCRATCHPAD= (not TMPDIR) as the canonical name for the scratch directory in new code. -10.4 Prefer explicit paths to archives (e.g., link with ~$(LIBFILE)~) unless you intentionally maintain ~-L/-l~ naming. +10.1 Export-worthy variables in ALL_CAPS. +10.2 Do not assign to =REPO_HOME= in Makefiles. +10.3 Canonical scratch directory variable: =SCRATCHPAD=. +10.4 Prefer explicit paths (e.g., link with ~$(LIBFILE)~) unless you require ~-L/-l~ naming. * 11. Usage & Defaults -11.1 Domain makefiles put ~usage~ as the first target (direct calls print help and exit non-zero). -11.2 Orchestrator keeps ~all~ as the first target (so ~make~ “just works”). -11.3 Skip-if-empty logic is *not required*: if no sources match, domain targets naturally no-op (empty prereq lists). -11.4 Optional diagnostic: a ~check-pwd~ target in the orchestrator can warn if ~$(CURDIR) ≠ $(REPO_HOME)/developer~. - -* 12. Release Conventions (machine/) -12.1 Released executables go to ~machine/~ and are checked in. -12.2 When a shared ~machine/~ is used across hosts, embed an arch tag in the filename at *build time* (not only at publish time). -12.3 Temporary binaries or architecture-variant scratch builds go to ~scratchpad/~. - -* 13. Summary (Do / Don’t) -13.1 Do keep domains independent; orchestrator stitches them via ~make -f~. -13.2 Do derive *what to build* from suffixes; don’t hard-code filenames. -13.3 Do confine writes to ~scratchpad/~ (and ~machine/~ for releases); never write in authored trees. -13.4 Do use ~-MMD -MP~ and include per-object depfiles; don’t require humans to rebuild deps. -13.5 Don’t recompute ~REPO_HOME~; don’t pass ~PWD~ unnecessarily; environment inheritance suffices. -13.6 Use clear names: public targets = ~lib_cli~, ~kmod~, ~clean~; domain files = ~target_lib_cli.mk~, ~targets_kmod.mk~. -13.7 Remember: Harmony **skeleton** pre-creates the standard tree; recipes may assume directories exist. +11.1 Domain makefiles may expose ~usage~ as the first target for direct calls. +11.2 Orchestrator keeps ~all~ as the first target (so ~make~ “just works”). +11.3 Skip-if-empty logic is not required; empty prereq lists no-op naturally. +11.4 Optional diagnostic: ~check-pwd~ can warn if ~$(CURDIR) ≠ $(REPO_HOME)/developer~. + +* 12. Release Conventions +12.1 Build outputs (CLIs and ~.ko~) remain in ~developer/scratchpad/~. +12.2 Promotion is explicit: copy **as-is** into ~release/machine~ (executables) or ~release/module~ (kernel modules). +12.3 No arch suffixing at build time; no per-arch subdirs; no symlinks. + +* 13. Alternatives considered (and why not chosen) +13.1 **Script adds arch suffix at release time:** Needs to “guess” artifact identity and build arch; risks drift if build host ≠ release host. +13.2 **Make target adds arch suffix at build time:** Hard-codes basename rules or requires extra metadata; increases cognitive load for developers. +13.3 **Per-arch release trees + symlinks:** Adds directory sprawl and symlink churn; complicates diffs and reviews. +13.4 **Chosen:** local build/use + explicit promotion. It is the least complex, avoids mismatch hazards, and keeps authored trees clean. Artifacts are reproducible from sources; release is a conscious, auditable copy. + +* 14. Summary (Do / Don’t) +14.1 Do keep domains independent; orchestrator stitches them via ~make -f~. +14.2 Do derive *what to build* from suffixes; don’t hard-code filenames. +14.3 Do confine writes to ~developer/scratchpad/~; never write in authored trees. +14.4 Don’t auto-publish; use explicit promotion to ~release/~. +14.5 Harmony skeleton pre-creates the standard tree; recipes may assume directories exist. diff --git a/release/aarch64/.githolder b/release/aarch64/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/armv7l/.githolder b/release/armv7l/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/i686/.githolder b/release/i686/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/ppc64le/.githolder b/release/ppc64le/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/riscv64/.githolder b/release/riscv64/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/s390x/.githolder b/release/s390x/.githolder deleted file mode 100644 index e69de29..0000000 diff --git a/release/x86_64/.githolder b/release/x86_64/.githolder deleted file mode 100644 index e69de29..0000000 -- 2.20.1