From: Thomas Walker Lynch Date: Thu, 16 Oct 2025 14:54:53 +0000 (+0000) Subject: revised build envornment doc by Triskel X-Git-Url: https://git.reasoningtechnology.com/style/static/git-favicon.png?a=commitdiff_plain;h=d010374bda4e4d79ba9ec46126233093db7675de;p=Harmony.git revised build envornment doc by Triskel --- diff --git a/document/build_environmnt.org b/document/build_environmnt.org new file mode 100644 index 0000000..4fb50c0 --- /dev/null +++ b/document/build_environmnt.org @@ -0,0 +1,146 @@ +#+TITLE: RT Harmony Build Environment (Orchestrator + Build Domains: lib_cli, kmod) +#+OPTIONS: toc:nil + +* 0. Scope +A concise contract for building user-space artifacts (static libraries + CLI executables) and Linux kernel modules (out-of-tree, *.ko) in one repo without target collisions. The design keeps authored sources read-only, writes artifacts to scratchpad, and exposes a small, stable public interface via an orchestrator Makefile. + +* 1. Core Principles +1.1 Minimize complexity (YAGNI). Prefer one happy path; add branches only when mandated. +1.2 Variable-driven config: the project declares *what* to build; domain makefiles define *how*. +1.3 Automatic discovery: detect work by suffix/patterns—~*.lib.c~, ~*.cli.c~, ~*.mod.c~—never hard-code basenames. +1.4 Authored sources are read-only to scripts; all synthesis happens in ~scratchpad/~. +1.5 Build domains are independent and reusable; orchestration is done by separate ~make -f~ calls. + +* 2. Repo Layout, Authority, and Skeleton +2.1 =REPO_HOME= is exported by the environment and is the **authoritative project root**. Never recompute or override it in Makefiles. +2.2 Invoke the build from =$REPO_HOME/developer=. +2.3 Domain makefiles are vendored under: + =$(REPO_HOME)/tool_shared/third_party/RT-project-share/release/make/= +2.4 Directory semantics: + 2.4.1 =developer/cc/= **authored** C sources (read-only to scripts). + 2.4.2 =scratchpad/= **SCRATCHPAD** (intermediates, depfiles, kbuild outputs). Git-ignored. + 2.4.3 =machine/= released executables (version-controlled). Prefer arch suffix in names. +2.5 **Harmony skeleton invariants (pre-created directories):** The Harmony skeleton lays down the standard tree *before* any build runs (e.g., =developer/=, =developer/cc/=, =scratchpad/=, =machine/=, =release/make/=, =tool_shared/...=). Domain makefiles and recipes may assume these exist; =mkdir -p= in build rules is unnecessary unless a domain explicitly introduces new subtrees. + +* 3. Build Domains & Files +3.1 Orchestrator (project-local): =developer/tool/makefile= + - Public targets: ~all~, ~lib_cli~, ~kmod~, ~clean~ (~usage~ optional; ~all~ is first). + - Includes =environment_RT_1.mk= and invokes domain makefiles via ~make -f~. +3.2 Domain: user-space build → =target_lib_cli.mk= (shared; under RT-project-share) + - Builds static lib from ~*.lib.c~ and CLI executables from ~*.cli.c~. + - Public targets: ~library~, ~cli~, ~clean~. ~cli~ depends on ~library~. +3.3 Domain: kernel modules → =targets_kmod.mk= (shared; under RT-project-share) + - Builds out-of-tree modules (.ko) from ~*.mod.c~ using Kbuild. + - Public targets: ~kmod~, ~clean~. + +* 4. Public-Target Orchestration (strict) +4.1 The **orchestrator** owns public targets and their order. Domain makefiles expose callable, explicit targets but are not public entrypoints. +4.2 Example (shape): +#+BEGIN_SRC makefile +# developer/tool/makefile — public surface +RT_INCOMMON := $(REPO_HOME)/tool_shared/third_party/RT-project-share/release +include $(RT_INCOMMON)/make/environment_RT_1.mk + +.PHONY: all lib_cli kmod clean +all: lib_cli kmod + +lib_cli: + @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk cli + +kmod: + @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk kmod + +clean: + @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk clean + @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk clean +#+END_SRC +4.3 Result: no target collisions (each domain runs in its own ~make -f~). ~all~ remains the single happy path. + +* 5. Discovery & Suffix Contract +5.1 Suffixes define intent; scripts don’t embed filenames: + 5.1.1 ~*.lib.c~ → compiled into a static archive. + 5.1.2 ~*.cli.c~ → linked into CLI executables. + 5.1.3 ~*.mod.c~ → compiled by Kbuild into ~.ko~. +5.2 Discovery happens in the domain makefiles; the orchestrator can optionally export a filtered list (e.g., subset kmods), but domains must work without it. + +* 6. Authored vs Write Areas (contract) +6.1 **Authored** (=developer/cc/= and other documented trees): tools do not write, delete, or generate within. +6.2 **SCRATCHPAD** (=scratchpad/=): all intermediates (.o, .d, temp .c for Kbuild) and module outputs (.ko). ~.gitignore~ contains this. +6.3 **machine/**: released executables (checked in). If shared across hosts, embed arch in filename (e.g., ~prog-$(shell uname -m)~). “Machine-like” *temporary* outputs go to ~scratchpad/~, not ~machine/~. + +* 7. Dependencies +7.1 User-space domain enables per-object depfiles: ~CFLAGS += -MMD -MP~; domain includes ~$(OBJECT:%.o=%.d)~. +7.2 Depfiles live beside objects in ~scratchpad/~ and are removed by ~clean~. +7.3 No separate “make dependency” step is required; changes to headers or structure are picked up automatically. +7.4 Kbuild manages its own deps internally when invoked for kmods. + +* 8. Kbuild Integration (kmod) +8.1 Kbuild interface directory: =/lib/modules/$(uname -r)/build=. +8.2 All Kbuild artifacts are directed to project SCRATCHPAD via ~O=$(REPO_HOME)/$(SCRATCHPAD)~; sources remain authored. +8.3 Pattern (shape): +#+BEGIN_SRC makefile +# make/targets_kmod.mk +ifndef REPO_HOME + $(error REPO_HOME is not set; build must be done in a project environment) +endif + +# If not provided by caller, discover from authored sources (recommended default) +ifndef KBUILD_BASE_List +KBUILD_BASE_List := $(basename $(notdir $(wildcard $(REPO_HOME)/developer/cc/*.mod.c))) +endif + +KDIR := /lib/modules/$(shell uname -r)/build +KBUILD_OUTPUT_DIR := $(REPO_HOME)/$(SCRATCHPAD) + +KERNEL_OBJS_M := $(addsuffix .o,$(KBUILD_BASE_List)) +KMOD_TARGETS := $(addsuffix .ko,$(addprefix $(KBUILD_OUTPUT_DIR)/,$(KBUILD_BASE_List))) + +.PHONY: kmod clean +kmod: $(KMOD_TARGETS) + +$(KBUILD_OUTPUT_DIR)/%.c: $(REPO_HOME)/developer/cc/%.mod.c + cp $< $@ + +$(KBUILD_OUTPUT_DIR)/%.ko: $(KBUILD_OUTPUT_DIR)/%.c + $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m=$*.o + +clean: + $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m="$(KERNEL_OBJS_M)" clean +#+END_SRC + +* 9. User-Space Domain (lib + CLI) +9.1 Domain name: =target_lib_cli.mk=. +9.2 Pattern (shape): +#+BEGIN_SRC makefile +# target_lib_cli.mk (shape) +# Scan *.lib.c / *.cli.c; derive OBJECT_LIB / OBJECT_EXEC under $(SCRATCHPAD) +# -include $(OBJECT_LIB:.o=.d) $(OBJECT_EXEC:.o=.d) +# library: builds $(LIBFILE) from OBJECT_LIB (empty archive is OK) +# cli: depends on $(LIBFILE); links each $(EXECDIR)/% from %.cli.o + $(LIBFILE) +#+END_SRC + +* 10. Variable Conventions +10.1 Only variables intended for export to sub-processes are ALL_CAPS (environment-style). +10.2 =REPO_HOME= is authoritative (exported by env); do not assign to it in Makefiles. +10.3 Use =SCRATCHPAD= (not TMPDIR) as the canonical name for the scratch directory in new code. +10.4 Prefer explicit paths to archives (e.g., link with ~$(LIBFILE)~) unless you intentionally maintain ~-L/-l~ naming. + +* 11. Usage & Defaults +11.1 Domain makefiles put ~usage~ as the first target (direct calls print help and exit non-zero). +11.2 Orchestrator keeps ~all~ as the first target (so ~make~ “just works”). +11.3 Skip-if-empty logic is *not required*: if no sources match, domain targets naturally no-op (empty prereq lists). +11.4 Optional diagnostic: a ~check-pwd~ target in the orchestrator can warn if ~$(CURDIR) ≠ $(REPO_HOME)/developer~. + +* 12. Release Conventions (machine/) +12.1 Released executables go to ~machine/~ and are checked in. +12.2 When a shared ~machine/~ is used across hosts, embed an arch tag in the filename at *build time* (not only at publish time). +12.3 Temporary binaries or architecture-variant scratch builds go to ~scratchpad/~. + +* 13. Summary (Do / Don’t) +13.1 Do keep domains independent; orchestrator stitches them via ~make -f~. +13.2 Do derive *what to build* from suffixes; don’t hard-code filenames. +13.3 Do confine writes to ~scratchpad/~ (and ~machine/~ for releases); never write in authored trees. +13.4 Do use ~-MMD -MP~ and include per-object depfiles; don’t require humans to rebuild deps. +13.5 Don’t recompute ~REPO_HOME~; don’t pass ~PWD~ unnecessarily; environment inheritance suffices. +13.6 Use clear names: public targets = ~lib_cli~, ~kmod~, ~clean~; domain files = ~target_lib_cli.mk~, ~targets_kmod.mk~. +13.7 Remember: Harmony **skeleton** pre-creates the standard tree; recipes may assume directories exist. diff --git a/document/developer_buils_environment.org b/document/developer_buils_environment.org deleted file mode 100644 index a882618..0000000 --- a/document/developer_buils_environment.org +++ /dev/null @@ -1,146 +0,0 @@ -#+TITLE: RT Harmony Build Environment (Orchestrator + Roles: lib_cli, kmod) -#+OPTIONS: toc:nil - -* 0. Scope -A concise contract for building user-space artifacts (static libraries + CLI executables) and Linux kernel modules (out-of-tree, *.ko) in one repo without target collisions. The design keeps authored sources read-only, writes artifacts to scratchpad, and exposes a small, stable public interface via an orchestrator Makefile. - -* 1. Core Principles -1.1 Minimize complexity (YAGNI). Prefer one happy path; add branches only when mandated. -1.2 Variable-driven config: the project declares *what* to build; role files define *how*. -1.3 Automatic discovery: detect work by suffix/patterns—~*.lib.c~, ~*.cli.c~, ~*.mod.c~—never hard-code basenames. -1.4 Authored sources are read-only to scripts; all synthesis happens in ~scratchpad/~. -1.5 Roles are independent and reusable; orchestration is done by separate ~make -f~ calls. - -* 2. Repo Layout & Authority -2.1 =REPO_HOME= is exported by the environment and is the **authoritative project root**. Never recompute or override it in Makefiles. -2.2 Invoke the build from =$REPO_HOME/developer=. -2.3 Shared includes live under: - =$(REPO_HOME)/tool_shared/third_party/RT-project-share/release/make/= - (project-vendored RT-project-share). -2.4 Directory semantics: - - =developer/cc/= **authored** C sources (read-only to scripts). - - =scratchpad/= **SCRATCHPAD** (intermediates, depfiles, kbuild outputs). Git-ignored. - - =machine/= released executables (version-controlled). Prefer arch suffix in names. - -* 3. Roles & Files -3.1 Orchestrator (project-local): =developer/tool/makefile= - - Public targets: ~all~, ~lib_cli~, ~kmod~, ~clean~ (~usage~ optional; ~all~ is first). - - Includes =environment_RT_1.mk= and invokes role files via ~make -f~. -3.2 Role: user-space build → =target_lib_cli.mk= (shared; under RT-project-share) - - Builds static lib from ~*.lib.c~ and CLI executables from ~*.cli.c~. - - Public targets: ~library~, ~cli~, ~clean~. ~cli~ depends on ~library~. -3.3 Role: kernel modules → =targets_kmod.mk= (shared; under RT-project-share) - - Builds out-of-tree modules (.ko) from ~*.mod.c~ using Kbuild. - - Public targets: ~kmod~, ~clean~. - -* 4. Public-Target Orchestration (strict) -4.1 The **orchestrator** owns public targets and their order. Roles expose callable, explicit targets but are not public entrypoints. -4.2 Example (shape): -#+BEGIN_SRC makefile -# developer/tool/makefile — public surface -RT_INCOMMON := $(REPO_HOME)/tool_shared/third_party/RT-project-share/release -include $(RT_INCOMMON)/make/environment_RT_1.mk - -.PHONY: all lib_cli kmod clean -all: lib_cli kmod - -lib_cli: - @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk cli - -kmod: - @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk kmod - -clean: - @$(MAKE) -f $(RT_INCOMMON)/make/target_lib_cli.mk clean - @$(MAKE) -f $(RT_INCOMMON)/make/targets_kmod.mk clean -#+END_SRC -4.3 Result: no target collisions (each role runs in its own ~make -f~). ~all~ remains the single happy path. - -* 5. Discovery & Suffix Contract -5.1 Suffixes define intent; scripts don’t embed filenames: - - ~*.lib.c~ → compiled into a static archive. - - ~*.cli.c~ → linked into CLI executables. - - ~*.mod.c~ → compiled by Kbuild into ~.ko~. -5.2 Discovery happens in the role files; the orchestrator can optionally export a filtered list (e.g., subset kmods), but roles must work without it. - -* 6. Authored vs Write Areas (contract) -6.1 **Authored** (=developer/cc/= and other documented trees): tools do not write, delete, or generate within. -6.2 **SCRATCHPAD** (=scratchpad/=): all intermediates (.o, .d, temp .c for Kbuild) and module outputs (.ko). ~.gitignore~ contains this. -6.3 **machine/**: released executables (checked in). If shared across hosts, embed arch in filename (e.g., ~prog-$(shell uname -m)~). “Machine-like” *temporary* outputs go to ~scratchpad/~, not ~machine/~. - -* 7. Dependencies -7.1 User-space role enables per-object depfiles: ~CFLAGS += -MMD -MP~; role includes ~$(OBJECT:%.o=%.d)~. -7.2 Depfiles live beside objects in ~scratchpad/~ and are removed by ~clean~. -7.3 No separate “make dependency” step is required; changes to headers or structure are picked up automatically. -7.4 Kbuild manages its own deps internally when invoked for kmods. - -* 8. Kbuild Integration (kmod) -8.1 Kbuild interface directory: =/lib/modules/$(uname -r)/build=. -8.2 All Kbuild artifacts are directed to project SCRATCHPAD via ~O=$(REPO_HOME)/$(SCRATCHPAD)~; sources remain authored. -8.3 Pattern (shape): -#+BEGIN_SRC makefile -# make/targets_kmod.mk -ifndef REPO_HOME - $(error REPO_HOME is not set; build must be done in a project environment) -endif - -# If not provided by caller, discover from authored sources (recommended default) -ifndef KBUILD_BASE_List -KBUILD_BASE_List := $(basename $(notdir $(wildcard $(REPO_HOME)/developer/cc/*.mod.c))) -endif - -KDIR := /lib/modules/$(shell uname -r)/build -KBUILD_OUTPUT_DIR := $(REPO_HOME)/$(SCRATCHPAD) - -KERNEL_OBJS_M := $(addsuffix .o,$(KBUILD_BASE_List)) -KMOD_TARGETS := $(addsuffix .ko,$(addprefix $(KBUILD_OUTPUT_DIR)/,$(KBUILD_BASE_List))) - -.PHONY: kmod clean -kmod: $(KMOD_TARGETS) - -$(KBUILD_OUTPUT_DIR)/%.c: $(REPO_HOME)/developer/cc/%.mod.c - cp $< $@ - -$(KBUILD_OUTPUT_DIR)/%.ko: $(KBUILD_OUTPUT_DIR)/%.c - $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m=$*.o - -clean: - $(MAKE) -C $(KDIR) M=$(REPO_HOME) O=$(KBUILD_OUTPUT_DIR) obj-m="$(KERNEL_OBJS_M)" clean -#+END_SRC - -* 9. User-Space Role (lib + CLI) -9.1 Role name: =target_lib_cli.mk=. -9.2 Pattern (shape): -#+BEGIN_SRC makefile -# target_lib_cli.mk (shape only) -# Scan *.lib.c / *.cli.c; derive OBJECT_LIB / OBJECT_EXEC under $(SCRATCHPAD) -# -include $(OBJECT_LIB:.o=.d) $(OBJECT_EXEC:.o=.d) -# library target builds $(LIBFILE) from OBJECT_LIB (empty archive is OK) -# cli target depends on $(LIBFILE), then links each $(EXECDIR)/% from %.cli.o + $(LIBFILE) -#+END_SRC - -* 10. Variable Conventions -10.1 Only variables intended for export to sub-processes are ALL_CAPS (environment-style). -10.2 =REPO_HOME= is authoritative (exported by env); do not assign to it in Makefiles. -10.3 Use =SCRATCHPAD= (not TMPDIR) as the canonical name for the scratch directory in new code; keep a compatibility alias only if needed. -10.4 Prefer explicit paths to archives (e.g., link with ~$(LIBFILE)~) unless you intentionally maintain ~-L/-l~ naming. - -* 11. Usage & Defaults -11.1 Role files put ~usage~ as the first target (direct calls print help and exit non-zero). -11.2 Orchestrator keeps ~all~ as the first target (so ~make~ “just works”). -11.3 Skip-if-empty logic is *not required*: if no sources match, role targets naturally no-op (empty prereq lists). -11.4 Optional diagnostic: a ~check-pwd~ target in the orchestrator can warn if ~$(CURDIR) ≠ $(REPO_HOME)/developer~. - -* 12. Release Conventions (machine/) -12.1 Released executables go to ~machine/~ and are checked in. -12.2 When a shared ~machine/~ is used across hosts, embed an arch tag in the filename at *build time* (not only at publish time). -12.3 Temporary binaries or architecture-variant scratch builds go to ~scratchpad/~. - -* 13. Summary (Do / Don’t) -13.1 Do keep roles independent; orchestrator stitches them via ~make -f~. -13.2 Do derive *what to build* from suffixes; don’t hard-code filenames. -13.3 Do confine writes to ~scratchpad/~ (and ~machine/~ for releases); never write in authored trees. -13.4 Do use ~-MMD -MP~ and include per-object depfiles; don’t require humans to rebuild deps. -13.5 Don’t recompute ~REPO_HOME~; don’t pass ~PWD~ unnecessarily; environment inheritance suffices. -13.6 Use clear names: public targets = ~lib_cli~, ~kmod~, ~clean~; role files = ~target_lib_cli.mk~, ~targets_kmod.mk~. -