From: Thomas Walker Lynch Date: Sun, 8 Mar 2026 16:23:07 +0000 (+0000) Subject: doc updates X-Git-Url: https://git.reasoningtechnology.com/%5B%5E?a=commitdiff_plain;h=7fd4aec227677b51f0995a11208cd8fd29aa6746;p=Harmony doc updates --- diff --git a/administrator/authored/document/setup.js b/administrator/authored/document/setup.js deleted file mode 100644 index 378fdfd..0000000 --- a/administrator/authored/document/setup.js +++ /dev/null @@ -1,4 +0,0 @@ -window.RT_REPO_ROOT = "../../../"; -document.write(''); -document.write(''); -document.write(''); diff --git a/administrator/document/01_Workflow_Build_Contract.html b/administrator/document/01_Workflow_Build_Contract.html deleted file mode 100644 index ac4f8fb..0000000 --- a/administrator/document/01_Workflow_Build_Contract.html +++ /dev/null @@ -1,96 +0,0 @@ - - - - - Workflow and Build Contract - - - - - - - - - - -

The Three Circular Loops

Development Loop: Developers author code, run experiments, and promote work product to the consumer/release/ directory via the release write script.
Developer-Tester Loop: Testers validate release candidates and file issues. Developers address these and promote new candidates. (A single person can play both roles locally to speed this up).
Release Branching Loop: Once a candidate meets the goals for a version, the tester creates a release branch (Major.Minor). Released code then has bugs filed against it, restarting the cycle.

- -

Administrator Role

Responsibilities:

Setup the project directory and keep it in sync with the Harmony skeleton.
Maintain role environments (apart from role-specific tool/setup files).
Install and maintain shared tools, addressing issues with the project workflow.

- -

Consumer Role

Responsibilities:

Act as the end-user simulation environment.
Consume and deploy artifacts exclusively from the consumer/release/ target.
Never author or modify code; strictly run local builds or deployments for architecture-specific testing.

- -

Developer Role

Responsibilities:

Write and modify authored/ source.
Run builds and stage artifacts in scratchpad/stage.
Spot test in experiment/.
Execute the release write script to copy artifacts to consumer/release for consumption/testing.

- - -

Tester Role

Responsibilities:

Validate candidates under consumer/release/.
Run regression suites.
Approve for quality and completeness, and create release branches.

- -

Entering the Project

- To enter a project, cd to the top-level directory and source the setup file for your desired role: -

- - . setup administrator - . setup developer - . setup tester - . setup consumer - -

- It is common to have multiple terminal sessions or IDEs open, each running under a different role environment. -

- -

Release Promotion

- Building and promotion are separate activities. The developer compiles and stages files in developer/scratchpad/stage. The developer then runs release write to transfer those files to consumer/release. -

- The consumer/release directory is strictly an untracked deployment target. No tools may rebuild during promotion, and no builds are run directly inside the release directory. -

- -

Separation of Roles

- To maintain integrity, strict boundaries exist between workspaces: -

A tester never patches code in the developer/ directory. Instead, the tester files issues or proposes code fixes on a separate branch.
A developer never writes into the tester/ directory. Instead, a developer adds tests to developer/experiment/ and offers to share them.

- - - - diff --git a/administrator/document/Harmony.md b/administrator/document/Harmony.md deleted file mode 100644 index 45fb023..0000000 --- a/administrator/document/Harmony.md +++ /dev/null @@ -1,42 +0,0 @@ -# Harmony â RT project skeleton - -Tiny, opinionated starter project skeleton that we use across RT projects independent of language being coded. - -Pick a role, source the env, build your thing, then release. - -## Roles (source these, donât execute) -- `env_developer` â dev workflow -- `env_tester` â test + repro -- `env_toolsmith` â shared tools + env wiring - -Developers work under `developer/`, testers under `tester/`, toolsmiths wire `shared/` and env scripts. - -## Layout (why it exists) -- `document/` â project docs (+ RT conventions in org) -- `developer/` â dev code, experiments, dev-specific docs/tools -- `tester/` â tests, fixtures, repro steps -- `shared/` â shared tools/env for all roles -- `shared/third_party/` â third-party tools -- `shared/third_party/python/` â your venv lives here (not committed) -- `release/` â publishable artifacts -- `tmp/` â scratch (gitignored) - -Empty directories are tracked with `.githolder` (kept out of release archives). - -## Quick start -```bash -# choose a role (must be sourced) -. setup developer # or env_tester / env_toolsmith - -# create the Python venv under shared/third_party/python/ (literally 'python' instead of 'venv' -./scripts/python_venv_bootstrap.sh - -# re-enter later -. setup developer - -# where used - -In public projects, this structure has been used with Python, Java, C, C++, and Lisp projects. - -Note the related https://github.com/Thomas-Walker-Lynch/RT-project-share project. It has the generic makefile used on C/C++ projects and other shared tools. Note the project https://github.com/Thomas-Walker-Lynch/RT_gcc for a more fully featured cpp. Note the projects https://github.com/Thomas-Walker-Lynch/Mosaic, and https://github.com/Thomas-Walker-Lynch/Mosaic for Java examples of this project skeleton being used for a Java testing and dependency grapph build tool, respectively. - diff --git a/administrator/tool/after_pull b/administrator/tool/after_pull deleted file mode 100755 index 946d9ad..0000000 --- a/administrator/tool/after_pull +++ /dev/null @@ -1,124 +0,0 @@ -#!/usr/bin/env -S python3 -B -# -*- mode: python; coding: utf-8; python-indent-offset: 2; indent-tabs-mode: nil -*- - -""" -set_project_permissions â normalize a freshly cloned project to Harmony policies. - -usage: - set_project_permissions [default] - set_project_permissions help | --help | -h - -notes: - â¢ Must be run from the toolsmith environment (ENV=tool/env, ROLE=toolsmith). - â¢ Starts at $REPO_HOME. - â¢ Baseline is umask-077 congruence: - - directories â 0700 - - files â 0600, but preserve owner-exec (â 0700 for executables) - applied to the entire repo, including release/, EXCEPT: - - release/kmod/*.ko â 0440 - â¢ Skips .git/ and symlinks. -""" - -import os, sys, stat - -# Must match shared/authored/env policy: -DEFAULT_UMASK = 0o077 # reminder only; effective modes below implement 077 congruence. - -DIR_MODE_077 = 0o700 - -def die(msg, code=1): - print(f"set_project_permissions: {msg}", file=sys.stderr) - sys.exit(code) - -def require_toolsmith_env(): - env = os.environ.get("ENV", "") - role = os.environ.get("ROLE", "") - if env != "tool/env" or role != "toolsmith": - hint = ( - "This script should be run from the toolsmith environment.\n" - "Try: source ./env_toolsmith (then re-run: set_project_permissions default)" - ) - die(f"bad environment: ENV='{env}' ROLE='{role}'.\n{hint}") - -def repo_home(): - rh = os.environ.get("REPO_HOME") - if not rh: - die("REPO_HOME is not set (did you source shared/authored/env?)") - return os.path.realpath(rh) - -def show_path(p, rh): - return p.replace(rh, "$REPO_HOME", 1) if p.startswith(rh) else p - -def is_git_dir(path): - return os.path.basename(path.rstrip(os.sep)) == ".git" - -def file_target_mode_077_preserve_exec(current_mode: int) -> int: - # Base 0600, add owner exec if currently set; drop all group/other. - target = 0o600 - if current_mode & stat.S_IXUSR: - target |= stat.S_IXUSR - return target - -def set_mode_if_needed(path, target, rh): - try: - st = os.lstat(path) - except FileNotFoundError: - return 0 - cur = stat.S_IMODE(st.st_mode) - if cur == target: - return 0 - os.chmod(path, target) - print(f"+ chmod {oct(target)[2:]} '{show_path(path, rh)}'") - return 1 - -def apply_policy(rh): - changed = 0 - release_root = os.path.join(rh, "release") - for dirpath, dirnames, filenames in os.walk(rh, topdown=True, followlinks=False): - # prune .git - dirnames[:] = [d for d in dirnames if d != ".git"] - - # directories: 0700 everywhere (incl. release/) - changed += set_mode_if_needed(dirpath, DIR_MODE_077, rh) - - # files: 0600 (+owner exec) everywhere, except release/kmod/*.ko â 0440 - rel_from_repo = os.path.relpath(dirpath, rh) - under_release = rel_from_repo == "release" or rel_from_repo.startswith("release"+os.sep) - top_under_release = "" - if under_release: - rel_from_release = os.path.relpath(dirpath, release_root) - top_under_release = (rel_from_release.split(os.sep, 1)[0] if rel_from_release != "." else "") - - for fn in filenames: - p = os.path.join(dirpath, fn) - if os.path.islink(p): - continue - try: - st = os.lstat(p) - except FileNotFoundError: - continue - - if under_release and top_under_release == "kmod" and fn.endswith(".ko"): - target = 0o440 - else: - target = file_target_mode_077_preserve_exec(stat.S_IMODE(st.st_mode)) - - changed += set_mode_if_needed(p, target, rh) - return changed - -def cmd_default(): - require_toolsmith_env() - rh = repo_home() - total = apply_policy(rh) - print(f"changes: {total}") - -def main(): - if len(sys.argv) == 1 or sys.argv[1] in ("default",): - return cmd_default() - if sys.argv[1] in ("help", "--help", "-h"): - print(__doc__.strip()); return 0 - # unknown command â help - print(__doc__.strip()); return 1 - -if __name__ == "__main__": - sys.exit(main()) diff --git a/administrator/tool/release b/administrator/tool/release deleted file mode 100755 index d72bf60..0000000 --- a/administrator/tool/release +++ /dev/null @@ -1,291 +0,0 @@ -#!/usr/bin/env -S python3 -B -# -*- mode: python; coding: utf-8; python-indent-offset: 2; indent-tabs-mode: nil -*- - -import os, sys, shutil, stat, pwd, grp, glob, tempfile - -HELP = """usage: release {write|clean|ls|help|dry write} [DIR] - write [DIR] Writes released files into $REPO_HOME/release. If [DIR] is specified, only writes files found in scratchpad/DIR. - clean [DIR] Remove the contents of the release directories. If [DIR] is specified, clean only the contents of that release directory. - ls List release/ as an indented tree: PERMS OWNER NAME (root-level dotfiles printed first). - help Show this message. - dry write [DIR] - Preview what write would do without modifying the filesystem. -""" - -SETUP_MUST_BE = "developer/tool/setup" -DEFAULT_DIR_MODE = 0o700 # 077-congruent dirs - -def exit_with_status(msg, code=1): - print(f"release: {msg}", file=sys.stderr) - sys.exit(code) - -def assert_env(): - env = os.environ.get("ENV", "") - if env != ENV_MUST_BE: - hint = ( - "SETUP is not 'developer/tool/setup'.\n" - "Enter the project with: . setup developer\n" - "That script exports: ROLE=developer; SETUP=$ROLE/tool/setup" - ) - exit_with_status(f"bad environment: ENV='{env}'. {hint}") - -def repo_home(): - rh = os.environ.get("REPO_HOME") - if not rh: - exit_with_status("REPO_HOME not set (did you '. setup developer'?)") - return rh - -def dpath(*parts): - return os.path.join(repo_home(), "developer", *parts) - -def rpath(*parts): - return os.path.join(repo_home(), "release", *parts) - -def dev_root(): - return dpath() - -def rel_root(): - return rpath() - -def _display_src(p_abs: str) -> str: - try: - if os.path.commonpath([dev_root()]) == os.path.commonpath([dev_root(), p_abs]): - return os.path.relpath(p_abs, dev_root()) - except Exception: - pass - return p_abs - -def _display_dst(p_abs: str) -> str: - try: - rel = os.path.relpath(p_abs, rel_root()) - rel = "" if rel == "." else rel - return "$REPO_HOME/release" + ("/" + rel if rel else "") - except Exception: - return p_abs - -def ensure_mode(path, mode): - try: os.chmod(path, mode) - except Exception: pass - -def ensure_dir(path, mode=DEFAULT_DIR_MODE, dry=False): - if dry: - if not os.path.isdir(path): - shown = _display_dst(path) if path.startswith(rel_root()) else ( - os.path.relpath(path, dev_root()) if path.startswith(dev_root()) else path - ) - print(f"(dry) mkdir -m {oct(mode)[2:]} '{shown}'") - return - os.makedirs(path, exist_ok=True) - ensure_mode(path, mode) - -def filemode(m): - try: return stat.filemode(m) - except Exception: return oct(m & 0o777) - -def owner_group(st): - try: return f"{pwd.getpwuid(st.st_uid).pw_name}:{grp.getgrgid(st.st_gid).gr_name}" - except Exception: return f"{st.st_uid}:{st.st_gid}" - -# ---------- LS (two-pass owner:group width) ---------- -def list_tree(root): - if not os.path.isdir(root): - return - entries = [] - def gather(path: str, depth: int, is_root: bool): - try: - it = list(os.scandir(path)) - except FileNotFoundError: - return - dirs = [e for e in it if e.is_dir(follow_symlinks=False)] - files = [e for e in it if not e.is_dir(follow_symlinks=False)] - dirs.sort(key=lambda e: e.name); files.sort(key=lambda e: e.name) - - if is_root: - for f in (e for e in files if e.name.startswith(".")): - st = os.lstat(f.path); entries.append((False, depth, filemode(st.st_mode), owner_group(st), f.name)) - for d in dirs: - st = os.lstat(d.path); entries.append((True, depth, filemode(st.st_mode), owner_group(st), d.name + "/")) - gather(d.path, depth + 1, False) - for f in (e for e in files if not e.name.startswith(".")): - st = os.lstat(f.path); entries.append((False, depth, filemode(st.st_mode), owner_group(st), f.name)) - else: - for d in dirs: - st = os.lstat(d.path); entries.append((True, depth, filemode(st.st_mode), owner_group(st), d.name + "/")) - gather(d.path, depth + 1, False) - for f in files: - st = os.lstat(f.path); entries.append((False, depth, filemode(st.st_mode), owner_group(st), f.name)) - gather(root, depth=1, is_root=True) - - ogw = 0 - for (_isdir, _depth, _perms, ownergrp, _name) in entries: - if len(ownergrp) > ogw: ogw = len(ownergrp) - - print("release/") - for (_isdir, depth, perms, ownergrp, name) in entries: - indent = " " * depth - print(f"{perms} {ownergrp:<{ogw}} {indent}{name}") -# ---------- end LS ---------- - -def iter_src_files(topdir, src_root): - base = os.path.join(src_root, topdir) if topdir else src_root - if not os.path.isdir(base): - return - yield - if topdir == "kmod": - for p in sorted(glob.glob(os.path.join(base, "*.ko"))): - yield (p, os.path.basename(p)) - else: - for root, dirs, files in os.walk(base): - dirs.sort(); files.sort() - for fn in files: - src = os.path.join(root, fn) - rel = os.path.relpath(src, base) - yield (src, rel) - -def _target_mode_from_source(src_abs: str) -> int: - """077 policy: files 0600; if source has owner-exec, make 0700.""" - try: - sm = stat.S_IMODE(os.stat(src_abs).st_mode) - except FileNotFoundError: - return 0o600 - return 0o700 if (sm & stat.S_IXUSR) else 0o600 - -def copy_one(src_abs, dst_abs, dry=False): - src_show = _display_src(src_abs) - dst_show = _display_dst(dst_abs) - parent = os.path.dirname(dst_abs) - os.makedirs(parent, exist_ok=True) - target_mode = _target_mode_from_source(src_abs) - - def _is_writable_dir(p): return os.access(p, os.W_OK) - flip_needed = not _is_writable_dir(parent) - restore_mode = None - parent_show = _display_dst(parent) - - if dry: - if flip_needed: - print(f"(dry) chmod u+w '{parent_show}'") - if os.path.exists(dst_abs): - print(f"(dry) unlink '{dst_show}'") - # show final mode we will set - print(f"(dry) install -m {oct(target_mode)[2:]} -D '{src_show}' '{dst_show}'") - if flip_needed: - print(f"(dry) chmod u-w '{parent_show}'") - return - - try: - if flip_needed: - try: - st_parent = os.stat(parent) - restore_mode = stat.S_IMODE(st_parent.st_mode) - os.chmod(parent, restore_mode | stat.S_IWUSR) - except PermissionError: - exit_with_status(f"cannot write: parent dir not writable and chmod failed on {parent_show}") - - # Atomic replace with enforced 077-compliant mode - fd, tmp_path = tempfile.mkstemp(prefix='.tmp.', dir=parent) - try: - with os.fdopen(fd, "wb") as tmpf, open(src_abs, "rb") as sf: - shutil.copyfileobj(sf, tmpf) - tmpf.flush() - os.chmod(tmp_path, target_mode) - os.replace(tmp_path, dst_abs) - finally: - try: - if os.path.exists(tmp_path): - os.unlink(tmp_path) - except Exception: - pass - finally: - if restore_mode is not None: - try: os.chmod(parent, restore_mode) - except Exception: pass - - print(f"+ install -m {oct(target_mode)[2:]} '{src_show}' '{dst_show}'") - -def write_one_dir(topdir, dry): - rel_root_dir = rpath() - src_root = dpath("scratchpad") - src_dir = os.path.join(src_root, topdir) - dst_dir = os.path.join(rel_root_dir, topdir) - - if not os.path.isdir(src_dir): - exit_with_status( - f"cannot write: expected '{_display_src(src_dir)}' to exist. " - f"Create scratchpad/{topdir} (Makefiles may need to populate it)." - ) - - ensure_dir(dst_dir, DEFAULT_DIR_MODE, dry=dry) - - wrote = False - for src_abs, rel in iter_src_files(topdir, src_root): - dst_abs = os.path.join(dst_dir, rel) - copy_one(src_abs, dst_abs, dry=dry) - wrote = True - if not wrote: - msg = "no matching artifacts found" - if topdir == "kmod": msg += " (looking for *.ko)" - print(f"(info) {msg} in {_display_src(src_dir)}") - -def cmd_write(dir_arg, dry=False): - assert_env() - ensure_dir(rpath(), DEFAULT_DIR_MODE, dry=dry) - - src_root = dpath("scratchpad") - if not os.path.isdir(src_root): - exit_with_status(f"cannot find developer scratchpad at '{_display_src(src_root)}'") - - if dir_arg: - write_one_dir(dir_arg, dry=dry) - else: - subs = sorted([e.name for e in os.scandir(src_root) if e.is_dir(follow_symlinks=False)]) - if not subs: - print(f"(info) nothing to release; no subdirectories found under {_display_src(src_root)}") - return - for td in subs: - write_one_dir(td, dry=dry) - -def _clean_contents(dir_path): - if not os.path.isdir(dir_path): return - for name in os.listdir(dir_path): - p = os.path.join(dir_path, name) - if os.path.isdir(p) and not os.path.islink(p): - shutil.rmtree(p, ignore_errors=True) - else: - try: os.unlink(p) - except FileNotFoundError: pass - -def cmd_clean(dir_arg): - assert_env() - rel_root_dir = rpath() - if not os.path.isdir(rel_root_dir): - return - if dir_arg: - _clean_contents(os.path.join(rel_root_dir, dir_arg)) - else: - for e in os.scandir(rel_root_dir): - if e.is_dir(follow_symlinks=False): - _clean_contents(e.path) - -def CLI(): - if len(sys.argv) < 2: - print(HELP); return - cmd, *args = sys.argv[1:] - if cmd == "write": - cmd_write(args[0] if args else None, dry=False) - elif cmd == "clean": - cmd_clean(args[0] if args else None) - elif cmd == "ls": - list_tree(rpath()) - elif cmd == "help": - print(HELP) - elif cmd == "dry": - if args and args[0] == "write": - cmd_write(args[1] if len(args) >= 2 else None, dry=True) - else: - print(HELP) - else: - print(HELP) - -if __name__ == "__main__": - CLI() diff --git a/document/Product_Development_Roles_and_Workflow.html b/document/Product_Development_Roles_and_Workflow.html new file mode 100644 index 0000000..4f46c62 --- /dev/null +++ b/document/Product_Development_Roles_and_Workflow.html @@ -0,0 +1,123 @@ + + + + + Product Development Roles and Workflow + + + + + + + + + + +

Roles as Hats

+ In Harmony, a role is a hat a person wears. There can be multiple people sharing a single role, and a single person can wear many hats. Some roles interact directly with the project directory structure, while others guide the process from outside the codebase. +

+ +

Workspace Roles

+ These roles interact directly with the repository. To enter a workspace, change directory to the top-level of the project and source the setup file for the desired role: +

+ + > . setup administrator + > . setup developer + > . setup tester + > . setup consumer + +

+ It is common for a person to have multiple terminal sessions or IDEs open, each running under a different role environment. +

+ +

Administrator Role

Responsibilities:

Set up the project directory and keep it in sync with the Harmony skeleton.
Maintain role environments (apart from role-specific tool/setup files).
Install and maintain shared and third_party tools, addressing issues with the project workflow. Note that the term "third_party" encompasses any software not authored within this specific project.

+ +

Developer Role

Responsibilities and Boundaries:

Write and modify authored/ source.
Run builds and stage artifacts in scratchpad/stage, then execute the release write script to copy artifacts to consumer/release for testing.
Run experiments in experiment/. These experiments can sometimes be promoted to formal tests, but there is no requirement to do so. The developer role should not blur into the tester role; experiments are informal, whereas tests are formal and retained.
Strict Boundary: A developer never writes into the tester/ directory. Instead, a developer adds tests to developer/experiment/ and offers to share them.

+ +

Tester Role

Responsibilities and Boundaries:

Evaluate candidates under consumer/release/ and run regression suites to confirm: +
- That the code does not crash.
- That consumers do not have a bad experience.
- That the goals specified by the product manager are met.
+
File issues and communicate feedback to the developers.
Strict Boundary: A tester never patches code in the developer/ directory. Instead, the tester files issues or proposes code fixes on a separate branch.

+ +

Consumer Role

Responsibilities:

Act as the end-user simulation environment.
Consume and deploy artifacts exclusively from the consumer/release/ target.
Never author or modify code; strictly run local builds or deployments for architecture-specific testing.
Report issues. (Anyone can report an issue, and consumers regularly do).

+ +

External Roles

+ These roles drive the project forward but do not have a dedicated setup <role> workspace, as they do not directly build or test the code in that capacity. If these individuals need to interface with the code, they simply put on a workspace role hat, such as administer, developer, tester, or consumer. +

+ +

Product Manager

+ The product manager receives specifications from the architect and sets the specific goals for a release. When the project manager provides updates indicating readiness, the product manager makes the final decision to cut a project release. +

+ +

Project Manager

+ The project manager owns the schedule (such as the Gantt chart) and monitors progress. They coordinate code reviews, read issues and announcements, reformulate the schedule, and instruct the product manager. When parts of the codebase are outsourced, the project manager serves as the primary point of contact with the external teams. +

+ +

Architect

+ The architect writes the original specification, sets the technical direction, and performs code reviews. The architect might be a contractor or an in-house team member. +

+ +

The Four Interacting Loops

+ The project moves forward through the continuous interaction of four distinct operational cycles. +

+ +

Developer Loop: Write code, make to the stage directory, promote to the consumer release directory, announce the release to the tester, read issues, and repeat.
Tester Loop: Read developer announcements, read the consumer release directory, run tests, and file issues.
Product Manager Loop: Provide specifications to the developer, receive progress updates from the project manager, and instruct the project administrator to make new git branch releases and deploy.
Project Manager Loop: Own the Gantt chart, coordinate code reviews, read issues and announcements, reformulate the Gantt chart, and instruct the product manager.

+ +

Release Promotion Mechanics

+ Building and promotion are separate activities. The developer compiles and stages files in developer/scratchpad/stage. The developer then runs release write to transfer those files to consumer/release. +

+ The consumer/release directory is strictly an untracked deployment target. No tools are permitted to rebuild during promotion, and no builds are run directly inside the release directory. +

+ + + +