From: Thomas Walker Lynch Date: Fri, 6 Nov 2026 11:45:19 +0000 (+0000) Subject: . X-Git-Url: https://git.reasoningtechnology.com/%5B%5E?a=commitdiff_plain;h=f79720aef6786a731ea56cc6b87970bae04333d9;p=Man_In_Grey . --- diff --git a/document/Man_In_Grey.html b/document/Man_In_Grey.html index caeb436..7920375 100644 --- a/document/Man_In_Grey.html +++ b/document/Man_In_Grey.html @@ -26,7 +26,7 @@ @@ -67,31 +67,28 @@ The ordered instructions the composers place, encoded as binary data in CBOR. - A journal entry. It consists of an instruction name followed by arguments. + A journal entry. It consists of an instruction operation specifier followed by arguments. In the sense of this manual, a symbol is a placeholder for an instruction result. Currently it is merely an index into the journal's instruction list. - A grammar that describes the available instructions, and for each instruction specifies the number of arguments it takes. + A grammar that describes the available instruction formats. Each instruction currently has the format of an instruction operation specifier followed by arguments. A compiled program that is invoked with a journal on its standard input, runs the guard on it, and when the guard approves the journal, runs the interpreter on it. - A program, started by the gate. It is given a journal, and either rejects it, or approves it by writing the one line guard <name> check passed and exiting 0. + A program, started by the gate. It is given a journal, and either rejects it, or approves it by writing the one line guard <language> check passed and exiting 0. Any semantic that participates in the guard code's approval process. - The program, started by the gate once the guard has approved the journal, that executes the journal's instructions in order. + The interpreter is invoked by the gate after, and only if, the guard has approved the journal. The interperter walks the journal while executing the instructions in order. - The two user ids a process carries. The effective uid determines what the process is permitted to do, and the real uid names the account the process was started for. See . - - - For each program, the account of its effective uid. The gate has two: account A, the gate invoking account, of its real uid; and account B, of its effective uid, which is the gate owner account when the gate is setuid. See . + The two user ids a process carries. The effective uid determines what the process is permitted to do, and the real uid identifies the ultimate invoking process's real uid. See . @@ -110,6 +107,9 @@

The journal is encoded in CBOR, whose strings are length-prefixed. Decoding CBOR makes maps, lists, strings and bytes, and cannot run code, so a journal reaches account B as data and nothing else.

+ Man_In_Grey replaces the template, not the syntax of the file being written. Content is still to be well formed for the program that reads the file, so a composer writing HTML escapes a < in its text as HTML requires. An instruction language can take that work over too, as the example in does. +

@@ -128,46 +128,40 @@ - Account and uid -

- Each account has a number, its user id, or uid. A file records the uid of the account that owns it, and a process carries two uids, its real uid and its effective uid. When a person logs in, the login program starts his shell with both uids set to that of his account, and every process started from the shell inherits them. In everyday speech such processes are said to be owned by the person, or to run under his account. As the two uids of a process can differ, this manual names the one it means, as in "the guard runs with the effective uid of account B". -

- The effective uid determines a process's permissions. The kernel compares it with a file's owner and permission bits when the process opens the file, and only a process whose effective uid is 0, root's, can do what is reserved to root. The real uid names the account the process was started for. Among its other uses, the kernel lets a process send a signal to another process whose real uid equals its own real or effective uid. -

- A process inherits both uids from the process that invoked it, unless the program file it runs has its setuid permission bit set. The kernel then takes the effective uid from the file's owner instead, and the real uid is inherited as before. A process therefore gets its effective uid in one of two ways: from the process that invoked it, or from the owner of a setuid program file. Groups work the same way, with a real gid, an effective gid and a setgid bit, beside the supplementary groups, which are inherited unchanged. -

- By convention, the gate is the only program of Man_In_Grey ever given the setuid bit. For each of the other programs, the account of its effective uid is named after the program: the orchestrator account, the composer account, the guard account and the interpreter account. The gate has two accounts. Account A, the gate invoking account, is the account of the gate's real uid, which the gate inherits from the program that invoked it. It is that program's real uid, and usually its effective uid too. Account B is the account of the gate's effective uid. When the gate is setuid, account B is the gate owner account, the account that owns the gate's file. -

- When the gate is not setuid, it inherits its effective uid from the invoking program as well, so accounts A and B are the same whenever that program's two uids are. They are also the same when the gate is setuid and owned by account A, as when a gate's owner invokes it. The gate starts the guard and the interpreter, and neither of their files is setuid, so both keep the gate's uids: the real uid of account A and the effective uid of account B. The guard account and the interpreter account are therefore account B. -

- The guard knows accounts A and B for certain, as the kernel gives them to it as its real and effective uid. However, apart from knowing that account A provided the journal, it has no way of externally verifying the journal's provenance. Note the further discussion of this topic in . -

+ Permissions - With an orchestrator + uid

- Man_In_Grey's orchestrator calls each composer as a function in its own process, so the composers run with the orchestrator's uids, and the composer account is the orchestrator account. The orchestrator then invokes the gate, so account A is the account of the orchestrator's real uid. When the orchestrator is started from a person's shell, both of its uids are those of his account, and the orchestrator account, the composer account and account A are all his account. + Each account has a number called its user id, or uid. Each file has an owner, noted with a uid. Each process is associated with two uids, its real uid and its effective uid. When a person logs in, the login program starts his shell with both uids set to that of his account, and every process started from the shell inherits them. In colloquial speech such processes are said to be owned by the person, or to run under his account. However, the two uids associated with a process can differ, so these colloquialisms are imprecise. +

+ The effective uid determines a process's permissions. So for example when opening a file, the kernel compares the process's effective uid with a file's owner and permission bits to determine if the process is allowed to open the file. Among its other uses, the kernel lets a process send a signal to another process whose real uid equals its own real or effective uid. +

+ A process inherits both uids from the process that invoked it, unless the program file it runs has its setuid permission bit set. In which case, the kernel then takes the effective uid from the file's owner instead, while the real uid remains inherited as before. A process therefore gets its effective uid in one of two ways: from the process that invoked it, or from the owner of a setuid program file. (Groups work the same way, with a real gid, an effective gid and a setgid bit, beside the supplementary groups, which are inherited unchanged.) +

+ By convention, the gate is the only program of Man_In_Grey that can optionally be given the setuid bit. When the gate is not setuid, it inherits its effective and real uids from the program that invoked it. When it is setuid, it inherits its real uid from the program that invoked it, and it inherits its effective uid from its file owner. Thus, the gate process has reliable information directly from the kernel on who invoked it, as that is its real uid. It also has reliable information directly from the kernel on who owned its executable file, as that becomes its effective uid.

- - - Without an orchestrator

- Man_In_Grey's orchestrator is one program that can invoke the gate, and any program can: an application acting as its own orchestrator, a shell pipeline, a service, or a command that sends a saved journal. All the gate knows is that a program whose real uid is account A's invoked it. + Due to this mechanism the effective uid for a setuid gate program can be different than the effective uid of the program that invoked it. In the chain of events of program A running and then invoking the gate, the effective uid in that thread of execution will start as the effective uid for program A, and then the effective uid will change to match the uid of the gate executable file owner. We can say that the effective uid change from A to B, where A is the invoking programs effective uid, an B is the owner of the gate executable file.

+ + + + Orchestrator

- Account A therefore answers for every journal it gives the gate. The guard cannot tell a journal from a careful composer from one typed by hand, copied from elsewhere, or made by code another account slipped into one of account A's programs. What bounds account A is everything its constraints allow, not what its composers happen to produce, so the constraints for each account are best written as narrowly as that account needs. + The orchestrator is a utility program provided with Man_In_Grey. The orchestrator calls one or more composer programs, then sends the resulting journal to a gate program. Like all processes, when the orchestrator program runs, it has a real and effective uid. When the orchestrator is run from a shell, these are both inherited from the shell, and are typically pointing to the login user. In any case, the composers that it invokes will then run with the same effective and real uids as the orchestrator. When the gate executable file has its setuid bit set, when the gate is invoke by the orchestrator, the gate will inherit its real uid from the orchestrator, and will take its effective uid from the gate's executable file owner.

- Likewise, anything that can run code with account A's uid has account A's use of the gate. Keeping the composers to account A, as describes, is one part of keeping that code to account A; the other programs account A runs are the rest. + An application making use of Man_In_Grey can bypass the orchestrator and call the composers and gate on its own. When this happens the gate's real ID will be that of the application. For a setuid gate, its effective uid still comes from its file owner.

- The gate's own protections hold whatever program invokes it, since none of them depends on the caller. The invoking program sets more of the gate's starting state than its arguments, standard input and environment, though: it can leave file descriptors open, lower resource limits, and send signals. The gate settles the first two before anything else; signals it cannot stop. See and . + It is possible that the application is design such that the gate executable file does not have its setuid bit set, or that it does have its setuid set but the gate owner is the same as that of the orchestrator or that of the direct calling application. In such a case, the execution thread will keep the same real and effective uids the entire time it runs. In this mode the Man_In_Grey can not achieve anything the application could not have achieved on its own, but it still might be a convenient expression of the desired functionality.

@@ -175,18 +169,38 @@ Gate

- A setuid program owned by root runs with root's effective uid, while its real uid still names the account that invoked it. That is how passwd lets a person change his own password in /etc/shadow, and how it knows whose password to change. A setuid gate works the same way: owned by account B, it runs with the effective uid of account B and the real uid of account A. + The gate is compiled because Linux ignores the setuid bit on a script started with #!. Given no arguments, the gate prints its usage. Its command flags reports the gate's real and effective uid and gid with their names, whether they differ, the name of the gate's instruction language, and the paths of the guard and the interpreter.

+ + 2026-11-06 07:55:01 Z [:] ThomasÂ·developer@IdealX + Â§/home/ThomasÂ·developer/project/Man_In_Grey/consumerÂ§ + > ./Man_In_Grey_gate/Man_In_Grey_file_gate flags + real.uid=2005 + real.user_name=ThomasÂ·developer + real.gid=2005 + real.group_name=ThomasÂ·developer + effective.uid=2005 + effective.user_name=ThomasÂ·developer + effective.gid=2005 + effective.group_name=ThomasÂ·developer + uid_differ=0 + gid_differ=0 + language=file + guard.path=/home/ThomasÂ·developer/project/Man_In_Grey/consumer/Man_In_Grey/guard_file.py + interpreter.path=/home/ThomasÂ·developer/project/Man_In_Grey/consumer/Man_In_Grey/interpreter_file.py + +

- The gate is compiled because Linux ignores the setuid bit on a script started with #!: between the kernel opening the script and the interpreter reading it, the script could be swapped. Given no arguments, the gate prints its usage. Its command flags reports account A, account B, whether they differ, and the guard's name and the paths of the guard and the interpreter. Its command admit takes the journal on standard input, with the options constraints <file> and check_only. When the gate is asked to admit a journal, it: + Th admit command takes the journal on standard input, with the options constraints <file> and check_only. When the gate is asked to admit a journal, it:

opens /dev/null on any of descriptors 0, 1 and 2 the invoking program left closed, and closes every descriptor above 2, so nothing the invoking program opened reaches the guard or the interpreter;
raises each resource limit that could stop the guard or the interpreter partway, file size and CPU time to unlimited, address space and data size to 1 GiB, stack size to 8 MiB, and open files to 64, as far as the hard limit allows, or past it when account B is root; and refuses when one stays too low;
finds the guard and the interpreter at paths fixed when the gate is compiled, relative to the gate itself, so account A cannot name others;
refuses unless the guard and the interpreter, and every directory above them, are owned by root or by account B, and writable by no other account;
clears the environment, setting only PATH and LANG, sets the file mode mask to 077, marks the process not dumpable, and changes directory to /;
records account A in MIG_GATE_INVOKING_UID, MIG_GATE_INVOKING_GID and MIG_GATE_INVOKING_NAME;
records its real ids, account A's, in MIG_REAL_UID, MIG_REAL_GID and MIG_REAL_USER_NAME;
reads the journal from its standard input, up to 16 MiB;
starts /usr/bin/python3 -E -s -B on the guard, with the journal on its standard input, and goes on only when the guard approves the journal;
starts the interpreter the same way, with the same journal.

- A gate that is not setuid keeps the uids of the program that invoked it, so accounts A and B are normally the same, and the guard still applies. When they are the same, the guard accepts constraints given with constraints <file>, checked as its own files are, since nothing runs with more than account A's own rights. When they differ, or account B is root, the guard refuses them and reads only its own constraints file. A gate that is not setuid is the convenient way to use Man_In_Grey on files the person owns, and the way to prototype an instruction language before making a gate setuid for it. + A gate that is not setuid keeps the uids of the program that invoked it, so accounts A and B are normally the same, and the guard still applies. When they are the same, the guard and the interpreter accept constraints given with constraints <file>, which the gate passes to both, checked as their own files are, since nothing runs with more than account A's own rights. When they differ, or account B is root, both refuse them and read only the constraints file installed beside them. A gate that is not setuid is the convenient way to use Man_In_Grey on files the person owns, and the way to prototype an instruction language before making a gate setuid for it.

One gate per instruction language

- Man_In_Grey's general gate is the library module Gate.lib.c, whose function Man_In_Grey_gateÂ·GateÂ·run takes the guard's name, the guard's path and the interpreter's path, and the command line. An instruction language's gate is a *.CLI.c of three lines that calls it with the three fixed. The gate of the file language: + Man_In_Grey's general gate is the library module Gate.lib.c, whose function Man_In_Grey_gateÂ·GateÂ·run takes the name of the instruction language, the guard's path and the interpreter's path, and the command line. An instruction language's gate is a *.CLI.c of three lines that calls it with the three fixed. The gate of the file language:

#include "Gate.lib.c" @@ -217,7 +231,7 @@ }

- The general gate is curried, with its first three arguments supplied when the gate is compiled. Whoever invokes a gate can choose neither program it runs; if he could, he could give it a guard that approves everything. The guard's name holds no space and no control character, and the gate refuses to run when it does. + The general gate is curried, with its first three arguments supplied when the gate is compiled. Whoever invokes a gate can choose neither program it runs; if he could, he could give it a guard that approves everything. The language name holds no space and no control character, and the gate refuses to run when it does. Man_In_Grey has two gates: the file language's, above, and Man_In_Grey_emplace_bio_gate, the gate of the example language in .

@@ -260,7 +274,7 @@ Verdict

- A guard reads the journal on its standard input. Its standard error is for the person: one line per finding, as <tag>: <message>, the first two name: and version:, then such lines as check_passed: wellformed or constraint_broken: <what is wrong>. Its standard output is for the verdict alone. A guard approves a journal by writing exactly one line there, guard <name> check passed, with the name the gate has compiled in, and exiting 0. The gate takes the journal as approved on that line and that status together, and on nothing else, and only then admits it to the interpreter; a guard that refuses exits 1, and one that cannot check exits 2. + A guard reads the journal on its standard input. Its standard error is for the person: one line per finding, as <tag>: <message>, the first two name: and version:, then such lines as check_passed: wellformed or constraint_broken: <what is wrong>. Its standard output is for the verdict alone. A guard approves a journal by writing exactly one line there, guard <language> check passed, with the language name the gate has compiled in, and exiting 0. The gate takes the journal as approved on that line and that status together, and on nothing else, and only then admits it to the interpreter; a guard that refuses exits 1, and one that cannot check exits 2.

The verdict travels out of band from the findings. A finding quotes the journal, a file name for instance, and goes to standard error, so nothing in a journal can put a line on standard output. A crash cannot approve a journal either: an uncaught Python exception exits 1, and a guard that dies leaves no verdict. @@ -271,7 +285,7 @@ Interpreter

- The gate starts the interpreter only after the guard has approved the journal, with the effective uid of account B and the same journal on its standard input. The interpreter does not check the constraints; that is the guard's work. It decodes the journal, checks once more that the journal is well formed, and then executes the instructions in order. + The gate starts the interpreter only after the guard has approved the journal, with the effective uid of account B and the same journal on its standard input. The interpreter does not check the constraints; that is the guard's work. It reads them, from the same file as the guard, only for what execution needs, as the example language's interpreter takes from them the directory its pages go in. It decodes the journal, checks once more that the journal is well formed, and then executes the instructions in order.

The interpreter writes one line to standard output for each thing it does, as <tag>: <message>, and the gate copies each line to the person and to the system log by the output rule. Its exit status is 0 when the journal was executed, 1 when execution failed, and 2 when the journal could not be read or is not well formed. @@ -289,6 +303,9 @@

The instruction language is not specific to Man_In_Grey. An application that mounts home directories has a language of mount and unmount; one that manages accounts has a language for making and deleting them. Each gate starts the guard and the interpreter of one instruction language, and they refuse a journal written in another.

+ A language can be made as narrow as its application. The narrower it is, the less a journal can say, and the less the guard has to rule out. The file language can name any file, so its constraints list the files allowed. The example language, , can name no file at all: what it writes, and where, follows from the account that gives the journal. +

The file language @@ -394,20 +411,27 @@ Building

- Man_In_Grey is a Harmony project with two namespaces: Man_In_Grey.copy, the Python programs, and Man_In_Grey_gate.make, the general gate and the file language's gate. In the developer role, from its directory developer/, build "*" and promote write place them in the following layout, which is how each part finds the others: + Man_In_Grey is a Harmony project with two namespaces: Man_In_Grey.copy, the Python programs, and Man_In_Grey_gate.make, the general gate and the gates of the file language and the example language. In the developer role, from its directory developer/, build "*" and promote write place them in the following layout, which is how each part finds the others:

- consumer/Man_In_Grey/Man_In_Grey the command a person runs - consumer/Man_In_Grey/Man_In_Grey.py the orchestrator - consumer/Man_In_Grey/Journal.py symbols, instructions, the journal, CBOR - consumer/Man_In_Grey/FileLanguage.py the file language: interface, checks, execution - consumer/Man_In_Grey/Guard.py the general guard - consumer/Man_In_Grey/Interpreter.py the general interpreter - consumer/Man_In_Grey/guard_file.py the file language's guard - consumer/Man_In_Grey/interpreter_file.py the file language's interpreter - consumer/Man_In_Grey/constraints_file.json its constraints, refusing everything - consumer/Man_In_Grey_gate/Man_In_Grey_file_gate the file language's gate - consumer/Man_In_Grey_gate/libMan_In_Grey.a the general gate, for other gates + consumer/Man_In_Grey/Man_In_Grey the command a person runs + consumer/Man_In_Grey/Man_In_Grey.py the orchestrator + consumer/Man_In_Grey/install_gate installs a gate setuid, run by root + consumer/Man_In_Grey/Journal.py symbols, instructions, the journal, CBOR + consumer/Man_In_Grey/Constraint.py loading the constraints + consumer/Man_In_Grey/Guard.py the general guard + consumer/Man_In_Grey/Interpreter.py the general interpreter + consumer/Man_In_Grey/FileLanguage.py the file language: interface, checks, execution + consumer/Man_In_Grey/guard_file.py the file language's guard + consumer/Man_In_Grey/interpreter_file.py the file language's interpreter + consumer/Man_In_Grey/constraints_file.json its constraints, refusing everything + consumer/Man_In_Grey/EmplaceBioLanguage.py the example language + consumer/Man_In_Grey/guard_emplace_bio.py its guard + consumer/Man_In_Grey/interpreter_emplace_bio.py its interpreter + consumer/Man_In_Grey/constraints_emplace_bio.json its constraints, refusing everything + consumer/Man_In_Grey_gate/Man_In_Grey_file_gate the file language's gate + consumer/Man_In_Grey_gate/Man_In_Grey_emplace_bio_gate the example language's gate + consumer/Man_In_Grey_gate/libMan_In_Grey.a the general gate, for other gates

The same layout is staged in developer/scratchpad/made/, so an experiment in the developer role of the Man_In_Grey project runs from there. Both sides need the Python package cbor2, each in its own Python. Man_In_Grey, and any program using the journal interface, runs on the first python3 on PATH, which in a Harmony role is the project's linked Python when there is one; install the package into it with python3 -m pip install cbor2. The gate always starts /usr/bin/python3 -E -s, so the guard and the interpreter use neither a person's Python nor a copy installed with pip install --user into ~/.local. The system Python needs the package from the distribution, installed by root: python-cbor2 on Arch, python3-cbor2 on Debian, and on openSUSE, which names packages for the Python they are built for, python313-cbor2 for Python 3.13. To see what the interpreter will see: /usr/bin/python3 -E -s -c 'import cbor2; print(cbor2.__file__)'. @@ -444,7 +468,7 @@

- Man_In_Grey orchestrate <path> runs the one composer in a composer file, or every composer in a composer directory. No option given to Man_In_Grey reaches compose; a composer's settings are written in the composer itself. + Man_In_Grey orchestrate <path> runs the one composer in a composer file, or every composer in a composer directory. No option given to Man_In_Grey reaches compose; a composer's settings are written in the composer itself. The journal_iface a composer is given is that of the gate's instruction language: the orchestrator asks the gate for its flags, and loads the module named for its language, FileLanguage.py for file and EmplaceBioLanguage.py for emplace_bio.

@@ -463,10 +487,10 @@ Run the composer in a composer file, or every composer in a composer directory, and invoke the gate with the journal. - The gate to invoke. Default: ../Man_In_Grey_gate/Man_In_Grey_file_gate from Man_In_Grey.py. + The gate to invoke, which also sets the instruction language the composers write in. Default: ../Man_In_Grey_gate/Man_In_Grey_file_gate from Man_In_Grey.py. - Constraints for a run in which accounts A and B are the same, given to the guard; refused when they differ or account B is root. + Constraints for a run in which accounts A and B are the same, given to the guard and the interpreter; refused when they differ or account B is root. Print the journal after composing. @@ -486,27 +510,248 @@ Installing a setuid gate

- As root, copy consumer/Man_In_Grey/ and consumer/Man_In_Grey_gate/ side by side into a directory owned by root and writable by no other account, make every file owned by root and not group- or other-writable, put the application's constraints in constraints_file.json, and make the gate owned by account B and setuid. The kernel ignores the setuid and setgid bits on a file system mounted nosuid, as removable drives and some separately mounted /home directories are, so the directory is to be on a file system mounted without it. For root as account B: + A setuid gate is installed by root, with install_gate, which is promoted beside Man_In_Grey:

- chown root:root Man_In_Grey_gate/Man_In_Grey_file_gate - chmod 4755 Man_In_Grey_gate/Man_In_Grey_file_gate - Man_In_Grey_gate/Man_In_Grey_file_gate flags + sudo consumer/Man_In_Grey/install_gate install <gate> owner <account> into <directory> [constraints <file>]

- 4755 is the mode 755 with the setuid bit, 4000, added. For an account B other than root, give the gate account B as its owner and account B's group as its group, and set the setgid bit, 2000, too: chmod 6755. Only root can give a file away, so an interpreter whose effective uid is that of an account B other than root can make files only account B owns, and its group has to be account B's group too. flags reports account A, account B, whether they differ, and the guard's name and the paths the gate finds the guard and the interpreter at. Promotion replaces files and sets neither the owner nor the setuid bit, so the installation is redone after each promotion that changes the interpreter. The installation step is a candidate for Man_In_Grey itself, once there is a language for installing programs. + It copies Man_In_Grey/ and the one gate named from Man_In_Grey_gate/ into the directory, side by side, as the gate's compiled paths expect. Every file and directory it installs is owned by root and writable by no other account, except the gate, which it gives to account B, the owner account named, and then makes setuid. The order matters, since changing a file's owner clears its setuid bit. For root as account B the gate's mode is 4755, the mode 755 with the setuid bit, 4000, added. For any other account B it is 6755, with the setgid bit, 2000, too, and account B's group as the gate's group. Only root can give a file away, so an interpreter whose effective uid is that of an account B other than root makes files account B owns, with the effective gid's group; setgid makes that account B's group. +

+ The constraints given with constraints <file> are installed as constraints_<language>.json. Without the option, an installed constraints file is kept, and a first installation gets the one promoted with Man_In_Grey, which refuses everything. install_gate refuses a directory that is not absolute, that an account other than root owns or can write, or that is on a file system mounted nosuid, where the kernel ignores the setuid and setgid bits, as it does on removable drives and some separately mounted /home directories. It ends by printing the installed gate's flags, as root sees them: real uid root's, effective uid account B's. Promotion replaces files and sets neither the owner nor the setuid bit, so the installation is redone after each promotion that changes the gate, the guard or the interpreter. The installation is a candidate for Man_In_Grey itself, once there is a language for installing programs.

Testing

- The tester's test tester/authored/write_file runs the promoted Man_In_Grey through a gate that is not setuid, so accounts A and B are the same, with constraints it writes into its scratchpad directory, and checks twenty-five things. Two runs of composers using every mode and unlink are compared with expected/run_1 and expected/run_2, with backup times replaced by the letters YYYYMMDDTHHMMSSZ. A file opened in create mode is written once and refused the second time. A journal with one allowed file and one the constraints do not list is refused whole. A composer giving a wrong mode fails before any journal reaches the gate. A composer directory holding a composer other accounts can write is refused by the orchestrator. A check_only run is approved by the guard and changes nothing. A run under a file size limit the gate cannot raise is refused before the guard starts. Each refusal leaves every file as it was. Since the gate checks ownership on every run, every directory from / down to the promoted interpreter is to be writable by no account but its owner. In the tester role, from its directory tester/: + The tester has two tests. Each runs the promoted Man_In_Grey through a gate that is not setuid, so accounts A and B are the same, with constraints it writes into its scratchpad directory. The test tester/authored/write_file runs the file language's gate, and checks twenty-five things. Two runs of composers using every mode and unlink are compared with expected/run_1 and expected/run_2, with backup times replaced by the letters YYYYMMDDTHHMMSSZ. A file opened in create mode is written once and refused the second time. A journal with one allowed file and one the constraints do not list is refused whole. A composer giving a wrong mode fails before any journal reaches the gate. A composer directory holding a composer other accounts can write is refused by the orchestrator. A check_only run is approved by the guard and changes nothing. A run under a file size limit the gate cannot raise is refused before the guard starts. Each refusal leaves every file as it was. Since the gate checks ownership on every run, every directory from / down to the promoted interpreter is to be writable by no account but its owner. In the tester role, from its directory tester/:

authored/write_file/run test authored/write_file/run clean +

+ The test tester/authored/emplace_bio runs the example language's gate, and checks twenty-six things. A page with every style and with text that looks like markup is compared with expected/page.html, and its picture with the one given. A later run with a JPEG places it and removes the PNG, and a run with no picture removes the JPEG. A check_only run changes nothing. Seven journals are then made without the journal interface and given to the gate directly, as a program other than the orchestrator could: one whose text instruction carries a file name, one with a style the language does not have, one with text before any paragraph, one with an SVG picture, one with more text than its constraints allow, one from an account its constraints do not list, and one for a page that has been made a symbolic link. Each is refused by the check that is to refuse it, and changes nothing: +

+ + authored/emplace_bio/run test + authored/emplace_bio/run clean + + + + + + Example: emplace_bio +

+ A host publishes a bio page for each of the accounts it allows, at /srv/http/bio/<user_name>.html, with the account's picture beside it. The directory belongs to the web server's account, http on Arch and www-data on Debian, so a person cannot write his page there himself. He writes a composer giving his display name, his text and his picture, and runs Man_In_Grey with the gate Man_In_Grey_emplace_bio_gate, which the administrator has installed setuid to the web server's account. The interpreter then runs with the effective uid of http, and builds the page. +

+ Here account B is not root, and the gate stands between two ordinary accounts. The example is made so that a reader can see each step is safe: a person can change only his own page, and only to a page made of his text and picture. +

+ + + The instruction language + + + The page's title and heading. Given once. + + + Starts a paragraph. + + + A run of text in the current paragraph. style is a list drawn from strong, emphasis, underline and strikethrough; an empty list is plain text. + + + The bytes of a PNG or JPEG image. At most once. + + +

+ No instruction names a file. The page's file names come from the user name of the real uid, which the kernel gives the guard and the interpreter, so a person can write only his own page. The directory comes from the constraints, which the administrator installs. A journal describes the whole page: a run that gives no picture removes the picture an earlier run placed. +

+ Decoration travels out of band. A run's styles are a list beside its text, not marks in it, and the interpreter turns each style into one fixed tag with no attribute: , , and <s>, nested in that order. It writes the rest of the page from a fixed form, and escapes every text it is given, so nothing a person gives becomes markup. There is nothing to sanitize: an attribute such as onload, or a <script> element, cannot be expressed in the language. +

+ + + + The composer +

+ The composer is Thomas's own program, running with his uid. It holds the text in code and reads the picture from beside the composer directory: +

+ + # composer/bio.py + + from pathlib import Path + + DISPLAY_NAME = "Thomas" + + BIO = [ + [("I write " ,[]) ,("software" ,["emphasis"]) ,(" and " ,[]) ,("documentation" ,["strong"]) ,("." ,[])] + ,[("Text such as this is shown as written." ,[])] + ] + + def compose(prov ,journal_iface): + journal_iface.display_name(DISPLAY_NAME) + for paragraph in BIO: + journal_iface.paragraph() + for content ,style in paragraph: + journal_iface.text(content ,style) + journal_iface.picture((Path(prov.composer_root_dpath).parent / "picture.jpg").read_bytes()) + +

+ A composer could as well read a file in a notation its author likes, such as *emphasis*, and turn it into runs. That conversion runs with the person's own uid, so a mistake in it stays on his side; the gate only ever receives runs and style names. +

+ + + + The constraints + + { + "gate_invoking_account": ["Thomas", "Alice"], + "directory": "/srv/http/bio", + "text_max_bytes": 8192, + "picture_max_bytes": 1048576 + } + +

+ One constraints file serves every account, since the page's name follows from the account. gate_invoking_account lists the accounts that can have a page. text_max_bytes bounds the display name and text together, and picture_max_bytes the picture. +

+ + + + The guard + + + Only the four instructions, with exactly their arguments; one display name; at most one picture; text only within a paragraph, and no paragraph without text; every style one of the four, none twice; no control character in any text. + + + Account A is listed in gate_invoking_account; the text and the picture are within their limits; the constraints give an absolute directory. + + + The picture begins with the signature of a PNG or a JPEG file. SVG is refused on purpose: it is a document that can hold script, and run it when opened by itself. + + + The directory exists, is reached through no symbolic link, and can be written by the effective uid's account; the account's page and pictures, where they exist, are regular files. + + +

+ The filesystem check asks whether the effective uid can write the directory. Python's os.access, like the system call access under it, answers for the real uid unless asked with effective_ids=True, since it was made for setuid programs to ask what the person who ran them could do. +

+ + + + The interpreter +

+ The interpreter writes the picture and the page to temporary files in the directory, gives each permission 0644, and renames each into place, then removes a picture the page no longer shows. The files are owned by http, the effective uid that made them. The page Thomas's composer gives: +

+ + <!DOCTYPE html> + <html> + <head> + <meta charset="utf-8"> + <meta http-equiv="Content-Security-Policy" content="default-src 'none'; img-src 'self'"> + <meta name="viewport" content="width=device-width, initial-scale=1"> + <title>Thomas</title> + </head> + <body> + <h1>Thomas</h1> + <img src="Thomas.jpg" alt="Thomas"> + I write software and documentation. + Text such as this is shown as written. + </body> + </html> + +

+ The interpreter's own form is built the way a composer builds a file: constant strings with the escaped texts between them. The Content-Security-Policy line tells the browser to run no script and load nothing but images from the same site, a second wall behind the first, since the page cannot hold script anyway. +

+ + + + Installing +

+ The administrator makes the directory, owned by the web server's account, and installs the gate, as root, with the constraints above in bio.json: +

+ + sudo install -d -o http -g http -m 755 /srv/http/bio + sudo consumer/Man_In_Grey/install_gate install Man_In_Grey_emplace_bio_gate owner http into /usr/local/lib/Man_In_Grey constraints bio.json + + + install_gate: installed /usr/local/lib/Man_In_Grey/Man_In_Grey/ + install_gate: installed /usr/local/lib/Man_In_Grey/Man_In_Grey/constraints_emplace_bio.json from /root/bio.json + install_gate: installed /usr/local/lib/Man_In_Grey/Man_In_Grey_gate/Man_In_Grey_emplace_bio_gate, owner http:http, mode 6755 + install_gate: its flags, as seen by root: + real.uid=0 + real.user_name=root + real.gid=0 + real.group_name=root + effective.uid=996 + effective.user_name=http + effective.gid=995 + effective.group_name=http + uid_differ=1 + gid_differ=1 + language=emplace_bio + guard.path=/usr/local/lib/Man_In_Grey/Man_In_Grey/guard_emplace_bio.py + interpreter.path=/usr/local/lib/Man_In_Grey/Man_In_Grey/interpreter_emplace_bio.py + install_gate: run it as: /usr/local/lib/Man_In_Grey/Man_In_Grey/Man_In_Grey orchestrate <composer> gate /usr/local/lib/Man_In_Grey/Man_In_Grey_gate/Man_In_Grey_emplace_bio_gate + +

+ The web server is to serve /srv/http/bio as static files, with no server-side processing, which is how it serves an .html file unless configured otherwise. +

+ + + + Running +

+ Thomas runs the installed Man_In_Grey from his bio directory, which holds composer/ and picture.jpg: +

+ + /usr/local/lib/Man_In_Grey/Man_In_Grey/Man_In_Grey orchestrate composer gate /usr/local/lib/Man_In_Grey/Man_In_Grey_gate/Man_In_Grey_emplace_bio_gate + + + gate: asked: real uid 2006 (Thomas), effective uid 996 (http), journal of 1857 bytes + guard: name: emplace_bio + guard: version: 1 + guard: constraints: /usr/local/lib/Man_In_Grey/Man_In_Grey/constraints_emplace_bio.json + guard: journal: 10 instructions + guard: check_passed: wellformed + guard: check_passed: constraint + guard: check_passed: content + guard: check_passed: filesystem + gate: admitted: guard emplace_bio check passed + interpreter: emplaced: /srv/http/bio/Thomas.jpg + interpreter: emplaced: /srv/http/bio/Thomas.html + gate: executed: the journal was executed + +

+ An account the constraints do not list is refused before anything is written: +

+ + gate: asked: real uid 2007 (alice), effective uid 996 (http), journal of 1855 bytes + guard: name: emplace_bio + guard: version: 1 + guard: constraints: /usr/local/lib/Man_In_Grey/Man_In_Grey/constraints_emplace_bio.json + guard: journal: 10 instructions + guard: check_passed: wellformed + guard: constraint_broken: gate invoking account 'alice' is not allowed to give journals to this gate + gate: refused: by the guard + +

+ The constraints list Alice, and account names are compared exactly, so the account alice is not her. A person who gives constraints <file> to this gate is refused too, since its real and effective ids differ. +

+ + + + Why it is safe +

A journal names no file, and the guard refuses an instruction with any argument the language does not have, so a journal cannot point the interpreter anywhere. The page's name comes from the kernel's real uid, and its directory from the administrator's constraints.
The interpreter runs with the effective uid of http, not root, so even a fault in it reaches only what the web server's account can write.
The page's text cannot become markup, since the interpreter escapes it and writes every tag itself. The page cannot become server code either: its name always ends in .html, never .php or .htaccess.
The guard refuses a page or picture that has been made a symbolic link, and a directory reached through one, so a link cannot redirect the write to another file of http's.
The picture is published as given. The content check confirms its format, not what it shows, nor what its metadata says: a JPEG from a phone can carry the place it was taken. A composer that is to strip such metadata does so before placing the picture.