From: Thomas Walker Lynch Date: Fri, 4 Oct 2024 08:45:26 +0000 (+0000) Subject: more extraction of the depency graph tool X-Git-Url: https://git.reasoningtechnology.com/usr/lib/python2.7/sre_constants.py?a=commitdiff_plain;h=67f403fa1ca7d1bbb62122f1979c2672ccb9add6;p=GQL-to-Cypher more extraction of the depency graph tool --- diff --git a/developer/document/cycle_dection.html b/developer/document/cycle_dection.html deleted file mode 100644 index 3c9fb85..0000000 --- a/developer/document/cycle_dection.html +++ /dev/null @@ -1,105 +0,0 @@ - - - - Dependency Build Algorithm - - - -

Cycle Detection in Dependency Graph

-

Overview

-

- The is_acyclic_q function is designed to detect cycles in a dependency graph using a depth-first search (DFS) algorithm. It starts from a list of root node labels and traverses the graph to ensure that there are no cycles. If a cycle is detected, the function marks the nodes involved and continues to explore other parts of the graph. -

-

Key Concepts

- -

Functions

-

1. is_acyclic_q

-

- Purpose: To determine if the dependency graph is acyclic (i.e., contains no cycles). -

-

- Parameters: -

-

-

- Returns: -

-

-

- Process: -

-

-

2. is_acyclic_q_descend

-

- Purpose: To perform the actual DFS traversal and cycle detection for a given path. -

-

- Parameters: -

-

-

- Returns: -

-

-

- Process: -

-

-

Usage

-

- The is_acyclic_q function is used to ensure that the dependency graph defined in the build file is free of cycles. This is crucial for preventing infinite loops and ensuring that the build process can proceed smoothly. -

- - diff --git a/developer/document/dependency_graph.html b/developer/document/dependency_graph.html deleted file mode 100644 index 378c52d..0000000 --- a/developer/document/dependency_graph.html +++ /dev/null @@ -1,137 +0,0 @@ - - - - Dependency Build Algorithm - - - -

Cycle Detection in Dependency Graph

-

Overview

-

- The is_acyclic_q function is designed to detect cycles in a dependency graph using a depth-first search (DFS) algorithm. It starts from a list of root node labels and traverses the graph to ensure that there are no cycles. If a cycle is detected, the function marks the nodes involved and continues to explore other parts of the graph. -

-

Key Concepts

- -

Functions

-

1. is_acyclic_q

-

- Purpose: To determine if the dependency graph is acyclic (i.e., contains no cycles). -

-

- Parameters: -

-

-

- Returns: -

-

-

- Process: -

-

-

2. is_acyclic_q_descend

-

- Purpose: To perform the actual DFS traversal and cycle detection for a given path. -

-

- Parameters: -

-

-

- Returns: -

-

-

- Process: -

-

-

Usage

-

- The is_acyclic_q function is used to ensure that the dependency graph defined in the build file is free of cycles. This is crucial for preventing infinite loops and ensuring that the build process can proceed smoothly. -

- - - - -

2. Run Build Scripts

-
    -
  1. Traverse the queue starting from the tail (the most recently added nodes).
    2. -
  2. For each node, attempt to build it by checking the file dates of the target node and its dependencies: -
      -
    1. If the target file is older than any dependency, execute the node’s build function.
      2. -
    2. After building, recheck the file dates. If the file date has been updated, mark the node as successfully built.
      3. -
    3. If the build fails or the file date is not updated, mark the node with an error in its property list.
      4. -
    -
    3. -
  3. Nodes with dependencies marked with errors will not be built, and errors will propagate up the dependency tree.
    4. -
- -

3. Final Reporting and Status

-
    -
  1. If the root node is successfully built, report the success and any other successfully built nodes.
    2. -
  2. If an error has propagated to the root, report the failure.
    3. -
  3. Keep a list of all successfully built nodes and provide a final summary of the build status.
    4. -
- -

4. Node Definitions

-

Each node in the dependency graph is defined by a property dictionary. A node is either a symbol or a path:

-
    -
  1. Symbol Nodes: These represent abstract concepts or commands and always trigger a build unless marked with an error.
    2. -
  2. Path Nodes: These represent file paths. A path node is considered built if its target file is newer than its dependencies.
    3. -
-

Both node types are identified by a label, and their dependencies are stored as a list of node labels. The presence of an error property indicates that the node has failed to build or encountered a problem during processing.

- - - diff --git a/developer/document/deps.gradle b/developer/document/deps.gradle deleted file mode 100644 index 8427e1f..0000000 --- a/developer/document/deps.gradle +++ /dev/null @@ -1,240 +0,0 @@ -// useful regular expressions -def base = "[a-zA-Z0-9_-]+" -def ext = "[a-zA-Z0-9_-]+$" -def name = "${filebase}\.${fileext}" -def path = "[.]+/${name}" - - -// Predefined returnable functions for comparison -def no_match_f = { target -> - println "No match found for target: ${target}" -} - -def dependency_list_f = { jarFilePath, javaFilePath -> - println "Building JAR file for target: ${jarFilePath}" - println "Target: ${jarFilePath} depends on: ${javaFilePath}" - return [jarFilePath, javaFilePath] -} - -// Main function to handle target matching -def target_executor(token) { - // Match against the expected executor path pattern - def match = token =~ /(?:^executor\/)(${base})\.class$/ - - // If there’s no match, return the no_match function - if (!match) { - return no_match_f.curry(token) - } - - // Extract the base name of the matched target (e.g., "RuleNameListRegx") - def targetBase = match[0][1] - - // Define the expected Java and JAR file paths - def javaFilePath = "${javaCompInDir}${targetBase}.java" - def jarFilePath = "${executorDir}${targetBase}.jar" - - // Check if the corresponding Java file exists in the javac directory - def javaFile = new File(javaFilePath) - if (!javaFile.exists()) { - // Return the function to print the missing Java file message - return no_java_file_f.curry(javaFilePath) - } - - // Return the function that handles the JAR build step - return build_jar_f.curry(jarFilePath, javaFilePath) -} - -// Example usage: -def f = target_executor("executor/RuleNameListRegx.class") -if (f == no_match_f) { - println "No match found" -} else if (f == no_java_file_f) { - println "No corresponding Java file" -} else { - f() // Call the returned function to perform the build -} - - - - -// useful regular expressions -def base = "[a-zA-Z0-9_-]+" -def ext = "[a-zA-Z0-9_-]+$" -def name = "${filebase}\.${fileext} -def path = "[.]+/${name}" - -def target_executor( token ){ - - // "(?:" ... ) matches are thrown away - def match = token =~ /(?:^executor/)(${base})(?:${ext})/ - - // if there is a corresponding primary javac/primary/%1.java file, - // then creating the executable wrapper depends upon the .jar - // file, which will later chain back to the java file. - - ... - -// returns a function that produces the jar file, or returns -// a function that, if invoked prints the message that there is -// no corresponding java file. - - - return ... - -} - - -// Define the build target, dependencies, and functions as triples -def build_triples = [ - [ - targets: [""] - deps: ["jvm/%1.class"] - ] - - ,[ - targets: ["(?:${path}/) - ... - - ], -] - - - - -def programMap = [ - [target: ["exectutor/ - - // RuleNameListRegx does not require ANTLR or external tools - "RuleNameListRegx": [ - "wrapped_jar_from_java": "RuleNameListRegx", - "java_depends": [], - "grammar": [], - "program_deps": [] - ], - - // RuleNameList does not require ANTLR or external tools - "RuleNameList": [ - "wrapped_jar_from_java": "RuleNameList", - "java_depends": [], - "grammar": [], - "program_deps": [] - ], - - // Synthesize_SyntaxAnnotate_PrintVisitorMethod has dependencies on Java classes - "Synthesize_SyntaxAnnotate_PrintVisitorMethod": [ - "wrapped_jar_from_java": "Synthesize_SyntaxAnnotate_PrintVisitorMethod", - "java_depends": ["StringUtils"], - "grammar": [], - "program_deps": [] - ], - - // Synthesize_SyntaxAnnotate_PrintVisitor depends on Synthesize_SyntaxAnnotate_PrintVisitorMethod - "Synthesize_SyntaxAnnotate_PrintVisitor": [ - "wrapped_jar_from_java": "Synthesize_SyntaxAnnotate_PrintVisitor", - "java_depends": ["StringUtils", "Synthesize_SyntaxAnnotate_PrintVisitorMethod"], - "grammar": [], - "program_deps": [] - ], - - // ANTLRv4_RuleNameList requires the ANTLRv4 grammar and is used in parsing ANTLR grammars - "ANTLRv4_RuleNameList": [ - "wrapped_jar_from_java": "ANTLRv4_RuleNameList", - "java_depends": [], - "grammar": ["ANTLRv4"], - "program_deps": [] - ], - - // Arithmetic_Echo requires the Arithmetic grammar and ANTLR-generated files - "Arithmetic_Echo": [ - "wrapped_jar_from_java": "Arithmetic_Echo", - "java_depends": ["Arithmetic_Echo_PrintVisitor"], - "grammar": ["Arithmetic"], - "program_deps": [] - ], - - // Arithmetic_Echo__Test also depends on the Arithmetic grammar - "Arithmetic_Echo__Test": [ - "wrapped_jar_from_java": "Arithmetic_Echo__Test", - "java_depends": ["Arithmetic_Echo_PrintVisitor"], - "grammar": ["Arithmetic"], - "program_deps": [] - ], - - // Arithmetic_SyntaxAnnotate has dependencies on Arithmetic and generated files - "Arithmetic_SyntaxAnnotate": [ - "wrapped_jar_from_java": "Arithmetic_SyntaxAnnotate", - "java_depends": ["Arithmetic_SyntaxAnnotate_PrintVisitor"], - "grammar": ["Arithmetic"], - "program_deps": [] - ], - - // Arithmetic_SyntaxAnnotate__Test has similar dependencies as Arithmetic_SyntaxAnnotate - "Arithmetic_SyntaxAnnotate__Test": [ - "wrapped_jar_from_java": "Arithmetic_SyntaxAnnotate__Test", - "java_depends": ["Arithmetic_SyntaxAnnotate_PrintVisitor"], - "grammar": ["Arithmetic"], - "program_deps": [] - ], - - // Arithmetic2_SyntaxAnnotate has its own grammar and dependencies - "Arithmetic2_SyntaxAnnotate": [ - "wrapped_jar_from_java": "Arithmetic2_SyntaxAnnotate", - "java_depends": ["Arithmetic2_SyntaxAnnotate_PrintVisitor"], - "grammar": ["Arithmetic2"], - "program_deps": [] - ], - - // Arithmetic2_SyntaxAnnotate__Test has the same dependencies as Arithmetic2_SyntaxAnnotate - "Arithmetic2_SyntaxAnnotate__Test": [ - "wrapped_jar_from_java": "Arithmetic2_SyntaxAnnotate__Test", - "java_depends": ["Arithmetic2_SyntaxAnnotate_PrintVisitor"], - "grammar": ["Arithmetic2"], - "program_deps": [] - ], - - // ANTLRv4_SyntaxAnnotate requires the ANTLRv4 grammar - "ANTLRv4_SyntaxAnnotate": [ - "wrapped_jar_from_java": "ANTLRv4_SyntaxAnnotate", - "java_depends": ["ANTLRv4_SyntaxAnnotate_PrintVisitor"], - "grammar": ["ANTLRv4"], - "program_deps": [] - ], - - // Arithmetic_Swap has multiple dependencies on generated files and grammars - "Arithmetic_Swap": [ - "wrapped_jar_from_java": "Arithmetic_Swap", - "java_depends": ["Arithmetic_SwapVisitor", "Arithmetic_Echo_PrintVisitor"], - "grammar": ["Arithmetic"], - "program_deps": [] - ], - - // GQL-related program with its own grammar and dependencies - "GQL_20240412_SyntaxAnnotate": [ - "wrapped_jar_from_java": "GQL_20240412_SyntaxAnnotate", - "java_depends": ["GQL_20240412_SyntaxAnnotate_PrintVisitor"], - "grammar": ["GQL_20240412"], - "program_deps": [] - ], - - "GQL_20240412_SyntaxAnnotate__Test": [ - "wrapped_jar_from_java": "GQL_20240412_SyntaxAnnotate__Test", - "java_depends": ["GQL_20240412_SyntaxAnnotate_PrintVisitor"], - "grammar": ["GQL_20240412"], - "program_deps": [] - ], - - // Additional tools - "TerminalToCategory": [ - "wrapped_jar_from_java": "TerminalToCategory", - "java_depends": [], - "grammar": [], - "program_deps": [] - ], - - "GrammarSplitter": [ - "wrapped_jar_from_java": "GrammarSplitter", - "java_depends": [], - "grammar": [], - "program_deps": [] - ] -] diff --git a/developer/document/node_labels_unique_q.txt b/developer/document/node_labels_unique_q.txt deleted file mode 100644 index b978a74..0000000 --- a/developer/document/node_labels_unique_q.txt +++ /dev/null @@ -1,15 +0,0 @@ - -predicate == is_well_formed_q - -We can not check that the node labels are unique because the given value -is a single node, and the code is stateless. Besides there is no contract with -the programmer on how to use the predicated, so the programmer could call the -predicate multiple times on the same node. Now can we test this condition in -our do_markup_graph routine because of the way lookup works, it will always -return the same node for the same label. This would be a truly difficult check -to perform because the map does not given an error but just takes the second of -the duplicate key definitions (is this really true?) besides, it would require -a formal proof of the recognizer functions that they do not return different -definitions for different keys to match regexprs against. I've been mulling -this over. As we currently the programmer provides the map and function -definitions, we don't even know which nodes will be in the graph... diff --git a/document/directory_naming.txt b/document/directory_naming.txt index aca6100..137f5fa 100644 --- a/document/directory_naming.txt +++ b/document/directory_naming.txt @@ -1,44 +1,126 @@ -Naming Conventions for Directories +Directories Naming Convention +----------------------------- - Our shop has a convention of naming directories after the role of the person, - the name of the program, or generally, the agent, who is going to use the - files in the directory. In short, the name is often the answer to the question - "who are the files are for?". +Property based file organization - Sometimes there is not a good answer to that question. A good example is the - documents directory. There really isn't a good term for the role that people - are playing when the read the documents. Perhaps, 'readers'? This is not a - job function, and it is a somewhat ambiguous. Perhaps, 'projectologist'? Ah - nah .. + I am experimenting with a file system where instead of having directories, we + have collections of files that have a property in common. - When we can not answer the question of who the files are for, we instead - choose another common property shared by each and every file in the directory. - It is often the case that a property that each and every file has in common - will be singular. Hence in the example in the prior paragraph, when each file - in a directory has the property of being a document, the directory gets called - 'document'. + In this distribution the project is on a conventional file system, but we still + have the directory names. These following properties have been used, in + order of preference: -Top Level Directory + 1. Who the file is for. This is typically a role of a contributor + to the project, or the name of a program. - The top level of a github project is of course named after the project. Though - we people like to see actors related to project names. Look at all the - mythical animals on the covers of the O'Reilley manuals. + 2. The role that the file plays in the project. This is more general than + saying a file is for a specific program, or even kind of program. As an + example, the 'tool' directory. + + 3. A role that the directory plays. As an example, 'scratch_pad'. Another + example is 'deprecated' which is a short term archived file. + + In this third category the file property is imposed upon the file, + rather than being a description of the file's contents. + + One side effect of grouping files based on a shared property is that + the directory name, coming from the property name, is often singular. + +Who the file is for + + These file groups fit the model well: developer, tester, user. Also in the + developer directory, directories named after programs work well, as examples, + python, groovy, cc, linker, javac, etc. + +Generalization of who the file is for, e.g. executor + + Sometimes multiple related actors operate on the files in a directory. In + which case we give the directory a more general name that describes the + actors as a group. + + So if we had a directory that held a mix of files for various compilers we might + name the directory 'compiler_input' or even 'compiler'. If the files are + interpreted and multiple interpreters are involved, then 'interpreter_input'. + + A such generalization you will see in this project is 'executor'. An executor + is any program that runs another program; examples include shells that + interpret shell scripts; various language interpreter; and most famously the + machine loader, which will load an instruction sequence into memory and point + the program counter at it. + +document + + What role do people play when reading documents? That this question does not + have an obvious direct answer says something about our values, and this in + turn might explain why so few people actually read the documents. + + An author will call the person reading his work 'the reader', perhaps even + addressing a reader directly, 'Dear Reader'. However, the are a large number + of reader programs and devices, such as Adobe Reader and a bar card reader. + Unlike toolsmith, developer, and tester, being a 'reader' is not a job + title that a person is going to be addressed by. + + A person might reach for latin and use the word, 'lector'. However it is + a stretch to get the meaning from this one. Perhaps we move to Greek, and + imagine the role 'projectologist'. lol. Perhaps shortened to 'ologist'? + + Actually, there is a word for a person who studies books, articles, and + documents, a 'student'. It is actually a fairly accurate description of the + role a person plays while trying to learn about the project. The only drawback + is that it is not a professional role title. Perhaps it is not much of a + stretch to say someone is still in the student phase learning about a project. + I tried this label out for a while, but it did not really fit. + + There is a clear property choice. Each file in the document directory has the + property of being a document, hence, the directory name is 'document'. The + document directory at the top level is for the project manager, while the + document directory in the developer's directory is about the code being + developed and hold to build it. + + +Purpose - tool, temporary, deprecated + + For some other directories it is even more of a stretch to try and say + they are for an actor. + + Ancient man would make up spirits in such situations. Perhaps then files in a + temporary directory are for the 'tanuki', a real animal in Japan who that has + taken on mythical proportions causing things to disappear or playing tricks. A + directory of deprecated files could said to be for 'Lethe', the river that + carries away the memories of those who are reincarnated - which reminds me of + the rivers in so many places that carry away trash. I tried these names + out. It was good entertainment, but when trying to explain them I met with + blank expressions. + + Since in these cases we can not answer the question of who the files are for, + we instead choose another common property shared by each and every file in the + directory. Hence we end up with 'tool' for the directory with tools, instead + of saying they are for the tool smith, or even for Hephaestus, besides tool + smiths such as Hephaestus create tools, and the tool directory is full of + tools that were already created. + +Top level directory + + The top level of a github project is of course named after the project. Here + to us programmers have appealed to mythology to find actors. Just look at all + the mythical animals on the covers of the O'Reilley manuals. The top level directory of our git project is reserved for project manager to - use. The project manager builds the directory structure, initializes the - repository, installs tools that will be needed, and generally administers - the project. + use. The 'project manager', in this context, builds the directory structure, + initializes the repository, installs tools that will be needed, and generally + administers the project. In the environment, the top level directory is located at `$REPO_HOME` -Developer +developer The developer's directory is located at `$REPO_HOME/developer`. This directory contains the developer's workspace. Developers are free to - organize it in any manner they see fit, though they should continue to follow the - convention of naming directories after the agents that operate on the contained files. + organize it in any manner they see fit, though they should continue to follow + the convention of naming directories after properties when it is practical to + do so. As examples, @@ -51,32 +133,21 @@ Developer the project involves files for another tool or compiler, the directory is named after that tool. -Executor - - Sometimes multiple related actors operate on the files in a directory. In - which case we give the directory a more general name that describes the - actors as a group. - - So if we had a directory that held a mix of files for various compilers we might - name the directory 'compiler_input' or even 'compiler'. - - One common generalization is 'executor'. An executor is any program that runs - another program; examples include shells that interpret shell scripts; - various language interpreter; and most famously the machine loader, which will - load an instruction sequence into memory and point the program counter at it. - -Temporary +scratch_pad - This is a scratch pad directory used by programs. Files will appear - and disappear from here. There is no reason a developer can not manually - add a file, but scripts such as `make clean`, might delete it. Directories - with this name should be git ignored. + This is a temporary directory used by programs. Files in this directory are + typically removed by the program that placed the file, or by a `clean` script. + There is no reason a developer can not manually add a file, but scripts such + as `make clean`, might delete it. Directories with this name should be git + ignored. -Deprecated +deprecated As a developer I often have files that I set aside just in case I want to look at them again. Sometimes I plan to bring them back later. Unlike temporary files, they are not deleted by any clean script or any program that is using them as - intermediate files. This directory is also git ignored. + intermediate files. This directory is similar in intent to `git stash` or + using to and going back to look at old versions. The contents of this directory + do end up in the repo. LocalWords: projectologist