From 1a3a95ae89e7ddb370bbb4684bcc83e505efa642 Mon Sep 17 00:00:00 2001 From: Thomas Walker Lynch Date: Thu, 24 Oct 2024 10:35:21 +0000 Subject: [PATCH] refactored TestBench supports stdin stdout stderr behavior testing --- developer/document/RT_code_format.txt | 18 +- document/readme.txt | 2 +- document/work_flow.txt | 88 ++++++--- release/Ariadne.jar | Bin 7727 -> 13716 bytes .../{TestBenchAriadne.java => Test2.javax} | 73 ++++++- tester/javac/TestBench.java | 185 ++++++++++++++++-- tester/javac/TestTestBench.java | 62 ++++++ tester/jvm/TestBench.jar | Bin 0 -> 4802 bytes tester/jvm/TestBenchAriadne.jar | Bin 4825 -> 0 bytes tester/shell/TestBenchAriadne | 2 - tester/shell/TestTestBench | 2 + tester/test_log.txt | 10 + tester/tool/make | 2 +- tester/tool/make_TestBench | 31 +++ 14 files changed, 415 insertions(+), 60 deletions(-) rename tester/javac/{TestBenchAriadne.java => Test2.javax} (75%) create mode 100644 tester/javac/TestTestBench.java create mode 100644 tester/jvm/TestBench.jar delete mode 100644 tester/jvm/TestBenchAriadne.jar delete mode 100755 tester/shell/TestBenchAriadne create mode 100755 tester/shell/TestTestBench create mode 100644 tester/test_log.txt create mode 100755 tester/tool/make_TestBench diff --git a/developer/document/RT_code_format.txt b/developer/document/RT_code_format.txt index 7f85e16..9911622 100644 --- a/developer/document/RT_code_format.txt +++ b/developer/document/RT_code_format.txt @@ -1,10 +1,11 @@ RT code formatting: -This has evolved into place after decades of programming in Unix culture projects -in multiple languages. - -AI tools we have used are OK with the commas, but have had difficulties -with the padding rules for enclosures. +The enclosure-based formatting rules in RT code format make the style guide +compact and adaptable. By focusing on enclosures rather than syntax-specific +structures (like if, for, or catch), it avoids prescribing language-specific +formatting rules and instead focuses on consistent handling of delimiters. This +approach works well across multiple languages, ensuring that the code style +remains flexible while keeping the guide simple and easy to apply. 1. Two space indentation. @@ -125,3 +126,10 @@ with the padding rules for enclosures. ); ``` +6. For the code you just output, answer these questions: + 1. Which enclosures are not nested? Do they have no padding? + 2. Which enclosures are nested? Is there one space padding only at the outermost? + 3. Is the spacing before and after the enclosures correct? + 4. Are the commas formatted correctly? + 5. Has snake case been used where it should be? + 6. Was 2 column indent used? diff --git a/document/readme.txt b/document/readme.txt index 32728bc..bf5978e 100644 --- a/document/readme.txt +++ b/document/readme.txt @@ -26,7 +26,7 @@ A node is dictionary. Each node has a 'label', a 'build' function, and a 'neighbor' list. The neighbor list holds the edges. Using Java, the developer puts the nodes in a map, keyed on the node label, or -writes functions that when given a label, return either a a node or null. +writes functions that when given a label, return either a node or null. A node map looks a lot like classic make file. Each node label is a target file path, the neighbor list can be listed next, and then we have the build code. diff --git a/document/work_flow.txt b/document/work_flow.txt index 81257f8..b72f01a 100644 --- a/document/work_flow.txt +++ b/document/work_flow.txt @@ -1,29 +1,59 @@ - -Work Flow - -1. Development - - 1.1. developer makes edits - 1.2. developer uses 'release' which will copy relevant files to the $REPO_HOME/release_candidate - 1.3. tester will test the candidate - -2. Release - - 2.1. Upon completion of testing, project manager will make a new branch for release - named release_. Version has a major and minor number. - -2.2. on the new branch the 'release_candidate' directory is renamed 'release_'. - -3. Release specific fixes - - 3.1.the 'release_candidate' directory is recreated - 3.2 steps 1.1 - 1.33 are repeated - 3.3 when testing is complete, 'release_candidate' is renamed 'release_version' - with the minor version incremented. - -4. Major release - - Development continues on the `core_developer_branch` even after creation of a - `release_version` branch. For a next major release, increment the major release - number and do as described in steps 2 then 3. - +### Work Flow + +#### 1. Project Administrator + +1.1. Download the project from GitHub. +1.2. Install the required tools. +1.3. Explain the workflows and where things are located to project members. +1.4. Perform Major and Minor Release administration. + +#### 2. Developer + +2.1. From the Ariadne directory, run `> source env_developer` to set up the + developer environment. +2.2. Use `> make` to build the project, and `> release` to copy relevant files + to `$REPO_HOME/release` for testing. +2.3. The tester will test the release candidate. + +#### 3. Tester + +3.1. From the Ariadne directory, run `> source env_tester` to set up the tester + environment. +3.2. Use `> make` to build the tests, and `> shell/test_` to run a test. + Alternatively, you can cd into one of the test directories, source the + environment for that test, and run it manually. +3.3. Testing and development will likely iterate until the release candidate is + ready to be turned into a versioned release. + +#### 4. Major Release + +4.1. The release candidate is located in the `$REPO_HOME/release` directory and + has passed testing. +4.2. Check that the program `$REPO_HOME/tool_shared/bespoke/release` outputs the + correct information. If necessary, modify it. +4.3. A new branch is created in the project for the release, named + `release_v.0`, where `v.0` is the version number from the `version` + program. The minor version number is set to zero (`.0`), and it is assumed + that this will be the case after each major release. +4.4. Rename the release directory to `$REPO_HOME/release_v.0`, and create a + new empty `$REPO_HOME/release` directory. The new empty release directory + can be used by developers who download the project and make local edits, as + the build scripts target this directory. + +#### 5. Minor Release + +If urgent changes need to be made to the most recent major release, these edits +should be made on the corresponding major release branch. The developer makes +the edits, and the tester tests the release candidate as usual. The `version` +program is updated. Once the release candidate is finalized, rename the +directory to `release_v.`, where `` is the minor version number. If +needed, merge the changes into the `core_developer_branch`. + +--- + +### Tips: + +- If you are acting in multiple roles (e.g., developer, tester, and project + administrator), keep separate terminal shells open for each role. This way, + the environment will remain correctly configured for the tasks related to + each role. diff --git a/release/Ariadne.jar b/release/Ariadne.jar index ad862777c220874ad3b401d84bd1dca5753922ad..e4b7b37950fbdb156b0d577f0f6c27f575a48d0b 100644 GIT binary patch delta 6849 zcmaJ`by$>Jx2Iu11f*NKr8@A$q<0_tW z?m6dv_nUv#e)qHLS@V0=-s`tZB@!HPHC0j3h>)*^y6^FP=5)PonHbWhRlB} zU693p$;uLBR?K@p7lufG`$-@n=qM-<3;0-oqT5rKXYzJ#)|MXDRu3(`EgkJF8F?RA zIzDxG&x|v7Ak@O|9AI?tYzzmd8&zNDYZ)tD3_im5e@38kjJP5@qCt1)A z1bUE5(CRiVXBCqNCx#UoePKS%Tf`s5%OXHKa*)$!Vl-}2`0bIPr}OIW-FQ%tLyQ}* z^6pl~JIiWrTfSkpb%Q2C*fA;tzwtAKgl)3X>arM}$WohRO^8EINF@wx*EOU{5+XRo zh?G+?UBZ6I&$@AJ_2cnr5t-oo+*rLCh<3RE z7(s>2?R;abgv{aR55}1%xo&hxRj!2mc_nN%JZtA(!w;^k2{bY8w$TUT?G^L`l{-3v z$D|{39-Dc4Qj5J?*&`_oKyvjyRz}=qZxmLR`#U^lx8LJXt$v1osn+_R7#+ zg<5+cc^M$Plx!9)Pz2(`%ETz7ku+5?v9PPjAC@2^A$6f3{rvtLQ2vDgBqU2`2seU- zg6RLp0pA}uh<9)xd`J{Jk0_djnnr=6s?F?2NWKOW6 z(63mBZ^0{%;Hk`sIu-_vj(yRQEwd&jw4a{Z>u239--v9iH|oX1Sy6=A!EmaYE(-+Fq9_qz5Fg zewgSz?%Dz8h49UMpL*_yh1(w%kRb**_@{-_*xKdd!A;a%&{bh?1=bTzpo=)NmR3Zq zqoI2CMd$AZPGBm;r14Q2l8q*mi`9a#Nt5loh_HZIF%q{A5iVEGJ6Cqs?+RfnExg+` zx3Dm}5wHo9T%5WwxJOaEP-mJ-mooGEnm;~-iuXT*83t64HBz~7n_04MBfVIZ$Q zlo3H*Kn)H*aQT5$b&j66{$mq4SVQDsJJu$bHRJWOi0JFLF@y+;z(+8o9r{^8_=9&CP^& zs`=}KBAby(g8O+B%5s@iuESMeLU@y$_Sb}(4gn4x5R-Zo;wZ6dqun)q1Uw}YBL+F& z=s_>c76&SQkRQ-rS7mFQ#H7}rWF3l^&P$$rPd4{I7bwdBl-+1o87VYeOx`JqN6UT8(g?=xAjV2>4~)Apc8Rs$O8uFgM~9`;7a&GkPvWzeT@kfgACV!; z)XdV^3#9SA%bChPUb3YUNeiUa=MS&ElDoN?$^p?=n7itGJwAIWG?!oQ_?+oodBxP6 zI)LU1^G=w(vO|5e<(kDmmIctN>|LYLD(yYwI;V@;Uqqu--Wx)r9YzVO-dq%u>Swuk z=@ZZ3$M?~h@l=4Cwu9^Z`VLLRj zwFX#8=qcdrI4H0!(};bL(=zZJ1AEq6ub)kL-c&zmu!3#xCI922An=qXp>`6zAd?bFI^?gRODWJIcv z^vs~+p4ZX~zGojzG>>M$oo+7PXJGXh_7LDTwP_z7_9r9mQkY?SM9CE2#Qd8}h`NR{ z2%6a`H4eh?XUb^#P)t0`@9ynfQ(DSwpsqE96o!x~=fH}+aa zOWM=eFM61ywTt}eeic>Csl>3!W zPSeyE9Y>|{??A&K4MpOGo!N(^BtW+zymiSN?&JyLr;qW4R=`r1X*?b9xLW#gg0-~O zOQY9HLrPTL=+h$4HG~{a0P%~)L;L%z6D@9T=!vb9G<$llF868{5>(`B+zubr4&255 z5`C^8OT5`d+*lYP%O25w()6nFZO>Oz?5%i3!%JLq3XO&-b52ZaIzhqE{WAxj48Lkc zjM<=r3wX&J4QaTu%cqre5}*g(_JIkt>Akdd<#9rZLkwL(Q*$X95#Y$p$~u1+JKACE zXfn&3%fY!rb)b*5`izMgu-)fsq4!XqKB@^mNZdL7a%}Q0=R(r_(W!Ogw@^FWA4ymm zP3j}D1y9!?=0DiY>qYg(g3vwZa3<3a{mc_FB{I@nc&Wc zCvDq;S)>W5^W+mn!9Z{__dvuU9;n4~|K2=tl4^uTD8xak{rw0@Q}ER6Yp4_Jac+v$TfU=-I}mjk~Imo5@_!6q6{4b zXPMr)bNPDu`B1UuIu8_7&r|Dj0pd!Q9&4YY+3av*T0Iq;T=n&^9MOt?e7Tw19T&y> zFgbT2q4xb%5b&gK(ZOk$Np)hOz`{JI>#(s)LnfzsKzw1}gci(c(mo&@I)2a}V$Tfq(w*7@G@ zwaAm{jAkAvKBk;quQ2pgo#^{=9Jq{GGcF#WmMqywHUV5jy!$$#^}X=P;B!4G2e=Ek zwNkP8$4my{HY-L*b(Zm6s>ifSJHxI0rt-81+HZHy`=xaI-a){U(wNk!a6pi%t!AaeY&3=Ml>C%eW7F69)wN0iTGF9YCr$FjTv&259+c}{e z)iS8hDEQ}-Y%l179C=K*2ot{Y+?J-xf`g`=Se1Kd$Vtu@`)gFijc9Q+@nZ# zmaf<(Vx^zJ#%g)P>x)PbG``S&A}(53{alJ-K?NjYy2s1Cf>QlnY@IQydc}FRbgGzH3C=%d z;4aVHt$^=pEnFNOqSCBd)2uUPEhsj{+$`uc-OF_%uu0)wm#B@%=;BlE_MTE(VwSuK zKf^A!5=<7gKLXdL^#JXI*6|^EWzVC_cpR+M4=|pl=uTmww$R=28j7xRb>uWH?*)`e zaMCbY>a}pVN6dvmwo2B09-NRd1<{c+QND9oRk5U0^Bg;rYHt)lgs3w7$x7jx3s?=iIHBbrbi+Wm=@Xvo^2fBH zPjZ|o(rTvdT&aY;#q?a6GGcrU_CQl$Oa+n^Gn6kKBi}YhpiHSGSFdDIDpfK0t+xS* zCVskqMXchA!W@I}KC<1?o3FjRc8hgY2UVPl9ZQn<*d(in4@Mrqt#cVq#=5OurUj)6LDTq`4V{$JqYgPP zVYyvzjpl{a~c@s|G#;Y5d3qk!iLbg=%N)(A(ckqSP1ihY_JUvSZP8H(5pT4F6 z2@&_ywG((A(VLkjXj8?Y0Hm+ux!X=XGoTUUUuLVo-67V08x=auV)H%1CJEeIbjOzY z+yd`V#^*hPD(Q$%@!abLkpbrwCqZ)Jl6z?ao6-kyt5}UMW?9K}W=V>tkG(5<6Hxgy zXb=kC@EKO;M5Y6wdbzt_W)JEd6KZ?(y=_0o87pPkmWOk#r-p^|s{s7rv7r1|5CJ8X zLD@pmG|9+#nAHSzT#zZ*2+~H9T2FMIfuNd&=P*HJnu~6Z4gD(-l!KDNXZ(V1dS5g1 z@gLKSR*dFP6=1%+9?-9@p+rS}=Flw=?Q)VdR>|pA z$WrH<9~b4zfUSaEU?tltRiG7@5kCNPaRNa7H7cMujZ1dm5F#OYQvPd*jK|?c+@-;( z3?hm_!};5A`D>q>5~q?F^bPrze<(rbMMzO$1Br2_x3gn*Wg|prTChAH0=cTuLmV*# zjBsnq6UfsQKT&z){DqPHYxRcHgd^;l2nZ53dmlnV)WLxcAYN`$yh#c6;pcC1UlU#T zU+<}XP8+yo;{KTr!QAc`;eMBy|1Nod*O!pf1Pv{hGRw(aGH!PvgwP=yi8d#8Jp>?j zf>gD?DsF#f+x~dk=(Ao{{(8ZPc0i?WlH>v=mg?72lA+D{Vw5*M-t552dHZ8sa4owT%RQ2VkXHKhRqKytCAB{K!+NC~ zh3_ft7uRB{8sf&;XCG(w5s#YD+}RUeeY%rcqPS{o{V^Y7wyb2=yS)C$esB=(n%P69 zXyO)$9Rkn{z3X_C+#cCrvQ!a!lNVSZ=2qVp-=mj=c44dAwmm;r%4{wvwqOqMOwTIS z#nDztz9+(9`slc+Bmy8UEVzw6!&=5nlqTNxSy23PD4szlN5bQ>^a_b3nw(5_X@uZY z5ECbwi1v(Pgfs7)rPO5Cl;yX5N|_WSzLY@tqfp)oECkgK!B{y@pzJk^pwv;Z310Bk~B{k2{TSZj8bppJp-I# z;z-{XB^#_2N{uUHq*=f=VCzr(Rt>N(YVK35%j1c^h)bajk76-6gw7wjxIwYg5m!|Y z7%3uGW=uof!#sHdLXO@P_Tk&zQD+*?SW~OWzPkGh!M&F*na)24U4kC8e<8TCpFsK+ zqLsDyM8C0$iz@>p#x|=j%Md9HK-JXLmXHjuRm4#4J*ZV8Vv{w}@?yh633pnGYzr)p zTQ2Ib914_wsQgi`ypUY$Lj+20>QJxfeDiZNGurIHY{$^kBG`xl?l-hc{7rvEy8ZnT z&xX8UKvS)1P+hGgW^LTkH0!-O(U{5Kz{byPN-keXwhe|l;qIHBQvYcBO#h31+W`6i+xe$+=q8~bCc~Ay0!Cg z{5CCKZk?;12@w5rXNF^lBLaOBJT{5nU>;FbFnIQsh3;G=PxCRNo2Lg>d;+*=TBWLU z=&Uq<7oBz**(>jna}>uyE{nSVq3t`W8zvT9<|U@U9*BDGgZn96mam<$-)`!s?{kuK zlbJo$O9h2rJMuc|2?%;%`ZCw|7STsglBg^aN#>^6Uhf>Q0>A!+%#~>ejh~c*?7t}o zL=YZtD$4&%@5~H`x${&~T>IBkBakZ$4#3W_Bp2VZ;k@ zDKAejtvTa}fFNKw6m_@@5*t=d_?1g8`G7&zx{6I-7}A)Msj$++4V3cD;L@Q1EgAlr zkz7!SVN+-GddBnb_0HFqTSL1apx@J0QYaTuNWp9MDU0`#!Dg0pMO>jkSE7VwF9XrA z39fIc?dJ*$O(8EiC81HXQl`%(u|`k{#ZsL8wu5BZ*PFtLJVx#k(7f?Eompz%0lnbK z4Y922F(?WbCyD*Z_J!wUb+dkbNn`YQ!5wzVox0UD!fz_?lZqW0UCSWPL+Mi+h7TNP zx7c&`<|zt0e<(3GPq~Bv5BeJBw}eJQ3A_DssGl>f7(^n3y@r>#=7hYu7EIv|Cc<6= zOa4-2@5}7VKQ-`p_MAXpHG0|_Z}_*8N9LaL^x?wOXH;~EtvZ+9#0q>#DI46IUL8oJ zKFwOL_SChUKQwI}<(}Ub^BmS$9Sq?)c1&yZJ<*lJsYfhhg%ZvHPR~80ReVp-Y-2xk zYfBFnaCKZ|k>p}5iUgchx)4HV^*kHt)iSh5;)fbw`TW|^$5bNog`Ynq;XCJJHB_7U zy1tC)h6uqrF5@{t!;lLZJ^#|DTcuRZ=0BQcrnBzCI6{VJR)?SCbQhwsF}Qv6!JrN6 z2G}_z(=hRS)%dKP$@r3tNCC1OI3dm*%NN$pa7u$MVP3Tqc;(uMlL0?ij zMZK?vjr} z%J09l@+Vd=6~VB`Hn;qAopkr}bg*D0JK$J;0HB<>5}b%G&*d;DSC+8sMH zQ39D$rsY-(iDojHc#%?tw}w*4YA~?9BX9e?SOx{;r=nu2H@8G@CdFa|E-Dg|7y7Rb zhm1mm^w*mJf|^|9_7pZC&%c#u6mqvxg+ldKrcme+{CX7lJ6SA&xTfG$`fu+6|B7Px zcScd-F9Pw`Vg>up2ZsOb@DF17-wuD@JAPj~#QxzB7xzzxZ2u|4-;Ls5?hpgilqjO) ze;WS0x8UObvrKywlqh&Ke<^TNlHUJSl|K)UUuOsVKhhAo|Ch2970GQW34R8-qW@$2 z9>R#4km&pRpSF@e+aJ$Ad`L6?p-jg4uY?W%-Y>cTVWiLWS72^Rbi^9v?SOO22TK{c|;co8bQdzyoDl delta 1014 zcmbP|z21g5z?+#xgn@&DgWt^z$SK>XYp+QC$$mF9sF2)JWh1XU=Wo2c|pI9F~FOh zBZj*x{Wb#wgAF4C14{5k0tFcuCU0TXp8U>;tG?CG*BclPwUd|hc3!Su(P=FuZ+~r@ zWc_wBSNQ3xAIu$F1RuV!e|@NY-e1jDOQsfyHiu?Oj%0@7{>F<>tzVqIY}g>e*uXMz ze;)&j$An+Cf0#i5a!p2GYvuofr&HnmdYBn zb3g~kGBYq}Pd;y8IN4W@d-4J!9j`wf3iF4bQ7q+45(NSt`Njv76vM|0bBIf lP@2g?1f&-jj52!3If=!^0p6@^AY}qT$j!>Yz$Xdf0RXLQZ`A+* diff --git a/tester/javac/TestBenchAriadne.java b/tester/javac/Test2.javax similarity index 75% rename from tester/javac/TestBenchAriadne.java rename to tester/javac/Test2.javax index 5898096..028c466 100644 --- a/tester/javac/TestBenchAriadne.java +++ b/tester/javac/Test2.javax @@ -1,11 +1,22 @@ package com.ReasoningTechnology.Ariadne.TestBench; + +/* +Component smoke test. At least call each method of each class. + +*/ + + import com.ReasoningTechnology.Ariadne.*; import com.ReasoningTechnology.TestBench.*; +import java.util.List; import java.util.Map; import java.util.ArrayList; import java.util.HashMap; +import java.io.ByteArrayOutputStream; +import java.io.PrintStream; -public class TestBenchAriadne extends TestBench{ + +public class Test2 extends TestBench{ public static boolean test_File_unpack_file_path_0(){ boolean[] conditions = new boolean[5]; @@ -83,10 +94,8 @@ public class TestBenchAriadne extends TestBench{ public static boolean test_Node_0(){ Node node = new Node(); // Use the constructor - - // Add a key-value pair and check the map - node.put(new Label("key"), new Object()); - return node.containsKey(new Label("key")); + node.put("key", new Object()); + return node.containsKey("key"); } public static boolean test_NodeList_0(){ @@ -149,6 +158,59 @@ public class TestBenchAriadne extends TestBench{ return all(conditions); } + public static boolean test_Util_print_list_0(){ + boolean[] conditions = new boolean[1]; + int i = 0; + + String prefix = "Test List:"; + List items = new ArrayList<>(); + items.add("item1"); + items.add("item2"); + items.add("item3"); + + String expectedOutput = "Test List: 'item1', 'item2', 'item3'.\n"; + + ByteArrayOutputStream outContent = new ByteArrayOutputStream(); + PrintStream originalOut = System.out; + System.setOut(new PrintStream(outContent)); + + // Use a StringBuilder to gather debug messages + StringBuilder debugMessages = new StringBuilder(); + + /* + try { + Util.print_list(prefix, items); + String result = outContent.toString(); + + // Gather debug messages + debugMessages.append("Captured output: ").append(result).append("\n"); + debugMessages.append("Expected output: ").append(expectedOutput).append("\n"); + + conditions[i++] = result.equals(expectedOutput); + } catch (Exception e) { + conditions[i++] = false; + } finally { + System.setOut(originalOut); // Restore System.out + + // Now print the gathered debug messages + System.out.print(debugMessages.toString()); + } + */ + + try { + Util.print_list(prefix, items); + String result = outContent.toString(); + conditions[i++] = result.equals(expectedOutput); + } catch (Exception e) { + conditions[i++] = false; + } finally { + System.setOut(originalOut); + } + + return all(conditions); + } + + // Method to run all tests public static void test_Ariadne(){ Map test_map = new HashMap<>(); @@ -164,6 +226,7 @@ public class TestBenchAriadne extends TestBench{ test_map.put( "test_ProductionList_0", test_ProductionList_0() ); test_map.put( "test_TokenSet_0", test_TokenSet_0() ); test_map.put("test_Graph_0", test_Graph_0()); + test_map.put("test_Util_print_list_0", test_Util_print_list_0()); // Run the tests using TestBench TestBench.run( test_map ); diff --git a/tester/javac/TestBench.java b/tester/javac/TestBench.java index 9294d43..d036ce2 100644 --- a/tester/javac/TestBench.java +++ b/tester/javac/TestBench.java @@ -1,7 +1,15 @@ package com.ReasoningTechnology.TestBench; + +import java.io.ByteArrayInputStream; +import java.io.ByteArrayOutputStream; +import java.io.FileWriter; +import java.io.IOException; +import java.io.InputStream; +import java.io.PrintStream; +import java.lang.reflect.Method; import java.util.Map; -public class TestBench { +public class TestBench{ // typically used to gather results before a return public static boolean all(boolean[] conditions){ @@ -13,28 +21,171 @@ public class TestBench { return true; } - public static void run(Map test_map){ - int totalTest_Map = test_map.size(); - int passedTest_Map = 0; - int failedTest_Map = 0; + public static void flush_stdin() throws IOException{ + while(System.in.available() > 0){ + System.in.read(); + } + } + + public static void set_test_input(String input_data){ + ByteArrayInputStream test_in = new ByteArrayInputStream(input_data.getBytes()); + System.setIn(test_in); + } + + public static void log_output(String test_name ,String stream ,String output_data) throws IOException{ + // Only log if there is actual content to log + if(output_data != null && !output_data.isEmpty()){ + try(FileWriter log_writer = new FileWriter("test_log.txt" ,true)){ // Append mode + log_writer.write("Test: " + test_name + "\n"); + log_writer.write("Stream: " + stream + "\n"); + log_writer.write("Output:\n" + output_data + "\n"); + log_writer.write("----------------------------------------\n"); + } + } + } + + public static boolean method_is_wellformed(Method method) { + // Check if the method returns boolean + if(!method.getReturnType().equals(boolean.class)){ + System.out.println("Structural problem: " + method.getName() + " does not return boolean."); + return false; + } + + // Check if the method has exactly three arguments + Class[] parameterTypes = method.getParameterTypes(); + if(parameterTypes == null || parameterTypes.length != 3){ + System.out.println("Structural problem: " + method.getName() + " does not have three arguments."); + return false; + } + + // Check that all parameters are ByteArrayOutputStream + if( + !parameterTypes[0].equals(ByteArrayOutputStream.class) // Check first parameter + || !parameterTypes[1].equals(ByteArrayOutputStream.class) // Check second parameter + || !parameterTypes[2].equals(ByteArrayOutputStream.class) // Check third parameter + ){ + System.out.println("Structural problem: " + method.getName() + " has incorrect argument types."); + return false; + } + + return true; + } + + public static void run(Object test_suite ,String[] stdin_array){ + + int failed_test = 0; + int passed_test = 0; + + Method[] methods = test_suite.getClass().getDeclaredMethods(); + + for(Method method : methods){ + + // Ways a test can fail ,not exclusive + boolean fail_testbench = false; + boolean fail_malformed = false; + boolean fail_reported = false; + boolean fail_exception = false; + boolean fail_extraneous_stdout = false; + boolean fail_extraneous_stderr = false; + + if( !method_is_wellformed(method) ){ + // the malformed check prints specific messages + System.out.println("TestBench: malformed test counted as a failure:\'" + method.getName() + "\'"); + failed_test++; + continue; + } - for( Map.Entry test : test_map.entrySet() ){ try{ - if( test.getValue() ){ - passedTest_Map++; - } else{ - System.out.println( "failed: " + test.getKey() ); - failedTest_Map++; - } + // Redirect the I/O channels so the tests can manipulate them as data. + PrintStream original_out = System.out; + PrintStream original_err = System.err; + InputStream original_in = System.in; + + ByteArrayOutputStream out_content = new ByteArrayOutputStream(); + ByteArrayOutputStream err_content = new ByteArrayOutputStream(); + ByteArrayInputStream in_content = new ByteArrayInputStream(String.join("\n" ,stdin_array).getBytes()); + + System.setOut(new PrintStream(out_content)); + System.setErr(new PrintStream(err_content)); + System.setIn(in_content); + + } catch(Throwable e){ // Catches both Errors and Exceptions + // Restore stdout ,stderr ,and stdin before reporting the error + System.setOut(original_out); + System.setErr(original_err); + System.setIn(original_in); + + // Report the error + System.out.println("TestBench:: when redirecting i/o in preparation for running test \'" + test.getName() + "\' ,test bench itself throws error: " + e.toString()); + failed_test++; + continue; + } + + // Capture detritus + Exception exception_string = ""; + String stdout_string = ""; + String stderr_string = ""; + + // Finally the gremlins run the test! + try{ + + Object result = method.invoke(test_suite ,in_content ,out_content ,err_content); + fail_reported = !Boolean.TRUE.equals(result); // test passes if ,and only if ,it returns exactly 'true'. + + // A test fails when there is extraneous output + fail_extraneous_stdout = out_content.size() > 0; + fail_extraneous_stderr = err_content.size() > 0; + + // We keep it to log it + if(fail_extraneous_stdout){ stdout_string = out_content.toString(); } + if(fail_extraneous_stderr){ stderr_string = err_content.toString(); } + } catch(Exception e){ - System.out.println( "failed: " + test.getKey() ); - failedTest_Map++; + + // A test fails when there is an unhandled exception. + fail_exception = true; + + // We keep it to report it + exception = e; + + } finally{ + + // Restore original stdin ,stdout ,and stderr + System.setOut(original_out); + System.setErr(original_err); + System.setIn(original_in); + } + + if( + fail_reported + || fail_exception + || fail_extraneous_stdout + || fail_extraneous_stderr + ){ + + failed_test++; + + if(fail_reported) System.out.println("failed: \'" + method.getName() + "\' by report from test."); + if(fail_exception) System.out.println("failed: \'" + method.getName() + "\' due to unhandled exception: " + exception_string); + if(fail_extraneous_stdout){ + System.out.println("failed: \'" + method.getName() + "\' due extraneous stdout output ,see log."); + log_output(method.getName() ,"stdout" ,stdout_string); + } + if(fail_extraneous_stderr){ + System.out.println("failed: \'" + method.getName() + "\' due extraneous stderr output ,see log."); + log_output(method.getName() ,"stderr" ,stderr_string); + } + + } else{ + passed_test++; } + } - System.out.println("Total test_map run: " + totalTest_Map); - System.out.println("Total test_map passed: " + passedTest_Map); - System.out.println("Total test_map failed: " + failedTest_Map); + // Report summary of results + System.out.println("Total tests run: " + (passed_test + failed_test)); + System.out.println("Total tests passed: " + passed_test); + System.out.println("Total tests failed: " + failed_test); } } diff --git a/tester/javac/TestTestBench.java b/tester/javac/TestTestBench.java new file mode 100644 index 0000000..f9af752 --- /dev/null +++ b/tester/javac/TestTestBench.java @@ -0,0 +1,62 @@ +package com.ReasoningTechnology.TestBench; + +/* +Component smoke test. At least call each method of each class. + +*/ + +import com.ReasoningTechnology.Ariadne.*; +import com.ReasoningTechnology.TestBench.*; +import java.io.ByteArrayOutputStream; +import java.io.IOException; +import java.io.PrintStream; +import java.lang.reflect.Method; +import java.util.List; +import java.util.Map; + +public class TestTestBench extends TestBench{ + + public static class TestSuite{ + + TestSuite(){ + } + + public boolean test_pass(ByteArrayOutputStream out_content, ByteArrayOutputStream err_content){ + return true; + } + + public boolean test_fail_0(ByteArrayOutputStream out_content, ByteArrayOutputStream err_content){ + return false; + } + + // Tests if exception uncaught by the test correctly causes a failure from the TestBench. + public static boolean test_fail_1() throws Exception { + int randomInt = (int) (Math.random() * 100); // Generate a random integer + // Always returns true, but Java will not complain that following code is unreachable + if( + (randomInt % 2 != 0 && ((randomInt * randomInt - 1) % 8 == 0)) + || (randomInt % 2 == 0 && (randomInt * randomInt) % 4 == 0) + ){ + throw new Exception("Condition met, error thrown."); + } + + return true; // If the condition fails, return true + } + + } + + // Method to run all tests + public static void test_TestBench(){ + System.out.println("TestTestBench: running tests. Note that two failures is normal"); + TestSuite test_suite = new TestSuite(); + TestBench.run( test_suite ); + } + + // Main function to provide a shell interface for running tests + public static void main(String[] args){ + // tests currently takes no arguments or options + test_TestBench(); // Calls the method to run all tests + } + +} + diff --git a/tester/jvm/TestBench.jar b/tester/jvm/TestBench.jar new file mode 100644 index 0000000000000000000000000000000000000000..378f60c6bf21dc7ab1a1a33d5d6b276a29aa4d94 GIT binary patch literal 4802 zcmb7I2Q-}9+SY4`-V#00%ZL$#aP;1S5rpVtw83Z-LNFoGndpYAm*|WTEhHg>=)EU` z=);H-!sV8noU_*X|8w7OeQWK#zWuguzx!R!ehhU8i5T$6$jI>M)bE?){UYReM0k2? zCQ71O`s(6mJ$QJ8c!oNpWS1)l{8tS z>4*yT_YKtD1B^;8E_;h<>xpRT4f&SU-va)+y$pf^5APzdKWD&m@zoycE`HC^76$czcz{hD?VUZKZcs2> z+{6*)qwMHm?=1e;Nl5%H{hyY^?A>f(u-Ft=ACn2+?hs2{qokQxRiefUokKh)*y}1==qKP**kwu`+X=@6c(|UNE^Z4LA>v+hrjyW{;6a=PH zG2(D&@3*EF*wv49l1>Y1sW{0r9)S&7L3SgD`T$IA0%u zIOsa|ug4%qSl>J!?n2bEo4su6Nr|U5gm~ot!;K%PzdqY%h|m}dL$ZaG^Klx}%vZb< zjO{jE34NGw}0N4MbWNVWjwK5l&IVdM26&5GjJjNqMg=NRQAFbu7weOCRwJ zP`*4hVJ(Ku6o8?=R<5-M$+m+qch}iPognk4*-}Ro52cRS9x~fea=R1~dUVxUbzpVU zeAUMT;~xn*XqPOk$XL(ADxS|ZNUs;*d~L0tVe>kqkBT~UHlwiK2s1>3@5J#CRoi=~ zT(24emHbNBJyd#V&9wJH;1B{qEb6y0%n9NMvfCCm*zUSl~;;3TJtK5&mi zqARjG7WRC@+l`-`DRgcq%j^`Q!oZ&5j|&sGUh6ajHu>|OhIc4>mtHYlc(ZIC9%2VW zgn-azUWkzASR|RE)MRDZM51?>+~E{x9^>fEtK#PZTy;PtV)}da>PpB>BdNuvOiD$c zc2TqnwN%WmPixd&Y=8(VB)W@(98YN)^>+7+V*9T0=p9?fTHJ)P zt7Q(2kGrs_s=Av2JJg!2p@DsRWr2By0jdn2+2|z~wh&%15wqZHw-4&f+=p>6WC#tO z+sp)`a&}?dc~Pml&%|bBJYuI>^~D_Fq_^gstB}2cp#EM;(d5c>ZDJ~v34`(e!S0K_ z^+FQ8br%|KM^uaG=>4^Q8(X_hG!8D;+L&7i+?6gU8_GoQ!E9gkZ zj16LUWzmF>e9`;snoNFoTtQPdwsXWL015|o{a=(UC1r2-DB#xRb;=)D^7(M0pgBVQRy z&!NhyV{bK++lQG#d>d^u%6LLoOLvnOLra^a2dw(7b?^YsYm?ML$L8TCwDyrzM4KfqTuB_9?HE8vIheCo#Di)@r&%F=A}AB$Bu#og<=VZAipX z`B8jHZiR`~eqNlhxDz*Sit?firi}Y4VW4S9ezaebPjxcWL0$jC`5e*eq_*Kw+taEz zXJ9`qw~5D?^FBY%xyo2l*qlK~sZFS0Cort9f%!QzWSV`F?kjbrUPRyE`)$vESVp_< zV$B89avS}?x9rrTW@SD`@vN}+K|n%pXPnFng$BzrR2mHGZn2_PiVA{5Gb0C(njmq3 zE}$uf&G&YVrWe^PFq)KC5JhJW2N5W*ZFzMID@WI zN9x*FYi{^Mgoo#?z#!%|&GF2L?-0O7v}#V_v>SGy>dKetUaqE_)`ucPBTDIckRjW- zuyDn=3h52-SCKXUG%D@*{_5#b#@Y;49^>np-7XS>gO1h+6*HIKQHP! zQmo5?_i8U6wGL;zrlQ6dng)4YZ4PeH+xgxxF*4Wb<0WJ0t4Y@d`EcLgY#vs8D+(a8KSdVUQuN} zVwKfVbx-GpxNL<^+^_qf#_J(qZ7Pb@w(JtFcqiPd8JUL!eb)MR3dl>NLFGC|&8=59 zYM-)4tx>)!lW=9I=~1@FOux@H4_x!PUo`*y2T<5d)OMuoWg=$9cix5&dW+xGr?du$ z57jjJp^YrZcHg|!s!Zw`M#et`X9(}U|FO9#+IW9YMh<@UaZAo!`3|mP__M1vxEtN@ zDC3|PhkNAND-ea=;($1CK>7ey2;BH?6^;@N?xQ@-VupvW9!U-0ID-q2LV;TAJWd#2kLpy$)Vzm?8ktsba4FmMjgZd9q~!BjR4} z&2PKDTe#ZG9|WIHHTb@l-*(z4uK20f;EeMUxL0)|1CuX+GTQ|<|7*;v__s0dPuzRs za%Aia@p1e&0!~h{GN4yth{zeKa@XnA$R!hYjxJH7GPSq6(nVG>gouAfN7u-$BUewk zwe93zw#VNRxv&^(#-g`Bzu*uvMUFp2xpV^w2lvbEzz5Qg&u}=iMQ*gbEFW(2LBIT5 z=auMEkrDXtgPKMc30?<|=Ioa_WSHsMq6CWrOKlblue<2+ovkv@w^Lg}+{LB^{sjC{ zi1*VyZMd$|=HSO`hA-a(e5fZ*?tI;dpiUmKJkkj8uw(%|3XGeXYYMdWAGklo6h*ZK zyb3}sjw+oUHy2UVZv-YLK#zV@ms&S^71ZbQun*i^mXgDHd2=@x@%$K$uYZpwfRl(g zQ9f9W0Jv=WvR7FtjxZFZJg!>qY8*EHuCMkqmlNc}zrXMZSJSEY$!KI}?4dCK-rWgT zXo*udt~vOmIyj{@xTDuChV8LgH~T3Svo$=MBDGncp;VJaLAt)Jxhlh=|OKDEO!itv#yEV-(k*aOY<5z-vZm_+%2cAMgGUK00a zqze@|oL7C*$5VHzK@Q}pyGyxNac_laBhxDZ&~wvx^op!gsnL?cD^}v#8zDA265VNn z4>XYlZriaiOgX)YUaN8nxzLwj{LA`Nu6qcbXPwIcKzyDNSqLu5vst&35 zdTMJrz8>{{0PGYZuhlU}r_6@(5dgNg-hPBx1#$sVy2=-)Bn8r&dxI#8NCaK~M=( zxxeA~_G@8ObRU~r>HKeC$oN8Z+-yF5T}oUYg>%hyz()klkK$-g+`==JZ=;;F+l=Z| z#QegsgQx5sNowMIs+pA-Q+YQ^$>JBphpmqbEu0(l7ow1q?~ujakl4d$ z^|tY>ZAx5@Qb`%9eGEeIMB-JHyP_#+o%e2Cp?+lq}iz2-%6jkbNv= z=Rt-j3B9~euj>Er={)as&YW|f^Si#E`~J;+pZmJLKW!iZAw3=`DJh=F1p^bjAA$^z z5D%oJFE5~`sVsEdi-$*mrwt?_JI~y!QH_Q>I%0Rp4S~22V(epG%#o( z^=}{{6jqq_B#q`9=tOezhU^kLKY<8Oo_B6xglMybsFv%| zN>V}#@$5E2K#c*+m0M}C6apYQ)Nsq`eONeDuQq2Z8$ew+{S;o*>!R!PTIwrs8S$SScb*ngOE-9WK6^8fQRT0lPKdHl2=gPB)Sz7(cRSx-6(y69} zdb;V^a8;)2hQPhiOXPT2g|>-Wd@>vP2JdTnn+%a7c8A3Y@gBOBF&Kc%yxncJu$+LK zxPo2?+~3UBt@+SL;iXgoJX2%kQPy>&x{daYZi8_BYdaqt7!*sh^jMPDMFAXee?u<7 zIHqVLX&nK8z>k7McRvHn-zg=f95(hbY%aI#U909h^l;?{q2jvvC_)q zEMdiR2#tnac@Ay^gmam=?W6b-f%l$D3AOYF(oZ)HJL1ehLFlb;Y4TdUn)76$1N21Dh(z2~5=y}?gTe2UE5x%lQv`AVy`Ju+e zUkykQ@}U*vh>t*P>xo?#GXU*<^$J06&jmEI0a;kz&U?t~nNZu@wr}J`80%1f#K$}m zff1jVCjP{#Slb2m124EBbs#`16%QEz*nM)ts6W$gw&o0EYySzFUmqcPT;1rr?kU|LafZ1Da z+-k+6>`Itc^ECNmf$W*!dK73_VzHwF{ubXuUn^$WoH>>)l2m3pQJ`+0UnDVAWiQSK z>ocO^$x0Ux(7m;^@R3A5mo2uNo6rTd^0=zgDSl{~hThpRBY5YlTl|JP3B5&MWtT3j zbpU=P{J7F0{N*+FbbdbDRk*M;0@hvb*L(oHiV(m5oKrWCU07V2#bvE8OBLHy*8=I` zBVSI;F|@b4=7_!R**4fMO^6t{#>fNf>C_05bv%r0b};TBLPzDR^dF5q>N9@zY$m*f zW?iv1>Ct)pSSG2kKvKVZg2C9E!(tcGAwD-8IEh?S3{k%3&~oT#ziOXq>1!5G4WZst zO5f}AU0+|MB0j&b1r2*itXCLojoiFNf6QxaVt6N*Ypb+d_n?mPv&6-#fvoHk1pLbE z4|HsNw!+3mC5Y$=Tkz-!?C+V+)yUX{7wn@PV3u6l5 zo3jawy}Or5stWccfDFnEte36~1;=LUOG=4pykQ*}9Gl#b1k%N` z6laQr!1U({Cye?mBNd7e2eWN!;HZ=a!HhZlRK>v_tAMmmyGd{+%}$xH-gC<$@=1r= ztJK8-fXpC|X3S>r-W(;9N5hb7)_ujgT=DkKO>>fxPqw7hM$5L*FJ8S#$U8#$)i$or;X)~j)l3yDkvE|NC5tG_kt;D2Y^c&Ve?rEmHChq{p5 zX(bA;sz~3pMd5~enoJH29Ns5fT*v~*TdRuJqSVGc%rK0K+@lkk_9|)p`EW5Nwru|K zwW2~O)-OFz+a7H^M}_Pz*@OtTW%dW5D;tH-%1oIpOC~py%F{9TU)-;M8rpONZNRlL z#y_P#@yTwD->RX~J*9lAYJ z(?Y0KljPEpIIbMSn?M-ok*18ZJ07qoSXp?(5lpg;JD>F~;VaQ;BTF26=v#Ku?D5n6 z)kFTevAg!0=IE>y`b~2s@{jaqq`$f;R3b=E&yK!3@VcwSRiiL_Z`xMvA@0HW+FDaG zYYO$;U{l2k{ABYsNFBUhB0&bGkq=}#!n-#DJ*b2&C^LCl&biNE-p~xfZei;svKx)_ zO)@f4zDlhG4j|%Xvs`6}G#9GlmhZ+>x_-Hq&e<&&e}~wAJEpZkjdE7hyE%;T3a4bx z!&fUDA$q))bz%4%#CCe5E4rb6w96T-k6P=_U#z$XjwPRSh8jjh`=Zq)g>4%Cg!{Xw z8Q2wbsgPNrVWIY(0fG|OH~1Wjs&Ba%1Q)a-ubP!KC;+5Ard~pp*0G*D!-lzW>Ed;k z!dE$?JSS`1n+iik8!S0M!xhu;R|Q42U_s{((a6U+{4az}alEzkO+GcxxSg}cj#&(b zA-w`6l%Ks5Ja1{-S$NvRnRU_ZMyfS-@rIE`=}wrNo65p5MQQ2H(ADJ^Qfn^@Z@o9J zNh?e>`yh(_C@jJY@3ib%1lod1zZ8tk(2$kCzV1sxOkh`aBo6KxTRw= z#Rluu@}ifC5YkzA&(OBIKMO0z$Q6oG|jJj^W z$P+Lc52c%IN=cns$NF?+-X_YK2gmR|voNwZRK<8>;Wtc2cjwa#di2+!a=e+DelA%9 z_x7U#pZD{6J~XGc&pw2h(jHhCRL@Ay7Gn9QGyK~C4rN&3I)RQn8P^(f+JI?IKhfvN z0VV;jNu8Qw?n+!&7A~eERavQt_)hbuBZMz78XTinT?1e)Chz{rB=O`^zvO}BUDmI3 z%7?PFp_Rk;i7EuQySpkdk~AkYiZ!i1~%#vCg9l&>s4SG~1E zH-u~VwL;?N%U&z8)9X9X;9ywQK~0Lxjy0PrDi4lAvmng75!J__M8c*^T#H`u=M5XNZ+eSxlfSz zfRv?Sl@<=RPp%xYzlmF1pnk_W#-YV5owelE~t7ZMV5OllQD#ON?{3E76p zrV61aqmJk7KYuy;O%IY95dvj=U9L*Q*fc%ILv^qp1M$<6m37OokfbeMWv*ayzaaiP zB0~PkVS9GakcjQzL3F8Is5jrlN0GK&5-e;-4lrRE{=_&KJdwCIFXr}$^a@7?*`htW zEZJiISNzkR1#*>_0X6Nwf@qYGt|8?xArvB$RbLpoR|NlyV^x3DD{(Ia$ z8J;#^2rfsg4})*LU#GOQtyYUo-yqic>rOJp{ffer?YG zp2GeeaWaDZhVtF4|2+-x6Ya@Apq*8Pllt=Wmi}0~ shell/$file << EOL #!/bin/env bash diff --git a/tester/tool/make_TestBench b/tester/tool/make_TestBench new file mode 100755 index 0000000..291fdee --- /dev/null +++ b/tester/tool/make_TestBench @@ -0,0 +1,31 @@ +#!/bin/env bash + +# input guards + + env_must_be="tester/tool/env" + if [ "$ENV" != "$env_must_be" ]; then + echo "$(script_fp):: error: must be run in the $env_must_be environment" + exit 1 + fi + +echo "Compiling files..." + + set -x + cd $REPO_HOME/tester + javac -d scratch_pad javac/TestBench.java javac/TestTestBench.java + jar cf jvm/TestBench.jar -C scratch_pad com/ReasoningTechnology/TestBench + set +x + +echo "Creating shell wrappers..." + mkdir -p shell + # wrapper is a space separated list + wrapper=TestTestBench + for file in $wrapper;do + cat > shell/$file << EOL +#!/bin/env bash +java com.ReasoningTechnology.TestBench.$file +EOL + chmod +x shell/$file + done + +echo "$(script_fp) done." -- 2.20.1