tyarn.1 (5916B)
1 \.TH TYARN 1 2020-12-24 2 .SH NAME 3 tyarn \- scenario testing of Unix command line tools 4 .SH SYNOPSIS 5 .B tyarn \fR[-12CdEehlstv] \fIscenarios\fR \fIimplementation...\fR 6 .SH DESCRIPTION 7 tyarn is a scenario testing tool. tyarn translates scenarios written 8 in a natural language to implementations written in shell code. Scenarios 9 written in a natural language should be supplied in the \fIscenarios\fR file, 10 shell code implementations for each scenario specified in \fIscenarios\fR 11 should be supplied in one or more \fIimplementation...\fR files. 12 The scenario below describes a simple test for a copy utility, 13 the SCENARIO line contains the name of the scenario (which must be unique). 14 .PP 15 .nf 16 .RS 17 SCENARIO copy a file 18 GIVEN a file F 19 WHEN the file F is copied to G 20 THEN the file G exists 21 .RE 22 .fi 23 .PP 24 A scenario is composed of a set of steps, at a minimum a scenario must include 25 at least one WHEN step and at least one THEN step, though it may also provide 26 more than one of each. A scenario may also 27 optionally include ASSUMING, GIVEN, and FINALLY steps. 28 GIVEN describes a condition that must be satisfied before the test can be 29 executed. WHEN describes an instruction that will be executed as part of 30 the test. THEN is used to assert a condition is true 31 now that the test instruction(s) in the WHEN section have been executed. 32 FINALLY steps will be executed at the end of each scenario regardless 33 of whether the scenario passed or failed. 34 The ASSUMING step can be used to state an assertion that must be 35 true in order for the scenario to be run. If the assertion is false 36 then the scenario is skipped and will not be executed. This can be 37 useful for testing on different platforms where you may want to 38 skip tests that are not supported on the platform on which the tests 39 are being run. In addition 40 to these step types, a special AND type can also be used 41 to indicate that the current step has the same type as the previous step. 42 For example, the scenario: 43 .PP 44 .nf 45 .RS 46 SCENARIO copy a file 47 GIVEN a file F 48 GIVEN a file A 49 WHEN the file F is copied to G 50 WHEN the file A is copied to B 51 THEN the file A exists 52 THEN the file B exists 53 .RE 54 .fi 55 .PP 56 can instead be written as: 57 .PP 58 .RS 59 .nf 60 SCENARIO copy a file 61 GIVEN a file F 62 AND a file A 63 WHEN the file F is copied to G 64 AND the file A is copied to B 65 THEN the file A exists 66 AND the file B exists 67 .RE 68 .fi 69 .PP 70 The AND step type mostly helps scenarios be a bit more readable in English. 71 .PP 72 A corresponding implementation must be provided for each scenario step, 73 for example shown below are the implementation steps for the 74 example scenario given above. 75 .PP 76 .RS 77 .nf 78 IMPLEMENTS GIVEN a file F 79 touch F 80 .PP 81 IMPLEMENTS WHEN the file F is copied to G 82 cp F G 83 .PP 84 IMPLEMENTS THEN the file G exists 85 test -e G 86 .fi 87 .RE 88 .PP 89 The implementations above are a straightforward translation from natural 90 langauge to shell code. Scenario implementations can be paramaterised 91 using POSIX regular expressions to avoid repetition. For example, the implementations 92 above can be written as: 93 .RS 94 .nf 95 .PP 96 IMPLEMENTS GIVEN a file ([A-Za-z0-9]+) 97 touch "$MATCH_1" 98 .PP 99 IMPLEMENTS WHEN the file ([A-Za-z0-9]+) is copied to ([A-Za-z0-9]+) 100 cp "$MATCH_1" "$MATCH_2" 101 .PP 102 IMPLEMENTS THEN the file ([A-Za-z0-9]+) exists 103 test -e "$MATCH_1" 104 .fi 105 .RE 106 .PP 107 For each capture an environment variable MATCH_N is defined, 108 where N is the nth capture. 109 Written this way the implementations may be used for any scenario 110 step they are a match for. 111 .PP 112 Scenarios are executed in a temporary directory, the environment 113 variable DATADIR is set to the path of this temporary directory. 114 The environment variable SRCDIR is set to the path of the working 115 directory tyarn is executed from. 116 .SH OPTIONS 117 .TP 118 .BR \-C 119 Don't cleanup temporary files and directories 120 .TP 121 .BR \-d 122 Enable debug 123 .TP 124 .BR \-E 125 Exit early (exit when first scenario fails) 126 .TP 127 .BR \-e=\fINAME=VALUE\fR 128 Add an environment variable to the test's environment 129 (this argument can be repeated to add multiple variables) 130 .TP 131 .BR \-h 132 Display help message and exit 133 .TP 134 .BR \-l=\fISHELL_LIBRARY\fR 135 Set shell library to be used by scenario implementation. This is a shell script that will be included at the top of every scenario's implementation 136 .TP 137 .BR \-s=\fISHELL\fR 138 Execute scenario implementations using SHELL. If unspecified tyarn defaults to sh 139 .TP 140 .BR \-t=\fIDIR\fR 141 Use DIR as temporary directory for tests, DIR is a template e.g. /tmp/foo.XXXXXXXX. There must be at least six trailing Xs 142 .TP 143 .BR \-v 144 Show verbose messages 145 .TP 146 .BR \-1 147 Display any messages printed to stdout during execution of scenario steps 148 .TP 149 .BR \-2 150 Display any messages printed to stderr during execution of scenario steps 151 .SH "SEE ALSO" 152 yarn(1) 153 .SH COMPATIBILITY 154 tyarn aims to be faithful to yarn, but there are some differences: 155 .IP \(bu 156 tyarn requires statements to be indented with four spaces, 157 yarn allows tab or space indentation. 158 .IP \(bu 159 yarn supports Perl Compatible Regular Expressions (PCREs), tyarn supports 160 POSIX extended regular expressions only. 161 .IP \(bu 162 yarn treats whitespace differently to tyarn. tyarn will 163 ignore surplus whitespace between a step keyword and the text 164 following it i.e. in 'GIVEN foo' the amount of space between 'GIVEN' and 'foo' 165 is ignored. The text following the initial whitespace is always preserved, 166 whereas yarn normalises whitespace for the entire line. 167 .IP \(bu 168 yarn allows implementations and scenarios to be written in the same file, 169 tyarn requires that implementations and scenarios be specified in 170 separate files. In addition only a single scenario file can be supplied to 171 tyarn, whereas yarn will execute scenarios from multiple files in a single 172 invocation. 173 .SH NOTES 174 yarn(1) refers to the testing tool written by Lars Wirzenius, not the package 175 manager, the testing tool in fact predates the package manager. See 176 http://git.liw.fi/cgi-bin/cgit/cgit.cgi/cmdtest/tree/README.yarn for 177 more information about the original yarn(1) tool that tyarn is based on.