The following command-line options are available when running tests:
---verbose::
- This makes the test more verbose. Specifically, the
- command being run and their output if any are also
- output.
-
--debug::
This may help the person who is developing a new test.
It causes the command defined with test_debug to run.
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
+--root=<dir>::
+ This runs the testsuites specified under a seperate directory.
+ However, caution is advised, as not all tests are maintained
+ with this relocation in mind, so some tests may behave
+ differently.
+
+ Pointing this argument at a tmpfs filesystem can improve the
+ speed of the test suite for some users.
+
When invoking the test suite via "make test" any of the above options
can be specified as follows:
Writing Tests
-------------
-The test script is written as a shell script. It should start
-with the standard "#!/bin/bash" with copyright notices, and an
+The test script is written as a shell script. It should start with
+the standard "#!/usr/bin/env bash" with copyright notices, and an
assignment to variable 'test_description', like this:
- #!/bin/bash
+ #!/usr/bin/env bash
#
# Copyright (c) 2005 Junio C Hamano
#
- If the script is invoked with command line argument --help
(or -h), it shows the test_description and exits.
- - Creates a temporary directory with default notmuch-config and empty
- mail store. This directory is 'test/tmp.<test-basename>'. The path
- to notmuch-config is exported in NOTMUCH_CONFIG environment
- variable and mail store path is stored in MAIL_DIR variable.
+ - Creates a temporary directory with default notmuch-config and a
+ mail store with a corpus of mail, (initially, 50 early messages
+ sent to the notmuch list). This directory is
+ test/tmp.<test-basename>. The path to notmuch-config is exported in
+ NOTMUCH_CONFIG environment variable and mail store path is stored
+ in MAIL_DIR variable.
- Defines standard test helper functions for your scripts to
use. These functions are designed to make all scripts behave
There are a handful helper functions defined in the test harness
library for your script to use.
- - test_expect_success <message> <script>
+ test_expect_success <message> <script>
This takes two strings as parameter, and evaluates the
<script>. If it yields success, test is considered
successful. <message> should state what it is testing.
- - test_expect_failure <message> <script>
+ test_expect_failure <message> <script>
This is NOT the opposite of test_expect_success, but is used
to mark a test that demonstrates a known breakage. Unlike
success and "still broken" on failure. Failures from these
tests won't cause -i (immediate) to stop.
- - test_begin_subtest <message>
+ test_begin_subtest <message>
Set the test description message for a subsequent test_expect_equal
invocation (see below).
- - test_expect_equal <output> <expected>
+ test_expect_equal <output> <expected>
This is an often-used convenience function built on top of
test_expect_success. It uses the message from the last
will generate a failure and print the difference of the two
strings.
- - test_debug <script>
+ test_expect_equal_file <output> <expected>
+
+ Identical to test_exepect_equal, except that <output> and
+ <expected> are files instead of strings. This is a much more
+ robust method to compare formatted textual information, since it
+ also notices whitespace and closing newline differences.
+
+ test_expect_equal_failure <output> <expected>
+
+ This works similar to test_expect_equal (see above) but is used to
+ mark a test that demonstrates a known breakage, (that is, the
+ author of the test expects "output" and "expected" to differ until
+ the breakage is fixed). See test_expect_failure for details.
+
+ test_debug <script>
This takes a single argument, <script>, and evaluates it only
when the test script is started with --debug command line
argument. This is primarily meant for use during the
development of a new test script.
- - test_done
+ test_emacs <emacs-lisp-expressions>
+
+ This function executes the provided emacs lisp script within
+ emacs. The script can be a sequence of emacs lisp expressions,
+ (that is, they will be evaluated within a progn form). The lisp
+ expressions can call `message' to generate output on stdout to be
+ examined by the calling test script.
+
+ test_done
Your test script must have test_done at the end. Its purpose
is to summarize successes and failures in the test script and
exit with an appropriate error code.
-k
\ No newline at end of file
+
+There are also a number of mail-specific functions which are useful in
+writing tests:
+
+ generate_message
+
+ Generates a message with an optional template. Most tests will
+ actually prefer to call add_message. See below.
+
+ add_message
+
+ Generate a message and add it to the database (by calling "notmuch
+ new"). It is sufficient to simply call add_message with no
+ arguments if you don't care about the content of the message. If
+ more control is needed, arguments can be provide to specify many
+ different header values for the new message. See the documentation
+ within test-lib.sh or refer to many example calls within existing
+ tests.
+
+ add_email_corpus
+
+ This function should be called at the beginning of a test file
+ when a test needs to operate on a non-empty body of messages. It
+ will initialize the mail database to a known state of 50 sample
+ messages, (culled from the early history of the notmuch mailing
+ list).
projects / notmuch / blobdiff