Pointing this argument at a tmpfs filesystem can improve the
speed of the test suite for some users.
+Certain tests require precomputed databases to complete. You can fetch these
+databases with
+
+ make download-test-databases
+
+If you do not download the test databases, the relevant tests will be
+skipped.
+
When invoking the test suite via "make test" any of the above options
can be specified as follows:
TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient ./emacs
make test TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient
+Some tests may require a c compiler. You can choose the name and flags similarly
+to with emacs, e.g.
+
+ make test TEST_CC=gcc TEST_CFLAGS="-g -O2"
+
+Quiet Execution
+---------------
+
+Normally, when new script starts and when test PASSes you get a message
+printed on screen. This printing can be disabled by setting the
+NOTMUCH_TEST_QUIET variable to a non-null value. Message on test
+failures and skips are still printed.
+
Skipping Tests
--------------
If, for any reason, you need to skip one or more tests, you can do so
items, so you cannot arbitrarily skip any test and expect the
remaining tests to be unaffected.
+Currently we do not consider skipped tests as build failures. For
+maximum robustness, when setting up automated build processes, you
+should explicitely skip tests, rather than relying on notmuch's
+detection of missing prerequisites. In the future we may treat tests
+unable to run because of missing prerequisites, but not explicitely
+skipped by the user, as failures.
+
Writing Tests
-------------
The test script is written as a shell script. It should start with
After assigning test_description, the test script should source
test-lib.sh like this:
- . ./test-lib.sh
+ . ./test-lib.sh || exit 1
This test harness library does the following things:
test_expect_equal_file <file1> <file2>
- Identical to test_exepect_equal, except that <file1> and <file2>
+ Identical to test_expect_equal, except that <file1> and <file2>
are files instead of strings. This is a much more robust method to
compare formatted textual information, since it also notices
whitespace and closing newline differences.
+ test_expect_equal_json <output> <expected>
+
+ Identical to test_expect_equal, except that the two strings are
+ treated as JSON and canonicalized before equality testing. This is
+ useful to abstract away from whitespace differences in the expected
+ output and that generated by running a notmuch command.
+
test_debug <script>
This takes a single argument, <script>, and evaluates it only
generated script that should be called instead of notmuch to do
the counting. The notmuch_counter_value() function prints the
current counter value.
+
+There are also functions which remove various environment-dependent
+values from notmuch output; these are useful to ensure that test
+results remain consistent across different machines.
+
+ notmuch_search_sanitize
+ notmuch_show_sanitize
+ notmuch_show_sanitize_all
+ notmuch_json_show_sanitize
+
+ All these functions should receive the text to be sanitized as the
+ input of a pipe, e.g.
+ output=`notmuch search "..." | notmuch_search_sanitize`