- gdb(1)
- gpg(1)
- python(1)
+ - xapian-metadata(1)
-If your system lacks these tools or have older, non-upgreable versions
+If your system lacks these tools or have older, non-upgradable versions
of these, please (possibly compile and) install these to some other
path, for example /usr/local/bin or /opt/gnu/bin. Then prepend the
chosen directory to your PATH before running the tests.
e.g. env PATH=/opt/gnu/bin:$PATH make test
+For FreeBSD you need to install latest gdb from ports or packages and
+provide path to it in TEST_GDB environment variable before executing
+the tests, native FreeBSD gdb does not not work. If you install
+coreutils, which provides GNU versions of basic utils like 'date' and
+'base64' on FreeBSD, the test suite will use these instead of the
+native ones. This provides robustness against portability issues with
+these system tools. Most often the tests are written, reviewed and
+tested on Linux system so such portability issues arise from time to
+time.
+
Running Tests
-------------
The easiest way to run tests is to say "make test", (or simply run the
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
---root=<dir>::
- This runs the testsuites specified under a separate directory.
- However, caution is advised, as not all tests are maintained
- with this relocation in mind, so some tests may behave
- differently.
-
- Pointing this argument at a tmpfs filesystem can improve the
- speed of the test suite for some users.
-
-Certain tests require precomputed databases to complete. You can fetch these
-databases with
-
- make download-test-databases
-
-If you do not download the test databases, the relevant tests will be
-skipped.
-
When invoking the test suite via "make test" any of the above options
can be specified as follows:
make test TEST_CC=gcc TEST_CFLAGS="-g -O2"
+Parallel Execution
+------------------
+If either the moreutils or GNU "parallel" utility is available all
+tests will be run in parallel. If the NOTMUCH_TEST_SERIALIZE variable
+is non-null all tests will be executed sequentially.
+
Quiet Execution
---------------
-
Normally, when new script starts and when test PASSes you get a message
printed on screen. This printing can be disabled by setting the
NOTMUCH_TEST_QUIET variable to a non-null value. Message on test
Currently we do not consider skipped tests as build failures. For
maximum robustness, when setting up automated build processes, you
-should explicitely skip tests, rather than relying on notmuch's
+should explicitly skip tests, rather than relying on notmuch's
detection of missing prerequisites. In the future we may treat tests
-unable to run because of missing prerequisites, but not explicitely
+unable to run because of missing prerequisites, but not explicitly
skipped by the user, as failures.
+Testing installed notmuch
+-------------------------
+
+Systems integrators (e.g. Linux distros) may wish to test an installed
+version of notmuch. This can be done be running
+
+ $ NOTMUCH_TEST_INSTALLED=1 ./test/notmuch-test
+
+In this scenario the test suite does not assume a built tree, and in
+particular cannot rely on the output of 'configure'. You may want to
+set certain feature environment variables ('NOTMUCH_HAVE_*') directly
+if you know those apply to your installed notmuch). Consider also
+setting TERM=dumb if the value of TERM cannot be used (e.g. in a
+chroot with missing terminfo). Note that having a built tree may cause
+surprising/broken results for NOTMUCH_TEST_INSTALLED, so consider
+cleaning first.
+
Writing Tests
-------------
The test script is written as a shell script. It is to be named as
There are a handful helper functions defined in the test harness
library for your script to use.
- test_expect_success <message> <script>
+ test_begin_subtest <message>
- This takes two strings as parameter, and evaluates the
+ Set the test description message for a subsequent test_expect_*
+ invocation (see below).
+
+ test_expect_success <script>
+
+ This takes a string as parameter, and evaluates the
<script>. If it yields success, test is considered
- successful. <message> should state what it is testing.
+ successful.
- test_begin_subtest <message>
+ test_expect_code <code> <script>
- Set the test description message for a subsequent test_expect_equal
- invocation (see below).
+ This takes two strings as parameter, and evaluates the <script>.
+ If it yields <code> exit status, test is considered successful.
test_subtest_known_broken
projects / notmuch / blobdiff