test: document test_expect_equal_file

[notmuch] / test / README

diff --git a/test/README b/test/README
index 50b3acd236989c95189a87ca73f3d890e07583ef..f9ac6073d97b4739c80b8ba5ec6da00a88d0e154 100644 (file)
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when running tests:
        As the names depend on the tests' file names, it is safe to
        run the tests with this option in parallel.
 
        As the names depend on the tests' file names, it is safe to
        run the tests with this option in parallel.
 

+--root=<dir>::
+       This runs the testsuites specified under a seperate directory.
+       However, caution is advised, as not all tests are maintained
+       with this relocation in mind, so some tests may behave
+       differently.
+
+       Pointing this argument at a tmpfs filesystem can improve the
+       speed of the test suite for some users.
+

 When invoking the test suite via "make test" any of the above options
 can be specified as follows:
 
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:
 


@@ -68,11 +77,11 @@ remaining tests to be unaffected.
 
 Writing Tests
 -------------
 
 Writing Tests
 -------------

-The test script is written as a shell script.  It should start
-with the standard "#!/bin/bash" with copyright notices, and an
+The test script is written as a shell script.  It should start with
+the standard "#!/usr/bin/env bash" with copyright notices, and an

 assignment to variable 'test_description', like this:
 
 assignment to variable 'test_description', like this:
 

-       #!/bin/bash
+       #!/usr/bin/env bash

        #
        # Copyright (c) 2005 Junio C Hamano
        #
        #
        # Copyright (c) 2005 Junio C Hamano
        #


@@ -94,10 +103,12 @@ This test harness library does the following things:
  - If the script is invoked with command line argument --help
    (or -h), it shows the test_description and exits.
 
  - If the script is invoked with command line argument --help
    (or -h), it shows the test_description and exits.
 

- - Creates a temporary directory with default notmuch-config and empty
-   mail store. This directory is 'test/tmp.<test-basename>'. The path
-   to notmuch-config is exported in NOTMUCH_CONFIG environment
-   variable and mail store path is stored in MAIL_DIR variable.
+ - Creates a temporary directory with default notmuch-config and a
+   mail store with a corpus of mail, (initially, 50 early messages
+   sent to the notmuch list). This directory is
+   test/tmp.<test-basename>. The path to notmuch-config is exported in
+   NOTMUCH_CONFIG environment variable and mail store path is stored
+   in MAIL_DIR variable.

 
  - Defines standard test helper functions for your scripts to
    use.  These functions are designed to make all scripts behave
 
  - Defines standard test helper functions for your scripts to
    use.  These functions are designed to make all scripts behave


@@ -115,13 +126,13 @@ Test harness library
 There are a handful helper functions defined in the test harness
 library for your script to use.
 
 There are a handful helper functions defined in the test harness
 library for your script to use.
 

- - test_expect_success <message> <script>
+ test_expect_success <message> <script>

 
    This takes two strings as parameter, and evaluates the
    <script>.  If it yields success, test is considered
    successful.  <message> should state what it is testing.
 
 
    This takes two strings as parameter, and evaluates the
    <script>.  If it yields success, test is considered
    successful.  <message> should state what it is testing.
 

- - test_expect_failure <message> <script>
+ test_expect_failure <message> <script>

 
    This is NOT the opposite of test_expect_success, but is used
    to mark a test that demonstrates a known breakage.  Unlike
 
    This is NOT the opposite of test_expect_success, but is used
    to mark a test that demonstrates a known breakage.  Unlike


@@ -130,12 +141,12 @@ library for your script to use.
    success and "still broken" on failure.  Failures from these
    tests won't cause -i (immediate) to stop.
 
    success and "still broken" on failure.  Failures from these
    tests won't cause -i (immediate) to stop.
 

- - test_begin_subtest <message>
+ test_begin_subtest <message>

 
    Set the test description message for a subsequent test_expect_equal
    invocation (see below).
 
 
    Set the test description message for a subsequent test_expect_equal
    invocation (see below).
 

- - test_expect_equal <output> <expected>
+ test_expect_equal <output> <expected>

 
    This is an often-used convenience function built on top of
    test_expect_success. It uses the message from the last
 
    This is an often-used convenience function built on top of
    test_expect_success. It uses the message from the last


@@ -145,16 +156,63 @@ library for your script to use.
    will generate a failure and print the difference of the two
    strings.
 
    will generate a failure and print the difference of the two
    strings.
 

- - test_debug <script>
+ test_expect_equal_file <output> <expected>
+
+   Identical to test_exepect_equal, except that <output> and
+   <expected> are files instead of strings.  This is a much more
+   robust method to compare formatted textual information, since it
+   also notices whitespace and closing newline differences.
+
+ test_expect_equal_failure <output> <expected>
+
+   This works similar to test_expect_equal (see above) but is used to
+   mark a test that demonstrates a known breakage, (that is, the
+   author of the test expects "output" and "expected" to differ until
+   the breakage is fixed). See test_expect_failure for details.
+
+ test_debug <script>

 
    This takes a single argument, <script>, and evaluates it only
    when the test script is started with --debug command line
    argument.  This is primarily meant for use during the
    development of a new test script.
 
 
    This takes a single argument, <script>, and evaluates it only
    when the test script is started with --debug command line
    argument.  This is primarily meant for use during the
    development of a new test script.
 

- - test_done
+ test_emacs <emacs-lisp-expressions>
+
+   This function executes the provided emacs lisp script within
+   emacs. The script can be a sequence of emacs lisp expressions,
+   (that is, they will be evaluated within a progn form). The lisp
+   expressions can call `message' to generate output on stdout to be
+   examined by the calling test script.
+
+ test_done

 
    Your test script must have test_done at the end.  Its purpose
    is to summarize successes and failures in the test script and
    exit with an appropriate error code.
 
    Your test script must have test_done at the end.  Its purpose
    is to summarize successes and failures in the test script and
    exit with an appropriate error code.

-k
\ No newline at end of file
+
+There are also a number of mail-specific functions which are useful in
+writing tests:
+
+  generate_message
+
+    Generates a message with an optional template. Most tests will
+    actually prefer to call add_message. See below.
+
+  add_message
+
+    Generate a message and add it to the database (by calling "notmuch
+    new"). It is sufficient to simply call add_message with no
+    arguments if you don't care about the content of the message. If
+    more control is needed, arguments can be provide to specify many
+    different header values for the new message. See the documentation
+    within test-lib.sh or refer to many example calls within existing
+    tests.
+
+  add_email_corpus
+
+    This function should be called at the beginning of a test file
+    when a test needs to operate on a non-empty body of messages. It
+    will initialize the mail database to a known state of 50 sample
+    messages, (culled from the early history of the notmuch mailing
+    list).