perf-test: add argument parsing for performance tests

[notmuch] / test / README

diff --git a/test/README b/test/README
index bde6db0c04a4979ea29ac1451e5d0e6b80b8fb50..81c232ddd0a3fc90e1dbaa74f34f1252da004193 100644 (file)
--- a/test/README
+++ b/test/README
@@ -6,6 +6,19 @@ When fixing bugs or enhancing notmuch, you are strongly encouraged to
 add tests in this directory to cover what you are trying to fix or
 enhance.
 
+Prerequisites
+-------------
+Some tests require external dependencies to run. Without them, they
+will be skipped, or (rarely) marked failed. Please install these, so
+that you know if you break anything.
+
+  - dtach(1)
+  - emacs(1)
+  - emacsclient(1)
+  - gdb(1)
+  - gpg(1)
+  - python(1)
+
 Running Tests
 -------------
 The easiest way to run tests is to say "make test", (or simply run the
@@ -43,7 +56,7 @@ The following command-line options are available when running tests:
        run the tests with this option in parallel.
 
 --root=<dir>::
-       This runs the testsuites specified under a seperate directory.
+       This runs the testsuites specified under a separate directory.
        However, caution is advised, as not all tests are maintained
        with this relocation in mind, so some tests may behave
        differently.
@@ -56,12 +69,12 @@ can be specified as follows:
 
        make test OPTIONS="--verbose"
 
-You can choose an emacs binary to run the tests in one of the
-following ways.
+You can choose an emacs binary (and corresponding emacsclient) to run
+the tests in one of the following ways.
 
-       TEST_EMACS=my-special-emacs make test
-       TEST_EMACS=my-special-emacs ./emacs
-       make test TEST_EMACS=my-special-emacs
+       TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient make test
+       TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient ./emacs
+       make test TEST_EMACS=my-special-emacs TEST_EMACSCLIENT=my-emacsclient
 
 Skipping Tests
 --------------
@@ -163,12 +176,12 @@ library for your script to use.
    will generate a failure and print the difference of the two
    strings.
 
- test_expect_equal_file <output> <expected>
+ test_expect_equal_file <file1> <file2>
 
-   Identical to test_exepect_equal, except that <output> and
-   <expected> are files instead of strings.  This is a much more
-   robust method to compare formatted textual information, since it
-   also notices whitespace and closing newline differences.
+   Identical to test_exepect_equal, except that <file1> and <file2>
+   are files instead of strings.  This is a much more robust method to
+   compare formatted textual information, since it also notices
+   whitespace and closing newline differences.
 
  test_debug <script>
 
@@ -189,6 +202,14 @@ library for your script to use.
    tests that may run in the same Emacs instance.  Use `let' instead
    so the scope of the changed variables is limited to a single test.
 
+ test_emacs_expect_t <emacs-lisp-expressions>
+
+  This function executes the provided emacs lisp script within
+  emacs in a manner similar to 'test_emacs'. The expressions should
+  return the value `t' to indicate that the test has passed. If the
+  test does not return `t' then it is considered failed and all data
+  returned by the test is reported to the tester.
+
  test_done
 
    Your test script must have test_done at the end.  Its purpose