notmuch/test/README, branch 0.16_rc2

test/README: mention the test_expect_equal_json and sanitize functions

2013-03-07T13:38:26Z

also fix one typo

test: Make the emacsclient binary user-configurable

2012-11-29T13:24:12Z

And require that if TEST_EMACS is specified, so is TEST_EMACSCLIENT. Previously, the test framework always used "emacsclient", even if the Emacs in use was overridden by TEST_EMACS. This causes problems if both Emacs 23 and Emacs 24 are installed, the Emacs 23 emacsclient is the system default, but TEST_EMACS is set to emacs24. Specifically, with an Emacs 24 server and an Emacs 23 client, emacs tests that run very quickly may produce no output from emacsclient, causing the test to fail. The Emacs server uses a very simple line-oriented protocol in which the client sends a request to evaluate an expression and the server sends a request to print the result of evaluation. Prior to Emacs bzr commit 107565 on March 11th, 2012 (released in Emacs 24.1), if multiple commands were sent to the emacsclient between when it sent the evaluation command and when it entered its receive loop, it would only process the first response command, ignoring the rest of the received buffer. This wasn't a problem with the Emacs 23 server because it sent only the command to print the evaluation result. However, the Emacs 24 server first sends an unprompted command specifying the PID of the Emacs server, then processes the evaluation request, then sends the command to print the result. If the evaluation is fast enough, it can send both of these commands before emacsclient enters the receive loop. Hence, if an Emacs 24 server is used with an Emacs 23 emacsclient, it may miss the response printing command, ultimately causing intermittent notmuch test failures.

Fixed 2 misspellings of word 'separate'

2012-11-16T19:50:55Z

"Seperate: The second most common misspelling on the Internet..." (from http://www.re-vision.com/spelling/separate.html). s/seperate/separate/ in debian/NEWS.Debian & test/README done.

test: make test_expect_equal_file() arguments flexible

2012-09-02T02:09:26Z

Before the change, test_expect_equal_file() function treated the first argument as "actual output file" and the second argument as "expected output file". When the test fails, the files are copied for later inspection. The first files was copied to "$testname.output" and the second file to "$testname.expected". The argument order for test_expect_equal_file() is often wrong which results in confusing diff output and incorrectly named files. The patch solves the issue by changing test_expect_equal_file() to treat arguments just as two files, without any special properties (like "actual" and "expected"). The file names for copying is now based on the given file name: "$testname.$file1" and "$testname.$file2". E.g. if test_expect_equal_file() is called with "OUTPUT" and "EXPECTED", the copied files can be named "emacs.1.OUTPUT" and "emacs.1.EXPECTED". The down side of this approach is that diff argument order depends on test_expect_equal_file() argument order. So sometimes we get diff from expected to actual results, and sometimes the other way around. But the files are always named correctly.

test: Add `test_emacs_expect_t'.

2012-01-25T11:25:44Z

Add a new test function to allow simpler testing of emacs functionality. `test_emacs_expect_t' takes one argument - a lisp expression to evaluate. The test passes if the expression returns `t', otherwise it fails and the output is reported to the tester.

Document external dependencies in the test suite

2012-01-21T12:28:47Z

Add an explicit note to the README explaining what programs are necessary and the perhaps-surprising behavior of skipping tests if they aren't present. Signed-off-by: Ethan Glasser-Camp

test: allow user to choose which emacs to run tests with.

2011-12-28T19:50:04Z

As we start to pay more attention to emacs24, it helps to be able to select a different version of emacs to run the tests with to verify version specific bugs. A separate variable TEST_EMACS is needed to avoid being overwritten by the make variable EMACS in Makefile.config For what it's worth, the value of emacs is chosen at the time tmp.emacs/run_emacs is created, so is fixed for all subtests.

test: add functions to count how much times notmuch was called

2011-12-08T00:03:00Z

The patch adds two auxiliary functions and a variable: notmuch_counter_reset $notmuch_counter_command notmuch_counter_value They allow to count how many times notmuch binary is called. notmuch_counter_reset() function generates a script that counts how many times it is called and resets the counter to zero. The function sets $notmuch_counter_command variable to the path to the generated script that should be called instead of notmuch to do the counting. The notmuch_counter_value() function returns the current counter value.

test: add test-binaries target

2011-12-06T13:37:44Z

The goal here is to have a simple way of making sure all of the binaries needed to run tests are available.

test: remove documentation for test_expect_equal_failure.

2011-09-10T17:54:56Z

The actual function was deleted in commit 4cc67276