<feed xmlns='http://www.w3.org/2005/Atom'>
<title>notmuch/test/README, branch 0.21_rc2</title>
<subtitle>thread-based email index, search, and tagging</subtitle>
<id>https://git.notmuchmail.org/git/notmuch/atom?h=0.21_rc2</id>
<link rel='self' href='https://git.notmuchmail.org/git/notmuch/atom?h=0.21_rc2'/>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/'/>
<updated>2015-09-25T12:18:22Z</updated>
<entry>
<title>test suite: don't consider skipped individual tests as failing</title>
<updated>2015-09-25T12:18:22Z</updated>
<author>
<name>David Bremner</name>
<email>david@tethera.net</email>
</author>
<published>2015-09-24T11:13:30Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=3cf7ed26c06fb3fa7145948fd9a9f2973037a5fd'/>
<id>urn:sha1:3cf7ed26c06fb3fa7145948fd9a9f2973037a5fd</id>
<content type='text'>
It isn't completely clear what we want to do here, but

1) We currently don't fail if we skip a whole test file (mainly because
we neglect to count those skipped tests properly). This change at least
makes the two kinds of skipping consistent.

2) Automated build environments may have good reasons for building with
a minimal set of prereqs, and we don't want to discourage running our
test suite by breaking builds.
</content>
</entry>
<entry>
<title>test: make script exit (1) if it "fails" to source (.) a file</title>
<updated>2015-08-07T19:56:39Z</updated>
<author>
<name>Tomi Ollila</name>
<email>tomi.ollila@iki.fi</email>
</author>
<published>2015-08-06T09:13:36Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=02a2eeb427d6b424029f6e5e5ddad4c6ec987741'/>
<id>urn:sha1:02a2eeb427d6b424029f6e5e5ddad4c6ec987741</id>
<content type='text'>
The files (test) scripts source (with builtin command `.`) provides
information which the scripts depend, and without the `source` to
succeed allowing script to continue may lead to dangerous situations
(e.g. rm -rf "${undefined_variable}"/*).

At the end of all source (.) lines construct ' || exit 1' was added;
In our case the script script will exit if it cannot find (or read) the
file to be sourced. Additionally script would also exits if the last
command of the sourced file exited nonzero.
</content>
</entry>
<entry>
<title>test: add support for compiling and running C snippets</title>
<updated>2015-03-28T23:33:56Z</updated>
<author>
<name>David Bremner</name>
<email>david@tethera.net</email>
</author>
<published>2015-03-14T08:02:03Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=1e69afe8af3512c45061cfde99c9f98e1f94e20c'/>
<id>urn:sha1:1e69afe8af3512c45061cfde99c9f98e1f94e20c</id>
<content type='text'>
This is to limit the copy-pasta involved in running C tests. I decided
to keep things simple and not try to provide an actual C skeleton.

The setting of LD_LIBRARY_PATH is to force using the built libnotmuch
rather than any potential system one.
</content>
</entry>
<entry>
<title>test: add machinery to download and verify databases</title>
<updated>2014-03-11T22:51:22Z</updated>
<author>
<name>David Bremner</name>
<email>david@tethera.net</email>
</author>
<published>2014-03-01T20:12:44Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=8413582b6edc859285d68c9f7ad83236788567c0'/>
<id>urn:sha1:8413582b6edc859285d68c9f7ad83236788567c0</id>
<content type='text'>
Note that it is intentional that the checksum file is not
downloaded. The intent is to check those into git.
</content>
</entry>
<entry>
<title>test: implement and document NOTMUCH_TEST_QUIET variable usage</title>
<updated>2013-12-09T15:29:11Z</updated>
<author>
<name>Tomi Ollila</name>
<email>tomi.ollila@iki.fi</email>
</author>
<published>2013-12-05T12:46:00Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=88e6a2995a0108f6ceb94924bd7e6c3623c43dec'/>
<id>urn:sha1:88e6a2995a0108f6ceb94924bd7e6c3623c43dec</id>
<content type='text'>
When NOTMUCH_TEST_QUIET environment variable is set to non-null value
messages when new test script starts and when test PASSes are disabled.
This eases picking the cases when tests FAIL (as those are still printed).
</content>
</entry>
<entry>
<title>test/README: mention the test_expect_equal_json and *sanitize* functions</title>
<updated>2013-03-07T13:38:26Z</updated>
<author>
<name>Aaron Ecay</name>
<email>aaronecay@gmail.com</email>
</author>
<published>2013-03-06T03:34:42Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=ed5150d2df65ed7d7db94a5f8ed43ec0b9b5104d'/>
<id>urn:sha1:ed5150d2df65ed7d7db94a5f8ed43ec0b9b5104d</id>
<content type='text'>
also fix one typo
</content>
</entry>
<entry>
<title>test: Make the emacsclient binary user-configurable</title>
<updated>2012-11-29T13:24:12Z</updated>
<author>
<name>Austin Clements</name>
<email>amdragon@MIT.EDU</email>
</author>
<published>2012-11-28T03:24:59Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=d59d9c81522d9127dde089ae9457f44de53f28e9'/>
<id>urn:sha1:d59d9c81522d9127dde089ae9457f44de53f28e9</id>
<content type='text'>
And require that if TEST_EMACS is specified, so is TEST_EMACSCLIENT.

Previously, the test framework always used "emacsclient", even if the
Emacs in use was overridden by TEST_EMACS.  This causes problems if
both Emacs 23 and Emacs 24 are installed, the Emacs 23 emacsclient is
the system default, but TEST_EMACS is set to emacs24.  Specifically,
with an Emacs 24 server and an Emacs 23 client, emacs tests that run
very quickly may produce no output from emacsclient, causing the test
to fail.

The Emacs server uses a very simple line-oriented protocol in which
the client sends a request to evaluate an expression and the server
sends a request to print the result of evaluation.  Prior to Emacs bzr
commit 107565 on March 11th, 2012 (released in Emacs 24.1), if
multiple commands were sent to the emacsclient between when it sent
the evaluation command and when it entered its receive loop, it would
only process the first response command, ignoring the rest of the
received buffer.  This wasn't a problem with the Emacs 23 server
because it sent only the command to print the evaluation result.
However, the Emacs 24 server first sends an unprompted command
specifying the PID of the Emacs server, then processes the evaluation
request, then sends the command to print the result.  If the
evaluation is fast enough, it can send both of these commands before
emacsclient enters the receive loop.  Hence, if an Emacs 24 server is
used with an Emacs 23 emacsclient, it may miss the response printing
command, ultimately causing intermittent notmuch test failures.
</content>
</entry>
<entry>
<title>Fixed 2 misspellings of word 'separate'</title>
<updated>2012-11-16T19:50:55Z</updated>
<author>
<name>Tomi Ollila</name>
<email>tomi.ollila@iki.fi</email>
</author>
<published>2012-11-16T18:18:14Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=3977b250cd317d2fe8b607fc274fb09f7a9d7cf5'/>
<id>urn:sha1:3977b250cd317d2fe8b607fc274fb09f7a9d7cf5</id>
<content type='text'>
"Seperate: The second most common misspelling on the Internet..."
(from http://www.re-vision.com/spelling/separate.html).

s/seperate/separate/ in debian/NEWS.Debian &amp; test/README done.
</content>
</entry>
<entry>
<title>test: make test_expect_equal_file() arguments flexible</title>
<updated>2012-09-02T02:09:26Z</updated>
<author>
<name>Dmitry Kurochkin</name>
<email>dmitry.kurochkin@gmail.com</email>
</author>
<published>2012-02-01T07:19:54Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=1ffb38296121f8cdec3f4193b8f3d31495a652ec'/>
<id>urn:sha1:1ffb38296121f8cdec3f4193b8f3d31495a652ec</id>
<content type='text'>
Before the change, test_expect_equal_file() function treated the first
argument as "actual output file" and the second argument as "expected
output file".  When the test fails, the files are copied for later
inspection.  The first files was copied to "$testname.output" and the
second file to "$testname.expected".  The argument order for
test_expect_equal_file() is often wrong which results in confusing
diff output and incorrectly named files.

The patch solves the issue by changing test_expect_equal_file() to
treat arguments just as two files, without any special properties
(like "actual" and "expected").  The file names for copying is now
based on the given file name: "$testname.$file1" and
"$testname.$file2".  E.g. if test_expect_equal_file() is called with
"OUTPUT" and "EXPECTED", the copied files can be named
"emacs.1.OUTPUT" and "emacs.1.EXPECTED".

The down side of this approach is that diff argument order depends on
test_expect_equal_file() argument order.  So sometimes we get diff
from expected to actual results, and sometimes the other way around.
But the files are always named correctly.
</content>
</entry>
<entry>
<title>test: Add `test_emacs_expect_t'.</title>
<updated>2012-01-25T11:25:44Z</updated>
<author>
<name>David Edmondson</name>
<email>dme@dme.org</email>
</author>
<published>2012-01-24T16:14:05Z</published>
<link rel='alternate' type='text/html' href='https://git.notmuchmail.org/git/notmuch/commit/?id=260975e8aff635b6b99db835ee8d40ec33ee916c'/>
<id>urn:sha1:260975e8aff635b6b99db835ee8d40ec33ee916c</id>
<content type='text'>
Add a new test function to allow simpler testing of emacs
functionality.

`test_emacs_expect_t' takes one argument - a lisp expression to
evaluate. The test passes if the expression returns `t', otherwise it
fails and the output is reported to the tester.
</content>
</entry>
</feed>
