summaryrefslogtreecommitdiff
path: root/test/test-lib.sh
AgeCommit message (Collapse)Author
2017-05-31test: define GMime version dependant breakageDavid Bremner
We have some tests where the gmime 3 behaviour seems like a bug fix, others where it's less clear, so we allow both possibilities.
2017-05-26test-lib.sh: add "atexit" functionalityTomi Ollila
New function at_exit_function registers given function to be called at script termination. Functions so registered are called in the reverse order of their registration; no arguments are passed. Function is called only once; re-adding with function name already registered will remove previous registration. New function rm_exit_function can be used to remove registration. Modules (and possibly test-lib.sh functions) in future commits will register such functions.
2017-03-12fix out of tree testsTomi Ollila
Use $NOTMUCH_SRCDIR/ instead of $TEST_DIRECTORY/../ (in those 2 places) where reference to source directory instead of build directory is required.
2017-03-09test: require test_begin_subtest before test_expect_codeJani Nikula
Unify the subtests by requiring test_begin_subtest before test_expect_code. (Similar change for test_expect_success has already been done.) This increases clarity in the test scripts by having a separate line for the start of the subtest with the heading, and makes it possible to simplify the test infrastructure by making all subtests similar.
2017-03-09test: require test_begin_subtest before test_expect_successJani Nikula
Unify the subtests by requiring test_begin_subtest before test_expect_success. (Similar change for test_expect_code will follow.) This increases clarity in the test scripts by having a separate line for the start of the subtest with the heading, and makes it possible to simplify the test infrastructure by making all subtests similar.
2017-03-09test: drop the implicit prereq check mechanism from test_expect_*Jani Nikula
The only place where we use the implicit prereq check is T000-basic.sh where we check that it works. It's an added complication that we don't use. Remove it. The test_have_prereq function can still be used for the same effect in subtests that use test_begin_subtest. For now, this will make it impossible to have prereqs in one-line subtests that don't require test_begin_subtest. This will be fixed in follow-up work.
2017-03-09test: ensure test_begin_subtest has been called before test_expect_*Jani Nikula
This is the expectation, increase robustness of the test suite by requiring it.
2017-03-09test: only accept short and long options, not silly in-betweensJani Nikula
It's not notmuch style to accept sloppy parameter names.
2017-03-09test: remove unused and no-op --long-tests parameterJani Nikula
It's been unused since its introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: remove unused test_external and test_external_without_stderrJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git"), only causing maintenance burden.
2017-03-09test: remove unused filter functionsJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: remove unused regexp convenience variablesJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: suppress diff for broken test without V=1Jani Nikula
Known broken tests are, well, known broken. Do not print the result diff for them unless V=1 is specified. Now that the test description is printed also when known broken tests fail, the user can also skip to running the individual failing tests.
2017-03-09test: print test description also for failing known broken testsJani Nikula
With the test description, the user can see the test script name, and debug with that alone.
2017-02-28Merge branch 'release'David Bremner
Second gnugpg test suite fix
2017-02-27test: move GNUPGHOME to TEST_TMPDIRDavid Bremner
We already use this directory for dtach sockets, so it makes sense to put gnupg sockets there as well. There doesn't seem to be a clean way to put a fully functional socket in a different location than GNUPGHOME.
2017-02-27Revert "test: use gpgconf --create-socketdir if available"David Bremner
This reverts commit e7b88e8b0a93ca83d807edc00e0c97af54c5b5f1. It turns out that this does not work well in environments without a running systemd (or some other provider of /run/user)
2017-02-26cli/show: list all filenames of a message in the formatted outputJani Nikula
Instead of just having the first filename for the message, list all duplicate filenames of the message as a list in the formatted outputs. This bumps the format version to 3.
2017-02-23test: replaced use of python with $NOTMUCH_PYTHON (twice)Tomi Ollila
$NOTMUCH_PYTHON is sourced from sh.config, configured by ./configure and stated to be used as: "Name of python command to use in configure and the test suite."
2017-02-21Merge branch 'release'David Bremner
Merge changes to use gpgconf --create-socketdir
2017-02-21test: use gpgconf --create-socketdir if availableDavid Bremner
This enables the shortened socket pathes in /run or equivalent. The explicit call to gpgconf is needed for nonstandard GNUPGHOME settings. (amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
2017-01-08test: allow user to choose which gdb to run tests withTomi Ollila
The variable used for selecting gdb is TEST_GDB, consistent with TEST_CC and TEST_EMACS{,CLIENT}.
2016-11-24Merge branch 'release'David Bremner
Initial set of changes for 0.23.3
2016-11-24tests: account for varying-size cryptographic signaturesDaniel Kahn Gillmor
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its signatures, which makes them about 32 octets larger when ascii-armored. This change in size means that the size of the MIME parts will vary depending on the version of gpg that the user has installed. at any rate, the signature part should be non-zero (this is true for basically any MIME part), so we just test for that instead of an exact size.
2016-10-25test/test-lib.sh: execute basename(1)s lazier in test_expect_equal_file ()Tomi Ollila
Moved the 2 basename(1) executions to the test failure branch in test_expect_equal_file (). The output of basename(1) executions in function test_expect_equal_file () are only used when tests fails -- when all tests pass these 2 basename(1) executions are no longer done at all.
2016-10-22test: use vt100 as "smart" terminal for known dumb/unknown terminalsTomi Ollila
Otherwise use whatever user environment has set for TERM so that there is more chance to test on users' actual environments.
2016-10-22test-lib.sh: rename $DTACH_TERM to $SMART_TERMTomi Ollila
2016-09-24test: test-lib.sh: use $BASH instead of ${SHELL-sh} to relaunchTomi Ollila
In case of the test script is to be relaunced under valgrind, or --tee is requested, use the $BASH shell variable to locate the command interpreter. The $SHELL variable is re-set by non-interactive shells so in case the shell uses some other shell (e.g. zsh) for interactive use these bash scripts continue to work.
2016-09-17test: make it possible to have multiple corporaJani Nikula
We largely use the corpus under test/corpus for testing. Unfortunately, many of our tests have grown to depend on having exactly this set of messages, making it hard to add new message files for testing specific cases. We do use a lot of add_message from within the tests, but it's not possible to use that for adding broken messages, and adding several messages at once can get unwieldy. Move the basic corpus under tests/corpora/default, and make it possible to add new, independent corpora along its side. This means tons of renames with a few tweaks to add_email_corpus function in test-lib.sh to let tests specify which corpus to use.
2016-06-13test: factor out some boilerplate from C testsDavid Bremner
The trick of having a common header file doesn't work to share between test scripts, so make an include file in the test directory. The use of #include <notmuch-test.h> looks slightly pretentious, but the include file is not actually in the current (temporary) directory.
2016-06-11test: test_python: set PYTHONPATH to the python execution environmentTomi Ollila
Place PYTHONPATH to the environment when python is executed in a way that current shell environment is not affected. This also allows adding the old value of PYTHONPATH to the end of the new value (otherwise it would have been appended again and again when test_python is called). At the same time, use -B option to avoid writing .pyc files to bindings/python/* (which are not cleared out by distclean). Drop the (unused) prefix code which preserved the original stdout of the python program and opened sys.stdout to OUTPUT. In place of that there is now note how (debug) information can be printed to original stdout.
2016-06-11test: set LD_LIBRARY_PATH early and keep its old contentsTomi Ollila
Previously LD_LIBRARY_PATH was exported (and environment changed) in the middle of test case execution, when a function setting it was called. Previously the old contents of LD_LIBRARY_PATH was lost (if any) when it was re-set and exported. In some systems the old contents of LD_LIBRARY_PATH was needed to e.g. locate suitable gmime library.
2016-06-11test: add function die () and have use of it in add_email_corpus ()Tomi Ollila
Added die() function to test-lib.sh with the following first use of it: If notmuch new fails during email corpus addition the database is most probably inexistent or broken and the added corpus would be unusable while running single tests, giving misleading failures ("only" full 'make test' cleans out old corpus).
2016-06-11test-lib.sh: renamed die...()s to trap...()s and exit...()Tomi Ollila
Now the function names more accurately describes what the functions do.
2016-06-05Use https instead of http where possibleDaniel Kahn Gillmor
Many of the external links found in the notmuch source can be resolved using https instead of http. This changeset addresses as many as i could find, without touching the e-mail corpus or expected outputs found in tests.
2016-05-25CLI: add notmuch-config support for named queriesDavid Bremner
Most of the infrastructure here is general, only the validation/dispatch is hardcoded to a particular prefix. A notable change in behaviour is that notmuch-config now opens the database e.g. on every call to list, which fails with an error message if the database doesn't exit yet.
2016-05-25CLI: add optional config data to dump output.David Bremner
Note that it changes the default dump output format, but doesn't break existing notmuch-restore. It might break user scripts though.
2016-05-19test: copyright information updatesTomi Ollila
Files in test directories had only copyright of a single individual, of which code was adapted here as a base of the test system. Since then many Notmuch Developers have contributed to the test system, which is now acknowledged with a constant string in some of the test files. The README file in test directory instructed new files contain a copyright notice, but that has never been done (and it is also not needed). To simplify things a bit (and lessen confusion) this instruction is now removed. As a side enchangement, all of the 3 entries in the whole source tree cd'ing to `dirname` of "$0" now uses syntax cd "$(dirname "$0")". This makes these particular lines work when current working directory is e.g. /c/Program Files/notmuch/test/. (Probably it would fail elsewhere, though.)
2016-05-13lib/cli: add library API / CLI for compile time optionsDavid Bremner
This is intentionally low tech; if we have more than two options it may make sense to build up what infrastructure is provided.
2016-04-12configure: add test for default xapian backendDavid Bremner
This is mainly for the test suite. We already expect the tests to be run in the same environment as configure was run, at least to get the name of the python interpreter. So we are not really imposing a new restriction.
2016-01-29test: add broken S/MIME signature verification test for notmuch CLIDavid Bremner
The test is pretty much cut and paste from the PGP/MIME version, with obvious updates taken from notmuch output. This also requires setting up gpgsm infrastucture.
2016-01-29test: initial tests for S/MIME and notmuch-emacsDavid Bremner
Test the ability of notmuch-mua-mail to send S/MIME signed (and encrypted) messages; this really relies on existing functionality in message-mode. The generated keys and messages will later be useful for testing the notmuch CLI.
2016-01-26test: refactor directory name sanitizationDavid Bremner
test_C and notmuch_search_files_sanitize were giving different output on the same path, which is not technically wrong, but slightly confusing.
2015-12-30test: Unset ALTERNATE_EDITOR before running emacsclientMichal Sojka
ALTERNATE_EDITOR causes emacsclient to run an alternate editor if the emacs server is not ready. This can collide with intended functionality in test-lib.sh. If the ALTERNATE_EDITOR is set but empty, emacsclient runs emacs daemon and tries to connect to it. When this happens the emacs run by test-lib.sh fails to start the server and the subsequent attempts to use the server fail because the daemon started by emacsclient does not know about notmuch-test-progn. This leads to test suite failure due to time out on any emacs test.
2015-08-13lib: API to retrieve database revision and UUIDAustin Clements
This exposes the committed database revision to library users along with a UUID that can be used to detect when revision numbers are no longer comparable (e.g., because the database has been replaced).
2015-08-07test: make script exit (1) if it "fails" to source (.) a fileTomi Ollila
The files (test) scripts source (with builtin command `.`) provides information which the scripts depend, and without the `source` to succeed allowing script to continue may lead to dangerous situations (e.g. rm -rf "${undefined_variable}"/*). At the end of all source (.) lines construct ' || exit 1' was added; In our case the script script will exit if it cannot find (or read) the file to be sourced. Additionally script would also exits if the last command of the sourced file exited nonzero.
2015-08-07emacs: make modifications to message Fcc vars buffer-localDavid Bremner
Previously we globally modified these variables, which tended to cause problems for people using message-mode, but not notmuch-mua-mail, to send mail. User visible changes: - Calling notmuch-fcc-header-setup is no longer optional. OTOH, it seems to do the right thing if notmuch-fcc-dirs is set to nil. - The Fcc header is visible during message composition - The name in the mode line is changed, and no longer matches exactly the menu label. - Previously notmuch-mua-send-and-exit was never called. Either we misunderstood define-mail-user-agent, or it had a bug. So there was no difference if the user called message-send-and-exit directly. Now there will be. - User bindings to C-c C-c and C-c C-s in message-mode-map are overridden. The user can override them in notmuch-message-mode-map, but then they're on their own for Fcc handling.
2015-08-04test: use the python interpreter in sh.configDavid Bremner
The configure script chooses "python" if both python and python{2,3} exist exists, so this could change the version of python used to run the test suite. The checking for ${NOTMUCH_PYTHON} in the test suite is arguably over-engineering, since the configure step will fail if it can't find it.
2015-06-14test: add initial ruby testsDavid Bremner
This is pretty much a line by line translation of the existing python tests, with two new tests for the count API.
2015-05-31cli: change "setup" to "set up" where used as a verbJ. Lewis Muir
The word "setup" is a noun, not a verb. Change occurrences of "setup" where used as a verb to "set up".