aboutsummaryrefslogtreecommitdiff
path: root/test/test-lib.sh
AgeCommit message (Collapse)Author
2017-02-27test: move GNUPGHOME to TEST_TMPDIRDavid Bremner
We already use this directory for dtach sockets, so it makes sense to put gnupg sockets there as well. There doesn't seem to be a clean way to put a fully functional socket in a different location than GNUPGHOME.
2017-02-27Revert "test: use gpgconf --create-socketdir if available"David Bremner
This reverts commit e7b88e8b0a93ca83d807edc00e0c97af54c5b5f1. It turns out that this does not work well in environments without a running systemd (or some other provider of /run/user)
2017-02-21test: use gpgconf --create-socketdir if availableDavid Bremner
This enables the shortened socket pathes in /run or equivalent. The explicit call to gpgconf is needed for nonstandard GNUPGHOME settings. (amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
2016-11-24tests: account for varying-size cryptographic signaturesDaniel Kahn Gillmor
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its signatures, which makes them about 32 octets larger when ascii-armored. This change in size means that the size of the MIME parts will vary depending on the version of gpg that the user has installed. at any rate, the signature part should be non-zero (this is true for basically any MIME part), so we just test for that instead of an exact size.
2016-10-22test: use vt100 as "smart" terminal for known dumb/unknown terminalsTomi Ollila
Otherwise use whatever user environment has set for TERM so that there is more chance to test on users' actual environments.
2016-10-22test-lib.sh: rename $DTACH_TERM to $SMART_TERMTomi Ollila
2016-09-24test: test-lib.sh: use $BASH instead of ${SHELL-sh} to relaunchTomi Ollila
In case of the test script is to be relaunced under valgrind, or --tee is requested, use the $BASH shell variable to locate the command interpreter. The $SHELL variable is re-set by non-interactive shells so in case the shell uses some other shell (e.g. zsh) for interactive use these bash scripts continue to work.
2016-09-17test: make it possible to have multiple corporaJani Nikula
We largely use the corpus under test/corpus for testing. Unfortunately, many of our tests have grown to depend on having exactly this set of messages, making it hard to add new message files for testing specific cases. We do use a lot of add_message from within the tests, but it's not possible to use that for adding broken messages, and adding several messages at once can get unwieldy. Move the basic corpus under tests/corpora/default, and make it possible to add new, independent corpora along its side. This means tons of renames with a few tweaks to add_email_corpus function in test-lib.sh to let tests specify which corpus to use.
2016-06-13test: factor out some boilerplate from C testsDavid Bremner
The trick of having a common header file doesn't work to share between test scripts, so make an include file in the test directory. The use of #include <notmuch-test.h> looks slightly pretentious, but the include file is not actually in the current (temporary) directory.
2016-06-11test: test_python: set PYTHONPATH to the python execution environmentTomi Ollila
Place PYTHONPATH to the environment when python is executed in a way that current shell environment is not affected. This also allows adding the old value of PYTHONPATH to the end of the new value (otherwise it would have been appended again and again when test_python is called). At the same time, use -B option to avoid writing .pyc files to bindings/python/* (which are not cleared out by distclean). Drop the (unused) prefix code which preserved the original stdout of the python program and opened sys.stdout to OUTPUT. In place of that there is now note how (debug) information can be printed to original stdout.
2016-06-11test: set LD_LIBRARY_PATH early and keep its old contentsTomi Ollila
Previously LD_LIBRARY_PATH was exported (and environment changed) in the middle of test case execution, when a function setting it was called. Previously the old contents of LD_LIBRARY_PATH was lost (if any) when it was re-set and exported. In some systems the old contents of LD_LIBRARY_PATH was needed to e.g. locate suitable gmime library.
2016-06-11test: add function die () and have use of it in add_email_corpus ()Tomi Ollila
Added die() function to test-lib.sh with the following first use of it: If notmuch new fails during email corpus addition the database is most probably inexistent or broken and the added corpus would be unusable while running single tests, giving misleading failures ("only" full 'make test' cleans out old corpus).
2016-06-11test-lib.sh: renamed die...()s to trap...()s and exit...()Tomi Ollila
Now the function names more accurately describes what the functions do.
2016-06-05Use https instead of http where possibleDaniel Kahn Gillmor
Many of the external links found in the notmuch source can be resolved using https instead of http. This changeset addresses as many as i could find, without touching the e-mail corpus or expected outputs found in tests.
2016-05-25CLI: add notmuch-config support for named queriesDavid Bremner
Most of the infrastructure here is general, only the validation/dispatch is hardcoded to a particular prefix. A notable change in behaviour is that notmuch-config now opens the database e.g. on every call to list, which fails with an error message if the database doesn't exit yet.
2016-05-25CLI: add optional config data to dump output.David Bremner
Note that it changes the default dump output format, but doesn't break existing notmuch-restore. It might break user scripts though.
2016-05-19test: copyright information updatesTomi Ollila
Files in test directories had only copyright of a single individual, of which code was adapted here as a base of the test system. Since then many Notmuch Developers have contributed to the test system, which is now acknowledged with a constant string in some of the test files. The README file in test directory instructed new files contain a copyright notice, but that has never been done (and it is also not needed). To simplify things a bit (and lessen confusion) this instruction is now removed. As a side enchangement, all of the 3 entries in the whole source tree cd'ing to `dirname` of "$0" now uses syntax cd "$(dirname "$0")". This makes these particular lines work when current working directory is e.g. /c/Program Files/notmuch/test/. (Probably it would fail elsewhere, though.)
2016-05-13lib/cli: add library API / CLI for compile time optionsDavid Bremner
This is intentionally low tech; if we have more than two options it may make sense to build up what infrastructure is provided.
2016-04-12configure: add test for default xapian backendDavid Bremner
This is mainly for the test suite. We already expect the tests to be run in the same environment as configure was run, at least to get the name of the python interpreter. So we are not really imposing a new restriction.
2016-01-29test: add broken S/MIME signature verification test for notmuch CLIDavid Bremner
The test is pretty much cut and paste from the PGP/MIME version, with obvious updates taken from notmuch output. This also requires setting up gpgsm infrastucture.
2016-01-29test: initial tests for S/MIME and notmuch-emacsDavid Bremner
Test the ability of notmuch-mua-mail to send S/MIME signed (and encrypted) messages; this really relies on existing functionality in message-mode. The generated keys and messages will later be useful for testing the notmuch CLI.
2016-01-26test: refactor directory name sanitizationDavid Bremner
test_C and notmuch_search_files_sanitize were giving different output on the same path, which is not technically wrong, but slightly confusing.
2015-12-30test: Unset ALTERNATE_EDITOR before running emacsclientMichal Sojka
ALTERNATE_EDITOR causes emacsclient to run an alternate editor if the emacs server is not ready. This can collide with intended functionality in test-lib.sh. If the ALTERNATE_EDITOR is set but empty, emacsclient runs emacs daemon and tries to connect to it. When this happens the emacs run by test-lib.sh fails to start the server and the subsequent attempts to use the server fail because the daemon started by emacsclient does not know about notmuch-test-progn. This leads to test suite failure due to time out on any emacs test.
2015-08-13lib: API to retrieve database revision and UUIDAustin Clements
This exposes the committed database revision to library users along with a UUID that can be used to detect when revision numbers are no longer comparable (e.g., because the database has been replaced).
2015-08-07test: make script exit (1) if it "fails" to source (.) a fileTomi Ollila
The files (test) scripts source (with builtin command `.`) provides information which the scripts depend, and without the `source` to succeed allowing script to continue may lead to dangerous situations (e.g. rm -rf "${undefined_variable}"/*). At the end of all source (.) lines construct ' || exit 1' was added; In our case the script script will exit if it cannot find (or read) the file to be sourced. Additionally script would also exits if the last command of the sourced file exited nonzero.
2015-08-07emacs: make modifications to message Fcc vars buffer-localDavid Bremner
Previously we globally modified these variables, which tended to cause problems for people using message-mode, but not notmuch-mua-mail, to send mail. User visible changes: - Calling notmuch-fcc-header-setup is no longer optional. OTOH, it seems to do the right thing if notmuch-fcc-dirs is set to nil. - The Fcc header is visible during message composition - The name in the mode line is changed, and no longer matches exactly the menu label. - Previously notmuch-mua-send-and-exit was never called. Either we misunderstood define-mail-user-agent, or it had a bug. So there was no difference if the user called message-send-and-exit directly. Now there will be. - User bindings to C-c C-c and C-c C-s in message-mode-map are overridden. The user can override them in notmuch-message-mode-map, but then they're on their own for Fcc handling.
2015-08-04test: use the python interpreter in sh.configDavid Bremner
The configure script chooses "python" if both python and python{2,3} exist exists, so this could change the version of python used to run the test suite. The checking for ${NOTMUCH_PYTHON} in the test suite is arguably over-engineering, since the configure step will fail if it can't find it.
2015-06-14test: add initial ruby testsDavid Bremner
This is pretty much a line by line translation of the existing python tests, with two new tests for the count API.
2015-05-31cli: change "setup" to "set up" where used as a verbJ. Lewis Muir
The word "setup" is a noun, not a verb. Change occurrences of "setup" where used as a verb to "set up".
2015-03-29test: add support for compiling and running C snippetsDavid Bremner
This is to limit the copy-pasta involved in running C tests. I decided to keep things simple and not try to provide an actual C skeleton. The setting of LD_LIBRARY_PATH is to force using the built libnotmuch rather than any potential system one.
2015-01-19test: prepare test-lib.sh for possible test system debug sessionTomi Ollila
When something in tests fails one possibility to test is to run the test script as `bash -x TXXX-testname.sh`. As stderr (fd 2) was redirected to separate file during test execution also this set -x (xtrace) output would also go there. test-lib.sh saves the stderr to fd 7 from where it can be restored, and bash has BASH_XTRACEFD variable, which is now given the same value 7, making bash to output all xtrade information (consistently) there. This lib file used to save fd's 1 & 2 to 6 & 7 (respectively) in test_begin_subtest(), but as those needs to be set *before* XTRACEFD variable is set those are now saved at the beginning of the lib (once). This is safe and simple thing to do. To make xtrace output more verbose PS4 variable was set to contain the source file, line number and if execution is in function, that function name. Setting this variable has no effect when not xtracing. As it is known that fd 6 is redirected stdout, printing status can now use that fd, instead of saving stdout to fd 5 and use it.
2015-01-17test-lib: Add dummy subject to force empty subjectJesse Rosenthal
At the moment, the test-lib fills in any missing headers. This makes it impossible to test our handling of empty subjects. This will allow us to use a special dummy subject -- `@FORCE_EMPTY` -- to force the subject to remain empty.
2014-10-25Merge tag '0.18.2_rc1'David Bremner
notmuch 0.18.2~rc1 release Conflicts: NEWS debian/changelog
2014-10-25test: kill '"filename": "signature.asc"' from json outputDavid Bremner
This starts to appear with emacs24.4, so we can't easily have it in our expected output.
2014-07-13test: make test_emacs call post-command-hookDavid Bremner
The unread/read changes will use the post-command-hook. test_emacs does not call the post-command-hook. This adds a notmuch-test-progn which takes a list of commands as argument and executes them in turn but runs the post-command-hook after each one. The caller can batch operations (ie to stop post-command-hook from being interleaved) by wrapping the batch of operations inside a progn. We also explicitly run the post-command-hook before getting the output from a test; this makes sense as this will be a place the user would be seeing the information.
2014-06-13test: use --quick when starting emacs.David Bremner
At least in emacs24, this removes the "site-lisp" directories from the load path in addition to enforcing --no-site-lisp --no-init-file. This works around a slightly mysterious bug on Debian that causes test-lib.el not to load when there is cl-lib.el(c) in some site-lisp directory. It should be harmless in general since we really don't want to load any files from addon packages to emacs.
2014-03-11test: add notmuch_search_files_sanitize and use itJani Nikula
We do this in a lot of places, so make it a helper in the test-lib.
2014-03-11test: add utility function to sort a json listDavid Bremner
So far we only need this one place, but it's a bit messy to inline
2014-03-09support for generating decreasing dates in bash 4.0 and 4.1Tomi Ollila
The printf builtin "%(fmt)T" specifier (which allows time values to use strftime-like formatting) is introduced in bash 4.2. Trying to execute this in pre-4.2 bash will fail -- and if this happens execute the fallback piece of perl code to do the same thing.
2014-03-09test: NOTMUCH_SKIP_TESTS accepts test names with or without Tddd- prefixTomi Ollila
The test names assigned to NOTMUCH_SKIP_TESTS variable can now be given with or without the Tddd- prefix for tester convenience: The test name without Tddd -prefix stays constant even when test filenames are renumbered. The test name with Tddd -prefix is printed out when tests run.
2014-03-06test: Print the number of the test along with its nameAustin Clements
Previously, we stripped the "Tnnn-" part from the test name when printing its description at the beginning of each test. However, this makes it difficult to find the source script for a test (e.g., when a test fails). Put this prefix back.
2014-01-18test: leave T\d\d\d- in variable $this_test and introduce $this_test_bareTomi Ollila
Script `notmuch-test` expects the results file have T\d\d\d- part intact so the results files (and some test output files) are now name as such. Without this change `notmuch-test` will exit in case the test script it was executing exited with nonzero value. The T\d\d\d- part is dropped in new variable $this_test_bare which is used in progress informational messages and when loading .el files in emacs tests (whenever $this_test_bare.el exists).
2014-01-13test: renamed test scripts to format T\d\d\d-name.shTomi Ollila
All test scripts to be executed are now named as T\d\d\d-name.sh, numers in increments of 10. This eases adding new tests and developers to see which are test scripts that are executed by test suite and in which order.
2013-12-28Merge tag '0.17_rc4'David Bremner
notmuch 0.17~rc4 release
2013-12-22test: remove call to notmuch-hello from emacs_deliver_messageDavid Bremner
There is an obscure bug in notmuch-hello that very occasionally causes emacs_deliver_message to fail. Since it it doesn't serve any actual purpose in the function we delete it, and leave tracking down the the bug for another day.
2013-12-22test: add emacs_fcc_message that does not use smtp-dummyDavid Bremner
Most of the tests previously using emacs_deliver_message do not use the actual transmitted message, so we replace it with a simpler (and presumably more reliable function) that only saves (and indexes) an fcc copy of the message.
2013-12-09test: implement and document NOTMUCH_TEST_QUIET variable usageTomi Ollila
When NOTMUCH_TEST_QUIET environment variable is set to non-null value messages when new test script starts and when test PASSes are disabled. This eases picking the cases when tests FAIL (as those are still printed).
2013-12-09test: print empty line at the beginning of test script, not at endTomi Ollila
In preparation for quiet mode print empty line before writing the test description. This is done now in function designed for it -- it will also be called when test fails.
2013-12-09test: resolve `basename "$0" .sh` once for all in test-lib.shTomi Ollila
test-lib.sh sometimes did equivalent of `basename "$0" .sh`, sometimes skipping the basename part and sometimes .sh part. This worked as we never had path components in $0 (more than ./) nor .sh ending. Now the equivalent of `basename "$0" .sh` is done once and used everywhere. In the future we may have .sh suffix in test names -- removing those is a good idea.
2013-12-09test: give unique timestamps to messagesDavid Bremner
The choice of decreasing timestamps is a hack which reduces the number of existing tests which fail. This can be changed to increasing if/when somebody wants update another 47 tests.