summaryrefslogtreecommitdiff
path: root/test/test-lib.sh
AgeCommit message (Collapse)Author
2017-12-08indexing: Change from try_decrypt to decryptDaniel Kahn Gillmor
the command-line interface for indexing (reindex, new, insert) used --try-decrypt; and the configuration records used index.try_decrypt. But by comparison with "show" and "reply", there doesn't seem to be any reason for the "try" prefix. This changeset adjusts the command-line interface and the configuration interface. For the moment, i've left indexopts_{set,get}_try_decrypt alone. The subsequent changeset will address those.
2017-12-04test: unbreak performance testsDavid Bremner
In 8e7fb88237aedea22 Jani replaced the use of $(pwd -P) to find the TEST_DIRECTORY in order to better support out of tree builds. Unfortunately the performance-tests need a different value for the variable and were thus broken. This commit splits out the setting of this variable for the two sets of tests. Performance tests still don't work out of tree, because the handling of the downloaded corpus needs to be updated.
2017-10-21tests: emacs_fcc_message: allow passing --arguments to notmuch newDaniel Kahn Gillmor
Subsequent patches may want to send GNU-style --long-arguments to notmuch new in the test suite, in particular when invoking emacs_fcc_message. This changeset makes that possible.
2017-10-21test: use source path in add_gnupg_homeJani Nikula
Make a distinction between source and build directories.
2017-10-21test: remove --root option and fix TMP_DIRECTORY cleanupJani Nikula
The primary motivation here is to fix TMP_DIRECTORY cleanup prior to running each test when the current working directory is not the test subdirectory. Tests with failures would leave their TMP_DIRECTORY directory behind for debugging, and repeated out-of-tree test runs would have old temp directories. (This lead to e.g. T310-emacs.sh hanging because emacs would prompt for overwriting files.) We remove the likely anyway defunct --root test option while at it, just to be on the safe side when doing 'rm -rf' on the TMP_DIRECTORY.
2017-10-20test: mkdir and cd to $TMP_DIRECTORY instead of $test for testingJani Nikula
$TMP_DIRECTORY is a full path, while $test is not.
2017-10-20test: use source and build paths in test-lib.shJani Nikula
Make a distinction between source and build directories.
2017-10-20test: check for notmuch binary in test-lib.shJani Nikula
Move notmuch executable check into common code. Redundant for notmuch-test runs, but works also for individual tests.
2017-10-14tests: prepare for more crypto tests (using add_gnupg_home)Daniel Kahn Gillmor
Move add_gnupg_home to test-lib.sh to prepare it for reuse.
2017-09-04test: notmuch_drop_mail_headers() style updateTomi Ollila
Changed "" quotes to '' as we're not supposed to dynamically alter python program (via shell $variable expansion). Added space to python program to match general python style. Replaced $* with 'idiomatic' "$@" to serve as better example.
2017-09-03test/crypto: remove headers more robustlyDavid Bremner
In [1], Vladimir Panteleev observed that the In-Reply-To and References headers could be wrapped in the 'default' output format of notmuch-reply, depending on the version of Emacs creating the message. In my own experiments notmuch-reply sometimes wraps headers with only one message-id if that message-id is long enough. However it happens, this causes the previous approach using grep to fail. Since I found the proposed unwrapping shell fragment in [1] a bit hard to follow, I decided to write a little python script instead. Then Tomi suggested a slight generalization of my script, and here we are. [1] id:20170817175145.3204-7-notmuch@thecybershadow.net
2017-08-23test: Use small Python script for JSON normalization instead of json.toolVladimir Panteleev
json.tool does not sort or otherwise normalize the order of JSON keys in its output, which can result in test failures on some test systems. Instead, use a one-line Python script passed to the interpreter directly on its command line. Use sort_keys=True for json.dump to ensure the key order is normalized. The script works with both Python 2 and 3. * test/test-lib.sh: Update test_expect_equal_json.
2017-08-20test: move generate_message, add_message into test-lib-common.shDavid Bremner
The plan is to use at least the former in the perf test suite.
2017-05-31test: define GMime version dependant breakageDavid Bremner
We have some tests where the gmime 3 behaviour seems like a bug fix, others where it's less clear, so we allow both possibilities.
2017-05-26test-lib.sh: add "atexit" functionalityTomi Ollila
New function at_exit_function registers given function to be called at script termination. Functions so registered are called in the reverse order of their registration; no arguments are passed. Function is called only once; re-adding with function name already registered will remove previous registration. New function rm_exit_function can be used to remove registration. Modules (and possibly test-lib.sh functions) in future commits will register such functions.
2017-03-12fix out of tree testsTomi Ollila
Use $NOTMUCH_SRCDIR/ instead of $TEST_DIRECTORY/../ (in those 2 places) where reference to source directory instead of build directory is required.
2017-03-09test: require test_begin_subtest before test_expect_codeJani Nikula
Unify the subtests by requiring test_begin_subtest before test_expect_code. (Similar change for test_expect_success has already been done.) This increases clarity in the test scripts by having a separate line for the start of the subtest with the heading, and makes it possible to simplify the test infrastructure by making all subtests similar.
2017-03-09test: require test_begin_subtest before test_expect_successJani Nikula
Unify the subtests by requiring test_begin_subtest before test_expect_success. (Similar change for test_expect_code will follow.) This increases clarity in the test scripts by having a separate line for the start of the subtest with the heading, and makes it possible to simplify the test infrastructure by making all subtests similar.
2017-03-09test: drop the implicit prereq check mechanism from test_expect_*Jani Nikula
The only place where we use the implicit prereq check is T000-basic.sh where we check that it works. It's an added complication that we don't use. Remove it. The test_have_prereq function can still be used for the same effect in subtests that use test_begin_subtest. For now, this will make it impossible to have prereqs in one-line subtests that don't require test_begin_subtest. This will be fixed in follow-up work.
2017-03-09test: ensure test_begin_subtest has been called before test_expect_*Jani Nikula
This is the expectation, increase robustness of the test suite by requiring it.
2017-03-09test: only accept short and long options, not silly in-betweensJani Nikula
It's not notmuch style to accept sloppy parameter names.
2017-03-09test: remove unused and no-op --long-tests parameterJani Nikula
It's been unused since its introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: remove unused test_external and test_external_without_stderrJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git"), only causing maintenance burden.
2017-03-09test: remove unused filter functionsJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: remove unused regexp convenience variablesJani Nikula
They've been unused since their introduction in commit 0083854b1204 ("Copy test framework from Git").
2017-03-09test: suppress diff for broken test without V=1Jani Nikula
Known broken tests are, well, known broken. Do not print the result diff for them unless V=1 is specified. Now that the test description is printed also when known broken tests fail, the user can also skip to running the individual failing tests.
2017-03-09test: print test description also for failing known broken testsJani Nikula
With the test description, the user can see the test script name, and debug with that alone.
2017-02-28Merge branch 'release'David Bremner
Second gnugpg test suite fix
2017-02-27test: move GNUPGHOME to TEST_TMPDIRDavid Bremner
We already use this directory for dtach sockets, so it makes sense to put gnupg sockets there as well. There doesn't seem to be a clean way to put a fully functional socket in a different location than GNUPGHOME.
2017-02-27Revert "test: use gpgconf --create-socketdir if available"David Bremner
This reverts commit e7b88e8b0a93ca83d807edc00e0c97af54c5b5f1. It turns out that this does not work well in environments without a running systemd (or some other provider of /run/user)
2017-02-26cli/show: list all filenames of a message in the formatted outputJani Nikula
Instead of just having the first filename for the message, list all duplicate filenames of the message as a list in the formatted outputs. This bumps the format version to 3.
2017-02-23test: replaced use of python with $NOTMUCH_PYTHON (twice)Tomi Ollila
$NOTMUCH_PYTHON is sourced from sh.config, configured by ./configure and stated to be used as: "Name of python command to use in configure and the test suite."
2017-02-21Merge branch 'release'David Bremner
Merge changes to use gpgconf --create-socketdir
2017-02-21test: use gpgconf --create-socketdir if availableDavid Bremner
This enables the shortened socket pathes in /run or equivalent. The explicit call to gpgconf is needed for nonstandard GNUPGHOME settings. (amended according to id:m2fujatr4k.fsf@guru.guru-group.fi)
2017-01-08test: allow user to choose which gdb to run tests withTomi Ollila
The variable used for selecting gdb is TEST_GDB, consistent with TEST_CC and TEST_EMACS{,CLIENT}.
2016-11-24Merge branch 'release'David Bremner
Initial set of changes for 0.23.3
2016-11-24tests: account for varying-size cryptographic signaturesDaniel Kahn Gillmor
GnuPG 2.1.16 is now injecting the full issuer fingerprint in its signatures, which makes them about 32 octets larger when ascii-armored. This change in size means that the size of the MIME parts will vary depending on the version of gpg that the user has installed. at any rate, the signature part should be non-zero (this is true for basically any MIME part), so we just test for that instead of an exact size.
2016-10-25test/test-lib.sh: execute basename(1)s lazier in test_expect_equal_file ()Tomi Ollila
Moved the 2 basename(1) executions to the test failure branch in test_expect_equal_file (). The output of basename(1) executions in function test_expect_equal_file () are only used when tests fails -- when all tests pass these 2 basename(1) executions are no longer done at all.
2016-10-22test: use vt100 as "smart" terminal for known dumb/unknown terminalsTomi Ollila
Otherwise use whatever user environment has set for TERM so that there is more chance to test on users' actual environments.
2016-10-22test-lib.sh: rename $DTACH_TERM to $SMART_TERMTomi Ollila
2016-09-24test: test-lib.sh: use $BASH instead of ${SHELL-sh} to relaunchTomi Ollila
In case of the test script is to be relaunced under valgrind, or --tee is requested, use the $BASH shell variable to locate the command interpreter. The $SHELL variable is re-set by non-interactive shells so in case the shell uses some other shell (e.g. zsh) for interactive use these bash scripts continue to work.
2016-09-17test: make it possible to have multiple corporaJani Nikula
We largely use the corpus under test/corpus for testing. Unfortunately, many of our tests have grown to depend on having exactly this set of messages, making it hard to add new message files for testing specific cases. We do use a lot of add_message from within the tests, but it's not possible to use that for adding broken messages, and adding several messages at once can get unwieldy. Move the basic corpus under tests/corpora/default, and make it possible to add new, independent corpora along its side. This means tons of renames with a few tweaks to add_email_corpus function in test-lib.sh to let tests specify which corpus to use.
2016-06-13test: factor out some boilerplate from C testsDavid Bremner
The trick of having a common header file doesn't work to share between test scripts, so make an include file in the test directory. The use of #include <notmuch-test.h> looks slightly pretentious, but the include file is not actually in the current (temporary) directory.
2016-06-11test: test_python: set PYTHONPATH to the python execution environmentTomi Ollila
Place PYTHONPATH to the environment when python is executed in a way that current shell environment is not affected. This also allows adding the old value of PYTHONPATH to the end of the new value (otherwise it would have been appended again and again when test_python is called). At the same time, use -B option to avoid writing .pyc files to bindings/python/* (which are not cleared out by distclean). Drop the (unused) prefix code which preserved the original stdout of the python program and opened sys.stdout to OUTPUT. In place of that there is now note how (debug) information can be printed to original stdout.
2016-06-11test: set LD_LIBRARY_PATH early and keep its old contentsTomi Ollila
Previously LD_LIBRARY_PATH was exported (and environment changed) in the middle of test case execution, when a function setting it was called. Previously the old contents of LD_LIBRARY_PATH was lost (if any) when it was re-set and exported. In some systems the old contents of LD_LIBRARY_PATH was needed to e.g. locate suitable gmime library.
2016-06-11test: add function die () and have use of it in add_email_corpus ()Tomi Ollila
Added die() function to test-lib.sh with the following first use of it: If notmuch new fails during email corpus addition the database is most probably inexistent or broken and the added corpus would be unusable while running single tests, giving misleading failures ("only" full 'make test' cleans out old corpus).
2016-06-11test-lib.sh: renamed die...()s to trap...()s and exit...()Tomi Ollila
Now the function names more accurately describes what the functions do.
2016-06-05Use https instead of http where possibleDaniel Kahn Gillmor
Many of the external links found in the notmuch source can be resolved using https instead of http. This changeset addresses as many as i could find, without touching the e-mail corpus or expected outputs found in tests.
2016-05-25CLI: add notmuch-config support for named queriesDavid Bremner
Most of the infrastructure here is general, only the validation/dispatch is hardcoded to a particular prefix. A notable change in behaviour is that notmuch-config now opens the database e.g. on every call to list, which fails with an error message if the database doesn't exit yet.
2016-05-25CLI: add optional config data to dump output.David Bremner
Note that it changes the default dump output format, but doesn't break existing notmuch-restore. It might break user scripts though.