aboutsummaryrefslogtreecommitdiff
path: root/test
AgeCommit message (Collapse)Author
2019-05-29cli/show: add information about which headers were protectedDaniel Kahn Gillmor
The header-mask member of the per-message crypto object allows a clever UI frontend to mark whether a header was protected (or not). And if it was protected, it contains enough information to show useful detail to an interested user. For example, an MUA could offer a "show what this message's Subject looked like on the wire" feature in expert mode. As before, we only handle Subject for now, but we might be able to handle other headers in the future. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net> Amended by db: tweaked schemata notation.
2019-05-29cli/show: emit payload subject instead of outside subjectDaniel Kahn Gillmor
Correctly fix the two outstanding tests so that the protected (hidden) subject is properly reported. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-29cli/show: add tests for viewing protected headersDaniel Kahn Gillmor
Here we add several variant e-mail messages, some of which have correctly-structured protected headers, and some of which do not. The goal of the tests is to ensure that the right protected subjects get reported. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-29test: new test framework to compare json partsJameson Graef Rollins
This makes it easier to write fairly compact, readable tests of json output, without needing to sanitize away parts that we don't care about. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-29cli/show: emit headers after emitting bodyDaniel Kahn Gillmor
This paves the way for emitting protected headers after verification and decryption, because it means that the headers will only be emitted after the body has been parsed. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-26test: report summary even when abortingDaniel Kahn Gillmor
In certain cases of test suite failure, the summary report was not being printed. In particular, any failure on the parallel test suite, and any aborted test in the serialized test suite would end up hiding the summary. It's better to always show the summary where we can (while preserving the return code). If we do abort due to this high-level failure, though, we should also announce to the user that we're doing so as close to the end of the process as possible, to make it easier to find the problem. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-26cli/show: emit new whole-message crypto status outputDaniel Kahn Gillmor
This allows MUAs that don't want to think about per-mime-part cryptographic status to have a simple high-level overview of the message's cryptographic state. Sensibly structured encrypted and/or signed messages will work fine with this. The only requirement for the simplest encryption + signing is that the message have all of its encryption and signing protection (the "cryptographic envelope") in a contiguous set of MIME layers at the very outside of the message itself. This is because messages with some subparts signed or encrypted, but with other subparts with no cryptographic protection is very difficult to reason about, and even harder for the user to make sense of or work with. For further characterization of the Cryptographic Envelope and some of the usability tradeoffs, see here: https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html#cryptographic-envelope
2019-05-25emacs: Drop content-free "Unknown signature status" buttonDaniel Kahn Gillmor
When we have not been able to evaluate the signature status of a given MIME part, showing a content-free (and interaction-free) "[ Unknown signature status ]" button doesn't really help the user at all, and takes up valuable screen real-estate. A visual reminder that a given message is *not* signed isn't helpful unless it is always present, in which case we'd want to see "[ Unknown signature status ]" buttons on all messages, even ones that don't have a signing structure, but i don't think we want that. Amended by db to drop the unused initialization of 'label'
2019-05-25test: allow disabling timeout with NOTMUCH_TEST_TIMEOUT=0Daniel Kahn Gillmor
To aid in diagnosing test suite tooling that interacts poorly with coreutils' timeout, it's handy to be able to bypass it entirely. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-25lib/database: index user headers.David Bremner
This essentially involves calling _notmuch_message_gen_terms once for each user defined header.
2019-05-25lib: setup user headers in query parserDavid Bremner
These tests will need to be updated if the Xapian query print/debug format changes.
2019-05-25cli/config: check syntax of user configured field namesDavid Bremner
These restrictions are meant to prevent incompatibilities with the Xapian query parser (which will split at non-word characters) and clashes with future notmuch builtin fields.
2019-05-25cli/config: support user header index configDavid Bremner
We don't do anything with this configuration information information yet, but nonetheless add a couple of regression tests to make sure we don't break standard functionality when we do use the configuration information.
2019-05-23test-lib.sh: colors to test output when parallel(1) is run on ttyTomi Ollila
Done via $COLORS_WITHOUT_TTY environment variable as passing options to commands through parallel(1) does not look trivial. Reorganized color checking in test-lib.sh a bit for this (perhaps were not fully necessary but rest still an improvement): - color checking commands in subshell are not run before arg parsing (args may disable colors with --no-color) - [ -t 1 ] is checked before forking subshell
2019-05-23test-lib.sh: "tidied" emacs_deliver_message ()Tomi Ollila
Added initialization and checking of smtp_dummy_port like it was done with smtp_dummy_pid. Made those function-local variables. One 8 spaces to tab consistency conversion. And last, but definitely not least; while doing above noticed that there were quite a few double-quoted strings where $@ was in the middle of it -- replaced those with $* for robustness ("...$@..." expands params to separate words, "...$*..." params expands to single word).
2019-05-22test: redirect STDIN from /dev/nullTomi Ollila
Without this stdin may be anything that parent process provided for it. Test processes might have tried to read something from it, which would have caused undeterministic behavior. E.g. gdb(1) tries to do tty related ioctls on fd 0 (and fd 1 and fd 2, but those are redirected to 'test.output' before test runs).
2019-05-20test/emacs: revert invalid-from test to pre-86f89385 behaviourDavid Bremner
To the best of my understanding, this original behaviour was what Carl's homebrew parser produced. With commit 86f89385 Austin switched to using GMime (2.6). This produced arguably worse results, but since the input was bad, we could live with it. Now with GMime 3.0 we are getting the original results again, and there is no reason to consider this test broken.
2019-05-20tests: fail and report when a parallel build fails (or times out)Daniel Kahn Gillmor
When a parallel build fails (or when it times out, if timeout is present), the test suite should not blithely succeed. Catch these failures and at least report them. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-20tests: make timeout configurable with NOTMUCH_TEST_TIMEOUT (default: 2m)Daniel Kahn Gillmor
The current 2 minute timeout is reasonable, but to exercise the test suite or induce timeout failures, we might want to make it shorter. This makes it configurable so you can run (for example): make check NOTMUCH_TEST_TIMEOUT=10s We stick with the default of 2m. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-10test: show what emacs sees of an encrypted message when crypto is disabledDaniel Kahn Gillmor
Some users may set notmuch-crypto-process-mime to nil, disabling all crypto use. We should have a baseline for what that looks like.
2019-05-10test: avoid unnecessary extraction of the test fingerprintDaniel Kahn Gillmor
FINGERPRINT is already exported by add_gnupg_home, so this is unnecessary. This change also happens to get rid of the superfluous check-trustdb spew from the test suite that looked like this: gpg: checking the trustdb gpg: marginals needed: 3 completes needed: 1 trust model: pgp gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-10test: let the OS choose a port for smtp-dummyDavid Bremner
This should avoid potential collisions if we start running multiple smtp-dummy processes in parallel.
2019-05-10emacs: test notmuch-show during message decryptionDaniel Kahn Gillmor
We did not have a test showing what message decryption looks like within notmuch-emacs. This change gives us a baseline for future work on the notmuch-emacs interface. This differs from previous revisions of this patch in that it should be insensitive to the order in which the local filesystem readdir()s the underlying maildir. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-07tests: environment variable to specify that tests should be serializedJameson Graef Rollins
If NOTMUCH_TEST_SERIALIZE is non-null all tests will be run in series, rather than in parallel.
2019-05-07tests: run all tests in parallel, if availableJameson Graef Rollins
If either the moreutils or GNU parallel utility are available, run all tests in parallel. On my eight core machine this makes for a ~x7 speed-up in the full test suite (1m24s -> 12s). The design of the test suite makes this parallelization trivial.
2019-05-07tests: remove entangling corpus index optimizationJameson Graef Rollins
The add_email_corpus test utility includes logic that tries to re-use an index of the corpus if available. This was seemingly done as an optimization, so that every test that uses the corpus didn't have to create it's own index of the corpus. However, this has the perverse side effect of entangling tests together, and breaks parallelization. Forcing each test to do it's own index does increase the overall time of the test slightly (~6%), but this will be more than made up for in the next patch that introduces paraellization.
2019-05-07tests: remove some redundant pre-cleanup of the corpus MAIL_DIRJameson Graef Rollins
add_email_corpus itself does an rm -rf $MAIL_DIR, so these are not necessary.
2019-05-07test/crypto: add_gnupg_home should have ultimate trust on "its own" keyDaniel Kahn Gillmor
The typical use case for gpg is that if you control a secret key, you mark it with "ultimate" ownertrust. The opaque --import-ownertrust mechanism is GnuPG's standard mechanism to set up ultimate ownertrust (the ":6:" means "ultimate", for whatever reason). We adjust the test suite to match this change, inverting the sense of one test: since the default is now that the user ID of the suite's own key is valid, we change the test to make sure that the user ID is not emitted when it is *not* valid. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-07test: simplify user ID handlingDaniel Kahn Gillmor
The user ID on the self-test is a little bit clunky-looking. It also may end up showing up elsewhere in the test suite. Centralizing the user ID in one place should make it easier to handle if it ever changes, and should make tests easier to read. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-07test/crypto: clarify the difference between ownertrust and validityDaniel Kahn Gillmor
This is a subtle difference, but the output of notmuch shouldn't ever change based on ownertrust itself -- notmuch is intended to show valid User IDs, and to avoid showing invalid User IDs. It so happens that setting ownertrust of a key to ultimate sets all associated user IDs to "full" validity, so the test is correct, but just misnamed. Signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-05Merge tag 0.28.4David Bremner
No functionality changes merged, since the bug in question was already fixed on master.
2019-05-05cli/show: avoid empty write to stdout in format_part_rawDavid Bremner
Previously if the input was exactly a multiple of the internal buffer size, notmuch would attempt to fwrite nothing to stdout, but still expected fwrite to return 1, causing a failure that looked like this: $ notmuch show --format=raw id:87o96f1cya.fsf@codeaurora.org ...entire message shown as expected.. Error: Write failed $ echo $? 1 To fix the problem don't call fwrite at all when there's nothing to write. Amended by db: add some tests of message sizes likely to cause this problem.
2019-05-03cli/notmuch-show: support gzipped filesDavid Bremner
This drops "file" from mime_node_context and just uses a local variable. It also uses the new gzip aware utility routines recently added to util/gmime-extra.c. The use of gzopen / gzfile in addition is a bit icky, but the choice is between that, and providing yet another readline implimentation that understands GMime streams.
2019-05-03lib/message_file: open gzipped filesDavid Bremner
Rather than storing the lower level stdio FILE object, we store a GMime stream. This allows both transparent decompression, and passing the stream into GMime for parsing. As a side effect, we can let GMime close the underlying OS stream (indeed, that stream isn't visible here anymore). This change is enough to get notmuch-{new,search} working, but there is still some work required for notmuch-show, to be done in a following commit.
2019-05-03gmime-cleanup: simplify T355-smime.shDaniel Kahn Gillmor
GMime 3.0 and later can handle User ID as expected. signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03gmime-cleanup: tests should only care about gmime 3Daniel Kahn Gillmor
note that "notmuch-show for message with invalid From" is still broken in T310-emacs.sh. It would be good to debug what's going on there and try to get it fixed! signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03gmime-cleanup: always support session keysDaniel Kahn Gillmor
Our minimum version of GMime 3.0 always supports good session key handling. signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-05-03gmime-cleanup: remove obsolete gpg_path configuration option and crypto contextsDaniel Kahn Gillmor
Note that we do keep ignoring the gpg_path configuration option, though, to avoid breakage of existing installations. It is ignored like any other unknown configuration option, but we at least document that it is ignored so that people who find it in their legacy configs can know that it's safe to drop. signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
2019-04-26crypto: Avoid pretending to verify signatures on unsigned encrypted mailDaniel Kahn Gillmor
Unsigned encrypted mail shows up with a weird empty signature list. If we successfully decrypted and there was no signature in it, we should just not show a sigstatus at all. The documentation for g_mime_decrypt_result_get_signatures says: a GMimeSignatureList or NULL if the stream was not signed.
2019-04-24tests: move FINGERPRINT definition to add_gnupg_homeDaniel Kahn Gillmor
If a test has added a GnuPG homedir, it may well want to know the fingerprint. This saves us from having to redefine this magic string in multiple places when more tests eventually use the GnuPG homedir.
2019-04-17lib: add 'body:' field, stop indexing headers twice.David Bremner
The new `body:` field (in Xapian terms) or prefix (in slightly sloppier notmuch) terms allows matching terms that occur only in the body. Unprefixed query terms should continue to match anywhere (header or body) in the message. This follows a suggestion of Olly Betts to use the facility (since Xapian 1.0.4) to add the same field with multiple prefixes. The double indexing of previous versions is thus replaced with a query time expension of unprefixed query terms to the various prefixed equivalent. Reindexing will be needed for 'body:' searches to work correctly; otherwise they will also match messages where the term occur in headers (demonstrated by the new tests in T530-upgrade.sh)
2019-04-14test: add test for checking forwarded messagesÖrjan Ekeberg
Add test of forwarding messages from within emacs. The first test checks that a references header is properly added to the new message. The second test checks that the send-hook of the forwarding message adds a forwarded-tag to the original message.
2019-03-28test: Add emacs attachment check tests.David Edmondson
2019-03-28lib: use phrase search for anything not ending in '*'David Bremner
Anything that does not look like a wildcard should be safe to quote. This should fix the problem searching for xapian keywords.
2019-03-28test: add two known broken tests searching for xapian keywordsDavid Bremner
Thanks to plujon for pointing out this problem on IRC. The underlying issue is that the quotes are stripped before the field processors get the query string, and the heuristic for putting them back is not quite right.
2019-03-11Prepend regerror() messages with "regexp error: "Luis Ressel
The exact error messages returned by regerror() aren't standardized; relying on them isn't portable. Thus, add a a prefix to make clear that the subsequent message is a regexp parsing error, and only look for this prefix in the test suite, ignoring the rest of the message.
2019-03-11test/T030-config: Separate stdout and stderr outputLuis Ressel
POSIX doesn't specify the flushing behaviour of the STDOUT stream, so it's invalid to assume a particular order between the stdout and stderr output. The current test breaks on musl due to this.
2019-03-06Merge branch 'release'David Bremner
Changes from 0.28.3
2019-03-05lib/string_map: fix return type of string_cmpDavid Bremner
I can't figure out how checking the sign of a bool ever worked. The following program demonstrates the problem (i.e. for me it prints 1). #include <stdio.h> #include <stdbool.h> int main(int argc, char **argv) { bool x; x = -1; printf("x = %d\n", x); } This seems to be mandated by the C99 standard 6.3.1.2.
2019-03-05lib: Add known broken test for string_map binary search.David Bremner
Because the string_map functions are not exported, we test via message properties.