notmuch/test/corpora, branch 0.36_rc1

test: start new corpus of test messages for indexing code

2022-02-20T00:14:18Z

This particular message is not recognized by notmuch as mail, but is fine according to e.g. mutt. The trigger for this bad behaviour seems to be a second "From " ocurring at the beginning of the line but inside an attachment.

test/crypto: test message with rfc822 attachment.

2021-08-30T23:24:59Z

This is intended to help track down a display problem in the emacs front end

tests/smime: add tests for S/MIME SignedData

2020-04-30T20:57:26Z

Add a simple S/MIME SignedData message, taken from an upcoming draft of https://datatracker.ietf.org/doc/draft-autocrypt-lamps-protected-headers/ RFC 8551 describes a SignedData, a one-part clearsigned object that is more resistant to common patterns of MTA message munging than multipart/signed (but has the downside that it is only readable by clients that implement S/MIME). To make sure sure notmuch can handle this kind of object, we want to know a few things: Already working: - Is the content of the SignedData object indexed? It actually is right now because of dumb luck -- i think we're indexing the raw CMS object and it happens to contain the cleartext of the message in a way that we can consume it before passing it on to Xapian. - Are we accidentally indexing the embedded PKCS#7 certificates? We don't want to, and for some reason I don't understand, our indexing is actually skipping the embedded certificates already. That's good! Still need fixing: - do we know the MIME type of the embedded part? - do we know that the message is signed? - can notmuch-show read its content? - can notmuch-show indicate the signature validity? - can notmuch-reply properly quote and attribute content? Signed-off-by: Daniel Kahn Gillmor

tests: Add S/MIME messages to protected-headers corpus

2020-04-30T20:55:19Z

These sample messages are taken directly from the Protected Headers draft: https://www.ietf.org/id/draft-autocrypt-lamps-protected-headers-02.html Note that this commit doesn't strictly pass the common git pre-commit hook due to introducing some trailing whitespace. That's just the nature of the corpus, though. We should have that trailing whitespace, so I've made this commit with --no-verify. Signed-off-by: Daniel Kahn Gillmor

test: add test for "Mixed-Up Mime" message mangling

2019-09-15T05:20:03Z

Some MTAs mangle e-mail messages in transit in ways that are repairable. Microsoft Exchange (in particular, the version running today on Office365's mailservers) appears to mangle multipart/encrypted messages in a way that makes them undecryptable by the recipient. I've documented this in section 4.1 "Mixed-up encryption" of draft -00 of https://tools.ietf.org/html/draft-dkg-openpgp-pgpmime-message-mangling Fortunately, it's possible to repair such a message, and notmuch can do that so that a user who receives an encrypted message from a user of office365.com can still decrypt the message. Enigmail already knows about this particular kind of mangling. It describes it as "broken PGP email format probably caused by an old Exchange server", and it tries to repair by directly changing the message held by the user. if this kind of repair goes wrong, the repair process can cause data loss (https://sourceforge.net/p/enigmail/bugs/987/, yikes). The tests introduced here are currently broken. In subsequent patches, i'll introduce a non-destructive form of repair for notmuch so that notmuch users can read mail that has been mangled in this way, and the tests will succeed. Signed-off-by: Daniel Kahn Gillmor

test: avoid showing legacy-display parts

2019-09-01T11:32:56Z

Enigmail generates a "legacy-display" part when it sends encrypted mail with a protected Subject: header. This part is intended to display the Subject for mail user agents that are capable of decryption, but do not know how to deal with embedded protected headers. This part is the first child of a two-part multipart/mixed cryptographic payload within a cryptographic envelope that includes encryption (that is, it is not just a cleartext signed message). It uses Content-Type: text/rfc822-headers. That is: A └┬╴multipart/encrypted B ├─╴application/pgp-encrypted C └┬╴application/octet-stream * ╤ D └┬╴multipart/mixed; protected-headers=v1 (cryptographic payload) E ├─╴text/rfc822-headers; protected-headers=v1 (legacy-display part) F └─╴… (actual message body) In discussions with jrollins, i've come to the conclusion that a legacy-display part should be stripped entirely from "notmuch show" and "notmuch reply" now that these tools can understand and interpret protected headers. You can tell when a message part is a protected header part this way: * is the payload (D) multipart/mixed with exactly two children? * is its first child (E) Content-Type: text/rfc822-headers? * does the first child (E) have the property protected-headers=v1? * do all the headers in the body of the first child (E) match the protected headers in the payload part (D) itself? If this is the case, and we already know how to deal with the protected header, then there is no reason to try to render the legacy-display part itself for the user. Furthermore, when indexing, if we are indexing properly, we should avoid indexing the text in E as part of the message body. 'notmuch reply' is an interesting case: the standard use of 'notmuch reply' will end up omitting all mention of protected Subject:. The right fix is for the replying MUA to be able to protect its headers, and for it to set them appropriately based on headers found in the original message. If a replying MUA is unable to protect headers, but still wants the user to be able to see the original header, a replying MUA that notices that the original message's subject differs from the proposed reply subject may choose to include the original's subject in the quoted/attributed text. (this would be a stopgap measure; it's not even clear that there is user demand for it) This test suite change indicates what we want to happen for this case (the tests are currently broken), and includes three additional TODO suggestions of subtle cases for anyone who wants to flesh out the test suite even further. (i believe all these cases should be already fixed by the rest of this series, but haven't had time to write the tests for the unusual cases) Signed-off-by: Daniel Kahn Gillmor

test: signature verification during decryption (session keys)

2019-06-08T23:14:00Z

When the user knows the signer's key, we want "notmuch show" to be able to verify the signature of an encrypted and signed message regardless of whether we are using a stashed session key or not. I wrote this test because I was surprised to see signature verification failing when viewing some encrypted messages after upgrading to GPGME 1.13.0-1 in debian experimental. The added tests here all pass with GPGME 1.12.0, but the final test fails with 1.13.0, due to some buggy updates to GPGME upstream: see https://dev.gnupg.org/T3464 for more details. While the bug needs to be fixed in GPGME, notmuch's test suite needs to make sure that GMime is doing what we expect it to do; i was a bit surprised that it hadn't caught the problem, hence this patch. I've fixed this bug in debian experimental with gpgme 1.13.0-2, so the tests should pass on any debian system. I've also fixed it in the gpgme packages (1.13.0-2~ppa1) in the ubuntu xenial PPA (ppa:notmuch/notmuch) that notmuch uses for Travis CI. Signed-off-by: Daniel Kahn Gillmor

test: try indexing nested messages and protected headers

2019-05-29T11:15:28Z

We want to make sure that internally-forwarded messages don't end up "bubbling up" when they aren't actually the cryptographic payload. Signed-off-by: Daniel Kahn Gillmor

test: protected headers should work when both encrypted and signed.

2019-05-29T11:14:57Z

Up to this point, we've tested protected headers on messages that have either been encrypted or signed, but not both. This adds a couple tests of signed+encrypted messages, one where the subject line is masked (outside subject line is "Subject Unavailable") and another where it is not (outside Subject: matches inner Subject:) See the discussion at https://dkg.fifthhorseman.net/blog/e-mail-cryptography.html#protected-headers for more details about the nuances between signed, stripped, and stubbed headers. Signed-off-by: Daniel Kahn Gillmor

test: show cryptographic envelope information for signed mails

2019-05-29T11:13:06Z

Make sure that we emit the correct cryptographic envelope status for cleartext signed messages. Signed-off-by: Daniel Kahn Gillmor