By default, emacs hides the User-Agent and References headers when
composing mail. This is a good thing for users, but a bad thing for
testing, since we can create ugly or invalid headers and not have it
show up in the tests.
By setting message-hidden-headers to an empty list, we force emacs to
show all the headers, so we can check that they're correct. Users
won't see this, but it will let us catch future bugs.
As a side-effect, this breaks all the reply tests, since there is a
bug with the References and User-Agent headers, fixed in the next commit.
Bug 1: Replying from alternate addresses
----------------------------------------
The reply code was inconsistent in its use of symbols and strings for
header names being passed to message.el functions. This caused the
From header to be lookup up incorrectly, causing an additional From
header to be added with the user's primary address instead of the
correct alternate address.
This is fixed by using symbols everywhere, i.e. never using strings
for header names when interacting with message.el.
This change also removes our use of `mail-header`, since we don't use
it anywhere else, and using assq makes it clear how the header lists
are expected to work.
Bug 2: Duplicate headers in emacs 23.2
--------------------------------------
The message.el code in emacs 23.2 assumes that header names will
always be passed as symbols, so our use of strings caused
problems. The symptom was that on 23.2 (and presumably on earlier
versions) the reply message would end up with two of some headers.
Converting everything to symbols also fixes this issue.
test: Tests for reply from alternate addresses in emacs
Since the recent reply changes were pushed, there has been a bug that
causes emacs to always reply from the primary address, even if the
JSON or default CLI reply output uses an alternate address.
This adds two tests to the emacs test library based on the two "Reply
form..." tests in the reply test library. One is currently marked
broken.
Austin Clements [Tue, 27 Mar 2012 01:40:26 +0000 (21:40 -0400)]
emacs: Fix mis-named argument to notmuch-get-bodypart-internal
Previously, this function took an argument called "message-id", even
though it was a general query, rather than a message ID. This changes
it to "query".
Austin Clements [Tue, 27 Mar 2012 21:59:52 +0000 (17:59 -0400)]
reply: Move reply citation printing to the recursive MIME walk
This makes more logical sense, since it makes the recursive printer
responsible for the entire reply body and lets it start at the root of
the MIME tree instead of the first child. (We could move reply header
creation in there, too, but if we ever support proper reply to
multiple messages, we'll want just one set of reply headers computed
from the entire message set and many bodies.)
Austin Clements [Tue, 27 Mar 2012 21:59:50 +0000 (17:59 -0400)]
show/reply: Unify the code that extracts text parts
Previously, show and reply had separate implementations of decoding
and printing text parts. Now both use show's implementation, which
was more complete. Show's implementation has been extended with an
option to add reply quoting to the extracted part (this is implemented
as a named flag to avoid naked booleans, even though it's the only
flag it can take).
Austin Clements [Tue, 27 Mar 2012 01:37:16 +0000 (21:37 -0400)]
emacs: Escape all message ID queries
This adds a lib function to turn a message ID into a properly escaped
message ID query and uses this function wherever we previously
hand-constructed ID queries. Wherever this new function is used,
documentation has been clarified to refer to "id: queries" instead of
"message IDs".
This fixes the broken test introduced by the previous patch.
Jani Nikula [Mon, 26 Mar 2012 21:04:09 +0000 (00:04 +0300)]
cli: refactor "notmuch tag" data structures for tagging operations
To simplify code, keep all tagging operations in a single array
instead of separate add and remove arrays. Apply tag changes in the
order specified on the command line, instead of first removing and
then adding the tags.
This results in a minor functional change: If a tag is both added and
removed, the last specified operation is now used. Previously the tag
was always added. Change the relevant test to reflect the new
behaviour.
Tomi Ollila [Wed, 21 Mar 2012 19:32:29 +0000 (21:32 +0200)]
NEWS: consistent 2-space indentation in new content
Some 0.12 NEWS items descriptions were indented with 3 spaces whereas
all other lines are indented with 2 spaces. Brought those
escaped lines back in line with others.
Tomi Ollila [Wed, 14 Mar 2012 14:59:59 +0000 (16:59 +0200)]
configure: store $IFS to $DEFAULT_IFS readonly variable
In the future, IFS value needs to be changed in a few places
in configure -- and then restored. Store the original value
to $DEFAULT_IFS for easy restoration.
Justus Winter [Mon, 5 Mar 2012 10:23:44 +0000 (11:23 +0100)]
Do not try to parse the options for --build and --host arguments
Formerly the code assumed the arguments to be triples and threw an
error if this was not the case. But those arguments are only there for
compatibility with autotools and are not used within the build system,
so just dropping the code parsing these values makes the build system
more robust.
emacs: Use the new JSON reply format and message-cite-original
Use the new JSON reply format to create replies in emacs. Quote HTML
parts nicely by using mm-display-part to turn them into displayable
text, then quoting them with message-cite-original. This is very
useful for users who regularly receive HTML-only email.
Use message-mode's message-cite-original function to create the
quoted body for reply messages. In order to make this act like the
existing notmuch defaults, you will need to set the following in
your emacs configuration:
This new JSON format for replies includes headers generated for a
reply message as well as the headers of the original message. Using
this data, a client can intelligently create a reply. For example, the
emacs client will be able to create replies with quoted HTML parts by
parsing the HTML parts.
Austin Clements [Fri, 16 Mar 2012 03:13:12 +0000 (23:13 -0400)]
emacs: Fix search tab completion in terminals
In X, Emacs distinguishes the tab key, which produces a 'tab event;
from C-i, which produces a ?\t event. However, in a terminal, these
are indistinguishable and only produce a ?\t event. In order to
simplify things, Emacs automatically translates from 'tab to ?\t (see
"Function key translations" in M-x describe-bindings), so functions
only need to be bound to ?\t to work in all situations.
Previously, the search tab completion code usedq (kbd "<tab>"), which
produced the event sequence [tab], which only matched the 'tab event
and hence only worked in X. This patch changes it to (kbd "TAB"),
which matches the general ?\t event and works in all situations.
Mark Walters [Wed, 14 Mar 2012 12:26:54 +0000 (12:26 +0000)]
lib: fix an exclude bug
When the exclude tags contain a tag that does not occur anywhere in
the Xapian database the exclusion fails. We modify the way the query
is constructed to `work around' this. (In fact the new code is cleaner
anyway.)
It also seems to fix another exclusion failure bug reported by
jrollins but we have not yet worked out why it helps in that case.
Mark Walters [Wed, 14 Mar 2012 12:26:53 +0000 (12:26 +0000)]
test: add tests for message only search
This adds three tests for --output=messages searches. One test is for
the case when one exclude tag does not occur in the Xapian
database. This triggers a Xapian bug in some cases and causes the
whole exclusion to fail. The next commit avoids this bug.
Mark Walters [Wed, 14 Mar 2012 12:26:52 +0000 (12:26 +0000)]
test: the test for the exclude code mistakenly excludes the tag "="
The tests for the exclude code in search and count use the line
notmuch config set search.exclude_tags = deleted
which actually sets the exclude tags to be "=" and "deleted". Remove
the "=" from this line.
Austin Clements [Wed, 14 Mar 2012 02:31:30 +0000 (22:31 -0400)]
lib: Expose query debug output via an environment variable
Allow query debugging to be enabled at run-time by setting the
NOTMUCH_DEBUG_QUERY environment variable to a non-empty string.
Previously, enabling query debugging required recompiling, but parsed
queries are often useful for tracking down bugs in situations where
recompiling is inconvenient.
Dmitry Kurochkin [Sat, 10 Mar 2012 01:24:51 +0000 (05:24 +0400)]
test: use subtest name for generated message subject by default
Before the change, messages generated by generate_message() used "Test
message #N" for default subject where N is the generated messages
counter. Since message subject is commonly present in expected
results, there is a chance of breaking other tests when a new
generate_message() call is added. The patch changes default subject
value for generated messages to subtest name if it is available. If
subtest name is not available (i.e. message is generated during test
initialization), the old default value is used (in this case it is
fine to have the counter in the subject).
Another benefit of this change is a sane default value for subject in
generated messages, which would allow to simplify code like:
test_begin_subtest "test for a cool feature"
add_message [subject]="message for test for a cool feature"
Dmitry Kurochkin [Sat, 10 Mar 2012 01:24:50 +0000 (05:24 +0400)]
test: remove "Generate some messages" test from raw
Before the change, the first subtest in raw format tests just
generated messages and checked that they are added successfully. This
is not really a raw format test, it is creating of environment
required for other subtests to run. The patch removes the first
subtest from raw and replaces it with bare add_message calls, similar
to how it is done in other tests.
TODO: we should check that test environment was created successfully.
Currently, many tests do add_message(), notmuch new and other calls
without checking the results. We should come up with a general
solution for this, i.e. if any command during test initialization
fails, all tests should be skipped with appropriate error message.
Austin Clements [Tue, 6 Mar 2012 18:48:43 +0000 (18:48 +0000)]
show: Convert raw format to the new self-recursive style, properly support interior parts
This is fully compatible for root and leaf parts, but now has proper
support for interior parts. This requires some design decisions that
were guided by what I would want if I were to save a part.
Specifically:
- Leaf parts are printed without headers and with transfer decoding.
This is what makes sense for saving attachments. (Furthermore, the
transfer decoding is necessary since, without the headers, the
caller would not be able to interpret non-transfer-decoded output.)
- Message parts are printed with their message headers, but without
enclosing part headers. This is what makes sense for saving a
message as a whole (which is a message part) and for saving attached
messages. This is symmetric for whole messages and for attached
messages, though we special-case the whole message for performance
reasons (and corner-case correctness reasons: given malformed input,
GMime may not be able to reproduce it from the parsed
representation).
- Multipart parts are printed with their headers and all child parts.
It's not clear what the best thing to do for multipart is, but this
was the most natural to implement and can be justified because such
parts can't be interpreted without their headers.
As an added benefit, we can move the special-case code for part 0 into
the raw formatter.
Austin Clements [Tue, 6 Mar 2012 18:48:38 +0000 (18:48 +0000)]
test: Fix malformed multipart message
Previously, there was only one CRLF between the terminating boundary
of the embedded multipart/alternative and the boundary of the
containing multipart. However, according the RFC 1341, 7.2.1:
The boundary must be followed immediately either by another CRLF and
the header fields for the next part, or by two CRLFs, in which case
there are no header fields for the next part
and
The CRLF preceding the encapsulation line is considered part of the
boundary so that it is possible to have a part that does not end
with a CRLF (line break).
Thus, there must be *two* CRLFs between these boundaries: one that
ends the terminating boundary and one that begins the enclosing
boundary.
While GMime accepted the message we had before, it could not produce
such a message.
Justus Winter [Sat, 17 Mar 2012 16:41:27 +0000 (17:41 +0100)]
Fix error reporting in notmuch_database_find_message_by_filename
Formerly it was possible for *message_ret to be left
uninitialized. The documentation however clearly states that "[o]n any
failure or when the message is not found, this function initializes
'*message' to NULL".
Thomas Jost [Tue, 21 Feb 2012 19:35:35 +0000 (20:35 +0100)]
build: Require gmime >= 2.6.7
gmime-2.6 had a bug [1] which made it impossible to tell why a signature
verification failed when the signer key was unavailable (empty "sigstatus" field
in the JSON output). Since 00b5623d the corresponding test is marked as broken
when using gmime-2.6 (2.4 is fine).
This bug has been fixed in gmime 2.6.5, which is now the minimal gmime-2.6
version required for building notmuch (gmime-2.4 is still available). As a
consequence the version check in test/crypto can be removed.
[Added by db]
Although less unambigously a bug, Gmime 2.6 prior to 2.6.7 also was
more strict about parsing, and rejected messages with initial "From "
headers. This restriction is relaxed in [2]. For reasons explained in [3],
we want to keep this more relaxed parsing for now.
Fix configure script to properly detect gmime-2.6 if available.
Previously, the configure script would appear to detect gmime-2.6 if
present. However, the binaries would end up being compiled against
gmime-2.4. The addition of a break fixes things so that now gmime-2.6
will be used if available, falling back to gmime-2.4.
Jani Nikula [Sun, 4 Mar 2012 08:25:38 +0000 (10:25 +0200)]
emacs: fix MML quoting in replies
The reply MML quoting added in commit ae438cc unintentionally MML
quotes also the signature/encryption MML tags added via
message-setup-hook, causing the reply not to be signed/encrypted.
MML quote just the original message in the temp buffer before
inserting it to the message buffer, to not interfere with message mode
hooks or message construction in general.
See [1] and [2] for bug reports.
Thanks to Tim Bielawa <tbielawa@redhat.com> for testing.
David Bremner [Sat, 3 Mar 2012 12:56:15 +0000 (08:56 -0400)]
Make exclusion visible again
This reverts
dfee0f9 man: remove search.exclude_tags from notmuch-config.1 for 0.12 e83409d NEWS: revert NEWS item for exclude tags for 0.12 e77b031 config: disable addition of exclude tags for 0.12
Justus Winter [Fri, 2 Mar 2012 14:58:39 +0000 (15:58 +0100)]
Actually close the xapian database in notmuch_database_close
Formerly the xapian database object was deleted and closed in its
destructor once the object was garbage collected. Explicitly call
close() so that the database and the associated lock is released
immediately.
Mark Walters [Thu, 1 Mar 2012 22:30:45 +0000 (22:30 +0000)]
emacs: notmuch.el ignore excluded matches
This is a small change to make notmuch.el ignore excluded matches. In
the future it could do something better like add a button for
rerunning the search with the excludes (particularly if nothing
matches with the excludes) or having them invisible and allowing the
visibility to be toggled.
Mark Walters [Thu, 1 Mar 2012 22:30:44 +0000 (22:30 +0000)]
emacs: show: recognize the exclude flag.
Show mode will recognize the exclude flag by not opening excluding
messages by default, and will start at the first matching non-excluded
message. If there are no matching non-excluded messages it will go to
the first matching (necessarily excluded) message.
Mark Walters [Thu, 1 Mar 2012 22:30:43 +0000 (22:30 +0000)]
cli: omit excluded messages in results where appropriate.
In all cases of notmuch count/search/show where the results returned
cannot reflect the exclude flag return just the matched not-excluded
results. If the caller wishes to have all the matched results (i.e.,
including the excluded ones) they should call with the
--no-exclude option.
The relevant cases are
count: both threads and messages
search: all cases except the summary view
show: mbox format
Mark Walters [Thu, 1 Mar 2012 22:30:41 +0000 (22:30 +0000)]
test: update tests to reflect the exclude flag
notmuch show outputs the exclude flag so many tests using notmuch
show failed. This commit adds "excluded:0" or "excluded: false" to
the expected outputs. After this commit there should be no failing
tests.
Mark Walters [Thu, 1 Mar 2012 22:30:40 +0000 (22:30 +0000)]
cli: Make notmuch-show respect excludes.
This adds the excludes to notmuch-show.c. We do not exclude when only
a single message (or part) is requested. notmuch-show will output the
exclude information when either text or json format is requested. As
this changes the output from notmuch-show it breaks many tests (in a
trivial and expected fashion).
Mark Walters [Thu, 1 Mar 2012 22:30:39 +0000 (22:30 +0000)]
test: update search test to reflect exclude flag
notmuch-search.c now returns all matching threads even if it the
match is a search.tag_excluded message (but with a mark indicating
this). Update the test to reflect this.
projects / notmuch / log