Jani Nikula [Sun, 23 Nov 2014 11:15:12 +0000 (13:15 +0200)]
lib: drop the deprecation message for single-message mbox files
We generally do not support mbox files, but for historical reasons
we've supported single-message mbox files, with a deprecation
message. We've tried dropping the support altogether, but backed out
of it because we'd need to stop indexing them, while keeping support
for previously indexed files. This would be more complicated than
simply supporting single-message mbox files. Therefore, drop the
deprecation message, and just silently accept single-message mboxes.
Jesse Rosenthal [Sat, 22 Nov 2014 13:17:16 +0000 (08:17 -0500)]
lib: Use email address instead of empty real name.
Currently, if a From-header is of the form:
"" <address@example.com>
the empty string will be treated as a valid real-name, and the entry
in the search results will be empty.
The new behavior here is that we treat an empty real-name field as if
it were null, so that the email address will be used in the search
results instead.
David Edmondson [Tue, 18 Nov 2014 07:03:17 +0000 (07:03 +0000)]
emacs: `with-current-notmuch-show-message' should not leak `coding-system-for-read'
`with-current-notmuch-show-message' applies a `no-conversion' coding
system when reading a raw message from notmuch. That coding system
should _not_ be applied when the body of the macro is evaluated, as it
can cause file operations used during that evaluation to incorrectly
apply the `no-conversion' coding system.
This was discovered when a user's .signature file contained non-ASCII
characters. When a message is forwarded, the `no-conversion' coding
system was applied to the reading of the .signature file, resulting in
raw rather than UTF-8 interpretation of the data.
David Bremner [Thu, 13 Nov 2014 20:59:43 +0000 (21:59 +0100)]
NEWS: deprecate notmuch deliver
notmuch-deliver has no commits for about 2.5 years. notmuch-insert has
all the features that deliver does, and as far as I understand the
error handling has now caught up.
Jani Nikula [Mon, 10 Nov 2014 19:27:52 +0000 (21:27 +0200)]
NEWS: notmuch insert, search updates
News for
- cli: add support for notmuch search --duplicate=N with --output=messages
- cli/insert: add post-insert hook
- cli/insert: require succesful message indexing for success statu
Michal Sojka [Wed, 5 Nov 2014 00:25:56 +0000 (01:25 +0100)]
cli: search: Convert --output to keyword argument
Now, when address related outputs are in a separate command, it makes
no sense to combine multiple --output options in search command line.
Using switch statement to handle different outputs is more readable
than a series of if statements.
Michal Sojka [Wed, 5 Nov 2014 00:25:55 +0000 (01:25 +0100)]
cli: Introduce "notmuch address" command
This moves address-related functionality from search command to the
new address command. The implementation shares almost all code and
some command line options.
Options --offset and --limit were intentionally not included in the
address command, because they refer to messages numbers, which users
do not see in the output. This could confuse users because, for
example, they could see more addresses in the output that what was
specified with --limit. This functionality can be correctly
reimplemented for address subcommand later.
Also useless values of --exclude flag were not included in the address
command.
Michal Sojka [Wed, 5 Nov 2014 00:25:52 +0000 (01:25 +0100)]
cli: search: Convert ctx. to ctx->
In the next commit, notmuch_search_command will be refactored to
several smaller functions. In order to simplify the next commit to
verbatim move of several lines to new functions with search_context_t*
argument, we convert all references to this structure to pointer
dereferences. To do so we rename the context variable and use the
original name ctx as the pointer to the renamed structure.
Michal Sojka [Wed, 5 Nov 2014 00:25:51 +0000 (01:25 +0100)]
cli: search: Move more variables into search_context_t
In order to share some command line options between search and address
subcommands we need to add corresponding variables to the context
structure. While we are at it, we also add notmuch_database_t to unify
parameters of all do_search_* functions and to simplify subsequent
commits.
Tomi Ollila [Mon, 4 Aug 2014 17:39:32 +0000 (20:39 +0300)]
devel: make man-to-mdwn.pl to work with generated manual pages
The new manual pages converted from rst using sphinx or rst2man
has somewhat different syntax. man-to-mdwn.pl is now adjusted
to produce even better output from this syntax. The changes also
include using utf-8 locale (e.g. for tables and generated hypens)
and and quite a few bugs fixes.
This tool still produces better results than just using the
html pages generated using sphinx / rst2html. For example those
tools don't create inter-page hyperlinks -- and the preformatted
pages written by man-to-mdwn.pl just works well with manual page
content.
Jesse Rosenthal [Fri, 31 Oct 2014 17:33:25 +0000 (13:33 -0400)]
test: Make gen-threads work with python3
python3 doesn't allow dictionaries to be initialized with non-string
keywords. This presents problems on systems in which "python" means
"python3". We instead initalize the dictionary using the dict
comprehension and then update it with the values from the tree. This
will work with both python2 and python3.
Tomi Ollila [Sat, 1 Nov 2014 09:39:04 +0000 (11:39 +0200)]
configure: move make {,install} instructions to the end
There was theorical possibility that writing the config files could
have skipped (by interruption) after the instructions how to make
notmuch was printed out.
Michal Sojka [Fri, 31 Oct 2014 21:53:58 +0000 (22:53 +0100)]
cli: search: Add --output={sender,recipients}
The new outputs allow printing senders, recipients or both of matching
messages. To print both, the user can use --output=sender and
--output=recipients simultaneously.
Currently, the same address can appear multiple times in the output.
The next commit will change this. For this reason, tests are
introduced there.
We use mailbox_t rather than InternetAddressMailbox because we will
need to extend it in a following commit.
Michal Sojka [Fri, 31 Oct 2014 21:53:57 +0000 (22:53 +0100)]
cli: search: Convert --output to keyword-flag argument
This converts "notmuch search" to use the recently introduced
keyword-flag argument parser. At this point, it only makes the code
slightly less readable but following commits that add new --output
keywords will profit from this.
Michal Sojka [Fri, 31 Oct 2014 21:53:55 +0000 (22:53 +0100)]
cli: search: Refactor passing of command line options
Many functions that implement the search command need to access command
line options. Instead of passing each option in a separate variable, put
them in a structure and pass only this structure.
Jani Nikula [Mon, 20 Oct 2014 15:03:21 +0000 (18:03 +0300)]
test: use LDFLAGS in test/Makefile.local
Apparently the test binaries are built with minimal LDFLAGS, only
adding dependency specific LDFLAGS as needed. However because some of
the test binaries incorporate notmuch object files, it is necessary to
use the same link flags as notmuch. For example user provided
CFLAGS/CXXFLAGS/LDFLAGS with -fsanitize=undefined fails to build the
test binaries if the flags differ.
Jani Nikula [Sun, 28 Sep 2014 14:40:59 +0000 (17:40 +0300)]
cli/insert: add post-insert hook
The post-new hook might no longer be needed or run very often if
notmuch insert is being used. Therefore a post-insert hook is needed
(arguably pre-insert not so much, so don't add one). Also add the
--no-hooks option to skip hooks.
Austin Clements [Thu, 23 Oct 2014 12:30:41 +0000 (08:30 -0400)]
lib: Remove unnecessary thread linking steps when using ghost messages
Previously, it was necessary to link new messages to children to work
around some (though not all) problems with the old metadata-based
approach to stored thread IDs. With ghost messages, this is no longer
necessary, so don't bother with child linking when ghost messages are
in use.
Austin Clements [Thu, 23 Oct 2014 12:30:37 +0000 (08:30 -0400)]
lib: Implement ghost-based thread linking
This updates the thread linking code to use ghost messages instead of
user metadata to link messages into threads.
In contrast with the old approach, this is actually correct.
Previously, thread merging updated only the thread IDs of message
documents, not thread IDs stored in user metadata. As originally
diagnosed by Mark Walters [1] and as demonstrated by the broken
T260-thread-order test, this can cause notmuch to fail to link
messages even though they're in the same thread. In principle the old
approach could have been fixed by updating the user metadata thread
IDs as well, but these are not indexed and hence this would have
required a full scan of all stored thread IDs. Ghost messages solve
this problem naturally by reusing the exact same thread ID and message
ID representation and indexing as regular messages.
Furthermore, thanks to this greater symmetry, ghost messages are also
algorithmically simpler. We continue to support the old user metadata
format, so this patch can't delete any code, but when we do remove
support for the old format, several functions can simply be deleted.
Austin Clements [Thu, 23 Oct 2014 12:30:36 +0000 (08:30 -0400)]
lib: Internal support for querying and creating ghost messages
This updates the message abstraction to support ghost messages: it
adds a message flag that distinguishes regular messages from ghost
messages, and an internal function for initializing a newly created
(blank) message as a ghost message.
David Bremner [Sat, 25 Oct 2014 06:53:29 +0000 (08:53 +0200)]
test/emacs: force *Messages* buffer to be writable
In emacs 24.4 the messages buffer starts being read-only, which kills
these tests. This seems to be the point of the variable
inihibit-read-only, which has existed at least since emacs 21.
David Bremner [Wed, 10 Sep 2014 06:41:11 +0000 (08:41 +0200)]
test: simplify T360-symbol-hiding, use nm instead of objdump
After yet another variation in objdump output caused this test to fail
(on a Debian port, no less), I decided whatever putative benefit we
get from looking at the object files instead of the library isn't
worth the maintenence headache.
This version uses nm -P. nm -P should be portable, and fixed format.
It purposely doesn't use the -D argument, since that is non-POSIX and
nm on GNU/Linux seems do the right thing without it.
It still won't work out of the box on e.g. Mac OS/X. I think the right
thing to do there is to move some more configuration information into
sh.config.
Austin Clements [Fri, 3 Oct 2014 16:58:03 +0000 (12:58 -0400)]
test: Port atomicity test to Python
Previously, this was implemented using a horrible GDB script (because
there is no such thing as a non-horrible GDB script). This GDB script
often broke with newer versions of GDB for mysterious reasons. Port
the test script to GDB's Python API, which makes the code much cleaner
and, hopefully, more stable.
Ian Main [Wed, 1 Oct 2014 17:11:28 +0000 (10:11 -0700)]
VIM: Make starting in 'insert' mode for compose optional
This adds a variable to make starting in insert mode optional when
composing and replying to emails. I found it unusual to be started in
insert mode so I thought I'd make it optional as others may find this as
well.
Ian Main [Thu, 2 Oct 2014 23:47:15 +0000 (16:47 -0700)]
VIM: Use notmuch CLI for config
This patch switches from reading .notmuch-config directly to using
the CLI the same way that emacs does it. It actually uses less code
and is probably less error prone.
Jani Nikula [Fri, 3 Oct 2014 21:18:59 +0000 (23:18 +0200)]
cli/insert: require succesful message indexing for success status
Add --keep option to keep any remaining stuff in index or file. We
could distinguish between failures to index and failures to apply tags
or maildir sync, but for simplicity just have one.
Jani Nikula [Fri, 3 Oct 2014 21:18:58 +0000 (23:18 +0200)]
cli/insert: add fail path to add_file_to_database
Handle failures gracefully in add_file_to_database, renamed simply
add_file while at it. Add keep option to not remove the message from
database if tagging or tag syncing to maildir flags fails. Expand the
function documentation to cover the changes.
Austin Clements [Mon, 6 Oct 2014 23:17:09 +0000 (17:17 -0600)]
lib: Handle empty date value
In the interest of robustness, avoid undefined behavior of
sortable_unserialise if the date value is missing. This shouldn't
happen now, but ghost messages will have blank date values.
Austin Clements [Mon, 6 Oct 2014 23:17:08 +0000 (17:17 -0600)]
lib: Refactor _notmuch_database_link_message
This moves the code to retrieve and clear the metadata thread ID out
of _notmuch_database_link_message into its own function. This will
simplify future changes.
Austin Clements [Mon, 6 Oct 2014 23:17:07 +0000 (17:17 -0600)]
lib: Move message ID compression to _notmuch_message_create_for_message_id
Previously, this was performed by notmuch_database_add_message. This
happens to be the only caller currently (which is why this was safe),
but we're about to introduce more callers, and it makes more sense to
put responsibility for ID compression in the lower-level function
rather than requiring each caller to handle it.
David Bremner [Sun, 5 Oct 2014 05:53:28 +0000 (07:53 +0200)]
NEWS: mention the change in default build flags
It blows things up by a factor of six or so, so it's worth giving
people a heads up. It won't effect e.g. Debian, that already builds
with -g and then strips.
W. Trevor King [Tue, 7 Oct 2014 17:36:13 +0000 (10:36 -0700)]
NEWS: Document "nmbug: Translate to Python"
For more details, see the commit message for 7f2cb3be (nmbug:
Translate to Python, 2014-10-03). I realized while writing this that
the 7f2cb3be commit message has:
* 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.
But we dropped that exec in favor of the subprocess approach between
v3 and v4, I just forgot to update the commit message [1].
Austin Clements [Fri, 3 Oct 2014 16:58:03 +0000 (12:58 -0400)]
test: Port atomicity test to Python
Previously, this was implemented using a horrible GDB script (because
there is no such thing as a non-horrible GDB script). This GDB script
often broke with newer versions of GDB for mysterious reasons. Port
the test script to GDB's Python API, which makes the code much cleaner
and, hopefully, more stable.
W. Trevor King [Fri, 3 Oct 2014 18:20:57 +0000 (11:20 -0700)]
nmbug: Translate to Python
This allows us to capture stdout and stderr separately, and do other
explicit subprocess manipulation without resorting to external
packages. It should be compatible with Python 2.7 and later
(including the 3.x series).
Most of the user-facing interface is the same, but there are a few
changes, where reproducing the original interface was too difficult or
I saw a change to make the underlying Git UI accessible:
* 'nmbug help' has been split between the general 'nmbug --help' and
the command-specific 'nmbug COMMAND --help'.
* Commands are no longer split into "most common", "other useful", and
"less common" sets. If we need something like this, I'd prefer
workflow examples highlighting common commands in the module
docstring (available with 'nmbug --help').
* 'nmbug commit' now only uses a single argument for the optional
commit-message text. I wanted to expose more of the underlying 'git
commit' UI, since I personally like to write my commit messages in
an editor with the notes added by 'git commit -v' to jog my memory.
Unfortunately, we're using 'git commit-tree' instead of 'git
commit', and commit-tree is too low-level for editor-launching. I'd
be interested in rewriting commit() to use 'git commit', but that
seemed like it was outside the scope of this rewrite. So I'm not
supporting all of Git's commit syntax in this patch, but I can at
least match 'git commit -m MESSAGE' in requiring command-line commit
messages to be a single argument.
* The default repository for 'nmbug push' and 'nmbug fetch' is now the
current branch's upstream (branch.<name>.remote) instead of
'origin'. When we have to, we extract this remote by hand, but
where possible we just call the Git command without a repository
argument, and leave it to Git to figure out the default.
* 'nmbug push' accepts multiple refspecs if you want to explicitly
specify what to push. Otherwise, the refspec(s) pushed depend on
push.default. The Perl version hardcoded 'master' as the pushed
refspec.
* 'nmbug pull' defaults to the current branch's upstream
(branch.<name>.remote and branch.<name>.merge) instead of hardcoding
'origin' and 'master'. It also supports multiple refspecs if for
some crazy reason you need an octopus merge (but mostly to avoid
breaking consistency with 'git pull').
* 'nmbug log' now execs 'git log', as there's no need to keep the
Python process around once we've launched Git there.
* 'nmbug status' now catches stderr, and doesn't print errors like:
No upstream configured for branch 'master'
The Perl implementation had just learned to avoid crashing on that
case, but wasn't yet catching the dying subprocess's stderr.
* 'nmbug archive' now accepts positional arguments for the tree-ish
and additional 'git archive' options. For example, you can run:
$ nmbug archive HEAD -- --format tar.gz
I wish I could have preserved the argument order from 'git archive'
(with the tree-ish at the end), but I'm not sure how to make
argparse accept arbitrary possitional arguments (some of which take
arguments). Flipping the order to put the tree-ish first seemed
easiest.
* 'nmbug merge' and 'pull' no longer checkout HEAD before running
their command, because blindly clobbering the index seems overly
risky.
* In order to avoid creating a dirty index, 'nmbug commit' now uses
the default index (instead of nmbug.index) for composing the commit.
That way the index matches the committed tree. To avoid leaving a
broken index after a failed commit, I've wrapped the whole thing in
a try/except block that resets the index to match the pre-commit
treeish on errors. That means that 'nmbug commit' will ignore
anything you've cached in the index via direct Git calls, and you'll
either end up with an index matching your notmuch tags and the new
HEAD (after a successful commit) or an index matching the original
HEAD (after a failed commit).
David Bremner [Sat, 4 Oct 2014 18:47:00 +0000 (20:47 +0200)]
doc: build notmuch-emacs info/html docs, link from index
Although this manual is far from complete, it may be helpful for
someone. In particular building it as part of the standard build
process makes it easier to find problems when editing the
notmuch-emacs-manual.