Carl Worth [Wed, 4 Nov 2009 17:26:50 +0000 (09:26 -0800)]
notmuch.el: Make archive-thread more efficient for already archived messages.
The approach here is to move the optimization from mark-read to the
more general remove-tag. Namely, don't call out to a "notmuch tag"
command to remove a tag that's not there already.
Carl Worth [Wed, 4 Nov 2009 16:42:38 +0000 (08:42 -0800)]
notmuch.el: Start implementing the magic space bar.
Currently this will either advance by screenfuls, or to the next
message if it's already within a screenful, and will mark each message
read as it is left.
It doesn't yet complete the magic by archiving the messages nor by
advancing to the next thread in the search.
Carl Worth [Wed, 4 Nov 2009 16:48:57 +0000 (08:48 -0800)]
notmuch.el: Redefine behavior of notmuch-show-previous-message
Now, if the user has manually moved point to somewhere within a
message, executing the previous-message command onece will rewind
point only to the beginning of the current message. Previously this
would go back to the previous message, (which the user can now do
easily and naturally by simply executing the command one more time).
Carl Worth [Wed, 4 Nov 2009 02:24:13 +0000 (18:24 -0800)]
notmuch.el: Add more complete documentation to the major modes.
These now provide a summary of the most useful features/bindings
as well as a complete printout of the relevant mode maps to show
all available keybindings.
Carl Worth [Wed, 4 Nov 2009 01:18:04 +0000 (17:18 -0800)]
notmuch.el: Make archive-thread advance to next line.
This is the command in notmuch-search mode and it's cer convenient
for it to advance to the next line there. (It would be even more
convenient if it didn't also take forever, but as mentioned before
that's an issue we'll need to fix in Xapian.)
Carl Worth [Wed, 4 Nov 2009 00:55:20 +0000 (16:55 -0800)]
notmuch.el: Add bindings for scrolling to notmuch-search mode.
We turn on the scroll-preserve-screen-position option which seems
like what's desired here, (though that's not what I normally use
when editing files---but I think scrolling through a list of email
threads is different).
Carl Worth [Wed, 4 Nov 2009 00:47:34 +0000 (16:47 -0800)]
Drop inapplicable copyright statements.
I had put these in here since I had originally planned to copy
liberally from the body of the implementation of 'compile in order
to get process output into a buffer. But once I found call-process
in the documentation of emacs, that was all I needed.
And all the code I've written since has been entirely my own with
just the help of emacs documentation.
Carl Worth [Tue, 3 Nov 2009 21:34:05 +0000 (13:34 -0800)]
notmuch.el: Add command to (a)rchive a thread from notmuch-show mode.
This is our first race-free implementation of archive-thread! It
acts only on the messages explcitly contained in the buffer, not
on an entire thread ID, so it's safe in the face of new messages
have been delivered for this thread since the view was made.
Carl Worth [Tue, 3 Nov 2009 21:16:40 +0000 (13:16 -0800)]
notmuch.el: Don't try to remove an "unread" tag that's not there.
This optimization wouldn't be necessary if we had a nice fast "notmuch
tag" command. But since it's currently fairly slow, (see Xapian defect
250: http://trac.xapian.org/ticket/250), we're willing to take some
extra care to avoid calling "notmuch tag" unnecessarily.
Carl Worth [Tue, 3 Nov 2009 21:13:50 +0000 (13:13 -0800)]
notmuch show: Remove custom "unread" hack, (printing tag in two locations).
I previously had a hack that special-cased the "unread" tag and
printed it on the same line as the message ID. But now that we are
printing all tags at the end of the one-line summary we don't need
this anymore. Get rid of it, and just read "unread" from the list of
tags just like any other tag.
Carl Worth [Tue, 3 Nov 2009 20:54:10 +0000 (12:54 -0800)]
notmuch.el: Add + and - bindings to add/remove tags from messages.
This is in notmuch-show mode rather than in notmuch-search mode,
(where we had + and - working already). This gives the same visual
feedback as in notmuch-search-mode, (the tags are manipulated first in
the database and then the list of tags in the buffer is updated).
Carl Worth [Tue, 3 Nov 2009 05:13:19 +0000 (21:13 -0800)]
notmuch show: Switch to control character to mark sections of output
We were previously using things like "%message{" which were not
guaranteed to never appear in an email message. Using a control
character (^L or '\f' instead of '%') gives us better assurance that
our delimiter doesn't show up in an original email message.
This still isn't entirely safe since we're decoding encoded text in
the body of the email message so almost all bets are off really.
Carl Worth [Tue, 3 Nov 2009 02:45:17 +0000 (18:45 -0800)]
notmuch.el: Add (n)ext and (p)revious bindings to notmuch-show mode.
Almost starting to get usable now. Still need to make it mark messages
as they are read, (by removing the unread tag), and selectively hiding
the full header.
Carl Worth [Tue, 3 Nov 2009 00:08:24 +0000 (16:08 -0800)]
Remove notmuch_message_get_header_size and notmuch_message_get_all_headers
The notmuch.c main program now uses GMime directly rather than using
these functions, and I'd rather not export any functions unless we
have good evidence that the functions are necessary.
Carl Worth [Mon, 2 Nov 2009 22:36:33 +0000 (14:36 -0800)]
Drop date.c file, (use identical function from GMime instead).
We had originally copied this function in at a time when notmuch
wasn't actually depending on the GMime library. Now that it does,
we might as well call the function that exists there rather
than having a private copy of it.
Carl Worth [Mon, 2 Nov 2009 17:11:37 +0000 (09:11 -0800)]
Add a simple configure script.
This is *not* based on autoconf. In fact, this doesn't actually
configure anything, (one can compile notmuch directly with just
"make" without running configure if the dependencies are all
satisfied).
The only thing that this configure script does is to check for the
presence of the various dependencies and provide some guidance to
the user if they are not all available.
Carl Worth [Sat, 31 Oct 2009 23:40:47 +0000 (16:40 -0700)]
notmuch: Add a talloc context argument to each top-level command function.
I had noticed several times earlier that having a talloc context
passed in would make things more convenient. I'm not exercising
that convenience yet, but the context is there now, (and there's
one fewer item on our TODO list).
Carl Worth [Sat, 31 Oct 2009 07:44:39 +0000 (00:44 -0700)]
notmuch.el: Switch from start-process to call-process
We now get the point staying right at the top where we want it.
We also don't get any extraneous noise about "Process notmuch
completed" or anything like that. Just the output in a read-only
buffer.
Carl Worth [Sat, 31 Oct 2009 06:42:39 +0000 (23:42 -0700)]
notmuch.el: Switch from compilation-start to start-process
Compilation mode does a bunch of things that we don't want. Instead
of trying to tear it down to what we want, let's start at the other
end and build up only things that we really want.
Carl Worth [Sat, 31 Oct 2009 00:17:16 +0000 (17:17 -0700)]
notmuch.el: Copy copyright information from compilation.el
I'm using that file as my reference here, so I'm likely to end up
copying some code here or there. Might as well be safe and just
copy the copyright statement.
Carl Worth [Fri, 30 Oct 2009 22:12:49 +0000 (15:12 -0700)]
notmuch-mode: Add an actualy notmuch-search-mode as well
Doesn't really do anything so far other than mark the buffer read-
only. This does have the benefit of giving us our own name rather
than "Compilation" for the mode.
Carl Worth [Fri, 30 Oct 2009 18:57:43 +0000 (11:57 -0700)]
TODO: Note that "notmuch show" exists now and list several new ideas.
The timestamp stuff we'll want to do soon, since it's a database
change, (though not a major one---at worst a handful of stale
timestamp documents would be left in the database).
Carl Worth [Fri, 30 Oct 2009 00:42:21 +0000 (17:42 -0700)]
Fix relative date formatting to not split one day into two formats.
We were aware of this bug when we wrote the function, (that a date
six days in the past would be treated as the "Friday" or as the
"Oct. 23" case depending on whether its time was before or after
the current time today). We thought it wouldn't be a problem, but
in practice it is. In scanning search results with this output,
the transition between formats makes it look like a day boundary,
(so it would be easy to mistakenly think "Oct. 23" is Thursday).
Fix this to avoid confusion, (still being careful to never print
"Thursday" for a date 7 days in the past when today is Thursday).
We're using a delimiter syntax that Keith is optimistic about
being able to easily parse in emacs. Note: We're not escaping
any occurrence of the delimiters in the message yet, so we'll
need to fix that.
Carl Worth [Wed, 28 Oct 2009 23:09:19 +0000 (16:09 -0700)]
notmuch setup/new: Print progress once per second instead of after 1000 files.
With the recent addition of full-text indexing, printing only once per
1000 files just isn't often enough. The new timer-based approach will
be reliable regardless of the speed of adding message.
Carl Worth [Wed, 28 Oct 2009 22:41:42 +0000 (15:41 -0700)]
index: Don't bother indexing quoted portions of messages (and signatures).
Our old notmuch-index-message.cc code had this, but I originally
left it out when adding indexing back in. I was concerned primarily
with mistakenly detecting signature markers and omitting important
text, (for example, I often do long lines of "----" as section
separators).
But now I see that there's a performance benefit to skippint the
quotations, (about 120 files/sec. instead of 95 files/sec.). I mitigated
the bogus signature checking by recognizing nothing other than the
all-time classic "-- ".
Carl Worth [Wed, 28 Oct 2009 20:35:10 +0000 (13:35 -0700)]
notmuch_database_add_message: Sanity check the file as the first thing
This avoids us wasting a bunch of time doing an expensive SHA-1 over a large
file only to discover later that it doesn't even *look* like an email message.
Carl Worth [Wed, 28 Oct 2009 20:09:08 +0000 (13:09 -0700)]
index: Store "Full Name <user@example.com>" addressses in the database
We put these is as a separate term so that they can be extracted.
We don't actually need this for searching, since typing an email
address in as a search term will already trigger a phrase search
that does exactly what's wanted.
Carl Worth [Wed, 28 Oct 2009 17:42:07 +0000 (10:42 -0700)]
Add full-text indexing using the GMime library for parsing.
This is based on the old notmuch-index-message.cc from early in
the history of notmuch, but considerably cleaned up now that
we have some experience with Xapian and know just what we want
to index, (rather than just blindly trying to index exactly
what sup does).
This does slow down notmuch_database_add_message a *lot*, but I've
got some ideas for getting some time back.
Carl Worth [Wed, 28 Oct 2009 17:35:26 +0000 (10:35 -0700)]
notmuch search: Clarify documentation of implicit Boolean operators
The original documentation of implicit AND is what we want, but
Xapian doesn't actually let us get that today. So be honest about
what the user can actually expect. And let's hope the Xapian
wizards give us the feature we want soon:
Carl Worth [Wed, 28 Oct 2009 08:46:24 +0000 (01:46 -0700)]
TODO: Remove a couple of since-completed items.
"notmuch tag" is implemented now and seems to work great (and fast).
As for the race condition, as noted in the description we're removing
it's not exposed directly in the API, but only in a client that
allows for looping over search results and removing the inbox tag
from all of them. But then, that's exactly what the "notmuch tag"
command does. So, as discussed, we've now documented that command
to highlight the issue. Problem resolved, (as well as we can).
Carl Worth [Wed, 28 Oct 2009 07:58:26 +0000 (00:58 -0700)]
notmuch help: Be less verbose by default and support detailed help
Putting all of our documentation into a single help message was getting
a bit unwieldy. Now, the simple output of "notmuch help" is a reasonable
reminder and a quick reference. Then we now support a new syntax of:
"notmuch help <command>" for the more detailed help messages.
This gives us freedom to put more detailed caveats, etc. into some
sub-commands without worrying about the usage statement getting too
long.
Carl Worth [Wed, 28 Oct 2009 06:57:37 +0000 (23:57 -0700)]
Fix segfault in case of the database lock not being available.
We were nicely reporting the lock-aquisition failure, but then marching
along trying to use the database object and just crashing badly.
So don't do that.
Carl Worth [Wed, 28 Oct 2009 06:55:08 +0000 (23:55 -0700)]
Update prefix so that "thread:" can be used in search strings.
It's convenient to be able to do things like:
notmuch tag -inbox thread:<thread-id>
(even though this can run into a race condition as noted in TODO--the fix
for the race is simply to not run "notmuch new" between reading a thread
with the (not yet existent) "notmuch show" and removing its inbox tag
with a command like the above). So we now allow such a thing.
Carl Worth [Wed, 28 Oct 2009 00:07:14 +0000 (17:07 -0700)]
Add new "notmuch tag" command for adding/removing tags.
This uses the same search functionality as "notmuch search" so
it should be quite powerful. And this global search might be
quick enough to be used for "automatic" adding of tags to new
messages.
Of course, this will all be a lot more useful when we can search
for actual text of messages and not just tags.
Carl Worth [Tue, 27 Oct 2009 23:19:20 +0000 (16:19 -0700)]
notmuch_database_add_message: Do not return a message on failure.
The recent, disastrous failure of "notmuch new" would have been
avoided with this change. The new_command function was basically
assuming that it would only get a message object on success so
wasn't destroying the message in the other cases.
Carl Worth [Tue, 27 Oct 2009 23:17:22 +0000 (16:17 -0700)]
notmuch_database_close: Explicitly flush the Xapian database.
This would have helped with the recent bug causing "notmuch new"
to not record any results in the database. I'm not sure why
the explicit flush would be required, (shouldn't the destructor
always ensure that things flush?), but perhaps some outstanding
references from the leak prevented that.
In any case, an explicit flush on close() seems to make sense.
Carl Worth [Tue, 27 Oct 2009 23:12:04 +0000 (16:12 -0700)]
Merge branch to fix broken "notmuch setup" and "notmuch new"
I'm trying to stick to a habit of fixing previously-introduced bugs
on side branches off of the commit that introduced the bug. The
idea here is to make it easy to find the commits to cherry pick
if bisecting in the future lands on one of the broken commits.
Carl Worth [Tue, 27 Oct 2009 23:07:27 +0000 (16:07 -0700)]
Fix "notmuch new" (bad performance, and no committing of results).
We were incorrectly only destroying messages in the case of
successful addition to the database, and not in other cases,
(such as failure due to FILE_NOT_EMAIL).
I'm still not entirely sure why this was performing abysmally, (as in
making an operation that should take a small fraction of a second take
10 seconds), nor why it was causing the database to entirely fail to
get new results.
Carl Worth [Tue, 27 Oct 2009 19:00:58 +0000 (12:00 -0700)]
Unbreak the "notmuch setup" command.
The recent addition of support for automatically adding tags to
new messages for "notmuch new" caused "notmuch setup" to segfault.
The fix is simple, (just need to move a destroy function to inside
a nearby if block).
Did I mention recently we need to add a test suite?
Carl Worth [Tue, 27 Oct 2009 17:19:46 +0000 (10:19 -0700)]
TODO: More notes on archive-thread and race conditions.
Interstingly, it's our simple "notmuch" client that's going to be the
most difficult to fix. There's just not as much information preserved
in the textual representation from "notmuch search" as there is in the
objects returned from notmuch_query_search_threads.
Carl Worth [Tue, 27 Oct 2009 17:04:48 +0000 (10:04 -0700)]
TODO: Add "notmuch tag" and thoughts on avoiding races in archiving threads.
The archive-thread race condition doesn't even exist now because there's
no command for modifying tags at the level of a thread (just individual
messages).
Carl Worth [Tue, 27 Oct 2009 03:11:58 +0000 (20:11 -0700)]
Fix incorrect name of _notmuch_thread_get_subject.
Somehow this naming with an underscore crept in, (but only in the
private header, so notmuch.c was compiling with no prototype). Fix
to be the notmuch_thread_get_subject originally intended.