Chris Wilson [Sat, 21 Nov 2009 19:54:25 +0000 (19:54 +0000)]
Permit opening the notmuch database in read-only mode.
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.
Signed-off-by: Chris Wilson <chris@chris-wilson.co.uk> Acked-by: Carl Worth <cworth@cworth.org>
The feature makes a lot of sense for the initial import, but it's not
as clear whether it makes sense for ongoing "notmuch new" runs. We
might need to make this opt-in by configuration.
Carl Worth [Sat, 21 Nov 2009 10:38:27 +0000 (11:38 +0100)]
Makefile: Fix the fallback emacs install path.
When pkg-config can't be used to find out where to install emacs
files, we fallback to a hard-coded directory. Only, we were falling
back to the wrong thing, (one that emacs doesn't look into by
default).
Carl Worth [Sat, 21 Nov 2009 00:00:41 +0000 (01:00 +0100)]
notmuch show: Don't hide a digital signature.
This was a bug that was introduced in copying the indexing code over
into notmuch-show.c. When indexing, we want to ignore the signature,
(it has no interesting terms). But when presenting the message, it's
important to present the signature to the user.
(And would be even better if we presented whether or not the signature
is good.)
Carl Worth [Fri, 20 Nov 2009 18:31:00 +0000 (19:31 +0100)]
add_message: Properly handle missing Message-ID once again.
There's been a fair amount of fallout from when we changed
message_file_get_header from returning NULL to returning "" for
missing headers. This is yet more fallout from that, (where we were
accepting an empty message-ID rather than generating one like we want
to).
Carl Worth [Fri, 20 Nov 2009 12:09:58 +0000 (13:09 +0100)]
notmuch.el: Don't use literal control characters in strings.
Avoding these is nicer to users, text editors, and our poor little
notmuch.el code itself that would get confused when seeing a copy of
itself in email. (Of course, we should still fix that bug, but this
workaround is good nonetheless.)
Carl Worth [Fri, 20 Nov 2009 11:58:42 +0000 (12:58 +0100)]
Makefile: Remove unused variable emacs_startdir
This was added in a prelimnary version of a previous commit that would
automatically load notmuch.el for anyone running emacs. It's not used
at all in the current Makefile.
Carl Worth [Fri, 20 Nov 2009 11:06:11 +0000 (12:06 +0100)]
Avoid access of a Xapian iterator's object when there's nothing there.
This eliminates a crash when a message (either corrupted or a non-mail
file that wasn't properly detected as not being mail) has no In-Reply-To
header, (and so few terms that trying to skip to the prefix of the
In-Reply-To terms actually brings us to the end of the termlist).
Adrian Perez [Thu, 19 Nov 2009 00:07:22 +0000 (01:07 +0100)]
Allow lone "not" search operators
As suggested by Keith in FLAG_PURE_NOT allows for expressions like:
notmuch search NOT tag:inbox
Note that this way a search like:
notmuch search foobar NOT tag:inbox
should not be written instead:
notmuch search foobar AND NOT tag:inbox
In my opinion, the latter feels more natural and is somewhat more explicit.
It gives a better clue of what the search is about instead of assuming that
an implicit AND operator is there.
Keith Packard [Wed, 18 Nov 2009 21:43:38 +0000 (13:43 -0800)]
Make reply/show/tag all require at least one search term
In particular, notmuch tag -inbox "" tended to take a long time to
run, happened if you hit 'a' on a blank line in the search view and
probably didn't have the desired effect.
Keith Packard [Wed, 18 Nov 2009 08:28:14 +0000 (00:28 -0800)]
Create a default notmuch-show-hook that highlights URLs and uses word-wrap
I created the notmuch-show-hook precisely so I could add these two
options, but I suspect most people will want them, so I just made them
the default. If you don't want them, you can use remove-hook to get
rid of this.
Keith Packard [Tue, 17 Nov 2009 06:06:49 +0000 (22:06 -0800)]
Add notmuch-show-hook to allow customization of show windows
I wanted to enable got-address-mode and visual-line-mode in my show
windows to make messages easier to read and URLs easier to
follow. This hook allows the user to run arbitrary code each time a
message is shown.
Stewart Smith [Wed, 18 Nov 2009 02:22:20 +0000 (13:22 +1100)]
count_files: sort directory in inode order before statting
Carl says: This has similar performance benefits as the previous
patch, and I fixed similar style issues here as well, (including
missing more of a commit message than the one-line summary).
Stewart Smith [Wed, 18 Nov 2009 01:56:40 +0000 (12:56 +1100)]
Read mail directory in inode number order
This gives a rather decent reduction in number of seeks required when
reading a Maildir that isn't in pagecache.
Most filesystems give some locality on disk based on inode numbers.
In ext[234] this is the inode tables, in XFS groups of sequential inode
numbers are together on disk and the most significant bits indicate
allocation group (i.e inode 1,000,000 is always after inode 1,000).
With this patch, we read in the whole directory, sort by inode number
before stat()ing the contents.
Ideally, directory is sequential and then we make one scan through the
file system stat()ing.
Since the universe is not ideal, we'll probably seek during reading the
directory and a fair bit while reading the inodes themselves.
However... with readahead, and stat()ing in inode order, we should be
in the best place possible to hit the cache.
In a (not very good) benchmark of "how long does it take to find the first
15,000 messages in my Maildir after 'echo 3 > /proc/sys/vm/drop_caches'",
this patch consistently cut at least 8 seconds off the scan time.
Without patch: 50 seconds
With patch: 38-42 seconds.
(I did this in a previous maildir reading project and saw large improvements too)
Stewart Smith [Wed, 18 Nov 2009 01:05:53 +0000 (12:05 +1100)]
Fix linking with gcc to use g++ to link in C++ libs.
Previously, Ubuntu 9.10, gcc 4.4.1 was getting:
/usr/bin/ld: lib/notmuch.a(database.o): in function global
constructors keyed to BOOLEAN_PREFIX_INTERNAL:database.cc(.text+0x3a):
error: undefined reference to 'std::ios_base::Init::Init()'
Carl Worth [Wed, 18 Nov 2009 11:29:30 +0000 (03:29 -0800)]
notmuch search: Avoid infinite stream of exceptions from "notmuch search"
That is, give a nice error message and exit if no search terms are
provided. Thanks to Priit Laes <plaes@plaes.org> for reporting the
error and providing an early version of the fix.
Jan Janak [Tue, 17 Nov 2009 22:18:47 +0000 (23:18 +0100)]
Older versions of install do not support -C.
Do not use -C cmdline option of install, older versions, commonly found in
distributions like Debian, do not seem to support it. Running make install
on such systems (tested on Debian Lenny) fails.
which was adding extra checks to avoid adding a self-referencing
message.
How many times am I going to fix a dumb regression like this and say
"we really need a test suite" before I actually sit down and write the
test suite?
Carl Worth [Wed, 18 Nov 2009 07:37:21 +0000 (23:37 -0800)]
notmuch help: Update documentation (following recent text from notmuch.1)
We take the recently created text from the notmuch manual page and
update the "notmuch help" command to use similar text. In particular,
we add a new "notmuch help search-terms" for documenting the search
syntax that is common to several commands.
Carl Worth [Wed, 18 Nov 2009 06:16:35 +0000 (22:16 -0800)]
man.1: A big update of the notmuch manual page.
I set out merely to add documentation for the recently-added options
for "notmuch search" (--first, --max-threads, and --sort), but ended
up revamping a lot. A significant change is a new SEARCH SYNTAX
section separate from "notmuch search" that is referred to in the
documentation of search, show, reply, and tag.
Also many sections were updated to reflect recent changes, (such as
the dropping of the NOTMUCH_BASE environment variable, the addition of
the .notmuch-config file, etc.)
Carl Worth [Wed, 18 Nov 2009 04:52:09 +0000 (20:52 -0800)]
notmuch search: Change default search order to be newest messages first.
This is what most people want for a _search_ command. It's often
different for actually reading mail in an inbox, (where it makes more
sense to have results displayed in chronological order), but in such a
case, ther user is likely using an interface that can simply pass the
--sort=oldest-first option to "notmuch search".
Here we're also change the sort enum from NOTMUCH_SORT_DATE and
NOTMUCH_SORT_DATE_REVERSE to NOTMUCH_SORT_OLDEST_FIRST and
NOTMUCH_SORT_NEWEST_FIRST. Similarly we replace the --reverse option
to "notmuch search" with two options: --sort=oldest-first and
--sort=newest-first.
Finally, these changes are all tracked in the emacs interface, (which
has no change in its behavior).
Carl Worth [Wed, 18 Nov 2009 03:11:05 +0000 (19:11 -0800)]
notmuch search: Return first 100 results as quickly as possible.
This is one of those cases where total time is not the metric of
interest. We increase the total time of the search, (by doing some
redundant work for the initial threads). But more significantly, we
give the user *some* results nearly instantaneously, (so that the user
might see the result of interest without ever even waiting for the
complete results to come in).
Carl Worth [Wed, 18 Nov 2009 02:50:13 +0000 (18:50 -0800)]
database: Make _parse_message_id static once again.
We had exposed this to the internal implementation for a short time,
(only while we had the silly code fetching In-Reply-To values from
message files instead of from the database). Make this private again
as it should be.
Carl Worth [Wed, 18 Nov 2009 02:48:38 +0000 (18:48 -0800)]
database: Add "replyto" to the database schema documentation.
Maybe ths lack of this documentation is why I forgot we were actually
storing this and wrote the ugly code to fetch In-Reply-To from message
files rather than from the database.
Carl Worth [Wed, 18 Nov 2009 02:44:02 +0000 (18:44 -0800)]
database: Rename "ref" prefix name to "reference"
Which is more consistent with the XREFERENCE prefix used in the terms
in the database. Also remove some stale documentation describing the
removal of resolved references from the database (we no longer do
this).
Carl Worth [Wed, 18 Nov 2009 02:37:45 +0000 (18:37 -0800)]
message_file_get_header: Use break where more clear than continue.
Calling continue here worked only because we set a flag before the
continue, and, check the flag at the beginning of the loop, and *then*
break. It's much more clear to just break in the first place.
Keith Packard [Wed, 18 Nov 2009 02:33:42 +0000 (18:33 -0800)]
Fix "too many open files" bug by closing message files when done with them.
The message file header parsing code parses only enough of the file to
find the desired header fields, then it leaves the file open until the
next header parsing call or when the message is no longer in use. If a
large number of messages end up being active, this will quickly run
out of file descriptors.
Here, we add support to explicitly close the message file within a
message, (_notmuch_message_close) and call that from thread
construction code.
Signed-off-by: Keith Packard <keithp@keithp.com> Edited-by: Carl Worth <cworth@cworth.org>:
Many portions of Keith's original patch have since been solved other
ways, (such as the code that changed the handling of the In-Reply-To
header). So the final version is clean enough that I think even Keith
would be happy to have his name on it.
Carl Worth [Wed, 18 Nov 2009 01:53:54 +0000 (17:53 -0800)]
notmuch show: Detect an internal error if a thread has no messages.
This really should be impossible---if there are no messages, then what
was the thread object created from? During recent debugging, it was
useful to have this error detected and reported.
Carl Worth [Wed, 18 Nov 2009 01:20:32 +0000 (17:20 -0800)]
add_message: Don't add any self-references to the database.
In our scheme it's illegal for any message to refer to itself, (nor
would it be useful for anything anyway). Cut these self-references off
at the source, before they trip up any internal errors.
Carl Worth [Tue, 17 Nov 2009 19:07:38 +0000 (11:07 -0800)]
message_get_thread_id: Generate internal error if message has no thread ID.
This case was happening when a message had its own message ID in its
In-Reply-To header. The thread-resolution code would find the
partially constructed message, (with no thread ID yet), get garbage
from this function, and then march right along with that garbage.
With this commit, a self-cyclic message like this will now trigger an
internal error rather than marching along silienty. (And a subsequent
commit will remove the call to this function in this case.)
Carl Worth [Tue, 17 Nov 2009 19:02:33 +0000 (11:02 -0800)]
Remove the talloc_owner argument from create_for_message_id.
This function has only one caller, and that one caller was passing the
same value for both talloc_owner and the notmuch database. Dropping
the redundant argument simplifies the documentation of this function
considerably.
Carl Worth [Wed, 18 Nov 2009 01:40:19 +0000 (17:40 -0800)]
get_in_reply_to: Implement via the database, not by opening mail file.
This reduces our reliance on open message_file objects, (so is a step
toward fixing the "too many open files" bug), but more importantly, it
means we don't load a self-referencing in-reply-to header, (since we
weed those out before adding any replyto terms to the database).
Carl Worth [Tue, 17 Nov 2009 21:05:16 +0000 (13:05 -0800)]
Makefile: Change default install prefix from /usr to /usr/local
We'll be a much more polite package this way. And the user can change
the prefix by editing Makefile.config. Still to be done is to make
configure write out Makefile.config and to add a --prefix option to
configure.
Carl Worth [Tue, 17 Nov 2009 20:50:46 +0000 (12:50 -0800)]
Makefile: Prefer directories as the target for install commands.
I was confusing myself with some rules installing to directories and
some installing to files. We do still install to a filename when
simultaneously renaming, (such as notmuch-completion.bash to notmuch).