Austin Clements [Sun, 6 Nov 2011 17:17:36 +0000 (12:17 -0500)]
Store "from" and "subject" headers in the database.
This is a rebase and cleanup of Istvan Marko's patch from
id:m3pqnj2j7a.fsf@zsu.kismala.com
Search retrieves these headers for every message in the search
results. Previously, this required opening and parsing every message
file. Storing them directly in the database significantly reduces IO
and computation, speeding up search by between 50% and 10X.
Taking full advantage of this requires a database rebuild, but it will
fall back to the old behavior for messages that do not have headers
stored in the database.
emacs: add notmuch-show-worker function for specifying crypto processing directly
The main reason to introduce this new unexposed function is to allow
the buffer redisplay crypto switch to behaving in a more expected way.
The prefix to notmuch-show-redisplay buffer now switches the crypto
processing of the current show buffer, as opposed to switching the
logic of the notmuch-crypto-process-mime customization variable. This
behavior is more intuitive.
Do not redirect test_emacs stderr to /dev/null. Test_emacs uses
emacsclient(1) now and it does not print unwanted messages (like
those from `message') to stderr. But it does print useful
errors, e.g. when emacs server connection fails, given expression
is not valid or undefined function is called.
David Bremner [Sun, 13 Nov 2011 01:24:25 +0000 (21:24 -0400)]
contrib/nmbug: new script for sharing tags with a given prefix.
The main idea is consider the notmuch database as analogous to the
work-tree. A bare git repo is maintained in the users home directory,
with a tree of the form tags/$message-id/$tag
Like notmuch and git, we have a set of subcommnds, mainly modelled on
git.
Implementation wise, the heavy lifting is in the following functions.
commit xapian -> git
checkout git -> xapian
merge fetched git + git -> xapian
status find differences between xapian, git, and remote git.
The central implementation trick, from an idea I think due to
tomprince on IRC is manipulate the git index directly from the xapian
tag information. The merge routine is still done using a temporary
checkout as I wasn't able to get it working with the index only.
There are also some convenience wrappers around git commands, like "fetch"
that essential just set GIT_DIR in the environment.
In order to encode tags (viewed as octet sequences) into filenames,
we whitelist a smallish set of characters and %hex escape anything outside.
The prefix is omitted in git, which lets one save and restore to
different prefixes (although this is only lightly tested).
Thanks to Tomi Ollila for a huge amount of feedback and patches while
putting this together.
Pieter Praet [Thu, 30 Jun 2011 08:20:50 +0000 (10:20 +0200)]
emacs: add keybind and function to stash Message-ID without prefix
Add function `notmuch-show-stash-message-id-stripped'
which stashes a Message-ID after ripping off the prefix and quotes,
add bind it to "I" key in `notmuch-show-stash-map'.
Simplifying `notmuch-show-get-message-id' instead might seem better,
but that would require concat'ing in 9 places instead of 1.
emacs: add invisible space after the search widget field in notmuch-hello
It is very convenient when C-e (bound to `widget-end-of-line') ignores
trailing spaces inside the search widget. But it only does so if a
widget is not followed by a newline (that is why it works in the saved
search widgets). The patch just adds an invisible space after the
search widget to get the desirable behavior of `widget-end-of-line'.
The extra space is also added to expected results of emacs tests.
Austin Clements [Wed, 9 Nov 2011 02:55:28 +0000 (21:55 -0500)]
emacs: Use a single buffer invisibility spec to fix quadratic search cost.
Buffer redisplay requires traversing the buffer's invisibility spec
for every part of the display that has an 'invisible text or overlay
property. Previously, the search buffer's invisibility spec list
contained roughly one entry for each search result. As a result,
redisplay took O(NM) time where N is the number of visible lines and M
is the total number of results. On a slow computer, this is enough to
make even buffer motion noticeably slow. Worse, during a search
operation, redisplay is triggered for each search result (even if
there are no visible buffer changes), so search was quadratic
(O(NM^2)) in the number of search results.
This change switches to using a single element buffer invisibility
spec. To un-hide authors, instead of removing an entry from the
invisibility spec, it simply removes the invisibility overlay from
those authors.
I tested using a query with 6633 results on a 9 year old machine.
Before this patch, Emacs took 70 seconds to fill the search buffer;
toward the end of the search, Emacs consumed 10-20x as much CPU as
notmuch; and moving point in the buffer took about a second. With
this patch, the same query takes 40 seconds, Emacs consumes ~3x the
CPU of notmuch by the end, and there's no noticeable lag to moving
point. (There's still some source of non-linearity, because Emacs and
notmuch consume roughly the same amount of CPU early in the search.)
Michal Sojka [Sat, 27 Aug 2011 12:38:26 +0000 (14:38 +0200)]
Do not query on notmuch-search exit
Emacs 23.2 queries by default about killing existing processes. This
is annoying when one wants to interrupt long search with 'q' key.
Disable this behavior for notmuch.
`point-invisible-p' does not work correctly when `invisible'
property is a list. There are standard `invisible-p' and related
functions that should be used instead.
emacs: improve hidden signatures handling in notmuch-show-advance-and-archive
Use `previous-single-char-property-change' instead of going
through each character by hand and testing it's visibility. This
fixes `notmuch-show-advance-and-archive' to work for the last
message in thread with hidden signature.
Set SCREENRC and SYSSCREENRC environment variables to "/dev/null"
as suggested by Jim Paris to avoid potential problems with
screen(1) configuration files.
Before the change, emacs run in daemon mode without any visible
buffers. Turns out that this affects emacs behavior in some
cases. In particular, `window-end' function returns `point-max'
instead of the last visible position. That makes it hard or
impossible to implement some tests. The patch runs emacs in a
detached screen(1) session. So that it works exactly as if it
has a visible window.
Note: screen terminates when emacs exits. So the patch does not
introduce new "running processes left behind" issues.
David Bremner [Fri, 28 Oct 2011 14:56:30 +0000 (11:56 -0300)]
lib/database.cc: use flush Xapian method instead of commit
Apparently the method was renamed in Xapian 1.1.0 but the old method
name will stay around for a while. It seems better to stick with the
old name to make notmuch compile with older versions of Xapian, at
least for now.
Tomi Ollila [Mon, 31 Oct 2011 12:01:28 +0000 (14:01 +0200)]
remove GCC visibility pragmas
libnotmuch.so.* linking fail on some environments. According to
David Bremner on irc:
"We jump through hoops with the linker script (notmuch.sym) so
the pragmas are not needed. And they are a little bizarre in a
library anyway..."
David Bremner [Sun, 23 Oct 2011 20:52:19 +0000 (17:52 -0300)]
xregcomp: don't consider every regex compilation failure an internal error.
This pushes the error handling up one step, but makes the function
more flexible. Running out of memory still triggers an internal error,
in the spirit of other xutils functions.
David Bremner [Sun, 23 Oct 2011 15:05:13 +0000 (12:05 -0300)]
xutil.c: remove duplicate copies, create new library libutil.a to contain xutil.
We keep the lib/xutil.c version. As a consequence, also factor out
_internal_error and associated macros. It might be overkill to make a
new file error_util.c for this, but _internal_error does not really
belong in database.cc.
David Bremner [Thu, 13 Oct 2011 01:23:33 +0000 (22:23 -0300)]
build system: target to make a Debian snapshot package.
Currently this builds a native package, but since the source package
is throw away, it should not matter too much, except for the extra
warnings from lintian.
The extra +1 is so that if $(VERSION) is the same as the last released
version (for example outside a git repo) then the versions still order
correctly.
David Bremner [Wed, 19 Oct 2011 21:18:24 +0000 (18:18 -0300)]
notmuch-restore: implement --accumulate option
Modify command line argument handling to take a --accumulate flag.
Test for extra arguments beyond the input file.
The --accumulate switch causes the union of the existing and new tags to be
applied, instead of replacing each message's tags as they are read in from the
dump file.
David Bremner [Fri, 21 Oct 2011 16:04:46 +0000 (13:04 -0300)]
test/dump-restore: add tests for restore --accumulate
Flesh out what ``notmuch restore --accumulate'' is supposed to do.
Its tests are currently XFAILed; the functionality will be added in
future patch(es).
David Bremner [Fri, 21 Oct 2011 00:14:30 +0000 (21:14 -0300)]
test/dump-restore: Fix quoting on grep
Thanks to Thomas Schwinge for noticing yet another place where quoting
matters. Since the shell translates \. to ., the regex passed to grep
is too generous without the quotes.
David Bremner [Wed, 19 Oct 2011 21:03:12 +0000 (18:03 -0300)]
test/dump-restore: expand test suite for dump-restore, make more robust
Several new tests are added, and existing use of test_begin_subtest is
replaced by test_expect_success to catch failing commands in cases where
we execute more than one command.
David Bremner [Fri, 21 Oct 2011 12:19:17 +0000 (09:19 -0300)]
cli: change argument parsing convention for subcommands
previously we deleted the subcommand name from argv before passing to
the subcommand. In this version, the deletion is done in the actual
subcommands. Although this causes some duplication of code, it allows
us to be more flexible about how we parse command line arguments in
the subcommand, including possibly using off-the-shelf routines like
getopt_long that expect the name of the command in argv[0].
David Bremner [Sun, 9 Oct 2011 13:29:22 +0000 (10:29 -0300)]
test: update dump-restore to use redirection instead of filename args
The idea here is that we want to deprecate the use of arguments to
dump and restore to specify paths, since in particular we want to use
the non-option arguments to dump to form a query.
The first test tests that the notmuch-show-refresh-view function
produces the exact same output for an unmodified show buffer. This
test should pass since the relevant functionality has already been
applied.
The second test tests show refresh for a show buffer that has been
modified by navigation and message visibility toggling. Ideally
refresh-view should preserve this state of the notmuch-show buffer.
Unfortunately it currently does not, so this test is know to be broken
and is marked as such.
This function, like the equivalent for notmuch-search, just refreshes
the current show view. Like in notmuch-search, this new function is
bound to "=". If a prefix is given then the redisplay happens with the
crypto-switch set, which displays the thread with the opposite logic
of whatever is set in the notmuch-crypto-process-mime customization
variable.
emacs: Add callback functions to crypto sigstatus button.
This adds two callback functions to the sigstatus button. If the sig
status is "good", then clicking the button displays the output of "gpg
--list-keys" on the key fingerprint. If the sigstatus is "bad", then
clicking the button will retrieve the key from the keyserver, and
redisplay the current buffer.
Thanks to David Bremner <bremner@unb.ca> for help with this.
test/multipart: test for absence of "Non-text part:" lines in reply for multipart/* and message/rfc822 parts
There's no reason to output "Non-text part:" lines for parts that are
not leaf nodes, eg. multipart/* and message/rfc822. We fix the text
here to test for their absence. The next patch will fix reply
accordingly.
Technically, this is a superfluous change, as the self.status variable
currently gets set in NotmuchErrors's __new__ function. However, in the
long run I would like to get rid of the weird __new__ implementation which
might be somewhat confusing for users (NotmuchError(status) returns a
different class, e.g. OutOfMemoryError)
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
python: help function Query._assert_query_is_initialized
Remove code duplication by using the new helper function. Also raise the
new fine grained exceptions in many cases, rather than the more generic
NotmuchErrors.
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
Catch up with the major version bump. I wonder if this could somehow be
automatically made the correct version number. Oh well, I hope it
doesn't change too often :-).
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
David Bremner [Tue, 4 Oct 2011 17:01:21 +0000 (14:01 -0300)]
debian: update packaging for new soname
we need
- a new changelog stanza, because the symbols files need a new version
- s/libnotmuch1/libnotmuch2/ everywhere
- update symbols file, s/.so.1/.so.2/, and bump minimum versions on changed
symbols (although the latter is just documentation)
David Bremner [Tue, 4 Oct 2011 16:47:04 +0000 (13:47 -0300)]
lib: bump SONAME
Based on discussions with amdragon, tschwinge, and others on IRC, I concluded that
1) symbol versioning was probably overkill for libnotmuch
2) It was also probably GNU ld specific
3) Most importantly, nobody could tell me on short notice how exactly it works.
So since the change to the notmuch_database_find_message breaks the
previous ABI, we need to bump the SONAME.
Ali Polatel [Tue, 4 Oct 2011 13:48:34 +0000 (16:48 +0300)]
ruby: Really add wrappers for database_find_message*
Commit 898613116db746aa0f915ae43da8aba28545203d only added wrapper
functions but did not register them. Register the functions in module's
initialization function.
Ali Polatel [Tue, 4 Oct 2011 13:06:20 +0000 (16:06 +0300)]
ruby: Add wrappers for database_find_message*
Two new wrappers:
Notmuch::Database.find_message(id) => Notmuch::Message or nil
Notmuch::Database.find_message_by_filename(path) => Notmuch::Message or nil
projects / notmuch / log