notmuch/lib, branch 0.18.2

lib: Separate all phrases indexed by _notmuch_message_gen_terms

2014-06-18T21:03:18Z

This adds a 100 termpos gap between all phrases indexed by _notmuch_message_gen_terms. This fixes a bug where terms from the end of one header and the beginning of another header could match together in a single phrase and a separate bug where term positions of un-prefixed terms overlapped. This fix only affects newly indexed messages. Messages that are already indexed won't benefit from this fix without re-indexing, but the fix won't make things any worse for existing messages.

lib: Index name and address of from/to headers as a phrase

2014-06-18T20:55:14Z

Previously, we indexed the name and address parts of from/to headers with two calls to _notmuch_message_gen_terms. In general, this indicates that these parts are separate phrases. However, because of an implementation quirk, the two calls to _notmuch_message_gen_terms generated adjacent term positions for the prefixed terms, which happens to be the right thing to do in this case, but the wrong thing to do for all other calls. Furthermore, _notmuch_message_gen_terms produced potentially overlapping term positions for the un-prefixed copies of the terms, which is simply wrong. This change indexes both the name and address in a single call to _notmuch_message_gen_terms, indicating that they should be part of a single phrase. This masks the problem with the un-prefixed terms (fixing the two known-broken tests) and puts us in a position to fix the unintentionally phrases generated by other calls to _notmuch_message_gen_terms.

lib: resurrect support for single-message mbox files

2014-06-14T01:59:04Z

This is effectively a revert of commit 6812136bf576d894591606d9e10096719054d1f9 Author: Jani Nikula Date: Mon Mar 31 00:21:48 2014 +0300 lib: drop support for single-message mbox files The intention was to drop support for indexing new single-message mbox files (and whether that was a good idea in the first place is arguable). However this inadvertently broke support for reading headers from previously indexed single-message mbox files, which is far worse. Distinguishing between the two cases would require more code than simply bringing back support for single-message mbox files.

build: add canonicalize_file_name to symbols exported from libnotmuch.so

2014-04-18T20:58:34Z

This is needed for our compat version of canonicalize_file_name to be used.

lib: replace the header parser with gmime

2014-04-05T15:53:04Z

The notmuch library includes a full blown message header parser. Yet the same message headers are parsed by gmime during indexing. Switch to gmime parsing completely. These are the main changes: * Gmime stops header parsing at the first invalid header, and presumes the message body starts from there. The current parser is quite liberal in accepting broken headers. The change means we will be much pickier about accepting invalid messages. * The current parser converts tabs used in header folding to spaces. Gmime preserve the tabs. Due to a broken python library used in mailman, there are plenty of mailing lists that produce headers with tabs in header folding, and we'll see plenty of tabs. (This change has been mitigated in preparatory patches.) * For pure header parsing, the current parser is likely faster than gmime, which parses the whole message rather than just the headers. Since we parse the message and its headers using gmime for indexing anyway, this avoids and extra header parsing round when adding new messages. In case of duplicate messages, we'll end up parsing the full message although just headers would be sufficient. All in all this should still speed up 'notmuch new'. * Calls to notmuch_message_get_header() may be slightly slower than previously for headers that are not indexed in the database, due to parsing of the whole message. Within the notmuch code base, notmuch reply is the only such user.

lib: drop support for single-message mbox files

2014-04-05T15:52:42Z

We've supported mbox files containing a single message for historical reasons, but the support has been deprecated, with a warning message while indexing, since Notmuch 0.15. Finally drop the support, and consider all mbox files non-email.

lib: make folder: prefix literal

2014-03-11T22:51:22Z

In xapian terms, convert folder: prefix from probabilistic to boolean prefix, matching the paths, relative from the maildir root, of the message files, ignoring the maildir new and cur leaf directories. folder:foo matches all message files in foo, foo/new, and foo/cur. folder:foo/new does *not* match message files in foo/new. folder:"" matches all message files in the top level maildir and its new and cur subdirectories. This change constitutes a database change: bump the database version and add database upgrade support for folder: terms. The upgrade also adds path: terms. Finally, fix the folder search test for literal folder: search, as some of the folder: matching capabilities are lost in the probabilistic to boolean prefix change.

lib: add support for path: prefix searches

2014-03-11T22:51:22Z

The path: prefix is a literal boolean prefix matching the paths, relative from the maildir root, of the message files. path:foo matches all message files in foo (but not in foo/new or foo/cur). path:foo/new matches all message files in foo/new. path:"" matches all message files in the top level maildir. path:foo/** matches all message files in foo and recursively in all subdirectories of foo. path:** matches all message files recursively, i.e. all messages.

lib: refactor folder term update after filename removal

2014-03-11T22:51:22Z

Abstract some blocks of code for reuse. No functional changes.

doc: notmuch_result_move_to_next -> notmuch_tags_move_to_next

2014-02-13T12:43:34Z

Fix typo in docs.