notmuch/notmuch-dump.c, branch 0.15.1

dump/restore: Use Xapian queries for batch-tag format

2013-01-07T02:40:32Z

This switches the new batch-tag format away from using a home-grown hex-encoding scheme for message IDs in the dump to simply using Xapian queries with Xapian quoting syntax. This has a variety of advantages beyond presenting a cleaner and more consistent interface. Foremost is that it will dramatically simplify the quoting for batch tagging, which shares the same input format. While the hex-encoding is no better or worse for the simple ID queries used by dump/restore, it becomes onerous for general-purpose queries used in batch tagging. It also better handles strange cases like "id:foo and bar", since this is no longer syntactically valid.

dump: Disallow \n in message IDs

2013-01-07T02:40:01Z

When we switch to using regular Xapian queries in the dump format, \n will cause problems, so we disallow it. Specially, while Xapian can quote and parse queries containing \n without difficultly, quoted queries containing \n still span multiple lines, which breaks the line-orientedness of the dump format. Strictly speaking, we could still round-trip these, but it would significantly complicate restore as well as scripts that deal with tag dumps. This complexity would come at absolutely no benefit: because of the RFC 2822 unfolding rules, no amount of standards negligence can produce a message with a message ID containing a line break (not even Outlook can do it!). Hence, we simply disallow it.

notmuch-dump: add --format=(batch-tag|sup)

2012-12-08T14:40:54Z

sup is the old format, and remains the default, at least until restore is converted to parse this format. Each line of the batch-tag format is modelled on the syntax of notmuch tag: - "notmuch tag" is omitted from the front of the line - The dump format only uses query strings of a single message-id. - Each space seperated tag/message-id is 'hex-encoded' to remove trouble-making characters. - It is permitted (and will be useful) for there to be no tags before the query. In particular this format won't have the same problem with e.g. spaces in message-ids or tags; they will be round-trip-able.

notmuch-dump: tidy formatting

2012-11-16T12:46:31Z

More uncrustify at work.

notmuch-dump: remove deprecated positional argument for output file

2012-08-06T11:52:33Z

The syntax --output=filename is a smaller change than deleting the output argument completely, and conceivably useful e.g. when running notmuch under a debugger.

lib/cli: Make notmuch_database_open return a status code

2012-05-05T13:11:57Z

It has been a long-standing issue that notmuch_database_open doesn't return any indication of why it failed. This patch changes its prototype to return a notmuch_status_t and set an out-argument to the database itself, like other functions that return both a status and an object. In the interest of atomicity, this also updates every use in the CLI so that notmuch still compiles. Since this patch does not update the bindings, the Python bindings test fails.

Use notmuch_database_destroy instead of notmuch_database_close

2012-04-28T12:27:33Z

Adapt the notmuch binaries source to the notmuch_database_close split. Signed-off-by: Justus Winter <4winter@informatik.uni-hamburg.de>

notmuch-dump: convert to command-line-arguments

2011-12-09T00:24:24Z

The output file is handled via positional arguments. There are currently no "normal" options.

notmuch-dump.c: whitespace cleanup

2011-12-04T15:28:42Z

dump: Don't sort the output by message id.

2011-11-28T15:57:45Z

Asking xapian to sort the messages for us causes suboptimal IO patterns. This would be useful, if we only wanted the first few results, but since we want everything anyway, this is pessimization. On 2011-10-29, a measurement on a 372981 messages instance showed that wall time can be reduced from 28 minutes (sorted by Message-ID) to 15 minutes (unsorted). Timings on 189605 messages: $ time notmuch.old dump 19.48user 5.83system 12:10.42elapsed 3%CPU (0avgtext+0avgdata 110656maxresident)k 3629584inputs+22720outputs (33major+7073minor)pagefaults 0swaps $ echo 3 > /proc/sys/vm/drop_caches $ time notmuch.new 14.89user 1.20system 3:23.58elapsed 7%CPU (0avgtext+0avgdata 46032maxresident)k 1256264inputs+22464outputs (43major+1990minor)pagefaults 0swaps