test: make it possible to have multiple corpora

We largely use the corpus under test/corpus for
testing. Unfortunately, many of our tests have grown to depend on
having exactly this set of messages, making it hard to add new message
files for testing specific cases.

We do use a lot of add_message from within the tests, but it's not
possible to use that for adding broken messages, and adding several
messages at once can get unwieldy.

Move the basic corpus under tests/corpora/default, and make it
possible to add new, independent corpora along its side. This means
tons of renames with a few tweaks to add_email_corpus function in
test-lib.sh to let tests specify which corpus to use.

25 files changed, 989 insertions, 0 deletions

diff --git a/test/corpora/default/cur/29:2, b/test/corpora/default/cur/29:2,
new file mode 100644
index 00000000..c76eff33
--- /dev/null
+++ b/test/corpora/default/cur/29:2,
@@ -0,0 +1,21 @@
+From: "Keith Packard" <keithp@keithp.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 18:04:31 -0800
+Subject: [notmuch] archive
+In-Reply-To: <20091117232137.GA7669@griffis1.net>
+References: <20091117232137.GA7669@griffis1.net>
+Message-ID: <yunzl6kd1w0.fsf@aiko.keithp.com>
+
+On Tue, 17 Nov 2009 18:21:38 -0500, Aron Griffis <agriffis at n01se.net> wrote:
+
+> Just subscribed, I'd like to catch up on the previous postings,
+> but the archive link seems to be bogus?
+
+Yeah, the archive appears broken and will need to wait until Carl
+arrives in Barcelona to get fixed.
+
+--
+keith.packard at intel.com
+
+
+
diff --git a/test/corpora/default/cur/30:2, b/test/corpora/default/cur/30:2,
new file mode 100644
index 00000000..a5b94a0a
--- /dev/null
+++ b/test/corpora/default/cur/30:2,
@@ -0,0 +1,75 @@
+From: "Stewart Smith" <stewart@flamingspork.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 13:22:20 +1100
+Subject: [notmuch] [PATCH] count_files: sort directory in inode order before
+	statting
+Message-ID: <1258510940-7018-1-git-send-email-stewart@flamingspork.com>
+
+---
+ notmuch-new.c |   30 ++++++++++--------------------
+ 1 files changed, 10 insertions(+), 20 deletions(-)
+
+diff --git a/notmuch-new.c b/notmuch-new.c
+index 11fad8c..c5f841a 100644
+--- a/notmuch-new.c
++++ b/notmuch-new.c
+@@ -308,36 +308,26 @@ add_files (notmuch_database_t *notmuch,
+ static void
+ count_files (const char *path, int *count)
+ {
+-    DIR *dir;
+-    struct dirent *e, *entry = NULL;
+-    int entry_length;
+-    int err;
++    struct dirent *entry = NULL;
+     char *next;
+     struct stat st;
++    struct dirent **namelist = NULL;
+ 
+-    dir = opendir (path);
++    int n_entries= scandir(path, &namelist, 0, ino_cmp);
+ 
+-    if (dir == NULL) {
++    if (n_entries == -1) {
+ 	fprintf (stderr, "Warning: failed to open directory %s: %s\n",
+ 		 path, strerror (errno));
+ 	goto DONE;
+     }
+ 
+-    entry_length = offsetof (struct dirent, d_name) +
+-	pathconf (path, _PC_NAME_MAX) + 1;
+-    entry = malloc (entry_length);
++    int i=0;
+ 
+     while (!interrupted) {
+-	err = readdir_r (dir, entry, &e);
+-	if (err) {
+-	    fprintf (stderr, "Error reading directory: %s\n",
+-		     strerror (errno));
+-	    free (entry);
+-	    goto DONE;
+-	}
++        if (i == n_entries)
++            break;
+ 
+-	if (e == NULL)
+-	    break;
++        entry= namelist[i++];
+ 
+ 	/* Ignore special directories to avoid infinite recursion.
+ 	 * Also ignore the .notmuch directory.
+@@ -376,8 +366,8 @@ count_files (const char *path, int *count)
+   DONE:
+     if (entry)
+ 	free (entry);
+-
+-    closedir (dir);
++    if (namelist)
++        free (namelist);
+ }
+ 
+ int
+-- 
+1.6.3.3
+
+
diff --git a/test/corpora/default/cur/31:2, b/test/corpora/default/cur/31:2,
new file mode 100644
index 00000000..88f17ca9
--- /dev/null
+++ b/test/corpora/default/cur/31:2,
@@ -0,0 +1,31 @@
+From: "Jjgod Jiang" <gzjjgod@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 11:50:17 +0800
+Subject: [notmuch] Mac OS X/Darwin compatibility issues
+Message-ID: <ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com>
+
+Hi,
+
+When I tried to compile notmuch under Mac OS X 10.6, several issues
+arisen:
+
+1. g++ reports 'warning: command line option "-Wmissing-declarations"
+is valid for C/ObjC but not for C++'
+
+2.
+notmuch-reply.c: In function ?address_is_users?:
+notmuch-reply.c:87: warning: passing argument 2 of
+?notmuch_config_get_user_other_email? from incompatible pointer type
+
+That's due to the size incompatibility of 'unsigned int' and 'size_t'
+(size_t is uint64_t in Mac OS X).
+
+3. Several errors about missing GNU extensions like getline() and strndup():
+
+warning: implicit declaration of function ?getline?
+error: ?strndup? was not declared in this scope
+
+We can implement these with fgets() and strncpy() though.
+
+- Jiang
+
diff --git a/test/corpora/default/cur/32:2, b/test/corpora/default/cur/32:2,
new file mode 100644
index 00000000..c1633cdd
--- /dev/null
+++ b/test/corpora/default/cur/32:2,
@@ -0,0 +1,165 @@
+From: "Jan Janak" <jan@ryngle.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 05:57:03 +0100
+Subject: [notmuch] [PATCH] notmuch new: Support for conversion of spool
+	subdirectories into tags
+Message-ID: <1258520223-15328-1-git-send-email-jan@ryngle.com>
+
+This patch makes it possible to convert subdirectory names inside the
+spool directory into message tags. Messages stored in subdirectory
+"foo" will be marked with tag "foo". Message duplicates found in several
+subdirectories will be given one tag per subdirectory.
+
+This feature can be used to synchronize notmuch's tags with with gmail
+tags, for example. Gmail IMAP servers convert tags to IMAP
+subdirectories and those subdirectories can be converted back to tags
+in notmuch.
+
+The patch modifies notmuch_database_add_message function to return a
+pointer to the message even if a message duplicate was found in the
+database. This is needed if we want to add a tag for each subdirectory
+in which a message duplicate was found.
+
+In addition to that, it makes the pointer to notmuch_config_t global
+(previously it was a local variable in notmuch_new_command). The
+configuration data structure is used by the function which converts
+subdirectory names to tags.
+
+Finally, there is a new function called subdir_to_tag. The function
+extracts the name of the subdirectory inside the spool from the full
+path of the message (also removing Maildir's cur,dir,and tmp
+subdirectories) and adds it as a new tag to the message.
+
+Signed-off-by: Jan Janak <jan at ryngle.com>
+---
+ lib/database.cc |    3 +-
+ notmuch-new.c   |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++++-
+ 2 files changed, 74 insertions(+), 3 deletions(-)
+
+diff --git a/lib/database.cc b/lib/database.cc
+index 3c8d626..f7799d2 100644
+--- a/lib/database.cc
++++ b/lib/database.cc
+@@ -949,7 +949,8 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
+ 
+   DONE:
+     if (message) {
+-	if (ret == NOTMUCH_STATUS_SUCCESS && message_ret)
++		if ((ret == NOTMUCH_STATUS_SUCCESS ||
++			 ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) && message_ret)
+ 	    *message_ret = message;
+ 	else
+ 	    notmuch_message_destroy (message);
+diff --git a/notmuch-new.c b/notmuch-new.c
+index 83a05ba..d94ce16 100644
+--- a/notmuch-new.c
++++ b/notmuch-new.c
+@@ -19,6 +19,9 @@
+  */
+ 
+ #include "notmuch-client.h"
++#include <libgen.h>
++
++static notmuch_config_t *config = 0;
+ 
+ static volatile sig_atomic_t do_add_files_print_progress = 0;
+ 
+@@ -45,6 +48,69 @@ tag_inbox_and_unread (notmuch_message_t *message)
+     notmuch_message_add_tag (message, "unread");
+ }
+ 
++/*
++ * Extracts the sub-directory from the filename and adds it as a new tag to
++ * the message. The filename must begin with the database directory configured
++ * in the configuration file. This prefix is then removed. If the remaining
++ * sub-directory ends with one of the Maildir special directories (/tmp, /new,
++ * /cur) then they are removed as well. If there is anything left then the
++ * function adds it as a new tag to the message.
++ *
++ * The function does nothing if it cannot extract a sub-directory from
++ * filename.
++ */
++static void
++subdir_to_tag (char* filename, notmuch_message_t *message)
++{
++	const char* db_path;
++	char *msg_dir, *tmp;
++	int db_path_len, msg_dir_len;
++
++	if (config == NULL) return;
++    db_path = notmuch_config_get_database_path (config);
++	if (db_path == NULL) return;
++	db_path_len = strlen(db_path);
++
++	/* Make a copy of the string as dirname may need to modify it. */
++	tmp = talloc_strdup(message, filename);
++	msg_dir = dirname(tmp);
++	msg_dir_len = strlen(msg_dir);
++
++	/* If msg_dir starts with db_path, remove it, including the / which delimits
++	 * it from the rest of the directory name. */
++	if (db_path_len < msg_dir_len &&
++		!strncmp(db_path, msg_dir, db_path_len)) {
++		msg_dir += db_path_len + 1;
++		msg_dir_len -= db_path_len + 1;
++	} else {
++		/* If we get here, either the message filename is not inside the
++		 * database directory configured in the configuration file, or it is a
++		 * file in the root directory of the database. Either way we just skip
++		 * it because we do not know how to convert it to a meaningful
++		 * subdirectory string that we could add as tag. */
++		goto out;
++	}
++
++	/* Special conditioning for Maildirs. If the remainder of the directory
++	 * name ends with /new, /cur, or /tmp then remove it. */
++	if ((msg_dir_len >= 4) &&
++		(!strncmp(msg_dir + msg_dir_len - 4, "/new", 4) ||
++		 !strncmp(msg_dir + msg_dir_len - 4, "/cur", 4) ||
++		 !strncmp(msg_dir + msg_dir_len - 4, "/tmp", 4))) {
++		msg_dir[msg_dir_len - 4] = '\0';
++	}
++
++	/* If, after all the modifications, we still have a subdirectory, add it
++	 * as tag. */
++	if (strlen(msg_dir)) {
++		notmuch_message_add_tag (message, msg_dir);
++	}
++
++out:
++	talloc_free(tmp);
++}
++
++
+ static void
+ add_files_print_progress (add_files_state_t *state)
+ {
+@@ -186,10 +252,15 @@ add_files_recursive (notmuch_database_t *notmuch,
+ 		    case NOTMUCH_STATUS_SUCCESS:
+ 			state->added_messages++;
+ 			tag_inbox_and_unread (message);
++			subdir_to_tag(next, message);
+ 			break;
+ 		    /* Non-fatal issues (go on to next file) */
+ 		    case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID:
+-		        /* Stay silent on this one. */
++		        /* Stay silent on this one. The message already exists in the
++				 * database, that means we may have found a duplicate in
++				 * another directory. If that's the case then we add another
++				 * tag to the message with the sub-directory. */
++				subdir_to_tag(next, message);
+ 			break;
+ 		    case NOTMUCH_STATUS_FILE_NOT_EMAIL:
+ 			fprintf (stderr, "Note: Ignoring non-mail file: %s\n",
+@@ -386,7 +457,6 @@ int
+ notmuch_new_command (void *ctx,
+ 		     unused (int argc), unused (char *argv[]))
+ {
+-    notmuch_config_t *config;
+     notmuch_database_t *notmuch;
+     add_files_state_t add_files_state;
+     double elapsed;
+-- 
+1.6.3.3
+
+
diff --git a/test/corpora/default/cur/33:2, b/test/corpora/default/cur/33:2,
new file mode 100644
index 00000000..a9b32524
--- /dev/null
+++ b/test/corpora/default/cur/33:2,
@@ -0,0 +1,13 @@
+From: "Rolland Santimano" <rollandsantimano@yahoo.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 21:12:23 -0800 (PST)
+Subject: [notmuch] Link to mailing list archives ?
+Message-ID: <736613.51770.qm@web113505.mail.gq1.yahoo.com>
+
+The link[1] provided on the list page[2] is broken:
+[1] http://notmuchmail.org/pipermail/notmuch/
+[2] http://notmuchmail.org/mailman/listinfo/notmuch
+
+
+      
+
diff --git a/test/corpora/default/cur/34:2, b/test/corpora/default/cur/34:2,
new file mode 100644
index 00000000..b94dd069
--- /dev/null
+++ b/test/corpora/default/cur/34:2,
@@ -0,0 +1,46 @@
+From: "Alexander Botero-Lowry" <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 21:45:36 -0800
+Subject: [notmuch] Mac OS X/Darwin compatibility issues
+In-Reply-To: <ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com>
+References: <ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com>
+Message-ID: <86einw2xof.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+
+On Wed, 18 Nov 2009 11:50:17 +0800, Jjgod Jiang <gzjjgod at gmail.com> wrote:
+> Hi,
+> 
+> When I tried to compile notmuch under Mac OS X 10.6, several issues
+> arisen:
+> 
+> 1. g++ reports 'warning: command line option "-Wmissing-declarations"
+> is valid for C/ObjC but not for C++'
+> 
+I got that too. I presume it's newly supported in GCC4.4?
+
+> 3. Several errors about missing GNU extensions like getline() and strndup():
+> 
+strndup from V8:
+
+char* strndup(char* str, size_t n) {
+  // Stupid implementation of strndup since macos isn't born with
+  // one.
+  size_t len = strlen(str);
+  if (len <= n)
+    return StrDup(str);
+  char* result = new char[n+1];
+  size_t i;
+  for (i = 0; i <= n; i++)
+    result[i] = str[i];
+  result[i] = '\0';
+  return result;
+}
+
+> warning: implicit declaration of function ?getline?
+> error: ?strndup? was not declared in this scope
+> 
+for getline do you mind trying #define _GNU_SOURCE 1
+before #include <stdio.h> in the offending files? The FreeBSD man pages
+mentions that as a way of enabling the GNU version of getline().
+
+Alex
+
diff --git a/test/corpora/default/cur/35:2, b/test/corpora/default/cur/35:2,
new file mode 100644
index 00000000..d7276705
--- /dev/null
+++ b/test/corpora/default/cur/35:2,
@@ -0,0 +1,24 @@
+From: "Jjgod Jiang" <gzjjgod@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 14:14:27 +0800
+Subject: [notmuch] Mac OS X/Darwin compatibility issues
+In-Reply-To: <86einw2xof.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+References: <ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com>
+	<86einw2xof.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+Message-ID: <ddd65cda0911172214t60d22b63hcfeb5a19ab54a39b@mail.gmail.com>
+
+Hi,
+
+On Wed, Nov 18, 2009 at 1:45 PM, Alexander Botero-Lowry
+<alex.boterolowry at gmail.com> wrote:
+> for getline do you mind trying #define _GNU_SOURCE 1
+> before #include <stdio.h> in the offending files? The FreeBSD man pages
+> mentions that as a way of enabling the GNU version of getline().
+
+It seems even _GNU_SOURCE is defined, getline is still not present.
+the C lib in Mac OS X simply doesn't have it. See also [1].
+
+- Jiang
+
+[1] http://stackoverflow.com/questions/1117108/compiling-c-code-using-gnu-c-getline-on-mac-osx
+
diff --git a/test/corpora/default/cur/36:2, b/test/corpora/default/cur/36:2,
new file mode 100644
index 00000000..4cd0d200
--- /dev/null
+++ b/test/corpora/default/cur/36:2,
@@ -0,0 +1,25 @@
+From: "Alexander Botero-Lowry" <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 22:19:29 -0800
+Subject: [notmuch] Mac OS X/Darwin compatibility issues
+In-Reply-To: <ddd65cda0911172214t60d22b63hcfeb5a19ab54a39b@mail.gmail.com>
+References: <ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com>
+	<86einw2xof.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+	<ddd65cda0911172214t60d22b63hcfeb5a19ab54a39b@mail.gmail.com>
+Message-ID: <86d43g2w3y.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+
+On Wed, 18 Nov 2009 14:14:27 +0800, Jjgod Jiang <gzjjgod at gmail.com> wrote:
+> Hi,
+> 
+> On Wed, Nov 18, 2009 at 1:45 PM, Alexander Botero-Lowry
+> <alex.boterolowry at gmail.com> wrote:
+> > for getline do you mind trying #define _GNU_SOURCE 1
+> > before #include <stdio.h> in the offending files? The FreeBSD man pages
+> > mentions that as a way of enabling the GNU version of getline().
+> 
+> It seems even _GNU_SOURCE is defined, getline is still not present.
+> the C lib in Mac OS X simply doesn't have it. See also [1].
+> 
+Alas. Since it's ostensibly based on the FreeBSD one, I figured there
+was a chance that would fix the problem. :/
+
diff --git a/test/corpora/default/cur/37:2, b/test/corpora/default/cur/37:2,
new file mode 100644
index 00000000..4e17e82b
--- /dev/null
+++ b/test/corpora/default/cur/37:2,
@@ -0,0 +1,22 @@
+From: "Alexander Botero-Lowry" <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 00:02:56 -0800
+Subject: [notmuch] request for pull
+Message-ID: <86aayk2rbj.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+
+The following changes since commit e8c9c3e6a534fc6c2919c2c1de63cea7250eb488:
+  Ingmar Vanhassel (1):
+        Makefile: Manual pages shouldn't be executable
+
+are available in the git repository at:
+
+  git://alexbl.net/notmuch.git master
+
+Alexander Botero-Lowry (2):
+      Error out if no query is supplied to search instead of going into an infinite loop
+      set a local truncate-line variable in notmuch-search-mode, so that subjects don't wrap and make the output look weird
+
+ notmuch-search.c |    5 +++++
+ notmuch.el       |    1 +
+ 2 files changed, 6 insertions(+), 0 deletions(-)
+
diff --git a/test/corpora/default/cur/38:2, b/test/corpora/default/cur/38:2,
new file mode 100644
index 00000000..f5537ffe
--- /dev/null
+++ b/test/corpora/default/cur/38:2,
@@ -0,0 +1,40 @@
+From: "Keith Packard" <keithp@keithp.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 00:29:59 -0800
+Subject: [notmuch] [PATCH] Create a default notmuch-show-hook that
+	highlights URLs and uses word-wrap
+Message-ID: <1258532999-9316-1-git-send-email-keithp@keithp.com>
+
+I created the notmuch-show-hook precisely so I could add these two
+options, but I suspect most people will want them, so I just made them
+the default. If you don't want them, you can use remove-hook to get
+rid of this.
+
+Signed-off-by: Keith Packard <keithp at keithp.com>
+---
+ notmuch.el |    8 ++++++++
+ 1 files changed, 8 insertions(+), 0 deletions(-)
+
+diff --git a/notmuch.el b/notmuch.el
+index 1bb1294..c95cb43 100644
+--- a/notmuch.el
++++ b/notmuch.el
+@@ -698,6 +698,14 @@ view, (remove the \"inbox\" tag from each), with either
+   :options '(goto-address)
+   :group 'notmuch)
+ 
++; Make show mode a bit prettier, highlighting URLs and using word wrap
++
++(defun notmuch-show-pretty-hook ()
++  (goto-address-mode 1)
++  (visual-line-mode))
++
++(add-hook 'notmuch-show-hook 'notmuch-show-pretty-hook)
++
+ (defun notmuch-show (thread-id &optional parent-buffer)
+   "Run \"notmuch show\" with the given thread ID and display results.
+ 
+-- 
+1.6.5.2
+
+
diff --git a/test/corpora/default/cur/39:2, b/test/corpora/default/cur/39:2,
new file mode 100644
index 00000000..637b3c74
--- /dev/null
+++ b/test/corpora/default/cur/39:2,
@@ -0,0 +1,32 @@
+From: "Alexander Botero-Lowry" <alex.boterolowry@gmail.com>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 00:52:34 -0800
+Subject: [notmuch] [PATCH] Create a default notmuch-show-hook that
+ highlights URLs and uses word-wrap
+In-Reply-To: <1258532999-9316-1-git-send-email-keithp@keithp.com>
+References: <1258532999-9316-1-git-send-email-keithp@keithp.com>
+Message-ID: <867hto2p0t.fsf@fortitudo.i-did-not-set--mail-host-address--so-tickle-me>
+
+On Wed, 18 Nov 2009 00:29:59 -0800, Keith Packard <keithp at keithp.com> wrote:
+> I created the notmuch-show-hook precisely so I could add these two
+> options, but I suspect most people will want them, so I just made them
+> the default. If you don't want them, you can use remove-hook to get
+> rid of this.
+> 
+Yes, hooks should be added for search as well. :)
+
+> +; Make show mode a bit prettier, highlighting URLs and using word wrap
+> +
+> +(defun notmuch-show-pretty-hook ()
+> +  (goto-address-mode 1)
+> +  (visual-line-mode))
+> +
+visual-line-mode turns out to make subject look pretty ugly if there is a
+continuation. It doesn't do much good for the citation headers
+either. We probably need to do our own intelligent wrapping rather then
+use visual-line-mode to make this actually look right.
+
+goto-address-mode is important though. :)
+
+alex
+
diff --git a/test/corpora/default/cur/40:2, b/test/corpora/default/cur/40:2,
new file mode 100644
index 00000000..91a15a8d
--- /dev/null
+++ b/test/corpora/default/cur/40:2,
@@ -0,0 +1,31 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 01:42:02 -0800
+Subject: [notmuch] [PATCH 1/2] Close message file after parsing message
+ headers
+In-Reply-To: <yunbpj0etua.fsf@aiko.keithp.com>
+References: <1258471718-6781-1-git-send-email-dottedmag@dottedmag.net>
+	<87lji5cbwo.fsf@yoom.home.cworth.org> <yunbpj0etua.fsf@aiko.keithp.com>
+Message-ID: <87pr7gqidx.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 13:15:25 -0800, Keith Packard <keithp at keithp.com> wrote:
+> Threading the message also involves displaying the from and to contents,
+> which requires opening the message file. The alternative to the fix I
+> provided is to just parse all of the message headers when first opening
+> the message; it could then be immediately closed and the hash referred
+> to for all header data. Given the choice, just having the caller say
+> when it has finished with a message is probably a reasonable option...
+
+Hi Keith,
+
+Once I finally got back on the ground again, I pushed out a revised
+version of your patch, (didn't need the reply-to stuff anymore since I
+had fixed that differently in the meantime).
+
+I'm pretty happy with the state of this portion of the code now.
+
+Thanks Keith and Mikhail for your input on and code to fix this bug.
+
+-Carl
+
+
diff --git a/test/corpora/default/cur/41:2, b/test/corpora/default/cur/41:2,
new file mode 100644
index 00000000..da22cc02
--- /dev/null
+++ b/test/corpora/default/cur/41:2,
@@ -0,0 +1,37 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:08:10 -0800
+Subject: [notmuch] Working with Maildir storage?
+In-Reply-To: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+References: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+Message-ID: <87ocn0qh6d.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 14:00:54 -0500, Lars Kellogg-Stedman <lars at seas.harvard.edu> wrote:
+> I saw the LWN article and decided to take a look at notmuch.  I'm
+> currently using mutt and mairix to index and read a collection of
+> Maildir mail folders (around 40,000 messages total).
+
+Welcome, Lars!
+
+I hadn't even seen that Keith's blog post had been picked up by lwn.net.
+That's very interesting. So, thanks for coming and trying out notmuch.
+
+>   Error opening
+>   /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S:
+>   Too many open files
+
+Sadly, the lwn article coincided with me having just introduced this
+bug, and then getting on a Trans-Atlantic flight. So I fixed the bug
+fairly quickly, but there was quite a bit of latency before I could push
+the fix out. It should be fixed now.
+
+> I'm curious if this is expected behavior (i.e., notmuch does not work
+> with Maildir) or if something else is going on.
+
+Notmuch works just fine with maildir---it's one of the things that it
+likes the best.
+
+Happy hacking,
+
+-Carl
+
diff --git a/test/corpora/default/cur/42:2, b/test/corpora/default/cur/42:2,
new file mode 100644
index 00000000..98fa75fa
--- /dev/null
+++ b/test/corpora/default/cur/42:2,
@@ -0,0 +1,30 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:19:26 -0800
+Subject: [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands
+ remove inbox (and unread) tags
+In-Reply-To: <1258493565-13508-1-git-send-email-keithp@keithp.com>
+References: <1258493565-13508-1-git-send-email-keithp@keithp.com>
+Message-ID: <87k4xoqgnl.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 13:32:45 -0800, Keith Packard <keithp at keithp.com> wrote:
+> When closing a thread view, mark the thread as archived by removing
+> the "inbox" tag, and for the 'x' variant, the "unread" tag as well,
+> then kill the buffer and update the search window view as well.
+> 
+> This makes 'x' much the same as 'a', but instead of taking you to the
+> next message, it takes you back to the search window instead.
+
+I don't like this---but that's because I use 'x' precisely *because* it
+preserves these tags.
+
+Otherwise, you might as well just remove inbox and unread as soon as the
+message is presented to the user. And that's a bug in a lot of other
+email programs that I'm unwilling to replicate.
+
+We may run into a need to define different ways that people like to work
+with their email here. (I know that so far I've just been coding up the
+way I want my mail to work.)
+
+-Carl
+
diff --git a/test/corpora/default/cur/43:2, b/test/corpora/default/cur/43:2,
new file mode 100644
index 00000000..2f6c8bc8
--- /dev/null
+++ b/test/corpora/default/cur/43:2,
@@ -0,0 +1,26 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:22:12 -0800
+Subject: [notmuch] archive
+In-Reply-To: <yunzl6kd1w0.fsf@aiko.keithp.com>
+References: <20091117232137.GA7669@griffis1.net>
+	<yunzl6kd1w0.fsf@aiko.keithp.com>
+Message-ID: <87iqd8qgiz.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 18:04:31 -0800, Keith Packard <keithp at keithp.com> wrote:
+> On Tue, 17 Nov 2009 18:21:38 -0500, Aron Griffis <agriffis at n01se.net> wrote:
+> 
+> > Just subscribed, I'd like to catch up on the previous postings,
+> > but the archive link seems to be bogus?
+> 
+> Yeah, the archive appears broken and will need to wait until Carl
+> arrives in Barcelona to get fixed.
+
+Fixed it in transit in Frankfurt---with only moments to spare on my
+battery and no outlets in sight.
+
+Thanks for the report, Aron. And welcome to notmuch!
+
+-Carl (who wants to reply to a lot more mail, but will have to wait
+ until later for that)
+
diff --git a/test/corpora/default/cur/44:2, b/test/corpora/default/cur/44:2,
new file mode 100644
index 00000000..c896c189
--- /dev/null
+++ b/test/corpora/default/cur/44:2,
@@ -0,0 +1,29 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:43:50 -0800
+Subject: [notmuch] [PATCH] Older versions of install do not support -C.
+In-Reply-To: <1258496327-12086-1-git-send-email-jan@ryngle.com>
+References: <1258496327-12086-1-git-send-email-jan@ryngle.com>
+Message-ID: <87hbssqfix.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 23:18:47 +0100, Jan Janak <jan at ryngle.com> wrote:
+> Do not use -C cmdline option of install, older versions, commonly found in
+> distributions like Debian, do not seem to support it. Running make install
+> on such systems (tested on Debian Lenny) fails.
+> 
+> Signed-off-by: Jan Janak <jan at ryngle.com>
+
+Thanks, Jan. This is pushed now.
+
+And did I say welcome to notmuch yet? (It's easy to lose track with all
+the newcomers---which I'm not complaining about---especially since so
+many are sharing code.)
+
+-Carl
+
+PS. I actually really like the behavior of -C (especially when
+installing a low-level library to avoid big waterfalls of needless
+recompiles). But since we're *not* actually installing a library (yet)
+I'm happy with this patch rather than writing code in configure to check
+if "install -C" works or not.
+
diff --git a/test/corpora/default/cur/45:2, b/test/corpora/default/cur/45:2,
new file mode 100644
index 00000000..806b0e89
--- /dev/null
+++ b/test/corpora/default/cur/45:2,
@@ -0,0 +1,41 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:49:52 -0800
+Subject: [notmuch] What a great idea!
+In-Reply-To: <f35dbb950911171435ieecd458o853c873e35f4be95@mail.gmail.com>
+References: <f35dbb950911171435ieecd458o853c873e35f4be95@mail.gmail.com>
+Message-ID: <87fx8cqf8v.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 23:35:30 +0100, Jan Janak <jan at ryngle.com> wrote:
+> First of all, notmuch is a wonderful idea, both the cmdline tool and
+> the emacs interface! Thanks a lot for writing it, I was really excited
+> when I read the announcement today.
+
+Ah, here's where I planned a nice welcome. So welcome (again), Jan! :-)
+
+I've been having a lot of fun with notmuch already, (though there have
+been some days of pain before it was functional enough and my
+email-reply latency went way up). But regardless---I got through that,
+and I'm able to work more efficiently with notmuch now than I could with
+sup before. So I'm happy.
+
+And I'm delighted when other people find this interesting as well.
+
+> Have you considered sending an announcement to the org-mode mailing list?
+> http://orgmode.org
+
+Thanks for the idea. I think I may have looked into org-mode years ago,
+(when I was investigating planner-mode and various emacs "personal wiki"
+systems for keeping random notes and what-not).
+
+> Various ways of searching/referencing emails from emacs were discussed
+> there several times and none of them were as elegant as notmuch (not
+> even close). Maybe notmuch would attract some of the developers
+> there..
+
+Yeah. I'll drop them a mail. Having a real emacs wizard on board would
+be nice. (I'm afraid the elisp I've written so far for this project is
+fairly grim.)
+
+-Carl
+
diff --git a/test/corpora/default/cur/46:2, b/test/corpora/default/cur/46:2,
new file mode 100644
index 00000000..bbd1b37f
--- /dev/null
+++ b/test/corpora/default/cur/46:2,
@@ -0,0 +1,57 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 03:02:43 -0800
+Subject: [notmuch] New to the list
+In-Reply-To: <1258498485-sup-142@elly>
+References: <1258498485-sup-142@elly>
+Message-ID: <87bpj0qeng.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 23:57:18 +0100, Israel Herraiz <isra at herraiz.org> wrote:
+> I have subscribed to the list. As suggested by the welcome message, I
+> am introducing myself. My name is Israel Herraiz, and I have done a
+> couple of contributions to Sup, the probably well-known here e-mail
+> client.
+
+Welcome, Israel!
+
+I'm glad people read that little bit of text in the welcome email and
+are introducing themselves. I like to think of our new notmuch community
+as a very personable place.
+
+> "Not much" sounds interesting, and I wonder whether it could be
+> integrated with the views of Sup (inbox, threads, etc). So I have
+> subscribed to the list to keep an eye on what's going on here.
+> 
+> I have just heard of "Not much". I have not even tried to download the
+> code yet.
+
+Yes, take a look. If you're already an emacs user, then you'll find the
+interface of notmuch very comfortable, (looks a lot like sup, but lives
+inside of emacs). Even outside of emacs, the command line interface of
+notmuch gives view *fairly* similar to those of sup:
+
+    notmuch search tag:inbox		# Very much like sup's inbox
+
+    notmuch show thread:some-thread-id	# A lot like sup's thread -view
+
+The command-line output right now isn't nearly as neat as sup's, (it
+doesn't elide comments--it doesn't do the indenting of threads, etc.),
+even though the command-line interface has all the information it needs
+to do that. The reason for that is to let the emacs code own most of the
+formatting, (so that it can be more flexible--such as making hidden
+things visible, changing column widths, etc.).
+
+But one thing I wonder is if there would be situations where it would
+make sense to get the cleaner output directly out of the command-line
+tool.
+
+For example, for someone who isn't an emacs user, the command-line
+interface might be their only introduction to what the "notmuch
+experience" is like. So maybe "notmuch show" should give nice clean
+output by default and then the emacs code could call "notmuch show
+--format=emacs-friendly" or something to get the current output.
+
+That's an idea anyway.
+
+-Carl
+
diff --git a/test/corpora/default/cur/47:2, b/test/corpora/default/cur/47:2,
new file mode 100644
index 00000000..9de5532c
--- /dev/null
+++ b/test/corpora/default/cur/47:2,
@@ -0,0 +1,84 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 03:15:31 -0800
+Subject: [notmuch] Introducing myself
+In-Reply-To: <20091118002059.067214ed@hikari>
+References: <20091118002059.067214ed@hikari>
+Message-ID: <87aaykqe24.fsf@yoom.home.cworth.org>
+
+On Wed, 18 Nov 2009 00:20:59 +0100, Adrian Perez de Castro <aperez at igalia.com> wrote:
+> I have just heard about Not Much today in some random Linux-related news
+> site (LWN?), my name is Adrian Perez and I work as systems administrator
+
+Welcome to notmuch, Adrian! We're glad to have you here.
+
+> by default on most distribution. I got to have some mailboxes indexed and
+> basic searching working a couple of months ago. Lately I have been very
+> busy and had no time for coding, and them... boom! Not Much appears -- and
+> it is almost exactly what I was trying to do, but faster. I have been
+> playing a bit with Not Much today, and I think it has potential.
+
+It's funny, because I had the exact same experience with sup a couple of
+months ago. I had been frustrated for years with email programs, and had
+been thinking about how I'd like things to work n the back of my mind
+for a long time, (but never *quite* getting to the point where I would
+commit to writing an email system myself).
+
+And then... boom! I found sup and was instantly hooked. It had so much
+of what I had imagined, (and much of what I hadn't yet imagined) that I
+was quite delighted.
+
+It was really quite by accident that I ended up inventing a different
+system. I had started out just trying to speedup index creation for sup.
+If I hadn't run into the problem that it was very difficult[*] to create a
+sup-compatible index from C code, I might have stopped there.
+
+So I'd written a bunch of functional code, only to find myself stuck at
+the very last step, (hooking it up to the existing sup interface). Then
+Keith suggested emacs and it all seemed pretty easy since I'd already
+done all the Xapian work. So it's funny, I was only willing to commit to
+this project because I wasn't consciously aware I was working on it.
+Otherwise it would have seemed to overwhelming to start. :-)
+
+Anyway, that's a lot of off-topic rambling off of your introduction. But
+I'm glad that notmuch can now give that same "boom!" to others, and I'm
+glad you see potential in it.
+
+> Also, I would like to share one idea I had in mind, that you might find
+> interesting: One thing I have found very annoying is having to re-tag my
+> mail when the indexes get b0rked (it happened a couple of times to me while
+> using Sup), so I was planning to mails as read/unread and adding the tags
+> not just to the index, but to the mail text itself, e.g. by adding a
+> "X-Tags" header field or by reusing the "Keywords" one. This way, the index
+> could be totally recreated by re-reading the mail directories, and this
+> would also allow to a tools like OfflineIMAP [1] to get the mails into a
+> local maildir, tagging and indexing the mails with the e-mail reader and
+> then syncing back the messages with the "X-Tags" header to the IMAP server.
+> This would allow to use the mail reader from a different computer and still
+> have everything tagged finely.
+
+It is an interesting idea. But there's also something really comforting
+about the email indexed never modifying the mail files. If you're
+reading the notmuch commit logs closely you'll see that I'm not actually
+careful enough to be trusted with your mail (but I try). So I like that
+I don't even have to trust myself---the worst that happens is that I
+have to recreate my index.
+
+And as Keith mentioned, we've got the "notmuch dump; notmuch restore"
+idea working exactly as it did in sup. (Though I am thinking of also
+adding thread IDs to that now---more on that later.)
+
+The big annoyance I had with sup index creation, (I ended up having to
+do it more than once too), was that it takes *forever*. Right now,
+notmuch is a little bit faster, but not a lot faster. And I've got some
+ideas to fix that. It would be really nice if index creation were pain
+free. (And maybe it is for some user with small amounts of mail---oh, to
+have only 40000 messages to have to index!).
+
+-Carl
+
+[*] The problem here is that sup puts serialized ruby data structures
+into the data field of its Xapian documents. So being compatible with
+sup means being able to recreate serialized data structures for a
+particular version of ruby.
+
diff --git a/test/corpora/default/cur/48:2, b/test/corpora/default/cur/48:2,
new file mode 100644
index 00000000..419e21db
--- /dev/null
+++ b/test/corpora/default/cur/48:2,
@@ -0,0 +1,17 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 03:22:32 -0800
+Subject: [notmuch] [PATCH] Typsos
+In-Reply-To: <1258500222-32066-1-git-send-email-ingmar@exherbo.org>
+References: <1258500222-32066-1-git-send-email-ingmar@exherbo.org>
+Message-ID: <878we4qdqf.fsf@yoom.home.cworth.org>
+
+On Wed, 18 Nov 2009 00:23:42 +0100, Ingmar Vanhassel <ingmar at exherbo.org> wrote:
+>  17 files changed, 30 insertions(+), 30 deletions(-)
+
+Yikes. That's a lot of typos.
+
+Thanks Ingmar, for cleaning up after my sloppy keyboarding. Pushed.
+
+-Carl
+
diff --git a/test/corpora/default/cur/49:2, b/test/corpora/default/cur/49:2,
new file mode 100644
index 00000000..b244f8ce
--- /dev/null
+++ b/test/corpora/default/cur/49:2,
@@ -0,0 +1,33 @@
+From: "Carl Worth" <cworth@cworth.org>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 03:31:23 -0800
+Subject: [notmuch] [PATCH] Error out if no query is supplied to search
+ instead of going into an infinite loop
+In-Reply-To: <cf0c4d610911171623q3e27a0adx802e47039b57604b@mail.gmail.com>
+References: <cf0c4d610911171623q3e27a0adx802e47039b57604b@mail.gmail.com>
+Message-ID: <877htoqdbo.fsf@yoom.home.cworth.org>
+
+On Tue, 17 Nov 2009 16:23:53 -0800, Alex Botero-Lowry <alex.boterolowry at gmail.com> wrote:
+> In this case error out when no query is supplied. There seems to be an
+> infinite-loop casued by i think notmuch_query_search_threads having
+> an exception:
+> A Xapian exception occurred: Syntax: <expression> AND <expression>
+> A Xapian exception occurred: Syntax: <expression> AND <expression>
+> A Xapian exception occurred: Syntax: <expression> AND <expression>
+> 
+> I'll look into that bug specifically a bit later.
+> 
+> It might be better to do a usage instead of just throwing an error here?
+
+Definitely.
+
+Priit Laes reported the same thing in IRC and I've just committed a
+patch to give a nice error message:
+
+$ ./notmuch search
+Error: notmuch search requires at least one search term.
+
+Thanks for the report!
+
+-Carl
+
diff --git a/test/corpora/default/cur/50:2, b/test/corpora/default/cur/50:2,
new file mode 100644
index 00000000..44e8be5d
--- /dev/null
+++ b/test/corpora/default/cur/50:2,
@@ -0,0 +1,39 @@
+From: "Chris Wilson" <chris@chris-wilson.co.uk>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 11:34:54 +0000
+Subject: [notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once
+Message-ID: <1258544095-16616-1-git-send-email-chris@chris-wilson.co.uk>
+
+Currently the same `pkg-config ...` is executed for every target, so
+just store the results in a variable.
+---
+ Makefile |    9 +++++----
+ 1 files changed, 5 insertions(+), 4 deletions(-)
+
+diff --git a/Makefile b/Makefile
+index 96aaa73..023b2ec 100644
+--- a/Makefile
++++ b/Makefile
+@@ -4,15 +4,16 @@ CFLAGS=-O2
+ 
+ # Additional flags that we will append to whatever the user set.
+ # These aren't intended for the user to manipulate.
+-extra_cflags = `pkg-config --cflags glib-2.0 gmime-2.4 talloc`
+-extra_cxxflags = `xapian-config --cxxflags`
++extra_cflags := $(shell pkg-config --cflags glib-2.0 gmime-2.4 talloc)
++extra_cxxflags := $(shell xapian-config --cxxflags)
+ 
+ # Now smash together user's values with our extra values
+ override CFLAGS += $(WARN_FLAGS) $(extra_cflags)
+ override CXXFLAGS += $(WARN_FLAGS) $(extra_cflags) $(extra_cxxflags)
+ 
+-override LDFLAGS += `pkg-config --libs glib-2.0 gmime-2.4 talloc` \
+-			`xapian-config --libs`
++override LDFLAGS += \
++	$(shell pkg-config --libs glib-2.0 gmime-2.4 talloc) \
++	$(shell xapian-config --libs)
+ 
+ # Include our local Makefile.local first so that its first target is default
+ include Makefile.local
+-- 
+1.6.5.2
diff --git a/test/corpora/default/cur/51:2, b/test/corpora/default/cur/51:2,
new file mode 100644
index 00000000..f522f69e
--- /dev/null
+++ b/test/corpora/default/cur/51:2,
@@ -0,0 +1,12 @@
+From: "Aron Griffis" <agriffis@n01se.net>
+To: notmuch@notmuchmail.org
+Date: Tue, 17 Nov 2009 18:21:38 -0500
+Subject: [notmuch] archive
+Message-ID: <20091117232137.GA7669@griffis1.net>
+
+Just subscribed, I'd like to catch up on the previous postings,
+but the archive link seems to be bogus?
+
+Thanks,
+Aron
+
diff --git a/test/corpora/default/cur/52:2, b/test/corpora/default/cur/52:2,
new file mode 100644
index 00000000..60283404
--- /dev/null
+++ b/test/corpora/default/cur/52:2,
@@ -0,0 +1,39 @@
+Message-ID: <4EFC743A.3060609@april.org>
+Date: Thu, 29 Dec 2010 15:07:54 +0100
+From: "=?ISO-8859-1?Q?Fran=E7ois_Boulogne?=" <boulogne.f@gmail.com>
+User-Agent: Mozilla/5.0 (X11; Linux i686;
+ rv:9.0) Gecko/20111224 Thunderbird/9.0.1
+MIME-Version: 1.0
+To: Allan McRae <allan@archlinux.org>, 
+ "Discussion about the Arch User Repository (AUR)" <aur-general@archlinux.org>
+References: <4EFC3931.6030007@april.org> <4EFC3D62.4030202@archlinux.org>
+In-Reply-To: <4EFC3D62.4030202@archlinux.org>
+Content-Type: text/plain; charset=ISO-8859-1
+Content-Transfer-Encoding: 8bit
+Subject: Re: [aur-general] Guidelines: cp, mkdir vs install
+
+Le 29/12/2011 11:13, Allan McRae a écrit :
+> On 29/12/11 19:56, François Boulogne wrote:
+>> Hi,
+>>
+>> Looking to improve the quality of my packages, I read again the guidelines.
+>> https://wiki.archlinux.org/index.php/Arch_Packaging_Standards
+>>
+>> However, it don't see anything about the install command like
+>> install -d $pkgdir/usr/{bin,share/man/man1,share/locale}
+>>
+>> Some contributors on AUR use cp or mkdir to install files/dir (when no
+>> makefile is provided) and others use install command.
+>>
+>> What's the opinion of TU on this point?
+>>
+> 
+> Use install with -m specifying the correct permissions
+> 
+
+Thank you Allan
+
+
+-- 
+François Boulogne.
+https://www.sciunto.org
diff --git a/test/corpora/default/cur/53:2, b/test/corpora/default/cur/53:2,
new file mode 100644
index 00000000..7a1e2e58
--- /dev/null
+++ b/test/corpora/default/cur/53:2,
@@ -0,0 +1,20 @@
+From: Olivier Berger <olivier.berger@it-sudparis.eu>
+To: olivier.berger@it-sudparis.eu
+Subject: Essai =?iso-8859-1?Q?accentu=E9?=
+User-Agent: Notmuch/0.10.1 (http://notmuchmail.org) Emacs/23.3.1 (i486-pc-linux-gnu)
+X-Draft-From: ("nnimap+localdovecot:INBOX" 44228)
+Date: Fri, 16 Dec 2010 16:49:59 +0100
+Message-ID: <877h1wv7mg.fsf@inf-8657.int-evry.fr>
+MIME-Version: 1.0
+Content-Type: text/plain; charset=iso-8859-1
+Content-Transfer-Encoding: quoted-printable
+
+Du texte accentu=E9 pour =E7a ...
+
+=E0 la bonne heure !
+--=20
+Olivier BERGER=20
+http://www-public.it-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8
+Ingenieur Recherche - Dept INF
+Institut TELECOM, SudParis (http://www.it-sudparis.eu/), Evry (France)
+