test: make it possible to have multiple corpora

We largely use the corpus under test/corpus for
testing. Unfortunately, many of our tests have grown to depend on
having exactly this set of messages, making it hard to add new message
files for testing specific cases.

We do use a lot of add_message from within the tests, but it's not
possible to use that for adding broken messages, and adding several
messages at once can get unwieldy.

Move the basic corpus under tests/corpora/default, and make it
possible to add new, independent corpora along its side. This means
tons of renames with a few tweaks to add_email_corpus function in
test-lib.sh to let tests specify which corpus to use.

3 files changed, 180 insertions, 0 deletions

diff --git a/test/corpora/default/foo/new/03:2, b/test/corpora/default/foo/new/03:2,
new file mode 100644
index 00000000..c154ac5e
--- /dev/null
+++ b/test/corpora/default/foo/new/03:2,
@@ -0,0 +1,93 @@
+Date: Tue, 17 Nov 2009 14:00:54 -0500
+From: Lars Kellogg-Stedman <lars@seas.harvard.edu>
+To: notmuch@notmuchmail.org
+Message-ID: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+MIME-Version: 1.0
+User-Agent: Mutt/1.5.19 (2009-01-05)
+Subject: [notmuch] Working with Maildir storage?
+X-BeenThere: notmuch@notmuchmail.org
+X-Mailman-Version: 2.1.12
+Precedence: list
+List-Id: "Use and development of the notmuch mail system."
+	<notmuch.notmuchmail.org>
+List-Unsubscribe: <http://notmuchmail.org/mailman/options/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=unsubscribe>
+List-Archive: <http://notmuchmail.org/pipermail/notmuch>
+List-Post: <mailto:notmuch@notmuchmail.org>
+List-Help: <mailto:notmuch-request@notmuchmail.org?subject=help>
+List-Subscribe: <http://notmuchmail.org/mailman/listinfo/notmuch>,
+	<mailto:notmuch-request@notmuchmail.org?subject=subscribe>
+Content-Type: multipart/mixed; boundary="===============1685355122=="
+Sender: notmuch-bounces@notmuchmail.org
+Errors-To: notmuch-bounces@notmuchmail.org
+
+
+--===============1685355122==
+Content-Type: multipart/signed; micalg=pgp-sha256;
+	protocol="application/pgp-signature"; boundary="5Dr6Wqe9hdyl7LAI"
+Content-Disposition: inline
+
+
+--5Dr6Wqe9hdyl7LAI
+Content-Type: text/plain; charset=us-ascii
+Content-Disposition: inline
+Content-Transfer-Encoding: quoted-printable
+
+I saw the LWN article and decided to take a look at notmuch.  I'm
+currently using mutt and mairix to index and read a collection of
+Maildir mail folders (around 40,000 messages total).
+
+notmuch indexed the messages without complaint, but my attempt at
+searching bombed out. Running, for example:
+
+  notmuch search storage
+
+Resulted in 4604 lines of errors along the lines of:
+
+  Error opening
+  /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=
+=3D3026:2,S:
+  Too many open files
+
+I'm curious if this is expected behavior (i.e., notmuch does not work
+with Maildir) or if something else is going on.
+
+Cheers,
+
+--=20
+Lars Kellogg-Stedman <lars@seas.harvard.edu>
+Senior Technologist, Computing and Information Technology
+Harvard University School of Engineering and Applied Sciences
+
+
+--5Dr6Wqe9hdyl7LAI
+Content-Type: application/pgp-signature
+Content-Disposition: inline
+
+-----BEGIN PGP SIGNATURE-----
+Version: GnuPG v1.4.9 (GNU/Linux)
+
+iQEcBAEBCAAGBQJLAvLmAAoJENdGlQYxQazYRtcH/0usClQ1Z+EoTsA+URwIK6hD
+FsZUxFxRjMuOQRn2idZ/zhhg5jJj11ZaHjqxSkDvi2ywkTKUf1vX9LLzVy5hSR9M
+E6XQUd5QWAQXo1VsTeKkukIL0YqsPjdgrT8+Yt+OS2NvhEncql23oxnL2/pHkIFq
+r0NdTmVV5Jcar7w9J6X1Mi9m229a/9jV5FImsWISkIhIWznXU5SiU6zIw8xhP4E0
+xhvVSNJnFryjVHtva870aSQduhHfeLPzpYhqbkMPvlq+bcz6Q/Q2SwxJcGLNMPHa
+os9s9FGhCvFKUhVzezHWPgXNCcNT8qK89rcUldb5Oq4jaJb8RCZCYABplfoyaFs=
+=vO4s
+-----END PGP SIGNATURE-----
+
+--5Dr6Wqe9hdyl7LAI--
+
+--===============1685355122==
+Content-Type: text/plain; charset="us-ascii"
+MIME-Version: 1.0
+Content-Transfer-Encoding: 7bit
+Content-Disposition: inline
+
+_______________________________________________
+notmuch mailing list
+notmuch@notmuchmail.org
+http://notmuchmail.org/mailman/listinfo/notmuch
+
+--===============1685355122==--
+
diff --git a/test/corpora/default/foo/new/09:2, b/test/corpora/default/foo/new/09:2,
new file mode 100644
index 00000000..26b51b1f
--- /dev/null
+++ b/test/corpora/default/foo/new/09:2,
@@ -0,0 +1,33 @@
+From: "Mikhail Gusarov" <dottedmag@dottedmag.net>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:50:48 +0600
+Subject: [notmuch] Working with Maildir storage?
+In-Reply-To: <20091117203301.GV3165@dottiness.seas.harvard.edu> (Lars
+	Kellogg-Stedman's message of "Tue, 17 Nov 2009 15:33:01 -0500")
+References: <20091117190054.GU3165@dottiness.seas.harvard.edu>
+	<87iqd9rn3l.fsf@vertex.dottedmag>
+	<20091117203301.GV3165@dottiness.seas.harvard.edu>
+Message-ID: <87fx8can9z.fsf@vertex.dottedmag>
+
+
+Twas brillig at 15:33:01 17.11.2009 UTC-05 when lars at seas.harvard.edu did gyre and gimble:
+
+ LK> Is the list archived anywhere?  The obvious archives
+ LK> (http://notmuchmail.org/pipermail/notmuch/) aren't available, and I
+ LK> think I subscribed too late to get the patch (I only just saw the
+ LK> discussion about it).
+
+ LK> It doesn't look like the patch is in git yet.
+
+Just has been pushed
+
+-- 
+  http://fossarchy.blogspot.com/
+-------------- next part --------------
+A non-text attachment was scrubbed...
+Name: not available
+Type: application/pgp-signature
+Size: 834 bytes
+Desc: not available
+URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20091118/0e33d964/attachment.pgp>
+
diff --git a/test/corpora/default/foo/new/10:2, b/test/corpora/default/foo/new/10:2,
new file mode 100644
index 00000000..4211d734
--- /dev/null
+++ b/test/corpora/default/foo/new/10:2,
@@ -0,0 +1,54 @@
+From: "Mikhail Gusarov" <dottedmag@dottedmag.net>
+To: notmuch@notmuchmail.org
+Date: Wed, 18 Nov 2009 02:51:18 +0600
+Subject: [notmuch] [PATCH] Handle rename of message file
+Message-ID: <1258491078-29658-1-git-send-email-dottedmag@dottedmag.net>
+
+If message file has been renamed, just update filename in the DB.
+
+Signed-off-by: Mikhail Gusarov <dottedmag at dottedmag.net>
+---
+ lib/database.cc |   21 ++++++++++++---------
+ 1 files changed, 12 insertions(+), 9 deletions(-)
+
+diff --git a/lib/database.cc b/lib/database.cc
+index 3c8d626..c4eb8b6 100644
+--- a/lib/database.cc
++++ b/lib/database.cc
+@@ -925,20 +925,23 @@ notmuch_database_add_message (notmuch_database_t *notmuch,
+ 	if (private_status == NOTMUCH_PRIVATE_STATUS_NO_DOCUMENT_FOUND) {
+ 	    _notmuch_message_set_filename (message, filename);
+ 	    _notmuch_message_add_term (message, "type", "mail");
++
++	    ret = _notmuch_database_link_message (notmuch, message, message_file);
++	    if (ret)
++		goto DONE;
++
++	    date = notmuch_message_file_get_header (message_file, "date");
++	    _notmuch_message_set_date (message, date);
++
++	    _notmuch_message_index_file (message, filename);
++	} else if (strcmp(notmuch_message_get_filename(message), filename)) {
++	    /* Message file has been moved/renamed */
++	    _notmuch_message_set_filename (message, filename);
+ 	} else {
+ 	    ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
+ 	    goto DONE;
+ 	}
+ 
+-	ret = _notmuch_database_link_message (notmuch, message, message_file);
+-	if (ret)
+-	    goto DONE;
+-
+-	date = notmuch_message_file_get_header (message_file, "date");
+-	_notmuch_message_set_date (message, date);
+-
+-	_notmuch_message_index_file (message, filename);
+-
+ 	_notmuch_message_sync (message);
+     } catch (const Xapian::Error &error) {
+ 	fprintf (stderr, "A Xapian exception occurred: %s.\n",
+-- 
+1.6.3.3
+
+