lib: add 'body:' field, stop indexing headers twice.

The new `body:` field (in Xapian terms) or prefix (in slightly
sloppier notmuch) terms allows matching terms that occur only in the
body.

Unprefixed query terms should continue to match anywhere (header or
body) in the message.

This follows a suggestion of Olly Betts to use the facility (since
Xapian 1.0.4) to add the same field with multiple prefixes. The double
indexing of previous versions is thus replaced with a query time
expension of unprefixed query terms to the various prefixed
equivalent.

Reindexing will be needed for 'body:' searches to work correctly;
otherwise they will also match messages where the term occur in
headers (demonstrated by the new tests in T530-upgrade.sh)

2 files changed, 59 insertions, 0 deletions

diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh
index 69ebec68..2124dde2 100755
--- a/test/T530-upgrade.sh
+++ b/test/T530-upgrade.sh
@@ -117,4 +117,20 @@ MAIL_DIR/bar/new/21:2,
 MAIL_DIR/bar/new/22:2,
 MAIL_DIR/cur/51:2,"
 
+test_begin_subtest "body: same as unprefixed before reindex"
+notmuch search --output=messages body:close > OUTPUT
+notmuch search --output=messages close  > EXPECTED
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest "body: subset of unprefixed after reindex"
+notmuch reindex '*'
+notmuch search --output=messages body:close | sort > BODY
+notmuch search --output=messages close | sort > UNPREFIXED
+diff -e UNPREFIXED BODY | cut -c2- > OUTPUT
+cat <<EOF > EXPECTED
+d
+d
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
diff --git a/test/T740-body.sh b/test/T740-body.sh
new file mode 100755
index 00000000..548b30a4
--- /dev/null
+++ b/test/T740-body.sh
@@ -0,0 +1,43 @@
+#!/usr/bin/env bash
+test_description='search body'
+. $(dirname "$0")/test-lib.sh || exit 1
+
+add_message "[body]=thebody-1" "[subject]=subject-1"
+add_message "[body]=nothing-to-see-here-1" "[subject]=thebody-1"
+
+test_begin_subtest 'search with body: prefix'
+notmuch search body:thebody | notmuch_search_sanitize > OUTPUT
+cat <<EOF > EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; subject-1 (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest 'search without body: prefix'
+notmuch search thebody | notmuch_search_sanitize > OUTPUT
+cat <<EOF > EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; subject-1 (inbox unread)
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; thebody-1 (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest 'negated body: prefix'
+notmuch search thebody and not body:thebody | notmuch_search_sanitize > OUTPUT
+cat <<EOF > EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; thebody-1 (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest 'search unprefixed for prefixed term'
+notmuch search subject | notmuch_search_sanitize > OUTPUT
+cat <<EOF > EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; subject-1 (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_begin_subtest 'search with body: prefix for term only in subject'
+notmuch search body:subject | notmuch_search_sanitize > OUTPUT
+cat <<EOF > EXPECTED
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
+test_done