David Bremner [Tue, 19 Apr 2022 11:32:14 +0000 (08:32 -0300)]
CLI/git: cache git indices
If the private index file matches a previously known revision of the
database, we can update the index incrementally using the recorded
lastmod counter. This is typically much faster than a full update,
although it could be slower in the case of large changes to the
database.
The "git-read-tree HEAD" is also a bottleneck, but unfortunately
sometimes is needed. Cache the index checksum and hash to reduce the
number of times the operation is run. The overall design is a
simplified version of the PrivateIndex class.
David Bremner [Sat, 16 Apr 2022 22:44:10 +0000 (19:44 -0300)]
CLI/git: add @timed decorator, time a few functions
Perf will show which binaries are using the CPU cycles, and standard
python profilers will show which python functions, but neither is
great at finding which call to an external binary is taking time, or
locating I/O hotspots.
David Bremner [Sun, 10 Apr 2022 17:22:57 +0000 (14:22 -0300)]
CLI/git: replace enumeration of tags with sexp query.
Unlike the (current) infix query parser provided by Xapian, the
notmuch specific sexp query parser supports prefixed wildcard queries,
so use those. In addition to being somewhat faster, this avoids
needing to escape all of the user's tags to pass via the shell.
David Bremner [Sun, 15 May 2022 16:47:54 +0000 (13:47 -0300)]
CLI/git: rename environment variables.
Although the code required to support both new and old environment
variables is small, it complicates the semantics of configuration, and
make the documentation harder to follow.
W. Trevor King [Mon, 19 Jan 2015 00:40:10 +0000 (16:40 -0800)]
CLI/git: Add an 'init' command
For folks that want to start versioning a new tag-space, instead of
cloning one that someone else has already started.
The empty-blob hash-object call avoids errors like:
$ nmbug commit
error: invalid object 100644 e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 for
'tags/...'
fatal: git-write-tree: error building trees
'git HASH(0x9ef3eb8) write-tree' exited with nonzero value
David Bremner suggested [1]:
$ git hash-object -w /dev/null
instead of my Python version of:
$ git hash-object -w --stdin <&-
but I expect that closing stdin is more portable than the /dev/null
path (which doesn't exist on Windows, for example).
The --bare init and use of NMBGIT as the work tree (what could go
wrong with an empty commit?) are suggestions from Michal Sojka [2].
David Bremner [Sat, 7 May 2022 16:53:08 +0000 (13:53 -0300)]
CLI/git: drop support for python < 3.2
Debian stable had python 3.4.2 3 releases ago (approximately 6 years
ago), so attempting to keep track of the changes in python is probably
no longer worthwhile. We already require python 3.5 for the
python-cffi bindings (although those are not yet used in notmuch-git).
jao [Wed, 1 Jun 2022 09:53:36 +0000 (10:53 +0100)]
emacs: notmuch-show-header-line: allow format strings and functions
If a string value is assigned to notmuch-show-header-line, it's used
as a format string to be passed passed to format-spec with `%s`
substituted by the message's subject. If a function is given, it's
called with the subject as argument, and its return value used as
header line.
As before, t means displaying the subject and nil not using any header
line.
David Bremner [Sat, 21 May 2022 18:06:30 +0000 (15:06 -0300)]
test: replace deprecated use of notmuch_database_open
This is a bit more involved than replacing the use of
notmuch_database_open_verbose, as we have to effectively inline the
definition of notmuch_database_open.
Michael J Gruber [Thu, 26 May 2022 16:52:32 +0000 (18:52 +0200)]
test: make T450 independent of application/octet-stream interpretation
The actual content type of `application/octet-stream` is up to content
type detection of the reader, and thus may not be stable across
implementations or versions. This showed up when
introduced a test for omitting a part of type `text/html` because it
expected a part of type `application/octet-stream` to remain in place,
i.e. a part of "unstable type". In particular, tests with `fd46fc19`
would succeed on RHEL/EPEL but fail on all current Fedoras with
```
FAIL multipart/alternative hides html by default
--- T450-emacs-show.16.notmuch-show-multipart-alternative 2022-05-26 15:34:42.100557244 +0000
+++ T450-emacs-show.16.OUTPUT 2022-05-26 15:34:42.102557207 +0000
@@ -24,7 +24,7 @@
uses 64 as the
buffer size.
[ text/html (hidden) ]
-[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-diff) ]
+[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: application/octet-stream (as text/x-patch) ]
From e3bc4bbd7b9d0d086816ab5f8f2d6ffea1dd3ea4 Mon Sep 17 00:00:00 2001
From: Alexander Botero-Lowry <alex.boterolowry@gmail.com>
Date: Tue, 17 Nov 2009 11:30:39 -0800
```
due to the different type detected.
Fix this by giving that message a specicific type of `text/x-diff` in
the test corpus, and adjust all affected test outputs.
Signed-off-by: Michael J Gruber <git@grubix.eu> Amended-by: db, fix some trailing whitespace
This variable is important for people who want to change the default
behaviour when displaying multipart/alternative messages. Previously
it was undocumented. Add a defcustom to help users and copy some
documentation from the wiki. The usual machinery of re-using
docstrings is a bit tricky to use here because it mangles the example
lisp code, and the link to the info node should not be in e.g. the
html page.
Add a simple test to make sure the switch from defvar to defcustom did
not break something obvious.
7228fe68 ("configure: restructure gmime cert validity checker code",
2022-04-09) restructured generated C code to repurpose it later on. This
put usage of `validity` within an `#if`, resulting in an "unused
warning" if that `#if` is not executed.
Put the variable declariation inside the same if branch and, thus, quel
the warning.
a1d139de ("lib: add sexp: prefix to Xapian (infix) query parser.",
2022-04-09) introduced sfsexp infix queries. This requires the infix
preprocessor to be built in in a way which does not require sfsexp when
notmuch is built without it.
Make the preprocessor throw a Xapian error in this case (and fix the
build).
David Bremner [Tue, 12 Apr 2022 20:15:56 +0000 (22:15 +0200)]
test/smime: fix signature verification test with newer gmime.
The extra machinery to check for the actual output format is justified
by the possibility that distros may patch this newer output format
into older versions of gmime.
Amended-by: Michael J Gruber <git@grubix.eu> Signed-off-by: Michael J Gruber <git@grubix.eu> Amended-again-by: db
David Bremner [Fri, 25 Feb 2022 02:41:03 +0000 (22:41 -0400)]
lib: do not phrase parse prefixed bracketed subexpressions
Since Xapian does not preserve quotes when passing the subquery to a
field processor, we have to make a guess as to what the user
intended. Here the added assumption is that a string surrounded by
parens is not intended to be a phrase.
David Bremner [Sat, 12 Feb 2022 20:27:34 +0000 (16:27 -0400)]
test/emacs: split out tagging related tests
T310-emacs is one of the largest and longest running sets of
tests. Splitting out the tagging operations will help maintainability
as well as potentially improve the parallel running time of the test
suite. Some slowdown in running the tests sequentially may result
since there is repeated setup.
David Bremner [Sat, 12 Feb 2022 20:27:33 +0000 (16:27 -0400)]
test: split variable settings to their own file
This allows sharing more variable settings between the (correctness)
tests and the performance-tests. Unfortunately it seems a bit tricky
to move settings to test-lib-common.sh, as that is sourced late in
test-lib.sh, and moving it earlier breaks things.
David Bremner [Sun, 16 Jan 2022 19:31:00 +0000 (15:31 -0400)]
devel/notmuch-web: single user web front end using python-cffi
Originally contributed by Brian Sniffen [1]. Quite a few unpublished
fixes from Daniel Kahn Gilmour. We've been running it a few years now
as the "official" archive of the notmuch mailing list. There also a
few fixes from myself and Austin Ray. Finally I converted it to the
new python bindings.
This commit squashes the several years of development history and
moves it under devel, in recognition of the now established role it
plays in the project infrastructure.
David Bremner [Sun, 6 Feb 2022 13:38:16 +0000 (09:38 -0400)]
doc: add configuration section to notmuch-insert(1)
This is partially redudant given some existing cross references, but
it is useful to have all of the config keys listed in one place, to
help keep track of them if nothing else.
David Bremner [Sun, 6 Feb 2022 13:07:13 +0000 (09:07 -0400)]
doc: alphabetize options in notmuch-config(1)
Originally (I think) these were in the order generated by notmuch
setup. As the number of options grows, and several are not in the
initial setup generated file, the original order becomes less useful
for users. This commit alphabetizes the keys to help users
search. There is only one content change, an added cross-reference
from user.other_email to user.primary_email.
David Bremner [Sat, 12 Feb 2022 02:47:03 +0000 (22:47 -0400)]
CLI/insert: escape envelope from
The idea is to do as little parsing and modification of the delivered
message as possible. Luckily the position of the "envelope header"
lets us escape it by replacing the first 5 characters of the stream
with a regular header name (with ':').
David Bremner [Sat, 5 Feb 2022 19:52:10 +0000 (15:52 -0400)]
test: start new corpus of test messages for indexing code
This particular message is not recognized by notmuch as mail, but is
fine according to e.g. mutt. The trigger for this bad behaviour seems
to be a second "From " ocurring at the beginning of the line but
inside an attachment.
David Bremner [Wed, 9 Feb 2022 01:01:16 +0000 (21:01 -0400)]
test: known broken test for list(db.config) in python-cffi bindings
As of notmuch 0.34.2 [1], the python-cffi bindings make available the
configuration from both a config file and the database when accessing
Database.config like a dictionary. It is therefore confusing that the
iterator operations only work on the configuration information stored
in the database.