| Age | Commit message (Collapse) | Author |
|---|
|
It is wasteful to remove a filename term when the whole message
document is about to be removed from the database. Profiling with perf
shows this takes a significant portion of the time when cleaning up
removed files in the database.
The logic of n_d_remove_message becomes a bit more convoluted here in
order to make the change minimal.
It is possible that this function can be further optimized, since the
expansion of filename terms into filenames is probably not needed
here.
|
|
Instead of skipping indexing all attachments, we check of a (user
configured) mime type that is indexable as text.
|
|
_notmuch_message_delete can return (at least)
NOTMUCH_STATUS_XAPIAN_EXCEPTION, which we should not ignore.
|
|
In order for a database to actually be writeable, it must be the case that it
is open, not just the correct type of Xapian object. By explicitely
checking, we are able to provide better error reporting, in particular
for the previously broken test in T566-lib-message.
|
|
Commit [0] left the stemmer object accessible, but did not add
de-allocation code to notmuch_database_destroy. This commit corrects
that oversight.
Leak originally reported by Austin Ray [1].
[0]: 3202e0d1feba1ab955ba1c07098c00208f8f0ada
[1]: id:20220105224538.m36lnjn7rf3ieonc@athena
|
|
If we know the configuration is split, but there is no mail root
defined, this indicates a (lack of) configuration error. Currently
this can only arise in XDG configurations.
|
|
It should not be necesary to have any config information here, hence
passing "" to n_d_open_with_config.
|
|
This will help provide more meaningful error messages without special
casing on the client side.
|
|
This change addresses two known issues with large sets of changes to
the database. The first is that as reported by Steven Allen [1],
notmuch commits are not "flushed" when they complete, which means that
if there is an open transaction when the database closes (or e.g. the
program crashes) then all changes since the last commit will be
discarded (nothing is irrecoverably lost for "notmuch new", as the
indexing process just restarts next time it is run). This does not
really "fix" the issue reported in [1]; that seems rather difficult
given how transactions work in Xapian. On the other hand, with the
default settings, this should mean one only loses less than a minutes
worth of work. The second issue is the occasionally reported "storm"
of disk writes when notmuch finishes. I don't yet have a test for
this, but I think committing as we go should reduce the amount of work
when finalizing the database.
[1]: id:20151025210215.GA3754@stebalien.com
|
|
Unfortunately, it doesn't make a difference if we call
cancel_transaction or not, all uncommited changes are discarded if
there is an open (unflushed) transaction.
|
|
The spacing of the declaration was wrong in ea30110.
|
|
Since most memory allocation is (ultimately) in the talloc context
defined by a notmuch_database_t pointer, this gives a more complete
view of memory still allocated at program shutdown.
We also change the talloc report in notmuch.c to mode "a" to avoid
clobbering the newly reported log.
|
|
This will allow more precise return values from various open related functions.
|
|
|
|
This promotes _choose_xapian_path from static to extern linkage in
order to share between open.cc and database.cc.
|
|
Introduce a new configuration value for the mail root, and use it to
locate mail messages in preference to the database.path (which
previously implied the mail messages were also in this location.
Initially only a subset of the CLI is tested in a split
configuration. Further changes will be needed for the remainder of the
CLI to work in split configurations.
|
|
This removes duplication between the struct element and the
configuration string_map entry. Create a simple wrapper for setting
the database path that makes sure the trailing / is stripped.
|
|
In the future Xapian will apparently support this more conveniently
for the cases other than READ_ONLY => READ_ONLY
Conceptually this function seems to fit better in lib/open.cc;
database.cc is still large enough that moving the function makes
sense.
|
|
Include the (currently unused) mode argument which will specify which
mode to re-open the database in. Functionality and docs to be
finalized in a followup commit.
|
|
This is the result of running
$ uncrustify --replace --config ../devel/uncrustify.cfg *.c *.h *.cc
in the lib directory
|
|
This will help share code with n_d_open_with_config.
|
|
The "back end" function takes an open notmuch database, which should
know its own path (i.e. the path needs to be cached in the
configuration data).
|
|
Reduce the size of database.cc, and prepare for extending the database
opening API
|
|
Reduce the size of database.cc, and limit the scope of prefix_table,
make sure it's accessed via a well-defined internal API.
|
|
database.cc is uncomfortably large, and some of the static data
structures do not need to be shared as much as they are.
This is a somewhat small piece to factor out, but it will turn out to
be helpful to further refactoring.
|
|
static_cast is a bit tricky to understand and error prone, so add a
second pointer to (potentially the same) Xapian database object that
we know has the right subclass.
|
|
The plan is to change the underlying representation.
|
|
I'm not sure what the point of modifying that right before destroying
the object is. In a future commit I want to remove that element of the
object, so simplify that task.
|
|
The error message and name were confusing when called in some "read
only" context.
|
|
Also clarify documentation of error return from n_d_needs_upgrade.
|
|
notmuch_database_get_version previously returned 0 on some errors, but
did not document this. Luckily 0 is not a valid database version.
|
|
In order to mimic the "best effort" API of Xapian to provide
information from a closed database when possible, do not
destroy the Xapian database object too early.
Because the pointer to a Xapian database is no longer nulled on close,
introduce a flag to track whether the notmuch database is open or not.
|
|
This will be mandatory as of Xapian 1.5. The API is also more
consistent with the FieldProcessor API, which helps code re-use a bit.
Note that this switches to using the built-in Xapian support for
prefixes on ranges (i.e. deleted code at beginning of
ParseTimeRangeProcessor::operator(), added prefix to constructor).
Another side effect of the migration is that we are generating smaller
queries, using one OP_VALUE_RANGE instead of an AND of two OP_VALUE_*
queries.
|
|
The old API was deprecated in Xapian 1.3.4 and (will be) removed in 1.5.0
|
|
Xapian 1.4 is over 3 years old now (1.4.0 released 2016-06-24),
and 1.2 has been deprecated in Notmuch version 0.27 (2018-06-13).
Xapian 1.4 supports compaction, field processors and retry locking;
conditionals checking compaction and field processors were removed
but user may want to disable retry locking at configure time so it
is kept.
|
|
This is the result of running
$ uncrustify --replace --config ../devel/uncrustify.cfg *.c *.h *.cc
in the lib directory
|
|
This essentially involves calling _notmuch_message_gen_terms once for
each user defined header.
|
|
This should not change the indexing process yet as nothing calls
_notmuch_message_gen_terms with a user prefix name. On the other hand,
it should not break anything either.
_notmuch_database_prefix does a linear walk of the list of (built-in)
prefixes, followed by a logarithmic time search of the list of user
prefixes. The latter is probably not really noticable.
|
|
This will be used to avoid needing a database access to resolve a db
prefix from the corresponding UI prefix (e.g. when indexing). Arguably
the setup of the separate header map does not belong here, since it is
about indexing rather than querying, but we currently don't have any
other indexing setup to do.
|
|
These tests will need to be updated if the Xapian
query print/debug format changes.
|
|
Several of these #defines were not actually used in the notmuch
codebase any longer. And as of GMime 3.0, g_mime_init takes no
arguments, so we can also drop the bogus RFC2047 argument that we were
passing and then #defining away.
signed-off-by: Daniel Kahn Gillmor <dkg@fifthhorseman.net>
|
|
The new `body:` field (in Xapian terms) or prefix (in slightly
sloppier notmuch) terms allows matching terms that occur only in the
body.
Unprefixed query terms should continue to match anywhere (header or
body) in the message.
This follows a suggestion of Olly Betts to use the facility (since
Xapian 1.0.4) to add the same field with multiple prefixes. The double
indexing of previous versions is thus replaced with a query time
expension of unprefixed query terms to the various prefixed
equivalent.
Reindexing will be needed for 'body:' searches to work correctly;
otherwise they will also match messages where the term occur in
headers (demonstrated by the new tests in T530-upgrade.sh)
|
|
We missed this when we changed to binary fields.
|
|
|
|
Although the situation is complicated by the value fields (which are
taken from a single file), this comment is now more false than true.
|
|
This change allows queries of the form
thread:{from:me} and thread:{from:jian} and not thread:{from:dave}
This is still somewhat brute-force, but it's a big improvement over
both the shell script solution and the previous proposal [1], because it
does not build the whole thread structure just generate a
query. A further potential optimization is to replace the calls to
notmuch with more specialized Xapian code; in particular it's not
likely that reading all of the message metadata is a win here.
[1]: id:20170820213240.20526-1-david@tethera.net
|
|
This reverts commit 4f5bbaf7e2cecfe5022ba4b28915cccfb7ccb12d.
|
|
This change allows queries of the form
thread:{from:me} and thread:{from:jian} and not thread:{from:dave}
This is still somewhat brute-force, but it's a big improvement over
both the shell script solution and the previous proposal [1], because it
does not build the whole thread structure just generate a
query. A further potential optimization is to replace the calls to
notmuch with more specialized Xapian code; in particular it's not
likely that reading all of the message metadata is a win here.
[1]: id:20170820213240.20526-1-david@tethera.net
|
|
If we're going to reuse the crypto code across both the library and
the client, then it needs to report error states properly and not
write to stderr.
|
|
C99 stdbool turned 18 this year. There really is no reason to use our
own, except in the library interface for backward
compatibility. Convert the lib internally to stdbool.
|
