thread-based email index, search, and tagging https://git.notmuchmail.org/git/notmuch/atom?h=0.26_rc2 2017-12-08T12:08:47Z 2017-12-08T12:08:47Z Daniel Kahn Gillmor dkg@fifthhorseman.net 2017-12-08T06:24:00Z urn:sha1:6a9626a2fdddf6115bcf97982fd10053bf48e942 There are some situations where the user wants to get rid of the cleartext index of a message. For example, if they're indexing encrypted messages normally, but suddenly they run across a message that they really don't want any trace of in their index. In that case, the natural thing to do is: notmuch reindex --decrypt=false id:whatever@example.biz But of course, clearing the cleartext index without clearing the stashed session key is just silly. So we do the expected thing and also destroy any stashed session keys while we're destroying the index of the cleartext. Note that stashed session keys are stored in the xapian database, but xapian does not currently allow safe deletion (see https://trac.xapian.org/ticket/742). As a workaround, after removing session keys and cleartext material from the database, the user probably should do something like "notmuch compact" to try to purge whatever recoverable data is left in the xapian freelist. This problem really needs to be addressed within xapian, though, if we want it fixed right. 2017-10-21T22:53:19Z Daniel Kahn Gillmor dkg@fifthhorseman.net 2017-10-21T02:25:41Z urn:sha1:4dfcc8c9b2e1dbb965f69283dca50c7581c88050 If we see index options that ask us to decrypt when indexing a message, and we encounter an encrypted part, we'll try to descend into it. If we can decrypt, we add the property index.decryption=success. If we can't decrypt (or recognize the encrypted type of mail), we add the property index.decryption=failure. Note that a single message may have both values of the "index.decryption" property: "success" and "failure". For example, consider a message that includes multiple layers of encryption. If we manage to decrypt the outer layer ("index.decryption=success"), but fail on the inner layer ("index.decryption=failure"). Because of the property name, this will be automatically cleared (and possibly re-set) during re-indexing. This means it will subsequently correspond to the actual semantics of the stored index. 2017-10-21T22:53:08Z Daniel Kahn Gillmor dkg@fifthhorseman.net 2017-10-21T02:25:40Z urn:sha1:0bb05ff693737c5b91d6a64f6209984a6c418c46 This allows us to create new properties that will be automatically set during indexing, and cleared during re-indexing, just by choice of property name. 2017-10-10T01:27:16Z Jani Nikula jani@nikula.org 2017-10-07T08:44:05Z urn:sha1:008a5e92eb157e2bb8622cb2fbf644deba5ba4b4 C99 stdbool turned 18 this year. There really is no reason to use our own, except in the library interface for backward compatibility. Convert the lib internally to stdbool. 2017-09-06T00:51:57Z David Bremner david@tethera.net 2017-08-27T23:58:22Z urn:sha1:debfae20db9a45ce7e12ac51db2b0104cd3b0be2 This is still a bit stopgap to be only choosing one set of headers, but this seems like a more defensible set of headers to choose. 2017-08-30T00:56:21Z David Bremner david@tethera.net 2017-08-20T01:07:26Z urn:sha1:0a40ea4b48357c69de6456a305f75b7bda649c9d I considered a higher level interface where the caller passes a tag name rather than a flag character, but the role of the "unread" tag is particularly confusing with such an interface. 2017-08-30T00:51:10Z David Bremner david@tethera.net 2017-08-20T01:07:25Z urn:sha1:8a8fb39b0c0199f224f4477cbcab2d7a578fee0f In a future commit this will allow querying maildir flags seperately from tags to allow resolving certain conflicts. 2017-08-23T10:55:12Z Daniel Kahn Gillmor dkg@fifthhorseman.net 2017-08-17T23:14:26Z urn:sha1:eb232ee0aba8f031fe4f0cb509682a321d85e06e There are at least three places in notmuch that can trigger an indexing action: * notmuch new * notmuch insert * notmuch reindex I have plans to add some indexing options (e.g. indexing the cleartext of encrypted parts, external filters, automated property injection) that should properly be available in all places where indexing happens. I also want those indexing options to be exposed by (and constrained by) the libnotmuch C API. This isn't yet an API break because we've never made a release with notmuch_param_t. These indexing options are relevant in the listed places (and in the libnotmuch analogues), but they aren't relevant in the other kinds of functionality that notmuch offers (e.g. dump/restore, tagging, search, show, reply). So i think a generic "param" object isn't well-suited for this case. In particular: * a param object sounds like it could contain parameters for some other (non-indexing) operation. This sounds confusing -- why would i pass non-indexing parameters to a function that only does indexing? * bremner suggests online a generic param object would actually be passed as a list of param objects, argv-style. In this case (at least in the obvious argv implementation), the params might be some sort of generic string. This introduces a problem where the API of the library doesn't grow as new options are added, which means that when code outside the library tries to use a feature, it first has to test for it, and have code to handle it not being available. The indexopts approach proposed here instead makes it clear at compile time and at dynamic link time that there is an explicit dependency on that feature, which allows automated tools to keep track of what's needed and keeps the actual code simple. My proposal adds the notmuch_indexopts_t as an opaque struct, so that we can extend the list of options without causing ABI breakage. The cost of this proposal appears to be that the "boilerplate" API increases a little bit, with a generic constructor and destructor function for the indexopts struct. More patches will follow that make use of this indexopts approach. 2017-08-02T01:17:47Z Daniel Kahn Gillmor dkg@fifthhorseman.net 2017-06-04T12:32:34Z urn:sha1:5b93fa6e70c905e3c5f2a8109683db29ccfd5bcf This new function asks the database to reindex a given message. The parameter `indexopts` is currently ignored, but is intended to provide an extensible API to support e.g. changing the encryption or filtering status (e.g. whether and how certain non-plaintext parts are indexed). 2017-08-02T01:17:47Z David Bremner david@tethera.net 2017-06-04T12:32:33Z urn:sha1:34d77539925f8e743830a2c8df0a079956ae8be3 Testing will be provided via use in notmuch_message_reindex