From: Mohsin Kaleem Date: Sun, 10 Mar 2024 18:57:41 +0000 (+0000) Subject: emacs: Add new option notmuch-search-hide-excluded X-Git-Url: https://git.notmuchmail.org/git?p=notmuch;a=commitdiff_plain;h=HEAD;hp=1979145b91fa85d6952b94db561a46238265d910 emacs: Add new option notmuch-search-hide-excluded The new notmuch-search-hide-excluded option allows users to configure whether to show or hide excluded messages (as determined by search.exclude_tags in the local notmuch config file). It defaults to true for now to maintain backwards-compatibility with how notmuch-{search,tree} already worked. New commands notmuch-search-toggle-hide-excluded and notmuch-tree-toggle-exclude have also been added. They toggle the value of notmuch-search-hide-excluded for the search in the current search or tree buffer. It's bound to "i" in the respective keymaps for these modes. Lastly I've amended some calls to notmuch-tree and notmuch-unthreaded which didn't pass through the buffer local value of notmuch-search-oldest-first (and now notmuch-search-exclude). Examples of where I've done this include: + notmuch-jump-search + notmuch-tree-from-search-current-query + notmuch-unthreaded-from-search-current-query + notmuch-tree-from-search-thread A new test file for Emacs has been added which covers the usage of the new `notmuch-search-hide-excluded' option and interactively hiding or showing mail with excluded tags. These test cover the basic usage of the `notmuch-search-toggle-hide-excluded' command in notmuch-search, notmuch-tree and notmuch-unthreaded searches. These tests also cover the persistence of the current value of the hide-excluded mail option as a user switches from between these different search commands. [1]: id:87ilxlxsng.fsf@kisara.moe Amended-by: db, fix indentation in T461-emacs-search-exclude.sh --- diff --git a/.dir-locals.el b/.dir-locals.el index fc75ae61..b3ddffe8 100644 --- a/.dir-locals.el +++ b/.dir-locals.el @@ -15,7 +15,7 @@ (emacs-lisp-mode (indent-tabs-mode . t) (tab-width . 8)) - (shell-mode + (sh-mode (indent-tabs-mode . t) (tab-width . 8) (sh-basic-offset . 4) diff --git a/.gitignore b/.gitignore index 468b660a..eda6d9cf 100644 --- a/.gitignore +++ b/.gitignore @@ -1,18 +1,24 @@ +*.[ao] +*.stamp +*cscope* +*~ +.*.swp +/.deps /.first-build-message +/.stamps /Makefile.config +/bindings/python-cffi/build/ +/lib/libnotmuch*.dylib +/lib/libnotmuch.so* +/nmbug +/notmuch +/notmuch-git +/notmuch-shared +/releases /sh.config +/sphinx.config /version.stamp +/bindings/python-cffi/_notmuch_config.py TAGS tags -*cscope* -/.deps -/notmuch -/notmuch-shared -/lib/libnotmuch.so* -/lib/libnotmuch*.dylib -*.[ao] -*~ -.*.swp -/releases -/.stamps -*.stamp +__pycache__ diff --git a/.travis.yml b/.travis.yml index f9516bde..5bb03de6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,27 +1,29 @@ language: c -dist: xenial +dist: bionic addons: apt: sources: - sourceline: 'ppa:xapian-backports/ppa' - - sourceline: 'ppa:notmuch/notmuch' packages: - dtach - libxapian-dev - libgmime-3.0-dev - libtalloc-dev - python3-sphinx + - python3-cffi + - python3-pytest + - python3-setuptools + - libpython3-all-dev - gpgsm script: - ./configure - - make download-test-databases - make test notifications: irc: channels: - - "chat.freenode.net#notmuch" + - "irc.libera.chat#notmuch" on_success: change diff --git a/AUTHORS b/AUTHORS index 5fe5006f..6e872084 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,5 +1,6 @@ -Carl Worth is the primary author of Notmuch. -But there's really not much that he's done. There's been a lot of +Carl Worth was the original author of Notmuch. +David Bremner has maintained Notmuch since release 0.6 (2011). But +there's really not much that they've done. There's been a lot of standing on shoulders here: William Morgan deserves credit for providing the primary inspiration @@ -21,10 +22,108 @@ engine that does the really heavy lifting, as well as the various system libraries, compilers, and the kernel that make it all work (thanks GNU, thanks Linux). Thanks to everyone who has played a part! +The following list of people have at least 15 lines of code in the +Notmuch 0.31 release (calculated by devel/author-scan.sh). + + David Bremner + Carl Worth + Jani Nikula + Austin Clements + Daniel Kahn Gillmor + Mark Walters + Floris Bruynooghe + David Edmondson + Tomi Ollila + Sebastian Spaeth + Ali Polatel + Michal Sojka + Justus Winter + Sebastien Binet + W. Trevor King + Jameson Graef Rollins + Felipe Contreras + Jonas Bernoulli + Pieter Praet + Peter Feigl + Dmitry Kurochkin + Peter Wang + Gregor Zattler + Daniel Schoepe + Keith Packard + Adam Wolfe Gordon + Stefano Zacchiroli + Vincent Breitmoser + laochailan + Ben Gamari + Aaron Ecay + l-m-h@web.de + Thomas Jost + Jesse Rosenthal + Dirk Hohndel + Blake Jones + Damien Cassou + Anton Khirnov + Matt Armstrong + Vladimir Panteleev + William Casarin + Örjan Ekeberg + Jan Janak + Patrick Totzke + Ruben Pollan + rhn + Ioan-Adrian Ratiu + Ethan Glasser-Camp + Chunyang Xu + Todd + Chris Wilson + Yuri Volchkov + Cédric Cabessa + Mark Anderson + Jed Brown + Maxime Coste + Ludovic LANGE + Sebastian Poeplau + Mikhail + Keith Amidon + Gaute Hope + martin f. krafft + Jeffrey C. Ollie + Jameson Rollins + Scott Henson + Bart Trojanowski + Vladimir Marek + Servilio Afre Puentes + Tomas Carnecky + Kevin McCarthy + Kevin J. McCarthy + Scott Robinson + Wael M. Nasreddine + Charles Celerier + Olly Betts + Istvan Marko + Florian Klink + Thibaut Horel + Joel Borggrén-Franck + Ingmar Vanhassel + Olivier Taïbi + Ian Main + Alexander Botero-Lowry + Luis Ressel + Sergei Shilovsky + Trevor Jim + Uli Scholler + Matthew Lear + Jinwoo Lee + Amadeusz Å»ołnowski + Here is an incomplete list of other people that have made contributions to Notmuch (whether by code, bug reporting/fixes, ideas, inspiration, testing or feedback): -Martin Krafft -Keith Packard -Jamey Sharp + Martin Krafft + Jamey Sharp + +The Notmuch project acknowledges the contributions of the following +organizations via their employees + + Google LLC diff --git a/INSTALL b/INSTALL index f1236e71..8054fafa 100644 --- a/INSTALL +++ b/INSTALL @@ -32,13 +32,6 @@ Talloc, and zlib which are each described below: Xapian is available from https://xapian.org - Note: Notmuch will work best with Xapian 1.0.18 (or later) or - Xapian 1.1.4 (or later). Previous versions of Xapian (whether - 1.0 or 1.1) had a performance bug that made notmuch very slow - when modifying tags. This would cause distracting pauses when - reading mail while notmuch would wait for Xapian when removing - the "inbox" and "unread" tags from messages in a thread. - GMime ----- GMime provides decoding of MIME email messages for Notmuch. @@ -48,6 +41,15 @@ Talloc, and zlib which are each described below: GMime is available from https://github.com/jstedfast/gmime + Sfsexp + ------ + + sfsexp is the "small fast s-expression" library. Notmuch + optionally use it to provide a second query parser. + + sfsexp is available from https://github.com/mjsottile/sfsexp. + In Debian Bookworm and later, install libsexp-dev. + Talloc ------ Talloc is a memory-pool allocator used by Notmuch. @@ -95,7 +97,7 @@ dependencies with a single simple command line. For example: For Fedora and similar: - sudo yum install xapian-core-devel gmime-devel libtalloc-devel zlib-devel python3-sphinx texinfo info + sudo dnf install xapian-core-devel gmime30-devel libtalloc-devel zlib-devel python3-sphinx texinfo info On other systems, a similar command can be used, but the details of the package names may be different. diff --git a/Makefile.global b/Makefile.global index 0aee5876..7a7a3c6d 100644 --- a/Makefile.global +++ b/Makefile.global @@ -1,3 +1,4 @@ +# -*- makefile-gmake -*- # Here's the (hopefully simple) versioning scheme. # # Releases of notmuch have a two-digit version (0.1, 0.2, etc.). We @@ -16,7 +17,7 @@ else DATE:=$(shell date +%F) endif -VERSION:=$(shell cat ${srcdir}/version) +VERSION:=$(shell cat ${srcdir}/version.txt) ELPA_VERSION:=$(subst ~,_,$(VERSION)) ifeq ($(filter release release-message pre-release update-versions,$(MAKECMDGOALS)),) ifeq ($(IS_GIT),yes) @@ -35,10 +36,10 @@ endif endif UPSTREAM_TAG=$(subst ~,_,$(VERSION)) -DEB_TAG=debian/$(UPSTREAM_TAG)-1 RELEASE_HOST=notmuchmail.org RELEASE_DIR=/srv/notmuchmail.org/www/releases +DOC_DIR=/srv/notmuchmail.org/www/doc/latest RELEASE_URL=https://notmuchmail.org/releases TAR_FILE=$(PACKAGE)-$(VERSION).tar.xz ELPA_FILE:=$(PACKAGE)-emacs-$(ELPA_VERSION).tar @@ -49,10 +50,9 @@ DETACHED_SIG_FILE=$(TAR_FILE).asc PV_FILE=bindings/python/notmuch/version.py # Smash together user's values with our extra values -STD_CFLAGS := -std=gnu99 -FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(CPPFLAGS) $(STD_CFLAGS) $(CFLAGS) $(WARN_CFLAGS) $(extra_cflags) $(CONFIGURE_CFLAGS) -FINAL_CXXFLAGS = $(CPPFLAGS) $(CXXFLAGS) $(WARN_CXXFLAGS) $(extra_cflags) $(extra_cxxflags) $(CONFIGURE_CXXFLAGS) -FINAL_NOTMUCH_LDFLAGS = $(LDFLAGS) -Lutil -lnotmuch_util -Llib -lnotmuch +FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(WARN_CFLAGS) $(extra_cflags) $(CPPFLAGS) $(CONFIGURE_CFLAGS) $(CFLAGS) +FINAL_CXXFLAGS = $(WARN_CXXFLAGS) $(extra_cflags) $(extra_cxxflags) $(CPPFLAGS) $(CONFIGURE_CXXFLAGS) $(CXXFLAGS) +FINAL_NOTMUCH_LDFLAGS = -Lutil -lnotmuch_util -Llib -lnotmuch $(LDFLAGS) ifeq ($(LIBDIR_IN_LDCONFIG),0) FINAL_NOTMUCH_LDFLAGS += $(RPATH_LDFLAGS) endif diff --git a/Makefile.local b/Makefile.local index 3c6dacbc..7699c208 100644 --- a/Makefile.local +++ b/Makefile.local @@ -1,7 +1,7 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- .PHONY: all -all: notmuch notmuch-shared build-man build-info ruby-bindings +all: notmuch notmuch-shared build-man build-info ruby-bindings python-cffi-bindings notmuch-git nmbug ifeq ($(MAKECMDGOALS),) ifeq ($(shell cat .first-build-message 2>/dev/null),) @NOTMUCH_FIRST_BUILD=1 $(MAKE) --no-print-directory all @@ -19,7 +19,7 @@ endif # Depend (also) on the file 'version'. In case of ifeq ($(IS_GIT),yes) # this file may already have been updated. -version.stamp: $(srcdir)/version +version.stamp: $(srcdir)/version.txt echo $(VERSION) > $@ $(TAR_FILE): @@ -30,12 +30,12 @@ $(TAR_FILE): echo "Warning: No signed tag for $(VERSION)"; \ fi ; \ git archive --format=tar --prefix=$(PACKAGE)-$(VERSION)/ $$ref > $(TAR_FILE).tmp - echo $(VERSION) > version.tmp + echo $(VERSION) > version.txt.tmp ct=`git --no-pager log -1 --pretty=format:%ct $$ref` ; \ tar --owner root --group root --append -f $(TAR_FILE).tmp \ --transform s_^_$(PACKAGE)-$(VERSION)/_ \ - --transform 's_.tmp$$__' --mtime=@$$ct version.tmp - rm version.tmp + --transform 's_.tmp$$__' --mtime=@$$ct version.txt.tmp + rm version.txt.tmp xz -C sha256 -9 < $(TAR_FILE).tmp > $(TAR_FILE) @echo "Source is ready for release in $(TAR_FILE)" @@ -45,6 +45,15 @@ $(SHA256_FILE): $(TAR_FILE) $(DETACHED_SIG_FILE): $(TAR_FILE) gpg --armor --detach-sign $^ +CLEAN := $(CLEAN) notmuch-git +notmuch-git: notmuch-git.py + cp $< $@ + chmod ugo+x $@ + +CLEAN := $(CLEAN) nmbug +nmbug: notmuch-git + ln -s $< $@ + .PHONY: dist dist: $(TAR_FILE) @@ -66,19 +75,20 @@ update-versions: release: verify-source-tree-and-version $(MAKE) VERSION=$(VERSION) verify-newer $(MAKE) VERSION=$(VERSION) clean + $(MAKE) VERSION=$(VERSION) sphinx-html $(MAKE) VERSION=$(VERSION) test git tag -s -m "$(PACKAGE) $(VERSION) release" $(UPSTREAM_TAG) $(MAKE) VERSION=$(VERSION) $(SHA256_FILE) $(DETACHED_SIG_FILE) ln -sf $(TAR_FILE) $(DEB_TAR_FILE) pristine-tar commit $(DEB_TAR_FILE) $(UPSTREAM_TAG) - git tag -s -m "$(PACKAGE) Debian $(VERSION)-1 upload (same as $(VERSION))" $(DEB_TAG) mkdir -p releases - mv $(TAR_FILE) $(SHA256_FILE) $(DETACHED_SIG_FILE) releases + mv $(TAR_FILE) $(DEB_TAR_FILE) $(SHA256_FILE) $(DETACHED_SIG_FILE) releases $(MAKE) VERSION=$(VERSION) release-message > $(PACKAGE)-$(VERSION).announce ifeq ($(REALLY_UPLOAD),yes) - git push origin $(VERSION) $(DEB_TAG) release pristine-tar + git push origin $(VERSION) release pristine-tar cd releases && scp $(TAR_FILE) $(SHA256_FILE) $(DETACHED_SIG_FILE) $(RELEASE_HOST):$(RELEASE_DIR) ssh $(RELEASE_HOST) "rm -f $(RELEASE_DIR)/LATEST-$(PACKAGE)-* ; ln -s $(TAR_FILE) $(RELEASE_DIR)/LATEST-$(TAR_FILE)" + rsync --verbose --delete --recursive doc/_build/html/ $(RELEASE_HOST):$(DOC_DIR) endif @echo "Please send a release announcement using $(PACKAGE)-$(VERSION).announce as a template." @@ -87,24 +97,29 @@ pre-release: $(MAKE) VERSION=$(VERSION) clean $(MAKE) VERSION=$(VERSION) test git tag -s -m "$(PACKAGE) $(VERSION) release" $(UPSTREAM_TAG) - git tag -s -m "$(PACKAGE) Debian $(VERSION)-1 upload (same as $(VERSION))" $(DEB_TAG) - $(MAKE) VERSION=$(VERSION) $(TAR_FILE) + $(MAKE) VERSION=$(VERSION) $(SHA256_FILE) $(DETACHED_SIG_FILE) ln -sf $(TAR_FILE) $(DEB_TAR_FILE) pristine-tar commit $(DEB_TAR_FILE) $(UPSTREAM_TAG) mkdir -p releases - mv $(TAR_FILE) $(DEB_TAR_FILE) releases + mv $(TAR_FILE) $(DEB_TAR_FILE) $(SHA256_FILE) $(DETACHED_SIG_FILE) releases +ifeq ($(REALLY_UPLOAD),yes) + git push origin $(UPSTREAM_TAG) release pristine-tar + cd releases && scp $(TAR_FILE) $(SHA256_FILE) $(DETACHED_SIG_FILE) $(RELEASE_HOST):$(RELEASE_DIR) +endif .PHONY: debian-snapshot debian-snapshot: make VERSION=$(VERSION) clean - TMPFILE=$$(mktemp /tmp/notmuch.XXXXXX); \ - cp debian/changelog $${TMPFILE}; \ - EDITOR=/bin/true dch -b -v $(VERSION)+1 \ - -D UNRELEASED 'test build, not for upload'; \ - echo '3.0 (native)' > debian/source/format; \ - debuild -us -uc; \ - mv -f $${TMPFILE} debian/changelog; \ - echo '3.0 (quilt)' > debian/source/format + RETVAL=0 && \ + TMPFILE=$$(mktemp /tmp/notmuch.XXXXXX) && \ + cp debian/changelog $${TMPFILE} && \ + (EDITOR=/bin/true dch -b -v $(VERSION)+1 \ + -D UNRELEASED 'test build, not for upload' && \ + echo '3.0 (native)' > debian/source/format && \ + debuild -us -uc); RETVAL=$$? \ + mv -f $${TMPFILE} debian/changelog; \ + echo '3.0 (quilt)' > debian/source/format; \ + exit $$RETVAL .PHONY: release-message release-message: @@ -118,8 +133,7 @@ release-message: @echo "Which can be verified with:" @echo "" @echo " $(RELEASE_URL)/$(SHA256_FILE)" - @echo -n " " - @cat releases/$(SHA256_FILE) + @sed "s/^/ /" releases/$(SHA256_FILE) @echo "" @echo " $(RELEASE_URL)/$(DETACHED_SIG_FILE)" @echo " (signed by `getent passwd "$$USER" | cut -d: -f 5 | cut -d, -f 1`)" @@ -167,7 +181,7 @@ release-checks: .PHONY: verify-newer verify-newer: - @echo -n "Checking that no $(VERSION) release already exists..." + @printf %s "Checking that no $(VERSION) release already exists..." @wget -q --no-check-certificate -O /dev/null $(RELEASE_URL)/$(TAR_FILE) ; \ case $$? in \ 8) echo "Good." ;; \ @@ -228,6 +242,7 @@ notmuch_client_srcs = \ gmime-filter-reply.c \ hooks.c \ notmuch.c \ + notmuch-client-init.c \ notmuch-compact.c \ notmuch-config.c \ notmuch-count.c \ @@ -290,7 +305,7 @@ CLEAN := $(CLEAN) notmuch notmuch-shared $(notmuch_client_modules) CLEAN := $(CLEAN) version.stamp notmuch-*.tar.gz.tmp CLEAN := $(CLEAN) .deps -DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config +DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config sphinx.config CPPCHECK_STAMPS := $(SRCS:%=.stamps/cppcheck/%) .PHONY: cppcheck diff --git a/NEWS b/NEWS index 4aefdf6b..cf8107f2 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,833 @@ +Notmuch 0.38.3 (2024-03-09) +=========================== + +CLI +--- + +Fix a bug in configuration code that caused the notmuch command to +erroneously report "Error: could not locate database" under some +circumstances. + +Notmuch 0.38.2 (2023-12-01) +=========================== + +Library +------- + +Make sorting of string maps lexicographic on (key,value) pairs. This +avoids some test failures due to variation in message property output +order. + +Emacs +----- + +Avoid extra separators after the last address in `notmuch-emacs-mua`. + + +Notmuch 0.38.1 (2023-10-26) +=========================== + +CLI +--- + +Report parse errors in config files. + +Emacs +----- + +Fix image toggling for Emacs >= 29.1. + +notmuch-mutt +------------ + +Fix syntax error in script. + +Notmuch 0.38 (2023-09-12) +========================= + +General +------- + +Support relative lastmod queries (see notmuch-sexp-queries(7) and +notmuch-search-terms(7) for details). + +Support indexing of designated attachments as text (see +notmuch-config(1) for details). + +CLI +--- + +Add options --offset and --limit to notmuch-show(1). + +Emacs +----- + +New commands notmuch-search-edit-search and notmuch-tree-edit-search. + +Introduce notmuch-tree-outline-mode. + +Some compatibility fixes for Emacs 29. At least one issue (hiding +images) remains in 0.38. + +Support completion when piping to external command. + +Fix regression in updating tag display introduced by 0.37. + +Library +------- + +Fix bug creating database when database.path is not set. + +Incremental performance improvements for message deletion. + +Catch Xapian exceptions when deleting messages. + +Sync removed message properties to the database. + +Replace use of thread-unsafe Query::MatchAll in the infix query +parser. + +Notmuch-Mutt +------------ + +Be more careful when clearing the results directory. + +Ruby +---- + +Use `database_open_with_config`, and provide compatible path search +semantics. + +Bugfix for query.get_sort + +Test Suite +---------- + +Support testing installed version of notmuch. + +Adapt to some breaking changes in glib handling of init files. + +Replace OpenPGP key used in test suite. + +Performance Tests +----------------- + +Update signatures for performance test corpus. + +Notmuch 0.37 (2022-08-21) +========================= + +Library +------- + +Fix uninitialized field in message objects. + +Improve exception handling and error propagation for message objects. + +Sexp Queries +------------ + +Add one sided lastmod ranges for sexp queries. + +Expand macro parameters inside regex and wildcard modifiers. + +Command Line Interface +---------------------- + +`notmuch help` now works for external commands. + +`NOTMUCH_CONFIG` is now passed to external commands and hooks. + +Promote the development tool `nmbug` to a user facing tool +`notmuch-git`. See notmuch-git(1) for details. + +Emacs +----- + +The function `notmuch-mua-mail` now moves point depending on the +provided arguments. + +Restrict what mime types are inlined in replies and on refresh. + +The functions in notmuch-query.el are now obsolete and may be removed +in a future version of Notmuch. + +Add some controls for lazy display of message bodies (See "Dealing +with large messages and threads" in the notmuch-emacs documentation). + +Allow the user to select (with '%') a different duplicate message file +to display. + +Use `message-dont-reply-to-names` in `notmuch-message-mode`. + +Support custom header-line format for notmuch-show mode. + +Notmuch 0.36 (2022-04-25) +========================= + +Library +------- + +Add the `sexp` prefix to the infix (traditional) query parser. This +allows specific subqueries to be parsed by the sexp parser (with +appropriate quoting). See `notmuch-search-terms(7)` for details. + +Add another heuristic to regexp fields to prevent phrase parsing of +bracketed sub-expressions. + +Command Line Interface +---------------------- + +Envelope from ("From ") headers are now escaped as X-Envelope-From: in +input to `notmuch-insert`. This prevents creating mbox files when +calling `notmuch-insert` from e.g. `postfix`. + +Python (CFFI) Bindings +---------------------- + +Use the `config_pairs` API in ConfigIterator. This returns all +matching key-value pairs, not just those that happen to be stored in +the database. + +Documentation +------------- + +Reorganize documentation for `notmuch-config`. Add a few links from +other man pages. + +Emacs +----- + +Bind the usual undo key sequences to new command +"notmuch-tag-undo". This allows transparent undo of tagging +operations. + +Tests +----- + +Fix smime.4 with newer gmime. Unset `XDG_DATA_HOME` and `MAILDIR` for tests. + +New add-on tool: notmuch-web +----------------------------- + +The new devel/ tool `notmuch-web` is a very thin web client. It +supports a full search interface for one user: there is no facility +for multiple users provided today. See the notmuch-web README file +for more information. + +Be careful about running it on a network-connected system: it will +expose a web interface that requires no authentication but exposes +your mail store. + +Notmuch 0.35 (2022-02-06) +========================= + +Library +------- + +Implement the `date` and `lastmod` fields in the S-expression parser. + +Ignore trailing `/` for pathnames in both query parsers. + +Rename configuration option `built_with.sexpr_query` to +`built_with.sexp_queries`. + +Do not assume a default mail root in split (e.g. XDG) configurations. + +Fix some small memory leaks in `notmuch_database_open_with_config`. + +CLI +--- + +Improve handling of leading/trailing punctuation and space for +configuration lists. + +Only ignore `.notmuch` at the top level in `notmuch new`. + +Optionally show extra headers in `notmuch show`. See +`show.extra_headers` in notmuch-config(1). + +Emacs +----- + +Drop `C-TAB` binding in hello mode, document `backtab`. + +Fix visual glitch in search mode by running `notmuch-search-hook` +lazily. + +Don't add space to completion candidates, improves compatibility with +third party completion frameworks. + +Make citation formatting more robust against whitespace. + +Use `--excludes=false` when generating the 'All tags' section. + +Use cached copy of message body for `Fcc`, avoiding variant bodies for +signed and/or encrypted messages. + +Add notmuch-logo.svg and use it in notmuch-hello view, replacing +the .png version. + +Make header line in show buffers optional. + +Add customizable names for search buffers. + +Build +----- + +Fix out-of-tree build for `python-cffi` bindings. + +Rearrange position of {C,CXX,CPP,LD}FLAGS, prevent some clashes with +installed version of notmuch. + +Ignore more configure options. + +Test Suite +---------- + +Replace some uses of `gdb` in the test suite with `LD_PRELOAD` based +shims. + +Use `--with-colons` for gpgsm, fix compatibility with newer gnupg. + +Python bindings +--------------- + +Add `matched` property to message objects. + +Users are reminded that the old python bindings in bindings/python are +deprecated; this will probably be the last major release that ships +them. + +Completion +---------- + +Use `database.mail_root` for path completion in bash/zsh. + +Notmuch 0.34.3 (2022-01-09) +=========================== + +Library +------- + +Do not crash when presented with a .notmuch directory without a +xapian/ subdirectory. + +Python Bindings (notmuch2) +-------------------------- + +Database constructor now searches for configuration by default. Pass +`config=Database.CONFIG.EMPTY` to disable. + +The `Message.replies()` method now returns OwnedMessage objects, to +prevent certain memory de-allocation errors. + +Fix for importing `notmuch2` module when building bindings +documentation. + +Notmuch 0.34.2 (2021-12-09) +=========================== + +Library +------- + +Fix a bug that wrongly resolved conflict between the `database_path` +parameter to `notmuch_database_open_with_config` and configuration +item `database.path` in favour of the latter. + +Python Bindings (notmuch2) +-------------------------- + +When building the documentation for the `notmuch2` python module, +import from the built module, not a system wide installed one. + +The notmuch2.Database constructor now uses the library function +`notmuch_database_open_with_config` to support the same configuration +and database location options as the library does. + +Fix some unprintable exception objects. + +Notmuch 0.34.1 (2021-11-03) +=========================== + +Library +------- + +Fix for deallocation and nulling of output parameter for +notmuch_database_{open_with,create_with,load}_config when errors +occur. This change fixes a potential use-after-free bug that has been +present since 0.32. This release also improves the documentation of +status returns for the same 3 functions. + +Notmuch 0.34 (2021-10-20) +========================= + +General +------- + +An optional new s-expression based query parser is available if +notmuch is built with the `sfsexp` library. See +notmuch-sexp-queries(7) for syntax, and use `notmuch config get +built_with.sexpr_query` to check if notmuch is compiled with +s-expression query support. + +CLI +--- + +Support multiple `Delivered-To` headers in notmuch-reply(1). + +Emacs +----- + +Functions are now allowed in `notmuch-search-result-format`. + +Improvements to unthreaded view on large threads. + +Tolerate bad/missing working directory for most commands. + +Allow customization of tree drawing symbols in notmuch-tree mode. + +Notmuch 0.33.2 (2021-09-30) +=========================== + +Tests +----- + +Improve reliability of T355-smime by changing gpgsm initialization. + +Notmuch 0.33.1 (2021-09-10) +=========================== + +General +------- + +Replace the fully-qualified-domain-name of the host with "localhost" +in the default email address. This should fix two flaky subtests in +T590-libconfig. + +Notmuch 0.33 (2021-09-03) +========================= + +Library +------- + +Correct documentation about transactions. + +Add a configurable automatic commit of transactions. See +`database.autocommit` in notmuch-config(1). + +Document the algorithm used to find a database. + +CLI +--- + +Define format version 5, which supports sorting the output of +notmuch-show. + +Emacs +----- + +`notmuch` no longer sets `mail-user-agent` on load. To restore the +previous behaviour of using notmuch to send mail by default, customize +`mail-user-agent` to `notmuch-user-agent`. + +`notmuch-company` now works in `org-msg`. + +Improve the display of messages from long threads in unthreaded mode. + +Prefer email addresses over User ID when showing valid signatures. + +Define a new face `notmuch-jump-key`. + +New commands in notmuch-tree view: `notmuch-tree-filter` and `notmuch-tree-filter-by-tag`. + +Honour `notmuch-show-text/html-blocked-images` when using `w3m` to +render html. + +Support toggling sort order in notmuch-tree mode. + +Ruby +---- + +Memory management of allocated notmuch objects (database, messages, +etc...) is now done via the Ruby GC. This removes all constraints on +the order of object destruction. Database close and destroy are +split, following an old library API change. + +Vim +--- + +Respect excluded tags when showing a thread. + +Documentation +------------- + +Fix doc build for Sphinx 4.0. + +Improve the markup and linking of the documentation. + +Notmuch 0.32.3 (2021-08-17) +=========================== + +Library +------- + +Restore location of database via `MAILDIR` environment variable, which +was broken in 0.32. + +Bump libnotmuch minor version to match the documentation in +`notmuch.h`. + +Correct documentation for deprecated database opening functions to +point out that they (still) do not load configuration information. + +CLI +--- + +Restore "notmuch config get built_with.*", which was broken in 0.32. + +Notmuch 0.32.2 (2021-06-27) +=========================== + +General +------- + +Fix a bug from 2017 that can add duplicate thread-id terms to message +documents. + +CLI +--- + +Fix small memory leak in notmuch new. + +Emacs +----- + +Add `(require 'seq)` for `seq-some`. + +Documentation +------------- + +Fix man page build for Sphinx 4.x. Fix variable name in emacs docs. + +Tests +----- + +Fix backup creation in `perf-test/T00-new`. Check openssl +prerequisite in `add_gpgsm_home`. + +Notmuch 0.32.1 (2021-05-15) +=========================== + +General +------- + +Restore handling of relative values for `database.path` that was +broken by 0.32. Extend this handling to `database.mail_root`, +`database.backup_dir`, and `database.hook_dir`. + +Reload certain metadata from Xapian database in +notmuch_database_reopen. This fixes a bug when adding messages to the +database in a pre-new hook. + +Fix default of `$HOME/mail` for `database.path`. In release 0.32, this +default worked only in "notmuch config". + +Emacs +----- + +Restore the dynamically bound variables `tag-changes` and `query` in +in `notmuch-before-tag-hook` and `notmuch-after-tag-hook`. + +Add `notmuch-jump-key` face to fontify keys in `notmuch-jump` and +related functions. To ensure backward compatibility, the new face +inherits from `minibuffer-prompt`. + +Notmuch 0.32 (2021-05-02) +========================= + +General +------- + +This release includes a significant overhaul of the configuration +management facilities for notmuch. The previous distinction between +configuration items that can be modified via plain text configuration +files and those that must be set in the database via the "notmuch +config" subcommand is gone, and all configuration items can be set in +both ways. The external configuration file overrides configuration +items in the database. The location of database, hooks, and +configuration files is now more flexible, with several new +configuration variables. In particular XDG locations are now supported +as fallbacks for database, configuration and hooks. For more +information see `notmuch-config(1)`. + +Library +------- + +To support the new configuration facilities, several functions and +constants have been added to the notmuch API. Most notably: + +- `notmuch_database_create_with_config` +- `notmuch_database_open_with_config` +- `notmuch_database_load_config` +- `notmuch_config_get` + +A previously requested API change is that `notmuch_database_reopen` is +now exposed (and generalized). + +The previously severe slowdowns from large numbers calls to +notmuch_database_remove_message or notmuch_message_delete in one +session has been fixed. + +As always, the canonical source of API documentation is +`lib/notmuch.h`, or the doxygen formatted documentation in `notmuch(3)`. + +CLI +--- + +The `notmuch config set` subcommand gained a `--database` argument to +specify that the database should be updated, rather than a config file. + +The speed of `notmuch new` and `notmuch reindex` in dealing with large +numbers of mail file deletions is significantly improved. + +Emacs +----- + +Completion related updates include: de-duplicating tags offered for +completion, use the actual initial input in address completion, allow +users to opt out of notmuch address completion, and do not force Ido +when prompting for senders. + +Some keymaps used to contain bindings for unnamed commands. These +lambda expressions have been replaced by named commands (symbols), to +ease customization. + +Lexical binding is now used in all notmuch-emacs libraries. + +Fix bug in calling `notmuch-mua-mail` with a non-nil RETURN-ACTION. + +Removed, inlined or renamed functions and variables: + `notmuch-address-locate-command`, + `notmuch-documentation-first-line`, `notmuch-folder`, + `notmuch-hello-trim', `notmuch-hello-versions` => `notmuch-version`, + `notmuch-remove-if-not`, `notmuch-search-disjunctive-regexp`, + `notmuch-sexp-eof`, `notmuch-split-content-type`, and + `notmuch-tree-button-activate`. + +Keymaps are no longer fset, which means they need to be referred to in +define-key directly (without quotes). If your Emacs configuration has a +keybinding like: + (define-key 'notmuch-show-mode-map "7" 'foo) +you should change it to: + (define-key notmuch-show-mode-map "7" 'foo) + +Notmuch 0.31.4 (2021-02-18) +=========================== + +Library +------- + +Fix include bug triggered by glib 2.67. + +Test +---- + +Fix race condition in T568-lib-thread. + +Notmuch 0.31.3 (2020-12-25) +=========================== + +Bindings +-------- + +Fix for exclude tags in notmuch2 bindings. + +Build +----- + +Portability update for T360-symbol-hiding. + +Library +------- + +Fix for memory error in notmuch_database_get_config_list. + +Notmuch 0.31.2 (2020-11-08) +=========================== + +Build +----- + +Catch one more occurrence of "version" in the build system, which +caused the file to be regenerated in the release tarball. + +Notmuch 0.31.1 (2020-11-08) +=========================== + +Library +------- + +Fix a memory initialization bug in notmuch_database_get_config_list. + +Build +----- + +Rename file 'version' to 'version.txt'. The old file name conflicted +with a C++ header for some compilers. + +Replace use of coreutils `realpath` in configure. + +Notmuch 0.31 (2020-09-05) +========================= + +Emacs +----- + +Notmuch now supports Emacs 27.1. You may need to set +`mml-secure-openpgp-sign-with-sender` and/or +`mml-secure-smime-sign-with-sender` to continue signing messages. + +The minimum supported major version of GNU Emacs is now 25.1. + +Add support for moving between threads after notmuch-tree-from-search-thread. + +New `notmuch-unthreaded` mode (added in Notmuch 0.30) + + Unthreaded view is a mode where each matching message is shown on a + separate line. + + The main key entries to unthreaded view are + + 'u' enter a query to view in unthreaded mode (works in hello, + search, show and tree mode) + + 'U' view the current query in unthreaded mode (works from search, + show and tree) + + Saved searches can also specify that they should open in unthreaded + view. + + Currently it is not possible to specify the sort order: it will + always be newest first. + +Notmuch-Mutt +------------ + +The shell pipeline executed by notmuch-mutt, which symlinked matched +files to a maildir for mutt to access is replaced with internal perl +processing. This search operation is now more portable, and somewhat +faster. + +Library +------- + +Improve exception handling in the library. This should +largely eliminate terminations inside the library due to uncaught +exceptions or internal errors. No doubt there are a few uncovered +code paths still; please report them as bugs. + +Add `notmuch_message_get_flag_st` and +`notmuch_message_has_maildir_flag_st`, and deprecate the existing +non-status providing versions. + +Move memory de-allocation from `notmuch_database_close` to +`notmuch_database_destroy`. + +Handle relative filenames in `notmuch_database_index_file`, as +promised in the documentation. + +Python Bindings +--------------- + +Documentation for the python bindings is merged into the main +sphinx-doc documentation tree. The merged documentation can be built +with e.g. `make sphinx-html` + +Dependencies +------------ + +We now support building notmuch against Xapian 1.5 (the current +development version). + +Test Suite +---------- + +Test suite fixes for compatibility with Emacs 27.1. + +Build System +------------ + +Man pages are now compressed reproducibly. + +Notmuch 0.30 (2020-07-10) +========================= + +S/MIME +------ + +Handle S/MIME (PKCS#7) messages -- one-part signed messages, encrypted +messages, and multilayer messages. Treat them symmetrically to +OpenPGP messages. This includes handling protected headers +gracefully. + +If you're using Notmuch with S/MIME, you currently need to configure +gpgsm appropriately. + +Mixed-up MIME Repair +-------------------- + +Detect and automatically repair a common form of message mangling +created by Microsoft Exchange (see index.repaired=mixedup in +notmuch-properties(7)). + +Protected Headers +----------------- + +Avoid indexing the legacy-display part of an encrypted message that +has protected headers (see +index.repaired=skip-protected-headers-legacy-display in +notmuch-properties(7)). + +Python +------ + +Drop support for python2, focus on python3. + +Introduce new CFFI-based python bindings in the python module named +"notmuch2". Officially deprecate (but still support) the older +"notmuch" module. + +Dependencies +------------ + +Support for Xapian 1.2 is removed. The minimum supported version of +Xapian is now 1.4.0. + +Notmuch 0.29.3 (2019-11-27) +=========================== + +General +------- + +Fix for use-after-free in notmuch_config_list_{key,val}. + +Fix for double close of file in notmuch-dump. + +Debian +------ + +Drop python2 support from shipped debian packaging. + +Notmuch 0.29.2 (2019-10-19) +=========================== + +General +------- + +Fix for file descriptor leak when opening gzipped mail files. Thanks +to James Troup for the bug report and the fix. + Notmuch 0.29.1 (2019-06-11) =========================== @@ -36,7 +866,7 @@ Command Line Interface ---------------------- `notmuch show` now supports --body=false and --include-html with ---format=text +--format=text. Fix several performance problems with `notmuch reindex`. @@ -48,7 +878,7 @@ information about cryptographic protections for the Subject header. Emacs ----- -Optionally check for missing attachements in outgoing messages (see +Optionally check for missing attachments in outgoing messages (see function `notmuch-mua-attachment-check`). Bind `B` to browse URLs in current message. diff --git a/README b/README index 0aa9a080..03bbb57f 100644 --- a/README +++ b/README @@ -73,5 +73,5 @@ information. There is also an IRC channel dedicated to talk about using and developing notmuch: - IRC server: irc.freenode.net + IRC server: irc.libera.chat Channel: #notmuch diff --git a/bindings/Makefile.local b/bindings/Makefile.local index 18f95835..9875123a 100644 --- a/bindings/Makefile.local +++ b/bindings/Makefile.local @@ -1,16 +1,28 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := bindings # force the shared library to be built -ruby-bindings: lib/$(LINKER_NAME) +ruby-bindings: $(dir)/ruby.stamp + +$(dir)/ruby.stamp: lib/$(LINKER_NAME) ifeq ($(HAVE_RUBY_DEV),1) cd $(dir)/ruby && \ EXTRA_LDFLAGS="$(NO_UNDEFINED_LDFLAGS)" \ LIBNOTMUCH="../../lib/$(LINKER_NAME)" \ NOTMUCH_SRCDIR='$(NOTMUCH_SRCDIR)' \ $(RUBY) extconf.rb --vendor - $(MAKE) -C $(dir)/ruby + $(MAKE) -C $(dir)/ruby CFLAGS="$(CFLAGS) -pipe -fno-plt -fPIC" && touch $@ +endif + +python-cffi-bindings: $(dir)/python-cffi.stamp + +$(dir)/python-cffi.stamp: lib/$(LINKER_NAME) +ifeq ($(HAVE_PYTHON3_CFFI),1) + cd $(dir)/python-cffi && \ + ${PYTHON} setup.py build --build-lib build/stage && \ + mkdir -p build/stage/tests && cp tests/*.py build/stage/tests && \ + touch ../python-cffi.stamp endif CLEAN += $(patsubst %,$(dir)/ruby/%, \ @@ -19,4 +31,10 @@ CLEAN += $(patsubst %,$(dir)/ruby/%, \ init.o message.o messages.o mkmf.log notmuch.so query.o \ status.o tags.o thread.o threads.o) -CLEAN += bindings/ruby/.vendorarchdir.time +CLEAN += bindings/ruby/.vendorarchdir.time $(dir)/ruby.stamp + +CLEAN += bindings/python-cffi/build $(dir)/python-cffi.stamp +CLEAN += bindings/python-cffi/__pycache__ + +DISTCLEAN += bindings/python-cffi/_notmuch_config.py \ + bindings/python-cffi/notmuch2.egg-info diff --git a/bindings/python-cffi/MANIFEST.in b/bindings/python-cffi/MANIFEST.in new file mode 100644 index 00000000..9ef81f24 --- /dev/null +++ b/bindings/python-cffi/MANIFEST.in @@ -0,0 +1,2 @@ +include MANIFEST.in +include tox.ini diff --git a/bindings/python-cffi/notmuch2/__init__.py b/bindings/python-cffi/notmuch2/__init__.py new file mode 100644 index 00000000..f281edc1 --- /dev/null +++ b/bindings/python-cffi/notmuch2/__init__.py @@ -0,0 +1,62 @@ +"""Pythonic API to the notmuch database. + +Creating Objects +================ + +Only the :class:`Database` object is meant to be created by the user. +All other objects should be created from this initial object. Users +should consider their signatures implementation details. + +Errors +====== + +All errors occurring due to errors from the underlying notmuch database +are subclasses of the :exc:`NotmuchError`. Due to memory management +it is possible to try and use an object after it has been freed. In +this case a :exc:`ObjectDestroyedError` will be raised. + +Memory Management +================= + +Libnotmuch uses a hierarchical memory allocator, this means all +objects have a strict parent-child relationship and when the parent is +freed all the children are freed as well. This has some implications +for these Python bindings as parent objects need to be kept alive. +This is normally schielded entirely from the user however and the +Python objects automatically make sure the right references are kept +alive. It is however the reason the :class:`BaseObject` exists as it +defines the API all Python objects need to implement to work +correctly. + +Collections and Containers +========================== + +Libnotmuch exposes nearly all collections of things as iterators only. +In these python bindings they have sometimes been exposed as +:class:`collections.abc.Container` instances or subclasses of this +like :class:`collections.abc.Set` or :class:`collections.abc.Mapping` +etc. This gives a more natural API to work with, e.g. being able to +treat tags as sets. However it does mean that the +:meth:`__contains__`, :meth:`__len__` and frieds methods on these are +usually more and essentially O(n) rather than O(1) as you might +usually expect from Python containers. +""" + +from notmuch2 import _capi +from notmuch2._base import * +from notmuch2._database import * +from notmuch2._errors import * +from notmuch2._message import * +from notmuch2._tags import * +from notmuch2._thread import * + + +NOTMUCH_TAG_MAX = _capi.lib.NOTMUCH_TAG_MAX +del _capi + + +# Re-home all the objects to the package. This leaves __qualname__ intact. +for x in locals().copy().values(): + if hasattr(x, '__module__'): + x.__module__ = __name__ +del x diff --git a/bindings/python-cffi/notmuch2/_base.py b/bindings/python-cffi/notmuch2/_base.py new file mode 100644 index 00000000..1cf03c88 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_base.py @@ -0,0 +1,238 @@ +import abc +import collections.abc + +from notmuch2 import _capi as capi +from notmuch2 import _errors as errors + + +__all__ = ['NotmuchObject', 'BinString'] + + +class NotmuchObject(metaclass=abc.ABCMeta): + """Base notmuch object syntax. + + This base class exists to define the memory management handling + required to use the notmuch library. It is meant as an interface + definition rather than a base class, though you can use it as a + base class to ensure you don't forget part of the interface. It + only concerns you if you are implementing this package itself + rather then using it. + + libnotmuch uses a hierarchical memory allocator, where freeing the + memory of a parent object also frees the memory of all child + objects. To make this work seamlessly in Python this package + keeps references to parent objects which makes them stay alive + correctly under normal circumstances. When an object finally gets + deleted the :meth:`__del__` method will be called to free the + memory. + + However during some peculiar situations, e.g. interpreter + shutdown, it is possible for the :meth:`__del__` method to have + been called, whele there are still references to an object. This + could result in child objects asking their memory to be freed + after the parent has already freed the memory, making things + rather unhappy as double frees are not taken lightly in C. To + handle this case all objects need to follow the same protocol to + destroy themselves, see :meth:`destroy`. + + Once an object has been destroyed trying to use it should raise + the :exc:`ObjectDestroyedError` exception. For this see also the + convenience :class:`MemoryPointer` descriptor in this module which + can be used as a pointer to libnotmuch memory. + """ + + @abc.abstractmethod + def __init__(self, parent, *args, **kwargs): + """Create a new object. + + Other then for the toplevel :class:`Database` object + constructors are only ever called by internal code and not by + the user. Per convention their signature always takes the + parent object as first argument. Feel free to make the rest + of the signature match the object's requirement. The object + needs to keep a reference to the parent, so it can check the + parent is still alive. + """ + + @property + @abc.abstractmethod + def alive(self): + """Whether the object is still alive. + + This indicates whether the object is still alive. The first + thing this needs to check is whether the parent object is + still alive, if it is not then this object can not be alive + either. If the parent is alive then it depends on whether the + memory for this object has been freed yet or not. + """ + + def __del__(self): + self._destroy() + + @abc.abstractmethod + def _destroy(self): + """Destroy the object, freeing all memory. + + This method needs to destroy the object on the + libnotmuch-level. It must ensure it's not been destroyed by + it's parent object yet before doing so. It also must be + idempotent. + """ + + +class MemoryPointer: + """Data Descriptor to handle accessing libnotmuch pointers. + + Most :class:`NotmuchObject` instances will have one or more CFFI + pointers to C-objects. Once an object is destroyed this pointer + should no longer be used and a :exc:`ObjectDestroyedError` + exception should be raised on trying to access it. This + descriptor simplifies implementing this, allowing the creation of + an attribute which can be assigned to, but when accessed when the + stored value is *None* it will raise the + :exc:`ObjectDestroyedError` exception:: + + class SomeOjb: + _ptr = MemoryPointer() + + def __init__(self, ptr): + self._ptr = ptr + + def destroy(self): + somehow_free(self._ptr) + self._ptr = None + + def do_something(self): + return some_libnotmuch_call(self._ptr) + """ + + def __get__(self, instance, owner): + try: + val = getattr(instance, self.attr_name, None) + except AttributeError: + # We're not on 3.6+ and self.attr_name does not exist + self.__set_name__(instance, 'dummy') + val = getattr(instance, self.attr_name, None) + if val is None: + raise errors.ObjectDestroyedError() + return val + + def __set__(self, instance, value): + try: + setattr(instance, self.attr_name, value) + except AttributeError: + # We're not on 3.6+ and self.attr_name does not exist + self.__set_name__(instance, 'dummy') + setattr(instance, self.attr_name, value) + + def __set_name__(self, instance, name): + self.attr_name = '_memptr_{}_{:x}'.format(name, id(instance)) + + +class BinString(str): + """A str subclass with binary data. + + Most data in libnotmuch should be valid ASCII or valid UTF-8. + However since it is a C library these are represented as + bytestrings instead which means on an API level we can not + guarantee that decoding this to UTF-8 will both succeed and be + lossless. This string type converts bytes to unicode in a lossy + way, but also makes the raw bytes available. + + This object is a normal unicode string for most intents and + purposes, but you can get the original bytestring back by calling + ``bytes()`` on it. + """ + + def __new__(cls, data, encoding='utf-8', errors='ignore'): + if not isinstance(data, bytes): + data = bytes(data, encoding=encoding) + strdata = str(data, encoding=encoding, errors=errors) + inst = super().__new__(cls, strdata) + inst._bindata = data + return inst + + @classmethod + def from_cffi(cls, cdata): + """Create a new string from a CFFI cdata pointer.""" + return cls(capi.ffi.string(cdata)) + + def __bytes__(self): + return self._bindata + + +class NotmuchIter(NotmuchObject, collections.abc.Iterator): + """An iterator for libnotmuch iterators. + + It is tempting to use a generator function instead, but this would + not correctly respect the :class:`NotmuchObject` memory handling + protocol and in some unsuspecting cornercases cause memory + trouble. You probably want to sublcass this in order to wrap the + value returned by :meth:`__next__`. + + :param parent: The parent object. + :type parent: NotmuchObject + :param iter_p: The CFFI pointer to the C iterator. + :type iter_p: cffi.cdata + :param fn_destory: The CFFI notmuch_*_destroy function. + :param fn_valid: The CFFI notmuch_*_valid function. + :param fn_get: The CFFI notmuch_*_get function. + :param fn_next: The CFFI notmuch_*_move_to_next function. + """ + _iter_p = MemoryPointer() + + def __init__(self, parent, iter_p, + *, fn_destroy, fn_valid, fn_get, fn_next): + self._parent = parent + self._iter_p = iter_p + self._fn_destroy = fn_destroy + self._fn_valid = fn_valid + self._fn_get = fn_get + self._fn_next = fn_next + + def __del__(self): + self._destroy() + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._iter_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def _destroy(self): + if self.alive: + try: + self._fn_destroy(self._iter_p) + except errors.ObjectDestroyedError: + pass + self._iter_p = None + + def __iter__(self): + """Return the iterator itself. + + Note that as this is an iterator and not a container this will + not return a new iterator. Thus any elements already consumed + will not be yielded by the :meth:`__next__` method anymore. + """ + return self + + def __next__(self): + if not self._fn_valid(self._iter_p): + self._destroy() + raise StopIteration() + obj_p = self._fn_get(self._iter_p) + self._fn_next(self._iter_p) + return obj_p + + def __repr__(self): + try: + self._iter_p + except errors.ObjectDestroyedError: + return '' + else: + return '' diff --git a/bindings/python-cffi/notmuch2/_build.py b/bindings/python-cffi/notmuch2/_build.py new file mode 100644 index 00000000..65d7dcb6 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_build.py @@ -0,0 +1,346 @@ +import cffi +from _notmuch_config import * + +ffibuilder = cffi.FFI() +ffibuilder.set_source( + 'notmuch2._capi', + r""" + #include + #include + #include + + #if LIBNOTMUCH_MAJOR_VERSION < 5 + #error libnotmuch version not supported by notmuch2 python bindings + #endif + #if LIBNOTMUCH_MINOR_VERSION < 1 + #ERROR libnotmuch version < 5.1 not supported + #endif + """, + include_dirs=[NOTMUCH_INCLUDE_DIR], + library_dirs=[NOTMUCH_LIB_DIR], + libraries=['notmuch'], +) +ffibuilder.cdef( + r""" + void free(void *ptr); + typedef int... time_t; + + #define LIBNOTMUCH_MAJOR_VERSION ... + #define LIBNOTMUCH_MINOR_VERSION ... + #define LIBNOTMUCH_MICRO_VERSION ... + + #define NOTMUCH_TAG_MAX ... + + typedef enum _notmuch_status { + NOTMUCH_STATUS_SUCCESS = 0, + NOTMUCH_STATUS_OUT_OF_MEMORY, + NOTMUCH_STATUS_READ_ONLY_DATABASE, + NOTMUCH_STATUS_XAPIAN_EXCEPTION, + NOTMUCH_STATUS_FILE_ERROR, + NOTMUCH_STATUS_FILE_NOT_EMAIL, + NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID, + NOTMUCH_STATUS_NULL_POINTER, + NOTMUCH_STATUS_TAG_TOO_LONG, + NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW, + NOTMUCH_STATUS_UNBALANCED_ATOMIC, + NOTMUCH_STATUS_UNSUPPORTED_OPERATION, + NOTMUCH_STATUS_UPGRADE_REQUIRED, + NOTMUCH_STATUS_PATH_ERROR, + NOTMUCH_STATUS_ILLEGAL_ARGUMENT, + NOTMUCH_STATUS_MALFORMED_CRYPTO_PROTOCOL, + NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION, + NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL, + NOTMUCH_STATUS_NO_CONFIG, + NOTMUCH_STATUS_NO_DATABASE, + NOTMUCH_STATUS_DATABASE_EXISTS, + NOTMUCH_STATUS_BAD_QUERY_SYNTAX, + NOTMUCH_STATUS_NO_MAIL_ROOT, + NOTMUCH_STATUS_CLOSED_DATABASE, + NOTMUCH_STATUS_LAST_STATUS + } notmuch_status_t; + typedef enum { + NOTMUCH_DATABASE_MODE_READ_ONLY = 0, + NOTMUCH_DATABASE_MODE_READ_WRITE + } notmuch_database_mode_t; + typedef int notmuch_bool_t; + typedef enum _notmuch_message_flag { + NOTMUCH_MESSAGE_FLAG_MATCH, + NOTMUCH_MESSAGE_FLAG_EXCLUDED, + NOTMUCH_MESSAGE_FLAG_GHOST, + } notmuch_message_flag_t; + typedef enum { + NOTMUCH_SORT_OLDEST_FIRST, + NOTMUCH_SORT_NEWEST_FIRST, + NOTMUCH_SORT_MESSAGE_ID, + NOTMUCH_SORT_UNSORTED + } notmuch_sort_t; + typedef enum { + NOTMUCH_EXCLUDE_FLAG, + NOTMUCH_EXCLUDE_TRUE, + NOTMUCH_EXCLUDE_FALSE, + NOTMUCH_EXCLUDE_ALL + } notmuch_exclude_t; + typedef enum { + NOTMUCH_DECRYPT_FALSE, + NOTMUCH_DECRYPT_TRUE, + NOTMUCH_DECRYPT_AUTO, + NOTMUCH_DECRYPT_NOSTASH, + } notmuch_decryption_policy_t; + + // These are fully opaque types for us, we only ever use pointers. + typedef struct _notmuch_database notmuch_database_t; + typedef struct _notmuch_query notmuch_query_t; + typedef struct _notmuch_threads notmuch_threads_t; + typedef struct _notmuch_thread notmuch_thread_t; + typedef struct _notmuch_messages notmuch_messages_t; + typedef struct _notmuch_message notmuch_message_t; + typedef struct _notmuch_tags notmuch_tags_t; + typedef struct _notmuch_string_map_iterator notmuch_message_properties_t; + typedef struct _notmuch_directory notmuch_directory_t; + typedef struct _notmuch_filenames notmuch_filenames_t; + typedef struct _notmuch_config_pairs notmuch_config_pairs_t; + typedef struct _notmuch_indexopts notmuch_indexopts_t; + + const char * + notmuch_status_to_string (notmuch_status_t status); + + notmuch_status_t + notmuch_database_create_with_config (const char *database_path, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **error_message); + notmuch_status_t + notmuch_database_open_with_config (const char *database_path, + notmuch_database_mode_t mode, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **error_message); + notmuch_status_t + notmuch_database_close (notmuch_database_t *database); + notmuch_status_t + notmuch_database_destroy (notmuch_database_t *database); + const char * + notmuch_database_get_path (notmuch_database_t *database); + unsigned int + notmuch_database_get_version (notmuch_database_t *database); + notmuch_bool_t + notmuch_database_needs_upgrade (notmuch_database_t *database); + notmuch_status_t + notmuch_database_begin_atomic (notmuch_database_t *notmuch); + notmuch_status_t + notmuch_database_end_atomic (notmuch_database_t *notmuch); + unsigned long + notmuch_database_get_revision (notmuch_database_t *notmuch, + const char **uuid); + notmuch_status_t + notmuch_database_index_file (notmuch_database_t *database, + const char *filename, + notmuch_indexopts_t *indexopts, + notmuch_message_t **message); + notmuch_status_t + notmuch_database_remove_message (notmuch_database_t *database, + const char *filename); + notmuch_status_t + notmuch_database_find_message (notmuch_database_t *database, + const char *message_id, + notmuch_message_t **message); + notmuch_status_t + notmuch_database_find_message_by_filename (notmuch_database_t *notmuch, + const char *filename, + notmuch_message_t **message); + notmuch_tags_t * + notmuch_database_get_all_tags (notmuch_database_t *db); + + notmuch_query_t * + notmuch_query_create (notmuch_database_t *database, + const char *query_string); + const char * + notmuch_query_get_query_string (const notmuch_query_t *query); + notmuch_database_t * + notmuch_query_get_database (const notmuch_query_t *query); + void + notmuch_query_set_omit_excluded (notmuch_query_t *query, + notmuch_exclude_t omit_excluded); + void + notmuch_query_set_sort (notmuch_query_t *query, notmuch_sort_t sort); + notmuch_sort_t + notmuch_query_get_sort (const notmuch_query_t *query); + notmuch_status_t + notmuch_query_add_tag_exclude (notmuch_query_t *query, const char *tag); + notmuch_status_t + notmuch_query_search_threads (notmuch_query_t *query, + notmuch_threads_t **out); + notmuch_status_t + notmuch_query_search_messages (notmuch_query_t *query, + notmuch_messages_t **out); + notmuch_status_t + notmuch_query_count_messages (notmuch_query_t *query, unsigned int *count); + notmuch_status_t + notmuch_query_count_threads (notmuch_query_t *query, unsigned *count); + void + notmuch_query_destroy (notmuch_query_t *query); + + notmuch_bool_t + notmuch_threads_valid (notmuch_threads_t *threads); + notmuch_thread_t * + notmuch_threads_get (notmuch_threads_t *threads); + void + notmuch_threads_move_to_next (notmuch_threads_t *threads); + void + notmuch_threads_destroy (notmuch_threads_t *threads); + + const char * + notmuch_thread_get_thread_id (notmuch_thread_t *thread); + notmuch_messages_t * + notmuch_message_get_replies (notmuch_message_t *message); + int + notmuch_thread_get_total_messages (notmuch_thread_t *thread); + notmuch_messages_t * + notmuch_thread_get_toplevel_messages (notmuch_thread_t *thread); + notmuch_messages_t * + notmuch_thread_get_messages (notmuch_thread_t *thread); + int + notmuch_thread_get_matched_messages (notmuch_thread_t *thread); + const char * + notmuch_thread_get_authors (notmuch_thread_t *thread); + const char * + notmuch_thread_get_subject (notmuch_thread_t *thread); + time_t + notmuch_thread_get_oldest_date (notmuch_thread_t *thread); + time_t + notmuch_thread_get_newest_date (notmuch_thread_t *thread); + notmuch_tags_t * + notmuch_thread_get_tags (notmuch_thread_t *thread); + void + notmuch_thread_destroy (notmuch_thread_t *thread); + + notmuch_bool_t + notmuch_messages_valid (notmuch_messages_t *messages); + notmuch_message_t * + notmuch_messages_get (notmuch_messages_t *messages); + void + notmuch_messages_move_to_next (notmuch_messages_t *messages); + void + notmuch_messages_destroy (notmuch_messages_t *messages); + notmuch_tags_t * + notmuch_messages_collect_tags (notmuch_messages_t *messages); + + const char * + notmuch_message_get_message_id (notmuch_message_t *message); + const char * + notmuch_message_get_thread_id (notmuch_message_t *message); + const char * + notmuch_message_get_filename (notmuch_message_t *message); + notmuch_filenames_t * + notmuch_message_get_filenames (notmuch_message_t *message); + notmuch_bool_t + notmuch_message_get_flag (notmuch_message_t *message, + notmuch_message_flag_t flag); + void + notmuch_message_set_flag (notmuch_message_t *message, + notmuch_message_flag_t flag, + notmuch_bool_t value); + time_t + notmuch_message_get_date (notmuch_message_t *message); + const char * + notmuch_message_get_header (notmuch_message_t *message, + const char *header); + notmuch_tags_t * + notmuch_message_get_tags (notmuch_message_t *message); + notmuch_status_t + notmuch_message_add_tag (notmuch_message_t *message, const char *tag); + notmuch_status_t + notmuch_message_remove_tag (notmuch_message_t *message, const char *tag); + notmuch_status_t + notmuch_message_remove_all_tags (notmuch_message_t *message); + notmuch_status_t + notmuch_message_maildir_flags_to_tags (notmuch_message_t *message); + notmuch_status_t + notmuch_message_tags_to_maildir_flags (notmuch_message_t *message); + notmuch_status_t + notmuch_message_freeze (notmuch_message_t *message); + notmuch_status_t + notmuch_message_thaw (notmuch_message_t *message); + notmuch_status_t + notmuch_message_get_property (notmuch_message_t *message, + const char *key, const char **value); + notmuch_status_t + notmuch_message_add_property (notmuch_message_t *message, + const char *key, const char *value); + notmuch_status_t + notmuch_message_remove_property (notmuch_message_t *message, + const char *key, const char *value); + notmuch_status_t + notmuch_message_remove_all_properties (notmuch_message_t *message, + const char *key); + notmuch_message_properties_t * + notmuch_message_get_properties (notmuch_message_t *message, + const char *key, notmuch_bool_t exact); + notmuch_bool_t + notmuch_message_properties_valid (notmuch_message_properties_t + *properties); + void + notmuch_message_properties_move_to_next (notmuch_message_properties_t + *properties); + const char * + notmuch_message_properties_key (notmuch_message_properties_t *properties); + const char * + notmuch_message_properties_value (notmuch_message_properties_t + *properties); + void + notmuch_message_properties_destroy (notmuch_message_properties_t + *properties); + void + notmuch_message_destroy (notmuch_message_t *message); + + notmuch_bool_t + notmuch_tags_valid (notmuch_tags_t *tags); + const char * + notmuch_tags_get (notmuch_tags_t *tags); + void + notmuch_tags_move_to_next (notmuch_tags_t *tags); + void + notmuch_tags_destroy (notmuch_tags_t *tags); + + notmuch_bool_t + notmuch_filenames_valid (notmuch_filenames_t *filenames); + const char * + notmuch_filenames_get (notmuch_filenames_t *filenames); + void + notmuch_filenames_move_to_next (notmuch_filenames_t *filenames); + void + notmuch_filenames_destroy (notmuch_filenames_t *filenames); + notmuch_indexopts_t * + notmuch_database_get_default_indexopts (notmuch_database_t *db); + notmuch_status_t + notmuch_indexopts_set_decrypt_policy (notmuch_indexopts_t *indexopts, + notmuch_decryption_policy_t decrypt_policy); + notmuch_decryption_policy_t + notmuch_indexopts_get_decrypt_policy (const notmuch_indexopts_t *indexopts); + void + notmuch_indexopts_destroy (notmuch_indexopts_t *options); + + notmuch_status_t + notmuch_database_set_config (notmuch_database_t *db, const char *key, const char *value); + notmuch_status_t + notmuch_database_get_config (notmuch_database_t *db, const char *key, char **value); + notmuch_config_pairs_t * + notmuch_config_get_pairs (notmuch_database_t *db, const char *prefix); + notmuch_bool_t + notmuch_config_pairs_valid (notmuch_config_pairs_t *config_list); + const char * + notmuch_config_pairs_key (notmuch_config_pairs_t *config_list); + const char * + notmuch_config_pairs_value (notmuch_config_pairs_t *config_list); + void + notmuch_config_pairs_move_to_next (notmuch_config_pairs_t *config_list); + void + notmuch_config_pairs_destroy (notmuch_config_pairs_t *config_list); + """ +) + + +if __name__ == '__main__': + ffibuilder.compile(verbose=True) diff --git a/bindings/python-cffi/notmuch2/_config.py b/bindings/python-cffi/notmuch2/_config.py new file mode 100644 index 00000000..603fdcbf --- /dev/null +++ b/bindings/python-cffi/notmuch2/_config.py @@ -0,0 +1,101 @@ +import collections.abc + +import notmuch2._base as base +import notmuch2._capi as capi +import notmuch2._errors as errors + + +__all__ = ['ConfigMapping'] + + +class ConfigIter(base.NotmuchIter): + + def __init__(self, parent, iter_p): + super().__init__( + parent, iter_p, + fn_destroy=capi.lib.notmuch_config_pairs_destroy, + fn_valid=capi.lib.notmuch_config_pairs_valid, + fn_get=capi.lib.notmuch_config_pairs_key, + fn_next=capi.lib.notmuch_config_pairs_move_to_next) + + def __next__(self): + # skip pairs whose value is NULL + while capi.lib.notmuch_config_pairs_valid(super()._iter_p): + val_p = capi.lib.notmuch_config_pairs_value(super()._iter_p) + key_p = capi.lib.notmuch_config_pairs_key(super()._iter_p) + if key_p == capi.ffi.NULL: + # this should never happen + raise errors.NullPointerError + key = base.BinString.from_cffi(key_p) + capi.lib.notmuch_config_pairs_move_to_next(super()._iter_p) + if val_p != capi.ffi.NULL and base.BinString.from_cffi(val_p) != "": + return key + self._destroy() + raise StopIteration + +class ConfigMapping(base.NotmuchObject, collections.abc.MutableMapping): + """The config key/value pairs loaded from the database, config file, + and and/or defaults. + + The entries are exposed as a :class:`collections.abc.MutableMapping` object. + Note that setting a value to an empty string is the same as deleting it. + + Mutating (deleting or updating values) in the map persists only in + the database, which can be shadowed by config files. + + :param parent: the parent object + :param ptr_name: the name of the attribute on the parent which will + return the memory pointer. This allows this object to + access the pointer via the parent's descriptor and thus + trigger :class:`MemoryPointer`'s memory safety. + + """ + + def __init__(self, parent, ptr_name): + self._parent = parent + self._ptr = lambda: getattr(parent, ptr_name) + + @property + def alive(self): + return self._parent.alive + + def _destroy(self): + pass + + def __getitem__(self, key): + if isinstance(key, str): + key = key.encode('utf-8') + val_pp = capi.ffi.new('char**') + ret = capi.lib.notmuch_database_get_config(self._ptr(), key, val_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + val = base.BinString.from_cffi(val_pp[0]) + capi.lib.free(val_pp[0]) + if val == '': + raise KeyError + return val + + def __setitem__(self, key, val): + if isinstance(key, str): + key = key.encode('utf-8') + if isinstance(val, str): + val = val.encode('utf-8') + ret = capi.lib.notmuch_database_set_config(self._ptr(), key, val) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def __delitem__(self, key): + self[key] = "" + + def __iter__(self): + """Return an iterator over the config items. + + :raises NullPointerError: If the iterator can not be created. + """ + config_pairs_p = capi.lib.notmuch_config_get_pairs(self._ptr(), b'') + if config_pairs_p == capi.ffi.NULL: + raise KeyError + return ConfigIter(self._parent, config_pairs_p) + + def __len__(self): + return sum(1 for t in self) diff --git a/bindings/python-cffi/notmuch2/_database.py b/bindings/python-cffi/notmuch2/_database.py new file mode 100644 index 00000000..d7485b4d --- /dev/null +++ b/bindings/python-cffi/notmuch2/_database.py @@ -0,0 +1,856 @@ +import collections +import configparser +import enum +import functools +import os +import pathlib +import weakref + +import notmuch2._base as base +import notmuch2._config as config +import notmuch2._capi as capi +import notmuch2._errors as errors +import notmuch2._message as message +import notmuch2._query as querymod +import notmuch2._tags as tags + + +__all__ = ['Database', 'AtomicContext', 'DbRevision'] + + +def _config_pathname(): + """Return the path of the configuration file. + + :rtype: pathlib.Path + """ + cfgfname = os.getenv('NOTMUCH_CONFIG', '~/.notmuch-config') + return pathlib.Path(os.path.expanduser(cfgfname)) + + +class Mode(enum.Enum): + READ_ONLY = capi.lib.NOTMUCH_DATABASE_MODE_READ_ONLY + READ_WRITE = capi.lib.NOTMUCH_DATABASE_MODE_READ_WRITE + +class ConfigFile(enum.Enum): + EMPTY = b'' + SEARCH = capi.ffi.NULL + +class QuerySortOrder(enum.Enum): + OLDEST_FIRST = capi.lib.NOTMUCH_SORT_OLDEST_FIRST + NEWEST_FIRST = capi.lib.NOTMUCH_SORT_NEWEST_FIRST + MESSAGE_ID = capi.lib.NOTMUCH_SORT_MESSAGE_ID + UNSORTED = capi.lib.NOTMUCH_SORT_UNSORTED + + +class QueryExclude(enum.Enum): + TRUE = capi.lib.NOTMUCH_EXCLUDE_TRUE + FLAG = capi.lib.NOTMUCH_EXCLUDE_FLAG + FALSE = capi.lib.NOTMUCH_EXCLUDE_FALSE + ALL = capi.lib.NOTMUCH_EXCLUDE_ALL + + +class DecryptionPolicy(enum.Enum): + FALSE = capi.lib.NOTMUCH_DECRYPT_FALSE + TRUE = capi.lib.NOTMUCH_DECRYPT_TRUE + AUTO = capi.lib.NOTMUCH_DECRYPT_AUTO + NOSTASH = capi.lib.NOTMUCH_DECRYPT_NOSTASH + + +class Database(base.NotmuchObject): + """Toplevel access to notmuch. + + A :class:`Database` can be opened read-only or read-write. + Modifications are not atomic by default, use :meth:`begin_atomic` + for atomic updates. If the underlying database has been modified + outside of this class a :exc:`XapianError` will be raised and the + instance must be closed and a new one created. + + You can use an instance of this class as a context-manager. + + :cvar MODE: The mode a database can be opened with, an enumeration + of ``READ_ONLY`` and ``READ_WRITE`` + :cvar SORT: The sort order for search results, ``OLDEST_FIRST``, + ``NEWEST_FIRST``, ``MESSAGE_ID`` or ``UNSORTED``. + :cvar EXCLUDE: Which messages to exclude from queries, ``TRUE``, + ``FLAG``, ``FALSE`` or ``ALL``. See the query documentation + for details. + :cvar CONFIG: Control loading of config file. Enumeration of + ``EMPTY`` (don't load a config file), and ``SEARCH`` (search as + in :ref:`config_search`) + :cvar AddedMessage: A namedtuple ``(msg, dup)`` used by + :meth:`add` as return value. + :cvar STR_MODE_MAP: A map mapping strings to :attr:`MODE` items. + This is used to implement the ``ro`` and ``rw`` string + variants. + + :ivar closed: Boolean indicating if the database is closed or + still open. + + :param path: The directory of where the database is stored. If + ``None`` the location will be searched according to + :ref:`database` + :type path: str, bytes, os.PathLike or pathlib.Path + :param mode: The mode to open the database in. One of + :attr:`MODE.READ_ONLY` OR :attr:`MODE.READ_WRITE`. For + convenience you can also use the strings ``ro`` for + :attr:`MODE.READ_ONLY` and ``rw`` for :attr:`MODE.READ_WRITE`. + :type mode: :attr:`MODE` or str. + + :param config: Where to load the configuration from, if any. + :type config: :attr:`CONFIG.EMPTY`, :attr:`CONFIG.SEARCH`, str, bytes, os.PathLike, pathlib.Path + :raises KeyError: if an unknown mode string is used. + :raises OSError: or subclasses if the configuration file can not + be opened. + :raises configparser.Error: or subclasses if the configuration + file can not be parsed. + :raises NotmuchError: or subclasses for other failures. + """ + + MODE = Mode + SORT = QuerySortOrder + EXCLUDE = QueryExclude + CONFIG = ConfigFile + AddedMessage = collections.namedtuple('AddedMessage', ['msg', 'dup']) + _db_p = base.MemoryPointer() + STR_MODE_MAP = { + 'ro': MODE.READ_ONLY, + 'rw': MODE.READ_WRITE, + } + + @staticmethod + def _cfg_path_encode(path): + if isinstance(path,ConfigFile): + path = path.value + elif path is None: + path = capi.ffi.NULL + elif not hasattr(os, 'PathLike') and isinstance(path, pathlib.Path): + path = bytes(path) + else: + path = os.fsencode(path) + return path + + @staticmethod + def _db_path_encode(path): + if path is None: + path = capi.ffi.NULL + elif not hasattr(os, 'PathLike') and isinstance(path, pathlib.Path): + path = bytes(path) + else: + path = os.fsencode(path) + return path + + def __init__(self, path=None, mode=MODE.READ_ONLY, config=CONFIG.SEARCH): + if isinstance(mode, str): + mode = self.STR_MODE_MAP[mode] + self.mode = mode + + db_pp = capi.ffi.new('notmuch_database_t **') + cmsg = capi.ffi.new('char**') + ret = capi.lib.notmuch_database_open_with_config(self._db_path_encode(path), + mode.value, + self._cfg_path_encode(config), + capi.ffi.NULL, + db_pp, cmsg) + if cmsg[0]: + msg = capi.ffi.string(cmsg[0]).decode(errors='replace') + capi.lib.free(cmsg[0]) + else: + msg = None + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret, msg) + self._db_p = db_pp[0] + self.closed = False + + @classmethod + def create(cls, path=None, config=ConfigFile.EMPTY): + """Create and open database in READ_WRITE mode. + + This is creates a new notmuch database and returns an opened + instance in :attr:`MODE.READ_WRITE` mode. + + :param path: The directory of where the database is stored. + If ``None`` the location will be read searched by the + notmuch library (see notmuch(3)::notmuch_open_with_config). + :type path: str, bytes or os.PathLike + + :param config: The pathname of the notmuch configuration file. + :type config: :attr:`CONFIG.EMPTY`, :attr:`CONFIG.SEARCH`, str, bytes, os.PathLike, pathlib.Path + + :raises OSError: or subclasses if the configuration file can not + be opened. + :raises configparser.Error: or subclasses if the configuration + file can not be parsed. + :raises NotmuchError: if the config file does not have the + database.path setting. + :raises FileError: if the database already exists. + + :returns: The newly created instance. + """ + + db_pp = capi.ffi.new('notmuch_database_t **') + cmsg = capi.ffi.new('char**') + ret = capi.lib.notmuch_database_create_with_config(cls._db_path_encode(path), + cls._cfg_path_encode(config), + capi.ffi.NULL, + db_pp, cmsg) + if cmsg[0]: + msg = capi.ffi.string(cmsg[0]).decode(errors='replace') + capi.lib.free(cmsg[0]) + else: + msg = None + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret, msg) + + # Now close the db and let __init__ open it. Inefficient but + # creating is not a hot loop while this allows us to have a + # clean API. + ret = capi.lib.notmuch_database_destroy(db_pp[0]) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + return cls(path, cls.MODE.READ_WRITE, config=config) + + @staticmethod + def default_path(cfg_path=None): + """Return the path of the user's default database. + + This reads the user's configuration file and returns the + default path of the database. + + :param cfg_path: The pathname of the notmuch configuration file. + If not specified tries to use the pathname provided in the + :envvar:`NOTMUCH_CONFIG` environment variable and falls back + to :file:`~/.notmuch-config`. + :type cfg_path: str, bytes, os.PathLike or pathlib.Path. + + :returns: The path of the database, which does not necessarily + exists. + :rtype: pathlib.Path + :raises OSError: or subclasses if the configuration file can not + be opened. + :raises configparser.Error: or subclasses if the configuration + file can not be parsed. + :raises NotmuchError: if the config file does not have the + database.path setting. + + .. deprecated:: 0.35 + Use the ``config`` parameter to :meth:`__init__` or :meth:`__create__` instead. + """ + if not cfg_path: + cfg_path = _config_pathname() + if not hasattr(os, 'PathLike') and isinstance(cfg_path, pathlib.Path): + cfg_path = bytes(cfg_path) + parser = configparser.ConfigParser() + with open(cfg_path) as fp: + parser.read_file(fp) + try: + return pathlib.Path(parser.get('database', 'path')) + except configparser.Error: + raise errors.NotmuchError( + 'No database.path setting in {}'.format(cfg_path)) + + def __del__(self): + self._destroy() + + @property + def alive(self): + try: + self._db_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def _destroy(self): + try: + ret = capi.lib.notmuch_database_destroy(self._db_p) + except errors.ObjectDestroyedError: + ret = capi.lib.NOTMUCH_STATUS_SUCCESS + else: + self._db_p = None + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def close(self): + """Close the notmuch database. + + Once closed most operations will fail. This can still be + useful however to explicitly close a database which is opened + read-write as this would otherwise stop other processes from + reading the database while it is open. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_database_close(self._db_p) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + self.closed = True + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc_value, traceback): + self.close() + + @property + def path(self): + """The pathname of the notmuch database. + + This is returned as a :class:`pathlib.Path` instance. + + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + return self._cache_path + except AttributeError: + ret = capi.lib.notmuch_database_get_path(self._db_p) + self._cache_path = pathlib.Path(os.fsdecode(capi.ffi.string(ret))) + return self._cache_path + + @property + def version(self): + """The database format version. + + This is a positive integer. + + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + return self._cache_version + except AttributeError: + ret = capi.lib.notmuch_database_get_version(self._db_p) + self._cache_version = ret + return ret + + @property + def needs_upgrade(self): + """Whether the database should be upgraded. + + If *True* the database can be upgraded using :meth:`upgrade`. + Not doing so may result in some operations raising + :exc:`UpgradeRequiredError`. + + A read-only database will never be upgradable. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_database_needs_upgrade(self._db_p) + return bool(ret) + + def upgrade(self, progress_cb=None): + """Upgrade the database to the latest version. + + Upgrade the database, optionally with a progress callback + which should be a callable which will be called with a + floating point number in the range of [0.0 .. 1.0]. + """ + raise NotImplementedError + + def atomic(self): + """Return a context manager to perform atomic operations. + + The returned context manager can be used to perform atomic + operations on the database. + + .. note:: Unlinke a traditional RDBMS transaction this does + not imply durability, it only ensures the changes are + performed atomically. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ctx = AtomicContext(self, '_db_p') + return ctx + + def revision(self): + """The currently committed revision in the database. + + Returned as a ``(revision, uuid)`` namedtuple. + + :raises ObjectDestroyedError: if used after destroyed. + """ + raw_uuid = capi.ffi.new('char**') + rev = capi.lib.notmuch_database_get_revision(self._db_p, raw_uuid) + return DbRevision(rev, capi.ffi.string(raw_uuid[0])) + + def get_directory(self, path): + raise NotImplementedError + + def default_indexopts(self): + """Returns default index options for the database. + + :raises ObjectDestroyedError: if used after destroyed. + + :returns: :class:`IndexOptions`. + """ + opts = capi.lib.notmuch_database_get_default_indexopts(self._db_p) + return IndexOptions(self, opts) + + def add(self, filename, *, sync_flags=False, indexopts=None): + """Add a message to the database. + + Add a new message to the notmuch database. The message is + referred to by the pathname of the maildir file. If the + message ID of the new message already exists in the database, + this adds ``pathname`` to the list of list of files for the + existing message. + + :param filename: The path of the file containing the message. + :type filename: str, bytes, os.PathLike or pathlib.Path. + :param sync_flags: Whether to sync the known maildir flags to + notmuch tags. See :meth:`Message.flags_to_tags` for + details. + :type sync_flags: bool + :param indexopts: The indexing options, see + :meth:`default_indexopts`. Leave as `None` to use the + default options configured in the database. + :type indexopts: :class:`IndexOptions` or `None` + + :returns: A tuple where the first item is the newly inserted + messages as a :class:`Message` instance, and the second + item is a boolean indicating if the message inserted was a + duplicate. This is the namedtuple ``AddedMessage(msg, + dup)``. + :rtype: Database.AddedMessage + + If an exception is raised, no message was added. + + :raises XapianError: A Xapian exception occurred. + :raises FileError: The file referred to by ``pathname`` could + not be opened. + :raises FileNotEmailError: The file referreed to by + ``pathname`` is not recognised as an email message. + :raises ReadOnlyDatabaseError: The database is opened in + READ_ONLY mode. + :raises UpgradeRequiredError: The database must be upgraded + first. + :raises ObjectDestroyedError: if used after destroyed. + """ + if not hasattr(os, 'PathLike') and isinstance(filename, pathlib.Path): + filename = bytes(filename) + msg_pp = capi.ffi.new('notmuch_message_t **') + opts_p = indexopts._opts_p if indexopts else capi.ffi.NULL + ret = capi.lib.notmuch_database_index_file( + self._db_p, os.fsencode(filename), opts_p, msg_pp) + ok = [capi.lib.NOTMUCH_STATUS_SUCCESS, + capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID] + if ret not in ok: + raise errors.NotmuchError(ret) + msg = message.Message(self, msg_pp[0], db=self) + if sync_flags: + msg.tags.from_maildir_flags() + return self.AddedMessage( + msg, ret == capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) + + def remove(self, filename): + """Remove a message from the notmuch database. + + Removing a message which is not in the database is just a + silent nop-operation. + + :param filename: The pathname of the file containing the + message to be removed. + :type filename: str, bytes, os.PathLike or pathlib.Path. + + :returns: True if the message is still in the database. This + can happen when multiple files contain the same message ID. + The true/false distinction is fairly arbitrary, but think + of it as ``dup = db.remove_message(name); if dup: ...``. + :rtype: bool + + :raises XapianError: A Xapian exception occurred. + :raises ReadOnlyDatabaseError: The database is opened in + READ_ONLY mode. + :raises UpgradeRequiredError: The database must be upgraded + first. + :raises ObjectDestroyedError: if used after destroyed. + """ + if not hasattr(os, 'PathLike') and isinstance(filename, pathlib.Path): + filename = bytes(filename) + ret = capi.lib.notmuch_database_remove_message(self._db_p, + os.fsencode(filename)) + ok = [capi.lib.NOTMUCH_STATUS_SUCCESS, + capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID] + if ret not in ok: + raise errors.NotmuchError(ret) + if ret == capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: + return True + else: + return False + + def find(self, msgid): + """Return the message matching the given message ID. + + If a message with the given message ID is found a + :class:`Message` instance is returned. Otherwise a + :exc:`LookupError` is raised. + + :param msgid: The message ID to look for. + :type msgid: str + + :returns: The message instance. + :rtype: Message + + :raises LookupError: If no message was found. + :raises OutOfMemoryError: When there is no memory to allocate + the message instance. + :raises XapianError: A Xapian exception occurred. + :raises ObjectDestroyedError: if used after destroyed. + """ + msg_pp = capi.ffi.new('notmuch_message_t **') + ret = capi.lib.notmuch_database_find_message(self._db_p, + msgid.encode(), msg_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + msg_p = msg_pp[0] + if msg_p == capi.ffi.NULL: + raise LookupError + msg = message.Message(self, msg_p, db=self) + return msg + + def get(self, filename): + """Return the :class:`Message` given a pathname. + + If a message with the given pathname exists in the database + return the :class:`Message` instance for the message. + Otherwise raise a :exc:`LookupError` exception. + + :param filename: The pathname of the message. + :type filename: str, bytes, os.PathLike or pathlib.Path + + :returns: The message instance. + :rtype: Message + + :raises LookupError: If no message was found. This is also + a subclass of :exc:`KeyError`. + :raises OutOfMemoryError: When there is no memory to allocate + the message instance. + :raises XapianError: A Xapian exception occurred. + :raises ObjectDestroyedError: if used after destroyed. + """ + if not hasattr(os, 'PathLike') and isinstance(filename, pathlib.Path): + filename = bytes(filename) + msg_pp = capi.ffi.new('notmuch_message_t **') + ret = capi.lib.notmuch_database_find_message_by_filename( + self._db_p, os.fsencode(filename), msg_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + msg_p = msg_pp[0] + if msg_p == capi.ffi.NULL: + raise LookupError + msg = message.Message(self, msg_p, db=self) + return msg + + @property + def tags(self): + """Return an immutable set with all tags used in this database. + + This returns an immutable set-like object implementing the + collections.abc.Set Abstract Base Class. Due to the + underlying libnotmuch implementation some operations have + different performance characteristics then plain set objects. + Mainly any lookup operation is O(n) rather then O(1). + + Normal usage treats tags as UTF-8 encoded unicode strings so + they are exposed to Python as normal unicode string objects. + If you need to handle tags stored in libnotmuch which are not + valid unicode do check the :class:`ImmutableTagSet` docs for + how to handle this. + + :rtype: ImmutableTagSet + + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + ref = self._cached_tagset + except AttributeError: + tagset = None + else: + tagset = ref() + if tagset is None: + tagset = tags.ImmutableTagSet( + self, '_db_p', capi.lib.notmuch_database_get_all_tags) + self._cached_tagset = weakref.ref(tagset) + return tagset + + @property + def config(self): + """Return a mutable mapping with the settings stored in this database. + + This returns an mutable dict-like object implementing the + collections.abc.MutableMapping Abstract Base Class. + + :rtype: Config + + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + ref = self._cached_config + except AttributeError: + config_mapping = None + else: + config_mapping = ref() + if config_mapping is None: + config_mapping = config.ConfigMapping(self, '_db_p') + self._cached_config = weakref.ref(config_mapping) + return config_mapping + + def _create_query(self, query, *, + omit_excluded=EXCLUDE.TRUE, + sort=SORT.UNSORTED, # Check this default + exclude_tags=None): + """Create an internal query object. + + :raises OutOfMemoryError: if no memory is available to + allocate the query. + """ + if isinstance(query, str): + query = query.encode('utf-8') + query_p = capi.lib.notmuch_query_create(self._db_p, query) + if query_p == capi.ffi.NULL: + raise errors.OutOfMemoryError() + capi.lib.notmuch_query_set_omit_excluded(query_p, omit_excluded.value) + capi.lib.notmuch_query_set_sort(query_p, sort.value) + if exclude_tags is not None: + for tag in exclude_tags: + if isinstance(tag, str): + tag = tag.encode('utf-8') + capi.lib.notmuch_query_add_tag_exclude(query_p, tag) + return querymod.Query(self, query_p) + + def messages(self, query, *, + omit_excluded=EXCLUDE.TRUE, + sort=SORT.UNSORTED, # Check this default + exclude_tags=None): + """Search the database for messages. + + :returns: An iterator over the messages found. + :rtype: MessageIter + + :raises OutOfMemoryError: if no memory is available to + allocate the query. + :raises ObjectDestroyedError: if used after destroyed. + """ + query = self._create_query(query, + omit_excluded=omit_excluded, + sort=sort, + exclude_tags=exclude_tags) + return query.messages() + + def count_messages(self, query, *, + omit_excluded=EXCLUDE.TRUE, + sort=SORT.UNSORTED, # Check this default + exclude_tags=None): + """Search the database for messages. + + :returns: An iterator over the messages found. + :rtype: MessageIter + + :raises ObjectDestroyedError: if used after destroyed. + """ + query = self._create_query(query, + omit_excluded=omit_excluded, + sort=sort, + exclude_tags=exclude_tags) + return query.count_messages() + + def threads(self, query, *, + omit_excluded=EXCLUDE.TRUE, + sort=SORT.UNSORTED, # Check this default + exclude_tags=None): + query = self._create_query(query, + omit_excluded=omit_excluded, + sort=sort, + exclude_tags=exclude_tags) + return query.threads() + + def count_threads(self, query, *, + omit_excluded=EXCLUDE.TRUE, + sort=SORT.UNSORTED, # Check this default + exclude_tags=None): + query = self._create_query(query, + omit_excluded=omit_excluded, + sort=sort, + exclude_tags=exclude_tags) + return query.count_threads() + + def status_string(self): + raise NotImplementedError + + def __repr__(self): + return 'Database(path={self.path}, mode={self.mode})'.format(self=self) + + +class AtomicContext: + """Context manager for atomic support. + + This supports the notmuch_database_begin_atomic and + notmuch_database_end_atomic API calls. The object can not be + directly instantiated by the user, only via ``Database.atomic``. + It does keep a reference to the :class:`Database` instance to keep + the C memory alive. + + :raises XapianError: When this is raised at enter time the atomic + section is not active. When it is raised at exit time the + atomic section is still active and you may need to try using + :meth:`force_end`. + :raises ObjectDestroyedError: if used after destroyed. + """ + + def __init__(self, db, ptr_name): + self._db = db + self._ptr = lambda: getattr(db, ptr_name) + self._exit_fn = lambda: None + + def __del__(self): + self._destroy() + + @property + def alive(self): + return self.parent.alive + + def _destroy(self): + pass + + def __enter__(self): + ret = capi.lib.notmuch_database_begin_atomic(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + self._exit_fn = self._end_atomic + return self + + def _end_atomic(self): + ret = capi.lib.notmuch_database_end_atomic(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def __exit__(self, exc_type, exc_value, traceback): + self._exit_fn() + + def force_end(self): + """Force ending the atomic section. + + This can only be called once __exit__ has been called. It + will attempt to close the atomic section (again). This is + useful if the original exit raised an exception and the atomic + section is still open. But things are pretty ugly by now. + + :raises XapianError: If exiting fails, the atomic section is + not ended. + :raises UnbalancedAtomicError: If the database was currently + not in an atomic section. + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_database_end_atomic(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def abort(self): + """Abort the transaction. + + Aborting a transaction will not commit any of the changes, but + will also implicitly close the database. + """ + self._exit_fn = lambda: None + self._db.close() + + +@functools.total_ordering +class DbRevision: + """A database revision. + + The database revision number increases monotonically with each + commit to the database. Which means user-visible changes can be + ordered. This object is sortable with other revisions. It + carries the UUID of the database to ensure it is only ever + compared with revisions from the same database. + """ + + def __init__(self, rev, uuid): + self._rev = rev + self._uuid = uuid + + @property + def rev(self): + """The revision number, a positive integer.""" + return self._rev + + @property + def uuid(self): + """The UUID of the database, consider this opaque.""" + return self._uuid + + def __eq__(self, other): + if isinstance(other, self.__class__): + if self.uuid != other.uuid: + return False + return self.rev == other.rev + else: + return NotImplemented + + def __lt__(self, other): + if self.__class__ is other.__class__: + if self.uuid != other.uuid: + return False + return self.rev < other.rev + else: + return NotImplemented + + def __repr__(self): + return 'DbRevision(rev={self.rev}, uuid={self.uuid})'.format(self=self) + + +class IndexOptions(base.NotmuchObject): + """Indexing options. + + This represents the indexing options which can be used to index a + message. See :meth:`Database.default_indexopts` to create an + instance of this. It can be used e.g. when indexing a new message + using :meth:`Database.add`. + """ + _opts_p = base.MemoryPointer() + + def __init__(self, parent, opts_p): + self._parent = parent + self._opts_p = opts_p + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._opts_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def _destroy(self): + if self.alive: + capi.lib.notmuch_indexopts_destroy(self._opts_p) + self._opts_p = None + + @property + def decrypt_policy(self): + """The decryption policy. + + This is an enum from the :class:`DecryptionPolicy`. See the + `index.decrypt` section in :man:`notmuch-config` for details + on the options. **Do not set this to + :attr:`DecryptionPolicy.TRUE`** without considering the + security of your index. + + You can change this policy by assigning a new + :class:`DecryptionPolicy` to this property. + + :raises ObjectDestroyedError: if used after destroyed. + + :returns: A :class:`DecryptionPolicy` enum instance. + """ + raw = capi.lib.notmuch_indexopts_get_decrypt_policy(self._opts_p) + return DecryptionPolicy(raw) + + @decrypt_policy.setter + def decrypt_policy(self, val): + ret = capi.lib.notmuch_indexopts_set_decrypt_policy( + self._opts_p, val.value) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret, msg) diff --git a/bindings/python-cffi/notmuch2/_errors.py b/bindings/python-cffi/notmuch2/_errors.py new file mode 100644 index 00000000..17c3ad9c --- /dev/null +++ b/bindings/python-cffi/notmuch2/_errors.py @@ -0,0 +1,124 @@ +from notmuch2 import _capi as capi + + +class NotmuchError(Exception): + """Base exception for errors originating from the notmuch library. + + Usually this will have two attributes: + + :status: This is a numeric status code corresponding to the error + code in the notmuch library. This is normally fairly + meaningless, it can also often be ``None``. This exists mostly + to easily create new errors from notmuch status codes and + should not normally be used by users. + + :message: A user-facing message for the error. This can + occasionally also be ``None``. Usually you'll want to call + ``str()`` on the error object instead to get a sensible + message. + """ + + @classmethod + def exc_type(cls, status): + """Return correct exception type for notmuch status.""" + types = { + capi.lib.NOTMUCH_STATUS_OUT_OF_MEMORY: + OutOfMemoryError, + capi.lib.NOTMUCH_STATUS_READ_ONLY_DATABASE: + ReadOnlyDatabaseError, + capi.lib.NOTMUCH_STATUS_XAPIAN_EXCEPTION: + XapianError, + capi.lib.NOTMUCH_STATUS_FILE_ERROR: + FileError, + capi.lib.NOTMUCH_STATUS_FILE_NOT_EMAIL: + FileNotEmailError, + capi.lib.NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: + DuplicateMessageIdError, + capi.lib.NOTMUCH_STATUS_NULL_POINTER: + NullPointerError, + capi.lib.NOTMUCH_STATUS_TAG_TOO_LONG: + TagTooLongError, + capi.lib.NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW: + UnbalancedFreezeThawError, + capi.lib.NOTMUCH_STATUS_UNBALANCED_ATOMIC: + UnbalancedAtomicError, + capi.lib.NOTMUCH_STATUS_UNSUPPORTED_OPERATION: + UnsupportedOperationError, + capi.lib.NOTMUCH_STATUS_UPGRADE_REQUIRED: + UpgradeRequiredError, + capi.lib.NOTMUCH_STATUS_PATH_ERROR: + PathError, + capi.lib.NOTMUCH_STATUS_ILLEGAL_ARGUMENT: + IllegalArgumentError, + capi.lib.NOTMUCH_STATUS_NO_CONFIG: + NoConfigError, + capi.lib.NOTMUCH_STATUS_NO_DATABASE: + NoDatabaseError, + capi.lib.NOTMUCH_STATUS_DATABASE_EXISTS: + DatabaseExistsError, + capi.lib.NOTMUCH_STATUS_BAD_QUERY_SYNTAX: + QuerySyntaxError, + } + return types[status] + + def __new__(cls, *args, **kwargs): + """Return the correct subclass based on status.""" + # This is simplistic, but the actual __init__ will fail if the + # signature is wrong anyway. + if args: + status = args[0] + else: + status = kwargs.get('status', None) + if status and cls == NotmuchError: + exc = cls.exc_type(status) + return exc.__new__(exc, *args, **kwargs) + else: + return super().__new__(cls) + + def __init__(self, status=None, message=None): + self.status = status + self.message = message + + def __str__(self): + if self.message: + return self.message + elif self.status: + char_str = capi.lib.notmuch_status_to_string(self.status) + return capi.ffi.string(char_str).decode(errors='replace') + else: + return 'Unknown error' + + +class OutOfMemoryError(NotmuchError): pass +class ReadOnlyDatabaseError(NotmuchError): pass +class XapianError(NotmuchError): pass +class FileError(NotmuchError): pass +class FileNotEmailError(NotmuchError): pass +class DuplicateMessageIdError(NotmuchError): pass +class NullPointerError(NotmuchError): pass +class TagTooLongError(NotmuchError): pass +class UnbalancedFreezeThawError(NotmuchError): pass +class UnbalancedAtomicError(NotmuchError): pass +class UnsupportedOperationError(NotmuchError): pass +class UpgradeRequiredError(NotmuchError): pass +class PathError(NotmuchError): pass +class IllegalArgumentError(NotmuchError): pass +class NoConfigError(NotmuchError): pass +class NoDatabaseError(NotmuchError): pass +class DatabaseExistsError(NotmuchError): pass +class QuerySyntaxError(NotmuchError): pass + +class ObjectDestroyedError(NotmuchError): + """The object has already been destroyed and it's memory freed. + + This occurs when :meth:`destroy` has been called on the object but + you still happen to have access to the object. This should not + normally occur since you should never call :meth:`destroy` by + hand. + """ + + def __str__(self): + if self.message: + return self.message + else: + return 'Memory already freed' diff --git a/bindings/python-cffi/notmuch2/_message.py b/bindings/python-cffi/notmuch2/_message.py new file mode 100644 index 00000000..d4b34e91 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_message.py @@ -0,0 +1,724 @@ +import collections +import contextlib +import os +import pathlib +import weakref + +import notmuch2._base as base +import notmuch2._capi as capi +import notmuch2._errors as errors +import notmuch2._tags as tags + + +__all__ = ['Message'] + + +class Message(base.NotmuchObject): + """An email message stored in the notmuch database retrieved via a query. + + This should not be directly created, instead it will be returned + by calling methods on :class:`Database`. A message keeps a + reference to the database object since the database object can not + be released while the message is in use. + + Note that this represents a message in the notmuch database. For + full email functionality you may want to use the :mod:`email` + package from Python's standard library. You could e.g. create + this as such:: + + notmuch_msg = db.find(msgid) # or from a query + parser = email.parser.BytesParser(policy=email.policy.default) + with notmuch_msg.path.open('rb) as fp: + email_msg = parser.parse(fp) + + Most commonly the functionality provided by notmuch is sufficient + to read email however. + + Messages are considered equal when they have the same message ID. + This is how libnotmuch treats messages as well, the + :meth:`pathnames` function returns multiple results for + duplicates. + + :param parent: The parent object. This is probably one off a + :class:`Database`, :class:`Thread` or :class:`Query`. + :type parent: NotmuchObject + :param db: The database instance this message is associated with. + This could be the same as the parent. + :type db: Database + :param msg_p: The C pointer to the ``notmuch_message_t``. + :type msg_p: + :param dup: Whether the message was a duplicate on insertion. + :type dup: None or bool + """ + _msg_p = base.MemoryPointer() + + def __init__(self, parent, msg_p, *, db): + self._parent = parent + self._msg_p = msg_p + self._db = db + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._msg_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_message_destroy(self._msg_p) + self._msg_p = None + + @property + def messageid(self): + """The message ID as a string. + + The message ID is decoded with the ignore error handler. This + is fine as long as the message ID is well formed. If it is + not valid ASCII then this will be lossy. So if you need to be + able to write the exact same message ID back you should use + :attr:`messageidb`. + + Note that notmuch will decode the message ID value and thus + strip off the surrounding ``<`` and ``>`` characters. This is + different from Python's :mod:`email` package behaviour which + leaves these characters in place. + + :returns: The message ID. + :rtype: :class:`BinString`, this is a normal str but calling + bytes() on it will return the original bytes used to create + it. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_message_id(self._msg_p) + return base.BinString(capi.ffi.string(ret)) + + @property + def threadid(self): + """The thread ID. + + The thread ID is decoded with the surrogateescape error + handler so that it is possible to reconstruct the original + thread ID if it is not valid UTF-8. + + :returns: The thread ID. + :rtype: :class:`BinString`, this is a normal str but calling + bytes() on it will return the original bytes used to create + it. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_thread_id(self._msg_p) + return base.BinString(capi.ffi.string(ret)) + + @property + def path(self): + """A pathname of the message as a pathlib.Path instance. + + If multiple files in the database contain the same message ID + this will be just one of the files, chosen at random. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_filename(self._msg_p) + return pathlib.Path(os.fsdecode(capi.ffi.string(ret))) + + @property + def pathb(self): + """A pathname of the message as a bytes object. + + See :attr:`path` for details, this is the same but does return + the path as a bytes object which is faster but less convenient. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_filename(self._msg_p) + return capi.ffi.string(ret) + + def filenames(self): + """Return an iterator of all files for this message. + + If multiple files contained the same message ID they will all + be returned here. The files are returned as instances of + :class:`pathlib.Path`. + + :returns: Iterator yielding :class:`pathlib.Path` instances. + :rtype: iter + + :raises ObjectDestroyedError: if used after destroyed. + """ + fnames_p = capi.lib.notmuch_message_get_filenames(self._msg_p) + return PathIter(self, fnames_p) + + def filenamesb(self): + """Return an iterator of all files for this message. + + This is like :meth:`pathnames` but the files are returned as + byte objects instead. + + :returns: Iterator yielding :class:`bytes` instances. + :rtype: iter + + :raises ObjectDestroyedError: if used after destroyed. + """ + fnames_p = capi.lib.notmuch_message_get_filenames(self._msg_p) + return FilenamesIter(self, fnames_p) + + @property + def ghost(self): + """Indicates whether this message is a ghost message. + + A ghost message if a message which we know exists, but it has + no files or content associated with it. This can happen if + it was referenced by some other message. Only the + :attr:`messageid` and :attr:`threadid` attributes are valid + for it. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_flag( + self._msg_p, capi.lib.NOTMUCH_MESSAGE_FLAG_GHOST) + return bool(ret) + + @property + def excluded(self): + """Indicates whether this message was excluded from the query. + + When a message is created from a search, sometimes messages + that where excluded by the search query could still be + returned by it, e.g. because they are part of a thread + matching the query. the :meth:`Database.query` method allows + these messages to be flagged, which results in this property + being set to *True*. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_flag( + self._msg_p, capi.lib.NOTMUCH_MESSAGE_FLAG_EXCLUDED) + return bool(ret) + + @property + def matched(self): + """Indicates whether this message was matched by the query. + + When a thread is created from a search, some of the + messages may not match the original query. This property + is set to *True* for those that do match. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_get_flag( + self._msg_p, capi.lib.NOTMUCH_MESSAGE_FLAG_MATCH) + return bool(ret) + + @property + def date(self): + """The message date as an integer. + + The time the message was sent as an integer number of seconds + since the *epoch*, 1 Jan 1970. This is derived from the + message's header, you can get the original header value with + :meth:`header`. + + :raises ObjectDestroyedError: if used after destroyed. + """ + return capi.lib.notmuch_message_get_date(self._msg_p) + + def header(self, name): + """Return the value of the named header. + + Returns the header from notmuch, some common headers are + stored in the database, others are read from the file. + Headers are returned with their newlines stripped and + collapsed concatenated together if they occur multiple times. + You may be better off using the standard library email + package's ``email.message_from_file(msg.path.open())`` if that + is not sufficient for you. + + :param header: Case-insensitive header name to retrieve. + :type header: str or bytes + + :returns: The header value, an empty string if the header is + not present. + :rtype: str + + :raises LookupError: if the header is not present. + :raises NullPointerError: For unexpected notmuch errors. + :raises ObjectDestroyedError: if used after destroyed. + """ + # The returned is supposedly guaranteed to be UTF-8. Header + # names must be ASCII as per RFC x822. + if isinstance(name, str): + name = name.encode('ascii') + ret = capi.lib.notmuch_message_get_header(self._msg_p, name) + if ret == capi.ffi.NULL: + raise errors.NullPointerError() + hdr = capi.ffi.string(ret) + if not hdr: + raise LookupError + return hdr.decode(encoding='utf-8') + + @property + def tags(self): + """The tags associated with the message. + + This behaves as a set. But removing and adding items to the + set removes and adds them to the message in the database. + + :raises ReadOnlyDatabaseError: When manipulating tags on a + database opened in read-only mode. + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + ref = self._cached_tagset + except AttributeError: + tagset = None + else: + tagset = ref() + if tagset is None: + tagset = tags.MutableTagSet( + self, '_msg_p', capi.lib.notmuch_message_get_tags) + self._cached_tagset = weakref.ref(tagset) + return tagset + + @contextlib.contextmanager + def frozen(self): + """Context manager to freeze the message state. + + This allows you to perform atomic tag updates:: + + with msg.frozen(): + msg.tags.clear() + msg.tags.add('foo') + + Using This would ensure the message never ends up with no tags + applied at all. + + It is safe to nest calls to this context manager. + + :raises ReadOnlyDatabaseError: if the database is opened in + read-only mode. + :raises UnbalancedFreezeThawError: if you somehow managed to + call __exit__ of this context manager more than once. Why + did you do that? + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_message_freeze(self._msg_p) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + self._frozen = True + try: + yield + except Exception: + # Only way to "rollback" these changes is to destroy + # ourselves and re-create. Behold. + msgid = self.messageid + self._destroy() + with contextlib.suppress(Exception): + new = self._db.find(msgid) + self._msg_p = new._msg_p + new._msg_p = None + del new + raise + else: + ret = capi.lib.notmuch_message_thaw(self._msg_p) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + self._frozen = False + + @property + def properties(self): + """A map of arbitrary key-value pairs associated with the message. + + Be aware that properties may be used by other extensions to + store state in. So delete or modify with care. + + The properties map is somewhat special. It is essentially a + multimap-like structure where each key can have multiple + values. Therefore accessing a single item using + :meth:`PropertiesMap.get` or :meth:`PropertiesMap.__getitem__` + will only return you the *first* item if there are multiple + and thus are only recommended if you know there to be only one + value. + + Instead the map has an additional :meth:`PropertiesMap.all` + method which can be used to retrieve all properties of a given + key. This method also allows iterating of a a subset of the + keys starting with a given prefix. + """ + try: + ref = self._cached_props + except AttributeError: + props = None + else: + props = ref() + if props is None: + props = PropertiesMap(self, '_msg_p') + self._cached_props = weakref.ref(props) + return props + + def replies(self): + """Return an iterator of all replies to this message. + + This method will only work if the message was created from a + thread. Otherwise it will yield no results. + + :returns: An iterator yielding :class:`OwnedMessage` instances. + :rtype: MessageIter + """ + # The notmuch_messages_valid call accepts NULL and this will + # become an empty iterator, raising StopIteration immediately. + # Hence no return value checking here. + msgs_p = capi.lib.notmuch_message_get_replies(self._msg_p) + return MessageIter(self, msgs_p, db=self._db, msg_cls=OwnedMessage) + + def __hash__(self): + return hash(self.messageid) + + def __eq__(self, other): + if isinstance(other, self.__class__): + return self.messageid == other.messageid + + +class OwnedMessage(Message): + """An email message owned by parent thread object. + + This subclass of Message is used for messages that are retrieved + from the notmuch database via a parent :class:`notmuch2.Thread` + object, which "owns" this message. This means that when this + message object is destroyed, by calling :func:`del` or + :meth:`_destroy` directly or indirectly, the message is not freed + in the notmuch API and the parent :class:`notmuch2.Thread` object + can return the same object again when needed. + """ + + @property + def alive(self): + return self._parent.alive + + def _destroy(self): + pass + + +class FilenamesIter(base.NotmuchIter): + """Iterator for binary filenames objects.""" + + def __init__(self, parent, iter_p): + super().__init__(parent, iter_p, + fn_destroy=capi.lib.notmuch_filenames_destroy, + fn_valid=capi.lib.notmuch_filenames_valid, + fn_get=capi.lib.notmuch_filenames_get, + fn_next=capi.lib.notmuch_filenames_move_to_next) + + def __next__(self): + fname = super().__next__() + return capi.ffi.string(fname) + + +class PathIter(FilenamesIter): + """Iterator for pathlib.Path objects.""" + + def __next__(self): + fname = super().__next__() + return pathlib.Path(os.fsdecode(fname)) + + +class PropertiesMap(base.NotmuchObject, collections.abc.MutableMapping): + """A mutable mapping to manage properties. + + Both keys and values of properties are supposed to be UTF-8 + strings in libnotmuch. However since the uderlying API uses + bytestrings you can use either str or bytes to represent keys and + all returned keys and values use :class:`BinString`. + + Also be aware that ``iter(this_map)`` will return duplicate keys, + while the :class:`collections.abc.KeysView` returned by + :meth:`keys` is a :class:`collections.abc.Set` subclass. This + means the former will yield duplicate keys while the latter won't. + It also means ``len(list(iter(this_map)))`` could be different + than ``len(this_map.keys())``. ``len(this_map)`` will correspond + with the length of the default iterator. + + Be aware that libnotmuch exposes all of this as iterators, so + quite a few operations have O(n) performance instead of the usual + O(1). + """ + Property = collections.namedtuple('Property', ['key', 'value']) + _marker = object() + + def __init__(self, msg, ptr_name): + self._msg = msg + self._ptr = lambda: getattr(msg, ptr_name) + + @property + def alive(self): + if not self._msg.alive: + return False + try: + self._ptr + except errors.ObjectDestroyedError: + return False + else: + return True + + def _destroy(self): + pass + + def __iter__(self): + """Return an iterator which iterates over the keys. + + Be aware that a single key may have multiple values associated + with it, if so it will appear multiple times here. + """ + iter_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) + return PropertiesKeyIter(self, iter_p) + + def __len__(self): + iter_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) + it = base.NotmuchIter( + self, iter_p, + fn_destroy=capi.lib.notmuch_message_properties_destroy, + fn_valid=capi.lib.notmuch_message_properties_valid, + fn_get=capi.lib.notmuch_message_properties_key, + fn_next=capi.lib.notmuch_message_properties_move_to_next, + ) + return len(list(it)) + + def __getitem__(self, key): + """Return **the first** peroperty associated with a key.""" + if isinstance(key, str): + key = key.encode('utf-8') + value_pp = capi.ffi.new('char**') + ret = capi.lib.notmuch_message_get_property(self._ptr(), key, value_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + if value_pp[0] == capi.ffi.NULL: + raise KeyError + return base.BinString.from_cffi(value_pp[0]) + + def keys(self): + """Return a :class:`collections.abc.KeysView` for this map. + + Even when keys occur multiple times this is a subset of set() + so will only contain them once. + """ + return collections.abc.KeysView({k: None for k in self}) + + def items(self): + """Return a :class:`collections.abc.ItemsView` for this map. + + The ItemsView treats a ``(key, value)`` pair as unique, so + dupcliate ``(key, value)`` pairs will be merged together. + However duplicate keys with different values will be returned. + """ + items = set() + props_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) + while capi.lib.notmuch_message_properties_valid(props_p): + key = capi.lib.notmuch_message_properties_key(props_p) + value = capi.lib.notmuch_message_properties_value(props_p) + items.add((base.BinString.from_cffi(key), + base.BinString.from_cffi(value))) + capi.lib.notmuch_message_properties_move_to_next(props_p) + capi.lib.notmuch_message_properties_destroy(props_p) + return PropertiesItemsView(items) + + def values(self): + """Return a :class:`collecions.abc.ValuesView` for this map. + + All unique property values are included in the view. + """ + values = set() + props_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) + while capi.lib.notmuch_message_properties_valid(props_p): + value = capi.lib.notmuch_message_properties_value(props_p) + values.add(base.BinString.from_cffi(value)) + capi.lib.notmuch_message_properties_move_to_next(props_p) + capi.lib.notmuch_message_properties_destroy(props_p) + return PropertiesValuesView(values) + + def __setitem__(self, key, value): + """Add a key-value pair to the properties. + + You may prefer to use :meth:`add` for clarity since this + method usually implies implicit overwriting of an existing key + if it exists, while for properties this is not the case. + """ + self.add(key, value) + + def add(self, key, value): + """Add a key-value pair to the properties.""" + if isinstance(key, str): + key = key.encode('utf-8') + if isinstance(value, str): + value = value.encode('utf-8') + ret = capi.lib.notmuch_message_add_property(self._ptr(), key, value) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def __delitem__(self, key): + """Remove all properties with this key.""" + if isinstance(key, str): + key = key.encode('utf-8') + ret = capi.lib.notmuch_message_remove_all_properties(self._ptr(), key) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def remove(self, key, value): + """Remove a key-value pair from the properties.""" + if isinstance(key, str): + key = key.encode('utf-8') + if isinstance(value, str): + value = value.encode('utf-8') + ret = capi.lib.notmuch_message_remove_property(self._ptr(), key, value) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def pop(self, key, default=_marker): + try: + value = self[key] + except KeyError: + if default is self._marker: + raise + else: + return default + else: + self.remove(key, value) + return value + + def popitem(self): + try: + key = next(iter(self)) + except StopIteration: + raise KeyError + value = self.pop(key) + return (key, value) + + def clear(self): + ret = capi.lib.notmuch_message_remove_all_properties(self._ptr(), + capi.ffi.NULL) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def getall(self, prefix='', *, exact=False): + """Return an iterator yielding all properties for a given key prefix. + + The returned iterator yields all peroperties which start with + a given key prefix as ``(key, value)`` namedtuples. If called + with ``exact=True`` then only properties which exactly match + the prefix are returned, those a key longer than the prefix + will not be included. + + :param prefix: The prefix of the key. + """ + if isinstance(prefix, str): + prefix = prefix.encode('utf-8') + props_p = capi.lib.notmuch_message_get_properties(self._ptr(), + prefix, exact) + return PropertiesIter(self, props_p) + + +class PropertiesKeyIter(base.NotmuchIter): + + def __init__(self, parent, iter_p): + super().__init__( + parent, + iter_p, + fn_destroy=capi.lib.notmuch_message_properties_destroy, + fn_valid=capi.lib.notmuch_message_properties_valid, + fn_get=capi.lib.notmuch_message_properties_key, + fn_next=capi.lib.notmuch_message_properties_move_to_next) + + def __next__(self): + item = super().__next__() + return base.BinString.from_cffi(item) + + +class PropertiesIter(base.NotmuchIter): + + def __init__(self, parent, iter_p): + super().__init__( + parent, + iter_p, + fn_destroy=capi.lib.notmuch_message_properties_destroy, + fn_valid=capi.lib.notmuch_message_properties_valid, + fn_get=capi.lib.notmuch_message_properties_key, + fn_next=capi.lib.notmuch_message_properties_move_to_next, + ) + + def __next__(self): + if not self._fn_valid(self._iter_p): + self._destroy() + raise StopIteration + key = capi.lib.notmuch_message_properties_key(self._iter_p) + value = capi.lib.notmuch_message_properties_value(self._iter_p) + capi.lib.notmuch_message_properties_move_to_next(self._iter_p) + return PropertiesMap.Property(base.BinString.from_cffi(key), + base.BinString.from_cffi(value)) + + +class PropertiesItemsView(collections.abc.Set): + + __slots__ = ('_items',) + + def __init__(self, items): + self._items = items + + @classmethod + def _from_iterable(self, it): + return set(it) + + def __len__(self): + return len(self._items) + + def __contains__(self, item): + return item in self._items + + def __iter__(self): + yield from self._items + + +collections.abc.ItemsView.register(PropertiesItemsView) + + +class PropertiesValuesView(collections.abc.Set): + + __slots__ = ('_values',) + + def __init__(self, values): + self._values = values + + def __len__(self): + return len(self._values) + + def __contains__(self, value): + return value in self._values + + def __iter__(self): + yield from self._values + + +collections.abc.ValuesView.register(PropertiesValuesView) + + +class MessageIter(base.NotmuchIter): + + def __init__(self, parent, msgs_p, *, db, msg_cls=Message): + self._db = db + self._msg_cls = msg_cls + super().__init__(parent, msgs_p, + fn_destroy=capi.lib.notmuch_messages_destroy, + fn_valid=capi.lib.notmuch_messages_valid, + fn_get=capi.lib.notmuch_messages_get, + fn_next=capi.lib.notmuch_messages_move_to_next) + + def __next__(self): + msg_p = super().__next__() + return self._msg_cls(self, msg_p, db=self._db) diff --git a/bindings/python-cffi/notmuch2/_query.py b/bindings/python-cffi/notmuch2/_query.py new file mode 100644 index 00000000..1db6ec96 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_query.py @@ -0,0 +1,83 @@ +from notmuch2 import _base as base +from notmuch2 import _capi as capi +from notmuch2 import _errors as errors +from notmuch2 import _message as message +from notmuch2 import _thread as thread + + +__all__ = [] + + +class Query(base.NotmuchObject): + """Private, minimal query object. + + This is not meant for users and is not a full implementation of + the query API. It is only an intermediate used internally to + match libnotmuch's memory management. + """ + _query_p = base.MemoryPointer() + + def __init__(self, db, query_p): + self._db = db + self._query_p = query_p + + @property + def alive(self): + if not self._db.alive: + return False + try: + self._query_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_query_destroy(self._query_p) + self._query_p = None + + @property + def query(self): + """The query string as seen by libnotmuch.""" + q = capi.lib.notmuch_query_get_query_string(self._query_p) + return base.BinString.from_cffi(q) + + def messages(self): + """Return an iterator over all the messages found by the query. + + This executes the query and returns an iterator over the + :class:`Message` objects found. + """ + msgs_pp = capi.ffi.new('notmuch_messages_t**') + ret = capi.lib.notmuch_query_search_messages(self._query_p, msgs_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + return message.MessageIter(self, msgs_pp[0], db=self._db) + + def count_messages(self): + """Return the number of messages matching this query.""" + count_p = capi.ffi.new('unsigned int *') + ret = capi.lib.notmuch_query_count_messages(self._query_p, count_p) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + return count_p[0] + + def threads(self): + """Return an iterator over all the threads found by the query.""" + threads_pp = capi.ffi.new('notmuch_threads_t **') + ret = capi.lib.notmuch_query_search_threads(self._query_p, threads_pp) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + return thread.ThreadIter(self, threads_pp[0], db=self._db) + + def count_threads(self): + """Return the number of threads matching this query.""" + count_p = capi.ffi.new('unsigned int *') + ret = capi.lib.notmuch_query_count_threads(self._query_p, count_p) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + return count_p[0] diff --git a/bindings/python-cffi/notmuch2/_tags.py b/bindings/python-cffi/notmuch2/_tags.py new file mode 100644 index 00000000..ee5d2a34 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_tags.py @@ -0,0 +1,359 @@ +import collections.abc + +import notmuch2._base as base +import notmuch2._capi as capi +import notmuch2._errors as errors + + +__all__ = ['ImmutableTagSet', 'MutableTagSet', 'TagsIter'] + + +class ImmutableTagSet(base.NotmuchObject, collections.abc.Set): + """The tags associated with a message thread or whole database. + + Both a thread as well as the database expose the union of all tags + in messages associated with them. This exposes these as a + :class:`collections.abc.Set` object. + + Note that due to the underlying notmuch API the performance of the + implementation is not the same as you would expect from normal + sets. E.g. the :meth:`__contains__` and :meth:`__len__` are O(n) + rather then O(1). + + Tags are internally stored as bytestrings but normally exposed as + unicode strings using the UTF-8 encoding and the *ignore* decoder + error handler. However the :meth:`iter` method can be used to + return tags as bytestrings or using a different error handler. + + Note that when doing arithmetic operations on tags, this class + will return a plain normal set as it is no longer associated with + the message. + + :param parent: the parent object + :param ptr_name: the name of the attribute on the parent which will + return the memory pointer. This allows this object to + access the pointer via the parent's descriptor and thus + trigger :class:`MemoryPointer`'s memory safety. + :param cffi_fn: the callable CFFI wrapper to retrieve the tags + iter. This can be one of notmuch_database_get_all_tags, + notmuch_thread_get_tags or notmuch_message_get_tags. + """ + + def __init__(self, parent, ptr_name, cffi_fn): + self._parent = parent + self._ptr = lambda: getattr(parent, ptr_name) + self._cffi_fn = cffi_fn + + def __del__(self): + self._destroy() + + @property + def alive(self): + return self._parent.alive + + def _destroy(self): + pass + + @classmethod + def _from_iterable(cls, it): + return set(it) + + def __iter__(self): + """Return an iterator over the tags. + + Tags are yielded as unicode strings, decoded using the + "ignore" error handler. + + :raises NullPointerError: If the iterator can not be created. + """ + return self.iter(encoding='utf-8', errors='ignore') + + def iter(self, *, encoding=None, errors='strict'): + """Aternate iterator constructor controlling string decoding. + + Tags are stored as bytes in the notmuch database, in Python + it's easier to work with unicode strings and thus is what the + normal iterator returns. However this method allows you to + specify how you would like to get the tags, defaulting to the + bytestring representation instead of unicode strings. + + :param encoding: Which codec to use. The default *None* does not + decode at all and will return the unmodified bytes. + Otherwise this is passed on to :func:`str.decode`. + :param errors: If using a codec, this is the error handler. + See :func:`str.decode` to which this is passed on. + + :raises NullPointerError: When things do not go as planned. + """ + # self._cffi_fn should point either to + # notmuch_database_get_all_tags, notmuch_thread_get_tags or + # notmuch_message_get_tags. nothmuch.h suggests these never + # fail, let's handle NULL anyway. + tags_p = self._cffi_fn(self._ptr()) + if tags_p == capi.ffi.NULL: + raise errors.NullPointerError() + tags = TagsIter(self, tags_p, encoding=encoding, errors=errors) + return tags + + def __len__(self): + return sum(1 for t in self) + + def __contains__(self, tag): + if isinstance(tag, str): + tag = tag.encode() + for msg_tag in self.iter(): + if tag == msg_tag: + return True + else: + return False + + def __eq__(self, other): + return tuple(sorted(self.iter())) == tuple(sorted(other.iter())) + + def issubset(self, other): + return self <= other + + def issuperset(self, other): + return self >= other + + def union(self, other): + return self | other + + def intersection(self, other): + return self & other + + def difference(self, other): + return self - other + + def symmetric_difference(self, other): + return self ^ other + + def copy(self): + return set(self) + + def __hash__(self): + return hash(tuple(self.iter())) + + def __repr__(self): + return '<{name} object at 0x{addr:x} tags={{{tags}}}>'.format( + name=self.__class__.__name__, + addr=id(self), + tags=', '.join(repr(t) for t in self)) + + +class MutableTagSet(ImmutableTagSet, collections.abc.MutableSet): + """The tags associated with a message. + + This is a :class:`collections.abc.MutableSet` object which can be + used to manipulate the tags of a message. + + Note that due to the underlying notmuch API the performance of the + implementation is not the same as you would expect from normal + sets. E.g. the ``in`` operator and variants are O(n) rather then + O(1). + + Tags are bytestrings and calling ``iter()`` will return an + iterator yielding bytestrings. However the :meth:`iter` method + can be used to return tags as unicode strings, while all other + operations accept either byestrings or unicode strings. In case + unicode strings are used they will be encoded using utf-8 before + being passed to notmuch. + """ + + # Since we subclass ImmutableTagSet we inherit a __hash__. But we + # are mutable, setting it to None will make the Python machinery + # recognise us as unhashable. + __hash__ = None + + def add(self, tag): + """Add a tag to the message. + + :param tag: The tag to add. + :type tag: str or bytes. A str will be encoded using UTF-8. + + :param sync_flags: Whether to sync the maildir flags with the + new set of tags. Leaving this as *None* respects the + configuration set in the database, while *True* will always + sync and *False* will never sync. + :param sync_flags: NoneType or bool + + :raises TypeError: If the tag is not a valid type. + :raises TagTooLongError: If the added tag exceeds the maximum + length, see ``notmuch_cffi.NOTMUCH_TAG_MAX``. + :raises ReadOnlyDatabaseError: If the database is opened in + read-only mode. + """ + if isinstance(tag, str): + tag = tag.encode() + if not isinstance(tag, bytes): + raise TypeError('Not a valid type for a tag: {}'.format(type(tag))) + ret = capi.lib.notmuch_message_add_tag(self._ptr(), tag) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def discard(self, tag): + """Remove a tag from the message. + + :param tag: The tag to remove. + :type tag: str of bytes. A str will be encoded using UTF-8. + :param sync_flags: Whether to sync the maildir flags with the + new set of tags. Leaving this as *None* respects the + configuration set in the database, while *True* will always + sync and *False* will never sync. + :param sync_flags: NoneType or bool + + :raises TypeError: If the tag is not a valid type. + :raises TagTooLongError: If the tag exceeds the maximum + length, see ``notmuch_cffi.NOTMUCH_TAG_MAX``. + :raises ReadOnlyDatabaseError: If the database is opened in + read-only mode. + """ + if isinstance(tag, str): + tag = tag.encode() + if not isinstance(tag, bytes): + raise TypeError('Not a valid type for a tag: {}'.format(type(tag))) + ret = capi.lib.notmuch_message_remove_tag(self._ptr(), tag) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def clear(self): + """Remove all tags from the message. + + :raises ReadOnlyDatabaseError: If the database is opened in + read-only mode. + """ + ret = capi.lib.notmuch_message_remove_all_tags(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def from_maildir_flags(self): + """Update the tags based on the state in the message's maildir flags. + + This function examines the filenames of 'message' for maildir + flags, and adds or removes tags on 'message' as follows when + these flags are present: + + Flag Action if present + ---- ----------------- + 'D' Adds the "draft" tag to the message + 'F' Adds the "flagged" tag to the message + 'P' Adds the "passed" tag to the message + 'R' Adds the "replied" tag to the message + 'S' Removes the "unread" tag from the message + + For each flag that is not present, the opposite action + (add/remove) is performed for the corresponding tags. + + Flags are identified as trailing components of the filename + after a sequence of ":2,". + + If there are multiple filenames associated with this message, + the flag is considered present if it appears in one or more + filenames. (That is, the flags from the multiple filenames are + combined with the logical OR operator.) + """ + ret = capi.lib.notmuch_message_maildir_flags_to_tags(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + def to_maildir_flags(self): + """Update the message's maildir flags based on the notmuch tags. + + If the message's filename is in a maildir directory, that is a + directory named ``new`` or ``cur``, and has a valid maildir + filename then the flags will be added as such: + + 'D' if the message has the "draft" tag + 'F' if the message has the "flagged" tag + 'P' if the message has the "passed" tag + 'R' if the message has the "replied" tag + 'S' if the message does not have the "unread" tag + + Any existing flags unmentioned in the list above will be + preserved in the renaming. + + Also, if this filename is in a directory named "new", rename it to + be within the neighboring directory named "cur". + + In case there are multiple files associated with the message + all filenames will get the same logic applied. + """ + ret = capi.lib.notmuch_message_tags_to_maildir_flags(self._ptr()) + if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: + raise errors.NotmuchError(ret) + + +class TagsIter(base.NotmuchObject, collections.abc.Iterator): + """Iterator over tags. + + This is only an iterator, not a container so calling + :meth:`__iter__` does not return a new, replenished iterator but + only itself. + + :param parent: The parent object to keep alive. + :param tags_p: The CFFI pointer to the C-level tags iterator. + :param encoding: Which codec to use. The default *None* does not + decode at all and will return the unmodified bytes. + Otherwise this is passed on to :func:`str.decode`. + :param errors: If using a codec, this is the error handler. + See :func:`str.decode` to which this is passed on. + + :raises ObjectDestroyedError: if used after destroyed. + """ + _tags_p = base.MemoryPointer() + + def __init__(self, parent, tags_p, *, encoding=None, errors='strict'): + self._parent = parent + self._tags_p = tags_p + self._encoding = encoding + self._errors = errors + + def __del__(self): + self._destroy() + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._tags_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def _destroy(self): + if self.alive: + try: + capi.lib.notmuch_tags_destroy(self._tags_p) + except errors.ObjectDestroyedError: + pass + self._tags_p = None + + def __iter__(self): + """Return the iterator itself. + + Note that as this is an iterator and not a container this will + not return a new iterator. Thus any elements already consumed + will not be yielded by the :meth:`__next__` method anymore. + """ + return self + + def __next__(self): + if not capi.lib.notmuch_tags_valid(self._tags_p): + self._destroy() + raise StopIteration() + tag_p = capi.lib.notmuch_tags_get(self._tags_p) + tag = capi.ffi.string(tag_p) + if self._encoding: + tag = tag.decode(encoding=self._encoding, errors=self._errors) + capi.lib.notmuch_tags_move_to_next(self._tags_p) + return tag + + def __repr__(self): + try: + self._tags_p + except errors.ObjectDestroyedError: + return '' + else: + return '' diff --git a/bindings/python-cffi/notmuch2/_thread.py b/bindings/python-cffi/notmuch2/_thread.py new file mode 100644 index 00000000..e883f308 --- /dev/null +++ b/bindings/python-cffi/notmuch2/_thread.py @@ -0,0 +1,194 @@ +import collections.abc +import weakref + +from notmuch2 import _base as base +from notmuch2 import _capi as capi +from notmuch2 import _errors as errors +from notmuch2 import _message as message +from notmuch2 import _tags as tags + + +__all__ = ['Thread'] + + +class Thread(base.NotmuchObject, collections.abc.Iterable): + _thread_p = base.MemoryPointer() + + def __init__(self, parent, thread_p, *, db): + self._parent = parent + self._thread_p = thread_p + self._db = db + + @property + def alive(self): + if not self._parent.alive: + return False + try: + self._thread_p + except errors.ObjectDestroyedError: + return False + else: + return True + + def __del__(self): + self._destroy() + + def _destroy(self): + if self.alive: + capi.lib.notmuch_thread_destroy(self._thread_p) + self._thread_p = None + + @property + def threadid(self): + """The thread ID as a :class:`BinString`. + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_thread_get_thread_id(self._thread_p) + return base.BinString.from_cffi(ret) + + def __len__(self): + """Return the number of messages in the thread. + + :raises ObjectDestroyedError: if used after destroyed. + """ + return capi.lib.notmuch_thread_get_total_messages(self._thread_p) + + def toplevel(self): + """Return an iterator of the toplevel messages. + + :returns: An iterator yielding :class:`Message` instances. + + :raises ObjectDestroyedError: if used after destroyed. + """ + msgs_p = capi.lib.notmuch_thread_get_toplevel_messages(self._thread_p) + return message.MessageIter(self, msgs_p, + db=self._db, + msg_cls=message.OwnedMessage) + + def __iter__(self): + """Return an iterator over all the messages in the thread. + + :returns: An iterator yielding :class:`Message` instances. + + :raises ObjectDestroyedError: if used after destroyed. + """ + msgs_p = capi.lib.notmuch_thread_get_messages(self._thread_p) + return message.MessageIter(self, msgs_p, + db=self._db, + msg_cls=message.OwnedMessage) + + @property + def matched(self): + """The number of messages in this thread which matched the query. + + Of the messages in the thread this gives the count of messages + which did directly match the search query which this thread + originates from. + + :raises ObjectDestroyedError: if used after destroyed. + """ + return capi.lib.notmuch_thread_get_matched_messages(self._thread_p) + + @property + def authors(self): + """A comma-separated string of all authors in the thread. + + Authors of messages which matched the query the thread was + retrieved from will be at the head of the string, ordered by + date of their messages. Following this will be the authors of + the other messages in the thread, also ordered by date of + their messages. Both groups of authors are separated by the + ``|`` character. + + :returns: The stringified list of authors. + :rtype: BinString + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_thread_get_authors(self._thread_p) + return base.BinString.from_cffi(ret) + + @property + def subject(self): + """The subject of the thread, taken from the first message. + + The thread's subject is taken to be the subject of the first + message according to query sort order. + + :returns: The thread's subject. + :rtype: BinString + + :raises ObjectDestroyedError: if used after destroyed. + """ + ret = capi.lib.notmuch_thread_get_subject(self._thread_p) + return base.BinString.from_cffi(ret) + + @property + def first(self): + """Return the date of the oldest message in the thread. + + The time the first message was sent as an integer number of + seconds since the *epoch*, 1 Jan 1970. + + :raises ObjectDestroyedError: if used after destroyed. + """ + return capi.lib.notmuch_thread_get_oldest_date(self._thread_p) + + @property + def last(self): + """Return the date of the newest message in the thread. + + The time the last message was sent as an integer number of + seconds since the *epoch*, 1 Jan 1970. + + :raises ObjectDestroyedError: if used after destroyed. + """ + return capi.lib.notmuch_thread_get_newest_date(self._thread_p) + + @property + def tags(self): + """Return an immutable set with all tags used in this thread. + + This returns an immutable set-like object implementing the + collections.abc.Set Abstract Base Class. Due to the + underlying libnotmuch implementation some operations have + different performance characteristics then plain set objects. + Mainly any lookup operation is O(n) rather then O(1). + + Normal usage treats tags as UTF-8 encoded unicode strings so + they are exposed to Python as normal unicode string objects. + If you need to handle tags stored in libnotmuch which are not + valid unicode do check the :class:`ImmutableTagSet` docs for + how to handle this. + + :rtype: ImmutableTagSet + + :raises ObjectDestroyedError: if used after destroyed. + """ + try: + ref = self._cached_tagset + except AttributeError: + tagset = None + else: + tagset = ref() + if tagset is None: + tagset = tags.ImmutableTagSet( + self, '_thread_p', capi.lib.notmuch_thread_get_tags) + self._cached_tagset = weakref.ref(tagset) + return tagset + + +class ThreadIter(base.NotmuchIter): + + def __init__(self, parent, threads_p, *, db): + self._db = db + super().__init__(parent, threads_p, + fn_destroy=capi.lib.notmuch_threads_destroy, + fn_valid=capi.lib.notmuch_threads_valid, + fn_get=capi.lib.notmuch_threads_get, + fn_next=capi.lib.notmuch_threads_move_to_next) + + def __next__(self): + thread_p = super().__next__() + return Thread(self, thread_p, db=self._db) diff --git a/bindings/python-cffi/setup.py b/bindings/python-cffi/setup.py new file mode 100644 index 00000000..55fb2d24 --- /dev/null +++ b/bindings/python-cffi/setup.py @@ -0,0 +1,25 @@ +import setuptools +from _notmuch_config import * + +with open(NOTMUCH_VERSION_FILE) as fp: + VERSION = fp.read().strip() + +setuptools.setup( + name='notmuch2', + version=VERSION, + description='Pythonic bindings for the notmuch mail database using CFFI', + author='Floris Bruynooghe', + author_email='flub@devork.be', + setup_requires=['cffi>=1.0.0'], + install_requires=['cffi>=1.0.0'], + packages=setuptools.find_packages(exclude=['tests']), + cffi_modules=['notmuch2/_build.py:ffibuilder'], + classifiers=[ + 'Development Status :: 3 - Alpha', + 'Intended Audience :: Developers', + 'License :: OSI Approved :: GNU General Public License (GPL)', + 'Programming Language :: Python :: 3', + 'Topic :: Communications :: Email', + 'Topic :: Software Development :: Libraries', + ], +) diff --git a/bindings/python-cffi/tests/conftest.py b/bindings/python-cffi/tests/conftest.py new file mode 100644 index 00000000..fe90c787 --- /dev/null +++ b/bindings/python-cffi/tests/conftest.py @@ -0,0 +1,149 @@ +import email.message +import mailbox +import pathlib +import shutil +import socket +import subprocess +import textwrap +import time +import os + +import pytest + + +def pytest_report_header(): + which = shutil.which('notmuch') + vers = subprocess.run(['notmuch', '--version'], stdout=subprocess.PIPE) + return ['{} ({})'.format(vers.stdout.decode(errors='replace').strip(),which)] + + +@pytest.fixture(scope='function') +def tmppath(tmpdir): + """The tmpdir fixture wrapped in pathlib.Path.""" + return pathlib.Path(str(tmpdir)) + + +@pytest.fixture +def notmuch(maildir): + """Return a function which runs notmuch commands on our test maildir. + + This uses the notmuch-config file created by the ``maildir`` + fixture. + """ + def run(*args): + """Run a notmuch command. + + This function runs with a timeout error as many notmuch + commands may block if multiple processes are trying to open + the database in write-mode. It is all too easy to + accidentally do this in the unittests. + """ + cfg_fname = maildir.path / 'notmuch-config' + cmd = ['notmuch'] + list(args) + env = os.environ.copy() + env['NOTMUCH_CONFIG'] = str(cfg_fname) + proc = subprocess.run(cmd, + timeout=120, + env=env) + proc.check_returncode() + return run + + +@pytest.fixture +def maildir(tmppath): + """A basic test interface to a valid maildir directory. + + This creates a valid maildir and provides a simple mechanism to + deliver test emails to it. It also writes a notmuch-config file + in the top of the maildir. + """ + cur = tmppath / 'cur' + cur.mkdir() + new = tmppath / 'new' + new.mkdir() + tmp = tmppath / 'tmp' + tmp.mkdir() + cfg_fname = tmppath/'notmuch-config' + with cfg_fname.open('w') as fp: + fp.write(textwrap.dedent("""\ + [database] + path={tmppath!s} + [user] + name=Some Hacker + primary_email=dst@example.com + [new] + tags=unread;inbox; + ignore= + [search] + exclude_tags=deleted;spam; + [maildir] + synchronize_flags=true + """.format(tmppath=tmppath))) + return MailDir(tmppath) + + +class MailDir: + """An interface around a correct maildir.""" + + def __init__(self, path): + self._path = pathlib.Path(path) + self.mailbox = mailbox.Maildir(str(path)) + self._idcount = 0 + + @property + def path(self): + """The pathname of the maildir.""" + return self._path + + def _next_msgid(self): + """Return a new unique message ID.""" + msgid = '{}@{}'.format(self._idcount, socket.getfqdn()) + self._idcount += 1 + return msgid + + def deliver(self, + subject='Test mail', + body='This is a test mail', + to='dst@example.com', + frm='src@example.com', + headers=None, + new=False, # Move to new dir or cur dir? + keywords=None, # List of keywords or labels + seen=False, # Seen flag (cur dir only) + replied=False, # Replied flag (cur dir only) + flagged=False): # Flagged flag (cur dir only) + """Deliver a new mail message in the mbox. + + This does only adds the message to maildir, does not insert it + into the notmuch database. + + :returns: A tuple of (msgid, pathname). + """ + msgid = self._next_msgid() + when = time.time() + msg = email.message.EmailMessage() + msg.add_header('Received', 'by MailDir; {}'.format(time.ctime(when))) + msg.add_header('Message-ID', '<{}>'.format(msgid)) + msg.add_header('Date', time.ctime(when)) + msg.add_header('From', frm) + msg.add_header('To', to) + msg.add_header('Subject', subject) + if headers: + for h, v in headers: + msg.add_header(h, v) + msg.set_content(body) + mdmsg = mailbox.MaildirMessage(msg) + if not new: + mdmsg.set_subdir('cur') + if flagged: + mdmsg.add_flag('F') + if replied: + mdmsg.add_flag('R') + if seen: + mdmsg.add_flag('S') + boxid = self.mailbox.add(mdmsg) + basename = boxid + if mdmsg.get_info(): + basename += mailbox.Maildir.colon + mdmsg.get_info() + msgpath = self.path / mdmsg.get_subdir() / basename + return (msgid, msgpath) diff --git a/bindings/python-cffi/tests/test_base.py b/bindings/python-cffi/tests/test_base.py new file mode 100644 index 00000000..d3280a67 --- /dev/null +++ b/bindings/python-cffi/tests/test_base.py @@ -0,0 +1,116 @@ +import pytest + +from notmuch2 import _base as base +from notmuch2 import _errors as errors + + +class TestNotmuchObject: + + def test_no_impl_methods(self): + class Object(base.NotmuchObject): + pass + with pytest.raises(TypeError): + Object() + + def test_impl_methods(self): + + class Object(base.NotmuchObject): + + def __init__(self): + pass + + @property + def alive(self): + pass + + def _destroy(self, parent=False): + pass + + Object() + + def test_del(self): + destroyed = False + + class Object(base.NotmuchObject): + + def __init__(self): + pass + + @property + def alive(self): + pass + + def _destroy(self, parent=False): + nonlocal destroyed + destroyed = True + + o = Object() + o.__del__() + assert destroyed + + +class TestMemoryPointer: + + @pytest.fixture + def obj(self): + class Cls: + ptr = base.MemoryPointer() + return Cls() + + def test_unset(self, obj): + with pytest.raises(errors.ObjectDestroyedError): + obj.ptr + + def test_set(self, obj): + obj.ptr = 'some' + assert obj.ptr == 'some' + + def test_cleared(self, obj): + obj.ptr = 'some' + obj.ptr + obj.ptr = None + with pytest.raises(errors.ObjectDestroyedError): + obj.ptr + + def test_two_instances(self, obj): + obj2 = obj.__class__() + obj.ptr = 'foo' + obj2.ptr = 'bar' + assert obj.ptr != obj2.ptr + + +class TestBinString: + + def test_type(self): + s = base.BinString(b'foo') + assert isinstance(s, str) + + def test_init_bytes(self): + s = base.BinString(b'foo') + assert s == 'foo' + + def test_init_str(self): + s = base.BinString('foo') + assert s == 'foo' + + def test_bytes(self): + s = base.BinString(b'foo') + assert bytes(s) == b'foo' + + def test_invalid_utf8(self): + s = base.BinString(b'\x80foo') + assert s == 'foo' + assert bytes(s) == b'\x80foo' + + def test_errors(self): + s = base.BinString(b'\x80foo', errors='replace') + assert s == '�foo' + assert bytes(s) == b'\x80foo' + + def test_encoding(self): + # pound sign: '£' == '\u00a3' latin-1: b'\xa3', utf-8: b'\xc2\xa3' + with pytest.raises(UnicodeDecodeError): + base.BinString(b'\xa3', errors='strict') + s = base.BinString(b'\xa3', encoding='latin-1', errors='strict') + assert s == '£' + assert bytes(s) == b'\xa3' diff --git a/bindings/python-cffi/tests/test_config.py b/bindings/python-cffi/tests/test_config.py new file mode 100644 index 00000000..2a7f42f0 --- /dev/null +++ b/bindings/python-cffi/tests/test_config.py @@ -0,0 +1,60 @@ +import collections.abc + +import pytest + +import notmuch2._database as dbmod + +import notmuch2._config as config + + +class TestIter: + + @pytest.fixture + def db(self, maildir): + with dbmod.Database.create(maildir.path) as db: + yield db + + def test_type(self, db): + assert isinstance(db.config, collections.abc.MutableMapping) + assert isinstance(db.config, config.ConfigMapping) + + def test_alive(self, db): + assert db.config.alive + + def test_set_get(self, maildir): + # Ensure get-set works from different db objects + with dbmod.Database.create(maildir.path, config=dbmod.Database.CONFIG.EMPTY) as db0: + db0.config['spam'] = 'ham' + with dbmod.Database(maildir.path, config=dbmod.Database.CONFIG.EMPTY) as db1: + assert db1.config['spam'] == 'ham' + + def test_get_keyerror(self, db): + with pytest.raises(KeyError): + val = db.config['not-a-key'] + print(repr(val)) + + def test_iter(self, db): + def has_prefix(x): + return x.startswith('TEST.') + + assert [ x for x in db.config if has_prefix(x) ] == [] + db.config['TEST.spam'] = 'TEST.ham' + db.config['TEST.eggs'] = 'TEST.bacon' + assert { x for x in db.config if has_prefix(x) } == {'TEST.spam', 'TEST.eggs'} + assert { x for x in db.config.keys() if has_prefix(x) } == {'TEST.spam', 'TEST.eggs'} + assert { x for x in db.config.values() if has_prefix(x) } == {'TEST.ham', 'TEST.bacon'} + assert { (x, y) for (x,y) in db.config.items() if has_prefix(x) } == \ + {('TEST.spam', 'TEST.ham'), ('TEST.eggs', 'TEST.bacon')} + + def test_len(self, db): + defaults = len(db.config) + db.config['spam'] = 'ham' + assert len(db.config) == defaults + 1 + db.config['eggs'] = 'bacon' + assert len(db.config) == defaults + 2 + + def test_del(self, db): + db.config['spam'] = 'ham' + assert db.config.get('spam') == 'ham' + del db.config['spam'] + assert db.config.get('spam') is None diff --git a/bindings/python-cffi/tests/test_database.py b/bindings/python-cffi/tests/test_database.py new file mode 100644 index 00000000..f1d12ea6 --- /dev/null +++ b/bindings/python-cffi/tests/test_database.py @@ -0,0 +1,342 @@ +import collections +import configparser +import os +import pathlib + +import pytest + +import notmuch2 +import notmuch2._errors as errors +import notmuch2._database as dbmod +import notmuch2._message as message + + +@pytest.fixture +def db(maildir): + with dbmod.Database.create(maildir.path, config=notmuch2.Database.CONFIG.EMPTY) as db: + yield db + + +class TestDefaultDb: + """Tests for reading the default database. + + The error cases are fairly undefined, some relevant Python error + will come out if you give it a bad filename or if the file does + not parse correctly. So we're not testing this too deeply. + """ + + def test_config_pathname_default(self, monkeypatch): + monkeypatch.delenv('NOTMUCH_CONFIG', raising=False) + user = pathlib.Path('~/.notmuch-config').expanduser() + assert dbmod._config_pathname() == user + + def test_config_pathname_env(self, monkeypatch): + monkeypatch.setenv('NOTMUCH_CONFIG', '/some/random/path') + assert dbmod._config_pathname() == pathlib.Path('/some/random/path') + + def test_default_path_nocfg(self, monkeypatch, tmppath): + monkeypatch.setenv('NOTMUCH_CONFIG', str(tmppath/'foo')) + with pytest.raises(FileNotFoundError): + dbmod.Database.default_path() + + def test_default_path_cfg_is_dir(self, monkeypatch, tmppath): + monkeypatch.setenv('NOTMUCH_CONFIG', str(tmppath)) + with pytest.raises(IsADirectoryError): + dbmod.Database.default_path() + + def test_default_path_parseerr(self, monkeypatch, tmppath): + cfg = tmppath / 'notmuch-config' + with cfg.open('w') as fp: + fp.write('invalid') + monkeypatch.setenv('NOTMUCH_CONFIG', str(cfg)) + with pytest.raises(configparser.Error): + dbmod.Database.default_path() + + def test_default_path_parse(self, monkeypatch, tmppath): + cfg = tmppath / 'notmuch-config' + with cfg.open('w') as fp: + fp.write('[database]\n') + fp.write('path={!s}'.format(tmppath)) + monkeypatch.setenv('NOTMUCH_CONFIG', str(cfg)) + assert dbmod.Database.default_path() == tmppath + + def test_default_path_param(self, monkeypatch, tmppath): + cfg_dummy = tmppath / 'dummy' + monkeypatch.setenv('NOTMUCH_CONFIG', str(cfg_dummy)) + cfg_real = tmppath / 'notmuch_config' + with cfg_real.open('w') as fp: + fp.write('[database]\n') + fp.write('path={!s}'.format(cfg_real/'mail')) + assert dbmod.Database.default_path(cfg_real) == cfg_real/'mail' + + +class TestCreate: + + def test_create(self, tmppath, db): + assert tmppath.joinpath('.notmuch/xapian/').exists() + + def test_create_already_open(self, tmppath, db): + with pytest.raises(errors.NotmuchError): + db.create(tmppath) + + def test_create_existing(self, tmppath, db): + with pytest.raises(errors.DatabaseExistsError): + dbmod.Database.create(path=tmppath) + + def test_close(self, db): + db.close() + + def test_del_noclose(self, db): + del db + + def test_close_del(self, db): + db.close() + del db + + def test_closed_attr(self, db): + assert not db.closed + db.close() + assert db.closed + + def test_ctx(self, db): + with db as ctx: + assert ctx is db + assert not db.closed + assert db.closed + + def test_path(self, db, tmppath): + assert db.path == tmppath + + def test_version(self, db): + assert db.version > 0 + + def test_needs_upgrade(self, db): + assert db.needs_upgrade in (True, False) + + +class TestAtomic: + + def test_exit_early(self, db): + with pytest.raises(errors.UnbalancedAtomicError): + with db.atomic() as ctx: + ctx.force_end() + + def test_exit_late(self, db): + with db.atomic() as ctx: + pass + with pytest.raises(errors.UnbalancedAtomicError): + ctx.force_end() + + def test_abort(self, db): + with db.atomic() as txn: + txn.abort() + assert db.closed + + +class TestRevision: + + def test_single_rev(self, db): + r = db.revision() + assert isinstance(r, dbmod.DbRevision) + assert isinstance(r.rev, int) + assert isinstance(r.uuid, bytes) + assert r is r + assert r == r + assert r <= r + assert r >= r + assert not r < r + assert not r > r + + def test_diff_db(self, tmppath): + dbpath0 = tmppath.joinpath('db0') + dbpath0.mkdir() + dbpath1 = tmppath.joinpath('db1') + dbpath1.mkdir() + db0 = dbmod.Database.create(path=dbpath0) + db1 = dbmod.Database.create(path=dbpath1) + r_db0 = db0.revision() + r_db1 = db1.revision() + assert r_db0 != r_db1 + assert r_db0.uuid != r_db1.uuid + + def test_cmp(self, db, maildir): + rev0 = db.revision() + _, pathname = maildir.deliver() + db.add(pathname, sync_flags=False) + rev1 = db.revision() + assert rev0 < rev1 + assert rev0 <= rev1 + assert not rev0 > rev1 + assert not rev0 >= rev1 + assert not rev0 == rev1 + assert rev0 != rev1 + + # XXX add tests for revisions comparisons + +class TestMessages: + + def test_add_message(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(pathname, sync_flags=False) + assert isinstance(msg, message.Message) + assert msg.path == pathname + assert msg.messageid == msgid + + def test_add_message_str(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(str(pathname), sync_flags=False) + + def test_add_message_bytes(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(os.fsencode(bytes(pathname)), sync_flags=False) + + def test_remove_message(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(pathname, sync_flags=False) + assert db.find(msgid) + dup = db.remove(pathname) + with pytest.raises(LookupError): + db.find(msgid) + + def test_remove_message_str(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(pathname, sync_flags=False) + assert db.find(msgid) + dup = db.remove(str(pathname)) + with pytest.raises(LookupError): + db.find(msgid) + + def test_remove_message_bytes(self, db, maildir): + msgid, pathname = maildir.deliver() + msg, dup = db.add(pathname, sync_flags=False) + assert db.find(msgid) + dup = db.remove(os.fsencode(bytes(pathname))) + with pytest.raises(LookupError): + db.find(msgid) + + def test_find_message(self, db, maildir): + msgid, pathname = maildir.deliver() + msg0, dup = db.add(pathname, sync_flags=False) + msg1 = db.find(msgid) + assert isinstance(msg1, message.Message) + assert msg1.messageid == msgid == msg0.messageid + assert msg1.path == pathname == msg0.path + + def test_find_message_notfound(self, db): + with pytest.raises(LookupError): + db.find('foo') + + def test_get_message(self, db, maildir): + msgid, pathname = maildir.deliver() + msg0, _ = db.add(pathname, sync_flags=False) + msg1 = db.get(pathname) + assert isinstance(msg1, message.Message) + assert msg1.messageid == msgid == msg0.messageid + assert msg1.path == pathname == msg0.path + + def test_get_message_str(self, db, maildir): + msgid, pathname = maildir.deliver() + db.add(pathname, sync_flags=False) + msg = db.get(str(pathname)) + assert msg.messageid == msgid + + def test_get_message_bytes(self, db, maildir): + msgid, pathname = maildir.deliver() + db.add(pathname, sync_flags=False) + msg = db.get(os.fsencode(bytes(pathname))) + assert msg.messageid == msgid + + +class TestTags: + # We just want to test this behaves like a set at a hight level. + # The set semantics are tested in detail in the test_tags module. + + def test_type(self, db): + assert isinstance(db.tags, collections.abc.Set) + + def test_none(self, db): + itags = iter(db.tags) + with pytest.raises(StopIteration): + next(itags) + assert len(db.tags) == 0 + assert not db.tags + + def test_some(self, db, maildir): + _, pathname = maildir.deliver() + msg, _ = db.add(pathname, sync_flags=False) + msg.tags.add('hello') + itags = iter(db.tags) + assert next(itags) == 'hello' + with pytest.raises(StopIteration): + next(itags) + assert 'hello' in msg.tags + + def test_cache(self, db): + assert db.tags is db.tags + + def test_iters(self, db): + i1 = iter(db.tags) + i2 = iter(db.tags) + assert i1 is not i2 + + +class TestQuery: + + @pytest.fixture + def db(self, maildir, notmuch): + """Return a read-only notmuch2.Database. + + The database will have 3 messages, 2 threads. + """ + msgid, _ = maildir.deliver(body='foo') + maildir.deliver(body='bar') + maildir.deliver(body='baz', + headers=[('In-Reply-To', '<{}>'.format(msgid))]) + notmuch('new') + with dbmod.Database(maildir.path, 'rw', config=notmuch2.Database.CONFIG.EMPTY) as db: + yield db + + def test_count_messages(self, db): + assert db.count_messages('*') == 3 + + def test_messages_type(self, db): + msgs = db.messages('*') + assert isinstance(msgs, collections.abc.Iterator) + + def test_message_no_results(self, db): + msgs = db.messages('not_a_matching_query') + with pytest.raises(StopIteration): + next(msgs) + + def test_message_match(self, db): + msgs = db.messages('*') + msg = next(msgs) + assert isinstance(msg, notmuch2.Message) + + def test_count_threads(self, db): + assert db.count_threads('*') == 2 + + def test_threads_type(self, db): + threads = db.threads('*') + assert isinstance(threads, collections.abc.Iterator) + + def test_threads_no_match(self, db): + threads = db.threads('not_a_matching_query') + with pytest.raises(StopIteration): + next(threads) + + def test_threads_match(self, db): + threads = db.threads('*') + thread = next(threads) + assert isinstance(thread, notmuch2.Thread) + + def test_use_threaded_message_twice(self, db): + thread = next(db.threads('*')) + for msg in thread.toplevel(): + assert isinstance(msg, notmuch2.Message) + assert msg.alive + del msg + for msg in thread: + assert isinstance(msg, notmuch2.Message) + assert msg.alive + del msg diff --git a/bindings/python-cffi/tests/test_errors.py b/bindings/python-cffi/tests/test_errors.py new file mode 100644 index 00000000..c2519f86 --- /dev/null +++ b/bindings/python-cffi/tests/test_errors.py @@ -0,0 +1,8 @@ +from notmuch2 import _capi as capi +from notmuch2 import _errors as errors + +def test_status_no_message(): + exc = errors.NotmuchError(capi.lib.NOTMUCH_STATUS_PATH_ERROR) + assert exc.status == capi.lib.NOTMUCH_STATUS_PATH_ERROR + assert exc.message is None + assert str(exc) == 'Path supplied is illegal for this function' diff --git a/bindings/python-cffi/tests/test_message.py b/bindings/python-cffi/tests/test_message.py new file mode 100644 index 00000000..56701d05 --- /dev/null +++ b/bindings/python-cffi/tests/test_message.py @@ -0,0 +1,229 @@ +import collections.abc +import time +import pathlib + +import pytest + +import notmuch2 + + +class TestMessage: + MaildirMsg = collections.namedtuple('MaildirMsg', ['msgid', 'path']) + + @pytest.fixture + def maildir_msg(self, maildir): + msgid, path = maildir.deliver() + return self.MaildirMsg(msgid, path) + + @pytest.fixture + def db(self, maildir): + with notmuch2.Database.create(maildir.path) as db: + yield db + + @pytest.fixture + def msg(self, db, maildir_msg): + msg, dup = db.add(maildir_msg.path, sync_flags=False) + yield msg + + def test_type(self, msg): + assert isinstance(msg, notmuch2.NotmuchObject) + assert isinstance(msg, notmuch2.Message) + + def test_alive(self, msg): + assert msg.alive + + def test_hash(self, msg): + assert hash(msg) + + def test_eq(self, db, msg): + copy = db.get(msg.path) + assert msg == copy + + def test_messageid_type(self, msg): + assert isinstance(msg.messageid, str) + assert isinstance(msg.messageid, notmuch2.BinString) + assert isinstance(bytes(msg.messageid), bytes) + + def test_messageid(self, msg, maildir_msg): + assert msg.messageid == maildir_msg.msgid + + def test_messageid_find(self, db, msg): + copy = db.find(msg.messageid) + assert msg.messageid == copy.messageid + + def test_threadid_type(self, msg): + assert isinstance(msg.threadid, str) + assert isinstance(msg.threadid, notmuch2.BinString) + assert isinstance(bytes(msg.threadid), bytes) + + def test_path_type(self, msg): + assert isinstance(msg.path, pathlib.Path) + + def test_path(self, msg, maildir_msg): + assert msg.path == maildir_msg.path + + def test_pathb_type(self, msg): + assert isinstance(msg.pathb, bytes) + + def test_pathb(self, msg, maildir_msg): + assert msg.path == maildir_msg.path + + def test_filenames_type(self, msg): + ifn = msg.filenames() + assert isinstance(ifn, collections.abc.Iterator) + + def test_filenames(self, msg): + ifn = msg.filenames() + fn = next(ifn) + assert fn == msg.path + assert isinstance(fn, pathlib.Path) + with pytest.raises(StopIteration): + next(ifn) + assert list(msg.filenames()) == [msg.path] + + def test_filenamesb_type(self, msg): + ifn = msg.filenamesb() + assert isinstance(ifn, collections.abc.Iterator) + + def test_filenamesb(self, msg): + ifn = msg.filenamesb() + fn = next(ifn) + assert fn == msg.pathb + assert isinstance(fn, bytes) + with pytest.raises(StopIteration): + next(ifn) + assert list(msg.filenamesb()) == [msg.pathb] + + def test_ghost_no(self, msg): + assert not msg.ghost + + def test_matched_no(self,msg): + assert not msg.matched + + def test_date(self, msg): + # XXX Someone seems to treat things as local time instead of + # UTC or the other way around. + now = int(time.time()) + assert abs(now - msg.date) < 3600*24 + + def test_header(self, msg): + assert msg.header('from') == 'src@example.com' + + def test_header_not_present(self, msg): + with pytest.raises(LookupError): + msg.header('foo') + + def test_freeze(self, msg): + with msg.frozen(): + msg.tags.add('foo') + msg.tags.add('bar') + msg.tags.discard('foo') + assert 'foo' not in msg.tags + assert 'bar' in msg.tags + + def test_freeze_err(self, msg): + msg.tags.add('foo') + try: + with msg.frozen(): + msg.tags.clear() + raise Exception('oops') + except Exception: + assert 'foo' in msg.tags + else: + pytest.fail('Context manager did not raise') + + def test_replies_type(self, msg): + assert isinstance(msg.replies(), collections.abc.Iterator) + + def test_replies(self, msg): + with pytest.raises(StopIteration): + next(msg.replies()) + + +class TestProperties: + + @pytest.fixture + def props(self, maildir): + msgid, path = maildir.deliver() + with notmuch2.Database.create(maildir.path) as db: + msg, dup = db.add(path, sync_flags=False) + yield msg.properties + + def test_type(self, props): + assert isinstance(props, collections.abc.MutableMapping) + + def test_add_single(self, props): + props['foo'] = 'bar' + assert props['foo'] == 'bar' + props.add('bar', 'baz') + assert props['bar'] == 'baz' + + def test_add_dup(self, props): + props.add('foo', 'bar') + props.add('foo', 'baz') + assert props['foo'] == 'bar' + assert (set(props.getall('foo', exact=True)) + == {('foo', 'bar'), ('foo', 'baz')}) + + def test_len(self, props): + props.add('foo', 'a') + props.add('foo', 'b') + props.add('bar', 'a') + assert len(props) == 3 + assert len(props.keys()) == 2 + assert len(props.values()) == 2 + assert len(props.items()) == 3 + + def test_del(self, props): + props.add('foo', 'a') + props.add('foo', 'b') + del props['foo'] + with pytest.raises(KeyError): + props['foo'] + + def test_remove(self, props): + props.add('foo', 'a') + props.add('foo', 'b') + props.remove('foo', 'a') + assert props['foo'] == 'b' + + def test_view_abcs(self, props): + assert isinstance(props.keys(), collections.abc.KeysView) + assert isinstance(props.values(), collections.abc.ValuesView) + assert isinstance(props.items(), collections.abc.ItemsView) + + def test_pop(self, props): + props.add('foo', 'a') + props.add('foo', 'b') + val = props.pop('foo') + assert val == 'a' + + def test_pop_default(self, props): + with pytest.raises(KeyError): + props.pop('foo') + assert props.pop('foo', 'default') == 'default' + + def test_popitem(self, props): + props.add('foo', 'a') + assert props.popitem() == ('foo', 'a') + with pytest.raises(KeyError): + props.popitem() + + def test_clear(self, props): + props.add('foo', 'a') + props.clear() + assert len(props) == 0 + + def test_getall(self, props): + props.add('foo', 'a') + assert set(props.getall('foo')) == {('foo', 'a')} + + def test_getall_prefix(self, props): + props.add('foo', 'a') + props.add('foobar', 'b') + assert set(props.getall('foo')) == {('foo', 'a'), ('foobar', 'b')} + + def test_getall_exact(self, props): + props.add('foo', 'a') + props.add('foobar', 'b') + assert set(props.getall('foo', exact=True)) == {('foo', 'a')} diff --git a/bindings/python-cffi/tests/test_tags.py b/bindings/python-cffi/tests/test_tags.py new file mode 100644 index 00000000..f2c6209d --- /dev/null +++ b/bindings/python-cffi/tests/test_tags.py @@ -0,0 +1,242 @@ +"""Tests for the behaviour of immutable and mutable tagsets. + +This module tests the Pythonic behaviour of the sets. +""" + +import collections +import subprocess +import textwrap + +import pytest + +from notmuch2 import _database as database +from notmuch2 import _tags as tags + + +class TestImmutable: + + @pytest.fixture + def tagset(self, maildir, notmuch): + """An non-empty immutable tagset. + + This will have the default new mail tags: inbox, unread. + """ + maildir.deliver() + notmuch('new') + with database.Database(maildir.path, config=database.Database.CONFIG.EMPTY) as db: + yield db.tags + + def test_type(self, tagset): + assert isinstance(tagset, tags.ImmutableTagSet) + assert isinstance(tagset, collections.abc.Set) + + def test_hash(self, tagset, maildir, notmuch): + h0 = hash(tagset) + notmuch('tag', '+foo', '*') + with database.Database(maildir.path, config=database.Database.CONFIG.EMPTY) as db: + h1 = hash(db.tags) + assert h0 != h1 + + def test_eq(self, tagset): + assert tagset == tagset + + def test_neq(self, tagset, maildir, notmuch): + notmuch('tag', '+foo', '*') + with database.Database(maildir.path, config=database.Database.CONFIG.EMPTY) as db: + assert tagset != db.tags + + def test_contains(self, tagset): + print(tuple(tagset)) + assert 'unread' in tagset + assert 'foo' not in tagset + + def test_isdisjoint(self, tagset): + assert tagset.isdisjoint(set(['spam', 'ham'])) + assert not tagset.isdisjoint(set(['inbox'])) + + def test_issubset(self, tagset): + assert {'inbox'} <= tagset + assert {'inbox'}.issubset(tagset) + assert tagset <= {'inbox', 'unread', 'spam'} + assert tagset.issubset({'inbox', 'unread', 'spam'}) + + def test_issuperset(self, tagset): + assert {'inbox', 'unread', 'spam'} >= tagset + assert {'inbox', 'unread', 'spam'}.issuperset(tagset) + assert tagset >= {'inbox'} + assert tagset.issuperset({'inbox'}) + + def test_iter(self, tagset): + expected = sorted(['unread', 'inbox']) + found = [] + for tag in tagset: + assert isinstance(tag, str) + found.append(tag) + assert expected == sorted(found) + + def test_special_iter(self, tagset): + expected = sorted([b'unread', b'inbox']) + found = [] + for tag in tagset.iter(): + assert isinstance(tag, bytes) + found.append(tag) + assert expected == sorted(found) + + def test_special_iter_codec(self, tagset): + for tag in tagset.iter(encoding='ascii', errors='surrogateescape'): + assert isinstance(tag, str) + + def test_len(self, tagset): + assert len(tagset) == 2 + + def test_and(self, tagset): + common = tagset & {'unread'} + assert isinstance(common, set) + assert isinstance(common, collections.abc.Set) + assert common == {'unread'} + common = tagset.intersection({'unread'}) + assert isinstance(common, set) + assert isinstance(common, collections.abc.Set) + assert common == {'unread'} + + def test_or(self, tagset): + res = tagset | {'foo'} + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'unread', 'inbox', 'foo'} + res = tagset.union({'foo'}) + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'unread', 'inbox', 'foo'} + + def test_sub(self, tagset): + res = tagset - {'unread'} + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox'} + res = tagset.difference({'unread'}) + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox'} + + def test_rsub(self, tagset): + res = {'foo', 'unread'} - tagset + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'foo'} + + def test_xor(self, tagset): + res = tagset ^ {'unread', 'foo'} + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox', 'foo'} + res = tagset.symmetric_difference({'unread', 'foo'}) + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox', 'foo'} + + def test_rxor(self, tagset): + res = {'unread', 'foo'} ^ tagset + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox', 'foo'} + + def test_copy(self, tagset): + res = tagset.copy() + assert isinstance(res, set) + assert isinstance(res, collections.abc.Set) + assert res == {'inbox', 'unread'} + + +class TestMutableTagset: + + @pytest.fixture + def tagset(self, maildir, notmuch): + """An non-empty mutable tagset. + + This will have the default new mail tags: inbox, unread. + """ + _, pathname = maildir.deliver() + notmuch('new') + with database.Database(maildir.path, + mode=database.Mode.READ_WRITE, + config=database.Database.CONFIG.EMPTY) as db: + msg = db.get(pathname) + yield msg.tags + + def test_type(self, tagset): + assert isinstance(tagset, collections.abc.MutableSet) + assert isinstance(tagset, tags.MutableTagSet) + + def test_hash(self, tagset): + assert not isinstance(tagset, collections.abc.Hashable) + with pytest.raises(TypeError): + hash(tagset) + + def test_add(self, tagset): + assert 'foo' not in tagset + tagset.add('foo') + assert 'foo' in tagset + + def test_discard(self, tagset): + assert 'inbox' in tagset + tagset.discard('inbox') + assert 'inbox' not in tagset + + def test_discard_not_present(self, tagset): + assert 'foo' not in tagset + tagset.discard('foo') + + def test_clear(self, tagset): + assert len(tagset) > 0 + tagset.clear() + assert len(tagset) == 0 + + def test_from_maildir_flags(self, maildir, notmuch): + _, pathname = maildir.deliver(flagged=True) + notmuch('new') + with database.Database(maildir.path, + mode=database.Mode.READ_WRITE, + config=database.Database.CONFIG.EMPTY) as db: + msg = db.get(pathname) + msg.tags.discard('flagged') + msg.tags.from_maildir_flags() + assert 'flagged' in msg.tags + + def test_to_maildir_flags(self, maildir, notmuch): + _, pathname = maildir.deliver(flagged=True) + notmuch('new') + with database.Database(maildir.path, + mode=database.Mode.READ_WRITE, + config=database.Database.CONFIG.EMPTY) as db: + msg = db.get(pathname) + flags = msg.path.name.split(',')[-1] + assert 'F' in flags + msg.tags.discard('flagged') + msg.tags.to_maildir_flags() + flags = msg.path.name.split(',')[-1] + assert 'F' not in flags + + def test_isdisjoint(self, tagset): + assert tagset.isdisjoint(set(['spam', 'ham'])) + assert not tagset.isdisjoint(set(['inbox'])) + + def test_issubset(self, tagset): + assert {'inbox'} <= tagset + assert {'inbox'}.issubset(tagset) + assert not {'spam'} <= tagset + assert not {'spam'}.issubset(tagset) + assert tagset <= {'inbox', 'unread', 'spam'} + assert tagset.issubset({'inbox', 'unread', 'spam'}) + assert not {'inbox', 'unread', 'spam'} <= tagset + assert not {'inbox', 'unread', 'spam'}.issubset(tagset) + + def test_issuperset(self, tagset): + assert {'inbox', 'unread', 'spam'} >= tagset + assert {'inbox', 'unread', 'spam'}.issuperset(tagset) + assert tagset >= {'inbox'} + assert tagset.issuperset({'inbox'}) + + def test_union(self, tagset): + assert {'spam'}.union(tagset) == {'inbox', 'unread', 'spam'} + assert tagset.union({'spam'}) == {'inbox', 'unread', 'spam'} diff --git a/bindings/python-cffi/tests/test_thread.py b/bindings/python-cffi/tests/test_thread.py new file mode 100644 index 00000000..619d2aac --- /dev/null +++ b/bindings/python-cffi/tests/test_thread.py @@ -0,0 +1,109 @@ +import collections.abc +import time + +import pytest + +import notmuch2 + + +@pytest.fixture +def thread(maildir, notmuch): + """Return a single thread with one matched message.""" + msgid, _ = maildir.deliver(body='foo') + maildir.deliver(body='bar', + headers=[('In-Reply-To', '<{}>'.format(msgid))]) + notmuch('new') + with notmuch2.Database(maildir.path, config=notmuch2.Database.CONFIG.EMPTY) as db: + yield next(db.threads('foo')) + + +def test_type(thread): + assert isinstance(thread, notmuch2.Thread) + assert isinstance(thread, collections.abc.Iterable) + + +def test_threadid(thread): + assert isinstance(thread.threadid, notmuch2.BinString) + assert thread.threadid + + +def test_len(thread): + assert len(thread) == 2 + + +def test_toplevel_type(thread): + assert isinstance(thread.toplevel(), collections.abc.Iterator) + + +def test_toplevel(thread): + msgs = thread.toplevel() + assert isinstance(next(msgs), notmuch2.Message) + with pytest.raises(StopIteration): + next(msgs) + + +def test_toplevel_reply(thread): + msg = next(thread.toplevel()) + assert isinstance(next(msg.replies()), notmuch2.Message) + + +def test_iter(thread): + msgs = list(iter(thread)) + assert len(msgs) == len(thread) + for msg in msgs: + assert isinstance(msg, notmuch2.Message) + + +def test_matched(thread): + assert thread.matched == 1 + +def test_matched_iter(thread): + count = 0 + msgs = list(iter(thread)) + for msg in msgs: + if msg.matched: + count += 1 + assert count == thread.matched + +def test_authors_type(thread): + assert isinstance(thread.authors, notmuch2.BinString) + + +def test_authors(thread): + assert thread.authors == 'src@example.com' + + +def test_subject(thread): + assert thread.subject == 'Test mail' + + +def test_first(thread): + # XXX Someone seems to treat things as local time instead of + # UTC or the other way around. + now = int(time.time()) + assert abs(now - thread.first) < 3600*24 + + +def test_last(thread): + # XXX Someone seems to treat things as local time instead of + # UTC or the other way around. + now = int(time.time()) + assert abs(now - thread.last) < 3600*24 + + +def test_first_last(thread): + # Sadly we only have second resolution so these will always be the + # same time in our tests. + assert thread.first <= thread.last + + +def test_tags_type(thread): + assert isinstance(thread.tags, notmuch2.ImmutableTagSet) + + +def test_tags_cache(thread): + assert thread.tags is thread.tags + + +def test_tags(thread): + assert 'inbox' in thread.tags diff --git a/bindings/python-cffi/tox.ini b/bindings/python-cffi/tox.ini new file mode 100644 index 00000000..7cf93be0 --- /dev/null +++ b/bindings/python-cffi/tox.ini @@ -0,0 +1,19 @@ +[pytest] +minversion = 3.0 +addopts = -ra --cov=notmuch2 --cov=tests + +[tox] +envlist = py35,py36,py37,py38,pypy35,pypy36 + +[testenv] +deps = + cffi + pytest + pytest-cov +commands = pytest --cov={envsitepackagesdir}/notmuch2 {posargs} + +[testenv:pypy35] +basepython = pypy3.5 + +[testenv:pypy36] +basepython = pypy3.6 diff --git a/bindings/python/notmuch/compat.py b/bindings/python/notmuch/compat.py index c931329e..4a94e05c 100644 --- a/bindings/python/notmuch/compat.py +++ b/bindings/python/notmuch/compat.py @@ -47,7 +47,10 @@ if sys.version_info[0] == 2: return value else: - from configparser import SafeConfigParser + from configparser import ConfigParser as SafeConfigParser + + if not hasattr(SafeConfigParser, 'readfp'): # py >= 3.12 + SafeConfigParser.readfp = SafeConfigParser.read_file class Python3StringMixIn(object): def __str__(self): diff --git a/bindings/python/notmuch/database.py b/bindings/python/notmuch/database.py index 88ca836e..8fb507fa 100644 --- a/bindings/python/notmuch/database.py +++ b/bindings/python/notmuch/database.py @@ -65,7 +65,7 @@ class Database(object): .. note:: Any function in this class can and will throw an - :exc:`NotInitializedError` if the database was not intitialized + :exc:`NotInitializedError` if the database was not initialized properly. """ _std_db_path = None @@ -273,9 +273,9 @@ class Database(object): return Database._get_version(self._db) def get_revision (self): - """Returns the committed database revison and UUID + """Returns the committed database revision and UUID - :returns: (revison, uuid) The database revision as a positive integer + :returns: (revision, uuid) The database revision as a positive integer and the UUID of the database. """ self._assert_db_is_initialized() @@ -574,7 +574,7 @@ class Database(object): in the meantime. In this case, you should close and reopen the database and retry. :exc:`NotInitializedError` if - the database was not intitialized. + the database was not initialized. """ self._assert_db_is_initialized() msg_p = NotmuchMessageP() @@ -600,7 +600,7 @@ class Database(object): case, you should close and reopen the database and retry. :raises: :exc:`NotInitializedError` if the database was not - intitialized. + initialized. *Added in notmuch 0.9*""" self._assert_db_is_initialized() @@ -616,7 +616,7 @@ class Database(object): """Returns :class:`Tags` with a list of all tags found in the database :returns: :class:`Tags` - :execption: :exc:`NotmuchError` with :attr:`STATUS`.NULL_POINTER + :exception: :exc:`NotmuchError` with :attr:`STATUS`.NULL_POINTER on error """ self._assert_db_is_initialized() diff --git a/bindings/python/notmuch/message.py b/bindings/python/notmuch/message.py index 6e32b5f7..e71dbe3e 100644 --- a/bindings/python/notmuch/message.py +++ b/bindings/python/notmuch/message.py @@ -46,7 +46,7 @@ import sys class Message(Python3StringMixIn): - """Represents a single Email message + r"""Represents a single Email message Technically, this wraps the underlying *notmuch_message_t* structure. A user will usually not create these objects themselves diff --git a/bindings/python/notmuch/messages.py b/bindings/python/notmuch/messages.py index cae5da50..3801c666 100644 --- a/bindings/python/notmuch/messages.py +++ b/bindings/python/notmuch/messages.py @@ -32,7 +32,7 @@ from .tag import Tags from .message import Message class Messages(object): - """Represents a list of notmuch messages + r"""Represents a list of notmuch messages This object provides an iterator over a list of notmuch messages (Technically, it provides a wrapper for the underlying diff --git a/bindings/python/notmuch/query.py b/bindings/python/notmuch/query.py index cc70e2aa..ffb86df1 100644 --- a/bindings/python/notmuch/query.py +++ b/bindings/python/notmuch/query.py @@ -95,7 +95,7 @@ class Query(object): :exc:`NullPointerError` if the query creation failed (e.g. too little memory). :exc:`NotInitializedError` if the underlying db was not - intitialized. + initialized. """ db._assert_db_is_initialized() # create reference to parent db to keep it alive @@ -140,7 +140,7 @@ class Query(object): _search_threads.restype = c_uint def search_threads(self): - """Execute a query for threads + r"""Execute a query for threads Execute a query for threads, returning a :class:`Threads` iterator. The returned threads are owned by the query and as such, will only be diff --git a/bindings/python/notmuch/version.py b/bindings/python/notmuch/version.py index ca9e6a8c..fd4152ee 100644 --- a/bindings/python/notmuch/version.py +++ b/bindings/python/notmuch/version.py @@ -1,3 +1,3 @@ # this file should be kept in sync with ../../../version -__VERSION__ = '0.29.1' +__VERSION__ = '0.38.3' SOVERSION = '5' diff --git a/bindings/python/setup.py b/bindings/python/setup.py index d986f0c6..6308b9f9 100644 --- a/bindings/python/setup.py +++ b/bindings/python/setup.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 """ This file is part of notmuch. diff --git a/bindings/ruby/database.c b/bindings/ruby/database.c index 416eb709..ed224ef7 100644 --- a/bindings/ruby/database.c +++ b/bindings/ruby/database.c @@ -23,7 +23,20 @@ VALUE notmuch_rb_database_alloc (VALUE klass) { - return Data_Wrap_Struct (klass, NULL, NULL, NULL); + return Data_Wrap_Notmuch_Object (klass, ¬much_rb_database_type, NULL); +} + +/* + * call-seq: DB.destroy => nil + * + * Destroys the database, freeing all resources allocated for it. + */ +VALUE +notmuch_rb_database_destroy (VALUE self) +{ + notmuch_rb_object_destroy (self, ¬much_rb_database_type); + + return Qnil; } /* @@ -45,43 +58,59 @@ notmuch_rb_database_initialize (int argc, VALUE *argv, VALUE self) notmuch_database_t *database; notmuch_status_t ret; + path = NULL; + create = 0; + mode = NOTMUCH_DATABASE_MODE_READ_ONLY; + /* Check arguments */ - rb_scan_args (argc, argv, "11", &pathv, &hashv); + rb_scan_args (argc, argv, "02", &pathv, &hashv); - SafeStringValue (pathv); - path = RSTRING_PTR (pathv); + if (!NIL_P (pathv)) { + SafeStringValue (pathv); + path = RSTRING_PTR (pathv); + } if (!NIL_P (hashv)) { - Check_Type (hashv, T_HASH); - create = RTEST (rb_hash_aref (hashv, ID2SYM (ID_db_create))); - modev = rb_hash_aref (hashv, ID2SYM (ID_db_mode)); - if (NIL_P (modev)) - mode = NOTMUCH_DATABASE_MODE_READ_ONLY; - else if (!FIXNUM_P (modev)) - rb_raise (rb_eTypeError, ":mode isn't a Fixnum"); - else { - mode = FIX2INT (modev); - switch (mode) { - case NOTMUCH_DATABASE_MODE_READ_ONLY: - case NOTMUCH_DATABASE_MODE_READ_WRITE: - break; - default: - rb_raise ( rb_eTypeError, "Invalid mode"); + VALUE rmode, rcreate; + VALUE kwargs[2]; + static ID keyword_ids[2]; + + if (!keyword_ids[0]) { + keyword_ids[0] = rb_intern_const ("mode"); + keyword_ids[1] = rb_intern_const ("create"); + } + + rb_get_kwargs (hashv, keyword_ids, 0, 2, kwargs); + + rmode = kwargs[0]; + rcreate = kwargs[1]; + + if (rmode != Qundef) { + if (!FIXNUM_P (rmode)) + rb_raise (rb_eTypeError, ":mode isn't a Fixnum"); + else { + mode = FIX2INT (rmode); + switch (mode) { + case NOTMUCH_DATABASE_MODE_READ_ONLY: + case NOTMUCH_DATABASE_MODE_READ_WRITE: + break; + default: + rb_raise ( rb_eTypeError, "Invalid mode"); + } } } - } else { - create = 0; - mode = NOTMUCH_DATABASE_MODE_READ_ONLY; + if (rcreate != Qundef) + create = RTEST (rcreate); } - Check_Type (self, T_DATA); + rb_check_typeddata (self, ¬much_rb_database_type); if (create) ret = notmuch_database_create (path, &database); else - ret = notmuch_database_open (path, mode, &database); + ret = notmuch_database_open_with_config (path, mode, NULL, NULL, &database, NULL); notmuch_rb_status_raise (ret); - DATA_PTR (self) = database; + DATA_PTR (self) = notmuch_rb_object_create (database, "notmuch_rb_database"); return self; } @@ -113,12 +142,12 @@ notmuch_rb_database_open (int argc, VALUE *argv, VALUE klass) VALUE notmuch_rb_database_close (VALUE self) { - notmuch_status_t ret; notmuch_database_t *db; + notmuch_status_t ret; Data_Get_Notmuch_Database (self, db); - ret = notmuch_database_destroy (db); - DATA_PTR (self) = NULL; + + ret = notmuch_database_close (db); notmuch_rb_status_raise (ret); return Qnil; @@ -266,7 +295,7 @@ notmuch_rb_database_get_directory (VALUE self, VALUE pathv) ret = notmuch_database_get_directory (db, path, &dir); notmuch_rb_status_raise (ret); if (dir) - return Data_Wrap_Struct (notmuch_rb_cDirectory, NULL, NULL, dir); + return Data_Wrap_Notmuch_Object (notmuch_rb_cDirectory, ¬much_rb_directory_type, dir); return Qnil; } @@ -293,7 +322,7 @@ notmuch_rb_database_add_message (VALUE self, VALUE pathv) ret = notmuch_database_index_file (db, path, NULL, &message); notmuch_rb_status_raise (ret); - return rb_assoc_new (Data_Wrap_Struct (notmuch_rb_cMessage, NULL, NULL, message), + return rb_assoc_new (Data_Wrap_Notmuch_Object (notmuch_rb_cMessage, ¬much_rb_message_type, message), (ret == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) ? Qtrue : Qfalse); } @@ -344,7 +373,7 @@ notmuch_rb_database_find_message (VALUE self, VALUE idv) notmuch_rb_status_raise (ret); if (message) - return Data_Wrap_Struct (notmuch_rb_cMessage, NULL, NULL, message); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessage, ¬much_rb_message_type, message); return Qnil; } @@ -370,7 +399,7 @@ notmuch_rb_database_find_message_by_filename (VALUE self, VALUE pathv) notmuch_rb_status_raise (ret); if (message) - return Data_Wrap_Struct (notmuch_rb_cMessage, NULL, NULL, message); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessage, ¬much_rb_message_type, message); return Qnil; } @@ -395,21 +424,28 @@ notmuch_rb_database_get_all_tags (VALUE self) rb_raise (notmuch_rb_eBaseError, "%s", msg); } - return Data_Wrap_Struct (notmuch_rb_cTags, NULL, NULL, tags); + return notmuch_rb_tags_get (tags); } /* - * call-seq: DB.query(query) => QUERY + * call-seq: + * DB.query(query) => QUERY + * DB.query(query, sort:, excluded_tags:, omit_excluded:) => QUERY * - * Retrieve a query object for the query string 'query' + * Retrieve a query object for the query string 'query'. When using keyword + * arguments they are passwed to the query object. */ VALUE -notmuch_rb_database_query_create (VALUE self, VALUE qstrv) +notmuch_rb_database_query_create (int argc, VALUE *argv, VALUE self) { + VALUE qstrv; + VALUE opts; const char *qstr; notmuch_query_t *query; notmuch_database_t *db; + rb_scan_args (argc, argv, "1:", &qstrv, &opts); + Data_Get_Notmuch_Database (self, db); SafeStringValue (qstrv); @@ -419,5 +455,39 @@ notmuch_rb_database_query_create (VALUE self, VALUE qstrv) if (!query) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cQuery, NULL, NULL, query); + if (!NIL_P (opts)) { + VALUE sort, exclude_tags, omit_excluded; + VALUE kwargs[3]; + static ID keyword_ids[3]; + + if (!keyword_ids[0]) { + keyword_ids[0] = rb_intern_const ("sort"); + keyword_ids[1] = rb_intern_const ("exclude_tags"); + keyword_ids[2] = rb_intern_const ("omit_excluded"); + } + + rb_get_kwargs (opts, keyword_ids, 0, 3, kwargs); + + sort = kwargs[0]; + exclude_tags = kwargs[1]; + omit_excluded = kwargs[2]; + + if (sort != Qundef) + notmuch_query_set_sort (query, FIX2UINT (sort)); + + if (exclude_tags != Qundef) { + for (int i = 0; i < RARRAY_LEN (exclude_tags); i++) { + VALUE e = RARRAY_AREF (exclude_tags, i); + notmuch_query_add_tag_exclude (query, RSTRING_PTR (e)); + } + } + + if (omit_excluded != Qundef) { + notmuch_exclude_t omit; + omit = FIXNUM_P (omit_excluded) ? FIX2UINT (omit_excluded) : RTEST(omit_excluded); + notmuch_query_set_omit_excluded (query, omit); + } + } + + return Data_Wrap_Notmuch_Object (notmuch_rb_cQuery, ¬much_rb_query_type, query); } diff --git a/bindings/ruby/defs.h b/bindings/ruby/defs.h index 48544ca2..a2cb38c8 100644 --- a/bindings/ruby/defs.h +++ b/bindings/ruby/defs.h @@ -23,6 +23,7 @@ #include #include +#include extern VALUE notmuch_rb_cDatabase; extern VALUE notmuch_rb_cDirectory; @@ -32,7 +33,6 @@ extern VALUE notmuch_rb_cThreads; extern VALUE notmuch_rb_cThread; extern VALUE notmuch_rb_cMessages; extern VALUE notmuch_rb_cMessage; -extern VALUE notmuch_rb_cTags; extern VALUE notmuch_rb_eBaseError; extern VALUE notmuch_rb_eDatabaseError; @@ -47,85 +47,100 @@ extern VALUE notmuch_rb_eUnbalancedFreezeThawError; extern VALUE notmuch_rb_eUnbalancedAtomicError; extern ID ID_call; -extern ID ID_db_create; -extern ID ID_db_mode; /* RSTRING_PTR() is new in ruby-1.9 */ #if !defined(RSTRING_PTR) # define RSTRING_PTR(v) (RSTRING((v))->ptr) #endif /* !defined (RSTRING_PTR) */ -#define Data_Get_Notmuch_Database(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "database closed"); \ - Data_Get_Struct ((obj), notmuch_database_t, (ptr)); \ +extern const rb_data_type_t notmuch_rb_object_type; +extern const rb_data_type_t notmuch_rb_database_type; +extern const rb_data_type_t notmuch_rb_directory_type; +extern const rb_data_type_t notmuch_rb_query_type; +extern const rb_data_type_t notmuch_rb_threads_type; +extern const rb_data_type_t notmuch_rb_thread_type; +extern const rb_data_type_t notmuch_rb_messages_type; +extern const rb_data_type_t notmuch_rb_message_type; +extern const rb_data_type_t notmuch_rb_tags_type; + +#define Data_Get_Notmuch_Rb_Object(obj, type, ptr) \ + do { \ + (ptr) = rb_check_typeddata ((obj), (type)); \ + if (RB_UNLIKELY (!(ptr))) { \ + VALUE cname = rb_class_name (CLASS_OF ((obj))); \ + rb_raise (rb_eRuntimeError, "%"PRIsVALUE" object destroyed", cname); \ + } \ } while (0) -#define Data_Get_Notmuch_Directory(obj, ptr) \ +#define Data_Get_Notmuch_Object(obj, type, ptr) \ do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "directory destroyed"); \ - Data_Get_Struct ((obj), notmuch_directory_t, (ptr)); \ + notmuch_rb_object_t *rb_wrapper; \ + Data_Get_Notmuch_Rb_Object ((obj), (type), rb_wrapper); \ + (ptr) = rb_wrapper->nm_object; \ } while (0) -#define Data_Get_Notmuch_FileNames(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "filenames destroyed"); \ - Data_Get_Struct ((obj), notmuch_filenames_t, (ptr)); \ - } while (0) +#define Data_Wrap_Notmuch_Object(klass, type, ptr) \ + TypedData_Wrap_Struct ((klass), (type), notmuch_rb_object_create ((ptr), "notmuch_rb_object: " __location__)) -#define Data_Get_Notmuch_Query(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "query destroyed"); \ - Data_Get_Struct ((obj), notmuch_query_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Database(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_database_type, (ptr)) -#define Data_Get_Notmuch_Threads(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "threads destroyed"); \ - Data_Get_Struct ((obj), notmuch_threads_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Directory(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_directory_type, (ptr)) -#define Data_Get_Notmuch_Messages(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "messages destroyed"); \ - Data_Get_Struct ((obj), notmuch_messages_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Query(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_query_type, (ptr)) -#define Data_Get_Notmuch_Thread(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "thread destroyed"); \ - Data_Get_Struct ((obj), notmuch_thread_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Threads(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_threads_type, (ptr)) -#define Data_Get_Notmuch_Message(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "message destroyed"); \ - Data_Get_Struct ((obj), notmuch_message_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Messages(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_messages_type, (ptr)) -#define Data_Get_Notmuch_Tags(obj, ptr) \ - do { \ - Check_Type ((obj), T_DATA); \ - if (DATA_PTR ((obj)) == NULL) \ - rb_raise (rb_eRuntimeError, "tags destroyed"); \ - Data_Get_Struct ((obj), notmuch_tags_t, (ptr)); \ - } while (0) +#define Data_Get_Notmuch_Thread(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_thread_type, (ptr)) + +#define Data_Get_Notmuch_Message(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_message_type, (ptr)) + +#define Data_Get_Notmuch_Tags(obj, ptr) \ + Data_Get_Notmuch_Object ((obj), ¬much_rb_tags_type, (ptr)) + +typedef struct { + void *nm_object; +} notmuch_rb_object_t; + +static inline void * +notmuch_rb_object_create (void *nm_object, const char *name) +{ + notmuch_rb_object_t *rb_wrapper = talloc_named_const (NULL, sizeof (*rb_wrapper), name); + + if (RB_UNLIKELY (!rb_wrapper)) + return NULL; + + rb_wrapper->nm_object = nm_object; + talloc_steal (rb_wrapper, nm_object); + return rb_wrapper; +} + +static inline void +notmuch_rb_object_free (void *rb_wrapper) +{ + talloc_free (rb_wrapper); +} + +static inline void +notmuch_rb_object_destroy (VALUE rb_object, const rb_data_type_t *type) +{ + notmuch_rb_object_t *rb_wrapper; + + Data_Get_Notmuch_Rb_Object (rb_object, type, rb_wrapper); + + /* Call the corresponding notmuch_*_destroy function */ + ((void (*)(void *)) type->data) (rb_wrapper->nm_object); + notmuch_rb_object_free (rb_wrapper); + DATA_PTR (rb_object) = NULL; +} /* status.c */ void @@ -135,6 +150,9 @@ notmuch_rb_status_raise (notmuch_status_t status); VALUE notmuch_rb_database_alloc (VALUE klass); +VALUE +notmuch_rb_database_destroy (VALUE self); + VALUE notmuch_rb_database_initialize (int argc, VALUE *argv, VALUE klass); @@ -181,7 +199,7 @@ VALUE notmuch_rb_database_get_all_tags (VALUE self); VALUE -notmuch_rb_database_query_create (VALUE self, VALUE qstrv); +notmuch_rb_database_query_create (int argc, VALUE *argv, VALUE self); /* directory.c */ VALUE @@ -201,10 +219,7 @@ notmuch_rb_directory_get_child_directories (VALUE self); /* filenames.c */ VALUE -notmuch_rb_filenames_destroy (VALUE self); - -VALUE -notmuch_rb_filenames_each (VALUE self); +notmuch_rb_filenames_get (notmuch_filenames_t *fnames); /* query.c */ VALUE @@ -345,10 +360,7 @@ notmuch_rb_message_thaw (VALUE self); /* tags.c */ VALUE -notmuch_rb_tags_destroy (VALUE self); - -VALUE -notmuch_rb_tags_each (VALUE self); +notmuch_rb_tags_get (notmuch_tags_t *tags); /* init.c */ void diff --git a/bindings/ruby/directory.c b/bindings/ruby/directory.c index 0f37b391..f267d82f 100644 --- a/bindings/ruby/directory.c +++ b/bindings/ruby/directory.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_directory_destroy (VALUE self) { - notmuch_directory_t *dir; - - Data_Get_Struct (self, notmuch_directory_t, dir); - - notmuch_directory_destroy (dir); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_directory_type); return Qnil; } @@ -92,7 +87,7 @@ notmuch_rb_directory_get_child_files (VALUE self) fnames = notmuch_directory_get_child_files (dir); - return Data_Wrap_Struct (notmuch_rb_cFileNames, NULL, NULL, fnames); + return notmuch_rb_filenames_get (fnames); } /* @@ -111,5 +106,5 @@ notmuch_rb_directory_get_child_directories (VALUE self) fnames = notmuch_directory_get_child_directories (dir); - return Data_Wrap_Struct (notmuch_rb_cFileNames, NULL, NULL, fnames); + return notmuch_rb_filenames_get (fnames); } diff --git a/bindings/ruby/extconf.rb b/bindings/ruby/extconf.rb index 161de5a2..d914537c 100644 --- a/bindings/ruby/extconf.rb +++ b/bindings/ruby/extconf.rb @@ -19,6 +19,7 @@ if not ENV['LIBNOTMUCH'] end $LOCAL_LIBS += ENV['LIBNOTMUCH'] +$LIBS += " -ltalloc" # Create Makefile dir_config('notmuch') diff --git a/bindings/ruby/filenames.c b/bindings/ruby/filenames.c index 656c58e6..60c3fb8b 100644 --- a/bindings/ruby/filenames.c +++ b/bindings/ruby/filenames.c @@ -1,58 +1,11 @@ -/* The Ruby interface to the notmuch mail library - * - * Copyright © 2010 Ali Polatel - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see https://www.gnu.org/licenses/ . - * - * Author: Ali Polatel - */ - #include "defs.h" -/* - * call-seq: FILENAMES.destroy! => nil - * - * Destroys the filenames, freeing all resources allocated for it. - */ -VALUE -notmuch_rb_filenames_destroy (VALUE self) -{ - notmuch_filenames_t *fnames; - - Data_Get_Notmuch_FileNames (self, fnames); - - notmuch_filenames_destroy (fnames); - DATA_PTR (self) = NULL; - - return Qnil; -} - -/* - * call-seq: FILENAMES.each {|item| block } => FILENAMES - * - * Calls +block+ once for each element in +self+, passing that element as a - * parameter. - */ VALUE -notmuch_rb_filenames_each (VALUE self) +notmuch_rb_filenames_get (notmuch_filenames_t *fnames) { - notmuch_filenames_t *fnames; - - Data_Get_Notmuch_FileNames (self, fnames); + VALUE rb_array = rb_ary_new (); for (; notmuch_filenames_valid (fnames); notmuch_filenames_move_to_next (fnames)) - rb_yield (rb_str_new2 (notmuch_filenames_get (fnames))); - - return self; + rb_ary_push (rb_array, rb_str_new2 (notmuch_filenames_get (fnames))); + return rb_array; } diff --git a/bindings/ruby/init.c b/bindings/ruby/init.c index 5556b43e..2d1994af 100644 --- a/bindings/ruby/init.c +++ b/bindings/ruby/init.c @@ -22,13 +22,11 @@ VALUE notmuch_rb_cDatabase; VALUE notmuch_rb_cDirectory; -VALUE notmuch_rb_cFileNames; VALUE notmuch_rb_cQuery; VALUE notmuch_rb_cThreads; VALUE notmuch_rb_cThread; VALUE notmuch_rb_cMessages; VALUE notmuch_rb_cMessage; -VALUE notmuch_rb_cTags; VALUE notmuch_rb_eBaseError; VALUE notmuch_rb_eDatabaseError; @@ -43,8 +41,31 @@ VALUE notmuch_rb_eUnbalancedFreezeThawError; VALUE notmuch_rb_eUnbalancedAtomicError; ID ID_call; -ID ID_db_create; -ID ID_db_mode; + +const rb_data_type_t notmuch_rb_object_type = { + .wrap_struct_name = "notmuch_object", + .function = { + .dfree = notmuch_rb_object_free, + }, +}; + +#define define_type(id) \ + const rb_data_type_t notmuch_rb_ ## id ## _type = { \ + .wrap_struct_name = "notmuch_" #id, \ + .parent = ¬much_rb_object_type, \ + .data = ¬much_ ## id ## _destroy, \ + .function = { \ + .dfree = notmuch_rb_object_free, \ + }, \ + } + +define_type (database); +define_type (directory); +define_type (query); +define_type (threads); +define_type (thread); +define_type (messages); +define_type (message); /* * Document-module: Notmuch @@ -59,13 +80,11 @@ ID ID_db_mode; * the user: * * - Notmuch::Database - * - Notmuch::FileNames * - Notmuch::Query * - Notmuch::Threads * - Notmuch::Messages * - Notmuch::Thread * - Notmuch::Message - * - Notmuch::Tags */ void @@ -74,8 +93,6 @@ Init_notmuch (void) VALUE mod; ID_call = rb_intern ("call"); - ID_db_create = rb_intern ("create"); - ID_db_mode = rb_intern ("mode"); mod = rb_define_module ("Notmuch"); @@ -133,6 +150,30 @@ Init_notmuch (void) * Maximum allowed length of a tag */ rb_define_const (mod, "TAG_MAX", INT2FIX (NOTMUCH_TAG_MAX)); + /* + * Document-const: Notmuch::EXCLUDE_FLAG + * + * Only flag excluded results + */ + rb_define_const (mod, "EXCLUDE_FLAG", INT2FIX (NOTMUCH_EXCLUDE_FLAG)); + /* + * Document-const: Notmuch::EXCLUDE_TRUE + * + * Exclude messages from the results + */ + rb_define_const (mod, "EXCLUDE_TRUE", INT2FIX (NOTMUCH_EXCLUDE_TRUE)); + /* + * Document-const: Notmuch::EXCLUDE_FALSE + * + * Don't exclude anything + */ + rb_define_const (mod, "EXCLUDE_FALSE", INT2FIX (NOTMUCH_EXCLUDE_FALSE)); + /* + * Document-const: Notmuch::EXCLUDE_ALL + * + * Exclude all results + */ + rb_define_const (mod, "EXCLUDE_ALL", INT2FIX (NOTMUCH_EXCLUDE_ALL)); /* * Document-class: Notmuch::BaseError @@ -211,10 +252,11 @@ Init_notmuch (void) * * Notmuch database interaction */ - notmuch_rb_cDatabase = rb_define_class_under (mod, "Database", rb_cData); + notmuch_rb_cDatabase = rb_define_class_under (mod, "Database", rb_cObject); rb_define_alloc_func (notmuch_rb_cDatabase, notmuch_rb_database_alloc); rb_define_singleton_method (notmuch_rb_cDatabase, "open", notmuch_rb_database_open, -1); /* in database.c */ rb_define_method (notmuch_rb_cDatabase, "initialize", notmuch_rb_database_initialize, -1); /* in database.c */ + rb_define_method (notmuch_rb_cDatabase, "destroy!", notmuch_rb_database_destroy, 0); /* in database.c */ rb_define_method (notmuch_rb_cDatabase, "close", notmuch_rb_database_close, 0); /* in database.c */ rb_define_method (notmuch_rb_cDatabase, "path", notmuch_rb_database_path, 0); /* in database.c */ rb_define_method (notmuch_rb_cDatabase, "version", notmuch_rb_database_version, 0); /* in database.c */ @@ -230,14 +272,14 @@ Init_notmuch (void) rb_define_method (notmuch_rb_cDatabase, "find_message_by_filename", notmuch_rb_database_find_message_by_filename, 1); /* in database.c */ rb_define_method (notmuch_rb_cDatabase, "all_tags", notmuch_rb_database_get_all_tags, 0); /* in database.c */ - rb_define_method (notmuch_rb_cDatabase, "query", notmuch_rb_database_query_create, 1); /* in database.c */ + rb_define_method (notmuch_rb_cDatabase, "query", notmuch_rb_database_query_create, -1); /* in database.c */ /* * Document-class: Notmuch::Directory * * Notmuch directory */ - notmuch_rb_cDirectory = rb_define_class_under (mod, "Directory", rb_cData); + notmuch_rb_cDirectory = rb_define_class_under (mod, "Directory", rb_cObject); rb_undef_method (notmuch_rb_cDirectory, "initialize"); rb_define_method (notmuch_rb_cDirectory, "destroy!", notmuch_rb_directory_destroy, 0); /* in directory.c */ rb_define_method (notmuch_rb_cDirectory, "mtime", notmuch_rb_directory_get_mtime, 0); /* in directory.c */ @@ -245,23 +287,12 @@ Init_notmuch (void) rb_define_method (notmuch_rb_cDirectory, "child_files", notmuch_rb_directory_get_child_files, 0); /* in directory.c */ rb_define_method (notmuch_rb_cDirectory, "child_directories", notmuch_rb_directory_get_child_directories, 0); /* in directory.c */ - /* - * Document-class: Notmuch::FileNames - * - * Notmuch file names - */ - notmuch_rb_cFileNames = rb_define_class_under (mod, "FileNames", rb_cData); - rb_undef_method (notmuch_rb_cFileNames, "initialize"); - rb_define_method (notmuch_rb_cFileNames, "destroy!", notmuch_rb_filenames_destroy, 0); /* in filenames.c */ - rb_define_method (notmuch_rb_cFileNames, "each", notmuch_rb_filenames_each, 0); /* in filenames.c */ - rb_include_module (notmuch_rb_cFileNames, rb_mEnumerable); - /* * Document-class: Notmuch::Query * * Notmuch query */ - notmuch_rb_cQuery = rb_define_class_under (mod, "Query", rb_cData); + notmuch_rb_cQuery = rb_define_class_under (mod, "Query", rb_cObject); rb_undef_method (notmuch_rb_cQuery, "initialize"); rb_define_method (notmuch_rb_cQuery, "destroy!", notmuch_rb_query_destroy, 0); /* in query.c */ rb_define_method (notmuch_rb_cQuery, "sort", notmuch_rb_query_get_sort, 0); /* in query.c */ @@ -279,7 +310,7 @@ Init_notmuch (void) * * Notmuch threads */ - notmuch_rb_cThreads = rb_define_class_under (mod, "Threads", rb_cData); + notmuch_rb_cThreads = rb_define_class_under (mod, "Threads", rb_cObject); rb_undef_method (notmuch_rb_cThreads, "initialize"); rb_define_method (notmuch_rb_cThreads, "destroy!", notmuch_rb_threads_destroy, 0); /* in threads.c */ rb_define_method (notmuch_rb_cThreads, "each", notmuch_rb_threads_each, 0); /* in threads.c */ @@ -290,7 +321,7 @@ Init_notmuch (void) * * Notmuch messages */ - notmuch_rb_cMessages = rb_define_class_under (mod, "Messages", rb_cData); + notmuch_rb_cMessages = rb_define_class_under (mod, "Messages", rb_cObject); rb_undef_method (notmuch_rb_cMessages, "initialize"); rb_define_method (notmuch_rb_cMessages, "destroy!", notmuch_rb_messages_destroy, 0); /* in messages.c */ rb_define_method (notmuch_rb_cMessages, "each", notmuch_rb_messages_each, 0); /* in messages.c */ @@ -302,7 +333,7 @@ Init_notmuch (void) * * Notmuch thread */ - notmuch_rb_cThread = rb_define_class_under (mod, "Thread", rb_cData); + notmuch_rb_cThread = rb_define_class_under (mod, "Thread", rb_cObject); rb_undef_method (notmuch_rb_cThread, "initialize"); rb_define_method (notmuch_rb_cThread, "destroy!", notmuch_rb_thread_destroy, 0); /* in thread.c */ rb_define_method (notmuch_rb_cThread, "thread_id", notmuch_rb_thread_get_thread_id, 0); /* in thread.c */ @@ -321,7 +352,7 @@ Init_notmuch (void) * * Notmuch message */ - notmuch_rb_cMessage = rb_define_class_under (mod, "Message", rb_cData); + notmuch_rb_cMessage = rb_define_class_under (mod, "Message", rb_cObject); rb_undef_method (notmuch_rb_cMessage, "initialize"); rb_define_method (notmuch_rb_cMessage, "destroy!", notmuch_rb_message_destroy, 0); /* in message.c */ rb_define_method (notmuch_rb_cMessage, "message_id", notmuch_rb_message_get_message_id, 0); /* in message.c */ @@ -343,15 +374,4 @@ Init_notmuch (void) rb_define_method (notmuch_rb_cMessage, "tags_to_maildir_flags", notmuch_rb_message_tags_to_maildir_flags, 0); /* in message.c */ rb_define_method (notmuch_rb_cMessage, "freeze", notmuch_rb_message_freeze, 0); /* in message.c */ rb_define_method (notmuch_rb_cMessage, "thaw", notmuch_rb_message_thaw, 0); /* in message.c */ - - /* - * Document-class: Notmuch::Tags - * - * Notmuch tags - */ - notmuch_rb_cTags = rb_define_class_under (mod, "Tags", rb_cData); - rb_undef_method (notmuch_rb_cTags, "initialize"); - rb_define_method (notmuch_rb_cTags, "destroy!", notmuch_rb_tags_destroy, 0); /* in tags.c */ - rb_define_method (notmuch_rb_cTags, "each", notmuch_rb_tags_each, 0); /* in tags.c */ - rb_include_module (notmuch_rb_cTags, rb_mEnumerable); } diff --git a/bindings/ruby/message.c b/bindings/ruby/message.c index c55cf6e2..13c182f6 100644 --- a/bindings/ruby/message.c +++ b/bindings/ruby/message.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_message_destroy (VALUE self) { - notmuch_message_t *message; - - Data_Get_Notmuch_Message (self, message); - - notmuch_message_destroy (message); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_message_type); return Qnil; } @@ -89,7 +84,7 @@ notmuch_rb_message_get_replies (VALUE self) messages = notmuch_message_get_replies (message); - return Data_Wrap_Struct (notmuch_rb_cMessages, NULL, NULL, messages); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessages, ¬much_rb_messages_type, messages); } /* @@ -125,7 +120,7 @@ notmuch_rb_message_get_filenames (VALUE self) fnames = notmuch_message_get_filenames (message); - return Data_Wrap_Struct (notmuch_rb_cFileNames, NULL, NULL, fnames); + return notmuch_rb_filenames_get (fnames); } /* @@ -137,13 +132,18 @@ VALUE notmuch_rb_message_get_flag (VALUE self, VALUE flagv) { notmuch_message_t *message; + notmuch_bool_t is_set; + notmuch_status_t status; Data_Get_Notmuch_Message (self, message); if (!FIXNUM_P (flagv)) rb_raise (rb_eTypeError, "Flag not a Fixnum"); - return notmuch_message_get_flag (message, FIX2INT (flagv)) ? Qtrue : Qfalse; + status = notmuch_message_get_flag_st (message, FIX2INT (flagv), &is_set); + notmuch_rb_status_raise (status); + + return is_set ? Qtrue : Qfalse; } /* @@ -221,7 +221,7 @@ notmuch_rb_message_get_tags (VALUE self) if (!tags) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cTags, NULL, NULL, tags); + return notmuch_rb_tags_get (tags); } /* diff --git a/bindings/ruby/messages.c b/bindings/ruby/messages.c index a337feeb..6369d052 100644 --- a/bindings/ruby/messages.c +++ b/bindings/ruby/messages.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_messages_destroy (VALUE self) { - notmuch_messages_t *messages; - - Data_Get_Notmuch_Messages (self, messages); - - notmuch_messages_destroy (messages); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_messages_type); return Qnil; } @@ -53,7 +48,7 @@ notmuch_rb_messages_each (VALUE self) for (; notmuch_messages_valid (messages); notmuch_messages_move_to_next (messages)) { message = notmuch_messages_get (messages); - rb_yield (Data_Wrap_Struct (notmuch_rb_cMessage, NULL, NULL, message)); + rb_yield (Data_Wrap_Notmuch_Object (notmuch_rb_cMessage, ¬much_rb_message_type, message)); } return self; @@ -76,5 +71,5 @@ notmuch_rb_messages_collect_tags (VALUE self) if (!tags) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cTags, NULL, NULL, tags); + return notmuch_rb_tags_get (tags); } diff --git a/bindings/ruby/query.c b/bindings/ruby/query.c index 8b46d700..077def02 100644 --- a/bindings/ruby/query.c +++ b/bindings/ruby/query.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_query_destroy (VALUE self) { - notmuch_query_t *query; - - Data_Get_Notmuch_Query (self, query); - - notmuch_query_destroy (query); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_query_type); return Qnil; } @@ -50,7 +45,7 @@ notmuch_rb_query_get_sort (VALUE self) Data_Get_Notmuch_Query (self, query); - return FIX2INT (notmuch_query_get_sort (query)); + return INT2FIX (notmuch_query_get_sort (query)); } /* @@ -107,19 +102,21 @@ notmuch_rb_query_add_tag_exclude (VALUE self, VALUE tagv) } /* - * call-seq: QUERY.omit_excluded=(boolean) => nil + * call-seq: QUERY.omit_excluded=(fixnum) => nil * * Specify whether to omit excluded results or simply flag them. - * By default, this is set to +true+. + * By default, this is set to +Notmuch::EXCLUDE_TRUE+. */ VALUE notmuch_rb_query_set_omit_excluded (VALUE self, VALUE omitv) { notmuch_query_t *query; + notmuch_exclude_t value; Data_Get_Notmuch_Query (self, query); - notmuch_query_set_omit_excluded (query, RTEST (omitv)); + value = FIXNUM_P (omitv) ? FIX2UINT (omitv) : RTEST(omitv); + notmuch_query_set_omit_excluded (query, value); return Qnil; } @@ -142,7 +139,7 @@ notmuch_rb_query_search_threads (VALUE self) if (status) notmuch_rb_status_raise (status); - return Data_Wrap_Struct (notmuch_rb_cThreads, NULL, NULL, threads); + return Data_Wrap_Notmuch_Object (notmuch_rb_cThreads, ¬much_rb_threads_type, threads); } /* @@ -163,7 +160,7 @@ notmuch_rb_query_search_messages (VALUE self) if (status) notmuch_rb_status_raise (status); - return Data_Wrap_Struct (notmuch_rb_cMessages, NULL, NULL, messages); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessages, ¬much_rb_messages_type, messages); } /* diff --git a/bindings/ruby/tags.c b/bindings/ruby/tags.c index db8b4cfc..b64874d1 100644 --- a/bindings/ruby/tags.c +++ b/bindings/ruby/tags.c @@ -1,61 +1,13 @@ -/* The Ruby interface to the notmuch mail library - * - * Copyright © 2010, 2011 Ali Polatel - * - * This program is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see https://www.gnu.org/licenses/ . - * - * Author: Ali Polatel - */ - #include "defs.h" -/* - * call-seq: TAGS.destroy! => nil - * - * Destroys the tags, freeing all resources allocated for it. - */ -VALUE -notmuch_rb_tags_destroy (VALUE self) -{ - notmuch_tags_t *tags; - - Data_Get_Notmuch_Tags (self, tags); - - notmuch_tags_destroy (tags); - DATA_PTR (self) = NULL; - - return Qnil; -} - -/* - * call-seq: TAGS.each {|item| block } => TAGS - * - * Calls +block+ once for each element in +self+, passing that element as a - * parameter. - */ VALUE -notmuch_rb_tags_each (VALUE self) +notmuch_rb_tags_get (notmuch_tags_t *tags) { - const char *tag; - notmuch_tags_t *tags; - - Data_Get_Notmuch_Tags (self, tags); + VALUE rb_array = rb_ary_new (); for (; notmuch_tags_valid (tags); notmuch_tags_move_to_next (tags)) { - tag = notmuch_tags_get (tags); - rb_yield (rb_str_new2 (tag)); + const char *tag = notmuch_tags_get (tags); + rb_ary_push (rb_array, rb_str_new2 (tag)); } - - return self; + return rb_array; } diff --git a/bindings/ruby/thread.c b/bindings/ruby/thread.c index 9b295981..b20ed893 100644 --- a/bindings/ruby/thread.c +++ b/bindings/ruby/thread.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_thread_destroy (VALUE self) { - notmuch_thread_t *thread; - - Data_Get_Notmuch_Thread (self, thread); - - notmuch_thread_destroy (thread); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_thread_type); return Qnil; } @@ -88,7 +83,7 @@ notmuch_rb_thread_get_toplevel_messages (VALUE self) if (!messages) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cMessages, NULL, NULL, messages); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessages, ¬much_rb_messages_type, messages); } /* @@ -108,7 +103,7 @@ notmuch_rb_thread_get_messages (VALUE self) if (!messages) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cMessages, NULL, NULL, messages); + return Data_Wrap_Notmuch_Object (notmuch_rb_cMessages, ¬much_rb_messages_type, messages); } /* @@ -209,5 +204,5 @@ notmuch_rb_thread_get_tags (VALUE self) if (!tags) rb_raise (notmuch_rb_eMemoryError, "Out of memory"); - return Data_Wrap_Struct (notmuch_rb_cTags, NULL, NULL, tags); + return notmuch_rb_tags_get (tags); } diff --git a/bindings/ruby/threads.c b/bindings/ruby/threads.c index ed403a8f..50280260 100644 --- a/bindings/ruby/threads.c +++ b/bindings/ruby/threads.c @@ -28,12 +28,7 @@ VALUE notmuch_rb_threads_destroy (VALUE self) { - notmuch_threads_t *threads; - - Data_Get_Struct (self, notmuch_threads_t, threads); - - notmuch_threads_destroy (threads); - DATA_PTR (self) = NULL; + notmuch_rb_object_destroy (self, ¬much_rb_threads_type); return Qnil; } @@ -53,7 +48,7 @@ notmuch_rb_threads_each (VALUE self) for (; notmuch_threads_valid (threads); notmuch_threads_move_to_next (threads)) { thread = notmuch_threads_get (threads); - rb_yield (Data_Wrap_Struct (notmuch_rb_cThread, NULL, NULL, thread)); + rb_yield (Data_Wrap_Notmuch_Object (notmuch_rb_cThread, ¬much_rb_thread_type, thread)); } return self; diff --git a/command-line-arguments.c b/command-line-arguments.c index 169b12a3..5dea8281 100644 --- a/command-line-arguments.c +++ b/command-line-arguments.c @@ -47,17 +47,23 @@ _process_keyword_arg (const notmuch_opt_desc_t *arg_desc, char next, continue; *arg_desc->opt_keyword = keywords->value; - fprintf (stderr, "Warning: No known keyword option given for \"%s\", choosing value \"%s\"." - " Please specify the argument explicitly!\n", arg_desc->name, arg_desc->keyword_no_arg_value); + fprintf (stderr, + "Warning: No known keyword option given for \"%s\", choosing value \"%s\"." + " Please specify the argument explicitly!\n", arg_desc->name, + arg_desc->keyword_no_arg_value); return OPT_GIVEBACK; } - fprintf (stderr, "No matching keyword for option \"%s\" and default value \"%s\" is invalid.\n", arg_str, arg_desc->name); + fprintf (stderr, + "No matching keyword for option \"%s\" and default value \"%s\" is invalid.\n", + arg_str, + arg_desc->name); return OPT_FAILED; } if (next != '\0') - fprintf (stderr, "Unknown keyword argument \"%s\" for option \"%s\".\n", arg_str, arg_desc->name); + fprintf (stderr, "Unknown keyword argument \"%s\" for option \"%s\".\n", arg_str, + arg_desc->name); else fprintf (stderr, "Option \"%s\" needs a keyword argument.\n", arg_desc->name); return OPT_FAILED; @@ -74,7 +80,8 @@ _process_boolean_arg (const notmuch_opt_desc_t *arg_desc, char next, } else if (strcmp (arg_str, "false") == 0) { value = false; } else { - fprintf (stderr, "Unknown argument \"%s\" for (boolean) option \"%s\".\n", arg_str, arg_desc->name); + fprintf (stderr, "Unknown argument \"%s\" for (boolean) option \"%s\".\n", arg_str, + arg_desc->name); return OPT_FAILED; } @@ -202,6 +209,7 @@ parse_option (int argc, char **argv, const notmuch_opt_desc_t *options, int opt_ const notmuch_opt_desc_t *try; const char *next_arg = NULL; + if (opt_index < argc - 1 && strncmp (argv[opt_index + 1], "--", 2) != 0) next_arg = argv[opt_index + 1]; diff --git a/compat/Makefile.local b/compat/Makefile.local index bcb9f0ec..c58ca746 100644 --- a/compat/Makefile.local +++ b/compat/Makefile.local @@ -1,14 +1,10 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := compat extra_cflags += -I$(srcdir)/$(dir) notmuch_compat_srcs := -ifneq ($(HAVE_CANONICALIZE_FILE_NAME),1) -notmuch_compat_srcs += $(dir)/canonicalize_file_name.c -endif - ifneq ($(HAVE_GETLINE),1) notmuch_compat_srcs += $(dir)/getline.c $(dir)/getdelim.c endif diff --git a/compat/canonicalize_file_name.c b/compat/canonicalize_file_name.c deleted file mode 100644 index 000f9e78..00000000 --- a/compat/canonicalize_file_name.c +++ /dev/null @@ -1,18 +0,0 @@ -#include "compat.h" -#include -#undef _GNU_SOURCE -#include - -char * -canonicalize_file_name (const char *path) -{ -#ifdef PATH_MAX - char *resolved_path = malloc (PATH_MAX + 1); - if (resolved_path == NULL) - return NULL; - - return realpath (path, resolved_path); -#else -#error undefined PATH_MAX _and_ missing canonicalize_file_name not supported -#endif -} diff --git a/compat/compat.h b/compat/compat.h index 8f15e585..59e91618 100644 --- a/compat/compat.h +++ b/compat/compat.h @@ -37,14 +37,6 @@ extern "C" { #define _POSIX_PTHREAD_SEMANTICS 1 #endif -#if ! HAVE_CANONICALIZE_FILE_NAME -/* we only call this function from C, and this makes testing easier */ -#ifndef __cplusplus -char * -canonicalize_file_name (const char *path); -#endif -#endif - #if ! HAVE_GETLINE #include #include diff --git a/compat/have_strcasestr.c b/compat/have_strcasestr.c index 3cd1838d..8e004572 100644 --- a/compat/have_strcasestr.c +++ b/compat/have_strcasestr.c @@ -1,5 +1,6 @@ #define _GNU_SOURCE -#include +#include /* strcasecmp() in POSIX */ +#include /* strcasecmp() in *BSD */ int main () diff --git a/completion/Makefile.local b/completion/Makefile.local index 8e86c9d2..54df463c 100644 --- a/completion/Makefile.local +++ b/completion/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := completion diff --git a/completion/notmuch-completion.bash b/completion/notmuch-completion.bash index 15425697..3748846e 100644 --- a/completion/notmuch-completion.bash +++ b/completion/notmuch-completion.bash @@ -103,12 +103,12 @@ _notmuch_search_terms() COMPREPLY=( $(compgen -P "from:" -W "`_notmuch_email ${cur}`" -- ${cur##from:}) ) ;; path:*) - local path=`notmuch config get database.path` + local path=`notmuch config get database.mail_root` compopt -o nospace COMPREPLY=( $(compgen -d "$path/${cur##path:}" | sed "s|^$path/||" ) ) ;; folder:*) - local path=`notmuch config get database.path` + local path=`notmuch config get database.mail_root` compopt -o nospace COMPREPLY=( $(compgen -d "$path/${cur##folder:}" | \ sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) ) @@ -281,7 +281,7 @@ _notmuch_insert() $split && case "${prev}" in --folder) - local path=`notmuch config get database.path` + local path=`notmuch config get database.mail_root` compopt -o nospace COMPREPLY=( $(compgen -d "$path/${cur}" | \ sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) ) @@ -530,7 +530,7 @@ _notmuch_show() ! $split && case "${cur}" in -*) - local options="--entire-thread= --format= --exclude= --body= --format-version= --part= --verify --decrypt= --include-html ${_notmuch_shared_options}" + local options="--entire-thread= --format= --exclude= --body= --format-version= --part= --verify --decrypt= --include-html --limit= --offset= ${_notmuch_shared_options}" compopt -o nospace COMPREPLY=( $(compgen -W "$options" -- ${cur}) ) ;; diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch index e920f10b..0bdd7f77 100644 --- a/completion/zsh/_notmuch +++ b/completion/zsh/_notmuch @@ -69,8 +69,8 @@ _notmuch_term_mimetype() { _notmuch_term_path() { local ret=1 expl - local maildir="$(notmuch config get database.path)" - [[ -d $maildir ]] || { _message -e "database.path not found" ; return $ret } + local maildir="$(notmuch config get database.mail_root)" + [[ -d $maildir ]] || { _message -e "database.mail_root not found" ; return $ret } _description notmuch-folder expl 'maildir folder' _files "$expl[@]" -W $maildir -/ && ret=0 @@ -79,8 +79,8 @@ _notmuch_term_path() { _notmuch_term_folder() { local ret=1 expl - local maildir="$(notmuch config get database.path)" - [[ -d $maildir ]] || { _message -e "database.path not found" ; return $ret } + local maildir="$(notmuch config get database.mail_root)" + [[ -d $maildir ]] || { _message -e "database.mail_root not found" ; return $ret } _description notmuch-folder expl 'maildir folder' local ignoredfolders=( '*/(cur|new|tmp)' ) @@ -245,6 +245,8 @@ _notmuch_show() { '--exclude=[respect excluded tags setting]:exclude tags:(true false)' \ '--body=[output body]:output body content:(true false)' \ '--include-html[include text/html parts in the output]' \ + '--limit=[limit the number of displayed results]:limit: ' \ + '--offset=[skip displaying the first N results]:offset: ' \ '*::search term:_notmuch_search_term' } diff --git a/configure b/configure index 3c148e12..7afd08c7 100755 --- a/configure +++ b/configure @@ -48,13 +48,15 @@ case $PWD in ( *["$IFS"]* ) esac subdirs="util compat lib parse-time-string completion doc emacs" -subdirs="${subdirs} performance-test test test/test-databases" +subdirs="${subdirs} performance-test test" subdirs="${subdirs} bindings" # For a non-srcdir configure invocation (such as ../configure), create # the directory structure and copy Makefiles. if [ "$srcdir" != "." ]; then + NOTMUCH_BUILDDIR=$PWD + for dir in . ${subdirs}; do mkdir -p "$dir" cp "$srcdir"/"$dir"/Makefile.local "$dir" @@ -70,6 +72,16 @@ if [ "$srcdir" != "." ]; then mkdir bindings/ruby cp -a "$srcdir"/bindings/ruby/*.[ch] bindings/ruby cp -a "$srcdir"/bindings/ruby/extconf.rb bindings/ruby + + # Use the same hack to replicate python-cffi source for + # out-of-tree builds (again, not ideal). + mkdir bindings/python-cffi + cp -a "$srcdir"/bindings/python-cffi/tests \ + "$srcdir"/bindings/python-cffi/notmuch2 \ + "$srcdir"/bindings/python-cffi/setup.py \ + bindings/python-cffi/ +else + NOTMUCH_BUILDDIR=$NOTMUCH_SRCDIR fi # Set several defaults (optionally specified by the user in @@ -100,6 +112,7 @@ PREFIX=/usr/local LIBDIR= WITH_DOCS=1 WITH_API_DOCS=1 +WITH_PYTHON_DOCS=1 WITH_EMACS=1 WITH_DESKTOP=1 WITH_BASH=1 @@ -168,7 +181,7 @@ Fine tuning of some installation directories is available: --emacslispdir=DIR Emacs code [PREFIX/share/emacs/site-lisp] --emacsetcdir=DIR Emacs miscellaneous files [PREFIX/share/emacs/site-lisp] --bashcompletiondir=DIR Bash completions files [PREFIX/share/bash-completion/completions] - --zshcompletiondir=DIR Zsh completions files [PREFIX/share/zsh/functions/Completion/Unix] + --zshcompletiondir=DIR Zsh completions files [PREFIX/share/zsh/site-functions] Some features can be disabled (--with-feature=no is equivalent to --without-feature) : @@ -299,12 +312,22 @@ for option; do true elif [ "${option%%=*}" = '--host' ] ; then true + elif [ "${option%%=*}" = '--bindir' ] ; then + true + elif [ "${option%%=*}" = '--sbindir' ] ; then + true elif [ "${option%%=*}" = '--datadir' ] ; then true elif [ "${option%%=*}" = '--localstatedir' ] ; then true + elif [ "${option%%=*}" = '--sharedstatedir' ] ; then + true elif [ "${option%%=*}" = '--libexecdir' ] ; then true + elif [ "${option%%=*}" = '--exec-prefix' ] ; then + true + elif [ "${option%%=*}" = '--program-prefix' ] ; then + true elif [ "${option}" = '--disable-maintainer-mode' ] ; then true elif [ "${option}" = '--disable-dependency-tracking' ] ; then @@ -387,6 +410,30 @@ EOF exit 1 fi +printf "C compiler supports address sanitizer... " +test_cmdline="${CC} ${CFLAGS} ${CPPFLAGS} -fsanitize=address minimal.c ${LDFLAGS} -o minimal" +if ${test_cmdline} >/dev/null 2>&1 && ./minimal +then + printf "Yes.\n" + have_asan=1 +else + printf "Nope, skipping those tests.\n" + have_asan=0 +fi +unset test_cmdline + +printf "C compiler supports thread sanitizer... " +test_cmdline="${CC} ${CFLAGS} ${CPPFLAGS} -fsanitize=thread minimal.c ${LDFLAGS} -o minimal" +if ${test_cmdline} >/dev/null 2>&1 && ./minimal +then + printf "Yes.\n" + have_tsan=1 +else + printf "Nope, skipping those tests.\n" + have_tsan=0 +fi +unset test_cmdline + printf "Reading libnotmuch version from source... " cat > _libversion.c < @@ -422,15 +469,22 @@ else have_pkg_config=0 fi -printf "Checking for Xapian development files... " + + +printf "Checking for Xapian development files (>= 1.4.0)... " have_xapian=0 -for xapian_config in ${XAPIAN_CONFIG} xapian-config xapian-config-1.3; do +for xapian_config in ${XAPIAN_CONFIG} xapian-config; do if ${xapian_config} --version > /dev/null 2>&1; then xapian_version=$(${xapian_config} --version | sed -e 's/.* //') - printf "Yes (%s).\n" ${xapian_version} - have_xapian=1 - xapian_cxxflags=$(${xapian_config} --cxxflags) - xapian_ldflags=$(${xapian_config} --libs) + case $xapian_version in + 1.[4-9]* | 1.[1-9][0-9]* | [2-9]* | [1-9][0-9]*) + printf "Yes (%s).\n" ${xapian_version} + have_xapian=1 + xapian_cxxflags=$(${xapian_config} --cxxflags) + xapian_ldflags=$(${xapian_config} --libs) + ;; + *) printf "Xapian $xapian_version not supported... " + esac break fi done @@ -439,81 +493,10 @@ if [ ${have_xapian} = "0" ]; then errors=$((errors + 1)) fi -have_xapian_compact=0 -have_xapian_field_processor=0 -if [ ${have_xapian} = "1" ]; then - printf "Checking for Xapian compaction support... " - cat>_compact.cc< -class TestCompactor : public Xapian::Compactor { }; -EOF - if ${CXX} ${CXXFLAGS_for_sh} ${xapian_cxxflags} -c _compact.cc -o _compact.o > /dev/null 2>&1 - then - have_xapian_compact=1 - printf "Yes.\n" - else - printf "No.\n" - errors=$((errors + 1)) - fi - - rm -f _compact.o _compact.cc - - printf "Checking for Xapian FieldProcessor API... " - cat>_field_processor.cc< -class TitleFieldProcessor : public Xapian::FieldProcessor { }; -EOF - if ${CXX} ${CXXFLAGS_for_sh} ${xapian_cxxflags} -c _field_processor.cc -o _field_processor.o > /dev/null 2>&1 - then - have_xapian_field_processor=1 - printf "Yes.\n" - else - printf "No. (optional)\n" - fi - - rm -f _field_processor.o _field_processor.cc - - default_xapian_backend="" - # DB_RETRY_LOCK is only supported on Xapian > 1.3.2 - have_xapian_db_retry_lock=0 - if [ $WITH_RETRY_LOCK = "1" ]; then - printf "Checking for Xapian lock retry support... " - cat>_retry.cc< -int flag = Xapian::DB_RETRY_LOCK; -EOF - if ${CXX} ${CXXFLAGS_for_sh} ${xapian_cxxflags} -c _retry.cc -o _retry.o > /dev/null 2>&1 - then - have_xapian_db_retry_lock=1 - printf "Yes.\n" - else - printf "No. (optional)\n" - fi - rm -f _retry.o _retry.cc - fi - - printf "Testing default Xapian backend... " - cat >_default_backend.cc < -int main(int argc, char** argv) { - Xapian::WritableDatabase db("test.db",Xapian::DB_CREATE_OR_OPEN); -} -EOF - ${CXX} ${CXXFLAGS_for_sh} ${xapian_cxxflags} _default_backend.cc -o _default_backend ${xapian_ldflags} - ./_default_backend - if [ -f test.db/iamglass ]; then - default_xapian_backend=glass - else - default_xapian_backend=chert - fi - printf "%s\n" "${default_xapian_backend}"; - rm -rf test.db _default_backend _default_backend.cc -fi - GMIME_MINVER=3.0.3 -printf "Checking for GMime development files... " -if pkg-config --exists "gmime-3.0 > $GMIME_MINVER"; then +printf "Checking for GMime development files (>= $GMIME_MINVER)... " +if pkg-config --exists "gmime-3.0 >= $GMIME_MINVER"; then printf "Yes.\n" have_gmime=1 gmime_cflags=$(pkg-config --cflags gmime-3.0) @@ -538,7 +521,7 @@ int main () { if (error) return !! fprintf (stderr, "failed to instantiate parser with test/corpora/crypto/basic-encrypted.eml\n"); body = GMIME_MULTIPART_ENCRYPTED(g_mime_message_get_mime_part (g_mime_parser_construct_message (parser, NULL))); - if (body == NULL) return !! fprintf (stderr, "did not find a multipart encrypted message\n"); + if (body == NULL) return !! fprintf (stderr, "did not find a multipart encrypted message\n"); output = g_mime_multipart_encrypted_decrypt (body, GMIME_DECRYPT_EXPORT_SESSION_KEY, NULL, &decrypt_result, &error); if (error || output == NULL) return !! fprintf (stderr, "decryption failed\n"); @@ -551,16 +534,16 @@ int main () { } EOF if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XXXXXX"); then - printf 'No.\nCould not make tempdir for testing session-key support.\n' - errors=$((errors + 1)) + printf 'No.\nCould not make tempdir for testing session-key support.\n' + errors=$((errors + 1)) elif ${CC} ${CFLAGS} ${gmime_cflags} _check_session_keys.c ${gmime_ldflags} -o _check_session_keys \ - && GNUPGHOME=${TEMP_GPG} gpg --batch --quiet --import < "$srcdir"/test/gnupg-secret-key.asc \ - && SESSION_KEY=$(GNUPGHOME=${TEMP_GPG} ./_check_session_keys) \ - && [ $SESSION_KEY = 9:0BACD64099D1468AB07C796F0C0AC4851948A658A15B34E803865E9FC635F2F5 ] + && GNUPGHOME=${TEMP_GPG} gpg --batch --quiet --import < "$srcdir"/test/openpgp4-secret-key.asc \ + && SESSION_KEY=$(GNUPGHOME=${TEMP_GPG} ./_check_session_keys) \ + && [ $SESSION_KEY = 9:496A0B6D15A5E7BA762FB8E5FE6DEE421D4D9BBFCEAD1CDD0CCF636D07ADE621 ] then - printf "OK.\n" + printf "OK.\n" else - cat </dev/null; then - printf 'Your current GPGME development version is: %s\n' "$(gpgme-config --version)" - else - printf 'You do not have the GPGME development libraries installed.\n' - fi - errors=$((errors + 1)) + if GPGME_VERS="$(pkg-config --modversion gpgme || gpgme-config --version)"; then + printf 'Your current GPGME development version is: %s\n' "$GPGME_VERS" + else + printf 'You do not have the GPGME development libraries installed.\n' + fi + errors=$((errors + 1)) + fi + if [ -n "$TEMP_GPG" -a -d "$TEMP_GPG" ]; then + rm -rf "$TEMP_GPG" + fi + + cat > _check_gmime_cert.c < +#include + +int main () { + GError *error = NULL; + GMimeParser *parser = NULL; + GMimeApplicationPkcs7Mime *body = NULL; + GMimeSignatureList *sig_list = NULL; + GMimeSignature *sig = NULL; + GMimeCertificate *cert = NULL; + GMimeObject *output = NULL; + int len; + + g_mime_init (); + parser = g_mime_parser_new (); + g_mime_parser_init_with_stream (parser, g_mime_stream_file_open("$srcdir/test/corpora/pkcs7/smime-onepart-signed.eml", "r", &error)); + if (error) return !! fprintf (stderr, "failed to instantiate parser with test/corpora/pkcs7/smime-onepart-signed.eml\n"); + + body = GMIME_APPLICATION_PKCS7_MIME(g_mime_message_get_mime_part (g_mime_parser_construct_message (parser, NULL))); + if (body == NULL) return !! fprintf (stderr, "did not find a application/pkcs7 message\n"); + + sig_list = g_mime_application_pkcs7_mime_verify (body, GMIME_VERIFY_NONE, &output, &error); + if (error || output == NULL) return !! fprintf (stderr, "verify failed\n"); + + if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList found\n"); + len = g_mime_signature_list_length (sig_list); + if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", len); + sig = g_mime_signature_list_get_signature (sig_list, 0); + if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at position 0\n"); + cert = g_mime_signature_get_certificate (sig); + if (cert == NULL) return !! fprintf (stderr, "no GMimeCertificate found\n"); +#ifdef CHECK_VALIDITY + GMimeValidity validity = g_mime_certificate_get_id_validity (cert); + if (validity != GMIME_VALIDITY_FULL) return !! fprintf (stderr, "Got validity %d, expected %d\n", validity, GMIME_VALIDITY_FULL); +#endif +#ifdef CHECK_EMAIL + const char *email = g_mime_certificate_get_email (cert); + if (! email) return !! fprintf (stderr, "no email returned"); + if (email[0] == '<') return 2; +#endif + return 0; +} +EOF + + # see https://github.com/jstedfast/gmime/pull/90 + # should be fixed in GMime in 3.2.7, but some distros might patch + printf "Checking for GMime X.509 certificate validity... " + + if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XXXXXX"); then + printf 'No.\nCould not make tempdir for testing X.509 certificate validity support.\n' + errors=$((errors + 1)) + elif ${CC} -DCHECK_VALIDITY ${CFLAGS} ${gmime_cflags} _check_gmime_cert.c ${gmime_ldflags} -o _check_x509_validity \ + && echo disable-crl-checks > "$TEMP_GPG/gpgsm.conf" \ + && echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> "$TEMP_GPG/trustlist.txt" \ + && GNUPGHOME=${TEMP_GPG} gpgsm --batch --quiet --import < "$srcdir"/test/smime/ca.crt + then + if GNUPGHOME=${TEMP_GPG} ./_check_x509_validity; then + gmime_x509_cert_validity=1 + printf "Yes.\n" + else + gmime_x509_cert_validity=0 + printf "No.\n" + if pkg-config --exists "gmime-3.0 >= 3.2.7"; then + cat < _verify_sig_with_session_key.c < +#include + +int main () { + GError *error = NULL; + GMimeParser *parser = NULL; + GMimeMultipartEncrypted *body = NULL; + GMimeDecryptResult *result = NULL; + GMimeSignatureList *sig_list = NULL; + GMimeSignature *sig = NULL; + GMimeObject *output = NULL; + GMimeSignatureStatus status; + int len; + + g_mime_init (); + parser = g_mime_parser_new (); + g_mime_parser_init_with_stream (parser, g_mime_stream_file_open("$srcdir/test/corpora/crypto/encrypted-signed.eml", "r", &error)); + if (error) return !! fprintf (stderr, "failed to instantiate parser with test/corpora/pkcs7/smime-onepart-signed.eml\n"); + + body = GMIME_MULTIPART_ENCRYPTED(g_mime_message_get_mime_part (g_mime_parser_construct_message (parser, NULL))); + if (body == NULL) return !! fprintf (stderr, "did not find a multipart/encrypted message\n"); + + output = g_mime_multipart_encrypted_decrypt (body, GMIME_DECRYPT_NONE, "9:9E1CDF53BBF794EA34F894B5B68E1E56FB015EA69F81D2A5EAB7F96C7B65783E", &result, &error); + if (error || output == NULL) return !! fprintf (stderr, "decrypt failed\n"); + + sig_list = g_mime_decrypt_result_get_signatures (result); + if (sig_list == NULL) return !! fprintf (stderr, "sig_list is NULL\n"); + + if (sig_list == NULL) return !! fprintf (stderr, "no GMimeSignatureList found\n"); + len = g_mime_signature_list_length (sig_list); + if (len != 1) return !! fprintf (stderr, "expected 1 signature, got %d\n", len); + sig = g_mime_signature_list_get_signature (sig_list, 0); + if (sig == NULL) return !! fprintf (stderr, "no GMimeSignature found at position 0\n"); + status = g_mime_signature_get_status (sig); + if (status & GMIME_SIGNATURE_STATUS_KEY_MISSING) return !! fprintf (stderr, "signature status contains KEY_MISSING (see https://dev.gnupg.org/T3464)\n"); + + return 0; +} +EOF + if ! TEMP_GPG=$(mktemp -d "${TMPDIR:-/tmp}/notmuch.XXXXXX"); then + printf 'No.\nCould not make tempdir for testing signature verification when decrypting with session keys.\n' + errors=$((errors + 1)) + elif ${CC} ${CFLAGS} ${gmime_cflags} _verify_sig_with_session_key.c ${gmime_ldflags} -o _verify_sig_with_session_key \ + && GNUPGHOME=${TEMP_GPG} gpg --batch --quiet --import < "$srcdir"/test/openpgp4-secret-key.asc \ + && rm -f ${TEMP_GPG}/private-keys-v1.d/*.key + then + if GNUPGHOME=${TEMP_GPG} ./_verify_sig_with_session_key; then + gmime_verify_with_session_key=1 + printf "Yes.\n" + else + gmime_verify_with_session_key=0 + printf "No.\n" + cat </dev/null 2>&1 && compat/gen_zlib_pc > compat/zlib.pc && - PKG_CONFIG_PATH="$PKG_CONFIG_PATH":compat && + PKG_CONFIG_PATH=${PKG_CONFIG_PATH:+$PKG_CONFIG_PATH:}compat && export PKG_CONFIG_PATH rm -f compat/gen_zlib_pc fi @@ -641,6 +787,7 @@ if command -v ${BASHCMD} > /dev/null; then printf "Yes (%s).\n" "$bash_absolute" else have_bash=0 + bash_absolute= printf "No. (%s not found)\n" "${BASHCMD}" fi @@ -651,6 +798,7 @@ if command -v ${PERL} > /dev/null; then printf "Yes (%s).\n" "$perl_absolute" else have_perl=0 + perl_absolute= printf "No. (%s not found)\n" "${PERL}" fi @@ -671,6 +819,59 @@ if [ $have_python -eq 0 ]; then errors=$((errors + 1)) fi +have_python3=0 +if [ $have_python -eq 1 ]; then + printf "Checking for python3 (>= 3.5)..." + if "$python" -c 'import sys, sysconfig; assert sys.version_info >= (3,5)'; >/dev/null 2>&1; then + printf "Yes.\n" + have_python3=1 + else + printf "No (will not install CFFI-based python bindings).\n" + fi +fi + +have_python3_dev=0 +if [ $have_python3 -eq 1 ]; then + printf "Checking for python3 version ..." + python3_version=$("$python" -c 'import sysconfig; print(sysconfig.get_python_version());') + printf "(%s)\n" $python3_version + + printf "Checking for python $python3_version development files..." + if pkg-config --exists "python-$python3_version"; then + have_python3_dev=1 + printf "Yes.\n" + else + have_python3_dev=0 + printf "No (will not install CFFI-based python bindings).\n" + fi +fi + +have_python3_cffi=0 +have_python3_pytest=0 +if [ $have_python3_dev -eq 1 ]; then + printf "Checking for python3 cffi and setuptools... " + if "$python" -c 'import cffi,setuptools; cffi.FFI().verify()' >/dev/null 2>&1; then + printf "Yes.\n" + have_python3_cffi=1 + WITH_PYTHON_DOCS=1 + else + WITH_PYTHON_DOCS=0 + printf "No (will not install CFFI-based python bindings).\n" + fi + rm -rf __pycache__ # cffi.FFI().verify() uses this space + + printf "Checking for python3 pytest (>= 3.0)... " + conf=$(mktemp) + printf "[pytest]\nminversion=3.0\n" > $conf + if "$python" -m pytest -c $conf --version >/dev/null 2>&1; then + printf "Yes.\n" + have_python3_pytest=1 + else + printf "No (will not test CFFI-based python bindings).\n" + fi + rm -f $conf +fi + printf "Checking for valgrind development files... " if pkg-config --exists valgrind; then printf "Yes.\n" @@ -690,6 +891,19 @@ else WITH_BASH=0 fi +printf "Checking for sfsexp... " +if pkg-config --exists sfsexp; then + printf "Yes.\n" + have_sfsexp=1 + sfsexp_cflags=$(pkg-config --cflags sfsexp) + sfsexp_ldflags=$(pkg-config --libs sfsexp) +else + printf "No (will not enable s-expression queries).\n" + have_sfsexp=0 + sfsexp_cflags= + sfsexp_ldflags= +fi + if [ -z "${EMACSLISPDIR-}" ]; then EMACSLISPDIR="\$(prefix)/share/emacs/site-lisp" fi @@ -699,12 +913,12 @@ if [ -z "${EMACSETCDIR-}" ]; then fi if [ $WITH_EMACS = "1" ]; then - printf "Checking if emacs (>= 24) is available... " - if emacs --quick --batch --eval '(if (< emacs-major-version 24) (kill-emacs 1))' > /dev/null 2>&1; then - printf "Yes.\n" + printf "Checking if emacs (>= 25) is available... " + if emacs --quick --batch --eval '(if (< emacs-major-version 25) (kill-emacs 1))' > /dev/null 2>&1; then + printf "Yes.\n" else - printf "No (disabling emacs related parts of build)\n" - WITH_EMACS=0 + printf "No (disabling emacs related parts of build)\n" + WITH_EMACS=0 fi fi @@ -845,8 +1059,8 @@ EOF if [ $have_python -eq 0 ]; then echo " python interpreter" fi - if [ $have_xapian -eq 0 -o $have_xapian_compact -eq 0 ]; then - echo " Xapian library (>= version 1.2.6, including development files such as headers)" + if [ $have_xapian -eq 0 ]; then + echo " Xapian library (>= version 1.4.0, including development files such as headers)" echo " https://xapian.org/" fi if [ $have_zlib -eq 0 ]; then @@ -881,7 +1095,7 @@ On Debian and similar systems: Or on Fedora and similar systems: - sudo yum install xapian-core-devel gmime-devel libtalloc-devel zlib-devel + sudo dnf install xapian-core-devel gmime30-devel libtalloc-devel zlib-devel On other systems, similar commands can be used, but the details of the package names may be different. @@ -896,7 +1110,7 @@ to install pkg-config with a command such as: sudo apt-get install pkg-config Or: - sudo yum install pkgconfig + sudo dnf install pkgconfig But if pkg-config is not available for your system, then you will need to modify the configure script to manually set the cflags and ldflags @@ -970,6 +1184,22 @@ else fi rm -f compat/have_timegm +cat < _time_t.c +#include +#include +static_assert(sizeof(time_t) >= 8, "sizeof(time_t) < 8"); +EOF + +printf "Checking for 64 bit time_t... " +if ${CC} -c _time_t.c -o /dev/null +then + printf "Yes.\n" + have_64bit_time_t=1 +else + printf "No.\n" + have_64bit_time_t=0 +fi + printf "Checking for dirent.d_type... " if ${CC} -o compat/have_d_type "$srcdir"/compat/have_d_type.c > /dev/null 2>&1 then @@ -1053,7 +1283,8 @@ for flag in -Wmissing-declarations; do done printf "\n\t%s\n" "${WARN_CFLAGS}" -rm -f minimal minimal.c _libversion.c _libversion _libversion.sh _check_session_keys.c _check_session_keys +rm -f minimal minimal.c _time_t.c _libversion.c _libversion _libversion.sh _check_session_keys.c _check_session_keys _check_gmime_cert.c _check_x509_validity _check_email \ + _verify_sig_with_session_key.c _verify_sig_with_session_key # construct the Makefile.config cat > Makefile.config < Makefile.config < sh.config <) +NOTMUCH_GMIME_EMITS_ANGLE_BRACKETS=${gmime_emits_angle_brackets} -# Whether the Xapian version in use supports field processors -NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR=${have_xapian_field_processor} +# Whether GMime can verify signatures when decrypting with a session key: +NOTMUCH_GMIME_VERIFY_WITH_SESSION_KEY=${gmime_verify_with_session_key} -# Whether the Xapian version in use supports lock retry -NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK=${have_xapian_db_retry_lock} +# Flags needed to compile and link against zlib +NOTMUCH_ZLIB_CFLAGS="${zlib_cflags}" +NOTMUCH_ZLIB_LDFLAGS="${zlib_ldflags}" -# Which backend will Xapian use by default? -NOTMUCH_DEFAULT_XAPIAN_BACKEND=${default_xapian_backend} +# Does the C compiler support the sanitizers +NOTMUCH_HAVE_ASAN=${have_asan} +NOTMUCH_HAVE_TSAN=${have_tsan} # do we have man pages? NOTMUCH_HAVE_MAN=$((have_sphinx)) @@ -1362,6 +1613,9 @@ NOTMUCH_HAVE_MAN=$((have_sphinx)) NOTMUCH_HAVE_BASH=${have_bash} NOTMUCH_BASH_ABSOLUTE=${bash_absolute} +# Whether time_t is 64 bits (or more) +NOTMUCH_HAVE_64BIT_TIME_T=${have_64bit_time_t} + # Whether perl exists, and if so where NOTMUCH_HAVE_PERL=${have_perl} NOTMUCH_PERL_ABSOLUTE=${perl_absolute} @@ -1376,10 +1630,42 @@ NOTMUCH_RUBY=${RUBY} # building/testing ruby bindings. NOTMUCH_HAVE_RUBY_DEV=${have_ruby_dev} +# Is the python cffi package available? +NOTMUCH_HAVE_PYTHON3_CFFI=${have_python3_cffi} + +# Is the python pytest package available? +NOTMUCH_HAVE_PYTHON3_PYTEST=${have_python3_pytest} + +# Is the sfsexp library available? +NOTMUCH_HAVE_SFSEXP=${have_sfsexp} + +# And if so, flags needed at compile/link time for sfsexp +NOTMUCH_SFSEXP_CFLAGS="${sfsexp_cflags}" +NOTMUCH_SFSEXP_LDFLAGS="${sfsexp_ldflags}" + # Platform we are run on PLATFORM=${platform} EOF +{ + echo "# Generated by configure, run from doc/conf.py" + if [ $WITH_EMACS = "1" ]; then + echo "tags.add('WITH_EMACS')" + fi + if [ $WITH_PYTHON_DOCS = "1" ]; then + echo "tags.add('WITH_PYTHON')" + fi + printf "rsti_dir = '%s'\n" "$(cd emacs && pwd -P)" +} > sphinx.config + +cat > bindings/python-cffi/_notmuch_config.py < $@ install: all - mkdir -p $(DESTDIR)$(prefix)/bin + mkdir -p $(DESTDIR)$(prefix)/bin $(DESTDIR)$(mandir)/man1 $(DESTDIR)$(sysconfdir)/Muttrc.d sed "1s|^#!.*|#! $(PERL_ABSOLUTE)|" < $(NAME) > $(DESTDIR)$(prefix)/bin/$(NAME) chmod 755 $(DESTDIR)$(prefix)/bin/$(NAME) - install -D -m 644 $(NAME).1 $(DESTDIR)$(mandir)/man1/$(NAME).1 - install -D -m 644 $(NAME).rc $(DESTDIR)$(sysconfdir)/Muttrc.d/$(NAME).rc + install -m 644 $(NAME).1 $(DESTDIR)$(mandir)/man1/ + install -m 644 $(NAME).rc $(DESTDIR)$(sysconfdir)/Muttrc.d/ clean: rm -f notmuch-mutt.1 README.html diff --git a/contrib/notmuch-mutt/README b/contrib/notmuch-mutt/README index 26996c4a..c7520228 100644 --- a/contrib/notmuch-mutt/README +++ b/contrib/notmuch-mutt/README @@ -39,8 +39,6 @@ To *run* notmuch-mutt you will need Perl with the following libraries: (Debian package: libmail-box-perl) - Mail::Header (Debian package: libmailtools-perl) -- String::ShellQuote - (Debian package: libstring-shellquote-perl) - Term::ReadLine::Gnu (Debian package: libterm-readline-gnu-perl) diff --git a/contrib/notmuch-mutt/notmuch-mutt b/contrib/notmuch-mutt/notmuch-mutt index 0e46a8c1..b81252c8 100755 --- a/contrib/notmuch-mutt/notmuch-mutt +++ b/contrib/notmuch-mutt/notmuch-mutt @@ -2,7 +2,7 @@ # # notmuch-mutt - notmuch (of a) helper for Mutt # -# Copyright: © 2011-2015 Stefano Zacchiroli +# Copyright: © 2011-2015 Stefano Zacchiroli # License: GNU General Public License (GPL), version 3 or above # # See the bottom of this file for more documentation. @@ -12,11 +12,12 @@ use strict; use warnings; use File::Path; +use File::Basename; +use File::Find; use Getopt::Long qw(:config no_getopt_compat); use Mail::Header; use Mail::Box::Maildir; use Pod::Usage; -use String::ShellQuote; use Term::ReadLine; use Digest::SHA; @@ -25,9 +26,53 @@ my $xdg_cache_dir = "$ENV{HOME}/.cache"; $xdg_cache_dir = $ENV{XDG_CACHE_HOME} if $ENV{XDG_CACHE_HOME}; my $cache_dir = "$xdg_cache_dir/notmuch/mutt"; +sub die_dir($$) { + my ($maildir, $error) = @_; + die "notmuch-mutt: search cache maildir $maildir $error\n". + "Please ensure that the notmuch-mutt search cache Maildir\n". + "contains no subfolders or real mail data, only symlinks to mail\n"; +} + +sub die_subdir($$$) { + my ($maildir, $subdir, $error) = @_; + die_dir($maildir, "subdir $subdir $error"); +} -# create an empty maildir (if missing) or empty an existing maildir" -sub empty_maildir($) { +# check that the search cache maildir is that and not a real maildir +# otherwise there could be data loss when the search cache is emptied +sub check_search_cache_maildir($) { + my ($maildir) = (@_); + + return unless -e $maildir; + + -d $maildir or die_dir($maildir, 'is not a directory'); + + opendir(my $mdh, $maildir) or die_dir($maildir, "cannot be opened: $!"); + my @contents = grep { !/^\.\.?$/ } readdir $mdh; + closedir $mdh; + + my @required = ('cur', 'new', 'tmp'); + foreach my $d (@required) { + -l "$maildir/$d" and die_dir($maildir, "contains symlink $d"); + -e "$maildir/$d" or die_subdir($maildir, $d, 'is missing'); + -d "$maildir/$d" or die_subdir($maildir, $d, 'is not a directory'); + find(sub { + $_ eq '.' and return; + $_ eq '..' and return; + -l $_ or die_subdir($maildir, $d, "contains non-symlink $_"); + }, "$maildir/$d"); + } + + my %required = map { $_ => 1 } @required; + foreach my $d (@contents) { + -l "$maildir/$d" and die_dir( $maildir, "contains symlink $d"); + -d "$maildir/$d" or die_dir( $maildir, "contains non-directory $d"); + exists($required{$d}) or die_dir( $maildir, "contains directory $d"); + } +} + +# create an empty search cache maildir (if missing) or empty existing one +sub empty_search_cache_maildir($) { my ($maildir) = (@_); rmtree($maildir) if (-d $maildir); my $folder = new Mail::Box::Maildir(folder => $maildir, @@ -41,16 +86,18 @@ sub search($$$) { my ($maildir, $remove_dups, $query) = @_; my $dup_option = ""; - $query = shell_quote($query); - - if ($remove_dups) { - $dup_option = "--duplicate=1"; + my @args = qw/notmuch search --output=files/; + push @args, "--duplicate=1" if $remove_dups; + push @args, $query; + + check_search_cache_maildir($maildir); + empty_search_cache_maildir($maildir); + open my $pipe, '-|', @args or die "Running @args failed: $!\n"; + while (<$pipe>) { + chomp; + my $ln = "$maildir/cur/" . basename $_; + symlink $_, "$ln" or warn "Failed to symlink '$_', '$ln': $!\n"; } - - empty_maildir($maildir); - system("notmuch search --output=files $dup_option $query" - . " | sed -e 's: :\\\\ :g'" - . " | xargs -r -I searchoutput ln -s searchoutput $maildir/cur/"); } sub prompt($$) { @@ -119,21 +166,23 @@ sub thread_action($$@) { my $mid = get_message_id(); if (! defined $mid) { - empty_maildir($results_dir); die "notmuch-mutt: cannot find Message-Id, abort.\n"; } - my $search_cmd = 'notmuch search --output=threads ' . shell_quote("id:$mid"); - my $tid = `$search_cmd`; # get thread id - chomp($tid); - search($results_dir, $remove_dups, $tid); + $mid =~ s/ //g; # notmuch strips spaces before storing Message-Id + $mid =~ s/"/""""/g; # escape all double quote characters twice + + search($results_dir, $remove_dups, qq{thread:"{id:""$mid""}"}); } sub tag_action(@) { my $mid = get_message_id(); defined $mid or die "notmuch-mutt: cannot find Message-Id, abort.\n"; - system("notmuch", "tag", @_, "--", "id:$mid"); + $mid =~ s/ //g; # notmuch strips spaces before storing Message-Id + $mid =~ s/"/""/g; # escape all double quote characters + + system("notmuch", "tag", @_, "--", qq{id:"$mid"}); } sub die_usage() { diff --git a/debian/changelog b/debian/changelog index 72a52546..1177f085 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,436 @@ +notmuch (0.38.3-1) unstable; urgency=medium + + * New upstream bugfix release + * Bug fix: "Recommends transitional package gnupg-agent instead of + gpg-agent", thanks to Andreas Metzler (Closes: #1064114). + + -- David Bremner Sat, 09 Mar 2024 23:13:07 -0400 + +notmuch (0.38.2-1.1) unstable; urgency=medium + + * Non-maintainer upload. + * Rename libraries for 64-bit time_t transition. Closes: #1063205 + + -- Benjamin Drung Wed, 28 Feb 2024 23:56:48 +0000 + +notmuch (0.38.2-1) unstable; urgency=medium + + * New upstream bugfix release + + -- David Bremner Fri, 01 Dec 2023 07:51:09 -0400 + +notmuch (0.38.1-1) unstable; urgency=medium + + * New upstream bugfix release + + -- David Bremner Thu, 26 Oct 2023 19:58:42 -0300 + +notmuch (0.38.1~rc1-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Thu, 12 Oct 2023 19:53:10 -0300 + +notmuch (0.38.1~pre0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Sun, 01 Oct 2023 08:14:17 -0300 + +notmuch (0.38-2) unstable; urgency=medium + + * Restrict autopkgtests to amd64 and aarch64. There are failures in + remaining architectures, but the same tests pass at build time, so any + bugs are probably related to either the autopkgtest environment, or + the (new) upstream test runner for installed notmuch. + + -- David Bremner Wed, 13 Sep 2023 19:55:00 -0300 + +notmuch (0.38-1) unstable; urgency=medium + + * New upstream release + * Bug fix: "FTBFS: 6 tests failed.", thanks to Aurelien Jarno (Closes: + #1051111). + * Run most of upstream test suite as autopkgtests + + -- David Bremner Tue, 12 Sep 2023 08:33:24 -0300 + +notmuch (0.38~rc2-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Sun, 03 Sep 2023 09:10:24 -0300 + +notmuch (0.38~rc1-1) experimental; urgency=medium + + * New upstream release candidate + * Hopefully reduce/eliminate intermittent failures of T460 by + controlling Emacs native compilation. + * Disable T810-tsan on ppc64el + + -- David Bremner Sat, 26 Aug 2023 08:31:21 -0300 + +notmuch (0.38~rc0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Thu, 24 Aug 2023 10:56:06 -0300 + +notmuch (0.37-1) unstable; urgency=medium + + * New upstream release. + * Build-depend on emacs-el to work around #1017698 + + -- David Bremner Wed, 24 Aug 2022 09:12:19 -0700 + +notmuch (0.37~rc0-3) experimental; urgency=medium + + * Another no-change re-upload with binaries. + + -- David Bremner Sun, 14 Aug 2022 11:49:21 -0300 + +notmuch (0.37~rc0-2) experimental; urgency=medium + + * Binary upload for NEW (notmuch-git is a new binary package) + + -- David Bremner Sun, 14 Aug 2022 10:55:24 -0300 + +notmuch (0.37~rc0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Sun, 14 Aug 2022 07:28:22 -0300 + +notmuch (0.36-1) unstable; urgency=medium + + * New upstream release + + -- David Bremner Mon, 25 Apr 2022 08:47:41 -0300 + +notmuch (0.36~rc1-1) experimental; urgency=medium + + * New upstream release candidate + * Fix for build in environments where libsexp is not available + (i.e. outside Debian). + + -- David Bremner Sat, 16 Apr 2022 08:37:12 -0300 + +notmuch (0.36~rc0-1) experimental; urgency=medium + + * New upstream release candidate + * Re-enable test smime.4, allegedly fixed upstream. + + -- David Bremner Fri, 15 Apr 2022 08:45:10 -0300 + +notmuch (0.35-2) unstable; urgency=medium + + * Disable test smime.4, which is broken by gmime 3.2.9 thanks to Lucas + Nussbaum for the report (Closes: #1008462). + + -- David Bremner Mon, 28 Mar 2022 11:45:11 -0600 + +notmuch (0.35-1) unstable; urgency=medium + + * New upstream release + + -- David Bremner Sun, 06 Feb 2022 12:15:19 -0400 + +notmuch (0.35~rc0-2) experimental; urgency=medium + + * Reupload with binaries + + -- David Bremner Sat, 29 Jan 2022 21:53:29 -0400 + +notmuch (0.35~rc0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Sat, 29 Jan 2022 18:14:57 -0400 + +notmuch (0.34.3-1) unstable; urgency=medium + + * New upstream bugfix release, with several fixes for the notmuch2 + python module. + + -- David Bremner Sun, 09 Jan 2022 15:30:38 -0400 + +notmuch (0.34.2-1) unstable; urgency=medium + + * New upstream bugfix with release, with fixes database location in + library and notmuch2 python module. + * Build only against the default version of python, to avoid including + multiple .abi3.so files in python3-notmuch2 + + -- David Bremner Fri, 10 Dec 2021 09:35:43 -0400 + +notmuch (0.34.1-1) unstable; urgency=medium + + * New upstream bugfix release. Fixes a memory deallocation error in + libnotmuch. + + -- David Bremner Wed, 03 Nov 2021 10:20:33 -0300 + +notmuch (0.34-1) unstable; urgency=medium + + * New upstream release + * Adds s-expression based query parser (man notmuch-sexp-queries). + * Bug fix: "notmuch breaks on directory removal", thanks to Joerg + Jaspert (Closes: #922536). + * Respect notmuch-show-text/html-blocked-images for renderer w3m + (Closes: #934082). + * Bug fix: "add an option to change the database path", thanks to + Michael Gold (Closes: #887041) (actually fixed in 0.32) + + -- David Bremner Wed, 20 Oct 2021 11:15:23 -0300 + +notmuch (0.34~rc0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Fri, 15 Oct 2021 08:50:57 -0300 + +notmuch (0.33.2-1) unstable; urgency=medium + + * Upstream fix for flaky/hanging tests in T355-smime + + -- David Bremner Thu, 30 Sep 2021 08:27:10 -0300 + +notmuch (0.33.1-1) unstable; urgency=medium + + * Upstream fix for flaky tests in T590-libconfig + + -- David Bremner Fri, 10 Sep 2021 08:28:48 -0300 + +notmuch (0.33-2) unstable; urgency=medium + + * Disable two flaky tests in T590-libconfig. + + -- David Bremner Sat, 04 Sep 2021 11:29:44 -0700 + +notmuch (0.33-1) unstable; urgency=medium + + * New upstream release + * See /usr/share/doc/notmuch/NEWS.gz for user visible changes. + + -- David Bremner Fri, 03 Sep 2021 12:24:41 -0700 + +notmuch (0.33~rc0-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Thu, 26 Aug 2021 08:27:42 -0700 + +notmuch (0.32.3-1) unstable; urgency=medium + + * new upstream bugfix release + * fixes for a few configuration related bugs introduced in 0.32 + * bump libnotmuch minor version to match documentation. + + -- David Bremner Tue, 17 Aug 2021 17:16:09 -0700 + +notmuch (0.32.2-1) experimental; urgency=medium + + * New upstream bugfix release + * Fix for memory leak in "notmuch new" introduced in 0.32 + * Fix for bug from 2017 that can add duplicate thread-ids to messages. + + -- David Bremner Sat, 26 Jun 2021 22:33:56 -0300 + +notmuch (0.32.1-1) experimental; urgency=medium + + * New upstream bugfix release + * Configuration bug fixes (see /usr/share/doc/notmuch/NEWS.gz) + * Bug fix for {pre,after}-tag hooks in emacs, related to lexical scope + transition. + + -- David Bremner Sat, 15 May 2021 09:01:27 -0300 + +notmuch (0.32-1) experimental; urgency=medium + + * New upstream release + * Speedup for handling deleted message files + * New configuration features (see /usr/share/doc/notmuch/NEWS.gz) + * Emacs interface codebase cleanup + + -- David Bremner Sun, 02 May 2021 07:05:15 -0300 + +notmuch (0.32~rc2-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Wed, 28 Apr 2021 07:05:22 -0300 + +notmuch (0.32~rc1-1) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Sat, 24 Apr 2021 12:46:10 -0300 + +notmuch (0.31.4-2) unstable; urgency=medium + + * Cherry pick upstream commit 3f4de98e7c8, which fixes a bug where + duplicate message-ids can cause multiple thread-ids for some message + documents. + * Add build-dependency on xapian-tools, for new test + + -- David Bremner Mon, 28 Jun 2021 22:48:02 -0300 + +notmuch (0.31.4-1) unstable; urgency=medium + + * New upstream bugfix release + - Fix include bug triggered by glib 2.67 + - Fix race condition in T568-lib-thread + + -- David Bremner Thu, 18 Feb 2021 07:23:00 -0400 + +notmuch (0.31.3-2) unstable; urgency=medium + + * Don't install gdb on hppa (skip gdb based tests) + + -- David Bremner Sat, 26 Dec 2020 15:14:07 -0400 + +notmuch (0.31.3-1) unstable; urgency=medium + + * New upstream bugfix release + * Second fix for T360, fix regression on ppc64el + * Fix for exclude tags in notmuch2 python bindings + * Fix for memory error in notmuch_database_get_config_list + + -- David Bremner Fri, 25 Dec 2020 11:48:37 -0400 + +notmuch (0.31.2-5) unstable; urgency=medium + + * Use readelf instead of nm in T360, hopefully build in ppc64 + + -- David Bremner Sun, 13 Dec 2020 08:24:23 -0400 + +notmuch (0.31.2-4) unstable; urgency=medium + + * Move prerequisite to file targets from phony ones. Thanks to + Lucas Nussbaum for the report. (Closes: #976934). + + -- David Bremner Thu, 10 Dec 2020 21:02:20 -0400 + +notmuch (0.31.2-3) unstable; urgency=medium + + * Switch to debhelper compat level 13 + + -- David Bremner Mon, 09 Nov 2020 13:59:47 -0400 + +notmuch (0.31.2-2) unstable; urgency=medium + + * Run tests in verbose mode + + -- David Bremner Mon, 09 Nov 2020 08:45:38 -0400 + +notmuch (0.31.2-1) unstable; urgency=medium + + * Delete stray "version" file in upstream source + + -- David Bremner Sun, 08 Nov 2020 11:32:45 -0400 + +notmuch (0.31.1-1) unstable; urgency=medium + + * New upstream bugfix release. + - Portability / C++20 fixes + - Fix initialization bug in library config handling. + + -- David Bremner Sun, 08 Nov 2020 07:48:22 -0400 + +notmuch (0.31-1) unstable; urgency=medium + + * New upstream release + * Compatibility fixes for Emacs 27.1 + + -- David Bremner Sat, 05 Sep 2020 21:47:42 -0300 + +notmuch (0.31~rc2-1) experimental; urgency=medium + + * New upstream release candidate + * Bug fix: "suggest elpa-mailscripts", thanks to Sean Whitton (Closes: + #944269). + * Bug fix: "suggest mailscripts", thanks to Sean Whitton (Closes: + #944270). + * Bug fix: "please drop transitional package notmuch-emacs from + src:notmuch", thanks to Holger Levsen (Closes: #940738). + + -- David Bremner Tue, 25 Aug 2020 07:51:33 -0300 + +notmuch (0.31~rc1-1) experimental; urgency=medium + + * Fix buggy test in T562-lib-database + * Clean up generated file in source package. + + -- David Bremner Mon, 17 Aug 2020 21:05:46 -0300 + +notmuch (0.31~rc0-1) experimental; urgency=medium + + * New upstream release candidate. + * Update notmuch-emacs for compatibility with GNU Emacs 27.1. + + -- David Bremner Sun, 16 Aug 2020 11:08:14 -0300 + +notmuch (0.30-1) unstable; urgency=medium + + * New upstream release + * Improvements to S/MIME handling + * Repairs to some mangled MIME messages + * New python bindings (notmuch2) compatible with current python 3 + + -- David Bremner Fri, 10 Jul 2020 22:24:14 -0300 + +notmuch (0.30~rc3-1) experimental; urgency=medium + + * New upstream release candidate + * Mark two tests broken on legacy (32 bit time_t) architectures. + * Drop -std=c99 + + -- David Bremner Fri, 03 Jul 2020 06:48:51 -0300 + +notmuch (0.30~rc2-1) experimental; urgency=medium + + * New upstream release candidate. + * Upstream fixes for new python bindings (python3-notmuch2). + * Update debian/copyright (one new author). + + -- David Bremner Tue, 16 Jun 2020 08:32:16 -0300 + +notmuch (0.30~rc1-1) experimental; urgency=medium + + * New upstream release candidate + * Update debian/changelog (new copyright holders) + + -- David Bremner Sat, 06 Jun 2020 08:06:56 -0300 + +notmuch (0.30~rc0-2) experimental; urgency=medium + + * New upstream release candidate + + -- David Bremner Mon, 01 Jun 2020 21:01:27 -0300 + +notmuch (0.29.3-1) unstable; urgency=medium + + * New upstream bugfix release. + - fix use-after-free bug in libnotmuch + - fix double close of file in "notmuch dump" + + -- David Bremner Wed, 27 Nov 2019 08:19:57 -0400 + +notmuch (0.29.2-2) experimental; urgency=medium + + * Drop python-notmuch binary package. + * Drop python2 build-dependency (Closes: #937161). + * Convert to pybuild for python bindings + + -- David Bremner Sat, 02 Nov 2019 18:21:06 -0300 + +notmuch (0.29.2-1) unstable; urgency=medium + + * New upstream bug fix release: fix file descriptor leak with gzipped + files. Thanks to James Troup for reporting and the fix. + + -- David Bremner Sat, 19 Oct 2019 07:23:21 -0300 + notmuch (0.29.1-2) unstable; urgency=medium * Re-upload to unstable diff --git a/debian/compat b/debian/compat deleted file mode 100644 index b4de3947..00000000 --- a/debian/compat +++ /dev/null @@ -1 +0,0 @@ -11 diff --git a/debian/control b/debian/control index ff646c6b..4fded909 100644 --- a/debian/control +++ b/debian/control @@ -4,40 +4,62 @@ Priority: optional Maintainer: Carl Worth Uploaders: Jameson Graef Rollins , - David Bremner -Build-Conflicts: ruby1.8, gdb-minimal, gdb [ia64 mips mips64el] -Build-Depends: + David Bremner , +Build-Conflicts: + gdb [ia64 mips mips64el hppa], + gdb-minimal, + ruby1.8, +Build-Depends: dpkg-dev (>= 1.22.5), + bash-completion (>=1.9.0~), + debhelper-compat (= 13), + dh-elpa (>= 1.3), + dh-python, + desktop-file-utils, + doxygen, dpkg-dev (>= 1.17.14), - debhelper (>= 11~), - pkg-config, - libxapian-dev, + dtach (>= 0.8) , + emacs-nox | emacs-gtk | emacs-lucid | emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) | emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~), + emacs-el, + gdb [!ia64 !mips !mips64el !kfreebsd-any !alpha !hppa] , + git , + gnupg , + gpgsm , libgmime-3.0-dev (>= 3.0.3~), + libpython3-dev, + libsexp-dev, libtalloc-dev, + libxapian-dev, libz-dev, - python-all (>= 2.6.6-3~), - python3-all (>= 3.1.2-7~), - dh-python, - dh-elpa (>= 1.3), + pkg-config, + python3, + python3-cffi, + python3-pytest, + python3-pytest-cov, + python3-setuptools, python3-sphinx, - ruby, ruby-dev (>>1:1.9.3~), - emacs-nox | emacs-gtk | emacs-lucid | - emacs25-nox | emacs25 (>=25~) | emacs25-lucid (>=25~) | - emacs24-nox | emacs24 (>=24~) | emacs24-lucid (>=24~), - gdb [!ia64 !mips !mips64el !kfreebsd-any !alpha] , - dtach (>= 0.8) , - gpgsm , - gnupg , - bash-completion (>=1.9.0~), - texinfo -Standards-Version: 4.3.0 + ruby, + ruby-dev (>>1:1.9.3~), + texinfo, + xapian-tools , +Standards-Version: 4.4.1 Homepage: https://notmuchmail.org/ Vcs-Git: https://git.notmuchmail.org/git/notmuch -b release Vcs-Browser: https://git.notmuchmail.org/git/notmuch +Rules-Requires-Root: no Package: notmuch Architecture: any -Depends: libnotmuch5 (= ${binary:Version}), ${shlibs:Depends}, ${misc:Depends} -Recommends: elpa-notmuch | notmuch-vim | notmuch-mutt | alot, gnupg-agent, gpgsm +Depends: + libnotmuch5t64 (= ${binary:Version}), + ${misc:Depends}, + ${shlibs:Depends}, +Recommends: + elpa-notmuch | notmuch-vim | notmuch-mutt | alot, + gpg-agent, + gpgsm, +Suggests: + mailscripts, + notmuch-doc, Description: thread-based email index, search and tagging Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -46,11 +68,48 @@ Description: thread-based email index, search and tagging . This package contains the notmuch command-line interface -Package: libnotmuch5 +Package: notmuch-git +Architecture: all +Depends: + git, + notmuch, + python3, + ${misc:Depends} +Description: thread-based email index, search and tagging + Notmuch is a system for indexing, searching, reading, and tagging + large collections of email messages in maildir or mh format. It uses + the Xapian library to provide fast, full-text search with a very + convenient search syntax. + . + This package contains a simple tool to save, restore, and synchronize + notmuch tags via git repositories. + +Package: notmuch-doc +Architecture: all +Depends: + ${misc:Depends}, + ${sphinxdoc:Depends}, +Suggests: + notmuch +Description: thread-based email index, search and tagging + Notmuch is a system for indexing, searching, reading, and tagging + large collections of email messages in maildir or mh format. It uses + the Xapian library to provide fast, full-text search with a very + convenient search syntax. + . + This package contains the HTML documentation + +Package: libnotmuch5t64 +Provides: ${t64:Provides} +Replaces: libnotmuch5 +Breaks: libnotmuch5 (<< ${source:Version}) Section: libs Architecture: any -Depends: ${shlibs:Depends}, ${misc:Depends} -Pre-Depends: ${misc:Pre-Depends} +Depends: + ${misc:Depends}, + ${shlibs:Depends}, +Pre-Depends: + ${misc:Pre-Depends}, Description: thread-based email index, search and tagging (runtime) Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -63,7 +122,9 @@ Description: thread-based email index, search and tagging (runtime) Package: libnotmuch-dev Section: libdevel Architecture: any -Depends: ${misc:Depends}, libnotmuch5 (= ${binary:Version}) +Depends: + libnotmuch5t64 (= ${binary:Version}), + ${misc:Depends}, Description: thread-based email index, search and tagging (development) Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -73,25 +134,32 @@ Description: thread-based email index, search and tagging (development) This package provides the necessary development libraries and header files to allow you to develop new software using libnotmuch. -Package: python-notmuch +Package: python3-notmuch Architecture: all Section: python -Depends: ${misc:Depends}, ${python:Depends}, libnotmuch5 (>= ${source:Version}) -Provides: ${python:Provides} -XB-Python-Version: ${python:Versions} -Description: Python interface to the notmuch mail search and index library +Depends: + libnotmuch5t64 (>= ${source:Version}), + ${misc:Depends}, + ${python3:Depends}, +Description: Python 3 legacy interface to the notmuch mail search and index library Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses the Xapian library to provide fast, full-text search with a very convenient search syntax. . - This package provides a Python interface to the notmuch + This package provides a legacy Python 3 interface to the notmuch functionality, directly interfacing with a shared notmuch library. + . + New projects are encouraged to use python3-notmuch2 instead. -Package: python3-notmuch -Architecture: all +Package: python3-notmuch2 +Architecture: any Section: python -Depends: ${misc:Depends}, ${python3:Depends}, libnotmuch5 (>= ${source:Version}) +Depends: + libnotmuch5t64 (>= ${source:Version}), + ${misc:Depends}, + ${python3:Depends}, + ${shlibs:Depends}, Description: Python 3 interface to the notmuch mail search and index library Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -99,12 +167,17 @@ Description: Python 3 interface to the notmuch mail search and index library convenient search syntax. . This package provides a Python 3 interface to the notmuch - functionality, directly interfacing with a shared notmuch library. + functionality using CFFI bindings, which interface with a shared + notmuch library. + . + This is the preferred way to use notmuch via Python. Package: ruby-notmuch Architecture: any Section: ruby -Depends: ${shlibs:Depends}, ${misc:Depends} +Depends: + ${misc:Depends}, + ${shlibs:Depends}, Description: Ruby interface to the notmuch mail search and index library Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -114,16 +187,12 @@ Description: Ruby interface to the notmuch mail search and index library This package provides a Ruby interface to the notmuch functionality, directly interfacing with a shared notmuch library. -Package: notmuch-emacs -Section: oldlibs -Architecture: all -Depends: elpa-notmuch, ${misc:Depends} -Description: thread-based email index, search and tagging (transitional package) - This dummy package help ease transition to the new package elpa-notmuch - Package: elpa-notmuch Architecture: all -Depends: ${misc:Depends}, ${elpa:Depends} +Depends: + ${elpa:Depends}, + ${misc:Depends}, +Suggests: elpa-mailscripts Description: thread-based email index, search and tagging (emacs interface) Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -135,10 +204,18 @@ Description: thread-based email index, search and tagging (emacs interface) Package: notmuch-vim Architecture: all -Breaks: notmuch (<<0.6~254~) -Replaces: notmuch (<<0.6~254~) -Depends: ${misc:Depends}, notmuch, vim-addon-manager, vim-ruby, ruby-notmuch -Recommends: ruby-mail +Breaks: + notmuch (<<0.6~254~), +Replaces: + notmuch (<<0.6~254~), +Depends: + notmuch, + ruby-notmuch, + vim-addon-manager, + vim-ruby, + ${misc:Depends}, +Recommends: + ruby-mail, Description: thread-based email index, search and tagging (vim interface) Notmuch is a system for indexing, searching, reading, and tagging large collections of email messages in maildir or mh format. It uses @@ -151,13 +228,17 @@ Description: thread-based email index, search and tagging (vim interface) Package: notmuch-mutt Architecture: all Depends: + libmail-box-perl, + libmailtools-perl, + libterm-readline-gnu-perl, notmuch (>= 0.4), - libmail-box-perl, libmailtools-perl, - libstring-shellquote-perl, libterm-readline-gnu-perl, ${misc:Depends}, ${perl:Depends}, -Recommends: mutt -Enhances: notmuch, mutt +Recommends: + mutt, +Enhances: + mutt, + notmuch, Description: thread-based email index, search and tagging (Mutt interface) notmuch-mutt provides integration among the Mutt mail user agent and the Notmuch mail indexer. diff --git a/debian/copyright b/debian/copyright index 0931d9b9..ba221e6b 100644 --- a/debian/copyright +++ b/debian/copyright @@ -1,36 +1,100 @@ Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ Upstream-Name: notmuch -Source: git://notmuchmail.org/git/notmuch +Source: https://git.notmuchmail.org/git/notmuch Upstream-Contact: Notmuch Mailing List Files: * -Copyright: Copyright 2009 Carl Worth - Bart Trojanowski - Keith Packard - Alexander Botero-Lowry - Ingmar Vanhassel - Jed Brown - Jan Janak - Chris Wilson - Keith Amidon - Aneesh Kumar K.V - Mikhail Gusarov - Jeffrey C. Ollie - Jameson Graef Rollins - Stewart Smith - Adrian Perez - Kan-Ru Chen - James Rowe - Eric Anholt - Alec Berryman - Tassilo Horn - Stefan Schmidt - Rolland Santimano - Peter Wang - Lars Kellogg-Stedman - Holger Freyther - David Bremner - Alexander Botero-Lowry +Copyright: Copyright 2009-2020 + David Bremner + Carl Worth + Jani Nikula + Austin Clements + Daniel Kahn Gillmor + Mark Walters + Floris Bruynooghe + David Edmondson + Tomi Ollila + Sebastian Spaeth + Ali Polatel + Michal Sojka + Justus Winter + Sebastien Binet + W. Trevor King + Jameson Graef Rollins + Felipe Contreras + Pieter Praet + Peter Feigl + Dmitry Kurochkin + Peter Wang + Daniel Schoepe + Gregor Zattler + Keith Packard + Adam Wolfe Gordon + Stefano Zacchiroli + Vincent Breitmoser + laochailan + Ben Gamari + Aaron Ecay + Jesse Rosenthal + l-m-h@web.de + Thomas Jost + Dirk Hohndel + Blake Jones + Jonas Bernoulli + Damien Cassou + Vladimir Panteleev + Anton Khirnov + Matt Armstrong + Örjan Ekeberg + Jan Janak + Patrick Totzke + Chunyang Xu + rhn + Ruben Pollan + Ioan-Adrian Ratiu + Ethan Glasser-Camp + Todd + Chris Wilson + William Casarin + Yuri Volchkov + Cédric Cabessa + Mark Anderson + Jed Brown + Maxime Coste + Ludovic LANGE + Sebastian Poeplau + Mikhail + Gaute Hope + Keith Amidon + martin f. krafft + Jeffrey C. Ollie + Bart Trojanowski + Jameson Rollins + Scott Henson + Vladimir Marek + Servilio Afre Puentes + Kevin McCarthy + Tomas Carnecky + Kevin J. McCarthy + Scott Robinson + Wael M. Nasreddine + Charles Celerier + Olly Betts + Istvan Marko + Florian Klink + Thibaut Horel + Joel Borggrén-Franck + Ingmar Vanhassel + Olivier Taïbi + Ian Main + Alexander Botero-Lowry + Luis Ressel + Sergei Shilovsky + Trevor Jim + Jinwoo Lee + Uli Scholler + Matthew Lear + Amadeusz Å»ołnowski License: GPL-3+ Files: debian/* diff --git a/debian/elpa-notmuch.elpa b/debian/elpa-notmuch.elpa index 19e3ba51..7b3ce0fa 100644 --- a/debian/elpa-notmuch.elpa +++ b/debian/elpa-notmuch.elpa @@ -1,3 +1,3 @@ -emacs/*.el -emacs/notmuch-logo.png -debian/tmp/usr/share/info/* +debian/tmp/usr/share/emacs/site-lisp/*.el +debian/tmp/usr/share/emacs/site-lisp/notmuch-logo.svg +emacs/notmuch-pkg.el diff --git a/debian/elpa-notmuch.info b/debian/elpa-notmuch.info new file mode 100644 index 00000000..0ac0fbf6 --- /dev/null +++ b/debian/elpa-notmuch.info @@ -0,0 +1 @@ +usr/share/info/*.info diff --git a/debian/elpa-notmuch.lintian-overrides b/debian/elpa-notmuch.lintian-overrides new file mode 100644 index 00000000..aa275eda --- /dev/null +++ b/debian/elpa-notmuch.lintian-overrides @@ -0,0 +1,4 @@ +# elpa-notmuch is an elisp plugin for dealing with e-mail. We can +# already tell from the package name that it is an elisp package, so +# it belongs in Section: mail, and lintian is being too strict here. +elpa-notmuch: wrong-section-according-to-package-name elpa-notmuch => lisp diff --git a/debian/libnotmuch-dev.manpages b/debian/libnotmuch-dev.manpages new file mode 100644 index 00000000..9c4bd5d3 --- /dev/null +++ b/debian/libnotmuch-dev.manpages @@ -0,0 +1 @@ +usr/share/man/man3/notmuch.3.gz diff --git a/debian/libnotmuch5.install b/debian/libnotmuch5.install deleted file mode 100644 index a513b475..00000000 --- a/debian/libnotmuch5.install +++ /dev/null @@ -1 +0,0 @@ -usr/lib/*/libnotmuch.so.* diff --git a/debian/libnotmuch5.symbols b/debian/libnotmuch5.symbols deleted file mode 100644 index 308567b8..00000000 --- a/debian/libnotmuch5.symbols +++ /dev/null @@ -1,140 +0,0 @@ -libnotmuch.so.5 libnotmuch5 #MINVER# - notmuch_built_with@Base 0.23~rc0 - notmuch_config_list_destroy@Base 0.23~rc0 - notmuch_config_list_key@Base 0.23~rc0 - notmuch_config_list_move_to_next@Base 0.23~rc0 - notmuch_config_list_valid@Base 0.23~rc0 - notmuch_config_list_value@Base 0.23~rc0 - notmuch_database_add_message@Base 0.3 - notmuch_database_begin_atomic@Base 0.9~rc1 - notmuch_database_close@Base 0.13~rc1 - notmuch_database_compact@Base 0.17~rc1 - notmuch_database_create@Base 0.3 - notmuch_database_create_verbose@Base 0.20~rc1 - notmuch_database_destroy@Base 0.13~rc1 - notmuch_database_end_atomic@Base 0.9~rc1 - notmuch_database_find_message@Base 0.9~rc2 - notmuch_database_find_message_by_filename@Base 0.9~rc2 - notmuch_database_get_all_tags@Base 0.3 - notmuch_database_get_config@Base 0.23~rc0 - notmuch_database_get_config_list@Base 0.23~rc0 - notmuch_database_get_default_indexopts@Base 0.26~rc0 - notmuch_database_get_directory@Base 0.3 - notmuch_database_get_path@Base 0.3 - notmuch_database_get_revision@Base 0.21~rc1 - notmuch_database_get_version@Base 0.3 - notmuch_database_index_file@Base 0.26~rc0 - notmuch_database_needs_upgrade@Base 0.3 - notmuch_database_open@Base 0.3 - notmuch_database_open_verbose@Base 0.20~rc1 - notmuch_database_remove_message@Base 0.3 - notmuch_database_set_config@Base 0.23~rc0 - notmuch_database_status_string@Base 0.20~rc1 - notmuch_database_upgrade@Base 0.3 - notmuch_directory_delete@Base 0.21~rc1 - notmuch_directory_destroy@Base 0.3 - notmuch_directory_get_child_directories@Base 0.3 - notmuch_directory_get_child_files@Base 0.3 - notmuch_directory_get_mtime@Base 0.3 - notmuch_directory_set_mtime@Base 0.3 - notmuch_filenames_destroy@Base 0.3 - notmuch_filenames_get@Base 0.3 - notmuch_filenames_move_to_next@Base 0.3 - notmuch_filenames_valid@Base 0.3 - notmuch_indexopts_destroy@Base 0.26~rc0 - notmuch_indexopts_get_decrypt_policy@Base 0.26~rc0 - notmuch_indexopts_set_decrypt_policy@Base 0.26~rc0 - notmuch_message_add_property@Base 0.23~rc0 - notmuch_message_add_tag@Base 0.3 - notmuch_message_count_files@Base 0.26~rc0 - notmuch_message_count_properties@Base 0.27~rc0 - notmuch_message_destroy@Base 0.3 - notmuch_message_freeze@Base 0.3 - notmuch_message_get_database@Base 0.27~rc0 - notmuch_message_get_date@Base 0.3 - notmuch_message_get_filename@Base 0.3 - notmuch_message_get_filenames@Base 0.5 - notmuch_message_get_flag@Base 0.3 - notmuch_message_get_header@Base 0.3 - notmuch_message_get_message_id@Base 0.3 - notmuch_message_get_properties@Base 0.23~rc0 - notmuch_message_get_property@Base 0.23~rc0 - notmuch_message_get_replies@Base 0.3 - notmuch_message_get_tags@Base 0.3 - notmuch_message_get_thread_id@Base 0.3 - notmuch_message_has_maildir_flag@Base 0.26~rc0 - notmuch_message_maildir_flags_to_tags@Base 0.5 - notmuch_message_properties_destroy@Base 0.23~rc0 - notmuch_message_properties_key@Base 0.23~rc0 - notmuch_message_properties_move_to_next@Base 0.23~rc0 - notmuch_message_properties_valid@Base 0.23~rc0 - notmuch_message_properties_value@Base 0.23~rc0 - notmuch_message_reindex@Base 0.26~rc0 - notmuch_message_remove_all_properties@Base 0.23~rc0 - notmuch_message_remove_all_properties_with_prefix@Base 0.26~rc0 - notmuch_message_remove_all_tags@Base 0.3 - notmuch_message_remove_property@Base 0.23~rc0 - notmuch_message_remove_tag@Base 0.3 - notmuch_message_set_flag@Base 0.3 - notmuch_message_tags_to_maildir_flags@Base 0.5 - notmuch_message_thaw@Base 0.3 - notmuch_messages_collect_tags@Base 0.3 - notmuch_messages_destroy@Base 0.3 - notmuch_messages_get@Base 0.3 - notmuch_messages_move_to_next@Base 0.3 - notmuch_messages_valid@Base 0.3 - notmuch_query_add_tag_exclude@Base 0.12~rc1 - notmuch_query_count_messages@Base 0.3 - notmuch_query_count_messages_st@Base 0.21~rc1 - notmuch_query_count_threads@Base 0.10~rc1 - notmuch_query_count_threads_st@Base 0.21~rc1 - notmuch_query_create@Base 0.3 - notmuch_query_destroy@Base 0.3 - notmuch_query_get_database@Base 0.21~rc1 - notmuch_query_get_query_string@Base 0.4 - notmuch_query_get_sort@Base 0.4 - notmuch_query_search_messages@Base 0.3 - notmuch_query_search_messages_st@Base 0.20~rc1 - notmuch_query_search_threads@Base 0.3 - notmuch_query_search_threads_st@Base 0.20~rc1 - notmuch_query_set_omit_excluded@Base 0.13~rc1 - notmuch_query_set_sort@Base 0.3 - notmuch_status_to_string@Base 0.3 - notmuch_tags_destroy@Base 0.3 - notmuch_tags_get@Base 0.3 - notmuch_tags_move_to_next@Base 0.3 - notmuch_tags_valid@Base 0.3 - notmuch_thread_destroy@Base 0.3 - notmuch_thread_get_authors@Base 0.3 - notmuch_thread_get_matched_messages@Base 0.3 - notmuch_thread_get_messages@Base 0.16 - notmuch_thread_get_newest_date@Base 0.3 - notmuch_thread_get_oldest_date@Base 0.3 - notmuch_thread_get_subject@Base 0.3 - notmuch_thread_get_tags@Base 0.3 - notmuch_thread_get_thread_id@Base 0.3 - notmuch_thread_get_toplevel_messages@Base 0.3 - notmuch_thread_get_total_files@Base 0.26~rc0 - notmuch_thread_get_total_messages@Base 0.3 - notmuch_threads_destroy@Base 0.3 - notmuch_threads_get@Base 0.3 - notmuch_threads_move_to_next@Base 0.3 - notmuch_threads_valid@Base 0.3 - (c++)"typeinfo for Xapian::LogicError@Base" 0.6.1 - (c++)"typeinfo for Xapian::RuntimeError@Base" 0.6.1 - (c++)"typeinfo for Xapian::DocNotFoundError@Base" 0.6.1 - (c++)"typeinfo for Xapian::InvalidArgumentError@Base" 0.6.1 - (c++)"typeinfo for Xapian::Error@Base" 0.6.1 - (c++)"typeinfo for Xapian::DatabaseError@Base" 0.24~rc0 - (c++)"typeinfo for Xapian::DatabaseModifiedError@Base" 0.24~rc0 - (c++|optional=present with Xapian 1.4)"typeinfo for Xapian::QueryParserError@Base" 0.23~rc0 - (c++)"typeinfo for Xapian::QueryParser::add_valuerangeprocessor(Xapian::ValueRangeProcessor*)::ShimRangeProcessor@Base" 0.25~rc0 - (c++)"typeinfo name for Xapian::QueryParser::add_valuerangeprocessor(Xapian::ValueRangeProcessor*)::ShimRangeProcessor@Base" 0.25~rc0 - (c++)"typeinfo name for Xapian::LogicError@Base" 0.6.1 - (c++)"typeinfo name for Xapian::RuntimeError@Base" 0.6.1 - (c++)"typeinfo name for Xapian::DocNotFoundError@Base" 0.6.1 - (c++)"typeinfo name for Xapian::InvalidArgumentError@Base" 0.6.1 - (c++)"typeinfo name for Xapian::Error@Base" 0.6.1 - (c++)"typeinfo name for Xapian::DatabaseError@Base" 0.24~rc0 - (c++)"typeinfo name for Xapian::DatabaseModifiedError@Base" 0.24~rc0 - (c++|optional=present with Xapian 1.4)"typeinfo name for Xapian::QueryParserError@Base" 0.23~rc0 diff --git a/debian/libnotmuch5t64.install b/debian/libnotmuch5t64.install new file mode 100644 index 00000000..a513b475 --- /dev/null +++ b/debian/libnotmuch5t64.install @@ -0,0 +1 @@ +usr/lib/*/libnotmuch.so.* diff --git a/debian/libnotmuch5t64.lintian-overrides b/debian/libnotmuch5t64.lintian-overrides new file mode 100644 index 00000000..affb63b4 --- /dev/null +++ b/debian/libnotmuch5t64.lintian-overrides @@ -0,0 +1 @@ +libnotmuch5t64: package-name-doesnt-match-sonames libnotmuch5 diff --git a/debian/libnotmuch5t64.symbols b/debian/libnotmuch5t64.symbols new file mode 100644 index 00000000..5715dec0 --- /dev/null +++ b/debian/libnotmuch5t64.symbols @@ -0,0 +1,166 @@ +libnotmuch.so.5 libnotmuch5t64 #MINVER# +* Build-Depends-Package: libnotmuch-dev + notmuch_built_with@Base 0.23~rc0 + notmuch_config_get@Base 0.32~rc0 + notmuch_config_get_bool@Base 0.32~rc0 + notmuch_config_get_pairs@Base 0.32~rc0 + notmuch_config_get_values@Base 0.32~rc0 + notmuch_config_get_values_string@Base 0.32~rc0 + notmuch_config_list_destroy@Base 0.23~rc0 + notmuch_config_list_key@Base 0.23~rc0 + notmuch_config_list_move_to_next@Base 0.23~rc0 + notmuch_config_list_valid@Base 0.23~rc0 + notmuch_config_list_value@Base 0.23~rc0 + notmuch_config_pairs_destroy@Base 0.32~rc0 + notmuch_config_pairs_key@Base 0.32~rc0 + notmuch_config_pairs_move_to_next@Base 0.32~rc0 + notmuch_config_pairs_valid@Base 0.32~rc0 + notmuch_config_pairs_value@Base 0.32~rc0 + notmuch_config_path@Base 0.32~rc0 + notmuch_config_set@Base 0.32~rc0 + notmuch_config_values_destroy@Base 0.32~rc0 + notmuch_config_values_get@Base 0.32~rc0 + notmuch_config_values_move_to_next@Base 0.32~rc0 + notmuch_config_values_start@Base 0.32~rc0 + notmuch_config_values_valid@Base 0.32~rc0 + notmuch_database_add_message@Base 0.3 + notmuch_database_begin_atomic@Base 0.9~rc1 + notmuch_database_close@Base 0.13~rc1 + notmuch_database_compact@Base 0.17~rc1 + notmuch_database_compact_db@Base 0.32~rc0 + notmuch_database_create@Base 0.3 + notmuch_database_create_verbose@Base 0.20~rc1 + notmuch_database_create_with_config@Base 0.32~rc0 + notmuch_database_destroy@Base 0.13~rc1 + notmuch_database_end_atomic@Base 0.9~rc1 + notmuch_database_find_message@Base 0.9~rc2 + notmuch_database_find_message_by_filename@Base 0.9~rc2 + notmuch_database_get_all_tags@Base 0.3 + notmuch_database_get_config@Base 0.23~rc0 + notmuch_database_get_config_list@Base 0.23~rc0 + notmuch_database_get_default_indexopts@Base 0.26~rc0 + notmuch_database_get_directory@Base 0.3 + notmuch_database_get_path@Base 0.3 + notmuch_database_get_revision@Base 0.21~rc1 + notmuch_database_get_version@Base 0.3 + notmuch_database_index_file@Base 0.26~rc0 + notmuch_database_load_config@Base 0.32~rc0 + notmuch_database_needs_upgrade@Base 0.3 + notmuch_database_open@Base 0.3 + notmuch_database_open_verbose@Base 0.20~rc1 + notmuch_database_open_with_config@Base 0.32~rc0 + notmuch_database_remove_message@Base 0.3 + notmuch_database_reopen@Base 0.32~rc0 + notmuch_database_set_config@Base 0.23~rc0 + notmuch_database_status_string@Base 0.20~rc1 + notmuch_database_upgrade@Base 0.3 + notmuch_directory_delete@Base 0.21~rc1 + notmuch_directory_destroy@Base 0.3 + notmuch_directory_get_child_directories@Base 0.3 + notmuch_directory_get_child_files@Base 0.3 + notmuch_directory_get_mtime@Base 0.3 + notmuch_directory_set_mtime@Base 0.3 + notmuch_filenames_destroy@Base 0.3 + notmuch_filenames_get@Base 0.3 + notmuch_filenames_move_to_next@Base 0.3 + notmuch_filenames_valid@Base 0.3 + notmuch_indexopts_destroy@Base 0.26~rc0 + notmuch_indexopts_get_decrypt_policy@Base 0.26~rc0 + notmuch_indexopts_set_decrypt_policy@Base 0.26~rc0 + notmuch_message_add_property@Base 0.23~rc0 + notmuch_message_add_tag@Base 0.3 + notmuch_message_count_files@Base 0.26~rc0 + notmuch_message_count_properties@Base 0.27~rc0 + notmuch_message_destroy@Base 0.3 + notmuch_message_freeze@Base 0.3 + notmuch_message_get_database@Base 0.27~rc0 + notmuch_message_get_date@Base 0.3 + notmuch_message_get_filename@Base 0.3 + notmuch_message_get_filenames@Base 0.5 + notmuch_message_get_flag@Base 0.3 + notmuch_message_get_flag_st@Base 0.31~rc0 + notmuch_message_get_header@Base 0.3 + notmuch_message_get_message_id@Base 0.3 + notmuch_message_get_properties@Base 0.23~rc0 + notmuch_message_get_property@Base 0.23~rc0 + notmuch_message_get_replies@Base 0.3 + notmuch_message_get_tags@Base 0.3 + notmuch_message_get_thread_id@Base 0.3 + notmuch_message_has_maildir_flag@Base 0.26~rc0 + notmuch_message_has_maildir_flag_st@Base 0.31~rc0 + notmuch_message_maildir_flags_to_tags@Base 0.5 + notmuch_message_properties_destroy@Base 0.23~rc0 + notmuch_message_properties_key@Base 0.23~rc0 + notmuch_message_properties_move_to_next@Base 0.23~rc0 + notmuch_message_properties_valid@Base 0.23~rc0 + notmuch_message_properties_value@Base 0.23~rc0 + notmuch_message_reindex@Base 0.26~rc0 + notmuch_message_remove_all_properties@Base 0.23~rc0 + notmuch_message_remove_all_properties_with_prefix@Base 0.26~rc0 + notmuch_message_remove_all_tags@Base 0.3 + notmuch_message_remove_property@Base 0.23~rc0 + notmuch_message_remove_tag@Base 0.3 + notmuch_message_set_flag@Base 0.3 + notmuch_message_tags_to_maildir_flags@Base 0.5 + notmuch_message_thaw@Base 0.3 + notmuch_messages_collect_tags@Base 0.3 + notmuch_messages_destroy@Base 0.3 + notmuch_messages_get@Base 0.3 + notmuch_messages_move_to_next@Base 0.3 + notmuch_messages_valid@Base 0.3 + notmuch_query_add_tag_exclude@Base 0.12~rc1 + notmuch_query_count_messages@Base 0.3 + notmuch_query_count_messages_st@Base 0.21~rc1 + notmuch_query_count_threads@Base 0.10~rc1 + notmuch_query_count_threads_st@Base 0.21~rc1 + notmuch_query_create@Base 0.3 + notmuch_query_create_with_syntax@Base 0.34~rc0 + notmuch_query_destroy@Base 0.3 + notmuch_query_get_database@Base 0.21~rc1 + notmuch_query_get_query_string@Base 0.4 + notmuch_query_get_sort@Base 0.4 + notmuch_query_search_messages@Base 0.3 + notmuch_query_search_messages_st@Base 0.20~rc1 + notmuch_query_search_threads@Base 0.3 + notmuch_query_search_threads_st@Base 0.20~rc1 + notmuch_query_set_omit_excluded@Base 0.13~rc1 + notmuch_query_set_sort@Base 0.3 + notmuch_status_to_string@Base 0.3 + notmuch_tags_destroy@Base 0.3 + notmuch_tags_get@Base 0.3 + notmuch_tags_move_to_next@Base 0.3 + notmuch_tags_valid@Base 0.3 + notmuch_thread_destroy@Base 0.3 + notmuch_thread_get_authors@Base 0.3 + notmuch_thread_get_matched_messages@Base 0.3 + notmuch_thread_get_messages@Base 0.16 + notmuch_thread_get_newest_date@Base 0.3 + notmuch_thread_get_oldest_date@Base 0.3 + notmuch_thread_get_subject@Base 0.3 + notmuch_thread_get_tags@Base 0.3 + notmuch_thread_get_thread_id@Base 0.3 + notmuch_thread_get_toplevel_messages@Base 0.3 + notmuch_thread_get_total_files@Base 0.26~rc0 + notmuch_thread_get_total_messages@Base 0.3 + notmuch_threads_destroy@Base 0.3 + notmuch_threads_get@Base 0.3 + notmuch_threads_move_to_next@Base 0.3 + notmuch_threads_valid@Base 0.3 + (c++)"typeinfo for Xapian::LogicError@Base" 0.6.1 + (c++)"typeinfo for Xapian::RuntimeError@Base" 0.6.1 + (c++)"typeinfo for Xapian::DocNotFoundError@Base" 0.6.1 + (c++)"typeinfo for Xapian::InvalidArgumentError@Base" 0.6.1 + (c++)"typeinfo for Xapian::Error@Base" 0.6.1 + (c++)"typeinfo for Xapian::DatabaseError@Base" 0.24~rc0 + (c++)"typeinfo for Xapian::DatabaseModifiedError@Base" 0.24~rc0 + (c++)"typeinfo for Xapian::DatabaseOpeningError@Base" 0.32~rc0 + (c++|optional=present with Xapian 1.4)"typeinfo for Xapian::QueryParserError@Base" 0.23~rc0 + (c++)"typeinfo name for Xapian::LogicError@Base" 0.6.1 + (c++)"typeinfo name for Xapian::RuntimeError@Base" 0.6.1 + (c++)"typeinfo name for Xapian::DocNotFoundError@Base" 0.6.1 + (c++)"typeinfo name for Xapian::InvalidArgumentError@Base" 0.6.1 + (c++)"typeinfo name for Xapian::Error@Base" 0.6.1 + (c++)"typeinfo name for Xapian::DatabaseError@Base" 0.24~rc0 + (c++)"typeinfo name for Xapian::DatabaseModifiedError@Base" 0.24~rc0 + (c++)"typeinfo name for Xapian::DatabaseOpeningError@Base" 0.32~rc0 + (c++|optional=present with Xapian 1.4)"typeinfo name for Xapian::QueryParserError@Base" 0.23~rc0 diff --git a/debian/not-installed b/debian/not-installed new file mode 100644 index 00000000..fd929459 --- /dev/null +++ b/debian/not-installed @@ -0,0 +1,3 @@ +usr/share/applications/mimeinfo.cache +usr/share/info/dir +usr/share/emacs/site-lisp/*.elc diff --git a/debian/notmuch-doc.install b/debian/notmuch-doc.install new file mode 100644 index 00000000..fa902fe1 --- /dev/null +++ b/debian/notmuch-doc.install @@ -0,0 +1 @@ +doc/_build/html usr/share/doc/notmuch diff --git a/debian/notmuch-emacs.maintscript b/debian/notmuch-emacs.maintscript index 6f93feb7..8e3004ec 100644 --- a/debian/notmuch-emacs.maintscript +++ b/debian/notmuch-emacs.maintscript @@ -1 +1 @@ -rm_conffile /etc/emacs/site-start.d/50notmuch.el +rm_conffile /etc/emacs/site-start.d/50notmuch.el 0.33-1~ diff --git a/debian/notmuch-git.install b/debian/notmuch-git.install new file mode 100644 index 00000000..2be08276 --- /dev/null +++ b/debian/notmuch-git.install @@ -0,0 +1,2 @@ +notmuch-git /usr/bin +nmbug /usr/bin diff --git a/debian/notmuch-git.manpages b/debian/notmuch-git.manpages new file mode 100644 index 00000000..e0895c86 --- /dev/null +++ b/debian/notmuch-git.manpages @@ -0,0 +1,2 @@ +usr/share/man/man1/notmuch-git.1.gz +usr/share/man/man1/nmbug.1.gz diff --git a/debian/notmuch-mutt.install b/debian/notmuch-mutt.install index 9b468bdb..8314f883 100644 --- a/debian/notmuch-mutt.install +++ b/debian/notmuch-mutt.install @@ -1,2 +1,2 @@ -usr/bin/notmuch-mutt etc/Muttrc.d/notmuch-mutt.rc +usr/bin/notmuch-mutt diff --git a/debian/notmuch-vim.dirs b/debian/notmuch-vim.dirs index c6373e42..2b531314 100644 --- a/debian/notmuch-vim.dirs +++ b/debian/notmuch-vim.dirs @@ -1,4 +1,4 @@ -usr/share/vim/registry -usr/share/vim/addons/plugin usr/share/vim/addons/doc +usr/share/vim/addons/plugin usr/share/vim/addons/syntax +usr/share/vim/registry diff --git a/debian/notmuch-vim.install b/debian/notmuch-vim.install index a1af708d..cf898738 100644 --- a/debian/notmuch-vim.install +++ b/debian/notmuch-vim.install @@ -1,4 +1,4 @@ -vim/notmuch.vim usr/share/vim/addons/plugin vim/notmuch.txt usr/share/vim/addons/doc -vim/syntax/notmuch-*.vim usr/share/vim/addons/syntax +vim/notmuch.vim usr/share/vim/addons/plugin vim/notmuch.yaml usr/share/vim/registry +vim/syntax/notmuch-*.vim usr/share/vim/addons/syntax diff --git a/debian/notmuch.install b/debian/notmuch.install index 0cce21bd..60f09712 100644 --- a/debian/notmuch.install +++ b/debian/notmuch.install @@ -1,5 +1,5 @@ usr/bin/notmuch usr/bin/notmuch-emacs-mua +usr/share/applications/notmuch-emacs-mua.desktop usr/share/bash-completion usr/share/zsh/vendor-completions -emacs/notmuch-emacs-mua.desktop usr/share/applications diff --git a/debian/notmuch.maintscript b/debian/notmuch.maintscript index 6f93feb7..8e3004ec 100644 --- a/debian/notmuch.maintscript +++ b/debian/notmuch.maintscript @@ -1 +1 @@ -rm_conffile /etc/emacs/site-start.d/50notmuch.el +rm_conffile /etc/emacs/site-start.d/50notmuch.el 0.33-1~ diff --git a/debian/notmuch.manpages b/debian/notmuch.manpages index f9fcb54a..d7e8cf59 100644 --- a/debian/notmuch.manpages +++ b/debian/notmuch.manpages @@ -1,18 +1,20 @@ -usr/share/man/man5/notmuch-hooks.5.gz -usr/share/man/man1/notmuch-dump.1.gz -usr/share/man/man1/notmuch-count.1.gz +usr/share/man/man1/notmuch-address.1.gz usr/share/man/man1/notmuch-compact.1.gz +usr/share/man/man1/notmuch-config.1.gz +usr/share/man/man1/notmuch-count.1.gz +usr/share/man/man1/notmuch-dump.1.gz usr/share/man/man1/notmuch-emacs-mua.1.gz +usr/share/man/man1/notmuch-insert.1.gz usr/share/man/man1/notmuch-new.1.gz -usr/share/man/man1/notmuch.1.gz usr/share/man/man1/notmuch-reindex.1.gz -usr/share/man/man1/notmuch-address.1.gz -usr/share/man/man1/notmuch-tag.1.gz usr/share/man/man1/notmuch-reply.1.gz -usr/share/man/man1/notmuch-search.1.gz usr/share/man/man1/notmuch-restore.1.gz -usr/share/man/man1/notmuch-insert.1.gz +usr/share/man/man1/notmuch-search.1.gz +usr/share/man/man1/notmuch-setup.1.gz usr/share/man/man1/notmuch-show.1.gz -usr/share/man/man1/notmuch-config.1.gz +usr/share/man/man1/notmuch-tag.1.gz +usr/share/man/man1/notmuch.1.gz +usr/share/man/man5/notmuch-hooks.5.gz usr/share/man/man7/notmuch-properties.7.gz usr/share/man/man7/notmuch-search-terms.7.gz +usr/share/man/man7/notmuch-sexp-queries.7.gz diff --git a/debian/python-notmuch.install b/debian/python-notmuch.install deleted file mode 100644 index b2cc1360..00000000 --- a/debian/python-notmuch.install +++ /dev/null @@ -1 +0,0 @@ -usr/lib/python2* diff --git a/debian/python3-notmuch.install b/debian/python3-notmuch.install deleted file mode 100644 index 4606faae..00000000 --- a/debian/python3-notmuch.install +++ /dev/null @@ -1 +0,0 @@ -usr/lib/python3* diff --git a/debian/rules b/debian/rules index ebd10481..a77ffa15 100755 --- a/debian/rules +++ b/debian/rules @@ -1,15 +1,18 @@ #!/usr/bin/make -f +include /usr/share/dpkg/architecture.mk -python3_all = py3versions -s | xargs -n1 | xargs -t -I {} env {} +ifeq ($(DEB_HOST_ARCH),ppc64el) + export NOTMUCH_SKIP_TESTS = T810-tsan +endif export DEB_BUILD_MAINT_OPTIONS = hardening=+all %: - dh $@ --with python2,python3,elpa + dh $@ --with python3,elpa,sphinxdoc override_dh_auto_configure: BASHCMD=/bin/bash ./configure --prefix=/usr \ - --libdir=/usr/lib/$$(dpkg-architecture -q DEB_TARGET_MULTIARCH) \ + --libdir=/usr/lib/${DEB_TARGET_MULTIARCH} \ --includedir=/usr/include \ --mandir=/usr/share/man \ --infodir=/usr/share/info \ @@ -18,21 +21,20 @@ override_dh_auto_configure: --localstatedir=/var override_dh_auto_build: - dh_auto_build -- V=1 - dh_auto_build --sourcedirectory bindings/python - cd bindings/python && $(python3_all) setup.py build + dh_auto_build -- V=1 all sphinx-html + PYBUILD_NAME=notmuch dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python + PYBUILD_NAME=notmuch2 dh_auto_build --buildsystem=pybuild --sourcedirectory bindings/python-cffi $(MAKE) -C contrib/notmuch-mutt override_dh_auto_clean: dh_auto_clean - dh_auto_clean --sourcedirectory bindings/python - cd bindings/python && $(python3_all) setup.py clean -a + PYBUILD_NAME=notmuch dh_auto_clean --buildsystem=pybuild --sourcedirectory bindings/python dh_auto_clean --sourcedirectory bindings/ruby $(MAKE) -C contrib/notmuch-mutt clean override_dh_auto_install: dh_auto_install - dh_auto_install --sourcedirectory bindings/python - cd bindings/python && $(python3_all) setup.py install --install-layout=deb --root=$(CURDIR)/debian/tmp + PYBUILD_NAME=notmuch dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python + PYBUILD_NAME=notmuch2 dh_auto_install --buildsystem=pybuild --sourcedirectory bindings/python-cffi $(MAKE) -C contrib/notmuch-mutt DESTDIR=$(CURDIR)/debian/tmp install dh_auto_install --sourcedirectory bindings/ruby diff --git a/debian/tests/control b/debian/tests/control new file mode 100644 index 00000000..80be1deb --- /dev/null +++ b/debian/tests/control @@ -0,0 +1,18 @@ +Test-command: env NOTMUCH_TEST_INSTALLED=1 TERM=dumb + NOTMUCH_HAVE_MAN=1 NOTMUCH_HAVE_SFSEXP=1 NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK=1 + NOTMUCH_HAVE_PYTHON3_CFFI=1 NOTMUCH_HAVE_PYTHON3_PYTEST=1 + NOTMUCH_HAVE_ASAN=1 NOTMUCH_HAVE_TSAN=1 + ./test/notmuch-test +Restrictions: allow-stderr +Architecture: amd64, arm64 +Depends: @, + build-essential, + dtach, + emacs-nox, + gdb, + git, + gnupg, + gpgsm, + libtalloc-dev, + man, + xapian-tools diff --git a/debian/upstream/metadata b/debian/upstream/metadata new file mode 100644 index 00000000..8f266aa8 --- /dev/null +++ b/debian/upstream/metadata @@ -0,0 +1,6 @@ +Bug-Database: https://nmbug.notmuchmail.org/status/ +Bug-Submit: mailto:notmuch@notmuchmail.org +FAQ: https://notmuchmail.org/faq/ +Repository: https://git.notmuchmail.org/git/notmuch +Repository-Browse: https://git.notmuchmail.org/git/notmuch +Screenshots: https://notmuchmail.org/screenshots/ diff --git a/devel/author-scan.sh b/devel/author-scan.sh new file mode 100644 index 00000000..23854f39 --- /dev/null +++ b/devel/author-scan.sh @@ -0,0 +1,11 @@ +#!/bin/sh + +FILE_EXCLUDE='corpora' +AUTHOR_EXCLUDE='uncrustify' +# based on the FSF guideline, for want of a better idea. +THRESHOLD=15 + +git ls-files | grep -v -e "$FILE_EXCLUDE" | tr '\n' '\0' | xargs -0 -n 1 \ + git blame -w --line-porcelain -- | \ + sed -n "/$AUTHOR_EXCLUDE/d; s/^[aA][uU][tT][hH][Oo][rR] //p" | \ + sort -fd | uniq -ic | awk "\$1 >= $THRESHOLD" | sort -nr diff --git a/devel/check-notmuch-commit b/devel/check-notmuch-commit new file mode 100755 index 00000000..eca5fb96 --- /dev/null +++ b/devel/check-notmuch-commit @@ -0,0 +1,31 @@ +#!/bin/sh + +# Usage suggestion: +# git rebase -i --exec devel/check-notmuch-commit origin/master + +set -e + +quick=0 +case "$1" in + -q|-Q|--quick) + quick=1 + ;; +esac + +if [ $quick = 0 ]; then + make test +fi + +unset uconf +for file in $(git diff --name-only --diff-filter=AM HEAD^); do + case $file in + *.c|*.h|*.cc|*.hh) + uncrustify --replace -c "${uconf=$(dirname "$0")/uncrustify.cfg}" "$file" + ;; + *.el) + emacs -Q --batch "$file" --eval '(indent-region (point-min) (point-max) nil)' -f save-buffer + ;; + esac +done + +git diff --quiet diff --git a/devel/emacs-keybindings.org b/devel/emacs-keybindings.org index 464b9467..218677c2 100644 --- a/devel/emacs-keybindings.org +++ b/devel/emacs-keybindings.org @@ -1,58 +1,60 @@ -|-----------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| -| Key | Search Mode | Show Mode | Tree Mode | -|-----------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| -| a | notmuch-search-archive-thread | notmuch-show-archive-message-then-next-or-next-thread | notmuch-tree-archive-message-then-next | -| b | notmuch-search-scroll-down | notmuch-show-resend-message | notmuch-show-resend-message | -| c | notmuch-search-stash-map | notmuch-show-stash-map | notmuch-show-stash-map | -| d | | | | -| e | | | (notmuch-tree-button-activate) | -| f | | notmuch-show-forward-message | notmuch-show-forward-message | -| g | | | | -| h | | notmuch-show-toggle-visibility-headers | | -| i | | | | -| j | notmuch-jump-search | notmuch-jump-search | notmuch-jump-search | -| k | notmuch-tag-jump | notmuch-tag-jump | notmuch-tag-jump | -| l | notmuch-search-filter | notmuch-show-filter-thread | | -| m | notmuch-mua-new-mail | notmuch-mua-new-mail | notmuch-mua-new-mail | -| n | notmuch-search-next-thread | notmuch-show-next-open-message | notmuch-tree-next-matching-message | -| o | notmuch-search-toggle-order | | | -| p | notmuch-search-previous-thread | notmuch-show-previous-open-message | notmuch-tree-prev-matching-message | -| q | notmuch-bury-or-kill-this-buffer | notmuch-bury-or-kill-this-buffer | notmuch-bury-or-kill-this-buffer | -| r | notmuch-search-reply-to-thread-sender | notmuch-show-reply-sender | notmuch-show-reply-sender | -| s | notmuch-search | notmuch-search | notmuch-search | -| t | notmuch-search-filter-by-tag | toggle-truncate-lines | | -| u | | | | -| v | | | notmuch-show-view-all-mime-parts | -| w | | notmuch-show-save-attachments | notmuch-show-save-attachments | -| x | notmuch-bury-or-kill-this-buffer | notmuch-show-archive-message-then-next-or-exit | notmuch-tree-quit | -| y | | | | -| z | notmuch-tree | notmuch-tree | notmuch-tree-to-tree | -| A | | notmuch-show-archive-thread-then-next | notmuch-tree-archive-thread | -| F | | notmuch-show-forward-open-messages | | -| G | notmuch-poll-and-refresh-this-buffer | notmuch-poll-and-refresh-this-buffer | notmuch-poll-and-refresh-this-buffer | -| N | | notmuch-show-next-message | notmuch-tree-next-message | -| O | | | | -| P | | notmuch-show-previous-message | notmuch-tree-prev-message | -| R | notmuch-search-reply-to-thread | notmuch-show-reply | notmuch-show-reply | -| S | | | notmuch-search-from-tree-current-query | -| V | | notmuch-show-view-raw-message | notmuch-show-view-raw-message | -| X | | notmuch-show-archive-thread-then-exit | | -| Z | notmuch-tree-from-search-current-query | notmuch-tree-from-show-current-query | | -| =!= | | notmuch-show-toggle-elide-non-matching | | -| =#= | | notmuch-show-print-message | | -| =$= | | notmuch-show-toggle-process-crypto | | -| =*= | notmuch-search-tag-all | notmuch-show-tag-all | notmuch-tree-tag-thread | -| + | notmuch-search-add-tag | notmuch-show-add-tag | notmuch-tree-add-tag | -| - | notmuch-search-remove-tag | notmuch-show-remove-tag | notmuch-tree-remove-tag | -| . | | notmuch-show-part-map | | -| < | notmuch-search-first-thread | notmuch-show-toggle-thread-indentation | | -| | notmuch-search-scroll-down | notmuch-show-rewind | notmuch-tree-scroll-message-window-back | -| | notmuch-search-show-thread | notmuch-show-toggle-message | notmuch-tree-show-message | -| | notmuch-search-scroll-up | notmuch-show-advance | notmuch-tree-scroll-or-next | -| | | notmuch-show-next-button | notmuch-show-next-button | -| | | notmuch-show-previous-button | notmuch-show-previous-button | -| = | notmuch-refresh-this-buffer | notmuch-refresh-this-buffer | notmuch-tree-refresh-view | -| > | notmuch-search-last-thread | | | -| ? | notmuch-help | notmuch-help | notmuch-help | -| \vert | | notmuch-show-pipe-message | notmuch-show-pipe-message | -|-----------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| +|--------------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| +| Key | Search Mode | Show Mode | Tree Mode | +|--------------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| +| a | notmuch-search-archive-thread | notmuch-show-archive-message-then-next-or-next-thread | notmuch-tree-archive-message-then-next | +| b | notmuch-search-scroll-down | notmuch-show-resend-message | notmuch-show-resend-message | +| c | notmuch-search-stash-map | notmuch-show-stash-map | notmuch-show-stash-map | +| d | | | | +| e | | | (notmuch-tree-button-activate) | +| f | | notmuch-show-forward-message | notmuch-show-forward-message | +| g | | | | +| h | | notmuch-show-toggle-visibility-headers | | +| i | notmuch-search-toggle-hide-excluded | | notmuch-tree-toggle-hide-excluded | +| j | notmuch-jump-search | notmuch-jump-search | notmuch-jump-search | +| k | notmuch-tag-jump | notmuch-tag-jump | notmuch-tag-jump | +| l | notmuch-search-filter | notmuch-show-filter-thread | notmuch-tree-filter | +| m | notmuch-mua-new-mail | notmuch-mua-new-mail | notmuch-mua-new-mail | +| n | notmuch-search-next-thread | notmuch-show-next-open-message | notmuch-tree-next-matching-message | +| o | notmuch-search-toggle-order | | notmuch-tree-toggle-order | +| p | notmuch-search-previous-thread | notmuch-show-previous-open-message | notmuch-tree-prev-matching-message | +| q | notmuch-bury-or-kill-this-buffer | notmuch-bury-or-kill-this-buffer | notmuch-bury-or-kill-this-buffer | +| r | notmuch-search-reply-to-thread-sender | notmuch-show-reply-sender | notmuch-show-reply-sender | +| s | notmuch-search | notmuch-search | notmuch-search | +| t | notmuch-search-filter-by-tag | toggle-truncate-lines | notmuch-tree-filter-by-tag | +| u | | | | +| v | | | notmuch-show-view-all-mime-parts | +| w | | notmuch-show-save-attachments | notmuch-show-save-attachments | +| x | notmuch-bury-or-kill-this-buffer | notmuch-show-archive-message-then-next-or-exit | notmuch-tree-quit | +| y | | | | +| z | notmuch-tree | notmuch-tree | notmuch-tree-to-tree | +| A | | notmuch-show-archive-thread-then-next | notmuch-tree-archive-thread | +| F | | notmuch-show-forward-open-messages | | +| G | notmuch-poll-and-refresh-this-buffer | notmuch-poll-and-refresh-this-buffer | notmuch-poll-and-refresh-this-buffer | +| N | | notmuch-show-next-message | notmuch-tree-next-message | +| O | | | | +| P | | notmuch-show-previous-message | notmuch-tree-prev-message | +| R | notmuch-search-reply-to-thread | notmuch-show-reply | notmuch-show-reply | +| S | | | notmuch-search-from-tree-current-query | +| V | | notmuch-show-view-raw-message | notmuch-show-view-raw-message | +| X | | notmuch-show-archive-thread-then-exit | | +| Z | notmuch-tree-from-search-current-query | notmuch-tree-from-show-current-query | | +| =!= | | notmuch-show-toggle-elide-non-matching | | +| =#= | | notmuch-show-print-message | | +| =%= | | notmuch-show-replace-msg | | +| =$= | | notmuch-show-toggle-process-crypto | | +| =*= | notmuch-search-tag-all | notmuch-show-tag-all | notmuch-tree-tag-thread | +| + | notmuch-search-add-tag | notmuch-show-add-tag | notmuch-tree-add-tag | +| - | notmuch-search-remove-tag | notmuch-show-remove-tag | notmuch-tree-remove-tag | +| . | | notmuch-show-part-map | | +| < | notmuch-search-first-thread | notmuch-show-toggle-thread-indentation | | +| | notmuch-search-scroll-down | notmuch-show-rewind | notmuch-tree-scroll-message-window-back | +| | notmuch-search-show-thread | notmuch-show-toggle-message | notmuch-tree-show-message | +| | notmuch-search-scroll-up | notmuch-show-advance | notmuch-tree-scroll-or-next | +| | | notmuch-show-next-button | notmuch-show-next-button | +| | | notmuch-show-previous-button | notmuch-show-previous-button | +| = | notmuch-refresh-this-buffer | notmuch-refresh-this-buffer | notmuch-tree-refresh-view | +| > | notmuch-search-last-thread | | | +| ? | notmuch-help | notmuch-help | notmuch-help | +| \vert | | notmuch-show-pipe-message | notmuch-show-pipe-message | +| [remap undo] | notmuch-tag-undo | notmuch-tag-undo | notmuch-tag-undo | +|--------------+----------------------------------------+-------------------------------------------------------+-----------------------------------------| diff --git a/devel/gen-testdb.sh b/devel/gen-testdb.sh deleted file mode 100755 index 61ae48a3..00000000 --- a/devel/gen-testdb.sh +++ /dev/null @@ -1,131 +0,0 @@ -#!/usr/bin/env bash -# -# NAME -# gen-testdb.sh - generate test databases -# -# SYNOPSIS -# gen-testdb.sh -v NOTMUCH-VERSION [-c CORPUS-PATH] [-s TAR-SUFFIX] -# -# DESCRIPTION -# Generate a tarball containing the specified test corpus and -# the corresponding notmuch database, indexed using a specific -# version of notmuch, resulting in a specific version of the -# database. -# -# The specific version of notmuch will be built on the fly. -# Therefore the script must be run within a git repository to be -# able to build the old versions of notmuch. -# -# This script reuses the test infrastructure, and the script -# must be run from within the test directory. -# -# The output tarballs, named database-.tar.gz, are -# placed in the test/test-databases directory. -# -# OPTIONS -# -v NOTMUCH-VERSION -# Notmuch version in terms of a git tag or commit to use -# for generating the database. Required. -# -# -c CORPUS-PATH -# Path to a corpus to use for generating the -# database. Due to CWD changes within the test -# infrastructure, use absolute paths. Defaults to the -# test corpus. -# -# -s TAR-SUFFIX -# Suffix for the tarball basename. Empty by default. -# -# EXAMPLE -# -# Generate a database indexed with notmuch 0.17. Use the default -# test corpus. Name the tarball database-v1.tar.gz to reflect -# the fact that notmuch 0.17 used database version 1. -# -# $ cd test -# $ ../devel/gen-testdb.sh -v 0.17 -s v1 -# -# CAVEATS -# Test infrastructure options won't work. -# -# Any existing databases with the same name will be overwritten. -# -# It may not be possible to build old versions of notmuch with -# the set of dependencies that satisfy building the current -# version of notmuch. -# -# AUTHOR -# Jani Nikula -# -# LICENSE -# Same as notmuch test infrastructure (GPLv2+). -# - -test_description="database generation abusing test infrastructure" - -# immediate exit on subtest failure; see test_failure_ in test-lib.sh -immediate=t - -VERSION= -CORPUS= -SUFFIX= - -while getopts v:c:s: opt; do - case "$opt" in - v) VERSION="$OPTARG";; - c) CORPUS="$OPTARG";; - s) SUFFIX="-$OPTARG";; - esac -done -shift `expr $OPTIND - 1` - -. ./test-lib.sh || exit 1 - -SHORT_CORPUS=$(basename ${CORPUS:-database}) -DBNAME=${SHORT_CORPUS}${SUFFIX} -TARBALLNAME=${DBNAME}.tar.xz - -CORPUS=${CORPUS:-${TEST_DIRECTORY}/corpus} - -test_expect_code 0 "notmuch version specified on the command line" \ - "test -n ${VERSION}" - -test_expect_code 0 "the specified version ${VERSION} refers to a commit" \ - "git show ${VERSION} >/dev/null 2>&1" - -BUILD_DIR="notmuch-${VERSION}" -test_expect_code 0 "generate snapshot of notmuch version ${VERSION}" \ - "git -C $TEST_DIRECTORY/.. archive --prefix=${BUILD_DIR}/ --format=tar ${VERSION} | tar x" - -# force version string -git describe --match '[0-9.]*' ${VERSION} > ${BUILD_DIR}/version - -test_expect_code 0 "configure and build notmuch version ${VERSION}" \ - "make -C ${BUILD_DIR}" - -# use the newly built notmuch -export PATH=./${BUILD_DIR}:$PATH - -test_begin_subtest "verify the newly built notmuch version" -test_expect_equal "`notmuch --version`" "notmuch `cat ${BUILD_DIR}/version`" - -# replace the existing mails, if any, with the specified corpus -rm -rf ${MAIL_DIR} -cp -a ${CORPUS} ${MAIL_DIR} - -test_expect_code 0 "index the corpus" \ - "notmuch new" - -# wrap the resulting mail store and database in a tarball - -cp -a ${MAIL_DIR} ${TMP_DIRECTORY}/${DBNAME} -tar Jcf ${TMP_DIRECTORY}/${TARBALLNAME} -C ${TMP_DIRECTORY} ${DBNAME} -mkdir -p ${TEST_DIRECTORY}/test-databases -cp -a ${TMP_DIRECTORY}/${TARBALLNAME} ${TEST_DIRECTORY}/test-databases -test_expect_code 0 "create the output tarball ${TARBALLNAME}" \ - "test -f ${TEST_DIRECTORY}/test-databases/${TARBALLNAME}" - -# generate a checksum file -test_expect_code 0 "compute checksum" \ - "(cd ${TEST_DIRECTORY}/test-databases/ && sha256sum ${TARBALLNAME} > ${TARBALLNAME}.sha256)" -test_done diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug deleted file mode 100755 index c35dd75d..00000000 --- a/devel/nmbug/nmbug +++ /dev/null @@ -1,852 +0,0 @@ -#!/usr/bin/env python -# -# Copyright (c) 2011-2014 David Bremner -# W. Trevor King -# -# This program is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see https://www.gnu.org/licenses/ . - -""" -Manage notmuch tags with Git - -Environment variables: - -* NMBGIT specifies the location of the git repository used by nmbug. - If not specified $HOME/.nmbug is used. -* NMBPREFIX specifies the prefix in the notmuch database for tags of - interest to nmbug. If not specified 'notmuch::' is used. -""" - -from __future__ import print_function -from __future__ import unicode_literals - -import codecs as _codecs -import collections as _collections -import functools as _functools -import inspect as _inspect -import locale as _locale -import logging as _logging -import os as _os -import re as _re -import shutil as _shutil -import subprocess as _subprocess -import sys as _sys -import tempfile as _tempfile -import textwrap as _textwrap -try: # Python 3 - from urllib.parse import quote as _quote - from urllib.parse import unquote as _unquote -except ImportError: # Python 2 - from urllib import quote as _quote - from urllib import unquote as _unquote - - -__version__ = '0.3' - -_LOG = _logging.getLogger('nmbug') -_LOG.setLevel(_logging.WARNING) -_LOG.addHandler(_logging.StreamHandler()) - -NMBGIT = _os.path.expanduser( - _os.getenv('NMBGIT', _os.path.join('~', '.nmbug'))) -_NMBGIT = _os.path.join(NMBGIT, '.git') -if _os.path.isdir(_NMBGIT): - NMBGIT = _NMBGIT - -TAG_PREFIX = _os.getenv('NMBPREFIX', 'notmuch::') -_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}') -_TAG_DIRECTORY = 'tags/' -_TAG_FILE_REGEX = _re.compile(_TAG_DIRECTORY + '(?P[^/]*)/(?P[^/]*)') - -# magic hash for Git (git hash-object -t blob /dev/null) -_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391' - - -try: - getattr(_tempfile, 'TemporaryDirectory') -except AttributeError: # Python < 3.2 - class _TemporaryDirectory(object): - """ - Fallback context manager for Python < 3.2 - - See PEP 343 for details on context managers [1]. - - [1]: https://www.python.org/dev/peps/pep-0343/ - """ - def __init__(self, **kwargs): - self.name = _tempfile.mkdtemp(**kwargs) - - def __enter__(self): - return self.name - - def __exit__(self, type, value, traceback): - _shutil.rmtree(self.name) - - - _tempfile.TemporaryDirectory = _TemporaryDirectory - - -def _hex_quote(string, safe='+@=:,'): - """ - quote('abc def') -> 'abc%20def'. - - Wrap urllib.parse.quote with additional safe characters (in - addition to letters, digits, and '_.-') and lowercase hex digits - (e.g. '%3a' instead of '%3A'). - """ - uppercase_escapes = _quote(string, safe) - return _HEX_ESCAPE_REGEX.sub( - lambda match: match.group(0).lower(), - uppercase_escapes) - - -_ENCODED_TAG_PREFIX = _hex_quote(TAG_PREFIX, safe='+@=,') # quote ':' - - -def _xapian_quote(string): - """ - Quote a string for Xapian's QueryParser. - - Xapian uses double-quotes for quoting strings. You can escape - internal quotes by repeating them [1,2,3]. - - [1]: https://trac.xapian.org/ticket/128#comment:2 - [2]: https://trac.xapian.org/ticket/128#comment:17 - [3]: https://trac.xapian.org/changeset/13823/svn - """ - return '"{0}"'.format(string.replace('"', '""')) - - -def _xapian_unquote(string): - """ - Unquote a Xapian-quoted string. - """ - if string.startswith('"') and string.endswith('"'): - return string[1:-1].replace('""', '"') - return string - - -class SubprocessError(RuntimeError): - "A subprocess exited with a nonzero status" - def __init__(self, args, status, stdout=None, stderr=None): - self.status = status - self.stdout = stdout - self.stderr = stderr - msg = '{args} exited with {status}'.format(args=args, status=status) - if stderr: - msg = '{msg}: {stderr}'.format(msg=msg, stderr=stderr) - super(SubprocessError, self).__init__(msg) - - -class _SubprocessContextManager(object): - """ - PEP 343 context manager for subprocesses. - - 'expect' holds a tuple of acceptable exit codes, otherwise we'll - raise a SubprocessError in __exit__. - """ - def __init__(self, process, args, expect=(0,)): - self._process = process - self._args = args - self._expect = expect - - def __enter__(self): - return self._process - - def __exit__(self, type, value, traceback): - for name in ['stdin', 'stdout', 'stderr']: - stream = getattr(self._process, name) - if stream: - stream.close() - setattr(self._process, name, None) - status = self._process.wait() - _LOG.debug( - 'collect {args} with status {status} (expected {expect})'.format( - args=self._args, status=status, expect=self._expect)) - if status not in self._expect: - raise SubprocessError(args=self._args, status=status) - - def wait(self): - return self._process.wait() - - -def _spawn(args, input=None, additional_env=None, wait=False, stdin=None, - stdout=None, stderr=None, encoding=_locale.getpreferredencoding(), - expect=(0,), **kwargs): - """Spawn a subprocess, and optionally wait for it to finish. - - This wrapper around subprocess.Popen has two modes, depending on - the truthiness of 'wait'. If 'wait' is true, we use p.communicate - internally to write 'input' to the subprocess's stdin and read - from it's stdout/stderr. If 'wait' is False, we return a - _SubprocessContextManager instance for fancier handling - (e.g. piping between processes). - - For 'wait' calls when you want to write to the subprocess's stdin, - you only need to set 'input' to your content. When 'input' is not - None but 'stdin' is, we'll automatically set 'stdin' to PIPE - before calling Popen. This avoids having the subprocess - accidentally inherit the launching process's stdin. - """ - _LOG.debug('spawn {args} (additional env. var.: {env})'.format( - args=args, env=additional_env)) - if not stdin and input is not None: - stdin = _subprocess.PIPE - if additional_env: - if not kwargs.get('env'): - kwargs['env'] = dict(_os.environ) - kwargs['env'].update(additional_env) - p = _subprocess.Popen( - args, stdin=stdin, stdout=stdout, stderr=stderr, **kwargs) - if wait: - if hasattr(input, 'encode'): - input = input.encode(encoding) - (stdout, stderr) = p.communicate(input=input) - status = p.wait() - _LOG.debug( - 'collect {args} with status {status} (expected {expect})'.format( - args=args, status=status, expect=expect)) - if stdout is not None: - stdout = stdout.decode(encoding) - if stderr is not None: - stderr = stderr.decode(encoding) - if status not in expect: - raise SubprocessError( - args=args, status=status, stdout=stdout, stderr=stderr) - return (status, stdout, stderr) - if p.stdin and not stdin: - p.stdin.close() - p.stdin = None - if p.stdin: - p.stdin = _codecs.getwriter(encoding=encoding)(stream=p.stdin) - stream_reader = _codecs.getreader(encoding=encoding) - if p.stdout: - p.stdout = stream_reader(stream=p.stdout) - if p.stderr: - p.stderr = stream_reader(stream=p.stderr) - return _SubprocessContextManager(args=args, process=p, expect=expect) - - -def _git(args, **kwargs): - args = ['git', '--git-dir', NMBGIT] + list(args) - return _spawn(args=args, **kwargs) - - -def _get_current_branch(): - """Get the name of the current branch. - - Return 'None' if we're not on a branch. - """ - try: - (status, branch, stderr) = _git( - args=['symbolic-ref', '--short', 'HEAD'], - stdout=_subprocess.PIPE, stderr=_subprocess.PIPE, wait=True) - except SubprocessError as e: - if 'not a symbolic ref' in e: - return None - raise - return branch.strip() - - -def _get_remote(): - "Get the default remote for the current branch." - local_branch = _get_current_branch() - (status, remote, stderr) = _git( - args=['config', 'branch.{0}.remote'.format(local_branch)], - stdout=_subprocess.PIPE, wait=True) - return remote.strip() - - -def get_tags(prefix=None): - "Get a list of tags with a given prefix." - if prefix is None: - prefix = TAG_PREFIX - (status, stdout, stderr) = _spawn( - args=['notmuch', 'search', '--output=tags', '*'], - stdout=_subprocess.PIPE, wait=True) - return [tag for tag in stdout.splitlines() if tag.startswith(prefix)] - - -def archive(treeish='HEAD', args=()): - """ - Dump a tar archive of the current nmbug tag set. - - Using 'git archive'. - - Each tag $tag for message with Message-Id $id is written to - an empty file - - tags/encode($id)/encode($tag) - - The encoding preserves alphanumerics, and the characters - "+-_@=.:," (not the quotes). All other octets are replaced with - '%' followed by a two digit hex number. - """ - _git(args=['archive', treeish] + list(args), wait=True) - - -def clone(repository): - """ - Create a local nmbug repository from a remote source. - - This wraps 'git clone', adding some options to avoid creating a - working tree while preserving remote-tracking branches and - upstreams. - """ - with _tempfile.TemporaryDirectory(prefix='nmbug-clone.') as workdir: - _spawn( - args=[ - 'git', 'clone', '--no-checkout', '--separate-git-dir', NMBGIT, - repository, workdir], - wait=True) - _git(args=['config', '--unset', 'core.worktree'], wait=True, expect=(0, 5)) - _git(args=['config', 'core.bare', 'true'], wait=True) - _git(args=['branch', 'config', 'origin/config'], wait=True) - existing_tags = get_tags() - if existing_tags: - _LOG.warning( - 'Not checking out to avoid clobbering existing tags: {}'.format( - ', '.join(existing_tags))) - else: - checkout() - - -def _is_committed(status): - return len(status['added']) + len(status['deleted']) == 0 - - -def commit(treeish='HEAD', message=None): - """ - Commit prefix-matching tags from the notmuch database to Git. - """ - status = get_status() - - if _is_committed(status=status): - _LOG.warning('Nothing to commit') - return - - _git(args=['read-tree', '--empty'], wait=True) - _git(args=['read-tree', treeish], wait=True) - try: - _update_index(status=status) - (_, tree, _) = _git( - args=['write-tree'], - stdout=_subprocess.PIPE, - wait=True) - (_, parent, _) = _git( - args=['rev-parse', treeish], - stdout=_subprocess.PIPE, - wait=True) - (_, commit, _) = _git( - args=['commit-tree', tree.strip(), '-p', parent.strip()], - input=message, - stdout=_subprocess.PIPE, - wait=True) - _git( - args=['update-ref', treeish, commit.strip()], - stdout=_subprocess.PIPE, - wait=True) - except Exception as e: - _git(args=['read-tree', '--empty'], wait=True) - _git(args=['read-tree', treeish], wait=True) - raise - -def _update_index(status): - with _git( - args=['update-index', '--index-info'], - stdin=_subprocess.PIPE) as p: - for id, tags in status['deleted'].items(): - for line in _index_tags_for_message(id=id, status='D', tags=tags): - p.stdin.write(line) - for id, tags in status['added'].items(): - for line in _index_tags_for_message(id=id, status='A', tags=tags): - p.stdin.write(line) - - -def fetch(remote=None): - """ - Fetch changes from the remote repository. - - See 'merge' to bring those changes into notmuch. - """ - args = ['fetch'] - if remote: - args.append(remote) - _git(args=args, wait=True) - - -def checkout(): - """ - Update the notmuch database from Git. - - This is mainly useful to discard your changes in notmuch relative - to Git. - """ - status = get_status() - with _spawn( - args=['notmuch', 'tag', '--batch'], stdin=_subprocess.PIPE) as p: - for id, tags in status['added'].items(): - p.stdin.write(_batch_line(action='-', id=id, tags=tags)) - for id, tags in status['deleted'].items(): - p.stdin.write(_batch_line(action='+', id=id, tags=tags)) - - -def _batch_line(action, id, tags): - """ - 'notmuch tag --batch' line for adding/removing tags. - - Set 'action' to '-' to remove a tag or '+' to add the tags to a - given message id. - """ - tag_string = ' '.join( - '{action}{prefix}{tag}'.format( - action=action, prefix=_ENCODED_TAG_PREFIX, tag=_hex_quote(tag)) - for tag in tags) - line = '{tags} -- id:{id}\n'.format( - tags=tag_string, id=_xapian_quote(string=id)) - return line - - -def _insist_committed(): - "Die if the the notmuch tags don't match the current HEAD." - status = get_status() - if not _is_committed(status=status): - _LOG.error('\n'.join([ - 'Uncommitted changes to {prefix}* tags in notmuch', - '', - "For a summary of changes, run 'nmbug status'", - "To save your changes, run 'nmbug commit' before merging/pull", - "To discard your changes, run 'nmbug checkout'", - ]).format(prefix=TAG_PREFIX)) - _sys.exit(1) - - -def pull(repository=None, refspecs=None): - """ - Pull (merge) remote repository changes to notmuch. - - 'pull' is equivalent to 'fetch' followed by 'merge'. We use the - Git-configured repository for your current branch - (branch..repository, likely 'origin', and - branch..merge, likely 'master'). - """ - _insist_committed() - if refspecs and not repository: - repository = _get_remote() - args = ['pull'] - if repository: - args.append(repository) - if refspecs: - args.extend(refspecs) - with _tempfile.TemporaryDirectory(prefix='nmbug-pull.') as workdir: - for command in [ - ['reset', '--hard'], - args]: - _git( - args=command, - additional_env={'GIT_WORK_TREE': workdir}, - wait=True) - checkout() - - -def merge(reference='@{upstream}'): - """ - Merge changes from 'reference' into HEAD and load the result into notmuch. - - The default reference is '@{upstream}'. - """ - _insist_committed() - with _tempfile.TemporaryDirectory(prefix='nmbug-merge.') as workdir: - for command in [ - ['reset', '--hard'], - ['merge', reference]]: - _git( - args=command, - additional_env={'GIT_WORK_TREE': workdir}, - wait=True) - checkout() - - -def log(args=()): - """ - A simple wrapper for 'git log'. - - After running 'nmbug fetch', you can inspect the changes with - 'nmbug log HEAD..@{upstream}'. - """ - # we don't want output trapping here, because we want the pager. - args = ['log', '--name-status', '--no-renames'] + list(args) - with _git(args=args, expect=(0, 1, -13)) as p: - p.wait() - - -def push(repository=None, refspecs=None): - "Push the local nmbug Git state to a remote repository." - if refspecs and not repository: - repository = _get_remote() - args = ['push'] - if repository: - args.append(repository) - if refspecs: - args.extend(refspecs) - _git(args=args, wait=True) - - -def status(): - """ - Show pending updates in notmuch or git repo. - - Prints lines of the form - - ng Message-Id tag - - where n is a single character representing notmuch database status - - * A - - Tag is present in notmuch database, but not committed to nmbug - (equivalently, tag has been deleted in nmbug repo, e.g. by a - pull, but not restored to notmuch database). - - * D - - Tag is present in nmbug repo, but not restored to notmuch - database (equivalently, tag has been deleted in notmuch). - - * U - - Message is unknown (missing from local notmuch database). - - The second character (if present) represents a difference between - local and upstream branches. Typically 'nmbug fetch' needs to be - run to update this. - - * a - - Tag is present in upstream, but not in the local Git branch. - - * d - - Tag is present in local Git branch, but not upstream. - """ - status = get_status() - # 'output' is a nested defaultdict for message status: - # * The outer dict is keyed by message id. - # * The inner dict is keyed by tag name. - # * The inner dict values are status strings (' a', 'Dd', ...). - output = _collections.defaultdict( - lambda : _collections.defaultdict(lambda : ' ')) - for id, tags in status['added'].items(): - for tag in tags: - output[id][tag] = 'A' - for id, tags in status['deleted'].items(): - for tag in tags: - output[id][tag] = 'D' - for id, tags in status['missing'].items(): - for tag in tags: - output[id][tag] = 'U' - if _is_unmerged(): - for id, tag in _diff_refs(filter='A'): - output[id][tag] += 'a' - for id, tag in _diff_refs(filter='D'): - output[id][tag] += 'd' - for id, tag_status in sorted(output.items()): - for tag, status in sorted(tag_status.items()): - print('{status}\t{id}\t{tag}'.format( - status=status, id=id, tag=tag)) - - -def _is_unmerged(ref='@{upstream}'): - try: - (status, fetch_head, stderr) = _git( - args=['rev-parse', ref], - stdout=_subprocess.PIPE, stderr=_subprocess.PIPE, wait=True) - except SubprocessError as e: - if 'No upstream configured' in e.stderr: - return - raise - (status, base, stderr) = _git( - args=['merge-base', 'HEAD', ref], - stdout=_subprocess.PIPE, wait=True) - return base != fetch_head - - -def get_status(): - status = { - 'deleted': {}, - 'missing': {}, - } - index = _index_tags() - maybe_deleted = _diff_index(index=index, filter='D') - for id, tags in maybe_deleted.items(): - (_, stdout, stderr) = _spawn( - args=['notmuch', 'search', '--output=files', 'id:{0}'.format(id)], - stdout=_subprocess.PIPE, - wait=True) - if stdout: - status['deleted'][id] = tags - else: - status['missing'][id] = tags - status['added'] = _diff_index(index=index, filter='A') - _os.remove(index) - return status - - -def _index_tags(): - "Write notmuch tags to the nmbug.index." - path = _os.path.join(NMBGIT, 'nmbug.index') - query = ' '.join('tag:"{tag}"'.format(tag=tag) for tag in get_tags()) - prefix = '+{0}'.format(_ENCODED_TAG_PREFIX) - _git( - args=['read-tree', '--empty'], - additional_env={'GIT_INDEX_FILE': path}, wait=True) - with _spawn( - args=['notmuch', 'dump', '--format=batch-tag', '--', query], - stdout=_subprocess.PIPE) as notmuch: - with _git( - args=['update-index', '--index-info'], - stdin=_subprocess.PIPE, - additional_env={'GIT_INDEX_FILE': path}) as git: - for line in notmuch.stdout: - if line.strip().startswith('#'): - continue - (tags_string, id) = [_.strip() for _ in line.split(' -- id:')] - tags = [ - _unquote(tag[len(prefix):]) - for tag in tags_string.split() - if tag.startswith(prefix)] - id = _xapian_unquote(string=id) - for line in _index_tags_for_message( - id=id, status='A', tags=tags): - git.stdin.write(line) - return path - - -def _index_tags_for_message(id, status, tags): - """ - Update the Git index to either create or delete an empty file. - - Neither 'id' nor the tags in 'tags' should be encoded/escaped. - """ - mode = '100644' - hash = _EMPTYBLOB - - if status == 'D': - mode = '0' - hash = '0000000000000000000000000000000000000000' - - for tag in tags: - path = 'tags/{id}/{tag}'.format( - id=_hex_quote(string=id), tag=_hex_quote(string=tag)) - yield '{mode} {hash}\t{path}\n'.format(mode=mode, hash=hash, path=path) - - -def _diff_index(index, filter): - """ - Get an {id: {tag, ...}} dict for a given filter. - - For example, use 'A' to find added tags, and 'D' to find deleted tags. - """ - s = _collections.defaultdict(set) - with _git( - args=[ - 'diff-index', '--cached', '--diff-filter', filter, - '--name-only', 'HEAD'], - additional_env={'GIT_INDEX_FILE': index}, - stdout=_subprocess.PIPE) as p: - # Once we drop Python < 3.3, we can use 'yield from' here - for id, tag in _unpack_diff_lines(stream=p.stdout): - s[id].add(tag) - return s - - -def _diff_refs(filter, a='HEAD', b='@{upstream}'): - with _git( - args=['diff', '--diff-filter', filter, '--name-only', a, b], - stdout=_subprocess.PIPE) as p: - # Once we drop Python < 3.3, we can use 'yield from' here - for id, tag in _unpack_diff_lines(stream=p.stdout): - yield id, tag - - -def _unpack_diff_lines(stream): - "Iterate through (id, tag) tuples in a diff stream." - for line in stream: - match = _TAG_FILE_REGEX.match(line.strip()) - if not match: - message = 'non-tag line in diff: {!r}'.format(line.strip()) - if line.startswith(_TAG_DIRECTORY): - raise ValueError(message) - _LOG.info(message) - continue - id = _unquote(match.group('id')) - tag = _unquote(match.group('tag')) - yield (id, tag) - - -def _help(parser, command=None): - """ - Show help for an nmbug command. - - Because some folks prefer: - - $ nmbug help COMMAND - - to - - $ nmbug COMMAND --help - """ - if command: - parser.parse_args([command, '--help']) - else: - parser.parse_args(['--help']) - - -if __name__ == '__main__': - import argparse - - parser = argparse.ArgumentParser( - description=__doc__.strip(), - formatter_class=argparse.RawDescriptionHelpFormatter) - parser.add_argument( - '-v', '--version', action='version', - version='%(prog)s {}'.format(__version__)) - parser.add_argument( - '-l', '--log-level', - choices=['critical', 'error', 'warning', 'info', 'debug'], - help='Log verbosity. Defaults to {!r}.'.format( - _logging.getLevelName(_LOG.level).lower())) - - help = _functools.partial(_help, parser=parser) - help.__doc__ = _help.__doc__ - subparsers = parser.add_subparsers( - title='commands', - description=( - 'For help on a particular command, run: ' - "'%(prog)s ... --help'.")) - for command in [ - 'archive', - 'checkout', - 'clone', - 'commit', - 'fetch', - 'help', - 'log', - 'merge', - 'pull', - 'push', - 'status', - ]: - func = locals()[command] - doc = _textwrap.dedent(func.__doc__).strip().replace('%', '%%') - subparser = subparsers.add_parser( - command, - help=doc.splitlines()[0], - description=doc, - formatter_class=argparse.RawDescriptionHelpFormatter) - subparser.set_defaults(func=func) - if command == 'archive': - subparser.add_argument( - 'treeish', metavar='TREE-ISH', nargs='?', default='HEAD', - help=( - 'The tree or commit to produce an archive for. Defaults ' - "to 'HEAD'.")) - subparser.add_argument( - 'args', metavar='ARG', nargs='*', - help=( - "Argument passed through to 'git archive'. Set anything " - 'before , see git-archive(1) for details.')) - elif command == 'clone': - subparser.add_argument( - 'repository', - help=( - 'The (possibly remote) repository to clone from. See the ' - 'URLS section of git-clone(1) for more information on ' - 'specifying repositories.')) - elif command == 'commit': - subparser.add_argument( - 'message', metavar='MESSAGE', default='', nargs='?', - help='Text for the commit message.') - elif command == 'fetch': - subparser.add_argument( - 'remote', metavar='REMOTE', nargs='?', - help=( - 'Override the default configured in branch..remote ' - 'to fetch from a particular remote repository (e.g. ' - "'origin').")) - elif command == 'help': - subparser.add_argument( - 'command', metavar='COMMAND', nargs='?', - help='The command to show help for.') - elif command == 'log': - subparser.add_argument( - 'args', metavar='ARG', nargs='*', - help="Additional argument passed through to 'git log'.") - elif command == 'merge': - subparser.add_argument( - 'reference', metavar='REFERENCE', default='@{upstream}', - nargs='?', - help=( - 'Reference, usually other branch heads, to merge into ' - "our branch. Defaults to '@{upstream}'.")) - elif command == 'pull': - subparser.add_argument( - 'repository', metavar='REPOSITORY', default=None, nargs='?', - help=( - 'The "remote" repository that is the source of the pull. ' - 'This parameter can be either a URL (see the section GIT ' - 'URLS in git-pull(1)) or the name of a remote (see the ' - 'section REMOTES in git-pull(1)).')) - subparser.add_argument( - 'refspecs', metavar='REFSPEC', default=None, nargs='*', - help=( - 'Refspec (usually a branch name) to fetch and merge. See ' - 'the entry in the OPTIONS section of ' - 'git-pull(1) for other possibilities.')) - elif command == 'push': - subparser.add_argument( - 'repository', metavar='REPOSITORY', default=None, nargs='?', - help=( - 'The "remote" repository that is the destination of the ' - 'push. This parameter can be either a URL (see the ' - 'section GIT URLS in git-push(1)) or the name of a remote ' - '(see the section REMOTES in git-push(1)).')) - subparser.add_argument( - 'refspecs', metavar='REFSPEC', default=None, nargs='*', - help=( - 'Refspec (usually a branch name) to push. See ' - 'the entry in the OPTIONS section of ' - 'git-push(1) for other possibilities.')) - - args = parser.parse_args() - - if args.log_level: - level = getattr(_logging, args.log_level.upper()) - _LOG.setLevel(level) - - if not getattr(args, 'func', None): - parser.print_usage() - _sys.exit(1) - - if args.func == help: - arg_names = ['command'] - else: - (arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__) - kwargs = {key: getattr(args, key) for key in arg_names if key in args} - try: - args.func(**kwargs) - except SubprocessError as e: - if _LOG.level == _logging.DEBUG: - raise # don't mask the traceback - _LOG.error(str(e)) - _sys.exit(1) diff --git a/devel/nmbug/notmuch-report b/devel/nmbug/notmuch-report index eaceb2ce..9a6a31cc 100755 --- a/devel/nmbug/notmuch-report +++ b/devel/nmbug/notmuch-report @@ -1,10 +1,9 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 # # Copyright (c) 2011-2012 David Bremner # # dependencies -# - python 2.6 for json -# - argparse; either python 2.7, or install separately +# - python3 or python2.7 # # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -371,9 +370,11 @@ header_template = config['meta'].get('header', ''' border-bottom-right-radius: {border_radius}; }} tbody:nth-child(4n+1) tr td {{ + color: #000; background-color: #ffd96e; }} tbody:nth-child(4n+3) tr td {{ + color: #000; background-color: #bce; }} hr {{ diff --git a/devel/notmuch-web/nmgunicorn.py b/devel/notmuch-web/nmgunicorn.py new file mode 100644 index 00000000..e71ba12a --- /dev/null +++ b/devel/notmuch-web/nmgunicorn.py @@ -0,0 +1,11 @@ +#!/usr/bin/env python3 + +# to launch nmweb from gunicorn. + +from nmweb import urls, index, search, show +import web + +app = web.application(urls, globals()) + +# get the wsgi app from web.py application object +wsgiapp = app.wsgifunc() diff --git a/devel/notmuch-web/nmweb.py b/devel/notmuch-web/nmweb.py new file mode 100755 index 00000000..e0e87b49 --- /dev/null +++ b/devel/notmuch-web/nmweb.py @@ -0,0 +1,366 @@ +#!/usr/bin/env python + +from __future__ import absolute_import + +try: + from urllib.parse import quote_plus + from urllib.parse import unquote_plus +except ImportError: + from urllib import quote_plus + from urllib import unquote_plus + +from datetime import datetime +from mailbox import MaildirMessage +import mimetypes +import email +import re +import html +import os + +import bleach +import web +from notmuch2 import Database +from jinja2 import Environment, FileSystemLoader # FIXME to PackageLoader +from jinja2 import Markup +try: + import bjoern # from https://github.com/jonashaag/bjoern/ + use_bjoern = True +except: + use_bjoern = False + +# Configuration options +safe_tags = bleach.sanitizer.ALLOWED_TAGS + \ + [u'div', u'span', u'p', u'br', u'table', u'tr', u'td', u'th'] +linkify_plaintext = True # delays page load by about 0.02s of 0.20s budget +show_thread_nav = True # delays page load by about 0.04s of 0.20s budget + +prefix = os.environ.get('NMWEB_PREFIX', "http://localhost:8080") +webprefix = os.environ.get('NMWEB_STATIC', prefix + "/static") +cachedir = os.environ.get('NMWEB_CACHE', "static/cache") # special for webpy server; changeable if using your own +cachepath = os.environ.get('NMWEB_CACHE_PATH', cachedir) # location of static cache in the local filesystem + +if 'NMWEB_DEBUG' in os.environ: + web.config.debug = True +else: + web.config.debug = False + +# End of config options + +env = Environment(autoescape=True, + loader=FileSystemLoader('templates')) + +urls = ( + '/', 'index', + '/search/(.*)', 'search', + '/show/(.*)', 'show', +) + +def urlencode_filter(s): + if type(s) == 'Markup': + s = s.unescape() + s = s.encode('utf8') + s = quote_plus(s) + return Markup(s) +env.filters['url'] = urlencode_filter + +class index: + def GET(self): + web.header('Content-type', 'text/html') + base = env.get_template('base.html') + template = env.get_template('index.html') + db = Database() + tags = db.tags + return template.render(tags=tags, + title="Notmuch webmail", + prefix=prefix, + sprefix=webprefix) + +class search: + def GET(self, terms): + redir = False + if web.input(terms=None).terms: + redir = True + terms = web.input().terms + terms = unquote_plus (terms) + if web.input(afters=None).afters: + afters = web.input(afters=None).afters[:-3] + else: + afters = '0' + if web.input(befores=None).befores: + befores = web.input(befores=None).befores + else: + befores = '4294967296' # 2^32 + try: + if int(afters) > 0 or int(befores) < 4294967296: + redir = True + terms += ' date:@%s..@%s' % (int(afters), int(befores)) + except ValueError: + pass + if redir: + raise web.seeother('/search/%s' % quote_plus(terms.encode('utf8'))) + web.header('Content-type', 'text/html') + db = Database() + ts = db.threads(query=terms, sort=Database.SORT.NEWEST_FIRST) + template = env.get_template('search.html') + return template.generate(terms=terms, + ts=ts, + title=terms, + prefix=prefix, + sprefix=webprefix) + +def format_time_range(start, end): + if end-start < (60*60*24): + time = datetime.fromtimestamp(start).strftime('%Y %b %d %H:%M') + else: + start = datetime.fromtimestamp(start).strftime("%Y %b %d") + end = datetime.fromtimestamp(end).strftime("%Y %b %d") + time = "%s through %s" % (start, end) + return time +env.globals['format_time_range'] = format_time_range + +def mailto_addrs(msg,header_name): + try: + hdr = msg.header(header_name) + except LookupError: + return '' + + frm = email.utils.getaddresses([hdr]) + return ', '.join(['%s' % ((l, p) if p else (l, l)) for (p, l) in frm]) +env.globals['mailto_addrs'] = mailto_addrs + +def link_msg(msg): + lnk = quote_plus(msg.messageid.encode('utf8')) + try: + subj = html.escape(msg.header('Subject')) + except LookupError: + subj = "" + out = '%s' % (prefix, lnk, subj) + return out +env.globals['link_msg'] = link_msg + +def show_msgs(msgs): + r = '
    ' + for msg in msgs: + red = 'color:black; font-style:normal' + if msg.matched: + red = 'color:red; font-style:italic' + frm = mailto_addrs(msg,'From') + lnk = link_msg(msg) + tags = ", ".join(msg.tags) + rs = show_msgs(msg.replies()) + r += '
  • %s—%s [%s] %s
    • ' % (red, frm, lnk, tags, rs) + r += '
' + return r +env.globals['show_msgs'] = show_msgs + +# As email.message.walk, but showing close tags as well +def mywalk(self): + yield self + if self.is_multipart(): + for subpart in self.get_payload(): + for subsubpart in mywalk(subpart): + yield subsubpart + yield 'close-div' + +class show: + def GET(self, mid): + web.header('Content-type', 'text/html') + db = Database() + try: + m = db.find(mid) + except: + raise web.notfound("No such message id.") + template = env.get_template('show.html') + # FIXME add reply-all link with email.urils.getaddresses + # FIXME add forward link using mailto with body parameter? + return template.render(m=m, + mid=mid, + title=m.header('Subject'), + prefix=prefix, + sprefix=webprefix) + +def thread_nav(m): + if not show_thread_nav: return + db = Database() + thread = next(db.threads('thread:'+m.threadid)) + prv = None + found = False + nxt = None + for msg in thread: + if m == msg: + found = True + elif not found: + prv = msg + else: # found message, but not on this loop + nxt = msg + break + yield "
    " + if prv: yield "
  • Previous message (by thread): %s
    • " % link_msg(prv) + if nxt: yield "
  • Next message (by thread): %s
    • " % link_msg(nxt) + yield "

Thread:

" + # FIXME show now takes three queries instead of 1; + # can we yield the message body while computing the thread shape? + thread = next(db.threads('thread:'+m.threadid)) + yield show_msgs(thread.toplevel()) + return +env.globals['thread_nav'] = thread_nav + +def format_message(nm_msg, mid): + fn = list(nm_msg.filenames())[0] + msg = MaildirMessage(open(fn, 'rb')) + return format_message_walk(msg, mid) + +def decodeAnyway(txt, charset='ascii'): + try: + out = txt.decode(charset) + except: + try: + out = txt.decode('utf-8') + except UnicodeDecodeError: + out = txt.decode('latin1') + return out + +def require_protocol_prefix(attrs, new=False): + if not new: + return attrs + link_text = attrs[u'_text'] + if link_text.startswith(('http:', 'https:', 'mailto:', 'git:', 'id:')): + return attrs + return None + +# Bleach doesn't even try to linkify id:... text, so no point invoking this yet +def modify_id_links(attrs, new=False): + if attrs[(None, u'href')].startswith(u'id:'): + attrs[(None, u'href')] = prefix + "/show/" + attrs[(None, u'href')][3:] + return attrs + +def css_part_id(content_type, parts=[]): + c = content_type.replace('/', '-') + out = "-".join(parts + [c]) + return out + +def format_message_walk(msg, mid): + counter = 0 + cid_refd = [] + parts = ['main'] + for part in mywalk(msg): + if part == 'close-div': + parts.pop() + yield '' + elif part.get_content_maintype() == 'multipart': + yield '
' % \ + (part.get_content_subtype(), css_part_id(part.get_content_type(), parts)) + parts.append(part.get_content_subtype()) + if part.get_content_subtype() == 'alternative': + yield '
    ' + for subpart in part.get_payload(): + yield ('
  • %s
    • ' % + (css_part_id(subpart.get_content_type(), parts), + subpart.get_content_type())) + yield '
' + elif part.get_content_type() == 'message/rfc822': + # FIXME extract subject, date, to/cc/from into a separate template and use it here + yield '
' + elif part.get_content_maintype() == 'text': + if part.get_content_subtype() == 'plain': + yield '
' % css_part_id(part.get_content_type(), parts) + yield '
'
+        out = part.get_payload(decode=True)
+        out = decodeAnyway(out, part.get_content_charset('ascii'))
+        out = html.escape(out)
+        out = out.encode('ascii', 'xmlcharrefreplace').decode('ascii')
+        if linkify_plaintext: out = bleach.linkify(out, callbacks=[require_protocol_prefix])
+        yield out
+        yield '
' + elif part.get_content_subtype() == 'html': + yield '
' % css_part_id(part.get_content_type(), parts) + unb64 = part.get_payload(decode=True) + decoded = decodeAnyway(unb64, part.get_content_charset('ascii')) + cid_refd += find_cids(decoded) + part.set_payload(bleach.clean(replace_cids(decoded, mid), tags=safe_tags). + encode(part.get_content_charset('ascii'), 'xmlcharrefreplace')) + (filename, cid) = link_to_cached_file(part, mid, counter) + counter += 1 + yield '' % \ + os.path.join(prefix, cachedir, mid, filename) + yield '
' + else: + yield '
' % css_part_id(part.get_content_type(), parts) + (filename, cid) = link_to_cached_file(part, mid, counter) + counter += 1 + yield '%s (%s)' % (os.path.join(prefix, + cachedir, + mid, + filename), + filename, + part.get_content_type()) + yield '
' + elif part.get_content_maintype() == 'image': + (filename, cid) = link_to_cached_file(part, mid, counter) + if cid not in cid_refd: + counter += 1 + yield '%s' % (os.path.join(prefix, + cachedir, + mid, + filename), + filename) + else: + (filename, cid) = link_to_cached_file(part, mid, counter) + counter += 1 + yield '%s (%s)' % (os.path.join(prefix, + cachedir, + mid, + filename), + filename, + part.get_content_type()) +env.globals['format_message'] = format_message + +def replace_cids(body, mid): + return body.replace('cid:', os.path.join(prefix, cachedir, mid)+'/') + +def find_cids(body): + return re.findall(r'cid:([^ "\'>]*)', body) + +def link_to_cached_file(part, mid, counter): + filename = part.get_filename() + if not filename: + ext = mimetypes.guess_extension(part.get_content_type()) + if not ext: + ext = '.bin' + filename = 'part-%03d%s' % (counter, ext) + try: + os.makedirs(os.path.join(cachepath, mid)) + except OSError: + pass + fn = os.path.join(cachepath, mid, filename) # FIXME escape mid, filename + fp = open(fn, 'wb') + if part.get_content_maintype() == 'text': + data = part.get_payload(decode=True) + data = decodeAnyway(data, part.get_content_charset('ascii')).encode('utf-8') + else: + try: + data = part.get_payload(decode=True) + except: + data = part.get_payload(decode=False) + if data: + fp.write(data) + fp.close() + if 'Content-ID' in part: + cid = part['Content-ID'] + if cid[0] == '<' and cid[-1] == '>': cid = cid[1:-1] + cid_fn = os.path.join(cachepath, mid, cid) # FIXME escape mid, cid + try: + os.unlink(cid_fn) + except OSError: + pass + os.link(fn, cid_fn) + return (filename, cid) + else: + return (filename, None) + +if __name__ == '__main__': + app = web.application(urls, globals()) + if use_bjoern: + bjoern.run(app.wsgifunc(), "127.0.0.1", 8080) + else: + app.run() diff --git a/devel/notmuch-web/static/css/jquery-ui.css b/devel/notmuch-web/static/css/jquery-ui.css new file mode 120000 index 00000000..eba7c769 --- /dev/null +++ b/devel/notmuch-web/static/css/jquery-ui.css @@ -0,0 +1 @@ +/usr/share/javascript/jquery-ui/themes/base/jquery-ui.min.css \ No newline at end of file diff --git a/devel/notmuch-web/static/css/notmuch-0.1.css b/devel/notmuch-web/static/css/notmuch-0.1.css new file mode 100644 index 00000000..0f085644 --- /dev/null +++ b/devel/notmuch-web/static/css/notmuch-0.1.css @@ -0,0 +1,15 @@ +pre { + white-space: pre-wrap; +} + +.message-rfc822 { + border: 1px solid; + border-radius: 25px; +} + +.embedded-html { + frameborder: 0; + border: 0; + scrolling: no; + width: 100%; +} diff --git a/devel/notmuch-web/static/js/jquery-ui.js b/devel/notmuch-web/static/js/jquery-ui.js new file mode 120000 index 00000000..5c053bab --- /dev/null +++ b/devel/notmuch-web/static/js/jquery-ui.js @@ -0,0 +1 @@ +/usr/share/javascript/jquery-ui/jquery-ui.min.js \ No newline at end of file diff --git a/devel/notmuch-web/static/js/jquery.js b/devel/notmuch-web/static/js/jquery.js new file mode 120000 index 00000000..7fff8870 --- /dev/null +++ b/devel/notmuch-web/static/js/jquery.js @@ -0,0 +1 @@ +/usr/share/javascript/jquery/jquery.min.js \ No newline at end of file diff --git a/devel/notmuch-web/static/js/notmuch-0.1.js b/devel/notmuch-web/static/js/notmuch-0.1.js new file mode 100644 index 00000000..ed6e9f4a --- /dev/null +++ b/devel/notmuch-web/static/js/notmuch-0.1.js @@ -0,0 +1,35 @@ +$(function(){ + $("#after").datepicker({ + altField: "#afters", + altFormat: "@", + changeMonth: true, + changeYear: true, + defaultDate: "-7d", + minDate: "01/01/1970", + yearRange: "2000:+0", + onSelect: function(selectedDate) { + $("#before").datepicker("option","minDate",selectedDate); + } + }); + $("#before").datepicker({ + altField: "#befores", + altFormat: "@", + changeMonth: true, + changeYear: true, + defaultDate: "+1d", + maxDate: "+1d", + yearRange: "2000:+0", + onSelect: function(selectedDate) { + $("#after").datepicker("option","maxDate",selectedDate); + } + }); + $(function(){ + $('.multipart-alternative').tabs() + }); + $(function(){ + $('.embedded-html').on('load',function(){ + this.style.height = this.contentWindow.document.body.offsetHeight + 'px'; + }); + }); +}); + diff --git a/devel/notmuch-web/templates/base.html b/devel/notmuch-web/templates/base.html new file mode 100644 index 00000000..90d92931 --- /dev/null +++ b/devel/notmuch-web/templates/base.html @@ -0,0 +1,39 @@ + + + + + + + + + + +{{title}} + +
+
+{% block searchform %} +
+ + + + +
+{% endblock searchform %} +

{{title}}

+
+
+{% block content %} +

Common tags

+ +
+{% endblock content %} +
+ diff --git a/devel/notmuch-web/templates/index.html b/devel/notmuch-web/templates/index.html new file mode 100644 index 00000000..0eb3fd3c --- /dev/null +++ b/devel/notmuch-web/templates/index.html @@ -0,0 +1,9 @@ +{% extends "base.html" %} +{% block content %} +

Common tags

+ +{% endblock content %} diff --git a/devel/notmuch-web/templates/search.html b/devel/notmuch-web/templates/search.html new file mode 100644 index 00000000..6719c356 --- /dev/null +++ b/devel/notmuch-web/templates/search.html @@ -0,0 +1,10 @@ +{% extends "base.html" %} +

{{ terms|e }}

+{% block content %} +{% for t in ts %} +

{{ t.subject|e }}

+

{{ t.authors|e }}

+

{{ format_time_range(t.first,t.last)|e }}

+ {{ show_msgs(t.toplevel())|safe }} +{% endfor %} +{% endblock content %} diff --git a/devel/notmuch-web/templates/show.html b/devel/notmuch-web/templates/show.html new file mode 100644 index 00000000..690f5464 --- /dev/null +++ b/devel/notmuch-web/templates/show.html @@ -0,0 +1,15 @@ +{% extends "base.html" %} +{% block content %} +{% set headers = ['Subject', 'Date'] %} +{% set addr_headers = ['To', 'Cc', 'From'] %} +{% for header in headers: %} +

{{header}}: {{m.header(header)|e}}

+{% endfor %} +{% for header in addr_headers: %} +

{{header}}: {{mailto_addrs(m,header)|safe}}

+{% endfor %} +
+{% for part in format_message(m,mid): %}{{ part|safe }}{% endfor %} +{% for b in thread_nav(m): %}{{b|safe}}{% endfor %} +
+{% endblock content %} diff --git a/devel/notmuch-web/todo b/devel/notmuch-web/todo new file mode 100644 index 00000000..3c885bd9 --- /dev/null +++ b/devel/notmuch-web/todo @@ -0,0 +1,14 @@ +review escaping and safety handling mail from Bad People + +revise template loader---can we make this faster? + +add reply-all link with email.urils.getaddresses + +change current reply links to quote body + +add forward link using mailto with body parameter? + +unescape the current search term, including translating back dates + + +later: json support, iOS app? diff --git a/devel/printmimestructure b/devel/printmimestructure deleted file mode 100755 index 70e0a5c0..00000000 --- a/devel/printmimestructure +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/env python -# -*- coding: utf-8 -*- - -# Author: Daniel Kahn Gillmor -# License: GPLv3+ - -# This script reads a MIME message from stdin and produces a treelike -# representation on it stdout. - -# Example: -# -# 0 dkg@alice:~$ printmimestructure < 'Maildir/cur/1269025522.M338697P12023.monkey,S=6459,W=6963:2,Sa' -# └┬╴multipart/signed 6546 bytes -# ├─╴text/plain inline 895 bytes -# └─╴application/pgp-signature inline [signature.asc] 836 bytes -# 0 dkg@alice:~$ - - -# If you want to number the parts, i suggest piping the output through -# something like "cat -n" - -from __future__ import print_function - -import email -import sys - -def print_part(z, prefix): - fname = '' if z.get_filename() is None else ' [' + z.get_filename() + ']' - cset = '' if z.get_charset() is None else ' (' + z.get_charset() + ')' - disp = z.get_params(None, header='Content-Disposition') - if (disp is None): - disposition = '' - else: - disposition = '' - for d in disp: - if d[0] in [ 'attachment', 'inline' ]: - disposition = ' ' + d[0] - if z.is_multipart(): - nbytes = len(z.as_string()) - else: - nbytes = len(z.get_payload()) - - print('{}{}{}{}{} {:d} bytes'.format( - prefix, - z.get_content_type(), - cset, - disposition, - fname, - nbytes, - )) - -def test(z, prefix=''): - if (z.is_multipart()): - print_part(z, prefix+'┬╴') - if prefix.endswith('└'): - prefix = prefix.rpartition('└')[0] + ' ' - if prefix.endswith('├'): - prefix = prefix.rpartition('├')[0] + '│' - parts = z.get_payload() - i = 0 - while (i < parts.__len__()-1): - test(parts[i], prefix + '├') - i += 1 - test(parts[i], prefix + '└') - # FIXME: show epilogue? - else: - print_part(z, prefix+'─╴') - -test(email.message_from_file(sys.stdin), '└') diff --git a/devel/release-checks.sh b/devel/release-checks.sh index 7ba94822..c0accf78 100755 --- a/devel/release-checks.sh +++ b/devel/release-checks.sh @@ -29,7 +29,7 @@ append_emsg () emsgs="${emsgs:+$emsgs\n} $1" } -for f in ./version debian/changelog NEWS "$PV_FILE" +for f in ./version.txt debian/changelog NEWS "$PV_FILE" do if [ ! -f "$f" ]; then append_emsg "File '$f' is missing" elif [ ! -r "$f" ]; then append_emsg "File '$f' is unreadable" @@ -53,13 +53,13 @@ then else echo "Reading './version' file failed (surprisingly!)" exit 1 -fi < ./version +fi < ./version.txt readonly VERSION # In the rest of this file, tests collect list of errors to be fixed -echo -n "Checking that git working directory is clean... " +printf %s "Checking that git working directory is clean... " git_status=`git status --porcelain` if [ "$git_status" = '' ] then @@ -77,7 +77,7 @@ verfail () append_emsg " Please follow the instructions in RELEASING to choose a version" } -echo -n "Checking that '$VERSION' is good with digits and periods... " +printf %s "Checking that '$VERSION' is good with digits and periods... " case $VERSION in *[!0-9.]*) verfail "'$VERSION' contains other characters than digits and periods" ;; @@ -88,7 +88,7 @@ case $VERSION in *) verfail "'$VERSION' is a single number" ;; esac -echo -n "Checking that this is Debian package for notmuch... " +printf %s "Checking that this is Debian package for notmuch... " read deb_notmuch deb_version rest < debian/changelog if [ "$deb_notmuch" = 'notmuch' ] then @@ -98,7 +98,7 @@ else append_emsg "Package name '$deb_notmuch' is not 'notmuch' in debian/changelog" fi -echo -n "Checking that Debian package version is $VERSION-1... " +printf %s "Checking that Debian package version is $VERSION-1... " if [ "$deb_version" = "($VERSION-1)" ] then @@ -108,8 +108,8 @@ else append_emsg "Version '$deb_version' is not '($VERSION-1)' in debian/changelog" fi -echo -n "Checking that python bindings version is $VERSION... " -py_version=`python -c "with open('$PV_FILE') as vf: exec(vf.read()); print(__VERSION__)"` +printf %s "Checking that python bindings version is $VERSION... " +py_version=`python3 -c "with open('$PV_FILE') as vf: exec(vf.read()); print(__VERSION__)"` if [ "$py_version" = "$VERSION" ] then echo Yes. @@ -118,7 +118,7 @@ else append_emsg "Version '$py_version' is not '$VERSION' in $PV_FILE" fi -echo -n "Checking that NEWS header is tidy... " +printf %s "Checking that NEWS header is tidy... " if [ "`exec sed 's/./=/g; 1q' NEWS`" = "`exec sed '1d; 2q' NEWS`" ] then echo Yes. @@ -132,7 +132,7 @@ else fi fi -echo -n "Checking that this is Notmuch NEWS... " +printf %s "Checking that this is Notmuch NEWS... " read news_notmuch news_version news_date < NEWS if [ "$news_notmuch" = "Notmuch" ] then @@ -142,7 +142,7 @@ else append_emsg "First word '$news_notmuch' is not 'Notmuch' in NEWS file" fi -echo -n "Checking that NEWS version is $VERSION... " +printf %s "Checking that NEWS version is $VERSION... " if [ "$news_version" = "$VERSION" ] then echo Yes. @@ -154,7 +154,7 @@ fi #eval `date '+year=%Y mon=%m day=%d'` today0utc=`date --date=0Z +%s` # gnu date feature -echo -n "Checking that NEWS date is right... " +printf %s "Checking that NEWS date is right... " case $news_date in '('[2-9][0-9][0-9][0-9]-[01][0-9]-[0123][0-9]')') newsdate0utc=`nd=${news_date#\\(}; date --date="${nd%)} 0Z" +%s` @@ -176,12 +176,9 @@ case $news_date in esac year=`exec date +%Y` -echo -n "Checking that copyright in documentation contains 2009-$year... " +printf %s "Checking that copyright in documentation contains 2009-$year... " # Read the value of variable `copyright' defined in 'doc/conf.py'. -# As __file__ is not defined when python command is given from command line, -# it is defined before contents of 'doc/conf.py' (which dereferences __file__) -# is executed. -copyrightline=`exec python -c "with open('doc/conf.py') as cf: __file__ = ''; exec(cf.read()); print(copyright)"` +copyrightline=$(grep ^copyright doc/conf.py) case $copyrightline in *2009-$year*) echo Yes. ;; diff --git a/devel/schemata b/devel/schemata index 28332c6b..4e05cdac 100644 --- a/devel/schemata +++ b/devel/schemata @@ -14,7 +14,7 @@ are interleaved. Keys are printed as keywords (symbols preceded by a colon), e.g. (:id "123" :time 54321 :from "foobar"). Null is printed as nil, true as t and false as nil. -This is version 4 of the structured output format. +This is version 5 of the structured output format. Version history --------------- @@ -36,6 +36,10 @@ v4 - (notmuch 0.29) added message.crypto to identify overall message cryptographic state +v5 +- sorting support for notmuch show (no change to actual schema, + just new command line argument) + Common non-terminals -------------------- @@ -72,6 +76,7 @@ message = { # (format_message_sprinter) id: messageid, match: bool, + excluded: bool, filename: [string*], timestamp: unix_time, # date header as unix time date_relative: string, # user-friendly timestamp @@ -79,6 +84,7 @@ message = { headers: headers, crypto: crypto, + duplicate: integer, body?: [part] # omitted if --body=false } @@ -141,9 +147,11 @@ headers = { Cc?: string, Bcc?: string, Reply-To?: string, - Date: string + Date: string, + extra_header_pair* } +extra_header_pair= (header_name: string) # Encryption status (format_part_sprinter) encstatus = [{status: "good"|"bad"}] @@ -158,6 +166,7 @@ signature = { created?: unix_time, expires?: unix_time, userid?: string + email?: string # if status is not "good": keyid?: string errors?: sig_errors diff --git a/devel/try-emacs-mua b/devel/try-emacs-mua index 041f6216..585d6242 100755 --- a/devel/try-emacs-mua +++ b/devel/try-emacs-mua @@ -44,28 +44,10 @@ while looking at: " pdir "emacs\n\nexit emacs (y or n)? ") try-notmuch-emacs-directory (concat pdir "emacs/") load-path (cons try-notmuch-emacs-directory load-path))) -;; they say advice doesn't work for primitives (functions from c source) -;; well, these 'before' advice works for emacs 23.1 - 24.5 (at least) -;; ...and for our purposes 24.3 is enough (there is no load-prefer-newer there) -;; note also that the old, "obsolete" defadvice mechanism was used, but that -;; is the only one available for emacs 23 and 24 up to 24.3. - -(if (boundp 'load-prefer-newer) - (defadvice require (before before-require activate) - (unless (featurep feature) - (message "require: %s" feature))) - ;; else: special require "short-circuit"; after load feature is provided... - ;; ... in notmuch sources we always use require and there are no loops - (defadvice require (before before-require activate) - (unless (featurep feature) - (message "require: %s" feature) - (let ((name (symbol-name feature))) - (if (and (string-match "^notmuch" name) - (file-newer-than-file-p - (concat try-notmuch-emacs-directory name ".el") - (concat try-notmuch-emacs-directory name ".elc"))) - (load (concat try-notmuch-emacs-directory name ".el") nil nil t t) - ))))) +(define-advice require + (:before (feature &optional _filename _noerror) notmuch) + (unless (featurep feature) + (message "require: %s" feature))) (insert "Found notmuch emacs client in " try-notmuch-emacs-directory "\n") diff --git a/devel/uncrustify.cfg b/devel/uncrustify.cfg index c36c33d6..d203d4e1 100644 --- a/devel/uncrustify.cfg +++ b/devel/uncrustify.cfg @@ -119,3 +119,9 @@ cmt_star_cont = true # indent_brace = 0 indent_class = true + +# line width / line splitting +code_width 102 +ls_for_split_full True +ls_func_split_full True +ls_code_width True diff --git a/doc/Makefile.local b/doc/Makefile.local index b4e0c955..aafa77a0 100644 --- a/doc/Makefile.local +++ b/doc/Makefile.local @@ -1,10 +1,10 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := doc # You can set these variables from the command line. SPHINXOPTS := -q -SPHINXBUILD = WITH_EMACS=${WITH_EMACS} sphinx-build +SPHINXBUILD = env LD_LIBRARY_PATH=${NOTMUCH_BUILDDIR}/lib sphinx-build DOCBUILDDIR := $(dir)/_build # Internal variables. @@ -18,7 +18,9 @@ MAN7_RST := $(wildcard $(srcdir)/doc/man7/*.rst) MAN_RST_FILES := $(MAN1_RST) $(MAN5_RST) $(MAN7_RST) ALL_RST_FILES := $(MAN_RST_FILES) $(srcdir)/doc/notmuch-emacs.rst +COPY_ROFF1 := $(patsubst %,$(DOCBUILDDIR)/man/man1/%.1,nmbug notmuch-setup) MAN1_ROFF := $(patsubst $(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN1_RST:.rst=.1)) +MAN1_ROFF := $(MAN1_ROFF) $(COPY_ROFF1) MAN5_ROFF := $(patsubst $(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN5_RST:.rst=.5)) MAN7_ROFF := $(patsubst $(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN7_RST:.rst=.7)) MAN_ROFF_FILES := $(MAN1_ROFF) $(MAN5_ROFF) $(MAN7_ROFF) @@ -33,33 +35,43 @@ ifeq ($(WITH_EMACS),1) INFO_TEXI_FILES += $(DOCBUILDDIR)/texinfo/notmuch-emacs.texi endif -INFO_INFO_FILES := $(INFO_TEXI_FILES:.texi=.info) +COPY_INFO1 := $(patsubst $(DOCBUILDDIR)/man/man1/%.1,$(DOCBUILDDIR)/texinfo/%.info,$(COPY_ROFF1)) +INFO_INFO_FILES := $(INFO_TEXI_FILES:.texi=.info) $(COPY_INFO1) .PHONY: sphinx-html sphinx-texinfo sphinx-info .PHONY: install-man build-man apidocs install-apidocs %.gz: % - rm -f $@ && gzip --stdout $^ > $@ + rm -f $@ && gzip --no-name --stdout $^ > $@ ifeq ($(WITH_EMACS),1) -$(DOCBUILDDIR)/.roff.stamp sphinx-html sphinx-texinfo: docstring.stamp +$(DOCBUILDDIR)/.roff.stamp $(DOCBUILDDIR)/.html.stamp $(DOCBUILDDIR)/.texi.stamp : docstring.stamp +endif + +ifeq ($(HAVE_PYTHON3_CFFI),1) +DOC_PREREQS=bindings/python-cffi.stamp +else +DOC_PREREQS= endif sphinx-html: $(DOCBUILDDIR)/.html.stamp -$(DOCBUILDDIR)/.html.stamp: $(ALL_RST_FILES) +$(DOCBUILDDIR)/.html.stamp: $(ALL_RST_FILES) $(DOC_PREREQS) $(SPHINXBUILD) -b html -d $(DOCBUILDDIR)/html_doctrees $(ALLSPHINXOPTS) $(DOCBUILDDIR)/html touch $@ sphinx-texinfo: $(DOCBUILDDIR)/.texi.stamp -$(DOCBUILDDIR)/.texi.stamp: $(ALL_RST_FILES) +$(DOCBUILDDIR)/.texi.stamp: $(ALL_RST_FILES) $(DOC_PREREQS) $(SPHINXBUILD) -b texinfo -d $(DOCBUILDDIR)/texinfo_doctrees $(ALLSPHINXOPTS) $(DOCBUILDDIR)/texinfo touch $@ -sphinx-info: sphinx-texinfo +sphinx-info: $(DOCBUILDDIR)/.info.stamp + +$(DOCBUILDDIR)/.info.stamp: $(DOCBUILDDIR)/.texi.stamp $(DOC_PREREQS) $(MAKE) -C $(DOCBUILDDIR)/texinfo info + touch $@ # Use the man page converter that is available. We should never depend # on MAN_ROFF_FILES if a converter is not available. @@ -108,6 +120,11 @@ build-man: install-man: @echo "No sphinx, will not install man pages." else + +# it should be safe to depend on the stamp file, because it is created +# after all roff files are moved into place. +${MAN_GZIP_FILES}: ${DOCBUILDDIR}/.roff.stamp + build-man: ${MAN_GZIP_FILES} install-man: ${MAN_GZIP_FILES} mkdir -m0755 -p "$(DESTDIR)$(mandir)/man1" @@ -116,14 +133,13 @@ install-man: ${MAN_GZIP_FILES} install -m0644 $(filter %.1.gz,$(MAN_GZIP_FILES)) $(DESTDIR)/$(mandir)/man1 install -m0644 $(filter %.5.gz,$(MAN_GZIP_FILES)) $(DESTDIR)/$(mandir)/man5 install -m0644 $(filter %.7.gz,$(MAN_GZIP_FILES)) $(DESTDIR)/$(mandir)/man7 - cd $(DESTDIR)/$(mandir)/man1 && ln -sf notmuch.1.gz notmuch-setup.1.gz endif ifneq ($(HAVE_SPHINX)$(HAVE_MAKEINFO),11) build-info: @echo "Missing sphinx or makeinfo, not building info pages" else -build-info: sphinx-info +build-info: $(DOCBUILDDIR)/.info.stamp endif ifneq ($(HAVE_SPHINX)$(HAVE_MAKEINFO)$(HAVE_INSTALL_INFO),111) @@ -141,5 +157,7 @@ $(dir)/config.dox: version.stamp echo "INPUT=${srcdir}/lib/notmuch.h" >> $@ CLEAN := $(CLEAN) $(DOCBUILDDIR) $(DOCBUILDDIR)/.roff.stamp $(DOCBUILDDIR)/.texi.stamp -CLEAN := $(CLEAN) $(DOCBUILDDIR)/.html.stamp +CLEAN := $(CLEAN) $(DOCBUILDDIR)/.html.stamp $(DOCBUILDDIR)/.info.stamp CLEAN := $(CLEAN) $(MAN_GZIP_FILES) $(MAN_ROFF_FILES) $(dir)/conf.pyc $(dir)/config.dox + +CLEAN := $(CLEAN) $(dir)/__pycache__ diff --git a/doc/command-line.rst b/doc/command-line.rst new file mode 100644 index 00000000..543a5f9e --- /dev/null +++ b/doc/command-line.rst @@ -0,0 +1,36 @@ +Notmuch Command Line Interface +============================== + +Main commands +------------- + +.. toctree:: + :titlesonly: + + man1/notmuch + man1/notmuch-address + man1/notmuch-compact + man1/notmuch-config + man1/notmuch-count + man1/notmuch-dump + man1/notmuch-emacs-mua + man1/notmuch-git + man1/notmuch-insert + man1/notmuch-new + man1/notmuch-reindex + man1/notmuch-reply + man1/notmuch-restore + man1/notmuch-search + man1/notmuch-show + man1/notmuch-tag + man5/notmuch-hooks + +Aliases +------- + +.. toctree:: + :titlesonly: + + nmbug + notmuch-setup + diff --git a/doc/conf.py b/doc/conf.py index fc9738ff..ee1b336a 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -3,6 +3,10 @@ import sys import os +from pathlib import Path +sys.path.append(str(Path(__file__).parent)) + +extensions = [ 'sphinx.ext.autodoc', 'elisp' ] # The suffix of source filenames. source_suffix = '.rst' @@ -12,16 +16,26 @@ master_doc = 'index' # General information about the project. project = u'notmuch' -copyright = u'2009-2019, Carl Worth and many others' +copyright = u'2009-2024, Carl Worth and many others' location = os.path.dirname(__file__) for pathdir in ['.', '..']: - version_file = os.path.join(location,pathdir,'version') + version_file = os.path.join(location,pathdir,'version.txt') if os.path.exists(version_file): with open(version_file,'r') as infile: version=infile.read().replace('\n','') +# for autodoc +sys.path.insert(0, os.path.join(location, '..', 'bindings', 'python-cffi', 'build', 'stage')) + +# read generated config +for pathdir in ['.', '..']: + conf_file = os.path.join(location,pathdir,'sphinx.config') + if os.path.exists(conf_file): + with open(conf_file,'r') as infile: + exec(''.join(infile.readlines())) + # The full version, including alpha/beta/rc tags. release = version @@ -29,12 +43,23 @@ release = version # directories to ignore when looking for source files. exclude_patterns = ['_build'] -# If we don't have emacs (or the user configured --without-emacs), -# don't build the notmuch-emacs docs, as they need emacs to generate -# the docstring include files -if os.environ.get('WITH_EMACS') != '1': +if tags.has('WITH_EMACS'): + # Hacky reimplementation of include to workaround limitations of + # sphinx-doc + lines = ['.. include:: /../emacs/rstdoc.rsti\n\n'] # in the source tree + for file in ('notmuch.rsti', 'notmuch-lib.rsti', 'notmuch-hello.rsti', 'notmuch-show.rsti', 'notmuch-tag.rsti', 'notmuch-tree.rsti'): + lines.extend(open(rsti_dir+'/'+file)) + rst_epilog = ''.join(lines) + del lines +else: + # If we don't have emacs (or the user configured --without-emacs), + # don't build the notmuch-emacs docs, as they need emacs to generate + # the docstring include files exclude_patterns.append('notmuch-emacs.rst') +if not tags.has('WITH_PYTHON'): + exclude_patterns.append('python-bindings.rst') + # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' @@ -44,6 +69,8 @@ pygments_style = 'sphinx' # a list of builtin themes. html_theme = 'default' +# prevent generation of python module index +html_domain_indices=[] # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, @@ -57,6 +84,11 @@ htmlhelp_basename = 'notmuchdoc' # Despite the name, this actually affects manual pages as well. html_use_smartypants = False +# See: +# - https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-manpages_url +# - https://manpages.debian.org/ +manpages_url = 'https://manpages.debian.org/{page}.{section}.html' + # -- Options for manual page output --------------------------------------- # One entry per manual page. List of tuples @@ -64,6 +96,8 @@ html_use_smartypants = False notmuch_authors = u'Carl Worth and many others' +man_make_section_directory = False + man_pages = [ ('man1/notmuch', 'notmuch', u'thread-based email index, search, and tagging', @@ -93,6 +127,14 @@ man_pages = [ u'send mail with notmuch and emacs', [notmuch_authors], 1), + ('man1/notmuch-git', 'notmuch-git', + u'manage notmuch tags with git', + [notmuch_authors], 1), + + ('man1/notmuch-git', 'nmbug', + u'manage notmuch bugs with git', + [notmuch_authors], 1), + ('man5/notmuch-hooks', 'notmuch-hooks', u'hooks for notmuch', [notmuch_authors], 5), @@ -129,6 +171,14 @@ man_pages = [ u'syntax for notmuch queries', [notmuch_authors], 7), + ('man1/notmuch', 'notmuch-setup', + u'getting started with notmuch', + [notmuch_authors], 1), + + ('man7/notmuch-sexp-queries', 'notmuch-sexp-queries', + u's-expression syntax for notmuch queries', + [notmuch_authors], 7), + ('man1/notmuch-show', 'notmuch-show', u'show messages matching the given search terms', [notmuch_authors], 1), @@ -166,3 +216,11 @@ texinfo_documents += [ x[2], # description 'Miscellaneous' # category ) for x in man_pages] + +def setup(app): + import docutils.nodes + # define nmconfig role and directive for config items. + app.add_object_type('nmconfig','nmconfig', + indextemplate='pair: configuration item; %s', + ref_nodeclass=docutils.nodes.generated, + objname='config item' ) diff --git a/doc/doxygen.cfg b/doc/doxygen.cfg index 2ca15d41..4a022de1 100644 --- a/doc/doxygen.cfg +++ b/doc/doxygen.cfg @@ -27,7 +27,6 @@ INHERIT_DOCS = YES SEPARATE_MEMBER_PAGES = NO TAB_SIZE = 8 ALIASES = -TCL_SUBST = OPTIMIZE_OUTPUT_FOR_C = YES OPTIMIZE_OUTPUT_JAVA = NO OPTIMIZE_FOR_FORTRAN = NO @@ -264,12 +263,10 @@ GENERATE_TAGFILE = ALLEXTERNALS = NO EXTERNAL_GROUPS = NO EXTERNAL_PAGES = NO -PERL_PATH = /usr/bin/perl #--------------------------------------------------------------------------- # Configuration options related to the dot tool #--------------------------------------------------------------------------- CLASS_DIAGRAMS = NO -MSCGEN_PATH = HIDE_UNDOC_RELATIONS = YES HAVE_DOT = NO DOT_NUM_THREADS = 0 diff --git a/doc/elisp.py b/doc/elisp.py new file mode 100644 index 00000000..1b0392e6 --- /dev/null +++ b/doc/elisp.py @@ -0,0 +1,445 @@ +# Copyright (C) 2016 Sebastian Wiesner and Flycheck contributors + +# This file is not part of GNU Emacs. + +# This program is free software: you can redistribute it and/or modify it under +# the terms of the GNU General Public License as published by the Free Software +# Foundation, either version 3 of the License, or (at your option) any later +# version. + +# This program is distributed in the hope that it will be useful, but WITHOUT +# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS +# FOR A PARTICULAR PURPOSE. See the GNU General Public License for more +# details. + +# You should have received a copy of the GNU General Public License along with +# this program. If not, see . + + +from collections import namedtuple +from sphinx import addnodes +from sphinx.util import ws_re +from sphinx.roles import XRefRole +from sphinx.domains import Domain, ObjType +from sphinx.util.nodes import make_refnode +from sphinx.directives import ObjectDescription + + +def make_target(cell, name): + """Create a target name from ``cell`` and ``name``. + + ``cell`` is the name of a symbol cell, and ``name`` is a symbol name, both + as strings. + + The target names are used as cross-reference targets for Sphinx. + + """ + return '{cell}-{name}'.format(cell=cell, name=name) + + +def to_mode_name(symbol_name): + """Convert ``symbol_name`` to a mode name. + + Split at ``-`` and titlecase each part. + + """ + return ' '.join(p.title() for p in symbol_name.split('-')) + + +class Cell(namedtuple('Cell', 'objtype docname')): + """A cell in a symbol. + + A cell holds the object type and the document name of the description for + the cell. + + Cell objects are used within symbol entries in the domain data. + + """ + + pass + + +class KeySequence(namedtuple('KeySequence', 'keys')): + """A key sequence.""" + + PREFIX_KEYS = {'C-u'} + PREFIX_KEYS.update('M-{}'.format(n) for n in range(10)) + + @classmethod + def fromstring(cls, s): + return cls(s.split()) + + @property + def command_name(self): + """The command name in this key sequence. + + Return ``None`` for key sequences that are no command invocations with + ``M-x``. + + """ + try: + return self.keys[self.keys.index('M-x') + 1] + except ValueError: + return None + + @property + def has_prefix(self): + """Whether this key sequence has a prefix.""" + return self.keys[0] in self.PREFIX_KEYS + + def __str__(self): + return ' '.join(self.keys) + + +class EmacsLispSymbol(ObjectDescription): + """An abstract base class for directives documenting symbols. + + Provide target and index generation and registration of documented symbols + within the domain data. + + Deriving classes must have a ``cell`` attribute which refers to the cell + the documentation goes in, and a ``label`` attribute which provides a + human-readable name for what is documented, used in the index entry. + + """ + + cell_for_objtype = { + 'defcustom': 'variable', + 'defconst': 'variable', + 'defvar': 'variable', + 'defface': 'face' + } + + category_for_objtype = { + 'defcustom': 'Emacs variable (customizable)', + 'defconst': 'Emacs constant', + 'defvar': 'Emacs variable', + 'defface': 'Emacs face' + } + + @property + def cell(self): + """The cell in which to store symbol metadata.""" + return self.cell_for_objtype[self.objtype] + + @property + def label(self): + """The label for the documented object type.""" + return self.objtype + + @property + def category(self): + """Index category""" + return self.category_for_objtype[self.objtype] + + def handle_signature(self, signature, signode): + """Create nodes in ``signode`` for the ``signature``. + + ``signode`` is a docutils node to which to add the nodes, and + ``signature`` is the symbol name. + + Add the object type label before the symbol name and return + ``signature``. + + """ + label = self.label + ' ' + signode += addnodes.desc_annotation(label, label) + signode += addnodes.desc_name(signature, signature) + return signature + + def _add_index(self, name, target): + index_text = '{name}; {label}'.format( + name=name, label=self.category) + self.indexnode['entries'].append( + ('pair', index_text, target, '', None)) + + def _add_target(self, name, sig, signode): + target = make_target(self.cell, name) + if target not in self.state.document.ids: + signode['names'].append(name) + signode['ids'].append(target) + signode['first'] = (not self.names) + self.state.document.note_explicit_target(signode) + + obarray = self.env.domaindata['el']['obarray'] + symbol = obarray.setdefault(name, {}) + if self.cell in symbol: + self.state_machine.reporter.warning( + 'duplicate description of %s %s, ' % (self.objtype, name) + + 'other instance in ' + + self.env.doc2path(symbol[self.cell].docname), + line=self.lineno) + symbol[self.cell] = Cell(self.objtype, self.env.docname) + + return target + + def add_target_and_index(self, name, sig, signode): + target = self._add_target(name, sig, signode) + self._add_index(name, target) + + +class EmacsLispMinorMode(EmacsLispSymbol): + cell = 'function' + label = 'Minor Mode' + + def handle_signature(self, signature, signode): + """Create nodes in ``signode`` for the ``signature``. + + ``signode`` is a docutils node to which to add the nodes, and + ``signature`` is the symbol name. + + Add the object type label before the symbol name and return + ``signature``. + + """ + label = self.label + ' ' + signode += addnodes.desc_annotation(label, label) + signode += addnodes.desc_name(signature, to_mode_name(signature)) + return signature + + def _add_index(self, name, target): + return super()._add_index(to_mode_name(name), target) + + +class EmacsLispFunction(EmacsLispSymbol): + """A directive to document Emacs Lisp functions.""" + + cell_for_objtype = { + 'defun': 'function', + 'defmacro': 'function' + } + + def handle_signature(self, signature, signode): + function_name, *args = ws_re.split(signature) + label = self.label + ' ' + signode += addnodes.desc_annotation(label, label) + signode += addnodes.desc_name(function_name, function_name) + for arg in args: + is_keyword = arg.startswith('&') + node = (addnodes.desc_annotation + if is_keyword + else addnodes.desc_addname) + signode += node(' ' + arg, ' ' + arg) + + return function_name + + +class EmacsLispKey(ObjectDescription): + """A directive to document interactive commands via their bindings.""" + + label = 'Emacs command' + + def handle_signature(self, signature, signode): + """Create nodes to ``signode`` for ``signature``. + + ``signode`` is a docutils node to which to add the nodes, and + ``signature`` is the symbol name. + """ + key_sequence = KeySequence.fromstring(signature) + signode += addnodes.desc_name(signature, str(key_sequence)) + return str(key_sequence) + + def _add_command_target_and_index(self, name, sig, signode): + target_name = make_target('function', name) + if target_name not in self.state.document.ids: + signode['names'].append(name) + signode['ids'].append(target_name) + self.state.document.note_explicit_target(signode) + + obarray = self.env.domaindata['el']['obarray'] + symbol = obarray.setdefault(name, {}) + if 'function' in symbol: + self.state_machine.reporter.warning( + 'duplicate description of %s %s, ' % (self.objtype, name) + + 'other instance in ' + + self.env.doc2path(symbol['function'].docname), + line=self.lineno) + symbol['function'] = Cell(self.objtype, self.env.docname) + + index_text = '{name}; {label}'.format(name=name, label=self.label) + self.indexnode['entries'].append( + ('pair', index_text, target_name, '', None)) + + def _add_binding_target_and_index(self, binding, sig, signode): + reftarget = make_target('key', binding) + + if reftarget not in self.state.document.ids: + signode['names'].append(reftarget) + signode['ids'].append(reftarget) + signode['first'] = (not self.names) + self.state.document.note_explicit_target(signode) + + keymap = self.env.domaindata['el']['keymap'] + if binding in keymap: + self.state_machine.reporter.warning( + 'duplicate description of binding %s, ' % binding + + 'other instance in ' + + self.env.doc2path(keymap[binding]), + line=self.lineno) + keymap[binding] = self.env.docname + + index_text = '{name}; Emacs key binding'.format(name=binding) + self.indexnode['entries'].append( + ('pair', index_text, reftarget, '', None)) + + def add_target_and_index(self, name, sig, signode): + # If unprefixed M-x command index as function and not as key binding + sequence = KeySequence.fromstring(name) + if sequence.command_name and not sequence.has_prefix: + self._add_command_target_and_index(sequence.command_name, + sig, signode) + else: + self._add_binding_target_and_index(name, sig, signode) + + +class XRefModeRole(XRefRole): + """A role to cross-reference a minor mode. + + Like a normal cross-reference role but appends ``-mode`` to the reference + target and title-cases the symbol name like Emacs does when referring to + modes. + + """ + + fix_parens = False + lowercase = False + + def process_link(self, env, refnode, has_explicit_title, title, target): + refnode['reftype'] = 'minor-mode' + target = target + '-mode' + return (title if has_explicit_title else to_mode_name(target), target) + + +class EmacsLispDomain(Domain): + """A domain to document Emacs Lisp code.""" + + name = 'el' + label = '' + + object_types = { + # TODO: Set search prio for object types + # Types for user-facing options and commands + 'minor-mode': ObjType('minor-mode', 'function', 'mode', + cell='function'), + 'define-key': ObjType('key binding', cell='interactive'), + 'defcustom': ObjType('defcustom', 'defcustom', cell='variable'), + 'defface': ObjType('defface', 'defface', cell='face'), + # Object types for code + 'defun': ObjType('defun', 'defun', cell='function'), + 'defmacro': ObjType('defmacro', 'defmacro', cell='function'), + 'defvar': ObjType('defvar', 'defvar', cell='variable'), + 'defconst': ObjType('defconst', 'defconst', cell='variable') + } + directives = { + 'minor-mode': EmacsLispMinorMode, + 'define-key': EmacsLispKey, + 'defcustom': EmacsLispSymbol, + 'defvar': EmacsLispSymbol, + 'defconst': EmacsLispSymbol, + 'defface': EmacsLispSymbol, + 'defun': EmacsLispFunction, + 'defmacro': EmacsLispFunction + } + roles = { + 'mode': XRefModeRole(), + 'defvar': XRefRole(), + 'defconst': XRefRole(), + 'defcustom': XRefRole(), + 'defface': XRefRole(), + 'defun': XRefRole(), + 'defmacro': XRefRole() + } + + data_version = 1 + initial_data = { + # Our domain data attempts to somewhat mirror the semantics of Emacs + # Lisp, so we have an obarray which holds symbols which in turn have + # function, variable, face, etc. cells, and a keymap which holds the + # documentation for key bindings. + 'obarray': {}, + 'keymap': {} + } + + def clear_doc(self, docname): + """Clear all cells documented ``docname``.""" + for symbol in self.data['obarray'].values(): + for cell in list(symbol.keys()): + if docname == symbol[cell].docname: + del symbol[cell] + for binding in list(self.data['keymap']): + if self.data['keymap'][binding] == docname: + del self.data['keymap'][binding] + + def resolve_xref(self, env, fromdocname, builder, + objtype, target, node, contnode): + """Resolve a cross reference to ``target``.""" + if objtype == 'key': + todocname = self.data['keymap'].get(target) + if not todocname: + return None + reftarget = make_target('key', target) + else: + cell = self.object_types[objtype].attrs['cell'] + symbol = self.data['obarray'].get(target, {}) + if cell not in symbol: + return None + reftarget = make_target(cell, target) + todocname = symbol[cell].docname + + return make_refnode(builder, fromdocname, todocname, + reftarget, contnode, target) + + def resolve_any_xref(self, env, fromdocname, builder, + target, node, contnode): + """Return all possible cross references for ``target``.""" + nodes = ((objtype, self.resolve_xref(env, fromdocname, builder, + objtype, target, node, contnode)) + for objtype in ['key', 'defun', 'defvar', 'defface']) + return [('el:{}'.format(objtype), node) for (objtype, node) in nodes + if node is not None] + + def merge_warn_duplicate(self, objname, our_docname, their_docname): + self.env.warn( + their_docname, + "Duplicate declaration: '{}' also defined in '{}'.\n".format( + objname, their_docname)) + + def merge_keymapdata(self, docnames, our_keymap, their_keymap): + for key, docname in their_keymap.items(): + if docname in docnames: + if key in our_keymap: + our_docname = our_keymap[key] + self.merge_warn_duplicate(key, our_docname, docname) + else: + our_keymap[key] = docname + + def merge_obarraydata(self, docnames, our_obarray, their_obarray): + for objname, their_cells in their_obarray.items(): + our_cells = our_obarray.setdefault(objname, dict()) + for cellname, their_cell in their_cells.items(): + if their_cell.docname in docnames: + our_cell = our_cells.get(cellname) + if our_cell: + self.merge_warn_duplicate(objname, our_cell.docname, + their_cell.docname) + else: + our_cells[cellname] = their_cell + + def merge_domaindata(self, docnames, otherdata): + self.merge_keymapdata(docnames, self.data['keymap'], + otherdata['keymap']) + self.merge_obarraydata(docnames, self.data['obarray'], + otherdata['obarray']) + + def get_objects(self): + """Get all documented symbols for use in the search index.""" + for name, symbol in self.data['obarray'].items(): + for cellname, cell in symbol.items(): + yield (name, name, cell.objtype, cell.docname, + make_target(cellname, name), + self.object_types[cell.objtype].attrs['searchprio']) + + +def setup(app): + app.add_domain(EmacsLispDomain) + return {'version': '0.1', 'parallel_read_safe': True} diff --git a/doc/index.rst b/doc/index.rst index 4440d93a..fec3e6c2 100644 --- a/doc/index.rst +++ b/doc/index.rst @@ -2,34 +2,19 @@ Welcome to notmuch's documentation! =================================== -Contents: +Content +------- .. toctree:: :titlesonly: - man1/notmuch - man1/notmuch-address - man1/notmuch-compact - man1/notmuch-config - man1/notmuch-count - man1/notmuch-dump + command-line + queries notmuch-emacs - man1/notmuch-emacs-mua - man5/notmuch-hooks - man1/notmuch-insert - man1/notmuch-new - man7/notmuch-properties - man1/notmuch-reindex - man1/notmuch-reply - man1/notmuch-restore - man1/notmuch-search - man7/notmuch-search-terms - man1/notmuch-show - man1/notmuch-tag - -Indices and tables -================== + python-bindings + +Index and search +----------------- * :ref:`genindex` -* :ref:`modindex` * :ref:`search` diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst index 2a7df6f0..c70dde35 100644 --- a/doc/man1/notmuch-address.rst +++ b/doc/man1/notmuch-address.rst @@ -1,3 +1,5 @@ +.. _notmuch-address(1): + =============== notmuch-address =============== @@ -13,94 +15,102 @@ DESCRIPTION Search for messages matching the given search terms, and display the addresses from them. Duplicate addresses are filtered out. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . Supported options for **address** include -``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**) - Presents the results in either JSON, S-Expressions, newline - character separated plain-text (default), or null character - separated plain-text (compatible with **xargs(1)** -0 option where - available). - -``--format-version=N`` - Use the specified structured output format version. This is - intended for programs that invoke **notmuch(1)** internally. If - omitted, the latest supported version will be used. - -``--output=(sender|recipients|count|address)`` - Controls which information appears in the output. This option can - be given multiple times to combine different outputs. When - neither ``--output=sender`` nor ``--output=recipients`` is - given, ``--output=sender`` is implied. - - **sender** - Output all addresses from the *From* header. - - Note: Searching for **sender** should be much faster than - searching for **recipients**, because sender addresses are - cached directly in the database whereas other addresses need - to be fetched from message files. - - **recipients** - Output all addresses from the *To*, *Cc* and *Bcc* headers. - - **count** - Print the count of how many times was the address encountered - during search. - - Note: With this option, addresses are printed only after the - whole search is finished. This may take long time. - - **address** - Output only the email addresses instead of the full mailboxes - with names and email addresses. This option has no effect on - the JSON or S-Expression output formats. - -``--deduplicate=(no|mailbox|address)`` - Control the deduplication of results. - - **no** - Output all occurrences of addresses in the matching - messages. This is not applicable with ``--output=count``. - - **mailbox** - Deduplicate addresses based on the full, case sensitive name - and email address, or mailbox. This is effectively the same as - piping the ``--deduplicate=no`` output to **sort | uniq**, except - for the order of results. This is the default. - - **address** - Deduplicate addresses based on the case insensitive address - part of the mailbox. Of all the variants (with different name - or case), print the one occurring most frequently among the - matching messages. If ``--output=count`` is specified, include all - variants in the count. - -``--sort=``\ (**newest-first**\ \|\ **oldest-first**) - This option can be used to present results in either chronological - order (**oldest-first**) or reverse chronological order - (**newest-first**). - - By default, results will be displayed in reverse chronological - order, (that is, the newest results will be displayed first). - - However, if either ``--output=count`` or ``--deduplicate=address`` is - specified, this option is ignored and the order of the results is - unspecified. - -``--exclude=(true|false)`` - A message is called "excluded" if it matches at least one tag in - search.exclude\_tags that does not appear explicitly in the search - terms. This option specifies whether to omit excluded messages in - the search process. - - The default value, **true**, prevents excluded messages from - matching the search terms. - - **false** allows excluded messages to match search terms and - appear in displayed results. +.. program:: address + +.. option:: --format=(json|sexp|text|text0) + + Presents the results in either JSON, S-Expressions, newline + character separated plain-text (default), or null character + separated plain-text (compatible with :manpage:`xargs(1)` -0 + option where available). + +.. option:: --format-version=N + + Use the specified structured output format version. This is + intended for programs that invoke :any:`notmuch(1)` internally. If + omitted, the latest supported version will be used. + +.. option:: --output=(sender|recipients|count|address) + + Controls which information appears in the output. This option can + be given multiple times to combine different outputs. When + neither ``--output=sender`` nor ``--output=recipients`` is + given, ``--output=sender`` is implied. + + sender + Output all addresses from the *From* header. + + Note: Searching for **sender** should be much faster than + searching for **recipients**, because sender addresses are + cached directly in the database whereas other addresses need + to be fetched from message files. + + recipients + Output all addresses from the *To*, *Cc* and *Bcc* headers. + + count + Print the count of how many times was the address encountered + during search. + + Note: With this option, addresses are printed only after the + whole search is finished. This may take long time. + + address + Output only the email addresses instead of the full mailboxes + with names and email addresses. This option has no effect on + the JSON or S-Expression output formats. + +.. option:: --deduplicate=(no|mailbox|address) + + Control the deduplication of results. + + no + Output all occurrences of addresses in the matching + messages. This is not applicable with ``--output=count``. + + mailbox + Deduplicate addresses based on the full, case sensitive name + and email address, or mailbox. This is effectively the same as + piping the ``--deduplicate=no`` output to **sort | uniq**, except + for the order of results. This is the default. + + address + Deduplicate addresses based on the case insensitive address + part of the mailbox. Of all the variants (with different name + or case), print the one occurring most frequently among the + matching messages. If ``--output=count`` is specified, include all + variants in the count. + +.. option:: --sort=(newest-first|oldest-first) + + This option can be used to present results in either chronological + order (**oldest-first**) or reverse chronological order + (**newest-first**). + + By default, results will be displayed in reverse chronological + order, (that is, the newest results will be displayed first). + + However, if either ``--output=count`` or ``--deduplicate=address`` is + specified, this option is ignored and the order of the results is + unspecified. + +.. option:: --exclude=(true|false) + + A message is called "excluded" if it matches at least one tag in + search.exclude\_tags that does not appear explicitly in the search + terms. This option specifies whether to omit excluded messages in + the search process. + + The default value, **true**, prevents excluded messages from + matching the search terms. + + **false** allows excluded messages to match search terms and + appear in displayed results. EXIT STATUS =========== @@ -116,16 +126,16 @@ This command supports the following special exit status codes SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)**, -**notmuch-search(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-compact.rst b/doc/man1/notmuch-compact.rst index b05593ec..cb1c858b 100644 --- a/doc/man1/notmuch-compact.rst +++ b/doc/man1/notmuch-compact.rst @@ -1,3 +1,5 @@ +.. _notmuch-compact(1): + =============== notmuch-compact =============== @@ -24,37 +26,31 @@ process (which may be quite long) to protect data integrity. Supported options for **compact** include -``--backup=``\ - Save the current database to the given directory before replacing - it with the compacted database. The backup directory must not - exist and it must reside on the same mounted filesystem as the - current database. +.. program:: compact -``--quiet`` - Do not report database compaction progress to stdout. +.. option:: --backup= -ENVIRONMENT -=========== + Save the current database to the given directory before replacing + it with the compacted database. The backup directory must not + exist and it must reside on the same mounted filesystem as the + current database. -The following environment variables can be used to control the behavior -of notmuch. +.. option:: --quiet -**NOTMUCH\_CONFIG** - Specifies the location of the notmuch configuration file. Notmuch - will use ${HOME}/.notmuch-config if this variable is not set. + Do not report database compaction progress to stdout. SEE ALSO ======== -**notmuch(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-config.rst b/doc/man1/notmuch-config.rst index 28487079..bd34afa4 100644 --- a/doc/man1/notmuch-config.rst +++ b/doc/man1/notmuch-config.rst @@ -1,3 +1,5 @@ +.. _notmuch-config(1): + ============== notmuch-config ============== @@ -7,7 +9,7 @@ SYNOPSIS **notmuch** **config** **get** <*section*>.<*item*> -**notmuch** **config** **set** <*section*>.<*item*> [*value* ...] +**notmuch** **config** **set** [--database] <*section*>.<*item*> [*value* ...] **notmuch** **config** **list** @@ -17,132 +19,121 @@ DESCRIPTION The **config** command can be used to get or set settings in the notmuch configuration file and corresponding database. -Items marked **[STORED IN DATABASE]** are only in the database. They -should not be placed in the configuration file, and should be accessed -programmatically as described in the SYNOPSIS above. +.. program:: config -**get** - The value of the specified configuration item is printed to - stdout. If the item has multiple values (it is a list), each value - is separated by a newline character. +.. option:: get -**set** - The specified configuration item is set to the given value. To - specify a multiple-value item (a list), provide each value as a - separate command-line argument. + The value of the specified configuration item is printed to + stdout. If the item has multiple values (it is a list), each value + is separated by a newline character. - If no values are provided, the specified configuration item will - be removed from the configuration file. +.. option:: set -**list** - Every configuration item is printed to stdout, each on a separate - line of the form:: + The specified configuration item is set to the given value. To + specify a multiple-value item (a list), provide each value as a + separate command-line argument. - *section*.\ *item*\ =\ *value* + If no values are provided, the specified configuration item will + be removed from the configuration file. - No additional whitespace surrounds the dot or equals sign - characters. In a multiple-value item (a list), the values are - separated by semicolon characters. + With the `--database` option, updates configuration metadata + stored in the database, rather than the default (text) + configuration file. -The available configuration items are described below. +.. option:: list -**database.path** - The top-level directory where your mail currently exists and to - where mail will be delivered in the future. Files should be - individual email messages. Notmuch will store its database within - a sub-directory of the path configured here named ``.notmuch``. + Every configuration item is printed to stdout, each on a separate + line of the form:: - Default: ``$MAILDIR`` variable if set, otherwise ``$HOME/mail``. + section.item=value -**user.name** - Your full name. + No additional whitespace surrounds the dot or equals sign + characters. In a multiple-value item (a list), the values are + separated by semicolon characters. - Default: ``$NAME`` variable if set, otherwise read from - ``/etc/passwd``. +The available configuration items are described below. Non-absolute +paths are presumed relative to `$HOME` for items in section +**database**. -**user.primary\_email** - Your primary email address. +.. nmconfig:: built_with. - Default: ``$EMAIL`` variable if set, otherwise constructed from - the username and hostname of the current machine. + Compile time feature . Current possibilities include + "retry_lock" (configure option, included by default). + (since notmuch 0.30, "compact" and "field_processor" are + always included.) -**user.other\_email** - A list of other email addresses at which you receive email. +.. nmconfig:: database.autocommit - Default: not set. + How often to commit transactions to disk. `0` means wait until + command completes, otherwise an integer `n` specifies to commit to + disk after every `n` completed transactions. -**new.tags** - A list of tags that will be added to all messages incorporated by - **notmuch new**. + History: this configuration value was introduced in notmuch 0.33. - Default: ``unread;inbox``. +.. nmconfig:: database.backup_dir -**new.ignore** - A list to specify files and directories that will not be searched - for messages by **notmuch new**. Each entry in the list is either: + Directory to store tag dumps when upgrading database. - A file or a directory name, without path, that will be ignored, - regardless of the location in the mail store directory hierarchy. + History: this configuration value was introduced in notmuch 0.32. - Or: + Default: A sibling directory of the Xapian database called + `backups`. - A regular expression delimited with // that will be matched - against the path of the file or directory relative to the database - path. Matching files and directories will be ignored. The - beginning and end of string must be explicitly anchored. For - example, /.*/foo$/ would match "bar/foo" and "bar/baz/foo", but - not "foo" or "bar/foobar". +.. nmconfig:: database.hook_dir - Default: empty list. + Directory containing hooks run by notmuch commands. See + :any:`notmuch-hooks(5)`. -**search.exclude\_tags** - A list of tags that will be excluded from search results by - default. Using an excluded tag in a query will override that - exclusion. + History: this configuration value was introduced in notmuch 0.32. - Default: empty list. Note that **notmuch-setup(1)** puts - ``deleted;spam`` here when creating new configuration file. + Default: See HOOKS, below. -**maildir.synchronize\_flags** - If true, then the following maildir flags (in message filenames) - will be synchronized with the corresponding notmuch tags: +.. nmconfig:: database.mail_root - +--------+-----------------------------------------------+ - | Flag | Tag | - +========+===============================================+ - | D | draft | - +--------+-----------------------------------------------+ - | F | flagged | - +--------+-----------------------------------------------+ - | P | passed | - +--------+-----------------------------------------------+ - | R | replied | - +--------+-----------------------------------------------+ - | S | unread (added when 'S' flag is not present) | - +--------+-----------------------------------------------+ + The top-level directory where your mail currently exists and to + where mail will be delivered in the future. Files should be + individual email messages. - The **notmuch new** command will notice flag changes in filenames - and update tags, while the **notmuch tag** and **notmuch restore** - commands will notice tag changes and update flags in filenames. + History: this configuration value was introduced in notmuch 0.32. - If there have been any changes in the maildir (new messages added, - old ones removed or renamed, maildir flags changed, etc.), it is - advisable to run **notmuch new** before **notmuch tag** or - **notmuch restore** commands to ensure the tag changes are - properly synchronized to the maildir flags, as the commands expect - the database and maildir to be in sync. + Default: For compatibility with older configurations, the value of + database.path is used if :nmconfig:`database.mail_root` is unset. - Default: ``true``. +.. nmconfig:: database.path + + Notmuch will store its database here, (in + sub-directory named ``.notmuch`` if :nmconfig:`database.mail_root` + is unset). -**crypto.gpg_path** - Name (or full path) of gpg binary to use in verification and - decryption of PGP/MIME messages. NOTE: This configuration item is - deprecated, and will be ignored if notmuch is built against GMime - 3.0 or later. + Default: see :ref:`database` - Default: ``gpg``. +.. nmconfig:: git.path + + Default location for git repository for :any:`notmuch-git`. + +.. nmconfig:: git.safe_fraction + + Some :any:`notmuch-git` operations check that the fraction of + messages changed (in the database or in git, as appropriate) is not + too large. This item controls what fraction of total messages is + considered "not too large". + +.. nmconfig:: git.tag_prefix + + Default tag prefix (filter) for :any:`notmuch-git`. + +.. nmconfig:: index.as_text + + List of regular expressions (without delimiters) for MIME types to + be indexed as text. Currently this applies only to attachments. By + default the regex matches anywhere in the content type; if the + user wants an anchored match, they should include anchors in their + regexes. + + History: This configuration value was introduced in notmuch 0.38. + +.. nmconfig:: index.decrypt -**index.decrypt** **[STORED IN DATABASE]** Policy for decrypting encrypted messages during indexing. Must be one of: ``false``, ``auto``, ``nostash``, or ``true``. @@ -158,8 +149,8 @@ The available configuration items are described below. ``nostash`` is the same as ``true`` except that it will not stash newly-discovered session keys in the database. - From the command line (i.e. during **notmuch-new(1)**, - **notmuch-insert(1)**, or **notmuch-reindex(1)**), the user can + From the command line (i.e. during :any:`notmuch-new(1)`, + :any:`notmuch-insert(1)`, or :any:`notmuch-reindex(1)`), the user can override the database's stored decryption policy with the ``--decrypt=`` option. @@ -183,7 +174,7 @@ The available configuration items are described below. Stashed session keys are kept in the database as properties associated with the message. See ``session-key`` in - **notmuch-properties(7)** for more details about how they can be + :any:`notmuch-properties(7)` for more details about how they can be useful. Be aware that the notmuch index is likely sufficient (and a @@ -195,48 +186,200 @@ The available configuration items are described below. Default: ``auto``. -**index.header.** **[STORED IN DATABASE]** +.. _index.header: + +.. nmconfig:: index.header. + Define the query prefix , based on a mail header. For example ``index.header.List=List-Id`` will add a probabilistic prefix ``List:`` that searches the ``List-Id`` field. User defined prefixes must not start with 'a'...'z'; in particular adding a prefix with same name as a predefined prefix is not - supported. See **notmuch-search-terms(7)** for a list of existing + supported. See :any:`notmuch-search-terms(7)` for a list of existing prefixes, and an explanation of probabilistic prefixes. -**built_with.** - Compile time feature . Current possibilities include - "compact" (see **notmuch-compact(1)**) and "field_processor" (see - **notmuch-search-terms(7)**). +.. nmconfig:: maildir.synchronize_flags + + If true, then the following maildir flags (in message filenames) + will be synchronized with the corresponding notmuch tags: + + +--------+-----------------------------------------------+ + | Flag | Tag | + +========+===============================================+ + | D | draft | + +--------+-----------------------------------------------+ + | F | flagged | + +--------+-----------------------------------------------+ + | P | passed | + +--------+-----------------------------------------------+ + | R | replied | + +--------+-----------------------------------------------+ + | S | unread (added when 'S' flag is not present) | + +--------+-----------------------------------------------+ + + The :any:`notmuch-new(1)` command will notice flag changes in + filenames and update tags, while the :any:`notmuch-tag(1)` and + :any:`notmuch-restore(1)` commands will notice tag changes and + update flags in filenames. + + If there have been any changes in the maildir (new messages added, + old ones removed or renamed, maildir flags changed, etc.), it is + advisable to run :any:`notmuch-new(1)` before + :any:`notmuch-tag(1)` or :any:`notmuch-restore(1)` commands to + ensure the tag changes are properly synchronized to the maildir + flags, as the commands expect the database and maildir to be in + sync. + + Default: ``true``. + +.. nmconfig:: new.ignore + + A list to specify files and directories that will not be searched + for messages by :any:`notmuch-new(1)`. Each entry in the list is either: + + A file or a directory name, without path, that will be ignored, + regardless of the location in the mail store directory hierarchy. + + Or: + + A regular expression delimited with // that will be matched + against the path of the file or directory relative to the database + path. Matching files and directories will be ignored. The + beginning and end of string must be explicitly anchored. For + example, /.*/foo$/ would match "bar/foo" and "bar/baz/foo", but + not "foo" or "bar/foobar". + + Default: empty list. + +.. nmconfig:: new.tags + + A list of tags that will be added to all messages incorporated by + **notmuch new**. + + Default: ``unread;inbox``. + +.. nmconfig:: query. -**query.** **[STORED IN DATABASE]** Expansion for named query called . See - **notmuch-search-terms(7)** for more information about named + :any:`notmuch-search-terms(7)` for more information about named queries. -ENVIRONMENT -=========== +.. nmconfig:: search.exclude_tags + + A list of tags that will be excluded from search results by + default. Using an excluded tag in a query will override that + exclusion. + + Default: empty list. Note that :any:`notmuch-setup(1)` puts + ``deleted;spam`` here when creating new configuration file. + +.. nmconfig:: show.extra_headers + + By default :any:`notmuch-show(1)` includes the following headers + in structured output if they are present in the message: + `Subject`, `From`, `To`, `Cc`, `Bcc`, `Reply-To`, `Date`. This + option allows the specification of a list of further + headers to output. + + History: This configuration value was introduced in notmuch 0.35. + + Default: empty list. + +.. nmconfig:: squery. + + Expansion for named query called , using s-expression syntax. See + :any:`notmuch-sexp-queries(7)` for more information about s-expression + queries. + +.. nmconfig:: user.name + + Your full name. + + Default: ``$NAME`` variable if set, otherwise read from + ``/etc/passwd``. + +.. nmconfig:: user.other_email + + A list of other email addresses at which you receive email + (see also, :nmconfig:`user.primary_email`) + + Default: not set. + +.. nmconfig:: user.primary_email + + Your primary email address. + + Default: ``$EMAIL`` variable if set, otherwise constructed from + the username and hostname of the current machine. + +FILES +===== + +.. _config_search: + +CONFIGURATION +------------- + +Notmuch configuration file search order: + +1. File specified by :option:`notmuch --config` global option; see + :any:`notmuch(1)`. + +2. File specified by :envvar:`NOTMUCH_CONFIG` environment variable. + +3. ``$XDG_CONFIG_HOME/notmuch//config`` where ```` + is defined by :envvar:`NOTMUCH_PROFILE` environment variable if + set, ``$XDG_CONFIG_HOME/notmuch/default/config`` otherwise. + +4. ``$HOME/.notmuch-config.`` where ```` is defined + by :envvar:`NOTMUCH_PROFILE` environment variable if set, + ``$HOME/.notmuch-config`` otherwise. + +.. _database: + +DATABASE LOCATION +----------------- + +Notmuch database search order: + +1. Directory specified by :envvar:`NOTMUCH_DATABASE` environment variable. + +2. Directory specified by config key ``database.path``. + +3. ``$XDG_DATA_HOME/notmuch/`` where ```` + is defined by :envvar:`NOTMUCH_PROFILE` environment variable if + set, ``$XDG_DATA_HOME/notmuch/default`` otherwise. + +4. Directory specified by :envvar:`MAILDIR` environment variable. + +5. ``$HOME/mail`` + +HOOKS +----- + +Notmuch hook directory search order: + +1. Directory specified by ``database.hook_dir`` configuration option. -The following environment variables can be used to control the behavior -of notmuch. +2. ``$XDG_CONFIG_HOME/notmuch//hooks`` where ```` + is defined by :envvar:`NOTMUCH_PROFILE` environment variable if + set, ``$XDG_CONFIG_HOME/notmuch/default/hooks`` otherwise. -**NOTMUCH\_CONFIG** - Specifies the location of the notmuch configuration file. Notmuch - will use ${HOME}/.notmuch-config if this variable is not set. +3. ``/.notmuch/hooks`` SEE ALSO ======== -**notmuch(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-properties(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-properties(7)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-count.rst b/doc/man1/notmuch-count.rst index 0eac5dbe..4c9c9a1c 100644 --- a/doc/man1/notmuch-count.rst +++ b/doc/man1/notmuch-count.rst @@ -1,3 +1,5 @@ +.. _notmuch-count(1): + ============= notmuch-count ============= @@ -17,56 +19,63 @@ The number of matching messages (or threads) is output to stdout. With no search terms, a count of all messages (or threads) in the database will be displayed. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . Supported options for **count** include -``--output=(messages|threads|files)`` - **messages** - Output the number of matching messages. This is the default. +.. program:: count + +.. option:: --output=(messages|threads|files) + + messages + Output the number of matching messages. This is the default. + + threads + Output the number of matching threads. + + files + Output the number of files associated with matching + messages. This may be bigger than the number of matching + messages due to duplicates (i.e. multiple files having the + same message-id). + +.. option:: --exclude=(true|false) + + Specify whether to omit messages matching search.exclude\_tags from + the count (the default) or not. - **threads** - Output the number of matching threads. +.. option:: --batch - **files** - Output the number of files associated with matching - messages. This may be bigger than the number of matching - messages due to duplicates (i.e. multiple files having the - same message-id). + Read queries from a file (stdin by default), one per line, and + output the number of matching messages (or threads) to stdout, one + per line. On an empty input line the count of all messages (or + threads) in the database will be output. This option is not + compatible with specifying search terms on the command line. -``--exclude=(true|false)`` - Specify whether to omit messages matching search.exclude\_tags from - the count (the default) or not. +.. option:: --lastmod -``--batch`` - Read queries from a file (stdin by default), one per line, and - output the number of matching messages (or threads) to stdout, one - per line. On an empty input line the count of all messages (or - threads) in the database will be output. This option is not - compatible with specifying search terms on the command line. + Append lastmod (counter for number of database updates) and UUID + to the output. lastmod values are only comparable between + databases with the same UUID. -``--lastmod`` - Append lastmod (counter for number of database updates) and UUID - to the output. lastmod values are only comparable between - databases with the same UUID. +.. option:: --input= -``--input=``\ - Read input from given file, instead of from stdin. Implies - ``--batch``. + Read input from given file, instead of from stdin. Implies + ``--batch``. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-dump.rst b/doc/man1/notmuch-dump.rst index ec6335b2..a7ca39d0 100644 --- a/doc/man1/notmuch-dump.rst +++ b/doc/man1/notmuch-dump.rst @@ -1,3 +1,5 @@ +.. _notmuch-dump(1): + ============ notmuch-dump ============ @@ -19,97 +21,103 @@ recreated from the messages themselves. The output of notmuch dump is therefore the only critical thing to backup (and much more friendly to incremental backup than the native database files.) -See **notmuch-search-terms(7)** for details of the supported syntax +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . With no search terms, a dump of all messages in the database will be generated. A ``--`` argument instructs notmuch that the remaining arguments are search terms. Supported options for **dump** include -``--gzip`` - Compress the output in a format compatible with **gzip(1)**. - -``--format=(sup|batch-tag)`` - Notmuch restore supports two plain text dump formats, both with - one message-id per line, followed by a list of tags. - - **batch-tag** - The default **batch-tag** dump format is intended to more - robust against malformed message-ids and tags containing - whitespace or non-\ **ascii(7)** characters. Each line has the - form:: - - +<*encoded-tag*\ > +<*encoded-tag*\ > ... -- id:<*quoted-message-id*\ > - - Tags are hex-encoded by replacing every byte not matching the - regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is the two - digit hex encoding. The message ID is a valid Xapian query, - quoted using Xapian boolean term quoting rules: if the ID - contains whitespace or a close paren or starts with a double - quote, it must be enclosed in double quotes and double quotes - inside the ID must be doubled. The astute reader will notice - this is a special case of the batch input format for - **notmuch-tag(1)**; note that the single message-id query is - mandatory for **notmuch-restore(1)**. - - **sup** - The **sup** dump file format is specifically chosen to be - compatible with the format of files produced by sup-dump. So - if you've previously been using sup for mail, then the - **notmuch restore** command provides you a way to import all - of your tags (or labels as sup calls them). Each line has the - following form:: - - <*message-id*\ > **(** <*tag*\ > ... **)** - - with zero or more tags are separated by spaces. Note that - (malformed) message-ids may contain arbitrary non-null - characters. Note also that tags with spaces will not be - correctly restored with this format. - -``--include=(config|properties|tags)`` - Control what kind of metadata is included in the output. - - **config** - Output configuration data stored in the database. Each line - starts with "#@ ", followed by a space separated key-value - pair. Both key and value are hex encoded if needed. - - **properties** - Output per-message (key,value) metadata. Each line starts - with "#= ", followed by a message id, and a space separated - list of key=value pairs. Ids, keys and values are hex encoded - if needed. See **notmuch-properties(7)** for more details. - - **tags** - Output per-message boolean metadata, namely tags. See *format* above - for description of the output. - - The default is to include all available types of data. The option - can be specified multiple times to select some subset. As of - version 3 of the dump format, there is a header line of the - following form:: - - #notmuch-dump <*format*>:<*version*> <*included*> - - where <*included*> is a comma separated list of the above options. - -``--output=``\ - Write output to given file instead of stdout. +.. program:: dump + +.. option:: --gzip + + Compress the output in a format compatible with :manpage:`gzip(1)`. + +.. option:: --format=(sup|batch-tag) + + Notmuch restore supports two plain text dump formats, both with + one message-id per line, followed by a list of tags. + + batch-tag + The default **batch-tag** dump format is intended to more + robust against malformed message-ids and tags containing + whitespace or non-\ :manpage:`ascii(7)` characters. Each line + has the form:: + + +<*encoded-tag*\ > +<*encoded-tag*\ > ... -- id:<*quoted-message-id*\ > + + Tags are hex-encoded by replacing every byte not matching the + regex **[A-Za-z0-9@=.,\_+-]** with **%nn** where nn is the two + digit hex encoding. The message ID is a valid Xapian query, + quoted using Xapian boolean term quoting rules: if the ID + contains whitespace or a close paren or starts with a double + quote, it must be enclosed in double quotes and double quotes + inside the ID must be doubled. The astute reader will notice + this is a special case of the batch input format for + :any:`notmuch-tag(1)`; note that the single message-id query is + mandatory for :any:`notmuch-restore(1)`. + + sup + The **sup** dump file format is specifically chosen to be + compatible with the format of files produced by + :manpage:`sup-dump(1)`. So if you've previously been using sup + for mail, then the :any:`notmuch-restore(1)` command provides + you a way to import all of your tags (or labels as sup calls + them). Each line has the following form:: + + <*message-id*\ > **(** <*tag*\ > ... **)** + + with zero or more tags are separated by spaces. Note that + (malformed) message-ids may contain arbitrary non-null + characters. Note also that tags with spaces will not be + correctly restored with this format. + +.. option:: --include=(config|properties|tags) + + Control what kind of metadata is included in the output. + + config + Output configuration data stored in the database. Each line + starts with "#@ ", followed by a space separated key-value + pair. Both key and value are hex encoded if needed. + + properties + Output per-message (key,value) metadata. Each line starts + with "#= ", followed by a message id, and a space separated + list of key=value pairs. Ids, keys and values are hex encoded + if needed. See :any:`notmuch-properties(7)` for more details. + + tags + Output per-message boolean metadata, namely tags. See *format* above + for description of the output. + + The default is to include all available types of data. The option + can be specified multiple times to select some subset. As of + version 3 of the dump format, there is a header line of the + following form:: + + #notmuch-dump <*format*>:<*version*> <*included*> + + where <*included*> is a comma separated list of the above options. + +.. option:: --output= + + Write output to given file instead of stdout. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-properties(7)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-properties(7)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-emacs-mua.rst b/doc/man1/notmuch-emacs-mua.rst index a0476136..d8af08bd 100644 --- a/doc/man1/notmuch-emacs-mua.rst +++ b/doc/man1/notmuch-emacs-mua.rst @@ -1,3 +1,5 @@ +.. _notmuch-emacs-mua(1): + ================= notmuch-emacs-mua ================= @@ -15,67 +17,86 @@ subject, recipients, and message body, or mailto: URL. Supported options for **emacs-mua** include -``-h, --help`` - Display help. +.. program:: emacs-mua + +.. option:: -h, --help + + Display help. + +.. option:: -s, --subject= + + Specify the subject of the message. + +.. option:: --to= + + Specify a recipient (To). + +.. option:: -c, --cc= -``-s, --subject=``\ - Specify the subject of the message. + Specify a carbon-copy (Cc) recipient. -``--to=``\ - Specify a recipient (To). +.. option:: -b, --bcc= -``-c, --cc=``\ - Specify a carbon-copy (Cc) recipient. + Specify a blind-carbon-copy (Bcc) recipient. -``-b, --bcc=``\ - Specify a blind-carbon-copy (Bcc) recipient. +.. option:: -i, --body= -``-i, --body=``\ - Specify a file to include into the body of the message. + Specify a file to include into the body of the message. -``--hello`` - Go to the Notmuch hello screen instead of the message composition - window if no message composition parameters are given. +.. option:: --hello -``--no-window-system`` - Even if a window system is available, use the current terminal. + Go to the Notmuch hello screen instead of the message composition + window if no message composition parameters are given. -``--client`` - Use **emacsclient**, rather than **emacs**. For **emacsclient** to - work, you need an already running Emacs with a server, or use - ``--auto-daemon``. +.. option:: --no-window-system -``--auto-daemon`` - Automatically start Emacs in daemon mode, if the Emacs server is - not running. Applicable with ``--client``. Implies - ``--create-frame``. + Even if a window system is available, use the current terminal. -``--create-frame`` - Create a new frame instead of trying to use the current Emacs - frame. Applicable with ``--client``. This will be required when - Emacs is running (or automatically started with ``--auto-daemon``) - in daemon mode. +.. option:: --client -``--print`` - Output the resulting elisp to stdout instead of evaluating it. + Use :manpage:`emacsclient(1)`, rather than + :manpage:`emacs(1)`. For :manpage:`emacsclient(1)` to work, you + need an already running Emacs with a server, or use + ``--auto-daemon``. + +.. option:: --auto-daemon + + Automatically start Emacs in daemon mode, if the Emacs server is + not running. Applicable with ``--client``. Implies + ``--create-frame``. + +.. option:: --create-frame + + Create a new frame instead of trying to use the current Emacs + frame. Applicable with ``--client``. This will be required when + Emacs is running (or automatically started with ``--auto-daemon``) + in daemon mode. + +.. option:: --print + + Output the resulting elisp to stdout instead of evaluating it. The supported positional parameters and short options are a compatible -subset of the **mutt** MUA command-line options. The options and -positional parameters modifying the message can't be combined with the -mailto: URL. +subset of the :manpage:`mutt(1)` MUA command-line options. The options +and positional parameters modifying the message can't be combined with +the mailto: URL. Options may be specified multiple times. ENVIRONMENT VARIABLES ===================== -**EMACS** - Name of emacs command to invoke. Defaults to "emacs". +.. envvar:: EMACS + + Name of emacs command to invoke. Defaults to "emacs". + +.. envvar:: EMACSCLIENT -**EMACSCLIENT** - Name of emacsclient command to invoke. Defaults to "emacsclient". + Name of emacsclient command to invoke. Defaults to "emacsclient". SEE ALSO ======== -**notmuch(1)**, **emacsclient(1)**, **mutt(1)** +:any:`notmuch(1)`, +:manpage:`emacsclient(1)`, +:manpage:`mutt(1)` diff --git a/doc/man1/notmuch-git.rst b/doc/man1/notmuch-git.rst new file mode 100644 index 00000000..33a46f84 --- /dev/null +++ b/doc/man1/notmuch-git.rst @@ -0,0 +1,340 @@ +.. _notmuch-git(1): + +=========== +notmuch-git +=========== + +SYNOPSIS +======== + +**notmuch** **git** [-h] [-N] [-C *repo*] [-p *prefix*] [-v] [-l *log level*] *subcommand* + +**nmbug** [-h] [-C *repo*] [-p *prefix*] [-v] [-l *log level*] *subcommand* + +DESCRIPTION +=========== + +Manage notmuch tags with Git. + +OPTIONS +------- + +Supported options for `notmuch git` include + +.. program:: notmuch-git + +.. option:: -h, --help + + show help message and exit + +.. option:: -N, --nmbug + + Set defaults for :option:`--tag-prefix` and :option:`--git-dir` suitable for the + :any:`notmuch` bug tracker + +.. option:: -C , --git-dir + + Operate on git repository *repo*. See :ref:`repo_location` for + defaults. + +.. option:: -p , --tag-prefix + + Operate only on tags with prefix *prefix*. See :ref:`prefix_val` for + defaults. + +.. option:: -v, --version + + show notmuch-git's version number and exit + +.. option:: -l , --log-level + + Log verbosity, one of: `critical`, `error`, `warning`, `info`, + `debug`. Defaults to `warning`. + +SUBCOMMANDS +----------- + +For help on a particular subcommand, run: 'notmuch-git ... --help'. + +.. program:: notmuch-git + +.. option:: archive [tree-ish] [arg ...] + +Dump a tar archive of a committed tag set using 'git archive'. See +:any:`format` for details of the archive contents. + + .. describe:: tree-ish + + The tree or commit to produce an archive for. Defaults to 'HEAD'. + + .. describe:: arg + + If present, any optional arguments are passed through to + :manpage:`git-archive(1)`. Arguments to `git-archive` are reordered + so that *tree-ish* comes last. + +.. option:: checkout [-f|--force] + +Update the notmuch database from Git. + +This is mainly useful to discard your changes in notmuch relative +to Git. + + .. describe:: [-f|--force] + + Override checks that prevent modifying tags for large fractions of + messages in the database. See also :nmconfig:`git.safe_fraction`. + +.. option:: clone + +Create a local `notmuch git` repository from a remote source. + +This wraps 'git clone', adding some options to avoid creating a +working tree while preserving remote-tracking branches and +upstreams. + + .. describe:: repository + + The (possibly remote) repository to clone from. See the URLS + section of :manpage:`git-clone(1)` for more information on + specifying repositories. + +.. option:: commit [-f|--force] [message] + +Commit prefix-matching tags from the notmuch database to Git. + + .. describe:: message + + Optional text for the commit message. + + .. describe:: -f|--force + + Override checks that prevent modifying tags for large fractions of + messages in the database. See also :nmconfig:`git.safe_fraction`. + +.. option:: fetch [remote] + +Fetch changes from the remote repository. + + .. describe:: remote + + Override the default configured in `branch..remote` to fetch + from a particular remote repository (e.g. `origin`). + +.. option:: help + +Show brief help for an `notmuch git` command. + +.. option:: init [--format-version=N] + +Create an empty `notmuch git` repository. + +This wraps 'git init' with a few extra steps to support subsequent +status and commit commands. + + .. describe:: --format-version=N + + Create a repo in format version N. By default :any:`notmuch-git` + uses the highest supported version, which is the best choice for + most use-cases. + +.. option:: log [arg ...] + +A wrapper for 'git log'. + + .. describe:: arg + + Additional arguments are passed through to 'git log'. + +After running `notmuch git fetch`, you can inspect the changes with + +:: + + $ notmuch git log HEAD..@{upstream} + +.. option:: merge [reference] + +Merge changes from 'reference' into HEAD and load the result into notmuch. + + .. describe:: reference + + Reference, usually other branch heads, to merge into our + branch. Defaults to `@{upstream}`. + +.. option:: pull [repository] [refspec ...] + +Pull (merge) remote repository changes to notmuch. + +**pull** is equivalent to **fetch** followed by **merge**. We use the +Git-configured repository for your current branch +(`branch..repository`, likely `origin`, and `branch..merge`, +likely `master` or `main`). + + .. describe:: repository + + The "remote" repository that is the source of the pull. This parameter + can be either a URL (see the section GIT URLS in :manpage:`git-pull(1)`) or the + name of a remote (see the section REMOTES in :manpage:`git-pull(1)`). + + .. describe:: refspec + + Refspec (usually a branch name) to fetch and merge. See the + *refspec* entry in the OPTIONS section of :manpage:`git-pull(1`) for + other possibilities. + +.. option:: push [repository] [refspec] + +Push the local `notmuch git` Git state to a remote repository. + + .. describe:: repository + + The "remote" repository that is the destination of the push. This + parameter can be either a URL (see the section GIT URLS in + :manpage:`git-push(1)`) or the name of a remote (see the section + REMOTES in :manpage:`git-push(1)`). + + .. describe:: refspec + + Refspec (usually a branch name) to push. See the *refspec* entry in the OPTIONS section of + :manpage:`git-push(1)` for other possibilities. + +.. option:: status + +Show pending updates in notmuch or git repo. + +Prints lines of the form + +| ng Message-Id tag + +where n is a single character representing notmuch database status + + .. describe:: A + + Tag is present in notmuch database, but not committed to nmbug + (equivalently, tag has been deleted in nmbug repo, e.g. by a + pull, but not restored to notmuch database). + + .. describe:: D + + Tag is present in nmbug repo, but not restored to notmuch + database (equivalently, tag has been deleted in notmuch). + + .. describe:: U + + Message is unknown (missing from local notmuch database). + +The second character *g* (if present) represents a difference between +local and upstream branches. Typically `notmuch git fetch` needs to be +run to update this. + + .. describe:: a + + Tag is present in upstream, but not in the local Git branch. + + .. describe:: d + + Tag is present in local Git branch, but not upstream. + +.. _format: + +REPOSITORY CONTENTS +=================== + +The tags are stored in the git repo (and exported) as a set of empty +files. These empty files are contained within a directory named after +the message-id. + +In what follows `encode()` represents a POSIX filesystem safe +encoding. The encoding preserves alphanumerics, and the characters +`+-_@=.,:`. All other octets are replaced with `%` followed by a two +digit hex number. + +Currently :any:`notmuch-git` can read any format version, but can only +create (via :any:`init`) :ref:`version 1 ` repositories. + +.. _format_version_0: + +Version 0 +--------- + +This is the legacy format created by the `nmbug` tool prior to release +0.37. For a message with Message-Id *id*, for each tag *tag*, there +is an empty file with path + + tags/ `encode` (*id*) / `encode` (*tag*) + +.. _format_version_1: + +Version 1 +--------- + +In format version 1 and later, the format version is contained in a +top level file called FORMAT. + +For a message with Message-Id *id*, for each tag *tag*, there +is an empty file with path + + tags/ `hash1` (*id*) / `hash2` (*id*) `encode` (*id*) / `encode` (*tag*) + +The hash functions each represent one byte of the `blake2b` hex +digest. + +Compared to :ref:`version 0 `, this reduces the +number of subdirectories within each directory. + +.. _repo_location: + +REPOSITORY LOCATION +=================== + +:any:`notmuch-git` uses the first of the following with a non-empty +value to locate the git repository. + +- Option :option:`--git-dir`. + +- Environment variable :envvar:`NOTMUCH_GIT_DIR`. + +- Configuration item :nmconfig:`git.path` + +- If invoked as `nmbug` or with the :option:`--nmbug` option, + :code:`$HOME/.nmbug`; otherwise + :code:`$XDG_DATA_HOME/notmuch/$NOTMUCH_PROFILE/git`. + +.. _prefix_val: + +PREFIX VALUE +============ + +:any:`notmuch-git` uses the first of the following with a non-null +value to define the tag prefix. + +- Option :option:`--tag-prefix`. + +- Environment variable :envvar:`NOTMUCH_GIT_PREFIX`. + +- Configuration item :nmconfig:`git.tag_prefix`. + +- If invoked as `nmbug` or with the :option:`--nmbug` option, + :code:`notmuch::`, otherwise the empty string. + +ENVIRONMENT +=========== + +Variable :envvar:`NOTMUCH_PROFILE` influences :ref:`repo_location`. +If it is unset, 'default' is assumed. + +.. envvar:: NOTMUCH_GIT_DIR + + Default location of git repository. Overridden by :option:`--git-dir`. + +.. envvar:: NOTMUCH_GIT_PREFIX + + Default tag prefix (filter). Overridden by :option:`--tag-prefix`. + +SEE ALSO +======== + +:any:`notmuch(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-insert.rst b/doc/man1/notmuch-insert.rst index 86e2f567..e05bd0b5 100644 --- a/doc/man1/notmuch-insert.rst +++ b/doc/man1/notmuch-insert.rst @@ -1,3 +1,5 @@ +.. _notmuch-insert(1): + ============== notmuch-insert ============== @@ -12,12 +14,12 @@ DESCRIPTION **notmuch insert** reads a message from standard input and delivers it into the maildir directory given by configuration option -**database.path**, then incorporates the message into the notmuch +:nmconfig:`database.mail_root`, then incorporates the message into the notmuch database. It is an alternative to using a separate tool to deliver the -message then running **notmuch new** afterwards. +message then running :any:`notmuch-new(1)` afterwards. The new message will be tagged with the tags specified by the -**new.tags** configuration option, then by operations specified on the +:nmconfig:`new.tags` configuration option, then by operations specified on the command-line: tags prefixed by '+' are added while those prefixed by '-' are removed. @@ -25,58 +27,75 @@ If the new message is a duplicate of an existing message in the database (it has same Message-ID), it will be added to the maildir folder and notmuch database, but the tags will not be changed. -The **insert** command supports hooks. See **notmuch-hooks(5)** for +The **insert** command supports hooks. See :any:`notmuch-hooks(5)` for more details on hooks. Option arguments must appear before any tag operation arguments. Supported options for **insert** include -``--folder=<``\ folder\ **>** - Deliver the message to the specified folder, relative to the - top-level directory given by the value of **database.path**. The - default is the empty string, which means delivering to the - top-level directory. - -``--create-folder`` - Try to create the folder named by the ``--folder`` option, if it - does not exist. Otherwise the folder must already exist for mail - delivery to succeed. - -``--keep`` - Keep the message file if indexing fails, and keep the message - indexed if applying tags or maildir flag synchronization - fails. Ignore these errors and return exit status 0 to indicate - successful mail delivery. - -``--no-hooks`` - Prevent hooks from being run. - -``--world-readable`` - When writing mail to the mailbox, allow it to be read by users - other than the current user. Note that this does not override - umask. By default, delivered mail is only readable by the current - user. - -``--decrypt=(true|nostash|auto|false)`` - If ``true`` and the message is encrypted, try to decrypt the - message while indexing, stashing any session keys discovered. If - ``auto``, and notmuch already knows about a session key for the - message, it will try decrypting using that session key but will - not try to access the user's secret keys. If decryption is - successful, index the cleartext itself. Either way, the message - is always stored to disk in its original form (ciphertext). - - ``nostash`` is the same as ``true`` except that it will not stash - newly-discovered session keys in the database. - - Be aware that the index is likely sufficient (and a stashed - session key is certainly sufficient) to reconstruct the cleartext - of the message itself, so please ensure that the notmuch message - index is adequately protected. DO NOT USE ``--decrypt=true`` or - ``--decrypt=nostash`` without considering the security of your - index. - - See also ``index.decrypt`` in **notmuch-config(1)**. +.. program:: insert + +.. option:: --folder= + + Deliver the message to the specified folder, relative to the + top-level directory given by the value of :nmconfig:`database.mail_root`. The + default is the empty string, which means delivering to the + top-level directory. + +.. option:: --create-folder + + Try to create the folder named by the ``--folder`` option, if it + does not exist. Otherwise the folder must already exist for mail + delivery to succeed. + +.. option:: --keep + + Keep the message file if indexing fails, and keep the message + indexed if applying tags or maildir flag synchronization + fails. Ignore these errors and return exit status 0 to indicate + successful mail delivery. + +.. option:: --no-hooks + + Prevent hooks from being run. + +.. option:: --world-readable + + When writing mail to the mailbox, allow it to be read by users + other than the current user. Note that this does not override + umask. By default, delivered mail is only readable by the current + user. + +.. option:: --decrypt=(true|nostash|auto|false) + + If ``true`` and the message is encrypted, try to decrypt the + message while indexing, stashing any session keys discovered. If + ``auto``, and notmuch already knows about a session key for the + message, it will try decrypting using that session key but will + not try to access the user's secret keys. If decryption is + successful, index the cleartext itself. Either way, the message + is always stored to disk in its original form (ciphertext). + + ``nostash`` is the same as ``true`` except that it will not stash + newly-discovered session keys in the database. + + Be aware that the index is likely sufficient (and a stashed + session key is certainly sufficient) to reconstruct the cleartext + of the message itself, so please ensure that the notmuch message + index is adequately protected. DO NOT USE ``--decrypt=true`` or + ``--decrypt=nostash`` without considering the security of your + index. + + See also :nmconfig:`index.decrypt` in :any:`notmuch-config(1)`. + +CONFIGURATION +============= + +Indexing is influenced by the configuration options +:nmconfig:`index.decrypt` and :nmconfig:`index.header.\`. Tagging +is controlled by options :nmconfig:`new.tags` and +:nmconfig:`maildir.synchronize_flags`. See +:any:`notmuch-config(1)` for details. EXIT STATUS =========== @@ -101,14 +120,14 @@ status of the **insert** command. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-new.rst b/doc/man1/notmuch-new.rst index 20a1103b..f0ed8eb8 100644 --- a/doc/man1/notmuch-new.rst +++ b/doc/man1/notmuch-new.rst @@ -1,3 +1,5 @@ +.. _notmuch-new(1): + =========== notmuch-new =========== @@ -17,56 +19,74 @@ performing full-text indexing on new messages that are found. Each new message will automatically be tagged with both the **inbox** and **unread** tags. -You should run **notmuch new** once after first running **notmuch -setup** to create the initial database. The first run may take a long -time if you have a significant amount of mail (several hundred thousand -messages or more). Subsequently, you should run **notmuch new** whenever -new mail is delivered and you wish to incorporate it into the database. -These subsequent runs will be much quicker than the initial run. +You should run **notmuch new** once after first running +:any:`notmuch-setup(1)` to create the initial database. The first run +may take a long time if you have a significant amount of mail (several +hundred thousand messages or more). Subsequently, you should run +**notmuch new** whenever new mail is delivered and you wish to +incorporate it into the database. These subsequent runs will be much +quicker than the initial run. Invoking ``notmuch`` with no command argument will run **new** if -**notmuch setup** has previously been completed, but **notmuch new** has -not previously been run. +:any:`notmuch-setup(1)` has previously been completed, but **notmuch +new** has not previously been run. **notmuch new** updates tags according to maildir flag changes if the **maildir.synchronize\_flags** configuration option is enabled. See -**notmuch-config(1)** for details. +:any:`notmuch-config(1)` for details. -The **new** command supports hooks. See **notmuch-hooks(5)** for more +The **new** command supports hooks. See :any:`notmuch-hooks(5)` for more details on hooks. Supported options for **new** include -``--no-hooks`` - Prevents hooks from being run. +.. program:: new + +.. option:: --no-hooks + + Prevents hooks from being run. + +.. option:: --quiet + + Do not print progress or results. + +.. option:: --verbose + + Print file names being processed. Ignored when combined with + ``--quiet``. + +.. option:: --decrypt=(true|nostash|auto|false) + + If ``true``, when encountering an encrypted message, try to + decrypt it while indexing, and stash any discovered session keys. + If ``auto``, try to use any session key already known to belong to + this message, but do not attempt to use the user's secret keys. + If decryption is successful, index the cleartext of the message. -``--quiet`` - Do not print progress or results. + Be aware that the index is likely sufficient (and the session key + is certainly sufficient) to reconstruct the cleartext of the + message itself, so please ensure that the notmuch message index is + adequately protected. DO NOT USE ``--decrypt=true`` or + ``--decrypt=nostash`` without considering the security of your + index. -``--verbose`` - Print file names being processed. Ignored when combined with - ``--quiet``. + See also ``index.decrypt`` in :any:`notmuch-config(1)`. -``--decrypt=(true|nostash|auto|false)`` - If ``true``, when encountering an encrypted message, try to - decrypt it while indexing, and stash any discovered session keys. - If ``auto``, try to use any session key already known to belong to - this message, but do not attempt to use the user's secret keys. - If decryption is successful, index the cleartext of the message. +.. option:: --full-scan - Be aware that the index is likely sufficient (and the session key - is certainly sufficient) to reconstruct the cleartext of the - message itself, so please ensure that the notmuch message index is - adequately protected. DO NOT USE ``--decrypt=true`` or - ``--decrypt=nostash`` without considering the security of your - index. + By default notmuch-new uses directory modification times (mtimes) + to optimize the scanning of directories for new mail. This option turns + that optimization off. - See also ``index.decrypt`` in **notmuch-config(1)**. +CONFIGURATION +============= -``--full-scan`` - By default notmuch-new uses directory modification times (mtimes) - to optimize the scanning of directories for new mail. This option turns - that optimization off. +Indexing is influenced by the configuration options +:nmconfig:`index.decrypt`, :nmconfig:`index.header.\` +and :nmconfig:`new.ignore`. Tagging +is controlled by :nmconfig:`new.tags` and +:nmconfig:`maildir.synchronize_flags`. See +:any:`notmuch-config(1)` for details. EXIT STATUS =========== @@ -79,15 +99,15 @@ This command supports the following special exit status code SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-reindex.rst b/doc/man1/notmuch-reindex.rst index cd7c91a0..85dad249 100644 --- a/doc/man1/notmuch-reindex.rst +++ b/doc/man1/notmuch-reindex.rst @@ -1,3 +1,5 @@ +.. _notmuch-reindex(1): + =============== notmuch-reindex =============== @@ -12,7 +14,7 @@ DESCRIPTION Re-index all messages matching the search terms. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for <*search-term*\ >. The **reindex** command searches for all messages matching the @@ -21,28 +23,31 @@ messages using the supplied options. Supported options for **reindex** include -``--decrypt=(true|nostash|auto|false)`` - If ``true``, when encountering an encrypted message, try to - decrypt it while reindexing, stashing any session keys discovered. - If ``auto``, and notmuch already knows about a session key for the - message, it will try decrypting using that session key but will - not try to access the user's secret keys. If decryption is - successful, index the cleartext itself. +.. program:: reindex + +.. option:: --decrypt=(true|nostash|auto|false) + + If ``true``, when encountering an encrypted message, try to + decrypt it while reindexing, stashing any session keys discovered. + If ``auto``, and notmuch already knows about a session key for the + message, it will try decrypting using that session key but will + not try to access the user's secret keys. If decryption is + successful, index the cleartext itself. - ``nostash`` is the same as ``true`` except that it will not stash - newly-discovered session keys in the database. + ``nostash`` is the same as ``true`` except that it will not stash + newly-discovered session keys in the database. - If ``false``, notmuch reindex will also delete any stashed session - keys for all messages matching the search terms. + If ``false``, notmuch reindex will also delete any stashed session + keys for all messages matching the search terms. - Be aware that the index is likely sufficient (and a stashed - session key is certainly sufficient) to reconstruct the cleartext - of the message itself, so please ensure that the notmuch message - index is adequately protected. DO NOT USE ``--decrypt=true`` or - ``--decrypt=nostash`` without considering the security of your - index. + Be aware that the index is likely sufficient (and a stashed + session key is certainly sufficient) to reconstruct the cleartext + of the message itself, so please ensure that the notmuch message + index is adequately protected. DO NOT USE ``--decrypt=true`` or + ``--decrypt=nostash`` without considering the security of your + index. - See also ``index.decrypt`` in **notmuch-config(1)**. + See also ``index.decrypt`` in :any:`notmuch-config(1)`. EXAMPLES ======== @@ -84,17 +89,17 @@ https://trac.xapian.org/ticket/742: SEE ALSO ======== -**notmuch(1)**, -**notmuch-compact(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-compact(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-reply.rst b/doc/man1/notmuch-reply.rst index 5c64c4a6..4f39a959 100644 --- a/doc/man1/notmuch-reply.rst +++ b/doc/man1/notmuch-reply.rst @@ -1,3 +1,5 @@ +.. _notmuch-reply(1): + ============= notmuch-reply ============= @@ -34,64 +36,75 @@ The resulting message template is output to stdout. Supported options for **reply** include -``--format=``\ (**default**\ \|\ **json**\ \|\ **sexp**\ \|\ **headers-only**) - **default** - Includes subject and quoted message body as an RFC 2822 - message. +.. program:: reply + +.. option:: --duplicate=N + + Reply to duplicate number N. The numbering starts from 1, and + matches the order used by :option:`show --duplicate` and + :option:`search --output=files `. + +.. option:: --format=(default|json|sexp|headers-only) + + default + Includes subject and quoted message body as an RFC 2822 + message. + + json + Produces JSON output containing headers for a reply message + and the contents of the original message. This output can be + used by a client to create a reply message intelligently. + + sexp + Produces S-Expression output containing headers for a reply + message and the contents of the original message. This output + can be used by a client to create a reply message + intelligently. - **json** - Produces JSON output containing headers for a reply message - and the contents of the original message. This output can be - used by a client to create a reply message intelligently. + headers-only + Only produces In-Reply-To, References, To, Cc, and Bcc + headers. - **sexp** - Produces S-Expression output containing headers for a reply - message and the contents of the original message. This output - can be used by a client to create a reply message - intelligently. +.. option:: --format-version=N - **headers-only** - Only produces In-Reply-To, References, To, Cc, and Bcc - headers. + Use the specified structured output format version. This is + intended for programs that invoke :any:`notmuch(1)` internally. If + omitted, the latest supported version will be used. -``--format-version=N`` - Use the specified structured output format version. This is - intended for programs that invoke **notmuch(1)** internally. If - omitted, the latest supported version will be used. +.. option:: --reply-to=(all|sender) -``--reply-to=``\ (**all**\ \|\ **sender**) - **all** (default) - Replies to all addresses. + all (default) + Replies to all addresses. - **sender** - Replies only to the sender. If replying to user's own message - (Reply-to: or From: header is one of the user's configured - email addresses), try To:, Cc:, and Bcc: headers in this - order, and copy values from the first that contains something - other than only the user's addresses. + sender + Replies only to the sender. If replying to user's own message + (Reply-to: or From: header is one of the user's configured + email addresses), try To:, Cc:, and Bcc: headers in this + order, and copy values from the first that contains something + other than only the user's addresses. -``--decrypt=(false|auto|true)`` +.. option:: --decrypt=(false|auto|true) - If ``true``, decrypt any MIME encrypted parts found in the - selected content (i.e., "multipart/encrypted" parts). Status - of the decryption will be reported (currently only supported - with ``--format=json`` and ``--format=sexp``), and on successful - decryption the multipart/encrypted part will be replaced by - the decrypted content. + If ``true``, decrypt any MIME encrypted parts found in the + selected content (i.e., "multipart/encrypted" parts). Status + of the decryption will be reported (currently only supported + with ``--format=json`` and ``--format=sexp``), and on successful + decryption the multipart/encrypted part will be replaced by + the decrypted content. - If ``auto``, and a session key is already known for the - message, then it will be decrypted, but notmuch will not try - to access the user's secret keys. + If ``auto``, and a session key is already known for the + message, then it will be decrypted, but notmuch will not try + to access the user's secret keys. - Use ``false`` to avoid even automatic decryption. + Use ``false`` to avoid even automatic decryption. - Non-automatic decryption expects a functioning - **gpg-agent(1)** to provide any needed credentials. Without - one, the decryption will likely fail. + Non-automatic decryption expects a functioning + :manpage:`gpg-agent(1)` to provide any needed credentials. Without + one, the decryption will likely fail. - Default: ``auto`` + Default: ``auto`` -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . Note: It is most common to use **notmuch reply** with a search string @@ -116,15 +129,15 @@ This command supports the following special exit status codes SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-restore.rst b/doc/man1/notmuch-restore.rst index c0f47f26..ac6b4245 100644 --- a/doc/man1/notmuch-restore.rst +++ b/doc/man1/notmuch-restore.rst @@ -1,3 +1,5 @@ +.. _notmuch-restore(1): + =============== notmuch-restore =============== @@ -10,90 +12,96 @@ SYNOPSIS DESCRIPTION =========== -Restores the tags from the given file (see **notmuch dump**). +Restores the tags from the given file (see :any:`notmuch-dump(1)`). The input is read from the given filename, if any, or from stdin. Supported options for **restore** include -``--accumulate`` - The union of the existing and new tags is applied, instead of - replacing each message's tags as they are read in from the dump - file. - -``--format=(sup|batch-tag|auto)`` - Notmuch restore supports two plain text dump formats, with each - line specifying a message-id and a set of tags. For details of the - actual formats, see **notmuch-dump(1)**. - - **sup** - The **sup** dump file format is specifically chosen to be - compatible with the format of files produced by sup-dump. So - if you've previously been using sup for mail, then the - **notmuch restore** command provides you a way to import all - of your tags (or labels as sup calls them). - - **batch-tag** - The **batch-tag** dump format is intended to more robust - against malformed message-ids and tags containing whitespace - or non-\ **ascii(7)** characters. See **notmuch-dump(1)** for - details on this format. - - **notmuch restore** updates the maildir flags according to tag - changes if the **maildir.synchronize\_flags** configuration - option is enabled. See **notmuch-config(1)** for details. - - **auto** - This option (the default) tries to guess the format from the - input. For correctly formed input in either supported format, - this heuristic, based the fact that batch-tag format contains - no parentheses, should be accurate. - -``--include=(config|properties|tags)`` - Control what kind of metadata is restored. - - **config** - Restore configuration data to the database. Each configuration - line starts with "#@ ", followed by a space separated - key-value pair. Both key and value are hex encoded if needed. - - **properties** - Restore per-message (key,value) metadata. Each line starts - with "#= ", followed by a message id, and a space separated - list of key=value pairs. Ids, keys and values are hex encoded - if needed. See **notmuch-properties(7)** for more details. - - **tags** - Restore per-message metadata, namely tags. See *format* above - for more details. - - The default is to restore all available types of data. The option - can be specified multiple times to select some subset. - -``--input=``\ - Read input from given file instead of stdin. +.. program:: restore + +.. option:: --accumulate + + The union of the existing and new tags is applied, instead of + replacing each message's tags as they are read in from the dump + file. + +.. option:: --format=(sup|batch-tag|auto) + + Notmuch restore supports two plain text dump formats, with each + line specifying a message-id and a set of tags. For details of the + actual formats, see :any:`notmuch-dump(1)`. + + sup + The **sup** dump file format is specifically chosen to be + compatible with the format of files produced by sup-dump. So + if you've previously been using sup for mail, then the + **notmuch restore** command provides you a way to import all + of your tags (or labels as sup calls them). + + batch-tag + The **batch-tag** dump format is intended to more robust + against malformed message-ids and tags containing whitespace + or non-\ **ascii(7)** characters. See :any:`notmuch-dump(1)` for + details on this format. + + **notmuch restore** updates the maildir flags according to tag + changes if the **maildir.synchronize\_flags** configuration + option is enabled. See :any:`notmuch-config(1)` for details. + + auto + This option (the default) tries to guess the format from the + input. For correctly formed input in either supported format, + this heuristic, based the fact that batch-tag format contains + no parentheses, should be accurate. + +.. option:: --include=(config|properties|tags) + + Control what kind of metadata is restored. + + config + Restore configuration data to the database. Each configuration + line starts with "#@ ", followed by a space separated + key-value pair. Both key and value are hex encoded if needed. + + properties + Restore per-message (key,value) metadata. Each line starts + with "#= ", followed by a message id, and a space separated + list of key=value pairs. Ids, keys and values are hex encoded + if needed. See :any:`notmuch-properties(7)` for more details. + + tags + Restore per-message metadata, namely tags. See *format* above + for more details. + + The default is to restore all available types of data. The option + can be specified multiple times to select some subset. + +.. option:: --input= + + Read input from given file instead of stdin. GZIPPED INPUT ============= \ **notmuch restore** will detect if the input is compressed in -**gzip(1)** format and automatically decompress it while reading. This -detection does not depend on file naming and in particular works for -standard input. +:manpage:`gzip(1)` format and automatically decompress it while +reading. This detection does not depend on file naming and in +particular works for standard input. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-properties(7)**, -**notmuch-reply(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-properties(7)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-search.rst b/doc/man1/notmuch-search.rst index ed9ff4e5..b87737ea 100644 --- a/doc/man1/notmuch-search.rst +++ b/doc/man1/notmuch-search.rst @@ -1,3 +1,5 @@ +.. _notmuch-search(1): + ============== notmuch-search ============== @@ -19,123 +21,133 @@ in the thread, the number of matched messages and total messages in the thread, the names of all participants in the thread, and the subject of the newest (or oldest) message. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . Supported options for **search** include -``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**) - Presents the results in either JSON, S-Expressions, newline - character separated plain-text (default), or null character - separated plain-text (compatible with **xargs(1)** -0 option where - available). - -``--format-version=N`` - Use the specified structured output format version. This is - intended for programs that invoke **notmuch(1)** internally. If - omitted, the latest supported version will be used. - -``--output=(summary|threads|messages|files|tags)`` - **summary** - Output a summary of each thread with any message matching the - search terms. The summary includes the thread ID, date, the - number of messages in the thread (both the number matched and - the total number), the authors of the thread and the - subject. In the case where a thread contains multiple files - for some messages, the total number of files is printed in - parentheses (see below for an example). - - **threads** - Output the thread IDs of all threads with any message matching - the search terms, either one per line (``--format=text``), - separated by null characters (``--format=text0``), as a JSON array - (``--format=json``), or an S-Expression list (``--format=sexp``). - - **messages** - Output the message IDs of all messages matching the search - terms, either one per line (``--format=text``), separated by null - characters (``--format=text0``), as a JSON array (``--format=json``), - or as an S-Expression list (``--format=sexp``). - - **files** - Output the filenames of all messages matching the search - terms, either one per line (``--format=text``), separated by null - characters (``--format=text0``), as a JSON array (``--format=json``), - or as an S-Expression list (``--format=sexp``). - - Note that each message may have multiple filenames associated - with it. All of them are included in the output (unless - limited with the ``--duplicate=N`` option). This may be - particularly confusing for **folder:** or **path:** searches - in a specified directory, as the messages may have duplicates - in other directories that are included in the output, although - these files alone would not match the search. - - **tags** - Output all tags that appear on any message matching the search - terms, either one per line (``--format=text``), separated by null - characters (``--format=text0``), as a JSON array (``--format=json``), - or as an S-Expression list (``--format=sexp``). - -``--sort=``\ (**newest-first**\ \|\ **oldest-first**) - This option can be used to present results in either chronological - order (**oldest-first**) or reverse chronological order - (**newest-first**). - - Note: The thread order will be distinct between these two options - (beyond being simply reversed). When sorting by **oldest-first** - the threads will be sorted by the oldest message in each thread, - but when sorting by **newest-first** the threads will be sorted by - the newest message in each thread. - - By default, results will be displayed in reverse chronological - order, (that is, the newest results will be displayed first). - -``--offset=[-]N`` - Skip displaying the first N results. With the leading '-', start - at the Nth result from the end. - -``--limit=N`` - Limit the number of displayed results to N. - -``--exclude=(true|false|all|flag)`` - A message is called "excluded" if it matches at least one tag in - search.exclude\_tags that does not appear explicitly in the search - terms. This option specifies whether to omit excluded messages in - the search process. - - **true** (default) - Prevent excluded messages from matching the search terms. - - **all** - Additionally prevent excluded messages from appearing in - displayed results, in effect behaving as though the excluded - messages do not exist. - - **false** - Allow excluded messages to match search terms and appear in - displayed results. Excluded messages are still marked in the - relevant outputs. - - **flag** - Only has an effect when ``--output=summary``. The output is - almost identical to **false**, but the "match count" is the - number of matching non-excluded messages in the thread, rather - than the number of matching messages. - -``--duplicate=N`` - For ``--output=files``, output the Nth filename associated with - each message matching the query (N is 1-based). If N is greater - than the number of files associated with the message, don't print - anything. - - For ``--output=messages``, only output message IDs of messages - matching the search terms that have at least N filenames - associated with them. - - Note that this option is orthogonal with the **folder:** search - prefix. The prefix matches messages based on filenames. This - option filters filenames of the matching messages. +.. program:: search + +.. option:: --format=(json|sexp|text|text0) + + Presents the results in either JSON, S-Expressions, newline + character separated plain-text (default), or null character + separated plain-text (compatible with :manpage:`xargs(1)` -0 + option where available). + +.. option:: --format-version=N + + Use the specified structured output format version. This is + intended for programs that invoke :any:`notmuch(1)` internally. If + omitted, the latest supported version will be used. + +.. option:: --output=(summary|threads|messages|files|tags) + + summary (default) + Output a summary of each thread with any message matching the + search terms. The summary includes the thread ID, date, the + number of messages in the thread (both the number matched and + the total number), the authors of the thread and the + subject. In the case where a thread contains multiple files + for some messages, the total number of files is printed in + parentheses (see below for an example). + + threads + Output the thread IDs of all threads with any message matching + the search terms, either one per line (``--format=text``), + separated by null characters (``--format=text0``), as a JSON array + (``--format=json``), or an S-Expression list (``--format=sexp``). + + messages + Output the message IDs of all messages matching the search + terms, either one per line (``--format=text``), separated by null + characters (``--format=text0``), as a JSON array (``--format=json``), + or as an S-Expression list (``--format=sexp``). + + files + Output the filenames of all messages matching the search + terms, either one per line (``--format=text``), separated by null + characters (``--format=text0``), as a JSON array (``--format=json``), + or as an S-Expression list (``--format=sexp``). + + Note that each message may have multiple filenames associated + with it. All of them are included in the output (unless + limited with the ``--duplicate=N`` option). This may be + particularly confusing for **folder:** or **path:** searches + in a specified directory, as the messages may have duplicates + in other directories that are included in the output, although + these files alone would not match the search. + + tags + Output all tags that appear on any message matching the search + terms, either one per line (``--format=text``), separated by null + characters (``--format=text0``), as a JSON array (``--format=json``), + or as an S-Expression list (``--format=sexp``). + +.. option:: --sort=(newest-first|oldest-first) + + This option can be used to present results in either chronological + order (**oldest-first**) or reverse chronological order + (**newest-first**). + + Note: The thread order will be distinct between these two options + (beyond being simply reversed). When sorting by **oldest-first** + the threads will be sorted by the oldest message in each thread, + but when sorting by **newest-first** the threads will be sorted by + the newest message in each thread. + + By default, results will be displayed in reverse chronological + order, (that is, the newest results will be displayed first). + +.. option:: --offset=[-]N + + Skip displaying the first N results. With the leading '-', start + at the Nth result from the end. + +.. option:: --limit=N + + Limit the number of displayed results to N. + +.. option:: --exclude=(true|false|all|flag) + + A message is called "excluded" if it matches at least one tag in + search.exclude\_tags that does not appear explicitly in the search + terms. This option specifies whether to omit excluded messages in + the search process. + + true (default) + Prevent excluded messages from matching the search terms. + + all + Additionally prevent excluded messages from appearing in + displayed results, in effect behaving as though the excluded + messages do not exist. + + false + Allow excluded messages to match search terms and appear in + displayed results. Excluded messages are still marked in the + relevant outputs. + + flag + Only has an effect when ``--output=summary``. The output is + almost identical to **false**, but the "match count" is the + number of matching non-excluded messages in the thread, rather + than the number of matching messages. + +.. option:: --duplicate=N + + For ``--output=files``, output the Nth filename associated with + each message matching the query (N is 1-based). If N is greater + than the number of files associated with the message, don't print + anything. + + For ``--output=messages``, only output message IDs of messages + matching the search terms that have at least N filenames + associated with them. + + Note that this option is orthogonal with the **folder:** search + prefix. The prefix matches messages based on filenames. This + option filters filenames of the matching messages. EXAMPLE ======= @@ -164,16 +176,16 @@ This command supports the following special exit status codes SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** -**notmuch-address(1)** +:any:`notmuch(1)`, +:any:`notmuch-address(1)` +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst index becd3e79..c13d94de 100644 --- a/doc/man1/notmuch-show.rst +++ b/doc/man1/notmuch-show.rst @@ -1,3 +1,5 @@ +.. _notmuch-show(1): + ============ notmuch-show ============ @@ -12,7 +14,7 @@ DESCRIPTION Shows all messages matching the search terms. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for . The messages will be grouped and sorted based on the threading (all @@ -23,176 +25,223 @@ post-processor (such as the emacs interface to notmuch). Supported options for **show** include -``--entire-thread=(true|false)`` - If true, **notmuch show** outputs all messages in the thread of - any message matching the search terms; if false, it outputs only - the matching messages. For ``--format=json`` and ``--format=sexp`` - this defaults to true. For other formats, this defaults to false. - -``--format=(text|json|sexp|mbox|raw)`` - **text** (default for messages) - The default plain-text format has all text-content MIME parts - decoded. Various components in the output, (**message**, - **header**, **body**, **attachment**, and MIME **part**), will - be delimited by easily-parsed markers. Each marker consists of - a Control-L character (ASCII decimal 12), the name of the - marker, and then either an opening or closing brace, ('{' or - '}'), to either open or close the component. For a multipart - MIME message, these parts will be nested. - - **json** - The output is formatted with Javascript Object Notation - (JSON). This format is more robust than the text format for - automated processing. The nested structure of multipart MIME - messages is reflected in nested JSON output. By default JSON - output includes all messages in a matching thread; that is, by - default, ``--format=json`` sets ``--entire-thread``. The - caller can disable this behaviour by setting - ``--entire-thread=false``. The JSON output is always encoded - as UTF-8 and any message content included in the output will - be charset-converted to UTF-8. - - **sexp** - The output is formatted as the Lisp s-expression (sexp) - equivalent of the JSON format above. Objects are formatted as - property lists whose keys are keywords (symbols preceded by a - colon). True is formatted as ``t`` and both false and null are - formatted as ``nil``. As for JSON, the s-expression output is - always encoded as UTF-8. - - **mbox** - All matching messages are output in the traditional, Unix mbox - format with each message being prefixed by a line beginning - with "From " and a blank line separating each message. Lines - in the message content beginning with "From " (preceded by - zero or more '>' characters) have an additional '>' character - added. This reversible escaping is termed "mboxrd" format and - described in detail here: - - http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/mail-mbox-formats.html - - **raw** (default if ``--part`` is given) - Write the raw bytes of the given MIME part of a message to - standard out. For this format, it is an error to specify a - query that matches more than one message. - - If the specified part is a leaf part, this outputs the body of - the part after performing content transfer decoding (but no - charset conversion). This is suitable for saving attachments, - for example. - - For a multipart or message part, the output includes the part - headers as well as the body (including all child parts). No - decoding is performed because multipart and message parts - cannot have non-trivial content transfer encoding. Consumers - of this may need to implement MIME decoding and similar - functions. - -``--format-version=N`` - Use the specified structured output format version. This is - intended for programs that invoke **notmuch(1)** internally. If - omitted, the latest supported version will be used. - -``--part=N`` - Output the single decoded MIME part N of a single message. The - search terms must match only a single message. Message parts are - numbered in a depth-first walk of the message MIME structure, and - are identified in the 'json', 'sexp' or 'text' output formats. - - Note that even a message with no MIME structure or a single body - part still has two MIME parts: part 0 is the whole message - (headers and body) and part 1 is just the body. - -``--verify`` - Compute and report the validity of any MIME cryptographic - signatures found in the selected content (e.g., "multipart/signed" - parts). Status of the signature will be reported (currently only - supported with ``--format=json`` and ``--format=sexp``), and the - multipart/signed part will be replaced by the signed data. - -``--decrypt=(false|auto|true|stash)`` - If ``true``, decrypt any MIME encrypted parts found in the - selected content (e.g., "multipart/encrypted" parts). Status of - the decryption will be reported (currently only supported - with ``--format=json`` and ``--format=sexp``) and on successful - decryption the multipart/encrypted part will be replaced by - the decrypted content. - - ``stash`` behaves like ``true``, but upon successful decryption it - will also stash the message's session key in the database, and - index the cleartext of the message, enabling automatic decryption - in the future. - - If ``auto``, and a session key is already known for the - message, then it will be decrypted, but notmuch will not try - to access the user's keys. - - Use ``false`` to avoid even automatic decryption. - - Non-automatic decryption (``stash`` or ``true``, in the absence of - a stashed session key) expects a functioning **gpg-agent(1)** to - provide any needed credentials. Without one, the decryption will - fail. - - Note: setting either ``true`` or ``stash`` here implies - ``--verify``. - - Here is a table that summarizes each of these policies: - - +------------------------+-------+------+------+-------+ - | | false | auto | true | stash | - +========================+=======+======+======+=======+ - | Show cleartext if | | X | X | X | - | session key is | | | | | - | already known | | | | | - +------------------------+-------+------+------+-------+ - | Use secret keys to | | | X | X | - | show cleartext | | | | | - +------------------------+-------+------+------+-------+ - | Stash any newly | | | | X | - | recovered session keys,| | | | | - | reindexing message if | | | | | - | found | | | | | - +------------------------+-------+------+------+-------+ - - Note: ``--decrypt=stash`` requires write access to the database. - Otherwise, ``notmuch show`` operates entirely in read-only mode. - - Default: ``auto`` - -``--exclude=(true|false)`` - Specify whether to omit threads only matching search.exclude\_tags - from the search results (the default) or not. In either case the - excluded message will be marked with the exclude flag (except when - output=mbox when there is nowhere to put the flag). - - If ``--entire-thread`` is specified then complete threads are returned - regardless (with the excluded flag being set when appropriate) but - threads that only match in an excluded message are not returned - when ``--exclude=true.`` - - The default is ``--exclude=true.`` - -``--body=(true|false)`` - If true (the default) **notmuch show** includes the bodies of the - messages in the output; if false, bodies are omitted. - ``--body=false`` is only implemented for the text, json and sexp - formats and it is incompatible with ``--part > 0.`` - - This is useful if the caller only needs the headers as body-less - output is much faster and substantially smaller. - -``--include-html`` - Include "text/html" parts as part of the output (currently - only supported with ``--format=text``, ``--format=json`` and - ``--format=sexp``). By default, unless ``--part=N`` is used to - select a specific part or ``--include-html`` is used to include all - "text/html" parts, no part with content type "text/html" is included - in the output. - -A common use of **notmuch show** is to display a single thread of email -messages. For this, use a search term of "thread:" as can be -seen in the first column of output from the **notmuch search** command. +.. program:: show + +.. option:: --duplicate=N + + Output duplicate number N. The numbering starts from 1, and matches + the order used by :option:`search --duplicate` and + :option:`search --output=files ` + +.. option:: --entire-thread=(true|false) + + If true, **notmuch show** outputs all messages in the thread of + any message matching the search terms; if false, it outputs only + the matching messages. For ``--format=json`` and ``--format=sexp`` + this defaults to true. For other formats, this defaults to false. + +.. option:: --format=(text|json|sexp|mbox|raw) + + text (default for messages) + The default plain-text format has all text-content MIME parts + decoded. Various components in the output, (**message**, + **header**, **body**, **attachment**, and MIME **part**), will + be delimited by easily-parsed markers. Each marker consists of + a Control-L character (ASCII decimal 12), the name of the + marker, and then either an opening or closing brace, ('{' or + '}'), to either open or close the component. For a multipart + MIME message, these parts will be nested. + + json + The output is formatted with Javascript Object Notation + (JSON). This format is more robust than the text format for + automated processing. The nested structure of multipart MIME + messages is reflected in nested JSON output. By default JSON + output includes all messages in a matching thread; that is, by + default, ``--format=json`` sets ``--entire-thread``. The + caller can disable this behaviour by setting + ``--entire-thread=false``. The JSON output is always encoded + as UTF-8 and any message content included in the output will + be charset-converted to UTF-8. + + sexp + The output is formatted as the Lisp s-expression (sexp) + equivalent of the JSON format above. Objects are formatted as + property lists whose keys are keywords (symbols preceded by a + colon). True is formatted as ``t`` and both false and null are + formatted as ``nil``. As for JSON, the s-expression output is + always encoded as UTF-8. + + mbox + All matching messages are output in the traditional, Unix mbox + format with each message being prefixed by a line beginning + with "From " and a blank line separating each message. Lines + in the message content beginning with "From " (preceded by + zero or more '>' characters) have an additional '>' character + added. This reversible escaping is termed "mboxrd" format and + described in detail here: + + http://homepage.ntlworld.com/jonathan.deboynepollard/FGA/mail-mbox-formats.html + + raw (default if ``--part`` is given) + Write the raw bytes of the given MIME part of a message to + standard out. For this format, it is an error to specify a + query that matches more than one message. + + If the specified part is a leaf part, this outputs the body of + the part after performing content transfer decoding (but no + charset conversion). This is suitable for saving attachments, + for example. + + For a multipart or message part, the output includes the part + headers as well as the body (including all child parts). No + decoding is performed because multipart and message parts + cannot have non-trivial content transfer encoding. Consumers + of this may need to implement MIME decoding and similar + functions. + +.. option:: --format-version=N + + Use the specified structured output format version. This is + intended for programs that invoke :any:`notmuch(1)` internally. If + omitted, the latest supported version will be used. + +.. option:: --part=N + + Output the single decoded MIME part N of a single message. The + search terms must match only a single message. Message parts are + numbered in a depth-first walk of the message MIME structure, and + are identified in the 'json', 'sexp' or 'text' output formats. + + Note that even a message with no MIME structure or a single body + part still has two MIME parts: part 0 is the whole message + (headers and body) and part 1 is just the body. + +.. option:: --sort=(newest-first|oldest-first) + + This option can be used to present results in either chronological + order (**oldest-first**) or reverse chronological order + (**newest-first**). + + Only threads as a whole are reordered. Ordering of messages within + each thread will not be affected by this flag, since that order is + always determined by the thread's replies. + + By default, results will be displayed in reverse chronological + order, (that is, the newest results will be displayed first). + +.. option:: --offset=[-]N + + Skip displaying the first N results. With the leading '-', start + at the Nth result from the end. + +.. option:: --limit=N + + Limit the number of displayed results to N. + +.. option:: --verify + + Compute and report the validity of any MIME cryptographic + signatures found in the selected content (e.g., "multipart/signed" + parts). Status of the signature will be reported (currently only + supported with ``--format=json`` and ``--format=sexp``), and the + multipart/signed part will be replaced by the signed data. + +.. option:: --decrypt=(false|auto|true|stash) + + If ``true``, decrypt any MIME encrypted parts found in the + selected content (e.g., "multipart/encrypted" parts). Status of + the decryption will be reported (currently only supported + with ``--format=json`` and ``--format=sexp``) and on successful + decryption the multipart/encrypted part will be replaced by + the decrypted content. + + ``stash`` behaves like ``true``, but upon successful decryption it + will also stash the message's session key in the database, and + index the cleartext of the message, enabling automatic decryption + in the future. + + If ``auto``, and a session key is already known for the + message, then it will be decrypted, but notmuch will not try + to access the user's keys. + + Use ``false`` to avoid even automatic decryption. + + Non-automatic decryption (``stash`` or ``true``, in the absence of + a stashed session key) expects a functioning :manpage:`gpg-agent(1)` to + provide any needed credentials. Without one, the decryption will + fail. + + Note: setting either ``true`` or ``stash`` here implies + ``--verify``. + + Here is a table that summarizes each of these policies: + + +------------------------+-------+------+------+-------+ + | | false | auto | true | stash | + +========================+=======+======+======+=======+ + | Show cleartext if | | X | X | X | + | session key is | | | | | + | already known | | | | | + +------------------------+-------+------+------+-------+ + | Use secret keys to | | | X | X | + | show cleartext | | | | | + +------------------------+-------+------+------+-------+ + | Stash any newly | | | | X | + | recovered session keys,| | | | | + | reindexing message if | | | | | + | found | | | | | + +------------------------+-------+------+------+-------+ + + Note: ``--decrypt=stash`` requires write access to the database. + Otherwise, ``notmuch show`` operates entirely in read-only mode. + + Default: ``auto`` + +.. option:: --exclude=(true|false) + + Specify whether to omit threads only matching search.exclude\_tags + from the search results (the default) or not. In either case the + excluded message will be marked with the exclude flag (except when + output=mbox when there is nowhere to put the flag). + + If ``--entire-thread`` is specified then complete threads are returned + regardless (with the excluded flag being set when appropriate) but + threads that only match in an excluded message are not returned + when ``--exclude=true.`` + + The default is ``--exclude=true.`` + +.. option:: --body=(true|false) + + If true (the default) **notmuch show** includes the bodies of the + messages in the output; if false, bodies are omitted. + ``--body=false`` is only implemented for the text, json and sexp + formats and it is incompatible with ``--part > 0.`` + + This is useful if the caller only needs the headers as body-less + output is much faster and substantially smaller. + +.. option:: --include-html + + Include "text/html" parts as part of the output (currently + only supported with ``--format=text``, ``--format=json`` and + ``--format=sexp``). By default, unless ``--part=N`` is used to + select a specific part or ``--include-html`` is used to include all + "text/html" parts, no part with content type "text/html" is included + in the output. + +A common use of **notmuch show** is to display a single thread of +email messages. For this, use a search term of "thread:" as +can be seen in the first column of output from the +:any:`notmuch-search(1)` command. + +CONFIGURATION +============= + +Structured output (json / sexp) is influenced by the configuration +option :nmconfig:`show.extra_headers`. See +:any:`notmuch-config(1)` for details. EXIT STATUS =========== @@ -208,15 +257,15 @@ This command supports the following special exit status codes SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-tag(1)` diff --git a/doc/man1/notmuch-tag.rst b/doc/man1/notmuch-tag.rst index c2324f5a..ae311a23 100644 --- a/doc/man1/notmuch-tag.rst +++ b/doc/man1/notmuch-tag.rst @@ -1,3 +1,5 @@ +.. _notmuch-tag(1): + =========== notmuch-tag =========== @@ -14,7 +16,7 @@ DESCRIPTION Add/remove tags for all messages matching the search terms. -See **notmuch-search-terms(7)** for details of the supported syntax for +See :any:`notmuch-search-terms(7)` for details of the supported syntax for <*search-term*\ >. Tags prefixed by '+' are added while those prefixed by '-' are removed. @@ -28,26 +30,31 @@ beginning with '+' or '-' is provided by allowing the user to specify a **notmuch tag** updates the maildir flags according to tag changes if the **maildir.synchronize\_flags** configuration option is enabled. See -**notmuch-config(1)** for details. +:any:`notmuch-config(1)` for details. Supported options for **tag** include -``--remove-all`` - Remove all tags from each message matching the search terms before - applying the tag changes appearing on the command line. This - means setting the tags of each message to the tags to be added. If - there are no tags to be added, the messages will have no tags. +.. program:: tag + +.. option:: --remove-all + + Remove all tags from each message matching the search terms before + applying the tag changes appearing on the command line. This + means setting the tags of each message to the tags to be added. If + there are no tags to be added, the messages will have no tags. + +.. option:: --batch + + Read batch tagging operations from a file (stdin by default). + This is more efficient than repeated **notmuch tag** + invocations. See `TAG FILE FORMAT <#tag_file_format>`__ below for + the input format. This option is not compatible with specifying + tagging on the command line. -``--batch`` - Read batch tagging operations from a file (stdin by default). - This is more efficient than repeated **notmuch tag** - invocations. See `TAG FILE FORMAT <#tag_file_format>`__ below for - the input format. This option is not compatible with specifying - tagging on the command line. +.. option:: --input= -``--input=``\ - Read input from given file, instead of from stdin. Implies - ``--batch``. + Read input from given file, instead of from stdin. Implies + ``--batch``. TAG FILE FORMAT =============== @@ -100,15 +107,15 @@ of the tag **space in tags** SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, diff --git a/doc/man1/notmuch.rst b/doc/man1/notmuch.rst index d2cd8da5..c488f12a 100644 --- a/doc/man1/notmuch.rst +++ b/doc/man1/notmuch.rst @@ -1,3 +1,6 @@ +.. _notmuch(1): +.. _notmuch-setup(1): + ======= notmuch ======= @@ -15,8 +18,9 @@ reading, and tagging large collections of email messages. This page describes how to get started using notmuch from the command line, and gives a brief overview of the commands available. For more -information on e.g. **notmuch show** consult the **notmuch-show(1)** man -page, also accessible via **notmuch help show** +information on e.g. **notmuch show** consult the +:any:`notmuch-show(1)` man page, also accessible via **notmuch help +show** The quickest way to get started with Notmuch is to simply invoke the ``notmuch`` command with no arguments, which will interactively guide @@ -39,23 +43,31 @@ OPTIONS Supported global options for ``notmuch`` include -``--help`` [command-name] - Print a synopsis of available commands and exit. With an optional - command name, show the man page for that subcommand. +.. program:: notmuch + +.. option:: --help [command-name] + + Print a synopsis of available commands and exit. With an optional + command name, show the man page for that subcommand. + +.. option:: --version -``--version`` - Print the installed version of notmuch, and exit. + Print the installed version of notmuch, and exit. -``--config=FILE`` - Specify the configuration file to use. This overrides any - configuration file specified by ${NOTMUCH\_CONFIG}. +.. option:: --config=FILE -``--uuid=HEX`` - Enforce that the database UUID (a unique identifier which persists - until e.g. the database is compacted) is HEX; exit with an error - if it is not. This is useful to detect rollover in modification - counts on messages. You can find this UUID using e.g. ``notmuch - count --lastmod`` + Specify the configuration file to use. This overrides any + configuration file specified by :envvar:`NOTMUCH_CONFIG`. The empty + string is a permitted and sometimes useful value of *FILE*, which + tells ``notmuch`` to use only configuration metadata from the database. + +.. option:: --uuid=HEX + + Enforce that the database UUID (a unique identifier which persists + until e.g. the database is compacted) is HEX; exit with an error + if it is not. This is useful to detect rollover in modification + counts on messages. You can find this UUID using e.g. ``notmuch + count --lastmod`` All global options except ``--config`` can also be specified after the command. For example, ``notmuch subcommand --uuid=HEX`` is equivalent @@ -73,7 +85,7 @@ use, (or to reconfigure it later). The setup command will prompt for your full name, your primary email address, any alternate email addresses you use, and the directory containing your email archives. Your answers will be written to a -configuration file in ${NOTMUCH\_CONFIG} (if set) or +configuration file in :envvar:`NOTMUCH_CONFIG` (if set) or ${HOME}/.notmuch-config . This configuration file will be created with descriptive comments, making it easy to edit by hand later to change the configuration. Or you can run **notmuch setup** again to change the @@ -88,7 +100,8 @@ will do its best to detect those and ignore them. Mail storage that uses mbox format, (where one mbox file contains many messages), will not work with notmuch. If that's how your mail is currently stored, it is recommended you first convert it to maildir -format with a utility such as mb2md before running **notmuch setup .** +format with a utility such as :manpage:`mb2md(1)` before running +**notmuch setup**. Invoking ``notmuch`` with no command argument will run **setup** if the setup command has not previously been completed. @@ -97,40 +110,44 @@ OTHER COMMANDS -------------- Several of the notmuch commands accept search terms with a common -syntax. See **notmuch-search-terms**\ (7) for more details on the +syntax. See :any:`notmuch-search-terms(7)` for more details on the supported syntax. -The **search**, **show**, **address** and **count** commands are used -to query the email database. +The :any:`notmuch-search(1)`, :any:`notmuch-show(1)`, +:any:`notmuch-address(1)` and :any:`notmuch-count(1)` commands are +used to query the email database. -The **reply** command is useful for preparing a template for an email -reply. +The :any:`notmuch-reply(1)` command is useful for preparing a template +for an email reply. -The **tag** command is the only command available for manipulating -database contents. +The :any:`notmuch-tag(1)` command is the only command available for +manipulating database contents. -The **dump** and **restore** commands can be used to create a textual -dump of email tags for backup purposes, and to restore from that dump. +The :any:`notmuch-dump(1)` and :any:`notmuch-restore(1)` commands can +be used to create a textual dump of email tags for backup purposes, +and to restore from that dump. -The **config** command can be used to get or set settings in the notmuch -configuration file. +The :any:`notmuch-config(1)` command can be used to get or set +settings in the notmuch configuration file. -CUSTOM COMMANDS ---------------- +EXTERNAL COMMANDS +----------------- If the given command is not known to notmuch, notmuch tries to execute -the external **notmuch-** in ${PATH} instead. This allows -users to have their own notmuch related tools to be run via the +the external **notmuch-** in :envvar:`PATH` instead. This +allows users to have their own notmuch related tools to be run via the notmuch command. By design, this does not allow notmuch's own commands -to be overridden using external commands. +to be overridden using external commands. The environment variable +:envvar:`NOTMUCH_CONFIG` will be set according to :option:`--config`, +if the latter is present. OPTION SYNTAX ------------- All options accepting an argument can be used with '=' or ':' as a -separator. For the cases where it's not ambiguous (in particular -excluding boolean options), a space can also be used. The following -are all equivalent: +separator. Except for boolean options (which would be ambiguous), a +space can also be used as a separator. The following are all +equivalent: :: @@ -138,44 +155,74 @@ are all equivalent: notmuch --config:alt-config config get user.name notmuch --config alt-config config get user.name +.. _duplicate-files: + +DUPLICATE MESSAGE FILES +======================= + +Notmuch considers the :mailheader:`Message-ID` to be the primary +identifier of message. Per :rfc:`5322` the :mailheader:`Message-ID` is +supposed to be globally unique, but this fails in two distinct +ways. When you receive copies of a message via a mechanism like +:mailheader:`Cc` or via a mailing list, the copies are typically +interchangeable. In the case of some broken mail sending software, the +same :mailheader:`Message-ID` is used for completely unrelated +messages. The options :option:`search --duplicate` and :option:`show +--duplicate` options provide the user with control over which message +file is displayed. Front ends will need to provide their own +interface, see e.g. the Emacs front-end :any:`emacs-show-duplicates`. + ENVIRONMENT =========== The following environment variables can be used to control the behavior of notmuch. -**NOTMUCH\_CONFIG** - Specifies the location of the notmuch configuration file. Notmuch - will use ${HOME}/.notmuch-config if this variable is not set. +.. envvar:: NOTMUCH_CONFIG + + Specifies the location of the notmuch configuration file. See + :any:`notmuch-config(1)` for details. + +.. envvar:: NOTMUCH_DATABASE + + Specifies the location of the notmuch database. See + :any:`notmuch-config(1)` for details. + +.. envvar:: NOTMUCH_PROFILE + + Selects among notmuch configurations. See :any:`notmuch-config(1)` + for details. + +.. envvar:: NOTMUCH_TALLOC_REPORT + + Location to write a talloc memory usage report. See + **talloc\_enable\_leak\_report\_full** in :manpage:`talloc(3)` for more + information. -**NOTMUCH\_TALLOC\_REPORT** - Location to write a talloc memory usage report. See - **talloc\_enable\_leak\_report\_full** in **talloc(3)** for more - information. +.. envvar:: NOTMUCH_DEBUG_QUERY -**NOTMUCH\_DEBUG\_QUERY** - If set to a non-empty value, the notmuch library will print (to - stderr) Xapian queries it constructs. + If set to a non-empty value, the notmuch library will print (to + stderr) Xapian queries it constructs. SEE ALSO ======== -**notmuch-address(1)**, -**notmuch-compact(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-properties(7)**, -**notmuch-reindex(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch-address(1)`, +:any:`notmuch-compact(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-properties(7)`, +:any:`notmuch-reindex(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` The notmuch website: **https://notmuchmail.org** @@ -187,4 +234,4 @@ list . Subscription is not required before posting, but is available from the notmuchmail.org website. Real-time interaction with the Notmuch community is available via IRC -(server: irc.freenode.net, channel: #notmuch). +(server: irc.libera.chat, channel: #notmuch). diff --git a/doc/man5/notmuch-hooks.rst b/doc/man5/notmuch-hooks.rst index de2ed0c2..d778bdb8 100644 --- a/doc/man5/notmuch-hooks.rst +++ b/doc/man5/notmuch-hooks.rst @@ -1,3 +1,5 @@ +.. _notmuch-hooks(5): + ============= notmuch-hooks ============= @@ -5,42 +7,42 @@ notmuch-hooks SYNOPSIS ======== -$DATABASEDIR/.notmuch/hooks/* +/{pre-new, post-new, post-insert} DESCRIPTION =========== Hooks are scripts (or arbitrary executables or symlinks to such) that notmuch invokes before and after certain actions. These scripts reside -in the .notmuch/hooks directory within the database directory and must -have executable permissions. +in a directory defined as described in :any:`notmuch-config(1)`. They +must have executable permissions. The currently available hooks are described below. -**pre-new** - This hook is invoked by the **new** command before scanning or - importing new messages into the database. If this hook exits with - a non-zero status, notmuch will abort further processing of the - **new** command. +pre-new + This hook is invoked by the :any:`notmuch-new(1)` command before + scanning or importing new messages into the database. If this hook + exits with a non-zero status, notmuch will abort further + processing of the :any:`notmuch-new(1)` command. Typically this hook is used for fetching or delivering new mail to be imported into the database. -**post-new** - This hook is invoked by the **new** command after new messages - have been imported into the database and initial tags have been - applied. The hook will not be run if there have been any errors - during the scan or import. +post-new + This hook is invoked by the :any:`notmuch-new(1)` command after + any new messages have been imported into the database and initial + tags have been applied. The hook will not be run if there have + been any errors during the scan or import. Typically this hook is used to perform additional query-based tagging on the imported messages. -**post-insert** - This hook is invoked by the **insert** command after the message - has been delivered, added to the database, and initial tags have - been applied. The hook will not be run if there have been any - errors during the message delivery; what is regarded as successful - delivery depends on the ``--keep`` option. +post-insert + This hook is invoked by the :any:`notmuch-insert(1)` command after + the message has been delivered, added to the database, and initial + tags have been applied. The hook will not be run if there have + been any errors during the message delivery; what is regarded as + successful delivery depends on the ``--keep`` option. Typically this hook is used to perform additional query-based tagging on the delivered messages. @@ -48,15 +50,15 @@ The currently available hooks are described below. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -**notmuch-search-terms(7)**, -**notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man7/notmuch-properties.rst b/doc/man7/notmuch-properties.rst index a7d91d67..ff79f4c2 100644 --- a/doc/man7/notmuch-properties.rst +++ b/doc/man7/notmuch-properties.rst @@ -1,3 +1,5 @@ +.. _notmuch-properties(7): + ================== notmuch-properties ================== @@ -45,7 +47,7 @@ CONVENTIONS =========== Any property with a key that starts with "index." will be removed (and -possibly re-set) upon reindexing (see **notmuch-reindex(1)**). +possibly re-set) upon reindexing (see :any:`notmuch-reindex(1)`). MESSAGE PROPERTIES ================== @@ -53,7 +55,7 @@ MESSAGE PROPERTIES The following properties are set by notmuch internally in the course of its normal activity. -**index.decryption** +index.decryption If a message contains encrypted content, and notmuch tries to decrypt that content during indexing, it will add the property ``index.decryption=success`` when the cleartext was successfully @@ -70,15 +72,14 @@ of its normal activity. If notmuch never tried to decrypt an encrypted message during indexing (which is the default, see ``index.decrypt`` in - **notmuch-config(1)**), then this property will not be set on that + :any:`notmuch-config(1)`), then this property will not be set on that message. -**session-key** - - When **notmuch-show(1)** or **nomtuch-reply** encounters a message - with an encrypted part, if notmuch finds a ``session-key`` - property associated with the message, it will try that stashed - session key for decryption. +session-key + When :any:`notmuch-show(1)` or :any:`notmuch-reply(1)` encounters + a message with an encrypted part, if notmuch finds a + ``session-key`` property associated with the message, it will try + that stashed session key for decryption. If you do not want to use any stashed session keys that might be present, you should pass those programs ``--decrypt=false``. @@ -97,7 +98,7 @@ of its normal activity. message. This includes attachments, cryptographic signatures, and other material that cannot be reconstructed from the index alone. - See ``index.decrypt`` in **notmuch-config(1)** for more + See ``index.decrypt`` in :any:`notmuch-config(1)` for more details about how to set notmuch's policy on when to store session keys. @@ -109,8 +110,7 @@ of its normal activity. example, an AES-128 key might be stashed in a notmuch property as: ``session-key=7:14B16AF65536C28AF209828DFE34C9E0``. -**index.repaired** - +index.repaired Some messages arrive in forms that are confusing to view; they can be mangled by mail transport agents, or the sending mail user agent may structure them in a way that is confusing. If notmuch @@ -136,13 +136,13 @@ of its normal activity. SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-dump(1)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reindex(1)**, -**notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-show(1)**, -***notmuch-search-terms(7)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-reindex(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search-terms(7)`, +:any:`notmuch-show(1)` diff --git a/doc/man7/notmuch-search-terms.rst b/doc/man7/notmuch-search-terms.rst index 1dd2dc58..acc1c967 100644 --- a/doc/man7/notmuch-search-terms.rst +++ b/doc/man7/notmuch-search-terms.rst @@ -1,3 +1,5 @@ +.. _notmuch-search-terms(7): + ==================== notmuch-search-terms ==================== @@ -37,10 +39,9 @@ In addition to free text, the following prefixes can be used to force terms to match against specific portions of an email, (where indicate user-supplied values). -If notmuch is built with **Xapian Field Processors** (see below) some -of the prefixes with forms can be also used to restrict the -results to those whose value matches a regular expression (see -**regex(7)**) delimited with //, for example:: +Some of the prefixes with forms can be also used to restrict +the results to those whose value matches a regular expression (see +:manpage:`regex(7)`) delimited with //, for example:: notmuch search 'from:"/bob@.*[.]example[.]com/"' @@ -72,8 +73,9 @@ mimetype: tag: or tag:// or is: or is:// For **tag:** and **is:** valid tag values include **inbox** and - **unread** by default for new messages added by **notmuch new** as - well as any other tag values added manually with **notmuch tag**. + **unread** by default for new messages added by + :any:`notmuch-new(1)` as well as any other tag values added + manually with :any:`notmuch-tag(1)`. id: or mid: or mid:// For **id:** and **mid:**, message ID values are the literal @@ -84,11 +86,10 @@ thread: The **thread:** prefix can be used with the thread ID values that are generated internally by notmuch (and do not appear in email messages). These thread ID values can be seen in the first column - of output from **notmuch search** + of output from :any:`notmuch-search(1)` thread:{} - If notmuch is built with **Xapian Field Processors** (see below), - threads may be searched for indirectly by providing an arbitrary + Threads may be searched for indirectly by providing an arbitrary notmuch query in **{}**. For example, the following returns threads containing a message from mallory and one (not necessarily the same message) with Subject containing the word "crypto". @@ -152,24 +153,30 @@ date:.. or date: lastmod:.. The **lastmod:** prefix can be used to restrict the result by the database revision number of when messages were last modified (tags - were added/removed or filenames changed). This is usually used in - conjunction with the ``--uuid`` argument to **notmuch search** to - find messages that have changed since an earlier query. + were added/removed or filenames changed). Negative revisions are + interpreted relative to the most recent database revision (see + :option:`count --lastmod`). This is usually used in conjunction + with the ``--uuid`` argument to :any:`notmuch-search(1)` to find + messages that have changed since an earlier query. query: The **query:** prefix allows queries to refer to previously saved - queries added with **notmuch-config(1)**. Named queries are only - available if notmuch is built with **Xapian Field Processors** - (see below). + queries added with :any:`notmuch-config(1)`. property:= The **property:** prefix searches for messages with a particular = property pair. Properties are used internally by notmuch (and extensions) to add metadata to messages. A given key can be present on a given message with several different values. - See **notmuch-properties(7)** for more details. + See :any:`notmuch-properties(7)` for more details. + +sexp: + The **sexp:** prefix allows subqueries in the format + documented in :any:`notmuch-sexp-queries(7)`. Note that subqueries containing + spaces must be quoted, and any embedded double quotes must be escaped + (see :any:`quoting`). -User defined prefixes are also supported, see **notmuch-config(1)** for +User defined prefixes are also supported, see :any:`notmuch-config(1)` for details. Operators @@ -257,7 +264,7 @@ Boolean Probabilistic **body:**, **to:**, **attachment:**, **mimetype:** Special - **from:**, **query:**, **subject:** + **from:**, **query:**, **subject:**, **sexp:** Terms and phrases ----------------- @@ -275,11 +282,13 @@ the same phrase. - a.list.of.words Both parenthesised lists of terms and quoted phrases are ok with -probabilistic prefixes such as **to:**, **from:**, and **subject:**. In particular +probabilistic prefixes such as **to:**, **from:**, and **subject:**. +For prefixes supporting regex search, the parenthesised list should be +quoted. In particular :: - subject:(pizza free) + subject:"(pizza free)" is equivalent to @@ -295,6 +304,8 @@ Both of these will match a subject "Free Delicious Pizza" while will not. +.. _quoting: + Quoting ------- @@ -322,6 +333,13 @@ e.g. % notmuch search 'folder:"/^.*/(Junk|Spam)$/"' % notmuch search 'thread:"{from:mallory and date:2009}" and thread:{to:mallory}' +Double quotes within query strings need to be doubled to escape them. + +:: + + % notmuch search 'tag:"""quoted tag"""' + % notmuch search 'sexp:"(or ""wizard"" ""php"")"' + DATE AND TIME SEARCH ==================== @@ -353,23 +371,21 @@ since 1970-01-01 00:00:00 UTC. For example: date:@..@ -date:..! can be used as a shorthand for date:... The -expansion takes place before interpretation, and thus, for example, -date:monday..! matches from the beginning of Monday until the end of -Monday. -With **Xapian Field Processor** support (see below), non-range -date queries such as date:yesterday will work, but otherwise -will give unexpected results; if in doubt use date:yesterday..! - -Currently, we do not support spaces in range expressions. You can +Currently, spaces in range expressions are not supported. You can replace the spaces with '\_', or (in most cases) '-', or (in some cases) leave the spaces out altogether. Examples in this man page use spaces for clarity. -Open-ended ranges are supported (since Xapian 1.2.1), i.e. it's possible -to specify date:.. or date:.. to not limit the start or -end time, respectively. Pre-1.2.1 Xapian does not report an error on -open ended ranges, but it does not work as expected either. +Open-ended ranges are supported. I.e. it's possible to specify +date:.. or date:.. to not limit the start or +end time, respectively. + +Single expression +----------------- + +date: works as a shorthand for date:... +For example, date:monday matches from the beginning of Monday until +the end of Monday. Relative date and time ---------------------- @@ -446,38 +462,20 @@ Time zones Some time zone codes, e.g. UTC, EET. -XAPIAN FIELD PROCESSORS -======================= - -Certain optional features of the notmuch query processor rely on the -presence of the Xapian field processor API. You can determine if your -notmuch was built against a sufficiently recent version of Xapian by running - -:: - - % notmuch config get built_with.field_processor - -Currently the following features require field processor support: - -- non-range date queries, e.g. "date:today" -- named queries e.g. "query:my_special_query" -- regular expression searches, e.g. "subject:/^\\[SPAM\\]/" -- thread subqueries, e.g. "thread:{from:bob}" - SEE ALSO ======== -**notmuch(1)**, -**notmuch-config(1)**, -**notmuch-count(1)**, -**notmuch-dump(1)**, -**notmuch-hooks(5)**, -**notmuch-insert(1)**, -**notmuch-new(1)**, -**notmuch-reindex(1)**, -**notmuch-properties(1)**, -***notmuch-reply(1)**, -**notmuch-restore(1)**, -**notmuch-search(1)**, -***notmuch-show(1)**, -**notmuch-tag(1)** +:any:`notmuch(1)`, +:any:`notmuch-config(1)`, +:any:`notmuch-count(1)`, +:any:`notmuch-dump(1)`, +:any:`notmuch-hooks(5)`, +:any:`notmuch-insert(1)`, +:any:`notmuch-new(1)`, +:any:`notmuch-properties(7)`, +:any:`notmuch-reindex(1)`, +:any:`notmuch-reply(1)`, +:any:`notmuch-restore(1)`, +:any:`notmuch-search(1)`, +:any:`notmuch-show(1)`, +:any:`notmuch-tag(1)` diff --git a/doc/man7/notmuch-sexp-queries.rst b/doc/man7/notmuch-sexp-queries.rst new file mode 100644 index 00000000..858ff685 --- /dev/null +++ b/doc/man7/notmuch-sexp-queries.rst @@ -0,0 +1,366 @@ +.. _notmuch-sexp-queries(7): + +==================== +notmuch-sexp-queries +==================== + +SYNOPSIS +======== + +**notmuch** *subcommand* ``--query=sexp`` [option ...] ``--`` '(and (to santa) (date december))' + +DESCRIPTION +=========== + +Notmuch supports an alternative query syntax based on `S-expressions +`_ . It can be selected +with the command line ``--query=sexp`` or with the appropriate option +to the library function :c:func:`notmuch_query_create_with_syntax`. +Support for this syntax is currently optional, you can test if your +build of notmuch supports it with + +:: + + $ notmuch config get built_with.sexp_queries + + +S-EXPRESSIONS +------------- + +An *s-expression* is either an atom, or list of whitespace delimited +s-expressions inside parentheses. Atoms are either + +*basic value* + + A basic value is an unquoted string containing no whitespace, double quotes, or + parentheses. + +*quoted string* + + Double quotes (") delimit strings possibly containing whitespace + or parentheses. These can contain double quote characters by + escaping with backslash. E.g. ``"this is a quote \""``. + +S-EXPRESSION QUERIES +-------------------- + +An s-expression query is either an atom, the empty list, or a +*compound query* consisting of a prefix atom (first element) defining +a *field*, *logical operation*, or *modifier*, and 0 or more +subqueries. + +``*`` + + "*" matches any non-empty string in the current field. + +``()`` + + The empty list matches all messages + +*term* + + Match all messages containing *term*, possibly after stemming or + phrase splitting. For discussion of stemming in notmuch see + :any:`notmuch-search-terms(7)`. Stemming only applies to unquoted + terms (basic values) in s-expression queries. For information on + phrase splitting see :any:`fields`. + +``(`` *field* |q1| |q2| ... |qn| ``)`` + + Restrict the queries |q1| to |qn| to *field*, and combine with *and* + (for most fields) or *or*. See :any:`fields` for more information. + +``(`` *operator* |q1| |q2| ... |qn| ``)`` + + Combine queries |q1| to |qn|. Currently supported operators are + ``and``, ``or``, and ``not``. ``(not`` |q1| ... |qn| ``)`` is equivalent + to ``(and (not`` |q1| ``) ... (not`` |qn| ``))``. + +``(`` *modifier* |q1| |q2| ... |qn| ``)`` + + Combine queries |q1| to |qn|, and reinterpret the result (e.g. as a regular expression). + See :any:`modifiers` for more information. + +``(macro (`` |p1| ... |pn| ``) body)`` + + Define saved query with parameter substitution. The syntax is + recognized only in saved s-expression queries (see ``squery.*`` in + :any:`notmuch-config(1)`). Parameter names in ``body`` must be + prefixed with ``,`` to be expanded (see :any:`macro_examples`). + Macros may refer to other macros, but only to their own + parameters [#macro-details]_. + +.. _fields: + +FIELDS +`````` + +*Fields* [#aka-pref]_ +correspond to attributes of mail messages. Some are inherent (and +immutable) like ``subject``, while others ``tag`` and ``property`` are +settable by the user. Each concrete field in +:any:`the table below ` +is discussed further under "Search prefixes" in +:any:`notmuch-search-terms(7)`. The row *user* refers to user defined +fields, described in :any:`notmuch-config(1)`. + +Most fields are either *phrase fields* [#aka-prob]_ (which match +sequences of words), or *term fields* [#aka-bool]_ (which match exact +strings). *Phrase splitting* breaks the term (basic value or quoted +string) into words, ignore punctuation. Phrase splitting is applied to +terms in phrase (probabilistic) fields. Both phrase splitting and +stemming apply only in phrase fields. + +Each term or phrase field has an associated combining operator +(``and`` or ``or``) used to combine the queries from each element of +the tail of the list. This is generally ``or`` for those fields where +a message has one such attribute, and ``and`` otherwise. + +Term or phrase fields can contain arbitrarily complex queries made up +from terms, operators, and modifiers, but not other fields. + +Range fields take one or two arguments specifying lower and upper +bounds. One argument is interpreted as identical upper and lower +bounds. Either upper or lower bound may be specified as ``""`` or +``*`` to specify the lowest possible lower bound or highest possible +upper bound. + +``lastmod`` ranges support negative arguments, interpreted relative to +the most recent database revision (see :option:`count --lastmod`). + +.. _field-table: + +.. table:: Fields with supported modifiers + + +------------+-----------+-----------+-----------+-----------+----------+ + | field | combine | type | expand | wildcard | regex | + +============+===========+===========+===========+===========+==========+ + | *none* | and | | no | yes | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | *user* | and | phrase | no | yes | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | attachment | and | phrase | yes | yes | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | body | and | phrase | no | no | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | date | | range | no | no | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | folder | or | phrase | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | from | and | phrase | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | id | or | term | no | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | is | and | term | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | lastmod | | range | no | no | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | mid | or | term | no | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | mimetype | or | phrase | yes | yes | no | + +------------+-----------+-----------+-----------+-----------+----------+ + | path | or | term | no | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | property | and | term | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | subject | and | phrase | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | tag | and | term | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | thread | or | term | yes | yes | yes | + +------------+-----------+-----------+-----------+-----------+----------+ + | to | and | phrase | yes | yes | no | + +------------+-----------+-----------+-----------+-----------+----------+ + +.. _modifiers: + +MODIFIERS +````````` + +*Modifiers* refer to any prefixes (first elements of compound queries) +that are neither operators nor fields. + +``(infix`` *atom* ``)`` + + Interpret *atom* as an infix notmuch query (see + :any:`notmuch-search-terms(7)`). Not supported inside fields. + +``(matching`` |q1| |q2| ... |qn| ``)`` ``(of`` |q1| |q2| ... |qn| ``)`` + + Match all messages have the same values of the current field as + those matching all of |q1| ... |qn|. Supported in most term [#not-path]_ or + phrase fields. Most commonly used in the ``thread`` field. + +``(query`` *atom* ``)`` + + Expand to the saved query named by *atom*. See + :any:`notmuch-config(1)` for more. Note that the saved query must + be in infix syntax (:any:`notmuch-search-terms(7)`). Not supported + inside fields. + +``(regex`` *atom* ``)`` ``(rx`` *atom* ``)`` + + Interpret *atom* as a POSIX.2 regular expression (see + :manpage:`regex(7)`). This applies in term fields and a subset [#not-phrase]_ of + phrase fields (see :any:`field-table`). + +``(starts-with`` *subword* ``)`` + + Matches any term starting with *subword*. This applies in either + phrase or term :any:`fields `, or outside of fields [#not-body]_. Note that + a ``starts-with`` query cannot be part of a phrase. The + atom ``*`` is a synonym for ``(starts-with "")``. + +EXAMPLES +======== + +``Wizard`` + + Match all messages containing the word "wizard", ignoring case. + +``added`` + + Match all messages containing "added", but also those containing "add", "additional", + "Additional", "adds", etc... via stemming. + +``(and Bob Marley)`` + + Match messages containing words "Bob" and "Marley", or their stems + The words need not be adjacent. + +``(not Bob Marley)`` + + Match messages containing neither "Bob" nor "Marley", nor their stems. + +``"quick fox"`` ``quick-fox`` ``quick@fox`` + + Match the *phrase* "quick" followed by "fox" in phrase fields (or + outside a field). Match the literal string in a term field. + +``(folder (of (id 1234@invalid)))`` + + Match any message in the same folder as the one with Message-Id "1234\@invalid". + +``(id 1234@invalid blah@test)`` + + Matches Message-Id "1234\@invalid" *or* Message-Id "blah\@test". + +``(and (infix "date:2009-11-18..2009-11-18") (tag unread))`` + + Match messages in the given date range with tag unread. + +``(and (date 2009-11-18 2009-11-18) (tag unread))`` + + Match messages in the given date range with tag unread. + +``(and (date 2009-11-18 *) (tag unread))`` + + Match messages from 2009-11-18 or later with tag unread. + +``(and (date * 2009-11-18) (tag unread))`` + + Match messages from 2009-11-18 or earlier with tag unread. + +``(starts-with prelim)`` + + Match any words starting with "prelim". + +``(subject quick "brown fox")`` + + Match messages whose subject contains "quick" (anywhere, stemmed) and + the phrase "brown fox". + +``(subject (starts-with prelim))`` + + Matches any word starting with "prelim", inside a message subject. + +``(subject (starts-with quick) "brown fox")`` + + Match messages whose subject contains "quick brown fox", but also + "brown fox quicksand". + +``(thread (of (id 1234@invalid)))`` + + Match any message in the same thread as the one with Message-Id "1234\@invalid". + +``(thread (matching (from bob@example.com) (to bob@example.com)))`` + + Match any (messages in) a thread containing a message from + "bob\@example.com" and a (possibly distinct) message to + "bob\@example.com". + +``(to (or bob@example.com mallory@example.org))`` ``(or (to bob@example.com) (to mallory@example.org))`` + + Match in the "To" or "Cc" headers, "bob\@example.com", + "mallory\@example.org", and also "bob\@example.com.au" since it + contains the adjacent triple "bob", "example", "com". + +``(not (to *))`` + + Match messages with an empty or invalid 'To' and 'Cc' field. + +``(List *)`` + + Match messages with a non-empty List-Id header, assuming + configuration ``index.header.List=List-Id``. + +.. _macro_examples: + +MACRO EXAMPLES +-------------- + +A macro that takes two parameters and applies different fields to them. + +:: + + $ notmuch config set squery.TagSubject '(macro (tagname subj) (and (tag ,tagname) (subject ,subj)))' + $ notmuch search --query=sexp '(TagSubject inbox maildir)' + +Nested macros are allowed. + +:: + + $ notmuch config set squery.Inner '(macro (x) (subject ,x))' + $ notmuch config set squery.Outer '(macro (x y) (and (tag ,x) (Inner ,y)))' + $ notmuch search --query=sexp '(Outer inbox maildir)' + +Parameters can be re-used to reduce boilerplate. Any field, including +user defined fields is permitted within a macro. + +:: + + $ notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))' + $ notmuch search --query=sexp '(About notmuch)' + + +NOTES +===== + +.. [#macro-details] Technically macros implement lazy evaluation and + lexical scope. There is one top level scope + containing all macro definitions, but all + parameter definitions are local to a given macro. + +.. [#aka-pref] a.k.a. prefixes + +.. [#aka-prob] a.k.a. probabilistic prefixes + +.. [#aka-bool] a.k.a. boolean prefixes + +.. [#not-phrase] Due to the implementation of phrase fields in Xapian, + regex queries could only match individual words. + +.. [#not-body] Due to the way ``body`` is implemented in notmuch, + this modifier is not supported in the ``body`` field. + +.. [#not-path] Due to the way recursive ``path`` queries are implemented + in notmuch, this modifier is not supported in the + ``path`` field. + +.. |q1| replace:: `q`\ :sub:`1` +.. |q2| replace:: `q`\ :sub:`2` +.. |qn| replace:: `q`\ :sub:`n` + +.. |p1| replace:: `p`\ :sub:`1` +.. |p2| replace:: `p`\ :sub:`2` +.. |pn| replace:: `p`\ :sub:`n` diff --git a/doc/notmuch-emacs.rst b/doc/notmuch-emacs.rst index 1655e2f0..91af6d14 100644 --- a/doc/notmuch-emacs.rst +++ b/doc/notmuch-emacs.rst @@ -1,6 +1,8 @@ -============= -notmuch-emacs -============= +.. _notmuch-emacs: + +========================== +Emacs Frontend for Notmuch +========================== About this Manual ================= @@ -12,7 +14,7 @@ manual to refer to the Emacs interface to Notmuch. When this distinction is important, we’ll refer to the Emacs interface as *notmuch-emacs*. -Notmuch-emacs is highly customizable via the the Emacs customization +Notmuch-emacs is highly customizable via the Emacs customization framework (or just by setting the appropriate variables). We try to point out relevant variables in this manual, but in order to avoid duplication of information, you can usually find the most detailed @@ -46,39 +48,95 @@ a mouse or by positioning the cursor and pressing ```` | Customize Notmuch or this page. You can change the overall appearance of the notmuch-hello screen by -customizing the variable :index:`notmuch-hello-sections`. +customizing the variables + +.. el:defcustom:: notmuch-hello-sections + + |docstring::notmuch-hello-sections| + +.. el:defcustom:: notmuch-hello-thousands-separator + + |docstring::notmuch-hello-thousands-separator| + +.. el:defcustom:: notmuch-show-logo + + |docstring::notmuch-show-logo| + +.. el:defcustom:: notmuch-column-control + + Controls the number of columns for saved searches/tags in notmuch view. + + This variable has three potential types of values: + + .. describe:: t + + Automatically calculate the number of columns possible based + on the tags to be shown and the window width. + + .. describe:: integer + + A lower bound on the number of characters that will + be used to display each column. + + .. describe:: float + + A fraction of the window width that is the lower bound on the + number of characters that should be used for each column. + + So: + + - if you would like two columns of tags, set this to 0.5. + + - if you would like a single column of tags, set this to 1.0. + + - if you would like tags to be 30 characters wide, set this to 30. + + - if you don't want to worry about all of this nonsense, leave + this set to `t`. +.. el:defcustom:: notmuch-show-empty-saved-searches + |docstring::notmuch-show-empty-saved-searches| notmuch-hello key bindings -------------------------- -```` +.. el:define-key:: + Move to the next widget (button or text entry field) -```` +.. el:define-key:: + Move to the previous widget. -```` +.. el:define-key:: + Activate the current widget. -``g`` ``=`` +.. el:define-key:: g + = + Refresh the buffer; mainly update the counts of messages for various saved searches. -``G`` +.. el:define-key:: G + Import mail, See :ref:`importing` -``m`` +.. el:define-key:: m + Compose a message -``s`` +.. el:define-key:: s + Search the notmuch database using :ref:`notmuch-search` -``v`` +.. el:define-key:: v + Print notmuch version -``q`` +.. el:define-key:: q + Quit .. _saved-searches: @@ -96,25 +154,28 @@ The saved searches default to various common searches such as ``tag:inbox`` to access the inbox and ``tag:unread`` to access all unread mail, but there are several options for customization: -:index:`notmuch-saved-searches` +.. el:defcustom:: notmuch-saved-searches + The list of saved searches, including names, queries, and additional per-query options. -:index:`notmuch-saved-searches-sort-function` +.. el:defcustom:: notmuch-saved-search-sort-function + This variable controls how saved searches should be sorted. A value of ``nil`` displays the saved searches in the order they are stored in ‘notmuch-saved-searches’. -:index:`notmuch-column-control` - Controls the number of columns for displaying saved-searches/tags - Search Box ---------- The search box lets the user enter a Notmuch query. See section “Description” in Notmuch Query Syntax, for more info on Notmuch query syntax. A history of recent searches is also displayed by default. The -latter is controlled by the variable :index:`notmuch-hello-recent-searches-max`. +latter is controlled by the variable `notmuch-hello-recent-searches-max`. + +.. el:defcustom:: notmuch-hello-recent-searches-max + + |docstring::notmuch-hello-recent-searches-max| Known Tags ---------- @@ -123,15 +184,14 @@ One special kind of saved search provided by default is for each individual tag defined in the database. This can be controlled via the following variables. -:index:`notmuch-hello-tag-list-make-query` +.. el:defcustom:: notmuch-hello-tag-list-make-query + Control how to construct a search (“virtual folder”) from a given tag. -:index:`notmuch-hello-hide-tags` - Which tags not to display at all. +.. el:defcustom:: notmuch-hello-hide-tags -:index:`notmuch-column-control` - Controls the number of columns for displaying saved-searches/tags + Which tags not to display at all. .. _notmuch-search: @@ -150,31 +210,93 @@ The main purpose of the ``notmuch-search-mode`` buffer is to act as a menu of results that the user can explore further by pressing ```` on the appropriate line. -``n,C-n,`` +.. el:define-key:: n + C-n + + Move to next line -``p,C-p,`` +.. el:define-key:: + p + C-p + + Move to previous line -```` +.. el:define-key:: + Open thread on current line in :ref:`notmuch-show` mode -``g`` ``=`` +.. el:define-key:: g + = + Refresh the buffer -``?`` +.. el:define-key:: i + + Toggle whether to show messages with excluded tags in search results. + +.. el:define-key:: ? + Display full set of key bindings The presentation of results can be controlled by the following variables. -:index:`notmuch-search-result-format` - Control how each thread of messages is presented in the - ``notmuch-show-mode`` buffer +.. el:defcustom:: notmuch-search-result-format + + |docstring::notmuch-search-result-format| + + If the car of an element in notmuch-search-result-format is a + function, insert the result of calling the function into the buffer. + + This allows a user to generate custom fields in the output of a + search result. For example, with the following settings, the first + few characters on each line of the search result are used to show + information about some significant tags associated with the thread. + + .. code:: lisp + + (defun -notmuch-result-flags (format-string result) + (let ((tags-to-letters '(("flagged" . "!") + ("unread" . "u") + ("mine" . "m") + ("sent" . "s") + ("replied" . "r"))) + (tags (plist-get result :tags))) + (format format-string + (mapconcat (lambda (t2l) + (if (member (car t2l) tags) + (cdr t2l) + " ")) + tags-to-letters "")))) + + (setq notmuch-search-result-format '((-notmuch-result-flags . "%s ") + ("date" . "%12s ") + ("count" . "%9s ") + ("authors" . "%-30s ") + ("subject" . "%s ") + ("tags" . "(%s)"))) + + See also :el:defcustom:`notmuch-tree-result-format` and + :el:defcustom:`notmuch-unthreaded-result-format`. + +.. el:defcustom:: notmuch-search-oldest-first -:index:`notmuch-search-oldest-first` Display the oldest threads at the top of the buffer +It is also possible to customize how the name of buffers containing +search results is formatted using the following variables: + +.. el:defcustom:: notmuch-search-buffer-name-format + + |docstring::notmuch-search-buffer-name-format| + +.. el:defcustom:: notmuch-saved-search-buffer-name-format + + |docstring::notmuch-saved-search-buffer-name-format| + + .. _notmuch-show: notmuch-show @@ -188,40 +310,113 @@ signatures, already-read messages), are hidden. You can make these parts visible by clicking with the mouse button or by pressing RET after positioning the cursor on a hidden part. -```` +.. el:define-key:: + Scroll the current message (if necessary), advance to the next message, or advance to the next thread (if already on the last message of a thread). -``c`` +.. el:define-key:: c + :ref:`show-copy` -``N`` +.. el:define-key:: N + Move to next message -``P`` +.. el:define-key:: P + Move to previous message (or start of current message) -``n`` +.. el:define-key:: n + Move to next matching message -``p`` +.. el:define-key:: p + Move to previous matching message -``+,-`` +.. el:define-key:: + + - + Add or remove arbitrary tags from the current message. -``?`` +.. el:define-key:: ! + + |docstring::notmuch-show-toggle-elide-non-matching| + +.. el:define-key:: ? + Display full set of key bindings -Display of messages can be controlled by the following variables +Display of messages can be controlled by the following variables; see also :ref:`show-large`. + +.. el:defcustom:: notmuch-message-headers -:index:`notmuch-message-headers` |docstring::notmuch-message-headers| -:index:`notmuch-message-headers-visible` +.. el:defcustom:: notmuch-message-headers-visible + |docstring::notmuch-message-headers-visible| +.. el:defcustom:: notmuch-show-header-line + + |docstring::notmuch-show-header-line| + +.. el:defcustom:: notmuch-multipart/alternative-discouraged + + Which mime types to hide by default for multipart messages. + + Can either be a list of mime types (as strings) or a function + mapping a plist representing the current message to such a list. + The following example function would discourage `text/html` and + `multipart/related` generally, but discourage `text/plain` should + the message be sent from `whatever@example.com`. + + .. code:: lisp + + (defun my--determine-discouraged (msg) + (let* ((headers (plist-get msg :headers)) + (from (or (plist-get headers :From) ""))) + (cond + ((string-match "whatever@example.com" from) + (list "text/plain")) + (t + (list "text/html" "multipart/related"))))) + +.. _show-large: + +Dealing with large messages and threads +--------------------------------------- + +If you are finding :ref:`notmuch-show` is annoyingly slow displaying +large messages, you can customize +:el:defcustom:`notmuch-show-max-text-part-size`. If you want to speed up the +display of large threads (with or without large messages), there are +several options. First, you can display the same query in one of the +other modes. :ref:`notmuch-unthreaded` is the most robust for +extremely large queries, but :ref:`notmuch-tree` is also be faster +than :ref:`notmuch-show` in general, since it only renders a single +message a time. If you prefer to stay with the rendered thread +("conversation") view of :ref:`notmuch-show`, you can customize the +variables :el:defcustom:`notmuch-show-depth-limit`, +:el:defcustom:`notmuch-show-height-limit` and +:el:defcustom:`notmuch-show-max-text-part-size` to limit the amount of +rendering done initially. Note that these limits are implicitly +*OR*-ed together, and combinations might have surprising effects. + +.. el:defcustom:: notmuch-show-depth-limit + + |docstring::notmuch-show-depth-limit| + +.. el:defcustom:: notmuch-show-height-limit + + |docstring::notmuch-show-height-limit| + +.. el:defcustom:: notmuch-show-max-text-part-size + + |docstring::notmuch-show-max-text-part-size| + .. _show-copy: Copy to kill-ring @@ -232,44 +427,92 @@ but notmuch also provides some shortcuts. These keys are available in :ref:`notmuch-show`, and :ref:`notmuch-tree`. A subset are available in :ref:`notmuch-search`. -``c F`` ``notmuch-show-stash-filename`` +.. el:define-key:: c F + M-x notmuch-show-stash-filename + |docstring::notmuch-show-stash-filename| -``c G`` ``notmuch-show-stash-git-send-email`` +.. el:define-key:: c G + M-x notmuch-show-stash-git-send-email + |docstring::notmuch-show-stash-git-send-email| -``c I`` ``notmuch-show-stash-message-id-stripped`` +.. el:define-key:: c I + M-x notmuch-show-stash-message-id-stripped + |docstring::notmuch-show-stash-message-id-stripped| -``c L`` ``notmuch-show-stash-mlarchive-link-and-go`` +.. el:define-key:: c L + M-x notmuch-show-stash-mlarchive-link-and-go + |docstring::notmuch-show-stash-mlarchive-link-and-go| -``c T`` ``notmuch-show-stash-tags`` +.. el:define-key:: c T + M-x notmuch-show-stash-tags + |docstring::notmuch-show-stash-tags| -``c c`` ``notmuch-show-stash-cc`` +.. el:define-key:: c c + M-x notmuch-show-stash-cc + |docstring::notmuch-show-stash-cc| -``c d`` ``notmuch-show-stash-date`` +.. el:define-key:: c d + M-x notmuch-show-stash-date + |docstring::notmuch-show-stash-date| -``c f`` ``notmuch-show-stash-from`` +.. el:define-key:: c f + M-x notmuch-show-stash-from + |docstring::notmuch-show-stash-from| -``c i`` ``notmuch-show-stash-message-id`` +.. el:define-key:: c i + M-x notmuch-show-stash-message-id + |docstring::notmuch-show-stash-message-id| -``c l`` ``notmuch-show-stash-mlarchive-link`` +.. el:define-key:: c l + M-x notmuch-show-stash-mlarchive-link + |docstring::notmuch-show-stash-mlarchive-link| -``c s`` ``notmuch-show-stash-subject`` +.. el:define-key:: c s + M-x notmuch-show-stash-subject + |docstring::notmuch-show-stash-subject| -``c t`` ``notmuch-show-stash-to`` +.. el:define-key:: c t + M-x notmuch-show-stash-to + |docstring::notmuch-show-stash-to| -``c ?`` - Show all available copying commands +.. el:define-key:: c ? + M-x notmuch-subkeymap-help + + Show all available copying commands + +.. _emacs-show-duplicates: + +Dealing with duplicates +----------------------- + +If there are multiple files with the same :mailheader:`Message-ID` +(see :any:`duplicate-files`), then :any:`notmuch-show` displays the +number of duplicates and identifies the current duplicate. In the +following example duplicate 3 of 5 is displayed. + +.. code-block:: + :emphasize-lines: 1 + + M. Mustermann (Sat, 30 Jul 2022 10:33:10 -0300) (inbox signed) 3/5 + Subject: Re: Multiple files per message in emacs + To: notmuch@notmuchmail.org + +.. el:define-key:: % + M-x notmuch-show-choose-duplicate + + |docstring::notmuch-show-choose-duplicate| .. _notmuch-tree: @@ -281,42 +524,173 @@ email archives. Each line in the buffer represents a single message giving the relative date, the author, subject, and any tags. -``c`` +.. el:define-key:: c + :ref:`show-copy` -```` +.. el:define-key:: + Displays that message. -``N`` +.. el:define-key:: N + Move to next message -``P`` +.. el:define-key:: P + Move to previous message -``n`` +.. el:define-key:: n + Move to next matching message -``p`` +.. el:define-key:: p + Move to previous matching message -``g`` ``=`` +.. el:define-key:: o + M-x notmuch-tree-toggle-order + + |docstring::notmuch-tree-toggle-order| + +.. el:define-key:: l + M-x notmuch-tree-filter + + Filter or LIMIT the current search results based on an additional query string + +.. el:define-key:: t + M-x notmuch-tree-filter-by-tag + + Filter the current search results based on an additional tag + +.. el:define-key:: i + + Toggle whether to show messages with excluded tags in search results. + +.. el:define-key:: g + = + Refresh the buffer -``?`` +.. el:define-key:: ? + Display full set of key bindings +As is the case with :ref:`notmuch-search`, the presentation of results +can be controlled by the variable ``notmuch-search-oldest-first``. + +.. el:defcustom:: notmuch-tree-result-format + + |docstring::notmuch-tree-result-format| + + The following example shows how to optionally display recipients instead + of authors for sent mail (assuming the user is named Mustermann). + + .. code:: lisp + + (defun -notmuch-authors-or-to (format-string result) + (let* ((headers (plist-get result :headers)) + (to (plist-get headers :To)) + (author (plist-get headers :From)) + (face (if (plist-get result :match) + 'notmuch-tree-match-author-face + 'notmuch-tree-no-match-author-face))) + (propertize + (format format-string + (if (string-match "Mustermann" author) + (concat "To:" (notmuch-tree-clean-address to)) + author)) + 'face face))) + + (setq notmuch-tree-result-format + '(("date" . "%12s ") + (-notmuch-authors-or-to . "%-20.20s") + ((("tree" . "%s") + ("subject" . "%s")) + . " %-54s ") + ("tags" . "(%s)"))) + + See also :el:defcustom:`notmuch-search-result-format` and + :el:defcustom:`notmuch-unthreaded-result-format`. + +.. _notmuch-tree-outline: + +notmuch-tree-outline +-------------------- + +When this mode is set, each thread and subthread in the results +list is treated as a foldable section, with its first message as +its header. + +The mode just makes available in the tree buffer all the +keybindings in info:emacs#Outline_Mode, and binds the following +additional keys: + +.. el:define-key:: + + Cycle visibility state of the current message's tree. + +.. el:define-key:: + + Cycle visibility state of all trees in the buffer. + +The behaviour of this minor mode is affected by the following +customizable variables: + +.. el:defcustom:: notmuch-tree-outline-enabled + + |docstring::notmuch-tree-outline-enabled| + +.. el:defcustom:: notmuch-tree-outline-visibility + + |docstring::notmuch-tree-outline-visibility| + +.. el:defcustom:: notmuch-tree-outline-auto-close + + |docstring::notmuch-tree-outline-auto-close| + +.. el:defcustom:: notmuch-tree-outline-open-on-next + + |docstring::notmuch-tree-outline-open-on-next| + +.. _notmuch-unthreaded: + +notmuch-unthreaded +------------------ + +``notmuch-unthreaded-mode`` is similar to :any:`notmuch-tree` in that +each line corresponds to a single message, but no thread information +is presented. + +Keybindings are the same as :any:`notmuch-tree`. + +.. el:defcustom:: notmuch-unthreaded-result-format + + |docstring::notmuch-unthreaded-result-format| + + See also :el:defcustom:`notmuch-search-result-format` and + :el:defcustom:`notmuch-tree-result-format`. + Global key bindings =================== -Several features are accessible from anywhere in notmuch through the +Several features are accessible from most places in notmuch through the following key bindings: -``j`` +.. el:define-key:: j + Jump to saved searches using :ref:`notmuch-jump`. -``k`` +.. el:define-key:: k + Tagging operations using :ref:`notmuch-tag-jump` +.. el:define-key:: C-_ + C-/ + C-x u + + Undo previous tagging operation using :any:`notmuch-tag-undo` + .. _notmuch-jump: notmuch-jump @@ -342,14 +716,32 @@ prefix (:kbd:`C-u k`), notmuch displays a menu of the reverses of the operations specified in ``notmuch-tagging-keys``; i.e. each ``+tag`` is replaced by ``-tag`` and vice versa. -:index:`notmuch-tagging-keys` +.. el:defcustom:: notmuch-tagging-keys |docstring::notmuch-tagging-keys| + +notmuch-tag-undo +---------------- + +Each notmuch buffer supporting tagging operations (i.e. buffers in +:any:`notmuch-show`, :any:`notmuch-search`, :any:`notmuch-tree`, and +:any:`notmuch-unthreaded` mode) keeps a local stack of tagging +operations. These can be undone via :any:`notmuch-tag-undo`. By default +this is bound to the usual Emacs keys for undo. + +.. el:define-key:: C-_ + C-/ + C-x u + M-x notmuch-tag-undo + + |docstring::notmuch-tag-undo| + Buffer navigation ================= -:index:`notmuch-cycle-notmuch-buffers` +.. el:define-key:: M-x notmuch-cycle-notmuch-buffers + |docstring::notmuch-cycle-notmuch-buffers| Configuration @@ -360,12 +752,33 @@ Configuration Importing Mail -------------- -:index:`notmuch-poll` +.. el:define-key:: M-x notmuch-poll + |docstring::notmuch-poll| -:index:`notmuch-poll-script` +.. el:defcustom:: notmuch-poll-script + |docstring::notmuch-poll-script| +Sending Mail +------------ + +.. el:defcustom:: mail-user-agent + + Emacs consults the variable :code:`mail-user-agent` to choose a mail + sending package for commands like :code:`report-emacs-bug` and + :code:`compose-mail`. To use ``notmuch`` for this, customize this + variable to the symbol :code:`notmuch-user-agent`. + +.. el:defcustom:: message-dont-reply-to-names + + When composing mail replies, Emacs's message mode uses the + variable :code:`message-dont-reply-to-names` to exclude + recipients matching a given collection of regular expressions + or satisfying an arbitrary predicate. Notmuch's MUA inherits + this standard mechanism and will honour your customization of + this variable. + Init File --------- @@ -377,13 +790,3 @@ suffix exist it will be read instead (just one of these, chosen in this order). Most often users create ``~/.emacs.d/notmuch-config.el`` and just work with it. If Emacs was invoked with the ``-q`` or ``--no-init-file`` options, ``notmuch-init-file`` is not read. - -.. include:: ../emacs/rstdoc.rsti - -.. include:: ../emacs/notmuch.rsti - -.. include:: ../emacs/notmuch-lib.rsti - -.. include:: ../emacs/notmuch-show.rsti - -.. include:: ../emacs/notmuch-tag.rsti diff --git a/doc/python-bindings.rst b/doc/python-bindings.rst new file mode 100644 index 00000000..e1ad26ad --- /dev/null +++ b/doc/python-bindings.rst @@ -0,0 +1,5 @@ +Python Bindings +=============== + +.. automodule:: notmuch2 + :members: diff --git a/doc/queries.rst b/doc/queries.rst new file mode 100644 index 00000000..b76e71e0 --- /dev/null +++ b/doc/queries.rst @@ -0,0 +1,9 @@ +Notmuch Queries +=============== + +.. toctree:: + :titlesonly: + + man7/notmuch-search-terms + man7/notmuch-sexp-queries + man7/notmuch-properties diff --git a/emacs/Makefile.local b/emacs/Makefile.local index 141f5868..0f1f0eb2 100644 --- a/emacs/Makefile.local +++ b/emacs/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := emacs emacs_sources := \ @@ -42,7 +42,7 @@ emacs_mua := $(dir)/notmuch-emacs-mua emacs_mua_desktop := $(dir)/notmuch-emacs-mua.desktop emacs_images := \ - $(srcdir)/$(dir)/notmuch-logo.png + $(srcdir)/$(dir)/notmuch-logo.svg emacs_bytecode = $(emacs_sources:.el=.elc) emacs_docstrings = $(emacs_sources:.el=.rsti) diff --git a/emacs/coolj.el b/emacs/coolj.el index 350d537f..79d2a1b7 100644 --- a/emacs/coolj.el +++ b/emacs/coolj.el @@ -1,6 +1,6 @@ -;;; coolj.el --- automatically wrap long lines -*- coding:utf-8 -*- +;;; coolj.el --- automatically wrap long lines -*- lexical-binding: t; coding: utf-8 -*- -;; Copyright (C) 2000, 2001, 2004, 2005, 2006, 2007, 2008, 2009 Free Software Foundation, Inc. +;; Copyright (C) 2000, 2001, 2004-2009 Free Software Foundation, Inc. ;; Authors: Kai Grossjohann ;; Alex Schroeder @@ -25,13 +25,13 @@ ;;; Commentary: -;;; This is a simple derivative of some functionality from -;;; `longlines.el'. The key difference is that this version will -;;; insert a prefix at the head of each wrapped line. The prefix is -;;; calculated from the originating long line. +;; This is a simple derivative of some functionality from +;; `longlines.el'. The key difference is that this version will +;; insert a prefix at the head of each wrapped line. The prefix is +;; calculated from the originating long line. -;;; No minor-mode is provided, the caller is expected to call -;;; `coolj-wrap-region' to wrap the region of interest. +;; No minor-mode is provided, the caller is expected to call +;; `coolj-wrap-region' to wrap the region of interest. ;;; Code: @@ -45,14 +45,12 @@ Otherwise respect `fill-column'." :group 'coolj :type 'boolean) -(defcustom coolj-line-prefix-regexp "^\\(>+ \\)*" +(defcustom coolj-line-prefix-regexp "^\\(>+ ?\\)*" "Regular expression that matches line prefixes." :group 'coolj :type 'regexp) -(defvar coolj-wrap-point nil) - -(make-variable-buffer-local 'coolj-wrap-point) +(defvar-local coolj-wrap-point nil) (defun coolj-determine-prefix () "Determine the prefix for the current line." @@ -107,12 +105,12 @@ not need to be wrapped, move point to the next line and return t." If the line should not be broken, return nil; point remains on the line." (move-to-column fill-column) - (if (and (re-search-forward "[^ ]" (line-end-position) 1) - (> (current-column) fill-column)) - ;; This line is too long. Can we break it? - (or (coolj-find-break-backward prefix) - (progn (move-to-column fill-column) - (coolj-find-break-forward))))) + (and (re-search-forward "[^ ]" (line-end-position) 1) + (> (current-column) fill-column) + ;; This line is too long. Can we break it? + (or (coolj-find-break-backward prefix) + (progn (move-to-column fill-column) + (coolj-find-break-forward))))) (defun coolj-find-break-backward (prefix) "Move point backward to the first available breakpoint and return t. @@ -135,12 +133,12 @@ If no breakpoint is found, return nil." If no break point is found, return nil." (and (search-forward " " (line-end-position) 1) (progn (skip-chars-forward " " (line-end-position)) - (null (eolp))) + (null (eolp))) (if (and fill-nobreak-predicate - (run-hook-with-args-until-success - 'fill-nobreak-predicate)) - (coolj-find-break-forward) - t))) + (run-hook-with-args-until-success + 'fill-nobreak-predicate)) + (coolj-find-break-forward) + t))) (provide 'coolj) diff --git a/emacs/make-deps.el b/emacs/make-deps.el index 5b6db698..8c9e0a27 100644 --- a/emacs/make-deps.el +++ b/emacs/make-deps.el @@ -1,4 +1,4 @@ -;; make-deps.el --- compute make dependencies for Elisp sources +;;; make-deps.el --- compute make dependencies for Elisp sources -*- lexical-binding: t -*- ;; ;; Copyright © Austin Clements ;; @@ -23,7 +23,6 @@ (defun batch-make-deps () "Invoke `make-deps' for each file on the command line." - (setq debug-on-error t) (dolist (file command-line-args-left) (let ((default-directory command-line-default-directory)) @@ -37,8 +36,8 @@ This prints make dependencies to `standard-output' based on the top-level `require' expressions in the current buffer. Paths in rules will be given relative to DIR, or `default-directory'." - - (setq dir (or dir default-directory)) + (unless dir + (setq dir default-directory)) (save-excursion (goto-char (point-min)) (condition-case nil diff --git a/emacs/notmuch-address.el b/emacs/notmuch-address.el index 64887a43..f756254c 100644 --- a/emacs/notmuch-address.el +++ b/emacs/notmuch-address.el @@ -1,4 +1,4 @@ -;;; notmuch-address.el --- address completion with notmuch +;;; notmuch-address.el --- address completion with notmuch -*- lexical-binding: t -*- ;; ;; Copyright © David Edmondson ;; @@ -25,20 +25,22 @@ (require 'notmuch-parser) (require 'notmuch-lib) (require 'notmuch-company) -;; + (declare-function company-manual-begin "company") +;;; Cache internals + (defvar notmuch-address-last-harvest 0 - "Time of last address harvest") + "Time of last address harvest.") (defvar notmuch-address-completions (make-hash-table :test 'equal) "Hash of email addresses for completion during email composition. - This variable is set by calling `notmuch-address-harvest'.") +This variable is set by calling `notmuch-address-harvest'.") (defvar notmuch-address-full-harvest-finished nil - "t indicates that full completion address harvesting has been -finished. Use notmuch-address--harvest-ready to access as that -will load a saved hash if necessary (and available).") + "Whether full completion address harvesting has finished. +Use `notmuch-address--harvest-ready' to access as that will load +a saved hash if necessary (and available).") (defun notmuch-address--harvest-ready () "Return t if there is a full address hash available. @@ -47,24 +49,33 @@ If the hash is not present it attempts to load a saved hash." (or notmuch-address-full-harvest-finished (notmuch-address--load-address-hash))) +;;; Options + (defcustom notmuch-address-command 'internal "Determines how address completion candidates are generated. -If it is a string then that string should be an external program -which must take a single argument (searched string) and output a -list of completion candidates, one per line. +If this is a string, then that string should be an external +program, which must take a single argument (searched string) +and output a list of completion candidates, one per line. + +If this is the symbol `internal', then an implementation is used +that relies on the \"notmuch address\" command, but does not use +any third-party (i.e. \"external\") programs. -Alternatively, it can be the symbol 'internal, in which case -internal completion is used; the variable -`notmuch-address-internal-completion` can be used to customize -this case. +If this is the symbol `as-is', then Notmuch does not modify the +value of `message-completion-alist'. This option has to be set to +this value before `notmuch' is loaded, otherwise the modification +to `message-completion-alist' may already have taken place. This +setting obviously does not prevent `message-completion-alist' +from being modified at all; the user or some third-party package +may still modify it. -Finally, if this variable is nil then address completion is -disabled." +Finally, if this is nil, then address completion is disabled." :type '(radio - (const :tag "Use internal address completion" internal) - (const :tag "Disable address completion" nil) - (string :tag "Use external completion command")) + (const :tag "Use internal address completion" internal) + (string :tag "Use external completion command") + (const :tag "Disable address completion" nil) + (const :tag "Use default or third-party mechanism" as-is)) :group 'notmuch-send :group 'notmuch-address :group 'notmuch-external) @@ -72,12 +83,12 @@ disabled." (defcustom notmuch-address-internal-completion '(sent nil) "Determines how internal address completion generates candidates. -This should be a list of the form '(DIRECTION FILTER), where - DIRECTION is either sent or received and specifies whether the - candidates are searched in messages sent by the user or received - by the user (note received by is much faster), and FILTER is - either nil or a filter-string, such as \"date:1y..\" to append - to the query." +This should be a list of the form (DIRECTION FILTER), where +DIRECTION is either sent or received and specifies whether the +candidates are searched in messages sent by the user or received +by the user (note received by is much faster), and FILTER is +either nil or a filter-string, such as \"date:1y..\" to append to +the query." :type '(list :tag "Use internal address completion" (radio :tag "Base completion on messages you have" @@ -101,8 +112,8 @@ This should be a list of the form '(DIRECTION FILTER), where "Filename to save the cached completion addresses. All the addresses notmuch uses for address completion will be -cached in this file. This has obvious privacy implications so you -should make sure it is not somewhere publicly readable." +cached in this file. This has obvious privacy implications so +you should make sure it is not somewhere publicly readable." :type '(choice (const :tag "Off" nil) (file :tag "Filename")) :group 'notmuch-send @@ -110,12 +121,14 @@ should make sure it is not somewhere publicly readable." :group 'notmuch-external) (defcustom notmuch-address-selection-function 'notmuch-address-selection-function - "The function to select address from given list. The function is -called with PROMPT, COLLECTION, and INITIAL-INPUT as arguments -(subset of what `completing-read' can be called with). -While executed the value of `completion-ignore-case' is t. -See documentation of function `notmuch-address-selection-function' -to know how address selection is made by default." + "The function to select address from given list. + +The function is called with PROMPT, COLLECTION, and INITIAL-INPUT +as arguments (subset of what `completing-read' can be called +with). While executed the value of `completion-ignore-case' +is t. See documentation of function +`notmuch-address-selection-function' to know how address +selection is made by default." :type 'function :group 'notmuch-send :group 'notmuch-address @@ -126,15 +139,21 @@ to know how address selection is made by default." The completed address is passed as an argument to each function. Note that this hook will be invoked for completion in headers -matching `notmuch-address-completion-headers-regexp'. -" +matching `notmuch-address-completion-headers-regexp'." :type 'hook :group 'notmuch-address :group 'notmuch-hooks) +(defcustom notmuch-address-use-company t + "If available, use company mode for address completion." + :type 'boolean + :group 'notmuch-send + :group 'notmuch-address) + +;;; Setup + (defun notmuch-address-selection-function (prompt collection initial-input) - "Call (`completing-read' - PROMPT COLLECTION nil nil INITIAL-INPUT 'notmuch-address-history)" + "Default address selection function: delegate to completing read." (completing-read prompt collection nil nil initial-input 'notmuch-address-history)) @@ -146,22 +165,14 @@ matching `notmuch-address-completion-headers-regexp'. (defun notmuch-address-message-insinuate () (message "calling notmuch-address-message-insinuate is no longer needed")) -(defcustom notmuch-address-use-company t - "If available, use company mode for address completion" - :type 'boolean - :group 'notmuch-send - :group 'notmuch-address) - (defun notmuch-address-setup () - (let* ((setup-company (and notmuch-address-use-company - (require 'company nil t))) - (pair (cons notmuch-address-completion-headers-regexp - #'notmuch-address-expand-name))) - (when setup-company - (notmuch-company-setup)) - (unless (member pair message-completion-alist) - (setq message-completion-alist - (push pair message-completion-alist))))) + (unless (eq notmuch-address-command 'as-is) + (when (and notmuch-address-use-company + (require 'company nil t)) + (notmuch-company-setup)) + (cl-pushnew (cons notmuch-address-completion-headers-regexp + #'notmuch-address-expand-name) + message-completion-alist :test #'equal))) (defun notmuch-address-toggle-internal-completion () "Toggle use of internal completion for current buffer. @@ -171,38 +182,41 @@ toggles the setting in this buffer." (interactive) (if (local-variable-p 'notmuch-address-command) (kill-local-variable 'notmuch-address-command) - (notmuch-setq-local notmuch-address-command 'internal)) - (if (boundp 'company-idle-delay) - (if (local-variable-p 'company-idle-delay) - (kill-local-variable 'company-idle-delay) - (notmuch-setq-local company-idle-delay nil)))) + (setq-local notmuch-address-command 'internal)) + (when (boundp 'company-idle-delay) + (if (local-variable-p 'company-idle-delay) + (kill-local-variable 'company-idle-delay) + (setq-local company-idle-delay nil)))) + +;;; Completion (defun notmuch-address-matching (substring) "Returns a list of completion candidates matching SUBSTRING. The candidates are taken from `notmuch-address-completions'." (let ((candidates) (re (regexp-quote substring))) - (maphash (lambda (key val) + (maphash (lambda (key _val) (when (string-match re key) (push key candidates))) notmuch-address-completions) candidates)) (defun notmuch-address-options (original) - "Returns a list of completion candidates. Uses either -elisp-based implementation or older implementation requiring -external commands." + "Return a list of completion candidates. +Use either elisp-based implementation or older implementation +requiring external commands." (cond ((eq notmuch-address-command 'internal) (unless (notmuch-address--harvest-ready) ;; First, run quick synchronous harvest based on what the user - ;; entered so far + ;; entered so far. (notmuch-address-harvest original t)) (prog1 (notmuch-address-matching original) - ;; Then start the (potentially long-running) full asynchronous harvest if necessary + ;; Then start the (potentially long-running) full asynchronous + ;; harvest if necessary. (notmuch-address-harvest-trigger))) (t - (process-lines notmuch-address-command original)))) + (notmuch--process-lines notmuch-address-command original)))) (defun notmuch-address-expand-name () (cond @@ -229,49 +243,24 @@ external commands." (t (funcall notmuch-address-selection-function (format "Address (%s matches): " num-options) - ;; We put the first match as the initial - ;; input; we put all the matches as - ;; possible completions, moving the - ;; first match to the end of the list - ;; makes cursor up/down in the list work - ;; better. - (append (cdr options) (list (car options))) - (car options)))))) + options + orig))))) (if chosen (progn (push chosen notmuch-address-history) (delete-region beg end) (insert chosen) - (run-hook-with-args 'notmuch-address-post-completion-functions chosen)) + (run-hook-with-args 'notmuch-address-post-completion-functions + chosen)) (message "No matches.") (ding)))) (t nil))) -;; Copied from `w3m-which-command'. -(defun notmuch-address-locate-command (command) - "Return non-nil if `command' is an executable either on -`exec-path' or an absolute pathname." - (when (stringp command) - (if (and (file-name-absolute-p command) - (file-executable-p command)) - command - (setq command (file-name-nondirectory command)) - (catch 'found-command - (let (bin) - (dolist (dir exec-path) - (setq bin (expand-file-name command dir)) - (when (or (and (file-executable-p bin) - (not (file-directory-p bin))) - (and (file-executable-p (setq bin (concat bin ".exe"))) - (not (file-directory-p bin)))) - (throw 'found-command bin)))))))) +;;; Harvest (defun notmuch-address-harvest-addr (result) - (let ((name-addr (plist-get result :name-addr))) - (puthash name-addr t notmuch-address-completions))) - -(defun notmuch-address-harvest-handle-result (obj) - (notmuch-address-harvest-addr obj)) + (puthash (plist-get result :name-addr) + t notmuch-address-completions)) (defun notmuch-address-harvest-filter (proc string) (when (buffer-live-p (process-buffer proc)) @@ -280,18 +269,18 @@ external commands." (goto-char (point-max)) (insert string)) (notmuch-sexp-parse-partial-list - 'notmuch-address-harvest-handle-result (process-buffer proc))))) + 'notmuch-address-harvest-addr (process-buffer proc))))) (defvar notmuch-address-harvest-procs '(nil . nil) "The currently running harvests. -The car is a partial harvest, and the cdr is a full harvest") +The car is a partial harvest, and the cdr is a full harvest.") (defun notmuch-address-harvest (&optional addr-prefix synchronous callback) "Collect addresses completion candidates. It queries the notmuch database for messages sent/received (as -configured with `notmuch-address-command`) by the user, collects +configured with `notmuch-address-command') by the user, collects destination/source addresses from those messages and stores them in `notmuch-address-completions'. @@ -301,29 +290,30 @@ matching ADDR-PREFIX*' are queried. Address harvesting may take some time so the address collection runs asynchronously unless SYNCHRONOUS is t. In case of asynchronous execution, CALLBACK is called when harvesting finishes." - (let* ((sent (eq (car notmuch-address-internal-completion) 'sent)) (config-query (cadr notmuch-address-internal-completion)) - (prefix-query (when addr-prefix - (format "%s:%s*" (if sent "to" "from") addr-prefix))) + (prefix-query (and addr-prefix + (format "%s:%s*" + (if sent "to" "from") + addr-prefix))) (from-or-to-me-query (mapconcat (lambda (x) (concat (if sent "from:" "to:") x)) (notmuch-user-emails) " or ")) (query (if (or prefix-query config-query) (concat (format "(%s)" from-or-to-me-query) - (when prefix-query - (format " and (%s)" prefix-query)) - (when config-query - (format " and (%s)" config-query))) + (and prefix-query + (format " and (%s)" prefix-query)) + (and config-query + (format " and (%s)" config-query))) from-or-to-me-query)) - (args `("address" "--format=sexp" "--format-version=4" + (args `("address" "--format=sexp" "--format-version=5" ,(if sent "--output=recipients" "--output=sender") "--deduplicate=address" ,query))) (if synchronous (mapc #'notmuch-address-harvest-addr - (apply 'notmuch-call-notmuch-sexp args)) + (apply 'notmuch-call-notmuch-sexp args)) ;; Asynchronous (let* ((current-proc (if addr-prefix (car notmuch-address-harvest-procs) @@ -334,7 +324,6 @@ execution, CALLBACK is called when harvesting finishes." ;; Kill any existing process (when current-proc (kill-buffer (process-buffer current-proc))) ; this also kills the process - (setq current-proc (apply 'notmuch-start-notmuch proc-name proc-buf callback ; process sentinel @@ -351,25 +340,25 @@ execution, CALLBACK is called when harvesting finishes." "Version format of the save hash.") (defun notmuch-address--get-address-hash () - "Returns the saved address hash as a plist. + "Return the saved address hash as a plist. Returns nil if the save file does not exist, or it does not seem to be a saved address hash." - (when notmuch-address-save-filename - (condition-case nil - (with-temp-buffer - (insert-file-contents notmuch-address-save-filename) - (let ((name (read (current-buffer))) - (plist (read (current-buffer)))) - ;; We do two simple sanity checks on the loaded file. We just - ;; check a version is specified, not that it is the current - ;; version, as we are allowed to over-write and a save-file with - ;; an older version. - (when (and (string= name "notmuch-address-hash") - (plist-get plist :version)) - plist))) - ;; The error case catches any of the reads failing. - (error nil)))) + (and notmuch-address-save-filename + (condition-case nil + (with-temp-buffer + (insert-file-contents notmuch-address-save-filename) + (let ((name (read (current-buffer))) + (plist (read (current-buffer)))) + ;; We do two simple sanity checks on the loaded file. + ;; We just check a version is specified, not that + ;; it is the current version, as we are allowed to + ;; over-write and a save-file with an older version. + (and (string= name "notmuch-address-hash") + (plist-get plist :version) + plist))) + ;; The error case catches any of the reads failing. + (error nil)))) (defun notmuch-address--load-address-hash () "Read the saved address hash and set the corresponding variables." @@ -382,22 +371,23 @@ to be a saved address hash." notmuch-address-internal-completion) (equal (plist-get load-plist :version) notmuch-address--save-hash-version)) - (setq notmuch-address-last-harvest (plist-get load-plist :last-harvest) - notmuch-address-completions (plist-get load-plist :completions) - notmuch-address-full-harvest-finished t) + (setq notmuch-address-last-harvest (plist-get load-plist :last-harvest)) + (setq notmuch-address-completions (plist-get load-plist :completions)) + (setq notmuch-address-full-harvest-finished t) ;; Return t to say load was successful. t))) (defun notmuch-address--save-address-hash () (when notmuch-address-save-filename (if (or (not (file-exists-p notmuch-address-save-filename)) - ;; The file exists, check it is a file we saved + ;; The file exists, check it is a file we saved. (notmuch-address--get-address-hash)) (with-temp-file notmuch-address-save-filename - (let ((save-plist (list :version notmuch-address--save-hash-version - :completion-settings notmuch-address-internal-completion - :last-harvest notmuch-address-last-harvest - :completions notmuch-address-completions))) + (let ((save-plist + (list :version notmuch-address--save-hash-version + :completion-settings notmuch-address-internal-completion + :last-harvest notmuch-address-last-harvest + :completions notmuch-address-completions))) (print "notmuch-address-hash" (current-buffer)) (print save-plist (current-buffer)))) (message "\ @@ -409,18 +399,18 @@ appear to be an address savefile. Not overwriting." (let ((now (float-time))) (when (> (- now notmuch-address-last-harvest) 86400) (setq notmuch-address-last-harvest now) - (notmuch-address-harvest nil nil - (lambda (proc event) - ;; If harvest fails, we want to try - ;; again when the trigger is next - ;; called - (if (string= event "finished\n") - (progn - (notmuch-address--save-address-hash) - (setq notmuch-address-full-harvest-finished t)) - (setq notmuch-address-last-harvest 0))))))) - -;; + (notmuch-address-harvest + nil nil + (lambda (_proc event) + ;; If harvest fails, we want to try + ;; again when the trigger is next called. + (if (string= event "finished\n") + (progn + (notmuch-address--save-address-hash) + (setq notmuch-address-full-harvest-finished t)) + (setq notmuch-address-last-harvest 0))))))) + +;;; Standalone completion (defun notmuch-address-from-minibuffer (prompt) (if (not notmuch-address-command) @@ -439,7 +429,7 @@ appear to be an address savefile. Not overwriting." (let ((minibuffer-local-map rmap)) (read-string prompt))))) -;; +;;; _ (provide 'notmuch-address) diff --git a/emacs/notmuch-company.el b/emacs/notmuch-company.el index 3e12e7a9..7e05dc8f 100644 --- a/emacs/notmuch-company.el +++ b/emacs/notmuch-company.el @@ -1,37 +1,41 @@ ;;; notmuch-company.el --- Mail address completion for notmuch via company-mode -*- lexical-binding: t -*- - -;; Authors: Trevor Jim -;; Michal Sojka ;; -;; Keywords: mail, completion - -;; This program is free software; you can redistribute it and/or modify -;; it under the terms of the GNU General Public License as published by +;; Copyright © Trevor Jim +;; Copyright © Michal Sojka +;; +;; This file is part of Notmuch. +;; +;; Notmuch is free software: you can redistribute it and/or modify it +;; under the terms of the GNU General Public License as published by ;; the Free Software Foundation, either version 3 of the License, or ;; (at your option) any later version. - -;; This program is distributed in the hope that it will be useful, -;; but WITHOUT ANY WARRANTY; without even the implied warranty of -;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -;; GNU General Public License for more details. - +;; +;; Notmuch is distributed in the hope that it will be useful, but +;; WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +;; General Public License for more details. +;; ;; You should have received a copy of the GNU General Public License -;; along with this program. If not, see . +;; along with Notmuch. If not, see . +;; +;; Authors: Trevor Jim +;; Michal Sojka +;; Keywords: mail, completion ;;; Commentary: -;; To enable this, install company mode (https://company-mode.github.io/) +;; Mail address completion for notmuch via company-mode. To enable +;; this, install company mode from . ;; ;; NB company-minimum-prefix-length defaults to 3 so you don't get -;; completion unless you type 3 characters +;; completion unless you type 3 characters. ;;; Code: -(eval-when-compile (require 'cl)) (require 'notmuch-lib) -(defvar notmuch-company-last-prefix nil) -(make-variable-buffer-local 'notmuch-company-last-prefix) +(defvar-local notmuch-company-last-prefix nil) + (declare-function company-begin-backend "company") (declare-function company-grab "company") (declare-function company-mode "company") @@ -49,14 +53,13 @@ ;;;###autoload (defun notmuch-company-setup () (company-mode) - (make-local-variable 'company-backends) - (setq company-backends '(notmuch-company)) + (setq-local company-backends '(notmuch-company)) ;; Disable automatic company completion unless an internal ;; completion method is configured. Company completion (using ;; internal completion) can still be accessed via standard company ;; functions, e.g., company-complete. (unless (eq notmuch-address-command 'internal) - (notmuch-setq-local company-idle-delay nil))) + (setq-local company-idle-delay nil))) ;;;###autoload (defun notmuch-company (command &optional arg &rest _ignore) @@ -65,12 +68,15 @@ (require 'company) (let ((case-fold-search t) (completion-ignore-case t)) - (case command + (cl-case command (interactive (company-begin-backend 'notmuch-company)) - (prefix (and (derived-mode-p 'message-mode) - (looking-back (concat notmuch-address-completion-headers-regexp ".*") - (line-beginning-position)) - (setq notmuch-company-last-prefix (company-grab "[:,][ \t]*\\(.*\\)" 1 (point-at-bol))))) + (prefix (and (or (derived-mode-p 'message-mode) + (derived-mode-p 'org-msg-edit-mode)) + (looking-back + (concat notmuch-address-completion-headers-regexp ".*") + (line-beginning-position)) + (setq notmuch-company-last-prefix + (company-grab "[:,][ \t]*\\(.*\\)" 1 (point-at-bol))))) (candidates (cond ((notmuch-address--harvest-ready) ;; Update harvested addressed from time to time @@ -79,20 +85,22 @@ (t (cons :async (lambda (callback) - ;; First run quick asynchronous harvest based on what the user entered so far + ;; First run quick asynchronous harvest + ;; based on what the user entered so far (notmuch-address-harvest arg nil (lambda (_proc _event) (funcall callback (notmuch-address-matching arg)) - ;; Then start the (potentially long-running) full asynchronous harvest if necessary + ;; Then start the (potentially long-running) + ;; full asynchronous harvest if necessary (notmuch-address-harvest-trigger)))))))) (match (if (string-match notmuch-company-last-prefix arg) (match-end 0) 0)) - (post-completion (run-hook-with-args 'notmuch-address-post-completion-functions arg)) + (post-completion + (run-hook-with-args 'notmuch-address-post-completion-functions arg)) (no-cache t)))) - (provide 'notmuch-company) ;;; notmuch-company.el ends here diff --git a/emacs/notmuch-compat.el b/emacs/notmuch-compat.el index 2cedd39d..179bf59c 100644 --- a/emacs/notmuch-compat.el +++ b/emacs/notmuch-compat.el @@ -1,14 +1,31 @@ -;; Compatibility functions for earlier versions of emacs - +;;; notmuch-compat.el --- compatibility functions for earlier versions of emacs -*- lexical-binding: t -*- +;; ;; The functions in this file are copied from more modern versions of ;; emacs and are Copyright (C) 1985-1986, 1992, 1994-1995, 1999-2017 ;; Free Software Foundation, Inc. - -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; emacs master has a bugfix for folding long headers when sending -;; messages. Include the fix for earlier versions of emacs. To avoid -;; interfering with gnus we only run the hook when called from -;; notmuch-message-mode. +;; +;; This file is part of Notmuch. +;; +;; Notmuch is free software: you can redistribute it and/or modify it +;; under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. +;; +;; Notmuch is distributed in the hope that it will be useful, but +;; WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +;; General Public License for more details. +;; +;; You should have received a copy of the GNU General Public License +;; along with Notmuch. If not, see . + +;;; Code: + +;; Before Emacs 26.1 lines that are longer than 998 octets were not. +;; folded. Commit 77bbca8c82f6e553c42abbfafca28f55fc995d00 fixed +;; that. Until we drop support for Emacs 25 we have to backport that +;; fix. To avoid interfering with Gnus we only run the hook when +;; called from notmuch-message-mode. (declare-function mail-header-fold-field "mail-parse" nil) @@ -24,70 +41,18 @@ (unless (fboundp 'message--fold-long-headers) (add-hook 'message-header-hook 'notmuch-message--fold-long-headers)) -(if (fboundp 'setq-local) - (defalias 'notmuch-setq-local 'setq-local) - (defmacro notmuch-setq-local (var val) - "Set variable VAR to value VAL in current buffer. - -Backport of setq-local for emacs without setq-local (pre 24.3)." - `(set (make-local-variable ',var) ,val))) - -(if (fboundp 'read-char-choice) - (defalias 'notmuch-read-char-choice 'read-char-choice) - (defun notmuch-read-char-choice (prompt chars &optional inhibit-keyboard-quit) - "Read and return one of CHARS, prompting for PROMPT. -Any input that is not one of CHARS is ignored. - -If optional argument INHIBIT-KEYBOARD-QUIT is non-nil, ignore -keyboard-quit events while waiting for a valid input. - -This is an exact copy of this function from emacs 24 for use on -emacs 23, except with the one emacs 24 only function it calls -inlined." - (unless (consp chars) - (error "Called `read-char-choice' without valid char choices")) - (let (char done show-help (helpbuf " *Char Help*")) - (let ((cursor-in-echo-area t) - (executing-kbd-macro executing-kbd-macro) - (esc-flag nil)) - (save-window-excursion ; in case we call help-form-show - (while (not done) - (unless (get-text-property 0 'face prompt) - (setq prompt (propertize prompt 'face 'minibuffer-prompt))) - (setq char (let ((inhibit-quit inhibit-keyboard-quit)) - (read-key prompt))) - (and show-help (buffer-live-p (get-buffer helpbuf)) - (kill-buffer helpbuf)) - (cond - ((not (numberp char))) - ;; If caller has set help-form, that's enough. - ;; They don't explicitly have to add help-char to chars. - ((and help-form - (eq char help-char) - (setq show-help t) - ;; This is an inlined copy of help-form-show as that - ;; was introduced in emacs 24 too. - (let ((msg (eval help-form))) - (if (stringp msg) - (with-output-to-temp-buffer " *Char Help*" - (princ msg)))))) - ((memq char chars) - (setq done t)) - ((and executing-kbd-macro (= char -1)) - ;; read-event returns -1 if we are in a kbd macro and - ;; there are no more events in the macro. Attempt to - ;; get an event interactively. - (setq executing-kbd-macro nil)) - ((not inhibit-keyboard-quit) - (cond - ((and (null esc-flag) (eq char ?\e)) - (setq esc-flag t)) - ((memq char '(?\C-g ?\e)) - (keyboard-quit)))))))) - ;; Display the question with the answer. But without cursor-in-echo-area. - (message "%s%s" prompt (char-to-string char)) - char))) - -;; End of compatibility functions +;; `dlet' isn't available until Emacs 28.1. Below is a copy, with the +;; addition of `with-no-warnings'. +(defmacro notmuch-dlet (binders &rest body) + "Like `let*' but using dynamic scoping." + (declare (indent 1) (debug let)) + `(let (_) + (with-no-warnings ; Quiet "lacks a prefix" warning. + ,@(mapcar (lambda (binder) + `(defvar ,(if (consp binder) (car binder) binder))) + binders)) + (let* ,binders ,@body))) (provide 'notmuch-compat) + +;;; notmuch-compat.el ends here diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el index 4216f583..a1cf3ddd 100644 --- a/emacs/notmuch-crypto.el +++ b/emacs/notmuch-crypto.el @@ -1,4 +1,4 @@ -;;; notmuch-crypto.el --- functions for handling display of cryptographic metadata. +;;; notmuch-crypto.el --- functions for handling display of cryptographic metadata -*- lexical-binding: t -*- ;; ;; Copyright © Jameson Rollins ;; @@ -24,8 +24,12 @@ (require 'epg) (require 'notmuch-lib) +(declare-function notmuch-show-get-message-id "notmuch-show" (&optional bare)) + +;;; Options + (defcustom notmuch-crypto-process-mime t - "Should cryptographic MIME parts be processed? + "Whether to process cryptographic MIME parts. If this variable is non-nil signatures in multipart/signed messages will be verified and multipart/encrypted parts will be @@ -43,6 +47,18 @@ mode." :package-version '(notmuch . "0.25") :group 'notmuch-crypto) +(defcustom notmuch-crypto-get-keys-asynchronously t + "Whether to retrieve openpgp keys asynchronously." + :type 'boolean + :group 'notmuch-crypto) + +(defcustom notmuch-crypto-gpg-program epg-gpg-program + "The gpg executable." + :type 'string + :group 'notmuch-crypto) + +;;; Faces + (defface notmuch-crypto-part-header '((((class color) (background dark)) @@ -84,41 +100,43 @@ mode." :group 'notmuch-crypto :group 'notmuch-faces) +;;; Functions + (define-button-type 'notmuch-crypto-status-button-type - 'action (lambda (button) (message (button-get button 'help-echo))) + 'action (lambda (button) (message "%s" (button-get button 'help-echo))) 'follow-link t 'help-echo "Set notmuch-crypto-process-mime to process cryptographic mime parts." :supertype 'notmuch-button-type) (defun notmuch-crypto-insert-sigstatus-button (sigstatus from) + "Insert a button describing the signature status SIGSTATUS sent by user FROM." (let* ((status (plist-get sigstatus :status)) - (help-msg nil) (show-button t) - (label nil) (face 'notmuch-crypto-signature-unknown) - (button-action (lambda (button) (message (button-get button 'help-echo))))) + (button-action (lambda (button) (message (button-get button 'help-echo)))) + (keyid (concat "0x" (plist-get sigstatus :keyid))) + label help-msg) (cond ((string= status "good") - (let ((fingerprint (concat "0x" (plist-get sigstatus :fingerprint)))) - ;; if userid present, userid has full or greater validity - (if (plist-member sigstatus :userid) - (let ((userid (plist-get sigstatus :userid))) - (setq label (concat "Good signature by: " userid)) - (setq face 'notmuch-crypto-signature-good)) - (progn - (setq label (concat "Good signature by key: " fingerprint)) - (setq face 'notmuch-crypto-signature-good-key))) + (let ((fingerprint (concat "0x" (plist-get sigstatus :fingerprint))) + (email-or-userid (or (plist-get sigstatus :email) + (plist-get sigstatus :userid)))) + ;; If email or userid are present, they have full or greater validity. + (setq label (concat "Good signature by key: " fingerprint)) + (setq face 'notmuch-crypto-signature-good-key) + (when email-or-userid + (setq label (concat "Good signature by: " email-or-userid)) + (setq face 'notmuch-crypto-signature-good)) (setq button-action 'notmuch-crypto-sigstatus-good-callback) (setq help-msg (concat "Click to list key ID 0x" fingerprint ".")))) ((string= status "error") - (let ((keyid (concat "0x" (plist-get sigstatus :keyid)))) - (setq label (concat "Unknown key ID " keyid " or unsupported algorithm")) - (setq button-action 'notmuch-crypto-sigstatus-error-callback) - (setq help-msg (concat "Click to retrieve key ID " keyid " from keyserver and redisplay.")))) + (setq label (concat "Unknown key ID " keyid " or unsupported algorithm")) + (setq button-action 'notmuch-crypto-sigstatus-error-callback) + (setq help-msg (concat "Click to retrieve key ID " keyid + " from key server."))) ((string= status "bad") - (let ((keyid (concat "0x" (plist-get sigstatus :keyid)))) - (setq label (concat "Bad signature (claimed key ID " keyid ")")) - (setq face 'notmuch-crypto-signature-bad))) + (setq label (concat "Bad signature (claimed key ID " keyid ")")) + (setq face 'notmuch-crypto-signature-bad)) (status (setq label (concat "Unknown signature status: " status))) (t @@ -135,55 +153,119 @@ mode." :notmuch-from from) (insert "\n")))) -(declare-function notmuch-show-refresh-view "notmuch-show" (&optional reset-state)) - (defun notmuch-crypto-sigstatus-good-callback (button) - (let* ((sigstatus (button-get button :notmuch-sigstatus)) + (let* ((id (notmuch-show-get-message-id)) + (sigstatus (button-get button :notmuch-sigstatus)) (fingerprint (concat "0x" (plist-get sigstatus :fingerprint))) (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")) - (window (display-buffer buffer t nil))) + (window (display-buffer buffer))) (with-selected-window window (with-current-buffer buffer (goto-char (point-max)) - (call-process epg-gpg-program nil t t "--batch" "--no-tty" "--list-keys" fingerprint)) + (insert (format "-- Key %s in message %s:\n" + fingerprint id)) + (notmuch--call-process notmuch-crypto-gpg-program nil t t + "--batch" "--no-tty" "--list-keys" fingerprint)) (recenter -1)))) +(declare-function notmuch-show-refresh-view "notmuch-show" (&optional reset-state)) +(declare-function notmuch-show-get-message-id "notmuch-show" (&optional bare)) + +(defun notmuch-crypto--async-key-sentinel (process _event) + "When the user asks for a GPG key to be retrieved +asynchronously, handle completion of that task. + +If the retrieval is successful, the thread where the retrieval +was initiated is still displayed and the cursor has not moved, +redisplay the thread." + (let ((status (process-status process)) + (exit-status (process-exit-status process)) + (keyid (process-get process :gpg-key-id))) + (when (memq status '(exit signal)) + (message "Getting the GPG key %s asynchronously...%s." + keyid + (if (= exit-status 0) + "completed" + "failed")) + ;; If the original buffer is still alive and point didn't move + ;; (i.e. the user didn't move on or away), refresh the buffer to + ;; show the updated signature status. + (let ((show-buffer (process-get process :notmuch-show-buffer)) + (show-point (process-get process :notmuch-show-point))) + (when (and (bufferp show-buffer) + (buffer-live-p show-buffer) + (= show-point + (with-current-buffer show-buffer + (point)))) + (with-current-buffer show-buffer + (notmuch-show-refresh-view))))))) + +(defun notmuch-crypto--set-button-label (button label) + "Set the text displayed in BUTTON to LABEL." + (save-excursion + (let ((inhibit-read-only t)) + ;; This knows rather too much about how we typically format + ;; buttons. + (goto-char (button-start button)) + (forward-char 2) + (delete-region (point) (- (button-end button) 2)) + (insert label)))) + (defun notmuch-crypto-sigstatus-error-callback (button) + "When signature validation has failed, try to retrieve the +corresponding key when the status button is pressed." (let* ((sigstatus (button-get button :notmuch-sigstatus)) (keyid (concat "0x" (plist-get sigstatus :keyid))) - (buffer (get-buffer-create "*notmuch-crypto-gpg-out*")) - (window (display-buffer buffer t nil))) - (with-selected-window window - (with-current-buffer buffer - (goto-char (point-max)) - (call-process epg-gpg-program nil t t "--batch" "--no-tty" "--recv-keys" keyid) - (insert "\n") - (call-process epg-gpg-program nil t t "--batch" "--no-tty" "--list-keys" keyid)) - (recenter -1)) - (notmuch-show-refresh-view))) + (buffer (get-buffer-create "*notmuch-crypto-gpg-out*"))) + (if notmuch-crypto-get-keys-asynchronously + (progn + (notmuch-crypto--set-button-label + button (format "Retrieving key %s asynchronously..." keyid)) + (with-current-buffer buffer + (goto-char (point-max)) + (insert (format "--- Retrieving key %s:\n" keyid))) + (let ((p (notmuch--make-process + :name "notmuch GPG key retrieval" + :connection-type 'pipe + :buffer buffer + :stderr buffer + :command (list notmuch-crypto-gpg-program "--recv-keys" keyid) + :sentinel #'notmuch-crypto--async-key-sentinel))) + (process-put p :gpg-key-id keyid) + (process-put p :notmuch-show-buffer (current-buffer)) + (process-put p :notmuch-show-point (point)) + (message "Getting the GPG key %s asynchronously..." keyid))) + (let ((window (display-buffer buffer))) + (with-selected-window window + (with-current-buffer buffer + (goto-char (point-max)) + (insert (format "--- Retrieving key %s:\n" keyid)) + (notmuch--call-process notmuch-crypto-gpg-program nil t t "--recv-keys" keyid) + (insert "\n") + (notmuch--call-process notmuch-crypto-gpg-program nil t t "--list-keys" keyid)) + (recenter -1)) + (notmuch-show-refresh-view))))) (defun notmuch-crypto-insert-encstatus-button (encstatus) - (let* ((status (plist-get encstatus :status)) - (help-msg nil) - (label "Decryption not attempted") - (face 'notmuch-crypto-decryption)) - (cond - ((string= status "good") - (setq label "Decryption successful")) - ((string= status "bad") - (setq label "Decryption error")) - (t - (setq label (concat "Unknown encryption status" - (if status (concat ": " status)))))) - (insert-button - (concat "[ " label " ]") - :type 'notmuch-crypto-status-button-type - 'help-echo help-msg - 'face face - 'mouse-face face) - (insert "\n"))) + "Insert a button describing the encryption status ENCSTATUS." + (insert-button + (concat "[ " + (let ((status (plist-get encstatus :status))) + (cond + ((string= status "good") + "Decryption successful") + ((string= status "bad") + "Decryption error") + (t + (concat "Unknown encryption status" + (and status (concat ": " status)))))) + " ]") + :type 'notmuch-crypto-status-button-type + 'face 'notmuch-crypto-decryption + 'mouse-face 'notmuch-crypto-decryption) + (insert "\n")) -;; +;;; _ (provide 'notmuch-crypto) diff --git a/emacs/notmuch-draft.el b/emacs/notmuch-draft.el index e22e0d16..fcc45503 100644 --- a/emacs/notmuch-draft.el +++ b/emacs/notmuch-draft.el @@ -1,4 +1,4 @@ -;;; notmuch-draft.el --- functions for postponing and editing drafts +;;; notmuch-draft.el --- functions for postponing and editing drafts -*- lexical-binding: t -*- ;; ;; Copyright © Mark Walters ;; Copyright © David Bremner @@ -25,18 +25,24 @@ ;;; Code: +(require 'cl-lib) +(require 'pcase) +(require 'subr-x) + (require 'notmuch-maildir-fcc) (require 'notmuch-tag) (declare-function notmuch-show-get-message-id "notmuch-show" (&optional bare)) (declare-function notmuch-message-mode "notmuch-mua") +;;; Options + (defgroup notmuch-draft nil "Saving and editing drafts in Notmuch." :group 'notmuch) (defcustom notmuch-draft-tags '("+draft") - "List of tags changes to apply to a draft message when it is saved in the database. + "List of tag changes to apply when saving a draft message in the database. Tags starting with \"+\" (or not starting with either \"+\" or \"-\") in the list will be added, and tags starting with \"-\" @@ -75,9 +81,11 @@ postponing and resuming a message." :group 'notmuch-send) (defcustom notmuch-draft-save-plaintext 'ask - "Should notmuch save/postpone in plaintext messages that seem - like they are intended to be sent encrypted -(i.e with an mml encryption tag in it)." + "Whether to allow saving plaintext when it seems encryption is intended. +When a message contains mml tags, then that suggest it is +intended to be encrypted. If the user requests that such a +message is saved locally, then this option controls whether +that is allowed. Beside a boolean, this can also be `ask'." :type '(radio (const :tag "Never" nil) (const :tag "Ask every time" ask) @@ -85,13 +93,14 @@ postponing and resuming a message." :group 'notmuch-draft :group 'notmuch-crypto) +;;; Internal + (defvar notmuch-draft-encryption-tag-regex "<#\\(part encrypt\\|secure.*mode=.*encrypt>\\)" - "Regular expression matching mml tags indicating encryption of part or message") + "Regular expression matching mml tags indicating encryption of part or message.") -(defvar notmuch-draft-id nil - "Message-id of the most recent saved draft of this message") -(make-variable-buffer-local 'notmuch-draft-id) +(defvar-local notmuch-draft-id nil + "Message-id of the most recent saved draft of this message.") (defun notmuch-draft--mark-deleted () "Tag the last saved draft deleted. @@ -101,7 +110,7 @@ Used when a new version is saved, or the message is sent." (notmuch-tag notmuch-draft-id '("+deleted")))) (defun notmuch-draft-quote-some-mml () - "Quote the mml tags in `notmuch-draft-quoted-tags`." + "Quote the mml tags in `notmuch-draft-quoted-tags'." (save-excursion ;; First we deal with any secure tag separately. (message-goto-body) @@ -122,7 +131,7 @@ Used when a new version is saved, or the message is sent." (insert "!")))))) (defun notmuch-draft-unquote-some-mml () - "Unquote the mml tags in `notmuch-draft-quoted-tags`." + "Unquote the mml tags in `notmuch-draft-quoted-tags'." (save-excursion (when notmuch-draft-quoted-tags (let ((re (concat "<#!+/?\\(" @@ -136,43 +145,47 @@ Used when a new version is saved, or the message is sent." (let (secure-tag) (save-restriction (message-narrow-to-headers) - (setq secure-tag (message-fetch-field "X-Notmuch-Emacs-Secure" 't)) + (setq secure-tag (message-fetch-field "X-Notmuch-Emacs-Secure" t)) (message-remove-header "X-Notmuch-Emacs-Secure")) (message-goto-body) (when secure-tag (insert secure-tag "\n"))))) (defun notmuch-draft--has-encryption-tag () - "Returns t if there is an mml secure tag." + "Return non-nil if there is an mml secure tag." (save-excursion (message-goto-body) - (re-search-forward notmuch-draft-encryption-tag-regex nil 't))) + (re-search-forward notmuch-draft-encryption-tag-regex nil t))) (defun notmuch-draft--query-encryption () - "Checks if we should save a message that should be encrypted. + "Return non-nil if we should save a message that should be encrypted. `notmuch-draft-save-plaintext' controls the behaviour." - (case notmuch-draft-save-plaintext - ((ask) - (unless (yes-or-no-p "(Customize `notmuch-draft-save-plaintext' to avoid this warning) + (cl-case notmuch-draft-save-plaintext + ((ask) + (unless (yes-or-no-p + "(Customize `notmuch-draft-save-plaintext' to avoid this warning) This message contains mml tags that suggest it is intended to be encrypted. Really save and index an unencrypted copy? ") - (error "Save aborted"))) - ((nil) - (error "Refusing to save draft with encryption tags (see `notmuch-draft-save-plaintext')")) - ((t) - (ignore)))) + (error "Save aborted"))) + ((nil) + (error "Refusing to save draft with encryption tags (see `%s')" + 'notmuch-draft-save-plaintext)) + ((t) + (ignore)))) (defun notmuch-draft--make-message-id () ;; message-make-message-id gives the id inside a "<" ">" pair, ;; but notmuch doesn't want that form, so remove them. (concat "draft-" (substring (message-make-message-id) 1 -1))) +;;; Commands + (defun notmuch-draft-save () "Save the current draft message in the notmuch database. This saves the current message in the database with tags -`notmuch-draft-tags` (in addition to any default tags +`notmuch-draft-tags' (in addition to any default tags applied to newly inserted messages)." (interactive) (when (notmuch-draft--has-encryption-tag) @@ -183,7 +196,7 @@ applied to newly inserted messages)." ;; so that it is easier to search for the message, and the ;; latter so we have a way of accessing the saved message (for ;; example to delete it at a later time). We check that the - ;; user has these in `message-deletable-headers` (the default) + ;; user has these in `message-deletable-headers' (the default) ;; as otherwise they are doing something strange and we ;; shouldn't interfere. Note, since we are doing this in a new ;; buffer we don't change the version in the compose buffer. @@ -192,19 +205,21 @@ applied to newly inserted messages)." (message-remove-header "Message-ID") (message-add-header (concat "Message-ID: <" id ">"))) (t - (message "You have customized emacs so Message-ID is not a deletable header, so not changing it") + (message "You have customized emacs so Message-ID is not a %s" + "deletable header, so not changing it") (setq id nil))) (cond ((member 'Date message-deletable-headers) (message-remove-header "Date") (message-add-header (concat "Date: " (message-make-date)))) (t - (message "You have customized emacs so Date is not a deletable header, so not changing it"))) + (message "You have customized emacs so Date is not a deletable %s" + "header, so not changing it"))) (message-add-header "X-Notmuch-Emacs-Draft: True") (notmuch-draft-quote-some-mml) (notmuch-maildir-setup-message-for-saving) (notmuch-maildir-notmuch-insert-current-buffer - notmuch-draft-folder 't notmuch-draft-tags)) + notmuch-draft-folder t notmuch-draft-tags)) ;; We are now back in the original compose buffer. Note the ;; function notmuch-call-notmuch-process (called by ;; notmuch-maildir-notmuch-insert-current-buffer) signals an error @@ -223,16 +238,18 @@ applied to newly inserted messages)." (defun notmuch-draft-resume (id) "Resume editing of message with id ID." - (let* ((tags (process-lines notmuch-command "search" "--output=tags" + ;; Used by command `notmuch-show-resume-message'. + (let* ((tags (notmuch--process-lines notmuch-command "search" "--output=tags" "--exclude=false" id)) (draft (equal tags (notmuch-update-tags tags notmuch-draft-tags)))) (when (or draft (yes-or-no-p "Message does not appear to be a draft: edit as new? ")) - (switch-to-buffer (get-buffer-create (concat "*notmuch-draft-" id "*"))) + (pop-to-buffer-same-window + (get-buffer-create (concat "*notmuch-draft-" id "*"))) (setq buffer-read-only nil) (erase-buffer) (let ((coding-system-for-read 'no-conversion)) - (call-process notmuch-command nil t nil "show" "--format=raw" id)) + (notmuch--call-process notmuch-command nil t nil "show" "--format=raw" id)) (mime-to-mml) (goto-char (point-min)) (when (re-search-forward "^$" nil t) @@ -259,12 +276,12 @@ applied to newly inserted messages)." ;; If the resumed message was a draft then set the draft ;; message-id so that we can delete the current saved draft if the ;; message is resaved or sent. - (setq notmuch-draft-id (when draft id))))) + (setq notmuch-draft-id (and draft id))))) +;;; _ (add-hook 'message-send-hook 'notmuch-draft--mark-deleted) - (provide 'notmuch-draft) ;;; notmuch-draft.el ends here diff --git a/emacs/notmuch-emacs-mua b/emacs/notmuch-emacs-mua index a5214977..254e6407 100755 --- a/emacs/notmuch-emacs-mua +++ b/emacs/notmuch-emacs-mua @@ -41,6 +41,9 @@ CREATE_FRAME= ELISP= MAILTO= HELLO= +TO_SEP= +CC_SEP= +BCC_SEP= # Short options compatible with mutt(1). while getopts :s:c:b:i:h opt; do @@ -86,13 +89,16 @@ while getopts :s:c:b:i:h opt; do ELISP="${ELISP} (message-goto-subject) (insert \"${OPTARG}\")" ;; --to) - ELISP="${ELISP} (message-goto-to) (insert \"${OPTARG}, \")" + ELISP="${ELISP} (message-goto-to) (insert \"${TO_SEP}${OPTARG}\")" + TO_SEP=", " ;; --cc|c) - ELISP="${ELISP} (message-goto-cc) (insert \"${OPTARG}, \")" + ELISP="${ELISP} (message-goto-cc) (insert \"${CC_SEP}${OPTARG}\")" + CC_SEP=", " ;; --bcc|b) - ELISP="${ELISP} (message-goto-bcc) (insert \"${OPTARG}, \")" + ELISP="${ELISP} (message-goto-bcc) (insert \"${BCC_SEP}${OPTARG}\")" + BCC_SEP=", " ;; --body|i) ELISP="${ELISP} (message-goto-body) (insert-file \"${OPTARG}\")" diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el index aff8beb5..b6d1e2ae 100644 --- a/emacs/notmuch-hello.el +++ b/emacs/notmuch-hello.el @@ -1,4 +1,4 @@ -;;; notmuch-hello.el --- welcome to notmuch, a frontend +;;; notmuch-hello.el --- welcome to notmuch, a frontend -*- lexical-binding: t -*- ;; ;; Copyright © David Edmondson ;; @@ -21,17 +21,26 @@ ;;; Code: -(eval-when-compile (require 'cl)) (require 'widget) (require 'wid-edit) ; For `widget-forward'. (require 'notmuch-lib) (require 'notmuch-mua) -(declare-function notmuch-search "notmuch" (&optional query oldest-first target-thread target-line continuation)) -(declare-function notmuch-poll "notmuch" ()) +(declare-function notmuch-search "notmuch" + (&optional query oldest-first target-thread target-line + no-display)) +(declare-function notmuch-poll "notmuch-lib" ()) (declare-function notmuch-tree "notmuch-tree" - (&optional query query-context target buffer-name open-target)) + (&optional query query-context target buffer-name + open-target unthreaded parent-buffer + oldest-first hide-excluded)) +(declare-function notmuch-unthreaded "notmuch-tree" + (&optional query query-context target buffer-name + open-target oldest-first hide-excluded)) + + +;;; Options (defun notmuch-saved-search-get (saved-search field) "Get FIELD from SAVED-SEARCH. @@ -44,17 +53,19 @@ lists (NAME QUERY COUNT-QUERY)." ((keywordp (car saved-search)) (plist-get saved-search field)) ;; It is not a plist so it is an old-style entry. - ((consp (cdr saved-search)) ;; It is a list (NAME QUERY COUNT-QUERY) - (case field - (:name (first saved-search)) - (:query (second saved-search)) - (:count-query (third saved-search)) - (t nil))) - (t ;; It is a cons-cell (NAME . QUERY) - (case field - (:name (car saved-search)) - (:query (cdr saved-search)) - (t nil))))) + ((consp (cdr saved-search)) + (pcase-let ((`(,name ,query ,count-query) saved-search)) + (cl-case field + (:name name) + (:query query) + (:count-query count-query) + (t nil)))) + (t + (pcase-let ((`(,name . ,query) saved-search)) + (cl-case field + (:name name) + (:query query) + (t nil)))))) (defun notmuch-hello-saved-search-to-plist (saved-search) "Return a copy of SAVED-SEARCH in plist form. @@ -63,7 +74,7 @@ If saved search is a plist then just return a copy. In other cases, for backwards compatibility, convert to plist form and return that." (if (keywordp (car saved-search)) - (copy-seq saved-search) + (copy-sequence saved-search) (let ((fields (list :name :query :count-query)) plist-search) (dolist (field fields plist-search) @@ -85,21 +96,32 @@ searches so they still work in customize." :tag "Saved Search" :args '((list :inline t :format "%v" - (group :format "%v" :inline t (const :format " Name: " :name) (string :format "%v")) - (group :format "%v" :inline t (const :format " Query: " :query) (string :format "%v"))) + (group :format "%v" :inline t + (const :format " Name: " :name) + (string :format "%v")) + (group :format "%v" :inline t + (const :format " Query: " :query) + (string :format "%v"))) (checklist :inline t :format "%v" - (group :format "%v" :inline t (const :format "Shortcut key: " :key) (key-sequence :format "%v")) - (group :format "%v" :inline t (const :format "Count-Query: " :count-query) (string :format "%v")) - (group :format "%v" :inline t (const :format "" :sort-order) + (group :format "%v" :inline t + (const :format "Shortcut key: " :key) + (key-sequence :format "%v")) + (group :format "%v" :inline t + (const :format "Count-Query: " :count-query) + (string :format "%v")) + (group :format "%v" :inline t + (const :format "" :sort-order) (choice :tag " Sort Order" (const :tag "Default" nil) (const :tag "Oldest-first" oldest-first) (const :tag "Newest-first" newest-first))) - (group :format "%v" :inline t (const :format "" :search-type) + (group :format "%v" :inline t + (const :format "" :search-type) (choice :tag " Search Type" (const :tag "Search mode" nil) - (const :tag "Tree mode" tree)))))) + (const :tag "Tree mode" tree) + (const :tag "Unthreaded mode" unthreaded)))))) (defcustom notmuch-saved-searches `((:name "inbox" :query "tag:inbox" :key ,(kbd "i")) @@ -120,19 +142,24 @@ a plist. Supported properties are shown. If not present then the :query property is used. :sort-order Specify the sort order to be used for the search. - Possible values are 'oldest-first 'newest-first or - nil. Nil means use the default sort order. - :search-type Specify whether to run the search in search-mode - or tree mode. Set to 'tree to specify tree - mode, set to nil (or anything except tree) to - specify search mode. + Possible values are `oldest-first', `newest-first' + or nil. Nil means use the default sort order. + :excluded Whether to show mail with excluded tags in the + search. Possible values are `hide', `show', + or nil. Nil means use the default value of + `notmuch-search-hide-excluded'. + :search-type Specify whether to run the search in search-mode, + tree mode or unthreaded mode. Set to `tree' to + specify tree mode, \\='unthreaded to specify + unthreaded mode, and set to nil (or anything + except tree and unthreaded) to specify search + mode. Other accepted forms are a cons cell of the form (NAME . QUERY) or a list of the form (NAME QUERY COUNT-QUERY)." -;; The saved-search format is also used by the all-tags notmuch-hello -;; section. This section generates its own saved-search list in one of -;; the latter two forms. - + ;; The saved-search format is also used by the all-tags notmuch-hello + ;; section. This section generates its own saved-search list in one of + ;; the latter two forms. :get 'notmuch-hello--saved-searches-to-plist :type '(repeat notmuch-saved-search-plist) :tag "List of Saved Searches" @@ -176,6 +203,8 @@ fields of the search." (defvar notmuch-hello-indent 4 "How much to indent non-headers.") +(defimage notmuch-hello-logo ((:type svg :file "notmuch-logo.svg"))) + (defcustom notmuch-show-logo t "Should the notmuch logo be shown?" :type 'boolean @@ -265,7 +294,7 @@ International Bureau of Weights and Measures." :group 'notmuch-hello :group 'notmuch-hooks) -(defvar notmuch-hello-url "https://notmuchmail.org" +(defconst notmuch-hello-url "https://notmuchmail.org" "The `notmuch' web site.") (defvar notmuch-hello-custom-section-options @@ -351,51 +380,69 @@ supported for \"Customized queries section\" items." :group 'notmuch-hello :type 'boolean) +;;; Internal variables + (defvar notmuch-hello-hidden-sections nil - "List of sections titles whose contents are hidden") + "List of sections titles whose contents are hidden.") (defvar notmuch-hello-first-run t - "True if `notmuch-hello' is run for the first time, set to nil -afterwards.") - -(defun notmuch-hello-nice-number (n) - (let (result) - (while (> n 0) - (push (% n 1000) result) - (setq n (/ n 1000))) - (setq result (or result '(0))) - (apply #'concat - (number-to-string (car result)) - (mapcar (lambda (elem) - (format "%s%03d" notmuch-hello-thousands-separator elem)) - (cdr result))))) - -(defun notmuch-hello-trim (search) - "Trim whitespace." - (if (string-match "^[[:space:]]*\\(.*[^[:space:]]\\)[[:space:]]*$" search) - (match-string 1 search) - search)) - -(defun notmuch-hello-search (&optional search) - (unless (null search) - (setq search (notmuch-hello-trim search)) - (let ((history-delete-duplicates t)) - (add-to-history 'notmuch-search-history search))) - (notmuch-search search notmuch-search-oldest-first)) - -(defun notmuch-hello-add-saved-search (widget) - (interactive) - (let ((search (widget-value - (symbol-value - (widget-get widget :notmuch-saved-search-widget)))) + "True if `notmuch-hello' is run for the first time, set to nil afterwards.") + +;;; Widgets for inserters + +(define-widget 'notmuch-search-item 'item + "A recent search." + :format "%v\n" + :value-create 'notmuch-search-item-value-create) + +(defun notmuch-search-item-value-create (widget) + (let ((value (widget-get widget :value))) + (widget-insert (make-string notmuch-hello-indent ?\s)) + (widget-create 'editable-field + :size (widget-get widget :size) + :parent widget + :action #'notmuch-hello-search + value) + (widget-insert " ") + (widget-create 'push-button + :parent widget + :notify #'notmuch-hello-add-saved-search + "save") + (widget-insert " ") + (widget-create 'push-button + :parent widget + :notify #'notmuch-hello-delete-search-from-history + "del"))) + +(defun notmuch-search-item-field-width () + (max 8 ; Don't let the search boxes be less than 8 characters wide. + (- (window-width) + notmuch-hello-indent ; space at bol + notmuch-hello-indent ; space at eol + 1 ; for the space before the [save] button + 6 ; for the [save] button + 1 ; for the space before the [del] button + 5))) ; for the [del] button + +;;; Widget actions + +(defun notmuch-hello-search (widget &rest _event) + (let ((search (widget-value widget))) + (when search + (setq search (string-trim search)) + (let ((history-delete-duplicates t)) + (add-to-history 'notmuch-search-history search))) + (notmuch-search search notmuch-search-oldest-first))) + +(defun notmuch-hello-add-saved-search (widget &rest _event) + (let ((search (widget-value (widget-get widget :parent))) (name (completing-read "Name for saved search: " notmuch-saved-searches))) ;; If an existing saved search with this name exists, remove it. (setq notmuch-saved-searches - (loop for elem in notmuch-saved-searches - if (not (equal name - (notmuch-saved-search-get elem :name))) - collect elem)) + (cl-loop for elem in notmuch-saved-searches + unless (equal name (notmuch-saved-search-get elem :name)) + collect elem)) ;; Add the new one. (customize-save-variable 'notmuch-saved-searches (add-to-list 'notmuch-saved-searches @@ -403,50 +450,61 @@ afterwards.") (message "Saved '%s' as '%s'." search name) (notmuch-hello-update))) -(defun notmuch-hello-delete-search-from-history (widget) - (interactive) - (let ((search (widget-value - (symbol-value - (widget-get widget :notmuch-saved-search-widget))))) - (setq notmuch-search-history (delete search - notmuch-search-history)) +(defun notmuch-hello-delete-search-from-history (widget &rest _event) + (when (y-or-n-p "Are you sure you want to delete this search? ") + (let ((search (widget-value (widget-get widget :parent)))) + (setq notmuch-search-history + (delete search notmuch-search-history))) (notmuch-hello-update))) +;;; Button utilities + +;; `notmuch-hello-query-counts', `notmuch-hello-nice-number' and +;; `notmuch-hello-insert-buttons' are used outside this section. +;; All other functions that are defined in this section are only +;; used by these two functions. + (defun notmuch-hello-longest-label (searches-alist) - (or (loop for elem in searches-alist - maximize (length (notmuch-saved-search-get elem :name))) + (or (cl-loop for elem in searches-alist + maximize (length (notmuch-saved-search-get elem :name))) 0)) (defun notmuch-hello-reflect-generate-row (ncols nrows row list) (let ((len (length list))) - (loop for col from 0 to (- ncols 1) - collect (let ((offset (+ (* nrows col) row))) - (if (< offset len) - (nth offset list) - ;; Don't forget to insert an empty slot in the - ;; output matrix if there is no corresponding - ;; value in the input matrix. - nil))))) + (cl-loop for col from 0 to (- ncols 1) + collect (let ((offset (+ (* nrows col) row))) + (if (< offset len) + (nth offset list) + ;; Don't forget to insert an empty slot in the + ;; output matrix if there is no corresponding + ;; value in the input matrix. + nil))))) (defun notmuch-hello-reflect (list ncols) "Reflect a `ncols' wide matrix represented by `list' along the diagonal." ;; Not very lispy... (let ((nrows (ceiling (length list) ncols))) - (loop for row from 0 to (- nrows 1) - append (notmuch-hello-reflect-generate-row ncols nrows row list)))) - -(defun notmuch-hello-widget-search (widget &rest ignore) - (if (widget-get widget :notmuch-search-type) - (notmuch-tree (widget-get widget - :notmuch-search-terms)) - (notmuch-search (widget-get widget - :notmuch-search-terms) - (widget-get widget - :notmuch-search-oldest-first)))) + (cl-loop for row from 0 to (- nrows 1) + append (notmuch-hello-reflect-generate-row ncols nrows row list)))) + +(defun notmuch-hello-widget-search (widget &rest _ignore) + (let ((search-terms (widget-get widget :notmuch-search-terms)) + (oldest-first (widget-get widget :notmuch-search-oldest-first)) + (exclude (widget-get widget :notmuch-search-hide-excluded))) + (cl-case (widget-get widget :notmuch-search-type) + (tree + (let ((n (notmuch-search-format-buffer-name (widget-value widget) "tree" t))) + (notmuch-tree search-terms nil nil n nil nil nil oldest-first exclude))) + (unthreaded + (let ((n (notmuch-search-format-buffer-name (widget-value widget) + "unthreaded" t))) + (notmuch-unthreaded search-terms nil nil n nil oldest-first exclude))) + (t + (notmuch-search search-terms oldest-first exclude))))) (defun notmuch-saved-search-count (search) - (car (process-lines notmuch-command "count" search))) + (car (notmuch--process-lines notmuch-command "count" search))) (defun notmuch-hello-tags-per-line (widest) "Determine how many tags to show per line and how wide they @@ -459,19 +517,17 @@ should be. Returns a cons cell `(tags-per-line width)'." ;; Count is 9 wide (8 digits plus space), 1 for the space ;; after the name. (+ 9 1 (max notmuch-column-control widest))))) - ((floatp notmuch-column-control) (let* ((available-width (- (window-width) notmuch-hello-indent)) - (proposed-width (max (* available-width notmuch-column-control) widest))) + (proposed-width (max (* available-width notmuch-column-control) + widest))) (floor available-width proposed-width))) - (t (max 1 (/ (- (window-width) notmuch-hello-indent) ;; Count is 9 wide (8 digits plus space), 1 for the space ;; after the name. (+ 9 1 widest))))))) - (cons tags-per-line (/ (max 1 (- (window-width) notmuch-hello-indent ;; Count is 9 wide (8 digits plus @@ -489,8 +545,7 @@ If FILTER is a function, it is called with QUERY as a parameter and the string it returns is used as the query. If nil is returned, the entry is hidden. -Otherwise, FILTER is ignored. -" +Otherwise, FILTER is ignored." (cond ((functionp filter) (funcall filter query)) ((stringp filter) @@ -510,7 +565,8 @@ with any properties in the original saved-search. The values :show-empty-searches, :filter and :filter-count from options will be handled as specified for -`notmuch-hello-insert-searches'." +`notmuch-hello-insert-searches'. :disable-includes can be used to +turn off the default exclude processing in `notmuch-count(1)'" (with-temp-buffer (dolist (elem query-list nil) (let ((count-query (or (notmuch-saved-search-get elem :count-query) @@ -521,31 +577,44 @@ options will be handled as specified for (notmuch-hello-filtered-query count-query (or (plist-get options :filter-count) (plist-get options :filter)))) - "\n"))) - - (unless (= (call-process-region (point-min) (point-max) notmuch-command - t t nil "count" "--batch") 0) - (notmuch-logged-error "notmuch count --batch failed" - "Please check that the notmuch CLI is new enough to support `count + "\n"))) + (unless (= (notmuch--call-process-region (point-min) (point-max) notmuch-command + t t nil "count" + (if (plist-get options :disable-excludes) + "--exclude=false" + "--exclude=true") + "--batch") 0) + (notmuch-logged-error + "notmuch count --batch failed" + "Please check that the notmuch CLI is new enough to support `count --batch'. In general we recommend running matching versions of the CLI and emacs interface.")) - (goto-char (point-min)) + (cl-mapcan + (lambda (elem) + (let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)) + (search-query (plist-get elem-plist :query)) + (filtered-query (notmuch-hello-filtered-query + search-query (plist-get options :filter))) + (message-count (prog1 (read (current-buffer)) + (forward-line 1)))) + (when (and filtered-query (or (plist-get options :show-empty-searches) + (> message-count 0))) + (setq elem-plist (plist-put elem-plist :query filtered-query)) + (list (plist-put elem-plist :count message-count))))) + query-list))) - (notmuch-remove-if-not - #'identity - (mapcar - (lambda (elem) - (let* ((elem-plist (notmuch-hello-saved-search-to-plist elem)) - (search-query (plist-get elem-plist :query)) - (filtered-query (notmuch-hello-filtered-query - search-query (plist-get options :filter))) - (message-count (prog1 (read (current-buffer)) - (forward-line 1)))) - (when (and filtered-query (or (plist-get options :show-empty-searches) (> message-count 0))) - (setq elem-plist (plist-put elem-plist :query filtered-query)) - (plist-put elem-plist :count message-count)))) - query-list)))) +(defun notmuch-hello-nice-number (n) + (let (result) + (while (> n 0) + (push (% n 1000) result) + (setq n (/ n 1000))) + (setq result (or result '(0))) + (apply #'concat + (number-to-string (car result)) + (mapcar (lambda (elem) + (format "%s%03d" notmuch-hello-thousands-separator elem)) + (cdr result))))) (defun notmuch-hello-insert-buttons (searches) "Insert buttons for SEARCHES. @@ -571,15 +640,19 @@ with `notmuch-hello-query-counts'." (mapc (lambda (elem) ;; (not elem) indicates an empty slot in the matrix. (when elem - (if (> column-indent 0) - (widget-insert (make-string column-indent ? ))) + (when (> column-indent 0) + (widget-insert (make-string column-indent ? ))) (let* ((name (plist-get elem :name)) (query (plist-get elem :query)) - (oldest-first (case (plist-get elem :sort-order) + (oldest-first (cl-case (plist-get elem :sort-order) (newest-first nil) (oldest-first t) (otherwise notmuch-search-oldest-first))) - (search-type (eq (plist-get elem :search-type) 'tree)) + (exclude (cl-case (plist-get elem :excluded) + (hide t) + (show nil) + (otherwise notmuch-search-hide-excluded))) + (search-type (plist-get elem :search-type)) (msg-count (plist-get elem :count))) (widget-insert (format "%8s " (notmuch-hello-nice-number msg-count))) @@ -588,21 +661,21 @@ with `notmuch-hello-query-counts'." :notmuch-search-terms query :notmuch-search-oldest-first oldest-first :notmuch-search-type search-type + :notmuch-search-hide-excluded exclude name) (setq column-indent (1+ (max 0 (- column-width (length name))))))) - (setq count (1+ count)) + (cl-incf count) (when (eq (% count tags-per-line) 0) (setq column-indent 0) (widget-insert "\n"))) reordered-list) - ;; If the last line was not full (and hence did not include a ;; carriage return), insert one now. (unless (eq (% count tags-per-line) 0) (widget-insert "\n")))) -(defimage notmuch-hello-logo ((:type png :file "notmuch-logo.png"))) +;;; Mode (defun notmuch-hello-update () "Update the notmuch-hello buffer." @@ -621,7 +694,7 @@ with `notmuch-hello-query-counts'." (dolist (window (window-list)) (let ((last-buf (window-parameter window 'notmuch-hello-last-buffer)) (cur-buf (window-buffer window))) - (when (not (eq last-buf cur-buf)) + (unless (eq last-buf cur-buf) ;; This window changed or is new. Update recorded buffer ;; for next time. (set-window-parameter window 'notmuch-hello-last-buffer cur-buf) @@ -634,46 +707,28 @@ with `notmuch-hello-query-counts'." ;; Refresh hello as soon as we get back to redisplay. On Emacs ;; 24, we can't do it right here because something in this ;; hook's call stack overrides hello's point placement. + ;; FIXME And on Emacs releases that we still support? (run-at-time nil nil #'notmuch-hello t)) - (when (null hello-buf) + (unless hello-buf ;; Clean up hook (remove-hook 'window-configuration-change-hook #'notmuch-hello-window-configuration-change)))) -;; the following variable is defined as being defconst in notmuch-version.el -(defvar notmuch-emacs-version) - -(defun notmuch-hello-versions () - "Display the notmuch version(s)" - (interactive) - (let ((notmuch-cli-version (notmuch-cli-version))) - (message "notmuch version %s" - (if (string= notmuch-emacs-version notmuch-cli-version) - notmuch-cli-version - (concat notmuch-cli-version - " (emacs mua version " notmuch-emacs-version ")"))))) - (defvar notmuch-hello-mode-map - (let ((map (if (fboundp 'make-composed-keymap) - ;; Inherit both widget-keymap and - ;; notmuch-common-keymap. We have to use - ;; make-sparse-keymap to force this to be a new - ;; keymap (so that when we modify map it does not - ;; modify widget-keymap). - (make-composed-keymap (list (make-sparse-keymap) widget-keymap)) - ;; Before Emacs 24, keymaps didn't support multiple - ;; inheritance,, so just copy the widget keymap since - ;; it's unlikely to change. - (copy-keymap widget-keymap)))) + ;; Inherit both widget-keymap and notmuch-common-keymap. We have + ;; to use make-sparse-keymap to force this to be a new keymap (so + ;; that when we modify map it does not modify widget-keymap). + (let ((map (make-composed-keymap (list (make-sparse-keymap) widget-keymap)))) (set-keymap-parent map notmuch-common-keymap) - (define-key map "v" 'notmuch-hello-versions) - (define-key map (kbd "") 'widget-backward) + ;; Currently notmuch-hello-mode supports free text entry, but not + ;; tagging operations, so provide standard undo. + (define-key map [remap notmuch-tag-undo] #'undo) map) "Keymap for \"notmuch hello\" buffers.") -(fset 'notmuch-hello-mode-map notmuch-hello-mode-map) (define-derived-mode notmuch-hello-mode fundamental-mode "notmuch-hello" - "Major mode for convenient notmuch navigation. This is your entry portal into notmuch. + "Major mode for convenient notmuch navigation. This is your entry +portal into notmuch. Saved searches are \"bookmarks\" for arbitrary queries. Hit RET or click on a saved search to view matching threads. Edit saved @@ -703,18 +758,18 @@ The screen may be customized via `\\[customize]'. Complete list of currently available key bindings: \\{notmuch-hello-mode-map}" - (setq notmuch-buffer-refresh-function #'notmuch-hello-update) - ;;(setq buffer-read-only t) -) + (setq notmuch-buffer-refresh-function #'notmuch-hello-update)) + +;;; Inserters (defun notmuch-hello-generate-tag-alist (&optional hide-tags) "Return an alist from tags to queries to display in the all-tags section." - (mapcar (lambda (tag) - (cons tag (concat "tag:" (notmuch-escape-boolean-term tag)))) - (notmuch-remove-if-not - (lambda (tag) - (not (member tag hide-tags))) - (process-lines notmuch-command "search" "--output=tags" "*")))) + (cl-mapcan (lambda (tag) + (and (not (member tag hide-tags)) + (list (cons tag + (concat "tag:" + (notmuch-escape-boolean-term tag)))))) + (notmuch--process-lines notmuch-command "search" "--output=tags" "*"))) (defun notmuch-hello-insert-header () "Insert the default notmuch-hello header." @@ -729,7 +784,9 @@ Complete list of currently available key bindings: ;; dark background. (setq image (cons 'image (append (cdr image) - (list :background (face-background 'notmuch-hello-logo-background))))) + (list :background + (face-background + 'notmuch-hello-logo-background))))) (insert-image image)) (widget-insert " ")) @@ -738,21 +795,21 @@ Complete list of currently available key bindings: (let ((widget-link-prefix "") (widget-link-suffix "")) (widget-create 'link - :notify (lambda (&rest ignore) + :notify (lambda (&rest _ignore) (browse-url notmuch-hello-url)) :help-echo "Visit the notmuch website." "notmuch") (widget-insert ". ") (widget-insert "You have ") (widget-create 'link - :notify (lambda (&rest ignore) + :notify (lambda (&rest _ignore) (notmuch-hello-update)) :help-echo "Refresh" (notmuch-hello-nice-number - (string-to-number (car (process-lines notmuch-command "count"))))) + (string-to-number + (car (notmuch--process-lines notmuch-command "count" "--exclude=false"))))) (widget-insert " messages.\n"))) - (defun notmuch-hello-insert-saved-searches () "Insert the saved-searches section." (let ((searches (notmuch-hello-query-counts @@ -764,7 +821,7 @@ Complete list of currently available key bindings: (when searches (widget-insert "Saved searches: ") (widget-create 'push-button - :notify (lambda (&rest ignore) + :notify (lambda (&rest _ignore) (customize-variable 'notmuch-saved-searches)) "edit") (widget-insert "\n\n") @@ -780,76 +837,35 @@ Complete list of currently available key bindings: ;; search boxes. :size (max 8 (- (window-width) notmuch-hello-indent (length "Search: "))) - :action (lambda (widget &rest ignore) - (notmuch-hello-search (widget-value widget)))) + :action #'notmuch-hello-search) ;; Add an invisible dot to make `widget-end-of-line' ignore ;; trailing spaces in the search widget field. A dot is used ;; instead of a space to make `show-trailing-whitespace' ;; happy, i.e. avoid it marking the whole line as trailing ;; spaces. - (widget-insert ".") - (put-text-property (1- (point)) (point) 'invisible t) + (widget-insert (propertize "." 'invisible t)) (widget-insert "\n")) (defun notmuch-hello-insert-recent-searches () "Insert recent searches." (when notmuch-search-history (widget-insert "Recent searches: ") - (widget-create 'push-button - :notify (lambda (&rest ignore) - (when (y-or-n-p "Are you sure you want to clear the searches? ") - (setq notmuch-search-history nil) - (notmuch-hello-update))) - "clear") + (widget-create + 'push-button + :notify (lambda (&rest _ignore) + (when (y-or-n-p "Are you sure you want to clear the searches? ") + (setq notmuch-search-history nil) + (notmuch-hello-update))) + "clear") (widget-insert "\n\n") - (let ((start (point))) - (loop for i from 1 to notmuch-hello-recent-searches-max - for search in notmuch-search-history do - (let ((widget-symbol (intern (format "notmuch-hello-search-%d" i)))) - (set widget-symbol - (widget-create 'editable-field - ;; Don't let the search boxes be - ;; less than 8 characters wide. - :size (max 8 - (- (window-width) - ;; Leave some space - ;; at the start and - ;; end of the - ;; boxes. - (* 2 notmuch-hello-indent) - ;; 1 for the space - ;; before the - ;; `[save]' button. 6 - ;; for the `[save]' - ;; button. - 1 6 - ;; 1 for the space - ;; before the `[del]' - ;; button. 5 for the - ;; `[del]' button. - 1 5)) - :action (lambda (widget &rest ignore) - (notmuch-hello-search (widget-value widget))) - search)) - (widget-insert " ") - (widget-create 'push-button - :notify (lambda (widget &rest ignore) - (notmuch-hello-add-saved-search widget)) - :notmuch-saved-search-widget widget-symbol - "save") - (widget-insert " ") - (widget-create 'push-button - :notify (lambda (widget &rest ignore) - (when (y-or-n-p "Are you sure you want to delete this search? ") - (notmuch-hello-delete-search-from-history widget))) - :notmuch-saved-search-widget widget-symbol - "del")) - (widget-insert "\n")) - (indent-rigidly start (point) notmuch-hello-indent)) - nil)) + (let ((width (notmuch-search-item-field-width))) + (dolist (search (seq-take notmuch-search-history + notmuch-hello-recent-searches-max)) + (widget-create 'notmuch-search-item :value search :size width))))) (defun notmuch-hello-insert-searches (title query-list &rest options) - "Insert a section with TITLE showing a list of buttons made from QUERY-LIST. + "Insert a section with TITLE showing a list of buttons made from +QUERY-LIST. QUERY-LIST should ideally be a plist but for backwards compatibility other forms are also accepted (see @@ -863,33 +879,36 @@ Supports the following entries in OPTIONS as a plist: :show-empty-searches - show buttons with no matching messages :hide-if-empty - hide if no buttons would be shown (only makes sense without :show-empty-searches) -:filter - This can be a function that takes the search query as its argument and - returns a filter to be used in conjunction with the query for that search or nil - to hide the element. This can also be a string that is used as a combined with - each query using \"and\". -:filter-count - Separate filter to generate the count displayed each search. Accepts - the same values as :filter. If :filter and :filter-count are specified, this - will be used instead of :filter, not in conjunction with it." +:filter - This can be a function that takes the search query as + its argument and returns a filter to be used in conjunction + with the query for that search or nil to hide the + element. This can also be a string that is used as a combined + with each query using \"and\". +:filter-count - Separate filter to generate the count displayed + each search. Accepts the same values as :filter. If :filter + and :filter-count are specified, this will be used instead of + :filter, not in conjunction with it." + (widget-insert title ": ") - (if (and notmuch-hello-first-run (plist-get options :initially-hidden)) - (add-to-list 'notmuch-hello-hidden-sections title)) + (when (and notmuch-hello-first-run (plist-get options :initially-hidden)) + (add-to-list 'notmuch-hello-hidden-sections title)) (let ((is-hidden (member title notmuch-hello-hidden-sections)) (start (point))) (if is-hidden (widget-create 'push-button - :notify `(lambda (widget &rest ignore) - (setq notmuch-hello-hidden-sections - (delete ,title notmuch-hello-hidden-sections)) - (notmuch-hello-update)) + :notify (lambda (&rest _ignore) + (setq notmuch-hello-hidden-sections + (delete title notmuch-hello-hidden-sections)) + (notmuch-hello-update)) "show") (widget-create 'push-button - :notify `(lambda (widget &rest ignore) - (add-to-list 'notmuch-hello-hidden-sections - ,title) - (notmuch-hello-update)) + :notify (lambda (&rest _ignore) + (add-to-list 'notmuch-hello-hidden-sections + title) + (notmuch-hello-update)) "hide")) (widget-insert "\n") - (when (not is-hidden) + (unless is-hidden (let ((searches (apply 'notmuch-hello-query-counts query-list options))) (when (or (not (plist-get options :hide-if-empty)) searches) @@ -911,7 +930,7 @@ following: options)) (defun notmuch-hello-insert-inbox () - "Show an entry for each saved search and inboxed messages for each tag" + "Show an entry for each saved search and inboxed messages for each tag." (notmuch-hello-insert-searches "What's in your inbox" (append notmuch-saved-searches @@ -919,12 +938,13 @@ following: :filter "tag:inbox")) (defun notmuch-hello-insert-alltags () - "Insert a section displaying all tags and associated message counts" + "Insert a section displaying all tags and associated message counts." (notmuch-hello-insert-tags-section nil :initially-hidden (not notmuch-show-all-tags-list) :hide-tags notmuch-hello-hide-tags - :filter notmuch-hello-tag-list-make-query)) + :filter notmuch-hello-tag-list-make-query + :disable-excludes t)) (defun notmuch-hello-insert-footer () "Insert the notmuch-hello footer." @@ -932,57 +952,51 @@ following: (widget-insert "Hit `?' for context-sensitive help in any Notmuch screen.\n") (widget-insert "Customize ") (widget-create 'link - :notify (lambda (&rest ignore) + :notify (lambda (&rest _ignore) (customize-group 'notmuch)) :button-prefix "" :button-suffix "" "Notmuch") (widget-insert " or ") (widget-create 'link - :notify (lambda (&rest ignore) + :notify (lambda (&rest _ignore) (customize-variable 'notmuch-hello-sections)) :button-prefix "" :button-suffix "" "this page.") (let ((fill-column (- (window-width) notmuch-hello-indent))) (center-region start (point))))) +;;; Hello! + ;;;###autoload (defun notmuch-hello (&optional no-display) "Run notmuch and display saved searches, known tags, etc." (interactive) - (notmuch-assert-cli-sane) ;; This may cause a window configuration change, so if the ;; auto-refresh hook is already installed, avoid recursive refresh. (let ((notmuch-hello-auto-refresh nil)) (if no-display (set-buffer "*notmuch-hello*") - (switch-to-buffer "*notmuch-hello*"))) - + (pop-to-buffer-same-window "*notmuch-hello*"))) ;; Install auto-refresh hook (when notmuch-hello-auto-refresh (add-hook 'window-configuration-change-hook #'notmuch-hello-window-configuration-change)) - (let ((target-line (line-number-at-pos)) (target-column (current-column)) (inhibit-read-only t)) - ;; Delete all editable widget fields. Editable widget fields are ;; tracked in a buffer local variable `widget-field-list' (and ;; others). If we do `erase-buffer' without properly deleting the ;; widgets, some widget-related functions are confused later. (mapc 'widget-delete widget-field-list) - (erase-buffer) - (unless (eq major-mode 'notmuch-hello-mode) (notmuch-hello-mode)) - (let ((all (overlay-lists))) ;; Delete all the overlays. (mapc 'delete-overlay (car all)) (mapc 'delete-overlay (cdr all))) - (mapc (lambda (section) (let ((point-before (point))) @@ -995,7 +1009,6 @@ following: (widget-insert "\n")))) notmuch-hello-sections) (widget-setup) - ;; Move point back to where it was before refresh. Use line and ;; column instead of point directly to be insensitive to additions ;; and removals of text within earlier lines. @@ -1005,12 +1018,7 @@ following: (run-hooks 'notmuch-hello-refresh-hook) (setq notmuch-hello-first-run nil)) -(defun notmuch-folder () - "Deprecated function for invoking notmuch---calling `notmuch' is preferred now." - (interactive) - (notmuch-hello)) - -;; +;;; _ (provide 'notmuch-hello) diff --git a/emacs/notmuch-jump.el b/emacs/notmuch-jump.el index 3e20b8c7..3161ed95 100644 --- a/emacs/notmuch-jump.el +++ b/emacs/notmuch-jump.el @@ -1,4 +1,4 @@ -;;; notmuch-jump.el --- User-friendly shortcut keys +;;; notmuch-jump.el --- User-friendly shortcut keys -*- lexical-binding: t -*- ;; ;; Copyright © Austin Clements ;; @@ -22,15 +22,12 @@ ;;; Code: -(eval-when-compile (require 'cl)) - (require 'notmuch-lib) (require 'notmuch-hello) -(eval-and-compile - (unless (fboundp 'window-body-width) - ;; Compatibility for Emacs pre-24 - (defalias 'window-body-width 'window-width))) +(declare-function notmuch-search "notmuch") +(declare-function notmuch-tree "notmuch-tree") +(declare-function notmuch-unthreaded "notmuch-tree") ;;;###autoload (defun notmuch-jump-search () @@ -41,7 +38,6 @@ keys configured in the :key property of `notmuch-saved-searches'. Typically these shortcuts are a single key long, so this is a fast way to jump to a saved search from anywhere in Notmuch." (interactive) - ;; Build the action map (let (action-map) (dolist (saved-search notmuch-saved-searches) @@ -51,23 +47,39 @@ fast way to jump to a saved search from anywhere in Notmuch." (let ((name (plist-get saved-search :name)) (query (plist-get saved-search :query)) (oldest-first - (case (plist-get saved-search :sort-order) + (cl-case (plist-get saved-search :sort-order) (newest-first nil) (oldest-first t) - (otherwise (default-value 'notmuch-search-oldest-first))))) + (otherwise (default-value 'notmuch-search-oldest-first)))) + (exclude (cl-case (plist-get saved-search :excluded) + (hide t) + (show nil) + (otherwise notmuch-search-hide-excluded)))) (push (list key name - (if (eq (plist-get saved-search :search-type) 'tree) - `(lambda () (notmuch-tree ',query)) - `(lambda () (notmuch-search ',query ',oldest-first)))) + (cond + ((eq (plist-get saved-search :search-type) 'tree) + (lambda () (notmuch-tree query nil nil nil nil nil nil + oldest-first exclude))) + ((eq (plist-get saved-search :search-type) 'unthreaded) + (lambda () (notmuch-unthreaded query nil nil nil nil + oldest-first exclude))) + (t + (lambda () (notmuch-search query oldest-first exclude))))) action-map))))) (setq action-map (nreverse action-map)) - (if action-map (notmuch-jump action-map "Search: ") - (error "To use notmuch-jump, please customize shortcut keys in notmuch-saved-searches.")))) + (error "To use notmuch-jump, %s" + "please customize shortcut keys in notmuch-saved-searches.")))) + +(defface notmuch-jump-key + '((t :inherit minibuffer-prompt)) + "Default face used for keys in `notmuch-jump' and related." + :group 'notmuch-faces) (defvar notmuch-jump--action nil) +;;;###autoload (defun notmuch-jump (action-map prompt) "Interactively prompt for one of the keys in ACTION-MAP. @@ -82,9 +94,7 @@ ACTION-MAP must be a list of triples of the form where KEY is a key binding, LABEL is a string label to display in the buffer, and ACTION is a nullary function to call. LABEL may be null, in which case the action will still be bound, but will -not appear in the pop-up buffer. -" - +not appear in the pop-up buffer." (let* ((items (notmuch-jump--format-actions action-map)) ;; Format the table of bindings and the full prompt (table @@ -109,7 +119,6 @@ not appear in the pop-up buffer. (notmuch-jump--action nil)) ;; Read the action (read-from-minibuffer full-prompt nil minibuffer-map) - ;; If we got an action, do it (when notmuch-jump--action (funcall notmuch-jump--action)))) @@ -120,21 +129,18 @@ not appear in the pop-up buffer. Returns a list of strings, one for each item with a label in ACTION-MAP. These strings can be inserted into a tabular buffer." - ;; Compute the maximum key description width (let ((key-width 1)) - (dolist (entry action-map) + (pcase-dolist (`(,key ,_desc) action-map) (setq key-width (max key-width - (string-width (format-kbd-macro (first entry)))))) + (string-width (format-kbd-macro key))))) ;; Format each action - (mapcar (lambda (entry) - (let ((key (format-kbd-macro (first entry))) - (desc (second entry))) - (concat - (propertize key 'face 'minibuffer-prompt) - (make-string (- key-width (length key)) ? ) - " " desc))) + (mapcar (pcase-lambda (`(,key ,desc)) + (setq key (format-kbd-macro key)) + (concat (propertize key 'face 'notmuch-jump-key) + (make-string (- key-width (length key)) ? ) + " " desc)) action-map))) (defun notmuch-jump--insert-items (width items) @@ -169,43 +175,42 @@ buffer." "Translate ACTION-MAP into a minibuffer keymap." (let ((map (make-sparse-keymap))) (set-keymap-parent map notmuch-jump-minibuffer-map) - (dolist (action action-map) - (if (= (length (first action)) 1) - (define-key map (first action) - `(lambda () (interactive) - (setq notmuch-jump--action ',(third action)) - (exit-minibuffer))))) + (pcase-dolist (`(,key ,_name ,fn) action-map) + (when (= (length key) 1) + (define-key map key + (lambda () + (interactive) + (setq notmuch-jump--action fn) + (exit-minibuffer))))) ;; By doing this in two passes (and checking if we already have a ;; binding) we avoid problems if the user specifies a binding which ;; is a prefix of another binding. - (dolist (action action-map) - (if (> (length (first action)) 1) - (let* ((key (elt (first action) 0)) - (keystr (string key)) - (new-prompt (concat prompt (format-kbd-macro keystr) " ")) - (action-submap nil)) - (unless (lookup-key map keystr) - (dolist (act action-map) - (when (= key (elt (first act) 0)) - (push (list (substring (first act) 1) - (second act) - (third act)) - action-submap))) - ;; We deal with backspace specially - (push (list (kbd "DEL") - "Backup" - (apply-partially #'notmuch-jump action-map prompt)) - action-submap) - (setq action-submap (nreverse action-submap)) - (define-key map keystr - `(lambda () (interactive) - (setq notmuch-jump--action - ',(apply-partially #'notmuch-jump action-submap new-prompt)) - (exit-minibuffer))))))) + (pcase-dolist (`(,key ,_name ,_fn) action-map) + (when (> (length key) 1) + (let* ((key (elt key 0)) + (keystr (string key)) + (new-prompt (concat prompt (format-kbd-macro keystr) " ")) + (action-submap nil)) + (unless (lookup-key map keystr) + (pcase-dolist (`(,k ,n ,f) action-map) + (when (= key (elt k 0)) + (push (list (substring k 1) n f) action-submap))) + ;; We deal with backspace specially + (push (list (kbd "DEL") + "Backup" + (apply-partially #'notmuch-jump action-map prompt)) + action-submap) + (setq action-submap (nreverse action-submap)) + (define-key map keystr + (lambda () + (interactive) + (setq notmuch-jump--action + (apply-partially #'notmuch-jump + action-submap + new-prompt)) + (exit-minibuffer))))))) map)) -;; - (provide 'notmuch-jump) ;;; notmuch-jump.el ends here diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el index 8acad267..bf9c4a53 100644 --- a/emacs/notmuch-lib.el +++ b/emacs/notmuch-lib.el @@ -1,4 +1,4 @@ -;;; notmuch-lib.el --- common variables, functions and function declarations +;;; notmuch-lib.el --- common variables, functions and function declarations -*- lexical-binding: t -*- ;; ;; Copyright © Carl Worth ;; @@ -19,22 +19,23 @@ ;; ;; Authors: Carl Worth -;; This is an part of an emacs-based interface to the notmuch mail system. - ;;; Code: +(require 'cl-lib) +(require 'pcase) +(require 'subr-x) + (require 'mm-util) (require 'mm-view) (require 'mm-decode) -(require 'cl) + (require 'notmuch-compat) (unless (require 'notmuch-version nil t) (defconst notmuch-emacs-version "unknown" "Placeholder variable when notmuch-version.el[c] is not available.")) -(autoload 'notmuch-jump-search "notmuch-jump" - "Jump to a saved search by shortcut key." t) +;;; Groups (defgroup notmuch nil "Notmuch mail reader for Emacs." @@ -54,9 +55,8 @@ (defgroup notmuch-send nil "Sending messages from Notmuch." - :group 'notmuch) - -(custom-add-to-group 'notmuch-send 'message 'custom-group) + :group 'notmuch + :group 'message) (defgroup notmuch-tag nil "Tags and tagging in Notmuch." @@ -82,6 +82,8 @@ "Graphical attributes for displaying text" :group 'notmuch) +;;; Options + (defcustom notmuch-command "notmuch" "Name of the notmuch binary. @@ -101,6 +103,17 @@ search results. Note that any filtered searches created by search." :type 'boolean :group 'notmuch-search) +(make-variable-buffer-local 'notmuch-search-oldest-first) + +(defcustom notmuch-search-hide-excluded t + "Hide mail tagged with a excluded tag. + +Excluded tags are defined in the users configuration file under +the search section. When this variable is true, any mail with +such a tag will not be shown in the search output." + :type 'boolean + :group 'notmuch-search) +(make-variable-buffer-local 'notmuch-search-hide-excluded) (defcustom notmuch-poll-script nil "[Deprecated] Command to run to incorporate new mail into the notmuch database. @@ -129,11 +142,6 @@ the user's needs: (string :tag "Custom script")) :group 'notmuch-external) -;; - -(defvar notmuch-search-history nil - "Variable to store notmuch searches history.") - (defcustom notmuch-archive-tags '("-inbox") "List of tag changes to apply to a message or a thread when it is archived. @@ -148,18 +156,27 @@ For example, if you wanted to remove an \"inbox\" tag and add an :group 'notmuch-search :group 'notmuch-show) +;;; Variables + +(defvar notmuch-search-history nil + "Variable to store notmuch searches history.") + (defvar notmuch-common-keymap (let ((map (make-sparse-keymap))) (define-key map "?" 'notmuch-help) + (define-key map "v" 'notmuch-version) (define-key map "q" 'notmuch-bury-or-kill-this-buffer) (define-key map "s" 'notmuch-search) + (define-key map "t" 'notmuch-search-by-tag) (define-key map "z" 'notmuch-tree) + (define-key map "u" 'notmuch-unthreaded) (define-key map "m" 'notmuch-mua-new-mail) (define-key map "g" 'notmuch-refresh-this-buffer) (define-key map "=" 'notmuch-refresh-this-buffer) (define-key map (kbd "M-=") 'notmuch-refresh-all-buffers) (define-key map "G" 'notmuch-poll-and-refresh-this-buffer) (define-key map "j" 'notmuch-jump-search) + (define-key map [remap undo] 'notmuch-tag-undo) map) "Keymap shared by all notmuch modes.") @@ -178,6 +195,8 @@ For example, if you wanted to remove an \"inbox\" tag and add an (select-window (posn-window (event-start last-input-event))) (button-activate button))) +;;; CLI Utilities + (defun notmuch-command-to-string (&rest args) "Synchronously invoke \"notmuch\" with the given list of arguments. @@ -185,10 +204,10 @@ If notmuch exits with a non-zero status, output from the process will appear in a buffer named \"*Notmuch errors*\" and an error will be signaled. -Otherwise the output will be returned" +Otherwise the output will be returned." (with-temp-buffer - (let* ((status (apply #'call-process notmuch-command nil t nil args)) - (output (buffer-string))) + (let ((status (apply #'notmuch--call-process notmuch-command nil t nil args)) + (output (buffer-string))) (notmuch-check-exit-status status (cons notmuch-command args) output) output))) @@ -198,7 +217,7 @@ Otherwise the output will be returned" (defun notmuch-cli-sane-p () "Return t if the cli seems to be configured sanely." (unless notmuch--cli-sane-p - (let ((status (call-process notmuch-command nil nil nil + (let ((status (notmuch--call-process notmuch-command nil nil nil "config" "get" "user.primary_email"))) (setq notmuch--cli-sane-p (= status 0)))) notmuch--cli-sane-p) @@ -207,7 +226,7 @@ Otherwise the output will be returned" (unless (notmuch-cli-sane-p) (notmuch-logged-error "notmuch cli seems misconfigured or unconfigured." -"Perhaps you haven't run \"notmuch setup\" yet? Try running this + "Perhaps you haven't run \"notmuch setup\" yet? Try running this on the command line, and then retry your notmuch command"))) (defun notmuch-cli-version () @@ -220,13 +239,31 @@ on the command line, and then retry your notmuch command"))) (match-string 2 long-string) "unknown"))) +(defvar notmuch-emacs-version) + +(defun notmuch-version () + "Display the notmuch version. +The versions of the Emacs package and the `notmuch' executable +should match, but if and only if they don't, then this command +displays both values separately." + (interactive) + (let ((cli-version (notmuch-cli-version))) + (message "notmuch version %s" + (if (string= notmuch-emacs-version cli-version) + cli-version + (concat cli-version + " (emacs mua version " notmuch-emacs-version ")"))))) + +;;; Notmuch Configuration + (defun notmuch-config-get (item) "Return a value from the notmuch configuration." (let* ((val (notmuch-command-to-string "config" "get" item)) (len (length val))) ;; Trim off the trailing newline (if the value is empty or not - ;; configured, there will be no newline) - (if (and (> len 0) (= (aref val (- len 1)) ?\n)) + ;; configured, there will be no newline). + (if (and (> len 0) + (= (aref val (- len 1)) ?\n)) (substring val 0 -1) val))) @@ -249,17 +286,21 @@ on the command line, and then retry your notmuch command"))) (defun notmuch-user-emails () (cons (notmuch-user-primary-email) (notmuch-user-other-email))) +;;; Commands + (defun notmuch-poll () "Run \"notmuch new\" or an external script to import mail. Invokes `notmuch-poll-script', \"notmuch new\", or does nothing depending on the value of `notmuch-poll-script'." (interactive) + (message "Polling mail...") (if (stringp notmuch-poll-script) - (unless (string= notmuch-poll-script "") - (unless (equal (call-process notmuch-poll-script nil nil) 0) + (unless (string-empty-p notmuch-poll-script) + (unless (equal (notmuch--call-process notmuch-poll-script nil nil) 0) (error "Notmuch: poll script `%s' failed!" notmuch-poll-script))) - (notmuch-call-notmuch-process "new"))) + (notmuch-call-notmuch-process "new")) + (message "Polling mail...done")) (defun notmuch-bury-or-kill-this-buffer () "Undisplay the current buffer. @@ -271,17 +312,7 @@ it, in which case it is killed." (bury-buffer) (kill-buffer))) -(defun notmuch-documentation-first-line (symbol) - "Return the first line of the documentation string for SYMBOL." - (let ((doc (documentation symbol))) - (if doc - (with-temp-buffer - (insert (documentation symbol t)) - (goto-char (point-min)) - (let ((beg (point))) - (end-of-line) - (buffer-substring beg (point)))) - ""))) +;;; Describe Key Bindings (defun notmuch-prefix-key-description (key) "Given a prefix key code, return a human-readable string representation. @@ -293,9 +324,8 @@ This is basically just `format-kbd-macro' but we also convert ESC to M-." "M-" (concat desc " ")))) - (defun notmuch-describe-key (actual-key binding prefix ua-keys tail) - "Prepend cons cells describing prefix-arg ACTUAL-KEY and ACTUAL-KEY to TAIL + "Prepend cons cells describing prefix-arg ACTUAL-KEY and ACTUAL-KEY to TAIL. It does not prepend if ACTUAL-KEY is already listed in TAIL." (let ((key-string (concat prefix (key-description actual-key)))) @@ -312,10 +342,15 @@ It does not prepend if ACTUAL-KEY is already listed in TAIL." tail))) ;; Documentation for command (push (cons key-string - (or (and (symbolp binding) (get binding 'notmuch-doc)) - (and (functionp binding) (notmuch-documentation-first-line binding)))) + (or (and (symbolp binding) + (get binding 'notmuch-doc)) + (and (functionp binding) + (let ((doc (documentation binding))) + (and doc + (string-match "\\`.+" doc) + (match-string 0 doc)))))) tail))) - tail) + tail) (defun notmuch-describe-remaps (remap-keymap ua-keys base-keymap prefix tail) ;; Remappings are represented as a binding whose first "event" is @@ -323,13 +358,13 @@ It does not prepend if ACTUAL-KEY is already listed in TAIL." ;; binding whose "key" is 'remap, and whose "binding" is itself a ;; keymap that maps not from keys to commands, but from old (remapped) ;; functions to the commands to use in their stead. - (map-keymap - (lambda (command binding) - (mapc - (lambda (actual-key) - (setq tail (notmuch-describe-key actual-key binding prefix ua-keys tail))) - (where-is-internal command base-keymap))) - remap-keymap) + (map-keymap (lambda (command binding) + (mapc (lambda (actual-key) + (setq tail + (notmuch-describe-key actual-key binding + prefix ua-keys tail))) + (where-is-internal command base-keymap))) + remap-keymap) tail) (defun notmuch-describe-keymap (keymap ua-keys base-keymap &optional prefix tail) @@ -352,9 +387,13 @@ prefix argument. PREFIX and TAIL are used internally." (notmuch-describe-remaps binding ua-keys base-keymap prefix tail) (notmuch-describe-keymap - binding ua-keys base-keymap (notmuch-prefix-key-description key) tail)))) + binding ua-keys base-keymap + (notmuch-prefix-key-description key) + tail)))) (binding - (setq tail (notmuch-describe-key (vector key) binding prefix ua-keys tail))))) + (setq tail + (notmuch-describe-key (vector key) + binding prefix ua-keys tail))))) keymap) tail) @@ -364,11 +403,15 @@ prefix argument. PREFIX and TAIL are used internally." (while (string-match "\\\\{\\([^}[:space:]]*\\)}" doc beg) (let ((desc (save-match-data - (let* ((keymap-name (substring doc (match-beginning 1) (match-end 1))) + (let* ((keymap-name (substring doc + (match-beginning 1) + (match-end 1))) (keymap (symbol-value (intern keymap-name))) (ua-keys (where-is-internal 'universal-argument keymap t)) (desc-alist (notmuch-describe-keymap keymap ua-keys keymap)) - (desc-list (mapcar (lambda (arg) (concat (car arg) "\t" (cdr arg))) desc-alist))) + (desc-list (mapcar (lambda (arg) + (concat (car arg) "\t" (cdr arg))) + desc-alist))) (mapconcat #'identity desc-list "\n"))))) (setq doc (replace-match desc 1 1 doc))) (setq beg (match-end 0))) @@ -381,13 +424,14 @@ This is similar to `describe-function' for the current major mode, but bindings tables are shown with documentation strings rather than command names. By default, this uses the first line of each command's documentation string. A command can override -this by setting the 'notmuch-doc property of its command symbol. +this by setting the \\='notmuch-doc property of its command symbol. A command that supports a prefix argument can explicitly document -its prefixed behavior by setting the 'notmuch-prefix-doc property +its prefixed behavior by setting the \\='notmuch-prefix-doc property of its command symbol." (interactive) - (let* ((mode major-mode) - (doc (substitute-command-keys (notmuch-substitute-command-keys (documentation mode t))))) + (let ((doc (substitute-command-keys + (notmuch-substitute-command-keys + (documentation major-mode t))))) (with-current-buffer (generate-new-buffer "*notmuch-help*") (insert doc) (goto-char (point-min)) @@ -398,17 +442,18 @@ of its command symbol." "Show help for a subkeymap." (interactive) (let* ((key (this-command-keys-vector)) - (prefix (make-vector (1- (length key)) nil)) - (i 0)) + (prefix (make-vector (1- (length key)) nil)) + (i 0)) (while (< i (length prefix)) (aset prefix i (aref key i)) - (setq i (1+ i))) - + (cl-incf i)) (let* ((subkeymap (key-binding prefix)) (ua-keys (where-is-internal 'universal-argument nil t)) (prefix-string (notmuch-prefix-key-description prefix)) - (desc-alist (notmuch-describe-keymap subkeymap ua-keys subkeymap prefix-string)) - (desc-list (mapcar (lambda (arg) (concat (car arg) "\t" (cdr arg))) desc-alist)) + (desc-alist (notmuch-describe-keymap + subkeymap ua-keys subkeymap prefix-string)) + (desc-list (mapcar (lambda (arg) (concat (car arg) "\t" (cdr arg))) + desc-alist)) (desc (mapconcat #'identity desc-list "\n"))) (with-help-window (help-buffer) (with-current-buffer standard-output @@ -416,9 +461,10 @@ of its command symbol." (insert desc))) (pop-to-buffer (help-buffer))))) -(defvar notmuch-buffer-refresh-function nil +;;; Refreshing Buffers + +(defvar-local notmuch-buffer-refresh-function nil "Function to call to refresh the current buffer.") -(make-variable-buffer-local 'notmuch-buffer-refresh-function) (defun notmuch-refresh-this-buffer () "Refresh the current buffer." @@ -448,9 +494,11 @@ be displayed." (with-current-buffer buffer (notmuch-refresh-this-buffer)))))) +;;; String Utilities + (defun notmuch-prettify-subject (subject) - ;; This function is used by `notmuch-search-process-filter' which - ;; requires that we not disrupt its' matching state. + ;; This function is used by `notmuch-search-process-filter', + ;; which requires that we not disrupt its matching state. (save-match-data (if (and subject (string-match "^[ \t]*$" subject)) @@ -469,7 +517,6 @@ This includes newlines, tabs, and other funny characters." The caller is responsible for prepending the term prefix and a colon. This performs minimal escaping in order to produce user-friendly queries." - (save-match-data (if (or (equal term "") ;; To be pessimistic, only pass through terms composed @@ -492,8 +539,6 @@ This replaces spaces, percents, and double quotes in STR with (replace-regexp-in-string "[ %\"]" (lambda (match) (format "%%%02x" (aref match 0))) str)) -;; - (defun notmuch-common-do-stash (text) "Common function to stash text in kill ring, and display in minibuffer." (if text @@ -505,47 +550,47 @@ This replaces spaces, percents, and double quotes in STR with (kill-new "") (message "Nothing to stash!"))) -;; - -(defun notmuch-remove-if-not (predicate list) - "Return a copy of LIST with all items not satisfying PREDICATE removed." - (let (out) - (while list - (when (funcall predicate (car list)) - (push (car list) out)) - (setq list (cdr list))) - (nreverse out))) +;;; Generic Utilities (defun notmuch-plist-delete (plist property) - (let* ((xplist (cons nil plist)) - (pred xplist)) - (while (cdr pred) - (when (eq (cadr pred) property) - (setcdr pred (cdddr pred))) - (setq pred (cddr pred))) - (cdr xplist))) - -(defun notmuch-split-content-type (content-type) - "Split content/type into 'content' and 'type'" - (split-string content-type "/")) + (let (p) + (while plist + (unless (eq property (car plist)) + (setq p (plist-put p (car plist) (cadr plist)))) + (setq plist (cddr plist))) + p)) + +;;; MML Utilities (defun notmuch-match-content-type (t1 t2) - "Return t if t1 and t2 are matching content types, taking wildcards into account" - (let ((st1 (notmuch-split-content-type t1)) - (st2 (notmuch-split-content-type t2))) - (if (or (string= (cadr st1) "*") - (string= (cadr st2) "*")) - ;; Comparison of content types should be case insensitive. - (string= (downcase (car st1)) (downcase (car st2))) - (string= (downcase t1) (downcase t2))))) - -(defvar notmuch-multipart/alternative-discouraged - '( - ;; Avoid HTML parts. + "Return t if t1 and t2 are matching content types. +Take wildcards into account." + (and (stringp t1) + (stringp t2) + (let ((st1 (split-string t1 "/")) + (st2 (split-string t2 "/"))) + (if (or (string= (cadr st1) "*") + (string= (cadr st2) "*")) + ;; Comparison of content types should be case insensitive. + (string= (downcase (car st1)) + (downcase (car st2))) + (string= (downcase t1) + (downcase t2)))))) + +(defcustom notmuch-multipart/alternative-discouraged + '(;; Avoid HTML parts. "text/html" - ;; multipart/related usually contain a text/html part and some associated graphics. - "multipart/related" - )) + ;; multipart/related usually contain a text/html part and some + ;; associated graphics. + "multipart/related") + "Which mime types to hide by default for multipart messages. + +Can either be a list of mime types (as strings) or a function +mapping a plist representing the current message to such a list. +See Info node `(notmuch-emacs) notmuch-show' for a sample function." + :group 'notmuch-show + :type '(radio (repeat :tag "MIME Types" string) + (function :tag "Function"))) (defun notmuch-multipart/alternative-determine-discouraged (msg) "Return the discouraged alternatives for the specified message." @@ -572,7 +617,7 @@ for this message, if present." (defun notmuch-parts-filter-by-type (parts type) "Given a list of message parts, return a list containing the ones matching the given type." - (remove-if-not + (cl-remove-if-not (lambda (part) (notmuch-match-content-type (plist-get part :content-type) type)) parts)) @@ -594,12 +639,14 @@ the given type." (set-buffer-multibyte nil)) (let ((args `("show" "--format=raw" ,(format "--part=%s" (plist-get part :id)) - ,@(when process-crypto '("--decrypt=true")) + ,@(and process-crypto '("--decrypt=true")) ,(notmuch-id-to-query (plist-get msg :id)))) (coding-system-for-read - (if binaryp 'no-conversion - (let ((coding-system (mm-charset-to-coding-system - (plist-get part :content-charset)))) + (if binaryp + 'no-conversion + (let ((coding-system + (mm-charset-to-coding-system + (plist-get part :content-charset)))) ;; Sadly, ;; `mm-charset-to-coding-system' seems ;; to return things that are not @@ -611,7 +658,8 @@ the given type." ;; charset is US-ASCII. RFC6657 ;; complicates this somewhat. 'us-ascii))))) - (apply #'call-process notmuch-command nil '(t nil) nil args) + (apply #'notmuch--call-process + notmuch-command nil '(t nil) nil args) (buffer-string)))))) (when (and cache data) (plist-put part plist-elem data)) @@ -640,20 +688,6 @@ If CACHE is non-nil, the content of this part will be saved in MSG (if it isn't already)." (notmuch--get-bodypart-raw msg part process-crypto nil cache)) -;; Workaround: The call to `mm-display-part' below triggers a bug in -;; Emacs 24 if it attempts to use the shr renderer to display an HTML -;; part with images in it (demonstrated in 24.1 and 24.2 on Debian and -;; Fedora 17, though unreproducable in other configurations). -;; `mm-shr' references the variable `gnus-inhibit-images' without -;; first loading gnus-art, which defines it, resulting in a -;; void-variable error. Hence, we advise `mm-shr' to ensure gnus-art -;; is loaded. -(if (>= emacs-major-version 24) - (defadvice mm-shr (before load-gnus-arts activate) - (require 'gnus-art nil t) - (ad-disable-advice 'mm-shr 'before 'load-gnus-arts) - (ad-activate 'mm-shr))) - (defun notmuch-mm-display-part-inline (msg part content-type process-crypto) "Use the mm-decode/mm-view functions to display a part in the current buffer, if possible." @@ -664,9 +698,11 @@ current buffer, if possible." ;; `gnus-decoded' charset. Otherwise, we'll fetch the binary ;; part content and let mm-* decode it. (let* ((have-content (plist-member part :content)) - (charset (if have-content 'gnus-decoded + (charset (if have-content + 'gnus-decoded (plist-get part :content-charset))) - (handle (mm-make-handle (current-buffer) `(,content-type (charset . ,charset))))) + (handle (mm-make-handle (current-buffer) + `(,content-type (charset . ,charset))))) ;; If the user wants the part inlined, insert the content and ;; test whether we are able to inline it (which includes both ;; capability and suitability tests). @@ -677,14 +713,17 @@ current buffer, if possible." (when (mm-inlinable-p handle) (set-buffer display-buffer) (mm-display-part handle) + (plist-put part :undisplayer (mm-handle-undisplayer handle)) t)))))) +;;; Generic Utilities + ;; Converts a plist of headers to an alist of headers. The input plist should ;; have symbols of the form :Header as keys, and the resulting alist will have ;; symbols of the form 'Header as keys. (defun notmuch-headers-plist-to-alist (plist) - (loop for (key value . rest) on plist by #'cddr - collect (cons (intern (substring (symbol-name key) 1)) value))) + (cl-loop for (key value . rest) on plist by #'cddr + collect (cons (intern (substring (symbol-name key) 1)) value))) (defun notmuch-face-ensure-list-form (face) "Return FACE in face list form. @@ -697,7 +736,7 @@ single element face list." (list face))) (defun notmuch-apply-face (object face &optional below start end) - "Combine FACE into the 'face text property of OBJECT between START and END. + "Combine FACE into the \\='face text property of OBJECT between START and END. This function combines FACE with any existing faces between START and END in OBJECT. Attributes specified by FACE take precedence @@ -710,7 +749,6 @@ must be a face name (a symbol or string), a property list of face attributes, or a list of these. If START and/or END are omitted, they default to the beginning/end of OBJECT. For convenience when applied to strings, this returns OBJECT." - ;; A face property can have three forms: a face name (a string or ;; symbol), a property list, or a list of these two forms. In the ;; list case, the faces will be combined, with the earlier faces @@ -746,6 +784,8 @@ returned by FUNC." (put-text-property start next prop (funcall func value) object) (setq start next)))) +;;; Running Notmuch + (defun notmuch-logged-error (msg &optional extra) "Log MSG and EXTRA to *Notmuch errors* and signal MSG. @@ -753,7 +793,6 @@ This logs MSG and EXTRA to the *Notmuch errors* buffer and signals MSG as an error. If EXTRA is non-nil, text referring the user to the *Notmuch errors* buffer will be appended to the signaled error. This function does not return." - (with-current-buffer (get-buffer-create "*Notmuch errors*") (goto-char (point-max)) (unless (bobp) @@ -766,8 +805,7 @@ signaled error. This function does not return." (insert extra) (unless (bolp) (newline))))) - (error "%s" (concat msg (when extra - " (see *Notmuch errors* for more details)")))) + (error "%s%s" msg (if extra " (see *Notmuch errors* for more details)" ""))) (defun notmuch-check-async-exit-status (proc msg &optional command err) "If PROC exited abnormally, pop up an error buffer and signal an error. @@ -778,11 +816,12 @@ arguments passed to the sentinel. COMMAND and ERR, if provided, are passed to `notmuch-check-exit-status'. If COMMAND is not provided, it is taken from `process-command'." (let ((exit-status - (case (process-status proc) + (cl-case (process-status proc) ((exit) (process-exit-status proc)) ((signal) msg)))) (when exit-status - (notmuch-check-exit-status exit-status (or command (process-command proc)) + (notmuch-check-exit-status exit-status + (or command (process-command proc)) nil err)))) (defun notmuch-check-exit-status (exit-status command &optional output err) @@ -797,7 +836,6 @@ command and its arguments. OUTPUT, if provided, is a string giving the output of command. ERR, if provided, is the error output of command. OUTPUT and ERR will be included in the error message." - (cond ((eq exit-status 0) t) ((eq exit-status 20) @@ -809,33 +847,64 @@ You may need to restart Emacs or upgrade your notmuch Emacs package.")) Emacs requested a newer output format than supported by the notmuch CLI. You may need to restart Emacs or upgrade your notmuch package.")) (t - (let* ((command-string - (mapconcat (lambda (arg) - (shell-quote-argument - (cond ((stringp arg) arg) - ((symbolp arg) (symbol-name arg)) - (t "*UNKNOWN ARGUMENT*")))) - command " ")) - (extra - (concat "command: " command-string "\n" - (if (integerp exit-status) - (format "exit status: %s\n" exit-status) - (format "exit signal: %s\n" exit-status)) - (when err - (concat "stderr:\n" err)) - (when output - (concat "stdout:\n" output))))) - (if err - ;; We have an error message straight from the CLI. - (notmuch-logged-error - (replace-regexp-in-string "[ \n\r\t\f]*\\'" "" err) extra) - ;; We only have combined output from the CLI; don't inundate - ;; the user with it. Mimic `process-lines'. - (notmuch-logged-error (format "%s exited with status %s" - (car command) exit-status) - extra)) - ;; `notmuch-logged-error' does not return. - )))) + (pcase-let* + ((`(,command . ,args) command) + (command (if (equal (file-name-nondirectory command) + notmuch-command) + notmuch-command + command)) + (command-string + (mapconcat (lambda (arg) + (shell-quote-argument + (cond ((stringp arg) arg) + ((symbolp arg) (symbol-name arg)) + (t "*UNKNOWN ARGUMENT*")))) + (cons command args) + " ")) + (extra + (concat "command: " command-string "\n" + (if (integerp exit-status) + (format "exit status: %s\n" exit-status) + (format "exit signal: %s\n" exit-status)) + (and err (concat "stderr:\n" err)) + (and output (concat "stdout:\n" output))))) + (if err + ;; We have an error message straight from the CLI. + (notmuch-logged-error + (replace-regexp-in-string "[ \n\r\t\f]*\\'" "" err) extra) + ;; We only have combined output from the CLI; don't inundate + ;; the user with it. Mimic `process-lines'. + (notmuch-logged-error (format "%s exited with status %s" + command exit-status) + extra)) + ;; `notmuch-logged-error' does not return. + )))) + +(defmacro notmuch--apply-with-env (func &rest args) + `(let ((default-directory "~")) + (apply ,func ,@args))) + +(defun notmuch--process-lines (program &rest args) + "Wrap process-lines, binding DEFAULT-DIRECTORY to a safe +default" + (notmuch--apply-with-env #'process-lines program args)) + +(defun notmuch--make-process (&rest args) + "Wrap make-process, binding DEFAULT-DIRECTORY to a safe +default" + (notmuch--apply-with-env #'make-process args)) + +(defun notmuch--call-process-region (start end program + &optional delete buffer display + &rest args) + "Wrap call-process-region, binding DEFAULT-DIRECTORY to a safe +default" + (notmuch--apply-with-env + #'call-process-region start end program delete buffer display args)) + +(defun notmuch--call-process (program &optional infile destination display &rest args) + "Wrap call-process, binding DEFAULT-DIRECTORY to a safe default" + (notmuch--apply-with-env #'call-process program infile destination display args)) (defun notmuch-call-notmuch--helper (destination args) "Helper for synchronous notmuch invocation commands. @@ -843,18 +912,17 @@ You may need to restart Emacs or upgrade your notmuch package.")) This wraps `call-process'. DESTINATION has the same meaning as for `call-process'. ARGS is as described for `notmuch-call-notmuch-process'." - (let (stdin-string) (while (keywordp (car args)) - (case (car args) - (:stdin-string (setq stdin-string (cadr args) - args (cddr args))) + (cl-case (car args) + (:stdin-string (setq stdin-string (cadr args)) + (setq args (cddr args))) (otherwise (error "Unknown keyword argument: %s" (car args))))) (if (null stdin-string) - (apply #'call-process notmuch-command nil destination nil args) + (apply #'notmuch--call-process notmuch-command nil destination nil args) (insert stdin-string) - (apply #'call-process-region (point-min) (point-max) + (apply #'notmuch--call-process-region (point-min) (point-max) notmuch-command t destination nil args)))) (defun notmuch-call-notmuch-process (&rest args) @@ -881,7 +949,6 @@ notmuch's output as an S-expression and returns the parsed value. Like `notmuch-call-notmuch-process', if notmuch exits with a non-zero status, this will report its output and signal an error." - (with-temp-buffer (let ((err-file (make-temp-file "nmerr"))) (unwind-protect @@ -909,59 +976,29 @@ when the process exits, or nil for none. The caller must *not* invoke `set-process-sentinel' directly on the returned process, as that will interfere with the handling of stderr and the exit status." - - (let (err-file err-buffer proc err-proc - ;; Find notmuch using Emacs' `exec-path' - (command (or (executable-find notmuch-command) - (error "Command not found: %s" notmuch-command)))) - (if (fboundp 'make-process) - (progn - (setq err-buffer (generate-new-buffer " *notmuch-stderr*")) - ;; Emacs 25 and newer has `make-process', which allows - ;; redirecting stderr independently from stdout to a - ;; separate buffer. As this allows us to avoid using a - ;; temporary file and shell invocation, use it when - ;; available. - (setq proc (make-process - :name name - :buffer buffer - :command (cons command args) - :connection-type 'pipe - :stderr err-buffer) - err-proc (get-buffer-process err-buffer)) - (process-put proc 'err-buffer err-buffer) - - (process-put err-proc 'err-file err-file) - (process-put err-proc 'err-buffer err-buffer) - (set-process-sentinel err-proc #'notmuch-start-notmuch-error-sentinel)) - - ;; On Emacs versions before 25, there is no way to capture - ;; stdout and stderr separately for asynchronous processes, or - ;; even to redirect stderr to a file, so we use a trivial shell - ;; wrapper to send stderr to a temporary file and clean things - ;; up in the sentinel. - (setq err-file (make-temp-file "nmerr")) - (let ((process-connection-type nil)) ;; Use a pipe - (setq proc (apply #'start-process name buffer - "/bin/sh" "-c" - "exec 2>\"$1\"; shift; exec \"$0\" \"$@\"" - command err-file args))) - (process-put proc 'err-file err-file)) - + (let* ((command (or (executable-find notmuch-command) + (error "Command not found: %s" notmuch-command))) + (err-buffer (generate-new-buffer " *notmuch-stderr*")) + (proc (notmuch--make-process + :name name + :buffer buffer + :command (cons command args) + :connection-type 'pipe + :stderr err-buffer)) + (err-proc (get-buffer-process err-buffer))) + (process-put proc 'err-buffer err-buffer) (process-put proc 'sub-sentinel sentinel) - (process-put proc 'real-command (cons notmuch-command args)) (set-process-sentinel proc #'notmuch-start-notmuch-sentinel) + (set-process-sentinel err-proc #'notmuch-start-notmuch-error-sentinel) proc)) (defun notmuch-start-notmuch-sentinel (proc event) "Process sentinel function used by `notmuch-start-notmuch'." - (let* ((err-file (process-get proc 'err-file)) - (err-buffer (or (process-get proc 'err-buffer) - (find-file-noselect err-file))) - (err (when (not (zerop (buffer-size err-buffer))) - (with-current-buffer err-buffer (buffer-string)))) - (sub-sentinel (process-get proc 'sub-sentinel)) - (real-command (process-get proc 'real-command))) + (let* ((err-buffer (process-get proc 'err-buffer)) + (err (and (buffer-live-p err-buffer) + (not (zerop (buffer-size err-buffer))) + (with-current-buffer err-buffer (buffer-string)))) + (sub-sentinel (process-get proc 'sub-sentinel))) (condition-case err (progn ;; Invoke the sub-sentinel, if any @@ -973,40 +1010,51 @@ status." ;; and there's no point in telling the user that (but we ;; still check for and report stderr output below). (when (buffer-live-p (process-buffer proc)) - (notmuch-check-async-exit-status proc event real-command err)) + (notmuch-check-async-exit-status proc event nil err)) ;; If that didn't signal an error, then any error output was ;; really warning output. Show warnings, if any. (let ((warnings - (when err - (with-current-buffer err-buffer - (goto-char (point-min)) - (end-of-line) - ;; Show first line; stuff remaining lines in the - ;; errors buffer. - (let ((l1 (buffer-substring (point-min) (point)))) - (skip-chars-forward "\n") - (cons l1 (unless (eobp) - (buffer-substring (point) (point-max))))))))) + (and err + (with-current-buffer err-buffer + (goto-char (point-min)) + (end-of-line) + ;; Show first line; stuff remaining lines in the + ;; errors buffer. + (let ((l1 (buffer-substring (point-min) (point)))) + (skip-chars-forward "\n") + (cons l1 (and (not (eobp)) + (buffer-substring (point) + (point-max))))))))) (when warnings (notmuch-logged-error (car warnings) (cdr warnings))))) (error ;; Emacs behaves strangely if an error escapes from a sentinel, ;; so turn errors into messages. - (message "%s" (error-message-string err)))) - (when err-file (ignore-errors (delete-file err-file))))) - -(defun notmuch-start-notmuch-error-sentinel (proc event) - (let* ((err-file (process-get proc 'err-file)) - ;; When `make-process' is available, use the error buffer - ;; associated with the process, otherwise the error file. - (err-buffer (or (process-get proc 'err-buffer) - (find-file-noselect err-file)))) - (when err-buffer (kill-buffer err-buffer)))) - -;; This variable is used only buffer local, but it needs to be -;; declared globally first to avoid compiler warnings. -(defvar notmuch-show-process-crypto nil) -(make-variable-buffer-local 'notmuch-show-process-crypto) + (message "%s" (error-message-string err)))))) + +(defun notmuch-start-notmuch-error-sentinel (proc _event) + (unless (process-live-p proc) + (let ((buffer (process-buffer proc))) + (when (buffer-live-p buffer) + (kill-buffer buffer))))) + +(defvar-local notmuch-show-process-crypto nil) + +(defun notmuch--run-show (search-terms &optional duplicate) + "Return a list of threads of messages matching SEARCH-TERMS. + +A thread is a forest or list of trees. A tree is a two element +list where the first element is a message, and the second element +is a possibly empty forest of replies." + (let ((args '("show" "--format=sexp" "--format-version=5"))) + (when notmuch-show-process-crypto + (setq args (append args '("--decrypt=true")))) + (when duplicate + (setq args (append args (list (format "--duplicate=%d" duplicate))))) + (setq args (append args search-terms)) + (apply #'notmuch-call-notmuch-sexp args))) + +;;; Generic Utilities (defun notmuch-interactive-region () "Return the bounds of the current interactive region. @@ -1018,14 +1066,20 @@ region if the region is active, or both `point' otherwise." (list (point) (point)))) (define-obsolete-function-alias - 'notmuch-search-interactive-region - 'notmuch-interactive-region + 'notmuch-search-interactive-region + 'notmuch-interactive-region "notmuch 0.29") -(provide 'notmuch-lib) +(defun notmuch--inline-override-types () + "Override mm-inline-override-types to stop application/* +parts from being displayed unless the user has customized +it themselves." + (if (equal mm-inline-override-types + (eval (car (get 'mm-inline-override-types 'standard-value)))) + (cons "application/.*" mm-inline-override-types) + mm-inline-override-types)) +;;; _ -;; Local Variables: -;; byte-compile-warnings: (not cl-functions) -;; End: +(provide 'notmuch-lib) ;;; notmuch-lib.el ends here diff --git a/emacs/notmuch-logo.png b/emacs/notmuch-logo.png deleted file mode 100644 index 53b5e6a4..00000000 Binary files a/emacs/notmuch-logo.png and /dev/null differ diff --git a/emacs/notmuch-logo.svg b/emacs/notmuch-logo.svg new file mode 100644 index 00000000..2c65a73b --- /dev/null +++ b/emacs/notmuch-logo.svg @@ -0,0 +1,27 @@ + + + + + + + + + + + + + + + + + + + + + + diff --git a/emacs/notmuch-maildir-fcc.el b/emacs/notmuch-maildir-fcc.el index ae56bacd..51020788 100644 --- a/emacs/notmuch-maildir-fcc.el +++ b/emacs/notmuch-maildir-fcc.el @@ -1,36 +1,38 @@ -;;; notmuch-maildir-fcc.el --- - -;; This file is free software; you can redistribute it and/or modify -;; it under the terms of the GNU General Public License as published -;; by the Free Software Foundation; either version 2, or (at your -;; option) any later version. - -;; This program is distributed in the hope that it will be useful, -;; but WITHOUT ANY WARRANTY; without even the implied warranty of -;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -;; GNU General Public License for more details. - +;;; notmuch-maildir-fcc.el --- inserting using a fcc handler -*- lexical-binding: t -*- + +;; Copyright © Jesse Rosenthal +;; +;; This file is part of Notmuch. +;; +;; Notmuch is free software: you can redistribute it and/or modify it +;; under the terms of the GNU General Public License as published by +;; the Free Software Foundation, either version 3 of the License, or +;; (at your option) any later version. +;; +;; Notmuch is distributed in the hope that it will be useful, but +;; WITHOUT ANY WARRANTY; without even the implied warranty of +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +;; General Public License for more details. +;; ;; You should have received a copy of the GNU General Public License -;; along with GNU Emacs; see the file COPYING. If not, write to the -;; Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, -;; Boston, MA 02110-1301, USA. - -;;; Commentary: - -;; To use this as the fcc handler for message-mode, -;; customize the notmuch-fcc-dirs variable +;; along with Notmuch. If not, see . +;; +;; Authors: Jesse Rosenthal ;;; Code: -(eval-when-compile (require 'cl)) +(require 'seq) + (require 'message) (require 'notmuch-lib) (defvar notmuch-maildir-fcc-count 0) +;;; Options + (defcustom notmuch-fcc-dirs "sent" - "Determines the Fcc Header which says where to save outgoing mail. + "Determines the Fcc Header which says where to save outgoing mail. Three types of values are permitted: @@ -39,16 +41,17 @@ Three types of values are permitted: - a string: the value of `notmuch-fcc-dirs' is the Fcc header to be used. -- a list: the folder is chosen based on the From address of the - current message using a list of regular expressions and - corresponding folders: +- an alist: the folder is chosen based on the From address of + the current message according to an alist mapping regular + expressions to folders or nil: ((\"Sebastian@SSpaeth.de\" . \"privat\") (\"spaetz@sspaeth.de\" . \"OUTBOX.OSS\") (\".*\" . \"defaultinbox\")) - If none of the regular expressions match the From address, no - Fcc header will be added. + If none of the regular expressions match the From address, or + if the cdr of the matching entry is nil, then no Fcc header + will be added. If `notmuch-maildir-use-notmuch-insert' is set (the default) then the header should be of the form \"folder +tag1 -tag2\" where @@ -68,62 +71,52 @@ database.path option in the notmuch configuration file). In all cases you will be prompted to create the folder or directory if it does not exist yet when sending a mail." - :type '(choice - (const :tag "No FCC header" nil) - (string :tag "A single folder") - (repeat :tag "A folder based on the From header" - (cons regexp (string :tag "Folder")))) - :require 'notmuch-fcc-initialization - :group 'notmuch-send) + :type '(choice + (const :tag "No FCC header" nil) + (string :tag "A single folder") + (repeat :tag "A folder based on the From header" + (cons regexp (choice (const :tag "No FCC header" nil) + (string :tag "Folder"))))) + :require 'notmuch-fcc-initialization + :group 'notmuch-send) -(defcustom notmuch-maildir-use-notmuch-insert 't - "Should fcc use notmuch insert instead of simple fcc" +(defcustom notmuch-maildir-use-notmuch-insert t + "Should fcc use notmuch insert instead of simple fcc." :type '(choice :tag "Fcc Method" (const :tag "Use notmuch insert" t) (const :tag "Use simple fcc" nil)) :group 'notmuch-send) -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Functions which set up the fcc header in the message buffer. +;;; Functions which set up the fcc header in the message buffer. (defun notmuch-fcc-header-setup () "Add an Fcc header to the current message buffer. -Sets the Fcc header based on the values of `notmuch-fcc-dirs'. - -Originally intended to be use a hook function, but now called directly -by notmuch-mua-mail" - +If the Fcc header is already set, then keep it as-is. +Otherwise set it according to `notmuch-fcc-dirs'." (let ((subdir (cond ((or (not notmuch-fcc-dirs) (message-field-value "Fcc")) ;; Nothing set or an existing header. nil) - ((stringp notmuch-fcc-dirs) notmuch-fcc-dirs) - ((and (listp notmuch-fcc-dirs) (stringp (car notmuch-fcc-dirs))) ;; Old style - no longer works. (error "Invalid `notmuch-fcc-dirs' setting (old style)")) - ((listp notmuch-fcc-dirs) - (let* ((from (message-field-value "From")) - (match - (catch 'first-match - (dolist (re-folder notmuch-fcc-dirs) - (when (string-match-p (car re-folder) from) - (throw 'first-match re-folder)))))) - (if match - (cdr match) - (message "No Fcc header added.") - nil))) - + (if-let ((match (seq-some (let ((from (message-field-value "From"))) + (pcase-lambda (`(,regexp . ,folder)) + (and (string-match-p regexp from) + (cons t folder)))) + notmuch-fcc-dirs))) + (cdr match) + (message "No Fcc header added.") + nil)) (t (error "Invalid `notmuch-fcc-dirs' setting (neither string nor list)"))))) - (when subdir (if notmuch-maildir-use-notmuch-insert (notmuch-maildir-add-notmuch-insert-style-fcc-header subdir) @@ -132,10 +125,10 @@ by notmuch-mua-mail" (defun notmuch-maildir-add-notmuch-insert-style-fcc-header (subdir) ;; Notmuch insert does not accept absolute paths, so check the user ;; really want this header inserted. - (when (or (not (= (elt subdir 0) ?/)) - (y-or-n-p (format "Fcc header %s is an absolute path and notmuch insert is requested.\nInsert header anyway? " - subdir))) + (y-or-n-p (format "Fcc header %s is an absolute path %s %s" subdir + "and notmuch insert is requested." + "Insert header anyway? "))) (message-add-header (concat "Fcc: " subdir)))) (defun notmuch-maildir-add-file-style-fcc-header (subdir) @@ -148,9 +141,7 @@ by notmuch-mua-mail" subdir (concat (notmuch-database-path) "/" subdir)))))) -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Functions for saving a message either using notmuch insert or file -;; fcc. First functions common to the two cases. +;;; Functions for saving a message using either method. (defmacro with-temporary-notmuch-message-buffer (&rest body) "Set-up a temporary copy of the current message-mode buffer." @@ -158,15 +149,20 @@ by notmuch-mua-mail" (buf (current-buffer)) (mml-externalize-attachments message-fcc-externalize-attachments)) (with-current-buffer (get-buffer-create " *message temp*") + (message-clone-locals buf) ;; for message-encoded-mail-cache (erase-buffer) (insert-buffer-substring buf) ,@body))) (defun notmuch-maildir-setup-message-for-saving () - "Setup message for saving. Should be called on a temporary copy. + "Setup message for saving. +This should be called on a temporary copy. This is taken from the function message-do-fcc." - (message-encode-message-body) + (if (not message-encoded-mail-cache) + (message-encode-message-body) + (erase-buffer) + (insert message-encoded-mail-cache)) (save-restriction (message-narrow-to-headers) (mail-encode-encoded-word-buffer)) @@ -180,38 +176,36 @@ This is taken from the function message-do-fcc." "Process Fcc headers in the current buffer. This is a rearranged version of message mode's message-do-fcc." - (let (list file) + (let (files file) (save-excursion (save-restriction (message-narrow-to-headers) (setq file (message-fetch-field "fcc" t))) (when file (with-temporary-notmuch-message-buffer + (notmuch-maildir-setup-message-for-saving) (save-restriction (message-narrow-to-headers) (while (setq file (message-fetch-field "fcc" t)) - (push file list) + (push file files) (message-remove-header "fcc" nil t))) - (notmuch-maildir-setup-message-for-saving) ;; Process FCC operations. - (while list - (setq file (pop list)) - (notmuch-fcc-handler file)) + (mapc #'notmuch-fcc-handler files) (kill-buffer (current-buffer))))))) (defun notmuch-fcc-handler (fcc-header) "Store message with notmuch insert or normal (file) fcc. -If `notmuch-maildir-use-notmuch-insert` is set then store the +If `notmuch-maildir-use-notmuch-insert' is set then store the message using notmuch insert. Otherwise store the message using normal fcc." (message "Doing Fcc...") (if notmuch-maildir-use-notmuch-insert (notmuch-maildir-fcc-with-notmuch-insert fcc-header) - (notmuch-maildir-fcc-file-fcc fcc-header))) + (notmuch-maildir-fcc-file-fcc fcc-header)) + (message "Doing Fcc...done")) -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Functions for saving a message using notmuch insert. +;;; Functions for saving a message using notmuch insert. (defun notmuch-maildir-notmuch-insert-current-buffer (folder &optional create tags) "Use notmuch insert to put the current buffer in the database. @@ -220,11 +214,11 @@ This inserts the current buffer as a message into the notmuch database in folder FOLDER. If CREATE is non-nil it will supply the --create-folder flag to create the folder if necessary. TAGS should be a list of tag changes to apply to the inserted message." - (let* ((args (append (when create (list "--create-folder")) - (list (concat "--folder=" folder)) - tags))) - (apply 'notmuch-call-notmuch-process - :stdin-string (buffer-string) "insert" args))) + (apply 'notmuch-call-notmuch-process + :stdin-string (buffer-string) "insert" + (append (and create (list "--create-folder")) + (list (concat "--folder=" folder)) + tags))) (defun notmuch-maildir-fcc-with-notmuch-insert (fcc-header &optional create) "Store message with notmuch insert. @@ -238,9 +232,8 @@ quoting each space with an immediately preceding backslash or surrounding the entire folder name in double quotes. If CREATE is non-nil then create the folder if necessary." - (let* ((args (split-string-and-unquote fcc-header)) - (folder (car args)) - (tags (cdr args))) + (pcase-let ((`(,folder . ,tags) + (split-string-and-unquote fcc-header))) (condition-case nil (notmuch-maildir-notmuch-insert-current-buffer folder create tags) ;; Since there are many reasons notmuch insert could fail, e.g., @@ -248,19 +241,16 @@ If CREATE is non-nil then create the folder if necessary." ;; typo, or just the user want a new folder, let the user decide ;; how to deal with it. (error - (let ((response (notmuch-read-char-choice - "Insert failed: (r)etry, (c)reate folder, (i)gnore, or (e)dit the header? " - '(?r ?c ?i ?e)))) - (case response - (?r (notmuch-maildir-fcc-with-notmuch-insert fcc-header)) - (?c (notmuch-maildir-fcc-with-notmuch-insert fcc-header 't)) - (?i 't) - (?e (notmuch-maildir-fcc-with-notmuch-insert - (read-from-minibuffer "Fcc header: " fcc-header))))))))) - + (let ((response (read-char-choice "Insert failed: \ +\(r)etry, (c)reate folder, (i)gnore, or (e)dit the header? " '(?r ?c ?i ?e)))) + (cl-case response + (?r (notmuch-maildir-fcc-with-notmuch-insert fcc-header)) + (?c (notmuch-maildir-fcc-with-notmuch-insert fcc-header t)) + (?i t) + (?e (notmuch-maildir-fcc-with-notmuch-insert + (read-from-minibuffer "Fcc header: " fcc-header))))))))) -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Functions for saving a message using file fcc. +;;; Functions for saving a message using file fcc. (defun notmuch-maildir-fcc-host-fixer (hostname) (replace-regexp-in-string "/\\|:" @@ -273,16 +263,16 @@ If CREATE is non-nil then create the folder if necessary." t)) (defun notmuch-maildir-fcc-make-uniq-maildir-id () - (let* ((ftime (float-time)) - (microseconds (mod (* 1000000 ftime) 1000000)) - (hostname (notmuch-maildir-fcc-host-fixer (system-name)))) - (setq notmuch-maildir-fcc-count (+ notmuch-maildir-fcc-count 1)) - (format "%d.%d_%d_%d.%s" - ftime - (emacs-pid) - microseconds - notmuch-maildir-fcc-count - hostname))) + (let* ((ftime (float-time)) + (microseconds (mod (* 1000000 ftime) 1000000)) + (hostname (notmuch-maildir-fcc-host-fixer (system-name)))) + (cl-incf notmuch-maildir-fcc-count) + (format "%d.%d_%d_%d.%s" + ftime + (emacs-pid) + microseconds + notmuch-maildir-fcc-count + hostname))) (defun notmuch-maildir-fcc-dir-is-maildir-p (dir) (and (file-exists-p (concat dir "/cur/")) @@ -309,9 +299,7 @@ if successful, nil if not." (write-file (concat destdir "/tmp/" msg-id)) msg-id) (t - (error (format "Can't write to %s. Not a maildir." - destdir)) - nil)))) + (error "Can't write to %s. Not a maildir." destdir))))) (defun notmuch-maildir-fcc-move-tmp-to-new (destdir msg-id) (add-name-to-file @@ -321,54 +309,54 @@ if successful, nil if not." (defun notmuch-maildir-fcc-move-tmp-to-cur (destdir msg-id &optional mark-seen) (add-name-to-file (concat destdir "/tmp/" msg-id) - (concat destdir "/cur/" msg-id ":2," (when mark-seen "S")))) + (concat destdir "/cur/" msg-id ":2," (and mark-seen "S")))) (defun notmuch-maildir-fcc-file-fcc (fcc-header) "Write the message to the file specified by FCC-HEADER. -It offers the user a chance to correct the header, or filesystem, -if needed." +If that fails, then offer the user a chance to correct the header +or filesystem." (if (notmuch-maildir-fcc-dir-is-maildir-p fcc-header) - (notmuch-maildir-fcc-write-buffer-to-maildir fcc-header 't) + (notmuch-maildir-fcc-write-buffer-to-maildir fcc-header t) ;; The fcc-header is not a valid maildir see if the user wants to ;; fix it in some way. - (let* ((prompt (format "Fcc %s is not a maildir: (r)etry, (c)reate folder, (i)gnore, or (e)dit the header? " - fcc-header)) - (response (notmuch-read-char-choice prompt '(?r ?c ?i ?e)))) - (case response - (?r (notmuch-maildir-fcc-file-fcc fcc-header)) - (?c (if (file-writable-p fcc-header) - (notmuch-maildir-fcc-create-maildir fcc-header) - (message "No permission to create %s." fcc-header) - (sit-for 2)) - (notmuch-maildir-fcc-file-fcc fcc-header)) - (?i 't) - (?e (notmuch-maildir-fcc-file-fcc - (read-from-minibuffer "Fcc header: " fcc-header))))))) + (let* ((prompt (format "Fcc %s is not a maildir: \ +\(r)etry, (c)reate folder, (i)gnore, or (e)dit the header? " fcc-header)) + (response (read-char-choice prompt '(?r ?c ?i ?e)))) + (cl-case response + (?r (notmuch-maildir-fcc-file-fcc fcc-header)) + (?c (if (file-writable-p fcc-header) + (notmuch-maildir-fcc-create-maildir fcc-header) + (message "No permission to create %s." fcc-header) + (sit-for 2)) + (notmuch-maildir-fcc-file-fcc fcc-header)) + (?i t) + (?e (notmuch-maildir-fcc-file-fcc + (read-from-minibuffer "Fcc header: " fcc-header))))))) (defun notmuch-maildir-fcc-write-buffer-to-maildir (destdir &optional mark-seen) - "Writes the current buffer to maildir destdir. If mark-seen is -non-nil, it will write it to cur/, and mark it as read. It should -return t if successful, and nil otherwise." + "Write the current buffer to maildir destdir. + +If mark-seen is non-nil, then write it to \"cur/\", and mark it +as read, otherwise write it to \"new/\". Return t if successful, +and nil otherwise." (let ((orig-buffer (buffer-name))) (with-temp-buffer (insert-buffer-substring orig-buffer) (catch 'link-error (let ((msg-id (notmuch-maildir-fcc-save-buffer-to-tmp destdir))) (when msg-id - (cond (mark-seen - (condition-case err - (notmuch-maildir-fcc-move-tmp-to-cur destdir msg-id t) - (file-already-exists - (throw 'link-error nil)))) - (t - (condition-case err - (notmuch-maildir-fcc-move-tmp-to-new destdir msg-id) - (file-already-exists - (throw 'link-error nil)))))) + (condition-case nil + (if mark-seen + (notmuch-maildir-fcc-move-tmp-to-cur destdir msg-id t) + (notmuch-maildir-fcc-move-tmp-to-new destdir msg-id)) + (file-already-exists + (throw 'link-error nil)))) (delete-file (concat destdir "/tmp/" msg-id)))) t))) +;;; _ + (provide 'notmuch-maildir-fcc) ;;; notmuch-maildir-fcc.el ends here diff --git a/emacs/notmuch-message.el b/emacs/notmuch-message.el index 0164472f..0856a2e9 100644 --- a/emacs/notmuch-message.el +++ b/emacs/notmuch-message.el @@ -1,4 +1,4 @@ -;;; notmuch-message.el --- message-mode functions specific to notmuch +;;; notmuch-message.el --- message-mode functions specific to notmuch -*- lexical-binding: t -*- ;; ;; Copyright © Jesse Rosenthal ;; @@ -21,6 +21,10 @@ ;;; Code: +(require 'cl-lib) +(require 'pcase) +(require 'subr-x) + (require 'message) (require 'notmuch-tag) @@ -50,20 +54,20 @@ the \"inbox\" tag, you would set: :type '(repeat string) :group 'notmuch-send) -(defconst notmuch-message-queued-tag-changes nil - "List of messages and corresponding tag-changes to be applied when sending a message. +(defvar-local notmuch-message-queued-tag-changes nil + "List of tag changes to be applied when sending a message. -This variable is overridden by buffer-local versions in message -buffers where tag changes should be triggered when sending off -the message. Each item in this list is a list of strings, where -the first is a notmuch query and the rest are the tag changes to -be applied to the matching messages.") +A list of queries and tag changes that are to be applied to them +when the message that was composed in the current buffer is being +send. Each item in this list is a list of strings, where the +first is a notmuch query and the rest are the tag changes to be +applied to the matching messages.") (defun notmuch-message-apply-queued-tag-changes () - ;; Apply the tag changes queued in the buffer-local variable notmuch-message-queued-tag-changes. - (dolist (query-and-tags notmuch-message-queued-tag-changes) - (notmuch-tag (car query-and-tags) - (cdr query-and-tags)))) + ;; Apply the tag changes queued in the buffer-local variable + ;; notmuch-message-queued-tag-changes. + (pcase-dolist (`(,query . ,tags) notmuch-message-queued-tag-changes) + (notmuch-tag query tags))) (add-hook 'message-send-hook 'notmuch-message-apply-queued-tag-changes) diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el index 7fdd76bc..bf62b656 100644 --- a/emacs/notmuch-mua.el +++ b/emacs/notmuch-mua.el @@ -1,4 +1,4 @@ -;;; notmuch-mua.el --- emacs style mail-user-agent +;;; notmuch-mua.el --- emacs style mail-user-agent -*- lexical-binding: t -*- ;; ;; Copyright © David Edmondson ;; @@ -21,7 +21,10 @@ ;;; Code: +(eval-when-compile (require 'subr-x)) + (require 'message) +(require 'gmm-utils) (require 'mm-view) (require 'format-spec) @@ -30,25 +33,27 @@ (require 'notmuch-draft) (require 'notmuch-message) -(eval-when-compile (require 'cl)) - (declare-function notmuch-show-insert-body "notmuch-show" (msg body depth)) (declare-function notmuch-fcc-header-setup "notmuch-maildir-fcc" ()) (declare-function notmuch-maildir-message-do-fcc "notmuch-maildir-fcc" ()) (declare-function notmuch-draft-postpone "notmuch-draft" ()) (declare-function notmuch-draft-save "notmuch-draft" ()) -;; +(defvar notmuch-show-indent-multipart) +(defvar notmuch-show-insert-header-p-function) +(defvar notmuch-show-max-text-part-size) +(defvar notmuch-show-insert-text/plain-hook) -(defcustom notmuch-mua-send-hook '(notmuch-mua-message-send-hook) +;;; Options + +(defcustom notmuch-mua-send-hook nil "Hook run before sending messages." :type 'hook :group 'notmuch-send :group 'notmuch-hooks) (defcustom notmuch-mua-compose-in 'current-window - (concat - "Where to create the mail buffer used to compose a new message. + "Where to create the mail buffer used to compose a new message. Possible values are `current-window' (default), `new-window' and `new-frame'. If set to `current-window', the mail buffer will be displayed in the current window, so the old buffer will be @@ -57,18 +62,14 @@ or `new-frame', the mail buffer will be displayed in a new window/frame that will be destroyed when the buffer is killed. You may want to customize `message-kill-buffer-on-exit' accordingly." - (when (< emacs-major-version 24) - " Due to a known bug in Emacs 23, you should not set -this to `new-window' if `message-kill-buffer-on-exit' is -disabled: this would result in an incorrect behavior.")) :group 'notmuch-send :type '(choice (const :tag "Compose in the current window" current-window) (const :tag "Compose mail in a new window" new-window) (const :tag "Compose mail in a new frame" new-frame))) (defcustom notmuch-mua-user-agent-function nil - "Function used to generate a `User-Agent:' string. If this is -`nil' then no `User-Agent:' will be generated." + "Function used to generate a `User-Agent:' string. +If this is `nil' then no `User-Agent:' will be generated." :type '(choice (const :tag "No user agent string" nil) (const :tag "Full" notmuch-mua-user-agent-full) (const :tag "Notmuch" notmuch-mua-user-agent-notmuch) @@ -78,20 +79,36 @@ disabled: this would result in an incorrect behavior.")) :group 'notmuch-send) (defcustom notmuch-mua-hidden-headers nil - "Headers that are added to the `message-mode' hidden headers -list." + "Headers that are added to the `message-mode' hidden headers list." :type '(repeat string) :group 'notmuch-send) +(defcustom notmuch-identities nil + "Identities that can be used as the From: address when composing a new message. + +If this variable is left unset, then a list will be constructed from the +name and addresses configured in the notmuch configuration file." + :type '(repeat string) + :group 'notmuch-send) + +(defcustom notmuch-always-prompt-for-sender nil + "Always prompt for the From: address when composing or forwarding a message. + +This is not taken into account when replying to a message, because in that case +the From: header is already filled in by notmuch." + :type 'boolean + :group 'notmuch-send) + (defgroup notmuch-reply nil - "Replying to messages in notmuch" + "Replying to messages in notmuch." :group 'notmuch) (defcustom notmuch-mua-cite-function 'message-cite-original - "*Function for citing an original message. + "Function for citing an original message. + Predefined functions include `message-cite-original' and -`message-cite-original-without-signature'. -Note that these functions use `mail-citation-hook' if that is non-nil." +`message-cite-original-without-signature'. Note that these +functions use `mail-citation-hook' if that is non-nil." :type '(radio (function-item message-cite-original) (function-item message-cite-original-without-signature) (function-item sc-cite-original) @@ -106,13 +123,13 @@ Note that these functions use `mail-citation-hook' if that is non-nil." This function specifies which parts of a mime message with multiple parts get a header." :type '(radio (const :tag "No part headers" - notmuch-show-reply-insert-header-p-never) + notmuch-show-reply-insert-header-p-never) (const :tag "All except multipart/* and hidden parts" - notmuch-show-reply-insert-header-p-trimmed) + notmuch-show-reply-insert-header-p-trimmed) (const :tag "Only for included text parts" - notmuch-show-reply-insert-header-p-minimal) + notmuch-show-reply-insert-header-p-minimal) (const :tag "Exactly as in show view" - notmuch-show-insert-header-p) + notmuch-show-insert-header-p) (function :tag "Other")) :group 'notmuch-reply) @@ -125,29 +142,34 @@ to `notmuch-mua-send-hook'." :type 'regexp :group 'notmuch-send) -;; +;;; Various functions (defun notmuch-mua-attachment-check () - "Signal an error if the message text indicates that an -attachment is expected but no MML referencing an attachment is -found. + "Signal an error an attachement is expected but missing. + +Signal an error if the message text indicates that an attachment +is expected but no MML referencing an attachment is found. Typically this is added to `notmuch-mua-send-hook'." (when (and ;; When the message mentions attachment... (save-excursion (message-goto-body) - (loop while (re-search-forward notmuch-mua-attachment-regexp (point-max) t) - ;; For every instance of the "attachment" string - ;; found, examine the text properties. If the text - ;; has either a `face' or `syntax-table' property - ;; then it is quoted text and should *not* cause the - ;; user to be asked about a missing attachment. - if (let ((props (text-properties-at (match-beginning 0)))) - (not (or (memq 'syntax-table props) - (memq 'face props)))) - return t - finally return nil)) + ;; Limit search from reaching other possible parts of the message + (let ((search-limit (search-forward "\n<#" nil t))) + (message-goto-body) + (cl-loop while (re-search-forward notmuch-mua-attachment-regexp + search-limit t) + ;; For every instance of the "attachment" string + ;; found, examine the text properties. If the text + ;; has either a `face' or `syntax-table' property + ;; then it is quoted text and should *not* cause the + ;; user to be asked about a missing attachment. + if (let ((props (text-properties-at (match-beginning 0)))) + (not (or (memq 'syntax-table props) + (memq 'face props)))) + return t + finally return nil))) ;; ...but doesn't have a part with a filename... (save-excursion (message-goto-body) @@ -159,17 +181,14 @@ Typically this is added to `notmuch-mua-send-hook'." (defun notmuch-mua-get-switch-function () "Get a switch function according to `notmuch-mua-compose-in'." - (cond ((eq notmuch-mua-compose-in 'current-window) - 'switch-to-buffer) - ((eq notmuch-mua-compose-in 'new-window) - 'switch-to-buffer-other-window) - ((eq notmuch-mua-compose-in 'new-frame) - 'switch-to-buffer-other-frame) - (t (error "Invalid value for `notmuch-mua-compose-in'")))) + (pcase notmuch-mua-compose-in + ('current-window 'switch-to-buffer) + ('new-window 'switch-to-buffer-other-window) + ('new-frame 'switch-to-buffer-other-frame) + (_ (error "Invalid value for `notmuch-mua-compose-in'")))) (defun notmuch-mua-maybe-set-window-dedicated () - "Set the selected window as dedicated according to -`notmuch-mua-compose-in'." + "Set the selected window as dedicated according to `notmuch-mua-compose-in'." (when (or (eq notmuch-mua-compose-in 'new-frame) (eq notmuch-mua-compose-in 'new-window)) (set-window-dedicated-p (selected-window) t))) @@ -194,19 +213,20 @@ Typically this is added to `notmuch-mua-send-hook'." (defun notmuch-mua-add-more-hidden-headers () "Add some headers to the list that are hidden by default." (mapc (lambda (header) - (when (not (member header message-hidden-headers)) + (unless (member header message-hidden-headers) (push header message-hidden-headers))) notmuch-mua-hidden-headers)) (defun notmuch-mua-reply-crypto (parts) "Add mml sign-encrypt flag if any part of original message is encrypted." - (loop for part in parts - if (notmuch-match-content-type (plist-get part :content-type) "multipart/encrypted") - do (mml-secure-message-sign-encrypt) - else if (notmuch-match-content-type (plist-get part :content-type) "multipart/*") - do (notmuch-mua-reply-crypto (plist-get part :content)))) - -;; There is a bug in emacs 23's message.el that results in a newline + (cl-loop for part in parts + for type = (plist-get part :content-type) + if (notmuch-match-content-type type "multipart/encrypted") + do (mml-secure-message-sign-encrypt) + else if (notmuch-match-content-type type "multipart/*") + do (notmuch-mua-reply-crypto (plist-get part :content)))) + +;; There is a bug in Emacs' message.el that results in a newline ;; not being inserted after the References header, so the next header ;; is concatenated to the end of it. This function fixes the problem, ;; while guarding against the possibility that some current or future @@ -215,29 +235,27 @@ Typically this is added to `notmuch-mua-send-hook'." (funcall original-func header references) (unless (bolp) (insert "\n"))) -(defun notmuch-mua-reply (query-string &optional sender reply-all) - (let ((args '("reply" "--format=sexp" "--format-version=4")) - (process-crypto notmuch-show-process-crypto) - reply - original) +;;; Mua reply + +(defun notmuch-mua-reply (query-string &optional sender reply-all duplicate) + (let* ((duparg (and duplicate (list (format "--duplicate=%d" duplicate)))) + (args `("reply" "--format=sexp" "--format-version=5" ,@duparg)) + (process-crypto notmuch-show-process-crypto) + reply + original) (when process-crypto (setq args (append args '("--decrypt=true")))) - (if reply-all (setq args (append args '("--reply-to=all"))) (setq args (append args '("--reply-to=sender")))) (setq args (append args (list query-string))) - ;; Get the reply object as SEXP, and parse it into an elisp object. (setq reply (apply #'notmuch-call-notmuch-sexp args)) - ;; Extract the original message to simplify the following code. (setq original (plist-get reply :original)) - ;; Extract the headers of both the reply and the original message. (let* ((original-headers (plist-get original :headers)) (reply-headers (plist-get reply :reply-headers))) - ;; If sender is non-nil, set the From: header to its value. (when sender (plist-put reply-headers :From sender)) @@ -245,28 +263,27 @@ Typically this is added to `notmuch-mua-send-hook'." ;; Overlay the composition window on that being used to read ;; the original message. ((same-window-regexps '("\\*mail .*"))) - - ;; We modify message-header-format-alist to get around a bug in message.el. - ;; See the comment above on notmuch-mua-insert-references. + ;; We modify message-header-format-alist to get around + ;; a bug in message.el. See the comment above on + ;; notmuch-mua-insert-references. (let ((message-header-format-alist - (loop for pair in message-header-format-alist - if (eq (car pair) 'References) - collect (cons 'References - (apply-partially - 'notmuch-mua-insert-references - (cdr pair))) - else - collect pair))) + (cl-loop for pair in message-header-format-alist + if (eq (car pair) 'References) + collect (cons 'References + (apply-partially + 'notmuch-mua-insert-references + (cdr pair))) + else + collect pair))) (notmuch-mua-mail (plist-get reply-headers :To) (notmuch-sanitize (plist-get reply-headers :Subject)) (notmuch-headers-plist-to-alist reply-headers) nil (notmuch-mua-get-switch-function)))) - - ;; Create a buffer-local queue for tag changes triggered when sending the reply + ;; Create a buffer-local queue for tag changes triggered when + ;; sending the reply. (when notmuch-message-replied-tags - (setq-local notmuch-message-queued-tag-changes - (list (cons query-string notmuch-message-replied-tags)))) - + (setq notmuch-message-queued-tag-changes + (list (cons query-string notmuch-message-replied-tags)))) ;; Insert the message body - but put it in front of the signature ;; if one is present, and after any other content ;; message*setup-hooks may have added to the message body already. @@ -275,75 +292,80 @@ Typically this is added to `notmuch-mua-send-hook'." (narrow-to-region (point) (point-max)) (goto-char (point-max)) (if (re-search-backward message-signature-separator nil t) - (if message-signature-insert-empty-line - (forward-line -1)) + (when message-signature-insert-empty-line + (forward-line -1)) (goto-char (point-max)))) - (let ((from (plist-get original-headers :From)) (date (plist-get original-headers :Date)) (start (point))) - ;; notmuch-mua-cite-function constructs a citation line based ;; on the From and Date headers of the original message, which ;; are assumed to be in the buffer. (insert "From: " from "\n") (insert "Date: " date "\n\n") - - (insert (with-temp-buffer - (let - ;; Don't attempt to clean up messages, excerpt - ;; citations, etc. in the original message before - ;; quoting. - ((notmuch-show-insert-text/plain-hook nil) - ;; Don't omit long parts. - (notmuch-show-max-text-part-size 0) - ;; Insert headers for parts as appropriate for replying. - (notmuch-show-insert-header-p-function notmuch-mua-reply-insert-header-p-function) - ;; Ensure that any encrypted parts are - ;; decrypted during the generation of the reply - ;; text. - (notmuch-show-process-crypto process-crypto) - ;; Don't indent multipart sub-parts. - (notmuch-show-indent-multipart nil)) - ;; We don't want sigstatus buttons (an information leak and usually wrong anyway). - (letf (((symbol-function 'notmuch-crypto-insert-sigstatus-button) #'ignore) - ((symbol-function 'notmuch-crypto-insert-encstatus-button) #'ignore)) - (notmuch-show-insert-body original (plist-get original :body) 0) - (buffer-substring-no-properties (point-min) (point-max)))))) - + (insert + (with-temp-buffer + (let + ;; Don't attempt to clean up messages, excerpt + ;; citations, etc. in the original message before + ;; quoting. + ((notmuch-show-insert-text/plain-hook nil) + ;; Don't omit long parts. + (notmuch-show-max-text-part-size 0) + ;; Insert headers for parts as appropriate for replying. + (notmuch-show-insert-header-p-function + notmuch-mua-reply-insert-header-p-function) + ;; Ensure that any encrypted parts are + ;; decrypted during the generation of the reply + ;; text. + (notmuch-show-process-crypto process-crypto) + ;; Don't indent multipart sub-parts. + (notmuch-show-indent-multipart nil) + ;; Stop certain mime types from being inlined + (mm-inline-override-types (notmuch--inline-override-types))) + ;; We don't want sigstatus buttons (an information leak and usually wrong anyway). + (cl-letf (((symbol-function 'notmuch-crypto-insert-sigstatus-button) #'ignore) + ((symbol-function 'notmuch-crypto-insert-encstatus-button) #'ignore)) + (notmuch-show-insert-body original (plist-get original :body) 0) + (buffer-substring-no-properties (point-min) (point-max)))))) (set-mark (point)) (goto-char start) ;; Quote the original message according to the user's configured style. (funcall notmuch-mua-cite-function))) - ;; Crypto processing based crypto content of the original message (when process-crypto (notmuch-mua-reply-crypto (plist-get original :body)))) - ;; Push mark right before signature, if any. (message-goto-signature) (unless (eobp) (end-of-line -1)) (push-mark) - (message-goto-body) (set-buffer-modified-p nil)) +;;; Mode and keymap + +(defvar notmuch-message-mode-map + (let ((map (make-sparse-keymap))) + (define-key map [remap message-send-and-exit] #'notmuch-mua-send-and-exit) + (define-key map [remap message-send] #'notmuch-mua-send) + (define-key map (kbd "C-c C-p") #'notmuch-draft-postpone) + (define-key map (kbd "C-x C-s") #'notmuch-draft-save) + map) + "Keymap for `notmuch-message-mode'.") + (define-derived-mode notmuch-message-mode message-mode "Message[Notmuch]" - "Notmuch message composition mode. Mostly like `message-mode'" + "Notmuch message composition mode. Mostly like `message-mode'." (notmuch-address-setup)) (put 'notmuch-message-mode 'flyspell-mode-predicate 'mail-mode-flyspell-verify) -(define-key notmuch-message-mode-map (kbd "C-c C-c") #'notmuch-mua-send-and-exit) -(define-key notmuch-message-mode-map (kbd "C-c C-s") #'notmuch-mua-send) -(define-key notmuch-message-mode-map (kbd "C-c C-p") #'notmuch-draft-postpone) -(define-key notmuch-message-mode-map (kbd "C-x C-s") #'notmuch-draft-save) +;;; New messages (defun notmuch-mua-pop-to-buffer (name switch-function) - "Pop to buffer NAME, and warn if it already exists and is -modified. This function is notmuch addaptation of -`message-pop-to-buffer'." + "Pop to buffer NAME, and warn if it already exists and is modified. +Like `message-pop-to-buffer' but enable `notmuch-message-mode' +instead of `message-mode' and SWITCH-FUNCTION is mandatory." (let ((buffer (get-buffer name))) (if (and buffer (buffer-name buffer)) @@ -355,34 +377,49 @@ modified. This function is notmuch addaptation of (select-window window)) (funcall switch-function buffer) (set-buffer buffer)) - (when (and (buffer-modified-p) - (not (prog1 - (y-or-n-p - "Message already being composed; erase? ") - (message nil)))) - (error "Message being composed"))) + (when (buffer-modified-p) + (if (y-or-n-p "Message already being composed; erase? ") + (message nil) + (error "Message being composed")))) (funcall switch-function name) (set-buffer name)) (erase-buffer) (notmuch-message-mode))) -(defun notmuch-mua-mail (&optional to subject other-headers continue +(defun notmuch-mua--remove-dont-reply-to-names () + (when-let* ((nr (if (functionp message-dont-reply-to-names) + message-dont-reply-to-names + (gmm-regexp-concat message-dont-reply-to-names))) + (nr-filter + (if (functionp nr) + (lambda (mail) (and (not (funcall nr mail)) mail)) + (lambda (mail) (and (not (string-match-p nr mail)) mail))))) + (dolist (header '("To" "Cc")) + (when-let ((v (message-fetch-field header))) + (let* ((tokens (mapcar #'string-trim (message-tokenize-header v))) + (good-tokens (delq nil (mapcar nr-filter tokens))) + (addr (and good-tokens (mapconcat #'identity good-tokens ", ")))) + (message-replace-header header addr)))))) + +;;;#autoload +(defun notmuch-mua-mail (&optional to subject other-headers _continue switch-function yank-action send-actions - return-action &rest ignored) - "Invoke the notmuch mail composition window." + return-action &rest _ignored) + "Invoke the notmuch mail composition window. + +The position of point when the function returns differs depending +on the values of TO and SUBJECT. If both are non-nil, point is +moved to the message's body. If SUBJECT is nil but TO isn't, +point is moved to the \"Subject:\" header. Otherwise, point is +moved to the \"To:\" header." (interactive) - (when notmuch-mua-user-agent-function (let ((user-agent (funcall notmuch-mua-user-agent-function))) - (when (not (string= "" user-agent)) + (unless (string-empty-p user-agent) (push (cons 'User-Agent user-agent) other-headers)))) - - (unless (assq 'From other-headers) - (push (cons 'From (message-make-from - (notmuch-user-name) (notmuch-user-primary-email))) other-headers)) - (notmuch-mua-pop-to-buffer (message-buffer-name "mail" to) - (or switch-function (notmuch-mua-get-switch-function))) + (or switch-function + (notmuch-mua-get-switch-function))) (let ((headers (append ;; The following is copied from `message-mail' @@ -393,69 +430,43 @@ modified. This function is notmuch addaptation of ;; https://lists.gnu.org/archive/html/emacs-devel/2011-01/msg00337.html ;; We need to convert any string input, eg from rmail-start-mail. (dolist (h other-headers other-headers) - (if (stringp (car h)) (setcar h (intern (capitalize (car h)))))))) - (args (list yank-action send-actions)) + (when (stringp (car h)) + (setcar h (intern (capitalize (car h)))))))) ;; Cause `message-setup-1' to do things relevant for mail, ;; such as observe `message-default-mail-headers'. (message-this-is-mail t)) - ;; message-setup-1 in Emacs 23 does not accept return-action - ;; argument. Pass it only if it is supplied by the caller. This - ;; will never be the case when we're called by `compose-mail' in - ;; Emacs 23. - (when return-action (nconc args '(return-action))) - (apply 'message-setup-1 headers args)) + (unless (assq 'From headers) + (push (cons 'From (message-make-from + (notmuch-user-name) + (notmuch-user-primary-email))) + headers)) + (message-setup-1 headers yank-action send-actions return-action)) (notmuch-fcc-header-setup) + (notmuch-mua--remove-dont-reply-to-names) (message-sort-headers) (message-hide-headers) (set-buffer-modified-p nil) (notmuch-mua-maybe-set-window-dedicated) - - (message-goto-to)) - -(defcustom notmuch-identities nil - "Identities that can be used as the From: address when composing a new message. - -If this variable is left unset, then a list will be constructed from the -name and addresses configured in the notmuch configuration file." - :type '(repeat string) - :group 'notmuch-send) - -(defcustom notmuch-always-prompt-for-sender nil - "Always prompt for the From: address when composing or forwarding a message. - -This is not taken into account when replying to a message, because in that case -the From: header is already filled in by notmuch." - :type 'boolean - :group 'notmuch-send) + (cond + ((and to subject) (message-goto-body)) + (to (message-goto-subject)) + (t (message-goto-to)))) (defvar notmuch-mua-sender-history nil) -;; Workaround: Running `ido-completing-read' in emacs 23.1, 23.2 and 23.3 -;; without some explicit initialization fill freeze the operation. -;; Hence, we advice `ido-completing-read' to ensure required initialization -;; is done. -(if (and (= emacs-major-version 23) (< emacs-minor-version 4)) - (defadvice ido-completing-read (before notmuch-ido-mode-init activate) - (ido-init-completion-maps) - (add-hook 'minibuffer-setup-hook 'ido-minibuffer-setup) - (add-hook 'choose-completion-string-functions - 'ido-choose-completion-string) - (ad-disable-advice 'ido-completing-read 'before 'notmuch-ido-mode-init) - (ad-activate 'ido-completing-read))) - (defun notmuch-mua-prompt-for-sender () "Prompt for a sender from the user's configured identities." (if notmuch-identities - (ido-completing-read "Send mail from: " notmuch-identities - nil nil nil 'notmuch-mua-sender-history - (car notmuch-identities)) + (completing-read "Send mail from: " notmuch-identities + nil nil nil 'notmuch-mua-sender-history + (car notmuch-identities)) (let* ((name (notmuch-user-name)) (addrs (cons (notmuch-user-primary-email) (notmuch-user-other-email))) (address - (ido-completing-read (concat "Sender address for " name ": ") addrs - nil nil nil 'notmuch-mua-sender-history - (car addrs)))) + (completing-read (concat "Sender address for " name ": ") addrs + nil nil nil 'notmuch-mua-sender-history + (car addrs)))) (message-make-from name address)))) (put 'notmuch-mua-new-mail 'notmuch-prefix-doc "... and prompt for sender") @@ -466,8 +477,8 @@ If PROMPT-FOR-SENDER is non-nil, the user will be prompted for the From: address first." (interactive "P") (let ((other-headers - (when (or prompt-for-sender notmuch-always-prompt-for-sender) - (list (cons 'From (notmuch-mua-prompt-for-sender)))))) + (and (or prompt-for-sender notmuch-always-prompt-for-sender) + (list (cons 'From (notmuch-mua-prompt-for-sender)))))) (notmuch-mua-mail nil nil other-headers nil (notmuch-mua-get-switch-function)))) (defun notmuch-mua-new-forward-messages (messages &optional prompt-for-sender) @@ -476,16 +487,16 @@ the From: address first." If PROMPT-FOR-SENDER is non-nil, the user will be prompteed for the From: address." (let* ((other-headers - (when (or prompt-for-sender notmuch-always-prompt-for-sender) - (list (cons 'From (notmuch-mua-prompt-for-sender))))) - forward-subject ;; Comes from the first message and is - ;; applied later. - forward-references ;; List of accumulated message-references of forwarded messages - forward-queries) ;; List of corresponding message-query - + (and (or prompt-for-sender notmuch-always-prompt-for-sender) + (list (cons 'From (notmuch-mua-prompt-for-sender))))) + ;; Comes from the first message and is applied later. + forward-subject + ;; List of accumulated message-references of forwarded messages. + forward-references + ;; List of corresponding message-query. + forward-queries) ;; Generate the template for the outgoing message. (notmuch-mua-mail nil "" other-headers nil (notmuch-mua-get-switch-function)) - (save-excursion ;; Insert all of the forwarded messages. (mapc (lambda (id) @@ -495,7 +506,8 @@ the From: address." (with-current-buffer temp-buffer (erase-buffer) (let ((coding-system-for-read 'no-conversion)) - (call-process notmuch-command nil t nil "show" "--format=raw" id)) + (notmuch--call-process notmuch-command nil t nil + "show" "--format=raw" id)) ;; Because we process the messages in reverse order, ;; always generate a forwarded subject, then use the ;; last (i.e. first) one. @@ -510,7 +522,6 @@ the From: address." ;; `message-forward-make-body' always puts the message at ;; the top, so do them in reverse order. (reverse messages)) - ;; Add in the appropriate subject. (save-restriction (message-narrow-to-headers) @@ -519,46 +530,44 @@ the From: address." (message-remove-header "References") (message-add-header (concat "References: " (mapconcat 'identity forward-references " ")))) - - ;; Create a buffer-local queue for tag changes triggered when sending the message + ;; Create a buffer-local queue for tag changes triggered when + ;; sending the message. (when notmuch-message-forwarded-tags - (setq-local notmuch-message-queued-tag-changes - (loop for id in forward-queries - collect - (cons id - notmuch-message-forwarded-tags)))) - + (setq notmuch-message-queued-tag-changes + (cl-loop for id in forward-queries + collect + (cons id notmuch-message-forwarded-tags)))) ;; `message-forward-make-body' shows the User-agent header. Hide ;; it again. (message-hide-headers) (set-buffer-modified-p nil)))) -(defun notmuch-mua-new-reply (query-string &optional prompt-for-sender reply-all) +(defun notmuch-mua-new-reply (query-string &optional prompt-for-sender reply-all duplicate) "Compose a reply to the message identified by QUERY-STRING. If PROMPT-FOR-SENDER is non-nil, the user will be prompted for the From: address first. If REPLY-ALL is non-nil, the message -will be addressed to all recipients of the source message." - -;; In current emacs (24.3) select-active-regions is set to t by -;; default. The reply insertion code sets the region to the quoted -;; message to make it easy to delete (kill-region or C-w). These two -;; things combine to put the quoted message in the primary selection. -;; -;; This is not what the user wanted and is a privacy risk (accidental -;; pasting of the quoted message). We can avoid some of the problems -;; by let-binding select-active-regions to nil. This fixes if the -;; primary selection was previously in a non-emacs window but not if -;; it was in an emacs window. To avoid the problem in the latter case -;; we deactivate mark. - - (let ((sender - (when prompt-for-sender - (notmuch-mua-prompt-for-sender))) +will be addressed to all recipients of the source message. If +DUPLICATE is non-nil, based the reply on that duplicate file" + ;; `select-active-regions' is t by default. The reply insertion code + ;; sets the region to the quoted message to make it easy to delete + ;; (kill-region or C-w). These two things combine to put the quoted + ;; message in the primary selection. + ;; + ;; This is not what the user wanted and is a privacy risk (accidental + ;; pasting of the quoted message). We can avoid some of the problems + ;; by let-binding select-active-regions to nil. This fixes if the + ;; primary selection was previously in a non-emacs window but not if + ;; it was in an emacs window. To avoid the problem in the latter case + ;; we deactivate mark. + (let ((sender (and prompt-for-sender + (notmuch-mua-prompt-for-sender))) (select-active-regions nil)) - (notmuch-mua-reply query-string sender reply-all) + (notmuch-mua-reply query-string sender reply-all duplicate) (deactivate-mark))) +;;; Checks + (defun notmuch-mua-check-no-misplaced-secure-tag () "Query user if there is a misplaced secure mml tag. @@ -570,11 +579,11 @@ tag, or the user confirms they mean it." (goto-char (point-max)) (or ;; We are always fine if there is no secure tag. - (not (search-backward "<#secure" nil 't)) + (not (search-backward "<#secure" nil t)) ;; There is a secure tag, so it must be at the start of the ;; body, with no secure tag earlier (i.e., in the headers). (and (= (point) body-start) - (not (search-backward "<#secure" nil 't))) + (not (search-backward "<#secure" nil t))) ;; The user confirms they means it. (yes-or-no-p "\ There is a <#secure> tag not at the start of the body. It is @@ -601,45 +610,47 @@ The <#secure> tag at the start of the body is not followed by a newline. It is likely that the message will be sent unsigned and unencrypted. Really send? ")))) +;;; Finishing commands + (defun notmuch-mua-send-common (arg &optional exit) (interactive "P") (run-hooks 'notmuch-mua-send-hook) (when (and (notmuch-mua-check-no-misplaced-secure-tag) (notmuch-mua-check-secure-tag-has-newline)) - (letf (((symbol-function 'message-do-fcc) #'notmuch-maildir-message-do-fcc)) - (if exit - (message-send-and-exit arg) - (message-send arg))))) + (cl-letf (((symbol-function 'message-do-fcc) + #'notmuch-maildir-message-do-fcc)) + (if exit + (message-send-and-exit arg) + (message-send arg))))) +;;;#autoload (defun notmuch-mua-send-and-exit (&optional arg) (interactive "P") - (notmuch-mua-send-common arg 't)) + (notmuch-mua-send-common arg t)) +;;;#autoload (defun notmuch-mua-send (&optional arg) (interactive "P") (notmuch-mua-send-common arg)) +;;;#autoload (defun notmuch-mua-kill-buffer () (interactive) (message-kill-buffer)) -(defun notmuch-mua-message-send-hook () - "The default function used for `notmuch-mua-send-hook', this -simply runs the corresponding `message-mode' hook functions." - (run-hooks 'message-send-hook)) - -;; +;;; _ +;;;#autoload (define-mail-user-agent 'notmuch-user-agent - 'notmuch-mua-mail 'notmuch-mua-send-and-exit - 'notmuch-mua-kill-buffer 'notmuch-mua-send-hook) + 'notmuch-mua-mail + 'notmuch-mua-send-and-exit + 'notmuch-mua-kill-buffer + 'notmuch-mua-send-hook) ;; Add some more headers to the list that `message-mode' hides when ;; composing a message. (notmuch-mua-add-more-hidden-headers) -;; - (provide 'notmuch-mua) ;;; notmuch-mua.el ends here diff --git a/emacs/notmuch-parser.el b/emacs/notmuch-parser.el index bb0379c1..710c60e1 100644 --- a/emacs/notmuch-parser.el +++ b/emacs/notmuch-parser.el @@ -1,4 +1,4 @@ -;;; notmuch-parser.el --- streaming S-expression parser +;;; notmuch-parser.el --- streaming S-expression parser -*- lexical-binding: t -*- ;; ;; Copyright © Austin Clements ;; @@ -21,7 +21,9 @@ ;;; Code: -(require 'cl) +(require 'cl-lib) +(require 'pcase) +(require 'subr-x) (defun notmuch-sexp-create-parser () "Return a new streaming S-expression parser. @@ -33,19 +35,15 @@ complete S-expression from the input. However, it extends this with an additional function that requires the next value in the input to be a list and descends into it, allowing its elements to be read one at a time or further descended into. Both functions -can return 'retry to indicate that not enough input is available. +can return \\='retry to indicate that not enough input is available. The parser always consumes input from point in the current buffer. Hence, the caller is allowed to delete any data before point and may resynchronize after an error by moving point." - (vector 'notmuch-sexp-parser - ;; List depth - 0 - ;; Partial parse position marker - nil - ;; Partial parse state - nil)) + 0 ; List depth + nil ; Partial parse position marker + nil)) ; Partial parse state (defmacro notmuch-sexp--depth (sp) `(aref ,sp 1)) (defmacro notmuch-sexp--partial-pos (sp) `(aref ,sp 2)) @@ -54,13 +52,12 @@ point and may resynchronize after an error by moving point." (defun notmuch-sexp-read (sp) "Consume and return the value at point in the current buffer. -Returns 'retry if there is insufficient input to parse a complete +Returns \\='retry if there is insufficient input to parse a complete value (though it may still move point over whitespace). If the parser is currently inside a list and the next token ends the -list, this moves point just past the terminator and returns 'end. +list, this moves point just past the terminator and returns \\='end. Otherwise, this moves point to just past the end of the value and returns the value." - (skip-chars-forward " \n\r\t") (cond ((eobp) 'retry) ((= (char-after) ?\)) @@ -70,7 +67,7 @@ returns the value." ;; error to be consistent with all other code paths. (read (current-buffer)) ;; Go up a level and return an end token - (decf (notmuch-sexp--depth sp)) + (cl-decf (notmuch-sexp--depth sp)) (forward-char) 'end)) ((= (char-after) ?\() @@ -80,7 +77,7 @@ returns the value." ;; parse, extend the partial parse to figure out when we ;; have a complete list. (catch 'return - (when (null (notmuch-sexp--partial-state sp)) + (unless (notmuch-sexp--partial-state sp) (let ((start (point))) (condition-case nil (throw 'return (read (current-buffer))) @@ -94,8 +91,8 @@ returns the value." (notmuch-sexp--partial-state sp))) ;; A complete value is available if we've ;; reached depth 0. - (depth (first new-state))) - (assert (>= depth 0)) + (depth (car new-state))) + (cl-assert (>= depth 0)) (if (= depth 0) ;; Reset partial parse state (setf (notmuch-sexp--partial-state sp) nil @@ -128,34 +125,23 @@ returns the value." (defun notmuch-sexp-begin-list (sp) "Parse the beginning of a list value and enter the list. -Returns 'retry if there is insufficient input to parse the +Returns \\='retry if there is insufficient input to parse the beginning of the list. If this is able to parse the beginning of a list, it moves point past the token that opens the list and returns t. Later calls to `notmuch-sexp-read' will return the elements inside the list. If the input in buffer is not the beginning of a list, throw invalid-read-syntax." - (skip-chars-forward " \n\r\t") (cond ((eobp) 'retry) ((= (char-after) ?\() (forward-char) - (incf (notmuch-sexp--depth sp)) + (cl-incf (notmuch-sexp--depth sp)) t) (t ;; Skip over the bad character like `read' does (forward-char) (signal 'invalid-read-syntax (list (string (char-before))))))) -(defun notmuch-sexp-eof (sp) - "Signal an error if there is more data in SP's buffer. - -Moves point to the beginning of any trailing data or to the end -of the buffer if there is only trailing whitespace." - - (skip-chars-forward " \n\r\t") - (unless (eobp) - (error "Trailing garbage following expression"))) - (defvar notmuch-sexp--parser nil "The buffer-local notmuch-sexp-parser instance. @@ -173,15 +159,13 @@ complete value in the list. It operates incrementally and should be called whenever the input buffer has been extended with additional data. The caller just needs to ensure it does not move point in the input buffer." - ;; Set up the initial state (unless (local-variable-p 'notmuch-sexp--parser) - (set (make-local-variable 'notmuch-sexp--parser) - (notmuch-sexp-create-parser)) - (set (make-local-variable 'notmuch-sexp--state) 'begin)) + (setq-local notmuch-sexp--parser (notmuch-sexp-create-parser)) + (setq-local notmuch-sexp--state 'begin)) (let (done) (while (not done) - (case notmuch-sexp--state + (cl-case notmuch-sexp--state (begin ;; Enter the list (if (eq (notmuch-sexp-begin-list notmuch-sexp--parser) 'retry) @@ -190,22 +174,21 @@ move point in the input buffer." (result ;; Parse a result (let ((result (notmuch-sexp-read notmuch-sexp--parser))) - (case result + (cl-case result (retry (setq done t)) (end (setq notmuch-sexp--state 'end)) (t (with-current-buffer result-buffer (funcall result-function result)))))) (end - ;; Any trailing data is unexpected - (notmuch-sexp-eof notmuch-sexp--parser) + ;; Skip over trailing whitespace. + (skip-chars-forward " \n\r\t") + ;; Any trailing data is unexpected. + (unless (eobp) + (error "Trailing garbage following expression")) (setq done t))))) ;; Clear out what we've parsed (delete-region (point-min) (point))) (provide 'notmuch-parser) -;; Local Variables: -;; byte-compile-warnings: (not cl-functions) -;; End: - ;;; notmuch-parser.el ends here diff --git a/emacs/notmuch-pkg.el.tmpl b/emacs/notmuch-pkg.el.tmpl index de97baac..85c631de 100644 --- a/emacs/notmuch-pkg.el.tmpl +++ b/emacs/notmuch-pkg.el.tmpl @@ -3,4 +3,4 @@ "notmuch" %VERSION% "Emacs based front-end (MUA) for notmuch" - nil) + '((emacs "25.1"))) diff --git a/emacs/notmuch-print.el b/emacs/notmuch-print.el index d9b3d449..8d9f1b08 100644 --- a/emacs/notmuch-print.el +++ b/emacs/notmuch-print.el @@ -1,4 +1,4 @@ -;;; notmuch-print.el --- printing messages from notmuch. +;;; notmuch-print.el --- printing messages from notmuch -*- lexical-binding: t -*- ;; ;; Copyright © David Edmondson ;; @@ -25,6 +25,8 @@ (declare-function notmuch-show-get-prop "notmuch-show" (prop &optional props)) +;;; Options + (defcustom notmuch-print-mechanism 'notmuch-print-lpr "How should printing be done?" :group 'notmuch-show @@ -36,17 +38,17 @@ (function :tag "Use muttprint then evince" notmuch-print-muttprint/evince) (function :tag "Using a custom function"))) -;; Utility functions: +;;; Utility functions (defun notmuch-print-run-evince (file) - "View FILE using 'evince'." + "View FILE using `evince'." (start-process "evince" nil "evince" file)) (defun notmuch-print-run-muttprint (&optional output) - "Pass the contents of the current buffer to 'muttprint'. + "Pass the contents of the current buffer to `muttprint'. Optional OUTPUT allows passing a list of flags to muttprint." - (apply #'call-process-region (point-min) (point-max) + (apply #'notmuch--call-process-region (point-min) (point-max) ;; Reads from stdin. "muttprint" nil nil nil @@ -54,9 +56,9 @@ Optional OUTPUT allows passing a list of flags to muttprint." "--printed-headers" "Date_To_From_CC_Newsgroups_*Subject*_/Tags/" output)) -;; User-visible functions: +;;; User-visible functions -(defun notmuch-print-lpr (msg) +(defun notmuch-print-lpr (_msg) "Print a message buffer using lpr." (lpr-buffer)) @@ -76,11 +78,11 @@ Optional OUTPUT allows passing a list of flags to muttprint." (ps-print-buffer ps-file) (notmuch-print-run-evince ps-file))) -(defun notmuch-print-muttprint (msg) +(defun notmuch-print-muttprint (_msg) "Print a message using muttprint." (notmuch-print-run-muttprint)) -(defun notmuch-print-muttprint/evince (msg) +(defun notmuch-print-muttprint/evince (_msg) "Preview a message buffer using muttprint and evince." (let ((ps-file (make-temp-file "notmuch" nil ".ps"))) (notmuch-print-run-muttprint (list "--printer" (concat "TO_FILE:" ps-file))) @@ -91,6 +93,8 @@ Optional OUTPUT allows passing a list of flags to muttprint." (set-buffer-modified-p nil) (funcall notmuch-print-mechanism msg)) +;;; _ + (provide 'notmuch-print) ;;; notmuch-print.el ends here diff --git a/emacs/notmuch-query.el b/emacs/notmuch-query.el index 563e4acf..2a46144c 100644 --- a/emacs/notmuch-query.el +++ b/emacs/notmuch-query.el @@ -1,4 +1,4 @@ -;;; notmuch-query.el --- provide an emacs api to query notmuch +;;; notmuch-query.el --- provide an emacs api to query notmuch -*- lexical-binding: t -*- ;; ;; Copyright © David Bremner ;; @@ -23,57 +23,51 @@ (require 'notmuch-lib) -(defun notmuch-query-get-threads (search-terms) - "Return a list of threads of messages matching SEARCH-TERMS. +;;; Basic query function -A thread is a forest or list of trees. A tree is a two element -list where the first element is a message, and the second element -is a possibly empty forest of replies. -" - (let ((args '("show" "--format=sexp" "--format-version=4"))) - (if notmuch-show-process-crypto - (setq args (append args '("--decrypt=true")))) - (setq args (append args search-terms)) - (apply #'notmuch-call-notmuch-sexp args))) +(define-obsolete-function-alias + 'notmuch-query-get-threads + #'notmuch--run-show + "notmuch 0.37") -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Mapping functions across collections of messages. +;;; Mapping functions across collections of messages (defun notmuch-query-map-aux (mapper function seq) - "private function to do the actual mapping and flattening" - (apply 'append - (mapcar - (lambda (tree) - (funcall mapper function tree)) - seq))) + "Private function to do the actual mapping and flattening." + (cl-mapcan (lambda (tree) + (funcall mapper function tree)) + seq)) (defun notmuch-query-map-threads (fn threads) - "apply FN to every thread in THREADS. Flatten results to a list. - -See the function notmuch-query-get-threads for more information." + "Apply function FN to every thread in THREADS. +Flatten results to a list. See the function +`notmuch-query-get-threads' for more information." (notmuch-query-map-aux 'notmuch-query-map-forest fn threads)) (defun notmuch-query-map-forest (fn forest) - "apply function to every message in a forest. Flatten results to a list. - -See the function notmuch-query-get-threads for more information. -" + "Apply function FN to every message in FOREST. +Flatten results to a list. See the function +`notmuch-query-get-threads' for more information." (notmuch-query-map-aux 'notmuch-query-map-tree fn forest)) (defun notmuch-query-map-tree (fn tree) - "Apply function FN to every message in TREE. Flatten results to a list + "Apply function FN to every message in TREE. +Flatten results to a list. See the function +`notmuch--run-show' for more information." + (cons (funcall fn (car tree)) + (notmuch-query-map-forest fn (cadr tree)))) -See the function notmuch-query-get-threads for more information." - (cons (funcall fn (car tree)) (notmuch-query-map-forest fn (cadr tree)))) - -;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; -;; Predefined queries +;;; Predefined queries (defun notmuch-query-get-message-ids (&rest search-terms) - "Return a list of message-ids of messages that match SEARCH-TERMS" + "Return a list of message-ids of messages that match SEARCH-TERMS." (notmuch-query-map-threads (lambda (msg) (plist-get msg :id)) - (notmuch-query-get-threads search-terms))) + (notmuch--run-show search-terms))) + +;;; Everything in this library is obsolete +(dolist (fun '(map-aux map-threads map-forest map-tree get-message-ids)) + (make-obsolete (intern (format "notmuch-query-%s" fun)) nil "notmuch 0.37")) (provide 'notmuch-query) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index e13ca3d7..4c0ad74d 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el @@ -1,4 +1,4 @@ -;;; notmuch-show.el --- displaying notmuch forests. +;;; notmuch-show.el --- displaying notmuch forests -*- lexical-binding: t -*- ;; ;; Copyright © Carl Worth ;; Copyright © David Edmondson @@ -23,7 +23,6 @@ ;;; Code: -(eval-when-compile (require 'cl)) (require 'mm-view) (require 'message) (require 'mm-decode) @@ -33,26 +32,36 @@ (require 'notmuch-lib) (require 'notmuch-tag) -(require 'notmuch-query) (require 'notmuch-wash) (require 'notmuch-mua) (require 'notmuch-crypto) (require 'notmuch-print) (require 'notmuch-draft) -(declare-function notmuch-call-notmuch-process "notmuch" (&rest args)) +(declare-function notmuch-call-notmuch-process "notmuch-lib" (&rest args)) (declare-function notmuch-search-next-thread "notmuch" nil) (declare-function notmuch-search-previous-thread "notmuch" nil) -(declare-function notmuch-search-show-thread "notmuch" nil) +(declare-function notmuch-search-show-thread "notmuch") (declare-function notmuch-foreach-mime-part "notmuch" (function mm-handle)) (declare-function notmuch-count-attachments "notmuch" (mm-handle)) (declare-function notmuch-save-attachments "notmuch" (mm-handle &optional queryp)) (declare-function notmuch-tree "notmuch-tree" - (&optional query query-context target buffer-name open-target)) + (&optional query query-context target buffer-name + open-target unthreaded parent-buffer)) (declare-function notmuch-tree-get-message-properties "notmuch-tree" nil) +(declare-function notmuch-unthreaded "notmuch-tree" + (&optional query query-context target buffer-name + open-target)) (declare-function notmuch-read-query "notmuch" (prompt)) (declare-function notmuch-draft-resume "notmuch-draft" (id)) +(defvar shr-blocked-images) +(defvar gnus-blocked-images) +(defvar shr-content-function) +(defvar w3m-ignored-image-url-regexp) + +;;; Options + (defcustom notmuch-message-headers '("Subject" "To" "Cc" "Date") "Headers that should be shown in a message, in this order. @@ -74,6 +83,59 @@ visible for any given message." :type 'boolean :group 'notmuch-show) +(defcustom notmuch-show-header-line t + "Show a header line in notmuch show buffers. + +If t (the default), the header line will contain the current +message's subject. + +If a string, this value is interpreted as a format string to be +passed to `format-spec` with `%s` as the substitution variable +for the message's subject. E.g., to display the subject trimmed +to a maximum of 80 columns, you could use \"%>-80s\" as format. + +If you assign to this variable a function, it will be called with +the subject as argument, and the return value will be used as the +header line format. Since the function is called with the +message buffer as the current buffer, it is also possible to +access any other properties of the message, using for instance +notmuch-show functions such as +`notmuch-show-get-message-properties'. + +Finally, if this variable is set to nil, no header is +displayed." + :type '(choice (const :tag "No header" ni) + (const :tag "Subject" t) + (string :tag "Format") + (function :tag "Function")) + :group 'notmuch-show) + +(defcustom notmuch-show-depth-limit nil + "Depth beyond which message bodies are displayed lazily. + +If bound to an integer, any message with tree depth greater than +this will have its body display lazily, initially +inserting only a button. + +If this variable is set to nil (the default) no such lazy +insertion is done." + :type '(choice (const :tag "No limit" nil) + (number :tag "Limit" 10)) + :group 'notmuch-show) + +(defcustom notmuch-show-height-limit nil + "Height (from leaves) beyond which message bodies are displayed lazily. + +If bound to an integer, any message with height in the message +tree greater than this will have its body displayed lazily, +initially only a button. + +If this variable is set to nil (the default) no such lazy +display is done." + :type '(choice (const :tag "No limit" nil) + (number :tag "Limit" 10)) + :group 'notmuch-show) + (defcustom notmuch-show-relative-dates t "Display relative dates in the message summary line." :type 'boolean @@ -90,10 +152,11 @@ visible for any given message." :group 'notmuch-show :group 'notmuch-hooks) -(defcustom notmuch-show-insert-text/plain-hook '(notmuch-wash-wrap-long-lines - notmuch-wash-tidy-citations - notmuch-wash-elide-blank-lines - notmuch-wash-excerpt-citations) +(defcustom notmuch-show-insert-text/plain-hook + '(notmuch-wash-wrap-long-lines + notmuch-wash-tidy-citations + notmuch-wash-elide-blank-lines + notmuch-wash-excerpt-citations) "Functions used to improve the display of text/plain parts." :type 'hook :options '(notmuch-wash-convert-inline-patch-to-part @@ -155,38 +218,37 @@ indentation." :type '(choice (const nil) regexp) :group 'notmuch-show) -(defvar notmuch-show-thread-id nil) -(make-variable-buffer-local 'notmuch-show-thread-id) +;;; Variables -(defvar notmuch-show-parent-buffer nil) -(make-variable-buffer-local 'notmuch-show-parent-buffer) +(defvar-local notmuch-show-thread-id nil) -(defvar notmuch-show-query-context nil) -(make-variable-buffer-local 'notmuch-show-query-context) +(defvar-local notmuch-show-parent-buffer nil) -(defvar notmuch-show-process-crypto nil) -(make-variable-buffer-local 'notmuch-show-process-crypto) +(defvar-local notmuch-show-query-context nil) -(defvar notmuch-show-elide-non-matching-messages nil) -(make-variable-buffer-local 'notmuch-show-elide-non-matching-messages) +(defvar-local notmuch-show-process-crypto nil) -(defvar notmuch-show-indent-content t) -(make-variable-buffer-local 'notmuch-show-indent-content) +(defvar-local notmuch-show-elide-non-matching-messages nil) + +(defvar-local notmuch-show-indent-content t) + +(defvar-local notmuch-show-single-message nil) (defvar notmuch-show-attachment-debug nil - "If t log stdout and stderr from attachment handlers + "If t log stdout and stderr from attachment handlers. When set to nil (the default) stdout and stderr from attachment handlers is discarded. When set to t the stdout and stderr from each attachment handler is logged in buffers with names beginning -\" *notmuch-part*\". This option requires emacs version at least -24.3 to work.") +\" *notmuch-part*\".") + +;;; Options (defcustom notmuch-show-stash-mlarchive-link-alist - '(("Gmane" . "https://mid.gmane.org/") - ("MARC" . "https://marc.info/?i=") + '(("MARC" . "https://marc.info/?i=") ("Mail Archive, The" . "https://mid.mail-archive.com/") - ("LKML" . "https://lkml.kernel.org/r/") + ("Lore" . "https://lore.kernel.org/r/") + ("Notmuch" . "https://nmbug.notmuchmail.org/nmweb/show/") ;; FIXME: can these services be searched by `Message-Id' ? ;; ("MarkMail" . "http://markmail.org/") ;; ("Nabble" . "http://nabble.com/") @@ -211,7 +273,7 @@ return the ML archive reference URI." (function :tag "Function returning the URL"))) :group 'notmuch-show) -(defcustom notmuch-show-stash-mlarchive-link-default "Gmane" +(defcustom notmuch-show-stash-mlarchive-link-default "MARC" "Default Mailing List Archive to use when stashing links. This is used when `notmuch-show-stash-mlarchive-link' isn't @@ -260,14 +322,16 @@ position of the message in the thread." :type 'boolean :group 'notmuch-show) +;;; Utilities + (defmacro with-current-notmuch-show-message (&rest body) - "Evaluate body with current buffer set to the text of current message" + "Evaluate body with current buffer set to the text of current message." `(save-excursion (let ((id (notmuch-show-get-message-id))) (let ((buf (generate-new-buffer (concat "*notmuch-msg-" id "*")))) - (with-current-buffer buf + (with-current-buffer buf (let ((coding-system-for-read 'no-conversion)) - (call-process notmuch-command nil t nil "show" "--format=raw" id)) + (notmuch--call-process notmuch-command nil t nil "show" "--format=raw" id)) ,@body) (kill-buffer buf))))) @@ -275,6 +339,8 @@ position of the message in the thread." "Enable Visual Line mode." (visual-line-mode t)) +;;; Commands + ;; DEPRECATED in Notmuch 0.16 since we now have convenient part ;; commands. We'll keep the command around for a version or two in ;; case people want to bind it themselves. @@ -290,13 +356,12 @@ position of the message in the thread." ;; ;; Any MIME part not explicitly mentioned here will be handled by an ;; external viewer as configured in the various mailcap files. - (let ((mm-inline-media-tests '( - ("text/.*" ignore identity) - ("application/pgp-signature" ignore identity) - ("multipart/alternative" ignore identity) - ("multipart/mixed" ignore identity) - ("multipart/related" ignore identity) - ))) + (let ((mm-inline-media-tests + '(("text/.*" ignore identity) + ("application/pgp-signature" ignore identity) + ("multipart/alternative" ignore identity) + ("multipart/mixed" ignore identity) + ("multipart/related" ignore identity)))) (mm-display-parts (mm-dissect-buffer))))) (defun notmuch-show-save-attachments () @@ -313,7 +378,6 @@ position of the message in the thread." FN is called with one argument, the message properties. It should operation on the contents of the current buffer." - ;; Remake the header to ensure that all information is available. (let* ((to (notmuch-show-get-to)) (cc (notmuch-show-get-cc)) @@ -322,11 +386,10 @@ operation on the contents of the current buffer." (date (notmuch-show-get-date)) (tags (notmuch-show-get-tags)) (depth (notmuch-show-get-depth)) - (header (concat "Subject: " subject "\n" "To: " to "\n" - (if (not (string= cc "")) + (if (not (string-empty-p cc)) (concat "Cc: " cc "\n") "") "From: " from "\n" @@ -342,8 +405,10 @@ operation on the contents of the current buffer." (indenting notmuch-show-indent-content)) (with-temp-buffer (insert all) - (if indenting - (indent-rigidly (point-min) (point-max) (- (* notmuch-show-indent-messages-width depth)))) + (when indenting + (indent-rigidly (point-min) + (point-max) + (- (* notmuch-show-indent-messages-width depth)))) ;; Remove the original header. (goto-char (point-min)) (re-search-forward "^$" (point-max) nil) @@ -356,6 +421,8 @@ operation on the contents of the current buffer." (interactive) (notmuch-show-with-message-as-text 'notmuch-print-message)) +;;; Headers + (defun notmuch-show-fontify-header () (let ((face (cond ((looking-at "[Tt]o:") @@ -366,7 +433,6 @@ operation on the contents of the current buffer." 'message-header-subject) (t 'message-header-other)))) - (overlay-put (make-overlay (point) (re-search-forward ":")) 'face 'message-header-name) (overlay-put (make-overlay (point) (re-search-forward ".*$")) @@ -386,70 +452,68 @@ operation on the contents of the current buffer." (defun notmuch-show-update-tags (tags) "Update the displayed tags of the current message." (save-excursion - (goto-char (notmuch-show-message-top)) - (if (re-search-forward "(\\([^()]*\\))$" (line-end-position) t) - (let ((inhibit-read-only t)) - (replace-match (concat "(" - (notmuch-tag-format-tags tags (notmuch-show-get-prop :orig-tags)) - ")")))))) + (let ((inhibit-read-only t) + (start (notmuch-show-message-top)) + (depth (notmuch-show-get-prop :depth)) + (orig-tags (notmuch-show-get-prop :orig-tags)) + (props (notmuch-show-get-message-properties)) + (extent (notmuch-show-message-extent))) + (goto-char start) + (notmuch-show-insert-headerline props depth tags orig-tags) + (put-text-property start (1+ start) + :notmuch-message-properties props) + (put-text-property (car extent) (cdr extent) :notmuch-message-extent extent) + ;; delete original headerline, but do not save to kill ring + (delete-region (point) (1+ (line-end-position)))))) (defun notmuch-clean-address (address) "Try to clean a single email ADDRESS for display. Return a cons cell of (AUTHOR_EMAIL AUTHOR_NAME). Return (ADDRESS nil) if parsing fails." (condition-case nil - (let (p-name p-address) - ;; It would be convenient to use `mail-header-parse-address', - ;; but that expects un-decoded mailbox parts, whereas our - ;; mailbox parts are already decoded (and hence may contain - ;; UTF-8). Given that notmuch should handle most of the awkward - ;; cases, some simple string deconstruction should be sufficient - ;; here. - (cond - ;; "User " style. - ((string-match "\\(.*\\) <\\(.*\\)>" address) - (setq p-name (match-string 1 address) - p-address (match-string 2 address))) - - ;; "" style. - ((string-match "<\\(.*\\)>" address) - (setq p-address (match-string 1 address))) - - ;; Everything else. - (t - (setq p-address address))) - - (when p-name - ;; Remove elements of the mailbox part that are not relevant for - ;; display, even if they are required during transport: - ;; - ;; Backslashes. - (setq p-name (replace-regexp-in-string "\\\\" "" p-name)) - - ;; Outer single and double quotes, which might be nested. - (loop - with start-of-loop - do (setq start-of-loop p-name) - - when (string-match "^\"\\(.*\\)\"$" p-name) - do (setq p-name (match-string 1 p-name)) - - when (string-match "^'\\(.*\\)'$" p-name) - do (setq p-name (match-string 1 p-name)) - - until (string= start-of-loop p-name))) - - ;; If the address is 'foo@bar.com ' then show just - ;; 'foo@bar.com'. - (when (string= p-name p-address) - (setq p-name nil)) - - (cons p-address p-name)) + (let (p-name p-address) + ;; It would be convenient to use `mail-header-parse-address', + ;; but that expects un-decoded mailbox parts, whereas our + ;; mailbox parts are already decoded (and hence may contain + ;; UTF-8). Given that notmuch should handle most of the awkward + ;; cases, some simple string deconstruction should be sufficient + ;; here. + (cond + ;; "User " style. + ((string-match "\\(.*\\) <\\(.*\\)>" address) + (setq p-name (match-string 1 address)) + (setq p-address (match-string 2 address))) + + ;; "" style. + ((string-match "<\\(.*\\)>" address) + (setq p-address (match-string 1 address))) + ;; Everything else. + (t + (setq p-address address))) + (when p-name + ;; Remove elements of the mailbox part that are not relevant for + ;; display, even if they are required during transport: + ;; + ;; Backslashes. + (setq p-name (replace-regexp-in-string "\\\\" "" p-name)) + ;; Outer single and double quotes, which might be nested. + (cl-loop with start-of-loop + do (setq start-of-loop p-name) + when (string-match "^\"\\(.*\\)\"$" p-name) + do (setq p-name (match-string 1 p-name)) + when (string-match "^'\\(.*\\)'$" p-name) + do (setq p-name (match-string 1 p-name)) + until (string= start-of-loop p-name))) + ;; If the address is 'foo@bar.com ' then show just + ;; 'foo@bar.com'. + (when (string= p-name p-address) + (setq p-name nil)) + (cons p-address p-name)) (error (cons address nil)))) (defun notmuch-show-clean-address (address) - "Try to clean a single email ADDRESS for display. Return -unchanged ADDRESS if parsing fails." + "Try to clean a single email ADDRESS for display. +Return unchanged ADDRESS if parsing fails." (let* ((clean-address (notmuch-clean-address address)) (p-address (car clean-address)) (p-name (cdr clean-address))) @@ -459,19 +523,54 @@ unchanged ADDRESS if parsing fails." ;; Otherwise format the name and address together. (concat p-name " <" p-address ">")))) -(defun notmuch-show-insert-headerline (headers date tags depth) +(defun notmuch-show--mark-height (tree) + "Calculate and cache height (distance from deepest descendent)" + (let* ((msg (car tree)) + (children (cadr tree)) + (cached-height (plist-get msg :height))) + (or cached-height + (let ((height + (if (null children) 0 + (1+ (apply #'max (mapcar #'notmuch-show--mark-height children)))))) + (plist-put msg :height height) + height)))) + +(defun notmuch-show-insert-headerline (msg-plist depth tags &optional orig-tags) "Insert a notmuch style headerline based on HEADERS for a message at DEPTH in the current thread." - (let ((start (point))) - (insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width depth)) - (notmuch-sanitize - (notmuch-show-clean-address (plist-get headers :From))) + (let* ((start (point)) + (headers (plist-get msg-plist :headers)) + (duplicate (or (plist-get msg-plist :duplicate) 0)) + (file-count (length (plist-get msg-plist :filename))) + (date (or (and notmuch-show-relative-dates + (plist-get msg-plist :date_relative)) + (plist-get headers :Date))) + (from (notmuch-sanitize + (notmuch-show-clean-address (plist-get headers :From))))) + (when (string-match "\\cR" from) + ;; If the From header has a right-to-left character add + ;; invisible U+200E LEFT-TO-RIGHT MARK character which forces + ;; the header paragraph as left-to-right text. + (insert (propertize (string ?\x200e) 'invisible t))) + (insert (if notmuch-show-indent-content + (notmuch-show-spaces-n (* notmuch-show-indent-messages-width + depth)) + "") + from " (" date ") (" - (notmuch-tag-format-tags tags tags) - ")\n") - (overlay-put (make-overlay start (point)) 'face 'notmuch-message-summary-face))) + (notmuch-tag-format-tags tags (or orig-tags tags)) + ")") + (insert + (if (> file-count 1) + (let ((txt (format "%d/%d\n" duplicate file-count))) + (concat + (notmuch-show-spaces-n (max 0 (- (window-width) (+ (current-column) (length txt))))) + txt)) + "\n")) + (overlay-put (make-overlay start (point)) + 'face 'notmuch-message-summary-face))) (defun notmuch-show-insert-header (header header-value) "Insert a single header." @@ -483,38 +582,36 @@ message at DEPTH in the current thread." (mapc (lambda (header) (let* ((header-symbol (intern (concat ":" header))) (header-value (plist-get headers header-symbol))) - (if (and header-value - (not (string-equal "" header-value))) - (notmuch-show-insert-header header header-value)))) + (when (and header-value + (not (string-equal "" header-value))) + (notmuch-show-insert-header header header-value)))) notmuch-message-headers) (save-excursion (save-restriction (narrow-to-region start (point-max)) (run-hooks 'notmuch-show-markup-headers-hook))))) +;;; Parts + (define-button-type 'notmuch-show-part-button-type 'action 'notmuch-show-part-button-default 'follow-link t 'face 'message-mml :supertype 'notmuch-button-type) -(defun notmuch-show-insert-part-header (nth content-type declared-type &optional name comment) - (let ((button) - (base-label (concat (when name (concat name ": ")) +(defun notmuch-show-insert-part-header (_nth content-type declared-type + &optional name comment) + (let ((base-label (concat (and name (concat name ": ")) declared-type - (unless (string-equal declared-type content-type) - (concat " (as " content-type ")")) + (and (not (string-equal declared-type content-type)) + (concat " (as " content-type ")")) comment))) - - (setq button - (insert-button - (concat "[ " base-label " ]") - :base-label base-label - :type 'notmuch-show-part-button-type - :notmuch-part-hidden nil)) - (insert "\n") - ;; return button - button)) + (prog1 (insert-button + (concat "[ " base-label " ]") + :base-label base-label + :type 'notmuch-show-part-button-type + :notmuch-part-hidden nil) + (insert "\n")))) (defun notmuch-show-toggle-part-invisibility (&optional button) (interactive) @@ -522,8 +619,9 @@ message at DEPTH in the current thread." (when button (let ((overlay (button-get button 'overlay)) (lazy-part (button-get button :notmuch-lazy-part))) - ;; We have a part to toggle if there is an overlay or if there is a lazy part. - ;; If neither is present we cannot toggle the part so we just return nil. + ;; We have a part to toggle if there is an overlay or if there + ;; is a lazy part. If neither is present we cannot toggle the + ;; part so we just return nil. (when (or overlay lazy-part) (let* ((show (button-get button :notmuch-part-hidden)) (new-start (button-start button)) @@ -546,11 +644,27 @@ message at DEPTH in the current thread." (when show (button-put button :notmuch-lazy-part nil) (notmuch-show-lazy-part lazy-part button)) - ;; else there must be an overlay. - (overlay-put overlay 'invisible (not show)) + (let* ((part (plist-get properties :notmuch-part)) + (undisplayer (plist-get part :undisplayer)) + (mime-type (plist-get part :computed-type)) + (redisplay-data (button-get button + :notmuch-redisplay-data)) + (imagep (string-match "^image/" mime-type))) + (cond + ((and imagep (not show) undisplayer) + ;; call undisplayer thunk created by gnus. + (funcall undisplayer) + ;; there is an extra newline left + (delete-region + (+ 1 (button-end button)) + (+ 2 (button-end button)))) + ((and imagep show redisplay-data) + (notmuch-show-lazy-part redisplay-data button)) + (t + (overlay-put overlay 'invisible (not show))))) t))))))) -;; Part content ID handling +;;; Part content ID handling (defvar notmuch-show--cids nil "Alist from raw content ID to (MSG PART).") @@ -569,15 +683,17 @@ message at DEPTH in the current thread." ;; alternative (even if we can't render it). (push (list content-id msg part) notmuch-show--cids))) ;; Recurse on sub-parts - (let ((ctype (notmuch-split-content-type - (downcase (plist-get part :content-type))))) - (cond ((equal (first ctype) "multipart") - (mapc (apply-partially #'notmuch-show--register-cids msg) - (plist-get part :content))) - ((equal ctype '("message" "rfc822")) - (notmuch-show--register-cids - msg - (first (plist-get (first (plist-get part :content)) :body))))))) + (when-let ((type (plist-get part :content-type))) + (pcase-let ((`(,type ,subtype) + (split-string (downcase type) "/"))) + (cond ((equal type "multipart") + (mapc (apply-partially #'notmuch-show--register-cids msg) + (plist-get part :content))) + ((and (equal type "message") + (equal subtype "rfc822")) + (notmuch-show--register-cids + msg + (car (plist-get (car (plist-get part :content)) :body)))))))) (defun notmuch-show--get-cid-content (cid) "Return a list (CID-content content-type) or nil. @@ -586,36 +702,33 @@ This will only find parts from messages that have been inserted into the current buffer. CID must be a raw content ID, without enclosing angle brackets, a cid: prefix, or URL encoding. This will return nil if the CID is unknown or cannot be retrieved." - (let ((descriptor (cdr (assoc cid notmuch-show--cids)))) - (when descriptor - (let* ((msg (first descriptor)) - (part (second descriptor)) - ;; Request caching for this content, as some messages - ;; reference the same cid: part many times (hundreds!). - (content (notmuch-get-bodypart-binary - msg part notmuch-show-process-crypto 'cache)) - (content-type (plist-get part :content-type))) - (list content content-type))))) + (when-let ((descriptor (cdr (assoc cid notmuch-show--cids)))) + (pcase-let ((`(,msg ,part) descriptor)) + ;; Request caching for this content, as some messages + ;; reference the same cid: part many times (hundreds!). + (list (notmuch-get-bodypart-binary + msg part notmuch-show-process-crypto 'cache) + (plist-get part :content-type))))) (defun notmuch-show-setup-w3m () "Instruct w3m how to retrieve content from a \"related\" part of a message." (interactive) - (if (boundp 'w3m-cid-retrieve-function-alist) - (unless (assq 'notmuch-show-mode w3m-cid-retrieve-function-alist) - (push (cons 'notmuch-show-mode #'notmuch-show--cid-w3m-retrieve) - w3m-cid-retrieve-function-alist))) + (when (and (boundp 'w3m-cid-retrieve-function-alist) + (not (assq 'notmuch-show-mode w3m-cid-retrieve-function-alist))) + (push (cons 'notmuch-show-mode #'notmuch-show--cid-w3m-retrieve) + w3m-cid-retrieve-function-alist)) (setq mm-html-inhibit-images nil)) (defvar w3m-current-buffer) ;; From `w3m.el'. -(defun notmuch-show--cid-w3m-retrieve (url &rest args) +(defun notmuch-show--cid-w3m-retrieve (url &rest _args) ;; url includes the cid: prefix and is URL encoded (see RFC 2392). (let* ((cid (url-unhex-string (substring url 4))) (content-and-type (with-current-buffer w3m-current-buffer (notmuch-show--get-cid-content cid)))) (when content-and-type - (insert (first content-and-type)) - (second content-and-type)))) + (insert (car content-and-type)) + (cadr content-and-type)))) ;; MIME part renderers @@ -623,8 +736,9 @@ will return nil if the CID is unknown or cannot be retrieved." (mapcar (lambda (inner-part) (plist-get inner-part :content-type)) (plist-get part :content))) -(defun notmuch-show-insert-part-multipart/alternative (msg part content-type nth depth button) - (let ((chosen-type (car (notmuch-multipart/alternative-choose msg (notmuch-show-multipart/*-to-list part)))) +(defun notmuch-show-insert-part-multipart/alternative (msg part _content-type _nth depth _button) + (let ((chosen-type (car (notmuch-multipart/alternative-choose + msg (notmuch-show-multipart/*-to-list part)))) (inner-parts (plist-get part :content)) (start (point))) ;; This inserts all parts of the chosen type rather than just one, @@ -632,8 +746,8 @@ will return nil if the CID is unknown or cannot be retrieved." ;; should be chosen if there are more than one that match? (mapc (lambda (inner-part) (let* ((inner-type (plist-get inner-part :content-type)) - (hide (not (or notmuch-show-all-multipart/alternative-parts - (string= chosen-type inner-type))))) + (hide (not (or notmuch-show-all-multipart/alternative-parts + (string= chosen-type inner-type))))) (notmuch-show-insert-bodypart msg inner-part depth hide))) inner-parts) @@ -641,99 +755,87 @@ will return nil if the CID is unknown or cannot be retrieved." (indent-rigidly start (point) 1))) t) -(defun notmuch-show-insert-part-multipart/related (msg part content-type nth depth button) +(defun notmuch-show-insert-part-multipart/related (msg part _content-type _nth depth _button) (let ((inner-parts (plist-get part :content)) (start (point))) - ;; Render the primary part. FIXME: Support RFC 2387 Start header. (notmuch-show-insert-bodypart msg (car inner-parts) depth) ;; Add hidden buttons for the rest (mapc (lambda (inner-part) (notmuch-show-insert-bodypart msg inner-part depth t)) (cdr inner-parts)) - (when notmuch-show-indent-multipart (indent-rigidly start (point) 1))) t) -(defun notmuch-show-insert-part-multipart/signed (msg part content-type nth depth button) +(defun notmuch-show-insert-part-multipart/signed (msg part _content-type _nth depth button) (when button (button-put button 'face 'notmuch-crypto-part-header)) - ;; Insert a button detailing the signature status. (notmuch-crypto-insert-sigstatus-button (car (plist-get part :sigstatus)) (notmuch-show-get-header :From msg)) - (let ((inner-parts (plist-get part :content)) (start (point))) ;; Show all of the parts. (mapc (lambda (inner-part) (notmuch-show-insert-bodypart msg inner-part depth)) inner-parts) - (when notmuch-show-indent-multipart (indent-rigidly start (point) 1))) t) -(defun notmuch-show-insert-part-multipart/encrypted (msg part content-type nth depth button) +(defun notmuch-show-insert-part-multipart/encrypted (msg part _content-type _nth depth button) (when button (button-put button 'face 'notmuch-crypto-part-header)) - ;; Insert a button detailing the encryption status. (notmuch-crypto-insert-encstatus-button (car (plist-get part :encstatus))) - ;; Insert a button detailing the signature status. (notmuch-crypto-insert-sigstatus-button (car (plist-get part :sigstatus)) (notmuch-show-get-header :From msg)) - (let ((inner-parts (plist-get part :content)) (start (point))) ;; Show all of the parts. (mapc (lambda (inner-part) (notmuch-show-insert-bodypart msg inner-part depth)) inner-parts) - (when notmuch-show-indent-multipart (indent-rigidly start (point) 1))) t) -(defun notmuch-show-insert-part-application/pgp-encrypted (msg part content-type nth depth button) +(defun notmuch-show-insert-part-application/pgp-encrypted (_msg _part _content-type _nth _depth _button) t) -(defun notmuch-show-insert-part-multipart/* (msg part content-type nth depth button) +(defun notmuch-show-insert-part-multipart/* (msg part _content-type _nth depth _button) (let ((inner-parts (plist-get part :content)) (start (point))) ;; Show all of the parts. (mapc (lambda (inner-part) (notmuch-show-insert-bodypart msg inner-part depth)) inner-parts) - - (when notmuch-show-indent-multipart - (indent-rigidly start (point) 1))) - t) - -(defun notmuch-show-insert-part-message/rfc822 (msg part content-type nth depth button) - (let* ((message (car (plist-get part :content))) - (body (car (plist-get message :body))) - (start (point))) - - ;; Override `notmuch-message-headers' to force `From' to be - ;; displayed. - (let ((notmuch-message-headers '("From" "Subject" "To" "Cc" "Date"))) - (notmuch-show-insert-headers (plist-get message :headers))) - - ;; Blank line after headers to be compatible with the normal - ;; message display. - (insert "\n") - - ;; Show the body - (notmuch-show-insert-bodypart msg body depth) - (when notmuch-show-indent-multipart (indent-rigidly start (point) 1))) t) -(defun notmuch-show-insert-part-text/plain (msg part content-type nth depth button) +(defun notmuch-show-insert-part-message/rfc822 (msg part _content-type _nth depth _button) + (let ((message (car (plist-get part :content)))) + (and + message + (let ((body (car (plist-get message :body))) + (start (point))) + ;; Override `notmuch-message-headers' to force `From' to be + ;; displayed. + (let ((notmuch-message-headers '("From" "Subject" "To" "Cc" "Date"))) + (notmuch-show-insert-headers (plist-get message :headers))) + ;; Blank line after headers to be compatible with the normal + ;; message display. + (insert "\n") + ;; Show the body + (notmuch-show-insert-bodypart msg body depth) + (when notmuch-show-indent-multipart + (indent-rigidly start (point) 1)) + t)))) + +(defun notmuch-show-insert-part-text/plain (msg part _content-type _nth depth button) ;; For backward compatibility we want to apply the text/plain hook ;; to the whole of the part including the part button if there is ;; one. @@ -747,7 +849,7 @@ will return nil if the CID is unknown or cannot be retrieved." (run-hook-with-args 'notmuch-show-insert-text/plain-hook msg depth)))) t) -(defun notmuch-show-insert-part-text/calendar (msg part content-type nth depth button) +(defun notmuch-show-insert-part-text/calendar (msg part _content-type _nth _depth _button) (insert (with-temp-buffer (insert (notmuch-get-bodypart-text msg part notmuch-show-process-crypto)) ;; notmuch-get-bodypart-text does no newline conversion. @@ -760,7 +862,8 @@ will return nil if the CID is unknown or cannot be retrieved." (unwind-protect (progn (unless (icalendar-import-buffer file t) - (error "Icalendar import error. See *icalendar-errors* for more information")) + (error "Icalendar import error. %s" + "See *icalendar-errors* for more information")) (set-buffer (get-file-buffer file)) (setq result (buffer-substring (point-min) (point-max))) (set-buffer-modified-p nil) @@ -770,45 +873,44 @@ will return nil if the CID is unknown or cannot be retrieved." t) ;; For backwards compatibility. -(defun notmuch-show-insert-part-text/x-vcalendar (msg part content-type nth depth button) - (notmuch-show-insert-part-text/calendar msg part content-type nth depth button)) - -(if (version< emacs-version "25.3") - ;; https://bugs.gnu.org/28350 - ;; - ;; For newer emacs, we fall back to notmuch-show-insert-part-*/* - ;; (see notmuch-show-handlers-for) - (defun notmuch-show-insert-part-text/enriched (msg part content-type nth depth button) - ;; By requiring enriched below, we ensure that the function enriched-decode-display-prop - ;; is defined before it will be shadowed by the letf below. Otherwise the version - ;; in enriched.el may be loaded a bit later and used instead (for the first time). - (require 'enriched) - (letf (((symbol-function 'enriched-decode-display-prop) - (lambda (start end &optional param) (list start end)))) - (notmuch-show-insert-part-*/* msg part content-type nth depth button)))) +(defun notmuch-show-insert-part-text/x-vcalendar (msg part _content-type _nth depth _button) + (notmuch-show-insert-part-text/calendar msg part nil nil depth nil)) + +(when (version< emacs-version "25.3") + ;; https://bugs.gnu.org/28350 + ;; + ;; For newer emacs, we fall back to notmuch-show-insert-part-*/* + ;; (see notmuch-show-handlers-for) + (defun notmuch-show-insert-part-text/enriched + (msg part content-type nth depth button) + ;; By requiring enriched below, we ensure that the function + ;; enriched-decode-display-prop is defined before it will be + ;; shadowed by the letf below. Otherwise the version in + ;; enriched.el may be loaded a bit later and used instead (for + ;; the first time). + (require 'enriched) + (cl-letf (((symbol-function 'enriched-decode-display-prop) + (lambda (start end &optional _param) (list start end)))) + (notmuch-show-insert-part-*/* msg part content-type nth depth button)))) (defun notmuch-show-get-mime-type-of-application/octet-stream (part) ;; If we can deduce a MIME type from the filename of the attachment, ;; we return that. - (if (plist-get part :filename) - (let ((extension (file-name-extension (plist-get part :filename))) - mime-type) - (if extension - (progn - (mailcap-parse-mimetypes) - (setq mime-type (mailcap-extension-to-mime extension)) - (if (and mime-type - (not (string-equal mime-type "application/octet-stream"))) - mime-type - nil)) - nil)))) + (and (plist-get part :filename) + (let ((extension (file-name-extension (plist-get part :filename)))) + (and extension + (progn + (mailcap-parse-mimetypes) + (let ((mime-type (mailcap-extension-to-mime extension))) + (and mime-type + (not (string-equal mime-type "application/octet-stream")) + mime-type))))))) (defun notmuch-show-insert-part-text/html (msg part content-type nth depth button) (if (eq mm-text-html-renderer 'shr) ;; It's easier to drive shr ourselves than to work around the ;; goofy things `mm-shr' does (like irreversibly taking over ;; content ID handling). - ;; FIXME: If we block an image, offer a button to load external ;; images. (let ((shr-blocked-images notmuch-show-text/html-blocked-images)) @@ -822,10 +924,12 @@ will return nil if the CID is unknown or cannot be retrieved." (let ((mm-inline-text-html-with-w3m-keymap nil) ;; FIXME: If we block an image, offer a button to load external ;; images. - (gnus-blocked-images notmuch-show-text/html-blocked-images)) + (gnus-blocked-images notmuch-show-text/html-blocked-images) + (w3m-ignored-image-url-regexp notmuch-show-text/html-blocked-images)) (notmuch-show-insert-part-*/* msg part content-type nth depth button)))) -;; These functions are used by notmuch-show--insert-part-text/html-shr +;;; Functions used by notmuch-show--insert-part-text/html-shr + (declare-function libxml-parse-html-region "xml.c") (declare-function shr-insert-document "shr") @@ -841,49 +945,48 @@ will return nil if the CID is unknown or cannot be retrieved." ;; shr strips the "cid:" part of URL, but doesn't ;; URL-decode it (see RFC 2392). (let ((cid (url-unhex-string url))) - (first (notmuch-show--get-cid-content cid)))))) + (car (notmuch-show--get-cid-content cid)))))) (shr-insert-document dom) t)) -(defun notmuch-show-insert-part-*/* (msg part content-type nth depth button) +(defun notmuch-show-insert-part-*/* (msg part content-type _nth _depth _button) ;; This handler _must_ succeed - it is the handler of last resort. (notmuch-mm-display-part-inline msg part content-type notmuch-show-process-crypto) t) -;; Functions for determining how to handle MIME parts. +;;; Functions for determining how to handle MIME parts. (defun notmuch-show-handlers-for (content-type) "Return a list of content handlers for a part of type CONTENT-TYPE." (let (result) (mapc (lambda (func) - (if (functionp func) - (push func result))) + (when (functionp func) + (push func result))) ;; Reverse order of prefrence. (list (intern (concat "notmuch-show-insert-part-*/*")) - (intern (concat - "notmuch-show-insert-part-" - (car (notmuch-split-content-type content-type)) - "/*")) + (intern (concat "notmuch-show-insert-part-" + (car (split-string content-type "/")) + "/*")) (intern (concat "notmuch-show-insert-part-" content-type)))) result)) -;; +;;; Parts (defun notmuch-show-insert-bodypart-internal (msg part content-type nth depth button) ;; Run the handlers until one of them succeeds. - (loop for handler in (notmuch-show-handlers-for content-type) - until (condition-case err - (funcall handler msg part content-type nth depth button) - ;; Specifying `debug' here lets the debugger run if - ;; `debug-on-error' is non-nil. - ((debug error) - (insert "!!! Bodypart handler `" (prin1-to-string handler) "' threw an error:\n" - "!!! " (error-message-string err) "\n") - nil)))) + (cl-loop for handler in (notmuch-show-handlers-for content-type) + until (condition-case err + (funcall handler msg part content-type nth depth button) + ;; Specifying `debug' here lets the debugger run if + ;; `debug-on-error' is non-nil. + ((debug error) + (insert "!!! Bodypart handler `" (prin1-to-string handler) + "' threw an error:\n" + "!!! " (error-message-string err) "\n") + nil)))) (defun notmuch-show-create-part-overlays (button beg end) - "Add an overlay to the part between BEG and END" - + "Add an overlay to the part between BEG and END." ;; If there is no button (i.e., the part is text/plain and the first ;; part) or if the part has no content then we don't make the part ;; toggleable. @@ -893,8 +996,7 @@ will return nil if the CID is unknown or cannot be retrieved." t)) (defun notmuch-show-record-part-information (part beg end) - "Store PART as a text property from BEG to END" - + "Store PART as a text property from BEG to END." ;; Record part information. Since we already inserted subparts, ;; don't override existing :notmuch-part properties. (notmuch-map-text-property beg end :notmuch-part @@ -905,13 +1007,15 @@ will return nil if the CID is unknown or cannot be retrieved." ;; watch out for sticky specs of t, which means all properties are ;; front-sticky/rear-nonsticky. (notmuch-map-text-property beg end 'front-sticky - (lambda (v) (if (listp v) - (pushnew :notmuch-part v) - v))) + (lambda (v) + (if (listp v) + (cl-pushnew :notmuch-part v) + v))) (notmuch-map-text-property beg end 'rear-nonsticky - (lambda (v) (if (listp v) - (pushnew :notmuch-part v) - v)))) + (lambda (v) + (if (listp v) + (cl-pushnew :notmuch-part v) + v)))) (defun notmuch-show-lazy-part (part-args button) ;; Insert the lazy part after the button for the part. We would just @@ -931,15 +1035,20 @@ will return nil if the CID is unknown or cannot be retrieved." (part-end (copy-marker (point) t)) ;; We have to save the depth as we can't find the depth ;; when narrowed. - (depth (notmuch-show-get-depth))) + (depth (notmuch-show-get-depth)) + (mime-type (plist-get (cadr part-args) :computed-type))) (save-restriction (narrow-to-region part-beg part-end) (delete-region part-beg part-end) + (when (and mime-type (string-match "^image/" mime-type)) + (button-put button :notmuch-redisplay-data part-args)) (apply #'notmuch-show-insert-bodypart-internal part-args) - (indent-rigidly part-beg part-end (* notmuch-show-indent-messages-width depth))) + (indent-rigidly part-beg + part-end + (* notmuch-show-indent-messages-width depth))) (goto-char part-end) (delete-char 1) - (notmuch-show-record-part-information (second part-args) + (notmuch-show-record-part-information (cadr part-args) (button-start button) part-end) ;; Create the overlay. If the lazy-part turned out to be empty/not @@ -948,7 +1057,8 @@ will return nil if the CID is unknown or cannot be retrieved." (defun notmuch-show-mime-type (part) "Return the correct mime-type to use for PART." - (let ((content-type (downcase (plist-get part :content-type)))) + (when-let ((content-type (plist-get part :content-type))) + (setq content-type (downcase content-type)) (or (and (string= content-type "application/octet-stream") (notmuch-show-get-mime-type-of-application/octet-stream part)) (and (string= content-type "inline patch") @@ -963,13 +1073,13 @@ The function should take two parameters, PART and HIDE, and should return non-NIL if a header button should be inserted for this part.") -(defun notmuch-show-insert-header-p (part hide) +(defun notmuch-show-insert-header-p (part _hide) ;; Show all part buttons except for the first part if it is text/plain. (let ((mime-type (notmuch-show-mime-type part))) (not (and (string= mime-type "text/plain") (<= (plist-get part :id) 1))))) -(defun notmuch-show-reply-insert-header-p-never (part hide) +(defun notmuch-show-reply-insert-header-p-never (_part _hide) nil) (defun notmuch-show-reply-insert-header-p-trimmed (part hide) @@ -988,33 +1098,45 @@ this part.") HIDE determines whether to show or hide the part and the button as follows: If HIDE is nil, show the part and the button. If HIDE is t, hide the part initially and show the button." - - (let* ((content-type (downcase (plist-get part :content-type))) + (let* ((content-type (plist-get part :content-type)) (mime-type (notmuch-show-mime-type part)) (nth (plist-get part :id)) + (height (plist-get msg :height)) (long (and (notmuch-match-content-type mime-type "text/*") (> notmuch-show-max-text-part-size 0) - (> (length (plist-get part :content)) notmuch-show-max-text-part-size))) + (> (length (plist-get part :content)) + notmuch-show-max-text-part-size))) + (deep (and notmuch-show-depth-limit + (> depth notmuch-show-depth-limit))) + (high (and notmuch-show-height-limit + (> height notmuch-show-height-limit))) (beg (point)) ;; This default header-p function omits the part button for ;; the first (or only) part if this is text/plain. - (button (when (funcall notmuch-show-insert-header-p-function part hide) - (notmuch-show-insert-part-header nth mime-type content-type (plist-get part :filename)))) - ;; Hide the part initially if HIDE is t, or if it is too long + (button (and (or deep long high + (funcall notmuch-show-insert-header-p-function part hide)) + (notmuch-show-insert-part-header + nth mime-type + (and content-type (downcase content-type)) + (plist-get part :filename)))) + ;; Hide the part initially if HIDE is t, or if it is too long/deep ;; and we have a button to allow toggling. (show-part (not (or (equal hide t) + (and deep button) + (and high button) (and long button)))) - (content-beg (point))) - + (content-beg (point)) + (part-data (list msg part mime-type nth depth button))) ;; Store the computed mime-type for later use (e.g. by attachment handlers). (plist-put part :computed-type mime-type) - - (if show-part - (notmuch-show-insert-bodypart-internal msg part mime-type nth depth button) + (cond + (show-part + (apply #'notmuch-show-insert-bodypart-internal part-data) + (when (and button (string-match "^image/" mime-type)) + (button-put button :notmuch-redisplay-data part-data))) + (t (when button - (button-put button :notmuch-lazy-part - (list msg part mime-type nth depth button)))) - + (button-put button :notmuch-lazy-part part-data)))) ;; Some of the body part handlers leave point somewhere up in the ;; part, so we make sure that we're down at the end. (goto-char (point-max)) @@ -1031,12 +1153,10 @@ is t, hide the part initially and show the button." (defun notmuch-show-insert-body (msg body depth) "Insert the body BODY at depth DEPTH in the current thread." - ;; Register all content IDs for this message. According to RFC ;; 2392, content IDs are *global*, but it's okay if an MUA treats ;; them as only global within a message. - (notmuch-show--register-cids msg (first body)) - + (notmuch-show--register-cids msg (car body)) (mapc (lambda (part) (notmuch-show-insert-bodypart msg part depth)) body)) (defun notmuch-show-make-symbol (type) @@ -1048,6 +1168,40 @@ is t, hide the part initially and show the button." (defvar notmuch-show-previous-subject "") (make-variable-buffer-local 'notmuch-show-previous-subject) +(defun notmuch-show-choose-duplicate (duplicate) + "Display message file with index DUPLICATE in place of the current one. + +Message file indices are based on the order the files are +discovered by `notmuch new' (and hence are somewhat arbitrary), +and correspond to those passed to the \"\\-\\-duplicate\" arguments +to the CLI. + +When called interactively, the function will prompt for the index +of the file to display. An error will be signaled if the index +is out of range." + (interactive "Nduplicate: ") + (let ((count (length (notmuch-show-get-prop :filename)))) + (when (or (> duplicate count) + (< duplicate 1)) + (error "Duplicate %d out of range [1,%d]" duplicate count))) + (notmuch-show-move-to-message-top) + (save-excursion + (let* ((extent (notmuch-show-message-extent)) + (id (notmuch-show-get-message-id)) + (depth (notmuch-show-get-depth)) + (inhibit-read-only t) + (new-msg (notmuch--run-show (list id) duplicate))) + ;; clean up existing overlays to avoid extending them. + (dolist (o (overlays-in (car extent) (cdr extent))) + (delete-overlay o)) + ;; pretend insertion is happening at end of buffer + (narrow-to-region (point-min) (car extent)) + ;; Insert first, then delete, to avoid marker for start of next + ;; message being in same place as the start of this one. + (notmuch-show-insert-msg new-msg depth) + (widen) + (delete-region (point) (cdr extent))))) + (defun notmuch-show-insert-msg (msg depth) "Insert the message MSG at depth DEPTH in the current thread." (let* ((headers (plist-get msg :headers)) @@ -1057,18 +1211,9 @@ is t, hide the part initially and show the button." content-start content-end headers-start headers-end (bare-subject (notmuch-show-strip-re (plist-get headers :Subject)))) - (setq message-start (point-marker)) - - (notmuch-show-insert-headerline headers - (or (if notmuch-show-relative-dates - (plist-get msg :date_relative) - nil) - (plist-get headers :Date)) - (plist-get msg :tags) depth) - + (notmuch-show-insert-headerline msg depth (plist-get msg :tags)) (setq content-start (point-marker)) - ;; Set `headers-start' to point after the 'Subject:' header to be ;; compatible with the existing implementation. This just sets it ;; to after the first header. @@ -1078,14 +1223,11 @@ is t, hide the part initially and show the button." ;; If the subject of this message is the same as that of the ;; previous message, don't display it when this message is ;; collapsed. - (when (not (string= notmuch-show-previous-subject - bare-subject)) + (unless (string= notmuch-show-previous-subject bare-subject) (forward-line 1)) (setq headers-start (point-marker))) (setq headers-end (point-marker)) - (setq notmuch-show-previous-subject bare-subject) - ;; A blank line between the headers and the body. (insert "\n") (notmuch-show-insert-body msg (plist-get msg :body) @@ -1094,37 +1236,35 @@ is t, hide the part initially and show the button." (unless (bolp) (insert "\n")) (setq content-end (point-marker)) - ;; Indent according to the depth in the thread. - (if notmuch-show-indent-content - (indent-rigidly content-start content-end (* notmuch-show-indent-messages-width depth))) - + (when notmuch-show-indent-content + (indent-rigidly content-start + content-end + (* notmuch-show-indent-messages-width depth))) (setq message-end (point-max-marker)) - ;; Save the extents of this message over the whole text of the ;; message. - (put-text-property message-start message-end :notmuch-message-extent (cons message-start message-end)) - + (put-text-property message-start message-end + :notmuch-message-extent + (cons message-start message-end)) ;; Create overlays used to control visibility (plist-put msg :headers-overlay (make-overlay headers-start headers-end)) (plist-put msg :message-overlay (make-overlay headers-start content-end)) - (plist-put msg :depth depth) - ;; Save the properties for this message. Currently this saves the ;; entire message (augmented it with other stuff), which seems ;; like overkill. We might save a reduced subset (for example, not ;; the content). (notmuch-show-set-message-properties msg) - ;; Set header visibility. (notmuch-show-headers-visible msg notmuch-message-headers-visible) - ;; Message visibility depends on whether it matched the search ;; criteria. (notmuch-show-message-visible msg (and (plist-get msg :match) (not (plist-get msg :excluded)))))) +;;; Toggle commands + (defun notmuch-show-toggle-process-crypto () "Toggle the processing of cryptographic MIME parts." (interactive) @@ -1137,7 +1277,8 @@ is t, hide the part initially and show the button." (defun notmuch-show-toggle-elide-non-matching () "Toggle the display of non-matching messages." (interactive) - (setq notmuch-show-elide-non-matching-messages (not notmuch-show-elide-non-matching-messages)) + (setq notmuch-show-elide-non-matching-messages + (not notmuch-show-elide-non-matching-messages)) (message (if notmuch-show-elide-non-matching-messages "Showing matching messages only." "Showing all messages.")) @@ -1152,12 +1293,15 @@ is t, hide the part initially and show the button." "Content is not indented.")) (notmuch-show-refresh-view)) +;;; Main insert functions + (defun notmuch-show-insert-tree (tree depth) "Insert the message tree TREE at depth DEPTH in the current thread." (let ((msg (car tree)) (replies (cadr tree))) ;; We test whether there is a message or just some replies. (when msg + (notmuch-show--mark-height tree) (notmuch-show-insert-msg msg depth)) (notmuch-show-insert-thread replies (1+ depth)))) @@ -1169,6 +1313,8 @@ is t, hide the part initially and show the button." "Insert the forest of threads FOREST." (mapc (lambda (thread) (notmuch-show-insert-thread thread 0)) forest)) +;;; Link buttons + (defvar notmuch-id-regexp (concat ;; Match the id: prefix only if it begins a word (to disallow, for @@ -1218,17 +1364,19 @@ buttons for a corresponding notmuch search." (url-unhex-string (match-string 0 mid-cid))))) (push (list (match-beginning 0) (match-end 0) (notmuch-id-to-query mid)) links))) - (dolist (link links) + (pcase-dolist (`(,beg ,end ,link) links) ;; Remove the overlay created by goto-address-mode - (remove-overlays (first link) (second link) 'goto-address t) - (make-text-button (first link) (second link) + (remove-overlays beg end 'goto-address t) + (make-text-button beg end :type 'notmuch-button-type 'action `(lambda (arg) - (notmuch-show ,(third link) current-prefix-arg)) + (notmuch-show ,link current-prefix-arg)) 'follow-link t 'help-echo "Mouse-1, RET: search for this message" 'face goto-address-mail-face))))) +;;; Show command + ;;;###autoload (defun notmuch-show (thread-id &optional elide-toggle parent-buffer query-context buffer-name) "Run \"notmuch show\" with the given thread ID and display results. @@ -1255,46 +1403,35 @@ matched." (let ((buffer-name (generate-new-buffer-name (or buffer-name (concat "*notmuch-" thread-id "*")))) - ;; We override mm-inline-override-types to stop application/* - ;; parts from being displayed unless the user has customized - ;; it themselves. - (mm-inline-override-types - (if (equal mm-inline-override-types - (eval (car (get 'mm-inline-override-types 'standard-value)))) - (cons "application/*" mm-inline-override-types) - mm-inline-override-types))) - (switch-to-buffer (get-buffer-create buffer-name)) + (mm-inline-override-types (notmuch--inline-override-types))) + + (pop-to-buffer-same-window (get-buffer-create buffer-name)) ;; No need to track undo information for this buffer. (setq buffer-undo-list t) - (notmuch-show-mode) - ;; Set various buffer local variables to their appropriate initial ;; state. Do this after enabling `notmuch-show-mode' so that they ;; aren't wiped out. - (setq notmuch-show-thread-id thread-id - notmuch-show-parent-buffer parent-buffer - notmuch-show-query-context (if (or (string= query-context "") - (string= query-context "*")) - nil query-context) - - notmuch-show-process-crypto notmuch-crypto-process-mime - ;; If `elide-toggle', invert the default value. - notmuch-show-elide-non-matching-messages + (setq notmuch-show-thread-id thread-id) + (setq notmuch-show-parent-buffer parent-buffer) + (setq notmuch-show-query-context + (if (or (string= query-context "") + (string= query-context "*")) + nil + query-context)) + (setq notmuch-show-process-crypto notmuch-crypto-process-mime) + ;; If `elide-toggle', invert the default value. + (setq notmuch-show-elide-non-matching-messages (if elide-toggle (not notmuch-show-only-matching-messages) notmuch-show-only-matching-messages)) - (add-hook 'post-command-hook #'notmuch-show-command-hook nil t) (jit-lock-register #'notmuch-show-buttonise-links) - (notmuch-tag-clear-cache) - (let ((inhibit-read-only t)) (if (notmuch-show--build-buffer) ;; Messages were inserted into the buffer. (current-buffer) - ;; No messages were inserted - presumably none matched the ;; query. (kill-buffer (current-buffer)) @@ -1311,9 +1448,22 @@ and THREAD. The next query is THREAD alone, and serves as a fallback if the prior matches no messages." (let (queries) (push (list thread) queries) - (if context (push (list thread "and (" context ")") queries)) + (when context + (push (list thread "and (" context ")") queries)) queries)) +(defun notmuch-show--header-line-format () + "Compute the header line format of a notmuch-show buffer." + (when notmuch-show-header-line + (let* ((s (notmuch-sanitize + (notmuch-show-strip-re (notmuch-show-get-subject)))) + (subject (replace-regexp-in-string "%" "%%" s))) + (cond ((stringp notmuch-show-header-line) + (format-spec notmuch-show-header-line `((?s . ,subject)))) + ((functionp notmuch-show-header-line) + (funcall notmuch-show-header-line subject)) + (notmuch-show-header-line subject))))) + (defun notmuch-show--build-buffer (&optional state) "Display messages matching the current buffer context. @@ -1321,9 +1471,10 @@ Apply the previously saved STATE if supplied, otherwise show the first relevant message. If no messages match the query return NIL." - (let* ((cli-args (cons "--exclude=false" - (when notmuch-show-elide-non-matching-messages - (list "--entire-thread=false")))) + (let* ((cli-args (list "--exclude=false")) + (cli-args (if notmuch-show-elide-non-matching-messages (cons "--entire-thread=false" cli-args) cli-args)) + ;; "part 0 is the whole message (headers and body)" notmuch-show(1) + (cli-args (if notmuch-show-single-message (cons "--part=0" cli-args) cli-args)) (queries (notmuch-show--build-queries notmuch-show-thread-id notmuch-show-query-context)) (forest nil) @@ -1332,34 +1483,28 @@ If no messages match the query return NIL." (notmuch-show-previous-subject "")) ;; Use results from the first query that returns some. (while (and (not forest) queries) - (setq forest (notmuch-query-get-threads + (setq forest (notmuch--run-show (append cli-args (list "'") (car queries) (list "'")))) + (when (and forest notmuch-show-single-message) + (setq forest (list (list (list forest))))) (setq queries (cdr queries))) (when forest (notmuch-show-insert-forest forest) - ;; Store the original tags for each message so that we can ;; display changes. (notmuch-show-mapc (lambda () (notmuch-show-set-prop :orig-tags (notmuch-show-get-tags)))) - - ;; Set the header line to the subject of the first message. - (setq header-line-format - (replace-regexp-in-string "%" "%%" - (notmuch-sanitize - (notmuch-show-strip-re - (notmuch-show-get-subject))))) - + (setq header-line-format (notmuch-show--header-line-format)) (run-hooks 'notmuch-show-hook) - (if state (notmuch-show-apply-state state) ;; With no state to apply, just go to the first message. (notmuch-show-goto-first-wanted-message))) - ;; Report back to the caller whether any messages matched. forest)) +;;; Refresh command + (defun notmuch-show-capture-state () "Capture the state of the current buffer. @@ -1374,7 +1519,7 @@ This includes: (list win-id-combo (notmuch-show-get-message-ids-for-open-messages)))) (defun notmuch-show-get-query () - "Return the current query in this show buffer" + "Return the current query in this show buffer." (if notmuch-show-query-context (concat notmuch-show-thread-id " and (" @@ -1385,9 +1530,9 @@ This includes: (defun notmuch-show-goto-message (msg-id) "Go to message with msg-id." (goto-char (point-min)) - (unless (loop if (string= msg-id (notmuch-show-get-message-id)) - return t - until (not (notmuch-show-goto-message-next))) + (unless (cl-loop if (string= msg-id (notmuch-show-get-message-id)) + return t + until (not (notmuch-show-goto-message-next))) (goto-char (point-min)) (message "Message-id not found.")) (notmuch-show-message-adjust)) @@ -1401,13 +1546,12 @@ This includes: - moving to the correct current message in every displayed window." (let ((win-msg-alist (car state)) (open (cadr state))) - ;; Open those that were open. (goto-char (point-min)) - (loop do (notmuch-show-message-visible (notmuch-show-get-message-properties) - (member (notmuch-show-get-message-id) open)) - until (not (notmuch-show-goto-message-next))) - + (cl-loop do (notmuch-show-message-visible + (notmuch-show-get-message-properties) + (member (notmuch-show-get-message-id) open)) + until (not (notmuch-show-goto-message-next))) (dolist (win-msg-pair win-msg-alist) (with-selected-window (car win-msg-pair) ;; Go to the previously open message in this window @@ -1422,6 +1566,7 @@ non-nil) then the state of the buffer (open/closed messages) is reset based on the original query." (interactive "P") (let ((inhibit-read-only t) + (mm-inline-override-types (notmuch--inline-override-types)) (state (unless reset-state (notmuch-show-capture-state)))) ;; `erase-buffer' does not seem to remove overlays, which can lead @@ -1429,13 +1574,14 @@ reset based on the original query." ;; manually. (remove-overlays) (erase-buffer) - (unless (notmuch-show--build-buffer state) ;; No messages were inserted. (kill-buffer (current-buffer)) (ding) (message "Refreshing the buffer resulted in no messages!")))) +;;; Keymaps + (defvar notmuch-show-stash-map (let ((map (make-sparse-keymap))) (define-key map "c" 'notmuch-show-stash-cc) @@ -1452,7 +1598,7 @@ reset based on the original query." (define-key map "G" 'notmuch-show-stash-git-send-email) (define-key map "?" 'notmuch-subkeymap-help) map) - "Submap for stash commands") + "Submap for stash commands.") (fset 'notmuch-show-stash-map notmuch-show-stash-map) (defvar notmuch-show-part-map @@ -1464,13 +1610,14 @@ reset based on the original query." (define-key map "m" 'notmuch-show-choose-mime-of-part) (define-key map "?" 'notmuch-subkeymap-help) map) - "Submap for part commands") + "Submap for part commands.") (fset 'notmuch-show-part-map notmuch-show-part-map) (defvar notmuch-show-mode-map (let ((map (make-sparse-keymap))) (set-keymap-parent map notmuch-common-keymap) (define-key map "Z" 'notmuch-tree-from-show-current-query) + (define-key map "U" 'notmuch-unthreaded-from-show-current-query) (define-key map (kbd "") 'widget-backward) (define-key map (kbd "M-TAB") 'notmuch-show-previous-button) (define-key map (kbd "") 'notmuch-show-previous-button) @@ -1508,13 +1655,15 @@ reset based on the original query." (define-key map "#" 'notmuch-show-print-message) (define-key map "!" 'notmuch-show-toggle-elide-non-matching) (define-key map "$" 'notmuch-show-toggle-process-crypto) + (define-key map "%" 'notmuch-show-choose-duplicate) (define-key map "<" 'notmuch-show-toggle-thread-indentation) (define-key map "t" 'toggle-truncate-lines) (define-key map "." 'notmuch-show-part-map) (define-key map "B" 'notmuch-show-browse-urls) map) "Keymap for \"notmuch show\" buffers.") -(fset 'notmuch-show-mode-map notmuch-show-mode-map) + +;;; Mode (define-derived-mode notmuch-show-mode fundamental-mode "notmuch-show" "Major mode for viewing a thread with notmuch. @@ -1545,31 +1694,37 @@ All currently available key bindings: \\{notmuch-show-mode-map}" (setq notmuch-buffer-refresh-function #'notmuch-show-refresh-view) - (setq buffer-read-only t - truncate-lines t) + (setq buffer-read-only t) + (setq truncate-lines t) (setq imenu-prev-index-position-function - #'notmuch-show-imenu-prev-index-position-function) + #'notmuch-show-imenu-prev-index-position-function) (setq imenu-extract-index-name-function - #'notmuch-show-imenu-extract-index-name-function)) + #'notmuch-show-imenu-extract-index-name-function)) + +;;; Tree commands (defun notmuch-tree-from-show-current-query () - "Call notmuch tree with the current query" + "Call notmuch tree with the current query." (interactive) (notmuch-tree notmuch-show-thread-id notmuch-show-query-context (notmuch-show-get-message-id))) +(defun notmuch-unthreaded-from-show-current-query () + "Call notmuch unthreaded with the current query." + (interactive) + (notmuch-unthreaded notmuch-show-thread-id + notmuch-show-query-context + (notmuch-show-get-message-id))) + +;;; Movement related functions. + (defun notmuch-show-move-to-message-top () (goto-char (notmuch-show-message-top))) (defun notmuch-show-move-to-message-bottom () (goto-char (notmuch-show-message-bottom))) -(defun notmuch-show-message-adjust () - (recenter 0)) - -;; Movement related functions. - ;; There's some strangeness here where a text property applied to a ;; region a->b is not found when point is at b. We walk backwards ;; until finding the property. @@ -1610,11 +1765,10 @@ of the current message." effects." (save-excursion (goto-char (point-min)) - (loop do (funcall function) - while (notmuch-show-goto-message-next)))) + (cl-loop do (funcall function) + while (notmuch-show-goto-message-next)))) -;; Functions relating to the visibility of messages and their -;; components. +;;; Functions relating to the visibility of messages and their components. (defun notmuch-show-message-visible (props visible-p) (overlay-put (plist-get props :message-overlay) 'invisible (not visible-p)) @@ -1624,13 +1778,13 @@ effects." (overlay-put (plist-get props :headers-overlay) 'invisible (not visible-p)) (notmuch-show-set-prop :headers-visible visible-p props)) -;; Functions for setting and getting attributes of the current -;; message. +;;; Functions for setting and getting attributes of the current message. (defun notmuch-show-set-message-properties (props) (save-excursion (notmuch-show-move-to-message-top) - (put-text-property (point) (+ (point) 1) :notmuch-message-properties props))) + (put-text-property (point) (+ (point) 1) + :notmuch-message-properties props))) (defun notmuch-show-get-message-properties () "Return the properties of the current message as a plist. @@ -1665,13 +1819,13 @@ It gets property PROP from PROPS or, if PROPS is nil, the current message in either tree or show. This means that several utility functions in notmuch-show can be used directly by notmuch-tree as they just need the correct message properties." - (let ((props (or props - (cond ((eq major-mode 'notmuch-show-mode) - (notmuch-show-get-message-properties)) - ((eq major-mode 'notmuch-tree-mode) - (notmuch-tree-get-message-properties)) - (t nil))))) - (plist-get props prop))) + (plist-get (or props + (cond ((eq major-mode 'notmuch-show-mode) + (notmuch-show-get-message-properties)) + ((eq major-mode 'notmuch-tree-mode) + (notmuch-tree-get-message-properties)) + (t nil))) + prop)) (defun notmuch-show-get-message-id (&optional bare) "Return an id: query for the Message-Id of the current message. @@ -1696,10 +1850,10 @@ current thread." ;; dme: Would it make sense to use a macro for many of these? -;; XXX TODO figure out what to do about multiple filenames (defun notmuch-show-get-filename () "Return the filename of the current message." - (car (notmuch-show-get-prop :filename))) + (let ((duplicate (notmuch-show-get-duplicate))) + (nth (1- duplicate) (notmuch-show-get-prop :filename)))) (defun notmuch-show-get-header (header &optional props) "Return the named header of the current message, if any." @@ -1711,6 +1865,10 @@ current thread." (defun notmuch-show-get-date () (notmuch-show-get-header :Date)) +(defun notmuch-show-get-duplicate () + ;; if no duplicate property exists, assume first file + (or (notmuch-show-get-prop :duplicate) 1)) + (defun notmuch-show-get-timestamp () (notmuch-show-get-prop :timestamp)) @@ -1758,15 +1916,15 @@ marked as unread, i.e. the tag changes in (apply 'notmuch-show-tag-message (notmuch-tag-change-list notmuch-show-mark-read-tags unread)))) -(defun notmuch-show-seen-current-message (start end) +(defun notmuch-show-seen-current-message (_start _end) "Mark the current message read if it is open. We only mark it read once: if it is changed back then that is a user decision and we should not override it." (when (and (notmuch-show-message-visible-p) (not (notmuch-show-get-prop :seen))) - (notmuch-show-mark-read) - (notmuch-show-set-prop :seen t))) + (notmuch-show-mark-read) + (notmuch-show-set-prop :seen t))) (defvar notmuch-show--seen-has-errored nil) (make-variable-buffer-local 'notmuch-show--seen-has-errored) @@ -1776,15 +1934,16 @@ user decision and we should not override it." ;; We need to redisplay to get window-start and window-end correct. (redisplay) (save-excursion - (condition-case err + (condition-case nil (funcall notmuch-show-mark-read-function (window-start) (window-end)) ((debug error) (unless notmuch-show--seen-has-errored - (setq notmuch-show--seen-has-errored 't) + (setq notmuch-show--seen-has-errored t) (setq header-line-format (concat header-line-format - (propertize " [some mark read tag changes may have failed]" - 'face font-lock-warning-face))))))))) + (propertize + " [some mark read tag changes may have failed]" + 'face font-lock-warning-face))))))))) (defun notmuch-show-filter-thread (query) "Filter or LIMIT the current thread based on a new query string. @@ -1792,12 +1951,11 @@ user decision and we should not override it." Reshows the current thread with matches defined by the new query-string." (interactive (list (notmuch-read-query "Filter thread: "))) (let ((msg-id (notmuch-show-get-message-id))) - (setq notmuch-show-query-context (if (string= query "") nil query)) + (setq notmuch-show-query-context (if (string-empty-p query) nil query)) (notmuch-show-refresh-view t) (notmuch-show-goto-message msg-id))) -;; Functions for getting attributes of several messages in the current -;; thread. +;;; Functions for getting attributes of several messages in the current thread. (defun notmuch-show-get-message-ids-for-open-messages () "Return a list of all id: queries for open messages in the current thread." @@ -1805,14 +1963,13 @@ Reshows the current thread with matches defined by the new query-string." (let (message-ids done) (goto-char (point-min)) (while (not done) - (if (notmuch-show-message-visible-p) - (setq message-ids (append message-ids (list (notmuch-show-get-message-id))))) - (setq done (not (notmuch-show-goto-message-next))) - ) - message-ids - ))) + (when (notmuch-show-message-visible-p) + (setq message-ids + (append message-ids (list (notmuch-show-get-message-id))))) + (setq done (not (notmuch-show-goto-message-next)))) + message-ids))) -;; Commands typically bound to keys. +;;; Commands typically bound to keys. (defun notmuch-show-advance () "Advance through thread. @@ -1839,16 +1996,13 @@ current window), advance to the next open message." (> visible-end-of-this-message (window-end))) ;; The bottom of this message is not visible - scroll. (scroll-up nil)) - ((not (= end-of-this-message (point-max))) ;; This is not the last message - move to the next visible one. (notmuch-show-next-open-message)) - ((not (= (point) (point-max))) ;; This is the last message, but the cursor is not at the end of ;; the buffer. Move it there. (goto-char (point-max))) - (t ;; This is the last message - change the return value (setq ret t))) @@ -1866,11 +2020,12 @@ archives the entire current thread, (apply changes in thread from the search from which this thread was originally shown." (interactive) - (if (notmuch-show-advance) - (notmuch-show-archive-thread-then-next))) + (when (notmuch-show-advance) + (notmuch-show-archive-thread-then-next))) (defun notmuch-show-rewind () - "Backup through the thread (reverse scrolling compared to \\[notmuch-show-advance-and-archive]). + "Backup through the thread (reverse scrolling compared to \ +\\[notmuch-show-advance-and-archive]). Specifically, if the beginning of the previous email is fewer than `window-height' lines from the current point, move to it @@ -1885,9 +2040,9 @@ any effects from previous calls to (let ((start-of-message (notmuch-show-message-top)) (start-of-window (window-start))) (cond - ;; Either this message is properly aligned with the start of the - ;; window or the start of this message is not visible on the - ;; screen - scroll. + ;; Either this message is properly aligned with the start of the + ;; window or the start of this message is not visible on the + ;; screen - scroll. ((or (= start-of-message start-of-window) (< start-of-message start-of-window)) (scroll-down) @@ -1908,13 +2063,15 @@ any effects from previous calls to (defun notmuch-show-reply (&optional prompt-for-sender) "Reply to the sender and all recipients of the current message." (interactive "P") - (notmuch-mua-new-reply (notmuch-show-get-message-id) prompt-for-sender t)) + (notmuch-mua-new-reply (notmuch-show-get-message-id) prompt-for-sender t + (notmuch-show-get-prop :duplicate))) (put 'notmuch-show-reply-sender 'notmuch-prefix-doc "... and prompt for sender") (defun notmuch-show-reply-sender (&optional prompt-for-sender) "Reply to the sender of the current message." (interactive "P") - (notmuch-mua-new-reply (notmuch-show-get-message-id) prompt-for-sender nil)) + (notmuch-mua-new-reply (notmuch-show-get-message-id) prompt-for-sender nil + (notmuch-show-get-prop :duplicate))) (put 'notmuch-show-forward-message 'notmuch-prefix-doc "... and prompt for sender") @@ -1942,6 +2099,9 @@ any effects from previous calls to (message-resend addresses) (notmuch-bury-or-kill-this-buffer))) +(defun notmuch-show-message-adjust () + (recenter 0)) + (defun notmuch-show-next-message (&optional pop-at-end) "Show the next message. @@ -1996,7 +2156,7 @@ to show, nil otherwise." (notmuch-show-message-visible props (plist-get props :match)))) (defun notmuch-show-goto-first-wanted-message () - "Move to the first open message and mark it read" + "Move to the first open message and mark it read." (goto-char (point-min)) (unless (notmuch-show-message-visible-p) (notmuch-show-next-open-message)) @@ -2022,12 +2182,16 @@ to show, nil otherwise." "View the original source of the current message." (interactive) (let* ((id (notmuch-show-get-message-id)) - (buf (get-buffer-create (concat "*notmuch-raw-" id "*"))) + (duplicate (notmuch-show-get-duplicate)) + (args (if (> duplicate 1) + (list (format "--duplicate=%d" duplicate) id) + (list id))) + (buf (get-buffer-create (format "*notmuch-raw-%s-%d*" id duplicate))) (inhibit-read-only t)) - (switch-to-buffer buf) + (pop-to-buffer-same-window buf) (erase-buffer) (let ((coding-system-for-read 'no-conversion)) - (call-process notmuch-command nil t nil "show" "--format=raw" id)) + (apply #'notmuch--call-process notmuch-command nil t nil "show" "--format=raw" args)) (goto-char (point-min)) (set-buffer-modified-p nil) (setq buffer-read-only t) @@ -2056,33 +2220,36 @@ message." (interactive (let ((query-string (if current-prefix-arg "Pipe all open messages to command: " "Pipe message to command: "))) - (list current-prefix-arg (read-string query-string)))) + (list current-prefix-arg (read-shell-command query-string)))) (let (shell-command) (if entire-thread (setq shell-command (concat notmuch-command " show --format=mbox --exclude=false " (shell-quote-argument - (mapconcat 'identity (notmuch-show-get-message-ids-for-open-messages) " OR ")) + (mapconcat 'identity + (notmuch-show-get-message-ids-for-open-messages) + " OR ")) " | " command)) (setq shell-command (concat notmuch-command " show --format=raw " - (shell-quote-argument (notmuch-show-get-message-id)) " | " command))) + (shell-quote-argument (notmuch-show-get-message-id)) + " | " command))) (let ((cwd default-directory) (buf (get-buffer-create (concat "*notmuch-pipe*")))) (with-current-buffer buf - (setq buffer-read-only nil) - (erase-buffer) - ;; Use the originating buffer's working directory instead of - ;; that of the pipe buffer. - (cd cwd) - (let ((exit-code (call-process-shell-command shell-command nil buf))) - (goto-char (point-max)) - (set-buffer-modified-p nil) - (setq buffer-read-only t) - (unless (zerop exit-code) - (switch-to-buffer-other-window buf) - (message (format "Command '%s' exited abnormally with code %d" - shell-command exit-code)))))))) + (setq buffer-read-only t) + (let ((inhibit-read-only t)) + (erase-buffer) + ;; Use the originating buffer's working directory instead of + ;; that of the pipe buffer. + (cd cwd) + (let ((exit-code (call-process-shell-command shell-command nil buf))) + (goto-char (point-max)) + (set-buffer-modified-p nil) + (unless (zerop exit-code) + (pop-to-buffer buf) + (message (format "Command '%s' exited abnormally with code %d" + shell-command exit-code))))))))) (defun notmuch-show-tag-message (&rest tag-changes) "Change tags for the current message. @@ -2167,9 +2334,10 @@ argument, hide all of the messages." (interactive) (save-excursion (goto-char (point-min)) - (loop do (notmuch-show-message-visible (notmuch-show-get-message-properties) - (not current-prefix-arg)) - until (not (notmuch-show-goto-message-next)))) + (cl-loop do (notmuch-show-message-visible + (notmuch-show-get-message-properties) + (not current-prefix-arg)) + until (not (notmuch-show-goto-message-next)))) (force-window-update)) (defun notmuch-show-next-button () @@ -2188,7 +2356,9 @@ argument, hide all of the messages." If SHOW is non-nil, open the next item in a show buffer. Otherwise just highlight the next item in the search buffer. If PREVIOUS is non-nil, move to the previous item in the -search results instead." +search results instead. + +Return non-nil on success." (interactive "P") (let ((parent-buffer notmuch-show-parent-buffer)) (notmuch-bury-or-kill-this-buffer) @@ -2337,10 +2507,12 @@ kill-ring." (defun notmuch-show-stash-mlarchive-link (&optional mla) "Copy an ML Archive URI for the current message to the kill-ring. -This presumes that the message is available at the selected Mailing List Archive. +This presumes that the message is available at the selected +Mailing List Archive. -If optional argument MLA is non-nil, use the provided key instead of prompting -the user (see `notmuch-show-stash-mlarchive-link-alist')." +If optional argument MLA is non-nil, use the provided key instead +of prompting the user (see +`notmuch-show-stash-mlarchive-link-alist')." (interactive) (let ((url (cdr (assoc (or mla @@ -2357,18 +2529,23 @@ the user (see `notmuch-show-stash-mlarchive-link-alist')." (concat url (notmuch-show-get-message-id t)))))) (defun notmuch-show-stash-mlarchive-link-and-go (&optional mla) - "Copy an ML Archive URI for the current message to the kill-ring and visit it. + "Copy an ML Archive URI for the current message to the + kill-ring and visit it. -This presumes that the message is available at the selected Mailing List Archive. +This presumes that the message is available at the selected +Mailing List Archive. -If optional argument MLA is non-nil, use the provided key instead of prompting -the user (see `notmuch-show-stash-mlarchive-link-alist')." +If optional argument MLA is non-nil, use the provided key instead +of prompting the user (see +`notmuch-show-stash-mlarchive-link-alist')." (interactive) (notmuch-show-stash-mlarchive-link mla) (browse-url (current-kill 0 t))) (defun notmuch-show-stash-git-helper (addresses prefix) - "Escape, trim, quote, and add PREFIX to each address in list of ADDRESSES, and return the result as a single string." + "Normalize all ADDRESSES while adding PREFIX. +Escape, trim, quote and add PREFIX to each address in list +of ADDRESSES, and return the result as a single string." (mapconcat (lambda (x) (concat prefix "\"" ;; escape double-quotes @@ -2381,10 +2558,12 @@ the user (see `notmuch-show-stash-mlarchive-link-alist')." addresses " ")) (put 'notmuch-show-stash-git-send-email 'notmuch-prefix-doc - "Copy From/To/Cc of current message to kill-ring in a form suitable for pasting to git send-email command line.") + "Copy From/To/Cc of current message to kill-ring. +Use a form suitable for pasting to git send-email command line.") (defun notmuch-show-stash-git-send-email (&optional no-in-reply-to) - "Copy From/To/Cc/Message-Id of current message to kill-ring in a form suitable for pasting to git send-email command line. + "Copy From/To/Cc/Message-Id of current message to kill-ring. +Use a form suitable for pasting to git send-email command line. If invoked with a prefix argument (or NO-IN-REPLY-TO is non-nil), omit --in-reply-to=." @@ -2404,7 +2583,7 @@ omit --in-reply-to=." (list (notmuch-show-get-message-id t)) "--in-reply-to=")))) " "))) -;; Interactive part functions and their helpers +;;; Interactive part functions and their helpers (defun notmuch-show-generate-part-buffer (msg part) "Return a temporary buffer containing the specified part's content." @@ -2427,7 +2606,7 @@ MIME-TYPE is given then set the handle's mime-type to MIME-TYPE." (buf (notmuch-show-generate-part-buffer msg part)) (computed-type (or mime-type (plist-get part :computed-type))) (filename (plist-get part :filename)) - (disposition (if filename `(attachment (filename . ,filename))))) + (disposition (and filename `(attachment (filename . ,filename))))) (mm-make-handle buf (list computed-type) nil nil disposition))) (defun notmuch-show-apply-to-current-part-handle (fn &optional mime-type) @@ -2437,10 +2616,9 @@ This ensures that the temporary buffer created for the mm-handle is destroyed when FN returns. If MIME-TYPE is given then force part to be treated as if it had that mime-type." (let ((handle (notmuch-show-current-part-handle mime-type))) - ;; emacs 24.3+ puts stdout/stderr into the calling buffer so we - ;; call it from a temp-buffer, unless - ;; notmuch-show-attachment-debug is non-nil in which case we put - ;; it in " *notmuch-part*". + ;; Emacs puts stdout/stderr into the calling buffer so we call + ;; it from a temp-buffer, unless notmuch-show-attachment-debug + ;; is non-nil, in which case we put it in " *notmuch-part*". (unwind-protect (if notmuch-show-attachment-debug (with-current-buffer (generate-new-buffer " *notmuch-part*") @@ -2480,7 +2658,6 @@ part to be treated as if it had that mime-type." (interactive) (notmuch-show-apply-to-current-part-handle #'mm-pipe-part)) - (defun notmuch-show--mm-display-part (handle) "Use mm-display-part to display HANDLE in a new buffer. @@ -2488,7 +2665,7 @@ If the part is displayed in an external application then close the new buffer." (let ((buf (get-buffer-create (generate-new-buffer-name (concat " *notmuch-internal-part*"))))) - (switch-to-buffer buf) + (pop-to-buffer-same-window buf) (if (eq (mm-display-part handle) 'external) (kill-buffer buf) (goto-char (point-min)) @@ -2496,11 +2673,12 @@ the new buffer." (view-buffer buf 'kill-buffer-if-not-modified)))) (defun notmuch-show-choose-mime-of-part (mime-type) - "Choose the mime type to use for displaying part" + "Choose the mime type to use for displaying part." (interactive (list (completing-read "Mime type to use (default text/plain): " (mailcap-mime-types) nil nil nil nil "text/plain"))) - (notmuch-show-apply-to-current-part-handle #'notmuch-show--mm-display-part mime-type)) + (notmuch-show-apply-to-current-part-handle #'notmuch-show--mm-display-part + mime-type)) (defun notmuch-show-imenu-prev-index-position-function () "Move point to previous message in notmuch-show buffer. @@ -2527,9 +2705,9 @@ beginning of the line." message." `(save-excursion (save-restriction - (let ((extent (notmuch-show-message-extent))) - (narrow-to-region (car extent) (cdr extent)) - ,@body)))) + (let ((extent (notmuch-show-message-extent))) + (narrow-to-region (car extent) (cdr extent)) + ,@body)))) (defun notmuch-show--gather-urls () "Gather any URLs in the current message." @@ -2540,14 +2718,20 @@ message." (push (match-string-no-properties 0) urls)) (reverse urls)))) -(defun notmuch-show-browse-urls () - "Offer to browse any URLs in the current message." - (interactive) - (let ((urls (notmuch-show--gather-urls))) +(defun notmuch-show-browse-urls (&optional kill) + "Offer to browse any URLs in the current message. +With a prefix argument, copy the URL to the kill ring rather than +browsing." + (interactive "P") + (let ((urls (notmuch-show--gather-urls)) + (prompt (if kill "Copy URL to kill ring: " "Browse URL: ")) + (fn (if kill #'kill-new #'browse-url))) (if urls - (browse-url (completing-read "Browse URL: " (cdr urls) nil nil (car urls))) + (funcall fn (completing-read prompt urls nil nil nil nil (car urls))) (message "No URLs found.")))) +;;; _ + (provide 'notmuch-show) ;;; notmuch-show.el ends here diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el index 0500927d..81101828 100644 --- a/emacs/notmuch-tag.el +++ b/emacs/notmuch-tag.el @@ -1,4 +1,4 @@ -;;; notmuch-tag.el --- tag messages within emacs +;;; notmuch-tag.el --- tag messages within emacs -*- lexical-binding: t -*- ;; ;; Copyright © Damien Cassou ;; Copyright © Carl Worth @@ -20,19 +20,20 @@ ;; ;; Authors: Carl Worth ;; Damien Cassou -;; + ;;; Code: -;; -(require 'cl) (require 'crm) + (require 'notmuch-lib) -(declare-function notmuch-search-tag "notmuch" tag-changes) -(declare-function notmuch-show-tag "notmuch-show" tag-changes) -(declare-function notmuch-tree-tag "notmuch-tree" tag-changes) +(declare-function notmuch-search-tag "notmuch" + (tag-changes &optional beg end only-matched)) +(declare-function notmuch-show-tag "notmuch-show" (tag-changes)) +(declare-function notmuch-tree-tag "notmuch-tree" (tag-changes)) +(declare-function notmuch-jump "notmuch-jump" (action-map prompt)) -(autoload 'notmuch-jump "notmuch-jump") +;;; Keys (define-widget 'notmuch-tag-key-type 'list "A single key tagging binding." @@ -40,7 +41,9 @@ :args '((list :inline t :format "%v" (key-sequence :tag "Key") - (radio :tag "Tag operations" (repeat :tag "Tag list" (string :format "%v" :tag "change")) + (radio :tag "Tag operations" + (repeat :tag "Tag list" + (string :format "%v" :tag "change")) (variable :tag "Tag variable")) (string :tag "Name")))) @@ -63,15 +66,15 @@ The key `notmuch-tag-jump-reverse-key' (k by default) should not be used (either as a key, or as the start of a key sequence) as it is already bound: it switches the menu to a menu of the reverse tagging operations. The reverse of a tagging operation is -the same list of individual tag-ops but with `+tag` replaced by -`-tag` and vice versa. +the same list of individual tag-ops but with `+tag' replaced by +`-tag' and vice versa. If setting this variable outside of customize then it should be a list of triples (lists of three elements). Each triple should be of the form (key-binding tagging-operations name). KEY-BINDING can be a single character or a key sequence; TAGGING-OPERATIONS should either be a list of individual tag operations each of the -form `+tag` or `-tag`, or the variable name of a variable that is +form `+tag' or `-tag', or the variable name of a variable that is a list of tagging operations; NAME should be a name for the tagging operation, if omitted or empty than then name is taken from TAGGING-OPERATIONS." @@ -79,8 +82,10 @@ from TAGGING-OPERATIONS." :type '(repeat notmuch-tag-key-type) :group 'notmuch-tag) +;;; Faces and Formats + (define-widget 'notmuch-tag-format-type 'lazy - "Customize widget for notmuch-tag-format and friends" + "Customize widget for notmuch-tag-format and friends." :type '(alist :key-type (regexp :tag "Tag") :extra-offset -3 :value-type @@ -111,7 +116,7 @@ from TAGGING-OPERATIONS." '((t :foreground "red")) "Default face used for the unread tag. -Used in the default value of `notmuch-tag-formats`." +Used in the default value of `notmuch-tag-formats'." :group 'notmuch-faces) (defface notmuch-tag-flagged @@ -123,7 +128,7 @@ Used in the default value of `notmuch-tag-formats`." (:foreground "blue"))) "Face used for the flagged tag. -Used in the default value of `notmuch-tag-formats`." +Used in the default value of `notmuch-tag-formats'." :group 'notmuch-faces) (defcustom notmuch-tag-formats @@ -132,26 +137,29 @@ Used in the default value of `notmuch-tag-formats`." (notmuch-tag-format-image-data tag (notmuch-tag-star-icon)))) "Custom formats for individual tags. -This is an association list that maps from tag name regexps to -lists of formatting expressions. The first entry whose car -regexp-matches a tag will be used to format that tag. The regexp -is implicitly anchored, so to match a literal tag name, just use -that tag name (if it contains special regexp characters like -\".\" or \"*\", these have to be escaped). The cdr of the -matching entry gives a list of Elisp expressions that modify the -tag. If the list is empty, the tag will simply be hidden. -Otherwise, each expression will be evaluated in order: for the -first expression, the variable `tag' will be bound to the tag -name; for each later expression, the variable `tag' will be bound -to the result of the previous expression. In this way, each +This is an association list of the form ((MATCH EXPR...)...), +mapping tag name regexps to lists of formatting expressions. + +The first entry whose MATCH regexp-matches a tag is used to +format that tag. The regexp is implicitly anchored, so to match +a literal tag name, just use that tag name (if it contains +special regexp characters like \".\" or \"*\", these have to be +escaped). + +The cdr of the matching entry gives a list of Elisp expressions +that modify the tag. If the list is empty, the tag is simply +hidden. Otherwise, each expression EXPR is evaluated in order: +for the first expression, the variable `tag' is bound to the tag +name; for each later expression, the variable `tag' is bound to +the result of the previous expression. In this way, each expression can build on the formatting performed by the previous -expression. The result of the last expression will displayed in +expression. The result of the last expression is displayed in place of the tag. For example, to replace a tag with another string, simply use that string as a formatting expression. To change the foreground of a tag to red, use the expression - (propertize tag 'face '(:foreground \"red\")) + (propertize tag \\='face \\='(:foreground \"red\")) See also `notmuch-tag-format-image', which can help replace tags with images." @@ -165,7 +173,7 @@ with images." (t :inverse-video t)) "Face used to display deleted tags. -Used in the default value of `notmuch-tag-deleted-formats`." +Used in the default value of `notmuch-tag-deleted-formats'." :group 'notmuch-faces) (defcustom notmuch-tag-deleted-formats @@ -173,7 +181,7 @@ Used in the default value of `notmuch-tag-deleted-formats`." (".*" (notmuch-apply-face tag `notmuch-tag-deleted))) "Custom formats for tags when deleted. -For deleted tags the formats in `notmuch-tag-formats` are applied +For deleted tags the formats in `notmuch-tag-formats' are applied first and then these formats are applied on top; that is `tag' passed to the function is the tag with all these previous formattings applied. The formatted can access the original @@ -183,7 +191,7 @@ By default this shows deleted tags with strike-through in red, unless strike-through is not available (e.g., emacs is running in a terminal) in which case it uses inverse video. To hide deleted tags completely set this to - '((\".*\" nil)) + \\='((\".*\" nil)) See `notmuch-tag-formats' for full documentation." :group 'notmuch-show @@ -194,14 +202,14 @@ See `notmuch-tag-formats' for full documentation." '((t :underline "green")) "Default face used for added tags. -Used in the default value for `notmuch-tag-added-formats`." +Used in the default value for `notmuch-tag-added-formats'." :group 'notmuch-faces) (defcustom notmuch-tag-added-formats '((".*" (notmuch-apply-face tag 'notmuch-tag-added))) "Custom formats for tags when added. -For added tags the formats in `notmuch-tag-formats` are applied +For added tags the formats in `notmuch-tag-formats' are applied first and then these formats are applied on top. To disable special formatting of added tags, set this variable to @@ -212,6 +220,8 @@ See `notmuch-tag-formats' for full documentation." :group 'notmuch-faces :type 'notmuch-tag-format-type) +;;; Icons + (defun notmuch-tag-format-image-data (tag data) "Replace TAG with image DATA, if available. @@ -230,8 +240,8 @@ DATA is the content of an SVG picture (e.g., as returned by (defun notmuch-tag-star-icon () "Return SVG data representing a star icon. This can be used with `notmuch-tag-format-image-data'." -" - + " + - + - + ") +;;; track history of tag operations +(defvar-local notmuch-tag-history nil + "Buffer local history of `notmuch-tag' function.") +(put 'notmuch-tag-history 'permanent-local t) + +;;; Format Handling + (defvar notmuch-tag--format-cache (make-hash-table :test 'equal) "Cache of tag format lookup. Internal to `notmuch-tag-format-tag'.") @@ -272,32 +289,34 @@ This can be used with `notmuch-tag-format-image-data'." "Clear the internal cache of tag formats." (clrhash notmuch-tag--format-cache)) -(defun notmuch-tag--get-formats (tag format-alist) +(defun notmuch-tag--get-formats (tag alist) "Find the first item whose car regexp-matches TAG." (save-match-data ;; Don't use assoc-default since there's no way to distinguish a ;; missing key from a present key with a null cdr. - (assoc* tag format-alist - :test (lambda (tag key) - (and (eq (string-match key tag) 0) - (= (match-end 0) (length tag))))))) + (cl-assoc tag alist + :test (lambda (tag key) + (and (eq (string-match key tag) 0) + (= (match-end 0) (length tag))))))) -(defun notmuch-tag--do-format (tag formatted-tag formats) +(defun notmuch-tag--do-format (bare-tag tag formats) "Apply a tag-formats entry to TAG." (cond ((null formats) ;; - Tag not in `formats', - formatted-tag) ;; the format is the tag itself. + tag) ;; the format is the tag itself. ((null (cdr formats)) ;; - Tag was deliberately hidden, nil) ;; no format must be returned (t ;; Tag was found and has formats, we must apply all the ;; formats. TAG may be null so treat that as a special case. - (let ((bare-tag tag) - (tag (copy-sequence (or formatted-tag "")))) + (let ((return-tag (copy-sequence (or tag "")))) (dolist (format (cdr formats)) - (setq tag (eval format))) - (if (and (null formatted-tag) (equal tag "")) + (setq return-tag + (eval format + `((bare-tag . ,bare-tag) + (tag . ,return-tag))))) + (if (and (null tag) (equal return-tag "")) nil - tag))))) + return-tag))))) (defun notmuch-tag-format-tag (tags orig-tags tag) "Format TAG according to `notmuch-tag-formats'. @@ -309,13 +328,15 @@ are not in TAGS) are shown using formats from are in TAGS but are not in ORIG-TAGS) are shown using formats from `notmuch-tag-added-formats' and tags which have not been changed (the normal case) are shown using formats from -`notmuch-tag-formats'" +`notmuch-tag-formats'." (let* ((tag-state (cond ((not (member tag tags)) 'deleted) ((not (member tag orig-tags)) 'added))) - (formatted-tag (gethash (cons tag tag-state) notmuch-tag--format-cache 'missing))) + (formatted-tag (gethash (cons tag tag-state) + notmuch-tag--format-cache + 'missing))) (when (eq formatted-tag 'missing) (let ((base (notmuch-tag--get-formats tag notmuch-tag-formats)) - (over (case tag-state + (over (cl-case tag-state (deleted (notmuch-tag--get-formats tag notmuch-tag-deleted-formats)) (added (notmuch-tag--get-formats @@ -323,7 +344,6 @@ changed (the normal case) are shown using formats from (otherwise nil)))) (setq formatted-tag (notmuch-tag--do-format tag tag base)) (setq formatted-tag (notmuch-tag--do-format tag formatted-tag over)) - (puthash (cons tag tag-state) formatted-tag notmuch-tag--format-cache))) formatted-tag)) @@ -334,21 +354,22 @@ changed (the normal case) are shown using formats from (notmuch-apply-face (mapconcat #'identity ;; nil indicated that the tag was deliberately hidden - (delq nil (mapcar - (apply-partially #'notmuch-tag-format-tag tags orig-tags) - all-tags)) + (delq nil (mapcar (apply-partially #'notmuch-tag-format-tag + tags orig-tags) + all-tags)) " ") face t))) +;;; Hooks + (defcustom notmuch-before-tag-hook nil "Hooks that are run before tags of a message are modified. -'tag-changes' will contain the tags that are about to be added or removed as +`tag-changes' will contain the tags that are about to be added or removed as a list of strings of the form \"+TAG\" or \"-TAG\". -'query' will be a string containing the search query that determines -the messages that are about to be tagged" - +`query' will be a string containing the search query that determines +the messages that are about to be tagged." :type 'hook :options '(notmuch-hl-line-mode) :group 'notmuch-hooks) @@ -356,49 +377,49 @@ the messages that are about to be tagged" (defcustom notmuch-after-tag-hook nil "Hooks that are run after tags of a message are modified. -'tag-changes' will contain the tags that were added or removed as +`tag-changes' will contain the tags that were added or removed as a list of strings of the form \"+TAG\" or \"-TAG\". -'query' will be a string containing the search query that determines -the messages that were tagged" +`query' will be a string containing the search query that determines +the messages that were tagged." :type 'hook :options '(notmuch-hl-line-mode) :group 'notmuch-hooks) +;;; User Input + (defvar notmuch-select-tag-history nil - "Variable to store minibuffer history for -`notmuch-select-tag-with-completion' function.") + "Minibuffer history of `notmuch-select-tag-with-completion' function.") (defvar notmuch-read-tag-changes-history nil - "Variable to store minibuffer history for -`notmuch-read-tag-changes' function.") + "Minibuffer history of `notmuch-read-tag-changes' function.") (defun notmuch-tag-completions (&rest search-terms) "Return a list of tags for messages matching SEARCH-TERMS. -Returns all tags if no search terms are given." - (if (null search-terms) - (setq search-terms (list "*"))) +Return all tags if no search terms are given." + (unless search-terms + (setq search-terms (list "*"))) (split-string (with-output-to-string (with-current-buffer standard-output - (apply 'call-process notmuch-command nil t + (apply 'notmuch--call-process notmuch-command nil t nil "search" "--output=tags" "--exclude=false" search-terms))) "\n+" t)) (defun notmuch-select-tag-with-completion (prompt &rest search-terms) - (let ((tag-list (apply #'notmuch-tag-completions search-terms))) - (completing-read prompt tag-list nil nil nil 'notmuch-select-tag-history))) + (completing-read prompt + (apply #'notmuch-tag-completions search-terms) + nil nil nil 'notmuch-select-tag-history)) (defun notmuch-read-tag-changes (current-tags &optional prompt initial-input) "Prompt for tag changes in the minibuffer. -CURRENT-TAGS is a list of tags that are present on the message or -messages to be changed. These are offered as tag removal +CURRENT-TAGS is a list of tags that are present on the message +or messages to be changed. These are offered as tag removal completions. CURRENT-TAGS may contain duplicates. PROMPT, if non-nil, is the query string to present in the minibuffer. It defaults to \"Tags\". INITIAL-INPUT, if non-nil, will be the initial input in the minibuffer." - (let* ((all-tag-list (notmuch-tag-completions)) (add-tag-list (mapcar (apply-partially 'concat "+") all-tag-list)) (remove-tag-list (mapcar (apply-partially 'concat "-") current-tags)) @@ -413,17 +434,11 @@ initial input in the minibuffer." (set-keymap-parent map crm-local-completion-map) (define-key map " " 'self-insert-command) map))) - (delete "" (completing-read-multiple - prompt - ;; Append the separator to each completion so when the - ;; user completes a tag they can immediately begin - ;; entering another. `completing-read-multiple' - ;; ultimately splits the input on crm-separator, so we - ;; don't need to strip this back off (we just need to - ;; delete "empty" entries caused by trailing spaces). - (mapcar (lambda (tag-op) (concat tag-op crm-separator)) tag-list) - nil nil initial-input - 'notmuch-read-tag-changes-history)))) + (completing-read-multiple prompt tag-list + nil nil initial-input + 'notmuch-read-tag-changes-history))) + +;;; Tagging (defun notmuch-update-tags (tags tag-changes) "Return a copy of TAGS with additions and removals from TAG-CHANGES. @@ -434,9 +449,9 @@ present or a \"-\" to indicate that the tag should be removed from TAGS if present." (let ((result-tags (copy-sequence tags))) (dolist (tag-change tag-changes) - (let ((op (string-to-char tag-change)) - (tag (unless (string= tag-change "") (substring tag-change 1)))) - (case op + (let ((tag (and (not (string-empty-p tag-change)) + (substring tag-change 1)))) + (cl-case (aref tag-change 0) (?+ (unless (member tag result-tags) (push tag result-tags))) (?- (setq result-tags (delete tag result-tags))) @@ -448,37 +463,59 @@ from TAGS if present." "Use batch tagging if the tagging query is longer than this. This limits the length of arguments passed to the notmuch CLI to -avoid system argument length limits and performance problems.") +avoid system argument length limits and performance problems. -(defun notmuch-tag (query tag-changes) +NOTE: this variable is no longer used.") + +(make-obsolete-variable 'notmuch-tag-argument-limit nil "notmuch 0.36") + +(defun notmuch-tag (query tag-changes &optional omit-hist) "Add/remove tags in TAG-CHANGES to messages matching QUERY. QUERY should be a string containing the search-terms. -TAG-CHANGES is a list of strings of the form \"+tag\" or -\"-tag\" to add or remove tags, respectively. +TAG-CHANGES is a list of strings of the form \"+tag\" or \"-tag\" +to add or remove tags, respectively. OMIT-HIST disables history +tracking if non-nil. Note: Other code should always use this function to alter tags of messages instead of running (notmuch-call-notmuch-process \"tag\" ..) directly, so that hooks specified in notmuch-before-tag-hook and notmuch-after-tag-hook will be run." ;; Perform some validation - (mapc (lambda (tag-change) - (unless (string-match-p "^[-+]\\S-+$" tag-change) - (error "Tag must be of the form `+this_tag' or `-that_tag'"))) - tag-changes) + (dolist (tag-change tag-changes) + (unless (string-match-p "^[-+]\\S-+$" tag-change) + (error "Tag must be of the form `+this_tag' or `-that_tag'"))) (unless query (error "Nothing to tag!")) - (unless (null tag-changes) - (run-hooks 'notmuch-before-tag-hook) - (if (<= (length query) notmuch-tag-argument-limit) - (apply 'notmuch-call-notmuch-process "tag" - (append tag-changes (list "--" query))) - ;; Use batch tag mode to avoid argument length limitations - (let ((batch-op (concat (mapconcat #'notmuch-hex-encode tag-changes " ") - " -- " query))) - (notmuch-call-notmuch-process :stdin-string batch-op "tag" "--batch"))) + (when tag-changes + (notmuch-dlet ((tag-changes tag-changes) + (query query)) + (run-hooks 'notmuch-before-tag-hook)) + (with-temp-buffer + (insert (concat (mapconcat #'notmuch-hex-encode tag-changes " ") " -- " query)) + (unless (= 0 + (notmuch--call-process-region + (point-min) (point-max) notmuch-command t t nil "tag" "--batch")) + (notmuch-logged-error "notmuch tag failed" (buffer-string)))) + (unless omit-hist + (push (list :query query :tag-changes tag-changes) notmuch-tag-history))) + (notmuch-dlet ((tag-changes tag-changes) + (query query)) (run-hooks 'notmuch-after-tag-hook))) +(defun notmuch-tag-undo () + "Undo the previous tagging operation in the current buffer. Uses +buffer local variable `notmuch-tag-history' to determine what +that operation was." + (interactive) + (when (null notmuch-tag-history) + (error "no further notmuch undo information")) + (let* ((action (pop notmuch-tag-history)) + (query (plist-get action :query)) + (changes (notmuch-tag-change-list (plist-get action :tag-changes) t))) + (notmuch-tag query changes t)) + (notmuch-refresh-this-buffer)) + (defun notmuch-tag-change-list (tags &optional reverse) "Convert TAGS into a list of tag changes. @@ -502,7 +539,7 @@ begin with a \"+\" or a \"-\". If REVERSE is non-nil, replace all Creates and displays a jump menu for the tagging operations specified in `notmuch-tagging-keys'. If REVERSE is set then it offers a menu of the reverses of the operations specified in -`notmuch-tagging-keys'; i.e. each `+tag` is replaced by `-tag` +`notmuch-tagging-keys'; i.e. each `+tag' is replaced by `-tag' and vice versa." ;; In principle this function is simple, but it has to deal with ;; lots of cases: different modes (search/show/tree), whether a name @@ -511,28 +548,28 @@ and vice versa." ;; REVERSE is specified. (interactive "P") (let (action-map) - (dolist (binding notmuch-tagging-keys) - (let* ((tag-function (case major-mode + (pcase-dolist (`(,key ,tag ,name) notmuch-tagging-keys) + (let* ((tag-function (cl-case major-mode (notmuch-search-mode #'notmuch-search-tag) (notmuch-show-mode #'notmuch-show-tag) (notmuch-tree-mode #'notmuch-tree-tag))) - (key (first binding)) - (forward-tag-change (if (symbolp (second binding)) - (symbol-value (second binding)) - (second binding))) + (tag (if (symbolp tag) + (symbol-value tag) + tag)) (tag-change (if reverse - (notmuch-tag-change-list forward-tag-change 't) - forward-tag-change)) - (name (or (and (not (string= (third binding) "")) - (third binding)) - (and (symbolp (second binding)) - (symbol-name (second binding))))) + (notmuch-tag-change-list tag t) + tag)) + (name (or (and (not (string= name "")) + name) + (and (symbolp name) + (symbol-name name)))) (name-string (if name - (if reverse (concat "Reverse " name) + (if reverse + (concat "Reverse " name) name) (mapconcat #'identity tag-change " ")))) (push (list key name-string - `(lambda () (,tag-function ',tag-change))) + (lambda () (funcall tag-function tag-change))) action-map))) (push (list notmuch-tag-jump-reverse-key (if reverse @@ -543,10 +580,8 @@ and vice versa." (setq action-map (nreverse action-map)) (notmuch-jump action-map "Tag: "))) -;; +;;; _ (provide 'notmuch-tag) -;; Local Variables: -;; byte-compile-warnings: (not cl-functions) -;; End: +;;; notmuch-tag.el ends here diff --git a/emacs/notmuch-tree.el b/emacs/notmuch-tree.el index c00315e8..faec89c4 100644 --- a/emacs/notmuch-tree.el +++ b/emacs/notmuch-tree.el @@ -1,4 +1,4 @@ -;;; notmuch-tree.el --- displaying notmuch forests. +;;; notmuch-tree.el --- displaying notmuch forests -*- lexical-binding: t -*- ;; ;; Copyright © Carl Worth ;; Copyright © David Edmondson @@ -27,20 +27,29 @@ (require 'mail-parse) (require 'notmuch-lib) -(require 'notmuch-query) (require 'notmuch-show) (require 'notmuch-tag) (require 'notmuch-parser) +(require 'notmuch-jump) -(eval-when-compile (require 'cl)) -(declare-function notmuch-search "notmuch" (&optional query oldest-first target-thread target-line)) -(declare-function notmuch-call-notmuch-process "notmuch" (&rest args)) +(declare-function notmuch-search "notmuch" + (&optional query oldest-first target-thread target-line + no-display)) +(declare-function notmuch-call-notmuch-process "notmuch-lib" (&rest args)) (declare-function notmuch-read-query "notmuch" (prompt)) (declare-function notmuch-search-find-thread-id "notmuch" (&optional bare)) (declare-function notmuch-search-find-subject "notmuch" ()) -;; the following variable is defined in notmuch.el -(defvar notmuch-search-query-string) +;; For `notmuch-tree-next-thread-from-search'. +(declare-function notmuch-search-next-thread "notmuch" ()) +(declare-function notmuch-search-previous-thread "notmuch" ()) +(declare-function notmuch-tree-from-search-thread "notmuch" ()) + +;; this variable distinguishes the unthreaded display from the normal tree display +(defvar-local notmuch-tree-unthreaded nil + "A buffer local copy of argument unthreaded to the function notmuch-tree.") + +;;; Options (defgroup notmuch-tree nil "Showing message and thread structure." @@ -51,27 +60,113 @@ :type 'boolean :group 'notmuch-tree) +(defcustom notmuch-unthreaded-show-out t + "View selected messages in new window rather than split-pane." + :type 'boolean + :group 'notmuch-tree) + +(defun notmuch-tree-show-out () + (if notmuch-tree-unthreaded + notmuch-unthreaded-show-out + notmuch-tree-show-out)) + +(defcustom notmuch-tree-thread-symbols + '((prefix . " ") + (top . "─") + (top-tee . "┬") + (vertical . "│") + (vertical-tee . "├") + (bottom . "╰") + (arrow . "►")) + "Strings used to draw trees in notmuch tree results. +Symbol keys denote where the corresponding string value is used: +`prefix' is used at the top of the tree, followed by `top' if it +has no children or `top-tee' if it does; `vertical' is a bar +connecting with a response down the list skipping the current +one, while `vertical-tee' marks the current message as a reply to +the previous one; `bottom' is used at the bottom of threads. +Finally, the `arrrow' string in the list is used as a pointer to +every message. + +Common customizations include setting `prefix' to \"-\", to see +equal-length prefixes, and `arrow' to an empty string or to a +different kind of arrow point." + :type '(alist :key-type symbol :value-type string) + :group 'notmuch-tree) + +(defconst notmuch-tree--field-names + '(choice :tag "Field" + (const :tag "Date" "date") + (const :tag "Authors" "authors") + (const :tag "Subject" "subject") + (const :tag "Tree" "tree") + (const :tag "Tags" "tags") + (function))) + (defcustom notmuch-tree-result-format `(("date" . "%12s ") ("authors" . "%-20s") - ((("tree" . "%s")("subject" . "%s")) ." %-54s ") + ((("tree" . "%s") + ("subject" . "%s")) + . " %-54s ") + ("tags" . "(%s)")) + "Result formatting for tree view. + +List of pairs of (field . format-string). Supported field +strings are: \"date\", \"authors\", \"subject\", \"tree\", +\"tags\". It is also supported to pass a function in place of a +field-name. In this case the function is passed the thread +object (plist) and format string. + +Tree means the thread tree box graphics. The field may +also be a list in which case the formatting rules are +applied recursively and then the output of all the fields +in the list is inserted according to format-string. + +Note that the author string should not contain whitespace +\(put it in the neighbouring fields instead)." + + :type `(alist :key-type (choice ,notmuch-tree--field-names + (alist :key-type ,notmuch-tree--field-names + :value-type (string :tag "Format"))) + :value-type (string :tag "Format")) + :group 'notmuch-tree) + +(defcustom notmuch-unthreaded-result-format + `(("date" . "%12s ") + ("authors" . "%-20s") + ((("subject" . "%s")) ." %-54s ") ("tags" . "(%s)")) - "Result formatting for Tree view. Supported fields are: date, - authors, subject, tree, tags. Tree means the thread tree - box graphics. The field may also be a list in which case - the formatting rules are applied recursively and then the - output of all the fields in the list is inserted - according to format-string. - -Note the author string should not contain - whitespace (put it in the neighbouring fields instead). - For example: - (setq notmuch-tree-result-format \(\(\"authors\" . \"%-40s\"\) - \(\"subject\" . \"%s\"\)\)\)" - :type '(alist :key-type (string) :value-type (string)) + "Result formatting for unthreaded tree view. + +List of pairs of (field . format-string). Supported field +strings are: \"date\", \"authors\", \"subject\", \"tree\", +\"tags\". It is also supported to pass a function in place of a +field-name. In this case the function is passed the thread +object (plist) and format string. + +Tree means the thread tree box graphics. The field may +also be a list in which case the formatting rules are +applied recursively and then the output of all the fields +in the list is inserted according to format-string. + +Note that the author string should not contain whitespace +\(put it in the neighbouring fields instead)." + + :type `(alist :key-type (choice ,notmuch-tree--field-names + (alist :key-type ,notmuch-tree--field-names + :value-type (string :tag "Format"))) + :value-type (string :tag "Format")) :group 'notmuch-tree) -;; Faces for messages that match the query. +(defun notmuch-tree-result-format () + (if notmuch-tree-unthreaded + notmuch-unthreaded-result-format + notmuch-tree-result-format)) + +;;; Faces +;;;; Faces for messages that match the query + (defface notmuch-tree-match-face '((t :inherit default)) "Default face used in tree mode face for matching messages" @@ -93,7 +188,7 @@ Note the author string should not contain (:foreground "dark blue")) (t (:bold t))) - "Face used in tree mode for the date in messages matching the query." + "Face used in tree mode for the author in messages matching the query." :group 'notmuch-tree :group 'notmuch-faces) @@ -105,7 +200,8 @@ Note the author string should not contain (defface notmuch-tree-match-tree-face nil - "Face used in tree mode for the thread tree block graphics in messages matching the query." + "Face used in tree mode for the thread tree block graphics in +messages matching the query." :group 'notmuch-tree :group 'notmuch-faces) @@ -122,10 +218,11 @@ Note the author string should not contain :group 'notmuch-tree :group 'notmuch-faces) -;; Faces for messages that do not match the query. +;;;; Faces for messages that do not match the query + (defface notmuch-tree-no-match-face '((t (:foreground "gray"))) - "Default face used in tree mode face for non-matching messages" + "Default face used in tree mode face for non-matching messages." :group 'notmuch-tree :group 'notmuch-faces) @@ -143,13 +240,14 @@ Note the author string should not contain (defface notmuch-tree-no-match-tree-face nil - "Face used in tree mode for the thread tree block graphics in messages matching the query." + "Face used in tree mode for the thread tree block graphics in +messages matching the query." :group 'notmuch-tree :group 'notmuch-faces) (defface notmuch-tree-no-match-author-face nil - "Face used in tree mode for the date in messages matching the query." + "Face used in tree mode for non-matching authors." :group 'notmuch-tree :group 'notmuch-faces) @@ -159,101 +257,127 @@ Note the author string should not contain :group 'notmuch-tree :group 'notmuch-faces) -(defvar notmuch-tree-previous-subject - "The subject of the most recent result shown during the async display") -(make-variable-buffer-local 'notmuch-tree-previous-subject) +;;; Variables + +(defvar-local notmuch-tree-previous-subject + "The subject of the most recent result shown during the async display.") + +(defvar-local notmuch-tree-basic-query nil + "A buffer local copy of argument query to the function notmuch-tree.") -(defvar notmuch-tree-basic-query nil - "A buffer local copy of argument query to the function notmuch-tree") -(make-variable-buffer-local 'notmuch-tree-basic-query) +(defvar-local notmuch-tree-query-context nil + "A buffer local copy of argument query-context to the function notmuch-tree.") -(defvar notmuch-tree-query-context nil - "A buffer local copy of argument query-context to the function notmuch-tree") -(make-variable-buffer-local 'notmuch-tree-query-context) +(defvar-local notmuch-tree-target-msg nil + "A buffer local copy of argument target to the function notmuch-tree.") -(defvar notmuch-tree-target-msg nil - "A buffer local copy of argument target to the function notmuch-tree") -(make-variable-buffer-local 'notmuch-tree-target-msg) +(defvar-local notmuch-tree-open-target nil + "A buffer local copy of argument open-target to the function notmuch-tree.") -(defvar notmuch-tree-open-target nil - "A buffer local copy of argument open-target to the function notmuch-tree") -(make-variable-buffer-local 'notmuch-tree-open-target) +(defvar-local notmuch-tree-parent-buffer nil) -(defvar notmuch-tree-message-window nil +(defvar-local notmuch-tree-message-window nil "The window of the message pane. It is set in both the tree buffer and the child show buffer. It is used to try and close the message pane when quitting tree view or the child show buffer.") -(make-variable-buffer-local 'notmuch-tree-message-window) (put 'notmuch-tree-message-window 'permanent-local t) -(defvar notmuch-tree-message-buffer nil +(defvar-local notmuch-tree-message-buffer nil "The buffer name of the show buffer in the message pane. This is used to try and make sure we don't close the message pane if the user has loaded a different buffer in that window.") -(make-variable-buffer-local 'notmuch-tree-message-buffer) (put 'notmuch-tree-message-buffer 'permanent-local t) -(defun notmuch-tree-to-message-pane (func) - "Execute FUNC in message pane. +;;; Tree wrapper commands -This function returns a function (so can be used as a keybinding) -which executes function FUNC in the message pane if it is -open (if the message pane is closed it does nothing)." - `(lambda () - ,(concat "(In message pane) " (documentation func t)) +(defmacro notmuch-tree--define-do-in-message-window (name cmd) + "Define NAME as a command that calls CMD interactively in the message window. +If the message pane is closed then this command does nothing. +Avoid using this macro in new code; it will be removed." + `(defun ,name () + ,(concat "(In message window) " (documentation cmd t)) (interactive) (when (window-live-p notmuch-tree-message-window) (with-selected-window notmuch-tree-message-window - (call-interactively #',func))))) - -(defun notmuch-tree-inherit-from-message-pane (sym) - "Return value of SYM in message-pane if open, or tree-pane if not" + (call-interactively #',cmd))))) + +(notmuch-tree--define-do-in-message-window + notmuch-tree-previous-message-button + notmuch-show-previous-button) +(notmuch-tree--define-do-in-message-window + notmuch-tree-next-message-button + notmuch-show-next-button) +(notmuch-tree--define-do-in-message-window + notmuch-tree-toggle-message-process-crypto + notmuch-show-toggle-process-crypto) + +(defun notmuch-tree--message-process-crypto () + "Return value of `notmuch-show-process-crypto' in the message window. +If that window isn't alive, then return the current value. +Avoid using this function in new code; it will be removed." (if (window-live-p notmuch-tree-message-window) (with-selected-window notmuch-tree-message-window - (symbol-value sym)) - (symbol-value sym))) - -(defun notmuch-tree-button-activate (&optional button) - "Activate BUTTON or button at point - -This function does not give an error if there is no button." - (interactive) - (let ((button (or button (button-at (point))))) - (when button (button-activate button)))) - -(defun notmuch-tree-close-message-pane-and (func) - "Close message pane and execute FUNC. - -This function returns a function (so can be used as a keybinding) -which closes the message pane if open and then executes function -FUNC." - `(lambda () - ,(concat "(Close message pane and) " (documentation func t)) + notmuch-show-process-crypto) + notmuch-show-process-crypto)) + +(defmacro notmuch-tree--define-close-message-window-and (name cmd) + "Define NAME as a variant of CMD. + +NAME determines the value of `notmuch-show-process-crypto' in the +message window, closes the window, and then call CMD interactively +with that value let-bound. If the message window does not exist, +then NAME behaves like CMD." + `(defun ,name () + ,(concat "(Close message pane and) " (documentation cmd t)) (interactive) (let ((notmuch-show-process-crypto - (notmuch-tree-inherit-from-message-pane 'notmuch-show-process-crypto))) + (notmuch-tree--message-process-crypto))) (notmuch-tree-close-message-window) - (call-interactively #',func)))) + (call-interactively #',cmd)))) + +(notmuch-tree--define-close-message-window-and + notmuch-tree-help + notmuch-help) +(notmuch-tree--define-close-message-window-and + notmuch-tree-new-mail + notmuch-mua-new-mail) +(notmuch-tree--define-close-message-window-and + notmuch-tree-jump-search + notmuch-jump-search) +(notmuch-tree--define-close-message-window-and + notmuch-tree-forward-message + notmuch-show-forward-message) +(notmuch-tree--define-close-message-window-and + notmuch-tree-reply-sender + notmuch-show-reply-sender) +(notmuch-tree--define-close-message-window-and + notmuch-tree-reply + notmuch-show-reply) +(notmuch-tree--define-close-message-window-and + notmuch-tree-view-raw-message + notmuch-show-view-raw-message) + +;;; Keymap (defvar notmuch-tree-mode-map (let ((map (make-sparse-keymap))) (set-keymap-parent map notmuch-common-keymap) - ;; The following override the global keymap. - ;; Override because we want to close message pane first. - (define-key map [remap notmuch-help] (notmuch-tree-close-message-pane-and #'notmuch-help)) - ;; Override because we first close message pane and then close tree buffer. + ;; These bindings shadow common bindings with variants + ;; that additionally close the message window. (define-key map [remap notmuch-bury-or-kill-this-buffer] 'notmuch-tree-quit) - ;; Override because we close message pane after the search query is entered. - (define-key map [remap notmuch-search] 'notmuch-tree-to-search) - ;; Override because we want to close message pane first. - (define-key map [remap notmuch-mua-new-mail] (notmuch-tree-close-message-pane-and #'notmuch-mua-new-mail)) - ;; Override because we want to close message pane first. - (define-key map [remap notmuch-jump-search] (notmuch-tree-close-message-pane-and #'notmuch-jump-search)) + (define-key map [remap notmuch-search] 'notmuch-tree-to-search) + (define-key map [remap notmuch-help] 'notmuch-tree-help) + (define-key map [remap notmuch-mua-new-mail] 'notmuch-tree-new-mail) + (define-key map [remap notmuch-jump-search] 'notmuch-tree-jump-search) + (define-key map "o" 'notmuch-tree-toggle-order) + (define-key map "i" 'notmuch-tree-toggle-hide-excluded) (define-key map "S" 'notmuch-search-from-tree-current-query) + (define-key map "U" 'notmuch-unthreaded-from-tree-current-query) + (define-key map "Z" 'notmuch-tree-from-unthreaded-current-query) ;; these use notmuch-show functions directly (define-key map "|" 'notmuch-show-pipe-message) @@ -263,22 +387,26 @@ FUNC." (define-key map "b" 'notmuch-show-resend-message) ;; these apply to the message pane - (define-key map (kbd "M-TAB") (notmuch-tree-to-message-pane #'notmuch-show-previous-button)) - (define-key map (kbd "") (notmuch-tree-to-message-pane #'notmuch-show-previous-button)) - (define-key map (kbd "TAB") (notmuch-tree-to-message-pane #'notmuch-show-next-button)) - (define-key map "$" (notmuch-tree-to-message-pane #'notmuch-show-toggle-process-crypto)) + (define-key map (kbd "M-TAB") 'notmuch-tree-previous-message-button) + (define-key map (kbd "") 'notmuch-tree-previous-message-button) + (define-key map (kbd "TAB") 'notmuch-tree-next-message-button) + (define-key map "$" 'notmuch-tree-toggle-message-process-crypto) ;; bindings from show (or elsewhere) but we close the message pane first. - (define-key map "f" (notmuch-tree-close-message-pane-and #'notmuch-show-forward-message)) - (define-key map "r" (notmuch-tree-close-message-pane-and #'notmuch-show-reply-sender)) - (define-key map "R" (notmuch-tree-close-message-pane-and #'notmuch-show-reply)) - (define-key map "V" (notmuch-tree-close-message-pane-and #'notmuch-show-view-raw-message)) + (define-key map "f" 'notmuch-tree-forward-message) + (define-key map "r" 'notmuch-tree-reply-sender) + (define-key map "R" 'notmuch-tree-reply) + (define-key map "V" 'notmuch-tree-view-raw-message) + (define-key map "l" 'notmuch-tree-filter) + (define-key map "t" 'notmuch-tree-filter-by-tag) + (define-key map "E" 'notmuch-tree-edit-search) ;; The main tree view bindings (define-key map (kbd "RET") 'notmuch-tree-show-message) (define-key map [mouse-1] 'notmuch-tree-show-message) - (define-key map "x" 'notmuch-tree-quit) - (define-key map "A" 'notmuch-tree-archive-thread) + (define-key map "x" 'notmuch-tree-archive-message-then-next-or-exit) + (define-key map "X" 'notmuch-tree-archive-thread-then-exit) + (define-key map "A" 'notmuch-tree-archive-thread-then-next) (define-key map "a" 'notmuch-tree-archive-message-then-next) (define-key map "z" 'notmuch-tree-to-tree) (define-key map "n" 'notmuch-tree-next-matching-message) @@ -294,15 +422,17 @@ FUNC." (define-key map " " 'notmuch-tree-scroll-or-next) (define-key map (kbd "DEL") 'notmuch-tree-scroll-message-window-back) (define-key map "e" 'notmuch-tree-resume-message) - map)) -(fset 'notmuch-tree-mode-map notmuch-tree-mode-map) + map) + "Keymap for \"notmuch tree\" buffers.") + +;;; Message properties (defun notmuch-tree-get-message-properties () "Return the properties of the current message as a plist. Some useful entries are: :headers - Property list containing the headers :Date, :Subject, :From, etc. -:tags - Tags for this message" +:tags - Tags for this message." (save-excursion (beginning-of-line) (get-text-property (point) :notmuch-message-properties))) @@ -310,7 +440,9 @@ Some useful entries are: (defun notmuch-tree-set-message-properties (props) (save-excursion (beginning-of-line) - (put-text-property (point) (+ (point) 1) :notmuch-message-properties props))) + (put-text-property (point) + (+ (point) 1) + :notmuch-message-properties props))) (defun notmuch-tree-set-prop (prop val &optional props) (let ((inhibit-read-only t) @@ -320,9 +452,8 @@ Some useful entries are: (notmuch-tree-set-message-properties props))) (defun notmuch-tree-get-prop (prop &optional props) - (let ((props (or props - (notmuch-tree-get-message-properties)))) - (plist-get props prop))) + (plist-get (or props (notmuch-tree-get-message-properties)) + prop)) (defun notmuch-tree-set-tags (tags) "Set the tags of the current message." @@ -343,9 +474,10 @@ Some useful entries are: (defun notmuch-tree-get-match () "Return whether the current message is a match." - (interactive) (notmuch-tree-get-prop :match)) +;;; Update display + (defun notmuch-tree-refresh-result () "Redisplay the current message line. @@ -363,7 +495,8 @@ updated." ;; from overwriting the buffer local copy of ;; notmuch-tree-previous-subject if this is called while the ;; buffer is displaying. - (let ((notmuch-tree-previous-subject (notmuch-tree-get-prop :previous-subject))) + (let ((notmuch-tree-previous-subject + (notmuch-tree-get-prop :previous-subject))) (delete-region (point) (1+ (line-end-position))) (notmuch-tree-insert-msg msg)) (let ((new-end (line-end-position))) @@ -387,8 +520,10 @@ NOT change the database." (when (string= tree-msg-id (notmuch-show-get-message-id)) (notmuch-show-update-tags new-tags))))))) +;;; Commands (and some helper functions used by them) + (defun notmuch-tree-tag (tag-changes) - "Change tags for the current message" + "Change tags for the current message." (interactive (list (notmuch-read-tag-changes (notmuch-tree-get-tags) "Tag message"))) (notmuch-tag (notmuch-tree-get-message-id) tag-changes) @@ -428,26 +563,47 @@ NOT change the database." (notmuch-search query))) (defun notmuch-tree-to-tree () - "Run a query and display results in Tree view" + "Run a query and display results in tree view." (interactive) (let ((query (notmuch-read-query "Notmuch tree view search: "))) (notmuch-tree-close-message-window) (notmuch-tree query))) +(defun notmuch-tree-archive-thread-then-next () + "Archive all messages in the current buffer, then show next thread from search." + (interactive) + (notmuch-tree-archive-thread) + (notmuch-tree-next-thread)) + +(defun notmuch-unthreaded-from-tree-current-query () + "Switch from tree view to unthreaded view." + (interactive) + (unless notmuch-tree-unthreaded + (notmuch-tree-refresh-view 'unthreaded))) + +(defun notmuch-tree-from-unthreaded-current-query () + "Switch from unthreaded view to tree view." + (interactive) + (when notmuch-tree-unthreaded + (notmuch-tree-refresh-view 'tree))) + (defun notmuch-search-from-tree-current-query () - "Call notmuch search with the current query" + "Call notmuch search with the current query." (interactive) (notmuch-tree-close-message-window) - (notmuch-search (notmuch-tree-get-query))) + (notmuch-search (notmuch-tree-get-query) + notmuch-search-oldest-first + notmuch-search-hide-excluded)) (defun notmuch-tree-message-window-kill-hook () "Close the message pane when exiting the show buffer." (let ((buffer (current-buffer))) (when (and (window-live-p notmuch-tree-message-window) (eq (window-buffer notmuch-tree-message-window) buffer)) - ;; We do not want an error if this is the sole window in the - ;; frame and I do not know how to test for that in emacs pre - ;; 24. Hence we just ignore-errors. + ;; We could check whether this is the only window in its frame, + ;; but simply ignoring the error that is thrown otherwise is + ;; what we had to do for Emacs 24 and we stick to that because + ;; it is still the simplest approach. (ignore-errors (delete-window notmuch-tree-message-window))))) @@ -471,9 +627,14 @@ NOT change the database." (setq notmuch-tree-message-window (split-window-vertically (/ (window-height) 4))) (with-selected-window notmuch-tree-message-window - ;; Since we are only displaying one message do not indent. - (let ((notmuch-show-indent-messages-width 0) - (notmuch-show-only-matching-messages t)) + (let (;; Since we are only displaying one message do not indent. + (notmuch-show-indent-messages-width 0) + (notmuch-show-single-message t) + ;; Ensure that `pop-to-buffer-same-window' uses the + ;; window we want it to use. + (display-buffer-overriding-action + '((display-buffer-same-window) + (inhibit-same-window . nil)))) (setq buffer (notmuch-show id)))) ;; We need the `let' as notmuch-tree-message-window is buffer local. (let ((window notmuch-tree-message-window)) @@ -488,12 +649,13 @@ NOT change the database." "Show the current message (in whole window)." (interactive) (let ((id (notmuch-tree-get-message-id)) - (inhibit-read-only t) - buffer) + (inhibit-read-only t)) (when id ;; We close the window to kill off un-needed buffers. (notmuch-tree-close-message-window) - (notmuch-show id)))) + ;; n-s-s-m is buffer local, so use inner let. + (let ((notmuch-show-single-message t)) + (notmuch-show id))))) (defun notmuch-tree-show-message (arg) "Show the current message. @@ -501,13 +663,13 @@ NOT change the database." Shows in split pane or whole window according to value of `notmuch-tree-show-out'. A prefix argument reverses the choice." (interactive "P") - (if (or (and notmuch-tree-show-out (not arg)) - (and (not notmuch-tree-show-out) arg)) + (if (or (and (notmuch-tree-show-out) (not arg)) + (and (not (notmuch-tree-show-out)) arg)) (notmuch-tree-show-message-out) (notmuch-tree-show-message-in))) (defun notmuch-tree-scroll-message-window () - "Scroll the message window (if it exists)" + "Scroll the message window (if it exists)." (interactive) (when (window-live-p notmuch-tree-message-window) (with-selected-window notmuch-tree-message-window @@ -516,7 +678,7 @@ Shows in split pane or whole window according to value of (scroll-up))))) (defun notmuch-tree-scroll-message-window-back () - "Scroll the message window back(if it exists)" + "Scroll the message window back (if it exists)." (interactive) (when (window-live-p notmuch-tree-message-window) (with-selected-window notmuch-tree-message-window @@ -525,22 +687,24 @@ Shows in split pane or whole window according to value of (scroll-down))))) (defun notmuch-tree-scroll-or-next () - "Scroll the message window. If it at end go to next message." + "Scroll the message window. +If it at end go to next message." (interactive) (when (notmuch-tree-scroll-message-window) (notmuch-tree-next-matching-message))) -(defun notmuch-tree-quit () +(defun notmuch-tree-quit (&optional kill-both) "Close the split view or exit tree." - (interactive) - (unless (notmuch-tree-close-message-window) + (interactive "P") + (when (or (not (notmuch-tree-close-message-window)) kill-both) (kill-buffer (current-buffer)))) (defun notmuch-tree-close-message-window () "Close the message-window. Return t if close succeeds." (interactive) (when (and (window-live-p notmuch-tree-message-window) - (eq (window-buffer notmuch-tree-message-window) notmuch-tree-message-buffer)) + (eq (window-buffer notmuch-tree-message-window) + notmuch-tree-message-buffer)) (delete-window notmuch-tree-message-window) (unless (get-buffer-window-list notmuch-tree-message-buffer) (kill-buffer notmuch-tree-message-buffer)) @@ -555,7 +719,8 @@ message will be \"unarchived\", i.e. the tag changes in `notmuch-archive-tags' will be reversed." (interactive "P") (when notmuch-archive-tags - (notmuch-tree-tag (notmuch-tag-change-list notmuch-archive-tags unarchive)))) + (notmuch-tree-tag + (notmuch-tag-change-list notmuch-archive-tags unarchive)))) (defun notmuch-tree-archive-message-then-next (&optional unarchive) "Archive the current message and move to next matching message." @@ -563,6 +728,21 @@ message will be \"unarchived\", i.e. the tag changes in (notmuch-tree-archive-message unarchive) (notmuch-tree-next-matching-message)) +(defun notmuch-tree-archive-thread-then-exit () + "Archive all messages in the current buffer, then exit notmuch-tree." + (interactive) + (notmuch-tree-archive-thread) + (notmuch-tree-quit t)) + +(defun notmuch-tree-archive-message-then-next-or-exit () + "Archive current message, then show next open message in current thread. + +If at the last open message in the current thread, then exit back +to search results." + (interactive) + (notmuch-tree-archive-message) + (notmuch-tree-next-matching-message t)) + (defun notmuch-tree-next-message () "Move to next message." (interactive) @@ -577,63 +757,118 @@ message will be \"unarchived\", i.e. the tag changes in (when (window-live-p notmuch-tree-message-window) (notmuch-tree-show-message-in))) -(defun notmuch-tree-prev-matching-message () +(defun notmuch-tree-goto-matching-message (&optional prev) + "Move to the next or previous matching message. + +Returns t if there was a next matching message in the thread to show, +nil otherwise." + (let ((dir (if prev -1 nil)) + (eobfn (if prev #'bobp #'eobp))) + (while (and (not (funcall eobfn)) + (not (notmuch-tree-get-match))) + (forward-line dir)) + (not (funcall eobfn)))) + +(defun notmuch-tree-matching-message (&optional prev pop-at-end) + "Move to the next or previous matching message." + (interactive "P") + (forward-line (if prev -1 nil)) + (if (and (not (notmuch-tree-goto-matching-message prev)) pop-at-end) + (notmuch-tree-quit pop-at-end) + (when (window-live-p notmuch-tree-message-window) + (notmuch-tree-show-message-in)))) + +(defun notmuch-tree-prev-matching-message (&optional pop-at-end) "Move to previous matching message." - (interactive) - (forward-line -1) - (while (and (not (bobp)) (not (notmuch-tree-get-match))) - (forward-line -1)) - (when (window-live-p notmuch-tree-message-window) - (notmuch-tree-show-message-in))) + (interactive "P") + (notmuch-tree-matching-message t pop-at-end)) -(defun notmuch-tree-next-matching-message () +(defun notmuch-tree-next-matching-message (&optional pop-at-end) "Move to next matching message." - (interactive) - (forward-line) - (while (and (not (eobp)) (not (notmuch-tree-get-match))) - (forward-line)) - (when (window-live-p notmuch-tree-message-window) - (notmuch-tree-show-message-in))) + (interactive "P") + (notmuch-tree-matching-message nil pop-at-end)) -(defun notmuch-tree-refresh-view () +(defun notmuch-tree-refresh-view (&optional view) "Refresh view." (interactive) (when (get-buffer-process (current-buffer)) (error "notmuch tree process already running for current buffer")) (let ((inhibit-read-only t) (basic-query notmuch-tree-basic-query) + (unthreaded (cond ((eq view 'unthreaded) t) + ((eq view 'tree) nil) + (t notmuch-tree-unthreaded))) (query-context notmuch-tree-query-context) (target (notmuch-tree-get-message-id))) (erase-buffer) (notmuch-tree-worker basic-query query-context - target))) + target + nil + unthreaded + notmuch-search-oldest-first + notmuch-search-hide-excluded))) (defun notmuch-tree-thread-top () (when (notmuch-tree-get-message-properties) (while (not (or (notmuch-tree-get-prop :first) (eobp))) (forward-line -1)))) -(defun notmuch-tree-prev-thread () +(defun notmuch-tree-prev-thread-in-tree () + "Move to the previous thread in the current tree" (interactive) (forward-line -1) - (notmuch-tree-thread-top)) + (notmuch-tree-thread-top) + (not (bobp))) -(defun notmuch-tree-next-thread () +(defun notmuch-tree-next-thread-in-tree () + "Get the next thread in the current tree. Returns t if a thread was +found or nil if not." (interactive) (forward-line 1) (while (not (or (notmuch-tree-get-prop :first) (eobp))) - (forward-line 1))) + (forward-line 1)) + (not (eobp))) + +(defun notmuch-tree-next-thread-from-search (&optional previous) + "Move to the next thread in the parent search results, if any. + +If PREVIOUS is non-nil, move to the previous item in the +search results instead." + (interactive "P") + (let ((parent-buffer notmuch-tree-parent-buffer)) + (notmuch-tree-quit t) + (when (buffer-live-p parent-buffer) + (switch-to-buffer parent-buffer) + (if previous + (notmuch-search-previous-thread) + (notmuch-search-next-thread)) + (notmuch-tree-from-search-thread)))) + +(defun notmuch-tree-next-thread (&optional previous) + "Move to the next thread in the current tree or parent search results. + +If PREVIOUS is non-nil, move to the previous thread in the tree or +search results instead." + (interactive) + (unless (if previous (notmuch-tree-prev-thread-in-tree) + (notmuch-tree-next-thread-in-tree)) + (notmuch-tree-next-thread-from-search previous))) + +(defun notmuch-tree-prev-thread () + "Move to the previous thread in the current tree or parent search results." + (interactive) + (notmuch-tree-next-thread t)) (defun notmuch-tree-thread-mapcar (function) - "Iterate through all messages in the current thread - and call FUNCTION for side effects." + "Call FUNCTION for each message in the current thread. +FUNCTION is called for side effects only." (save-excursion (notmuch-tree-thread-top) - (loop collect (funcall function) - do (forward-line) - while (and (notmuch-tree-get-message-properties) - (not (notmuch-tree-get-prop :first)))))) + (cl-loop collect (funcall function) + do (forward-line) + while (and (notmuch-tree-get-message-properties) + (not (notmuch-tree-get-prop :first)))))) (defun notmuch-tree-get-messages-ids-thread-search () "Return a search string for all message ids of messages in the current thread." @@ -642,7 +877,7 @@ message will be \"unarchived\", i.e. the tag changes in " or ")) (defun notmuch-tree-tag-thread (tag-changes) - "Tag all messages in the current thread" + "Tag all messages in the current thread." (interactive (let ((tags (apply #'append (notmuch-tree-thread-mapcar (lambda () (notmuch-tree-get-tags)))))) @@ -669,7 +904,7 @@ buffer." (notmuch-tree-tag-thread (notmuch-tag-change-list notmuch-archive-tags unarchive)))) -;; Functions below here display the tree buffer itself. +;;; Functions for displaying the tree buffer itself (defun notmuch-tree-clean-address (address) "Try to clean a single email ADDRESS for display. Return @@ -683,18 +918,22 @@ unchanged ADDRESS if parsing fails." (or p-name p-address))) (defun notmuch-tree-format-field (field format-string msg) - "Format a FIELD of MSG according to FORMAT-STRING and return string" + "Format a FIELD of MSG according to FORMAT-STRING and return string." (let* ((headers (plist-get msg :headers)) (match (plist-get msg :match))) (cond ((listp field) (format format-string (notmuch-tree-format-field-list field msg))) + ((functionp field) + (funcall field format-string msg)) + ((string-equal field "date") (let ((face (if match 'notmuch-tree-match-date-face 'notmuch-tree-no-match-date-face))) - (propertize (format format-string (plist-get msg :date_relative)) 'face face))) + (propertize (format format-string (plist-get msg :date_relative)) + 'face face))) ((string-equal field "tree") (let ((tree-status (plist-get msg :tree-status)) @@ -739,7 +978,7 @@ unchanged ADDRESS if parsing fails." (format format-string (notmuch-tag-format-tags tags orig-tags face))))))) (defun notmuch-tree-format-field-list (field-list msg) - "Format fields of MSG according to FIELD-LIST and return string" + "Format fields of MSG according to FIELD-LIST and return string." (let ((face (if (plist-get msg :match) 'notmuch-tree-match-face 'notmuch-tree-no-match-face)) @@ -750,17 +989,17 @@ unchanged ADDRESS if parsing fails." (notmuch-apply-face result-string face t))) (defun notmuch-tree-insert-msg (msg) - "Insert the message MSG according to notmuch-tree-result-format" + "Insert the message MSG according to notmuch-tree-result-format." ;; We need to save the previous subject as it will get overwritten ;; by the insert-field calls. (let ((previous-subject notmuch-tree-previous-subject)) - (insert (notmuch-tree-format-field-list notmuch-tree-result-format msg)) + (insert (notmuch-tree-format-field-list (notmuch-tree-result-format) msg)) (notmuch-tree-set-message-properties msg) (notmuch-tree-set-prop :previous-subject previous-subject) (insert "\n"))) (defun notmuch-tree-goto-and-insert-msg (msg) - "Insert msg at the end of the buffer. Move point to msg if it is the target" + "Insert msg at the end of the buffer. Move point to msg if it is the target." (save-excursion (goto-char (point-max)) (notmuch-tree-insert-msg msg)) @@ -772,7 +1011,8 @@ unchanged ADDRESS if parsing fails." (goto-char (point-max)) (forward-line -1) (when notmuch-tree-open-target - (notmuch-tree-show-message-in))))) + (notmuch-tree-show-message-in) + (notmuch-tree-command-hook))))) (defun notmuch-tree-insert-tree (tree depth tree-status first last) "Insert the message tree TREE at depth DEPTH in the current thread. @@ -780,51 +1020,53 @@ unchanged ADDRESS if parsing fails." A message tree is another name for a single sub-thread: i.e., a message together with all its descendents." (let ((msg (car tree)) - (replies (cadr tree))) - - (cond - ((and (< 0 depth) (not last)) - (push "├" tree-status)) - ((and (< 0 depth) last) - (push "╰" tree-status)) - ((and (eq 0 depth) first last) -;; (push "─" tree-status)) choice between this and next line is matter of taste. - (push " " tree-status)) - ((and (eq 0 depth) first (not last)) - (push "┬" tree-status)) - ((and (eq 0 depth) (not first) last) - (push "╰" tree-status)) - ((and (eq 0 depth) (not first) (not last)) - (push "├" tree-status))) - - (push (concat (if replies "┬" "─") "►") tree-status) - (setq msg (plist-put msg :first (and first (eq 0 depth)))) - (setq msg (plist-put msg :tree-status tree-status)) - (setq msg (plist-put msg :orig-tags (plist-get msg :tags))) - (notmuch-tree-goto-and-insert-msg msg) - (pop tree-status) - (pop tree-status) - - (if last - (push " " tree-status) - (push "│" tree-status)) - + (replies (cadr tree)) + ;; outline level, computed from the message's depth and + ;; whether or not it's the first message in the tree. + (level (1+ (if (and (eq 0 depth) (not first)) 1 depth)))) + (cond + ((and (< 0 depth) (not last)) + (push (alist-get 'vertical-tee notmuch-tree-thread-symbols) tree-status)) + ((and (< 0 depth) last) + (push (alist-get 'bottom notmuch-tree-thread-symbols) tree-status)) + ((and (eq 0 depth) first last) + (push (alist-get 'prefix notmuch-tree-thread-symbols) tree-status)) + ((and (eq 0 depth) first (not last)) + (push (alist-get 'top-tee notmuch-tree-thread-symbols) tree-status)) + ((and (eq 0 depth) (not first) last) + (push (alist-get 'bottom notmuch-tree-thread-symbols) tree-status)) + ((and (eq 0 depth) (not first) (not last)) + (push (alist-get 'vertical-tee notmuch-tree-thread-symbols) tree-status))) + (push (concat (alist-get (if replies 'top-tee 'top) notmuch-tree-thread-symbols) + (alist-get 'arrow notmuch-tree-thread-symbols)) + tree-status) + (setq msg (plist-put msg :first (and first (eq 0 depth)))) + (setq msg (plist-put msg :tree-status tree-status)) + (setq msg (plist-put msg :orig-tags (plist-get msg :tags))) + (setq msg (plist-put msg :level level)) + (notmuch-tree-goto-and-insert-msg msg) + (pop tree-status) + (pop tree-status) + (if last + (push " " tree-status) + (push (alist-get 'vertical notmuch-tree-thread-symbols) tree-status)) (notmuch-tree-insert-thread replies (1+ depth) tree-status))) (defun notmuch-tree-insert-thread (thread depth tree-status) - "Insert the collection of sibling sub-threads THREAD at depth DEPTH in the current forest." + "Insert the collection of sibling sub-threads THREAD at depth +DEPTH in the current forest." (let ((n (length thread))) - (loop for tree in thread - for count from 1 to n - - do (notmuch-tree-insert-tree tree depth tree-status (eq count 1) (eq count n))))) + (cl-loop for tree in thread + for count from 1 to n + do (notmuch-tree-insert-tree tree depth tree-status + (eq count 1) + (eq count n))))) (defun notmuch-tree-insert-forest-thread (forest-thread) "Insert a single complete thread." - (let (tree-status) - ;; Reset at the start of each main thread. - (setq notmuch-tree-previous-subject nil) - (notmuch-tree-insert-thread forest-thread 0 tree-status))) + ;; Reset at the start of each main thread. + (setq notmuch-tree-previous-subject nil) + (notmuch-tree-insert-thread forest-thread 0 nil)) (defun notmuch-tree-insert-forest (forest) "Insert a forest of threads. @@ -846,51 +1088,57 @@ Pressing \\[notmuch-tree-show-message] on any line displays that message. Complete list of currently available key bindings: \\{notmuch-tree-mode-map}" - (setq notmuch-buffer-refresh-function #'notmuch-tree-refresh-view) (hl-line-mode 1) - (setq buffer-read-only t - truncate-lines t)) + (setq buffer-read-only t) + (setq truncate-lines t) + (when notmuch-tree-outline-enabled (notmuch-tree-outline-mode 1))) + +(defvar notmuch-tree-process-exit-functions nil + "Functions called when the process inserting a tree of results finishes. -(defun notmuch-tree-process-sentinel (proc msg) - "Add a message to let user know when \"notmuch tree\" exits" +Functions in this list are called with one argument, the process +object, and with the tree results buffer as the current buffer.") + +(defun notmuch-tree-process-sentinel (proc _msg) + "Add a message to let user know when \"notmuch tree\" exits." (let ((buffer (process-buffer proc)) (status (process-status proc)) - (exit-status (process-exit-status proc)) - (never-found-target-thread nil)) + (exit-status (process-exit-status proc))) (when (memq status '(exit signal)) - (kill-buffer (process-get proc 'parse-buf)) - (if (buffer-live-p buffer) - (with-current-buffer buffer - (save-excursion - (let ((inhibit-read-only t) - (atbob (bobp))) - (goto-char (point-max)) - (if (eq status 'signal) - (insert "Incomplete search results (tree view process was killed).\n")) - (when (eq status 'exit) - (insert "End of search results.") - (unless (= exit-status 0) - (insert (format " (process returned %d)" exit-status))) - (insert "\n"))))))))) + (kill-buffer (process-get proc 'parse-buf)) + (when (buffer-live-p buffer) + (with-current-buffer buffer + (save-excursion + (let ((inhibit-read-only t)) + (goto-char (point-max)) + (when (eq status 'signal) + (insert "Incomplete search results (tree view process was killed).\n")) + (when (eq status 'exit) + (insert "End of search results.") + (unless (= exit-status 0) + (insert (format " (process returned %d)" exit-status))) + (insert "\n")))) + (run-hook-with-args 'notmuch-tree-process-exit-functions proc)))))) (defun notmuch-tree-process-filter (proc string) - "Process and filter the output of \"notmuch show\" for tree view" + "Process and filter the output of \"notmuch show\" for tree view." (let ((results-buf (process-buffer proc)) - (parse-buf (process-get proc 'parse-buf)) - (inhibit-read-only t) - done) + (parse-buf (process-get proc 'parse-buf)) + (inhibit-read-only t)) (if (not (buffer-live-p results-buf)) - (delete-process proc) + (delete-process proc) (with-current-buffer parse-buf - ;; Insert new data - (save-excursion - (goto-char (point-max)) - (insert string)) + ;; Insert new data + (save-excursion + (goto-char (point-max)) + (insert string)) (notmuch-sexp-parse-partial-list 'notmuch-tree-insert-forest-thread results-buf))))) -(defun notmuch-tree-worker (basic-query &optional query-context target open-target) +(defun notmuch-tree-worker (basic-query &optional query-context target + open-target unthreaded oldest-first + exclude) "Insert the tree view of the search in the current buffer. This is is a helper function for notmuch-tree. The arguments are @@ -898,10 +1146,14 @@ the same as for the function notmuch-tree." (interactive) (notmuch-tree-mode) (add-hook 'post-command-hook #'notmuch-tree-command-hook t t) + (setq notmuch-search-oldest-first oldest-first) + (setq notmuch-search-hide-excluded exclude) + (setq notmuch-tree-unthreaded unthreaded) (setq notmuch-tree-basic-query basic-query) (setq notmuch-tree-query-context (if (or (string= query-context "") (string= query-context "*")) - nil query-context)) + nil + query-context)) (setq notmuch-tree-target-msg target) (setq notmuch-tree-open-target open-target) ;; Set the default value for `notmuch-show-process-crypto' in this @@ -909,20 +1161,21 @@ the same as for the function notmuch-tree." ;; (such as reply) do. It is a buffer local variable so setting it ;; will not affect genuine show buffers. (setq notmuch-show-process-crypto notmuch-crypto-process-mime) - (erase-buffer) (goto-char (point-min)) (let* ((search-args (concat basic-query - (if query-context (concat " and (" query-context ")")) - )) - (message-arg "--entire-thread")) - (if (equal (car (process-lines notmuch-command "count" search-args)) "0") - (setq search-args basic-query)) + (and query-context + (concat " and (" query-context ")")))) + (sort-arg (if oldest-first "--sort=oldest-first" "--sort=newest-first")) + (message-arg (if unthreaded "--unthreaded" "--entire-thread")) + (exclude-arg (if exclude "--exclude=true" "--exclude=false"))) + (when (equal (car (notmuch--process-lines notmuch-command "count" search-args)) "0") + (setq search-args basic-query)) (notmuch-tag-clear-cache) (let ((proc (notmuch-start-notmuch "notmuch-tree" (current-buffer) #'notmuch-tree-process-sentinel - "show" "--body=false" "--format=sexp" "--format-version=4" - message-arg search-args)) + "show" "--body=false" "--format=sexp" "--format-version=5" + sort-arg message-arg exclude-arg search-args)) ;; Use a scratch buffer to accumulate partial output. ;; This buffer will be killed by the sentinel, which ;; should be called no matter how the process dies. @@ -932,7 +1185,7 @@ the same as for the function notmuch-tree." (set-process-query-on-exit-flag proc nil)))) (defun notmuch-tree-get-query () - "Return the current query in this tree buffer" + "Return the current query in this tree buffer." (if notmuch-tree-query-context (concat notmuch-tree-basic-query " and (" @@ -940,8 +1193,29 @@ the same as for the function notmuch-tree." ")") notmuch-tree-basic-query)) -(defun notmuch-tree (&optional query query-context target buffer-name open-target) - "Display threads matching QUERY in Tree View. +(defun notmuch-tree-toggle-order () + "Toggle the current search order. + +This command toggles the sort order for the current search. The +default sort order is defined by `notmuch-search-oldest-first'." + (interactive) + (setq notmuch-search-oldest-first (not notmuch-search-oldest-first)) + (notmuch-tree-refresh-view)) + +(defun notmuch-tree-toggle-hide-excluded () + "Toggle whether to hide excluded messages. + +This command toggles whether to hide excluded messages for the current +search. The default value for this is defined by `notmuch-search-hide-excluded'." + (interactive) + (setq notmuch-search-hide-excluded (not notmuch-search-hide-excluded)) + (notmuch-tree-refresh-view)) + +;;;###autoload +(defun notmuch-tree (&optional query query-context target buffer-name + open-target unthreaded parent-buffer + oldest-first hide-excluded) + "Display threads matching QUERY in tree view. The arguments are: QUERY: the main query. This can be any query but in many cases will be @@ -953,25 +1227,277 @@ The arguments are: current if it appears in the tree view results. BUFFER-NAME: the name of the buffer to display the tree view. If it is nil \"*notmuch-tree\" followed by QUERY is used. - OPEN-TARGET: If TRUE open the target message in the message pane." - (interactive) - (if (null query) - (setq query (notmuch-read-query "Notmuch tree view search: "))) - (let ((buffer (get-buffer-create (generate-new-buffer-name - (or buffer-name - (concat "*notmuch-tree-" query "*"))))) + OPEN-TARGET: If TRUE open the target message in the message pane. + UNTHREADED: If TRUE only show matching messages in an unthreaded view." + (interactive + (list + ;; Prompt for a query + nil + ;; Fill other args with nil. + nil nil nil nil nil nil + ;; Populate these from the default value of these options. + (default-value 'notmuch-search-oldest-first) + (default-value 'notmuch-search-hide-excluded))) + (unless query + (setq query (notmuch-read-query (concat "Notmuch " + (if unthreaded "unthreaded " "tree ") + "view search: ")))) + (let* ((name + (or buffer-name + (notmuch-search-buffer-title query + (if unthreaded "unthreaded" "tree")))) + (buffer (get-buffer-create (generate-new-buffer-name name))) (inhibit-read-only t)) - - (switch-to-buffer buffer)) + (pop-to-buffer-same-window buffer)) ;; Don't track undo information for this buffer - (set 'buffer-undo-list t) + (setq buffer-undo-list t) + (notmuch-tree-worker query query-context target open-target + unthreaded oldest-first hide-excluded) + (setq notmuch-tree-parent-buffer parent-buffer) + (setq truncate-lines t)) - (notmuch-tree-worker query query-context target open-target) +(defun notmuch-unthreaded (&optional query query-context target buffer-name + open-target oldest-first hide-excluded) + "Display threads matching QUERY in unthreaded view. - (setq truncate-lines t)) +See function NOTMUCH-TREE for documentation of the arguments" + (interactive + (list + ;; Prompt for a query + nil + ;; Fill other args with nil. + nil nil nil nil + ;; Populate these from the default value of these options. + (default-value 'notmuch-search-oldest-first) + (default-value 'notmuch-search-hide-excluded))) + (notmuch-tree query query-context target buffer-name open-target + t nil oldest-first hide-excluded)) + +(defun notmuch-tree-filter (query) + "Filter or LIMIT the current search results based on an additional query string. + +Runs a new tree search matching only messages that match both the +current search results AND the additional query string provided." + (interactive (list (notmuch-read-query "Filter search: "))) + (let ((notmuch-show-process-crypto (notmuch-tree--message-process-crypto)) + (grouped-query (notmuch-group-disjunctive-query-string query)) + (grouped-original-query (notmuch-group-disjunctive-query-string + (notmuch-tree-get-query)))) + (notmuch-tree-close-message-window) + (notmuch-tree (if (string= grouped-original-query "*") + grouped-query + (concat grouped-original-query " and " grouped-query))))) +(defun notmuch-tree-filter-by-tag (tag) + "Filter the current search results based on a single TAG. -;; +Run a new search matching only messages that match the current +search results and that are also tagged with the given TAG." + (interactive + (list (notmuch-select-tag-with-completion "Filter by tag: " + notmuch-tree-basic-query))) + (let ((notmuch-show-process-crypto (notmuch-tree--message-process-crypto))) + (notmuch-tree-close-message-window) + (notmuch-tree (concat notmuch-tree-basic-query " and tag:" tag) + notmuch-tree-query-context + nil + nil + nil + notmuch-tree-unthreaded + nil + notmuch-search-oldest-first + notmuch-search-hide-excluded))) + +(defun notmuch-tree-edit-search (query) + "Edit the current search" + (interactive (list (read-from-minibuffer "Edit search: " + notmuch-tree-basic-query))) + (let ((notmuch-show-process-crypto (notmuch-tree--message-process-crypto))) + (notmuch-tree-close-message-window) + (notmuch-tree query + notmuch-tree-query-context + nil + nil + nil + notmuch-tree-unthreaded + nil + notmuch-search-oldest-first))) + +;;; Tree outline mode +;;;; Custom variables +(defcustom notmuch-tree-outline-enabled nil + "Whether to automatically activate `notmuch-tree-outline-mode' in tree views." + :type 'boolean) + +(defcustom notmuch-tree-outline-visibility 'hide-others + "Default state of the forest outline for `notmuch-tree-outline-mode'. + +This variable controls the state of a forest initially and after +a movement command. If set to nil, all trees are displayed while +the symbol hide-all indicates that all trees in the forest should +be folded and hide-other that only the first one should be +unfolded." + :type '(choice (const :tag "Show all" nil) + (const :tag "Hide others" hide-others) + (const :tag "Hide all" hide-all))) + +(defcustom notmuch-tree-outline-auto-close nil + "Close message and tree windows when moving past the last message." + :type 'boolean) + +(defcustom notmuch-tree-outline-open-on-next nil + "Open new messages under point if they are closed when moving to next one. + +When this flag is set, using the command +`notmuch-tree-outline-next' with point on a header for a new +message that is not shown will open its `notmuch-show' buffer +instead of moving point to next matching message." + :type 'boolean) + +;;;; Helper functions +(defsubst notmuch-tree-outline--pop-at-end (pop-at-end) + (if notmuch-tree-outline-auto-close (not pop-at-end) pop-at-end)) + +(defun notmuch-tree-outline--set-visibility () + (when (and notmuch-tree-outline-mode (> (point-max) (point-min))) + (cl-case notmuch-tree-outline-visibility + (hide-others (notmuch-tree-outline-hide-others)) + (hide-all (outline-hide-body))))) + +(defun notmuch-tree-outline--on-exit (proc) + (when (eq (process-status proc) 'exit) + (notmuch-tree-outline--set-visibility))) + +(add-hook 'notmuch-tree-process-exit-functions #'notmuch-tree-outline--on-exit) + +(defsubst notmuch-tree-outline--level (&optional props) + (or (plist-get (or props (notmuch-tree-get-message-properties)) :level) 0)) + +(defsubst notmuch-tree-outline--message-open-p () + (and (buffer-live-p notmuch-tree-message-buffer) + (get-buffer-window notmuch-tree-message-buffer) + (let ((id (notmuch-tree-get-message-id))) + (and id + (with-current-buffer notmuch-tree-message-buffer + (string= (notmuch-show-get-message-id) id)))))) + +(defsubst notmuch-tree-outline--at-original-match-p () + (and (notmuch-tree-get-prop :match) + (equal (notmuch-tree-get-prop :orig-tags) + (notmuch-tree-get-prop :tags)))) + +(defun notmuch-tree-outline--next (prev thread pop-at-end &optional open-new) + (cond (thread + (notmuch-tree-thread-top) + (if prev + (outline-backward-same-level 1) + (outline-forward-same-level 1)) + (when (> (notmuch-tree-outline--level) 0) (outline-show-branches)) + (notmuch-tree-outline--next nil nil pop-at-end t)) + ((and (or open-new notmuch-tree-outline-open-on-next) + (notmuch-tree-outline--at-original-match-p) + (not (notmuch-tree-outline--message-open-p))) + (notmuch-tree-outline-hide-others t)) + (t (outline-next-visible-heading (if prev -1 1)) + (unless (notmuch-tree-get-prop :match) + (notmuch-tree-matching-message prev pop-at-end)) + (notmuch-tree-outline-hide-others t)))) + +;;;; User commands +(defun notmuch-tree-outline-hide-others (&optional and-show) + "Fold all threads except the one around point. +If AND-SHOW is t, make the current message visible if it's not." + (interactive) + (save-excursion + (while (and (not (bobp)) (> (notmuch-tree-outline--level) 1)) + (outline-previous-heading)) + (outline-hide-sublevels 1)) + (when (> (notmuch-tree-outline--level) 0) + (outline-show-subtree) + (when and-show (notmuch-tree-show-message nil)))) + +(defun notmuch-tree-outline-next (&optional pop-at-end) + "Next matching message in a forest, taking care of thread visibility. +A prefix argument reverses the meaning of `notmuch-tree-outline-auto-close'." + (interactive "P") + (let ((pop (notmuch-tree-outline--pop-at-end pop-at-end))) + (if (null notmuch-tree-outline-visibility) + (notmuch-tree-matching-message nil pop) + (notmuch-tree-outline--next nil nil pop)))) + +(defun notmuch-tree-outline-previous (&optional pop-at-end) + "Previous matching message in forest, taking care of thread visibility. +With prefix, quit the tree view if there is no previous message." + (interactive "P") + (if (null notmuch-tree-outline-visibility) + (notmuch-tree-prev-matching-message pop-at-end) + (notmuch-tree-outline--next t nil pop-at-end))) + +(defun notmuch-tree-outline-next-thread () + "Next matching thread in forest, taking care of thread visibility." + (interactive) + (if (null notmuch-tree-outline-visibility) + (notmuch-tree-next-thread) + (notmuch-tree-outline--next nil t nil))) + +(defun notmuch-tree-outline-previous-thread () + "Previous matching thread in forest, taking care of thread visibility." + (interactive) + (if (null notmuch-tree-outline-visibility) + (notmuch-tree-prev-thread) + (notmuch-tree-outline--next t t nil))) + +;;;; Mode definition +(defvar notmuch-tree-outline-mode-lighter nil + "The lighter mark for notmuch-tree-outline mode. +Usually empty since outline-minor-mode's lighter will be active.") + +(define-minor-mode notmuch-tree-outline-mode + "Minor mode allowing message trees to be folded as outlines. + +When this mode is set, each thread and subthread in the results +list is treated as a foldable section, with its first message as +its header. + +The mode just makes available in the tree buffer all the +keybindings in `outline-minor-mode', and binds the following +additional keys: + +\\{notmuch-tree-outline-mode-map} + +The customizable variable `notmuch-tree-outline-visibility' +controls how navigation in the buffer is affected by this mode: + + - If it is set to nil, `notmuch-tree-outline-previous', + `notmuch-tree-outline-next', and their thread counterparts + behave just as the corresponding notmuch-tree navigation keys + when this mode is not enabled. + + - If, on the other hand, `notmuch-tree-outline-visibility' is + set to a non-nil value, these commands hiding the outlines of + the trees you are not reading as you move to new messages. + +To enable notmuch-tree-outline-mode by default in all +notmuch-tree buffers, just set +`notmuch-tree-outline-mode-enabled' to t." + :lighter notmuch-tree-outline-mode-lighter + :keymap `((,(kbd "TAB") . outline-cycle) + (,(kbd "M-TAB") . outline-cycle-buffer) + ("n" . notmuch-tree-outline-next) + ("p" . notmuch-tree-outline-previous) + (,(kbd "M-n") . notmuch-tree-outline-next-thread) + (,(kbd "M-p") . notmuch-tree-outline-previous-thread)) + (outline-minor-mode notmuch-tree-outline-mode) + (unless (derived-mode-p 'notmuch-tree-mode) + (user-error "notmuch-tree-outline-mode is only meaningful for notmuch trees!")) + (if notmuch-tree-outline-mode + (progn (setq-local outline-regexp "^[^\n]+") + (setq-local outline-level #'notmuch-tree-outline--level) + (notmuch-tree-outline--set-visibility)) + (setq-local outline-regexp (default-value 'outline-regexp)) + (setq-local outline-level (default-value 'outline-level)))) + +;;; _ (provide 'notmuch-tree) diff --git a/emacs/notmuch-version.el.tmpl b/emacs/notmuch-version.el.tmpl index abf52f17..97308295 100644 --- a/emacs/notmuch-version.el.tmpl +++ b/emacs/notmuch-version.el.tmpl @@ -1,5 +1,4 @@ -;;; notmuch-version.el --- Version of notmuch -;; -*- emacs-lisp -*- +;;; notmuch-version.el --- version of notmuch -*- emacs-lisp -*- ;; ;; %AG% ;; diff --git a/emacs/notmuch-wash.el b/emacs/notmuch-wash.el index 54108d93..fd8a9d1e 100644 --- a/emacs/notmuch-wash.el +++ b/emacs/notmuch-wash.el @@ -1,4 +1,4 @@ -;;; notmuch-wash.el --- cleaning up message bodies +;;; notmuch-wash.el --- cleaning up message bodies -*- lexical-binding: t -*- ;; ;; Copyright © Carl Worth ;; Copyright © David Edmondson @@ -24,11 +24,14 @@ ;;; Code: (require 'coolj) +(require 'diff-mode) (require 'notmuch-lib) -(declare-function notmuch-show-insert-bodypart "notmuch-show" (msg part depth &optional hide)) + +(declare-function notmuch-show-insert-bodypart "notmuch-show" + (msg part depth &optional hide)) (defvar notmuch-show-indent-messages-width) -;; +;;; Options (defgroup notmuch-wash nil "Cleaning up messages for display." @@ -128,6 +131,8 @@ or at the window width (whichever one is lower)." (integer :tag "number of characters")) :group 'notmuch-wash) +;;; Faces + (defface notmuch-wash-toggle-button '((t (:inherit font-lock-comment-face))) "Face used for buttons toggling the visibility of washed away @@ -141,6 +146,8 @@ message parts." :group 'notmuch-wash :group 'notmuch-faces) +;;; Buttons + (defun notmuch-wash-toggle-invisible-action (cite-button) ;; Toggle overlay visibility (let ((overlay (button-get cite-button 'overlay))) @@ -186,24 +193,25 @@ message parts." (let* ((type (overlay-get overlay 'type)) (invis-spec (overlay-get overlay 'invisible)) (state (if (invisible-p invis-spec) "hidden" "visible")) - (label-format (symbol-value (intern-soft (concat "notmuch-wash-button-" - type "-" state "-format")))) - (lines-count (count-lines (overlay-start overlay) (overlay-end overlay)))) + (label-format (symbol-value + (intern-soft + (format "notmuch-wash-button-%s-%s-format" + type state)))) + (lines-count (count-lines (overlay-start overlay) + (overlay-end overlay)))) (format label-format lines-count))) -(defun notmuch-wash-region-to-button (msg beg end type &optional prefix) - "Auxiliary function to do the actual making of overlays and buttons +(defun notmuch-wash-region-to-button (beg end type &optional prefix) + "Auxiliary function to do the actual making of overlays and buttons. BEG and END are buffer locations. TYPE should a string, either \"citation\" or \"signature\". Optional PREFIX is some arbitrary text to insert before the button, probably for indentation. Note that PREFIX should not include a newline." - ;; This uses some slightly tricky conversions between strings and ;; symbols because of the way the button code works. Note that ;; replacing intern-soft with make-symbol will cause this to fail, ;; since the newly created symbol has no plist. - (let ((overlay (make-overlay beg end)) (button-type (intern-soft (concat "notmuch-wash-button-" type "-toggle-type")))) @@ -213,8 +221,8 @@ that PREFIX should not include a newline." (goto-char (1+ end)) (save-excursion (goto-char beg) - (if prefix - (insert-before-markers prefix)) + (when prefix + (insert-before-markers prefix)) (let ((button-beg (point))) (insert-before-markers (notmuch-wash-button-label overlay) "\n") (let ((button (make-button button-beg (1- (point)) @@ -222,23 +230,24 @@ that PREFIX should not include a newline." :type button-type))) (overlay-put overlay 'notmuch-wash-button button)))))) -(defun notmuch-wash-excerpt-citations (msg depth) +;;; Hook functions + +(defun notmuch-wash-excerpt-citations (_msg _depth) "Excerpt citations and up to one signature." (goto-char (point-min)) (beginning-of-line) - (if (and (< (point) (point-max)) - (re-search-forward notmuch-wash-original-regexp nil t)) - (let* ((msg-start (match-beginning 0)) - (msg-end (point-max)) - (msg-lines (count-lines msg-start msg-end))) - (notmuch-wash-region-to-button - msg msg-start msg-end "original"))) + (when (and (< (point) (point-max)) + (re-search-forward notmuch-wash-original-regexp nil t)) + (notmuch-wash-region-to-button (match-beginning 0) + (point-max) + "original")) (while (and (< (point) (point-max)) (re-search-forward notmuch-wash-citation-regexp nil t)) (let* ((cite-start (match-beginning 0)) (cite-end (match-end 0)) (cite-lines (count-lines cite-start cite-end))) - (overlay-put (make-overlay cite-start cite-end) 'face 'notmuch-wash-cited-text) + (overlay-put (make-overlay cite-start cite-end) + 'face 'notmuch-wash-cited-text) (when (> cite-lines (+ notmuch-wash-citation-lines-prefix notmuch-wash-citation-lines-suffix 1)) @@ -248,54 +257,45 @@ that PREFIX should not include a newline." (goto-char cite-end) (forward-line (- notmuch-wash-citation-lines-suffix)) (notmuch-wash-region-to-button - msg hidden-start (point-marker) + hidden-start (point-marker) "citation"))))) - (if (and (not (eobp)) - (re-search-forward notmuch-wash-signature-regexp nil t)) - (let* ((sig-start (match-beginning 0)) - (sig-end (match-end 0)) - (sig-lines (count-lines sig-start (point-max)))) - (if (<= sig-lines notmuch-wash-signature-lines-max) - (let ((sig-start-marker (make-marker)) - (sig-end-marker (make-marker))) - (set-marker sig-start-marker sig-start) - (set-marker sig-end-marker (point-max)) - (overlay-put (make-overlay sig-start-marker sig-end-marker) 'face 'message-cited-text) - (notmuch-wash-region-to-button - msg sig-start-marker sig-end-marker - "signature")))))) - -;; + (when (and (not (eobp)) + (re-search-forward notmuch-wash-signature-regexp nil t)) + (let ((sig-start (match-beginning 0))) + (when (<= (count-lines sig-start (point-max)) + notmuch-wash-signature-lines-max) + (let ((sig-start-marker (make-marker)) + (sig-end-marker (make-marker))) + (set-marker sig-start-marker sig-start) + (set-marker sig-end-marker (point-max)) + (overlay-put (make-overlay sig-start-marker sig-end-marker) + 'face 'message-cited-text) + (notmuch-wash-region-to-button + sig-start-marker sig-end-marker + "signature")))))) -(defun notmuch-wash-elide-blank-lines (msg depth) +(defun notmuch-wash-elide-blank-lines (_msg _depth) "Elide leading, trailing and successive blank lines." - ;; Algorithm derived from `article-strip-multiple-blank-lines' in ;; `gnus-art.el'. - ;; Make all blank lines empty. (goto-char (point-min)) (while (re-search-forward "^[[:space:]\t]+$" nil t) (replace-match "" nil t)) - ;; Replace multiple empty lines with a single empty line. (goto-char (point-min)) (while (re-search-forward "^\n\\(\n+\\)" nil t) (delete-region (match-beginning 1) (match-end 1))) - ;; Remove a leading blank line. (goto-char (point-min)) - (if (looking-at "\n") - (delete-region (match-beginning 0) (match-end 0))) - + (when (looking-at "\n") + (delete-region (match-beginning 0) (match-end 0))) ;; Remove a trailing blank line. (goto-char (point-max)) - (if (looking-at "\n") - (delete-region (match-beginning 0) (match-end 0)))) - -;; + (when (looking-at "\n") + (delete-region (match-beginning 0) (match-end 0)))) -(defun notmuch-wash-tidy-citations (msg depth) +(defun notmuch-wash-tidy-citations (_msg _depth) "Improve the display of cited regions of a message. Perform several transformations on the message body: @@ -306,27 +306,20 @@ Perform several transformations on the message body: text, - Remove citation trailers standing alone after a block of cited text." - ;; Remove lines of repeated citation leaders with no other content. (goto-char (point-min)) (while (re-search-forward "\\(^>[> ]*\n\\)\\{2,\\}" nil t) (replace-match "\\1")) - - ;; Remove citation leaders standing alone before a block of cited - ;; text. + ;; Remove citation leaders standing alone before a block of cited text. (goto-char (point-min)) (while (re-search-forward "\\(\n\\|^[^>].*\\)\n\\(^>[> ]*\n\\)" nil t) (replace-match "\\1\n")) - - ;; Remove citation trailers standing alone after a block of cited - ;; text. + ;; Remove citation trailers standing alone after a block of cited text. (goto-char (point-min)) (while (re-search-forward "\\(^>[> ]*\n\\)\\(^$\\|^[^>].*\\)" nil t) (replace-match "\\2"))) -;; - -(defun notmuch-wash-wrap-long-lines (msg depth) +(defun notmuch-wash-wrap-long-lines (_msg depth) "Wrap long lines in the message. If `notmuch-wash-wrap-lines-length' is a number, this will wrap @@ -334,7 +327,6 @@ the message lines to the minimum of the width of the window or its value. Otherwise, this function will wrap long lines in the message at the window width. When doing so, citation leaders in the wrapped text are maintained." - (let* ((coolj-wrap-follows-window-size nil) (indent (* depth notmuch-show-indent-messages-width)) (limit (if (numberp notmuch-wash-wrap-lines-length) @@ -348,11 +340,7 @@ the wrapped text are maintained." 2))) (coolj-wrap-region (point-min) (point-max)))) -;; - -(require 'diff-mode) - -(defvar diff-file-header-re) ; From `diff-mode.el'. +;;;; Convert Inline Patches (defun notmuch-wash-subject-to-filename (subject &optional maxlen) "Convert a mail SUBJECT into a filename. @@ -376,10 +364,10 @@ filename, before trimming any trailing . and - characters." Return the patch sequence number N from the last \"[PATCH N/M]\" style prefix in SUBJECT, or nil if such a prefix can't be found." - (when (string-match - "^ *\\(\\[[^]]*\\] *\\)*\\[[^]]*?\\([0-9]+\\)/[0-9]+[^]]*\\].*" - subject) - (string-to-number (substring subject (match-beginning 2) (match-end 2))))) + (and (string-match + "^ *\\(\\[[^]]*\\] *\\)*\\[[^]]*?\\([0-9]+\\)/[0-9]+[^]]*\\].*" + subject) + (string-to-number (substring subject (match-beginning 2) (match-end 2))))) (defun notmuch-wash-subject-to-patch-filename (subject) "Convert a patch mail SUBJECT into a filename. @@ -393,12 +381,11 @@ original filename the sender had." (notmuch-wash-subject-to-filename subject 52))) (defun notmuch-wash-convert-inline-patch-to-part (msg depth) - "Convert an inline patch into a fake 'text/x-diff' attachment. + "Convert an inline patch into a fake `text/x-diff' attachment. Given that this function guesses whether a buffer includes a patch and then guesses the extent of the patch, there is scope for error." - (goto-char (point-min)) (when (re-search-forward diff-file-header-re nil t) (beginning-of-line -1) @@ -406,12 +393,12 @@ for error." (patch-end (point-max)) part) (goto-char patch-start) - (if (or - ;; Patch ends with signature. - (re-search-forward notmuch-wash-signature-regexp nil t) - ;; Patch ends with bugtraq comment. - (re-search-forward "^\\*\\*\\* " nil t)) - (setq patch-end (match-beginning 0))) + (when (or + ;; Patch ends with signature. + (re-search-forward notmuch-wash-signature-regexp nil t) + ;; Patch ends with bugtraq comment. + (re-search-forward "^\\*\\*\\* " nil t)) + (setq patch-end (match-beginning 0))) (save-restriction (narrow-to-region patch-start patch-end) (setq part (plist-put part :content-type "inline patch")) @@ -424,7 +411,7 @@ for error." (delete-region (point-min) (point-max)) (notmuch-show-insert-bodypart nil part depth))))) -;; +;;; _ (provide 'notmuch-wash) diff --git a/emacs/notmuch.el b/emacs/notmuch.el index 773d1206..2a73ffa5 100644 --- a/emacs/notmuch.el +++ b/emacs/notmuch.el @@ -1,4 +1,4 @@ -;;; notmuch.el --- run notmuch within emacs +;;; notmuch.el --- run notmuch within emacs -*- lexical-binding: t -*- ;; ;; Copyright © Carl Worth ;; @@ -18,7 +18,7 @@ ;; along with Notmuch. If not, see . ;; ;; Authors: Carl Worth -;; Homepage: https://notmuchmail.org/ +;; Homepage: https://notmuchmail.org ;;; Commentary: @@ -62,13 +62,14 @@ ;; ;; TL;DR: notmuch-emacs from MELPA and notmuch from distro packages is ;; NOT SUPPORTED. -;; + ;;; Code: -(eval-when-compile (require 'cl)) (require 'mm-view) (require 'message) +(require 'hl-line) + (require 'notmuch-lib) (require 'notmuch-tag) (require 'notmuch-show) @@ -79,24 +80,38 @@ (require 'notmuch-message) (require 'notmuch-parser) +;;; Options + (defcustom notmuch-search-result-format `(("date" . "%12s ") ("count" . "%-7s ") ("authors" . "%-20s ") ("subject" . "%s ") ("tags" . "(%s)")) - "Search result formatting. Supported fields are: - date, count, authors, subject, tags -For example: - (setq notmuch-search-result-format \(\(\"authors\" . \"%-40s\"\) - \(\"subject\" . \"%s\"\)\)\) + "Search result formatting. + +List of pairs of (field . format-string). Supported field +strings are: \"date\", \"count\", \"authors\", \"subject\", +\"tags\". It is also supported to pass a function in place of a +field name. In this case the function is passed the thread +object (plist) and format string. + Line breaks are permitted in format strings (though this is currently experimental). Note that a line break at the end of an \"authors\" field will get elided if the authors list is long; place it instead at the beginning of the following field. To enter a line break when setting this variable with setq, use \\n. To enter a line break in customize, press \\[quoted-insert] C-j." - :type '(alist :key-type (string) :value-type (string)) + :type '(alist + :key-type + (choice + (const :tag "Date" "date") + (const :tag "Count" "count") + (const :tag "Authors" "authors") + (const :tag "Subject" "subject") + (const :tag "Tags" "tags") + function) + :value-type (string :tag "Format")) :group 'notmuch-search) ;; The name of this variable `notmuch-init-file' is consistent with the @@ -111,28 +126,34 @@ there will be called at other points of notmuch execution." :type 'file :group 'notmuch) -(defvar notmuch-query-history nil - "Variable to store minibuffer history for notmuch queries") +(defcustom notmuch-search-hook '(notmuch-hl-line-mode) + "List of functions to call when notmuch displays the search results." + :type 'hook + :options '(notmuch-hl-line-mode) + :group 'notmuch-search + :group 'notmuch-hooks) + +;;; Mime Utilities (defun notmuch-foreach-mime-part (function mm-handle) (cond ((stringp (car mm-handle)) - (dolist (part (cdr mm-handle)) - (notmuch-foreach-mime-part function part))) - ((bufferp (car mm-handle)) - (funcall function mm-handle)) - (t (dolist (part mm-handle) - (notmuch-foreach-mime-part function part))))) + (dolist (part (cdr mm-handle)) + (notmuch-foreach-mime-part function part))) + ((bufferp (car mm-handle)) + (funcall function mm-handle)) + (t (dolist (part mm-handle) + (notmuch-foreach-mime-part function part))))) (defun notmuch-count-attachments (mm-handle) (let ((count 0)) (notmuch-foreach-mime-part (lambda (p) (let ((disposition (mm-handle-disposition p))) - (and (listp disposition) - (or (equal (car disposition) "attachment") - (and (equal (car disposition) "inline") - (assq 'filename disposition))) - (incf count)))) + (and (listp disposition) + (or (equal (car disposition) "attachment") + (and (equal (car disposition) "inline") + (assq 'filename disposition))) + (cl-incf count)))) mm-handle) count)) @@ -141,34 +162,22 @@ there will be called at other points of notmuch execution." (lambda (p) (let ((disposition (mm-handle-disposition p))) (and (listp disposition) - (or (equal (car disposition) "attachment") - (and (equal (car disposition) "inline") - (assq 'filename disposition))) - (or (not queryp) - (y-or-n-p - (concat "Save '" (cdr (assq 'filename disposition)) "' "))) - (mm-save-part p)))) + (or (equal (car disposition) "attachment") + (and (equal (car disposition) "inline") + (assq 'filename disposition))) + (or (not queryp) + (y-or-n-p + (concat "Save '" (cdr (assq 'filename disposition)) "' "))) + (mm-save-part p)))) mm-handle)) -(require 'hl-line) - -(defun notmuch-hl-line-mode () - (prog1 (hl-line-mode) - (when hl-line-overlay - (overlay-put hl-line-overlay 'priority 1)))) - -(defcustom notmuch-search-hook '(notmuch-hl-line-mode) - "List of functions to call when notmuch displays the search results." - :type 'hook - :options '(notmuch-hl-line-mode) - :group 'notmuch-search - :group 'notmuch-hooks) +;;; Keymap (defvar notmuch-search-mode-map (let ((map (make-sparse-keymap))) (set-keymap-parent map notmuch-common-keymap) (define-key map "x" 'notmuch-bury-or-kill-this-buffer) - (define-key map (kbd "") 'notmuch-search-scroll-down) + (define-key map (kbd "DEL") 'notmuch-search-scroll-down) (define-key map "b" 'notmuch-search-scroll-down) (define-key map " " 'notmuch-search-scroll-up) (define-key map "<" 'notmuch-search-first-thread) @@ -178,9 +187,11 @@ there will be called at other points of notmuch execution." (define-key map "r" 'notmuch-search-reply-to-thread-sender) (define-key map "R" 'notmuch-search-reply-to-thread) (define-key map "o" 'notmuch-search-toggle-order) + (define-key map "i" 'notmuch-search-toggle-hide-excluded) (define-key map "c" 'notmuch-search-stash-map) (define-key map "t" 'notmuch-search-filter-by-tag) (define-key map "l" 'notmuch-search-filter) + (define-key map "E" 'notmuch-search-edit-search) (define-key map [mouse-1] 'notmuch-search-show-thread) (define-key map "k" 'notmuch-tag-jump) (define-key map "*" 'notmuch-search-tag-all) @@ -188,10 +199,22 @@ there will be called at other points of notmuch execution." (define-key map "-" 'notmuch-search-remove-tag) (define-key map "+" 'notmuch-search-add-tag) (define-key map (kbd "RET") 'notmuch-search-show-thread) + (define-key map (kbd "M-RET") 'notmuch-tree-from-search-thread) (define-key map "Z" 'notmuch-tree-from-search-current-query) + (define-key map "U" 'notmuch-unthreaded-from-search-current-query) map) "Keymap for \"notmuch search\" buffers.") -(fset 'notmuch-search-mode-map notmuch-search-mode-map) + +;;; Internal Variables + +(defvar notmuch-query-history nil + "Variable to store minibuffer history for notmuch queries.") + +(defvar-local notmuch-search-query-string nil) +(defvar-local notmuch-search-target-thread nil) +(defvar-local notmuch-search-target-line nil) + +;;; Stashing (defvar notmuch-search-stash-map (let ((map (make-sparse-keymap))) @@ -199,7 +222,7 @@ there will be called at other points of notmuch execution." (define-key map "q" 'notmuch-stash-query) (define-key map "?" 'notmuch-subkeymap-help) map) - "Submap for stash commands") + "Submap for stash commands.") (fset 'notmuch-search-stash-map notmuch-search-stash-map) (defun notmuch-search-stash-thread-id () @@ -210,13 +233,9 @@ there will be called at other points of notmuch execution." (defun notmuch-stash-query () "Copy current query to kill-ring." (interactive) - (notmuch-common-do-stash (notmuch-search-get-query))) + (notmuch-common-do-stash notmuch-search-query-string)) -(defvar notmuch-search-query-string) -(defvar notmuch-search-target-thread) -(defvar notmuch-search-target-line) - -(defvar notmuch-search-disjunctive-regexp "\\<[oO][rR]\\>") +;;; Movement (defun notmuch-search-scroll-up () "Move forward through search results by one window's worth." @@ -261,19 +280,26 @@ there will be called at other points of notmuch execution." (goto-char (point-max)) (forward-line -2) (let ((beg (notmuch-search-result-beginning))) - (when beg (goto-char beg)))) + (when beg + (goto-char beg)))) (defun notmuch-search-first-thread () "Select the first thread in the search results." (interactive) (goto-char (point-min))) +;;; Faces + (defface notmuch-message-summary-face - '((((class color) (background light)) (:background "#f0f0f0")) - (((class color) (background dark)) (:background "#303030"))) - "Face for the single-line message summary in notmuch-show-mode." - :group 'notmuch-show - :group 'notmuch-faces) + `((((class color) (background light)) + ,@(and (>= emacs-major-version 27) '(:extend t)) + :background "#f0f0f0") + (((class color) (background dark)) + ,@(and (>= emacs-major-version 27) '(:extend t)) + :background "#303030")) + "Face for the single-line message summary in notmuch-show-mode." + :group 'notmuch-show + :group 'notmuch-faces) (defface notmuch-search-date '((t :inherit default)) @@ -335,7 +361,7 @@ there will be called at other points of notmuch execution." "Face used in search mode face for flagged threads. This face is the default value for the \"flagged\" tag in -`notmuch-search-line-faces`." +`notmuch-search-line-faces'." :group 'notmuch-search :group 'notmuch-faces) @@ -345,10 +371,12 @@ This face is the default value for the \"flagged\" tag in "Face used in search mode for unread threads. This face is the default value for the \"unread\" tag in -`notmuch-search-line-faces`." +`notmuch-search-line-faces'." :group 'notmuch-search :group 'notmuch-faces) +;;; Mode + (define-derived-mode notmuch-search-mode fundamental-mode "notmuch-search" "Major mode displaying results of a notmuch search. @@ -379,19 +407,17 @@ new, global search. Complete list of currently available key bindings: \\{notmuch-search-mode-map}" - (make-local-variable 'notmuch-search-query-string) - (make-local-variable 'notmuch-search-oldest-first) - (make-local-variable 'notmuch-search-target-thread) - (make-local-variable 'notmuch-search-target-line) (setq notmuch-buffer-refresh-function #'notmuch-search-refresh-view) - (set (make-local-variable 'scroll-preserve-screen-position) t) + (setq-local scroll-preserve-screen-position t) (add-to-invisibility-spec (cons 'ellipsis t)) (setq truncate-lines t) (setq buffer-read-only t) (setq imenu-prev-index-position-function - #'notmuch-search-imenu-prev-index-position-function) + #'notmuch-search-imenu-prev-index-position-function) (setq imenu-extract-index-name-function - #'notmuch-search-imenu-extract-index-name-function)) + #'notmuch-search-imenu-extract-index-name-function)) + +;;; Search Results (defun notmuch-search-get-result (&optional pos) "Return the result object for the thread at POS (or point). @@ -403,46 +429,46 @@ If there is no thread at POS (or point), returns nil." "Return the point at the beginning of the thread at POS (or point). If there is no thread at POS (or point), returns nil." - (when (notmuch-search-get-result pos) - ;; We pass 1+point because previous-single-property-change starts - ;; searching one before the position we give it. - (previous-single-property-change (1+ (or pos (point))) - 'notmuch-search-result nil (point-min)))) + (and (notmuch-search-get-result pos) + ;; We pass 1+point because previous-single-property-change starts + ;; searching one before the position we give it. + (previous-single-property-change (1+ (or pos (point))) + 'notmuch-search-result nil + (point-min)))) (defun notmuch-search-result-end (&optional pos) "Return the point at the end of the thread at POS (or point). The returned point will be just after the newline character that ends the result line. If there is no thread at POS (or point), -returns nil" - (when (notmuch-search-get-result pos) - (next-single-property-change (or pos (point)) 'notmuch-search-result - nil (point-max)))) +returns nil." + (and (notmuch-search-get-result pos) + (next-single-property-change (or pos (point)) + 'notmuch-search-result nil + (point-max)))) (defun notmuch-search-foreach-result (beg end fn) "Invoke FN for each result between BEG and END. -FN should take one argument. It will be applied to the -character position of the beginning of each result that overlaps -the region between points BEG and END. As a special case, if (= -BEG END), FN will be applied to the result containing point -BEG." - - (lexical-let ((pos (notmuch-search-result-beginning beg)) - ;; End must be a marker in case fn changes the - ;; text. - (end (copy-marker end)) - ;; Make sure we examine at least one result, even if - ;; (= beg end). - (first t)) +FN should take one argument. It will be applied to the character +position of the beginning of each result that overlaps the region +between points BEG and END. As a special case, if (= BEG END), +FN will be applied to the result containing point BEG." + (let ((pos (notmuch-search-result-beginning beg)) + ;; End must be a marker in case fn changes the + ;; text. + (end (copy-marker end)) + ;; Make sure we examine at least one result, even if + ;; (= beg end). + (first t)) ;; We have to be careful if the region extends beyond the results. ;; In this case, pos could be null or there could be no result at ;; pos. (while (and pos (or (< pos end) first)) (when (notmuch-search-get-result pos) (funcall fn pos)) - (setq pos (notmuch-search-result-end pos) - first nil)))) + (setq pos (notmuch-search-result-end pos)) + (setq first nil)))) ;; Unindent the function argument of notmuch-search-foreach-result so ;; the indentation of callers doesn't get out of hand. (put 'notmuch-search-foreach-result 'lisp-indent-function 2) @@ -455,16 +481,17 @@ BEG." output)) (defun notmuch-search-find-thread-id (&optional bare) - "Return the thread for the current thread + "Return the thread for the current thread. -If BARE is set then do not prefix with \"thread:\"" +If BARE is set then do not prefix with \"thread:\"." (let ((thread (plist-get (notmuch-search-get-result) :thread))) - (when thread (concat (unless bare "thread:") thread)))) + (when thread + (concat (and (not bare) "thread:") thread)))) (defun notmuch-search-find-stable-query () "Return the stable queries for the current thread. -This returns a list (MATCHED-QUERY UNMATCHED-QUERY) for the +Return a list (MATCHED-QUERY UNMATCHED-QUERY) for the matched and unmatched messages in the current thread." (plist-get (notmuch-search-get-result) :query)) @@ -476,27 +503,27 @@ is nil, include both matched and unmatched messages. If there are no messages in the region then return nil." (let ((query-list nil) (all (not only-matched))) (dolist (queries (notmuch-search-properties-in-region :query beg end)) - (when (first queries) - (push (first queries) query-list)) - (when (and all (second queries)) - (push (second queries) query-list))) - (when query-list - (concat "(" (mapconcat 'identity query-list ") or (") ")")))) + (when (car queries) + (push (car queries) query-list)) + (when (and all (cadr queries)) + (push (cadr queries) query-list))) + (and query-list + (concat "(" (mapconcat 'identity query-list ") or (") ")")))) (defun notmuch-search-find-authors () - "Return the authors for the current thread" + "Return the authors for the current thread." (plist-get (notmuch-search-get-result) :authors)) (defun notmuch-search-find-authors-region (beg end) - "Return a list of authors for the current region" + "Return a list of authors for the current region." (notmuch-search-properties-in-region :authors beg end)) (defun notmuch-search-find-subject () - "Return the subject for the current thread" + "Return the subject for the current thread." (plist-get (notmuch-search-get-result) :subject)) (defun notmuch-search-find-subject-region (beg end) - "Return a list of authors for the current region" + "Return a list of authors for the current region." (notmuch-search-properties-in-region :subject beg end)) (defun notmuch-search-show-thread (&optional elide-toggle) @@ -504,48 +531,68 @@ no messages in the region then return nil." With a prefix argument, invert the default value of `notmuch-show-only-matching-messages' when displaying the -thread." +thread. + +Return non-nil on success." (interactive "P") - (let ((thread-id (notmuch-search-find-thread-id)) - (subject (notmuch-search-find-subject))) - (if (> (length thread-id) 0) + (let ((thread-id (notmuch-search-find-thread-id))) + (if thread-id (notmuch-show thread-id elide-toggle (current-buffer) notmuch-search-query-string ;; Name the buffer based on the subject. - (concat "*" (truncate-string-to-width subject 30 nil nil t) "*")) - (message "End of search results.")))) + (format "*%s*" (truncate-string-to-width + (notmuch-search-find-subject) + 30 nil nil t))) + (message "End of search results.") + nil))) (defun notmuch-tree-from-search-current-query () - "Call notmuch tree with the current query" + "Tree view of current query." (interactive) - (notmuch-tree notmuch-search-query-string)) + (notmuch-tree notmuch-search-query-string + nil nil nil nil nil nil + notmuch-search-oldest-first + notmuch-search-hide-excluded)) + +(defun notmuch-unthreaded-from-search-current-query () + "Unthreaded view of current query." + (interactive) + (notmuch-unthreaded notmuch-search-query-string + nil nil nil nil + notmuch-search-oldest-first + notmuch-search-hide-excluded)) (defun notmuch-tree-from-search-thread () - "Show the selected thread with notmuch-tree" + "Show the selected thread with notmuch-tree." (interactive) (notmuch-tree (notmuch-search-find-thread-id) - notmuch-search-query-string + notmuch-search-query-string nil - (notmuch-prettify-subject (notmuch-search-find-subject)) - t)) + (notmuch-prettify-subject (notmuch-search-find-subject)) + t nil (current-buffer) + notmuch-search-oldest-first + notmuch-search-hide-excluded)) (defun notmuch-search-reply-to-thread (&optional prompt-for-sender) "Begin composing a reply-all to the entire current thread in a new buffer." (interactive "P") - (let ((message-id (notmuch-search-find-thread-id))) - (notmuch-mua-new-reply message-id prompt-for-sender t))) + (notmuch-mua-new-reply (notmuch-search-find-thread-id) + prompt-for-sender t)) (defun notmuch-search-reply-to-thread-sender (&optional prompt-for-sender) "Begin composing a reply to the entire current thread in a new buffer." (interactive "P") - (let ((message-id (notmuch-search-find-thread-id))) - (notmuch-mua-new-reply message-id prompt-for-sender nil))) + (notmuch-mua-new-reply (notmuch-search-find-thread-id) + prompt-for-sender nil)) + +;;; Tags (defun notmuch-search-set-tags (tags &optional pos) - (let ((new-result (plist-put (notmuch-search-get-result pos) :tags tags))) - (notmuch-search-update-result new-result pos))) + (notmuch-search-update-result + (plist-put (notmuch-search-get-result pos) :tags tags) + pos)) (defun notmuch-search-get-tags (&optional pos) (plist-get (notmuch-search-get-result pos) :tags)) @@ -555,18 +602,17 @@ thread." (notmuch-search-foreach-result beg end (lambda (pos) (setq output (append output (notmuch-search-get-tags pos))))) - output)) + (delete-dups output))) (defun notmuch-search-interactive-tag-changes (&optional initial-input) "Prompt for tag changes for the current thread or region. -Returns (TAG-CHANGES REGION-BEGIN REGION-END)." - (let* ((region (notmuch-interactive-region)) - (beg (first region)) (end (second region)) - (prompt (if (= beg end) "Tag thread" "Tag region"))) - (cons (notmuch-read-tag-changes - (notmuch-search-get-tags-region beg end) prompt initial-input) - region))) +Return (TAG-CHANGES REGION-BEGIN REGION-END)." + (pcase-let ((`(,beg ,end) (notmuch-interactive-region))) + (list (notmuch-read-tag-changes (notmuch-search-get-tags-region beg end) + (if (= beg end) "Tag thread" "Tag region") + initial-input) + beg end))) (defun notmuch-search-tag (tag-changes &optional beg end only-matched) "Change tags for the currently selected thread or region. @@ -581,8 +627,8 @@ is inactive this applies to the thread at point. If ONLY-MATCHED is non-nil, only tag matched messages." (interactive (notmuch-search-interactive-tag-changes)) (unless (and beg end) - (setq beg (car (notmuch-interactive-region)) - end (cadr (notmuch-interactive-region)))) + (setq beg (car (notmuch-interactive-region))) + (setq end (cadr (notmuch-interactive-region)))) (let ((search-string (notmuch-search-find-stable-query-region beg end only-matched))) (notmuch-tag search-string tag-changes) @@ -625,6 +671,8 @@ This function advances the next thread when finished." (when (eq beg end) (notmuch-search-next-thread))) +;;; Search Results + (defun notmuch-search-update-result (result &optional pos) "Replace the result object of the thread at POS (or point) by RESULT and redraw it. @@ -653,8 +701,8 @@ of the result." (min init-point (- new-end 1))))) (goto-char new-point))))) -(defun notmuch-search-process-sentinel (proc msg) - "Add a message to let user know when \"notmuch search\" exits" +(defun notmuch-search-process-sentinel (proc _msg) + "Add a message to let user know when \"notmuch search\" exits." (let ((buffer (process-buffer proc)) (status (process-status proc)) (exit-status (process-exit-status proc)) @@ -662,28 +710,28 @@ of the result." (when (memq status '(exit signal)) (catch 'return (kill-buffer (process-get proc 'parse-buf)) - (if (buffer-live-p buffer) - (with-current-buffer buffer - (save-excursion - (let ((inhibit-read-only t) - (atbob (bobp))) - (goto-char (point-max)) - (if (eq status 'signal) - (insert "Incomplete search results (search process was killed).\n")) - (when (eq status 'exit) - (insert "End of search results.\n") - ;; For version mismatch, there's no point in - ;; showing the search buffer - (when (or (= exit-status 20) (= exit-status 21)) - (kill-buffer) - (throw 'return nil)) - (if (and atbob + (when (buffer-live-p buffer) + (with-current-buffer buffer + (save-excursion + (let ((inhibit-read-only t) + (atbob (bobp))) + (goto-char (point-max)) + (when (eq status 'signal) + (insert "Incomplete search results (search process was killed).\n")) + (when (eq status 'exit) + (insert "End of search results.\n") + ;; For version mismatch, there's no point in + ;; showing the search buffer + (when (or (= exit-status 20) (= exit-status 21)) + (kill-buffer) + (throw 'return nil)) + (when (and atbob (not (string= notmuch-search-target-thread "found"))) - (set 'never-found-target-thread t))))) - (when (and never-found-target-thread + (setq never-found-target-thread t))))) + (when (and never-found-target-thread notmuch-search-target-line) - (goto-char (point-min)) - (forward-line (1- notmuch-search-target-line))))))))) + (goto-char (point-min)) + (forward-line (1- notmuch-search-target-line))))))))) (define-widget 'notmuch--custom-face-edit 'lazy "Custom face edit with a tag Edit Face" @@ -703,7 +751,7 @@ Here is an example of how to color search results based on tags. (the following text would be placed in your ~/.emacs file): (setq notmuch-search-line-faces \\='((\"unread\" . (:foreground \"green\")) - (\"deleted\" . (:foreground \"red\" + (\"deleted\" . (:foreground \"red\" :background \"blue\")))) The FACE must be a face name (a symbol or string), a property @@ -714,7 +762,7 @@ the above settings would have a green foreground and blue background." :type '(alist :key-type (string) :value-type (radio (face :tag "Face name") - (notmuch--custom-face-edit))) + (notmuch--custom-face-edit))) :group 'notmuch-search :group 'notmuch-faces) @@ -747,89 +795,89 @@ non-authors is found, assume that all of the authors match." (visible-string formatted-authors) (invisible-string "") (padding "")) - ;; Truncate the author string to fit the specification. - (if (> (length formatted-authors) - (length formatted-sample)) - (let ((visible-length (- (length formatted-sample) - (length "... ")))) - ;; Truncate the visible string according to the width of - ;; the display string. - (setq visible-string (substring formatted-authors 0 visible-length) - invisible-string (substring formatted-authors visible-length)) - ;; If possible, truncate the visible string at a natural - ;; break (comma or pipe), as incremental search doesn't - ;; match across the visible/invisible border. - (when (string-match "\\(.*\\)\\([,|] \\)\\([^,|]*\\)" visible-string) - ;; Second clause is destructive on `visible-string', so - ;; order is important. - (setq invisible-string (concat (match-string 3 visible-string) - invisible-string) - visible-string (concat (match-string 1 visible-string) - (match-string 2 visible-string)))) - ;; `visible-string' may be shorter than the space allowed - ;; by `format-string'. If so we must insert some padding - ;; after `invisible-string'. - (setq padding (make-string (- (length formatted-sample) - (length visible-string) - (length "...")) - ? )))) - + (when (> (length formatted-authors) + (length formatted-sample)) + (let ((visible-length (- (length formatted-sample) + (length "... ")))) + ;; Truncate the visible string according to the width of + ;; the display string. + (setq visible-string (substring formatted-authors 0 visible-length)) + (setq invisible-string (substring formatted-authors visible-length)) + ;; If possible, truncate the visible string at a natural + ;; break (comma or pipe), as incremental search doesn't + ;; match across the visible/invisible border. + (when (string-match "\\(.*\\)\\([,|] \\)\\([^,|]*\\)" visible-string) + ;; Second clause is destructive on `visible-string', so + ;; order is important. + (setq invisible-string (concat (match-string 3 visible-string) + invisible-string)) + (setq visible-string (concat (match-string 1 visible-string) + (match-string 2 visible-string)))) + ;; `visible-string' may be shorter than the space allowed + ;; by `format-string'. If so we must insert some padding + ;; after `invisible-string'. + (setq padding (make-string (- (length formatted-sample) + (length visible-string) + (length "...")) + ? )))) ;; Use different faces to show matching and non-matching authors. (if (string-match "\\(.*\\)|\\(.*\\)" visible-string) ;; The visible string contains both matching and ;; non-matching authors. - (setq visible-string (notmuch-search-author-propertize visible-string) - ;; The invisible string must contain only non-matching - ;; authors, as the visible-string contains both. - invisible-string (propertize invisible-string - 'face 'notmuch-search-non-matching-authors)) + (progn + (setq visible-string (notmuch-search-author-propertize visible-string)) + ;; The invisible string must contain only non-matching + ;; authors, as the visible-string contains both. + (setq invisible-string (propertize invisible-string + 'face 'notmuch-search-non-matching-authors))) ;; The visible string contains only matching authors. (setq visible-string (propertize visible-string - 'face 'notmuch-search-matching-authors) - ;; The invisible string may contain both matching and - ;; non-matching authors. - invisible-string (notmuch-search-author-propertize invisible-string))) - + 'face 'notmuch-search-matching-authors)) + ;; The invisible string may contain both matching and + ;; non-matching authors. + (setq invisible-string (notmuch-search-author-propertize invisible-string))) ;; If there is any invisible text, add it as a tooltip to the ;; visible text. - (when (not (string= invisible-string "")) - (setq visible-string (propertize visible-string 'help-echo (concat "..." invisible-string)))) - + (unless (string-empty-p invisible-string) + (setq visible-string + (propertize visible-string + 'help-echo (concat "..." invisible-string)))) ;; Insert the visible and, if present, invisible author strings. (insert visible-string) - (when (not (string= invisible-string "")) + (unless (string-empty-p invisible-string) (let ((start (point)) overlay) (insert invisible-string) (setq overlay (make-overlay start (point))) + (overlay-put overlay 'evaporate t) (overlay-put overlay 'invisible 'ellipsis) (overlay-put overlay 'isearch-open-invisible #'delete-overlay))) (insert padding)))) (defun notmuch-search-insert-field (field format-string result) - (cond - ((string-equal field "date") - (insert (propertize (format format-string (plist-get result :date_relative)) - 'face 'notmuch-search-date))) - ((string-equal field "count") - (insert (propertize (format format-string - (format "[%s/%s]" (plist-get result :matched) - (plist-get result :total))) - 'face 'notmuch-search-count))) - ((string-equal field "subject") - (insert (propertize (format format-string - (notmuch-sanitize (plist-get result :subject))) - 'face 'notmuch-search-subject))) - - ((string-equal field "authors") - (notmuch-search-insert-authors - format-string (notmuch-sanitize (plist-get result :authors)))) - - ((string-equal field "tags") - (let ((tags (plist-get result :tags)) - (orig-tags (plist-get result :orig-tags))) - (insert (format format-string (notmuch-tag-format-tags tags orig-tags))))))) + (pcase field + ((pred functionp) + (insert (funcall field format-string result))) + ("date" + (insert (propertize (format format-string (plist-get result :date_relative)) + 'face 'notmuch-search-date))) + ("count" + (insert (propertize (format format-string + (format "[%s/%s]" (plist-get result :matched) + (plist-get result :total))) + 'face 'notmuch-search-count))) + ("subject" + (insert (propertize (format format-string + (notmuch-sanitize (plist-get result :subject))) + 'face 'notmuch-search-subject))) + ("authors" + (notmuch-search-insert-authors format-string + (notmuch-sanitize (plist-get result :authors)))) + ("tags" + (let ((tags (plist-get result :tags)) + (orig-tags (plist-get result :orig-tags))) + (insert (format format-string (notmuch-tag-format-tags tags orig-tags))))))) (defun notmuch-search-show-result (result pos) "Insert RESULT at POS." @@ -855,12 +903,19 @@ sets the :orig-tag property." (setq notmuch-search-target-thread "found") (goto-char pos)))) +(defvar-local notmuch--search-hook-run nil + "Flag used to ensure the notmuch-search-hook is only run once per buffer") + +(defun notmuch--search-hook-wrapper () + (unless notmuch--search-hook-run + (setq notmuch--search-hook-run t) + (run-hooks 'notmuch-search-hook))) + (defun notmuch-search-process-filter (proc string) - "Process and filter the output of \"notmuch search\"" + "Process and filter the output of \"notmuch search\"." (let ((results-buf (process-buffer proc)) (parse-buf (process-get proc 'parse-buf)) - (inhibit-read-only t) - done) + (inhibit-read-only t)) (when (buffer-live-p results-buf) (with-current-buffer parse-buf ;; Insert new data @@ -868,7 +923,11 @@ sets the :orig-tag property." (goto-char (point-max)) (insert string)) (notmuch-sexp-parse-partial-list 'notmuch-search-append-result - results-buf))))) + results-buf)) + (with-current-buffer results-buf + (notmuch--search-hook-wrapper))))) + +;;; Commands (and some helper functions used by them) (defun notmuch-search-tag-all (tag-changes) "Add/remove tags from all messages in current search buffer. @@ -879,84 +938,122 @@ See `notmuch-tag' for information on the format of TAG-CHANGES." (notmuch-search-get-tags-region (point-min) (point-max)) "Tag all"))) (notmuch-search-tag tag-changes (point-min) (point-max) t)) -(defun notmuch-search-buffer-title (query) +(defcustom notmuch-search-buffer-name-format "*notmuch-%t-%s*" + "Format for the name of search results buffers. + +In this spec, %s will be replaced by a description of the search +query and %t by its type (search, tree or unthreaded). The +buffer name is formatted using `format-spec': see its docstring +for additional parameters for the s and t format specifiers. + +See also `notmuch-saved-search-buffer-name-format'" + :type 'string + :group 'notmuch-search) + +(defcustom notmuch-saved-search-buffer-name-format "*notmuch-saved-%t-%s*" + "Format for the name of search results buffers for saved searches. + +In this spec, %s will be replaced by the saved search name and %t +by its type (search, tree or unthreaded). The buffer name is +formatted using `format-spec': see its docstring for additional +parameters for the s and t format specifiers. + +See also `notmuch-search-buffer-name-format'" + :type 'string + :group 'notmuch-search) + +(defun notmuch-search-format-buffer-name (query type saved) + "Compose a buffer name for the given QUERY, TYPE (search, tree, +unthreaded) and whether it's SAVED (t or nil)." + (let ((fmt (if saved + notmuch-saved-search-buffer-name-format + notmuch-search-buffer-name-format))) + (format-spec fmt `((?t . ,(or type "search")) (?s . ,query))))) + +(defun notmuch-search-buffer-title (query &optional type) "Returns the title for a buffer with notmuch search results." (let* ((saved-search (let (longest (longest-length 0)) - (loop for tuple in notmuch-saved-searches - if (let ((quoted-query (regexp-quote (notmuch-saved-search-get tuple :query)))) - (and (string-match (concat "^" quoted-query) query) - (> (length (match-string 0 query)) - longest-length))) - do (setq longest tuple)) + (cl-loop for tuple in notmuch-saved-searches + if (let ((quoted-query + (regexp-quote + (notmuch-saved-search-get tuple :query)))) + (and (string-match (concat "^" quoted-query) query) + (> (length (match-string 0 query)) + longest-length))) + do (setq longest tuple)) longest)) (saved-search-name (notmuch-saved-search-get saved-search :name)) + (saved-search-type (notmuch-saved-search-get saved-search :search-type)) (saved-search-query (notmuch-saved-search-get saved-search :query))) (cond ((and saved-search (equal saved-search-query query)) ;; Query is the same as saved search (ignoring case) - (concat "*notmuch-saved-search-" saved-search-name "*")) + (notmuch-search-format-buffer-name saved-search-name + saved-search-type + t)) (saved-search - (concat "*notmuch-search-" - (replace-regexp-in-string (concat "^" (regexp-quote saved-search-query)) - (concat "[ " saved-search-name " ]") - query) - "*")) - (t - (concat "*notmuch-search-" query "*")) - ))) + (let ((query (replace-regexp-in-string + (concat "^" (regexp-quote saved-search-query)) + (concat "[ " saved-search-name " ]") + query))) + (notmuch-search-format-buffer-name query saved-search-type t))) + (t (notmuch-search-format-buffer-name query type nil))))) (defun notmuch-read-query (prompt) "Read a notmuch-query from the minibuffer with completion. PROMPT is the string to prompt with." - (lexical-let* - ((all-tags - (mapcar (lambda (tag) (notmuch-escape-boolean-term tag)) - (process-lines notmuch-command "search" "--output=tags" "*"))) - (completions - (append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:" - "subject:" "attachment:") - (mapcar (lambda (tag) (concat "tag:" tag)) all-tags) - (mapcar (lambda (tag) (concat "is:" tag)) all-tags) - (mapcar (lambda (mimetype) (concat "mimetype:" mimetype)) (mailcap-mime-types))))) - (let ((keymap (copy-keymap minibuffer-local-map)) - (current-query (case major-mode - (notmuch-search-mode (notmuch-search-get-query)) - (notmuch-show-mode (notmuch-show-get-query)) - (notmuch-tree-mode (notmuch-tree-get-query)))) - (minibuffer-completion-table - (completion-table-dynamic - (lambda (string) - ;; generate a list of possible completions for the current input - (cond - ;; this ugly regexp is used to get the last word of the input - ;; possibly preceded by a '(' - ((string-match "\\(^\\|.* (?\\)\\([^ ]*\\)$" string) - (mapcar (lambda (compl) - (concat (match-string-no-properties 1 string) compl)) - (all-completions (match-string-no-properties 2 string) - completions))) - (t (list string))))))) - ;; this was simpler than convincing completing-read to accept spaces: - (define-key keymap (kbd "TAB") 'minibuffer-complete) - (let ((history-delete-duplicates t)) - (read-from-minibuffer prompt nil keymap nil - 'notmuch-search-history current-query nil))))) + (let* ((all-tags + (mapcar (lambda (tag) (notmuch-escape-boolean-term tag)) + (notmuch--process-lines notmuch-command "search" "--output=tags" "*"))) + (completions + (append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:" + "subject:" "attachment:") + (mapcar (lambda (tag) (concat "tag:" tag)) all-tags) + (mapcar (lambda (tag) (concat "is:" tag)) all-tags) + (mapcar (lambda (mimetype) (concat "mimetype:" mimetype)) + (mailcap-mime-types)))) + (keymap (copy-keymap minibuffer-local-map)) + (current-query (cl-case major-mode + (notmuch-search-mode (notmuch-search-get-query)) + (notmuch-show-mode (notmuch-show-get-query)) + (notmuch-tree-mode (notmuch-tree-get-query)))) + (minibuffer-completion-table + (completion-table-dynamic + (lambda (string) + ;; Generate a list of possible completions for the current input. + (cond + ;; This ugly regexp is used to get the last word of the input + ;; possibly preceded by a '('. + ((string-match "\\(^\\|.* (?\\)\\([^ ]*\\)$" string) + (mapcar (lambda (compl) + (concat (match-string-no-properties 1 string) compl)) + (all-completions (match-string-no-properties 2 string) + completions))) + (t (list string))))))) + ;; This was simpler than convincing completing-read to accept spaces: + (define-key keymap (kbd "TAB") 'minibuffer-complete) + (let ((history-delete-duplicates t)) + (read-from-minibuffer prompt nil keymap nil + 'notmuch-search-history current-query nil)))) (defun notmuch-search-get-query () - "Return the current query in this search buffer" + "Return the current query in this search buffer." notmuch-search-query-string) (put 'notmuch-search 'notmuch-doc "Search for messages.") ;;;###autoload -(defun notmuch-search (&optional query oldest-first target-thread target-line no-display) +(defun notmuch-search (&optional query oldest-first hide-excluded target-thread + target-line no-display) "Display threads matching QUERY in a notmuch-search buffer. If QUERY is nil, it is read interactively from the minibuffer. Other optional parameters are used as follows: OLDEST-FIRST: A Boolean controlling the sort order of returned threads + HIDE-EXCLUDED: A boolean controlling whether to omit threads with excluded + tags. TARGET-THREAD: A thread ID (without the thread: prefix) that will be made current if it appears in the search results. TARGET-LINE: The line number to move to if the target thread does not @@ -971,46 +1068,48 @@ the configured default sort order." (list ;; Prompt for a query nil - ;; Use the default search order (if we're doing a search from a - ;; search buffer, ignore any buffer-local overrides) - (default-value 'notmuch-search-oldest-first))) + ;; Use the default search order and exclude value (if we're doing a + ;; search from a search buffer, ignore any buffer-local overrides) + (default-value 'notmuch-search-oldest-first) + (default-value 'notmuch-search-hide-excluded))) (let* ((query (or query (notmuch-read-query "Notmuch search: "))) (buffer (get-buffer-create (notmuch-search-buffer-title query)))) (if no-display (set-buffer buffer) - (switch-to-buffer buffer)) + (pop-to-buffer-same-window buffer)) (notmuch-search-mode) ;; Don't track undo information for this buffer - (set 'buffer-undo-list t) - (set 'notmuch-search-query-string query) - (set 'notmuch-search-oldest-first oldest-first) - (set 'notmuch-search-target-thread target-thread) - (set 'notmuch-search-target-line target-line) + (setq buffer-undo-list t) + (setq notmuch-search-query-string query) + (setq notmuch-search-oldest-first oldest-first) + (setq notmuch-search-target-thread target-thread) + (setq notmuch-search-target-line target-line) + (setq notmuch-search-hide-excluded hide-excluded) (notmuch-tag-clear-cache) - (let ((proc (get-buffer-process (current-buffer))) - (inhibit-read-only t)) - (if proc - (error "notmuch search process already running for query `%s'" query) - ) + (when (get-buffer-process buffer) + (error "notmuch search process already running for query `%s'" query)) + (let ((inhibit-read-only t)) (erase-buffer) (goto-char (point-min)) (save-excursion (let ((proc (notmuch-start-notmuch "notmuch-search" buffer #'notmuch-search-process-sentinel - "search" "--format=sexp" "--format-version=4" + "search" "--format=sexp" "--format-version=5" (if oldest-first "--sort=oldest-first" "--sort=newest-first") - query)) - ;; Use a scratch buffer to accumulate partial output. - ;; This buffer will be killed by the sentinel, which - ;; should be called no matter how the process dies. - (parse-buf (generate-new-buffer " *notmuch search parse*"))) - (process-put proc 'parse-buf parse-buf) + (if hide-excluded + "--exclude=true" + "--exclude=false") + query))) + ;; Use a scratch buffer to accumulate partial output. + ;; This buffer will be killed by the sentinel, which + ;; should be called no matter how the process dies. + (process-put proc 'parse-buf + (generate-new-buffer " *notmuch search parse*")) (set-process-filter proc 'notmuch-search-process-filter) - (set-process-query-on-exit-flag proc nil)))) - (run-hooks 'notmuch-search-hook))) + (set-process-query-on-exit-flag proc nil)))))) (defun notmuch-search-refresh-view () "Refresh the current view. @@ -1021,13 +1120,22 @@ the new search results, then point will be placed on the same thread. Otherwise, point will be moved to attempt to be in the same relative position within the new buffer." (interactive) - (let ((target-line (line-number-at-pos)) - (oldest-first notmuch-search-oldest-first) - (target-thread (notmuch-search-find-thread-id 'bare)) - (query notmuch-search-query-string)) - ;; notmuch-search erases the current buffer. - (notmuch-search query oldest-first target-thread target-line t) - (goto-char (point-min)))) + (notmuch-search notmuch-search-query-string + notmuch-search-oldest-first + notmuch-search-hide-excluded + (notmuch-search-find-thread-id 'bare) + (line-number-at-pos) + t) + (goto-char (point-min))) + +(defun notmuch-search-toggle-hide-excluded () + "Toggle whether to hide excluded messages. + +This command toggles whether to hide excluded messages for the current +search. The default value for this is defined by `notmuch-search-hide-excluded'." + (interactive) + (setq notmuch-search-hide-excluded (not notmuch-search-hide-excluded)) + (notmuch-search-refresh-view)) (defun notmuch-search-toggle-order () "Toggle the current search order. @@ -1035,15 +1143,13 @@ same relative position within the new buffer." This command toggles the sort order for the current search. The default sort order is defined by `notmuch-search-oldest-first'." (interactive) - (set 'notmuch-search-oldest-first (not notmuch-search-oldest-first)) + (setq notmuch-search-oldest-first (not notmuch-search-oldest-first)) (notmuch-search-refresh-view)) (defun notmuch-group-disjunctive-query-string (query-string) "Group query if it contains a complex expression. - -Enclose QUERY-STRING in parentheses if it matches -`notmuch-search-disjunctive-regexp'." - (if (string-match-p notmuch-search-disjunctive-regexp query-string) +Enclose QUERY-STRING in parentheses if contains \"OR\" operators." + (if (string-match-p "\\<[oO][rR]\\>" query-string) (concat "( " query-string " )") query-string)) @@ -1059,16 +1165,34 @@ current search results AND the additional query string provided." (notmuch-search (if (string= grouped-original-query "*") grouped-query (concat grouped-original-query " and " grouped-query)) - notmuch-search-oldest-first))) + notmuch-search-oldest-first + notmuch-search-hide-excluded))) (defun notmuch-search-filter-by-tag (tag) - "Filter the current search results based on a single tag. + "Filter the current search results based on a single TAG. -Runs a new search matching only messages that match both the -current search results AND that are tagged with the given tag." +Run a new search matching only messages that match the current +search results and that are also tagged with the given TAG." (interactive - (list (notmuch-select-tag-with-completion "Filter by tag: " notmuch-search-query-string))) - (notmuch-search (concat notmuch-search-query-string " and tag:" tag) notmuch-search-oldest-first)) + (list (notmuch-select-tag-with-completion "Filter by tag: " + notmuch-search-query-string))) + (notmuch-search (concat notmuch-search-query-string " and tag:" tag) + notmuch-search-oldest-first + notmuch-search-hide-excluded)) + +(defun notmuch-search-by-tag (tag) + "Display threads matching TAG in a notmuch-search buffer." + (interactive + (list (notmuch-select-tag-with-completion "Notmuch search tag: "))) + (notmuch-search (concat "tag:" tag) + (default-value 'notmuch-search-oldest-first) + (default-value 'notmuch-search-hide-excluded))) + +(defun notmuch-search-edit-search (query) + "Edit the current search" + (interactive (list (read-from-minibuffer "Edit search: " + notmuch-search-query-string))) + (notmuch-search query notmuch-search-oldest-first)) ;;;###autoload (defun notmuch () @@ -1077,7 +1201,7 @@ current search results AND that are tagged with the given tag." (notmuch-hello)) (defun notmuch-interesting-buffer (b) - "Is the current buffer of interest to a notmuch user?" + "Whether the current buffer's major-mode is a notmuch mode." (with-current-buffer b (memq major-mode '(notmuch-show-mode notmuch-search-mode @@ -1089,10 +1213,9 @@ current search results AND that are tagged with the given tag." (defun notmuch-cycle-notmuch-buffers () "Cycle through any existing notmuch buffers (search, show or hello). -If the current buffer is the only notmuch buffer, bury it. If no -notmuch buffers exist, run `notmuch'." +If the current buffer is the only notmuch buffer, bury it. +If no notmuch buffers exist, run `notmuch'." (interactive) - (let (start first) ;; If the current buffer is a notmuch buffer, remember it and then ;; bury it. @@ -1101,42 +1224,46 @@ notmuch buffers exist, run `notmuch'." (bury-buffer)) ;; Find the first notmuch buffer. - (setq first (loop for buffer in (buffer-list) - if (notmuch-interesting-buffer buffer) - return buffer)) + (setq first (cl-loop for buffer in (buffer-list) + if (notmuch-interesting-buffer buffer) + return buffer)) (if first ;; If the first one we found is any other than the starting ;; buffer, switch to it. (unless (eq first start) - (switch-to-buffer first)) + (pop-to-buffer-same-window first)) (notmuch)))) +;;; Integrations +;;;; Hl-line Support + +(defun notmuch-hl-line-mode () + (prog1 (hl-line-mode) + (when hl-line-overlay + (overlay-put hl-line-overlay 'priority 1)))) + ;;;; Imenu Support (defun notmuch-search-imenu-prev-index-position-function () "Move point to previous message in notmuch-search buffer. -This function is used as a value for -`imenu-prev-index-position-function'." +Used as`imenu-prev-index-position-function' in notmuch buffers." (notmuch-search-previous-thread)) (defun notmuch-search-imenu-extract-index-name-function () "Return imenu name for line at point. -This function is used as a value for -`imenu-extract-index-name-function'. Point should be at the -beginning of the line." - (let ((subject (notmuch-search-find-subject)) - (author (notmuch-search-find-authors))) - (format "%s (%s)" subject author))) +Used as `imenu-extract-index-name-function' in notmuch buffers. +Point should be at the beginning of the line." + (format "%s (%s)" + (notmuch-search-find-subject) + (notmuch-search-find-authors))) -(setq mail-user-agent 'notmuch-user-agent) +;;; _ (provide 'notmuch) ;; After provide to avoid loops if notmuch was require'd via notmuch-init-file. -(if init-file-user ; don't load init file if the -q option was used. - (let ((init-file (locate-file notmuch-init-file '("/") - (get-load-suffixes)))) - (if init-file (load init-file nil t t)))) +(when init-file-user ; don't load init file if the -q option was used. + (load notmuch-init-file t t nil t)) ;;; notmuch.el ends here diff --git a/emacs/rstdoc.el b/emacs/rstdoc.el index 2225aefc..5b8a9d01 100644 --- a/emacs/rstdoc.el +++ b/emacs/rstdoc.el @@ -1,4 +1,4 @@ -;;; rstdoc.el --- help generate documentation from docstrings -*-lexical-binding: t-*- +;;; rstdoc.el --- help generate documentation from docstrings -*- lexical-binding: t -*- ;; Copyright (C) 2018 David Bremner @@ -24,7 +24,6 @@ ;; ;;; Commentary: -;; ;; Rstdoc provides a facility to extract all of the docstrings defined in ;; an elisp source file. Usage: @@ -33,16 +32,15 @@ ;;; Code: -(provide 'rstdoc) - (defun rstdoc-batch-extract () - "Extract docstrings to and from the files on the command line" + "Extract docstrings to and from the files on the command line." (apply #'rstdoc-extract command-line-args-left)) (defun rstdoc-extract (in-file out-file) - "Write docstrings from IN-FILE to OUT-FILE" + "Write docstrings from IN-FILE to OUT-FILE." (load-file in-file) (let* ((definitions (cdr (assoc (expand-file-name in-file) load-history))) + (text-quoting-style 'grave) (doc-hash (make-hash-table :test 'eq))) (mapc (lambda (elt) @@ -63,14 +61,19 @@ (defun rstdoc--insert-docstring (symbol docstring) (insert (format "\n.. |docstring::%s| replace::\n" symbol)) - (insert (replace-regexp-in-string "^" " " (rstdoc--rst-quote-string docstring))) + (insert (replace-regexp-in-string "^" " " + (rstdoc--rst-quote-string docstring))) (insert "\n")) (defvar rst--escape-alist - '( ("\\\\='" . "\\\\'") - ("\\([^\\]\\)'" . "\\1`") - ("^[[:space:]\t]*$" . "|br|") - ("^[[:space:]\t]" . "|indent| ")) + '( ("\\\\='" . "\001") + ("`\\([^\n`']*\\)[`']" . "\002\\1\002") ;; good enough for now... + ("`" . "\\\\`") + ("\001" . "'") + ("\002" . "`") + ("[*]" . "\\\\*") + ("^[[:space:]]*$" . "|br|") + ("^[[:space:]]" . "|indent| ")) "list of (regex . replacement) pairs") (defun rstdoc--rst-quote-string (str) @@ -82,4 +85,6 @@ (replace-match (cdr pair)))) (buffer-substring (point-min) (point-max)))) +(provide 'rstdoc) + ;;; rstdoc.el ends here diff --git a/gmime-filter-reply.c b/gmime-filter-reply.c index 2b067669..35349cc8 100644 --- a/gmime-filter-reply.c +++ b/gmime-filter-reply.c @@ -43,29 +43,31 @@ static void filter_reset (GMimeFilter *filter); static GMimeFilterClass *parent_class = NULL; +static GType type = 0; +static const GTypeInfo info = { + .class_size = sizeof (GMimeFilterReplyClass), + .base_init = NULL, + .base_finalize = NULL, + .class_init = (GClassInitFunc) g_mime_filter_reply_class_init, + .class_finalize = NULL, + .class_data = NULL, + .instance_size = sizeof (GMimeFilterReply), + .n_preallocs = 0, + .instance_init = (GInstanceInitFunc) g_mime_filter_reply_init, + .value_table = NULL, +}; + + +void +g_mime_filter_reply_module_init (void) +{ + type = g_type_register_static (GMIME_TYPE_FILTER, "GMimeFilterReply", &info, (GTypeFlags) 0); + parent_class = (GMimeFilterClass *) g_type_class_ref (GMIME_TYPE_FILTER); +} GType g_mime_filter_reply_get_type (void) { - static GType type = 0; - - if (! type) { - static const GTypeInfo info = { - .class_size = sizeof (GMimeFilterReplyClass), - .base_init = NULL, - .base_finalize = NULL, - .class_init = (GClassInitFunc) g_mime_filter_reply_class_init, - .class_finalize = NULL, - .class_data = NULL, - .instance_size = sizeof (GMimeFilterReply), - .n_preallocs = 0, - .instance_init = (GInstanceInitFunc) g_mime_filter_reply_init, - .value_table = NULL, - }; - - type = g_type_register_static (GMIME_TYPE_FILTER, "GMimeFilterReply", &info, (GTypeFlags) 0); - } - return type; } @@ -76,8 +78,6 @@ g_mime_filter_reply_class_init (GMimeFilterReplyClass *klass, unused (void *clas GObjectClass *object_class = G_OBJECT_CLASS (klass); GMimeFilterClass *filter_class = GMIME_FILTER_CLASS (klass); - parent_class = (GMimeFilterClass *) g_type_class_ref (GMIME_TYPE_FILTER); - object_class->finalize = g_mime_filter_reply_finalize; filter_class->copy = filter_copy; diff --git a/gmime-filter-reply.h b/gmime-filter-reply.h index 5a1e606e..988fe2d6 100644 --- a/gmime-filter-reply.h +++ b/gmime-filter-reply.h @@ -21,14 +21,22 @@ #include +void g_mime_filter_reply_module_init (void); + G_BEGIN_DECLS #define GMIME_TYPE_FILTER_REPLY (g_mime_filter_reply_get_type ()) -#define GMIME_FILTER_REPLY(obj) (G_TYPE_CHECK_INSTANCE_CAST ((obj), GMIME_TYPE_FILTER_REPLY, GMimeFilterReply)) -#define GMIME_FILTER_REPLY_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST ((klass), GMIME_TYPE_FILTER_REPLY, GMimeFilterReplyClass)) -#define GMIME_IS_FILTER_REPLY(obj) (G_TYPE_CHECK_INSTANCE_TYPE ((obj), GMIME_TYPE_FILTER_REPLY)) -#define GMIME_IS_FILTER_REPLY_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), GMIME_TYPE_FILTER_REPLY)) -#define GMIME_FILTER_REPLY_GET_CLASS(obj) (G_TYPE_INSTANCE_GET_CLASS ((obj), GMIME_TYPE_FILTER_REPLY, GMimeFilterReplyClass)) +#define GMIME_FILTER_REPLY(obj) (G_TYPE_CHECK_INSTANCE_CAST ((obj), \ + GMIME_TYPE_FILTER_REPLY, \ + GMimeFilterReply)) +#define GMIME_FILTER_REPLY_CLASS(klass) (G_TYPE_CHECK_CLASS_CAST ((klass), GMIME_TYPE_FILTER_REPLY, \ + GMimeFilterReplyClass)) +#define GMIME_IS_FILTER_REPLY(obj) (G_TYPE_CHECK_INSTANCE_TYPE ((obj), \ + GMIME_TYPE_FILTER_REPLY)) +#define GMIME_IS_FILTER_REPLY_CLASS(klass) (G_TYPE_CHECK_CLASS_TYPE ((klass), \ + GMIME_TYPE_FILTER_REPLY)) +#define GMIME_FILTER_REPLY_GET_CLASS(obj) (G_TYPE_INSTANCE_GET_CLASS ((obj), GMIME_TYPE_FILTER_REPLY, \ + GMimeFilterReplyClass)) typedef struct _GMimeFilterReply GMimeFilterReply; typedef struct _GMimeFilterReplyClass GMimeFilterReplyClass; diff --git a/hooks.c b/hooks.c index 59c58070..0cf72e74 100644 --- a/hooks.c +++ b/hooks.c @@ -24,19 +24,27 @@ #include int -notmuch_run_hook (const char *db_path, const char *hook) +notmuch_run_hook (notmuch_database_t *notmuch, const char *hook) { char *hook_path; + const char *config_path; int status = 0; pid_t pid; - hook_path = talloc_asprintf (NULL, "%s/%s/%s/%s", db_path, ".notmuch", - "hooks", hook); + hook_path = talloc_asprintf (notmuch, "%s/%s", + notmuch_config_get (notmuch, NOTMUCH_CONFIG_HOOK_DIR), + hook); if (hook_path == NULL) { fprintf (stderr, "Out of memory\n"); return 1; } + config_path = notmuch_config_path (notmuch); + if (setenv ("NOTMUCH_CONFIG", config_path, 1)) { + perror ("setenv"); + return 1; + } + /* Check access before fork() for speed and simplicity of error handling. */ if (access (hook_path, X_OK) == -1) { /* Ignore ENOENT. It's okay not to have a hook, hook dir, or even diff --git a/lib/Makefile.local b/lib/Makefile.local index 5dc057c0..4e766305 100644 --- a/lib/Makefile.local +++ b/lib/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := lib @@ -59,7 +59,14 @@ libnotmuch_cxx_srcs = \ $(dir)/config.cc \ $(dir)/regexp-fields.cc \ $(dir)/thread.cc \ - $(dir)/thread-fp.cc + $(dir)/thread-fp.cc \ + $(dir)/features.cc \ + $(dir)/prefix.cc \ + $(dir)/open.cc \ + $(dir)/init.cc \ + $(dir)/parse-sexp.cc \ + $(dir)/sexp-fp.cc \ + $(dir)/lastmod-fp.cc libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o) diff --git a/lib/add-message.cc b/lib/add-message.cc index 8c92689b..b16748fd 100644 --- a/lib/add-message.cc +++ b/lib/add-message.cc @@ -40,20 +40,14 @@ parse_references (void *ctx, static const char * _notmuch_database_generate_thread_id (notmuch_database_t *notmuch) { - /* 16 bytes (+ terminator) for hexadecimal representation of - * a 64-bit integer. */ - static char thread_id[17]; - Xapian::WritableDatabase *db; - - db = static_cast (notmuch->xapian_db); notmuch->last_thread_id++; - sprintf (thread_id, "%016" PRIx64, notmuch->last_thread_id); + sprintf (notmuch->thread_id_str, "%016" PRIx64, notmuch->last_thread_id); - db->set_metadata ("last_thread_id", thread_id); + notmuch->writable_xapian_db->set_metadata ("last_thread_id", notmuch->thread_id_str); - return thread_id; + return notmuch->thread_id_str; } static char * @@ -161,7 +155,7 @@ _resolve_message_id_to_thread_id_old (notmuch_database_t *notmuch, * can return the thread ID stored in the metadata. Otherwise, we * generate a new thread ID and store it there. */ - db = static_cast (notmuch->xapian_db); + db = notmuch->writable_xapian_db; metadata_key = _get_metadata_thread_id_key (ctx, message_id); thread_id_string = notmuch->xapian_db->get_metadata (metadata_key); @@ -370,13 +364,9 @@ _consume_metadata_thread_id (void *ctx, notmuch_database_t *notmuch, if (stored_id.empty ()) { return NULL; } else { - Xapian::WritableDatabase *db; - - db = static_cast (notmuch->xapian_db); - /* Clear the metadata for this message ID. We don't need it * anymore. */ - db->set_metadata (metadata_key, ""); + notmuch->writable_xapian_db->set_metadata (metadata_key, ""); return talloc_strdup (ctx, stored_id.c_str ()); } @@ -414,14 +404,17 @@ static notmuch_status_t _notmuch_database_link_message (notmuch_database_t *notmuch, notmuch_message_t *message, notmuch_message_file_t *message_file, - bool is_ghost) + bool is_ghost, + bool is_new) { void *local = talloc_new (NULL); notmuch_status_t status; const char *thread_id = NULL; /* Check if the message already had a thread ID */ - if (notmuch->features & NOTMUCH_FEATURE_GHOSTS) { + if (! is_new) { + thread_id = notmuch_message_get_thread_id (message); + } else if (notmuch->features & NOTMUCH_FEATURE_GHOSTS) { if (is_ghost) thread_id = notmuch_message_get_thread_id (message); } else { @@ -477,7 +470,7 @@ notmuch_database_index_file (notmuch_database_t *notmuch, notmuch_message_t *message = NULL; notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS, ret2; notmuch_private_status_t private_status; - bool is_ghost = false, is_new = false; + notmuch_bool_t is_ghost = false, is_new = false; notmuch_indexopts_t *def_indexopts = NULL; const char *date; @@ -525,7 +518,9 @@ notmuch_database_index_file (notmuch_database_t *notmuch, is_new = true; break; case NOTMUCH_PRIVATE_STATUS_SUCCESS: - is_ghost = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_GHOST); + ret = notmuch_message_get_flag_st (message, NOTMUCH_MESSAGE_FLAG_GHOST, &is_ghost); + if (ret) + goto DONE; is_new = false; break; default: @@ -534,7 +529,9 @@ notmuch_database_index_file (notmuch_database_t *notmuch, goto DONE; } - _notmuch_message_add_filename (message, filename); + ret = _notmuch_message_add_filename (message, filename); + if (ret) + goto DONE; if (is_new || is_ghost) { _notmuch_message_add_term (message, "type", "mail"); @@ -544,7 +541,7 @@ notmuch_database_index_file (notmuch_database_t *notmuch, } ret = _notmuch_database_link_message (notmuch, message, - message_file, is_ghost); + message_file, is_ghost, is_new); if (ret) goto DONE; diff --git a/lib/built-with.c b/lib/built-with.c index 320be6c5..275e72b8 100644 --- a/lib/built-with.c +++ b/lib/built-with.c @@ -25,13 +25,15 @@ notmuch_bool_t notmuch_built_with (const char *name) { if (STRNCMP_LITERAL (name, "compact") == 0) { - return HAVE_XAPIAN_COMPACT; + return true; } else if (STRNCMP_LITERAL (name, "field_processor") == 0) { - return HAVE_XAPIAN_FIELD_PROCESSOR; + return true; } else if (STRNCMP_LITERAL (name, "retry_lock") == 0) { return HAVE_XAPIAN_DB_RETRY_LOCK; } else if (STRNCMP_LITERAL (name, "session_key") == 0) { return true; + } else if (STRNCMP_LITERAL (name, "sexp_queries") == 0) { + return HAVE_SFSEXP; } else { return false; } diff --git a/lib/config.cc b/lib/config.cc index 8ee4da01..6cd15fab 100644 --- a/lib/config.cc +++ b/lib/config.cc @@ -22,6 +22,9 @@ #include "notmuch-private.h" #include "database-private.h" +#include +#include + static const std::string CONFIG_PREFIX = "C"; struct _notmuch_config_list { @@ -31,6 +34,20 @@ struct _notmuch_config_list { char *current_val; }; +struct _notmuch_config_values { + const char *iterator; + size_t tok_len; + const char *string; + void *children; /* talloc_context */ +}; + +struct _notmuch_config_pairs { + notmuch_string_map_iterator_t *iter; +}; + +static const char *_notmuch_config_key_to_string (notmuch_config_key_t key); +static char *_expand_path (void *ctx, const char *key, const char *val); + static int _notmuch_config_list_destroy (notmuch_config_list_t *list) { @@ -45,21 +62,30 @@ notmuch_database_set_config (notmuch_database_t *notmuch, const char *value) { notmuch_status_t status; - Xapian::WritableDatabase *db; status = _notmuch_database_ensure_writable (notmuch); if (status) return status; + if (! notmuch->config) { + if ((status = _notmuch_config_load_from_database (notmuch))) + return status; + } + try { - db = static_cast (notmuch->xapian_db); - db->set_metadata (CONFIG_PREFIX + key, value); + notmuch->writable_xapian_db->set_metadata (CONFIG_PREFIX + key, value); } catch (const Xapian::Error &error) { status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; notmuch->exception_reported = true; _notmuch_database_log (notmuch, "Error: A Xapian exception occurred setting metadata: %s\n", error.get_msg ().c_str ()); } + + if (status) + return status; + + _notmuch_string_map_set (notmuch->config, key, value); + return NOTMUCH_STATUS_SUCCESS; } @@ -86,17 +112,25 @@ notmuch_database_get_config (notmuch_database_t *notmuch, const char *key, char **value) { - std::string strval; + const char *stored_val; notmuch_status_t status; + if (! notmuch->config) { + if ((status = _notmuch_config_load_from_database (notmuch))) + return status; + } + if (! value) return NOTMUCH_STATUS_NULL_POINTER; - status = _metadata_value (notmuch, key, strval); - if (status) - return status; - - *value = strdup (strval.c_str ()); + stored_val = _notmuch_string_map_get (notmuch->config, key); + if (! stored_val) { + /* XXX in principle this API should be fixed so empty string + * is distinguished from not found */ + *value = strdup (""); + } else { + *value = strdup (stored_val); + } return NOTMUCH_STATUS_SUCCESS; } @@ -115,7 +149,6 @@ notmuch_database_get_config_list (notmuch_database_t *notmuch, goto DONE; } - talloc_set_destructor (list, _notmuch_config_list_destroy); list->notmuch = notmuch; list->current_key = NULL; list->current_val = NULL; @@ -124,9 +157,11 @@ notmuch_database_get_config_list (notmuch_database_t *notmuch, new(&(list->iterator)) Xapian::TermIterator (notmuch->xapian_db->metadata_keys_begin (CONFIG_PREFIX + (prefix ? prefix : ""))); + talloc_set_destructor (list, _notmuch_config_list_destroy); } catch (const Xapian::Error &error) { - _notmuch_database_log (notmuch, "A Xapian exception occurred getting metadata iterator: %s.\n", + _notmuch_database_log (notmuch, + "A Xapian exception occurred getting metadata iterator: %s.\n", error.get_msg ().c_str ()); notmuch->exception_reported = true; status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; @@ -135,8 +170,15 @@ notmuch_database_get_config_list (notmuch_database_t *notmuch, *out = list; DONE: - if (status && list) - talloc_free (list); + if (status) { + if (list) { + talloc_free (list); + if (status != NOTMUCH_STATUS_XAPIAN_EXCEPTION) + _notmuch_config_list_destroy (list); + } + } else { + talloc_set_destructor (list, _notmuch_config_list_destroy); + } return status; } @@ -150,13 +192,19 @@ notmuch_config_list_valid (notmuch_config_list_t *metadata) return true; } +static inline char * +_key_from_iterator (notmuch_config_list_t *list) +{ + return talloc_strdup (list, (*list->iterator).c_str () + CONFIG_PREFIX.length ()); +} + const char * notmuch_config_list_key (notmuch_config_list_t *list) { if (list->current_key) talloc_free (list->current_key); - list->current_key = talloc_strdup (list, (*list->iterator).c_str () + CONFIG_PREFIX.length ()); + list->current_key = _key_from_iterator (list); return list->current_key; } @@ -166,7 +214,7 @@ notmuch_config_list_value (notmuch_config_list_t *list) { std::string strval; notmuch_status_t status; - const char *key = notmuch_config_list_key (list); + char *key = _key_from_iterator (list); /* TODO: better error reporting?? */ status = _metadata_value (list->notmuch, key, strval); @@ -177,6 +225,7 @@ notmuch_config_list_value (notmuch_config_list_t *list) talloc_free (list->current_val); list->current_val = talloc_strdup (list, strval.c_str ()); + talloc_free (key); return list->current_val; } @@ -191,3 +240,487 @@ notmuch_config_list_destroy (notmuch_config_list_t *list) { talloc_free (list); } + +notmuch_status_t +_notmuch_config_load_from_database (notmuch_database_t *notmuch) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + notmuch_config_list_t *list; + + if (notmuch->config == NULL) + notmuch->config = _notmuch_string_map_create (notmuch); + + if (unlikely (notmuch->config == NULL)) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + status = notmuch_database_get_config_list (notmuch, "", &list); + if (status) + return status; + + for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { + const char *key = notmuch_config_list_key (list); + char *normalized_val = NULL; + + /* If we opened from a given path, do not overwrite it */ + if (strcmp (key, "database.path") == 0 && + (notmuch->params & NOTMUCH_PARAM_DATABASE) && + notmuch->xapian_db) + continue; + + normalized_val = _expand_path (list, key, notmuch_config_list_value (list)); + _notmuch_string_map_append (notmuch->config, key, normalized_val); + talloc_free (normalized_val); + } + + return status; +} + +notmuch_config_values_t * +notmuch_config_get_values (notmuch_database_t *notmuch, notmuch_config_key_t key) +{ + const char *key_str = _notmuch_config_key_to_string (key); + + if (! key_str) + return NULL; + + return notmuch_config_get_values_string (notmuch, key_str); +} + +notmuch_config_values_t * +notmuch_config_get_values_string (notmuch_database_t *notmuch, const char *key_str) +{ + notmuch_config_values_t *values = NULL; + bool ok = false; + + values = talloc (notmuch, notmuch_config_values_t); + if (unlikely (! values)) + goto DONE; + + values->children = talloc_new (values); + + values->string = _notmuch_string_map_get (notmuch->config, key_str); + if (! values->string) + goto DONE; + + values->iterator = strsplit_len (values->string, ';', &(values->tok_len)); + ok = true; + + DONE: + if (! ok) { + if (values) + talloc_free (values); + return NULL; + } + return values; +} + +notmuch_bool_t +notmuch_config_values_valid (notmuch_config_values_t *values) +{ + if (! values) + return false; + + return (values->iterator != NULL); +} + +const char * +notmuch_config_values_get (notmuch_config_values_t *values) +{ + return talloc_strndup (values->children, values->iterator, values->tok_len); +} + +void +notmuch_config_values_start (notmuch_config_values_t *values) +{ + if (values == NULL) + return; + if (values->children) { + talloc_free (values->children); + } + + values->children = talloc_new (values); + + values->iterator = strsplit_len (values->string, ';', &(values->tok_len)); +} + +void +notmuch_config_values_move_to_next (notmuch_config_values_t *values) +{ + values->iterator += values->tok_len; + values->iterator = strsplit_len (values->iterator, ';', &(values->tok_len)); +} + +void +notmuch_config_values_destroy (notmuch_config_values_t *values) +{ + talloc_free (values); +} + +notmuch_config_pairs_t * +notmuch_config_get_pairs (notmuch_database_t *notmuch, + const char *prefix) +{ + notmuch_config_pairs_t *pairs = talloc (notmuch, notmuch_config_pairs_t); + + pairs->iter = _notmuch_string_map_iterator_create (notmuch->config, prefix, false); + return pairs; +} + +notmuch_bool_t +notmuch_config_pairs_valid (notmuch_config_pairs_t *pairs) +{ + return _notmuch_string_map_iterator_valid (pairs->iter); +} + +void +notmuch_config_pairs_move_to_next (notmuch_config_pairs_t *pairs) +{ + _notmuch_string_map_iterator_move_to_next (pairs->iter); +} + +const char * +notmuch_config_pairs_key (notmuch_config_pairs_t *pairs) +{ + return _notmuch_string_map_iterator_key (pairs->iter); +} + +const char * +notmuch_config_pairs_value (notmuch_config_pairs_t *pairs) +{ + return _notmuch_string_map_iterator_value (pairs->iter); +} + +void +notmuch_config_pairs_destroy (notmuch_config_pairs_t *pairs) +{ + _notmuch_string_map_iterator_destroy (pairs->iter); + talloc_free (pairs); +} + +static char * +_expand_path (void *ctx, const char *key, const char *val) +{ + char *expanded_val; + + if ((strcmp (key, "database.path") == 0 || + strcmp (key, "database.mail_root") == 0 || + strcmp (key, "database.hook_dir") == 0 || + strcmp (key, "database.backup_path") == 0 ) && + val[0] != '/') + expanded_val = talloc_asprintf (ctx, "%s/%s", getenv ("HOME"), val); + else + expanded_val = talloc_strdup (ctx, val); + + return expanded_val; +} + +notmuch_status_t +_notmuch_config_load_from_file (notmuch_database_t *notmuch, + GKeyFile *file, + char **status_string) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + gchar **groups = NULL, **keys, *val; + + if (notmuch->config == NULL) + notmuch->config = _notmuch_string_map_create (notmuch); + + if (unlikely (notmuch->config == NULL)) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + groups = g_key_file_get_groups (file, NULL); + for (gchar **grp = groups; *grp; grp++) { + keys = g_key_file_get_keys (file, *grp, NULL, NULL); + for (gchar **keys_p = keys; *keys_p; keys_p++) { + char *absolute_key = talloc_asprintf (notmuch, "%s.%s", *grp, *keys_p); + char *normalized_val; + GError *gerr = NULL; + + /* If we opened from a given path, do not overwrite it */ + if (strcmp (absolute_key, "database.path") == 0 && + (notmuch->params & NOTMUCH_PARAM_DATABASE) && + notmuch->xapian_db) + continue; + + val = g_key_file_get_string (file, *grp, *keys_p, &gerr); + if (gerr) { + if (status_string) + IGNORE_RESULT (asprintf (status_string, + "GLib: %s\n", + gerr->message)); + g_error_free (gerr); + } + if (! val) { + status = NOTMUCH_STATUS_FILE_ERROR; + goto DONE; + } + + normalized_val = _expand_path (notmuch, absolute_key, val); + _notmuch_string_map_set (notmuch->config, absolute_key, normalized_val); + g_free (val); + talloc_free (absolute_key); + talloc_free (normalized_val); + if (status) + goto DONE; + } + g_strfreev (keys); + } + + DONE: + if (groups) + g_strfreev (groups); + + return status; +} + +notmuch_status_t +notmuch_config_get_bool (notmuch_database_t *notmuch, notmuch_config_key_t key, notmuch_bool_t *val) +{ + const char *key_string, *val_string; + + key_string = _notmuch_config_key_to_string (key); + if (! key_string) { + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + } + + val_string = _notmuch_string_map_get (notmuch->config, key_string); + if (! val_string) { + *val = FALSE; + return NOTMUCH_STATUS_SUCCESS; + } + + if (strcase_equal (val_string, "false") || strcase_equal (val_string, "no")) + *val = FALSE; + else if (strcase_equal (val_string, "true") || strcase_equal (val_string, "yes")) + *val = TRUE; + else + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + + return NOTMUCH_STATUS_SUCCESS; +} + +static const char * +_get_name_from_passwd_file (void *ctx) +{ + long pw_buf_size; + char *pw_buf; + struct passwd passwd, *ignored; + const char *name; + int e; + + pw_buf_size = sysconf (_SC_GETPW_R_SIZE_MAX); + if (pw_buf_size == -1) pw_buf_size = 64; + pw_buf = (char *) talloc_size (ctx, pw_buf_size); + + while ((e = getpwuid_r (getuid (), &passwd, pw_buf, + pw_buf_size, &ignored)) == ERANGE) { + pw_buf_size = pw_buf_size * 2; + pw_buf = (char *) talloc_zero_size (ctx, pw_buf_size); + } + + if (e == 0) { + char *comma = strchr (passwd.pw_gecos, ','); + if (comma) + name = talloc_strndup (ctx, passwd.pw_gecos, + comma - passwd.pw_gecos); + else + name = talloc_strdup (ctx, passwd.pw_gecos); + } else { + name = talloc_strdup (ctx, ""); + } + + talloc_free (pw_buf); + + return name; +} + +static char * +_get_username_from_passwd_file (void *ctx) +{ + long pw_buf_size; + char *pw_buf; + struct passwd passwd, *ignored; + char *name; + int e; + + pw_buf_size = sysconf (_SC_GETPW_R_SIZE_MAX); + if (pw_buf_size == -1) pw_buf_size = 64; + pw_buf = (char *) talloc_zero_size (ctx, pw_buf_size); + + while ((e = getpwuid_r (getuid (), &passwd, pw_buf, + pw_buf_size, &ignored)) == ERANGE) { + pw_buf_size = pw_buf_size * 2; + pw_buf = (char *) talloc_zero_size (ctx, pw_buf_size); + } + + if (e == 0) + name = talloc_strdup (ctx, passwd.pw_name); + else + name = talloc_strdup (ctx, ""); + + talloc_free (pw_buf); + + return name; +} + +static const char * +_get_email_from_passwd_file (void *ctx) +{ + char *email; + + char *username = _get_username_from_passwd_file (ctx); + + email = talloc_asprintf (ctx, "%s@localhost", username); + + talloc_free (username); + return email; +} + +static const char * +_notmuch_config_key_to_string (notmuch_config_key_t key) +{ + switch (key) { + case NOTMUCH_CONFIG_DATABASE_PATH: + return "database.path"; + case NOTMUCH_CONFIG_MAIL_ROOT: + return "database.mail_root"; + case NOTMUCH_CONFIG_HOOK_DIR: + return "database.hook_dir"; + case NOTMUCH_CONFIG_BACKUP_DIR: + return "database.backup_dir"; + case NOTMUCH_CONFIG_EXCLUDE_TAGS: + return "search.exclude_tags"; + case NOTMUCH_CONFIG_NEW_TAGS: + return "new.tags"; + case NOTMUCH_CONFIG_NEW_IGNORE: + return "new.ignore"; + case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS: + return "maildir.synchronize_flags"; + case NOTMUCH_CONFIG_PRIMARY_EMAIL: + return "user.primary_email"; + case NOTMUCH_CONFIG_OTHER_EMAIL: + return "user.other_email"; + case NOTMUCH_CONFIG_USER_NAME: + return "user.name"; + case NOTMUCH_CONFIG_AUTOCOMMIT: + return "database.autocommit"; + case NOTMUCH_CONFIG_EXTRA_HEADERS: + return "show.extra_headers"; + case NOTMUCH_CONFIG_INDEX_AS_TEXT: + return "index.as_text"; + default: + return NULL; + } +} + +static const char * +_notmuch_config_default (notmuch_database_t *notmuch, notmuch_config_key_t key) +{ + char *path; + const char *name, *email; + + switch (key) { + case NOTMUCH_CONFIG_DATABASE_PATH: + path = getenv ("MAILDIR"); + if (path) + path = talloc_strdup (notmuch, path); + else + path = talloc_asprintf (notmuch, "%s/mail", + getenv ("HOME")); + return path; + case NOTMUCH_CONFIG_MAIL_ROOT: + /* by default, mail root is the same as database path */ + return notmuch_database_get_path (notmuch); + case NOTMUCH_CONFIG_EXCLUDE_TAGS: + return ""; + case NOTMUCH_CONFIG_NEW_TAGS: + return "unread;inbox"; + case NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS: + return "true"; + case NOTMUCH_CONFIG_USER_NAME: + name = getenv ("NAME"); + if (name) + name = talloc_strdup (notmuch, name); + else + name = _get_name_from_passwd_file (notmuch); + return name; + case NOTMUCH_CONFIG_PRIMARY_EMAIL: + email = getenv ("EMAIL"); + if (email) + email = talloc_strdup (notmuch, email); + else + email = _get_email_from_passwd_file (notmuch); + return email; + case NOTMUCH_CONFIG_INDEX_AS_TEXT: + case NOTMUCH_CONFIG_NEW_IGNORE: + return ""; + case NOTMUCH_CONFIG_AUTOCOMMIT: + return "8000"; + case NOTMUCH_CONFIG_EXTRA_HEADERS: + case NOTMUCH_CONFIG_HOOK_DIR: + case NOTMUCH_CONFIG_BACKUP_DIR: + case NOTMUCH_CONFIG_OTHER_EMAIL: + return NULL; + default: + case NOTMUCH_CONFIG_LAST: + INTERNAL_ERROR ("illegal key enum %d", key); + } +} + +notmuch_status_t +_notmuch_config_load_defaults (notmuch_database_t *notmuch) +{ + notmuch_config_key_t key; + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + + if (notmuch->config == NULL) + notmuch->config = _notmuch_string_map_create (notmuch); + + for (key = NOTMUCH_CONFIG_FIRST; + key < NOTMUCH_CONFIG_LAST; + key = notmuch_config_key_t (key + 1)) { + const char *val = notmuch_config_get (notmuch, key); + const char *key_string = _notmuch_config_key_to_string (key); + + val = _notmuch_string_map_get (notmuch->config, key_string); + if (! val) { + if (key == NOTMUCH_CONFIG_MAIL_ROOT && (notmuch->params & NOTMUCH_PARAM_SPLIT)) + status = NOTMUCH_STATUS_NO_MAIL_ROOT; + + _notmuch_string_map_set (notmuch->config, key_string, _notmuch_config_default (notmuch, + key)); + } + } + return status; +} + +const char * +notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key) +{ + + return _notmuch_string_map_get (notmuch->config, _notmuch_config_key_to_string (key)); +} + +const char * +notmuch_config_path (notmuch_database_t *notmuch) +{ + return notmuch->config_path; +} + +notmuch_status_t +notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val) +{ + + return notmuch_database_set_config (notmuch, _notmuch_config_key_to_string (key), val); +} + +void +_notmuch_config_cache (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val) +{ + if (notmuch->config == NULL) + notmuch->config = _notmuch_string_map_create (notmuch); + + _notmuch_string_map_set (notmuch->config, _notmuch_config_key_to_string (key), val); +} diff --git a/lib/database-private.h b/lib/database-private.h index 87ae1bdf..61232f1a 100644 --- a/lib/database-private.h +++ b/lib/database-private.h @@ -32,12 +32,18 @@ #include "notmuch-private.h" +#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0])) + #ifdef SILENCE_XAPIAN_DEPRECATION_WARNINGS #define XAPIAN_DEPRECATED(D) D #endif #include +#if HAVE_SFSEXP +#include +#endif + /* Bit masks for _notmuch_database::features. Features are named, * independent aspects of the database schema. * @@ -154,11 +160,12 @@ operator&= (_notmuch_features &a, _notmuch_features b) /* * Configuration options for xapian database fields */ -typedef enum notmuch_field_flags { +typedef enum { NOTMUCH_FIELD_NO_FLAGS = 0, NOTMUCH_FIELD_EXTERNAL = 1 << 0, NOTMUCH_FIELD_PROBABILISTIC = 1 << 1, NOTMUCH_FIELD_PROCESSOR = 1 << 2, + NOTMUCH_FIELD_STRIP_TRAILING_SLASH = 1 << 3, } notmuch_field_flag_t; /* @@ -184,24 +191,75 @@ operator& (notmuch_field_flag_t a, notmuch_field_flag_t b) Xapian::QueryParser::FLAG_WILDCARD | \ Xapian::QueryParser::FLAG_PURE_NOT) +/* + * explicit and implied parameters to open */ +typedef enum { + NOTMUCH_PARAM_NONE = 0, + /* database passed explicitely */ + NOTMUCH_PARAM_DATABASE = 1 << 0, + /* config file passed explicitely */ + NOTMUCH_PARAM_CONFIG = 1 << 1, + /* profile name passed explicitely */ + NOTMUCH_PARAM_PROFILE = 1 << 2, + /* split (e.g. XDG) configuration */ + NOTMUCH_PARAM_SPLIT = 1 << 3, +} notmuch_open_param_t; + +/* + * define bitwise operators to hide casts */ + +inline notmuch_open_param_t +operator| (notmuch_open_param_t a, notmuch_open_param_t b) +{ + return static_cast( + static_cast(a) | static_cast(b)); +} + +inline notmuch_open_param_t& +operator|= (notmuch_open_param_t &a, notmuch_open_param_t b) +{ + a = a | b; + return a; +} + +inline notmuch_open_param_t +operator& (notmuch_open_param_t a, notmuch_open_param_t b) +{ + return static_cast( + static_cast(a) & static_cast(b)); +} + struct _notmuch_database { bool exception_reported; - char *path; + /* Path to actual database */ + const char *xapian_path; + + /* Path to config loaded, if any */ + const char *config_path; - notmuch_database_mode_t mode; int atomic_nesting; /* true if changes have been made in this atomic section */ bool atomic_dirty; Xapian::Database *xapian_db; - + Xapian::WritableDatabase *writable_xapian_db; + bool open; /* Bit mask of features used by this database. This is a * bitwise-OR of NOTMUCH_FEATURE_* values (above). */ enum _notmuch_features features; unsigned int last_doc_id; + + /* 16 bytes (+ terminator) for hexadecimal representation of + * a 64-bit integer. */ + char thread_id_str[17]; uint64_t last_thread_id; + /* How many transactions have successfully completed since we last committed */ + int transaction_count; + /* when to commit and reset the counter */ + int transaction_threshold; + /* error reporting; this value persists only until the * next library call. May be NULL */ char *status_string; @@ -217,15 +275,26 @@ struct _notmuch_database { */ unsigned long view; Xapian::QueryParser *query_parser; + Xapian::Stem *stemmer; Xapian::TermGenerator *term_gen; - Xapian::ValueRangeProcessor *value_range_processor; - Xapian::ValueRangeProcessor *date_range_processor; - Xapian::ValueRangeProcessor *last_mod_range_processor; + Xapian::RangeProcessor *value_range_processor; + Xapian::RangeProcessor *date_range_processor; + Xapian::RangeProcessor *last_mod_range_processor; /* XXX it's slightly gross to use two parallel string->string maps * here, but at least they are small */ notmuch_string_map_t *user_prefix; notmuch_string_map_t *user_header; + + /* Cached and possibly overridden configuration */ + notmuch_string_map_t *config; + + /* Track what parameters were specified when opening */ + notmuch_open_param_t params; + + /* list of regular expressions to check for text indexing */ + regex_t *index_as_text; + size_t index_as_text_length; }; /* Prior to database version 3, features were implied by the database @@ -263,4 +332,64 @@ _notmuch_database_find_doc_ids (notmuch_database_t *notmuch, const char *value, Xapian::PostingIterator *begin, Xapian::PostingIterator *end); + +#define NOTMUCH_DATABASE_VERSION 3 + +/* features.cc */ + +_notmuch_features +_notmuch_database_parse_features (const void *ctx, const char *features, unsigned int version, + char mode, char **incompat_out); + +char * +_notmuch_database_print_features (const void *ctx, unsigned int features); + +/* prefix.cc */ +notmuch_status_t +_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch); + +notmuch_status_t +_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch); + +#if __cplusplus +/* query.cc */ +notmuch_status_t +_notmuch_query_string_to_xapian_query (notmuch_database_t *notmuch, + std::string query_string, + Xapian::Query &output, + std::string &msg); + +notmuch_status_t +_notmuch_query_expand (notmuch_database_t *notmuch, const char *field, Xapian::Query subquery, + Xapian::Query &output, std::string &msg); + +/* regexp-fields.cc */ +notmuch_status_t +_notmuch_regexp_to_query (notmuch_database_t *notmuch, Xapian::valueno slot, std::string field, + std::string regexp_str, + Xapian::Query &output, std::string &msg); + +/* thread-fp.cc */ +notmuch_status_t +_notmuch_query_name_to_query (notmuch_database_t *notmuch, const std::string name, + Xapian::Query &output); + +#if HAVE_SFSEXP +/* parse-sexp.cc */ +notmuch_status_t +_notmuch_sexp_string_to_xapian_query (notmuch_database_t *notmuch, const char *querystr, + Xapian::Query &output); +#endif + +/* parse-time-vrp.h */ +notmuch_status_t +_notmuch_date_strings_to_query (Xapian::valueno slot, const std::string &from, const std::string &to, + Xapian::Query &output, std::string &msg); + +/* lastmod-fp.h */ +notmuch_status_t +_notmuch_lastmod_strings_to_query (notmuch_database_t *notmuch, + const std::string &from, const std::string &to, + Xapian::Query &output, std::string &msg); +#endif #endif diff --git a/lib/database.cc b/lib/database.cc index 24b7ec43..737a3f30 100644 --- a/lib/database.cc +++ b/lib/database.cc @@ -19,10 +19,6 @@ */ #include "database-private.h" -#include "parse-time-vrp.h" -#include "query-fp.h" -#include "thread-fp.h" -#include "regexp-fields.h" #include "string-util.h" #include @@ -39,8 +35,6 @@ using namespace std; -#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0])) - typedef struct { const char *name; const char *prefix; @@ -52,11 +46,26 @@ typedef struct { #define STRINGIFY(s) _SUB_STRINGIFY (s) #define _SUB_STRINGIFY(s) #s -#if HAVE_XAPIAN_DB_RETRY_LOCK -#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK) -#else -#define DB_ACTION Xapian::DB_CREATE_OR_OPEN -#endif +#define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception (__location__, message, error) + +static void +_log_xapian_exception (const char *where, notmuch_database_t *notmuch, const Xapian::Error error) +{ + _notmuch_database_log (notmuch, + "A Xapian exception occurred at %s: %s\n", + where, + error.get_msg ().c_str ()); + notmuch->exception_reported = true; +} + +notmuch_database_mode_t +_notmuch_database_mode (notmuch_database_t *notmuch) +{ + if (notmuch->writable_xapian_db) + return NOTMUCH_DATABASE_MODE_READ_WRITE; + else + return NOTMUCH_DATABASE_MODE_READ_ONLY; +} /* Here's the current schema for our database (for NOTMUCH_DATABASE_VERSION): * @@ -245,82 +254,6 @@ typedef struct { * same thread. */ -/* With these prefix values we follow the conventions published here: - * - * https://xapian.org/docs/omega/termprefixes.html - * - * as much as makes sense. Note that I took some liberty in matching - * the reserved prefix values to notmuch concepts, (for example, 'G' - * is documented as "newsGroup (or similar entity - e.g. a web forum - * name)", for which I think the thread is the closest analogue in - * notmuch. This in spite of the fact that we will eventually be - * storing mailing-list messages where 'G' for "mailing list name" - * might be even a closer analogue. I'm treating the single-character - * prefixes preferentially for core notmuch concepts (which will be - * nearly universal to all mail messages). - */ - -static const -prefix_t prefix_table[] = { - /* name term prefix flags */ - { "type", "T", NOTMUCH_FIELD_NO_FLAGS }, - { "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS }, - { "replyto", "XREPLYTO", NOTMUCH_FIELD_NO_FLAGS }, - { "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS }, - { "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, - { "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, - { "body", "", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "thread", "G", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "tag", "K", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "is", "K", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "id", "Q", NOTMUCH_FIELD_EXTERNAL }, - { "mid", "Q", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "path", "P", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "property", "XPROPERTY", NOTMUCH_FIELD_EXTERNAL }, - /* - * Unconditionally add ':' to reduce potential ambiguity with - * overlapping prefixes and/or terms that start with capital - * letters. See Xapian document termprefixes.html for related - * discussion. - */ - { "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, -#if HAVE_XAPIAN_FIELD_PROCESSOR - { "date", NULL, NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, - { "query", NULL, NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROCESSOR }, -#endif - { "from", "XFROM", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC | - NOTMUCH_FIELD_PROCESSOR }, - { "to", "XTO", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "attachment", "XATTACHMENT", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "mimetype", "XMIMETYPE", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC }, - { "subject", "XSUBJECT", NOTMUCH_FIELD_EXTERNAL | - NOTMUCH_FIELD_PROBABILISTIC | - NOTMUCH_FIELD_PROCESSOR }, -}; - -static void -_setup_query_field_default (const prefix_t *prefix, notmuch_database_t *notmuch) -{ - if (prefix->prefix) - notmuch->query_parser->add_prefix ("", prefix->prefix); - if (prefix->flags & NOTMUCH_FIELD_PROBABILISTIC) - notmuch->query_parser->add_prefix (prefix->name, prefix->prefix); - else - notmuch->query_parser->add_boolean_prefix (prefix->name, prefix->prefix); -} notmuch_string_map_iterator_t * _notmuch_database_user_headers (notmuch_database_t *notmuch) @@ -328,161 +261,6 @@ _notmuch_database_user_headers (notmuch_database_t *notmuch) return _notmuch_string_map_iterator_create (notmuch->user_header, "", false); } -const char * -_user_prefix (void *ctx, const char *name) -{ - return talloc_asprintf (ctx, "XU%s:", name); -} - -static notmuch_status_t -_setup_user_query_fields (notmuch_database_t *notmuch) -{ - notmuch_config_list_t *list; - notmuch_status_t status; - - notmuch->user_prefix = _notmuch_string_map_create (notmuch); - if (notmuch->user_prefix == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - - notmuch->user_header = _notmuch_string_map_create (notmuch); - if (notmuch->user_header == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - - status = notmuch_database_get_config_list (notmuch, CONFIG_HEADER_PREFIX, &list); - if (status) - return status; - - for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { - - prefix_t query_field; - - const char *key = notmuch_config_list_key (list) - + sizeof (CONFIG_HEADER_PREFIX) - 1; - - _notmuch_string_map_append (notmuch->user_prefix, - key, - _user_prefix (notmuch, key)); - - _notmuch_string_map_append (notmuch->user_header, - key, - notmuch_config_list_value (list)); - - query_field.name = talloc_strdup (notmuch, key); - query_field.prefix = _user_prefix (notmuch, key); - query_field.flags = NOTMUCH_FIELD_PROBABILISTIC - | NOTMUCH_FIELD_EXTERNAL; - - _setup_query_field_default (&query_field, notmuch); - } - - notmuch_config_list_destroy (list); - - return NOTMUCH_STATUS_SUCCESS; -} - -#if HAVE_XAPIAN_FIELD_PROCESSOR -static void -_setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) -{ - if (prefix->flags & NOTMUCH_FIELD_PROCESSOR) { - Xapian::FieldProcessor *fp; - - if (STRNCMP_LITERAL (prefix->name, "date") == 0) - fp = (new DateFieldProcessor ())->release (); - else if (STRNCMP_LITERAL (prefix->name, "query") == 0) - fp = (new QueryFieldProcessor (*notmuch->query_parser, notmuch))->release (); - else if (STRNCMP_LITERAL (prefix->name, "thread") == 0) - fp = (new ThreadFieldProcessor (*notmuch->query_parser, notmuch))->release (); - else - fp = (new RegexpFieldProcessor (prefix->name, prefix->flags, - *notmuch->query_parser, notmuch))->release (); - - /* we treat all field-processor fields as boolean in order to get the raw input */ - if (prefix->prefix) - notmuch->query_parser->add_prefix ("", prefix->prefix); - notmuch->query_parser->add_boolean_prefix (prefix->name, fp); - } else { - _setup_query_field_default (prefix, notmuch); - } -} -#else -static inline void -_setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) -{ - _setup_query_field_default (prefix, notmuch); -} -#endif - -const char * -_find_prefix (const char *name) -{ - unsigned int i; - - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - if (strcmp (name, prefix_table[i].name) == 0) - return prefix_table[i].prefix; - } - - INTERNAL_ERROR ("No prefix exists for '%s'\n", name); - - return ""; -} - -/* Like find prefix, but include the possibility of user defined - * prefixes specific to this database */ - -const char * -_notmuch_database_prefix (notmuch_database_t *notmuch, const char *name) -{ - unsigned int i; - - /*XXX TODO: reduce code duplication */ - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - if (strcmp (name, prefix_table[i].name) == 0) - return prefix_table[i].prefix; - } - - if (notmuch->user_prefix) - return _notmuch_string_map_get (notmuch->user_prefix, name); - - return NULL; -} - -static const struct { - /* NOTMUCH_FEATURE_* value. */ - _notmuch_features value; - /* Feature name as it appears in the database. This name should - * be appropriate for displaying to the user if an older version - * of notmuch doesn't support this feature. */ - const char *name; - /* Compatibility flags when this feature is declared. */ - const char *flags; -} feature_names[] = { - { NOTMUCH_FEATURE_FILE_TERMS, - "multiple paths per message", "rw" }, - { NOTMUCH_FEATURE_DIRECTORY_DOCS, - "relative directory paths", "rw" }, - /* Header values are not required for reading a database because a - * reader can just refer to the message file. */ - { NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES, - "from/subject/message-ID in database", "w" }, - { NOTMUCH_FEATURE_BOOL_FOLDER, - "exact folder:/path: search", "rw" }, - { NOTMUCH_FEATURE_GHOSTS, - "mail documents for missing messages", "w" }, - /* Knowledge of the index mime-types are not required for reading - * a database because a reader will just be unable to query - * them. */ - { NOTMUCH_FEATURE_INDEXED_MIMETYPES, - "indexed MIME types", "w" }, - { NOTMUCH_FEATURE_LAST_MOD, - "modification tracking", "w" }, - /* Existing databases will work fine for all queries not involving - * 'body:' */ - { NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY, - "index body and headers separately", "w" }, -}; - const char * notmuch_status_to_string (notmuch_status_t status) { @@ -515,12 +293,26 @@ notmuch_status_to_string (notmuch_status_t status) return "Operation requires a database upgrade"; case NOTMUCH_STATUS_PATH_ERROR: return "Path supplied is illegal for this function"; + case NOTMUCH_STATUS_IGNORED: + return "Argument was ignored"; + case NOTMUCH_STATUS_ILLEGAL_ARGUMENT: + return "Illegal argument for function"; case NOTMUCH_STATUS_MALFORMED_CRYPTO_PROTOCOL: return "Crypto protocol missing, malformed, or unintelligible"; case NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION: return "Crypto engine initialization failure"; case NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL: return "Unknown crypto protocol"; + case NOTMUCH_STATUS_NO_CONFIG: + return "No configuration file found"; + case NOTMUCH_STATUS_NO_DATABASE: + return "No database found"; + case NOTMUCH_STATUS_DATABASE_EXISTS: + return "Database exists, not recreated"; + case NOTMUCH_STATUS_BAD_QUERY_SYNTAX: + return "Syntax error in query"; + case NOTMUCH_STATUS_NO_MAIL_ROOT: + return "No mail root found"; default: case NOTMUCH_STATUS_LAST_STATUS: return "Unknown error status value"; @@ -676,117 +468,19 @@ notmuch_database_find_message (notmuch_database_t *notmuch, } } -notmuch_status_t -notmuch_database_create (const char *path, notmuch_database_t **database) -{ - char *status_string = NULL; - notmuch_status_t status; - - status = notmuch_database_create_verbose (path, database, - &status_string); - - if (status_string) { - fputs (status_string, stderr); - free (status_string); - } - - return status; -} - -notmuch_status_t -notmuch_database_create_verbose (const char *path, - notmuch_database_t **database, - char **status_string) -{ - notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; - notmuch_database_t *notmuch = NULL; - char *notmuch_path = NULL; - char *message = NULL; - struct stat st; - int err; - - if (path == NULL) { - message = strdup ("Error: Cannot create a database for a NULL path.\n"); - status = NOTMUCH_STATUS_NULL_POINTER; - goto DONE; - } - - if (path[0] != '/') { - message = strdup ("Error: Database path must be absolute.\n"); - status = NOTMUCH_STATUS_PATH_ERROR; - goto DONE; - } - - err = stat (path, &st); - if (err) { - IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: %s.\n", - path, strerror (errno))); - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - if (! S_ISDIR (st.st_mode)) { - IGNORE_RESULT (asprintf (&message, "Error: Cannot create database at %s: " - "Not a directory.\n", - path)); - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - notmuch_path = talloc_asprintf (NULL, "%s/%s", path, ".notmuch"); - - err = mkdir (notmuch_path, 0755); - - if (err) { - IGNORE_RESULT (asprintf (&message, "Error: Cannot create directory %s: %s.\n", - notmuch_path, strerror (errno))); - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - status = notmuch_database_open_verbose (path, - NOTMUCH_DATABASE_MODE_READ_WRITE, - ¬much, &message); - if (status) - goto DONE; - - /* Upgrade doesn't add these feature to existing databases, but - * new databases have them. */ - notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES; - notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES; - notmuch->features |= NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY; - - status = notmuch_database_upgrade (notmuch, NULL, NULL); - if (status) { - notmuch_database_close (notmuch); - notmuch = NULL; - } - - DONE: - if (notmuch_path) - talloc_free (notmuch_path); - - if (message) { - if (status_string) - *status_string = message; - else - free (message); - } - if (database) - *database = notmuch; - else - talloc_free (notmuch); - return status; -} - notmuch_status_t _notmuch_database_ensure_writable (notmuch_database_t *notmuch) { - if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY) { + if (_notmuch_database_mode (notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY) { _notmuch_database_log (notmuch, "Cannot write to a read-only database.\n"); return NOTMUCH_STATUS_READ_ONLY_DATABASE; } + if (! notmuch->open) { + _notmuch_database_log (notmuch, "Cannot write to a closed database.\n"); + return NOTMUCH_STATUS_CLOSED_DATABASE; + } + return NOTMUCH_STATUS_SUCCESS; } @@ -807,289 +501,6 @@ _notmuch_database_new_revision (notmuch_database_t *notmuch) return new_revision; } -/* Parse a database features string from the given database version. - * Returns the feature bit set. - * - * For version < 3, this ignores the features string and returns a - * hard-coded set of features. - * - * If there are unrecognized features that are required to open the - * database in mode (which should be 'r' or 'w'), return a - * comma-separated list of unrecognized but required features in - * *incompat_out suitable for presenting to the user. *incompat_out - * will be allocated from ctx. - */ -static _notmuch_features -_parse_features (const void *ctx, const char *features, unsigned int version, - char mode, char **incompat_out) -{ - _notmuch_features res = static_cast<_notmuch_features>(0); - unsigned int namelen, i; - size_t llen = 0; - const char *flags; - - /* Prior to database version 3, features were implied by the - * version number. */ - if (version == 0) - return NOTMUCH_FEATURES_V0; - else if (version == 1) - return NOTMUCH_FEATURES_V1; - else if (version == 2) - return NOTMUCH_FEATURES_V2; - - /* Parse the features string */ - while ((features = strtok_len_c (features + llen, "\n", &llen)) != NULL) { - flags = strchr (features, '\t'); - if (! flags || flags > features + llen) - continue; - namelen = flags - features; - - for (i = 0; i < ARRAY_SIZE (feature_names); ++i) { - if (strlen (feature_names[i].name) == namelen && - strncmp (feature_names[i].name, features, namelen) == 0) { - res |= feature_names[i].value; - break; - } - } - - if (i == ARRAY_SIZE (feature_names) && incompat_out) { - /* Unrecognized feature */ - const char *have = strchr (flags, mode); - if (have && have < features + llen) { - /* This feature is required to access this database in - * 'mode', but we don't understand it. */ - if (! *incompat_out) - *incompat_out = talloc_strdup (ctx, ""); - *incompat_out = talloc_asprintf_append_buffer ( - *incompat_out, "%s%.*s", **incompat_out ? ", " : "", - namelen, features); - } - } - } - - return res; -} - -static char * -_print_features (const void *ctx, unsigned int features) -{ - unsigned int i; - char *res = talloc_strdup (ctx, ""); - - for (i = 0; i < ARRAY_SIZE (feature_names); ++i) - if (features & feature_names[i].value) - res = talloc_asprintf_append_buffer ( - res, "%s\t%s\n", feature_names[i].name, feature_names[i].flags); - - return res; -} - -notmuch_status_t -notmuch_database_open (const char *path, - notmuch_database_mode_t mode, - notmuch_database_t **database) -{ - char *status_string = NULL; - notmuch_status_t status; - - status = notmuch_database_open_verbose (path, mode, database, - &status_string); - - if (status_string) { - fputs (status_string, stderr); - free (status_string); - } - - return status; -} - -notmuch_status_t -notmuch_database_open_verbose (const char *path, - notmuch_database_mode_t mode, - notmuch_database_t **database, - char **status_string) -{ - notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; - void *local = talloc_new (NULL); - notmuch_database_t *notmuch = NULL; - char *notmuch_path, *xapian_path, *incompat_features; - char *message = NULL; - struct stat st; - int err; - unsigned int i, version; - static int initialized = 0; - - if (path == NULL) { - message = strdup ("Error: Cannot open a database for a NULL path.\n"); - status = NOTMUCH_STATUS_NULL_POINTER; - goto DONE; - } - - if (path[0] != '/') { - message = strdup ("Error: Database path must be absolute.\n"); - status = NOTMUCH_STATUS_PATH_ERROR; - goto DONE; - } - - if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) { - message = strdup ("Out of memory\n"); - status = NOTMUCH_STATUS_OUT_OF_MEMORY; - goto DONE; - } - - err = stat (notmuch_path, &st); - if (err) { - IGNORE_RESULT (asprintf (&message, "Error opening database at %s: %s\n", - notmuch_path, strerror (errno))); - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"))) { - message = strdup ("Out of memory\n"); - status = NOTMUCH_STATUS_OUT_OF_MEMORY; - goto DONE; - } - - /* Initialize the GLib type system and threads */ -#if ! GLIB_CHECK_VERSION (2, 35, 1) - g_type_init (); -#endif - - /* Initialize gmime */ - if (! initialized) { - g_mime_init (); - initialized = 1; - } - - notmuch = talloc_zero (NULL, notmuch_database_t); - notmuch->exception_reported = false; - notmuch->status_string = NULL; - notmuch->path = talloc_strdup (notmuch, path); - - strip_trailing (notmuch->path, '/'); - - notmuch->mode = mode; - notmuch->atomic_nesting = 0; - notmuch->view = 1; - try { - string last_thread_id; - string last_mod; - - if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) { - notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path, - DB_ACTION); - } else { - notmuch->xapian_db = new Xapian::Database (xapian_path); - } - - /* Check version. As of database version 3, we represent - * changes in terms of features, so assume a version bump - * means a dramatically incompatible change. */ - version = notmuch_database_get_version (notmuch); - if (version > NOTMUCH_DATABASE_VERSION) { - IGNORE_RESULT (asprintf (&message, - "Error: Notmuch database at %s\n" - " has a newer database format version (%u) than supported by this\n" - " version of notmuch (%u).\n", - notmuch_path, version, NOTMUCH_DATABASE_VERSION)); - notmuch->mode = NOTMUCH_DATABASE_MODE_READ_ONLY; - notmuch_database_destroy (notmuch); - notmuch = NULL; - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - /* Check features. */ - incompat_features = NULL; - notmuch->features = _parse_features ( - local, notmuch->xapian_db->get_metadata ("features").c_str (), - version, mode == NOTMUCH_DATABASE_MODE_READ_WRITE ? 'w' : 'r', - &incompat_features); - if (incompat_features) { - IGNORE_RESULT (asprintf (&message, - "Error: Notmuch database at %s\n" - " requires features (%s)\n" - " not supported by this version of notmuch.\n", - notmuch_path, incompat_features)); - notmuch->mode = NOTMUCH_DATABASE_MODE_READ_ONLY; - notmuch_database_destroy (notmuch); - notmuch = NULL; - status = NOTMUCH_STATUS_FILE_ERROR; - goto DONE; - } - - notmuch->last_doc_id = notmuch->xapian_db->get_lastdocid (); - last_thread_id = notmuch->xapian_db->get_metadata ("last_thread_id"); - if (last_thread_id.empty ()) { - notmuch->last_thread_id = 0; - } else { - const char *str; - char *end; - - str = last_thread_id.c_str (); - notmuch->last_thread_id = strtoull (str, &end, 16); - if (*end != '\0') - INTERNAL_ERROR ("Malformed database last_thread_id: %s", str); - } - - /* Get current highest revision number. */ - last_mod = notmuch->xapian_db->get_value_upper_bound ( - NOTMUCH_VALUE_LAST_MOD); - if (last_mod.empty ()) - notmuch->revision = 0; - else - notmuch->revision = Xapian::sortable_unserialise (last_mod); - notmuch->uuid = talloc_strdup ( - notmuch, notmuch->xapian_db->get_uuid ().c_str ()); - - notmuch->query_parser = new Xapian::QueryParser; - notmuch->term_gen = new Xapian::TermGenerator; - notmuch->term_gen->set_stemmer (Xapian::Stem ("english")); - notmuch->value_range_processor = new Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_TIMESTAMP); - notmuch->date_range_processor = new ParseTimeValueRangeProcessor (NOTMUCH_VALUE_TIMESTAMP); - notmuch->last_mod_range_processor = new Xapian::NumberValueRangeProcessor (NOTMUCH_VALUE_LAST_MOD, "lastmod:"); - - notmuch->query_parser->set_default_op (Xapian::Query::OP_AND); - notmuch->query_parser->set_database (*notmuch->xapian_db); - notmuch->query_parser->set_stemmer (Xapian::Stem ("english")); - notmuch->query_parser->set_stemming_strategy (Xapian::QueryParser::STEM_SOME); - notmuch->query_parser->add_valuerangeprocessor (notmuch->value_range_processor); - notmuch->query_parser->add_valuerangeprocessor (notmuch->date_range_processor); - notmuch->query_parser->add_valuerangeprocessor (notmuch->last_mod_range_processor); - - for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { - const prefix_t *prefix = &prefix_table[i]; - if (prefix->flags & NOTMUCH_FIELD_EXTERNAL) { - _setup_query_field (prefix, notmuch); - } - } - status = _setup_user_query_fields (notmuch); - } catch (const Xapian::Error &error) { - IGNORE_RESULT (asprintf (&message, "A Xapian exception occurred opening database: %s\n", - error.get_msg ().c_str ())); - notmuch_database_destroy (notmuch); - notmuch = NULL; - status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; - } - - DONE: - talloc_free (local); - - if (message) { - if (status_string) - *status_string = message; - else - free (message); - } - - if (database) - *database = notmuch; - else - talloc_free (notmuch); - return status; -} - notmuch_status_t notmuch_database_close (notmuch_database_t *notmuch) { @@ -1098,68 +509,25 @@ notmuch_database_close (notmuch_database_t *notmuch) /* Many Xapian objects (and thus notmuch objects) hold references to * the database, so merely deleting the database may not suffice to * close it. Thus, we explicitly close it here. */ - if (notmuch->xapian_db != NULL) { + if (notmuch->open) { try { - /* If there's an outstanding transaction, it's unclear if - * closing the Xapian database commits everything up to - * that transaction, or may discard committed (but - * unflushed) transactions. To be certain, explicitly - * cancel any outstanding transaction before closing. */ - if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE && - notmuch->atomic_nesting) - (static_cast (notmuch->xapian_db)) - ->cancel_transaction (); - /* Close the database. This implicitly flushes - * outstanding changes. */ + * outstanding changes. If there is an open (non-flushed) + * transaction, ALL pending changes will be discarded */ notmuch->xapian_db->close (); } catch (const Xapian::Error &error) { status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; if (! notmuch->exception_reported) { - _notmuch_database_log (notmuch, "Error: A Xapian exception occurred closing database: %s\n", + _notmuch_database_log (notmuch, + "Error: A Xapian exception occurred closing database: %s\n", error.get_msg ().c_str ()); } } } - - delete notmuch->term_gen; - notmuch->term_gen = NULL; - delete notmuch->query_parser; - notmuch->query_parser = NULL; - delete notmuch->xapian_db; - notmuch->xapian_db = NULL; - delete notmuch->value_range_processor; - notmuch->value_range_processor = NULL; - delete notmuch->date_range_processor; - notmuch->date_range_processor = NULL; - delete notmuch->last_mod_range_processor; - notmuch->last_mod_range_processor = NULL; - + notmuch->open = false; return status; } -notmuch_status_t -_notmuch_database_reopen (notmuch_database_t *notmuch) -{ - if (notmuch->mode != NOTMUCH_DATABASE_MODE_READ_ONLY) - return NOTMUCH_STATUS_UNSUPPORTED_OPERATION; - - try { - notmuch->xapian_db->reopen (); - } catch (const Xapian::Error &error) { - if (! notmuch->exception_reported) { - _notmuch_database_log (notmuch, "Error: A Xapian exception reopening database: %s\n", - error.get_msg ().c_str ()); - notmuch->exception_reported = true; - } - return NOTMUCH_STATUS_XAPIAN_EXCEPTION; - } - - notmuch->view++; - - return NOTMUCH_STATUS_SUCCESS; -} - static int unlink_cb (const char *path, unused (const struct stat *sb), @@ -1225,36 +593,58 @@ notmuch_database_compact (const char *path, notmuch_compact_status_cb_t status_cb, void *closure) { - void *local; - char *notmuch_path, *xapian_path, *compact_xapian_path; notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS; notmuch_database_t *notmuch = NULL; - struct stat statbuf; - bool keep_backup; char *message = NULL; - local = talloc_new (NULL); - if (! local) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - - ret = notmuch_database_open_verbose (path, - NOTMUCH_DATABASE_MODE_READ_WRITE, - ¬much, - &message); + ret = notmuch_database_open_with_config (path, + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", + NULL, + ¬much, + &message); if (ret) { if (status_cb) status_cb (message, closure); - goto DONE; + return ret; } - if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) { - ret = NOTMUCH_STATUS_OUT_OF_MEMORY; - goto DONE; - } + _notmuch_config_cache (notmuch, NOTMUCH_CONFIG_DATABASE_PATH, path); - if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, "xapian"))) { - ret = NOTMUCH_STATUS_OUT_OF_MEMORY; + return notmuch_database_compact_db (notmuch, + backup_path, + status_cb, + closure); +} + +notmuch_status_t +notmuch_database_compact_db (notmuch_database_t *notmuch, + const char *backup_path, + notmuch_compact_status_cb_t status_cb, + void *closure) +{ + void *local; + const char *xapian_path, *compact_xapian_path; + const char *path; + notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS; + struct stat statbuf; + bool keep_backup; + char *message; + + ret = _notmuch_database_ensure_writable (notmuch); + if (ret) + return ret; + + path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH); + if (! path) + return NOTMUCH_STATUS_PATH_ERROR; + + local = talloc_new (NULL); + if (! local) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + ret = _notmuch_choose_xapian_path (local, path, &xapian_path, &message); + if (ret) goto DONE; - } if (! (compact_xapian_path = talloc_asprintf (local, "%s.compact", xapian_path))) { ret = NOTMUCH_STATUS_OUT_OF_MEMORY; @@ -1291,11 +681,8 @@ notmuch_database_compact (const char *path, try { NotmuchCompactor compactor (status_cb, closure); - - compactor.set_renumber (false); - compactor.add_source (xapian_path); - compactor.set_destdir (compact_xapian_path); - compactor.compact (); + notmuch->xapian_db->compact (compact_xapian_path, Xapian::DBCOMPACT_NO_RENUMBER, 0, + compactor); } catch (const Xapian::Error &error) { _notmuch_database_log (notmuch, "Error while compacting: %s\n", error.get_msg ().c_str ()); ret = NOTMUCH_STATUS_XAPIAN_EXCEPTION; @@ -1349,8 +736,33 @@ notmuch_status_t notmuch_database_destroy (notmuch_database_t *notmuch) { notmuch_status_t status; + const char *talloc_report; + + talloc_report = getenv ("NOTMUCH_TALLOC_REPORT"); + if (talloc_report && strcmp (talloc_report, "") != 0) { + FILE *report = fopen (talloc_report, "a"); + if (report) { + talloc_report_full (notmuch, report); + } + } status = notmuch_database_close (notmuch); + + delete notmuch->term_gen; + notmuch->term_gen = NULL; + delete notmuch->query_parser; + notmuch->query_parser = NULL; + delete notmuch->xapian_db; + notmuch->xapian_db = NULL; + delete notmuch->value_range_processor; + notmuch->value_range_processor = NULL; + delete notmuch->date_range_processor; + notmuch->date_range_processor = NULL; + delete notmuch->last_mod_range_processor; + notmuch->last_mod_range_processor = NULL; + delete notmuch->stemmer; + notmuch->stemmer = NULL; + talloc_free (notmuch); return status; @@ -1359,7 +771,7 @@ notmuch_database_destroy (notmuch_database_t *notmuch) const char * notmuch_database_get_path (notmuch_database_t *notmuch) { - return notmuch->path; + return notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH); } unsigned int @@ -1370,7 +782,13 @@ notmuch_database_get_version (notmuch_database_t *notmuch) const char *str; char *end; - version_string = notmuch->xapian_db->get_metadata ("version"); + try { + version_string = notmuch->xapian_db->get_metadata ("version"); + } catch (const Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (notmuch, error); + return 0; + } + if (version_string.empty ()) return 0; @@ -1388,9 +806,17 @@ notmuch_database_get_version (notmuch_database_t *notmuch) notmuch_bool_t notmuch_database_needs_upgrade (notmuch_database_t *notmuch) { - return notmuch->mode == NOTMUCH_DATABASE_MODE_READ_WRITE && - ((NOTMUCH_FEATURES_CURRENT & ~notmuch->features) || - (notmuch_database_get_version (notmuch) < NOTMUCH_DATABASE_VERSION)); + unsigned int version; + + if (_notmuch_database_mode (notmuch) != NOTMUCH_DATABASE_MODE_READ_WRITE) + return FALSE; + + if (NOTMUCH_FEATURES_CURRENT & ~notmuch->features) + return TRUE; + + version = notmuch_database_get_version (notmuch); + + return (version > 0 && version < NOTMUCH_DATABASE_VERSION); } static volatile sig_atomic_t do_progress_notify = 0; @@ -1431,11 +857,10 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, notmuch_query_t *query = NULL; unsigned int count = 0, total = 0; - status = _notmuch_database_ensure_writable (notmuch); - if (status) - return status; + if (_notmuch_database_mode (notmuch) != NOTMUCH_DATABASE_MODE_READ_WRITE) + return NOTMUCH_STATUS_READ_ONLY_DATABASE; - db = static_cast (notmuch->xapian_db); + db = notmuch->writable_xapian_db; target_features = notmuch->features | NOTMUCH_FEATURES_CURRENT; new_features = NOTMUCH_FEATURES_CURRENT & ~notmuch->features; @@ -1590,8 +1015,8 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, mtime = Xapian::sortable_unserialise ( document.get_value (NOTMUCH_VALUE_TIMESTAMP)); - directory = _notmuch_directory_create (notmuch, term.c_str () + 10, - NOTMUCH_FIND_CREATE, &status); + directory = _notmuch_directory_find_or_create (notmuch, term.c_str () + 10, + NOTMUCH_FIND_CREATE, &status); notmuch_directory_set_mtime (directory, mtime); notmuch_directory_destroy (directory); @@ -1640,7 +1065,8 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, if (private_status) { _notmuch_database_log (notmuch, "Upgrade failed while creating ghost messages.\n"); - status = COERCE_STATUS (private_status, "Unexpected status from _notmuch_message_initialize_ghost"); + status = COERCE_STATUS (private_status, + "Unexpected status from _notmuch_message_initialize_ghost"); goto DONE; } @@ -1652,7 +1078,7 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, } status = NOTMUCH_STATUS_SUCCESS; - db->set_metadata ("features", _print_features (local, notmuch->features)); + db->set_metadata ("features", _notmuch_database_print_features (local, notmuch->features)); db->set_metadata ("version", STRINGIFY (NOTMUCH_DATABASE_VERSION)); DONE: @@ -1684,7 +1110,7 @@ notmuch_database_upgrade (notmuch_database_t *notmuch, notmuch_status_t notmuch_database_begin_atomic (notmuch_database_t *notmuch) { - if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY || + if (_notmuch_database_mode (notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY || notmuch->atomic_nesting > 0) goto DONE; @@ -1692,7 +1118,7 @@ notmuch_database_begin_atomic (notmuch_database_t *notmuch) return NOTMUCH_STATUS_UPGRADE_REQUIRED; try { - (static_cast (notmuch->xapian_db))->begin_transaction (false); + notmuch->writable_xapian_db->begin_transaction (false); } catch (const Xapian::Error &error) { _notmuch_database_log (notmuch, "A Xapian exception occurred beginning transaction: %s.\n", error.get_msg ().c_str ()); @@ -1713,20 +1139,28 @@ notmuch_database_end_atomic (notmuch_database_t *notmuch) if (notmuch->atomic_nesting == 0) return NOTMUCH_STATUS_UNBALANCED_ATOMIC; - if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY || + if (_notmuch_database_mode (notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY || notmuch->atomic_nesting > 1) goto DONE; - db = static_cast (notmuch->xapian_db); + db = notmuch->writable_xapian_db; try { db->commit_transaction (); - - /* This is a hack for testing. Xapian never flushes on a - * non-flushed commit, even if the flush threshold is 1. - * However, we rely on flushing to test atomicity. */ + notmuch->transaction_count++; + + /* Xapian never flushes on a non-flushed commit, even if the + * flush threshold is 1. However, we rely on flushing to test + * atomicity. On the other hand, we can't straight replace + * XAPIAN_FLUSH_THRESHOLD with our autocommit counter, because + * the former also applies outside notmuch atomic + * commits. Hence the follow complicated test */ const char *thresh = getenv ("XAPIAN_FLUSH_THRESHOLD"); - if (thresh && atoi (thresh) == 1) + if ((notmuch->transaction_threshold > 0 && + notmuch->transaction_count >= notmuch->transaction_threshold) || + (thresh && atoi (thresh) == 1)) { db->commit (); + notmuch->transaction_count = 0; + } } catch (const Xapian::Error &error) { _notmuch_database_log (notmuch, "A Xapian exception occurred committing transaction: %s.\n", error.get_msg ().c_str ()); @@ -1863,7 +1297,7 @@ _notmuch_database_find_directory_id (notmuch_database_t *notmuch, return NOTMUCH_STATUS_SUCCESS; } - directory = _notmuch_directory_create (notmuch, path, flags, &status); + directory = _notmuch_directory_find_or_create (notmuch, path, flags, &status); if (status || ! directory) { *directory_id = -1; return status; @@ -1942,7 +1376,7 @@ _notmuch_database_relative_path (notmuch_database_t *notmuch, const char *db_path, *relative; unsigned int db_path_len; - db_path = notmuch_database_get_path (notmuch); + db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); db_path_len = strlen (db_path); relative = path; @@ -1973,8 +1407,8 @@ notmuch_database_get_directory (notmuch_database_t *notmuch, *directory = NULL; try { - *directory = _notmuch_directory_create (notmuch, path, - NOTMUCH_FIND_LOOKUP, &status); + *directory = _notmuch_directory_find_or_create (notmuch, path, + NOTMUCH_FIND_LOOKUP, &status); } catch (const Xapian::Error &error) { _notmuch_database_log (notmuch, "A Xapian exception occurred getting directory: %s.\n", error.get_msg ().c_str ()); @@ -2020,9 +1454,11 @@ notmuch_database_remove_message (notmuch_database_t *notmuch, &message); if (status == NOTMUCH_STATUS_SUCCESS && message) { - status = _notmuch_message_remove_filename (message, filename); + if (notmuch_message_count_files (message) > 1) { + status = _notmuch_message_remove_filename (message, filename); + } if (status == NOTMUCH_STATUS_SUCCESS) - _notmuch_message_delete (message); + status = _notmuch_message_delete (message); else if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) _notmuch_message_sync (message); @@ -2073,7 +1509,8 @@ notmuch_database_find_message_by_filename (notmuch_database_t *notmuch, status = NOTMUCH_STATUS_OUT_OF_MEMORY; } } catch (const Xapian::Error &error) { - _notmuch_database_log (notmuch, "Error: A Xapian exception occurred finding message by filename: %s\n", + _notmuch_database_log (notmuch, + "Error: A Xapian exception occurred finding message by filename: %s\n", error.get_msg ().c_str ()); notmuch->exception_reported = true; status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; @@ -2138,3 +1575,15 @@ notmuch_database_status_string (const notmuch_database_t *notmuch) { return notmuch->status_string; } + +bool +_notmuch_database_indexable_as_text (notmuch_database_t *notmuch, const char *mime_string) +{ + for (size_t i = 0; i < notmuch->index_as_text_length; i++) { + if (regexec (¬much->index_as_text[i], mime_string, 0, NULL, 0) == 0) { + return true; + } + } + + return false; +} diff --git a/lib/directory.cc b/lib/directory.cc index af71f402..5cf64d7f 100644 --- a/lib/directory.cc +++ b/lib/directory.cc @@ -49,6 +49,21 @@ struct _notmuch_directory { time_t mtime; }; +#define LOG_XAPIAN_EXCEPTION(directory, error) _log_xapian_exception (__location__, directory, error) + +static void +_log_xapian_exception (const char *where, notmuch_directory_t *dir, const Xapian::Error error) +{ + notmuch_database_t *notmuch = dir->notmuch; + + _notmuch_database_log (notmuch, + "A Xapian exception occurred at %s: %s\n", + where, + error.get_msg ().c_str ()); + notmuch->exception_reported = true; +} + + /* We end up having to call the destructor explicitly because we had * to use "placement new" in order to initialize C++ objects within a * block that we allocated with talloc. So C++ is making talloc @@ -94,12 +109,11 @@ find_directory_document (notmuch_database_t *notmuch, * NOTMUCH_STATUS_SUCCESS and this returns NULL. */ notmuch_directory_t * -_notmuch_directory_create (notmuch_database_t *notmuch, - const char *path, - notmuch_find_flags_t flags, - notmuch_status_t *status_ret) +_notmuch_directory_find_or_create (notmuch_database_t *notmuch, + const char *path, + notmuch_find_flags_t flags, + notmuch_status_t *status_ret) { - Xapian::WritableDatabase *db; notmuch_directory_t *directory; notmuch_private_status_t private_status; const char *db_path; @@ -114,7 +128,7 @@ _notmuch_directory_create (notmuch_database_t *notmuch, path = _notmuch_database_relative_path (notmuch, path); - if (create && notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY) + if (create && _notmuch_database_mode (notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY) INTERNAL_ERROR ("Failure to ensure database is writable"); directory = talloc (notmuch, notmuch_directory_t); @@ -176,10 +190,10 @@ _notmuch_directory_create (notmuch_database_t *notmuch, directory->doc.add_value (NOTMUCH_VALUE_TIMESTAMP, Xapian::sortable_serialise (0)); - db = static_cast (notmuch->xapian_db); - directory->document_id = _notmuch_database_generate_doc_id (notmuch); - db->replace_document (directory->document_id, directory->doc); + directory->notmuch-> + writable_xapian_db + -> replace_document (directory->document_id, directory->doc); talloc_free (local); } @@ -187,7 +201,7 @@ _notmuch_directory_create (notmuch_database_t *notmuch, directory->doc.get_value (NOTMUCH_VALUE_TIMESTAMP)); } catch (const Xapian::Error &error) { _notmuch_database_log (notmuch, - "A Xapian exception occurred creating a directory: %s.\n", + "A Xapian exception occurred finding/creating a directory: %s.\n", error.get_msg ().c_str ()); notmuch->exception_reported = true; notmuch_directory_destroy (directory); @@ -213,20 +227,18 @@ notmuch_directory_set_mtime (notmuch_directory_t *directory, time_t mtime) { notmuch_database_t *notmuch = directory->notmuch; - Xapian::WritableDatabase *db; notmuch_status_t status; status = _notmuch_database_ensure_writable (notmuch); if (status) return status; - db = static_cast (notmuch->xapian_db); - try { directory->doc.add_value (NOTMUCH_VALUE_TIMESTAMP, Xapian::sortable_serialise (mtime)); - db->replace_document (directory->document_id, directory->doc); + directory->notmuch + ->writable_xapian_db->replace_document (directory->document_id, directory->doc); directory->mtime = mtime; @@ -251,15 +263,19 @@ notmuch_filenames_t * notmuch_directory_get_child_files (notmuch_directory_t *directory) { char *term; - notmuch_filenames_t *child_files; + notmuch_filenames_t *child_files = NULL; term = talloc_asprintf (directory, "%s%u:", _find_prefix ("file-direntry"), directory->document_id); - child_files = _create_filenames_for_terms_with_prefix (directory, - directory->notmuch, - term); + try { + child_files = _create_filenames_for_terms_with_prefix (directory, + directory->notmuch, + term); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (directory, error); + } talloc_free (term); @@ -270,14 +286,18 @@ notmuch_filenames_t * notmuch_directory_get_child_directories (notmuch_directory_t *directory) { char *term; - notmuch_filenames_t *child_directories; + notmuch_filenames_t *child_directories = NULL; term = talloc_asprintf (directory, "%s%u:", _find_prefix ("directory-direntry"), directory->document_id); - child_directories = _create_filenames_for_terms_with_prefix (directory, - directory->notmuch, term); + try { + child_directories = _create_filenames_for_terms_with_prefix (directory, + directory->notmuch, term); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (directory, error); + } talloc_free (term); @@ -288,15 +308,14 @@ notmuch_status_t notmuch_directory_delete (notmuch_directory_t *directory) { notmuch_status_t status; - Xapian::WritableDatabase *db; status = _notmuch_database_ensure_writable (directory->notmuch); if (status) return status; try { - db = static_cast (directory->notmuch->xapian_db); - db->delete_document (directory->document_id); + directory->notmuch-> + writable_xapian_db->delete_document (directory->document_id); } catch (const Xapian::Error &error) { _notmuch_database_log (directory->notmuch, "A Xapian exception occurred deleting directory entry: %s.\n", @@ -306,7 +325,7 @@ notmuch_directory_delete (notmuch_directory_t *directory) } notmuch_directory_destroy (directory); - return NOTMUCH_STATUS_SUCCESS; + return status; } void diff --git a/lib/features.cc b/lib/features.cc new file mode 100644 index 00000000..cf0196c8 --- /dev/null +++ b/lib/features.cc @@ -0,0 +1,114 @@ +#include "database-private.h" + +static const struct { + /* NOTMUCH_FEATURE_* value. */ + _notmuch_features value; + /* Feature name as it appears in the database. This name should + * be appropriate for displaying to the user if an older version + * of notmuch doesn't support this feature. */ + const char *name; + /* Compatibility flags when this feature is declared. */ + const char *flags; +} feature_names[] = { + { NOTMUCH_FEATURE_FILE_TERMS, + "multiple paths per message", "rw" }, + { NOTMUCH_FEATURE_DIRECTORY_DOCS, + "relative directory paths", "rw" }, + /* Header values are not required for reading a database because a + * reader can just refer to the message file. */ + { NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES, + "from/subject/message-ID in database", "w" }, + { NOTMUCH_FEATURE_BOOL_FOLDER, + "exact folder:/path: search", "rw" }, + { NOTMUCH_FEATURE_GHOSTS, + "mail documents for missing messages", "w" }, + /* Knowledge of the index mime-types are not required for reading + * a database because a reader will just be unable to query + * them. */ + { NOTMUCH_FEATURE_INDEXED_MIMETYPES, + "indexed MIME types", "w" }, + { NOTMUCH_FEATURE_LAST_MOD, + "modification tracking", "w" }, + /* Existing databases will work fine for all queries not involving + * 'body:' */ + { NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY, + "index body and headers separately", "w" }, +}; + +char * +_notmuch_database_print_features (const void *ctx, unsigned int features) +{ + unsigned int i; + char *res = talloc_strdup (ctx, ""); + + for (i = 0; i < ARRAY_SIZE (feature_names); ++i) + if (features & feature_names[i].value) + res = talloc_asprintf_append_buffer ( + res, "%s\t%s\n", feature_names[i].name, feature_names[i].flags); + + return res; +} + + +/* Parse a database features string from the given database version. + * Returns the feature bit set. + * + * For version < 3, this ignores the features string and returns a + * hard-coded set of features. + * + * If there are unrecognized features that are required to open the + * database in mode (which should be 'r' or 'w'), return a + * comma-separated list of unrecognized but required features in + * *incompat_out suitable for presenting to the user. *incompat_out + * will be allocated from ctx. + */ +_notmuch_features +_notmuch_database_parse_features (const void *ctx, const char *features, unsigned int version, + char mode, char **incompat_out) +{ + _notmuch_features res = static_cast<_notmuch_features>(0); + unsigned int namelen, i; + size_t llen = 0; + const char *flags; + + /* Prior to database version 3, features were implied by the + * version number. */ + if (version == 0) + return NOTMUCH_FEATURES_V0; + else if (version == 1) + return NOTMUCH_FEATURES_V1; + else if (version == 2) + return NOTMUCH_FEATURES_V2; + + /* Parse the features string */ + while ((features = strtok_len_c (features + llen, "\n", &llen)) != NULL) { + flags = strchr (features, '\t'); + if (! flags || flags > features + llen) + continue; + namelen = flags - features; + + for (i = 0; i < ARRAY_SIZE (feature_names); ++i) { + if (strlen (feature_names[i].name) == namelen && + strncmp (feature_names[i].name, features, namelen) == 0) { + res |= feature_names[i].value; + break; + } + } + + if (i == ARRAY_SIZE (feature_names) && incompat_out) { + /* Unrecognized feature */ + const char *have = strchr (flags, mode); + if (have && have < features + llen) { + /* This feature is required to access this database in + * 'mode', but we don't understand it. */ + if (! *incompat_out) + *incompat_out = talloc_strdup (ctx, ""); + *incompat_out = talloc_asprintf_append_buffer ( + *incompat_out, "%s%.*s", **incompat_out ? ", " : "", + namelen, features); + } + } + } + + return res; +} diff --git a/lib/index.cc b/lib/index.cc index 158ba5cf..629dcb22 100644 --- a/lib/index.cc +++ b/lib/index.cc @@ -148,8 +148,6 @@ notmuch_filter_discard_non_term_class_init (NotmuchFilterDiscardNonTermClass *kl GObjectClass *object_class = G_OBJECT_CLASS (klass); GMimeFilterClass *filter_class = GMIME_FILTER_CLASS (klass); - parent_class = (GMimeFilterClass *) g_type_class_ref (GMIME_TYPE_FILTER); - object_class->finalize = notmuch_filter_discard_non_term_finalize; filter_class->copy = filter_copy; @@ -240,29 +238,33 @@ filter_reset (GMimeFilter *gmime_filter) * * Returns: a new #NotmuchFilterDiscardNonTerm filter. **/ +static GType type = 0; + +static const GTypeInfo info = { + .class_size = sizeof (NotmuchFilterDiscardNonTermClass), + .base_init = NULL, + .base_finalize = NULL, + .class_init = (GClassInitFunc) notmuch_filter_discard_non_term_class_init, + .class_finalize = NULL, + .class_data = NULL, + .instance_size = sizeof (NotmuchFilterDiscardNonTerm), + .n_preallocs = 0, + .instance_init = NULL, + .value_table = NULL, +}; + +void +_notmuch_filter_init () { + type = g_type_register_static (GMIME_TYPE_FILTER, "NotmuchFilterDiscardNonTerm", &info, + (GTypeFlags) 0); + parent_class = (GMimeFilterClass *) g_type_class_ref (GMIME_TYPE_FILTER); +} + static GMimeFilter * notmuch_filter_discard_non_term_new (GMimeContentType *content_type) { - static GType type = 0; NotmuchFilterDiscardNonTerm *filter; - if (! type) { - static const GTypeInfo info = { - .class_size = sizeof (NotmuchFilterDiscardNonTermClass), - .base_init = NULL, - .base_finalize = NULL, - .class_init = (GClassInitFunc) notmuch_filter_discard_non_term_class_init, - .class_finalize = NULL, - .class_data = NULL, - .instance_size = sizeof (NotmuchFilterDiscardNonTerm), - .n_preallocs = 0, - .instance_init = NULL, - .value_table = NULL, - }; - - type = g_type_register_static (GMIME_TYPE_FILTER, "NotmuchFilterDiscardNonTerm", &info, (GTypeFlags) 0); - } - filter = (NotmuchFilterDiscardNonTerm *) g_object_new (type, NULL); filter->content_type = content_type; filter->state = 0; @@ -369,9 +371,32 @@ _index_content_type (notmuch_message_t *message, GMimeObject *part) static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *part, + GMimeObject *part, _notmuch_message_crypto_t *msg_crypto); +static void +_index_pkcs7_part (notmuch_message_t *message, + notmuch_indexopts_t *indexopts, + GMimeObject *part, + _notmuch_message_crypto_t *msg_crypto); + +static bool +_indexable_as_text (notmuch_message_t *message, GMimeObject *part) +{ + GMimeContentType *content_type = g_mime_object_get_content_type (part); + notmuch_database_t *notmuch = notmuch_message_get_database (message); + + if (content_type) { + char *mime_string = g_mime_content_type_get_mime_type (content_type); + if (mime_string) { + bool ret = _notmuch_database_indexable_as_text (notmuch, mime_string); + g_free (mime_string); + return ret; + } + } + return false; +} + /* Callback to generate terms for each mime part of a message. */ static void _index_mime_part (notmuch_message_t *message, @@ -433,7 +458,7 @@ _index_mime_part (notmuch_message_t *message, g_mime_multipart_get_part (multipart, i)); if (i == GMIME_MULTIPART_ENCRYPTED_CONTENT) { _index_encrypted_mime_part (message, indexopts, - GMIME_MULTIPART_ENCRYPTED (part), + part, msg_crypto); } else { if (i != GMIME_MULTIPART_ENCRYPTED_VERSION) { @@ -449,7 +474,8 @@ _index_mime_part (notmuch_message_t *message, msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) { toindex = _notmuch_repair_crypto_payload_skip_legacy_display (child); if (toindex != child) - notmuch_message_add_property (message, "index.repaired", "skip-protected-headers-legacy-display"); + notmuch_message_add_property (message, "index.repaired", + "skip-protected-headers-legacy-display"); } _index_mime_part (message, indexopts, toindex, msg_crypto); } @@ -461,11 +487,17 @@ _index_mime_part (notmuch_message_t *message, mime_message = g_mime_message_part_get_message (GMIME_MESSAGE_PART (part)); - _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), msg_crypto); + _index_mime_part (message, indexopts, g_mime_message_get_mime_part (mime_message), + msg_crypto); goto DONE; } + if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) { + _index_pkcs7_part (message, indexopts, part, msg_crypto); + goto DONE; + } + if (! (GMIME_IS_PART (part))) { _notmuch_database_log (notmuch_message_get_database (message), "Warning: Not indexing unknown mime part: %s.\n", @@ -482,9 +514,11 @@ _index_mime_part (notmuch_message_t *message, _notmuch_message_add_term (message, "tag", "attachment"); _notmuch_message_gen_terms (message, "attachment", filename); - /* XXX: Would be nice to call out to something here to parse - * the attachment into text and then index that. */ - goto DONE; + if (! _indexable_as_text (message, part)) { + /* XXX: Would be nice to call out to something here to parse + * the attachment into text and then index that. */ + goto DONE; + } } byte_array = g_byte_array_new (); @@ -540,7 +574,7 @@ _index_mime_part (notmuch_message_t *message, static void _index_encrypted_mime_part (notmuch_message_t *message, notmuch_indexopts_t *indexopts, - GMimeMultipartEncrypted *encrypted_data, + GMimeObject *encrypted_data, _notmuch_message_crypto_t *msg_crypto) { notmuch_status_t status; @@ -556,6 +590,7 @@ _index_encrypted_mime_part (notmuch_message_t *message, bool attempted = false; GMimeDecryptResult *decrypt_result = NULL; bool get_sk = (notmuch_indexopts_get_decrypt_policy (indexopts) == NOTMUCH_DECRYPT_TRUE); + clear = _notmuch_crypto_decrypt (&attempted, notmuch_indexopts_get_decrypt_policy (indexopts), message, encrypted_data, get_sk ? &decrypt_result : NULL, &err); if (! attempted) @@ -584,7 +619,8 @@ _index_encrypted_mime_part (notmuch_message_t *message, notmuch_status_to_string (status)); if (get_sk) { status = notmuch_message_add_property (message, "session-key", - g_mime_decrypt_result_get_session_key (decrypt_result)); + g_mime_decrypt_result_get_session_key ( + decrypt_result)); if (status) _notmuch_database_log (notmuch, "failed to add session-key " "property (%d)\n", status); @@ -592,11 +628,14 @@ _index_encrypted_mime_part (notmuch_message_t *message, g_object_unref (decrypt_result); } GMimeObject *toindex = clear; - if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, GMIME_OBJECT (encrypted_data), GMIME_MULTIPART_ENCRYPTED_CONTENT) && + + if (_notmuch_message_crypto_potential_payload (msg_crypto, clear, encrypted_data, + GMIME_MULTIPART_ENCRYPTED_CONTENT) && msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) { toindex = _notmuch_repair_crypto_payload_skip_legacy_display (clear); if (toindex != clear) - notmuch_message_add_property (message, "index.repaired", "skip-protected-headers-legacy-display"); + notmuch_message_add_property (message, "index.repaired", + "skip-protected-headers-legacy-display"); } _index_mime_part (message, indexopts, toindex, msg_crypto); g_object_unref (clear); @@ -608,6 +647,59 @@ _index_encrypted_mime_part (notmuch_message_t *message, } +static void +_index_pkcs7_part (notmuch_message_t *message, + notmuch_indexopts_t *indexopts, + GMimeObject *part, + _notmuch_message_crypto_t *msg_crypto) +{ + GMimeApplicationPkcs7Mime *pkcs7; + GMimeSecureMimeType p7type; + GMimeObject *mimeobj = NULL; + GMimeSignatureList *sigs = NULL; + GError *err = NULL; + notmuch_database_t *notmuch = NULL; + + pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part); + p7type = g_mime_application_pkcs7_mime_get_smime_type (pkcs7); + notmuch = notmuch_message_get_database (message); + _index_content_type (message, part); + + if (p7type == GMIME_SECURE_MIME_TYPE_SIGNED_DATA) { + sigs = g_mime_application_pkcs7_mime_verify (pkcs7, GMIME_VERIFY_NONE, &mimeobj, &err); + if (sigs == NULL) { + _notmuch_database_log (notmuch, + "Failed to verify PKCS#7 SignedData during indexing. (%d:%d) [%s]\n", + err->domain, err->code, err->message); + g_error_free (err); + goto DONE; + } + _notmuch_message_add_term (message, "tag", "signed"); + GMimeObject *toindex = mimeobj; + if (_notmuch_message_crypto_potential_payload (msg_crypto, mimeobj, part, 0) && + msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) { + toindex = _notmuch_repair_crypto_payload_skip_legacy_display (mimeobj); + if (toindex != mimeobj) + notmuch_message_add_property (message, "index.repaired", + "skip-protected-headers-legacy-display"); + } + _index_mime_part (message, indexopts, toindex, msg_crypto); + } else if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) { + _notmuch_message_add_term (message, "tag", "encrypted"); + _index_encrypted_mime_part (message, indexopts, + part, + msg_crypto); + } else { + _notmuch_database_log (notmuch, "Cannot currently handle PKCS#7 smime-type '%s'\n", + g_mime_object_get_content_type_parameter (part, "smime-type")); + } + DONE: + if (mimeobj) + g_object_unref (mimeobj); + if (sigs) + g_object_unref (sigs); +} + static notmuch_status_t _notmuch_message_index_user_headers (notmuch_message_t *message, GMimeMessage *mime_message) { diff --git a/lib/indexopts.c b/lib/indexopts.c index 1e8d180e..2ffd1942 100644 --- a/lib/indexopts.c +++ b/lib/indexopts.c @@ -20,6 +20,10 @@ #include "notmuch-private.h" +struct _notmuch_indexopts { + _notmuch_crypto_t crypto; +}; + notmuch_indexopts_t * notmuch_database_get_default_indexopts (notmuch_database_t *db) { @@ -31,8 +35,9 @@ notmuch_database_get_default_indexopts (notmuch_database_t *db) char *decrypt_policy; notmuch_status_t err = notmuch_database_get_config (db, "index.decrypt", &decrypt_policy); + if (err) - return ret; + return NULL; if (decrypt_policy) { if ((! (strcasecmp (decrypt_policy, "true"))) || diff --git a/lib/init.cc b/lib/init.cc new file mode 100644 index 00000000..cf29200f --- /dev/null +++ b/lib/init.cc @@ -0,0 +1,21 @@ +#include "notmuch-private.h" + +#include + +static void do_init () +{ + /* Initialize the GLib type system and threads */ +#if ! GLIB_CHECK_VERSION (2, 35, 1) + g_type_init (); +#endif + + g_mime_init (); + _notmuch_filter_init (); +} + +void +_notmuch_init () +{ + static std::once_flag initialized; + std::call_once (initialized, do_init); +} diff --git a/lib/lastmod-fp.cc b/lib/lastmod-fp.cc new file mode 100644 index 00000000..f85efd28 --- /dev/null +++ b/lib/lastmod-fp.cc @@ -0,0 +1,83 @@ +/* lastmod-fp.cc - lastmod range query glue + * + * This file is part of notmuch. + * + * Copyright © 2022 David Bremner + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see https://www.gnu.org/licenses/ . + * + * Author: David Bremner + */ + +#include "database-private.h" +#include "lastmod-fp.h" + +notmuch_status_t +_notmuch_lastmod_strings_to_query (notmuch_database_t *notmuch, + const std::string &from, const std::string &to, + Xapian::Query &output, std::string &msg) +{ + long from_idx = 0L, to_idx = LONG_MAX; + long current; + std::string str; + + /* revision should not change, but for the avoidance of doubt, + * grab for both ends of range, if needed*/ + current = notmuch_database_get_revision (notmuch, NULL); + + try { + if (from.empty ()) + from_idx = 0L; + else + from_idx = std::stol (from); + } catch (std::logic_error &e) { + msg = "bad 'from' revision: '" + from + "'"; + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (from_idx < 0) + from_idx += current; + + try { + if (EMPTY_STRING (to)) + to_idx = LONG_MAX; + else + to_idx = std::stol (to); + } catch (std::logic_error &e) { + msg = "bad 'to' revision: '" + to + "'"; + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (to_idx < 0) + to_idx += current; + + output = Xapian::Query (Xapian::Query::OP_VALUE_RANGE, NOTMUCH_VALUE_LAST_MOD, + Xapian::sortable_serialise (from_idx), + Xapian::sortable_serialise (to_idx)); + return NOTMUCH_STATUS_SUCCESS; +} + +Xapian::Query +LastModRangeProcessor::operator() (const std::string &begin, const std::string &end) +{ + + Xapian::Query output; + std::string msg; + + if (_notmuch_lastmod_strings_to_query (notmuch, begin, end, output, msg)) + throw Xapian::QueryParserError (msg); + + return output; +} + diff --git a/lib/lastmod-fp.h b/lib/lastmod-fp.h new file mode 100644 index 00000000..8168fe7b --- /dev/null +++ b/lib/lastmod-fp.h @@ -0,0 +1,41 @@ +/* lastmod-fp.h - database revision query glue + * + * This file is part of notmuch. + * + * Copyright © 2022 David Bremner + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see https://www.gnu.org/licenses/ . + * + * Author: David Bremner + */ + +#ifndef NOTMUCH_LASTMOD_FP_H +#define NOTMUCH_LASTMOD_FP_H + +#include + +class LastModRangeProcessor : public Xapian::RangeProcessor { +protected: + notmuch_database_t *notmuch; + +public: + LastModRangeProcessor (notmuch_database_t *notmuch_, const std::string prefix_) + : Xapian::RangeProcessor (NOTMUCH_VALUE_LAST_MOD, prefix_, 0), notmuch (notmuch_) + { + } + + Xapian::Query operator() (const std::string &begin, const std::string &end); +}; + +#endif /* NOTMUCH_LASTMOD_FP_H */ diff --git a/lib/message-file.c b/lib/message-file.c index e1db26fb..68f646a4 100644 --- a/lib/message-file.c +++ b/lib/message-file.c @@ -64,21 +64,38 @@ _notmuch_message_file_open_ctx (notmuch_database_t *notmuch, if (unlikely (message == NULL)) return NULL; - message->filename = talloc_strdup (message, filename); + const char *prefix = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); + + if (prefix == NULL) + goto FAIL; + + if (*filename == '/') { + if (strncmp (filename, prefix, strlen (prefix)) != 0) { + _notmuch_database_log (notmuch, "Error opening %s: path outside mail root\n", + filename); + errno = 0; + goto FAIL; + } + message->filename = talloc_strdup (message, filename); + } else { + message->filename = talloc_asprintf (message, "%s/%s", prefix, filename); + } + if (message->filename == NULL) goto FAIL; talloc_set_destructor (message, _notmuch_message_file_destructor); - message->stream = g_mime_stream_gzfile_open (filename); + message->stream = g_mime_stream_gzfile_open (message->filename); if (message->stream == NULL) goto FAIL; return message; FAIL: - _notmuch_database_log (notmuch, "Error opening %s: %s\n", - filename, strerror (errno)); + if (errno) + _notmuch_database_log (notmuch, "Error opening %s: %s\n", + filename, strerror (errno)); _notmuch_message_file_close (message); return NULL; @@ -124,7 +141,6 @@ _notmuch_message_file_parse (notmuch_message_file_t *message) { GMimeParser *parser; notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; - static int initialized = 0; bool is_mbox; if (message->message) @@ -132,10 +148,7 @@ _notmuch_message_file_parse (notmuch_message_file_t *message) is_mbox = _is_mbox (message->stream); - if (! initialized) { - g_mime_init (); - initialized = 1; - } + _notmuch_init (); message->headers = g_hash_table_new_full (strcase_hash, strcase_equal, free, g_free); @@ -278,11 +291,16 @@ _notmuch_message_file_get_header (notmuch_message_file_t *message, if (value) return value; - if (strcasecmp (header, "received") == 0) { + if (strcasecmp (header, "received") == 0 || + strcasecmp (header, "delivered-to") == 0) { /* - * The Received: header is special. We concatenate all - * instances of the header as we use this when analyzing the - * path the mail has taken from sender to recipient. + * The Received: header is special. We concatenate all instances of the + * header as we use this when analyzing the path the mail has taken + * from sender to recipient. + * + * Similarly, multiple instances of Delivered-To may be present. We + * concatenate them so the one with highest priority may be picked (eg. + * primary_email before other_email). */ decoded = _notmuch_message_file_get_combined_header (message, header); } else { diff --git a/lib/message-property.cc b/lib/message-property.cc index ecf7e140..7f520340 100644 --- a/lib/message-property.cc +++ b/lib/message-property.cc @@ -25,6 +25,20 @@ #include "database-private.h" #include "message-private.h" +#define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception (__location__, message, error) + +static void +_log_xapian_exception (const char *where, notmuch_message_t *message, const Xapian::Error error) +{ + notmuch_database_t *notmuch = notmuch_message_get_database (message); + + _notmuch_database_log (notmuch, + "A Xapian exception occurred at %s: %s\n", + where, + error.get_msg ().c_str ()); + notmuch->exception_reported = true; +} + notmuch_status_t notmuch_message_get_property (notmuch_message_t *message, const char *key, const char **value) { @@ -43,11 +57,13 @@ notmuch_message_count_properties (notmuch_message_t *message, const char *key, u return NOTMUCH_STATUS_NULL_POINTER; notmuch_string_map_t *map; + map = _notmuch_message_property_map (message); if (! map) return NOTMUCH_STATUS_NULL_POINTER; notmuch_string_map_iterator_t *matcher = _notmuch_string_map_iterator_create (map, key, true); + if (! matcher) return NOTMUCH_STATUS_OUT_OF_MEMORY; @@ -81,10 +97,15 @@ _notmuch_message_modify_property (notmuch_message_t *message, const char *key, c term = talloc_asprintf (message, "%s=%s", key, value); - if (delete_it) - private_status = _notmuch_message_remove_term (message, "property", term); - else - private_status = _notmuch_message_add_term (message, "property", term); + try { + if (delete_it) + private_status = _notmuch_message_remove_term (message, "property", term); + else + private_status = _notmuch_message_add_term (message, "property", term); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } if (private_status) return COERCE_STATUS (private_status, @@ -121,15 +142,22 @@ _notmuch_message_remove_all_properties (notmuch_message_t *message, const char * if (status) return status; - _notmuch_message_invalidate_metadata (message, "property"); if (key) term_prefix = talloc_asprintf (message, "%s%s%s", _find_prefix ("property"), key, prefix ? "" : "="); else term_prefix = _find_prefix ("property"); - /* XXX better error reporting ? */ - _notmuch_message_remove_terms (message, term_prefix); + try { + /* XXX better error reporting ? */ + _notmuch_message_remove_terms (message, term_prefix); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + + if (! _notmuch_message_frozen (message)) + _notmuch_message_sync (message); return NOTMUCH_STATUS_SUCCESS; } diff --git a/lib/message.cc b/lib/message.cc index 5c9b58b2..46638f80 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -68,7 +68,7 @@ struct maildir_flag_tag { }; /* ASCII ordered table of Maildir flags and associated tags */ -static struct maildir_flag_tag flag2tag[] = { +static const struct maildir_flag_tag flag2tag[] = { { 'D', "draft", false }, { 'F', "flagged", false }, { 'P', "passed", false }, @@ -90,6 +90,20 @@ _notmuch_message_destructor (notmuch_message_t *message) return 0; } +#define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception (__location__, message, error) + +static void +_log_xapian_exception (const char *where, notmuch_message_t *message, const Xapian::Error error) +{ + notmuch_database_t *notmuch = notmuch_message_get_database (message); + + _notmuch_database_log (notmuch, + "A Xapian exception occurred at %s: %s\n", + where, + error.get_msg ().c_str ()); + notmuch->exception_reported = true; +} + static notmuch_message_t * _notmuch_message_create_for_document (const void *talloc_owner, notmuch_database_t *notmuch, @@ -155,6 +169,7 @@ _notmuch_message_create_for_document (const void *talloc_owner, message->doc = doc; message->termpos = 0; + message->modified = false; return message; } @@ -263,7 +278,7 @@ _notmuch_message_create_for_message_id (notmuch_database_t *notmuch, return NULL; } - if (notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY) + if (_notmuch_database_mode (notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY) INTERNAL_ERROR ("Failure to ensure database is writable."); try { @@ -274,7 +289,8 @@ _notmuch_message_create_for_message_id (notmuch_database_t *notmuch, doc_id = _notmuch_database_generate_doc_id (notmuch); } catch (const Xapian::Error &error) { - _notmuch_database_log (notmuch_message_get_database (message), "A Xapian exception occurred creating message: %s\n", + _notmuch_database_log (notmuch, + "A Xapian exception occurred creating message: %s\n", error.get_msg ().c_str ()); notmuch->exception_reported = true; *status_ret = NOTMUCH_PRIVATE_STATUS_XAPIAN_EXCEPTION; @@ -306,6 +322,7 @@ _notmuch_message_get_term (notmuch_message_t *message, return NULL; const std::string &term = *i; + if (strncmp (term.c_str (), prefix, prefix_len)) return NULL; @@ -324,23 +341,6 @@ _notmuch_message_get_term (notmuch_message_t *message, return value; } -/* - * For special applications where we only want the thread id, reading - * in all metadata is a heavy I/O penalty. - */ -const char * -_notmuch_message_get_thread_id_only (notmuch_message_t *message) -{ - - Xapian::TermIterator i = message->doc.termlist_begin (); - Xapian::TermIterator end = message->doc.termlist_end (); - - message->thread_id = _notmuch_message_get_term (message, i, end, - _find_prefix ("thread")); - return message->thread_id; -} - - static void _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field) { @@ -443,13 +443,11 @@ _notmuch_message_ensure_metadata (notmuch_message_t *message, void *field) /* all the way without an exception */ break; } catch (const Xapian::DatabaseModifiedError &error) { - notmuch_status_t status = _notmuch_database_reopen (message->notmuch); + notmuch_status_t status = notmuch_database_reopen (message->notmuch, + NOTMUCH_DATABASE_MODE_READ_ONLY); if (status != NOTMUCH_STATUS_SUCCESS) INTERNAL_ERROR ("unhandled error from notmuch_database_reopen: %s\n", notmuch_status_to_string (status)); - } catch (const Xapian::Error &error) { - INTERNAL_ERROR ("A Xapian exception occurred fetching message metadata: %s\n", - error.get_msg ().c_str ()); } } message->last_view = message->notmuch->view; @@ -507,7 +505,13 @@ _notmuch_message_get_doc_id (notmuch_message_t *message) const char * notmuch_message_get_message_id (notmuch_message_t *message) { - _notmuch_message_ensure_metadata (message, message->message_id); + try { + _notmuch_message_ensure_metadata (message, message->message_id); + } catch (const Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NULL; + } + if (! message->message_id) INTERNAL_ERROR ("Message with document ID of %u has no message ID.\n", message->doc_id); @@ -557,9 +561,7 @@ notmuch_message_get_header (notmuch_message_t *message, const char *header) return talloc_strdup (message, value.c_str ()); } catch (Xapian::Error &error) { - _notmuch_database_log (notmuch_message_get_database (message), "A Xapian exception occurred when reading header: %s\n", - error.get_msg ().c_str ()); - message->notmuch->exception_reported = true; + LOG_XAPIAN_EXCEPTION (message, error); return NULL; } } @@ -589,7 +591,12 @@ _notmuch_message_get_in_reply_to (notmuch_message_t *message) const char * notmuch_message_get_thread_id (notmuch_message_t *message) { - _notmuch_message_ensure_metadata (message, message->thread_id); + try { + _notmuch_message_ensure_metadata (message, message->thread_id); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NULL; + } if (! message->thread_id) INTERNAL_ERROR ("Message with document ID of %u has no thread ID.\n", message->doc_id); @@ -712,6 +719,8 @@ _notmuch_message_remove_terms (notmuch_message_t *message, const char *prefix) /* Ignore failure to remove non-existent term. */ } } + + _notmuch_message_invalidate_metadata (message, "property"); } @@ -719,7 +728,7 @@ _notmuch_message_remove_terms (notmuch_message_t *message, const char *prefix) * properties, along with any automatic tags*/ /* According to Xapian API docs, none of these calls throw * exceptions */ -notmuch_private_status_t +static notmuch_private_status_t _notmuch_message_remove_indexed_terms (notmuch_message_t *message) { Xapian::TermIterator i; @@ -751,9 +760,9 @@ _notmuch_message_remove_indexed_terms (notmuch_message_t *message) const char *tag = notmuch_tags_get (tags); - if (STRNCMP_LITERAL (tag, "encrypted") != 0 && - STRNCMP_LITERAL (tag, "signed") != 0 && - STRNCMP_LITERAL (tag, "attachment") != 0) { + if (strcmp (tag, "encrypted") != 0 && + strcmp (tag, "signed") != 0 && + strcmp (tag, "attachment") != 0) { std::string term = tag_prefix + tag; message->doc.add_term (term); } @@ -785,7 +794,7 @@ is_maildir (const char *p) } /* Add "folder:" term for directory. */ -static notmuch_status_t +NODISCARD static notmuch_status_t _notmuch_message_add_folder_terms (notmuch_message_t *message, const char *directory) { @@ -821,7 +830,10 @@ _notmuch_message_add_folder_terms (notmuch_message_t *message, *folder = '\0'; } - _notmuch_message_add_term (message, "folder", folder); + if (notmuch_status_t status = COERCE_STATUS (_notmuch_message_add_term (message, "folder", + folder), + "adding folder term")) + return status; talloc_free (folder); @@ -832,12 +844,17 @@ _notmuch_message_add_folder_terms (notmuch_message_t *message, #define RECURSIVE_SUFFIX "/**" /* Add "path:" terms for directory. */ -static notmuch_status_t +NODISCARD static notmuch_status_t _notmuch_message_add_path_terms (notmuch_message_t *message, const char *directory) { + notmuch_status_t status; + /* Add exact "path:" term. */ - _notmuch_message_add_term (message, "path", directory); + status = COERCE_STATUS (_notmuch_message_add_term (message, "path", directory), + "adding path term"); + if (status) + return status; if (strlen (directory)) { char *path, *p; @@ -850,7 +867,10 @@ _notmuch_message_add_path_terms (notmuch_message_t *message, for (p = path + strlen (path) - 1; p > path; p--) { if (*p == '/') { strcpy (p, RECURSIVE_SUFFIX); - _notmuch_message_add_term (message, "path", path); + status = COERCE_STATUS (_notmuch_message_add_term (message, "path", path), + "adding path term"); + if (status) + return status; } } @@ -858,7 +878,10 @@ _notmuch_message_add_path_terms (notmuch_message_t *message, } /* Recursive all-matching path:** for consistency. */ - _notmuch_message_add_term (message, "path", "**"); + status = COERCE_STATUS (_notmuch_message_add_term (message, "path", "**"), + "adding path term"); + if (status) + return status; return NOTMUCH_STATUS_SUCCESS; } @@ -877,6 +900,7 @@ _notmuch_message_add_directory_terms (void *ctx, notmuch_message_t *message) const char *direntry, *directory; char *colon; const std::string &term = *i; + notmuch_status_t term_status; /* Terminate loop at first term without desired prefix. */ if (strncmp (term.c_str (), direntry_prefix, direntry_prefix_len)) @@ -897,8 +921,13 @@ _notmuch_message_add_directory_terms (void *ctx, notmuch_message_t *message) message->notmuch, directory_id); - _notmuch_message_add_folder_terms (message, directory); - _notmuch_message_add_path_terms (message, directory); + term_status = _notmuch_message_add_folder_terms (message, directory); + if (term_status) + return term_status; + + term_status = _notmuch_message_add_path_terms (message, directory); + if (term_status) + return term_status; } return status; @@ -914,6 +943,7 @@ _notmuch_message_add_filename (notmuch_message_t *message, { const char *relative, *directory; notmuch_status_t status; + notmuch_private_status_t private_status; void *local = talloc_new (message); char *direntry; @@ -937,10 +967,25 @@ _notmuch_message_add_filename (notmuch_message_t *message, /* New file-direntry allows navigating to this message with * notmuch_directory_get_child_files() . */ - _notmuch_message_add_term (message, "file-direntry", direntry); + private_status = _notmuch_message_add_term (message, "file-direntry", direntry); + switch (private_status) { + case NOTMUCH_PRIVATE_STATUS_SUCCESS: + break; + case NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG: + _notmuch_database_log (message->notmuch, "filename too long for file-direntry term: %s\n", + filename); + return NOTMUCH_STATUS_PATH_ERROR; + default: + return COERCE_STATUS (private_status, "adding file-direntry term"); + } - _notmuch_message_add_folder_terms (message, directory); - _notmuch_message_add_path_terms (message, directory); + status = _notmuch_message_add_folder_terms (message, directory); + if (status) + return status; + + status = _notmuch_message_add_path_terms (message, directory); + if (status) + return status; talloc_free (local); @@ -1079,7 +1124,7 @@ _notmuch_message_ensure_filename_list (notmuch_message_t *message) *colon = '\0'; - db_path = notmuch_database_get_path (message->notmuch); + db_path = notmuch_config_get (message->notmuch, NOTMUCH_CONFIG_MAIL_ROOT); directory = _notmuch_database_get_directory_path (local, message->notmuch, @@ -1104,7 +1149,12 @@ _notmuch_message_ensure_filename_list (notmuch_message_t *message) const char * notmuch_message_get_filename (notmuch_message_t *message) { - _notmuch_message_ensure_filename_list (message); + try { + _notmuch_message_ensure_filename_list (message); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NULL; + } if (message->filename_list == NULL) return NULL; @@ -1120,7 +1170,12 @@ notmuch_message_get_filename (notmuch_message_t *message) notmuch_filenames_t * notmuch_message_get_filenames (notmuch_message_t *message) { - _notmuch_message_ensure_filename_list (message); + try { + _notmuch_message_ensure_filename_list (message); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NULL; + } return _notmuch_filenames_create (message, message->filename_list); } @@ -1128,20 +1183,50 @@ notmuch_message_get_filenames (notmuch_message_t *message) int notmuch_message_count_files (notmuch_message_t *message) { - _notmuch_message_ensure_filename_list (message); + try { + _notmuch_message_ensure_filename_list (message); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return -1; + } return _notmuch_string_list_length (message->filename_list); } +notmuch_status_t +notmuch_message_get_flag_st (notmuch_message_t *message, + notmuch_message_flag_t flag, + notmuch_bool_t *is_set) +{ + if (! is_set) + return NOTMUCH_STATUS_NULL_POINTER; + + try { + if (flag == NOTMUCH_MESSAGE_FLAG_GHOST && + ! NOTMUCH_TEST_BIT (message->lazy_flags, flag)) + _notmuch_message_ensure_metadata (message, NULL); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + + *is_set = NOTMUCH_TEST_BIT (message->flags, flag); + return NOTMUCH_STATUS_SUCCESS; +} + notmuch_bool_t notmuch_message_get_flag (notmuch_message_t *message, notmuch_message_flag_t flag) { - if (flag == NOTMUCH_MESSAGE_FLAG_GHOST && - ! NOTMUCH_TEST_BIT (message->lazy_flags, flag)) - _notmuch_message_ensure_metadata (message, NULL); + notmuch_bool_t is_set; + notmuch_status_t status; - return NOTMUCH_TEST_BIT (message->flags, flag); + status = notmuch_message_get_flag_st (message, flag, &is_set); + + if (status) + return FALSE; + else + return is_set; } void @@ -1163,9 +1248,7 @@ notmuch_message_get_date (notmuch_message_t *message) try { value = message->doc.get_value (NOTMUCH_VALUE_TIMESTAMP); } catch (Xapian::Error &error) { - _notmuch_database_log (notmuch_message_get_database (message), "A Xapian exception occurred when reading date: %s\n", - error.get_msg ().c_str ()); - message->notmuch->exception_reported = true; + LOG_XAPIAN_EXCEPTION (message, error); return 0; } @@ -1180,7 +1263,12 @@ notmuch_message_get_tags (notmuch_message_t *message) { notmuch_tags_t *tags; - _notmuch_message_ensure_metadata (message, message->tag_list); + try { + _notmuch_message_ensure_metadata (message, message->tag_list); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NULL; + } tags = _notmuch_tags_create (message, message->tag_list); /* _notmuch_tags_create steals the reference to the tag_list, but @@ -1261,9 +1349,7 @@ _notmuch_message_upgrade_last_mod (notmuch_message_t *message) void _notmuch_message_sync (notmuch_message_t *message) { - Xapian::WritableDatabase *db; - - if (message->notmuch->mode == NOTMUCH_DATABASE_MODE_READ_ONLY) + if (_notmuch_database_mode (message->notmuch) == NOTMUCH_DATABASE_MODE_READ_ONLY) return; if (! message->modified) @@ -1281,8 +1367,8 @@ _notmuch_message_sync (notmuch_message_t *message) _notmuch_database_new_revision ( message->notmuch))); - db = static_cast (message->notmuch->xapian_db); - db->replace_document (message->doc_id, message->doc); + message->notmuch->writable_xapian_db-> + replace_document (message->doc_id, message->doc); message->modified = false; } @@ -1292,12 +1378,10 @@ notmuch_status_t _notmuch_message_delete (notmuch_message_t *message) { notmuch_status_t status; - Xapian::WritableDatabase *db; - const char *mid, *tid, *query_string; + const char *mid, *tid; notmuch_message_t *ghost; notmuch_private_status_t private_status; notmuch_database_t *notmuch; - notmuch_query_t *query; unsigned int count = 0; bool is_ghost; @@ -1309,27 +1393,44 @@ _notmuch_message_delete (notmuch_message_t *message) if (status) return status; - db = static_cast (notmuch->xapian_db); - db->delete_document (message->doc_id); - - /* if this was a ghost to begin with, we are done */ - private_status = _notmuch_message_has_term (message, "type", "ghost", &is_ghost); - if (private_status) - return COERCE_STATUS (private_status, - "Error trying to determine whether message was a ghost"); - if (is_ghost) - return NOTMUCH_STATUS_SUCCESS; - - query_string = talloc_asprintf (message, "thread:%s", tid); - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - status = notmuch_query_count_messages (query, &count); - if (status) { - notmuch_query_destroy (query); - return status; + try { + Xapian::PostingIterator thread_doc, thread_doc_end; + Xapian::PostingIterator mail_doc, mail_doc_end; + + /* look for a non-ghost message in the same thread */ + /* if this was a ghost to begin with, we are done */ + private_status = _notmuch_message_has_term (message, "type", "ghost", &is_ghost); + if (private_status) + return COERCE_STATUS (private_status, + "Error trying to determine whether message was a ghost"); + + message->notmuch->writable_xapian_db->delete_document (message->doc_id); + + if (is_ghost) + return NOTMUCH_STATUS_SUCCESS; + + _notmuch_database_find_doc_ids (message->notmuch, "thread", tid, &thread_doc, + &thread_doc_end); + _notmuch_database_find_doc_ids (message->notmuch, "type", "mail", &mail_doc, &mail_doc_end); + + while (count == 0 && + thread_doc != thread_doc_end && + mail_doc != mail_doc_end) { + thread_doc.skip_to (*mail_doc); + if (thread_doc != thread_doc_end) { + if (*thread_doc == *mail_doc) { + count++; + } else { + mail_doc.skip_to (*thread_doc); + if (mail_doc != mail_doc_end && *thread_doc == *mail_doc) + count++; + } + } + } + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; } - if (count > 0) { /* reintroduce a ghost in its place because there are still * other active messages in this thread: */ @@ -1348,27 +1449,21 @@ _notmuch_message_delete (notmuch_message_t *message) notmuch_message_destroy (ghost); status = COERCE_STATUS (private_status, "Error converting to ghost message"); } else { - /* the thread is empty; drop all ghost messages from it */ - notmuch_messages_t *messages; - status = _notmuch_query_search_documents (query, - "ghost", - &messages); - if (status == NOTMUCH_STATUS_SUCCESS) { - notmuch_status_t last_error = NOTMUCH_STATUS_SUCCESS; - while (notmuch_messages_valid (messages)) { - message = notmuch_messages_get (messages); - status = _notmuch_message_delete (message); - if (status) /* we'll report the last failure we see; - * if there is more than one failure, we - * forget about previous ones */ - last_error = status; - notmuch_message_destroy (message); - notmuch_messages_move_to_next (messages); + /* the thread now contains only ghosts: delete them */ + try { + Xapian::PostingIterator doc, doc_end; + + _notmuch_database_find_doc_ids (message->notmuch, "thread", tid, &doc, &doc_end); + + for (; doc != doc_end; doc++) { + message->notmuch->writable_xapian_db->delete_document (*doc); } - status = last_error; + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; } + } - notmuch_query_destroy (query); return status; } @@ -1410,31 +1505,37 @@ _notmuch_message_close (notmuch_message_t *message) * * This change will not be reflected in the database until the next * call to _notmuch_message_sync. */ -notmuch_private_status_t +NODISCARD notmuch_private_status_t _notmuch_message_add_term (notmuch_message_t *message, const char *prefix_name, const char *value) { char *term; + notmuch_private_status_t status = NOTMUCH_PRIVATE_STATUS_SUCCESS; if (value == NULL) return NOTMUCH_PRIVATE_STATUS_NULL_POINTER; term = talloc_asprintf (message, "%s%s", _find_prefix (prefix_name), value); + if (strlen (term) > NOTMUCH_TERM_MAX) { + status = NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG; + goto DONE; + } - if (strlen (term) > NOTMUCH_TERM_MAX) - return NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG; - - message->doc.add_term (term, 0); - message->modified = true; + try { + message->doc.add_term (term, 0); + message->modified = true; + _notmuch_message_invalidate_metadata (message, prefix_name); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + status = NOTMUCH_PRIVATE_STATUS_XAPIAN_EXCEPTION; + } + DONE: talloc_free (term); - - _notmuch_message_invalidate_metadata (message, prefix_name); - - return NOTMUCH_PRIVATE_STATUS_SUCCESS; + return status; } /* Parse 'text' and add a term to 'message' for each parsed word. Each @@ -1479,7 +1580,7 @@ _notmuch_message_gen_terms (notmuch_message_t *message, * * This change will not be reflected in the database until the next * call to _notmuch_message_sync. */ -notmuch_private_status_t +NODISCARD notmuch_private_status_t _notmuch_message_remove_term (notmuch_message_t *message, const char *prefix_name, const char *value) @@ -1498,11 +1599,12 @@ _notmuch_message_remove_term (notmuch_message_t *message, try { message->doc.remove_term (term); message->modified = true; - } catch (const Xapian::InvalidArgumentError) { + } catch (const Xapian::InvalidArgumentError &error) { /* We'll let the philosophers try to wrestle with the * question of whether failing to remove that which was not * there in the first place is failure. For us, we'll silently * consider it all good. */ + LOG_XAPIAN_EXCEPTION (message, error); } talloc_free (term); @@ -1553,24 +1655,31 @@ notmuch_message_add_tag (notmuch_message_t *message, const char *tag) notmuch_private_status_t private_status; notmuch_status_t status; - status = _notmuch_database_ensure_writable (message->notmuch); - if (status) - return status; + try { + status = _notmuch_database_ensure_writable (message->notmuch); + if (status) + return status; - if (tag == NULL) - return NOTMUCH_STATUS_NULL_POINTER; + if (tag == NULL) + return NOTMUCH_STATUS_NULL_POINTER; - if (strlen (tag) > NOTMUCH_TAG_MAX) - return NOTMUCH_STATUS_TAG_TOO_LONG; + if (strlen (tag) > NOTMUCH_TAG_MAX) + return NOTMUCH_STATUS_TAG_TOO_LONG; - private_status = _notmuch_message_add_term (message, "tag", tag); - if (private_status) { - INTERNAL_ERROR ("_notmuch_message_add_term return unexpected value: %d\n", - private_status); - } + private_status = _notmuch_message_add_term (message, "tag", tag); + if (private_status) { + return COERCE_STATUS (private_status, + "_notmuch_message_remove_term return unexpected value: %d\n", + private_status); + } - if (! message->frozen) - _notmuch_message_sync (message); + if (! message->frozen) + _notmuch_message_sync (message); + + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } return NOTMUCH_STATUS_SUCCESS; } @@ -1581,24 +1690,30 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag) notmuch_private_status_t private_status; notmuch_status_t status; - status = _notmuch_database_ensure_writable (message->notmuch); - if (status) - return status; + try { + status = _notmuch_database_ensure_writable (message->notmuch); + if (status) + return status; - if (tag == NULL) - return NOTMUCH_STATUS_NULL_POINTER; + if (tag == NULL) + return NOTMUCH_STATUS_NULL_POINTER; - if (strlen (tag) > NOTMUCH_TAG_MAX) - return NOTMUCH_STATUS_TAG_TOO_LONG; + if (strlen (tag) > NOTMUCH_TAG_MAX) + return NOTMUCH_STATUS_TAG_TOO_LONG; - private_status = _notmuch_message_remove_term (message, "tag", tag); - if (private_status) { - INTERNAL_ERROR ("_notmuch_message_remove_term return unexpected value: %d\n", - private_status); - } + private_status = _notmuch_message_remove_term (message, "tag", tag); + if (private_status) { + return COERCE_STATUS (private_status, + "_notmuch_message_remove_term return unexpected value: %d\n", + private_status); + } - if (! message->frozen) - _notmuch_message_sync (message); + if (! message->frozen) + _notmuch_message_sync (message); + } catch (Xapian::Error &error) { + LOG_XAPIAN_EXCEPTION (message, error); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } return NOTMUCH_STATUS_SUCCESS; } @@ -1643,7 +1758,7 @@ _filename_is_in_maildir (const char *filename) return NULL; } -static void +static notmuch_status_t _ensure_maildir_flags (notmuch_message_t *message, bool force) { const char *flags; @@ -1658,8 +1773,10 @@ _ensure_maildir_flags (notmuch_message_t *message, bool force) message->maildir_flags = NULL; } } - - for (filenames = notmuch_message_get_filenames (message); + filenames = notmuch_message_get_filenames (message); + if (! filenames) + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + for (; notmuch_filenames_valid (filenames); notmuch_filenames_move_to_next (filenames)) { filename = notmuch_filenames_get (filenames); @@ -1685,13 +1802,38 @@ _ensure_maildir_flags (notmuch_message_t *message, bool force) } if (seen_maildir_info) message->maildir_flags = combined_flags; + return NOTMUCH_STATUS_SUCCESS; } notmuch_bool_t notmuch_message_has_maildir_flag (notmuch_message_t *message, char flag) { - _ensure_maildir_flags (message, false); - return message->maildir_flags && (strchr (message->maildir_flags, flag) != NULL); + notmuch_status_t status; + notmuch_bool_t ret; + + status = notmuch_message_has_maildir_flag_st (message, flag, &ret); + if (status) + return FALSE; + + return ret; +} + +notmuch_status_t +notmuch_message_has_maildir_flag_st (notmuch_message_t *message, + char flag, + notmuch_bool_t *is_set) +{ + notmuch_status_t status; + + if (! is_set) + return NOTMUCH_STATUS_NULL_POINTER; + + status = _ensure_maildir_flags (message, false); + if (status) + return status; + + *is_set = message->maildir_flags && (strchr (message->maildir_flags, flag) != NULL); + return NOTMUCH_STATUS_SUCCESS; } notmuch_status_t @@ -1700,7 +1842,9 @@ notmuch_message_maildir_flags_to_tags (notmuch_message_t *message) notmuch_status_t status; unsigned i; - _ensure_maildir_flags (message, true); + status = _ensure_maildir_flags (message, true); + if (status) + return status; /* If none of the filenames have any maildir info field (not even * an empty info with no flags set) then there's no information to * go on, so do nothing. */ @@ -1906,6 +2050,10 @@ notmuch_message_tags_to_maildir_flags (notmuch_message_t *message) char *to_set, *to_clear; notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + status = _notmuch_database_ensure_writable (message->notmuch); + if (status) + return status; + _get_maildir_flag_actions (message, &to_set, &to_clear); for (filenames = notmuch_message_get_filenames (message); @@ -1969,16 +2117,20 @@ notmuch_message_remove_all_tags (notmuch_message_t *message) status = _notmuch_database_ensure_writable (message->notmuch); if (status) return status; + tags = notmuch_message_get_tags (message); + if (! tags) + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; - for (tags = notmuch_message_get_tags (message); + for (; notmuch_tags_valid (tags); notmuch_tags_move_to_next (tags)) { tag = notmuch_tags_get (tags); private_status = _notmuch_message_remove_term (message, "tag", tag); if (private_status) { - INTERNAL_ERROR ("_notmuch_message_remove_term return unexpected value: %d\n", - private_status); + return COERCE_STATUS (private_status, + "_notmuch_message_remove_term return unexpected value: %d\n", + private_status); } } @@ -2099,8 +2251,10 @@ notmuch_message_reindex (notmuch_message_t *message, /* Save in case we need to delete message */ orig_thread_id = notmuch_message_get_thread_id (message); if (! orig_thread_id) { - /* XXX TODO: make up new error return? */ - INTERNAL_ERROR ("message without thread-id"); + /* the following is correct as long as there is only one reason + * n_m_get_thread_id returns NULL + */ + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; } /* strdup it because the metadata may be invalidated */ @@ -2163,7 +2317,11 @@ notmuch_message_reindex (notmuch_message_t *message, if (thread_id == NULL) thread_id = orig_thread_id; - _notmuch_message_add_term (message, "thread", thread_id); + ret = COERCE_STATUS (_notmuch_message_add_term (message, "thread", thread_id), + "adding thread term"); + if (ret) + goto DONE; + /* Take header values only from first filename */ if (found == 0) _notmuch_message_set_header_values (message, date, from, subject); @@ -2181,7 +2339,11 @@ notmuch_message_reindex (notmuch_message_t *message, } if (found == 0) { /* put back thread id to help cleanup */ - _notmuch_message_add_term (message, "thread", orig_thread_id); + ret = COERCE_STATUS (_notmuch_message_add_term (message, "thread", orig_thread_id), + "adding thread term"); + if (ret) + goto DONE; + ret = _notmuch_message_delete (message); } else { _notmuch_message_sync (message); diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h index 28ced3a2..367e23e6 100644 --- a/lib/notmuch-private.h +++ b/lib/notmuch-private.h @@ -31,6 +31,12 @@ #include "notmuch.h" +#include "xutil.h" +#include "error_util.h" +#include "string-util.h" +#include "crypto.h" +#include "repair.h" + NOTMUCH_BEGIN_DECLS #include @@ -47,14 +53,6 @@ NOTMUCH_BEGIN_DECLS #include -#include "gmime-extra.h" - -#include "xutil.h" -#include "error_util.h" -#include "string-util.h" -#include "crypto.h" -#include "repair.h" - #ifdef DEBUG # define DEBUG_DATABASE_SANITY 1 # define DEBUG_THREADING 1 @@ -76,7 +74,7 @@ NOTMUCH_BEGIN_DECLS #define NOTMUCH_CLEAR_BIT(valp, bit) \ (_NOTMUCH_VALID_BIT (bit) ? (*(valp) &= ~(1ull << (bit))) : *(valp)) -#define unused(x) x __attribute__ ((unused)) +#define unused(x) x ## _unused __attribute__ ((unused)) /* Thanks to Andrew Tridgell's (SAMBA's) talloc for this definition of * unlikely. The talloc source code comes to us via the GNU LGPL v. 3. @@ -123,16 +121,32 @@ typedef enum { */ #define NOTMUCH_MESSAGE_ID_MAX (200 - sizeof (NOTMUCH_METADATA_THREAD_ID_PREFIX)) -typedef enum _notmuch_private_status { +typedef enum { /* First, copy all the public status values. */ NOTMUCH_PRIVATE_STATUS_SUCCESS = NOTMUCH_STATUS_SUCCESS, NOTMUCH_PRIVATE_STATUS_OUT_OF_MEMORY = NOTMUCH_STATUS_OUT_OF_MEMORY, NOTMUCH_PRIVATE_STATUS_READ_ONLY_DATABASE = NOTMUCH_STATUS_READ_ONLY_DATABASE, NOTMUCH_PRIVATE_STATUS_XAPIAN_EXCEPTION = NOTMUCH_STATUS_XAPIAN_EXCEPTION, + NOTMUCH_PRIVATE_STATUS_FILE_ERROR = NOTMUCH_STATUS_FILE_ERROR, NOTMUCH_PRIVATE_STATUS_FILE_NOT_EMAIL = NOTMUCH_STATUS_FILE_NOT_EMAIL, NOTMUCH_PRIVATE_STATUS_NULL_POINTER = NOTMUCH_STATUS_NULL_POINTER, NOTMUCH_PRIVATE_STATUS_TAG_TOO_LONG = NOTMUCH_STATUS_TAG_TOO_LONG, NOTMUCH_PRIVATE_STATUS_UNBALANCED_FREEZE_THAW = NOTMUCH_STATUS_UNBALANCED_FREEZE_THAW, + NOTMUCH_PRIVATE_STATUS_UNBALANCED_ATOMIC = NOTMUCH_STATUS_UNBALANCED_ATOMIC, + NOTMUCH_PRIVATE_STATUS_UNSUPPORTED_OPERATION = NOTMUCH_STATUS_UNSUPPORTED_OPERATION, + NOTMUCH_PRIVATE_STATUS_UPGRADE_REQUIRED = NOTMUCH_STATUS_UPGRADE_REQUIRED, + NOTMUCH_PRIVATE_STATUS_PATH_ERROR = NOTMUCH_STATUS_PATH_ERROR, + NOTMUCH_PRIVATE_STATUS_IGNORED = NOTMUCH_STATUS_IGNORED, + NOTMUCH_PRIVATE_STATUS_ILLEGAL_ARGUMENT = NOTMUCH_STATUS_ILLEGAL_ARGUMENT, + NOTMUCH_PRIVATE_STATUS_MALFORMED_CRYPTO_PROTOCOL = NOTMUCH_STATUS_MALFORMED_CRYPTO_PROTOCOL, + NOTMUCH_PRIVATE_STATUS_FAILED_CRYPTO_CONTEXT_CREATION = NOTMUCH_STATUS_FAILED_CRYPTO_CONTEXT_CREATION, + NOTMUCH_PRIVATE_STATUS_UNKNOWN_CRYPTO_PROTOCOL = NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL, + NOTMUCH_PRIVATE_STATUS_NO_CONFIG = NOTMUCH_STATUS_NO_CONFIG, + NOTMUCH_PRIVATE_STATUS_NO_DATABASE = NOTMUCH_STATUS_NO_DATABASE, + NOTMUCH_PRIVATE_STATUS_DATABASE_EXISTS = NOTMUCH_STATUS_DATABASE_EXISTS, + NOTMUCH_PRIVATE_STATUS_NO_MAIL_ROOT = NOTMUCH_STATUS_NO_MAIL_ROOT, + NOTMUCH_PRIVATE_STATUS_BAD_QUERY_SYNTAX = NOTMUCH_STATUS_BAD_QUERY_SYNTAX, + NOTMUCH_PRIVATE_STATUS_CLOSED_DATABASE = NOTMUCH_STATUS_CLOSED_DATABASE, /* Then add our own private values. */ NOTMUCH_PRIVATE_STATUS_TERM_TOO_LONG = NOTMUCH_STATUS_LAST_STATUS, @@ -162,7 +176,7 @@ typedef enum _notmuch_private_status { (notmuch_status_t) private_status) /* Flags shared by various lookup functions. */ -typedef enum _notmuch_find_flags { +typedef enum { /* Lookup without creating any documents. This is the default * behavior. */ NOTMUCH_FIND_LOOKUP = 0, @@ -194,9 +208,6 @@ _notmuch_message_id_compressed (void *ctx, const char *message_id); notmuch_status_t _notmuch_database_ensure_writable (notmuch_database_t *notmuch); -notmuch_status_t -_notmuch_database_reopen (notmuch_database_t *notmuch); - void _notmuch_database_log (notmuch_database_t *notmuch, const char *format, ...); @@ -248,17 +259,24 @@ _notmuch_database_filename_to_direntry (void *ctx, notmuch_find_flags_t flags, char **direntry); +bool +_notmuch_database_indexable_as_text (notmuch_database_t *notmuch, + const char *mime_string); + /* directory.cc */ notmuch_directory_t * -_notmuch_directory_create (notmuch_database_t *notmuch, - const char *path, - notmuch_find_flags_t flags, - notmuch_status_t *status_ret); +_notmuch_directory_find_or_create (notmuch_database_t *notmuch, + const char *path, + notmuch_find_flags_t flags, + notmuch_status_t *status_ret); unsigned int _notmuch_directory_get_document_id (notmuch_directory_t *directory); +notmuch_database_mode_t +_notmuch_database_mode (notmuch_database_t *notmuch); + /* message.cc */ notmuch_message_t * @@ -458,11 +476,18 @@ _notmuch_database_link_message_to_parents (notmuch_database_t *notmuch, const char **thread_id); /* index.cc */ +void +_notmuch_filter_init (); + notmuch_status_t _notmuch_message_index_file (notmuch_message_t *message, notmuch_indexopts_t *indexopts, notmuch_message_file_t *message_file); +/* init.cc */ +void +_notmuch_init (); + /* messages.c */ typedef struct _notmuch_message_node { @@ -568,9 +593,6 @@ _notmuch_message_add_reply (notmuch_message_t *message, void _notmuch_message_remove_unprefixed_terms (notmuch_message_t *message); -const char * -_notmuch_message_get_thread_id_only (notmuch_message_t *message); - size_t _notmuch_message_get_thread_depth (notmuch_message_t *message); void @@ -635,6 +657,11 @@ _notmuch_string_map_append (notmuch_string_map_t *map, const char *key, const char *value); +void +_notmuch_string_map_set (notmuch_string_map_t *map, + const char *key, + const char *value); + const char * _notmuch_string_map_get (notmuch_string_map_t *map, const char *key); @@ -688,14 +715,30 @@ _notmuch_thread_create (void *ctx, /* indexopts.c */ -struct _notmuch_indexopts { - _notmuch_crypto_t crypto; -}; +struct _notmuch_indexopts; #define CONFIG_HEADER_PREFIX "index.header." #define EMPTY_STRING(s) ((s)[0] == '\0') +/* config.cc */ +notmuch_status_t +_notmuch_config_load_from_database (notmuch_database_t *db); + +notmuch_status_t +_notmuch_config_load_from_file (notmuch_database_t *db, GKeyFile *file, char **status_string); + +notmuch_status_t +_notmuch_config_load_defaults (notmuch_database_t *db); + +void +_notmuch_config_cache (notmuch_database_t *db, notmuch_config_key_t key, const char *val); + +/* open.cc */ +notmuch_status_t +_notmuch_choose_xapian_path (void *ctx, const char *database_path, const char **xapian_path, + char **message); + NOTMUCH_END_DECLS #ifdef __cplusplus @@ -714,6 +757,12 @@ _notmuch_talloc_steal (const void *new_ctx, const T *ptr) #undef talloc_steal #define talloc_steal _notmuch_talloc_steal #endif + +#if __cplusplus >= 201703L || __cppcheck__ +#define NODISCARD [[nodiscard]] +#else +#define NODISCARD /**/ +#endif #endif #endif diff --git a/lib/notmuch.h b/lib/notmuch.h index 34666b88..4e2b0fa4 100644 --- a/lib/notmuch.h +++ b/lib/notmuch.h @@ -58,7 +58,7 @@ NOTMUCH_BEGIN_DECLS * version in Makefile.local. */ #define LIBNOTMUCH_MAJOR_VERSION 5 -#define LIBNOTMUCH_MINOR_VERSION 2 +#define LIBNOTMUCH_MINOR_VERSION 6 #define LIBNOTMUCH_MICRO_VERSION 0 @@ -112,7 +112,7 @@ typedef int notmuch_bool_t; * A zero value (NOTMUCH_STATUS_SUCCESS) indicates that the function * completed without error. Any other value indicates an error. */ -typedef enum _notmuch_status { +typedef enum { /** * No error occurred. */ @@ -208,6 +208,30 @@ typedef enum _notmuch_status { * something that notmuch doesn't know how to handle. */ NOTMUCH_STATUS_UNKNOWN_CRYPTO_PROTOCOL, + /** + * Unable to load a config file + */ + NOTMUCH_STATUS_NO_CONFIG, + /** + * Unable to load a database + */ + NOTMUCH_STATUS_NO_DATABASE, + /** + * Database exists, so not (re)-created + */ + NOTMUCH_STATUS_DATABASE_EXISTS, + /** + * Syntax error in query + */ + NOTMUCH_STATUS_BAD_QUERY_SYNTAX, + /** + * No mail root could be deduced from parameters and environment + */ + NOTMUCH_STATUS_NO_MAIL_ROOT, + /** + * Database is not fully opened, or has been closed + */ + NOTMUCH_STATUS_CLOSED_DATABASE, /** * Not an actual status value. Just a way to find out how many * valid status values there are. @@ -236,6 +260,8 @@ typedef struct _notmuch_tags notmuch_tags_t; typedef struct _notmuch_directory notmuch_directory_t; typedef struct _notmuch_filenames notmuch_filenames_t; typedef struct _notmuch_config_list notmuch_config_list_t; +typedef struct _notmuch_config_values notmuch_config_values_t; +typedef struct _notmuch_config_pairs notmuch_config_pairs_t; typedef struct _notmuch_indexopts notmuch_indexopts_t; #endif /* __DOXYGEN__ */ @@ -267,6 +293,8 @@ typedef struct _notmuch_indexopts notmuch_indexopts_t; * * NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory. * + * NOTMUCH_STATUS_PATH_ERROR: filename is too long + * * NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to create the * database file (such as permission denied, or file not found, * etc.), or the database already exists. @@ -301,52 +329,205 @@ typedef enum { } notmuch_database_mode_t; /** - * Open an existing notmuch database located at 'path'. + * Deprecated alias for notmuch_database_open_with_config with + * config_path="" and error_message=NULL + * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32) + */ +NOTMUCH_DEPRECATED(5, 4) +notmuch_status_t +notmuch_database_open (const char *path, + notmuch_database_mode_t mode, + notmuch_database_t **database); +/** + * Deprecated alias for notmuch_database_open_with_config with + * config_path="" + * + * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32) + * + */ +NOTMUCH_DEPRECATED(5, 4) +notmuch_status_t +notmuch_database_open_verbose (const char *path, + notmuch_database_mode_t mode, + notmuch_database_t **database, + char **error_message); + +/** + * Open an existing notmuch database located at 'database_path', using + * configuration in 'config_path'. + * + * @param[in] database_path + * @parblock + * Path to existing database. + * + * A notmuch database is a Xapian database containing appropriate + * metadata. * * The database should have been created at some time in the past, * (not necessarily by this process), by calling - * notmuch_database_create with 'path'. By default the database should be - * opened for reading only. In order to write to the database you need to - * pass the NOTMUCH_DATABASE_MODE_READ_WRITE mode. + * notmuch_database_create. + * + * If 'database_path' is NULL, use the location specified * - * An existing notmuch database can be identified by the presence of a - * directory named ".notmuch" below 'path'. + * - in the environment variable NOTMUCH_DATABASE, if non-empty + * + * - in a configuration file, located as described under 'config_path' + * + * - by $XDG_DATA_HOME/notmuch/$PROFILE where XDG_DATA_HOME defaults + * to "$HOME/.local/share" and PROFILE as as discussed in + * 'profile' + * + * If 'database_path' is non-NULL, but does not appear to be a Xapian + * database, check for a directory '.notmuch/xapian' below + * 'database_path' (this is the behavior of + * notmuch_database_open_verbose pre-0.32). + * + * @endparblock + * @param[in] mode + * @parblock + * Mode to open database. Use one of #NOTMUCH_DATABASE_MODE_READ_WRITE + * or #NOTMUCH_DATABASE_MODE_READ_ONLY + * + * @endparblock + * @param[in] config_path + * @parblock + * Path to config file. + * + * Config file is key-value, with mandatory sections. See + * notmuch-config(5) for more information. The key-value pair + * overrides the corresponding configuration data stored in the + * database (see notmuch_database_get_config) + * + * If config_path is NULL use the path specified + * + * - in environment variable NOTMUCH_CONFIG, if non-empty + * + * - by XDG_CONFIG_HOME/notmuch/ where + * XDG_CONFIG_HOME defaults to "$HOME/.config". + * + * - by $HOME/.notmuch-config + * + * If config_path is "" (empty string) then do not + * open any configuration file. + * @endparblock + * @param[in] profile: + * @parblock + * Name of profile (configuration/database variant). + * + * If non-NULL, append to the directory / file path determined for + * config_path and database_path. + * + * If NULL then use + * - environment variable NOTMUCH_PROFILE if defined, + * - otherwise "default" for directories and "" (empty string) for paths. + * + * @endparblock + * @param[out] database + * @parblock + * Pointer to database object. May not be NULL. * * The caller should call notmuch_database_destroy when finished with * this database. * * In case of any failure, this function returns an error status and - * sets *database to NULL (after printing an error message on stderr). + * sets *database to NULL. * - * Return value: + * @endparblock + * @param[out] error_message + * If non-NULL, store error message from opening the database. + * Any such message is allocated by \a malloc(3) and should be freed + * by the caller. * - * NOTMUCH_STATUS_SUCCESS: Successfully opened the database. + * @retval NOTMUCH_STATUS_SUCCESS: Successfully opened the database. * - * NOTMUCH_STATUS_NULL_POINTER: The given 'path' argument is NULL. + * @retval NOTMUCH_STATUS_NULL_POINTER: The given \a database + * argument is NULL. * - * NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory. + * @retval NOTMUCH_STATUS_NO_CONFIG: No config file was found. Fatal. * - * NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the - * database file (such as permission denied, or file not found, + * @retval NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory. + * + * @retval NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the + * database or config file (such as permission denied, or file not found, * etc.), or the database version is unknown. * - * NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred. + * @retval NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred. + * + * @since libnotmuch 5.4 (notmuch 0.32) */ + notmuch_status_t -notmuch_database_open (const char *path, - notmuch_database_mode_t mode, - notmuch_database_t **database); +notmuch_database_open_with_config (const char *database_path, + notmuch_database_mode_t mode, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **error_message); + + /** - * Like notmuch_database_open, except optionally return an error - * message. This message is allocated by malloc and should be freed by - * the caller. + * Loads configuration from config file, database, and/or defaults + * + * For description of arguments, @see notmuch_database_open_with_config + * + * For errors other then NO_DATABASE and NO_CONFIG, *database is set to + * NULL. + * + * @retval NOTMUCH_STATUS_SUCCESS: Successfully loaded configuration. + * + * @retval NOTMUCH_STATUS_NO_CONFIG: No config file was loaded. Not fatal. + * + * @retval NOTMUCH_STATUS_NO_DATABASE: No config information was + * loaded from a database. Not fatal. + * + * @retval NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory. + * + * @retval NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the + * database or config file (such as permission denied, or file not found, + * etc.) + * + * @retval NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred. + * + * @since libnotmuch 5.4 (notmuch 0.32) */ notmuch_status_t -notmuch_database_open_verbose (const char *path, - notmuch_database_mode_t mode, - notmuch_database_t **database, - char **error_message); +notmuch_database_load_config (const char *database_path, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **error_message); + +/** + * Create a new notmuch database located at 'database_path', using + * configuration in 'config_path'. + * + * For description of arguments, @see notmuch_database_open_with_config + * + * In case of any failure, this function returns an error status and + * sets *database to NULL. + * + * @retval NOTMUCH_STATUS_SUCCESS: Successfully created the database. + * + * @retval NOTMUCH_STATUS_DATABASE_EXISTS: Database already exists, not created + * + * @retval NOTMUCH_STATUS_OUT_OF_MEMORY: Out of memory. + * + * @retval NOTMUCH_STATUS_FILE_ERROR: An error occurred trying to open the + * database or config file (such as permission denied, or file not found, + * etc.) + * + * @retval NOTMUCH_STATUS_XAPIAN_EXCEPTION: A Xapian exception occurred. + * + * @since libnotmuch 5.4 (notmuch 0.32) + */ + +notmuch_status_t +notmuch_database_create_with_config (const char *database_path, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **error_message); /** * Retrieve last status string for given database. @@ -370,11 +551,11 @@ notmuch_database_status_string (const notmuch_database_t *notmuch); * have no effect. * * For writable databases, notmuch_database_close commits all changes - * to disk before closing the database. If the caller is currently in - * an atomic section (there was a notmuch_database_begin_atomic - * without a matching notmuch_database_end_atomic), this will discard - * changes made in that atomic section (but still commit changes made - * prior to entering the atomic section). + * to disk before closing the database, unless the caller is currently + * in an atomic section (there was a notmuch_database_begin_atomic + * without a matching notmuch_database_end_atomic). In this case + * changes since the last commit are discarded. @see + * notmuch_database_end_atomic for more information. * * Return value: * @@ -410,6 +591,18 @@ notmuch_database_compact (const char *path, notmuch_compact_status_cb_t status_cb, void *closure); +/** + * Like notmuch_database_compact, but take an open database as a + * parameter. + * + * @since libnnotmuch 5.4 (notmuch 0.32) + */ +notmuch_status_t +notmuch_database_compact_db (notmuch_database_t *database, + const char *backup_path, + notmuch_compact_status_cb_t status_cb, + void *closure); + /** * Destroy the notmuch database, closing it if necessary and freeing * all associated resources. @@ -431,6 +624,8 @@ notmuch_database_get_path (notmuch_database_t *database); /** * Return the database format version of the given database. + * + * @retval 0 on error */ unsigned int notmuch_database_get_version (notmuch_database_t *database); @@ -444,6 +639,9 @@ notmuch_database_get_version (notmuch_database_t *database); * fail with NOTMUCH_STATUS_UPGRADE_REQUIRED. This always returns * FALSE for a read-only database because there's no way to upgrade a * read-only database. + * + * Also returns FALSE if an error occurs accessing the database. + * */ notmuch_bool_t notmuch_database_needs_upgrade (notmuch_database_t *database); @@ -494,7 +692,10 @@ notmuch_status_t notmuch_database_begin_atomic (notmuch_database_t *notmuch); /** - * Indicate the end of an atomic database operation. + * Indicate the end of an atomic database operation. If repeated + * (with matching notmuch_database_begin_atomic) "database.autocommit" + * times, commit the the transaction and all previous (non-cancelled) + * transactions to the database. * * Return value: * @@ -521,7 +722,8 @@ notmuch_database_end_atomic (notmuch_database_t *notmuch); * * The UUID is a NUL-terminated opaque string that uniquely identifies * this database. Two revision numbers are only comparable if they - * have the same database UUID. + * have the same database UUID. The string 'uuid' is owned by notmuch + * and should not be freed or modified by the user. */ unsigned long notmuch_database_get_revision (notmuch_database_t *notmuch, @@ -740,6 +942,19 @@ notmuch_database_find_message_by_filename (notmuch_database_t *notmuch, notmuch_tags_t * notmuch_database_get_all_tags (notmuch_database_t *db); +/** + * Reopen an open notmuch database. + * + * @param [in] db open notmuch database + * @param [in] mode mode (read only or read-write) for reopened database. + * + * @retval #NOTMUCH_STATUS_SUCCESS + * @retval #NOTMUCH_STATUS_ILLEGAL_ARGUMENT The passed database was not open. + * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION A Xapian exception occured + */ +notmuch_status_t +notmuch_database_reopen (notmuch_database_t *db, notmuch_database_mode_t mode); + /** * Create a new query for 'database'. * @@ -769,6 +984,16 @@ notmuch_query_t * notmuch_query_create (notmuch_database_t *database, const char *query_string); +typedef enum { + NOTMUCH_QUERY_SYNTAX_XAPIAN, + NOTMUCH_QUERY_SYNTAX_SEXP +} notmuch_query_syntax_t; + +notmuch_status_t +notmuch_query_create_with_syntax (notmuch_database_t *database, + const char *query_string, + notmuch_query_syntax_t syntax, + notmuch_query_t **output); /** * Sort values for notmuch_query_set_sort. */ @@ -948,7 +1173,10 @@ notmuch_query_search_threads_st (notmuch_query_t *query, notmuch_threads_t **out * * query = notmuch_query_create (database, query_string); * - * for (messages = notmuch_query_search_messages (query); + * if (notmuch_query_search_messages (query, &messages) != NOTMUCH_STATUS_SUCCESS) + * return EXIT_FAILURE; + * + * for (; * notmuch_messages_valid (messages); * notmuch_messages_move_to_next (messages)) * { @@ -1363,9 +1591,8 @@ notmuch_message_get_database (const notmuch_message_t *message); * message is valid, (which is until the query from which it derived * is destroyed). * - * This function will not return NULL since Notmuch ensures that every - * message has a unique message ID, (Notmuch will generate an ID for a - * message if the original file does not contain one). + * This function will return NULL if triggers an unhandled Xapian + * exception. */ const char * notmuch_message_get_message_id (notmuch_message_t *message); @@ -1379,8 +1606,8 @@ notmuch_message_get_message_id (notmuch_message_t *message); * notmuch_message_destroy on 'message' or until a query from which it * derived is destroyed). * - * This function will not return NULL since Notmuch ensures that every - * message belongs to a single thread. + * This function will return NULL if triggers an unhandled Xapian + * exception. */ const char * notmuch_message_get_thread_id (notmuch_message_t *message); @@ -1403,14 +1630,18 @@ notmuch_message_get_thread_id (notmuch_message_t *message); * NULL. (Note that notmuch_messages_valid will accept that NULL * value as legitimate, and simply return FALSE for it.) * - * The returned list will be destroyed when the thread is destroyed. + * This function also returns NULL if it triggers a Xapian exception. + * + * The returned list will be destroyed when the thread is + * destroyed. */ notmuch_messages_t * notmuch_message_get_replies (notmuch_message_t *message); /** * Get the total number of files associated with a message. - * @returns Non-negative integer + * @returns Non-negative integer for file count. + * @returns Negative integer for error. * @since libnotmuch 5.0 (notmuch 0.25) */ int @@ -1431,6 +1662,8 @@ notmuch_message_count_files (notmuch_message_t *message); * this function will arbitrarily return a single one of those * filenames. See notmuch_message_get_filenames for returning the * complete list of filenames. + * + * This function returns NULL if it triggers a Xapian exception. */ const char * notmuch_message_get_filename (notmuch_message_t *message); @@ -1444,6 +1677,8 @@ notmuch_message_get_filename (notmuch_message_t *message); * * Each filename in the iterator is an absolute filename, (the initial * component will match notmuch_database_get_path() ). + * + * This function returns NULL if it triggers a Xapian exception. */ notmuch_filenames_t * notmuch_message_get_filenames (notmuch_message_t *message); @@ -1465,7 +1700,7 @@ notmuch_message_reindex (notmuch_message_t *message, /** * Message flags. */ -typedef enum _notmuch_message_flag { +typedef enum { NOTMUCH_MESSAGE_FLAG_MATCH, NOTMUCH_MESSAGE_FLAG_EXCLUDED, @@ -1479,11 +1714,36 @@ typedef enum _notmuch_message_flag { /** * Get a value of a flag for the email corresponding to 'message'. + * + * returns FALSE in case of errors. + * + * @deprecated Deprecated as of libnotmuch 5.3 (notmuch 0.31). Please + * use notmuch_message_get_flag_st instead. */ +NOTMUCH_DEPRECATED (5, 3) notmuch_bool_t notmuch_message_get_flag (notmuch_message_t *message, notmuch_message_flag_t flag); +/** + * Get a value of a flag for the email corresponding to 'message'. + * + * @param message a message object + * @param flag flag to check + * @param is_set pointer to boolean to store flag value. + * + * @retval #NOTMUCH_STATUS_SUCCESS + * @retval #NOTMUCH_STATUS_NULL_POINTER is_set is NULL + * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION Accessing the database + * triggered an exception. + * + * @since libnotmuch 5.3 (notmuch 0.31) + */ +notmuch_status_t +notmuch_message_get_flag_st (notmuch_message_t *message, + notmuch_message_flag_t flag, + notmuch_bool_t *is_set); + /** * Set a value of a flag for the email corresponding to 'message'. */ @@ -1497,6 +1757,8 @@ notmuch_message_set_flag (notmuch_message_t *message, * For the original textual representation of the Date header from the * message call notmuch_message_get_header() with a header value of * "date". + * + * Returns 0 in case of error. */ time_t notmuch_message_get_date (notmuch_message_t *message); @@ -1601,8 +1863,10 @@ notmuch_message_remove_tag (notmuch_message_t *message, const char *tag); * See notmuch_message_freeze for an example showing how to safely * replace tag values. * - * NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in read-only - * mode so message cannot be modified. + * @retval #NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in + * read-only mode so message cannot be modified. + * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION: an exception was thrown + * accessing the database. */ notmuch_status_t notmuch_message_remove_all_tags (notmuch_message_t *message); @@ -1646,10 +1910,32 @@ notmuch_message_maildir_flags_to_tags (notmuch_message_t *message); * return TRUE if any filename of 'message' has maildir flag 'flag', * FALSE otherwise. * + * Deprecated wrapper for notmuch_message_has_maildir_flag_st + * + * @returns FALSE in case of error + * @deprecated libnotmuch 5.3 (notmuch 0.31) */ +NOTMUCH_DEPRECATED (5, 3) notmuch_bool_t notmuch_message_has_maildir_flag (notmuch_message_t *message, char flag); +/** + * check message for maildir flag + * + * @param [in,out] message message to check + * @param [in] flag flag to check for + * @param [out] is_set pointer to boolean + * + * @retval #NOTMUCH_STATUS_SUCCESS + * @retval #NOTMUCH_STATUS_NULL_POINTER is_set is NULL + * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION Accessing the database + * triggered an exception. + */ +notmuch_status_t +notmuch_message_has_maildir_flag_st (notmuch_message_t *message, + char flag, + notmuch_bool_t *is_set); + /** * Rename message filename(s) to encode tags as maildir flags. * @@ -1811,7 +2097,7 @@ notmuch_message_add_property (notmuch_message_t *message, const char *key, const /** * Remove a (key,value) pair from a message. * - * It is not an error to remove a non-existant (key,value) pair + * It is not an error to remove a non-existent (key,value) pair * * @returns * - NOTMUCH_STATUS_ILLEGAL_ARGUMENT: *key* may not contain an '=' character. @@ -1986,6 +2272,9 @@ notmuch_message_properties_destroy (notmuch_message_properties_t *properties); * valid string. Whereas when this function returns FALSE, * notmuch_tags_get will return NULL. * + * It is acceptable to pass NULL for 'tags', in which case this + * function will always return FALSE. + * See the documentation of notmuch_message_get_tags for example code * showing how to iterate over a notmuch_tags_t object. */ @@ -2085,6 +2374,8 @@ notmuch_directory_get_mtime (notmuch_directory_t *directory); * * The returned filenames will be the basename-entries only (not * complete paths). + * + * Returns NULL if it triggers a Xapian exception */ notmuch_filenames_t * notmuch_directory_get_child_files (notmuch_directory_t *directory); @@ -2095,6 +2386,8 @@ notmuch_directory_get_child_files (notmuch_directory_t *directory); * * The returned filenames will be the basename-entries only (not * complete paths). + * + * Returns NULL if it triggers a Xapian exception */ notmuch_filenames_t * notmuch_directory_get_child_directories (notmuch_directory_t *directory); @@ -2172,6 +2465,11 @@ notmuch_filenames_destroy (notmuch_filenames_t *filenames); * set config 'key' to 'value' * * @since libnotmuch 4.4 (notmuch 0.23) + * @retval #NOTMUCH_STATUS_READ_ONLY_DATABASE: Database was opened in + * read-only mode so message cannot be modified. + * @retval #NOTMUCH_STATUS_XAPIAN_EXCEPTION: an exception was thrown + * accessing the database. + * @retval #NOTMUCH_STATUS_SUCCESS */ notmuch_status_t notmuch_database_set_config (notmuch_database_t *db, const char *key, const char *value); @@ -2186,6 +2484,7 @@ notmuch_database_set_config (notmuch_database_t *db, const char *key, const char * caller. * * @since libnotmuch 4.4 (notmuch 0.23) + * */ notmuch_status_t notmuch_database_get_config (notmuch_database_t *db, const char *key, char **value); @@ -2196,7 +2495,8 @@ notmuch_database_get_config (notmuch_database_t *db, const char *key, char **val * @since libnotmuch 4.4 (notmuch 0.23) */ notmuch_status_t -notmuch_database_get_config_list (notmuch_database_t *db, const char *prefix, notmuch_config_list_t **out); +notmuch_database_get_config_list (notmuch_database_t *db, const char *prefix, + notmuch_config_list_t **out); /** * Is 'config_list' iterator valid (i.e. _key, _value, _move_to_next can be called). @@ -2224,6 +2524,7 @@ notmuch_config_list_key (notmuch_config_list_t *config_list); * next call to notmuch_config_list_value or notmuch config_list_destroy * * @since libnotmuch 4.4 (notmuch 0.23) + * @retval NULL for errors */ const char * notmuch_config_list_value (notmuch_config_list_t *config_list); @@ -2245,6 +2546,255 @@ notmuch_config_list_move_to_next (notmuch_config_list_t *config_list); void notmuch_config_list_destroy (notmuch_config_list_t *config_list); +/** + * Configuration keys known to libnotmuch + */ +typedef enum { + NOTMUCH_CONFIG_FIRST, + NOTMUCH_CONFIG_DATABASE_PATH = NOTMUCH_CONFIG_FIRST, + NOTMUCH_CONFIG_MAIL_ROOT, + NOTMUCH_CONFIG_HOOK_DIR, + NOTMUCH_CONFIG_BACKUP_DIR, + NOTMUCH_CONFIG_EXCLUDE_TAGS, + NOTMUCH_CONFIG_NEW_TAGS, + NOTMUCH_CONFIG_NEW_IGNORE, + NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, + NOTMUCH_CONFIG_PRIMARY_EMAIL, + NOTMUCH_CONFIG_OTHER_EMAIL, + NOTMUCH_CONFIG_USER_NAME, + NOTMUCH_CONFIG_AUTOCOMMIT, + NOTMUCH_CONFIG_EXTRA_HEADERS, + NOTMUCH_CONFIG_INDEX_AS_TEXT, + NOTMUCH_CONFIG_LAST +} notmuch_config_key_t; + +/** + * get a configuration value from an open database. + * + * This value reflects all configuration information given at the time + * the database was opened. + * + * @param[in] notmuch database + * @param[in] key configuration key + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval NULL if 'key' unknown or if no value is known for + * 'key'. Otherwise returns a string owned by notmuch which should + * not be modified nor freed by the caller. + */ +const char * +notmuch_config_get (notmuch_database_t *notmuch, notmuch_config_key_t key); + +/** + * set a configuration value from in an open database. + * + * This value reflects all configuration information given at the time + * the database was opened. + * + * @param[in,out] notmuch database open read/write + * @param[in] key configuration key + * @param[in] val configuration value + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval returns any return value for notmuch_database_set_config. + */ +notmuch_status_t +notmuch_config_set (notmuch_database_t *notmuch, notmuch_config_key_t key, const char *val); + +/** + * Returns an iterator for a ';'-delimited list of configuration values + * + * These values reflect all configuration information given at the + * time the database was opened. + * + * @param[in] notmuch database + * @param[in] key configuration key + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval NULL in case of error. + */ +notmuch_config_values_t * +notmuch_config_get_values (notmuch_database_t *notmuch, notmuch_config_key_t key); + +/** + * Returns an iterator for a ';'-delimited list of configuration values + * + * These values reflect all configuration information given at the + * time the database was opened. + * + * @param[in] notmuch database + * @param[in] key configuration key + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval NULL in case of error. + */ +notmuch_config_values_t * +notmuch_config_get_values_string (notmuch_database_t *notmuch, const char *key); + +/** + * Is the given 'config_values' iterator pointing at a valid element. + * + * @param[in] values iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval FALSE if passed a NULL pointer, or the iterator is exhausted. + * + */ +notmuch_bool_t +notmuch_config_values_valid (notmuch_config_values_t *values); + +/** + * Get the current value from the 'values' iterator + * + * @param[in] values iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval a string with the same lifetime as the iterator + */ +const char * +notmuch_config_values_get (notmuch_config_values_t *values); + +/** + * Move the 'values' iterator to the next element + * + * @param[in,out] values iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + */ +void +notmuch_config_values_move_to_next (notmuch_config_values_t *values); + + +/** + * reset the 'values' iterator to the first element + * + * @param[in,out] values iterator. A NULL value is ignored. + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + */ +void +notmuch_config_values_start (notmuch_config_values_t *values); + +/** + * Destroy a config values iterator, along with any associated + * resources. + * + * @param[in,out] values iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + */ +void +notmuch_config_values_destroy (notmuch_config_values_t *values); + + +/** + * Returns an iterator for a (key, value) configuration pairs + * + * @param[in] notmuch database + * @param[in] prefix prefix for keys. Pass "" for all keys. + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval NULL in case of error. + */ +notmuch_config_pairs_t * +notmuch_config_get_pairs (notmuch_database_t *notmuch, + const char *prefix); + +/** + * Is the given 'config_pairs' iterator pointing at a valid element. + * + * @param[in] pairs iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval FALSE if passed a NULL pointer, or the iterator is exhausted. + * + */ +notmuch_bool_t +notmuch_config_pairs_valid (notmuch_config_pairs_t *pairs); + +/** + * Move the 'config_pairs' iterator to the next element + * + * @param[in,out] pairs iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + */ +void +notmuch_config_pairs_move_to_next (notmuch_config_pairs_t *pairs); + +/** + * Get the current key from the 'config_pairs' iterator + * + * @param[in] pairs iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval a string with the same lifetime as the iterator + */ +const char * +notmuch_config_pairs_key (notmuch_config_pairs_t *pairs); + +/** + * Get the current value from the 'config_pairs' iterator + * + * @param[in] pairs iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval a string with the same lifetime as the iterator + */ +const char * +notmuch_config_pairs_value (notmuch_config_pairs_t *pairs); + +/** + * Destroy a config_pairs iterator, along with any associated + * resources. + * + * @param[in,out] pairs iterator + * + * @since libnotmuch 5.4 (notmuch 0.32) + */ +void +notmuch_config_pairs_destroy (notmuch_config_pairs_t *pairs); + +/** + * get a configuration value from an open database as Boolean + * + * This value reflects all configuration information given at the time + * the database was opened. + * + * @param[in] notmuch database + * @param[in] key configuration key + * @param[out] val configuration value, converted to Boolean + * + * @since libnotmuch 5.4 (notmuch 0.32) + * + * @retval #NOTMUCH_STATUS_ILLEGAL_ARGUMENT if either key is unknown + * or the corresponding value does not convert to Boolean. + */ +notmuch_status_t +notmuch_config_get_bool (notmuch_database_t *notmuch, + notmuch_config_key_t key, + notmuch_bool_t *val); + +/** + * return the path of the config file loaded, if any + * + * @retval NULL if no config file was loaded + */ +const char * +notmuch_config_path (notmuch_database_t *notmuch); /** * get the current default indexing options for a given database. @@ -2257,6 +2807,7 @@ notmuch_config_list_destroy (notmuch_config_list_t *config_list); * added to the index. At the moment it is a featureless stub. * * @since libnotmuch 5.1 (notmuch 0.26) + * @retval NULL in case of error */ notmuch_indexopts_t * notmuch_database_get_default_indexopts (notmuch_database_t *db); @@ -2312,7 +2863,7 @@ notmuch_indexopts_destroy (notmuch_indexopts_t *options); */ notmuch_bool_t notmuch_built_with (const char *name); -/* @} */ +/**@}*/ #pragma GCC visibility pop diff --git a/lib/open.cc b/lib/open.cc new file mode 100644 index 00000000..463e38bf --- /dev/null +++ b/lib/open.cc @@ -0,0 +1,995 @@ +#include +#include + +#include "database-private.h" +#include "parse-time-vrp.h" +#include "lastmod-fp.h" +#include "path-util.h" + +#if HAVE_XAPIAN_DB_RETRY_LOCK +#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK) +#else +#define DB_ACTION Xapian::DB_CREATE_OR_OPEN +#endif + +notmuch_status_t +notmuch_database_open (const char *path, + notmuch_database_mode_t mode, + notmuch_database_t **database) +{ + char *status_string = NULL; + notmuch_status_t status; + + status = notmuch_database_open_with_config (path, mode, "", NULL, + database, &status_string); + if (status_string) { + fputs (status_string, stderr); + free (status_string); + } + + return status; +} + +notmuch_status_t +notmuch_database_open_verbose (const char *path, + notmuch_database_mode_t mode, + notmuch_database_t **database, + char **status_string) +{ + return notmuch_database_open_with_config (path, mode, "", NULL, + database, status_string); +} + +static const char * +_xdg_dir (void *ctx, + const char *xdg_root_variable, + const char *xdg_prefix, + const char *profile_name) +{ + const char *xdg_root = getenv (xdg_root_variable); + + if (! xdg_root) { + const char *home = getenv ("HOME"); + + if (! home) return NULL; + + xdg_root = talloc_asprintf (ctx, + "%s/%s", + home, + xdg_prefix); + } + + if (! profile_name) + profile_name = getenv ("NOTMUCH_PROFILE"); + + if (! profile_name) + profile_name = "default"; + + return talloc_asprintf (ctx, + "%s/notmuch/%s", + xdg_root, + profile_name); +} + +static notmuch_status_t +_choose_dir (notmuch_database_t *notmuch, + const char *profile, + notmuch_config_key_t key, + const char *xdg_var, + const char *xdg_subdir, + const char *subdir, + char **message = NULL) +{ + const char *parent; + const char *dir; + struct stat st; + int err; + + dir = notmuch_config_get (notmuch, key); + + if (dir) + return NOTMUCH_STATUS_SUCCESS; + + parent = _xdg_dir (notmuch, xdg_var, xdg_subdir, profile); + if (! parent) + return NOTMUCH_STATUS_PATH_ERROR; + + dir = talloc_asprintf (notmuch, "%s/%s", parent, subdir); + + err = stat (dir, &st); + if (err) { + if (errno == ENOENT) { + char *notmuch_path = dirname (talloc_strdup (notmuch, notmuch->xapian_path)); + dir = talloc_asprintf (notmuch, "%s/%s", notmuch_path, subdir); + } else { + IGNORE_RESULT (asprintf (message, "Error: Cannot stat %s: %s.\n", + dir, strerror (errno))); + return NOTMUCH_STATUS_FILE_ERROR; + } + } + + _notmuch_config_cache (notmuch, key, dir); + + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_load_key_file (notmuch_database_t *notmuch, + const char *path, + const char *profile, + GKeyFile **key_file) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + + if (path && EMPTY_STRING (path)) + goto DONE; + + if (! path) + path = getenv ("NOTMUCH_CONFIG"); + + if (path) + path = talloc_strdup (notmuch, path); + else { + const char *dir = _xdg_dir (notmuch, "XDG_CONFIG_HOME", ".config", profile); + + if (dir) { + path = talloc_asprintf (notmuch, "%s/config", dir); + if (access (path, R_OK) != 0) + path = NULL; + } + } + + if (! path) { + const char *home = getenv ("HOME"); + + path = talloc_asprintf (notmuch, "%s/.notmuch-config", home); + + if (! profile) + profile = getenv ("NOTMUCH_PROFILE"); + + if (profile) + path = talloc_asprintf (notmuch, "%s.%s", path, profile); + } + + *key_file = g_key_file_new (); + if (! g_key_file_load_from_file (*key_file, path, G_KEY_FILE_NONE, NULL)) { + status = NOTMUCH_STATUS_NO_CONFIG; + } + + DONE: + if (path) + notmuch->config_path = path; + + return status; +} + +static notmuch_status_t +_db_dir_exists (const char *database_path, char **message) +{ + struct stat st; + int err; + + err = stat (database_path, &st); + if (err) { + IGNORE_RESULT (asprintf (message, "Error: Cannot open database at %s: %s.\n", + database_path, strerror (errno))); + return NOTMUCH_STATUS_FILE_ERROR; + } + + if (! S_ISDIR (st.st_mode)) { + IGNORE_RESULT (asprintf (message, "Error: Cannot open database at %s: " + "Not a directory.\n", + database_path)); + return NOTMUCH_STATUS_FILE_ERROR; + } + + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_choose_database_path (notmuch_database_t *notmuch, + const char *profile, + GKeyFile *key_file, + const char **database_path, + char **message) +{ + notmuch_status_t status; + + if (! *database_path) { + *database_path = getenv ("NOTMUCH_DATABASE"); + } + + if (! *database_path && key_file) { + char *path = g_key_file_get_string (key_file, "database", "path", NULL); + if (path) { + if (path[0] == '/') + *database_path = talloc_strdup (notmuch, path); + else + *database_path = talloc_asprintf (notmuch, "%s/%s", getenv ("HOME"), path); + g_free (path); + } + } + if (! *database_path) { + *database_path = _xdg_dir (notmuch, "XDG_DATA_HOME", ".local/share", profile); + status = _db_dir_exists (*database_path, message); + if (status) { + *database_path = NULL; + } else { + notmuch->params |= NOTMUCH_PARAM_SPLIT; + } + } + + if (! *database_path) { + *database_path = getenv ("MAILDIR"); + } + + if (! *database_path) { + *database_path = talloc_asprintf (notmuch, "%s/mail", getenv ("HOME")); + status = _db_dir_exists (*database_path, message); + if (status) { + *database_path = NULL; + } + } + + if (*database_path == NULL) { + *message = strdup ("Error: could not locate database.\n"); + return NOTMUCH_STATUS_NO_DATABASE; + } + + if (*database_path[0] != '/') { + *message = strdup ("Error: Database path must be absolute.\n"); + return NOTMUCH_STATUS_PATH_ERROR; + } + + status = _db_dir_exists (*database_path, message); + if (status) { + IGNORE_RESULT (asprintf (message, + "Error: database path '%s' does not exist or is not a directory.\n", + *database_path)); + return NOTMUCH_STATUS_NO_DATABASE; + } + + if (*message) { + free (*message); + *message = NULL; + } + + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_mkdir (const char *path, char **message) +{ + if (g_mkdir_with_parents (path, 0755)) { + IGNORE_RESULT (asprintf (message, "Error: Cannot create directory %s: %s.\n", + path, strerror (errno))); + return NOTMUCH_STATUS_FILE_ERROR; + } + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_create_database_path (notmuch_database_t *notmuch, + const char *profile, + GKeyFile *key_file, + const char **database_path, + char **message) +{ + notmuch_status_t status; + + if (! *database_path) { + *database_path = getenv ("NOTMUCH_DATABASE"); + } + + if (! *database_path && key_file) { + char *path = g_key_file_get_string (key_file, "database", "path", NULL); + if (path) { + if (path[0] == '/') + *database_path = talloc_strdup (notmuch, path); + else + *database_path = talloc_asprintf (notmuch, "%s/%s", getenv ("HOME"), path); + g_free (path); + } + } + + if (! *database_path) { + *database_path = _xdg_dir (notmuch, "XDG_DATA_HOME", ".local/share", profile); + notmuch->params |= NOTMUCH_PARAM_SPLIT; + } + + if (*database_path[0] != '/') { + *message = strdup ("Error: Database path must be absolute.\n"); + return NOTMUCH_STATUS_PATH_ERROR; + } + + if ((status = _mkdir (*database_path, message))) + return status; + + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_database_t * +_alloc_notmuch (const char *database_path, const char *config_path, const char *profile) +{ + notmuch_database_t *notmuch; + + notmuch = talloc_zero (NULL, notmuch_database_t); + if (! notmuch) + return NULL; + + notmuch->exception_reported = false; + notmuch->status_string = NULL; + notmuch->writable_xapian_db = NULL; + notmuch->config_path = NULL; + notmuch->atomic_nesting = 0; + notmuch->transaction_count = 0; + notmuch->transaction_threshold = 0; + notmuch->view = 1; + notmuch->index_as_text = NULL; + notmuch->index_as_text_length = 0; + + notmuch->params = NOTMUCH_PARAM_NONE; + if (database_path) + notmuch->params |= NOTMUCH_PARAM_DATABASE; + if (config_path) + notmuch->params |= NOTMUCH_PARAM_CONFIG; + if (profile) + notmuch->params |= NOTMUCH_PARAM_PROFILE; + + return notmuch; +} + +static notmuch_status_t +_trial_open (const char *xapian_path, char **message_ptr) +{ + try { + Xapian::Database db (xapian_path); + } catch (const Xapian::DatabaseOpeningError &error) { + IGNORE_RESULT (asprintf (message_ptr, + "Cannot open Xapian database at %s: %s\n", + xapian_path, + error.get_msg ().c_str ())); + return NOTMUCH_STATUS_PATH_ERROR; + } catch (const Xapian::Error &error) { + IGNORE_RESULT (asprintf (message_ptr, + "A Xapian exception occurred opening database: %s\n", + error.get_msg ().c_str ())); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + return NOTMUCH_STATUS_SUCCESS; +} + +notmuch_status_t +_notmuch_choose_xapian_path (void *ctx, const char *database_path, + const char **xapian_path, char **message_ptr) +{ + notmuch_status_t status; + const char *trial_path, *notmuch_path; + + status = _db_dir_exists (database_path, message_ptr); + if (status) + goto DONE; + + trial_path = talloc_asprintf (ctx, "%s/xapian", database_path); + status = _trial_open (trial_path, message_ptr); + if (status != NOTMUCH_STATUS_PATH_ERROR) + goto DONE; + + if (*message_ptr) + free (*message_ptr); + + notmuch_path = talloc_asprintf (ctx, "%s/.notmuch", database_path); + status = _db_dir_exists (notmuch_path, message_ptr); + if (status) + goto DONE; + + trial_path = talloc_asprintf (ctx, "%s/xapian", notmuch_path); + status = _trial_open (trial_path, message_ptr); + + DONE: + if (status == NOTMUCH_STATUS_SUCCESS) + *xapian_path = trial_path; + return status; +} + +static void +_set_database_path (notmuch_database_t *notmuch, + const char *database_path) +{ + char *path = talloc_strdup (notmuch, database_path); + + strip_trailing (path, '/'); + + _notmuch_config_cache (notmuch, NOTMUCH_CONFIG_DATABASE_PATH, path); +} + +static void +_load_database_state (notmuch_database_t *notmuch) +{ + std::string last_thread_id; + std::string last_mod; + + notmuch->last_doc_id = notmuch->xapian_db->get_lastdocid (); + last_thread_id = notmuch->xapian_db->get_metadata ("last_thread_id"); + if (last_thread_id.empty ()) { + notmuch->last_thread_id = 0; + } else { + const char *str; + char *end; + + str = last_thread_id.c_str (); + notmuch->last_thread_id = strtoull (str, &end, 16); + if (*end != '\0') + INTERNAL_ERROR ("Malformed database last_thread_id: %s", str); + } + + /* Get current highest revision number. */ + last_mod = notmuch->xapian_db->get_value_upper_bound ( + NOTMUCH_VALUE_LAST_MOD); + if (last_mod.empty ()) + notmuch->revision = 0; + else + notmuch->revision = Xapian::sortable_unserialise (last_mod); + notmuch->uuid = talloc_strdup ( + notmuch, notmuch->xapian_db->get_uuid ().c_str ()); +} + +/* XXX This should really be done lazily, but the error reporting path in the indexing code + * would need to be redone to report any errors. + */ +notmuch_status_t +_ensure_index_as_text (notmuch_database_t *notmuch, char **message) +{ + int nregex = 0; + regex_t *regexv = NULL; + + if (notmuch->index_as_text) + return NOTMUCH_STATUS_SUCCESS; + + for (notmuch_config_values_t *list = notmuch_config_get_values (notmuch, + NOTMUCH_CONFIG_INDEX_AS_TEXT); + notmuch_config_values_valid (list); + notmuch_config_values_move_to_next (list)) { + regex_t *new_regex; + int rerr; + const char *str = notmuch_config_values_get (list); + size_t len = strlen (str); + + /* str must be non-empty, because n_c_get_values skips empty + * strings */ + assert (len > 0); + + regexv = talloc_realloc (notmuch, regexv, regex_t, nregex + 1); + new_regex = ®exv[nregex]; + + rerr = regcomp (new_regex, str, REG_EXTENDED | REG_NOSUB); + if (rerr) { + size_t error_size = regerror (rerr, new_regex, NULL, 0); + char *error = (char *) talloc_size (str, error_size); + + regerror (rerr, new_regex, error, error_size); + IGNORE_RESULT (asprintf (message, "Error in index.as_text: %s: %s\n", error, str)); + + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + } + nregex++; + } + + notmuch->index_as_text = regexv; + notmuch->index_as_text_length = nregex; + + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_finish_open (notmuch_database_t *notmuch, + const char *profile, + notmuch_database_mode_t mode, + GKeyFile *key_file, + char **message_ptr) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + char *incompat_features; + char *message = NULL; + const char *autocommit_str; + char *autocommit_end; + unsigned int version; + const char *database_path = notmuch_database_get_path (notmuch); + + try { + + if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) { + notmuch->writable_xapian_db = new Xapian::WritableDatabase (notmuch->xapian_path, + DB_ACTION); + notmuch->xapian_db = notmuch->writable_xapian_db; + } else { + notmuch->xapian_db = new Xapian::Database (notmuch->xapian_path); + } + + /* Check version. As of database version 3, we represent + * changes in terms of features, so assume a version bump + * means a dramatically incompatible change. */ + version = notmuch_database_get_version (notmuch); + if (version > NOTMUCH_DATABASE_VERSION) { + IGNORE_RESULT (asprintf (&message, + "Error: Notmuch database at %s\n" + " has a newer database format version (%u) than supported by this\n" + " version of notmuch (%u).\n", + database_path, version, NOTMUCH_DATABASE_VERSION)); + status = NOTMUCH_STATUS_FILE_ERROR; + goto DONE; + } + + /* Check features. */ + incompat_features = NULL; + notmuch->features = _notmuch_database_parse_features ( + notmuch, notmuch->xapian_db->get_metadata ("features").c_str (), + version, mode == NOTMUCH_DATABASE_MODE_READ_WRITE ? 'w' : 'r', + &incompat_features); + if (incompat_features) { + IGNORE_RESULT (asprintf (&message, + "Error: Notmuch database at %s\n" + " requires features (%s)\n" + " not supported by this version of notmuch.\n", + database_path, incompat_features)); + status = NOTMUCH_STATUS_FILE_ERROR; + goto DONE; + } + + _load_database_state (notmuch); + + notmuch->query_parser = new Xapian::QueryParser; + notmuch->term_gen = new Xapian::TermGenerator; + notmuch->term_gen->set_stemmer (Xapian::Stem ("english")); + notmuch->value_range_processor = new Xapian::NumberRangeProcessor (NOTMUCH_VALUE_TIMESTAMP); + notmuch->date_range_processor = new ParseTimeRangeProcessor (NOTMUCH_VALUE_TIMESTAMP, + "date:"); + notmuch->last_mod_range_processor = new LastModRangeProcessor (notmuch, "lastmod:"); + notmuch->query_parser->set_default_op (Xapian::Query::OP_AND); + notmuch->query_parser->set_database (*notmuch->xapian_db); + notmuch->stemmer = new Xapian::Stem ("english"); + notmuch->query_parser->set_stemmer (*notmuch->stemmer); + notmuch->query_parser->set_stemming_strategy (Xapian::QueryParser::STEM_SOME); + notmuch->query_parser->add_rangeprocessor (notmuch->value_range_processor); + notmuch->query_parser->add_rangeprocessor (notmuch->date_range_processor); + notmuch->query_parser->add_rangeprocessor (notmuch->last_mod_range_processor); + + /* Configuration information is needed to set up query parser */ + status = _notmuch_config_load_from_database (notmuch); + if (status) + goto DONE; + + if (key_file) + status = _notmuch_config_load_from_file (notmuch, key_file, &message); + if (status) + goto DONE; + + status = _choose_dir (notmuch, profile, + NOTMUCH_CONFIG_HOOK_DIR, + "XDG_CONFIG_HOME", + ".config", + "hooks", + &message); + if (status) + goto DONE; + + status = _choose_dir (notmuch, profile, + NOTMUCH_CONFIG_BACKUP_DIR, + "XDG_DATA_HOME", + ".local/share", + "backups", + &message); + if (status) + goto DONE; + status = _notmuch_config_load_defaults (notmuch); + if (status) + goto DONE; + + status = _ensure_index_as_text (notmuch, &message); + if (status) + goto DONE; + + autocommit_str = notmuch_config_get (notmuch, NOTMUCH_CONFIG_AUTOCOMMIT); + if (unlikely (! autocommit_str)) { + INTERNAL_ERROR ("missing configuration for autocommit"); + } + notmuch->transaction_threshold = strtoul (autocommit_str, &autocommit_end, 10); + if (*autocommit_end != '\0') + INTERNAL_ERROR ("Malformed database database.autocommit value: %s", autocommit_str); + + status = _notmuch_database_setup_standard_query_fields (notmuch); + if (status) + goto DONE; + + status = _notmuch_database_setup_user_query_fields (notmuch); + if (status) + goto DONE; + + } catch (const Xapian::Error &error) { + IGNORE_RESULT (asprintf (&message, "A Xapian exception occurred opening database: %s\n", + error.get_msg ().c_str ())); + status = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + DONE: + if (message_ptr) + *message_ptr = message; + return status; +} + +notmuch_status_t +notmuch_database_open_with_config (const char *database_path, + notmuch_database_mode_t mode, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **status_string) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + notmuch_database_t *notmuch = NULL; + char *message = NULL; + GKeyFile *key_file = NULL; + + _notmuch_init (); + + notmuch = _alloc_notmuch (database_path, config_path, profile); + if (! notmuch) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = _load_key_file (notmuch, config_path, profile, &key_file); + if (status) { + message = strdup ("Error: cannot load config file.\n"); + goto DONE; + } + + if ((status = _choose_database_path (notmuch, profile, key_file, + &database_path, + &message))) + goto DONE; + + status = _db_dir_exists (database_path, &message); + if (status) + goto DONE; + + _set_database_path (notmuch, database_path); + + status = _notmuch_choose_xapian_path (notmuch, database_path, + ¬much->xapian_path, &message); + if (status) + goto DONE; + + status = _finish_open (notmuch, profile, mode, key_file, &message); + + DONE: + if (key_file) + g_key_file_free (key_file); + + if (message) { + if (status_string) + *status_string = message; + else + free (message); + } + + if (status && notmuch) { + notmuch_database_destroy (notmuch); + notmuch = NULL; + } + + if (database) + *database = notmuch; + + if (notmuch) + notmuch->open = true; + + return status; +} + +notmuch_status_t +notmuch_database_create (const char *path, notmuch_database_t **database) +{ + char *status_string = NULL; + notmuch_status_t status; + + status = notmuch_database_create_verbose (path, database, + &status_string); + + if (status_string) { + fputs (status_string, stderr); + free (status_string); + } + + return status; +} + +notmuch_status_t +notmuch_database_create_verbose (const char *path, + notmuch_database_t **database, + char **status_string) +{ + return notmuch_database_create_with_config (path, "", NULL, database, status_string); +} + +notmuch_status_t +notmuch_database_create_with_config (const char *database_path, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **status_string) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + notmuch_database_t *notmuch = NULL; + const char *notmuch_path = NULL; + char *message = NULL; + GKeyFile *key_file = NULL; + + _notmuch_init (); + + notmuch = _alloc_notmuch (database_path, config_path, profile); + if (! notmuch) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = _load_key_file (notmuch, config_path, profile, &key_file); + if (status) { + message = strdup ("Error: cannot load config file.\n"); + goto DONE; + } + + status = _choose_database_path (notmuch, profile, key_file, + &database_path, &message); + switch (status) { + case NOTMUCH_STATUS_SUCCESS: + break; + case NOTMUCH_STATUS_NO_DATABASE: + if ((status = _create_database_path (notmuch, profile, key_file, + &database_path, &message))) + goto DONE; + break; + default: + goto DONE; + } + + _set_database_path (notmuch, database_path); + + if (key_file && ! (notmuch->params & NOTMUCH_PARAM_SPLIT)) { + char *mail_root = notmuch_canonicalize_file_name ( + g_key_file_get_string (key_file, "database", "mail_root", NULL)); + char *db_path = notmuch_canonicalize_file_name (database_path); + + if (mail_root && (0 != strcmp (mail_root, db_path))) + notmuch->params |= NOTMUCH_PARAM_SPLIT; + + free (mail_root); + free (db_path); + } + + if (notmuch->params & NOTMUCH_PARAM_SPLIT) { + notmuch_path = database_path; + } else { + if (! (notmuch_path = talloc_asprintf (notmuch, "%s/%s", database_path, ".notmuch"))) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = _mkdir (notmuch_path, &message); + if (status) + goto DONE; + } + + if (! (notmuch->xapian_path = talloc_asprintf (notmuch, "%s/%s", notmuch_path, "xapian"))) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = _trial_open (notmuch->xapian_path, &message); + if (status == NOTMUCH_STATUS_SUCCESS) { + notmuch_database_destroy (notmuch); + notmuch = NULL; + status = NOTMUCH_STATUS_DATABASE_EXISTS; + goto DONE; + } + + if (message) + free (message); + + status = _finish_open (notmuch, + profile, + NOTMUCH_DATABASE_MODE_READ_WRITE, + key_file, + &message); + if (status) + goto DONE; + + /* Upgrade doesn't add these feature to existing databases, but + * new databases have them. */ + notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES; + notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES; + notmuch->features |= NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY; + + status = notmuch_database_upgrade (notmuch, NULL, NULL); + if (status) { + notmuch_database_close (notmuch); + notmuch = NULL; + } + + DONE: + if (key_file) + g_key_file_free (key_file); + + if (message) { + if (status_string) + *status_string = message; + else + free (message); + } + if (status && notmuch) { + notmuch_database_destroy (notmuch); + notmuch = NULL; + } + + if (database) + *database = notmuch; + + if (notmuch) + notmuch->open = true; + return status; +} + +notmuch_status_t +notmuch_database_reopen (notmuch_database_t *notmuch, + notmuch_database_mode_t new_mode) +{ + notmuch_database_mode_t cur_mode = _notmuch_database_mode (notmuch); + + if (notmuch->xapian_db == NULL) { + _notmuch_database_log (notmuch, "Cannot reopen closed or nonexistent database\n"); + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + } + + try { + if (cur_mode == new_mode && + new_mode == NOTMUCH_DATABASE_MODE_READ_ONLY) { + notmuch->xapian_db->reopen (); + } else { + notmuch->xapian_db->close (); + + delete notmuch->xapian_db; + notmuch->xapian_db = NULL; + /* no need to free the same object twice */ + notmuch->writable_xapian_db = NULL; + + if (new_mode == NOTMUCH_DATABASE_MODE_READ_WRITE) { + notmuch->writable_xapian_db = new Xapian::WritableDatabase (notmuch->xapian_path, + DB_ACTION); + notmuch->xapian_db = notmuch->writable_xapian_db; + } else { + notmuch->xapian_db = new Xapian::Database (notmuch->xapian_path, + DB_ACTION); + } + } + + _load_database_state (notmuch); + } catch (const Xapian::Error &error) { + if (! notmuch->exception_reported) { + _notmuch_database_log (notmuch, "Error: A Xapian exception reopening database: %s\n", + error.get_msg ().c_str ()); + notmuch->exception_reported = true; + } + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + + notmuch->view++; + notmuch->open = true; + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_maybe_load_config_from_database (notmuch_database_t *notmuch, + GKeyFile *key_file, + const char *database_path, + const char *profile) +{ + char *message; /* ignored */ + + if (_db_dir_exists (database_path, &message)) + return NOTMUCH_STATUS_NO_DATABASE; + + _set_database_path (notmuch, database_path); + + if (_notmuch_choose_xapian_path (notmuch, database_path, ¬much->xapian_path, &message)) + return NOTMUCH_STATUS_NO_DATABASE; + + (void) _finish_open (notmuch, profile, NOTMUCH_DATABASE_MODE_READ_ONLY, key_file, &message); + + return NOTMUCH_STATUS_SUCCESS; +} + +notmuch_status_t +notmuch_database_load_config (const char *database_path, + const char *config_path, + const char *profile, + notmuch_database_t **database, + char **status_string) +{ + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS, warning = NOTMUCH_STATUS_SUCCESS; + notmuch_database_t *notmuch = NULL; + char *message = NULL; + GKeyFile *key_file = NULL; + + _notmuch_init (); + + notmuch = _alloc_notmuch (database_path, config_path, profile); + if (! notmuch) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = _load_key_file (notmuch, config_path, profile, &key_file); + switch (status) { + case NOTMUCH_STATUS_SUCCESS: + break; + case NOTMUCH_STATUS_NO_CONFIG: + warning = status; + break; + default: + message = strdup ("Error: cannot load config file.\n"); + goto DONE; + } + + status = _choose_database_path (notmuch, profile, key_file, + &database_path, &message); + switch (status) { + case NOTMUCH_STATUS_NO_DATABASE: + case NOTMUCH_STATUS_SUCCESS: + if (! warning) + warning = status; + break; + default: + goto DONE; + } + + + if (database_path) { + status = _maybe_load_config_from_database (notmuch, key_file, database_path, profile); + switch (status) { + case NOTMUCH_STATUS_NO_DATABASE: + case NOTMUCH_STATUS_SUCCESS: + if (! warning) + warning = status; + break; + default: + goto DONE; + } + } + + if (key_file) { + status = _notmuch_config_load_from_file (notmuch, key_file, &message); + if (status) + goto DONE; + } + status = _notmuch_config_load_defaults (notmuch); + if (status) + goto DONE; + + DONE: + if (status_string) + *status_string = message; + + if (status && + status != NOTMUCH_STATUS_NO_DATABASE + && status != NOTMUCH_STATUS_NO_CONFIG) { + notmuch_database_destroy (notmuch); + notmuch = NULL; + } + + if (database) + *database = notmuch; + + if (status) + return status; + else + return warning; +} diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc new file mode 100644 index 00000000..9cadbc13 --- /dev/null +++ b/lib/parse-sexp.cc @@ -0,0 +1,731 @@ +#include "database-private.h" + +#if HAVE_SFSEXP +#include "sexp.h" +#include "unicode-util.h" + +/* _sexp is used for file scope symbols to avoid clashing with + * definitions from sexp.h */ + +/* sexp_binding structs attach name to a sexp and a defining + * context. The latter allows lazy evaluation of parameters whose + * definition contains other parameters. Lazy evaluation is needed + * because a primary goal of macros is to change the parent field for + * a sexp. + */ + +typedef struct sexp_binding { + const char *name; + const sexp_t *sx; + const struct sexp_binding *context; + const struct sexp_binding *next; +} _sexp_binding_t; + +typedef enum { + SEXP_FLAG_NONE = 0, + SEXP_FLAG_FIELD = 1 << 0, + SEXP_FLAG_BOOLEAN = 1 << 1, + SEXP_FLAG_SINGLE = 1 << 2, + SEXP_FLAG_WILDCARD = 1 << 3, + SEXP_FLAG_REGEX = 1 << 4, + SEXP_FLAG_DO_REGEX = 1 << 5, + SEXP_FLAG_EXPAND = 1 << 6, + SEXP_FLAG_DO_EXPAND = 1 << 7, + SEXP_FLAG_ORPHAN = 1 << 8, + SEXP_FLAG_RANGE = 1 << 9, + SEXP_FLAG_PATHNAME = 1 << 10, +} _sexp_flag_t; + +/* + * define bitwise operators to hide casts */ + +inline _sexp_flag_t +operator| (_sexp_flag_t a, _sexp_flag_t b) +{ + return static_cast<_sexp_flag_t>( + static_cast(a) | static_cast(b)); +} + +inline _sexp_flag_t +operator& (_sexp_flag_t a, _sexp_flag_t b) +{ + return static_cast<_sexp_flag_t>( + static_cast(a) & static_cast(b)); +} + +typedef struct { + const char *name; + Xapian::Query::op xapian_op; + Xapian::Query initial; + _sexp_flag_t flags; +} _sexp_prefix_t; + +static _sexp_prefix_t prefixes[] = +{ + { "and", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_NONE }, + { "attachment", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD | SEXP_FLAG_EXPAND }, + { "body", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD }, + { "date", Xapian::Query::OP_INVALID, Xapian::Query::MatchAll, + SEXP_FLAG_RANGE }, + { "from", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "folder", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND | + SEXP_FLAG_PATHNAME }, + { "id", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX }, + { "infix", Xapian::Query::OP_INVALID, Xapian::Query::MatchAll, + SEXP_FLAG_SINGLE | SEXP_FLAG_ORPHAN }, + { "is", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "lastmod", Xapian::Query::OP_INVALID, Xapian::Query::MatchAll, + SEXP_FLAG_RANGE }, + { "matching", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_DO_EXPAND }, + { "mid", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX }, + { "mimetype", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD | SEXP_FLAG_EXPAND }, + { "not", Xapian::Query::OP_AND_NOT, Xapian::Query::MatchAll, + SEXP_FLAG_NONE }, + { "of", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_DO_EXPAND }, + { "or", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_NONE }, + { "path", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | + SEXP_FLAG_PATHNAME }, + { "property", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "query", Xapian::Query::OP_INVALID, Xapian::Query::MatchNothing, + SEXP_FLAG_SINGLE | SEXP_FLAG_ORPHAN }, + { "regex", Xapian::Query::OP_INVALID, Xapian::Query::MatchAll, + SEXP_FLAG_SINGLE | SEXP_FLAG_DO_REGEX }, + { "rx", Xapian::Query::OP_INVALID, Xapian::Query::MatchAll, + SEXP_FLAG_SINGLE | SEXP_FLAG_DO_REGEX }, + { "starts-with", Xapian::Query::OP_WILDCARD, Xapian::Query::MatchAll, + SEXP_FLAG_SINGLE }, + { "subject", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "tag", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "thread", Xapian::Query::OP_OR, Xapian::Query::MatchNothing, + SEXP_FLAG_FIELD | SEXP_FLAG_BOOLEAN | SEXP_FLAG_WILDCARD | SEXP_FLAG_REGEX | SEXP_FLAG_EXPAND }, + { "to", Xapian::Query::OP_AND, Xapian::Query::MatchAll, + SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD | SEXP_FLAG_EXPAND }, + { } +}; + +static notmuch_status_t _sexp_to_xapian_query (notmuch_database_t *notmuch, + const _sexp_prefix_t *parent, + const _sexp_binding_t *env, + const sexp_t *sx, + Xapian::Query &output); + +static notmuch_status_t +_sexp_combine_query (notmuch_database_t *notmuch, + const _sexp_prefix_t *parent, + const _sexp_binding_t *env, + Xapian::Query::op operation, + Xapian::Query left, + const sexp_t *sx, + Xapian::Query &output) +{ + Xapian::Query subquery; + + notmuch_status_t status; + + /* if we run out elements, return accumulator */ + + if (! sx) { + output = left; + return NOTMUCH_STATUS_SUCCESS; + } + + status = _sexp_to_xapian_query (notmuch, parent, env, sx, subquery); + if (status) + return status; + + return _sexp_combine_query (notmuch, + parent, + env, + operation, + Xapian::Query (operation, left, subquery), + sx->next, output); +} + +static notmuch_status_t +_sexp_parse_phrase (std::string term_prefix, const char *phrase, Xapian::Query &output) +{ + Xapian::Utf8Iterator p (phrase); + Xapian::Utf8Iterator end; + std::vector terms; + + while (p != end) { + Xapian::Utf8Iterator start; + while (p != end && ! Xapian::Unicode::is_wordchar (*p)) + p++; + + if (p == end) + break; + + start = p; + + while (p != end && Xapian::Unicode::is_wordchar (*p)) + p++; + + if (p != start) { + std::string word (start, p); + word = Xapian::Unicode::tolower (word); + terms.push_back (term_prefix + word); + } + } + output = Xapian::Query (Xapian::Query::OP_PHRASE, terms.begin (), terms.end ()); + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +resolve_binding (notmuch_database_t *notmuch, const _sexp_binding_t *env, const char *name, + const _sexp_binding_t **out) +{ + for (; env; env = env->next) { + if (strcmp (name, env->name) == 0) { + *out = env; + return NOTMUCH_STATUS_SUCCESS; + } + } + + _notmuch_database_log (notmuch, "undefined parameter '%s'\n", name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; +} + +static notmuch_status_t +_sexp_expand_term (notmuch_database_t *notmuch, + const _sexp_prefix_t *prefix, + const _sexp_binding_t *env, + const sexp_t *sx, + const char **out) +{ + notmuch_status_t status; + + if (! out) + return NOTMUCH_STATUS_NULL_POINTER; + + while (sx->ty == SEXP_VALUE && sx->aty == SEXP_BASIC && sx->val[0] == ',') { + const char *name = sx->val + 1; + const _sexp_binding_t *binding; + + status = resolve_binding (notmuch, env, name, &binding); + if (status) + return status; + + sx = binding->sx; + env = binding->context; + } + + if (sx->ty != SEXP_VALUE) { + _notmuch_database_log (notmuch, "'%s' expects single atom as argument\n", + prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + *out = sx->val; + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_sexp_parse_wildcard (notmuch_database_t *notmuch, + const _sexp_prefix_t *parent, + unused(const _sexp_binding_t *env), + std::string match, + Xapian::Query &output) +{ + + std::string term_prefix = parent ? _notmuch_database_prefix (notmuch, parent->name) : ""; + + if (parent && ! (parent->flags & SEXP_FLAG_WILDCARD)) { + _notmuch_database_log (notmuch, "'%s' does not support wildcard queries\n", parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + output = Xapian::Query (Xapian::Query::OP_WILDCARD, + term_prefix + Xapian::Unicode::tolower (match)); + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_sexp_parse_one_term (notmuch_database_t *notmuch, std::string term_prefix, const sexp_t *sx, + Xapian::Query &output) +{ + Xapian::Stem stem = *(notmuch->stemmer); + + if (sx->aty == SEXP_BASIC && unicode_word_utf8 (sx->val)) { + std::string term = Xapian::Unicode::tolower (sx->val); + + output = Xapian::Query ("Z" + term_prefix + stem (term)); + return NOTMUCH_STATUS_SUCCESS; + } else { + return _sexp_parse_phrase (term_prefix, sx->val, output); + } + +} + +notmuch_status_t +_sexp_parse_regex (notmuch_database_t *notmuch, + const _sexp_prefix_t *prefix, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, + const sexp_t *term, Xapian::Query &output) +{ + if (! parent) { + _notmuch_database_log (notmuch, "illegal '%s' outside field\n", + prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (! (parent->flags & SEXP_FLAG_REGEX)) { + _notmuch_database_log (notmuch, "'%s' not supported in field '%s'\n", + prefix->name, parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + std::string msg; /* ignored */ + const char *str; + notmuch_status_t status; + + status = _sexp_expand_term (notmuch, prefix, env, term, &str); + if (status) + return status; + + return _notmuch_regexp_to_query (notmuch, Xapian::BAD_VALUENO, parent->name, + str, output, msg); +} + + +static notmuch_status_t +_sexp_expand_query (notmuch_database_t *notmuch, + const _sexp_prefix_t *prefix, const _sexp_prefix_t *parent, + unused(const _sexp_binding_t *env), const sexp_t *sx, Xapian::Query &output) +{ + Xapian::Query subquery; + notmuch_status_t status; + std::string msg; + + if (! (parent->flags & SEXP_FLAG_EXPAND)) { + _notmuch_database_log (notmuch, "'%s' unsupported inside '%s'\n", prefix->name, parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + status = _sexp_combine_query (notmuch, NULL, NULL, prefix->xapian_op, prefix->initial, sx, + subquery); + if (status) + return status; + + status = _notmuch_query_expand (notmuch, parent->name, subquery, output, msg); + if (status) { + _notmuch_database_log (notmuch, "error expanding query %s\n", msg.c_str ()); + } + return status; +} + +static notmuch_status_t +_sexp_parse_infix (notmuch_database_t *notmuch, const sexp_t *sx, Xapian::Query &output) +{ + try { + output = notmuch->query_parser->parse_query (sx->val, NOTMUCH_QUERY_PARSER_FLAGS); + } catch (const Xapian::QueryParserError &error) { + _notmuch_database_log (notmuch, "Syntax error in infix query: %s\n", sx->val); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } catch (const Xapian::Error &error) { + if (! notmuch->exception_reported) { + _notmuch_database_log (notmuch, + "A Xapian exception occurred parsing query: %s\n", + error.get_msg ().c_str ()); + _notmuch_database_log_append (notmuch, + "Query string was: %s\n", + sx->val); + notmuch->exception_reported = true; + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + } + return NOTMUCH_STATUS_SUCCESS; +} + +static notmuch_status_t +_sexp_parse_header (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, const sexp_t *sx, Xapian::Query &output) +{ + _sexp_prefix_t user_prefix; + + user_prefix.name = sx->list->val; + user_prefix.flags = SEXP_FLAG_FIELD | SEXP_FLAG_WILDCARD; + + if (parent) { + _notmuch_database_log (notmuch, "nested field: '%s' inside '%s'\n", + sx->list->val, parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + parent = &user_prefix; + + return _sexp_combine_query (notmuch, parent, env, Xapian::Query::OP_AND, Xapian::Query::MatchAll, + sx->list->next, output); +} + +static _sexp_binding_t * +_sexp_bind (void *ctx, const _sexp_binding_t *env, const char *name, const sexp_t *sx, const + _sexp_binding_t *context) +{ + _sexp_binding_t *binding = talloc (ctx, _sexp_binding_t); + + binding->name = talloc_strdup (ctx, name); + binding->sx = sx; + binding->context = context; + binding->next = env; + return binding; +} + +static notmuch_status_t +maybe_apply_macro (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, const sexp_t *sx, const sexp_t *args, + Xapian::Query &output) +{ + const sexp_t *params, *param, *arg, *body; + void *local = talloc_new (notmuch); + _sexp_binding_t *new_env = NULL; + notmuch_status_t status = NOTMUCH_STATUS_SUCCESS; + + if (sx->list->ty != SEXP_VALUE || strcmp (sx->list->val, "macro") != 0) { + status = NOTMUCH_STATUS_IGNORED; + goto DONE; + } + + params = sx->list->next; + + if (! params || (params->ty != SEXP_LIST)) { + _notmuch_database_log (notmuch, "missing (possibly empty) list of arguments to macro\n"); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + body = params->next; + + if (! body) { + _notmuch_database_log (notmuch, "missing body of macro\n"); + status = NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + goto DONE; + } + + for (param = params->list, arg = args; + param && arg; + param = param->next, arg = arg->next) { + if (param->ty != SEXP_VALUE || param->aty != SEXP_BASIC) { + _notmuch_database_log (notmuch, "macro parameters must be unquoted atoms\n"); + status = NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + goto DONE; + } + new_env = _sexp_bind (local, new_env, param->val, arg, env); + } + + if (param && ! arg) { + _notmuch_database_log (notmuch, "too few arguments to macro\n"); + status = NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + goto DONE; + } + + if (! param && arg) { + _notmuch_database_log (notmuch, "too many arguments to macro\n"); + status = NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + goto DONE; + } + + status = _sexp_to_xapian_query (notmuch, parent, new_env, body, output); + + DONE: + if (local) + talloc_free (local); + + return status; +} + +static notmuch_status_t +maybe_saved_squery (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, const sexp_t *sx, Xapian::Query &output) +{ + char *key; + char *expansion = NULL; + notmuch_status_t status; + sexp_t *saved_sexp; + void *local = talloc_new (notmuch); + char *buf; + + key = talloc_asprintf (local, "squery.%s", sx->list->val); + if (! key) { + status = NOTMUCH_STATUS_OUT_OF_MEMORY; + goto DONE; + } + + status = notmuch_database_get_config (notmuch, key, &expansion); + if (status) + goto DONE; + if (EMPTY_STRING (expansion)) { + status = NOTMUCH_STATUS_IGNORED; + goto DONE; + } + + buf = talloc_strdup (local, expansion); + /* XXX TODO: free this memory */ + saved_sexp = parse_sexp (buf, strlen (expansion)); + if (! saved_sexp) { + _notmuch_database_log (notmuch, "invalid saved s-expression query: '%s'\n", expansion); + status = NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + goto DONE; + } + + status = maybe_apply_macro (notmuch, parent, env, saved_sexp, sx->list->next, output); + if (status == NOTMUCH_STATUS_IGNORED) + status = _sexp_to_xapian_query (notmuch, parent, env, saved_sexp, output); + + DONE: + if (local) + talloc_free (local); + + return status; +} + +static notmuch_status_t +_sexp_expand_param (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, const char *name, + Xapian::Query &output) +{ + notmuch_status_t status; + + const _sexp_binding_t *binding; + + status = resolve_binding (notmuch, env, name, &binding); + if (status) + return status; + + return _sexp_to_xapian_query (notmuch, parent, binding->context, binding->sx, + output); +} + +static notmuch_status_t +_sexp_parse_range (notmuch_database_t *notmuch, const _sexp_prefix_t *prefix, + const sexp_t *sx, Xapian::Query &output) +{ + const char *from, *to; + std::string msg; + + /* empty range matches everything */ + if (! sx) { + output = Xapian::Query::MatchAll; + return NOTMUCH_STATUS_SUCCESS; + } + + if (sx->ty == SEXP_LIST) { + _notmuch_database_log (notmuch, "expected atom as first argument of '%s'\n", prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + from = sx->val; + if (strcmp (from, "*") == 0) + from = ""; + + to = from; + + if (sx->next) { + if (sx->next->ty == SEXP_LIST) { + _notmuch_database_log (notmuch, "expected atom as second argument of '%s'\n", + prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (sx->next->next) { + _notmuch_database_log (notmuch, "'%s' expects maximum of two arguments\n", prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + to = sx->next->val; + if (strcmp (to, "*") == 0) + to = ""; + } + + if (strcmp (prefix->name, "date") == 0) { + notmuch_status_t status; + status = _notmuch_date_strings_to_query (NOTMUCH_VALUE_TIMESTAMP, from, to, output, msg); + if (status) { + if (! msg.empty ()) + _notmuch_database_log (notmuch, "%s\n", msg.c_str ()); + } + return status; + } + + if (strcmp (prefix->name, "lastmod") == 0) { + notmuch_status_t status; + status = _notmuch_lastmod_strings_to_query (notmuch, from, to, output, msg); + if (status) { + if (! msg.empty ()) + _notmuch_database_log (notmuch, "%s\n", msg.c_str ()); + } + return status; + } + + _notmuch_database_log (notmuch, "unimplimented range prefix: '%s'\n", prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; +} + +/* Here we expect the s-expression to be a proper list, with first + * element defining and operation, or as a special case the empty + * list */ + +static notmuch_status_t +_sexp_to_xapian_query (notmuch_database_t *notmuch, const _sexp_prefix_t *parent, + const _sexp_binding_t *env, const sexp_t *sx, Xapian::Query &output) +{ + notmuch_status_t status; + + if (sx->ty == SEXP_VALUE && sx->aty == SEXP_BASIC && sx->val[0] == ',') { + return _sexp_expand_param (notmuch, parent, env, sx->val + 1, output); + } + + if (sx->ty == SEXP_VALUE) { + std::string term_prefix = parent ? _notmuch_database_prefix (notmuch, parent->name) : ""; + + if (sx->aty == SEXP_BASIC && strcmp (sx->val, "*") == 0) { + return _sexp_parse_wildcard (notmuch, parent, env, "", output); + } + + char *atom = sx->val; + + if (parent && parent->flags & SEXP_FLAG_PATHNAME) + strip_trailing (atom, '/'); + + if (parent && (parent->flags & SEXP_FLAG_BOOLEAN)) { + output = Xapian::Query (term_prefix + atom); + return NOTMUCH_STATUS_SUCCESS; + } + + if (parent) { + return _sexp_parse_one_term (notmuch, term_prefix, sx, output); + } else { + Xapian::Query accumulator; + for (_sexp_prefix_t *prefix = prefixes; prefix->name; prefix++) { + if (prefix->flags & SEXP_FLAG_FIELD) { + Xapian::Query subquery; + term_prefix = _notmuch_database_prefix (notmuch, prefix->name); + status = _sexp_parse_one_term (notmuch, term_prefix, sx, subquery); + if (status) + return status; + accumulator = Xapian::Query (Xapian::Query::OP_OR, accumulator, subquery); + } + } + output = accumulator; + return NOTMUCH_STATUS_SUCCESS; + } + } + + /* Empty list */ + if (! sx->list) { + output = Xapian::Query::MatchAll; + return NOTMUCH_STATUS_SUCCESS; + } + + if (sx->list->ty == SEXP_LIST) { + _notmuch_database_log (notmuch, "unexpected list in field/operation position\n", + sx->list->val); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + status = maybe_saved_squery (notmuch, parent, env, sx, output); + if (status != NOTMUCH_STATUS_IGNORED) + return status; + + /* Check for user defined field */ + if (_notmuch_string_map_get (notmuch->user_prefix, sx->list->val)) { + return _sexp_parse_header (notmuch, parent, env, sx, output); + } + + if (strcmp (sx->list->val, "macro") == 0) { + _notmuch_database_log (notmuch, "macro definition not permitted here\n"); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + for (_sexp_prefix_t *prefix = prefixes; prefix && prefix->name; prefix++) { + if (strcmp (prefix->name, sx->list->val) == 0) { + if (prefix->flags & (SEXP_FLAG_FIELD | SEXP_FLAG_RANGE)) { + if (parent) { + _notmuch_database_log (notmuch, "nested field: '%s' inside '%s'\n", + prefix->name, parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + parent = prefix; + } + + if (parent && (prefix->flags & SEXP_FLAG_ORPHAN)) { + _notmuch_database_log (notmuch, "'%s' not supported inside '%s'\n", + prefix->name, parent->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if ((prefix->flags & SEXP_FLAG_SINGLE) && + (! sx->list->next || sx->list->next->next || sx->list->next->ty != SEXP_VALUE)) { + _notmuch_database_log (notmuch, "'%s' expects single atom as argument\n", + prefix->name); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + if (prefix->flags & SEXP_FLAG_RANGE) + return _sexp_parse_range (notmuch, prefix, sx->list->next, output); + + if (strcmp (prefix->name, "infix") == 0) { + return _sexp_parse_infix (notmuch, sx->list->next, output); + } + + if (strcmp (prefix->name, "query") == 0) { + return _notmuch_query_name_to_query (notmuch, sx->list->next->val, output); + } + + if (prefix->xapian_op == Xapian::Query::OP_WILDCARD) { + const char *str; + status = _sexp_expand_term (notmuch, prefix, env, sx->list->next, &str); + if (status) + return status; + + return _sexp_parse_wildcard (notmuch, parent, env, str, output); + } + + if (prefix->flags & SEXP_FLAG_DO_REGEX) { + return _sexp_parse_regex (notmuch, prefix, parent, env, sx->list->next, output); + } + + if (prefix->flags & SEXP_FLAG_DO_EXPAND) { + return _sexp_expand_query (notmuch, prefix, parent, env, sx->list->next, output); + } + + return _sexp_combine_query (notmuch, parent, env, prefix->xapian_op, prefix->initial, + sx->list->next, output); + } + } + + _notmuch_database_log (notmuch, "unknown prefix '%s'\n", sx->list->val); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; +} + +notmuch_status_t +_notmuch_sexp_string_to_xapian_query (notmuch_database_t *notmuch, const char *querystr, + Xapian::Query &output) +{ + const sexp_t *sx = NULL; + char *buf = talloc_strdup (notmuch, querystr); + + sx = parse_sexp (buf, strlen (querystr)); + if (! sx) { + _notmuch_database_log (notmuch, "invalid s-expression: '%s'\n", querystr); + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + + return _sexp_to_xapian_query (notmuch, NULL, NULL, sx, output); +} +#endif diff --git a/lib/parse-time-vrp.cc b/lib/parse-time-vrp.cc index 45db9597..6b07970b 100644 --- a/lib/parse-time-vrp.cc +++ b/lib/parse-time-vrp.cc @@ -24,66 +24,84 @@ #include "parse-time-vrp.h" #include "parse-time-string.h" -#define PREFIX "date:" - -/* See *ValueRangeProcessor in xapian-core/api/valuerangeproc.cc */ -Xapian::valueno -ParseTimeValueRangeProcessor::operator() (std::string &begin, std::string &end) +notmuch_status_t +_notmuch_date_strings_to_query (Xapian::valueno slot, + const std::string &begin, const std::string &end, + Xapian::Query &output, std::string &msg) { - time_t t, now; - std::string b; - - /* Require date: prefix in start of the range... */ - if (STRNCMP_LITERAL (begin.c_str (), PREFIX)) - return Xapian::BAD_VALUENO; - - /* ...and remove it. */ - begin.erase (0, sizeof (PREFIX) - 1); - b = begin; + double from = DBL_MIN, to = DBL_MAX; + time_t parsed_time, now; + std::string str; /* Use the same 'now' for begin and end. */ - if (time (&now) == (time_t) -1) - return Xapian::BAD_VALUENO; + if (time (&now) == (time_t) -1) { + msg = "unable to get current time"; + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + } if (! begin.empty ()) { - if (parse_time_string (begin.c_str (), &t, &now, PARSE_TIME_ROUND_DOWN)) - return Xapian::BAD_VALUENO; + if (parse_time_string (begin.c_str (), &parsed_time, &now, PARSE_TIME_ROUND_DOWN)) { + msg = "Didn't understand date specification '" + begin + "'"; + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } - begin.assign (Xapian::sortable_serialise ((double) t)); + from = (double) parsed_time; } if (! end.empty ()) { - if (end == "!" && ! b.empty ()) - end = b; + if (end == "!" && ! begin.empty ()) + str = begin; + else + str = end; + + if (parse_time_string (str.c_str (), &parsed_time, &now, PARSE_TIME_ROUND_UP_INCLUSIVE)) { + msg = "Didn't understand date specification '" + str + "'"; + return NOTMUCH_STATUS_BAD_QUERY_SYNTAX; + } + to = (double) parsed_time; + } - if (parse_time_string (end.c_str (), &t, &now, PARSE_TIME_ROUND_UP_INCLUSIVE)) - return Xapian::BAD_VALUENO; + output = Xapian::Query (Xapian::Query::OP_VALUE_RANGE, slot, + Xapian::sortable_serialise (from), + Xapian::sortable_serialise (to)); + return NOTMUCH_STATUS_SUCCESS; +} - end.assign (Xapian::sortable_serialise ((double) t)); - } +Xapian::Query +ParseTimeRangeProcessor::operator() (const std::string &begin, const std::string &end) +{ + + Xapian::Query output; + std::string msg; + + if (_notmuch_date_strings_to_query (slot, begin, end, output, msg)) + throw Xapian::QueryParserError (msg); - return valno; + return output; } -#if HAVE_XAPIAN_FIELD_PROCESSOR /* XXX TODO: is throwing an exception the right thing to do here? */ Xapian::Query DateFieldProcessor::operator() (const std::string & str) { - time_t from, to, now; + double from = DBL_MIN, to = DBL_MAX; + time_t parsed_time, now; /* Use the same 'now' for begin and end. */ if (time (&now) == (time_t) -1) throw Xapian::QueryParserError ("Unable to get current time"); - if (parse_time_string (str.c_str (), &from, &now, PARSE_TIME_ROUND_DOWN)) + if (parse_time_string (str.c_str (), &parsed_time, &now, PARSE_TIME_ROUND_DOWN)) throw Xapian::QueryParserError ("Didn't understand date specification '" + str + "'"); + else + from = (double) parsed_time; - if (parse_time_string (str.c_str (), &to, &now, PARSE_TIME_ROUND_UP_INCLUSIVE)) + if (parse_time_string (str.c_str (), &parsed_time, &now, PARSE_TIME_ROUND_UP_INCLUSIVE)) throw Xapian::QueryParserError ("Didn't understand date specification '" + str + "'"); + else + to = (double) parsed_time; - return Xapian::Query (Xapian::Query::OP_AND, - Xapian::Query (Xapian::Query::OP_VALUE_GE, 0, Xapian::sortable_serialise ((double) from)), - Xapian::Query (Xapian::Query::OP_VALUE_LE, 0, Xapian::sortable_serialise ((double) to))); + return Xapian::Query (Xapian::Query::OP_VALUE_RANGE, slot, + Xapian::sortable_serialise (from), + Xapian::sortable_serialise (to)); } -#endif diff --git a/lib/parse-time-vrp.h b/lib/parse-time-vrp.h index e6138d05..f495e716 100644 --- a/lib/parse-time-vrp.h +++ b/lib/parse-time-vrp.h @@ -26,22 +26,21 @@ #include /* see *ValueRangeProcessor in xapian-core/include/xapian/queryparser.h */ -class ParseTimeValueRangeProcessor : public Xapian::ValueRangeProcessor { -protected: - Xapian::valueno valno; +class ParseTimeRangeProcessor : public Xapian::RangeProcessor { public: - ParseTimeValueRangeProcessor (Xapian::valueno slot_) - : valno (slot_) - { - } + ParseTimeRangeProcessor (Xapian::valueno slot_, const std::string prefix_) + : Xapian::RangeProcessor(slot_, prefix_, 0) { } - Xapian::valueno operator() (std::string &begin, std::string &end); + Xapian::Query operator() (const std::string &begin, const std::string &end); }; -#if HAVE_XAPIAN_FIELD_PROCESSOR class DateFieldProcessor : public Xapian::FieldProcessor { - Xapian::Query operator() (const std::string & str); +private: + Xapian::valueno slot; +public: + DateFieldProcessor(Xapian::valueno slot_) : slot(slot_) { }; + Xapian::Query operator()(const std::string & str); }; -#endif + #endif /* NOTMUCH_PARSE_TIME_VRP_H */ diff --git a/lib/prefix.cc b/lib/prefix.cc new file mode 100644 index 00000000..06e2333a --- /dev/null +++ b/lib/prefix.cc @@ -0,0 +1,215 @@ +#include "database-private.h" +#include "query-fp.h" +#include "thread-fp.h" +#include "regexp-fields.h" +#include "parse-time-vrp.h" +#include "sexp-fp.h" + +typedef struct { + const char *name; + const char *prefix; + notmuch_field_flag_t flags; +} prefix_t; + +/* With these prefix values we follow the conventions published here: + * + * https://xapian.org/docs/omega/termprefixes.html + * + * as much as makes sense. Note that I took some liberty in matching + * the reserved prefix values to notmuch concepts, (for example, 'G' + * is documented as "newsGroup (or similar entity - e.g. a web forum + * name)", for which I think the thread is the closest analogue in + * notmuch. This in spite of the fact that we will eventually be + * storing mailing-list messages where 'G' for "mailing list name" + * might be even a closer analogue. I'm treating the single-character + * prefixes preferentially for core notmuch concepts (which will be + * nearly universal to all mail messages). + */ + +static const +prefix_t prefix_table[] = { + /* name term prefix flags */ + { "type", "T", NOTMUCH_FIELD_NO_FLAGS }, + { "reference", "XREFERENCE", NOTMUCH_FIELD_NO_FLAGS }, + { "replyto", "XREPLYTO", NOTMUCH_FIELD_NO_FLAGS }, + { "directory", "XDIRECTORY", NOTMUCH_FIELD_NO_FLAGS }, + { "file-direntry", "XFDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, + { "directory-direntry", "XDDIRENTRY", NOTMUCH_FIELD_NO_FLAGS }, + { "body", "", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "thread", "G", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "tag", "K", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "is", "K", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "id", "Q", NOTMUCH_FIELD_EXTERNAL }, + { "mid", "Q", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "path", "P", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR | NOTMUCH_FIELD_STRIP_TRAILING_SLASH }, + { "property", "XPROPERTY", NOTMUCH_FIELD_EXTERNAL }, + /* + * Unconditionally add ':' to reduce potential ambiguity with + * overlapping prefixes and/or terms that start with capital + * letters. See Xapian document termprefixes.html for related + * discussion. + */ + { "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR | NOTMUCH_FIELD_STRIP_TRAILING_SLASH }, + { "date", NULL, NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "query", NULL, NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "sexp", NULL, NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROCESSOR }, + { "from", "XFROM", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC | + NOTMUCH_FIELD_PROCESSOR }, + { "to", "XTO", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "attachment", "XATTACHMENT", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "mimetype", "XMIMETYPE", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC }, + { "subject", "XSUBJECT", NOTMUCH_FIELD_EXTERNAL | + NOTMUCH_FIELD_PROBABILISTIC | + NOTMUCH_FIELD_PROCESSOR }, +}; + +static const char * +_user_prefix (void *ctx, const char *name) +{ + return talloc_asprintf (ctx, "XU%s:", name); +} + +const char * +_find_prefix (const char *name) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { + if (strcmp (name, prefix_table[i].name) == 0) + return prefix_table[i].prefix; + } + + INTERNAL_ERROR ("No prefix exists for '%s'\n", name); + + return ""; +} + +/* Like find prefix, but include the possibility of user defined + * prefixes specific to this database */ + +const char * +_notmuch_database_prefix (notmuch_database_t *notmuch, const char *name) +{ + unsigned int i; + + /*XXX TODO: reduce code duplication */ + for (i = 0; i < ARRAY_SIZE (prefix_table); i++) { + if (strcmp (name, prefix_table[i].name) == 0) + return prefix_table[i].prefix; + } + + if (notmuch->user_prefix) + return _notmuch_string_map_get (notmuch->user_prefix, name); + + return NULL; +} + +static void +_setup_query_field_default (const prefix_t *prefix, notmuch_database_t *notmuch) +{ + if (prefix->prefix) + notmuch->query_parser->add_prefix ("", prefix->prefix); + if (prefix->flags & NOTMUCH_FIELD_PROBABILISTIC) + notmuch->query_parser->add_prefix (prefix->name, prefix->prefix); + else + notmuch->query_parser->add_boolean_prefix (prefix->name, prefix->prefix); +} + +static void +_setup_query_field (const prefix_t *prefix, notmuch_database_t *notmuch) +{ + if (prefix->flags & NOTMUCH_FIELD_PROCESSOR) { + Xapian::FieldProcessor *fp; + + if (STRNCMP_LITERAL (prefix->name, "date") == 0) + fp = (new DateFieldProcessor (NOTMUCH_VALUE_TIMESTAMP))->release (); + else if (STRNCMP_LITERAL (prefix->name, "query") == 0) + fp = (new QueryFieldProcessor (*notmuch->query_parser, notmuch))->release (); + else if (STRNCMP_LITERAL (prefix->name, "thread") == 0) + fp = (new ThreadFieldProcessor (*notmuch->query_parser, notmuch))->release (); + else if (STRNCMP_LITERAL (prefix->name, "sexp") == 0) + fp = (new SexpFieldProcessor (notmuch))->release (); + else + fp = (new RegexpFieldProcessor (prefix->name, prefix->flags, + *notmuch->query_parser, notmuch))->release (); + + /* we treat all field-processor fields as boolean in order to get the raw input */ + if (prefix->prefix) + notmuch->query_parser->add_prefix ("", prefix->prefix); + notmuch->query_parser->add_boolean_prefix (prefix->name, fp); + } else { + _setup_query_field_default (prefix, notmuch); + } +} + +notmuch_status_t +_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch) +{ + for (unsigned int i = 0; i < ARRAY_SIZE (prefix_table); i++) { + const prefix_t *prefix = &prefix_table[i]; + if (prefix->flags & NOTMUCH_FIELD_EXTERNAL) { + _setup_query_field (prefix, notmuch); + } + } + return NOTMUCH_STATUS_SUCCESS; +} + +notmuch_status_t +_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch) +{ + notmuch_string_map_iterator_t *list; + + notmuch->user_prefix = _notmuch_string_map_create (notmuch); + if (notmuch->user_prefix == NULL) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + notmuch->user_header = _notmuch_string_map_create (notmuch); + if (notmuch->user_header == NULL) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + list = _notmuch_string_map_iterator_create (notmuch->config, CONFIG_HEADER_PREFIX, FALSE); + if (! list) + INTERNAL_ERROR ("unable to read headers from configuration"); + + for (; _notmuch_string_map_iterator_valid (list); + _notmuch_string_map_iterator_move_to_next (list)) { + + prefix_t query_field; + + const char *key = _notmuch_string_map_iterator_key (list) + + sizeof (CONFIG_HEADER_PREFIX) - 1; + + _notmuch_string_map_append (notmuch->user_prefix, + key, + _user_prefix (notmuch, key)); + + _notmuch_string_map_append (notmuch->user_header, + key, + _notmuch_string_map_iterator_value (list)); + + query_field.name = talloc_strdup (notmuch, key); + query_field.prefix = _user_prefix (notmuch, key); + query_field.flags = NOTMUCH_FIELD_PROBABILISTIC + | NOTMUCH_FIELD_EXTERNAL; + + _setup_query_field_default (&query_field, notmuch); + } + + _notmuch_string_map_iterator_destroy (list); + + return NOTMUCH_STATUS_SUCCESS; +} diff --git a/lib/query-fp.cc b/lib/query-fp.cc index c39f5915..75b1d875 100644 --- a/lib/query-fp.cc +++ b/lib/query-fp.cc @@ -24,20 +24,33 @@ #include "query-fp.h" #include -#if HAVE_XAPIAN_FIELD_PROCESSOR - -Xapian::Query -QueryFieldProcessor::operator() (const std::string & name) +notmuch_status_t +_notmuch_query_name_to_query (notmuch_database_t *notmuch, const std::string name, + Xapian::Query &output) { std::string key = "query." + name; char *expansion; notmuch_status_t status; status = notmuch_database_get_config (notmuch, key.c_str (), &expansion); + if (status) + return status; + + output = notmuch->query_parser->parse_query (expansion, NOTMUCH_QUERY_PARSER_FLAGS); + return NOTMUCH_STATUS_SUCCESS; +} + +Xapian::Query +QueryFieldProcessor::operator() (const std::string & name) +{ + notmuch_status_t status; + Xapian::Query output; + + status = _notmuch_query_name_to_query (notmuch, name, output); if (status) { throw Xapian::QueryParserError ("error looking up key" + name); } - return parser.parse_query (expansion, NOTMUCH_QUERY_PARSER_FLAGS); + return output; + } -#endif diff --git a/lib/query-fp.h b/lib/query-fp.h index 8a8bde62..beaaf405 100644 --- a/lib/query-fp.h +++ b/lib/query-fp.h @@ -26,7 +26,6 @@ #include #include "notmuch.h" -#if HAVE_XAPIAN_FIELD_PROCESSOR class QueryFieldProcessor : public Xapian::FieldProcessor { protected: Xapian::QueryParser &parser; @@ -40,5 +39,5 @@ public: Xapian::Query operator() (const std::string & str); }; -#endif + #endif /* NOTMUCH_QUERY_FP_H */ diff --git a/lib/query.cc b/lib/query.cc index 792aba21..1c60c122 100644 --- a/lib/query.cc +++ b/lib/query.cc @@ -20,6 +20,7 @@ #include "notmuch-private.h" #include "database-private.h" +#include "xapian-extra.h" #include /* GHashTable, GPtrArray */ @@ -30,6 +31,7 @@ struct _notmuch_query { notmuch_string_list_t *exclude_terms; notmuch_exclude_t omit_excluded; bool parsed; + notmuch_query_syntax_t syntax; Xapian::Query xapian_query; std::set terms; }; @@ -84,9 +86,9 @@ _notmuch_query_destructor (notmuch_query_t *query) return 0; } -notmuch_query_t * -notmuch_query_create (notmuch_database_t *notmuch, - const char *query_string) +static notmuch_query_t * +_notmuch_query_constructor (notmuch_database_t *notmuch, + const char *query_string) { notmuch_query_t *query; @@ -105,7 +107,10 @@ notmuch_query_create (notmuch_database_t *notmuch, query->notmuch = notmuch; - query->query_string = talloc_strdup (query, query_string); + if (query_string) + query->query_string = talloc_strdup (query, query_string); + else + query->query_string = NULL; query->sort = NOTMUCH_SORT_NEWEST_FIRST; @@ -116,44 +121,146 @@ notmuch_query_create (notmuch_database_t *notmuch, return query; } -static notmuch_status_t -_notmuch_query_ensure_parsed (notmuch_query_t *query) +notmuch_query_t * +notmuch_query_create (notmuch_database_t *notmuch, + const char *query_string) { - if (query->parsed) - return NOTMUCH_STATUS_SUCCESS; - try { - query->xapian_query = - query->notmuch->query_parser-> - parse_query (query->query_string, NOTMUCH_QUERY_PARSER_FLAGS); + notmuch_query_t *query; + notmuch_status_t status; - /* Xapian doesn't support skip_to on terms from a query since - * they are unordered, so cache a copy of all terms in - * something searchable. - */ + status = notmuch_query_create_with_syntax (notmuch, query_string, + NOTMUCH_QUERY_SYNTAX_XAPIAN, + &query); + if (status) + return NULL; + + return query; +} + +notmuch_status_t +notmuch_query_create_with_syntax (notmuch_database_t *notmuch, + const char *query_string, + notmuch_query_syntax_t syntax, + notmuch_query_t **output) +{ + + notmuch_query_t *query; + + if (! output) + return NOTMUCH_STATUS_NULL_POINTER; - for (Xapian::TermIterator t = query->xapian_query.get_terms_begin (); - t != query->xapian_query.get_terms_end (); ++t) - query->terms.insert (*t); + query = _notmuch_query_constructor (notmuch, query_string); + if (! query) + return NOTMUCH_STATUS_OUT_OF_MEMORY; + + if (syntax == NOTMUCH_QUERY_SYNTAX_SEXP && ! HAVE_SFSEXP) { + _notmuch_database_log (notmuch, "sexp query parser not available"); + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; + } - query->parsed = true; + query->syntax = syntax; + *output = query; + + return NOTMUCH_STATUS_SUCCESS; +} + +static void +_notmuch_query_cache_terms (notmuch_query_t *query) +{ + /* Xapian doesn't support skip_to on terms from a query since + * they are unordered, so cache a copy of all terms in + * something searchable. + */ + + for (Xapian::TermIterator t = query->xapian_query.get_terms_begin (); + t != query->xapian_query.get_terms_end (); ++t) + query->terms.insert (*t); +} + +notmuch_status_t +_notmuch_query_string_to_xapian_query (notmuch_database_t *notmuch, + std::string query_string, + Xapian::Query &output, + std::string &msg) +{ + try { + if (query_string == "" || query_string == "*") { + output = xapian_query_match_all (); + } else { + output = + notmuch->query_parser-> + parse_query (query_string, NOTMUCH_QUERY_PARSER_FLAGS); + } } catch (const Xapian::Error &error) { - if (! query->notmuch->exception_reported) { - _notmuch_database_log (query->notmuch, + if (! notmuch->exception_reported) { + _notmuch_database_log (notmuch, "A Xapian exception occurred parsing query: %s\n", error.get_msg ().c_str ()); - _notmuch_database_log_append (query->notmuch, + _notmuch_database_log_append (notmuch, "Query string was: %s\n", - query->query_string); - query->notmuch->exception_reported = true; + query_string.c_str ()); + notmuch->exception_reported = true; } + msg = error.get_msg (); return NOTMUCH_STATUS_XAPIAN_EXCEPTION; } return NOTMUCH_STATUS_SUCCESS; } +static notmuch_status_t +_notmuch_query_ensure_parsed_xapian (notmuch_query_t *query) +{ + notmuch_status_t status; + std::string msg; /* ignored */ + + status = _notmuch_query_string_to_xapian_query (query->notmuch, query->query_string, + query->xapian_query, msg); + if (status) + return status; + + query->parsed = true; + + _notmuch_query_cache_terms (query); + + return NOTMUCH_STATUS_SUCCESS; +} + +#if HAVE_SFSEXP +static notmuch_status_t +_notmuch_query_ensure_parsed_sexpr (notmuch_query_t *query) +{ + notmuch_status_t status; + + if (query->parsed) + return NOTMUCH_STATUS_SUCCESS; + + status = _notmuch_sexp_string_to_xapian_query (query->notmuch, query->query_string, + query->xapian_query); + if (status) + return status; + + _notmuch_query_cache_terms (query); + return NOTMUCH_STATUS_SUCCESS; +} +#endif + +static notmuch_status_t +_notmuch_query_ensure_parsed (notmuch_query_t *query) +{ + if (query->parsed) + return NOTMUCH_STATUS_SUCCESS; + +#if HAVE_SFSEXP + if (query->syntax == NOTMUCH_QUERY_SYNTAX_SEXP) + return _notmuch_query_ensure_parsed_sexpr (query); +#endif + + return _notmuch_query_ensure_parsed_xapian (query); +} + const char * notmuch_query_get_query_string (const notmuch_query_t *query) { @@ -249,7 +356,6 @@ _notmuch_query_search_documents (notmuch_query_t *query, notmuch_messages_t **out) { notmuch_database_t *notmuch = query->notmuch; - const char *query_string = query->query_string; notmuch_mset_messages_t *messages; notmuch_status_t status; @@ -279,13 +385,9 @@ _notmuch_query_search_documents (notmuch_query_t *query, Xapian::MSet mset; Xapian::MSetIterator iterator; - if (strcmp (query_string, "") == 0 || - strcmp (query_string, "*") == 0) { - final_query = mail_query; - } else { - final_query = Xapian::Query (Xapian::Query::OP_AND, - mail_query, query->xapian_query); - } + final_query = Xapian::Query (Xapian::Query::OP_AND, + mail_query, query->xapian_query); + messages->base.excluded_doc_ids = NULL; if ((query->omit_excluded != NOTMUCH_EXCLUDE_FALSE) && (query->exclude_terms)) { @@ -606,7 +708,6 @@ notmuch_status_t _notmuch_query_count_documents (notmuch_query_t *query, const char *type, unsigned *count_out) { notmuch_database_t *notmuch = query->notmuch; - const char *query_string = query->query_string; Xapian::doccount count = 0; notmuch_status_t status; @@ -622,13 +723,8 @@ _notmuch_query_count_documents (notmuch_query_t *query, const char *type, unsign Xapian::Query final_query, exclude_query; Xapian::MSet mset; - if (strcmp (query_string, "") == 0 || - strcmp (query_string, "*") == 0) { - final_query = mail_query; - } else { - final_query = Xapian::Query (Xapian::Query::OP_AND, - mail_query, query->xapian_query); - } + final_query = Xapian::Query (Xapian::Query::OP_AND, + mail_query, query->xapian_query); exclude_query = _notmuch_exclude_tags (query); @@ -728,3 +824,51 @@ notmuch_query_get_database (const notmuch_query_t *query) { return query->notmuch; } + +notmuch_status_t +_notmuch_query_expand (notmuch_database_t *notmuch, const char *field, Xapian::Query subquery, + Xapian::Query &output, std::string &msg) +{ + std::set terms; + const std::string term_prefix = _find_prefix (field); + + if (_debug_query ()) { + fprintf (stderr, "Expanding subquery:\n%s\n", + subquery.get_description ().c_str ()); + } + + try { + Xapian::Enquire enquire (*notmuch->xapian_db); + Xapian::MSet mset; + + enquire.set_weighting_scheme (Xapian::BoolWeight ()); + enquire.set_query (subquery); + + mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ()); + + for (Xapian::MSetIterator iterator = mset.begin (); iterator != mset.end (); iterator++) { + Xapian::docid doc_id = *iterator; + Xapian::Document doc = notmuch->xapian_db->get_document (doc_id); + Xapian::TermIterator i = doc.termlist_begin (); + + for (i.skip_to (term_prefix); + i != doc.termlist_end () && ((*i).rfind (term_prefix, 0) == 0); i++) { + terms.insert (*i); + } + } + output = Xapian::Query (Xapian::Query::OP_OR, terms.begin (), terms.end ()); + if (_debug_query ()) { + fprintf (stderr, "Expanded query:\n%s\n", + subquery.get_description ().c_str ()); + } + + } catch (const Xapian::Error &error) { + _notmuch_database_log (notmuch, + "A Xapian exception occurred expanding query: %s\n", + error.get_msg ().c_str ()); + msg = error.get_msg (); + return NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } + + return NOTMUCH_STATUS_SUCCESS; +} diff --git a/lib/regexp-fields.cc b/lib/regexp-fields.cc index 198eb32f..3a775261 100644 --- a/lib/regexp-fields.cc +++ b/lib/regexp-fields.cc @@ -25,29 +25,34 @@ #include "regexp-fields.h" #include "notmuch-private.h" #include "database-private.h" +#include "xapian-extra.h" -#if HAVE_XAPIAN_FIELD_PROCESSOR -static void -compile_regex (regex_t ®exp, const char *str) +notmuch_status_t +compile_regex (regex_t ®exp, const char *str, std::string &msg) { int err = regcomp (®exp, str, REG_EXTENDED | REG_NOSUB); if (err != 0) { size_t len = regerror (err, ®exp, NULL, 0); char *buffer = new char[len]; - std::string msg = "Regexp error: "; + msg = "Regexp error: "; (void) regerror (err, ®exp, buffer, len); msg.append (buffer, len); delete[] buffer; - throw Xapian::QueryParserError (msg); + return NOTMUCH_STATUS_ILLEGAL_ARGUMENT; } + return NOTMUCH_STATUS_SUCCESS; } RegexpPostingSource::RegexpPostingSource (Xapian::valueno slot, const std::string ®exp) : slot_ (slot) { - compile_regex (regexp_, regexp.c_str ()); + std::string msg; + notmuch_status_t status = compile_regex (regexp_, regexp.c_str (), msg); + + if (status) + throw Xapian::QueryParserError (msg); } RegexpPostingSource::~RegexpPostingSource () @@ -142,25 +147,61 @@ _find_slot (std::string prefix) return Xapian::BAD_VALUENO; } -RegexpFieldProcessor::RegexpFieldProcessor (std::string prefix, +RegexpFieldProcessor::RegexpFieldProcessor (std::string field_, notmuch_field_flag_t options_, Xapian::QueryParser &parser_, notmuch_database_t *notmuch_) - : slot (_find_slot (prefix)), - term_prefix (_find_prefix (prefix.c_str ())), + : slot (_find_slot (field_)), + field (field_), + term_prefix (_find_prefix (field_.c_str ())), options (options_), parser (parser_), notmuch (notmuch_) { }; +notmuch_status_t +_notmuch_regexp_to_query (notmuch_database_t *notmuch, Xapian::valueno slot, std::string field, + std::string regexp_str, + Xapian::Query &output, std::string &msg) +{ + regex_t regexp; + notmuch_status_t status; + + status = compile_regex (regexp, regexp_str.c_str (), msg); + if (status) { + _notmuch_database_log_append (notmuch, "error compiling regex %s", msg.c_str ()); + return status; + } + + if (slot == Xapian::BAD_VALUENO) + slot = _find_slot (field); + + if (slot == Xapian::BAD_VALUENO) { + std::string term_prefix = _find_prefix (field.c_str ()); + std::vector terms; + + for (Xapian::TermIterator it = notmuch->xapian_db->allterms_begin (term_prefix); + it != notmuch->xapian_db->allterms_end (); ++it) { + if (regexec (®exp, (*it).c_str () + term_prefix.size (), + 0, NULL, 0) == 0) + terms.push_back (*it); + } + output = Xapian::Query (Xapian::Query::OP_OR, terms.begin (), terms.end ()); + } else { + RegexpPostingSource *postings = new RegexpPostingSource (slot, regexp_str); + output = Xapian::Query (postings->release ()); + } + return NOTMUCH_STATUS_SUCCESS; +} + Xapian::Query RegexpFieldProcessor::operator() (const std::string & str) { if (str.empty ()) { if (options & NOTMUCH_FIELD_PROBABILISTIC) { return Xapian::Query (Xapian::Query::OP_AND_NOT, - Xapian::Query::MatchAll, + xapian_query_match_all (), Xapian::Query (Xapian::Query::OP_WILDCARD, term_prefix)); } else { return Xapian::Query (term_prefix); @@ -169,23 +210,15 @@ RegexpFieldProcessor::operator() (const std::string & str) if (str.at (0) == '/') { if (str.length () > 1 && str.at (str.size () - 1) == '/') { + Xapian::Query query; std::string regexp_str = str.substr (1, str.size () - 2); - if (slot != Xapian::BAD_VALUENO) { - RegexpPostingSource *postings = new RegexpPostingSource (slot, regexp_str); - return Xapian::Query (postings->release ()); - } else { - std::vector terms; - regex_t regexp; - - compile_regex (regexp, regexp_str.c_str ()); - for (Xapian::TermIterator it = notmuch->xapian_db->allterms_begin (term_prefix); - it != notmuch->xapian_db->allterms_end (); ++it) { - if (regexec (®exp, (*it).c_str () + term_prefix.size (), - 0, NULL, 0) == 0) - terms.push_back (*it); - } - return Xapian::Query (Xapian::Query::OP_OR, terms.begin (), terms.end ()); - } + std::string msg; + notmuch_status_t status; + + status = _notmuch_regexp_to_query (notmuch, slot, field, regexp_str, query, msg); + if (status) + throw Xapian::QueryParserError (msg); + return query; } else { throw Xapian::QueryParserError ("unmatched regex delimiter in '" + str + "'"); } @@ -195,7 +228,8 @@ RegexpFieldProcessor::operator() (const std::string & str) * phrase parsing, when possible */ std::string query_str; - if (*str.rbegin () != '*' || str.find (' ') != std::string::npos) + if ((str.at (0) != '(' || *str.rbegin () != ')') && + (*str.rbegin () != '*' || str.find (' ') != std::string::npos)) query_str = '"' + str + '"'; else query_str = str; @@ -203,9 +237,16 @@ RegexpFieldProcessor::operator() (const std::string & str) return parser.parse_query (query_str, NOTMUCH_QUERY_PARSER_FLAGS, term_prefix); } else { /* Boolean prefix */ - std::string term = term_prefix + str; + std::string query_str; + std::string term; + + if (str.length () > 1 && str.at (str.size () - 1) == '/') + query_str = str.substr (0, str.size () - 1); + else + query_str = str; + + term = term_prefix + query_str; return Xapian::Query (term); } } } -#endif diff --git a/lib/regexp-fields.h b/lib/regexp-fields.h index 97778a1d..9c871de7 100644 --- a/lib/regexp-fields.h +++ b/lib/regexp-fields.h @@ -24,12 +24,17 @@ #ifndef NOTMUCH_REGEXP_FIELDS_H #define NOTMUCH_REGEXP_FIELDS_H -#if HAVE_XAPIAN_FIELD_PROCESSOR + #include #include #include "database-private.h" #include "notmuch-private.h" +notmuch_status_t +_notmuch_regex_to_query (notmuch_database_t *notmuch, Xapian::valueno slot, std::string field, + std::string regexp_str, + Xapian::Query &output, std::string &msg); + /* A posting source that returns documents where a value matches a * regexp. */ @@ -64,6 +69,7 @@ public: class RegexpFieldProcessor : public Xapian::FieldProcessor { protected: Xapian::valueno slot; + std::string field; std::string term_prefix; notmuch_field_flag_t options; Xapian::QueryParser &parser; @@ -79,5 +85,5 @@ public: Xapian::Query operator() (const std::string & str); }; -#endif + #endif /* NOTMUCH_REGEXP_FIELDS_H */ diff --git a/lib/sexp-fp.cc b/lib/sexp-fp.cc new file mode 100644 index 00000000..1fdf5225 --- /dev/null +++ b/lib/sexp-fp.cc @@ -0,0 +1,44 @@ +/* sexp-fp.cc - "sexp:" field processor glue + * + * This file is part of notmuch. + * + * Copyright © 2022 David Bremner + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see https://www.gnu.org/licenses/ . + * + * Author: David Bremner + */ + +#include "database-private.h" +#include "sexp-fp.h" +#include + +Xapian::Query +SexpFieldProcessor::operator() (const std::string & query_string) +{ + notmuch_status_t status; + Xapian::Query output; + +#if HAVE_SFSEXP + status = _notmuch_sexp_string_to_xapian_query (notmuch, query_string.c_str (), output); + if (status) { + throw Xapian::QueryParserError ("error parsing " + query_string); + } +#else + throw Xapian::QueryParserError ("sexp query parser not available"); +#endif + + return output; + +} diff --git a/lib/sexp-fp.h b/lib/sexp-fp.h new file mode 100644 index 00000000..341dfa7e --- /dev/null +++ b/lib/sexp-fp.h @@ -0,0 +1,41 @@ +/* sexp-fp.h - sexp field processor glue + * + * This file is part of notmuch. + * + * Copyright © 2022 David Bremner + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see https://www.gnu.org/licenses/ . + * + * Author: David Bremner + */ + +#ifndef NOTMUCH_SEXP_FP_H +#define NOTMUCH_SEXP_FP_H + +#include +#include "notmuch.h" + +class SexpFieldProcessor : public Xapian::FieldProcessor { +protected: + notmuch_database_t *notmuch; + +public: + SexpFieldProcessor (notmuch_database_t *notmuch_) : notmuch (notmuch_) + { + }; + + Xapian::Query operator() (const std::string & query_string); +}; + +#endif /* NOTMUCH_SEXP_FP_H */ diff --git a/lib/string-map.c b/lib/string-map.c index a88404c7..99bc2ea2 100644 --- a/lib/string-map.c +++ b/lib/string-map.c @@ -86,10 +86,14 @@ _notmuch_string_map_append (notmuch_string_map_t *map, static int cmppair (const void *pa, const void *pb) { + int cmp = 0; notmuch_string_pair_t *a = (notmuch_string_pair_t *) pa; notmuch_string_pair_t *b = (notmuch_string_pair_t *) pb; - return strcmp (a->key, b->key); + cmp = strcmp (a->key, b->key); + if (cmp == 0) + cmp = strcmp (a->value, b->value); + return cmp; } static void @@ -143,6 +147,24 @@ bsearch_first (notmuch_string_pair_t *array, size_t len, const char *key, bool e } +void +_notmuch_string_map_set (notmuch_string_map_t *map, + const char *key, + const char *val) +{ + notmuch_string_pair_t *pair; + + /* this means that calling string_map_set invalidates iterators */ + _notmuch_string_map_sort (map); + pair = bsearch_first (map->pairs, map->length, key, true); + if (! pair) + _notmuch_string_map_append (map, key, val); + else { + talloc_free (pair->value); + pair->value = talloc_strdup (map->pairs, val); + } +} + const char * _notmuch_string_map_get (notmuch_string_map_t *map, const char *key) { diff --git a/lib/tags.c b/lib/tags.c index c7d3f66f..ec5366ff 100644 --- a/lib/tags.c +++ b/lib/tags.c @@ -48,7 +48,7 @@ _notmuch_tags_create (const void *ctx, notmuch_string_list_t *list) notmuch_bool_t notmuch_tags_valid (notmuch_tags_t *tags) { - return tags->iterator != NULL; + return tags && (tags->iterator != NULL); } const char * diff --git a/lib/thread-fp.cc b/lib/thread-fp.cc index 73277006..3aa9c423 100644 --- a/lib/thread-fp.cc +++ b/lib/thread-fp.cc @@ -24,8 +24,6 @@ #include "thread-fp.h" #include -#if HAVE_XAPIAN_FIELD_PROCESSOR - Xapian::Query ThreadFieldProcessor::operator() (const std::string & str) { @@ -36,26 +34,20 @@ ThreadFieldProcessor::operator() (const std::string & str) if (str.size () <= 1 || str.at (str.size () - 1) != '}') { throw Xapian::QueryParserError ("missing } in '" + str + "'"); } else { + Xapian::Query subquery; + Xapian::Query query; + std::string msg; std::string subquery_str = str.substr (1, str.size () - 2); - notmuch_query_t *subquery = notmuch_query_create (notmuch, subquery_str.c_str ()); - notmuch_messages_t *messages; - std::set terms; - if (! subquery) - throw Xapian::QueryParserError ("failed to create subquery for '" + subquery_str + "'"); + status = _notmuch_query_string_to_xapian_query (notmuch, subquery_str, subquery, msg); + if (status) + throw Xapian::QueryParserError (msg); - status = notmuch_query_search_messages (subquery, &messages); + status = _notmuch_query_expand (notmuch, "thread", subquery, query, msg); if (status) - throw Xapian::QueryParserError ("failed to search messages for '" + subquery_str + "'"); + throw Xapian::QueryParserError (msg); - for (; notmuch_messages_valid (messages); notmuch_messages_move_to_next (messages)) { - std::string term = thread_prefix; - notmuch_message_t *message; - message = notmuch_messages_get (messages); - term += _notmuch_message_get_thread_id_only (message); - terms.insert (term); - } - return Xapian::Query (Xapian::Query::OP_OR, terms.begin (), terms.end ()); + return query; } } else { /* literal thread id */ @@ -64,4 +56,3 @@ ThreadFieldProcessor::operator() (const std::string & str) } } -#endif diff --git a/lib/thread-fp.h b/lib/thread-fp.h index de837d3e..00bf1aa2 100644 --- a/lib/thread-fp.h +++ b/lib/thread-fp.h @@ -26,7 +26,6 @@ #include #include "notmuch.h" -#if HAVE_XAPIAN_FIELD_PROCESSOR class ThreadFieldProcessor : public Xapian::FieldProcessor { protected: Xapian::QueryParser &parser; @@ -40,5 +39,5 @@ public: Xapian::Query operator() (const std::string & str); }; -#endif + #endif /* NOTMUCH_THREAD_FP_H */ diff --git a/lib/thread.cc b/lib/thread.cc index 6073e45c..60e9a666 100644 --- a/lib/thread.cc +++ b/lib/thread.cc @@ -25,7 +25,8 @@ #include /* GHashTable */ #ifdef DEBUG_THREADING -#define THREAD_DEBUG(format, ...) fprintf (stderr, format " (%s).\n", ##__VA_ARGS__, __location__) +#define THREAD_DEBUG(format, ...) fprintf (stderr, "DT: " format " (%s).\n", ##__VA_ARGS__, \ + __location__) #else #define THREAD_DEBUG(format, ...) do {} while (0) /* ignored */ #endif @@ -351,14 +352,16 @@ _thread_set_subject_from_message (notmuch_thread_t *thread, /* Add a message to this thread which is known to match the original * search specification. The 'sort' parameter controls whether the * oldest or newest matching subject is applied to the thread as a - * whole. */ -static void + * whole. Returns 0 on success. + */ +static int _thread_add_matched_message (notmuch_thread_t *thread, notmuch_message_t *message, notmuch_sort_t sort) { time_t date; notmuch_message_t *hashed_message; + notmuch_bool_t is_set; date = notmuch_message_get_date (message); @@ -375,7 +378,9 @@ _thread_add_matched_message (notmuch_thread_t *thread, _thread_set_subject_from_message (thread, message); } - if (! notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED)) + if (notmuch_message_get_flag_st (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED, &is_set)) + return -1; + if (! is_set) thread->matched_messages++; if (g_hash_table_lookup_extended (thread->message_hash, @@ -386,6 +391,7 @@ _thread_add_matched_message (notmuch_thread_t *thread, } _thread_add_matched_author (thread, _notmuch_message_get_author (hashed_message)); + return 0; } static bool @@ -395,7 +401,7 @@ _parent_via_in_reply_to (notmuch_thread_t *thread, notmuch_message_t *message) const char *in_reply_to; in_reply_to = _notmuch_message_get_in_reply_to (message); - THREAD_DEBUG ("checking message = %s in_reply_to=%s\n", + THREAD_DEBUG ("checking message = %s in_reply_to=%s", notmuch_message_get_message_id (message), in_reply_to); if (in_reply_to && (! EMPTY_STRING (in_reply_to)) && @@ -418,31 +424,31 @@ _parent_or_toplevel (notmuch_thread_t *thread, notmuch_message_t *message) const notmuch_string_list_t *references = _notmuch_message_get_references (message); - THREAD_DEBUG ("trying to reparent via references: %s\n", + THREAD_DEBUG ("trying to reparent via references: %s", notmuch_message_get_message_id (message)); for (notmuch_string_node_t *ref_node = references->head; ref_node; ref_node = ref_node->next) { - THREAD_DEBUG ("checking reference=%s\n", ref_node->string); + THREAD_DEBUG ("checking reference=%s", ref_node->string); if ((g_hash_table_lookup_extended (thread->message_hash, ref_node->string, NULL, (void **) &new_parent))) { size_t new_depth = _notmuch_message_get_thread_depth (new_parent); - THREAD_DEBUG ("got depth %lu\n", new_depth); + THREAD_DEBUG ("got depth %lu", new_depth); if (new_depth > max_depth || ! parent) { - THREAD_DEBUG ("adding at depth %lu parent=%s\n", new_depth, ref_node->string); + THREAD_DEBUG ("adding at depth %lu parent=%s", new_depth, ref_node->string); max_depth = new_depth; parent = new_parent; } } } if (parent) { - THREAD_DEBUG ("adding reply %s to parent=%s\n", + THREAD_DEBUG ("adding reply %s to parent=%s", notmuch_message_get_message_id (message), notmuch_message_get_message_id (parent)); _notmuch_message_add_reply (parent, message); } else { - THREAD_DEBUG ("adding as toplevel %s\n", + THREAD_DEBUG ("adding as toplevel %s", notmuch_message_get_message_id (message)); _notmuch_message_list_add_message (thread->toplevel_list, message); } @@ -477,13 +483,13 @@ _resolve_thread_relationships (notmuch_thread_t *thread) */ if (first_node) { message = first_node->message; - THREAD_DEBUG ("checking first message %s\n", + THREAD_DEBUG ("checking first message %s", notmuch_message_get_message_id (message)); if (_notmuch_message_list_empty (maybe_toplevel_list) || ! _parent_via_in_reply_to (thread, message)) { - THREAD_DEBUG ("adding first message as toplevel = %s\n", + THREAD_DEBUG ("adding first message as toplevel = %s", notmuch_message_get_message_id (message)); _notmuch_message_list_add_message (maybe_toplevel_list, message); } @@ -500,7 +506,8 @@ _resolve_thread_relationships (notmuch_thread_t *thread) notmuch_messages_valid (roots); notmuch_messages_move_to_next (roots)) { notmuch_message_t *message = notmuch_messages_get (roots); - if (_notmuch_messages_has_next (roots) || ! _notmuch_message_list_empty (thread->toplevel_list)) + if (_notmuch_messages_has_next (roots) || ! _notmuch_message_list_empty ( + thread->toplevel_list)) _parent_or_toplevel (thread, message); else _notmuch_message_list_add_message (thread->toplevel_list, message); @@ -625,7 +632,10 @@ _notmuch_thread_create (void *ctx, if ( _notmuch_doc_id_set_contains (match_set, doc_id)) { _notmuch_doc_id_set_remove (match_set, doc_id); - _thread_add_matched_message (thread, message, sort); + if (_thread_add_matched_message (thread, message, sort)) { + thread = NULL; + goto DONE; + } } _notmuch_message_close (message); diff --git a/mime-node.c b/mime-node.c index d4996a33..1c5d619b 100644 --- a/mime-node.c +++ b/mime-node.c @@ -78,13 +78,14 @@ mime_node_get_message_crypto_status (mime_node_t *node) notmuch_status_t mime_node_open (const void *ctx, notmuch_message_t *message, + int duplicate, _notmuch_crypto_t *crypto, mime_node_t **root_out) { const char *filename = notmuch_message_get_filename (message); mime_node_context_t *mctx; mime_node_t *root; notmuch_status_t status; - int fd; + int fd = -1; root = talloc_zero (ctx, mime_node_t); if (root == NULL) { @@ -103,20 +104,33 @@ mime_node_open (const void *ctx, notmuch_message_t *message, talloc_set_destructor (mctx, _mime_node_context_free); /* Fast path */ - fd = open (filename, O_RDONLY); + if (duplicate <= 0) + fd = open (filename, O_RDONLY); if (fd == -1) { - /* Slow path - for some reason the first file in the list is - * not available anymore. This is clearly a problem in the + /* Slow path - Either we are trying to open a specific file, or + * for some reason the first file in the list is + * not available anymore. The latter is clearly a problem in the * database, but we are not going to let this problem be a * show stopper */ notmuch_filenames_t *filenames; + int i = 1; + for (filenames = notmuch_message_get_filenames (message); notmuch_filenames_valid (filenames); - notmuch_filenames_move_to_next (filenames)) { - filename = notmuch_filenames_get (filenames); - fd = open (filename, O_RDONLY); - if (fd != -1) - break; + notmuch_filenames_move_to_next (filenames), i++) { + if (i >= duplicate) { + filename = notmuch_filenames_get (filenames); + fd = open (filename, O_RDONLY); + if (fd != -1) { + break; + } else { + if (duplicate > 0) { + fprintf (stderr, "Error opening %s: %s\n", filename, strerror (errno)); + status = NOTMUCH_STATUS_FILE_ERROR; + goto DONE; + } + } + } } talloc_free (filenames); @@ -192,6 +206,26 @@ set_signature_list_destructor (mime_node_t *node) } } +/* Unwrapped MIME part destructor */ +static int +_unwrapped_child_free (GMimeObject **proxy) +{ + g_object_unref (*proxy); + return 0; +} + +/* Set up unwrapped MIME part destructor */ +static void +set_unwrapped_child_destructor (mime_node_t *node) +{ + GMimeObject **proxy = talloc (node, GMimeObject *); + + if (proxy) { + *proxy = node->unwrapped_child; + talloc_set_destructor (proxy, _unwrapped_child_free); + } +} + /* Verify a signed mime node */ static void node_verify (mime_node_t *node, GMimeObject *part) @@ -200,8 +234,17 @@ node_verify (mime_node_t *node, GMimeObject *part) notmuch_status_t status; node->verify_attempted = true; - node->sig_list = g_mime_multipart_signed_verify ( - GMIME_MULTIPART_SIGNED (part), GMIME_ENCRYPT_NONE, &err); + if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) + node->sig_list = g_mime_application_pkcs7_mime_verify ( + GMIME_APPLICATION_PKCS7_MIME (part), GMIME_VERIFY_NONE, &node->unwrapped_child, &err); + else + node->sig_list = g_mime_multipart_signed_verify ( + GMIME_MULTIPART_SIGNED (part), GMIME_VERIFY_NONE, &err); + + if (node->unwrapped_child) { + node->nchildren = 1; + set_unwrapped_child_destructor (node); + } if (node->sig_list) set_signature_list_destructor (node); @@ -214,7 +257,8 @@ node_verify (mime_node_t *node, GMimeObject *part) status = _notmuch_message_crypto_potential_sig_list (node->ctx->msg_crypto, node->sig_list); if (status) /* this is a warning, not an error */ - fprintf (stderr, "Warning: failed to note signature status: %s.\n", notmuch_status_to_string (status)); + fprintf (stderr, "Warning: failed to note signature status: %s.\n", notmuch_status_to_string ( + status)); } /* Decrypt and optionally verify an encrypted mime node */ @@ -224,22 +268,23 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject *part) GError *err = NULL; GMimeDecryptResult *decrypt_result = NULL; notmuch_status_t status; - GMimeMultipartEncrypted *encrypteddata = GMIME_MULTIPART_ENCRYPTED (part); notmuch_message_t *message = NULL; - if (! node->decrypted_child) { + if (! node->unwrapped_child) { for (mime_node_t *parent = node; parent; parent = parent->parent) if (parent->envelope_file) { message = parent->envelope_file; break; } - node->decrypted_child = _notmuch_crypto_decrypt (&node->decrypt_attempted, + node->unwrapped_child = _notmuch_crypto_decrypt (&node->decrypt_attempted, node->ctx->crypto->decrypt, message, - encrypteddata, &decrypt_result, &err); + part, &decrypt_result, &err); + if (node->unwrapped_child) + set_unwrapped_child_destructor (node); } - if (! node->decrypted_child) { + if (! node->unwrapped_child) { fprintf (stderr, "Failed to decrypt part: %s\n", err ? err->message : "no error explanation given"); goto DONE; @@ -248,7 +293,8 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject *part) node->decrypt_success = true; status = _notmuch_message_crypto_successful_decryption (node->ctx->msg_crypto); if (status) /* this is a warning, not an error */ - fprintf (stderr, "Warning: failed to note decryption status: %s.\n", notmuch_status_to_string (status)); + fprintf (stderr, "Warning: failed to note decryption status: %s.\n", + notmuch_status_to_string (status)); if (decrypt_result) { /* This may be NULL if the part is not signed. */ @@ -257,9 +303,11 @@ node_decrypt_and_verify (mime_node_t *node, GMimeObject *part) node->verify_attempted = true; g_object_ref (node->sig_list); set_signature_list_destructor (node); - status = _notmuch_message_crypto_potential_sig_list (node->ctx->msg_crypto, node->sig_list); + status = _notmuch_message_crypto_potential_sig_list (node->ctx->msg_crypto, + node->sig_list); if (status) /* this is a warning, not an error */ - fprintf (stderr, "Warning: failed to note signature status: %s.\n", notmuch_status_to_string (status)); + fprintf (stderr, "Warning: failed to note signature status: %s.\n", + notmuch_status_to_string (status)); } if (node->ctx->crypto->decrypt == NOTMUCH_DECRYPT_TRUE && message) { @@ -336,7 +384,8 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject *part, int numchild) } /* Handle PGP/MIME parts (by definition not cryptographic payload parts) */ - if (GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt != NOTMUCH_DECRYPT_FALSE)) { + if (GMIME_IS_MULTIPART_ENCRYPTED (part) && (node->ctx->crypto->decrypt != + NOTMUCH_DECRYPT_FALSE)) { if (node->nchildren != 2) { /* this violates RFC 3156 section 4, so we won't bother with it. */ fprintf (stderr, "Error: %d part(s) for a multipart/encrypted " @@ -354,8 +403,23 @@ _mime_node_set_up_part (mime_node_t *node, GMimeObject *part, int numchild) } else { node_verify (node, part); } + } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) && + GMIME_SECURE_MIME_TYPE_SIGNED_DATA == g_mime_application_pkcs7_mime_get_smime_type ( + GMIME_APPLICATION_PKCS7_MIME (part))) { + /* If node->ctx->crypto->verify is false, it would be better + * to just unwrap (instead of verifying), but + * https://github.com/jstedfast/gmime/issues/67 */ + node_verify (node, part); + } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part) && + GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA == g_mime_application_pkcs7_mime_get_smime_type ( + GMIME_APPLICATION_PKCS7_MIME (part)) && + (node->ctx->crypto->decrypt != NOTMUCH_DECRYPT_FALSE)) { + node_decrypt_and_verify (node, part); + if (node->unwrapped_child && node->nchildren == 0) + node->nchildren = 1; } else { - if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, part, node->parent ? node->parent->part : NULL, numchild) && + if (_notmuch_message_crypto_potential_payload (node->ctx->msg_crypto, part, node->parent ? + node->parent->part : NULL, numchild) && node->ctx->msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL) { GMimeObject *clean_payload = _notmuch_repair_crypto_payload_skip_legacy_display (part); if (clean_payload != part) { @@ -380,13 +444,17 @@ mime_node_child (mime_node_t *parent, int child) return NULL; if (GMIME_IS_MULTIPART (parent->part)) { - if (child == GMIME_MULTIPART_ENCRYPTED_CONTENT && parent->decrypted_child) - sub = parent->decrypted_child; + if (child == GMIME_MULTIPART_ENCRYPTED_CONTENT && parent->unwrapped_child) + sub = parent->unwrapped_child; else sub = g_mime_multipart_get_part ( GMIME_MULTIPART (parent->part), child); } else if (GMIME_IS_MESSAGE (parent->part)) { sub = g_mime_message_get_mime_part (GMIME_MESSAGE (parent->part)); + } else if (GMIME_IS_APPLICATION_PKCS7_MIME (parent->part) && + parent->unwrapped_child && + child == 0) { + sub = parent->unwrapped_child; } else { /* This should have been caught by _mime_node_set_up_part */ INTERNAL_ERROR ("Unexpected GMimeObject type: %s", diff --git a/notmuch-client-init.c b/notmuch-client-init.c new file mode 100644 index 00000000..60db6ba4 --- /dev/null +++ b/notmuch-client-init.c @@ -0,0 +1,18 @@ +#include "notmuch-client.h" +#include "gmime-filter-reply.h" + +/* Caller is responsible for only calling this once */ + +void +notmuch_client_init (void) +{ +#if ! GLIB_CHECK_VERSION (2, 35, 1) + g_type_init (); +#endif + + g_mime_init (); + + g_mime_filter_reply_module_init (); + + talloc_enable_null_tracking (); +} diff --git a/notmuch-client.h b/notmuch-client.h index 74690054..1a87240d 100644 --- a/notmuch-client.h +++ b/notmuch-client.h @@ -49,6 +49,7 @@ #include #include #include +#include #include "talloc-extra.h" #include "crypto.h" @@ -64,7 +65,7 @@ struct sprinter; struct notmuch_show_params; typedef struct notmuch_show_format { - struct sprinter *(*new_sprinter)(const void *ctx, FILE *stream); + struct sprinter *(*new_sprinter)(notmuch_database_t * db, FILE *stream); notmuch_status_t (*part)(const void *ctx, struct sprinter *sprinter, struct mime_node *node, int indent, const struct notmuch_show_params *params); @@ -74,7 +75,10 @@ typedef struct notmuch_show_params { bool entire_thread; bool omit_excluded; bool output_body; + int duplicate; int part; + int offset; + int limit; _notmuch_crypto_t crypto; bool include_html; GMimeStream *out_stream; @@ -136,7 +140,7 @@ chomp_newline (char *str) * this. New (required) map fields can be added without increasing * this. */ -#define NOTMUCH_FORMAT_CUR 4 +#define NOTMUCH_FORMAT_CUR 5 /* The minimum supported structured output format version. Requests * for format versions below this will return an error. */ #define NOTMUCH_FORMAT_MIN 1 @@ -155,7 +159,7 @@ chomp_newline (char *str) */ extern int notmuch_format_version; -typedef struct _notmuch_config notmuch_config_t; +typedef struct _notmuch_conffile notmuch_conffile_t; /* Commands that support structured output should support the * following argument @@ -170,46 +174,46 @@ void notmuch_exit_if_unsupported_format (void); int -notmuch_count_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_count_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_dump_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_new_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_insert_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_reindex_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_reply_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_restore_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_search_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_search_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_address_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_address_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_setup_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_setup_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_tag_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_config_command (notmuch_database_t *notmuch, int argc, char *argv[]); int -notmuch_compact_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_compact_command (notmuch_database_t *notmuch, int argc, char *argv[]); const char * notmuch_time_relative_date (const void *ctx, time_t then); @@ -228,6 +232,7 @@ show_one_part (const char *filename, int part); void format_part_sprinter (const void *ctx, struct sprinter *sp, mime_node_t *node, + int duplicate, bool output_body, bool include_html); @@ -249,91 +254,71 @@ json_quote_chararray (const void *ctx, const char *str, const size_t len); char * json_quote_str (const void *ctx, const char *str); +/* notmuch-client-init.c */ + +void notmuch_client_init (void); + /* notmuch-config.c */ typedef enum { - NOTMUCH_CONFIG_OPEN = 1 << 0, - NOTMUCH_CONFIG_CREATE = 1 << 1, -} notmuch_config_mode_t; - -notmuch_config_t * -notmuch_config_open (void *ctx, - const char *filename, - notmuch_config_mode_t config_mode); + NOTMUCH_COMMAND_CONFIG_CREATE = 1 << 1, + NOTMUCH_COMMAND_DATABASE_EARLY = 1 << 2, + NOTMUCH_COMMAND_DATABASE_WRITE = 1 << 3, + NOTMUCH_COMMAND_DATABASE_CREATE = 1 << 4, + NOTMUCH_COMMAND_CONFIG_LOAD = 1 << 5, +} notmuch_command_mode_t; + +notmuch_conffile_t * +notmuch_conffile_open (notmuch_database_t *notmuch, + const char *filename, + bool create); void -notmuch_config_close (notmuch_config_t *config); +notmuch_conffile_close (notmuch_conffile_t *config); int -notmuch_config_save (notmuch_config_t *config); +notmuch_conffile_save (notmuch_conffile_t *config); bool -notmuch_config_is_new (notmuch_config_t *config); - -const char * -notmuch_config_get_database_path (notmuch_config_t *config); +notmuch_conffile_is_new (notmuch_conffile_t *config); void -notmuch_config_set_database_path (notmuch_config_t *config, - const char *database_path); - -const char * -notmuch_config_get_user_name (notmuch_config_t *config); +notmuch_conffile_set_database_path (notmuch_conffile_t *config, + const char *database_path); void -notmuch_config_set_user_name (notmuch_config_t *config, - const char *user_name); - -const char * -notmuch_config_get_user_primary_email (notmuch_config_t *config); +notmuch_conffile_set_user_name (notmuch_conffile_t *config, + const char *user_name); void -notmuch_config_set_user_primary_email (notmuch_config_t *config, - const char *primary_email); - -const char ** -notmuch_config_get_user_other_email (notmuch_config_t *config, - size_t *length); +notmuch_conffile_set_user_primary_email (notmuch_conffile_t *config, + const char *primary_email); void -notmuch_config_set_user_other_email (notmuch_config_t *config, - const char *other_email[], - size_t length); +notmuch_conffile_set_user_other_email (notmuch_conffile_t *config, + const char *other_email[], + size_t length); -const char ** -notmuch_config_get_new_tags (notmuch_config_t *config, - size_t *length); void -notmuch_config_set_new_tags (notmuch_config_t *config, - const char *new_tags[], - size_t length); - -const char ** -notmuch_config_get_new_ignore (notmuch_config_t *config, - size_t *length); - -void -notmuch_config_set_new_ignore (notmuch_config_t *config, - const char *new_ignore[], +notmuch_conffile_set_new_tags (notmuch_conffile_t *config, + const char *new_tags[], size_t length); -bool -notmuch_config_get_maildir_synchronize_flags (notmuch_config_t *config); - void -notmuch_config_set_maildir_synchronize_flags (notmuch_config_t *config, - bool synchronize_flags); - -const char ** -notmuch_config_get_search_exclude_tags (notmuch_config_t *config, size_t *length); +notmuch_conffile_set_new_ignore (notmuch_conffile_t *config, + const char *new_ignore[], + size_t length); void -notmuch_config_set_search_exclude_tags (notmuch_config_t *config, - const char *list[], - size_t length); +notmuch_conffile_set_maildir_synchronize_flags (notmuch_conffile_t *config, + bool synchronize_flags); +void +notmuch_conffile_set_search_exclude_tags (notmuch_conffile_t *config, + const char *list[], + size_t length); int -notmuch_run_hook (const char *db_path, const char *hook); +notmuch_run_hook (notmuch_database_t *notmuch, const char *hook); bool debugger_is_active (void); @@ -395,8 +380,10 @@ struct mime_node { struct mime_node_context *ctx; /* Internal: For successfully decrypted multipart parts, the - * decrypted part to substitute for the second child. */ - GMimeObject *decrypted_child; + * decrypted part to substitute for the second child; or, for + * PKCS#7 parts, the part returned after removing/processing the + * PKCS#7 transformation */ + GMimeObject *unwrapped_child; /* Internal: The next child for depth-first traversal and the part * number to assign it (or -1 if unknown). */ @@ -405,7 +392,8 @@ struct mime_node { }; /* Construct a new MIME node pointing to the root message part of - * message. If crypto->verify is true, signed child parts will be + * message. Use the duplicate-th filename if that parameter is + * positive. If crypto->verify is true, signed child parts will be * verified. If crypto->decrypt is NOTMUCH_DECRYPT_TRUE, encrypted * child parts will be decrypted using either stored session keys or * asymmetric crypto. If crypto->decrypt is NOTMUCH_DECRYPT_AUTO, @@ -423,6 +411,7 @@ struct mime_node { */ notmuch_status_t mime_node_open (const void *ctx, notmuch_message_t *message, + int duplicate, _notmuch_crypto_t *crypto, mime_node_t **node_out); /* Return a new MIME node for the requested child part of parent. @@ -443,13 +432,13 @@ mime_node_seek_dfs (mime_node_t *node, int n); const _notmuch_message_crypto_t * mime_node_get_message_crypto_status (mime_node_t *node); -typedef enum dump_formats { +typedef enum { DUMP_FORMAT_AUTO, DUMP_FORMAT_BATCH_TAG, DUMP_FORMAT_SUP } dump_format_t; -typedef enum dump_includes { +typedef enum { DUMP_INCLUDE_TAGS = 1, DUMP_INCLUDE_CONFIG = 2, DUMP_INCLUDE_PROPERTIES = 4 @@ -467,7 +456,7 @@ notmuch_database_dump (notmuch_database_t *notmuch, dump_include_t include, bool gzip_output); -/* If status is non-zero (i.e. error) print appropriate +/* If status indicates error print appropriate * messages to stderr. */ @@ -489,13 +478,24 @@ print_status_database (const char *loc, int status_to_exit (notmuch_status_t status); +notmuch_status_t +print_status_gzbytes (const char *loc, + gzFile file, + int bytes); + +/* the __location__ macro is defined in talloc.h */ +#define ASSERT_GZBYTES(file, bytes) ((print_status_gzbytes (__location__, file, bytes)) ? exit (1) : \ + 0) +#define GZPRINTF(file, fmt, ...) ASSERT_GZBYTES (file, gzprintf (file, fmt, ##__VA_ARGS__)); +#define GZPUTS(file, str) ASSERT_GZBYTES (file, gzputs (file, str)); + #include "command-line-arguments.h" -extern const char *notmuch_requested_db_uuid; extern const notmuch_opt_desc_t notmuch_shared_options []; -void notmuch_exit_if_unmatched_db_uuid (notmuch_database_t *notmuch); -void notmuch_process_shared_options (const char *subcommand_name); +notmuch_query_syntax_t shared_option_query_syntax (); + +void notmuch_process_shared_options (notmuch_database_t *notmuch, const char *subcommand_name); int notmuch_minimal_options (const char *subcommand_name, int argc, char **argv); @@ -505,11 +505,10 @@ int notmuch_minimal_options (const char *subcommand_name, struct _notmuch_client_indexing_cli_choices { int decrypt_policy; bool decrypt_policy_set; - notmuch_indexopts_t *opts; }; extern struct _notmuch_client_indexing_cli_choices indexing_cli_choices; extern const notmuch_opt_desc_t notmuch_shared_indexing_options []; notmuch_status_t -notmuch_process_shared_indexing_options (notmuch_database_t *notmuch); +notmuch_process_shared_indexing_options (notmuch_indexopts_t *opts); #endif diff --git a/notmuch-compact.c b/notmuch-compact.c index f8996cf4..40ffb428 100644 --- a/notmuch-compact.c +++ b/notmuch-compact.c @@ -27,9 +27,8 @@ status_update_cb (const char *msg, unused (void *closure)) } int -notmuch_compact_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_compact_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - const char *path = notmuch_config_get_database_path (config); const char *backup_path = NULL; notmuch_status_t ret; bool quiet = false; @@ -46,17 +45,12 @@ notmuch_compact_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - if (notmuch_requested_db_uuid) { - fprintf (stderr, "Error: --uuid not implemented for compact\n"); - return EXIT_FAILURE; - } - - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (NULL, argv[0]); if (! quiet) printf ("Compacting database...\n"); - ret = notmuch_database_compact (path, backup_path, - quiet ? NULL : status_update_cb, NULL); + ret = notmuch_database_compact_db (notmuch, backup_path, + quiet ? NULL : status_update_cb, NULL); if (ret) { fprintf (stderr, "Compaction failed: %s\n", notmuch_status_to_string (ret)); return EXIT_FAILURE; diff --git a/notmuch-config.c b/notmuch-config.c index 1b079e85..8123e438 100644 --- a/notmuch-config.c +++ b/notmuch-config.c @@ -24,6 +24,7 @@ #include #include +#include "path-util.h" #include "unicode-util.h" static const char toplevel_config_comment[] = @@ -31,110 +32,97 @@ static const char toplevel_config_comment[] = "\n" " For more information about notmuch, see https://notmuchmail.org"; -static const char database_config_comment[] = - " Database configuration\n" - "\n" - " The only value supported here is 'path' which should be the top-level\n" - " directory where your mail currently exists and to where mail will be\n" - " delivered in the future. Files should be individual email messages.\n" - " Notmuch will store its database within a sub-directory of the path\n" - " configured here named \".notmuch\".\n"; - -static const char new_config_comment[] = - " Configuration for \"notmuch new\"\n" - "\n" - " The following options are supported here:\n" - "\n" - "\ttags A list (separated by ';') of the tags that will be\n" - "\t added to all messages incorporated by \"notmuch new\".\n" - "\n" - "\tignore A list (separated by ';') of file and directory names\n" - "\t that will not be searched for messages by \"notmuch new\".\n" - "\n" - "\t NOTE: *Every* file/directory that goes by one of those\n" - "\t names will be ignored, independent of its depth/location\n" - "\t in the mail store.\n"; - -static const char user_config_comment[] = - " User configuration\n" - "\n" - " Here is where you can let notmuch know how you would like to be\n" - " addressed. Valid settings are\n" - "\n" - "\tname Your full name.\n" - "\tprimary_email Your primary email address.\n" - "\tother_email A list (separated by ';') of other email addresses\n" - "\t at which you receive email.\n" - "\n" - " Notmuch will use the various email addresses configured here when\n" - " formatting replies. It will avoid including your own addresses in the\n" - " recipient list of replies, and will set the From address based on the\n" - " address to which the original email was addressed.\n"; - -static const char maildir_config_comment[] = - " Maildir compatibility configuration\n" - "\n" - " The following option is supported here:\n" - "\n" - "\tsynchronize_flags Valid values are true and false.\n" - "\n" - "\tIf true, then the following maildir flags (in message filenames)\n" - "\twill be synchronized with the corresponding notmuch tags:\n" - "\n" - "\t\tFlag Tag\n" - "\t\t---- -------\n" - "\t\tD draft\n" - "\t\tF flagged\n" - "\t\tP passed\n" - "\t\tR replied\n" - "\t\tS unread (added when 'S' flag is not present)\n" - "\n" - "\tThe \"notmuch new\" command will notice flag changes in filenames\n" - "\tand update tags, while the \"notmuch tag\" and \"notmuch restore\"\n" - "\tcommands will notice tag changes and update flags in filenames\n"; - -static const char search_config_comment[] = - " Search configuration\n" - "\n" - " The following option is supported here:\n" - "\n" - "\texclude_tags\n" - "\t\tA ;-separated list of tags that will be excluded from\n" - "\t\tsearch results by default. Using an excluded tag in a\n" - "\t\tquery will override that exclusion.\n"; - -static const char crypto_config_comment[] = - " Cryptography related configuration\n" - "\n" - " The following old option is now ignored:\n" - "\n" - "\tgpgpath\n" - "\t\tThis option was used by older builds of notmuch to choose\n" - "\t\tthe version of gpg to use.\n" - "\t\tSetting $PATH is a better approach.\n"; +static const struct config_group { + const char *group_name; + const char *comment; +} group_comment_table [] = { + { + "database", + " Database configuration\n" + "\n" + " The only value supported here is 'path' which should be the top-level\n" + " directory where your mail currently exists and to where mail will be\n" + " delivered in the future. Files should be individual email messages.\n" + " Notmuch will store its database within a sub-directory of the path\n" + " configured here named \".notmuch\".\n" + }, + { + "user", + " User configuration\n" + "\n" + " Here is where you can let notmuch know how you would like to be\n" + " addressed. Valid settings are\n" + "\n" + "\tname Your full name.\n" + "\tprimary_email Your primary email address.\n" + "\tother_email A list (separated by ';') of other email addresses\n" + "\t at which you receive email.\n" + "\n" + " Notmuch will use the various email addresses configured here when\n" + " formatting replies. It will avoid including your own addresses in the\n" + " recipient list of replies, and will set the From address based on the\n" + " address to which the original email was addressed.\n" + }, + { + "new", + " Configuration for \"notmuch new\"\n" + "\n" + " The following options are supported here:\n" + "\n" + "\ttags A list (separated by ';') of the tags that will be\n" + "\t added to all messages incorporated by \"notmuch new\".\n" + "\n" + "\tignore A list (separated by ';') of file and directory names\n" + "\t that will not be searched for messages by \"notmuch new\".\n" + "\n" + "\t NOTE: *Every* file/directory that goes by one of those\n" + "\t names will be ignored, independent of its depth/location\n" + "\t in the mail store.\n" + }, + { + "search", + " Search configuration\n" + "\n" + " The following option is supported here:\n" + "\n" + "\texclude_tags\n" + "\t\tA ;-separated list of tags that will be excluded from\n" + "\t\tsearch results by default. Using an excluded tag in a\n" + "\t\tquery will override that exclusion.\n" + }, + { + "maildir", + " Maildir compatibility configuration\n" + "\n" + " The following option is supported here:\n" + "\n" + "\tsynchronize_flags Valid values are true and false.\n" + "\n" + "\tIf true, then the following maildir flags (in message filenames)\n" + "\twill be synchronized with the corresponding notmuch tags:\n" + "\n" + "\t\tFlag Tag\n" + "\t\t---- -------\n" + "\t\tD draft\n" + "\t\tF flagged\n" + "\t\tP passed\n" + "\t\tR replied\n" + "\t\tS unread (added when 'S' flag is not present)\n" + "\n" + "\tThe \"notmuch new\" command will notice flag changes in filenames\n" + "\tand update tags, while the \"notmuch tag\" and \"notmuch restore\"\n" + "\tcommands will notice tag changes and update flags in filenames\n" + }, +}; -struct _notmuch_config { +struct _notmuch_conffile { char *filename; GKeyFile *key_file; bool is_new; - - char *database_path; - char *crypto_gpg_path; - char *user_name; - char *user_primary_email; - const char **user_other_email; - size_t user_other_email_length; - const char **new_tags; - size_t new_tags_length; - const char **new_ignore; - size_t new_ignore_length; - bool maildir_synchronize_flags; - const char **search_exclude_tags; - size_t search_exclude_tags_length; }; static int -notmuch_config_destructor (notmuch_config_t *config) +notmuch_conffile_destructor (notmuch_conffile_t *config) { if (config->key_file) g_key_file_free (config->key_file); @@ -142,72 +130,8 @@ notmuch_config_destructor (notmuch_config_t *config) return 0; } -static char * -get_name_from_passwd_file (void *ctx) -{ - long pw_buf_size; - char *pw_buf; - struct passwd passwd, *ignored; - char *name; - int e; - - pw_buf_size = sysconf (_SC_GETPW_R_SIZE_MAX); - if (pw_buf_size == -1) pw_buf_size = 64; - pw_buf = talloc_size (ctx, pw_buf_size); - - while ((e = getpwuid_r (getuid (), &passwd, pw_buf, - pw_buf_size, &ignored)) == ERANGE) { - pw_buf_size = pw_buf_size * 2; - pw_buf = talloc_zero_size (ctx, pw_buf_size); - } - - if (e == 0) { - char *comma = strchr (passwd.pw_gecos, ','); - if (comma) - name = talloc_strndup (ctx, passwd.pw_gecos, - comma - passwd.pw_gecos); - else - name = talloc_strdup (ctx, passwd.pw_gecos); - } else { - name = talloc_strdup (ctx, ""); - } - - talloc_free (pw_buf); - - return name; -} - -static char * -get_username_from_passwd_file (void *ctx) -{ - long pw_buf_size; - char *pw_buf; - struct passwd passwd, *ignored; - char *name; - int e; - - pw_buf_size = sysconf (_SC_GETPW_R_SIZE_MAX); - if (pw_buf_size == -1) pw_buf_size = 64; - pw_buf = talloc_zero_size (ctx, pw_buf_size); - - while ((e = getpwuid_r (getuid (), &passwd, pw_buf, - pw_buf_size, &ignored)) == ERANGE) { - pw_buf_size = pw_buf_size * 2; - pw_buf = talloc_zero_size (ctx, pw_buf_size); - } - - if (e == 0) - name = talloc_strdup (ctx, passwd.pw_name); - else - name = talloc_strdup (ctx, ""); - - talloc_free (pw_buf); - - return name; -} - static bool -get_config_from_file (notmuch_config_t *config, bool create_new) +get_config_from_file (notmuch_conffile_t *config, bool create_new) { #define BUF_SIZE 4096 char *config_str = NULL; @@ -322,32 +246,21 @@ get_config_from_file (notmuch_config_t *config, bool create_new) * The default configuration also contains comments to guide the * user in editing the file directly. */ -notmuch_config_t * -notmuch_config_open (void *ctx, - const char *filename, - notmuch_config_mode_t config_mode) +notmuch_conffile_t * +notmuch_conffile_open (notmuch_database_t *notmuch, + const char *filename, + bool create) { - GError *error = NULL; - size_t tmp; char *notmuch_config_env = NULL; - int file_had_database_group; - int file_had_new_group; - int file_had_user_group; - int file_had_maildir_group; - int file_had_search_group; - int file_had_crypto_group; - notmuch_config_t *config = talloc_zero (ctx, notmuch_config_t); + notmuch_conffile_t *config = talloc_zero (notmuch, notmuch_conffile_t); if (config == NULL) { fprintf (stderr, "Out of memory.\n"); return NULL; } - talloc_set_destructor (config, notmuch_config_destructor); - - /* non-zero defaults */ - config->maildir_synchronize_flags = true; + talloc_set_destructor (config, notmuch_conffile_destructor); if (filename) { config->filename = talloc_strdup (config, filename); @@ -360,153 +273,42 @@ notmuch_config_open (void *ctx, config->key_file = g_key_file_new (); - if (config_mode & NOTMUCH_CONFIG_OPEN) { - bool create_new = (config_mode & NOTMUCH_CONFIG_CREATE) != 0; - - if (! get_config_from_file (config, create_new)) { - talloc_free (config); - return NULL; - } - } - - /* Whenever we know of configuration sections that don't appear in - * the configuration file, we add some comments to help the user - * understand what can be done. - * - * It would be convenient to just add those comments now, but - * apparently g_key_file will clear any comments when keys are - * added later that create the groups. So we have to check for the - * groups now, but add the comments only after setting all of our - * values. - */ - file_had_database_group = g_key_file_has_group (config->key_file, - "database"); - file_had_new_group = g_key_file_has_group (config->key_file, "new"); - file_had_user_group = g_key_file_has_group (config->key_file, "user"); - file_had_maildir_group = g_key_file_has_group (config->key_file, "maildir"); - file_had_search_group = g_key_file_has_group (config->key_file, "search"); - file_had_crypto_group = g_key_file_has_group (config->key_file, "crypto"); - - if (notmuch_config_get_database_path (config) == NULL) { - char *path = getenv ("MAILDIR"); - if (path) - path = talloc_strdup (config, path); - else - path = talloc_asprintf (config, "%s/mail", - getenv ("HOME")); - notmuch_config_set_database_path (config, path); - talloc_free (path); - } - - if (notmuch_config_get_user_name (config) == NULL) { - char *name = getenv ("NAME"); - if (name) - name = talloc_strdup (config, name); - else - name = get_name_from_passwd_file (config); - notmuch_config_set_user_name (config, name); - talloc_free (name); - } - - if (notmuch_config_get_user_primary_email (config) == NULL) { - char *email = getenv ("EMAIL"); - if (email) { - notmuch_config_set_user_primary_email (config, email); - } else { - char hostname[256]; - struct hostent *hostent; - const char *domainname; - - char *username = get_username_from_passwd_file (config); - - gethostname (hostname, 256); - hostname[255] = '\0'; - - hostent = gethostbyname (hostname); - if (hostent && (domainname = strchr (hostent->h_name, '.'))) - domainname += 1; - else - domainname = "(none)"; - - email = talloc_asprintf (config, "%s@%s.%s", - username, hostname, domainname); - - notmuch_config_set_user_primary_email (config, email); - - talloc_free (username); - talloc_free (email); - } - } - - if (notmuch_config_get_new_tags (config, &tmp) == NULL) { - const char *tags[] = { "unread", "inbox" }; - notmuch_config_set_new_tags (config, tags, 2); - } - - if (notmuch_config_get_new_ignore (config, &tmp) == NULL) { - notmuch_config_set_new_ignore (config, NULL, 0); + if (! get_config_from_file (config, create)) { + talloc_free (config); + return NULL; } - if (notmuch_config_get_search_exclude_tags (config, &tmp) == NULL) { - if (config->is_new) { - const char *tags[] = { "deleted", "spam" }; - notmuch_config_set_search_exclude_tags (config, tags, 2); - } else { - notmuch_config_set_search_exclude_tags (config, NULL, 0); + for (size_t i = 0; i < ARRAY_SIZE (group_comment_table); i++) { + const char *name = group_comment_table[i].group_name; + if (! g_key_file_has_group (config->key_file, name)) { + /* Force group to exist before adding comment */ + g_key_file_set_value (config->key_file, name, "dummy_key", "dummy_val"); + g_key_file_remove_key (config->key_file, name, "dummy_key", NULL); + if (config->is_new && (i == 0) ) { + const char *comment; + + comment = talloc_asprintf (config, "%s\n%s", + toplevel_config_comment, + group_comment_table[i].comment); + g_key_file_set_comment (config->key_file, name, NULL, comment, + NULL); + } else { + g_key_file_set_comment (config->key_file, name, NULL, + group_comment_table[i].comment, NULL); + } } } - - error = NULL; - config->maildir_synchronize_flags = - g_key_file_get_boolean (config->key_file, - "maildir", "synchronize_flags", &error); - if (error) { - notmuch_config_set_maildir_synchronize_flags (config, true); - g_error_free (error); - } - - /* Whenever we know of configuration sections that don't appear in - * the configuration file, we add some comments to help the user - * understand what can be done. */ - if (config->is_new) - g_key_file_set_comment (config->key_file, NULL, NULL, - toplevel_config_comment, NULL); - - if (! file_had_database_group) - g_key_file_set_comment (config->key_file, "database", NULL, - database_config_comment, NULL); - - if (! file_had_new_group) - g_key_file_set_comment (config->key_file, "new", NULL, - new_config_comment, NULL); - - if (! file_had_user_group) - g_key_file_set_comment (config->key_file, "user", NULL, - user_config_comment, NULL); - - if (! file_had_maildir_group) - g_key_file_set_comment (config->key_file, "maildir", NULL, - maildir_config_comment, NULL); - - if (! file_had_search_group) - g_key_file_set_comment (config->key_file, "search", NULL, - search_config_comment, NULL); - - if (! file_had_crypto_group) - g_key_file_set_comment (config->key_file, "crypto", NULL, - crypto_config_comment, NULL); - return config; } -/* Close the given notmuch_config_t object, freeing all resources. +/* Close the given notmuch_conffile_t object, freeing all resources. * * Note: Any changes made to the configuration are *not* saved by this - * function. To save changes, call notmuch_config_save before - * notmuch_config_close. + * function. To save changes, call notmuch_conffile_save before + * notmuch_conffile_close. */ void -notmuch_config_close (notmuch_config_t *config) +notmuch_conffile_close (notmuch_conffile_t *config) { talloc_free (config); } @@ -519,7 +321,7 @@ notmuch_config_close (notmuch_config_t *config) * printing a description of the error to stderr). */ int -notmuch_config_save (notmuch_config_t *config) +notmuch_conffile_save (notmuch_conffile_t *config) { size_t length; char *data, *filename; @@ -532,7 +334,7 @@ notmuch_config_save (notmuch_config_t *config) } /* Try not to overwrite symlinks. */ - filename = canonicalize_file_name (config->filename); + filename = notmuch_canonicalize_file_name (config->filename); if (! filename) { if (errno == ENOENT) { filename = strdup (config->filename); @@ -569,200 +371,81 @@ notmuch_config_save (notmuch_config_t *config) } bool -notmuch_config_is_new (notmuch_config_t *config) +notmuch_conffile_is_new (notmuch_conffile_t *config) { return config->is_new; } -static const char * -_config_get (notmuch_config_t *config, char **field, - const char *group, const char *key) -{ - /* read from config file and cache value, if not cached already */ - if (*field == NULL) { - char *value; - value = g_key_file_get_string (config->key_file, group, key, NULL); - if (value) { - *field = talloc_strdup (config, value); - free (value); - } - } - return *field; -} - static void -_config_set (notmuch_config_t *config, char **field, +_config_set (notmuch_conffile_t *config, const char *group, const char *key, const char *value) { g_key_file_set_string (config->key_file, group, key, value); - - /* drop the cached value */ - talloc_free (*field); - *field = NULL; -} - -static const char ** -_config_get_list (notmuch_config_t *config, - const char *section, const char *key, - const char ***outlist, size_t *list_length, size_t *ret_length) -{ - assert (outlist); - - /* read from config file and cache value, if not cached already */ - if (*outlist == NULL) { - - char **inlist = g_key_file_get_string_list (config->key_file, - section, key, list_length, NULL); - if (inlist) { - unsigned int i; - - *outlist = talloc_size (config, sizeof (char *) * (*list_length + 1)); - - for (i = 0; i < *list_length; i++) - (*outlist)[i] = talloc_strdup (*outlist, inlist[i]); - - (*outlist)[i] = NULL; - - g_strfreev (inlist); - } - } - - if (ret_length) - *ret_length = *list_length; - - return *outlist; } static void -_config_set_list (notmuch_config_t *config, +_config_set_list (notmuch_conffile_t *config, const char *group, const char *key, const char *list[], - size_t length, const char ***config_var ) -{ - g_key_file_set_string_list (config->key_file, group, key, list, length); - - /* drop the cached value */ - talloc_free (*config_var); - *config_var = NULL; -} - -const char * -notmuch_config_get_database_path (notmuch_config_t *config) + size_t length) { - char *db_path = (char *) _config_get (config, &config->database_path, "database", "path"); - - if (db_path && *db_path != '/') { - /* If the path in the configuration file begins with any - * character other than /, presume that it is relative to - * $HOME and update as appropriate. - */ - char *abs_path = talloc_asprintf (config, "%s/%s", getenv ("HOME"), db_path); - talloc_free (db_path); - db_path = config->database_path = abs_path; - } - - return db_path; + if (length > 1) + g_key_file_set_string_list (config->key_file, group, key, list, length); + else + g_key_file_set_string (config->key_file, group, key, list[0]); } void -notmuch_config_set_database_path (notmuch_config_t *config, - const char *database_path) +notmuch_conffile_set_database_path (notmuch_conffile_t *config, + const char *database_path) { - _config_set (config, &config->database_path, "database", "path", database_path); -} - -const char * -notmuch_config_get_user_name (notmuch_config_t *config) -{ - return _config_get (config, &config->user_name, "user", "name"); + _config_set (config, "database", "path", database_path); } void -notmuch_config_set_user_name (notmuch_config_t *config, - const char *user_name) -{ - _config_set (config, &config->user_name, "user", "name", user_name); -} - -const char * -notmuch_config_get_user_primary_email (notmuch_config_t *config) +notmuch_conffile_set_user_name (notmuch_conffile_t *config, + const char *user_name) { - return _config_get (config, &config->user_primary_email, "user", "primary_email"); + _config_set (config, "user", "name", user_name); } void -notmuch_config_set_user_primary_email (notmuch_config_t *config, - const char *primary_email) +notmuch_conffile_set_user_primary_email (notmuch_conffile_t *config, + const char *primary_email) { - _config_set (config, &config->user_primary_email, "user", "primary_email", primary_email); -} - -const char ** -notmuch_config_get_user_other_email (notmuch_config_t *config, size_t *length) -{ - return _config_get_list (config, "user", "other_email", - &(config->user_other_email), - &(config->user_other_email_length), length); -} - -const char ** -notmuch_config_get_new_tags (notmuch_config_t *config, size_t *length) -{ - return _config_get_list (config, "new", "tags", - &(config->new_tags), - &(config->new_tags_length), length); -} - -const char ** -notmuch_config_get_new_ignore (notmuch_config_t *config, size_t *length) -{ - return _config_get_list (config, "new", "ignore", - &(config->new_ignore), - &(config->new_ignore_length), length); + _config_set (config, "user", "primary_email", primary_email); } void -notmuch_config_set_user_other_email (notmuch_config_t *config, - const char *list[], - size_t length) +notmuch_conffile_set_user_other_email (notmuch_conffile_t *config, + const char *list[], + size_t length) { - _config_set_list (config, "user", "other_email", list, length, - &(config->user_other_email)); + _config_set_list (config, "user", "other_email", list, length); } void -notmuch_config_set_new_tags (notmuch_config_t *config, - const char *list[], - size_t length) -{ - _config_set_list (config, "new", "tags", list, length, - &(config->new_tags)); -} - -void -notmuch_config_set_new_ignore (notmuch_config_t *config, +notmuch_conffile_set_new_tags (notmuch_conffile_t *config, const char *list[], size_t length) { - _config_set_list (config, "new", "ignore", list, length, - &(config->new_ignore)); + _config_set_list (config, "new", "tags", list, length); } -const char ** -notmuch_config_get_search_exclude_tags (notmuch_config_t *config, size_t *length) +void +notmuch_conffile_set_new_ignore (notmuch_conffile_t *config, + const char *list[], + size_t length) { - return _config_get_list (config, "search", "exclude_tags", - &(config->search_exclude_tags), - &(config->search_exclude_tags_length), length); + _config_set_list (config, "new", "ignore", list, length); } void -notmuch_config_set_search_exclude_tags (notmuch_config_t *config, - const char *list[], - size_t length) +notmuch_conffile_set_search_exclude_tags (notmuch_conffile_t *config, + const char *list[], + size_t length) { - _config_set_list (config, "search", "exclude_tags", list, length, - &(config->search_exclude_tags)); + _config_set_list (config, "search", "exclude_tags", list, length); } @@ -834,19 +517,19 @@ validate_field_name (const char *str) typedef struct config_key { const char *name; - bool in_db; bool prefix; bool (*validate)(const char *); } config_key_info_t; -static struct config_key +static const struct config_key config_key_table[] = { - { "index.decrypt", true, false, NULL }, - { "index.header.", true, true, validate_field_name }, - { "query.", true, true, NULL }, + { "index.decrypt", false, NULL }, + { "index.header.", true, validate_field_name }, + { "query.", true, NULL }, + { "squery.", true, validate_field_name }, }; -static config_key_info_t * +static const config_key_info_t * _config_key_info (const char *item) { for (size_t i = 0; i < ARRAY_SIZE (config_key_table); i++) { @@ -860,95 +543,30 @@ _config_key_info (const char *item) return NULL; } -static bool -_stored_in_db (const char *item) -{ - config_key_info_t *info; - - info = _config_key_info (item); - - return (info && info->in_db); -} - static int -_print_db_config (notmuch_config_t *config, const char *name) +notmuch_config_command_get (notmuch_database_t *notmuch, char *item) { - notmuch_database_t *notmuch; - char *val; - - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_ONLY, ¬much)) - return EXIT_FAILURE; + notmuch_config_values_t *list; - /* XXX Handle UUID mismatch? */ - - if (print_status_database ("notmuch config", notmuch, - notmuch_database_get_config (notmuch, name, &val))) - return EXIT_FAILURE; - - puts (val); - - return EXIT_SUCCESS; -} - -static int -notmuch_config_command_get (notmuch_config_t *config, char *item) -{ - if (strcmp (item, "database.path") == 0) { - printf ("%s\n", notmuch_config_get_database_path (config)); - } else if (strcmp (item, "user.name") == 0) { - printf ("%s\n", notmuch_config_get_user_name (config)); - } else if (strcmp (item, "user.primary_email") == 0) { - printf ("%s\n", notmuch_config_get_user_primary_email (config)); - } else if (strcmp (item, "user.other_email") == 0) { - const char **other_email; - size_t i, length; - - other_email = notmuch_config_get_user_other_email (config, &length); - for (i = 0; i < length; i++) - printf ("%s\n", other_email[i]); - } else if (strcmp (item, "new.tags") == 0) { - const char **tags; - size_t i, length; - - tags = notmuch_config_get_new_tags (config, &length); - for (i = 0; i < length; i++) - printf ("%s\n", tags[i]); - } else if (STRNCMP_LITERAL (item, BUILT_WITH_PREFIX) == 0) { - printf ("%s\n", - notmuch_built_with (item + strlen (BUILT_WITH_PREFIX)) ? "true" : "false"); - } else if (_stored_in_db (item)) { - return _print_db_config (config, item); + if (STRNCMP_LITERAL (item, BUILT_WITH_PREFIX) == 0) { + if (notmuch_built_with (item + strlen (BUILT_WITH_PREFIX))) + puts ("true"); + else + puts ("false"); } else { - char **value; - size_t i, length; - char *group, *key; - - if (_item_split (item, &group, &key)) - return 1; - - value = g_key_file_get_string_list (config->key_file, - group, key, - &length, NULL); - if (value == NULL) { - fprintf (stderr, "Unknown configuration item: %s.%s\n", - group, key); - return 1; + for (list = notmuch_config_get_values_string (notmuch, item); + notmuch_config_values_valid (list); + notmuch_config_values_move_to_next (list)) { + const char *val = notmuch_config_values_get (list); + puts (val); } - - for (i = 0; i < length; i++) - printf ("%s\n", value[i]); - - g_strfreev (value); } - - return 0; + return EXIT_SUCCESS; } static int -_set_db_config (notmuch_config_t *config, const char *key, int argc, char **argv) +_set_db_config (notmuch_database_t *notmuch, const char *key, int argc, char **argv) { - notmuch_database_t *notmuch; const char *val = ""; if (argc > 1) { @@ -961,12 +579,11 @@ _set_db_config (notmuch_config_t *config, const char *key, int argc, char **argv val = argv[0]; } - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + if (print_status_database ("notmuch config", notmuch, + notmuch_database_reopen (notmuch, + NOTMUCH_DATABASE_MODE_READ_WRITE))) return EXIT_FAILURE; - /* XXX Handle UUID mismatch? */ - if (print_status_database ("notmuch config", notmuch, notmuch_database_set_config (notmuch, key, val))) return EXIT_FAILURE; @@ -979,10 +596,37 @@ _set_db_config (notmuch_config_t *config, const char *key, int argc, char **argv } static int -notmuch_config_command_set (notmuch_config_t *config, char *item, int argc, char *argv[]) +notmuch_config_command_set (notmuch_database_t *notmuch, + int argc, char *argv[]) { char *group, *key; - config_key_info_t *key_info; + const config_key_info_t *key_info; + notmuch_conffile_t *config; + bool update_database = false; + int opt_index, ret; + char *item; + + notmuch_opt_desc_t options[] = { + { .opt_bool = &update_database, .name = "database" }, + { } + }; + + opt_index = parse_arguments (argc, argv, options, 1); + if (opt_index < 0) + return EXIT_FAILURE; + + argc -= opt_index; + argv += opt_index; + + if (argc < 1) { + fprintf (stderr, "Error: notmuch config set requires at least " + "one argument.\n"); + return EXIT_FAILURE; + } + + item = argv[0]; + argv++; + argc--; if (STRNCMP_LITERAL (item, BUILT_WITH_PREFIX) == 0) { fprintf (stderr, "Error: read only option: %s\n", item); @@ -993,13 +637,18 @@ notmuch_config_command_set (notmuch_config_t *config, char *item, int argc, char if (key_info && key_info->validate && (! key_info->validate (item))) return 1; - if (key_info && key_info->in_db) { - return _set_db_config (config, item, argc, argv); + if (update_database) { + return _set_db_config (notmuch, item, argc, argv); } if (_item_split (item, &group, &key)) return 1; + config = notmuch_conffile_open (notmuch, + notmuch_config_path (notmuch), false); + if (! config) + return 1; + /* With only the name of an item, we clear it from the * configuration file. * @@ -1020,7 +669,11 @@ notmuch_config_command_set (notmuch_config_t *config, char *item, int argc, char break; } - return notmuch_config_save (config); + ret = notmuch_conffile_save (config); + + notmuch_conffile_close (config); + + return ret; } static @@ -1036,74 +689,30 @@ _notmuch_config_list_built_with () printf ("%sretry_lock=%s\n", BUILT_WITH_PREFIX, notmuch_built_with ("retry_lock") ? "true" : "false"); + printf ("%ssexp_queries=%s\n", + BUILT_WITH_PREFIX, + notmuch_built_with ("sexp_queries") ? "true" : "false"); } static int -_list_db_config (notmuch_config_t *config) -{ - notmuch_database_t *notmuch; - notmuch_config_list_t *list; - - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_ONLY, ¬much)) - return EXIT_FAILURE; - - /* XXX Handle UUID mismatch? */ - - - if (print_status_database ("notmuch config", notmuch, - notmuch_database_get_config_list (notmuch, "", &list))) - return EXIT_FAILURE; - - for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { - printf ("%s=%s\n", notmuch_config_list_key (list), notmuch_config_list_value (list)); - } - notmuch_config_list_destroy (list); - - return EXIT_SUCCESS; -} - -static int -notmuch_config_command_list (notmuch_config_t *config) +notmuch_config_command_list (notmuch_database_t *notmuch) { - char **groups; - size_t g, groups_length; - - groups = g_key_file_get_groups (config->key_file, &groups_length); - if (groups == NULL) - return 1; - - for (g = 0; g < groups_length; g++) { - char **keys; - size_t k, keys_length; - - keys = g_key_file_get_keys (config->key_file, - groups[g], &keys_length, NULL); - if (keys == NULL) - continue; - - for (k = 0; k < keys_length; k++) { - char *value; - - value = g_key_file_get_string (config->key_file, - groups[g], keys[k], NULL); - if (value != NULL) { - printf ("%s.%s=%s\n", groups[g], keys[k], value); - free (value); - } - } - - g_strfreev (keys); - } - - g_strfreev (groups); + notmuch_config_pairs_t *list; _notmuch_config_list_built_with (); - return _list_db_config (config); + for (list = notmuch_config_get_pairs (notmuch, ""); + notmuch_config_pairs_valid (list); + notmuch_config_pairs_move_to_next (list)) { + const char *value = notmuch_config_pairs_value (list); + if (value) + printf ("%s=%s\n", notmuch_config_pairs_key (list), value); + } + notmuch_config_pairs_destroy (list); + return EXIT_SUCCESS; } int -notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_config_command (notmuch_database_t *notmuch, int argc, char *argv[]) { int ret; int opt_index; @@ -1112,10 +721,6 @@ notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - if (notmuch_requested_db_uuid) - fprintf (stderr, "Warning: ignoring --uuid=%s\n", - notmuch_requested_db_uuid); - /* skip at least subcommand argument */ argc -= opt_index; argv += opt_index; @@ -1131,16 +736,11 @@ notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]) "one argument.\n"); return EXIT_FAILURE; } - ret = notmuch_config_command_get (config, argv[1]); + ret = notmuch_config_command_get (notmuch, argv[1]); } else if (strcmp (argv[0], "set") == 0) { - if (argc < 2) { - fprintf (stderr, "Error: notmuch config set requires at least " - "one argument.\n"); - return EXIT_FAILURE; - } - ret = notmuch_config_command_set (config, argv[1], argc - 2, argv + 2); + ret = notmuch_config_command_set (notmuch, argc, argv); } else if (strcmp (argv[0], "list") == 0) { - ret = notmuch_config_command_list (config); + ret = notmuch_config_command_list (notmuch); } else { fprintf (stderr, "Unrecognized argument for notmuch config: %s\n", argv[0]); @@ -1151,17 +751,10 @@ notmuch_config_command (notmuch_config_t *config, int argc, char *argv[]) } -bool -notmuch_config_get_maildir_synchronize_flags (notmuch_config_t *config) -{ - return config->maildir_synchronize_flags; -} - void -notmuch_config_set_maildir_synchronize_flags (notmuch_config_t *config, - bool synchronize_flags) +notmuch_conffile_set_maildir_synchronize_flags (notmuch_conffile_t *config, + bool synchronize_flags) { g_key_file_set_boolean (config->key_file, "maildir", "synchronize_flags", synchronize_flags); - config->maildir_synchronize_flags = synchronize_flags; } diff --git a/notmuch-count.c b/notmuch-count.c index d8ad7d6d..0d9046a8 100644 --- a/notmuch-count.c +++ b/notmuch-count.c @@ -64,10 +64,9 @@ count_files (notmuch_query_t *query) /* return 0 on success, -1 on failure */ static int print_count (notmuch_database_t *notmuch, const char *query_str, - const char **exclude_tags, size_t exclude_tags_length, int output, int print_lastmod) + notmuch_config_values_t *exclude_tags, int output, int print_lastmod) { notmuch_query_t *query; - size_t i; int count; unsigned int ucount; unsigned long revision; @@ -75,17 +74,24 @@ print_count (notmuch_database_t *notmuch, const char *query_str, int ret = 0; notmuch_status_t status; - query = notmuch_query_create (notmuch, query_str); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); - return -1; + status = notmuch_query_create_with_syntax (notmuch, query_str, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch count", notmuch, status)) { + ret = -1; + goto DONE; } - for (i = 0; i < exclude_tags_length; i++) { - status = notmuch_query_add_tag_exclude (query, exclude_tags[i]); + for (notmuch_config_values_start (exclude_tags); + notmuch_config_values_valid (exclude_tags); + notmuch_config_values_move_to_next (exclude_tags)) { + + status = notmuch_query_add_tag_exclude (query, + notmuch_config_values_get (exclude_tags)); if (status && status != NOTMUCH_STATUS_IGNORED) { print_status_query ("notmuch count", query, status); - return -1; + ret = -1; + goto DONE; } } @@ -127,8 +133,8 @@ print_count (notmuch_database_t *notmuch, const char *query_str, } static int -count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags, - size_t exclude_tags_length, int output, int print_lastmod) +count_file (notmuch_database_t *notmuch, FILE *input, notmuch_config_values_t *exclude_tags, + int output, int print_lastmod) { char *line = NULL; ssize_t line_len; @@ -137,8 +143,7 @@ count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags, while (! ret && (line_len = getline (&line, &line_size, input)) != -1) { chomp_newline (line); - ret = print_count (notmuch, line, exclude_tags, exclude_tags_length, - output, print_lastmod); + ret = print_count (notmuch, line, exclude_tags, output, print_lastmod); } if (line) @@ -148,15 +153,13 @@ count_file (notmuch_database_t *notmuch, FILE *input, const char **exclude_tags, } int -notmuch_count_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_count_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; char *query_str; int opt_index; int output = OUTPUT_MESSAGES; bool exclude = true; - const char **search_exclude_tags = NULL; - size_t search_exclude_tags_length = 0; + notmuch_config_values_t *exclude_tags = NULL; bool batch = false; bool print_lastmod = false; FILE *input = stdin; @@ -181,7 +184,7 @@ notmuch_count_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); if (input_file_name) { batch = true; @@ -200,29 +203,20 @@ notmuch_count_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_ONLY, ¬much)) - return EXIT_FAILURE; - - notmuch_exit_if_unmatched_db_uuid (notmuch); - - query_str = query_string_from_args (config, argc - opt_index, argv + opt_index); + query_str = query_string_from_args (notmuch, argc - opt_index, argv + opt_index); if (query_str == NULL) { fprintf (stderr, "Out of memory.\n"); return EXIT_FAILURE; } if (exclude) { - search_exclude_tags = notmuch_config_get_search_exclude_tags - (config, &search_exclude_tags_length); + exclude_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS); } if (batch) - ret = count_file (notmuch, input, search_exclude_tags, - search_exclude_tags_length, output, print_lastmod); + ret = count_file (notmuch, input, exclude_tags, output, print_lastmod); else - ret = print_count (notmuch, query_str, search_exclude_tags, - search_exclude_tags_length, output, print_lastmod); + ret = print_count (notmuch, query_str, exclude_tags, output, print_lastmod); notmuch_database_destroy (notmuch); diff --git a/notmuch-dump.c b/notmuch-dump.c index 65e02639..cb82d61f 100644 --- a/notmuch-dump.c +++ b/notmuch-dump.c @@ -21,7 +21,7 @@ #include "notmuch-client.h" #include "hex-escape.h" #include "string-util.h" -#include +#include "zlib-extra.h" static int database_dump_config (notmuch_database_t *notmuch, gzFile output) @@ -42,7 +42,7 @@ database_dump_config (notmuch_database_t *notmuch, gzFile output) notmuch_config_list_key (list)); goto DONE; } - gzprintf (output, "#@ %s", buffer); + GZPRINTF (output, "#@ %s", buffer); if (hex_encode (notmuch, notmuch_config_list_value (list), &buffer, &buffer_size) != HEX_SUCCESS) { @@ -51,7 +51,9 @@ database_dump_config (notmuch_database_t *notmuch, gzFile output) goto DONE; } - gzprintf (output, " %s\n", buffer); + GZPUTS (output, " "); + GZPUTS (output, buffer); + GZPUTS (output, "\n"); } ret = EXIT_SUCCESS; @@ -71,22 +73,22 @@ print_dump_header (gzFile output, int output_format, int include) { const char *sep = ""; - gzprintf (output, "#notmuch-dump %s:%d ", + GZPRINTF (output, "#notmuch-dump %s:%d ", (output_format == DUMP_FORMAT_SUP) ? "sup" : "batch-tag", NOTMUCH_DUMP_VERSION); if (include & DUMP_INCLUDE_CONFIG) { - gzputs (output, "config"); + GZPUTS (output, "config"); sep = ","; } if (include & DUMP_INCLUDE_PROPERTIES) { - gzprintf (output, "%sproperties", sep); + GZPRINTF (output, "%sproperties", sep); sep = ","; } if (include & DUMP_INCLUDE_TAGS) { - gzprintf (output, "%stags", sep); + GZPRINTF (output, "%stags", sep); } - gzputs (output, "\n"); + GZPUTS (output, "\n"); } static int @@ -115,7 +117,7 @@ dump_properties_message (void *ctx, fprintf (stderr, "Error: failed to hex-encode message-id %s\n", message_id); return 1; } - gzprintf (output, "#= %s", *buffer_p); + GZPRINTF (output, "#= %s", *buffer_p); first = false; } @@ -126,18 +128,18 @@ dump_properties_message (void *ctx, fprintf (stderr, "Error: failed to hex-encode key %s\n", key); return 1; } - gzprintf (output, " %s", *buffer_p); + GZPRINTF (output, " %s", *buffer_p); if (hex_encode (ctx, val, buffer_p, size_p) != HEX_SUCCESS) { fprintf (stderr, "Error: failed to hex-encode value %s\n", val); return 1; } - gzprintf (output, "=%s", *buffer_p); + GZPRINTF (output, "=%s", *buffer_p); } notmuch_message_properties_destroy (list); if (! first) - gzprintf (output, "\n", *buffer_p); + GZPRINTF (output, "\n", *buffer_p); return 0; } @@ -165,7 +167,7 @@ dump_tags_message (void *ctx, } if (output_format == DUMP_FORMAT_SUP) { - gzprintf (output, "%s (", message_id); + GZPRINTF (output, "%s (", message_id); } for (notmuch_tags_t *tags = notmuch_message_get_tags (message); @@ -174,12 +176,12 @@ dump_tags_message (void *ctx, const char *tag_str = notmuch_tags_get (tags); if (! first) - gzputs (output, " "); + GZPUTS (output, " "); first = 0; if (output_format == DUMP_FORMAT_SUP) { - gzputs (output, tag_str); + GZPUTS (output, tag_str); } else { if (hex_encode (ctx, tag_str, buffer_p, size_p) != HEX_SUCCESS) { @@ -187,12 +189,12 @@ dump_tags_message (void *ctx, tag_str); return EXIT_FAILURE; } - gzprintf (output, "+%s", *buffer_p); + GZPRINTF (output, "+%s", *buffer_p); } } if (output_format == DUMP_FORMAT_SUP) { - gzputs (output, ")\n"); + GZPUTS (output, ")\n"); } else { if (make_boolean_term (ctx, "id", message_id, buffer_p, size_p)) { @@ -200,7 +202,7 @@ dump_tags_message (void *ctx, message_id, strerror (errno)); return EXIT_FAILURE; } - gzprintf (output, " -- %s\n", *buffer_p); + GZPRINTF (output, " -- %s\n", *buffer_p); } return EXIT_SUCCESS; } @@ -230,11 +232,12 @@ database_dump_file (notmuch_database_t *notmuch, gzFile output, if (! query_str) query_str = ""; - query = notmuch_query_create (notmuch, query_str); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); + status = notmuch_query_create_with_syntax (notmuch, query_str, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch dump", notmuch, status)) return EXIT_FAILURE; - } + /* Don't ask xapian to sort by Message-ID. Xapian optimizes returning the * first results quickly at the expense of total time. */ @@ -316,7 +319,7 @@ notmuch_database_dump (notmuch_database_t *notmuch, ret = gzflush (output, Z_FINISH); if (ret) { - fprintf (stderr, "Error flushing output: %s\n", gzerror (output, NULL)); + fprintf (stderr, "Error flushing output: %s\n", gzerror_str (output)); goto DONE; } @@ -332,12 +335,12 @@ notmuch_database_dump (notmuch_database_t *notmuch, ret = gzclose_w (output); if (ret) { fprintf (stderr, "Error closing %s: %s\n", name_for_error, - gzerror (output, NULL)); + gzerror_str (output)); ret = EXIT_FAILURE; output = NULL; goto DONE; } else - output = NULL; + output = NULL; if (output_file_name) { ret = rename (tempname, output_file_name); @@ -359,18 +362,11 @@ notmuch_database_dump (notmuch_database_t *notmuch, } int -notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_dump_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; const char *query_str = NULL; int ret; - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) - return EXIT_FAILURE; - - notmuch_exit_if_unmatched_db_uuid (notmuch); - const char *output_file_name = NULL; int opt_index; @@ -397,7 +393,7 @@ notmuch_dump_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); if (include == 0) include = DUMP_INCLUDE_CONFIG | DUMP_INCLUDE_TAGS | DUMP_INCLUDE_PROPERTIES; diff --git a/notmuch-git.py b/notmuch-git.py new file mode 100644 index 00000000..97073c80 --- /dev/null +++ b/notmuch-git.py @@ -0,0 +1,1217 @@ +#!/usr/bin/env python3 +# +# Copyright (c) 2011-2014 David Bremner +# W. Trevor King +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see https://www.gnu.org/licenses/ . + +""" +Manage notmuch tags with Git +""" + +from __future__ import print_function +from __future__ import unicode_literals + +import codecs as _codecs +import collections as _collections +import functools as _functools +import inspect as _inspect +import locale as _locale +import logging as _logging +import os as _os +import re as _re +import subprocess as _subprocess +import sys as _sys +import tempfile as _tempfile +import textwrap as _textwrap +from urllib.parse import quote as _quote +from urllib.parse import unquote as _unquote +import json as _json + +_LOG = _logging.getLogger('notmuch-git') +_LOG.setLevel(_logging.WARNING) +_LOG.addHandler(_logging.StreamHandler()) + +NOTMUCH_GIT_DIR = None +TAG_PREFIX = None +FORMAT_VERSION = 1 + +_HEX_ESCAPE_REGEX = _re.compile('%[0-9A-F]{2}') +_TAG_DIRECTORY = 'tags/' +_TAG_FILE_REGEX = ( _re.compile(_TAG_DIRECTORY + '(?P[^/]*)/(?P[^/]*)'), + _re.compile(_TAG_DIRECTORY + '([0-9a-f]{2}/){2}(?P[^/]*)/(?P[^/]*)')) + +# magic hash for Git (git hash-object -t blob /dev/null) +_EMPTYBLOB = 'e69de29bb2d1d6434b8b29ae775ad8c2e48c5391' + +def _hex_quote(string, safe='+@=:,'): + """ + quote('abc def') -> 'abc%20def'. + + Wrap urllib.parse.quote with additional safe characters (in + addition to letters, digits, and '_.-') and lowercase hex digits + (e.g. '%3a' instead of '%3A'). + """ + uppercase_escapes = _quote(string, safe) + return _HEX_ESCAPE_REGEX.sub( + lambda match: match.group(0).lower(), + uppercase_escapes) + +def _xapian_quote(string): + """ + Quote a string for Xapian's QueryParser. + + Xapian uses double-quotes for quoting strings. You can escape + internal quotes by repeating them [1,2,3]. + + [1]: https://trac.xapian.org/ticket/128#comment:2 + [2]: https://trac.xapian.org/ticket/128#comment:17 + [3]: https://trac.xapian.org/changeset/13823/svn + """ + return '"{0}"'.format(string.replace('"', '""')) + + +def _xapian_unquote(string): + """ + Unquote a Xapian-quoted string. + """ + if string.startswith('"') and string.endswith('"'): + return string[1:-1].replace('""', '"') + return string + + +def timed(fn): + """Timer decorator""" + from time import perf_counter + + def inner(*args, **kwargs): + start_time = perf_counter() + rval = fn(*args, **kwargs) + end_time = perf_counter() + _LOG.info('{0}: {1:.8f}s elapsed'.format(fn.__name__, end_time - start_time)) + return rval + + return inner + + +class SubprocessError(RuntimeError): + "A subprocess exited with a nonzero status" + def __init__(self, args, status, stdout=None, stderr=None): + self.status = status + self.stdout = stdout + self.stderr = stderr + msg = '{args} exited with {status}'.format(args=args, status=status) + if stderr: + msg = '{msg}: {stderr}'.format(msg=msg, stderr=stderr) + super(SubprocessError, self).__init__(msg) + + +class _SubprocessContextManager(object): + """ + PEP 343 context manager for subprocesses. + + 'expect' holds a tuple of acceptable exit codes, otherwise we'll + raise a SubprocessError in __exit__. + """ + def __init__(self, process, args, expect=(0,)): + self._process = process + self._args = args + self._expect = expect + + def __enter__(self): + return self._process + + def __exit__(self, type, value, traceback): + for name in ['stdin', 'stdout', 'stderr']: + stream = getattr(self._process, name) + if stream: + stream.close() + setattr(self._process, name, None) + status = self._process.wait() + _LOG.debug( + 'collect {args} with status {status} (expected {expect})'.format( + args=self._args, status=status, expect=self._expect)) + if status not in self._expect: + raise SubprocessError(args=self._args, status=status) + + def wait(self): + return self._process.wait() + + +def _spawn(args, input=None, additional_env=None, wait=False, stdin=None, + stdout=None, stderr=None, encoding=_locale.getpreferredencoding(), + expect=(0,), **kwargs): + """Spawn a subprocess, and optionally wait for it to finish. + + This wrapper around subprocess.Popen has two modes, depending on + the truthiness of 'wait'. If 'wait' is true, we use p.communicate + internally to write 'input' to the subprocess's stdin and read + from it's stdout/stderr. If 'wait' is False, we return a + _SubprocessContextManager instance for fancier handling + (e.g. piping between processes). + + For 'wait' calls when you want to write to the subprocess's stdin, + you only need to set 'input' to your content. When 'input' is not + None but 'stdin' is, we'll automatically set 'stdin' to PIPE + before calling Popen. This avoids having the subprocess + accidentally inherit the launching process's stdin. + """ + _LOG.debug('spawn {args} (additional env. var.: {env})'.format( + args=args, env=additional_env)) + if not stdin and input is not None: + stdin = _subprocess.PIPE + if additional_env: + if not kwargs.get('env'): + kwargs['env'] = dict(_os.environ) + kwargs['env'].update(additional_env) + p = _subprocess.Popen( + args, stdin=stdin, stdout=stdout, stderr=stderr, **kwargs) + if wait: + if hasattr(input, 'encode'): + input = input.encode(encoding) + (stdout, stderr) = p.communicate(input=input) + status = p.wait() + _LOG.debug( + 'collect {args} with status {status} (expected {expect})'.format( + args=args, status=status, expect=expect)) + if stdout is not None: + stdout = stdout.decode(encoding) + if stderr is not None: + stderr = stderr.decode(encoding) + if status not in expect: + raise SubprocessError( + args=args, status=status, stdout=stdout, stderr=stderr) + return (status, stdout, stderr) + if p.stdin and not stdin: + p.stdin.close() + p.stdin = None + if p.stdin: + p.stdin = _codecs.getwriter(encoding=encoding)(stream=p.stdin) + stream_reader = _codecs.getreader(encoding=encoding) + if p.stdout: + p.stdout = stream_reader(stream=p.stdout) + if p.stderr: + p.stderr = stream_reader(stream=p.stderr) + return _SubprocessContextManager(args=args, process=p, expect=expect) + + +def _git(args, **kwargs): + args = ['git', '--git-dir', NOTMUCH_GIT_DIR] + list(args) + return _spawn(args=args, **kwargs) + + +def _get_current_branch(): + """Get the name of the current branch. + + Return 'None' if we're not on a branch. + """ + try: + (status, branch, stderr) = _git( + args=['symbolic-ref', '--short', 'HEAD'], + stdout=_subprocess.PIPE, stderr=_subprocess.PIPE, wait=True) + except SubprocessError as e: + if 'not a symbolic ref' in e: + return None + raise + return branch.strip() + + +def _get_remote(): + "Get the default remote for the current branch." + local_branch = _get_current_branch() + (status, remote, stderr) = _git( + args=['config', 'branch.{0}.remote'.format(local_branch)], + stdout=_subprocess.PIPE, wait=True) + return remote.strip() + +def _tag_query(prefix=None): + if prefix is None: + prefix = TAG_PREFIX + return '(tag (starts-with "{:s}"))'.format(prefix.replace('"','\\\"')) + +def count_messages(prefix=None): + "count messages with a given prefix." + (status, stdout, stderr) = _spawn( + args=['notmuch', 'count', '--query=sexp', _tag_query(prefix)], + stdout=_subprocess.PIPE, wait=True) + if status != 0: + _LOG.error("failed to run notmuch config") + _sys.exit(1) + return int(stdout.rstrip()) + +def get_tags(prefix=None): + "Get a list of tags with a given prefix." + (status, stdout, stderr) = _spawn( + args=['notmuch', 'search', '--exclude=false', '--query=sexp', '--output=tags', _tag_query(prefix)], + stdout=_subprocess.PIPE, wait=True) + return [tag for tag in stdout.splitlines()] + +def archive(treeish='HEAD', args=()): + """ + Dump a tar archive of the current notmuch-git tag set. + + Using 'git archive'. + + Each tag $tag for message with Message-Id $id is written to + an empty file + + tags/hash1(id)/hash2(id)/encode($id)/encode($tag) + + The encoding preserves alphanumerics, and the characters + "+-_@=.:," (not the quotes). All other octets are replaced with + '%' followed by a two digit hex number. + """ + _git(args=['archive', treeish] + list(args), wait=True) + + +def clone(repository): + """ + Create a local notmuch-git repository from a remote source. + + This wraps 'git clone', adding some options to avoid creating a + working tree while preserving remote-tracking branches and + upstreams. + """ + with _tempfile.TemporaryDirectory(prefix='notmuch-git-clone.') as workdir: + _spawn( + args=[ + 'git', 'clone', '--no-checkout', '--separate-git-dir', NOTMUCH_GIT_DIR, + repository, workdir], + wait=True) + _git(args=['config', '--unset', 'core.worktree'], wait=True, expect=(0, 5)) + _git(args=['config', 'core.bare', 'true'], wait=True) + (status, stdout, stderr) = _git(args=['show-ref', '--verify', + '--quiet', + 'refs/remotes/origin/config'], + expect=(0,1), + wait=True) + if status == 0: + _git(args=['branch', 'config', 'origin/config'], wait=True) + existing_tags = get_tags() + if existing_tags: + _LOG.warning( + 'Not checking out to avoid clobbering existing tags: {}'.format( + ', '.join(existing_tags))) + else: + checkout() + + +def _is_committed(status): + return len(status['added']) + len(status['deleted']) == 0 + + +class CachedIndex: + def __init__(self, repo, treeish): + self.cache_path = _os.path.join(repo, 'notmuch', 'index_cache.json') + self.index_path = _os.path.join(repo, 'index') + self.current_treeish = treeish + # cached values + self.treeish = None + self.hash = None + self.index_checksum = None + + self._load_cache_file() + + def _load_cache_file(self): + try: + with open(self.cache_path) as f: + data = _json.load(f) + self.treeish = data['treeish'] + self.hash = data['hash'] + self.index_checksum = data['index_checksum'] + except FileNotFoundError: + pass + except _json.JSONDecodeError: + _LOG.error("Error decoding cache") + _sys.exit(1) + + def __enter__(self): + self.read_tree() + return self + + def __exit__(self, type, value, traceback): + checksum = _read_index_checksum(self.index_path) + (_, hash, _) = _git( + args=['rev-parse', self.current_treeish], + stdout=_subprocess.PIPE, + wait=True) + + with open(self.cache_path, "w") as f: + _json.dump({'treeish': self.current_treeish, + 'hash': hash.rstrip(), 'index_checksum': checksum }, f) + + @timed + def read_tree(self): + current_checksum = _read_index_checksum(self.index_path) + (_, hash, _) = _git( + args=['rev-parse', self.current_treeish], + stdout=_subprocess.PIPE, + wait=True) + current_hash = hash.rstrip() + + if self.current_treeish == self.treeish and \ + self.index_checksum and self.index_checksum == current_checksum and \ + self.hash and self.hash == current_hash: + return + + _git(args=['read-tree', self.current_treeish], wait=True) + + +def check_safe_fraction(status): + safe = 0.1 + conf = _notmuch_config_get ('git.safe_fraction') + if conf and conf != '': + safe=float(conf) + + total = count_messages (TAG_PREFIX) + if total == 0: + _LOG.error('No existing tags with given prefix, stopping.') + _LOG.error('Use --force to override.') + exit(1) + change = len(status['added'])+len(status['deleted']) + fraction = change/total + _LOG.debug('total messages {:d}, change: {:d}, fraction: {:f}'.format(total,change,fraction)) + if fraction > safe: + _LOG.error('safe fraction {:f} exceeded, stopping.'.format(safe)) + _LOG.error('Use --force to override or reconfigure git.safe_fraction.') + exit(1) + +def commit(treeish='HEAD', message=None, force=False): + """ + Commit prefix-matching tags from the notmuch database to Git. + """ + + status = get_status() + + if _is_committed(status=status): + _LOG.warning('Nothing to commit') + return + + if not force: + check_safe_fraction (status) + + with CachedIndex(NOTMUCH_GIT_DIR, treeish) as index: + try: + _update_index(status=status) + (_, tree, _) = _git( + args=['write-tree'], + stdout=_subprocess.PIPE, + wait=True) + (_, parent, _) = _git( + args=['rev-parse', treeish], + stdout=_subprocess.PIPE, + wait=True) + (_, commit, _) = _git( + args=['commit-tree', tree.strip(), '-p', parent.strip()], + input=message, + stdout=_subprocess.PIPE, + wait=True) + _git( + args=['update-ref', treeish, commit.strip()], + stdout=_subprocess.PIPE, + wait=True) + except Exception as e: + _git(args=['read-tree', '--empty'], wait=True) + _git(args=['read-tree', treeish], wait=True) + raise + +@timed +def _update_index(status): + with _git( + args=['update-index', '--index-info'], + stdin=_subprocess.PIPE) as p: + for id, tags in status['deleted'].items(): + for line in _index_tags_for_message(id=id, status='D', tags=tags): + p.stdin.write(line) + for id, tags in status['added'].items(): + for line in _index_tags_for_message(id=id, status='A', tags=tags): + p.stdin.write(line) + + +def fetch(remote=None): + """ + Fetch changes from the remote repository. + + See 'merge' to bring those changes into notmuch. + """ + args = ['fetch'] + if remote: + args.append(remote) + _git(args=args, wait=True) + + +def init(remote=None,format_version=None): + """ + Create an empty notmuch-git repository. + + This wraps 'git init' with a few extra steps to support subsequent + status and commit commands. + """ + from pathlib import Path + parent = Path(NOTMUCH_GIT_DIR).parent + try: + _os.makedirs(parent) + except FileExistsError: + pass + + if not format_version: + format_version = 1 + + format_version=int(format_version) + + if format_version > 1 or format_version < 0: + _LOG.error("Illegal format version {:d}".format(format_version)) + _sys.exit(1) + + _spawn(args=['git', '--git-dir', NOTMUCH_GIT_DIR, 'init', + '--initial-branch=master', '--quiet', '--bare'], wait=True) + _git(args=['config', 'core.logallrefupdates', 'true'], wait=True) + # create an empty blob (e69de29bb2d1d6434b8b29ae775ad8c2e48c5391) + _git(args=['hash-object', '-w', '--stdin'], input='', wait=True) + allow_empty=('--allow-empty',) + if format_version >= 1: + allow_empty=() + # create a blob for the FORMAT file + (status, stdout, _) = _git(args=['hash-object', '-w', '--stdin'], stdout=_subprocess.PIPE, + input='{:d}\n'.format(format_version), wait=True) + verhash=stdout.rstrip() + _LOG.debug('hash of FORMAT blob = {:s}'.format(verhash)) + # Add FORMAT to the index + _git(args=['update-index', '--add', '--cacheinfo', '100644,{:s},FORMAT'.format(verhash)], wait=True) + + _git( + args=[ + 'commit', *allow_empty, '-m', 'Start a new notmuch-git repository' + ], + additional_env={'GIT_WORK_TREE': NOTMUCH_GIT_DIR}, + wait=True) + + +def checkout(force=None): + """ + Update the notmuch database from Git. + + This is mainly useful to discard your changes in notmuch relative + to Git. + """ + status = get_status() + + if not force: + check_safe_fraction(status) + + with _spawn( + args=['notmuch', 'tag', '--batch'], stdin=_subprocess.PIPE) as p: + for id, tags in status['added'].items(): + p.stdin.write(_batch_line(action='-', id=id, tags=tags)) + for id, tags in status['deleted'].items(): + p.stdin.write(_batch_line(action='+', id=id, tags=tags)) + + +def _batch_line(action, id, tags): + """ + 'notmuch tag --batch' line for adding/removing tags. + + Set 'action' to '-' to remove a tag or '+' to add the tags to a + given message id. + """ + tag_string = ' '.join( + '{action}{prefix}{tag}'.format( + action=action, prefix=_ENCODED_TAG_PREFIX, tag=_hex_quote(tag)) + for tag in tags) + line = '{tags} -- id:{id}\n'.format( + tags=tag_string, id=_xapian_quote(string=id)) + return line + + +def _insist_committed(): + "Die if the the notmuch tags don't match the current HEAD." + status = get_status() + if not _is_committed(status=status): + _LOG.error('\n'.join([ + 'Uncommitted changes to {prefix}* tags in notmuch', + '', + "For a summary of changes, run 'notmuch-git status'", + "To save your changes, run 'notmuch-git commit' before merging/pull", + "To discard your changes, run 'notmuch-git checkout'", + ]).format(prefix=TAG_PREFIX)) + _sys.exit(1) + + +def pull(repository=None, refspecs=None): + """ + Pull (merge) remote repository changes to notmuch. + + 'pull' is equivalent to 'fetch' followed by 'merge'. We use the + Git-configured repository for your current branch + (branch..repository, likely 'origin', and + branch..merge, likely 'master'). + """ + _insist_committed() + if refspecs and not repository: + repository = _get_remote() + args = ['pull'] + if repository: + args.append(repository) + if refspecs: + args.extend(refspecs) + with _tempfile.TemporaryDirectory(prefix='notmuch-git-pull.') as workdir: + for command in [ + ['reset', '--hard'], + args]: + _git( + args=command, + additional_env={'GIT_WORK_TREE': workdir}, + wait=True) + checkout() + + +def merge(reference='@{upstream}'): + """ + Merge changes from 'reference' into HEAD and load the result into notmuch. + + The default reference is '@{upstream}'. + """ + _insist_committed() + with _tempfile.TemporaryDirectory(prefix='notmuch-git-merge.') as workdir: + for command in [ + ['reset', '--hard'], + ['merge', reference]]: + _git( + args=command, + additional_env={'GIT_WORK_TREE': workdir}, + wait=True) + checkout() + + +def log(args=()): + """ + A simple wrapper for 'git log'. + + After running 'notmuch-git fetch', you can inspect the changes with + 'notmuch-git log HEAD..@{upstream}'. + """ + # we don't want output trapping here, because we want the pager. + args = ['log', '--name-status', '--no-renames'] + list(args) + with _git(args=args, expect=(0, 1, -13)) as p: + p.wait() + + +def push(repository=None, refspecs=None): + "Push the local notmuch-git Git state to a remote repository." + if refspecs and not repository: + repository = _get_remote() + args = ['push'] + if repository: + args.append(repository) + if refspecs: + args.extend(refspecs) + _git(args=args, wait=True) + + +def status(): + """ + Show pending updates in notmuch or git repo. + + Prints lines of the form + + ng Message-Id tag + + where n is a single character representing notmuch database status + + * A + + Tag is present in notmuch database, but not committed to notmuch-git + (equivalently, tag has been deleted in notmuch-git repo, e.g. by a + pull, but not restored to notmuch database). + + * D + + Tag is present in notmuch-git repo, but not restored to notmuch + database (equivalently, tag has been deleted in notmuch). + + * U + + Message is unknown (missing from local notmuch database). + + The second character (if present) represents a difference between + local and upstream branches. Typically 'notmuch-git fetch' needs to be + run to update this. + + * a + + Tag is present in upstream, but not in the local Git branch. + + * d + + Tag is present in local Git branch, but not upstream. + """ + status = get_status() + # 'output' is a nested defaultdict for message status: + # * The outer dict is keyed by message id. + # * The inner dict is keyed by tag name. + # * The inner dict values are status strings (' a', 'Dd', ...). + output = _collections.defaultdict( + lambda : _collections.defaultdict(lambda : ' ')) + for id, tags in status['added'].items(): + for tag in tags: + output[id][tag] = 'A' + for id, tags in status['deleted'].items(): + for tag in tags: + output[id][tag] = 'D' + for id, tags in status['missing'].items(): + for tag in tags: + output[id][tag] = 'U' + if _is_unmerged(): + for id, tag in _diff_refs(filter='A'): + output[id][tag] += 'a' + for id, tag in _diff_refs(filter='D'): + output[id][tag] += 'd' + for id, tag_status in sorted(output.items()): + for tag, status in sorted(tag_status.items()): + print('{status}\t{id}\t{tag}'.format( + status=status, id=id, tag=tag)) + + +def _is_unmerged(ref='@{upstream}'): + try: + (status, fetch_head, stderr) = _git( + args=['rev-parse', ref], + stdout=_subprocess.PIPE, stderr=_subprocess.PIPE, wait=True) + except SubprocessError as e: + if 'No upstream configured' in e.stderr: + return + raise + (status, base, stderr) = _git( + args=['merge-base', 'HEAD', ref], + stdout=_subprocess.PIPE, wait=True) + return base != fetch_head + +class DatabaseCache: + def __init__(self): + try: + from notmuch2 import Database + self._notmuch = Database() + except ImportError: + self._notmuch = None + self._known = {} + + def known(self,id): + if id in self._known: + return self._known[id]; + + if self._notmuch: + try: + _ = self._notmuch.find(id) + self._known[id] = True + except LookupError: + self._known[id] = False + else: + (_, stdout, stderr) = _spawn( + args=['notmuch', 'search', '--exclude=false', '--output=files', 'id:{0}'.format(id)], + stdout=_subprocess.PIPE, + wait=True) + self._known[id] = stdout != None + return self._known[id] + +@timed +def get_status(): + status = { + 'deleted': {}, + 'missing': {}, + } + db = DatabaseCache() + with PrivateIndex(repo=NOTMUCH_GIT_DIR, prefix=TAG_PREFIX) as index: + maybe_deleted = index.diff(filter='D') + for id, tags in maybe_deleted.items(): + if db.known(id): + status['deleted'][id] = tags + else: + status['missing'][id] = tags + status['added'] = index.diff(filter='A') + + return status + +class PrivateIndex: + def __init__(self, repo, prefix): + try: + _os.makedirs(_os.path.join(repo, 'notmuch')) + except FileExistsError: + pass + + file_name = 'notmuch/index' + self.index_path = _os.path.join(repo, file_name) + self.cache_path = _os.path.join(repo, 'notmuch', '{:s}.json'.format(_hex_quote(file_name))) + + self.current_prefix = prefix + + self.prefix = None + self.uuid = None + self.lastmod = None + self.checksum = None + self._load_cache_file() + self.file_tree = None + self._index_tags() + + def __enter__(self): + return self + + def __exit__(self, type, value, traceback): + checksum = _read_index_checksum(self.index_path) + (count, uuid, lastmod) = _read_database_lastmod() + with open(self.cache_path, "w") as f: + _json.dump({'prefix': self.current_prefix, 'uuid': uuid, 'lastmod': lastmod, 'checksum': checksum }, f) + + def _load_cache_file(self): + try: + with open(self.cache_path) as f: + data = _json.load(f) + self.prefix = data['prefix'] + self.uuid = data['uuid'] + self.lastmod = data['lastmod'] + self.checksum = data['checksum'] + except FileNotFoundError: + return None + except _json.JSONDecodeError: + _LOG.error("Error decoding cache") + _sys.exit(1) + + @timed + def _read_file_tree(self): + self.file_tree = {} + + with _git( + args=['ls-files', 'tags'], + additional_env={'GIT_INDEX_FILE': self.index_path}, + stdout=_subprocess.PIPE) as git: + for file in git.stdout: + dir=_os.path.dirname(file) + tag=_os.path.basename(file).rstrip() + if dir not in self.file_tree: + self.file_tree[dir]=[tag] + else: + self.file_tree[dir].append(tag) + + + def _clear_tags_for_message(self, id): + """ + Clear any existing index entries for message 'id' + + Neither 'id' nor the tags in 'tags' should be encoded/escaped. + """ + + if self.file_tree == None: + self._read_file_tree() + + dir = _id_path(id) + + if dir not in self.file_tree: + return + + for file in self.file_tree[dir]: + line = '0 0000000000000000000000000000000000000000\t{:s}/{:s}\n'.format(dir,file) + yield line + + + @timed + def _index_tags(self): + "Write notmuch tags to private git index." + prefix = '+{0}'.format(_ENCODED_TAG_PREFIX) + current_checksum = _read_index_checksum(self.index_path) + if (self.prefix == None or self.prefix != self.current_prefix + or self.checksum == None or self.checksum != current_checksum): + _git( + args=['read-tree', '--empty'], + additional_env={'GIT_INDEX_FILE': self.index_path}, wait=True) + + query = _tag_query() + clear_tags = False + (count,uuid,lastmod) = _read_database_lastmod() + if self.prefix == self.current_prefix and self.uuid \ + and self.uuid == uuid and self.checksum == current_checksum: + query = '(and (infix "lastmod:{:d}..")) {:s})'.format(self.lastmod+1, query) + clear_tags = True + with _spawn( + args=['notmuch', 'dump', '--format=batch-tag', '--query=sexp', '--', query], + stdout=_subprocess.PIPE) as notmuch: + with _git( + args=['update-index', '--index-info'], + stdin=_subprocess.PIPE, + additional_env={'GIT_INDEX_FILE': self.index_path}) as git: + for line in notmuch.stdout: + if line.strip().startswith('#'): + continue + (tags_string, id) = [_.strip() for _ in line.split(' -- id:')] + tags = [ + _unquote(tag[len(prefix):]) + for tag in tags_string.split() + if tag.startswith(prefix)] + id = _xapian_unquote(string=id) + if clear_tags: + for line in self._clear_tags_for_message(id=id): + git.stdin.write(line) + for line in _index_tags_for_message( + id=id, status='A', tags=tags): + git.stdin.write(line) + + @timed + def diff(self, filter): + """ + Get an {id: {tag, ...}} dict for a given filter. + + For example, use 'A' to find added tags, and 'D' to find deleted tags. + """ + s = _collections.defaultdict(set) + with _git( + args=[ + 'diff-index', '--cached', '--diff-filter', filter, + '--name-only', 'HEAD'], + additional_env={'GIT_INDEX_FILE': self.index_path}, + stdout=_subprocess.PIPE) as p: + # Once we drop Python < 3.3, we can use 'yield from' here + for id, tag in _unpack_diff_lines(stream=p.stdout): + s[id].add(tag) + return s + +def _read_index_checksum (index_path): + """Read the index checksum, as defined by index-format.txt in the git source + WARNING: assumes SHA1 repo""" + import binascii + try: + with open(index_path, 'rb') as f: + size=_os.path.getsize(index_path) + f.seek(size-20); + return binascii.hexlify(f.read(20)).decode('ascii') + except FileNotFoundError: + return None + +def _read_database_lastmod(): + with _spawn( + args=['notmuch', 'count', '--lastmod', '*'], + stdout=_subprocess.PIPE) as notmuch: + (count,uuid,lastmod_str) = notmuch.stdout.readline().split() + return (count,uuid,int(lastmod_str)) + +def _id_path(id): + hid=_hex_quote(string=id) + from hashlib import blake2b + + if FORMAT_VERSION==0: + return 'tags/{hid}'.format(hid=hid) + elif FORMAT_VERSION==1: + idhash = blake2b(hid.encode('utf8'), digest_size=2).hexdigest() + return 'tags/{dir1}/{dir2}/{hid}'.format( + hid=hid, + dir1=idhash[0:2],dir2=idhash[2:]) + else: + _LOG.error("Unknown format version",FORMAT_VERSION) + _sys.exit(1) + +def _index_tags_for_message(id, status, tags): + """ + Update the Git index to either create or delete an empty file. + + Neither 'id' nor the tags in 'tags' should be encoded/escaped. + """ + mode = '100644' + hash = _EMPTYBLOB + + if status == 'D': + mode = '0' + hash = '0000000000000000000000000000000000000000' + + for tag in tags: + path = '{ipath}/{tag}'.format(ipath=_id_path(id),tag=_hex_quote(string=tag)) + yield '{mode} {hash}\t{path}\n'.format(mode=mode, hash=hash, path=path) + + +def _diff_refs(filter, a='HEAD', b='@{upstream}'): + with _git( + args=['diff', '--diff-filter', filter, '--name-only', a, b], + stdout=_subprocess.PIPE) as p: + # Once we drop Python < 3.3, we can use 'yield from' here + for id, tag in _unpack_diff_lines(stream=p.stdout): + yield id, tag + + +def _unpack_diff_lines(stream): + "Iterate through (id, tag) tuples in a diff stream." + for line in stream: + match = _TAG_FILE_REGEX[FORMAT_VERSION].match(line.strip()) + if not match: + message = 'non-tag line in diff: {!r}'.format(line.strip()) + if line.startswith(_TAG_DIRECTORY): + raise ValueError(message) + _LOG.info(message) + continue + id = _unquote(match.group('id')) + tag = _unquote(match.group('tag')) + yield (id, tag) + + +def _help(parser, command=None): + """ + Show help for an notmuch-git command. + + Because some folks prefer: + + $ notmuch-git help COMMAND + + to + + $ notmuch-git COMMAND --help + """ + if command: + parser.parse_args([command, '--help']) + else: + parser.parse_args(['--help']) + +def _notmuch_config_get(key): + (status, stdout, stderr) = _spawn( + args=['notmuch', 'config', 'get', key], + stdout=_subprocess.PIPE, wait=True) + if status != 0: + _LOG.error("failed to run notmuch config") + _sys.exit(1) + return stdout.rstrip() + +def read_format_version(): + try: + (status, stdout, stderr) = _git( + args=['cat-file', 'blob', 'master:FORMAT'], + stdout=_subprocess.PIPE, stderr=_subprocess.PIPE, wait=True) + except SubprocessError as e: + _LOG.debug("failed to read FORMAT file from git, assuming format version 0") + return 0 + + return int(stdout) + +# based on BaseDirectory.save_data_path from pyxdg (LGPL2+) +def xdg_data_path(profile): + resource = _os.path.join('notmuch',profile,'git') + assert not resource.startswith('/') + _home = _os.path.expanduser('~') + xdg_data_home = _os.environ.get('XDG_DATA_HOME') or \ + _os.path.join(_home, '.local', 'share') + path = _os.path.join(xdg_data_home, resource) + return path + +if __name__ == '__main__': + import argparse + + parser = argparse.ArgumentParser( + description=__doc__.strip(), + formatter_class=argparse.RawDescriptionHelpFormatter) + parser.add_argument( + '-C', '--git-dir', metavar='REPO', + help='Git repository to operate on.') + parser.add_argument( + '-p', '--tag-prefix', metavar='PREFIX', + default = None, + help='Prefix of tags to operate on.') + parser.add_argument( + '-N', '--nmbug', action='store_true', + help='Set defaults for --tag-prefix and --git-dir for the notmuch bug tracker') + parser.add_argument( + '-l', '--log-level', + choices=['critical', 'error', 'warning', 'info', 'debug'], + help='Log verbosity. Defaults to {!r}.'.format( + _logging.getLevelName(_LOG.level).lower())) + + help = _functools.partial(_help, parser=parser) + help.__doc__ = _help.__doc__ + subparsers = parser.add_subparsers( + title='commands', + description=( + 'For help on a particular command, run: ' + "'%(prog)s ... --help'.")) + for command in [ + 'archive', + 'checkout', + 'clone', + 'commit', + 'fetch', + 'help', + 'init', + 'log', + 'merge', + 'pull', + 'push', + 'status', + ]: + func = locals()[command] + doc = _textwrap.dedent(func.__doc__).strip().replace('%', '%%') + subparser = subparsers.add_parser( + command, + help=doc.splitlines()[0], + description=doc, + formatter_class=argparse.RawDescriptionHelpFormatter) + subparser.set_defaults(func=func) + if command == 'archive': + subparser.add_argument( + 'treeish', metavar='TREE-ISH', nargs='?', default='HEAD', + help=( + 'The tree or commit to produce an archive for. Defaults ' + "to 'HEAD'.")) + subparser.add_argument( + 'args', metavar='ARG', nargs='*', + help=( + "Argument passed through to 'git archive'. Set anything " + 'before , see git-archive(1) for details.')) + elif command == 'checkout': + subparser.add_argument( + '-f', '--force', action='store_true', + help='checkout a large fraction of tags.') + elif command == 'clone': + subparser.add_argument( + 'repository', + help=( + 'The (possibly remote) repository to clone from. See the ' + 'URLS section of git-clone(1) for more information on ' + 'specifying repositories.')) + elif command == 'commit': + subparser.add_argument( + '-f', '--force', action='store_true', + help='commit a large fraction of tags.') + subparser.add_argument( + 'message', metavar='MESSAGE', default='', nargs='?', + help='Text for the commit message.') + elif command == 'fetch': + subparser.add_argument( + 'remote', metavar='REMOTE', nargs='?', + help=( + 'Override the default configured in branch..remote ' + 'to fetch from a particular remote repository (e.g. ' + "'origin').")) + elif command == 'help': + subparser.add_argument( + 'command', metavar='COMMAND', nargs='?', + help='The command to show help for.') + elif command == 'init': + subparser.add_argument( + '--format-version', metavar='VERSION', + default = None, + help='create format VERSION repository.') + elif command == 'log': + subparser.add_argument( + 'args', metavar='ARG', nargs='*', + help="Additional argument passed through to 'git log'.") + elif command == 'merge': + subparser.add_argument( + 'reference', metavar='REFERENCE', default='@{upstream}', + nargs='?', + help=( + 'Reference, usually other branch heads, to merge into ' + "our branch. Defaults to '@{upstream}'.")) + elif command == 'pull': + subparser.add_argument( + 'repository', metavar='REPOSITORY', default=None, nargs='?', + help=( + 'The "remote" repository that is the source of the pull. ' + 'This parameter can be either a URL (see the section GIT ' + 'URLS in git-pull(1)) or the name of a remote (see the ' + 'section REMOTES in git-pull(1)).')) + subparser.add_argument( + 'refspecs', metavar='REFSPEC', default=None, nargs='*', + help=( + 'Refspec (usually a branch name) to fetch and merge. See ' + 'the entry in the OPTIONS section of ' + 'git-pull(1) for other possibilities.')) + elif command == 'push': + subparser.add_argument( + 'repository', metavar='REPOSITORY', default=None, nargs='?', + help=( + 'The "remote" repository that is the destination of the ' + 'push. This parameter can be either a URL (see the ' + 'section GIT URLS in git-push(1)) or the name of a remote ' + '(see the section REMOTES in git-push(1)).')) + subparser.add_argument( + 'refspecs', metavar='REFSPEC', default=None, nargs='*', + help=( + 'Refspec (usually a branch name) to push. See ' + 'the entry in the OPTIONS section of ' + 'git-push(1) for other possibilities.')) + + args = parser.parse_args() + + nmbug_mode = False + notmuch_profile = _os.getenv('NOTMUCH_PROFILE','default') + + if args.nmbug or _os.path.basename(__file__) == 'nmbug': + nmbug_mode = True + + if args.git_dir: + NOTMUCH_GIT_DIR = args.git_dir + else: + if nmbug_mode: + default = _os.path.join('~', '.nmbug') + else: + default = _notmuch_config_get ('git.path') + if default == '': + default = xdg_data_path(notmuch_profile) + + NOTMUCH_GIT_DIR = _os.path.expanduser(_os.getenv('NOTMUCH_GIT_DIR', default)) + + _NOTMUCH_GIT_DIR = _os.path.join(NOTMUCH_GIT_DIR, '.git') + if _os.path.isdir(_NOTMUCH_GIT_DIR): + NOTMUCH_GIT_DIR = _NOTMUCH_GIT_DIR + + if args.tag_prefix: + TAG_PREFIX = args.tag_prefix + else: + if nmbug_mode: + prefix = 'notmuch::' + else: + prefix = _notmuch_config_get ('git.tag_prefix') + + TAG_PREFIX = _os.getenv('NOTMUCH_GIT_PREFIX', prefix) + + _ENCODED_TAG_PREFIX = _hex_quote(TAG_PREFIX, safe='+@=,') # quote ':' + + if args.log_level: + level = getattr(_logging, args.log_level.upper()) + _LOG.setLevel(level) + + # for test suite + for var in ['NOTMUCH_GIT_DIR', 'NOTMUCH_GIT_PREFIX', 'NOTMUCH_PROFILE', 'NOTMUCH_CONFIG' ]: + _LOG.debug('env {:s} = {:s}'.format(var, _os.getenv(var,'%None%'))) + + if _notmuch_config_get('built_with.sexp_queries') != 'true': + _LOG.error("notmuch git needs sexp query support") + _sys.exit(1) + + if not getattr(args, 'func', None): + parser.print_usage() + _sys.exit(1) + + # The following two lines are used by the test suite. + _LOG.debug('prefix = {:s}'.format(TAG_PREFIX)) + _LOG.debug('repository = {:s}'.format(NOTMUCH_GIT_DIR)) + + if args.func != init: + FORMAT_VERSION = read_format_version() + + _LOG.debug('FORMAT_VERSION={:d}'.format(FORMAT_VERSION)) + + if args.func == help: + arg_names = ['command'] + else: + (arg_names, varargs, varkw) = _inspect.getargs(args.func.__code__) + kwargs = {key: getattr(args, key) for key in arg_names if key in args} + try: + args.func(**kwargs) + except SubprocessError as e: + if _LOG.level == _logging.DEBUG: + raise # don't mask the traceback + _LOG.error(str(e)) + _sys.exit(1) diff --git a/notmuch-insert.c b/notmuch-insert.c index 1d3b0150..e44607ad 100644 --- a/notmuch-insert.c +++ b/notmuch-insert.c @@ -34,7 +34,7 @@ static volatile sig_atomic_t interrupted; static void handle_sigint (unused (int sig)) { - static char msg[] = "Stopping... \n"; + static const char msg[] = "Stopping... \n"; /* This write is "opportunistic", so it's okay to ignore the * result. It is not required for correctness, and if it does @@ -241,6 +241,26 @@ maildir_mktemp (const void *ctx, const char *maildir, bool world_readable, char return fd; } +static bool +write_buf (const char *buf, int fdout, ssize_t remain) +{ + const char *p = buf; + + do { + ssize_t written = write (fdout, p, remain); + if (written < 0 && errno == EINTR) + continue; + if (written <= 0) { + fprintf (stderr, "Error: writing to temporary file: %s", + strerror (errno)); + return false; + } + p += written; + remain -= written; + } while (remain > 0); + return true; +} + /* * Copy fdin to fdout, return true on success, and false on errors and * empty input. @@ -249,11 +269,13 @@ static bool copy_fd (int fdout, int fdin) { bool empty = true; + bool first = true; + const char *header = "X-Envelope-From: "; while (! interrupted) { ssize_t remain; char buf[4096]; - char *p; + const char *p = buf; remain = read (fdin, buf, sizeof (buf)); if (remain == 0) @@ -266,20 +288,18 @@ copy_fd (int fdout, int fdin) return false; } - p = buf; - do { - ssize_t written = write (fdout, p, remain); - if (written < 0 && errno == EINTR) - continue; - if (written <= 0) { - fprintf (stderr, "Error: writing to temporary file: %s", - strerror (errno)); + if (first && remain >= 5 && 0 == strncmp (buf, "From ", 5)) { + if (! write_buf (header, fdout, strlen (header))) return false; - } - p += written; - remain -= written; - empty = false; - } while (remain > 0); + p += 5; + remain -= 5; + } + + first = false; + + if (! write_buf (p, fdout, remain)) + return false; + empty = false; } return (! interrupted && ! empty); @@ -444,14 +464,12 @@ add_file (notmuch_database_t *notmuch, const char *path, tag_op_list_t *tag_ops, } int -notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_insert_command (notmuch_database_t *notmuch, int argc, char *argv[]) { notmuch_status_t status, close_status; - notmuch_database_t *notmuch; struct sigaction action; - const char *db_path; - const char **new_tags; - size_t new_tags_length; + const char *mail_root; + notmuch_config_values_t *new_tags = NULL; tag_op_list_t *tag_ops; char *query_string = NULL; const char *folder = ""; @@ -459,11 +477,13 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) bool keep = false; bool hooks = true; bool world_readable = false; - bool synchronize_flags; + notmuch_bool_t synchronize_flags; char *maildir; char *newpath; int opt_index; - unsigned int i; + notmuch_indexopts_t *indexopts = notmuch_database_get_default_indexopts (notmuch); + + void *local = talloc_new (NULL); notmuch_opt_desc_t options[] = { { .opt_string = &folder, .name = "folder", .allow_empty = true }, @@ -480,32 +500,41 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); + + mail_root = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); - db_path = notmuch_config_get_database_path (config); - new_tags = notmuch_config_get_new_tags (config, &new_tags_length); - synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config); + new_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_TAGS); - tag_ops = tag_op_list_create (config); + if (print_status_database ( + "notmuch insert", + notmuch, + notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, + &synchronize_flags))) + return EXIT_FAILURE; + + tag_ops = tag_op_list_create (local); if (tag_ops == NULL) { fprintf (stderr, "Out of memory.\n"); return EXIT_FAILURE; } - for (i = 0; i < new_tags_length; i++) { + for (; + notmuch_config_values_valid (new_tags); + notmuch_config_values_move_to_next (new_tags)) { const char *error_msg; - - error_msg = illegal_tag (new_tags[i], false); + const char *tag = notmuch_config_values_get (new_tags); + error_msg = illegal_tag (tag, false); if (error_msg) { fprintf (stderr, "Error: tag '%s' in new.tags: %s\n", - new_tags[i], error_msg); + tag, error_msg); return EXIT_FAILURE; } - if (tag_op_list_append (tag_ops, new_tags[i], false)) + if (tag_op_list_append (tag_ops, tag, false)) return EXIT_FAILURE; } - if (parse_tag_command_line (config, argc - opt_index, argv + opt_index, + if (parse_tag_command_line (local, argc - opt_index, argv + opt_index, &query_string, tag_ops)) return EXIT_FAILURE; @@ -519,14 +548,14 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - maildir = talloc_asprintf (config, "%s/%s", db_path, folder); + maildir = talloc_asprintf (local, "%s/%s", mail_root, folder); if (! maildir) { fprintf (stderr, "Out of memory\n"); return EXIT_FAILURE; } strip_trailing (maildir, '/'); - if (create_folder && ! maildir_create_folder (config, maildir, world_readable)) + if (create_folder && ! maildir_create_folder (local, maildir, world_readable)) return EXIT_FAILURE; /* Set up our handler for SIGINT. We do not set SA_RESTART so that copying @@ -538,19 +567,12 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) sigaction (SIGINT, &action, NULL); /* Write the message to the Maildir new directory. */ - newpath = maildir_write_new (config, STDIN_FILENO, maildir, world_readable); + newpath = maildir_write_new (local, STDIN_FILENO, maildir, world_readable); if (! newpath) { return EXIT_FAILURE; } - status = notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much); - if (status) - return keep ? NOTMUCH_STATUS_SUCCESS : status_to_exit (status); - - notmuch_exit_if_unmatched_db_uuid (notmuch); - - status = notmuch_process_shared_indexing_options (notmuch); + status = notmuch_process_shared_indexing_options (indexopts); if (status != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "Error: Failed to process index options. (%s)\n", notmuch_status_to_string (status)); @@ -558,10 +580,10 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) } /* Index the message. */ - status = add_file (notmuch, newpath, tag_ops, synchronize_flags, keep, indexing_cli_choices.opts); + status = add_file (notmuch, newpath, tag_ops, synchronize_flags, keep, indexopts); /* Commit changes. */ - close_status = notmuch_database_destroy (notmuch); + close_status = notmuch_database_close (notmuch); if (close_status) { /* Hold on to the first error, if any. */ if (! status) @@ -586,8 +608,12 @@ notmuch_insert_command (notmuch_config_t *config, int argc, char *argv[]) if (hooks && status == NOTMUCH_STATUS_SUCCESS) { /* Ignore hook failures. */ - notmuch_run_hook (db_path, "post-insert"); + notmuch_run_hook (notmuch, "post-insert"); } + notmuch_database_destroy (notmuch); + + talloc_free (local); + return status_to_exit (status); } diff --git a/notmuch-new.c b/notmuch-new.c index f079f62a..4a53e3eb 100644 --- a/notmuch-new.c +++ b/notmuch-new.c @@ -43,13 +43,14 @@ enum verbosity { typedef struct { const char *db_path; + const char *mail_root; + notmuch_indexopts_t *indexopts; int output_is_a_tty; enum verbosity verbosity; bool debug; bool full_scan; - const char **new_tags; - size_t new_tags_length; + notmuch_config_values_t *new_tags; const char **ignore_verbatim; size_t ignore_verbatim_length; regex_t *ignore_regex; @@ -65,7 +66,7 @@ typedef struct { _filename_list_t *removed_directories; _filename_list_t *directory_mtimes; - bool synchronize_flags; + notmuch_bool_t synchronize_flags; } add_files_state_t; static volatile sig_atomic_t do_print_progress = 0; @@ -81,7 +82,7 @@ static volatile sig_atomic_t interrupted; static void handle_sigint (unused (int sig)) { - static char msg[] = "Stopping... \n"; + static const char msg[] = "Stopping... \n"; /* This write is "opportunistic", so it's okay to ignore the * result. It is not required for correctness, and if it does @@ -245,19 +246,17 @@ _special_directory (const char *entry) } static bool -_setup_ignore (notmuch_config_t *config, add_files_state_t *state) +_setup_ignore (notmuch_database_t *notmuch, add_files_state_t *state) { - const char **ignore_list, **ignore; + notmuch_config_values_t *ignore_list; int nregex = 0, nverbatim = 0; const char **verbatim = NULL; regex_t *regex = NULL; - ignore_list = notmuch_config_get_new_ignore (config, NULL); - if (! ignore_list) - return true; - - for (ignore = ignore_list; *ignore; ignore++) { - const char *s = *ignore; + for (ignore_list = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_IGNORE); + notmuch_config_values_valid (ignore_list); + notmuch_config_values_move_to_next (ignore_list)) { + const char *s = notmuch_config_values_get (ignore_list); size_t len = strlen (s); if (len == 0) { @@ -276,8 +275,8 @@ _setup_ignore (notmuch_config_t *config, add_files_state_t *state) return false; } - r = talloc_strndup (config, s + 1, len - 2); - regex = talloc_realloc (config, regex, regex_t, nregex + 1); + r = talloc_strndup (notmuch, s + 1, len - 2); + regex = talloc_realloc (notmuch, regex, regex_t, nregex + 1); preg = ®ex[nregex]; rerr = regcomp (preg, r, REG_EXTENDED | REG_NOSUB); @@ -295,7 +294,7 @@ _setup_ignore (notmuch_config_t *config, add_files_state_t *state) talloc_free (r); } else { - verbatim = talloc_realloc (config, verbatim, const char *, + verbatim = talloc_realloc (notmuch, verbatim, const char *, nverbatim + 1); verbatim[nverbatim++] = s; } @@ -310,18 +309,18 @@ _setup_ignore (notmuch_config_t *config, add_files_state_t *state) } static char * -_get_relative_path (const char *db_path, const char *dirpath, const char *entry) +_get_relative_path (const char *mail_root, const char *dirpath, const char *entry) { - size_t db_path_len = strlen (db_path); + size_t mail_root_len = strlen (mail_root); /* paranoia? */ - if (strncmp (dirpath, db_path, db_path_len) != 0) { + if (strncmp (dirpath, mail_root, mail_root_len) != 0) { fprintf (stderr, "Warning: '%s' is not a subdirectory of '%s'\n", - dirpath, db_path); + dirpath, mail_root); return NULL; } - dirpath += db_path_len; + dirpath += mail_root_len; while (*dirpath == '/') dirpath++; @@ -349,7 +348,7 @@ _entry_in_ignore_list (add_files_state_t *state, const char *dirpath, if (state->ignore_regex_length == 0) return false; - path = _get_relative_path (state->db_path, dirpath, entry); + path = _get_relative_path (state->mail_root, dirpath, entry); if (! path) return false; @@ -371,14 +370,14 @@ add_file (notmuch_database_t *notmuch, const char *filename, add_files_state_t *state) { notmuch_message_t *message = NULL; - const char **tag; + const char *tag; notmuch_status_t status; status = notmuch_database_begin_atomic (notmuch); if (status) goto DONE; - status = notmuch_database_index_file (notmuch, filename, indexing_cli_choices.opts, &message); + status = notmuch_database_index_file (notmuch, filename, state->indexopts, &message); switch (status) { /* Success. */ case NOTMUCH_STATUS_SUCCESS: @@ -387,10 +386,17 @@ add_file (notmuch_database_t *notmuch, const char *filename, if (state->synchronize_flags) notmuch_message_maildir_flags_to_tags (message); - for (tag = state->new_tags; *tag != NULL; tag++) { - if (strcmp ("unread", *tag) != 0 || - ! notmuch_message_has_maildir_flag (message, 'S')) { - notmuch_message_add_tag (message, *tag); + for (notmuch_config_values_start (state->new_tags); + notmuch_config_values_valid (state->new_tags); + notmuch_config_values_move_to_next (state->new_tags)) { + notmuch_bool_t is_set; + + tag = notmuch_config_values_get (state->new_tags); + /* Currently all errors from has_maildir_flag are fatal */ + if ((status = notmuch_message_has_maildir_flag_st (message, 'S', &is_set))) + goto DONE; + if (strcmp ("unread", tag) != 0 || ! is_set) { + notmuch_message_add_tag (message, tag); } } @@ -398,12 +404,19 @@ add_file (notmuch_database_t *notmuch, const char *filename, break; /* Non-fatal issues (go on to next file). */ case NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID: - if (state->synchronize_flags) - notmuch_message_maildir_flags_to_tags (message); + if (state->synchronize_flags) { + status = notmuch_message_maildir_flags_to_tags (message); + if (print_status_message ("add_file", message, status)) + goto DONE; + } break; case NOTMUCH_STATUS_FILE_NOT_EMAIL: fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename); break; + case NOTMUCH_STATUS_PATH_ERROR: + fprintf (stderr, "Note: Ignoring non-indexable path: %s\n", filename); + (void) print_status_database ("add_file", notmuch, status); + break; case NOTMUCH_STATUS_FILE_ERROR: /* Someone renamed/removed the file between scandir and now. */ state->vanished_files++; @@ -592,11 +605,12 @@ add_files (notmuch_database_t *notmuch, continue; } - /* Ignore the .notmuch directory and any "tmp" directory + /* Ignore any top level .notmuch directory and any "tmp" directory * that appears within a maildir. */ if ((is_maildir && strcmp (entry->d_name, "tmp") == 0) || - strcmp (entry->d_name, ".notmuch") == 0) + (strcmp (entry->d_name, ".notmuch") == 0 + && (strcmp (path, state->mail_root)) == 0)) continue; next = talloc_asprintf (notmuch, "%s/%s", path, entry->d_name); @@ -669,8 +683,9 @@ add_files (notmuch_database_t *notmuch, char *absolute = talloc_asprintf (state->removed_directories, "%s/%s", path, filename); if (state->debug) - printf ("(D) add_files, pass 2: queuing passed directory %s for deletion from database\n", - absolute); + printf ( + "(D) add_files, pass 2: queuing passed directory %s for deletion from database\n", + absolute); _filename_list_add (state->removed_directories, absolute); } @@ -752,8 +767,9 @@ add_files (notmuch_database_t *notmuch, notmuch_filenames_get (db_subdirs)); if (state->debug) - printf ("(D) add_files, pass 3: queuing leftover directory %s for deletion from database\n", - absolute); + printf ( + "(D) add_files, pass 3: queuing leftover directory %s for deletion from database\n", + absolute); _filename_list_add (state->removed_directories, absolute); @@ -959,8 +975,7 @@ remove_filename (notmuch_database_t *notmuch, /* Recursively remove all filenames from the database referring to * 'path' (or to any of its children). */ static notmuch_status_t -_remove_directory (void *ctx, - notmuch_database_t *notmuch, +_remove_directory (notmuch_database_t *notmuch, const char *path, add_files_state_t *add_files_state) { @@ -976,7 +991,7 @@ _remove_directory (void *ctx, for (files = notmuch_directory_get_child_files (directory); notmuch_filenames_valid (files); notmuch_filenames_move_to_next (files)) { - absolute = talloc_asprintf (ctx, "%s/%s", path, + absolute = talloc_asprintf (notmuch, "%s/%s", path, notmuch_filenames_get (files)); status = remove_filename (notmuch, absolute, add_files_state); talloc_free (absolute); @@ -987,9 +1002,9 @@ _remove_directory (void *ctx, for (subdirs = notmuch_directory_get_child_directories (directory); notmuch_filenames_valid (subdirs); notmuch_filenames_move_to_next (subdirs)) { - absolute = talloc_asprintf (ctx, "%s/%s", path, + absolute = talloc_asprintf (notmuch, "%s/%s", path, notmuch_filenames_get (subdirs)); - status = _remove_directory (ctx, notmuch, absolute, add_files_state); + status = _remove_directory (notmuch, absolute, add_files_state); talloc_free (absolute); if (status) goto DONE; @@ -1039,10 +1054,67 @@ print_results (const add_files_state_t *state) printf ("\n"); } +static int +_maybe_upgrade (notmuch_database_t *notmuch, add_files_state_t *state) +{ + if (notmuch_database_needs_upgrade (notmuch)) { + time_t now = time (NULL); + struct tm *gm_time = gmtime (&now); + int err; + notmuch_status_t status; + const char *backup_dir = notmuch_config_get (notmuch, NOTMUCH_CONFIG_BACKUP_DIR); + const char *backup_name; + + err = mkdir (backup_dir, 0755); + if (err && errno != EEXIST) { + fprintf (stderr, "Failed to create %s: %s\n", backup_dir, strerror (errno)); + return EXIT_FAILURE; + } + + /* since dump files are written atomically, the amount of + * harm from overwriting one within a second seems + * relatively small. */ + backup_name = talloc_asprintf (notmuch, "%s/dump-%04d%02d%02dT%02d%02d%02d.gz", + backup_dir, + gm_time->tm_year + 1900, + gm_time->tm_mon + 1, + gm_time->tm_mday, + gm_time->tm_hour, + gm_time->tm_min, + gm_time->tm_sec); + + if (state->verbosity >= VERBOSITY_NORMAL) { + printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n"); + printf ("This process is safe to interrupt.\n"); + printf ("Backing up tags to %s...\n", backup_name); + } + + if (notmuch_database_dump (notmuch, backup_name, "", + DUMP_FORMAT_BATCH_TAG, DUMP_INCLUDE_DEFAULT, true)) { + fprintf (stderr, "Backup failed. Aborting upgrade."); + return EXIT_FAILURE; + } + + gettimeofday (&state->tv_start, NULL); + status = notmuch_database_upgrade ( + notmuch, + state->verbosity >= VERBOSITY_NORMAL ? upgrade_print_progress : NULL, + state); + if (status) { + printf ("Upgrade failed: %s\n", + notmuch_status_to_string (status)); + notmuch_database_destroy (notmuch); + return EXIT_FAILURE; + } + if (state->verbosity >= VERBOSITY_NORMAL) + printf ("Your notmuch database has now been upgraded.\n"); + } + return EXIT_SUCCESS; +} + int -notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_new_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; add_files_state_t add_files_state = { .verbosity = VERBOSITY_NORMAL, .debug = false, @@ -1051,9 +1123,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) }; struct timeval tv_start; int ret = 0; - struct stat st; - const char *db_path; - char *dot_notmuch_path; + const char *db_path, *mail_root; struct sigaction action; _filename_node_t *f; int opt_index; @@ -1078,7 +1148,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); /* quiet trumps verbose */ if (quiet) @@ -1086,103 +1156,68 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) else if (verbose) add_files_state.verbosity = VERBOSITY_VERBOSE; - add_files_state.new_tags = notmuch_config_get_new_tags (config, &add_files_state.new_tags_length); - add_files_state.synchronize_flags = notmuch_config_get_maildir_synchronize_flags (config); - db_path = notmuch_config_get_database_path (config); + add_files_state.indexopts = notmuch_database_get_default_indexopts (notmuch); + + add_files_state.new_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_TAGS); + + if (print_status_database ( + "notmuch new", + notmuch, + notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, + &add_files_state.synchronize_flags))) + return EXIT_FAILURE; + + db_path = notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH); add_files_state.db_path = db_path; - if (! _setup_ignore (config, &add_files_state)) + mail_root = notmuch_config_get (notmuch, NOTMUCH_CONFIG_MAIL_ROOT); + add_files_state.mail_root = mail_root; + + if (! _setup_ignore (notmuch, &add_files_state)) return EXIT_FAILURE; - for (i = 0; i < add_files_state.new_tags_length; i++) { - const char *error_msg; + for (notmuch_config_values_start (add_files_state.new_tags); + notmuch_config_values_valid (add_files_state.new_tags); + notmuch_config_values_move_to_next (add_files_state.new_tags)) { + const char *tag, *error_msg; - error_msg = illegal_tag (add_files_state.new_tags[i], false); + tag = notmuch_config_values_get (add_files_state.new_tags); + error_msg = illegal_tag (tag, false); if (error_msg) { - fprintf (stderr, "Error: tag '%s' in new.tags: %s\n", - add_files_state.new_tags[i], error_msg); + fprintf (stderr, "Error: tag '%s' in new.tags: %s\n", tag, error_msg); return EXIT_FAILURE; } } if (hooks) { - ret = notmuch_run_hook (db_path, "pre-new"); - if (ret) + /* Drop write lock to run hook */ + status = notmuch_database_reopen (notmuch, NOTMUCH_DATABASE_MODE_READ_ONLY); + if (print_status_database ("notmuch new", notmuch, status)) return EXIT_FAILURE; - } - dot_notmuch_path = talloc_asprintf (config, "%s/%s", db_path, ".notmuch"); + ret = notmuch_run_hook (notmuch, "pre-new"); + if (ret) + return EXIT_FAILURE; - if (stat (dot_notmuch_path, &st)) { - int count; + /* acquire write lock again */ + status = notmuch_database_reopen (notmuch, NOTMUCH_DATABASE_MODE_READ_WRITE); + if (print_status_database ("notmuch new", notmuch, status)) + return EXIT_FAILURE; + } - count = 0; - count_files (db_path, &count, &add_files_state); + if (notmuch_database_get_revision (notmuch, NULL) == 0) { + int count = 0; + count_files (mail_root, &count, &add_files_state); if (interrupted) return EXIT_FAILURE; if (add_files_state.verbosity >= VERBOSITY_NORMAL) printf ("Found %d total files (that's not much mail).\n", count); - if (notmuch_database_create (db_path, ¬much)) - return EXIT_FAILURE; + add_files_state.total_files = count; } else { - char *status_string = NULL; - if (notmuch_database_open_verbose (db_path, NOTMUCH_DATABASE_MODE_READ_WRITE, - ¬much, &status_string)) { - if (status_string) { - fputs (status_string, stderr); - free (status_string); - } + if (_maybe_upgrade (notmuch, &add_files_state)) return EXIT_FAILURE; - } - - notmuch_exit_if_unmatched_db_uuid (notmuch); - - if (notmuch_database_needs_upgrade (notmuch)) { - time_t now = time (NULL); - struct tm *gm_time = gmtime (&now); - - /* since dump files are written atomically, the amount of - * harm from overwriting one within a second seems - * relatively small. */ - - const char *backup_name = - talloc_asprintf (notmuch, "%s/dump-%04d%02d%02dT%02d%02d%02d.gz", - dot_notmuch_path, - gm_time->tm_year + 1900, - gm_time->tm_mon + 1, - gm_time->tm_mday, - gm_time->tm_hour, - gm_time->tm_min, - gm_time->tm_sec); - - if (add_files_state.verbosity >= VERBOSITY_NORMAL) { - printf ("Welcome to a new version of notmuch! Your database will now be upgraded.\n"); - printf ("This process is safe to interrupt.\n"); - printf ("Backing up tags to %s...\n", backup_name); - } - - if (notmuch_database_dump (notmuch, backup_name, "", - DUMP_FORMAT_BATCH_TAG, DUMP_INCLUDE_DEFAULT, true)) { - fprintf (stderr, "Backup failed. Aborting upgrade."); - return EXIT_FAILURE; - } - - gettimeofday (&add_files_state.tv_start, NULL); - status = notmuch_database_upgrade ( - notmuch, - add_files_state.verbosity >= VERBOSITY_NORMAL ? upgrade_print_progress : NULL, - &add_files_state); - if (status) { - printf ("Upgrade failed: %s\n", - notmuch_status_to_string (status)); - notmuch_database_destroy (notmuch); - return EXIT_FAILURE; - } - if (add_files_state.verbosity >= VERBOSITY_NORMAL) - printf ("Your notmuch database has now been upgraded.\n"); - } add_files_state.total_files = 0; } @@ -1190,7 +1225,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) if (notmuch == NULL) return EXIT_FAILURE; - status = notmuch_process_shared_indexing_options (notmuch); + status = notmuch_process_shared_indexing_options (add_files_state.indexopts); if (status != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "Error: Failed to process index options. (%s)\n", notmuch_status_to_string (status)); @@ -1206,14 +1241,11 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) action.sa_flags = SA_RESTART; sigaction (SIGINT, &action, NULL); - talloc_free (dot_notmuch_path); - dot_notmuch_path = NULL; - gettimeofday (&add_files_state.tv_start, NULL); - add_files_state.removed_files = _filename_list_create (config); - add_files_state.removed_directories = _filename_list_create (config); - add_files_state.directory_mtimes = _filename_list_create (config); + add_files_state.removed_files = _filename_list_create (notmuch); + add_files_state.removed_directories = _filename_list_create (notmuch); + add_files_state.directory_mtimes = _filename_list_create (notmuch); if (add_files_state.verbosity == VERBOSITY_NORMAL && add_files_state.output_is_a_tty && ! debugger_is_active ()) { @@ -1221,7 +1253,7 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) timer_is_active = true; } - ret = add_files (notmuch, db_path, &add_files_state); + ret = add_files (notmuch, mail_root, &add_files_state); if (ret) goto DONE; @@ -1233,14 +1265,15 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) if (do_print_progress) { do_print_progress = 0; generic_print_progress ("Cleaned up", "messages", - tv_start, add_files_state.removed_messages + add_files_state.renamed_messages, + tv_start, add_files_state.removed_messages + + add_files_state.renamed_messages, add_files_state.removed_files->count); } } gettimeofday (&tv_start, NULL); for (f = add_files_state.removed_directories->head, i = 0; f && ! interrupted; f = f->next, i++) { - ret = _remove_directory (config, notmuch, f->filename, &add_files_state); + ret = _remove_directory (notmuch, f->filename, &add_files_state); if (ret) goto DONE; if (do_print_progress) { @@ -1275,10 +1308,12 @@ notmuch_new_command (notmuch_config_t *config, int argc, char *argv[]) fprintf (stderr, "Note: A fatal error was encountered: %s\n", notmuch_status_to_string (ret)); - notmuch_database_destroy (notmuch); + notmuch_database_close (notmuch); if (hooks && ! ret && ! interrupted) - ret = notmuch_run_hook (db_path, "post-new"); + ret = notmuch_run_hook (notmuch, "post-new"); + + notmuch_database_destroy (notmuch); if (ret || interrupted) return EXIT_FAILURE; diff --git a/notmuch-reindex.c b/notmuch-reindex.c index 5a39ade1..e9a65456 100644 --- a/notmuch-reindex.c +++ b/notmuch-reindex.c @@ -26,7 +26,7 @@ static volatile sig_atomic_t interrupted; static void handle_sigint (unused (int sig)) { - static char msg[] = "Stopping... \n"; + static const char msg[] = "Stopping... \n"; /* This write is "opportunistic", so it's okay to ignore the * result. It is not required for correctness, and if it does @@ -49,11 +49,11 @@ reindex_query (notmuch_database_t *notmuch, const char *query_string, notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS; - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory.\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch reindex", notmuch, status)) return 1; - } /* reindexing is not interested in any special sort order */ notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED); @@ -83,14 +83,14 @@ reindex_query (notmuch_database_t *notmuch, const char *query_string, } int -notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_reindex_command (notmuch_database_t *notmuch, int argc, char *argv[]) { char *query_string = NULL; - notmuch_database_t *notmuch; struct sigaction action; int opt_index; int ret; notmuch_status_t status; + notmuch_indexopts_t *indexopts = notmuch_database_get_default_indexopts (notmuch); /* Set up our handler for SIGINT */ memset (&action, 0, sizeof (struct sigaction)); @@ -109,22 +109,16 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); - - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) - return EXIT_FAILURE; - - notmuch_exit_if_unmatched_db_uuid (notmuch); + notmuch_process_shared_options (notmuch, argv[0]); - status = notmuch_process_shared_indexing_options (notmuch); + status = notmuch_process_shared_indexing_options (indexopts); if (status != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "Error: Failed to process index options. (%s)\n", notmuch_status_to_string (status)); return EXIT_FAILURE; } - query_string = query_string_from_args (config, argc - opt_index, argv + opt_index); + query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index); if (query_string == NULL) { fprintf (stderr, "Out of memory\n"); return EXIT_FAILURE; @@ -135,7 +129,7 @@ notmuch_reindex_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - ret = reindex_query (notmuch, query_string, indexing_cli_choices.opts); + ret = reindex_query (notmuch, query_string, indexopts); notmuch_database_destroy (notmuch); diff --git a/notmuch-reply.c b/notmuch-reply.c index 2c30f6f9..44297251 100644 --- a/notmuch-reply.c +++ b/notmuch-reply.c @@ -65,8 +65,9 @@ format_part_reply (GMimeStream *stream, mime_node_t *node) GMimeContentDisposition *disposition = g_mime_object_get_content_disposition (node->part); if (g_mime_content_type_is_type (content_type, "application", "pgp-encrypted") || - g_mime_content_type_is_type (content_type, "application", "pgp-signature")) { - /* Ignore PGP/MIME cruft parts */ + g_mime_content_type_is_type (content_type, "application", "pgp-signature") || + g_mime_content_type_is_type (content_type, "application", "pkcs7-mime")) { + /* Ignore PGP/MIME and S/MIME cruft parts */ } else if (g_mime_content_type_is_type (content_type, "text", "*") && ! g_mime_content_type_is_type (content_type, "text", "html")) { show_text_part_content (node->part, stream, NOTMUCH_SHOW_TEXT_PART_REPLY); @@ -111,25 +112,26 @@ match_address (const char *str, const char *address, address_match_t mode) /* Match given string against user's configured "primary" and "other" * addresses according to mode. */ static const char * -address_match (const char *str, notmuch_config_t *config, address_match_t mode) +address_match (const char *str, notmuch_database_t *notmuch, address_match_t mode) { const char *primary; - const char **other; - size_t i, other_len; + notmuch_config_values_t *other = NULL; if (! str || *str == '\0') return NULL; - primary = notmuch_config_get_user_primary_email (config); + primary = notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL); if (match_address (str, primary, mode)) return primary; - other = notmuch_config_get_user_other_email (config, &other_len); - for (i = 0; i < other_len; i++) { - if (match_address (str, other[i], mode)) - return other[i]; - } + for (other = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_OTHER_EMAIL); + notmuch_config_values_valid (other); + notmuch_config_values_move_to_next (other)) { + const char *addr = notmuch_config_values_get (other); + if (match_address (str, addr, mode)) + return addr; + } return NULL; } @@ -137,26 +139,26 @@ address_match (const char *str, notmuch_config_t *config, address_match_t mode) * user's "primary" or "other" addresses. If so, return the matching * address, NULL otherwise. */ static const char * -user_address_in_string (const char *str, notmuch_config_t *config) +user_address_in_string (const char *str, notmuch_database_t *notmuch) { - return address_match (str, config, USER_ADDRESS_IN_STRING); + return address_match (str, notmuch, USER_ADDRESS_IN_STRING); } /* Do any of the addresses configured as one of the user's "primary" * or "other" addresses contain the given string. If so, return the * matching address, NULL otherwise. */ static const char * -string_in_user_address (const char *str, notmuch_config_t *config) +string_in_user_address (const char *str, notmuch_database_t *notmuch) { - return address_match (str, config, STRING_IN_USER_ADDRESS); + return address_match (str, notmuch, STRING_IN_USER_ADDRESS); } /* Is the given address configured as one of the user's "primary" or * "other" addresses. */ static bool -address_is_users (const char *address, notmuch_config_t *config) +address_is_users (const char *address, notmuch_database_t *notmuch) { - return address_match (address, config, STRING_IS_USER_ADDRESS) != NULL; + return address_match (address, notmuch, STRING_IS_USER_ADDRESS) != NULL; } /* Scan addresses in 'list'. @@ -174,7 +176,7 @@ address_is_users (const char *address, notmuch_config_t *config) */ static unsigned int scan_address_list (InternetAddressList *list, - notmuch_config_t *config, + notmuch_database_t *notmuch, GMimeMessage *message, GMimeAddressType type, const char **user_from) @@ -194,7 +196,7 @@ scan_address_list (InternetAddressList *list, group = INTERNET_ADDRESS_GROUP (address); group_list = internet_address_group_get_members (group); - n += scan_address_list (group_list, config, message, type, user_from); + n += scan_address_list (group_list, notmuch, message, type, user_from); } else { InternetAddressMailbox *mailbox; const char *name; @@ -205,7 +207,7 @@ scan_address_list (InternetAddressList *list, name = internet_address_get_name (address); addr = internet_address_mailbox_get_addr (mailbox); - if (address_is_users (addr, config)) { + if (address_is_users (addr, notmuch)) { if (user_from && *user_from == NULL) *user_from = addr; } else if (message) { @@ -323,7 +325,7 @@ get_bcc (GMimeMessage *message) */ static const char * add_recipients_from_message (GMimeMessage *reply, - notmuch_config_t *config, + notmuch_database_t *notmuch, GMimeMessage *message, bool reply_all) { @@ -345,7 +347,7 @@ add_recipients_from_message (GMimeMessage *reply, recipients = reply_to_map[i].get_header (message); - n += scan_address_list (recipients, config, reply, + n += scan_address_list (recipients, notmuch, reply, reply_to_map[i].recipient_type, &from_addr); if (! reply_all && n) { @@ -383,7 +385,7 @@ add_recipients_from_message (GMimeMessage *reply, * Return the address that was found, if any, and NULL otherwise. */ static const char * -guess_from_in_received_for (notmuch_config_t *config, const char *received) +guess_from_in_received_for (notmuch_database_t *notmuch, const char *received) { const char *ptr; @@ -391,7 +393,7 @@ guess_from_in_received_for (notmuch_config_t *config, const char *received) if (! ptr) return NULL; - return user_address_in_string (ptr, config); + return user_address_in_string (ptr, notmuch); } /* @@ -407,7 +409,7 @@ guess_from_in_received_for (notmuch_config_t *config, const char *received) * Return the address that was found, if any, and NULL otherwise. */ static const char * -guess_from_in_received_by (notmuch_config_t *config, const char *received) +guess_from_in_received_by (notmuch_database_t *notmuch, const char *received) { const char *addr; const char *by = received; @@ -445,7 +447,7 @@ guess_from_in_received_by (notmuch_config_t *config, const char *received) */ *(tld - 1) = '.'; - addr = string_in_user_address (domain, config); + addr = string_in_user_address (domain, notmuch); if (addr) { free (mta); return addr; @@ -462,18 +464,19 @@ guess_from_in_received_by (notmuch_config_t *config, const char *received) * (last Received: header added) and try to extract from them * indications to which email address this message was delivered. * - * The Received: header is special in our get_header function and is - * always concatenated. + * The Received: header is among special ones in our get_header function + * and is always concatenated. * * Return the address that was found, if any, and NULL otherwise. */ static const char * -guess_from_in_received_headers (notmuch_config_t *config, - notmuch_message_t *message) +guess_from_in_received_headers (notmuch_message_t *message) { const char *received, *addr; char *sanitized; + notmuch_database_t *notmuch = notmuch_message_get_database (message); + received = notmuch_message_get_header (message, "received"); if (! received) return NULL; @@ -482,9 +485,9 @@ guess_from_in_received_headers (notmuch_config_t *config, if (! sanitized) return NULL; - addr = guess_from_in_received_for (config, sanitized); + addr = guess_from_in_received_for (notmuch, sanitized); if (! addr) - addr = guess_from_in_received_by (config, sanitized); + addr = guess_from_in_received_by (notmuch, sanitized); talloc_free (sanitized); @@ -496,10 +499,13 @@ guess_from_in_received_headers (notmuch_config_t *config, * headers: Envelope-To, X-Original-To, and Delivered-To (searched in * that order). * + * The Delivered-To: header is among special ones in our get_header + * function and is always concatenated. + * * Return the address that was found, if any, and NULL otherwise. */ static const char * -get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message) +get_from_in_to_headers (notmuch_message_t *message) { size_t i; const char *tohdr, *addr; @@ -509,11 +515,13 @@ get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message) "Delivered-To", }; + notmuch_database_t *notmuch = notmuch_message_get_database (message); + for (i = 0; i < ARRAY_SIZE (to_headers); i++) { tohdr = notmuch_message_get_header (message, to_headers[i]); /* Note: tohdr potentially contains a list of email addresses. */ - addr = user_address_in_string (tohdr, config); + addr = user_address_in_string (tohdr, notmuch); if (addr) return addr; } @@ -523,7 +531,6 @@ get_from_in_to_headers (notmuch_config_t *config, notmuch_message_t *message) static GMimeMessage * create_reply_message (void *ctx, - notmuch_config_t *config, notmuch_message_t *message, GMimeMessage *mime_message, bool reply_all, @@ -531,7 +538,7 @@ create_reply_message (void *ctx, { const char *subject, *from_addr = NULL; const char *in_reply_to, *orig_references, *references; - + notmuch_database_t *notmuch = notmuch_message_get_database (message); /* * Use the below header order for limited headers, "pretty" order * otherwise. @@ -557,7 +564,7 @@ create_reply_message (void *ctx, g_mime_object_set_header (GMIME_OBJECT (reply), "References", references, NULL); - from_addr = add_recipients_from_message (reply, config, + from_addr = add_recipients_from_message (reply, notmuch, mime_message, reply_all); /* The above is all that is needed for limited headers. */ @@ -577,7 +584,7 @@ create_reply_message (void *ctx, * Delivered-To: headers. */ if (from_addr == NULL) - from_addr = get_from_in_to_headers (config, message); + from_addr = get_from_in_to_headers (message); /* * Check for a (for ) clause in Received: headers, @@ -585,14 +592,14 @@ create_reply_message (void *ctx, * of Received: headers */ if (from_addr == NULL) - from_addr = guess_from_in_received_headers (config, message); + from_addr = guess_from_in_received_headers (message); /* Default to user's primary address. */ if (from_addr == NULL) - from_addr = notmuch_config_get_user_primary_email (config); + from_addr = notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL); from_addr = talloc_asprintf (ctx, "%s <%s>", - notmuch_config_get_user_name (config), + notmuch_config_get (notmuch, NOTMUCH_CONFIG_USER_NAME), from_addr); g_mime_object_set_header (GMIME_OBJECT (reply), "From", from_addr, NULL); @@ -614,7 +621,7 @@ enum { }; static int -do_reply (notmuch_config_t *config, +do_reply (notmuch_database_t *notmuch, notmuch_query_t *query, notmuch_show_params_t *params, int format, @@ -635,14 +642,16 @@ do_reply (notmuch_config_t *config, return 1; if (count != 1) { - fprintf (stderr, "Error: search term did not match precisely one message (matched %u messages).\n", count); + fprintf (stderr, + "Error: search term did not match precisely one message (matched %u messages).\n", + count); return 1; } if (format == FORMAT_JSON) - sp = sprinter_json_create (config, stdout); + sp = sprinter_json_create (notmuch, stdout); else - sp = sprinter_sexp_create (config, stdout); + sp = sprinter_sexp_create (notmuch, stdout); } status = notmuch_query_search_messages (query, &messages); @@ -654,10 +663,10 @@ do_reply (notmuch_config_t *config, notmuch_messages_move_to_next (messages)) { message = notmuch_messages_get (messages); - if (mime_node_open (config, message, ¶ms->crypto, &node)) + if (mime_node_open (notmuch, message, params->duplicate, ¶ms->crypto, &node)) return 1; - reply = create_reply_message (config, config, message, + reply = create_reply_message (notmuch, message, GMIME_MESSAGE (node->part), reply_all, format == FORMAT_HEADERS_ONLY); if (! reply) @@ -674,7 +683,7 @@ do_reply (notmuch_config_t *config, /* Start the original */ sp->map_key (sp, "original"); - format_part_sprinter (config, sp, node, true, false); + format_part_sprinter (notmuch, sp, node, params->duplicate, true, false); /* End */ sp->end (sp); @@ -699,18 +708,19 @@ do_reply (notmuch_config_t *config, } int -notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_reply_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; notmuch_query_t *query; char *query_string; int opt_index; notmuch_show_params_t params = { .part = -1, + .duplicate = 0, .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO }, }; int format = FORMAT_DEFAULT; int reply_all = true; + notmuch_status_t status; notmuch_opt_desc_t options[] = { { .opt_keyword = &format, .name = "format", .keywords = @@ -730,6 +740,7 @@ notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]) { "auto", NOTMUCH_DECRYPT_AUTO }, { "true", NOTMUCH_DECRYPT_NOSTASH }, { 0, 0 } } }, + { .opt_int = ¶ms.duplicate, .name = "duplicate" }, { .opt_inherit = notmuch_shared_options }, { } }; @@ -738,11 +749,11 @@ notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); notmuch_exit_if_unsupported_format (); - query_string = query_string_from_args (config, argc - opt_index, argv + opt_index); + query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index); if (query_string == NULL) { fprintf (stderr, "Out of memory\n"); return EXIT_FAILURE; @@ -753,19 +764,13 @@ notmuch_reply_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_ONLY, ¬much)) - return EXIT_FAILURE; - - notmuch_exit_if_unmatched_db_uuid (notmuch); - - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch reply", notmuch, status)) return EXIT_FAILURE; - } - if (do_reply (config, query, ¶ms, format, reply_all) != 0) + if (do_reply (notmuch, query, ¶ms, format, reply_all) != 0) return EXIT_FAILURE; _notmuch_crypto_cleanup (¶ms.crypto); diff --git a/notmuch-restore.c b/notmuch-restore.c index 4b509d95..1cce004a 100644 --- a/notmuch-restore.c +++ b/notmuch-restore.c @@ -219,9 +219,8 @@ parse_sup_line (void *ctx, char *line, } int -notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_restore_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; bool accumulate = false; tag_op_flag_t flags = 0; tag_op_list_t *tag_ops; @@ -237,12 +236,17 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) int opt_index; int include = 0; int input_format = DUMP_FORMAT_AUTO; - - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + int errnum; + notmuch_bool_t synchronize_flags; + + if (print_status_database ( + "notmuch restore", + notmuch, + notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, + &synchronize_flags))) return EXIT_FAILURE; - if (notmuch_config_get_maildir_synchronize_flags (config)) + if (synchronize_flags) flags |= TAG_FLAG_MAILDIR_SYNC; notmuch_opt_desc_t options[] = { @@ -268,8 +272,7 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) goto DONE; } - notmuch_process_shared_options (argv[0]); - notmuch_exit_if_unmatched_db_uuid (notmuch); + notmuch_process_shared_options (notmuch, argv[0]); if (include == 0) { include = DUMP_INCLUDE_CONFIG | DUMP_INCLUDE_PROPERTIES | DUMP_INCLUDE_TAGS; @@ -309,7 +312,7 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) goto DONE; } - tag_ops = tag_op_list_create (config); + tag_ops = tag_op_list_create (notmuch); if (tag_ops == NULL) { fprintf (stderr, "Out of memory.\n"); ret = EXIT_FAILURE; @@ -339,7 +342,8 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) if (ret) goto DONE; } - if ((include & DUMP_INCLUDE_PROPERTIES) && line_len >= 2 && line[0] == '#' && line[1] == '=') { + if ((include & DUMP_INCLUDE_PROPERTIES) && line_len >= 2 && line[0] == '#' && line[1] == + '=') { ret = process_properties_line (notmuch, line + 2); if (ret) goto DONE; @@ -356,6 +360,7 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) } char *p; + for (p = line; (input_format == DUMP_FORMAT_AUTO) && *p; p++) { if (*p == '(') input_format = DUMP_FORMAT_SUP; @@ -376,9 +381,10 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) if (line_ctx != NULL) talloc_free (line_ctx); - line_ctx = talloc_new (config); + line_ctx = talloc_new (notmuch); - if ((include & DUMP_INCLUDE_PROPERTIES) && line_len >= 2 && line[0] == '#' && line[1] == '=') { + if ((include & DUMP_INCLUDE_PROPERTIES) && line_len >= 2 && line[0] == '#' && line[1] == + '=') { ret = process_properties_line (notmuch, line + 2); if (ret) goto DONE; @@ -448,10 +454,13 @@ notmuch_restore_command (notmuch_config_t *config, int argc, char *argv[]) if (notmuch) notmuch_database_destroy (notmuch); - if (input && gzclose_r (input)) { - fprintf (stderr, "Error closing %s: %s\n", - name_for_error, gzerror (input, NULL)); - ret = EXIT_FAILURE; + if (input) { + errnum = gzclose_r (input); + if (errnum) { + fprintf (stderr, "Error closing %s: %d\n", + name_for_error, errnum); + ret = EXIT_FAILURE; + } } return ret ? EXIT_FAILURE : EXIT_SUCCESS; diff --git a/notmuch-search.c b/notmuch-search.c index fd0b58c5..327e1445 100644 --- a/notmuch-search.c +++ b/notmuch-search.c @@ -52,9 +52,11 @@ typedef enum { typedef struct { notmuch_database_t *notmuch; + void *talloc_ctx; int format_sel; sprinter_t *format; int exclude; + int query_syntax; notmuch_query_t *query; int sort; int output; @@ -90,9 +92,13 @@ get_thread_query (notmuch_thread_t *thread, notmuch_messages_move_to_next (messages)) { notmuch_message_t *message = notmuch_messages_get (messages); const char *mid = notmuch_message_get_message_id (message); + notmuch_bool_t is_set; + char **buf; + + if (notmuch_message_get_flag_st (message, NOTMUCH_MESSAGE_FLAG_MATCH, &is_set)) + return -1; /* Determine which query buffer to extend */ - char **buf = notmuch_message_get_flag ( - message, NOTMUCH_MESSAGE_FLAG_MATCH) ? matched_out : unmatched_out; + buf = is_set ? matched_out : unmatched_out; /* Add this message's id: query. Since "id" is an exclusive * prefix, it is implicitly 'or'd together, so we only need to * join queries with a space. */ @@ -673,28 +679,29 @@ do_search_tags (const search_context_t *ctx) } static int -_notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int argc, char *argv[]) +_notmuch_search_prepare (search_context_t *ctx, int argc, char *argv[]) { char *query_str; - unsigned int i; - char *status_string = NULL; + + if (! ctx->talloc_ctx) + ctx->talloc_ctx = talloc_new (NULL); switch (ctx->format_sel) { case NOTMUCH_FORMAT_TEXT: - ctx->format = sprinter_text_create (config, stdout); + ctx->format = sprinter_text_create (ctx->talloc_ctx, stdout); break; case NOTMUCH_FORMAT_TEXT0: if (ctx->output == OUTPUT_SUMMARY) { fprintf (stderr, "Error: --format=text0 is not compatible with --output=summary.\n"); return EXIT_FAILURE; } - ctx->format = sprinter_text0_create (config, stdout); + ctx->format = sprinter_text0_create (ctx->talloc_ctx, stdout); break; case NOTMUCH_FORMAT_JSON: - ctx->format = sprinter_json_create (config, stdout); + ctx->format = sprinter_json_create (ctx->talloc_ctx, stdout); break; case NOTMUCH_FORMAT_SEXP: - ctx->format = sprinter_sexp_create (config, stdout); + ctx->format = sprinter_sexp_create (ctx->talloc_ctx, stdout); break; default: /* this should never happen */ @@ -703,20 +710,6 @@ _notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int ar notmuch_exit_if_unsupported_format (); - if (notmuch_database_open_verbose ( - notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_ONLY, &ctx->notmuch, &status_string)) { - - if (status_string) { - fputs (status_string, stderr); - free (status_string); - } - - return EXIT_FAILURE; - } - - notmuch_exit_if_unmatched_db_uuid (ctx->notmuch); - query_str = query_string_from_args (ctx->notmuch, argc, argv); if (query_str == NULL) { fprintf (stderr, "Out of memory.\n"); @@ -727,11 +720,11 @@ _notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int ar return EXIT_FAILURE; } - ctx->query = notmuch_query_create (ctx->notmuch, query_str); - if (ctx->query == NULL) { - fprintf (stderr, "Out of memory\n"); + if (print_status_database ("notmuch search", ctx->notmuch, + notmuch_query_create_with_syntax (ctx->notmuch, query_str, + shared_option_query_syntax (), + &ctx->query))) return EXIT_FAILURE; - } notmuch_query_set_sort (ctx->query, ctx->sort); @@ -744,21 +737,20 @@ _notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int ar } if (ctx->exclude != NOTMUCH_EXCLUDE_FALSE) { - const char **search_exclude_tags; - size_t search_exclude_tags_length; + notmuch_config_values_t *exclude_tags; notmuch_status_t status; - search_exclude_tags = notmuch_config_get_search_exclude_tags ( - config, &search_exclude_tags_length); + for (exclude_tags = notmuch_config_get_values (ctx->notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS); + notmuch_config_values_valid (exclude_tags); + notmuch_config_values_move_to_next (exclude_tags)) { - for (i = 0; i < search_exclude_tags_length; i++) { - status = notmuch_query_add_tag_exclude (ctx->query, search_exclude_tags[i]); + status = notmuch_query_add_tag_exclude (ctx->query, + notmuch_config_values_get (exclude_tags)); if (status && status != NOTMUCH_STATUS_IGNORED) { print_status_query ("notmuch search", ctx->query, status); return EXIT_FAILURE; } } - notmuch_query_set_omit_excluded (ctx->query, ctx->exclude); } @@ -778,6 +770,7 @@ static search_context_t search_context = { .format_sel = NOTMUCH_FORMAT_TEXT, .exclude = NOTMUCH_EXCLUDE_TRUE, .sort = NOTMUCH_SORT_NEWEST_FIRST, + .query_syntax = NOTMUCH_QUERY_SYNTAX_XAPIAN, .output = 0, .offset = 0, .limit = -1, /* unlimited */ @@ -801,7 +794,7 @@ static const notmuch_opt_desc_t common_options[] = { }; int -notmuch_search_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_search_command (notmuch_database_t *notmuch, int argc, char *argv[]) { search_context_t *ctx = &search_context; int opt_index, ret; @@ -828,21 +821,22 @@ notmuch_search_command (notmuch_config_t *config, int argc, char *argv[]) { } }; + ctx->notmuch = notmuch; ctx->output = OUTPUT_SUMMARY; opt_index = parse_arguments (argc, argv, options, 1); if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); if (ctx->output != OUTPUT_FILES && ctx->output != OUTPUT_MESSAGES && ctx->dupe != -1) { - fprintf (stderr, "Error: --duplicate=N is only supported with --output=files and --output=messages.\n"); + fprintf (stderr, + "Error: --duplicate=N is only supported with --output=files and --output=messages.\n"); return EXIT_FAILURE; } - if (_notmuch_search_prepare (ctx, config, - argc - opt_index, argv + opt_index)) + if (_notmuch_search_prepare (ctx, argc - opt_index, argv + opt_index)) return EXIT_FAILURE; switch (ctx->output) { @@ -867,7 +861,7 @@ notmuch_search_command (notmuch_config_t *config, int argc, char *argv[]) } int -notmuch_address_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_address_command (notmuch_database_t *notmuch, int argc, char *argv[]) { search_context_t *ctx = &search_context; int opt_index, ret; @@ -893,11 +887,13 @@ notmuch_address_command (notmuch_config_t *config, int argc, char *argv[]) { } }; + ctx->notmuch = notmuch; + opt_index = parse_arguments (argc, argv, options, 1); if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); if (! (ctx->output & (OUTPUT_SENDER | OUTPUT_RECIPIENTS))) ctx->output |= OUTPUT_SENDER; @@ -907,8 +903,7 @@ notmuch_address_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - if (_notmuch_search_prepare (ctx, config, - argc - opt_index, argv + opt_index)) + if (_notmuch_search_prepare (ctx, argc - opt_index, argv + opt_index)) return EXIT_FAILURE; ctx->addresses = g_hash_table_new_full (strcase_hash, strcase_equal, diff --git a/notmuch-setup.c b/notmuch-setup.c index cd1a52ff..9382e279 100644 --- a/notmuch-setup.c +++ b/notmuch-setup.c @@ -74,7 +74,7 @@ welcome_message_post_setup (void) { printf ("\n" "Notmuch is now configured, and the configuration settings are saved in\n" - "a file in your home directory named .notmuch-config . If you'd like to\n" + "a file in your home directory named .notmuch-config. If you'd like to\n" "change the configuration in the future, you can either edit that file\n" "directly or run \"notmuch setup\". To choose an alternate configuration\n" "location, set ${NOTMUCH_CONFIG}.\n\n" @@ -88,14 +88,17 @@ welcome_message_post_setup (void) } static void -print_tag_list (const char **tags, size_t tags_len) +print_tag_list (notmuch_config_values_t *tags) { - unsigned int i; + bool first = false; - for (i = 0; i < tags_len; i++) { - if (i != 0) + for (; + notmuch_config_values_valid (tags); + notmuch_config_values_move_to_next (tags)) { + if (! first) printf (" "); - printf ("%s", tags[i]); + first = false; + printf ("%s", notmuch_config_values_get (tags)); } } @@ -121,19 +124,14 @@ parse_tag_list (void *ctx, char *response) } int -notmuch_setup_command (notmuch_config_t *config, +notmuch_setup_command (notmuch_database_t *notmuch, int argc, char *argv[]) { char *response = NULL; size_t response_size = 0; - const char **old_other_emails; - size_t old_other_emails_len; GPtrArray *other_emails; - unsigned int i; - const char **new_tags; - size_t new_tags_len; - const char **search_exclude_tags; - size_t search_exclude_tags_len; + notmuch_config_values_t *new_tags, *search_exclude_tags, *emails; + notmuch_conffile_t *config; #define prompt(format, ...) \ do { \ @@ -149,33 +147,35 @@ notmuch_setup_command (notmuch_config_t *config, if (notmuch_minimal_options ("setup", argc, argv) < 0) return EXIT_FAILURE; - if (notmuch_requested_db_uuid) - fprintf (stderr, "Warning: ignoring --uuid=%s\n", - notmuch_requested_db_uuid); + config = notmuch_conffile_open (notmuch, + notmuch_config_path (notmuch), true); + if (! config) + return EXIT_FAILURE; - if (notmuch_config_is_new (config)) + if (notmuch_conffile_is_new (config)) welcome_message_pre_setup (); - prompt ("Your full name [%s]: ", notmuch_config_get_user_name (config)); + prompt ("Your full name [%s]: ", notmuch_config_get (notmuch, NOTMUCH_CONFIG_USER_NAME)); if (strlen (response)) - notmuch_config_set_user_name (config, response); + notmuch_conffile_set_user_name (config, response); prompt ("Your primary email address [%s]: ", - notmuch_config_get_user_primary_email (config)); + notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL)); if (strlen (response)) - notmuch_config_set_user_primary_email (config, response); + notmuch_conffile_set_user_primary_email (config, response); other_emails = g_ptr_array_new (); - old_other_emails = notmuch_config_get_user_other_email (config, - &old_other_emails_len); - for (i = 0; i < old_other_emails_len; i++) { - prompt ("Additional email address [%s]: ", old_other_emails[i]); + for (emails = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_OTHER_EMAIL); + notmuch_config_values_valid (emails); + notmuch_config_values_move_to_next (emails)) { + const char *email = notmuch_config_values_get (emails); + + prompt ("Additional email address [%s]: ", email); if (strlen (response)) g_ptr_array_add (other_emails, talloc_strdup (config, response)); else - g_ptr_array_add (other_emails, talloc_strdup (config, - old_other_emails[i])); + g_ptr_array_add (other_emails, talloc_strdup (config, email)); } do { @@ -184,57 +184,59 @@ notmuch_setup_command (notmuch_config_t *config, g_ptr_array_add (other_emails, talloc_strdup (config, response)); } while (strlen (response)); if (other_emails->len) - notmuch_config_set_user_other_email (config, - (const char **) - other_emails->pdata, - other_emails->len); + notmuch_conffile_set_user_other_email (config, + (const char **) + other_emails->pdata, + other_emails->len); g_ptr_array_free (other_emails, true); prompt ("Top-level directory of your email archive [%s]: ", - notmuch_config_get_database_path (config)); + notmuch_config_get (notmuch, NOTMUCH_CONFIG_DATABASE_PATH)); if (strlen (response)) { const char *absolute_path; absolute_path = make_path_absolute (config, response); - notmuch_config_set_database_path (config, absolute_path); + notmuch_conffile_set_database_path (config, absolute_path); } - new_tags = notmuch_config_get_new_tags (config, &new_tags_len); + new_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_NEW_TAGS); printf ("Tags to apply to all new messages (separated by spaces) ["); - print_tag_list (new_tags, new_tags_len); + print_tag_list (new_tags); prompt ("]: "); if (strlen (response)) { GPtrArray *tags = parse_tag_list (config, response); - notmuch_config_set_new_tags (config, (const char **) tags->pdata, - tags->len); + notmuch_conffile_set_new_tags (config, (const char **) tags->pdata, + tags->len); g_ptr_array_free (tags, true); } - - search_exclude_tags = notmuch_config_get_search_exclude_tags (config, &search_exclude_tags_len); + search_exclude_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS); printf ("Tags to exclude when searching messages (separated by spaces) ["); - print_tag_list (search_exclude_tags, search_exclude_tags_len); + print_tag_list (search_exclude_tags); prompt ("]: "); if (strlen (response)) { GPtrArray *tags = parse_tag_list (config, response); - notmuch_config_set_search_exclude_tags (config, - (const char **) tags->pdata, - tags->len); + notmuch_conffile_set_search_exclude_tags (config, + (const char **) tags->pdata, + tags->len); g_ptr_array_free (tags, true); } - if (notmuch_config_save (config)) + if (notmuch_conffile_save (config)) return EXIT_FAILURE; - if (notmuch_config_is_new (config)) + if (config) + notmuch_conffile_close (config); + + if (notmuch_conffile_is_new (config)) welcome_message_post_setup (); return EXIT_SUCCESS; diff --git a/notmuch-show.c b/notmuch-show.c index 21792a57..7fb40ce9 100644 --- a/notmuch-show.c +++ b/notmuch-show.c @@ -23,6 +23,21 @@ #include "sprinter.h" #include "zlib-extra.h" +static const char * +_get_filename (notmuch_message_t *message, int index) +{ + notmuch_filenames_t *filenames = notmuch_message_get_filenames (message); + int i = 1; + + for (; + notmuch_filenames_valid (filenames); + notmuch_filenames_move_to_next (filenames), i++) { + if (i >= index) + return notmuch_filenames_get (filenames); + } + return NULL; +} + static const char * _get_tags_as_string (const void *ctx, notmuch_message_t *message) { @@ -80,6 +95,20 @@ _get_disposition (GMimeObject *meta) return g_mime_content_disposition_get_disposition (disposition); } +static bool +_get_message_flag (notmuch_message_t *message, notmuch_message_flag_t flag) +{ + notmuch_bool_t is_set; + notmuch_status_t status; + + status = notmuch_message_get_flag_st (message, flag, &is_set); + + if (print_status_message ("notmuch show", message, status)) + INTERNAL_ERROR ("unexpected error getting message flag\n"); + + return is_set; +} + /* Emit a sequence of key/value pairs for the metadata of message. * The caller should begin a map before calling this. */ static void @@ -97,10 +126,10 @@ format_message_sprinter (sprinter_t *sp, notmuch_message_t *message) sp->string (sp, notmuch_message_get_message_id (message)); sp->map_key (sp, "match"); - sp->boolean (sp, notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH)); + sp->boolean (sp, _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH)); sp->map_key (sp, "excluded"); - sp->boolean (sp, notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED)); + sp->boolean (sp, _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED)); sp->map_key (sp, "filename"); if (notmuch_format_version >= 3) { @@ -195,6 +224,30 @@ _is_from_line (const char *line) return 0; } +/* Output extra headers if configured with the `show.extra_headers' + * configuration option + */ +static void +format_extra_headers_sprinter (sprinter_t *sp, GMimeMessage *message) +{ + GMimeHeaderList *header_list = g_mime_object_get_header_list (GMIME_OBJECT (message)); + + for (notmuch_config_values_t *extra_headers = notmuch_config_get_values ( + sp->notmuch, NOTMUCH_CONFIG_EXTRA_HEADERS); + notmuch_config_values_valid (extra_headers); + notmuch_config_values_move_to_next (extra_headers)) { + GMimeHeader *header; + const char *header_name = notmuch_config_values_get (extra_headers); + + header = g_mime_header_list_get_header (header_list, header_name); + if (header == NULL) + continue; + + sp->map_key (sp, g_mime_header_get_name (header)); + sp->string (sp, g_mime_header_get_value (header)); + } +} + void format_headers_sprinter (sprinter_t *sp, GMimeMessage *message, bool reply, const _notmuch_message_crypto_t *msg_crypto) @@ -255,6 +308,9 @@ format_headers_sprinter (sprinter_t *sp, GMimeMessage *message, sp->string (sp, g_mime_message_get_date_string (sp, message)); } + /* Output extra headers the user has configured, if any */ + if (! reply) + format_extra_headers_sprinter (sp, message); sp->end (sp); talloc_free (local); } @@ -426,6 +482,7 @@ format_part_sigstatus_sprinter (sprinter_t *sp, GMimeSignatureList *siglist) } int i; + for (i = 0; i < g_mime_signature_list_length (siglist); i++) { GMimeSignature *signature = g_mime_signature_list_get_signature (siglist, i); @@ -460,6 +517,11 @@ format_part_sigstatus_sprinter (sprinter_t *sp, GMimeSignatureList *siglist) sp->map_key (sp, "userid"); sp->string (sp, uid); } + const char *email = g_mime_certificate_get_valid_email (certificate); + if (email) { + sp->map_key (sp, "email"); + sp->string (sp, email); + } } } else if (certificate) { const char *key_id = g_mime_certificate_get_fpr16 (certificate); @@ -507,8 +569,8 @@ format_part_text (const void *ctx, sprinter_t *sp, mime_node_t *node, part_type, notmuch_message_get_message_id (message), indent, - notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? 1 : 0, - notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED) ? 1 : 0, + _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH) ? 1 : 0, + _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED) ? 1 : 0, notmuch_message_get_filename (message)); } else { char *content_string; @@ -611,6 +673,7 @@ format_omitted_part_meta_sprinter (sprinter_t *sp, GMimeObject *meta, GMimePart void format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node, + int duplicate, bool output_body, bool include_html) { @@ -622,10 +685,13 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node, sp->begin_map (sp); format_message_sprinter (sp, node->envelope_file); + sp->map_key (sp, "duplicate"); + sp->integer (sp, duplicate > 0 ? duplicate : 1); + if (output_body) { sp->map_key (sp, "body"); sp->begin_list (sp); - format_part_sprinter (ctx, sp, mime_node_child (node, 0), true, include_html); + format_part_sprinter (ctx, sp, mime_node_child (node, 0), -1, true, include_html); sp->end (sp); } @@ -656,7 +722,8 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node, sp->map_key (sp, "decrypted"); sp->begin_map (sp); sp->map_key (sp, "status"); - sp->string (sp, msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL ? "full" : "partial"); + sp->string (sp, msg_crypto->decryption_status == NOTMUCH_MESSAGE_DECRYPTED_FULL ? + "full" : "partial"); if (msg_crypto->payload_subject) { const char *subject = g_mime_message_get_subject GMIME_MESSAGE (node->part); @@ -759,7 +826,16 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node, sp->string_len (sp, (char *) part_content->data, part_content->len); g_object_unref (stream_memory); } else { - format_omitted_part_meta_sprinter (sp, meta, GMIME_PART (node->part)); + /* if we have a child part despite being a standard + * (non-multipart) MIME part, that means there is + * something to unwrap, which we will present in + * content: */ + if (node->nchildren) { + sp->map_key (sp, "content"); + sp->begin_list (sp); + nclose = 1; + } else + format_omitted_part_meta_sprinter (sp, meta, GMIME_PART (node->part)); } } else if (GMIME_IS_MULTIPART (node->part)) { sp->map_key (sp, "content"); @@ -779,7 +855,7 @@ format_part_sprinter (const void *ctx, sprinter_t *sp, mime_node_t *node, } for (i = 0; i < node->nchildren; i++) - format_part_sprinter (ctx, sp, mime_node_child (node, i), true, include_html); + format_part_sprinter (ctx, sp, mime_node_child (node, i), -1, true, include_html); /* Close content structures */ for (i = 0; i < nclose; i++) @@ -793,7 +869,8 @@ format_part_sprinter_entry (const void *ctx, sprinter_t *sp, mime_node_t *node, unused (int indent), const notmuch_show_params_t *params) { - format_part_sprinter (ctx, sp, node, params->output_body, params->include_html); + format_part_sprinter (ctx, sp, node, params->duplicate, params->output_body, + params->include_html); return NOTMUCH_STATUS_SUCCESS; } @@ -868,7 +945,7 @@ format_part_raw (unused (const void *ctx), unused (sprinter_t *sp), char buf[4096]; notmuch_status_t ret = NOTMUCH_STATUS_FILE_ERROR; - filename = notmuch_message_get_filename (node->envelope_file); + filename = _get_filename (node->envelope_file, params->duplicate); if (filename == NULL) { fprintf (stderr, "Error: Cannot get message filename.\n"); goto DONE; @@ -888,7 +965,7 @@ format_part_raw (unused (const void *ctx), unused (sprinter_t *sp), } if (ssize > 0 && fwrite (buf, ssize, 1, stdout) != 1) { - fprintf (stderr, "Error: Write %ld chars to stdout failed\n", ssize); + fprintf (stderr, "Error: Write %zd chars to stdout failed\n", ssize); goto DONE; } } @@ -944,20 +1021,24 @@ show_message (void *ctx, notmuch_status_t session_key_count_error = NOTMUCH_STATUS_SUCCESS; if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE) - session_key_count_error = notmuch_message_count_properties (message, "session-key", &session_keys); + session_key_count_error = notmuch_message_count_properties (message, "session-key", + &session_keys); - status = mime_node_open (local, message, &(params->crypto), &root); + status = mime_node_open (local, message, params->duplicate, &(params->crypto), &root); if (status) goto DONE; part = mime_node_seek_dfs (root, (params->part < 0 ? 0 : params->part)); if (part) status = format->part (local, sp, part, indent, params); - if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE && session_key_count_error == NOTMUCH_STATUS_SUCCESS) { + if (params->crypto.decrypt == NOTMUCH_DECRYPT_TRUE && session_key_count_error == + NOTMUCH_STATUS_SUCCESS) { unsigned int new_session_keys = 0; - if (notmuch_message_count_properties (message, "session-key", &new_session_keys) == NOTMUCH_STATUS_SUCCESS && + if (notmuch_message_count_properties (message, "session-key", &new_session_keys) == + NOTMUCH_STATUS_SUCCESS && new_session_keys > session_keys) { /* try a quiet re-indexing */ - notmuch_indexopts_t *indexopts = notmuch_database_get_default_indexopts (notmuch_message_get_database (message)); + notmuch_indexopts_t *indexopts = notmuch_database_get_default_indexopts ( + notmuch_message_get_database (message)); if (indexopts) { notmuch_indexopts_set_decrypt_policy (indexopts, NOTMUCH_DECRYPT_AUTO); print_status_message ("Error re-indexing message with --decrypt=stash", @@ -993,8 +1074,8 @@ show_messages (void *ctx, message = notmuch_messages_get (messages); - match = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH); - excluded = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED); + match = _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH); + excluded = _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED); next_indent = indent; @@ -1043,7 +1124,9 @@ do_show_single (void *ctx, return 1; if (count != 1) { - fprintf (stderr, "Error: search term did not match precisely one message (matched %u messages).\n", count); + fprintf (stderr, + "Error: search term did not match precisely one message (matched %u messages).\n", + count); return 1; } @@ -1066,16 +1149,28 @@ do_show_single (void *ctx, /* Formatted output of threads */ static int -do_show (void *ctx, - notmuch_query_t *query, - const notmuch_show_format_t *format, - sprinter_t *sp, - notmuch_show_params_t *params) +do_show_threaded (void *ctx, + notmuch_query_t *query, + const notmuch_show_format_t *format, + sprinter_t *sp, + notmuch_show_params_t *params) { notmuch_threads_t *threads; notmuch_thread_t *thread; notmuch_messages_t *messages; notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS; + int i; + + if (params->offset < 0) { + unsigned count; + notmuch_status_t s = notmuch_query_count_threads (query, &count); + if (print_status_query ("notmuch show", query, s)) + return 1; + + params->offset += count; + if (params->offset < 0) + params->offset = 0; + } status = notmuch_query_search_threads (query, &threads); if (print_status_query ("notmuch show", query, status)) @@ -1083,11 +1178,16 @@ do_show (void *ctx, sp->begin_list (sp); - for (; - notmuch_threads_valid (threads); - notmuch_threads_move_to_next (threads)) { + for (i = 0; + notmuch_threads_valid (threads) && (params->limit < 0 || i < params->offset + params->limit); + notmuch_threads_move_to_next (threads), i++) { thread = notmuch_threads_get (threads); + if (i < params->offset) { + notmuch_thread_destroy (thread); + continue; + } + messages = notmuch_thread_get_toplevel_messages (thread); if (messages == NULL) @@ -1107,6 +1207,66 @@ do_show (void *ctx, return res != NOTMUCH_STATUS_SUCCESS; } +static int +do_show_unthreaded (void *ctx, + notmuch_query_t *query, + const notmuch_show_format_t *format, + sprinter_t *sp, + notmuch_show_params_t *params) +{ + notmuch_messages_t *messages; + notmuch_message_t *message; + notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS; + notmuch_bool_t excluded; + int i; + + if (params->offset < 0) { + unsigned count; + notmuch_status_t s = notmuch_query_count_messages (query, &count); + if (print_status_query ("notmuch show", query, s)) + return 1; + + params->offset += count; + if (params->offset < 0) + params->offset = 0; + } + + status = notmuch_query_search_messages (query, &messages); + if (print_status_query ("notmuch show", query, status)) + return 1; + + sp->begin_list (sp); + + for (i = 0; + notmuch_messages_valid (messages) && (params->limit < 0 || i < params->offset + params->limit); + notmuch_messages_move_to_next (messages), i++) { + if (i < params->offset) { + continue; + } + + sp->begin_list (sp); + sp->begin_list (sp); + + message = notmuch_messages_get (messages); + + notmuch_message_set_flag (message, NOTMUCH_MESSAGE_FLAG_MATCH, TRUE); + excluded = _get_message_flag (message, NOTMUCH_MESSAGE_FLAG_EXCLUDED); + + if (! excluded || ! params->omit_excluded) { + status = show_message (ctx, format, sp, message, 0, params); + if (status && ! res) + res = status; + } else { + sp->null (sp); + } + notmuch_message_destroy (message); + sp->end (sp); + sp->end (sp); + } + sp->end (sp); + return res; +} + enum { NOTMUCH_FORMAT_NOT_SPECIFIED, NOTMUCH_FORMAT_JSON, @@ -1150,9 +1310,8 @@ static const notmuch_show_format_t *formatters[] = { }; int -notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_show_command (notmuch_database_t *notmuch, int argc, char *argv[]) { - notmuch_database_t *notmuch; notmuch_query_t *query; char *query_string; int opt_index, ret; @@ -1160,6 +1319,9 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) sprinter_t *sprinter; notmuch_show_params_t params = { .part = -1, + .duplicate = 0, + .offset = 0, + .limit = -1, /* unlimited */ .omit_excluded = true, .output_body = true, .crypto = { .decrypt = NOTMUCH_DECRYPT_AUTO }, @@ -1168,8 +1330,15 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) bool exclude = true; bool entire_thread_set = false; bool single_message; + bool unthreaded = FALSE; + notmuch_status_t status; + int sort = NOTMUCH_SORT_NEWEST_FIRST; notmuch_opt_desc_t options[] = { + { .opt_keyword = &sort, .name = "sort", .keywords = + (notmuch_keyword_t []){ { "oldest-first", NOTMUCH_SORT_OLDEST_FIRST }, + { "newest-first", NOTMUCH_SORT_NEWEST_FIRST }, + { 0, 0 } } }, { .opt_keyword = &format, .name = "format", .keywords = (notmuch_keyword_t []){ { "json", NOTMUCH_FORMAT_JSON }, { "text", NOTMUCH_FORMAT_TEXT }, @@ -1181,6 +1350,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) { .opt_bool = &exclude, .name = "exclude" }, { .opt_bool = ¶ms.entire_thread, .name = "entire-thread", .present = &entire_thread_set }, + { .opt_bool = &unthreaded, .name = "unthreaded" }, { .opt_int = ¶ms.part, .name = "part" }, { .opt_keyword = (int *) (¶ms.crypto.decrypt), .name = "decrypt", .keyword_no_arg_value = "true", .keywords = @@ -1192,6 +1362,9 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) { .opt_bool = ¶ms.crypto.verify, .name = "verify" }, { .opt_bool = ¶ms.output_body, .name = "body" }, { .opt_bool = ¶ms.include_html, .name = "include-html" }, + { .opt_int = ¶ms.duplicate, .name = "duplicate" }, + { .opt_int = ¶ms.limit, .name = "limit" }, + { .opt_int = ¶ms.offset, .name = "offset" }, { .opt_inherit = notmuch_shared_options }, { } }; @@ -1200,7 +1373,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); /* explicit decryption implies verification */ if (params.crypto.decrypt == NOTMUCH_DECRYPT_NOSTASH || @@ -1210,6 +1383,9 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) /* specifying a part implies single message display */ single_message = params.part >= 0; + /* specifying a duplicate also implies single message display */ + single_message = single_message || (params.duplicate > 0); + if (format == NOTMUCH_FORMAT_NOT_SPECIFIED) { /* if part was requested and format was not specified, use format=raw */ if (params.part >= 0) @@ -1253,10 +1429,20 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) (format != NOTMUCH_FORMAT_TEXT && format != NOTMUCH_FORMAT_JSON && format != NOTMUCH_FORMAT_SEXP)) { - fprintf (stderr, "Warning: --include-html only implemented for format=text, format=json and format=sexp\n"); + fprintf (stderr, + "Warning: --include-html only implemented for format=text, format=json and format=sexp\n"); + } + + if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE) { + status = notmuch_database_reopen (notmuch, NOTMUCH_DATABASE_MODE_READ_WRITE); + if (status) { + fprintf (stderr, "Error reopening database for READ_WRITE: %s\n", + notmuch_status_to_string (status)); + return EXIT_FAILURE; + } } - query_string = query_string_from_args (config, argc - opt_index, argv + opt_index); + query_string = query_string_from_args (notmuch, argc - opt_index, argv + opt_index); if (query_string == NULL) { fprintf (stderr, "Out of memory\n"); return EXIT_FAILURE; @@ -1267,44 +1453,36 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } - notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY; - if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE) - mode = NOTMUCH_DATABASE_MODE_READ_WRITE; - if (notmuch_database_open (notmuch_config_get_database_path (config), - mode, ¬much)) + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch show", notmuch, status)) return EXIT_FAILURE; - notmuch_exit_if_unmatched_db_uuid (notmuch); - - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory\n"); - return EXIT_FAILURE; - } + notmuch_query_set_sort (query, sort); /* Create structure printer. */ formatter = formatters[format]; - sprinter = formatter->new_sprinter (config, stdout); + sprinter = formatter->new_sprinter (notmuch, stdout); params.out_stream = g_mime_stream_stdout_new (); /* If a single message is requested we do not use search_excludes. */ if (single_message) { - ret = do_show_single (config, query, formatter, sprinter, ¶ms); + ret = do_show_single (notmuch, query, formatter, sprinter, ¶ms); } else { /* We always apply set the exclude flag. The * exclude=true|false option controls whether or not we return * threads that only match in an excluded message */ - const char **search_exclude_tags; - size_t search_exclude_tags_length; - unsigned int i; + notmuch_config_values_t *exclude_tags; notmuch_status_t status; - search_exclude_tags = notmuch_config_get_search_exclude_tags - (config, &search_exclude_tags_length); + for (exclude_tags = notmuch_config_get_values (notmuch, NOTMUCH_CONFIG_EXCLUDE_TAGS); + notmuch_config_values_valid (exclude_tags); + notmuch_config_values_move_to_next (exclude_tags)) { - for (i = 0; i < search_exclude_tags_length; i++) { - status = notmuch_query_add_tag_exclude (query, search_exclude_tags[i]); + status = notmuch_query_add_tag_exclude (query, + notmuch_config_values_get (exclude_tags)); if (status && status != NOTMUCH_STATUS_IGNORED) { print_status_query ("notmuch show", query, status); ret = -1; @@ -1317,7 +1495,10 @@ notmuch_show_command (notmuch_config_t *config, int argc, char *argv[]) params.omit_excluded = false; } - ret = do_show (config, query, formatter, sprinter, ¶ms); + if (unthreaded) + ret = do_show_unthreaded (notmuch, query, formatter, sprinter, ¶ms); + else + ret = do_show_threaded (notmuch, query, formatter, sprinter, ¶ms); } DONE: diff --git a/notmuch-tag.c b/notmuch-tag.c index 05b1837d..71ff06bf 100644 --- a/notmuch-tag.c +++ b/notmuch-tag.c @@ -27,7 +27,7 @@ static volatile sig_atomic_t interrupted; static void handle_sigint (unused (int sig)) { - static char msg[] = "Stopping... \n"; + static const char msg[] = "Stopping... \n"; /* This write is "opportunistic", so it's okay to ignore the * result. It is not required for correctness, and if it does @@ -39,8 +39,8 @@ handle_sigint (unused (int sig)) static char * -_optimize_tag_query (void *ctx, const char *orig_query_string, - const tag_op_list_t *list) +_optimize_tag_query_infix (void *ctx, const char *orig_query_string, + const tag_op_list_t *list) { /* This is subtler than it looks. Xapian ignores the '-' operator * at the beginning both queries and parenthesized groups and, @@ -88,6 +88,33 @@ _optimize_tag_query (void *ctx, const char *orig_query_string, return query_string; } +static char * +_optimize_tag_query (void *ctx, const char *orig_query_string, + notmuch_query_syntax_t stx, + const tag_op_list_t *list) +{ + char *query_string; + + if (stx == NOTMUCH_QUERY_SYNTAX_XAPIAN) + return _optimize_tag_query_infix (ctx, orig_query_string, list); + + /* Don't optimize if there are no tag changes. */ + if (tag_op_list_size (list) == 0) + return talloc_strdup (ctx, orig_query_string); + + query_string = talloc_asprintf (ctx, "(and %s", orig_query_string); + for (size_t i = 0; i < tag_op_list_size (list) && query_string; i++) { + query_string = talloc_asprintf_append_buffer ( + query_string, tag_op_list_isremove (list, i) ? " (tag \"%s\")" : " (not (tag \"%s\"))", + tag_op_list_tag (list, i)); + } + + if (query_string) + query_string = talloc_strdup_append_buffer (query_string, ")"); + + return query_string; +} + /* Tag messages matching 'query_string' according to 'tag_ops' */ static int @@ -104,7 +131,9 @@ tag_query (void *ctx, notmuch_database_t *notmuch, const char *query_string, if (! (flags & TAG_FLAG_REMOVE_ALL)) { /* Optimize the query so it excludes messages that already * have the specified set of tags. */ - query_string = _optimize_tag_query (ctx, query_string, tag_ops); + query_string = _optimize_tag_query (ctx, query_string, + shared_option_query_syntax (), + tag_ops); if (query_string == NULL) { fprintf (stderr, "Out of memory.\n"); return 1; @@ -112,11 +141,11 @@ tag_query (void *ctx, notmuch_database_t *notmuch, const char *query_string, flags |= TAG_FLAG_PRE_OPTIMIZED; } - query = notmuch_query_create (notmuch, query_string); - if (query == NULL) { - fprintf (stderr, "Out of memory.\n"); + status = notmuch_query_create_with_syntax (notmuch, query_string, + shared_option_query_syntax (), + &query); + if (print_status_database ("notmuch tag", notmuch, status)) return 1; - } /* tagging is not interested in any special sort order */ notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED); @@ -187,11 +216,10 @@ tag_file (void *ctx, notmuch_database_t *notmuch, tag_op_flag_t flags, } int -notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]) +notmuch_tag_command (notmuch_database_t *notmuch, int argc, char *argv[]) { tag_op_list_t *tag_ops = NULL; char *query_string = NULL; - notmuch_database_t *notmuch; struct sigaction action; tag_op_flag_t tag_flags = TAG_FLAG_NONE; bool batch = false; @@ -200,6 +228,7 @@ notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]) const char *input_file_name = NULL; int opt_index; int ret; + notmuch_bool_t synchronize_flags; /* Set up our handler for SIGINT */ memset (&action, 0, sizeof (struct sigaction)); @@ -220,7 +249,7 @@ notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]) if (opt_index < 0) return EXIT_FAILURE; - notmuch_process_shared_options (argv[0]); + notmuch_process_shared_options (notmuch, argv[0]); if (input_file_name) { batch = true; @@ -240,13 +269,13 @@ notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]) return EXIT_FAILURE; } } else { - tag_ops = tag_op_list_create (config); + tag_ops = tag_op_list_create (notmuch); if (tag_ops == NULL) { fprintf (stderr, "Out of memory.\n"); return EXIT_FAILURE; } - if (parse_tag_command_line (config, argc - opt_index, argv + opt_index, + if (parse_tag_command_line (notmuch, argc - opt_index, argv + opt_index, &query_string, tag_ops)) return EXIT_FAILURE; @@ -261,22 +290,23 @@ notmuch_tag_command (notmuch_config_t *config, int argc, char *argv[]) } } - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + if (print_status_database ( + "notmuch restore", + notmuch, + notmuch_config_get_bool (notmuch, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, + &synchronize_flags))) return EXIT_FAILURE; - notmuch_exit_if_unmatched_db_uuid (notmuch); - - if (notmuch_config_get_maildir_synchronize_flags (config)) + if (synchronize_flags) tag_flags |= TAG_FLAG_MAILDIR_SYNC; if (remove_all) tag_flags |= TAG_FLAG_REMOVE_ALL; if (batch) - ret = tag_file (config, notmuch, tag_flags, input); + ret = tag_file (notmuch, notmuch, tag_flags, input); else - ret = tag_query (config, notmuch, query_string, tag_ops, tag_flags); + ret = tag_query (notmuch, notmuch, query_string, tag_ops, tag_flags); notmuch_database_destroy (notmuch); diff --git a/notmuch-time.c b/notmuch-time.c index cc7ffc23..cd45818b 100644 --- a/notmuch-time.c +++ b/notmuch-time.c @@ -39,8 +39,8 @@ * */ #define MINUTE (60) -#define HOUR (60 *MINUTE) -#define DAY (24 *HOUR) +#define HOUR (60 * MINUTE) +#define DAY (24 * HOUR) #define RELATIVE_DATE_MAX 20 const char * notmuch_time_relative_date (const void *ctx, time_t then) diff --git a/notmuch.c b/notmuch.c index 4ef1484f..814b9e42 100644 --- a/notmuch.c +++ b/notmuch.c @@ -27,41 +27,60 @@ * * The return value will be used as notmuch exit status code, * preferably EXIT_SUCCESS or EXIT_FAILURE. + * + * Each subcommand should be passed either a config object, or an open + * database */ -typedef int (*command_function_t) (notmuch_config_t *config, int argc, char *argv[]); +typedef int (*command_function_t) (notmuch_database_t *notmuch, int argc, char *argv[]); typedef struct command { const char *name; command_function_t function; - notmuch_config_mode_t config_mode; + notmuch_command_mode_t mode; const char *summary; } command_t; static int -notmuch_help_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_help_command (notmuch_database_t *notmuch, int argc, char *argv[]); static int -notmuch_command (notmuch_config_t *config, int argc, char *argv[]); +notmuch_command (notmuch_database_t *notmuch, int argc, char *argv[]); static int _help_for (const char *topic); +static void +notmuch_exit_if_unmatched_db_uuid (notmuch_database_t *notmuch); + static bool print_version = false, print_help = false; -const char *notmuch_requested_db_uuid = NULL; +static const char *notmuch_requested_db_uuid = NULL; +static int query_syntax = NOTMUCH_QUERY_SYNTAX_XAPIAN; const notmuch_opt_desc_t notmuch_shared_options [] = { { .opt_bool = &print_version, .name = "version" }, { .opt_bool = &print_help, .name = "help" }, { .opt_string = ¬much_requested_db_uuid, .name = "uuid" }, + { .opt_keyword = &query_syntax, .name = "query", .keywords = + (notmuch_keyword_t []){ { "infix", NOTMUCH_QUERY_SYNTAX_XAPIAN }, + { "sexp", NOTMUCH_QUERY_SYNTAX_SEXP }, + { 0, 0 } } }, + { } }; +notmuch_query_syntax_t +shared_option_query_syntax () +{ + return query_syntax; +} + /* any subcommand wanting to support these options should call * inherit notmuch_shared_options and call - * notmuch_process_shared_options (subcommand_name); + * notmuch_process_shared_options (notmuch, subcommand_name); + * with notmuch = an open database, or NULL. */ void -notmuch_process_shared_options (const char *subcommand_name) +notmuch_process_shared_options (notmuch_database_t *notmuch, const char *subcommand_name) { if (print_version) { printf ("notmuch " STRINGIFY (NOTMUCH_VERSION) "\n"); @@ -72,6 +91,14 @@ notmuch_process_shared_options (const char *subcommand_name) int ret = _help_for (subcommand_name); exit (ret); } + + if (notmuch) { + notmuch_exit_if_unmatched_db_uuid (notmuch); + } else { + if (notmuch_requested_db_uuid) + fprintf (stderr, "Warning: ignoring --uuid=%s\n", + notmuch_requested_db_uuid); + } } /* This is suitable for subcommands that do not actually open the @@ -94,7 +121,7 @@ notmuch_minimal_options (const char *subcommand_name, return -1; /* We can't use argv here as it is sometimes NULL */ - notmuch_process_shared_options (subcommand_name); + notmuch_process_shared_options (NULL, subcommand_name); return opt_index; } @@ -114,20 +141,18 @@ const notmuch_opt_desc_t notmuch_shared_indexing_options [] = { notmuch_status_t -notmuch_process_shared_indexing_options (notmuch_database_t *notmuch) +notmuch_process_shared_indexing_options (notmuch_indexopts_t *opts) { - if (indexing_cli_choices.opts == NULL) - indexing_cli_choices.opts = notmuch_database_get_default_indexopts (notmuch); + if (opts == NULL) + return NOTMUCH_STATUS_NULL_POINTER; + if (indexing_cli_choices.decrypt_policy_set) { notmuch_status_t status; - if (indexing_cli_choices.opts == NULL) - return NOTMUCH_STATUS_OUT_OF_MEMORY; - status = notmuch_indexopts_set_decrypt_policy (indexing_cli_choices.opts, indexing_cli_choices.decrypt_policy); + status = notmuch_indexopts_set_decrypt_policy (opts, + indexing_cli_choices.decrypt_policy); if (status != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "Error: Failed to set index decryption policy to %d. (%s)\n", indexing_cli_choices.decrypt_policy, notmuch_status_to_string (status)); - notmuch_indexopts_destroy (indexing_cli_choices.opts); - indexing_cli_choices.opts = NULL; return status; } } @@ -135,42 +160,48 @@ notmuch_process_shared_indexing_options (notmuch_database_t *notmuch) } -static command_t commands[] = { - { NULL, notmuch_command, NOTMUCH_CONFIG_OPEN | NOTMUCH_CONFIG_CREATE, +static const command_t commands[] = { + { NULL, notmuch_command, NOTMUCH_COMMAND_CONFIG_CREATE | NOTMUCH_COMMAND_CONFIG_LOAD, "Notmuch main command." }, - { "setup", notmuch_setup_command, NOTMUCH_CONFIG_OPEN | NOTMUCH_CONFIG_CREATE, + { "setup", notmuch_setup_command, NOTMUCH_COMMAND_CONFIG_CREATE | NOTMUCH_COMMAND_CONFIG_LOAD, "Interactively set up notmuch for first use." }, - { "new", notmuch_new_command, NOTMUCH_CONFIG_OPEN, + { "new", notmuch_new_command, + NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE | + NOTMUCH_COMMAND_DATABASE_CREATE, "Find and import new messages to the notmuch database." }, - { "insert", notmuch_insert_command, NOTMUCH_CONFIG_OPEN, + { "insert", notmuch_insert_command, NOTMUCH_COMMAND_DATABASE_EARLY | + NOTMUCH_COMMAND_DATABASE_WRITE, "Add a new message into the maildir and notmuch database." }, - { "search", notmuch_search_command, NOTMUCH_CONFIG_OPEN, + { "search", notmuch_search_command, NOTMUCH_COMMAND_DATABASE_EARLY, "Search for messages matching the given search terms." }, - { "address", notmuch_address_command, NOTMUCH_CONFIG_OPEN, + { "address", notmuch_address_command, NOTMUCH_COMMAND_DATABASE_EARLY, "Get addresses from messages matching the given search terms." }, - { "show", notmuch_show_command, NOTMUCH_CONFIG_OPEN, + { "show", notmuch_show_command, NOTMUCH_COMMAND_DATABASE_EARLY, "Show all messages matching the search terms." }, - { "count", notmuch_count_command, NOTMUCH_CONFIG_OPEN, + { "count", notmuch_count_command, NOTMUCH_COMMAND_DATABASE_EARLY, "Count messages matching the search terms." }, - { "reply", notmuch_reply_command, NOTMUCH_CONFIG_OPEN, + { "reply", notmuch_reply_command, NOTMUCH_COMMAND_DATABASE_EARLY, "Construct a reply template for a set of messages." }, - { "tag", notmuch_tag_command, NOTMUCH_CONFIG_OPEN, + { "tag", notmuch_tag_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE, "Add/remove tags for all messages matching the search terms." }, - { "dump", notmuch_dump_command, NOTMUCH_CONFIG_OPEN, + { "dump", notmuch_dump_command, NOTMUCH_COMMAND_DATABASE_EARLY | NOTMUCH_COMMAND_DATABASE_WRITE, "Create a plain-text dump of the tags for each message." }, - { "restore", notmuch_restore_command, NOTMUCH_CONFIG_OPEN, + { "restore", notmuch_restore_command, NOTMUCH_COMMAND_DATABASE_EARLY | + NOTMUCH_COMMAND_DATABASE_WRITE, "Restore the tags from the given dump file (see 'dump')." }, - { "compact", notmuch_compact_command, NOTMUCH_CONFIG_OPEN, + { "compact", notmuch_compact_command, NOTMUCH_COMMAND_DATABASE_EARLY | + NOTMUCH_COMMAND_DATABASE_WRITE, "Compact the notmuch database." }, - { "reindex", notmuch_reindex_command, NOTMUCH_CONFIG_OPEN, + { "reindex", notmuch_reindex_command, NOTMUCH_COMMAND_DATABASE_EARLY | + NOTMUCH_COMMAND_DATABASE_WRITE, "Re-index all messages matching the search terms." }, - { "config", notmuch_config_command, NOTMUCH_CONFIG_OPEN, + { "config", notmuch_config_command, NOTMUCH_COMMAND_CONFIG_LOAD, "Get or set settings in the notmuch configuration file." }, #if WITH_EMACS { "emacs-mua", NULL, 0, "send mail with notmuch and emacs." }, #endif - { "help", notmuch_help_command, NOTMUCH_CONFIG_CREATE, /* create but don't save config */ + { "help", notmuch_help_command, NOTMUCH_COMMAND_CONFIG_CREATE, /* create but don't save config */ "This message, or more detailed help for the named command." } }; @@ -179,16 +210,18 @@ typedef struct help_topic { const char *summary; } help_topic_t; -static help_topic_t help_topics[] = { - { "search-terms", - "Common search term syntax." }, +static const help_topic_t help_topics[] = { { "hooks", "Hooks that will be run before or after certain commands." }, { "properties", "Message property conventions and documentation." }, + { "search-terms", + "Common infix search term syntax." }, + { "sexp-queries", + "Common s-expression search term syntax." }, }; -static command_t * +static const command_t * find_command (const char *name) { size_t i; @@ -206,8 +239,8 @@ int notmuch_format_version; static void usage (FILE *out) { - command_t *command; - help_topic_t *topic; + const command_t *command; + const help_topic_t *topic; unsigned int i; fprintf (out, @@ -244,14 +277,16 @@ void notmuch_exit_if_unsupported_format (void) { if (notmuch_format_version > NOTMUCH_FORMAT_CUR) { - fprintf (stderr, "\ + fprintf (stderr, + "\ A caller requested output format version %d, but the installed notmuch\n\ CLI only supports up to format version %d. You may need to upgrade your\n\ notmuch CLI.\n", notmuch_format_version, NOTMUCH_FORMAT_CUR); exit (NOTMUCH_EXIT_FORMAT_TOO_NEW); } else if (notmuch_format_version < NOTMUCH_FORMAT_MIN) { - fprintf (stderr, "\ + fprintf (stderr, + "\ A caller requested output format version %d, which is no longer supported\n\ by the notmuch CLI (it requires at least version %d). You may need to\n\ upgrade your notmuch front-end.\n", @@ -261,13 +296,14 @@ upgrade your notmuch front-end.\n", /* Warn about old version requests so compatibility issues are * less likely when we drop support for a deprecated format * versions. */ - fprintf (stderr, "\ + fprintf (stderr, + "\ A caller requested deprecated output format version %d, which may not\n\ be supported in the future.\n", notmuch_format_version); } } -void +static void notmuch_exit_if_unmatched_db_uuid (notmuch_database_t *notmuch) { const char *uuid = NULL; @@ -295,9 +331,7 @@ exec_man (const char *page) static int _help_for (const char *topic_name) { - command_t *command; - help_topic_t *topic; - unsigned int i; + char *page; if (! topic_name) { printf ("The notmuch mail system.\n\n"); @@ -314,28 +348,14 @@ _help_for (const char *topic_name) return EXIT_SUCCESS; } - command = find_command (topic_name); - if (command) { - char *page = talloc_asprintf (NULL, "notmuch-%s", command->name); - exec_man (page); - } + page = talloc_asprintf (NULL, "notmuch-%s", topic_name); + exec_man (page); - for (i = 0; i < ARRAY_SIZE (help_topics); i++) { - topic = &help_topics[i]; - if (strcmp (topic_name, topic->name) == 0) { - char *page = talloc_asprintf (NULL, "notmuch-%s", topic->name); - exec_man (page); - } - } - - fprintf (stderr, - "\nSorry, %s is not a known command. There's not much I can do to help.\n\n", - topic_name); return EXIT_FAILURE; } static int -notmuch_help_command (unused (notmuch_config_t *config), int argc, char *argv[]) +notmuch_help_command (unused(notmuch_database_t *notmuch), int argc, char *argv[]) { int opt_index; @@ -359,35 +379,24 @@ notmuch_help_command (unused (notmuch_config_t *config), int argc, char *argv[]) * to be more clever about this in the future. */ static int -notmuch_command (notmuch_config_t *config, +notmuch_command (notmuch_database_t *notmuch, unused(int argc), unused(char **argv)) { - char *db_path; - struct stat st; - /* If the user has never configured notmuch, then run + const char *config_path; + + /* If the user has not created a configuration file, then run * notmuch_setup_command which will give a nice welcome message, * and interactively guide the user through the configuration. */ - if (notmuch_config_is_new (config)) - return notmuch_setup_command (config, 0, NULL); - - /* Notmuch is already configured, but is there a database? */ - db_path = talloc_asprintf (config, "%s/%s", - notmuch_config_get_database_path (config), - ".notmuch"); - if (stat (db_path, &st)) { + config_path = notmuch_config_path (notmuch); + if (access (config_path, R_OK | F_OK) == -1) { if (errno != ENOENT) { - fprintf (stderr, "Error looking for notmuch database at %s: %s\n", - db_path, strerror (errno)); + fprintf (stderr, "Error: %s config file access failed: %s\n", config_path, + strerror (errno)); return EXIT_FAILURE; + } else { + return notmuch_setup_command (notmuch, 0, NULL); } - printf ("Notmuch is configured, but there's not yet a database at\n\n\t%s\n\n", - db_path); - printf ("You probably want to run \"notmuch new\" now to create that database.\n\n" - "Note that the first run of \"notmuch new\" can take a very long time\n" - "and that the resulting database will use roughly the same amount of\n" - "storage space as the email being indexed.\n\n"); - return EXIT_SUCCESS; } printf ("Notmuch is configured and appears to have a database. Excellent!\n\n" @@ -404,8 +413,8 @@ notmuch_command (notmuch_config_t *config, "or any other interface described at https://notmuchmail.org\n\n" "And don't forget to run \"notmuch new\" whenever new mail arrives.\n\n" "Have fun, and may your inbox never have much mail.\n\n", - notmuch_config_get_user_name (config), - notmuch_config_get_user_primary_email (config)); + notmuch_config_get (notmuch, NOTMUCH_CONFIG_USER_NAME), + notmuch_config_get (notmuch, NOTMUCH_CONFIG_PRIMARY_EMAIL)); return EXIT_SUCCESS; } @@ -420,11 +429,18 @@ notmuch_command (notmuch_config_t *config, * false on errors. */ static bool -try_external_command (char *argv[]) +try_external_command (const char *config_file_name, char *argv[]) { char *old_argv0 = argv[0]; bool ret = true; + if (config_file_name) { + if (setenv ("NOTMUCH_CONFIG", config_file_name, 1)) { + perror ("setenv"); + exit (1); + } + } + argv[0] = talloc_asprintf (NULL, "notmuch-%s", old_argv0); /* @@ -450,27 +466,22 @@ main (int argc, char *argv[]) void *local; char *talloc_report; const char *command_name = NULL; - command_t *command; + const command_t *command; const char *config_file_name = NULL; - notmuch_config_t *config = NULL; + notmuch_database_t *notmuch = NULL; int opt_index; - int ret; + int ret = EXIT_SUCCESS; notmuch_opt_desc_t options[] = { - { .opt_string = &config_file_name, .name = "config" }, + { .opt_string = &config_file_name, .name = "config", .allow_empty = TRUE }, { .opt_inherit = notmuch_shared_options }, { } }; - talloc_enable_null_tracking (); + notmuch_client_init (); local = talloc_new (NULL); - g_mime_init (); -#if ! GLIB_CHECK_VERSION (2, 35, 1) - g_type_init (); -#endif - /* Globally default to the current output format version. */ notmuch_format_version = NOTMUCH_FORMAT_CUR; @@ -483,38 +494,119 @@ main (int argc, char *argv[]) if (opt_index < argc) command_name = argv[opt_index]; - notmuch_process_shared_options (command_name); + notmuch_process_shared_options (NULL, command_name); command = find_command (command_name); /* if command->function is NULL, try external command */ if (! command || ! command->function) { /* This won't return if the external command is found. */ - if (try_external_command (argv + opt_index)) + if (try_external_command (config_file_name, argv + opt_index)) fprintf (stderr, "Error: Unknown command '%s' (see \"notmuch help\")\n", command_name); ret = EXIT_FAILURE; goto DONE; } - config = notmuch_config_open (local, config_file_name, command->config_mode); - if (! config) { - ret = EXIT_FAILURE; - goto DONE; + if (command->mode & NOTMUCH_COMMAND_DATABASE_EARLY) { + char *status_string = NULL; + notmuch_database_mode_t mode; + notmuch_status_t status; + + if (command->mode & NOTMUCH_COMMAND_DATABASE_WRITE || + command->mode & NOTMUCH_COMMAND_DATABASE_CREATE) + mode = NOTMUCH_DATABASE_MODE_READ_WRITE; + else + mode = NOTMUCH_DATABASE_MODE_READ_ONLY; + + if (command->mode & NOTMUCH_COMMAND_DATABASE_CREATE) { + status = notmuch_database_create_with_config (NULL, + config_file_name, + NULL, + ¬much, + &status_string); + if (status && status != NOTMUCH_STATUS_DATABASE_EXISTS) { + if (status_string) { + fputs (status_string, stderr); + free (status_string); + } + + if (status == NOTMUCH_STATUS_NO_CONFIG) + fputs ("Try running 'notmuch setup' to create a configuration.\n", stderr); + + return EXIT_FAILURE; + } + } + + if (notmuch == NULL) { + status = notmuch_database_open_with_config (NULL, + mode, + config_file_name, + NULL, + ¬much, + &status_string); + if (status) { + if (status_string) { + fputs (status_string, stderr); + free (status_string); + } + + return EXIT_FAILURE; + } + } } - ret = (command->function)(config, argc - opt_index, argv + opt_index); + if (command->mode & NOTMUCH_COMMAND_CONFIG_LOAD) { + char *status_string = NULL; + notmuch_status_t status; + status = notmuch_database_load_config (NULL, + config_file_name, + NULL, + ¬much, + &status_string); + if (status_string) { + fputs (status_string, stderr); + free (status_string); + status_string = NULL; + } - DONE: - if (config) - notmuch_config_close (config); + switch (status) { + case NOTMUCH_STATUS_NO_CONFIG: + if (! (command->mode & NOTMUCH_COMMAND_CONFIG_CREATE)) { + fputs ("Try running 'notmuch setup' to create a configuration.\n", stderr); + goto DONE; + } + break; + case NOTMUCH_STATUS_NO_DATABASE: + if (! command_name) { + printf ("Notmuch is configured, but no database was found.\n"); + printf ("You probably want to run \"notmuch new\" now to create a database.\n\n" + "Note that the first run of \"notmuch new\" can take a very long time\n" + "and that the resulting database will use roughly the same amount of\n" + "storage space as the email being indexed.\n\n"); + status = NOTMUCH_STATUS_SUCCESS; + goto DONE; + } + break; + case NOTMUCH_STATUS_SUCCESS: + break; + default: + fputs ("Error: unable to load config file.\n", stderr); + ret = 1; + goto DONE; + } + + } + ret = (command->function)(notmuch, argc - opt_index, argv + opt_index); + + DONE: talloc_report = getenv ("NOTMUCH_TALLOC_REPORT"); if (talloc_report && strcmp (talloc_report, "") != 0) { /* this relies on the previous call to * talloc_enable_null_tracking */ - FILE *report = fopen (talloc_report, "w"); + FILE *report = fopen (talloc_report, "a"); if (report) { talloc_report_full (NULL, report); } else { diff --git a/parse-time-string/Makefile.local b/parse-time-string/Makefile.local index 53534f3e..ee8030cc 100644 --- a/parse-time-string/Makefile.local +++ b/parse-time-string/Makefile.local @@ -1,3 +1,5 @@ +# -*- makefile-gmake -*- + dir := parse-time-string extra_cflags += -I$(srcdir)/$(dir) diff --git a/performance-test/Makefile.local b/performance-test/Makefile.local index 9dc260e3..b9f580c7 100644 --- a/performance-test/Makefile.local +++ b/performance-test/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := performance-test diff --git a/performance-test/README b/performance-test/README index fbc61028..1ca0df2f 100644 --- a/performance-test/README +++ b/performance-test/README @@ -16,6 +16,7 @@ In addition to having notmuch, you need: - xz. Some speedup can be gotten by installing "pixz", but this is probably only worthwhile if you are debugging the tests. - valgrind (for the memory tests) +- perf (optional, for more fine-grained timing) Getting set up to run tests: ---------------------------- @@ -23,11 +24,11 @@ Getting set up to run tests: First, you need to get the corpus. If you don't already have the gpg key for David Bremner, run - % gpg --search 'david@tethera.net' + % gpg --locate-external-key 'david@tethera.net' This should get you a key with fingerprint - 815B 6398 2A79 F8E7 C727 86C4 762B 57BB 7842 06AD + 7A18 807F 100A 4570 C596 8420 7E4E 65C8 720B 706B (the last 8 digits are printed as the "key id"). @@ -56,11 +57,24 @@ supports the following arguments --small / --medium / --large Choose corpus size. --debug Enable debugging. In particular don't delete - temporary directories. + temporary directories. +--perf Run perf record in place of /usr/bin/time. Perf output can be + found in a log directory. +--call-graph {fp,lbr,dwarf} Call graph option for perf record. Default is 'lbr'. When using the make targets, you can pass arguments to all test scripts by defining the make variable OPTIONS. +Log Directory +------------- + +The memory tests, and the time tests when option '--perf' is given +save their output in a directory named as follows + + log.$test_name-$corpus_size-$timestamp + +These directories are removed by "make clean". + Writing tests ------------- diff --git a/performance-test/T00-new.sh b/performance-test/T00-new.sh index a14dd13f..de260b2d 100755 --- a/performance-test/T00-new.sh +++ b/performance-test/T00-new.sh @@ -5,16 +5,16 @@ test_description='notmuch new' . $(dirname "$0")/perf-test-lib.sh || exit 1 uncache_database - time_start +manifest=$(mktemp manifestXXXXXX) +find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest +xargs tar uf backup.tar < $manifest + for i in $(seq 2 6); do time_run "notmuch new #$i" 'notmuch new' done -manifest=$(mktemp manifestXXXXXX) - -find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest # arithmetic context is to eat extra whitespace on e.g. some BSDs count=$((`wc -l < $manifest`)) @@ -26,6 +26,14 @@ perl -nle 'rename "$_.renamed", $_' $manifest time_run "new ($count mv back)" 'notmuch new' +perl -nle 'unlink $_; unlink $_.copy' $manifest + +time_run "new ($count rm)" 'notmuch new' + +tar xf backup.tar + +time_run "new ($count restore)" 'notmuch new' + perl -nle 'link $_, "$_.copy"' $manifest time_run "new ($count cp)" 'notmuch new' diff --git a/performance-test/T02-tag.sh b/performance-test/T02-tag.sh index 9c895d6a..47fdb0c2 100755 --- a/performance-test/T02-tag.sh +++ b/performance-test/T02-tag.sh @@ -11,4 +11,13 @@ time_run 'tag * +existing_tag' "notmuch tag +new_tag '*'" time_run 'tag * -existing_tag' "notmuch tag -new_tag '*'" time_run 'tag * -missing_tag' "notmuch tag -new_tag '*'" +time_run 'tag * +maildir_flag F' "notmuch tag +flagged '*'" +time_run 'tag * -maildir_flag F' "notmuch tag -flagged '*'" +time_run 'tag * +maildir_flag P' "notmuch tag +passed '*'" +time_run 'tag * -maildir_flag P' "notmuch tag -passed '*'" +time_run 'tag * +maildir_flag D' "notmuch tag +draft '*'" +time_run 'tag * -maildir_flag D' "notmuch tag -draft '*'" +time_run 'tag * +maildir_flag S' "notmuch tag -unread '*'" +time_run 'tag * -maildir_flag S' "notmuch tag +unread '*'" + time_done diff --git a/performance-test/T03-reindex.sh b/performance-test/T03-reindex.sh index 8db52a33..b58950d7 100755 --- a/performance-test/T03-reindex.sh +++ b/performance-test/T03-reindex.sh @@ -10,4 +10,32 @@ time_run 'reindex *' "notmuch reindex '*'" time_run 'reindex *' "notmuch reindex '*'" time_run 'reindex *' "notmuch reindex '*'" +manifest=$(mktemp manifestXXXXXX) + +find mail -type f ! -path 'mail/.notmuch/*' | sed -n '1~4 p' > $manifest +# arithmetic context is to eat extra whitespace on e.g. some BSDs +count=$((`wc -l < $manifest`)) + +xargs tar uf backup.tar < $manifest + +perl -nle 'rename $_, "$_.renamed"' $manifest + +time_run "reindex ($count mv)" "notmuch reindex '*'" + +perl -nle 'rename "$_.renamed", $_' $manifest + +time_run "reindex ($count mv back)" "notmuch reindex '*'" + +perl -nle 'unlink $_; unlink $_.copy' $manifest + +time_run "reindex ($count rm)" "notmuch reindex '*'" + +tar xf backup.tar + +time_run "reindex ($count restore)" "notmuch reindex '*'" + +perl -nle 'link $_, "$_.copy"' $manifest + +time_run "reindex ($count cp)" "notmuch reindex '*'" + time_done diff --git a/performance-test/T05-ruby.sh b/performance-test/T05-ruby.sh new file mode 100755 index 00000000..527ab28b --- /dev/null +++ b/performance-test/T05-ruby.sh @@ -0,0 +1,24 @@ +#!/usr/bin/env bash + +test_description='ruby bindings' + +. $(dirname "$0")/perf-test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_RUBY_DEV}" = "0" ]; then + echo "missing prerequisites: ruby development files" + exit 0 +fi + +time_start + +time_run 'print all messages' "$NOTMUCH_RUBY -I '$NOTMUCH_BUILDDIR/bindings/ruby' <<'EOF' +require 'notmuch' +db = Notmuch::Database.new('$MAIL_DIR') +100.times.each do + db.query('').search_messages.each do |msg| + puts msg.message_id + end +end +EOF" + +time_done diff --git a/performance-test/T06-emacs.sh b/performance-test/T06-emacs.sh new file mode 100755 index 00000000..c92bbd66 --- /dev/null +++ b/performance-test/T06-emacs.sh @@ -0,0 +1,84 @@ +#!/usr/bin/env bash + +test_description='emacs operations' + +. $(dirname "$0")/perf-test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs + +time_start + +print_emacs_header + +MSGS=$(notmuch search --output=messages "*" | shuf -n 50 | awk '{printf " \"%s\"",$1}') + +time_emacs "tag messages" \ +"(dolist (msg (list $MSGS)) + (notmuch-tag msg (list \"+test\")) + (notmuch-tag msg (list \"-test\"))))" + +time_emacs "show warmup" \ + '(notmuch-show "thread:{id:tip-4f8219875a0dad2cfad9e93a3fafcd9626db98d2@git.kernel.org}")' + +time_emacs "show thread #1" \ + '(notmuch-show "thread:{id:tip-4f8219875a0dad2cfad9e93a3fafcd9626db98d2@git.kernel.org}")' + +time_emacs "depth bound #1" \ + '(let ((notmuch-show-depth-limit 0)) + (notmuch-show "thread:{id:tip-4f8219875a0dad2cfad9e93a3fafcd9626db98d2@git.kernel.org}"))' + +time_emacs "height bound #1" \ + '(let ((notmuch-show-height-limit -1)) + (notmuch-show "thread:{id:tip-4f8219875a0dad2cfad9e93a3fafcd9626db98d2@git.kernel.org}"))' + +time_emacs "size bound #1" \ + '(let ((notmuch-show-max-text-part-size 1)) + (notmuch-show "thread:{id:tip-4f8219875a0dad2cfad9e93a3fafcd9626db98d2@git.kernel.org}"))' + +time_emacs "show thread #2" \ + '(notmuch-show "thread:{id:20101208005731.943729010@clark.site}")' + +time_emacs "depth bound #2" \ + '(let ((notmuch-show-depth-limit 0)) + (notmuch-show "thread:{id:20101208005731.943729010@clark.site}"))' + +time_emacs "height bound #2" \ + '(let ((notmuch-show-height-limit -1)) + (notmuch-show "thread:{id:20101208005731.943729010@clark.site}"))' + +time_emacs "size bound #2" \ + '(let ((notmuch-show-max-text-part-size 1)) + (notmuch-show "thread:{id:20101208005731.943729010@clark.site}"))' + +time_emacs "show thread #3" \ + '(notmuch-show "thread:{id:20120109014938.GE20796@mit.edu}")' + +time_emacs "depth bound #3" \ + '(let ((notmuch-show-depth-limit 0)) + (notmuch-show "thread:{id:20120109014938.GE20796@mit.edu}"))' + +time_emacs "height bound #3" \ + '(let ((notmuch-show-height-limit -1)) + (notmuch-show "thread:{id:20120109014938.GE20796@mit.edu}"))' + +time_emacs "size bound #3" \ + '(let ((notmuch-show-max-text-part-size 1)) + (notmuch-show "thread:{id:20120109014938.GE20796@mit.edu}"))' + +time_emacs "show thread #4" \ + '(notmuch-show "thread:{id:1280704593.25620.48.camel@mulgrave.site}")' + +time_emacs "depth bound #4" \ + '(let ((notmuch-show-depth-limit 0)) + (notmuch-show "thread:{id:1280704593.25620.48.camel@mulgrave.site}"))' + +time_emacs "height bound #4" \ + '(let ((notmuch-show-height-limit -1)) + (notmuch-show "thread:{id:1280704593.25620.48.camel@mulgrave.site}"))' + +time_emacs "size bound #4" \ + '(let ((notmuch-show-max-text-part-size 1)) + (notmuch-show "thread:{id:1280704593.25620.48.camel@mulgrave.site}"))' + +time_done diff --git a/performance-test/T07-git.sh b/performance-test/T07-git.sh new file mode 100755 index 00000000..11dfec05 --- /dev/null +++ b/performance-test/T07-git.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash + +test_description='notmuch-git' + +. $(dirname "$0")/perf-test-lib.sh || exit 1 + +time_start + +time_run 'init' "notmuch git init" + +time_run 'commit --force' "notmuch git commit --force" +time_run 'commit' "notmuch git -l error commit" +time_run 'commit' "notmuch git -l error commit" + +time_run 'checkout' "notmuch git checkout" + +time_run 'tag -inbox' "notmuch tag -inbox '*'" + +time_run 'checkout --force' "notmuch git checkout --force" + + + +time_done diff --git a/performance-test/download/notmuch-email-corpus-0.5.tar.xz.asc b/performance-test/download/notmuch-email-corpus-0.5.tar.xz.asc new file mode 100644 index 00000000..2318c2f6 --- /dev/null +++ b/performance-test/download/notmuch-email-corpus-0.5.tar.xz.asc @@ -0,0 +1,16 @@ +-----BEGIN PGP SIGNATURE----- + +iQIzBAABCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAmR119gACgkQA0U5G1Wq +FSHlSQ/+NSRj27PEZjaP2I+3j+rsMG3pnVckNcuOQyfgjJ+zEagMZyRu3vaIA/pX +xtBrNIX4l4CQIkqwyNjsuqJdzh6S3DeCWSEr1Q+GSBki+wQCBiRuDYY2HQoDezEK +4bMfniEWZpKJD8PfIabz0OOqMUsfXEYMd9kefew5/J4OGnDIv8E5pKfqvDNNO4rW +MhZ9w9uR9wkvmfmpO66kAgTfLllwiyNHWoWnzQfNmqM8eULFn7XxM1PEZShUEqXf +pTWCqqm5OyUcy8f+gy9Mb7DRRvnwLpHHRQlCzzH2c+ENQRpt1ErsgVKpHTVk4UsB +EML+zwyWEaQg7xVKWXRJDuGCF47S1GCQNUtvtx57HJl6Ds6N2mlr2KEGaI7qtiz5 +5hdaTc0L/TVN0WS+uCdfdDDozFErf1kwhA6Jnpi0YNNdK+wpFzj7ISvA+DNHwJ75 +TLBuJIU/h3QfX3NDC5xDbsWAgpv7a84e7ePO6+kAVkHsNYDbFjiunr5fRbqDsJcJ +B+aVGhKvFZbziC84Dn5Ae9Lpa40fQlxbdb+So2nDIiuR3P33vt7wr/e2ptVfrqkn +a1DM96n03VWexwEDMye3b3rOTXsN5Ul87zucg9xWm5JT75NGuqJ1WDJN/wwNPDro +ZXS1OHh7UKsU1tP2J9+gLiKYNBP4m4BQjEgYXpiYEoge9A1QplQ= +=5/Ep +-----END PGP SIGNATURE----- diff --git a/performance-test/perf-test-lib.sh b/performance-test/perf-test-lib.sh index b70288cc..c34f8cd6 100644 --- a/performance-test/perf-test-lib.sh +++ b/performance-test/perf-test-lib.sh @@ -1,6 +1,9 @@ . $(dirname "$0")/version.sh || exit 1 +debug="" corpus_size=large +perf_callgraph=lbr +use_perf=0 while test "$#" -ne 0 do @@ -9,6 +12,15 @@ do debug=t; shift ;; + -p|--perf) + use_perf=1; + shift + ;; + -c|--call-graph) + shift + perf_callgraph=$1 + shift + ;; -s|--small) corpus_size=small; shift @@ -29,6 +41,8 @@ done # Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set. . $(dirname "$0")/../test/export-dirs.sh || exit 1 +. "$NOTMUCH_SRCDIR/test/test-vars.sh" || exit 1 + # Where to run the tests TEST_DIRECTORY=$NOTMUCH_BUILDDIR/performance-test @@ -127,10 +141,20 @@ notmuch_new_with_cache () fi } +make_log_dir () { + local timestamp=$(date +%Y%m%dT%H%M%S) + log_dir=${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp} + mkdir -p "${log_dir}" +} + time_start () { add_email_corpus + if [[ "$use_perf" = 1 ]]; then + make_log_dir + fi + print_header notmuch_new_with_cache time_run @@ -140,9 +164,7 @@ memory_start () { add_email_corpus - local timestamp=$(date +%Y%m%dT%H%M%S) - log_dir="${TEST_DIRECTORY}/log.$(basename $0)-$corpus_size-${timestamp}" - mkdir -p ${log_dir} + make_log_dir notmuch_new_with_cache memory_run } @@ -188,12 +210,23 @@ print_header () printf "\t\t\tWall(s)\tUsr(s)\tSys(s)\tRes(K)\tIn/Out(512B)\n" } +print_emacs_header () +{ + printf "\t\t\tWall(s)\tGCs\tGC time(s)\n" +} + time_run () { printf " %-22s" "$1" test_count=$(($test_count+1)) - if test "$verbose" != "t"; then exec 4>test.output 3>&4; fi - if ! eval >&3 "/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2" ; then + if test "$verbose" != "t"; then exec 4>test.output 3>&4; else exec 3>&1; fi + if [[ "$use_perf" = 1 ]]; then + command_str="perf record --call-graph=${perf_callgraph} -o ${log_dir}/${test_count}.perf $2" + else + command_str="/usr/bin/time -f '%e\t%U\t%S\t%M\t%I/%O' $2" + fi + + if ! eval >&3 "$command_str" ; then test_failure=$(($test_failure + 1)) return 1 fi diff --git a/performance-test/version.sh b/performance-test/version.sh index f02527a7..357b9dad 100644 --- a/performance-test/version.sh +++ b/performance-test/version.sh @@ -1,3 +1,3 @@ # this should be both a valid Makefile fragment and valid POSIX(ish) shell. -PERFTEST_VERSION=0.4 +PERFTEST_VERSION=0.5 diff --git a/sprinter-json.c b/sprinter-json.c index c6ec8577..502f89fb 100644 --- a/sprinter-json.c +++ b/sprinter-json.c @@ -1,3 +1,4 @@ +#include #include #include #include @@ -124,11 +125,11 @@ json_string (struct sprinter *sp, const char *val) } static void -json_integer (struct sprinter *sp, int val) +json_integer (struct sprinter *sp, int64_t val) { struct sprinter_json *spj = json_begin_value (sp); - fprintf (spj->stream, "%d", val); + fprintf (spj->stream, "%" PRId64, val); } static void @@ -171,7 +172,7 @@ json_separator (struct sprinter *sp) } struct sprinter * -sprinter_json_create (const void *ctx, FILE *stream) +sprinter_json_create (notmuch_database_t *db, FILE *stream) { static const struct sprinter_json template = { .vtable = { @@ -191,11 +192,12 @@ sprinter_json_create (const void *ctx, FILE *stream) }; struct sprinter_json *res; - res = talloc (ctx, struct sprinter_json); + res = talloc (db, struct sprinter_json); if (! res) return NULL; *res = template; + res->vtable.notmuch = db; res->stream = stream; return &res->vtable; } diff --git a/sprinter-sexp.c b/sprinter-sexp.c index 6891ea42..e37cb1f9 100644 --- a/sprinter-sexp.c +++ b/sprinter-sexp.c @@ -18,6 +18,7 @@ * Author: Peter Feigl */ +#include #include #include #include @@ -161,11 +162,11 @@ sexp_keyword (struct sprinter *sp, const char *val) } static void -sexp_integer (struct sprinter *sp, int val) +sexp_integer (struct sprinter *sp, int64_t val) { struct sprinter_sexp *sps = sexp_begin_value (sp); - fprintf (sps->stream, "%d", val); + fprintf (sps->stream, "%" PRId64, val); } static void @@ -206,7 +207,7 @@ sexp_separator (struct sprinter *sp) } struct sprinter * -sprinter_sexp_create (const void *ctx, FILE *stream) +sprinter_sexp_create (notmuch_database_t *db, FILE *stream) { static const struct sprinter_sexp template = { .vtable = { @@ -226,11 +227,12 @@ sprinter_sexp_create (const void *ctx, FILE *stream) }; struct sprinter_sexp *res; - res = talloc (ctx, struct sprinter_sexp); + res = talloc (db, struct sprinter_sexp); if (! res) return NULL; *res = template; + res->vtable.notmuch = db; res->stream = stream; return &res->vtable; } diff --git a/sprinter-text.c b/sprinter-text.c index 648b54b1..99330a94 100644 --- a/sprinter-text.c +++ b/sprinter-text.c @@ -1,3 +1,4 @@ +#include #include #include #include @@ -44,11 +45,11 @@ text_string (struct sprinter *sp, const char *val) } static void -text_integer (struct sprinter *sp, int val) +text_integer (struct sprinter *sp, int64_t val) { struct sprinter_text *sptxt = (struct sprinter_text *) sp; - fprintf (sptxt->stream, "%d", val); + fprintf (sptxt->stream, "%" PRId64, val); } static void @@ -113,7 +114,7 @@ text_map_key (unused (struct sprinter *sp), unused (const char *key)) } struct sprinter * -sprinter_text_create (const void *ctx, FILE *stream) +sprinter_text_create (notmuch_database_t *db, FILE *stream) { static const struct sprinter_text template = { .vtable = { @@ -133,21 +134,22 @@ sprinter_text_create (const void *ctx, FILE *stream) }; struct sprinter_text *res; - res = talloc (ctx, struct sprinter_text); + res = talloc (db, struct sprinter_text); if (! res) return NULL; *res = template; + res->vtable.notmuch = db; res->stream = stream; return &res->vtable; } struct sprinter * -sprinter_text0_create (const void *ctx, FILE *stream) +sprinter_text0_create (notmuch_database_t *db, FILE *stream) { struct sprinter *sp; - sp = sprinter_text_create (ctx, stream); + sp = sprinter_text_create (db, stream); if (! sp) return NULL; diff --git a/sprinter.h b/sprinter.h index 182b1a8b..fd08641c 100644 --- a/sprinter.h +++ b/sprinter.h @@ -9,6 +9,11 @@ * (strings, integers and booleans). */ typedef struct sprinter { + /* + * Open notmuch database + */ + notmuch_database_t *notmuch; + /* Start a new map/dictionary structure. This should be followed by * a sequence of alternating calls to map_key and one of the * value-printing functions until the map is ended by end. @@ -33,7 +38,7 @@ typedef struct sprinter { */ void (*string)(struct sprinter *, const char *); void (*string_len)(struct sprinter *, const char *, size_t); - void (*integer)(struct sprinter *, int); + void (*integer)(struct sprinter *, int64_t); void (*boolean)(struct sprinter *, bool); void (*null)(struct sprinter *); @@ -65,20 +70,20 @@ typedef struct sprinter { /* Create a new unstructured printer that emits the default text format * for "notmuch search". */ struct sprinter * -sprinter_text_create (const void *ctx, FILE *stream); +sprinter_text_create (notmuch_database_t *db, FILE *stream); /* Create a new unstructured printer that emits the text format for * "notmuch search", with each field separated by a null character * instead of the newline character. */ struct sprinter * -sprinter_text0_create (const void *ctx, FILE *stream); +sprinter_text0_create (notmuch_database_t *db, FILE *stream); /* Create a new structure printer that emits JSON. */ struct sprinter * -sprinter_json_create (const void *ctx, FILE *stream); +sprinter_json_create (notmuch_database_t *db, FILE *stream); /* Create a new structure printer that emits S-Expressions. */ struct sprinter * -sprinter_sexp_create (const void *ctx, FILE *stream); +sprinter_sexp_create (notmuch_database_t *db, FILE *stream); #endif // NOTMUCH_SPRINTER_H diff --git a/status.c b/status.c index d0ae47f4..09d82a17 100644 --- a/status.c +++ b/status.c @@ -72,3 +72,17 @@ status_to_exit (notmuch_status_t status) return EXIT_FAILURE; } } + +notmuch_status_t +print_status_gzbytes (const char *loc, gzFile file, int bytes) +{ + if (bytes <= 0) { + int errnum; + const char *errstr = gzerror (file, &errnum); + fprintf (stderr, "%s: zlib error %s (%d)\n", loc, errstr, errnum); + return NOTMUCH_STATUS_FILE_ERROR; + } else { + return NOTMUCH_STATUS_SUCCESS; + } +} + diff --git a/tag-util.c b/tag-util.c index 1837b1ae..accf299e 100644 --- a/tag-util.c +++ b/tag-util.c @@ -323,7 +323,7 @@ tag_op_list_apply (notmuch_message_t *message, if (flags & TAG_FLAG_MAILDIR_SYNC) { status = notmuch_message_tags_to_maildir_flags (message); if (status) { - message_error (message, status, "synching tags to maildir"); + message_error (message, status, "syncing tags to maildir"); return status; } } diff --git a/tag-util.h b/tag-util.h index bbe54d99..411e8cae 100644 --- a/tag-util.h +++ b/tag-util.h @@ -123,7 +123,7 @@ tag_op_list_append (tag_op_list_t *list, /* * Apply a list of tag operations, in order, to a given message. * - * Flags can be bitwise ORed; see enum above for possibilies. + * Flags can be bitwise ORed; see enum above for possibilities. */ notmuch_status_t diff --git a/test/Makefile.local b/test/Makefile.local index 47244e8f..40574739 100644 --- a/test/Makefile.local +++ b/test/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := test diff --git a/test/README b/test/README index 3f54af58..a81808b1 100644 --- a/test/README +++ b/test/README @@ -25,6 +25,7 @@ that you know if you break anything. - gdb(1) - gpg(1) - python(1) + - xapian-metadata(1) If your system lacks these tools or have older, non-upgradable versions of these, please (possibly compile and) install these to some other @@ -79,14 +80,6 @@ The following command-line options are available when running tests: As the names depend on the tests' file names, it is safe to run the tests with this option in parallel. -Certain tests require precomputed databases to complete. You can fetch these -databases with - - make download-test-databases - -If you do not download the test databases, the relevant tests will be -skipped. - When invoking the test suite via "make test" any of the above options can be specified as follows: @@ -144,6 +137,23 @@ detection of missing prerequisites. In the future we may treat tests unable to run because of missing prerequisites, but not explicitly skipped by the user, as failures. +Testing installed notmuch +------------------------- + +Systems integrators (e.g. Linux distros) may wish to test an installed +version of notmuch. This can be done be running + + $ NOTMUCH_TEST_INSTALLED=1 ./test/notmuch-test + +In this scenario the test suite does not assume a built tree, and in +particular cannot rely on the output of 'configure'. You may want to +set certain feature environment variables ('NOTMUCH_HAVE_*') directly +if you know those apply to your installed notmuch). Consider also +setting TERM=dumb if the value of TERM cannot be used (e.g. in a +chroot with missing terminfo). Note that having a built tree may cause +surprising/broken results for NOTMUCH_TEST_INSTALLED, so consider +cleaning first. + Writing Tests ------------- The test script is written as a shell script. It is to be named as diff --git a/test/T000-basic.sh b/test/T000-basic.sh index 7fbdcfa3..642f918d 100755 --- a/test/T000-basic.sh +++ b/test/T000-basic.sh @@ -33,7 +33,7 @@ test_begin_subtest 'failure to clean up causes the test to fail' test_expect_code 2 'test_when_finished "(exit 2)"' EXPECTED=$NOTMUCH_SRCDIR/test/test.expected-output -suppress_diff_date() { +suppress_diff_date () { sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \ -e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/' } @@ -66,6 +66,7 @@ test_begin_subtest 'NOTMUCH_CONFIG is set and points to an existing file' test_expect_success 'test -f "${NOTMUCH_CONFIG}"' test_begin_subtest 'PATH is set to build directory' +test_subtest_broken_for_installed test_expect_equal \ "$(dirname ${TEST_DIRECTORY})" \ "$(echo $PATH|cut -f1 -d: | sed -e 's,/test/valgrind/bin$,,')" diff --git a/test/T010-help-test.sh b/test/T010-help-test.sh index da45d3ae..827edc14 100755 --- a/test/T010-help-test.sh +++ b/test/T010-help-test.sh @@ -12,13 +12,16 @@ test_expect_success 'notmuch help' test_begin_subtest 'notmuch --version' test_expect_success 'notmuch --version' -if [ $NOTMUCH_HAVE_MAN -eq 1 ]; then +if [ "${NOTMUCH_HAVE_MAN-0}" = "1" ]; then test_begin_subtest 'notmuch --help tag' test_expect_success 'notmuch --help tag' test_begin_subtest 'notmuch help tag' test_expect_success 'notmuch help tag' else + if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done + fi test_begin_subtest 'notmuch --help tag (man pages not available)' test_expect_success 'test_must_fail notmuch --help tag >/dev/null' diff --git a/test/T020-compact.sh b/test/T020-compact.sh index 58cd2ba7..d77db00d 100755 --- a/test/T020-compact.sh +++ b/test/T020-compact.sh @@ -10,20 +10,8 @@ notmuch tag +tag1 \* notmuch tag +tag2 subject:Two notmuch tag -tag1 +tag3 subject:Three -if [ $NOTMUCH_HAVE_XAPIAN_COMPACT -eq 0 ]; then - test_begin_subtest "Compact unsupported: error message" - output=$(notmuch compact --quiet 2>&1) - test_expect_equal "$output" "notmuch was compiled against a xapian version lacking compaction support. -Compaction failed: Unsupported operation" - - test_begin_subtest "Compact unsupported: status code" - test_expect_code 1 "notmuch compact" - - test_done -fi - test_begin_subtest "Running compact" -test_expect_success "notmuch compact --backup=${TEST_DIRECTORY}/xapian.old" +test_expect_success "notmuch compact --backup=${TMP_DIRECTORY}/xapian.old" test_begin_subtest "Compact preserves database" output=$(notmuch search \* | notmuch_search_sanitize) @@ -34,7 +22,7 @@ thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Three (inbox tag3 unread)" test_begin_subtest "Restoring Backup" test_expect_success 'rm -Rf ${MAIL_DIR}/.notmuch/xapian && - mv ${TEST_DIRECTORY}/xapian.old ${MAIL_DIR}/.notmuch/xapian' + mv ${TMP_DIRECTORY}/xapian.old ${MAIL_DIR}/.notmuch/xapian' test_begin_subtest "Checking restored backup" output=$(notmuch search \* | notmuch_search_sanitize) diff --git a/test/T030-config.sh b/test/T030-config.sh index 883541d5..621e0b69 100755 --- a/test/T030-config.sh +++ b/test/T030-config.sh @@ -3,13 +3,18 @@ test_description='"notmuch config"' . $(dirname "$0")/test-lib.sh || exit 1 +cp notmuch-config initial-config + test_begin_subtest "Get string value" test_expect_equal "$(notmuch config get user.name)" "Notmuch Test Suite" test_begin_subtest "Get list value" -test_expect_equal "$(notmuch config get new.tags)" "\ +cat < EXPECTED +inbox unread -inbox" +EOF +notmuch config get new.tags | sort > OUTPUT +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Set string value" notmuch config set foo.string "this is a string value" @@ -43,28 +48,57 @@ notmuch config set foo.nonexistent test_expect_equal "$(notmuch config get foo.nonexistent)" "" test_begin_subtest "List all items" -notmuch config list > STDOUT 2> STDERR -printf "%s\n====\n%s\n" "$(< STDOUT)" "$(< STDERR)" | notmuch_config_sanitize > OUTPUT - +notmuch config list 2>&1 | notmuch_config_sanitize > OUTPUT cat < EXPECTED -database.path=MAIL_DIR -user.name=Notmuch Test Suite -user.primary_email=test_suite@notmuchmail.org -user.other_email=test_suite_other@notmuchmail.org;test_suite@otherdomain.org -new.tags=unread;inbox; -new.ignore= -search.exclude_tags= -maildir.synchronize_flags=true -foo.string=this is another string value -foo.list=this;is another;list value; built_with.compact=something built_with.field_processor=something built_with.retry_lock=something -==== -Error opening database at MAIL_DIR/.notmuch: No such file or directory +built_with.sexp_queries=something +database.autocommit=8000 +database.mail_root=MAIL_DIR +database.path=MAIL_DIR +foo.list=this;is another;list value; +foo.string=this is another string value +index.as_text= +maildir.synchronize_flags=true +new.ignore= +new.tags=unread;inbox +search.exclude_tags= +user.name=Notmuch Test Suite +user.other_email=test_suite_other@notmuchmail.org;test_suite@otherdomain.org +user.primary_email=test_suite@notmuchmail.org EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Round trip config item with leading spaces" +test_subtest_known_broken +notmuch config set foo.bar " thing" +output=$(notmuch config get foo.bar) +test_expect_equal "${output}" " thing" + +test_begin_subtest "Round trip config item with leading tab" +test_subtest_known_broken +notmuch config set foo.bar " thing" +output=$(notmuch config get foo.bar) +test_expect_equal "${output}" " thing" + +test_begin_subtest "Round trip config item with embedded tab" +notmuch config set foo.bar "thing other" +output=$(notmuch config get foo.bar) +test_expect_equal "${output}" "thing other" + +test_begin_subtest "Round trip config item with embedded backslash" +notmuch config set foo.bar 'thing\other' +output=$(notmuch config get foo.bar) +test_expect_equal "${output}" "thing\other" + +test_begin_subtest "Round trip config item with embedded NL/CR" +notmuch config set foo.bar 'thing + other' +output=$(notmuch config get foo.bar) +test_expect_equal "${output}" "thing + other" + test_begin_subtest "Top level --config=FILE option" cp "${NOTMUCH_CONFIG}" alt-config notmuch --config=alt-config config set user.name "Another Name" @@ -76,7 +110,7 @@ test_expect_equal "$(notmuch --config:alt-config config get user.name)" \ "Another Name" test_begin_subtest "Top level --configFILE option" -test_expect_equal "$(notmuch --config alt-config config get user.name)" \ +test_expect_equal "$(notmuch --config alt-config config get user.name)" \ "Another Name" test_begin_subtest "Top level --config=FILE option changed the right file" @@ -96,14 +130,82 @@ test_expect_equal "$(notmuch --config=alt-config-link config get user.name)" \ test_begin_subtest "Writing config file through symlink follows symlink" test_expect_equal "$(readlink alt-config-link)" "alt-config" +test_begin_subtest "Round trip arbitrary key" +key=g${RANDOM}.m${RANDOM} +value=${RANDOM} +notmuch config set ${key} ${value} +output=$(notmuch config get ${key}) +test_expect_equal "${output}" "${value}" + +test_begin_subtest "Clear arbitrary key" +notmuch config set ${key} +output=$(notmuch config get ${key}) +test_expect_equal "${output}" "" + +db_path=${HOME}/database-path + test_begin_subtest "Absolute database path returned" notmuch config set database.path ${HOME}/Maildir test_expect_equal "$(notmuch config get database.path)" \ "${HOME}/Maildir" -test_begin_subtest "Relative database path properly expanded" +ln -s `pwd`/mail home/Maildir +add_email_corpus +test_begin_subtest "Relative database path expanded" notmuch config set database.path Maildir -test_expect_equal "$(notmuch config get database.path)" \ - "${HOME}/Maildir" +path=$(notmuch config get database.path | notmuch_dir_sanitize) +count=$(notmuch count '*') +test_expect_equal "${path} ${count}" \ + "CWD/home/Maildir 52" + +test_begin_subtest "Add config to database" +notmuch new +key=g${RANDOM}.m${RANDOM} +value=${RANDOM} +notmuch config set --database ${key} ${value} +notmuch dump --include=config > OUTPUT +cat < EXPECTED +#notmuch-dump batch-tag:3 config +#@ ${key} ${value} +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Roundtrip config to/from database" +notmuch new +key=g${RANDOM}.m${RANDOM} +value=${RANDOM} +notmuch config set --database ${key} ${value} +output=$(notmuch config get ${key}) +test_expect_equal "${output}" "${value}" + +test_begin_subtest "set built_with.* yields error" +test_expect_code 1 "notmuch config set built_with.compact false" + +test_begin_subtest "get built_with.{compact,field_processor} prints true" +for key in compact field_processor; do + notmuch config get built_with.${key} +done > OUTPUT +cat < EXPECTED +true +true +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get built_with.nonexistent prints false" +output=$(notmuch config get built_with.nonexistent) +test_expect_equal "$output" "false" + +test_begin_subtest "Bad utf8 reported as error" +cp initial-config bad-config +printf '[query]\nq3=from:\xff\n' >>bad-config +test_expect_code 1 "notmuch --config=./bad-config config list" + +test_begin_subtest "Specific error message about bad utf8" +notmuch --config=./bad-config config list 2>ERRORS +cat < EXPECTED +GLib: Key file contains key “q3” with value “from:�” which is not UTF-8 +Error: unable to load config file. +EOF +test_expect_equal_file EXPECTED ERRORS test_done diff --git a/test/T035-read-config.sh b/test/T035-read-config.sh new file mode 100755 index 00000000..ac0f420b --- /dev/null +++ b/test/T035-read-config.sh @@ -0,0 +1,476 @@ +#!/usr/bin/env bash +test_description='Various options for reading configuration' +. $(dirname "$0")/test-lib.sh || exit 1 + +backup_config () { + local test_name=$(basename $0 .sh) + cp ${NOTMUCH_CONFIG} notmuch-config-backup.${test_name} +} + +xdg_config () { + local dir + local profile=${1:-default} + if [[ $profile != default ]]; then + export NOTMUCH_PROFILE=$profile + fi + backup_config + dir="${HOME}/.config/notmuch/${profile}" + rm -rf $dir + mkdir -p $dir + CONFIG_PATH=$dir/config + mv ${NOTMUCH_CONFIG} ${CONFIG_PATH} + unset NOTMUCH_CONFIG +} + +restore_config () { + local test_name=$(basename $0 .sh) + export NOTMUCH_CONFIG="${TMP_DIRECTORY}/notmuch-config" + unset CONFIG_PATH + unset NOTMUCH_PROFILE + cp notmuch-config-backup.${test_name} ${NOTMUCH_CONFIG} +} + +add_email_corpus + +test_begin_subtest "count with saved query from config file" +backup_config +query_name="test${RANDOM}" +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> notmuch-config +notmuch count query:$query_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "count with saved query from config file (xdg)" +query_name="test${RANDOM}" +xdg_config +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch count query:$query_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "count with saved query from config file (xdg+profile)" +query_name="test${RANDOM}" +xdg_config work +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch count query:$query_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +cat < EXPECTED +Before: +#notmuch-dump batch-tag:3 tags + +After: +#notmuch-dump batch-tag:3 tags ++attachment +inbox +signed +unread -- id:20091118005829.GB25380@dottiness.seas.harvard.edu ++attachment +inbox +signed +unread -- id:20091118010116.GC25380@dottiness.seas.harvard.edu ++inbox +signed +unread -- id:20091117190054.GU3165@dottiness.seas.harvard.edu ++inbox +signed +unread -- id:20091117203301.GV3165@dottiness.seas.harvard.edu ++inbox +signed +unread -- id:20091118002059.067214ed@hikari ++inbox +signed +unread -- id:20091118005040.GA25380@dottiness.seas.harvard.edu ++inbox +signed +unread -- id:87iqd9rn3l.fsf@vertex.dottedmag +EOF + +test_begin_subtest "dump with saved query from config file" +backup_config +query_name="test${RANDOM}" +CONFIG_PATH=notmuch-config +printf "Before:\n" > OUTPUT +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +printf "\nAfter:\n" >> OUTPUT +printf "\n[query]\n${query_name} = tag:signed\n" >> ${CONFIG_PATH} +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "dump with saved query from config file (xdg)" +backup_config +query_name="test${RANDOM}" +xdg_config +printf "Before:\n" > OUTPUT +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +printf "\nAfter:\n" >> OUTPUT +printf "\n[query]\n${query_name} = tag:signed\n" >> ${CONFIG_PATH} +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "dump with saved query from config file (xdg+profile)" +backup_config +query_name="test${RANDOM}" +xdg_config work +printf "Before:\n" > OUTPUT +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +printf "\nAfter:\n" >> OUTPUT +printf "\n[query]\n${query_name} = tag:signed\n" >> ${CONFIG_PATH} +notmuch dump --include=tags query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "restore with xdg config" +backup_config +notmuch dump '*' > EXPECTED +notmuch tag -inbox '*' +xdg_config +notmuch restore --input=EXPECTED +notmuch dump > OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "restore with xdg+profile config" +backup_config +notmuch dump '*' > EXPECTED +notmuch tag -inbox '*' +xdg_config work +notmuch restore --input=EXPECTED +notmuch dump > OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Insert message with custom new.tags (xdg)" +backup_config +xdg_config +tag=test${RANDOM} +notmuch --config=${CONFIG_PATH} config set new.tags $tag +generate_message \ + "[subject]=\"insert-subject\"" \ + "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \ + "[body]=\"insert-message\"" +mkdir -p ${MAIL_DIR}/{cur,new,tmp} +notmuch insert < "$gen_msg_filename" +notmuch dump id:$gen_msg_id > OUTPUT +cat < EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++$tag -- id:$gen_msg_id +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Insert message with custom new.tags (xdg+profile)" +backup_config +tag=test${RANDOM} +xdg_config $tag +notmuch --config=${CONFIG_PATH} config set new.tags $tag +generate_message \ + "[subject]=\"insert-subject\"" \ + "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \ + "[body]=\"insert-message\"" +mkdir -p ${MAIL_DIR}/{cur,new,tmp} +notmuch insert < "$gen_msg_filename" +notmuch dump id:$gen_msg_id > OUTPUT +cat < EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++$tag -- id:$gen_msg_id +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "reindex with saved query from config file" +backup_config +query_name="test${RANDOM}" +count1=$(notmuch count --lastmod '*' | cut -f3) +printf "\n[query]\n${query_name} = tag:inbox\n" >> notmuch-config +notmuch reindex query:$query_name +count2=$(notmuch count --lastmod '*' | cut -f3) +restore_config +test_expect_success "test '$count2 -gt $count1'" + +test_begin_subtest "reindex with saved query from config file (xdg)" +query_name="test${RANDOM}" +count1=$(notmuch count --lastmod '*' | cut -f3) +xdg_config +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch reindex query:$query_name +count2=$(notmuch count --lastmod '*' | cut -f3) +restore_config +test_expect_success "test '$count2 -gt $count1'" + +test_begin_subtest "reindex with saved query from config file (xdg+profile)" +query_name="test${RANDOM}" +count1=$(notmuch count --lastmod '*' | cut -f3) +xdg_config $query_name +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch reindex query:$query_name +count2=$(notmuch count --lastmod '*' | cut -f3) +restore_config +test_expect_success "test '$count2 -gt $count1'" + + + +add_message '[from]="Sender "' \ + [to]=test_suite@notmuchmail.org \ + '[cc]="Other Parties "' \ + [subject]=notmuch-reply-test \ + '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ + '[body]="reply with CC"' + +cat < EXPECTED +Before: +After: +From: Notmuch Test Suite +Subject: Re: notmuch-reply-test +To: Sender +Cc: Other Parties +In-Reply-To: <${gen_msg_id}> +References: <${gen_msg_id}> + +On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: +> reply with CC +EOF + +test_begin_subtest "reply with saved query from config file" +backup_config +query_name="test${RANDOM}" +printf "Before:\n" > OUTPUT +notmuch reply query:$query_name 2>&1 >> OUTPUT +printf "\n[query]\n${query_name} = id:${gen_msg_id}\n" >> notmuch-config +printf "After:\n" >> OUTPUT +notmuch reply query:$query_name >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "reply with saved query from config file (xdg)" +backup_config +query_name="test${RANDOM}" +xdg_config +printf "Before:\n" > OUTPUT +notmuch reply query:$query_name 2>&1 >> OUTPUT +printf "\n[query]\n${query_name} = id:${gen_msg_id}\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch reply query:$query_name >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "reply with saved query from config file (xdg+profile)" +backup_config +query_name="test${RANDOM}" +xdg_config $query_name +printf "Before:\n" > OUTPUT +notmuch reply query:$query_name 2>&1 >> OUTPUT +printf "\n[query]\n${query_name} = id:${gen_msg_id}\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch reply query:$query_name >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +backup_database +test_begin_subtest "search with alternate config" +notmuch tag -- +foobar17 '*' +cp notmuch-config alt-config +notmuch --config=alt-config config set search.exclude_tags foobar17 +output=$(notmuch --config=alt-config count '*') +test_expect_equal "$output" "0" +restore_database + +cat < EXPECTED +Before: +After: +thread:XXX 2009-11-18 [1/2] Carl Worth| Alex Botero-Lowry; [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) +thread:XXX 2009-11-18 [1/2] Carl Worth| Ingmar Vanhassel; [notmuch] [PATCH] Typsos (inbox unread) +thread:XXX 2009-11-18 [1/3] Carl Worth| Adrian Perez de Castro, Keith Packard; [notmuch] Introducing myself (inbox signed unread) +thread:XXX 2009-11-18 [1/3] Carl Worth| Israel Herraiz, Keith Packard; [notmuch] New to the list (inbox unread) +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +thread:XXX 2009-11-18 [1/2] Carl Worth| Jan Janak; [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) +thread:XXX 2009-11-18 [1/3(4)] Carl Worth| Aron Griffis, Keith Packard; [notmuch] archive (inbox unread) +thread:XXX 2009-11-18 [1/2] Carl Worth| Keith Packard; [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) +thread:XXX 2009-11-18 [1/7] Carl Worth| Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard; [notmuch] Working with Maildir storage? (inbox signed unread) +thread:XXX 2009-11-18 [2/5] Carl Worth| Mikhail Gusarov, Keith Packard; [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) +thread:XXX 2009-11-17 [1/2] Carl Worth| Alex Botero-Lowry; [notmuch] preliminary FreeBSD support (attachment inbox unread) +EOF + +test_begin_subtest "search with saved query from config file" +query_name="test${RANDOM}" +backup_config +printf "Before:\n" > OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +printf "\n[query]\n${query_name} = from:cworth\n" >> notmuch-config +printf "After:\n" >> OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search with saved query from config file (xdg)" +query_name="test${RANDOM}" +xdg_config +printf "Before:\n" > OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +printf "\n[query]\n${query_name} = from:cworth\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search with saved query from config file (xdg + profile)" +query_name="test${RANDOM}" +xdg_config $query_name +printf "Before:\n" > OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +printf "\n[query]\n${query_name} = from:cworth\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch search query:$query_name 2>&1 | notmuch_search_sanitize >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +cat < EXPECTED +Before: +After: +Alex Botero-Lowry +Alexander Botero-Lowry +François Boulogne +Jjgod Jiang +EOF + +test_begin_subtest "address: saved query from config file" +backup_config +query_name="test${RANDOM}" +printf "Before:\n" > OUTPUT +notmuch address --deduplicate=no --output=sender query:$query_name 2>&1 | sort >> OUTPUT +printf "\n[query]\n${query_name} = from:gmail.com\n" >> notmuch-config +printf "After:\n" >> OUTPUT +notmuch address --output=sender query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "address: saved query from config file (xdg)" +query_name="test${RANDOM}" +xdg_config +printf "Before:\n" > OUTPUT +notmuch address --deduplicate=no --output=sender query:$query_name 2>&1 | sort >> OUTPUT +printf "\n[query]\n${query_name} = from:gmail.com\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch address --output=sender query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "address: saved query from config file (xdg+profile)" +query_name="test${RANDOM}" +xdg_config $query_name +printf "Before:\n" > OUTPUT +notmuch address --deduplicate=no --output=sender query:$query_name 2>&1 | sort >> OUTPUT +printf "\n[query]\n${query_name} = from:gmail.com\n" >> ${CONFIG_PATH} +printf "After:\n" >> OUTPUT +notmuch address --output=sender query:$query_name | sort >> OUTPUT +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "show with alternate config" +backup_database +cp notmuch-config alt-config +notmuch --config=alt-config config set search.exclude_tags foobar17 +notmuch tag -- +foobar17 '*' +output=$(notmuch --config=alt-config show '*' && echo OK) +restore_database +test_expect_equal "$output" "OK" + +test_begin_subtest "show with alternate config (xdg)" +backup_database +notmuch tag -- +foobar17 '*' +xdg_config +notmuch --config=${CONFIG_PATH} config set search.exclude_tags foobar17 +output=$(notmuch show '*' && echo OK) +restore_database +restore_config +test_expect_equal "$output" "OK" + +test_begin_subtest "show with alternate config (xdg+profile)" +backup_database +notmuch tag -- +foobar17 '*' +xdg_config foobar17 +notmuch --config=${CONFIG_PATH} config set search.exclude_tags foobar17 +output=$(notmuch show '*' && echo OK) +restore_database +restore_config +test_expect_equal "$output" "OK" + +# reset to known state +add_email_corpus + +test_begin_subtest "tag with saved query from config file" +backup_config +query_name="test${RANDOM}" +tag_name="tag${RANDOM}" +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> notmuch-config +notmuch tag +$tag_name -- query:${query_name} +notmuch count tag:$tag_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "tag with saved query from config file (xdg)" +xdg_config +query_name="test${RANDOM}" +tag_name="tag${RANDOM}" +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch tag +$tag_name -- query:${query_name} +notmuch count tag:$tag_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "tag with saved query from config file (xdg+profile)" +query_name="test${RANDOM}" +xdg_config ${query_name} +tag_name="tag${RANDOM}" +notmuch count query:$query_name > OUTPUT +printf "\n[query]\n${query_name} = tag:inbox\n" >> ${CONFIG_PATH} +notmuch tag +$tag_name -- query:${query_name} +notmuch count tag:$tag_name >> OUTPUT +cat < EXPECTED +0 +52 +EOF +restore_config +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "running compact (xdg)" +xdg_config +notmuch compact +output=$(notmuch count '*') +restore_config +test_expect_equal "52" "$output" + +test_begin_subtest "running compact (xdg + profile)" +xdg_config ${RANDOM} +notmuch compact +output=$(notmuch count '*') +restore_config +test_expect_equal "52" "$output" + +test_begin_subtest "run notmuch-new (xdg)" +xdg_config +generate_message +output=$(NOTMUCH_NEW --debug) +restore_config +test_expect_equal "$output" "Added 1 new message to the database." + +test_begin_subtest "run notmuch-new (xdg + profile)" +xdg_config ${RANDOM} +generate_message +output=$(NOTMUCH_NEW --debug) +restore_config +test_expect_equal "$output" "Added 1 new message to the database." + +test_done diff --git a/test/T040-setup.sh b/test/T040-setup.sh index fbfe200a..39846d34 100755 --- a/test/T040-setup.sh +++ b/test/T040-setup.sh @@ -6,11 +6,11 @@ test_description='"notmuch setup"' test_begin_subtest "Notmuch new without a config suggests notmuch setup" output=$(notmuch --config=new-notmuch-config new 2>&1) test_expect_equal "$output" "\ -Configuration file new-notmuch-config not found. +Error: cannot load config file. Try running 'notmuch setup' to create a configuration." test_begin_subtest "Create a new config interactively" -notmuch --config=new-notmuch-config > /dev/null < log.${test_count} < filtered-config +test_expect_equal_file ${expected_dir}/config-with-comments filtered-config + +test_begin_subtest "setup consistent with config-set for single items" +# note this relies on the config state from the previous test. +notmuch --config=new-notmuch-config config list > list.setup +notmuch --config=new-notmuch-config config set search.exclude_tags baz +notmuch --config=new-notmuch-config config list > list.config +test_expect_equal_file list.setup list.config + +test_begin_subtest "notmuch with a config but without a database suggests notmuch new" +notmuch 2>&1 | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +Notmuch is configured, but no database was found. +You probably want to run "notmuch new" now to create a database. + +Note that the first run of "notmuch new" can take a very long time +and that the resulting database will use roughly the same amount of +storage space as the email being indexed. + +EOF +test_expect_equal_file EXPECTED OUTPUT test_done diff --git a/test/T050-new.sh b/test/T050-new.sh index dfc8508f..52888be2 100755 --- a/test/T050-new.sh +++ b/test/T050-new.sh @@ -324,10 +324,22 @@ test_expect_equal "$output" "" OLDCONFIG=$(notmuch config get new.tags) -test_begin_subtest "Empty tags in new.tags are forbidden" +test_begin_subtest "Empty tags in new.tags are ignored" notmuch config set new.tags "foo;;bar" -output=$(NOTMUCH_NEW --debug 2>&1) -test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden" +output=$(NOTMUCH_NEW --quiet 2>&1) +test_expect_equal "$output" "" + +test_begin_subtest "leading/trailing whitespace in new.tags is ignored" +# avoid complications with leading spaces and "notmuch config" +sed -i 's/^tags=.*$/tags= fu bar ; ; bar /' notmuch-config +add_message +NOTMUCH_NEW --quiet +notmuch dump id:$gen_msg_id | sed 's/ --.*$//' > OUTPUT +cat <EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++bar +fu%20bar +EOF +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Tags starting with '-' in new.tags are forbidden" notmuch config set new.tags "-foo;bar" @@ -339,39 +351,80 @@ test_expect_code 1 "NOTMUCH_NEW --debug 2>&1" notmuch config set new.tags $OLDCONFIG +test_begin_subtest ".notmuch only ignored at top level" +generate_message '[dir]=foo/bar/.notmuch/cur' '[subject]="Do not ignore, very important"' +NOTMUCH_NEW > OUTPUT +notmuch search subject:Do-not-ignore | notmuch_search_sanitize >> OUTPUT +cat < EXPECTED +Added 1 new message to the database. +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Do not ignore, very important (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "RFC822 group names are indexed" +test_subtest_known_broken +generate_message [to]="undisclosed-recipients:" +NOTMUCH_NEW > OUTPUT +output=$(notmuch search --output=messages to:undisclosed-recipients) +test_expect_equal "${output}" "${gen_msg_id}" + +test_begin_subtest "Long directory names don't cause rescan" +test_subtest_known_broken +printf -v name 'z%.0s' {1..234} +generate_message [dir]=$name +NOTMUCH_NEW > OUTPUT +notmuch new >> OUTPUT +rm -r ${MAIL_DIR}/${name} +notmuch new >> OUTPUT +cat < EXPECTED +Added 1 new message to the database. +No new mail. +No new mail. Removed 1 message. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Long file names have reasonable diagnostics" +printf -v name 'f%.0s' {1..234} +generate_message "[filename]=$name" +notmuch new 2>&1 | notmuch_dir_sanitize >OUTPUT +rm ${MAIL_DIR}/${name} +cat < EXPECTED +Note: Ignoring non-indexable path: MAIL_DIR/$name +add_file: Path supplied is illegal for this function +filename too long for file-direntry term: MAIL_DIR/$name +Processed 1 file in almost no time. +No new mail. +EOF +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Xapian exception: read only files" -chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.${db_ending} +test_subtest_broken_for_root +chmod u-w ${MAIL_DIR}/.notmuch/xapian/*.* output=$(NOTMUCH_NEW --debug 2>&1 | sed 's/: .*$//' ) -chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.${db_ending} +chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.* test_expect_equal "$output" "A Xapian exception occurred opening database" -test_begin_subtest "Handle files vanishing between scandir and add_file" +make_shim dif-shim< -# A file for scandir to find. It won't get indexed, so can be empty. -touch ${MAIL_DIR}/vanish +WRAP_DLFUNC(notmuch_status_t, notmuch_database_index_file, \ + (notmuch_database_t *database, const char *filename, notmuch_indexopts_t *indexopts, notmuch_message_t **message)) -# Breakpoint to remove the file before indexing -cat < notmuch-new-vanish.gdb -set breakpoint pending on -set logging file notmuch-new-vanish-gdb.log -set logging on -break notmuch_database_index_file -commands -shell rm -f ${MAIL_DIR}/vanish -continue -end -run + if (unlink ("${MAIL_DIR}/vanish")) { + fprintf (stderr, "unlink failed\n"); + exit (42); + } + return notmuch_database_index_file_orig (database, filename, indexopts, message); +} EOF -${TEST_GDB} --batch-silent --return-child-result -x notmuch-new-vanish.gdb \ - --args notmuch new 2>OUTPUT 1>/dev/null -echo "exit status: $?" >> OUTPUT - -# Clean up the file in case gdb isn't available. -rm -f ${MAIL_DIR}/vanish +test_begin_subtest "Handle files vanishing between scandir and add_file" +# A file for scandir to find. It won't get indexed, so can be empty. +touch ${MAIL_DIR}/vanish +notmuch_with_shim dif-shim new 2>OUTPUT 1>/dev/null +echo "exit status: $?" >> OUTPUT cat < EXPECTED Unexpected error with file ${MAIL_DIR}/vanish add_file: Something went wrong trying to read or write a file @@ -380,13 +433,37 @@ exit status: 75 EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Relative database path expanded in new" +ln -s "$PWD/mail" home/Maildir +notmuch config set database.path Maildir +generate_message +NOTMUCH_NEW > OUTPUT +cat <EXPECTED +Added 1 new message to the database. +EOF +notmuch config set database.path ${MAIL_DIR} +rm home/Maildir +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Relative mail root (in db) expanded in new" +ln -s "$PWD/mail" home/Maildir +notmuch config set --database database.mail_root Maildir +generate_message +NOTMUCH_NEW > OUTPUT +cat <EXPECTED +Added 1 new message to the database. +EOF +notmuch config set database.mail_root +rm home/Maildir +test_expect_equal_file EXPECTED OUTPUT + add_email_corpus broken test_begin_subtest "reference loop does not crash" test_expect_code 0 "notmuch show --format=json id:mid-loop-12@example.org id:mid-loop-21@example.org > OUTPUT" test_begin_subtest "reference loop ordered by date" -threadid=$(notmuch search --output=threads id:mid-loop-12@example.org) -notmuch show --format=mbox $threadid | grep '^Date' > OUTPUT +threadid=$(notmuch search --output=threads id:mid-loop-12@example.org) +notmuch show --format=mbox $threadid | grep '^Date' > OUTPUT cat < EXPECTED Date: Thu, 16 Jun 2016 22:14:41 -0400 Date: Fri, 17 Jun 2016 22:14:41 -0400 diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh new file mode 100755 index 00000000..ebd06be1 --- /dev/null +++ b/test/T051-new-renames.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +test_description='"notmuch new" with directory renames' +. $(dirname "$0")/test-lib.sh || exit 1 + +for loop in {1..10}; do + +rm -rf ${MAIL_DIR} + +for i in {1..10}; do + generate_message '[dir]=foo' '[subject]="Message foo $i"' +done + +for i in {1..10}; do + generate_message '[dir]=bar' '[subject]="Message bar $i"' +done + +test_begin_subtest "Index the messages, round $loop" +output=$(NOTMUCH_NEW) +test_expect_equal "$output" "Added 20 new messages to the database." + +all_files=$(notmuch search --output=files \*) +count_foo=$(notmuch count folder:foo) + +test_begin_subtest "Rename folder" +mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz +output=$(NOTMUCH_NEW) +test_expect_equal "$output" "No new mail. Detected $count_foo file renames." + +test_begin_subtest "Rename folder back" +mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo +output=$(NOTMUCH_NEW) +test_expect_equal "$output" "No new mail. Detected $count_foo file renames." + +test_begin_subtest "Files remain the same" +output=$(notmuch search --output=files \*) +test_expect_equal "$output" "$all_files" + +done + +test_done diff --git a/test/T055-path-config.sh b/test/T055-path-config.sh new file mode 100755 index 00000000..1feb5624 --- /dev/null +++ b/test/T055-path-config.sh @@ -0,0 +1,396 @@ +#!/usr/bin/env bash +test_description='Configuration of mail-root and database path' +. $(dirname "$0")/test-lib.sh || exit 1 + +test_require_external_prereq xapian-metdata + +backup_config () { + local test_name=$(basename $0 .sh) + cp ${NOTMUCH_CONFIG} notmuch-config-backup.${test_name} +} + +restore_config () { + local test_name=$(basename $0 .sh) + export NOTMUCH_CONFIG="${TMP_DIRECTORY}/notmuch-config" + unset CONFIG_PATH + unset DATABASE_PATH + unset NOTMUCH_PROFILE + unset XAPIAN_PATH + unset MAILDIR + rm -f "$HOME/mail" + cp notmuch-config-backup.${test_name} ${NOTMUCH_CONFIG} +} + +split_config () { + local dir + backup_config + dir="$TMP_DIRECTORY/database.$test_count" + rm -rf $dir + mkdir $dir + notmuch config set database.path $dir + notmuch config set database.mail_root $MAIL_DIR + DATABASE_PATH=$dir + XAPIAN_PATH="$dir/xapian" +} + +symlink_config () { + local dir + backup_config + dir="$TMP_DIRECTORY/link.$test_count" + ln -s $MAIL_DIR $dir + notmuch config set database.path $dir + notmuch config set database.mail_root $MAIL_DIR + XAPIAN_PATH="$MAIL_DIR/.notmuch/xapian" + unset DATABASE_PATH +} + + +home_mail_config () { + local dir + backup_config + dir="${HOME}/mail" + ln -s $MAIL_DIR $dir + notmuch config set database.path + notmuch config set database.mail_root + XAPIAN_PATH="$MAIL_DIR/.notmuch/xapian" + unset DATABASE_PATH +} + +maildir_env_config () { + local dir + backup_config + dir="${HOME}/env_points_here" + ln -s $MAIL_DIR $dir + export MAILDIR=$dir + notmuch config set database.path + notmuch config set database.mail_root + XAPIAN_PATH="${MAIL_DIR}/.notmuch/xapian" + unset DATABASE_PATH +} + +xdg_config () { + local dir + local profile=${1:-default} + + if [[ $profile != default ]]; then + export NOTMUCH_PROFILE=$profile + fi + + backup_config + DATABASE_PATH="${HOME}/.local/share/notmuch/${profile}" + rm -rf $DATABASE_PATH + mkdir -p $DATABASE_PATH + + config_dir="${HOME}/.config/notmuch/${profile}" + mkdir -p ${config_dir} + CONFIG_PATH=$config_dir/config + mv ${NOTMUCH_CONFIG} $CONFIG_PATH + unset NOTMUCH_CONFIG + + XAPIAN_PATH="${DATABASE_PATH}/xapian" + notmuch --config=${CONFIG_PATH} config set database.mail_root ${TMP_DIRECTORY}/mail + notmuch --config=${CONFIG_PATH} config set database.path +} + +mailroot_only_config () { + local dir + + backup_config + notmuch config set database.mail_root ${TMP_DIRECTORY}/mail + notmuch config set database.path + DATABASE_PATH="${HOME}/.local/share/notmuch/default" + rm -rf $DATABASE_PATH + mkdir -p $DATABASE_PATH + XAPIAN_PATH="${DATABASE_PATH}/xapian" + mv mail/.notmuch/xapian $DATABASE_PATH +} + +for config in traditional split XDG XDG+profile symlink home_mail maildir_env mailroot_only; do + #start each set of tests with an known set of messages + add_email_corpus + + case $config in + traditional) + backup_config + XAPIAN_PATH="$MAIL_DIR/.notmuch/xapian" + ;; + split) + split_config + mv mail/.notmuch/xapian $DATABASE_PATH + ;; + XDG) + xdg_config + mv mail/.notmuch/xapian $DATABASE_PATH + ;; + XDG+profile) + xdg_config ${RANDOM} + mv mail/.notmuch/xapian $DATABASE_PATH + ;; + symlink) + symlink_config + ;; + home_mail) + home_mail_config + ;; + maildir_env) + maildir_env_config + ;; + mailroot_only) + mailroot_only_config + ;; + esac + + test_begin_subtest "count ($config)" + output=$(notmuch count '*') + test_expect_equal "$output" '52' + + test_begin_subtest "count+tag ($config)" + tag="tag${RANDOM}" + notmuch tag +$tag '*' + output=$(notmuch count tag:$tag) + notmuch tag -$tag '*' + test_expect_equal "$output" '52' + + test_begin_subtest "address ($config)" + notmuch address --deduplicate=no --sort=newest-first --output=sender --output=recipients path:foo >OUTPUT + cat <EXPECTED +Carl Worth +notmuch@notmuchmail.org +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "dump ($config)" + notmuch dump is:attachment and is:signed | sort > OUTPUT + cat < EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++attachment +inbox +signed +unread -- id:20091118005829.GB25380@dottiness.seas.harvard.edu ++attachment +inbox +signed +unread -- id:20091118010116.GC25380@dottiness.seas.harvard.edu +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "dump + tag + restore ($config)" + notmuch dump '*' > EXPECTED + notmuch tag -inbox '*' + notmuch restore < EXPECTED + notmuch dump > OUTPUT + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "reindex ($config)" + notmuch search --output=messages '*' > EXPECTED + notmuch reindex '*' + notmuch search --output=messages '*' > OUTPUT + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "use existing database ($config)" + output=$(notmuch new) + test_expect_equal "$output" 'No new mail.' + + test_begin_subtest "create database ($config)" + rm -rf $DATABASE_PATH/{.notmuch,}/xapian + notmuch new + output=$(notmuch count '*') + test_expect_equal "$output" '52' + + test_begin_subtest "detect new files ($config)" + generate_message + generate_message + notmuch new + output=$(notmuch count '*') + test_expect_equal "$output" '54' + + test_begin_subtest "Show a raw message ($config)" + add_message + notmuch show --format=raw id:$gen_msg_id > OUTPUT + test_expect_equal_file_nonempty $gen_msg_filename OUTPUT + rm -f $gen_msg_filename + + test_begin_subtest "reply ($config)" + add_message '[from]="Sender "' \ + [to]=test_suite@notmuchmail.org \ + [subject]=notmuch-reply-test \ + '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ + '[body]="basic reply test"' + notmuch reply id:${gen_msg_id} 2>&1 > OUTPUT + cat < EXPECTED +From: Notmuch Test Suite +Subject: Re: notmuch-reply-test +To: Sender +In-Reply-To: <${gen_msg_id}> +References: <${gen_msg_id}> + +On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: +> basic reply test +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "insert+search ($config)" + generate_message \ + "[subject]=\"insert-subject\"" \ + "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \ + "[body]=\"insert-message\"" + mkdir -p "$MAIL_DIR"/{cur,new,tmp} + notmuch insert < "$gen_msg_filename" + cur_msg_filename=$(notmuch search --output=files "subject:insert-subject") + test_expect_equal_file_nonempty "$cur_msg_filename" "$gen_msg_filename" + + test_begin_subtest "compact+search ($config)" + notmuch search --output=messages '*' | sort > EXPECTED + notmuch compact + notmuch search --output=messages '*' | sort > OUTPUT + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "upgrade backup ($config)" + features=$(xapian-metadata get $XAPIAN_PATH features | grep -v "^relative directory paths") + xapian-metadata set $XAPIAN_PATH features "$features" + output=$(notmuch new | grep Welcome) + test_expect_equal \ + "$output" \ + "Welcome to a new version of notmuch! Your database will now be upgraded." + + test_begin_subtest "notmuch +config -database suggests notmuch new ($config)" + mv "$XAPIAN_PATH" "${XAPIAN_PATH}.bak" + notmuch > OUTPUT +cat < EXPECTED +Notmuch is configured, but no database was found. +You probably want to run "notmuch new" now to create a database. + +Note that the first run of "notmuch new" can take a very long time +and that the resulting database will use roughly the same amount of +storage space as the email being indexed. + +EOF + mv "${XAPIAN_PATH}.bak" "$XAPIAN_PATH" + + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "Set config value ($config)" + name=${RANDOM} + value=${RANDOM} + notmuch config set test${test_count}.${name} ${value} + output=$(notmuch config get test${test_count}.${name}) + notmuch config set test${test_count}.${name} + output2=$(notmuch config get test${test_count}.${name}) + test_expect_equal "${output}+${output2}" "${value}+" + + test_begin_subtest "Set config value in database ($config)" + name=${RANDOM} + value=${RANDOM} + notmuch config set --database test${test_count}.${name} ${value} + output=$(notmuch config get test${test_count}.${name}) + notmuch config set --database test${test_count}.${name} + output2=$(notmuch config get test${test_count}.${name}) + test_expect_equal "${output}+${output2}" "${value}+" + + test_begin_subtest "Config list ($config)" + notmuch config list | notmuch_config_sanitize | \ + sed -e "s/^database.backup_dir=.*$/database.backup_dir/" \ + -e "s/^database.hook_dir=.*$/database.hook_dir/" \ + -e "s/^database.path=.*$/database.path/" \ + -e "s,^database.mail_root=CWD/home/mail,database.mail_root=MAIL_DIR," \ + -e "s,^database.mail_root=CWD/home/env_points_here,database.mail_root=MAIL_DIR," \ + > OUTPUT + cat < EXPECTED +built_with.compact=something +built_with.field_processor=something +built_with.retry_lock=something +built_with.sexp_queries=something +database.autocommit=8000 +database.backup_dir +database.hook_dir +database.mail_root=MAIL_DIR +database.path +index.as_text= +maildir.synchronize_flags=true +new.ignore= +new.tags=unread;inbox +search.exclude_tags= +user.name=Notmuch Test Suite +user.other_email=test_suite_other@notmuchmail.org;test_suite@otherdomain.org +user.primary_email=test_suite@notmuchmail.org +EOF + test_expect_equal_file EXPECTED OUTPUT + + test_begin_subtest "Config list from python ($config)" + test_python < OUTPUT +from notmuch2 import Database +db=Database(config=Database.CONFIG.SEARCH) +for key in list(db.config): + print(key) +EOF + cat < EXPECTED +database.autocommit +database.backup_dir +database.hook_dir +database.mail_root +database.path +maildir.synchronize_flags +new.tags +user.name +user.other_email +user.primary_email +EOF + test_expect_equal_file EXPECTED OUTPUT + + case $config in + XDG*) + test_begin_subtest "Set shadowed config value in database ($config)" + name=${RANDOM} + value=${RANDOM} + key=test${test_count}.${name} + notmuch config set --database ${key} ${value} + notmuch config set ${key} shadow${value} + output=$(notmuch --config='' config get ${key}) + notmuch config set --database ${key} + output2=$(notmuch --config='' config get ${key}) + notmuch config set ${key} + test_expect_equal "${output}+${output2}" "${value}+" + ;& + split) + test_begin_subtest "'to' header does not crash (python-cffi) ($config)" + echo 'notmuch@notmuchmail.org' > EXPECTED + test_python < OUTPUT + notmuch search subject:Do-not-ignore | notmuch_search_sanitize >> OUTPUT + cat < EXPECTED +Added 1 new message to the database. +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Do not ignore, very important (inbox unread) +EOF + test_expect_equal_file EXPECTED OUTPUT + ;& + mailroot_only) + test_begin_subtest "create database parent dir ($config)" + rm -r ${DATABASE_PATH} + notmuch new + test_expect_equal "$(xapian-metadata get ${XAPIAN_PATH} version)" 3 + ;; + home_mail|maildir_env) + test_begin_subtest "No errors from config list ($config)" + notmuch config list 2>OUTPUT 1>/dev/null + test_expect_equal_file /dev/null OUTPUT + ;; + *) + backup_database + test_begin_subtest ".notmuch without xapian/ handled gracefully ($config)" + rm -r $XAPIAN_PATH + test_expect_success "notmuch new" + restore_database + ;; + esac + + restore_config + rm -rf home/.local + rm -rf home/.config +done + +test_done diff --git a/test/T060-count.sh b/test/T060-count.sh index 0c0bf473..6e855b59 100755 --- a/test/T060-count.sh +++ b/test/T060-count.sh @@ -95,28 +95,32 @@ test_expect_equal_file EXPECTED OUTPUT backup_database test_begin_subtest "error message for database open" -dd if=/dev/zero of="${MAIL_DIR}/.notmuch/xapian/postlist.${db_ending}" count=3 +target=(${MAIL_DIR}/.notmuch/xapian/postlist.*) +dd if=/dev/zero of="$target" count=3 notmuch count '*' 2>OUTPUT 1>/dev/null output=$(sed 's/^\(A Xapian exception [^:]*\):.*$/\1/' OUTPUT) test_expect_equal "${output}" "A Xapian exception occurred opening database" restore_database -cat < count-files.gdb -set breakpoint pending on -set logging file count-files-gdb.log -set logging on -break count_files -commands -shell cp /dev/null ${MAIL_DIR}/.notmuch/xapian/postlist.${db_ending} -continue -end -run +make_shim qsm-shim< + +WRAP_DLFUNC (notmuch_status_t, notmuch_query_search_messages, (notmuch_query_t *query, notmuch_messages_t **messages)) + + /* XXX WARNING THIS CORRUPTS THE DATABASE */ + int fd = open ("target_postlist", O_WRONLY|O_TRUNC); + if (fd < 0) + exit (8); + close (fd); + + return notmuch_query_search_messages_orig(query, messages); +} EOF backup_database test_begin_subtest "error message from query_search_messages" -${TEST_GDB} --batch-silent --return-child-result -x count-files.gdb \ - --args notmuch count --output=files '*' 2>OUTPUT 1>/dev/null +ln -s ${MAIL_DIR}/.notmuch/xapian/postlist.* target_postlist +notmuch_with_shim qsm-shim count --output=files '*' 2>OUTPUT 1>/dev/null cat < EXPECTED notmuch count: A Xapian exception occurred A Xapian exception occurred performing query @@ -153,4 +157,28 @@ print("4: {} messages".format(query.count_messages())) EOF test_expect_equal_file EXPECTED OUTPUT +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + test_begin_subtest "and of exact terms (query=sexp)" + output=$(notmuch count --query=sexp '(and "wonderful" "wizard")') + test_expect_equal "$output" 1 + + test_begin_subtest "or of exact terms (query=sexp)" + output=$(notmuch count --query=sexp '(or "php" "wizard")') + test_expect_equal "$output" 2 + + test_begin_subtest "starts-with, case-insensitive (query=sexp)" + output=$(notmuch count --query=sexp '(starts-with FreeB)') + test_expect_equal "$output" 5 + + test_begin_subtest "query that matches no messages (query=sexp)" + count=$(notmuch count --query=sexp '(and (from keithp) (to keithp))') + test_expect_equal 0 "$count" + + test_begin_subtest "Compound subquery (query=sexp)" + output=$(notmuch count --query=sexp '(thread (of (from keithp) (subject Maildir)))') + test_expect_equal "$output" 7 + +fi + test_done diff --git a/test/T070-insert.sh b/test/T070-insert.sh index c8161e1e..73953272 100755 --- a/test/T070-insert.sh +++ b/test/T070-insert.sh @@ -15,7 +15,7 @@ notmuch new > /dev/null # They happen to be in the mail directory already but that is okay # since we do not call notmuch new hereafter. -gen_insert_msg() { +gen_insert_msg () { generate_message \ "[subject]=\"insert-subject\"" \ "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" \ @@ -43,13 +43,13 @@ test_begin_subtest "Permissions on inserted message should be 0600" test_expect_equal "600" "$(stat -c %a "$cur_msg_filename")" test_begin_subtest "Insert message adds default tags" -output=$(notmuch show --format=json "subject:insert-subject") +output=$(notmuch show --format=json "subject:insert-subject" | notmuch_json_show_sanitize) expected='[[[{ - "id": "'"${gen_msg_id}"'", + "id": "XXXXX", "crypto": {}, "match": true, "excluded": false, - "filename": ["'"${cur_msg_filename}"'"], + "filename": ["YYYYY"], "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["inbox","unread"], @@ -222,30 +222,44 @@ test_expect_equal "$output" "2" test_begin_subtest "Insert message, create invalid subfolder" gen_insert_msg -test_expect_code 1 "notmuch insert --folder=../G --create-folder $gen_msg_filename" +test_expect_code 1 "notmuch insert --folder=../G --create-folder < $gen_msg_filename" OLDCONFIG=$(notmuch config get new.tags) -test_begin_subtest "Empty tags in new.tags are forbidden" +test_begin_subtest "Empty tags in new.tags are ignored" notmuch config set new.tags "foo;;bar" gen_insert_msg -output=$(notmuch insert $gen_msg_filename 2>&1) -test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden" +notmuch insert < $gen_msg_filename +output=$(notmuch show --format=json id:$gen_msg_id) +test_json_nodes <<<"$output" \ + 'new_tags:[0][0][0]["tags"] = ["bar", "foo"]' + +test_begin_subtest "leading/trailing whitespace in new.tags is ignored" +# avoid complications with leading spaces and "notmuch config" +sed -i 's/^tags=.*$/tags= fu bar ; ; bar /' notmuch-config +gen_insert_msg +notmuch insert < $gen_msg_filename +notmuch dump id:$gen_msg_id | sed 's/ --.*$//' > OUTPUT +cat <EXPECTED +#notmuch-dump batch-tag:3 config,properties,tags ++bar +fu%20bar +EOF +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Tags starting with '-' in new.tags are forbidden" notmuch config set new.tags "-foo;bar" gen_insert_msg -output=$(notmuch insert $gen_msg_filename 2>&1) +output=$(notmuch insert < $gen_msg_filename 2>&1) test_expect_equal "$output" "Error: tag '-foo' in new.tags: tag starting with '-' forbidden" test_begin_subtest "Invalid tags set exit code" -test_expect_code 1 "notmuch insert $gen_msg_filename 2>&1" +test_expect_code 1 "notmuch insert < $gen_msg_filename 2>&1" notmuch config set new.tags $OLDCONFIG # DUPLICATE_MESSAGE_ID is not tested here, because it should actually pass. # pregenerate all of the test shims -for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR OUT_OF_MEMORY XAPIAN_EXCEPTION; do +for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR OUT_OF_MEMORY XAPIAN_EXCEPTION; do make_shim shim-$code < #include @@ -262,7 +276,7 @@ done gen_insert_msg -for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do +for code in FILE_NOT_EMAIL READ_ONLY_DATABASE UPGRADE_REQUIRED PATH_ERROR; do test_begin_subtest "EXIT_FAILURE when index_file returns $code" test_expect_code 1 "notmuch_with_shim shim-$code insert < \"$gen_msg_filename\"" @@ -278,4 +292,9 @@ for code in OUT_OF_MEMORY XAPIAN_EXCEPTION ; do test_expect_code 0 "notmuch_with_shim shim-$code insert --keep < \"$gen_msg_filename\"" done +test_begin_subtest "insert converts mboxes on delivery" +notmuch insert +unmboxed < "${TEST_DIRECTORY}"/corpora/insert/mbox-attachment.eml +output=$(notmuch count tag:unmboxed) +test_expect_equal "${output}" 1 + test_done diff --git a/test/T080-search.sh b/test/T080-search.sh index a3f0dead..515eb88f 100755 --- a/test/T080-search.sh +++ b/test/T080-search.sh @@ -34,6 +34,12 @@ add_message '[subject]="search by id"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000" output=$(notmuch search id:${gen_msg_id} | notmuch_search_sanitize) test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread)" +test_begin_subtest "Message-Id with spaces" +test_subtest_known_broken +add_message '[id]="message id@example.com"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --output=messages id:"message id@example.com") +test_expect_equal "$output" "messageid@example.com" + test_begin_subtest "Search by mid:" add_message '[subject]="search by mid"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' output=$(notmuch search mid:${gen_msg_id} | notmuch_search_sanitize) @@ -132,6 +138,7 @@ thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; Message-Id with spaces (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by mid (inbox unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread) thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by thread (inbox unread) diff --git a/test/T081-sexpr-search.sh b/test/T081-sexpr-search.sh new file mode 100755 index 00000000..8800b545 --- /dev/null +++ b/test/T081-sexpr-search.sh @@ -0,0 +1,1322 @@ +#!/usr/bin/env bash +test_description='"notmuch search" in several variations' +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" != "1" ]; then + printf "Skipping due to missing sfsexp library\n" + test_done +fi + +add_email_corpus + +for query in '()' '(not)' '(and)' '(or ())' '(or (not))' '(or (and))' \ + '(or (and) (or) (not (and)))'; do + test_begin_subtest "all messages: $query" + notmuch search '*' > EXPECTED + notmuch search --query=sexp "$query" > OUTPUT + test_expect_equal_file EXPECTED OUTPUT +done + +for query in '(or)' '(not ())' '(not (not))' '(not (and))' \ + '(not (or (and) (or) (not (and))))'; do + test_begin_subtest "no messages: $query" + notmuch search --query=sexp "$query" > OUTPUT + test_expect_equal_file /dev/null OUTPUT +done + +test_begin_subtest "and of exact terms" +notmuch search --query=sexp '(and "wonderful" "wizard")' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "and of stemmed terms" +notmuch search --query=sexp '(and wonderful wizard)' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "or of exact terms" +notmuch search --query=sexp '(or "php" "wizard")' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2010-12-29 [1/1] François Boulogne; [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "or of exact terms via field processor" +notmuch search 'sexp:"(or ""php"" ""wizard"")"' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2010-12-29 [1/1] François Boulogne; [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "single term in body" +notmuch search --query=sexp 'wizard' | notmuch_search_sanitize>OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "single term in body (case insensitive)" +notmuch search --query=sexp 'Wizard' | notmuch_search_sanitize>OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [1/3] Carl Worth| Jan Janak; [notmuch] What a great idea! (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "single term in body, stemmed version" +notmuch search arriv > EXPECTED +notmuch search --query=sexp arriv > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "single term in body, unstemmed version" +notmuch search --query=sexp '"arriv"' > OUTPUT +test_expect_equal_file /dev/null OUTPUT + +test_begin_subtest "Search by 'subject'" +add_message [subject]=subjectsearchtest '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp '(subject subjectsearchtest)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread)" + +test_begin_subtest "Search by 'subject' (case insensitive)" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(subject "Maildir")' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'subject' (utf-8):" +add_message [subject]=utf8-sübjéct '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp '(subject utf8 sübjéct)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)" + +test_begin_subtest "Search by 'subject' (utf-8, and):" +output=$(notmuch search --query=sexp '(subject (and utf8 sübjéct))' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)" + +test_begin_subtest "Search by 'subject' (utf-8, and outside):" +output=$(notmuch search --query=sexp '(and (subject utf8) (subject sübjéct))' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)" + +test_begin_subtest "Search by 'subject' (utf-8, or):" +notmuch search --query=sexp '(subject (or utf8 subjectsearchtest))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'subject' (utf-8, or outside):" +notmuch search --query=sexp '(or (subject utf8) (subject subjectsearchtest))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; subjectsearchtest (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'attachment'" +notmuch search attachment:notmuch-help.patch > EXPECTED +notmuch search --query=sexp '(attachment notmuch-help.patch)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'body'" +add_message '[subject]="body search"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [body]=bodysearchtest +output=$(notmuch search --query=sexp '(body bodysearchtest)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (inbox unread)" + +test_begin_subtest "Search by 'body' (phrase)" +add_message '[subject]="body search (phrase)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="body search (phrase)"' +add_message '[subject]="negative result"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="This phrase should not match the body search"' +output=$(notmuch search --query=sexp '(body "body search phrase")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; body search (phrase) (inbox unread)" + +test_begin_subtest "Search by 'body' (utf-8):" +add_message '[subject]="utf8-message-body-subject"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[body]="message body utf8: bödý"' +output=$(notmuch search --query=sexp '(body bödý)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-message-body-subject (inbox unread)" + +add_message "[body]=thebody-1" "[subject]=kryptonite-1" +add_message "[body]=nothing-to-see-here-1" "[subject]=thebody-1" + +test_begin_subtest 'search without body: prefix' +notmuch search thebody > EXPECTED +notmuch search --query=sexp '(and thebody)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest 'negated body: prefix' +notmuch search thebody and not body:thebody > EXPECTED +notmuch search --query=sexp '(and (not (body thebody)) thebody)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest 'search unprefixed for prefixed term' +notmuch search kryptonite > EXPECTED +notmuch search --query=sexp '(and kryptonite)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest 'search with body: prefix for term only in subject' +notmuch search body:kryptonite > EXPECTED +notmuch search --query=sexp '(body kryptonite)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'from'" +add_message '[subject]="search by from"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [from]=searchbyfrom +output=$(notmuch search --query=sexp '(from searchbyfrom)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] searchbyfrom; search by from (inbox unread)" + +test_begin_subtest "Search by 'from' (address)" +add_message '[subject]="search by from (address)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [from]=searchbyfrom@example.com +output=$(notmuch search --query=sexp '(from searchbyfrom@example.com)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] searchbyfrom@example.com; search by from (address) (inbox unread)" + +test_begin_subtest "Search by 'from' (name)" +add_message '[subject]="search by from (name)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[from]="Search By From Name "' +output=$(notmuch search --query=sexp '(from "Search By From Name")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)" + +test_begin_subtest "Search by 'from' (name and address)" +output=$(notmuch search --query=sexp '(from "Search By From Name ")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Search By From Name; search by from (name) (inbox unread)" + +add_message '[dir]=bad' '[subject]="To the bone"' +add_message '[dir]=.' '[subject]="Top level"' +add_message '[dir]=bad/news' '[subject]="Bears"' +mkdir -p "${MAIL_DIR}/duplicate/bad/news" +cp "$gen_msg_filename" "${MAIL_DIR}/duplicate/bad/news" + +add_message '[dir]=things' '[subject]="These are a few"' +add_message '[dir]=things/favorite' '[subject]="Raindrops, whiskers, kettles"' +add_message '[dir]=things/bad' '[subject]="Bites, stings, sad feelings"' + +test_begin_subtest "Search by 'folder' (multiple)" +output=$(notmuch search --query=sexp '(folder bad bad/news things/bad)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + +test_begin_subtest "Search by 'folder': top level." +notmuch search folder:'""' > EXPECTED +notmuch search --query=sexp '(folder "")' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'folder' with --output=files" +output=$(notmuch search --output=files --query=sexp '(folder bad/news)' | notmuch_search_files_sanitize) +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" + +test_begin_subtest "Search by 'folder' with --output=files (trailing /)" +output=$(notmuch search --output=files --query=sexp '(folder bad/news/)' | notmuch_search_files_sanitize) +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" + +test_begin_subtest "Search by 'folder' (multiple)" +output=$(notmuch search --query=sexp '(folder bad bad/news things/bad)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + +test_begin_subtest "Search by 'folder' (multiple, trailing /)" +output=$(notmuch search --query=sexp '(folder bad bad/news/ things/bad)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + +test_begin_subtest "Search by 'path' with --output=files" +output=$(notmuch search --output=files --query=sexp '(path bad/news)' | notmuch_search_files_sanitize) +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" + +test_begin_subtest "Search by 'path' with --output=files (trailing /)" +output=$(notmuch search --output=files --query=sexp '(path bad/news/)' | notmuch_search_files_sanitize) +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" + +test_begin_subtest "Search by 'path' specification (multiple)" +output=$(notmuch search --query=sexp '(path bad bad/news things/bad)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + +test_begin_subtest "Search by 'path' specification (multiple, trailing /)" +output=$(notmuch search --query=sexp '(path bad bad/news/ things/bad)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + +test_begin_subtest "Search by 'id'" +add_message '[subject]="search by id"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp "(id ${gen_msg_id})" | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread)" + +test_begin_subtest "Search by 'id' (or)" +add_message '[subject]="search by id"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp "(id non-existent-mid ${gen_msg_id})" | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by id (inbox unread)" + +test_begin_subtest "Search by 'is' (multiple)" +notmuch tag -inbox tag:searchbytag +notmuch search is:inbox AND is:unread | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(is inbox unread)' | notmuch_search_sanitize > OUTPUT +notmuch tag +inbox tag:searchbytag +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'mid'" +add_message '[subject]="search by mid"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp "(mid ${gen_msg_id})" | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by mid (inbox unread)" + +test_begin_subtest "Search by 'mid' (or)" +add_message '[subject]="search by mid"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +output=$(notmuch search --query=sexp "(mid non-existent-mid ${gen_msg_id})" | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by mid (inbox unread)" + +test_begin_subtest "Search by 'mimetype'" +notmuch search mimetype:text/html > EXPECTED +notmuch search --query=sexp '(mimetype text html)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +QUERYSTR="date:2009-11-18..2009-11-18 and tag:unread" +QUERYSTR2="query:test and subject:Maildir" +notmuch config set --database query.test "$QUERYSTR" +notmuch config set query.test2 "$QUERYSTR2" + +test_begin_subtest "ill-formed named query search" +notmuch search --query=sexp '(query)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'query' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "ill-formed named query search 2" +notmuch search --query=sexp '(to (query))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'query' not supported inside 'to' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search named query" +notmuch search --query=sexp '(query test)' > OUTPUT +notmuch search $QUERYSTR > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'subject' (utf-8, phrase-token):" +output=$(notmuch search --query=sexp '(subject utf8-sübjéct)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)" + +test_begin_subtest "search named query with other terms" +notmuch search --query=sexp '(and (query test) (subject Maildir))' > OUTPUT +notmuch search $QUERYSTR and subject:Maildir > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search nested named query" +notmuch search --query=sexp '(query test2)' > OUTPUT +notmuch search $QUERYSTR2 > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'subject' (utf-8, quoted string):" +output=$(notmuch search --query=sexp '(subject "utf8 sübjéct")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread)" + +test_begin_subtest "Search by 'subject' (combine phrase, term):" +output=$(notmuch search --query=sexp '(subject Mac "compatibility issues")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" + +test_begin_subtest "Search by 'subject' (combine phrase, term 2):" +notmuch search --query=sexp '(subject (or utf8 "compatibility issues"))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-sübjéct (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; utf8-message-body-subject (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'subject' (combine phrase, term 3):" +notmuch search --query=sexp '(subject issues X/Darwin)' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'tag'" +add_message '[subject]="search by tag"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +notmuch tag +searchbytag id:${gen_msg_id} +output=$(notmuch search --query=sexp '(tag searchbytag)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread)" + +test_begin_subtest "Search by 'tag' (multiple)" +notmuch tag -inbox tag:searchbytag +notmuch search tag:inbox AND tag:unread | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(tag inbox unread)' | notmuch_search_sanitize > OUTPUT +notmuch tag +inbox tag:searchbytag +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'tag' and 'subject'" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (tag inbox) (subject maildir))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search by 'thread'" +add_message '[subject]="search by thread"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' +thread_id=$(notmuch search id:${gen_msg_id} | sed -e "s/thread:\([a-f0-9]*\).*/\1/") +output=$(notmuch search --query=sexp "(thread ${thread_id})" | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by thread (inbox unread)" + +test_begin_subtest "Search by 'to'" +add_message '[subject]="search by to"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [to]=searchbyto +output=$(notmuch search --query=sexp '(to searchbyto)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread)" + +test_begin_subtest "Search by 'to' (address)" +add_message '[subject]="search by to (address)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' [to]=searchbyto@example.com +output=$(notmuch search --query=sexp '(to searchbyto@example.com)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread)" + +test_begin_subtest "Search by 'to' (name)" +add_message '[subject]="search by to (name)"' '[date]="Sat, 01 Jan 2000 12:00:00 -0000"' '[to]="Search By To Name "' +output=$(notmuch search --query=sexp '(to "Search By To Name")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)" + +test_begin_subtest "Search by 'to' (name and address)" +output=$(notmuch search --query=sexp '(to "Search By To Name ")' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread)" + +test_begin_subtest "starts-with, no prefix" +output=$(notmuch search --query=sexp '(starts-with prelim)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-17 [2/2] Alex Botero-Lowry, Carl Worth; [notmuch] preliminary FreeBSD support (attachment inbox unread)" + +test_begin_subtest "starts-with, case-insensitive" +notmuch search --query=sexp '(starts-with FreeB)' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [3/4] Alexander Botero-Lowry, Jjgod Jiang; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) +thread:XXX 2009-11-17 [2/2] Alex Botero-Lowry, Carl Worth; [notmuch] preliminary FreeBSD support (attachment inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, no prefix, all messages" +notmuch search --query=sexp '(starts-with "")' | notmuch_search_sanitize > OUTPUT +notmuch search '*' | notmuch_search_sanitize > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, attachment" +output=$(notmuch search --query=sexp '(attachment (starts-with not))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2009-11-18 [2/2] Lars Kellogg-Stedman; [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread)' + +test_begin_subtest "starts-with, folder" +notmuch search --output=files --query=sexp '(folder (starts-with bad))' | notmuch_search_files_sanitize > OUTPUT +cat < EXPECTED +MAIL_DIR/bad/msg-XXX +MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, from" +notmuch search --query=sexp '(from (starts-with Mik))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-17 [1/1] Mikhail Gusarov; [notmuch] [PATCH] Handle rename of message file (inbox unread) +thread:XXX 2009-11-17 [2/7] Mikhail Gusarov| Lars Kellogg-Stedman, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +thread:XXX 2009-11-17 [2/5] Mikhail Gusarov| Carl Worth, Keith Packard; [notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, id" +notmuch search --query=sexp --output=messages '(id (starts-with 877))' > OUTPUT +cat < EXPECTED +id:877h1wv7mg.fsf@inf-8657.int-evry.fr +id:877htoqdbo.fsf@yoom.home.cworth.org +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, is" +output=$(notmuch search --query=sexp '(is (starts-with searchby))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread)' + +test_begin_subtest "starts-with, mid" +notmuch search --query=sexp --output=messages '(mid (starts-with 877))' > OUTPUT +cat < EXPECTED +id:877h1wv7mg.fsf@inf-8657.int-evry.fr +id:877htoqdbo.fsf@yoom.home.cworth.org +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, mimetype" +notmuch search --query=sexp '(mimetype (starts-with sig))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-18 [2/2] Lars Kellogg-Stedman; [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) +thread:XXX 2009-11-18 [4/7] Lars Kellogg-Stedman, Mikhail Gusarov| Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +thread:XXX 2009-11-17 [1/3] Adrian Perez de Castro| Keith Packard, Carl Worth; [notmuch] Introducing myself (inbox signed unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +add_message '[subject]="message with properties"' +notmuch restore < OUTPUT +cat < EXPECTED +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; message with properties (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, subject" +notmuch search --query=sexp '(subject (starts-with FreeB))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2009-11-17 [2/2] Alex Botero-Lowry, Carl Worth; [notmuch] preliminary FreeBSD support (attachment inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, tag" +output=$(notmuch search --query=sexp '(tag (starts-with searchby))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by tag (inbox searchbytag unread)' + +add_message '[subject]="no tags"' +notag_mid=${gen_msg_id} +notmuch tag -unread -inbox id:${notag_mid} + +test_begin_subtest "negated starts-with, tag" +output=$(notmuch search --query=sexp '(tag (not (starts-with in)))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; no tags ()' + +test_begin_subtest "negated starts-with, tag 2" +output=$(notmuch search --query=sexp '(not (tag (starts-with in)))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; no tags ()' + +test_begin_subtest "negated starts-with, tag 3" +output=$(notmuch search --query=sexp '(not (tag (starts-with "")))' | notmuch_search_sanitize) +test_expect_equal "$output" 'thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; no tags ()' + +test_begin_subtest "starts-with, thread" +notmuch search --query=sexp '(thread (starts-with "00"))' > OUTPUT +notmuch search '*' > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, to" +notmuch search --query=sexp '(to (starts-with "search"))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (address) (inbox unread) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; search by to (name) (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for 'is'" +notmuch search not id:${notag_mid} > EXPECTED +notmuch search --query=sexp '(is *)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "negated wildcard search for 'is'" +notmuch search id:${notag_mid} > EXPECTED +notmuch search --query=sexp '(not (is *))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for 'property'" +notmuch search property:foo=bar > EXPECTED +notmuch search --query=sexp '(property *)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for 'tag'" +notmuch search not id:${notag_mid} > EXPECTED +notmuch search --query=sexp '(tag *)' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "negated wildcard search for 'tag'" +notmuch search id:${notag_mid} > EXPECTED +notmuch search --query=sexp '(not (tag *))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +add_message '[subject]="message with tag \"*\""' +notmuch tag '+*' id:${gen_msg_id} + +test_begin_subtest "search for 'tag' \"*\"" +output=$(notmuch search --query=sexp --output=messages '(tag "*")') +test_expect_equal "$output" "id:$gen_msg_id" + +test_begin_subtest "search for missing / empty to" +add_message [to]="undisclosed-recipients:" +notmuch search --query=sexp '(not (to *))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; search for missing / empty to (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Unbalanced parens" +# A code 1 indicates the error was handled (a crash will return e.g. 139). +test_expect_code 1 "notmuch search --query=sexp '('" + +test_begin_subtest "Unbalanced parens, error message" +notmuch search --query=sexp '(' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +invalid s-expression: '(' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "unknown prefix" +notmuch search --query=sexp '(foo)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +unknown prefix 'foo' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "list as prefix" +notmuch search --query=sexp '((foo))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +unexpected list in field/operation position +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "illegal nesting" +notmuch search --query=sexp '(subject (subject foo))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'subject' inside 'subject' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, no argument" +notmuch search --query=sexp '(starts-with)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'starts-with' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, list argument" +notmuch search --query=sexp '(starts-with (stuff))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'starts-with' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, too many arguments" +notmuch search --query=sexp '(starts-with a b c)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'starts-with' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "starts-with, illegal field" +notmuch search --query=sexp '(body (starts-with foo))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'body' does not support wildcard queries +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard, illegal field" +notmuch search --query=sexp '(body *)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'body' does not support wildcard queries +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search, exclude \"deleted\" messages from search" +notmuch config set search.exclude_tags deleted +generate_message '[subject]="Not deleted"' +not_deleted_id=$gen_msg_id +generate_message '[subject]="Deleted"' +notmuch new > /dev/null +notmuch tag +deleted id:$gen_msg_id +deleted_id=$gen_msg_id +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)" + +test_begin_subtest "Search, exclude \"deleted\" messages from message search --exclude=false" +output=$(notmuch search --query=sexp --exclude=false --output=messages '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "id:$not_deleted_id +id:$deleted_id" + +test_begin_subtest "Search, exclude \"deleted\" messages from search, overridden" +notmuch search --query=sexp '(and (subject deleted) (tag deleted))' | notmuch_search_sanitize > OUTPUT +cat < EXPECTED +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Deleted (deleted inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Search, exclude \"deleted\" messages from threads" +add_message '[subject]="Not deleted reply"' '[in-reply-to]="<$gen_msg_id>"' +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [1/2] Notmuch Test Suite; Not deleted reply (deleted inbox unread)" + +test_begin_subtest "Search, don't exclude \"deleted\" messages when --exclude=flag specified" +output=$(notmuch search --query=sexp --exclude=flag '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [1/2] Notmuch Test Suite; Deleted (deleted inbox unread)" + +test_begin_subtest "Search, don't exclude \"deleted\" messages from search if not configured" +notmuch config set search.exclude_tags +output=$(notmuch search --query=sexp '(subject deleted)' | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +thread:XXX 2001-01-05 [2/2] Notmuch Test Suite; Deleted (deleted inbox unread)" + +test_begin_subtest "regex at top level" +notmuch search --query=sexp '(rx foo)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +illegal 'rx' outside field +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regex in illegal field" +notmuch search --query=sexp '(body (regex foo))' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +'regex' not supported in field 'body' +EOF +test_expect_equal_file EXPECTED OUTPUT + +notmuch search --output=messages from:cworth > cworth.msg-ids + +test_begin_subtest "regexp 'from' search" +notmuch search --output=messages --query=sexp '(from (rx cworth))' > OUTPUT +test_expect_equal_file cworth.msg-ids OUTPUT + +test_begin_subtest "regexp search for 'from' 2" +notmuch search from:/cworth@cworth.org/ and subject:patch | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (from (rx cworth@cworth.org)) (subject patch))' \ + | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'folder' search" +notmuch search 'folder:/^bar$/' | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(folder (rx ^bar$))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'id' search" +notmuch search --output=messages --query=sexp '(id (rx yoom))' > OUTPUT +test_expect_equal_file cworth.msg-ids OUTPUT + +test_begin_subtest "unanchored 'is' search" +notmuch search tag:signed or tag:inbox > EXPECTED +notmuch search --query=sexp '(is (rx i))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "anchored 'is' search" +notmuch search tag:signed > EXPECTED +notmuch search --query=sexp '(is (rx ^si))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "combine regexp mid and subject" +notmuch search subject:/-C/ and mid:/y..m/ | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (subject (rx -C)) (mid (rx y..m)))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'path' search" +notmuch search 'path:/^bar$/' | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(path (rx ^bar$))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'property' search" +notmuch search property:foo=bar > EXPECTED +notmuch search --query=sexp '(property (rx foo=.*))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'property' search via field processor" +notmuch search property:foo=bar > EXPECTED +notmuch search 'sexp:"(property (rx foo=.*))"' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "anchored 'tag' search" +notmuch search tag:signed > EXPECTED +notmuch search --query=sexp '(tag (rx ^si))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "regexp 'thread' search" +notmuch search --output=threads '*' | grep '7$' > EXPECTED +notmuch search --output=threads --query=sexp '(thread (rx 7$))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Basic query that matches no messages" +count=$(notmuch count from:keithp and to:keithp) +test_expect_equal 0 "$count" + +test_begin_subtest "Same query against threads" +notmuch search --query=sexp '(and (thread (of (from keithp))) (thread (matching (to keithp))))' \ + | notmuch_search_sanitize > OUTPUT +cat< EXPECTED +thread:XXX 2009-11-18 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Mix thread and non-threads query" +notmuch search --query=sexp '(and (thread (matching keithp)) (to keithp))' | notmuch_search_sanitize > OUTPUT +cat< EXPECTED +thread:XXX 2009-11-18 [1/7] Lars Kellogg-Stedman| Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Compound subquery" +notmuch search --query=sexp '(thread (of (from keithp) (subject Maildir)))' | notmuch_search_sanitize > OUTPUT +cat< EXPECTED +thread:XXX 2009-11-18 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Compound subquery via field processor" +notmuch search 'sexp:"(thread (of (from keithp) (subject Maildir)))"' | notmuch_search_sanitize > OUTPUT +cat< EXPECTED +thread:XXX 2009-11-18 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "empty subquery" +notmuch search --query=sexp '(thread (of))' 1>OUTPUT 2>&1 +notmuch search '*' > EXPECTED +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "illegal expansion" +notmuch search --query=sexp '(id (of ego))' 1>OUTPUT 2>&1 +cat< EXPECTED +notmuch search: Syntax error in query +'of' unsupported inside 'id' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "(folder (of subquery))" +notmuch search --query=sexp --output=messages '(folder (of (id yun3a4cegoa.fsf@aiko.keithp.com)))' > OUTPUT +cat < EXPECTED +id:yun1vjwegii.fsf@aiko.keithp.com +id:yun3a4cegoa.fsf@aiko.keithp.com +id:1258509400-32511-1-git-send-email-stewart@flamingspork.com +id:1258506353-20352-1-git-send-email-stewart@flamingspork.com +id:20091118010116.GC25380@dottiness.seas.harvard.edu +id:20091118005829.GB25380@dottiness.seas.harvard.edu +id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "infix query" +notmuch search to:searchbyto | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(infix "to:searchbyto")' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "bad infix query 1" +notmuch search --query=sexp '(infix "from:/unbalanced")' 2>&1| notmuch_search_sanitize > OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +Syntax error in infix query: from:/unbalanced +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "bad infix query 2" +notmuch search --query=sexp '(infix "thread:{unbalanced")' 2>&1| notmuch_search_sanitize > OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +Syntax error in infix query: thread:{unbalanced +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "bad infix query 3: bad nesting" +notmuch search --query=sexp '(subject (infix "tag:inbox"))' 2>&1| notmuch_search_sanitize > OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +'infix' not supported inside 'subject' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "infix query that matches no messages" +notmuch search --query=sexp '(and (infix "from:keithp") (infix "to:keithp"))' > OUTPUT +test_expect_equal_file /dev/null OUTPUT + +test_begin_subtest "compound infix query" +notmuch search date:2009-11-18..2009-11-18 and tag:unread > EXPECTED +notmuch search --query=sexp '(infix "date:2009-11-18..2009-11-18 and tag:unread")' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "compound infix query 2" +notmuch search date:2009-11-18..2009-11-18 and tag:unread > EXPECTED +notmuch search --query=sexp '(and (infix "date:2009-11-18..2009-11-18") (infix "tag:unread"))' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, empty" +notmuch search from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date) (from keithp))'| notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, one argument" +notmuch search date:2009-11-18 and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date 2009-11-18) (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, two arguments" +notmuch search date:2009-11-17..2009-11-18 and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date 2009-11-17 2009-11-18) (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, lower bound only" +notmuch search date:2009-11-18.. and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date 2009-11-18 "") (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "date query, upper bound only" +notmuch search date:..2009-11-17 and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date "" 2009-11-17) (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "date query, lower bound only, using *" +notmuch search date:2009-11-18.. and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date 2009-11-18 *) (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "date query, upper bound only, using *" +notmuch search date:..2009-11-17 and from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (date * 2009-11-17) (from keithp))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "date query, illegal nesting 1" +notmuch search --query=sexp '(to (date))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'date' inside 'to' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, illegal nesting 2" +notmuch search --query=sexp '(to (date 2021-11-18))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'date' inside 'to' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, illegal nesting 3" +notmuch search --query=sexp '(date (to))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +expected atom as first argument of 'date' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, illegal nesting 4" +notmuch search --query=sexp '(date today (to))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +expected atom as second argument of 'date' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, too many arguments" +notmuch search --query=sexp '(date yesterday and tommorow)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'date' expects maximum of two arguments +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "date query, bad date" +notmuch search --query=sexp '(date "hawaiian-pizza-day")' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +Didn't understand date specification 'hawaiian-pizza-day' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, empty" +notmuch search from:keithp | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(and (lastmod) (from keithp))'| notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, one argument" +notmuch tag +4EFC743A.3060609@april.org id:4EFC743A.3060609@april.org +revision=$(notmuch count --lastmod '*' | cut -f3) +notmuch search lastmod:$revision..$revision | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(and (lastmod $revision))" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, one argument (negative)" +notmuch tag +4EFC743A.3060609@april.org id:4EFC743A.3060609@april.org +revision=$(notmuch count --lastmod '*' | cut -f3) +revision1=$((revision - 1)) +notmuch search lastmod:$revision1..$revision1 | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod -1)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, two arguments" +notmuch tag +keithp from:keithp +revision2=$(notmuch count --lastmod '*' | cut -f3) +notmuch search lastmod:$revision..$revision2 | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(and (lastmod $revision $revision2))" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, two arguments, first negative" +revdiff=$((revision2 - revision)) +notmuch search lastmod:$revision..$revision2 | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod -$revdiff $revision2)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, two arguments, second negative" +revdiff=$((revision2 - revision)) +notmuch search lastmod:..$revision | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod 0 -$revdiff)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, lower bound only" +notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod $revision \"\")" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, lower bound only (negative)" +notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod -$revdiff \"\")" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, upper bound only" +notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod \"\" $revision2)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, upper bound only (negative)" +notmuch search lastmod:..$revision | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod \"\" -$revdiff)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, lower bound only, using *" +notmuch search lastmod:$revision.. | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod $revision *)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, upper bound only, using *" +notmuch search lastmod:..$revision2 | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp "(lastmod * $revision2)" | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "lastmod query, illegal nesting 1" +notmuch search --query=sexp '(to (lastmod))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'lastmod' inside 'to' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, bad from revision" +notmuch search --query=sexp '(lastmod apples)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +bad 'from' revision: 'apples' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, bad to revision" +notmuch search --query=sexp '(lastmod 0 apples)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +bad 'to' revision: 'apples' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, illegal nesting 2" +notmuch search --query=sexp '(to (lastmod 2021-11-18))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'lastmod' inside 'to' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, illegal nesting 3" +notmuch search --query=sexp '(lastmod (to))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +expected atom as first argument of 'lastmod' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, illegal nesting 4" +notmuch search --query=sexp '(lastmod today (to))' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +expected atom as second argument of 'lastmod' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "lastmod query, too many arguments" +notmuch search --query=sexp '(lastmod yesterday and tommorow)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'lastmod' expects maximum of two arguments +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "user header (unknown header)" +notmuch search --query=sexp '(FooBar)' >& OUTPUT +cat < EXPECTED +notmuch search: Syntax error in query +unknown prefix 'FooBar' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "adding user header" +test_expect_code 0 "notmuch config set index.header.List \"List-Id\"" + +test_begin_subtest "reindexing" +test_expect_code 0 'notmuch reindex "*"' + +test_begin_subtest "wildcard search for user header" +grep -Ril List-Id ${MAIL_DIR} | sort | notmuch_dir_sanitize > EXPECTED +notmuch search --output=files --query=sexp '(List *)' | sort | notmuch_dir_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for user header via field processor" +grep -Ril List-Id ${MAIL_DIR} | sort | notmuch_dir_sanitize > EXPECTED +notmuch search --output=files 'sexp:"(List *)"' | sort | notmuch_dir_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "wildcard search for user header 2" +grep -Ril List-Id ${MAIL_DIR} | sort | notmuch_dir_sanitize > EXPECTED +notmuch search --output=files --query=sexp '(List (starts-with not))' | sort | notmuch_dir_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header" +notmuch search List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(List notmuch)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (list token)" +notmuch search List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(List notmuch.notmuchmail.org)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (quoted string)" +notmuch search 'List:"notmuch notmuchmail org"' | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(List "notmuch notmuchmail org")' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search for user header (atoms)" +notmuch search 'List:"notmuch notmuchmail org"' | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(List notmuch notmuchmail org)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "check saved query name" +test_expect_code 1 "notmuch config set squery.test '(subject utf8-sübjéct)'" + +test_begin_subtest "roundtrip saved query (database)" +notmuch config set --database squery.Test '(subject utf8-sübjéct)' +output=$(notmuch config get squery.Test) +test_expect_equal "$output" '(subject utf8-sübjéct)' + +test_begin_subtest "roundtrip saved query" +notmuch config set squery.Test '(subject override subject)' +output=$(notmuch config get squery.Test) +test_expect_equal "$output" '(subject override subject)' + +test_begin_subtest "unknown saved query" +notmuch search --query=sexp '(Unknown foo bar)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +unknown prefix 'Unknown' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "syntax error in saved query" +notmuch config set squery.Bad '(Bad' +notmuch search --query=sexp '(Bad foo bar)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +invalid saved s-expression query: '(Bad' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search by 'tag' and 'subject'" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.TagSubject '(and (tag inbox) (subject maildir))' +notmuch search --query=sexp '(TagSubject)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: illegal nesting" +notmuch config set squery.TagSubject '(and (tag inbox) (subject maildir))' +notmuch search --query=sexp '(subject (TagSubject))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +nested field: 'tag' inside 'subject' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: list as prefix" +notmuch config set squery.Bad2 '((and) (tag inbox) (subject maildir))' +notmuch search --query=sexp '(Bad2)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +unexpected list in field/operation position +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax" +notmuch config set squery.Bad3 '(macro a b)' +notmuch search --query=sexp '(Bad3)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +missing (possibly empty) list of arguments to macro +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 2" +notmuch config set squery.Bad4 '(macro ((a b)) a)' +notmuch search --query=sexp '(Bad4 1)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +macro parameters must be unquoted atoms +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 3" +notmuch config set squery.Bad5 '(macro (a b) a)' +notmuch search --query=sexp '(Bad5 1)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +too few arguments to macro +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 4" +notmuch search --query=sexp '(Bad5 1 2 3)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +too many arguments to macro +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 5" +notmuch config set squery.Bad5 '(macro (thing) (tag (rx ,thing)))' +notmuch search --query=sexp '(Bad5 (1 2))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'rx' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 6" +notmuch config set squery.Bad6 '(macro (thing) (tag (starts-with ,thing)))' +notmuch search --query=sexp '(Bad6 (1 2))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +'starts-with' expects single atom as argument +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: bad parameter syntax 7" +notmuch search --query=sexp '(subject (rx ,unknown))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +undefined parameter 'unknown' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Saved Search: macro without body" +notmuch config set squery.Bad3 '(macro (a b))' +notmuch search --query=sexp '(Bad3)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +missing body of macro +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "macro in query" +notmuch search --query=sexp '(macro (a) (and ,b (subject maildir)))' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +macro definition not permitted here +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "zero argument macro" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.TagSubject0 '(macro () (and (tag inbox) (subject maildir)))' +notmuch search --query=sexp '(TagSubject0)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "undefined argument" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Bad6 '(macro (a) (and ,b (subject maildir)))' +notmuch search --query=sexp '(Bad6 foo)' >OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +undefined parameter 'b' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Single argument macro" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.TagSubject1 '(macro (tagname) (and (tag ,tagname) (subject maildir)))' +notmuch search --query=sexp '(TagSubject1 inbox)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Single argument macro, list argument" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.ThingSubject '(macro (thing) (and ,thing (subject maildir)))' +notmuch search --query=sexp '(ThingSubject (tag inbox))' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "two argument macro" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.TagSubject2 '(macro (tagname subj) (and (tag ,tagname) (subject ,subj)))' +notmuch search --query=sexp '(TagSubject2 inbox maildir)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "macro in regex" +notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED +notmuch config set squery.D '(macro (tagname) (and (date 2009-11-17) (tag (rx ,tagname))))' +notmuch search --query=sexp '(D inbo)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "macro in wildcard" +notmuch search tag:inbox and date:2009-11-17 | notmuch_search_sanitize > EXPECTED +notmuch config set squery.W '(macro (tagname) (and (date 2009-11-17) (tag (starts-with ,tagname))))' +notmuch search --query=sexp '(W inbo)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "nested macros (shadowing)" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Inner '(macro (x) (subject ,x))' +notmuch config set squery.Outer '(macro (x y) (and (tag ,x) (Inner ,y)))' +notmuch search --query=sexp '(Outer inbox maildir)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "nested macros (no dynamic scope)" +notmuch config set squery.Inner2 '(macro (x) (subject ,y))' +notmuch config set squery.Outer2 '(macro (x y) (and (tag ,x) (Inner2 ,y)))' +notmuch search --query=sexp '(Outer2 inbox maildir)' > OUTPUT 2>&1 +cat < EXPECTED +notmuch search: Syntax error in query +undefined parameter 'y' +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "nested macros (shadowing, regex)" +notmuch search tag:/inbo/ and subject:/Maildi/ | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Inner3 '(macro (x) (subject (rx ,x)))' +notmuch config set squery.Outer3 '(macro (x y) (and (tag (rx ,x)) (Inner3 ,y)))' +notmuch search --query=sexp '(Outer3 inbo Maildi)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "nested macros (shadowing, wildcard)" +notmuch search tag:inbox and subject:maildir | notmuch_search_sanitize > EXPECTED +notmuch config set squery.Inner4 '(macro (x) (subject (starts-with ,x)))' +notmuch config set squery.Outer4 '(macro (x y) (and (tag (starts-with ,x)) (Inner4 ,y)))' +notmuch search --query=sexp '(Outer4 inbo maildi)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "combine macro and user defined header" +notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))' +notmuch search subject:notmuch or List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(About notmuch)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + + +test_begin_subtest "combine macro and user defined header" +notmuch config set squery.About '(macro (name) (or (subject ,name) (List ,name)))' +notmuch search subject:notmuch or List:notmuch | notmuch_search_sanitize > EXPECTED +notmuch search --query=sexp '(About notmuch)' | notmuch_search_sanitize > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + + +test_done diff --git a/test/T090-search-output.sh b/test/T090-search-output.sh index bf28d220..0d85c609 100755 --- a/test/T090-search-output.sh +++ b/test/T090-search-output.sh @@ -435,7 +435,7 @@ test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "search for non-existent message prints nothing" notmuch search "no-message-matches-this" > OUTPUT -echo -n >EXPECTED +: >EXPECTED test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "search --format=json for non-existent message prints proper empty json" diff --git a/test/T095-address.sh b/test/T095-address.sh index 817be538..0cafbe20 100755 --- a/test/T095-address.sh +++ b/test/T095-address.sh @@ -325,4 +325,11 @@ cat <EXPECTED EOF test_expect_equal_file EXPECTED OUTPUT +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + test_begin_subtest "sexpr query: all messages" + notmuch address '*' > EXPECTED + notmuch address --query=sexp '()' > OUTPUT + test_expect_equal_file EXPECTED OUTPUT +fi + test_done diff --git a/test/T100-search-by-folder.sh b/test/T100-search-by-folder.sh index 409cfdcc..b4f6294e 100755 --- a/test/T100-search-by-folder.sh +++ b/test/T100-search-by-folder.sh @@ -18,6 +18,12 @@ test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; T thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" +test_begin_subtest "search by path: specification (multiple)" +output=$(notmuch search path:bad path:bad/news path:things/bad | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; To the bone (inbox unread) +thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite; Bears (inbox unread) +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Bites, stings, sad feelings (inbox unread)" + test_begin_subtest "Top level folder" output=$(notmuch search folder:'""' | notmuch_search_sanitize) test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Top level (inbox unread)" @@ -28,8 +34,13 @@ test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1(2)] Notmuch Test Suite test_begin_subtest "Folder search with --output=files" output=$(notmuch search --output=files folder:bad/news | notmuch_search_files_sanitize) -test_expect_equal "$output" "MAIL_DIR/bad/news/msg-003 -MAIL_DIR/duplicate/bad/news/msg-003" +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" + +test_begin_subtest "Folder search with --output=files (trailing /)" +output=$(notmuch search --output=files folder:bad/news/ | notmuch_search_files_sanitize) +test_expect_equal "$output" "MAIL_DIR/bad/news/msg-XXX +MAIL_DIR/duplicate/bad/news/msg-XXX" test_begin_subtest "After removing duplicate instance of matching path" rm -r "${MAIL_DIR}/bad/news" @@ -39,7 +50,7 @@ test_expect_equal "$output" "" test_begin_subtest "Folder search with --output=files part #2" output=$(notmuch search --output=files folder:duplicate/bad/news | notmuch_search_files_sanitize) -test_expect_equal "$output" "MAIL_DIR/duplicate/bad/news/msg-003" +test_expect_equal "$output" "MAIL_DIR/duplicate/bad/news/msg-XXX" test_begin_subtest "After removing duplicate instance of matching path part #2" output=$(notmuch search folder:duplicate/bad/news | notmuch_search_sanitize) @@ -120,6 +131,13 @@ test_expect_equal "$output" "MAIL_DIR/bar/17:2, MAIL_DIR/bar/18:2, MAIL_DIR/cur/51:2," +test_begin_subtest "path: search (trailing /)" +output=$(notmuch search --output=files path:"bar/" | notmuch_search_files_sanitize | sort) +# cur/51:2, is a duplicate of bar/18:2, +test_expect_equal "$output" "MAIL_DIR/bar/17:2, +MAIL_DIR/bar/18:2, +MAIL_DIR/cur/51:2," + test_begin_subtest "top level path: search" output=$(notmuch search --output=files path:'""' | notmuch_search_files_sanitize | sort) test_expect_equal "$output" "MAIL_DIR/01:2, diff --git a/test/T131-show-limiting.sh b/test/T131-show-limiting.sh new file mode 100755 index 00000000..30d1f254 --- /dev/null +++ b/test/T131-show-limiting.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash +test_description='"notmuch show" --offset and --limit parameters' +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +show () { + local kind="$1" + shift + if [ "$kind" = messages ]; then + set -- --unthreaded "$@" + fi + notmuch show --body=false --format=text --entire-thread=false "$@" "*" | + sed -nre 's/^.message\{.*\.*/&/p' +} + +for outp in messages threads; do + test_begin_subtest "$outp: limit does the right thing" + show $outp | head -n 20 >expected + show $outp --limit=20 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: concatenation of limited shows" + show $outp | head -n 20 >expected + show $outp --limit=10 >output + show $outp --limit=10 --offset=10 >>output + test_expect_equal_file expected output + + test_begin_subtest "$outp: limit larger than result set" + N=$(notmuch count --output=$outp "*") + show $outp >expected + show $outp --limit=$((1 + N)) >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: limit = 0" + test_expect_equal "$(show $outp --limit=0)" "" + + test_begin_subtest "$outp: offset does the right thing" + # note: tail -n +N is 1-based + show $outp | tail -n +21 >expected + show $outp --offset=20 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: offset = 0" + show $outp >expected + show $outp --offset=0 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset" + show $outp | tail -n 20 >expected + show $outp --offset=-20 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset" + show $outp | tail -n 1 >expected + show $outp --offset=-1 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset combined with limit" + show $outp | tail -n 20 | head -n 10 >expected + show $outp --offset=-20 --limit=10 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset combined with equal limit" + show $outp | tail -n 20 >expected + show $outp --offset=-20 --limit=20 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset combined with large limit" + show $outp | tail -n 20 >expected + show $outp --offset=-20 --limit=50 >output + test_expect_equal_file expected output + + test_begin_subtest "$outp: negative offset larger than results" + N=$(notmuch count --output=$outp "*") + show $outp >expected + show $outp --offset=-$((1 + N)) >output + test_expect_equal_file expected output +done + +test_done diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh index 0cf69975..e9cc9cb0 100755 --- a/test/T140-excludes.sh +++ b/test/T140-excludes.sh @@ -5,8 +5,7 @@ test_description='"notmuch search, count and show" with excludes in several vari # Generates a thread consisting of a top level message and 'length' # replies. The subject of the top message 'subject: top message" # and the subject of the nth reply in the thread is "subject: reply n" -generate_thread () -{ +generate_thread () { local subject="$1" local length="$2" generate_message '[subject]="'"${subject}: top message"'"' '[body]="'"body of top message"'"' @@ -22,7 +21,7 @@ generate_thread () done notmuch new > /dev/null # We cannot retrieve the thread_id until after we have run notmuch new. - gen_thread_id=`notmuch search --output=threads id:${gen_thread_msg_id[0]}` + gen_thread_id=$(notmuch search --output=threads id:${gen_thread_msg_id[0]} 2>/dev/null) } ############################################# @@ -39,6 +38,16 @@ deleted_id=$gen_msg_id output=$(notmuch search subject:deleted | notmuch_search_sanitize) test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)" +test_begin_subtest "Search, exclude \"deleted\" messages; alternate config file" +cp ${NOTMUCH_CONFIG} alt-config +notmuch config set search.exclude_tags +notmuch --config=alt-config search subject:deleted | notmuch_search_sanitize > OUTPUT +cp alt-config ${NOTMUCH_CONFIG} +cat < EXPECTED +thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread) +EOF +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "Search, exclude \"deleted\" messages from message search" output=$(notmuch search --output=messages subject:deleted | notmuch_search_sanitize) test_expect_equal "$output" "id:$not_deleted_id" @@ -276,6 +285,21 @@ test_begin_subtest "Count, default exclusion: tag in query (threads)" output=$(notmuch count --output=threads tag:test and tag:deleted) test_expect_equal "$output" "3" +test_begin_subtest "Count, default exclusion, batch" +notmuch count --batch --output=messages< OUTPUT +tag:test +tag:test and tag:deleted +tag:test +tag:test and tag:deleted +EOF +cat <EXPECTED +2 +4 +2 +4 +EOF +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "Count, exclude=true: tag in query (messages)" output=$(notmuch count --exclude=true tag:test and tag:deleted) test_expect_equal "$output" "4" @@ -364,7 +388,7 @@ Subject: No messages excluded: single match: reply 4 Subject: No messages excluded: single match: reply 5" test_begin_subtest "Show, exclude=false" -output=$(notmuch show --exclude=false tag:test | notmuch_show_sanitize_all | egrep "Subject:|message{") +output=$(notmuch show --exclude=false tag:test | notmuch_show_sanitize_all | egrep "Subject:|message{") test_expect_equal "$output" " message{ id:XXXXX depth:0 match:1 excluded:1 filename:XXXXX Subject: All messages excluded: single match: reply 2 message{ id:XXXXX depth:0 match:1 excluded:1 filename:XXXXX diff --git a/test/T150-tagging.sh b/test/T150-tagging.sh index 208b4b98..273c0af3 100755 --- a/test/T150-tagging.sh +++ b/test/T150-tagging.sh @@ -2,6 +2,21 @@ test_description='"notmuch tag"' . $(dirname "$0")/test-lib.sh || exit 1 +test_query_syntax () { + # use a tag with a space to stress the query string munging code. + local new_tag="${RANDOM} ${RANDOM}" + test_begin_subtest "sexpr query: $1" + backup_database + notmuch tag --query=sexp "+${new_tag}" -- "$1" + notmuch dump > OUTPUT + restore_database + backup_database + notmuch tag "+${new_tag}" -- "$2" + notmuch dump > EXPECTED + restore_database + test_expect_equal_file_nonempty EXPECTED OUTPUT +} + add_message '[subject]=One' add_message '[subject]=Two' @@ -90,7 +105,7 @@ thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; One (inbox tag5 unread) thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; Two (inbox tag4 tag5 unread)" # generate a common input file for the next several tests. -cat > batch.in < batch.in <&1 | sed 's/: .*$//' ) -chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.${db_ending} +chmod u+w ${MAIL_DIR}/.notmuch/xapian/*.* test_expect_equal "$output" "A Xapian exception occurred opening database" +add_email_corpus + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + test_query_syntax '(and "wonderful" "wizard")' 'wonderful and wizard' + test_query_syntax '(or "php" "wizard")' 'php or wizard' + test_query_syntax 'wizard' 'wizard' + test_query_syntax 'Wizard' 'Wizard' + test_query_syntax '(attachment notmuch-help.patch)' 'attachment:notmuch-help.patch' + test_query_syntax '(mimetype text/html)' 'mimetype:text/html' + + test_begin_subtest "--batch --query=sexp" + notmuch dump --format=batch-tag > backup.tags + notmuch tag --batch --query=sexp < OUTPUT + cat < EXPECTED + #notmuch-dump batch-tag:3 config,properties,tags + +all +inbox +tag5 +unread -- id:msg-001@notmuch-test-suite + +all +inbox +tag4 +tag5 +unread -- id:msg-002@notmuch-test-suite +EOF + notmuch restore --format=batch-tag < backup.tags + test_expect_equal_file EXPECTED OUTPUT + +fi + test_done diff --git a/test/T160-json.sh b/test/T160-json.sh index 004adb4e..318c9788 100755 --- a/test/T160-json.sh +++ b/test/T160-json.sh @@ -1,20 +1,21 @@ #!/usr/bin/env bash test_description="--format=json output" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 test_begin_subtest "Show message: json" add_message "[subject]=\"json-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[bcc]=\"test_suite+bcc@notmuchmail.org\"" "[reply-to]=\"test_suite+replyto@notmuchmail.org\"" "[body]=\"json-show-message\"" -output=$(notmuch show --format=json "json-show-message") -test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"${gen_msg_filename}\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]" +output=$(notmuch show --format=json "json-show-message" | notmuch_json_show_sanitize) +test_expect_equal_json "$output" "[[[{\"id\": \"XXXXX\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"YYYYY\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]" # This should be the same output as above. test_begin_subtest "Show message: json --body=true" -output=$(notmuch show --format=json --body=true "json-show-message") -test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"${gen_msg_filename}\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]" +output=$(notmuch show --format=json --body=true "json-show-message" | notmuch_json_show_sanitize) +test_expect_equal_json "$output" "[[[{\"id\": \"XXXXX\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"YYYYY\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"json-show-message\n\"}]}, []]]]" test_begin_subtest "Show message: json --body=false" -output=$(notmuch show --format=json --body=false "json-show-message") -test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"${gen_msg_filename}\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}}, []]]]" +output=$(notmuch show --format=json --body=false "json-show-message" | notmuch_json_show_sanitize) +test_expect_equal_json "$output" "[[[{\"id\": \"XXXXX\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"YYYYY\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Bcc\": \"test_suite+bcc@notmuchmail.org\", \"Reply-To\": \"test_suite+replyto@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}}, []]]]" test_begin_subtest "Search message: json" add_message "[subject]=\"json-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"json-search-message\"" @@ -32,8 +33,8 @@ test_expect_equal_json "$output" "[{\"thread\": \"XXX\", test_begin_subtest "Show message: json, utf-8" add_message "[subject]=\"json-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-show-méssage\"" -output=$(notmuch show --format=json "jsön-show-méssage") -test_expect_equal_json "$output" "[[[{\"id\": \"${gen_msg_id}\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"${gen_msg_filename}\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"jsön-show-méssage\n\"}]}, []]]]" +output=$(notmuch show --format=json "jsön-show-méssage" | notmuch_json_show_sanitize) +test_expect_equal_json "$output" "[[[{\"id\": \"XXXXX\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"YYYYY\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\",\"unread\"], \"headers\": {\"Subject\": \"json-show-utf8-body-sübjéct\", \"From\": \"Notmuch Test Suite \", \"To\": \"Notmuch Test Suite \", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"text/plain\", \"content\": \"jsön-show-méssage\n\"}]}, []]]]" test_begin_subtest "Show message: json, inline attachment filename" subject='json-show-inline-attachment-filename' @@ -48,7 +49,7 @@ output=$(notmuch show --format=json "id:$id") filename=$(notmuch search --output=files "id:$id") # Get length of README after base64-encoding, minus additional newline. attachment_length=$(( $(base64 $NOTMUCH_SRCDIR/test/README | wc -c) - 1 )) -test_expect_equal_json "$output" "[[[{\"id\": \"$id\", \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"$filename\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"test_suite@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed\", \"content\": [{\"id\": 2, \"content-type\": \"text/plain\", \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"content-length\": $attachment_length, \"content-transfer-encoding\": \"base64\", \"content-disposition\": \"inline\", \"filename\": \"README\"}]}]}, []]]]" +test_expect_equal_json "$output" "[[[{\"id\": \"$id\", \"duplicate\": 1, \"crypto\": {}, \"match\": true, \"excluded\": false, \"filename\": [\"$filename\"], \"timestamp\": 946728000, \"date_relative\": \"2000-01-01\", \"tags\": [\"inbox\"], \"headers\": {\"Subject\": \"$subject\", \"From\": \"Notmuch Test Suite \", \"To\": \"test_suite@notmuchmail.org\", \"Date\": \"Sat, 01 Jan 2000 12:00:00 +0000\"}, \"body\": [{\"id\": 1, \"content-type\": \"multipart/mixed\", \"content\": [{\"id\": 2, \"content-type\": \"text/plain\", \"content\": \"This is a test message with inline attachment with a filename\"}, {\"id\": 3, \"content-type\": \"application/octet-stream\", \"content-length\": $attachment_length, \"content-transfer-encoding\": \"base64\", \"content-disposition\": \"inline\", \"filename\": \"README\"}]}]}, []]]]" test_begin_subtest "Search message: json, utf-8" add_message "[subject]=\"json-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-search-méssage\"" @@ -64,6 +65,25 @@ test_expect_equal_json "$output" "[{\"thread\": \"XXX\", \"tags\": [\"inbox\", \"unread\"]}]" +if [ -z "${NOTMUCH_TEST_INSTALLED-}" ]; then +test_begin_subtest "Search message: json, 64-bit timestamp" +if [ "${NOTMUCH_HAVE_64BIT_TIME_T-0}" != "1" ]; then + test_subtest_known_broken +fi +add_message "[subject]=\"json-search-64bit-timestamp-subject\"" "[date]=\"Tue, 01 Jan 2999 12:00:00 -0000\"" "[body]=\"json-search-64bit-timestamp-message\"" +output=$(notmuch search --format=json "json-search-64bit-timestamp-message" | notmuch_search_sanitize) +test_expect_equal_json "$output" "[{\"thread\": \"XXX\", + \"timestamp\": 32472187200, + \"date_relative\": \"the future\", + \"matched\": 1, + \"total\": 1, + \"authors\": \"Notmuch Test Suite\", + \"subject\": \"json-search-64bit-timestamp-subject\", + \"query\": [\"id:$gen_msg_id\", null], + \"tags\": [\"inbox\", + \"unread\"]}]" +fi # NOTMUCH_TEST_INSTALLED undefined / empty + test_begin_subtest "Format version: too low" test_expect_code 20 "notmuch search --format-version=0 \\*" @@ -79,6 +99,7 @@ cat < EXPECTED [ { "date_relative": "2001-01-05", + "duplicate": 1, "excluded": false, "filename": [ "${MAIL_DIR}/copy1", @@ -114,6 +135,7 @@ cat < EXPECTED [ { "date_relative": "2001-01-05", + "duplicate": 1, "excluded": false, "filename": "${MAIL_DIR}/copy1", "headers": { @@ -138,4 +160,46 @@ EOF output=$(notmuch show --format=json --body=false --format-version=2 id:message-id@example.com) test_expect_equal_json "$output" "$(cat EXPECTED)" +test_begin_subtest "show extra headers" +add_message "[subject]=\"extra-headers\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[in-reply-to]=\"\"" "[body]=\"extra-headers test\""\ + "[header]=\"Received: from mail.example.com (mail.example.com [1.1.1.1]) + by mail.notmuchmail.org (some MTA) with ESMTP id 12345678 + for ; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)\"" \ + +notmuch config set show.extra_headers "in-reply-to;received" +output=$(notmuch show --format=json --body=false id:${gen_msg_id} | notmuch_json_show_sanitize) +cat < EXPECTED +[ + [ + [ + { + "crypto": {}, + "date_relative": "2000-01-01", + "excluded": false, + "filename": [ + "YYYYY" + ], + "headers": { + "Date": "Sat, 01 Jan 2000 12:00:00 +0000", + "From": "Notmuch Test Suite ", + "In-Reply-To": "", + "Received": "from mail.example.com (mail.example.com [1.1.1.1])\tby mail.notmuchmail.org (some MTA) with ESMTP id 12345678\tfor ; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)", + "Subject": "extra-headers", + "To": "Notmuch Test Suite " + }, + "id": "XXXXX", + "match": true, + "tags": [ + "inbox", + "unread" + ], + "timestamp": 946728000 + }, + [] + ] + ] +] +EOF +test_expect_equal_json "${output}" "$(cat EXPECTED)" + test_done diff --git a/test/T170-sexp.sh b/test/T170-sexp.sh index 24be8351..0be94bd2 100755 --- a/test/T170-sexp.sh +++ b/test/T170-sexp.sh @@ -1,35 +1,36 @@ #!/usr/bin/env bash test_description="--format=sexp output" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 test_begin_subtest "Show message: sexp" add_message "[subject]=\"sexp-show-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[bcc]=\"test_suite+bcc@notmuchmail.org\"" "[reply-to]=\"test_suite+replyto@notmuchmail.org\"" "[body]=\"sexp-show-message\"" -output=$(notmuch show --format=sexp "sexp-show-message") -test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :filename (\"${gen_msg_filename}\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"sexp-show-message\n\")) :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" +output=$(notmuch show --format=sexp "sexp-show-message" | notmuch_sexp_show_sanitize) +test_expect_equal "$output" "((((:id \"XXXXX\" :match t :excluded nil :filename (\"YYYYY\") :timestamp 42 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"sexp-show-message\n\")) :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"GENERATED_DATE\")) ())))" # This should be the same output as above. test_begin_subtest "Show message: sexp --body=true" -output=$(notmuch show --format=sexp --body=true "sexp-show-message") -test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :filename (\"${gen_msg_filename}\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"sexp-show-message\n\")) :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" +output=$(notmuch show --format=sexp --body=true "sexp-show-message" | notmuch_sexp_show_sanitize) +test_expect_equal "$output" "((((:id \"XXXXX\" :match t :excluded nil :filename (\"YYYYY\") :timestamp 42 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"sexp-show-message\n\")) :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"GENERATED_DATE\")) ())))" test_begin_subtest "Show message: sexp --body=false" -output=$(notmuch show --format=sexp --body=false "sexp-show-message") -test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :filename (\"${gen_msg_filename}\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" +output=$(notmuch show --format=sexp --body=false "sexp-show-message" | notmuch_sexp_show_sanitize) +test_expect_equal "$output" "((((:id \"XXXXX\" :match t :excluded nil :filename (\"YYYYY\") :timestamp 42 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :crypto () :headers (:Subject \"sexp-show-subject\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Bcc \"test_suite+bcc@notmuchmail.org\" :Reply-To \"test_suite+replyto@notmuchmail.org\" :Date \"GENERATED_DATE\")) ())))" test_begin_subtest "Search message: sexp" add_message "[subject]=\"sexp-search-subject\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"sexp-search-message\"" -output=$(notmuch search --format=sexp "sexp-search-message" | notmuch_search_sanitize) -test_expect_equal "$output" "((:thread \"0000000000000002\" :timestamp 946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch Test Suite\" :subject \"sexp-search-subject\" :query (\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))" +output=$(notmuch search --format=sexp "sexp-search-message" | notmuch_sexp_search_sanitize) +test_expect_equal "$output" "((:thread \"XXX\" :timestamp 946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch Test Suite\" :subject \"sexp-search-subject\" :query (\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))" test_begin_subtest "Show message: sexp, utf-8" add_message "[subject]=\"sexp-show-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-show-méssage\"" -output=$(notmuch show --format=sexp "jsön-show-méssage") -test_expect_equal "$output" "((((:id \"${gen_msg_id}\" :match t :excluded nil :filename (\"${gen_msg_filename}\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"jsön-show-méssage\n\")) :crypto () :headers (:Subject \"sexp-show-utf8-body-sübjéct\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" +output=$(notmuch show --format=sexp "jsön-show-méssage" | notmuch_sexp_show_sanitize) +test_expect_equal "$output" "((((:id \"XXXXX\" :match t :excluded nil :filename (\"YYYYY\") :timestamp 42 :date_relative \"2000-01-01\" :tags (\"inbox\" \"unread\") :body ((:id 1 :content-type \"text/plain\" :content \"jsön-show-méssage\n\")) :crypto () :headers (:Subject \"sexp-show-utf8-body-sübjéct\" :From \"Notmuch Test Suite \" :To \"Notmuch Test Suite \" :Date \"GENERATED_DATE\")) ())))" test_begin_subtest "Search message: sexp, utf-8" add_message "[subject]=\"sexp-search-utf8-body-sübjéct\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[body]=\"jsön-search-méssage\"" -output=$(notmuch search --format=sexp "jsön-search-méssage" | notmuch_search_sanitize) -test_expect_equal "$output" "((:thread \"0000000000000004\" :timestamp 946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch Test Suite\" :subject \"sexp-search-utf8-body-sübjéct\" :query (\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))" +output=$(notmuch search --format=sexp "jsön-search-méssage" | notmuch_sexp_search_sanitize) +test_expect_equal "$output" "((:thread \"XXX\" :timestamp 946728000 :date_relative \"2000-01-01\" :matched 1 :total 1 :authors \"Notmuch Test Suite\" :subject \"sexp-search-utf8-body-sübjéct\" :query (\"id:$gen_msg_id\" nil) :tags (\"inbox\" \"unread\")))" test_begin_subtest "Show message: sexp, inline attachment filename" subject='sexp-show-inline-attachment-filename' @@ -44,6 +45,19 @@ output=$(notmuch show --format=sexp "id:$id") filename=$(notmuch search --output=files "id:$id") # Get length of README after base64-encoding, minus additional newline. attachment_length=$(( $(base64 $NOTMUCH_SRCDIR/test/README | wc -c) - 1 )) -test_expect_equal "$output" "((((:id \"$id\" :match t :excluded nil :filename (\"$filename\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\") :body ((:id 1 :content-type \"multipart/mixed\" :content ((:id 2 :content-type \"text/plain\" :content \"This is a test message with inline attachment with a filename\") (:id 3 :content-type \"application/octet-stream\" :content-disposition \"inline\" :filename \"README\" :content-transfer-encoding \"base64\" :content-length $attachment_length)))) :crypto () :headers (:Subject \"sexp-show-inline-attachment-filename\" :From \"Notmuch Test Suite \" :To \"test_suite@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" +test_expect_equal "$output" "((((:id \"$id\" :match t :excluded nil :filename (\"$filename\") :timestamp 946728000 :date_relative \"2000-01-01\" :tags (\"inbox\") :duplicate 1 :body ((:id 1 :content-type \"multipart/mixed\" :content ((:id 2 :content-type \"text/plain\" :content \"This is a test message with inline attachment with a filename\") (:id 3 :content-type \"application/octet-stream\" :content-disposition \"inline\" :filename \"README\" :content-transfer-encoding \"base64\" :content-length $attachment_length)))) :crypto () :headers (:Subject \"sexp-show-inline-attachment-filename\" :From \"Notmuch Test Suite \" :To \"test_suite@notmuchmail.org\" :Date \"Sat, 01 Jan 2000 12:00:00 +0000\")) ())))" + +test_begin_subtest "show extra headers" +add_message "[subject]=\"extra-headers\"" "[date]=\"Sat, 01 Jan 2000 12:00:00 -0000\"" "[in-reply-to]=\"\"" "[body]=\"extra-headers test\""\ + "[header]=\"Received: from mail.example.com (mail.example.com [1.1.1.1]) + by mail.notmuchmail.org (some MTA) with ESMTP id 12345678 + for ; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)\"" \ + +notmuch config set show.extra_headers "in-reply-to;received" +notmuch show --format=sexp --body=false id:${gen_msg_id} | notmuch_sexp_show_sanitize > OUTPUT +cat < EXPECTED +((((:id "XXXXX" :match t :excluded nil :filename ("YYYYY") :timestamp 42 :date_relative "2000-01-01" :tags ("inbox" "unread") :crypto () :headers (:Subject "extra-headers" :From "Notmuch Test Suite " :To "Notmuch Test Suite " :Date "GENERATED_DATE" :In-Reply-To "" :Received "from mail.example.com (mail.example.com [1.1.1.1])\011by mail.notmuchmail.org (some MTA) with ESMTP id 12345678\011for ; Sat, 10 Apr 2010 07:54:51 -0400 (EDT)")) ()))) +EOF +test_expect_equal_file EXPECTED OUTPUT test_done diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh index 6f715ff9..cfe48ac5 100755 --- a/test/T190-multipart.sh +++ b/test/T190-multipart.sh @@ -376,18 +376,18 @@ test_begin_subtest "--format=text --part=8, no part, expect error" test_expect_success "notmuch show --format=text --part=8 'id:87liy5ap00.fsf@yoom.home.cworth.org'" test_begin_subtest "--format=json --part=0, full message" -notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT +notmuch show --format=json --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | notmuch_json_show_sanitize >OUTPUT cat <EXPECTED -{"id": "87liy5ap00.fsf@yoom.home.cworth.org", "crypto": {}, "match": true, "excluded": false, "filename": ["${MAIL_DIR}/multipart"], "timestamp": 978709437, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth ", "To": "cworth@cworth.org", "Date": "Fri, 05 Jan 2001 15:43:57 +0000"}, "body": [ +{"id": "XXXXX", "crypto": {}, "match": true, "excluded": false, "filename": ["YYYYY"], "timestamp": 42, "date_relative": "2001-01-05", "tags": ["attachment","inbox","signed","unread"], "headers": {"Subject": "Multipart message", "From": "Carl Worth ", "To": "cworth@cworth.org", "Date": "GENERATED_DATE"}, "body": [ {"id": 1, "content-type": "multipart/signed", "content": [ {"id": 2, "content-type": "multipart/mixed", "content": [ -{"id": 3, "content-type": "message/rfc822", "content-disposition": "inline", "content": [{"headers": {"Subject": "html message", "From": "Carl Worth ", "To": "cworth@cworth.org", "Date": "Fri, 05 Jan 2001 15:42:57 +0000"}, "body": [ +{"id": 3, "content-type": "message/rfc822", "content-disposition": "inline", "content": [{"headers": {"Subject": "html message", "From": "Carl Worth ", "To": "cworth@cworth.org", "Date": "GENERATED_DATE"}, "body": [ {"id": 4, "content-type": "multipart/alternative", "content": [ -{"id": 5, "content-type": "text/html", "content-length": 71}, +{"id": 5, "content-type": "text/html", "content-length": "NONZERO"}, {"id": 6, "content-type": "text/plain", "content": "This is an embedded message, with a multipart/alternative part.\n"}]}]}]}, {"id": 7, "content-type": "text/plain", "content-disposition": "attachment", "filename": "attachment", "content": "This is a text attachment.\n"}, {"id": 8, "content-type": "text/plain", "content": "And this message is signed.\n\n-Carl\n"}]}, -{"id": 9, "content-type": "application/pgp-signature", "content-length": 197}]}]} +{"id": 9, "content-type": "application/pgp-signature", "content-length": "NONZERO"}]}]} EOF test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED)" @@ -485,7 +485,7 @@ notmuch show --format=raw 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT test_expect_equal_file "${MAIL_DIR}"/multipart OUTPUT test_begin_subtest "--format=raw --part=0, full message" -notmuch show --format=raw --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' >OUTPUT +notmuch show --format=raw --part=0 'id:87liy5ap00.fsf@yoom.home.cworth.org' | notmuch_json_show_sanitize >OUTPUT test_expect_equal_file "${MAIL_DIR}"/multipart OUTPUT test_begin_subtest "--format=raw --part=1, message body" @@ -683,7 +683,7 @@ test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED)" test_begin_subtest "'notmuch show --part' does not corrupt a part with CRLF pair" notmuch show --format=raw --part=3 id:base64-part-with-crlf > crlf.out -echo -n -e "\xEF\x0D\x0A" > crlf.expected +printf "\xEF\x0D\x0A" > crlf.expected test_expect_equal_file crlf.out crlf.expected @@ -725,13 +725,12 @@ EOF notmuch new > /dev/null -cat_expected_head () -{ +cat_expected_head () { cat <", "Subject": "html message", "To": "B "}, @@ -743,8 +742,8 @@ EOF cat_expected_head > EXPECTED.nohtml cat <> EXPECTED.nohtml "content": [ - { "id": 2, "content-charset": "UTF-8", "content-length": 21, "content-type": "text/html"}, - { "id": 3, "content-charset": "ISO-8859-1", "content-length": 20, "content-type": "text/html"}, + { "id": 2, "content-charset": "UTF-8", "content-length": "NONZERO", "content-type": "text/html"}, + { "id": 3, "content-charset": "ISO-8859-1", "content-length": "NONZERO", "content-type": "text/html"}, { "id": 4, "content-type": "text/plain", "content": "0.5 equals \\u00bd\\n"} ]}]},[]]]] EOF @@ -760,11 +759,11 @@ cat <> EXPECTED.withhtml EOF test_begin_subtest "html parts excluded by default" -notmuch show --format=json id:htmlmessage > OUTPUT +notmuch show --format=json id:htmlmessage | notmuch_json_show_sanitize > OUTPUT test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.nohtml)" test_begin_subtest "html parts included" -notmuch show --format=json --include-html id:htmlmessage > OUTPUT +notmuch show --format=json --include-html id:htmlmessage | notmuch_json_show_sanitize > OUTPUT test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)" test_begin_subtest "indexes mime-type #1" diff --git a/test/T210-raw.sh b/test/T210-raw.sh index 85e707d4..44082028 100755 --- a/test/T210-raw.sh +++ b/test/T210-raw.sh @@ -40,7 +40,7 @@ for pow in range(10,21): msg['To'] = msg['From'] msg['Message-Id'] = 'size-{:07d}@notmuch-test-suite'.format(size) content = "" - msg.set_content("") + msg.set_content("\n") padding = size - len(bytes(msg)) lines = [] while padding > 0: @@ -61,7 +61,18 @@ for pow in {10..20}; do notmuch show --format=raw subject:$size > OUTPUT test_expect_equal_file mail/size-$size OUTPUT test_begin_subtest "return value, message of size $size" - test_expect_success "notmuch show --format=raw subject:$size > /dev/null" + test_expect_success "notmuch show --format=raw subject:$size > /dev/null" done +add_email_corpus duplicate +ID=87r2ecrr6x.fsf@zephyr.silentflame.com +test_begin_subtest "raw content, duplicate files" +rm -f OUTPUT.raw +for dup in {1..5}; do + notmuch show --format=raw --duplicate=${dup} --format=raw id:${ID} | md5sum | cut -f1 -d' ' >> OUTPUT.raw +done +sort OUTPUT.raw > OUTPUT +notmuch search --output=files id:${ID} | xargs md5sum | cut -f1 -d ' ' | sort > EXPECTED +test_expect_equal_file_nonempty EXPECTED OUTPUT + test_done diff --git a/test/T220-reply.sh b/test/T220-reply.sh index b6d8f42a..120d7138 100755 --- a/test/T220-reply.sh +++ b/test/T220-reply.sh @@ -2,15 +2,14 @@ test_description="\"notmuch reply\" in several variations" . $(dirname "$0")/test-lib.sh || exit 1 -test_begin_subtest "Basic reply" add_message '[from]="Sender "' \ [to]=test_suite@notmuchmail.org \ [subject]=notmuch-reply-test \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="basic reply test"' -output=$(notmuch reply id:${gen_msg_id} 2>&1 && echo OK) -test_expect_equal "$output" "From: Notmuch Test Suite +cat < basic.expected +From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender In-Reply-To: <${gen_msg_id}> @@ -18,7 +17,19 @@ References: <${gen_msg_id}> On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: > basic reply test -OK" +OK +EOF + +test_begin_subtest "Basic reply" +notmuch reply id:${gen_msg_id} >OUTPUT 2>&1 && echo OK >> OUTPUT +test_expect_equal_file basic.expected OUTPUT + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + test_begin_subtest "Basic reply (query=sexp)" + notmuch reply --query=sexp "(id ${gen_msg_id})" >OUTPUT 2>&1 && echo OK >> OUTPUT + test_expect_equal_file basic.expected OUTPUT +fi test_begin_subtest "Multiple recipients" add_message '[from]="Sender "' \ @@ -245,6 +256,26 @@ On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: > From guessing OK" +test_begin_subtest "From guessing: multiple Delivered-To" +add_message '[from]="Sender "' \ + '[to]="Recipient "' \ + '[subject]="From guessing"' \ + '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ + '[body]="From guessing"' \ + '[header]="Delivered-To: test_suite_other@notmuchmail.org +Delivered-To: test_suite@notmuchmail.org"' + +output=$(notmuch reply id:${gen_msg_id} 2>&1 && echo OK) +test_expect_equal "$output" "From: Notmuch Test Suite +Subject: Re: From guessing +To: Sender , Recipient +In-Reply-To: <${gen_msg_id}> +References: <${gen_msg_id}> + +On Tue, 05 Jan 2010 15:43:56 -0000, Sender wrote: +> From guessing +OK" + test_begin_subtest "Reply with RFC 2047-encoded headers" add_message '[subject]="=?iso-8859-1?q?=e0=df=e7?="' \ '[from]="=?utf-8?q?=e2=98=83?= "' \ @@ -267,7 +298,8 @@ On Tue, 05 Jan 2010 15:43:56 -0000, ☃ wrote: OK" test_begin_subtest "Reply with RFC 2047-encoded headers (JSON)" -output=$(echo '{"answer":' && notmuch reply --format=json id:${gen_msg_id} 2>&1 && echo ', "success": "OK"}') +output=$(echo '{"answer":' && notmuch reply --format=json id:${gen_msg_id} 2>&1 | notmuch_json_show_sanitize \ + && echo ', "success": "OK"}') test_expect_equal_json "$output" ' { "answer": { "original": { @@ -281,14 +313,14 @@ test_expect_equal_json "$output" ' "crypto": {}, "date_relative": "2010-01-05", "excluded": false, - "filename": ["'${MAIL_DIR}'/msg-014"], + "filename": ["YYYYY"], "headers": { "Date": "Tue, 05 Jan 2010 15:43:56 +0000", "From": "\u2603 ", "Subject": "\u00e0\u00df\u00e7", "To": "Notmuch Test Suite " }, - "id": "'${gen_msg_id}'", + "id": "XXXXX", "match": false, "tags": [ "inbox", @@ -321,4 +353,40 @@ On Thu, 16 Jun 2016 22:14:41 -0400, Alice wrote: > Note the Cc: and cc: headers. OK" +add_email_corpus duplicate + +ID1=debian/2.6.1.dfsg-4-1-g87ea161@87ea161e851dfb1ea324af00e4ecfccc18875e15 + +test_begin_subtest "format json, --duplicate=2, duplicate key" +output=$(notmuch reply --format=json --duplicate=2 id:${ID1}) +test_json_nodes <<<"$output" "dup:['original']['duplicate']=2" + +test_begin_subtest "format json, subject, --duplicate=1" +output=$(notmuch reply --format=json --duplicate=1 id:${ID1}) +file=$(notmuch search --output=files id:${ID1} | head -n 1) +subject=$(sed -n 's/^Subject: \(.*\)$/\1/p' < $file) +test_json_nodes <<<"$output" "subject:['reply-headers']['Subject']=\"Re: $subject\"" + +test_begin_subtest "format json, subject, --duplicate=2" +output=$(notmuch reply --format=json --duplicate=2 id:${ID1}) +file=$(notmuch search --output=files id:${ID1} | tail -n 1) +subject=$(sed -n 's/^Subject: \(.*\)$/\1/p' < $file) +test_json_nodes <<<"$output" "subject:['reply-headers']['Subject']=\"Re: $subject\"" + +ID2=87r2geywh9.fsf@tethera.net +for dup in {1..2}; do + test_begin_subtest "format json, body, --duplicate=${dup}" + output=$(notmuch reply --format=json --duplicate=${dup} id:${ID2} | \ + $NOTMUCH_PYTHON -B "$NOTMUCH_SRCDIR"/test/json_check_nodes.py "body:['original']['body'][0]['content']" | \ + grep '^# body') + test_expect_equal "$output" "# body ${dup}" +done + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +for dup in {1..5}; do + test_begin_subtest "format json, --duplicate=${dup}, 'duplicate' key" + output=$(notmuch reply --format=json --duplicate=${dup} id:${ID3}) + test_json_nodes <<<"$output" "dup:['original']['duplicate']=${dup}" +done + test_done diff --git a/test/T230-reply-to-sender.sh b/test/T230-reply-to-sender.sh index bbeaa2b9..38fbe96a 100755 --- a/test/T230-reply-to-sender.sh +++ b/test/T230-reply-to-sender.sh @@ -43,7 +43,7 @@ add_message '[from]="Sender "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="Multiple recipients"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -60,7 +60,7 @@ add_message '[from]="Notmuch Test Suite "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="From Us, Multiple TO recipients"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Recipient , Someone Else @@ -78,7 +78,7 @@ add_message '[from]="Sender "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="reply with CC"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -96,7 +96,7 @@ add_message '[from]="Notmuch Test Suite "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="reply with CC"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Recipient @@ -113,7 +113,7 @@ add_message '[from]="Notmuch Test Suite "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="reply with CC"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test Cc: Other Parties @@ -130,7 +130,7 @@ add_message '[from]="Sender "' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="reply from alternate address"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -148,7 +148,7 @@ add_message '[from]="Sender "' \ '[body]="support for reply-to"' \ '[reply-to]="Sender "' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -166,7 +166,7 @@ add_message '[from]="Sender "' \ '[body]="support for reply-to with multiple recipients"' \ '[reply-to]="Sender "' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -184,7 +184,7 @@ add_message '[from]="Sender "' \ '[body]="Un-munging Reply-To"' \ '[reply-to]="Evil Munging List "' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: notmuch-reply-test To: Sender @@ -198,7 +198,7 @@ test_begin_subtest "Message with header of exactly 200 bytes" add_message '[subject]="This subject is exactly 200 bytes in length. Other than its length there is not much of note here. Note that the length of 200 bytes includes the Subject: and Re: prefixes with two spaces"' \ '[date]="Tue, 05 Jan 2010 15:43:56 -0000"' \ '[body]="200-byte header"' -output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) +output=$(notmuch reply --reply-to=sender id:${gen_msg_id}) test_expect_equal "$output" "From: Notmuch Test Suite Subject: Re: This subject is exactly 200 bytes in length. Other than its length there is not much of note here. Note that the length of 200 bytes diff --git a/test/T240-dump-restore.sh b/test/T240-dump-restore.sh index 0870ff92..c3f18839 100755 --- a/test/T240-dump-restore.sh +++ b/test/T240-dump-restore.sh @@ -65,7 +65,7 @@ test_begin_subtest "Accumulate with new tags" test_expect_success \ 'notmuch restore --input=dump.expected && notmuch restore --accumulate --input=dump-ABC_DEF.expected && - notmuch dump > OUTPUT.$test_count && + notmuch dump > OUTPUT.$test_count && notmuch restore --input=dump.expected && test_cmp dump-ABC_DEF.expected OUTPUT.$test_count' @@ -117,6 +117,19 @@ test_begin_subtest "dump -- from:cworth" notmuch dump -- from:cworth > dump-dash-cworth.actual test_expect_equal_file dump-cworth.expected dump-dash-cworth.actual + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + test_begin_subtest "dump --query=sexp -- '(from cworth)'" + notmuch dump --query=sexp -- '(from cworth)' > dump-dash-cworth.actual2 + test_expect_equal_file_nonempty dump-cworth.expected dump-dash-cworth.actual2 + + test_begin_subtest "dump --query=sexp --output=outfile '(from cworth)'" + notmuch dump --output=dump-outfile-cworth.actual2 --query=sexp '(from cworth)' + test_expect_equal_file dump-cworth.expected dump-outfile-cworth.actual2 + +fi + test_begin_subtest "dump --output=outfile from:cworth" notmuch dump --output=dump-outfile-cworth.actual from:cworth test_expect_equal_file dump-cworth.expected dump-outfile-cworth.actual @@ -126,6 +139,7 @@ notmuch dump --output=dump-outfile-dash-inbox.actual -- from:cworth test_expect_equal_file dump-cworth.expected dump-outfile-dash-inbox.actual test_begin_subtest "Check for a safe set of message-ids" +test_subtest_broken_for_installed notmuch search --output=messages from:cworth | sed s/^id:// > EXPECTED notmuch search --output=messages from:cworth | sed s/^id:// |\ $TEST_DIRECTORY/hex-xcode --direction=encode > OUTPUT @@ -233,9 +247,10 @@ notmuch dump --format=batch-tag > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count test_begin_subtest 'format=batch-tag, checking encoded output' +test_subtest_broken_for_installed NOTMUCH_DUMP_TAGS --format=batch-tag -- from:cworth |\ awk "{ print \"+$enc1 +$enc2 +$enc3 -- \" \$5 }" > EXPECTED.$test_count -NOTMUCH_DUMP_TAGS --format=batch-tag -- from:cworth > OUTPUT.$test_count +NOTMUCH_DUMP_TAGS --format=batch-tag -- from:cworth > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count test_begin_subtest 'restoring sane tags' @@ -322,6 +337,7 @@ EOF test_expect_equal_file EXPECTED OUTPUT +backup_database test_begin_subtest 'roundtripping random message-ids and tags' ${TEST_DIRECTORY}/random-corpus --config-path=${NOTMUCH_CONFIG} \ @@ -338,7 +354,7 @@ test_begin_subtest 'roundtripping random message-ids and tags' sort > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count +restore_database test_done -# Note the database is "poisoned" for sup format at this point. diff --git a/test/T300-encoding.sh b/test/T300-encoding.sh index 1e9d2a3d..6fcd10c5 100755 --- a/test/T300-encoding.sh +++ b/test/T300-encoding.sh @@ -31,18 +31,18 @@ test_expect_equal "$output" "thread:0000000000000002 2001-01-05 [1/1] Notmuch test_begin_subtest "RFC 2047 encoded word with spaces" add_message '[subject]="=?utf-8?q?encoded word with spaces?="' -output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize) -test_expect_equal "$output" "thread:0000000000000003 2001-01-05 [1/1] Notmuch Test Suite; encoded word with spaces (inbox unread)" +output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; encoded word with spaces (inbox unread)" test_begin_subtest "RFC 2047 encoded words back to back" add_message '[subject]="=?utf-8?q?encoded-words-back?==?utf-8?q?to-back?="' -output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize) -test_expect_equal "$output" "thread:0000000000000004 2001-01-05 [1/1] Notmuch Test Suite; encoded-words-backto-back (inbox unread)" +output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; encoded-words-backto-back (inbox unread)" test_begin_subtest "RFC 2047 encoded words without space before or after" add_message '[subject]="=?utf-8?q?encoded?=word without=?utf-8?q?space?=" ' -output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_show_sanitize) -test_expect_equal "$output" "thread:0000000000000005 2001-01-05 [1/1] Notmuch Test Suite; encodedword withoutspace (inbox unread)" +output=$(notmuch search id:${gen_msg_id} 2>&1 | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; encodedword withoutspace (inbox unread)" test_begin_subtest "Mislabeled Windows-1252 encoding" add_message '[content-type]="text/plain; charset=iso-8859-1"' \ diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh index 5f74305d..e96c1601 100755 --- a/test/T310-emacs.sh +++ b/test/T310-emacs.sh @@ -2,9 +2,11 @@ test_description="emacs interface" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 EXPECTED=$NOTMUCH_SRCDIR/test/emacs.expected-output +test_require_emacs add_email_corpus # syntax errors in test-lib.el cause mysterious failures @@ -39,13 +41,29 @@ test_emacs '(notmuch-search "tag:inbox") (test-output)' test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox OUTPUT +test_begin_subtest "Functions in search-result-format" +test_emacs '(let + ((notmuch-search-result-format + (quote ((notmuch-test-result-flags . "%s ") + ("date" . "%12s ") + ("count" . "%9s ") + ("authors" . "%-30s ") + ("subject" . "%s ") + ("tags" . "(%s)"))))) + (notmuch-search "tag:inbox") + (notmuch-test-wait) + (test-output))' +test_expect_equal_file $EXPECTED/search-result-format-function OUTPUT + test_begin_subtest "Incremental parsing of search results" -test_emacs "(ad-enable-advice 'notmuch-search-process-filter 'around 'pessimal) - (ad-activate 'notmuch-search-process-filter) - (notmuch-search \"tag:inbox\") - (notmuch-test-wait) - (ad-disable-advice 'notmuch-search-process-filter 'around 'pessimal) - (ad-activate 'notmuch-search-process-filter) +test_emacs "(cl-letf* (((symbol-function 'orig) + (symbol-function 'notmuch-search-process-filter)) + ((symbol-function 'notmuch-search-process-filter) + (lambda (proc string) + (cl-loop for char across string + do (orig proc (char-to-string char)))))) + (notmuch-search \"tag:inbox\") + (notmuch-test-wait)) (test-output)" test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox OUTPUT @@ -112,53 +130,6 @@ test_emacs '(notmuch-search "tag:inbox") (test-output)' test_expect_equal_file $EXPECTED/notmuch-show-thread-maildir-storage OUTPUT -test_begin_subtest "Add tag from search view" -os_x_darwin_thread=$(notmuch search --output=threads id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com) -test_emacs "(notmuch-search \"$os_x_darwin_thread\") - (notmuch-test-wait) - (execute-kbd-macro \"+tag-from-search-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)" - -test_begin_subtest "Remove tag from search view" -test_emacs "(notmuch-search \"$os_x_darwin_thread\") - (notmuch-test-wait) - (execute-kbd-macro \"-tag-from-search-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" - -test_begin_subtest "Add tag (large query)" -# We use a long query to force us into batch mode and use a funny tag -# that requires escaping for batch tagging. -test_emacs "(notmuch-tag (concat \"$os_x_darwin_thread\" \" or \" (mapconcat #'identity (make-list notmuch-tag-argument-limit \"x\") \"-\")) (list \"+tag-from-%-large-query\"))" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-%-large-query unread)" -notmuch tag -tag-from-%-large-query $os_x_darwin_thread - -test_begin_subtest "notmuch-show: add single tag to single message" -test_emacs "(notmuch-show \"$os_x_darwin_thread\") - (execute-kbd-macro \"+tag-from-show-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)" - -test_begin_subtest "notmuch-show: remove single tag from single message" -test_emacs "(notmuch-show \"$os_x_darwin_thread\") - (execute-kbd-macro \"-tag-from-show-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" - -test_begin_subtest "notmuch-show: add multiple tags to single message" -test_emacs "(notmuch-show \"$os_x_darwin_thread\") - (execute-kbd-macro \"+tag1-from-show-view +tag2-from-show-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag1-from-show-view tag2-from-show-view unread)" - -test_begin_subtest "notmuch-show: remove multiple tags from single message" -test_emacs "(notmuch-show \"$os_x_darwin_thread\") - (execute-kbd-macro \"-tag1-from-show-view -tag2-from-show-view\")" -output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" - test_begin_subtest "Message with .. in Message-Id:" add_message [id]=123..456@example '[subject]="Message with .. in Message-Id"' test_emacs '(notmuch-search "id:\"123..456@example\"") @@ -287,6 +258,7 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Verify that sent messages are saved/searchable (via FCC)" +test_subtest_broken_for_installed notmuch new > /dev/null output=$(notmuch search 'subject:"testing message sent via SMTP"' | notmuch_search_sanitize) test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; Testing message sent via SMTP (inbox)" @@ -379,6 +351,7 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Reply within emacs" +test_subtest_broken_for_installed test_emacs '(let ((message-hidden-headers ''())) (notmuch-search "subject:\"testing message sent via SMTP\"") (notmuch-test-wait) @@ -445,6 +418,31 @@ Sender writes: EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Reply with show.extra_headers set" +notmuch config set show.extra_headers Received +add_message '[from]="Sender "' \ + [to]=test_suite_other@notmuchmail.org + +test_emacs "(let ((message-hidden-headers '())) + (notmuch-search \"id:\\\"${gen_msg_id}\\\"\") + (notmuch-test-wait) + (notmuch-search-reply-to-thread) + (test-output))" +cat <EXPECTED +From: Notmuch Test Suite +To: Sender +Subject: Re: ${test_subtest_name} +In-Reply-To: <${gen_msg_id}> +Fcc: ${MAIL_DIR}/sent +References: <${gen_msg_id}> +--text follows this line-- +Sender writes: + +> This is just a test message (#${gen_msg_cnt}) +EOF +notmuch config set show.extra_headers +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "Reply from address in named group list within emacs" add_message '[from]="Sender "' \ '[to]=group:test_suite@notmuchmail.org,someone@example.com\;' \ @@ -640,7 +638,7 @@ References: --text follows this line-- test_suite@notmuchmail.org writes: -> This is just a test message (#7) +> This is just a test message (#${gen_msg_cnt}) EOF test_expect_equal_file EXPECTED OUTPUT @@ -843,7 +841,7 @@ test_emacs '(notmuch-show "id:\"bought\"") (notmuch-show-stash-message-id-stripped) (notmuch-show-stash-tags) (notmuch-show-stash-filename) - (notmuch-show-stash-mlarchive-link "Gmane") + (notmuch-show-stash-mlarchive-link "Notmuch") (notmuch-show-stash-mlarchive-link "MARC") (notmuch-show-stash-mlarchive-link "Mail Archive, The") (switch-to-buffer @@ -864,7 +862,7 @@ id:bought bought inbox,stashtest ${gen_msg_filename} -https://mid.gmane.org/bought +https://nmbug.notmuchmail.org/nmweb/show/bought https://marc.info/?i=bought https://mid.mail-archive.com/bought EOF @@ -1033,9 +1031,8 @@ End of search results. === MESSAGES === YYY/notmuch_fail exited with status 1 (see *Notmuch errors* for more details) === ERROR === -[XXX] YYY/notmuch_fail exited with status 1 -command: YYY/notmuch_fail search --format\=sexp --format-version\=4 --sort\=newest-first tag\:inbox +command: YYY/notmuch_fail search --format\=sexp --format-version\=5 --sort\=newest-first --exclude\=false tag\:inbox exit status: 1" test_begin_subtest "Search handles subprocess warnings" @@ -1069,30 +1066,6 @@ This is a warning (see *Notmuch errors* for more details) This is a warning This is another warning" -test_begin_subtest "Search thread tag operations are race-free" -add_message '[subject]="Search race test"' -gen_msg_id_1=$gen_msg_id -generate_message '[in-reply-to]="<'$gen_msg_id_1'>"' \ - '[references]="<'$gen_msg_id_1'>"' \ - '[subject]="Search race test two"' -test_emacs '(notmuch-search "subject:\"search race test\"") - (notmuch-test-wait) - (notmuch-poll) - (execute-kbd-macro "+search-thread-race-tag")' -output=$(notmuch search --output=messages 'tag:search-thread-race-tag') -test_expect_equal "$output" "id:$gen_msg_id_1" - -test_begin_subtest "Search global tag operations are race-free" -generate_message '[in-reply-to]="<'$gen_msg_id_1'>"' \ - '[references]="<'$gen_msg_id_1'>"' \ - '[subject]="Re: Search race test"' -test_emacs '(notmuch-search "subject:\"search race test\" -subject:two") - (notmuch-test-wait) - (notmuch-poll) - (execute-kbd-macro "*+search-global-race-tag")' -output=$(notmuch search --output=messages 'tag:search-global-race-tag') -test_expect_equal "$output" "id:$gen_msg_id_1" - test_begin_subtest "Term escaping" output=$(test_emacs "(mapcar 'notmuch-escape-boolean-term (list \"\" @@ -1129,4 +1102,10 @@ This text added by the hook. EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "notmuch-search with nonexistent CWD" +test_emacs '(test-log-error + (let ((default-directory "/nonexistent")) + (notmuch-search "*")))' +test_expect_equal "$(cat MESSAGES)" "COMPLETE" + test_done diff --git a/test/T315-emacs-tagging.sh b/test/T315-emacs-tagging.sh new file mode 100755 index 00000000..c26413ce --- /dev/null +++ b/test/T315-emacs-tagging.sh @@ -0,0 +1,166 @@ +#!/usr/bin/env bash + +test_description="emacs interface" +. $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +EXPECTED=$NOTMUCH_SRCDIR/test/emacs.expected-output + +test_require_emacs +add_email_corpus + +test_begin_subtest "Add tag from search view" +os_x_darwin_thread=$(notmuch search --output=threads id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com) +test_emacs "(notmuch-search \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+tag-from-search-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-search-view unread)" + +test_begin_subtest "Remove tag from search view" +test_emacs "(notmuch-search \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"-tag-from-search-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" + +test_begin_subtest "Add tag (large query)" +# We use a long query to force us into batch mode and use a funny tag +# that requires escaping for batch tagging. +test_emacs "(notmuch-tag (concat \"$os_x_darwin_thread\" \" or \" (mapconcat #'identity (make-list notmuch-tag-argument-limit \"x\") \"-\")) (list \"+tag-from-%-large-query\"))" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-%-large-query unread)" +notmuch tag -tag-from-%-large-query $os_x_darwin_thread + +test_begin_subtest "notmuch-show: add single tag to single message" +test_emacs "(notmuch-show \"$os_x_darwin_thread\") + (execute-kbd-macro \"+tag-from-show-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag-from-show-view unread)" + +test_begin_subtest "notmuch-show: remove single tag from single message" +test_emacs "(notmuch-show \"$os_x_darwin_thread\") + (execute-kbd-macro \"-tag-from-show-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" + +test_begin_subtest "notmuch-show: add multiple tags to single message" +test_emacs "(notmuch-show \"$os_x_darwin_thread\") + (execute-kbd-macro \"+tag1-from-show-view +tag2-from-show-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox tag1-from-show-view tag2-from-show-view unread)" + +test_begin_subtest "notmuch-show: remove multiple tags from single message" +test_emacs "(notmuch-show \"$os_x_darwin_thread\") + (execute-kbd-macro \"-tag1-from-show-view -tag2-from-show-view\")" +output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) +test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox unread)" + +test_begin_subtest "notmuch-show: before-tag-hook is run, variables are defined" +output=$(test_emacs '(let ((notmuch-test-tag-hook-output nil) + (notmuch-before-tag-hook (function notmuch-test-tag-hook))) + (notmuch-show "id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com") + (execute-kbd-macro "+activate-hook\n") + (execute-kbd-macro "-activate-hook\n") + notmuch-test-tag-hook-output)') +test_expect_equal "$output" \ +'(("id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com" "-activate-hook") + ("id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com" "+activate-hook"))' + +test_begin_subtest "notmuch-show: after-tag-hook is run, variables are defined" +output=$(test_emacs '(let ((notmuch-test-tag-hook-output nil) + (notmuch-after-tag-hook (function notmuch-test-tag-hook))) + (notmuch-show "id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com") + (execute-kbd-macro "+activate-hook\n") + (execute-kbd-macro "-activate-hook\n") + notmuch-test-tag-hook-output)') +test_expect_equal "$output" \ +'(("id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com" "-activate-hook") + ("id:ddd65cda0911171950o4eea4389v86de9525e46052d3@mail.gmail.com" "+activate-hook"))' + + +test_begin_subtest "Search thread tag operations are race-free" +add_message '[subject]="Search race test"' +gen_msg_id_1=$gen_msg_id +generate_message '[in-reply-to]="<'$gen_msg_id_1'>"' \ + '[references]="<'$gen_msg_id_1'>"' \ + '[subject]="Search race test two"' +test_emacs '(notmuch-search "subject:\"search race test\"") + (notmuch-test-wait) + (notmuch-poll) + (execute-kbd-macro "+search-thread-race-tag")' +output=$(notmuch search --output=messages 'tag:search-thread-race-tag') +test_expect_equal "$output" "id:$gen_msg_id_1" + +test_begin_subtest "Search global tag operations are race-free" +generate_message '[in-reply-to]="<'$gen_msg_id_1'>"' \ + '[references]="<'$gen_msg_id_1'>"' \ + '[subject]="Re: Search race test"' +test_emacs '(notmuch-search "subject:\"search race test\" -subject:two") + (notmuch-test-wait) + (notmuch-poll) + (execute-kbd-macro "*+search-global-race-tag")' +output=$(notmuch search --output=messages 'tag:search-global-race-tag') +test_expect_equal "$output" "id:$gen_msg_id_1" + +test_begin_subtest "undo with empty history is an error" +test_emacs "(let ((notmuch-tag-history nil)) + (test-log-error + (notmuch-tag-undo))) + " +cat < EXPECTED +(error no further notmuch undo information) +EOF +test_expect_equal_file EXPECTED MESSAGES + +for mode in search show tree unthreaded; do + test_begin_subtest "undo tagging in $mode mode" + test_emacs "(let ((notmuch-tag-history nil)) + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+tag-to-be-undone-$mode\") + (notmuch-tag-undo) + (notmuch-test-wait))" + count=$(notmuch count "tag:tag-to-be-undone-$mode") + test_expect_equal "$count" "0" + + test_begin_subtest "undo tagging in $mode mode (multiple operations)" + test_emacs "(let ((notmuch-tag-history nil)) + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+one-$mode\") + (execute-kbd-macro \"+two-$mode\") + (notmuch-tag-undo) + (notmuch-test-wait) + (execute-kbd-macro \"+three-$mode\"))" + output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) + notmuch tag "-one-$mode" "-three-$mode" $os_x_darwin_thread + test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox one-$mode three-$mode unread)" + + test_begin_subtest "undo tagging in $mode mode (multiple undo)" + test_emacs "(let ((notmuch-tag-history nil)) + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+one-$mode\") + (execute-kbd-macro \"+two-$mode\") + (notmuch-tag-undo) + (notmuch-test-wait) + (notmuch-tag-undo) + (notmuch-test-wait) + (execute-kbd-macro \"+three-$mode\"))" + output=$(notmuch search $os_x_darwin_thread | notmuch_search_sanitize) + notmuch tag "-one-$mode" "-three-$mode" $os_x_darwin_thread + test_expect_equal "$output" "thread:XXX 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry; [notmuch] Mac OS X/Darwin compatibility issues (inbox three-$mode unread)" + + test_begin_subtest "undo tagging in $mode mode (via binding)" + test_emacs "(let ((notmuch-tag-history nil)) + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+tag-to-be-undone-$mode\") + (execute-kbd-macro (kbd \"C-x u\")) + (notmuch-test-wait))" + count=$(notmuch count "tag:tag-to-be-undone-$mode") + test_expect_equal "$count" "0" +done + +test_done diff --git a/test/T320-emacs-large-search-buffer.sh b/test/T320-emacs-large-search-buffer.sh index f61e8a97..617985e6 100755 --- a/test/T320-emacs-large-search-buffer.sh +++ b/test/T320-emacs-large-search-buffer.sh @@ -1,11 +1,14 @@ #!/usr/bin/env bash test_description="Emacs with large search results buffer" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 x=xxxxxxxxxx # 10 x=$x$x$x$x$x$x$x$x$x$x # 100 x=$x$x$x$x$x$x$x$x$x # 900 +test_require_emacs + # We generate a long subject here (over 900 bytes) so that the emacs # search results get large quickly. With 30 such messages we should # cross several 4kB page boundaries and see the bug. diff --git a/test/T330-emacs-subject-to-filename.sh b/test/T330-emacs-subject-to-filename.sh index eaf7c980..405b063b 100755 --- a/test/T330-emacs-subject-to-filename.sh +++ b/test/T330-emacs-subject-to-filename.sh @@ -2,6 +2,9 @@ test_description="emacs: mail subject to filename" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs # emacs server can't be started in a child process with $(test_emacs ...) test_emacs '(ignore)' > /dev/null diff --git a/test/T340-maildir-sync.sh b/test/T340-maildir-sync.sh index 7416dd61..a697317f 100755 --- a/test/T340-maildir-sync.sh +++ b/test/T340-maildir-sync.sh @@ -174,7 +174,7 @@ thread:XXX 2001-01-05 [1/1(3)] Notmuch Test Suite; Duplicated message (inbox r test_begin_subtest "Tag changes modify flags of multiple files" notmuch tag -replied subject:"Duplicated message" (cd $MAIL_DIR/cur/; ls duplicated*) > actual -test_expect_equal "$(< actual)" "duplicated-message-another-copy:2,S +test_expect_equal "$(< actual)" "duplicated-message-another-copy:2,S duplicated-message-copy:2,S duplicated-message:2,S" diff --git a/test/T350-crypto.sh b/test/T350-crypto.sh index 0aada4df..27c0e86d 100755 --- a/test/T350-crypto.sh +++ b/test/T350-crypto.sh @@ -6,21 +6,37 @@ test_description='PGP/MIME signature verification and decryption' . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 ################################################## +test_require_emacs add_gnupg_home -test_begin_subtest "emacs delivery of signed message" +test_begin_subtest "emacs delivery of signed message via fcc" test_expect_success \ 'emacs_fcc_message \ "test signed message 001" \ "This is a test signed message." \ "(mml-secure-message-sign)"' +test_begin_subtest "emacs delivery of signed message via fcc and smtp" +emacs_deliver_message \ + 'signed message sent via SMTP' \ + 'This is a test that messages are sent via SMTP' \ + "(add-hook 'message-send-mail-hook (lambda () (sleep-for 1))) + (mml-secure-message-sign)" +msg_file=$(notmuch search --output=files subject:signed-message-sent-via-SMTP) +test_expect_equal_message_body sent_message "$msg_file" + test_begin_subtest "signed part content-type indexing" -output=$(notmuch search mimetype:multipart/signed and mimetype:application/pgp-signature | notmuch_search_sanitize) -test_expect_equal "$output" "thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test signed message 001 (inbox signed)" +test_subtest_broken_for_installed +notmuch search mimetype:multipart/signed and mimetype:application/pgp-signature | notmuch_search_sanitize > OUTPUT +cat <EXPECTED +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test signed message 001 (inbox signed) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; signed message sent via SMTP (inbox signed) +EOF +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "signature verification" output=$(notmuch show --format=json --verify subject:"test signed message 001" \ @@ -33,7 +49,7 @@ expected='[[[{"id": "XXXXX", "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["inbox","signed"], - "crypto": {"signed": {"status": [{ "status": "good", "created": 946728000, "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'"}]}}, + "crypto": {"signed": {"status": [{ "status": "good", "created": 946728000, "email": "'"$SELF_EMAIL"'", "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'"}]}}, "headers": {"Subject": "test signed message 001", "From": "Notmuch Test Suite ", "To": "test_suite@notmuchmail.org", @@ -42,6 +58,7 @@ expected='[[[{"id": "XXXXX", "sigstatus": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "created": 946728000, + "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'"}], "content-type": "multipart/signed", "content": [{"id": 2, @@ -365,7 +382,7 @@ expected='[[[{"id": "XXXXX", "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["encrypted","inbox"], - "crypto": {"signed": {"status": [{ "status": "good", "created": 946728000, "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'"}], + "crypto": {"signed": {"status": [{ "status": "good", "created": 946728000, "fingerprint": "'$FINGERPRINT'", "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'"}], "encrypted": true }, "decrypted": {"status": "full"}}, "headers": {"Subject": "test encrypted message 002", @@ -377,6 +394,7 @@ expected='[[[{"id": "XXXXX", "sigstatus": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "created": 946728000, + "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'"}], "content-type": "multipart/encrypted", "content": [{"id": 2, @@ -450,7 +468,7 @@ expected='[[[{"id": "XXXXX", "Date": "Sat, 01 Jan 2000 12:00:00 +0000"}, "body": [{"id": 1, "sigstatus": [{"status": "error", - "keyid": "6D92612D94E46381", + "keyid": "'$(echo $FINGERPRINT | cut -c 25-)'", "errors": {"key-revoked": true}}], "content-type": "multipart/signed", "content": [{"id": 2, diff --git a/test/T355-smime.sh b/test/T355-smime.sh index 336da917..d2118c04 100755 --- a/test/T355-smime.sh +++ b/test/T355-smime.sh @@ -2,26 +2,13 @@ test_description='S/MIME signature verification and decryption' . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 -add_gpgsm_home () -{ - local fpr - [ -d ${GNUPGHOME} ] && return - _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; } - at_exit_function _gnupg_exit - mkdir -m 0700 "$GNUPGHOME" - gpgsm --no-tty --no-common-certs-import --disable-dirmngr --import < $NOTMUCH_SRCDIR/test/smime/test.crt >"$GNUPGHOME"/import.log 2>&1 - fpr=$(gpgsm --list-key test_suite@notmuchmail.org | sed -n 's/.*fingerprint: //p') - echo "$fpr S relax" >> $GNUPGHOME/trustlist.txt - test_debug "cat $GNUPGHOME/import.log" -} - +test_require_emacs test_require_external_prereq openssl test_require_external_prereq gpgsm -cp $NOTMUCH_SRCDIR/test/smime/key+cert.pem test_suite.pem - -FINGERPRINT=$(openssl x509 -fingerprint -in test_suite.pem -noout | sed -e 's/^.*=//' -e s/://g) +FINGERPRINT=$(openssl x509 -sha1 -fingerprint -in "$NOTMUCH_SRCDIR/test/smime/key+cert.pem" -noout | sed -e 's/^.*=//' -e s/://g) add_gpgsm_home @@ -37,7 +24,7 @@ test_begin_subtest "emacs delivery of S/MIME encrypted + signed message" test_expect_success \ 'emacs_fcc_message \ "test encrypted message 001" \ - "<#secure method=smime mode=signencrypt keyfile=\\\"test_suite.pem\\\" certfile=\\\"test_suite.pem\\\">\nThis is a test encrypted message.\n"' + "<#secure method=smime mode=signencrypt>\nThis is a test encrypted message.\n"' test_begin_subtest "Signature verification (openssl)" notmuch show --format=raw subject:"test signed message 001" |\ @@ -48,6 +35,11 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "signature verification (notmuch CLI)" +if [ $NOTMUCH_GMIME_EMITS_ANGLE_BRACKETS == 1 ]; then + EXPECTED_EMAIL_ADDR='' +else + EXPECTED_EMAIL_ADDR='test_suite@notmuchmail.org' +fi output=$(notmuch show --format=json --verify subject:"test signed message 001" \ | notmuch_json_show_sanitize \ | sed -e 's|"created": [-1234567890]*|"created": 946728000|g' \ @@ -59,7 +51,7 @@ expected='[[[{"id": "XXXXX", "timestamp": 946728000, "date_relative": "2000-01-01", "tags": ["inbox","signed"], - "crypto": {"signed": {"status": [{"fingerprint": "'$FINGERPRINT'", "status": "good","userid": "CN=Notmuch Test Suite","expires": 424242424, "created": 946728000}]}}, + "crypto": {"signed": {"status": [{"fingerprint": "'$FINGERPRINT'", "status": "good","userid": "CN=Notmuch Test Suite", "email": "'$EXPECTED_EMAIL_ADDR'", "expires": 424242424, "created": 946728000}]}}, "headers": {"Subject": "test signed message 001", "From": "Notmuch Test Suite ", "To": "test_suite@notmuchmail.org", @@ -68,6 +60,7 @@ expected='[[[{"id": "XXXXX", "sigstatus": [{"fingerprint": "'$FINGERPRINT'", "status": "good", "userid": "CN=Notmuch Test Suite", + "email": "'$EXPECTED_EMAIL_ADDR'", "expires": 424242424, "created": 946728000}], "content-type": "multipart/signed", @@ -78,7 +71,7 @@ expected='[[[{"id": "XXXXX", "content-disposition": "attachment", "content-length": "NONZERO", "content-transfer-encoding": "base64", - "content-type": "application/x-pkcs7-signature", + "content-type": "application/pkcs7-signature", "filename": "smime.p7s"}]}]}, []]]]' test_expect_equal_json \ @@ -87,11 +80,128 @@ test_expect_equal_json \ test_begin_subtest "Decryption and signature verification (openssl)" notmuch show --format=raw subject:"test encrypted message 001" |\ - openssl smime -decrypt -recip test_suite.pem |\ + openssl smime -decrypt -recip $NOTMUCH_SRCDIR/test/smime/key+cert.pem |\ openssl smime -verify -CAfile $NOTMUCH_SRCDIR/test/smime/test.crt 2>OUTPUT cat < EXPECTED Verification successful EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "Decryption (notmuch CLI)" +notmuch show --decrypt=true subject:"test encrypted message 001" |\ + grep "^This is a" > OUTPUT +cat < EXPECTED +This is a test encrypted message. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Cryptographic message status (encrypted+signed)" +output=$(notmuch show --format=json --decrypt=true subject:"test encrypted message 001") +test_json_nodes <<<"$output" \ + 'crypto_encrypted:[0][0][0]["crypto"]["decrypted"]["status"]="full"' \ + 'crypto_sigok:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \ + 'crypto_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="616F46CD73834C63847756AF0DFB64A6E0972A47"' \ + 'crypto_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Notmuch Test Suite"' + +test_begin_subtest "encrypted+signed message is known to be encrypted, but signature is unknown" +output=$(notmuch search subject:"test encrypted message 001") +test_expect_equal "$output" "thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message 001 (encrypted inbox)" + +test_begin_subtest "Encrypted body is not indexed" +output=$(notmuch search 'this is a test encrypted message') +test_expect_equal "$output" "" + +test_begin_subtest "Reindex cleartext" +test_expect_success "notmuch reindex --decrypt=true subject:'test encrypted message 001'" + +test_begin_subtest "signature is now known" +output=$(notmuch search subject:"test encrypted message 001") +test_expect_equal "$output" "thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)" + +test_begin_subtest "Encrypted body is indexed" +output=$(notmuch search 'this is a test encrypted message') +test_expect_equal "$output" "thread:0000000000000002 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message 001 (encrypted inbox signed)" + +add_email_corpus pkcs7 + +test_begin_subtest "index PKCS#7 SignedData message" +output=$(notmuch search --output=messages Thanks) +expected=id:smime-onepart-signed@protected-headers.example +test_expect_equal "$expected" "$output" + +test_begin_subtest "do not index embedded certificates from PKCS#7 SignedData" +output=$(notmuch search --output=messages 'LAMPS Certificate') +expected='' +test_expect_equal "$expected" "$output" + +test_begin_subtest "know the MIME type of the embedded part in PKCS#7 SignedData" +output=$(notmuch search --output=messages 'mimetype:text/plain') +expected=id:smime-onepart-signed@protected-headers.example +test_expect_equal "$expected" "$output" + +test_begin_subtest "PKCS#7 SignedData message is tagged 'signed'" +output=$(notmuch dump id:smime-onepart-signed@protected-headers.example) +expected='#notmuch-dump batch-tag:3 config,properties,tags ++inbox +signed +unread -- id:smime-onepart-signed@protected-headers.example' +test_expect_equal "$expected" "$output" + +test_begin_subtest "show contents of PKCS#7 SignedData message" +output=$(notmuch show --format=raw --part=2 id:smime-onepart-signed@protected-headers.example) +whitespace=' ' +expected="Bob, we need to cancel this contract. + +Please start the necessary processes to make that happen today. + +Thanks, Alice +--${whitespace} +Alice Lovelace +President +OpenPGP Example Corp" +test_expect_equal "$expected" "$output" + +test_begin_subtest "reply to PKCS#7 SignedData message with proper quoting and attribution" +output=$(notmuch reply id:smime-onepart-signed@protected-headers.example) +expected="From: Notmuch Test Suite +Subject: Re: The FooCorp contract +To: Alice Lovelace , Bob Babbage +In-Reply-To: +References: + +On Tue, 26 Nov 2019 20:11:29 -0400, Alice Lovelace wrote: +> Bob, we need to cancel this contract. +>${whitespace} +> Please start the necessary processes to make that happen today. +>${whitespace} +> Thanks, Alice +> --${whitespace} +> Alice Lovelace +> President +> OpenPGP Example Corp" +test_expect_equal "$expected" "$output" + +test_begin_subtest "show PKCS#7 SignedData outputs valid JSON" +output=$(notmuch show --format=json id:smime-onepart-signed@protected-headers.example) +test_valid_json "$output" + +if [ -z "${NOTMUCH_TEST_INSTALLED-}" ]; then +test_begin_subtest "Verify signature on PKCS#7 SignedData message" +if [ "${NOTMUCH_HAVE_64BIT_TIME_T-0}" != "1" ]; then + test_subtest_known_broken +fi +output=$(notmuch show --format=json id:smime-onepart-signed@protected-headers.example) + +test_json_nodes <<<"$output" \ + 'created:[0][0][0]["crypto"]["signed"]["status"][0]["created"]=1574813489' \ + 'expires:[0][0][0]["crypto"]["signed"]["status"][0]["expires"]=2611032858' \ + 'fingerprint:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"' \ + 'status:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' +fi # NOTMUCH_TEST_INSTALLED undefined / empty + +test_begin_subtest "Verify signature on PKCS#7 SignedData message signer User ID" +if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then + test_subtest_known_broken +fi +test_json_nodes <<<"$output" \ + 'userid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice Lovelace"' + test_done diff --git a/test/T356-protected-headers.sh b/test/T356-protected-headers.sh index 925805df..9f640331 100755 --- a/test/T356-protected-headers.sh +++ b/test/T356-protected-headers.sh @@ -1,14 +1,14 @@ #!/usr/bin/env bash -# TODO: -# * check S/MIME as well as PGP/MIME - test_description='Message decryption with protected headers' . $(dirname "$0")/test-lib.sh || exit 1 ################################################## +test_require_external_prereq gpgsm + add_gnupg_home +add_gpgsm_home add_email_corpus protected-headers @@ -69,12 +69,12 @@ test_json_nodes <<<"$output" \ test_begin_subtest "show cryptographic envelope on signed mail" output=$(notmuch show --verify --format=json id:simple-signed-mail@crypto.notmuchmail.org) test_json_nodes <<<"$output" \ - 'crypto:[0][0][0]["crypto"]={"signed": {"status": [{"created": 1525609971, "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'", "status": "good"}]}}' + 'crypto:[0][0][0]["crypto"]={"signed": {"status": [{"created": 1662554210, "fingerprint": "'$FINGERPRINT'", "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'", "status": "good"}]}}' test_begin_subtest "verify signed protected header" output=$(notmuch show --verify --format=json id:signed-protected-header@crypto.notmuchmail.org) test_json_nodes <<<"$output" \ - 'crypto:[0][0][0]["crypto"]={"signed": {"status": [{"created": 1525350527, "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'", "status": "good"}], "headers": ["Subject"]}}' + 'crypto:[0][0][0]["crypto"]={"signed": {"status": [{"created": 1662554263, "fingerprint": "'$FINGERPRINT'", "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'", "status": "good"}], "headers": ["Subject"]}}' test_begin_subtest "protected subject does not leak by default in replies" output=$(notmuch reply --decrypt=true --format=json id:protected-header@crypto.notmuchmail.org) @@ -115,7 +115,7 @@ test_begin_subtest "verify protected header is both signed and encrypted" output=$(notmuch show --decrypt=true --format=json id:encrypted-signed@crypto.notmuchmail.org) test_json_nodes <<<"$output" \ 'crypto:[0][0][0]["crypto"]={ - "signed":{"status": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'", "created": 1525812676}], + "signed":{"status": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "email": "'"$SELF_EMAIL"'", "userid": "'"$SELF_USERID"'", "created": 1662550328}], "encrypted": true, "headers": ["Subject"]},"decrypted": {"status": "full", "header-mask": {"Subject": "Subject Unavailable"}}}' \ 'subject:[0][0][0]["headers"]["Subject"]="Rhinoceros dinner"' @@ -123,7 +123,7 @@ test_begin_subtest "verify protected header is signed even when not masked" output=$(notmuch show --decrypt=true --format=json id:encrypted-signed-not-masked@crypto.notmuchmail.org) test_json_nodes <<<"$output" \ 'crypto:[0][0][0]["crypto"]={ - "signed":{"status": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'", "created": 1525812676}], + "signed":{"status": [{"status": "good", "fingerprint": "'$FINGERPRINT'", "userid": "'"$SELF_USERID"'", "email": "'"$SELF_EMAIL"'", "created": 1662550328}], "encrypted": true, "headers": ["Subject"]},"decrypted": {"status": "full"}}' \ 'subject:[0][0][0]["headers"]["Subject"]="Rhinoceros dinner"' @@ -155,6 +155,46 @@ test_begin_subtest "identify message that had a legacy display part skipped duri output=$(notmuch search --output=messages property:index.repaired=skip-protected-headers-legacy-display) test_expect_equal "$output" id:protected-with-legacy-display@crypto.notmuchmail.org +for variant in multipart-signed onepart-signed; do + test_begin_subtest "verify signed PKCS#7 subject ($variant)" + output=$(notmuch show --verify --format=json "id:smime-${variant}@protected-headers.example") + test_json_nodes <<<"$output" \ + 'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \ + 'sig_good:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \ + 'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"' \ + 'not_encrypted:[0][0][0]["crypto"]!"decrypted"' + test_begin_subtest "verify signed PKCS#7 subject ($variant) signer User ID" + if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then + test_subtest_known_broken + fi + test_json_nodes <<<"$output" \ + 'sig_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice Lovelace"' +done + +for variant in sign+enc sign+enc+legacy-disp; do + test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant)" + output=$(notmuch show --decrypt=true --format=json "id:smime-${variant}@protected-headers.example") + test_json_nodes <<<"$output" \ + 'signed_subject:[0][0][0]["crypto"]["signed"]["headers"]=["Subject"]' \ + 'sig_good:[0][0][0]["crypto"]["signed"]["status"][0]["status"]="good"' \ + 'sig_fpr:[0][0][0]["crypto"]["signed"]["status"][0]["fingerprint"]="702BA4B157F1E2B7D16B0C6A5FFC8A7DE2057DEB"' \ + 'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}' + test_begin_subtest "confirm signed and encrypted PKCS#7 subject ($variant) signer User ID" + if [ $NOTMUCH_GMIME_X509_CERT_VALIDITY -ne 1 ]; then + test_subtest_known_broken + fi + test_json_nodes <<<"$output" \ + 'sig_uid:[0][0][0]["crypto"]["signed"]["status"][0]["userid"]="CN=Alice Lovelace"' + +done + +test_begin_subtest "confirm encryption-protected PKCS#7 subject (enc+legacy-disp)" +output=$(notmuch show --decrypt=true --format=json "id:smime-enc+legacy-disp@protected-headers.example") +test_json_nodes <<<"$output" \ + 'encrypted:[0][0][0]["crypto"]["decrypted"]={"status":"full","header-mask":{"Subject":"..."}}' \ + 'no_sig:[0][0][0]["crypto"]!"signed"' + + # TODO: test that a part that looks like a legacy-display in # multipart/signed, but not encrypted, is indexed and not stripped. diff --git a/test/T357-index-decryption.sh b/test/T357-index-decryption.sh index 1ac2836a..a7497489 100755 --- a/test/T357-index-decryption.sh +++ b/test/T357-index-decryption.sh @@ -4,9 +4,11 @@ test_description='indexing decrypted mail' . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 ################################################## +test_require_emacs add_gnupg_home # create a test encrypted message @@ -112,12 +114,10 @@ test_expect_equal \ "$expected" # try inserting it with decryption, should appear as a single copy -# (note: i think thread id skips 4 because of duplicate message-id -# insertion, above) test_begin_subtest "message cleartext is present with insert --decrypt=true" notmuch insert --folder=sent --decrypt=true <<<"$contents" -output=$(notmuch search wumpus) -expected='thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (encrypted inbox unread)' +output=$(notmuch search wumpus | notmuch_search_sanitize) +expected='thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (encrypted inbox unread)' test_expect_equal \ "$output" \ "$expected" @@ -127,9 +127,9 @@ test_expect_equal \ test_begin_subtest 'tagging all messages' test_expect_success 'notmuch tag +blarney "encrypted message"' test_begin_subtest "verify that tags have not changed" -output=$(notmuch search tag:blarney) -expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) -thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)' +output=$(notmuch search tag:blarney | notmuch_search_sanitize) +expected='thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)' test_expect_equal \ "$output" \ "$expected" @@ -138,14 +138,14 @@ test_expect_equal \ test_begin_subtest 'reindex old messages' test_expect_success 'notmuch reindex --decrypt=true tag:encrypted and not property:index.decryption=success' test_begin_subtest "reindexed encrypted message, including cleartext" -output=$(notmuch search wumpus) +output=$(notmuch search wumpus | notmuch_search_sanitize) test_expect_equal \ "$output" \ "$expected" # and the same search, but by property ($expected is untouched): test_begin_subtest "emacs search by property for both messages" -output=$(notmuch search property:index.decryption=success) +output=$(notmuch search property:index.decryption=success | notmuch_search_sanitize) test_expect_equal \ "$output" \ "$expected" @@ -154,7 +154,7 @@ test_expect_equal \ test_begin_subtest 'reindex in auto mode' test_expect_success 'notmuch reindex tag:encrypted and property:index.decryption=success' test_begin_subtest "reindexed encrypted messages, should not have changed" -output=$(notmuch search wumpus) +output=$(notmuch search wumpus | notmuch_search_sanitize) test_expect_equal \ "$output" \ "$expected" @@ -188,9 +188,9 @@ test_expect_equal \ # ensure that the tags remain even when we are dropping the cleartext. test_begin_subtest "verify that tags remain without cleartext" -output=$(notmuch search tag:blarney) -expected='thread:0000000000000001 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) -thread:0000000000000005 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)' +output=$(notmuch search tag:blarney | notmuch_search_sanitize) +expected='thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 001 (blarney encrypted inbox) +thread:XXX 2000-01-01 [1/1] Notmuch Test Suite; test encrypted message for cleartext index 002 (blarney encrypted inbox unread)' test_expect_equal \ "$output" \ "$expected" @@ -199,7 +199,7 @@ test_begin_subtest "index cleartext without keeping session keys" test_expect_success "notmuch reindex --decrypt=nostash tag:blarney" test_begin_subtest "Ensure that the indexed terms are present" -output=$(notmuch search wumpus) +output=$(notmuch search wumpus | notmuch_search_sanitize) test_expect_equal \ "$output" \ "$expected" @@ -226,6 +226,7 @@ output=$(notmuch dump | LC_ALL=C sort) expected='#= simple-encrypted@crypto.notmuchmail.org index.decryption=failure #notmuch-dump batch-tag:3 config,properties,tags +encrypted +inbox +unread -- id:basic-encrypted@crypto.notmuchmail.org ++encrypted +inbox +unread -- id:encrypted-rfc822-attachment@crypto.notmuchmail.org +encrypted +inbox +unread -- id:encrypted-signed@crypto.notmuchmail.org +encrypted +inbox +unread -- id:simple-encrypted@crypto.notmuchmail.org' test_expect_equal \ @@ -306,6 +307,9 @@ test_json_nodes <<<"$output" "$goodsig" test_begin_subtest "verify signature with stashed session key" output=$(notmuch show --format=json id:encrypted-signed@crypto.notmuchmail.org) +if [ $NOTMUCH_GMIME_VERIFY_WITH_SESSION_KEY -ne 1 ]; then + test_subtest_known_broken +fi test_json_nodes <<<"$output" "$goodsig" # TODO: test removal of a message from the message store between diff --git a/test/T358-emacs-protected-headers.sh b/test/T358-emacs-protected-headers.sh index bca78531..96e42bf3 100755 --- a/test/T358-emacs-protected-headers.sh +++ b/test/T358-emacs-protected-headers.sh @@ -2,8 +2,10 @@ test_description="protected headers in emacs interface" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 # testing protected headers with emacs +test_require_emacs add_gnupg_home add_email_corpus protected-headers diff --git a/test/T360-symbol-hiding.sh b/test/T360-symbol-hiding.sh index 43921cb4..ff06ff69 100755 --- a/test/T360-symbol-hiding.sh +++ b/test/T360-symbol-hiding.sh @@ -11,14 +11,18 @@ test_description='exception symbol hiding' . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + test_begin_subtest 'running test' run_test mkdir -p ${PWD}/fakedb/.notmuch $TEST_DIRECTORY/symbol-test ${PWD}/fakedb ${PWD}/nonexistent 2>&1 \ - | notmuch_dir_sanitize | sed -e "s,\`,\',g" -e "s,${NOTMUCH_DEFAULT_XAPIAN_BACKEND},backend,g" > OUTPUT + | notmuch_dir_sanitize | sed -e "s,\`,\',g" -e "s,No [^[:space:]]* database,No XXXXXX database,g" > OUTPUT cat < EXPECTED -A Xapian exception occurred opening database: Couldn't stat 'CWD/fakedb/.notmuch/xapian' -caught No backend database found at path 'CWD/nonexistent' +Cannot open Xapian database at CWD/fakedb/.notmuch/xapian: Couldn't stat 'CWD/fakedb/.notmuch/xapian' +caught No XXXXXX database found at path 'CWD/nonexistent' EOF test_expect_equal_file EXPECTED OUTPUT @@ -26,8 +30,9 @@ test_begin_subtest 'checking output' test_expect_equal "$result" "$output" test_begin_subtest 'comparing existing to exported symbols' -nm -P $NOTMUCH_BUILDDIR/lib/libnotmuch.so | awk '$2 == "T" && $1 ~ "^notmuch" {print $1}' | sort | uniq > ACTUAL -sed -n 's/^\(notmuch_[a-zA-Z0-9_]*\)[[:blank:]]*(.*/\1/p' $NOTMUCH_SRCDIR/lib/notmuch.h | sort | uniq > EXPORTED +readelf -Ws $NOTMUCH_BUILDDIR/lib/libnotmuch.so | sed -e 's/\[[^]]*\]//' |\ + awk '$4 == "FUNC" && $5 == "GLOBAL" && $7 != "UND" {print $8}' | sort -u > ACTUAL +sed -n 's/^\(notmuch_[a-zA-Z0-9_]*\)[[:blank:]]*(.*/\1/p' $NOTMUCH_SRCDIR/lib/notmuch.h | sort -u > EXPORTED test_expect_equal_file EXPORTED ACTUAL test_done diff --git a/test/T370-search-folder-coherence.sh b/test/T370-search-folder-coherence.sh index 0a2727e7..cf202bb3 100755 --- a/test/T370-search-folder-coherence.sh +++ b/test/T370-search-folder-coherence.sh @@ -24,8 +24,8 @@ test_expect_equal "$output" "No new mail." test_begin_subtest "Multiple files for same message" cat <EXPECTED -MAIL_DIR/msg-001 -MAIL_DIR/spam/msg-001 +MAIL_DIR/msg-XXX +MAIL_DIR/spam/msg-XXX EOF notmuch search --output=files id:$id_x | notmuch_search_files_sanitize >OUTPUT test_expect_equal_file EXPECTED OUTPUT diff --git a/test/T380-atomicity.sh b/test/T380-atomicity.sh index 45de2228..0f9e6d2e 100755 --- a/test/T380-atomicity.sh +++ b/test/T380-atomicity.sh @@ -67,12 +67,12 @@ if test_require_external_prereq gdb; then ${TEST_GDB} -tty /dev/null -batch -x $NOTMUCH_SRCDIR/test/atomicity.py notmuch 1>gdb.out 2>&1 # Get the final, golden output - notmuch search '*' > expected + notmuch search '*' 2>/dev/null > expected # Check output against golden output outcount=$(cat outcount) - echo -n > searchall - echo -n > expectall + : > searchall + : > expectall for ((i = 0; i < $outcount; i++)); do if ! cmp -s search.$i expected; then # Find the range of interruptions that match this output diff --git a/test/T385-transactions.sh b/test/T385-transactions.sh new file mode 100755 index 00000000..d8bb502d --- /dev/null +++ b/test/T385-transactions.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env bash +test_description='transactions' +. $(dirname "$0")/test-lib.sh || exit 1 + +make_shim no-close < +#include +notmuch_status_t +notmuch_database_close (notmuch_database_t *notmuch) +{ + return notmuch_database_begin_atomic (notmuch); +} +EOF + +for i in `seq 1 1024` +do + generate_message '[subject]="'"subject $i"'"' \ + '[body]="'"body $i"'"' +done + +test_begin_subtest "initial new" +NOTMUCH_NEW > OUTPUT +cat < EXPECTED +Added 1024 new messages to the database. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Some changes saved with open transaction" +notmuch config set database.autocommit 1000 +rm -r ${MAIL_DIR}/.notmuch +notmuch_with_shim no-close new +output=$(notmuch count '*') +test_expect_equal "$output" "1000" + +test_done diff --git a/test/T390-python.sh b/test/T390-python.sh index 9f71ce3c..21912431 100755 --- a/test/T390-python.sh +++ b/test/T390-python.sh @@ -4,6 +4,10 @@ test_description="python bindings" test_require_external_prereq ${NOTMUCH_PYTHON} +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + add_email_corpus add_gnupg_home diff --git a/test/T391-python-cffi.sh b/test/T391-python-cffi.sh new file mode 100755 index 00000000..0059b050 --- /dev/null +++ b/test/T391-python-cffi.sh @@ -0,0 +1,23 @@ +#!/usr/bin/env bash +test_description="python bindings (pytest)" +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + +if [ "${NOTMUCH_HAVE_PYTHON3_CFFI-0}" = "0" -o "${NOTMUCH_HAVE_PYTHON3_PYTEST-0}" = "0" ]; then + test_done +fi + +test_begin_subtest "python cffi tests (NOTMUCH_CONFIG set)" +pytest_dir=$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage +printf "[pytest]\nminversion = 3.0\naddopts = -ra\n" > $pytest_dir/pytest.ini +test_expect_success "(cd $pytest_dir && ${NOTMUCH_PYTHON} -m pytest --verbose --log-file=$TMP_DIRECTORY/test.output)" + +test_begin_subtest "python cffi tests (NOTMUCH_CONFIG unset)" +pytest_dir=$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage +printf "[pytest]\nminversion = 3.0\naddopts = -ra\n" > $pytest_dir/pytest.ini +unset NOTMUCH_CONFIG +test_expect_success "(cd $pytest_dir && ${NOTMUCH_PYTHON} -m pytest --verbose --log-file=$TMP_DIRECTORY/test.output)" +test_done diff --git a/test/T392-python-cffi-notmuch.sh b/test/T392-python-cffi-notmuch.sh new file mode 100755 index 00000000..06161219 --- /dev/null +++ b/test/T392-python-cffi-notmuch.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +test_description="python bindings (notmuch test suite)" +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_PYTHON3_CFFI-0}" = "0" -o "${NOTMUCH_HAVE_PYTHON3_PYTEST-0}" = "0" ]; then + test_done +fi + +add_email_corpus + +cat < recurse.py +from notmuch2 import Database +def show_msgs(msgs, level): + print('{:s} {:s}'.format(' ' * level*4, type(msgs).__name__)) + for msg in msgs: + print('{:s} {:s}'.format(' ' * (level*4+2), type(msg).__name__)) + replies=msg.replies() + show_msgs(replies, level+1) +db = Database(config=Database.CONFIG.SEARCH) +msg=db.find("87ocn0qh6d.fsf@yoom.home.cworth.org") +threads = db.threads(query="thread:"+msg.threadid) +thread = next (threads) +show_msgs(thread, 0) +EOF + +test_begin_subtest "recursive traversal of replies (no crash)" +test_python < recurse.py +error=$? +test_expect_equal "${error}" 0 + +test_begin_subtest "recursive traversal of replies (output)" +test_python < recurse.py +tail -n 10 < OUTPUT > OUTPUT.sample +cat < EXPECTED + OwnedMessage + MessageIter + OwnedMessage + MessageIter + OwnedMessage + MessageIter + OwnedMessage + MessageIter + OwnedMessage + MessageIter +EOF +test_expect_equal_file EXPECTED OUTPUT.sample + +test_done diff --git a/test/T395-ruby.sh b/test/T395-ruby.sh index a0b76eb8..d0c6bb17 100755 --- a/test/T395-ruby.sh +++ b/test/T395-ruby.sh @@ -2,101 +2,106 @@ test_description="ruby bindings" . $(dirname "$0")/test-lib.sh || exit 1 -if [ "${NOTMUCH_HAVE_RUBY_DEV}" = "0" ]; then +if [ -z "${NOTMUCH_TEST_INSTALLED-}" -a "${NOTMUCH_HAVE_RUBY_DEV-0}" = "0" ]; then test_subtest_missing_external_prereq_["ruby development files"]=t fi add_email_corpus +test_ruby() { + ( + cat <<-EOF + require 'notmuch' + db = Notmuch::Database.new() + EOF + cat + ) | if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + ruby + else + $NOTMUCH_RUBY -I "$NOTMUCH_BUILDDIR/bindings/ruby" + fi> OUTPUT + test_expect_equal_file EXPECTED OUTPUT +} + test_begin_subtest "compare thread ids" +notmuch search --sort=oldest-first --output=threads tag:inbox > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') -end -@db = Notmuch::Database.new($maildir) -@q = @db.query('tag:inbox') -@q.sort = Notmuch::SORT_OLDEST_FIRST -for t in @q.search_threads do - print t.thread_id, "\n" +q = db.query('tag:inbox') +q.sort = Notmuch::SORT_OLDEST_FIRST +q.search_threads.each do |t| + puts 'thread:%s' % t.thread_id end EOF -notmuch search --sort=oldest-first --output=threads tag:inbox | sed s/^thread:// > EXPECTED -test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "compare message ids" +notmuch search --sort=oldest-first --output=messages tag:inbox > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') -end -@db = Notmuch::Database.new($maildir) -@q = @db.query('tag:inbox') -@q.sort = Notmuch::SORT_OLDEST_FIRST -for m in @q.search_messages do - print m.message_id, "\n" +q = db.query('tag:inbox') +q.sort = Notmuch::SORT_OLDEST_FIRST +q.search_messages.each do |m| + puts 'id:%s' % m.message_id end EOF -notmuch search --sort=oldest-first --output=messages tag:inbox | sed s/^id:// > EXPECTED -test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "get non-existent file" +echo nil > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') -end -@db = Notmuch::Database.new($maildir) -result = @db.find_message_by_filename('i-dont-exist') -print (result == nil) +p db.find_message_by_filename('i-dont-exist') EOF -test_expect_equal "$(cat OUTPUT)" "true" test_begin_subtest "count messages" +notmuch count --output=messages tag:inbox > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') -end -@db = Notmuch::Database.new($maildir) -@q = @db.query('tag:inbox') -print @q.count_messages(),"\n" +puts db.query('tag:inbox').count_messages() EOF -notmuch count --output=messages tag:inbox > EXPECTED -test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "count threads" +notmuch count --output=threads tag:inbox > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') -end -@db = Notmuch::Database.new($maildir) -@q = @db.query('tag:inbox') -print @q.count_threads(),"\n" +puts db.query('tag:inbox').count_threads() EOF -notmuch count --output=threads tag:inbox > EXPECTED -test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "get all tags" +notmuch search --output=tags '*' > EXPECTED +test_ruby <<"EOF" +db.all_tags.each do |tag| + puts tag +end +EOF + +notmuch config set search.exclude_tags deleted +generate_message '[subject]="Good"' +generate_message '[subject]="Bad"' "[in-reply-to]=\<$gen_msg_id\>" +notmuch new > /dev/null +notmuch tag +deleted id:$gen_msg_id + +test_begin_subtest "omit excluded all" +notmuch search --output=threads --exclude=all tag:inbox > EXPECTED +test_ruby <<"EOF" +q = db.query('tag:inbox') +q.add_tag_exclude('deleted') +q.omit_excluded = Notmuch::EXCLUDE_ALL +q.search_threads.each do |t| + puts 'thread:%s' % t.thread_id +end +EOF + +test_begin_subtest "check sort argument" +notmuch search --sort=oldest-first --output=threads tag:inbox > EXPECTED test_ruby <<"EOF" -require 'notmuch' -$maildir = ENV['MAIL_DIR'] -if not $maildir then - abort('environment variable MAIL_DIR must be set') +q = db.query('tag:inbox', sort: Notmuch::SORT_OLDEST_FIRST) +q.search_threads.each do |t| + puts 'thread:%s' % t.thread_id end -@db = Notmuch::Database.new($maildir) -@t = @db.all_tags() -for tag in @t do - print tag,"\n" +EOF + +test_begin_subtest "check exclude_tags argument" +notmuch search --output=threads --exclude=all tag:inbox > EXPECTED +test_ruby <<"EOF" +q = db.query('tag:inbox', exclude_tags: %w[deleted], omit_excluded: Notmuch::EXCLUDE_ALL) +q.search_threads.each do |t| + puts 'thread:%s' % t.thread_id end EOF -notmuch search --output=tags '*' > EXPECTED -test_expect_equal_file EXPECTED OUTPUT test_done diff --git a/test/T400-hooks.sh b/test/T400-hooks.sh index 49c690eb..35bf70c0 100755 --- a/test/T400-hooks.sh +++ b/test/T400-hooks.sh @@ -2,7 +2,7 @@ test_description='hooks' . $(dirname "$0")/test-lib.sh || exit 1 -HOOK_DIR=${MAIL_DIR}/.notmuch/hooks +test_require_external_prereq xapian-delve create_echo_hook () { local TOKEN="${RANDOM}" @@ -15,17 +15,46 @@ EOF echo "${TOKEN}" > ${2} } -create_failing_hook () { +create_printenv_hook () { mkdir -p ${HOOK_DIR} cat <"${HOOK_DIR}/${1}" #!/bin/sh -exit 13 +printenv "${2}" > "${3}" EOF chmod +x "${HOOK_DIR}/${1}" } -rm_hooks () { - rm -rf ${HOOK_DIR} +create_write_hook () { + local TOKEN="${RANDOM}" + mkdir -p ${HOOK_DIR} + cat <"${HOOK_DIR}/${1}" +#!/bin/sh +if xapian-delve ${MAIL_DIR}/.notmuch/xapian | grep -q "writing = false"; then + echo "${TOKEN}" > ${3} +fi +EOF + chmod +x "${HOOK_DIR}/${1}" + echo "${TOKEN}" > ${2} +} + +create_change_hook () { + mkdir -p ${HOOK_DIR} + cat <"${HOOK_DIR}/${1}" +#!/bin/sh +notmuch insert --no-hooks < ${2} > /dev/null +rm -f ${2} +EOF + chmod +x "${HOOK_DIR}/${1}" +} + +create_failing_hook () { + local HOOK_DIR=${2} + mkdir -p ${HOOK_DIR} + cat <"${HOOK_DIR}/${1}" +#!/bin/sh +exit 13 +EOF + chmod +x "${HOOK_DIR}/${1}" } # add a message to generate mail dir and database @@ -33,89 +62,175 @@ add_message # create maildir structure for notmuch-insert mkdir -p "$MAIL_DIR"/{cur,new,tmp} -test_begin_subtest "pre-new is run" -rm_hooks -generate_message -create_echo_hook "pre-new" expected output -notmuch new > /dev/null -test_expect_equal_file expected output - -test_begin_subtest "post-new is run" -rm_hooks -generate_message -create_echo_hook "post-new" expected output -notmuch new > /dev/null -test_expect_equal_file expected output - -test_begin_subtest "post-insert hook is run" -rm_hooks -generate_message -create_echo_hook "post-insert" expected output -notmuch insert < "$gen_msg_filename" -test_expect_equal_file expected output - -test_begin_subtest "pre-new is run before post-new" -rm_hooks -generate_message -create_echo_hook "pre-new" pre-new.expected pre-new.output -create_echo_hook "post-new" post-new.expected post-new.output -notmuch new > /dev/null -test_expect_equal_file post-new.expected post-new.output - -test_begin_subtest "pre-new non-zero exit status (hook status)" -rm_hooks -generate_message -create_failing_hook "pre-new" -output=`notmuch new 2>&1` -test_expect_equal "$output" "Error: pre-new hook failed with status 13" - -# depends on the previous subtest leaving broken hook behind -test_begin_subtest "pre-new non-zero exit status (notmuch status)" -test_expect_code 1 "notmuch new" - -# depends on the previous subtests leaving 1 new message behind -test_begin_subtest "pre-new non-zero exit status aborts new" -rm_hooks -output=$(NOTMUCH_NEW) -test_expect_equal "$output" "Added 1 new message to the database." - -test_begin_subtest "post-new non-zero exit status (hook status)" -rm_hooks -generate_message -create_failing_hook "post-new" -NOTMUCH_NEW 2>output.stderr >output -cat output.stderr >> output -echo "Added 1 new message to the database." > expected -echo "Error: post-new hook failed with status 13" >> expected -test_expect_equal_file expected output - -# depends on the previous subtest leaving broken hook behind -test_begin_subtest "post-new non-zero exit status (notmuch status)" -test_expect_code 1 "notmuch new" - -test_begin_subtest "post-insert hook does not affect insert status" -rm_hooks -generate_message -create_failing_hook "post-insert" -test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null" - -test_begin_subtest "hook without executable permissions" -rm_hooks -mkdir -p ${HOOK_DIR} -cat <"${HOOK_DIR}/pre-new" -#!/bin/sh -echo foo +ORIG_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +for config in traditional profile explicit relative XDG split; do + unset NOTMUCH_PROFILE + export NOTMUCH_CONFIG=${ORIG_NOTMUCH_CONFIG} + EXPECTED_CONFIG=${NOTMUCH_CONFIG} + notmuch config set database.hook_dir + notmuch config set database.path ${MAIL_DIR} + case $config in + traditional) + HOOK_DIR=${MAIL_DIR}/.notmuch/hooks + ;; + profile) + dir=${HOME}/.config/notmuch/other + mkdir -p ${dir} + HOOK_DIR=${dir}/hooks + EXPECTED_CONFIG=${dir}/config + cp ${NOTMUCH_CONFIG} ${EXPECTED_CONFIG} + export NOTMUCH_PROFILE=other + unset NOTMUCH_CONFIG + ;; + explicit) + HOOK_DIR=${HOME}/.notmuch-hooks + mkdir -p $HOOK_DIR + notmuch config set database.hook_dir $HOOK_DIR + ;; + relative) + HOOK_DIR=${HOME}/.notmuch-hooks + mkdir -p $HOOK_DIR + notmuch config set database.hook_dir .notmuch-hooks + ;; + XDG) + HOOK_DIR=${HOME}/.config/notmuch/default/hooks + ;; + split) + dir="$TMP_DIRECTORY/database.$test_count" + notmuch config set database.path $dir + notmuch config set database.mail_root $MAIL_DIR + HOOK_DIR=${dir}/hooks + ;; + esac + + test_begin_subtest "pre-new is run [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_echo_hook "pre-new" expected output $HOOK_DIR + notmuch new > /dev/null + test_expect_equal_file expected output + + test_begin_subtest "post-new is run [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_echo_hook "post-new" expected output $HOOK_DIR + notmuch new > /dev/null + test_expect_equal_file expected output + + test_begin_subtest "post-insert hook is run [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_echo_hook "post-insert" expected output $HOOK_DIR + notmuch insert < "$gen_msg_filename" + test_expect_equal_file expected output + + test_begin_subtest "pre-new is run before post-new [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_echo_hook "pre-new" pre-new.expected pre-new.output $HOOK_DIR + create_echo_hook "post-new" post-new.expected post-new.output $HOOK_DIR + notmuch new > /dev/null + test_expect_equal_file post-new.expected post-new.output + + test_begin_subtest "pre-new non-zero exit status (hook status) [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_failing_hook "pre-new" $HOOK_DIR + output=`notmuch new 2>&1` + test_expect_equal "$output" "Error: pre-new hook failed with status 13" + + # depends on the previous subtest leaving broken hook behind + test_begin_subtest "pre-new non-zero exit status (notmuch status) [${config}]" + test_expect_code 1 "notmuch new" + + # depends on the previous subtests leaving 1 new message behind + test_begin_subtest "pre-new non-zero exit status aborts new [${config}]" + rm -rf ${HOOK_DIR} + output=$(NOTMUCH_NEW) + test_expect_equal "$output" "Added 1 new message to the database." + + test_begin_subtest "post-new non-zero exit status (hook status) [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_failing_hook "post-new" $HOOK_DIR + NOTMUCH_NEW 2>output.stderr >output + cat output.stderr >> output + echo "Added 1 new message to the database." > expected + echo "Error: post-new hook failed with status 13" >> expected + test_expect_equal_file expected output + + # depends on the previous subtest leaving broken hook behind + test_begin_subtest "post-new non-zero exit status (notmuch status) [${config}]" + test_expect_code 1 "notmuch new" + + test_begin_subtest "post-insert hook does not affect insert status [${config}]" + rm -rf ${HOOK_DIR} + generate_message + create_failing_hook "post-insert" $HOOK_DIR + test_expect_success "notmuch insert < \"$gen_msg_filename\" > /dev/null" + + test_begin_subtest "hook without executable permissions [${config}]" + rm -rf ${HOOK_DIR} + mkdir -p ${HOOK_DIR} + cat <"${HOOK_DIR}/pre-new" + #!/bin/sh + echo foo EOF -output=`notmuch new 2>&1` -test_expect_code 1 "notmuch new" - -test_begin_subtest "hook execution failure" -rm_hooks -mkdir -p ${HOOK_DIR} -cat <"${HOOK_DIR}/pre-new" -no hashbang, execl fails + output=`notmuch new 2>&1` + test_expect_code 1 "notmuch new" + + test_begin_subtest "hook execution failure [${config}]" + rm -rf ${HOOK_DIR} + mkdir -p ${HOOK_DIR} + cat <"${HOOK_DIR}/pre-new" + no hashbang, execl fails +EOF + chmod +x "${HOOK_DIR}/pre-new" + test_expect_code 1 "notmuch new" + + test_begin_subtest "post-new with write access [${config}]" + rm -rf ${HOOK_DIR} + create_write_hook "post-new" write.expected write.output $HOOK_DIR + NOTMUCH_NEW + test_expect_equal_file write.expected write.output + + test_begin_subtest "pre-new with write access [${config}]" + rm -rf ${HOOK_DIR} + create_write_hook "pre-new" write.expected write.output $HOOK_DIR + NOTMUCH_NEW + test_expect_equal_file write.expected write.output + + test_begin_subtest "add message in pre-new [${config}]" + rm -rf ${HOOK_DIR} + generate_message '[subject]="add msg in pre-new"' + id1=$gen_msg_id + create_change_hook "pre-new" $gen_msg_filename $HOOK_DIR + generate_message '[subject]="add msg in new"' + NOTMUCH_NEW + notmuch search id:$id1 or id:$gen_msg_id | notmuch_search_sanitize > OUTPUT + cat < EXPECTED + thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; add msg in pre-new (inbox unread) + thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; add msg in new (inbox unread) EOF -chmod +x "${HOOK_DIR}/pre-new" -test_expect_code 1 "notmuch new" + test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "NOTMUCH_CONFIG is set" + create_printenv_hook "pre-new" NOTMUCH_CONFIG OUTPUT + NOTMUCH_NEW + cat < EXPECTED +${EXPECTED_CONFIG} +EOF + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "NOTMUCH_CONFIG is set by --config" + create_printenv_hook "pre-new" NOTMUCH_CONFIG OUTPUT + cp "${EXPECTED_CONFIG}" "${EXPECTED_CONFIG}.alternate" + notmuch --config "${EXPECTED_CONFIG}.alternate" new + cat < EXPECTED +${EXPECTED_CONFIG}.alternate +EOF + test_expect_equal_file_nonempty EXPECTED OUTPUT + + rm -rf ${HOOK_DIR} +done test_done diff --git a/test/T405-external.sh b/test/T405-external.sh new file mode 100755 index 00000000..0e1d9646 --- /dev/null +++ b/test/T405-external.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +test_description='hooks' +. $(dirname "$0")/test-lib.sh || exit 1 + +create_echo_script () { + local TOKEN="${RANDOM}" + mkdir -p ${BIN_DIR} + cat <"${BIN_DIR}/${1}" +#!/bin/sh +echo "${TOKEN}" > ${3} +EOF + chmod +x "${BIN_DIR}/${1}" + echo "${TOKEN}" > ${2} +} + +create_printenv_script () { + mkdir -p ${BIN_DIR} + cat <"${BIN_DIR}/${1}" +#!/bin/sh +printenv "${2}" > "${3}" +EOF + chmod +x "${BIN_DIR}/${1}" +} + +# add a message to generate mail dir and database +add_message + +BIN_DIR=`pwd`/bin +PATH=$BIN_DIR:$PATH + +test_begin_subtest "'notmuch foo' runs notmuch-foo" +rm -rf ${BIN_DIR} +create_echo_script "notmuch-foo" EXPECTED OUTPUT $HOOK_DIR +notmuch foo +test_expect_equal_file_nonempty EXPECTED OUTPUT + +create_printenv_script "notmuch-printenv" NOTMUCH_CONFIG OUTPUT + +test_begin_subtest "NOTMUCH_CONFIG is set" +notmuch printenv +cat < EXPECTED +${NOTMUCH_CONFIG} +EOF +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "NOTMUCH_CONFIG is set by --config" +cp "${NOTMUCH_CONFIG}" "${NOTMUCH_CONFIG}.alternate" +cat < EXPECTED +${NOTMUCH_CONFIG}.alternate +EOF +notmuch --config "${NOTMUCH_CONFIG}.alternate" printenv +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_done diff --git a/test/T410-argument-parsing.sh b/test/T410-argument-parsing.sh index b31d239a..40b625fe 100755 --- a/test/T410-argument-parsing.sh +++ b/test/T410-argument-parsing.sh @@ -2,8 +2,12 @@ test_description="argument parsing" . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + test_begin_subtest "sanity check" -$TEST_DIRECTORY/arg-test pos1 --keyword=one --boolean --string=foo pos2 --int=7 --flag=one --flag=three > OUTPUT +$TEST_DIRECTORY/arg-test pos1 --keyword=one --boolean --string=foo pos2 --int=7 --flag=one --flag=three > OUTPUT cat < EXPECTED boolean 1 keyword 1 diff --git a/test/T420-emacs-test-functions.sh b/test/T420-emacs-test-functions.sh index bfc10be3..2dfd7c56 100755 --- a/test/T420-emacs-test-functions.sh +++ b/test/T420-emacs-test-functions.sh @@ -2,6 +2,7 @@ test_description="emacs test function sanity" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 test_begin_subtest "emacs test function sanity" test_emacs_expect_t 't' diff --git a/test/T430-emacs-address-cleaning.sh b/test/T430-emacs-address-cleaning.sh index 02d3b411..7d3d61ff 100755 --- a/test/T430-emacs-address-cleaning.sh +++ b/test/T430-emacs-address-cleaning.sh @@ -2,6 +2,9 @@ test_description="emacs address cleaning" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs test_begin_subtest "notmuch-test-address-clean part 1" test_emacs_expect_t '(notmuch-test-address-cleaning-1)' diff --git a/test/T440-emacs-hello.sh b/test/T440-emacs-hello.sh index d23c1fca..842781a4 100755 --- a/test/T440-emacs-hello.sh +++ b/test/T440-emacs-hello.sh @@ -2,9 +2,11 @@ test_description="emacs notmuch-hello view" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 EXPECTED=$NOTMUCH_SRCDIR/test/emacs.expected-output +test_require_emacs add_email_corpus test_begin_subtest "User-defined section with inbox tag" @@ -66,4 +68,22 @@ test_emacs '(notmuch-hello) notmuch tag -$tag '*' test_expect_equal_file $EXPECTED/notmuch-hello-long-names OUTPUT +test_begin_subtest "All tags show up" +tag=exclude_me +notmuch tag +$tag '*' +notmuch config set search.exclude_tags $tag +test_emacs '(notmuch-hello) + (test-output)' +notmuch tag -$tag '*' +test_expect_equal_file $EXPECTED/notmuch-hello-all-tags OUTPUT + +test_done +test_begin_subtest "notmuch-hello with nonexistent CWD" +test_emacs ' + (notmuch-hello) + (test-log-error + (let ((default-directory "/nonexistent")) + (notmuch-hello-update)))' +test_expect_equal "$(cat MESSAGES)" "COMPLETE" + test_done diff --git a/test/T450-emacs-show.sh b/test/T450-emacs-show.sh index de1755d2..559df8aa 100755 --- a/test/T450-emacs-show.sh +++ b/test/T450-emacs-show.sh @@ -2,9 +2,11 @@ test_description="emacs notmuch-show view" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 EXPECTED=$NOTMUCH_SRCDIR/test/emacs-show.expected-output +test_require_emacs add_email_corpus test_begin_subtest "Hiding Original Message region at beginning of a message" @@ -60,6 +62,17 @@ test_emacs '(let ((notmuch-crypto-process-mime nil)) (test-visible-output))' test_expect_equal_file $EXPECTED/notmuch-show-process-crypto-mime-parts-on OUTPUT +test_begin_subtest "notmuch-search-show-thread returns non-nil on success" +test_emacs_expect_t '(notmuch-search "id:20091117203301.GV3165@dottiness.seas.harvard.edu") + (notmuch-test-wait) + (and (notmuch-search-show-thread) + (not (notmuch-show-next-thread)))' + +test_begin_subtest "notmuch-search-show-thread returns nil when there are no messages" +test_emacs_expect_t '(notmuch-search "id:non-existing-id") + (notmuch-test-wait) + (not (notmuch-search-show-thread))' + test_begin_subtest "notmuch-show: don't elide non-matching messages" test_emacs '(let ((notmuch-show-only-matching-messages nil)) (notmuch-search "from:lars@seas.harvard.edu and subject:\"Maildir storage\"") @@ -78,6 +91,62 @@ test_emacs '(let ((notmuch-show-only-matching-messages t)) (test-visible-output))' test_expect_equal_file $EXPECTED/notmuch-show-elide-non-matching-messages-on OUTPUT +test_begin_subtest "Hide bodies of messages by depth" +test_emacs '(let ((notmuch-show-depth-limit -1)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +test_expect_equal_file $EXPECTED/notmuch-show-depth OUTPUT + + +test_begin_subtest "Hide bodies of messages by height" +test_emacs '(let ((notmuch-show-height-limit -1)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +# folding all messages by height or depth should look the same +test_expect_equal_file $EXPECTED/notmuch-show-depth OUTPUT + +test_begin_subtest "Hide bodies of messages; show only leaves." +test_emacs '(let ((notmuch-show-height-limit 0)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +test_expect_equal_file $EXPECTED/notmuch-show-height-0 OUTPUT + +test_begin_subtest "Hide bodies of messages (depth > 1)" +test_emacs '(let ((notmuch-show-depth-limit 1)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +test_expect_equal_file $EXPECTED/notmuch-show-depth-1 OUTPUT + +test_begin_subtest "Hide bodies of messages by size" +test_emacs '(let ((notmuch-show-max-text-part-size 1)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +test_expect_equal_file $EXPECTED/notmuch-show-size OUTPUT + +test_begin_subtest "Hide bodies of messages by size > 450" +test_emacs '(let ((notmuch-show-max-text-part-size 450)) + (notmuch-search "thread:{id:87ocn0qh6d.fsf@yoom.home.cworth.org}") + (notmuch-test-wait) + (notmuch-search-show-thread) + (notmuch-test-wait) + (test-visible-output))' +test_expect_equal_file $EXPECTED/notmuch-show-size-450 OUTPUT + test_begin_subtest "notmuch-show: elide non-matching messages (w/ notmuch-show-toggle-elide-non-matching)" test_emacs '(let ((notmuch-show-only-matching-messages nil)) (notmuch-search "from:lars@seas.harvard.edu and subject:\"Maildir storage\"") @@ -177,7 +246,7 @@ test_emacs "(let ((notmuch-command \"$PWD/notmuch_fail\")) (let ((inhibit-read-only t)) (erase-buffer))) (condition-case err (notmuch-show \"*\") - (error (message \"%s\" (second err)))) + (error (message \"%s\" (cadr err)))) (notmuch-test-wait) (with-current-buffer \"*Messages*\" (test-output \"MESSAGES\")) @@ -189,9 +258,8 @@ test_expect_equal "$(notmuch_emacs_error_sanitize notmuch_fail OUTPUT MESSAGES E === MESSAGES === This is an error (see *Notmuch errors* for more details) === ERROR === -[XXX] This is an error -command: YYY/notmuch_fail show --format\\=sexp --format-version\\=4 --decrypt\\=true --exclude\\=false \\' \\* \\' +command: YYY/notmuch_fail show --format\\=sexp --format-version\\=5 --decrypt\\=true --exclude\\=false \\' \\* \\' exit status: 1 stderr: This is an error @@ -208,6 +276,10 @@ test_emacs '(notmuch-show "id:'$gen_msg_id'") output=$(head -1 OUTPUT.raw|cut -f1-4 -d' ') test_expect_equal "$output" "Notmuch Test Suite " +test_begin_subtest "multipart/alternative hides html by default" +test_emacs '(notmuch-show "id:cf0c4d610911171136h1713aa59w9cf9aa31f052ad0a@mail.gmail.com") + (test-visible-output)' +test_expect_equal_file $EXPECTED/notmuch-show-multipart-alternative OUTPUT # switching to the crypto corpus, using gpg from here on: add_gnupg_home @@ -218,6 +290,14 @@ test_emacs '(notmuch-show "id:basic-encrypted@crypto.notmuchmail.org") (test-visible-output)' test_expect_equal_file $EXPECTED/notmuch-show-decrypted-message OUTPUT +test_begin_subtest "show encrypted rfc822 message" +if ${TEST_EMACS} --quick --batch --eval '(kill-emacs (if (version< emacs-version "28") 0 1))'; then + test_subtest_known_broken +fi +test_emacs '(notmuch-show "id:encrypted-rfc822-attachment@crypto.notmuchmail.org") + (test-visible-output)' +test_expect_code 1 'fgrep "!!!" OUTPUT' + test_begin_subtest "show undecryptable message" test_emacs '(notmuch-show "id:simple-encrypted@crypto.notmuchmail.org") (test-visible-output)' @@ -229,4 +309,107 @@ test_emacs '(let ((notmuch-crypto-process-mime nil)) (test-visible-output))' test_expect_equal_file $EXPECTED/notmuch-show-decrypted-message-no-crypto OUTPUT +test_begin_subtest "notmuch-show with nonexistent CWD" +tid=$(notmuch search --limit=1 --output=threads '*' | sed s/thread://) +test_emacs "(test-log-error + (let ((default-directory \"/nonexistent\")) + (notmuch-show \"$tid\")))" +test_expect_equal "$(cat MESSAGES)" "COMPLETE" + +add_email_corpus attachment + +test_begin_subtest "tar not inlined by default" +test_emacs '(notmuch-show "id:874llc2bkp.fsf@curie.anarc.at") + (test-visible-output "OUTPUT")' +cat < EXPECTED +Antoine Beaupré (2018-03-19) (attachment inbox) +Subject: Re: bug: "no top level messages" crash on Zen email loops +To: David Bremner , notmuch@notmuchmail.org +Date: Mon, 19 Mar 2018 13:56:54 -0400 + +[ multipart/mixed ] +[ text/plain ] +And obviously I forget the frigging attachment. +[ zendesk-email-loop2.tgz: application/x-gtar-compressed ] +[ text/plain ] + +PS: don't we have a "you forgot to actually attach the damn file" plugin +when we detect the word "attachment" and there's no attach? :p +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "tar not inlined by default on refresh" +test_emacs '(notmuch-show "id:874llc2bkp.fsf@curie.anarc.at") + (notmuch-show-refresh-view) + (test-visible-output "OUTPUT")' +cat < EXPECTED +Antoine Beaupré (2018-03-19) (attachment inbox) +Subject: Re: bug: "no top level messages" crash on Zen email loops +To: David Bremner , notmuch@notmuchmail.org +Date: Mon, 19 Mar 2018 13:56:54 -0400 + +[ multipart/mixed ] +[ text/plain ] +And obviously I forget the frigging attachment. +[ zendesk-email-loop2.tgz: application/x-gtar-compressed ] +[ text/plain ] + +PS: don't we have a "you forgot to actually attach the damn file" plugin +when we detect the word "attachment" and there's no attach? :p +EOF +test_expect_equal_file EXPECTED OUTPUT + +add_email_corpus duplicate + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +test_begin_subtest "duplicate=3, subject" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 3) + (test-visible-output \"OUTPUT\")" +output=$(grep "Subject:" OUTPUT) +file=$(notmuch search --output=files id:${ID3} | head -n 3 | tail -n 1) +subject=$(grep '^Subject:' $file) +test_expect_equal "$output" "$subject" + +FILE3=$(notmuch search --output=files --duplicate=3 "id:${ID3}") +test_begin_subtest "duplicate=3, stash" +test_emacs_expect_t \ + "(notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 3) + (notmuch-show-stash-filename) + (notmuch-test-expect-equal (list (car kill-ring)) (list \"${FILE3}\"))" + +test_begin_subtest "duplicate=0" +test_emacs "(test-log-error + (notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 0))" +cat < EXPECTED +(error Duplicate 0 out of range [1,5]) +EOF +test_expect_equal_file EXPECTED MESSAGES + +test_begin_subtest "duplicate=1000" +test_emacs "(test-log-error + (notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 1000))" +cat < EXPECTED +(error Duplicate 1000 out of range [1,5]) +EOF +test_expect_equal_file EXPECTED MESSAGES +test_begin_subtest "duplicate=4" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 4) + (test-visible-output \"OUTPUT\")" +test_expect_equal_file_nonempty $EXPECTED/notmuch-show-duplicate-4 OUTPUT + +FILE4=$(notmuch search --output=files --duplicate=4 "id:${ID3}") +test_begin_subtest "duplicate=4, raw" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 4) + (notmuch-show-view-raw-message) + (test-visible-output \"OUTPUT\")" +subject4=$(grep '^Subject:' $FILE4) +subject=$(grep '^Subject:' OUTPUT) +test_expect_equal "$subject4" "$subject" + test_done diff --git a/test/T453-emacs-reply.sh b/test/T453-emacs-reply.sh new file mode 100755 index 00000000..0a27d066 --- /dev/null +++ b/test/T453-emacs-reply.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash + +test_description="emacs reply" +. $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +EXPECTED=$NOTMUCH_SRCDIR/test/emacs-reply.expected-output + +test_require_emacs + +add_email_corpus attachment + +test_begin_subtest "tar not inlined by default" +test_emacs '(notmuch-mua-new-reply "id:874llc2bkp.fsf@curie.anarc.at") + (test-visible-output "OUTPUT.raw")' +cat < EXPECTED +From: Notmuch Test Suite +To: Antoine Beaupré +Subject: Re: bug: "no top level messages" crash on Zen email loops +In-Reply-To: <874llc2bkp.fsf@curie.anarc.at> +Fcc: MAIL_DIR/sent +--text follows this line-- +Antoine Beaupré writes: + +> And obviously I forget the frigging attachment. +> +> +> PS: don't we have a "you forgot to actually attach the damn file" plugin +> when we detect the word "attachment" and there's no attach? :p +EOF +notmuch_dir_sanitize < OUTPUT.raw > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +add_email_corpus duplicate + +ID2=87r2geywh9.fsf@tethera.net +for dup in {1..2}; do + test_begin_subtest "body, duplicate=${dup}" + test_emacs "(notmuch-show \"id:${ID2}\") + (notmuch-test-wait) + (notmuch-show-choose-duplicate $dup) + (notmuch-test-wait) + (notmuch-show-reply) + (test-visible-output \"OUTPUT.raw\")" + output=$(grep '^> # body' OUTPUT.raw) + test_expect_equal "$output" "> # body ${dup}" +done + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +test_begin_subtest "duplicate=3, subject" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-test-wait) + (notmuch-show-choose-duplicate 3) + (notmuch-test-wait) + (notmuch-show-reply) + (test-visible-output \"OUTPUT\")" +output=$(sed -n 's/^Subject: //p' OUTPUT) +file=$(notmuch search --output=files id:${ID3} | head -n 3 | tail -n 1) +subject=$(sed -n 's/^Subject: //p' $file) +test_expect_equal "$output" "Re: $subject" + +test_begin_subtest "duplicate=4" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-show-choose-duplicate 4) + (notmuch-test-wait) + (notmuch-show-reply) + (test-visible-output \"OUTPUT.raw\")" +notmuch_dir_sanitize < OUTPUT.raw > OUTPUT +test_expect_equal_file_nonempty $EXPECTED/notmuch-reply-duplicate-4 OUTPUT + +test_done diff --git a/test/T454-emacs-dont-reply-names.sh b/test/T454-emacs-dont-reply-names.sh new file mode 100755 index 00000000..3a770177 --- /dev/null +++ b/test/T454-emacs-dont-reply-names.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash + +test_description="message-dont-reply-to-names in emacs replies" +. $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +EXPECTED=$NOTMUCH_SRCDIR/test/emacs-show.expected-output + +test_require_emacs + +add_email_corpus default + +test_begin_subtest "regular expression" +test_emacs '(let ((message-dont-reply-to-names "notmuchmail\\|noreply\\|harvard")) + (notmuch-mua-new-reply + "id:20091117203301.GV3165@dottiness.seas.harvard.edu" nil t) + (test-visible-output "OUTPUT-FULL.raw"))' + +notmuch_dir_sanitize < OUTPUT-FULL.raw > OUTPUT-FULL +head -6 OUTPUT-FULL > OUTPUT + +cat < EXPECTED +From: Notmuch Test Suite +To: Mikhail Gusarov +Subject: Re: [notmuch] Working with Maildir storage? +In-Reply-To: <20091117203301.GV3165@dottiness.seas.harvard.edu> +Fcc: MAIL_DIR/sent +--text follows this line-- +EOF + +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "predicate" +test_emacs '(let ((message-dont-reply-to-names + (lambda (m) (string-prefix-p "Mikhail" m)))) + (notmuch-mua-new-reply + "id:20091117203301.GV3165@dottiness.seas.harvard.edu" nil t) + (test-visible-output "OUTPUT-FULL-PRED.raw"))' + +notmuch_dir_sanitize < OUTPUT-FULL-PRED.raw > OUTPUT-FULL-PRED +head -7 OUTPUT-FULL-PRED > OUTPUT-PRED + +cat < EXPECTED-PRED +From: Notmuch Test Suite +To: Lars Kellogg-Stedman +Cc: notmuch@notmuchmail.org +Subject: Re: [notmuch] Working with Maildir storage? +In-Reply-To: <20091117203301.GV3165@dottiness.seas.harvard.edu> +Fcc: MAIL_DIR/sent +--text follows this line-- +EOF + +test_expect_equal_file EXPECTED-PRED OUTPUT-PRED + +test_begin_subtest "nil value" +test_emacs '(let ((message-dont-reply-to-names nil)) + (notmuch-mua-new-reply + "id:20091117203301.GV3165@dottiness.seas.harvard.edu" nil t) + (test-visible-output "OUTPUT-FULL-NIL.raw"))' + +notmuch_dir_sanitize < OUTPUT-FULL-NIL.raw > OUTPUT-FULL-NIL +head -7 OUTPUT-FULL-NIL > OUTPUT-NIL + +cat < EXPECTED-NIL +From: Notmuch Test Suite +To: Lars Kellogg-Stedman , Mikhail Gusarov +Cc: notmuch@notmuchmail.org +Subject: Re: [notmuch] Working with Maildir storage? +In-Reply-To: <20091117203301.GV3165@dottiness.seas.harvard.edu> +Fcc: MAIL_DIR/sent +--text follows this line-- +EOF + +test_expect_equal_file EXPECTED-NIL OUTPUT-NIL + +test_done diff --git a/test/T455-emacs-charsets.sh b/test/T455-emacs-charsets.sh index cb1297ca..db03bb67 100755 --- a/test/T455-emacs-charsets.sh +++ b/test/T455-emacs-charsets.sh @@ -2,11 +2,14 @@ test_description="emacs notmuch-show charset handling" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 UTF8_YEN=$'\xef\xbf\xa5' BIG5_YEN=$'\xa2\x44' +test_require_emacs + # Add four messages with unusual encoding requirements: # # 1) text/plain in quoted-printable big5 diff --git a/test/T460-emacs-tree.sh b/test/T460-emacs-tree.sh index cb2c90b8..6ef5c54a 100755 --- a/test/T460-emacs-tree.sh +++ b/test/T460-emacs-tree.sh @@ -2,9 +2,11 @@ test_description="emacs tree view interface" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 EXPECTED=$NOTMUCH_SRCDIR/test/emacs-tree.expected-output +test_require_emacs add_email_corpus test_begin_subtest "Basic notmuch-tree view in emacs" @@ -98,7 +100,7 @@ test_emacs '(notmuch-hello) (notmuch-test-wait) (test-output) (delete-other-windows)' -test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox OUTPUT +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox-oldest-first OUTPUT test_begin_subtest "Tree view of a single thread (from search)" test_emacs '(notmuch-hello) @@ -177,4 +179,71 @@ output=$(test_emacs '(notmuch-tree "tag:inbox") (notmuch-show-stash-message-id)') test_expect_equal "$output" "\"Stashed: id:1258493565-13508-1-git-send-email-keithp@keithp.com\"" +test_begin_subtest "Functions in tree-result-format" +test_emacs ' +(let + ((notmuch-tree-result-format + (quote (("date" . "%12s ") + ("authors" . "%-20s") + ((("tree" . "%s") + ("subject" . "%s")) . " %-54s ") + (notmuch-test-result-flags . "(%s)"))))) + (notmuch-tree "tag:inbox") + (notmuch-test-wait) + (test-output)) +' +test_expect_equal_file $EXPECTED/result-format-function OUTPUT + +test_begin_subtest "notmuch-tree with nonexistent CWD" +test_emacs '(test-log-error + (let ((default-directory "/nonexistent")) + (notmuch-tree "*")))' +test_expect_equal "$(cat MESSAGES)" "COMPLETE" + +# reinitialize database for outline tests +add_email_corpus + +test_begin_subtest "start in outline mode" +test_emacs '(let ((notmuch-tree-outline-enabled t)) + (notmuch-tree "tag:inbox") + (notmuch-test-wait) + (test-visible-output))' +# folding all messages by height or depth should look the same +test_expect_equal_file $EXPECTED/inbox-outline OUTPUT + +test_begin_subtest "outline-cycle-buffer" +test_emacs '(let ((notmuch-tree-outline-enabled t)) + (notmuch-tree "tag:inbox") + (notmuch-test-wait) + (outline-cycle-buffer) + (outline-cycle-buffer) + (notmuch-test-wait) + (test-visible-output))' +# folding all messages by height or depth should look the same +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox OUTPUT + +test_done + +add_email_corpus duplicate + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +test_begin_subtest "duplicate=3, subject" +test_emacs "(notmuch-tree \"id:${ID3}\") + (notmuch-test-wait) + (notmuch-tree-show-message t) + (notmuch-show-choose-duplicate 3) + (test-visible-output \"OUTPUT\")" +output=$(grep "Subject:" OUTPUT) +file=$(notmuch search --output=files id:${ID3} | head -n 3 | tail -n 1) +subject=$(grep '^Subject:' $file) +test_expect_equal "$output" "$subject" + +test_begin_subtest "duplicate=4" +test_emacs "(notmuch-show \"id:${ID3}\") + (notmuch-test-wait) + (notmuch-tree-show-message t) + (notmuch-show-choose-duplicate 4) + (test-visible-output \"OUTPUT\")" +test_expect_equal_file_nonempty $NOTMUCH_SRCDIR/test/emacs-show.expected-output/notmuch-show-duplicate-4 OUTPUT + test_done diff --git a/test/T461-emacs-search-exclude.sh b/test/T461-emacs-search-exclude.sh new file mode 100755 index 00000000..47f74682 --- /dev/null +++ b/test/T461-emacs-search-exclude.sh @@ -0,0 +1,185 @@ +#!/usr/bin/env bash + +test_description="exclude options persist between Emacs search and tree modes" +. $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +EXPECTED=$NOTMUCH_SRCDIR/test/emacs-exclude.expected-output + +test_require_emacs +add_email_corpus +notmuch config set search.exclude_tags deleted +notmuch tag +deleted -- 'from:"Stewart Smith"' or 'from:"Chris Wilson"' + +# Basic test cases just asserting exclude option is working and consistent. + +test_begin_subtest "Search doesn't contain excluded mail by default" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox-without-excluded OUTPUT + +test_begin_subtest "Toggling exclude in search will show excluded mail" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-search-toggle-hide-excluded) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Tree search doesn't contain excluded mail by default" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-tree-from-search-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox-without-excluded OUTPUT + +test_begin_subtest "Toggling exclude in tree search will show excluded mail" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-tree-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Unthreaded search doesn't contain excluded mail by default" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-unthreaded-from-search-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-unthreaded-tag-inbox-without-excluded OUTPUT + +test_begin_subtest "Toggling exclude in unthreaded will show excluded mail" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-unthreaded-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-unthreaded-tag-inbox-with-excluded OUTPUT + +# Cycling from search to tree to unthreaded and vice versa will persist the current +# value of notmuch-search-hide-excluded. + +test_begin_subtest "Value of hide-excluded from search persists into tree search" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-search-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-tree-from-search-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Value of hide-excluded from search persists into unthreaded" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-search-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-unthreaded-from-search-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-unthreaded-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Value of hide-excluded from tree persists into search" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-tree-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-search-from-tree-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Value of hide-excluded from tree persists into unthreaded" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-tree-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-unthreaded-from-tree-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-unthreaded-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Value of hide-excluded from unthreaded persists into tree" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-unthreaded-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-tree-from-unthreaded-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-tree-tag-inbox-with-excluded OUTPUT + +test_begin_subtest "Value of hide-excluded from unthreaded persists into search" +test_emacs '(notmuch-hello) + (goto-char (point-min)) + (re-search-forward "inbox") + (widget-button-press (1- (point))) + (notmuch-test-wait) + (notmuch-unthreaded-from-search-current-query) + (notmuch-test-wait) + (notmuch-tree-toggle-hide-excluded) + (notmuch-test-wait) + (notmuch-search-from-tree-current-query) + (notmuch-test-wait) + (test-output) + (delete-other-windows)' +test_expect_equal_file $EXPECTED/notmuch-search-tag-inbox-with-excluded OUTPUT + +test_done diff --git a/test/T465-emacs-unthreaded.sh b/test/T465-emacs-unthreaded.sh new file mode 100755 index 00000000..a3ff85fd --- /dev/null +++ b/test/T465-emacs-unthreaded.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash + +test_description="emacs unthreaded interface" +. $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs + +EXPECTED=$NOTMUCH_SRCDIR/test/emacs-unthreaded.expected-output + +generate_message "[id]=large-thread-1" '[subject]="large thread"' +printf " 2001-01-05 Notmuch Test Suite large thread%43s(inbox unread)\n" >> EXPECTED.unthreaded + +for num in $(seq 2 64); do + prev=$((num - 1)) + generate_message '[subject]="large thread"' "[id]=large-thread-$num" "[in-reply-to]=\" + printf " 2001-01-05 Notmuch Test Suite large thread%43s(inbox unread)\n" >> EXPECTED.unthreaded +done +printf "End of search results.\n" >> EXPECTED.unthreaded + +notmuch new > new.output 2>&1 + +test_begin_subtest "large thread" +test_emacs '(let ((max-lisp-eval-depth 10)) + (notmuch-unthreaded "subject:large-thread") + (notmuch-test-wait) + (test-output))' +test_expect_equal_file EXPECTED.unthreaded OUTPUT + +test_begin_subtest "message from large thread (status)" +output=$(test_emacs '(let ((max-lisp-eval-depth 10)) + (notmuch-unthreaded "subject:large-thread") + (notmuch-test-wait) + (notmuch-tree-show-message nil) + (notmuch-test-wait) + "SUCCESS")' ) +test_expect_equal "$output" '"SUCCESS"' + +add_email_corpus +test_begin_subtest "Functions in unthreaded-result-format" +test_emacs ' +(let + ((notmuch-unthreaded-result-format + (quote (("date" . "%12s ") + ("authors" . "%-20s") + ("subject" . "%-54s") + (notmuch-test-result-flags . "(%s)"))))) + (notmuch-unthreaded "tag:inbox") + (notmuch-test-wait) + (test-output)) +' +test_expect_equal_file $EXPECTED/result-format-function OUTPUT + +test_begin_subtest "notmuch-unthreaded with nonexistent CWD" +test_emacs '(test-log-error + (let ((default-directory "/nonexistent")) + (notmuch-unthreaded "*")))' +test_expect_equal "$(cat MESSAGES)" "COMPLETE" + +add_email_corpus duplicate + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +test_begin_subtest "duplicate=3, subject" +test_emacs "(let ((notmuch-tree-show-out t)) + (notmuch-unthreaded \"id:${ID3}\") + (notmuch-test-wait) + (notmuch-tree-show-message nil) + (notmuch-show-choose-duplicate 3) + (test-visible-output \"OUTPUT\"))" +output=$(grep "Subject:" OUTPUT) +file=$(notmuch search --output=files id:${ID3} | head -n 3 | tail -n 1) +subject=$(grep '^Subject:' $file) +test_expect_equal "$output" "$subject" + +test_begin_subtest "duplicate=4" +test_emacs "(let ((notmuch-tree-show-out t)) + (notmuch-unthreaded \"id:${ID3}\") + (notmuch-test-wait) + (notmuch-tree-show-message nil) + (notmuch-show-choose-duplicate 4) + (test-visible-output \"OUTPUT\"))" +test_expect_equal_file_nonempty $NOTMUCH_SRCDIR/test/emacs-show.expected-output/notmuch-show-duplicate-4 OUTPUT + + +test_done diff --git a/test/T480-hex-escaping.sh b/test/T480-hex-escaping.sh index 2c5bbb63..8bddf3e7 100755 --- a/test/T480-hex-escaping.sh +++ b/test/T480-hex-escaping.sh @@ -2,6 +2,10 @@ test_description="hex encoding and decoding" . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + test_begin_subtest "round trip" find $NOTMUCH_SRCDIR/test/corpora/default -type f -print | sort | xargs cat > EXPECTED $TEST_DIRECTORY/hex-xcode --direction=encode < EXPECTED | $TEST_DIRECTORY/hex-xcode --direction=decode > OUTPUT @@ -14,13 +18,13 @@ test_expect_equal "$tag_enc1" "comic_swear=%24%26%5e%25%24%5e%25%5c%5c%2f%2f-+%2 test_begin_subtest "round trip newlines" printf 'this\n tag\t has\n spaces\n' > EXPECTED.$test_count -$TEST_DIRECTORY/hex-xcode --direction=encode < EXPECTED.$test_count |\ +$TEST_DIRECTORY/hex-xcode --direction=encode < EXPECTED.$test_count |\ $TEST_DIRECTORY/hex-xcode --direction=decode > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count test_begin_subtest "round trip 8bit chars" echo '%c3%91%c3%a5%c3%b0%c3%a3%c3%a5%c3%a9-%c3%8f%c3%8a' > EXPECTED.$test_count -$TEST_DIRECTORY/hex-xcode --direction=decode < EXPECTED.$test_count |\ +$TEST_DIRECTORY/hex-xcode --direction=decode < EXPECTED.$test_count |\ $TEST_DIRECTORY/hex-xcode --direction=encode > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count @@ -37,13 +41,13 @@ test_expect_equal "$tag_enc1" "comic_swear=%24%26%5e%25%24%5e%25%5c%5c%2f%2f-+%2 test_begin_subtest "round trip newlines (in-place)" printf 'this\n tag\t has\n spaces\n' > EXPECTED.$test_count -$TEST_DIRECTORY/hex-xcode --in-place --direction=encode < EXPECTED.$test_count |\ +$TEST_DIRECTORY/hex-xcode --in-place --direction=encode < EXPECTED.$test_count |\ $TEST_DIRECTORY/hex-xcode --in-place --direction=decode > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count test_begin_subtest "round trip 8bit chars (in-place)" echo '%c3%91%c3%a5%c3%b0%c3%a3%c3%a5%c3%a9-%c3%8f%c3%8a' > EXPECTED.$test_count -$TEST_DIRECTORY/hex-xcode --in-place --direction=decode < EXPECTED.$test_count |\ +$TEST_DIRECTORY/hex-xcode --in-place --direction=decode < EXPECTED.$test_count |\ $TEST_DIRECTORY/hex-xcode --in-place --direction=encode > OUTPUT.$test_count test_expect_equal_file EXPECTED.$test_count OUTPUT.$test_count diff --git a/test/T490-parse-time-string.sh b/test/T490-parse-time-string.sh index d1c70cfa..3b6e48c4 100755 --- a/test/T490-parse-time-string.sh +++ b/test/T490-parse-time-string.sh @@ -2,15 +2,17 @@ test_description="date/time parser module" . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + # Sanity/smoke tests for the date/time parser independent of notmuch -_date () -{ +_date () { date -d "$*" +%s } -_parse_time () -{ +_parse_time () { ${TEST_DIRECTORY}/parse-time --format=%s "$*" } diff --git a/test/T500-search-date.sh b/test/T500-search-date.sh index f84b0962..85ff831f 100755 --- a/test/T500-search-date.sh +++ b/test/T500-search-date.sh @@ -14,9 +14,6 @@ test_expect_equal "$output" "thread:XXX 2010-12-16 [1/1] Olivier Berger; Essai test_begin_subtest "Absolute date field" output=$(notmuch search date:2010-12-16 | notmuch_search_sanitize) -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then - test_subtest_known_broken -fi test_expect_equal "$output" "thread:XXX 2010-12-16 [1/1] Olivier Berger; Essai accentué (inbox unread)" test_begin_subtest "Absolute time range with TZ" diff --git a/test/T510-thread-replies.sh b/test/T510-thread-replies.sh index 8b96a1db..35f3ff83 100755 --- a/test/T510-thread-replies.sh +++ b/test/T510-thread-replies.sh @@ -10,6 +10,7 @@ test_description='test of proper handling of in-reply-to and references headers' # non-RFC-compliant headers' . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 test_begin_subtest "Use References when In-Reply-To is broken" add_message '[id]="foo@one.com"' \ @@ -178,7 +179,7 @@ test_expect_equal_json "$output" "$expected" add_email_corpus threading test_begin_subtest "reply to ghost" -notmuch show --entire-thread=true id:000-real-root@example.org | grep ^Subject: | head -1 > OUTPUT +notmuch show --entire-thread=true id:000-real-root@example.org | grep ^Subject: | head -1 > OUTPUT cat < EXPECTED Subject: root message EOF diff --git a/test/T520-show.sh b/test/T520-show.sh index 16222650..6bcf109c 100755 --- a/test/T520-show.sh +++ b/test/T520-show.sh @@ -3,6 +3,13 @@ test_description='"notmuch show"' . $(dirname "$0")/test-lib.sh || exit 1 +test_query_syntax () { + test_begin_subtest "sexpr query: $1" + sexp=$(notmuch show --format=json --query=sexp "$1") + infix=$(notmuch show --format=json "$2") + test_expect_equal_json "$sexp" "$infix" +} + add_email_corpus test_begin_subtest "exit code for show invalid query" @@ -10,4 +17,68 @@ notmuch show foo.. exit_code=$? test_expect_equal 1 $exit_code +test_begin_subtest "notmuch show --sort=newest-first" +notmuch show --entire-thread=true '*' > EXPECTED +notmuch show --entire-thread=true --sort=newest-first '*' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch show --sort=oldest-first" +notmuch show --entire-thread=true '*' | grep ^depth:0 > EXPECTED +notmuch show --entire-thread=true --sort=oldest-first '*' | grep ^depth:0 > OLDEST +perl -e 'print reverse<>' OLDEST > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch show --sort for single thread" +QUERY="id:yun1vjwegii.fsf@aiko.keithp.com" +notmuch show --entire-thread=true --sort=newest-first $QUERY > EXPECTED +notmuch show --entire-thread=true --sort=oldest-first $QUERY > OUTPUT +test_expect_equal_file EXPECTED OUTPUT + + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + test_query_syntax '(and "wonderful" "wizard")' 'wonderful and wizard' + test_query_syntax '(or "php" "wizard")' 'php or wizard' + test_query_syntax 'wizard' 'wizard' + test_query_syntax 'Wizard' 'Wizard' + test_query_syntax '(attachment notmuch-help.patch)' 'attachment:notmuch-help.patch' + +fi + +add_email_corpus duplicate + +ID1=debian/2.6.1.dfsg-4-1-g87ea161@87ea161e851dfb1ea324af00e4ecfccc18875e15 + +test_begin_subtest "format json, --duplicate=2, duplicate key" +output=$(notmuch show --format=json --duplicate=2 id:${ID1}) +test_json_nodes <<<"$output" "dup:['duplicate']=2" + +test_begin_subtest "format json, subject, --duplicate=1" +output=$(notmuch show --format=json --duplicate=1 id:${ID1}) +file=$(notmuch search --output=files id:${ID1} | head -n 1) +subject=$(sed -n 's/^Subject: \(.*\)$/\1/p' < $file) +test_json_nodes <<<"$output" "subject:['headers']['Subject']=\"$subject\"" + +test_begin_subtest "format json, subject, --duplicate=2" +output=$(notmuch show --format=json --duplicate=2 id:${ID1}) +file=$(notmuch search --output=files id:${ID1} | tail -n 1) +subject=$(sed -n 's/^Subject: \(.*\)$/\1/p' < $file) +test_json_nodes <<<"$output" "subject:['headers']['Subject']=\"$subject\"" + +ID2=87r2geywh9.fsf@tethera.net +for dup in {1..2}; do + test_begin_subtest "format json, body, --duplicate=${dup}" + output=$(notmuch show --format=json --duplicate=${dup} id:${ID2} | \ + $NOTMUCH_PYTHON -B "$NOTMUCH_SRCDIR"/test/json_check_nodes.py "body:['body'][0]['content']" | \ + grep '^# body') + test_expect_equal "$output" "# body ${dup}" +done + +ID3=87r2ecrr6x.fsf@zephyr.silentflame.com +for dup in {1..5}; do + test_begin_subtest "format json, --duplicate=${dup}, 'duplicate' key" + output=$(notmuch show --format=json --duplicate=${dup} id:${ID3}) + test_json_nodes <<<"$output" "dup:['duplicate']=${dup}" +done + test_done diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh index 2124dde2..5f0de2ed 100755 --- a/test/T530-upgrade.sh +++ b/test/T530-upgrade.sh @@ -1,136 +1,76 @@ #!/usr/bin/env bash -test_description="database upgrade" - +test_description='database upgrades' . $(dirname "$0")/test-lib.sh || exit 1 -dbtarball=database-v1.tar.xz - -# XXX: Accomplish the same with test lib helpers -if [ ! -e ${TEST_DIRECTORY}/test-databases/${dbtarball} ]; then - test_subtest_missing_external_prereq_["${dbtarball} - fetch with 'make download-test-databases'"]=t -fi - -test_begin_subtest "database checksum" -test_expect_success \ - '( cd $TEST_DIRECTORY/test-databases && - sha256sum --quiet --check --status ${dbtarball}.sha256 )' - -tar xf $TEST_DIRECTORY/test-databases/${dbtarball} -C ${MAIL_DIR} --strip-components=1 - -test_begin_subtest "folder: search does not work with old database version" -output=$(notmuch search folder:foo) -test_expect_equal "$output" "" - -test_begin_subtest "path: search does not work with old database version" -output=$(notmuch search path:foo) -test_expect_equal "$output" "" - -test_begin_subtest "pre upgrade dump" -test_expect_success 'notmuch dump | sort > pre-upgrade-dump' - -test_begin_subtest "database upgrade from format version 1" -output=$(notmuch new | sed -e 's/^Backing up tags to .*$/Backing up tags to FILENAME/') -test_expect_equal "$output" "\ -Welcome to a new version of notmuch! Your database will now be upgraded. -This process is safe to interrupt. -Backing up tags to FILENAME -Your notmuch database has now been upgraded. -No new mail." - -test_begin_subtest "tag backup matches pre-upgrade dump" -gunzip -c ${MAIL_DIR}/.notmuch/dump-*.gz | sort > backup-dump -test_expect_equal_file pre-upgrade-dump backup-dump - -test_begin_subtest "folder: no longer matches in the middle of path" -output=$(notmuch search folder:baz) -test_expect_equal "$output" "" - -test_begin_subtest "folder: search" -output=$(notmuch search --output=files folder:foo | notmuch_search_files_sanitize | sort) -test_expect_equal "$output" "MAIL_DIR/foo/06:2, -MAIL_DIR/foo/cur/07:2, -MAIL_DIR/foo/cur/08:2, -MAIL_DIR/foo/new/03:2, -MAIL_DIR/foo/new/09:2, -MAIL_DIR/foo/new/10:2," - -test_begin_subtest "top level folder: search" -output=$(notmuch search --output=files folder:'""' | notmuch_search_files_sanitize | sort) -# bar/18:2, is a duplicate of cur/51:2, -test_expect_equal "$output" "MAIL_DIR/01:2, -MAIL_DIR/02:2, -MAIL_DIR/bar/18:2, -MAIL_DIR/cur/29:2, -MAIL_DIR/cur/30:2, -MAIL_DIR/cur/31:2, -MAIL_DIR/cur/32:2, -MAIL_DIR/cur/33:2, -MAIL_DIR/cur/34:2, -MAIL_DIR/cur/35:2, -MAIL_DIR/cur/36:2, -MAIL_DIR/cur/37:2, -MAIL_DIR/cur/38:2, -MAIL_DIR/cur/39:2, -MAIL_DIR/cur/40:2, -MAIL_DIR/cur/41:2, -MAIL_DIR/cur/42:2, -MAIL_DIR/cur/43:2, -MAIL_DIR/cur/44:2, -MAIL_DIR/cur/45:2, -MAIL_DIR/cur/46:2, -MAIL_DIR/cur/47:2, -MAIL_DIR/cur/48:2, -MAIL_DIR/cur/49:2, -MAIL_DIR/cur/50:2, -MAIL_DIR/cur/51:2, -MAIL_DIR/cur/52:2, -MAIL_DIR/cur/53:2, -MAIL_DIR/new/04:2," - -test_begin_subtest "path: search" -output=$(notmuch search --output=files path:"bar" | notmuch_search_files_sanitize | sort) -# cur/51:2, is a duplicate of bar/18:2, -test_expect_equal "$output" "MAIL_DIR/bar/17:2, -MAIL_DIR/bar/18:2, -MAIL_DIR/cur/51:2," - -test_begin_subtest "top level path: search" -output=$(notmuch search --output=files path:'""' | notmuch_search_files_sanitize | sort) -test_expect_equal "$output" "MAIL_DIR/01:2, -MAIL_DIR/02:2," - -test_begin_subtest "recursive path: search" -output=$(notmuch search --output=files path:"bar/**" | notmuch_search_files_sanitize | sort) -# cur/51:2, is a duplicate of bar/18:2, -test_expect_equal "$output" "MAIL_DIR/bar/17:2, -MAIL_DIR/bar/18:2, -MAIL_DIR/bar/baz/05:2, -MAIL_DIR/bar/baz/23:2, -MAIL_DIR/bar/baz/24:2, -MAIL_DIR/bar/baz/cur/25:2, -MAIL_DIR/bar/baz/cur/26:2, -MAIL_DIR/bar/baz/new/27:2, -MAIL_DIR/bar/baz/new/28:2, -MAIL_DIR/bar/cur/19:2, -MAIL_DIR/bar/cur/20:2, -MAIL_DIR/bar/new/21:2, -MAIL_DIR/bar/new/22:2, -MAIL_DIR/cur/51:2," - -test_begin_subtest "body: same as unprefixed before reindex" -notmuch search --output=messages body:close > OUTPUT -notmuch search --output=messages close > EXPECTED +test_require_external_prereq xapian-metadata + +XAPIAN_PATH=$MAIL_DIR/.notmuch/xapian +BACKUP_PATH=$MAIL_DIR/.notmuch/backups + +delete_feature () { + local key=$1 + features=$(xapian-metadata get $XAPIAN_PATH features | grep -v "^$key") + xapian-metadata set $XAPIAN_PATH features "$features" +} + +add_email_corpus + +for key in 'multiple paths per message' \ + 'relative directory paths' \ + 'exact folder:/path: search' \ + 'mail documents for missing messages' \ + 'modification tracking'; do + backup_database + test_begin_subtest "upgrade is triggered by missing '$key'" + delete_feature "$key" + output=$(notmuch new | grep Welcome) + test_expect_equal \ + "$output" \ + "Welcome to a new version of notmuch! Your database will now be upgraded." + + restore_database + + backup_database + test_begin_subtest "backup can be restored ['$key']" + notmuch dump > BEFORE + delete_feature "$key" + notmuch new + notmuch tag -inbox '*' + dump_file=$(echo ${BACKUP_PATH}/dump*) + notmuch restore --input=$dump_file + notmuch dump > AFTER + test_expect_equal_file BEFORE AFTER + restore_database +done + +for key in 'from/subject/message-ID in database' \ + 'indexed MIME types' \ + 'index body and headers separately'; do + backup_database + test_begin_subtest "upgrade not triggered by missing '$key'" + delete_feature "$key" + output=$(notmuch new | grep Welcome) + test_expect_equal "$output" "" + restore_database +done + +test_begin_subtest "upgrade with configured backup dir" +notmuch config set database.backup_dir ${HOME}/backups +delete_feature 'modification tracking' +notmuch new | grep Backing | notmuch_dir_sanitize | sed 's/dump-[0-9T]*/dump-XXX/' > OUTPUT +cat < EXPECTED +Backing up tags to CWD/home/backups/dump-XXX.gz... +EOF test_expect_equal_file EXPECTED OUTPUT -test_begin_subtest "body: subset of unprefixed after reindex" -notmuch reindex '*' -notmuch search --output=messages body:close | sort > BODY -notmuch search --output=messages close | sort > UNPREFIXED -diff -e UNPREFIXED BODY | cut -c2- > OUTPUT +test_begin_subtest "upgrade with relative configured backup dir" +notmuch config set database.backup_dir ${HOME}/backups +delete_feature 'modification tracking' +notmuch new | grep Backing | notmuch_dir_sanitize | sed 's/dump-[0-9T]*/dump-XXX/' > OUTPUT cat < EXPECTED -d -d +Backing up tags to CWD/home/backups/dump-XXX.gz... EOF test_expect_equal_file EXPECTED OUTPUT + test_done diff --git a/test/T550-db-features.sh b/test/T550-db-features.sh index 9d5a9e70..3048c7c4 100755 --- a/test/T550-db-features.sh +++ b/test/T550-db-features.sh @@ -3,6 +3,10 @@ test_description="database version and feature compatibility" . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + test_begin_subtest "future database versions abort open" ${TEST_DIRECTORY}/make-db-version ${MAIL_DIR} 9999 "" output=$(notmuch search x 2>&1 | sed 's/\(database at\) .*/\1 FILENAME/') diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh index 06a6b860..78cae1cd 100755 --- a/test/T560-lib-error.sh +++ b/test/T560-lib-error.sh @@ -16,13 +16,17 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open (NULL, 0, 0); + char* msg = NULL; + stat = notmuch_database_open_with_config (NULL, + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED == stdout == == stderr == -Error: Cannot open a database for a NULL path. +Error: could not locate database. EOF test_expect_equal_file EXPECTED OUTPUT @@ -34,7 +38,11 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open ("./nonexistent/foo", 0, 0); + char *msg = NULL; + stat = notmuch_database_open_with_config ("./nonexistent/foo", + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED @@ -52,7 +60,10 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_create ("./nonexistent/foo", &db); + char *msg = NULL; + + stat = notmuch_database_create_with_config ("./nonexistent/foo", "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED @@ -70,13 +81,17 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open (argv[1], 0, 0); + char* msg = NULL; + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED == stdout == == stderr == -Error opening database at CWD/nonexistent/foo/.notmuch: No such file or directory +Error: database path 'CWD/nonexistent/foo' does not exist or is not a directory. EOF test_expect_equal_file EXPECTED OUTPUT @@ -87,13 +102,17 @@ test_C <<'EOF' int main (int argc, char** argv) { notmuch_status_t stat; - stat = notmuch_database_create (NULL, NULL); + char *msg = NULL; + + stat = notmuch_database_create_with_config (NULL, "", NULL, NULL, &msg); + printf ("%s\n", notmuch_status_to_string (stat)); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED == stdout == +No mail root found == stderr == -Error: Cannot create a database for a NULL path. EOF test_expect_equal_file EXPECTED OUTPUT @@ -105,13 +124,17 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_create (argv[1], &db); + char *msg = NULL; + + stat = notmuch_database_create_with_config (argv[1], "", NULL, &db, &msg); + printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS); + if (msg) fputs (msg, stderr); } EOF cat <<'EOF' >EXPECTED == stdout == +1 == stderr == -Error: Cannot create database at CWD/nonexistent/foo: No such file or directory. EOF test_expect_equal_file EXPECTED OUTPUT @@ -123,7 +146,11 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, &db); + char* msg = NULL; + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); if (stat != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "error opening database: %d\n", stat); } @@ -148,7 +175,9 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, &db); + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &db, NULL); if (stat != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "error opening database: %d\n", stat); } @@ -206,13 +235,14 @@ int main (int argc, char** argv) char *msg = NULL; int fd; - stat = notmuch_database_open_verbose (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, &db, &msg); + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); if (stat != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); exit (1); } - path = talloc_asprintf (db, "%s/.notmuch/xapian/postlist.${db_ending}", argv[1]); - fd = open(path,O_WRONLY|O_TRUNC); + fd = open(argv[2],O_WRONLY|O_TRUNC); if (fd < 0) { fprintf (stderr, "error opening %s\n", argv[1]); exit (1); @@ -228,9 +258,10 @@ cat <<'EOF' > c_tail } EOF +POSTLIST_PATH=(${MAIL_DIR}/.notmuch/xapian/postlist.*) backup_database test_begin_subtest "Xapian exception finding message" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} { notmuch_message_t *message = NULL; stat = notmuch_database_find_message (db, "id:nonexistent", &message); @@ -245,27 +276,9 @@ EOF test_expect_equal_file EXPECTED OUTPUT.clean restore_database -backup_database -test_begin_subtest "Xapian exception getting tags" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} - { - notmuch_tags_t *tags = NULL; - tags = notmuch_database_get_all_tags (db); - stat = (tags == NULL); - } -EOF -sed 's/^\(A Xapian exception [^:]*\):.*$/\1/' < OUTPUT > OUTPUT.clean -cat <<'EOF' >EXPECTED -== stdout == -== stderr == -A Xapian exception occurred getting tags -EOF -test_expect_equal_file EXPECTED OUTPUT.clean -restore_database - backup_database test_begin_subtest "Xapian exception creating directory" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} { notmuch_directory_t *directory = NULL; stat = notmuch_database_get_directory (db, "none/existing", &directory); @@ -275,14 +288,14 @@ sed 's/^\(A Xapian exception [^:]*\):.*$/\1/' < OUTPUT > OUTPUT.clean cat <<'EOF' >EXPECTED == stdout == == stderr == -A Xapian exception occurred creating a directory +A Xapian exception occurred finding/creating a directory EOF test_expect_equal_file EXPECTED OUTPUT.clean restore_database backup_database test_begin_subtest "Xapian exception searching messages" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} { notmuch_messages_t *messages = NULL; notmuch_query_t *query=notmuch_query_create (db, "*"); @@ -301,7 +314,7 @@ restore_database backup_database test_begin_subtest "Xapian exception counting messages" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} { int count; notmuch_query_t *query=notmuch_query_create (db, "id:87ocn0qh6d.fsf@yoom.home.cworth.org"); diff --git a/test/T562-lib-database.sh b/test/T562-lib-database.sh new file mode 100755 index 00000000..fedfc9ed --- /dev/null +++ b/test/T562-lib-database.sh @@ -0,0 +1,427 @@ +#!/usr/bin/env bash +test_description="notmuch_database_* API" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +cat < c_head +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_status_t stat = NOTMUCH_STATUS_SUCCESS; + char *msg = NULL; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } +EOF + +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +test_begin_subtest "get status_string with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *str; + EXPECT0(notmuch_database_close (db)); + str = notmuch_database_status_string (db); + printf("%d\n", str == NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get path with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *path; + EXPECT0(notmuch_database_close (db)); + path = notmuch_database_get_path (db); + printf("%s\n", path); + } +EOF +cat < EXPECTED +== stdout == +MAIL_DIR +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get version with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + unsigned int version; + EXPECT0(notmuch_database_close (db)); + version = notmuch_database_get_version (db); + printf ("%u\n", version); + stat = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } +EOF +cat < EXPECTED +== stdout == +0 +== stderr == +A Xapian exception occurred at database.cc:XXX: Database has been closed +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "re-close a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_close (db); + printf ("%d\n", stat); + } +EOF +cat < EXPECTED +== stdout == +0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "destroy a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + unsigned int version; + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_destroy (db); + printf ("%d\n", stat); + } +EOF +cat < EXPECTED +== stdout == +0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "destroy an open db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + unsigned int version; + stat = notmuch_database_destroy (db); + printf ("%d\n", stat); + } +EOF +cat < EXPECTED +== stdout == +0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "check a closed db for upgrade" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_bool_t ret; + + EXPECT0(notmuch_database_close (db)); + ret = notmuch_database_needs_upgrade (db); + printf ("%d\n", ret == FALSE); + stat = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred at database.cc:XXX: Database has been closed +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "upgrade a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_upgrade (db, NULL, NULL); + printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "begin atomic section for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_begin_atomic (db); + printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS || + stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + stat = NOTMUCH_STATUS_SUCCESS; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "end atomic section for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + EXPECT0(notmuch_database_begin_atomic (db)); + stat = notmuch_database_end_atomic (db); + printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS || + stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + stat = NOTMUCH_STATUS_SUCCESS; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get revision for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *uuid; + unsigned long rev; + + EXPECT0(notmuch_database_close (db)); + rev = notmuch_database_get_revision (db, &uuid); + printf ("%d\n", rev, uuid); + } +EOF +cat < EXPECTED +== stdout == +53 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get directory for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_directory_t *dir; + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_get_directory (db, "/nonexistent", &dir); + printf ("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred finding/creating a directory: Database has been closed. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "index file with a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_message_t *msg; + const char *path = talloc_asprintf(db, "%s/01:2,", argv[1]); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_index_file (db, path, NULL, &msg); + printf ("%d\n", stat == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +Cannot write to a closed database. +EOF +test_expect_equal_file EXPECTED OUTPUT + +generate_message '[filename]=relative_path' +test_begin_subtest "index file (relative path)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_message_t *msg; + stat = notmuch_database_index_file (db, "relative_path", NULL, &msg); + printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "index file (absolute path outside mail root)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_message_t *msg; + stat = notmuch_database_index_file (db, "/dev/zero", NULL, &msg); + printf ("%d\n", stat == NOTMUCH_STATUS_FILE_ERROR); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +Error opening /dev/zero: path outside mail root +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "remove message file with a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_remove_message (db, "01:2,"); + printf ("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred finding/creating a directory: Database has been closed. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "find message by filename with a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_message_t *msg; + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_find_message_by_filename (db, "01:2,", &msg); + printf ("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred finding/creating a directory: Database has been closed. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting tags from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_tags_t *result; + EXPECT0(notmuch_database_close (db)); + result = notmuch_database_get_all_tags (db); + printf("%d\n", result == NULL); + stat = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred getting tags: Database has been closed. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get config from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + char *result; + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_get_config (db, "foo", &result); + printf("%d\n", stat == NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "set config in closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_set_config (db, "foo", "bar"); + printf("%d\n", stat == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +Cannot write to a closed database. +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get indexopts from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_indexopts_t *result; + EXPECT0(notmuch_database_close (db)); + result = notmuch_database_get_default_indexopts (db); + printf("%d\n", result != NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get decryption policy from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_indexopts_t *result; + result = notmuch_database_get_default_indexopts (db); + EXPECT0(notmuch_database_close (db)); + notmuch_decryption_policy_t policy = notmuch_indexopts_get_decrypt_policy (result); + printf ("%d\n", policy == NOTMUCH_DECRYPT_AUTO); + notmuch_indexopts_destroy (result); + printf ("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +1 +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "set decryption policy with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_indexopts_t *result; + result = notmuch_database_get_default_indexopts (db); + EXPECT0(notmuch_database_close (db)); + notmuch_decryption_policy_t policy = notmuch_indexopts_get_decrypt_policy (result); + stat = notmuch_indexopts_set_decrypt_policy (result, policy); + printf("%d\n%d\n", policy == NOTMUCH_DECRYPT_AUTO, stat == NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T563-lib-directory.sh b/test/T563-lib-directory.sh new file mode 100755 index 00000000..4711fcdf --- /dev/null +++ b/test/T563-lib-directory.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +test_description="notmuch_directory_* API" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +cat < c_head +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_directory_t *dir; + notmuch_status_t stat = NOTMUCH_STATUS_SUCCESS; + char *msg = NULL; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } + + EXPECT0(notmuch_database_get_directory (db, "bar", &dir)); + EXPECT0(notmuch_database_close (db)); +EOF + +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +test_begin_subtest "get child directories for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_filenames_t *children; + children = notmuch_directory_get_child_directories (dir); + printf ("%d\n", children == NULL); + stat = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred at directory.cc:XXX: Database has been closed +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get child filenames for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_filenames_t *children; + children = notmuch_directory_get_child_files (dir); + printf ("%d\n", children == NULL); + stat = NOTMUCH_STATUS_XAPIAN_EXCEPTION; + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred at directory.cc:XXX: Database has been closed +EOF +test_expect_equal_file EXPECTED OUTPUT + +backup_database +test_begin_subtest "delete directory document for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + stat = notmuch_directory_delete (dir); + printf ("%d\n", stat == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +Cannot write to a closed database. +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "get/set mtime of directory for a closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + time_t stamp = notmuch_directory_get_mtime (dir); + stat = notmuch_directory_set_mtime (dir, stamp); + printf ("%d\n", stat == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +Cannot write to a closed database. +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_done diff --git a/test/T564-lib-query.sh b/test/T564-lib-query.sh new file mode 100755 index 00000000..53a63bf6 --- /dev/null +++ b/test/T564-lib-query.sh @@ -0,0 +1,254 @@ +#!/usr/bin/env bash +test_description="notmuch_query_* API" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +cat < c_head +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_status_t stat; + char *msg = NULL; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } +EOF + +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +test_begin_subtest "roundtrip query string with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + const char *ret; + + EXPECT0(notmuch_database_close (db)); + query = notmuch_query_create (db, str); + ret = notmuch_query_get_query_string (query); + + printf("%s\n%s\n", str, ret); + } +EOF +cat < EXPECTED +== stdout == +id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "retrieve closed db from query" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + notmuch_database_t *db2; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + db2 = notmuch_query_get_database (query); + + printf("%d\n", db == db2); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "set omit_excluded on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + notmuch_query_set_omit_excluded (query, NOTMUCH_EXCLUDE_ALL); + + printf("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "roundtrip sort on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + notmuch_sort_t sort; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED); + sort = notmuch_query_get_sort (query); + printf("%d\n", sort == NOTMUCH_SORT_UNSORTED); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "add tag_exclude on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_query_add_tag_exclude (query, "spam"); + printf("%d\n", stat == NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search threads on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + notmuch_threads_t *threads; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_query_search_threads (query, &threads); + + printf("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred performing query: Database has been closed +Query string was: id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "search messages on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + notmuch_messages_t *messages; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_query_search_messages (query, &messages); + + printf("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred performing query: Database has been closed +Query string was: id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "count messages on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + unsigned int count; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_query_count_messages (query, &count); + + printf("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred performing query: Database has been closed +Query string was: id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "count threads on closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + unsigned int count; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + stat = notmuch_query_count_threads (query, &count); + + printf("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred performing query: Database has been closed +Query string was: id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "destroy query with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_query_t *query; + const char *str = "id:1258471718-6781-1-git-send-email-dottedmag@dottedmag.net"; + + query = notmuch_query_create (db, str); + EXPECT0(notmuch_database_close (db)); + notmuch_query_destroy (query); + + printf("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T565-lib-tags.sh b/test/T565-lib-tags.sh new file mode 100755 index 00000000..2a59f8dd --- /dev/null +++ b/test/T565-lib-tags.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +test_description="API tests for tags" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +cat < c_head +#include +#include +#include +#include +#include +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_status_t stat; + char *path; + char *msg = NULL; + int fd; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database\n%s\n%s\n", notmuch_status_to_string (stat), msg ? msg : ""); + exit (1); + } +EOF +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +POSTLIST_PATH=(${MAIL_DIR}/.notmuch/xapian/postlist.*) + +backup_database +test_begin_subtest "Xapian exception getting tags" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} + { + notmuch_tags_t *tags = NULL; + fd = open(argv[2],O_WRONLY|O_TRUNC); + if (fd < 0) { + fprintf (stderr, "error opening %s\n", argv[1]); + exit (1); + } + tags = notmuch_database_get_all_tags (db); + stat = (tags == NULL); + } +EOF +sed 's/^\(A Xapian exception [^:]*\):.*$/\1/' < OUTPUT > OUTPUT.clean +cat <<'EOF' >EXPECTED +== stdout == +== stderr == +A Xapian exception occurred getting tags +EOF +test_expect_equal_file EXPECTED OUTPUT.clean +restore_database + +test_begin_subtest "NULL tags are not valid" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_bool_t valid = TRUE; + valid = notmuch_tags_valid (NULL); + fprintf(stdout, "valid = %d\n", valid); + } +EOF +cat <<'EOF' >EXPECTED +== stdout == +valid = 0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T566-lib-message.sh b/test/T566-lib-message.sh new file mode 100755 index 00000000..69051937 --- /dev/null +++ b/test/T566-lib-message.sh @@ -0,0 +1,550 @@ +#!/usr/bin/env bash +test_description="API tests for notmuch_message_*" + +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ -n "${NOTMUCH_TEST_INSTALLED}" ]; then + test_done +fi + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +cat < c_head0 +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_status_t stat; + char *msg = NULL; + notmuch_message_t *message = NULL; + const char *id = "87pr7gqidx.fsf@yoom.home.cworth.org"; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } + EXPECT0(notmuch_database_find_message (db, id, &message)); +EOF + +cp c_head0 c_head +echo " EXPECT0(notmuch_database_close (db));" >> c_head + +test_begin_subtest "Handle getting message-id from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *id2; + id2=notmuch_message_get_message_id (message); + printf("%d\n%d\n", message != NULL, id2==NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting thread-id from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *id2; + id2=notmuch_message_get_thread_id (message); + printf("%d\n%d\n", message != NULL, id2==NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting header from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *from; + from=notmuch_message_get_header (message, "from"); + printf("%s\n%d\n", id, from == NULL); + } +EOF +cat < EXPECTED +== stdout == +87pr7gqidx.fsf@yoom.home.cworth.org +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +# XXX this test only tests the trivial code path +test_begin_subtest "Handle getting replies from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *replies; + replies = notmuch_message_get_replies (message); + printf("%d\n%d\n", message != NULL, replies==NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting message filename from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *filename; + filename = notmuch_message_get_filename (message); + printf("%d\n%d\n", message != NULL, filename == NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting all message filenames from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_filenames_t *filenames; + filenames = notmuch_message_get_filenames (message); + printf("%d\n%d\n", message != NULL, filenames == NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate over all message filenames from closed database" +cat c_head0 - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_filenames_t *filenames; + filenames = notmuch_message_get_filenames (message); + EXPECT0(notmuch_database_close (db)); + for (; notmuch_filenames_valid (filenames); + notmuch_filenames_move_to_next (filenames)) { + const char *filename = notmuch_filenames_get (filenames); + printf("%s\n", filename); + } + notmuch_filenames_destroy (filenames); + printf("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +MAIL_DIR/cur/40:2, +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting ghost flag from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_bool_t result; + result = notmuch_message_get_flag (message, NOTMUCH_MESSAGE_FLAG_GHOST); + printf("%d\n%d\n", message != NULL, result == FALSE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting date from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + time_t result; + result = notmuch_message_get_date (message); + printf("%d\n%d\n", message != NULL, result == 0); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle getting tags from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_tags_t *result; + result = notmuch_message_get_tags (message); + printf("%d\n%d\n", message != NULL, result == NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle counting files from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + int result; + result = notmuch_message_count_files (message); + printf("%d\n%d\n", message != NULL, result < 0); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle adding tag with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_add_tag (message, "boom"); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle removing tag with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_remove_tag (message, "boom"); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle read maildir flag with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_bool_t is_set = -1; + is_set = notmuch_message_has_maildir_flag (message, 'S'); + printf("%d\n%d\n", message != NULL, is_set == FALSE || is_set == TRUE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle checking maildir flag with closed db (new API)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + notmuch_bool_t out; + status = notmuch_message_has_maildir_flag_st (message, 'S', &out); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle converting maildir flags to tags with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_maildir_flags_to_tags (message); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "_notmuch_message_add_term catches exceptions" +cat c_head0 - c_tail <<'EOF' | test_private_C ${MAIL_DIR} + { + notmuch_private_status_t status; + /* This relies on Xapian throwing an exception for adding empty terms */ + status = _notmuch_message_add_term (message, "body", ""); + printf("%d\n%d\n", message != NULL, status != NOTMUCH_STATUS_SUCCESS ); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "_notmuch_message_remove_term catches exceptions" +cat c_head0 - c_tail <<'EOF' | test_private_C ${MAIL_DIR} + { + notmuch_private_status_t status; + /* Xapian throws the same exception for empty and non-existent terms; + * error string varies between Xapian versions. */ + status = _notmuch_message_remove_term (message, "tag", "nonexistent"); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_SUCCESS ); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "_notmuch_message_add_filename on closed db" +cat c_head - c_tail <<'EOF' | test_private_C ${MAIL_DIR} + { + notmuch_private_status_t status; + status = _notmuch_message_add_filename (message, "some-filename"); + printf("%d\n%d\n", message != NULL, status != NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "_notmuch_message_remove_filename on closed db" +cat c_head - c_tail <<'EOF' | test_private_C ${MAIL_DIR} + { + notmuch_private_status_t status; + status = _notmuch_message_remove_filename (message, "some-filename"); + printf("%d\n%d\n", message != NULL, status != NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle converting tags to maildir flags with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_tags_to_maildir_flags (message); + printf("%d\n%d\n", message != NULL, status != NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +POSTLIST_PATH=(${MAIL_DIR}/.notmuch/xapian/postlist.*) +test_begin_subtest "Handle converting tags to maildir flags with corrupted db" +backup_database +cat c_head0 - c_tail <<'EOF' | test_C ${MAIL_DIR} ${POSTLIST_PATH} + { + notmuch_status_t status; + + status = notmuch_message_add_tag (message, "draft"); + if (status) exit(1); + + int fd = open(argv[2],O_WRONLY|O_TRUNC); + if (fd < 0) { + fprintf (stderr, "error opening %s\n", argv[1]); + exit (1); + } + + status = notmuch_message_tags_to_maildir_flags (message); + printf("%d\n%d\n", message != NULL, status != NOTMUCH_STATUS_SUCCESS); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +restore_database +notmuch new +notmuch tag -draft id:87pr7gqidx.fsf@yoom.home.cworth.org +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle removing all tags with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_remove_all_tags (message); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle freezing message with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_freeze (message); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle thawing message with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_thaw (message); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_CLOSED_DATABASE); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle destroying message with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_message_destroy (message); + printf("%d\n%d\n", message != NULL, 1); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle retrieving closed db from message" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_database_t *db2; + db2 = notmuch_message_get_database (message); + printf("%d\n%d\n", message != NULL, db == db2); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "Handle reindexing message with closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_status_t status; + status = notmuch_message_reindex (message, NULL); + printf("%d\n%d\n", message != NULL, status == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +TERMLIST_PATH=(${MAIL_DIR}/.notmuch/xapian/termlist.*) +test_begin_subtest "remove message with corrupted db" +backup_database +cat c_head0 - c_tail <<'EOF' | test_private_C ${MAIL_DIR} ${TERMLIST_PATH} + { + notmuch_status_t status; + + int fd = open(argv[2],O_WRONLY|O_TRUNC); + if (fd < 0) { + fprintf (stderr, "error opening %s\n", argv[1]); + exit (1); + } + + stat = _notmuch_message_delete (message); + printf ("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); + } +EOF +cat < EXPECTED +== stdout == +1 +== stderr == +A Xapian exception occurred at message.cc:XXX: EOF reading block YYY +EOF +sed 's/EOF reading block [0-9]*/EOF reading block YYY/' < OUTPUT > OUTPUT.clean +test_expect_equal_file EXPECTED OUTPUT.clean +restore_database + +test_done diff --git a/test/T568-lib-thread.sh b/test/T568-lib-thread.sh new file mode 100755 index 00000000..b4c24ca2 --- /dev/null +++ b/test/T568-lib-thread.sh @@ -0,0 +1,341 @@ +#!/usr/bin/env bash +test_description="API tests for notmuch_thread_*" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +test_begin_subtest "building database" +test_expect_success "NOTMUCH_NEW" + +test_begin_subtest "finding thread" +THREAD=$(notmuch search --output=threads id:20091117190054.GU3165@dottiness.seas.harvard.edu) +count=$(notmuch count $THREAD) +test_expect_equal "$count" "7" + +cat <<'EOF' > c_tail + if (stat) { + const char *stat_str = notmuch_database_status_string (db); + if (stat_str) + fputs (stat_str, stderr); + } + +} +EOF + +cat < c_head +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + notmuch_status_t stat; + char *msg = NULL; + notmuch_thread_t *thread = NULL; + notmuch_threads_t *threads = NULL; + notmuch_query_t *query = NULL; + const char *id = "${THREAD}"; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + NULL, NULL, &db, &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } + + query = notmuch_query_create (db, id); + EXPECT0(notmuch_query_search_threads (query, &threads)); + thread = notmuch_threads_get (threads); + EXPECT0(notmuch_database_close (db)); +EOF + +test_begin_subtest "get thread-id from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *id2; + id2 = notmuch_thread_get_thread_id (thread); + printf("%d\n%s\n", thread != NULL, id2); + } +EOF +thread_num=${THREAD#thread:} +cat < EXPECTED +== stdout == +1 +${thread_num} +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get total messages with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + int count; + count = notmuch_thread_get_total_messages (thread); + printf("%d\n%d\n", thread != NULL, count); + } +EOF +cat < EXPECTED +== stdout == +1 +7 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get total files with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + int count; + count = notmuch_thread_get_total_files (thread); + printf("%d\n%d\n", thread != NULL, count); + } +EOF +cat < EXPECTED +== stdout == +1 +7 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get top level messages with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages; + messages = notmuch_thread_get_toplevel_messages (thread); + printf("%d\n%d\n", thread != NULL, messages != NULL); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate over level messages with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages; + for (messages = notmuch_thread_get_toplevel_messages (thread); + notmuch_messages_valid (messages); + notmuch_messages_move_to_next (messages)) { + notmuch_message_t *message = notmuch_messages_get (messages); + const char *mid = notmuch_message_get_message_id (message); + printf("%s\n", mid); + } + } +EOF +cat < EXPECTED +== stdout == +20091117190054.GU3165@dottiness.seas.harvard.edu +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate over level messages with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages; + for (messages = notmuch_thread_get_toplevel_messages (thread); + notmuch_messages_valid (messages); + notmuch_messages_move_to_next (messages)) { + notmuch_message_t *message = notmuch_messages_get (messages); + const char *mid = notmuch_message_get_message_id (message); + printf("%s\n", mid); + } + } +EOF +cat < EXPECTED +== stdout == +20091117190054.GU3165@dottiness.seas.harvard.edu +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate over replies with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages = notmuch_thread_get_toplevel_messages (thread); + notmuch_message_t *message = notmuch_messages_get (messages); + notmuch_messages_t *replies; + for (replies = notmuch_message_get_replies (message); + notmuch_messages_valid (replies); + notmuch_messages_move_to_next (replies)) { + notmuch_message_t *message = notmuch_messages_get (replies); + const char *mid = notmuch_message_get_message_id (message); + + printf("%s\n", mid); + } + } +EOF +cat < EXPECTED +== stdout == +87iqd9rn3l.fsf@vertex.dottedmag +87ocn0qh6d.fsf@yoom.home.cworth.org +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate over all messages with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages; + for (messages = notmuch_thread_get_messages (thread); + notmuch_messages_valid (messages); + notmuch_messages_move_to_next (messages)) { + notmuch_message_t *message = notmuch_messages_get (messages); + const char *mid = notmuch_message_get_message_id (message); + printf("%s\n", mid); + } + } +EOF +cat < EXPECTED +== stdout == +20091117190054.GU3165@dottiness.seas.harvard.edu +87iqd9rn3l.fsf@vertex.dottedmag +20091117203301.GV3165@dottiness.seas.harvard.edu +87fx8can9z.fsf@vertex.dottedmag +yunaayketfm.fsf@aiko.keithp.com +20091118005040.GA25380@dottiness.seas.harvard.edu +87ocn0qh6d.fsf@yoom.home.cworth.org +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get authors from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *authors; + authors = notmuch_thread_get_authors (thread); + printf("%d\n%s\n", thread != NULL, authors); + } +EOF +cat < EXPECTED +== stdout == +1 +Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "get subject from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + const char *subject; + subject = notmuch_thread_get_subject (thread); + printf("%d\n%s\n", thread != NULL, subject); + } +EOF +cat < EXPECTED +== stdout == +1 +[notmuch] Working with Maildir storage? +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "oldest date from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + time_t stamp; + stamp = notmuch_thread_get_oldest_date (thread); + printf("%d\n%d\n", thread != NULL, stamp > 0); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "newest date from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + time_t stamp; + stamp = notmuch_thread_get_newest_date (thread); + printf("%d\n%d\n", thread != NULL, stamp > 0); + } +EOF +cat < EXPECTED +== stdout == +1 +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "iterate tags from closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_tags_t *tags; + const char *tag; + for (tags = notmuch_thread_get_tags (thread); + notmuch_tags_valid (tags); + notmuch_tags_move_to_next (tags)) + { + tag = notmuch_tags_get (tags); + printf ("%s\n", tag); + } + } +EOF +cat < EXPECTED +== stdout == +inbox +signed +unread +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "collect tags with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + notmuch_messages_t *messages = notmuch_thread_get_messages (thread); + + notmuch_tags_t *tags = notmuch_messages_collect_tags (messages); + + const char *tag; + for (tags = notmuch_thread_get_tags (thread); + notmuch_tags_valid (tags); + notmuch_tags_move_to_next (tags)) + { + tag = notmuch_tags_get (tags); + printf ("%s\n", tag); + } + notmuch_tags_destroy (tags); + notmuch_messages_destroy (messages); + + printf("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +inbox +signed +unread +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "destroy thread with closed database" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} + { + time_t stamp; + notmuch_thread_destroy (thread); + printf("SUCCESS\n"); + } +EOF +cat < EXPECTED +== stdout == +SUCCESS +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T570-revision-tracking.sh b/test/T570-revision-tracking.sh index a59e7c98..bcc97dd9 100755 --- a/test/T570-revision-tracking.sh +++ b/test/T570-revision-tracking.sh @@ -19,7 +19,12 @@ int main (int argc, char** argv) unsigned long rev; - stat = notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, &db); + char* msg = NULL; + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); + if (stat) fputs ("open failed\n", stderr); revision = notmuch_database_get_revision (db, &uuid); @@ -90,4 +95,31 @@ subtotal=$(notmuch count lastmod:..$lastmod) result=$(($subtotal == $total-1)) test_expect_equal 1 "$result" +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + test_begin_subtest 'exclude one message using negative lastmod (sexp)' + total=$(notmuch count '*') + notmuch tag +${RANDOM} id:4EFC743A.3060609@april.org + count=$(notmuch count --query=sexp '(lastmod -1 *)') + test_expect_equal 1 "$count" +fi + +test_begin_subtest 'exclude one message using negative lastmod' +total=$(notmuch count '*') +notmuch tag +${RANDOM} id:4EFC743A.3060609@april.org +count=$(notmuch count lastmod:-1..) +test_expect_equal 1 "$count" + +test_begin_subtest 'exclude one message using negative lastmod (second param)' +total=$(notmuch count '*') +notmuch tag +${RANDOM} id:4EFC743A.3060609@april.org +count=$(notmuch count lastmod:..-1) +test_expect_equal 51 "$count" + +test_begin_subtest 'negative lastmod (two parameters)' +notmuch tag +${RANDOM} '*' +before=$(notmuch count --lastmod '*' | cut -f3) +notmuch tag +${RANDOM} id:4EFC743A.3060609@april.org +count=$(notmuch count lastmod:-100..$before) +test_expect_equal 51 "$count" + test_done diff --git a/test/T585-thread-subquery.sh b/test/T585-thread-subquery.sh index bf9894d3..71ced149 100755 --- a/test/T585-thread-subquery.sh +++ b/test/T585-thread-subquery.sh @@ -14,9 +14,6 @@ count=$(notmuch count from:keithp and to:keithp) test_expect_equal 0 "$count" test_begin_subtest "Same query against threads" -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then - test_subtest_known_broken -fi notmuch search thread:{from:keithp} and thread:{to:keithp} | notmuch_search_sanitize > OUTPUT cat< EXPECTED thread:XXX 2009-11-18 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) @@ -24,9 +21,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Mix thread and non-threads query" -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then - test_subtest_known_broken -fi notmuch search thread:{from:keithp} and to:keithp | notmuch_search_sanitize > OUTPUT cat< EXPECTED thread:XXX 2009-11-18 [1/7] Lars Kellogg-Stedman| Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) @@ -34,9 +28,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Compound subquery" -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then - test_subtest_known_broken -fi notmuch search 'thread:"{from:keithp and date:2009}" and thread:{to:keithp}' | notmuch_search_sanitize > OUTPUT cat< EXPECTED thread:XXX 2009-11-18 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth; [notmuch] Working with Maildir storage? (inbox signed unread) @@ -44,9 +35,6 @@ EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "Syntax/quoting error in subquery" -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then - test_subtest_known_broken -fi notmuch search 'thread:{from:keithp and date:2009} and thread:{to:keithp}' 1>OUTPUT 2>&1 cat< EXPECTED notmuch search: A Xapian exception occurred diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh index 46f3a76d..9326ba3e 100755 --- a/test/T590-libconfig.sh +++ b/test/T590-libconfig.sh @@ -5,9 +5,24 @@ test_description="library config API" add_email_corpus +_libconfig_sanitize() { + ${NOTMUCH_PYTHON} /dev/fd/3 3<<'EOF' +import os, sys, pwd, socket + +pw = pwd.getpwuid(os.getuid()) +user = pw.pw_name +name = pw.pw_gecos.partition(",")[0] + +for l in sys.stdin: + if l[:4] == "08: ": + l = l.replace(user, "USERNAME", 1) + elif l[:4] == "10: ": + l = l.replace("'" + name, "'USER_FULL_NAME", 1) + sys.stdout.write(l) +EOF +} + cat < c_head -#include -#include #include int main (int argc, char** argv) @@ -15,9 +30,21 @@ int main (int argc, char** argv) notmuch_database_t *db; char *val; notmuch_status_t stat; + char *msg = NULL; - EXPECT0(notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, &db)); + for (int i = 1; i < argc; i++) + if (strcmp (argv[i], "%NULL%") == 0) argv[i] = NULL; + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + argv[2], + argv[3], + &db, + &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database\n%s\n%s\n", notmuch_status_to_string (stat), msg ? msg : ""); + exit (1); + } EOF cat < c_tail @@ -26,27 +53,27 @@ cat < c_tail EOF test_begin_subtest "notmuch_database_{set,get}_config" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} { - EXPECT0(notmuch_database_set_config (db, "testkey1", "testvalue1")); - EXPECT0(notmuch_database_set_config (db, "testkey2", "testvalue2")); - EXPECT0(notmuch_database_get_config (db, "testkey1", &val)); - printf("testkey1 = %s\n", val); - EXPECT0(notmuch_database_get_config (db, "testkey2", &val)); - printf("testkey2 = %s\n", val); + EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1")); + EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2")); + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); } EOF cat <<'EOF' >EXPECTED == stdout == -testkey1 = testvalue1 -testkey2 = testvalue2 +test.key1 = testvalue1 +test.key2 = testvalue2 == stderr == EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "notmuch_database_get_config_list: empty list" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% { notmuch_config_list_t *list; EXPECT0(notmuch_database_get_config_list (db, "nonexistent", &list)); @@ -61,9 +88,24 @@ valid = 0 EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "notmuch_database_get_config_list: closed db" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +{ + notmuch_config_list_t *list; + EXPECT0(notmuch_database_close (db)); + stat = notmuch_database_get_config_list (db, "nonexistent", &list); + printf("%d\n", stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "notmuch_database_get_config_list: all pairs" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% { notmuch_config_list_t *list; EXPECT0(notmuch_database_set_config (db, "zzzafter", "afterval")); @@ -78,18 +120,40 @@ EOF cat <<'EOF' >EXPECTED == stdout == aaabefore beforeval -testkey1 testvalue1 -testkey2 testvalue2 +test.key1 testvalue1 +test.key2 testvalue2 zzzafter afterval == stderr == EOF test_expect_equal_file EXPECTED OUTPUT -test_begin_subtest "notmuch_database_get_config_list: one prefix" +test_begin_subtest "notmuch_database_get_config_list: all pairs (closed db)" cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} { notmuch_config_list_t *list; - EXPECT0(notmuch_database_get_config_list (db, "testkey", &list)); + EXPECT0(notmuch_database_get_config_list (db, "", &list)); + EXPECT0(notmuch_database_close (db)); + for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { + printf("%s %d\n", notmuch_config_list_key (list), NULL == notmuch_config_list_value(list)); + } + notmuch_config_list_destroy (list); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +aaabefore 1 +test.key1 1 +test.key2 1 +zzzafter 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_database_get_config_list: one prefix" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_list_t *list; + EXPECT0(notmuch_database_get_config_list (db, "test.key", &list)); for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { printf("%s %s\n", notmuch_config_list_key (list), notmuch_config_list_value(list)); } @@ -98,14 +162,14 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} EOF cat <<'EOF' >EXPECTED == stdout == -testkey1 testvalue1 -testkey2 testvalue2 +test.key1 testvalue1 +test.key2 testvalue2 == stderr == EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "dump config" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% { EXPECT0(notmuch_database_set_config (db, "key with spaces", "value, with, spaces!")); } @@ -115,21 +179,867 @@ cat <<'EOF' >EXPECTED #notmuch-dump batch-tag:3 config #@ aaabefore beforeval #@ key%20with%20spaces value,%20with,%20spaces%21 -#@ testkey1 testvalue1 -#@ testkey2 testvalue2 +#@ test.key1 testvalue1 +#@ test.key2 testvalue2 #@ zzzafter afterval EOF test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "restore config" notmuch dump --include=config >EXPECTED -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% { - EXPECT0(notmuch_database_set_config (db, "testkey1", "mutatedvalue")); + EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue")); } EOF notmuch restore --include=config OUTPUT test_expect_equal_file EXPECTED OUTPUT +backup_database +test_begin_subtest "override config from file" +notmuch config set test.key1 overridden +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: traditional" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR); + printf("database.hook_dir = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +database.hook_dir = MAIL_DIR/.notmuch/hooks +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "NOTMUCH_CONFIG_HOOK_DIR: xdg" +dir="${HOME}/.config/notmuch/default/hooks" +mkdir -p $dir +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + const char *val = notmuch_config_get (db, NOTMUCH_CONFIG_HOOK_DIR); + printf("database.hook_dir = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +database.hook_dir = CWD/home/.config/notmuch/default/hooks +== stderr == +EOF +rmdir $dir +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_config_get_values" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_values_t *values; + EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c")); + for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +a +b +c +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "notmuch_config_get_values (ignore leading/trailing whitespace)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_values_t *values; + EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, " a ; b c ; d ")); + for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +a +b c +d +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "notmuch_config_get_values_string" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_values_t *values; + EXPECT0(notmuch_database_set_config (db, "test.list", "x;y;z")); + for (values = notmuch_config_get_values_string (db, "test.list"); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +x +y +z +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "notmuch_config_get_values (restart)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_values_t *values; + EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c")); + for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } + for (notmuch_config_values_start (values); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +a +b +c +a +b +c +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "notmuch_config_get_values, trailing ;" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_values_t *values; + EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_NEW_TAGS, "a;b;c")); + for (values = notmuch_config_get_values (db, NOTMUCH_CONFIG_NEW_TAGS); + notmuch_config_values_valid (values); + notmuch_config_values_move_to_next (values)) + { + puts (notmuch_config_values_get (values)); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +a +b +c +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "get config by key" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} +{ + printf("before = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS)); + EXPECT0(notmuch_database_set_config (db, "maildir.synchronize_flags", "false")); + printf("after = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS)); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +before = true +after = false +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "set config by key" +notmuch config set test.key1 overridden +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} +{ + printf("before = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS)); + EXPECT0(notmuch_config_set (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS, "false")); + printf("after = %s\n", notmuch_config_get (db, NOTMUCH_CONFIG_SYNC_MAILDIR_FLAGS)); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +before = true +after = false +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "load default values" +export MAILDIR=${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} '' %NULL% +{ + notmuch_config_key_t key; + for (key = NOTMUCH_CONFIG_FIRST; + key < NOTMUCH_CONFIG_LAST; + key = (notmuch_config_key_t)(key + 1)) { + const char *val = notmuch_config_get (db, key); + printf("%02d: '%s'\n", key, val ? val : "NULL" ); + } +} +EOF + +_libconfig_sanitize < OUTPUT > OUTPUT.clean + +cat <<'EOF' >EXPECTED +== stdout == +00: 'MAIL_DIR' +01: 'MAIL_DIR' +02: 'MAIL_DIR/.notmuch/hooks' +03: 'MAIL_DIR/.notmuch/backups' +04: '' +05: 'unread;inbox' +06: '' +07: 'true' +08: 'USERNAME@localhost' +09: 'NULL' +10: 'USER_FULL_NAME' +11: '8000' +12: 'NULL' +13: '' +== stderr == +EOF +unset MAILDIR +test_expect_equal_file EXPECTED OUTPUT.clean + +backup_database +test_begin_subtest "override config from \${NOTMUCH_CONFIG}" +notmuch config set test.key1 overridden +# second argument omitted to make argv[2] == NULL +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +notmuch config set test.key1 +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "override config from \${HOME}/.notmuch-config" +ovconfig=${HOME}/.notmuch-config +cp ${NOTMUCH_CONFIG} ${ovconfig} +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +notmuch --config=${ovconfig} config set test.key1 overridden-home +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +rm -f ${ovconfig} +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden-home +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch" +ovconfig=${HOME}/.config/notmuch/default/config +mkdir -p $(dirname ${ovconfig}) +cp ${NOTMUCH_CONFIG} ${ovconfig} +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +notmuch --config=${ovconfig} config set test.key1 overridden-xdg +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +rm -f ${ovconfig} +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden-xdg +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "override config from \${XDG_CONFIG_HOME}/notmuch with profile" +ovconfig=${HOME}/.config/notmuch/work/config +mkdir -p $(dirname ${ovconfig}) +cp ${NOTMUCH_CONFIG} ${ovconfig} +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +notmuch --config=${ovconfig} config set test.key1 overridden-xdg-profile +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +rm -f ${ovconfig} +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden-xdg-profile +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest "override config from \${HOME}/.notmuch-config.work (via args)" +ovconfig=${HOME}/.notmuch-config.work +cp ${NOTMUCH_CONFIG} ${ovconfig} +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +notmuch --config=${ovconfig} config set test.key1 overridden-profile +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% work +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +#rm -f ${ovconfig} +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden-profile +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "no config, fail to open database" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +cat c_head - c_tail <<'EOF' | test_C %NULL% '' %NULL% +{ + printf("NOT RUN"); +} +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +== stderr == +error opening database +No database found +Error: could not locate database. + +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "open database from NOTMUCH_DATABASE" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +export NOTMUCH_DATABASE=${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C %NULL% '' %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +unset NOTMUCH_DATABASE +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "NOTMUCH_DATABASE overrides config" +cp notmuch-config notmuch-config.bak +notmuch config set database.path /nonexistent +export NOTMUCH_DATABASE=${MAIL_DIR} +cat c_head - c_tail <<'EOF' | test_C %NULL% '' %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +unset NOTMUCH_DATABASE +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +cp notmuch-config.bak notmuch-config +test_expect_equal_file EXPECTED OUTPUT + +cat < c_head2 +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + char *val; + notmuch_status_t stat; + char *msg = NULL; + + for (int i = 1; i < argc; i++) + if (strcmp (argv[i], "%NULL%") == 0) argv[i] = NULL; + + stat = notmuch_database_load_config (argv[1], + argv[2], + argv[3], + &db, + &msg); + if (stat != NOTMUCH_STATUS_SUCCESS && stat != NOTMUCH_STATUS_NO_CONFIG) { + fprintf (stderr, "error opening database\n%d: %s\n%s\n", stat, + notmuch_status_to_string (stat), msg ? msg : ""); + exit (1); + } +EOF + + +test_begin_subtest "notmuch_database_get_config (ndlc)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + + +test_begin_subtest "notmuch_database_get_config_list: all pairs (ndlc)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_list_t *list; + EXPECT0(notmuch_database_get_config_list (db, "", &list)); + for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { + printf("%s %s\n", notmuch_config_list_key (list), notmuch_config_list_value(list)); + } + notmuch_config_list_destroy (list); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +aaabefore beforeval +key with spaces value, with, spaces! +test.key1 testvalue1 +test.key2 testvalue2 +zzzafter afterval +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_database_get_config_list: one prefix (ndlc)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_list_t *list; + EXPECT0(notmuch_database_get_config_list (db, "test.key", &list)); + for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next (list)) { + printf("%s %s\n", notmuch_config_list_key (list), notmuch_config_list_value(list)); + } + notmuch_config_list_destroy (list); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 testvalue1 +test.key2 testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "list by keys (ndlc)" +notmuch config set search.exclude_tags "foo;bar;fub" +notmuch config set new.ignore "sekrit_junk" +notmuch config set index.as_text "text/" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL% +{ + notmuch_config_key_t key; + for (key = NOTMUCH_CONFIG_FIRST; + key < NOTMUCH_CONFIG_LAST; + key = (notmuch_config_key_t)(key + 1)) { + const char *val = notmuch_config_get (db, key); + printf("%02d: '%s'\n", key, val ? val : "NULL" ); + } +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +00: 'MAIL_DIR' +01: 'MAIL_DIR' +02: 'MAIL_DIR/.notmuch/hooks' +03: 'MAIL_DIR/.notmuch/backups' +04: 'foo;bar;fub' +05: 'unread;inbox' +06: 'sekrit_junk' +07: 'true' +08: 'test_suite@notmuchmail.org' +09: 'test_suite_other@notmuchmail.org;test_suite@otherdomain.org' +10: 'Notmuch Test Suite' +11: '8000' +12: 'NULL' +13: 'text/' +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "load default values (ndlc, nonexistent config)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} /nonexistent %NULL% +{ + notmuch_config_key_t key; + for (key = NOTMUCH_CONFIG_FIRST; + key < NOTMUCH_CONFIG_LAST; + key = (notmuch_config_key_t)(key + 1)) { + const char *val = notmuch_config_get (db, key); + printf("%02d: '%s'\n", key, val ? val : "NULL" ); + } +} +EOF + +_libconfig_sanitize < OUTPUT > OUTPUT.clean + +cat <<'EOF' >EXPECTED +== stdout == +00: 'MAIL_DIR' +01: 'MAIL_DIR' +02: 'MAIL_DIR/.notmuch/hooks' +03: 'MAIL_DIR/.notmuch/backups' +04: '' +05: 'unread;inbox' +06: '' +07: 'true' +08: 'USERNAME@localhost' +09: 'NULL' +10: 'USER_FULL_NAME' +11: '8000' +12: 'NULL' +13: '' +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT.clean + +backup_database +test_begin_subtest "override config from \${HOME}/.notmuch-config (ndlc)" +ovconfig=${HOME}/.notmuch-config +cp ${NOTMUCH_CONFIG} ${ovconfig} +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +notmuch --config=${ovconfig} config set test.key1 overridden-home +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} %NULL% %NULL% +{ + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +rm -f ${ovconfig} +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = overridden-home +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT +restore_database + +test_begin_subtest "notmuch_config_get_pairs: prefix (ndlc)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_pairs_t *list; + for (list = notmuch_config_get_pairs (db, "user."); + notmuch_config_pairs_valid (list); + notmuch_config_pairs_move_to_next (list)) { + printf("%s %s\n", notmuch_config_pairs_key (list), notmuch_config_pairs_value(list)); + } + notmuch_config_pairs_destroy (list); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +user.name Notmuch Test Suite +user.other_email test_suite_other@notmuchmail.org;test_suite@otherdomain.org +user.primary_email test_suite@notmuchmail.org +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_config_get_pairs: all pairs (ndlc)" +cat c_head2 - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL% +{ + notmuch_config_pairs_t *list; + for (list = notmuch_config_get_pairs (db, ""); + notmuch_config_pairs_valid (list); + notmuch_config_pairs_move_to_next (list)) { + printf("%s %s\n", notmuch_config_pairs_key (list), notmuch_config_pairs_value(list)); + } + notmuch_config_pairs_destroy (list); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +aaabefore beforeval +database.autocommit 8000 +database.backup_dir MAIL_DIR/.notmuch/backups +database.hook_dir MAIL_DIR/.notmuch/hooks +database.mail_root MAIL_DIR +database.path MAIL_DIR +index.as_text text/ +key with spaces value, with, spaces! +maildir.synchronize_flags true +new.ignore sekrit_junk +new.tags unread;inbox +search.exclude_tags foo;bar;fub +show.extra_headers (null) +test.key1 testvalue1 +test.key2 testvalue2 +user.name Notmuch Test Suite +user.other_email test_suite_other@notmuchmail.org;test_suite@otherdomain.org +user.primary_email test_suite@notmuchmail.org +zzzafter afterval +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +cat < c_head3 +#include +int main (int argc, char **argv) { + notmuch_status_t stat; + notmuch_database_t *db = NULL; +EOF + +cat < c_tail3 + printf("db == NULL: %d\n", db == NULL); +} +EOF + +test_begin_subtest "open: database set to null on missing config" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} + notmuch_status_t st = notmuch_database_open_with_config(argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "/nonexistent", NULL, &db, NULL); +EOF +cat < EXPECTED +== stdout == +db == NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "open: database set to null on missing config (env)" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +export NOTMUCH_CONFIG="/nonexistent" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} + notmuch_status_t st = notmuch_database_open_with_config(argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + NULL, NULL, &db, NULL); +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat < EXPECTED +== stdout == +db == NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "create: database set to null on missing config" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} "/nonexistent" + notmuch_status_t st = notmuch_database_create_with_config(argv[1],argv[2], NULL, &db, NULL); +EOF +cat < EXPECTED +== stdout == +db == NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "create: database set to null on missing config (env)" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +export NOTMUCH_CONFIG="/nonexistent" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} + notmuch_status_t st = notmuch_database_create_with_config(argv[1], + NULL, NULL, &db, NULL); +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat < EXPECTED +== stdout == +db == NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "load_config: database set non-null on missing config" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} "/nonexistent" + notmuch_status_t st = notmuch_database_load_config(argv[1],argv[2], NULL, &db, NULL); +EOF +cat < EXPECTED +== stdout == +db == NULL: 0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "load_config: database non-null on missing config (env)" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +export NOTMUCH_CONFIG="/nonexistent" +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} + notmuch_status_t st = notmuch_database_load_config(argv[1], NULL, NULL, &db, NULL); +EOF +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +cat < EXPECTED +== stdout == +db == NULL: 0 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "load_config: database set to NULL on fatal error" +cat c_head3 - c_tail3 <<'EOF' | test_C + notmuch_status_t st = notmuch_database_load_config("relative", NULL, NULL, &db, NULL); +EOF +cat < EXPECTED +== stdout == +db == NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "open: database parameter overrides implicit config" +cp $NOTMUCH_CONFIG ${NOTMUCH_CONFIG}.bak +notmuch config set database.path ${MAIL_DIR}/nonexistent +cat c_head3 - c_tail3 <<'EOF' | test_C ${MAIL_DIR} + const char *path = NULL; + notmuch_status_t st = notmuch_database_open_with_config(argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + NULL, NULL, &db, NULL); + printf ("status: %d\n", st); + path = notmuch_database_get_path (db); + printf ("path: %s\n", path ? path : "(null)"); +EOF +cp ${NOTMUCH_CONFIG}.bak ${NOTMUCH_CONFIG} +cat < EXPECTED +== stdout == +status: 0 +path: MAIL_DIR +db == NULL: 0 +== stderr == +EOF +notmuch_dir_sanitize < OUTPUT > OUTPUT.clean +test_expect_equal_file EXPECTED OUTPUT.clean + +cat < c_body + notmuch_status_t st = notmuch_database_open_with_config(NULL, + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, NULL); + printf ("status == SUCCESS: %d\n", st == NOTMUCH_STATUS_SUCCESS); + if (db) { + const char *mail_root = NULL; + mail_root = notmuch_config_get (db, NOTMUCH_CONFIG_MAIL_ROOT); + printf ("mail_root: %s\n", mail_root ? mail_root : "(null)"); + } +EOF + +cat < EXPECTED.common +== stdout == +status == SUCCESS: 0 +db == NULL: 1 +== stderr == +EOF + +test_begin_subtest "open/error: config=empty with no mail root in db " +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +cat c_head3 c_body c_tail3 | test_C +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +notmuch_dir_sanitize < OUTPUT > OUTPUT.clean +test_expect_equal_file EXPECTED.common OUTPUT.clean + +test_begin_subtest "open/error: config=empty with no mail root in db (xdg)" +old_NOTMUCH_CONFIG=${NOTMUCH_CONFIG} +unset NOTMUCH_CONFIG +backup_database +mkdir -p home/.local/share/notmuch +mv mail/.notmuch home/.local/share/notmuch/default +cat c_head3 c_body c_tail3 | test_C +restore_database +export NOTMUCH_CONFIG=${old_NOTMUCH_CONFIG} +notmuch_dir_sanitize < OUTPUT > OUTPUT.clean +test_expect_equal_file EXPECTED.common OUTPUT.clean + test_done diff --git a/test/T590-thread-breakage.sh b/test/T590-thread-breakage.sh deleted file mode 100755 index aeb82cf4..00000000 --- a/test/T590-thread-breakage.sh +++ /dev/null @@ -1,124 +0,0 @@ -#!/usr/bin/env bash -# -# Copyright (c) 2016 Daniel Kahn Gillmor -# - -test_description='thread breakage during reindexing' - -# notmuch uses ghost documents to track messages we have seen references -# to but have never seen. Regardless of the order of delivery, message -# deletion, and reindexing, the list of ghost messages for a given -# stored corpus should not vary, so that threads can be reassmebled -# cleanly. -# -# In practice, we accept a small amount of variation (and therefore -# traffic pattern metadata leakage to be stored in the index) for the -# sake of efficiency. -# -# This test also embeds some subtests to ensure that indexing actually -# works properly and attempted fixes to threading issues do not break -# the expected contents of the index. - -. $(dirname "$0")/test-lib.sh || exit 1 - -message_a() { - mkdir -p ${MAIL_DIR}/cur - cat > ${MAIL_DIR}/cur/a < -From: Alice -To: Bob -Date: Thu, 31 Mar 2016 20:10:00 -0400 - -This is the first message in the thread. -Apple -EOF -} - -message_b() { - mkdir -p ${MAIL_DIR}/cur - cat > ${MAIL_DIR}/cur/b < -In-Reply-To: -References: -From: Bob -To: Alice -Date: Thu, 31 Mar 2016 20:15:00 -0400 - -This is the second message in the thread. -Banana -EOF -} - - -test_content_count() { - test_begin_subtest "${3:-looking for $2 instance of '$1'}" - count=$(notmuch count --output=threads "$1") - test_expect_equal "$count" "$2" -} - -test_thread_count() { - test_begin_subtest "${2:-Expecting $1 thread(s)}" - count=$(notmuch count --output=threads) - test_expect_equal "$count" "$1" -} - -test_ghost_count() { - test_begin_subtest "${2:-Expecting $1 ghosts(s)}" - ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian) - test_expect_equal "$ghosts" "$1" -} - -notmuch new >/dev/null - -test_thread_count 0 'There should be no threads initially' -test_ghost_count 0 'There should be no ghosts initially' - -message_a -notmuch new >/dev/null -test_thread_count 1 'One message in: one thread' -test_content_count apple 1 -test_content_count banana 0 -test_ghost_count 0 - -message_b -notmuch new >/dev/null -test_thread_count 1 'Second message in the same thread: one thread' -test_content_count apple 1 -test_content_count banana 1 -test_ghost_count 0 - -rm -f ${MAIL_DIR}/cur/a -notmuch new >/dev/null -test_thread_count 1 'First message removed: still only one thread' -test_content_count apple 0 -test_content_count banana 1 -test_ghost_count 1 'should be one ghost after first message removed' - -message_a -notmuch new >/dev/null -test_thread_count 1 'First message reappears: should return to the same thread' -test_content_count apple 1 -test_content_count banana 1 -test_ghost_count 0 - -rm -f ${MAIL_DIR}/cur/b -notmuch new >/dev/null -test_thread_count 1 'Removing second message: still only one thread' -test_content_count apple 1 -test_content_count banana 0 -test_begin_subtest 'No ghosts should remain after deletion of second message' -# this is known to fail; we are leaking ghost messages deliberately -test_subtest_known_broken -ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian) -test_expect_equal "$ghosts" "0" - -rm -f ${MAIL_DIR}/cur/a -notmuch new >/dev/null -test_thread_count 0 'All messages gone: no threads' -test_content_count apple 0 -test_content_count banana 0 -test_ghost_count 0 'No ghosts should remain after full thread deletion' - -test_done diff --git a/test/T592-thread-breakage.sh b/test/T592-thread-breakage.sh new file mode 100755 index 00000000..2334fcaf --- /dev/null +++ b/test/T592-thread-breakage.sh @@ -0,0 +1,128 @@ +#!/usr/bin/env bash +# +# Copyright (c) 2016 Daniel Kahn Gillmor +# + +test_description='thread breakage during reindexing' + +# notmuch uses ghost documents to track messages we have seen references +# to but have never seen. Regardless of the order of delivery, message +# deletion, and reindexing, the list of ghost messages for a given +# stored corpus should not vary, so that threads can be reassmebled +# cleanly. +# +# In practice, we accept a small amount of variation (and therefore +# traffic pattern metadata leakage to be stored in the index) for the +# sake of efficiency. +# +# This test also embeds some subtests to ensure that indexing actually +# works properly and attempted fixes to threading issues do not break +# the expected contents of the index. + +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + +message_a () { + mkdir -p ${MAIL_DIR}/cur + cat > ${MAIL_DIR}/cur/a < +From: Alice +To: Bob +Date: Thu, 31 Mar 2016 20:10:00 -0400 + +This is the first message in the thread. +Apple +EOF +} + +message_b () { + mkdir -p ${MAIL_DIR}/cur + cat > ${MAIL_DIR}/cur/b < +In-Reply-To: +References: +From: Bob +To: Alice +Date: Thu, 31 Mar 2016 20:15:00 -0400 + +This is the second message in the thread. +Banana +EOF +} + + +test_content_count () { + test_begin_subtest "${3:-looking for $2 instance of '$1'}" + count=$(notmuch count --output=threads "$1") + test_expect_equal "$count" "$2" +} + +test_thread_count () { + test_begin_subtest "${2:-Expecting $1 thread(s)}" + count=$(notmuch count --output=threads) + test_expect_equal "$count" "$1" +} + +test_ghost_count () { + test_begin_subtest "${2:-Expecting $1 ghosts(s)}" + ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian) + test_expect_equal "$ghosts" "$1" +} + +notmuch new >/dev/null + +test_thread_count 0 'There should be no threads initially' +test_ghost_count 0 'There should be no ghosts initially' + +message_a +notmuch new >/dev/null +test_thread_count 1 'One message in: one thread' +test_content_count apple 1 +test_content_count banana 0 +test_ghost_count 0 + +message_b +notmuch new >/dev/null +test_thread_count 1 'Second message in the same thread: one thread' +test_content_count apple 1 +test_content_count banana 1 +test_ghost_count 0 + +rm -f ${MAIL_DIR}/cur/a +notmuch new >/dev/null +test_thread_count 1 'First message removed: still only one thread' +test_content_count apple 0 +test_content_count banana 1 +test_ghost_count 1 'should be one ghost after first message removed' + +message_a +notmuch new >/dev/null +test_thread_count 1 'First message reappears: should return to the same thread' +test_content_count apple 1 +test_content_count banana 1 +test_ghost_count 0 + +rm -f ${MAIL_DIR}/cur/b +notmuch new >/dev/null +test_thread_count 1 'Removing second message: still only one thread' +test_content_count apple 1 +test_content_count banana 0 +test_begin_subtest 'No ghosts should remain after deletion of second message' +# this is known to fail; we are leaking ghost messages deliberately +test_subtest_known_broken +ghosts=$($NOTMUCH_BUILDDIR/test/ghost-report ${MAIL_DIR}/.notmuch/xapian) +test_expect_equal "$ghosts" "0" + +rm -f ${MAIL_DIR}/cur/a +notmuch new >/dev/null +test_thread_count 0 'All messages gone: no threads' +test_content_count apple 0 +test_content_count banana 0 +test_ghost_count 0 'No ghosts should remain after full thread deletion' + +test_done diff --git a/test/T595-reopen.sh b/test/T595-reopen.sh new file mode 100755 index 00000000..1a517423 --- /dev/null +++ b/test/T595-reopen.sh @@ -0,0 +1,123 @@ +#!/usr/bin/env bash +test_description="library reopen API" + +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus + +cat < c_head +#include + +int main (int argc, char** argv) +{ + notmuch_database_t *db; + char *val; + notmuch_status_t stat; + notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY; + + char *msg = NULL; + + for (int i = 1; i < argc; i++) + if (strcmp (argv[i], "%NULL%") == 0) argv[i] = NULL; + + if (argv[2] && (argv[2][0] == 'w' || argv[2][0] == 'W')) + mode = NOTMUCH_DATABASE_MODE_READ_WRITE; + + stat = notmuch_database_open_with_config (argv[1], + mode, + argv[3], + argv[4], + &db, + &msg); + if (stat != NOTMUCH_STATUS_SUCCESS) { + fprintf (stderr, "error opening database: %d %s\n", stat, msg ? msg : ""); + exit (1); + } +EOF + +cat < c_tail + EXPECT0(notmuch_database_destroy(db)); +} +EOF + +# The sequence of tests is important here + +test_begin_subtest "notmuch_database_reopen (read=>write)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} read ${NOTMUCH_CONFIG} +{ + EXPECT0(notmuch_database_reopen (db, NOTMUCH_DATABASE_MODE_READ_WRITE)); + EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1")); + EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2")); + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_database_reopen (read=>read)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} read ${NOTMUCH_CONFIG} +{ + EXPECT0(notmuch_database_reopen (db, NOTMUCH_DATABASE_MODE_READ_ONLY)); + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_database_reopen (write=>read)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} write ${NOTMUCH_CONFIG} +{ + EXPECT0(notmuch_database_reopen (db, NOTMUCH_DATABASE_MODE_READ_ONLY)); + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_database_reopen (write=>write)" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} write ${NOTMUCH_CONFIG} +{ + EXPECT0(notmuch_database_reopen (db, NOTMUCH_DATABASE_MODE_READ_WRITE)); + EXPECT0(notmuch_database_set_config (db, "test.key3", "testvalue3")); + EXPECT0(notmuch_database_get_config (db, "test.key1", &val)); + printf("test.key1 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key2", &val)); + printf("test.key2 = %s\n", val); + EXPECT0(notmuch_database_get_config (db, "test.key3", &val)); + printf("test.key3 = %s\n", val); +} +EOF +cat <<'EOF' >EXPECTED +== stdout == +test.key1 = testvalue1 +test.key2 = testvalue2 +test.key3 = testvalue3 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T600-named-queries.sh b/test/T600-named-queries.sh index abaee3b7..a7b84995 100755 --- a/test/T600-named-queries.sh +++ b/test/T600-named-queries.sh @@ -4,13 +4,13 @@ test_description='named queries' QUERYSTR="date:2009-11-18..2009-11-18 and tag:unread" -test_begin_subtest "error adding named query before initializing DB" -test_expect_code 1 "notmuch config set query.test \"$QUERYSTR\"" +test_begin_subtest "error adding named query to DB before initialization" +test_expect_code 1 "notmuch config set --database query.test \"$QUERYSTR\"" add_email_corpus -test_begin_subtest "adding named query" -test_expect_success "notmuch config set query.test \"$QUERYSTR\"" +test_begin_subtest "adding named query (database)" +test_expect_success "notmuch config set --database query.test \"$QUERYSTR\"" test_begin_subtest "adding nested named query" QUERYSTR2="query:test and subject:Maildir" @@ -32,16 +32,28 @@ test_begin_subtest "dump named queries" notmuch dump | grep '^#@' > OUTPUT cat< QUERIES.BEFORE #@ query.test date%3a2009-11-18..2009-11-18%20and%20tag%3aunread -#@ query.test2 query%3atest%20and%20subject%3aMaildir EOF test_expect_equal_file QUERIES.BEFORE OUTPUT +test_begin_subtest 'dumping large queries' +# This value is just large enough to trigger a limitation of gzprintf +# to 8191 bytes in total (by default). +repeat=1329 +notmuch config set --database query.big "$(seq -s' ' $repeat)" +notmuch dump --include=config > OUTPUT +notmuch config set --database query.big +printf "#notmuch-dump batch-tag:3 config\n#@ query.big " > EXPECTED +seq -s'%20' $repeat >> EXPECTED +cat <> EXPECTED +#@ query.test date%3a2009-11-18..2009-11-18%20and%20tag%3aunread +EOF +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest "delete named queries" notmuch dump > BEFORE -notmuch config set query.test +notmuch config set --database query.test notmuch dump | grep '^#@' > OUTPUT cat< EXPECTED -#@ query.test2 query%3atest%20and%20subject%3aMaildir EOF test_expect_equal_file EXPECTED OUTPUT @@ -53,17 +65,11 @@ test_expect_equal_file QUERIES.BEFORE OUTPUT test_begin_subtest "search named query" notmuch search query:test > OUTPUT notmuch search $QUERYSTR > EXPECTED -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then - test_subtest_known_broken -fi test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "search named query with other terms" notmuch search query:test and subject:Maildir > OUTPUT notmuch search $QUERYSTR and subject:Maildir > EXPECTED -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -ne 1 ]; then - test_subtest_known_broken -fi test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "search nested named query" diff --git a/test/T610-message-property.sh b/test/T610-message-property.sh index 53a0be3b..a7cbe048 100755 --- a/test/T610-message-property.sh +++ b/test/T610-message-property.sh @@ -6,17 +6,13 @@ test_description="message property API" add_email_corpus cat < c_head -#include -#include -#include -#include #include void print_properties (notmuch_message_t *message, const char *prefix, notmuch_bool_t exact) { notmuch_message_properties_t *list; for (list = notmuch_message_get_properties (message, prefix, exact); notmuch_message_properties_valid (list); notmuch_message_properties_move_to_next (list)) { - printf("%s\n", notmuch_message_properties_value(list)); + printf("%s = %s\n", notmuch_message_properties_key(list), notmuch_message_properties_value(list)); } notmuch_message_properties_destroy (list); } @@ -27,8 +23,12 @@ int main (int argc, char** argv) notmuch_message_t *message = NULL; const char *val; notmuch_status_t stat; + char* msg = NULL; - EXPECT0(notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, &db)); + EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &db, &msg)); + if (msg) fputs (msg, stderr); EXPECT0(notmuch_database_find_message(db, "4EFC743A.3060609@april.org", &message)); if (message == NULL) { fprintf (stderr, "unable to find message"); @@ -65,7 +65,7 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} EXPECT0(notmuch_message_get_property (message, "testkey2", &val)); printf("testkey2 = %s\n", val); - /* remove non-existant value for key */ + /* remove non-existent value for key */ EXPECT0(notmuch_message_remove_property (message, "testkey2", "this value has spaces and = sign")); EXPECT0(notmuch_message_get_property (message, "testkey2", &val)); printf("testkey2 = %s\n", val); @@ -89,17 +89,6 @@ testkey2 = NULL EOF test_expect_equal_file EXPECTED OUTPUT -test_begin_subtest "notmuch_message_remove_all_properties" -cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} -EXPECT0(notmuch_message_remove_all_properties (message, NULL)); -print_properties (message, "", FALSE); -EOF -cat <<'EOF' >EXPECTED -== stdout == -== stderr == -EOF -test_expect_equal_file EXPECTED OUTPUT - test_begin_subtest "testing string map binary search (via message properties)" cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} { @@ -157,7 +146,28 @@ print_properties (message, "testkey1", TRUE); EOF cat <<'EOF' >EXPECTED == stdout == -testvalue1 +testkey1 = testvalue1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "notmuch_message_remove_all_properties" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +EXPECT0(notmuch_message_remove_all_properties (message, NULL)); +EXPECT0(notmuch_database_destroy(db)); +EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &db, &msg)); +if (msg) fputs (msg, stderr); +EXPECT0(notmuch_database_find_message(db, "4EFC743A.3060609@april.org", &message)); +if (message == NULL) { + fprintf (stderr, "unable to find message"); + exit (1); +} +print_properties (message, "", FALSE); +EOF +cat <<'EOF' >EXPECTED +== stdout == == stderr == EOF test_expect_equal_file EXPECTED OUTPUT @@ -171,10 +181,9 @@ print_properties (message, "testkey1", TRUE); EOF cat <<'EOF' >EXPECTED == stdout == -alice -bob -testvalue1 -testvalue2 +testkey1 = alice +testkey1 = bob +testkey1 = testvalue2 == stderr == EOF test_expect_equal_file EXPECTED OUTPUT @@ -188,13 +197,12 @@ print_properties (message, "testkey", FALSE); EOF cat <<'EOF' >EXPECTED == stdout == -alice -bob -testvalue1 -testvalue2 -alice3 -bob3 -testvalue3 +testkey1 = alice +testkey1 = bob +testkey1 = testvalue2 +testkey3 = alice3 +testkey3 = bob3 +testkey3 = testvalue3 == stderr == EOF test_expect_equal_file EXPECTED OUTPUT @@ -236,7 +244,7 @@ test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "dump message properties" cat < PROPERTIES -#= 4EFC743A.3060609@april.org fancy%20key%20with%20%c3%a1cc%c3%a8nts=import%20value%20with%20= testkey1=alice testkey1=bob testkey1=testvalue1 testkey1=testvalue2 testkey3=alice3 testkey3=bob3 testkey3=testvalue3 +#= 4EFC743A.3060609@april.org fancy%20key%20with%20%c3%a1cc%c3%a8nts=import%20value%20with%20= testkey1=alice testkey1=bob testkey1=testvalue2 testkey3=alice3 testkey3=bob3 testkey3=testvalue3 EOF cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} EXPECT0(notmuch_message_add_property (message, "fancy key with áccènts", "import value with =")); @@ -247,7 +255,7 @@ test_expect_equal_file PROPERTIES OUTPUT test_begin_subtest "dump _only_ message properties" cat < EXPECTED #notmuch-dump batch-tag:3 properties -#= 4EFC743A.3060609@april.org fancy%20key%20with%20%c3%a1cc%c3%a8nts=import%20value%20with%20= testkey1=alice testkey1=bob testkey1=testvalue1 testkey1=testvalue2 testkey3=alice3 testkey3=bob3 testkey3=testvalue3 +#= 4EFC743A.3060609@april.org fancy%20key%20with%20%c3%a1cc%c3%a8nts=import%20value%20with%20= testkey1=alice testkey1=bob testkey1=testvalue2 testkey3=alice3 testkey3=bob3 testkey3=testvalue3 EOF notmuch dump --include=properties > OUTPUT test_expect_equal_file EXPECTED OUTPUT @@ -316,7 +324,6 @@ EOF cat <<'EOF' > EXPECTED testkey1 = alice testkey1 = bob -testkey1 = testvalue1 testkey1 = testvalue2 EOF test_expect_equal_file EXPECTED OUTPUT @@ -332,7 +339,6 @@ EOF cat <<'EOF' > EXPECTED testkey1 = alice testkey1 = bob -testkey1 = testvalue1 testkey1 = testvalue2 testkey3 = alice3 testkey3 = bob3 @@ -350,4 +356,49 @@ for (key,val) in msg.get_properties("testkey",True): EOF test_expect_equal_file /dev/null OUTPUT +test_begin_subtest "notmuch_message_remove_all_properties_with_prefix" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +EXPECT0(notmuch_message_remove_all_properties_with_prefix (message, "testkey3")); +print_properties (message, "", FALSE); +EOF +cat <<'EOF' >EXPECTED +== stdout == +fancy key with áccènts = import value with = +testkey1 = alice +testkey1 = bob +testkey1 = testvalue2 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "edit property on removed message without uncaught exception" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +EXPECT0(notmuch_database_remove_message (db, notmuch_message_get_filename (message))); +stat = notmuch_message_remove_property (message, "example", "example"); +if (stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION) + fprintf (stderr, "unable to remove properties on message"); +EOF +cat <<'EOF' >EXPECTED +== stdout == +== stderr == +unable to remove properties on message +EOF +test_expect_equal_file EXPECTED OUTPUT + +add_email_corpus + +test_begin_subtest "remove all properties on removed message without uncaught exception" +cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} +EXPECT0(notmuch_database_remove_message (db, notmuch_message_get_filename (message))); +stat = notmuch_message_remove_all_properties_with_prefix (message, ""); +if (stat == NOTMUCH_STATUS_XAPIAN_EXCEPTION) + fprintf (stderr, "unable to remove properties on message"); +EOF +cat <<'EOF' >EXPECTED +== stdout == +== stderr == +unable to remove properties on message +EOF +test_expect_equal_file EXPECTED OUTPUT + test_done diff --git a/test/T620-lock.sh b/test/T620-lock.sh index 7aaaff2a..99cc7010 100755 --- a/test/T620-lock.sh +++ b/test/T620-lock.sh @@ -9,9 +9,6 @@ if [ $NOTMUCH_HAVE_XAPIAN_DB_RETRY_LOCK -ne 1 ]; then test_subtest_known_broken fi test_C ${MAIL_DIR} <<'EOF' -#include -#include -#include #include void @@ -43,15 +40,25 @@ main (int argc, char **argv) if (child == 0) { notmuch_database_t *db2; + char* msg = NULL; sleep (1); - EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_WRITE, &db2)); + + EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &db2, &msg)); + if (msg) fputs (msg, stderr); + taggit (db2, "child"); EXPECT0 (notmuch_database_close (db2)); } else { notmuch_database_t *db; + char* msg = NULL; - EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_WRITE, &db)); + EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &db, &msg)); + if (msg) fputs (msg, stderr); taggit (db, "parent"); sleep (2); EXPECT0 (notmuch_database_close (db)); diff --git a/test/T630-emacs-draft.sh b/test/T630-emacs-draft.sh index d7903ce7..c443f417 100755 --- a/test/T630-emacs-draft.sh +++ b/test/T630-emacs-draft.sh @@ -1,7 +1,9 @@ #!/usr/bin/env bash test_description="Emacs Draft Handling" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 +test_require_emacs add_email_corpus notmuch config set search.exclude_tags deleted diff --git a/test/T640-database-modified.sh b/test/T640-database-modified.sh index 274105c7..2c3fa735 100755 --- a/test/T640-database-modified.sh +++ b/test/T640-database-modified.sh @@ -10,11 +10,8 @@ test_begin_subtest "catching DatabaseModifiedError in _notmuch_message_ensure_me first_id=$(notmuch search --output=messages '*'| head -1 | sed s/^id://) test_C ${MAIL_DIR} < -#include #include -#include -#include + int main (int argc, char **argv) { @@ -26,14 +23,22 @@ main (int argc, char **argv) notmuch_query_t *query; notmuch_tags_t *tags; int i; + char* msg = NULL; - EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_ONLY, &ro_db)); + EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &ro_db, &msg)); + if (msg) fputs (msg, stderr); assert(ro_db); EXPECT0 (notmuch_database_find_message (ro_db, "${first_id}", &ro_message)); assert(ro_message); - EXPECT0 (notmuch_database_open (path, NOTMUCH_DATABASE_MODE_READ_WRITE, &rw_db)); + EXPECT0(notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_WRITE, + "", NULL, &rw_db, &msg)); + if (msg) fputs (msg, stderr); + query = notmuch_query_create(rw_db, ""); EXPECT0 (notmuch_query_search_messages (query, &messages)); diff --git a/test/T650-regexp-query.sh b/test/T650-regexp-query.sh index 43af3b47..a9844501 100755 --- a/test/T650-regexp-query.sh +++ b/test/T650-regexp-query.sh @@ -2,10 +2,6 @@ test_description='regular expression searches' . $(dirname "$0")/test-lib.sh || exit 1 -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 0 ]; then - test_done -fi - add_message '[dir]=bad' '[subject]="To the bone"' add_message '[dir]=.' '[subject]="Top level"' add_message '[dir]=bad/news' '[subject]="Bears"' @@ -69,6 +65,31 @@ thread:XXX 2001-01-05 [1/1] Notmuch Test Suite; - (inbox unread) EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "bracketed subject search (with dquotes)" +notmuch search subject:notmuch and subject:show > EXPECTED +notmuch search 'subject:"(show notmuch)"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "bracketed subject search (with dquotes and operator 'or')" +notmuch search subject:notmuch or subject:show > EXPECTED +notmuch search 'subject:"(notmuch or show)"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "bracketed subject search (with dquotes and operator 'and')" +notmuch search subject:notmuch and subject:show > EXPECTED +notmuch search 'subject:"(notmuch and show)"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "bracketed subject search (with phrase, operator 'or')" +notmuch search 'subject:"mailing list"' or subject:FreeBSD > EXPECTED +notmuch search 'subject:"(""mailing list"" or FreeBSD)"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "bracketed subject search (with phrase, operator 'and')" +notmuch search search 'subject:"notmuch show"' and subject:commands > EXPECTED +notmuch search 'subject:"(""notmuch show"" and commands)"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + test_begin_subtest "xapian wildcard search for from:" notmuch search --output=messages 'from:cwo*' > OUTPUT test_expect_equal_file cworth.msg-ids OUTPUT diff --git a/test/T670-duplicate-mid.sh b/test/T670-duplicate-mid.sh index c17ccb69..8fec291e 100755 --- a/test/T670-duplicate-mid.sh +++ b/test/T670-duplicate-mid.sh @@ -2,10 +2,25 @@ test_description="duplicate message ids" . $(dirname "$0")/test-lib.sh || exit 1 +test_require_external_prereq xapian-delve + add_message '[id]="duplicate"' '[subject]="message 1" [filename]=copy1' add_message '[id]="duplicate"' '[subject]="message 2" [filename]=copy2' add_message '[id]="duplicate"' '[subject]="message 0" [filename]=copy0' + +test_begin_subtest 'at most 1 thread-id per xapian document' +db=${MAIL_DIR}/.notmuch/xapian +for doc in $(xapian-delve -1 -t '' "$db" | grep '^[1-9]'); do + xapian-delve -1 -r "$doc" "$db" | grep -c '^G' +done > OUTPUT.raw +sort -u < OUTPUT.raw > OUTPUT +cat < EXPECTED +0 +1 +EOF +test_expect_equal_file EXPECTED OUTPUT + test_begin_subtest 'search: first indexed subject preserved' cat < EXPECTED thread:XXX 2001-01-05 [1/1(3)] Notmuch Test Suite; message 1 (inbox unread) @@ -48,11 +63,7 @@ notmuch search --output=files subject:'"message 2"' | notmuch_dir_sanitize > OUT test_expect_equal_file EXPECTED OUTPUT test_begin_subtest 'Regexp search for second subject' -# Note that missing field processor support really means the test -# doesn't make sense, but it happens to pass. -if [ $NOTMUCH_HAVE_XAPIAN_FIELD_PROCESSOR -eq 1 ]; then - test_subtest_known_broken -fi +test_subtest_known_broken cat <EXPECTED MAIL_DIR/copy0 MAIL_DIR/copy1 diff --git a/test/T700-reindex.sh b/test/T700-reindex.sh index 9e795896..af34ad7c 100755 --- a/test/T700-reindex.sh +++ b/test/T700-reindex.sh @@ -4,10 +4,25 @@ test_description='reindexing messages' add_email_corpus + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" = "1" ]; then + + count=$(notmuch count --lastmod '*' | cut -f 3) + for query in '()' '(not)' '(and)' '(or ())' '(or (not))' '(or (and))' \ + '(or (and) (or) (not (and)))'; do + test_begin_subtest "reindex all messages: $query" + notmuch reindex --query=sexp "$query" + output=$(notmuch count --lastmod '*' | cut -f 3) + count=$((count + 1)) + test_expect_equal "$output" "$count" + done + +fi + notmuch tag +usertag1 '*' -notmuch search '*' | notmuch_search_sanitize > initial-threads -notmuch search --output=messages '*' > initial-message-ids +notmuch search '*' 2>1 | notmuch_search_sanitize > initial-threads +notmuch search --output=messages '*' 2>/dev/null > initial-message-ids notmuch dump > initial-dump test_begin_subtest 'reindex preserves threads' @@ -33,6 +48,15 @@ notmuch reindex '*' notmuch dump > OUTPUT test_expect_equal_file initial-dump OUTPUT +test_begin_subtest 'reindex preserves tags with special prefixes' +notmuch tag +attachment2 +encrypted2 +signed2 '*' +notmuch dump > EXPECTED +notmuch reindex '*' +notmuch dump > OUTPUT +notmuch tag -attachment2 -encrypted2 -signed2 '*' +test_expect_equal_file EXPECTED OUTPUT + +backup_database test_begin_subtest 'reindex moves a message between threads' notmuch search --output=threads id:87iqd9rn3l.fsf@vertex.dottedmag > EXPECTED # re-parent @@ -40,7 +64,19 @@ sed -i 's/1258471718-6781-1-git-send-email-dottedmag@dottedmag.net/87iqd9rn3l.fs notmuch reindex id:1258471718-6781-2-git-send-email-dottedmag@dottedmag.net notmuch search --output=threads id:1258471718-6781-2-git-send-email-dottedmag@dottedmag.net > OUTPUT test_expect_equal_file EXPECTED OUTPUT +restore_database + +backup_database +test_begin_subtest 'reindex detects removal of all files' +notmuch search --output=messages not id:20091117232137.GA7669@griffis1.net> EXPECTED +# remove both copies +mv $MAIL_DIR/cur/51:2,* duplicate-message-2.eml +notmuch reindex id:20091117232137.GA7669@griffis1.net +notmuch search --output=messages '*' > OUTPUT +test_expect_equal_file EXPECTED OUTPUT +restore_database +backup_database test_begin_subtest 'reindex detects removal of all files' notmuch search --output=messages not id:20091117232137.GA7669@griffis1.net> EXPECTED # remove both copies @@ -48,6 +84,7 @@ mv $MAIL_DIR/cur/51:2,* duplicate-message-2.eml notmuch reindex id:20091117232137.GA7669@griffis1.net notmuch search --output=messages '*' > OUTPUT test_expect_equal_file EXPECTED OUTPUT +restore_database test_begin_subtest "reindex preserves properties" cat < prop-dump @@ -75,4 +112,11 @@ notmuch reindex '*' notmuch search '*' | notmuch_search_sanitize > OUTPUT test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "reindex after removing corpus" +tar cf backup.tar mail/cur +find mail/cur -type f -delete +test_expect_success "notmuch reindex '*'" +tar xf backup.tar + test_done diff --git a/test/T710-message-id.sh b/test/T710-message-id.sh index e73d6ba9..a2d8ec71 100755 --- a/test/T710-message-id.sh +++ b/test/T710-message-id.sh @@ -3,6 +3,10 @@ test_description="message id parsing" . $(dirname "$0")/test-lib.sh || exit 1 +if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_done +fi + test_begin_subtest "good message ids" ${TEST_DIRECTORY}/message-id-parse <OUTPUT <018b1a8f2d1df62e804ce88b65401304832dfbbf.1346614915.git.jani@nikula.org> @@ -29,7 +33,7 @@ GOOD: 1258787708-21121-2-git-send-email-keithp@keithp.com EOF test_expect_equal_file EXPECTED OUTPUT -test_begin_subtest "<> delimeters are required" +test_begin_subtest "<> delimiters are required" ${TEST_DIRECTORY}/message-id-parse <OUTPUT 018b1a8f2d1df62e804ce88b65401304832dfbbf.1346614915.git.jani@nikula.org> <1530507300.raoomurnbf.astroid@strange.none diff --git a/test/T720-emacs-attachment-warnings.sh b/test/T720-emacs-attachment-warnings.sh index c8d2bcc2..6e03f39d 100755 --- a/test/T720-emacs-attachment-warnings.sh +++ b/test/T720-emacs-attachment-warnings.sh @@ -2,6 +2,9 @@ test_description="emacs attachment warnings" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs test_begin_subtest "notmuch-test-attachment-warning part 1" test_emacs_expect_t '(notmuch-test-attachment-warning-1)' diff --git a/test/T720-lib-lifetime.sh b/test/T720-lib-lifetime.sh index 3d94d4df..e5afeaa2 100755 --- a/test/T720-lib-lifetime.sh +++ b/test/T720-lib-lifetime.sh @@ -23,7 +23,13 @@ int main (int argc, char** argv) { notmuch_database_t *db; notmuch_status_t stat; - stat = notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, &db); + char* msg = NULL; + + stat = notmuch_database_open_with_config (argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + "", NULL, &db, &msg); + if (msg) fputs (msg, stderr); + if (stat != NOTMUCH_STATUS_SUCCESS) { fprintf (stderr, "error opening database: %d\n", stat); exit (1); diff --git a/test/T730-emacs-forwarding.sh b/test/T730-emacs-forwarding.sh index 45e61568..7b6ebf15 100755 --- a/test/T730-emacs-forwarding.sh +++ b/test/T730-emacs-forwarding.sh @@ -2,6 +2,9 @@ test_description="emacs forwarding" . $(dirname "$0")/test-lib.sh || exit 1 +. $NOTMUCH_SRCDIR/test/test-lib-emacs.sh || exit 1 + +test_require_emacs test_begin_subtest "Forward setting the correct references header" # Check that, when forwarding a message, the new message has diff --git a/test/T750-gzip.sh b/test/T750-gzip.sh index fac41d39..5648896f 100755 --- a/test/T750-gzip.sh +++ b/test/T750-gzip.sh @@ -58,13 +58,13 @@ test_expect_equal_file EXPECTED OUTPUT test_begin_subtest "notmuch search --output=files with partially gzipped mail store" notmuch search --output=files '*' | notmuch_search_files_sanitize > OUTPUT cat < EXPECTED -MAIL_DIR/msg-001.gz -MAIL_DIR/msg-002.gz -MAIL_DIR/msg-003.gz -MAIL_DIR/msg-004 -MAIL_DIR/msg-005.gz -MAIL_DIR/msg-006 -MAIL_DIR/msg-007.gz +MAIL_DIR/msg-XXX.gz +MAIL_DIR/msg-XXX.gz +MAIL_DIR/msg-XXX.gz +MAIL_DIR/msg-XXX +MAIL_DIR/msg-XXX.gz +MAIL_DIR/msg-XXX +MAIL_DIR/msg-XXX.gz EOF test_expect_equal_file EXPECTED OUTPUT @@ -171,7 +171,7 @@ test_expect_equal_file EXPECTED OUTPUT add_email_corpus lkml test_begin_subtest "new doesn't run out of file descriptors with many gzipped files" ulimit -n 200 -gzip --recursive ${MAIL_DIR} +find ${MAIL_DIR} -name .notmuch -prune -o -type f -print0 | xargs -0 gzip -- test_expect_success "notmuch new" test_done diff --git a/test/T750-user-header.sh b/test/T750-user-header.sh index 204c052a..03c43656 100755 --- a/test/T750-user-header.sh +++ b/test/T750-user-header.sh @@ -2,17 +2,10 @@ test_description='indexing user specified headers' . $(dirname "$0")/test-lib.sh || exit 1 -test_begin_subtest "error adding user header before initializing DB" -notmuch config set index.header.List List-Id 2>&1 | notmuch_dir_sanitize > OUTPUT -cat < EXPECTED -Error opening database at MAIL_DIR/.notmuch: No such file or directory -EOF -test_expect_equal_file EXPECTED OUTPUT - add_email_corpus -notmuch search '*' | notmuch_search_sanitize > initial-threads -notmuch search --output=messages '*' > initial-message-ids +notmuch search '*' 2>1 | notmuch_search_sanitize > initial-threads +notmuch search --output=messages '*' 2>/dev/null > initial-message-ids notmuch dump > initial-dump test_begin_subtest "adding illegal prefix name, bad utf8" @@ -108,4 +101,22 @@ MAIL_DIR/new/04:2, EOF test_expect_equal_file EXPECTED OUTPUT +test_begin_subtest "index user header, config from file" +field_name="Test" +printf "\n[index]\nheader.${field_name} = List-Id\n" >> notmuch-config +notmuch reindex '*' +notmuch search --output=files ${field_name}:notmuch | notmuch_search_files_sanitize | sort > OUTPUT +cat < EXPECTED +MAIL_DIR/bar/baz/05:2, +MAIL_DIR/bar/baz/23:2, +MAIL_DIR/bar/baz/24:2, +MAIL_DIR/bar/cur/20:2, +MAIL_DIR/bar/new/21:2, +MAIL_DIR/bar/new/22:2, +MAIL_DIR/foo/cur/08:2, +MAIL_DIR/foo/new/03:2, +MAIL_DIR/new/04:2, +EOF +test_expect_equal_file EXPECTED OUTPUT + test_done diff --git a/test/T760-as-text.sh b/test/T760-as-text.sh new file mode 100755 index 00000000..744567f2 --- /dev/null +++ b/test/T760-as-text.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +test_description='index attachments as text' +. $(dirname "$0")/test-lib.sh || exit 1 + +add_email_corpus indexing +test_begin_subtest "empty as_text; skip text/x-diff" +messages=$(notmuch count id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain) +count=$(notmuch count id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz) +test_expect_equal "$messages,$count" "1,0" + +notmuch config set index.as_text "^text/" +add_email_corpus indexing + +test_begin_subtest "as_index is text/; find text/x-diff" +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain > EXPECTED +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "reindex with empty as_text, skips text/x-diff" +notmuch config set index.as_text +notmuch reindex '*' +messages=$(notmuch count id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain) +count=$(notmuch count id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz) +test_expect_equal "$messages,$count" "1,0" + +test_begin_subtest "reindex with empty as_text; skips application/pdf" +notmuch config set index.as_text +notmuch reindex '*' +gmessages=$(notmuch count id:871qo9p4tf.fsf@tethera.net) +count=$(notmuch count id:871qo9p4tf.fsf@tethera.net and body:not-really-PDF) +test_expect_equal "$messages,$count" "1,0" + +test_begin_subtest "reindex with as_text as text/; finds text/x-diff" +notmuch config set index.as_text "^text/" +notmuch reindex '*' +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain > EXPECTED +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "reindex with as_text as text/; skips application/pdf" +notmuch config set index.as_text "^text/" +notmuch config set index.as_text +notmuch reindex '*' +messages=$(notmuch count id:871qo9p4tf.fsf@tethera.net) +count=$(notmuch count id:871qo9p4tf.fsf@tethera.net and body:not-really-PDF) +test_expect_equal "$messages,$count" "1,0" + +test_begin_subtest "as_text has multiple regexes" +notmuch config set index.as_text "blahblah;^text/" +notmuch reindex '*' +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain > EXPECTED +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "as_text is non-anchored regex" +notmuch config set index.as_text "e.t/" +notmuch reindex '*' +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain > EXPECTED +notmuch search id:20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain and ersatz > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "as_text is 'application/pdf'" +notmuch config set index.as_text "^application/pdf$" +notmuch reindex '*' +notmuch search id:871qo9p4tf.fsf@tethera.net > EXPECTED +notmuch search id:871qo9p4tf.fsf@tethera.net and '"not really PDF"' > OUTPUT +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "as_text is bad regex" +notmuch config set index.as_text '[' +notmuch reindex '*' >& OUTPUT +cat< EXPECTED +Error in index.as_text: Invalid regular expression: [ +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T800-asan.sh b/test/T800-asan.sh new file mode 100755 index 00000000..9ce6baa7 --- /dev/null +++ b/test/T800-asan.sh @@ -0,0 +1,44 @@ +#!/usr/bin/env bash +test_description='run code with ASAN enabled against the library' +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_ASAN-0}" != "1" ]; then + printf "Skipping due to missing ASAN support\n" + test_done +fi + +if [ -n "${LD_PRELOAD-}" ]; then + printf "Skipping due to ASAN LD_PRELOAD restrictions\n" + test_done +fi + +add_email_corpus + +TEST_CFLAGS="${TEST_CFLAGS:-} -fsanitize=address" + +test_begin_subtest "open and destroy" +test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} < +#include + +int main(int argc, char **argv) { + notmuch_database_t *db = NULL; + + notmuch_status_t st = notmuch_database_open_with_config(argv[1], + NOTMUCH_DATABASE_MODE_READ_ONLY, + argv[2], NULL, &db, NULL); + + printf("db != NULL: %d\n", db != NULL); + if (db != NULL) + notmuch_database_destroy(db); + return 0; +} +EOF +cat < EXPECTED +== stdout == +db != NULL: 1 +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T810-tsan.sh b/test/T810-tsan.sh new file mode 100755 index 00000000..4071e296 --- /dev/null +++ b/test/T810-tsan.sh @@ -0,0 +1,92 @@ +#!/usr/bin/env bash + +test_directory=$(cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null && pwd) + +test_description='run code with TSan enabled against the library' +# Note it is hard to ensure race conditions are deterministic so this +# only provides best effort detection. + +. "$test_directory"/test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_TSAN-0}" != "1" ]; then + printf "Skipping due to missing TSan support\n" + test_done +fi + +export TSAN_OPTIONS="suppressions=$test_directory/T810-tsan.suppressions" +TEST_CFLAGS="${TEST_CFLAGS:-} -fsanitize=thread" + +cp -r ${MAIL_DIR} ${MAIL_DIR}-2 + +test_begin_subtest "create" +test_C ${MAIL_DIR} ${MAIL_DIR}-2 < +#include + +void *thread (void *arg) { + char *mail_dir = arg; + /* + * Calls into notmuch_query_search_messages which was using the thread-unsafe + * Xapian::Query::MatchAll. + */ + EXPECT0(notmuch_database_create (mail_dir, NULL)); + return NULL; +} + +int main (int argc, char **argv) { + pthread_t t1, t2; + EXPECT0(pthread_create (&t1, NULL, thread, argv[1])); + EXPECT0(pthread_create (&t2, NULL, thread, argv[2])); + EXPECT0(pthread_join (t1, NULL)); + EXPECT0(pthread_join (t2, NULL)); + return 0; +} +EOF +cat < EXPECTED +== stdout == +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +add_email_corpus +rm -r ${MAIL_DIR}-2 +cp -r ${MAIL_DIR} ${MAIL_DIR}-2 + +test_begin_subtest "query" +test_C ${MAIL_DIR} ${MAIL_DIR}-2 < +#include + +void *thread (void *arg) { + char *mail_dir = arg; + notmuch_database_t *db; + /* + * 'from' is NOTMUCH_FIELD_PROBABILISTIC | NOTMUCH_FIELD_PROCESSOR and an + * empty string gets us to RegexpFieldProcessor::operator which was using + * the tread-unsafe Xapian::Query::MatchAll. + */ + EXPECT0(notmuch_database_open_with_config (mail_dir, + NOTMUCH_DATABASE_MODE_READ_ONLY, + NULL, NULL, &db, NULL)); + notmuch_query_t *query = notmuch_query_create (db, "from:\"\""); + notmuch_messages_t *messages; + EXPECT0(notmuch_query_search_messages (query, &messages)); + return NULL; +} + +int main (int argc, char **argv) { + pthread_t t1, t2; + EXPECT0(pthread_create (&t1, NULL, thread, argv[1])); + EXPECT0(pthread_create (&t2, NULL, thread, argv[2])); + EXPECT0(pthread_join (t1, NULL)); + EXPECT0(pthread_join (t2, NULL)); + return 0; +} +EOF +cat < EXPECTED +== stdout == +== stderr == +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_done diff --git a/test/T810-tsan.suppressions b/test/T810-tsan.suppressions new file mode 100644 index 00000000..80dc062f --- /dev/null +++ b/test/T810-tsan.suppressions @@ -0,0 +1,3 @@ +# It's unclear how TSan-friendly GLib is: +# https://gitlab.gnome.org/GNOME/glib/-/issues/1672 +called_from_lib:libglib*.so diff --git a/test/T850-git.sh b/test/T850-git.sh new file mode 100755 index 00000000..831e4678 --- /dev/null +++ b/test/T850-git.sh @@ -0,0 +1,402 @@ +#!/usr/bin/env bash +test_description='"notmuch git" to save and restore tags' +. $(dirname "$0")/test-lib.sh || exit 1 + +if [ "${NOTMUCH_HAVE_SFSEXP-0}" != "1" ]; then + printf "Skipping due to missing sfsexp library\n" + test_done +fi + +# be very careful using backup_database / restore_database in this +# file, as they fool the cache invalidation checks in notmuch-git. + +add_email_corpus + +git config --global user.email notmuch@example.org +git config --global user.name "Notmuch Test Suite" + +test_begin_subtest "init" +test_expect_success "notmuch git -p '' -C remote.git init" + +test_begin_subtest "init (git.path)" +notmuch config set git.path configured.git +notmuch git init +notmuch config set git.path +output=$(git -C configured.git rev-parse --is-bare-repository) +test_expect_equal "$output" "true" + +test_begin_subtest "clone" +test_expect_success "notmuch git -p '' -C tags.git clone remote.git" + +test_begin_subtest "initial commit needs force" +test_expect_code 1 "notmuch git -C tags.git commit" + +test_begin_subtest "committing new prefix requires force" +notmuch git -C force-prefix.git init +notmuch tag +new-prefix::foo id:20091117190054.GU3165@dottiness.seas.harvard.edu +test_expect_code 1 "notmuch git -l debug -p 'new-prefix::' -C force-prefix.git commit" +notmuch tag -new-prefix::foo id:20091117190054.GU3165@dottiness.seas.harvard.edu + +test_begin_subtest "committing new prefix works with force" +notmuch tag +new-prefix::foo id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -l debug -p 'new-prefix::' -C force-prefix.git commit --force +git -C force-prefix.git ls-tree -r --name-only HEAD | notmuch_git_sanitize | xargs dirname | sort -u > OUTPUT +notmuch tag -new-prefix::foo id:20091117190054.GU3165@dottiness.seas.harvard.edu +cat <EXPECTED +20091117190054.GU3165@dottiness.seas.harvard.edu +EOF +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "checkout new prefix requires force" +test_expect_code 1 "notmuch git -l debug -p 'new-prefix::' -C force-prefix.git checkout" + +test_begin_subtest "checkout new prefix works with force" +notmuch dump > BEFORE +notmuch git -l debug -p 'new-prefix::' -C force-prefix.git checkout --force +notmuch dump --include=tags id:20091117190054.GU3165@dottiness.seas.harvard.edu | grep -v '^#' > OUTPUT +notmuch restore < BEFORE +cat < EXPECTED ++inbox +new-prefix%3a%3afoo +signed +unread -- id:20091117190054.GU3165@dottiness.seas.harvard.edu +EOF +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "commit" +notmuch git -C tags.git commit --force +git -C tags.git ls-tree -r --name-only HEAD | notmuch_git_sanitize | xargs dirname | sort -u > OUTPUT +notmuch search --output=messages '*' | sed s/^id:// | sort > EXPECTED +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "commit --force succeeds" +notmuch git -C force.git init +test_expect_success "notmuch git -C force.git commit --force" + +test_begin_subtest "changing git.safe_fraction succeeds" +notmuch config set git.safe_fraction 1 +notmuch git -C force2.git init +test_expect_success "notmuch git -C force2.git commit" +notmuch config set git.safe_fraction + +test_begin_subtest "commit, with quoted tag" +notmuch git -C clone2.git clone tags.git +git -C clone2.git ls-tree -r --name-only HEAD | grep /inbox > BEFORE +notmuch tag '+"quoted tag"' '*' +notmuch git -C clone2.git commit +notmuch tag '-"quoted tag"' '*' +git -C clone2.git ls-tree -r --name-only HEAD | grep /inbox > AFTER +test_expect_equal_file_nonempty BEFORE AFTER + +test_begin_subtest "commit (incremental)" +notmuch tag +test id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git commit +git -C tags.git ls-tree -r --name-only HEAD | notmuch_git_sanitize | \ + grep 20091117190054 | sort > OUTPUT +echo "--------------------------------------------------" >> OUTPUT +notmuch tag -test id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git commit +git -C tags.git ls-tree -r --name-only HEAD | notmuch_git_sanitize | \ + grep 20091117190054 | sort >> OUTPUT +cat < EXPECTED +20091117190054.GU3165@dottiness.seas.harvard.edu/inbox +20091117190054.GU3165@dottiness.seas.harvard.edu/signed +20091117190054.GU3165@dottiness.seas.harvard.edu/test +20091117190054.GU3165@dottiness.seas.harvard.edu/unread +-------------------------------------------------- +20091117190054.GU3165@dottiness.seas.harvard.edu/inbox +20091117190054.GU3165@dottiness.seas.harvard.edu/signed +20091117190054.GU3165@dottiness.seas.harvard.edu/unread +EOF +test_expect_equal_file_nonempty EXPECTED OUTPUT + +test_begin_subtest "commit (change prefix)" +notmuch tag +test::one id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git -p 'test::' commit --force +git -C tags.git ls-tree -r --name-only HEAD | + grep 20091117190054 | notmuch_git_sanitize | sort > OUTPUT +echo "--------------------------------------------------" >> OUTPUT +notmuch tag -test::one id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git commit --force +git -C tags.git ls-tree -r --name-only HEAD | notmuch_git_sanitize | \ + grep 20091117190054 | sort >> OUTPUT +cat < EXPECTED +20091117190054.GU3165@dottiness.seas.harvard.edu/one +-------------------------------------------------- +20091117190054.GU3165@dottiness.seas.harvard.edu/inbox +20091117190054.GU3165@dottiness.seas.harvard.edu/signed +20091117190054.GU3165@dottiness.seas.harvard.edu/unread +EOF +test_expect_equal_file_nonempty EXPECTED OUTPUT + +backup_database +test_begin_subtest "large checkout needs --force" +notmuch tag -inbox '*' +test_expect_code 1 "notmuch git -C tags.git checkout" +restore_database + +test_begin_subtest "checkout (git.safe_fraction)" +notmuch git -C force3.git clone tags.git +notmuch dump > BEFORE +notmuch tag -inbox '*' +notmuch config set git.safe_fraction 1 +notmuch git -C force3.git checkout +notmuch config set git.safe_fraction +notmuch dump > AFTER +test_expect_equal_file_nonempty BEFORE AFTER + +test_begin_subtest "checkout" +notmuch dump > BEFORE +notmuch tag -inbox '*' +notmuch git -C tags.git checkout --force +notmuch dump > AFTER +test_expect_equal_file_nonempty BEFORE AFTER + +test_begin_subtest "archive" +notmuch git -C tags.git archive | tar tf - | \ + grep 20091117190054.GU3165@dottiness.seas.harvard.edu | notmuch_git_sanitize | sort > OUTPUT +cat < EXPECTED +20091117190054.GU3165@dottiness.seas.harvard.edu/ +20091117190054.GU3165@dottiness.seas.harvard.edu/inbox +20091117190054.GU3165@dottiness.seas.harvard.edu/signed +20091117190054.GU3165@dottiness.seas.harvard.edu/unread +EOF +notmuch git -C tags.git checkout +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "status" +notmuch tag +test id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git status > OUTPUT +cat < EXPECTED +A 20091117190054.GU3165@dottiness.seas.harvard.edu test +EOF +notmuch git -C tags.git checkout +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "status (global config argument)" +cp notmuch-config notmuch-config.new +notmuch --config=notmuch-config.new config set git.path tags.git +notmuch tag +test id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch --config=./notmuch-config.new git status > OUTPUT +cat < EXPECTED +A 20091117190054.GU3165@dottiness.seas.harvard.edu test +EOF +notmuch --config=notmuch-config.new git checkout +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "fetch" +notmuch tag +test2 id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C remote.git commit --force +notmuch tag -test2 id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git fetch +notmuch git -C tags.git status > OUTPUT +cat < EXPECTED + a 20091117190054.GU3165@dottiness.seas.harvard.edu test2 +EOF +notmuch git -C tags.git checkout +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "merge" +notmuch git -C tags.git merge +notmuch dump id:20091117190054.GU3165@dottiness.seas.harvard.edu | grep -v '^#' > OUTPUT +cat < EXPECTED ++inbox +signed +test2 +unread -- id:20091117190054.GU3165@dottiness.seas.harvard.edu +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "push" +notmuch tag +test3 id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git commit +notmuch tag -test3 id:20091117190054.GU3165@dottiness.seas.harvard.edu +notmuch git -C tags.git push +notmuch git -C remote.git checkout +notmuch dump id:20091117190054.GU3165@dottiness.seas.harvard.edu | grep -v '^#' > OUTPUT +cat < EXPECTED ++inbox +signed +test2 +test3 +unread -- id:20091117190054.GU3165@dottiness.seas.harvard.edu +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "environment passed through when run as 'notmuch git'" +env NOTMUCH_GIT_DIR=foo NOTMUCH_GIT_PREFIX=bar NOTMUCH_PROFILE=default notmuch git -C tags.git -p '' -ldebug status |& \ + grep '^env ' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +env NOTMUCH_GIT_DIR = foo +env NOTMUCH_GIT_PREFIX = bar +env NOTMUCH_PROFILE = default +env NOTMUCH_CONFIG = CWD/notmuch-config +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "--nmbug argument sets defaults" +notmuch git -ldebug --nmbug status |& grep '^\(prefix\|repository\)' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +prefix = notmuch:: +repository = CWD/home/.nmbug +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "invoke as nmbug sets defaults" +test_subtest_broken_for_installed +"$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^\(prefix\|repository\)' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +prefix = notmuch:: +repository = CWD/home/.nmbug +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_DIR works when invoked as nmbug" +test_subtest_broken_for_installed +NOTMUCH_GIT_DIR=`pwd`/foo "$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +repository = CWD/foo +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_DIR works when invoked as 'notmuch git'" +NOTMUCH_GIT_DIR=`pwd`/remote.git notmuch git -ldebug status |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +repository = CWD/remote.git +EOF +test_expect_equal_file EXPECTED OUTPUT + + +test_begin_subtest "env variable NOTMUCH_GIT_DIR overrides config when invoked as 'nmbug'" +test_subtest_broken_for_installed +notmuch config set git.path `pwd`/bar +NOTMUCH_GIT_DIR=`pwd`/remote.git "$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +notmuch config set git.path +cat < EXPECTED +repository = CWD/remote.git +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_DIR overrides config when invoked as 'notmuch git'" +notmuch config set git.path `pwd`/bar +NOTMUCH_GIT_DIR=`pwd`/remote.git notmuch git -ldebug status |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +notmuch config set git.path +cat < EXPECTED +repository = CWD/remote.git +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_PREFIX works when invoked as 'nmbug'" +test_subtest_broken_for_installed +NOTMUCH_GIT_PREFIX=env:: "$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^prefix' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +prefix = env:: +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_PREFIX works when invoked as nmbug" +test_subtest_broken_for_installed +NOTMUCH_GIT_PREFIX=foo:: "$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^prefix' | notmuch_dir_sanitize > OUTPUT +cat < EXPECTED +prefix = foo:: +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_PREFIX overrides config when invoked as 'nmbug'" +test_subtest_broken_for_installed +notmuch config set git.tag_prefix config:: +NOTMUCH_GIT_PREFIX=env:: "$NOTMUCH_BUILDDIR"/nmbug -ldebug status |& grep '^prefix' | notmuch_dir_sanitize > OUTPUT +notmuch config set git.path +cat < EXPECTED +prefix = env:: +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "env variable NOTMUCH_GIT_PREFIX overrides config when invoked as 'notmuch git'" +notmuch config set git.tag_prefix config:: +NOTMUCH_GIT_PREFIX=env:: notmuch git -ldebug status |& grep '^prefix' | notmuch_dir_sanitize > OUTPUT +notmuch config set git.path +cat < EXPECTED +prefix = env:: +EOF +test_expect_equal_file EXPECTED OUTPUT + + +test_begin_subtest "init, xdg default location" +repo=home/.local/share/notmuch/default/git +notmuch git -ldebug init |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +git -C $repo rev-parse --absolute-git-dir | notmuch_dir_sanitize >> OUTPUT +cat < EXPECTED +repository = CWD/$repo +CWD/$repo +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "init, xdg default location, with profile" +repo=home/.local/share/notmuch/work/git +NOTMUCH_PROFILE=work notmuch git -ldebug init |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +git -C $repo rev-parse --absolute-git-dir | notmuch_dir_sanitize >> OUTPUT +cat < EXPECTED +repository = CWD/$repo +CWD/$repo +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "init, configured location" +repo=configured-tags +notmuch config set git.path `pwd`/$repo +notmuch git -ldebug init |& grep '^repository' | notmuch_dir_sanitize > OUTPUT +notmuch config set git.path +git -C $repo rev-parse --absolute-git-dir | notmuch_dir_sanitize >> OUTPUT +cat < EXPECTED +repository = CWD/$repo +CWD/$repo +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "configured tag prefix" +notmuch config set git.tag_prefix test:: +notmuch git -ldebug status |& grep '^prefix' > OUTPUT +notmuch config set git.tag_prefix +cat < EXPECTED +prefix = test:: +EOF +test_expect_equal_file EXPECTED OUTPUT + +test_begin_subtest "default version is 1" +notmuch git -l debug -C default-version.git init +output=$(git -C default-version.git cat-file blob HEAD:FORMAT) +test_expect_equal "${output}" 1 + +test_begin_subtest "illegal version" +test_expect_code 1 "notmuch git -l debug -C default-version.git init --format-version=42" + +hash=("" "8d/c3/") # for use in synthetic repo contents. +for ver in {0..1}; do + test_begin_subtest "init version=${ver}" + notmuch git -C version-${ver}.git -p "test${ver}::" init --format-version=${ver} + output=$(git -C version-${ver}.git ls-tree -r --name-only HEAD) + expected=("" "FORMAT") + test_expect_equal "${output}" "${expected[${ver}]}" + + test_begin_subtest "initial commit version=${ver}" + notmuch tag "+test${ver}::a" "+test${ver}::b" id:20091117190054.GU3165@dottiness.seas.harvard.edu + notmuch git -C version-${ver}.git -p "test${ver}::" commit --force + git -C version-${ver}.git ls-tree -r --name-only HEAD | grep -v FORMAT > OUTPUT +cat < EXPECTED +tags/${hash[${ver}]}20091117190054.GU3165@dottiness.seas.harvard.edu/a +tags/${hash[${ver}]}20091117190054.GU3165@dottiness.seas.harvard.edu/b +EOF + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "second commit repo=${ver}" + notmuch tag "+test${ver}::c" "+test${ver}::d" id:20091117190054.GU3165@dottiness.seas.harvard.edu + notmuch git -C version-${ver}.git -p "test${ver}::" commit --force + git -C version-${ver}.git ls-tree -r --name-only HEAD | grep -v FORMAT > OUTPUT +cat < EXPECTED +tags/${hash[$ver]}20091117190054.GU3165@dottiness.seas.harvard.edu/a +tags/${hash[$ver]}20091117190054.GU3165@dottiness.seas.harvard.edu/b +tags/${hash[$ver]}20091117190054.GU3165@dottiness.seas.harvard.edu/c +tags/${hash[$ver]}20091117190054.GU3165@dottiness.seas.harvard.edu/d +EOF + test_expect_equal_file_nonempty EXPECTED OUTPUT + + test_begin_subtest "checkout repo=${ver} " + notmuch dump > BEFORE + notmuch tag -test::${ver}::a '*' + notmuch git -C version-${ver}.git -p "test${ver}::" checkout --force + notmuch dump > AFTER + test_expect_equal_file_nonempty BEFORE AFTER +done + +test_done diff --git a/test/aggregate-results.sh b/test/aggregate-results.sh index 75400e6e..6845fcf0 100755 --- a/test/aggregate-results.sh +++ b/test/aggregate-results.sh @@ -8,9 +8,16 @@ failed=0 broken=0 total=0 all_skipped=0 +rep_failed=0 for file do + if [ ! -f "$file" ]; then + echo "'$file' does not exist!" + rep_failed=$((rep_failed + 1)) + continue + fi + has_total=0 while read type value do case $type in @@ -24,18 +31,23 @@ do broken=$((broken + value)) ;; total) total=$((total + value)) + has_total=1 if [ "$value" -eq 0 ]; then all_skipped=$((all_skipped + 1)) fi esac done <"$file" + if [ "$has_total" -eq 0 ]; then + echo "'$file' lacks 'total ...'; results may be inconsistent." + failed=$((failed + 1)) + fi done pluralize_s () { [ "$1" -eq 1 ] && s='' || s='s'; } echo "Notmuch test suite complete." -if [ "$fixed" -eq 0 ] && [ "$failed" -eq 0 ]; then +if [ "$fixed" -eq 0 ] && [ "$failed" -eq 0 ] && [ "$rep_failed" -eq 0 ]; then pluralize_s "$total" printf "All $total test$s " if [ "$broken" -eq 0 ]; then @@ -70,10 +82,16 @@ if [ "$all_skipped" -ne 0 ]; then echo "All tests in $all_skipped file$s skipped." fi +if [ "$rep_failed" -ne 0 ]; then + pluralize_s "$rep_failed" + echo "$rep_failed test$s failed to report results." +fi + # Note that we currently do not consider skipped tests as failing the # build. -if [ "$success" -gt 0 ] && [ "$fixed" -eq 0 ] && [ "$failed" -eq 0 ] +if [ "$success" -gt 0 ] && [ "$fixed" -eq 0 ] && + [ "$failed" -eq 0 ] && [ "$rep_failed" -eq 0 ] then exit 0 else diff --git a/test/corpora/attachment/x-gtar-compressed.eml b/test/corpora/attachment/x-gtar-compressed.eml new file mode 100644 index 00000000..258a74d1 --- /dev/null +++ b/test/corpora/attachment/x-gtar-compressed.eml @@ -0,0 +1,136 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Mon, 19 Mar 2018 13:56:54 -0400 +Received: from marcos.anarc.at ([206.248.172.91]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1exz1i-0002aa-If + for david@tethera.net; Mon, 19 Mar 2018 13:56:54 -0400 +Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: anarcat) with ESMTPSA id 718A610E04F +From: =?utf-8?Q?Antoine_Beaupr=C3=A9?= +To: David Bremner , notmuch@notmuchmail.org +Subject: Re: bug: "no top level messages" crash on Zen email loops +In-Reply-To: <87a7v42bv9.fsf@curie.anarc.at> +References: <87d10042pu.fsf@curie.anarc.at> <87woy8vx7i.fsf@tesseract.cs.unb.ca> <87a7v42bv9.fsf@curie.anarc.at> +Date: Mon, 19 Mar 2018 13:56:54 -0400 +Message-ID: <874llc2bkp.fsf@curie.anarc.at> +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary="=-=-=" +X-Spam_score: 0.0 +X-Spam_score_int: 0 +X-Spam_bar: / + +--=-=-= +Content-Type: text/plain + +And obviously I forget the frigging attachment. + + +--=-=-= +Content-Type: application/x-gtar-compressed +Content-Disposition: attachment; filename=zendesk-email-loop2.tgz +Content-Transfer-Encoding: base64 + +H4sIAJX1r1oAA+xbaVPbyNbO19Gv6CF1K0mBrM2SLYOY8QoGbAzegKlbVFtq28KyJNSSF269//09 +Lclm35LJTebOiARbvZw+a5+npcPIDh08EMwoED58r0uEKyeK7FPKqfc+V9cHSZFVJZeTlWz2gyjJ +OUn9gNTvxtGdK6IhDhD6gF0cmDh8dtxr/X/Ra3Rrf0mVpaym5FQlc6orWl7P5hQpmxdzmi4qSj4D +Y2xSkLdq7XeuwQysPWt/WZIVOba/LGqalJPA/jktJ35A4neR+MH1N7f/KQmjwOVbOBwX0A6NfN8L +wt8Tr8iY3nSXO+OPA3tku9jhO14BpXrYTIb8Ht9mcMhViGPPSECsF0edEpPAMKuAhoE3RVNo9mhm +1Y0+O56JnbFHQ/QHbAIZEX6kf3/hfhkskZUssLwzugUDh/biC5rb4RhV241OC9kWkrJFVZHEqqiK +3C9DL0A7z7Czu406EdlCkoIa4ASyKOXhV0HOFZQc4sWsKKLP1UrnC+ig7eMpXx4Tc0ICvkcCantu +AbHWIqWYUttFSiabkdBnoKLCXF7Of0Ge+1DEFakjMiNOAa1u2yEOI1pATW8LUdMLiMFLGRUF5Dqy +QaeGmhFRSGhIjVLxvNq+VMWtymG9cdmu7zWrlS3ul/iuVzyqV7b2O42jy9pxs3N5dNy/LMOX02K7 +kzQ3qu12ca+6dVruVS7rzctKs91nrRXuFxyFnkNw4BpjPEXru0tQoEkM10OzRGojlhMY3wvI0rFp +WGCmwUtiIUXVECWm51oUgcV8MM8IBoEoiobAYIkunle6hkDbD33EIkMSgA6yGd+ztIyNLSlzY9El +Ze6JPr/c/YekyxlJkzIS7GqSpKe+NF287EVt5kY1WS7HbpStfbUbaffc6IFkXhRKTwv1bM99eUSR +yfM5Au8boc5ReyZl5EQK0/bHJEDFarmyzxerbVnV+PZ+EbxT1QT4jwZ2SL+wyU0PmY5N3BCZJAjt +oQ0CktjxwN2ItYq9F23wnAbL2Ww+q0lFuShKr6qwTyxQYfZWhSKoUCpkRbTJ0gT63O2U36HC9U6S +ib9Z3hRDjD7aVZ4n8MzmUiqWq2VZlHM5XftamRT9JZluiGsROkm4mHvBRHrOF0AMSc5AalbFrxVH +L5WknMzScFX9k8SpgAMVXhvL1UDUAtqzwyM8QO0k7zydgE6J7yzjrPKW0Wxc8Z4Ez0vENQhs3iPC +1yuQ/HQlp5801AtFv1QxzmsDSckOzeHgUsplAQANTT2HZXOg582cpOetS+oHUfj7HWvtcnWXv+X2 +DsH7o55cSoVslWVLkcH9pbRc9qml2tHgipiw+Z57UbDSTKqQVfiiMYaNmBAXGhIX4xr2lNzmLykj +cmXPDSH8+c7SB7NNIye0fRyEAnZCErg4hGnbHBp4kWvhYGls8LxhXE6BDBu1Zl+XMAjzWFOypGU3 +YL45xgEloRGFQz5/u2aAXQpbC191Tc+CXayAcrAzQXIpQv4BXVLfcynhmcUDsBWY1nG4uA/kn9ph +yKKG5Sp+RFwSgOdZMLmBbYcEBXSRKAwl99CRgpQlHzOwAO1BGgY+bIvP5lVFy+miLMt5GJlO5Zmb +8kXTBPFDvg6LgVxDmYgcS7h82x6BhqIAFIdmhrSNsBFQzNMxhj12G5lGAJlxQSwh/dxGlnHHiqCX +a8NyqRAuwm1EV11AJzQAjYt6HlC4zrQ/NpRNOdob+eOjA9ITx7WS2r7qdxZd3PA79SNrTq8ts7G8 +tptddWTAjLFhsTBkG0ohiF0y9Arwb5p6vG0VbJdf99DUm5hd+TTRF8zUSiHzjPXNymQkNRljzzgy +G/Ogrzfz+pV442Fts0lmvd5hq7dcqBPVy17NDhzizlsnh61KLWfnG/kDf7h57Cw2ezUXq6Q66YkL +pxYe1azRebBnXWiT3nH7qnR81VfzWAoWVDmzRq4iir090eotdGtfEMpWcZSjEC65aqPeqmXdUjXn +BV4w3FzOTFKpn8tndauy31ke2KcTTdZPz9STzRPjK03Hge1GkEQcQDGTyM/cteP/ihkPi8L1/szO +iaMDoX5Fg9HeZrNfypYOdHvzpNs5L0/y43arNtwTivsDyTpzrIDcXC33RktJGeXV3l4Qnkxvjqwr +eeDX/H1ltF+JDs91dzZoNoh/le+PvRHBfpdG+PrcOVnsh2fyvjsrDejgeKYuOuc3m1HPOSlNatHs +tGtP8y0tS2oz/8yiE41cn12U+62uV5meluoGBGm5UazybQaXC0hc3cPm7yypDXvFzIDkiMyZUTlo +j/dPawOwqoToFH6FgQFnA2y0XMc5HjQkd+HOLYWqejAVvBPDKEhSDjzi0C4vDiriYrNF1F6xfXI4 +PfcOAnMZHXswRGcU6hOzM9mviMOLRrEgiTDp4sKNzsODidf30hZf7FuNKh4W4SrkoaHIF5f62UxJ +GhiZmyKuRxd86JxUJVO1nIYNk3UYelI9syrHchQq5ym1DolOrhs6vZ3cbk+Pa9VieU1f79jl42sq +t7JLOgi7XbN6YVbPvSXNFxnfMEKdNaIbrB8oZvekeDNeQG5mlC77l+3LXA8w/Em62jC4yRaj8ogf +d1kL25p76UHIut1hEwhaduA41EM8Kh9Vi80/xUD9iXM17VnnEzyw5ovh9awYNI+/0UAzudI6mam8 +M5y/0WSMzn2bobebjE2+bzP0fpMxIk/YDL3FZKtse0TcETvmZ3NanuM4nn9/Kn+AFlgOFXwHkPV7 +Uvx15EGe5v3AdkM8cAjHffzIoxYcNylBbJdCS4Zp4l0N4YE3g9axTZFjuwTxHz9y3L79AONtcVxn +jN0Jm4kYimX7HDZD5pX3UWOGASZkYjelH3oJccK2dnZIxQbnQjNohfVhy0IumSPbBapTQENwpI+n +QK8NMQD0+mNAF/HCc2yHW/G3KV6ioe0anIXGxPGHkXOPApxEmISWZ0ZTUFLSCGuPw9CnBUGADpq5 +RbUZZIC9GN+eDzDHSjhO1mcHbQpRyCSNKAK5YkluxcQOhSNeQNixLplicBTNbByzMCcDpq/p3dVT +FHmHAWFsCsTlIyqkyJIKoJQM6hgcKCKezxiCdSHUrZWC/MCzIjNkCvkUkHSPwG6iEtC4SyH9xlyw +FQ2OIdeQ4GkMW7HjxEQswshSMILJcl4QWwywKCS3kNGKXQVWAj9K9dQnyPG8CWNrjoOYG7YgW5yZ +BnCk58zWPpYAZTqGxZ3lrxxXYrcBGcFMugVORVb+wzjjuIodL4kmrjf/ba3jCSE+bFnYnCBvGLMO +H/ECicZpLDaIY4NnRnCSdsEIiMYPfVK9rNTAlMCMArFBgiE2ya+oZ1P7JesAKUaeAqKBaQ8sP/KQ +ZQP8DxNfX6kTwioK7/lR7AAG95ILrI2vy2pWBs9nDwqgD3zYSsVgJksmA6n7gsRxiVkvCWz4Shbs +C+APAuHi4lFqoLXOMqgax6TpGRxbArTIpBqAT9jDoW3CSYVJNPQcx5tnwDSe+ymMDZOQWWkTJ+gd ++ZhSOE5bv6HSMgmSJGa2WNCsBs3B9QwODht8EjJWzDWz+CAKkyC3wbwuSdwcnImEqQ+m5CHuwTUz +6JQpC0INGIh3kHX/nUhL1XsHSgpwmBkLM1lwPFCIsJp0GTBSbLepxeLGerZdGgYQYLB5ULalxDG3 +WiXhLNnVgNERe/oBQkIMASrlvCdVlEHNRHmJkKBptj2z6ZSQp9z61ySLvHRBCK13V/gC9ifBzGbe +wJ51JLG1hequmQETrq4/kgMyz07I/zb+vFQ1DqfOt2WqnV8rx+XOeauKGC3U6paO6mW0wQtCXykL +QqVTQWfsSSs7WqOYos3sgx1BqDY30AazPBh+Pje4eWauZLxgJHROhQWjJrHp6dcEoKdzM1Zobexy +O6yHfRBs7XII7UwhmGNf4tlz4pmhVDbuCr2BUrTPOm7lX4uvVGIFGNwGEmJ6NFw6SQZezzAp3WB9 +CIWJL1joP/EtQgNwM9CWCQ6JfQoqXn3bjgf8H6MoxCSBZyFhemfgWZCvWCNbYm5bDJBIovivX+0p +c0XswhEKTljg/AAZt8GfrcQU4nbMx45lz27nr3sl0V+g7Rgh8LDLjsZwtJfy/mIbwhe0McRT21kW +0KejyLQtjPYMDtRrkU9bPRJYsPlsFQMbO1sUdM6Dg9rDdCK1b0AwSWaUQDwvKHzMxtd2qpX7/CRD +0MeByn5gzNuAjcGtoM2OAPRSyv7uY5izI/i70PFmrGNwr6GdN2Mdg3sN7bwZ6wDCYmvvgPMGZMgU +9wzw2QC2HdbvAq6IH0EHG7v3xhrc7egdAe9mUKqi96MlSKNP46Wn0dIT3K+zpsG9Ap1eFOxe8jW4 +F8gkEnfeiMHA6V9FYW/EYJDcX0BhqQneBsRA7U9CsZTIPTy2Mwhgq3oAytKBgMwSUt+OzSCgDe49 +2OwlX7BBV29x58eQi1n3PrK7Ded3Y7uHLBrcG2He60yDX71MInHTF9FiqljYGb4ZLaZY0eBeQIsr +n3kbZIRj3Kug8Y2QkUGv10Djk5DxsYsZ3LvwI6z5rCkBW74TjQK1xKoP8ajBfQ0ifRKPQuC8H5Ey +y8bII06kO9QHO63y86c0hdfi69PuPZgJWAXG7u4IDKLEiCVBWyzB2yGZUhPSSfxtBZBSLEfNMQgV +I7nYBdOXShu3sMCyqe/AjrIsuJ5LNnZT2MBowZ7psyE4VtrG21aCgBsJPZvMi8ms3RgvgGdO7tGM +AmfjhX0pNjZEydNBLyREY4h5l6iLp/eBJeMj1r7BgYexaYnuk99fCd7h3PCjK2P+HtdT9V9y5lRS +JFXKaWpeVHN6Ts3ns+LXF4C9Vv8lyuK6/ksVFVb/lctK/9R//Teu/8X6L12vFtVvqP9SxIKq//D6 +LzmTf6r+q9g/2kpqwMQfXgP2TKXWMxUoL3c/qNTS3lOplVdK+do3VGo9NvjjMqNnSp+e7XlQqaX8 +lyu13l4FFGuwokga7PG6JGez361S6xkVvr1S6+3lZ3H1WbVUrspSuVSVf85KrfeJU6xVS1oizute +/rer1Hpv+dQ3VGq9tyhsXam1Ppq6XvyYYx4fJf3fvltRFs4puvWUUvKS9NcqypLyP2VRFhlqN165 +oUuji173Cotdunm1lLs9s3W4SZvjC8+cuYqiTIY31R9QzRPOWsv6jVKea5KrRjK1j+3lqV1tFA9J +ZVqN5EpjJLWpqmQ3xVZrNuiMgqgf9AcXE+fgZqlqp5V6y86OvcPNrq2NbkTBHmud5cnZnhb6tQa5 +yC7qWX3veulH0ol85kndWaT1W+cVkndzFbl0IOTnLW0zur7YrPhFnHOspi17/n5vNrW0vcNydCQO +m3mnP7Mq0+yPLcr6uc1Ir498tecfDoOlM+rIzdzNYE4qVLFMs3rWyGbn3qQhBAv3Iqv7ujA/rxw0 +G4PhsJRfnp1dYKl0IB6eDh2hW7v2Wvnzi9Z5kJPxZlXWpe5o8+ZakrsHttroyz133mgdLIr+5pk+ +PiG6WLxpErNUuTruurp6bZcP3Vnvahx2hKMeaU310qA5znfnZbWvXI+F69pVb/KeoqzTxrGVHVxv +/hxFWWzWdNY8xAvXGbv7IZ87qJzW6XM1WHbZPpbdw8XeeLQq8SkuOmU6j/hnirweFW49KgJiRF4v +3Ko07bl3SDRvOVxWrvDZk0VAjNSDKiAYNArq067bwycVcrg3lTsdr14syPL3Kel63rw/T0nX39Xg +jwrCNEl5oSDsJRjxkxSEvVaF8KAmgXvwWjXGvpKyFf/ZjhT/FRLHziDxKjalEfn35wc1G/HT8YSK +MMcONOGJJ8RjqSCpX7bYw++AIMszOELZawZKyJQ9uWYPsd3lGg1m0DJ5GxC/12GPtZOJsPIUogzu +oNlz2YN0VlNjw4ERwxnQS56uU9siAxyw1xGA4tNH7n/EbCD2x1mg3ydZ/2JwyGYP2sMImF8iipc0 +eYgfT2Vv/7RbFjlunghjszdlHlhljWVH3m8/uAjkve75TxHIP0Ugf5kikHuUE5UApvThyKP6iw0E +QBTz64dylQ15Y3cnsVGsVdYkiQYn/gvcgDhOqirWLCYt1MfmbUtizeSGvcgJg+QtEBh8RQ8chVlq +4yn9A0vMXKlPxGxK0GR57EiIPpoq+9lOXlcBVdjNHrL6NYwyTkB5ibsygryDQcVhYWizI8JquVtx +Hl9MwBl24PSRSAisPyWgmIoYf2yj1HGz4HZ3Vkkp2tMRKyV24ri7TTcbKPFM1pplvAfmvbdzxITT +T2YU4BlYNkiyTPzV4ARFHUhkIIqiCacXrCoS0YaKaWr5vDYksiX/xhwVqP1/O2e32yYQROH7fQpk +KelNaQA7TVW6lnrTF2h7XYEgjpWAkXGUKk/fmeFvzQKmaVRZyvkUyXgJy3J2dnYHxqy8yygrwiS9 +jR7l/FL7xfLrRfCN/qhVZclPzI1giYq3GcUuJW0Env+RPurD3er07ifvQ5FvpGLWfVPpQ3ObNL8e +bRz09wcg9WnId7u49LPDD0BD8cNuQouXveQISTFVFTruUxrfbw9ubUL7KNnybXGR2812z4M7rDKt +qEM6s2Kdr47658vVIRk0BdPSxRzZLw4YgumeFp3piOEcn6loD6eqhv1Q7YWa79/J/zg/8y3NP73S +1lNp9f5HdLfLolFnJcIMeMLahcQ0JPmunhfSOsB0K8fetnF3fuwnfmoZeQ+qimaP/S7frI/XVzwD +SPH4sfzYvldVp9w/6aaVqdwp3ZaWbsF1q5tWnXL+tSHnhHSxYEmnlbXknJBjXDaaHzhv7SEqSxbq +OXHrldnCnpCCOEiD25HZUNTTqtXPktl5V0lHW42WzpiKq+K3Vn0dg06wZq6g2ao43U6t7Ja+sJ1a +2S2d007Os5I0DXJwi7WEBloZwcFAAti8OKFKyVFVUk6VnOQY2Tm04NpvZJW2+BU/RPl9l3o1XD/N +OL0zcJqOEY3Mj0X4FwqT0cj8WIR8exeNTGtl5CgNysG5LycFkWs+Gd/wwGkuvEoIe8mYEUvU6jXG +jFgiafU3tjgVmGkllyXLyRm+xJwU6UuzWKLNKOYF1tr4L9kve6i8XbAi6+rss66momVkXb056vyv +PH06h/e/Lb0b35P3v3krvP/tf1D3/yErzqX//er9f+h/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABg +lD9gaWU0AHgAAA== +--=-=-= +Content-Type: text/plain + + +PS: don't we have a "you forgot to actually attach the damn file" plugin +when we detect the word "attachment" and there's no attach? :p + +--=-=-=-- diff --git a/test/corpora/crypto/basic-encrypted.eml b/test/corpora/crypto/basic-encrypted.eml index 1ba4698a..b139a735 100644 --- a/test/corpora/crypto/basic-encrypted.eml +++ b/test/corpora/crypto/basic-encrypted.eml @@ -17,12 +17,11 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBACp70e7KPy9OYaheIrkLzmhq1lRqmy51aL1jBL0K/qN7rfK -BZEG1cR8jeLjTFdPKPLVKJI80r7FgKI0ywvWvl6R1aE1Ty5BnVXT9XzCrEH7fqCl -SKK82EvolXTohAZHUrh6K66eQQTTIAC1n7B0A8hErzkgaM4+seN3LlvezT6TLNKM -ATpqsEbM2MVrGgw0b3oUsGGAPEt2MmjNEYsriKnqwt6dJDZc//XyhjgMQayiD8da -N1gT3oqgu/gKCpBZDYzHf9OtVi2UnlFDWy6rrMZLjWDnIv4ve9Pn/qolwHVjzdJ1 -ZfjNC5t0z3XADKGrjN9wutr4qm7STW1rHAXHP68TQTxI0qgJKjPXNKWEw6g= -=pJG4 +hF4DHXHP849rSK8SAQdAYbv9NFaU2Fbd6JbfE87h/yZNyWLJYZ2EseU0WyOz7Agw +/+KTbbIqRcEYhnpQhQXBQ2wqIN5gmdRhaqrj5q0VLV2BOKNJKqXGs/W4DghXwfAu +0oMBqjTd/mMbF0nJLw3bPX+LW47RHQdZ8vUVPlPr0ALg8kqgcfy95Qqy5h796Uyq +xs+I/UUOt7fzTDAw0B4qkRbdSangwYy80N4X43KrAfKSstBH3/7O4285XZr86YhF +rEtsBuwhoXI+DaG3uYZBBMTkzfButmBKHwB2CmWutmVpQL087A== +=lhSz -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/crypto/encrypted-rfc822-attachment b/test/corpora/crypto/encrypted-rfc822-attachment new file mode 100644 index 00000000..e7bad9b0 --- /dev/null +++ b/test/corpora/crypto/encrypted-rfc822-attachment @@ -0,0 +1,47 @@ +Content-Type: multipart/encrypted; protocol="application/pgp-encrypted"; boundary="===============9060418334135509864==" +MIME-Version: 1.0 +From: Notmuch test suite +To: test_suite@notmuchmail.org +Subject: testing encrypted rfc822 attachments +Date: Sat, 03 Jul 2021 16:00:02 -0300 +Message-ID: +User-Agent: alot/0.9.1 + +--===============9060418334135509864== +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: application/pgp-encrypted; charset="us-ascii" + +Version: 1 +--===============9060418334135509864== +MIME-Version: 1.0 +Content-Transfer-Encoding: 7bit +Content-Type: application/octet-stream; charset="us-ascii" + +-----BEGIN PGP MESSAGE----- + +hF4DHXHP849rSK8SAQdAtWMNBhjdh9Xyig/GYM1RN27B7LmCk0oRkguuFMROXk8w +wLmcm3HiikqziJzlPY8sBLEx7cLQCkCrg6KarCHGvcqajUJluLAZgrwUsmUpOFy0 +0ukBQ6EnlpNlxdcmOElG9ZYn2Dgp1Gq4eo9rHb9f/TBLxuTr1Yr0vZ5f38bVn+tB +0FupuYE5h/PAM2JPjVg4rh1pIN2yWvNdFBHCm0Yt3AquS9bwQymUOjh0ZCey5ERy +WJtUQ7u+f2Hfm5K9CPPqlM7GPqeR8nv2/cgJxsA9nGn4tco7pHEkM1Kwe1dLlTbh +ZqEMKu+dVK/RqRUjDVdAonM7wz3Uah6HJH3a6W4g2mPiPPkOpnbTClCLiK0H8NRv +9rbO69kHoNar0YTpTAAbkxlKkdKeM/xAS2UFIdE+tXSofeCHmsFCmcohKH7Rt66l +rU7BJE7dtTiaPFYgHBuzbrkA+oH8gU9l7PQ1L16zo1rRVRb8IIBPSHY64Ekpy3L4 +FQ3Ttg8e26d2X+6eGca6RIXksTfqOniC33Fal6gBIUqP1w4ky1wjJ+wujkSuf0QB +yFWPP3o16o9X3nEIa2L8v/engSM5p7pezZfVJej/5hWnuPP3YmbDpn0ir21Qye96 +qxq7XI7GSWukxisxC8WakNA1LHnXo5DvHB8wm3lJIfEfIJ9RHhZqyRMptC563XCZ +A1NvY+4sKgx9g8PBRai019ZaTMYJRQqE3VlU+xHbJWMf4aymYyeY8ut8HvFhC9Tk +4D3eol7/LgHhXA9Db4L5MJX8iHtXhqyt62vmMXcFzC1Wi8Co/HnchzGQx2vXnUIZ +IlomEAvN1RZC2UV8q2NHKzKnfr43WfnIsdt1fQwEfQeTKX6wfhq8FDR5NmEeLeTw +ddGA95bZEkT2xYh+2g7G1c5Z33l5WLeriA/SUMWK1oafUtcx81lJnmK/xSIg3PKZ +jKIbTwv4tKjmGUJE5xb/RTUfOTG+v7HOBo0avwofvqz5k7Do8v0BEEmhewjcIKBf +h2Go8VxFtDj/OG0J98PS3OVQtnv4jzepXUxr2ZW7Z0aPFUHQ2F97Yio3nxb7r7um +OjKhNgIPp+iAWisqqw+kwWxPx/5GHxVYsOM+fqpFEjELGGNv+ImTG7jS1V5agqIr +qU+dxYydDKnQi/5xXeE4TO0MEFhCIttOQARw3DPtZmjLuXFjqvaasTV4YyXwrKCJ +0sfemg3wxkZxY0NI13g3CaJGtSzXdZVfkzpdVxRAZoETT7rLNDbawuq0AzSava2u +nB/vgcDWfPr4jW21 +=r4Vv +-----END PGP MESSAGE----- + +--===============9060418334135509864==-- diff --git a/test/corpora/crypto/encrypted-signed.eml b/test/corpora/crypto/encrypted-signed.eml index 0345e3e9..f5d5d120 100644 --- a/test/corpora/crypto/encrypted-signed.eml +++ b/test/corpora/crypto/encrypted-signed.eml @@ -17,19 +17,18 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBAC9z781zV7QAInGMKHX6TKU5Xw/OkoWXahpDL88F6Ocm5R9 -7M9z2ocvlyrbgRhqE+nvFeGH/K7rVkBBT6TAcdIe/C8Qzbd3stPPcx1PlunGROj7 -H/WAcmDksK3HkXpHwmInUtzNw1pkhOoLy/sFSbPvtyg8GCUzXbafHAIIo0rB2tLB -DwGWD3l4WdcyQWuYD9QJKuDIqdWo8E3TTcKkiOAt/6liwPNZ0jGzDeCuSTnWFj6Z -AiXGeNtD3I1tCN/8T3NjEKOCQ+bdT5Y06dDaL61FpQ23eIuSUgksVxjnkEAb6iPe -07gjzcyNuGP3WPI/0qu0wtZwpAQxvaNygDsQj/OjR5kn9luBd/VqodM3TWWS8miV -m0z1tYbqYAQWW6TS7fXlsyXoOxTLW5MCfe3D36VSErL/NJItETklVKzNfKjMmRKx -CI2ZUzugxPWSLQzOp5yl7iICk8e+vS9TkQw2j0nXAQYLYgmqZMhf4av5GlFv3tQu -heO4XLT6NBDTHMFTDbgW42kE0N4MDPc29AqVFGImcTHvflF4Vp0qIbSJdIcHwKkU -5LKqvicAa0lsIoJbsW3lHrzowyjov2vLH/VGd/wIX+MS3KT7cySdyp8HVMcwwyZu -Y9nrTN/7G1FwKWlcGa4uJNcFFkYlcEymZj1EX2cyrdezPtX7K5vhwBYddptFD+Bn -IVkghRut3UDeXe83F8OutWiZfK5EVYABq/aP3//hIbQl2o4Dkd3z9m+8LobrIV5s -NXjAjU5WQOjRLoHBebG2HkMpFsWhXD/Fb/Bb58VOpdI= -=x12v +hF4DHXHP849rSK8SAQdAwRMXQWEiw2ZldkVtILgy2w/vFrKxAgAxFmyDGvtbiTow +xLMHH7PgH+S0JIjEOM0jFglpyMdzvNBWII726THKbFnOcR38CsbK55lCYw2uhABS +0sD1AU/yQZNNDwjuJT6wjtxMdfI4DaFMD5XTpioNtRcphLpN7MHbYr4MYCxszRY7 +ZLIAkUCoY/3D26dcOkB0EMGELsRsPmKaCq/m3FVvxHHku94MzLnjV/eGRE40AiSe +JzteAKwXzUAYzYw+LCj9WPZvy1Rs37I4VgukEgSYXMidgc9fUQ4yYKesbyQ8/iMW +Ryo+X2yu7Iv3a7pp2qdArsJwatWpyRuASVVA7nZlNQS0YGh/fuSX3x8TJuSPliFr +sdTVuE6sJIlqttH9CRgxMjLY5YbpF3lBTqlv2tmz0VERhWKKsh5VOiUvJvZ3p5hn +FqnD1bcoWoBPZRNOoF6PzGkvWQIqysGLgg24wZr7ZRgZ8mHRgykBN6cmZHQKf16m +zqqf8sppaRal6d/L72EzmeHEusUn6FlWcEDlXLc5anghYdna7qhbLqsoY7X504SQ +Mx4tj1P4XVhgoXdLRR2EHOLrzUCrkiQ9cKfoAcUFJTcbGGYYiYwzUCkZWhsHOKsQ +y7tajvWUzGaJ8aiZ1dfdUraOzrvOOif4TdnFJrTpM0Agy6IH8tbm8EnhNOxkjQPr +3t7eS3JcuC0= +=qz8x -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/default/bar/baz/05:2, b/test/corpora/default/bar/baz/05:2, index 75b05fa4..ce174b52 100644 --- a/test/corpora/default/bar/baz/05:2, +++ b/test/corpora/default/bar/baz/05:2, @@ -63,7 +63,7 @@ and +Envelope-to: david@tethera.net +Delivery-date: Wed, 28 Nov 2012 18:41:46 -0400 +Received: from [199.188.72.155] (helo=yantan.tethera.net) + by tesseract.cs.unb.ca with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) + (Exim 4.72) + (envelope-from ) + id 1TdqKA-00017j-3c + for david@tethera.net; Wed, 28 Nov 2012 18:41:46 -0400 +Received: from wagner.debian.org ([217.196.43.132]) + by yantan.tethera.net with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqK9-00072z-AF + for david@tethera.net; Wed, 28 Nov 2012 18:41:45 -0400 +Received: from localhost ([::1] helo=wagner.debian.org) + by wagner.debian.org with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqK8-0007GZ-67 + for david@tethera.net; Wed, 28 Nov 2012 22:41:44 +0000 +Received: from vasks.debian.org ([217.196.43.140]) + by wagner.debian.org with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqIc-0006jm-OC; Wed, 28 Nov 2012 22:40:11 +0000 +Received: from gladky-anton-guest by vasks.debian.org with local (Exim 4.72) + (envelope-from ) + id 1TdqIc-0003j1-DE; Wed, 28 Nov 2012 22:40:10 +0000 +Date: Wed, 28 Nov 2012 22:40:10 +0000 +From: Anton Gladky +To: 691896@bugs.debian.org, control@bugs.debian.org, + 691896-submitter@bugs.debian.org +Message-ID: +X-PTS-Approved: Yes +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on wagner.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM + autolearn=ham version=3.3.1 +Subject: [87ea161] Fix for Bug#691896 committed to git +X-BeenThere: debian-science-maintainers@lists.alioth.debian.org +X-Mailman-Version: 2.1.13 +Precedence: list +List-Id: Mailing list for maintainer discussions and BTS messages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +MIME-Version: 1.0 +Content-Type: text/plain; charset="us-ascii" +Content-Transfer-Encoding: 7bit +Sender: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +Errors-To: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +X-SA-Exim-Connect-IP: ::1 +X-SA-Exim-Mail-From: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +X-SA-Exim-Scanned: No (on wagner.debian.org); SAEximRunCond expanded to false +X-Spam-Score: 1.3 +X-Spam_bar: + + + +tags 691896 + pending +thanks + +Hello, + + The following change has been committed for this bug by + Anton Gladky on Wed, 31 Oct 2012 08:16:42 +0100. + The fix will be in the next upload. +==================================== +Minor fixes in README.Debian. (Closes: #691896) + + +==================================== + +You can check the diff of the fix at: + + ;a=commitdiff;h=87ea161 + + + +-- +debian-science-maintainers mailing list +debian-science-maintainers@lists.alioth.debian.org +http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-science-maintainers diff --git a/test/corpora/duplicate/msg-1-2:2, b/test/corpora/duplicate/msg-1-2:2, new file mode 100644 index 00000000..dc0476e7 --- /dev/null +++ b/test/corpora/duplicate/msg-1-2:2, @@ -0,0 +1,113 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Wed, 28 Nov 2012 18:42:39 -0400 +Received: from [199.188.72.155] (helo=yantan.tethera.net) + by tesseract.cs.unb.ca with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) + (Exim 4.72) + (envelope-from ) + id 1TdqL1-00017s-3b + for david@tethera.net; Wed, 28 Nov 2012 18:42:39 -0400 +Received: from wagner.debian.org ([217.196.43.132]) + by yantan.tethera.net with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqL0-00073Z-Gw + for david@tethera.net; Wed, 28 Nov 2012 18:42:38 -0400 +Received: from localhost ([::1] helo=alioth.debian.org) + by wagner.debian.org with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqKz-0007sR-PR + for david@tethera.net; Wed, 28 Nov 2012 22:42:37 +0000 +Received: from buxtehude.debian.org ([140.211.166.26]) + by wagner.debian.org with esmtp (Exim 4.72) + (envelope-from ) id 1TdqKU-0007SW-IG + for debian-science-maintainers@lists.alioth.debian.org; + Wed, 28 Nov 2012 22:42:07 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.72) + (envelope-from ) + id 1TdqKR-0001dH-UL; Wed, 28 Nov 2012 22:42:03 +0000 +X-Loop: owner@bugs.debian.org +Resent-From: Anton Gladky +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Science Maintainers + +X-Loop: owner@bugs.debian.org +Resent-Date: Wed, 28 Nov 2012 22:42:02 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 691896 +X-Debian-PR-Package: gmsh +X-Debian-PR-Keywords: pending +X-Debian-PR-Source: gmsh +Received: via spool by 691896-submit@bugs.debian.org id=B691896.13541424145158 + (code B ref 691896); Wed, 28 Nov 2012 22:42:02 +0000 +Received: (at 691896) by bugs.debian.org; 28 Nov 2012 22:40:14 +0000 +Received: from wagner.debian.org ([217.196.43.132]) + by buxtehude.debian.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) + (Exim 4.72) (envelope-from ) + id 1TdqIg-0001Kh-Ba; Wed, 28 Nov 2012 22:40:14 +0000 +Received: from vasks.debian.org ([217.196.43.140]) + by wagner.debian.org with esmtp (Exim 4.72) + (envelope-from ) + id 1TdqIc-0006jm-OC; Wed, 28 Nov 2012 22:40:11 +0000 +Received: from gladky-anton-guest by vasks.debian.org with local (Exim 4.72) + (envelope-from ) + id 1TdqIc-0003j1-DE; Wed, 28 Nov 2012 22:40:10 +0000 +Date: Wed, 28 Nov 2012 22:40:10 +0000 +From: Anton Gladky +To: 691896@bugs.debian.org, control@bugs.debian.org, + 691896-submitter@bugs.debian.org +Message-ID: +X-PTS-Approved: Yes +Resent-Sender: Debian BTS +X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on wagner.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,FREEMAIL_FROM, + RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 +Subject: Bug#691896: [87ea161] Fix for Bug#691896 committed to git +x-debian-approved: yes +X-BeenThere: debian-science-maintainers@lists.alioth.debian.org +X-Mailman-Version: 2.1.13 +Precedence: list +Reply-To: Anton Gladky , 691896@bugs.debian.org +List-Id: Mailing list for maintainer discussions and BTS messages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +MIME-Version: 1.0 +Content-Type: text/plain; charset="us-ascii" +Content-Transfer-Encoding: 7bit +Sender: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +Errors-To: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +X-SA-Exim-Connect-IP: ::1 +X-SA-Exim-Mail-From: debian-science-maintainers-bounces+david=tethera.net@lists.alioth.debian.org +X-SA-Exim-Scanned: No (on wagner.debian.org); SAEximRunCond expanded to false +X-Spam-Score: 1.3 +X-Spam_bar: + + + +tags 691896 + pending +thanks + +Hello, + + The following change has been committed for this bug by + Anton Gladky on Wed, 31 Oct 2012 08:16:42 +0100. + The fix will be in the next upload. +==================================== +Minor fixes in README.Debian. (Closes: #691896) + + +==================================== + +You can check the diff of the fix at: + + ;a=commitdiff;h=87ea161 + +-- +debian-science-maintainers mailing list +debian-science-maintainers@lists.alioth.debian.org +http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/debian-science-maintainers diff --git a/test/corpora/duplicate/msg-2-1:2, b/test/corpora/duplicate/msg-2-1:2, new file mode 100644 index 00000000..e118d784 --- /dev/null +++ b/test/corpora/duplicate/msg-2-1:2, @@ -0,0 +1,43 @@ +From: David Bremner +To: Samuel Bronson , 695159@bugs.debian.org, Debian Bug Tracking System +Subject: Re: Bug#695159: debian-el: Shouldn't put downloaded bugs loose in ~/ +In-Reply-To: <87vcch7hxy.fsf@naesten.dyndns.org> +References: <87vcch7hxy.fsf@naesten.dyndns.org> +Date: Thu, 25 Oct 2018 10:41:38 -0300 +Message-ID: <87r2geywh9.fsf@tethera.net> +MIME-Version: 1.0 +Content-Type: text/plain + + +Control: severity -1 minor +Control: tag -1 moreinfo + +Samuel Bronson writes: + +> Package: debian-el +> Version: 35.2+nmu1 +> Severity: normal +> File: /usr/share/emacs/site-lisp/debian-el/debian-bug.el +> +> Dear Maintainer, +> +> After being mildly annoyed with this for ages, it finally occurred to me +> to file a bug about it: +> +> It's rather rude of `getdebian-bug-get-bug-as-email' to default to +> sticking downloaded mbox files loose in ~/, isn't it? +> +> (And might it not make sense to try and use the same files as the bts(1) +> command from the devscripts package?) +> + +Hi Samuel + +There is already a variable "debian-bug-download-directory" which can be +customized. Is there an obviously better default? I guess we could put +things in /tmp by default, with a minor privacy leak on multiuser +systems. + +d + +# body 1 diff --git a/test/corpora/duplicate/msg-2-2:2, b/test/corpora/duplicate/msg-2-2:2, new file mode 100644 index 00000000..a7549c47 --- /dev/null +++ b/test/corpora/duplicate/msg-2-2:2, @@ -0,0 +1,140 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 25 Oct 2018 09:45:10 -0400 +Received: from muffat.debian.org ([2607:f8f0:614:1::1274:33]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1gFfwj-0004Y9-69 + for david@tethera.net; Thu, 25 Oct 2018 09:45:10 -0400 +Received: from ticharich.debian.org ([2001:41c8:1000:21::21:23]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=ticharich.debian.org,EMAIL=hostmaster@ticharich.debian.org (verified) + by muffat.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1gFfwi-0004A1-J6 + for david@tethera.net; Thu, 25 Oct 2018 13:45:08 +0000 +Received: from localhost ([::1] helo=ticharich.debian.org) + by ticharich.debian.org with esmtp (Exim 4.89) + (envelope-from ) + id 1gFfwh-0002Ex-6w + for david@tethera.net; Thu, 25 Oct 2018 13:45:07 +0000 +Received: from mailly.debian.org ([2001:41b8:202:deb:6564:a62:52c3:4b72]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=mailly.debian.org,EMAIL=hostmaster@mailly.debian.org (verified) + by ticharich.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1gFfwg-0002Em-NI + for dispatch+emacs-goodies-el@tracker.debian.org; Thu, 25 Oct 2018 13:45:06 +0000 +Received: from quantz.debian.org ([2001:41c8:1000:21::21:28]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=quantz.debian.org,EMAIL=hostmaster@quantz.debian.org (verified) + by mailly.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1gFfwg-0004h1-AC + for dispatch+emacs-goodies-el@tracker.debian.org; Thu, 25 Oct 2018 13:45:06 +0000 +Received: from qa by quantz.debian.org with local (Exim 4.89) + (envelope-from ) + id 1gFfwf-0007Lg-TQ + for dispatch+emacs-goodies-el@tracker.debian.org; Thu, 25 Oct 2018 13:45:05 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=buxtehude.debian.org,EMAIL=hostmaster@buxtehude.debian.org (verified) + by quantz.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1gFfwf-0007J0-3r; Thu, 25 Oct 2018 13:45:05 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1gFfwc-0003jj-VU; Thu, 25 Oct 2018 13:45:02 +0000 +X-Loop: owner@bugs.debian.org +Subject: Bug#695159: debian-el: Shouldn't put downloaded bugs loose in ~/ +Reply-To: David Bremner , 695159@bugs.debian.org +Resent-From: David Bremner +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Emacsen team +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 25 Oct 2018 13:45:01 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 695159 +X-Debian-PR-Package: debian-el +X-Debian-PR-Keywords: +References: <87vcch7hxy.fsf@naesten.dyndns.org> <87vcch7hxy.fsf@naesten.dyndns.org> +X-Debian-PR-Source: debian-el, emacs-goodies-el +Received: via spool by submit@bugs.debian.org id=B.154047490313415 + (code B); Thu, 25 Oct 2018 13:45:01 +0000 +Received: (at submit) by bugs.debian.org; 25 Oct 2018 13:41:43 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.1-bugs.debian.org_2005_01_02 + (2015-04-28) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-19.5 required=4.0 tests=BAYES_00,FROMDEVELOPER,GMAIL, + HAS_BUG_NUMBER,TXREP autolearn=ham autolearn_force=no + version=3.4.1-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 16; hammy, 110; neutral, 41; spammy, + 2. spammytokens:0.971-+--privacy, 0.857-+--customized + hammytokens:0.000-+--UD:el, 0.000-+--H*F:U*bremner, 0.000-+--Maintainer, + 0.000-+--sitelisp, 0.000-+--site-lisp +Received: from fethera.tethera.net ([2607:5300:60:c5::1]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1gFftO-0003U6-RV; Thu, 25 Oct 2018 13:41:42 +0000 +Received: from remotemail by fethera.tethera.net with local (Exim 4.89) + (envelope-from ) + id 1gFftL-0004VK-V8; Thu, 25 Oct 2018 09:41:39 -0400 +Received: (nullmailer pid 477 invoked by uid 1000); + Thu, 25 Oct 2018 13:41:38 -0000 +From: David Bremner +To: Samuel Bronson , 695159@bugs.debian.org, Debian Bug Tracking System +In-Reply-To: <87vcch7hxy.fsf@naesten.dyndns.org> +Date: Thu, 25 Oct 2018 10:41:38 -0300 +Message-ID: <87r2geywh9.fsf@tethera.net> +MIME-Version: 1.0 +Content-Type: text/plain +Delivered-To: submit@bugs.debian.org +Delivered-To: emacs-goodies-el@packages.qa.debian.org +Delivered-To: dispatch+emacs-goodies-el@tracker.debian.org +X-Loop: dispatch@tracker.debian.org +X-Distro-Tracker-Keyword: bts +X-Distro-Tracker-Package: emacs-goodies-el +List-Id: +X-Debian: tracker.debian.org +X-Debian-Package: emacs-goodies-el +X-PTS-Package: emacs-goodies-el +X-PTS-Keyword: bts +X-Distro-Tracker-Team: emacsen +X-Spam_score: -2.3 +X-Spam_score_int: -22 +X-Spam_bar: -- + + +Control: severity -1 minor +Control: tag -1 moreinfo + +Samuel Bronson writes: + +> Package: debian-el +> Version: 35.2+nmu1 +> Severity: normal +> File: /usr/share/emacs/site-lisp/debian-el/debian-bug.el +> +> Dear Maintainer, +> +> After being mildly annoyed with this for ages, it finally occurred to me +> to file a bug about it: +> +> It's rather rude of `getdebian-bug-get-bug-as-email' to default to +> sticking downloaded mbox files loose in ~/, isn't it? +> +> (And might it not make sense to try and use the same files as the bts(1) +> command from the devscripts package?) +> + +Hi Samuel + +There is already a variable "debian-bug-download-directory" which can be +customized. Is there an obviously better default? I guess we could put +things in /tmp by default, with a minor privacy leak on multiuser +systems. + +d + +# body 2 diff --git a/test/corpora/duplicate/msg-3-1:2, b/test/corpora/duplicate/msg-3-1:2, new file mode 100644 index 00000000..8bb6a8c2 --- /dev/null +++ b/test/corpora/duplicate/msg-3-1:2, @@ -0,0 +1,183 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 20 Dec 2018 13:27:11 -0500 +Received: from muffat.debian.org ([2607:f8f0:614:1::1274:33]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32M-0005ae-Ko + for david@tethera.net; Thu, 20 Dec 2018 13:27:11 -0500 +Received: from alioth-lists-01.debian.net ([2001:ba8:0:2c77:0:4:0:1]) + by muffat.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32K-0008PT-Nm + for david@tethera.net; Thu, 20 Dec 2018 18:27:08 +0000 +Received: from localhost ([::1] helo=alioth-lists-01.debian.net) + by alioth-lists-01.debian.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32I-0003to-Kn + for bremner@debian.org; Thu, 20 Dec 2018 18:27:06 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + by alioth-lists-01.debian.net with esmtps + (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) + (envelope-from ) id 1ga32H-0003tO-29 + for pkg-emacsen-addons@lists.alioth.debian.org; Thu, 20 Dec 2018 18:27:05 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32F-0005PP-9m; Thu, 20 Dec 2018 18:27:03 +0000 +X-Loop: owner@bugs.debian.org +Resent-From: Sean Whitton +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Emacs addons team + +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 20 Dec 2018 18:27:02 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 916805 +X-Debian-PR-Package: src:assess-el +X-Debian-PR-Keywords: ftbfs +References: <87k1k6h43h.fsf@zephyr.silentflame.com> +X-Debian-PR-Source: assess-el +Received: via spool by 916805-submit@bugs.debian.org id=B916805.154533033319811 + (code B ref 916805); Thu, 20 Dec 2018 18:27:02 +0000 +Received: (at 916805) by bugs.debian.org; 20 Dec 2018 18:25:33 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 + (2018-09-13) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-8.8 required=4.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PGPSIGNATURE,RCVD_IN_DNSWL_LOW, + SORTED_RECIPS,SPF_HELO_PASS,SPF_PASS,SUSPICIOUS_RECIPS,TXREP + autolearn=ham autolearn_force=no + version=3.4.2-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 32; hammy, 83; neutral, 22; spammy, 1. + spammytokens:0.902-+--emails + hammytokens:0.000-+--HX-ME-Sender:xms, + 0.000-+--H*RU:10.202.2.43, + 0.000-+--Hx-spam-relays-external:10.202.2.43, 0.000-+--H*F:U*spwhitton, + 0.000-+--H*F:D*spwhitton.name +Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) (envelope-from ) + id 1ga30m-00059C-Kv; Thu, 20 Dec 2018 18:25:32 +0000 +Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) + by mailout.west.internal (Postfix) with ESMTP id C50B712F6; + Thu, 20 Dec 2018 13:25:30 -0500 (EST) +Received: from mailfrontend1 ([10.202.2.162]) + by compute3.internal (MEProxy); Thu, 20 Dec 2018 13:25:31 -0500 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name; + h=from:to:subject:date:message-id:mime-version:content-type; s= + fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lqnNZlsr/mb10zqE=; b=X9S4kI14 + zxdg9IKMIyALL2Eao3GTyyojuATJkgCqJvZWd8RWpty6RBStZfyeOqR1L3Gr5m3/ + EVeiHQyFNnPor2xjMDmblfPS/u09JxVlc0KMpT0XXRfNWsVQn+U40nNRX15kXzZ/ + D1rYhxpxzKRzU2tByUULCgbGlXAwJQtOXMDw3mpj1BxcoO13H/0H/KQTQ+AcpiOw + BV3JFKL/jA+mH8uAPIgNM2mUYZz5REO89eh3lPhLyc7tw745X+4ywZlo/Piqa0+6 + BCldY9/nDR3csAUKx1+3hkpJPdqFALBWvG3SelGt44BqcoLsOJLB8QH6trCro39o + fEnBaUBTAkTAHA== +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lq + nNZlsr/mb10zqE=; b=QYqLjUBZtfbO0DUlagvXRP2BPnpTgkvIna9/uCewkdoIH + /LuPW6DZUNWQa55qiDquqKXcs0tTODTEzYgeIDgqC+DDBTHnFQvXdWyS1X3o4sLL + 8dTKk8lv7M1/zKFxyg/ycNvPJGS9m4ZucGbxjwdgAcozhg7W1Qztxt9eVhPVnenS + 5sdeJ9mjIE7lYkKX4QVsXPOi86j6QlfMNyi/OnBfX2+95QiA/xPE/wEq4MYlLNm7 + Av1P/8OrI4ImDKkOEivarktL+isYL7OXyGB4GfUTsydiy9dhP7RKPxrai1kJRu5S + b2470KXNatu2WkyMFrsdcwrSqyKIe096k5xPfVI2A== +X-ME-Sender: +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudejfedguddugecutefuodetggdotefrod + ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu + fedttdenucfuohhrthgvugcurhgvtghiphhsucdlgedtmdenucfjughrpefhvffufffkgg + gtsehgtderredttddtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhi + thhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqnecurfgrrhgrmhepmhgrihhlfhhroh + hmpehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgenucevlhhushhtvghr + ufhiiigvpedt +X-ME-Proxy: + + + +From: Sean Whitton +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, + 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, + 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, + 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 +Message-ID: <87r2ecrr6x.fsf@zephyr.silentflame.com> +MIME-Version: 1.0 +Received-SPF: pass client-ip=2607:f8f0:614:1::1274:39; + envelope-from=debbugs@buxtehude.debian.org; helo=buxtehude.debian.org +x-debian-approved: yes +Subject: [Pkg-emacsen-addons] Bug#916805: Increase severity to 'serious' +X-BeenThere: pkg-emacsen-addons@alioth-lists.debian.net +X-Mailman-Version: 2.1.23 +Precedence: list +List-Id: Maintainers list for Emacs addon packages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Reply-To: Sean Whitton , 916805@bugs.debian.org +Content-Type: multipart/mixed; boundary="===============5317466403067656157==" +Errors-To: pkg-emacsen-addons-bounces+bremner=debian.org@alioth-lists.debian.net +Sender: "Pkg-emacsen-addons" + +X-Spam_score: 2.8 +X-Spam_score_int: 28 +X-Spam_bar: ++ + +--===============5317466403067656157== +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha512; protocol="application/pgp-signature" + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +=2D-=20 +Sean Whitton + +--=-=-= +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlwb3pYACgkQaVt65L8G +YkDLrhAAhORxZzZDE5vlXRm89JYA3jd9OyleioZvDDRCrpEd7CQ5AHiMMJizW1lU +gn6OBIoW4O04TZ5oOuUnDHK/rS0G4zgNCJUyNf06zVECmdvkzspNNpQ3J5aOi4t2 +lhjRIFOKA9ifGsEqYLwP2dork1xFuyHEqHkDH8zpCTvdzkWky1bwAD/Pj5dArd7t +FeQGsPm7/64H1/rHk8pSP2pQgRsMDX6rIdx3vuQ7r+NssdRq+II4e479l02TiCDi +FBOX+n3nPXxREPdZ9EKL4SauL/AnRqpeC9GX6fC9OOnQeQ1xVTzNWKa6ixrqkFoH +TI/vy51p16jFNgdkLkyLtZA8Tq72TIAKWbZC0GFzWJVNASWu7WDIoMn5pgoi454w +TgsvK9MOnEYeABiDUa1ppaoMiP4+3j5yT0eWttTMSkcKjk1Ap1o+RfUxlIGl0Rog +ShbG2y6Mv8FERtjzPVQ7VMLDN9zRIbtlSJFm7CboPNSAygzzzaA/RIN/e8MdbZoM +a8AT9KiAVHEEcw+nWFAatAew5VP9iRZVgrVdWBszuaWOolxnYvpAL45WanqG0eab +VMe66+rZ8momI0MsM9JcqBwXO+fOf8CrPSO9PL8VFEJXFLZQS7asFStJf2l8msWE +3IYhvk4B6Nf1R96XzpXLlkOnoGtcnPVAvotrGU/rDfk5i/WF810= +=mWfF +-----END PGP SIGNATURE----- +--=-=-=-- + + +--===============5317466403067656157== +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Disposition: inline + +X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KUGtnLWVtYWNz +ZW4tYWRkb25zIG1haWxpbmcgbGlzdApQa2ctZW1hY3Nlbi1hZGRvbnNAYWxpb3RoLWxpc3RzLmRl +Ymlhbi5uZXQKaHR0cHM6Ly9hbGlvdGgtbGlzdHMuZGViaWFuLm5ldC9jZ2ktYmluL21haWxtYW4v +bGlzdGluZm8vcGtnLWVtYWNzZW4tYWRkb25zCg== + +--===============5317466403067656157==-- + diff --git a/test/corpora/duplicate/msg-3-2:2, b/test/corpora/duplicate/msg-3-2:2, new file mode 100644 index 00000000..292d515f --- /dev/null +++ b/test/corpora/duplicate/msg-3-2:2, @@ -0,0 +1,184 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 20 Dec 2018 13:27:12 -0500 +Received: from mailly.debian.org ([2001:41b8:202:deb:6564:a62:52c3:4b72]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32O-0005ap-Hn + for david@tethera.net; Thu, 20 Dec 2018 13:27:12 -0500 +Received: from alioth-lists-01.debian.net ([2001:ba8:0:2c77:0:4:0:1]) + by mailly.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32O-0004V7-4x + for david@tethera.net; Thu, 20 Dec 2018 18:27:12 +0000 +Received: from localhost ([::1] helo=alioth-lists-01.debian.net) + by alioth-lists-01.debian.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32N-0003wA-Sc + for bremner@debian.org; Thu, 20 Dec 2018 18:27:11 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + by alioth-lists-01.debian.net with esmtps + (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) + (envelope-from ) id 1ga32M-0003vQ-Q9 + for pkg-emacsen-addons@lists.alioth.debian.org; Thu, 20 Dec 2018 18:27:10 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32K-0005Sn-7B; Thu, 20 Dec 2018 18:27:08 +0000 +X-Loop: owner@bugs.debian.org +Resent-From: Sean Whitton +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Emacs addons team + +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 20 Dec 2018 18:27:06 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 916808 +X-Debian-PR-Package: src:hydra-el +X-Debian-PR-Keywords: ftbfs +References: <87bm5ih3w5.fsf@zephyr.silentflame.com> +X-Debian-PR-Source: hydra-el +Received: via spool by 916808-submit@bugs.debian.org id=B916808.154533033319830 + (code B ref 916808); Thu, 20 Dec 2018 18:27:06 +0000 +Received: (at 916808) by bugs.debian.org; 20 Dec 2018 18:25:33 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 + (2018-09-13) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-11.3 required=4.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PGPSIGNATURE,RCVD_IN_DNSWL_LOW, + SORTED_RECIPS,SPF_HELO_PASS,SPF_PASS,SUSPICIOUS_RECIPS,TXREP + autolearn=unavailable autolearn_force=no + version=3.4.2-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 0; hammy, 115; neutral, 22; spammy, 1. + spammytokens:0.902-+--emails + hammytokens:0.000-+--HX-ME-Sender:xms, + 0.000-+--H*RU:10.202.2.43, + 0.000-+--Hx-spam-relays-external:10.202.2.43, 0.000-+--H*F:U*spwhitton, + 0.000-+--H*F:D*spwhitton.name +Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) (envelope-from ) + id 1ga30m-00059C-Kv; Thu, 20 Dec 2018 18:25:32 +0000 +Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) + by mailout.west.internal (Postfix) with ESMTP id C50B712F6; + Thu, 20 Dec 2018 13:25:30 -0500 (EST) +Received: from mailfrontend1 ([10.202.2.162]) + by compute3.internal (MEProxy); Thu, 20 Dec 2018 13:25:31 -0500 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name; + h=from:to:subject:date:message-id:mime-version:content-type; s= + fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lqnNZlsr/mb10zqE=; b=X9S4kI14 + zxdg9IKMIyALL2Eao3GTyyojuATJkgCqJvZWd8RWpty6RBStZfyeOqR1L3Gr5m3/ + EVeiHQyFNnPor2xjMDmblfPS/u09JxVlc0KMpT0XXRfNWsVQn+U40nNRX15kXzZ/ + D1rYhxpxzKRzU2tByUULCgbGlXAwJQtOXMDw3mpj1BxcoO13H/0H/KQTQ+AcpiOw + BV3JFKL/jA+mH8uAPIgNM2mUYZz5REO89eh3lPhLyc7tw745X+4ywZlo/Piqa0+6 + BCldY9/nDR3csAUKx1+3hkpJPdqFALBWvG3SelGt44BqcoLsOJLB8QH6trCro39o + fEnBaUBTAkTAHA== +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lq + nNZlsr/mb10zqE=; b=QYqLjUBZtfbO0DUlagvXRP2BPnpTgkvIna9/uCewkdoIH + /LuPW6DZUNWQa55qiDquqKXcs0tTODTEzYgeIDgqC+DDBTHnFQvXdWyS1X3o4sLL + 8dTKk8lv7M1/zKFxyg/ycNvPJGS9m4ZucGbxjwdgAcozhg7W1Qztxt9eVhPVnenS + 5sdeJ9mjIE7lYkKX4QVsXPOi86j6QlfMNyi/OnBfX2+95QiA/xPE/wEq4MYlLNm7 + Av1P/8OrI4ImDKkOEivarktL+isYL7OXyGB4GfUTsydiy9dhP7RKPxrai1kJRu5S + b2470KXNatu2WkyMFrsdcwrSqyKIe096k5xPfVI2A== +X-ME-Sender: +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudejfedguddugecutefuodetggdotefrod + ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu + fedttdenucfuohhrthgvugcurhgvtghiphhsucdlgedtmdenucfjughrpefhvffufffkgg + gtsehgtderredttddtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhi + thhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqnecurfgrrhgrmhepmhgrihhlfhhroh + hmpehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgenucevlhhushhtvghr + ufhiiigvpedt +X-ME-Proxy: + + + +From: Sean Whitton +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, + 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, + 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, + 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 +Message-ID: <87r2ecrr6x.fsf@zephyr.silentflame.com> +MIME-Version: 1.0 +X-CrossAssassin-Score: 3 +Received-SPF: pass client-ip=2607:f8f0:614:1::1274:39; + envelope-from=debbugs@buxtehude.debian.org; helo=buxtehude.debian.org +x-debian-approved: yes +Subject: [Pkg-emacsen-addons] Bug#916808: Increase severity to 'serious' +X-BeenThere: pkg-emacsen-addons@alioth-lists.debian.net +X-Mailman-Version: 2.1.23 +Precedence: list +List-Id: Maintainers list for Emacs addon packages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Reply-To: Sean Whitton , 916808@bugs.debian.org +Content-Type: multipart/mixed; boundary="===============8231894308137086149==" +Errors-To: pkg-emacsen-addons-bounces+bremner=debian.org@alioth-lists.debian.net +Sender: "Pkg-emacsen-addons" + +X-Spam_score: 2.8 +X-Spam_score_int: 28 +X-Spam_bar: ++ + +--===============8231894308137086149== +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha512; protocol="application/pgp-signature" + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +=2D-=20 +Sean Whitton + +--=-=-= +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlwb3pYACgkQaVt65L8G +YkDLrhAAhORxZzZDE5vlXRm89JYA3jd9OyleioZvDDRCrpEd7CQ5AHiMMJizW1lU +gn6OBIoW4O04TZ5oOuUnDHK/rS0G4zgNCJUyNf06zVECmdvkzspNNpQ3J5aOi4t2 +lhjRIFOKA9ifGsEqYLwP2dork1xFuyHEqHkDH8zpCTvdzkWky1bwAD/Pj5dArd7t +FeQGsPm7/64H1/rHk8pSP2pQgRsMDX6rIdx3vuQ7r+NssdRq+II4e479l02TiCDi +FBOX+n3nPXxREPdZ9EKL4SauL/AnRqpeC9GX6fC9OOnQeQ1xVTzNWKa6ixrqkFoH +TI/vy51p16jFNgdkLkyLtZA8Tq72TIAKWbZC0GFzWJVNASWu7WDIoMn5pgoi454w +TgsvK9MOnEYeABiDUa1ppaoMiP4+3j5yT0eWttTMSkcKjk1Ap1o+RfUxlIGl0Rog +ShbG2y6Mv8FERtjzPVQ7VMLDN9zRIbtlSJFm7CboPNSAygzzzaA/RIN/e8MdbZoM +a8AT9KiAVHEEcw+nWFAatAew5VP9iRZVgrVdWBszuaWOolxnYvpAL45WanqG0eab +VMe66+rZ8momI0MsM9JcqBwXO+fOf8CrPSO9PL8VFEJXFLZQS7asFStJf2l8msWE +3IYhvk4B6Nf1R96XzpXLlkOnoGtcnPVAvotrGU/rDfk5i/WF810= +=mWfF +-----END PGP SIGNATURE----- +--=-=-=-- + + +--===============8231894308137086149== +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Disposition: inline + +X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KUGtnLWVtYWNz +ZW4tYWRkb25zIG1haWxpbmcgbGlzdApQa2ctZW1hY3Nlbi1hZGRvbnNAYWxpb3RoLWxpc3RzLmRl +Ymlhbi5uZXQKaHR0cHM6Ly9hbGlvdGgtbGlzdHMuZGViaWFuLm5ldC9jZ2ktYmluL21haWxtYW4v +bGlzdGluZm8vcGtnLWVtYWNzZW4tYWRkb25zCg== + +--===============8231894308137086149==-- + diff --git a/test/corpora/duplicate/msg-3-3:2, b/test/corpora/duplicate/msg-3-3:2, new file mode 100644 index 00000000..bff2473c --- /dev/null +++ b/test/corpora/duplicate/msg-3-3:2, @@ -0,0 +1,178 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 20 Dec 2018 13:27:16 -0500 +Received: from muffat.debian.org ([2607:f8f0:614:1::1274:33]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32S-0005aw-5h + for david@tethera.net; Thu, 20 Dec 2018 13:27:16 -0500 +Received: from ticharich.debian.org ([2001:41c8:1000:21::21:23]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=ticharich.debian.org,EMAIL=hostmaster@ticharich.debian.org (verified) + by muffat.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32R-0008QQ-PL + for david@tethera.net; Thu, 20 Dec 2018 18:27:15 +0000 +Received: from localhost ([::1] helo=ticharich.debian.org) + by ticharich.debian.org with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32Q-0008BB-Ad + for david@tethera.net; Thu, 20 Dec 2018 18:27:14 +0000 +Received: from muffat.debian.org ([2607:f8f0:614:1::1274:33]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=muffat.debian.org,EMAIL=hostmaster@muffat.debian.org (verified) + by ticharich.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32Q-0008Ai-2N + for dispatch+haskell-mode@tracker.debian.org; Thu, 20 Dec 2018 18:27:14 +0000 +Received: from quantz.debian.org ([2001:41c8:1000:21::21:28]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=quantz.debian.org,EMAIL=hostmaster@quantz.debian.org (verified) + by muffat.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32O-0008QE-Eo + for dispatch+haskell-mode@tracker.debian.org; Thu, 20 Dec 2018 18:27:12 +0000 +Received: from qa by quantz.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32M-0006hb-Vn + for dispatch+haskell-mode@tracker.debian.org; Thu, 20 Dec 2018 18:27:10 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + from C=NA,ST=NA,L=Ankh Morpork,O=Debian SMTP,OU=Debian SMTP CA,CN=buxtehude.debian.org,EMAIL=hostmaster@buxtehude.debian.org (verified) + by quantz.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32J-0006gG-1x + for haskell-mode@packages.qa.debian.org; Thu, 20 Dec 2018 18:27:07 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32H-0005SF-MF; Thu, 20 Dec 2018 18:27:05 +0000 +X-Loop: owner@bugs.debian.org +Subject: Bug#916807: Increase severity to 'serious' +Reply-To: Sean Whitton , 916807@bugs.debian.org +Resent-From: Sean Whitton +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Emacs addons team +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 20 Dec 2018 18:27:04 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 916807 +X-Debian-PR-Package: src:haskell-mode +X-Debian-PR-Keywords: ftbfs +References: <87efaeh3zr.fsf@zephyr.silentflame.com> +X-Debian-PR-Source: haskell-mode +Received: via spool by 916807-submit@bugs.debian.org id=B916807.154533033319820 + (code B ref 916807); Thu, 20 Dec 2018 18:27:04 +0000 +Received: (at 916807) by bugs.debian.org; 20 Dec 2018 18:25:33 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 + (2018-09-13) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-11.3 required=4.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PGPSIGNATURE,RCVD_IN_DNSWL_LOW, + SORTED_RECIPS,SPF_HELO_PASS,SPF_PASS,SUSPICIOUS_RECIPS,TXREP + autolearn=unavailable autolearn_force=no + version=3.4.2-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 0; hammy, 115; neutral, 22; spammy, 1. + spammytokens:0.902-+--emails hammytokens:0.000-+--HX-ME-Sender:xms, + 0.000-+--H*RU:10.202.2.43, + 0.000-+--Hx-spam-relays-external:10.202.2.43, + 0.000-+--H*F:D*spwhitton.name, 0.000-+--H*F:U*spwhitton +Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga30m-00059C-Kv; Thu, 20 Dec 2018 18:25:32 +0000 +Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) + by mailout.west.internal (Postfix) with ESMTP id C50B712F6; + Thu, 20 Dec 2018 13:25:30 -0500 (EST) +Received: from mailfrontend1 ([10.202.2.162]) + by compute3.internal (MEProxy); Thu, 20 Dec 2018 13:25:31 -0500 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name; + h=from:to:subject:date:message-id:mime-version:content-type; s= + fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lqnNZlsr/mb10zqE=; b=X9S4kI14 + zxdg9IKMIyALL2Eao3GTyyojuATJkgCqJvZWd8RWpty6RBStZfyeOqR1L3Gr5m3/ + EVeiHQyFNnPor2xjMDmblfPS/u09JxVlc0KMpT0XXRfNWsVQn+U40nNRX15kXzZ/ + D1rYhxpxzKRzU2tByUULCgbGlXAwJQtOXMDw3mpj1BxcoO13H/0H/KQTQ+AcpiOw + BV3JFKL/jA+mH8uAPIgNM2mUYZz5REO89eh3lPhLyc7tw745X+4ywZlo/Piqa0+6 + BCldY9/nDR3csAUKx1+3hkpJPdqFALBWvG3SelGt44BqcoLsOJLB8QH6trCro39o + fEnBaUBTAkTAHA== +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lq + nNZlsr/mb10zqE=; b=QYqLjUBZtfbO0DUlagvXRP2BPnpTgkvIna9/uCewkdoIH + /LuPW6DZUNWQa55qiDquqKXcs0tTODTEzYgeIDgqC+DDBTHnFQvXdWyS1X3o4sLL + 8dTKk8lv7M1/zKFxyg/ycNvPJGS9m4ZucGbxjwdgAcozhg7W1Qztxt9eVhPVnenS + 5sdeJ9mjIE7lYkKX4QVsXPOi86j6QlfMNyi/OnBfX2+95QiA/xPE/wEq4MYlLNm7 + Av1P/8OrI4ImDKkOEivarktL+isYL7OXyGB4GfUTsydiy9dhP7RKPxrai1kJRu5S + b2470KXNatu2WkyMFrsdcwrSqyKIe096k5xPfVI2A== +X-ME-Sender: +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudejfedguddugecutefuodetggdotefrod + ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu + fedttdenucfuohhrthgvugcurhgvtghiphhsucdlgedtmdenucfjughrpefhvffufffkgg + gtsehgtderredttddtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhi + thhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqnecurfgrrhgrmhepmhgrihhlfhhroh + hmpehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgenucevlhhushhtvghr + ufhiiigvpedt +X-ME-Proxy: + + + +From: Sean Whitton +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 +Message-ID: <87r2ecrr6x.fsf@zephyr.silentflame.com> +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha512; protocol="application/pgp-signature" +X-CrossAssassin-Score: 2 +Delivered-To: haskell-mode@packages.qa.debian.org +Delivered-To: dispatch+haskell-mode@tracker.debian.org +X-Loop: dispatch@tracker.debian.org +X-Distro-Tracker-Keyword: bts +X-Distro-Tracker-Package: haskell-mode +List-Id: +X-Debian: tracker.debian.org +X-Debian-Package: haskell-mode +X-PTS-Package: haskell-mode +X-PTS-Keyword: bts +Precedence: list +List-Unsubscribe: +X-Spam_score: 2.8 +X-Spam_score_int: 28 +X-Spam_bar: ++ + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +=2D-=20 +Sean Whitton + +--=-=-= +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlwb3pYACgkQaVt65L8G +YkDLrhAAhORxZzZDE5vlXRm89JYA3jd9OyleioZvDDRCrpEd7CQ5AHiMMJizW1lU +gn6OBIoW4O04TZ5oOuUnDHK/rS0G4zgNCJUyNf06zVECmdvkzspNNpQ3J5aOi4t2 +lhjRIFOKA9ifGsEqYLwP2dork1xFuyHEqHkDH8zpCTvdzkWky1bwAD/Pj5dArd7t +FeQGsPm7/64H1/rHk8pSP2pQgRsMDX6rIdx3vuQ7r+NssdRq+II4e479l02TiCDi +FBOX+n3nPXxREPdZ9EKL4SauL/AnRqpeC9GX6fC9OOnQeQ1xVTzNWKa6ixrqkFoH +TI/vy51p16jFNgdkLkyLtZA8Tq72TIAKWbZC0GFzWJVNASWu7WDIoMn5pgoi454w +TgsvK9MOnEYeABiDUa1ppaoMiP4+3j5yT0eWttTMSkcKjk1Ap1o+RfUxlIGl0Rog +ShbG2y6Mv8FERtjzPVQ7VMLDN9zRIbtlSJFm7CboPNSAygzzzaA/RIN/e8MdbZoM +a8AT9KiAVHEEcw+nWFAatAew5VP9iRZVgrVdWBszuaWOolxnYvpAL45WanqG0eab +VMe66+rZ8momI0MsM9JcqBwXO+fOf8CrPSO9PL8VFEJXFLZQS7asFStJf2l8msWE +3IYhvk4B6Nf1R96XzpXLlkOnoGtcnPVAvotrGU/rDfk5i/WF810= +=mWfF +-----END PGP SIGNATURE----- +--=-=-=-- + diff --git a/test/corpora/duplicate/msg-3-4:2, b/test/corpora/duplicate/msg-3-4:2, new file mode 100644 index 00000000..861719d1 --- /dev/null +++ b/test/corpora/duplicate/msg-3-4:2, @@ -0,0 +1,184 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 20 Dec 2018 13:27:16 -0500 +Received: from mailly.debian.org ([2001:41b8:202:deb:6564:a62:52c3:4b72]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32S-0005ax-Kz + for david@tethera.net; Thu, 20 Dec 2018 13:27:16 -0500 +Received: from alioth-lists-01.debian.net ([2001:ba8:0:2c77:0:4:0:1]) + by mailly.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) + (envelope-from ) + id 1ga32S-0004W6-8i + for david@tethera.net; Thu, 20 Dec 2018 18:27:16 +0000 +Received: from localhost ([::1] helo=alioth-lists-01.debian.net) + by alioth-lists-01.debian.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32S-0003xQ-0U + for bremner@debian.org; Thu, 20 Dec 2018 18:27:16 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + by alioth-lists-01.debian.net with esmtps + (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) + (envelope-from ) id 1ga32Q-0003wj-1x + for pkg-emacsen-addons@lists.alioth.debian.org; Thu, 20 Dec 2018 18:27:14 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32O-0005Te-Sa; Thu, 20 Dec 2018 18:27:12 +0000 +X-Loop: owner@bugs.debian.org +Resent-From: Sean Whitton +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Emacs addons team + +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 20 Dec 2018 18:27:11 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 916811 +X-Debian-PR-Package: src:weechat-el +X-Debian-PR-Keywords: ftbfs +References: <8736quh3la.fsf@zephyr.silentflame.com> +X-Debian-PR-Source: weechat-el +Received: via spool by 916811-submit@bugs.debian.org id=B916811.154533033319851 + (code B ref 916811); Thu, 20 Dec 2018 18:27:11 +0000 +Received: (at 916811) by bugs.debian.org; 20 Dec 2018 18:25:33 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 + (2018-09-13) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-15.2 required=4.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PGPSIGNATURE,RCVD_IN_DNSWL_LOW, + SORTED_RECIPS,SPF_HELO_PASS,SPF_PASS,SUSPICIOUS_RECIPS,TXREP + autolearn=unavailable autolearn_force=no + version=3.4.2-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 0; hammy, 115; neutral, 22; spammy, 1. + spammytokens:0.902-+--emails + hammytokens:0.000-+--HX-ME-Sender:xms, + 0.000-+--H*RU:10.202.2.43, + 0.000-+--Hx-spam-relays-external:10.202.2.43, + 0.000-+--H*F:D*spwhitton.name, 0.000-+--H*F:U*spwhitton +Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) (envelope-from ) + id 1ga30m-00059C-Kv; Thu, 20 Dec 2018 18:25:32 +0000 +Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) + by mailout.west.internal (Postfix) with ESMTP id C50B712F6; + Thu, 20 Dec 2018 13:25:30 -0500 (EST) +Received: from mailfrontend1 ([10.202.2.162]) + by compute3.internal (MEProxy); Thu, 20 Dec 2018 13:25:31 -0500 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name; + h=from:to:subject:date:message-id:mime-version:content-type; s= + fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lqnNZlsr/mb10zqE=; b=X9S4kI14 + zxdg9IKMIyALL2Eao3GTyyojuATJkgCqJvZWd8RWpty6RBStZfyeOqR1L3Gr5m3/ + EVeiHQyFNnPor2xjMDmblfPS/u09JxVlc0KMpT0XXRfNWsVQn+U40nNRX15kXzZ/ + D1rYhxpxzKRzU2tByUULCgbGlXAwJQtOXMDw3mpj1BxcoO13H/0H/KQTQ+AcpiOw + BV3JFKL/jA+mH8uAPIgNM2mUYZz5REO89eh3lPhLyc7tw745X+4ywZlo/Piqa0+6 + BCldY9/nDR3csAUKx1+3hkpJPdqFALBWvG3SelGt44BqcoLsOJLB8QH6trCro39o + fEnBaUBTAkTAHA== +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lq + nNZlsr/mb10zqE=; b=QYqLjUBZtfbO0DUlagvXRP2BPnpTgkvIna9/uCewkdoIH + /LuPW6DZUNWQa55qiDquqKXcs0tTODTEzYgeIDgqC+DDBTHnFQvXdWyS1X3o4sLL + 8dTKk8lv7M1/zKFxyg/ycNvPJGS9m4ZucGbxjwdgAcozhg7W1Qztxt9eVhPVnenS + 5sdeJ9mjIE7lYkKX4QVsXPOi86j6QlfMNyi/OnBfX2+95QiA/xPE/wEq4MYlLNm7 + Av1P/8OrI4ImDKkOEivarktL+isYL7OXyGB4GfUTsydiy9dhP7RKPxrai1kJRu5S + b2470KXNatu2WkyMFrsdcwrSqyKIe096k5xPfVI2A== +X-ME-Sender: +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudejfedguddugecutefuodetggdotefrod + ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu + fedttdenucfuohhrthgvugcurhgvtghiphhsucdlgedtmdenucfjughrpefhvffufffkgg + gtsehgtderredttddtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhi + thhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqnecurfgrrhgrmhepmhgrihhlfhhroh + hmpehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgenucevlhhushhtvghr + ufhiiigvpedt +X-ME-Proxy: + + + +From: Sean Whitton +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, + 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, + 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, + 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 +Message-ID: <87r2ecrr6x.fsf@zephyr.silentflame.com> +MIME-Version: 1.0 +X-CrossAssassin-Score: 5 +Received-SPF: pass client-ip=2607:f8f0:614:1::1274:39; + envelope-from=debbugs@buxtehude.debian.org; helo=buxtehude.debian.org +x-debian-approved: yes +Subject: [Pkg-emacsen-addons] Bug#916811: Increase severity to 'serious' +X-BeenThere: pkg-emacsen-addons@alioth-lists.debian.net +X-Mailman-Version: 2.1.23 +Precedence: list +List-Id: Maintainers list for Emacs addon packages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Reply-To: Sean Whitton , 916811@bugs.debian.org +Content-Type: multipart/mixed; boundary="===============5359377007738902760==" +Errors-To: pkg-emacsen-addons-bounces+bremner=debian.org@alioth-lists.debian.net +Sender: "Pkg-emacsen-addons" + +X-Spam_score: 2.8 +X-Spam_score_int: 28 +X-Spam_bar: ++ + +--===============5359377007738902760== +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha512; protocol="application/pgp-signature" + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +=2D-=20 +Sean Whitton + +--=-=-= +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlwb3pYACgkQaVt65L8G +YkDLrhAAhORxZzZDE5vlXRm89JYA3jd9OyleioZvDDRCrpEd7CQ5AHiMMJizW1lU +gn6OBIoW4O04TZ5oOuUnDHK/rS0G4zgNCJUyNf06zVECmdvkzspNNpQ3J5aOi4t2 +lhjRIFOKA9ifGsEqYLwP2dork1xFuyHEqHkDH8zpCTvdzkWky1bwAD/Pj5dArd7t +FeQGsPm7/64H1/rHk8pSP2pQgRsMDX6rIdx3vuQ7r+NssdRq+II4e479l02TiCDi +FBOX+n3nPXxREPdZ9EKL4SauL/AnRqpeC9GX6fC9OOnQeQ1xVTzNWKa6ixrqkFoH +TI/vy51p16jFNgdkLkyLtZA8Tq72TIAKWbZC0GFzWJVNASWu7WDIoMn5pgoi454w +TgsvK9MOnEYeABiDUa1ppaoMiP4+3j5yT0eWttTMSkcKjk1Ap1o+RfUxlIGl0Rog +ShbG2y6Mv8FERtjzPVQ7VMLDN9zRIbtlSJFm7CboPNSAygzzzaA/RIN/e8MdbZoM +a8AT9KiAVHEEcw+nWFAatAew5VP9iRZVgrVdWBszuaWOolxnYvpAL45WanqG0eab +VMe66+rZ8momI0MsM9JcqBwXO+fOf8CrPSO9PL8VFEJXFLZQS7asFStJf2l8msWE +3IYhvk4B6Nf1R96XzpXLlkOnoGtcnPVAvotrGU/rDfk5i/WF810= +=mWfF +-----END PGP SIGNATURE----- +--=-=-=-- + + +--===============5359377007738902760== +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Disposition: inline + +X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KUGtnLWVtYWNz +ZW4tYWRkb25zIG1haWxpbmcgbGlzdApQa2ctZW1hY3Nlbi1hZGRvbnNAYWxpb3RoLWxpc3RzLmRl +Ymlhbi5uZXQKaHR0cHM6Ly9hbGlvdGgtbGlzdHMuZGViaWFuLm5ldC9jZ2ktYmluL21haWxtYW4v +bGlzdGluZm8vcGtnLWVtYWNzZW4tYWRkb25zCg== + +--===============5359377007738902760==-- + diff --git a/test/corpora/duplicate/msg-3-5:2, b/test/corpora/duplicate/msg-3-5:2, new file mode 100644 index 00000000..af26374d --- /dev/null +++ b/test/corpora/duplicate/msg-3-5:2, @@ -0,0 +1,179 @@ +Return-path: +Envelope-to: david@tethera.net +Delivery-date: Thu, 20 Dec 2018 13:27:21 -0500 +Received: from alioth-lists-01.debian.net ([2001:ba8:0:2c77:0:4:0:1]) + by fethera.tethera.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32X-0005bA-Go + for david@tethera.net; Thu, 20 Dec 2018 13:27:21 -0500 +Received: from localhost ([::1] helo=alioth-lists-01.debian.net) + by alioth-lists-01.debian.net with esmtp (Exim 4.89) + (envelope-from ) + id 1ga32U-0003yc-Ai + for david@tethera.net; Thu, 20 Dec 2018 18:27:18 +0000 +Received: from buxtehude.debian.org ([2607:f8f0:614:1::1274:39]) + by alioth-lists-01.debian.net with esmtps + (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) + (envelope-from ) id 1ga32S-0003xN-6J + for debian-science-maintainers@lists.alioth.debian.org; + Thu, 20 Dec 2018 18:27:16 +0000 +Received: from debbugs by buxtehude.debian.org with local (Exim 4.89) + (envelope-from ) + id 1ga32R-0005U9-0Z; Thu, 20 Dec 2018 18:27:15 +0000 +X-Loop: owner@bugs.debian.org +Subject: Bug#916867: Increase severity to 'serious' +Resent-From: Sean Whitton +Resent-To: debian-bugs-dist@lists.debian.org +Resent-CC: Debian Science Maintainers + +X-Loop: owner@bugs.debian.org +Resent-Date: Thu, 20 Dec 2018 18:27:13 +0000 +Resent-Message-ID: +X-Debian-PR-Message: followup 916867 +X-Debian-PR-Package: src:hkl +X-Debian-PR-Keywords: ftbfs +References: <87sgyt2xyg.fsf@zephyr.silentflame.com> +X-Debian-PR-Source: hkl +Received: via spool by 916867-submit@bugs.debian.org id=B916867.154533033319861 + (code B ref 916867); Thu, 20 Dec 2018 18:27:13 +0000 +Received: (at 916867) by bugs.debian.org; 20 Dec 2018 18:25:33 +0000 +X-Spam-Checker-Version: SpamAssassin 3.4.2-bugs.debian.org_2005_01_02 + (2018-09-13) on buxtehude.debian.org +X-Spam-Level: +X-Spam-Status: No, score=-13.0 required=4.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,PGPSIGNATURE,RCVD_IN_DNSWL_LOW, + SORTED_RECIPS,SPF_HELO_PASS,SPF_PASS,SUSPICIOUS_RECIPS,TXREP + autolearn=unavailable autolearn_force=no + version=3.4.2-bugs.debian.org_2005_01_02 +X-Spam-Bayes: score:0.0000 Tokens: new, 0; hammy, 115; neutral, 22; spammy, 1. + spammytokens:0.902-+--emails + hammytokens:0.000-+--HX-ME-Sender:xms, + 0.000-+--H*RU:10.202.2.43, + 0.000-+--Hx-spam-relays-external:10.202.2.43, + 0.000-+--H*F:D*spwhitton.name, 0.000-+--H*F:U*spwhitton +Received: from wout2-smtp.messagingengine.com ([64.147.123.25]) + by buxtehude.debian.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) + (Exim 4.89) (envelope-from ) + id 1ga30m-00059C-Kv; Thu, 20 Dec 2018 18:25:32 +0000 +Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) + by mailout.west.internal (Postfix) with ESMTP id C50B712F6; + Thu, 20 Dec 2018 13:25:30 -0500 (EST) +Received: from mailfrontend1 ([10.202.2.162]) + by compute3.internal (MEProxy); Thu, 20 Dec 2018 13:25:31 -0500 +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=spwhitton.name; + h=from:to:subject:date:message-id:mime-version:content-type; s= + fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lqnNZlsr/mb10zqE=; b=X9S4kI14 + zxdg9IKMIyALL2Eao3GTyyojuATJkgCqJvZWd8RWpty6RBStZfyeOqR1L3Gr5m3/ + EVeiHQyFNnPor2xjMDmblfPS/u09JxVlc0KMpT0XXRfNWsVQn+U40nNRX15kXzZ/ + D1rYhxpxzKRzU2tByUULCgbGlXAwJQtOXMDw3mpj1BxcoO13H/0H/KQTQ+AcpiOw + BV3JFKL/jA+mH8uAPIgNM2mUYZz5REO89eh3lPhLyc7tw745X+4ywZlo/Piqa0+6 + BCldY9/nDR3csAUKx1+3hkpJPdqFALBWvG3SelGt44BqcoLsOJLB8QH6trCro39o + fEnBaUBTAkTAHA== +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= + messagingengine.com; h=content-type:date:from:message-id + :mime-version:subject:to:x-me-proxy:x-me-proxy:x-me-sender + :x-me-sender:x-sasl-enc; s=fm1; bh=jAd0CK1DR6lBA7NSLH5FsHIYQv3lq + nNZlsr/mb10zqE=; b=QYqLjUBZtfbO0DUlagvXRP2BPnpTgkvIna9/uCewkdoIH + /LuPW6DZUNWQa55qiDquqKXcs0tTODTEzYgeIDgqC+DDBTHnFQvXdWyS1X3o4sLL + 8dTKk8lv7M1/zKFxyg/ycNvPJGS9m4ZucGbxjwdgAcozhg7W1Qztxt9eVhPVnenS + 5sdeJ9mjIE7lYkKX4QVsXPOi86j6QlfMNyi/OnBfX2+95QiA/xPE/wEq4MYlLNm7 + Av1P/8OrI4ImDKkOEivarktL+isYL7OXyGB4GfUTsydiy9dhP7RKPxrai1kJRu5S + b2470KXNatu2WkyMFrsdcwrSqyKIe096k5xPfVI2A== +X-ME-Sender: +X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedtkedrudejfedguddugecutefuodetggdotefrod + ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfhuthenuceurghilhhouhhtmecu + fedttdenucfuohhrthgvugcurhgvtghiphhsucdlgedtmdenucfjughrpefhvffufffkgg + gtsehgtderredttddtnecuhfhrohhmpefuvggrnhcuhghhihhtthhonhcuoehsphifhhhi + thhtohhnsehsphifhhhithhtohhnrdhnrghmvgeqnecurfgrrhgrmhepmhgrihhlfhhroh + hmpehsphifhhhithhtohhnsehsphifhhhithhtohhnrdhnrghmvgenucevlhhushhtvghr + ufhiiigvpedt +X-ME-Proxy: + + + +From: Sean Whitton +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, + 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, + 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, + 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 +Message-ID: <87r2ecrr6x.fsf@zephyr.silentflame.com> +MIME-Version: 1.0 +X-CrossAssassin-Score: 6 +Received-SPF: pass client-ip=2607:f8f0:614:1::1274:39; + envelope-from=debbugs@buxtehude.debian.org; helo=buxtehude.debian.org +x-debian-approved: yes +X-BeenThere: debian-science-maintainers@alioth-lists.debian.net +X-Mailman-Version: 2.1.23 +Precedence: list +List-Id: Mailing list for maintainer discussions and BTS messages + +List-Unsubscribe: , + +List-Archive: +List-Post: +List-Help: +List-Subscribe: , + +Reply-To: Sean Whitton , 916867@bugs.debian.org +Content-Type: multipart/mixed; boundary="===============7686561040995282884==" +Errors-To: debian-science-maintainers-bounces+david=tethera.net@alioth-lists.debian.net +Sender: "debian-science-maintainers" + +X-Spam_score: 2.8 +X-Spam_score_int: 28 +X-Spam_bar: ++ + +--===============7686561040995282884== +Content-Type: multipart/signed; boundary="=-=-="; + micalg=pgp-sha512; protocol="application/pgp-signature" + +--=-=-= +Content-Type: text/plain +Content-Transfer-Encoding: quoted-printable + +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +=2D-=20 +Sean Whitton + +--=-=-= +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCgAdFiEEm5FwB64DDjbk/CSLaVt65L8GYkAFAlwb3pYACgkQaVt65L8G +YkDLrhAAhORxZzZDE5vlXRm89JYA3jd9OyleioZvDDRCrpEd7CQ5AHiMMJizW1lU +gn6OBIoW4O04TZ5oOuUnDHK/rS0G4zgNCJUyNf06zVECmdvkzspNNpQ3J5aOi4t2 +lhjRIFOKA9ifGsEqYLwP2dork1xFuyHEqHkDH8zpCTvdzkWky1bwAD/Pj5dArd7t +FeQGsPm7/64H1/rHk8pSP2pQgRsMDX6rIdx3vuQ7r+NssdRq+II4e479l02TiCDi +FBOX+n3nPXxREPdZ9EKL4SauL/AnRqpeC9GX6fC9OOnQeQ1xVTzNWKa6ixrqkFoH +TI/vy51p16jFNgdkLkyLtZA8Tq72TIAKWbZC0GFzWJVNASWu7WDIoMn5pgoi454w +TgsvK9MOnEYeABiDUa1ppaoMiP4+3j5yT0eWttTMSkcKjk1Ap1o+RfUxlIGl0Rog +ShbG2y6Mv8FERtjzPVQ7VMLDN9zRIbtlSJFm7CboPNSAygzzzaA/RIN/e8MdbZoM +a8AT9KiAVHEEcw+nWFAatAew5VP9iRZVgrVdWBszuaWOolxnYvpAL45WanqG0eab +VMe66+rZ8momI0MsM9JcqBwXO+fOf8CrPSO9PL8VFEJXFLZQS7asFStJf2l8msWE +3IYhvk4B6Nf1R96XzpXLlkOnoGtcnPVAvotrGU/rDfk5i/WF810= +=mWfF +-----END PGP SIGNATURE----- +--=-=-=-- + + +--===============7686561040995282884== +Content-Type: text/plain; charset="utf-8" +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Disposition: inline + +LS0gCmRlYmlhbi1zY2llbmNlLW1haW50YWluZXJzIG1haWxpbmcgbGlzdApkZWJpYW4tc2NpZW5j +ZS1tYWludGFpbmVyc0BhbGlvdGgtbGlzdHMuZGViaWFuLm5ldApodHRwczovL2FsaW90aC1saXN0 +cy5kZWJpYW4ubmV0L2NnaS1iaW4vbWFpbG1hbi9saXN0aW5mby9kZWJpYW4tc2NpZW5jZS1tYWlu +dGFpbmVycw== + +--===============7686561040995282884==-- + diff --git a/test/corpora/indexing/PATCH-1-2-system_data_types.7-srcfix.txt:2,S b/test/corpora/indexing/PATCH-1-2-system_data_types.7-srcfix.txt:2,S new file mode 100644 index 00000000..1361c6f2 --- /dev/null +++ b/test/corpora/indexing/PATCH-1-2-system_data_types.7-srcfix.txt:2,S @@ -0,0 +1,282 @@ +From mboxrd@z Thu Jan 1 00:00:00 1970 +Return-Path: +X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on + aws-us-west-2-korg-lkml-1.web.codeaurora.org +X-Spam-Level: +X-Spam-Status: No, score=-8.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, + DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, + HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE, + SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no + version=3.4.0 +Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) + by smtp.lore.kernel.org (Postfix) with ESMTP id AFE3FC4727E + for ; Wed, 30 Sep 2020 10:12:21 +0000 (UTC) +Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) + by mail.kernel.org (Postfix) with ESMTP id 4E0D62074A + for ; Wed, 30 Sep 2020 10:12:21 +0000 (UTC) +Authentication-Results: mail.kernel.org; + dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Osm9Pn67" +Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand + id S1725823AbgI3KMU (ORCPT ); + Wed, 30 Sep 2020 06:12:20 -0400 +Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50038 "EHLO + lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org + with ESMTP id S1725779AbgI3KMU (ORCPT + ); Wed, 30 Sep 2020 06:12:20 -0400 +Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) + by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5026DC061755 + for ; Wed, 30 Sep 2020 03:12:20 -0700 (PDT) +Received: by mail-pf1-x443.google.com with SMTP id b124so832681pfg.13 + for ; Wed, 30 Sep 2020 03:12:20 -0700 (PDT) +DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=gmail.com; s=20161025; + h=date:from:to:cc:subject:message-id:references:mime-version + :content-disposition:in-reply-to:user-agent; + bh=qR1FJVXOhU6/g+m4SoSco3vMtV+CNvRvNyXS1xuG+T4=; + b=Osm9Pn67G380QiA1ORltntJShSHlKg/KZZfKV8ebvfEXJw9893EO0N6J6GDR+zkmHi + TOQuIe7x9y95Pipm54rWWEW33U3gwoXRHsPc2Kivm6L8Ixb+f0T0rMPKw/FOkL8OGo9t + WmmSvnlErAXHqBq9aRAJJsf2bSlDgdAyYY1Qe6PSq2hKi2rg+sOy1Vaj4RqZ6jTK/DWY + tX28Ql0XS3kKWp0Lc8MNsSP+SXlcdwHQYll5LeReAg1oi++hICgWphuMmo3OH+2B1WtO + hMH7VuUONqbuE1aLoZ6PyyUlCeN1soJd8bKY0cmY0TKCsw0Jvkuh/XzYDVNi6wOSM6Ez + okpA== +X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; + d=1e100.net; s=20161025; + h=x-gm-message-state:date:from:to:cc:subject:message-id:references + :mime-version:content-disposition:in-reply-to:user-agent; + bh=qR1FJVXOhU6/g+m4SoSco3vMtV+CNvRvNyXS1xuG+T4=; + b=TJU+duGLhruSES/5sJy4y1wfcltfokDpA58edkSUJyasvsooUo67VNtOB3ZK49iHm5 + C/cjy0ExxTECB0aM6p+B1jcePdWoPUaVBY9bVd/Q5DNhm4KhTO8ON96gB43d2rLWLOiK + /Y1vCu+MwOpY0JQTojbC140s/JYccR/KPapTmbUkRzrpmeoYqw8CbBPV60rQxYCn9GUu + FeCXJY5q9OfaYW1viQZoBL5n1IMMpJDVa61Q8gZ33b3wRCvQv/x1eZCsVlYpjcqf7Umc + /Amx3i27cxvo8pSvvwiTzrlJHJv0Gkytz13i7s+zW+XKzZRyzy3yirtU2DFTGat6FeMn + H8Ig== +X-Gm-Message-State: AOAM530Yon7xNOW6kiuy6bVpbpwbzR/9pldRB49OtZaSAHAZg7Gyf7qE + JXgAH20rZzYlwqOZyeZCeAwtWh09PeI= +X-Google-Smtp-Source: ABdhPJxzyZAVDBtMwQ5+dUqVg37y/LgZByrSaTxvhS6wnx6sJuG8ROItw0CwDAg939XUVADeje/nZQ== +X-Received: by 2002:a63:c547:: with SMTP id g7mr1563654pgd.234.1601460739764; + Wed, 30 Sep 2020 03:12:19 -0700 (PDT) +Received: from localhost.localdomain ([1.129.172.177]) + by smtp.gmail.com with ESMTPSA id k14sm1804437pjd.45.2020.09.30.03.12.17 + (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); + Wed, 30 Sep 2020 03:12:19 -0700 (PDT) +Date: Wed, 30 Sep 2020 20:12:15 +1000 +From: "G. Branden Robinson" +To: "Michael Kerrisk (man-pages)" +Cc: Jakub Wilk , linux-man@vger.kernel.org +Subject: Re: [PATCH 1/2] system_data_types.7: srcfix +Message-ID: <20200930101213.2m2pt3jrspvcrxfx@localhost.localdomain> +References: <20200925080330.184303-1-colomar.6.4.3@gmail.com> + <20200927061015.4obt73pdhyh7wecu@localhost.localdomain> + <20200928132959.x4koforqnzohxh5u@jwilk.net> + <9b8303fe-969e-c9f0-e3cd-0590b342d5bf@gmail.com> +MIME-Version: 1.0 +Content-Type: multipart/signed; micalg=pgp-sha256; + protocol="application/pgp-signature"; boundary="jg2hlfugxpumieke" +Content-Disposition: inline +In-Reply-To: <9b8303fe-969e-c9f0-e3cd-0590b342d5bf@gmail.com> +User-Agent: NeoMutt/20180716 +Precedence: bulk +List-ID: +X-Mailing-List: linux-man@vger.kernel.org + + +--jg2hlfugxpumieke +Content-Type: multipart/mixed; boundary="wl6i3r6gpq7ibouc" +Content-Disposition: inline + + +--wl6i3r6gpq7ibouc +Content-Type: text/plain; charset=us-ascii +Content-Disposition: inline +Content-Transfer-Encoding: quoted-printable + +Hi Jakub and Michael, + +At 2020-09-29T14:13:26+0200, Michael Kerrisk (man-pages) wrote: +> On 9/28/20 3:29 PM, Jakub Wilk wrote: +> > Hi Branden! +> >=20 +> > In groff_man_style(7) you wrote: +> >> Unused macro arguments are more often simply omitted, or good style +> >> suggests that a more appropriate macro be chosen, that earlier +> >> arguments are more important than later ones, or that arguments +> >> have identical significance such that skipping any is superfluous. +> >=20 +> > After 15 minutes of gawking at this sentence, I still don't +> > understand what are you trying to say here. The sentence should be +> > either thoroughly rephrased or removed. +>=20 +> I must say that I too found it hard to parse. I presume, Branden, +> that you mean: +>=20 +> [[ +> Unused macro arguments are more often simply omitted, or good style=20 +> suggests +> EITHER (1)=20 +> that a more appropriate macro be chosen,=20 +> (2) +> that earlier arguments are more important than later ones, or +> (3) +> that arguments have=20 +> identical significance such that skipping any is superfluous. +> ]] + +You got it. But it was too much work. + +> But it takes a few scans to work that out. Perhaps break this into +> smaller pieces, or add some explicit structuring elements to the +> sentence? + +I was trying to be comprehensive with respect to several anti-patterns I +had in mind. However, using the anti-patterns concretely is premature +at that point in the page. So I both expanded and relocated the +material. + +I'm attaching what I've just committed to groff git. + +Further feedback is welcome, of course; revision of documentation is a +process that is never completed, only abandoned. And I haven't given up +yet. :) + +Thank you both for your reviews. + +Regards, +Branden + +--wl6i3r6gpq7ibouc +Content-Type: text/x-diff; charset=us-ascii +Content-Disposition: attachment; filename="excise_standardese.diff" +Content-Transfer-Encoding: quoted-printable + +commit dd2c4cf05a659ae7127e342924668ff0fa0deaa1 +Author: G. Branden Robinson +Date: Wed Sep 30 19:56:38 2020 +1000 + + groff_man_style(7): Clarify empty macro arguments. + =20 + Rewrite some ersatz standardese I had managed to concoct regarding why + empty macro arguments are usually not needed. Put an expanded + discussion, with anti-patterns and remedies, in section "Notes", with + forward reference from subsection "Macro reference preliminaries". + =20 + Thanks to Jakub Wilk and Michael Kerrisk for the critique. + +diff --git a/tmac/groff_man.7.man.in b/tmac/groff_man.7.man.in +index c62d97ba..b96cbaf4 100644 +--- a/tmac/groff_man.7.man.in ++++ b/tmac/groff_man.7.man.in +@@ -281,23 +281,8 @@ but the + package is designed such that this should seldom be necessary. + _ifstyle()dnl + . +-Unused macro arguments are more often simply omitted, +-.\" antipattern: '.TP ""' (just '.TP' will do) +-or good style suggests that a more appropriate macro be chosen, +-.\" antipattern: '.BI "" italic bold' (use '.IB' instead) +-that earlier arguments are more important than later ones, +-.\" antipattern: '.TH foo 1 "" "foo "1.2.3"' (don't skip the date!) +-.\" antipattern: '.IP "" 4n' (use .TP or .RS/.RE, depending on needs) +-or that arguments have identical significance such that skipping any is +-superfluous. +-.\" antipattern: '.B one two "" three' (pointless) +-.\" Technically, the above has a side-effect of additional space +-.\" between "two" and "three", but there are much more obvious ways of +-.\" getting it if desired. +-.\" .B "one two three" +-.\" .B one "two " three +-.\" .B one two " three" +-.\" .B one two\~ three ++See section \(lqNotes\(rq below for examples of cases where better ++alternatives to empty arguments in macro calls are available. + _endif()dnl + . + Most macro arguments are strings that will be output as text; +@@ -3235,6 +3220,63 @@ Some tips on troubleshooting your man pages follow. + . + . + .TP ++\(bu Do I ever need to use an empty macro argument ("")? ++Probably not. ++. ++When this seems necessary, ++often a shorter or clearer alternative is available. ++. ++.\" antipattern: '.TP ""' (just '.TP' will do) ++.\" antipattern: '.BI "" italic bold' (use '.IB' instead) ++.\" antipattern: '.TH foo 1 "" "foo 1.2.3"' (don't skip the date!) ++.\" antipattern: '.IP "" 4n' (use .TP or .RS/.RE, depending on needs) ++.\" antipattern: '.B one two "" three' (pointless) ++.\" Technically, the above has a side-effect of additional space ++.\" between "two" and "three", but there are much more obvious ways of ++.\" getting it if desired. ++.\" .B "one two three" ++.\" .B one "two " three ++.\" .B one two " three" ++.\" .B one two\~ three ++.TS ++c c ++lfCB lfCB. ++Instead of.\|.\. .\|.\|.do this. ++_ ++\&.TP \(dq\(dq .TP ++\&.BI \(dq\(dq italic-text bold-text .IB italic-text bold-text ++\&.TH foo 1 \(dq\(dq \(dqfoo 1.2.3\(dq .TH foo 1 \ ++\f(CIyyyy\fP-\f(CImm\fP-\f(CIdd\fP \(dqfoo 1.2.3\(dq ++\&.IP \(dq\(dq 4n .TP 4n ++\&.B one two \(dq\(dq three .B one two three ++.TE ++. ++. ++.IP ++In the title heading ++.RB ( .TH ), ++the date of the page's last revision is more important than packaging ++information; ++it should not be omitted. ++. ++Ideally, ++a page maintainer will keep both up to date. ++. ++. ++.IP ++In the last example, ++the empty argument does have a subtly different effect than its ++suggested replacement; ++the empty argument becomes an additional space character\(embut it is a ++regular breaking space, ++so it can be discarded at the end of an output line. ++. ++It is better not to be subtle, ++particularly with space, ++which can be overlooked in source and rendered forms. ++. ++. ++.TP + .RB \(bu " .RS" " doesn't indent relative to my indented paragraph" + The + .B .RS + +--wl6i3r6gpq7ibouc-- + +--jg2hlfugxpumieke +Content-Type: application/pgp-signature; name="signature.asc" + +-----BEGIN PGP SIGNATURE----- + +iQIzBAEBCAAdFiEEh3PWHWjjDgcrENwa0Z6cfXEmbc4FAl90WfUACgkQ0Z6cfXEm +bc5raQ/9GhXG/5U7McaEEu+aW1IgaTYTMbsMpew5u3tBlj3/IenGzsy8wDO912BD +aHPSedYoc485k1Vh/Kowyx569RhyIXiMtH7uINCEtheMSUNgITNFqXo8mhaqVMlU +3JoV12btQddOIqHnGX6c5V9Z38KXFmVctD6CxjLaWGLp/Bu9tSKwSaHOOmtUYyOv +fYpMzr0amd4z9f+O8PPnToqBhwUitEvis1ZHYU6gIj8VwOjD0gNsWjA9HR3uC3c9 +GK/R5przMANrNejzSgofm0/yAL6a61WhqhYEtzLUYu2NFnsyNJWzITNsNnoxzgQ5 +liKL0Onmw0YWjOo4Z9Zht9Iyd6JhJxW0uRwlpFhE6UlCkFHK8nbv3NbHT2xlx/po +rxY5jDC3Ap3+mdYHY8k5o8vFd4QOXc2bSTuDRZoWtFZQsjnl4Fpkqks1W54Txq4y +o3Vu9aOPx//Jfi8sDc/qD/mFnyUu+AMFWjIj8UxQN4HmbrbXg/DEczRfP68DjOiX +ssy/0Rmm/H1cu7oBMoSss63mpk/NvPTSzzCR+VhU4PHQ7rxSZYS105tzkBVfe37e +hSS00rQVWe2YnI1KkfJHFjzveHiPXf+IxC0Z4PpJuLhl+pIZ/FgxJ5yEkX0XVUIy +aYRzKI3JaJktYl6WvulKSBPzQxIyOgrqVkZW4lv/uTh64pE6E5w= +=oeam +-----END PGP SIGNATURE----- + +--jg2hlfugxpumieke-- + diff --git a/test/corpora/indexing/fake-pdf:2,S b/test/corpora/indexing/fake-pdf:2,S new file mode 100644 index 00000000..60a7a47f --- /dev/null +++ b/test/corpora/indexing/fake-pdf:2,S @@ -0,0 +1,11 @@ +From: David Bremner +To: example@example.com +Subject: attachment content type +Date: Thu, 05 Jan 2023 08:02:36 -0400 +Message-ID: <871qo9p4tf.fsf@tethera.net> +MIME-Version: 1.0 +Content-Type: application/pdf +Content-Disposition: attachment; filename=fake.pdf +Content-Transfer-Encoding: base64 + +dGhpcyBpcyBub3QgcmVhbGx5IFBERgo= \ No newline at end of file diff --git a/test/corpora/insert/mbox-attachment.eml b/test/corpora/insert/mbox-attachment.eml new file mode 100644 index 00000000..98a8fc91 --- /dev/null +++ b/test/corpora/insert/mbox-attachment.eml @@ -0,0 +1,83 @@ +From david@tethera.net Sat Feb 5 09:19:10 2022 +From: David Bremner +To: David Bremner +Subject: Re: [RFC PATCH v2 12/12] emacs: whitespace cleanup for keybindings +Date: Sat, 05 Feb 2022 10:19:09 -0400 +Message-ID: <87k0e9o0pu.fsf@tethera.net> +MIME-Version: 1.0 +Content-Type: multipart/mixed; boundary="=-=-=" + +--=-=-= +Content-Type: text/plain +Content-Disposition: inline + + +I figured out the race condition in the tests. The previous test was +still running when the failing test started, the joys of using a shared +emacs for running all of the tests in one file. + +The attached diff is split into the the commits that introduce the tests +in question in my working series, but you should be able to just apply +it on top of the posted series if you want. + + +--=-=-= +Content-Type: text/x-diff +Content-Disposition: inline; filename=0001-test-fixups.patch + +From fc88cba7f1f37b9cf3b296eace2422dd0e173502 Mon Sep 17 00:00:00 2001 +From: David Bremner +Date: Thu, 3 Feb 2022 21:05:05 -0400 +Subject: [PATCH] test fixups + +--- + test/T315-emacs-tagging.sh | 9 ++++----- + 1 file changed, 4 insertions(+), 5 deletions(-) + +diff --git a/test/T315-emacs-tagging.sh b/test/T315-emacs-tagging.sh +index c9e3e53a..c26413ce 100755 +--- a/test/T315-emacs-tagging.sh ++++ b/test/T315-emacs-tagging.sh +@@ -119,7 +119,8 @@ for mode in search show tree unthreaded; do + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+tag-to-be-undone-$mode\") +- (notmuch-tag-undo))" ++ (notmuch-tag-undo) ++ (notmuch-test-wait))" + count=$(notmuch count "tag:tag-to-be-undone-$mode") + test_expect_equal "$count" "0" + +@@ -128,9 +129,7 @@ for mode in search show tree unthreaded; do + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+one-$mode\") +- (notmuch-test-wait) + (execute-kbd-macro \"+two-$mode\") +- (notmuch-test-wait) + (notmuch-tag-undo) + (notmuch-test-wait) + (execute-kbd-macro \"+three-$mode\"))" +@@ -143,7 +142,6 @@ for mode in search show tree unthreaded; do + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+one-$mode\") +- (notmuch-test-wait) + (execute-kbd-macro \"+two-$mode\") + (notmuch-tag-undo) + (notmuch-test-wait) +@@ -159,7 +157,8 @@ for mode in search show tree unthreaded; do + (notmuch-$mode \"$os_x_darwin_thread\") + (notmuch-test-wait) + (execute-kbd-macro \"+tag-to-be-undone-$mode\") +- (execute-kbd-macro (kbd \"C-x u\")))" ++ (execute-kbd-macro (kbd \"C-x u\")) ++ (notmuch-test-wait))" + count=$(notmuch count "tag:tag-to-be-undone-$mode") + test_expect_equal "$count" "0" + done +-- +2.30.2 + + +--=-=-=-- diff --git a/test/corpora/mangling/mixed-up.eml b/test/corpora/mangling/mixed-up.eml index a09f6191..d4702798 100644 --- a/test/corpora/mangling/mixed-up.eml +++ b/test/corpora/mangling/mixed-up.eml @@ -21,13 +21,9 @@ VmVyc2lvbjogMQ0K Content-Type: application/octet-stream Content-Transfer-Encoding: base64 -LS0tLS1CRUdJTiBQR1AgTUVTU0FHRS0tLS0tDQoNCmhJd0R4RTAyM3ExVXF4WUJCQUNwNzBlN0tQ -eTlPWWFoZUlya0x6bWhxMWxScW15NTFhTDFqQkwwSy9xTjdyZksNCkJaRUcxY1I4amVMalRGZFBL -UExWS0pJODByN0ZnS0kweXd2V3ZsNlIxYUUxVHk1Qm5WWFQ5WHpDckVIN2ZxQ2wNClNLSzgyRXZv -bFhUb2hBWkhVcmg2SzY2ZVFRVFRJQUMxbjdCMEE4aEVyemtnYU00K3NlTjNMbHZlelQ2VExOS00N -CkFUcHFzRWJNMk1WckdndzBiM29Vc0dHQVBFdDJNbWpORVlzcmlLbnF3dDZkSkRaYy8vWHloamdN -UWF5aUQ4ZGENCk4xZ1Qzb3FndS9nS0NwQlpEWXpIZjlPdFZpMlVubEZEV3k2cnJNWkxqV0RuSXY0 -dmU5UG4vcW9sd0hWanpkSjENClpmak5DNXQwejNYQURLR3JqTjl3dXRyNHFtN1NUVzFySEFYSFA2 -OFRRVHhJMHFnSktqUFhOS1dFdzZnPQ0KPXBKRzQNCi0tLS0tRU5EIFBHUCBNRVNTQUdFLS0tLS0N -Cg== +wV4DHXHP849rSK8SAQdA4L/4ntrLU9OGEHy0JIryOhNPoVCN1MgZieW5E+1wDjMw +GZL3EGlGTYWPhAm6wmDXsMHWMtuBVawUN0lZDuUoQtWzXYZrn3HiKQZn2ahfIrrs +0oMBLz48JurPlrHCIHfsVa/YbfZOTun/TPa5zcjX/Vi0+FgOHEFCmHzK/VqlqsdW +PYgQPvn4rQL25GACLzzJGrDvvJKS4fEo2p3pf6SGDFtBeyKb0AdyXoBWfXelS+ST +rMTVqEDTT+71MToOcmYPX2QJRIMEVix8tCmvMkUXni8AjurehQ== --=-=-=-- diff --git a/test/corpora/pkcs7/smime-onepart-signed.eml b/test/corpora/pkcs7/smime-onepart-signed.eml new file mode 100644 index 00000000..070303b7 --- /dev/null +++ b/test/corpora/pkcs7/smime-onepart-signed.eml @@ -0,0 +1,51 @@ +Received: from localhost (localhost [127.0.0.1]); Tue, 26 Nov 2019 + 20:11:46 -0400 (UTC-04:00) +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-mime; name="smime.p7m"; + smime-type="signed-data" +MIME-Version: 1.0 +From: Alice Lovelace +To: Bob Babbage +Date: Tue, 26 Nov 2019 20:11:29 -0400 +Subject: The FooCorp contract +Message-ID: + +MIIHRQYJKoZIhvcNAQcCoIIHNjCCBzICAQExDTALBglghkgBZQMEAgEwggHJBgkq +hkiG9w0BBwGgggG6BIIBtkNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNl +dD0idXMtYXNjaWkiDQpGcm9tOiBBbGljZSBMb3ZlbGFjZSA8YWxpY2VAc21pbWUu +ZXhhbXBsZT4NClRvOiBCb2IgQmFiYmFnZSA8Ym9iQHNtaW1lLmV4YW1wbGU+DQpE +YXRlOiBUdWUsIDI2IE5vdiAyMDE5IDIwOjExOjI5IC0wNDAwDQpTdWJqZWN0OiBU +aGUgRm9vQ29ycCBjb250cmFjdA0KTWVzc2FnZS1JRDogPHNtaW1lLW9uZXBhcnQt +c2lnbmVkQHByb3RlY3RlZC1oZWFkZXJzLmV4YW1wbGU+DQoNCkJvYiwgd2UgbmVl +ZCB0byBjYW5jZWwgdGhpcyBjb250cmFjdC4NCg0KUGxlYXNlIHN0YXJ0IHRoZSBu +ZWNlc3NhcnkgcHJvY2Vzc2VzIHRvIG1ha2UgdGhhdCBoYXBwZW4gdG9kYXkuDQoN +ClRoYW5rcywgQWxpY2UNCi0tIA0KQWxpY2UgTG92ZWxhY2UNClByZXNpZGVudA0K +T3BlblBHUCBFeGFtcGxlIENvcnANCqCCA3IwggNuMIICVqADAgECAhRngrRZc1JL +wfRxRxlq8P0RiqpMCzANBgkqhkiG9w0BAQ0FADAtMSswKQYDVQQDEyJTYW1wbGUg +TEFNUFMgQ2VydGlmaWNhdGUgQXV0aG9yaXR5MCAXDTE5MTEyMDA2NTQxOFoYDzIw +NTIwOTI3MDY1NDE4WjAZMRcwFQYDVQQDEw5BbGljZSBMb3ZlbGFjZTCCASIwDQYJ +KoZIhvcNAQEBBQADggEPADCCAQoCggEBAMPurfll0bYkDPMkY1kNn2xXsAqHSGVF ++gWNNk3mbhF6BABhLJqDjei5aLXFE3Rq9/RRNivCMrTipF1XsbMIAKgQqr/GI1Q6 +yN8lfNsK5uU3d9kw5cOyEooGpOGUrvlKMD0LPGDt6MaiJj+KJ2TR73Wd4rfRIIJo +FMmV9HZkOs+Tvcg8x6SzGhNq18X2HD10MD78eLXKm039obRD+z2JwWvGvrLbNBey +O5A+aMxmCPXRoP1xrNZWBFgKB+WGYDRXW5CXXChthTwMBXFWf4aBpurKMZAyjK2E +grQafn6h/DFddQz/NtT6Dr7UhJ2hfFFEW2rYbNsiqQAdllCb4FucWuECAwEAAaOB +lzCBlDAMBgNVHRMBAf8EAjAAMB4GA1UdEQQXMBWBE2FsaWNlQHNtaW1lLmV4YW1w +bGUwEwYDVR0lBAwwCgYIKwYBBQUHAwQwDwYDVR0PAQH/BAUDAwegADAdBgNVHQ4E +FgQUrC5UWqT9VRivLuhmRDjRJdHXAHkwHwYDVR0jBBgwFoAUt1JNc8CIPbLDeloM +85T394Cid9swDQYJKoZIhvcNAQENBQADggEBAHvqjhjPvKtVIVyleoutwa10jir3 +dooJcQIILM1AunjJ6yHpuuppkc0m3BhwnlOptTKb2EnvSIkTiMY037IBlHWW217Q +cUpggEozgQm6Yb77aGptRovPi2XToEdpA8K//02I1jur1H1z8HqzVjMeHCqRaG3Z +r4C2AngGSkb6D4yZkxBX8CjtHAsUon06UxYsGYRcVykgk3Qek9qxPScSX8yai1K7 +7xGcKUCLfIV/JMpv7ysPtXG7Jd62oNnp1T/3+KoP9JlLs5AiPLC13fjeYALPcHVG +UXEwdIDp1AB/Zu0a6apHQqICncqRhEB4+hompiQHtlp3TqeAWXQbQUc437sxggHZ +MIIB1QIBATBFMC0xKzApBgNVBAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0ZSBB +dXRob3JpdHkCFGeCtFlzUkvB9HFHGWrw/RGKqkwLMAsGCWCGSAFlAwQCAaBpMBgG +CSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE5MTEyNzAw +MTEyOVowLwYJKoZIhvcNAQkEMSIEILsI9kL3zfZiVOEDjAUWrbjHjGMLoGUwEqYH +pOA9XZ+QMA0GCSqGSIb3DQEBAQUABIIBAGDat8UYN9MShlKEw3hYVVUk6HKO6Xjp +rdgCBKpoyoWJy0VJis0xHxaT2gn/+TPu8a5l6RslgeALjMyflzyzAmrqnknQQG8K +bvbt/MwpU/TxnmxT+2oP9TVmAx/IQOq4pQ35uK7peSPck2CcTvZjHTeVBWcsLVEk +hELoSD8XFRBo34qdinBzW0/sMlyK1XnlN7khKry1g7uaXcurVqptRA1rWOvCOt72 +aElKG/Q7OoVgHxbUpdzV3Hqe9/UeTRDUqCs++on2pLlA0TA0Pq8RQ0hDHD/p0t41 +1RAT1/RbnGQiVfRilMan+VGT4shokb1RoANy/1rOO9ZKlyWToYdRl9E= diff --git a/test/corpora/protected-headers/double-wrapped-with-phony-protected-header.eml b/test/corpora/protected-headers/double-wrapped-with-phony-protected-header.eml index b05cb545..115019b0 100644 --- a/test/corpora/protected-headers/double-wrapped-with-phony-protected-header.eml +++ b/test/corpora/protected-headers/double-wrapped-with-phony-protected-header.eml @@ -17,22 +17,25 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBACkvfKZEkuRUQ2ujdel8U2ufplGxE2oNOK+CI5S1O8cS9vE -DIkVIXAtpZcCc31pYBTRl0TwCrLKFT/siYfshbxyWjMZjX/Jc38Yjg9pDFTIZ312 -LoM5uH22f1X8O8020HgH+CQk9T4s9bBuvxTvJ6GQvK/ssnoYsGr9TGcjjh3uMdLp -AXkkF76a2iimkq2163ee/8X0vgI+2fx6EjJJvlcSIlDcUvhYHIt8kjnlADSBMpho -gaMa90baGlE1RAK9nSBC+4ty0fIlfsgcecRtFEifFRj6foYPFIFzkgwhRkXovouG -FyXi8QrDVS8cz61I03PMVsFHo4FtJw9cAfvTh45QFGl+inW2pSvZyRnyu6uHDe61 -NqUTJOVN4B+dFPbKafUKuJ4YGXLsDoQoE8VF0lwznA7AOATmqPQpp+Anq40C/4Su -Zf1hGaBTuYjlChSTMxX+wV22+PQwJmK3tl1NQRFGlR1pQZWdNcu6/6RGooiVZSg+ -VsmtZjgpZa8aaEEnrsIEVPfvbIZ4OQhmgNi4CYNB306UOjIh3/8m+8JmlkxPiGXW -gnzNUTuwKytlZnIgT1o9a7PAkz+ZiHhMLmk5nPN+dlwsVN7Ff1FHqLIMbKaZbeKK -txvhw7/NdaCALnjamqtDJTc4kL50F44DC0im0U9hcoy8X/HBrYkTGfHgRttCp5V/ -XisGT6/rzyUzTi2usZpRtl3WhHrE0Uj0w2Bm/Qqe64vNd3F8xwuJ5qMZ3QLVxoX0 -MPTajY1pLgfMViqLaLV8fR8hLmattxaO92sbVuxHiaba8er3jzO2HfmRLqesio7u -8FXZQnBgeqBkoRlrHhvScuZLJVU1I4UHd9s3mcR+IY5VvjxdPMcnxTNqcRB/He4H -MrrH26P0uSFe6WJYQVXEDt4OO73ROyFZE0+rSw1z+VnjmHVIzUVvvFqwJZo6Y/0v -1+3ab4TGMPJSkfQYHY8/O1RF67BNlA== -=gizc +hF4DHXHP849rSK8SAQdAhuTX3AYQs007gCtcGsYQmtZg5DrD3ev6Dm+1Fq9vfjQw +bHP9+UKDAmWL5XW2+zlFQVb1CcWn8QUXwjJBZvHEpwyxRWJNYqNlhsnXsX+5woGs +0ukB6Jq0OJydXSKubEnI5Bxs7UC7rekUfxUH2ij6ZXmz5Xyc/h5rF2rJO1uaPVVx +LuVOF+ZX0muBzegGMiUtunHmY0Akis7m3xDl2ZRVViRdW+LPoxb4oAGoPq5LHfHp +5adDtrno13K6oO0NriVEPsSl6xFCLXOqTj600F8658o16yA8fiZj4kY8G8NunmMD +lN4UNw/EQuWIe343dKaQo+iH52atihw90OG4ahmjvkhBC/iIiXZ3J5m10VoJ7Sxy +G6O84UW06pCAqvPmkjnkgpflGdpM3Qv66Uu9ss6jAnRLP2LVv+IVWrwle/Yp6B8V +1adoFUMphkp534vTR4v0xJ/6WaI1YzIwlT2zd2zCMoqwT5weuX509pTbae0iGLRG +Q+l0ImN4AxsUdgWIW+GtMn3GVWCXk2BDLcr1xkl5lJVlOIbIvbpTwqYFMVAfnDGc +s6JZS3Bwg9bfmxSkM4GhZlDiCdZE7f0IUBSnRRHPcMkaox0g7ibF8aG0himZRqhD +3z6gtPmWO+c5i1iV6pQbntB7MY80vprz7gzmvmadbGFS+A4uCaH2afWr1eBb6nGx +W6deJxHaCQrrxujz20y5osSMhdAk47WPkZhphU/8ERncZ1F8aYJhUprAbKFxNjt6 +I4p8lrfAe7iM4UxqtSfVx2rPV9/Nqm8rFVH5l4f/0P93qsBNdF7rVQl9Ry2kVqkk +86ct9ZK2QrOtsnuSYFzhlaDo9pOHkhlMOmQXAgSb12VP87z65NTYRGbElypmQl8F +MXMxrljqnCbPaQaPq/37GEHTRJ7rrUvCQ2L6Ljp+F0m555eYHNiZRaxYA17uABOf +RIr5gfB5ICmnH2EGIYiEkSeX3k8+8xHffrzTpOAqv/5oJBDlRcu6qyVSAumNMPYX +fKqlNDhKh9xmK7K88vWwTgeOBm/sOXcfuAkr9nXqDqdLHb+duLiH+xjvBdl+ewTx +gpzIkoTSsd0fgmNqmmfwmmU4oIQu4QX3/zh5HiQS70xogRxxU3przSFrrtNkiSkI +Oe5fIqQ3tlUSzutTEoaSxeU= +=4vL7 -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/encrypted-message-with-forwarded-attachment.eml b/test/corpora/protected-headers/encrypted-message-with-forwarded-attachment.eml index eea66a94..d43331bd 100644 --- a/test/corpora/protected-headers/encrypted-message-with-forwarded-attachment.eml +++ b/test/corpora/protected-headers/encrypted-message-with-forwarded-attachment.eml @@ -17,17 +17,22 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBAC9RgjF0vsqVqHMB8fauhazs2XoTMKkANrDS6ECANm0wcvO -tU1huRepG8ezoow/OgZ0Yd9y/zw6w+Frrx1PhVEr01lQsUdRq7INq2FRia015Q6Q -eOgSv9Q8wg4Vcy9XD1wI2Un71nDvbNwqx+hiR9m8vhiWfXH1MvxVQUWcUocUMtLA -uAEB+fx5ag3Qr42VAgyymvNrHJKtuhdj7CvdT/a5oVbZV7ilflFlYms7Wq0jSex+ -Jrb+/CnNLow4LehrOpf+IfgPumo0nBbseB17rAM9vtjNy+tHEqPsB0YFIpVR9FOp -zJITbWeFyGbOd5vMk9xbEFbw58JR8PPqsYJK41RleU2QoPEO69hoV0tXzjby5JQZ -2G/SrH+m9tggi3rWxHx9XuNKJP4iK9wZnO4k5DFaUXq6PGCYkgDi/K1RuUcJjcv7 -ob6Yp/cTLxHMmIS9VNNjUnnoaD71ndzYsZoaI6MTMX7/4eu5roeE3887NU5af/wS -ep6POG8WFJzKwc4dvAPd0NBVojdrftJkYKONsYL5KN8TY8SqUPxiXReGwg2evQqb -aGEU02zdRGYtmNSneGl20dJ39cHoW7B66ek9OQkgilSHQq4adPleq07r3HSv87jk -xNYoQ7xH2fahqbosW8N5uI9L2sdGVmTBNZgejiNyZoUn47tFEt4Uocg= -=/ZB1 +hF4DHXHP849rSK8SAQdACh9CzQl7cqCVO0bpHfyqL0Kr/V2TtUO68yhOYuMMWXEw +v+aH1aLYervLMNDSC8Fj6XY9myqaybbOfHnLZIQg0XkR3JIKhWR3LpZtfpFGSr/H +0ukBgdihDN26YqD+UETkBN0lg/fd4+cvVyOJSwuU5DQFdsOk1znvEercz1UVpKxz +gg1lniZzRYCyPSl9NcZVtNXggz6z9XqxT0cVE3WCH2IsVrJPfgnFu/P63pAhp43X +IxJe5mB6y5eaX5Dcvj48Z2E+3tVVn1gKGwLzPH40WIy98KM4dv59I0ZghM8h1dVr +LUUgdiz17fo5oWphzdgn57ERk9B/pApkzujrB4Sum9qy9y/Q2DWFKdOARNJSZ/Li +KZVXZLVEyp1WeAU262blQRtqWXq30wU9PazFo3wsUUVoLCu5SWsRpircSIWrh3gH +iBE/YdPYQNJ6kJoz17AdIq0a4jH3ae881P6eBW22zMxdqD4zfkT43iVQAtuki4JS +iBJ2vHoMU3z+rJ59ea+P595QHCGNyfkgl50e0E8rh9j+9/CExyD6F4GwAZe9VA6o +hYFQ3U089Cp/Cd5jVojUWIAOWqwb6nB7w3SabPZIbhhJSBdQEGe4Xh3TN3k5cpLr +TxDJz4zf220EQ2pt0BJwDu8fdskMXC6ytQettvR46+UtFOOG9eClnAFkgeuqwKmv +k1YyYMFWQ3Bv/IHOFrcu3negBQpP4ln5px6zggNoeNLzaZ9PN9Y2lHnoQTkw/93L +qyjLD+xEnzgdPYZvVsNA1iI6LMrCandDwKySQyfZadj/G3/nLFcoThvn03NNUyCK +Sh0Vy2gJhxD04HF8nkR/E5MLFCQ3x0qpGbOoanctgs/glxCpnzWRcSZERlilQ+3Y +0r5UkO5XJU3Z3CP4uJnjykgB8Qjq/WzERgOtOQ/bVaWBWkKzxS/Pj1sJ47kKMtBJ +3r5LXymlkA70RA0cJIv1F/At +=vLN3 -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/encrypted-signed-not-masked.eml b/test/corpora/protected-headers/encrypted-signed-not-masked.eml index 8dfd7c39..9f0e51b9 100644 --- a/test/corpora/protected-headers/encrypted-signed-not-masked.eml +++ b/test/corpora/protected-headers/encrypted-signed-not-masked.eml @@ -17,18 +17,17 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBADAJ03D4w48sefkQsBWXUc1spTljROjVN+y5a2yCKtYMt3M -wWMeQyem5hwLpLYRCfeIzXCrlBfpZffuOkA5okGGVEWFvJ5a1kZNZnH5Wg0ccBp7 -KBGnJY0gS/BlrKK2Sjmk9Z3ww7GAgDGPbc7mc3Csj9G38UvneBdrQgm6kZR3GNLA -6AGLN3KJETruI3Js6++aG+7tSkJ8Vo4WCVUR7oQROwF601X0QF/XghCoJCrx8B/1 -cw6Yb2wQj2nv3gw1rqWVsPVpAKsMc1yHx/2Vsee/VPtt4f67fSAMuJF3EJ6JkcK7 -tM761v69GoJGgvsie45pb1N2l/GfVMuwWU0wZhEsF7eXxqPzoE/kIGX1XIqleLaw -On2kPSM5RgqV6gLOcw4WaFPi0oMbDhltNs72SV9cV6ZhhuwEQRq+u/K76NKLwte2 -R1JutAiuPZVF0WanmmiN6RbIpWOB5XxQfWagfr4vcf/03TaLP4hJMnqUdFMk20HP -eI8TMQxkfryZK2Z6VxEBVdXhK05VEdkolmc4j9U+76A96Gd5zbYPApirkebmZatS -X3rKKAiBqwWrFXi/7LNDoCwhRRmqDuHXruh3vZEcz+xiPfJh0G31GJQgIpE15Sv6 -trf20u3CXAFjHg9zPpSFV7uAOsqv7bg+xtG9PgN4aLCiVbXHsT0z6PAz+6K+SiKw -QW8ZOtLikj5HyLAz/TDcsIShFaM3QHk2qq9RY10kmxlQVrf9Oyh3Wmc= -=om0O +hF4DHXHP849rSK8SAQdAhRhcdC3r33MMW+D1PgLAoIhstxnEWOjI0s5wyH2gKTIw +ceJ7nVvmnqLNWbEaoLp2tVH1+cI4guvgqzV60BoxNU5G5YMGqzMS4VQ47N4BENmn +0sDEAWcBT1+PGFlXAeotmryF5ErMXesEVJJHru/KHZxsP3wwRp3qZjrwIWVgVlVo +SpJJ+hwIRqCTbdsw0ejKY0+BJwQ/z+GPmYkWE+ARxkzwMzfA47PW+iNudrQNEJuu +ALnsbyVX0/JA6E8jhvIA2OHb6G2yqFA499I6kWiSkT3m2SikuyKGybwWbaUDZVJW +Y+JN2L8snDUHJKfl5JMhAgu2+fx/joE9PxM6fdx8rJJbpmTrPqrYf6vYyuq5BGYV +snGJozwiN/cqR+PruMT7i5/Dhs/EDl3Z9D8iH7FEDtIzpSJNvGFIHetXC4JXgO7d +C48g+0uuUL3SSuZviz+OgLJbyDu1Pc3tCrtBUs0zYJGio5ghJljU5tUnCNhbBwd7 +oWgYxOmSzbZrs73E5Lpvnq+juZpXSGNoYzaZHacp2FTpo0LlZ2k6o8kx6eYfOlHs +JN+43CnzTnR6eZYkfjIaBbjYKi0rMH2DRJjMXyeYRLjEi9ET2fFX3WWn487snIjw +om8EPhsOxQ== +=wKPm -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/encrypted-signed.eml b/test/corpora/protected-headers/encrypted-signed.eml index c97d8c3c..f64b99bf 100644 --- a/test/corpora/protected-headers/encrypted-signed.eml +++ b/test/corpora/protected-headers/encrypted-signed.eml @@ -17,18 +17,17 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBADAJ03D4w48sefkQsBWXUc1spTljROjVN+y5a2yCKtYMt3M -wWMeQyem5hwLpLYRCfeIzXCrlBfpZffuOkA5okGGVEWFvJ5a1kZNZnH5Wg0ccBp7 -KBGnJY0gS/BlrKK2Sjmk9Z3ww7GAgDGPbc7mc3Csj9G38UvneBdrQgm6kZR3GNLA -6AGLN3KJETruI3Js6++aG+7tSkJ8Vo4WCVUR7oQROwF601X0QF/XghCoJCrx8B/1 -cw6Yb2wQj2nv3gw1rqWVsPVpAKsMc1yHx/2Vsee/VPtt4f67fSAMuJF3EJ6JkcK7 -tM761v69GoJGgvsie45pb1N2l/GfVMuwWU0wZhEsF7eXxqPzoE/kIGX1XIqleLaw -On2kPSM5RgqV6gLOcw4WaFPi0oMbDhltNs72SV9cV6ZhhuwEQRq+u/K76NKLwte2 -R1JutAiuPZVF0WanmmiN6RbIpWOB5XxQfWagfr4vcf/03TaLP4hJMnqUdFMk20HP -eI8TMQxkfryZK2Z6VxEBVdXhK05VEdkolmc4j9U+76A96Gd5zbYPApirkebmZatS -X3rKKAiBqwWrFXi/7LNDoCwhRRmqDuHXruh3vZEcz+xiPfJh0G31GJQgIpE15Sv6 -trf20u3CXAFjHg9zPpSFV7uAOsqv7bg+xtG9PgN4aLCiVbXHsT0z6PAz+6K+SiKw -QW8ZOtLikj5HyLAz/TDcsIShFaM3QHk2qq9RY10kmxlQVrf9Oyh3Wmc= -=om0O +hF4DHXHP849rSK8SAQdAliiUK1XTMeUUmjJ6n4yEEBGVPmQuVTD4wdJZx4r2J3Mw +uSVIq8qBJ7NrZfvDa63807usyKpk0Io3tdH83hn3wIKKmUMTroaRZMP1BdJ3aVdh +0sDEAf18AhE5POknHzzolNbTXmQkPNspdNpu0ANdKkukvUZjfWvssZ1WN3jB08Fc +Wr2Byez+UyLggj1bm+HuzbcZGPfyMOpoajFG0+yK4Ccqpf3XapNJn7v1ic2QhR/B +pmWpJR0cXDigNLYQ/RoNEiSur2+0hzeVkDiYQtDYD02Cv4rnhyJdCalyriRaLOZr +4Tau4xfK9PdOydahZZHEFllDU+yfEb9xtBZ+DEGh5AyR1/sG8ORnJc+5m/z54zmJ +W3MmZ7M+4O1q5l5MbDwHq1TrQF5R9JSXyeGTW2XscsIPdvY5bExMwyMoW/XY4E8+ +GJ12UVpC/EWt2Nisfx2LhIoxtwaZm2r0Xkt6hY5DSMkpHLZtGPOOe1QxtXF1qXH2 +owTZynRxlmaWslOXdhtFRHyfGiFK4pNoV6fVB0e2khKKfXynN9Zx99GJ+DoJsOFT +AYOadP1WlbIbSaimv/xSpaijE2XReKTLfN6aK1vSe90r96nCWr31fcd6x9ax0nVx +jDReMRJrHA== +=8mRD -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/misplaced-protected-header.eml b/test/corpora/protected-headers/misplaced-protected-header.eml index f1a72f0d..c93ce0f7 100644 --- a/test/corpora/protected-headers/misplaced-protected-header.eml +++ b/test/corpora/protected-headers/misplaced-protected-header.eml @@ -17,19 +17,27 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBACwbgx3N72gYKIU63tNE6kf6UA5ed39VFXh3zdM6eDdA0bG -DWt5yROckkCeCvMoFaRswK8MiX8aGG0GdH6VKhyn7HjT/Dm84QLwoB0ccZs3MnwU -aJ9yTC9HbX3yfTVZYOu0w47NZho/LXX2Yd1pi8OUgrPg44fjgvx2kNRQ9EsNBdLA -/AGMhwwcTPHjyWQ4XYZoL6WeVJfq2C0m3hQ3bxrKuAzW53HrSa4tPCXzX3G8KEz5 -sSk3ZOmajSvLde0LG8bxwexgAHC/Wd07e2HgHtZ/H+Cw9oYLgwcgVyXg7sGVrMrs -IlwW0Njf93DJmJZuTD8P9XJc3h1VzKA+YhbtnofFZw4JexpHcC+R8Lcso16Mkp91 -7Ig0E8WTZ+K+judGS010b5ND2ETyc+TYY4/XJ2R90pbNrRLNTFG+P2HUob6PBCwE -rXot6TeBSgm+k4bvl9aMKyrBSplKktQey4WsdblbJnJUxSl/rMpW6xwglkyIgrCU -vbhffqgB8y1JLmK6Ow/A6Pzi3T6Zn95zu2GN8+yAOzDhGwlAfIV85TYnX6ybOkX/ -Amoh7qNS17pzc6ch/mif/RsSPYo+y2UQuVFhG+kOy9oGAQOOHeiCWZPa09o3R2Jn -myMg1FPgoDgsjE6QpD0mx9ORdPGC2e8jwrifS/W9eHJ2QG+mNkcKlAr5b8WiUTkq -hEZ+BaaVhbXN8EuHHTJT6YojusCIsXI0BMF1su1KupQw+dwQnys8wuy45Fr3H58x -zqHoU9KzdQGLbeJTgA== -=+EWE +hF4DHXHP849rSK8SAQdAlqPAqdaaB751ZhN8drQheW77zbyPrbofTZgTa/32UFow +ShGDg7SQxQD+LssHmGT0g4pUPo8JGor8BIvnvcZa4qRiZJI1i/VWUVoLc+TBT442 +0ukBxtUeKC0V35/zRgS1ZXrbkiBh8TGy0lee/eM9yvfT5FmFhotzoQkexDSEPxjb +g8Qr/IcH0v2XOEmHux3hQpngY/LRYoCv1c8Niu2Zfan1p7zefOZjAxRraeihHi6I +KyyA8WIdiIac97vO1CRyXXATJabjeijH/BaUDlPvQ39c9Mdie+HeaVYcaC3e9oW3 +sjn8fN+KjkjLXwPWhgMDpvP9f3ZD8oCHkN7Utq4OTwz8gb0IbUIkElF4G6O3AckV +UXwrASodKBL58IWYz6VWhyjYxoV2R2hLNtnQWWvKgYzru2Mdb/ONZBM/JMXMjFUt +oxDJs2pb2sun2q6n1sWKTrS9MXbxeFEcp2v3hMXrmP8nLeNiPJ+HlerFCSZasDOm +c/3X97zIneJUosm3ltqLunSSE4vp+FAGIRaibMjyE2u2niDReINxX58E7o9cfJL0 +9Af6olZA1gQ/8t5qMei5qTN3wi4a1ieqTN4yD4v7r8yZE5PmlQfyY0joXVepm8Tx +9/rDLM/gU/zogoo17enlU3ipvLZKsVDkLIlv2SdT1AF0ROnjLBGJ2iWzKst9Rg7/ +mXzEqv9CSVM2lXd3qUqra7reeaVD3vd00zmLq3yM/2sHeuLAc40R0MkIOSa89eRP +iXu4k9+/m4WU4bNPugaH8Szm7OuLQNoB7mAB6t9GJGq7Y8DKxMn5FSUwLtKkGkUQ +YdGCw2/salvqoQIDJVMteKpM4pWi9sFoVbxAU+djinJejSjSmY+iKHcMYeQwOpja +33+W2RM40JOUZ7yc+LzgWy/ahyD27cdS1TPaghlQxPeSjcEfekrjgGW+2wIMmCxy +xyMPWeN+Mf7e/fNh6YDbld9BKk6zY4fNVWlC+gR/aMVapomNgW1/FS7F7vk5qY0Q +pAf1GS1YR4pKpo3sSEaLHw5rhpI0pT5ZvB09+GYRhsBQzP3LvnBCsxcVvi7yERY2 +QVlHejpTWwuRcTQCXP9UXa7z9UhEOieHwy6xNRTi4HVWHbXMxVunvlUnBCnQdFIr +s4bcOjY94Q29FeRjaweYIZBUSabXDLJhLZNH1u6Z9Z8dBZ6Th1+ou77m0rEN6B3d +mf83pKPrOR8ASw88rRGq8O9yVpFR4mD7HiE7YNofsPBtk0Cz178WLCi/D8b9A80h +R/jVyxLmrCPHQUFfu46JgytBta+VBYwnYFHDynnYczuB0gydvuPJ4cJvfn3vPA== +=DmA/ -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/nested-rfc822-message.eml b/test/corpora/protected-headers/nested-rfc822-message.eml index 059783ce..4d5073bb 100644 --- a/test/corpora/protected-headers/nested-rfc822-message.eml +++ b/test/corpora/protected-headers/nested-rfc822-message.eml @@ -17,16 +17,17 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBADCWqobSSS78XdrxhBh5W01OZbUMdnrwrYJsiG9fQoVfFHN -eALvOfviTcSBD97/jO2IRL2W8hyF7k1BVAYMwSuxe4qLbLdsxK1i4KBRIFRkm990 -ipBddgFXV16WNO2cTK7boEJ7Xfjp/zjoS2z2YUXsdGx3OSJciyHBVJki2UfkL9LA -egHa7dsw6BxoNbAkrD+ijVbsFrKHeeJIlWkNbSYOk/YLmqLAEy1CYvSvC8ZSBtQT -fVYc37fc3RB0vQC+Vu5k5d/I5Z1/Yz+McBJDMNvcn4yoFiXemY8YVFvj7iC0sbuq -lwitvgMYaljhb8RUQAa3Dy08Jju09DIBcCgRsx32U+3aqZ0MhU6CRgt8kc9oK1g4 -yBVppqpX6hCXjtt9LUArY3DIchRb+IWTXsb+eDR700GXDyNMk1G5WUl0eLuw75uz -EqU5Tjh36fP0ceMESjaxuxyhhw1jjE3ON7vqFQRVcs7UtazbxznWQH3Z73mDmY3G -q9JGMOOqVnnFdnEq8vDFF7m+Cp3N1ieyXUXjn3aLtvSRMmVV20Q5QXSFg8nP6juT -Yn1xZjqOodSeig1ITZZF58Whv+LHGtzDHwV8 -=cNYF +hF4DHXHP849rSK8SAQdAc+vNxgvGe1T23qYt8zZ2dtmU4+DzCiMlKBFrI64sICMw +foE2ym+RPLXzL2SbgDaUqgYbkiKmo6HrFSURtlDae0lFHrmLYZHToVWXF14DGsyu +0sDgAcsIt7i8j6XWpAI9slRqjDAEPBp+4EFJKL+BdIuEYa6z1ULrv5BUimUi6D1o +UE+k49xE6iFOpgSEghF1+ZneE3bj8rqJJwA+sAjth/Kp8vNccA42mCyn3Avxvk9q +aMw7GWvTRJ5oc+RGo6BZukQtApQTbLzOeI92w68XNmQaSq4+LKUA4+CTZpasR/WA +CR2/MWfwW3vmZilPbW2249Nj8CAawaDxTsIY9i5bHE0HjbfJhBBNffDPoNCh/+Pb +6wIZ9/6xLHAzxFtY+qvDhVO/nWOLrdd9ACZudoD/x4qITc9IFo4F8bWF5iKPs67q +wtw1qFTT8ODc2WWhOizDByOkk/D+Z3mrlsOC/x7ioho0IIeWldkfaET2ucc9FI9S +SPX+huu6vnPAGO21T4EMqevwDLNGWMQBolHSlU3SRnNyU1bqUNDF+/9RiOk/NFj9 +FTwMj8FAI6/q0kZLQUF34h4BPF8/v1TmVZKniaVXqQIE78MvFWXV4FXYvP4w+IdK +q634Ah+cC9NVEB+U6H4aSB9IojUurd+RvD+4x7rd+EtLFdc= +=HmgM -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/no-protected-header-attribute.eml b/test/corpora/protected-headers/no-protected-header-attribute.eml index 880f60e3..cf786c02 100644 --- a/test/corpora/protected-headers/no-protected-header-attribute.eml +++ b/test/corpora/protected-headers/no-protected-header-attribute.eml @@ -17,13 +17,13 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBA/9GY8NN4NDwpNttr/hTXpS701Z8TDr3hC89obZNnNpYxSct -p+YkS+FsPMLimIDfU1meG8R+YgtQOJIhmKPHW8CLQ1heBsX0Dcv2oLxXodqNGD7M -/szVRR6duVnALPgmV66vkcBHKbsiuv8EO86C7G1hAnXfs0H47WoeUz9dQ6RaHdKw -AVbxw7KWVbiP+S4SO1rvNsAL1xiRPA0FFmDRMyoFRC/618dGS6HitkD0UR708oVt -PooD4Rk22c8b549wvZ88flGk+WBCLhyXAuWYPHwag1DLzLjWH5r+XmK2O7JoQZeq -k7JM/M8QM+xetFaPmsWs52IynhXyWpXBBanm9NEsNEiIB59480D7tJ0oivo8T24d -izSAMGATP26ReatoXltCl9x8uUfUSAjWt8iJ1+n/3ds= -=hGDA +hF4DHXHP849rSK8SAQdAQd0btUvfUMmCgmDv3mG8d2tfmr3MYsOdE4TeSqk6sFsw +8Wsp5J4s2t9ua6ScvqCVtlUVUL4Z6BsACArwy0/XwGQ8JQ6BUuYH7JSWb6O6tzed +0r8BYDEY7/8+QCgXneo6k3wvPHrzTsvg9fxmCSTbvA+8JCrEzbvM5l/wQmSDf18x +difPsrCT75x3eYgOy7NNYIAg97teybVWZ4raPQ/SdJ9J0TmPTjQF07HPndrsYGQC +rEKvu6oq2/HL4NWOWMNNixs0u6ALRpPuUUXKgwSaZUG8+juOZ3yFe56bd1IvwkWK +ZkKwa9gBn7WA+eNpcJauux/ta6LuXiNvNHUGUnd9puVepi9GO3XAxYcOzqF6ly1V +iw== +=c3OU -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/phony-protected-header-bad-encryption.eml b/test/corpora/protected-headers/phony-protected-header-bad-encryption.eml index 15dc08ab..d0e11a10 100644 --- a/test/corpora/protected-headers/phony-protected-header-bad-encryption.eml +++ b/test/corpora/protected-headers/phony-protected-header-bad-encryption.eml @@ -18,13 +18,10 @@ Subject: this should not show up as a protected header -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBA/9ZaOuxGtLVWiA7KQfB+4td1AILd1uy039UDb+9YwlhmJTq -mNqVJu+ZkFniZPMliM0z1QRBkBeL2Q7MrHAdYxYBKrDHKVja4O7jwqeKjy5BzQCW -fnyT+sb2Mh+dz5P2voF3XJHgqzhFY1rtVEatXSZADwwIVU6oZqGZ8GOELNGSd9KX -ASNElH7WGZB/TQ5X+MktzOLExx5QWaRK9skogI2RRoOquS7KpMcjzb2FWaJDjr1s -hd8FCQVjWuUDrolMGH8cgeq9iUBlHMzfPY6/jeGHNrjk12wwhBNcq6O95uzXtIRS -BM2xnwCYec6wYJ46fHukTgv+286nSQcV0XT6a+qM5GMgV5DMHW2vSyl6kTszJ3EP -xvQBfPCItA== -=Gkxz +BADBADBAD49rSK8SAQdAeybb8KrIaEFV5+Ks5loaz651PudVdzS8ombK8EW7Mnsw +kEppTiE4jo6ZocHvhjSzPdEK4MPh0qmKvu1RrHa23dc1n7Cutg1FjOb6ZRloTisz +0jEB3YxBhFDBZyWSGeAeZx93JaNcV6CBjOfZ6GJhzkqmSs73VdwFUWQxcoV4q5sL +GYCW +=BADC -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/protected-header.eml b/test/corpora/protected-headers/protected-header.eml index dec822c2..a10612f9 100644 --- a/test/corpora/protected-headers/protected-header.eml +++ b/test/corpora/protected-headers/protected-header.eml @@ -18,13 +18,12 @@ Subject: this should not show up as a protected header -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBA/9ZaOuxGtLVWiA7KQfB+4td1AILd1uy039UDb+9YwlhmJTq -mNqVJu+ZkFniZPMliM0z1QRBkBeL2Q7MrHAdYxYBKrDHKVja4O7jwqeKjy5BzQCW -fnyT+sb2Mh+dz5P2voF3XJHgqzhFY1rtVEatXSZADwwIVU6oZqGZ8GOELNGSd9KX -ASNElH7WGZB/TQ5X+MktzOLExx5QWaRK9skogI2RRoOquS7KpMcjzb2FWaJDjr1s -RGboX7NG3xCvNUV2ByFTvLOeo7eO1GfUsabTUbMMvh3AE1UvHgCu8VJiRrMdmPln -BM2xnwCYec6wYJ46fHukTgv+286nSQcV0XT6a+qM5GMgV5DMHW2vSyl6kTszJ3EP -xvQBfPCItA== -=Gkxz +hF4DHXHP849rSK8SAQdA3xy46BwN9R4tHbyAqwTMeOSfrrNzqsvCT9hqcVfRIzQw +ThEINU6n9x5QgU2L/mtSaAMkw7ikOmzrJkvjEEE913dvUsw80+Q3QDYODETYzXBN +0qIBrEpD0pYlXiQECDAYqox9JBkOPi3K6c4TLdACG2q7oOtHzbApzHC636oOFAXB +uhARzvr/TN5/Fr4KDM2j4LqdsxSwE2mOJn4lM8EfPAm0jxbDmHKOpjpk/QnKR7ry +BbctXwYIwlkp6voRsAJ/zG3XcLpbO/w+a5U14P6qCp2RM+bNWKtZTT4Gl0yCBk3A +CYGvWb797ZA+5BhraXCWAQcscec= +=WKBy -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/protected-with-legacy-display.eml b/test/corpora/protected-headers/protected-with-legacy-display.eml index 8c5dd251..9a3c7644 100644 --- a/test/corpora/protected-headers/protected-with-legacy-display.eml +++ b/test/corpora/protected-headers/protected-with-legacy-display.eml @@ -17,24 +17,26 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBBACkgwtKOAP+UlKYDzYkZY+gDuMNKnHjWIvv2Cdnovy40QzL -5sbuib40y7orO+MqYMCWpoFtgBVsGiOUE3bZAg8n3Ji38/zVGwQveu6sh7SAy0Q9 -zFEvLhtajw17nPe+QH2UmIyfVikA57Mot13THq4i6C4ozVCyhyIltx+sNJkmw9Lp -AdQd+cgCMRSMbi++eRwIi4zgxKrfAoGOmdMiVzBrh3yZqnbI0rCxJIKu7gEWuQLT -7BuvN2bJUkPGLAUhUanFararVoD7WWOl67IlWFkyncES0PRskUf9coV68WZnYjsR -Y3LdLnha1sdMwUNeBKQ44XBd2e7mXbDSp1cSjTDf9euwB4m7uQFTLwoQ8Of+LmQD -KMHzjmucbkNAIpfAjcDusTA/oaaqUiEgGIgYYMDqG1CaaxdT55S7tMjW5yJryQmo -pg65jrUMgEn5XHZ+KI2OsCmwGdoBYNau8p1a2hsiKhHJmLUeEAu34gFI3hylIOC0 -0KC40d0zTSb0s7SZuTrD6vYgiXG9aFktHvAWFH0ATCts7qyiRN7k5jt7yWfRntE2 -UCexTGE3TH7aju+IqDPC1XsaKF4T3CVhdr8WmKCa+0VOaw7xHRGYnzq9y91GcaCx -8AcoZ3kYs+f2LIn+T667A0KKP4Z6OmLjCx3b1RvRUQYR9taruEMAQbIuAajiyTe9 -KfUrsUULZfInE50x+OneYvDhzoSgSJoHIK+18X/wo6YcyleJ9fZxCQ/vaXTDkAeF -ve7TFcbIqmJ4MHygXILHUuDwp7P4t/tIL7SZwja70P3digjsgoNZY29VTnU8uyIb -d6eOjgpeNVhRjDWxbUvhFD7i4rHCi/bbXFlW0cCXoiaVQBtYmiNysRoRZOv0h3TW -q/+/UmqkaQFnF3zp5sr87y+ValItgPWmb9Ds0lyAoSvQx35zVh8DFfH04m7hmsb7 -gcvemlPTAnQWkIMC3c/bZWgt8tNcG7tQeUMWd9n4281y/hApbm90x2NLzEqvVcRq -K0iIgVxbCHSKqGh4TtbIwpNhzSP+KHYkZ8h6+QUDRwGEV9QqZKg= -=2O0V +hF4DHXHP849rSK8SAQdAGiLZx0/sI1uoQ27OPpwunlzzlY7ba4E2YNU+AErRA1gw +YYKlhgfzSgO0LpbxHJfithVAkrGYvj5HNWGxriXEsOyfy+Ax0FtAJuzYr6DGKGx1 +0ukBxLoRfWuoo2C9gXwojieSI9lTh7V+FaVUoVmTXhd/WFjWLtHbkIVmvXI1WVyi +wM5NfDi3Ho995P+DMZxTKkdqtbdeYgUK7oCw3FHsYsNSTf3XFazjEvHg/TvOWd5u +m9SbZAKfOBrEGGCzNHfgyluaBKdzVX04GPfZ2GblHlnU9AnGSWys1i8kxmuQYAKp +UrXDlMcVB4V5bcGd1KMvGyhnKzICXWhXiJhEiNOc+uhl97jxXRujflfT6S1+8thv +o86332XixGu8o5svVAEWobN8LUXpHZZlnK9a0Zftf4v8ATHEzQLAa5vdx+BYN8ua +e4dmCtxA4XCfRD58FJ6EwjDqhv45KYnJP2W5eZujQ7Pl1m3HJXGwFQmtnOSB/9dw +M+y1Aif07VBYE3LmUUqmS0HLZoqmOEoh6rKldzyxFmtfZyn73n/zcUoQblEWTE7z +lxIqpCmo8jHPcs1tm9QD3sUmqQ/YXwmqZbD3pOn0PIXZKVY8/DaeggMWKQ/UhCWa +7Z8g2GVq17AjHsS9n3ShDhf6B/8qI+jjaZQqH0W6KLmDQixjf1BoPnTrXNjcloJk +uf0YAuol05fXNAiyPbFNO9zoFPxm8ZVEZG9nbcnNOz7ac/Aea6hqhxHnzNFPU09K +J92FZ08XXDlrt0jw11Z/i606U/7kX6Zy4vCtZjGB4h04msBiLQwI0POIcY28SJ0U +W1AqcReye6lQTz47AkOKAfVQl9hQP++G7nZXlxUQ+z0VRqBEqd/QJdHgoe6X4ctd +r8093odiz6/DXJNwDTHPkaV5IseghzSLYyjmbLR5DUjnfuxKw5zpG+mK3X2PDx1B +LtUNfBGmnLN3jBa8Q/i2WYxYpAuMZzJcCcocxW0H+yBf8+rZNpIvi/RsTklKkaap +EOgP9sZXlgJePUbBmdd4Wwx7WTsjna6ckNp/9WE8CuDy6x9Zkc97Rkd+Oxc/KKtF +1mQ/VdRZj3trlABnHmF0H/H4Qlrt//P/PCl3qRZpE5v34OHDlTT6UjLh5ahWZ3hf +pj3cSKy9uajnWPFf9tnI0/9cWYbllaCMhIMbDZXRM3F4H03bi2k= +=VKCz -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/signed-protected-header.eml b/test/corpora/protected-headers/signed-protected-header.eml index c3a21b85..f5efaff0 100644 --- a/test/corpora/protected-headers/signed-protected-header.eml +++ b/test/corpora/protected-headers/signed-protected-header.eml @@ -6,7 +6,7 @@ Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; protocol="application/pgp-signature"; - micalg=pgp-sha512 + micalg=pgp-sha256 --=-=-= Content-Type: text/plain; protected-headers="v1" @@ -20,10 +20,9 @@ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- -iLMEAQEKAB0WIQRa6rEfXjPc6HXdt1ttkmEtlORjgQUCWusAfwAKCRBtkmEtlORj -geIJA/0WcyxlwDfXRMbiGE/crLBYhLpXK6ZMzjEn6HQDntMIk3Kr61rAwL8edKGx -gbxr1+XlMYRt+PJDhi8iI0odDI1YjiBjjc0bXUoDn60UcjL2MPGshI3426CA7cqB -cMaoRHajfdxYjSzzfh8duVgi0vmUnsyoePBhANRbDIVmCQS11g== -=c4cq +iHUEARYIAB0WIQSaOv5sYAZaFI/UtYp+ar6SRkXMYAUCYxiQlwAKCRB+ar6SRkXM +YIm6AP0UlyfUbhd7bG4Azs0rby3qPUXOC1DtbSpQegSuR7nGgAEAub3WeYgEVVOS +fsnuNE9Q/LnPTS5m85eMa1s1bS8fcAE= +=O+fm -----END PGP SIGNATURE----- --=-=-=-- diff --git a/test/corpora/protected-headers/simple-signed-mail.eml b/test/corpora/protected-headers/simple-signed-mail.eml index ebf4b786..cd1e9fc3 100644 --- a/test/corpora/protected-headers/simple-signed-mail.eml +++ b/test/corpora/protected-headers/simple-signed-mail.eml @@ -6,7 +6,7 @@ Message-ID: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; protocol="application/pgp-signature"; - micalg=pgp-sha512 + micalg=pgp-sha256 --=-=-= Content-Type: text/plain @@ -19,10 +19,9 @@ Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- -iLMEAQEKAB0WIQRa6rEfXjPc6HXdt1ttkmEtlORjgQUCWu718wAKCRBtkmEtlORj -gUXaA/4/m6CPRgC9JODRKRWo3Szi5D3zg7uf29DIJu9m2vVRw5o0ZeHcxLb26UPe -qdjPq6GBclkXdeTH9Nv2TW5cToJmMA9UvESeRRzbe6ytvswNEYdSbiYAsv/k9t6K -KQO2ZSbsbVlkh8xVYC3ORiUS775YrPxVT6QlPkMKAXw3l3Zwcg== -=jnDO +iHUEARYIAB0WIQSaOv5sYAZaFI/UtYp+ar6SRkXMYAUCYxiQYgAKCRB+ar6SRkXM +YJkmAP9TEGDYF4GZcHaxWDZYf6EKHmNqu1RPYuwEN8QdVbUIxAEA7IiFYPQtKXgr +wyEYNcJ8aD1CYCGhR8pTA9oT/Vp16Qk= +=a+TS -----END PGP SIGNATURE----- --=-=-=-- diff --git a/test/corpora/protected-headers/smime-enc+legacy-disp.eml b/test/corpora/protected-headers/smime-enc+legacy-disp.eml new file mode 100644 index 00000000..6f5c9417 --- /dev/null +++ b/test/corpora/protected-headers/smime-enc+legacy-disp.eml @@ -0,0 +1,50 @@ +Received: from localhost (localhost [127.0.0.1]); Wed, 27 Nov 2019 + 01:27:28 -0700 (UTC-07:00) +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-mime; name="smime.p7m"; + smime-type="enveloped-data" +From: Alice Lovelace +To: Bob Babbage +Date: Wed, 27 Nov 2019 01:27:00 -0700 +Message-ID: +Subject: ... + +MIIG5QYJKoZIhvcNAQcDoIIG1jCCBtICAQAxggLCMIIBXQIBADBFMC0xKzApBgNV +BAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkCFCJT7jBtAgsf +As31ycE+Ot95phvCMA0GCSqGSIb3DQEBAQUABIIBADEhlzhFzYj6tUAdsRCrSiLl +d9cgKtlAesJ4cDY4szFWAbnwrCmEcFxjFDUOjbfQCYCG80Sxd+xntni73I7PI2rR +QLjk3w9VhLwFRyzy7qyJi2CavjKTxysX9f36+FXA+THfVQRM5ypiyYJg91X51PNX +hJj3DHrnxqKeSl/z1hdt9r+s6XAUCBSvL99BGnODWhNIZtPDzt8fMNcgarfw+D5F +IZJb6+wX30tkztHkpHHKrrDPveyfnlS/p06Gi3ekrrhBtMQMRb9PA/E+ivDPktsm +aKg0Oauw4oZSKW3f4ukYhbnndbbagNsnTfs/QFy/p+hhKTrfCd0h1N8mTzedVX0w +ggFdAgEAMEUwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1 +dGhvcml0eQIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEBBQAEggEA +FaK5QaPXJ133D2uybQt//oeDm6PkCAFW9YVOgjnLLz6FD54Dt2i1KCQu1Xlg9W3P +1zJdYXOftDgilylNfmt/muEsvbRfFtMWUq0VGirHz//BWmY2cW/ocinFO514iviL +MLE1umsXRNwVIVIk/uh7AmqXjPkRZgRgIMUbSbtmW4DDja+ZM0vmqFQ1iUIlApth +FpjFfPDHHD8isLTbGi2iK6dEN3DIJFGbg5o3nK6yAhVZ7x3LfFNSNVDDSY5mPFG9 +Vm6uRgEE3Y5P6DbXXo6MHTgg0XY2f4y6MEWhOg37NT9aFAfzBBxJ1oSBWpOOfZnV +K1DvAwPaemSRz9oWDcBM8DCCBAUGCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIsFkN +8DEx8muAggPgWGF2WsPq3/a9jUa5GA0YFPiINuETCGTNaEXiVxnT0h0CF+EhZ0T2 +HFCiZEM0dzO05zt9WdVvAREaCSh7ZWG9D9wJF9x+tqQbzMuJ2AdKuoOH73kClvkx +pHxANLhkY7hzIqRb/eLG5D7Xh8iCDiFecXDh7EHqD/R+sfLN9aHKOcKyY36kesBQ +R8aHZbbFnnD+oXSDNIPcntGG3BSGMxsWuOp+rpTKeIHWFIungDNKsLIy3kWleENw +FVIcjUF6QhI1HYW6BeXuVq40GV2OOkmB24rYEW1Jg0hAtY+5rn2mRoyxvUC87bjQ +hLu6xgPmhun9J324eM5aYVwkmVBnRW9hyxClZ7Sv0zlL7lGQ0VQG+zWHeJ+h/M2j +mQpLgAUEGxxNCm5ASHuXPIN6pSvrOVplrT8kKLPpmMYEwmTX2/rBO4P8I8uNrqYD +AyX8p0/l2ArczkWzGTz2luBahrD+cTZPApe5SeyXOxWBl1Lmb0G8o4twBeeBLiHP +XwYvttx0JYG/hc/lmMpEemJqwj9uZ3wGD03dIhhDX2Oj4ek/7jT6yqJh8C1H+PqA ++HNfNXsFQDrRORoqJS8YVEiYRDQNyePy2ugzLTh88nPtJp92hY7bk9zl3AYaiVFH ++szlLoyzfM9D+geZemR8XfI2ijGnrWMlnyPah/zA6J6RwemhuiMklZGYG85hMU9H +K4CFVM+m7xYxKpwFVnmkVZjzWInirJhehElhtCXpx/IFGxH9CPbCyEZV1WVStrl/ +0fWTGicMXez6hVQCadWCXy96/eLIXOrC54gSoIJX2TD6jdVEu1YptutyGI6KdQ2p +yXwhs98Uj7DM3nmFeAcjjN3e8pPoX7aG8eP+MfmHlWN6jA44jMaJmIdp9J20g74J +MdjvnHa/cGibW/RamPiFObN0F94A83vcpUfU/zZ8cFHi/3/lN6Rm9+3/giGRZa9E +Y6e2/CEq1cUbPQ09fPwRJmjZCfDce71DKe+ZFGdYtFR7JwDEeZ6BB4Ff4rXctcWD +PgUJqUGv/SXBcFn4cNUK9MYYqVu1ovd/T7FMf+i3c5MH6BRCvft/i5aeBR+A26Gk +2awtBPYdHW6+AslrFjncBbtPDlU6vX9AWuC0k0MQYnNkTWS8gTvsriXJZ6Zu5iFE +ExNuFz7YcnMKnguOn2ph5azzeMm83AYzWXzZPu3mdr5Siuu/Ke38oADKP+BZ08Za +XVvKvvfnRPXO9kG9hgvEMRU9KOcxn82XoGPNZib+9SPa2zYx5P6HX1Bqe/cmKAen +FKEiJLSTP2/pc6AWAICqJl978HaUHfMFiN7jEUppAifpAWqNcIGSW5w= + diff --git a/test/corpora/protected-headers/smime-multipart-signed.eml b/test/corpora/protected-headers/smime-multipart-signed.eml new file mode 100644 index 00000000..f05d2d98 --- /dev/null +++ b/test/corpora/protected-headers/smime-multipart-signed.eml @@ -0,0 +1,68 @@ +Received: from localhost (localhost [127.0.0.1]); Tue, 26 Nov 2019 + 20:03:17 -0400 (UTC-04:00) +MIME-Version: 1.0 +Content-Type: multipart/signed; boundary="179"; + protocol="application/pkcs7-signature"; micalg="sha-256" +From: Alice Lovelace +To: Bob Babbage +Date: Tue, 26 Nov 2019 20:03:00 -0400 +Subject: The FooCorp contract +Message-ID: + +--179 +Content-Type: text/plain; charset="us-ascii"; protected-headers="v1" +From: Alice Lovelace +To: Bob Babbage +Date: Tue, 26 Nov 2019 20:03:00 -0400 +Subject: The FooCorp contract +Message-ID: + +Bob, we need to cancel this contract. + +Please start the necessary processes to make that happen today. + +(this is the 'smime-multipart-signed' message) + +Thanks, Alice +-- +Alice Lovelace +President +Example Corp + +--179 +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-signature; name="smime.p7s" + +MIIFhQYJKoZIhvcNAQcCoIIFdjCCBXICAQExDTALBglghkgBZQMEAgEwCwYJKoZI +hvcNAQcBoIIDcjCCA24wggJWoAMCAQICFGeCtFlzUkvB9HFHGWrw/RGKqkwLMA0G +CSqGSIb3DQEBDQUAMC0xKzApBgNVBAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0 +ZSBBdXRob3JpdHkwIBcNMTkxMTIwMDY1NDE4WhgPMjA1MjA5MjcwNjU0MThaMBkx +FzAVBgNVBAMTDkFsaWNlIExvdmVsYWNlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8A +MIIBCgKCAQEAw+6t+WXRtiQM8yRjWQ2fbFewCodIZUX6BY02TeZuEXoEAGEsmoON +6LlotcUTdGr39FE2K8IytOKkXVexswgAqBCqv8YjVDrI3yV82wrm5Td32TDlw7IS +igak4ZSu+UowPQs8YO3oxqImP4onZNHvdZ3it9EggmgUyZX0dmQ6z5O9yDzHpLMa +E2rXxfYcPXQwPvx4tcqbTf2htEP7PYnBa8a+sts0F7I7kD5ozGYI9dGg/XGs1lYE +WAoH5YZgNFdbkJdcKG2FPAwFcVZ/hoGm6soxkDKMrYSCtBp+fqH8MV11DP821PoO +vtSEnaF8UURbaths2yKpAB2WUJvgW5xa4QIDAQABo4GXMIGUMAwGA1UdEwEB/wQC +MAAwHgYDVR0RBBcwFYETYWxpY2VAc21pbWUuZXhhbXBsZTATBgNVHSUEDDAKBggr +BgEFBQcDBDAPBgNVHQ8BAf8EBQMDB6AAMB0GA1UdDgQWBBSsLlRapP1VGK8u6GZE +ONEl0dcAeTAfBgNVHSMEGDAWgBS3Uk1zwIg9ssN6WgzzlPf3gKJ32zANBgkqhkiG +9w0BAQ0FAAOCAQEAe+qOGM+8q1UhXKV6i63BrXSOKvd2iglxAggszUC6eMnrIem6 +6mmRzSbcGHCeU6m1MpvYSe9IiROIxjTfsgGUdZbbXtBxSmCASjOBCbphvvtoam1G +i8+LZdOgR2kDwr//TYjWO6vUfXPwerNWMx4cKpFobdmvgLYCeAZKRvoPjJmTEFfw +KO0cCxSifTpTFiwZhFxXKSCTdB6T2rE9JxJfzJqLUrvvEZwpQIt8hX8kym/vKw+1 +cbsl3rag2enVP/f4qg/0mUuzkCI8sLXd+N5gAs9wdUZRcTB0gOnUAH9m7RrpqkdC +ogKdypGEQHj6GiamJAe2WndOp4BZdBtBRzjfuzGCAdkwggHVAgEBMEUwLTErMCkG +A1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0eQIUZ4K0WXNS +S8H0cUcZavD9EYqqTAswCwYJYIZIAWUDBAIBoGkwGAYJKoZIhvcNAQkDMQsGCSqG +SIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTkxMTI3MDAwMzAwWjAvBgkqhkiG9w0B +CQQxIgQgGeoQw8WDmjB606EKGR5n1oMuV7Te1VjfA2oB2ebW390wDQYJKoZIhvcN +AQEBBQAEggEABblYEWSnYyzL3jTS3AoPr93YKksIZr5q/b8Y5/1rMxdYxPm+iReO +RHRgpbFQeiqZXzRXtMohfoIkh7RmdQoSV4OpwiUmNU+f0ZEAu8cMVJM6gdyUD+1D +JwDNr+YNLV/1UUGhqx0FExOa/4O92KYBD4eRQw4KDWrkfh9dlSj0Bsl4thrZYGLz +e7ut3FN5TBruZfmqMy50xZ9yUW91YyQUBLiIcuF185y5ZW/aQCxBKBbrNNGXLJbo +8yKFJqSPiWZvwUmVQvfgL182hg823OJTtP4VImcUakTF0+k+BM//qqKXYrlX/tZn +QzG+4ZH/XM1vgHl7ShjHS6TSOHz2ODqD6Q== + +--179-- + diff --git a/test/corpora/protected-headers/smime-onepart-signed.eml b/test/corpora/protected-headers/smime-onepart-signed.eml new file mode 100644 index 00000000..028e02c8 --- /dev/null +++ b/test/corpora/protected-headers/smime-onepart-signed.eml @@ -0,0 +1,54 @@ +Received: from localhost (localhost [127.0.0.1]); Tue, 26 Nov 2019 + 20:06:17 -0400 (UTC-04:00) +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-mime; name="smime.p7m"; + smime-type="signed-data" +MIME-Version: 1.0 +From: Alice Lovelace +To: Bob Babbage +Date: Tue, 26 Nov 2019 20:06:00 -0400 +Subject: The FooCorp contract +Message-ID: + +MIIHhQYJKoZIhvcNAQcCoIIHdjCCB3ICAQExDTALBglghkgBZQMEAgEwggIJBgkq +hkiG9w0BBwGgggH6BIIB9kNvbnRlbnQtVHlwZTogdGV4dC9wbGFpbjsgY2hhcnNl +dD0idXMtYXNjaWkiOyBwcm90ZWN0ZWQtaGVhZGVycz0idjEiDQpGcm9tOiBBbGlj +ZSBMb3ZlbGFjZSA8YWxpY2VAc21pbWUuZXhhbXBsZT4NClRvOiBCb2IgQmFiYmFn +ZSA8Ym9iQHNtaW1lLmV4YW1wbGU+DQpEYXRlOiBUdWUsIDI2IE5vdiAyMDE5IDIw +OjA2OjAwIC0wNDAwDQpTdWJqZWN0OiBUaGUgRm9vQ29ycCBjb250cmFjdA0KTWVz +c2FnZS1JRDogPHNtaW1lLW9uZXBhcnQtc2lnbmVkQHByb3RlY3RlZC1oZWFkZXJz +LmV4YW1wbGU+DQoNCkJvYiwgd2UgbmVlZCB0byBjYW5jZWwgdGhpcyBjb250cmFj +dC4NCg0KUGxlYXNlIHN0YXJ0IHRoZSBuZWNlc3NhcnkgcHJvY2Vzc2VzIHRvIG1h +a2UgdGhhdCBoYXBwZW4gdG9kYXkuDQoNCih0aGlzIGlzIHRoZSAnc21pbWUtb25l +cGFydC1zaWduZWQnIG1lc3NhZ2UpDQoNClRoYW5rcywgQWxpY2UNCi0tIA0KQWxp +Y2UgTG92ZWxhY2UNClByZXNpZGVudA0KRXhhbXBsZSBDb3JwDQqgggNyMIIDbjCC +AlagAwIBAgIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQENBQAwLTEr +MCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0eTAgFw0x +OTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowGTEXMBUGA1UEAxMOQWxpY2Ug +TG92ZWxhY2UwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDD7q35ZdG2 +JAzzJGNZDZ9sV7AKh0hlRfoFjTZN5m4RegQAYSyag43ouWi1xRN0avf0UTYrwjK0 +4qRdV7GzCACoEKq/xiNUOsjfJXzbCublN3fZMOXDshKKBqThlK75SjA9Czxg7ejG +oiY/iidk0e91neK30SCCaBTJlfR2ZDrPk73IPMeksxoTatfF9hw9dDA+/Hi1yptN +/aG0Q/s9icFrxr6y2zQXsjuQPmjMZgj10aD9cazWVgRYCgflhmA0V1uQl1wobYU8 +DAVxVn+GgabqyjGQMoythIK0Gn5+ofwxXXUM/zbU+g6+1ISdoXxRRFtq2GzbIqkA +HZZQm+BbnFrhAgMBAAGjgZcwgZQwDAYDVR0TAQH/BAIwADAeBgNVHREEFzAVgRNh +bGljZUBzbWltZS5leGFtcGxlMBMGA1UdJQQMMAoGCCsGAQUFBwMEMA8GA1UdDwEB +/wQFAwMHoAAwHQYDVR0OBBYEFKwuVFqk/VUYry7oZkQ40SXR1wB5MB8GA1UdIwQY +MBaAFLdSTXPAiD2yw3paDPOU9/eAonfbMA0GCSqGSIb3DQEBDQUAA4IBAQB76o4Y +z7yrVSFcpXqLrcGtdI4q93aKCXECCCzNQLp4yesh6brqaZHNJtwYcJ5TqbUym9hJ +70iJE4jGNN+yAZR1ltte0HFKYIBKM4EJumG++2hqbUaLz4tl06BHaQPCv/9NiNY7 +q9R9c/B6s1YzHhwqkWht2a+AtgJ4BkpG+g+MmZMQV/Ao7RwLFKJ9OlMWLBmEXFcp +IJN0HpPasT0nEl/MmotSu+8RnClAi3yFfyTKb+8rD7VxuyXetqDZ6dU/9/iqD/SZ +S7OQIjywtd343mACz3B1RlFxMHSA6dQAf2btGumqR0KiAp3KkYRAePoaJqYkB7Za +d06ngFl0G0FHON+7MYIB2TCCAdUCAQEwRTAtMSswKQYDVQQDEyJTYW1wbGUgTEFN +UFMgQ2VydGlmaWNhdGUgQXV0aG9yaXR5AhRngrRZc1JLwfRxRxlq8P0RiqpMCzAL +BglghkgBZQMEAgGgaTAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3 +DQEJBTEPFw0xOTExMjcwMDA2MDBaMC8GCSqGSIb3DQEJBDEiBCAKDM98nuDl98sK +i4SDvP2xlxr2SdV/xNVYs6SeGCBRuTANBgkqhkiG9w0BAQEFAASCAQAcryWkSIbG +rrc/aDF1Z4KRnoRpr+fOutQSLV7k0Tgezt+X/kJCIiuLvjUxLrTux1yUWCKUPb6T +KLYASPJpwDXrNzqmGs1pJmWHTZwUhbFVXt16FaQZkDSATtvhQU39Rsot2j1pP/UV +J7+5FPQwNc4dt7MFW7jU4TBHo2VrzjZ2K8ioELPxsixOCAp3ytkhf1Umw6bC5M/u +oWjsa6xzAl4fw5+pxZw0JdbrYn5kmPiekSsYy2/+yOwzrtIYtHW5dY7DoWWXDXtD +cmCGHkO8qry+MnMy3PwvXiX0warQo1fnhXB5tlk2K9YdiDcOtnAshEBXAudnxlPK +JGzeJVUfbfM0 + diff --git a/test/corpora/protected-headers/smime-sign+enc+legacy-disp.eml b/test/corpora/protected-headers/smime-sign+enc+legacy-disp.eml new file mode 100644 index 00000000..7ec33968 --- /dev/null +++ b/test/corpora/protected-headers/smime-sign+enc+legacy-disp.eml @@ -0,0 +1,102 @@ +Received: from localhost (localhost [127.0.0.1]); Wed, 27 Nov 2019 + 01:24:28 -0700 (UTC-07:00) +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-mime; name="smime.p7m"; + smime-type="enveloped-data" +From: Alice Lovelace +To: Bob Babbage +Date: Wed, 27 Nov 2019 01:24:00 -0700 +Message-ID: +Subject: ... + +MIIQjQYJKoZIhvcNAQcDoIIQfjCCEHoCAQAxggLCMIIBXQIBADBFMC0xKzApBgNV +BAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkCFCJT7jBtAgsf +As31ycE+Ot95phvCMA0GCSqGSIb3DQEBAQUABIIBAFbDR6j4ZB/Mo9BQygYItwFc +P+4rO4d1ak51hc1DpSqyhiMcGahA3yxDRbZ4W1rbmC/s3d5+OWXKYgs1nNMQJ48F +f45BtNTNslPZ1+NZVbkoVJO8Bxv1rjB8/qWuSUsroqzn9enS8DUBxxPL5aSWKQQN +G2IaH9BUkMXLPUYA46GATly94IS4fZqwBtNNBP5eiIIPc9Ogjy+7At5GG7rVMN0M +G5FL0oq52SYUe1167jp378JI+2dkA1q5+Cru/ZE2Rdw3DrMDAFO5GwC7fWKg4zPm +IHZj92caVj1IyfTmGogT2o5tLMqn61BkptqxZwHDr3FI/aYo4vcHgmlKR/TdbHww +ggFdAgEAMEUwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1 +dGhvcml0eQIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEBBQAEggEA +hXeYVSUsT1EBZ/+AjwyEcnlM0kuFMaNvGlBMhAZzAsy012rrZTWbqWkcA3abgm/M +CuZX7mQL0I79KZdmClGpLx6gQFjLemHaClQV0ZNdX4DxakWuME/kCMqbo4MZXStT +a0MHlKUdoMt72Rz4YBzNQCL7ePaii5w6Nd2KD7yJAirLYUMJEjVweVaMI9y9LmbO +vb0g0iuoUe0vp9B20LRcIX37nN5D1GG4tHLPjBD43gC8iqxZQf0uah2cWD1mAG5R +oBgIDKXPy2eVbcMdSaOirDKYZ49WFe9Lad9q3mHHbFs6K6/yuBm/thMEdCJKZTHo +jiPvYdYF8IJfEd368I+DujCCDa0GCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIsb1a +JX/RU9aAgg2I0VXWfs5fc/Yad2qvawUVNX+LObjA6/+t9WxuV2emOeBYzQGjo7q+ +xaIXQwbbF1ej27efGhxUYDwBNS56c0uI0Ta7jxv5OFZhzQGLRzoFp0bbZ+uVC4eP +bFHarRQiPzlg900XASO0RW+UOtqN5raZ3Ry2lKwXxuStZ0pX666Rz4c8PrmMb4/B +aQYn6iKcT6fDU2TpSbWY9iph6kZczSeewK+pIj9nXfjDKXScs8D2Raezev2ciq/V +ZRpRH8JxieimI2yeBmEzTCq11TDYycDfMHB6reGaiCGX//8kAWtskzRyNlV61unY +ZKSNhVKLwKmCQh1V1Nd3oLApT41EeM2oWedUqNBYqB+XGCD4DUYdm1e+4h73d4dn +JTkCdadxEn+9RRvZ4YMlw3mvT997Dy3rTXT29dj14TstZZf2O63pY0TpYy0HZy6Z +Jug1qoe/vdcJ9SPOSfJE6VWCeVjxB+eGgheFLKqzK8Hs/Bm0/wDKpSFgEpOPnkJ4 +HJ2Uzgn1Emo6gBDJt+qn3s2UnowcMsTgellhKvgzVq59LTyRyWL5U8XMBsXT4qjm +0LkRvDkOIjMQH7kqvWbpPlnWpLKo/VVoxifldEegWAqFVrP7f5Y+nNQttAYV79uk +MXvR+5YFkvmQAerfllPqXBJdbB65ovikSVsy/kAboGpRG1oAZ4ODdwdGyiGIzyyc +lE0x/8+gY8BqWzRtWX4GySKyZ50/+xkJe5ss0IXPCgq/09bdihsRn57v4V4SpdDO +k3g/Dce+LzCRL8uTbUhrhZnjKSjRc3fFaD/BpLYjEDbnGF0ICslN3vb2xWUK1u4M +uUH9r7lH/DCb0+TxIBtxOnP7W02bz8gGJAxEVEqk6pjxxOYqfS9/uBrrAY8P21Y9 +PFLdeHzEdYemq3il+4S7OU3uNUuAYijxmCRs7JQxZ9puA0iaTME9gK1yikzsLtVZ +f+9osk2nYgfXvlL0AiYabd5cU2GNW33TkdDMNBsB7lx77J9erVLZpPKNo4vgHA7b +owrDaYe0AgcZm79fvmR0RdtIZI91MouEhkdhaPiXmypmszjR/M0Ot3Y+oU/ks+yV +Sle0S0h4V8wJRJYG/9VVurm8012ke2U3EGFlVnSv/IYtpssC+U4McRCmakKCrGU7 +OhL5JKBQN/DFTu4pV39IQlLLhg3wzA2FSkyIL5gEbS6sP9GTPo5LlNm2nYfJQX9A +sHKSrfh68dvjSNExxi/8hdmFnnRwbAnUCI/WObGOkKdheOfdQ1AAHtLO7G65X1Cx +RctbAJWa93M+iRUN6qnB+vIbPPnI1Mc7i6mPYzgtPrM9bYqEZz69pQtHcGTfxOrU +tm+/h36CRzJBfXodBZbwQ9mZAzfkKdlArlZYIeBUw3ORQnQ7UlJgG8KsZpUhTxCc +gvMoExtlvkXcYLRUBFfZWyOi6FePzQjuCK1w58OdweJgXprEAWSvyxhmVdg4jUpX +MYKE0tZI9xwujyWjACO0myYqTdmsqyds+BgfBn96XiA9OFUH2C0/GAomhNs8uPSO +T3Gt7Ld/FByxEVrtl9A37X6bAwZO01j5tHmdXFPmMVep0R8zsWtPn3RyGAjcgcq6 +50wJRwhvofdI7wilZ0KUBsAaPj3MK52cRyD19VXKNNwt2bLDV6gcWQ8+QEMusxfp +1Dc9N9DSs+w3lGsFfpoeQ53/fXcVNJm6Bv89bH9anLGYdCdRGvZsvw+xRuglykqb +xLtL2lB6wzlRFREJoWTzCVsdpIZ8znPmk1cB0wDlbMeu6sddHmv+6fpyuvQfQmdj +D8WLRTuyxax94TmBlhJCFYxmO/y4Ivlx5C60GIRTkHpBYL/M0RjrbIszXEqcogzU +bdwjLIhdEnpJ5vy0uXwhltce8BDpenmHE7y1kHvPBiUG3vB7AIXqhohFsJU3AYUj +d1TvFKS2AsizUTLuq0Ydbnz3AxMfmnZe8qYkNu2zRygL2xTa58f/MwsHKakk3OmS +9JFZLrkkVWZKXoARctuahYtWBAsykaWVNnB6zGcdX1MGVccl930Z6QWHyydtZpQc +ivNdEGdGv9B0K7/ngNdVgD5Wd29AMMFnS8+55mLfRZDCjUmshSySaf6Ein4HD9Hr +vk6dJvBPjnI5UjeUPjmH+wcZKIjLHW/aV/6/zoxzBh61rWFlr/daec+CFZE/+epr +LRRYSmv8oY47fF4duDDhoexcvP/CH+A2Hr40OfciL4vKy3nuUDCNa59xO9JWv4NL +n3MQypC9bcaVPkXa7TK3ECq1Jgv8gwfdh5/ovG5OdZA4uIcO+aqcskt/PD252c63 +0Znww3RXXf46KT4GdKO5A377ixkUMkznnCMvottmkPxjnhQjAsQg3bJeQk8EoX8f +Pq0If4i7SRBSDtb2OH1pPmk0RVPtxlRDTVj3vS3Lci4xADFgC09n9nIvPO/55aau +O6StbJtLmpubS5giuDH3uftwuyRiLqm3gtbSKPdoTk+dJhHXbbpBknL4XYTPxSsR +IIaRds6w30vf7/IscyunMcquJlsO929SSa93UevKEIZbqbV9oGIqwkiUMdVZK09g +rW0F//Ts4a5nYdEQth/fq3JnwqeHvvUfKdasK4TtrTnUBX7qZk/K3Y1fZwjKdd/8 +t9t1z7Kb2d9hWwtY7xP8liDluVFTsq8NM54ZC2218X5ViWz1yFmF2LXvRixsmYJv +Tz8lUUnC2B/Etm1kkU4zrYK0/L77EikKVl+B7BXfEqx6ow41j7e1YZYaqmZ9mph+ +UieSdzqVYxhPwT25DrkU3r74iS28gKsbFhUaNklaFOO5iDWsKgBXT+wdZqlYQ6Fo +oPe66025iJMwK8t+d53jEduHezHO2sTMAuf2hpdaZo7+rP/hRTReAR6CmI7nkWhP +z5Kno9S+XhiSP+WTSpsoA4ubx0T94mL8NOVvSZA76TZ3ObVAP5VI/bwv6Grighor +Kpsjt7dhSJRv+RHv95sAWBeW1Fgv8XOPSAZOmpJV2qc3x3Qmj0MXIR+7+3GlUr8+ +Dit3CE1hwtxgOW0tc8kuBTfQD+wNSa9r0eUyFscEBBljpEVbLjgjVdNv4Hc+fsbT +g1JzZuUIDQZoEO2xLjxD+I7vLZKQa0J1JeZ7O+NqmSxsvSnwCWtJEWNMMxYNfwsP +rdj1zPLqn3rzSBqhroNbaDGn86BTwIqfhr+AKbvevxS6bI8IbyKm9u3BFr9cuawx +Sp1QM3NtqNStV67qR4A6U/ZyPUJdO1bxo8F3oRmJqOt7Jc93rFgkhBJ2+eMtrA75 +Om5tB9LBVSl5U5yLP0COO1QE5pqk5yuhJLT9Dyss8bWDRbSWKj83e4YXhPnq71Bm +001czylLVNUlDc69Tf7FXjtIxh2yjvOT3zeLBPXOjU0it+gAma4vgrh8/mMXnNiq +OLsVow8aKqm+Ofd6m13K5riDFgXgNI9lbvPKUSWlEqDMEqXk1oAqD4Nb5NTGSFpQ +Q4G+cHAxJCu7vcXBaZnP8uMP5IAkdg5jIPvvMRwg/aqkl/KbL98oYZ5+1xrOMuKA +LT1uCJ4MMB0lWsa1He4jPe8LneSupw7vAXlbo2VzcOI6oCSY5hV+cGQRY+LjW81q +Cu5nLq8bwgnZMSlPmwr0YrKmvh8YKyGOrmTadxykC5IC+XbrLDsw2Jd9mLIjUQ/V +4ibjeb+e0QGob22WOplCLnHGW/SnYei8KG1dxs/ahS+8vQdrI880ZJx2QJnrz0Ej +ux6tKv4mvUkqYA5hlTFeT3PTr54yA+YLcCLMfBDx4ykPQnYUBj7ONHuNSUYt1CJy +faZ7cWAbhgH+wlTFdVBVeW5D4FRbM8dMTPXyfC5ygwTJOiDu3vQKyyDkmiX7sEaC +P1JN2V55uacyR8ZAG5+Mlc4ZMx83kAIZZXTCdqa1EX8yda31FI2rDHmvW/82bmjL +pvI4Nnn9+zzJtDVCJ0B2VAZ3Edov5GzPikm3un4+mvyhUZpH4sbT0+VhPCsr1+zn +bDJyNw4AswxaaJKh2+7wBiU6h+9TP/lI8SAJHtZL7zHBH8tD10ptksLRWDs9vYqp +/3T86S2vxJL5DvLFJSAZrYOE3InS+keGmTMCdAl9I8zIworC/8uQp0N8ESebEVjA +aHotBk59lj/OW4JZ3tQkcdQWkpnUfW/x9xE2wthacHlRzYDDsFByjEqkQr0MU8VF +EGij9RCC97zyFrhv0xJm1C6wX0pcuEcuPTNBf38WyBTIfmVHHz/I5YKk5cdWG7Hq +fmccV5GKrs2BseR683HM+/u50sq0km9UrqjgFR1DjfDoRKp0guP9PqkJAnwG2nv1 +hmNtXumzkF0otP5LDKLJ84MGP8Wnb006iEdD48Lra+clRAIIuLX4A0wRQjViDp7n +OByI6ZcQd4DTMHnFPRvMkNMLYn13LghD6P9TTjQZ0KCOCwmc2TMCIhJlvzOYX6Cc +wJZYLO1ltgfnHEuh8ijv0u3d/BUpsknYKBSJGUyMEZ9iUtbFPVfXBGSTi3gcWHtl +IrM7wjswJwHWSvZKWUs+YWWJTwj0apG6ViGllwOAqR9C48uLKgFWPbMoTpolnp69 +eiij5ZHxB0i7SI80D+r65b+fqaFzVIJXVEI0zu/mIilbYBnGkhLI/Naw1m2e1qVJ +mi1JBjXLAT3pEJDh8b3Lpgw= + diff --git a/test/corpora/protected-headers/smime-sign+enc.eml b/test/corpora/protected-headers/smime-sign+enc.eml new file mode 100644 index 00000000..aa09d19f --- /dev/null +++ b/test/corpora/protected-headers/smime-sign+enc.eml @@ -0,0 +1,95 @@ +Received: from localhost (localhost [127.0.0.1]); Wed, 27 Nov 2019 + 01:15:28 -0700 (UTC-07:00) +MIME-Version: 1.0 +Content-Transfer-Encoding: base64 +Content-Type: application/pkcs7-mime; name="smime.p7m"; + smime-type="enveloped-data" +From: Alice Lovelace +To: Bob Babbage +Date: Wed, 27 Nov 2019 01:15:00 -0700 +Message-ID: +Subject: ... + +MIIPVQYJKoZIhvcNAQcDoIIPRjCCD0ICAQAxggLCMIIBXQIBADBFMC0xKzApBgNV +BAMTIlNhbXBsZSBMQU1QUyBDZXJ0aWZpY2F0ZSBBdXRob3JpdHkCFCJT7jBtAgsf +As31ycE+Ot95phvCMA0GCSqGSIb3DQEBAQUABIIBAKswTlBs+STeesZIYAf7Gqsj +Za0rdUeDTSxt8RCa010EHb2lqKzHRwwPJkClLm6Glb09nYnQiFrEl6jbWTG3hMRD +OSt9kyqeg+MxXr2g4LoXAT+8hg/qBoF//tX+bzxhx0gx8wjxBc3bvp4esCJro7Aq +tx56BtVsIO6TA0NT0CaOcnMhIo09raR6JQX+DoPynKeXihny6TFDP7eopCgorCfR +o59O3ZMvaui6Q9KixZy3Yae8fa0ZdJu3FahIZTPdBHzbmirLxcYgp+cbTpW+Yno2 +X5GJ8eq8Y0qcc/8r6Xd3REarUxO2YbO2D6cgDj+aNnnsoG1/9psaYl8W1MSc2/Qw +ggFdAgEAMEUwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1 +dGhvcml0eQIUZ4K0WXNSS8H0cUcZavD9EYqqTAswDQYJKoZIhvcNAQEBBQAEggEA +RHhTarDqNLzXSaBokp2L3EwDv11KiGtMSMUQuPelNoC2nNYU1yzAF4jd+1UUo4Uu +quiHg5Hn44a9MejrVmQRLd5IEJiZGD8m5JguuOjn0ooyA6EEWUpMn6hOAKlaCiXd +kwTivKfhQFJe9Eb6TKqtvT2IEu3kXFfJKi+VyQw49+RXBmajDKJoHtumMJs8k4Ll +kJah+wD+snwHg2LCiJeSVHmpf4RvSiIJSvk206IeTxN3JecNbBpKLtIoy/CjWEZv +G3Pj/zkBbb+XhHbXo+Zk/e3aLToVG/cldx6Ti8zArOYNAzgt1G7dmJ3mnNPitEwN +O4qIozhT2Qn8P95AEV5PsDCCDHUGCSqGSIb3DQEHATAUBggqhkiG9w0DBwQIUzdf +vwulBs+AggxQMK121v6lO7W1r96RW0rsOHzsIvGyfyRTT1UuZRxVL09BQZstI5ss +5Zv8BogoKA0mLaNBKM755joUbzF5f/jMYhkW3q0Het9/HRH0mOnCSnoT4i2yzNdi +0tj8ixPT4sgPe9FOTkke9CzoJ967kj9D8u7Ik2goojttt3ViJkv3a1qrWDMiJRIJ +gOTTA6ZaQep5L92vtCobhD+i7iaktEpmbYucXs8jjMmwyxCFxHXGD/fwDk3UDgeu +8a5f66YepZdbLKB61A3rBwJMvQubuXEIEb04tG0Fgwx3Ao2NshN+XRk/y+uhQKdC +5ZduTxk5sokA+H4nzVv0IUkAAI+8FwY5ZWFGlncKUM/wvrGHQq3R/utChFauOHxD +7vZQLM91TcQzVWdHfJGPtp+ekjRlu9UqatQgc1ogObw3PGYlJc90Gl7AZHAsYncU +jsMbdsweuFuYNHJ8lR5VMo6L4bCNMy+tQBOfYTF1el+i9S3r3SWdBP+uLiKgDQ52 +/o4shxoi+YOf9k8wRR0iDKqwzcJuABplpgA9qjsQNqBKF5t5p3l3ihH1mfh8FaPL +ab0aDC7uunY5g44qXcG9YS+j5wUFuxgYyGkVcJq3xIit9YbEy8uPxJFz4g0vNC+r +uUSsztbLyHkhv7vnCTAlmjgG9eDpW/tEC/85pLOV1HUooD05eRfkjU+1XsccX8DG +iCax2C6W3cc1SC/d3a1+27OcgvPdDcb7zuL3v6qgqbN+7GDrcQHQRFMd2vd6+xGk +NWZQMBZVHmdCcKGl9YaH0RgkGH5beTRKEV1wBafuVOwTEwl/FuZzD4oHrOaP3GLO +cLxi44her/hNxtxDc2Lw0VQcxD8A55OkCt9+u9M5/YPj41FWyH6kdh86p958gzF5 +EpwCnQDe+s7OrwFVV00DEJhqtEcxRCSSW8dS4hVEhVxQJ56liJP+VZ+LTUJBelt4 +mfSpSqxeJnmyY0nmhEbZKVbK95a1WYMJCEpk2n1g/bQGqJKRryGwbEF9WqqHuvPo +Bv/BfinoUL3Kd3g+hgSCR4mCg5EhEsCx21jEqEggzb2XMcA+knGUYxSWj322pZfW +LDh50gkL3GQSmm9fOvjdK40GwZv8HUdLXuAQ/J19PafMaDkd4jzRi37VBqdDgLY3 +u6K+oFKhG4oqQYa/er+ZGAqqldTmu8HGCsjm6kGZvSAocJg0UnLPBNI0/iB0BYGf +KJk302jy8kfAXGSiWrYDNbTuDzFMD0zsbHbM07AOOROGwKv5TxAF1EHHTxGb3IKI +jRkVBL7QdRtDH03zlxv0lnFwiuCrzLrQdUuEG/0wt8RaNr+p8hAo0YEGbB9jmbax +CSLLWeNbMOo8eIi3Mft4qmDXp3TEuHHru8kbvA36vQ8+dunSf2BcecyM6UAYBqaw +SCcxQmEcyMuyjSLVerVfMl5lwlmM+qabxHq0hpJHnCR3Vl2qX3CiRWpVlNaBVyTf +793bAm7DU7G+Tzt5gdgE4s41aZt8fFXyclhH1QLPNSnctxJjuW1gJJ0h51iCQJp2 +TgzDw35oqvBxbN3yqCFjScsQXPXYErGWkLrAkUurff4x/ZAizFkmjjdpyaIK9JBw +QRyrYYQ8pJhXJe9BrP3OS6evFlsWZW1MaoQcOUMWsuVucE0e4AQRGlPixDjJWW7L +I6AQ3KUW6ggzDJksaYHDiuEoBa7vcYoTar+/AhNjYMjkQX/3kptQryqy+xke0t8O +EPQER0Wur2IpvM6YsvI/SoeFwxMb4Zm5AFvvibiCCmmoJc4A9E1tZ/sMstHyZ5iu +tJqu1M5B0DIoFdB5pzbZYCkgN2n7EY23JS7E/ozOrzYuOIVUJVtB5awqmuSLmI+N +R91g4FMEfLYC1HYKYlaknX2zmrx8+Z8MEJNM2K0q8wPBnm86OpGeJmlZhFwT2x0R +eJpKcfLGroXYh2Gb6BxwIfKjOOTXCoIFP02JbTJ7clc/2ei0BN6JxywPkH4renaP +SkuNBgbexfZGBhMTlR+CtKLEUmw5bxBTDwjjcvzWDPhy/VurLQxhOqYnbhZW21SV +4qMrJ4uGXEhylnP0FD+HR4mB2epYcW3dFj4cGN3B2Y5NnOTw0Z7fi4S0BPdvYjP9 +LL5WZ6p90mII9wcunGCRnLUUYumRnIbhVHIBTTIRI5PUSVFfEuotrDZ9oZcwYkO7 +fQX21gJCzvJyp8ft01HX4Kc4mN/FMPgGcmq70N335yQ4mQ/eSvTNn7E+35ZGn9f8 +PI7QPJRhdUkBZCnwyv+OwK2VzySxnqNfPaZk168foGRd9eFCw80L4U+SuLDQH6ZT +o++VKk4Ce2jx1khoig16wic0dVFwt4bmybNz4u/qdobYr5fs7dKPHHO02SBvAl60 +16foheiBtV2VA8mEBA1BhcNmKYegu+RGhmGfNDuZB8XdbPQ6M+N+ilEj/6rr+wgD +gcmEyAGNwJkmWpbyrm9M4lDtzemv5N5V32ppGizEt6c0xlkiULllwGdWey3+YRez +7b+Kl/uIpDuRbp5Tf43dyPsy/cx4DNm5kAB4CcyyVlXPaqXm0llEPYBmaMW3O+D2 +5v4Wj1qwIRO5qgI8FyVnX6sm/oucfg5l172edaCG8f42gIMNfQBgWVMsSG7Nt00x +dJo/OGtACwnY47ohMFG0BejWueAksdnqVWCIto989iBHgegNx5jUCycB/YOm0xh0 +pfeNjA9PwZMUpjlqrjDFIan/UFYAZH5ISSV7G30oRKJ3TTEshShXP2K3cn7Fa9W+ +H/jyTEQGfCiTq7Xx5FrOIJBmKjylkF7oGlIBxJgKKRm0iD/sGNTaSJ6Pl8/K6dEz +zsMwEFTawnWVq32Xn3d6/+FADZ9lGhC5WwVgaQHRb/9Ejt1mBdptmXjEj5w0YOib +xFer54LrQgvBWEYRqDneh3bI53BudbTl7YitqULVGETe+k1T0NbcyElrr2Y/NKHk +rPMarAfByookkJrDtVh3VrAm2ows7OwvKGyoNybjlyczjt7xosatZ1xkgb9mtR5i +E2l9ajSR4SzQjHoboRyOCwl5ZgLV/+yp3jTkNcUkFDRtkVbGfascBIMe0ifUGfvP +mJ9AQHZxdfm99KlQjCZzR8CBUvR+zsT43jr91CQKSSEvPMl6vVRV2thiWw3VGgP+ +c8i5zj6+zCnlEdSWiIeFwOJ9/ewKSdU9pGrA0OQtXbYQlDCKuGK1Vgy6jJCeglDH +T6gVNy5ip593wWWfOVxVEWUygi6JCdS27b5+P/wlNjTrzpZ4yWDCpyogyrT1gf1/ +GgvdGuWWinKSLOyh1fJ1p9WoDWcqH98QhJXLV+X3OC+tmMofytmHgXN8jjVsWSRa +VWrFUarMs2hZDWf6e6ncwvMC8QliiszrKXQNckxvBuh5hug9WKurVj4CIWnoqXFh +OqlO+VbqZSj+TT5pCN//370vsIZIn5UbrpDmUP0rUvdTGz9iWQRUl6R2g2h286s6 +pAGHv9luXCoPJ5uPTwcbBSl/js6J+K5McyqRl4fucacfVFnMuDpET/tT1eAROP3F +DOBKqV5YOO0rWMexzMLJUEQ/eGSwfp7wv8on7jeGxAexMqyWCrhRk9G2ZwiT4L7Q +rX4NIDj6oujCCkeFUATs0pGKwEFGmpbEUfDOsioWoVYJZPsO9kAGq6bhbKACOkeZ +v95ha/3CleYXGUUNtzLsCx+c9Zp/Wl+0PcT3ZSWhmRbXiIvz+ntHVe47PHxbvH6a +ZG7YGc/9u3jTvJJyYtQO54uGET/eFWSxCUo5/VfsheOuLdXN7JnVi6ooF+c7WUZd +61FwfDwNf8z0GWs3EotozrWyBgKS5VFP99vZM64nSqu9v5PSzmb0AY/Zc5KhVXVY +zQqmO3keXq92Fejtgyd/O9ITZf5GkMQVU7+IT52JxFRQplkbTHJj4HRGtGHtIyPW +Rmf9qSZz8QgVyAUKK1k+kLBJTHN3CWIB6S9hO42HWEFvLVl8wPWW5aLYTsVMGnMU +aZ35M35odjrvY9B0INMpL53Hm7qH1w/h9QCv+xsFmanYsoylwbuKW2TcSnWB74C7 +Wy0NmCkaM+JweOgygffWicLGJ3jKWccykTUZtodz1ectNHh24puZICnvfzwjte+n +eSQqJfHMsra6V8BcshpwmvPylHnkU+2KyhQ8430OR/qaXAYJ7EWRBEFe4EIpxzfL +zQF0LwbhpAstpcjOlJfEHmQiWx8ASzE1LMSfZo148sXYEWsJL7t5tWs= + diff --git a/test/corpora/protected-headers/subjectless-protected-header.eml b/test/corpora/protected-headers/subjectless-protected-header.eml index 7163b9ae..8c7fb3eb 100644 --- a/test/corpora/protected-headers/subjectless-protected-header.eml +++ b/test/corpora/protected-headers/subjectless-protected-header.eml @@ -17,13 +17,12 @@ Subject: this should not show up as a protected header -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBA/9ZaOuxGtLVWiA7KQfB+4td1AILd1uy039UDb+9YwlhmJTq -mNqVJu+ZkFniZPMliM0z1QRBkBeL2Q7MrHAdYxYBKrDHKVja4O7jwqeKjy5BzQCW -fnyT+sb2Mh+dz5P2voF3XJHgqzhFY1rtVEatXSZADwwIVU6oZqGZ8GOELNGSd9KX -ASNElH7WGZB/TQ5X+MktzOLExx5QWaRK9skogI2RRoOquS7KpMcjzb2FWaJDjr1s -RGboX7NG3xCvNUV2ByFTvLOeo7eO1GfUsabTUbMMvh3AE1UvHgCu8VJiRrMdmPln -BM2xnwCYec6wYJ46fHukTgv+286nSQcV0XT6a+qM5GMgV5DMHW2vSyl6kTszJ3EP -xvQBfPCItA== -=Gkxz +hF4DHXHP849rSK8SAQdAxAZy4nBUDdm2u4sgr1inLki0LMCVcsVlax6Pd0AiZAow +iYz940UtZwQNRRb640w1bB2pAvg5Nn8hJK5ye3qtyUWNW1VEvAa+GXndI/Qt0+7x +0qIBOXRBCrkOxB10iCvSDVoOMZPj8GgvQwpsnslATJbsp9jV74fU7eFCKE5VWKUw +FTos+VX1YFZyf2RsznHXdi0CrL2rkUNoLby4SEUa/urd6GKb3xuOjJlIYN4Fh9xz +e33+Dl3NHohNooytxoJuTNiXbNWe7kidfOwMGzvdYsegk/WMqMLg8DmZPvl0BcYI +o+4ZU3kuhbk5Pup5nOV6OLrs7A0= +=BF8X -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/corpora/protected-headers/wrapped-protected-header.eml b/test/corpora/protected-headers/wrapped-protected-header.eml index 9a3c1384..87f3f8da 100644 --- a/test/corpora/protected-headers/wrapped-protected-header.eml +++ b/test/corpora/protected-headers/wrapped-protected-header.eml @@ -20,14 +20,13 @@ Content-Type: application/octet-stream -----BEGIN PGP MESSAGE----- -hIwDxE023q1UqxYBA/9ZaOuxGtLVWiA7KQfB+4td1AILd1uy039UDb+9YwlhmJTq -mNqVJu+ZkFniZPMliM0z1QRBkBeL2Q7MrHAdYxYBKrDHKVja4O7jwqeKjy5BzQCW -fnyT+sb2Mh+dz5P2voF3XJHgqzhFY1rtVEatXSZADwwIVU6oZqGZ8GOELNGSd9KX -ASNElH7WGZB/TQ5X+MktzOLExx5QWaRK9skogI2RRoOquS7KpMcjzb2FWaJDjr1s -RGboX7NG3xCvNUV2ByFTvLOeo7eO1GfUsabTUbMMvh3AE1UvHgCu8VJiRrMdmPln -BM2xnwCYec6wYJ46fHukTgv+286nSQcV0XT6a+qM5GMgV5DMHW2vSyl6kTszJ3EP -xvQBfPCItA== -=Gkxz +hF4DHXHP849rSK8SAQdAbnAG8Oxige+8PXg1B/Ex8Nc/IcBW7R4Cmnq3rArI3g0w +rb1P2LngDMpqMxRGAkucBU/omYHuyDyLIoQZc84XYQy9N+M/u4HK187tyXaKx970 +0qIBnZhdiVm9RFn8CvQLG1hhw8E6UFm/YlURkMoaP66HIU9WLFAlmHrZPOnXJBr5 +2qOWnqSttuD/1Bjt1R2dguoltYqv1iBkwDlE2mWubSTkDp3Pf3QeJGz3Q727+bHV +MI3k/5sNfJyyx9lIB3nyjwa/+Ap5orrPBwe+Y8tRdLO9xtvIFO+U9l9L6yPTYPyz +4P+LVzDS+6tnWxPiLeEz/sRGmtA= +=pPju -----END PGP MESSAGE----- --=-=-=-- diff --git a/test/emacs-address-cleaning.el b/test/emacs-address-cleaning.el index 8423245f..6eda0ebc 100644 --- a/test/emacs-address-cleaning.el +++ b/test/emacs-address-cleaning.el @@ -1,6 +1,6 @@ (defun notmuch-test-address-cleaning-1 () (notmuch-test-expect-equal (notmuch-show-clean-address "dme@dme.org") - "dme@dme.org")) + "dme@dme.org")) (defun notmuch-test-address-cleaning-2 () (let* ((input '("foo@bar.com" diff --git a/test/emacs-attachment-warnings.el b/test/emacs-attachment-warnings.el index 200ca7ba..8f4918ef 100644 --- a/test/emacs-attachment-warnings.el +++ b/test/emacs-attachment-warnings.el @@ -1,3 +1,4 @@ +(require 'cl-lib) (require 'notmuch-mua) (defun attachment-check-test (&optional fn) @@ -12,7 +13,8 @@ Return `t' if the message would be sent, otherwise `nil'" (condition-case nil ;; Force `y-or-n-p' to always return `nil', as if the user ;; pressed "n". - (letf (((symbol-function 'y-or-n-p) (lambda (&rest args) nil))) + (cl-letf (((symbol-function 'y-or-n-p) + (lambda (&rest args) nil))) (notmuch-mua-attachment-check) t) ('error nil)) @@ -36,6 +38,12 @@ Return `t' if the message would be sent, otherwise `nil'" ;; fontification properties. For fontification to happen we need to ;; allow some time for redisplay. (sit-for 0.01))) + (t . (lambda () + ;; "attach" is only mentioned in a forwarded message. + (insert "Hello\n") + (insert "<#mml type=message/rfc822 disposition=inline>\n") + (insert "X-Has-Attach:\n") + (insert "<#/mml>\n"))) ;; These should not be okay: (nil . (lambda () (insert "Here is an attachment:\n"))) @@ -49,20 +57,25 @@ Return `t' if the message would be sent, otherwise `nil'" ;; looking at fontification properties. For fontification ;; to happen we need to allow some time for redisplay. (sit-for 0.01))) + (nil . (lambda () + ;; "attachment" is mentioned before a forwarded message. + (insert "I also attach something.\n") + (insert "<#mml type=message/rfc822 disposition=inline>\n") + (insert "X-Has-Attach:\n") + (insert "<#/mml>\n"))) )) (defun notmuch-test-attachment-warning-1 () (let (output expected) - (mapcar (lambda (test) - (let* ((expect (car test)) - (body (cdr test)) - (result (attachment-check-test body))) - (push expect expected) - (push (if (eq result expect) - result - ;; In the case of a failure, include the test - ;; details to make it simpler to debug. - (format "%S <-- %S" result body)) - output))) - attachment-check-tests) + (dolist (test attachment-check-tests) + (let* ((expect (car test)) + (body (cdr test)) + (result (attachment-check-test body))) + (push expect expected) + (push (if (eq result expect) + result + ;; In the case of a failure, include the test + ;; details to make it simpler to debug. + (format "%S <-- %S" result body)) + output))) (notmuch-test-expect-equal output expected))) diff --git a/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-with-excluded b/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-with-excluded new file mode 100644 index 00000000..ce1d7118 --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-with-excluded @@ -0,0 +1,25 @@ + 2009-11-17 [5/5] Mikhail Gusarov, Carl Worth, Keith Packard [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 [2/2] Alex Botero-Lowry, Carl Worth [notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 [1/1] Mikhail Gusarov [notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 [2/2] Keith Packard, Carl Worth [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 [2/2] Jan Janak, Carl Worth [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-17 [3/3] Jan Janak, Carl Worth [notmuch] What a great idea! (inbox unread) + 2009-11-17 [3/3] Israel Herraiz, Keith Packard, Carl Worth [notmuch] New to the list (inbox unread) + 2009-11-17 [3/3] Adrian Perez de Castro, Keith Packard, Carl Worth [notmuch] Introducing myself (inbox signed unread) + 2009-11-17 [3/3] Aron Griffis, Keith Packard, Carl Worth [notmuch] archive (inbox unread) + 2009-11-17 [2/2] Ingmar Vanhassel, Carl Worth [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 [2/2] Alex Botero-Lowry, Carl Worth [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 [2/2] Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 [1/1] Stewart Smith [notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. (deleted inbox unread) + 2009-11-18 [1/1] Stewart Smith [notmuch] [PATCH 2/2] Read mail directory in inode number order (deleted inbox unread) + 2009-11-18 [1/1] Stewart Smith [notmuch] [PATCH] count_files: sort directory in inode order before statting (deleted inbox unread) + 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 [1/1] Jan Janak [notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 [1/1] Rolland Santimano [notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 [1/1] Alexander Botero-Lowry [notmuch] request for pull (inbox unread) + 2009-11-18 [2/2] Keith Packard, Alexander Botero-Lowry [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 [1/1] Chris Wilson [notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once (deleted inbox unread) + 2010-12-16 [1/1] Olivier Berger Essai accentué (inbox unread) + 2010-12-29 [1/1] François Boulogne [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-without-excluded b/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-without-excluded new file mode 100644 index 00000000..8a874320 --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-search-tag-inbox-without-excluded @@ -0,0 +1,21 @@ + 2009-11-17 [5/5] Mikhail Gusarov, Carl Worth, Keith Packard [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 [7/7] Lars Kellogg-Stedman, Mikhail Gusarov, Keith Packard, Carl Worth [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 [2/2] Alex Botero-Lowry, Carl Worth [notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 [1/1] Mikhail Gusarov [notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 [2/2] Keith Packard, Carl Worth [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 [2/2] Jan Janak, Carl Worth [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-17 [3/3] Jan Janak, Carl Worth [notmuch] What a great idea! (inbox unread) + 2009-11-17 [3/3] Israel Herraiz, Keith Packard, Carl Worth [notmuch] New to the list (inbox unread) + 2009-11-17 [3/3] Adrian Perez de Castro, Keith Packard, Carl Worth [notmuch] Introducing myself (inbox signed unread) + 2009-11-17 [3/3] Aron Griffis, Keith Packard, Carl Worth [notmuch] archive (inbox unread) + 2009-11-17 [2/2] Ingmar Vanhassel, Carl Worth [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 [2/2] Alex Botero-Lowry, Carl Worth [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 [2/2] Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 [4/4] Jjgod Jiang, Alexander Botero-Lowry [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 [1/1] Jan Janak [notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 [1/1] Rolland Santimano [notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 [1/1] Alexander Botero-Lowry [notmuch] request for pull (inbox unread) + 2009-11-18 [2/2] Keith Packard, Alexander Botero-Lowry [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2010-12-16 [1/1] Olivier Berger Essai accentué (inbox unread) + 2010-12-29 [1/1] François Boulogne [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-with-excluded b/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-with-excluded new file mode 100644 index 00000000..5c6b2d7a --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-with-excluded @@ -0,0 +1,53 @@ + 2009-11-17 Mikhail Gusarov ┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Mikhail Gusarov ├─►[notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) + 2009-11-17 Carl Worth ╰┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Lars Kellogg-Stedman ┬►[notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov ├┬► ... (inbox signed unread) + 2009-11-17 Lars Kellogg-Stedman │╰┬► ... (inbox signed unread) + 2009-11-17 Mikhail Gusarov │ ├─► ... (inbox unread) + 2009-11-17 Keith Packard │ ╰┬► ... (inbox unread) + 2009-11-18 Lars Kellogg-Stedman │ ╰─► ... (inbox signed unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Alex Botero-Lowry ┬►[notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Mikhail Gusarov ─►[notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Keith Packard ┬►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Israel Herraiz ┬►[notmuch] New to the list (inbox unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Adrian Perez de Cast ┬►[notmuch] Introducing myself (inbox signed unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Aron Griffis ┬►[notmuch] archive (inbox unread) + 2009-11-18 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Ingmar Vanhassel ┬►[notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-18 Alex Botero-Lowry ┬►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (inbox unread) + 2009-11-18 Lars Kellogg-Stedman ┬►[notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman ╰─► ... (attachment inbox signed unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. (deleted inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH 2/2] Read mail directory in inode number order (deleted inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] count_files: sort directory in inode order before statting (deleted inbox unread) + 2009-11-18 Jjgod Jiang ┬►[notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low ╰┬► ... (inbox unread) + 2009-11-18 Jjgod Jiang ╰┬► ... (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─► ... (inbox unread) + 2009-11-18 Jan Janak ─►[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Rolland Santimano ─►[notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Alexander Botero-Low ─►[notmuch] request for pull (inbox unread) + 2009-11-18 Keith Packard ┬►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Chris Wilson ─►[notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once (deleted inbox unread) + 2010-12-16 Olivier Berger ─►Essai accentué (inbox unread) + 2010-12-29 François Boulogne ─►[aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-without-excluded b/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-without-excluded new file mode 100644 index 00000000..55806d18 --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-tree-tag-inbox-without-excluded @@ -0,0 +1,49 @@ + 2009-11-17 Mikhail Gusarov ┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Mikhail Gusarov ├─►[notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) + 2009-11-17 Carl Worth ╰┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Lars Kellogg-Stedman ┬►[notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov ├┬► ... (inbox signed unread) + 2009-11-17 Lars Kellogg-Stedman │╰┬► ... (inbox signed unread) + 2009-11-17 Mikhail Gusarov │ ├─► ... (inbox unread) + 2009-11-17 Keith Packard │ ╰┬► ... (inbox unread) + 2009-11-18 Lars Kellogg-Stedman │ ╰─► ... (inbox signed unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Alex Botero-Lowry ┬►[notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Mikhail Gusarov ─►[notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Keith Packard ┬►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Israel Herraiz ┬►[notmuch] New to the list (inbox unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Adrian Perez de Cast ┬►[notmuch] Introducing myself (inbox signed unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Aron Griffis ┬►[notmuch] archive (inbox unread) + 2009-11-18 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Ingmar Vanhassel ┬►[notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-18 Alex Botero-Lowry ┬►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (inbox unread) + 2009-11-18 Lars Kellogg-Stedman ┬►[notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman ╰─► ... (attachment inbox signed unread) + 2009-11-18 Jjgod Jiang ┬►[notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low ╰┬► ... (inbox unread) + 2009-11-18 Jjgod Jiang ╰┬► ... (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─► ... (inbox unread) + 2009-11-18 Jan Janak ─►[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Rolland Santimano ─►[notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Alexander Botero-Low ─►[notmuch] request for pull (inbox unread) + 2009-11-18 Keith Packard ┬►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2010-12-16 Olivier Berger ─►Essai accentué (inbox unread) + 2010-12-29 François Boulogne ─►[aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-with-excluded b/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-with-excluded new file mode 100644 index 00000000..d55818e8 --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-with-excluded @@ -0,0 +1,53 @@ + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) + 2009-11-17 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Alex Botero-Lowry [notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 Carl Worth [notmuch] preliminary FreeBSD support (inbox unread) + 2009-11-17 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Keith Packard [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Keith Packard [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-17 Keith Packard [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Jan Janak [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-17 Jan Janak [notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak [notmuch] What a great idea! (inbox unread) + 2009-11-17 Israel Herraiz [notmuch] New to the list (inbox unread) + 2009-11-17 Adrian Perez de Cast [notmuch] Introducing myself (inbox signed unread) + 2009-11-17 Aron Griffis [notmuch] archive (inbox unread) + 2009-11-17 Ingmar Vanhassel [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Alex Botero-Lowry [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Stewart Smith [notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. (deleted inbox unread) + 2009-11-18 Stewart Smith [notmuch] [PATCH 2/2] Read mail directory in inode number order (deleted inbox unread) + 2009-11-18 Keith Packard [notmuch] New to the list (inbox unread) + 2009-11-18 Keith Packard [notmuch] Introducing myself (inbox unread) + 2009-11-18 Keith Packard [notmuch] archive (inbox unread) + 2009-11-18 Stewart Smith [notmuch] [PATCH] count_files: sort directory in inode order before statting (deleted inbox unread) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Jan Janak [notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Rolland Santimano [notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] request for pull (inbox unread) + 2009-11-18 Keith Packard [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-18 Carl Worth [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-18 Carl Worth [notmuch] archive (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-18 Carl Worth [notmuch] What a great idea! (inbox unread) + 2009-11-18 Carl Worth [notmuch] New to the list (inbox unread) + 2009-11-18 Carl Worth [notmuch] Introducing myself (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (inbox unread) + 2009-11-18 Chris Wilson [notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once (deleted inbox unread) + 2010-12-16 Olivier Berger Essai accentué (inbox unread) + 2010-12-29 François Boulogne [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-without-excluded b/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-without-excluded new file mode 100644 index 00000000..80c67d07 --- /dev/null +++ b/test/emacs-exclude.expected-output/notmuch-unthreaded-tag-inbox-without-excluded @@ -0,0 +1,49 @@ + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) + 2009-11-17 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Alex Botero-Lowry [notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 Carl Worth [notmuch] preliminary FreeBSD support (inbox unread) + 2009-11-17 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Keith Packard [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Keith Packard [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-17 Keith Packard [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Jan Janak [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-17 Jan Janak [notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak [notmuch] What a great idea! (inbox unread) + 2009-11-17 Israel Herraiz [notmuch] New to the list (inbox unread) + 2009-11-17 Adrian Perez de Cast [notmuch] Introducing myself (inbox signed unread) + 2009-11-17 Aron Griffis [notmuch] archive (inbox unread) + 2009-11-17 Ingmar Vanhassel [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Alex Botero-Lowry [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman [notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Keith Packard [notmuch] New to the list (inbox unread) + 2009-11-18 Keith Packard [notmuch] Introducing myself (inbox unread) + 2009-11-18 Keith Packard [notmuch] archive (inbox unread) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Jan Janak [notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Rolland Santimano [notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] request for pull (inbox unread) + 2009-11-18 Keith Packard [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-18 Carl Worth [notmuch] Working with Maildir storage? (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-18 Carl Worth [notmuch] archive (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-18 Carl Worth [notmuch] What a great idea! (inbox unread) + 2009-11-18 Carl Worth [notmuch] New to the list (inbox unread) + 2009-11-18 Carl Worth [notmuch] Introducing myself (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Carl Worth [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (inbox unread) + 2010-12-16 Olivier Berger Essai accentué (inbox unread) + 2010-12-29 François Boulogne [aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-reply.expected-output/notmuch-reply-duplicate-4 b/test/emacs-reply.expected-output/notmuch-reply-duplicate-4 new file mode 100644 index 00000000..836f77b1 --- /dev/null +++ b/test/emacs-reply.expected-output/notmuch-reply-duplicate-4 @@ -0,0 +1,21 @@ +From: Notmuch Test Suite +To: Sean Whitton , 916811@bugs.debian.org, 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, 916876@bugs.debian.org +Subject: Re: [Pkg-emacsen-addons] Bug#916811: Increase severity to 'serious' +In-Reply-To: <87r2ecrr6x.fsf@zephyr.silentflame.com> +Fcc: MAIL_DIR/sent +--text follows this line-- +Sean Whitton writes: + +> control: severity -1 serious +> +> Hello, +> +> Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +> all the e-mails). +> +> -- +> Sean Whitton +> _______________________________________________ +> Pkg-emacsen-addons mailing list +> Pkg-emacsen-addons@alioth-lists.debian.net +> https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-emacsen-addons diff --git a/test/emacs-show.expected-output/notmuch-show-depth b/test/emacs-show.expected-output/notmuch-show-depth new file mode 100644 index 00000000..8299519c --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-depth @@ -0,0 +1,44 @@ +Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Subject: [notmuch] Working with Maildir storage? +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 14:00:54 -0500 + +[ multipart/mixed (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 01:02:38 +0600 + + [ multipart/mixed (hidden) ] + Lars Kellogg-Stedman (2009-11-17) (inbox signed) + Subject: Re: [notmuch] Working with Maildir storage? + To: Mikhail Gusarov + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 15:33:01 -0500 + + [ multipart/mixed (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:50:48 +0600 + + [ text/plain (hidden) ] + Keith Packard (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 13:24:13 -0800 + + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: Keith Packard + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 19:50:40 -0500 + + [ multipart/mixed (hidden) ] + Carl Worth (2009-11-18) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:08:10 -0800 + + [ text/plain (hidden) ] diff --git a/test/emacs-show.expected-output/notmuch-show-depth-1 b/test/emacs-show.expected-output/notmuch-show-depth-1 new file mode 100644 index 00000000..e7c376bb --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-depth-1 @@ -0,0 +1,119 @@ +Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Subject: [notmuch] Working with Maildir storage? +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 14:00:54 -0500 + +[ multipart/mixed ] +[ multipart/signed ] +[ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] +[ text/plain ] +I saw the LWN article and decided to take a look at notmuch. I'm +currently using mutt and mairix to index and read a collection of +Maildir mail folders (around 40,000 messages total). + +notmuch indexed the messages without complaint, but my attempt at +searching bombed out. Running, for example: + + notmuch search storage + +Resulted in 4604 lines of errors along the lines of: + + Error opening + /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S: + Too many open files + +I'm curious if this is expected behavior (i.e., notmuch does not work +with Maildir) or if something else is going on. + +Cheers, + +[ 4-line signature. Click/Enter to show. ] +[ application/pgp-signature ] +[ text/plain ] +[ 4-line signature. Click/Enter to show. ] + Mikhail Gusarov (2009-11-17) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 01:02:38 +0600 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0x9D20F6503E338888 or unsupported algorithm ] + [ text/plain ] + + Twas brillig at 14:00:54 17.11.2009 UTC-05 when lars@seas.harvard.edu did + gyre and gimble: + + LK> Resulted in 4604 lines of errors along the lines of: + + LK> Error opening + LK> + /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S: + LK> Too many open files + + See the patch just posted here. + + [ 2-line signature. Click/Enter to show. ] + [ application/pgp-signature ] + [ text/plain ] + [ 4-line signature. Click/Enter to show. ] + Lars Kellogg-Stedman (2009-11-17) (inbox signed) + Subject: Re: [notmuch] Working with Maildir storage? + To: Mikhail Gusarov + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 15:33:01 -0500 + + [ multipart/mixed (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:50:48 +0600 + + [ text/plain (hidden) ] + Keith Packard (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 13:24:13 -0800 + + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: Keith Packard + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 19:50:40 -0500 + + [ multipart/mixed (hidden) ] + Carl Worth (2009-11-18) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:08:10 -0800 + + On Tue, 17 Nov 2009 14:00:54 -0500, Lars Kellogg-Stedman wrote: + > I saw the LWN article and decided to take a look at notmuch. I'm + > currently using mutt and mairix to index and read a collection of + > Maildir mail folders (around 40,000 messages total). + + Welcome, Lars! + + I hadn't even seen that Keith's blog post had been picked up by lwn.net. + That's very interesting. So, thanks for coming and trying out notmuch. + + > Error opening + > /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S: + > Too many open files + + Sadly, the lwn article coincided with me having just introduced this + bug, and then getting on a Trans-Atlantic flight. So I fixed the bug + fairly quickly, but there was quite a bit of latency before I could push + the fix out. It should be fixed now. + + > I'm curious if this is expected behavior (i.e., notmuch does not work + > with Maildir) or if something else is going on. + + Notmuch works just fine with maildir---it's one of the things that it + likes the best. + + Happy hacking, + + -Carl diff --git a/test/emacs-show.expected-output/notmuch-show-duplicate-4 b/test/emacs-show.expected-output/notmuch-show-duplicate-4 new file mode 100644 index 00000000..6bf49d81 --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-duplicate-4 @@ -0,0 +1,20 @@ +Sean Whitton (2018-12-20) (inbox signed) 4/5 +Subject: [Pkg-emacsen-addons] Bug#916811: Increase severity to 'serious' +To: 916805@bugs.debian.org, 916807@bugs.debian.org, 916808@bugs.debian.org, 916809@bugs.debian.org, 916811@bugs.debian.org, 916867@bugs.debian.org, 916869@bugs.debian.org, 916872@bugs.debian.org, 916875@bugs.debian.org, 916876@bugs.debian.org +Date: Thu, 20 Dec 2018 18:25:26 +0000 + +[ multipart/mixed ] +[ multipart/signed ] +[ Unknown key ID 0x695B7AE4BF066240 or unsupported algorithm ] +[ text/plain ] +control: severity -1 serious + +Hello, + +Emacs 26.1 has reached Debian unstable (sooner than expected; sorry for +all the e-mails). + +[ 2-line signature. Click/Enter to show. ] +[ signature.asc: application/pgp-signature ] +[ text/plain ] +[ 4-line signature. Click/Enter to show. ] diff --git a/test/emacs-show.expected-output/notmuch-show-height-0 b/test/emacs-show.expected-output/notmuch-show-height-0 new file mode 100644 index 00000000..d646353e --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-height-0 @@ -0,0 +1,97 @@ +Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Subject: [notmuch] Working with Maildir storage? +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 14:00:54 -0500 + +[ multipart/mixed (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 01:02:38 +0600 + + [ multipart/mixed (hidden) ] + Lars Kellogg-Stedman (2009-11-17) (inbox signed) + Subject: Re: [notmuch] Working with Maildir storage? + To: Mikhail Gusarov + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 15:33:01 -0500 + + [ multipart/mixed (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:50:48 +0600 + + Twas brillig at 15:33:01 17.11.2009 UTC-05 when lars at seas.harvard.edu + did gyre and gimble: + + LK> Is the list archived anywhere? The obvious archives + LK> (http://notmuchmail.org/pipermail/notmuch/) aren't available, and I + LK> think I subscribed too late to get the patch (I only just saw the + LK> discussion about it). + + LK> It doesn't look like the patch is in git yet. + + Just has been pushed + + [ 10-line signature. Click/Enter to show. ] + Keith Packard (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 13:24:13 -0800 + + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: Keith Packard + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 19:50:40 -0500 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] + [ text/plain ] + > I've also pushed a slightly more complicated (and complete) fix to my + > private notmuch repository + + The version of lib/messages.cc in your repo doesn't build because it's + missing "#include " (for the uint32_t on line 466). + + [ 4-line signature. Click/Enter to show. ] + [ application/pgp-signature ] + [ text/plain ] + [ 4-line signature. Click/Enter to show. ] + Carl Worth (2009-11-18) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:08:10 -0800 + + On Tue, 17 Nov 2009 14:00:54 -0500, Lars Kellogg-Stedman wrote: + > I saw the LWN article and decided to take a look at notmuch. I'm + > currently using mutt and mairix to index and read a collection of + > Maildir mail folders (around 40,000 messages total). + + Welcome, Lars! + + I hadn't even seen that Keith's blog post had been picked up by lwn.net. + That's very interesting. So, thanks for coming and trying out notmuch. + + > Error opening + > /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S: + > Too many open files + + Sadly, the lwn article coincided with me having just introduced this + bug, and then getting on a Trans-Atlantic flight. So I fixed the bug + fairly quickly, but there was quite a bit of latency before I could push + the fix out. It should be fixed now. + + > I'm curious if this is expected behavior (i.e., notmuch does not work + > with Maildir) or if something else is going on. + + Notmuch works just fine with maildir---it's one of the things that it + likes the best. + + Happy hacking, + + -Carl diff --git a/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off b/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off index 1a06374d..0bb58330 100644 --- a/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off +++ b/test/emacs-show.expected-output/notmuch-show-indent-thread-content-off @@ -31,8 +31,8 @@ Cheers, [ application/pgp-signature ] [ text/plain ] [ 4-line signature. Click/Enter to show. ] - Mikhail Gusarov (2009-11-17) (inbox signed unread) - Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Mikhail Gusarov (2009-11-17) (inbox signed unread) +Lars Kellogg-Stedman (2009-11-17) (inbox signed) Subject: Re: [notmuch] Working with Maildir storage? To: Mikhail Gusarov Cc: notmuch@notmuchmail.org @@ -57,9 +57,9 @@ It doesn't look like the patch is in git yet. [ application/pgp-signature ] [ text/plain ] [ 4-line signature. Click/Enter to show. ] - Mikhail Gusarov (2009-11-17) (inbox unread) - Keith Packard (2009-11-17) (inbox unread) - Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) +Mikhail Gusarov (2009-11-17) (inbox unread) +Keith Packard (2009-11-17) (inbox unread) +Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) Subject: Re: [notmuch] Working with Maildir storage? To: Keith Packard Cc: notmuch@notmuchmail.org @@ -79,4 +79,4 @@ missing "#include " (for the uint32_t on line 466). [ application/pgp-signature ] [ text/plain ] [ 4-line signature. Click/Enter to show. ] - Carl Worth (2009-11-18) (inbox unread) +Carl Worth (2009-11-18) (inbox unread) diff --git a/test/emacs-show.expected-output/notmuch-show-multipart-alternative b/test/emacs-show.expected-output/notmuch-show-multipart-alternative new file mode 100644 index 00000000..e2951d2b --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-multipart-alternative @@ -0,0 +1,62 @@ +Alex Botero-Lowry (2009-11-17) (attachment inbox) +Subject: [notmuch] preliminary FreeBSD support +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 11:36:14 -0800 + +[ multipart/mixed ] +[ multipart/alternative ] +[ text/plain ] +I saw the announcement this morning, and was very excited, as I had been +hoping sup would be turned into a library, +since I like the concept more than the UI (I'd rather an emacs interface). + +I did a preliminary compile which worked out fine, but +sysconf(_SC_SC_GETPW_R_SIZE_MAX) returns -1 on +FreeBSD, so notmuch_config_open segfaulted. + +Attached is a patch that supplies a default buffer size of 64 in cases where +-1 is returned. + +http://www.opengroup.org/austin/docs/austin_328.txt - seems to indicate this +is acceptable behavior, +and +http://mail-index.netbsd.org/pkgsrc-bugs/2006/06/07/msg016808.htmlspecifically +uses 64 as the +buffer size. +[ text/html (hidden) ] +[ 0001-Deal-with-situation-where-sysconf-_SC_GETPW_R_SIZE_M.patch: text/x-diff ] +From e3bc4bbd7b9d0d086816ab5f8f2d6ffea1dd3ea4 Mon Sep 17 00:00:00 2001 +From: Alexander Botero-Lowry +Date: Tue, 17 Nov 2009 11:30:39 -0800 +Subject: [PATCH] Deal with situation where sysconf(_SC_GETPW_R_SIZE_MAX) returns -1 + +--- + notmuch-config.c | 2 ++ + 1 files changed, 2 insertions(+), 0 deletions(-) + +diff --git a/notmuch-config.c b/notmuch-config.c +index 248149c..e7220d8 100644 +--- a/notmuch-config.c ++++ b/notmuch-config.c +@@ -77,6 +77,7 @@ static char * + get_name_from_passwd_file (void *ctx) + { + long pw_buf_size = sysconf(_SC_GETPW_R_SIZE_MAX); ++ if (pw_buf_size == -1) pw_buf_size = 64; + char *pw_buf = talloc_zero_size (ctx, pw_buf_size); + struct passwd passwd, *ignored; + char *name; +@@ -101,6 +102,7 @@ static char * + get_username_from_passwd_file (void *ctx) + { + long pw_buf_size = sysconf(_SC_GETPW_R_SIZE_MAX); ++ if (pw_buf_size == -1) pw_buf_size = 64; + char *pw_buf = talloc_zero_size (ctx, pw_buf_size); + struct passwd passwd, *ignored; + char *name; +-- +1.6.5.2 + +[ text/plain ] +[ 4-line signature. Click/Enter to show. ] + Carl Worth (2009-11-17) (inbox unread) diff --git a/test/emacs-show.expected-output/notmuch-show-size b/test/emacs-show.expected-output/notmuch-show-size new file mode 100644 index 00000000..cdde467e --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-size @@ -0,0 +1,64 @@ +Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Subject: [notmuch] Working with Maildir storage? +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 14:00:54 -0500 + +[ multipart/mixed ] +[ multipart/signed ] +[ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] +[ text/plain (hidden) ] +[ application/pgp-signature ] +[ text/plain (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 01:02:38 +0600 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0x9D20F6503E338888 or unsupported algorithm ] + [ text/plain (hidden) ] + [ application/pgp-signature ] + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-17) (inbox signed) + Subject: Re: [notmuch] Working with Maildir storage? + To: Mikhail Gusarov + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 15:33:01 -0500 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] + [ text/plain (hidden) ] + [ application/pgp-signature ] + [ text/plain (hidden) ] + Mikhail Gusarov (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:50:48 +0600 + + [ text/plain (hidden) ] + Keith Packard (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 13:24:13 -0800 + + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: Keith Packard + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 19:50:40 -0500 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] + [ text/plain (hidden) ] + [ application/pgp-signature ] + [ text/plain (hidden) ] + Carl Worth (2009-11-18) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:08:10 -0800 + + [ text/plain (hidden) ] diff --git a/test/emacs-show.expected-output/notmuch-show-size-450 b/test/emacs-show.expected-output/notmuch-show-size-450 new file mode 100644 index 00000000..ec34612e --- /dev/null +++ b/test/emacs-show.expected-output/notmuch-show-size-450 @@ -0,0 +1,89 @@ +Lars Kellogg-Stedman (2009-11-17) (inbox signed) +Subject: [notmuch] Working with Maildir storage? +To: notmuch@notmuchmail.org +Date: Tue, 17 Nov 2009 14:00:54 -0500 + +[ multipart/mixed ] +[ multipart/signed ] +[ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] +[ text/plain (hidden) ] +[ application/pgp-signature ] +[ text/plain ] +[ 4-line signature. Click/Enter to show. ] + Mikhail Gusarov (2009-11-17) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 01:02:38 +0600 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0x9D20F6503E338888 or unsupported algorithm ] + [ text/plain ] + + Twas brillig at 14:00:54 17.11.2009 UTC-05 when lars@seas.harvard.edu did + gyre and gimble: + + LK> Resulted in 4604 lines of errors along the lines of: + + LK> Error opening + LK> + /home/lars/Mail/read-messages.2008/cur/1246413773.24928_27334.hostname,U=3026:2,S: + LK> Too many open files + + See the patch just posted here. + + [ 2-line signature. Click/Enter to show. ] + [ application/pgp-signature ] + [ text/plain ] + [ 4-line signature. Click/Enter to show. ] + Lars Kellogg-Stedman (2009-11-17) (inbox signed) + Subject: Re: [notmuch] Working with Maildir storage? + To: Mikhail Gusarov + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 15:33:01 -0500 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] + [ text/plain (hidden) ] + [ application/pgp-signature ] + [ text/plain ] + [ 4-line signature. Click/Enter to show. ] + Mikhail Gusarov (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:50:48 +0600 + + [ text/plain (hidden) ] + Keith Packard (2009-11-17) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 13:24:13 -0800 + + [ text/plain (hidden) ] + Lars Kellogg-Stedman (2009-11-18) (inbox signed unread) + Subject: Re: [notmuch] Working with Maildir storage? + To: Keith Packard + Cc: notmuch@notmuchmail.org + Date: Tue, 17 Nov 2009 19:50:40 -0500 + + [ multipart/mixed ] + [ multipart/signed ] + [ Unknown key ID 0xD74695063141ACD8 or unsupported algorithm ] + [ text/plain ] + > I've also pushed a slightly more complicated (and complete) fix to my + > private notmuch repository + + The version of lib/messages.cc in your repo doesn't build because it's + missing "#include " (for the uint32_t on line 466). + + [ 4-line signature. Click/Enter to show. ] + [ application/pgp-signature ] + [ text/plain ] + [ 4-line signature. Click/Enter to show. ] + Carl Worth (2009-11-18) (inbox unread) + Subject: [notmuch] Working with Maildir storage? + To: notmuch@notmuchmail.org + Date: Wed, 18 Nov 2009 02:08:10 -0800 + + [ text/plain (hidden) ] diff --git a/test/emacs-tree.expected-output/inbox-outline b/test/emacs-tree.expected-output/inbox-outline new file mode 100644 index 00000000..9119a916 --- /dev/null +++ b/test/emacs-tree.expected-output/inbox-outline @@ -0,0 +1,25 @@ + 2010-12-29 François Boulogne ─►[aur-general] Guidelines: cp, mkdir vs install (inbox unread) + 2010-12-16 Olivier Berger ─►Essai accentué (inbox unread) + 2009-11-18 Chris Wilson ─►[notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once (inbox unread) + 2009-11-18 Alex Botero-Lowry ┬►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-17 Ingmar Vanhassel ┬►[notmuch] [PATCH] Typsos (inbox unread) + 2009-11-17 Adrian Perez de Cast ┬►[notmuch] Introducing myself (inbox signed unread) + 2009-11-17 Israel Herraiz ┬►[notmuch] New to the list (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-17 Aron Griffis ┬►[notmuch] archive (inbox unread) + 2009-11-17 Keith Packard ┬►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Lars Kellogg-Stedman ┬►[notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov ┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-18 Keith Packard ┬►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low ─►[notmuch] request for pull (inbox unread) + 2009-11-18 Jjgod Jiang ┬►[notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Rolland Santimano ─►[notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Jan Janak ─►[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] count_files: sort directory in inode order before statting (inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH 2/2] Read mail directory in inode number order (inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. (inbox unread) + 2009-11-18 Lars Kellogg-Stedman ┬►[notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-17 Mikhail Gusarov ─►[notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Alex Botero-Lowry ┬►[notmuch] preliminary FreeBSD support (attachment inbox unread) +End of search results. diff --git a/test/emacs-tree.expected-output/notmuch-tree-tag-inbox-oldest-first b/test/emacs-tree.expected-output/notmuch-tree-tag-inbox-oldest-first new file mode 100644 index 00000000..588fc583 --- /dev/null +++ b/test/emacs-tree.expected-output/notmuch-tree-tag-inbox-oldest-first @@ -0,0 +1,53 @@ + 2009-11-17 Mikhail Gusarov ┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Mikhail Gusarov ├─►[notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 (inbox unread) + 2009-11-17 Carl Worth ╰┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers (inbox unread) + 2009-11-17 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Lars Kellogg-Stedman ┬►[notmuch] Working with Maildir storage? (inbox signed unread) + 2009-11-17 Mikhail Gusarov ├┬► ... (inbox signed unread) + 2009-11-17 Lars Kellogg-Stedman │╰┬► ... (inbox signed unread) + 2009-11-17 Mikhail Gusarov │ ├─► ... (inbox unread) + 2009-11-17 Keith Packard │ ╰┬► ... (inbox unread) + 2009-11-18 Lars Kellogg-Stedman │ ╰─► ... (inbox signed unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Alex Botero-Lowry ┬►[notmuch] preliminary FreeBSD support (attachment inbox unread) + 2009-11-17 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Mikhail Gusarov ─►[notmuch] [PATCH] Handle rename of message file (inbox unread) + 2009-11-17 Keith Packard ┬►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] [PATCH] Older versions of install do not support -C. (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Jan Janak ┬►[notmuch] What a great idea! (inbox unread) + 2009-11-17 Jan Janak ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Israel Herraiz ┬►[notmuch] New to the list (inbox unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Adrian Perez de Cast ┬►[notmuch] Introducing myself (inbox signed unread) + 2009-11-18 Keith Packard ├─► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Aron Griffis ┬►[notmuch] archive (inbox unread) + 2009-11-18 Keith Packard ╰┬► ... (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-17 Ingmar Vanhassel ┬►[notmuch] [PATCH] Typsos (inbox unread) + 2009-11-18 Carl Worth ╰─► ... (inbox unread) + 2009-11-18 Alex Botero-Lowry ┬►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (attachment inbox unread) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (inbox unread) + 2009-11-18 Lars Kellogg-Stedman ┬►[notmuch] "notmuch help" outputs to stderr? (attachment inbox signed unread) + 2009-11-18 Lars Kellogg-Stedman ╰─► ... (attachment inbox signed unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. (inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH 2/2] Read mail directory in inode number order (inbox unread) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] count_files: sort directory in inode order before statting (inbox unread) + 2009-11-18 Jjgod Jiang ┬►[notmuch] Mac OS X/Darwin compatibility issues (inbox unread) + 2009-11-18 Alexander Botero-Low ╰┬► ... (inbox unread) + 2009-11-18 Jjgod Jiang ╰┬► ... (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─► ... (inbox unread) + 2009-11-18 Jan Janak ─►[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags (inbox unread) + 2009-11-18 Rolland Santimano ─►[notmuch] Link to mailing list archives ? (inbox unread) + 2009-11-18 Alexander Botero-Low ─►[notmuch] request for pull (inbox unread) + 2009-11-18 Keith Packard ┬►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Alexander Botero-Low ╰─►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap (inbox unread) + 2009-11-18 Chris Wilson ─►[notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once (inbox unread) + 2010-12-16 Olivier Berger ─►Essai accentué (inbox unread) + 2010-12-29 François Boulogne ─►[aur-general] Guidelines: cp, mkdir vs install (inbox unread) +End of search results. diff --git a/test/emacs-tree.expected-output/result-format-function b/test/emacs-tree.expected-output/result-format-function new file mode 100644 index 00000000..7eb24696 --- /dev/null +++ b/test/emacs-tree.expected-output/result-format-function @@ -0,0 +1,53 @@ + 2010-12-29 François Boulogne ─►[aur-general] Guidelines: cp, mkdir vs install ( ui) + 2010-12-16 Olivier Berger ─►Essai accentué ( ui) + 2009-11-18 Chris Wilson ─►[notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once ( ui) + 2009-11-18 Alex Botero-Lowry ┬►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop (& ui) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop ( ui) + 2009-11-17 Ingmar Vanhassel ┬►[notmuch] [PATCH] Typsos ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Adrian Perez de Cast ┬►[notmuch] Introducing myself ( =ui) + 2009-11-18 Keith Packard ├─► ... ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Israel Herraiz ┬►[notmuch] New to the list ( ui) + 2009-11-18 Keith Packard ├─► ... ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Jan Janak ┬►[notmuch] What a great idea! ( ui) + 2009-11-17 Jan Janak ├─► ... ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Jan Janak ┬►[notmuch] [PATCH] Older versions of install do not support -C. ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Aron Griffis ┬►[notmuch] archive ( ui) + 2009-11-18 Keith Packard ╰┬► ... ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Keith Packard ┬►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags ( ui) + 2009-11-18 Carl Worth ╰─►[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags ( ui) + 2009-11-17 Lars Kellogg-Stedman ┬►[notmuch] Working with Maildir storage? ( = i) + 2009-11-17 Mikhail Gusarov ├┬► ... ( =ui) + 2009-11-17 Lars Kellogg-Stedman │╰┬► ... ( =ui) + 2009-11-17 Mikhail Gusarov │ ├─► ... ( ui) + 2009-11-17 Keith Packard │ ╰┬► ... ( ui) + 2009-11-18 Lars Kellogg-Stedman │ ╰─► ... ( =ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-17 Mikhail Gusarov ┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers ( i) + 2009-11-17 Mikhail Gusarov ├─►[notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4 ( ui) + 2009-11-17 Carl Worth ╰┬►[notmuch] [PATCH 1/2] Close message file after parsing message headers ( ui) + 2009-11-17 Keith Packard ╰┬► ... ( ui) + 2009-11-18 Carl Worth ╰─► ... ( ui) + 2009-11-18 Keith Packard ┬►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap ( ui) + 2009-11-18 Alexander Botero-Low ╰─►[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap ( ui) + 2009-11-18 Alexander Botero-Low ─►[notmuch] request for pull ( ui) + 2009-11-18 Jjgod Jiang ┬►[notmuch] Mac OS X/Darwin compatibility issues ( ui) + 2009-11-18 Alexander Botero-Low ╰┬► ... ( ui) + 2009-11-18 Jjgod Jiang ╰┬► ... ( ui) + 2009-11-18 Alexander Botero-Low ╰─► ... ( ui) + 2009-11-18 Rolland Santimano ─►[notmuch] Link to mailing list archives ? ( ui) + 2009-11-18 Jan Janak ─►[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags ( ui) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] count_files: sort directory in inode order before statting ( ui) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH 2/2] Read mail directory in inode number order ( ui) + 2009-11-18 Stewart Smith ─►[notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs. ( ui) + 2009-11-18 Lars Kellogg-Stedman ┬►[notmuch] "notmuch help" outputs to stderr? (&=ui) + 2009-11-18 Lars Kellogg-Stedman ╰─► ... (&=ui) + 2009-11-17 Mikhail Gusarov ─►[notmuch] [PATCH] Handle rename of message file ( ui) + 2009-11-17 Alex Botero-Lowry ┬►[notmuch] preliminary FreeBSD support (& ui) + 2009-11-17 Carl Worth ╰─► ... ( ui) +End of search results. diff --git a/test/emacs-unthreaded.expected-output/result-format-function b/test/emacs-unthreaded.expected-output/result-format-function new file mode 100644 index 00000000..bcb10b96 --- /dev/null +++ b/test/emacs-unthreaded.expected-output/result-format-function @@ -0,0 +1,53 @@ + 2010-12-29 François Boulogne [aur-general] Guidelines: cp, mkdir vs install ( ui) + 2010-12-16 Olivier Berger Essai accentué ( ui) + 2009-11-18 Chris Wilson [notmuch] [PATCH 1/2] Makefile: evaluate pkg-config once( ui) + 2009-11-18 Carl Worth [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop( ui) + 2009-11-18 Carl Worth [notmuch] [PATCH] Typsos ( ui) + 2009-11-18 Carl Worth [notmuch] Introducing myself ( ui) + 2009-11-18 Carl Worth [notmuch] New to the list ( ui) + 2009-11-18 Carl Worth [notmuch] What a great idea! ( ui) + 2009-11-18 Carl Worth [notmuch] [PATCH] Older versions of install do not support -C.( ui) + 2009-11-18 Carl Worth [notmuch] archive ( ui) + 2009-11-18 Carl Worth [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags( ui) + 2009-11-18 Carl Worth [notmuch] Working with Maildir storage? ( ui) + 2009-11-18 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers( ui) + 2009-11-18 Alexander Botero-Low[notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap( ui) + 2009-11-18 Keith Packard [notmuch] [PATCH] Create a default notmuch-show-hook that highlights URLs and uses word-wrap( ui) + 2009-11-18 Alexander Botero-Low[notmuch] request for pull ( ui) + 2009-11-18 Alexander Botero-Low[notmuch] Mac OS X/Darwin compatibility issues ( ui) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues ( ui) + 2009-11-18 Alexander Botero-Low[notmuch] Mac OS X/Darwin compatibility issues ( ui) + 2009-11-18 Rolland Santimano [notmuch] Link to mailing list archives ? ( ui) + 2009-11-18 Jan Janak [notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags( ui) + 2009-11-18 Jjgod Jiang [notmuch] Mac OS X/Darwin compatibility issues ( ui) + 2009-11-18 Stewart Smith [notmuch] [PATCH] count_files: sort directory in inode order before statting( ui) + 2009-11-18 Keith Packard [notmuch] archive ( ui) + 2009-11-18 Keith Packard [notmuch] Introducing myself ( ui) + 2009-11-18 Keith Packard [notmuch] New to the list ( ui) + 2009-11-18 Stewart Smith [notmuch] [PATCH 2/2] Read mail directory in inode number order( ui) + 2009-11-18 Stewart Smith [notmuch] [PATCH] Fix linking with gcc to use g++ to link in C++ libs.( ui) + 2009-11-18 Lars Kellogg-Stedman[notmuch] "notmuch help" outputs to stderr? (&=ui) + 2009-11-18 Lars Kellogg-Stedman[notmuch] "notmuch help" outputs to stderr? (&=ui) + 2009-11-18 Lars Kellogg-Stedman[notmuch] Working with Maildir storage? ( =ui) + 2009-11-18 Alex Botero-Lowry [notmuch] [PATCH] Error out if no query is supplied to search instead of going into an infinite loop(& ui) + 2009-11-17 Ingmar Vanhassel [notmuch] [PATCH] Typsos ( ui) + 2009-11-17 Aron Griffis [notmuch] archive ( ui) + 2009-11-17 Adrian Perez de Cast[notmuch] Introducing myself ( =ui) + 2009-11-17 Israel Herraiz [notmuch] New to the list ( ui) + 2009-11-17 Jan Janak [notmuch] What a great idea! ( ui) + 2009-11-17 Jan Janak [notmuch] What a great idea! ( ui) + 2009-11-17 Jan Janak [notmuch] [PATCH] Older versions of install do not support -C.( ui) + 2009-11-17 Keith Packard [notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags( ui) + 2009-11-17 Keith Packard [notmuch] Working with Maildir storage? ( ui) + 2009-11-17 Keith Packard [notmuch] [PATCH 1/2] Close message file after parsing message headers( ui) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH] Handle rename of message file ( ui) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? ( ui) + 2009-11-17 Lars Kellogg-Stedman[notmuch] Working with Maildir storage? ( =ui) + 2009-11-17 Carl Worth [notmuch] preliminary FreeBSD support ( ui) + 2009-11-17 Alex Botero-Lowry [notmuch] preliminary FreeBSD support (& ui) + 2009-11-17 Mikhail Gusarov [notmuch] Working with Maildir storage? ( =ui) + 2009-11-17 Lars Kellogg-Stedman[notmuch] Working with Maildir storage? ( =ui) + 2009-11-17 Carl Worth [notmuch] [PATCH 1/2] Close message file after parsing message headers( ui) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 2/2] Include to get uint32_t in C++ file with gcc 4.4( ui) + 2009-11-17 Mikhail Gusarov [notmuch] [PATCH 1/2] Close message file after parsing message headers( ui) +End of search results. diff --git a/test/emacs.expected-output/notmuch-hello-all-tags b/test/emacs.expected-output/notmuch-hello-all-tags new file mode 100644 index 00000000..65e479fa --- /dev/null +++ b/test/emacs.expected-output/notmuch-hello-all-tags @@ -0,0 +1,11 @@ + Welcome to notmuch. You have 52 messages. + +Search: . + +All tags: [hide] + + 4 attachment 52 inbox 52 unread + 52 exclude_me 7 signed + + Hit `?' for context-sensitive help in any Notmuch screen. + Customize Notmuch or this page. diff --git a/test/emacs.expected-output/raw-message-cf0c4d-52ad0a b/test/emacs.expected-output/raw-message-cf0c4d-52ad0a index 75b05fa4..ce174b52 100644 --- a/test/emacs.expected-output/raw-message-cf0c4d-52ad0a +++ b/test/emacs.expected-output/raw-message-cf0c4d-52ad0a @@ -63,7 +63,7 @@ and &2 exit 1 fi diff --git a/test/ghost-report.cc b/test/ghost-report.cc index fad9a71d..9d9e7a7f 100644 --- a/test/ghost-report.cc +++ b/test/ghost-report.cc @@ -12,5 +12,6 @@ main (int argc, char **argv) } Xapian::Database db (argv[1]); + std::cout << db.get_termfreq ("Tghost") << std::endl; } diff --git a/test/gnupg-secret-key.NOTE b/test/gnupg-secret-key.NOTE deleted file mode 100644 index 604496c5..00000000 --- a/test/gnupg-secret-key.NOTE +++ /dev/null @@ -1,9 +0,0 @@ -How the crypto test gnupg secret was generated: - -GNUPGHOME=gnupghome gpg --quick-random --gen-key -kind: 1 (RSA/RSA) -size: 1024 -expire: 0 -name: Notmuch Test Suite -email: test_suite@notmuchmail.org -(no passphrase) diff --git a/test/gnupg-secret-key.asc b/test/gnupg-secret-key.asc deleted file mode 100644 index 6431b56c..00000000 --- a/test/gnupg-secret-key.asc +++ /dev/null @@ -1,34 +0,0 @@ ------BEGIN PGP PRIVATE KEY BLOCK----- -Version: GnuPG v1.4.10 (GNU/Linux) - -lQHYBE1Mm18BBADlMsMlUeO6usp/XuulgimqlCSphHcYZvH6+Sy7u7W4TpJzid7e -jEOCrk3UZi2XMPW9+snDMhV9e28HeRz61zAO9G/gedn4N+mKOyTaELEmj9SP2IG2 -ZTvdUvn30vWIHyfRIww3qEiSzNULKn6zTDfcg6BIY6ZDQ6GFSfH5EioxuQARAQAB -AAP8CM2/sS9JZWLHZHJrmsU6fygxlaarlxmyhxwLG9WZ+qUJ+xDQqWZkhssrMigP -7ZQehwLwZ7mvbvfOy/qwTPJMZjQMMuTGEzclwBTOTttSxEDS+kgYmZ05CBjIgXbo -8+k+L347l+kVRBFsi1cqOkCr+VZQwhOnbeNb8uJsUx27aAECAPD7jsBP73LRgoXQ -x650D2fzjjuomGVsIxSAPjkDRYmtorsRftaEy7DkvX3Ihu5WN6YRRjJavoL+f8ar -4escR40CAPN7NOFOGmiFZYzQcfJYQI2m7YDk4B51JxORFvLrvGT+LJnVwhtFsdGS -QnMyO4eNpKH0qeEkT5Zqha2oyAc0Yd0B/3f962YCmYlbDAvWjcbMvhV7G4DbazVp -2TNR0BhhEMiOlHuwmTO59s2iukuE5ifaVbwrj+NgpipTsaffKnhALlGjV7Q7Tm90 -bXVjaCBUZXN0IFN1aXRlIDx0ZXN0X3N1aXRlQG5vdG11Y2htYWlsLm9yZz4gKElO -U0VDVVJFISmIuAQTAQIAIgUCTUybXwIbAwYLCQgHAwIGFQgCCQoLBBYCAwECHgEC -F4AACgkQbZJhLZTkY4GJFAP9E0mOw+RUGdmqbxSbd2rm0/inUSYOC0Pvt/D05pUY -xzXDAMZwsy1DWhfS7bSgdD3YTM/22b/LJ2FmbLUF1cU6cNslmdPdfHDZ5+C4qpa1 -uW11y7djlBFAwxc3NBypT6Bmh/iIixrx413cw8CEU0lSZbSXUvbxZ7Rg4JYm2K6f -Y7SdAdgETUybXwEEAM74QJJWzPavquSF0IkKDFjEvI44WC1HGNsJF3JVuKv9G00P -RaHavNNcHEG8MorbfaWk7pipaEJ3+zbPKgp2vRCSJnLL6z813JIQqXJTZzu1ip63 -s4icfOfXkxFJ5AaFd/pVdi+wjmEwvv+YMtJT9DyXANo6b2eQu+0bMtP4Xuv/ABEB -AAEAA/wJArUJw450070K6eoXeg22wT0iq/O0aCExSzoI5Kmywytj6KnnAmp9TftL -WVgrkQntVjrhzPsYoB40JEMrGKd7QL/6LPTNWq3eFW38PSpCiG83T0rtmKCKqHB1 -Uo0B78AHfYYX7MUOEuCq2AhKTAdZukesoCpmVxcEFtjDEbOB8QIA3cvXrPJN/J2S -W61mdMT7KlaXZZD8Phs/TY2ZLAiMKUAP1dVYNDvRSDjZLvQrqKQjEAN5jM81cWAV -pvOIavLhOwIA7uMVIiaQ3vIy10C7ltiLT6YuJL/O6XDnXY/PDuXOatQahd/gmI0q -dGQLSaHIxYILPZPaW6t0orx+dduQ0ES0DQIA21nEKX0MZpYOY1eIt6OlKemsjL2a -UTdFhq/OgwVv+QRVHNdYQXmKpKDeW30lN/+BI3zyDTZjtehwKMMxNTu4AJu/iJ8E -GAECAAkFAk1Mm18CGwwACgkQbZJhLZTkY4H8kgQA4vHsTt8dlJdWJAu2SKZGOPRs -bIPu5XtRXe3RYbW5H7PqbAnrKIzlIKpkPNTwLL4wVXaF+R/aHa8ZKX3paohrPL74 -qpbffwtHXyVEwyWlw3m9mgti0de1dy1YvVasCe/UQ8Frc6uNmOwtlQE20k4R4cLI -SWXT1JrwPoKh9xe++90= -=rvTR ------END PGP PRIVATE KEY BLOCK----- diff --git a/test/json_check_nodes.py b/test/json_check_nodes.py index 17403c57..fd8f1607 100755 --- a/test/json_check_nodes.py +++ b/test/json_check_nodes.py @@ -1,4 +1,4 @@ -#!/usr/bin/env python +#!/usr/bin/env python3 import re import sys import json diff --git a/test/make-db-version.cc b/test/make-db-version.cc index 78feaf72..238584e2 100644 --- a/test/make-db-version.cc +++ b/test/make-db-version.cc @@ -17,6 +17,7 @@ main (int argc, char **argv) } std::string nmpath (argv[1]); + nmpath += "/.notmuch"; if (mkdir (nmpath.c_str (), 0777) < 0) { perror (("failed to create " + nmpath).c_str ()); diff --git a/test/notmuch-test b/test/notmuch-test index b58fd3b3..5d27e4d1 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -18,12 +18,22 @@ fi # Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set. . $(dirname "$0")/export-dirs.sh || exit 1 +set -eu + +# Where to run the tests +# XXX FIXME this code is duplicated with test-lib.sh +if [[ -n "${NOTMUCH_BUILDDIR}" ]]; then + TEST_DIRECTORY=$NOTMUCH_BUILDDIR/test +else + TEST_DIRECTORY=$NOTMUCH_SRCDIR/test +fi + TESTS= -for test in $NOTMUCH_TESTS; do +for test in ${NOTMUCH_TESTS-}; do TESTS="$TESTS $NOTMUCH_SRCDIR/test/$test" done -if [[ -z "$TESTS" ]]; then +if [ -z "$TESTS" ]; then TESTS="$NOTMUCH_SRCDIR/test/T[0-9][0-9][0-9]-*.sh" fi @@ -44,43 +54,46 @@ else TEST_TIMEOUT_CMD="" fi -trap 'e=$?; kill $!; exit $e' HUP INT TERM - META_FAILURE= +RES=0 # Run the tests -if test -z "$NOTMUCH_TEST_SERIALIZE" && command -v parallel >/dev/null ; then +if test -z "${NOTMUCH_TEST_SERIALIZE-}" && command -v parallel >/dev/null ; then test -t 1 && export COLORS_WITHOUT_TTY=t || : - if parallel -h | grep -q GNU ; then + if parallel --minversion 0 >/dev/null 2>&1 ; then echo "INFO: running tests with GNU parallel" - printf '%s\n' $TESTS | $TEST_TIMEOUT_CMD parallel + printf '%s\n' $TESTS | $TEST_TIMEOUT_CMD parallel || RES=$? else echo "INFO: running tests with moreutils parallel" - $TEST_TIMEOUT_CMD parallel -- $TESTS + $TEST_TIMEOUT_CMD parallel -- $TESTS || RES=$? fi - RES=$? - if [[ $RES != 0 ]]; then + if [ $RES != 0 ]; then META_FAILURE="parallel test suite returned error code $RES" fi else + trap 'e=$?; trap - 0; kill ${!-}; exit $e' 0 HUP INT TERM for test in $TESTS; do $TEST_TIMEOUT_CMD $test "$@" & - wait $! - # If the test failed without producing results, then it aborted, - # so we should abort, too. - RES=$? - testname=$(basename $test .sh) - if [[ $RES != 0 && ! -e "$NOTMUCH_BUILDDIR/test/test-results/$testname" ]]; then - META_FAILURE="Aborting on $testname (returned $RES)" - break - fi + wait $! && ev=0 || ev=$? + test $ev = 0 || RES=$ev done + trap - 0 HUP INT TERM + if [ $RES != 0 ]; then + META_FAILURE="some tests failed; first failed returned error code $RES" + fi fi -trap - HUP INT TERM # Report results +RESULT_FILES= +for file in $TESTS +do + file=${file##*/} # drop leading path components + file=${file%.sh} # drop trailing '.sh' + RESULT_FILES="$RESULT_FILES $TEST_DIRECTORY/test-results/$file" +done + echo -$NOTMUCH_SRCDIR/test/aggregate-results.sh $NOTMUCH_BUILDDIR/test/test-results/* -ev=$? +$NOTMUCH_SRCDIR/test/aggregate-results.sh $RESULT_FILES && ev=0 || ev=$? + if [ -n "$META_FAILURE" ]; then printf 'ERROR: %s\n' "$META_FAILURE" if [ $ev = 0 ]; then @@ -89,6 +102,6 @@ if [ -n "$META_FAILURE" ]; then fi # Clean up -rm -rf $NOTMUCH_BUILDDIR/test/test-results +rm -rf $TEST_DIRECTORY/test-results exit $ev diff --git a/test/notmuch-test.h b/test/notmuch-test.h index df852da9..ed713099 100644 --- a/test/notmuch-test.h +++ b/test/notmuch-test.h @@ -1,6 +1,21 @@ #ifndef _NOTMUCH_TEST_H #define _NOTMUCH_TEST_H +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include +#include +#include #include +#include +#include +#include +#include +#include +#include +#include + #include inline static void @@ -13,4 +28,23 @@ expect0 (int line, notmuch_status_t ret) } #define EXPECT0(v) expect0 (__LINE__, v); + +inline static void * +dlsym_next (const char *symbol) +{ + void *sym = dlsym (RTLD_NEXT, symbol); + char *str = dlerror (); + + if (str != NULL) { + fprintf (stderr, "finding symbol '%s' failed: %s", symbol, str); + exit (77); + } + return sym; +} + +#define WRAP_DLFUNC(_rtype, _func, _args) \ + _rtype _func _args; \ + _rtype _func _args { \ + static _rtype (*_func##_orig) _args = NULL; \ + if (! _func##_orig ) *(void **) (&_func##_orig) = dlsym_next (#_func); #endif diff --git a/test/openpgp4-secret-key.asc b/test/openpgp4-secret-key.asc new file mode 100644 index 00000000..182a8249 --- /dev/null +++ b/test/openpgp4-secret-key.asc @@ -0,0 +1,15 @@ +-----BEGIN PGP PRIVATE KEY BLOCK----- + +lFgEYxhtlxYJKwYBBAHaRw8BAQdA0PoNKr90DaQV1dIK77wbWm4RT+JQzqBkwIjA +HQM9RHYAAQDQ5wSfkOGXvKYroALWgibztISzXS5b8boGXykcHERo6w/ctDtOb3Rt +dWNoIFRlc3QgU3VpdGUgKElOU0VDVVJFISkgPHRlc3Rfc3VpdGVAbm90bXVjaG1h +aWwub3JnPoiQBBMWCAA4AhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEEmjr+ +bGAGWhSP1LWKfmq+kkZFzGAFAmMYbZwACgkQfmq+kkZFzGDtrwEAjQRn3xhEomah +wICjQjfi4RKNbvnRViZgosijDBANUAgA/28GrK1tPnQsXWqmuZxQ1Cd5ry4NAnj/ +4jsxD3cTbnEHnF0EYxhtlxIKKwYBBAGXVQEFAQEHQEOd3EyCD5qo4+QuHz0lruCG +VM6n6RI4dtAh3cX9uHwiAwEIBwAA/1oe+p5jNjNE5lEj4yTpYjCxCeC98MolbiAy +0yY7526wECqIeAQYFggAIBYhBJo6/mxgBloUj9S1in5qvpJGRcxgBQJjGG2XAhsM +AAoJEH5qvpJGRcxgBdsA/R9ZECoxai5QhOitDIAUZVCRr59Pm1VMPiJOOIla2N1p +AQCNESwJ9IJOdO/06q+bR2GG4WyEkB4VoVBiA3hFx/zZAA== +=uGTo +-----END PGP PRIVATE KEY BLOCK----- diff --git a/test/openpgp4-secret-key.asc.NOTE b/test/openpgp4-secret-key.asc.NOTE new file mode 100644 index 00000000..4693768e --- /dev/null +++ b/test/openpgp4-secret-key.asc.NOTE @@ -0,0 +1,5 @@ +The OpenPGPv4 secret key for the crypto tests was generated using: + +$ gpg --quick-generate-key \ + 'Notmuch Test Suite (INSECURE!) ' \ + future-default default never diff --git a/test/random-corpus.c b/test/random-corpus.c index 8ed7ff76..8ae08971 100644 --- a/test/random-corpus.c +++ b/test/random-corpus.c @@ -50,7 +50,7 @@ typedef struct { /* * Choose about half ascii as test characters, as ascii - * punctation and whitespace is the main cause of problems for + * punctuation and whitespace is the main cause of problems for * the (old) restore parser. * * We then favour code points with 2 byte encodings. Note that @@ -122,7 +122,8 @@ const notmuch_opt_desc_t notmuch_shared_options[] = { const char *notmuch_requested_db_uuid = NULL; void -notmuch_process_shared_options (unused (const char *dummy)) +notmuch_process_shared_options (unused (notmuch_database_t *notmuch), + unused (const char *dummy)) { } @@ -141,7 +142,6 @@ main (int argc, char **argv) void *ctx = talloc_new (NULL); const char *config_path = NULL; - notmuch_config_t *config; notmuch_database_t *notmuch; int num_messages = 500; @@ -179,17 +179,18 @@ main (int argc, char **argv) exit (1); } - config = notmuch_config_open (ctx, config_path, false); - if (config == NULL) - return 1; - - if (notmuch_database_open (notmuch_config_get_database_path (config), - NOTMUCH_DATABASE_MODE_READ_WRITE, ¬much)) + if (notmuch_database_open_with_config (NULL, + NOTMUCH_DATABASE_MODE_READ_WRITE, + config_path, + NULL, + ¬much, + NULL)) return 1; srandom (seed); int count; + for (count = 0; count < num_messages; count++) { int j; /* explicitly allow zero tags */ diff --git a/test/setup.expected-output/config-with-comments b/test/setup.expected-output/config-with-comments new file mode 100644 index 00000000..d925acea --- /dev/null +++ b/test/setup.expected-output/config-with-comments @@ -0,0 +1,81 @@ +# .notmuch-config - Configuration file for the notmuch mail system +# +# For more information about notmuch, see https://notmuchmail.org +# Database configuration +# +# The only value supported here is 'path' which should be the top-level +# directory where your mail currently exists and to where mail will be +# delivered in the future. Files should be individual email messages. +# Notmuch will store its database within a sub-directory of the path +# configured here named ".notmuch". +# +[database] +path=/path/to/maildir +# User configuration +# +# Here is where you can let notmuch know how you would like to be +# addressed. Valid settings are +# +# name Your full name. +# primary_email Your primary email address. +# other_email A list (separated by ';') of other email addresses +# at which you receive email. +# +# Notmuch will use the various email addresses configured here when +# formatting replies. It will avoid including your own addresses in the +# recipient list of replies, and will set the From address based on the +# address to which the original email was addressed. +# +[user] +name=Test Suite +primary_email=test.suite@example.com +other_email=another.suite@example.com +# Configuration for "notmuch new" +# +# The following options are supported here: +# +# tags A list (separated by ';') of the tags that will be +# added to all messages incorporated by "notmuch new". +# +# ignore A list (separated by ';') of file and directory names +# that will not be searched for messages by "notmuch new". +# +# NOTE: *Every* file/directory that goes by one of those +# names will be ignored, independent of its depth/location +# in the mail store. +# +[new] +tags=foo;bar; +# Search configuration +# +# The following option is supported here: +# +# exclude_tags +# A ;-separated list of tags that will be excluded from +# search results by default. Using an excluded tag in a +# query will override that exclusion. +# +[search] +exclude_tags=baz +# Maildir compatibility configuration +# +# The following option is supported here: +# +# synchronize_flags Valid values are true and false. +# +# If true, then the following maildir flags (in message filenames) +# will be synchronized with the corresponding notmuch tags: +# +# Flag Tag +# ---- ------- +# D draft +# F flagged +# P passed +# R replied +# S unread (added when 'S' flag is not present) +# +# The "notmuch new" command will notice flag changes in filenames +# and update tags, while the "notmuch tag" and "notmuch restore" +# commands will notice tag changes and update flags in filenames +# +[maildir] diff --git a/test/smime/0xE0972A47.p12 b/test/smime/0xE0972A47.p12 new file mode 100644 index 00000000..2c4a6d17 --- /dev/null +++ b/test/smime/0xE0972A47.p12 @@ -0,0 +1,62 @@ +Issuer ...: /CN=Notmuch Test Suite +Serial ...: 6F748C94BD0C67A9 +Subject ..: /CN=Notmuch Test Suite + aka ..: test_suite@notmuchmail.org +Keygrip ..: 1727B9C7108D50333614F3B1DD0807F624B31130 + +-----BEGIN PKCS12----- +MIIJ+AIBAzCCCb4GCSqGSIb3DQEHAaCCCa8EggmrMIIJpzCCBAcGCSqGSIb3DQEH +BqCCA/gwggP0AgEAMIID7QYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQIcfMY +MS7tOpcCAggAgIIDwFu7ZRNrXCb0eKei44aeBZPRs9YI/5EpMcFuc8j4/8T1HkIt +GuRe/HzRmoiLZcAMOzGC/hF8TkHlNeUZ7rOSpCg4UlBVWJS6avTMHHsakDvTV/7q +X5VNi4pLUuyEToGTAPHV+s5P/gYYG6mFPkwG/pDDlAcgMhgtuPY/lQp6IS/E6CaR +fhcnQiPq9ySTqO7UNwIyMwtAtSHkgBaje8UbOkQch4lg51i97rm9m4EMvklKtjXc +Ud4aTEuoZguPmdBdLvF5QxqJf6Bm9lHa1Awhru2gBWQf9TjX8bwK9Xsv8G6gPOwc +LVpIR9fMZtgBbc+heeJTjfn6VqEy881ckbkz+38hiN3pbLMuATM7QAY3u3N4whM6 +Hmfyl3iqba84Pl93zaUzqazAUeFdqcqSpAUGkS4gU6klr9qi3NicaGbry1DySYU7 +2h4xy3j7eiHxqdWaibdPoBC8CEbPaFj2qnOVsZykxG6zPvbEB+5sJ/a+T6xm1Btx +N6vXR7ObbXlpC4pRkS32ehuRbY6wc6H2KKepOMCu7x10tN0Up5ccNxvkT26QIrEE +LW296ijCLbsRhWymDtopWAZHcXXIu0fJ4tocSp2c3lojSEYu1jlMXR+Pa4R8EtgZ +lb5+NqISxjUlMMWzGDyhrp9ImcsZmpv6N8zPcZVyU+M1/h+p9ur/IOVZU9P1vIKy +kcM4pslr0JhLfnZCLZ+3Ux1yKAcndGZFPb1vZ83jyZKR38BVSGu53ODaBJBqSMHu +Mv2Na/qzvQBSVJuWF9cAhiVd7v9R/EvT0zmljN4w7l4EXsB5wRsO1wvlL+MhwaET +dIHbRH2GD3gERX6oTc3t3cgritVePk70rCxQDxn5zUbjW7dNIlIobAumLHBfgSxR +QCE6gxdTm5MW2O9hnfTSQvliVaGU1gd0M3BRiqeNpPPxnloGKnOEODM381F4HxyR +CzO2r/2aKJP+U5HxSf4cljp3/Lripxykzfqc9/xZshl+jGixsSSm+Ul916Hpj2Rt +j9vHg4H9YfJTGdvzxZcvZCvNSy3ygtjx0++SrI5hGHKjpVJIK2/9Wi39q5s6LkiA +RCjvuoBBcQXm++69X7QGWSsGFtwerCGnq3nAxGpHVKVGTvFYMAg6y1RR0zvE0SuM +MZegD8w45QyrmiPqSRM7/RtqVdA+r/wiJwWerUBq+mrCvJHB2NRcjiUiCJY1bjRU +ATMfB0uZaNInUXiLDGxp2mdBgdFVq7sYTbq+OvprzxeAjIvodxl3J9ThvJnt1fzK +RPCJw5COI60ibE3XTTCCBZgGCSqGSIb3DQEHAaCCBYkEggWFMIIFgTCCBX0GCyqG +SIb3DQEMCgECoIIE7jCCBOowHAYKKoZIhvcNAQwBAzAOBAiEe8CcxIIv9wICCAAE +ggTIujut93lYPUsKc/JNhZhUWS/RHHog6d8ZAjpFvXpyD8Z2z4A4PpgIn8eUSRW5 +Gwp8izR+16Tj3ht52pJ5Y1x27/S3l3sDlekEZ/33X/AdLFWAXbcibmwtRea1ucKZ +ze3DJM7CvuRvVSBG8XubPGi3pZkEjHBGQqgtsTnxlBp0PXl7wxfyT7F6gOH2DGYP +bYzNa2fnY8twEcUYhuksI/eh9Zwj9TrF0HWq1hwp0tDCfqutzshSX2GQ/p0raL3B +C2stHBjl0OVUfDHpqQ5OJWbQvGcJntECqu4gmSJohunObaUKcN8xs+FzB5czpmsT +W/pyR58nc8QhTttByqZN3EerhEogWDZj4tQ6dK8p6bqLO/0qqBehZGchfof5Evwj +VFsvVGD8xVLQWWAFnrQs5+U56NQEbmZzN5RCI7FEK2VVOeG03dpXyoAQyxuYrsYU +3znmoSleIqDDBFD21YePUcJZ0R8AQsvgV11tdwPWqr1hk0bIazLQ9rappGrTgkK8 +DFdQKSH1dRvjqtbuDyY7j5PXXJTXthVv9T9N7Vp6qU+pWBQ1Mz30J+fHX2ilEnbi +tQ49hwt1+/2Zkmwz3reoEnxYOKzCg/ySIpQ27/Hx4xZ+ecEzX/0IxCkHeAV3V3bB +1z8wFxWEh1s9hL6C8lRk/wQ9KsKaxM7BdLw7RjiqEwR4HgeCqMPdCVQQpILARDC8 +Poz8xUmjv7HyIvvyBUP12YdIj74Jjj0Mm2r/FDj7nsXxkjXMZEMMKK3oVaAMq8Bd +cO4VQXDd7bgNzLF9PKxWNjoCuQcPJXwMPqlFoc06BLPstEaR4enafv0Pd4l0pyME +YgezyVW+3yFEsbbB2UUs0r7oqxsDFU9/iHf8O3nu3NuKTJkux4uMlOTBKsm6sY7k +GduP2UA+WU27jHrf4zQQbkDLG1lJFfcaKzlcOmz5B9iZwugBz9Y28w5f2/12Kqrh +4tibFBUG0E85KAb1wnFUNUx06OMX229U1M0E1LHbcUJ9mcRipONPVn0FRi8XzaLK +023XRoihuoWhVUiB1OJ2eZW1JnUYRztfa3nfmGjXv4VGkxYlnTkE9z0PAAhf6t5A +7Ir0y1JUeOlBITTcojOp6qQ8tMQQ5wRk1oncHiw3WwJvFN6fOa9Q/+4ZmULHz0vV +Xl+Qio8B7/4jqZoT4e/gK6U/zHriznLzqp63LjP47eFRXTfuXslaCt7YF75Mq2J6 +VPA+qfYRw0K5BvDUkr8c+nLP2AiDaEYVBHGdBRTlWO9UkcB1F4cuZZiU5MZbxVrb +Db+zGWW6AT+4XTO4z9KmAqgTTv1+BQrLxNI+RG8JfQapUKQyB794F4kXK2yhd1P3 +XS9cwh24COiqbOpI1nB5qn7cn4RRHW156LWGF+VJFdxR6Wu3vZx/kZGevG9o1ARF +z1l9mbGyhwnUJO1EQwjbppvRou1bZuNbuRgLmHKEVPAv+J+7hLXZAnRdwoV0x91t +bpmy4qyxA/90DHguIhRVcKsYBrdShY7LXdZArECBhMY9R41D6v1yyhC6fL6PKR5g +DaluN2K9TBALzZH7NnNdE14l+56+kLc9Fq8JXsq3rxdeBTsNl09fHPf9w5VLkq4I +doNcPPlta0Q0xJNa/RYENCJpAMZdMFIJ558uMXwwVQYJKoZIhvcNAQkUMUgeRgBH +AG4AdQBQAEcAIABlAHgAcABvAHIAdABlAGQAIABjAGUAcgB0AGkAZgBpAGMAYQB0 +AGUAIABlADAAOQA3ADIAYQA0ADcwIwYJKoZIhvcNAQkVMRYEFGFvRs1zg0xjhHdW +rw37ZKbglypHMDEwITAJBgUrDgMCGgUABBSluQBa+tVpYVYmB/zAZuPE9NnargQI +XWSQTDEONWgCAggA +-----END PKCS12----- diff --git a/test/smime/README b/test/smime/README index 92803c77..6f276398 100644 --- a/test/smime/README +++ b/test/smime/README @@ -2,6 +2,8 @@ test.crt: self signed certificated % gpgsm --gen-key # needs gpgsm 2.1 key+cert.pem: cert + unencryped private - % gpsm --import test.crt + % gpgsm --import test.crt % gpgsm --export-private-key-p12 -out foo.p12 (no passphrase) % openssl pkcs12 -in ns.p12 -clcerts -nodes > key+cert.pem + +ca.crt: from https://tools.ietf.org/id/draft-dkg-lamps-samples-01.html#name-certificate-authority-certi diff --git a/test/smime/bob.p12 b/test/smime/bob.p12 new file mode 100644 index 00000000..774c77d0 --- /dev/null +++ b/test/smime/bob.p12 @@ -0,0 +1,58 @@ +-----BEGIN PKCS12----- +MIIKWAIBAzCCCh4GCSqGSIb3DQEHAaCCCg8EggoLMIIKBzCCBGcGCSqGSIb3DQEH +BqCCBFgwggRUAgEAMIIETQYJKoZIhvcNAQcBMBwGCiqGSIb3DQEMAQYwDgQICE8J +3kMad9UCAggAgIIEIPvHjK0eRQrnowMUsz1z1x/IxslNvG6DjPZjNHCkNYYmiRsg +Leu5nqKf4emWVvYpnlh+4Gql7pyJm3G3zSNhobPkW+P1Eh80tTBoUk7TIvvvmtrE +YEc/nRR1p1MgjISq4Q/CM6ccCCw6YEiQcj/0mSS7gmHUegD5glcWbVuqAT8M/p1z +98OP3z37G8ARRLNj1yyp0SVlt59Sx3WNbmYBqkQ96iukjMJvmjV7o6BFYUx46Llb +tphhdRgKXbK2r1R0TUlvE659TUwlrpGlaFpaGj1kLdzVAnjh1ZWnWO2a2BSj0LzG +qRyiLwqDFPLJLQEckfV+RPWiRrSewME8URNKdk6eewtHdhrehMo4ZJnOIum8qcSz +giW61SSyZJsFvILpmMYghIxWmPd/8cNIHBrdFEa7z3QKh5jcJNTCxz6yO9f8F830 +d+WDK7DbGkUW4mVTGg/lEYnCFZDF6S1mr0hx+cew1FbKjLpxfQllIIrLf5d2BF8H +0STpuylQDVVBFdTRHyeS6td5nulANgOProrRzy3aAKQmZ6iullKl+i2t/2TwfVP/ +gG+yszpOEf8U9txuvbiZ7j4XV158zdaaGiduDqMKLOvbdctwHAsR9ecx5C3NTRDl +ZlttNoXN9zhT4CkWk1w4sFk2KUurjVraIcjWVT7yOreaaK+6N09M0tnLPDJDTrow +8WwP/rZhA+t+CMrhqkFBxXsyo5VTM0jWJGO/NLpYXPhDPBsRq8rs1OCrUoVr34aR +cpUTNhyXkvJUarWDHs88lg0ps0G9/1dXI1AbEsQQg8u+QT2ztGYrg2OQxQyi1Mo4 +u/FkAcEbtlYYLmJjj/S2qVRPJgBALVjw9k5hnYRdAXWVDCJ96PMn1SKORvlMxnZ7 +djlhaztOhTLsiDzywVDYWLvQElunWcAGeDZykWNytwcEagc0VjWKHMibc0JOZQ1T +crGyOzTlt09xHj1NrItYefIwdtKuJfkAh03B5xI6rJ9ZbK9xidcVxyeRX0lEqdo9 +WHQrhHefAmeyo0TlfsN67kFDp5FLpwEtNaN0lyzpkl30aWZdtP5vkvtfmy5ugYIO +bXoVa+tO6k5V/VfUFUKdaY7xAX7XRzUUg4jB0D0CuaX+YS+GL+5wuQwIY1y2ihBb +CuCxlcP1lVEU4CVQba60VTudJtWyE7QpPhf+y81f1wRjwIihFvwzpUFWf8JVEppe +v3Yot3OWGBmhEqLkC9LELth8o5gLfyYHaXTYNd9aRTiI+0ZC5U3O4wUwYLTG3exM +rIDTzEMk/p4DYIHkNKVUiRJfGYdAwuRxf3IMcYWARTXlSzl1C3hWmZfvTPlKs1bB +OHTHP/P+qdOFjxOh+fbyqXPJauBAhHvHgrp3iI6t834wJou26oWNihM7OnWuyQRt +9DVxG4l+1VjtbQZfTDCCBZgGCSqGSIb3DQEHAaCCBYkEggWFMIIFgTCCBX0GCyqG +SIb3DQEMCgECoIIE7jCCBOowHAYKKoZIhvcNAQwBAzAOBAjqo0x2p5SqLAICCAAE +ggTIe6Ws+lu0CoNlCXGM2BEPV09wuRHTJe+KnesrmRbXPF9linG3d6G++tTkBHz/ +yr77/DV5aDYciV1pGAbLuX2lMwuqdxzJ4OBPBAjuX5H+IPRaTbxfHYYIwhG8oZzy +aHyVhHr9j0h7lzW7xSTYJuBNEJ58L42dfzpNRw9dyRPmcuhZqW14Z3xyDm8yjHfB +2p99y9/A4qSyJJSUM3O3nLdtIar3ktSTRAijgqq+s9wnsfozQRzWpYaqiRrdzwfO +HqXk54l3/lMSyLpfPl9LW7er6JbGI4jEyQ3x8WijATM5h/lkZKejh/mOaWCvs6G6 +fGzV4P35EsToYbOk9GX4jl4SyDBt3iEHYm5teDUhJmTcR39lAQuAfxN6rOn/TkoO +YLxtdD5DLiTfYZPCFyavLEsamr8A4p93torF6Rs7GsaHE6PmCcprzqx71KV0DZKv +tMY86RoiWPKLFxZcYt1yz9/95c1SO1s4i1GvLpJTEgQxLM2OhfEwDNKd2rMJoq1I +YIRPSP204dIVwwNdXN1vB2slhN2+/QMOqsEkWtTOpW2QoTGSze49hfmJGdu+91jd +XZBBMJQfY4q066/eE4IOW7ZZId5uMYxDRnGdEQjJsxyW8YHWLRGQvBC8gMkdbj8e +0wkXbe+jML7vG7t3hDhLEbj5sTquIMTWrTirPw4SxLCuGZAyJHFN3/nCaOSMFlCG +wEZHrAozgQXPBYU7p+uIkJ4lDc2ZtW8NM8U15gKZLDFfAE6Vg0jAtfFMqvNnX630 +xfo1z4jBd7VXbBFrPzrmvlTnb1XxNFcPycowzW9tgtN4YnNroCq98VpMC914tdpJ ++C/PI0eJ7M2ir3ajN0RabSm02JO9Hdwoa5OgqLwPYDwiFyQvKFGKqAF8Ph6pSEiZ +10OnH+DVgEY70A+Le+ZSDosMdrhZfHbCcIFitZJ3sYV/7Q118QckW3szcjmLHS5g +M6Whl2HhjLsAfsmCnoRlIwjx4g0TiuZcb4hGysq8QjD3Z8qqFK28m6OMHbASQfWg +U+Qg3vmEvVsnBxStFEIImS3QYQoaT0pk6zKUYsI/fOBnEgxsY0XwTfXzVw7hZDct +yhNIQVWmfgVZwUw0wLoNu3A5hupjUwQzQr4TPnKkFPI8qHmRrJgP8EA0U0019y3W +MlK0h/LAJEaUBS0goLJCJ8+1EWr6femjnyuU5hMizOm+3j0JexjWz5TQttioS7Q/ +vcxt5pA9yAWQdH9j72saKEoKmDi+kIPr4mimKJz99LhKp9A6Hj0f1P2V3As8JWyW +ZKmJKW7qMMCFADlALolobqzA60j6Zeo5jiEj/j2lVlUPPz47WO+uKeb+rx+hgTUc +Xrhq0+an5tvEXt/8wy3PJFqP+qqHGhOIuPLuhqPyzNowuXirIXsiWnI44/X48W91 +HPEoL3xaebQ6oyTP8dI4CCkkHgiLWL5mskjHMEXvcdR6k0ygmu8DGQCPfUweUZqZ +wfkhD/jwbVpLR5Y3chpatW0cJ2bsAWdxwtuxF05+fVEePUsR0x+2/v/8eDEHKYwt +aYlAhI48nyrKKVMmqvqcXnzmJlUaq05GnEcglFbv4MUExL7CxClls6QnVNiZFPrV +ffVsYT2A300xrm4pan89n3nuavjJn7L1JJdmMXwwVQYJKoZIhvcNAQkUMUgeRgBH +AG4AdQBQAEcAIABlAHgAcABvAHIAdABlAGQAIABjAGUAcgB0AGkAZgBpAGMAYQB0 +AGUAIAA0ADIAYgBiADIANAAwADYwIwYJKoZIhvcNAQkVMRYEFGaI9k+ZdE9/rxBZ +4rSdH1BCuyQGMDEwITAJBgUrDgMCGgUABBRJfL4XyIHpXmjbziCGCbSAOK9jKgQI +drOMeIgXcCYCAggA +-----END PKCS12----- diff --git a/test/smime/ca.crt b/test/smime/ca.crt new file mode 100644 index 00000000..b33d087f --- /dev/null +++ b/test/smime/ca.crt @@ -0,0 +1,20 @@ +-----BEGIN CERTIFICATE----- +MIIDLTCCAhWgAwIBAgIULXcNXGI2bZp38sV7cF6VcQfnKDwwDQYJKoZIhvcNAQEN +BQAwLTErMCkGA1UEAxMiU2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0 +eTAgFw0xOTExMjAwNjU0MThaGA8yMDUyMDkyNzA2NTQxOFowLTErMCkGA1UEAxMi +U2FtcGxlIExBTVBTIENlcnRpZmljYXRlIEF1dGhvcml0eTCCASIwDQYJKoZIhvcN +AQEBBQADggEPADCCAQoCggEBAMUfZ8+NYSh6h36zQcXBo5B6ficAcBJ1f3aLxyN8 +QXB83XuP8aDRWQ9uJvJpQkWVH4zx96/E/zI0t0lDMYtZNqra16h+gxbHJgoq2pRw +RCOiyYu/p2vzvvZ1dtFTMc/mIigjA/73kokui62j1EFy//fNVIihkVS3rAweq+fI +8qJHSMhdc2aYa9wOP0eGe/HTiDYgT4L4f2HTGMGGwQgj1vub0gpR4YHmNqr0GyEA +63mHUQUZpnmN1FEl+nVFA5Ntu4uF++qf/tkTji89/eXYBdKX2yUdTeTIKoCI65IL +EXxezjTc8aFjf/8E0aWGVZR/DtCsjWOh/s/mV7n/YPyb4+ECAwEAAaNDMEEwDwYD +VR0TAQH/BAUwAwEB/zAPBgNVHQ8BAf8EBQMDBwYAMB0GA1UdDgQWBBS3Uk1zwIg9 +ssN6WgzzlPf3gKJ32zANBgkqhkiG9w0BAQ0FAAOCAQEALsU91Bmhc6EgCNr7inY2 +2gYPnosJ+kZ1eC0hvHIK9e0Tx74RmhTOe8M2C9YXQKehHpRaX+DLcjup6scoH/bT +u0THbmzeOy29TTiFcyV9BK+SEKQWW4s98Fwdk9fPWcflHtYvqxjooAV3vHbt6Xmp +KrKDz/jdg7t0ptI4zSqAf3wNppiJoswlOHBUnH2W1MIYkWQ4jYj5socblVlklHOr +ykKUiEZAbjU+C1+0FhT4HgLjBB9R4H1H0JRKsggWiZBBJ6UpN0dTN4iD0mDVa0jy +sJqqWnIViy/xaSDcNaWJmU3o2KmkMkdpinoJ5uLkAHQqXjFaujdU1PkufeA7v3uG +Rw== +-----END CERTIFICATE----- diff --git a/test/symbol-test.cc b/test/symbol-test.cc index 9d73a571..9e956ddf 100644 --- a/test/symbol-test.cc +++ b/test/symbol-test.cc @@ -12,8 +12,10 @@ main (int argc, char **argv) if (argc != 3) return 1; - if (notmuch_database_open_verbose (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, - ¬much, &message)) { + if (notmuch_database_open_with_config (argv[1], NOTMUCH_DATABASE_MODE_READ_ONLY, + "", + NULL, + ¬much, &message)) { if (message) { fputs (message, stderr); free (message); diff --git a/test/test-databases/.gitignore b/test/test-databases/.gitignore deleted file mode 100644 index 9452199f..00000000 --- a/test/test-databases/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/*.tar.xz diff --git a/test/test-databases/Makefile b/test/test-databases/Makefile deleted file mode 100644 index b250a8be..00000000 --- a/test/test-databases/Makefile +++ /dev/null @@ -1,7 +0,0 @@ -# See Makefile.local for the list of files to be compiled in this -# directory. -all: - $(MAKE) -C ../.. all - -.DEFAULT: - $(MAKE) -C ../.. $@ diff --git a/test/test-databases/Makefile.local b/test/test-databases/Makefile.local deleted file mode 100644 index 7aedff70..00000000 --- a/test/test-databases/Makefile.local +++ /dev/null @@ -1,20 +0,0 @@ -# -*- makefile -*- - -TEST_DATABASE_MIRROR=https://notmuchmail.org/releases/test-databases - -dir := test/test-databases - -test_databases := $(dir)/database-v1.tar.xz - -%.tar.xz: - @exec 1>&2 ;\ - if command -v wget >/dev/null ;\ - then set -x; wget -nv -O $@ ${TEST_DATABASE_MIRROR}/$(notdir $@) ;\ - elif command -v curl >/dev/null ;\ - then set -x; curl -L -s -o $@ ${TEST_DATABASE_MIRROR}/$(notdir $@) ;\ - else echo Cannot fetch databases, no wget nor curl available; exit 1 ;\ - fi - -download-test-databases: ${test_databases} - -DATACLEAN := $(DATACLEAN) ${test_databases} diff --git a/test/test-databases/database-v1.tar.xz.sha256 b/test/test-databases/database-v1.tar.xz.sha256 deleted file mode 100644 index 2cc4f965..00000000 --- a/test/test-databases/database-v1.tar.xz.sha256 +++ /dev/null @@ -1 +0,0 @@ -4299e051b10e1fa7b33ea2862790a09ebfe96859681804e5251e130f800e69d2 database-v1.tar.xz diff --git a/test/test-lib-common.sh b/test/test-lib-common.sh index 2f7950ac..f5d72e12 100644 --- a/test/test-lib-common.sh +++ b/test/test-lib-common.sh @@ -24,11 +24,25 @@ # type die >/dev/null 2>&1 || die () { echo "$@" >&2; exit 1; } -if [[ -z "$NOTMUCH_SRCDIR" ]] || [[ -z "$NOTMUCH_BUILDDIR" ]]; then +if [[ -z "$NOTMUCH_SRCDIR" ]] || [ -z "${NOTMUCH_TEST_INSTALLED-}" -a -z "$NOTMUCH_BUILDDIR" ]; then echo "internal: srcdir or builddir not set" >&2 exit 1 fi +# Explicitly require external prerequisite. Useful when binary is +# called indirectly (e.g. from emacs). +# Returns success if dependency is available, failure otherwise. +test_require_external_prereq () { + local binary + binary="$1" + if [[ ${test_missing_external_prereq_["${binary}"]} == t ]]; then + # dependency is missing, call the replacement function to note it + eval "$binary" + else + true + fi +} + backup_database () { test_name=$(basename $0 .sh) rm -rf $TMP_DIRECTORY/notmuch-dir-backup."$test_name" @@ -47,7 +61,9 @@ LD_LIBRARY_PATH=${TEST_DIRECTORY%/*}/lib${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH} export LD_LIBRARY_PATH # configure output -. "$NOTMUCH_BUILDDIR/sh.config" || exit 1 +if [ -z "${NOTMUCH_TEST_INSTALLED-}" ]; then + . "$NOTMUCH_BUILDDIR/sh.config" || exit 1 +fi # load OS specifics if [[ -e "$NOTMUCH_SRCDIR/test/test-lib-$PLATFORM.sh" ]]; then @@ -105,8 +121,7 @@ fi gen_msg_cnt=0 gen_msg_filename="" gen_msg_id="" -generate_message () -{ +generate_message () { # This is our (bash-specific) magic for doing named parameters local -A template="($@)" local additional_headers @@ -225,8 +240,7 @@ EOF # # All of the arguments and return values supported by generate_message # are also supported here, so see that function for details. -add_message () -{ +add_message () { generate_message "$@" && notmuch new > /dev/null } @@ -303,7 +317,12 @@ export PATH MANPATH # Test repository test="tmp.$(basename "$0" .sh)" -TMP_DIRECTORY="$TEST_DIRECTORY/$test" +if [ -z "${NOTMUCH_TEST_INSTALLED-}" ]; then + TMP_DIRECTORY="$TEST_DIRECTORY/$test" +else + TMP_DIRECTORY=$(mktemp -d "${TMPDIR:-/tmp}/notmuch-$test.XXXXXX") +fi + test ! -z "$debug" || remove_tmp=$TMP_DIRECTORY rm -rf "$TMP_DIRECTORY" || { GIT_EXIT_OK=t @@ -311,6 +330,10 @@ rm -rf "$TMP_DIRECTORY" || { exit 1 } +# Provide a guess at a usable Python, to support running tests without +# running configure first. +NOTMUCH_PYTHON=${NOTMUCH_PYTHON-python3} + # A temporary home directory is needed by at least: # - emacs/"Sending a message via (fake) SMTP" # - emacs/"Reply within emacs" diff --git a/test/test-lib-emacs.sh b/test/test-lib-emacs.sh new file mode 100644 index 00000000..0ab58fc2 --- /dev/null +++ b/test/test-lib-emacs.sh @@ -0,0 +1,226 @@ +# +# Copyright (c) 2010-2020 Notmuch Developers +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see https://www.gnu.org/licenses/ . + +test_require_emacs () { + local ret=0 + test_require_external_prereq "$TEST_EMACS" || ret=1 + test_require_external_prereq "$TEST_EMACSCLIENT" || ret=1 + test_require_external_prereq dtach || ret=1 + return $ret +} + +# Deliver a message with emacs and add it to the database +# +# Uses emacs to generate and deliver a message to the mail store. +# Accepts arbitrary extra emacs/elisp functions to modify the message +# before sending, which is useful to doing things like attaching files +# to the message and encrypting/signing. +emacs_deliver_message () { + local subject body smtp_dummy_pid smtp_dummy_port + test_subtest_broken_for_installed + subject="$1" + body="$2" + shift 2 + # before we can send a message, we have to prepare the FCC maildir + mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} + # eval'ing smtp-dummy --background will set smtp_dummy_pid and -_port + smtp_dummy_pid= smtp_dummy_port= + eval `$TEST_DIRECTORY/smtp-dummy --background sent_message` + test -n "$smtp_dummy_pid" || return 1 + test -n "$smtp_dummy_port" || return 1 + + test_emacs \ + "(let ((message-send-mail-function 'message-smtpmail-send-it) + (mail-host-address \"example.com\") + (smtpmail-smtp-server \"localhost\") + (smtpmail-smtp-service \"${smtp_dummy_port}\")) + (notmuch-mua-mail) + (message-goto-to) + (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") + (message-goto-subject) + (insert \"${subject}\") + (message-goto-body) + (insert \"${body}\") + $* + (let ((mml-secure-smime-sign-with-sender t) + (mml-secure-openpgp-sign-with-sender t)) + (notmuch-mua-send-and-exit)))" + # In case message was sent properly, client waits for confirmation + # before exiting and resuming control here; therefore making sure + # that server exits by sending (KILL) signal to it is safe. + kill -9 $smtp_dummy_pid + notmuch new >/dev/null +} + +# Pretend to deliver a message with emacs. Really save it to a file +# and add it to the database +# +# Uses emacs to generate and deliver a message to the mail store. +# Accepts arbitrary extra emacs/elisp functions to modify the message +# before sending, which is useful to doing things like attaching files +# to the message and encrypting/signing. +# +# If any GNU-style long-arguments (like --quiet or --decrypt=true) are +# at the head of the argument list, they are sent directly to "notmuch +# new" after message delivery +emacs_fcc_message () { + local nmn_args subject body + nmn_args='' + while [[ "$1" =~ ^-- ]]; do + nmn_args="$nmn_args $1" + shift + done + subject="$1" + body="$2" + shift 2 + # before we can send a message, we have to prepare the FCC maildir + mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} + + test_emacs \ + "(let ((message-send-mail-function (lambda () t)) + (mail-host-address \"example.com\")) + (notmuch-mua-mail) + (message-goto-to) + (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") + (message-goto-subject) + (insert \"${subject}\") + (message-goto-body) + (insert \"${body}\") + $* + (let ((mml-secure-smime-sign-with-sender t) + (mml-secure-openpgp-sign-with-sender t)) + (notmuch-mua-send-and-exit)))" || return 1 + notmuch new $nmn_args >/dev/null +} + +test_emacs_expect_t () { + local result + test "$#" = 1 || + error "bug in the test script: not 1 parameter to test_emacs_expect_t" + if [ -z "$inside_subtest" ]; then + error "bug in the test script: test_emacs_expect_t without test_begin_subtest" + fi + + # Run the test. + if ! test_skip "$test_subtest_name" + then + test_emacs "(notmuch-test-run $1)" >/dev/null + + # Restore state after the test. + exec 1>&6 2>&7 # Restore stdout and stderr + inside_subtest= + + # test_emacs may update missing external prerequisites + test_check_missing_external_prereqs_ "$test_subtest_name" && return + + # Report success/failure. + result=$(cat OUTPUT) + if [ "$result" = t ] + then + test_ok_ + else + test_failure_ "${result}" + fi + else + # Restore state after the (non) test. + exec 1>&6 2>&7 # Restore stdout and stderr + inside_subtest= + fi +} + +emacs_generate_script () { + # Construct a little test script here for the benefit of the user, + # (who can easily run "run_emacs" to get the same emacs environment + # for investigating any failures). + if [ -z "${NOTMUCH_TEST_INSTALLED-}" ]; then + find_notmuch_el='--directory "$NOTMUCH_BUILDDIR/emacs"' + else + ### XXX FIXME: this should really use the installed emacs lisp files + find_notmuch_el='--directory "$NOTMUCH_SRCDIR/emacs"' + fi + + cat <"$TMP_DIRECTORY/run_emacs" +#!/bin/sh +export PATH=$PATH +export NOTMUCH_CONFIG=$NOTMUCH_CONFIG + +# Here's what we are using here: +# +# --quick Use minimal customization. This implies --no-init-file, +# --no-site-file and (emacs 24) --no-site-lisp +# +# --directory Ensure that the local elisp sources are found +# +# --load Force loading of notmuch.el and test-lib.el + +exec ${TEST_EMACS} ${find_notmuch_el} --quick \ + ${EXTRA_DIR} --load notmuch.el \ + --directory "$NOTMUCH_SRCDIR/test" --load test-lib.el \ + "\$@" +EOF + chmod a+x "$TMP_DIRECTORY/run_emacs" +} + +test_emacs () { + # test dependencies beforehand to avoid the waiting loop below + test_require_emacs || return + + if [ -z "$EMACS_SERVER" ]; then + emacs_tests="$NOTMUCH_SRCDIR/test/${this_test_bare}.el" + if [ -f "$emacs_tests" ]; then + load_emacs_tests="--eval '(load \"$emacs_tests\")'" + else + load_emacs_tests= + fi + server_name="notmuch-test-suite-$$" + # start a detached session with an emacs server + # user's TERM (or 'vt100' in case user's TERM is known dumb + # or unknown) is given to dtach which assumes a minimally + # VT100-compatible terminal -- and emacs inherits that + TERM=$SMART_TERM dtach -n "$TEST_TMPDIR/emacs-dtach-socket.$$" \ + sh -c "stty rows 24 cols 80; exec '$TMP_DIRECTORY/run_emacs' \ + --no-window-system \ + $load_emacs_tests \ + --eval '(setq server-name \"$server_name\")' \ + --eval '(server-start)' \ + --eval '(orphan-watchdog $$)'" || return + EMACS_SERVER="$server_name" + # wait until the emacs server is up + until test_emacs '()' >/dev/null 2>/dev/null; do + sleep 1 + done + fi + + # Clear test-output output file. Most Emacs tests end with a + # call to (test-output). If the test code fails with an + # exception before this call, the output file won't get + # updated. Since we don't want to compare against an output + # file from another test, so start out with an empty file. + rm -f OUTPUT + touch OUTPUT + + ${TEST_EMACSCLIENT} --socket-name="$EMACS_SERVER" --eval "(notmuch-test-progn $*)" +} + +time_emacs () { + rm -f MESSAGES + printf "%s" "$1" + shift + test_emacs "(test-time $*)" > emacs.out + tail -n 1 MESSAGES +} + +emacs_generate_script diff --git a/test/test-lib.el b/test/test-lib.el index 9946010b..4cfb8ef1 100644 --- a/test/test-lib.el +++ b/test/test-lib.el @@ -1,4 +1,4 @@ -;; test-lib.el --- auxiliary stuff for Notmuch Emacs tests. +;;; test-lib.el --- auxiliary stuff for Notmuch Emacs tests ;; ;; Copyright © Carl Worth ;; Copyright © David Edmondson @@ -20,7 +20,24 @@ ;; ;; Authors: Dmitry Kurochkin -(require 'cl) ;; This code is generally used uncompiled. +;;; Code: + +;; minimize impact of native compilation on the test suite. +;; These are the Emacs 29.1 version of the variables. +;; Leave trampolines enabled per Emacs upstream recommendations. +;; It is important to set these variables before loading any +;; .elc files. +(setq native-comp-jit-compilation nil) +(setq native-comp-speed -1) +(setq native-comp-async-jobs-number 1) + +(require 'cl-lib) + +;; Ensure that the dynamic variables that are defined by this library +;; are defined by the time that we let-bind them. This is needed +;; because starting with Emacs 27 undeclared variables in evaluated +;; interactive code (such as our tests) use lexical scope. +(require 'smtpmail) ;; `read-file-name' by default uses `completing-read' function to read ;; user input. It does not respect `standard-input' variable which we @@ -28,23 +45,6 @@ ;; `read' call. (setq read-file-name-function (lambda (&rest _) (read))) -;; Work around a bug in emacs 23.1 and emacs 23.2 which prevents -;; noninteractive (kill-emacs) from emacsclient. -(if (and (= emacs-major-version 23) (< emacs-minor-version 3)) - (defadvice kill-emacs (before disable-yes-or-no-p activate) - "Disable yes-or-no-p before executing kill-emacs" - (defun yes-or-no-p (prompt) t))) - -;; Emacs bug #2930: -;; 23.0.92; `accept-process-output' and `sleep-for' do not run sentinels -;; seems to be present in Emacs 23.1. -;; Running `list-processes' after `accept-process-output' seems to work -;; around this problem. -(if (and (= emacs-major-version 23) (= emacs-minor-version 1)) - (defadvice accept-process-output (after run-list-processes activate) - "run list-processes after executing accept-process-output" - (list-processes))) - (defun notmuch-test-wait () "Wait for process completion." (while (get-buffer-process (current-buffer)) @@ -75,7 +75,7 @@ invisible text." (let (str) (while (< start end) (let ((next-pos (next-char-property-change start end))) - (when (not (invisible-p start)) + (unless (invisible-p start) (setq str (concat str (buffer-substring-no-properties start next-pos)))) (setq start next-pos))) @@ -91,35 +91,25 @@ invisible text." (defun orphan-watchdog-check (pid) "Periodically check that the process with id PID is still running, quit if it terminated." - (if (not (test-process-running pid)) - (kill-emacs))) + (unless (test-process-running pid) + (kill-emacs))) (defun orphan-watchdog (pid) "Initiate orphan watchdog check." (run-at-time 60 60 'orphan-watchdog-check pid)) -(defun hook-counter (hook) - "Count how many times a hook is called. Increments -`hook'-counter variable value if it is bound, otherwise does -nothing." - (let ((counter (intern (concat (symbol-name hook) "-counter")))) - (if (boundp counter) - (set counter (1+ (symbol-value counter)))))) - -(defun add-hook-counter (hook) - "Add hook to count how many times `hook' is called." - (add-hook hook (apply-partially 'hook-counter hook))) - -(add-hook-counter 'notmuch-hello-mode-hook) -(add-hook-counter 'notmuch-hello-refresh-hook) - -(defadvice notmuch-search-process-filter (around pessimal activate disable) - "Feed notmuch-search-process-filter one character at a time." - (let ((string (ad-get-arg 1))) - (loop for char across string - do (progn - (ad-set-arg 1 (char-to-string char)) - ad-do-it)))) +(defvar notmuch-hello-mode-hook-counter -100 + "Tests that care about this counter must let-bind it to 0.") +(add-hook 'notmuch-hello-mode-hook + (lambda () (cl-incf notmuch-hello-mode-hook-counter))) + +(defvar notmuch-hello-refresh-hook-counter -100 + "Tests that care about this counter must let-bind it to 0.") +(add-hook 'notmuch-hello-refresh-hook + (lambda () (cl-incf notmuch-hello-refresh-hook-counter))) + +(defvar notmuch-test-tag-hook-output nil) +(defun notmuch-test-tag-hook () (push (cons query tag-changes) notmuch-test-tag-hook-output)) (defun notmuch-test-mark-links () "Enclose links in the current buffer with << and >>." @@ -152,23 +142,22 @@ nothing." "Output:\t" (prin1-to-string output) "\n")) (defun notmuch-test-expect-equal (output expected) - "Compare OUTPUT with EXPECTED. Report any discrepencies." - (if (equal output expected) - t - (cond - ((and (listp output) - (listp expected)) - ;; Reporting the difference between two lists is done by - ;; reporting differing elements of OUTPUT and EXPECTED - ;; pairwise. This is expected to make analysis of failures - ;; simpler. - (apply #'concat (loop for o in output - for e in expected - if (not (equal o e)) - collect (notmuch-test-report-unexpected o e)))) - - (t - (notmuch-test-report-unexpected output expected))))) + "Compare OUTPUT with EXPECTED. Report any discrepancies." + (cond + ((equal output expected) + t) + ((and (listp output) + (listp expected)) + ;; Reporting the difference between two lists is done by + ;; reporting differing elements of OUTPUT and EXPECTED + ;; pairwise. This is expected to make analysis of failures + ;; simpler. + (apply #'concat (cl-loop for o in output + for e in expected + if (not (equal o e)) + collect (notmuch-test-report-unexpected o e)))) + (t + (notmuch-test-report-unexpected output expected)))) (defun notmuch-post-command () (run-hooks 'post-command-hook)) @@ -179,6 +168,38 @@ nothing." (lambda (x) `(prog1 ,x (notmuch-post-command))) body))) +;; For testing functions in +;; notmuch-{search,tree,unsorted}-result-format +(defun notmuch-test-result-flags (format-string result) + (let ((tags-to-letters (quote (("attachment" . "&") + ("signed" . "=") + ("unread" . "u") + ("inbox" . "i")))) + (tags (plist-get result :tags))) + (format format-string + (mapconcat (lambda (t2l) + (if (member (car t2l) tags) + (cdr t2l) + " ")) + tags-to-letters "")))) + +;; Log any signalled error (and other messages) to MESSAGES +;; Log "COMPLETE" if forms complete without error. +(defmacro test-log-error (&rest body) + `(progn + (with-current-buffer "*Messages*" + (let ((inhibit-read-only t)) (erase-buffer))) + (condition-case err + (progn ,@body + (message "COMPLETE")) + (t (message "%s" err))) + (with-current-buffer "*Messages*" (test-output "MESSAGES")))) + +(defmacro test-time (&rest body) + `(let ((results (mapcar (lambda (x) (/ x 5.0)) (benchmark-run 5 ,@body)))) + (message "\t\t%0.2f\t%0.2f\t%0.2f" (nth 0 results) (nth 1 results) (nth 2 results)) + (with-current-buffer "*Messages*" (test-output "MESSAGES")))) + ;; For historical reasons, we hide deleted tags by default in the test ;; suite (setq notmuch-tag-deleted-formats @@ -194,12 +215,7 @@ nothing." (setq mm-text-html-renderer 'html2text) -;; Set some variables for S/MIME tests. - -(setq smime-keys '(("" "test_suite.pem" nil))) - -(setq mml-smime-use 'openssl) - -;; all test keys are without passphrase -(eval-after-load 'smime - '(defun smime-ask-passphrase (cache) nil)) +;; Set our own default for message-hidden-headers, to avoid tests +;; breaking when the Emacs default changes. +(setq message-hidden-headers + '("^References:" "^Face:" "^X-Face:" "^X-Draft-From:")) diff --git a/test/test-lib.sh b/test/test-lib.sh index 7f8a3a4d..059e110c 100644 --- a/test/test-lib.sh +++ b/test/test-lib.sh @@ -29,8 +29,8 @@ shopt -u xpg_echo # Ensure NOTMUCH_SRCDIR and NOTMUCH_BUILDDIR are set. . $(dirname "$0")/export-dirs.sh || exit 1 -# It appears that people try to run tests without building... -if [[ ! -x "$NOTMUCH_BUILDDIR/notmuch" ]]; then +# We need either a built tree, or a promise of an installed notmuch +if [ -z "${NOTMUCH_TEST_INSTALLED-}" -a ! -x "$NOTMUCH_BUILDDIR/notmuch" ]; then echo >&2 'You do not seem to have built notmuch yet.' exit 1 fi @@ -64,57 +64,14 @@ exec 6>&1 7>&2 BASH_XTRACEFD=7 export PS4='+(${BASH_SOURCE}:${LINENO}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' -# Keep the original TERM for say_color and test_emacs -ORIGINAL_TERM=$TERM +. "$NOTMUCH_SRCDIR/test/test-vars.sh" || exit 1 -# Set SMART_TERM to vt100 for known dumb/unknown terminal. -# Otherwise use whatever TERM is currently used so that -# users' actual TERM environments are being used in tests. -case ${TERM-} in - '' | dumb | unknown ) - SMART_TERM=vt100 ;; - *) - SMART_TERM=$TERM ;; -esac - -# For repeatability, reset the environment to known value. -LANG=C -LC_ALL=C -PAGER=cat -TZ=UTC -TERM=dumb -export LANG LC_ALL PAGER TERM TZ -GIT_TEST_CMP=${GIT_TEST_CMP:-diff -u} -if [[ ( -n "$TEST_EMACS" && -z "$TEST_EMACSCLIENT" ) || \ - ( -z "$TEST_EMACS" && -n "$TEST_EMACSCLIENT" ) ]]; then - echo "error: must specify both or neither of TEST_EMACS and TEST_EMACSCLIENT" >&2 - exit 1 -fi -TEST_EMACS=${TEST_EMACS:-${EMACS:-emacs}} -TEST_EMACSCLIENT=${TEST_EMACSCLIENT:-emacsclient} -TEST_GDB=${TEST_GDB:-gdb} -TEST_CC=${TEST_CC:-cc} -TEST_CFLAGS=${TEST_CFLAGS:-"-g -O0"} -TEST_SHIM_CFLAGS=${TEST_SHIM_CFLAGS:-"-fpic -shared"} -TEST_SHIM_LDFLAGS=${TEST_SHIM_LDFLAGS:-"-ldl"} - -# Protect ourselves from common misconfiguration to export -# CDPATH into the environment -unset CDPATH - -unset GREP_OPTIONS - -# For emacsclient -unset ALTERNATE_EDITOR - -add_gnupg_home () -{ - local output - [ -d ${GNUPGHOME} ] && return +add_gnupg_home () { + [ -e "${GNUPGHOME}/gpg.conf" ] && return _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; } at_exit_function _gnupg_exit - mkdir -m 0700 "$GNUPGHOME" - gpg --no-tty --import <$NOTMUCH_SRCDIR/test/gnupg-secret-key.asc >"$GNUPGHOME"/import.log 2>&1 + mkdir -p -m 0700 "$GNUPGHOME" + gpg --no-tty --import <$NOTMUCH_SRCDIR/test/openpgp4-secret-key.asc >"$GNUPGHOME"/import.log 2>&1 test_debug "cat $GNUPGHOME/import.log" if (gpg --quick-random --version >/dev/null 2>&1) ; then echo quick-random >> "$GNUPGHOME"/gpg.conf @@ -124,11 +81,32 @@ add_gnupg_home () echo no-emit-version >> "$GNUPGHOME"/gpg.conf # Change this if we ship a new test key - FINGERPRINT="5AEAB11F5E33DCE875DDB75B6D92612D94E46381" - SELF_USERID="Notmuch Test Suite (INSECURE!)" + FINGERPRINT="9A3AFE6C60065A148FD4B58A7E6ABE924645CC60" + SELF_USERID="Notmuch Test Suite (INSECURE!) " + SELF_EMAIL="test_suite@notmuchmail.org" printf '%s:6:\n' "$FINGERPRINT" | gpg --quiet --batch --no-tty --import-ownertrust } +add_gpgsm_home () { + test_require_external_prereq openssl + + local fpr + [ -e "$GNUPGHOME/gpgsm.conf" ] && return + _gnupg_exit () { gpgconf --kill all 2>/dev/null || true; } + at_exit_function _gnupg_exit + mkdir -p -m 0700 "$GNUPGHOME" + gpgsm --batch --no-tty --no-common-certs-import --pinentry-mode=loopback --passphrase-fd 3 \ + --disable-dirmngr --import >"$GNUPGHOME"/import.log 2>&1 3<<<'' <$NOTMUCH_SRCDIR/test/smime/0xE0972A47.p12 + fpr=$(gpgsm --batch --with-colons --list-key test_suite@notmuchmail.org | awk -F: '/^fpr/ {print $10}') + echo "$fpr S relax" >> "$GNUPGHOME/trustlist.txt" + gpgsm --quiet --batch --no-tty --no-common-certs-import --disable-dirmngr --import < $NOTMUCH_SRCDIR/test/smime/ca.crt + echo "4D:E0:FF:63:C0:E9:EC:01:29:11:C8:7A:EE:DA:3A:9A:7F:6E:C1:0D S" >> "$GNUPGHOME/trustlist.txt" + printf '%s::1\n' include-certs disable-crl-checks | gpgconf --output /dev/null --change-options gpgsm + gpgsm --batch --no-tty --no-common-certs-import --pinentry-mode=loopback --passphrase-fd 3 \ + --disable-dirmngr --import "$NOTMUCH_SRCDIR/test/smime/bob.p12" >>"$GNUPGHOME"/import.log 2>&1 3<<<'' + test_debug "cat $GNUPGHOME/import.log" +} + # Each test should start with something like this, after copyright notices: # # test_description='Description of this test... @@ -168,56 +146,53 @@ do done if test -n "$debug"; then - print_subtest () { - printf " %-4s" "[$((test_count - 1))]" - } + fmt_subtest () { + printf -v $1 " %-4s" "[$((test_count - 1))]" + } else - print_subtest () { - true - } + fmt_subtest () { + printf -v $1 '' + } fi test -n "$COLORS_WITHOUT_TTY" || [ -t 1 ] || color= -if [ -n "$color" ] && [ "$ORIGINAL_TERM" != 'dumb' ] && ( - TERM=$ORIGINAL_TERM && - export TERM && - tput bold - tput setaf - tput sgr0 - ) >/dev/null 2>&1 +if [ -n "$color" ] && [ "$ORIGINAL_TERM" != 'dumb' ] && + tput -T "$ORIGINAL_TERM" -S <<<$'bold\nsetaf\nsgr0\n' >/dev/null 2>&1 then color=t else color= fi -if test -n "$color"; then +if test -n "$color" +then + # _tput run in subshell (``) only + _tput () { exec tput -T "$ORIGINAL_TERM" "$@"; } + unset BOLD RED GREEN BROWN SGR0 say_color () { - ( - TERM=$ORIGINAL_TERM - export TERM case "$1" in - error) tput bold; tput setaf 1;; # bold red - skip) tput bold; tput setaf 2;; # bold green - pass) tput setaf 2;; # green - info) tput setaf 3;; # brown - *) test -n "$quiet" && return;; + error) b=${BOLD=`_tput bold`} + c=${RED=`_tput setaf 1`} ;; # bold red + skip) b=${BOLD=`_tput bold`} + c=${GREEN=`_tput setaf 2`} ;; # bold green + pass) b= c=${GREEN=`_tput setaf 2`} ;; # green + info) b= c=${BROWN=`_tput setaf 3`} ;; # brown + *) b= c=; test -n "$quiet" && return ;; esac - shift - printf " " - printf "$@" - tput sgr0 - print_subtest - ) + f=$2 + shift 2 + sgr0=${SGR0=`_tput sgr0`} + fmt_subtest st + printf " ${b}${c}${f}${sgr0}${st}" "$@" } else say_color() { test -z "$1" && test -n "$quiet" && return - shift - printf " " - printf "$@" - print_subtest + f=$2 + shift 2 + fmt_subtest st + printf " ${f}${st}" "$@" } fi @@ -241,8 +216,7 @@ then fi test_description_printed= -print_test_description () -{ +print_test_description () { test -z "$test_description_printed" || return 0 echo echo $this_test: "Testing ${test_description}" @@ -308,94 +282,9 @@ die () { exit 1 } -GIT_EXIT_OK= -# Note: TEST_TMPDIR *NOT* exported! -TEST_TMPDIR=$(mktemp -d "${TMPDIR:-/tmp}/notmuch-test-$$.XXXXXX") -# Put GNUPGHOME in TMPDIR to avoid problems with long paths. -export GNUPGHOME="${TEST_TMPDIR}/gnupg" trap 'trap_exit' EXIT trap 'trap_signal' HUP INT TERM -# Deliver a message with emacs and add it to the database -# -# Uses emacs to generate and deliver a message to the mail store. -# Accepts arbitrary extra emacs/elisp functions to modify the message -# before sending, which is useful to doing things like attaching files -# to the message and encrypting/signing. -emacs_deliver_message () -{ - local subject="$1" - local body="$2" - shift 2 - # before we can send a message, we have to prepare the FCC maildir - mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} - # eval'ing smtp-dummy --background will set smtp_dummy_pid and -_port - local smtp_dummy_pid= smtp_dummy_port= - eval `$TEST_DIRECTORY/smtp-dummy --background sent_message` - test -n "$smtp_dummy_pid" || return 1 - test -n "$smtp_dummy_port" || return 1 - - test_emacs \ - "(let ((message-send-mail-function 'message-smtpmail-send-it) - (mail-host-address \"example.com\") - (smtpmail-smtp-server \"localhost\") - (smtpmail-smtp-service \"${smtp_dummy_port}\")) - (notmuch-mua-mail) - (message-goto-to) - (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") - (message-goto-subject) - (insert \"${subject}\") - (message-goto-body) - (insert \"${body}\") - $* - (notmuch-mua-send-and-exit))" - - # In case message was sent properly, client waits for confirmation - # before exiting and resuming control here; therefore making sure - # that server exits by sending (KILL) signal to it is safe. - kill -9 $smtp_dummy_pid - notmuch new >/dev/null -} - -# Pretend to deliver a message with emacs. Really save it to a file -# and add it to the database -# -# Uses emacs to generate and deliver a message to the mail store. -# Accepts arbitrary extra emacs/elisp functions to modify the message -# before sending, which is useful to doing things like attaching files -# to the message and encrypting/signing. -# -# If any GNU-style long-arguments (like --quiet or --decrypt=true) are -# at the head of the argument list, they are sent directly to "notmuch -# new" after message delivery -emacs_fcc_message () -{ - local nmn_args='' - while [[ "$1" =~ ^-- ]]; do - nmn_args="$nmn_args $1" - shift - done - local subject="$1" - local body="$2" - shift 2 - # before we can send a message, we have to prepare the FCC maildir - mkdir -p "$MAIL_DIR"/sent/{cur,new,tmp} - - test_emacs \ - "(let ((message-send-mail-function (lambda () t)) - (mail-host-address \"example.com\")) - (notmuch-mua-mail) - (message-goto-to) - (insert \"test_suite@notmuchmail.org\nDate: 01 Jan 2000 12:00:00 -0000\") - (message-goto-subject) - (insert \"${subject}\") - (message-goto-body) - (insert \"${body}\") - $* - (notmuch-mua-send-and-exit))" || return 1 - notmuch new $nmn_args >/dev/null -} - # Add an existing, fixed corpus of email to the database. # # $1 is the corpus dir under corpora to add, using "default" if unset. @@ -404,8 +293,8 @@ emacs_fcc_message () # history of the notmuch mailing list, which allows for reliably # testing commands that need to operate on a not-totally-trivial # number of messages. -add_email_corpus () -{ +add_email_corpus () { + local corpus corpus=${1:-default} rm -rf ${MAIL_DIR} @@ -413,8 +302,7 @@ add_email_corpus () notmuch new >/dev/null || die "'notmuch new' failed while adding email corpus" } -test_begin_subtest () -{ +test_begin_subtest () { if [ -n "$inside_subtest" ]; then exec 1>&6 2>&7 # Restore stdout and stderr error "bug in test script: Missing test_expect_equal in ${BASH_SOURCE[1]}:${BASH_LINENO[0]}" @@ -434,8 +322,8 @@ test_begin_subtest () # not accept a test name. Instead, the caller should call # test_begin_subtest before calling this function in order to set the # name. -test_expect_equal () -{ +test_expect_equal () { + local output expected testname exec 1>&6 2>&7 # Restore stdout and stderr if [ -z "$inside_subtest" ]; then error "bug in the test script: test_expect_equal without test_begin_subtest" @@ -459,61 +347,115 @@ test_expect_equal () fi } -# Like test_expect_equal, but takes two filenames. -test_expect_equal_file () -{ - exec 1>&6 2>&7 # Restore stdout and stderr - if [ -z "$inside_subtest" ]; then - error "bug in the test script: test_expect_equal_file without test_begin_subtest" +test_diff_file_ () { + local file1 file2 testname basename1 basename2 + file1="$1" + file2="$2" + if ! test_skip "$test_subtest_name" + then + if diff -q "$file1" "$file2" >/dev/null ; then + test_ok_ + else + testname=$this_test.$test_count + basename1=`basename "$file1"` + basename2=`basename "$file2"` + cp "$file1" "$testname.$basename1" + cp "$file2" "$testname.$basename2" + test_failure_ "$(diff -u "$testname.$basename1" "$testname.$basename2")" fi - inside_subtest= - test "$#" = 2 || + fi +} + +# Like test_expect_equal, but takes two filenames. +test_expect_equal_file () { + exec 1>&6 2>&7 # Restore stdout and stderr + if [ -z "$inside_subtest" ]; then + error "bug in the test script: test_expect_equal_file without test_begin_subtest" + fi + inside_subtest= + test "$#" = 2 || error "bug in the test script: not 2 parameters to test_expect_equal_file" - file1="$1" - file2="$2" - if ! test_skip "$test_subtest_name" - then - if diff -q "$file1" "$file2" >/dev/null ; then - test_ok_ - else - testname=$this_test.$test_count - basename1=`basename "$file1"` - basename2=`basename "$file2"` - cp "$file1" "$testname.$basename1" - cp "$file2" "$testname.$basename2" - test_failure_ "$(diff -u "$testname.$basename1" "$testname.$basename2")" - fi + test_diff_file_ "$1" "$2" +} + +# Like test_expect_equal_file, but compare the part of the two files after the first blank line +test_expect_equal_message_body () { + exec 1>&6 2>&7 # Restore stdout and stderr + if [ -z "$inside_subtest" ]; then + error "bug in the test script: test_expect_equal_file without test_begin_subtest" + fi + test "$#" = 2 || + error "bug in the test script: not 2 parameters to test_expect_equal_file" + + for file in "$1" "$2"; do + if [ ! -s "$file" ]; then + test_failure_ "Missing or zero length file: $file" + inside_subtest= + return 1 + fi + done + + expected=$(sed '1,/^$/d' "$1") + output=$(sed '1,/^$/d' "$2") + test_expect_equal "$expected" "$output" +} + +# Like test_expect_equal, but takes two filenames. Fails if either is empty +test_expect_equal_file_nonempty () { + exec 1>&6 2>&7 # Restore stdout and stderr + if [ -z "$inside_subtest" ]; then + error "bug in the test script: test_expect_equal_file_nonempty without test_begin_subtest" fi + inside_subtest= + test "$#" = 2 || + error "bug in the test script: not 2 parameters to test_expect_equal_file_nonempty" + + for file in "$1" "$2"; do + if [ ! -s "$file" ]; then + test_failure_ "Missing or zero length file: $file" + return $? + fi + done + + test_diff_file_ "$1" "$2" } # Like test_expect_equal, but arguments are JSON expressions to be # canonicalized before diff'ing. If an argument cannot be parsed, it # is used unchanged so that there's something to diff against. test_expect_equal_json () { + local script output expected # The test suite forces LC_ALL=C, but this causes Python 3 to # decode stdin as ASCII. We need to read JSON in UTF-8, so # override Python's stdio encoding defaults. - local script='import json, sys; json.dump(json.load(sys.stdin), sys.stdout, sort_keys=True, indent=4)' + script='import json, sys; json.dump(json.load(sys.stdin), sys.stdout, sort_keys=True, indent=4)' output=$(echo "$1" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" \ - || echo "$1") + || echo "$1") expected=$(echo "$2" | PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "$script" \ - || echo "$2") + || echo "$2") shift 2 test_expect_equal "$output" "$expected" "$@" } +# Ensure that the argument is valid JSON data. +test_valid_json () { + PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c "import sys, json; json.load(sys.stdin)" <<<"$1" + test_expect_equal "$?" 0 +} + # Sort the top-level list of JSON data from stdin. test_sort_json () { PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -c \ - "import sys, json; json.dump(sorted(json.load(sys.stdin)),sys.stdout)" + "import sys, json; json.dump(sorted(json.load(sys.stdin)),sys.stdout)" } # test for json objects: # read the source of test/json_check_nodes.py (or the output when # invoking it without arguments) for an explanation of the syntax. test_json_nodes () { - exec 1>&6 2>&7 # Restore stdout and stderr + local output + exec 1>&6 2>&7 # Restore stdout and stderr if [ -z "$inside_subtest" ]; then error "bug in the test script: test_json_eval without test_begin_subtest" fi @@ -523,7 +465,7 @@ test_json_nodes () { if ! test_skip "$test_subtest_name" then - output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON "$TEST_DIRECTORY"/json_check_nodes.py "$@") + output=$(PYTHONIOENCODING=utf-8 $NOTMUCH_PYTHON -B "$NOTMUCH_SRCDIR"/test/json_check_nodes.py "$@") if [ "$?" = 0 ] then test_ok_ @@ -533,50 +475,16 @@ test_json_nodes () { fi } -test_emacs_expect_t () { - test "$#" = 1 || - error "bug in the test script: not 1 parameter to test_emacs_expect_t" - if [ -z "$inside_subtest" ]; then - error "bug in the test script: test_emacs_expect_t without test_begin_subtest" - fi - - # Run the test. - if ! test_skip "$test_subtest_name" - then - test_emacs "(notmuch-test-run $1)" >/dev/null - - # Restore state after the test. - exec 1>&6 2>&7 # Restore stdout and stderr - inside_subtest= - - # Report success/failure. - result=$(cat OUTPUT) - if [ "$result" = t ] - then - test_ok_ - else - test_failure_ "${result}" - fi - else - # Restore state after the (non) test. - exec 1>&6 2>&7 # Restore stdout and stderr - inside_subtest= - fi -} - -NOTMUCH_NEW () -{ +NOTMUCH_NEW () { notmuch new "${@}" | grep -v -E -e '^Processed [0-9]*( total)? file|Found [0-9]* total file' } -NOTMUCH_DUMP_TAGS () -{ +NOTMUCH_DUMP_TAGS () { # this relies on the default format being batch-tag, otherwise some tests will break notmuch dump --include=tags "${@}" | sed '/^#/d' | sort } -notmuch_drop_mail_headers () -{ +notmuch_drop_mail_headers () { $NOTMUCH_PYTHON -c ' import email, sys msg = email.message_from_file(sys.stdin) @@ -585,80 +493,99 @@ print(msg.as_string(False)) ' "$@" } -notmuch_search_sanitize () -{ - perl -pe 's/("?thread"?: ?)("?)................("?)/\1\2XXX\3/' +notmuch_debug_sanitize () { + grep -v '^D.:' +} + +notmuch_exception_sanitize () { + perl -pe 's,(A Xapian exception occurred at) .*?([^/]*[.]cc?):([0-9]*),\1 \2:XXX,' +} + +notmuch_search_sanitize () { + notmuch_debug_sanitize | perl -pe 's/("?thread"?: ?)("?)................("?)/\1\2XXX\3/' } -notmuch_search_files_sanitize () -{ - notmuch_dir_sanitize +notmuch_search_files_sanitize () { + notmuch_dir_sanitize | sed 's/msg-[0-9][0-9][0-9]/msg-XXX/' } -notmuch_dir_sanitize () -{ +notmuch_dir_sanitize () { sed -e "s,$MAIL_DIR,MAIL_DIR," -e "s,${PWD},CWD,g" "$@" } NOTMUCH_SHOW_FILENAME_SQUELCH='s,filename:.*/mail,filename:/XXX/mail,' -notmuch_show_sanitize () -{ +notmuch_show_sanitize () { sed -e "$NOTMUCH_SHOW_FILENAME_SQUELCH" } -notmuch_show_sanitize_all () -{ +notmuch_show_sanitize_all () { + notmuch_debug_sanitize | \ sed \ -e 's| filename:.*| filename:XXXXX|' \ -e 's| id:[^ ]* | id:XXXXX |' | \ notmuch_date_sanitize } -notmuch_json_show_sanitize () -{ +notmuch_json_show_sanitize () { sed \ -e 's|"id": "[^"]*",|"id": "XXXXX",|g' \ -e 's|"Date": "Fri, 05 Jan 2001 [^"]*0000"|"Date": "GENERATED_DATE"|g' \ -e 's|"filename": "signature.asc",||g' \ + -e 's|"duplicate": 1,||g' \ -e 's|"filename": \["/[^"]*"\],|"filename": \["YYYYY"\],|g' \ -e 's|"timestamp": 97.......|"timestamp": 42|g' \ - -e 's|"content-length": [1-9][0-9]*|"content-length": "NONZERO"|g' + -e 's|"content-length": [1-9][0-9]*|"content-length": "NONZERO"|g' +} + +notmuch_sexp_show_sanitize () { + sed \ + -e 's|:id "[^"]*"|:id "XXXXX"|g' \ + -e 's|:Date "Sat, 01 Jan 2000 [^"]*0000"|:Date "GENERATED_DATE"|g' \ + -e 's|:filename "signature.asc"||g' \ + -e 's|:duplicate 1 ||g' \ + -e 's|:filename ("/[^"]*")|:filename ("YYYYY")|g' \ + -e 's|:timestamp 9........|:timestamp 42|g' \ + -e 's|:content-length [1-9][0-9]*|:content-length "NONZERO"|g' +} + +notmuch_sexp_search_sanitize () { + sed -e 's|:thread "[^"]*"|:thread "XXX"|' } -notmuch_emacs_error_sanitize () -{ - local command=$1 +notmuch_emacs_error_sanitize () { + local command + command=$1 shift for file in "$@"; do echo "=== $file ===" - cat "$file" - done | sed \ - -e 's/^\[.*\]$/[XXX]/' \ + notmuch_debug_sanitize < "$file" + done | sed \ + -e '/^$/d' \ + -e '/^\[.*\]$/d' \ -e "s|^\(command: \)\{0,1\}/.*/$command|\1YYY/$command|" } -notmuch_date_sanitize () -{ +notmuch_date_sanitize () { sed \ -e 's/^Date: Fri, 05 Jan 2001 .*0000/Date: GENERATED_DATE/' } -notmuch_uuid_sanitize () -{ +# remove redundant parts of notmuch-git internal paths +notmuch_git_sanitize () { + sed -e 's,tags/\([0-9a-f]\{2\}/\)\{2\},,' -e '/FORMAT/d' +} +notmuch_uuid_sanitize () { sed 's/[0-9a-f]\{8\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{4\}-[0-9a-f]\{12\}/UUID/g' } -notmuch_built_with_sanitize () -{ +notmuch_built_with_sanitize () { sed 's/^built_with[.]\(.*\)=.*$/built_with.\1=something/' } -notmuch_config_sanitize () -{ +notmuch_config_sanitize () { notmuch_dir_sanitize | notmuch_built_with_sanitize } -notmuch_show_part () -{ +notmuch_show_part () { awk '/^\014part}/{ f=0 }; { if (f) { print $0 } } /^\014part{ ID: '"$1"'/{ f=1 }' } @@ -690,6 +617,7 @@ declare -A test_subtest_missing_external_prereq_ # declare prerequisite for the given external binary test_declare_external_prereq () { + local binary binary="$1" test "$#" = 2 && name=$2 || name="$binary(1)" @@ -703,19 +631,6 @@ $binary () { fi } -# Explicitly require external prerequisite. Useful when binary is -# called indirectly (e.g. from emacs). -# Returns success if dependency is available, failure otherwise. -test_require_external_prereq () { - binary="$1" - if [[ ${test_missing_external_prereq_["${binary}"]} == t ]]; then - # dependency is missing, call the replacement function to note it - eval "$binary" - else - true - fi -} - # You are not expected to call test_ok_ and test_failure_ directly, use # the text_expect_* functions instead. @@ -833,6 +748,18 @@ test_subtest_known_broken () { test_subtest_known_broken_=t } +test_subtest_broken_for_installed () { + if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + test_subtest_known_broken_=t + fi +} + +test_subtest_broken_for_root () { + if [ "$EUID" = "0" ]; then + test_subtest_known_broken_=t + fi +} + test_expect_success () { exec 1>&6 2>&7 # Restore stdout and stderr if [ -z "$inside_subtest" ]; then @@ -847,7 +774,7 @@ test_expect_success () { test_run_ "$1" run_ret="$?" # test_run_ may update missing external prerequisites - test_check_missing_external_prereqs_ "$@" || + test_check_missing_external_prereqs_ "$test_subtest_name" || if [ "$run_ret" = 0 -a "$eval_ret" = 0 ] then test_ok_ @@ -871,7 +798,7 @@ test_expect_code () { test_run_ "$2" run_ret="$?" # test_run_ may update missing external prerequisites, - test_check_missing_external_prereqs_ "$@" || + test_check_missing_external_prereqs_ "$test_subtest_name" || if [ "$run_ret" = 0 -a "$eval_ret" = "$1" ] then test_ok_ @@ -885,8 +812,8 @@ test_expect_code () { # but is a prefix that can be used in the test script, like: # # test_expect_success 'complain and die' ' -# do something && -# do something else && +# do something && +# do something else && # test_must_fail git checkout ../outerspace # ' # @@ -911,7 +838,7 @@ test_must_fail () { # - cmp's output is not nearly as easy to read as diff -u # - not all diff versions understand "-u" -test_cmp() { +test_cmp () { $GIT_TEST_CMP "$@" } @@ -946,12 +873,13 @@ test_done () { mkdir -p "$test_results_dir" test_results_path="$test_results_dir/$this_test" - echo "total $test_count" >> $test_results_path - echo "success $test_success" >> $test_results_path - echo "fixed $test_fixed" >> $test_results_path - echo "broken $test_broken" >> $test_results_path - echo "failed $test_failure" >> $test_results_path - echo "" >> $test_results_path + printf %s\\n \ + "success $test_success" \ + "fixed $test_fixed" \ + "broken $test_broken" \ + "failed $test_failure" \ + "total $test_count" \ + > $test_results_path [ -n "$EMACS_SERVER" ] && test_emacs '(kill-emacs)' @@ -965,89 +893,15 @@ test_done () { fi } -emacs_generate_script () { - # Construct a little test script here for the benefit of the user, - # (who can easily run "run_emacs" to get the same emacs environment - # for investigating any failures). - cat <"$TMP_DIRECTORY/run_emacs" -#!/bin/sh -export PATH=$PATH -export NOTMUCH_CONFIG=$NOTMUCH_CONFIG - -# Here's what we are using here: -# -# --quick Use minimal customization. This implies --no-init-file, -# --no-site-file and (emacs 24) --no-site-lisp -# -# --directory Ensure that the local elisp sources are found -# -# --load Force loading of notmuch.el and test-lib.el - -exec ${TEST_EMACS} --quick \ - --directory "$NOTMUCH_SRCDIR/emacs" --load notmuch.el \ - --directory "$NOTMUCH_SRCDIR/test" --load test-lib.el \ - "\$@" -EOF - chmod a+x "$TMP_DIRECTORY/run_emacs" -} - -test_emacs () { - # test dependencies beforehand to avoid the waiting loop below - missing_dependencies= - test_require_external_prereq dtach || missing_dependencies=1 - test_require_external_prereq emacs || missing_dependencies=1 - test_require_external_prereq ${TEST_EMACSCLIENT} || missing_dependencies=1 - test -z "$missing_dependencies" || return - - if [ -z "$EMACS_SERVER" ]; then - emacs_tests="$NOTMUCH_SRCDIR/test/${this_test_bare}.el" - if [ -f "$emacs_tests" ]; then - load_emacs_tests="--eval '(load \"$emacs_tests\")'" - else - load_emacs_tests= - fi - server_name="notmuch-test-suite-$$" - # start a detached session with an emacs server - # user's TERM (or 'vt100' in case user's TERM is known dumb - # or unknown) is given to dtach which assumes a minimally - # VT100-compatible terminal -- and emacs inherits that - TERM=$SMART_TERM dtach -n "$TEST_TMPDIR/emacs-dtach-socket.$$" \ - sh -c "stty rows 24 cols 80; exec '$TMP_DIRECTORY/run_emacs' \ - --no-window-system \ - $load_emacs_tests \ - --eval '(setq server-name \"$server_name\")' \ - --eval '(server-start)' \ - --eval '(orphan-watchdog $$)'" || return - EMACS_SERVER="$server_name" - # wait until the emacs server is up - until test_emacs '()' >/dev/null 2>/dev/null; do - sleep 1 - done - fi - - # Clear test-output output file. Most Emacs tests end with a - # call to (test-output). If the test code fails with an - # exception before this call, the output file won't get - # updated. Since we don't want to compare against an output - # file from another test, so start out with an empty file. - rm -f OUTPUT - touch OUTPUT - - ${TEST_EMACSCLIENT} --socket-name="$EMACS_SERVER" --eval "(notmuch-test-progn $*)" -} - -test_python() { +test_python () { # Note: if there is need to print debug information from python program, # use stdout = os.fdopen(6, 'w') or stderr = os.fdopen(7, 'w') - PYTHONPATH="$NOTMUCH_SRCDIR/bindings/python${PYTHONPATH:+:$PYTHONPATH}" \ + PYTHONPATH="$NOTMUCH_BUILDDIR/bindings/python-cffi/build/stage:$NOTMUCH_SRCDIR/bindings/python${PYTHONPATH:+:$PYTHONPATH}" \ $NOTMUCH_PYTHON -B - > OUTPUT } -test_ruby() { - MAIL_DIR=$MAIL_DIR $NOTMUCH_RUBY -I $NOTMUCH_SRCDIR/bindings/ruby> OUTPUT -} - test_C () { + local exec_file test_file exec_file="test${test_count}" test_file="${exec_file}.c" cat > ${test_file} @@ -1055,10 +909,24 @@ test_C () { echo "== stdout ==" > OUTPUT.stdout echo "== stderr ==" > OUTPUT.stderr ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr - notmuch_dir_sanitize OUTPUT.stdout OUTPUT.stderr > OUTPUT + notmuch_dir_sanitize OUTPUT.stdout OUTPUT.stderr | notmuch_exception_sanitize | notmuch_debug_sanitize > OUTPUT +} + +test_private_C () { + local exec_file test_file + exec_file="test${test_count}" + test_file="${exec_file}.c" + echo '#include ' > ${test_file} + cat >> ${test_file} + ${TEST_CC} ${TEST_CFLAGS} -I${NOTMUCH_SRCDIR}/test -I${NOTMUCH_SRCDIR}/lib -I${NOTMUCH_SRCDIR}/util -I${NOTMUCH_SRCDIR}/compat ${NOTMUCH_GMIME_CFLAGS} -o ${exec_file} ${test_file} ${NOTMUCH_BUILDDIR}/lib/libnotmuch.a ${NOTMUCH_GMIME_LDFLAGS} ${NOTMUCH_XAPIAN_LDFLAGS} ${NOTMUCH_BUILDDIR}/util/libnotmuch_util.a ${NOTMUCH_SFSEXP_LDFLAGS} ${NOTMUCH_BUILDDIR}/parse-time-string/libparse-time-string.a -ltalloc -lstdc++ + echo "== stdout ==" > OUTPUT.stdout + echo "== stderr ==" > OUTPUT.stderr + ./${exec_file} "$@" 1>>OUTPUT.stdout 2>>OUTPUT.stderr + notmuch_dir_sanitize OUTPUT.stdout OUTPUT.stderr | notmuch_exception_sanitize | notmuch_debug_sanitize > OUTPUT } make_shim () { + local base_name test_file shim_file base_name="$1" test_file="${base_name}.c" shim_file="${base_name}.so" @@ -1067,10 +935,16 @@ make_shim () { } notmuch_with_shim () { - base_name="$1" + local base_name shim_file notmuch_cmd + if [ -n "${NOTMUCH_TEST_INSTALLED-}" ]; then + notmuch_cmd="notmuch" + else + notmuch_cmd="notmuch-shared" + fi + base_name=$1 shift shim_file="${base_name}.so" - LD_PRELOAD=./${shim_file}${LD_PRELOAD:+:$LD_PRELOAD} notmuch-shared "$@" + LD_PRELOAD=${LD_PRELOAD:+:$LD_PRELOAD}:./${shim_file} $notmuch_cmd "$@" } # Creates a script that counts how much time it is executed and calls @@ -1122,13 +996,14 @@ test_init_ () { # Where to run the tests -TEST_DIRECTORY=$NOTMUCH_BUILDDIR/test +if [[ -n "${NOTMUCH_BUILDDIR}" ]]; then + TEST_DIRECTORY=$NOTMUCH_BUILDDIR/test +else + TEST_DIRECTORY=$NOTMUCH_SRCDIR/test +fi . "$NOTMUCH_SRCDIR/test/test-lib-common.sh" || exit 1 -emacs_generate_script - - # Use -P to resolve symlinks in our working directory so that the cwd # in subprocesses like git equals our $PWD (for pathname comparisons). cd -P "$TMP_DIRECTORY" || error "Cannot set up test environment" @@ -1214,17 +1089,6 @@ test -z "$NO_PYTHON" && test_set_prereq PYTHON ln -s x y 2>/dev/null && test -h y 2>/dev/null && test_set_prereq SYMLINKS rm -f y -# convert variable from configure to more convenient form -case "$NOTMUCH_DEFAULT_XAPIAN_BACKEND" in - glass) - db_ending=glass - ;; - chert) - db_ending=DB - ;; - *) - error "Unknown Xapian backend $NOTMUCH_DEFAULT_XAPIAN_BACKEND" -esac # declare prerequisites for external binaries used in tests test_declare_external_prereq dtach test_declare_external_prereq emacs @@ -1234,3 +1098,5 @@ test_declare_external_prereq gpg test_declare_external_prereq openssl test_declare_external_prereq gpgsm test_declare_external_prereq ${NOTMUCH_PYTHON} +test_declare_external_prereq xapian-metadata +test_declare_external_prereq xapian-delve diff --git a/test/test-vars.sh b/test/test-vars.sh new file mode 100644 index 00000000..02d60f89 --- /dev/null +++ b/test/test-vars.sh @@ -0,0 +1,62 @@ +# Common variable settings for (correctness) tests and performance +# tests. + +# Keep the original TERM for say_color and test_emacs +ORIGINAL_TERM=$TERM + +# Set SMART_TERM to vt100 for known dumb/unknown terminal. +# Otherwise use whatever TERM is currently used so that +# users' actual TERM environments are being used in tests. +case ${TERM-} in + '' | dumb | unknown ) + SMART_TERM=vt100 ;; + *) + SMART_TERM=$TERM ;; +esac + +# For repeatability, reset the environment to known value. +LANG=C +LC_ALL=C +PAGER=cat +TZ=UTC +TERM=dumb +export LANG LC_ALL PAGER TERM TZ +GIT_TEST_CMP=${GIT_TEST_CMP:-diff -u} +if [[ ( -n "$TEST_EMACS" && -z "$TEST_EMACSCLIENT" ) || \ + ( -z "$TEST_EMACS" && -n "$TEST_EMACSCLIENT" ) ]]; then + echo "error: must specify both or neither of TEST_EMACS and TEST_EMACSCLIENT" >&2 + exit 1 +fi +TEST_EMACS=${TEST_EMACS:-${EMACS:-emacs}} +TEST_EMACSCLIENT=${TEST_EMACSCLIENT:-emacsclient} +TEST_GDB=${TEST_GDB:-gdb} +TEST_CC=${TEST_CC:-cc} +TEST_CFLAGS=${TEST_CFLAGS:-"-g -O0"} +TEST_SHIM_CFLAGS=${TEST_SHIM_CFLAGS:-"-fpic -shared"} +TEST_SHIM_LDFLAGS=${TEST_SHIM_LDFLAGS:-"-ldl"} + +# Protect ourselves from common misconfiguration to export +# CDPATH into the environment +unset CDPATH + +unset GREP_OPTIONS + +# For lib/open.cc:_load_key_file +unset XDG_CONFIG_HOME + +# for lib/open.cc:_choose_database_path +unset XDG_DATA_HOME +unset MAILDIR + +# For emacsclient +unset ALTERNATE_EDITOR + +# for reproducibility +unset EMAIL +unset NAME + +GIT_EXIT_OK= +# Note: TEST_TMPDIR *NOT* exported! +TEST_TMPDIR=$(mktemp -d "${TMPDIR:-/tmp}/notmuch-test-$$.XXXXXX") +# Put GNUPGHOME in TMPDIR to avoid problems with long paths. +export GNUPGHOME="${TEST_TMPDIR}/gnupg" diff --git a/util/Makefile.local b/util/Makefile.local index f5d72f79..8a0b9bc3 100644 --- a/util/Makefile.local +++ b/util/Makefile.local @@ -1,4 +1,4 @@ -# -*- makefile -*- +# -*- makefile-gmake -*- dir := util extra_cflags += -I$(srcdir)/$(dir) @@ -6,7 +6,7 @@ extra_cflags += -I$(srcdir)/$(dir) libnotmuch_util_c_srcs := $(dir)/xutil.c $(dir)/error_util.c $(dir)/hex-escape.c \ $(dir)/string-util.c $(dir)/talloc-extra.c $(dir)/zlib-extra.c \ $(dir)/util.c $(dir)/gmime-extra.c $(dir)/crypto.c \ - $(dir)/repair.c \ + $(dir)/repair.c $(dir)/path-util.c \ $(dir)/unicode-util.c libnotmuch_util_modules := $(libnotmuch_util_c_srcs:.c=.o) diff --git a/util/crypto.c b/util/crypto.c index 0bb6f526..156a6550 100644 --- a/util/crypto.c +++ b/util/crypto.c @@ -34,7 +34,7 @@ GMimeObject * _notmuch_crypto_decrypt (bool *attempted, notmuch_decryption_policy_t decrypt, notmuch_message_t *message, - GMimeMultipartEncrypted *part, + GMimeObject *part, GMimeDecryptResult **decrypt_result, GError **err) { @@ -48,17 +48,30 @@ _notmuch_crypto_decrypt (bool *attempted, notmuch_message_properties_t *list = NULL; for (list = notmuch_message_get_properties (message, "session-key", TRUE); - notmuch_message_properties_valid (list); notmuch_message_properties_move_to_next (list)) { + notmuch_message_properties_valid (list); notmuch_message_properties_move_to_next ( + list)) { if (err && *err) { g_error_free (*err); *err = NULL; } if (attempted) *attempted = true; - ret = g_mime_multipart_encrypted_decrypt (part, - GMIME_DECRYPT_NONE, - notmuch_message_properties_value (list), - decrypt_result, err); + if (GMIME_IS_MULTIPART_ENCRYPTED (part)) { + ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED (part), + GMIME_DECRYPT_NONE, + notmuch_message_properties_value (list), + decrypt_result, err); + } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) { + GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part); + GMimeSecureMimeType type = g_mime_application_pkcs7_mime_get_smime_type (pkcs7); + if (type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) { + ret = g_mime_application_pkcs7_mime_decrypt (pkcs7, + GMIME_DECRYPT_NONE, + notmuch_message_properties_value ( + list), + decrypt_result, err); + } + } if (ret) break; } @@ -79,10 +92,20 @@ _notmuch_crypto_decrypt (bool *attempted, if (attempted) *attempted = true; GMimeDecryptFlags flags = GMIME_DECRYPT_NONE; + if (decrypt == NOTMUCH_DECRYPT_TRUE && decrypt_result) flags |= GMIME_DECRYPT_EXPORT_SESSION_KEY; - ret = g_mime_multipart_encrypted_decrypt (part, flags, NULL, - decrypt_result, err); + if (GMIME_IS_MULTIPART_ENCRYPTED (part)) { + ret = g_mime_multipart_encrypted_decrypt (GMIME_MULTIPART_ENCRYPTED (part), flags, NULL, + decrypt_result, err); + } else if (GMIME_IS_APPLICATION_PKCS7_MIME (part)) { + GMimeApplicationPkcs7Mime *pkcs7 = GMIME_APPLICATION_PKCS7_MIME (part); + GMimeSecureMimeType p7type = g_mime_application_pkcs7_mime_get_smime_type (pkcs7); + if (p7type == GMIME_SECURE_MIME_TYPE_ENVELOPED_DATA) { + ret = g_mime_application_pkcs7_mime_decrypt (pkcs7, flags, NULL, + decrypt_result, err); + } + } return ret; } @@ -108,7 +131,8 @@ _notmuch_message_crypto_new (void *ctx) } notmuch_status_t -_notmuch_message_crypto_potential_sig_list (_notmuch_message_crypto_t *msg_crypto, GMimeSignatureList *sigs) +_notmuch_message_crypto_potential_sig_list (_notmuch_message_crypto_t *msg_crypto, + GMimeSignatureList *sigs) { if (! msg_crypto) return NOTMUCH_STATUS_NULL_POINTER; @@ -137,7 +161,8 @@ _notmuch_message_crypto_potential_sig_list (_notmuch_message_crypto_t *msg_crypt bool -_notmuch_message_crypto_potential_payload (_notmuch_message_crypto_t *msg_crypto, GMimeObject *part, GMimeObject *parent, int childnum) +_notmuch_message_crypto_potential_payload (_notmuch_message_crypto_t *msg_crypto, GMimeObject *part, + GMimeObject *parent, int childnum) { const char *protected_headers = NULL; const char *forwarded = NULL; @@ -155,7 +180,8 @@ _notmuch_message_crypto_potential_payload (_notmuch_message_crypto_t *msg_crypto * encryption protocol should be "control information" metadata, * not payload. So we skip it. (see * https://tools.ietf.org/html/rfc1847#page-8) */ - if (parent && GMIME_IS_MULTIPART_ENCRYPTED (parent) && childnum == GMIME_MULTIPART_ENCRYPTED_VERSION) { + if (parent && GMIME_IS_MULTIPART_ENCRYPTED (parent) && childnum == + GMIME_MULTIPART_ENCRYPTED_VERSION) { const char *enc_type = g_mime_object_get_content_type_parameter (parent, "protocol"); GMimeContentType *ct = g_mime_object_get_content_type (part); if (ct && enc_type) { diff --git a/util/crypto.h b/util/crypto.h index f8bda0d1..3c5d384b 100644 --- a/util/crypto.h +++ b/util/crypto.h @@ -18,7 +18,7 @@ GMimeObject * _notmuch_crypto_decrypt (bool *attempted, notmuch_decryption_policy_t decrypt, notmuch_message_t *message, - GMimeMultipartEncrypted *part, + GMimeObject *part, GMimeDecryptResult **decrypt_result, GError **err); @@ -80,7 +80,8 @@ _notmuch_message_crypto_new (void *ctx); * consider a particular signature as relevant for the message. */ notmuch_status_t -_notmuch_message_crypto_potential_sig_list (_notmuch_message_crypto_t *msg_crypto, GMimeSignatureList *sigs); +_notmuch_message_crypto_potential_sig_list (_notmuch_message_crypto_t *msg_crypto, + GMimeSignatureList *sigs); /* call successful_decryption during a depth-first-search on a message * to indicate that a part was successfully decrypted. @@ -95,7 +96,8 @@ _notmuch_message_crypto_successful_decryption (_notmuch_message_crypto_t *msg_cr * this message. */ bool -_notmuch_message_crypto_potential_payload (_notmuch_message_crypto_t *msg_crypto, GMimeObject *part, GMimeObject *parent, int childnum); +_notmuch_message_crypto_potential_payload (_notmuch_message_crypto_t *msg_crypto, GMimeObject *part, + GMimeObject *parent, int childnum); #ifdef __cplusplus diff --git a/util/gmime-extra.c b/util/gmime-extra.c index 04d8ed3d..192cb078 100644 --- a/util/gmime-extra.c +++ b/util/gmime-extra.c @@ -101,11 +101,27 @@ g_mime_certificate_get_valid_userid (GMimeCertificate *cert) if (uid == NULL) return uid; GMimeValidity validity = g_mime_certificate_get_id_validity (cert); + if (validity == GMIME_VALIDITY_FULL || validity == GMIME_VALIDITY_ULTIMATE) return uid; return NULL; } +const char * +g_mime_certificate_get_valid_email (GMimeCertificate *cert) +{ + /* output e-mail address only if validity is FULL or ULTIMATE. */ + const char *email = g_mime_certificate_get_email(cert); + + if (email == NULL) + return email; + GMimeValidity validity = g_mime_certificate_get_id_validity (cert); + + if (validity == GMIME_VALIDITY_FULL || validity == GMIME_VALIDITY_ULTIMATE) + return email; + return NULL; +} + const char * g_mime_certificate_get_fpr16 (GMimeCertificate *cert) { diff --git a/util/gmime-extra.h b/util/gmime-extra.h index 094309ec..889e91f3 100644 --- a/util/gmime-extra.h +++ b/util/gmime-extra.h @@ -69,6 +69,10 @@ gint64 g_mime_utils_header_decode_date_unix (const char *date); * Return string for valid User ID (or NULL if no valid User ID exists) */ const char *g_mime_certificate_get_valid_userid (GMimeCertificate *cert); +/** + * Return string for valid e-mail address (or NULL if no valid e-mail address exists) + */ +const char *g_mime_certificate_get_valid_email (GMimeCertificate *cert); #ifdef __cplusplus } diff --git a/util/hex-escape.h b/util/hex-escape.h index 8703334c..83a4c6f1 100644 --- a/util/hex-escape.h +++ b/util/hex-escape.h @@ -5,7 +5,7 @@ extern "C" { #endif -typedef enum hex_status { +typedef enum { HEX_SUCCESS = 0, HEX_SYNTAX_ERROR, HEX_OUT_OF_MEMORY diff --git a/util/path-util.c b/util/path-util.c new file mode 100644 index 00000000..3267a967 --- /dev/null +++ b/util/path-util.c @@ -0,0 +1,27 @@ +/* + * SPDX-License-Identifier: GPL-3.0-or-later + */ + +#define _GNU_SOURCE + +#include "path-util.h" + +#include +#include + + +char * +notmuch_canonicalize_file_name (const char *path) +{ +#if HAVE_CANONICALIZE_FILE_NAME + return canonicalize_file_name (path); +#elif defined(PATH_MAX) + char *resolved_path = malloc (PATH_MAX + 1); + if (resolved_path == NULL) + return NULL; + + return realpath (path, resolved_path); +#else +#error undefined PATH_MAX _and_ missing canonicalize_file_name not supported +#endif +} diff --git a/util/path-util.h b/util/path-util.h new file mode 100644 index 00000000..ac85f696 --- /dev/null +++ b/util/path-util.h @@ -0,0 +1,19 @@ +/* + * SPDX-License-Identifier: GPL-3.0-or-later + */ + +#ifndef NOTMUCH_UTIL_PATH_UTIL_H_ +#define NOTMUCH_UTIL_PATH_UTIL_H_ + +#ifdef __cplusplus +extern "C" { +#endif + +char * +notmuch_canonicalize_file_name (const char *path); + +#ifdef __cplusplus +} +#endif + +#endif /* NOTMUCH_UTIL_PATH_UTIL_H_ */ diff --git a/util/repair.c b/util/repair.c index 9fba97b7..5b0dfdf4 100644 --- a/util/repair.c +++ b/util/repair.c @@ -27,18 +27,13 @@ _notmuch_crypto_payload_has_legacy_display (GMimeObject *payload) { GMimeMultipart *mpayload; const char *protected_header_parameter; - GMimeTextPart *legacy_display; - char *legacy_display_header_text = NULL; - GMimeStream *stream = NULL; - GMimeParser *parser = NULL; - GMimeObject *legacy_header_object = NULL, *first; - GMimeHeaderList *legacy_display_headers = NULL, *protected_headers = NULL; - bool ret = false; + GMimeObject *first; if (! g_mime_content_type_is_type (g_mime_object_get_content_type (payload), "multipart", "mixed")) return false; - protected_header_parameter = g_mime_object_get_content_type_parameter (payload, "protected-headers"); + protected_header_parameter = g_mime_object_get_content_type_parameter (payload, + "protected-headers"); if ((! protected_header_parameter) || strcmp (protected_header_parameter, "v1")) return false; if (! GMIME_IS_MULTIPART (payload)) @@ -49,66 +44,23 @@ _notmuch_crypto_payload_has_legacy_display (GMimeObject *payload) if (g_mime_multipart_get_count (mpayload) != 2) return false; first = g_mime_multipart_get_part (mpayload, 0); - if (! g_mime_content_type_is_type (g_mime_object_get_content_type (first), - "text", "rfc822-headers")) - return false; - protected_header_parameter = g_mime_object_get_content_type_parameter (first, "protected-headers"); + /* Early implementations that generated "Legacy Display" parts used + * Content-Type: text/rfc822-headers, but text/plain is more widely + * rendered, so it is now the standard choice. We accept either as a + * Legacy Display part. */ + if (! (g_mime_content_type_is_type (g_mime_object_get_content_type (first), + "text", "plain") || + g_mime_content_type_is_type (g_mime_object_get_content_type (first), + "text", "rfc822-headers"))) + return false; + protected_header_parameter = g_mime_object_get_content_type_parameter (first, + "protected-headers"); if ((! protected_header_parameter) || strcmp (protected_header_parameter, "v1")) return false; if (! GMIME_IS_TEXT_PART (first)) return false; - /* ensure that the headers in the first part all match the values - * found in the payload's own protected headers! if they don't, - * we should not treat this as a valid "legacy-display" part. - * - * Crafting a GMimeHeaderList object from the content of the - * text/rfc822-headers part is pretty clumsy; we should probably - * push something into GMime that makes this a one-shot - * operation. */ - if ((protected_headers = g_mime_object_get_header_list (payload), protected_headers) && - (legacy_display = GMIME_TEXT_PART (first), legacy_display) && - (legacy_display_header_text = g_mime_text_part_get_text (legacy_display), legacy_display_header_text) && - (stream = g_mime_stream_mem_new_with_buffer (legacy_display_header_text, strlen (legacy_display_header_text)), stream) && - (g_mime_stream_write (stream, "\r\n\r\n", 4) == 4) && - (g_mime_stream_seek (stream, 0, GMIME_STREAM_SEEK_SET) == 0) && - (parser = g_mime_parser_new_with_stream (stream), parser) && - (legacy_header_object = g_mime_parser_construct_part (parser, NULL), legacy_header_object) && - (legacy_display_headers = g_mime_object_get_header_list (legacy_header_object), legacy_display_headers)) { - /* walk through legacy_display_headers, comparing them against - * their values in the protected_headers: */ - ret = true; - for (int i = 0; i < g_mime_header_list_get_count (legacy_display_headers); i++) { - GMimeHeader *dh = g_mime_header_list_get_header_at (legacy_display_headers, i); - if (dh == NULL) { - ret = false; - goto DONE; - } - GMimeHeader *ph = g_mime_header_list_get_header (protected_headers, g_mime_header_get_name (dh)); - if (ph == NULL) { - ret = false; - goto DONE; - } - const char *dhv = g_mime_header_get_value (dh); - const char *phv = g_mime_header_get_value (ph); - if (dhv == NULL || phv == NULL || strcmp (dhv, phv)) { - ret = false; - goto DONE; - } - } - } - - DONE: - if (legacy_display_header_text) - g_free (legacy_display_header_text); - if (stream) - g_object_unref (stream); - if (parser) - g_object_unref (parser); - if (legacy_header_object) - g_object_unref (legacy_header_object); - - return ret; + return true; } GMimeObject * @@ -127,7 +79,7 @@ static bool _notmuch_is_mixed_up_mangled (GMimeObject *part) { GMimeMultipart *mpart = NULL; - GMimeObject *parts[3] = {NULL, NULL, NULL}; + GMimeObject *parts[3] = { NULL, NULL, NULL }; GMimeContentType *type = NULL; char *prelude_string = NULL; bool prelude_is_empty; diff --git a/util/string-util.c b/util/string-util.c index de8430b2..03d7648d 100644 --- a/util/string-util.c +++ b/util/string-util.c @@ -24,6 +24,7 @@ #include #include +#include char * strtok_len (char *s, const char *delim, size_t *len) @@ -37,6 +38,30 @@ strtok_len (char *s, const char *delim, size_t *len) return *len ? s : NULL; } +const char * +strsplit_len (const char *s, char delim, size_t *len) +{ + bool escaping = false; + size_t count = 0, last_nonspace = 0; + + /* Skip initial unescaped delimiters and whitespace */ + while (*s && (*s == delim || isspace (*s))) + s++; + + while (s[count] && (escaping || s[count] != delim)) { + if (! isspace (s[count])) + last_nonspace = count; + escaping = (s[count] == '\\'); + count++; + } + + if (count == 0) + return NULL; + + *len = last_nonspace + 1; + return s; +} + const char * strtok_len_c (const char *s, const char *delim, size_t *len) { @@ -160,6 +185,7 @@ parse_boolean_term (void *ctx, const char *str, /* Parse prefix */ str = skip_space (str); const char *pos = strchr (str, ':'); + if (! pos || pos == str) goto FAIL; *prefix_out = talloc_strndup (ctx, str, pos - str); diff --git a/util/string-util.h b/util/string-util.h index fb95a740..80647c5f 100644 --- a/util/string-util.h +++ b/util/string-util.h @@ -26,6 +26,20 @@ char *strtok_len (char *s, const char *delim, size_t *len); /* Const version of strtok_len. */ const char *strtok_len_c (const char *s, const char *delim, size_t *len); +/* Simplified version of strtok_len, with a single delimiter. + * Handles escaping delimiters with \ + * Usage pattern: + * + * const char *tok = input; + * const char *delim = ';'; + * size_t tok_len = 0; + * + * while ((tok = strsplit_len (tok + tok_len, delim, &tok_len)) != NULL) { + * // do stuff with string tok of length tok_len + * } + */ +const char *strsplit_len (const char *s, char delim, size_t *len); + /* Return a talloced string with str sanitized. * * Whitespace characters (tabs and newlines) are replaced with spaces, diff --git a/util/unicode-util.h b/util/unicode-util.h index 32d1e6ef..1bb9336a 100644 --- a/util/unicode-util.h +++ b/util/unicode-util.h @@ -4,9 +4,16 @@ #include #include +#ifdef __cplusplus +extern "C" { +#endif + /* The utf8 encoded string would tokenize as a single word, according * to xapian. */ bool unicode_word_utf8 (const char *str); typedef gunichar notmuch_unichar; +#ifdef __cplusplus +} +#endif #endif diff --git a/util/xapian-extra.h b/util/xapian-extra.h new file mode 100644 index 00000000..39c7f48f --- /dev/null +++ b/util/xapian-extra.h @@ -0,0 +1,15 @@ +#ifndef _XAPIAN_EXTRA_H +#define _XAPIAN_EXTRA_H + +#include +#include + +inline Xapian::Query +xapian_query_match_all (void) +{ + // Xapian::Query::MatchAll isn't thread safe (a static object with reference + // counting) so instead reconstruct the equivalent on demand. + return Xapian::Query (std::string ()); +} + +#endif diff --git a/util/zlib-extra.c b/util/zlib-extra.c index f691cccf..1f5f9dbe 100644 --- a/util/zlib-extra.c +++ b/util/zlib-extra.c @@ -47,6 +47,7 @@ gz_getline (void *talloc_ctx, char **bufptr, ssize_t *bytes_read, gzFile stream) int zlib_status = 0; (void) gzerror (stream, &zlib_status); switch (zlib_status) { + case Z_STREAM_END: case Z_OK: /* no data read before EOF */ if (offset == 0) @@ -80,7 +81,15 @@ const char * gz_error_string (util_status_t status, gzFile file) { if (status == UTIL_GZERROR) - return gzerror (file, NULL); + return gzerror_str (file); else return util_error_string (status); } + +const char * +gzerror_str (gzFile file) +{ + int dummy; + + return gzerror (file, &dummy); +} diff --git a/util/zlib-extra.h b/util/zlib-extra.h index 209fa998..7532339b 100644 --- a/util/zlib-extra.h +++ b/util/zlib-extra.h @@ -27,6 +27,11 @@ gz_getline (void *ctx, char **lineptr, ssize_t *bytes_read, gzFile stream); const char * gz_error_string (util_status_t status, gzFile stream); +/* Call gzerror with a dummy errno argument, the docs don't promise to + * support the NULL case */ +const char * +gzerror_str (gzFile file); + #ifdef __cplusplus } #endif diff --git a/version b/version deleted file mode 100644 index 25939d35..00000000 --- a/version +++ /dev/null @@ -1 +0,0 @@ -0.29.1 diff --git a/version.txt b/version.txt new file mode 100644 index 00000000..f2687f32 --- /dev/null +++ b/version.txt @@ -0,0 +1 @@ +0.38.3 diff --git a/vim/README b/vim/README index c137bacd..777c20c0 100644 --- a/vim/README +++ b/vim/README @@ -53,10 +53,10 @@ Enjoy ;) As an example to configure a key mapping to add the tag 'to-do' and archive, this is what I use: -let g:notmuch_rb_custom_search_maps = { +let g:notmuch_custom_search_maps = { \ 't': 'search_tag("+to-do -inbox")', \ } -let g:notmuch_rb_custom_show_maps = { +let g:notmuch_custom_show_maps = { \ 't': 'show_tag("+to-do -inbox")', \ } diff --git a/vim/notmuch.txt b/vim/notmuch.txt index 43741022..c98f2b53 100644 --- a/vim/notmuch.txt +++ b/vim/notmuch.txt @@ -89,7 +89,7 @@ s Send CONFIGURATION *notmuch-config* You can add the following configurations to your `.vimrc`, or -`~/.vim/plugin/notmuch.vim`. +`~/.vim/after/plugin/notmuch.vim`. *g:notmuch_folders* @@ -138,13 +138,13 @@ You can do the same for the thread view: If you want to count the threads instead of the messages in the folder view: > - let g:notmuch_folders_count_threads = 0 + let g:notmuch_folders_count_threads = 1 < *g:notmuch_reader* *g:notmuch_sendmail* -You can also configure your externail mail reader and sendemail program: +You can also configure your external mail reader and sendmail program: > let g:notmuch_reader = 'mutt -f %s' let g:notmuch_sendmail = 'sendmail' diff --git a/vim/notmuch.vim b/vim/notmuch.vim index ad8b7c80..c1c2f63d 100644 --- a/vim/notmuch.vim +++ b/vim/notmuch.vim @@ -317,6 +317,9 @@ ruby << EOF $curbuf.render do |b| q = $curbuf.query(get_cur_view) q.sort = Notmuch::SORT_OLDEST_FIRST + $exclude_tags.each { |t| + q.add_tag_exclude(t) + } msgs = q.search_messages msgs.each do |msg| m = Mail.read(msg.filename) @@ -666,7 +669,7 @@ ruby << EOF date = Time.at(e.newest_date).strftime(date_fmt) subject = e.messages.first['subject'] if $mail_installed - subject = Mail::Field.new("Subject: " + subject).to_s + subject = Mail::Field.parse("Subject: " + subject).to_s else subject = subject.force_encoding('utf-8') end