Mark Walters [Sun, 6 Apr 2014 05:44:45 +0000 (06:44 +0100)]
emacs: hello: add helper functions for saved-searches
Add helper functions to for saved searches to ease the transition to
the new plist form while maintaining backwards compatibility. They
will be used in the next patch.
Tomi Ollila [Wed, 19 Mar 2014 20:37:10 +0000 (22:37 +0200)]
emacs: add notmuch-version.el.tmpl and create notmuch-version.el from it
The notmuch cli program and emacs lisp versions may differ (especially
in remote usage). It helps to resolve problems if we can determine
the versions of notmuch cli and notmuch emacs mua separately.
The build process now creates notmuch-version.el from template file
by filling the version info to notmuch-emacs-version variable.
This version file will be as prerequisite to the target files
that use the version info for some purpose, like printing
it for the user to examine. The contents of the version.stamp
file is seldom read by the build system itself as the $(VERSION)
variable has the same information.
Thanks to Trevor, David and Mark for their contributions.
which breaks all of the FETCH_HEAD logic in nmbug (where FETCH_HEAD is
assumed to point to the master branch).
Instead of relying on FETCH_HEAD, use @{upstream} as the
remote-tracking branch that should be merged/diffed/integrated into
HEAD. @{upstream} was added in Git v1.7.0 (2010-02-12) [1], so
relying on it should be fairly safe. One tricky bit is that bare
repositories don't set upstream tracking branches by default:
$ git clone --bare http://nmbug.tethera.net/git/nmbug-tags.git nmbug-bare
$ cd nmbug-bare
$ git remote show origin
* remote origin
Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
Push URL: http://nmbug.tethera.net/git/nmbug-tags.git
HEAD branch: master
Local refs configured for 'git push':
config pushes to config (up to date)
master pushes to master (up to date)
While in a non-bare clone:
$ git clone http://nmbug.tethera.net/git/nmbug-tags.git
$ cd nmbug-tags
$ git remote show origin
* remote origin
Fetch URL: http://nmbug.tethera.net/git/nmbug-tags.git
Push URL: http://nmbug.tethera.net/git/nmbug-tags.git
HEAD branch: master
Remote branches:
config tracked
master tracked
Local branch configured for 'git pull':
master merges with remote master
Local ref configured for 'git push':
master pushes to master (up to date)
From the clone docs [2]:
--bare::
Make a 'bare' Git repository…
Also the branch heads at the remote are copied directly
to corresponding local branch heads, without mapping
them to `refs/remotes/origin/`. When this option is
used, neither remote-tracking branches nor the related
configuration variables are created.
To use @{upstream}, we need to the local vs. remote-tracking
distinction, so this commit adds 'nmbug clone', replacing the
previously suggested --bare clone with a non-bare --no-checkout
--separate-git-dir clone into a temporary work directory. After
which:
gives us the master-branch commit. Existing nmbug users will have to
run the configuration tweaks and re-fetch by hand. If you don't have
any local commits, you could also blow away your NMBGIT repository and
re-clone from scratch:
Besides removing the ambiguity of FETCH_HEAD, this commit allows users
to configure which upstream branch they want nmbug to track via 'git
config', in case they want to change their upstream repository.
Tomi Ollila [Sat, 5 Apr 2014 09:47:29 +0000 (12:47 +0300)]
release-checks: removed manual page version check
Manual pages are now generated and during the generation the version
string is read from `version` file, so this (currently failing) test
checking manual page versions can be removed.
While at it, changed the case pattern *[^0-9.]*
to its portable alternative *[!0-9.]*
Tomi Ollila [Thu, 3 Apr 2014 20:53:03 +0000 (23:53 +0300)]
doc: added 'Init File' section to notmuch-emacs info source
While adding that fixed (also other) typos noticed by aspell(1) run,
and capitalized Emacs and (most) Notmuch terms to match how emacs
Info documentation seems to look in general.
David Bremner [Mon, 27 Jan 2014 14:12:12 +0000 (10:12 -0400)]
compat: add canonicalize_file_name
the POSIX 2008 behaviour of realpath is not available everywhere so we
provide a simple wrapper function. We use (and provide) the gnu
extension canonicalize_file_name to make it cleaner to test for the
feature we need; otherwise we have to rely on realpath segfaulting if
the second argument is null.
David Bremner [Sun, 6 Apr 2014 12:53:39 +0000 (09:53 -0300)]
debian: really delete madduck from uploaders
I'm not sure how it got into debian/changelog without actually
happening, but actually delete martin from debian/control per
request in Debian bug #719100
David Bremner [Sun, 30 Mar 2014 14:54:15 +0000 (11:54 -0300)]
debian: ignore performance corpus when making source package
Currently "make debian-snapshot" will include the performance corpus
tarball in the source package, which slows things down and wastes disk
space. tar-ignore is needed twice to keep the default exclude rules
(e.g. to exclude .git)
Jani Nikula [Sun, 30 Mar 2014 21:21:49 +0000 (00:21 +0300)]
lib: replace the header parser with gmime
The notmuch library includes a full blown message header parser. Yet
the same message headers are parsed by gmime during indexing. Switch
to gmime parsing completely.
These are the main changes:
* Gmime stops header parsing at the first invalid header, and presumes
the message body starts from there. The current parser is quite
liberal in accepting broken headers. The change means we will be
much pickier about accepting invalid messages.
* The current parser converts tabs used in header folding to
spaces. Gmime preserve the tabs. Due to a broken python library used
in mailman, there are plenty of mailing lists that produce headers
with tabs in header folding, and we'll see plenty of tabs. (This
change has been mitigated in preparatory patches.)
* For pure header parsing, the current parser is likely faster than
gmime, which parses the whole message rather than just the
headers. Since we parse the message and its headers using gmime for
indexing anyway, this avoids and extra header parsing round when
adding new messages. In case of duplicate messages, we'll end up
parsing the full message although just headers would be
sufficient. All in all this should still speed up 'notmuch new'.
* Calls to notmuch_message_get_header() may be slightly slower than
previously for headers that are not indexed in the database, due to
parsing of the whole message. Within the notmuch code base, notmuch
reply is the only such user.
Jani Nikula [Sun, 30 Mar 2014 21:21:48 +0000 (00:21 +0300)]
lib: drop support for single-message mbox files
We've supported mbox files containing a single message for historical
reasons, but the support has been deprecated, with a warning message
while indexing, since Notmuch 0.15. Finally drop the support, and
consider all mbox files non-email.
Tomi Ollila [Sat, 29 Mar 2014 08:07:59 +0000 (10:07 +0200)]
emacs: add defcustom notmuch-init-file and load it if exists
So that users can easily organize their notmuch-specific configurations
to separate file and they don't have to have notmuch configurations in
*every* emacs installation they launch, especially if those need to
'(require notmuch) to make the configurations possible.
Austin Clements [Sun, 23 Mar 2014 21:22:26 +0000 (17:22 -0400)]
cli: Flush stdout before fork()ing to run hooks
Without this flush, if stdout is block buffered (which will happen if
it's a pipe or a file, for example) and the hook also writes to
stdout, then notmuch new's output will appear *after* the hook output.
This situation may be a little esoteric, but it's good practice to
flush before you fork anyway.
Austin Clements [Tue, 11 Mar 2014 22:19:54 +0000 (18:19 -0400)]
emacs: Use whitelist instead of blacklist for term escaping
Previously, the term escaper used a blacklist of characters that
needed escaping. This blacklist turned out to be somewhat incomplete;
for example, it did not contain non-whitespace ASCII control
characters or Unicode "fancy quotes", both of which do require the
term to be escaped.
Switch to a whitelist of characters that are definitely safe to leave
unquoted. This fixes the broken test introduced by the previous
patch.
Austin Clements [Tue, 11 Mar 2014 22:19:53 +0000 (18:19 -0400)]
test: Add broken test for Emacs boolean term escaping
The current term escaper gets most of these right, but fails to escape
things containing Unicode "fancy quotes" or things containing
non-whitespace control characters.
Jani Nikula [Wed, 12 Mar 2014 17:26:04 +0000 (19:26 +0200)]
test: conditionally test compact depending on configured support
I still have one machine with old enough Xapian to not have compaction
support. Make the tests check for unsupported compact operation when
compact is not available.
Mark Walters [Sat, 22 Mar 2014 11:51:12 +0000 (11:51 +0000)]
emacs: tree: use orig-tags in search
This uses the recent functionality to show the tag changes in the tree
buffer. Currently this is only used to show changes the tree buffer
makes itself: i.e., it does not make display any changes reflecting
tagging done by other notmuch-buffers.
Mark Walters [Sat, 22 Mar 2014 11:51:11 +0000 (11:51 +0000)]
emacs: search: use orig-tags in search
This uses the recent functionality to show the tag changes in the
search buffer. Currently this is only used to show changes the search
buffer makes itself: i.e., it does not make display any changes
reflecting tagging done by other notmuch-buffers.
Mark Walters [Sat, 22 Mar 2014 11:51:09 +0000 (11:51 +0000)]
emacs: show: mark tags changed since buffer loaded
This allows (and requires) the original-tags to be passed along with
the current-tags to be passed to notmuch-tag-format-tags. This allows
the tag formatting to show added and deleted tags.By default a removed
tag is displayed with strike-through in red (if strike-through is not
available, eg on a terminal, inverse video is used instead) and an
added tag is displayed underlined in green.
If the caller does not wish to use the new feature it can pass
current-tags for both arguments and, at this point, we do exactly that
in the three callers of this function.
Note, we cannot tidily allow original-tags to be optional because we would
need to distinguish nil meaning "we are not specifying original-tags"
from nil meaning there were no original-tags (an empty list).
We use this in subsequent patches to make it clear when a message was
unread when you first loaded a show buffer (previously the unread tag
could be removed before a user realised that it had been unread).
The code adds into the existing tag formatting code. The user can
specify exactly how a tag should be displayed normally, when deleted,
or when added.
Since the formatting code matches regexps a user can match all deleted
tags with a ".*" in notmuch-tag-deleted-formats. For example setting
notmuch-tag-deleted-formats to '((".*" nil)) tells notmuch not to show
deleted tags at all.
All the variables are customizable; however, more complicated cases
like changing the face depending on the type of display will require
custom lisp.
Currently this overrides notmuch-tag-deleted-formats for the tests
setting it to '((".*" nil)) so that they get removed from the display
and, thus, all tests still pass.
Mark Walters [Sat, 22 Mar 2014 11:51:08 +0000 (11:51 +0000)]
emacs: tag: add customize for deleted/added tag formats
Add customize options for deleted/added tag formats. These are not
used yet but will be later in the series.
We switch to using `notmuch-apply-face' rather than `propertize' in
the defcustom for faces so that the faces for deleted/added tags add
to the default face attributes for the tag.
We special case deleting the unread tag as that tag is a strong visual
cue and we don't need that cue when we are just saying it used to be
unread. Thus, we revert to the normal tag face with strikethough for
deleted unread tags.
Mark Walters [Sat, 22 Mar 2014 11:51:07 +0000 (11:51 +0000)]
emacs: tag split customise option for format-tags into a widget
We will re-use the customize option for format-tags for formattting
deleted tags to added tags in the next patch so split it into a
widget. There should be no functional change.
Austin Clements [Sat, 22 Mar 2014 11:51:06 +0000 (11:51 +0000)]
Make keys of notmuch-tag-formats regexps and use caching
This modifies `notmuch-tag-format-tag' to treat the keys of
`notmuch-tag-formats' as (anchored) regexps, rather than literal
strings. This is clearly more flexible, as it allows for prefix
matching, defining a fallback format, etc. This may cause compatibility
problems if people have customized `notmuch-tag-formats' to match tags
that contain regexp specials, but this seems unlikely.
Regular expression matching has quite a performance hit over string
lookup, so this also introduces a simple cache from exact tags to
formatted strings. The number of unique tags is likely to be quite
small, so this cache should have a high hit rate. In addition to
eliminating the regexp lookup in the common case, this cache stores
fully formatted tags, eliminating the repeated evaluation of potentially
expensive, user-specified formatting code. This makes regexp lookup at
least as fast as assoc for unformatted tags (e.g., inbox) and *faster*
than the current code for formatted tags (e.g., unread):
This combines our two face combining functions into one, easy to use
function with a much shorter name: `notmuch-apply-face'. This
function takes the full set of arguments that
`notmuch-combine-face-text-property' took, but takes them in a more
convenient order and provides smarter defaults that make the function
easy to use on both strings and buffers.
David Bremner [Thu, 13 Mar 2014 03:21:16 +0000 (00:21 -0300)]
doc: configure detection of sphinx and rst2man
Because sphinx-build does not provide a convenient way of listing
which builders exist, and some people actually have pre 1.0 sphinx, we
try loading a relevant python module.
Currently the assumption is that no python in path -> no sphinx-build
in path.
David Bremner [Thu, 13 Mar 2014 03:21:15 +0000 (00:21 -0300)]
doc: build man pages into hierarchy, fix help test.
It turns out there was a reason the old man pages were stored in a man
compatible hierarchy, namely so that we could run man on them before
installing.
Hardcode doc build location into test suite. This isn't ideal, but
let's unbreak the test suite for now.
Austin Clements [Tue, 11 Mar 2014 20:42:00 +0000 (16:42 -0400)]
util: Fix two corner-cases in boolean term quoting function
Previously, make_boolean_term did not quote empty boolean terms or
boolean terms that started with '('. These cases are incompatible
with Xapian: empty terms cannot be omitted, and boolean terms that
start with '(' trigger an alternate term quoting syntax.
Fix this by quoting empty terms and terms that contain '('.
David Bremner [Fri, 7 Mar 2014 12:03:23 +0000 (08:03 -0400)]
perf-test: use command line arguments for directories
It seems that between version 1.26 and 1.27 of gnu tar, directories to
be extracted read with --files-from are no longer recursively extacted.
This patch puts them on the command line instead.
David Bremner [Sat, 1 Mar 2014 23:22:59 +0000 (19:22 -0400)]
test: commit database checksum, ignore actual databases
The checksum file is used by the test infrastructure to verify the downloaded
test database is the one we had in mind. Note that this test is
rather strict, and the the checksum file needs to be recommitted when
the database is regenerated.
add a pattern .gitignore to ignore the actual databases
Jani Nikula [Tue, 11 Mar 2014 05:42:49 +0000 (07:42 +0200)]
test: add database upgrade test from format version 1 to 2
Test the upgrade from probabilistic to boolean folder: terms, and
addition of path: terms.
The test depends on the pre-built test corpus and database tarball and
checksum file being in place. If it's not, the test is skipped. The
mechanism to fetch the test database will be added later.
At the time of writing, a working test database and checksum file is
available at
http://notmuchmail.org/releases/test-databases/
It has been noted that some non-GNU environments make lack
sha256sum. We leave this portability issue for a followup patch.
Jani Nikula [Sat, 8 Feb 2014 19:20:42 +0000 (21:20 +0200)]
lib: make folder: prefix literal
In xapian terms, convert folder: prefix from probabilistic to boolean
prefix, matching the paths, relative from the maildir root, of the
message files, ignoring the maildir new and cur leaf directories.
folder:foo matches all message files in foo, foo/new, and foo/cur.
folder:foo/new does *not* match message files in foo/new.
folder:"" matches all message files in the top level maildir and its
new and cur subdirectories.
This change constitutes a database change: bump the database version
and add database upgrade support for folder: terms. The upgrade also
adds path: terms.
Finally, fix the folder search test for literal folder: search, as
some of the folder: matching capabilities are lost in the
probabilistic to boolean prefix change.
Jani Nikula [Sun, 9 Mar 2014 12:25:37 +0000 (14:25 +0200)]
test: rearrange the test corpus into subfolders, fix tests
We will need this for improved folder search tests, but having some
folders should exercise our code paths better anyway.
Modify the relevant test accordingly to make it pass.
This reorganization triggers a bug in the test suite, namely that it
expects the output of --output=files to be in a certain order. So we
add the fix for that into the same commit.
This mainly involves sorting, although the case --duplicate=$n
requires more subtlety.
Jani Nikula [Mon, 3 Feb 2014 19:51:41 +0000 (21:51 +0200)]
cli: sanitize tabs and newlines to spaces in notmuch search
Sanitize tabs and newlines to spaces rather than question marks in
--output=summary --format=text output.
This will also hide any difference in unfolding a header that has been
folded with a tab. Our own header parser replaces tabs with spaces,
while gmime would retain the tab.
David Bremner [Sat, 8 Mar 2014 14:00:39 +0000 (10:00 -0400)]
test: don't use $(dir) in recipes.
According the semantics of make, the expansion of $(dir) in recipes
uses dynamic scope, i.e. the value at the time the recipe is run. This
means if test/Makefile.local is not the last sub-makefile included,
all heck breaks loose.
David Bremner [Thu, 6 Mar 2014 12:52:02 +0000 (08:52 -0400)]
debian: add dependency on bash-completion
At some point we decided to only install bash completion for notmuch
if the bash-completion file was present. Add the corresponding debian
build dependency.
Austin Clements [Tue, 4 Mar 2014 06:40:08 +0000 (01:40 -0500)]
test: Print the number of the test along with its name
Previously, we stripped the "Tnnn-" part from the test name when
printing its description at the beginning of each test. However, this
makes it difficult to find the source script for a test (e.g., when a
test fails). Put this prefix back.
Jani Nikula [Sat, 1 Mar 2014 13:17:50 +0000 (15:17 +0200)]
emacs: use the originating buffer's working directory for pipe
Currently notmuch-show-pipe-message runs the command in the working
directory of the *notmuch-pipe* buffer if it exists, and the current
buffer's working directory (which is inherited to the new
*notmuch-pipe* buffer) otherwise. This is all very surprising to the
user, and it's difficult to know or change where the command will be
run.
Always use the current show buffer's working directory for piping. The
user can check that with M-x pwd and change it with M-x cd. This is
consistent with notmuch-show-pipe-part.
Austin Clements [Thu, 27 Feb 2014 18:10:54 +0000 (13:10 -0500)]
emacs: Simplify and fix `notmuch-mua-prompt-for-sender'
`notmuch-mua-prompt-for-sender' is over-engineered and often wrong.
It attempts to detect when all identities have the same name and
specialize the prompt to just the email address part. However, to do
this it uses `mail-extract-address-components', which is meant for
displaying email addresses, not general-purpose parsing, and hence
performs many canonicalizations that can interfere with this use. For
example, configuring notmuch-identities to ("Austin
<austin@example.com>"), will cause `notmuch-mua-prompt-for-sender' to
lose the name part entirely and return " <austin@example.com>".
This patch rewrites `notmuch-mua-prompt-for-sender' to simply prompt
for a full identity when notmuch-identities is configured, or to
prompt for a sender address when it isn't.
The original code also did several strange things, like using `eval'
and specifying that this function was interactive. As a side-effect,
this patch fixes these problems. And it adds a docstring.
David Bremner [Sat, 8 Feb 2014 14:25:17 +0000 (10:25 -0400)]
emacs: remove newlines from input to notmuch count --batch
Since a newline starts a new query in batch mode, this causes
mysterious crashes in the emacs interface if saved searches contain
newlines. See the discussion at
id:87wqhcxb5j.fsf@maritornes.cs.unb.ca
In general newlines seem to be just whitespace to the xapian query
parser, so this should be mainly harmless.
Austin Clements [Thu, 20 Feb 2014 19:16:35 +0000 (14:16 -0500)]
emacs: Fix exception when fetching empty or unconfigured settings
When "notmuch config" is called with the name of an empty or
unconfigured setting, it prints nothing (not even a new line).
Previously, `notmuch-config-get' assumed it would always print a
newline. As a result, when `notmuch-config-get' was called with the
name of an empty of unconfigured setting, it would attempt to
(substring "" 0 -1) to strip the newline, which would fail with a
(args-out-of-range "" 0 -1) exception.
Fix this by only stripping the newline if there actually is one.
Austin Clements [Thu, 20 Feb 2014 19:16:34 +0000 (14:16 -0500)]
emacs: Build forwarded message buffer more directly
Previously, we used `message-forward' to build forwarded messages, but
this function is simply too high-level to be a good fit for some of
what we do.
First, since `message-forward' builds a full forward message buffer
given the message to forward, we have to duplicate much of the logic
in `notmuch-mua-mail' to patch the notmuch-y things into the built
buffer.
Second, `message-forward' constructs the From header from
user-full-name and user-mail-address. As a result, if we prompt the
user for an identity, we have to parse it into name and address
components, just to have it put back together by `message-forward'.
This process is not entirely loss-less because
`mail-extract-address-components' does a lot of canonicalization
(since it's intended for displaying addresses, not for parsing them).
To fix these problems, don't use `message-forward' at all.
`message-forward' itself is basically just a call to `message-mail'
and `message-forward-make-body'. Do this ourselves, but call
`notmuch-mua-mail' instead of `message-mail' so we can directly build
a notmuch-y message and control the From header.
This also fixes a bug that was a direct consequence of our use of
`mail-extract-address-components': if the user chose an identity that
had no name part (or the name part matched the mailbox), we would bind
user-full-name to nil, which would cause an exception in the bowels of
message-mode because user-full-name is expected to always be a string
(even if it's just "").
Tomi Ollila [Tue, 18 Feb 2014 18:34:52 +0000 (20:34 +0200)]
nmbug-status: replace __values__() with values() in OrderedDict stub
Python dict() object does not have __values__() function which
OrderedDict().values() (the stub provided in nmbug-status) could call
to provide ordered list of values. By renaming this thinko to
values() will make our stub work as expected -- dict items listed out
in order those were added to the dictionary.
Austin Clements [Thu, 20 Feb 2014 04:24:24 +0000 (23:24 -0500)]
emacs: Avoid rebuilding .eldeps even when there's nothing to do
Previously, we updated .eldeps only if the file contents actually
needed to change. This was done to avoid unnecessary make restarts
(if the .eldeps rule changes the mtime of .eldeps, make has to restart
to collect the new dependencies). However, this meant that, after a
modification to any .el file that did not change dependencies, .eldeps
would always be out of date, so every make invocation would run the
.eldeps rule, which is both expensive because it starts up Emacs and
noisy. This was true even when there was nothing to do. E.g.,
$ make clean && make
...
$ touch emacs/notmuch-lib.el && make
...
$ make
Use "make V=1" to see the verbose compile lines.
EMACS emacs/.eldeps
make: Nothing to be done for `all'.
$ make
Use "make V=1" to see the verbose compile lines.
EMACS emacs/.eldeps
make: Nothing to be done for `all'.
Fix this by replacing .eldeps with two files with identical content.
One tracks the mtime of the dependency information and triggers the
Emacs call to rebuild dependencies only when it may be necessary. The
other tracks the content only; this rule over-triggers in the same way
the old rule did, but this rule is cheap and quiet.
David Bremner [Wed, 19 Feb 2014 01:05:14 +0000 (21:05 -0400)]
debian: update ruby binary extension install path
From Christian Hofstaedler, Debian bug 739120:
When ruby-defaults > 1:2.0 will be uploaded to sid, the version of
Ruby provided by the 'ruby' and 'ruby-dev' packages will change to
version 2.0. This version change includes a change to the path for
binary extensions.
This patch works for Ruby 1.9.1 as well as Ruby 2.0, so it's safe to
apply now.