performance.mdwn



# Optimize notmuch's performance

[[!toc levels=2]]

## Xapian 1.2.x database format: chert vs. flint

Xapian 1.2 (in contrast to 1.0.x) uses a new database format called
`chert` which is a lot faster than the old format `flint`.

Check whether you are on chert or flint:

    $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
    iamchert

If it does not say `iamchert`, but `iamflint`, than you should migrate
(read on).

There might be a file named `flintlock`, it is no indicator for your
database format, but also used in case of `chert`.

### Migrate from flint to chert

Make sure you are using xapian 1.2.x (depends on your distribution)

XXX: add examples for different distributions

Backup your notmuch tags:

    $ cd
    $ notmuch dump notmuch-dump-b4chert

Move away your old database:

    $ mv $(notmuch config get database.path)/.notmuch/ notmuch-db-b4chert

Find and import your messages, a new and fresh database will be
created:

    $ XAPIAN_PREFER_CHERT=1 notmuch new

I did not need the `XAPIAN_PREFER_CHERT`, but am not sure whether
there are situations making it necessary (certain compile-time options
to xapian?). This note is just here to be extended or removed by
somebody more knowledgeable.

After the initial import it is definitely not necessary to define
`XAPIAN_PREFER_CHERT` for subsequent calls of `notmuch new`.

Restore your tags:

    $ notmuch restore notmuch-dump-b4chert

Check whether you are on chert now:

    $ ls $(notmuch config get database.path)/.notmuch/xapian/iam*
    iamchert

  If it says `iamflint`, most likely you are still using xapian 1.0.x.

Clean-up: if you are sure everything is fine, you can delete the old
things. The dump of your tags you might want to keep anyway as a
backup - on the other hand you should have some mechanism in place to
make regular backups of your notmuch database:

    $ rm notmuch-dump-b4chert
    $ rm -R notmuch-db-b4chert

### Compact your database

In order to greatly reduce the size of your database, you should
compact it. Carl says "This functionality is not yet available in the
Xapian library interface or else I would probably make notmuch call it
after building the database." and Pieter "[compacting] will cause
writes to be a bit more expensive in the short term, but will
significantly reduce the db's size, which your SSD will surely
appreciate."

This needs xapian 1.2

Backup your database:

    $ cd
    $ notmuch dump notmuch-dump-b4compact

Compact it:

    $ cd $(notmuch config get database.path)/.notmuch
    $ xapian-compact --no-renumber xapian xapian-compacted

Remove old and renamed compacted, if compacting went well:

    $ rm -R xapian
    $ mv xapian-compacted xapian


### More information about this in the email archives:

* [Carl Worth](http://notmuchmail.org/pipermail/notmuch/2010/001105.html)

* [Sebastian Spaeth](http://notmuchmail.org/pipermail/notmuch/2011/004048.html)

* [Pieter Praet](http://notmuchmail.org/pipermail/notmuch/2011/004175.html)