mu-find - Online in the Cloud

PROGRAM:

NAME

mu_find - find e-mail messages in the mu database.

SYNOPSIS

mu find [options] <search expression>

DESCRIPTION

mu find is the mu command for searching e-mail message that were stored earlier using mu
index(1).

SEARCHING MAIL

mu find starts a search for messages in the database that match some search pattern. For
example:

$ mu find subject:snow from:john

would find all messages from John with 'snow' in the subject field, something like:

2009-03-05 17:57:33 EET Lucia <[email protected]> running in the snow
2009-03-05 18:38:24 EET Marius <[email protected]> Re: running in the snow

Note, this the default, plain-text output, which is the default, so you don't have to use
--format=plain. For other types of output (such as symlinks, XML or s-expressions), see
the discussion in the OPTIONS-section below about --format.

The search pattern is taken as a command-line parameter. If the search parameter consists
of multiple parts (as in the example) they are treated as if there were a logical AND
between them.

mu relies on the Xapian database for its searching capabilities, so it offers all the
search functionality that Xapian offers; for all the details, see:
http://xapian.org/docs/queryparser.html

One special feature of mu is that is does not distinguish between uppercase and lowercase,
nor the accented or unaccented versions of characters. All match. In general, mu tries to
be 'eager' in matching, as filtering out unwanted results is usually preferable over non
matching messages.

A wildcard search is a search where a * matches the last n character(s) in some string.
The string must always start with one or more characters before the wildcard. mu supports
wildcard searches for all fields except maildirs and paths. To get all mails with a
subject containing a word starting with com, you can use:

$ mu find 'subject:com*'

and get mails about computers, comments, compilation and so on. Note, when running from
the command-line it's important to put the query in quotes, otherwise the shell would
interpret the '*'. It is equally important to remember that the '*' invokes the wildcard
search only when used as the rightmost character of a search term. Furthermore, it is not
a regular expression.

The basic way to search a message is to type some words matching it, as you would do in an
internet search engine. For example,

$ mu find monkey banana

will find all messages that contain both 'monkey' and 'banana' in either body or subject
or one of the address-fields (to/from/cc).

As mentioned, matching is case-insensitive and accent-insensitive; thus

$ mu find Mönkey BÄNAÑå

yields the same results as the example above.

mu also recognizes prefixes for specific fields in a messages; for example:

$ mu find subject:penguin

to find messages with have the word penguin in the subject field. You can abbreviate
subject: to just s:. Here is the full table of the search fields and their abbreviations:

cc,c Cc (carbon-copy) recipient(s)
bcc,h Bcc (blind-carbon-copy) recipient(s)
from,f Message sender
to,t To: recipient(s)
subject,s Message subject
maildir,m Maildir
msgid,i Message-ID
prio,p Message priority ('low', 'normal' or 'high')
flag,g Message Flags
date,d Date-Range
size,z Message size
embed,e Search inside embedded text parts (messages, attachments)
file,j Attachment filename
mime,y MIME-type of one or more message parts
tag,x Tags for the message (X-Label and/or X-Keywords)
list,v Mailing list (e.g. the List-Id value)

There are also the special fields contact, which matches all contact-fields (from, to, cc
and bcc), and recip, which matches all recipient-fields (to, cc and bcc).

The meaning of most of the above fields should be clear, but some require some extra
discussion. First, the message flags field describes certain properties of the message, as
listed in the following table:

d,draft Draft Message
f,flagged Flagged
n,new New message (in new/ Maildir)
p,passed Passed ('Handled')
r,replied Replied
s,seen Seen
t,trashed Marked for deletion
a,attach Has attachment
z,signed Signed message
x,encrypted Encrypted message
l,list Mailing-list message

Using this, we can search e.g. for all signed messages that have an attachment:

$ mu find flag:signed flag:attach

Encrypted messages may be signed as well, but this is only visible after decrypting, and
thus, is invisible to mu.

The message-priority has three possible values: low, normal or high. We can match them
using prio: - for example, to get all high-priority messages with a subject containing
some bird:

$ mu find prio:high subject:nightingale

The Maildir field describes the directory path starting after the Maildir-base path, and
before the /cur/ or /new/ part. So for example, if there's a message with the file name
~/Maildir/lists/running/cur/1234.213:2,, you could find it (and all the other messages in
the same maildir) with:

$ mu find maildir:/lists/running

Note the starting '/'. If you want to match mails in the 'root' maildir, you can do with a
single '/':

$ mu find maildir:/

(and of course you can use the m: shortcut instead of maildir:)

The date: (or d:) search parameter is 'special' in the fact that it takes a range of
dates. For now, these dates are in ISO 8601 format (YYYYMMDDHHMM); you can leave out the
right part, and mu will add the rest, depending on whether this is the beginning or end of
the date interval. For example, for the beginning of the interval "201012" would be
interpreted as "20101201010000", or December 1, 2010 at 00:00, while for the end of the
interval, this would be interpreted as "20101231122359", or December 31, 2010 at 23:59. If
you omit the left part completely, the beginning date is assumed to be January 1, year 0
at 00:00. Likewise, if you omit the right part, the end data is assumed to be to the last
second of the year 9999.

To get all messages between (inclusive) the 5th of May 2009 and the 2nd of June 2010, you
could use:

$ mu find date:20090505..20100602

Non-numeric characters are ignored, so the following is equivalent but more readable:

$ mu find date:2009-05-05..2010-06-02

Precision is up to the minute and 24-hour notation for times is used, so another example
would be:

$ mu find date:2009-05-05/12:23..2010-06-02/17:18

mu also understand relative dates, in the form of a positive number followed by h (hour),
d (day), w (week), m (30 days) or y (365 days). Some examples to explain this:

5h five hours in the past
2w two weeks in the past
3m three times 30 days in the past
1y 365 days in the past

Using this notation, you can for example match messages between two and three weeks old:

$ mu find date:3w..2w

There are some special keywords for dates, namely 'now', meaning the present moment and
'today' for the beginning of today. So to get all messages sent or received today, you
could use:

$ mu find date:today..now

The size or z allows you to match size ranges -- that is, match messages that have a byte-
size within a certain range. Units (B (for bytes), K (for 1000 bytes) and M (for 1000 *
1000 bytes) are supported). For example, to get all messages between 10Kb and 2Mb
(assuming SI units), you could use:

$ mu find size:10K..2M

It's important to remember that if a search term includes spaces, you should quote those
parts. Thus, when we look at the following examples:

$ mu find maildir:/Sent Items yoghurt
$ mu find maildir:'/Sent Items' yoghurt

The first query searches for messages in the /Sent maildir matching Items and yoghurt,
while the second query searches the /Sent Items maildir searching for messages matching
yoghurt.

You can match all messages using "" (or ''):

$ mu find ""

OPTIONS

Note, some of the important options are described in the mu(1) man-page and not here, as
they apply to multiple mu-commands.

The find-command has various options that influence the way mu displays the results. If
you don't specify anything, the defaults are --fields="d f s", --sortfield=date and
--reverse.

-f, --fields=<fields>
specifies a string that determines which fields are shown in the output. This
string consists of a number of characters (such as 's' for subject or 'f' for
from), which will replace with the actual field in the output. Fields that are not
known will be output as-is, allowing for some simple formatting.

For example:

$ mu find subject:snow --fields "d f s"

would list the date, subject and sender of all messages with 'snow' in the their
subject.

The table of replacement characters is superset of the list mentions for search
parameters; the complete list:

t to: recipient
c cc: (carbon-copy) recipient
h Bcc: (blind carbon-copy, hidden) recipient
d Sent date of the message
f Message sender (from:)
g Message flags (flags)
l Full path to the message (location)
p Message priority (high, normal, low)
s Message subject
i Message-id
m maildir

The message flags are the same ones we already saw in the message flags above.
Thus, a message which is 'seen', has an attachment and is signed would have 'asz'
as its corresponding output string, while an encrypted new message would have 'nx'.

-s, --sortfield =<field> and -z,
--reverse specifies the field to sort the search results by, and the direction
(i.e., 'reverse' means that the sort should be reverted - Z-A). The following
fields are supported:

cc,c Cc (carbon-copy) recipient(s)
bcc,h Bcc (blind-carbon-copy) recipient(s)
date,d message sent date
from,f message sender
maildir,m maildir
msgid,i message id
prio,p message priority
subject,s message subject
to,t To:-recipient(s)

Thus, for example, to sort messages by date, you could specify:

$ mu find fahrrad --fields "d f s" --sortfield=date --reverse

Note, if you specify a sortfield, by default, messages are sorted in reverse
(descending) order (e.g., from lowest to highest). This is usually a good choice,
but for dates it may be more useful to sort in the opposite direction.

-n, --maxnum=<number>
If > 0, display maximally that number of entries. If not specified, all matching
entries are displayed.

--summary-len=<number>
If > 0, use that number of lines of the message to provide a summary.

--format=plain|links|xquery|xml|sexp
output results in the specified format.

The default is plain, i.e normal output with one line per message.

links outputs the results as a maildir with symbolic links to the found messages.
This enables easy integration with mail-clients (see below for more information).
See --linksdir and --clearlinks below.

xml formats the search results as XML.

sexp formats the search results as an s-expression as used in Lisp programming
environments.

xquery shows the Xapian query corresponding to your search terms. This is meant for
for debugging purposes.

--linksdir =<dir> and -c, --clearlinks
output the results as a maildir with symbolic links to the found messages. This
enables easy integration with mail-clients (see below for more information). mu
will create the maildir if it does not exist yet.

If you specify --clearlinks, all existing symlinks will be cleared from the target
maildir; this allows for re-use of the same directory. An alternative would be to
delete the target directory before, but this has a big chance of accidentally
removing something that should not be removed.

$ mu find grolsch --linksdir=~/Maildir/search --clearlinks

will store links to found messages in ~/Maildir/search. If the directory does not
exist yet, it will be created.

Note: when mu creates a Maildir for these links, it automatically inserts a
.noindex file, to exclude the directory from mu index.

--after=<timestamp> only show messages whose message files were
last modified (mtime) after <timestamp>. <timestamp> is a UNIX time_t value, the
number of seconds since 1970-01-01 (in UTC).

From the command line, you can use the date command to get this value. For example,
only consider messages modified (or created) in the last 5 minutes, you could
specify
--after=`date +%s --date='5 min ago'`
This is assuming the GNU date command.

--exec=<command>
the --exec command causes the command to be executed on each matched message; for
example, to see the raw text of all messages matching 'milkshake', you could use:
$ mu find milkshake --exec='less'
which is roughly equivalent to:
$ mu find milkshake --fields="l" | xargs less

-b, --bookmark=<bookmark>
use a bookmarked search query. Using this option, a query from your bookmark file
will be prepended to other search queries. See mu-bookmarks(1) for the details of
the bookmarks file.

--skip-dups,-u whenever there are multiple messages with the
same name, only show the first one. This is useful if you have copies of the same
message, which is a common occurrence when using e.g. Gmail together with
offlineimap.

--include-related,-r also include messages being refered to by
the matched messages -- i.e.. include messages that are part of the same message
thread as some matched messages. This is useful if you want Gmail-style
'conversations'. Note, finding these related messages make searches slower.

-t, --threads show messages in a 'threaded' format -- that is,
with indentation and arrows showing the conversation threads in the list of
matching messages.

Messages in the threaded list are indented based on the depth in the discussion,
and are prefix with a kind of arrow with thread-related information about the
message, as in the following table:

| | normal | orphan | duplicate |
|-------------+--------+--------+-----------|
| first child | `-> | `*> | `=> |
| other | |-> | |*> | |=> |

Here, an 'orphan' is a message without a parent message (in the list of matches),
and a duplicate is a message whose message-id was already seen before; not this may
not really be the same message, if the message-id was copied.

The algorithm used for determining the threads is based on Jamie Zawinksi's
description: http://www.jwz.org/doc/threading.html

Example queries
Here are some simple examples of mu search queries; you can make many more complicated
queries using various logical operators, parentheses and so on, but in the author's
experience, it's usually faster to find a message with a simple query just searching for
some words.

Find all messages with both 'bee' and 'bird' (in any field)

$ mu find bee AND bird

or shorter, because AND is implied:

$ mu find bee bird

Find all messages with either Frodo or Sam:

$ mu find 'Frodo OR Sam'

Find all messages with the 'wombat' as subject, and 'capibara' anywhere:

$ mu find subject:wombat capibara

Find all messages in the 'Archive' folder from Fred:

$ mu find from:fred maildir:/Archive

Find all unread messages with attachments:

$ mu find flag:attach flag:unread

Find all messages with PDF-attachments:

$ mu find mime:application/pdf

Find all messages with attached images:

$ mu find 'mime:image/*'

Note[1]: the argument needs to be quoted, or the shell will interpret the '*' Note[2]: the
'*' wild card can only be used as the last (rightmost) part of a search term. Note[3]:
non-word characters (such as € or ☺) are ignore in queries; you cannot search for them.

Integrating mu find with mail clients
mutt

For mutt you can use the following in your muttrc; pressing the F8 key will start a
search, and F9 will take you to the results.

# mutt macros for mu
macro index <F8> "<shell-escape>mu find --clearlinks --format=links --linksdir=~/Maildir/search " \
"mu find"
macro index <F9> "<change-folder-readonly>~/Maildir/search" \
"mu find results"

Wanderlust

Sam B suggested the following on the mu-mailing list. First add the following to
your Wanderlust configuration file:

(require 'elmo-search)
(elmo-search-register-engine
'mu 'local-file
:prog "/usr/local/bin/mu" ;; or wherever you've installed it
:args '("find" pattern "--fields" "l") :charset 'utf-8)

(setq elmo-search-default-engine 'mu)
;; for when you type "g" in folder or summary.
(setq wl-default-spec "[")

Now, you can search using the g key binding; you can also create permanent virtual
folders when the messages matching some expression by adding something like the
following to your folders file.

VFolders {
[date:today..now]!mu "Today"

[size:1m..100m]!mu "Big"

[flag:unread]!mu "Unread"
}

After restarting Wanderlust, the virtual folders should appear.

Wanderlust (old)

Another way to integrate mu and wanderlust is shown below; the aforementioned
method is recommended, but if that does not work for some reason, the below can be
an alternative.

(defvar mu-wl-mu-program "/usr/local/bin/mu")
(defvar mu-wl-search-folder "search")

(defun mu-wl-search ()
"search for messages with `mu', and jump to the results"
(let* ((muexpr (read-string "Find messages matching: "))
(sfldr (concat elmo-maildir-folder-path "/"
mu-wl-search-folder))
(cmdline (concat mu-wl-mu-program " find "
"--clearlinks --format=links --linksdir='" sfldr "' "
muexpr))
(rv (shell-command cmdline)))
(cond
((= rv 0) (message "Query succeeded"))
((= rv 2) (message "No matches found"))
(t (message "Error running query")))
(= rv 0)))

(defun mu-wl-search-and-goto ()
"search and jump to the folder with the results"
(interactive)
(when (mu-wl-search)
(wl-summary-goto-folder-subr
(concat "." mu-wl-search-folder)
'force-update nil nil t)
(wl-summary-sort-by-date)))

;; querying both in summary and folder
(define-key wl-summary-mode-map (kbd "Q") ;; => query
'(lambda()(interactive)(mu-wl-search-and-goto)))
(define-key wl-folder-mode-map (kbd "Q") ;; => query
'(lambda()(interactive)(mu-wl-search-and-goto)))

RETURN VALUE

mu find returns 0 upon successful completion; if the search was performed, there needs to
be a least one match. Anything else leads to a non-zero return value, for example:

| code | meaning |
|------+--------------------------------|
| 0 | ok |
| 1 | general error |
| 2 | no matches (for 'mu find') |
| 4 | database is corrupted |

ENCODING

mu find output is encoded according the locale for --format=plain (the default), and UTF-8
for all other formats (sexp, xml).

Use mu-find online using onworks.net services