Commit Graph

212 Commits

Author SHA1 Message Date
Robin Jarry 4cf0ea2a05 imap: use builtin idle support
go-imap supports IDLE since 1.2.0. Remove dependency to go-imap-idle.

Link: https://github.com/emersion/go-imap/commit/ac3f8e195ef1b6d
Signed-off-by: Robin Jarry <robin@jarry.cc>
2021-11-02 11:24:33 +01:00
Reto Brunner bc48628839 maildir: track the recent flag correctly
In the maildir worker we manually need to track the Recent flag in order for the
notification command etc to work.

Push that responsibility to the container, we must make sure to manually add the
flag though if one grabs the message info.
2021-11-01 13:30:45 +01:00
Reto Brunner 1687e558d3 notmuch/maildir: remove double emit of the dirinfo
There was some bug which could be worked around by double emitting an event.
However that proofed to be brittle:

We send the first message here from the worker goroutine:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g=
o#L306

Then Tick() is waked in the main goroutine and calls ProcessMessage:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/widgets/account.go#L100

ProcessMessage in the main goroutine reads types.Message state with
msg.getId() and msg.InResponseTo():
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#=
L74-76

Meanwhile in the worker goroutine we call PostMessage for a second
time with a pointer that points to the *same* previous message that
ProcessMessage is reading:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g=
o#L306

The second PostMessage call makes writes to message while
ProcessMessage in the main goroutine is possibly reading:
https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#=
L59

This led to a data race in the event loop

Reported-By: Wagner Riffel <w@104d.net>
2021-04-28 07:54:16 +02:00
Reto Brunner fda424bebc lib/parse: simplify parseAddressList 2021-02-22 11:47:31 +01:00
Reto Brunner 8ecf0b73f4 maildir: don't read the full file into memory 2021-02-08 09:06:42 +01:00
Reto Brunner db6848fe1b notmuch: don't read the full file into memory 2021-02-08 08:40:07 +01:00
Kalyan Sriram b56a688589 notmuch: trim excluded tags
Trims whitespace in list of excluded notmuch tags. This allows a comma
separated list with spaces to be correctly processed.
2020-12-02 09:08:19 +01:00
Reto Brunner a9330f4c63 notmuch: remove gc close hooks
We frequently had issues with notmuch segfaulting and my guess is that this
was due to the garbage collection magic used in the module.

This changes to a fork that ripped the functionality out.
2020-12-02 09:06:34 +01:00
Reto Brunner daf214a1f8 imap: strip <> from message-ids 2020-11-14 15:40:13 +01:00
Reto Brunner 18b0ea571a lib/parse: use go-message msgid parsing if it succeeds 2020-11-14 15:40:13 +01:00
Reto Brunner fc9ccc3000 remove models.Address in favor of go-message Address
We made a new type out of go-message/mail.Address without any real reason.
This suddenly made it necessary to convert from one to the other without actually
having any benefit whatsoever.
This commit gets rid of the additional type
2020-11-14 15:40:13 +01:00
Reto Brunner b6bcf89784 imap: add sort support 2020-10-11 09:18:45 +02:00
Reto Brunner c48f228fa5 notmuch: rename method to SetFlag 2020-09-27 19:00:58 +02:00
Reto Brunner 6654c970e6 notmuch: close tag object 2020-09-24 21:24:44 +02:00
Jeff Martin 0acb28645f handle message unknown charset error
This change handles message parse errors by printing the error when the
user tries to view the message. Specifically only handling unknown
charset errors in this patch, but there are many types of invalid
messages that can be handled in this way.

aerc currently leaves certain messages in the msglist in the pending
(spinner) state, and I'm unable to view or modify the message. aerc also
only prints parse errors with message when they are initially loaded.
This UX is a little better, because you can still see the header info
about the message, and if you try to view it, you will see the specific
error.
2020-08-31 22:00:28 +02:00
Reto Brunner c846307144 base models.Address on the mail.Address type
This allows us to hook into the std libs implementation of parsing related stuff.
For this, we need to get rid of the distinction between a mailbox and a host
to just a single "address" field.

However this is already the common case. All but one users immediately
concatenated the mbox/domain to a single address.

So this in effects makes it simpler for most cases and we simply do the
transformation in the special case.
2020-08-20 19:18:57 +02:00
ARaspiK fe1cabb077 Add support for :rmdir
The `:rmdir` command removes the current directory (`-f` is required if
the directory is not empty).

This is not supported on the notmuch backend.

An issue with the maildir backend is that some sync programs (e.g.
offlineimap) may recover the directory after it is deleted.  They need
to specifically be configured to accept deletions, or special commands
need to be executed (e.g. `offlineimap --delete-folder`) to properly
delete folders.

A danger of using this on the IMAP backend is that it is possible for a
new message to be added to the directory and for aerc to not show it
immediately (due to a slow connection) - using `:rmdir` at this moment
(with `-f` if the directory already contains messages) would delete the
directory and the new message that just arrived (and all other
contents).  This is documented in aerc(1) so that users are aware of
possible risks.
2020-08-19 11:38:57 +02:00
Reto Brunner f1a0fd20d6 improve date parsing for notmuch/maildir
If a message date would fail to parse, the worker would never receive
the MessageInfo it asked for, and so it wouldn't display the message.

The problem is the spec for date formats is too lax, so trying to ensure
we can parse every weird date format out there is not a strategy we want
to pursue. On the other hand, preventing the user from reading and
working with a message due to the error format is also not a solution.

The maildir and notmuch workers will now fallback to the internal date, which
is based on the received header if we can't parse the format of the Date header.

The UI will also fallback to the received header whenever the date header can't
be parsed.

This patch is based on the work done by Lyudmil Angelov <lyudmilangelov@gmail.com>
But tries to handle a parsing error a bit more gracefully instead of just returning
the zero date.
2020-08-10 08:00:52 +02:00
Reto Brunner 7a9fe3824f notmuch: manually close notmuch objects
There seems to be some race with the automatic closing that should happen
in theory... close it manually where we can to avoid the issue
2020-08-07 09:49:57 +02:00
Tero Koskinen f81cc2803c maildir: Provide nicer error message on invalid url
If accounts.conf contains an invalid maildir url, return a nice
error instead of panicking.

Log a couple of different error cases to provide extra
information about the error to the user.
2020-07-27 18:26:57 +02:00
Reto Brunner c574a838fa Remove hard coded bodystruct path everywhere
Aerc usually used the path []int{1} if it didn't know what the proper path is.
However this only works for multipart messages and breaks if it isn't one.

This patch removes all the hard coding and extracts the necessary helpers to lib.
2020-07-27 09:19:27 +02:00
Tobias Wölfel 494bd674a9 Add flag based search options
Provide search and filter with the option to specify more flag based
conditions.
Use '-x <flag>' to search for messages with a flag (seen, answered,
flagged) and '-X <flag>' to search for messages without a flag.
2020-07-25 08:08:08 +02:00
Andrew Jeffery 126c9437e8 Only send directory info once from maildir
The directory info only needs to be sent once for all the messages, this
reduces unnecessary messages being sent which could lock up the ui.
2020-07-19 11:16:05 +02:00
Reto Brunner 8446d48664 run go fmt 2020-07-17 17:50:24 +02:00
Lyudmil Angelov cc44027545 Make it easier to debug date parsing errors
When message dates failed to parse, the error displayed would try to
include the time object it failed to obtain, which would display as
something like 0001-01-01 00:00:00 UTC, which isn't of much help.

Instead, display the text we were trying to parse into a date, which
makes the problem easier to debug.
2020-07-11 13:51:32 +02:00
ARaspiK 0535f6333f Add additional flagging functionality
More mail flags can now be set, unset, and toggled, not just the
read/seen flag.

This functionality is implemented with a new `:flag` and `:unflag`
command, which are extensions to the matching `:read` and `:unread`
commands, adding support for different flags.  In fact, the
`read`/`unread` commands are now recognized aliases to `flag`/`unflag`.
The new commands are also well documented in aerc(1).

The change mostly extends the previous read/unread setting functionality
by adding a selection for the flag to change.
2020-07-08 09:13:03 +02:00
Reto Brunner 3e6189f243 notmuch: fix docstring 2020-07-08 01:02:46 +02:00
Reto Brunner 5f9c005872 notmuch: handle the answered flag 2020-07-05 15:40:12 +02:00
Srivathsan Murali 6c4ed3cfe2 notmuch: undefined variable when setting reply flag 2020-05-26 11:18:11 -04:00
Srivathsan Murali b1eb7ad18d Set AnsweredFlag on successful reply 2020-05-25 11:29:53 -04:00
Reto Brunner bae678e8f2 imap: Remove FetchMessageBodyPart.{Encoding,Charset}
Fixes https://todo.sr.ht/~sircmpwn/aerc2/352 exactly as suggested by emersion.
2020-05-16 20:03:42 +02:00
Reto Brunner fefb08f831 maildir: remove read handling from FetchMessageBodyPart 2020-05-11 09:47:21 -04:00
Reto Brunner ee1d971554 notmuch: remove read handling from FetchMessageBodyPart 2020-05-11 09:47:19 -04:00
elumbella d1600e4666 Guess date from received if not present 2020-05-06 10:00:33 -04:00
Grégoire Delattre 9a520c4098 Allow maildir subdirectories 2020-05-05 19:18:26 +02:00
Ben Fiedler 05fa79eb8e store.FetchFull: Change callback type to expose entire message
This is a prerequisite for allowing the FetchFull message to return both
the message content and the message headers.
2020-05-01 11:10:08 -04:00
Ben Fiedler 92f59ef94a imap/fetch.go: report error when fetching headers fails
Correct me if I'm wrong, but shouldn't this raise an error when it
fails?
2020-04-24 12:59:21 -04:00
Jeffas 08eabef17d Count recent messages in maildir exists total too 2020-03-09 09:13:46 -04:00
Amir Yalon ae7595e945 Ignore missing message part in getDecodedPart
The code that calls this function handles nil without issues.
2020-03-09 09:12:33 -04:00
Drew DeVault f3158b36f1 Initial support for PGP decryption & signatures 2020-03-03 16:49:52 -05:00
Simon Ser f3535703b0 worker/imap: rely on go-imap for charset handling
Set imap.CharsetReader so that go-imap can automatically decode all
encoded fields.
2020-03-03 16:48:00 -05:00
Galen Abell 6ff3c7a1ba Mark sent messages as "seen" in maildir
- Add maildir flags to complement a messages imap flags
- Set the "seen" flag on sent messages when using the maildir backend
- Cleanup AppendMessage interface to use models.Flag for both IMAP and
  maildir
2020-03-03 20:08:37 +01:00
Reto Brunner 979da6d6f7 notmuch: emit dirinfo upon label change 2020-03-01 11:58:39 +01:00
Reto Brunner f3dbecaaea remove the dirInfoUpdateRequest functionality
The notmuch worker followed suit in handling the dirInfo submission manually.
That removes the last user so we might as well remove the functionality.
2020-02-29 13:16:33 -05:00
Reto Brunner fb2df7e169 notmuch: align dirInfo logic to the maildir worker 2020-02-29 13:16:33 -05:00
Jeffas e8b7b3bcc1 Cleanup sorting logic
There was an unused error value as well as unnecessary usage of the sort
interface. There should now be less copying so a bit better performance
in some cases.
2020-02-28 20:54:14 -05:00
Jeffas 01c96e78df Update DirectoryInfo handling for maildir
This ensures that the directory info is up to date on events in the
maildir worker. This also sets up the initial dirinfo for other
directories and updates them when using built-in commands.

FS events are still only watched for the selected directory. This should
be changed in a future patch to watch other directories too in order to
cover UI updates for folders when an event occurs in a non-selected
folder.
2020-02-28 09:26:17 -05:00
Jeffas d6696f34b8 Make search more lenient
When we fail to read a single message we don't need to fail the whole
search, just log the error and we can still get results back.
2020-02-26 21:46:35 +01:00
Jeffas 00ed2ef06a Add missing notmuch build tags 2020-02-26 21:46:31 +01:00
Galen Abell 5ba1569734 Ensure new directory exists before unwatching old 2020-02-25 16:36:37 -05:00
Reto Brunner 5dd0f454c1 lib/msgstore: debounce directoryUpdateRequests
Apparently sending an event for every incoming messageInfo slows down
the application significantly.

Therefore this slows down the emmision rate, on the cost of being out of date
in some cases.
2020-02-25 11:05:26 -05:00
Reto Brunner 78dd043057 notmuch: refresh dirlist in background 2020-02-16 10:41:19 -05:00
Reto Brunner 8d216ab10e notmuch: add internal event loop 2020-02-16 10:41:18 -05:00
Reto Brunner bb09694f07 notmuch: emit DirInfo upon request 2020-02-16 10:41:18 -05:00
Reto Brunner 89b742dc6c Request DirInfo whenever the unread count may have changed
Actions such as read / unread or the addition of new messages do change
the read/unread/recent count. Hence we request an update from the workers.
Workers going over the network should probably cache the information and invalidate
it only if necessary
2020-02-16 10:41:17 -05:00
Reto Brunner 30c0a9fba4 imap: fix double closing idleStop
The idle restart code is at the end of handleMessage in the worker.
However if an unsupported msg comes in, we returned early, skipping the re-init.
That lead to a crash due to double closing idleStop in the next iteration.
2020-02-16 10:41:11 -05:00
Reto Brunner 3e7e236f50 notmuch: avoid stale DBs
Opening a notmuch DB gives you a snapshot of the stage at that specific time.
Prior to this, we only reopened the DB upon writing.
However, if say a mail sync program like offlineimap is fetching new mail,
we would never pick it up.

This commit caches a db for a while, so that we don't generate too much overhead
and does a reconnect cycle after that.

I hardcoded a value as I don't think that having an option would be beneficial.
Any write operation (meaning reading mail) anyhow flushes the DB by necessity.
(we need to close to commit tag changes, which changing the read state is)
2020-02-16 10:41:10 -05:00
Timmy Douglas 8c8096da97 worker/lib/parse: be more tolerant with parsing email addresses 2020-02-06 14:38:55 -05:00
Reto Brunner df5d9a3ec7 imap: emit messageinfo when changing read state.
We need to emit the changed msgInfo whenever we modify the state
2020-01-24 10:51:09 -05:00
Reto Brunner 472c421e85 worker/imap: don't decode in FetchFullMessage.
Doing that breaks `git am` as it expected the encoded variant.
Same is probably true for any sort of signature validation (gpg / dkim)
2020-01-18 19:00:02 -05:00
Reto Brunner da6fb1a155 maildir/notmuch: don't re-encode readers 2020-01-05 16:02:46 -05:00
Reto Brunner 3d85f75d9c imap: decode reader prior to returning them 2020-01-05 16:02:45 -05:00
Reto Brunner 9096049f75 FetchBodyParts: decode source in the workers
Previously the workers returned a mixture of decoded / encoded parts.
This lead to a whole bunch of issues.
This commit changes the msgviewer and the commands to assume parts to already
be decoded
2020-01-05 16:02:45 -05:00
Reto Brunner 63391b7dca Add labels to index format (%g)
Exposes the notmuch tags accordingly, stubs it for the maildir worker.
2019-12-27 10:20:29 -07:00
Reto Brunner 40ceee969b notmuch: emit LabelList event 2019-12-21 11:20:11 -05:00
Reto Brunner 2705d8460d add LabelList event 2019-12-21 11:20:11 -05:00
Leszek Cimała e84e402e48 decode(RFC 2047) imap headers to fix encoding in subject & filenames
Me again,
this time fixing encoding of subjects and attachments. It was problem in
IMAP backend. While other backends user MessageInfo() function which
generates MessageInfo decoded via go-message methodes, IMAP worker is
creating MessageInfo directly, so all non-utf8 subjects and filenames
were in raw form.

This patch fixes it. Not sure if we should care about errors (if
DecodeHeader fails it returns raw string back).

>From what I see, this should solve all encoding problem (tested only
IMAP). So, now I can focus on features. ;-)

Have a great weekend!
Leszek
2019-12-07 14:30:33 -05:00
Srivathsan Murali 59c24523af Parse Reply-To header while parsing envelope 2019-11-17 13:24:23 -05:00
Kevin Kuehler 8a848303fe worker/imap: Fix seqMap race condition
When deleting a message, sometimes FetchDirectoryContents will fire.
FetchDirectoryContents will return a smaller set of UIDs since messages
have been deleted. This operation races with fetching from the seqMap in
client.ExpungeUpdate. Only recreate the seqMap if it can grow so that
messages will continue to be expunged.

Signed-off-by: Kevin Kuehler <keur@xcf.berkeley.edu>
2019-11-10 17:19:23 -05:00
Reto Brunner 6dc537fbe3 Notmuch: be resilient to config errors
Right now notmuch panics if something goes wrong in the configure event.
This patch checks for that and returns an error instead, so that we can at least
get the UI up and running (and all the other accounts)

The experience will be completely degraded until another configure event occurs.
2019-11-01 11:02:01 -04:00
Matt Snider cd39e8f90c notmuch: ignore comments and blank lines when processing query-map file
A segmentation fault occurs when using the notmuch backend and a `query-map`
file that contains blank lines or comments.
2019-11-01 11:02:01 -04:00
Frode Aannevik 550ef0bc1f Fix: oauthbearer runtime error
Configure an oauthbearer source without a token_endpoint
parameter would panic due to nil pointer dereference

  Example
    source=imaps+oauthbearer://frode.aa%40gmail.com@imap.gmail.com:993
    source-cred-cmd=pass oatuh2 frode.aa@gmail.com

token_endpoint is not required as it will use the provided
password as access_token when it is not set
2019-10-16 11:04:41 -04:00
Jeffas d2a18e267c Change search flags
This changes the search flags for maildir and imap backends.

They now no longer use -t for searching all text. This seems to make
more sense as being the targeted recipient. I have similarly added Cc
for -c. The text search now resides under -a for all text.
2019-09-20 15:06:54 -04:00
Reto Brunner 3ec9fd216d notmuch: add sort functionality 2019-09-20 15:01:07 -04:00
Jeffas 90d26da58a Add sorting functionality
There is a command and config option. The criteria are a list of the
sort criterion and each can be individually reversed.

This only includes support for sorting in the maildir backend currently.
The other backends are not supported in this patch.
2019-09-20 14:56:02 -04:00
Jeffas 43435ba06c Add directory info messages
This populates the directory info model properly when requested,
allowing the fields to be relied upon elsewhere.

This also sends the dirinfo when new messages come in.
2019-09-18 19:34:30 -04:00
Jeffas c97d0d6320 Add basic searching to the maildir backend
Basic searching is supported with the following:
- read messages
- unread messages
- from addresses
- text in body
- text in subject
- text in all

The implementation loops through all messages in the selected directory.
It tries to be smart by detecting which parts of each message the search
query needs to use and only loads these from the filesystem.
2019-09-16 12:40:26 -04:00
Reto Brunner edfab1b201 notmuch: implement ModifyLabels 2019-09-16 12:19:15 -04:00
Reto Brunner 3f452ca283 notmuch: sync maildir flags
Syncs back special notmuch tag like unread to the underlying maildir store
2019-09-16 12:19:14 -04:00
Reto Brunner 83207c1e1d notmuch: extract all notmuch db operations.
For some reason the current code frequently segfaults due to an
invalid C memory address. This commit mediates that by never keeping an object
alive longer than absolutely necessary.
2019-09-16 12:19:13 -04:00
Reto Brunner a93b4de6f3 Add modify-labels command
This adds the event type as well as the command implementation, but no backend
supports it yet.
2019-09-12 16:17:24 -04:00
Jeffas 32381592fc Add new search behaviour for imap
This patch adds search behaviour to allow searching in the body of the
messages, the entire text (body + header), and searching just the from
header.
2019-09-11 12:24:27 -04:00
Wagner Riffel baa70469c3 all: rewrite references to strings.Index to strings.Contains
Signed-off-by: Wagner Riffel <wgrriffel@gmail.com>
2019-09-04 16:30:57 -10:00
Reto Brunner f13f9a8684 notmuch: add search / filter capabilities 2019-08-29 08:44:11 +09:00
Reto Brunner 94b9d557de extract search criteria parsing into the backends 2019-08-29 08:44:10 +09:00
Reto Brunner a2c5233f71 Notmuch: use adhoc write connection.
Notmuch only allows a single write connection, all other clients trying to
modify the db block. Hence we should only open one when we actually need it.

Apparently we also need to refresh the RO DB connection upon modification,
else we get stale message tag results
2019-08-26 09:56:44 +09:00
Wagner Riffel 90453b5db4 imap worker: do not copy mailbox status mutex
Signed-off-by: Wagner Riffel <wgrriffel@gmail.com>
2019-08-26 09:54:57 +09:00
Ben Burwell a1749411e6 maildir: Preserve flags when copying messages 2019-08-08 12:50:54 +09:00
Reto Brunner c38ddf8d30 Add notmuch backend
This commit introduces the notmuch backend.
The backend is conditionally compiled in if the "notmuch" tag is provided.

Most of the message types are implemented, with the notable exceptions
of DeleteMessages as well as any copy / move / append type.
Reason being, that those aren't normally applicable in a notmuch based workflow.

Changes v2 --> v3, based on review comments
* Use account config for configuration
2019-08-08 10:10:34 +09:00
Reto Brunner 07a4b0fef0 Extract message parsing to common worker module
Things like FetchEntityPartReader etc can be reused by most workers
working with raw email files from disk (or any reader for that matter).

This patch extract that common functionality in a separate package.
2019-08-08 10:10:31 +09:00
Jelle Besseling aeca428c14 Don't print imap errors to ui
This fixes ~sircmpwn/aerc2#245. This sets up the imap client to send
error messages to the logger of the worker. Errors now end up in the
bottom status line.

https://todo.sr.ht/~sircmpwn/aerc2/245
2019-08-08 09:47:15 +09:00
Ben Burwell 04ccbd09b1 Clean maildirs when opened
This removes old aborted deliveries from the tmp directory.
2019-08-02 09:26:07 -04:00
Nicolai Dagestad d526786c93 Fix error handling in maildir worker 2019-07-19 17:33:33 -04:00
Reto Brunner 6fed04bb9f Register worker in init.
This allows backends which can't always be compiled due to missing
dependencies (say libnotmuch) to be compiled conditionally with buildflags.
2019-07-19 15:37:55 -04:00
Ben Burwell 6473848d87 maildir: Watch for new messages
When a directory is opened, start watching its "new" subdirectory for
incoming messages using the fsnotify library. When creation events are
detected, run the Unseen routine to move the message from new to cur and
add new UIDs to the store, updating the UI's list of directory contents
as we go.
2019-07-17 17:29:11 -04:00
Ben Burwell eb0481b9cc Fix header decoding
Email headers can be encoded with different charsets, which is signalled
using a special character sequence. The go-message package provides two
different methods for accessing header values, Get(key) (actually
inherited from the embedded textproto.Header) which returns the raw
header value and Text(key), which returns the header's value decoded as
UTF-8.

Before, in the maildir backend, we were using the Get method which
sometimes resulted in encoded headers being displayed in the UI. This
patch replaces the incorrect usage of Get() with Text().
2019-07-17 15:58:15 -04:00
Drew DeVault dba16157de Fix missing format fields in maildir/container.go 2019-07-13 11:21:17 -04:00
Ben Burwell d15ba24a2f Implement maildir copy
Create a delivery in the destination directory with the content of the
source message.
2019-07-12 11:26:39 -04:00
Ben Burwell 1b8b6e218c Add maildir backend worker
Add the initial implementation of a backend for Maildir accounts. Much
of the functionality required is implemented in the go-message and
go-maildir libraries, so we use them as much as possible.

The maildir worker hooks into a new maildir:// URL scheme in the
accounts.conf file which points to a container of several maildir
directories. From there, the OpenDirectory, FetchDirectoryContents, etc
messages work on subdirectories. This is implemented as a Container
struct which handles mapping between the symbolic email folder names and
UIDs to the concrete directories and file names.
2019-07-12 11:09:50 -04:00
Daniel Bridges 06da4512b9 #190 Handle gmail duplicate folder 2019-07-11 19:38:40 -04:00