aboutsummaryrefslogtreecommitdiffstats
path: root/worker/maildir/worker.go
Commit message (Collapse)AuthorAgeFilesLines
* fswatcher: add a darwin fswatcher implementationTim Culverhouse2023-03-071-0/+4
| | | | | | | | | | Add a darwin implementation of FSWatcher using the fsevents package. The implementation is behind a darwin build flag. Co-authored-by: Ben Cohen <ben@bencohen.net> Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Ben Lee-Cohen <ben@lee-cohen.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: use FSWatcher interfaceTim Culverhouse2023-03-071-13/+5
| | | | | | | | Use the FSWatcher interface to create the watcher. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Ben Lee-Cohen <ben@lee-cohen.com> Acked-by: Robin Jarry <robin@jarry.cc>
* lint: add missing panic handlers in goroutinesRobin Jarry2023-01-061-0/+4
| | | | | | | | These issues were all reported by the new custom analyzer introduced in previous commit. Signed-off-by: Robin Jarry <robin@jarry.cc> Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
* model: change flags array to bitmaskRobin Jarry2023-01-041-2/+2
| | | | | | | Using a list of integers is not optimal. Use a bit mask instead. Signed-off-by: Robin Jarry <robin@jarry.cc> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* maildir: send MessagesDeleted to UI when moving messagesTim Culverhouse2022-12-141-0/+4
| | | | | | | | | | | | | | The maildir worker sends a MessagesMoved message to the UI when messages are moved, enabling the destination directory to update it's counts. The filesystem watcher sees the move and updates the directory currently selected. However, an update of the UIDs in the msgstore is not completed as it is in other workers. All other works, via some mechanism (direct or EXPUNGE update) issue a MessagesDeleted message after a move. Send this message to the UI for the maildir worker to have it work as all other workers do. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
* logging: rename package to logRobin Jarry2022-12-021-38/+38
| | | | | | | | | | Use the same name than the builtin "log" package. That way, we do not risk logging in the wrong place. Suggested-by: Tim Culverhouse <tim@timculverhouse.com> Signed-off-by: Robin Jarry <robin@jarry.cc> Tested-by: Bence Ferdinandy <bence@ferdinandy.com> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* logging: homogenize levelsRobin Jarry2022-12-021-10/+11
| | | | | | | | | | | | | | | | | | The main goal is to ensure that by default, the log file (if configured) does not grow out of proportions. Most of the logging messages in aerc are actually for debugging and/or trace purposes. Define clear rules for logging levels. Enforce these rules everywhere. After this patch, here is what the log file looks like after starting up with a single account: INFO 2022/11/24 20:26:16.147164 aerc.go:176: Starting up version 0.13.0-100-g683981479c60 (go1.18.7 amd64 linux) INFO 2022/11/24 20:26:17.546448 account.go:254: [work] connected. Signed-off-by: Robin Jarry <robin@jarry.cc> Tested-by: Bence Ferdinandy <bence@ferdinandy.com> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* maildir: drain limit channel on parse error when sortingTim Culverhouse2022-11-091-0/+1
| | | | | | | | | | | | | | Commit 155f0c3f28a1 ("maildir/sort: get MessageInfos in parallel") added parallel parsing of maildir messages. The parsing is limitted to 2x the number of CPUs using a buffered channel. The channel is drained at the end of the parsing, however it is not drained if a parse error is encountered. With enough errors, this can lead to a deadlock. Drain the channel when an error occurs. Fixes: 155f0c3f28a1 ("maildir/sort: get MessageInfos in parallel") Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: always sort by at least uidTim Culverhouse2022-11-091-1/+7
| | | | | | | | | | | | | | | | | | | | Commit 29205fdd07c0 ("maildir/search: get MessageInfos in parallel") introduced parallel searching of maildir backends, which improves search and filtering performance. When no filter is specified, the search algorithm is still used with no criteria because the FilterCriteria field always contains "filter" as a minimum. With the new parallelized search, the message order becomes random. Only use the search algorithm if there is a filter specified. If a filter *is* specified and a sort order is *not*, the messages will go through the search algorithm and become randomized, but not sorted back in order. Always default to a UID order sort (the same as prior to this commit) when no sort order is specified. Fixes: 29205fdd07c0 ("maildir/search: get MessageInfos in parallel") Reported-by: Ben Lee-Cohen <bdc@fastmail.fm> Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Ben Lee-Cohen <bdc@fastmail.fm>
* maildir: enable backend threadsTim Culverhouse2022-11-091-1/+85
| | | | | | | | | | Add thread capability to maildir backend. Enables the maildir worker to thread the entire folder, as opposed to only the displayed messages via client side threads. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Inwit <inwit@sindominio.net> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: speedup move to larger foldersBence Ferdinandy2022-11-091-7/+4
| | | | | | | | | | When moving or archiving to a very large folder, aerc requests directory info, which can takes several seconds to receive. Also, when moving, we post a MessagesDeleted message. Remove unnecessary call to directory info and change posted message to the correct MessagesMoved. Signed-off-by: Bence Ferdinandy <bence@ferdinandy.com> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* maildir: keep less data in memory for sortingTim Culverhouse2022-11-061-1/+13
| | | | | | | | | | | | | | | | Sorting opens and reads portions of every file within a directory in order to gather the data needed. Specifically, RFC822Headers and BodyStructure are not needed. The RFC822Headers field stores a lot of information, and the BodyStructure field requires parsing the body of the email. Don't set these two values when parsing. Note: in my testing, this dropped sorting a 52k archive from 2.2gb of ram usage, to < 500mb Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir/sort: get MessageInfos in parallelTim Culverhouse2022-11-061-6/+22
| | | | | | | | Sorting in the maildir worker requires reading each file in the directory. Use waitgroups and goroutines to read files in parallel. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: set SkipSort when checking mailTim Culverhouse2022-11-061-1/+2
| | | | | | | | | | | | | The maildir worker's check-mail command did not include the SkipSort directive in the DirectoryInfo, which would cause a fresh fetching of directory contents on the selected maiboxes. This is unneeded and resulted in every directory being sent to the msgstore and requesting a sort. Set SkipSort when checking mail to prevent extra calls to sort. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: replace ListFolder method with FolderMapJulian Pidancet2022-10-271-5/+5
| | | | | | | | | | | Replace ListFolder with a new method that returns a map indexed by folder names instead of a list of folder names. A map is simpler to use and more efficient in case we only want to check the presence of a specific folder in the Maildir store. Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com> Acked-by: Robin Jarry <robin@jarry.cc> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* maildir: move common maildir code out of workerJulian Pidancet2022-10-271-11/+11
| | | | | | | | | This change moves code that could be common to both notmuch and maildir workers in worker/lib. Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com> Acked-by: Robin Jarry <robin@jarry.cc> Acked-by: Tim Culverhouse <tim@timculverhouse.com>
* maildir: make checkmail update directory countsBen Cohen2022-10-181-0/+14
| | | | | | | | Non-selected folders will now have their total/unread/new counts updated in the background when a check-mail happens. Signed-off-by: Ben Cohen <ben@bencohen.net> Acked-by: Robin Jarry <robin@jarry.cc>
* logging: substitute %w for %vKoni Marti2022-10-021-28/+28
| | | | | | | | | | | | | | | | Subsitute the format specifier %w for %v in the logging facility. The logging functions use a fmt.Sprintf call behind the scene which does not recognize %w. %w should be used in fmt.Errorf when you want to wrap errors. Hence, the log entries that use %w are improperly formatted like this: ERROR 2022/10/02 09:13:57.724529 worker.go:439: could not get message info %!w(*fmt.wrapError=&{could not get structure: [snip] }) ^ Links: https://go.dev/blog/go1.13-errors Signed-off-by: Koni Marti <koni.marti@gmail.com> Acked-by: Moritz Poldrack <moritz@poldrack.dev>
* backends: send MessageInfoError on header fetching errorTim Culverhouse2022-09-251-1/+1
| | | | | | | | | | | | | | | | | | | | | When an error is encountered fetching a header, the backends respond with a type.Error worker message. On receipt of this message, the UI deletes all pending headers. The headers are all requested again as they remain on the screen, resulting in an infinite request loop - and an infinite logging loop. The user only ever sees the spinner unless they check the logs. A previous commit intended to fix this, however it introduced a regression where any message that was part of the fetch request would also be marked as erroneous. This commit is reverted with commit 2aad2fea7d36 ("msgstore: revert 9fdc7acf5b48"). Send an erroneous message info message from the backend when an error is encountered for a specific UID. Fixes: 01f80721e283 ("msgstore: post MessageInfo on erroneous fetch") Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: do not send dircontents on fs eventTim Culverhouse2022-08-301-13/+0
| | | | | | | | | | | | | | The maildir worker watches the file system for events (new mail, moved mail, copied mail, etc). On an event, the worker sends a DirectoryContents message and a DirectoryInfo message. Do not send DirectoryContents on FS Event in the maildir worker. The DirectoryInfo message already triggers the UI to request the new DirectoryContents, and does so in a more predictable way (IE any currently applied filter is applied). Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: implement MoveMessages handlingTim Culverhouse2022-08-221-0/+16
| | | | | | | | | | | | | Implement MoveMessages in the maildir worker. go-maildir supports Move operations by default, and is functionally equivalent to a OS-level rename. Creation date of the file is preserved by using Move, which is used by at least one maildir-to-IMAP synchronizer (isync/mbsync). The previous move method of copy-and-delete would reset the creation date of the message, and potentially cause sorting issues in other email clients. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* go: removed io/ioutilMoritz Poldrack2022-08-221-2/+1
| | | | | | | | | | Since the minimum required version of Go has been bumped to 1.16, the deprecation of io/ioutil can now be acted upon. This Commit removes the remaining dependencies on ioutil and replaces them with their io or os counterparts. Signed-off-by: Moritz Poldrack <git@moritz.sh> Acked-by: Robin Jarry <robin@jarry.cc>
* lint: work nicely with wrapped errors (errorlint)Moritz Poldrack2022-08-041-40/+40
| | | | | | | | Error wrapping as introduced in Go 1.13 adds some additional logic to use for comparing errors and adding information to it. Signed-off-by: Moritz Poldrack <moritz@poldrack.dev> Acked-by: Robin Jarry <robin@jarry.cc>
* lint: homogenize operations and minor fixes (gocritic)Moritz Poldrack2022-08-041-3/+5
| | | | | | | | | | | | | | | | | | Apply GoDoc comment policy (comments for humans should have a space after the //; machine-readable comments shouldn't) Use strings.ReplaceAll instead of strings.Replace when appropriate Remove if/else chains by replacing them with switches Use short assignment/increment notation Replace single case switches with if statements Combine else and if when appropriate Signed-off-by: Moritz Poldrack <moritz@poldrack.dev> Acked-by: Robin Jarry <robin@jarry.cc>
* lint: apply new formatting rulesMoritz Poldrack2022-08-011-4/+6
| | | | | | | Run `make fmt`. Signed-off-by: Moritz Poldrack <git@moritz.sh> Acked-by: Robin Jarry <robin@jarry.cc>
* checkmail: error if check-mail-cmd is not setTim Culverhouse2022-08-011-0/+4
| | | | | | | Send error message to UI if check-mail-cmd is required but not set. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* logging: use level-based logger functionsRobin Jarry2022-07-231-40/+39
| | | | | | | | | | | | Do not pass logger objects around anymore. Shuffle some messages to make them consistent with the new logging API. Avoid using %v when a more specific verb exists for the argument types. The loggers are completely disabled (i.e. Sprintf is not even called) by default. They are only enabled when redirecting stdout to a file. Signed-off-by: Robin Jarry <robin@jarry.cc> Acked-by: Moritz Poldrack <moritz@poldrack.dev>
* account: import mbox file to a folderKoni Marti2022-07-141-1/+3
| | | | | | | | | | | | | Append all messages from an mbox file to the selected folder with the import-mbox command. User confirmation is required when the folder already contains messages. A failed append will be retried a few times. If a backend timeout occurs, the entire import is stopped to prevent a hang. Signed-off-by: Koni Marti <koni.marti@gmail.com> Acked-by: Robin Jarry <robin@jarry.cc>
* account: export folder to mbox fileKoni Marti2022-07-141-0/+3
| | | | | | | | | Export all message in the current folder to an mbox file. If an error occurs during the export, aerc retries a few times before giving up to prevent a hang. Signed-off-by: Koni Marti <koni.marti@gmail.com> Acked-by: Robin Jarry <robin@jarry.cc>
* threading: enable filtering of server-side threadsTim Culverhouse2022-07-101-4/+19
| | | | | | | | | | | | | | This patch enables the filtering of a threaded view which uses server-built threads. Filtering is done server-side, in order to preserve the use of server-built threads. In adding this feature, the filtering of notmuch folders was brought up to feature parity with the other workers. The filters function the same (ie: they can be stacked). The notmuch filters, however, still use notmuch syntax for the filtering. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* threading: add backend capabilities to workersTim Culverhouse2022-07-101-0/+5
| | | | | | | | | | | This patch provides a method to report backend capabilities to the UI. The intial capabilities included in the report are Sort and Thread. Having these available to the UI enables the client to better handle server side threading. Signed-off-by: Koni Marti <koni.marti@gmail.com> Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* worker/maildir: implement Maildir++ supportAdnan Maolood2022-07-101-1/+12
| | | | | | | See https://www.courier-mta.org/maildir.html#maildircontents Signed-off-by: Adnan Maolood <me@adnano.co> Acked-by: Koni Marti <koni.marti@gmail.com>
* maildir: fix dirinfo.Unseen and Exists countingTim Culverhouse2022-06-141-2/+0
| | | | | | | | The maildir worker was adding Recent messages to the counts of Unseen and Exists, however these messages were already included in those counts. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* dirlist: update RUE counts for imap/maildir on move|copy|delete|archiveTim Culverhouse2022-06-071-13/+4
| | | | | | | | | | | | | | | | | | | When moving/copying/deleting/archiving a message in imap, the RUE counts displayed in the dirlist would not update properly. Maildir has (had) an implementation that recounts the entire directory and updates the DirectoryInfo after one of these actions. This patch implements a more efficient method of updating, and also enables it to apply to IMAP without any additional requests. Upon completion of the action, the counts are manually updated with the count of messages in the action and recent and/or unseen states of those messages. This is more efficient for maildir, because we aren't counting everything in the store. For IMAP, we get the updates for free because we are only performing the update after confirmation from the server that the action has happened. Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Acked-by: Robin Jarry <robin@jarry.cc>
* feat: add background mail polling option for all workersTim Culverhouse2022-05-311-11/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | Check for new mail (recent, unseen, exists counts) with an external command, or for imap with the STATUS command, at start or on reconnection and every X time duration IMAP: The selected folder is skipped, per specification. Additional config options are included for including/excluding folders explicitly. Maildir/Notmuch: An external command will be run in the background to check for new mail. An optional timeout can be used with maildir/notmuch. Default is 10s New account options: check-mail check-mail-cmd (maildir/notmuch only) check-mail-timeout (maildir/notmuch only), default 10s check-mail-include (IMAP only) check-mail-exclude (IMAP only) If unset, or set less than or equal to 0, check-mail will be ignored Signed-off-by: Tim Culverhouse <tim@timculverhouse.com> Tested-by: Moritz Poldrack <moritz@poldrack.dev> Acked-by: Robin Jarry <robin@jarry.cc>
* go vet: composite literal uses unkeyed fieldsMoritz Poldrack2022-03-181-1/+1
| | | | | | | | This commit fixes all occurrences of the abovementioned lint-error in the codebase. Signed-off-by: Moritz Poldrack <git@moritz.sh> Acked-by: Robin Jarry <robin@jarry.cc>
* maildir: fix data race in maildir workerwagner riffel2022-03-071-25/+82
| | | | | | | | | | | | | Fix a data race due dirInfo pointer being read in the main goroutine by NewMessageStore and written in the anonymous goroutine launched in Worker.getDirectoryInfo. To address the issue raised in https://todo.sr.ht/~rjarry/aerc/16, we use readdir(3) once, parse and cache its results, this replaces go-maildir library Dir.Flags based stat(3) and filepath.Glob causing the issue when N (emails) is large. Signed-off-by: wagner riffel <w@104d.net>
* maildir: defer the count of recent messagesRobin Jarry2022-01-301-25/+32
| | | | | | | | | | | | | | | Since commit 01c96e78dfe8 ("Update DirectoryInfo handling for maildir"), flags are checked for every message of a folder when entering it. Iterating over all messages of a folder takes a long time for large collections of emails. Only count the number of messages and state that the directory info counts are not accurate. Defer the parsing of message flags in a goroutine to have a more responsive UI. Fixes: https://todo.sr.ht/~rjarry/aerc/16 Suggested-by: Koni Marti <koni.marti@gmail.com> Signed-off-by: Robin Jarry <robin@jarry.cc>
* maildir,notmuch: pass in-memory message to callbackNguyễn Gia Phong2022-01-201-1/+7
| | | | | | | This fixes piped full message (:pipe -m) being empty. Fixes: 904ffacb0e52 ("maildir,notmuch: avoid leaking open files") Signed-off-by: Nguyễn Gia Phong <mcsinyx@disroot.org>
* maildir,notmuch: avoid leaking open filesNguyễn Gia Phong2022-01-191-0/+1
| | | | | | | | | | Previously, Message.NewReader returned the wrapped buffered reader without a reference to the opened file, so the files descriptors were left unclosed after reading. Now, the file reader is returned directly and closed on the call site. Buffering is not needed here because it is an implementation detail of go-message. Fixes: https://todo.sr.ht/~rjarry/aerc/9
* maildir: watch for message renamesRobin Jarry2021-12-061-2/+5
| | | | | | | | | | Messages flags can also be changed without changing directories. Changing flags in maildirs means renaming the message files. Also take renames into account. Link: https://cr.yp.to/proto/maildir.html Fixes: f4d3c8fc77f9 ("maildir: watch for external changes") Signed-off-by: Robin Jarry <robin@jarry.cc>
* maildir: watch for external changesRobin Jarry2021-12-061-4/+12
| | | | | | | | | | | | | | When a maildir is synchronized by an external process while aerc is running (e.g. mbsync), some emails may be moved out of "new" to "cur" or completely deleted. These deletions are ignored and aerc may assume these messages are still here, leading to errors. Take file deletions into account. Also, add "cur" to the watched folders since these can be modified as well. Signed-off-by: Robin Jarry <robin@jarry.cc>
* go.mod: change base git urlRobin Jarry2021-11-051-4/+4
| | | | | | | I'm not sure what are the implications but it seems required. Link: https://github.com/golang/go/issues/20883 Signed-off-by: Robin Jarry <robin@jarry.cc>
* maildir: track the recent flag correctlyReto Brunner2021-11-011-22/+24
| | | | | | | | In the maildir worker we manually need to track the Recent flag in order for the notification command etc to work. Push that responsibility to the container, we must make sure to manually add the flag though if one grabs the message info.
* notmuch/maildir: remove double emit of the dirinfoReto Brunner2021-04-281-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There was some bug which could be worked around by double emitting an event. However that proofed to be brittle: We send the first message here from the worker goroutine: https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g= o#L306 Then Tick() is waked in the main goroutine and calls ProcessMessage: https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/widgets/account.go#L100 ProcessMessage in the main goroutine reads types.Message state with msg.getId() and msg.InResponseTo(): https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#= L74-76 Meanwhile in the worker goroutine we call PostMessage for a second time with a pointer that points to the *same* previous message that ProcessMessage is reading: https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/maildir/worker.g= o#L306 The second PostMessage call makes writes to message while ProcessMessage in the main goroutine is possibly reading: https://git.sr.ht/~sircmpwn/aerc/tree/a5553438/item/worker/types/worker.go#= L59 This led to a data race in the event loop Reported-By: Wagner Riffel <w@104d.net>
* Add support for :rmdirARaspiK2020-08-191-0/+12
| | | | | | | | | | | | | | | | | | | | | The `:rmdir` command removes the current directory (`-f` is required if the directory is not empty). This is not supported on the notmuch backend. An issue with the maildir backend is that some sync programs (e.g. offlineimap) may recover the directory after it is deleted. They need to specifically be configured to accept deletions, or special commands need to be executed (e.g. `offlineimap --delete-folder`) to properly delete folders. A danger of using this on the IMAP backend is that it is possible for a new message to be added to the directory and for aerc to not show it immediately (due to a slow connection) - using `:rmdir` at this moment (with `-f` if the directory already contains messages) would delete the directory and the new message that just arrived (and all other contents). This is documented in aerc(1) so that users are aware of possible risks.
* maildir: Provide nicer error message on invalid urlTero Koskinen2020-07-271-1/+16
| | | | | | | | If accounts.conf contains an invalid maildir url, return a nice error instead of panicking. Log a couple of different error cases to provide extra information about the error to the user.
* Only send directory info once from maildirAndrew Jeffery2020-07-191-4/+5
| | | | | The directory info only needs to be sent once for all the messages, this reduces unnecessary messages being sent which could lock up the ui.
* Add additional flagging functionalityARaspiK2020-07-081-5/+6
| | | | | | | | | | | | | | More mail flags can now be set, unset, and toggled, not just the read/seen flag. This functionality is implemented with a new `:flag` and `:unflag` command, which are extensions to the matching `:read` and `:unread` commands, adding support for different flags. In fact, the `read`/`unread` commands are now recognized aliases to `flag`/`unflag`. The new commands are also well documented in aerc(1). The change mostly extends the previous read/unread setting functionality by adding a selection for the flag to change.
* Set AnsweredFlag on successful replySrivathsan Murali2020-05-251-0/+35
|
generated by cgit (git 2.34.1) at 2024-11-14 22:05:13 +0000