| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
| |
This has nothing to do at the root of the source tree.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Bence Ferdinandy <bence@ferdinandy.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
When go-maildir parses a folder and finds an error, it will still return
the valid keys it has found along with the error. Instead of returning
an empty list of UIDs log an error and proceed with the valid uids found
by go-maildir. When opening a folder with invalid files, show an error
message to the user.
References: https://todo.sr.ht/~rjarry/aerc/215
Signed-off-by: Bence Ferdinandy <bence@ferdinandy.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
For the maildir and notmuch backends, add the full path of the message
as Filename to templates. In case of a notmuch message having multiple
files associated with it, it returns a random path. Also add Filenames
to templates, which is a list of all associated message paths. This is
relevant for the notmuch backend, where a single messages is shown, if
there are multiple copies of it.
Changelog-added: Add filepath to messages in templates as .Filename(s)
Signed-off-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
| |
Add recipients fields (from, to, cc) to the header for filtering and
searching for the maildir and mbox backends.
Fixes: commit 8464b373 ("search: use a common api for all workers")
Reported-by: Vitaly Ovchinnikov <v@ovch.ru>
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Tested-by: Vitaly Ovchinnikov <v@ovch.ru>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Define a SearchCriteria structure. Update the FetchDirectoryContents,
FetchDirectoryThreaded and SearchDirectory worker messages to include
this SearchCriteria structure instead of a []string slice.
Parse the search arguments in a single place into a SearchCriteria
structure and use it to search/filter via the message store.
Update all workers to use that new API. Clarify the man page indicating
that notmuch supports searching with aerc's syntax and also with notmuch
specific syntax.
getopt is no longer needed, remove it from go.mod.
NB: to support more complex search filters in JMAP, we need to use an
email.Filter interface. Since GOB does not support encoding/decoding
interfaces, store the raw SearchCriteria and []SortCriterion values in
the cached FolderContents. Translate them to JMAP API objects when
sending an email.Query request to the server.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Tested-by: Inwit <inwit@sindominio.net>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Avoid importing code from worker/lib into lib. It should only be the
other way around. Move the message parsing code used by maildir,
notmuch, mbox and the eml viewer into a lib/rfc822 package.
Adapt imports accordingly.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Moritz Poldrack <moritz@poldrack.dev>
Tested-by: Inwit <inwit@sindominio.net>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The LimitHeaders function is used to optionally reduce memory usage of
aerc by only keeping certain headers in memory for the message list. The
function properly deletes header keys and values from the underlying
object, however the underlying data structure has a map and a slice -
which do not get resized after deletion: resulting in no actual memory
savings.
Create a new header and add only the headers we want to it. Return this
value and use in the MessageInfo struct.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the logging of stdout/stderr of mail checking process in maildir
worker if it returns an error.
The user still sees just the exit status, but log gets a more detailed
information for further analysis.
Signed-off-by: Vitaly Ovchinnikov <v@postbox.nz>
Acked-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
| |
github.com/mitchellh/go-homedir has not received any update since 2019.
The last release of github.com/kyoh86/xdg was in 2020 and it has been
marked as deprecated by its author.
Replace these with internal functions.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Reviewed-by: Moritz Poldrack <moritz@poldrack.dev>
|
|
|
|
|
|
|
| |
No functional change. This will allow reuse in other parts of aerc.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Koni Marti <koni.marti@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add the folder-map functionality to the maildir backend. If the
folder-map config option is specified, the folder-map worker middleware
is used.
Unroll the worker.PostMessageInfoError function for a streamlined
WorkerInteractor interface.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add a WorkerInteractor interface. Avoid exposing any public fields in
the types.Worker.
This will set the stage to implement a middleware pattern for the
workers, i.e. to map folder names between the ui and the backend.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
| |
Implement cancellation of cancellable requests. These include listing of
directory contents, searching, and sorting.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
Use the worker's logging functions to have all maildir related messages
prefixed by the account name.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Inwit <inwit@sindominio.net>
|
|
|
|
|
|
|
|
|
|
| |
Fix path separator for the maildir++ backend. Maildir++ already
substitutes '.' for os.PathSeparator. Returning '.' will thus break the
directory tree and the other logic for this backend.
Fixes: 2040fc18 ("imap: use delimiter from server")
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Enable partial header fetching by creating config values for headers to
specifically include, or specifically exclude. The References field will
always be fetched, regardless of the include list. Envelope data is
always fetched, but is not shown with :toggle-headers, since it isn't in
the RFC822 struct unless explicitly included in the list.
Partial headers can break the cache on changes. Update the cache tag key
to include the state of the partially-fetched headers.
Partial header fetching can have a significant performance increase for
IMAP, and for all backends a resource improvement. Some data to support
this is below. Gathered by opening aerc, selecting a mailbox with
approximately 800 messages and scrolling to the end.
Received measured with nethogs, RAM from btop
Received | RAM
-------------------------------------
All Headers | 9,656 kb | 103 MB
Minimum Headers | 896 kb | 36 MB
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
| |
To accommodate servers that use a delimiter other than "/" ("." being a
common alternative), the delimiter is fetched from the server when
connecting.
Signed-off-by: Moritz Poldrack <git@moritz.sh>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
| |
Retrieve message size not only in the MessageInfo but also in the
MessageHeaders function. The MessageHeaders function is used for an
memory-optimized maildir sorting. This fixes sorting by size in the
maildir backend.
Fixes: f04d83e8 ("messageinfo: report message sizes")
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit ef4504e6baf5 ("maildir: fix handling of FSEvents") ambitiously
tried to fix handling of file system events by handling different events
in "proper" ways. Distributions and OSes report these FSEvents
differently which creates a large amount of edge cases on what the right
handling of each individual event should be.
Revert part of ef4504e6baf5 which attempts to issue different messages
based on the event. Add a debounce to file system events and always
trigger a Refetch of the message list. This still fixes one of the
"fixes" the referenced patch attempted at, where the UI was only told to
refetch if the message count increased (but if messages disappeared
externally, the maildir never updated).
Fixes: ef4504e6baf5 ("maildir: fix handling of FSEvents")
Reported-by: Kirill Chibisov <contact@kchibisov.com>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Kirill Chibisov <contact@kchibisov.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Maildir uses a filesystem watcher to watch the currently selected
directory for changes, updating the UI as necessary. Not all changes get
updated based on the current logic. Send a MessageInfo if the event was
a Rename. Send a MessagesDeleted if the event was a Remove. Tell the UI
to refetch the list if the event was a Create.
Note: This leads to some unnecessary fetching in the UI, as renames also
come with a Create event in most cases. Koni suggested sending the
MessageInfo and having the UI perform a binary search and inserting the
message in the proper position. This is optimization is left out of this
series, with a TODO left in the code.
Fixes: https://todo.sr.ht/~rjarry/aerc/171
Fixes: 91ac21ac6155 ("msgstore: fetch message list based on OpenDirectory msg")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
| |
Report sizes of the message across all backends.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
| |
Handle headers in the search and filter commands, for searching and
filtering based on the Headers specified by the -H parameter, the syntax
for the -H parameter should be `Header: Key`.
Signed-off-by: Julian Marcos <jmjl@tilde.green>
Acked-by: Robin Jarry<robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Store the Directory RUE counts on the Directory data model. Use
DirectoryInfo messages to update the Directory model. Access Directories
via the dirlist instead of via the msgstore. Remove unused fields on
DirectoryInfo, all backends now give accurate counts.
Move refetch logic into dirlist
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry<robin@jarry.cc>
|
|
|
|
|
|
|
| |
Remove the unused field Attributes
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use the Backend interface to report Backend capabilities. Previously,
these were reported via a DirectoryInfo message, however they have
nothing to do with a directory and should be reported directly by the
backend. Add Capabilities method to Backend interface, satisfy this in
each backend, and use it on the UI side.
Remove Caps field from DirectoryInfo
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry<robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fetching the message list is done in a convoluted way. The UI receives a
DirectoryInfo message, which creates a message store. It then receives a
second DirectoryInfo (an oddity from the IMAP worker), and this
DirectoryInfo is passed to the message store which then requests a fetch
of the message list via store.Sort.
Use the OpenDirectory done response to tell the message store to fetch
messages. This makes the code easier to follow, and does not rely on
quirks from the IMAP worker.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
| |
Add a darwin implementation of FSWatcher using the fsevents package. The
implementation is behind a darwin build flag.
Co-authored-by: Ben Cohen <ben@bencohen.net>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Ben Lee-Cohen <ben@lee-cohen.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
Use the FSWatcher interface to create the watcher.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Ben Lee-Cohen <ben@lee-cohen.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
These issues were all reported by the new custom analyzer introduced in
previous commit.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
|
|
|
|
|
|
|
| |
Using a list of integers is not optimal. Use a bit mask instead.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The maildir worker sends a MessagesMoved message to the UI when messages
are moved, enabling the destination directory to update it's counts. The
filesystem watcher sees the move and updates the directory currently
selected. However, an update of the UIDs in the msgstore is not
completed as it is in other workers. All other works, via some mechanism
(direct or EXPUNGE update) issue a MessagesDeleted message after a move.
Send this message to the UI for the maildir worker to have it work as
all other workers do.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
|
|
|
|
|
|
|
| |
Filter and search with a date range in the maildir backend.
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
| |
Use the same name than the builtin "log" package. That way, we do not
risk logging in the wrong place.
Suggested-by: Tim Culverhouse <tim@timculverhouse.com>
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main goal is to ensure that by default, the log file (if configured)
does not grow out of proportions. Most of the logging messages in aerc
are actually for debugging and/or trace purposes.
Define clear rules for logging levels. Enforce these rules everywhere.
After this patch, here is what the log file looks like after starting up
with a single account:
INFO 2022/11/24 20:26:16.147164 aerc.go:176: Starting up version 0.13.0-100-g683981479c60 (go1.18.7 amd64 linux)
INFO 2022/11/24 20:26:17.546448 account.go:254: [work] connected.
Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit 155f0c3f28a1 ("maildir/sort: get MessageInfos in parallel") added
parallel parsing of maildir messages. The parsing is limitted to 2x the
number of CPUs using a buffered channel. The channel is drained at the
end of the parsing, however it is not drained if a parse error is
encountered. With enough errors, this can lead to a deadlock.
Drain the channel when an error occurs.
Fixes: 155f0c3f28a1 ("maildir/sort: get MessageInfos in parallel")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Commit 29205fdd07c0 ("maildir/search: get MessageInfos in parallel")
introduced parallel searching of maildir backends, which improves search
and filtering performance. When no filter is specified, the search
algorithm is still used with no criteria because the FilterCriteria
field always contains "filter" as a minimum. With the new parallelized
search, the message order becomes random. Only use the search algorithm
if there is a filter specified.
If a filter *is* specified and a sort order is *not*, the messages will
go through the search algorithm and become randomized, but not sorted
back in order. Always default to a UID order sort (the same as prior to
this commit) when no sort order is specified.
Fixes: 29205fdd07c0 ("maildir/search: get MessageInfos in parallel")
Reported-by: Ben Lee-Cohen <bdc@fastmail.fm>
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Ben Lee-Cohen <bdc@fastmail.fm>
|
|
|
|
|
|
|
|
|
|
| |
Add thread capability to maildir backend. Enables the maildir worker to
thread the entire folder, as opposed to only the displayed messages via
client side threads.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Tested-by: Inwit <inwit@sindominio.net>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
| |
When moving or archiving to a very large folder, aerc requests directory
info, which can takes several seconds to receive. Also, when moving, we
post a MessagesDeleted message. Remove unnecessary call to directory
info and change posted message to the correct MessagesMoved.
Signed-off-by: Bence Ferdinandy <bence@ferdinandy.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Sorting opens and reads portions of every file within a directory in
order to gather the data needed. Specifically, RFC822Headers and
BodyStructure are not needed. The RFC822Headers field stores a lot of
information, and the BodyStructure field requires parsing the body of
the email.
Don't set these two values when parsing.
Note: in my testing, this dropped sorting a 52k archive from 2.2gb of
ram usage, to < 500mb
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
Searching in the maildir worker requires reading each file in the
directory. Use waitgroups and goroutines to read files in parallel.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
Sorting in the maildir worker requires reading each file in the
directory. Use waitgroups and goroutines to read files in parallel.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The maildir worker's check-mail command did not include the SkipSort
directive in the DirectoryInfo, which would cause a fresh fetching of
directory contents on the selected maiboxes. This is unneeded and
resulted in every directory being sent to the msgstore and requesting a
sort.
Set SkipSort when checking mail to prevent extra calls to sort.
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
| |
Replace ListFolder with a new method that returns a map indexed by
folder names instead of a list of folder names. A map is simpler to use
and more efficient in case we only want to check the presence of a
specific folder in the Maildir store.
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
| |
This change moves code that could be common to both notmuch and maildir
workers in worker/lib.
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Acked-by: Robin Jarry <robin@jarry.cc>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previous change "maildir: hide invalid folders" prevents filepath.Walk
from descending into directories that don't contain new/, tmp/ or cur/
because the WalkDirFunc returns filepath.SkipDir.
We have to assume that these directories can be the parent of valid
maildir folders even if they're not themselves valid maildir folders, so
it is a mistake not to recurse into these subfolders. Fix it by
returning nil instead of filepath.SkipDir in the WalkDirFunc.
Fixes: 150aa0f498b9 ("maildir: hide invalid folders")
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
| |
Non-selected folders will now have their total/unread/new counts updated
in the background when a check-mail happens.
Signed-off-by: Ben Cohen <ben@bencohen.net>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The maildir worker currently populates the list of mail folders by
listing all the filesystem subdirectories in the maildir directory.
Although there's no official specification for maildir subfolders,
they should all have cur/ new/ and tmp/ subdirectories to be valid.
This patch prevents directories that don't have those subdirectories
present on the filesystem from appearing in the account folder list.
This is useful for example to prevent ".notmuch" and ".notmuch/xapian"
from showing up in the folder list if using notmuch to index emails
while using aerc's maildir backend.
Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
Acked-by: Tim Culverhouse <tim@timculverhouse.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Subsitute the format specifier %w for %v in the logging facility. The
logging functions use a fmt.Sprintf call behind the scene which does not
recognize %w. %w should be used in fmt.Errorf when you want to wrap
errors. Hence, the log entries that use %w are improperly formatted like
this:
ERROR 2022/10/02 09:13:57.724529 worker.go:439: could not get message
info %!w(*fmt.wrapError=&{could not get structure: [snip] })
^
Links: https://go.dev/blog/go1.13-errors
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When an error is encountered fetching a header, the backends respond
with a type.Error worker message. On receipt of this message, the UI
deletes all pending headers. The headers are all requested again as they
remain on the screen, resulting in an infinite request loop - and an
infinite logging loop. The user only ever sees the spinner unless they
check the logs.
A previous commit intended to fix this, however it introduced a
regression where any message that was part of the fetch request would
also be marked as erroneous. This commit is reverted with commit
2aad2fea7d36 ("msgstore: revert 9fdc7acf5b48").
Send an erroneous message info message from the backend when an error is
encountered for a specific UID.
Fixes: 01f80721e283 ("msgstore: post MessageInfo on erroneous fetch")
Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|
|
|
|
|
|
|
|
|
| |
Do not throw an error when the charset is unknown; the message entity
can still be read, but log the error instead.
Reported-by: falsifian
Signed-off-by: Koni Marti <koni.marti@gmail.com>
Acked-by: Robin Jarry <robin@jarry.cc>
|