headers: enable partial header fetching

Enable partial header fetching by creating config values for headers to
specifically include, or specifically exclude. The References field will
always be fetched, regardless of the include list. Envelope data is
always fetched, but is not shown with :toggle-headers, since it isn't in
the RFC822 struct unless explicitly included in the list.

Partial headers can break the cache on changes. Update the cache tag key
to include the state of the partially-fetched headers.

Partial header fetching can have a significant performance increase for
IMAP, and for all backends a resource improvement. Some data to support
this is below. Gathered by opening aerc, selecting a mailbox with
approximately 800 messages and scrolling to the end.

Received measured with nethogs, RAM from btop

		   Received  |   RAM
-------------------------------------
All Headers	|   9,656 kb | 103 MB
Minimum Headers	|     896 kb |  36 MB

Signed-off-by: Tim Culverhouse <tim@timculverhouse.com>
Acked-by: Moritz Poldrack <moritz@poldrack.dev>
Acked-by: Robin Jarry <robin@jarry.cc>

1 files changed, 24 insertions, 0 deletions

diff --git a/worker/lib/parse.go b/worker/lib/parse.go
index ac86292f..2baf13f7 100644
--- a/worker/lib/parse.go
+++ b/worker/lib/parse.go
@@ -310,6 +310,30 @@ func MessageInfo(raw RawMessage) (*models.MessageInfo, error) {
 	}, nil
 }
 
+// LimitHeaders returns a new Header with the specified headers included or
+// excluded
+func LimitHeaders(hdr *mail.Header, fields []string, exclude bool) {
+	fieldMap := make(map[string]struct{}, len(fields))
+	for _, f := range fields {
+		fieldMap[strings.ToLower(f)] = struct{}{}
+	}
+	curFields := hdr.Fields()
+	for curFields.Next() {
+		key := strings.ToLower(curFields.Key())
+		_, ok := fieldMap[key]
+		switch {
+		case exclude && ok:
+			curFields.Del()
+		case exclude && !ok:
+			// No-op: exclude but we didn't find it
+		case !exclude && ok:
+			// No-op: include and we found it
+		case !exclude && !ok:
+			curFields.Del()
+		}
+	}
+}
+
 // MessageHeaders populates a models.MessageInfo struct for the message.
 // based on the reader returned by NewReader. Minimal information is included.
 // There is no body structure or RFC822Headers set