ui: fix deadlocks in message channel

There are several ways the ui message channel can fill up leading to
deadlocks.

1) Invalidate() changes the value of uiState to DIRTY. The following
   call sequence:

      QueueRedraw()
      Invalidate()
      QueueRedraw()

   Leads to multiple nil messages being queued in the message channel
   whereas one could assume that the second QueueRedraw() would do
   nothing. This is caused by the tri-state nature of uiState.

2) We use the same channel to convey state change, keyboard events and
   redraw requests. Since a keyboard event almost always triggers
   a redraw, we end up trying to append a redraw message in the same
   goroutine that reads from the channel. This triggers a deadlock when
   there are more than 50 pending messages.

Solve the issue by using multiple channels, one per type of message that
needs to be sent to the main ui thread.

Remove QueueRedraw() and merge its functionality in Invalidate(). Only
use a DIRTY/CLEAN state to determine if something needs to be queued in
the redraw channel.

Use a channel for quitting instead of an atomic. Restructure some code
functions to have a cleaner API.

Use a for loop in the main thread and select from all channels.

Signed-off-by: Robin Jarry <robin@jarry.cc>
Tested-by: Koni Marti <koni.marti@gmail.com>
Tested-by: Maarten van Gompel <proycon@anaproy.nl>

1 files changed, 16 insertions, 17 deletions

diff --git a/main.go b/main.go
index a403c751..4705c438 100644
--- a/main.go
+++ b/main.go
@@ -12,7 +12,6 @@ import (
 	"time"
 
 	"git.sr.ht/~sircmpwn/getopt"
-	"github.com/gdamore/tcell/v2"
 	"github.com/mattn/go-isatty"
 	"github.com/xo/terminfo"
 
@@ -251,7 +250,6 @@ func main() {
 		setWindowTitle()
 	}
 
-	ui.ChannelEvents()
 	go func() {
 		defer log.PanicHandler()
 		err := hooks.RunHook(&hooks.AercStartup{Version: Version})
@@ -268,22 +266,23 @@ func main() {
 			log.Errorf("aerc-shutdown hook: %s", err)
 		}
 	}(time.Now())
-	for event := range libui.MsgChannel {
-		switch event := event.(type) {
-		case tcell.Event:
+loop:
+	for {
+		select {
+		case event := <-ui.Events:
 			ui.HandleEvent(event)
-		case *libui.AercFuncMsg:
-			event.Func()
-		case types.WorkerMessage:
-			aerc.HandleMessage(event)
-		}
-		if ui.ShouldExit() {
-			break
+		case msg := <-types.WorkerMessages:
+			aerc.HandleMessage(msg)
+		case callback := <-libui.Callbacks:
+			callback()
+		case <-libui.Redraw:
+			ui.Render()
+		case <-ui.Quit:
+			err = aerc.CloseBackends()
+			if err != nil {
+				log.Warnf("failed to close backends: %v", err)
+			}
+			break loop
 		}
-		ui.Render()
-	}
-	err = aerc.CloseBackends()
-	if err != nil {
-		log.Warnf("failed to close backends: %v", err)
 	}
 }