command:serve_commands: new command for running a command server.

This is an initial step towards improving BE's efficiency.

Previously, BE gets slow as the bug count increases for several
commands (e.g. `be list`), because it takes time to load the bugdir
information from disk at each invocation.  If you use a remote repo
(`be --repo http://localhost:8000/ list`), the server process may have
already loaded the repo from disk, but now your listing process has to
fetch everything over the wire.  This is even worse than loading it
from disk.

With the new `be serve-commands` and `be --server URL ...` pair, the
bugdir loading happens once on the server, and all the processing is
also carried out on the server.  This means that calls like `be
--server http://localhost:8000/ list` will scale much better than
other methods.  For example:

  $ time be --server http://localhost:8000/ list > /dev/null

  real    0m2.234s
  user    0m0.548s
  sys     0m0.114s
  $ time be --server http://localhost:8000/ list > /dev/null

  real    0m0.730s
  user    0m0.548s
  sys     0m0.112s
  $ time be list > /dev/null

  real    0m2.453s
  user    0m2.289s
  sys     0m0.166s
  $ time be list > /dev/null

  real    0m2.521s
  user    0m2.350s
  sys     0m0.172s

The first call to a cold server takes about the same time as a local
call, because you need to load the bugs from the filesystem.  However,
later calls to a warm server are 3x faster, while later local calls
are still slow.

This is currently a minimal working implementation.  There's a good
deal of code in libbe.command.serve that I'd like to abstract out into
a libbe.util library (since there's still a bunch of duplication
between libbe.command.serve and libbe.command.serve_commands).  The
remote calls are also not as fast as I'd like, likely due to library
load times.  This commit just locks in an initial working
implementation.

1 files changed, 9 insertions, 2 deletions

diff --git a/libbe/ui/command_line.py b/libbe/ui/command_line.py
index 164d2c7..1cafc36 100644
--- a/libbe/ui/command_line.py
+++ b/libbe/ui/command_line.py
@@ -230,6 +230,12 @@ class BE (libbe.command.Command):
                     arg=libbe.command.Argument(
                         name='repo', metavar='REPO', default='.',
                         completion_callback=libbe.command.util.complete_path)),
+                libbe.command.Option(name='server', short_name='s',
+                    help='Select BE command server (see `be help '
+                         'command-server`) rather than executing commands '
+                         'locally',
+                    arg=libbe.command.Argument(
+                        name='server', metavar='URL')),
                 libbe.command.Option(name='paginate',
                     help='Pipe all output into less (or if set, $PAGER).'),
                 libbe.command.Option(name='no-pager',
@@ -355,10 +361,11 @@ def main():
         return 1
 
     ui.storage_callbacks = libbe.command.StorageCallbacks(options['repo'])
-    command = Class(ui=ui)
+    command = Class(ui=ui, server=options['server'])
     ui.setup_command(command)
 
-    if command.name in ['new', 'comment', 'commit', 'import-xml', 'serve']:
+    if command.name in [
+        'new', 'comment', 'commit', 'import-xml', 'serve', 'serve-commands']:
         paginate = 'never'
     else:
         paginate = 'auto'