Merge assorted bugfixes and optimizations.

Highlights:
  * `be new` adds creator field like its supposed to (oops :p).
  * `be list --xml` uses <be-xml> format (was <bugs>).
  * `be import-xml` handles root comments appropriately.
  * `be` raises an appropriate help message.
  * `be help` works.
  * `be html` prints Comment.id.user() information.
  * better SavedSettingsObject._get_saved_settings() avoids data loss.
  * be-mbox-to-xml -> be-mail-to-xml and adds assorted format support.
  * be-xml-to-mbox and be-handle-mail work with new libbe layout.
  * BugDir.uuids() now caches on-disk uuids for speed.
  * Mercurial detection works for mercurial <= 1.1.2 _and_ >= 1.2
  * Fix bugs in VCS._children() relative/absolute path handling.

28 files changed, 462 insertions, 115 deletions

diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/body b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/body
new file mode 100644
index 0000000..6af098a
--- /dev/null
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/body
@@ -0,0 +1,19 @@
+On Wed, Jan 20, 2010 at 01:24:25PM -0500, W. Trevor King wrote:
+> Of course, incorperating interactive functionality in command output
+> (i.e. changing the bug target from the bug-show page), doesn't fit
+> into this model.  To do that, we'd have to abstract the default
+> command output the way we've already abstracted the commands and their
+> input...
+
+Does anyone know of any output-abstraction implementations to look at
+for inspiration.
+  * How would we handle the options we currently pass through
+    (shortlist, show_comments, etc.)?
+  * Would standard arguments know how to display themselves?
+    class Status (Argument):
+        def str(self, ui, command, *args, **kwargs):
+            ui.display_status(self, command, *args, **kwargs)
+    class Bug (Argument):
+        def str(self, ui, command, *args, **kwargs):
+            ui.display_bug(self, command, *args, **kwargs)
+    ...
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/values
new file mode 100644
index 0000000..378cc67
--- /dev/null
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/b8e5c376-32a4-42ea-b6b2-adbee069384a/values
@@ -0,0 +1,14 @@
+Alt-id: <20100120183646.GC14791@mjolnir>
+
+
+Author: '"W. Trevor King" <wking@drexel.edu>'
+
+
+Content-type: text/plain
+
+
+Date: Wed, 20 Jan 2010 18:36:46 +0000
+
+
+In-reply-to: <20100120182425.GB14791@mjolnir>
+
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/body b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/body
new file mode 100644
index 0000000..636137c
--- /dev/null
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/body
@@ -0,0 +1,76 @@
+On Wed, Jan 20, 2010 at 09:34:44AM -0500, W. Trevor King wrote:
+> On Sun, Dec 06, 2009 at 04:47:23AM -0500, W. Trevor King wrote:
+> > Steve, I've caught my CFBE branch up to my current pre-trunk BE and
+> > added dependency links to the bug page, so you should be all set once
+> > you get back to CFBE.
+> 
+> And I haven't pulled it up to date with my recent reorganization.  As
+> far as release tarballs go though, we don't have to port to Bazaar at
+> all, we can stuff a recent CFBE snapshot into the BE tarball.  How
+> do people feel about that?
+
+Ok, I've got CFBE working with my BE head:
+  http://www.physics.drexel.edu/~wking/code/hg/cfbe/
+However, I haven't reworked CFBE to take advantage of the new command
+structure.
+
+We'll need to extend libbe.command.base.Argument a bit as we work this
+out, but I expect we can auto-generate handlers for various commands
+with something along the lines of:
+
+<snip web.py>
+
+class CommandHandler (object):
+    def __init__(self, command):
+        self.command = command
+    def __call__(self, *args, **kwargs):
+        if GET:
+            template = self.env.get_template('command.html')
+            return template.render(command=self.command)
+	else:
+            try:
+                ret = libbe.ui.command_line.dispatch(
+                    self.command.ui, self.command, *args, **kwargs)
+            except libbe.command.UserError, e:
+                HANDLE ERROR
+            stdout = self.command.ui.get_stdout()
+            DISPLAY STDOUT OR REDIRECT...
+
+class WebInterface (libbe.command.UserInterface):
+    ...
+    def add_commands(self):
+        for command_name in libbe.command.commands():
+            Class = libbe.command.get_command_class(
+                command_name=command_name)
+            command = Class(ui=self)
+            self.command_name = cherrypy.expose(
+                CommandHandler(command))
+
+</snip web.py>
+
+<snip command.html>
+
+<form id="command-form" action="/command" method="post">
+    <fieldset>
+        {% for option in command.options %}
+	  {{ option_form_html(option) }}
+        {% endfor %}
+        {% for argument in command.args %}
+	  {{ argument_form_html(argument) }}
+        {% endfor %}
+    </fieldset>
+</form>
+
+{{ command.help() }}
+
+</snip command.html>
+
+Of course, incorperating interactive functionality in command output
+(i.e. changing the bug target from the bug-show page), doesn't fit
+into this model.  To do that, we'd have to abstract the default
+command output the way we've already abstracted the commands and their
+input...  This sounds like a lot of work, and it is, but the goal is
+that BE adds functionality (new commands, option, etc.), and CFBE,
+be-handle-mail, etc. automatically incorperate the new stuff.
+
+Thoughts?
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/values
new file mode 100644
index 0000000..fb6ab4e
--- /dev/null
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/comments/f5139012-e20b-4d24-90a5-10d969ddd364/values
@@ -0,0 +1,14 @@
+Alt-id: <20100120182425.GB14791@mjolnir>
+
+
+Author: '"W. Trevor King" <wking@drexel.edu>'
+
+
+Content-type: text/plain
+
+
+Date: Wed, 20 Jan 2010 18:24:25 +0000
+
+
+In-reply-to: <20100120143444.GA14451@mjolnir>
+
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/values
new file mode 100644
index 0000000..d1b7cbe
--- /dev/null
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/01c9a900-61f9-41f7-9b2f-dd8f89e25b1b/values
@@ -0,0 +1,14 @@
+creator: W. Trevor King <wking@drexel.edu>
+
+
+severity: minor
+
+
+status: open
+
+
+summary: Need command output abstraction for flexible UIs
+
+
+time: Wed, 20 Jan 2010 20:35:12 +0000
+
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/3438b72c-6244-4f1d-8722-8c8d41484e35/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/3438b72c-6244-4f1d-8722-8c8d41484e35/values
index 4d7cd5c..5c72e5f 100644
--- a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/3438b72c-6244-4f1d-8722-8c8d41484e35/values
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/3438b72c-6244-4f1d-8722-8c8d41484e35/values
@@ -1,3 +1,6 @@
+creator: W. Trevor King <wking@drexel.edu>
+
+
 severity: minor
 
 
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/814e39c0-68ee-4165-9166-19e2aee9c07d/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/814e39c0-68ee-4165-9166-19e2aee9c07d/values
index 9c9a294..5feb832 100644
--- a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/814e39c0-68ee-4165-9166-19e2aee9c07d/values
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/814e39c0-68ee-4165-9166-19e2aee9c07d/values
@@ -1,3 +1,6 @@
+creator: W. Trevor King <wking@drexel.edu>
+
+
 severity: minor
 
 
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/ed5eac05-80ed-411d-88a4-d2261b879713/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/ed5eac05-80ed-411d-88a4-d2261b879713/values
index 4a103e6..7b2813d 100644
--- a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/ed5eac05-80ed-411d-88a4-d2261b879713/values
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/ed5eac05-80ed-411d-88a4-d2261b879713/values
@@ -1,3 +1,6 @@
+creator: W. Trevor King <wking@drexel.edu>
+
+
 severity: minor
 
 
diff --git a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/f51dc5a7-37b7-4ce1-859a-b7cb58be6494/values b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/f51dc5a7-37b7-4ce1-859a-b7cb58be6494/values
index 3e4747c..bb6b222 100644
--- a/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/f51dc5a7-37b7-4ce1-859a-b7cb58be6494/values
+++ b/.be/bea86499-824e-4e77-b085-2d581fa9ccab/bugs/f51dc5a7-37b7-4ce1-859a-b7cb58be6494/values
@@ -1,3 +1,6 @@
+creator: Aaron Bentley
+
+
 extra_strings:
 - BLOCKS:47c8fd5f-1f5a-4048-bef7-bb4c9a37c411
 
diff --git a/NEWS b/NEWS
index 99d5bca..17d5aee 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,7 @@
+January 20, 2010
+ * Renamed 'be-mbox-to-xml' -> 'be-mail-to-xml' and added support for
+   several mailbox formats.
+
 January 3, 2010
  * Changed `be list --uuids` -> `be list --ids`
    Instead of UUIDs, it now outputs user ids: BUGDIR/BUG
diff --git a/interfaces/email/interactive/README b/interfaces/email/interactive/README
index 2070973..b25054c 100644
--- a/interfaces/email/interactive/README
+++ b/interfaces/email/interactive/README
@@ -28,10 +28,8 @@ first place.  There are four parsing styles:
     creating bugs         [be-bug:submit] new bug summary
     commenting on bugs    [be-bug:<bug-id>] commit message
     control               [be-bug] commit message
-    xml                   [be-bug:xml] commit message
 These are analogous to submit@bugs.debian.org, nnn@bugs.debian.org,
-and control@bugs.debian.org respectively.  The xml style has no Debian
-analog.
+and control@bugs.debian.org respectively.
 
 Creating bugs
 =============
@@ -108,15 +106,6 @@ shlex.split().
     --
     Goofy tagline ignored.
 
-XML
-===
-
-This interface allows users without access to the versioned source of
-the program to conveniently submit bugs and comments using be.  You
-should not attempt to compose emails for this interface by hand.  See
-the documentation for the `email-bugs' and `import-xml' be commands
-for details.
-
 Example emails
 ==============
 
diff --git a/interfaces/email/interactive/be-handle-mail b/interfaces/email/interactive/be-handle-mail
index 14ae6ac..c8343fc 100755
--- a/interfaces/email/interactive/be-handle-mail
+++ b/interfaces/email/interactive/be-handle-mail
@@ -374,8 +374,8 @@ class Message (object):
     def _subject_tag_type(self):
         """
         Parse subject tag, return (type, value), where type is one of
-        None, "new", "comment", "control", or "xml"; and value is None
-        except in the case of "comment", in which case it's the bug
+        None, "new", "comment", or "control"; and value is None except
+        in the case of "comment", in which case it's the bug
         ID/shortname.
         """
         tag,subject = self._split_subject()
@@ -619,7 +619,7 @@ class Message (object):
         bd = UI.storage_callbacks.get_bugdir()
         writeable = bd.storage.writeable
         bd.storage.writeable = False
-        if bd.vcs.versioned == False: # no way to tell what's changed
+        if bd.storage.versioned == False: # no way to tell what's changed
             bd.storage.writeable = writeable
             raise NotificationFailed('Not versioned')
 
@@ -656,7 +656,7 @@ class Message (object):
             commit_msg = self.commit_command.stdout
             assert commit_msg.startswith('Committed '), commit_msg
             after_revision = commit_msg[len('Committed '):]
-            before_revision = bd.vcs.revision_id(-2)
+            before_revision = bd.storage.revision_id(-2)
         else:
             before_revision = previous_revision
         if before_revision == None:
@@ -664,12 +664,12 @@ class Message (object):
             before_bd = libbe.bugdir.BugDir(from_disk=False,
                                             manipulate_encodings=False)
         else:
-            before_bd = bd.duplicate_bugdir(before_revision)
+            before_bd = libbe.bugdir.RevisionedBugDir(bd, before_revision)
         #after_bd = bd.duplicate_bugdir(after_revision)
         after_bd = bd # assume no changes since commit a few cycles ago
         return (before_bd, after_bd)
     def _subscriber_header(self, bd, previous_revision=None):
-        root_dir = os.path.basename(bd.root)
+        root_dir = os.path.basename(bd.storage.repo)
         if previous_revision == None:
             subject = 'Changes to %s on %s by %s' \
                 % (root_dir, THIS_SERVER, self.author_addr())
@@ -693,7 +693,7 @@ def generate_global_tags(tag_base=u'be-bug'):
     SUBJECT_TAG_START = u'[%s' % tag_base
     SUBJECT_TAG_RESPONSE = u'[%s]' % tag_base
     SUBJECT_TAG_NEW = u'[%s:submit]' % tag_base
-    SUBJECT_TAG_COMMENT = re.compile(u'\[%s:([\-0-9a-z]*)]' % tag_base)
+    SUBJECT_TAG_COMMENT = re.compile(u'\[%s:([\-0-9a-z/]*)]' % tag_base)
     SUBJECT_TAG_CONTROL = SUBJECT_TAG_RESPONSE
 
 def open_logfile(logpath=None):
@@ -718,15 +718,12 @@ def open_logfile(logpath=None):
             LOGPATH = os.path.join(_THIS_DIR, logpath)
     if LOGFILE == None and LOGPATH != u'none':
         LOGFILE = codecs.open(LOGPATH, u'a+',
-            libbe.utuil.encoding.get_filesystem_encoding())
+            libbe.util.encoding.get_filesystem_encoding())
 
 def close_logfile():
     if LOGFILE != None and LOGPATH not in [u'stderr', u'none']:
         LOGFILE.close()
 
-unitsuite = unittest.TestLoader().loadTestsFromModule(sys.modules[__name__])
-suite = unittest.TestSuite([unitsuite, doctest.DocTestSuite()])
-
 def test():
     result = unittest.TextTestRunner(verbosity=2).run(suite)
     num_errors = len(result.errors)
@@ -798,7 +795,6 @@ def main(args):
                         print send_pgp_mime.flatten(msg, to_unicode=True)
                     else:
                         send_pgp_mime.mail(msg, send_pgp_mime.sendmail)
-            self.commit_command.cleanup()
             close_logfile()
         UI.cleanup()
         sys.exit(0)
@@ -819,6 +815,7 @@ def main(args):
             LOGFILE.write(u'Uncaught exception:\n%s\n' % (e,))
             traceback.print_tb(sys.exc_traceback, file=LOGFILE)
             close_logfile()
+        m.commit_command.cleanup()
         UI.cleanup()
         sys.exit(1)
     else:
@@ -850,6 +847,7 @@ def main(args):
                     send_pgp_mime.mail(msg, send_pgp_mime.sendmail)
 
     close_logfile()
+    m.commit_command.cleanup()
     UI.cleanup()
 
 class GenerateGlobalTagsTestCase (unittest.TestCase):
@@ -900,9 +898,12 @@ class GenerateGlobalTagsTestCase (unittest.TestCase):
     def test_subject_tag_comment(self):
         "Should set SUBJECT_TAG_COMMENT global correctly"
         generate_global_tags(u'projectX-bug')
-        m = SUBJECT_TAG_COMMENT.match('[projectX-bug:xyz-123]')
+        m = SUBJECT_TAG_COMMENT.match('[projectX-bug:abc/xyz-123]')
         self.failUnlessEqual(len(m.groups()), 1)
-        self.failUnlessEqual(m.group(1), u'xyz-123')
+        self.failUnlessEqual(m.group(1), u'abc/xyz-123')
+
+unitsuite = unittest.TestLoader().loadTestsFromModule(sys.modules[__name__])
+suite = unittest.TestSuite([unitsuite, doctest.DocTestSuite()])
 
 if __name__ == "__main__":
     main(sys.argv)
diff --git a/libbe/bugdir.py b/libbe/bugdir.py
index 5967a7e..8389716 100644
--- a/libbe/bugdir.py
+++ b/libbe/bugdir.py
@@ -234,23 +234,22 @@ class BugDir (list, settings_object.SavedSettingsObject):
 
     # methods for managing bugs
 
-    def uuids(self):
-        uuids = []
-        # list the uuids in memory
-        for bug in self:
-            uuids.append(bug.uuid)
-            yield bug.uuid
-        if self.storage != None and self.storage.is_readable():
-            # and the ones that are still just in storage
-            child_uuids = libbe.util.id.child_uuids(
-                self.storage.children(self.id.storage()))
-            for id in child_uuids:
-                if id not in uuids:
-                    yield id
+    def uuids(self, use_cached_disk_uuids=True):
+        if use_cached_disk_uuids==False or not hasattr(self, '_uuids_cache'):
+            self._uuids_cache = []
+            # list bugs that are in storage
+            if self.storage != None and self.storage.is_readable():
+                child_uuids = libbe.util.id.child_uuids(
+                    self.storage.children(self.id.storage()))
+                for id in child_uuids:
+                    self._uuids_cache.append(id)
+        return list(set([bug.uuid for bug in self] + self._uuids_cache))
 
     def _clear_bugs(self):
         while len(self) > 0:
             self.pop()
+        if hasattr(self, '_uuids_cache'):
+            del(self._uuids_cache)
         self._bug_map_gen()
 
     def _load_bug(self, uuid):
@@ -264,9 +263,13 @@ class BugDir (list, settings_object.SavedSettingsObject):
                      from_storage=False)
         self.append(bg)
         self._bug_map_gen()
+        if hasattr(self, '_uuids_cache') and not bg.uuid in self._uuids_cache:
+            self._uuids_cache.append(bg.uuid)
         return bg
 
     def remove_bug(self, bug):
+        if hasattr(self, '_uuids_cache') and bug.uuid in self._uuids_cache:
+            self._uuids_cache.remove(bug.uuid)
         self.remove(bug)
         if self.storage != None and self.storage.is_writeable():
             bug.remove()
diff --git a/libbe/command/commit.py b/libbe/command/commit.py
index f4c275c..fd15630 100644
--- a/libbe/command/commit.py
+++ b/libbe/command/commit.py
@@ -38,7 +38,7 @@ class Commit (libbe.command.Command):
 
     >>> bd.extra_strings = ['hi there']
     >>> bd.flush_reload()
-    >>> ui.run(cmd, {'user-id':'Joe'}, ['Making a commit']) # doctest: +ELLIPSIS
+    >>> ui.run(cmd, args=['Making a commit']) # doctest: +ELLIPSIS
     Committed ...
     >>> ui.cleanup()
     >>> bd.cleanup()
diff --git a/libbe/command/help.py b/libbe/command/help.py
index 8e66405..1fc88f0 100644
--- a/libbe/command/help.py
+++ b/libbe/command/help.py
@@ -58,7 +58,7 @@ class Help (libbe.command.Command):
     def _run(self, **params):
         if params['topic'] == None:
             if hasattr(self.ui, 'help'):
-                self.ui.help()
+                print >> self.stdout, self.ui.help().rstrip('\n')
         elif params['topic'] in libbe.command.commands():
             module = libbe.command.get_command(params['topic'])
             Class = libbe.command.get_command_class(module,params['topic'])
diff --git a/libbe/command/html.py b/libbe/command/html.py
index d0985cc..0b4cf89 100644
--- a/libbe/command/html.py
+++ b/libbe/command/html.py
@@ -223,7 +223,7 @@ class HTMLGen (object):
                 comment_entries.append('<div class="comment root">')
             else:
                 comment_entries.append('<div class="comment">')
-            template_info = {}
+            template_info = {'shortname': comment.id.user()}
             for attr in ['uuid', 'author', 'date', 'body']:
                 value = getattr(comment, attr)
                 if attr == 'body':
@@ -611,7 +611,8 @@ class HTMLGen (object):
               <td class="bug_comment_label">Comment:</td>
               <td class="bug_comment">
             --------- Comment ---------<br/>
-            Name: %(uuid)s<br/>
+            ID: %(uuid)s<br/>
+            Short name: %(shortname)s<br/>
             From: %(author)s<br/>
             Date: %(date)s<br/>
             <br/>
diff --git a/libbe/command/import_xml.py b/libbe/command/import_xml.py
index 0656154..598ecb8 100644
--- a/libbe/command/import_xml.py
+++ b/libbe/command/import_xml.py
@@ -194,10 +194,19 @@ class Import_XML (libbe.command.Command):
 
         # protect against programmer error causing data loss:
         if croot_bug != None:
-            comms = [c.uuid for c in croot_comment.traverse()]
+            comms = []
+            for c in croot_comment.traverse():
+                comms.append(c.uuid)
+                if c.alt_id != None:
+                    comms.append(c.alt_id)
+            if croot_comment.uuid == libbe.comment.INVALID_UUID:
+                root_text = croot_bug.id.user()
+            else:
+                root_text = croot_comment.id.user()
             for new in root_comments:
-                assert new.uuid in comms, \
-                    "comment %s wasn't added to %s" % (new.uuid, croot_comment.uuid)
+                assert new.uuid in comms or new.alt_id in comms, \
+                    "comment %s (alt: %s) wasn't added to %s" \
+                    % (new.uuid, new.alt_id, root_text)
         for new in root_bugs:
             if not new in merged_bugs:
                 assert bugdir.has_bug(new.uuid), \
@@ -364,24 +373,39 @@ if libbe.TESTING == True:
               </bug>
             </be-xml>
             """
+            self.root_comment_xml = """
+            <be-xml>
+              <comment>
+                <uuid>c1</uuid>
+                <body>So long</body>
+              </comment>
+              <comment>
+                <uuid>c3</uuid>
+                <author>Jed</author>
+                <body>And thanks</body>
+              </comment>
+            </be-xml>
+            """
         def tearDown(self):
             self.bugdir.cleanup()
             self.ui.cleanup()
-        def _execute(self, params={}, args=[]):
-            self.ui.io.set_stdin(self.xml)
+        def _execute(self, xml, params={}, args=[]):
+            self.ui.io.set_stdin(xml)
             self.ui.run(self.cmd, params, args)
             self.bugdir.flush_reload()
         def testCleanBugdir(self):
             uuids = list(self.bugdir.uuids())
             self.failUnless(uuids == ['b'], uuids)
         def testNotAddOnly(self):
-            self._execute({}, ['-'])
+            bugB = self.bugdir.bug_from_uuid('b')
+            self._execute(self.xml, {}, ['-'])
             uuids = list(self.bugdir.uuids())
             self.failUnless(uuids == ['b'], uuids)
             bugB = self.bugdir.bug_from_uuid('b')
             self.failUnless(bugB.uuid == 'b', bugB.uuid)
             self.failUnless(bugB.creator == 'John', bugB.creator)
             self.failUnless(bugB.status == 'fixed', bugB.status)
+            self.failUnless(bugB.summary == 'a test bug', bugB.summary)
             estrs = ["don't forget your towel",
                      'helps with space travel',
                      'watch out for flying dolphins']
@@ -408,13 +432,87 @@ if libbe.TESTING == True:
             self.failUnless(c4.author == 'Jed', c4.author)
             self.failUnless(c4.body == 'And thanks\n', c4.body)
         def testAddOnly(self):
-            self._execute({'add-only':True}, ['-'])
+            bugB = self.bugdir.bug_from_uuid('b')
+            initial_bugB_summary = bugB.summary
+            self._execute(self.xml, {'add-only':True}, ['-'])
+            uuids = list(self.bugdir.uuids())
+            self.failUnless(uuids == ['b'], uuids)
+            bugB = self.bugdir.bug_from_uuid('b')
+            self.failUnless(bugB.uuid == 'b', bugB.uuid)
+            self.failUnless(bugB.creator == 'John', bugB.creator)
+            self.failUnless(bugB.status == 'open', bugB.status)
+            self.failUnless(bugB.summary == initial_bugB_summary, bugB.summary)
+            estrs = ["don't forget your towel",
+                     'helps with space travel']
+            self.failUnless(bugB.extra_strings == estrs, bugB.extra_strings)
+            comments = list(bugB.comments())
+            self.failUnless(len(comments) == 3,
+                            ['%s (%s)' % (c.uuid, c.alt_id) for c in comments])
+            c1 = bugB.comment_from_uuid('c1')
+            comments.remove(c1)
+            self.failUnless(c1.uuid == 'c1', c1.uuid)
+            self.failUnless(c1.alt_id == None, c1.alt_id)
+            self.failUnless(c1.author == 'Jane', c1.author)
+            self.failUnless(c1.body == 'Hello\n', c1.body)
+            c2 = bugB.comment_from_uuid('c2')
+            comments.remove(c2)
+            self.failUnless(c2.uuid == 'c2', c2.uuid)
+            self.failUnless(c2.alt_id == None, c2.alt_id)
+            self.failUnless(c2.author == 'Jess', c2.author)
+            self.failUnless(c2.body == 'World\n', c2.body)
+            c4 = comments[0]
+            self.failUnless(len(c4.uuid) == 36, c4.uuid)
+            self.failUnless(c4.alt_id == 'c3', c4.alt_id)
+            self.failUnless(c4.author == 'Jed', c4.author)
+            self.failUnless(c4.body == 'And thanks\n', c4.body)
+        def testRootCommentsNotAddOnly(self):
+            bugB = self.bugdir.bug_from_uuid('b')
+            initial_bugB_summary = bugB.summary
+            self._execute(self.root_comment_xml, {'comment-root':'/b'}, ['-'])
+            uuids = list(self.bugdir.uuids())
+            uuids = list(self.bugdir.uuids())
+            self.failUnless(uuids == ['b'], uuids)
+            bugB = self.bugdir.bug_from_uuid('b')
+            self.failUnless(bugB.uuid == 'b', bugB.uuid)
+            self.failUnless(bugB.creator == 'John', bugB.creator)
+            self.failUnless(bugB.status == 'open', bugB.status)
+            self.failUnless(bugB.summary == initial_bugB_summary, bugB.summary)
+            estrs = ["don't forget your towel",
+                     'helps with space travel']
+            self.failUnless(bugB.extra_strings == estrs, bugB.extra_strings)
+            comments = list(bugB.comments())
+            self.failUnless(len(comments) == 3,
+                            ['%s (%s, %s)' % (c.uuid, c.alt_id, c.body)
+                             for c in comments])
+            c1 = bugB.comment_from_uuid('c1')
+            comments.remove(c1)
+            self.failUnless(c1.uuid == 'c1', c1.uuid)
+            self.failUnless(c1.alt_id == None, c1.alt_id)
+            self.failUnless(c1.author == 'Jane', c1.author)
+            self.failUnless(c1.body == 'So long\n', c1.body)
+            c2 = bugB.comment_from_uuid('c2')
+            comments.remove(c2)
+            self.failUnless(c2.uuid == 'c2', c2.uuid)
+            self.failUnless(c2.alt_id == None, c2.alt_id)
+            self.failUnless(c2.author == 'Jess', c2.author)
+            self.failUnless(c2.body == 'World\n', c2.body)
+            c4 = comments[0]
+            self.failUnless(len(c4.uuid) == 36, c4.uuid)
+            self.failUnless(c4.alt_id == 'c3', c4.alt_id)
+            self.failUnless(c4.author == 'Jed', c4.author)
+            self.failUnless(c4.body == 'And thanks\n', c4.body)
+        def testRootCommentsAddOnly(self):
+            bugB = self.bugdir.bug_from_uuid('b')
+            initial_bugB_summary = bugB.summary
+            self._execute(self.root_comment_xml,
+                          {'comment-root':'/b', 'add-only':True}, ['-'])
             uuids = list(self.bugdir.uuids())
             self.failUnless(uuids == ['b'], uuids)
             bugB = self.bugdir.bug_from_uuid('b')
             self.failUnless(bugB.uuid == 'b', bugB.uuid)
             self.failUnless(bugB.creator == 'John', bugB.creator)
             self.failUnless(bugB.status == 'open', bugB.status)
+            self.failUnless(bugB.summary == initial_bugB_summary, bugB.summary)
             estrs = ["don't forget your towel",
                      'helps with space travel']
             self.failUnless(bugB.extra_strings == estrs, bugB.extra_strings)
diff --git a/libbe/command/list.py b/libbe/command/list.py
index 8baeaa2..44be71b 100644
--- a/libbe/command/list.py
+++ b/libbe/command/list.py
@@ -23,6 +23,8 @@ import re
 import libbe
 import libbe.bug
 import libbe.command
+import libbe.command.depend
+import libbe.command.target
 import libbe.command.util
 
 # get a list of * for cmp_*() comparing two bugs.
@@ -30,19 +32,31 @@ AVAILABLE_CMPS = [fn[4:] for fn in dir(libbe.bug) if fn[:4] == 'cmp_']
 AVAILABLE_CMPS.remove('attr') # a cmp_* template.
 
 class Filter (object):
-    def __init__(self, status, severity, assigned, extra_strings_regexps):
+    def __init__(self, status='all', severity='all', assigned='all',
+                 target='all', extra_strings_regexps=[]):
         self.status = status
         self.severity = severity
         self.assigned = assigned
+        self.target = target
         self.extra_strings_regexps = extra_strings_regexps
 
-    def __call__(self, bug):
-        if self.status != "all" and not bug.status in self.status:
+    def __call__(self, bugdir, bug):
+        if self.status != 'all' and not bug.status in self.status:
             return False
-        if self.severity != "all" and not bug.severity in self.severity:
+        if self.severity != 'all' and not bug.severity in self.severity:
             return False
-        if self.assigned != "all" and not bug.assigned in self.assigned:
+        if self.assigned != 'all' and not bug.assigned in self.assigned:
             return False
+        if self.target == 'all':
+            pass
+        else:
+            target_bug = libbe.command.target.bug_target(bugdir, bug)
+            if self.target in ['none', None]:
+                if target_bug.summary != None:
+                    return False
+            else:
+                if target_bug.summary != self.target:
+                    return False
         if len(bug.extra_strings) == 0:
             if len(self.extra_strings_regexps) > 0:
                 return False
@@ -140,9 +154,10 @@ class List (libbe.command.Command):
         bugdir.storage.writeable = False
         cmp_list, status, severity, assigned, extra_strings_regexps = \
             self._parse_params(params)
-        filter = Filter(status, severity, assigned, extra_strings_regexps)
+        filter = Filter(status, severity, assigned,
+                        extra_strings_regexps=extra_strings_regexps)
         bugs = [bugdir.bug_from_uuid(uuid) for uuid in bugdir.uuids()]
-        bugs = [b for b in bugs if filter(b) == True]
+        bugs = [b for b in bugs if filter(bugdir, b) == True]
         self.result = bugs
         if len(bugs) == 0 and params['xml'] == False:
             print >> self.stdout, "No matching bugs found"
@@ -191,13 +206,8 @@ class List (libbe.command.Command):
         if params['assigned'] == "all":
             assigned = "all"
         else:
-            possible_assignees = []
-            for bug in self.bugdir:
-                if bug.assigned != None \
-                        and not bug.assigned in possible_assignees:
-                    possible_assignees.append(bug.assigned)
             assigned = libbe.command.util.select_values(
-                params['assigned'], possible_assignees)
+                params['assigned'], libbe.command.util.assignees())
         for i in range(len(assigned)):
             if assigned[i] == '-':
                 assigned[i] = params['user-id']
@@ -218,7 +228,7 @@ class List (libbe.command.Command):
         if xml == True:
             print >> self.stdout, \
                 '<?xml version="1.0" encoding="%s" ?>' % self.stdout.encoding
-            print >> self.stdout, '<bugs>'
+            print >> self.stdout, '<be-xml>'
         if len(bugs) > 0:
             for bug in bugs:
                 if xml == True:
@@ -226,7 +236,7 @@ class List (libbe.command.Command):
                 else:
                     print >> self.stdout, bug.string(shortlist=True)
         if xml == True:
-            print >> self.stdout, '</bugs>'
+            print >> self.stdout, '</be-xml>'
 
     def _long_help(self):
         return """
diff --git a/libbe/command/new.py b/libbe/command/new.py
index ac0659b..be18306 100644
--- a/libbe/command/new.py
+++ b/libbe/command/new.py
@@ -38,6 +38,7 @@ class New (libbe.command.Command):
 
     >>> uuid_gen = libbe.util.id.uuid_gen
     >>> libbe.util.id.uuid_gen = lambda: 'X'
+    >>> ui._user_id = u'Fran\\xe7ois'
     >>> ret = ui.run(cmd, args=['this is a test',])
     Created bug with ID abc/X
     >>> libbe.util.id.uuid_gen = uuid_gen
@@ -45,6 +46,10 @@ class New (libbe.command.Command):
     >>> bug = bd.bug_from_uuid('X')
     >>> print bug.summary
     this is a test
+    >>> bug.creator
+    u'Fran\\xe7ois'
+    >>> bug.reporter
+    u'Fran\\xe7ois'
     >>> bug.time <= int(time.time())
     True
     >>> print bug.severity
@@ -80,6 +85,7 @@ class New (libbe.command.Command):
             summary = params['summary']
         bugdir = self._get_bugdir()
         bug = bugdir.new_bug(summary=summary.strip())
+        bug.creator = self._get_user_id()
         if params['reporter'] != None:
             bug.reporter = params['reporter']
         else:
diff --git a/libbe/command/target.py b/libbe/command/target.py
index 0161772..6bb348f 100644
--- a/libbe/command/target.py
+++ b/libbe/command/target.py
@@ -46,6 +46,7 @@ class Target (libbe.command.Command):
     >>> ui.io.stdout = StringIO.StringIO()
     >>> ret = ui.run(cmd, {'resolve':True}, ['tomorrow'])
     >>> output = ui.io.get_stdout().strip()
+    >>> bd.flush_reload()
     >>> target = bd.bug_from_uuid(output)
     >>> print target.summary
     tomorrow
@@ -185,15 +186,24 @@ def add_target(bugdir, bug, summary):
     return target
 
 def targets(bugdir):
+    """Generate all possible target bug summaries."""
     bugdir.load_all_bugs()
     for bug in bugdir:
         if bug.severity == 'target':
             yield bug.summary
 
-def complete_target(command, argument, fragment=None):
+def target_dict(bugdir):
     """
-    List possible command completions for fragment.
-
-    argument argument is not used.
+    Return a dict with bug UUID keys and bug summary values for all
+    target bugs.
     """
+    ret = {}
+    bugdir.load_all_bugs()
+    for bug in bugdir:
+        if bug.severity == 'target':
+            ret[bug.uuid] = bug.summary
+    return ret
+
+def complete_target(command, argument, fragment=None):
+    """List possible command completions for fragment."""
     return targets(command._get_bugdir())
diff --git a/libbe/command/util.py b/libbe/command/util.py
index c9618ad..977b596 100644
--- a/libbe/command/util.py
+++ b/libbe/command/util.py
@@ -34,12 +34,8 @@ def complete_command(command, argument, fragment=None):
     """
     return list(libbe.command.commands())
 
-def complete_path(command, argument, fragment=None):
-    """
-    List possible path completions for fragment.
-
-    command argument is not used.
-    """
+def comp_path(fragment=None):
+    """List possible path completions for fragment."""
     if fragment == None:
         fragment = '.'
     comps = glob.glob(fragment+'*') + glob.glob(fragment+'/*')
@@ -47,6 +43,10 @@ def complete_path(command, argument, fragment=None):
         comps.extend(glob.glob(comps[0]+'/*'))
     return comps
 
+def complete_path(command, argument, fragment=None):
+    """List possible path completions for fragment."""
+    return comp_path(fragment)
+
 def complete_status(command, argument, fragment=None):
     bd = command._get_bugdir()
     import libbe.bug
@@ -57,10 +57,13 @@ def complete_severity(command, argument, fragment=None):
     import libbe.bug
     return libbe.bug.severity_values
 
+def assignees(bugdir):
+    bugdir.load_all_bugs()
+    return list(set([bug.assigned for bug in bugdir
+                     if bug.assigned != None]))
+
 def complete_assigned(command, argument, fragment=None):
-    if fragment == None:
-        return []
-    return [fragment]
+    return assignees(command._get_bugdir())
 
 def complete_extra_strings(command, argument, fragment=None):
     if fragment == None:
diff --git a/libbe/storage/util/settings_object.py b/libbe/storage/util/settings_object.py
index 9f2b7af..ca94f23 100644
--- a/libbe/storage/util/settings_object.py
+++ b/libbe/storage/util/settings_object.py
@@ -204,18 +204,31 @@ class SavedSettingsObject(object):
             self._settings_loaded = True
 
     def save_settings(self):
-        """Load the settings from disk."""
+        """Save the settings to disk."""
         # Override.  Should save the dict output of ._get_saved_settings()
         settings = self._get_saved_settings()
         pass # write settings to disk....
 
     def _get_saved_settings(self):
+        """
+        In order to avoid overwriting unread on-disk data, make sure
+        we've loaded anything sitting on the disk.  In the current
+        implementation, all the settings are stored in a single file,
+        so we need to load _all_ the saved settings.  Another approach
+        would be per-setting saves, in which case you could skip this
+        step, since any setting changes would have forced that setting
+        load already.
+        """
         settings = {}
-        for k,v in self.settings.items():
-            if v != None and v != EMPTY:
-                settings[k] = v
-        for k in self.required_saved_properties:
-            settings[k] = getattr(self, self._setting_name_to_attr_name(k))
+        for k in self.settings_properties:
+            if k in self.settings and \
+                    not self.settings[k] in [None, EMPTY]:
+                settings[k] = self.settings[k]
+            else:
+                value = getattr(
+                    self, self._setting_name_to_attr_name(k))
+                if value not in [None, EMPTY, []]:
+                    settings[k] = value
         return settings
 
     def clear_cached_setting(self, setting=None):
@@ -313,7 +326,8 @@ if libbe.TESTING == True:
             self.failUnless(t.content_type == "text/plain", t.content_type)
             self.failUnless(t.settings["Content-type"] == EMPTY,
                             t.settings["Content-type"])
-            self.failUnless(t._get_saved_settings() == {},
+            self.failUnless(t._get_saved_settings() ==
+                            {"Content-type":"text/plain"},
                             t._get_saved_settings())
             t.content_type = "text/html"
             self.failUnless(t.content_type == "text/html",
diff --git a/libbe/storage/vcs/arch.py b/libbe/storage/vcs/arch.py
index f9b01fd..74ba371 100644
--- a/libbe/storage/vcs/arch.py
+++ b/libbe/storage/vcs/arch.py
@@ -304,7 +304,6 @@ class Arch(base.VCS):
                 self._invoke_client(
                 'file-find', '--unescaped', path, revision)
             relpath = output.rstrip('\n').splitlines()[-1]
-            print >> sys.stderr, 'getting', relpath
             return base.VCS._vcs_get_file_contents(self, relpath)
 
     def _vcs_path(self, id, revision):
diff --git a/libbe/storage/vcs/base.py b/libbe/storage/vcs/base.py
index 15460b0..9fc43c1 100644
--- a/libbe/storage/vcs/base.py
+++ b/libbe/storage/vcs/base.py
@@ -749,7 +749,8 @@ os.listdir(self.get_path("bugs")):
         if revision == None:
             id_to_path = self._cached_path_id.path
         else:
-            id_to_path = lambda id : self._vcs_path(id, revision)
+            id_to_path = lambda id : os.path.join(
+                self.repo, self._vcs_path(id, revision))
         if id==None:
             path = self.be_dir
         else:
@@ -772,9 +773,12 @@ os.listdir(self.get_path("bugs")):
             isdir = os.path.isdir
             listdir = os.listdir
         else:
-            id_to_path = lambda id : self._vcs_path(id, revision)
-            isdir = lambda path : self._vcs_isdir(path, revision)
-            listdir = lambda path : self._vcs_listdir(path, revision)
+            id_to_path = lambda id : os.path.join(
+                self.repo, self._vcs_path(id, revision))
+            isdir = lambda path : self._vcs_isdir(
+                self._u_rel_path(path), revision)
+            listdir = lambda path : self._vcs_listdir(
+                self._u_rel_path(path), revision)
         if id==None:
             path = self.be_dir
         else:
@@ -1046,7 +1050,8 @@ os.listdir(self.get_path("bugs")):
         if revision == None: # don't require connection
             return libbe.util.encoding.get_file_contents(
                 path, decode=True).rstrip('\n')
-        contents = self._vcs_get_file_contents(path, revision=revision)
+        relpath = self._u_rel_path(path)
+        contents = self._vcs_get_file_contents(relpath, revision=revision)
         if type(contents) != types.UnicodeType:
             contents = unicode(contents, self.encoding)
         return contents.strip()
diff --git a/libbe/storage/vcs/hg.py b/libbe/storage/vcs/hg.py
index 076943a..088a141 100644
--- a/libbe/storage/vcs/hg.py
+++ b/libbe/storage/vcs/hg.py
@@ -23,11 +23,21 @@ Mercurial (hg) backend.
 
 try:
     import mercurial
-    import mercurial.version
     import mercurial.dispatch
     import mercurial.ui
 except ImportError:
     mercurial = None
+
+try:
+    # mercurial >= 1.2
+    from mercurial.util import version
+except ImportError:
+    try:
+        # mercurial <= 1.1.2
+        from mercurial.version import get_version as version
+    except ImportError:
+        version = None
+
 import os
 import os.path
 import re
@@ -57,9 +67,9 @@ class Hg(base.VCS):
         self.__updated = [] # work around http://mercurial.selenic.com/bts/issue618
 
     def _vcs_version(self):
-        if mercurial == None:
+        if version == None:
             return None
-        return mercurial.version.get_version()
+        return version()
 
     def _u_invoke_client(self, *args, **kwargs):
         if 'cwd' not in kwargs:
diff --git a/libbe/ui/command_line.py b/libbe/ui/command_line.py
index 7ba6cf5..89d791d 100644
--- a/libbe/ui/command_line.py
+++ b/libbe/ui/command_line.py
@@ -250,6 +250,16 @@ class BE (libbe.command.Command):
     def full_version(self, *args):
         return libbe.version.version(verbose=True)
 
+class CommandLine (libbe.command.UserInterface):
+    def __init__(self, *args, **kwargs):
+        libbe.command.UserInterface.__init__(self, *args, **kwargs)
+        self.restrict_file_access = False
+        self.storage_callbacks = None
+    def help(self):
+        be = BE(ui=self)
+        self.setup_command(be)
+        return be.help()
+
 def dispatch(ui, command, args):
     parser = CmdOptionParser(command)
     try:
@@ -273,9 +283,7 @@ def dispatch(ui, command, args):
 
 def main():
     io = libbe.command.StdInputOutput()
-    ui = libbe.command.UserInterface(io)
-    ui.restrict_file_access = False
-    ui.storage_callbacks = None
+    ui = CommandLine(io)
     be = BE(ui=ui)
     ui.setup_command(be)
 
@@ -285,7 +293,14 @@ def main():
     except CallbackExit:
         return 0
     except libbe.command.UserError, e:
-        print >> ui.io.stdout, 'ERROR:\n', e
+        if str(e).endswith('COMMAND'):
+            # no command given, print usage string
+            print >> ui.io.stdout, 'ERROR:'
+            print >> ui.io.stdout, be.usage(), '\n', e
+            print >> ui.io.stdout, 'For example, try'
+            print >> ui.io.stdout, '  be help'
+        else:
+            print >> ui.io.stdout, 'ERROR:\n', e
         return 1
 
     command_name = args.pop(0)
@@ -299,7 +314,7 @@ def main():
     command = Class(ui=ui)
     ui.setup_command(command)
 
-    if command.name in ['comment', 'commit', 'serve']:
+    if command.name in ['comment', 'commit', 'import-xml', 'serve']:
         paginate = 'never'
     else:
         paginate = 'auto'
diff --git a/misc/xml/be-mbox-to-xml b/misc/xml/be-mail-to-xml
index 1fc41e0..5a1a88f 100755
--- a/misc/xml/be-mbox-to-xml
+++ b/misc/xml/be-mail-to-xml
@@ -24,15 +24,16 @@ followed by a blank line.
 
 import base64
 import email.utils
-from libbe.encoding import get_encoding, set_IO_stream_encodings
-from libbe.utility import time_to_str
-from mailbox import mbox, Message  # the mailbox people really want an on-disk copy
+from libbe.util.encoding import get_output_encoding
+from libbe.util.utility import time_to_str
+import mailbox # the mailbox people really want an on-disk copy
+import optparse
 from time import asctime, gmtime, mktime
 import types
 from xml.sax.saxutils import escape
 
-DEFAULT_ENCODING = get_encoding()
-set_IO_stream_encodings(DEFAULT_ENCODING)
+BREAK = u'--' # signature separator
+DEFAULT_ENCODING = get_output_encoding()
 
 KNOWN_IDS = []
 
@@ -57,6 +58,14 @@ def normalize_RFC_2822_date(date):
         'unparsable date: "%s"' % date
     return time_to_str(mktime(time_tuple))
 
+def strip_footer(body):
+    body_lines = body.splitlines()
+    for i,line in enumerate(body_lines):
+        if line.startswith(BREAK):
+            break
+        i += 1 # increment past the current valid line.
+    return u'\n'.join(body_lines[:i]).strip()
+
 def comment_message_to_xml(message, fields=None):
     if fields == None:
         fields = {}
@@ -128,7 +137,7 @@ def comment_message_to_xml(message, fields=None):
     body = message.get_payload(decode=True) # attempt to decode
     assert body != None, "Unable to decode?"
     if fields[u'content-type'].startswith(u"text/"):
-        body = unicode(body, encoding=charset).rstrip(u'\n')
+        body = strip_footer(unicode(body, encoding=charset))
     else:
         body = base64.encode(body)
     fields[u'body'] = body
@@ -140,8 +149,17 @@ def comment_message_to_xml(message, fields=None):
     lines.append(u"</comment>")
     return u'\n'.join(lines)
 
-def main(mbox_filename):
-    mb = mbox(mbox_filename)
+def main(argv):
+    parser = optparse.OptionParser(usage='%prog [options] mailbox')
+    formats = ['mbox', 'Maildir', 'MH', 'Babyl', 'MMDF']
+    parser.add_option('-f', '--format', type='choice', dest='format',
+                      help="Select the mailbox format from %s.  See the mailbox module's documention for descriptions of these formats." \
+                          % ', '.join(formats),
+                      default='mbox', choices=formats)
+    options,args = parser.parse_args(argv)
+    mailbox_file = args[1]
+    reader = getattr(mailbox, options.format)
+    mb = reader(mailbox_file, factory=None)
     print u'<?xml version="1.0" encoding="%s" ?>' % DEFAULT_ENCODING
     print u"<be-xml>"
     for message in mb:
@@ -151,4 +169,7 @@ def main(mbox_filename):
 
 if __name__ == "__main__":
     import sys
-    main(sys.argv[1])
+    import codecs
+
+    sys.stdout = codecs.getwriter(DEFAULT_ENCODING)(sys.stdout)
+    main(sys.argv)
diff --git a/misc/xml/be-xml-to-mbox b/misc/xml/be-xml-to-mbox
index 34d0aa3..81741bf 100755
--- a/misc/xml/be-xml-to-mbox
+++ b/misc/xml/be-xml-to-mbox
@@ -25,10 +25,9 @@ followed by a blank line.
 """
 
 #from mailbox import mbox, Message  # the mailbox people really want an on-disk copy
-import codecs
 import email.utils
-from libbe.encoding import get_encoding, set_IO_stream_encodings
-from libbe.utility import str_to_time as rfc2822_to_gmtime_integer
+from libbe.util.encoding import get_output_encoding
+from libbe.util.utility import str_to_time as rfc2822_to_gmtime_integer
 from time import asctime, gmtime
 import types
 try: # import core module, Python >= 2.5
@@ -40,8 +39,7 @@ from xml.sax.saxutils import unescape
 
 DEFAULT_DOMAIN = "invalid.com"
 DEFAULT_EMAIL = "dummy@" + DEFAULT_DOMAIN
-DEFAULT_ENCODING = get_encoding()
-set_IO_stream_encodings(DEFAULT_ENCODING)
+DEFAULT_ENCODING = get_output_encoding()
 
 def rfc2822_to_asctime(rfc2822_string):
     """Convert an RFC 2822-fomatted string into a asctime string.
@@ -124,6 +122,11 @@ class Bug (LimitedAttrDict):
             else:
                 self[field.tag] = text
 
+def wrap_id(id):
+    if "@" not in id:
+        return "<%s@%s>" % (id, DEFAULT_DOMAIN)
+    return id
+
 class Comment (LimitedAttrDict):
     _attrs = [u"uuid",
               u"alt-id",
@@ -144,7 +147,9 @@ class Comment (LimitedAttrDict):
         elif "alt-id" in self: id = self["alt-id"]
         else:                  id = None
         if id != None:
-            print "Message-id: <%s@%s>" % (id, DEFAULT_DOMAIN)
+            print "Message-id: %s" % wrap_id(id)
+        if "alt-id" in self:
+            print "Alt-id: %s" % wrap_id(self["alt-id"])
         print "Date: %s" % self["date"]
         print "From: %s" % self["author"]
         subject = ""
@@ -157,7 +162,7 @@ class Comment (LimitedAttrDict):
         print "Subject: %s" % subject
         if "in-reply-to" not in self.keys():
             self["in-reply-to"] = bug["uuid"]
-        print "In-Reply-To: <%s@%s>" % (self["in-reply-to"], DEFAULT_DOMAIN)
+        print "In-Reply-To: %s" % wrap_id(self["in-reply-to"])
         if "extra-strings" in self:
             for estr in self["extra_strings"]:
                 print "X-Extra-String: %s" % estr
@@ -197,7 +202,11 @@ def print_to_mbox(element):
             print_to_mbox(elt)
 
 if __name__ == "__main__":
+    import codecs
     import sys
+    
+    sys.stdin = codecs.getreader(DEFAULT_ENCODING)(sys.stdin)
+    sys.stdout = codecs.getwriter(DEFAULT_ENCODING)(sys.stdout)
 
     if len(sys.argv) == 1: # no filename given, use stdin
         xml_unicode = sys.stdin.read()