From e2c1aa813380694b2bc8f6cb07345aab23c71e27 Mon Sep 17 00:00:00 2001 From: "W. Trevor King" Date: Mon, 20 Jul 2009 11:44:11 -0400 Subject: Added interfaces/email/interactive/README and be-handle-mail options. The README should give enough info to install and use the interface. While I was writing it, I thought that be-handle-mail could use the --be-dir, --tag-base, and --test options. generate_global_tags() helps implement the --tag-base option. I set up a unittest framework since checking is currently a pipe-in-emails-by-hand sort of arrangement, which can be slow ;). Currently only generate_global_tags() is tested. I also restored "show" to ALLOWED_COMMANDS, since it seems to have wandered off ;). --- interfaces/email/interactive/README | 144 ++++++++++++++++++++++++++++ interfaces/email/interactive/be-handle-mail | 115 +++++++++++++++++++--- 2 files changed, 248 insertions(+), 11 deletions(-) create mode 100644 interfaces/email/interactive/README (limited to 'interfaces/email/interactive') diff --git a/interfaces/email/interactive/README b/interfaces/email/interactive/README new file mode 100644 index 0000000..a1f21ef --- /dev/null +++ b/interfaces/email/interactive/README @@ -0,0 +1,144 @@ +Overview +======== + +The interactive email interface to Bugs Everywhere (BE) attempts to +provide a Debian-bug-tracking-system-style interface to a BE +repository. Users can mail in bug reports, comments, or control +requests, which will be committed to the served repository. +Developers can then pull the changes they approve of from the served +repository into their other repositories and push updates back onto +the served repository. + +For details about the Debian bug tracking system that inspired this +interface, see http://www.debian.org/Bugs . + +Architecture +============ + +In order to reduce setup costs, the entire interface can piggyback on +an existing email address, although from a security standpoint it's +probably best to create a dedicated user. Incoming email is filtered +by procmail, with matching emails being piped into be-handle-mail for +execution. + +Once be-handle-mail recieves the email, the parsing method is selected +according to the subject tag that procmail used grab the email in the +first place. There are three parsing styles: + Style Subject + creating bugs [be-bug:submit] new bug summary + commenting on bugs [be-bug:] human-specific subject + control [be-bug] human-specific subject +These are analagous to submit@bugs.debian.org, nnn@bugs.debian.org, +and control@bugs.debian.org respectively. + +Creating bugs +============= + +The create-style interface creates a bug whose summary is given by the +email's post-tag subject. The body of the email must begin with a +psuedo-header containing at least the "Version" field. Anything after +the pseudo-header and before a line starting with '--' is, if present, +attached as the bugs first comment. + + From jdoe@example.com Fri Apr 18 12:00:00 2008 + From: John Doe + Date: Fri, 18 Apr 2008 12:00:00 +0000 + Content-Type: text/plain; charset=UTF-8 + Content-Transfer-Encoding: 8bit + Subject: [be-bug:submit] Need tests for the email interface. + + Version: XYZ + Severity: minor + + Someone should write up a series of test emails to send into + be-handle mail so we can test changes quickly without having to + use procmail. + + -- + Goofy tagline not included. + +Commenting on bugs +================== + +The comment-style interface appends a comment to the bug specified in +the subject tag. The the first non-multipart body is attached with +the appropriate content-type. In the case of "text/plain" contents, +anything following a line starting with '--' is stripped. + + From jdoe@example.com Fri Apr 18 12:00:00 2008 + From: John Doe + Date: Fri, 18 Apr 2008 12:00:00 +0000 + Content-Type: text/plain; charset=UTF-8 + Content-Transfer-Encoding: 8bit + Subject: [be-bug:XYZ] Isolated problem in baz() + + Finally tracked it down to the bar() call. Some sort of + string<->unicode conversion problem. Solution ideas? + + -- + Goofy tagline not included. + +Controlling bugs +================ + +The control-style consists of a list of allowed be commands, with one +command per line. Blank lines and lines beginning with '#' are +ignored, as well anything following a line starting with '--'. All the +listed commands are executed in order and their output returned. +Note that currently arguments are split on spaces, so + "John Doe" -> ['"John', 'Doe"'] +I'm thinking about how to fix this, but for the time being it's best +to avoid spaces. + + From jdoe@example.com Fri Apr 18 12:00:00 2008 + From: John Doe + Date: Fri, 18 Apr 2008 12:00:00 +0000 + Content-Type: text/plain; charset=UTF-8 + Content-Transfer-Encoding: 8bit + Subject: [be-bug] I'll handle XYZ by release 1.2.3 + + assign XYZ John + status XYZ assigned + severity XYZ critical + target XYZ 1.2.3 + + -- + Goofy tagline ignored. + +Example emails +============== + +Take a look at my interfaces/email/interactive/examples for some +more examples. + +Procmail rules +============== + +The file _procmailrc as it stands is fairly appropriate for as a +dedicated user's ~/.procmailrc. It forwards matching mail to +be-handle-mail, which should be installed somewhere in the user's +path. All non-matching mail is dumped into /dev/null. Everything +procmail does will be logged to ~/be-mail/procmail.log. + +If you're piggybacking the interface on top of an existing account, +you probably only need to add the be-handle-mail stanza to your +existing ~/.procmailrc, since you will still want to recieve non-bug +emails. + +Note that you will probably have to add a + --be-dir /path/to/served/repo +option to the be-handle-mail invocation so it knows what repo to +serve. + +Multiple repos may be served by the same email address by adding +multiple be-handle-mail stanzas, each matching a different tag, for +example the "[be-bug" portion of the stanza could be "[projectX-bug", +"[projectY-bug", etc. If you change the base tag, be sure to add a + --tag-base "projectX-bug" +or equivalent to your be-handle-mail invocation. + +Testing +======= + +Send test emails in to be-handle-mail with something like + cat examples/blank | ./be-handle-mail -o -l - -a diff --git a/interfaces/email/interactive/be-handle-mail b/interfaces/email/interactive/be-handle-mail index 1feba92..a82f723 100755 --- a/interfaces/email/interactive/be-handle-mail +++ b/interfaces/email/interactive/be-handle-mail @@ -50,6 +50,8 @@ import send_pgp_mime import sys import time import traceback +import doctest +import unittest HANDLER_ADDRESS = u"BE Bugs " _THIS_DIR = os.path.abspath(os.path.dirname(__file__)) @@ -57,12 +59,13 @@ BE_DIR = _THIS_DIR LOGPATH = os.path.join(_THIS_DIR, u"be-handle-mail.log") LOGFILE = None -SUBJECT_TAG_BASE = u"[be-bug" -SUBJECT_TAG_RESPONSE = u"%s]" % SUBJECT_TAG_BASE -SUBJECT_TAG_NEW = u"%s:submit]" % SUBJECT_TAG_BASE -SUBJECT_TAG_COMMENT = re.compile(u"%s:([\-0-9a-z]*)]" - % SUBJECT_TAG_BASE.replace("[","\[")) -SUBJECT_TAG_CONTROL = SUBJECT_TAG_RESPONSE +# Tag strings generated by generate_global_tags() +SUBJECT_TAG_BASE = u"be-bug" +SUBJECT_TAG_RESPONSE = None +SUBJECT_TAG_START = None +SUBJECT_TAG_NEW = None +SUBJECT_TAG_COMMENT = None +SUBJECT_TAG_CONTROL = None BREAK = u"--" NEW_REQUIRED_PSEUDOHEADERS = [u"Version"] @@ -70,8 +73,8 @@ NEW_OPTIONAL_PSEUDOHEADERS = [u"Reporter", u"Assign", u"Depend", u"Severity", u"Status", u"Tag", u"Target"] CONTROL_COMMENT = u"#" ALLOWED_COMMANDS = [u"assign", u"comment", u"commit", u"depend", u"help", - u"list", u"merge", u"new", u"open", u"severity", u"status", - u"tag", u"target"] + u"list", u"merge", u"new", u"open", u"severity", u"show", + u"status", u"tag", u"target"] AUTOCOMMIT = True @@ -321,9 +324,9 @@ class Message (object): Validate the subject line. """ tag,subject = self._split_subject() - if not tag.startswith(SUBJECT_TAG_BASE): + if not tag.startswith(SUBJECT_TAG_START): raise InvalidSubject( - self, u"Subject must start with '%s'" % SUBJECT_TAG_BASE) + self, u"Subject must start with '%s'" % SUBJECT_TAG_START) tag_type,value = self._subject_tag_type() if tag_type == None: raise InvalidSubject(self, u"Invalid tag '%s'" % tag) @@ -483,6 +486,19 @@ class Message (object): response_body.attach(message) return send_pgp_mime.attach_root(self.response_header, response_body) +def generate_global_tags(tag_base=u"be-bug"): + """ + Generate a series of tags from a base tag string. + """ + global SUBJECT_TAG_BASE, SUBJECT_TAG_START, SUBJECT_TAG_RESPONSE, \ + SUBJECT_TAG_NEW, SUBJECT_TAG_COMMENT, SUBJECT_TAG_CONTROL + SUBJECT_TAG_BASE = tag_base + SUBJECT_TAG_START = u"[%s" % tag_base + SUBJECT_TAG_RESPONSE = u"[%s]" % tag_base + SUBJECT_TAG_NEW = u"[%s:submit]" % tag_base + SUBJECT_TAG_COMMENT = re.compile(u"\[%s:([\-0-9a-z]*)]" % tag_base) + SUBJECT_TAG_CONTROL = SUBJECT_TAG_RESPONSE + def open_logfile(logpath=None): """ If logpath=None, default to global LOGPATH. @@ -511,13 +527,27 @@ def close_logfile(): if LOGFILE != None and LOGPATH not in [u"stderr", u"none"]: LOGFILE.close() +def test(): + unitsuite = unittest.TestLoader().loadTestsFromModule(sys.modules[__name__]) + suite = unittest.TestSuite([unitsuite, doctest.DocTestSuite()]) + result = unittest.TextTestRunner(verbosity=2).run(suite) + num_errors = len(result.errors) + num_failures = len(result.failures) + num_bad = num_errors + num_failures + return num_bad def main(): from optparse import OptionParser - global AUTOCOMMIT + global AUTOCOMMIT, BE_DIR usage="be-handle-mail [options]\n\n%s" % (__doc__) parser = OptionParser(usage=usage) + parser.add_option('-b', '--be-dir', dest='be_dir', default=BE_DIR, + metavar="DIR", + help='Select the BE directory to serve (%default).') + parser.add_option('-t', '--tag-base', dest='tag_base', + default=SUBJECT_TAG_BASE, metavar="TAG", + help='Set the subject tag base (%default).') parser.add_option('-o', '--output', dest='output', action='store_true', help="Don't mail the generated message, print it to stdout instead. Useful for testing be-handle-mail functionality without the whole mail transfer agent and procmail setup.") parser.add_option('-l', '--logfile', dest='logfile', metavar='LOGFILE', @@ -525,9 +555,20 @@ def main(): parser.add_option('-a', '--disable-autocommit', dest='autocommit', default=True, action='store_false', help='Disable the autocommit after parsing the email.') + parser.add_option('--test', dest='test', action='store_true', + help='Run internal unit-tests and exit.') options,args = parser.parse_args() + + if options.test == True: + num_bad = test() + if num_bad > 126: + num_bad = 1 + sys.exit(num_bad) + + BE_DIR = options.be_dir AUTOCOMMIT = options.autocommit + generate_global_tags(options.tag_base) msg_text = sys.stdin.read() libbe.encoding.set_IO_stream_encodings(ENCODING) # _after_ reading message @@ -560,5 +601,57 @@ def main(): send_pgp_mime.mail(response, send_pgp_mime.sendmail) close_logfile() +class GenerateGlobalTagsTestCase (unittest.TestCase): + def setUp(self): + super(GenerateGlobalTagsTestCase, self).setUp() + self.save_global_tags() + def tearDown(self): + self.restore_global_tags() + super(GenerateGlobalTagsTestCase, self).tearDown() + def save_global_tags(self): + self.saved_globals = [SUBJECT_TAG_BASE, SUBJECT_TAG_START, + SUBJECT_TAG_RESPONSE, SUBJECT_TAG_NEW, + SUBJECT_TAG_COMMENT, SUBJECT_TAG_CONTROL] + def restore_global_tags(self): + global SUBJECT_TAG_BASE, SUBJECT_TAG_START, SUBJECT_TAG_RESPONSE, \ + SUBJECT_TAG_NEW, SUBJECT_TAG_COMMENT, SUBJECT_TAG_CONTROL + SUBJECT_TAG_BASE, SUBJECT_TAG_START, SUBJECT_TAG_RESPONSE, \ + SUBJECT_TAG_NEW, SUBJECT_TAG_COMMENT, SUBJECT_TAG_CONTROL = \ + self.saved_globals + def test_restore_global_tags(self): + "Test global tag restoration by teardown function." + global SUBJECT_TAG_BASE + self.failUnlessEqual(SUBJECT_TAG_BASE, u"be-bug") + SUBJECT_TAG_BASE = "projectX-bug" + self.failUnlessEqual(SUBJECT_TAG_BASE, u"projectX-bug") + self.restore_global_tags() + self.failUnlessEqual(SUBJECT_TAG_BASE, u"be-bug") + def test_subject_tag_base(self): + "Should set SUBJECT_TAG_BASE global correctly" + generate_global_tags(u"projectX-bug") + self.failUnlessEqual(SUBJECT_TAG_BASE, u"projectX-bug") + def test_subject_tag_start(self): + "Should set SUBJECT_TAG_START global correctly" + generate_global_tags(u"projectX-bug") + self.failUnlessEqual(SUBJECT_TAG_START, u"[projectX-bug") + def test_subject_tag_response(self): + "Should set SUBJECT_TAG_RESPONSE global correctly" + generate_global_tags(u"projectX-bug") + self.failUnlessEqual(SUBJECT_TAG_RESPONSE, u"[projectX-bug]") + def test_subject_tag_new(self): + "Should set SUBJECT_TAG_NEW global correctly" + generate_global_tags(u"projectX-bug") + self.failUnlessEqual(SUBJECT_TAG_NEW, u"[projectX-bug:submit]") + def test_subject_tag_control(self): + "Should set SUBJECT_TAG_CONTROL global correctly" + generate_global_tags(u"projectX-bug") + self.failUnlessEqual(SUBJECT_TAG_CONTROL, u"[projectX-bug]") + def test_subject_tag_comment(self): + "Should set SUBJECT_TAG_COMMENT global correctly" + generate_global_tags(u"projectX-bug") + m = SUBJECT_TAG_COMMENT.match("[projectX-bug:xyz-123]") + self.failUnlessEqual(len(m.groups()), 1) + self.failUnlessEqual(m.group(1), u"xyz-123") + if __name__ == "__main__": main() -- cgit