Continuous Build Integration

If you don't know what Continuous Build Integration is, then you should because it makes the development process work very well.

Here at UnnamedStartup® we're using Mercurial (a.k.a. Hg) as our revision control system and we wanted to use buildbot to manage our continuous integration according to the following diagram:


I've just got the closed loop from developer to Hg to Buildbot to email. It could have gone smoother. Here's a few of the challenges I faced, and how I solved them.

1. Sending Check-in Notifications from Mercurial to Buildbot - There is a user contribution to buildbot for doing this. You can download hg_buildbot.py and make sure it is executable to the user(s) commiting to your Mercurial repository. Follow the instructions in the comments of that file to have mercurial call this script. Please note that if you are submitting changes via https, I hit a bug and ended up changing to committing via ssh to work around it (for what it's worth, ssh is faster than https).
   
2. Configure Buildbot Sources - The hg_buildbot.py script is expecting buildbot to be accepting changes from a PBChangeSource. Configure buidbot like so:
   
from buildbot.changes.pb import PBChangeSource
c['sources'].append(PBChangeSource())

   
3. Unified email recipients list - I've configured Mercurial to send email notifications like so:
   

   
   .hg/hgrc
   
   ...
   [hooks]
   #callback to the notifier extension when changegroups are constructed
   changegroup.notify = python:hgext.notify.hook
   
   [notify]
   #only send out emails if a changegroup is pushed to the master repository
   sources = serve
   # set this to True when you need to do testing
   test = False
   config = /usr/local/share/hg/my_email_notifications
   template = Subject: Changes in repository: {desc|firstline|strip}\nFrom: {author}\n\ndetails:   {baseurl}/rev/{node|short}\nchangeset: {rev}:{node|short}\nuser:      {author}\ndate:      {date|date}\ndescription:\n{desc}\n
   ...
   
   
   /usr/local/share/hg/my_email_notifications
   
   [reposubs]
   * = "Developer 1"<dev1@unamedstartup.com>, "Developer 2"<dev2@unamedstartup.com>
   
   

   
   So we now want to use the same list when telling our developers that the build failed. Since buildbot configuration file is just python we can embed this parsing code directly in our configuration file:
   

   
   emailcfg = open("/usr/local/share/hg/my_email_notifications")
   emailcfg.readline()
   import re
   emailparser = re.compile("<(.+@.+)>")
   emails = map(lambda s: emailparser.search(s).group(1),
                emailcfg.readline().split("=")[1].split(","))
   emailcfg.close()
   

   
   Granted, this isn't going to handle changes to the my_email_notications file very well, but you should get the idea. The important thing is that we aren't maintaining two lists of emails.
   
4. Sending email to an authenticating smtp server - Out of the box buildbot can only send email to an open SMTP server... I'm not sure who's dumb enough to leave their email server open like that, but we don't. So a little reading through the twisted libraries and I found that twisted kind-of supports ESMTP. I had to wrap this up in a buildbot notifier. Here's the code for that:
   

   
   from buildbot.status.mail import MailNotifier
   class ESMTPMailNotifier(MailNotifier):
     def __init__(self, username=None, password=None, port=25, *args, **kwargs):
       MailNotifier.__init__(self,*args,**kwargs)
       self._username = username
       self._password = password
       self._port = port
     def sendMessage(self, m, recipients):
           from twisted.internet.ssl import ClientContextFactory 
           from twisted.internet import reactor
           from twisted.mail.smtp import ESMTPSenderFactory
           from StringIO import StringIO
           from twisted.internet import defer
           s = m.as_string() 
           ds = []
           for recip in recipients:
              if not hasattr(m,'read'):
                   # It's not a file
                   m = StringIO(str(m))
              d = defer.Deferred()
              factory = ESMTPSenderFactory(self._username, self._password,
                                           self.fromaddr, recip, m, d,
                                           contextFactory=ClientContextFactory())
              reactor.connectTCP(self.relayhost, self._port, factory)
              ds.append(d)
           return defer.DeferredList(ds)
    
   from buildbot.status import mail 
   c['status'].append(ESMTPMailNotifier(username="yourusername",
                                        password="yourpassword",
                                        fromaddr="you@yourcompany.net",
                                        relayhost="smtp.gmail.com",
                                        mode="all",
                                        extraRecipients=emails,
                                        sendToInterestedUsers=False))
   

   
   

Granted, this isn't a complete guide to how to set up Mercurial and Buildbot, but I hope this will help you get over some of the minor hurdles I had to jump over.