How to integrate Jira, Bitbucket & Bamboo with Google Hangouts Chat

Here at Toro, we’re big fans of Atlassian's suite of products. We use them on a daily basis, particularly Jira, Bitbucket, Bamboo, and the soon to be defunct (at time of writing) HipChat.

One thing we loved about HipChat was how easy it was to integrate with the other Atlassian products we use. This allowed us to get messages in HipChat when Bamboo builds completed, when Jira issues were opened or updated, and when any activity in Bitbucket would occur (such as commits, pull requests being approved, comments, etc).

Once we got wind of the pending HipChat shutdown, our DevOps team quickly got to work on identifying a new IM platform to use. The criteria was quite simple:

• Cost

• Ease-of-use

• Feature Set (Chat History/Secure Attachments/Integration with Google Directory)

• Ability to Integrate it to our existing suite of software to be notified of build/development events.

Our team went over a few of the most popular platforms, (and some that weren’t very popular), but in the end we decided to go with Google Hangouts Chat, particularly since we already use Google G Suite for email, document storage and productivity applications.

Some of the things we liked about Google Hangouts Chat:

• Unlimited chat history which is very handy when looking back to old conversations.

• Ability to attach Google Drive documents which means attachments are secured by default rather than just uploading them on a public S3 bucket which is what HipChat does.

• Threaded conversations take a bit of getting used to but they do assist in keeping a conversation on topic

• Cost. We already subscribe to Google G Suite so there was no additional cost for the Chat app.

Some of the things that we didn’t like about Google Hangouts Chat:

• We didn’t love the UI. Maybe that was just because we were used to Hipchat. But it definitely wasn’t as slick as Slack and didn’t include a dark theme.

• Integration was much more limited. Compared to solutions like Slack the number of integrations was very limited and some of our must have integrations such as the Atlassian product suite didn’t exist.

Eat your own dog food

Toro is also a big fan of eating our own dog food. When I was asked to integrate Atlassian’s products with Google Hangouts I quickly got to work and started reading up on Atlassian’s and Google’s documentation. The rest of this post describes how each app was integrated using Toro Integrate.

Creating services for Google Hangouts Chat

The first thing I did was create a few Gloop Services to send IMs to Google Hangouts using Webhooks in Chat. Toro Integrate is compatible with the OpenAPI specification, but Google has their own format called the API Discovery Service (the Google Hangout API Discovery file is here). Fortunately, we have an integration that reads in all of Google’s APIs and converts them to OpenAPI, and also adds them to our Marketplace! So to get Toro Integrate sending IMs was simply a matter of importing the Open API specification from the Toro Marketplace.

Unfortunately, I found that the API specification from Google was missing a field required for Bots to use Chat webhooks called token, so I added it as an input to the generated ChatSpacesMessagesCreate service, and mapped it as a query string parameter, as shown below:

Another small change I made was toggle the input properties of the generated service from allowNull = false to  allowNull = true. This change was made because the API doesn’t require these fields at runtime, and it removed warnings from the Coder UI while I was developing the Integrations.

With the amended code in place, I added the API key as a package property (which could easily be retrieved using a one liner), and wrapped the generated service with a new one that had the bare minimum of input properties, to make the resulting code much cleaner and easier to use.

You can see in the above service that at line 3 I use the aforementioned one-liner to get the API key. For bonus points the service will post the message to a test room (lines 4-7) if the calling code didn’t provide a spacesId. This also uses the one-liner code.

Finally, I created a Bitbucket, Jira, and Bamboo Webhook for each room, and saved the space IDs and tokens in a text file, for use later. Below is a gif that shows the simple wrapper service successfully sending a message to a room as a bot.

Integrating Bitbucket with Google Hangouts Chat

First cab off the rank now that we were chatting with Google was to add a Bitbucket integration, since that sent the most events to HipChat. So I got to work creating creating Gloop Models from the sample JSON payloads in the Atlassian documentation, then created a simple Gloop Service and Rest API that Bitbucket could POST to. Since we had a separate webhook for each repository, I added the Google spacesId and token input properties as parameters to the API itself, meaning each webhook can be configured in Bitbucket to choose which room the messages would get sent to.

You might notice that in the API, the Body Parameter is mapped to a Gloop Model called requestBody whose allowExtraProperties is true (when it’s true, the model icon appears with a * badge/overlay icon on the top left). This was done because at runtime we don’t know what fields the payload will contain.

Next up I started playing with a test git repository in Bitbucket. I configured the webhook in the repository, then started pushing commits, and playing with PRs and branches, in order to have the webhook invoked, and saving payloads in the tracker so I could re-send them at will. This would allow me to test my code and have the resulting messages sent to Google as similar as possible to what was getting sent to HipChat.

This took a little longer than expected because the example payloads in the Atlassian documentation were missing a lot of fields. However, it turns out that Bitbucket has an Open API schema in the Toro Marketplace which contains the models required in the webhooks too!

Once all the models were configured in Integrate, I set about adding the logic in Gloop to parse the X-Event-Key header (which helps determine what the payload looks like), and the fields in the request payload itself to determine what sort of IM message to send (or whether to send a message at all).

The screenshot above shows the main parts of the webhook service. As you can see the value of the header is checked at line 8, then depending on the header value and values in the payload, different Hangouts messages are built. Finally, at line 63, if there’s a message to be sent, it’s logged to the logger and sent to Google using the wrapper service I discussed earlier.

Unfortunately Google Hangouts doesn’t have the ability to style the messages a lot, so I added some emojis to make them look prettier.

Integrating Jira with Google Hangouts Chat

Next up was Jira. This turned out to be much easier than Bitbucket. We were only logging new issues and status changes to Google Hangouts. With Jira we had 2 options with regards to how the webhook in Integrate would send tokens and space ids to Google.

• One webhook per project (using JQL to filter the events), and include the token and space id in the webhook request

• One webhook for all our development projects, and have Toro Integrate lookup the token and space id at runtime.

To keep things consistent with the other webhooks, I went with option 1. We had our DevOps configure the webhooks using JQL to ensure events from certain Jira projects went to their corresponding rooms.

Jira’s payload is mostly the same format regardless of what happens, but the payload does differ because fields appear and disappear based on what happened (and custom fields in issues in particular changes the structure of the payload). For this reason I decided to create a Gloop Model from a few request JSON payloads that Jira had sent, and included the fields I needed to build the IM, anything else I either deleted from the model, or left them there in case I wanted to use them later.

For any model properties that I kept, I ensured that allowExtraProperties was true so if any new fields pop up in the future, Gloop won’t error out.

Below is a screenshot of the resulting Jira webhook API, and as you can see, it’s quite simple.

The webhook service is also rather simple. It gets the event from the request body payload (unlike Bitbucket which sends it as a header), and builds an IM based on the contents of the payload.

You might notice on lines 6-8, there are 3 services for getting emojis based on the Issue data. This was done for the same reasons emojis were added to the Bitbucket Integration (to make them prettier). Below is a screenshot of one of these services (GetEmojiForStatus).

With a little testing to ensure only the proper events from Jira were being logged, we were done! To keep things clean in the room, all events related to a particular issue are also placed in the same thread, as you can see below.

One thing to note with Jira though, I noticed that the equal sign at the end of our token wasn’t being escaped properly by Jira, so I removed it from the URL, and had Integrate add the equal sign at runtime.

Integrating Bamboo with Google Hangouts Chat

Bamboo was the easiest of the lot, it literally took around 20 minutes to get it working! It needed a bit of ‘creative thinking’ though. The problem with Bamboo is that there was no generic webhook functionality included.

However, while going through the options, I noticed a little tip about the Slack option under the Slack webhook URL field:

Hmm, that looks interesting! So I searched for the Slack Incoming webhook API and came across this page. Point 4 was of particular interest. I figured if I could emulate Slack, I could at least get something over HTTP. So I went about creating the webhook, related service and API, and a Gloop Model from the payload that Bamboo had sent after a few attempts at testing it all. Below is the Gloop Model created from the slack-like payload we got from Bamboo:

Since Bamboo allows separate webhooks per build project (similar to how Bitbucket allows separate webhooks per repository), I made the API the same as the Bitbucket API, whereby the token and spacesId would be part of the request.

Not only was Bamboo sending me the payloads as expected, the payloads already included a nice IM in a markup format that was compatible with Google! The only hiccup I had was that sometimes Bamboo would send the payload in the request body, and other times as a parameter called payload. I suspect it could be because of how I configured the webhook.

Either way I added a step in the service (at lines 3 and 4) to check for the payload parameter, and to use that instead of the request body. Apart from that everything else worked!

Once I was happy with the code, all that I needed to do was reconfigure the webhooks (or more accurately, the CNAME of the DNS record they were pointing to, to avoid having to reconfigure them all again), then deploy the code to the target server.

How to implement Google Chat with your instance of Jira/Bitbucket/Bamboo today

1. Get Toro Integrate by either:

   1. Installing Toro Integrate & Toro Coder Studio on your own computer; or

   2. Instantly provisioning an instance of Toro Integrate on Toro Cloud

2. Download the packages from the marketplace.

   • Bamboo
   • Bitbucket
   • Jira
     
     

3. Configure the bots in hangouts. To do this:

   1. Click on the members drop down at the top of your desired room, then select Configure webhooks.

      

   2. Click on the Add button, and enter the name and (optionally) the URL of the avatar of your bot, then click Save.

      

   3. Copy the link that Google has generated. Save the URL for Step 4, by clicking on the Copy button (shown below). The URL will be in the following format:

      https://chat.googleapis.com/v1/spaces/<SPACE ID>/messages?key=<API KEY>&token=<TOKEN>

       

4. In Toro Integrate, add your hangouts key as an application property called hangouts.key (shown below at line 30). The value of the property is the API KEY from the URL in step 3.

   

5. If you’re using the Jira bot, populate the jira.base.url property in the JiraGoogleBot package properties file.

6. Start your Integrate packages (if they aren’t already started), then configure your Atlassian app webhooks as follows:

   Application	Webhook URL
   Jira	<SERVER URL>/api/jira/hook?spacesId=<SPACE ID>&token=<TOKEN>
   Bamboo	<SERVER URL>/api/bamboo/hook?spacesId=<SPACE ID>&token=<TOKEN>
   Bitbucket	<SERVER URL>/api/bitbucket/hook?spacesId=<SPACE ID>&token=<TOKEN>

    

   Don’t forget to add your space ID and token to the web hooks!

7. Done!