We killed the connector. Long live the API schema.

Why Enterprise Application Integration platforms have traditionally used connectors

A connector (oftentimes called an SDK) is a piece of code that wraps an external API so that it can communicate with your application. Huh? Ok, well, let’s run through practical example...

Say, for example, that you wanted to update a contact in your CRM system every time there was a change to the same contact in your ERP (or accounting system). The providers of both the CRM and ERP system document their APIs on their websites. This documentation describes all of the operations available in the API, the method by which you should connect to the API, security and other settings.

However, this is just documentation. It doesn’t provide any code to actually implement the API in your integration. A developer will typically require a connector that wraps all of the operations of the API and expose them as methods that they can call in their code. In our example, the developer would require a connector for each of the APIs that will be consumed and the connector will expose the operations as methods such as getContact and updateContact.

Most of the Enterprise Application Integration (EAI) platform software industry use connectors to facilitate communication between the software vendors application and an API. The connector is specific to the EAI platform in question and wraps the API in a language that the integration platform understands, making all of the operations of the API accessible to the user of the application.

Why we killed the connector

What’s wrong with connectors? They go out of date. Often. Really often. You see, publishers frequently change their APIs, oftentimes with little to no warning. With a connector, you are reliant on the integration platform vendor to:

• have a connector for each API you would like to integrate;

• proactively monitor when publishers make changes to their API; and

• update the connector to support changes to the API.

What’s worse is that connectors are usually coded by hand. That makes creating and updating them time-consuming and error prone. Maintaining connectors for even a few hundred APIs is a nightmare. Yet there are tens of thousands of public APIs and they are growing at an exponential rate. Creating and maintaining a connector for each of them is near impossible.

The connector is dead. Long live the API schema.

With the introduction of Toro Integrate v3, we eliminated the dependency on connectors. Instead, you can integrate directly with any API by simply importing an API schema that describes the API in a machine and human readable format. Toro Integrate parses the API schema and automatically generates a service for each operation in the API that can then be invoked immediately or incorporated into a larger integration service. All without writing a single line of code.

If the API changes, just import the publisher’s latest schema and you’re ready to go again. There is no need to wait in hope that the vendor of your application integration platform will ship a new connector to incorporate the latest API changes.

What is an API schema?

An API schema is a resource generated from an API (application programming interface) DL (description language). A DL is a structured description of a RESTful web API that can be read by a human and processed by a machine. Usually, they are defined in JSON but can also be in YAML, XML or even Markdown. So when we refer to an API schema we are actually referring to the JSON (YAML or XML) resource of an API DL.

API schemas provide machine readable structured documentation that together with regular documentation and an API explorer environment (or development sandbox) will make an API more discoverable. API schemas have several other benefits for the publisher of an API including streamlining the development process and making an API more maintainable. We will go into more details of the benefits of an API first development methodology using API schemas in another article.

OK so there is just one format for API schemas, right?

Nope. But the industry is getting there. As you can imagine there wasn’t a consensus on the definition of API DL’s from day one. In fact, for a quite a while there were API definition wars from competing formats. These have largely been won now by a specification called OpenAPI. Most of the industry heavyweights, including those that used to promote their own competing formats, have now joined the OpenAPI bandwagon.

Given that OpenAPI has largely emerged as the industry standard API DL we won’t go into the advantages and disadvantages of all the competing formats. However, here are some of the major formats you may see promoted:

OpenAPI

This is an industry alliance that has formed around an open source API DL project called Swagger. Swagger was donated by Smartbear Software to the OpenAPI Initiative and is governed under the Linux Foundation. You will still see many APIs using or referring to the Swagger definition. This typically refers to a Swagger v2.x API definition. From v3.0 the Swagger definition became known as OpenAPI.

RAML

RAML was developed by Mulesoft and was a modelling language designed to be suitable to document both RESTful web APIs as well as SOAP and RPC protocols. Mulesoft is now a member of the OpenAPI Initiative.

API Blueprint

API Blueprint was developed by Apiary (now part of Oracle) and is a documentation-focused API DL defined in Markdown. Oracle + Apiary is now a member of the OpenAPI Initiative.

WADL

WADL (Web Application Description Language) uses XML and focuses on machine readability. WADL attempted to be the REST equivalent of SOAP, however, WADL never really took off and is not widely adopted.

Fortunately, Toro Integrate supports the major API DL formats as well as APIs defined in the older SOAP protocol. This means that no matter what API DL format is used, you should be able to import it to Integrate, automatically generate the services for all the operations of the API, and start integrating.

Cool, I’m on board, but what if the API I want to consume doesn’t publish an API schema?

Companies and services including Slack, Bitbucket, Xero, Marketo, BigCommerce, eBay, Box, Yammer, Lyft, SendGrid, Zuora, Insightly, and Avalara already make API schemas publicly available. API publishers want to make their APIs more discoverable and as API schemas facilitate this, it is within their interest to publish an API schema.

However, what do you do if the API you want to consume doesn’t publish an API schema?

One option is to read the publishers API documentation and manually implement the operations you want to consume yourself. This is easy enough to do with Toro Integrate as making an ad hoc connection to a RESTful web API can be done by navigating through a few steps of a wizard. However, this is not terribly convenient if there are dozens of operations you would like to use.

TORO Docs to OpenAPI creates schemas for APIs without schemas

Fortunately, API documentation is typically generated from templates so the formatting is consistent. We built a Chrome extension called Toro Docs to OpenAPI that leverages this consistent formatting of API documentation in order to map the CSS classes contained within the page to the operations of an API. The extension then crawls the page and generates an OpenAPI compliant schema. Toro Docs to OpenAPI is free to download and use, doesn’t require any other software, and the schemas generated by it can be used for any purpose. You can download Toro Docs to OpenAPI yourself and give it a try.

Finally, there are over 1,200 API schemas, and growing every day, in the Toro Marketplace. Those created with Toro Docs to OpenAPI can be opened and edited directly within the extension. All of the schemas in the Toro Marketplace can be downloaded to Toro Integrate, including the free Desktop Edition.

Get started consuming an API using API schemas

To consume an API using an API schema today, follow these steps: