Kevin Montalbo

APIs are recognized as the cornerstone of any digital transformation project. Through APIs organizations can securely expose data to internal systems, customers and business partners. They can also be a path to new revenue opportunities by creating digital products out of an organization's data or business processes. 

Our guest for today describes APIs as the ultimate do over solution for enterprise it. And in this round of cocktails, we discuss how organizations can establish a formal API program. We also take a look at both top down and bottom up organizational approaches towards digital transformation and discuss the align define and design approach to implementing APIs. 

Joining me today is Toro Cloud CEO and founder David Brown. Good morning, David. 

David Brown

Good morning. 

Kevin Montalbo

And our guest for today is a highly experienced microservice architecture consultant with more than 20 years of experience in architecting building and deploying software products for fortune 500 companies as well as early stage start ups. He's also the founder of LaunchAny, a consulting firm that specializes in digital transformation API design development and cloud infrastructure. He's also the author of the book, a practical approach to API design from principles to practice. James Higginbotham. Welcome to the show. How are you?

James Higginbotham

Great. Thanks for having me.

Kevin Montalbo

All right. James Higginbotham. Let's jump right in. So you've previously discussed that when an organization undergoes digital transformation, they need to create a formal API program. So what should be included in a formal API program?

James Higginbotham

Well, first, it's important to recognize that APIs are really the ultimate do over for an enterprise, whether they're producing product or whether it's just for their it department. It's helping the organizations to start thinking about the digital capabilities that they bring to their workforce, their partners, their customers. So what I like to do is I like to look at an API program from eight core disciplines as part of that digital transformation process. The first is their program strategy. Why it exists what they're doing with APIs. The second is to the process and governance, you know, how are they operating? What kind of checks do they have in place and workflows. The third is their portfolio management. How their APIs take the shape of the organization and vice versa? How are their APIs incorporated into the overall portfolio? What kind of API styles are they offering rest? Graph QLR PC, maybe even some async APIs like web hooks and so on. And how does that all together help people get things done? Then we need to look at the discovery and the adoption side. So, is the API documented well and is it discoverable? Are we making sure that people can find the APIs they need as part of that transformation program? Fifth? Are they on boarding easily? Are they one morning developers easily by that? I mean, you know, how easy is it for a developer to get up and running with an API or how difficult have they made it? Perhaps? We focus also on design and delivery. How are they're designed, how we deliver the APIs through the standard STLC, the software development life cycle. We look at the management analytics aspects. how are they tracking the APIs their usage? What ones are really being used heavily? Which ones aren't being used at all? And then finally, we encourage product ownership of APIs. So how do we look at our APIs overall as products so that we don't just release them once and then forget about them and treat them more like one off projects? But rather we treat them like products that we continue to release and improve, mature and grow them over time.

David Brown

That last point kind of comes full circle back to where you started, right? In terms of determining from an organizational perspective, why are we doing this? It's part of a digital transformation strategy, but we may be trying to create new digi digital products out of our data or services or processes. Um And so let's talk about the sort of organizational requirements from the perspective of people rather than the technology in terms of getting everyone on board as to what the strategy is here and, and why we're doing it and getting everyone, everyone's buying. What, what's the process there do you recommend?

James Higginbotham

Yeah, so I see a few things, one is making sure that all the executives are in alignment. Some digital transformation projects come from the executives down, but others start kind of from the ground up where there's a, a need and a recognition, but we haven't fully experienced the full buy in of the executives. So being able to hone the message, understand and show through execution, the ability to transform the organization in some particular area and then reincorporating that messaging throughout. The other thing is training, making sure that teams are trained up on what this means. So we'll oftentimes fall back into the patterns that we always have the way we build things. We al we won't initially look for APIs. We're gonna look at data stores. What's the data store that has the information I need? And who do I need to talk to? Get access to it? And we don't look for those APIs, we don't look at developer portals. One of the other things that we see is, is having that executive buy in and having developer portals shifts the conversation quite a bit. I've been in organizations where they've had all these different APIs and they weren't gaining adoption. They, they would come to me and say, well, we've got some API started but no one's using them. How do we help with that? And so it's evangelizing that throughout the organization. So there's a lot of preparation. 

David Brown

So that, that develop a poll you're talking about is about giving some accessibility to the API so that people within the organization can discover what's available

James Higginbotham

It is. Um And there are multiple personas that needed to look at a portal. We often times focus on the developer, you know, where's the reference documentation that I need to write the code to integrate with this API and that's super important. But one of the factors that is not always thought about is the evangelistic capabilities that the portal gives. So one of the clients that I had, they went from a few APIs and a developer portal that was behind a login screen inside their API management layer that only about five people had access to, to a portal where vps that had never written a line of code before were running around showing the portal and being able to demonstrate not just the reference documentation, but what are the capabilities that the APIs are delivering to the organization and to the marketplace and they were actually evangelizing all over the organization. So every time they would travel to one of their other um office locations across North America and even overseas, they would, they would be showing the portal and being able to see, see what we're doing today., look, actually there's a new capability that's been released that I didn't even see last time I pulled this up and they become evangelists and help spread that word. And that is absolutely instrumental to a digital transformation effort. Otherwise, it kind of stays inside of it and gets locked inside of it and siloed and it doesn't break out of that.

David Brown

Yeah, for and for those that are not familiar with developer portals, what kind of things would you expect to see in a developer portal? 

James Higginbotham

So some of the primary things we see is the API documentation um how to use an API we oftentimes have documentation on how to get started or how to register your application so that you can access the API but really developer portals can take on a life of their own and they really should be treated like a product like anything else. And that means understanding who's gonna be visiting the portal and bringing content to bear that helps people use that portal effectively. And that means helping people understand what exists, you know, what kinds of things already existed there and keeping it in a very high level form of discussion or content rather than something that's very technical. So that the non technical people in the team can, can go to the portal and reference it and see what kind of APIs do we have? Um How do I use an API, how do I get started? And then they also will often times have documentation for style guides to tell people kind of, here's how to, how you want to design your API. So they all look consistent. There'll be reference documentation on how to actually use them. They'll oftentimes have examples as well and even reference applications you can download or view or, or interact with and and, and, and documentation that you can interact with as well to try out the API. So it should be, you know, kind of multifaceted, not just focused on, on the few developers in your organization that may be needing to use it.

David Brown

And speaking of users and giving accessibility to the APIs, there is obviously a challenge associated with permissions and access to data because it's natural tendency is to restrict access, right? So to provide the minimum level of permissions. And so if you're creating an API, you might tend to want to restrict the number of people that are allowed to access that API but when you're talking about a large dispersed organization, um how do you balance that challenge of providing only that access that is required from security and privacy point of view to mass adoption and digital transformation and agile organization that can discover these APIs and respond quickly.

James Higginbotham

Yeah, it is a very big challenge and a lot of the organizations I work with are dealing with, with anything from PI I data to regulatory requirements that restrict what they're able to do. So they often do a few different things. One is they categorize or classify their APIs um as to what kind of P I data if any it may be offering. And so they'll have kind of a base level of acceptance that if someone on boards with an API one, they're gonna do it in a sandbox environment, not production and they're working with an API that doesn't have those kinds of rigid requirements. They don't offer any kind of that, that concerning data. The, the other thing that they do is they oftentimes partition their APIs so they may partition them into different portfolios, some of those portfolios would be private and would be only used for internal purposes. And furthermore, they may be restricted inside of certain data centers that have certain types of compliance and auditability that others may not. And they'll use those kinds of techniques to try to keep the organization as open as possible and then have the kinds of a typical process approval process as you would expect for the more restrictive APIs. That way they're encouraging APIs to be built and deployed, they're reviewed for the types of data that they're exposing and then they're categorized and partitioned accordingly.

David Brown

Hm. And there's something else I want to dive into as well. This is something you mentioned earlier about api adoption occurring from both a bottom up and a top down demand approach. So we often think of digital transformation strategies as being driven by board level decisions or executive decisions, um operational efficiencies from, you know, management level or new products from management level. But in actual fact, a lot of it is demand driven, right? So the bottom up approach where you just have developers trying to solve problems on a day to day basis, right? So where is digital transformation realistically occurring in an organization? Because we think of it being applied as a strategic move and decision being driven by top level management? But is that actually what's occurring or is it just people solving problems and using APIs to solve problems?

James Higginbotham

Yeah, it's, it's definitely a a bit of both. I mean, we we see the need for the executive buy in and the executive vision and execution from the top down to really drive, drive the the budgets necessary to take a large organization. We'll say 25,000 developers, you know, 2 3000 scrum teams and try to shift them over to a new direction and transform themselves that takes kind of that top down and complete buy in and, and large budgets. But they don't often start that way from what I've seen. I sometimes see organizations that you'll see things like solution architects and mid-level managers that really understand the value of reusing APIs many of those folks that are doing that saw the last um the last couple of decades with our service oriented architectures and soap and things like that and they saw how you could either build services or in this case APIs that were reusable and that we could leverage time and again. And then they also sold the one off integrations that came up from all the different ecommerce solutions and supply chain solutions that required us to rewrite and rewrite new and different integrations every time. So I think there's, there's kind of a combination and in the case of kind of the bottom up or mid up sort of approach, they'll oftentimes take something that has already been budgeted and is in flight and they'll start to use that to demonstrate the values that can exist and the approaches and the shift in mindset that can exist and then they take that upward to get future follow up and, and buy in from the executives to get the larger budgets they need, which means they have to be a little scrappy in the beginning And until they can get everything officially adopted and, and, and at a larger scale

David Brown

And they're much more likely to get by and at being able to demonstrate something. Right.

James Higginbotham

Absolutely. Yeah. A, a demo is worth 1000 words. And if you can take,, if you can take something instead of talking about it and, and powerpoint sliding someone to death and you can come up with a vision, have a team to be able to execute that either in a skunk works way or with some sort of buy in. But in a very scoped and focused way for something you're already gonna deliver, that can help kind of overcome some of those um concerns and demonstrate value early on.

David Brown

Particularly if you can render some pretty chats as well.

James Higginbotham

Absolutely. Yeah, if you, if you can go and say, well, look, we, we built this API and this was actually fundamental in this feature for our mobile app and we're running millions of API requests per day through this or per hour through this or whatever it is that wow, that goes heaps into really being able to transform somebody's mind and say, well, if we can do it here, we can do it in the rest of the organization.

David Brown

Totally. Let's talk about the technology considerations from a developer's perspective. So what are some of the you you've written about some of the essential considerations for developers when creating API specifications run us through those?

James Higginbotham

Yeah. So, so a lot of what I like to do is encourage developers to, to slow down a little bit. We, we are always eager and I'm a developer as well to, to kind of get in and start writing code. And while code is really absolutely fun and exciting to build and see it all come together, it's oftentimes important for us to step back a bit. And so I encourage development teams to slow down and align, to find and design their API aligning meaning, making sure that we understand what the APIs need to do, the outcomes it needs to produce, making sure that we're breaking down those assumptions that every line of our code is going to have wrapped into it. And then once we've aligned on what it needs to do, then we can spend some time defining what the API is actually gonna look like the operations, it's gonna deliver um Perhaps its name, its scope, its purpose kind of at a high level. It's what I call the API model. It's, it doesn't get into the specifics of how it's gonna work with the protocol details, but it is going to get into the specifics of what it needs to do. And then finally, the design phase helps us take that model and apply it either as a rest API GR PC graph QL, whatever the case is, maybe it's a combination where we use rest to make mod modifications and retrieve individual resources and then we use graph QL for really robust query language support. So align, define and design helps developers to kind of go from what we need to build, to how to build it in a in a very simple and approachable way that's rapid fire to get us into the code as quick as possible. But to ensure that we're not having to throw a lot of that code away and scramble at the last minute to try to make a change because we misunderstood something that we could have understood from the beginning.

David Brown

Is there some sort of formal procedure they can follow in terms of getting requirements liaising with stakeholders, do you recommend like some sort of formal workbook or procedure to, to log and document the requirements and to execute, mock, mock the solution? Um What do you suggest?

James Higginbotham

Yeah, I do. So I've been developing a process for some time around this kind of align to find design approach. And I've, I've documented in a book that I have out currently. It's a practical approach to API design is the title of it. You can, you can find it at the APIDesignbook.com and it sort of lays out a process of how to go from user stories or job stories into kind of what your API needs to do the activities and the steps and kind of helps you break that down. And, and it's a collaborative approach. So you want to get subject matter experts in the room, your domain experts and pull it all together and, and start to work together even if they don't understand. Htp some of your subject matter experts aren't that technical to that extreme or to that depth. That's OK to work through the process to figure out what, what you need to deliver and then to arrive at that design that you need to, to deliver to the marketplace to meet what's needed. Now, it, it assumes that you're never gonna nail that design right off the bat. It assumes that you're probably gonna have to make some changes, but it helps you hone in on that design early and then you can use mocking or prototyping styles to turn your design into some sort of a small little mock version that you can use to both test out the design and to accelerate the development testing and delivery of that API since, you know, front end developers can use that mock to start tying it into their front end code while the backend developers may be building the full API implementation, integrating with legacy systems, multiple data source and so on. So that, that's what I really recommend. It's really approachable, really lightweight. Um And it, and it's really driven and just kind of breaking things down into steps and then turning those into operations for our APIs

David Brown

And there was an important point that you mentioned as well is don't expect to get it right? The first time you are, it is an iterative process, right? You're gonna need to keep going through the process to improve and, and, and you know, you can update APIs at a time.

James Higginbotham

Absolutely. So, I like to remind people API are forever. Once you have your first production integration, it's gonna be very hard to, to to encourage that integration, that team that integrated that at your API to make a code change. So you have to learn to design evolvable APIs and, and you can, so you can add to them, but you can't make fundamental changes. That doesn't mean however that we can't go through a design process where we establish a life cycle for that design and we can allow people to integrate with it early that are willing to absorb some of those changes that may need to be made as you discover more things. So it's a, it's about setting expectations and having maybe early adopters be willing to take on a little bit of that extra burden to get access to your API first. And then once it solidifies it, it's forever, you can add to it, you can improve it, but you're not gonna be able to make some fundamental changes. So going through a design process helps you kind of get there quicker and then put it in front of people and, and allow them to try working with either the mock or the eventual evolving implementation and, and, and make those changes while you can before everything is locked into place and you're only gonna be able to add to it but not make some naming changes or something that, that you decide you don't like down the road.

David Brown

And when, and how do you decide on the implementation, the choice between R and graphQL and Gr PC?

James Higginbotham

Yeah. II I like to do it. So in that align defined design process that define helps us understand what the API needs to do. And it's at that point where we start to transition and design that we can start asking some questions who's gonna use it? What, what are my developers capable of or comfortable with? Um I if it's, you know, what kind of performance needs do we have? So if we have, if we need high performance, GR PC is usually really good selection because it's taking advantage of HTP two and some of the improvements there. And, and some of the compression and other things that exist underneath the covers. You can have more persistent connections with GR PC so that when you make a new connection or when you make a new request, the connection already exists, If you want and you can have bidirectional communication and those kinds of things. So it's during that time where we have to determine what's the, the best way to do it, who's going to use it? How are they going to use it? And it's a lot of discussions I get into with, particularly with developers that are very strict about, well, this is the way I do things and this is the only way I do things they'll say, well, which one's best? And my answer is the, the, the best API style is the one that's best for your consumer, not for the provider, the provider's job is to make the consumer's job as easy as possible. So we have to put ourselves in the mindset of who's going to use the API and figure out what API style they might need. The other thing is to think about and not versus so in the improv, improvisational world, we are taught the yes and approach to improv the idea that we take something that somebody threw at us or said to us or something. And we add to it and with the API style selection, it's the same thing. Um You know, I find a lot of organizations still use rest, but they're using rest and GR PC or they're exploring GR PC and Graph QL. And they're also looking at async API styles like web hooks and servers and events and web sockets to be able to push things back and forth so that it's not just strictly request response. So it's during that kind of a defined design transition where we start to ask ourselves, OK, how are people going to use this now that we know what it needs to do, how are they going to use it and what styles are right for these operations that we're gonna bring forth?

David Brown

There's a lot of technical decisions you've just mentioned there, right? And design styles and implementation options. If you're new to APIs as a developer, where do you start?

James Higginbotham

I recommend first of all, um starting with rest and what I do is I say start with rest but really start with HTP. Get really good at http. Understand it now, you don't have to have like some of my friends do have memorized RFC S practically but understand what HTP can do because I oftentimes see teams building APIs or struggling to consume APIs struggle with just using HDP. We use it every day in our browser, but we sometimes forget about all of the, the amazing things built into the protocol from caching and, and entity tags and you know, detecting when something has changed and all kinds of things that are actually built into HTP. And then once you understand that, then you can get a little bit more advanced and you can start seeing well, how can I optimize my queries? Maybe graph QL will do that for me. How can I optimize my interactions or get bidirectional communication or avoid all the boilerplate code. Well, GR PC can help me with that. So, so getting in getting a good understanding of, of some HTP basics and what it can do will also save you time in trying to reim implement what's already in HTP inside your API thereby making it longer to get to market and perhaps more difficult to consume by someone else down the road. So just starting with HTP fundamentals is, is, is a key factor if you're going to look for anything to start and then rest based APIs kind of come with their own sort of myths and misunderstandings. So just understanding how to use HTP, how to use the HTP methods, the response codes, um serializing things and Jason maybe looking at some different media types or starting with something like Jason API, that's kind of there and opinionated and ready, ready to go can help jumpstart that process so that you don't have to to take it all in at once. You can kind of do it a little bit at a time

David Brown

And the tool sets for rest are also readily available and reasonably mature.

James Higginbotham

Now they are um that that's the one thing that I really find organizations challenged with is they sometimes want to adopt things like GR PC or graph QL. And while tools for those worlds are emerging, there's still, there's just not as many of them. And and so the default tends to be rest in http. So any tool you pick up can make a get request, a post request, whatever you need. But to formulate the right standard request response formats for GR PC or for graph QL is a lot more difficult and, and there are tools coming out to make it easier to make it better. But if you're just starting out and you want to be able to leverage some of those tools to get you going faster to keep you from having to sit at the command line and give you kind of a graphical interface to it, almost like an ID to work with an API. Then the rest is a really a great way to go to get started.

David Brown

Even as a software company, we took the same approach. Rest is where we got started. We've totally nailed rest. but graph QL and GR PC are well under development. And so we'll come out with tool sets for those which are equally accessible to developers as our rest tool set. So as I'm sure all of the other software publishers in this space are also doing so the the tool sets will evolve the rest. They are great. I'm guessing there was also a opportunity for you there to, to plug your own educational material and book for, for, for the developers to learn this sort of stuff. So tell us about your, your books and I think you, you also have one under under development as well.

James Higginbotham

I do. Yeah. So I mentioned earlier the APIdesignbook.com will take you over to Lean Pub and Keith Casey and I put together that book about five or six years ago and it, and it has a lot of this process that I mentioned. So it's there and that was used to build our API training courses that we've delivered to thousands and thousands of people all over the world. So we go in and we, we start from what is an API and how does H DB work all the way up to consuming APIs and designing them as part of our training. So if you, if you get the book, then you have kind of the basics of what we deliver in training. We've done that on site and we also do it virtually. but I'm also working on the new books. So um Vaughn Vernon who's known well in the Domain Driven Design Space has a signature series from Addison Wesley. And we're working on an API design book that really takes kind of what Keith and I started and it blends in some of the domain driven design techniques that are out there as well as some other insights that have gained by, by helping teams design APIs and train them how to scale. design processes as part of the Digital transformation. So we're, we're taking that kind of aligned design or align, defined design process and expand it a little bit talking about how to do things like event storming and job stories and how to design evolvable APIs and how to go from a model to rest R PC graph QL documentation and testing strategies and how to scale a manager API program all in one book. So I'm looking, I'm, I'm currently putting that together now and it should be out sometime spring of 2021. So really excited about that as well.

David Brown

That's exciting big project. How, how long out of interest, how long does it take you to write a book like that?

James Higginbotham

Yeah. So Keith, it took Keith and I quite a few months, but we did a lean approach to the first book we released. So we would release a few chapters at a time and the sooner you bought the book, the lower the price was and, and then we just bump the price up with each subsequent release. So if you got in early and supported us, then then you got a little bit of a discount. That's how we did the first one and, and we kind of spread that over a couple of years because we would be out on the road training quite often. And so it would take a little while to build it. So it took a couple of years really to build that one, this new book,, I'm really hoping to have wrapped up in the next couple of months and then of course, with the editorial review and all the other steps,, to go to print,, that'll bring it sometime in, in probably spring,,, of next year. So, so it takes a little while, um, even with as many times as I've taught the material, it takes a little while to get it into print form and really get it in a way that everybody can, can understand it with great numbers of examples to, to demonstrate different um success and, and failure paths through the process of, of you know, growing your API program and and helping to design them and then scale that, that process out.

David Brown

Brilliant, looking forward to seeing it and maybe we can get you back when it's out and we can talk about some of the concepts which you mentioned in there.

James Higginbotham

That'd be great.

David Brown

And James, how can the listeners follow you on social media?

James Higginbotham

Yeah. So I'm on Twitter at LaunchAny. You can also go to launchany.com. And also if you go to APIdeveloperweekly.com, you can sign up for a weekly hand curated newsletter. So I pull together a lot of really interesting articles every week that cover api details, kind of the business side, the technical side as well as some tangentially related technology news as well.

David Brown

Awesome James. Thank you so much for your time. It's been a pleasure having you on the show and as I say, we look forward to getting you back when that new book is published and gaining your insights.

James Higginbotham

All right. Thanks a lot. Looking forward to it.

Kevin Montalbo

All right. That's a wrap for this episode. Thank you very much James Higginbotham for being with us to our listeners. What did you think of this episode? Is your organization taking a bottom up or a top down approach to digital transformation? How are you implementing your API strategy? Let us know in the comments from the podcast platform you're listening to. Also, please visit our website at www.torocloud.com for our blogs and our products. You can also find us on social media, Facebook, LinkedIn, Youtube, Twitter and Instagram, talk to us there because we listen, just look for Toro Cloud again. Thank you very much for listening to us today. This has been James Higginbotham, David Brown and Kevin Montalbo at your service for Coding over cocktails.