Kevin Montalbo

APIs represent a contract between a system and its consumers. They have evolved from pure system to system integration to evolve many other stakeholders making it very important that careful consideration be given to an APIs design. Our guest for this episode is known as the API handyman who shares with us how to design APIs by diving into four layers of consistency, how to make your APIs more discoverable, defining and setting the boundaries between an API gateway and API implementation and choosing the right API architecture and technology for the right problem.

Kevin Montalbo

Welcome to episode 46 of the Coding over cocktails podcast. My name is Kevin Montalbo. Joining us from Sydney Australia is Toro Cloud CEO and founder David Brown. What's up? David 

David Brown

Good day Kevin.

Kevin Montalbo

All our guest for today is a senior API architect at NATA based in Paris France. His daily job involves helping people understand what APIs are, why they matter and how to do them. Known as the API handyman. He authored the book, the design of web APIs which teaches readers how to gather requirements, balanced business and technical goals and how to adopt a consumer first mindset while teaching effective practices using numerous interesting examples. I'm very happy to announce as well that we're giving away a copy of the book to one lucky listener. So thanks to our friends at Manning publications, stick around until the end of the show to find out more about our giveaway without further ado joining us for a round of cocktails is Arnold Lore. Hi, Arnold. Welcome to the show.

Arnaud Lauret

Hi. Thank you. Hi.

Kevin Montalbo

Alright. So let's dive right in, in your daily job at the Texas. You regularly help people and consult on API design. What are the main challenges that organizations are facing when designing APIs?

Arnaud Lauret

There are actually many changes we can, we could talk about it for days. So I think we don't have days actually for this podcast that um let's let's pick one of the topics I really like is actually APIs our application programming interfaces that there are nowadays, there are far more than that because they are not just meant to connect pieces of software together. That's their primary intent, obviously. But now as at an organization, a team or business units, a company, a group, people must understand that APIs are a new way of creating business, a new way of making your company work better. And so actually, the main challenge regarding API design is to make people understand that they have to design better APIs because UN until a few years ago, we most companies let the it people alone design APIs because they were just technical connectors. But now when we design them, we need to gather the it people, people with business knowledge, people who understand what means developer experience to ensure the creation of APIs that do the job that, that do something that is useful for people outside your organization, it can be your team, your business unit or your group and to ensure that it can be used in various use cases. Because as as an example, if people start a project for maybe reworking their website, their mobile application, they can be tempted to create an API that is really tight too close to the user interface that are created and they instead of doing that, they should rethink. OK. Yes, we want to do a mobile application or a website. But what are the functions we want to put in them from a business perspective? Do we want to create bank accounts to wire transfer. Yes, I'm calling from the banking industry. and you have to think to that first and afterwards you will use them in your mobile application or website. But you will also be able to provide these features to to partners or to fulfill of use cases. So that's really for me, the main challenge, it's changing mindsets from for the it people to understand that they have to do more. And sometimes you, you just have to explain them and they get the thing very quickly. But some, some of the time you really have to work a lot with people. because sometimes the broke out tends to be too focused on really technical issue and you have to talk to business to make them understand that you have to invest in APIs.

David Brown

It's an interesting perspective because you're right. APIs came from system to system integration primarily initially. So, and now we've brought people into the picture and stakeholders and business users and end users and departments and all these requirements and different devices and the like and um and, and, and they need to liaise with still with those um technical people are gonna be implementing the API and there's gonna be a whole bunch of communication going on. And I guess technical people aren't known for their communication skills. They prefer to sit in front of a computer coding away and and, and and, and so I can imagine you, you met, you brought up this when we asked you the question, the problems, the challenges that organizations face designing APIs. This is the one you, you said is a particular favorite of yours. I imagine introducing that people aspect to the API design process is it just makes it that much more complicated. Um You've now in your book, you the design of web APIs, you've talked about designing a predictable API and it should be consistent, adaptable and discoverable. I'd like to dive in more into that consistency aspect. What do you mean by making it consistent?

Arnaud Lauret

So let's take an example. Um If on a, if on a washing machine, you see you have AAA button on the control panel with an icon which is a triangle going to the right. Yeah. Do you know what it means?

David Brown

Fast forward,

Arnaud Lauret

fast or play or? Yeah, if people are able to understand that it means OK, starts what this device should do. And we know that because we have seen this burden elsewhere, we have seen this burden especially on media devices or people who are as old as me on workman or cassette players and whatever. Yeah, but this is actually a consistent design. It's a design that looks like something you already have seen and with consistent design, you are, you are ensuring that people will understand quickly how to use your API what it does and so on because you are replicating something else.

David Brown

So what are the, what are the attributes of a consistent design and what are you replicating?

Arnaud Lauret

Yeah. So actually there are four levels of consistency. at minimum, your API should be consistent with itself. It means that if you decided that a customer ID was a string name, account number. You should ensure that everywhere in your API where you are, you, you need to retrieve that information or you need to provide it, it is name and type that way. Yeah, if you change names, people will be lost, we people won't be able to make the connection between the values operations, the values that are noted in your at an upper level, your API should be consistent inside your let's say your organization, it can be your team, it can be a business unit, it can be your group. So your API all the APIs constructing your API surface must share common features like how you handle security, how you manage error, how you represent versioning everything you can that way once you have learned to use the first API in your organization, when you shift to the next one, even if the business is completely different, at least you have a common base, you don't have to relearn everything Yeah. Next level is trying to create APIs that are consistent with your business domain. For example, in the banking industry, there is standard which is called ISO 20,022. It's a terrible and ugly standard from my designer's perspective, but it's a standard that you should care about. If you are, if you want to create APIs that will be used by corporate banking software because this software actually understand pretty well is 0 20,022 and that can meet standard in travel in health. So always take a look at that to ensure that what you are building conforms to the standards. That way your IP L will be easier to use. And last level is trying to make APIs like every other people around the world are doing that. So do not try to be too different. That mean, yeah. that means for example, I'm trying to, for example, um yeah, it could be, there are two ways of seeing that first, there are standards that are not domain related. For example, if you want to represent countries or currencies, there are io codes for that that everyone use. So if you use them, your API will be interoperable. And there are also if you are doing raised APIs, you are, you should follow the HTTP protocol behavior and semantic. So if you are using the gate http method. Don't dare to use it to delete something, for example, because it's against the protocol. And there are also common practices. For example, if you want to return data in a list, there are various way of doing that, but there are ways that are more common than others. For example, in my company, we decided to return list as an object containing a property name, items containing the list, either this list contains bananas or carrots or whatever. There are also common patterns regarding the definition of your wrestle stuff. So try to see how people actually do them and replicate that because if you do like some of our APIs especially famous ones, people who have used them when they come and use your APIs, they will feel just like home. So

David Brown

that's why I guess that principle, I'm sorry, I'm sorry, Anna, I guess that principle applies regardless of the type of API whether it be restful or graphql, you can still make them consistent with each other in terms of those principles you just outlined totally. OK. So is ensuring your API is discoverable, um which is one of those three meanings of a predictable API, is it as simple as ensuring you have some good do human readable documentation and API description format such as open API for machine readable formats. And and there may be a postman collection free HTTP client. Is that what you're talking about discoverability or is it more than that?

Arnaud Lauret

Yeah, there is actually far more than that. Because first very simply OK, you, you made some documentation, you created a postman correction. But if nobody knows how to find it, if nobody knows your API actually exist, that document is worth nothing. So actually you need to have a place some people put into portal developer, portal, whatever or an API registry or yeah, but you need to have a place where people can find all of your APIs. And so you will put in there your your documentation, your open API files and postman collection. regarding documentation reference documentation, which is usually based on open API file depending on your context. They may be not enough for internal API, they can be enough. But when you try, when you provide APIs to the outside world, you need use case documentation that actually explain what you can do with API because the reference documentation is a kind of list of ingredients that you need the recipe to actually use those ingredients. And on top of that, you need a very short and business focused description of your API that will help people to understand how this API can solve that problem. So that's for the usual way of how people think about discoverability. But there is something else that you need to think about. It's when you when you're actually using the API you can make it discoverable in the sense of, ok, I, I have seen the documentation. I, I know that my first request could be to read a list of customers and I do get slash customers. But if inside the restaurant you provide information to explain. Ok, now you have a list of customers. you can you can read a customer direct a customer or whatever maybe create them B got. If you provide inside the response, the links to other operations like you would do in a website that make your API actually ver because some people don't like to read documentation, they just want to start to use the API. And so if you give them directly, what are the next possible operation? It could be interesting

David Brown

in the response itself, which was actually the original definition of a restful API. Right.

Arnaud Lauret

Exactly. And even if you don't want to do rest APIs making APIs consistent, make them discoverable because OK, if I see a get slash sho I can guess, but I can read a user by sending a guest slash user slash user ID, I can guess, but I can probably delete the user by trying to send a delete slash user slash user id because this API is this design is consistent with common practices and this design also actually respects the HTTP protocol. So making APR discoverable means, make them easy to find. But also when you use them, you need to provide by any means information of how people can use it, how people may find over operation and other features. Interesting.

David Brown

Um What, what is the role? There's some confusion, I think sometimes in, in terms of the role of an API gateway in the whole serving of an API. can you, we're gonna dive into the API gateway and the implementation of it? So first of all, can you just define what is the role of the API gateway?

Arnaud Lauret

in my opinion, and the API gets where is only supposed to expose API in a secure way. That means it's a kind of proxy, but it's a little bit more than the proxy. It may handle high security level such as OK, our as an API gateway, I will ensure that only this application will be able to consume this API. Hm I may also say, OK, this application can consume this API, but it can only use a subset of all of the available operations. Hm. An API gateway may also provide features such as throttling to say, OK, you can use the API but you can only do, let's say 10 calls per minute. Yes, that is for me, the role of the API gateway. If you put something else in that inside it, sometimes it can be interesting to do that. But most of the time you are for me working on the domain of what should be inside the implementation. And

David Brown

this is what I wanted to get to because I know you have some thoughts on this matter in terms of where the boundaries should lie between the API gateway and the implementation. So because I know a lot of API gateways will allow you to do transformations and and caching and perhaps throughout the lookups and all these sort of various things and it's the, the lines are starting to blur between the gateway and the implementation of the service being proxy. So, so can we dive into that? What, what, where should those boundaries lie? And what's the reasons for that?

Arnaud Lauret

In my opinion, what is so what, what is tied to business, business rules shouldn't me must not be in the gateway. So if you need for practical purpose to say, OK, we are returning that as Jason, but it could be interesting to return them as CS V OK? It could be just the technical transformation without business involved, maybe it could be put in the gateway. But if you are starting to say, OK, now if the user is of type admin, we will do this or do that and call the service and call two operations instead of one take some data, h them, stop them. And that's business rules and that must not be in the API get where it must be inside the implementation. So for me, the separation is really OK. If it's high level security, if it's as a general concern, just technical concern and so on, it can work. But the implementation must be the owner of all. What is tied to the business domain, the business world, the business intelligence.

David Brown

And what is the reason for that? And what's the danger of if, if you have logic in both places

Arnaud Lauret

For what I've seen, I, I would say, remember what I earned with EB I worked in companies having U SBS, we have put a lot of business logic in there and it was really a mess because it was a prior technology needing really specific skills to develop in that. And when we wanted to get rid of that, it was really, really, really hard. But if we had stick to say, OK, we are creating our application, we are there the whole, the business logic and so on. And on top of that, we had maybe a, a security layer doing all that stuff. If you want to change it, we can. So based on my experience developing business logic inside gateways or ESD for that matter requires are actually usually specific skills you need to understand the product, how it works, how you can develop on it. Sometimes you don't have all the features you would have using regular programming language such as java.net or whatever. You, you, you don't have unit tests. You don't have you can create um put in place C I CD easily and so on, you can test in isolation. And so that's why I prefer to keep the business logic outside of the API gateway.

David Brown

Yeah, because like you said, it is like an ESB the API gateway is almost mimicking that ESB type environment where you have smart pipes, you know, so as opposed to the, you know, um smart endpoints and, and and dumb pipes, right? So it's um I totally get it as well. As of course, you mentioned the development process you have around the deployment process to those to the application logic as well, makes a lot of sense. Um So where does the, so you mentioned the security should obviously that proxy concept and the authentication of the security is lying in the API gateway? But will you have circumstances where there is some level of security in the, the service itself, the business logic as well? So where should the boundaries lie between the two?

Arnaud Lauret

Yeah, so definitely putting an API gateway in front of your implementations, whatever it is, a good old monolith or macro services or whatever the API A gateway will only provide high level security. As I said, it will ensure that only registered consumers can use API in some sort of way. So let's say I grant access to my users API to some consumer. I just say OK, this API is allowed to use my user's API. And inside this API, the consumer can use all of the functions, let's say, create users, delete users, search user of the student. So if there is an end user using this application, let's say a no me and I trigger an API call that is supposed to delete a user. We get where we say, OK, who is making this call? It's application but it is allowed to use the CP I and this specific operation. So the coal is sent then to the implementation, the implementation needs to ensure that the coal comes from a known gateway obviously. And it needs to ensure that actually the person me who is requesting to delete user 1234 is actually allowed to do that. It means that OK, user 1234 is someone who is in my team. So I can delete this user because this person is no more in my team. But if it's not the case, the implementation to say no, no no cannot delete this specific user. You can delete over users but not this one because it did not belong is in, in, in its in his data perimeter. So that's what I call application local security. And most of the time actually don't really get it at first time because they think, OK, we have an API gateway, it will handle all of the security. But no, because the API gateway don't know the business logic. The A P A gateway don't know who is allowed to delete a specific user or not, who is allowed to read a specific user or not. This is the job of the implementation. And that's really important because being able to, to access data that you are not allowed to is actually the API security 101 failure. We see every two or three weeks. For instance, there was I don't remember the name of the application but it was an application on IO si think that allow users to record phone calls. And so the phone calls were stalled somewhere and to get phone calls, all you all you have to do is sending a request. Basically telling, OK, give me the phone calls for phone number, whatever. OK? And what obviously if hackers look at what is doing the application when they see that? What is the first move they say, OK, what happened if I put the phone number of someone else who may be using the service? And obviously, in that case, they were able to retrieve the conversation because the implementation was not checking. But ok, I get, I get give me the, the, the conversation of the phone number but this phone number does not match with the user who is actually connected.

David Brown

Mm That, that example you just gave there. I mean that would,  it's a record level permission but it's at like an account level ID. So I can imagine some sort of database saying sort of account ID and they're associated with some sort of token being passed on by the API gateway are allowed to read and write to these records in our database. But I'm imagining particularly in the finance industry which in which you work in a lot, it gets a lot more finely grained than that and it would get quite complex when you're down to the record level of, of who's allowed to read and write to records. Are there any particular, are there, are there frameworks around like security type frameworks around to, to do this sort of stuff or do you really have to spin this stuff yourself?

Arnaud Lauret

No, I in as far as I know, no, you can, for example, in my, how many, when a consumer is actually making an API code on the behalf of someone first the gateway and delegate the authentication of the end user to an identity provider using some and then we get we know who is the user, what are his roles regarding this application? But it's only a fraction of what is actually needed to handle application level security. But at least the gateway transmits all this information to the implementation inside a token AJ G BT token. And that gives some hints for the implementation to do its job. But most of the time the implementation needs to know. OK. the users are stored in this table and we know how to make the joint with the over table to ensure that this user is actually allowed to do that. And that really depends on the business you are doing. what is your implementation, how it works and so on. So actually there is no magic frameworks that will handle all the possibilities. Wire frameworks can handle rule based security and so on. But sometimes you have to go to prevent that. For example, let's say that a bank councilor try to make a wire transfer on the behalf of the owner of a bank account. Um This person can be allowed to make wire transfer from €100 or dollars to $1000 but not beyond. Yes. And that is that is actually application level security. Yes. And that cannot be handled by a kind of generic role based access mechanism. You have to actually write your own customer.

David Brown

I was hoping you'd say yes, go here and you can buy one of those off the shelf, but sadly not. Um, it's interesting, like a lot of the stuff we're talking about in terms of design and stuff, which is still very topical today. I mean, we've been working with APIs for several years now and, and people are still struggling with a lot of these concepts. Um, I'm guessing at some point in time, a PS themselves will just become hardware, their end points and mechanisms and, and all this stuff which in these foundation days will seem, you know, old hat. What are you seeing in the space? Like we have rest and graph QL and AC KP I and GR PC some architecture is trending more than others, is it? Um what do you, what do you see happening in the API space?

Arnaud Lauret

Actually, yes, there are new or not so new technology because Cr PC has been there for a long time, graphical stuff to be there also for a few years. And I think there will always be new ways of making software talking to each other. very important for me and especially when people say come to me and say, OK, we don't want to do a rest api we want to do with fancy new ways of doing API because it's brand new and we want to do brand new things and say, OK, but why are you picking this particular technology? What problem are you trying to solve that can be solved? And can it be actually solved with this particular technology? And, and as an architect, I try to moderate people and say, OK, it's not because it's brand new but it is actually the solution. It's not because you like it but you need to use it always. So regarding all these different ways of doing API si my family said, OK, what is the problem we are trying to solve? And we must choose the adapted solution. If the adaptive solution is doing a good or rest API we will do a good or rest API and that has some advantage because people are used to them, people can use them quickly and they are easier to implement. But if we need to do, I think KP I because we are inside our company, we want to um to use C A fa for actual really good reasons. Hm We'll do that. And if we need to let's say, creates a back end for front end for mobile application and we need to optimize network codes and so on, maybe you can do graph QR. But the problem with graph Q is it can be really tricky. you can do crazy thing with it and especially you can just destroy your system because you consumer can send really complex requests. And if you don't have mechanism that prevent them to do that, they will send those basic requests and your system will just go down or you will have to pay 300 percent more than what you expected on your Adobe us or Google cloud provider. So, yeah, there are new ways of doing things and there will always be. But in my opinion, and actually, it's not really an opinion for me, it's a fact, choose the adapted tool to your context. And on top of that, all the design principles, if you have learned actual design principles and not just the technology, you will be able to apply those principles on rest on graph QL on GL PC or NCKP, I so make your api whatever they are consistent, usable and so on and everything would be OK, but just be sure to choose to choose the right technology for what you want to do. Not just because it's fun, it's new and, Netflix is doing that. Yeah, but actually, well, not Netflix

David Brown

sensible advice. And thank you very much for that. You make it all sound very accessible and easy to understand. Um how can our listeners learn more about what you're writing about and discover things like the design of a web APIs the book you've published with Manning.

Arnaud Lauret

They can come to to my blog I since March, I started to write to write more frequently post a week. So I learn more experience, my fellows, my Dots. And so people can learn from, from that. They can follow me on Twitter with handle Api handyman. And obviously on Twitter. And on my blog, people can find a link to by by my book, but actually, they can read it online on the money website. You can read it freely. I think it's five minutes a day. You, you, you can actually read it. So you can see what is actually speaking of. So you can test it before buying it. And and actually if you like the book or if you didn't like it, I'd like to have feedback from readers because I may work on a new version of the book if I'm permits. So having feedback could be interesting.

David Brown

I did that myself. Actually, I went to the Mannings and I was reading section of the book and it starts to obfuscate the, the, the, the text after you've been reading it for a while. It, it's quite cool.

Arnaud Lauret

Yeah. And the, the figures, you, you can still read the figures afterwards.

David Brown

Yes.

Arnaud Lauret

Yeah. So, sometimes they are interesting just by themselves

David Brown

and of course, your blogger is at a p handyman dot IO as well. Thank you for joining us today.

Arnaud Lauret

Thank you for inviting me.

Kevin Montalbo

Hey listeners, hope you had a wonderful time listening to that episode. For those who stuck around. We've got a special surprise for you. We're giving away a digital copy of our no Laureates book, The Design of Web APIs courtesy of Manning publications. Simply follow us on Twitter at Toro Cloud like and retweet our contest post which is pinned on the top of our Twitter feed. One random winner will be chosen and contacted by DMS. Good luck. So what did you think of that podcast episode? Let us know in the comments section from the podcast platform you're listening to. Also please visit our website at www.torocloud.com for a transcript of this episode as well as our blogs and our products. We're also on social media, Facebook, linkedin, youtube, Twitter and Instagram, talk to us there because we listen, just look for Toro Cloud on behalf of the team here. Thank you very much for listening to us today. This has been Kevin Montalbo for coding over cocktails. Cheers.