Contract Driven Development – Deploying your Microservices Independently without Integration Testing

Hello and welcome to this talk about contract driven development. We’re going to be talking about deploying your microservices independently without integration testing. My name is Hari Krishnan. I’m a consultant and a coach. I help both unicorn startups and enterprises with their transformation activities.

 

My interests include distributed systems and high performance application architectures and these are some of the conferences I speak often at. Let’s start off with some context setting on the topic. Why do we really need contract driven development? Let’s say we’re building a mobile application that requests data from a backend and then displays the same. The application requesting the data is the consumer and the application responding with the data.

 

Let’s call it the provider. Now, with the terminology out of the way, let’s look at how we would go about building the consumer. Now, one way is to wait for the dependency, which is the actual provider application, to become available as a reference so that then we can build it out against it. However, this is not very productive because it’s a sequential style of development. So usual practice would be to stand up a mock server to emulate the provider so that the consumer application development can progress independent of whether the provider itself is actually built or not.

 

While this looks good on paper, there is a fundamental flaw with this approach which is the mock server need not be truly representative of the real provider. By which I mean as a consumer application developer, I may wrongly assume that I could send the ID as a string wherein the actual provider application may be expecting an integer, and likewise the provider may be responding with the name and SKU of the product. While I may be wrongly assuming that I’d be getting the name and the price, now, what does it all lead to? When we deploy these applications together, there is going to be broken integration and such issues are not really easy to find on our local environment. For the consumer application developer like we saw and also in the continuous integration environment because we might pretty much be leveraging the same handrolled stub, right?

 

And for the provider there is no emulation of the consumer. So thereby even provider application development is also happening in isolation. And when you put these two applications together in an environment such as integration testing, that’s when you realize there is a compatibility issue. Now this first of all compromises your integration testing environment and worse, if at all you have a leak from integration testing and these issues do happen to make it to production ends up in creating incidents which we do not like, right? And because these issues are usually found on the right hand side much later in the cycle, they’re also quite expensive to fix.

 

Wouldn’t it be nice to prevent such incidents from happening altogether and to be able to identify such compatibility issues very much on the left hand side? What do we want to achieve. We’d like to shift left and identify compatibility issues without really integration testing applications. Now, one way of solving this problem could be to reduce the ambiguity in communication between the teams so that there are no assumptions which can lead to problems, right? So putting it down in an industry standard specification such as Open API or WSDL is a good way of governing both application development on the consumer side and provider side, so that you have the two applications talking to each other well when they are deployed together.

 

However, do you think just having an API specification standard or adopting one solves your problem? That we are trying to look for a solution? Not necessarily. API specifications in themselves are just describing what the API looks like, right? It is up to us how we enforce these API specifications on the application development itself.

 

And that’s where the difference is between API specifications and executable contracts. What we’d like to achieve is to be able to take API specifications and treat them like executable contracts. Now that’s exactly what Specmatic is able to do. Specmatic is a tool which can read your Open API specification or WSDL specification and then convert that and make it available as a contractor stub server for your consumer application, so that you can independently continue on the development of the consumer application. This stub is significantly different in comparison to handrolled stubs because it is truly representative of the provider because it is based off of the mutually agreed upon API specification.

 

Now, we have to keep the equation balanced on the provider side also. So for which Specmatic is able to convert your API specification into contract as test and make sure that it is verifying that the provider is adhering to the contract also. Now with this, you are able to independently develop both consumer and provider applications while still being sure that when you deploy them together, they will work well with each other. Now, let’s take a look at a live demo of what this will look like on the provider side. Let’s say I have this API specification and then I have the system under test, which is the provider itself.

 

Now, I’d like to be able to test if the provider application is in line with the API specification. What I’m going to do is take Specmatic, leverage the API specification as contract as test, and then verify if the provider is able to match up. So for this, I’m quickly going to show you an example of an API specification. I have a typical ecommerce like OpenAPI specification here. It’s got a bunch of parts with products and orders.

 

And on each of these parts it’s got multiple operations to support crud operations for a product or an order. And it’s also detailing out the schema in terms of what are the type of values you can support and whatnot. So that’s typically this API specification. And then I also have this application here which says the team is claiming that they have completed building this application and it should be able to support all the operations that we saw in the API specification itself. Now how as a person who is supposed to be responsible for this application going live in production, that this is indeed adhering to the specification we saw earlier.

 

So what I’m going to do now is leverage Specmatic to take the API specification and run it as a test. So what I have here as a basic setup is I’m using Specmatics JUnit support while I am using the JUnit support within a Java springboard application, Kotlin application, Specmatic itself is language agnostic and platform agnostic. You could use it from a command line also. It’s just an executable. But for the purpose of convenience, I have this JUnit support extension here and then some plumbing to start and stop the app and the coordinates of where the app itself is running so that Specmatic can hit it.

 

Now how does Specmatic know where to pull the API specification from? That’s where this JSON configuration comes in. I’ve directly pointed it to one of the Git repositories where I’m hosting the API specification which we saw on this wagon editor. Now with this, let’s actually run the test. And what you will notice is a bunch of tests getting generated and run against your application without you having to write any line of code.

 

And this is all I have done. And you notice we got twelve tests, right? That’s money for nothing and tests for free. Who doesn’t want free tests, right? And let’s actually analyze what this tests are actually doing.

 

Let’s take the first one for example. It says Fetch product details response. Okay? Get Products. Now if you recollect the API specification, the first operation if you remember, is a Get operation on the Path products and its Fetch Product details.

 

So Specmatic was able to convert each of these operations inside your open API specification into a test scenario and run it. Now what did it do? It generated a request based off of the API specification. It knew that there is a URL called Products and there’s an ID and that ID is supposed to be number. So this Specmatic kind of went ahead and generated a value which is adhering to the schema.

 

And then when the response came back from the application, it verified that this is indeed in line with the schema that is expected from the actual API specification itself. So thereby the tests are green. Now that’s great. Obviously it’s good news that the application is in line with the API specification, but that’s not necessarily the reality for most of us, right? Incidents do not happen in the Happy Path scenarios.

 

Incidents are hiding in the places that we are not looking for them. Right? So let’s throw a curveball at this whole application. So what I’m going to do is I have this quick snippet here called Specmatic Generator Test. I’m going to paste that in and say Specmatic, I want you to do some extra testing on this application and see if there are any loopholes beyond what you have already verified.

 

So right now you’re saying twelve tests. Just hold on to that number in your head and let’s rerun the test and see what happens. Now the interesting bit here is we want to test boundary conditions, correct? The happy path is when you expect given a we are just checking, right? We are not really exploring what are the possibilities.

 

We just checked so far. But this time around we are going to be exploring what are the boundaries of the application itself. Now, this time around you have 26 tests, which is a lot more than twelve and only 16 of them passed, which means we also have test failures now. Now that’s interesting. Let’s take a look at and analyze what these scenarios are like.

 

So earlier you saw only scenario I did not have positive or negative. Now you have labeling. It says this scenario is a positive scenario and this one’s a negative and the negative one is what failed. So let’s take a look at it. So on the negative scenario, the first one I’m looking at, the error we see here.

 

Oh my. It’s a 500 internal server error. How did that happen? It looks like Specmatic tried to send a null for a value that is not supposed to be null according to the API specification. Right?

 

Now what would have been ideal is if the application had done a null check and then responded with an appropriate 422 or a 400. But then there is no null check in the application, which is evident here and the application instead threw a 500, which is not desirable. This could easily have become an incident in production. But lucky for us, we have Specmatic and we were able to identify it right here and we can now fix it. Now isn’t this awesome that I did not have to write any line of code and I have so much ability to both verify the positive and the boundary conditions for an application, all with just OpenAPI specification.

 

Okay, now that we’ve seen the power of open API specifications and Specmatic and how it is able to leverage it on the provider side, let’s switch gears and look at how this can help the consumer side of the equation. So let’s again look at a quick demo here. What I have is an API specification called Products. YAML, this is a fairly straightforward one. This is not even like 30 40 lines, which is fairly straightforward for any small application.

 

All it says is I have a path called Get Products and given an ID, it gives me back a product for that particular ID which has the name and the SKU. Now this is all I have to start off with and I need to build a mobile application. Now how do I go about my business? So first thing obviously is I want to play around with this API and see what values I get. So what I have done here is I’ve imported it into Postman and I have pretty much the ability to do that because Postman recognizes open API and I’ve got it in.

 

The next step is I need to have a stop server, right? One option is to hand roll it. I could just hard code some values in a mock or a record and replay service. But why will I do that? I have the ability to use Specmatic.

 

So what I’m going to do is start a new terminal and I’m going to say go into this folder called Smartmocks where we have this file lying around. I’m going to say Specmatic stop this file called Products YAML, and that’s pretty much all it says. Hey, there is this server called 9000 running on localhost and we have a step server going. So what does this mean? Let’s actually take a look.

 

Let’s try sending a value. I just have some random number and I send the request and I get back a response which is a name and an SKU magic, right? And this is Specmatic doing the job for us. And every time I send a different value or just send the same value, it gives me different responses for the name and the SKU. Now this is happening because Specmatic is able to guess what data it is supposed to respond with based off of the schema which we saw earlier, which said both name and SKU are strings and so thereby Spectra is generating values within that boundary.

 

Now usually this is not useful as a mobile application developer or a client application developer. I would want specific values. Like for example I want for one I want a particular book to come back to me. So I’m going to say one and magically Specmatic gave me back the name Mythical Manmont and SKU for that book. Now how did this happen?

 

What we did here is if you notice there is a folder called Products underscore Data which has a convention based naming here which is based off of the Products YAML, I just have underscore Data. Now under that I can have as many JSONs as I want and each JSON is basically a request response pair. Excuse me, what I’ve done here is for the product ID one, I want to return Mythical Manmont and the status code and whatnot. Now thereby I’m able to get back this information. Now let’s actually see if we can fool Specmatic right?

 

Now the issue earlier which we noticed was that as the mobile application developer I was wrongly assuming that the server is going to be giving me name and price and not name and SKU. And I was going to say name and SKU, sorry, name and price is basically this $10 for this book for example. And now let’s take a look at what happened. The Stub server running picked up this file change and then said error, right? And it says in scenario get products key name price in the Stub server was not part of the contract.

 

Now that’s very helpful for me, right? Because I would have otherwise wrongly assumed and set up my Stubbing incorrectly and maybe this would have turned out to be a production incident. But instead what now happened is I am completely safeguarded as an engineer from making any wrong assumptions because every time I make a mistake, hey, Specmatic is able to point me hey, this is not the right value. You should probably be adhering to the specification and only if you go back to saying SKU and give any value which can be any string right within that format and now it’s going to be fine and this error will magically go away. That’s why we call this smartmocks.

 

And now when you go back to your postman, then I send the request again. This time you get back any string. So now that we’ve seen both sides of the equation for the consumer and the provider, let’s actually look at the anatomy of a test for the consumer itself. Now, typically for a component test, your system under test needs to be isolated from its dependencies and that’s where Specmatic comes in to provide your smart mock. And for a component test there are three parts to it.

 

Any test to that matter has arrange act assert which is the standard set of steps. And during the arrange step, just like you would use any tool like in unit testing like Mokito, here you are testing an API. So thereby you’re setting up an expectation at the API level. With Specmatic, you saw how I set up the expectation with a JSON file but you could also do this over an Http endpoint if you were to dynamically if you want to set it up dynamically at Runtime. Now Specmatic thoroughly verifies this expectation against the open API and only then stores it into its local storage.

 

And then your component test can invoke the actual system under test and in turn the system under test interacts with Specmatic Stubserver. And Specmatic Stubserver looks up if it has any corresponding data based on the expectation, or it will generate some random data and give it back to the system under test and which in turn returns to the test. And the test asserts if the response is good. Now if you look at this setup, this is very helpful to isolate the system under test while still being sure that you are emulating the provider fairly well otherwise. Now that we’ve seen both sides of the story for the consumer and the provider in terms of how we could use contract as test and Contract as Stub server, let’s look at backward compatibility.

 

Compatibility issues begin to grow a lot more with API evolution. And now API evolution is natural, right? We want to be able to add features. The difficulty, however, is when you are adding a new capability to an API for supporting a newer consumer or a client or a new feature, existing consumers and clients can potentially become incompatible. And that’s where it might be helpful if you could just take a contract and another version of the same contract and just verify it and say, are these two compatible?

 

That would be a nice experiment to run, right, without really getting into a situation where you have to make the changes on the consumer and the provider applications, deploy it, and only then realize that they are not compatible. Let’s take a look at a live demo of this sort of comparison. So let’s take a look at a popquest. Which of these changes are backward compatible in a request? If I add a mandatory or a required field, is that change a backward compatible change to the API?

 

Now, this is a fairly straightforward answer because it’s easy to guess, right? Because if you add a new parameter to the request, obviously existing clients will not be able to send it and thereby it is a backward incompatible change. Now, why take my word for it? Let’s actually test this out. So I’m going to kill this stuff server which was running before.

 

I’m going to go back and go into this folder called backward compatibility. And what I have here are two files, products v one, Products v two. At this point, both of these files are pretty much identical. And they have this one operation wherein you can create a product by posting product details. And by now we are quite familiar with what the product has, right?

 

It’s got a name and an SKU of which only name is required. SKU is optional and then in response it gives you the ID of the product. Let’s assume this is the API we have to work with and we need to understand if these two are backward compatible. So obviously, before we make changes, we want to make sure that we are starting on a clean slate. So I’m going to say Specmatic compare Products v one with Products v two.

 

And when I do that, you will notice, my bad, I did not put the extension. Then yes, the newer contract is backward compatible. Now for the change that we saw in the pop quiz, right, which is basically if I add a new parameter, if I add a parameter which is required. So the quick one to do here is maybe I will make SKU also mandatory. Now, if I do this, what is going to happen?

 

I’m going to run the compile command again and this time around you see this issue, right? It says New contract expects key named SKU in the request. But it is missing from the older contract and therefore new contract is not backward compatible. Okay, so that just proves what we already guessed. That’s not really very groundbreaking, earth shattering.

 

Let’s look at something more complex. What if I change an optional nullable field to an optional non nullable field? Let me repeat it. An optional nullable field to an optional non nullable field. Let’s actually look at it with an example, right?

 

So it’s easier to process. We are back to square one here. Both contracts are compatible. And you will notice in the V one SKU is the optional field and it is nullable. Now what I’m going to do is, according to the quiz question, going to take the optional nullable field and make it optional non nullable.

 

Now, this is not straightforward to process in order to guess whether it’s going to be backward compatibility breaking change or not because there are at least three or four permutation combinations. So let’s try and check what happens. And this time Specmatic says, hey, this is a string in the new contract and nullable in the old contract. Just basically you made something mandatory. But the older contract, it was nullable.

 

And that is actually correct how? Specmatic is able to point it out to us. Because whenever you are going to send SKU or supposed to send the value and not a null, but in the earlier case you could send a null. Thereby it is a backward incompatible change. Now this is getting progressively harder, right?

 

To figure out if this is a backward compatibility change or not, compatible change or not, let’s take it to the next level. What if I change a schema component that is referenced in request and response? Okay, I have an example of such a contract here which is a fairly large one. I do not expect you to read all of it. But what I’d like to draw your attention to is this one particular schema component called Address.

 

Now let’s actually take a look at where Address is used. So I’m going to see address is part of storage. It is part of warehouse and it is also part of cart response. Okay? Now the other problem is storage itself is part of request but address is also part of directly response.

 

Now, if the same schema is part of both request and response. Now how would you make a decision whether it’s backward compatible or not? Now that doesn’t stop there, right? The next one is the hierarchy, which is basically what you noticed because address was directly part of the storage and one more element and it is also directly appearing under the master element, which means you don’t know what sort of issues it can create if you change a value. What can make it even more complicated is if this schema is referenced across multiple files, which is normal, for example with OpenAPI remote references, then that can make life a lot more harder.

 

So for an engineer who’s probably just joined the team. And you ask me to make the change here to say address which is required now and it’s got these four parameters which are required. I want you to remove this from required to making it optional. Now, is this change backward compatible? I’m pretty sure at least I am not smart enough to compute this mentally.

 

I need the help of something like Specmatic to figure this out. And isn’t this very powerful to do because all you’ve noticed is I’ve been able to experiment which change is compatible or not just by changing the specifications and not really writing any code. Now, this would be a great way of collaborating between teams to understand if a change made for one team can break the compatibility with another consumer or client team. Now, with that out of the way, let’s further move to the next topic which is called central contract repo. Are we on the same page?

 

What do I mean by that? Now, we’ve seen contract test contract as stub. We’ve seen backward compatibility verification and all of this hard work that we’ve done. And still it is possible that let’s say I am the provider developer and I make a change to the application. And then I either miss updating the Open API specification or I update the wrong file for whatever reason.

 

And likewise, let’s say if I am the consumer application developer, I somehow ended up referring to the older version of the contract or I am not looking at the latest version that someone sent me over email and thereby I have ended up using the wrong API specification as a reference for building my consumer application. Now, if this happens, irrespective of whether you’re using Specmatic and all the smart mock and contract test and whatnot, you can still end up with a broken integration. Now, that’s not a desirable place to be, right? What we want is to have the API specification as a single source of truth. It is code, practically.

 

And how do we treat code? We keep it in a version control system and that’s where what we believe is open API or any other API specification standards you’re using should reside in a central contract repository, which is a version control system. And if you are treating it like code, then you will end up having something like a pull request, a merge request process, which is a good thing because you can now have some sort of a style or an API lenter with probably tools such as toplight spectral. And once you have gone through the basic process there, then you can do the backward compatibility testing which we just spoke about, right? And verify that the change that is being proposed is compatible or not.

 

Now, Specmatic, you noticed I was able to compare two files. Likewise, it can also compare two versions of the same file in a version control system and thereby give you a go ahead whether it is compatible or not. If it is compatible, you’ll go ahead and review and merge it manually. After sorry, you would do a manual review and then merge it and pretty much evolve the API further. Or if you realize that it’s a backward incompatible change, then you may have to think about a versioning strategy for your API itself.

 

Now, with this pretty much we have covered four topics, right? Which is Contract Testers. Contract Tester. Contract versus Contract backward compatibility testing. And then the central Contract repo.

 

How do all these pieces come together eventually in the CI pipeline to solve our initial problem, which is to shift left the identification of compatibility issues. Now that we know that the API specifications is being stored in the central Contract repository, which is a single source of truth and Specmatic is able to read it, make it available as Contract as step server for your consumer application and Contract as test for your provider application. Let’s see what happens further in the continuous integration environment of the consumer application. After the unit tests are done for component testing, you don’t need to look for another tool to stub out the provider. You can use the Specmatic stubserver which you are using on your local environment itself environment as well, like I mentioned earlier.

 

So now that you’ve been able to emulate the provider for the consumer, let’s look at what happens for the provider itself. Now, after the unit testing is done, we run the Contract test first to verify that the provider is adhering to the API specifications and only after that we run the component testing. Now that we have adhered with the API specification both on our local and on our CI for both the consumer and the provider applications, we can confidently deploy to this integration testing environment and be sure that it’s going to work. And you also have an unblocked path to production. And pretty much none of the compatibility related incidents are bound to happen now in production at all because we have completely shifted left the identification of compatibility issues to the local environment or at most to the CI very much in the green right, which is what the heat map is representing.

 

So which means the cost of fixing such issues is also fairly low. And that is how Specmatic and Contract driven development can help you identify compatibility issues early in your development cycle without depending on integration testing and then allow you to confidently deploy your applications independently to production. With that, I’ll open up for the Q and A on the Discord server and thank you very much. Please do feel free to reach out to me on my social handle here and do check out Specmatic Ion. Thank you.