I am trying to adopt Pact. I understand the consumer side of the equation and it looks very nice. But I am confused about the producer side.
It seems the documentation advocate running the provider app, and verifying the contracts against a running server.
I prefer not to do it. First, I need to curate a database with proper information for each pact, which is painful to say the least. Second, starting up the application is going to be a hassle - did I mention it is a monolith? -, finally, there are POSTS which are going to mutate the state of the database, and make test running brittle.
What I want to do is to do a mockMvc style testing with the pacts. I would like to mock my services, and just test the endpoint, which I think what should be tested in this case.
How can I achieve this with Pact?
Well if you don't test your contracts against your Provider that loses the whole point of Contract testing, since your contracts aren't tested against both sides. Because the main point is that Consumers dictate how the Provider should behave and in your case you would like to bypass the provider with a mockMvc and there is no point for doing contract testing only against your Consumer not the Provider. Even though your Provider is a monolith it's still better to run it and test with a contract, then to run all the microservices for end-to-end testing.
Yes you can achieve it with PACT, however I have the same opinion with Cotnic that it beats the purpose of having PACT on provider side. The main purpose of PACT is to verify that your server as the provider is working as according to the agreement (PACT). Therefore in my opinion the proper way to use PACT as a CONTRACT is by running it against a fully deployed server, and using #State to "Prepare" the Server (db, startup applications, etc)
Anyway, if you are using Spring, you probably can have a look at this sample for using Pact with MockMvc
https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
Pact-JVM now supports Spring mockmvc tests to verify a Spring or Springboot provider. See https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
Related
We have 5 microservices (more on the way) that communicate with each other asynchronously. 3 of these microservices do not have any API. Those consume data from a message queue, do some processing, and write data into another queue. 2 of these microservices do have APIs, and those also consume data from the queues but send the response back to the caller.
Given that, for testing the service interactions, correctness of contracts, and end-to-end flow:
what would be the best way to test the asynchronous services that read from and write to queues?
would consumer-contract test be applicable anywhere?
I feel end-to-end production testing is possible, but can something more granular and effective be done?
First, let's correct one thing - you do have APIs. The messages that the "API-less" services read must have some defined content or format. That's an API. You should be testing it, both positive and negative.
Before you get to whole-system testing (in a test or staging environment that mimics your production environment) your testing should probably be in layers, much as in any other system.
Unit tests to test the behavior of each class. In your message-handling classes, for example, call each with messages that prompt specific actions and test that it works correctly. These can usually be run very quickly so it's easy to run them often while developing code.
Integration tests to make sure your interaction with next-level external systems works. You can use, for example, testcontainers to run isolated instances of the queueing system that your services interact with.
Exactly what form these tests take depends on what languages and frameworks your system is built with. I briefly looked at the Pact tool you referenced and it is taking a similar approach.
Looks like with Pact it is possible to do message-based contract testing. I think I have a path forward with this.
I am new to PACT and trying to use pact-net for contract testing for a .net microservice. I understand the concept of consumer test which generates a pact file.
There is the concept of a provider state middleware which is responsible for making sure that the provider's state matches the Given() condition in the generated pact.
I am bit confused on the following or how to achieve this:
The provider tests are run against the actual service. So we start the provider service before tests are run. My provider service interacts with a database to store and retrieve records. PACT also mentions that all the dependencies of a service should be stubbed.
So we run the actual provider api that is running against the actual db?
If we running the api against actual db how do we inject the data into the db? Should we be using the provider api's own endpoints to add the Given() data?
If the above is not the correct approach then what is?
All the basic blog articles I have come across do not explain this and usually have examples with no provider states or states that are just some text files on the file system.
Help appreciated.
I'm going to add to Matt's comment, you have three options:
Do your provider test with a connected environment but you will have to do some cleanup manually afterwards and make sure your data is always available in your db or/and the external APIs are always up and running. Simple to write but can be very hard to maintain.
You mock your API calls but call the real database.
You mock all your external dependencies: the API and the DB calls.
For 2) or 3) you will have to have test routes and inject the provider state middleware in your provider test fixture. Then, you can configure provider states to be called to generate in-memory data if solution 3) or add some data-init if you are in solution 2)
You can find an example here: https://github.com/pact-foundation/pact-net/tree/master/Samples/EventApi/Provider.Api.Web.Tests
The provider tests are run against the actual service
Do you mean against a live environment, or the actual service running locally to the unit test (the former is not recommended, because of (2) above).
This is one of the exceptions to that rule. You can choose to use a real DB or an in-memory one - whatever is most convenient. It's common to use docker and tools like that for testing.
In your case, I'd have a specific test-only set of routes that respond to the provider state handler endpoints, that also have access to the repository code and can manipulate state of the system.
I'm implementing a service that needs to call another service that calculates a result in a way I cannot know.
Let's say I have the following scenario:
I have some place in my code, calling a HTTP request to a defined endpoint another service returning a defined result. Now, I mustn't dictate how the result will be calculated, however I can define the result output data type I'm expecting. I want to put emphasis on this, since otherwise I would just implement the calculation logic in my service.
I would then describe it to the user:
You need to provide an HTTP service, with this exact endpoint, receiving these exact parameters, delivering this exact result type, but how you calculate the result is your job. I just need the URL of your service.
Afterwards the user of my service would configure the URL to their HTTP service into my service, so that I can make a HTTP request to {url}/defined-endpoint.
I couldn't think of another name but "service-injection" to describe this concept, since it has a resemblance to dependency injection, just that in code you don't provide an object instance, but a service that is called via http.
My question is: Is there a pattern for this concept or an alternative that more elegantly solves the general problem of outsourcing a calculation to another service?
You are defining a contract of how the interface between your service and the other service. This means that as long as the contract is respected by both parties the integration and communication will succeed. Not sure if "service-injection" is a good terminology for this. You are not injecting something in your own service, you are simply delegating the calculation to another one, but you don't inject the logic of the service into your own. And that is good because then you have a good separation of concerns and loose coupling. As long as the contract is respected, both services can be changed in whatever way it is needed and the integration would still hold.
Is there a pattern for this concept or an alternative that more elegantly solves the general problem of outsourcing a calculation to another service?
This is just how things work in a micro-services ecosystem. You have multiple services exposing APIs that communicate with each other to provide as a whole a higher-order functionality.
We have two micro-services: Provider and Consumer, both are built independently. Consumer micro-service makes a mistake in how it consumes Provider service (for whatever reason) and as a result, incorrect pact is published to the Pact Broker.
Consumer service build is successful (and can go all the way to release!), but next Provider service build will fail for the wrong reason. So we end up with the broken Provider service build and a broken release of Consumer.
What is the best practice to guard against situations like this?
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Thanks!
This is the nature of consumer-driven contracts - the consumer gets a significant say in the API!
As a general rule, if the contract doesn't change, there is no need to run the Provider build, albeit there is currently no easy way to know this in the Broker (see feature request https://github.com/bethesque/pact_broker/issues/48).
As for solutions you could use one or more of the below strategies.
Effective use of code branches
It is of course very important that new assumptions on the contract be validated by the Provider before the Consumer can be safely released. Have branches tested against the Provider before you merge into master.
But most importantly - you must be collaborating closely with the Provider team!
Use source control to detect a modified contract:
If you also checked the master pact files into source control, your CI build could conditionally act - if the contract has changed, you must wait for a green provider build, if not you can safely deploy!
Store in separate repository
If you really want the provider to maintain control, you could store contracts in an intermediate repository or file location managed by the provider. I'd recommend this is a last resort as it negates much of the collaboration pact intends to facilitate.
Use Pact Broker Webhooks:
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Yes, this is possible using web hooks on the Pact Broker. You could trigger a build on the Provider as soon as a new contract is submitted to the server.
You could envisage this step working with options 1 and 2.
See Using Pact where the Consumer team is different from the Provider team in our FAQ for more on this use case.
You're spot on, that is one of the current things lacking with the Pact workflow and it's something I've been meaning of working towards once a few other things align.
That being said, in the meantime, this isn't solving your current problem, so I'm going to suggest a potential workaround in your process. Instead of running the test for the consumer, them passing, and then releasing it straight away, you could have the test run on the consumer, then wait for the provider test to come back green before releasing the consumer/provider together. Another way would be to version your provider/consumer interactions (api versioning) so that you can release the consumer beforehand, but isn't "turned on" until the correct version of the provider is released.
None of these solutions are great and I wholeheartedly agree. This is something that I'm quite passionate about and will be working on soon to fix the developer experience with pact broker and releasing the consumer/provider in a better fashion.
Any and all comments are welcome. Cheers.
I think the problem might be caused by the fact that contracts are generated on the consumer side. It means that consumers can modify those contracts how they want. But in the end producer's build will suffer due to incorrect contracts generated by consumers.
Is there any way that contracts are defined by producer? As I think the producer is responsible for maintaining its own contracts. For instance, in case of Spring Cloud Contracts it is recommended to have contacts defined in producer sources (e.g. in the same git repo with producer source code) or in a separate scm repo that can be managed by producer and consumer together.
I am confused between these two nuances:
Is EJB itself middleware, or is there any middleware used in the deployment of EJB?
Same goes for RMI- is RMI itself middleware or is middleware used in RMI?
Define Middleware, if it is in the middle what is it between? I agree with the basic idea of this Wikipedia definition:
Middleware is a computer software that provides services to software
applications beyond those available from the operating system. It can
be described as "software glue".[1] Middleware makes it easier for
software developers to perform communication and input/output, so they
can focus on the specific purpose of their application
So the key idea is you write software and exploit something more sophisticated than the plain operating system. I would not say that middleware is only doing communication and input/output, as I'll explain later.
Now define EJBs. There are two things here: the EJB itself, that is application software that you write; you write an EJB as part of your application development so it is not middleware. But you write to a specification defined by Java EE, and deploy your EJB to an EJB Container provided by an Application Server. The EJB Container and App Server are providing something more sophisticated than the operating system. So the container and server are middleware.
The EJB Container facilities include communications (eg. RMI access and JDBC database access) but also include things such as security and transactions.
EJBs are a component of Java EE, which is a middleware.
RMI is another one, and also another component of Java EE.
You can see that these terms aren't too precise.
I agree with EJP.
Middleware, as it says, is software that provides service for distributed applications, it connects kernel(like server) and user apps.
EJB is a component architecture in server-side and is part of Java EE, it is built on RMI. So both of them are components of middleware.
For me, I liken middleware to the UNIX philosophy per McIlroy: "Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."
The middleware is all about handling the "text streams".
Each program does one thing, does it well. Having said that, it works on its own, but is also written to work together with others. If a program is to work on its own, then in my view, it is asynchronous. For this you need the middleware if it's to work with others.
I think the RPC (RMI) stuff is too tightly coupled, and synchronous, so it fails my definition of middleware. I think the EJB is trying to do far too much than handle "text streams".
There's obviously more to this topic. Try Middleware but it all gets too complicated for me, probably because people are trying to define middleware as stuff that allows programs from different vendors to talk to each other. Now you get into rivalry and competition, "standards" and ISO stuff.