Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 10 months ago.
The community reviewed whether to reopen this question 10 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I am trying to understand the better tool between PACT and Spring Cloud Contract to implement Consumer Driver Contract Tests.
I dont find any clear examples to find the pros and cons.
I want to implement CDCT, and I dont use Spring in my project. From what i understood I am assuming PACT is good to go with.
Any information or suggestions are welcomed.
Thank you.
Spring Cloud Contract allows you to define and test contracts for both REST APIs and messaging. It provides a clear and easy to use statically typed Groovy DSL and also allows for defining contracts via yaml. Both with the DSL and with yaml, defining contracts is very intuitive to anyone familiar with the standard HTTP/messaging terms, for example:
request {
method PUT()
url '/fraudcheck'
body([
"client.id": $(regex('[0-9]{10}')),
loanAmount: 99999
])
headers {
contentType('application/json')
}
}
response {
status OK()
body([
fraudCheckStatus: "FRAUD",
"rejection.reason": "Amount too high"
])
headers {
contentType applicationJson()
}
}
The producer-side tests are automatically generated by SCC and added to the project during build. If a correct implementation fulfilling the contracts is not in place, the project will not be built and deployed. If they pass, the stubs for consumer will be generated and published along with the corresponding artifact version.
On the consumer side, for HTTP, SCC provides the Stubrunner, that launches a Wiremock (in-memory http server) instance, that provides stubbed responses to matching requests. Stubrunner works with external artifact repositories such as Nexus and Artifactory and also with local m2 repositories.
SCC integrates with SpringBoot seamlessly and also integrates with Spring Cloud out of the box and can be used in place of service discovery during integration tests.
It also integrates out of the box with Pact and allows to leverage additional Pact features via hooks while using only the SCC contracts definitions.
SCC also provides a Docker-based option for implementing and testing contracts in technologies other than JVM.
DISCLAIMER: I’m a Spring Cloud Contract committer.
In contract testing, we want to make sure that the provider and consumer of an API are compatible with each other. Therefore we define our expectations as a contract using a DSL and test it against the consumer or provider of the API and if it is successful then publish it (via manually copying it or publishing to a shared repository) to the other side of the contract (Provider or consumer).
Two main providers of such functionality are: Pact and SCC (Spring Cloud Contract)
PACT: Pact is a consumer-driven contract meaning that we define and validate the contract on the consumer side and then push it to the provider side for validation.
Additionally, Pact is not limited to a specific framework or even language (pact-JVM is there for a Java application). Also, it integrates well with Spring (Boot) and you can define pact test while starting the whole application (#SpringBootTest) or during Slice testing (#WebMvcTest).
SCC: Generally it encourages provider-driven contracts, meaning that you define the contract on the provider side and if it validates then publish it to the consumer side. However, we can do it the other way around or even we can define the contract separately and test it manually against the producer and consumer.
Related
I'm just starting with Consumer-Driven Contracts for our microservice setup, and as they are mainly Spring Boot apps natural choice is Spring Cloud Contract.
But then comes the confusion. In all examples in docs contracts are defined on the producer side, and consumer retrieves them from maven repo. How it is consumer-driven if the producer defines them?
My understanding was that conceptually consumer defines them as its expectation from the producer for part of the API it uses. In addition to the previous question, is it possible with Spring Cloud Contract to define a contract on the consumer side and verify it on producers?
Thanks for this question. Yes, Spring Cloud Contract can be consumer-driven or producer driven.
The contract definitions can lay either with the producer or in an external repository. The concept of consumer-driven is not related to the fact where the contract are stored. It's related to the fact who creates them or actually who drives the change of those contracts. Another thing is whether each consumer can define a set of their own requirements. That means that one consumer can have different requirements than another one.
So in Spring Cloud Contract for the producer-driven approach, the producer just creates the contracts and it doesn't really care if different consumers use their API in a different way. The producer of the API creates e.g. 1 contract definition for all the consumers and it's the consumer's problem to align to that contract.
With consumer-driven contracts with Spring Cloud Contract, it's the consumer that suggests the changes and creates the contract definitions. The consumer sends a PR either to the producer team's repo or to a separate repo where all the contract definitions are stored. It's not the producer, but the consumer who does it. The consumer can, in the meantime, create the stubs locally without asking the producer for the permission, and prototype the API. Once the consumer has finished its work and of course talked with the producer team about the changes before, then a pull request can be sent with the suggested changes. Important thing to remember is that each consumer has their own folder with their requirements. E.g. if there are consumers example1 and example2 and there's a producer producer1 then under producer1's contract folder there will be 2 folders, one example1 and the second example2. Both will contain expectations for the particular consumer.
Then the producer takes the PR over and the verification happens on the producer side. The tests are generated. Once the implementation is written and tests are passing the stubs can be uploaded.
Finally, on the consumer side, the consumers can switch to start fetching the uploaded stubs to always get the fresh version of the stubs.
You can check a video of how full cycle of CDC is done here: https://www.youtube.com/watch?v=pDkC_00hhvA
Consumer Driven Contract tutorial: https://cloud-samples.spring.io/spring-cloud-contract-samples/tutorials/stubs_per_consumer.html
Producer Contract tutorial with contracts on the producer side: https://cloud-samples.spring.io/spring-cloud-contract-samples/tutorials/contracts_on_the_producer_side.html
Producer Contract tutorial with contracts in an external repo: https://cloud-samples.spring.io/spring-cloud-contract-samples/tutorials/contracts_external.html
All the workflows related to doing contract tests with Spring Cloud Contract can be found here - https://docs.spring.io/spring-cloud-contract/docs/current/reference/html/using.html#using
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
My question addresses a conceptual issue with consumer driven contracts (CDCs), not with Spring-Cloud-Contract in particular (though tagged otherwise).
A CDC usually guarantees a particular behavior of a service (producer/supplier) to its client (consumer). Many architectures (e.g. Hexagonal/Clean/Onion Architecture) have an adapter/layer to provide the technical interface(s) of the service (REST/SOAP/Messaging/...). They separate the adapter from the business logic. When tests are written based on a CDC it is possible to just test the adapter (classical unit test) and mock away the business logic.
However, the contract formally does not only specify a particular mapping of the result (from the business logic) to the data transfer object as returned by the adapter. Moreover it contains semantics wrt. to the request/response behavior: If the consumer sends a specific request with meaningful data, the producer should respond with a particular correct answer (with respect to the request).
My point for discussion/question is: Should the test on the producer side then be an integration test (integrating business logic) to check if the business logic will return the correct answer? In this case the next layer (behind business logic) might be mocked away as long as the business logic is an integral part of the test case.
In Spring CDCs (but I think in other CDC implementations as well) it is possible to distinguish between consumer and producer side and the generated test cases. For example it would be possible to reduce the producer part to check for formal correctness of the response (e.g., if a particular data item is contained and validates against a pattern) while it specifies a concrete response for the consumer part at the same time. This might lead to the (mis-) understanding on the consumer side (and backed by the consumer tests) that the concrete specified data would be returned by the producer if the consumer had sent the specified request from the CDC.
What do users of CDCs think? How do you specify behavior and to what extent do you test it on the producer side?
We describe it in the documentation - https://docs.spring.io/spring-cloud-contract/docs/2.2.5.RELEASE/reference/htmlsingle/#getting-started-introducing-spring-cloud-contract-purposes . Let me write it here for your convenience
Spring Cloud Contract’s purpose is NOT to start writing business
features in the contracts. Assume that we have a business use case of
fraud check. If a user can be a fraud for 100 different reasons, we
would assume that you would create two contracts, one for the positive
case and one for the negative case. Contract tests are used to test
contracts between applications and not to simulate full behavior.
If you check Pact's documentation (https://docs.pact.io/faq#do-i-still-need-end-to-end-tests) you'll find
Contract tests replace a certain class of system integration test (the
ones you do to make sure that you're using the API correctly and that
the API responds the way you expect). They don't replace the tests
that ensure that the core business logic of your services is working.
In case of Spring Cloud Contract - if you have a controller that delegates work to a service, then you should mock out the service and test only whether your controller can take a request and send back a proper response
I am trying to adopt Pact. I understand the consumer side of the equation and it looks very nice. But I am confused about the producer side.
It seems the documentation advocate running the provider app, and verifying the contracts against a running server.
I prefer not to do it. First, I need to curate a database with proper information for each pact, which is painful to say the least. Second, starting up the application is going to be a hassle - did I mention it is a monolith? -, finally, there are POSTS which are going to mutate the state of the database, and make test running brittle.
What I want to do is to do a mockMvc style testing with the pacts. I would like to mock my services, and just test the endpoint, which I think what should be tested in this case.
How can I achieve this with Pact?
Well if you don't test your contracts against your Provider that loses the whole point of Contract testing, since your contracts aren't tested against both sides. Because the main point is that Consumers dictate how the Provider should behave and in your case you would like to bypass the provider with a mockMvc and there is no point for doing contract testing only against your Consumer not the Provider. Even though your Provider is a monolith it's still better to run it and test with a contract, then to run all the microservices for end-to-end testing.
Yes you can achieve it with PACT, however I have the same opinion with Cotnic that it beats the purpose of having PACT on provider side. The main purpose of PACT is to verify that your server as the provider is working as according to the agreement (PACT). Therefore in my opinion the proper way to use PACT as a CONTRACT is by running it against a fully deployed server, and using #State to "Prepare" the Server (db, startup applications, etc)
Anyway, if you are using Spring, you probably can have a look at this sample for using Pact with MockMvc
https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
Pact-JVM now supports Spring mockmvc tests to verify a Spring or Springboot provider. See https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
I have read that on Pact, the consumer is the one initiating the request. I have a service (let's call it A) that is used to draw pictures that will then be submitted (via POST) to a service (let's call it B) that will process those inputs and generate albums.
My question is: If the service B is the expert on knowing which kind of inputs and in which format should receive in order to create albums, how come is service A the consumer and therefore the one that will be writing the contract? Shouldn't it be the service B that specifies what kind of data should be receiving?
From [1]:
A component that initiates a HTTP request to another component (the service Provider). Note that this does not depend on the way the data flows - whether it is a GET or a PUT / POST / PATCH, the Consumer is the initiator of the HTTP request.
But I think your question is really asking about consumer driven contract testing [2]. You're conflating who is building the service with expertise / understanding of the requirements for its existence. The Consumer knows what it needs from the service, and the Provider knows how to implement it.
Consumer driven contracts reverses the typical API design process of creating an API first and then consumers coming along to use it. First, the consumer specifies what it requires, publishes the expectations as a contract and the Provider then implements it.
Pact generally assumes this mode, but it's not strictly necessary (i.e. the Provider API can exist in advance).
[1] https://docs.pact.io/documentation/how_does_pact_work.html
[2] https://martinfowler.com/articles/consumerDrivenContracts.html
We have two micro-services: Provider and Consumer, both are built independently. Consumer micro-service makes a mistake in how it consumes Provider service (for whatever reason) and as a result, incorrect pact is published to the Pact Broker.
Consumer service build is successful (and can go all the way to release!), but next Provider service build will fail for the wrong reason. So we end up with the broken Provider service build and a broken release of Consumer.
What is the best practice to guard against situations like this?
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Thanks!
This is the nature of consumer-driven contracts - the consumer gets a significant say in the API!
As a general rule, if the contract doesn't change, there is no need to run the Provider build, albeit there is currently no easy way to know this in the Broker (see feature request https://github.com/bethesque/pact_broker/issues/48).
As for solutions you could use one or more of the below strategies.
Effective use of code branches
It is of course very important that new assumptions on the contract be validated by the Provider before the Consumer can be safely released. Have branches tested against the Provider before you merge into master.
But most importantly - you must be collaborating closely with the Provider team!
Use source control to detect a modified contract:
If you also checked the master pact files into source control, your CI build could conditionally act - if the contract has changed, you must wait for a green provider build, if not you can safely deploy!
Store in separate repository
If you really want the provider to maintain control, you could store contracts in an intermediate repository or file location managed by the provider. I'd recommend this is a last resort as it negates much of the collaboration pact intends to facilitate.
Use Pact Broker Webhooks:
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Yes, this is possible using web hooks on the Pact Broker. You could trigger a build on the Provider as soon as a new contract is submitted to the server.
You could envisage this step working with options 1 and 2.
See Using Pact where the Consumer team is different from the Provider team in our FAQ for more on this use case.
You're spot on, that is one of the current things lacking with the Pact workflow and it's something I've been meaning of working towards once a few other things align.
That being said, in the meantime, this isn't solving your current problem, so I'm going to suggest a potential workaround in your process. Instead of running the test for the consumer, them passing, and then releasing it straight away, you could have the test run on the consumer, then wait for the provider test to come back green before releasing the consumer/provider together. Another way would be to version your provider/consumer interactions (api versioning) so that you can release the consumer beforehand, but isn't "turned on" until the correct version of the provider is released.
None of these solutions are great and I wholeheartedly agree. This is something that I'm quite passionate about and will be working on soon to fix the developer experience with pact broker and releasing the consumer/provider in a better fashion.
Any and all comments are welcome. Cheers.
I think the problem might be caused by the fact that contracts are generated on the consumer side. It means that consumers can modify those contracts how they want. But in the end producer's build will suffer due to incorrect contracts generated by consumers.
Is there any way that contracts are defined by producer? As I think the producer is responsible for maintaining its own contracts. For instance, in case of Spring Cloud Contracts it is recommended to have contacts defined in producer sources (e.g. in the same git repo with producer source code) or in a separate scm repo that can be managed by producer and consumer together.