I am new to PACT and trying to use pact-net for contract testing for a .net microservice. I understand the concept of consumer test which generates a pact file.
There is the concept of a provider state middleware which is responsible for making sure that the provider's state matches the Given() condition in the generated pact.
I am bit confused on the following or how to achieve this:
The provider tests are run against the actual service. So we start the provider service before tests are run. My provider service interacts with a database to store and retrieve records. PACT also mentions that all the dependencies of a service should be stubbed.
So we run the actual provider api that is running against the actual db?
If we running the api against actual db how do we inject the data into the db? Should we be using the provider api's own endpoints to add the Given() data?
If the above is not the correct approach then what is?
All the basic blog articles I have come across do not explain this and usually have examples with no provider states or states that are just some text files on the file system.
Help appreciated.
I'm going to add to Matt's comment, you have three options:
Do your provider test with a connected environment but you will have to do some cleanup manually afterwards and make sure your data is always available in your db or/and the external APIs are always up and running. Simple to write but can be very hard to maintain.
You mock your API calls but call the real database.
You mock all your external dependencies: the API and the DB calls.
For 2) or 3) you will have to have test routes and inject the provider state middleware in your provider test fixture. Then, you can configure provider states to be called to generate in-memory data if solution 3) or add some data-init if you are in solution 2)
You can find an example here: https://github.com/pact-foundation/pact-net/tree/master/Samples/EventApi/Provider.Api.Web.Tests
The provider tests are run against the actual service
Do you mean against a live environment, or the actual service running locally to the unit test (the former is not recommended, because of (2) above).
This is one of the exceptions to that rule. You can choose to use a real DB or an in-memory one - whatever is most convenient. It's common to use docker and tools like that for testing.
In your case, I'd have a specific test-only set of routes that respond to the provider state handler endpoints, that also have access to the repository code and can manipulate state of the system.
Related
How to create a mock server to call a webservice in my providerservice class.
I created the clientPactTest class with some interactions, created json file, published it, but got stuck at the Verfication part in the ProciderService.
In my scenario i have to call a webservice in my Provider service. Its a typescript project.
Please answer my queries:
1) Right now iam running my webservice at command line at localhost port 8080. Then I created a ProviderService with a Pact mentioning the provider url and hostname as localhost and port 8080. So that all my tests hit the webservice which I started at command line. Is this the right approach?
2) Iam a bit confused as at many websites its mentioned that we dont run the actual service for pacts verification and we should create a mock server. So if I create a mock server, do i have re write all the endpoints in my providerservice file, the way it is mentioned in the actual webservice ?
Please clarify.
Thanks,
Aruna
With provider side verification you always need to run the real provider to ensure it behaves as expected by its consumers. On the consumer side you use the mock service in place of the provider, so perhaps this is where the confusion lies.
In the case where your provider service also is a consumer of another service, the pattern repeats itself.
See https://docs.pact.io/best_practices/provider#only-stub-layers-beneath-where-contents-of-the-request-body-are-extracted and related sections on that page.
The general advice is that you should use local stubs for any downstream service(s) during pact provider testing
I am trying to adopt Pact. I understand the consumer side of the equation and it looks very nice. But I am confused about the producer side.
It seems the documentation advocate running the provider app, and verifying the contracts against a running server.
I prefer not to do it. First, I need to curate a database with proper information for each pact, which is painful to say the least. Second, starting up the application is going to be a hassle - did I mention it is a monolith? -, finally, there are POSTS which are going to mutate the state of the database, and make test running brittle.
What I want to do is to do a mockMvc style testing with the pacts. I would like to mock my services, and just test the endpoint, which I think what should be tested in this case.
How can I achieve this with Pact?
Well if you don't test your contracts against your Provider that loses the whole point of Contract testing, since your contracts aren't tested against both sides. Because the main point is that Consumers dictate how the Provider should behave and in your case you would like to bypass the provider with a mockMvc and there is no point for doing contract testing only against your Consumer not the Provider. Even though your Provider is a monolith it's still better to run it and test with a contract, then to run all the microservices for end-to-end testing.
Yes you can achieve it with PACT, however I have the same opinion with Cotnic that it beats the purpose of having PACT on provider side. The main purpose of PACT is to verify that your server as the provider is working as according to the agreement (PACT). Therefore in my opinion the proper way to use PACT as a CONTRACT is by running it against a fully deployed server, and using #State to "Prepare" the Server (db, startup applications, etc)
Anyway, if you are using Spring, you probably can have a look at this sample for using Pact with MockMvc
https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
Pact-JVM now supports Spring mockmvc tests to verify a Spring or Springboot provider. See https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-spring
I'm trying to create an end-to-end Application Map using Application Insights. Note all dependencies and metrics are captured and sent using the SDK.
Take the following scenario:
Windows service (batch processing) > (calls) WebAPI > (queries db)
I have 2 Application Insight resources - Windows Service and WebAPI. Both are capturing metrics but in isolation. How can I create a dependency using the SDK between resource 1 (i.e. service) and resource 2 (i.e. WebAPI)? I need to be able to view the Application Map for resource 1 and be able to see the entire end-to-end view of windows service > web service > db.
I can currently see only windows service > WebApi (App Map resource 1) or WebApi > db (App Map resource 2). Need to bring both together somehow?
Application Insights sdk only collect dependencies automatically for HTTP dependencies. Also it only works when the application insights profiler is running on the machine (often installed on azure websites through the Application Insights Extension).
If you happen to be in one of the situations where the new beta sdk is not collecting dependencies for you. You can do that on your own by writing a little bit of code yourself.
The sdk's autocollection code is open source and you can use it to guide you as to how to track these dependencies. The idea is to append the dependency telemetry's target field with the hash of the target component's instrumentation key and set the dependency type to "Application Insights".
Here is how to compute the hash: Compute Hash
Here is how to add it to the target field and set the right dependency type on the dependency telemetry object: Add component correlation to DependencyTelemetryTarget
A little word of caution. There may soon be a change to the format in which the target field is captured / the name of the dependency type (see this discussion). If and when that happens, it would be an easy enough change for you too.
My recommendation would be to use the same Application Insights resources (e.g. instrumentation key) for both your Windows Service and Web API.
You can separate your telemetry for those two services by adding a custom property indicating the service for all telemetry you emit. Easiest way to do this would be to implement a telemetry initializer (see here for documentation).
It is not possible today. Possible ways -
Use a single InstrumentationKey and identify by a custom property (as
suggested by #EranG
Export the data for both the apps and do your own thing
Please vote on this uservoice. Product team is already considering implementing this functionality in future.
We have two micro-services: Provider and Consumer, both are built independently. Consumer micro-service makes a mistake in how it consumes Provider service (for whatever reason) and as a result, incorrect pact is published to the Pact Broker.
Consumer service build is successful (and can go all the way to release!), but next Provider service build will fail for the wrong reason. So we end up with the broken Provider service build and a broken release of Consumer.
What is the best practice to guard against situations like this?
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Thanks!
This is the nature of consumer-driven contracts - the consumer gets a significant say in the API!
As a general rule, if the contract doesn't change, there is no need to run the Provider build, albeit there is currently no easy way to know this in the Broker (see feature request https://github.com/bethesque/pact_broker/issues/48).
As for solutions you could use one or more of the below strategies.
Effective use of code branches
It is of course very important that new assumptions on the contract be validated by the Provider before the Consumer can be safely released. Have branches tested against the Provider before you merge into master.
But most importantly - you must be collaborating closely with the Provider team!
Use source control to detect a modified contract:
If you also checked the master pact files into source control, your CI build could conditionally act - if the contract has changed, you must wait for a green provider build, if not you can safely deploy!
Store in separate repository
If you really want the provider to maintain control, you could store contracts in an intermediate repository or file location managed by the provider. I'd recommend this is a last resort as it negates much of the collaboration pact intends to facilitate.
Use Pact Broker Webhooks:
I was hoping that Pact Broker can trigger the Provider tests automatically when contracts are published and notify Consumers if they fail, but it doesn't seem to be the case.
Yes, this is possible using web hooks on the Pact Broker. You could trigger a build on the Provider as soon as a new contract is submitted to the server.
You could envisage this step working with options 1 and 2.
See Using Pact where the Consumer team is different from the Provider team in our FAQ for more on this use case.
You're spot on, that is one of the current things lacking with the Pact workflow and it's something I've been meaning of working towards once a few other things align.
That being said, in the meantime, this isn't solving your current problem, so I'm going to suggest a potential workaround in your process. Instead of running the test for the consumer, them passing, and then releasing it straight away, you could have the test run on the consumer, then wait for the provider test to come back green before releasing the consumer/provider together. Another way would be to version your provider/consumer interactions (api versioning) so that you can release the consumer beforehand, but isn't "turned on" until the correct version of the provider is released.
None of these solutions are great and I wholeheartedly agree. This is something that I'm quite passionate about and will be working on soon to fix the developer experience with pact broker and releasing the consumer/provider in a better fashion.
Any and all comments are welcome. Cheers.
I think the problem might be caused by the fact that contracts are generated on the consumer side. It means that consumers can modify those contracts how they want. But in the end producer's build will suffer due to incorrect contracts generated by consumers.
Is there any way that contracts are defined by producer? As I think the producer is responsible for maintaining its own contracts. For instance, in case of Spring Cloud Contracts it is recommended to have contacts defined in producer sources (e.g. in the same git repo with producer source code) or in a separate scm repo that can be managed by producer and consumer together.
I'm trying to write test cases for my web services in a way that I can rollback any database changes they may make. I can try to surround them with a transaction scope, but how do I specify a context for the transaction? In other words, how does the transaction know which database and server to rollback on? In my case, the SQL server runs locally as well as the web services. Before you tell me to call the web services directly without a client, please understand that the web services have very specific runtime environment settings that would be a royal pain to reproduce for my test cases. Perhaps, the transaction scope isn't what I want to use, is there an alternative? Is there a database function I could call to start a transaction? Thanks.
First you are not doing unit testing. Unit test is about testing single small unit of code (function). When you test a function you are creating unit test for each execution path so that you have full coverage of tested code. But your system under tests includes client to service communication and service to database communication = several tiers of code + configuration. That is called integration testing.
The problem here is how did you design your service? Does your service flow transactions? Transaction flow allows starting transaction at your client and pass it to the service (distributed transaction). It is not default behavior and it requires special configuration of WCF bindings. If you use this approach you can do the same in your test. Start transaction at test and rollback the transaction at the end of the test. If you didn't design service to flow transaction you simply can't use it because your transaction started in the test will not affect the service. In that case you have several choices:
Create manual compensation. At the end of each test run custom SQL to move data to initial state. This simulates rollback. I don't recommend this approach.
Recreate database at the beginning of each test. This is slow but fully acceptable because integration tests are usually run only on build server few times per day.
Don't test WCF service level. WCF service should be only some wrapper on the top of business logic or data access logic. So don't test service level but instead test the wrapped layer. You can probably use transactions there. This approach can be well combined with previous one so that you have some small set of complex integration tests which requires database recreation and some bigger set of tests which can do rollback and use the same database.