I recently encountered a question where a person asked me what would you do in the following scenario:
You have service A, service B, and service C interacting with one another. Service A can only perform its full functionality if it receives response from B and C. However, C has a lot of requests queued and will take a long time to respond. How would service A handle this scenario? Will service A wait and wait until C will respond even after getting the response from B? How will you make this architecture faster?
I see a fundamental issue with this statement:
Service A can only perform its full functionality if it receives response from B and C
If your components (services) are not autonomous you have a serious flaw in the initial design. When decomposing a system you want to end up with a logical boundary that allows each "Service"/"Component" autonomous and explicitly a technical authority of the part of the system it owns.
So Service A will do it's job and publish an event that Service B subscribes to and will do it's bit and then B will publish an event that C will subscribe to an ddo it's portion of the business process.
If they can do the work in parallel then the creator (the client who send the initial commands with the data) can send the command directly to each component and you might have a component tracking the business process state by subscribing to events the component would publish when they complete their units of work.
The actual design is very context dependent so it would be hard to decompose the domain without the specific details.
Does that make sense?
Your C service is a bottleneck of your system. Solutions are:
Scale C service. You should have a load balancer and more C service nodes.
Use cache for C service results.
Change your architecture with splitting C logic into 2 microservices.
Related
We are designing a new system, which will be based on microservices.
I would like to consult, whether it is considered an anti-pattern to produce and consume from the same queue.
The service should be a REST-based microservice.
The backend microservice should handle large-scale operations on IoT devices, which may not always be connected to the internet, the API must be asynchronous and provide robust features such as the number of X retries before final failure, etc.
The API should immediately return a UUID with 201 responses from the POST request and expose 'Get Current Status' (pending, active, done.. etc.) of the operation status of the IoT device, by accepting the UUID in the get request.
We've thought about two solutions (described at a high level) for managing this task:
Implement both API GW Microservice and the logic handler Microservice.
The API GW will expose the REST API methods and will publish messages via RabbitMQ that will be consumed by multiple instances of the logic handler microservice.
The main drawback is that we will need to manage multiple deployments, and keep consistency between the APIs exposed in service A, to the consumer in service B.
Implement one microservice, that exposes the relevant APIs, and in order to handle the scale and the asynchronous operations, will publish the request to own queue over RabbitMQ, being consumed by the same microservice at a background worker.
The second solution looks easier to develop, maintain, and update because all the logic including the REST API's handling will take care of the same microservice.
To some members of the team, this solution looks a bit dirty, and we can't decide whether it is an anti-pattern to consume messages of your own queue.
Any advice will be helpful.
I would like to consult, whether it is considered an anti-pattern to produce and consume from the same queue.
That definitely sounds suspicious, not that I have a lot of experience with queues. But, you also talk about microservices and "producing" & consuming from those - that sounds fine, there's no reason why a microservice (and by extension, it's API) can't do both. But then I'm a bit confused because in reading the rest of the question I don't really see how that issue is a factor.
Having some kind of separation between the API and the microservice sounds wise, because you can change the microservices implementation without affecting callers, assuming it's a non-breaking change. It means you have the ability to solve API / consumer problems, and backend problems, separately.
Version control is just a part of good engineering practice, and probably not an ideal reason to bend your architecture. You should be able to run more than one version in parallel - a lot of API providers operate a N+2 model, where they support the current version, plus the last two (major) releases. That way you give your consumers a reasonable runway for upgrading.
As long as you keep the two concerns modular so you'd be able to separate them when it would make sense it doesn't matter.
I'd think in the longer term you'd probably want to treat them as two aspects of the same service as they'd probably have different update cycle (e.g. the gateway part may need things like auth, maybe additional api in gRPC, etc.),different security reqs (one can accessible to the outside where the other consumes internal resource) different scalability concerns (you'd probably need more resources for the processing) etc.
I am trying to implement a server which has to offer REST APIs. As the time goes by I may have to add new REST APIs based on the need of that hour. Well I can do it with a simple spring REST API service where I can add the new API and re-deploy the application to server.
But it would have been nicer if I could just go on adding APIs to the server whenever there is a need without even stopping the server ! Is it even possible ?
I would appreciate any input on this topic.
Not familiar with Spring, but surely REST APIs can be written in Python and once running can be served through Tomcat as with any other HTTP server.
The question is, why don't you want to re-start the server? Is it because you are afraid of downtime and missing out on some requests?
If so probably you could adopt one of these two strategies (laid out in layman's terms):
Load Balancing: You have 2+ servers running in a cluster behind a common HTTP server (let's say A, B, C and D, for example): bring offline A and B, update them, bring them back online while bringing offline C and D, update them, and bring them all back online.
Blue/Green: Similar to the previous one but with 2 clusters (one active and one idle - could be just 1 server per cluster, doesn't matter): update the idle one, swap it with the currently active one (i.e. channel all traffic from one to the other using the HTTP server).
We are using Micro services architecture where top services are used for exposing REST API's to end user and backend services does the work of querying database.
When we get 1 user request we make ~30k requests to backend service. We are using RxJava for top service so all 30K requests gets executed in parallel.
We are using haproxy to distribute the load between backend services.
However when we get 3-5 user requests we are getting network connection Exceptions, No Route to Host Exception, Socket connection Exception.
What are the best practices for this kind of use case?
Well you ended up with the classical microservice mayhem. It's completely irrelevant what technologies you employ - the problem lays within the way you applied the concept of microservices!
It is natural in this architecture, that services call each other (preferably that should happen asynchronously!!). Since I know only little about your service APIs I'll have to make some assumptions about what went wrong in your backend:
I assume that a user makes a request to one service. This service will now (obviously synchronously) query another service and receive these 30k records you described. Since you probably have to know more about these records you now have to make another request per record to a third service/endpoint to aggregate all the information your frontend requires!
This shows me that you probably got the whole thing with bounded contexts wrong! So much for the analytical part. Now to the solution:
Your API should return all the information along with the query that enumerates them! Sometimes that could seem like a contradiction to the kind of isolation and authority over data/state that the microservices pattern specifies - but it is not feasible to isolate data/state in one service only because that leads to the problem you currently have - all other services HAVE to query that data every time to be able to return correct data to the frontend! However it is possible to duplicate it as long as the authority over the data/state is clear!
Let me illustrate that with an example: Let's assume you have a classical shop system. Articles are grouped. Now you would probably write two microservices - one that handles articles and one that handles groups! And you would be right to do so! You might have already decided that the group-service will hold the relation to the articles assigned to a group! Now if the frontend wants to show all items in a group - what happens: The group service receives the request and returns 30'000 Article numbers in a beautiful JSON array that the frontend receives. This is where it all goes south: The frontend now has to query the article-service for every article it received from the group-service!!! Aaand your're screwed!
Now there are multiple ways to solve this problem: One is (as previously mentioned) to duplicate article information to the group-service: So every time an article is assigned to a group using the group-service, it has to read all the information for that article form the article-service and store it to be able to return it with the get-me-all-the-articles-in-group-x query. This is fairly simple but keep in mind that you will need to update this information when it changes in the article-service or you'll be serving stale data from the group-service. Event-Sourcing can be a very powerful tool in this use case and I suggest you read up on it! You can also use simple messages sent from one service (in this case the article-service) to a message bus of your preference and make the group-service listen and react to these messages.
Another very simple quick-and-dirty solution to your problem could also be just to provide a new REST endpoint on the articles services that takes an array of article-ids and returns the information to all of them which would be much quicker. This could probably solve your problem very quickly.
A good rule of thumb in a backend with microservices is to aspire for a constant number of these cross-service calls which means your number of calls that go across service boundaries should never be directly related to the amount of data that was requested! We closely monitory what service calls are made because of a given request that comes through our API to keep track of what services calls what other services and where our performance bottlenecks will arise or have been caused. Whenever we detect that a service makes many (there is no fixed threshold but everytime I see >4 I start asking questions!) calls to other services we investigate why and how this could be fixed! There are some great metrics tools out there that can help you with tracing requests across service boundaries!
Let me know if this was helpful or not, and whatever solution you implemented!
Requirement: I need to create a background worker/task that will get data from an external source ( message queue) at certain intervals ( i.e. 10s) and update a database. Need to run non stop 24hrs. An ASP.NET application is placing the data to the message queue.
Possible solutions:
Windows service with timer
Pros: Takes load away from web server
Cons: Separate deployment overhead, Not load balanced
Use one of the methods described here : background task
Pros: No separation deployment required, Can be load balanced - if one server goes down another can pick it up
Cons: Overhead on web server (however, in my case with max 100 concurrent users and seeing the web server resources are under-utilized, I do not think it will be an issue)
Question: What would be a recommended solution and why?
I am looking for a .net based solution.
You shouldn't go with the second option unless there's a really good reason for it. Decoupling your background jobs from your web application brings a number of advantages:
Scalability - It's up to you where to deploy the service. It can share the same server with the web application or you can easily move it to a different server if you see the load going up.
Robustness - If there's a critical bug in either the web application or the service this won't bring the other component down.
Maintanance - Yes, there's a slight overhead as you will have to adjust your deployment process but it's as simple as copying all binaries from the output folder and you will have to do it once only. On the other hand, you won't have to redeploy the application thus brining it down for some time if you just need to fix a small bug in the service.
etc.
Though I recommend you to go with the first option I don't like the idea with timer. There's a much simpler and robust solution. I would implement a WCF service with MSMQ binding as it provides you with a lot of nice features out of the box:
You won't have to implement polling logic. On start up the service will connect to the queue and will sit waiting for new messages.
You can easily use transaction to process queue messages. For example, if there's something wrong with the database and you can't write to it the message which is being processed at the moment won't get lost. This will get back to the queue to be processed later.
You can deploy as many services listening to the same queue as you wish to ensure scalability and availability. WCF will make sure that the same queue message is not processed by more than one service that is if a message is being processed by service A, service B will skip it and get the next available message.
Many other features you can learn about here.
I suggest reading this article for a WCF + MSMQ service sample and see how simple it is to implement one and use the features I mentioned above. As soon as you are done with the WCF service you can easily host it in a windows service.
Hope it helps!
Our website has a long running calculation process which keeps the client waiting for a few minutes until it's finished. We've decided we need a design change, and to farm out the processing to a windows or a WCF service, while the client is presented with another page, while we're doing all the calculations.
What's the best way of implementing the service though?
We've looked at background worker processes, but it looks like these are problematic because if IIS can periodically shut down threads
It seems the best thing to use is either a Windows service or a WCF service. Does anyone have a view on which is better for this purpose?
If we host the service on another machine, would it have to be a WCF service?
It looks like it's difficult to have the service (whatever type it is) to communicate back to the website - maybe instead the service can update its results to a database, and the website polls that for the required results later on.
It's an open ended question I know, but does anyone have any ideas?
thanks
I don't think that the true gain in terms of performance will come from the design change.
If I were to chose between windows service and WCF I would go with the Windows service because I would be able to fix an affinity and prioritize as I want. However I will have to implement the logic for serving multiple clients in the same time (which in a WCF service approach will be handled by IIS).
So in terms of performance if you use .NET framework for both the WCF service and Windows service the performance difference will not be major. Windows service would be more "controllable", WCF would be more straight-forward and with no big performance penalties.
For these types of tasks I would focus on highly optimizing the single thread calculation. If you have a complex calculation, can it be written in native code (C or C++)? You could make a .DLL file that is highly optimized and is used by either the Windows service or the WCF service. Using this approach will allow you to select best compiler option and make best use of your machine resources. Also nothing stops you from creating multiple threads in the .DLL function.
The link between the website and the service can be ensured in both cases: through sockets for Windows service (extra code for creating the protocol) or directly through SOAP for the WCF. If you push the results in a database the difficulty would be letting the website (and knowing to wich particular user session) know that the data is there.
So that's what I would do.
Hope it helps.
Cheers!
One way to do this is:
The Client submits the calculation request using a Call to a WCF Service (can be hosted in IIS)
The calculation request is stored in a database With a unique ID
The ID is returned to the Client
A Windows Service (or serveral on several different machines) poll the database for New requests
The Windows service performs the calculation and stores the result to a result table With the ID
The Client polls the result table (using a WCF service) With the ID
When the calculation is finished the result is returned to the client