I need to create a web UI for a client company to allow them to monitor which subscribers (Microservices) are up and if they are part of a worker queue, how many of them are running.
Although this is easy to implement by keeping sending a signal to another instance of NATS (to minimize overheads) for monitoring, I would like to ask if there is any built-in feature that can provide this functionality.
This link suggests that there is a monitoring endpoint built-in but I am not sure if I can get what I need out of it. So do I have to start inventing the wheel, or is there something that can be used out of the box?
Related
I am building a microservice application, and I try to follow the best practices. I use event sourcing and event driven state transfer in many places, but I realized that sometimes I just need to call another service in an asynchronous way to kindly ask it to do something e.g. send out a registration email (as the email service is a technical component and not a domain). I noticed that many times, services just call the other's service API endpoint, but that wouldn't be asynchronous. As I don't expect any returned value when calling from another service, the command would only produce events, RPC is not necessary.
In the end, my plan is to implement commands/actions that can be triggered by clients from REST API (then the commands may also produce responses) or by events or other services from RabbitMQ or similar. This leads me to how should I define the data structure of the command/action, is there any specification for that? Or existing solutions for Python? Or should I do something differently?
I am in the process of delivering a product.
What my product needs to do is spin up and down multiple instances on google cloud.
Basically, I have a main instance which I plan to control the rest of them. A 'worker factory' if you will.
What would be a good way to communicate results from the workers back to the main instance.
The actually worker already inserts its results to a sql table also hosted by google, before spinning down.
However I am looking to additionally send off a log file to the main instance.
What would be the best way of doing this?
p.s - I am not extremely familiar with networking, however I am looking to learn
There are likely many ways to achieve your goal. One thought is to use Publish/Subscribe. In this concept, you create a resource called a topic and your main instance would register a subscription to the topic. Now, when your workers wish to transmit information they would executed a publish on the topic. There is no networking knowledge needed to make this all work. The topic becomes a centralized drop-off for information published.
Here are the docs on pub/sub: https://cloud.google.com/pubsub/docs/
Before I get to my question, let me sketch out a sample set of microservices to illustrate my dilemma.
Scenario outline
Suppose I have 4 microservices:
An activation service where features supplied to our customers are (de)activated. A registration service where members can be added and changed. A secured key service that is able to generate secure keys (in a multi step process) for members to be used when communicating with them with the outside world. And a communication service that is used to communicate about our members with external vendors.
The secured key service may however only request secured keys if this is a feature that is activated. Additionally, the communication service may only communicate about members that have a secured key AND if the communication feature itself is activated.
Because they are microservices, each of the services has it's own datastore and is completely self sufficient. That is, any data that is required from the other microservices is duplicated locally and kept in sync by means of asynchronous messages from the other microservices.
The dilemma
I'm actually facing two main dilemma's. The first is (pretty obviously) data synchronization. When there are multiple data stores that need to be kept in sync you have to account for messages getting lost or processed out of order. But there are plenty of out of the box solutions for this and when all fails you could even fall back to some kind of ETL process to keep things in sync.
The main issue I'm facing however is the actions that need to be performed. In the above example the secured key service must perform an action when it either
Receives a message from the registration service for a new member when it already knows that the secured keys feature is active in the activation service
Receives a message from the activation service that the secured keys feature is now active when it already knows about members from the registration service
In both cases this means that a message from the external system must lead to both an update in the local copy of the data as well as some logic that needs to be processed.
The question
Now to the actual question :)
What is the recommended way to cope with either bugs or new insights when it comes to handling those messages? Suppose there is a bug in the message handler from the activation service. The handler does update the internal data structure, but it fails to detect that there are already registered members and thus never starts the secure key generation process. Alternatively it could be that there's no bug, but we decide that there is something else we want the handler to do.
The system will have no reason to resubmit or reprocess messages (as the message didn't fail), but there's no real way for us to re-trigger the behavior that's behind the message.
I hope it's clear what I'm asking (and I do apologize if it should be posted on any of the other 170 Stack... sites, I only really know of StackOverflow)
I don't know what is the recommended way, I know how this is done in DDD and maybe this can help you as DDD and microservices are friends.
What you have is a long-running/multi-step process that involves information from multiple microservices. In DDD this can be implemented using a Saga/Process manager. The Saga maintains a local state by subscribing to events from both the registration service and the activation service. As the events come, the Saga check to see if it has all the information it needs to generate secure keys by submitting a CreateSecureKey command. The events may come in any order and even can be duplicated but this is not a problem as the Saga can compensate for this.
In case of bugs or new features, you could create special scripts or other processes that search for a particular situation and handle it by submitting specific compensating commands, without reprocessing all the past events.
In case of new features you may even have to process old events that now are interesting for your business process. You do this in the same way, by querying the events source for the newly interesting old events and send them to the newly updated Saga. After that import process, you subscribe the Saga to these newly interesting events and the Saga continues to function as usual.
The project I am working on is moving from an n-tier to a SOA architecture so I have been reading up on good SOA practices. I'm struggling to understand the dynamic between avoiding RPC style services in favor of event driven services, and the requirement of User Interfaces to retrieve data and do it speedily.
So for instance, ideally a SOA architecture would be composed of repeatable business process wherein you could simply publish a message onto an ESB which would handle finding the services that handle that message. So rather than executing a procedure called "Setup New User" which set out to do all the tasks related to new user setup, you would publish a message into the ESB that just contained the new user's details and had the appropriate document type "New User" and then the ESB would find services that handled that event that would then do whatever domain specific new user provisioning was required.
However, sometimes you just need data. Maybe you have a page that shows some list of user associated data. You can't just fire off a message into the ESB because you need data back and you need it now. Also, you aren't really triggering any business processes; you're just retrieving data from previously invoked business processes (the processes that caused the user to be associated with the data for instance). So to give a concrete example, maybe I just want to see the list of 10 Netflix movies a user has watched recently.
How do you reconcile these disparate types of services in a single SOA system?
In an ESB, where event-driven approach is followed, you have all kinds of listeners, that detect events and act accordingly. These listeners may wait for the appearance of direct messages via some protocol at certain endpoint for example. No matter what the trigger is - a purely business event that starts a business process or a technical call that just needs to retrieve some data, it is still an event that is handled by the ESB. So you are not technically breaking the event-driven approach - it is enforced by your ESB solution. Moreover keep in mind SOA doesn't impose such limitation - you do not have to implement everything in event driven manner.
In your case (provided, you don't have a dedicated BPM solution in place), I'd identify and implement two kinds of services on two purely conceptual layers in the ESB:
Technical services (the event is an incoming direct message for retrieval/modification of data), that can be either called directly by another system (via the ESB) or called by other process services.
Process services on the top (business) layer that are being triggered in a event-driven way (using topic queue for example, where process services listen for their triggering event)
However, this may not be the most optimal approach. I've been discussing business processes in a dedicated business process layer versus process services in the ESB in this topic. Feel free to check it out, because it is kind of related with your question.
I have two asp.net applications, one of which is a publisher of data, one is a subscriber. It's possible there will be more subscribers too. The list is not anonymous, I know my applications.
What would be the best strategy to notify the subscribers when there is data in the publisher?
There are several message brokers and queueing systems on the internet, most are hard to implement and require code that I don't really find desirable (for instance running a service on the machines of the applications, where the services do the communication).
I don't mind running a server somehow that brokers messages, like a hub, but I would prefer a hub that offers an HTTP API I can call from my publisher, and that calls an HTTP page on the subscribers (kind of like PubSubHubbub but that appears to be very much about RSS feeds).
It's a bit unfeasable to write my own queueing system because there are some hard problems in that as well. Also calling the list of subscribers from publisher directly will cause some performance issues that I also want to avoid.
Do you know of solutions I can try?
PS: if this message is better suited for Programmers#StackExchange, feel free to move it!