I have application that should notify user based on some interval pattern like:
Event
> Pushes
Pattern: Immediately - 3 day - 7 day - 12 day
If user made action for event pushes should stops for this event. It is possible to have multiple same type events that should send push when event occurred.
Also I do not want to bother user for example when the one have 5 events to send x5 more pushes, but reduce by taking together pushes that should happens next day (or some other interval) by sending one push for example 'reminder: you have 5 events'.
So now I decide this kind of solution, when event occurred, insert into db all pushes for event that should be send later with datetime for send. If user take action, pushes marks as redundant for this event. And before sending analyze interval for example take all pushes for next 24hour, send one and mark all others as already sent.
Is it ok, or maybe exists better solutions?
I have experience building same application with you. What I'm I doing is :
CMS -> redis -> worker
CMS, are used for creating push notification content, including the time when that content should be sent
Redis, are used for storing the delayed jobs data
Worker, php application that pulling delayed jobs data from Redis. I use Laravel on here, I take advantage from Laravel queue delayed dispatching.
Previously I have try use database and message broker SQS as queue driver. Why I'm switch to redis ? First, when I using database, is too costly, due the traffic of my queue data is very huge. Then when I use SQS, it's better than database, but SQS cannot hold delayed data with weeks age. So my last choice is Redis. Of course we can use another serivce such as Rabbitmq.
Related
I am using SignalR in my web api to provide real-time functionality to my client apps (mobile and web). Everything works ok but there is something that worries me a bit:
The clients get updated when different things happen in the backend. For example, when one of the clients does a CRUD operation on a resource that will be notified by SignalR. But, what happens when something happens on the client, let's say the mobile app, and the device data connection is dropped?.
It could happen that another client has done any action over a resource and when SignalR broadcasts the message it doesn't arrive to that client. So, that client will have an old view sate.
As I have read, it seems that there's no way to know if a meesage has been sent and received ok by all the clients. So, beside checking the network state and doing a full reload of the resource list when this happens is there any way to be sure message synchronization has been accomplished correctly on all the clients?
As you've suggested, ASP NET Core SignalR places the responsibility on the application for managing message buffering if that's required.
If an eventually consistent view is an issue (because order of operations is important, for example) and the full reload proves to be an expensive operation, you could manage some persistent queue of message events as far back as it makes sense to do so (until a full reload would be preferable) and take a page from message buses and event sourcing, with an onus on the client in a "dumb broker/smart consumer"-style approach.
It's not an exact match of your case, but credit where credit is due, there's a well thought out example of handling queuing up SignalR events here: https://stackoverflow.com/a/56984518/13374279 You'd have to adapt that some and give a numerical order to the queued events.
The initial state load and any subsequent events could have an aggregate version attached to them; at any time that the client receives an event from SignalR, it can compare its currently known state against what was received and determine whether it has missed events, be it from a disconnection or a delay in the hub connection starting up after the initial fetch; if the client's version is out of date and within the depth of your queue, you can issue a request to the server to replay the events out to that connection to bring the client back up to sync.
Some reading into immediate consistency vs eventual consistency may be helpful to come up with a plan. Hope this helps!
I have 5 nodes & 1 Notary node corda private network and a web client with UI & RESTful services.
There are quite a few states and its attributes that are managed by users using the UI. I need to understand how to handle timeouts and avoid multiple updates or Errors
Scenario 1
User is viewing a specific un-consumed current state of a feature.
User performs an edit and updates the state
Upon receiving the request RESTful component use CORDA RPCClient to start the flow. It sets the timeout value e.g. 2 secs
CORDA flow runs the configured rules and it has to sync / collect signatures from all the participating nodes (4). Complete processing takes more than 2 secs (Some file processing, multiple states update etc. I can change the timeout to higher value based on specific use cases. It can surely happen anytime. Need to understand what is the recommended way of handling)
As time taken is higher than provided, CORDA RPCClient throws exception. For the RESTFul service / User transaction has failed.
Behind the scenes CORDA is processing and collecting signatures and updating nodes. From CORDA perspective everything looks fine and changed set is committed to the ledger.
Question:
Is there a way to know transaction submitted is in progress so RESTful service should wait
If user submits again we do check for transaction hash is the latest one associated with unconsumed state and reject if not (It was provided to UI while querying.
Any recommended way of handling.
Scenario 2
User viewing a specific un-consumed current state of a feature.
User performs an edit and updates the state
Upon receiving the request RESTful component use CORDA RPCClient to start the flow. It sets the timeout value e.g. 2 secs
CORDA flow runs the configured rules and it has to sync / collect signatures from all the participating nodes (4). One of the nodes is down or not reachable. Flow hangs / waits for the node to be live again.
RESTFul service / UI receives a timeout exception. User refreshes the view and submits the change again. Querying the current node will return old data and user will try to make change again and submit. Same will happen at CORDA layer transaction will be of latest unconsumed state (comparing the tx hash as state is not committed, it will proceed further and will hang / waits for the node to be live again. It waits for long time i have waited for a minute it did not quite trying.
Now the node comes up and will be syncing with peers. Notary will give exception as there are two states / requests pending to form the next state in chain. Transaction fails.
Question:
Is there a way to know transaction submitted is in progress so RESTful service should wait
Any recommended way of handling.
Is there a way to provide timeout values for node communication.
Do i need to keep monitoring if the node is active or not and accordingly tailor user experience.
Appreciate all the help and support for above issue. Please let me know if there is any additional information needed.
Timeouts
As of Corda 3.3, there is no way to set a timeout either on a Corda RPC request, a flow, or a message to another node. If another node is down when you try to contact it as part of a flow, the message will simply remain in your outbound message queue until it can be successfully delivered.
Checking flow progress
Each flow has a unique run ID. When you start a flow via RPC (e.g. using CordaRPCOps.startFlowDynamic), you get back a FlowHandle. The flow's unique run ID is then available via FlowHandle.id. Once you have this ID, you can check whether the flow is still in progress by checking whether it is still present in the list of current state machines (i.e. flows):
val flowInProgress = flowHandle.id in cordaRPCOps.stateMachinesSnapshot().map { it.id }
You can also monitor the state machine manager to wait until the flow completes, then get its result:
val flowUpdates = cordaRPCOps.stateMachinesFeed().updates
flowUpdates.subscribe {
if (it.id == flowHandle.id && it is StateMachineUpdate.Removed) {
val int = it.result.getOrThrow()
// Handle result.
}
}
Handling duplicate requests
The flow will throw an exception if you try and consume the same state twice, either when you query the vault to retrieve the state or when you try to notarise the transaction. I'd suggest letting the user start the flow again, then handling any double-spend errors appropriately and reflecting them on the front-end (e.g. via an error message and automatic refresh).
I am building a project from scratch using event-sourcing with Java and Cassandra.
My apps we be based on microservices and in some use cases information will be processed asynchronously. I was wondering what part a Message Queue (such as Rabbit, Active MQ Artemis, Kafka, etc) would play to improve the technology stack in this environment and if I understand the scenarios if I won't use it.
I would start with separating messaging infrastructure like RabbitMQ from event streaming/storing/processing like Kafka. These are two different things made for two (or more) different purposes.
Concerning the event sourcing, you have to have a place where you must store events. This storage must be append-only and support fast reads of unstructured data based on an identity. One example of such persistence is the EventStore.
Event sourcing goes together with CQRS, which means you have to project your changes (event) to another store, which you can query. This is done by projecting events to that store, this is where events get processed to change the domain object state. It is important to understand that using message infrastructure for projections is generally a bad idea. This is due to the nature of messaging and two-phase commit issue.
If you look at how events get persisted, you can see that they get saved to the store as one transaction. If you then need to publish events, this will be another transaction. Since you are dealing with two different pieces of infrastructure, things can get broken.
The messaging issue as such is that messages are usually guaranteed to be delivered "at least once" and the order of messages is usually not guaranteed. Also, when your message consumer fails and NACKs the message, it will be redelivered but usually a bit later, again breaking the sequence.
The ordering and duplication concerns, whoever, do not apply to event streaming servers like Kafka. Also, the EventStore will guarantee once only event delivery in order if you use catch-up subscription.
In my experience, messages are used to send commands and to implement event-driven architecture to connect independent services in a reactive way. Event stores, at the other hand, are used to persist events and only events that get there are then projected to the query store and also get published to the message bus.
Make sure you are clear on the distinction between send(command) and publish(event). Udi Dahan touches on that topic in his essay on busses and brokers.
In most cases where you are event sourcing, you do not want to be reconstructing state from published events. If you need state, then query the technical authority/book of record for the history, and reconstruct the state from the history.
On the other hand, event driven activity off of a message queue should be fine. When a single event (plus the subscriber's state) has everything you need, then running off of the bus is fine.
In some cases, you might do both. For example, if you were updating cached views, you'd subscribe to various BobChanged events to know when your cached data was stale; to rebuild a stale view, you would reload a representation of the history and transform it into an updated view.
In the world of event-sourcing applications, message queues usually allow you to implement publish-subscribe pattern style of communication between producers and consumers. Also, they usually help you with delivery guarantees: which messages were delivered to which subscribers and which ones were not.
But they don't store all messages indefinitely. You need to have an event store to do any kind of event sourcing.
The question is not 'to queue or not to queue', but it is more like:
can this thing store huge volume of events indefinitely?
does it have publish-subscribe capabilities?
does it provide at-least-once delivery guarantees?
So, you should use something like Kafka or EventStore to have all that out-of-the-box. Alternatively, you can combine event store with message queue manually, but this is going to be more involved.
I am building a system that processes orders. Each order will follow a workflow. So this order can be, e.g., booked,accepted,payment approved,cancelled and so on.
Every time a status of a order changes I will post this change to SNS. To know if a status order has changed I will need to make a request to a external API, and compare to the last known status.
The question is: What is the best place to store the last known order status?
1. A SQS queue. So every time I read a message from queue, check status using the external API, delete the message and insert another one with the new status.
2. Use a database (like Dynamo DB) to control the order status.
You should not use the word "store" to describe something happening with stateful facts and a queue. Stateful, factual information should be stored -- persisted -- to a database.
The queue messages should be treated as "hints" on what work needs to be done -- a request to consider the reasonableness of a proposed action, and if reasonable, perform the action.
What I mean by this, is that when a queue consumer sees a message to create an order, it should check the database and create the order if not already present. Update an order? Check the database to see whether the order is in a correct status for the update to occur. (Canceling an order that has already shipped would be an example of a mismatched state).
Queues, by design, can't be as precise and atomic in their operation as a database should. The Two Generals Problem is one of several scenarios that becomes an issue in dealing with queues (and indeed with designing a queue system) -- messages can be lost or delivered more than once.
What happens in a "queue is authoritative" scenario when a message is delivered (received from the queue) more than once? What happens if a message is lost? There's nothing wrong with using a queue, but I respectfully suggest that in this scenario the queue should not be treated as authoritative.
I will go with the database option instead of SQS:
1) option SQS:
You will have one application which will change the status
Add the status value into SQS
Now another application will check your messages and send notification, delete the message
2) Option DynamoDB:
Insert you updated status in DynamoDB
Configure a Lambda function on update of that field
Lambda function will send notifcation
The database option looks clear additionally, you don't have to worry about maintaining any queue plus you can read one message from the queue at a time unless you implement parallel reader to read from the queue. In a database, you can update multiple rows and it will trigger the lambda and you don't have to worry about it.
Hope that helps
I'm writing an application where the user will create an appointment, and instantly get an email confirming their appointment. I'd also like to send an email the day of their appointment, to remind them to actually show up.
I'm in ASP.NET (2.0) on MS SQL . The immediate email is no problem, but I'm not sure about the best way to address the reminder email. Basically, I can think of three approaches:
Set up a SQL job that runs every night, kicking off SQL emails to people that have appointments that day.
Somehow send the email with a "do not deliver before" flag, although this seems like something I might be inventing.
Write another application that runs at a certain time every night.
Am I missing something obvious? How can I accomplish this?
Choice #1 would be the best option, create a table of emails to send, and update the table as you send each email. It's also best not to delete the entry but mark it as sent, you never know when you'll have a problem oneday and want to resend out emails, I've seen this happen many times in similar setups.
One caution - tightly coupling the transmission of the initial email in the web application can result in a brittle architecture (e.g. SMTP server not available) - and lost messages.
You can introduce an abstraction layer via an MSMQ for both the initial and the reminder email - and have a service sweeping the queue on a scheduled basis. The initial message can be flagged with an attribute that means "SEND NOW" - the reminder message can be flagged as "SCHEDULED" - and the sweeper simply needs to send any messages that it finds that are of the "SEND NOW" or that are "SCHEDULED" and have a toBeSentDate >= the current date. Once the message is successfully sent - the unit of work can be concluded by deleting the message from the queue.
This approach ensures messages are not lost - and enables the distribution of load to off-peak hours by adjusting the service polling interval.
As Rob Williams points out - my suggestion of MSMQ is a bit of overkill for this specific question...but it is a viable approach to keep in mind when you start looking at problems of scale - and you want (or need) to minimize/reduce database read/write activity (esepcially during peak processing periods).
Hat tip to Rob.
For every larger project I usually also create a service which performs regular or periodical tasks.
The service updates its status and time of last execution somewhere in the database, so that the information is available for applications.
For example, the application posts commands to a command queue, and the service processes them at the schedule time.
I find this solution easier to handle than SQL Server Tasks or Jobs, since it's only a single service that you need to install, rather than ensuring all required Jobs are set up correctly.
Also, as the service is written in C#, I have a more powerful programming language (plus libraries) at hand than T-SQL.
If it's really pure T-SQL stuff that needs to be handled, there will be a Execute_Daily stored procedure that the service is going to call on date change.
Create a separate batch service, as others have suggested, but use it to send ALL of the emails.
The web app should record the need to send notifications in a database table, both for the immediate notice and for the reminder notice, with both records annotated with the desired send date/time.
Using MSMQ is overkill--you already have a database and a simple application. As the complexity grows, MSMQ or something similar might help with that complexity and scalability.
The service should periodically (every few minutes to a few hours) scan the database table for notifications (emails) to send in the near future, send them, and mark them as sent if successful. You could eventually leverage this to also send text messages (SMS) or instant messages (IMs), etc.
While you are at it, you should consider using the Command design pattern, and implement this service as a reusable Command executor. I have done this recently with a web application that needs to keep real estate listing (MLS) data synchronized with a third-party provider.
Your option 2 certainly seems like something you are inventing. I know that my mail system won't hold messages for future delivery if you were to send me something like that.
I don't think you're missing anything obvious. You will need something that runs the day of the appointment to send emails. Whether that might be better as a SQL job or as a separate application would be up to your application architecture.
I would recommend the first option, using either an SQL or other application to run automatically every day to send the e-mails. It's simple, and it works.
Microsoft Office has a delivery delay feature, but I think that is an Outlook thing rather than an Exchange/Mail Server thing, so you're going to have to go with option 1 or 3. Or option 4 would be to write a service. That way you won't have to worry about scheduled tasks to get the option 3 application to run.
If you are planning on having this app hosted at a cheap hosting service (like GoDaddy), then what I'd recommend is to spin off a worker thread in Global.asax at Application_Start and having it sleep, wake-up, send emails, sleep...
Because you won't be able to run something on the SQL Server machine, and you won't be able to install your own service.
I do this, and it works fine.