I'm currently working on an IRC-Chat and we want to add the option to chat with other people privately (User-To-User) which works fine, but the messages aren't stored, meaning that a user loses all private messages after disconnecting. They also can't message a person once they have disconnected.
All of this isn't the case with channels, where messages are stored for X time, allowing a asynchronous communication.
Is there a way of allowing asynchronous messaging for private User-To-User messages without storing the messages in an extra system? Or is this simply a limitation of IRC?
IRC isn't designed to store messages - it's a feature, not a bug. One way to get around this is to configure the server to 'simulate' past messages (essentially pretending to be the other user to send past messages). To do this, you would need to store messages - however, it would be on the server itself, not on a separate database or a third party.
Related
We have a simple application, who upon every update of an entity sends out a notification to SNS(it could very well have been any other queuing system). Clients are listening to these notifications and they do a get of updated entity based on these notifications.
The problem we are facing is, when clients do a get, sometimes data is not replicated and we return 404 or sometimes stale data(even worse).
How can we mitigate this while sending notifications?
Here are Few strategies to mitigate this with pros and cons
Instead of sending notification from application send notification using database streams
For example dynamodb streams ans aws lambda. This pattern can be useful in the case of multiregion deployment as well. where all the subscriber, publisher will subscribe to their regional database streams. And also atomicity of sending message and writing to database is preserved. And we wont loose events in the case of regional failure.
Send delayed messages to your broker
Some borkers like activemq and sqs support this functionality, but SNS does not. A workaround for that could be writing to sqs queue which then writes to sns. This might be a good option when your database does not support streams.
Send special error code for retry-able gets
Since we know that eventual consistency is there we can return special error code to clients, so that they can retry based on this error code. The retry strategy should be exponential backoff. but this may mean giving away your problems to clients. Also we should have some sort of versioning in place.
Fetch from another region
If entity is not found in the same region application can go to another region or master database to fetch it. NOTE Don't do this. as it is an anti pattern. I am mentioning it here just for the sake of completion.
Send the full entity in message
If entities to be fetched by rest service is small and there are no security constrain around who can access what, we can send the full entity in message. This is ensure that client don't have to do explicit fetch of it every time a new message is arrived.
The SignalR server hub does not retain a list of groups so I assume that when I send out a broadcast message to a group it is sent to everyone connected to the hub and the clients filter out the message based on their participating groups. If I send a message to a specific group it seems that it is sent down the wire to everyone and can be intercepted by someone not in the specified group who knows how to use a browser debugger. If I want to have a secure broadcast (not even sent over the wire to some clients) I assume I need separate hubs or do I need separate sub domains?
Separate hubs do not sound like a good idea. While you probably can validate users on connect and refuse connections for non-authorized users what if you need a way to create isolated hubs dynamically. If however this functionality is enough for you you can use the Authorize attribute to secure either hubs or hub methods.
For cases where you do need to replicate the same functionality for different groups of users you can use SignalR groups but you need to verify that the user belongs to the group each time he connects. The SignalR documentation contains an example on how to do that verification. Note that everyone can try to connect to your groups and you should validate upon connection and not depend on the fact that you do not call the add method for a certain client.
SignalR only sends to the clients in the group. It's just a design issue that you can't retrieve a list of connections in a group. Groups, as pretty much everything else in SignalR, are subscription based, so the server knows which connections need to receive a message, but it's buried deep in the internals.
I would point you to the relevant server-side code, but don't have time to look it up right now.
It's easy to see in the client-side code though since it's not a lot of code - so you can verify there is no filtering going on there.
Groups also aren't separated per hub.
Could you validate my approach for using Firebase to manage a user notification system?
Basically I want to have user specific channels as well as more general channels which hold notifications. These notifications would appear on an intranet if the user has not viewed them before.
The idea being a server side action will update Firebase endpoints using the REST API either for a specific user or broadcast to everyone. The specific user messages I can easily mark as read and therefore not show them again, its the general broadcast I am struggling slightly with.
I could add a flag(user ID) to the general broadcast to indicate its read but I am concerned about performance as the client would have to check historic broadcast messages for the existence of this flag. I could add a user id to create a new endpoint which should be quicker.
e.g. /notification/general/ - contains the message, this triggers the client which then checks to see if /users/USERID/MessageID exists if it doesnt display the message and create this end point.
Is there something I am missing or is that the best approach?
Are the messages always consumed in-order? If so you could have each client remember the ID of the last message it read in each public channel. You could then use "startAt" on the queue to limit it to only new messages.
If they're not consumed in order, then you'll need some way of storing data about which ones were read and which ones weren't. Perhaps you can have each message get sent out to everyone's personal queue, and then have each user remove read messages.
Since there are already undividual user messages, why not just deliver the broadcasts to everyone individually (think email) rather than trying to store a single copy and figure out who read it.
To reduce bulk, you could store the message content separately, and simply store the ids in a user's queue. Then when they are viewed, you flag them per-user without any additional complexity.
At 100k of users receiving 100 messages a day including broadcasts, with a standard firebase ID (around 20 characters), that comes out to 210,000,000 characters a year (i.e. nothing for a database, and probably still far less than the actual bulk of storing the message body), assuming they never expire and get deleted.
I'm considering updating an existing site to use SignalR. My site polls a third party service for data changes, does some magic on it, and clients poll it once every few minutes to refresh their view with any updates.
SignalR seems like a great way to eliminate the polling from the client, but I want to know how SignalR handles dropped & reconnected connections, especially with regards to mobile web apps which may have been suspended for some time. Will it automatically negotiate and queue up any updates that were missed in the meantime, or does the client need to resynch from scratch in these cases? I looked but couldn't find any docs on this so guidance would be appreciated.
All this is definitely possible since the client keeps track of the last message id it saw. If it happened to miss messages, it'll get those the next time it goes back to the server (asking for all messages since the last one it saw).
By default the server side of SignalR stores messages in memory (and it purges those every few seconds), but you can change it to persist to some persistent store (see IMessageStore) if you're thinking about clients going offline and catching up.
You could even persist messages yourself in your own app logic while SignalR stores stuff in memory. It really depends on the application.
We haven't added any special support for mobile clients, but you can persist the message id in whatever local storage you need to for your mobile client.
Those details aren't very specific but what you want to do is all possible with SignalR.
Read Understanding and Handling Connection Lifetime Events in SignalR, especially these sections:
How to continuously reconnect - required to recover from a disconnected state;
How to notify the user about disconnections - so your app can not only inform the user, but detect state changes (disconnected, reconnecting, reconnected) to refresh your app's state in other ways.
That document was written in 2014 and basically obsoletes many of the wrong or incomplete StackOverflow SignalR-related questions/answers from the 2011-2012 era.
Is there any mail queue concept in ASP.NET?
I want to send thousands of different mail to thousands of users (i.e. each user will have a different mail). I want to send the mail at a particular time, so each user receives it at a constant time.
There really is not mail queue in the Core framework. You can send individual messages synchronously or asynchronously, but you can't really send a bunch at once.
You can queue your messages by storing them to a database or file server and then kicking off a job to loop through your saved messages and send them off.
Also, not all of your users will receive the messages at the same time, even if you could send them at the same time. There are too many external variables and dependencies (network traffic, mail server loads, spam filters) to accurately predicate when or even if your users receive their messages.
There's no native MailQueue concept within .NET framework. The queue will have to be implemented yourself. In your case, you would like the mails for each recipients to be sent at about the same time for all batches. Am I right?
Well, this is a bit tricky. You can use any SMTP server, localhost or external ones. But that also mean although you can dispatch to the SMTP server at a specific time, there's no guarantee it will reach the recipients immediately.
There are a whole bunch of stuff on mail delivery which are not exactly programming related (grey listing, spam filtering etc etc).
The alternative is to have full control on the sending and have your app directly sending the mails to the recipients' mail servers. Well that is workable and I suggest you use a commercial or a good open source component for that. Anyhow, there's still a whole bunch of issues you need to deal with, (e.g. some receiving mail server like Yahoo might block the sending a few times and let it through after a few retries).
I've posted a related question, take a look at the replies here.