All i am using a quartz schedular for scheduling a job in an asp.net mvc application.This schedular schedules a job after fixed interval of time.
http://quartznet.sourceforge.net/
The service i have created basically runs every minute.It reads the message from the
message que(database in my case) every 1min , sends an email and updates the message sent status
to true.
I am having some problems though.TO be specific the problem is the service sends the same email twice because of the reasons mentioned below.
In some cases the service gets called as soon as an email is send before the db update happens.As The database update does not happen after sending email and service is invoked again,the processed message is again read from the database as unread message and gets resent.
The same message is read again from database.Thus the service ends of sending same message twice.
How do i handle this case in my code.
public void Execute(JobExecutionContext context)
{
List<QueuedEmail> lstQueuedEmail =
_svcQueuedEmail.Filter((x => x.IsSent == false)).Take(NO_OF_MAILS_TO_SEND).ToList();
if (lstQueuedEmail.Count > 0)
{
foreach (var queuedEmail in lstQueuedEmail)
{
try
{
bool emailSendStatus = false;
emailSendStatus = EmailHelper.SendEmail(queuedEmail.From, queuedEmail.To, queuedEmail.Subject,
queuedEmail.Body, queuedEmail.FromName);
QueuedEmail objQueuedEmail =
_svcQueuedEmail.Filter(x => x.Id == queuedEmail.Id).FirstOrDefault();
if (emailSendStatus)
{
objQueuedEmail.IsSent = true;
objQueuedEmail.SentOnUtc = DateTime.UtcNow;
}
else
{
objQueuedEmail.IsSent = false;
if (objQueuedEmail.SentTries == null)
{
objQueuedEmail.SentTries = 1;
}
else
{
objQueuedEmail.SentTries += 1;
}
}
_svcQueuedEmail.Update(objQueuedEmail);
}
catch (Exception)
{
//log error
}
}
}
}
Assuming you have two states for an email: "Pending" and "Sent".
You should add a third an intermediary state called "Sending" and as soon as you read the email from the Queue you should change it's status to something like "Executing" so other threads/services won't get it again.
Related
We use PushSharp 4.0.10 to send iOS Push Notifications:
https://github.com/Redth/PushSharp
Recently we recieved this email from Apple Developer:
"If you still send push notifications with the legacy binary protocol, it's time to update to the HTTP/2-based Apple Push Notification service (APNs) provider API. You'll be able to take advantage of great features, such as authentication with a JSON Web Token, improved error messaging, and per-notification feedback.
To give you additional time to prepare, the deadline to upgrade to the APNs provider API has been extended to March 31, 2021. We recommend upgrading as soon as possible, as APNs will no longer support the legacy binary protocol after this date."
My question is: Will PushSharp 4.0.10 still work after March 31, 2021?
There is a discussion about this but the thread was closed. But there are still some suggestions on this thread that you might want to try.
The Apple Push Notification service (APNs) will no longer support the legacy binary protocol as of November 2020
https://github.com/Redth/PushSharp/issues/923
**
EDIT - 25th March 2021
The deadline is close and #Ashita Shah asked some code snippet so I hope the following can save your time.
Add the following class dotAPNSService to your project. You can customise this structure according to your needs. Also I didn't focus the best of best coding C# standards when implementing my own push notification service. You can implement LINQ, Tasks async etc. I tested this dotAPNS library and it works perfectly fine. For Android you can still use PushSharp.
Before you implement the dotAPNSService helper class, get the following from your Apple developer account. The ApnsJwtOptions values should be:
BundleId - your app’s bundle ID. Should not include specific topics (i.e. com.myapp but not com.myapp.voip).
CertFilePath - path to the .p8 certificate you have downloaded from the Developer Center.
KeyId - The 10-character Key ID you obtained from your developer account
TeamId - The 10-character Team ID you use for developing your company’s apps. Obtain this value from your developer account.
public class dotAPNSService : IDisposable
{
public event EventHandler OnTokenExpiredHandler;
private ApnsJwtOptions options = null;
public dotAPNSService()
{
options = new ApnsJwtOptions()
{
BundleId = "com.xx.xxxx",
CertFilePath = "../../certificate.p8",
KeyId = "The_Key_Id",
TeamId = "The_Team_Id"
};
}
public void SendNotifications(String[] deviceTokens, String title, String body)
{
if (deviceTokens == null || deviceTokens.Length <= 0)
{
return;
}
if (String.IsNullOrEmpty(title))
{
return;
}
if (String.IsNullOrEmpty(body))
{
return;
}
// once you've gathered all the information needed and created an options instance, it's time to call
var apns = ApnsClient.CreateUsingJwt(new HttpClient(), options);
// start the process
foreach (String deviceToken in deviceTokens)
{
var push = new ApplePush(ApplePushType.Alert)
.AddAlert(title, body)
.AddToken(deviceToken);
Send(apns, push, deviceToken);
}
}
public void SendSilentNotifications(String[] deviceTokens)
{
try
{
if (deviceTokens == null || deviceTokens.Length <= 0)
{
return;
}
// once you've gathered all the information needed and created an options instance, it's time to call
var apns = ApnsClient.CreateUsingJwt(new HttpClient(), options);
// start the process
foreach (String deviceToken in deviceTokens)
{
var push = new ApplePush(ApplePushType.Background)
.AddContentAvailable()
.AddToken(deviceToken);
Send(apns, push, deviceToken);
}
}
finally
{
}
}
private void Send(ApnsClient apns, ApplePush push, String deviceToken)
{
try
{
var response = apns.SendAsync(push);
if (response.Result.Reason == ApnsResponseReason.Success)
{
// the notification has been sent!
}
else
{
Boolean removeToken = false;
switch (response.Result.Reason)
{
case ApnsResponseReason.BadDeviceToken:
removeToken = true;
break;
case ApnsResponseReason.TooManyRequests:
break;
}
// remove the token from database?
if (removeToken)
OnTokenExpired(new ExpiredTokenEventArgs(deviceToken));
}
}
catch (TaskCanceledException)
{
// ERROR - HTTP request timed out, you can use the deviceToken to log the error
}
catch (HttpRequestException ex)
{
// ERROR - HTTP request failed, you can use the deviceToken to log the error
}
}
protected virtual void OnTokenExpired(ExpiredTokenEventArgs args)
{
try
{
EventHandler handler = OnTokenExpiredHandler;
if (handler != null)
{
ISynchronizeInvoke target = handler.Target as ISynchronizeInvoke;
if (target != null && target.InvokeRequired)
target.Invoke(handler, new object[] { this, args });
else
handler(this, args);
}
}
catch (Exception ex)
{
}
}
}
These are the namespaces of the dotAPNSService helper class:
using System;
using System.ComponentModel;
using System.Net.Http;
using System.Threading.Tasks;
using dotAPNS;
In order to use the dotAPNSService helper on your project just pull the tokens from the database and then pass them to it. For instance, to send silent notifications:
public void SendScheduledSilentNotifications()
{
try
{
IList<User> users = _accountService.GetUsers(true);
if (users != null && users.Count > 0)
{
List<String> deviceTokens = new List<String>();
foreach (User user in users)
{
if (!String.IsNullOrEmpty(user.DeviceToken))
deviceTokens.Add(user.DeviceToken);
}
if (deviceTokens.Count > 0)
{
using (dotAPNSService service = new dotAPNSService())
{
service.OnTokenExpiredHandler += new EventHandler(OnTokenExpired);
service.SendSilentNotifications(deviceTokens.ToArray());
}
}
}
}
finally
{
}
}
To remove the expired tokens from the database you can use the following:
private void OnTokenExpired(object sender, EventArgs e)
{
if (e == null)
return;
if (e.GetType() == typeof(ExpiredTokenEventArgs))
{
var args = (ExpiredTokenEventArgs)e;
User user = _accountService.GetUserByDeviceToken(args.Token);
if (user != null)
{
user.DeviceToken = String.Empty;
Boolean success = !(_accountService.SaveUser(user) == null);
if (success)
// INFO - expired device token has been removed from database
else
// INFO - something went wrong
}
}
}
You can download the source code from here:
https://github.com/alexalok/dotAPNS
The API is now sending thousands of silent notifications at one time and there are no delays, crashes etc. Hope this code snippet helps and saves your time!
I'm using Confluent.Kafka(1.4.4) in a .netCore project as a message broker. In the startup of the project I set only "bootstrapservers" to the specific servers which were in the appSetting.json file and I produce messages in an API when necessary with the code below in related class:
public async Task WriteMessage<T>(string topicName, T message)
{
using (var p = new ProducerBuilder<Null, string>(_producerConfig).Build())
{
try
{
var serializedMessage= JsonConvert.SerializeObject(message);
var dr = await p.ProduceAsync(topicName, new Message<Null, string> { Value = serializedMessage });
logger.LogInformation($"Delivered '{dr.Value}' to '{dr.TopicPartitionOffset}'");
}
catch (ProduceException<Null, string> e)
{
logger.LogInformation($"Delivery failed: {e.Error.Reason}");
}
}
}
I have also added the following code In the consumer solution :
public async Task Run()
{
using (var consumerBuilder = new ConsumerBuilder<Ignore, string>(_consumerConfig).Build())
{
consumerBuilder.Subscribe(new List<string>() { "ActiveMemberCardForPanClubEvent", "CreatePanClubEvent", "RemovePanClubEvent"
});
CancellationTokenSource cts = new CancellationTokenSource();
Console.CancelKeyPress += (_, e) =>
{
e.Cancel = true; // prevent the process from terminating.
cts.Cancel();
};
try
{
while (true)
{
try
{
var consumer = consumerBuilder.Consume(cts.Token);
if (consumer.Message != null)
{
using (LogContext.PushProperty("RequestId", Guid.NewGuid()))
{
//Do something
logger.LogInformation($"Consumed message '{consumer.Message.Value}' at: '{consumer.TopicPartitionOffset}'.");
await DoJob(consumer.Topic, consumer.Message.Value);
consumer.Topic.Remove(0, consumer.Topic.Length);
}
}
else
{
logger.LogInformation($"message is null for topic '{consumer.Topic}'and partition : '{consumer.TopicPartitionOffset}' .");
consumer.Topic.Remove(0, consumer.Topic.Length);
}
}
catch (ConsumeException e)
{
logger.LogInformation($"Error occurred: {e.Error.Reason}");
}
}
}
catch (OperationCanceledException)
{
// Ensure the consumer leaves the group cleanly and final offsets are committed.
consumerBuilder.Close();
}
}
}
I produce a message and when the consumer project is run everything goes perfectly and the message is being read in the consumer solution.
The problem is raised when the consumer project is not run and I queue a message in the API with the message producer in API. After running consumers there is not any valid message for that topic that it's message is being produced.
I am familiar with and have experiences with message brokers and I know that by sending a message it will be on the bus until it is being used but I don't understand why it doesn't work with Kafka in this project.
The default setting for the "auto.offset.reset" Consumer property is "latest".
That means (in the context of no offsets being written yet) if you write a message to some topic and then subsequently start the consumer, it will skip past any messages written before the consumer was started. This could be why your consumer is not seeing the messages queued by your producer.
The solution is to set "auto.offset.reset" to "earliest" which means that the consumer will start from the earliest offset on the topic.
https://docs.confluent.io/current/installation/configuration/consumer-configs.html#auto.offset.reset
I am trying to use logstash to receive events from TCP socket, and output them to a Kafka topic. My current configuration is able to do that perfectly, but I want to be able to conduct events to Kafka in a transactional manner. I mean, the system should not send the events to kafka, until commit message is received:
START TXN 123 --No message sent to Kafka
123 - Event1 Message --No message sent to Kafka
123 - Event2 Message --No message sent to Kafka
123 - Event3 Message --No message sent to Kafka
COMMIT TXN 123 --Event1, Event2, Event3 messages sent to Kafka
Is there any possibility to achieve this using logstash only or should I introduce any other transaction coordinator in between source and logstash? Here is my current config:
input {
tcp {
port => 9000
}
}
output {
kafka {
bootstrap_servers => "localhost:9092"
topic_id => "alpayk"
}
}
I tried to use use logstash' s aggregate filter for this purpose, but I couldn' t end up with something works.
Thank you very much in advance
I finally decided to use Apache Flume for this purpose. I modified its netcat source so that uncommited messages reside in flume' s heap, and as soon as a commit message is received for a transaction, all messages are sent to kafka sink.
I am going to change the message storing location from flume heap to an external cache, so that I will be able to expire the stored messages if transaction abends or rolls back.
Below is my piece of code for that transaction logic:
String eventMessage = new String(body);
int indexOfTrxIdSeparator = eventMessage.indexOf("-");
if (indexOfTrxIdSeparator != -1) {
String txnId = eventMessage.substring(0, indexOfTrxIdSeparator).trim();
String message = eventMessage.substring(indexOfTrxIdSeparator + 1).trim();
ArrayList<Event> events = cachedEvents.get(txnId);
if (message.equals("COMMIT")) {
System.out.println("##### COMMIT RECEIVED");
if (events != null) {
for (Event eventItem : events) {
ChannelException ex = null;
try {
source.getChannelProcessor().processEvent(eventItem);
} catch (ChannelException chEx) {
ex = chEx;
}
if (ex == null) {
counterGroup.incrementAndGet("events.processed");
} else {
counterGroup.incrementAndGet("events.failed");
logger.warn("Error processing event. Exception follows.", ex);
}
}
cachedEvents.remove(txnId);
}
} else {
System.out.println("##### MESSAGE RECEIVED: " + message);
if (events == null) {
events = new ArrayList<Event>();
}
events.add(EventBuilder.withBody(message.getBytes()));
cachedEvents.put(txnId, events);
}
}
I added this code into processEvents method of the netcat source of Flume. I didn' t want to work with Ruby code, that' s why I decided to switch to Flume. However the same thing could also be done in logstash.
Thank you
I am trying to create a Tcp socket server that accepts multiple clients. However, for the past couple of days, I haven't been able to overcome a certain obstacle. I believe I've isolated the problem to be in the TcpClient.BeginRead(callbackMethod) Method.
Basically, distinct clients activate this method but the callback isn't invoked/triggered until they actually send data into their outgoing stream. However, the encoding.ASCII.Getstring() Method I perform on the bytes that come in via the stream outputs an unwanted "0/0/0/" depending on the order the beginread methods were started. Why is this happening? Why? Please help.
The Situation/Scenario in Order
Event 1.) ClientOne Connects which then triggers a BeginRead with asynchronous call back.(Now callback is waiting for data)
Event 2.) ClientTwo Connects which then triggers a BeginRead with asynchronous call back. (Now callback is waiting for data)
Event 3.) If ClientOne sends a message first, the data definitely is serviced, however, the Encoding.ASCII.GetString(3 arguments) outputs "0/" for every byte. I think ClientTwo's BeginRead is interfering with ClientOne's BeginRead somehow.
Event 3. (Not 4)) If ClientTwo sends a message first, the data is serviced and decoded/stringified correctly using Encoding.ASCII.GetString(3 arguments).
Source Code
void onCompleteAcceptTcpClient(IAsyncResult iar){TcpListener tcpl = (TcpListener)iar.AsyncState;
try
{
mTCPClient = tcpl.EndAcceptTcpClient(iar);
var ClientEndPoint = mTCPClient.Client.RemoteEndPoint;
Console.log(ClientEndPoint.ToString());
Console.log("Client Connected...");
_sockets.Add(mTCPClient);
tcpl.BeginAcceptTcpClient(onCompleteAcceptTcpClient, tcpl);
mRx = new byte[512];
_sockets.Last().GetStream().BeginRead(mRx, 0, mRx.Length, onCompleteReadFromTCPClientStream, mTCPClient);
}
catch (Exception exc)
{
MessageBox.Show(exc.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
void **onCompleteReadFromTCPClientStream**(IAsyncResult iar)
{
foreach (string message in messages)//For Testing previous saved messages
{
printLine("Checking previous saved messages: " + message);
}
TcpClient tcpc = new TcpClient();
int nCountReadBytes = 0;
try
{
tcpc = (TcpClient)iar.AsyncState;
nCountReadBytes = tcpc.GetStream().EndRead(iar);
printLine(nCountReadBytes.GetType().ToString());
if (nCountReadBytes == 0)
{
MessageBox.Show("Client disconnected.");
return;
}
string foo;
/*THE ENCODING OUTPUTS "0/" FOR EVERY BYTE WHEN AN OLDER CALLBACK'S DATA IS DECODED*/
foo = Encoding.ASCII.GetString(mRx, 0, nCountReadBytes);
messages.Add(foo);
foreach (string message in messages)
{
console.log(message);
}
mRx = new byte[512];
//(reopens the callback)
tcpc.GetStream().BeginRead(mRx, 0, mRx.Length, onCompleteReadFromTCPClientStream, tcpc);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message, "Error", MessageBoxButtons.OK, MessageBoxIcon.Error);
}
}
I'm using Web API to stream large files to clients, but I'd like to log if the download was successful or not. That is, if the server sent the entire content of the file.
Is there some way to get a a callback or event when the HttpResponseMessage completes sending data?
Perhaps something like this:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
// This doesn't exist, but it illustrates what I'm trying to do.
response.OnComplete(context =>
{
if (context.Success)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
EDIT: I've now tested this using a real connection (via fiddler).
I inherited StreamContent and added my own OnComplete action which checks for an exception:
public class StreamContentWithCompletion : StreamContent
{
public StreamContentWithCompletion(Stream stream) : base (stream) { }
public StreamContentWithCompletion(Stream stream, Action<Exception> onComplete) : base(stream)
{
this.OnComplete = onComplete;
}
public Action<Exception> OnComplete { get; set; }
protected override Task SerializeToStreamAsync(Stream stream, TransportContext context)
{
var t = base.SerializeToStreamAsync(stream, context);
t.ContinueWith(x =>
{
if (this.OnComplete != null)
{
// The task will be in a faulted state if something went wrong.
// I observed the following exception when I aborted the fiddler session:
// 'System.Web.HttpException (0x800704CD): The remote host closed the connection.'
if (x.IsFaulted)
this.OnComplete(x.Exception.GetBaseException());
else
this.OnComplete(null);
}
}, TaskContinuationOptions.ExecuteSynchronously);
return t;
}
}
Then I use it like so:
var stream = GetMyStream();
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContentWithCompletion(stream, ex =>
{
if (ex == null)
Log.Info("File downloaded successfully.");
else
Log.Warn("File download was terminated by client.");
});
response.Content.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
return response;
I am not sure if there is direct signaling that all is ok, but you can use a trick to find out that the connection is exist just before you end it up, and right after you fully send the file.
For example the Response.IsClientConnected is return true if the client is still connected, so you can check something like:
// send the file, make a flush
Response.Flush();
// and now the file is fully sended check if the client is still connected
if(Response.IsClientConnected)
{
// log that all looks ok until the last byte.
}
else
{
// the client is not connected, so maybe have lost some data
}
// and now close the connection.
Response.End();
if the server sent the entire content of the file
Actually there is nothing to do :)
This might sound very simplistic but you will know if an exception is raised - if you care about server delivering and not client cancelling halfway. IsClientConnected is based on ASP.NET HttpResponse not the WebApi.