Recently we implemented a scheduled callback for our users, and it seemed to coincide with an increase in the abandoned calls metric.
As of now, right after we schedule a callback, we take the call out of the queue and then hang up.
So, the question is :
If a call is taken out of a queue, i.e.: by using , does it count as an abandoned call on the metrics?
According to the documentation:
Abandoned conversations may occur because the customer hung up or disconnected due to technical issues (for example, a lost signal while calling from a mobile device). Abandons can happen either in a queue that a customer enters from an IVR or during a transfer to another queue. Abandoned conversations do not include calls disconnected at any point during the IVR flow.
If you are hanging up the call as part of the process of setting up your callback, you should be ok, though it's hard to tell without an example of your code. It's possible that, if you have confirmed to the user that they will get a callback that they are able to hangup before your code hangs them up and that might cause an increase. You might want to check the Abandoned Phase to see where these abandonments are happening. If you are getting an increase in the Callback Request phase, you might be fine with that and just want to exclude the Abandoned value of Follow-Up when checking these metrics.
Related
I am using an Axon Event Tracking processor. Sometimes events take longer that 10 seconds to process.
This seems to cause the message to be processed again and this appears in the log "Releasing claim of token X/0 failed. It was owned by another node."
If I up the number of segments it does not log this BUT the event is still processed twice so I think this might be misleading. (I think I was mistaken about this)
I have tried adjusting the fetchDelay, cleanupDelay and tokenClaimInterval. None of which has fixed this. Is there a property or something that I am missing?
Edit
The scenario taking longer than 10 seconds is making a HTTP request to an external service.
I'm using axon 4.1.2 with all default configuration when using with Spring auto configuration. I cannot see the Releasing claim on token and preparing for retry in [timeout]s log.
I was having this issue with a single segment and 2 instances of the application. I realised I hadn't increased the number of segments like I thought I had.
After further investigation I have discovered that adding an additional segment seems to have stopped this. Even if I have for example 2 segments and 6 applications it still doesn't reappear, however I'm not sure how this is different to my original scenario of 1 segment and 2 application?
I didn't realise it would be possible for multiple threads to grab the same tracking token and process the same event. It sounds like the best action would be to put an idem-potency check before the HTTP call?
The Releasing claim of token [event-processor-name]/[segment-id] failed. It was owned by another node. message can only occur in three scenarios:
You are performing a merge operation of two segments which fails because the given thread doesn't own both segments.
The main event processing loop of the TrackingEventProcessor is stopped, but releasing the token claim fails because the token is already claimed by another thread.
The main event processing loop has caught an Exception, making it retry with a exponential back-off, and it tries to release the claim (which might fail with the given message).
I am guessing it's not options 1 and 2, so that would leave us with option 3. This should also mean you are seeing other WARN level messages, like:
Releasing claim on token and preparing for retry in [timeout]s
Would you be able to share whether that's the case? That way we can pinpoint a little better what the exact problem is you are encountering.
By the way, very likely you have several processes (event handling threads of the TrackingEventProcessor) stealing the TrackingToken from one another. As they're stealing an un-updated token, both (or more) will handled the same event. Hence why you see the event handler being invoked twice.
Obviously undesirable behavior and something we should resolve for you. I would like to ask you to provide answers to my comments under the question, as right now I have to little to go on. Let us figure this out #Dan!
Update
Thanks for updating your question #dan, that's very helpful.
From what you've shared, I am fairly confident that both instances are stealing the token from one another. This does depend though on whether both are using the same database for the token_entry table (although I am assuming they are).
If they are using the same table, then they should "nicely" share their work, unless one of them takes to long. If it takes to long, the token will be claimed by another process. This other process in this case is the thread of the TEP of your other application instance. The "claim timeout" is defaulted to 10 seconds, which also corresponds with the long running event handling process.
This claimTimeout is adjustable though, by invoking the Builder of the JpaTokenStore/JdbcTokenStore (depending on which you are using / auto wiring) and calling the JpaTokenStore.Builder#claimTimeout(TemporalAmount) method. And, I think this would be required on your end, giving the fact you have a long running operation.
There are of course different ways of tackling this. Like, making sure the TEP is only ran on a single instance (not really fault tolerant though), or offloading this long running operation to a schedule task which is triggered by the event.
But, I think we've found the issue at least, so I'd suggest to tweak the claimTimeout and see if the problem persists.
Let us know if this resolves the problem on your end #dan!
I want to pull messages off a MQS queue in a C client, and would love to do so asynchronously so I don't have to start (explicitly) multithreading. The messages will be forwarded to another system that acts "transactionally" but is completely incompatible with XA. So I'd like to have a way to explicitly commit (and thereby remove) a message that's been successfully handed off to the other system, and not commit if this failed, so that the last message is retained for a more successful later attempt.
I've read about the SYNCPOINT option and understand how I'd use that around a regular GET, but I haven's seen any hints on how to make asynchronous message retrieval have transactional behavior like this. Any hints, please?
I think you are describing using the asynchronous callback capability, ie you register a routine to be called when a message arrives, and ask for any get to be under syncpoint... An explanation of how some of it works is in here, https://share.confex.com/share/117/webprogram/Handout/Session9513/share_advanced_mqi.pdf page 4+
Effectively you get called with the MQ message under syncpoint, do your processing with another system, then commit or rollback the message before returning.
Be aware without the use of e.g. XA 2 phase commit, there is always going to be the windows of e.g. committing to the external system and a power outage means the message under the unit of work gets rolled back inside MQ as you didnt have time to perform the commit.
Edit: my misunderstanding, didn't realise that the application was using a callback to retrieve messages, which is indeed fully asynchronous behavior. Disregard the answer below.
Do MQGET with MQGMO_SYNCPOINT, then issue either MQCMIT or MQBACK.
"Asynchronous" and "synchronous" may be misnomers - these are your patterns of using MQ - whether you wait for a reply message or not, these patterns do not affect how MQ processes your calls. Transaction management (unit of work management) works across any MQI calls that use SYNCPOINT, no matter if they are part of a request/reply pattern or not.
I have a server that keeps 2 booleans. These booleans change every 15 seconds.
Whenever I wake my Apple Watch, I want the complication to show the current state. How can I do it withough exhausting the budget early on?
The best way would be to fetch the newest state into the complication whenever I wake my watch. The only possible way I see would be to poll the server (either directly or via my phone) every 15 seconds. The problem is that I'd soon use up all the allotted time.
It would be great if I could make the complication only update when the watch was woken up. Can that be done?
Is there a way to not fetch data unless you need it?
No.
By "waking the watch," you're speaking of activating the watch either by interacting with it, or by raising your wrist. Regardless of the manner, the watch can either wake to the watch face, or to the last activity (which is controlled by the Wake Screen setting).
If it wakes to the watch face, this is independent of your app, watch extension, or complication controller. There is no notification you can use to handle that scenario.
If it wakes to an activity, it may not be your activity. If it were your activity, all you could do would be to stop updating when your watch app was active.
Either way, there is no contingency to only update the complication when the watch is awake.
If you think about what you're asking, it runs contrary to Apple's guidelines, as users expect to glance at the watch face and already see current complication data. The system expects you to provide updates when the watch is not awake, so the information will be immediately visible when the watch does wake.
Updating Complication Data
Of the available update approaches, these won't handle your requirements:
PKPushTypeComplication push notifications
This would be ideal, if you were not updating frequently and constantly throughout the day.
Apple applies a daily limit to the number of pushes of this type that you send from your server. If you exceed the limit, subsequent pushes are not delivered.
Scheduled automatic updates
The issue here is that the minimum scheduled update interval is 10 minutes, so you wouldn't have current complication info for the remaining 9-3/4 minutes.
Scheduled updates are useful for apps whose data changes at predictable times. When a scheduled update occurs, ClockKit calls the requestedUpdateDidBegin or requestedUpdateBudgetExhausted method of your data source first.
These approaches might handle your requirements, but you'd have to try them and determine which one meets your needs.
Watch extension + background NSURLSession
This may place more of a drain on battery, but it would work even if not in range of the phone.
Manual update via WCSession transferCurrentComplicationUserInfo
If you're in range of your phone, the more likely approach would be for your phone to (always) poll your server, then provide constant updates.
When your iOS app receives updated data intended for your complication, it can use the Watch Connectivity framework to update your complication right away. The transferCurrentComplicationUserInfo: method of WCSession sends a high priority message to your WatchKit extension, waking it up as needed to deliver the data. Upon receiving the data, extend or reload your timeline as needed to force ClockKit to request the new data from your data source.
Note that complication transfers are budgeted in iOS 10. This approach would not work if you were performing large numbers of updates per day.
Having said all that, an alternate approach would be to monitor these booleans on the server side, and only send out a notification if there was a problem or change. You didn't explain exactly what these booleans indicate, but there are other approaches for monitoring, which would avoid you having to constantly poll a server (from your phone or watch).
If that's not an option, you really should consider either viewing the server data on your phone, or switching to a far less frequent update interval for your complication. Apple discourages such frequent updates, and even their own (stock or weather) complications are not updating several times a minute.
Update:
What if I only wanted to update the complication when user requested it himself(=clicked the complication)? He is not really interested in the state all the time.
Can you (only) update a complication when the watch awakes? No.
Can you update an extension when the extension awakes? Yes.
The user needs to consider what a complication is meant to be. It's designed to be regularly updated to show current information. There's no mechanism to not update the complication because the user is mostly not interested in knowing the state.
Could you tap on a complication to open its app? Yes. But the complication itself would be showing stale data, and the user would have to do more than raise their wrist to see current state.
If you consider what the user is asking, they're not describing a complication (which shows current state), but a way to see the current state when they request it.
That mechanism is different from a complication. They're really describing a Glance (or an app) which the user swipes up (or opens) to see.
If they want live updates, it can be done via WCSession updateApplicationContext.
Use the updateApplicationContext:error: method to communicate recent state information to the counterpart. When the counterpart wakes, it can use this information to update its own state. Sending a new dictionary with this method overwrites the previous dictionary.
The way that works is your phone sends background updates to the watch. The watch stores the most recent update, which will be available to it when your app or Glance wakes up. The user views the app or Glance, and it displays the most recent value which the watch stored. While open, it continues to update itself as new updates arrive. While closed (e.g., inactive, asleep), the watch stores the most recent update on behalf of your app or glance.
If the user doesn't need the app or glance to update itself every 15 seconds, then you wouldn't need to poll, and you could simply use a NSURLSession to fetch the current state when the extension awakes.
If you explain to the user that there's no way to update a complication when the user raises their wrist, and that they don't care about knowing the state all the time (which a complication is meant to do), then show them what a Glance can do, you'll find it far easier to accomplish what the user seems to want, ideally without unnecessarily draining the battery.
I'm currently investigating Rebus but being unable to find good documentation this process is proving difficult. I am hoping someone can help me understand this exciting product.
I have read that during message processing, if something goes wrong the message will return to the queue.
Is the message returned to the front of the queue or placed on the end? If placed on the front this will be problem because the queue in essence becomes blocked with a message that may not be able to be processed - at least until it times out or retries exceeded.
Does Rebus have support for an out-of-the-box separate Retry queue?
Can I specify the interval between retries?
Can I specify an exponential backoff interval for retries as in Apache ActiveMQ?
Thanks
1) The queue transaction is rolled back, effectively moving the message back in front - therefore, it will be immediately retried.
After 5 failed attempts (at least that is the default), Rebus will move the message to the error queue. The default retry mechanism is intentionally very swift - this way, the input queue will never be clogged by poisonous messages.
If you need more sophisticated retries, I suggest you tage a look at bus.Defer - it can defer delivery of a message to the future. It requires that you have a timeout manager(*) running though.
2) I guess that's what I call "error queue", except there's no retry :)
I did create a solution some time, though, where I coded a simple endpoint that would periodically empty the error queue and move all the messages back into the original source queue, as a form of crude automatic second-level retry mechanism.
3) No. NServiceBus has the concept of second-level retries, but this is something that I've never really needed (enough) with Rebus. But with Rebus, you're on your own here - it should be fairly easy to do some intelligent bus.Defer that can then be easily adapted to each kind of error that you're expecting.
4) See (3)
I hope that clarifies a bit :)
(*) The timeout manager can be a separate endpoint whose only job in life is to receive a message, hold on to it for a while (i.e. save it to a database), and then return it to the sender when the time has elapsed. The timeout manager can be hosted in-process though, but using the .Timeouts(t => t.???) configuration spell.
I didn't see a situation quite like mine, so here goes:
Scenario highlights: The user wants a system that includes custom SMS alerts. A component of the functionality is to have a way to identify a start based on user input, then send SMS with personalized message according to a pre-defined interval after the trigger. I've never used Twilio before and am noodling around with the implementation.
First Pass Solution: Using Twilio account, I designated the .aspx that will receive the inbound triggering alert/SMS via GET. The receiving page declares and instantiates my SMSAlerter object within page load, which responds immediately with a first SMS and kicks off the System.Timer.Timer. Elementary, and functional to a point.
Problem: The alerts continue to be sent if the interval for the timer is a short time span. I tested it at a minute interval and was successful. When I went to 10 minutes, the immediate SMS is sent and the first message 10 minutes later is sent, but nothing after that.
My Observation: Since there is no interaction with the resource after the inbound text, the Session times out if left at default 20 minutes. Increasing Session timeout doesn't work, and even if it did does not seem correct since the interval will be on the order of hours, not minutes.
Using Cache to store each new SMSAlerter might be the way to go. For any SMSAlerter that is created, the schedule is used for roughly 12 hours and is replaced with a new SMSAlerter object when the same user notifies the system the following day. Is there a better way? Am I over/under-simplifying? I am not anticipating heavy traffic now (tens of users), but the user is thinking big.
Thank you for comments, suggestions. I didn't include the code, because the question is about design, not syntax.
I think your timer is going out of scope about 20 minutes after the original request, killing the timer. I have a feeling that if you keep refreshing the aspx page it won't happen - but obviously that doesn't help much.
You could launch a new thread that has the System.Timers.Timer object so it stays alive, and doesn't go out of scope when there are no follow up requests to the server. But this isn't a great idea to be honest - although it might help with understanding the issue.
Ultimately, you'll need some sort of continuously running service - as you don't want to depend on the app pool for this, so I'd suggest a Windows Service running in the background to handle it, which is going to be suitable for a long term solution.
Hope this helps!
(Edited slightly to make the windows service aspect clearer)