I've a Model Product that fires retrieved event ProductRetrieved and a listener CheckProductValidity that throws an exception depending on the API path (if-else condition).
Also, I've a update query that I have implemented in Console\Kernal.php that runs everyday at 00:00 Hours.
Problem: CheckProductValidity throws an exception for scheduled tasks. How do I make an exception in listener to allow retrieval of model Product data when it is done by scheduler.
Possible solution: Use unsetEventDispatcher and setEventDispatcher but at times this update query may take more than usual time. Also, cron also sends notifications and processes jobs (all depending on Product) so that might cause problems.
Not really a solution but this is how I fixed it.
// fix to catch if artisan schedule:run has intiated this check;
$parameters = app('request')->server->get('argv');
$allowed_commands = ['schedule:run', 'migrate:refresh', 'db:seed', 'queue:work'];
if ($parameters[0]==='artisan'
&& in_array($parameters[1], $allowed_commands))
return true;
In the listener I added this code which would check if the request was a result of artisan command or a route.
Related
I have ASP.Net 4.7.2 window service which is processing NServiceBus messages. Currently it is deployed to On-Premise server. It has retry mechanism as well and working fine. Now I am going to containerizing it. While running into docker window container, it is doing SQL operation using Entity framework and giving exception as mentioned below:
The configured execution strategy 'SqlRetryingExecutionStrategy' does not support user-initiated transactions. Use the execution strategy returned by 'DbContext.Database.CreateExecutionStrategy()' to execute all the operations in the transaction as a retriable unit.
While running locally by installing manually or on On-Premise server, it is working fine but in container it is throwing exception.
Can any one help me what can be the root cause?
It sounds like the piece of code does manual transaction management and is not wrapped within an execution strategy execute.
if your code initiates a transaction using BeginTransaction() you are defining your own group of operations that need to be treated as a unit, and everything inside the transaction would need to be played back shall a failure occur.
The solution is to manually invoke the execution strategy with a delegate representing everything that needs to be executed. If a transient failure occurs, the execution strategy will invoke the delegate again.
https://learn.microsoft.com/en-us/ef/core/miscellaneous/connection-resiliency#execution-strategies-and-transactions
using var db = new SomeContext();
var strategy = db.Database.CreateExecutionStrategy();
strategy.Execute(
() =>
{
using var context = new SomeContext();
using var transaction = context.Database.BeginTransaction();
context.SaveChanges();
transaction.Commit();
});
``
All,
I am using Change Feed Processor Library.Want to know the best way to handle service failure along with the exceptions/errors scenario's in ProcessChangesAsync method. Below are the events am referring to.
1) Service failure - Service having the processor library crashed in the middle of some operation. How to start the process from the same document(doc on failure instance)? is there any inbuilt mechanism where change feed will start with the last failed documents? E.g. Let assume,in current batch we have 10 docs.5 processed successfully and then service breaks because of network failure or by some other reasons.Will my process starts with 6th document once service is re-started? How to achieve this?
2) Exception and Errors- Any errors in ProcessChangesAsync method can be handle using try catch at the global level but how to persist those failure records and make them available for the next batch? Again,looking for any available inbuilt mechanism in change feed process.
1) The Processor Library, by default, checkpoints after a successful run of ProcessChangesAsync. In the latest library version, you can customize the Checkpointer to do manual checkpoints in case you need it. If for some reason the processor shuts down before checkpointing, then it will start processing next from the the last successful checkpoint stored in the Leases collection. In your case, it will start with the first document again, so you will never lose a change but you could experience double processing (this is an "at least once" model).
2) There is no built-in mechanism that you can leverage, handling exceptions within the ProcessChangesAsync is your responsibility. You could not only add a global try/catch but, in the case you are looping over the documents, add a try/catch inside the loop, to handle a failing document (maybe send it to queue for later analysis/post-process) without losing the batch. If you require logging for those errors (I'm assuming that's what you mean by persisting errors?), then the latest version is compatible with LibLog, so plugging your own custom logging is as simple as:
using Microsoft.Azure.Documents.ChangeFeedProcessor.Logging;
var hostName = "SampleHost";
var tracelogProvider = new TraceLogProvider(); //You can use any provider supported by LibLog
using (tracelogProvider.OpenNestedContext(hostName))
{
LogProvider.SetCurrentLogProvider(tracelogProvider);
// After this, create IChangeFeedProcessor instance and start/stop it.
}
Source
Extra info for the comments
To avoid exceptions halting the batch or causing a batch to be reprocessed, you can have handling like this:
public async Task ProcessChangesAsync(IChangeFeedObserverContext context, IReadOnlyList<Document> documents, CancellationToken cancellationToken)
{
try
{
foreach(var document in documents)
{
try
{
// Do your work for the document
}
catch(Exception ex)
{
// Something happened with the current document, handle it, send it to a queue / another storage to analyze, log it. This catch will make the loop continue with the next.
}
}
}
catch(Exception ex)
{
// Something unhandled happened, log it and avoid throwing it again so the next batch is processed
}
}
I want to use Datastore from Cloud Compute through Java and I am following Getting started with Google Cloud Datastore.
My use case is quite standard - read one entity (lookup), modify it and save the new version. I want to do it in a transaction so that if two processes do this, the second one won't overwrite the changes made by the first one.
I managed to issue a transaction and it works. However I don't know what would happen if the transaction fails:
How to identify a failed transaction? Probably a DatastoreException with some specific code or name will be thrown?
Should I issue a rollback explicitly? Can I assume that if a transaction fails, nothing from it will be written?
Should I retry?
Is there any documentation on that?
How to identify a failed transaction? Probably a DatastoreException
with some specific code or name will be thrown?
Your code should always ensure that a transaction is either successfully committed or rolled back. Here's an example:
// Begin the transaction.
BeginTransactionRequest begin = BeginTransactionRequest.newBuilder()
.build();
ByteString txn = datastore.beginTransaction(begin)
.getTransaction();
try {
// Zero or more transactional lookup()s or runQuerys().
// ...
// Followed by a commit().
CommitRequest commit = CommitRequest.newBuilder()
.setTransaction(txn)
.addMutation(...)
.build();
datastore.commit(commit);
} catch (Exception e) {
// If a transactional operation fails for any reason,
// attempt to roll back.
RollbackRequest rollback = RollbackRequest.newBuilder()
.setTransaction(txn);
.build();
try {
datastore.rollback(rollback);
} catch (DatastoreException de) {
// Rollback may fail due to a transient error or if
// the transaction was already committed.
}
// Propagate original exception.
throw e;
}
An exception might be thrown by commit() or by another lookup() or runQuery() call inside the try block. In each case, it's important to clean up the transaction.
Should I issue a rollback explicitly? Can I assume that if a
transaction fails, nothing from it will be written?
Unless you're sure that the commit() succeeded, you should explicitly issue a rollback() request. However, a failed commit() does not necessarily mean that no data was written. See the note on this page.
Should I retry?
You can retry using exponential backoff. However, frequent transaction failures may indicate that you are attempting to write too frequently to an entity group.
Is there any documentation on that?
https://cloud.google.com/datastore/docs/concepts/transactions
I have the current setup:
A regular Symfony2 web request can create and persist Job entity which also creates a Gearman Job, lets say this occurs in process 1. The Gearman Job is executed by a Gearman Worker which is passed the Job entity's ID.
I also use Symfony to create a Gearman Worker, this is run as a PHP CLI process, lets call this process 2.
For those not familiar with Gearman the worker code operates something like so:
for loop 5 times
get job from gearman (blocking method call)
get job entity from database
do stuff
Essentially this code keeps a Symfony2 instance running to handle 5 Jobs before the worker dies.
My issue is this: On the first job that the worker handles Doctrine2 is able to retrieve the created job from the database without issue using the following code:
$job = $this->doctrine
->getRepository('AcmeJobBundle:Job')
->findOneById($job->workload()); // workload is the job id
However, once this job completes and the for loop increments to wait for a second job, lets say this arrives from another Symfony2 web request on process 3 creating the Job with ID 2, the call to the Doctrine2 repository returns null even though the entity is definitely in the database.
Restarting the worker solves the issue, so when it carries out it's first loop it can pick up Job 2.
Does anyone know why this happens? Does the first call of getRepository or findOneById do some sort of table caching from MySQL that doesn't allow it to see the subsequently added Job 2?
Does MySQL only show a snapshot of the DB to a given connection as long as it is held open?
I've also tried resetting the entityManager before making the second call to findOneBy to no avail.
Thanks for any advice in advance, this one is really stumping me.
Update:
I've created a single process test case to rule out whether or not it was the concurrency causing the problem, and the test case executes as expected. It seems the only time the repository can't find job 2 is when it is added to the DB on another process.
// Job 1 already exists
$job = $this->doctrine
->getRepository('AcmeJobBundle:Job')
->findOneById(1);
$job->getId(); // this is fine.
$em->persist(new Job()); // creates job 2
$em->flush();
$job = $this->doctrine
->getRepository('AcmeJobBundle:Job')
->findOneById(2);
$job->getId(); // this is fine too, no exception.
Perhaps one process tries to load entity before it has saved by the second process.
Doctrine caches loaded entities by their id, so that when you get a second request for the same object it loads without making another query to the database. You can reed more about Doctrine IdentityMap here
is it possible to stop a web service from executing?
I have a flex web application that searches clients with both full name and client id, when searching by name sometimes the usuer just types the last name and it takes a long time.
Since the app is used when clients are waiting in line, I would like to be able to stop the search and use their full name or id instead, and avoid waiting for the results and then having to search the user manually within the results.
thanks
edit: Sorry, I didn't explain myself correctly, when I meant "web service" I actually meant mx.rpc.soap.mxml.WebService, I want to stop it from waiting for the result event and the fault event. thanks.
There is actually a cancel(..) method explicitly for this purpose, though it is a little burried. Using the cancel method will cause the result and fault handlers not to be called and will also remove the busy cursor etc.
Depending on how you run your searches (ie. separate worker process etc), it is also possible to extend this by added in a cancelSearch() web service method to kill these worker processes and free up server resources etc.
private var _searchToken:AsyncToken;
public function doSearch(query:String):void
{
_searchToken = this.searchService.doSearch(query);
}
protected function doSearch_resultHandler(event:ResultEvent):void
{
trace("doSearch result");
trace("TODO: Do stuff with results");
_searchToken = null;
}
protected function doSearch_faultHandler(event:FaultEvent):void
{
trace("doSearch fault: " + event.fault);
_searchToken = null;
}
public function cancelSearch():void
{
var searchMessageId:String = _searchToken.message.messageId;
// Cancels the last service invocation or an invokation with the
// specified ID. Even though the network operation may still
// continue, no result or fault event is dispatched.
searchService.getOperation("doSearch").cancel(searchMessageId);
_searchToken = null;
trace("The search was cancelled, result/fault handlers not called");
// TODO: If your web service search method is using worker processes
// to do a search and is likely to continue processing for some time,
// you may want to implement a 'cancel()' method on the web service
// to stop any search threads that may be running.
}
Update
You could use disconnect() to remove any pending request responders, but it also disconnects the service's connection. Then call initialize().
/Update
You cannot stop the web service from executing, because that's beyond the Flex app's control, but you can limit the processing of the web service's response. For instance on the app, have a button like Cancel Search which sets a boolean bSearchCanceled to true.
The result handler for the web service call checks bSearchCanceled; if true just return.