I am using JPA2 #entity to represent my models. I need to put data asynchronously to the datastore. How would I do that? If a JPA Dao cannot do that, can I use AsyncDatastoreService on JPA entities -- instead of JPQL? Does anyone have an example?
(I know in Python this would be a joke as I can extend ndb.Model to create models and then call put_async on them. But in Java there is no ndb.Model to extend so I am using JPA.)
I am very much hoping not to have to use the following ThreadManager.createBackgroundThread. But so far it's all I have found. Does anyone else have something much simpler (like python's put_async)?
import com.google.appengine.api.ThreadManager;
import java.util.concurrent.AtomicLong;
AtomicLong counter = new AtomicLong();
Thread thread = ThreadManager.createBackgroundThread(new Runnable() {
public void run() {
try {
while (true) {
counter.incrementAndGet();
Thread.sleep(10);
}
} catch (InterruptedException ex) {
throw new RuntimeException("Interrupted in loop:", ex);
}
}
});
thread.start();
Related
I am trying to implement non blocking retries with single topic fixed back-off.
I am able to do so, thanks to documentation https://docs.spring.io/spring-kafka/reference/html/#single-topic-fixed-delay-retries.
Now I also need to perform a few blocked/local retries on main topic. I have been trying to implement this using DefaultErrorHandler as below:
#Bean
public DefaultErrorHandler retryErrorHandler() {
return new DefaultErrorHandler(new FixedBackOff(2000, 3));
}
This does not seem to work with RetryableTopic.
I have also tried the following approach retry-topic-combine-blocking https://docs.spring.io/spring-kafka/reference/html/#retry-topic-combine-blocking using ListenerContainerFactoryConfigurer
but issue I am facing here is creating beans KafkaConsumerBackoffManager, DeadLetterPublishingRecovererFactory and especially KafkaConsumerBackoffManager.
I need to know if this another way to achieve this using spring kafka framework or is there a way to construct above beans ?
We're currently working on improving configuration for the non-blocking retries components.
For now, as documented here, you should inject these beans such as:
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory,
#Qualifier(RetryTopicInternalBeanNames
.INTERNAL_BACKOFF_CLOCK_BEAN_NAME) Clock clock) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, clock);
lcfc.setBlockingRetryableExceptions(MyBlockingRetryException.class, MyOtherBlockingRetryException.class);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
return lcfc;
}}
Also, there's a known issue where if you try to inject the beans before the first #KafkaListener bean with retryable topic is processed, the feature's component's beans won't be present in the context yet and will throw an error.
Does that happen to you?
We're currently working on a fix for this, but we should be able to work around that if that's your problem.
EDIT: Since the problem is that components are not instantiated yet, the most guaranteed workaround is to provide the components yourself.
Here's a sample on how to do that. Of course, adjust it accordingly if you need any further customization.
#Configuration
public static class SO71705876Configuration {
#Bean(name = RetryTopicInternalBeanNames.LISTENER_CONTAINER_FACTORY_CONFIGURER_NAME)
public ListenerContainerFactoryConfigurer lcfc(KafkaConsumerBackoffManager kafkaConsumerBackoffManager,
DeadLetterPublishingRecovererFactory deadLetterPublishingRecovererFactory) {
ListenerContainerFactoryConfigurer lcfc = new ListenerContainerFactoryConfigurer(kafkaConsumerBackoffManager, deadLetterPublishingRecovererFactory, Clock.systemUTC());
lcfc.setBlockingRetryableExceptions(IllegalArgumentException.class, IllegalStateException.class);
lcfc.setBlockingRetriesBackOff(new FixedBackOff(500, 5)); // Optional
return lcfc;
}
#Bean(name = RetryTopicInternalBeanNames.KAFKA_CONSUMER_BACKOFF_MANAGER)
public KafkaConsumerBackoffManager backOffManager(ApplicationContext context) {
PartitionPausingBackOffManagerFactory managerFactory =
new PartitionPausingBackOffManagerFactory();
managerFactory.setApplicationContext(context);
return managerFactory.create();
}
#Bean(name = RetryTopicInternalBeanNames.DEAD_LETTER_PUBLISHING_RECOVERER_FACTORY_BEAN_NAME)
public DeadLetterPublishingRecovererFactory dlprFactory(DestinationTopicResolver resolver) {
return new DeadLetterPublishingRecovererFactory(resolver);
}
#Bean(name = RetryTopicInternalBeanNames.DESTINATION_TOPIC_CONTAINER_NAME)
public DestinationTopicResolver destinationTopicResolver(ApplicationContext context) {
return new DefaultDestinationTopicResolver(Clock.systemUTC(), context);
}
In the next release this should not be a problem anymore. Please let me know if that works for you, or if any further adjustment to this workaround is necessary.
Thanks.
One of the cool features in Prism 6 is the deep linking and passing parameters. In a lot of cases, you'd want to use this parameter to look up data from a web service. Ideally this would be using async/await to get the data. Where is the best place to do this? The OnNavigatedTo method for example is a void.
Although I don't have a case for Deep Linking yet, I am doing many loads on many pages inside OnNavigatedTo and it is working great!
Here is a sample:
public void OnNavigatedTo(NavigationParameters parameters)
{
if (parameters != null &&
parameters.ContainsKey("MyKey"))
{
SomePrivateFieldInViewModel = (YourVariable)parameters["MyKey"];
//SomeWork
}
GetItems();
}
private async void GetItems()
{
try
{
SomeListInViewModel = await WebServices.GetEntity(SomePrivateFieldInViewModel);
//SomeWork
}
catch (Exception ex)
{
//SomeWork
}
}
I am working on making a scheduler, just like Windows Scheduler using Quartz.Net.
In Windows Scheduler, there is an option to stop a task from running if it takes more than the specified time. I have to implement the same in my scheduler.
But I am not able to find any extension method/setting to configure Trigger or Job accordingly.
I request some inputs or suggestions for it.
You can write small code to set a custom timout running on another thread. Implement IInterruptableJob interface and make a call to its Interrupt() method from that thread when the job should be interrupted. You can modify the following sample code as per your need. Please make necessary checks/config inputs wherever required.
public class MyCustomJob : IInterruptableJob
{
private Thread runner;
public void Execute(IJobExecutionContext context)
{
int timeOutInMinutes = 20; //Read this from some config or db.
TimeSpan timeout = TimeSpan.FromMinutes(timeOutInMinutes);
//Run your job here.
//As your job needs to be interrupted, let us create a new task for that.
var task = new Task(() =>
{
Thread.Sleep(timeout);
Interrupt();
});
task.Start();
runner = new Thread(PerformScheduledWork);
runner.Start();
}
private void PerformScheduledWork()
{
//Do what you wish to do in the schedled task.
}
public void Interrupt()
{
try
{
runner.Abort();
}
catch (Exception)
{
//log it!
}
finally
{
//do what you wish to do as a clean up task.
}
}
}
I want to use Rollback() or commit() functions after multiple process.
There is no error, but it does not commit() to update DB.
Here is my example code,
public void startTransaction(){
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
Ads_A_Connection.Open();
Ads_B_Connection.Open();
AdsTransaction aTxn = Ads_A_Connection.BeginTransaction();
AdsTransaction bTxn = Ads_B_Connection.BeginTransaction();
try{
string aResult = this.process1(Ads_A_Connection);
this.process2(Ads_B_Connection, aResult);
this.process3(Ads_A_Connection. Ads_B_Connection);
aTxn.Commit();
bTxn.Commit();
// there is no error, but it couldn't commit.
}catch(Exception e){
aTxn.Rollback();
bTxn.Rollback();
}
}
}
public string process1(conn){
// Insert data
return result;
}
public void process2(conn. aResult){
// update
}
public void process3(aConn, bConn){
// delete
// update
}
I guess, its because out of using scope. because I tried to put all the code into
startTransaction() method, then it works. but it look too dirty.
How can I use rollback() or commit() after multiple (METHOD) process?
anybody know, please advice me.
Thanks!
[EDIT]
I just add TransactionScope before connection,
using (TransactionScope scope = new TransactionScope())
{
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
.
.
but it makes an error, it say "Error 5047: The transaction command was not in valid sequence."
I need a little more hint please :)
To extend what Etch mentioned, their are several issues with manually managing transactions on your connections:
You need to pass the SQL connection around your methods
Need to manually remember to commit or rollback when you are finished
If you have more than one connection to manage under a transaction, you should really use DTC or XA to enroll the transactions into a Distributed / 2 phase transaction.
TransactionScopes are supported with the Advantage Database Server, although you will need to enable the MSDTC service and possibly also enable XA compliance.
Note that I'm assuming that the advantage .NET client has some sort of connection pooling mechanism - this makes the cost of obtaining connections very lightweight.
Ultimately, this means that your code can be refactored to something like the following, which is easier to maintain:
private void Method1()
{
using(Ads_A_Connection = new AdsConnection(Ads_A_connection_string))
{
Ads_A_Connection.Open();
string aResult = this.process1(Ads_A_Connection);
} // Can logically 'close' the connection here, although it is actually now held by the transaction manager
}
private void Method2()
{
using(Ads_B_Connection = new AdsConnection(Ads_B_connection_string))
{
Ads_B_Connection.Open();
this.process2(Ads_B_Connection, aResult);
} // Can logically 'close' the connection here, although it is actually now held by the transaction manager
}
public void MyServiceWhichNeedToBeTransactional(){
using(TransactionScope ts = new TransactionScope()) { // NB : Watch isolation here. Recommend change to READ_COMMITTED
try{
Method1();
Method2();
ts.Complete();
}
catch(Exception e){
// Do Logging etc. No need to rollback, as this is done by default if Complete() not called
}
}
}
TransactionScope is your friend!
TransactionScope
I am new to Nhibernate and slowing working my way thru learning it. I tried to implement a session manager class to help me get the session for my db calls. Below is the code for it. Can someone please say if this is architecturally correct and foresee any issue of scalability or performance?
public static class StaticSessionManager
{
private static ISession _session;
public static ISession GetCurrentSession()
{
if (_session == null)
OpenSession();
return _session;
}
private static void OpenSession()
{
_session = (new Configuration()).Configure().BuildSessionFactory().OpenSession();
}
public static void CloseSession()
{
if (_session != null)
{
_session.Close();
_session = null;
}
}
}
and in my data provider class, I use the following code to get data.
public class GenericDataProvider<T>
{
NHibernate.ISession _session;
public GenericDataProvider()
{
this._session = StaticSessionManager.GetCurrentSession();
}
public T GetById(object id)
{
using (ITransaction tx = _session.BeginTransaction())
{
try
{
T obj = _session.Get<T>(id);
tx.Commit();
return obj;
}
catch (Exception ex)
{
tx.Rollback();
StaticSessionManager.CloseSession();
throw ex;
}
}
}
}
and then
public class UserDataProvider : GenericDataProvider<User>
{
public User GetUserById(Guid uid)
{
return GetById(uid)
}
}
Final usage in Page
UserDataProvider udp = new UserDataProvider();
User u = udp.GetUserById(xxxxxx-xxx-xxx);
Is this something that is correct? Will instantiating lot of data providers in a single page cause issues?
I am also facing an issue right now, where if I do a same read operation from multiple machines at the same time, Nhibernate throws random errors- which I think is due to transactions.
Please advice.
From what I can see you are building the session factory if you have a null session. You should only call BuildSessionFactory() once when the application starts.
Where you do this is up to you, some people build the SessionFactory inside Global.asax in the method application_start or in your case have a static property for sessionFactory instead of session in your StaticSessionManager class.
I suspect your errors are due to the fact that your session factory is being built multiple times!
Another point is that some people open a transaction _session.BeginTransaction() at the beginning of each request and either commit or rollback at the end of each request. This gives you a unit of work which means you can lose the
using (ITransaction tx = _session.BeginTransaction())
{
...
}
on every method. All of this is open for debate but I use this method for 99% of all my code with no trouble at all.