I am using #KafkaListener(topics = "${topic}") to consume messages from a topic in a spring-boot application and I need this running periodically. spring-kafka version is 2.2.4.RELEASE.
One way to achieve this could have been batching every 6 hours using fetch.max.wait.ms, but 6 hours seems too much for this configuration.
Hence, I am looking for a way to shut down the application after processing, and restart it every 6 hours.
Other way is something like below, but it does not guarantee that application had finished processing within the sleep time(30 sec in below example).
public class Application {
public static void main(String[] args) throws InterruptedException {
ConfigurableApplicationContext run = SpringApplication.run(Application.class, args);
Thread.sleep(30000);
run.close();
}
}
What is the graceful way to shutdown the consumer, to make sure that shutdown happens only after it has processed the batch of messages?
See this answer.
Shut down the application when all of the container instances go idle.
Related
Spring Kafka version - 2.8.5
```#KafkaListener
consumeMessages(#Payload List<String> messages,Ack ack)
{
//process records in separate Thread and acknowledge
// Thread is not available nack it after 2 seconds
}```
Nack records should be reprocessed after 2 seconds in KafkaListener. However, Skipped records were not processed by KafkaListener. The missing message is consumed again after restarting the Spring Boot app.
You can't use nack() from another thread; only from the listener thread
#Override
public void nack(long sleepMillis) {
Assert.state(Thread.currentThread().equals(ListenerConsumer.this.consumerThread),
"nack() can only be called on the consumer thread");
Even when called from the listener thread, nack() does not support "skipping" records; when using manual commits, it is the application's responsibility to commit the offset to skip a record (by calling acknowledge()).
I have a Quarkus application where I use the event bus.
the code in question looks like this:
#ConsumeEvent(value = "execution-request", blocking = true)
#Transactional
#TransactionConfiguration(timeout = 3600)
public void consume(final Message<ExecutionRequest> msg) {
try {
execute(...);
} catch (final Exception e) {
// some logging
}
}
private void execute(...)
throws InterruptedException {
// it actually runs a long running task, but for
// this example this has the same effect
Thread.sleep(65000);
}
Why do I still get a
WARN [io.ver.cor.imp.BlockedThreadChecker] (vertx-blocked-thread-checker) Thread Thread[vert.x-worker-thread-0,5,main] has been blocked for 63066 ms, time limit is 60000 ms: io.vertx.core.VertxException: Thread blocked
I'm I doing something wrong? Is the blocking parameter at the ConsumeEvent annotation not enough to let that handle in a separate Worker?
Your annotation is working as designed; the method is running in a worker thread. You can tell by both the name of the thread "vert.x-worker-thread-0", and by the 60 second timeout before the warnings were logged. The eventloop thread only has a 3 second timeout, I believe.
The default Vert.x worker thread pool is not designed for "very" long running blocking code, as stated in their docs:
Warning:
Blocking code should block for a reasonable amount of time (i.e no more than a few seconds). Long blocking operations or polling operations (i.e a thread that spin in a loop polling events in a blocking fashion) are precluded. When the blocking operation lasts more than the 10 seconds, a message will be printed on the console by the blocked thread checker. Long blocking operations should use a dedicated thread managed by the application, which can interact with verticles using the event-bus or runOnContext
That message mentions blocking for more than 10 seconds triggers a warning, but I think that's a typo; the default is actually 60.
To avoid the warning, you'll need to create a dedicated WorkerExecutor (via vertx.createSharedWorkerExecutor) configured with a very high maxExcecuteTime. However, it does not appear you can tell the #ConsumeEvent annotation to use it instead of the default worker pool, so you'd need to manually create an event bus consumer, as well, or use a regular #ConsumeEvent annotation, but call workerExectur.executeBlocking inside of it.
Trying to figure out why my console app won't stop running.
Using the following approach in a dotnet core application main method:
await new HostBuilder().
...
.ConfigureServices((hostContext, services) =>
{
services.AddHostedService<MyHostedService>();
})
.UseConsoleLifetime()
.Build()
.RunAsync();
Publishing and scheduling that task from the Windows Task Scheduler using the following settings works:
All good so far. All code is properly executed. However, the task stays running, the process never ends. (not even after pressing refresh on the UI of the task scheduler)
Is this expected? If not, how do I get the process to terminate?
If expected, does it still make sense then, to use Generic Host / Hosted Service in a scheduled console app that just starts, runs, and stops?
Answer based on Microsoft.Extensions.Hosting 2.2.0
This behavior is expected, due to your usage of the Generic Host:
It keeps running until shut down or disposed, and you have no shutdown mechanism in place. I assume you expect the Generic Host to shut down after IHostedService.StartAsync(CancellationToken) of your MyHostedService ran to completion. This is not the case, because there might be other IHostedService implementations registered and executed in sequence, and/or a long running BackgroundService which returns control when its ExecuteAsync(CancellationToken) is not completing synchronously to allow other services to run in parallel.
To stop your application gracefully after your MyHostedService completes when running the host via RunAsync, you should constructor-inject the IApplicationLifetime into your MyHostedService and call StopApplication after your Task has completed.
internal class MyHostedService : IHostedService
{
private readonly IApplicationLifetime _appLifetime;
public MyHostedService(IApplicationLifetime appLifetime)
{
_appLifetime = appLifetime;
}
public async Task StartAsync(CancellationToken cancellationToken)
{
await Task.Delay(1000); //your scheduled work
_appLifetime.StopApplication();
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
Also, the application may be stopped via either AppDomain.CurrentDomain.ProcessExit or Console.CancelKeyPress, both events are subscribed to by the ConsoleLifetime, which is pre-registered as the default lifetime implementation.
You can read more about lifetime management in the docs.
Microsoft.Extensions.Hosting 3.0.0 - currently in preview - marked IApplicationLifetime obsolete and recommends using IHostApplicationLifetime instead
According to the documentation, Task#call() is "invoked when the Task is executed ".
Consider the following program:
import javafx.application.Application;
import javafx.concurrent.Task;
import javafx.stage.Stage;
public class TestTask extends Application {
Long start;
public void start(Stage stage) {
start = System.currentTimeMillis();
new Thread(new Taskus()).start();
}
public static void main(String[] args) {
launch();
}
class Taskus extends Task<Void> {
public Taskus() {
stateProperty().addListener((obs, oldValue, newValue) -> {
try {
System.out.println(newValue + " at " + (System.currentTimeMillis()-start));
} catch (Exception e) {
e.printStackTrace();
}
});
}
public Void call() throws InterruptedException {
for (int i = 0; i < 10000; i++) {
// Could be a lot longer.
}
System.out.println("Some code already executed." + " at " + (System.currentTimeMillis()-start));
Thread.sleep(3000);
return null;
}
}
}
Executing this program gives me the following output:
Some code already executed. after 5 milliseconds
SCHEDULED after 5 milliseconds
RUNNING after 7 milliseconds
SUCCEEDED after 3005 milliseconds
Why is the call() method invoked before the task is even scheduled? This makes no sense to me. In the task where I first saw the issue my task executed a few seconds before the task went into the SCHEDULED state. What if I want to give the user some feedback on the state, and nothing happens until the task has already been executed for a few seconds?
Why is the call() method invoked before the task is even scheduled?
TLDR; version: It's not. It's merely invoked before you get notified that it's been scheduled.
You have two threads running, essentially independently: the thread you explicitly create, and the FX Application Thread. When you start your application thread, it will invoke Taskus.call() on that thread. However, changes to the the task's properties are made on the FX Application Thread via calls to Platform.runLater(...).
So when you call start() on your thread, the following occurs behind the scenes:
A new thread is started
On that thread, an internal call() method in Task is called. That method:
Schedules a runnable to execute on the FX Application Thread, that changes the stateProperty of the task to SCHEDULED
Schedules a runnable to execute on the FX Application Thread, that changes the stateProperty of the task to RUNNING
Invokes your call method
When the FX Application Thread receives the runnable that changes the state of the task from READY to SCHEDULED, and later from SCHEDULED to RUNNING, it effects those changes and notifies any listeners. Since this is on a different thread to the code in your call method, there is no "happens-before" relationship between code in your call method and code in your stateProperty listeners. In other words, there is no guarantee as to which will happen first. In particular, if the FX Application Thread is already busy doing something (rendering the UI, processing user input, processing other Runnables passed to Platform.runLater(...), etc), it will finish those before it makes the changes to the task's stateProperty.
What you are guaranteed is that the changes to SCHEDULED and to RUNNING will be scheduled on the FX Application thread (but not necessarily executed) before your call method is invoked, and that the change to SCHEDULED will be executed before the change to RUNNING is executed.
Here's an analogy. Suppose I take requests from customers to write software. Think of my workflow as the background thread. Suppose I have an admin assistant who communicates with the customers for me. Think of her workflow as the FX Application thread. So when I receive a request from a customer, I tell my admin assistant to email the customer and notify them I received the request (SCHEDULED). My admin assistant dutifully puts that on her "to-do" list. A short while later, I tell my admin assistant to email the customer telling them I have started working on their project (RUNNING), and she adds that to her "to-do" list. I then start working on the project. I do a little work on the project, and then go onto Twitter and post a tweet (your System.out.println("Some code already executed")) "Working on a project for xxx, it's really interesting!". Depending on the number of things already on my assistant's "to-do" list, it's perfectly possible the tweet may appear before she sends the emails to the customer, and so perfectly possible the customer sees that I have started work on the project before seeing the email saying the work is scheduled, even though from the perspective of my workflow, everything occurred in the correct order.
This is typically what you want: the status property is designed to be used to update the UI, so it must run on the FX Application Thread. Since you are running your task on a different thread, you presumably want it to do just that: run in a different thread of execution.
It seems unlikely to me that a change to the scheduled state would be observed a significant amount of time (more than one frame rendering pulse, typically 1/60th second) after the call method actually started executing: if this is happening you are likely blocking the FX Application thread somewhere to prevent it from seeing those changes. In your example, the time delay is clearly minimal (less than a millisecond).
If you want to do something when the task starts, but don't care which thread you do it on, just do that at the beginning of the call method. (In terms of the analogy above, this would be the equivalent of me sending the emails to the customer, instead of requesting that my assistant do it.)
If you really need code in your call method to happen after some user notification has occurred on the FX Application Thread, you need to use the following pattern:
public class Taskus extends Task<Void> {
#Override
public Void call() throws Exception {
FutureTask<Void> uiUpdate = new FutureTask<Void>(() -> {
System.out.println("Task has started");
// do some UI update here...
return null ;
});
Platform.runLater(uiUpdate);
// wait for update:
uiUpdate.get();
for (int i = 0; i < 10000; i++) {
// any VM implementation worth using is going
// to ignore this loop, by the way...
}
System.out.println("Some code already executed." + " at " + (System.currentTimeMillis()-start));
Thread.sleep(3000);
return null ;
}
}
In this example, you are guaranteed to see "Task has started" before you see "Some code already executed". Additionally, since displaying the "Task has started" method happens on the same thread (the FX Application thread) as the changes in state to SCHEDULED and RUNNING, and since displaying the "Task has started" message is scheduled after those changes in state, you are guaranteed to see the transitions to SCHEDULED and RUNNING before you see the "Task has started" message. (In terms of the analogy, this is the same as me asking my assistant to send the emails, and then not starting any work until I know she has sent them.)
Also note that if you replace your original call to
System.out.println("Some code already executed." + " at " + (System.currentTimeMillis()-start));
with
Platform.runLater(() ->
System.out.println("Some code already executed." + " at " + (System.currentTimeMillis()-start)));
then you are also guaranteed to see the calls in the order you are expecting:
SCHEDULED after 5 milliseconds
RUNNING after 7 milliseconds
Some code already executed. after 8 milliseconds
SUCCEEDED after 3008 milliseconds
This last version is the equivalent in the analogy of me asking my assistant to post the tweet for me.
I am working on an asp.net mvc 5 web application , deployed inside IIS-8, and i have a method inside my application to perform a long running task which mainly scans our network for servers & VMs and update our database with the scan results. method execution might last between 30-40 minutes to complete on production environment. and i am using a schedule tool named Hangfire which will call this method 2 times a day.
here is the job definition inside the startup.cs file, which will call the method at 8:01 am & 8:01 pm:-
public void Configuration(IAppBuilder app)
{
var options = new SqlServerStorageOptions
{
PrepareSchemaIfNecessary = false
};
GlobalConfiguration.Configuration.UseSqlServerStorage("scanservice",options);
RecurringJob.AddOrUpdate(() => ss.Scan(), "01 8,20 ***");
}
and here is the method which is being called twice a day by the schedule tool:-
public void Scan()
{
Service ss = new Service();
ss.NetworkScan().Wait();
}
Finally the method which do the real scan is (i only provide a high level description of what the method will do):-
public async Task<ScanResult> NetworkScan()
{
// retrieve the server info from the DB
// loop over all servers & then execute some power shell commands to scan the network & retrieve the info for each server one by one...
// after the shell command completed for each server, i will update the related server info inside the DB
currently i did some tests on our test environment and every thing worked well ,, where the scan took around 25 seconds to scan 2 test servers.but now we are planning to move the application to production and we have around 120++ servers to scan. so i estimate the method execution to take around 30 -40 minutes to complete on the production environment. so my question is how i can make sure that this execution will never expire , and the ScanNetwork() method will complete till the end?
Instead of worrying about your task timing out, perhaps you could start a new task for each server. In this way each task will be very short lived, and any exceptions caused by scanning a single server will not effect all the others. Additionally, if your application is restarted in IIS any scans which were not yet completed will be resumed. With all scans happening in one sequential task this is not possible. You will likely also see the total time to complete a scan of your entire network plummet, as the majority of time would likely be spent waiting on remote servers.
public void Scan()
{
Service ss = new Service();
foreach (var server in ss.GetServers())
{
BackgroundJob.Enqueue<Service>(s => s.ServerScan(server));
}
}
Now your scheduled task will simply enqueue one new task for each server.