I'm trying to understand these concepts
Event-driven
Asynchronous
non-blocking I/O
Imagine you read from a socket with this pseudo code
void processIO(socket)
{
data = socket.read();
doSomething(data);
}
The read method is used in a blocking mode. That means it does not continue until the data is read. The thread, this is running on is blocked and does not continue until the data is read. doSomething is only called once the data is read. If you did this on a main thread of an app, it would probably not be able to update its UI and would behave like frozen until the data is received.
async void processIO(socket)
{
data = await socket.readAsync();
doSomething(data);
}
This function is asynchronous and it itself calls an asynchronous readAsync() method.
In this way, the thread this runs on is not blocked. It is interrupted at the await statement, and is available for other things in your app to run. Once the data is read, it resumes after the await statement and continues with doing something with the data.
This is one way of doing non-blockig data processing however it is not event driven.
void setupRead(socket)
{
socket.on_data(do_something);
}
void do_something(data)
{
// process data
}
void main()
{
socket = new Socket(111)
setupRead(socket)
while (true) {
processEvents()
}
}
This last example demonstrates event driven IO. You register a callback on some resource to be called when some data arrives. In the mean time your code may do other things or do nothing. This is a non-blocking, asynchronous and event driven approach. The UI get's refreshed and the app may do whatever it needs to.
Event Driven, means you setup event callbacks and wait for the events to happen.
Asynchronous means, you do other stuff while waiting such as refreshing UI, processing user input or read from write from other resources.
Non-blocking means the thread that started the listening, is not blocked until an event arrives, it does whatever else there is to do. Such as handle other events in the mean time.
I have (in the past) written cross-platform (Windows/Unix) applications which, when started from the command line, handled a user-typed Ctrl-C combination in the same way (i.e. to terminate the application cleanly).
Is it possible on Windows to send a Ctrl-C/SIGINT/equivalent to a process from another (unrelated) process to request that it terminate cleanly (giving it an opportunity to tidy up resources etc.)?
I have done some research around this topic, which turned out to be more popular than I anticipated. KindDragon's reply was one of the pivotal points.
I wrote a longer blog post on the topic and created a working demo program, which demonstrates using this type of system to close a command line application in a couple of nice fashions. That post also lists external links that I used in my research.
In short, those demo programs do the following:
Start a program with a visible window using .Net, hide with pinvoke, run for 6 seconds, show with pinvoke, stop with .Net.
Start a program without a window using .Net, run for 6 seconds, stop by attaching console and issuing ConsoleCtrlEvent
Edit: The amended solution from #KindDragon for those who are interested in the code here and now. If you plan to start other programs after stopping the first one, you should re-enable CTRL+C handling, otherwise the next process will inherit the parent's disabled state and will not respond to CTRL+C.
[DllImport("kernel32.dll", SetLastError = true)]
static extern bool AttachConsole(uint dwProcessId);
[DllImport("kernel32.dll", SetLastError = true, ExactSpelling = true)]
static extern bool FreeConsole();
[DllImport("kernel32.dll")]
static extern bool SetConsoleCtrlHandler(ConsoleCtrlDelegate HandlerRoutine, bool Add);
delegate bool ConsoleCtrlDelegate(CtrlTypes CtrlType);
// Enumerated type for the control messages sent to the handler routine
enum CtrlTypes : uint
{
CTRL_C_EVENT = 0,
CTRL_BREAK_EVENT,
CTRL_CLOSE_EVENT,
CTRL_LOGOFF_EVENT = 5,
CTRL_SHUTDOWN_EVENT
}
[DllImport("kernel32.dll")]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool GenerateConsoleCtrlEvent(CtrlTypes dwCtrlEvent, uint dwProcessGroupId);
public void StopProgram(Process proc)
{
//This does not require the console window to be visible.
if (AttachConsole((uint)proc.Id))
{
// Disable Ctrl-C handling for our program
SetConsoleCtrlHandler(null, true);
GenerateConsoleCtrlEvent(CtrlTypes.CTRL_C_EVENT, 0);
//Moved this command up on suggestion from Timothy Jannace (see comments below)
FreeConsole();
// Must wait here. If we don't and re-enable Ctrl-C
// handling below too fast, we might terminate ourselves.
proc.WaitForExit(2000);
//Re-enable Ctrl-C handling or any subsequently started
//programs will inherit the disabled state.
SetConsoleCtrlHandler(null, false);
}
}
Also, plan for a contingency solution if AttachConsole() or the sent signal should fail, for instance sleeping then this:
if (!proc.HasExited)
{
try
{
proc.Kill();
}
catch (InvalidOperationException e){}
}
The closest that I've come to a solution is the SendSignal 3rd party app. The author lists source code and an executable. I've verified that it works under 64-bit windows (running as a 32-bit program, killing another 32-bit program), but I've not figured out how to embed the code into a windows program (either 32-bit or 64-bit).
How it works:
After much digging around in the debugger I discovered that the entry point that actually does the behavior associated with a signal like ctrl-break is kernel32!CtrlRoutine. The function had the same prototype as ThreadProc, so it can be used with CreateRemoteThread directly, without having to inject code. However, that's not an exported symbol! It's at different addresses (and even has different names) on different versions of Windows. What to do?
Here is the solution I finally came up with. I install a console ctrl handler for my app, then generate a ctrl-break signal for my app. When my handler gets called, I look back at the top of the stack to find out the parameters passed to kernel32!BaseThreadStart. I grab the first param, which is the desired start address of the thread, which is the address of kernel32!CtrlRoutine. Then I return from my handler, indicating that I have handled the signal and my app should not be terminated. Back in the main thread, I wait until the address of kernel32!CtrlRoutine has been retrieved. Once I've got it, I create a remote thread in the target process with the discovered start address. This causes the ctrl handlers in the target process to be evaluated as if ctrl-break had been pressed!
The nice thing is that only the target process is affected, and any process (even a windowed process) can be targeted. One downside is that my little app can't be used in a batch file, since it will kill it when it sends the ctrl-break event in order to discover the address of kernel32!CtrlRoutine.
(Precede it with start if running it in a batch file.)
I guess I'm a bit late on this question but I'll write something anyway for anyone having the same problem.
This is the same answer as I gave to this question.
My problem was that I'd like my application to be a GUI application but the processes executed should be run in the background without any interactive console window attached. I think this solution should also work when the parent process is a console process. You may have to remove the "CREATE_NO_WINDOW" flag though.
I managed to solve this using GenerateConsoleCtrlEvent() with a wrapper app. The tricky part is just that the documentation is not really clear on exactly how it can be used and the pitfalls with it.
My solution is based on what is described here. But that didn't really explain all the details either and with an error, so here is the details on how to get it working.
Create a new helper application "Helper.exe". This application will sit between your application (parent) and the child process you want to be able to close. It will also create the actual child process. You must have this "middle man" process or GenerateConsoleCtrlEvent() will fail.
Use some kind of IPC mechanism to communicate from the parent to the helper process that the helper should close the child process. When the helper get this event it calls "GenerateConsoleCtrlEvent(CTRL_BREAK, 0)" which closes down itself and the child process. I used an event object for this myself which the parent completes when it wants to cancel the child process.
To create your Helper.exe create it with CREATE_NO_WINDOW and CREATE_NEW_PROCESS_GROUP. And when creating the child process create it with no flags (0) meaning it will derive the console from its parent. Failing to do this will cause it to ignore the event.
It is very important that each step is done like this. I've been trying all different kinds of combinations but this combination is the only one that works. You can't send a CTRL_C event. It will return success but will be ignored by the process. CTRL_BREAK is the only one that works. Doesn't really matter since they will both call ExitProcess() in the end.
You also can't call GenerateConsoleCtrlEvent() with a process groupd id of the child process id directly allowing the helper process to continue living. This will fail as well.
I spent a whole day trying to get this working. This solution works for me but if anyone has anything else to add please do. I went all over the net finding lots of people with similar problems but no definite solution to the problem. How GenerateConsoleCtrlEvent() works is also a bit weird so if anyone knows more details on it please share.
Somehow GenerateConsoleCtrlEvent() return error if you call it for another process, but you can attach to another console application and send event to all child processes.
void SendControlC(int pid)
{
AttachConsole(pid); // attach to process console
SetConsoleCtrlHandler(NULL, TRUE); // disable Control+C handling for our app
GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0); // generate Control+C event
}
Edit:
For a GUI App, the "normal" way to handle this in Windows development would be to send a WM_CLOSE message to the process's main window.
For a console app, you need to use SetConsoleCtrlHandler to add a CTRL_C_EVENT.
If the application doesn't honor that, you could call TerminateProcess.
Here is the code I use in my C++ app.
Positive points :
Works from console app
Works from Windows service
No delay required
Does not close the current app
Negative points :
The main console is lost and a new one is created (see FreeConsole)
The console switching give strange results...
// Inspired from http://stackoverflow.com/a/15281070/1529139
// and http://stackoverflow.com/q/40059902/1529139
bool signalCtrl(DWORD dwProcessId, DWORD dwCtrlEvent)
{
bool success = false;
DWORD thisConsoleId = GetCurrentProcessId();
// Leave current console if it exists
// (otherwise AttachConsole will return ERROR_ACCESS_DENIED)
bool consoleDetached = (FreeConsole() != FALSE);
if (AttachConsole(dwProcessId) != FALSE)
{
// Add a fake Ctrl-C handler for avoid instant kill is this console
// WARNING: do not revert it or current program will be also killed
SetConsoleCtrlHandler(nullptr, true);
success = (GenerateConsoleCtrlEvent(dwCtrlEvent, 0) != FALSE);
FreeConsole();
}
if (consoleDetached)
{
// Create a new console if previous was deleted by OS
if (AttachConsole(thisConsoleId) == FALSE)
{
int errorCode = GetLastError();
if (errorCode == 31) // 31=ERROR_GEN_FAILURE
{
AllocConsole();
}
}
}
return success;
}
Usage example :
DWORD dwProcessId = ...;
if (signalCtrl(dwProcessId, CTRL_C_EVENT))
{
cout << "Signal sent" << endl;
}
A solution that I have found from here is pretty simple if you have python 3.x available in your command line. First, save a file (ctrl_c.py) with the contents:
import ctypes
import sys
kernel = ctypes.windll.kernel32
pid = int(sys.argv[1])
kernel.FreeConsole()
kernel.AttachConsole(pid)
kernel.SetConsoleCtrlHandler(None, 1)
kernel.GenerateConsoleCtrlEvent(0, 0)
sys.exit(0)
Then call:
python ctrl_c.py 12345
If that doesn't work, I recommend trying out the windows-kill project:
Choco: https://github.com/ElyDotDev/windows-kill
Node: https://github.com/ElyDotDev/node-windows-kill
void SendSIGINT( HANDLE hProcess )
{
DWORD pid = GetProcessId(hProcess);
FreeConsole();
if (AttachConsole(pid))
{
// Disable Ctrl-C handling for our program
SetConsoleCtrlHandler(NULL, true);
GenerateConsoleCtrlEvent(CTRL_C_EVENT, 0); // SIGINT
//Re-enable Ctrl-C handling or any subsequently started
//programs will inherit the disabled state.
SetConsoleCtrlHandler(NULL, false);
WaitForSingleObject(hProcess, 10000);
}
}
Thanks to jimhark's answer and other answers here, I found a way to do it in PowerShell:
$ProcessID = 1234
$MemberDefinition = '
[DllImport("kernel32.dll")]public static extern bool FreeConsole();
[DllImport("kernel32.dll")]public static extern bool AttachConsole(uint p);
[DllImport("kernel32.dll")]public static extern bool GenerateConsoleCtrlEvent(uint e, uint p);
public static void SendCtrlC(uint p) {
FreeConsole();
if (AttachConsole(p)) {
GenerateConsoleCtrlEvent(0, p);
FreeConsole();
}
AttachConsole(uint.MaxValue);
}'
Add-Type -Name 'dummyName' -Namespace 'dummyNamespace' -MemberDefinition $MemberDefinition
[dummyNamespace.dummyName]::SendCtrlC($ProcessID)
What made things work was sending the GenerateConsoleCtrlEvent to the desired process group instead of sending it to all processes that share the console of the calling process and then AttachConsole back to the current process' parent's console.
Yes. The https://github.com/ElyDotDev/windows-kill project does exactly what you want:
windows-kill -SIGINT 1234
It should be made crystal clear because at the moment it isn't.
There is a modified and compiled version of SendSignal to send Ctrl-C (by default it only sends Ctrl+Break). Here are some binaries:
(2014-3-7) : I built both 32-bit and 64-bit version with Ctrl-C, it's called SendSignalCtrlC.exe and you can download it at: https://dl.dropboxusercontent.com/u/49065779/sendsignalctrlc/x86/SendSignalCtrlC.exe https://dl.dropboxusercontent.com/u/49065779/sendsignalctrlc/x86_64/SendSignalCtrlC.exe -- Juraj Michalak
I have also mirrored those files just in case:
32-bit version: https://www.dropbox.com/s/r96jxglhkm4sjz2/SendSignalCtrlC.exe?dl=0
64-bit version: https://www.dropbox.com/s/hhe0io7mcgcle1c/SendSignalCtrlC64.exe?dl=0
Disclaimer: I didn't build those files. No modification was made to the compiled
original files. The only platform tested is the 64-bit Windows 7. It is recommended to adapt the source available at http://www.latenighthacking.com/projects/2003/sendSignal/ and compile it yourself.
In Java, using JNA with the Kernel32.dll library, similar to a C++ solution. Runs the CtrlCSender main method as a Process which just gets the console of the process to send the Ctrl+C event to and generates the event. As it runs separately without a console the Ctrl+C event does not need to be disabled and enabled again.
CtrlCSender.java - Based on Nemo1024's and KindDragon's answers.
Given a known process ID, this consoless application will attach the console of targeted process and generate a CTRL+C Event on it.
import com.sun.jna.platform.win32.Kernel32;
public class CtrlCSender {
public static void main(String args[]) {
int processId = Integer.parseInt(args[0]);
Kernel32.INSTANCE.AttachConsole(processId);
Kernel32.INSTANCE.GenerateConsoleCtrlEvent(Kernel32.CTRL_C_EVENT, 0);
}
}
Main Application - Runs CtrlCSender as a separate consoless process
ProcessBuilder pb = new ProcessBuilder();
pb.command("javaw", "-cp", System.getProperty("java.class.path", "."), CtrlCSender.class.getName(), processId);
pb.redirectErrorStream();
pb.redirectOutput(ProcessBuilder.Redirect.INHERIT);
pb.redirectError(ProcessBuilder.Redirect.INHERIT);
Process ctrlCProcess = pb.start();
ctrlCProcess.waitFor();
I found all this too complicated and used SendKeys to send a CTRL-C keystroke to the command line window (i.e. cmd.exe window) as a workaround.
A friend of mine suggested a complete different way of solving the problem and it worked for me. Use a vbscript like below. It starts and application, let it run for 7 seconds and close it using ctrl+c.
'VBScript Example
Set WshShell = WScript.CreateObject("WScript.Shell")
WshShell.Run "notepad.exe"
WshShell.AppActivate "notepad"
WScript.Sleep 7000
WshShell.SendKeys "^C"
// Send [CTRL-C] to interrupt a batch file running in a Command Prompt window, even if the Command Prompt window is not visible,
// without bringing the Command Prompt window into focus.
// [CTRL-C] will have an effect on the batch file, but not on the Command Prompt window itself -- in other words,
// [CTRL-C] will not have the same visible effect on a Command Prompt window that isn't running a batch file at the moment
// as bringing a Command Prompt window that isn't running a batch file into focus and pressing [CTRL-C] on the keyboard.
ulong ulProcessId = 0UL;
// hwC = Find Command Prompt window HWND
GetWindowThreadProcessId (hwC, (LPDWORD) &ulProcessId);
AttachConsole ((DWORD) ulProcessId);
SetConsoleCtrlHandler (NULL, TRUE);
GenerateConsoleCtrlEvent (CTRL_C_EVENT, 0UL);
SetConsoleCtrlHandler (NULL, FALSE);
FreeConsole ();
SIGINT can be send to program using windows-kill, by syntax windows-kill -SIGINT PID, where PID can be obtained by Microsoft's pslist.
Regarding catching SIGINTs, if your program is in Python then you can implement SIGINT processing/catching like in this solution.
Based on process id, we can send the signal to process to terminate forcefully or gracefully or any other signal.
List all process :
C:\>tasklist
To kill the process:
C:\>Taskkill /IM firefox.exe /F
or
C:\>Taskkill /PID 26356 /F
Details:
http://tweaks.com/windows/39559/kill-processes-from-command-prompt/
// down = acquire the resource
// up = release the resource
typedef int semaphore;
semaphore resource_1;
semaphore resource_2;
void process_A(void) {
down(&resource_1);
down(&resource_2);
use_both_resources();
up(&resource_2);
up(&resource_1);
}
If the resource return in the same order as it acquired, i.e,
void process_A(void) {
down(&resource_1);
down(&resource_2);
use_both_resources();
up(&resource_1);
up(&resource_2);
}
Would that cause any potential problem.
Thanks for any explanation!
The important part is if you are taking the locks in the same order in different threads or not.
The order of release has no effect; there's nothing stopping the program from releasing the second lock after the first one has been released, (unless you're taking new locks in between, but then you're back at the first case; taking the locks in the correct order.)
If you have two functions that try to take the same two locks, in different orders, they could grab one lock each, and wait forever for the other one to release their lock. Example code:
down(first_lock)
down(second_lock)
running concurrently with
down(second_lock)
down(first_lock)
they could both take their first lock before any of them take their second lock, and then they'll deadlock.
I'm developing a JavaFX application for read data from a serial device and show a notification when a new device is connected to the computer.
I have a task DeviceDetectorTask which scans all the ports and creates an event when a new device is connected. This task must be submited every 3 seconds.
When a device is detected the user can press a button to read all the data contained in it. This is performed by another task ReadDeviceTask. At this point and while the ReadDeviceTask is running scan operations should not be performed (I cannot read and scan one port at the same time). So only one of the two task can be running at a time.
My actual solution is:
public class DeviceTaskQueue {
private ExecutorService executorService = Executors.newSingleThreadExecutor();
public void submit(Runnable task) {
executorService.submit(task);
}
}
public class ScanScheduler {
private ScheduledExecutorService executor = Executors.newSingleThreadScheduledExecutor();
public void start() {
AddScanTask task = new AddScanTask();
executor.scheduleAtFixedRate(task, 0, 3, TimeUnit.SECONDS);
}
}
public class AddScanTask implements Runnable {
#Autowired
DeviceTaskQueue deviceTaskQueue;
#Override
public void run() {
deviceTaskQueue.submit(new DeviceDetectorTask());
}
}
public class ViewController {
#Autowired
DeviceTaskQueue deviceTaskQueue;
#FXML
private readDataFromDevice() {
deviceTaskQueue.submit(new ReadDeviceTask());
}
}
My question is: is it ok to add a task to the ExecutorService from the task AddScanTask which has been scheduled by the ScheduledExecutorService?
Yes, An Executor May Post Task To Another Executor
To answer your simple question in last line:
is it ok to add a task to the ExecutorService from the task AddScanTask which has been scheduled by the ScheduledExecutorService?
Yes. Certainly you can submit a Callable/Runnable from any other code. That the submitting code happens to be running from another executor is irrelevant, as code run from an executor is still “normal” Java code, just running on a different thread.
That is the whole point of the executor, to handle the juggling of threads in a manner convenient to you the programmer. Making multi-threaded coding easier and less error-prone is why these classes were added to Java. See the extremely helpful book, Java Concurrency in Practice by Brian Goetz et al. And see other writings by Goetz.
In your case you have two executors each with their own thread, each executing a series of submitted tasks. One has tasks submitted automatically (timed) while the other has tasks submitted manually (arbitrarily). Each executes on their own thread independent of one another. With multiple cores they may execute simultaneously.
Therein lies the bigger problem: In your scenario you don't want them to be independent. You want the reading tasks to block the scanning tasks.
Bigger Problem
The problem you present is that a regularly occurring activity (scanning) must halt when an arbitrary event (reading) happens. That means the two activities must coordinate with one another. The question is how to coordinate.
Semaphores
When the arbitrary event is happening, it should raise a flag. The recurring activity, when it runs, should always check for that flag. If raised, wait until the flag lowers before proceeding with scan. The ScheduledExecutorService is designed for this, tolerating a task that may run for a time longer than the scheduled period. If one execution of the task runs long, the SES does not run again, so it does not pile up a backlog of executions. That is just the behavior you want.
Vice versa, if the recurring activity is executing, it should raise a flag. The arbitrary event’s first to-do item is to check for that flag. If raised, wait until lowered. Then proceed, first raising its own flag and then proceeding with the task at hand (scanning).
Perhaps your scenario should be designed with a single flag rather than scanner and reader each having their own. I would have to think about it more and probably know more about your scenario.
The technical term for such flags is semaphore.
Unfortunately your comment says you cannot alter the scanner’s source code. So you cannot implement the semaphores and coordinate the activities. So I am stuck, cannot see a solution.
Hack
Given your frozen code, one hack solution, which I do not recommend, is that the regularly occurring activity (the scanning) not actually do the work but instead post a scanning task on another thread (another executor). That other executor would also be the same executor used to post the arbitrary activity (the reading). So there is one single queue of to-do items, a mix of scanning and reading jobs, submitted to a single-thread executor. The single-thread means they get done one at a time in sequence of their submission.
I do not like this hack because if any of the to-do items takes a long while you will begin to accumulate a backlog. That could be a mess.
By the way, no need for the DeviceTaskQueue in your example code. Just call the instance of the ExecutorService directly to submit a task. That is the job of an ExecutorService, and wrapping it adds no value that I can see.
this is my first post here. I'm asking because I ran out of clues and I was unable to find anything about this specific issue.
My question is: In Adobe AIR, is there a way to do a synchronous usleep() equivalent (delay execution of 200ms), alternatively is there a way to specify the SQLite busy timeout somewhere?
I have an AIR application which uses the database in synchronous mode because the code cannot cope with the need of events/callbacks in SQL queries.
The database sometimes is accessed from another application, such that it is busy. Hence the execute() of a statement throws SQLerror 3119 detail 2206. In this case the command shall be retried after a short delay.
As there is another application running on the computer I want to try to avoid busy waiting, however I'm stuck with it because of three things:
First, I was unable to find a way to give the SQLConnection a busy timeout value, like it is possible in C with the function sqlite3_busy_timeout()
Second, I was unable to find the equivalent of the C usleep() command in Adobe AIR / Actionscript.
Third, I am unable to use events/timers/callbacks etc. at this location. The SQL execute() must be synchronous because it is called from deeply nested classes and functions in zillion of places all around in the application.
If the application could cope with events/callbacks while doing SQL I would use an asynchronous database anyway, so this problem cannot be solved using events. The retry must be done on the lowest level without using the AIR event processing facility.
The lowest level of code looks like:
private static function retried(fn:Function):void {
var loops:int = 0;
for (;;) {
try {
fn();
if (loops)
trace("database available again, "+loops+" loops");
return;
} catch (e:Error) {
if (e is SQLError && e.errorID==3119) {
if (!loops)
trace("database locked, retrying");
loops++;
// Braindead AIR does not provide a synchronous sleep
// so we busy loop here
continue;
}
trace(e.getStackTrace());
trace(e);
throw e;
}
}
}
One sample use of this function is:
protected static function begin(conn:SQLConnection):void {
retried(function():void{
conn.begin(SQLTransactionLockType.EXCLUSIVE);
});
}
Output of this code is something like:
database locked, retrying
database available again, 5100 loops
Read: The application loops over 500 times a second. I would like to reduce this to 5 loops somehow to reduce CPU load while waiting, because the App shall run on Laptops while on battery.
Thanks.
-Tino