Can anyone please share their experience of invoking unix scripts from a Java EE env, either servlet or EJBs? Note that these scripts are to be invoked for real time processing and not offline processing.
Spawning processes from a Java EE container is probably not the right way to this.
If these are shell scripts they will not be portable.
If you want to use transaction support, the scripts could be rewritten as Jobs using Quartz Scheduler.
This is more likely the Java EE way to do things like that.
EDIT:With your requirements added in the commented this should work
Process process = new ProcessBuilder(command).start();
More details here
please note that if you use scripts and/or pipes (no native executables) you must include the shell to invoke the command (and setup pipes)
One possibility would be to write a small application that listens to a JMS queue and invokes the scripts. That way, the script execution is separated from the app server, so doesn't run into any spec limitations.
The biggest problem you will have is if your app server memory image is large, when you fork to run the script you may well run out of memory and have the fork fail. When you fork, the system needs to make a complete copy of the executable image. It doesn't make a physical copy, but it does need to make a virtual one. So, if you have a large Java EE heap, like 4G of real memory (i.e. not just Java heap, total process size), then you need an extra "free" 4G of real RAM and/or Swap for the fork to have enough virtual space to happen.
Yes, you're going to immediately exec sh or some other command that isn't going to suck up a gazillion resources. But the system can't know that, and so it needs to act as if it's going to have to run two copies of your Java EE container at once, even for a nanosecond.
If you don't have the resources for the fork, the fork fails.
If you're strapped for space, then what you can do is create a little mini exec launcher daemon. Then instead of your Java EE app forking the process, you just open a socket to your daemon, and IT forks the process. Obviously the expectation is that this little daemon is consuming much fewer resources than your container, so it's cheap to fork.
The daemon can be as simple taking the command line to execute over the socket, and just execing what it gets (potentially unsafe, naturally, but...), or simple rpc with a command code and some arguments. Whatever is appropriate for your project. You can write it in Java, a scripting language (Python, Perl, Ruby), whatever. Lots of ways to do that.
Related
Context
I'm trying to implement a sort-of orchestrator pattern for our applications.
Basically, we have three different and independent applications developped in Qt that communicate with each other using Web Socket. We'll call them "core", "business" and "ui". This is a flexibility aim as we can simply develop a new application in a more suiting technology and connect it to the others via the same communication protocole.
Now the idea is to have a simple launcher that allows us to specify which part to start. We launch this "orchestrator-like" application and it starts all required processes from a configuration file.
Everything is done in Qt currently (QML for the UI interfaces).
Initial Issue
I've made a custom class oriented towards reading the configuration file, preparing the processes, and starting them with their respective arguments.
This uses a std::map of QProcess related to their name in the configuration file and launch them using QProcess::start(<process_path>) method.
The catch is that everything went smoothly until recently. The sub processes are started and runs perfectly ; everything goes on as normal until we reach some point were the "ui" part crashes (usually an LLVM memory error or vector:: length error).
At first we thought about a memory leak or a code error but after much debugging we found that the application had no error whatsoever when we ran each part individually (without using the custom orchestrator class).
Question / Concerns
So, our question is: could it be that the QProcess:start() method actually shares the same stack with its parent? Three processes having the same parent, it would not be surprising than a vector of ~500 elements stored in each application can exceed the stack size when returned.
Information
We use MacOS Big Sur, IDE is Qt Creator, using Qt 5.15.0 and C++11.
Tried using valgrind but as read here and here, this seems a dead-end for now. The errors below were seen in the .crash file following the application exit.
libc++abi: terminating with uncaught exception of type std::length_error: vector
ui(2503,0x108215e00) malloc: can't allocate region :*** mach_vm_map(size=140280206704640, flags: 100) failed (error code=3) ui(2503,0x108215e00) malloc: *** set a breakpoint in malloc_error_break to debug LLVM ERROR: out of memory
Also tried to redirects or completly remove the application's output. First changing the setProcessChannelMode when starting the application, then with startDetached instead of start. Then, commented my Log method dumping log info into the corresponding Qt output (info/warning/critical/fatal/debug).
As suggested by #stanislav888, we could rewrite the application manager part in bash scripts and it would probably do the trick but I'd like to understand the root issue to avoid future mistakes.
It looks like a bad design. Application running and orchestrating through bash or PowerShell script looks much more better.
But anyway.
You could try to suppress orchestrated applications output and see what happen. The programs output might flood memory and make crash.
You must check what particular trouble cause the crash. Use memory "coredump" and system error messages to understand the all problem details.
I sure the community need that details. Because "оut of memory", "stack depth exceeded" and same errors make big difference.
Try to write bash or PowerShell script which does the same workflow as the Qt application. Hope it is not hard. But it will help you to figure out the issue.
At least you can run this script from the application.
I am wondering if it is somehow possible to have an OSGi environment with GoGo shell running with different consoles for input and output.
It is not very handy to write gogo commands into a console where your system is logging a lot of data.
Is there a good solution for that?
There are a number of solutions for your question.
You can use the telnet or ssh interface to gogo. Apache Felix provides both in separate bundles.
Gogo registers a CommandProcessor from which you can make a CommandSession by supplying an Input and Output Stream. It is quite easy to make a small Java Swing program that acts as a shell
Stop logging so much :-)
We all know that Core Dumps are an essential diagnostic tools for analysing various processes in Unix . I know both jstack and gcore are both used for generating Javacore files or Core Dumps but I have a doubt that Gcore is mainly used for Processes and Jstack is used for threads .
As from an Operating System perspective Process and Threads though interrelated (Process comprises of Threads only) they are relatively different from each other w.r.t memory/speed/execution . So is that gcore will diagnose the process and jstack will analyse the threads in that process ???
GCore act at OS level and you got a dump of native code that is currently running. From a java point of view, it is not really understandable.
JStack get you the stack trace at VM level (the java stack) of all thread your application have. You can find from that what is the real java code executed at a point.
Clearly, GCore is almost never used (too low level, native code...). Only really strange issue with native library or stuff like that will perhaps need this kind of tool.
There is also jmap that can generate a hprof file which is the heap data from you VM. A tool like 'Memory Analyser Tool' can open the hprof, and you can drill down on what was going on (on memory side).
If your VM crash because of a OutOfMemory, you can also set parameter to get the hprof when the event occurs. It helps to understand why (too many users, A DB query that fetch too much data...)
Last thing is the fact that you can add a debug option when your start your VM, so that you can connect to it, and put debug on running process. It can help if you have some strange issue that you are not able to reproduce in your local environment.
I have created a memory dump of an ASP.NET process on a server using the following command: .dump /ma mydump.dmp. I am trying to identify a memory leak.
I want to look at the dump file in more detail on my local development PC. I read somewhere that it is advisable to debug on the same machine as you create the dump file. However, I have also read that some developers do analyse the dump file on their local development PC's. What is the best approach?
I notice that when I create a dump file using the command above the W3WP process memory increases by about 1.5 times. Why this this? I suppose this should be avoided on a live server.
Analyzing on the same machine can save you from SOS loading issues thereafter. Unless you are familiar with WinDbg and SOS, you will find it confusing and frustrating then.
If you have to use another machine for analysis, make sure you read carefully this blog post, http://blogs.msdn.com/b/dougste/archive/2009/02/18/failed-to-load-data-access-dll-0x80004005-or-what-is-mscordacwks-dll.aspx as it shows you how to copy the necessary files from the source machine (where the dump is captured) to the target machine (the one you launch WinDbg).
For your second question, as you use WinDbg to attach to the process directly, and use .dump command to capture the dump, the target process unfortunately is modified. Not easy to explain in a few words. The recommended way is to use ADPlus.exe or Debug Diag. Even procdump from SysInternals is better. Those tools are designed for dump capture and they have minimal impact on the target processes.
For memory leak from unmanaged libraries, you should use memory leak rule of Debug Diag. for managed memory leak, you can simply capture hang dumps when memory usage is high.
I am no expert on WinDBG but I once had to analyse a dump file on my ASP.NET site to find a StackOverflowException.
While I got a dump file of my live site (I had no choice since that was what was failing), originally I tried to analyse that dump file on my local dev PC but ran into problems when trying to load the CLR data from it. The reason being that the exact version of the .NET framework differed between my dev PC and the server - both were .NET 4 but I imagine my dev PC had some cumulative updates installed that the server did not. The SOS module simply refused to load because of this discrepancy. I actually wrote a blog post about my findings.
So to answer part of your question it may be that you have no choice but to run WinDBG from your server, at least you can be sure that the dump file will match your environment.
It is not necessary to debug on the actual machine unless the problem is difficult to manifest on your development machine.
So long as you have the pdbs with the private symbols then the symbols should be resolved and call stacks correctly displayed and the correct version of .NET installed.
In terms of looking at memory leaks you should enable Gflags user stack trace and take memory dumps at 2 intervals so you can compare the memory usage before and after the action that provokes the memory leak, remember to disable gflags afterwards!
You could also run DebugDiag on the server which has automated memory pressure analysis scripts that will work with .Net leaks.
We are using RDI (IBM Rational Developer for System i) to do cobol development work, we are eager to write automation test cases for our program, to make the testing work easier. But we don't know how to use script to compile and run cobol, which on i-series server.
For now, our solution is that we use scripts prepare test data (insert data to database/files),and then run cobol on RDI manually, finally, run scripts to check the results. It makes our work easier, but still not real automation test.
So, I want to know if there are some methods to invoke the compile&run process according to scripts, such eclipse headless or telnet technologies.
We've already found the solution: use telnet to compile/run program. Because green screen is one kind of telnet, it reliable.