Run Rscript from Spring MVC with Wildfly 9 - r

I am trying to run Rscript from JAVA code. I am able to do so. Now I am trying to run same JAVA code from a Spring MVC project and using Wildfly 9 to run the project. For the first time when I am trying to execute JAVA code (to run Rscript) is working fine and giving correct result, but on running 2nd time it is giving error and Wildfly stops running. Below is the error that I am getting:
A fatal error has been detected by the Java Runtime Environment:
Internal Error (0xc0000029), pid=6768, tid=8456
JRE version: Java(TM) SE Runtime Environment (7.0_75-b13) (build 1.7.0_75-b13)
Java VM: Java HotSpot(TM) Client VM (24.75-b04 mixed mode, sharing windows-x86 )
Problematic frame:
C [ntdll.dll+0xa096a]
Failed to write core dump. Minidumps are not enabled by default on client versions of Windows
The JAVA code is below:
package com.test.util;
import org.rosuda.JRI.Rengine;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class RunRScript {
private static final Logger logger = LoggerFactory
.getLogger(RunRScript.class);
public void runScript() {
// Create an R vector in the form of a string.
String javaVector = "c(1,2,3,4,5)";
// Start Rengine.
Rengine engine = new Rengine(new String[] { "--no-save" }, false, null);
// The vector that was created in JAVA context is stored in 'rVector' which is a variable in R context.
engine.eval("rVector=" + javaVector);
//Calculate MEAN of vector using R syntax.
engine.eval("meanVal=mean(rVector)");
//Retrieve MEAN value
double mean = engine.eval("meanVal").asDouble();
//Print output values
logger.info("Mean of given vector is=" + mean);
}
}
I am using Windows 8 64-bit and R-2.15.0. Please let me know if my question is not clear or you need any other information. Thanks in advance.

You can't call JRI engine with that code. According to the documentation, JRI doesn't allow more that one engine instance per JVM, so you shouldn't create more than one engine.
This line:
// Start Rengine.
Rengine engine = new Rengine(new String[] { "--no-save" }, false, null);
Must be called only once. You have to ensure that only one engine is started in your JVM.
On the other hand, JRI uses by default the same environment to handle all the calls (eval, assign, etc...) so the rest of your code must be synchronized, otherwise you can suffer race conditions every time two different threads are executing eval methods.
If you have trouble getting it working, you can replace JRI by Rserve, that doesn't need JRI library loaded into the JVM and allow concurrency (each thread must use its own RConnection).
But with Rserve you should setup your engine only once, as well as using JRI.
You can use a #PostConstruct method:
/**
* This method initializes REngine properly and make all the operations needed
* to set up the environment.
*
* This RServe implementation must run R in a separate process and check the connection.
*
*/
#PostConstruct
public void setUpR() {//NOSONAR
REngine engine = null;
try {
if(LOGGER.isInfoEnabled()) {
LOGGER.info("Starting RServe process...");
}
ProcessBuilder builder = new ProcessBuilder("/bin/sh", "-c", String.format("echo 'library(Rserve);Rserve(FALSE,args=\"--no-save --slave --RS-conf %s\")'|%s --no-save --slave", rserveConf, rexe));
builder.inheritIO();
Process rProcess = builder.start();
if(LOGGER.isInfoEnabled()) {
LOGGER.info("Waiting for Rserve to start...");
}
int execCodeResult = rProcess.waitFor();
if(execCodeResult != SUCCESS_CODE) {
LOGGER.error(String.format("Unexpected error code starting RServe: %d", execCodeResult));
} else {
LOGGER.error("RServe started successfully");
}
if(LOGGER.isInfoEnabled()) {
LOGGER.info("Opening connection to RServe daemon....");
}
engine = new RConnection();
if(LOGGER.isInfoEnabled()) {
LOGGER.info(String.format("Obtaining R server version: %d", ((RConnection)engine).getServerVersion()));
}
} catch(Exception e) {
LOGGER.error("Unexpected error setting up RServe environment", e);
} finally {
closeRServeConnection(engine);
}
}
You can do the same with JRI:
/**
* This method initializes REngine properly and make all the operations needed
* to set up the environment.
*
* This JRI implementation must load JRI library and starts JRIEngine
*
*/
#PostConstruct
public void setUpR() {//NOSONAR
try {
//make sure JRI lib can be loaded (it must be present in java.library.path parameter)
//This line is necessary because Rengine.versionCheck() will execute a System.exit if
//it can't load JRI library.
System.loadLibrary("jri");
// just making sure we have the right version of everything
if (!Rengine.versionCheck()) {
LOGGER.error("** Version mismatch - Java files don't match library version.");
LOGGER.error(String.format("Invalid versions. Rengine must have the same version of native library. Rengine version: %d. RNI library version: %d", Rengine.getVersion(), Rengine.rniGetVersion()));
}
// Enables debug traces
Rengine.DEBUG = 1;
if(LOGGER.isInfoEnabled()) {
LOGGER.info("Creating Rengine (with arguments)");
}
// 1) we pass the arguments from the command line
// 2) we won't use the main loop at first, we'll start it later
// (that's the "false" as second argument)
// 3) no callback class will be used
this.engine = REngine.engineForClass("org.rosuda.REngine.JRI.JRIEngine", new String[] { "--no-save" }, new REngineStdOutCallback(LOGGER), false);
if(LOGGER.isInfoEnabled()) {
LOGGER.info("Rengine created...");
LOGGER.info("Loading blockFunction from " + getBlockFunction());
}
REXP result = engine.parseAndEval(getBlockFunction());
if(result == null) {
LOGGER.error("blockFunction is not loaded!");
} else if(LOGGER.isInfoEnabled()) {
LOGGER.info("blockFunction loaded successfully");
}
} catch(Exception|UnsatisfiedLinkError e) {
LOGGER.error("Unexpected error setting up R", e);
}
}
And then reuse the same Rengine instance in each call (make sure you synchronize the access to this instance).
You have more examples in this repo

Related

dotnet core TopShelf Windows Service fails to start

I have a dotnet core console application build to connect to a Sql Service Broker instance to monitor table changes.
The app monitors one table that is updated from an ERP system and then publishes messages to our bus.
It runs fine when running as a console application, or debugging in my IDE.
I am having an issue when using TopShelf to configure it as a windows service.
Here is the entry point:
private static void Main(string[] args)
{
RegisterComponents();
var serviceHost = HostFactory.Run(sc =>
{
sc.Service<ISalesOrderMonitorService>(s =>
{
var sqlListener = _container.ResolveNamed<SqlDependencyEx>(ListenerKey.SalesOrder);
var changeHandler = _container.Resolve<ISalesOrderChangeHandler>();
var listenerConfig = _container.ResolveNamed<ListenerConfiguration>(ListenerKey.SalesOrder);
var logger = _container.Resolve<ILogger<SalesOrder>>();
s.ConstructUsing(f =>
new SalesOrderMonitorService(sqlListener, changeHandler, listenerConfig, logger));
s.WhenStarted(tc => tc.Start());
s.WhenStopped(tc => tc.Stop());
});
});
var exitCode = (int) Convert.ChangeType(serviceHost, serviceHost.GetType());
Environment.ExitCode = exitCode;
}
The "worker" class:
public abstract class ServiceBase<T, TZ> : IService<T>
where T : IChangeHandler
{
protected readonly IChangeHandler ChangeHandler;
protected readonly SqlDependencyEx Listener;
protected readonly ListenerConfiguration ListenerConfiguration;
protected readonly ILogger<TZ> Logger;
protected ServiceBase(SqlDependencyEx listener, IChangeHandler changeHandler,
ListenerConfiguration listenerConfiguration, ILogger<TZ> logger)
{
Logger = logger;
ListenerConfiguration = listenerConfiguration;
Listener = listener;
ChangeHandler = changeHandler;
}
public virtual void Start()
{
try
{
Listener.TableChanged += (o, e) => ChangeHandler.Process(e);
Listener.Start();
Logger.LogDebug(
$"Listening to changes on the {ListenerConfiguration.Table} table in the {ListenerConfiguration.Database} database");
}
catch (Exception e)
{
Logger.LogError(e, e.Message);
throw;
}
}
public virtual void Stop()
{
Listener.Stop();
}
Install through TopShelf is no problem:
c:>{ServiceName}.exe install -username "serviceAccount" -password "superSecret" -servicename "ServiceName" -servicedescription "Description" -displayname "Service DisplayName" --autostart
When I go to start the service - I get this:
This is misleading because the event viewer shows this:
This is happening way faster than 30 seconds. This is definitely related to how I am configuring TopShelf.
As stated - the application works just fine when run "debug" or even as just an exe console.
I got it figured out. Actually both comments from #DotNetPadawan and #Lex Li indirectly got me there.
For starters - enabling the remote debugger clued me in that my appsetting.json was not being read into my IConfiguration. That was really confusing because everything works fine running locally with a debugger or even just starting the exe.
The link Lex Li points out did not provide the answer - however that article had this reference:
Host and Deploy aspnetcore as a Windows Service
It was here that I found this little nugget:
The current working directory returned by calling GetCurrentDirectory for a Windows Service is the C:\WINDOWS\system32 folder. The system32 folder isn't a suitable location to store a service's files (for example, settings files). Use one of the following approaches to maintain and access a service's assets and settings files.
The link explains how to conditionally set the current directory if the app is running as a service.
var isConsole = args.Contains("-mode:console");
if (!isConsole)
{
var pathToExe = Process.GetCurrentProcess().MainModule?.FileName;
var pathToContentRoot = Path.GetDirectoryName(pathToExe);
Directory.SetCurrentDirectory(pathToContentRoot);
}
Putting this out there for anyone else that runs into this problem.
Admittedly - netcore 3.0 is likely the better way to go - but I don't have the bandwidth to upgrade everything is this repo (lots of shared stuff) to 3.0. I needed to get this working.

Log event datetime with.Net Core Console logger

I'm using logging to Console output, that built-in to .Net Core framework.
Here initialization of the logger:
var serviceCollection = new ServiceCollection();
serviceCollection.AddSingleton(new LoggerFactory()
.AddConsole());
Also for logging I'm using Microsoft.Extensions.Logging.LoggerExtensions class with methods Log...
Here an example of logging in my App:
_logger.LogInformation(eventId, "Action is started.");
Where _logger is instance of ILogger<T> class and initialized in the class constructor with built-in dependency injection.
As result of calling of the above method Console output shows following string:
info: NameSpaceName.ClassName[eventId] Action is started.
I would like to display date-time in the Console output, that points to time, when the Log method is executed, but it seems that Log.. methods don't contain any methods that allow to display date time.
Does it exist some method or additioanl classes-formatters that allow to display the action datetime in console output without passing it to the method as part of the message?
The feature was added into version 3 of the Microsoft.Extensions.Logging.Console(here is the pr). You can activate this with setting the TimestampFormat:
new ServiceCollection()
.AddLogging(opt =>
{
opt.AddConsole(c =>
{
c.TimestampFormat = "[HH:mm:ss] ";
});
})
Example in .NET 5 (ASP.NET Core):
public void ConfigureServices(IServiceCollection services)
{
services.AddLogging(options =>
{
options.AddSimpleConsole(c =>
{
c.TimestampFormat = "[yyyy-MM-dd HH:mm:ss] ";
// c.UseUtcTimestamp = true; // something to consider
});
});
// ...
}
Output example:
[2020-12-13 12:55:44] info: Microsoft.Hosting.Lifetime[0] Application is shutting down...
For ASP.NET Core, you might prefer to use configuration file appsettings.json over wiring it directly into the code.
{
"Logging": {
"Console": {
"TimestampFormat": "[yyyy-MM-dd HH:mm:ss] "
}
}
}
This works out of the box, provided that Host.CreateDefaultBuilder() is invoked in Program.cs
Built-in .NET Core console logger doesn't log date-time. Track this issue to get more details. The easiest workaround is:
logger.Log(LogLevel.Information, 1, someObj, null, (s, e) => DateTime.Now + " " + s.ToString());
I wrote a custom console logger to automatically log the timestamp and do other useful tricks:
[2017.06.15 23:46:44] info: WebHost[1] Request starting HTTP/1.1 GET http://localhost:6002/hc

Flags for PermGen not working as expected: -XX:+CMSClassUnloadingEnabled and -XX:+CMSPermGenSweepingEnabled

I have the following code which generates (intentionally) a PermGen java.lang.OutOfMemoryError:
public class Main {
public static void main(String[] args) throws InstantiationException, IllegalAccessException, InterruptedException {
String name = "MyClass";
DynamicClassLoader cl = new DynamicClassLoader();
int i = 0;
while (true) {
//code for generating the binary for a class to be loaded.
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
cw.visit(Opcodes.V1_7, Opcodes.ACC_PUBLIC, name + ++i, null, "java/lang/Object", null);
MethodVisitor con = cw.visitMethod(Opcodes.ACC_PUBLIC, "", "()V", null, null);
con.visitCode();
con.visitVarInsn(Opcodes.ALOAD, 0);
con.visitMethodInsn(Opcodes.INVOKESPECIAL, "java/lang/Object", "", "()V", false);
con.visitInsn(Opcodes.RETURN);
con.visitMaxs(1, 1);
cw.visitEnd();
//binary code for class successfully generated
Class clazz = cl.defineClass(name + i, cw.toByteArray());
Object o = clazz.newInstance();
System.out.println(o.getClass().getName());
}
}
private static class DynamicClassLoader extends ClassLoader {
public Class defineClass(String name, byte[] b) {
return defineClass(name, b, 0, b.length);
}
}
}
I run this code in Java 7. As expected, it gets java.lang.OutOfMemoryError: PermGen space error.
When I try to run this program with the mentioned flags as follows:
java -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled -jar target/permgen.jar
, I still get the same error, at exact the same point as when I run it without flags.
I expected that if I put those flags, if not a total refresh of PermGen would occur, at least a partial improvement would be seen. But this is not the case.
Question: Am I misunderstanding the meaning of those flags? Can you detail if that's the case, please? Otherwise, any suggestions?
N.B. Output of java -version is:
java version "1.7.0_95"
OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-3)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
All classloaders keep strong references to all classes they load. In your example, you keep reusing the single instance of DynamicClassLoader. This classloader in turn keeps strong references to all classes you load. So the Garbage Collector never sees unreferenced objects it can collect.
If you modify your test case to use separate classloaders, the Garbage Collector should be able to identify that your classes are not being used and should reclaim the memory.

exePath exception when using SAP .net connector 3.0 in ASP.net

Have a WPF app that I am converting to ASP.net and I am having issues with SAP.
When I run this line I get an exception.
RfcDestinationManager.RegisterDestinationConfiguration(Backend);
Exception Message {"exePath must be specified when not running inside a stand alone exe."}
Stack Trace
at System.Configuration.ConfigurationManager.OpenExeConfigurationImpl(ConfigurationFileMap fileMap, Boolean isMachine, ConfigurationUserLevel userLevel, String exePath, Boolean preLoad)
at System.Configuration.ConfigurationManager.OpenExeConfiguration(ConfigurationUserLevel userLevel)
at SAP.Middleware.Connector.RfcConfigParameters..cctor()
googling around I saw as similar issue here exePath must be specified when not running inside a stand alone exe
The issue seems to be using ConfigurationManager.OpenExeConfiguration rather that System.Web.Configuration.WebConfigurationManagerwhich is what I need to use. Problem is I can't change that as its part of the SAP.Net Connector.
Is there anything I can do?
Edit: My BackendConfig code
public class BackendConfig : IDestinationConfiguration
{
public RfcConfigParameters GetParameters(String destinationName)
{
if ("P38".Equals(destinationName))
{
var parms = new RfcConfigParameters
{
{RfcConfigParameters.AppServerHost, "SAPSERVER"},
{RfcConfigParameters.SystemNumber, "86"},
{RfcConfigParameters.SncMode, "1"},
{RfcConfigParameters.SncPartnerName, "p:SAP#SERVER"},
{RfcConfigParameters.Client, "010"},
{RfcConfigParameters.Language, "EN"},
{RfcConfigParameters.PoolSize, "5"}
};
return parms;
}
else return null;
}
// The following two are not used in this example:
public bool ChangeEventsSupported()
{
return false;
}
public event RfcDestinationManager.ConfigurationChangeHandler ConfigurationChanged;
}

Behaviour of JerseyTest Grizzly Web Server on Unix

We have created a test suite and in order to run it we are using embedded Grizzly Web Server with JerseyTest framework.
We are extending a custom class from JerseyTest and in its constructor we are creating ApplicationDescriptor and then call superclass setupTestEnvironment() which essentially starts embedded grizzly web server.
Few of our test cases are extending this custom class to start grizzly server directly. However, we are not stopping this embedded server anywhere in the code.
The test cases run fine on windows but on Unix they fail with java.net.BindException port 9998 is in use by another process.
It becomes obvious these tests should fail with similar error on windows too if we are not stopping embedded web server in the code. How they are running fine on windows and failing on unix. Has this something to do with how Unix spawns threads or processes?
P.S. We have also tested whether port 9998 is in use by some other process using netstat -a | grep 9998 but no other process using that port could be found.
i had a similar problem and i did fix it by not using the default port if already used. just add following code to your test case:
#Override
protected int getPort(int defaultPort) {
ServerSocket server = null;
int port = -1;
try {
server = new ServerSocket(defaultPort);
port = server.getLocalPort();
} catch (IOException e) {
// ignore
} finally {
if (server != null) {
try {
server.close();
} catch (IOException e) {
// ignore
}
}
}
if ((port != -1) || (defaultPort == 0)) {
return port;
}
return getPort(0);
}
I had the same problem, when I was writing my integration tests. I didn't get to test on a Windows machine, but on my Unix machine I found the problem was that by default the JerseyTest class utilizes #After on it's tearDown method to close the embedded server. Since I had overridden this method to do clean up on my side, I had to call super.tearDown()
#After
public void tearDown() throws Exception{
super.tearDown();
...
}
After doing this, everything worked as expected.

Resources