Adding new paths for native libraries at runtime in Java9 - reflection

I have an app that includes some native code. In an effort to make things easy for users to install and run, I have been extending the private field ClassLoader.sys_paths at runtime as described in this previous question: Adding new paths for native libraries at runtime in Java. With this hack the app can run with a simple unadorned java -jar app.jar command and the native objects will be located correctly in the installation directory.
This technique no longer works in Java 9 (see How to solve InaccessibleObjectException ("Unable to make {member} accessible: module {A} does not 'opens {package}' to {B}") on Java 9?). This article describes the changes to Java 9 and suggests a solution (adding arguments to the command line to open the module), but this defeats my original intent of allowing a simple execution of the java command.
Is there any way to change the native library search path at runtime, or is this considered too much of a loaded gun to leave lying around?

Related

Stuck with creating Keyword library using Java in Eclipse and using that JAR file in RIDE

I'm new to ROBOT framework and must create library keyword using java and utilize the JAR files in RIDE editor. I have followed the below steps. But still have confusions in where to start up. Please help me to achieve this.
Steps:
1) Installed Python, wxPython, pybot, RIDE, selenium2library successfully and I'm able to work with RIDE using predefined keywords.
2) For creating own java library, Installed Eclipse Mars, Java, ROBOT IDE manually using the link https://github.com/NitorCreations/RobotFramework-EclipseIDE/wiki/Installation
The question here is,
Anything i'm missing to install?
How to proceed further with Robot Framework coding?
Note: I'm reading many materials in parts to proceed things. But that is confusing me what are the steps to be followed. Is there any single material that i can get having the sequences of steps that need to be followed for my requirements?
The below links I referred with,
https://github.com/NitorCreations/RobotFramework-EclipseIDE/wiki/Usage
Maven error "Failure to transfer..."
Create a Maven project in Eclipse complains "Could not resolve archetype"
http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#using-set-test-suite-global-variable-keywords
Let's start with the basics. RIDE is an editor for Robot Script files. Robot Script files are executed by the Robot Framework robot application. It is important to understand the difference between these two applications. One creates files and the other uses them.
Assuming for a moment here that you have some custom functionality in a Jar file that you would like to make available inside the Robot Script files test cases. If this is indeed the case then your focus should be on create Robot Test Libraries. In the documentation there are numerous java examples given for basing your own library on.
This means that you do not need Maven, unless you want to, to create your library. Also Eclipse IDE is not needed nor is RIDE. The library you create can be used by Robot Framework even when it's run from the command line. Feel free to use them for what they are good at: writing Robot Scripts efficiently and running Robot through menu options instead of command line windows.
When you are familiar with Eclipse and comfortable with it's features, then I can also recommend using the Eclipse Application RED which is a mature Eclipse add on for creating, running and debugging Robot scripts.
There are two ways of implementing a Java routine in Robot Framework and it depends a bit on how you run Robot Framework: Python or Jython. Running Python means that you need to run the Java in a separate JVM container and connect to it. The simplest way is via Command Line interaction. This may cause some security concerns. The second approach is to use the Remote Library approach. A good example of this is the Robot Framework Sikuli Library. An added bonus is that you can run Robot and your Java application on two separate machines/servers.
The Jython way has been discussed on StackOverflow before and I gladly refer to it. Please bear in mind that although the examples refer to RIDE, the Library org.company.Keywordlibrary works in all IDE's, provided robot framework is run using Jython.
In the Jython documentation it clearly explains how to setup an Eclipse based IDE with Jython. Do keep in mind that it's recommended to use the same interpreter in development and production. I.e. if you create your scripts in Jython, do run them in 'production' also using Jython. Otherwise you open yourself up to interpreter differences where functionality does not work or behaves differently then expected.
An alternative approach with Jython is the all-in-one Robot Framework Maven JAR plugin. Adding a custom Java Library in this setup is not too difficult. A nice short tutorial can be found on the blog of Valerio Bruno. If this is your preferred route for development, then you may want to read the RED howto on integrating the Robot Framework Maven jar.
Working on your custom functionality is likely going to be a independent project in Eclipse, but can be linked to an Robot project via the Class Path settings in red.xml if you use RED or the CLASS_PATH environment variable.
Done! I have installed RED and required add on using following below steps. It worked successfully with predefined keywords. Now I'm gonna create user defined keywords/libraries using the same editor. I don't know where to start up. Will it work creating just a java project? Should I need to include any additional libraries?
Steps followed to proceed with RED:
1. RED Version Conclusion: https://github.com/nokia/RED/blob/master/release.md
2. RED Installation Instructions: https://github.com/nokia/RED/blob/master/installation.md
3. ROBOT Framework prerequisites using RED: https://github.com/nokia/RED/blob/master/red_help/first_steps/download_install.html.md
4. To verify Installations in Eclipse: Help -> Installation Details to check if RED is visible on installed features list
5. Create Project and Tests using RED in Eclipse: https://github.com/nokia/RED/blob/master/red_help/first_steps/create_run.html.md
6. RED Steps: https://github.com/nokia/RED/blob/master/First_steps.md
7. Quick STart: https://github.com/nokia/RED/blob/master/red_help/user_guide/quick_start.html.md

Static link sqlite in Lazarus

I am building an application with Lazarus where I use a sqlite database to store thousands of records. Right now I am linking to the sqlite library dynamically via the sqlite3.dll.
Is it possible to link to it statically? Where can I find the Lazarus compatible lib file to do that?
Note:
I only started using Lazarus and Free Pascal a month ago so something that might look very obvious to one, might not be for me. So bear with me a bit.
Cheers
Actual static linking is difficult since the TSQLite3Connection component is inherently designed to actively load the SQLite3 DLL. In other words, it's not linking against the library when you compile the program, the component is coded to dynamically load the DLL at run time.
If you are looking to have a totally self contained program, then you can accomplish this two different ways.
Create a new TSQLite3Connection component that links statically against sqlite3 instead of loading the DLL dynamically.
Include the sqlite3.dll as a resource in your program and have your program automatically deploy it before it runs.
Solution #1 is not trivial and not for the faint of heart. I've done it, and I intended to include a link to the component, but the result isn't stable. The problem is that you have to compile a static version of sqlite3, which isn't a real problem, but you have to do it with something like gcc under MinGW and that introduces issues. Compiling with gcc under MinGW means you have to then link in libgcc.a, and because FreePascal's internal linker doesn't know how to interpret stdcall symbols properly, you also have to link against MinGW's libkernel32.a, and libmsvcrt.a. The result just isn't stable. Crashes galore.
Solution #2 should be fairly easy, but the Lazarus maintainers make it a little hard. The part where you store the dll inside the executable as a resource is easy enough to do. And so is writing it out as a temp file. The problem is that you can't tell the TSQLite3Connection component where to find it after. So it looks in the executable's folder, or in system folders. Neither of which can necessarily be written to by the executable. The only place you can guarantee that your program will be able to write to is a temp folder. So what I did is created a new version of TSQLite3Connection component call TSQLite3DynConnection, meaning you can dynamically specify where the DLL is. I made a published property called ClientLibrary where you can specify the location of the dll (it doesn't have to end in .dll, so you can use system temp filename generation routines). You can get this component at: http://icculus.org/~kfitzner/misc/sqlite3dyndll.zip. It will compile against Lazarus 1.6.2 FP 3.0.0, or FP 1.0.6 / FP 2.6.0, which are the two versions I use.
I'll update this answer if I can get the statically linked version stable.
2 Dec 2016 update: I managed to get a static version stable.

Mixing Shared and Static Libraries

I'm using premake5 to build a complex application on multiple platforms. My application links against both static and shared/dynamic external libraries.
There seems to be significant build chain dependencies that break premake generated 'gmake' make files in this case.
Case in Point:
If you mix shared and static libraries in premake 'link' statements, GCC seems to get confused and expect your shared library references to actually be static libraries. When it can't find them the link stage fails. This is normally handled by prefixing your shared libs with '-Bdynamic'. Unfortunately there is no way to tell premake5 that an external link lib is static or dynamic, so you have to manually fixup the make files, which defeats the purpose of a build utility.
This is kind of a showstopper. I don't think you can just feed "-Bdynamic" into the linkoptions because it must be followed by the list of shared libraries.
Seems like a bug in gmake action (or at least a missing functionality)
For those, the best approach is to go to the Premake page on Github (https://github.com/premake/premake-core) and create a new issue.
And if you have the time to provide a small reproductible project (a static lib project, a dynamic one, and an application using both, each with only 1 cpp or some simple stuff + the premake script) and attach it to the issue, it would also be really appreciated (and much easier to treat this issue ^^)

How to debug problems with the javafx native bundle for Windows?

I'm trying to package a JavaFX/DukeScript application into native bundles.
For Windows that's achieved with Inno Setup, that's what the javafxpackager uses.
I can't find much documentation about it, so I'll create a question and provide the answers I find below.
Related questions: 1, 2, 3, 4
So I've created a build.xml Ant script using e(fx)clipse (see example here).
I've been following this excellent blog post.
Now, the bundle task worked and produced an installer and executable app.
I've been having trouble running the app and couldn't see a stack trace.
So here's how to display a window with the stack trace when running the executable from the command line:
goGPS_FX.exe /Debug
That brings up a popup, with the stack trace. In my case, a jar dependency was missing. So just for testing, I manually added that jar under deploy\bundles\goGPS_FX\app\libs.
For the JavaFX launcher to include this new jar, I need to modify the file app\package.cfg.
As far as I understand, that's JavaFX specific, it's not needed by Inno Setup.
Also, I couldn't find much information about this file.
Anyway, there's a property called app.classpath we can update.
The other relevant property is app.mainclass. Here the mainclass is not specified with the standard dotted notation, but rather as a path:
app.mainclass=org/gogpsproject/GoGPS_Fx
Finally, other properties might be available. One I've found was:
jvmarg.1=-Djavafx.verbose=true
So that's a way to pass arguments to the jvm, should you need to.

java.lang.UnsatisfiedLinkError: no db_java-4.2 in java.library.path

I'm trying to access a Berkeley DB database file (say Test.db) from the java code and then want to read all the pairs from it. I'm using the com.sleepycat.db.* and com.sleepycat.bdb.* packages ( NOTE: Not Oracle Berkeley DB JE )to do this task. For the implementation, I did exactly as described in this BerkeleyDB Java API Tutorial
So, when I run this project from eclipse I got this error
Exception in thread "main" java.lang.UnsatisfiedLinkError: no db_java-4.2 in java.library.path
and from the stack trace, source for this exception occured from this line
env = new DbEnv(0);
I tried to set it from the java command
$java -D<name>=<value>
but still it didn't work.
My goal is to export this project into a jar file and use it to read BDBs anywhere . Anyone who knows how to get out of this, I appreciate your help. It would be great if I can be redirected to any working tutorial for accessing Berkeley DBs from java code too. Thanks !!
SOLUTION: After a lot of work, I found out that the Sleepycat API that I'm working on is using native libraries that are written in C/C++ through the JNI. Berkeley DB that is installed on my PC didn't have the shared object file db_java-4.2 (to be accurate, file name is libdb_java-4.2) which means that this BDB is not configured for java APIs.
So, the solution is to start the installation of BDB from scratch again by enabling the java configuration.
I had the same problem.
Like kK_Pulla mentioned, the sleepycat API using native libraries written in C/C++ through the Java Native Interface. So making Sleepycat API calls means the java code is ultimately going to call compiled C/C++ code.
If you are getting the "java.lang.UnsatisfiedLinkError: no db_java-4.2 in java.library.path" error then it is likely that, at least it was in my case, you have not included the relevant native libraries in your project.
I would describe what I did to fix it in my case. Firstly let me describe my environment.
IntelliJ maven Java project on a linux machine.
Built Berkeley DB version 18.1.40.
The Berkeley build included the --enable-java switch in the configuration phase.
I included the db.jar (found in the build_unix directory) as a module dependency through the Intellij menus File> Project Structure. I selected Modules under Project settings on the left pane and then under the dependencies tab I added the db.jar file as a dependency.
This was the state of my project before I encountered the UnsatisfiedLinkError. To fix that I added the native libraries (found in the unix_build/.libs directory) to the project. You can do this by selecting the File menu> then Project Structure. On the left pane under project settings select Libraries and then click the + sign on the right pane to add the directory for the native libs.
This fixed my problem.

Resources