Does anyone know what the difference is between adding an appropriate JAR-file (eg. Apache XALAN) to a JRE's endorsed directory and adding it to the application's classpath?
Is it possible to take a jar-file that can be added to the endorsed lib and instead add it to the classpath?
Tecnically you probaly can do that, but the difference is that the jar files in the endorsed directory are loaded by the bootstrap classloader, which is probably not the same classloader as the one that loads your jars from the classpath.
There is a classic XML problem with the Xerces and Xalan XML implementations which are out in the endorsed directory. Because newer applications sometimes require newer versions of both libraries, and the classes have the same names, there is a classpath problem.
You can replace the Xerces and Xalan libraries in the endorsed dir (backup your old ones!) but that can possibly screw up other applications which use the same JRE installation.
I've even seen this problem within 1 application where one library depends on one version, and another library depends on another version of xerces. Very troublesome, and there's no "general approach" to this, or at least none that I found.
There are some interesting articles easily found by Google on this, try to find one which best matches your situation or problem.
Related
So I've been trying to do some AWS lambda with scala and sbt. And one of the recommendations is
Minimize your deployment package size to its runtime necessities. This
will reduce the amount of time that it takes for your deployment
package to be downloaded and unpacked ahead of invocation. For
functions authored in Java or .NET Core, avoid uploading the entire
AWS SDK library as part of your deployment package.
I've been using sbt-assembly to create fat jars for my code and dependencies, but it seems like sbt-assembly will package all library dependencies when I only use like 10% of the aws-core library which adds a lot of content. Was wondering if there is something I can do to cut down on the number of dependencies to what is actually imported in my code (and their dependencies).
As far as I know, there is no direct and safe way to selectively contains the dependent classes in a fat jar generated by the sbt-compile plugin.
First of all, you should understand that sbt plugins just provide a settings and jar files required to invoke methods in your project. It means that the dependent plugin is brought to your project with the pre-compiled jar file, which is determined by the version that you specified on your build setting (e.g., plugins.sbt in your project dir).
For example, the jars of sbt-assembly are brought to your project from this link when you specify that you want to use the sbt-assembly (although it is brought to your project when you use recent sbt version by default).
Therefore, at least you may have two choices to shrink your jar files.
Compile the jar file from the scratch
For the sbt-aws, its source code is provided on this link, so you may selectively compile the source codes to get the classes that your program is going to use.
Use the tool for shrinking jar file
There are several tools to shrink your jar file based on the dependencies. The most popular tool is proguard; it seems that there is a proguard support for sbt.
Warning
As mentioned in another stack overflow answer, selectively choosing some classes from the jar may cause your program crash depending on the input value and several other conditions. You've said that only 10 percent of the jar file is used, but you cannot ensure whether other classes are required from your code and library that your project depends on. When you use the tool for helping you to shrink the jar file, be careful when the program is security critical one.
I have a jar that works on Java 8.
I would like to create a new jar, that is going to be Multi-Release JAR but empty, just with 'patched' classes in META-INF/versions.
I would like to have a separate jar, so people can include it on Java9, otherwise, they use the default one. Why? Because so many tools are not yet prepared for Java9 MR-Jars.
Would this be possible? Would Java9 MR-Jar override classes from others jars?
Why?
The idea behind Multi-Release jars is that they provide simple patching. In my humble opinion, the way MR jars works is not satisfying.
There are two reasons why I can't make 2 separate Jars:
try to make cross-compile source base that works with Java8 and Java9. You would end up with folders like java, java8 and java9... and then have the build produce two jars, two poms... Yeah, good luck.
Imagine that I even build a library for java9. What about transient dependencies? That would mean that all other libraries that uses mine, would need to have jre8 version that depends on my jre8 version. Just because there is Java9 version!
Here is the story:
My A is a Java library built on Java8 but packaged as Multi-Release Jar which means it contains additional classes for when jar is run on Java9. Additional classes are built separately on JDK9 and I copied them manually (yeah, I know, but it works for now).
Unfortunately, some tools and servers (Jetty) are not aware of MR Jars and this makes them NOT working.
For that reason, I have A-jre8 version of my library, that comes without any extra classes, so servers can use it.
However, if user is using library B that depends on my A, he will still get the MRJar version of A and this will fail again. I want to be able to prevent this somehow. And I can't say to B: hey, could you make B-jre8?
Possible solution
JAR is just about packaging!
Allow the separate jar to patch existing jar.
In my case, I would just include A.jar9 and Java would consider A.jar and A.jar9 together as a package. No need for META-INF/versions. Very clean. And, best of all, it would help in situations like above! If run on Java8, the jar9 jar would make no difference; if run on Java9 the jar9 jar would patch the jar with the same name. Simple as that. No transitive dependency hell.
Rename classes in META-INF/versions.
Common Oracle, have you ever heard about the classpath scanning? Could you at least rename the classes in versions to e.g. *.class9 so not to be caught by existing classpath scanners.
As it is today (Java v9.0.4) - no.
I am working at a library needing some dependencies.
For ease of deployment, I want to create a JAR file containing everything, including the dependencies.
I have tried sbt-assembly - this works, but it may be inadvisable due to legal reasons, so I'm looking for a solution where the resulting JAR file has the original JAR files inside, and where the classpath entry in MANIFEST.MF is set up such that client classes may just add this "nested JAR file" into their classpaths.
Is something like this even possible? sbt-one-jar nearly does, what I want, but only for executables - my product will result in a library, so this is not a perfect fit.
As I've used SBT so far, an SBT plugin would be easiest to use, as it is rather too much work to convert everyting to maven or gradle or ... now.
After thinking a bit more about how class lookup works, we dediced to abandon this experiment.
Basically classes are loaded by ClassLoader instances, and the standard class loaders for applications use a fixed strategy of how to find classes in JAR files or directories.
It seems that to allow a library to be located in a hierarchical JAR file, we must also provide the user of this library (i.e. the library client) with a special classloader so that our client may load all needed classes from the hierarchical JAR.
This is too much work to be worth it - the whole idea of a hierarchical JAR was enteratained only to simplify deployment, and having to juggle own classloaders would nullify this simplification.
In short - possible, but probably not worth the effort.
So i received a java api from a client and the main code is in main.jar. But the instructions he gave me require me to add these other jars (a.jar, b.jar, etc..) into the classpath whenever I want to use main.jar. These other supporting jars are things like Xerces, jakarta-oro, and a few other publicly available libraries. The problem is i don't know what versions they are, so i'm not sure if there would be issues if i just update the pom.xml file in my app to depend on main.jar and also have dependencies to these other jars as well with the latest versions of them.
Whats the best strategy for using main.jar in my maven application? I'm planning on adding main.jar to our internal maven repository, but what should i do about the xerces, jakarta-oro, and other jars when i dont know what versions they are?
Thanks
If you are lucky the file /META-INF/MANIFEST.MF inside a.jar, b.jar etc. contains an entry "Implementation-Version" or some other useful information which tell you what version they are. If not, you can download the latest release(s) from the project web site and check if they have the same file size as your bundled dependencies.
You may also come to the idea to bundle the dependencies with the main.jar in one big jar, but this may become funny, when you have the dependencies twice in your classpath at some point in the future...
What about just asking the client what version numbers this dependencies have?
If you don't have any information about these third-party libraries, just add them to src/resources/META-INF/lib and commit to SVN. That's the best way, if we're talking about black box approach.
As part of improvements to our build process, we are currently debating whether we should have separate project/solution files on our CI production environment from our local development environments.
The reason this has come about is because of reference problems we experienced in our previous project. On a frequent basis people would mistakenly add a reference to an assembly in the wrong location, which would mean it would work okay on their local environment, but might break on someone else's or on the build machine.
Also, the reference paths are in the csproj.user files which means these must be committed to source control, so everyone has to share these same settings.
So we are thinking about having separate projects and solutions on our CI server, so that when we do a build it uses these projects rather than local development ones.
It has obvious drawbacks such as an overhead to maintaining these separate files and the associated process that would need to be defined and followed, but it has benefits in that we would be in more control over EXACTLY what happens in the production environment.
What I haven't been able to find is anything on this subject - can't believe we are the only people to think about this - so all thoughts are welcome.
I know it's anachronistic. But the single best way I've found to handle the references issue is to have a folder mapped to a drive letter such as R: and then all projects build into or copy output into that folder also. Then all references are R:\SomeFile.dll etc. This gets you around the problem that sometimes references are added by absolute path and sometimes they are added relatively. (there's something to do with "HintPath" which I can't really remember)
The nice thing then, is that you can still use the same solution files on your build server. Which to be honest is an absolute must as you lose the certainty that what is being built on the dev machine is the same as on the build server otherwise.
In our largest project (a system comprising of many applications) we have the following structure
/3rdPartyAssemblies /App1 /App2 /App3 /.....
All external assemblies are added to 3rdPartyAssemblies/Vendor/Version/...
We have a CoreBuild.sln file which acts as an MSBuild script for all of the assemblies that are shared to ensure building in dependancy order (ie, make sure App1.Interfaces is built before App2 as App2 has a reference to App1.Interfaces).
All inter-application references target the /bin folder (we don't use bin/debug and bin/release, just bin, this way the references remain the same and we just change the release configuration depending on the build target).
Cruise Control builds the core solution for any dependencies before building any other app, and because the 3rdPartAssemblies folder is present on the server we ensure developer machines and build server have the same development layout.
Usually, you would be creating Build projects/scripts in some form or another for your Production, and so putting together another Solution file doesn't come in the picture.
It would be easier to train everyone to use project references, and create a directory under the project file structure for external assembly references. This way everyone follows the same environment.
We have changed our project structure (making use of SVN Externals) where each project is now completely self-contained. That is, any references never go outwith the project directory (for example, if Project A references ASM X, then ASM X exists within a subfolder of ProjectA)
I suspect that this should go some way towards helping solve some of our problems, but I can still see some advantages of having more control over the build projects.
#David - believe it or not this is what we actually have just now, and yet it's still causing us problems!
We're making some changes though, which are forced upon us due to moving to TeamCity and multiple build agents - so we can't have references to directories outwith the current project, as I've mentioned in my previous answer.
Look at the Externals section of this link to see what I mean - http://www.dummzeuch.de/delphi/subversion/english.html
I would strongly recommend against this.
Reference paths aren't only stored in the .user file. A hint path is stored in the project file itself. You should never have to check a .user file into source control.
Let there be one set of (okay, possibly versioned) solution/project files which all developers use, and the Release configurations of which are what you're ultimately building in production. Having separate project files is going to cause confusion down the road, when some project setting is tweaked, not carried across, and slipped into production.
You might also check this out:
http://www.objectsharp.com/cs/blogs/barry/archive/2004/10/29/988.aspx
http://bytes.com/forum/thread268546.html