Vaadin Flow 14, Jetty embedded and static files - jar

I'm trying to create app based on Jetty 9.4.20 (embedded) and Vaadin Flow 14.0.12.
It based on very nice project vaadin14-embedded-jetty.
I want to package app with one main-jar and all dependency libs must be in folder 'libs' near main-jar.
I remove maven-assembly-plugin, instead use maven-dependency-plugin and maven-jar-plugin. In maven-dependency-plugin i add section <execution>get-dependencies</execution> where i unpack directories META-INF/resources/,META-INF/services/ from Vaadin Flow libs to the result JAR.
In this case app work fine. But if i comment section <execution>get-dependencies</execution> then result package didn't contain that directories and app didn't work.
It just cannot give some static files from Vaadin Flow libs.
This error occurs only if i launch packaged app with ...
$ java -jar vaadin14-embedded-jetty-1.0-SNAPSHOT.jar
... but from Intellij Idea it launch correctly.
There was an opinion that is Jetty staring with wrong ClassLoader and cannot maintain requests to static files in Jar-libs.

The META-INF/services/ files MUST be maintained from the Jetty libs.
That's important for Jetty to use java.util.ServiceLoader.
If you are merging contents of JAR files into a single JAR file, that's called a "uber jar".
There are many techniques to do this, but if you are using maven-assembly-plugin or maven-dependency-plugin to build this "uber jar" then you will not be merging critical files that have the same name across multiple JAR files.
Consider using maven-shade-plugin and it's associated Resource Transformers to properly merge these files.
http://maven.apache.org/plugins/maven-shade-plugin/
http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html
The ServicesResourceTransformer is the one that merges META-INF/services/ files, use it.
As for static content, that works fine, but you have to setup your Base Resource properly.
Looking at your source, you do the following ...
final URI webRootUri = ManualJetty.class.getResource("/webapp/").toURI();
final WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));
That won't work reliably in 100% of cases (as you have noticed when running in the IDE vs command line).
The Class.getResource(String) is only reliable if you lookup a file (not a directory).
Consider that the Jetty Project Embedded Cookbook recipes have techniques for this.
See:
WebAppContextFromClasspath.java
ResourceHandlerFromClasspath.java
DefaultServletFileServer.java
DefaultServletMultipleBases.java
XmlEnhancedServer.java
MultipartMimeUploadExample.java
Example:
// Figure out what path to serve content from
ClassLoader cl = ManualJetty.class.getClassLoader();
// We look for a file, as ClassLoader.getResource() is not
// designed to look for directories (we resolve the directory later)
URL f = cl.getResource("webapp/index.html");
if (f == null)
{
throw new RuntimeException("Unable to find resource directory");
}
// Resolve file to directory
URI webRootUri = f.toURI().resolve("./").normalize();
System.err.println("WebRoot is " + webRootUri);
WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));

Related

Why does `getResourceAsStream` sometimes load a resource even when there is a typo in the resource path?

I have a Jar (we'll call it a.jar) with a resource in it at path foo/bar.txt and a function as follows:
object FooBarLoader {
fun loadFooBarText() = javaClass.getResourceAsStream("foo//bar.txt")
?.bufferedReader()
?.readLines()
?.joinToString("\n")
}
When I test the function in a unit test (JUnit 4, running with Gradle 6), it loads the text from the resource file despite the obvious typo (the // in the middle of the resource path).
I also have a CLI application (in b.jar) that has a dependency on a.jar. When the CLI application calls loadFooBarText(), it got a null result due to the resource not being found. This was fixed by fixing the typo (// -> /) in the function in a.jar. No other changes were needed to fix it.
So, my question is why did the wrong path work in one situation (unit tests of a.jar) and not the other (call from b.jar)?
How do you run the unit test with a.jar ? Just run it in your IDE or use command java -jar a.jar ?
If you ran it just in IDE,I think difference is the search path between local files and zip files .
Your first application searches the file in your target directory and the second application searches it in the jar which is a compressed file.
When searching files in local path, command will be changed to right one by system.
The two commands below are the same in both Windows/Linux.
cd work//abc/ddd
cd work/abc/ddd
But when searching files in a jar file which is actually compressed zip file, path should be a restrict written or else the program will find nothing.

FileReader can't find R Script

I try run my R Script within JavaFx. I use Renjin for this purpose and it seems to work properly with statements I run internally. But I want to run an external R Script. The project is set up with Maven so the path should be easy as the R Script is in the resources folder. The path works when I load FXML files, so I'm pretty confused why it can't find my Script.
Here's a short example:
package survey;
import javax.script.*;
import org.renjin.script.*;
import java.io.FileReader;
public class calcFunction {
public static void main(String[] args) throws Exception {
// create a script engine manager:
RenjinScriptEngineFactory factory = new RenjinScriptEngineFactory();
// create a Renjin engine:
ScriptEngine engine = factory.getScriptEngine();
engine.put("x", 4);
engine.put("y", 5);
engine.eval(new FileReader("/test.R"));
}
}
Is something missing? Thanks in advance!
EDIT1:
With my FXML files it works with the "/" path like this:
root = FXMLLoader.load(getClass().getResource("/moduleDa.fxml"));
EDIT2:
Someone who deleted his comment proposed this:
engine.eval(new FileReader(new File(".").getAbsolutePath()+"/test.R"));
It works if the script is in the root directory, where the pom.xml file is located. #James_D made it work so the R script can be located in the resources folder - thanks a lot!
If your R script is bundled as part of the application, it can't be treated as a file - you need to treat it as a resource. Typically, you will deploy your application as a Jar file, and the resources will be elements within that jar file (they won't be files in their own right).
So just treat the R script as a resource and load it as such. I don't know the renjin framework, but I assume ScriptEngine here is a javax.script.ScriptEngine, in which case ScriptEngine.eval(...) takes a Reader as a parameter, and so (if your R script is located in the root of the class path) you can do
engine.eval(new InputStreamReader(getClass().getResourceAsStream("/test.R")));

Spring Boot - Loading a .properties file from WEB-INF/ folder

I have a Spring Boot 1.3 application (deployed as .war) that needs to be able read a .properties file from the following location:
WEB-INF/application.properties (outside the classpath, but relative to the app root folder)
...as opposed to:
WEB-INF/classes/application.properties(inside the classpath, gets loaded automatically)
What worked in Spring Boot 1.3 was the following #PropertySource annotation:
#SpringBootApplication
#PropertySource(value = {"WEB-INF/application.properties"})
public class MyApplication {
public static void main(String[] args) {
SpringApplication.run(MyApplication.class, args);
}
}
It correctly fetched the .properties file relative to the app root. However that stops working after an update to Spring Boot 1.4.0.RC1.
Since then I've tried the following:
#PropertySource("classpath:../application.properties")
#PropertySource("file:WEB-INF/application.properties")
and also
spring.config.location=classpath:../
spring.config.location=file:src/main/webapp/WEB-INF/
spring.config.location=WEB-INF/application.properties
But haven't had any luck loading the .properties.
I'd normally put the .properties file inside the classpath, but in this case this is NOT an option due to the way our deployment works on a remote location.
I'd also prefer to not use an absolute path, as that'll be a nightmare to support with multiple customers.
Edit: Just to be clear - the .properties I'd like to read aren't located outside the JAR (in my case - WAR) file, but inside - just not on the classpath, but directly in the WEB-INF/ folder where normally other resources (pages, images) would be.
As I mentioned in duplicate SO question:
Put this line into your application.properties:
logging.config=file:log4j.xml
Second option is to pass system variable to -Dlogging.config=file:log4j.xml
In this case it is expected to be located in current directory outside of the JAR file.
REACTION ON COMMENT:
If you are using WAR file, your main class is not used at all. So PropertySource annotation doesn't have any effect there.
If the .properties is packed in the .war file. Then you can try the following (assuming that the WEB-INF directory is located in the root of the .war file.
#PropertySource("classpath:/WEB-INF/conf/application.properties")
Turns out this issue was caused by a bug with the SpringBootTestContextBootstrapper in Spring Boot 1.4.0.RC1: https://github.com/spring-projects/spring-boot/issues/6371

Can't deploy precompiled, merged webapp to Azure

I'm trying to deploy an ASP.NET web application to Azure. It's hybrid Web Forms, MVC, and WebAPI, and there are a TON of aspx/ascx files, such that they really need to be precompiled or every deploy will render the site sluggish for awhile.
I am trying to deploy via SCM integration with GitHub via kudu, with precompiled views, all merged to a single assembly.
Note that:
Deploy works fine with precompilation disabled.
Deploy works fine from Visual Studio
Build works fine if I copy the msbuild command from the Azure log, replace the relevant paths, and run it locally on my Windows 8.1 machine.
I've set up the Advanced Precompile settings as:
Don't allow precompiled site to be udpatable
Don't emit debug information
Merge all pages and control outputs to a single assembly = AppViews.dll
Here's the .deployment file for Azure
[config]
project = WebSite/WebSite.csproj
SCM_BUILD_ARGS=/p:Configuration=Release;PublishProfile=azure-prod /v:n
You notice I'm sending the verbosity /v to "normal" for extra diagnostic information.
Here is info I get toward the tail of the deployment log:
AspNetPreCompile:
D:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v \ -p D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\Source -c D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir
GenerateAssemblyInfoFromExistingAssembleInfo:
Creating directory "obj\Release\AssemblyInfo".
D:\Windows\Microsoft.NET\Framework\v4.0.30319\Csc.exe /out:obj\Release\AssemblyInfo\AssemblyInfo.dll /target:library Properties\AssemblyInfo.cs
AspNetMerge:
Running aspnet_merge.exe.
D:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools\aspnet_merge.exe D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir -w AppViews.dll -copyattrs obj\Release\AssemblyInfo\AssemblyInfo.dll -a
aspnet_merge : error 1003: The directory 'D:\home\site\repository\WebSite\obj\Release\AspnetCompileMerge\TempBuildDir' does not exist. [D:\home\site\repository\WebSite\WebSite.csproj]
Done Building Project "D:\home\site\repository\WebSite\WebSite.csproj" (Build;pipelinePreDeployCopyAllFilesToOneFolder target(s)) -- FAILED.
Build FAILED.
It looks like aspnet_compiler.exe runs, but doesn't do what it's supposed to, which is why the TempBuildDir directory (supposed to be the output of the compiler) does not exist in time for the AspNetMerge target. Contrast that with my system, where that directory DOES in fact exist, containing the marker aspx/ascx/etc. files, static content, a PrecompiledApp.config file, and a whole mess of stuff in the bin directory.
aspnet_compiler.exe has an -errorstack flag but it's not clear to me how I could get MSBuild to add this just via the .deployment file, or even if that app is really even throwing an error.
I could just deploy via Visual Studio, but I would really like to take advantage of the SCM integration so I can just push to my prod branch and let it go. Any suggestions?
I replied on https://github.com/projectkudu/kudu/issues/1341, but copying my answer here in case someone lands here...
Way back, we had found that aspnet_compiler.exe was not working within Azure Websites due to how it dealt with the profile folder. We made a change at the time that's a bit of a hack but got us going: we turned it into a no-op, by pointing HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\aspnet_compiler.exe to our own dummy exe (D:\Program Files (x86)\aspnet_compiler\KuduAspNetCompiler.exe).
But trying it now, it appears to work correctly today, likely thanks to improvements in the Azure Websites hosting environment. So we will try getting rid of this hack and doing a full test pass to make sure it doesn't cause any major regressions. If all goes well, we can get that into production, which should enable those scenarios.
In the short term, you may be able to work around this by having your build script:
copy aspnet_compiler.exe from D:\Windows\Microsoft.NET\Framework\v4.0.30319 into your own site files, but under a different name (e.g. aspnet_compiler2.exe)
convince msbuild to use that one
Note: This GitHub issue on projectkudu will eventually make this solution obsolete, but for the meantime, that issue is filed as Backlog, and this works right now.
Thank you thank you David Ebbo. With this information, I was able to bootstrap my build to work for the short term.
First, I downloaded the aspnet_compiler.exe from the Azure instance using the Diagnostic Console available at https://{WEBSITE_NAME}.scm.azurewebsites.net/DebugConsole and added that to my own repository. This way there's no question about any difference between 32/64-bit, etc. I renamed it to azure_aspnet_compiler.exe in my repository.
Second, the AspNetCompiler task doesn't give you the option to change the tool name. It's hardcoded, but as a virtual property so it's overrideable. So I had to create my own task class, and package it in its own assembly, which I built in Release mode and also included in my repository.
public class AzureAspNetCompiler : Microsoft.Build.Tasks.AspNetCompiler
{
private string _toolName = "aspnet_compiler.exe";
protected override string ToolName
{
get { return _toolName; }
}
public string CustomToolName // Because ToolName cannot have a setter
{
get { return _toolName; }
set { _toolName = value; }
}
}
Next I needed to replace the AspNetPreCompile task in MSBuild, but I couldn't figure out how to do that directly. But that task wasn't doing anything anyway, so why not just run right after it?
I added this to the top of my Website.csproj file to import the DLL containing the AzureAspNetCompiler class. Note that the path is relative to the Website.csproj file I'm editing.
<UsingTask TaskName="AzureBuildTargets.AzureAspNetCompiler"
AssemblyFile="..\DeploymentTools\AzureBuildTargets.dll" />
Then I added this right below it, which is basically stealing the MSBuild target definition of AspNetPreCompile from C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v11.0\Web\Transform\Microsoft.Web.Publishing.AspNetCompileMerge.targets, with some of the property setting stuff near the top of it left out (because the original task will do that for us anyway.) Just take note of the ToolPath and CustomToolName values at the bottom of the (renamed) AzureAspNetCompiler element.
<PropertyGroup>
<!--Relative to solution root apparently-->
<LocalRepoDeploymentTools>.\DeploymentTools</LocalRepoDeploymentTools>
<AzureAspnetCompilerPath>$([System.IO.Path]::GetFullPath($(LocalRepoDeploymentTools)))</AzureAspnetCompilerPath>
</PropertyGroup>
<Target Name="NoReallyAspNetPreCompile" AfterTargets="AspNetPreCompile">
<AzureAspNetCompiler
PhysicalPath="$(_PreAspnetCompileMergeSingleTargetFolderFullPath)"
TargetPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)"
VirtualPath="$(_AspNetCompilerVirtualPath)"
Force="$(_AspNetCompilerForce)"
Debug="$(DebugSymbols)"
Updateable="$(EnableUpdateable)"
KeyFile="$(_AspNetCompileMergeKeyFile)"
KeyContainer="$(_AspNetCompileMergeKeyContainer)"
DelaySign="$(DelaySign)"
AllowPartiallyTrustedCallers="$(AllowPartiallyTrustedCallers)"
FixedNames="$(_AspNetCompilerFixedNames)"
Clean="$(Clean)"
MetabasePath="$(_AspNetCompilerMetabasePath)"
ToolPath="$(AzureAspnetCompilerPath)"
CustomToolName="azure_aspnet_compiler.exe"
/>
<!--
Removing APP_DATA is done here so that the output groups reflect the fact that App_data is
not present
-->
<RemoveDir Condition="'$(DeleteAppDataFolder)' == 'true' And Exists('$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data')"
Directories="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\App_Data" />
<CollectFilesinFolder Condition="'$(UseMerge)' != 'true'"
RootPath="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)" >
<Output TaskParameter="Result" ItemName="_AspnetCompileMergePrecompiledOutputNoMetadata" />
</CollectFilesinFolder>
<ItemGroup Condition="'$(UseMerge)' != 'true'">
<FileWrites Include="$(_PostAspnetCompileMergeSingleTargetFolderFullPath)\**"/>
</ItemGroup>
With this in place, everything works as I would expect it to.

Setting Jetty resourcebase to static file embedded in the same jar file

I am trying to access static resource (eg. first.html) packed inside the same .jar file (testJetty.jar), which also has a class which starts the jetty (v.8) server (MainTest.java). I am unable to set the resource base correctly.
The structure of my jar file (testJetty.jar):
testJetty.jar
first.html
MainTest.java
==
Works fine on local machine, but when I wrap it in jar file and then run it, it doesn't work, giving "404: File not found" error.
I tried to set the resourcebase with the following values, all of which failed:
a) Tried setting it to .
resource_handler.setResourceBase("."); // Results in directory containing the jar file, D:\Work\eclipseworkspace\testJettyResult
b) Tried getting it from getResource
ClassLoader loader = this.getClass().getClassLoader();
File indexLoc = new File(loader.getResource("first.html").getFile());
String htmlLoc = indexLoc.getAbsolutePath();
resource_handler.setResourceBase(htmloc); // Results in D:\Work\eclipseworkspace\testJettyResult\file:\D:\Work\eclipseworkspace\testJettyResult\testJetty1.jar!\first.html
c) Tried getting the webdir
String webDir = this.getClass().getProtectionDomain()
.getCodeSource().getLocation().toExternalForm();
resource_handler.setResourceBase(webdir); // Results in D:/Work/eclipseworkspace/testJettyResult/testJetty1.jar
None of these 3 approaches worked.
Any help or alternative would be appreciated
Thanks
abbas
The solutions provided in this thread work but I think some clarity to the solution could be useful.
If you are building a fat jar and use the ProtectionDomain way you may hit some issues because you are loading the whole jar!
class.getProtectionDomain().getCodeSource().getLocation().toExternalForm();
So the better solution is the other provided solution
contextHandler.setResourceBase(
YourClass.class
.getClassLoader()
.getResource("WEB-INF")
.toExternalForm());
The problem here is if you are building a fat jar you are not really dumping your webapp resources into WEB-INF but are probably going into the root of the jar, so a simple workaround is to create a folder XXX and use the second approach as follows:
contextHandler.setResourceBase(
YourClass.class
.getClassLoader()
.getResource("XXX")
.toExternalForm());
Or change your build tool to export the webapp files into that given directory. Maybe Maven does this on a Jar for you but gradle does not.
Not unusually, I found a solution to my problem. The 3rd approach mentioned by Stephen in Embedded Jetty : how to use a .war that is included in the .jar from which Jetty starts? worked!
So, I changed from Resource_handler to WebAppContext, where WebAppContext is pointing to the same jar (testJetty.jar) and it worked!
String webDir = MainTest.class.getProtectionDomain()
.getCodeSource().getLocation().toExternalForm(); ; // Results in D:/Work/eclipseworkspace/testJettyResult/testJetty.jar
WebAppContext webappContext = new WebAppContext(webDir, "/");
It looks like ClassLoader.getResource does not understand an empty string or . or / as an argument. In my jar file I had to move all stuf to WEB-INF(any other wrapping dir will do). So the code looks like
contextHandler.setResourceBase(EmbeddedJetty.class.getClassLoader().getResource("WEB-INF").toExternalForm());
so the context looks like this then:
ContextHandler:744 - Started o.e.j.w.WebAppContext#48b3806{/,jar:file:/Users/xxx/projects/dropbox/ui/target/ui-1.0-SNAPSHOT.jar!/WEB-INF,AVAILABLE}

Resources