I have a Jar (we'll call it a.jar) with a resource in it at path foo/bar.txt and a function as follows:
object FooBarLoader {
fun loadFooBarText() = javaClass.getResourceAsStream("foo//bar.txt")
?.bufferedReader()
?.readLines()
?.joinToString("\n")
}
When I test the function in a unit test (JUnit 4, running with Gradle 6), it loads the text from the resource file despite the obvious typo (the // in the middle of the resource path).
I also have a CLI application (in b.jar) that has a dependency on a.jar. When the CLI application calls loadFooBarText(), it got a null result due to the resource not being found. This was fixed by fixing the typo (// -> /) in the function in a.jar. No other changes were needed to fix it.
So, my question is why did the wrong path work in one situation (unit tests of a.jar) and not the other (call from b.jar)?
How do you run the unit test with a.jar ? Just run it in your IDE or use command java -jar a.jar ?
If you ran it just in IDE，I think difference is the search path between local files and zip files .
Your first application searches the file in your target directory and the second application searches it in the jar which is a compressed file.
When searching files in local path, command will be changed to right one by system.
The two commands below are the same in both Windows/Linux.
cd work//abc/ddd
cd work/abc/ddd
But when searching files in a jar file which is actually compressed zip file, path should be a restrict written or else the program will find nothing.
Related
I'm trying to create app based on Jetty 9.4.20 (embedded) and Vaadin Flow 14.0.12.
It based on very nice project vaadin14-embedded-jetty.
I want to package app with one main-jar and all dependency libs must be in folder 'libs' near main-jar.
I remove maven-assembly-plugin, instead use maven-dependency-plugin and maven-jar-plugin. In maven-dependency-plugin i add section <execution>get-dependencies</execution> where i unpack directories META-INF/resources/,META-INF/services/ from Vaadin Flow libs to the result JAR.
In this case app work fine. But if i comment section <execution>get-dependencies</execution> then result package didn't contain that directories and app didn't work.
It just cannot give some static files from Vaadin Flow libs.
This error occurs only if i launch packaged app with ...
$ java -jar vaadin14-embedded-jetty-1.0-SNAPSHOT.jar
... but from Intellij Idea it launch correctly.
There was an opinion that is Jetty staring with wrong ClassLoader and cannot maintain requests to static files in Jar-libs.
The META-INF/services/ files MUST be maintained from the Jetty libs.
That's important for Jetty to use java.util.ServiceLoader.
If you are merging contents of JAR files into a single JAR file, that's called a "uber jar".
There are many techniques to do this, but if you are using maven-assembly-plugin or maven-dependency-plugin to build this "uber jar" then you will not be merging critical files that have the same name across multiple JAR files.
Consider using maven-shade-plugin and it's associated Resource Transformers to properly merge these files.
http://maven.apache.org/plugins/maven-shade-plugin/
http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html
The ServicesResourceTransformer is the one that merges META-INF/services/ files, use it.
As for static content, that works fine, but you have to setup your Base Resource properly.
Looking at your source, you do the following ...
final URI webRootUri = ManualJetty.class.getResource("/webapp/").toURI();
final WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));
That won't work reliably in 100% of cases (as you have noticed when running in the IDE vs command line).
The Class.getResource(String) is only reliable if you lookup a file (not a directory).
Consider that the Jetty Project Embedded Cookbook recipes have techniques for this.
See:
WebAppContextFromClasspath.java
ResourceHandlerFromClasspath.java
DefaultServletFileServer.java
DefaultServletMultipleBases.java
XmlEnhancedServer.java
MultipartMimeUploadExample.java
Example:
// Figure out what path to serve content from
ClassLoader cl = ManualJetty.class.getClassLoader();
// We look for a file, as ClassLoader.getResource() is not
// designed to look for directories (we resolve the directory later)
URL f = cl.getResource("webapp/index.html");
if (f == null)
{
throw new RuntimeException("Unable to find resource directory");
}
// Resolve file to directory
URI webRootUri = f.toURI().resolve("./").normalize();
System.err.println("WebRoot is " + webRootUri);
WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));
u-boot support many platform. and there are files with same file name. it's hard to determine which file is involved in the make process for a certain platform. how could I get all files that be used in make process?
You could:
first, build u-boot for your target platform,
then look for all object files that resulted from the compilation process.
For example, if your u-boot main makefile were located in /opt/u-boot-2019.01,
the following commands would give you all the object files that were compiled:
cd /opt/u-boot-2019.01
find . -name "*.o"
You can then correlate the list of files you retrieved with the content of your target board's config file, usually located in the configs sub-directory.
In my case, the first object files to be displayed are:
./scripts/kconfig/zconf.tab.o
./scripts/kconfig/conf.o
./scripts/dtc/srcpos.o
./scripts/dtc/dtc.o
./scripts/dtc/treesource.o
./scripts/dtc/util.o
./scripts/dtc/fstree.o
./scripts/dtc/checks.o
./scripts/dtc/flattree.o
./scripts/dtc/dtc-parser.tab.o
./scripts/dtc/livetree.o
./scripts/dtc/dtc-lexer.lex.o
./scripts/dtc/data.o
./arch/arm/cpu/built-in.o
./arch/arm/cpu/armv8/built-in.o
./arch/arm/cpu/armv8/cpu-dt.o
./arch/arm/cpu/armv8/cache_v8.o
./arch/arm/cpu/armv8/generic_timer.o
./arch/arm/cpu/armv8/exceptions.o
./arch/arm/cpu/armv8/lowlevel_init.o
./arch/arm/cpu/armv8/fwcall.o
./arch/arm/cpu/armv8/cpu.o
./arch/arm/cpu/armv8/start.o
./arch/arm/cpu/armv8/cache.o
./arch/arm/cpu/armv8/transition.o
./arch/arm/cpu/armv8/tlb.o
./arch/arm/mach-sunxi/built-in.o
./arch/arm/mach-sunxi/clock.o
./arch/arm/mach-sunxi/dram_helpers.o
./arch/arm/mach-sunxi/pinmux.o
./arch/arm/mach-sunxi/prcm.o
./arch/arm/mach-sunxi/board.o
./arch/arm/mach-sunxi/clock_sun6i.o
./arch/arm/mach-sunxi/cpu_info.o
./arch/arm/mach-sunxi/rsb.o
For example, the fact that the file ./arch/arm/mach-sunxi/rsb.o is in the list means that ./arch/arm/mach-sunxi/rsb.c was compiled during the build process and contributed to the resulting u-boot image.
I'm using skipper to receive the files, sharp to resize (and save) and fs unlink to remove the old image. But I got a very weird error this time that concerns me a lot:
error: ** Grunt :: An error occurred. **
error:
Aborted due to warnings.
Running "copy:dev" (copy) task
Warning: Unable to read "assets/images/users/c8e303ca-1036-4f52-88c7-fda7e01b6bba.jpg" file (Error code: ENOENT).
error: Looks like a Grunt error occurred--
error: Please fix it, then restart Sails to continue running tasks (e.g. watching for changes in assets)
error: Or if you're stuck, check out the troubleshooting tips below.
error: Troubleshooting tips:
error:
error: *-> Are "grunt" and related grunt task modules installed locally? Run npm install if you're not sure.
error:
error: *-> You might have a malformed LESS, SASS, CoffeeScript file, etc.
error:
error: *-> Or maybe you don't have permissions to access the .tmp directory?
error: e.g., (edited for privacy)/sails/.tmp ?
error:
error: If you think this might be the case, try running:
error: sudo chown -R 1000 (edited for privacy)/sails/.tmp
Grunt stopped running and to have that in production is a big NoNo... I believe that this is caused because of concurrency with fs.unlinkSync(fname). The error is also intermittent and very hard to reproduce in some machines (IO ops/sec maybe?).
I have the following controller action:
var id = 1; // for example
req.file('avatar').upload({
dirname: require('path').resolve(sails.config.appPath, 'assets/images')
}, function(err, files){
var avatar = files.pop();
//file name operations here. output is defined as the path + id + filetype
//...
sharp(avatar.fd)
.resize(800, 800)
.toFile(output, (err, info)=>{
if(err){
res.badRequest();
} else {
fs.unlinkSync(avatar.fd);
res.ok();
}
});
});
Now I've been thinking about a few solutions:
Output the new image directly to .temp
Unlink when files exists on .tmp. Explanation: Grunt already copied the old file so removing it would be safe!
But I don't know if this is some spaghetti code or even if a better solution exists.
EDIT: My solution was, as proposed by arbuthnott, wrap a controller like this:
get : function(req, res){
var filepath = req.path.slice(1,req.path.length);
//remove '/' root identifier. path.resolve() could be used
if(fs.existsSync(path.resolve(filepath))){
return res.sendfile(filepath);
} else {
return res.notFound();
}
}
I think you are on the right track about the error. You are making some rapid changes to in the assets folder. If I read your code right:
Add an image with user-generated filename to assets/images (ex cat.jpg)
Copy/resize the file to an id filename in assets/images (ex abc123.jpg)
Delete the original upload (cat.jpg)
(I don't know the details of sharp, there may be more under the hood there)
If sails is running in dev mode, then Grunt will be trying to watch the whole assets/ folder, and copy all the changes to .tmp/public/. It's easy to imagine Grunt may register a change, but when it gets around to copying the added file (assets/images/cat.jpg) it is already gone.
I have two suggestions for the solution:
One:
Like you suggested, upload your original to the .tmp folder (maybe even a custom subfolder of .tmp). Still place your sized copy into /assets/images/, and it will be copied to /.tmp/public/ where it can be accessed as an asset by the running app. But Grunt will ignore the quick add-then-delete in the .tmp folder.
Two:
Do a bit of general thinking about both what you want to include in version control, and what Grunt tasks you want to be running in production. Note that if you use sails lift --prod then Grunt watch is turned off by default, and this error would not even occur. Generally, I don't feel like we want Grunt to do too much in production, it is more of a development shortcut. Specifically, Grunt watch can use a lot of resources on a production server.
The note about version control is just that you probably want some of the contents of assets/images/ to be in version control (images used by the site, etc), but maybe not in the case of user-uploaded avatars. Make sure you have a way to differentiate these contents (subdirectories or whatever). Then they can be easily .git-ignore'd or whatever is appropriate.
Hope this helps, good luck!
I am trying to access static resource (eg. first.html) packed inside the same .jar file (testJetty.jar), which also has a class which starts the jetty (v.8) server (MainTest.java). I am unable to set the resource base correctly.
The structure of my jar file (testJetty.jar):
testJetty.jar
first.html
MainTest.java
==
Works fine on local machine, but when I wrap it in jar file and then run it, it doesn't work, giving "404: File not found" error.
I tried to set the resourcebase with the following values, all of which failed:
a) Tried setting it to .
resource_handler.setResourceBase("."); // Results in directory containing the jar file, D:\Work\eclipseworkspace\testJettyResult
b) Tried getting it from getResource
ClassLoader loader = this.getClass().getClassLoader();
File indexLoc = new File(loader.getResource("first.html").getFile());
String htmlLoc = indexLoc.getAbsolutePath();
resource_handler.setResourceBase(htmloc); // Results in D:\Work\eclipseworkspace\testJettyResult\file:\D:\Work\eclipseworkspace\testJettyResult\testJetty1.jar!\first.html
c) Tried getting the webdir
String webDir = this.getClass().getProtectionDomain()
.getCodeSource().getLocation().toExternalForm();
resource_handler.setResourceBase(webdir); // Results in D:/Work/eclipseworkspace/testJettyResult/testJetty1.jar
None of these 3 approaches worked.
Any help or alternative would be appreciated
Thanks
abbas
The solutions provided in this thread work but I think some clarity to the solution could be useful.
If you are building a fat jar and use the ProtectionDomain way you may hit some issues because you are loading the whole jar!
class.getProtectionDomain().getCodeSource().getLocation().toExternalForm();
So the better solution is the other provided solution
contextHandler.setResourceBase(
YourClass.class
.getClassLoader()
.getResource("WEB-INF")
.toExternalForm());
The problem here is if you are building a fat jar you are not really dumping your webapp resources into WEB-INF but are probably going into the root of the jar, so a simple workaround is to create a folder XXX and use the second approach as follows:
contextHandler.setResourceBase(
YourClass.class
.getClassLoader()
.getResource("XXX")
.toExternalForm());
Or change your build tool to export the webapp files into that given directory. Maybe Maven does this on a Jar for you but gradle does not.
Not unusually, I found a solution to my problem. The 3rd approach mentioned by Stephen in Embedded Jetty : how to use a .war that is included in the .jar from which Jetty starts? worked!
So, I changed from Resource_handler to WebAppContext, where WebAppContext is pointing to the same jar (testJetty.jar) and it worked!
String webDir = MainTest.class.getProtectionDomain()
.getCodeSource().getLocation().toExternalForm(); ; // Results in D:/Work/eclipseworkspace/testJettyResult/testJetty.jar
WebAppContext webappContext = new WebAppContext(webDir, "/");
It looks like ClassLoader.getResource does not understand an empty string or . or / as an argument. In my jar file I had to move all stuf to WEB-INF(any other wrapping dir will do). So the code looks like
contextHandler.setResourceBase(EmbeddedJetty.class.getClassLoader().getResource("WEB-INF").toExternalForm());
so the context looks like this then:
ContextHandler:744 - Started o.e.j.w.WebAppContext#48b3806{/,jar:file:/Users/xxx/projects/dropbox/ui/target/ui-1.0-SNAPSHOT.jar!/WEB-INF,AVAILABLE}
So I have a bunch of data that I want to load into database from CSV. I've hacked together a solution that works in local development, but when I deploy to meteor.com, it no longer works.
I'm loading the csv file in the folder /server/data/:
function readData(name){
var fs = __meteor_bootstrap__.require('fs');
var path = __meteor_bootstrap__.require('path');
var base = path.resolve('.');
var data = fs.readFileSync(path.join(base, '/server/data/', name));
return CSVToArray(data);
}
After I deploy to meteor.com, i got:
INFO Error: ENOENT, no such file or directory '/meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/public/data/categories.csv'
at Object.openSync (fs.js:240:18)
at Object.readFileSync (fs.js:128:15)
at readData (app/server/models.js:10:16)
at app/server/categories.js:6:7
at /meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/server.js:132:63
at Array.forEach (native)
at Function.<anonymous> (/meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/underscore.js:76:11)
at /meteor/containers/98eb1286-120b-ee84-8e98-ce673fa2eab7/bundle/server/server.js:132:7
Any idea how I can get meteor to see the csv file after deployment?
I realize this question is old, but it still ranks high on certain keyword searches. So, if you're using Meteor 0.6.5+, you can use the new Assets API.
The issue is that meteor only bundles files that it knows about (ie. JS/CSS/HTML/+more depending on which packages you use) up when it deploys.
Try putting the file you need in the public directory (this directory is exempt from the above rule).
Thanks to SamuelDavis and Tom Coleman's tips. I ended up figuring out what the problem is. Turns out the bundled app is no longer formated as client, public, and server. I ended up debugging it by running meteor bundle to create a tarball. extract the tarball and took a look inside to find where the data folder is. Tom was also right that the data folder needed to be in the public folder in order to get bundled in.
It appears that the base directory is not in the same location that contains the file '/server/data/xxx.csv'.
Before you try anything else, log the base path after calling "var base = path.resolve('.'). If that value is what you expected, log the files that appear in that directory. Again if the files are what you expected, navigate into the /server folder and print out those directories and so forth.
This should pinpoint you to which folder and/or directory is missing and should indicate where you should place the CSV file in future.