I see that Java 8 has significantly cleaned up reading the contents of a file into a String:
String contents = new String(Files.readAllBytes(Paths.get(new URI(someUrl))));
I am wondering if there is something similar (cleaner/less code/more concise) for copying directories recursively. In Java 7 land, it's still something like:
public void copyFolder(File src, File dest) throws IOException{
if(src.isDirectory()){
if(!dest.exists()){
dest.mkdir();
}
String files[] = src.list();
for (String file : files) {
File srcFile = new File(src, file);
File destFile = new File(dest, file);
copyFolder(srcFile,destFile);
}
} else {
InputStream in = new FileInputStream(src);
OutputStream out = new FileOutputStream(dest);
byte[] buffer = new byte[1024];
int length;
while ((length = in.read(buffer)) > 0){
out.write(buffer, 0, length);
}
in.close();
out.close();
}
}
Any improvements here in Java 8?
In this way the code looks a bit simpler
import static java.nio.file.StandardCopyOption.*;
public void copyFolder(Path src, Path dest) throws IOException {
try (Stream<Path> stream = Files.walk(src)) {
stream.forEach(source -> copy(source, dest.resolve(src.relativize(source))));
}
}
private void copy(Path source, Path dest) {
try {
Files.copy(source, dest, REPLACE_EXISTING);
} catch (Exception e) {
throw new RuntimeException(e.getMessage(), e);
}
}
Using Files.walkFileTree:
you don't need to worry about closing Streams.
(some other answers here forget that while using Files.walk)
handles IOException elegantly.
(Some other answers here would become more difficult when adding proper exception handling instead of a simple printStackTrace)
public void copyFolder(Path source, Path target, CopyOption... options)
throws IOException {
Files.walkFileTree(source, new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs)
throws IOException {
Files.createDirectories(target.resolve(source.relativize(dir).toString()));
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs)
throws IOException {
Files.copy(file, target.resolve(source.relativize(file).toString()), options);
return FileVisitResult.CONTINUE;
}
});
}
What this does is
walk recursively over all files in the directory.
When a directory is encountered (preVisitDirectory):
create the corresponding one in the target directory.
When a regular file is encountered (visitFile):
copy it.
options can be used to tailor the copy to your needs. For example to overwrite existing files in the target directory, use copyFolder(source, target, StandardCopyOption.REPLACE_EXISTING);
How about the following code
public void copyFolder(File src, File dest) throws IOException {
try (Stream<Path> stream = Files.walk(src.toPath())) {
stream.forEachOrdered(sourcePath -> {
try {
Files.copy(
/*Source Path*/
sourcePath,
/*Destination Path */
src.toPath().resolve(dest.toPath().relativize(sourcePath)));
} catch (IOException e) {
throw new UncheckedIOException(e);
}
});
}
}
This version uses Files.walk and Path parameters as Java 8 suggests.
public static void copyFolder(Path src, Path dest) {
try {
Files.walk( src ).forEach( s -> {
try {
Path d = dest.resolve( src.relativize(s) );
if( Files.isDirectory( s ) ) {
if( !Files.exists( d ) )
Files.createDirectory( d );
return;
}
Files.copy( s, d );// use flag to override existing
} catch( Exception e ) {
e.printStackTrace();
}
});
} catch( Exception ex ) {
ex.printStackTrace();
}
}
and one more version:
static void copyFolder(File src, File dest){
// checks
if(src==null || dest==null)
return;
if(!src.isDirectory())
return;
if(dest.exists()){
if(!dest.isDirectory()){
//System.out.println("destination not a folder " + dest);
return;
}
} else {
dest.mkdir();
}
if(src.listFiles()==null || src.listFiles().length==0)
return;
String strAbsPathSrc = src.getAbsolutePath();
String strAbsPathDest = dest.getAbsolutePath();
try {
Files.walkFileTree(src.toPath(), new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(Path file,
BasicFileAttributes attrs) throws IOException {
File dstFile = new File(strAbsPathDest + file.toAbsolutePath().toString().substring(strAbsPathSrc.length()));
if(dstFile.exists())
return FileVisitResult.CONTINUE;
if(!dstFile.getParentFile().exists())
dstFile.getParentFile().mkdirs();
//System.out.println(file + " " + dstFile.getAbsolutePath());
Files.copy(file, dstFile.toPath());
return FileVisitResult.CONTINUE;
}
});
} catch (IOException e) {
//e.printStackTrace();
return;
}
return;
}
its code use java8 Files.walkFileTree function.
my version:
static private void copyFolder(File src, File dest) {
// checks
if(src==null || dest==null)
return;
if(!src.isDirectory())
return;
if(dest.exists()){
if(!dest.isDirectory()){
//System.out.println("destination not a folder " + dest);
return;
}
} else {
dest.mkdir();
}
File[] files = src.listFiles();
if(files==null || files.length==0)
return;
for(File file: files){
File fileDest = new File(dest, file.getName());
//System.out.println(fileDest.getAbsolutePath());
if(file.isDirectory()){
copyFolder(file, fileDest);
}else{
if(fileDest.exists())
continue;
try {
Files.copy(file.toPath(), fileDest.toPath());
} catch (IOException e) {
//e.printStackTrace();
}
}
}
}
Can be used to copy source (file or directory) to target (directory)
void copy(Path source, Path target, boolean override) throws IOException {
Path target = target.resolve(source.toFile().getName());
Files.walkFileTree(source, new FileVisitor<Path>() {
#Override
public FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) throws IOException {
Path targetDir = target.resolve(source.relativize(dir));
if(Files.notExists(targetDir)) {
Files.createDirectory(targetDir);
}
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
Files.copy(file, target.resolve(source.relativize(file))));
return FileVisitResult.CONTINUE;
}
#Override
public FileVisitResult visitFileFailed(Path file, IOException exc) throws IOException {
throw new RuntimeException("Copying file " + file + " failed", exc);
// Consider looking at FileVisitResult options...
}
#Override
public FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
if (exc != null) {
// TODO...
}
return FileVisitResult.CONTINUE; // Or whatever works for you
}
});
}
Related
In a REST service adding a circuit breaker with hystrix, I could do the following:
#HystrixCommand(fallbackMethod = "getBackupResult")
#GetMapping(value = "/result")
public ResponseEntity<ResultDto> getResult(#RequestParam("request") String someRequest) {
ResultDto resultDto = service.parserRequest(someRequest);
return new ResponseEntity<>(resultDto, HttpStatus.OK);
}
public ResponseEntity<ResultDto> getBackupResult(#RequestParam("request") String someRequest) {
ResultDto resultDto = new ResultDto();
return new ResponseEntity<>(resultDto, HttpStatus.OK);
}
Is there something similar I can do for the gRPC call?
public void parseRequest(ParseRequest request, StreamObserver<ParseResponse> responseObserver) {
try {
ParseResponse parseResponse = service.parseRequest(request.getSomeRequest());
responseObserver.onNext(parseResponse);
responseObserver.onCompleted();
} catch (Exception e) {
logger.error("Failed to execute parse request.", e);
responseObserver.onError(new StatusException(Status.INTERNAL));
}
}
I solved my problem by implementing the circuit-breaker on my client. I used the sentinel library
To react on exceptions ratio for example I added this rule:
private static final String KEY = "callGRPC";
private void callGRPC(List<String> userAgents) {
initDegradeRule();
ManagedChannel channel = ManagedChannelBuilder.forAddress(grpcHost, grpcPort).usePlaintext()
.build();
for (String userAgent : userAgents) {
Entry entry = null;
try {
entry = SphU.entry(KEY);
UserAgentServiceGrpc.UserAgentServiceBlockingStub stub
= UserAgentServiceGrpc.newBlockingStub(channel);
UserAgentParseRequest request = UserAgentRequest.newBuilder().setUserAgent(userAgent).build();
UserAgentResponse userAgentResponse = stub.getUserAgentDetails(request);
} catch (BlockException e) {
logger.error("Circuit-breaker is on and the call has been blocked");
} catch (Throwable t) {
logger.error("Exception was thrown", t);
} finally {
if (entry != null) {
entry.exit();
}
}
}
channel.shutdown();
}
private void initDegradeRule() {
List<DegradeRule> rules = new ArrayList<DegradeRule>();
DegradeRule rule = new DegradeRule();
rule.setResource(KEY);
rule.setCount(0.5);
rule.setGrade(RuleConstant.DEGRADE_GRADE_EXCEPTION_RATIO);
rule.setTimeWindow(60);
rules.add(rule);
DegradeRuleManager.loadRules(rules);
}
We have an application which saves images in a directory in an SD card. The directory is created if it does not already exist.
Here is the code that has done the job before the arrival of API 23:
OutputStream output;
File filepath = Environment.getExternalStorageDirectory();
File dir = new File(filepath.getAbsolutePath()
+ "/fingerprint/");
dir.mkdirs();
Bitmap bitmap = m_bitmap;
File file = new File(dir, userId.getText().toString() + ".jpg");
try {
output = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, output);
output.flush();
output.close();
} catch (Exception e) {
e.printStackTrace();
}
For some reasons, the directory is no longer created and files are no longer saved in Android 6.0. We introduced these methods to update our code with API changes in Marshmallow.
private void requestPermission(final Context context){
if(ActivityCompat.shouldShowRequestPermissionRationale((Activity)context, Manifest.permission.WRITE_EXTERNAL_STORAGE)) {
new AlertDialog.Builder(context)
.setMessage(context.getResources().getString(R.string.permission_storage))
.setPositiveButton(R.string.allow, new DialogInterface.OnClickListener() {
#Override
public void onClick(DialogInterface dialog, int which) {
ActivityCompat.requestPermissions((Activity) context,
new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE},
REQUEST_WRITE_EXTERNAL_STORAGE);
}
}).show();
} else {
ActivityCompat.requestPermissions((Activity)context,
new String[]{Manifest.permission.WRITE_EXTERNAL_STORAGE},
REQUEST_WRITE_EXTERNAL_STORAGE);
}
}
#Override
public void onRequestPermissionsResult(int requestCode,String permissions[], int[] grantResults) {
switch (requestCode) {
case REQUEST_WRITE_EXTERNAL_STORAGE: {
if (grantResults.length == 1 && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
Toast.makeText(CaptureFingerprintActivity.this,
getResources().getString(R.string.permission_storage_success),
Toast.LENGTH_SHORT).show();
} else {
Toast.makeText(CaptureFingerprintActivity.this,
getResources().getString(R.string.permission_storage_failure),
Toast.LENGTH_SHORT).show();
super.onRequestPermissionsResult(requestCode, permissions, grantResults);
}
return;
}
}
}
And the code that creates the directory and saves files is also updated.
OutputStream output;
File storageDir = null;
if (Environment.MEDIA_MOUNTED.equals(Environment.getExternalStorageState())) {
if (PackageManager.PERMISSION_GRANTED == ActivityCompat.checkSelfPermission(CaptureFingerprintActivity.this, Manifest.permission.WRITE_EXTERNAL_STORAGE)) {
storageDir = new File(Environment.Environment.getExternalStorageDirectory()
, "fingerprint");
if (!storageDir.exists()) {
storageDir.mkdir();
} else {
requestPermission(CaptureFingerprintActivity.this);
}
}
Bitmap bitmap = m_bitmap;
File file = new File(storageDir, userId.getText().toString() + ".jpg");
try {
output = new FileOutputStream(file);
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, output);
output.flush();
output.close();
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
This still doesn't seem to be creating directories.
I built a web application using spring MVC, everything is working fine except the file upload in which I got random FileNotFoundExceptions. I found some solutions online like using a different tmp folder but I keep getting random error.
My code is:
#RequestMapping(value="/upload", method=RequestMethod.POST)
public #ResponseBody String handleFileUpload(#RequestParam("file") final MultipartFile multipartFile,
#RequestHeader("email") final String email, #RequestHeader("password") String password){
if (authenticateUser(email, password)) {
if (!multipartFile.isEmpty()) {
System.out.println("Start processing");
Thread thread = new Thread(){
public void run(){
ProcessCSV obj = new ProcessCSV();
try {
File file = multipartToFile(multipartFile);
if(file !=null) {
obj.extractEvents(file, email, cluster, session);
}
else {
System.out.println("null File");
}
} catch (IOException e) {
System.out.println("File conversion error");
e.printStackTrace();
}
}
};
thread.start();
return "true";
} else {
return "false";
}
}
else {
return "false";
}
}
and:
public File multipartToFile(MultipartFile multipartFile) throws IOException {
File uploadFile = null;
if(multipartFile != null && multipartFile.getSize() > 0) {
uploadFile = new File("/tmp/" + multipartFile.getOriginalFilename());
FileOutputStream fos = null;
try {
uploadFile.createNewFile();
fos = new FileOutputStream(uploadFile);
IOUtils.copy(multipartFile.getInputStream(), fos);
} catch (FileNotFoundException e) {
System.out.println("File conversion error");
e.printStackTrace();
} catch (IOException e) {
System.out.println("File conversion error");
e.printStackTrace();
} finally {
if (fos != null) {
try {
fos.close();
} catch (IOException e) {
System.out.println("File conversion error");
e.printStackTrace();
}
}
}
}
else {
System.out.println("null MultipartFile");
}
return uploadFile;
}
and the configuration file:
multipart.maxFileSize: 100MB
multipart.maxRequestSize: 100MB
multipart.location = ${user.home}
server.port = 8090
I used different versions of the multipartToFile function, one was using multipartfile.transferTo() but I was getting the same random error. Any advice?
Thank you
EDIT stack trace:
java.io.IOException: java.io.FileNotFoundException: /Users/aaa/upload_07720775_4b37_4b86_b370_40280388f3a4_00000003.tmp (No such file or directory)
at org.apache.catalina.core.ApplicationPart.write(ApplicationPart.java:121)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest$StandardMultipartFile.transferTo(StandardMultipartHttpServletRequest.java:260)
at main.RESTController.multipartToFile(RESTController.java:358)
at main.RESTController$1.run(RESTController.java:241)
Caused by: java.io.FileNotFoundException: /Users/aaa/upload_07720775_4b37_4b86_b370_40280388f3a4_00000003.tmp (No such file or directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.<init>(FileInputStream.java:138)
at org.apache.tomcat.util.http.fileupload.disk.DiskFileItem.write(DiskFileItem.java:392)
at org.apache.catalina.core.ApplicationPart.write(ApplicationPart.java:119)
... 3 more
I had just had a night of terror with this error. I found out that MultiPartFile is only recognisable to and by the #Controller class. So if you pass it to another bean which is not a controller, Spring will not be able to help you. It somewhat makes sense that the #Controller is tightly bound to the front screen (communication from the browser to the system - Controllers are the entry point from the browser). So any conversation must happen there in the Controller.
In my case, I did something like the following:
#Controller
public class FileUploadingController{
#PostMapping("/uploadHistoricData")
public String saveUploadedDataFromBrowser(#RequestParam("file") MultipartFile file) {
try {
String pathToFile = "/home/username/destination/"
new File(pathToFile).mkdir();
File newFile = new File(pathToFile + "/uploadedFile.csv");
file.transferTo(newFile); //transfer the uploaded file data to a java.io.File which can be passed between layers
dataService.processUploadedFile( newFile);
} catch (IOException e) {
//handle your exception here please
}
return "redirect:/index?successfulDataUpload";
}
}`
I had the same problem, it looks like MultipartFile is using different current dir internally, so all not absolute paths are not working.
I had to convert my path to an absolute path and then it worked.
It is working inside #RestController and in other beans too.
Path path = Paths.get(filename).toAbsolutePath();
fileToImport.transferTo(path.toFile());
fileToImport is MultipartFile.
I have a piece of code that does pull with rebase:
private void pullWithRebase(Git git) throws GitAPIException {
git.checkout().setName("master").call();
List<Ref> branches = git.branchList().setListMode(ListBranchCommand.ListMode.ALL).call();
String remoteMasterBranchName = "refs/remotes/origin/master";
for (Ref ref : branches) {
if (remoteMasterBranchName.equals(ref.getName())) {
PullResult result = git.pull().setRemoteBranchName("master").setRebase(true).call();
return;
}
}
}
However it doesn't work if any conflicts occur while merging. If they do occur, I want to accept mine
I ended up just merging two branches and directly resolving any conflicts by modifying files.
private static final String CONFLICT_HEAD_MARKER = "<<<<<<<";
private static final String CONFLICT_BORDER_MARKER = "=======";
private static final String CONFLICT_UPSTREAM_MARKER = ">>>>>>>";
private boolean acceptHead;
public void resolve(File file) throws IOException {
File temp = new File(file.getParent(), "temp" + System.currentTimeMillis());
try(BufferedReader reader = new BufferedReader(new FileReader(file));
BufferedWriter writer = new BufferedWriter(new FileWriter(temp))) {
String currentLine;
boolean removePartition = false;
while((currentLine = reader.readLine()) != null) {
if (currentLine.contains(CONFLICT_HEAD_MARKER)) {
removePartition = !acceptHead;
continue;
} else if (currentLine.contains(CONFLICT_BORDER_MARKER)) {
removePartition = acceptHead;
continue;
} else if (currentLine.contains(CONFLICT_UPSTREAM_MARKER)) {
removePartition = false;
continue;
}
if (!removePartition) {
writer.write(currentLine + System.getProperty("line.separator"));
}
}
}
FileUtils.forceDelete(file);
FileUtils.moveFile(temp, file);
}
I'm trying to access a XML file from client side in GWT. But it looks like the sendRequest method is not getting fired at all.
I'm able to see the xml in the browser. Do I need to do any thing in the server side?
Any help is appreciated.
Here's my code
String xmlurl = "http://localhost:8888/test.xml";
RequestBuilder builder = new RequestBuilder(RequestBuilder.GET, URL.encode(xmlurl));
try {
Request request = builder.sendRequest(null, new RequestCallback() {
public void onError(Request request, Throwable exception) {
System.out.println(exception);
}
public void onResponseReceived(Request request, Response response) {
if (200 == response.getStatusCode()) {
System.out.println(response.getText());
} else {
System.out.println(response.getStatusCode());
}
}
});
} catch (RequestException e) {
System.out.println("exception"+e);
}
I tried the following code too, but have the same problem. The developer tool shows response status as 200 and correct response text. Only, its not working in the code.
String xmlurl = "http://127.0.0.1:8888/test.xml";
httpGetFile(xmlurl, new AsyncCallback<String>() {
public void onFailure(Throwable caught) {
xmlData = "Error";
}
public void onSuccess(String xmlText) {
xmlData = xmlText;
}
});
public static void httpGetFile(final String url, final AsyncCallback<String> callback) {
final RequestBuilder rb = new RequestBuilder(RequestBuilder.GET, url);
rb.setCallback(new RequestCallback() {
public void onResponseReceived(Request request, Response response) {
try {
System.out.println("dafadfdf");
final int responseCode = response.getStatusCode() / 100;
if (url.startsWith("file:/") || (responseCode == 2)) {
callback.onSuccess(response.getText());
} else {
callback.onFailure(new IllegalStateException("HttpError#" + response.getStatusCode() + " - " + response.getStatusText()));
}
} catch (Throwable e) {
callback.onFailure(e);
}
}
public void onError(Request request, Throwable exception) {
callback.onFailure(exception);
}
});
try {
rb.send();
} catch (RequestException e) {
callback.onFailure(e);
}
}
Always Use logging instead of System.out.print statements https://developers.google.com/web-toolkit/doc/latest/DevGuideLogging
Step 1 - Add logging statements to failure, success and try catch statements. Clean up the exception.
Step 2 - "Parsing the XML" should be done inside the "onSuccess" method of the rb callback.
You do not need a RequestBuilder at all to access an XML file. You can use an ExternalTextResource for this:
https://developers.google.com/web-toolkit/doc/latest/DevGuideClientBundle#TextResource