Facing Performance issue with JGit add command with big repository - jgit

My local repository already has more than 10,000 objects. I am seeing performance issues while using the git.add() command to add one more file to the index. Below is the JGit code snippet which I am using to interface my Java program with Git:
String absoluteLocalGitPath = "c:\\localGitRepo\\.git"
FileRepositoryBuilder repositoryBuilder = new FileRepositoryBuilder();
repositoryBuilder.setMustExist(true);
repositoryBuilder.setGitDir(new File(absoluteLocalGitPath));
repository = repositoryBuilder.build();
git = new Git(repository);
AddCommand addCommand = git.add();
addCommand.addFilepattern("folder1/obj10001.obj");
addCommand.call();
Here path passed in file pattern is the relative path to c://gitLocalRepo.

Related

Vaadin Flow 14, Jetty embedded and static files

I'm trying to create app based on Jetty 9.4.20 (embedded) and Vaadin Flow 14.0.12.
It based on very nice project vaadin14-embedded-jetty.
I want to package app with one main-jar and all dependency libs must be in folder 'libs' near main-jar.
I remove maven-assembly-plugin, instead use maven-dependency-plugin and maven-jar-plugin. In maven-dependency-plugin i add section <execution>get-dependencies</execution> where i unpack directories META-INF/resources/,META-INF/services/ from Vaadin Flow libs to the result JAR.
In this case app work fine. But if i comment section <execution>get-dependencies</execution> then result package didn't contain that directories and app didn't work.
It just cannot give some static files from Vaadin Flow libs.
This error occurs only if i launch packaged app with ...
$ java -jar vaadin14-embedded-jetty-1.0-SNAPSHOT.jar
... but from Intellij Idea it launch correctly.
There was an opinion that is Jetty staring with wrong ClassLoader and cannot maintain requests to static files in Jar-libs.
The META-INF/services/ files MUST be maintained from the Jetty libs.
That's important for Jetty to use java.util.ServiceLoader.
If you are merging contents of JAR files into a single JAR file, that's called a "uber jar".
There are many techniques to do this, but if you are using maven-assembly-plugin or maven-dependency-plugin to build this "uber jar" then you will not be merging critical files that have the same name across multiple JAR files.
Consider using maven-shade-plugin and it's associated Resource Transformers to properly merge these files.
http://maven.apache.org/plugins/maven-shade-plugin/
http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html
The ServicesResourceTransformer is the one that merges META-INF/services/ files, use it.
As for static content, that works fine, but you have to setup your Base Resource properly.
Looking at your source, you do the following ...
final URI webRootUri = ManualJetty.class.getResource("/webapp/").toURI();
final WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));
That won't work reliably in 100% of cases (as you have noticed when running in the IDE vs command line).
The Class.getResource(String) is only reliable if you lookup a file (not a directory).
Consider that the Jetty Project Embedded Cookbook recipes have techniques for this.
See:
WebAppContextFromClasspath.java
ResourceHandlerFromClasspath.java
DefaultServletFileServer.java
DefaultServletMultipleBases.java
XmlEnhancedServer.java
MultipartMimeUploadExample.java
Example:
// Figure out what path to serve content from
ClassLoader cl = ManualJetty.class.getClassLoader();
// We look for a file, as ClassLoader.getResource() is not
// designed to look for directories (we resolve the directory later)
URL f = cl.getResource("webapp/index.html");
if (f == null)
{
throw new RuntimeException("Unable to find resource directory");
}
// Resolve file to directory
URI webRootUri = f.toURI().resolve("./").normalize();
System.err.println("WebRoot is " + webRootUri);
WebAppContext context = new WebAppContext();
context.setBaseResource(Resource.newResource(webRootUri));

Remove local Git repo when working with JGit

I am building a small tool to propose some Git commands to users not familiar with Git. The commands are not intended to modify the repo, just consult some information.
I am creating the tool in Java, using JGit which seems to be the best match to do this kind of stuff.
The issue I face so far is that I create a temporary folder to host the repo content, but I am unable to delete it automatically at the end of the execution.
Here is the code (I removed the try/catch stuff to simplify the reading):
// Create temporary folder
Path folderPath = Paths.get(System.getProperty("user.dir"));
File localRepoFolder = Files.createTempDirectory(folderPath, "local-repo").toFile();
// Clone the repo
CloneCommand clone = new CloneCommand();
clone.setURI("https://myrepo");
clone.setNoCheckout(true);
clone.setDirectory(localRepoFolder);
clone.setCredentialsProvider(new UsernamePasswordCredentialsProvider("user", "password"));
Git gitRepo = clone.call();
// Do some stuff
[...]
// Cleanup before closing
gitRepo.getRepository().close();
gitRepo.close();
localRepoFolder.deleteOnExit();
I searched quite a lot on this topic, but I get everywhere that it should be automatically deleted... Am I missing something?
I would use something like Apache Commons IO (http://commons.apache.org/proper/commons-io/) which has a FileUtils.deleteDirectory

Listing files under a specific folder from remote repository with JGit

I'm trying to list files from HEAD from a remote repository (Github). I read examples from the JGit documentation, but most of the time these are referencing to a local repository.
The only piece of code I found about remote repository is:
Collection<Ref> refs = Git.lsRemoteRepository()
.setHeads(true)
.setTags(true)
.setRemote("https://github/example/example.git")
.call();
for (Ref ref : refs) {
System.out.println("Ref: " + ref);
}
But this code is just listing references, like HEAD. Could anyone help listing files from a subfolder inside my remote repository?
LsRemoteCommand is the counterpart of git ls-remote and only lists references of a remote repository. In order to list files contained in a repository, you have to first clone the repository .
For example:
Git git = Git.cloneRepository()
.setURI( "https://github.com/eclipse/jgit.git" )
.setDirectory( "/path/to/repo" )
.call();
See this link for more on cloning repositories with JGit: http://www.codeaffine.com/2015/11/30/jgit-clone-repository/

Not able to initialize tess-two (Could not initialize Tesseract API error)

I wrote a sample Android app. I am getting 'Tesseract(native): Could not initialize Tesseract API with language=eng!' error.
I did include
compile 'com.rmtheis:tess-two:5.4.0'
in the gradle file
Also copied all 'data files' 3.04.00 version to 'tessdata' directory.
I debugged Java portion of 'init' code it seems to be working fine, it's failing inside 'nativeside'.
Any suggestions what could be going wrong with my code. Here are few lines of code I am using to init
final String lang = "eng";
TessBaseAPI baseApi = new TessBaseAPI();
File externalDir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS);
String externalDirPath = externalDir.getAbsolutePath() + "/";
flag = baseApi.init(externalDirPath, lang);
The problem was not with tess-two, it was with my app, I deployed it on Marshmallow, it requires different way to get WRITE_EXTERNAL_STORAGE permission. I was not aware of it, now I fixed that issue my app is working fine.

JGit PullCommand Exception

We are using git to maintain our source. URL like git#xx.xx.xx.xx:XYZ.git. I'm using JGit to pull the changes.
UsernamePasswordCredentialsProvider user = new UsernamePasswordCredentialsProvider("xxxx", "xxxx");
localPath = "E:\\murugan\\Test\\GIT_LOCALDEPY";
Git git = new Git(localRepo);
PullCommand pcmd = git.pull();
pcmd.setCredentialsProvider(user);
pcmd.call();
I'm getting the following exception when I execute the code.
org.eclipse.jgit.errors.UnsupportedCredentialItem: ssh://git#xx.xx.xx.xx:22:
org.eclipse.jgit.transport.CredentialItem$StringType:Passphrase for C:\Users\Murugan.SOLVER\.ssh\id_rsa
If username/password security is not an issue, you can specify the credentials as part of the connection in the .git/config file of the local Git repo:
[remote "origin"]
url = ssh://<user>:<pwd>#<host>:22/<remote-path-to-repo>/
fetch = +refs/heads/*:refs/remotes/origin/*
You have to configure your SSH parameters on your machine before using Git. Here is a link, from github, for configuring it.
https://help.github.com/categories/56/articles
especially this one: https://help.github.com/articles/generating-ssh-keys
This will help you set up everything properly (you should adapt everything, since you are probably not connecting to GitHub)

Resources