I want to store following directory tree structure using neo4j local database and Gremlin in Java.
(ROOT)
/ \
Dir2 Dir3
/ \ \
Dir4 Dir5 Dir6
/
Dir7
I have defined a method StorePath(String path).
What i want : When i call StorePath(path) with path = "Root\Dir2\Dir4\Dir7" then data should be stored as follows
Root
/
Dir2
/
Dir4
/
Dir7
where Root and Dir* are vertices with blank edges.
Please help me with the java code.
private static final RelationshipType SUB_DIR = DynamicRelationshipType.withName("SUB_DIR");
public void storePath(String path) {
Node dir = graphDb.getReferenceNode();
for (String name : path.split(File.separator)) {
dir = obtainSubDir(dir, name);
}
}
private Node obtainSubDir(Node dir, String name) {
Node subDir = getSubDir(dir,name);
if (subDir!=null) return subDir;
return createSubDir(dir, name);
}
private Node getSubDir(Node dir, String name) {
for (Relationship rel : dir.getRelationships(SUB_DIR, Direction.OUTGOING)) {
final Node subDir = rel.getEndNode();
if (subDir.getProperty("name", "").equals(name)) return subDir;
}
return null;
}
private Node createSubDir(Node dir, String name) {
Node subDir = dir.getGraphDatabase().createNode();
subDir.setProperty("name", name);
dir.createRelationshipTo(subDir, SUB_DIR);
return subDir;
}
Related
I'm trying to read the path /var/cache/apt/archives with the following permissions:
drwxr-xr-x 3 root root 90112 ago 2 14:36 archives
And I got the following error:
ERROR: Error opening directory '/var/cache/apt/archives/partial': Permission denied
Can somebody give me a hand with this?
The source code is the following:
using Gtk;
using GLib;
private int64[] get_folder_data (File file, string space = "", Cancellable? cancellable = null) throws Error
{
FileEnumerator enumerator = file.enumerate_children (
"standard::*",
FileQueryInfoFlags.NOFOLLOW_SYMLINKS,
cancellable);
int64 files = 0;
int64 size = 0;
int64[] data = new int64[2];
FileInfo info = null;
while (cancellable.is_cancelled () == false && ((info = enumerator.next_file (cancellable)) != null)) {
if (info.get_file_type () == FileType.DIRECTORY) {
File subdir = file.resolve_relative_path (info.get_name ());
get_folder_data (subdir, space + " ", cancellable);
} else {
files += 1;//Sum Files
size += info.get_size ();//Accumulates Size
}
}
if (cancellable.is_cancelled ()) {
throw new IOError.CANCELLED ("Operation was cancelled");
}
data[0] = files;
data[1] = size;
stdout.printf ("APT CACHE SIZE: %s\n", files.to_string());
stdout.printf ("APT CACHE FILES: %s\n", size.to_string());
return data;
}
public static int main (string[] args) {
Gtk.init (ref args);
File APT_CACHE_PATH = File.new_for_path ("/var/cache/apt/archives");
try {
get_folder_data (APT_CACHE_PATH, "", new Cancellable ());
} catch (Error e) {
stdout.printf ("ERROR: %s\n", e.message);
}
Gtk.main ();
return 0;
}
And the command I used for compile is the following:
valac --pkg gtk+-3.0 --pkg glib-2.0 --pkg gio-2.0 apt-cache.vala
If you run your app as a normal user, you have to exclude the "partial" dir, it has more restrictive permissions (0700):
drwx------ 2 _apt root 4096 Jul 29 11:36 /var/cache/apt/archives/partial
One way to exclude the partial dir is to just ignore any dir that is inaccessible:
int64[] data = new int64[2];
FileEnumerator enumerator = null;
try {
enumerator = file.enumerate_children (
"standard::*",
FileQueryInfoFlags.NOFOLLOW_SYMLINKS,
cancellable);
}
catch (IOError e) {
stderr.printf ("WARNING: Unable to get size of dir '%s': %s\n", file.get_path (), e.message);
data[0] = 0;
data[1] = 0;
return data;
}
In addition it might be a good idea to always explicitly ignore the partial folder.
If you are planning to make your utility useful for the root user as well, you might even think of adding a command line option like "--include-partial-dir".
Also the same thing can be done with simple bash commands which is much easier than writing your own program.
du -sh /var/cache/apt/archives
find /var/cache/apt/archives -type f | wc -l
Note that du and find also warn about the inaccessible partial dir:
$ du -sh /var/cache/apt/archives
du: cannot read directory '/var/cache/apt/archives/partial': Permission denied
4.6G /var/cache/apt/archives
$ find /var/cache/apt/archives -type f | wc -l
find: '/var/cache/apt/archives/partial': Permission denied
3732
I wrote a Spock test to learn how to use JGit. The general idea of the test follows these steps:
Create a "TestRepo" directory
Initialize a new Git repository there ("TestRepo/.git")
Create a new File in the parent directory (TestRepo) and set its text to something to take up space
Call "git status"
(debug) Groovy dump the returned Status object
Assert that the returned Status object has the file listed as untracked.
When I run the below test, it fails. Why?
state.dump() prints
Status#38989dff
diff=org.eclipse.jgit.lib.IndexDiff#72def3cd
clean=true
hasUncommittedChanges=false
Code below:
class GitActionsSpec extends Specification {
public static final ROOT_DIR_PATH = Paths.get(System.getProperty("user.home"), "TestRepo")
public static final ROOT_DIR_STRING = ROOT_DIR_PATH.toString()
public static final GIT_DIR_PATH = ROOT_DIR_PATH.resolve(".git")
#Shared
Git git
/**
* Creates a repository in rootDirPath
*/
def setupSpec() {
if (Files.exists(ROOT_DIR_PATH)) {
deleteDirectory(ROOT_DIR_PATH)
}
Files.createDirectory(ROOT_DIR_PATH)
/*
GitActions.createRepoIn(File parentDirectory) {
return Git.init().setDirectory(f).call()
}
*/
git = GitActions.createRepoIn(ROOT_DIR_PATH.toFile())
assert git.repository.getDirectory().exists()
}
// The actual test
def "A newly-created file should be listed as 'untracked'"() {
given: "A new file"
Path file = ROOT_DIR_PATH.relativize(ROOT_DIR_PATH.resolve("file.txt"))
file.text = "filler text"
assert Files.exists(file)
when: "user requests the status"
Status state = git.status().addPath(file.toString()).call()
then: "Git lists that file as untracked"
println state.dump()
!state.getUntracked().isEmpty()
}
def cleanupSpec() {
git.close()
deleteDirectory(ROOT_DIR_PATH)
}
def deleteDirectory(Path directory) {
Files.walkFileTree(directory, new SimpleFileVisitor<Path>() {
#Override
FileVisitResult postVisitDirectory(Path dir, IOException exc) throws IOException {
Files.delete(dir)
return FileVisitResult.CONTINUE
}
#Override
FileVisitResult visitFile(Path file, BasicFileAttributes attrs) throws IOException {
Files.delete(file)
return FileVisitResult.CONTINUE
}
})
}
}
Turns out the issue lied in the code that sets up file.
File's toString() returns A, not B:
A: /home/user/Project/Module/file.txt
B: /home/user/TestRepo/file.txt
I am building a VSIX package to support a custom language in Visual Studio using MPF. I am in a custom designer and I need to find the files referenced in the project to resolve some dependencies. Where can I access this list?
I assume, that you´re using MPF to implement the project system for your custom language service. When doing so, you probably have a project root node which is derived from either ProjectNode or HierarchyNode...
If so, you could share the root node´s instance with the designer and try to find files by traversing the hierarchy, for instance...
internal class HierarchyVisitor
{
private readonly Func<HierarchyNode, bool> filterCallback;
public HierarchyVisitor(
Func<HierarchyNode, bool> filter)
{
this.filterCallback = filter;
}
public IEnumerable<HierarchyNode> Visit(
HierarchyNode node)
{
var stack = new Stack<HierarchyNode>();
stack.Push(node);
while (stack.Any())
{
HierarchyNode next = stack.Pop();
if (this.filterCallback(next))
{
yield return next;
}
for (
HierarchyNode child = next.FirstChild;
child != null;
child = child.NextSibling)
{
stack.Push(child);
}
}
}
}
To get a list of all nodes in the hierarchy, you could just do...
ProjectNode root = ...
var visitor = new HierarchyVisitor(x => true);
IEnumerable<HierarchyNode> flatList = visitor.Visit(root);
Or to filter for a certain file type, you could try something like this...
ProjectNode root = ...
var visitor = new HierarchyVisitor((HierarchyNode x) =>
{
const string XmlFileExtension = ".xml";
string path = new Uri(x.Url, UriKind.Absolut).LocalPath;
return string.Compare(
XmlFileExtension,
Path.GetFileExtension(path),
StringComparison.InvariantCultureIgnoreCase) == 0;
});
IEnumerable<HierarchyNode> xmlFiles = visitor.Visit(root);
I've a dir (with sub dirs) template that is kept as a resource inside a jar file. During run
time I need to extract it (template) to tmp dir change some content and finally publish it as a zipped artifact.
My question is: how to extract this content easily? I was trying getResource() as well as getResourceAsStream()..
Following code works fine here: (Java7)
String s = this.getClass().getResource("").getPath();
if (s.contains("jar!")) {
// we have a jar file
// format: file:/location...jar!...path-in-the-jar
// we only want to have location :)
int excl = s.lastIndexOf("!");
s = s.substring(0, excl);
s = s.substring("file:/".length());
Path workingDirPath = workingDir = Files.createTempDirectory("demo")
try (JarFile jf = new JarFile(s);){
Enumeration<JarEntry> entries = jf.entries();
while (entries.hasMoreElements()) {
JarEntry je = entries.nextElement();
String name = je.getName();
if (je.isDirectory()) {
// directory found
Path dir = workingDirPath.resolve(name);
Files.createDirectory(dir);
} else {
Path file = workingDirPath.resolve(name);
try (InputStream is = jf.getInputStream(je);) {
Files.copy(is, file, StandardCopyOption.REPLACE_EXISTING);
}
}
}
}
} else {
// debug mode: no jar
}
I have following directory structure,
Dir1
|___Dir2
|___Dir3
|___Dir4
|___File1.gz
|___File2.gz
|___File3.gz
The subdirectories are just nested and donot contain any files
I am trying to use the following for recursing through a directory on HDFS.If its a directory I append /* to the path and addInputPath
arg[0] = "path/to/Dir1"; // given at command line
FileStatus fs = new FileStatus();
Path q = new Path(args[0]);
FileInputFormat.addInputPath(job,q);
Path p = new Path(q.toString()+"/*");
fs.setPath(p);
while(fs.isDirectory())
{
fs.setPath(new Path(p.toString()+"/*"));
FileInputFormat.addInputPath(job,fs.getPath());
}
But the code doesnt seem to go in the while loop and I get not a File Exception
Where is the if statement you are referring to?
Anyway, you may have a look at these utility methods which add all files within a directory to a job's input:
Utils:
public static Path[] getRecursivePaths(FileSystem fs, String basePath)
throws IOException, URISyntaxException {
List<Path> result = new ArrayList<Path>();
basePath = fs.getUri() + basePath;
FileStatus[] listStatus = fs.globStatus(new Path(basePath+"/*"));
for (FileStatus fstat : listStatus) {
readSubDirectory(fstat, basePath, fs, result);
}
return (Path[]) result.toArray(new Path[result.size()]);
}
private static void readSubDirectory(FileStatus fileStatus, String basePath,
FileSystem fs, List<Path> paths) throws IOException, URISyntaxException {
if (!fileStatus.isDir()) {
paths.add(fileStatus.getPath());
}
else {
String subPath = fileStatus.getPath().toString();
FileStatus[] listStatus = fs.globStatus(new Path(subPath + "/*"));
if (listStatus.length == 0) {
paths.add(fileStatus.getPath());
}
for (FileStatus fst : listStatus) {
readSubDirectory(fst, subPath, fs, paths);
}
}
}
Use it in your job runner class:
...
Path[] inputPaths = Utils.getRecursivePaths(fs, inputPath);
FileInputFormat.setInputPaths(job, inputPaths);
...