Recently I explored some libraries where I can visualize graphs from Neo4j, but didn't get a clear view wich one to choose, so I have a couple thought I'd like to share.
My requirements and wishes - after search throught REST API with cypher query visualize graphs with all relationships, nodes. And result make interactive - able to drag/drop... opportunity to reveal other nodes/rels on click. Import nodes throught json data and be able to show quickly up to at least 400 elements.
So I more or less explored further libraries:
Three.js - mostly for 3D. (If I don't need 3D then better not to choose this one)
Arbor.js - drawing leaves up to you, mostly layout library
VivaGraphJS - easy to use, but almost no documentation :(
D3.js - Looks fine, but uses SVG
Cytoscape.js - looks fine, but I read that I can't use it with Neo4j. Is D3.js the right choice for real-time visualization of Neo4j Graph DB data why?
Processing.js - have it's own language similar to Java. I'm not so much in Java-like language (yet).
And I have a couple questions:
1) Whats your experience - benefits, weakneses of libraries?
2) I suppose I want to use canvas insted of SVG? (a lot of small elements on the screen).
3) Arbor is special library for layout, but others don't have layout algorithms or what?
I'll appreciate any opinion! :-)
I pondered a similar situation. I chose sigma.js.
1) Whats your experience - benefits, weakneses of libraries?
sigma.js is specific to graph rendering. d3 is a full on toolbox for visualization. Since I only want to draw graphs, the concise sigma.js interface was easier to work with. I think this is the video where the creator talks about how sigma.js is able to do incremental rendering and thus perform better and handle more vertices/edges smoothly.
2) I suppose I want to use canvas insted of SVG? (a lot of small
elements on the screen).
sigma.js renders to the canvas.
Here are the sigma.js examples. The "hide nodes" example shows mouse interactions. Additionaly, here is a blog post showing how one can easily integrate neo4j w/ sigma using a very minimal amount of data massaging code written in node.js. It would be easy to port this to any server language.
I don't see sigma.js in that list which you might want to take a look at as it is very useful (example)
You will have to somehow handle the rendering of the received data (i.e. go from a data structure to its visual representation) and this depends on what are you trying to show / say with that visualisation, how you want to do it and how you want it to look like.
As a starting point, i would go with D3 and Sigma.
I note that Gephi is not on your list. As a standalone visualiser/editor, its the best that I have used (you just get the neo4j plugin, and it will import the source files of the database and let you view it). Its also Open Source, so you can include it as part of a project.
Also missing from your list is the yFiles family of graph visualization libraries.
The JavaScript variant of if uses both Canvas, WebGL, and SVG. This can give very good performance and high-quality visualizations at the same time. For hair-ball-like graphs (thousands of dots and lines, but probably no text) you can use WebGL and Canvas and when you zoom in to actually make sense of the data, you can use SVG to get detailed high-quality visualizations with labels, icons, data-bound visualizations, etc. You can use both techniques in the same visualization, even at the same time, benefiting from the best of both worlds (demo).
Connecting yFiles to Neo4j is easy using the JavaScript Bolt driver.
Aiming at 400 elements in the view is a good idea. While larger graphs tend to look really nice, most of the time you won't gain a lot of insights by looking at "hair-balls". These almost artistic visualizations all look very similar and as such carry little to no information. Often you could just as well display a static image showing the first google image search hit for a "hairball graph" query :-)
In order to better support the user during exploration, filtering, grouping, nesting, and dynamic interaction should be added to the viewer application. How this can be implemented, very much depends on the business domain in the visualization, though and there are only very few generic approaches that work for generic graphs.
Disclaimer: I work for the company that creates the above mentioned visualization library. I do not represent it on SO, though.
Related
This is not a joke. and I want a tutorial.
I want to make a 3D (node editor / programming language / graph editor) like this picture:
How can I do that?
These are just some thoughts on the matter, I think some planning will be required.
I'm unaware of any tutorials explaining how to build node editors, but since so many node editors exist, perhaps your best source for inspiration can be other people's codebases.
I think you can look at prior art:
https://architecture.mit.edu/computation/project/visual-programming-three-dimensions-visual-representations-computational-mental
Also, I think you can plan it like this:
figure out what it will look like (maybe you can make a mockup on blender first)
think about what kind of interactions will be allowed, how will the mouse work?
Consider using an existing 3d framework like openframeworks or unity
design a command palette or list of keyboard shortcuts of things that will be required
study existing 2d note editors, consider evaluating the work of John Paul Morrison (Flow Based Programming) and things like NoFlo or PyFlow among many others, you can study the ones that have actual users like the Unreal Node Editor, or more recently I think Unity also has one.
design an execution engine (how will the code actually run, input ports, output ports, etc).
consider the format that you'll use for saving/loading the networks, is it sqlite? json? will you need to store the coordinates and perhaps the dimensions of the nodes, and the connections between the nodes?
if a new node is to be located, where will it be put? do you need to indicate an arbitrary point in 3d space, or is it better to have the new node positions to be relative to existing objects?
component property editors may be important, each node will probably have it's own set of properties, will each node be custom, with a custom ui for editing? or will there be a general property editor for all the nodes? these decisions should be made.
custom node properties can be used to make the ui easier for the user, however
the global property editor will allow you to create many components relatively inexpensively.
what operations need to be supported? do you need auto-layouting algorithms, like graphviz's dot?
will the runtime allow for step by step execution? node statistics? real time visualization?
consider who is your target market for this tool, and why this tool will be useful outside the existing tools, is it music artists using the next vvvvv ? or is it graphic artists making the next shader? will be useful for web development? or for a specific niche?
I hope you'll find these thoughts and ideas useful in your process, and I look forward to the things you will build.
I'm looking for some tools to draw a directed graph with relatively small number of nodes (up to 50). As it is meant to be shown to non-technical people - I need my graph to be descriptive and PRETTY. Additionally I need to use some custom images (word-clouds) as nodes - can anyone give an example of a tool that would work for me ?
Take a look at Graphviz, where using custom images for nodes is certainly possible. Prettyness is also feasible, at least in my opinion.
We currently have a dynamically updated network graph with around 1,500 nodes and 2,000 edges. It's ever-growing. Our current layout engine uses Prefuse - the force directed layout in particular - and it takes about 10 minutes with a hefty server to get a nice, stable layout.
I've looked a little GraphViz's sfpd algorithm, but haven't tested it yet...
Are there faster alternatives I should look at?
I don't care about the visual appearance of the nodes and edges - we process that separately - just putting x, y on the nodes.
We do need to be able to tinker with the layout properties for specific parts of the graph, for instance, applying special tighter or looser springs for certain nodes.
Thanks in advance, and please comment if you need more specific information to answer!
EDIT: I'm particularly looking for speed comparisons between the layout engine options. Benchmarks, specific examples, or just personal experience would suffice!
I wrote a JavaScript-based graph drawing library VivaGraph.js.
It calculates layout and renders graph with 2K+ vertices, 8.5K edges in ~10-15 seconds. If you don't need rendering part it should be even faster.
Here is a video demonstrating it in action: WebGL Graph Rendering With VivaGraphJS.
Online demo is available here. WebGL is required to view the demo but is not needed to calculate graphs layouts. The library also works under node.js, thus could be used as a service.
Example of API usage (layout only):
var graph = Viva.Graph.graph(),
layout = Viva.Graph.Layout.forceDirected(graph);
graph.addLink(1, 2);
layout.run(50); // runs 50 iterations of graph layout
// print results:
graph.forEachNode(function(node) { console.log(node.position); })
Hope this helps :)
I would have a look at OGDF, specifically http://www.ogdf.net/doku.php/tech:howto:frcl
I have not used OGDF, but I do know that Fast Multipole Multilevel is a good performant algorithm and when you're dealing with the types of runtimes involved with force directed layout with the number of nodes you want, that matters a lot.
Why, among other reasons, that algorithm is awesome: Fast Multipole method. The fast multipole method is a matrix multiplication approximation which reduces the O() runtime of matrix multiplication for approximation to a small degree. Ideally, you'd have code from something like this: http://mgarland.org/files/papers/layoutgpu.pdf but I can't find it anywhere; maybe a CUDA solution isn't up your alley anyways.
Good luck.
The Gephi Toolkit might be what you need: some layouts are very fast yet with a good quality: http://gephi.org/toolkit/
30 secondes to 2 minutes are enough to layout such a graph, depending on your machine.
You can use the ForAtlas layout, or the Yifan Hu Multilevel layout.
For very large graphs (+50K nodes and 500K links), the OpenOrd layout wil
In a commercial scenario, you might also want to look at the family of yFiles graph layout and visualization libraries.
Even the JavaScript version of it can perform layouts for thousands of nodes and edges using different arrangement styles. The "organic" layout style is an implementation of a force directed layout algorithm similar in nature to the one used in Neo4j's browser application. But there are a lot more layout algorithms available that can give better visualizations for certain types of graph structures and diagrams. Depending on the settings and structure of the problem, some of the algorithms take only seconds, while more complex implementations can also bring your JavaScript engine to its knees. The Java and .net based variants still perform quite a bit better, as of today, but the JavaScript engines are catching up.
You can play with these algorithms and settings in this online demo.
Disclaimer: I work for yWorks, which is the maker of these libraries, but I do not represent my employer on SO.
I would take a look at http://neo4j.org/ its open source which is beneficial in your case so you can customize it to your needs. The github account can be found here.
UPDATE: I had posted this on UI.stackexchange also for views on different kinds od visualisation. I am posting this here for finding out the programming techniques and tools required to do so.
Let us have the following three sets of information
Now I want to combine all of this data and show it all together. Telling it like a story. Giving inter-relations. Showing similarities in terms, concepts etc. to get the following (Note that in the diagram below, the colored relations may not be exact, they are merely indicative of a node of information)
Situation: I need to tell somebody the relation between two or more important things through the commonness of concepts, keywords, behaviours in those things.
One way that I figured out would be to use circles for concepts.
So that all concepts connected to thing A would be connected to it and all concept related to B would be connected to it. And the common concepts would be connected to both. That way 2 things can be easily compared.
Problem: To build such a graph/visualisation manually would be cumbersome. Especially to add, arrange, update and manipulate.
Question: Is there a good way to do it. Also, Is there a tool available for doing this?
I hope this make the question much more clear. :)
Where does this data (the concepts, keywords, and relations between them etc...) come from? If it's in a database somewhere you could write soem code to generate a graphiz file then open it in a graphiz visualizer. There might be some tools out there that allow interactive editing of a graphiz graph, it looks like WebDot may and there are probably others.
How to display the hierarchical data on User Interface
You're talking about Venn diagrams. I think there should be plenty of online and offline tools that can help making these.
graphiz has been mentioned already, although that would be used more to show a flow of a system, or a treeview.
When you're talking about software development and want to display a design through diagrams, a complete diagram solution already exist as UML. And there are plenty of UMT tools that can help here. A commercial version is Altova UModel, which has some very nice features. You could probably use Use Cases as the most logical diagram type.
Also see Wikipedia for more info about use case diagrams. Reconsidering the image you've added, I do tend to consider it to be a usecase. Since UML is based on XML, it should be possible to transform your data through a stylesheet to UML, then use a random UML tool to display the diagrams.To convert your data to XML, well... If it's in Excel then exporting it to XML should not be too difficult.
Why is your sample image an Use Case? Well, you have actors (Pinguin, Koala, Tulips) and you have actions. (well, kind of actions: Cause for concern, some kind of animal, linked to movie, bites your nose off...) And finally, there are associations between the actors and the actions connecting them all in some way. Thus Data--(export)->XML--(Styleheet)->UML--(UML tool)->Diagram.
D3: Data-Driven Documents JS library
In working on a shortest path algorithm across a network I would like to generate a picture of the network. I'd like to represent nodes (circles), links (lines), cost to traverse the link (number in the middle of the link line), and capacity of the link (number on the link line next to the node it represents) in the picture. Is there any library/software out there that would help to automate creating this picture?
I can do this manually in Visio or with some drawing application but I'd like to generate them from code as I change/tweak the network.
Sounds like a job for GraphViz , it generates graphs from a short text description file. I've used it to produce connected node graphs and I believe it should be possible to add link labels, as you require.
If you're using python, Nodebox draws pretty graphs.
One of the big problems in displaying networks like this is figuring out where to put the nodes on the display screen. If arranging nodes is logically simple given your network, then an off-the-shelf product is likely to suit your needs.
If the arrangements are much more complicated, you may have to accept a certain amount of manual intervention to get this to work with off-the-shelf stuff, or byte the bullet and program the whole thing yourself.
.NET is one choice, and once you've mastered the Graphics class it's easy to use and plenty fast for something like this. However, there are probably better languages/platforms than .NET for something graphics-oriented like this.
Update: .NET is much better for 2D graphics than I knew. The key is finding a fast workaround to the pitifully slow GetPixel() and SetPixel() methods in the Bitmap class. Once you can read and write individual pixels easily and quickly, you can do whatever you want as a programmer.
Did you by chance check out the R programming language? I'm not positive but I believe that you can make images and such out of graphs. r-project.org
There are a bunch of visualizations of various algorithms here: Algorithmics Animation Workshop