I've a network graph
Now I've some connected nodes and as you can see most of the nodes only have one connected node that is their degree is 1. Now I'd like to remove such nodes to clear the clutter. Unable to find how to since last 2 days. No such helper functions available in visjs documentation. Would appreciate help.
I believe the algorithm suggested by the 1st answer -by macramole- (before updates) would actually hide the non-connected nodes (degree 0), instead of the ones with degree 1.
I would probably just iterate over all the edges in the network while keeping 'degree' counters for each node that is an endpoint in the edge you are visiting (you can obtain these nodes by grabbing the edge.from and edge.to values, as shown above). You would increment the degree counter for a node, whenever the node is 'hit' in this search through the edges.
Eventually you'll end up with the degree value for each node in the network, at which point you can decide which ones to hide.
Updating this answer now to include my suggested code (note: nodes and edges are vis DataSet instances):
Example code:
var nodeToDegrees = {}; // keeps a map of node ids to degrees
var nodeFrom, nodeTo;
for (edge in edges) {
nodeFrom = edge.from;
nodeTo = edge.to;
nodeToDegrees[nodeFrom] = nodeToDegrees[nodeFrom] ? nodeToDegrees[nodeFrom] + 1 : 0;
nodeToDegrees[nodeTo] = nodeToDegrees[nodeTo] ? nodeToDegrees[nodeTo] + 1 : 0;
}
for (node in nodes) {
if (nodeToDegrees[node.id] = 1) nodes.update([{node.id, hidden: true}]);
}
This might work:
var DEGREES_HIDDEN = 1;
for ( var node of nodes ) {
node.cantLinks = 0;
for ( var link of links ) {
if ( link.from == node.id || link.to == node.id ) {
node.cantLinks++;
}
}
}
for ( var node of nodes ) {
if ( node.cantLinks <= DEGREES_HIDDEN ) {
node.hidden = true;
}
}
Nodes and links are arrays not vis.DataSet, I create the latter after doing that.
Doesn't look very nice perfomance wise but it does get the job done. Hope you find it useful.
Related
I need to calculate the average of the values of points that fall in a polygon grid.
Is like a join one to many based in condition Polyogon.contains(point)
//In:
val pointValueRdd : RDD[Feature[Point,Double]]
val squareGridRdd : RDD[Feature[Polygon]]
//Needed Out:
val squareGridRdd : RDD[Feature[Polygon,(accum:Double,count:Int)]]
//or
val squareGridRdd : RDD[Feature[Polygon,average:Double]]
is possible to use some quadtree index ?
I read:clip to grid but i don not if it is the right tool.
http://geotrellis.readthedocs.io/en/latest/guide/vectors.html#cliptogrid
The next image show the grid in blue, and the points
Some advice we welcome
There is a straightforward way to accomplish this if your polygon grid can be expressed as a LayoutDefinition in GeoTrellis. A LayoutDefinition defines the grid of tiles that are used by GeoTrellis layers to represent large collection of raster tiles. It can also be used to perform transformations between a grid key (SpatialKey) in the grid space and bounding boxes (Extents) of in the map space.
I won't assume that you can represent the grid by a LayoutDefinition and instead will show an example that solve the more general case. If you can represent your polygon grid by a LayoutDefinition, that approach will be more straightforward. However here is a code snippet of the more general approach. This was compiled but not tested, so if you find problems with it let me know. This is being included in our examples in the doc-examples project with this PR: https://github.com/locationtech/geotrellis/pull/2489
import geotrellis.raster._
import geotrellis.spark._
import geotrellis.spark.tiling._
import geotrellis.vector._
import org.apache.spark.HashPartitioner
import org.apache.spark.rdd.RDD
import java.util.UUID
// see https://stackoverflow.com/questions/47359243/geotrellis-get-the-points-that-fall-in-a-polygon-grid-rdd
val pointValueRdd : RDD[Feature[Point,Double]] = ???
val squareGridRdd : RDD[Polygon] = ???
// Since we'll be grouping by key and then joining, it's work defining a partitioner
// up front.
val partitioner = new HashPartitioner(100)
// First, we'll determine the bounds of the Polygon grid
// and the average height and width, to create a GridExtent
val (extent, totalHeight, totalWidth, totalCount) =
squareGridRdd
.map { poly =>
val e = poly.envelope
(e, e.height, e.width, 1)
}
.reduce { case ((extent1, height1, width1, count1), (extent2, height2, width2, count2)) =>
(extent1.combine(extent2), height1 + height2, width1 + width2, count1 + count2)
}
val gridExtent = GridExtent(extent, totalHeight / totalCount, totalWidth / totalCount)
// We then use this for to construct a LayoutDefinition, that represents "tiles"
// that are 1x1.
val layoutDefinition = LayoutDefinition(gridExtent, tileCols = 1, tileRows = 1)
// Now that we have a layout, we can cut the polygons up per SpatialKey, as well as
// assign points to a SpatialKey, using ClipToGrid
// In order to keep track of which polygons go where, we'll assign a UUID to each
// polygon, so that they can be reconstructed. If we were dealing with PolygonFeatures,
// we could store the feature data as well. If those features already had IDs, we could
// also just use those IDs instead of UUIDs.
// We'll also store the original polygon, as they are not too big and it makes
// the reconstruction process (which might be prone to floating point errors) a little
// easier. For more complex polygons this might not be the most performant strategy.
// We then group by key to produce a set of polygons that intersect each key.
val cutPolygons: RDD[(SpatialKey, Iterable[Feature[Geometry, (Polygon, UUID)]])] =
squareGridRdd
.map { poly => Feature(poly, (poly, UUID.randomUUID)) }
.clipToGrid(layoutDefinition)
.groupByKey(partitioner)
// While we could also use clipToGrid for the points, we can
// simply use the mapTransform on the layout to determien what SpatialKey each
// point should be assigned to.
// We also group this by key to produce the set of points that intersect the key.
val pointsToKeys: RDD[(SpatialKey, Iterable[PointFeature[Double]])] =
pointValueRdd
.map { pointFeature =>
(layoutDefinition.mapTransform.pointToKey(pointFeature.geom), pointFeature)
}
.groupByKey(partitioner)
// Now we can join those two RDDs and perform our point in polygon tests.
// Use a left outer join so that polygons with no points can be recorded.
// Once we have the point information, we key the RDD by the UUID and
// reduce the results.
val result: RDD[Feature[Polygon, Double]] =
cutPolygons
.leftOuterJoin(pointsToKeys)
.flatMap { case (_, (polyFeatures, pointsOption)) =>
pointsOption match {
case Some(points) =>
for(
Feature(geom, (poly, uuid)) <- polyFeatures;
Feature(point, value) <- points if geom.intersects(point)
) yield {
(uuid, Feature(poly, (value, 1)))
}
case None =>
for(Feature(geom, (poly, uuid)) <- polyFeatures) yield {
(uuid, Feature(poly, (0.0, 0)))
}
}
}
.reduceByKey { case (Feature(poly1, (accum1, count1)), Feature(poly2, (accum2, count2))) =>
Feature(poly1, (accum1 + accum2, count1 + count2))
}
.map { case (_, feature) =>
// We no longer need the UUID; also compute the mean
feature.mapData { case (acc, c) => acc / c }
}
☼ Hello !
I want to get the critical path of an activity list, but through Neo4j.
For this, I need the Earliest Times (Start and Finish). The Earliest Start of an activity equals the greatest Earliest Finish of its predecessors, and so on.
I already had something "working". But my problem is that I just need to "recall the function". I can go down by hand, but I can't do it indefinitely...
The Activity List
Here is my code :
// LEVEL 1
/****** collect (start.successors) as startSucessors *****/
MATCH (project:Project)-[:CONTAINS]->(:Activity{tag:'Start'})-[s:ENABLES]->(:Activity)
WHERE ID(project)=toInteger(322)
WITH collect(endNode(s)) AS startSuccessors
/**** foreach node in startSucessors ****/
UNWIND startSuccessors AS node
/**** collect (node.predecessors) as nodePredecessors ****/
MATCH (activity:Activity)-[p:ENABLES]->(node)
WITH collect(startNode(p)) AS nodePredecessors, node, startSuccessors
/**** foreach activity in nodePredecessors ****/
UNWIND nodePredecessors AS activity
/**** IF (node.ES is null OR node.ES < activity.EF) ****/
WITH node, activity, startSuccessors,(node.ES = 0) AS cond1, (node.ES < activity.EF) AS cond2
MERGE (activity)-[:ENABLES]->(node)
ON MATCH SET node.ES =
CASE
/**if**/ WHEN cond1 OR cond2
/**node.ES = activity.EF**/ THEN activity.EF
END
ON MATCH SET node.EF = node.ES + node.ET
// LEVEL 2
/**T.O.D.O. : loop for each node in startSuccessors and their nodes **/
WITH startSuccessors
UNWIND startSuccessors AS node
MERGE (node)-[s2:ENABLES]->(successor:Activity)
WITH collect(successor) AS nodeSuccessors,node
UNWIND nodeSuccessors AS successor
CREATE UNIQUE (act:Activity)-[p2:ENABLES]->(successor)
WITH successor, node,act, (successor.ES = 0) AS cond3, (successor.ES < act.EF) AS cond4
MERGE (act)-[p2:ENABLES]->(successor)
ON MATCH SET successor.ES =
CASE
/**if**/ WHEN cond3 OR cond4
/**node.ES = activity.EF**/ THEN act.EF
END
ON MATCH SET successor.EF = successor.ES + successor.ET
Here is the result
Earliest Times Query Result
The second problem is that if I rerun the query, the ES and EF properties disappear ... (prove below)
Problem when rerunning the query
To repair this, I have to run this query :
MATCH (p:Project) WHERE ID(p)=322
MATCH (p)-[:CONTAINS]->(one:Activity{tag:'one'}),(p)-[:CONTAINS]->(zrht:Activity{tag:'zrht'}),(p)-[:CONTAINS]->(ore:Activity{tag:'ore'}),(p)-[:CONTAINS]->(bam:Activity{tag:'bam'}),(p)-[:CONTAINS]->(two:Activity{tag:'two'})
SET one.EF = 0,one.ES = 0,one.LF=0,one.LS=0,zrht.EF = 0,zrht.ES = 0,zrht.LF=0,zrht.LS=0,ore.EF = 0,ore.ES = 0,ore.LF=0,ore.LS=0,bam.EF = 0,bam.ES = 0,bam.LF=0,bam.LS=0,two.EF = 0,two.ES = 0,two.LF=0,two.LS=0
This javascript code reaches what I want to do.
Thank you very much for your help.
☼ I finally found what I was looking for : Project Management with Neo4j
In hopes it will help other to find in a quicker way ;)
I've a graph in OrientDB with vertexes Area & Place with edges visited. Your average path goes Area > visited > Place > visited > Place > visited > Place > visited > Place and so on. It tracks which places user visited after the previous one. visited contains YYYYmmDD datestamp.
I'm trying to find out all Area vertexes based on arbitrary Place vertexes for certain day - i.e. I want to know from which areas users came to a certain place after visiting certain place first.
Traversing from any single Place in the path would be easy but I need to to follow the path for only for a specific datestamp. What I did was that I created index for datestamp to get day's visited edges quickly and then finds the one that has in to the first Place. However now I can't figure out how to create a fast query that finds all Area vertexes based on the first Place while also making sure that the path contains second Place as well. I can get path between first and second Place via shortestPath() but I still have the same problem with extending the path to include Area vertexes.
I found some theory on the subject but if somebody could point me to the right direction how to use OrientDB to do this instead of pure graph theory I would really appreciate it - I've been working on this for the past week now. Originally this was done via bruteforce by traversing everything and then selecting but as the database grows it's not obviously sustainable.
I created the three Vertices 'Area', 'Place' and 'User' and the two Edges 'visited' and 'placed' where datestamp is a property on the edge 'visited.
In this way you don't have to insert everytime the User as a property on the edge.
Edit
Try this JavaScript function that has three parameters(places,date,propertyPlace)
var g=orient.getGraph();
var myPlaces=places.substring(1,places.length-1).split(",");
var b=g.command("sql","select from Area");
var result=[];
if(checkPlaces){
for(i=0;i<b.length;i++){
var listPlaces=[];
for(ind=0;ind<myPlaces.length;ind++){
listPlaces.push(myPlaces[ind]);
}
if(search(b[i],listPlaces)){
result.push(b[i]);
}
}
}
return result;
function checkPlaces() {
for(index=0;index<myPlaces.length;index++){
var place=g.command("sql","select from Place where "+ propertyPlace + "='"+myPlaces[index]+"'");
if(place.length==0){
return false;
}
}
return true;
}
function checkDate(edge){
var datestamp=edge.getRecord().field("datestamp");
var year=datestamp.getYear()+1900;
var month=datestamp.getMonth()+1;
var day=datestamp.getDate();
var app=date.split("-");
var myYear=parseInt(app[0]);
var myMonth=parseInt(app[1]);
var myDay=parseInt(app[2]);
if(year==myYear && month==myMonth && day==myDay){
return true;
}
return false;
}
function search(v,places){
var edge=v.getRecord().field("out_visited");
if(edge!=null){
var edgeIterator=edge.iterator();
while(edgeIterator.hasNext()){
var edge = edgeIterator.next();
if (checkDate(edge)) {
var v1 = edge.field("in");
if(v1!=null){
var name = v1.field(propertyPlace);
for(j=0;j<places.length;j++){
if(name==(places[j])) {
places.splice(j, 1);
break;
}
}
if(places.length==0){
return true;
}
else if(search(v1,places)){
return true;
}
}
}
}
}
return false;
}
Using the following command
select expand(result) from (select myFunction("[place1,place2]","2015-12-03","name") as result)
Let me know if it works
This is not a solution to this exact problem but instead a workaround I came up with. Inspired by Orientdb get last vertex from each path when Traversing by edge property
I changed to structure so that Area is now created per visitation instead of being static and it includes yyyymmdd timestamp also. Now I can use Area to start the query and use visited edges to get Place vertices only for a certain date.
Here's the query:
SELECT $path, $depth FROM (
TRAVERSE * FROM (
SELECT outE('visited') FROM (
SELECT EXPAND(rid) FROM INDEX:area.dt WHERE key = 20151205
)
) WHILE (#class = 'visited' AND dt = 20151205) OR #class = 'place')
WHERE #class = 'place' AND NOT (outE() contains (dt=20151205))
This returns correct paths with vertices and edges so you can verify it's only for a certain day. However note that Area is not contained in the path and I still need to figure out how to do that but if you want, you can just traverse the first visited edge backwards in the path and get it that way.
I have coded a table driven LR(1) parser and it is working very well however I am having a bit of a disconnect on the stage of turing a parse into a syntax tree/abstract syntax tree. This is a project that I m very passionate about but I have really just hit a dead end here. Thank you for your help in advance.
Edit: Also my parser just uses a 2d array and an action object that tells it where to go next or if its a reduction where to go and how many items to pop. I noticed that many people use the visitor pattern. Im not sure how they know what type of node to make.
Here is the pushdown automata for context
while (lexer.hasNext() || parseStack.size() > 0) {
Action topOfStack = parseStack.peek();
token = parseStack.size() > 0 ? lexer.nextToken() : new Token(TokenType.EOF, "EOF");
topOfStack.setToken(token);
int row = topOfStack.getTransitionIndex();
int column = getTerminalIndex(token.getLexeme());
column = token.getType() == TokenType.IDENTIFIER
&& !terminalsContain(token.getLexeme()) ? 0 : column;
Action action = actionTable[row][column];
if (action instanceof Accept) {
System.out.println("valid parse!!!!!!");
} else if (action instanceof Reduction) {
Reduction reduction = (Reduction) action;
popStack(parseStack, reduction.getNumberOfItemsToPop());
column = reduction.getTransitionIndex();
row = parseStack.peek().getTransitionIndex();
parseStack.push(new Action(gotoTable[row][column]));
lexer.backupTokenStream();
} else if (action != null) {
parseStack.push(actionTable[row][column]);
} else {
System.out.println("Parse error");
System.out.println("On token: " + token.getLexeme());
break;
}
Each reduction in the LR parsing process corresponds to an internal node in the parse tree. The rule being reduced is the internal AST node, and the items popped off the stack correspond to the children of that internal node. The item pushed for the goto corresponds to the internal node, while those pushed by shift actions correspond to leaves (tokens) of the AST.
Putting all that together, you can easily build an AST by createing a new internal node every time you do a reduction and wiring everything together appropriately.
Ok I am trying to make a dynamic pathing system so the player can move from point A to point B without having predefined paths. Note this game is all text based no graphics. The player can move in 10 directions: up, down, n, e, s, w, sw, se, nw and ne.
The map of the entire world is in a database, each row of the database contains a room or a node, each room/node has directions it's capable of going. The room may not be sequential persay. An Example:
Map Number, Room Number, Direction_N, Direction_S, Direction_E, Direction_W, etc.
1 1 1/3 1/100 1/1381 1/101
The Direction_N indicates it goes to Map 1 Room 3, Direction_S Map 1 Room 100, etc...
Ok, I reworked the code with suggestions (thank you guys by the way!) here is revised code. It seems to find the rooms now, even vast distances! But now the issue is finding the shortest path to the destination, I tried traversing the collection but the path is not coming out right...
In the image link below, I have start point in red square in center and stop point at red square at upper left. This returns visitedStartRooms = 103 and visitedStopRooms = 86, when it's only about 16 rooms.
Quess my missing piece of the puzzle is I am not sure how to sort out the rooms in those collection to gain the true shortest route.
Example of map
Here is the new code
public void findRoute(ROOM_INFO startRoom, ROOM_INFO destinationRoom)
{
Dictionary<ROOM_INFO, bool> visitedStartRooms = new Dictionary<ROOM_INFO, bool>();
Dictionary<ROOM_INFO, bool> visitedStopRooms = new Dictionary<ROOM_INFO, bool>();
List<string> directions = new List<string>();
startQueue.Enqueue(startRoom); // Queue up the initial room
destinationQueue.Enqueue(destinationRoom);
visitedStartRooms.Add(startRoom, true);// say we have been there, done that
visitedStopRooms.Add(destinationRoom, true);
string direction = "";
bool foundRoom = false;
while (startQueue.Count != 0 || destinationQueue.Count != 0)
{
ROOM_INFO currentStartRoom = startQueue.Dequeue(); // remove room from queue to check out.
ROOM_INFO currentDestinationRoom = destinationQueue.Dequeue();
ROOM_INFO startNextRoom = new ROOM_INFO();
ROOM_INFO stopNextRoom = new ROOM_INFO();
if (currentStartRoom.Equals(destinationRoom))
{
break;
}
else
{
// Start from destination and work to Start Point.
foreach (string exit in currentDestinationRoom.exitData)
{
stopNextRoom = extractMapRoom(exit); // get adjacent room
if (stopNextRoom.Equals(startRoom))
{
visitedStopRooms.Add(stopNextRoom, true);
foundRoom = true;
break;
}
if (stopNextRoom.mapNumber != 0 && stopNextRoom.roomNumber != 0)
{
if (!visitedStopRooms.ContainsKey(stopNextRoom))
{
if (visitedStartRooms.ContainsKey(stopNextRoom))
{
foundRoom = true;
}
else
{
destinationQueue.Enqueue(stopNextRoom);
visitedStopRooms.Add(stopNextRoom, true);
}
}
}
}
if (foundRoom)
{
break;
}
}
// start from the start and work way to destination point
foreach (string exit in currentStartRoom.exitData)
{
startNextRoom = extractMapRoom(exit); // get adjacent room
if (startNextRoom.Equals(destinationRoom))
{
visitedStartRooms.Add(startNextRoom, true);
foundRoom = true;
break;
}
if (startNextRoom.mapNumber != 0 && startNextRoom.roomNumber != 0)
{
if (!visitedStartRooms.ContainsKey(startNextRoom))
{
if (visitedStopRooms.ContainsKey(startNextRoom))
{
foundRoom = true;
break;
}
else
{
startQueue.Enqueue(startNextRoom);
visitedStartRooms.Add(startNextRoom, true);
}
}
}
}
if (foundRoom)
{
break;
}
}
}
You have a good start. There are a few basic improvements that will help. First, to be able to reconstruct your path, you should create a new data structure to store visited rooms. But for each entry, you want to store the room, plus the previous room in the path back to the starting point. A good data structure for this would be a dictionary where the key is the room identifier, and the value is the previous room identifier. To know if you've visited a room, you look to see if it exists in that data structure, not your openList queue. With this new structure, you can properly check if you've visited a room, and you can reconstruct the path back by repeatedly looking up the previous room in the same structure until you get to the origination.
The second improvement will increase the performance quite a bit. Instead of just doing a breadth-first search from the start point until you bump into the destination, as you currently do, instead create matching data structures like you have for the start room search, but have them be for the destination room. After you've looked one room away from the start, look one room away from the destination. Repeat this...two rooms away from start, then two rooms away from destination.. etc., working your way out, until you discover a room that has been visited by both your search from start and your search from destination. Build the path from this room back to the start, and back to the destination, and that will be your shortest path.
The problem you are trying to solve is the shortest path problem with unweighted edges or where the weights of all edges are equal. The weight of an edge is the time/cost to move from one room to another. If the cost to move from one room to another varies depending on which pair of rooms you're talking about, then the problem is more complicated, and the algorithm you started with and for which I've suggested modifications, will not work as it is. Here are some links about it:
Shortest path (fewest nodes) for unweighted graph
http://en.wikipedia.org/wiki/Shortest_path_problem
You may also be interested in the A* algorithm which uses a different approach. It uses a hueristic approach to concentrate the search in a subset of the solution space more likely to contain the shortest path. http://en.wikipedia.org/wiki/A%2a_search_algorithm
But A* is probably overkill in your case since the weight of all edges is the same between all rooms.