I have set up a Janusgraph Cluster with Cassandra + ES. The cluster has been set up to support ConfiguredGraphFactory. Also, I am connecting the gremlin cluster remotely. I have set up a client and am able to create a graph using :
client.submit(String.format("ConfiguredGraphFactory.create(\"%s\")", graphName));
However, I am not able to get the traversalSource of the graph created using the gremlin driver. Do I have to create raw gremlin queries and traverse the graph using client.submit or is there a way to get it through the gremlin driver using Emptygraph.Instance().
To get the remote traversal reference, you need to pass in a variable name that is bound to your graph traversal. This binding is usually done as part of the "globals" in your startup script when you start the remote server (the start up script is configured to run as part of the gremlin-server.yaml).
There is currently no inherent way to dynamically bind a variable to a graph or traversal reference, but I plan on fixing this at some point.
A short term fix is to bind your graph and traversal references to a method that will be variably defined, and then create some mechanism to change the variable dynamically.
To further explain a potential solution:
Update your server's startup script to bind g to something variable:
globals << [g : DynamicBindingTool.getBoundGraphTraversal()]
Create DynamicBindingTool, which has to do two things:
A. Provide a way to setBoundGraph() which may look something like:
setBoundGraph(graphName) {
this.boundGraph = ConfiguredGraphFactory.open(graphName);
}
B. Provide a way to getBoundGraphTraversal() which may look something like:
getBoundGraphTraversal() {
this.boundGraph.traversal();
}
You can include these sorts of functions in your start-up script or perhaps even create a separate jar that you attach to your Gremlin Server.
Finally, I would like to note that the proposed example solution does not take into account a multi-node JanusGraph cluster, i.e. your notion of the current bound graph would not be shared across the JG nodes. To make this a multi-node solution, you can update the functions to define the bound graph on an external database or even piggybacked on a JanusGraph graph.
For example, something like this would be a multi-node safe implementation:
setBoundGraph(graphName) {
def managementGraph = ConfiguredGraphFactory.open("managementGraph");
managementGraph.traversal().V().has("boundGraph", true).remove();
def v = managementGraph.addVertex();
v.property("boundGraph", true);
v.property("graph.graphname", graphName);
}
and:
getBoundGraphTraversal() {
def managementGraph = ConfiguredGraphFactory.open("managementGraph");
def graphName = managementGraph.traversal().V().has("boundGraph", true).values("graph.graphname");
return ConfiguredGraphFactory.open(graphName).traversal();
}
EDIT:
Unfortunately, the above "short-term trick" will not work as the global bindings are evaluated once and stored in a Map for the duration of the sever life cycle. Please see here for more information and updates on fixes: https://issues.apache.org/jira/browse/TINKERPOP-1839.
Related
We appear to have a problem with MDriven generating the same ECO_ID for multiple objects. For the most part it seems to happen in conjunction with unexpected process shutdowns and/or server shutdowns, but it does also happen during normal activity.
Our system consists of one ASP.NET application and one WinForms application. The ASP.NET app is setup in IIS to use a single worker process. We have a mixture of WebForms and MVC, including ApiControllers. We're using a rather old version of the ECO packages: 7.0.0.10021. We're on VS 2017, target framework is 4.7.1.
We have it configured to use 64 bit integers for object id:s. Database is Firebird. SQL configuration is set to use ReadCommitted transaction isolation.
As far as I can tell we have configured EcoSpaceStrategyHandler with EcoSpaceStrategyHandler.SessionStateMode.Never, which should mean that EcoSpaces are not reused at all, right? (Why would I even use EcoSpaceStrategyHandler in this case, instead of just creating EcoSpace normally with the new keyword?)
We have created MasterController : Controller and MasterApiController : ApiController classes that we use for all our controllers. These have a EcoSpace property that simply does this:
if (ecoSpace == null)
{
if (ecoSpaceStrategyHandler == null)
ecoSpaceStrategyHandler = new EcoSpaceStrategyHandler(
EcoSpaceStrategyHandler.SessionStateMode.Never,
typeof(DiamondsEcoSpace),
null,
false
);
ecoSpace = (DiamondsEcoSpace)ecoSpaceStrategyHandler.GetEcoSpace();
}
return ecoSpace;
I.e. if no strategy handler has been created, create one specifying no pooling and no session state persisting of eco spaces. Then, if no ecospace has been fetched, fetch one from the strategy handler. Return the ecospace. Is this an acceptable approach? Why would it be better than simply doing this:
if (ecoSpace = null)
ecoSpace = new DiamondsEcoSpace();
return ecoSpace;
In aspx we have a master page that has an EcoSpaceManager. It has been configured to use a pool but SessionStateMode is Never. It has EnableViewState set to true. Is this acceptable? Does it mean that EcoSpaces will be pooled but inactivated between round trips?
It is possible that we receive multiple incoming API calls in tight succession, so that one API call hasn't been completed before the next one comes in. I assume that this means that multiple instances of MasterApiController can execute simultaneously but in separate threads. There may of course also be MasterController instances executing MVC requests and also the WinForms app may be running some batch job or other.
But as far as I understand id reservation is made at the beginning of any UpdateDatabase call, in this way:
update "ECO_ID" set "BOLD_ID" = "BOLD_ID" + :N;
select "BOLD_ID" from "ECO_ID";
If the returned value is K, this will reserve N new id:s ranging from K - N to K - 1. Using ReadCommitted transactions everywhere should ensure that the update locks the id data row, forcing any concurrent save operations to wait, then fetches the update result without interference from other transactions, then commits. At that point any other pending save operation can proceed with its own id reservation. I fail to see how this could result in the same ID being used for multiple objects.
I should note that it does seem like it sometimes produces id duplicates within one single UpdateDatabase, i.e. when saving a set of new related objects, some of them end up with the same id. I haven't really confirmed this though.
Any ideas what might be going on here? What should I look for?
The issue is most likely that you use ReadCommitted isolation.
This allows for 2 systems to simultaneously start a transaction, read the current value, increase the batch, and then save after each other.
You must use Serializable isolation for key generation; ie only read things not currently in a write operation.
MDriven use 2 settings for isolation level UpdateIsolationLevel and FetchIsolationLevel.
Set your UpdateIsolationLevel to Serializable
I have one Azure Function which is adding documents to cosmos db. One way of adding is by creating a cosmos client and then calling client.CreateDocumentAsync( ) and other is creating an Output binding to Azure Function with IAsyncCollector documents and then calling documents.AddAsync().
I would like to know what's the difference between these two and which one is more preferable.
Thanks
Two main differences:
Output binding maintains a single static instance of the client across executions. When you are calling client.CreateDocumentAsync you are in charge of maintaining the client instance, and the recommendation is to follow the singleton pattern to avoid opening multiple connections and not sharing them across executions.
Output binding actually does a UpsertDocumentAsync when AddAsync is called on the IAsyncCollector, you can check the source code: https://github.com/Azure/azure-webjobs-sdk-extensions/blob/dev/src/WebJobs.Extensions.CosmosDB/Bindings/CosmosDBAsyncCollector.cs#L28
We want to remove elements from Map bin based on size. There will be multiple threads which will try to do above operation. So writing an UDF to do this operation will make it synchronized between threads. But remove_by_rank_range is not working inside lua. Below is the error iwe are getting:
attempt to call field 'remove_by_rank_range' (a nil value)
sample lua code:
function delete(rec)
local testBinMap = rec.testBin
map.remove_by_rank_range(testBinMap, 0, 5)
end
The Lua map API does not include most of the operations of the Map data type, as implemented in the clients (for example, the Java client's MapOperation class).
The performance of the native map operations is significantly higher, so why would you use a UDF here, instead of calling remove_by_rank_range from the client?
The next thing to be aware of is that any write operation, whether it's a UDF or a client calling the map remove_by_rank_range method, first grabs a lock on the record. I answered another stackoverflow question about this request flow. Your UDF doesn't give any advantage to the problem you described over the client map operation.
If you want to cap the size of your map you should be doing it at the very same time you're adding new elements to the map. The two operations would be wrapped together with operate() - an insert, followed by the remove. I have an example of how to do this in rbotzer/aerospike-cdt-examples.
I have a node on firebase that lists all the players in the game. This list will update as and when new players join. And when the current user ( me ) disconnects, I would like to remove myself from the list.
As the list will change over time, at the moment I disconnect, I would like to update this list and update firebase.
This is the way I am thinking of doing it, but it doesn't work as .update doesnt accept a function. Only the object. But if I create the object beforehand, when .onDisconnect calls, it will not be the latest object... How should I go about doing this?
payload.onDisconnect().update( () => {
const withoutMe = state.roomObj
const index = withoutMe.players.indexOf( state.userObj.name )
if ( index > -1 ) {
withoutMe.players.splice( index, 1 )
}
return withoutMe
})
The onDisconnect handler was made for this use-case. But it requires that the data of the write operation is known at the time that you set the onDisconnect. If you think about it, this should make sense: since the onDisconnect happens after your client is disconnected, the data of the data of that write operation must be known before the disconnect.
It sounds like you're building a so-called presence system: a list that contains a node for each user that is currently online. The Firebase documentation has an example of such a presence system. The key difference from your approach is that it in the documentation each user only modifies their own node.
So: when the user comes online, they write a node for themselves. And then when they get disconnected, that node gets removed. Since all users write their node under the same parent, that parent will reflect the users that are online.
The actual implementation is a bit more involved since it deals with some edge cases too. So I recommend you check out the code in the documentation I linked, and use that as the basis for your own similar system.
I have a PlayerPref that spawn more objects in OnStartLocalPlayer().
So in OnStartLocalPlayer() it call Command(assume that called on server) that instantiate GameObject and setup some values of its scripts. At the end it calls SpanWithClientAuthority()...
The thing is that on owner client and on server those script tweeks are correct, but on all other clients it lost all that settings(ex. gameobject ref etc). What do I do wrong?
Once more in nutshell: playerPref GO must have ref list of several other objects, and those objects must have ref to that playerPref GO. (making them part of playerPref GO is not a solution).
If I understand your problem correctly, you need the references to be set across all clients who have the same game object. [Command]'s are for client to server. What you need is a [ClientRpc]. Make the OnStartLocalPlayer() call a [ClientRpc] function, in that function (ex: RpcSetRefs()) set the references you need each client to have.