If I have a #Load Ref<Driver> in my entity Car and I make a query for Cars, when are those refs actually loaded from the datastore?
For example, in the code below are the Driver's loaded from the datastore when this query runs? Or are they loaded when I return the Cars at the end of my endpoint and it is serialized?
// The Query itself is Iterable
Query<Car> q = ofy().load().type(Car.class).filter("vin >", "123456789");
for (Car car: q) {
System.out.println(car.toString());
}
If you have the #Load annotation, Refs are loaded for each chunk of iteration. So if your chunk size is 30, then as each chunk of 30 cars is loaded their drivers will be loaded.
If you remove the #Load annotation, each Ref is loaded separately on demand.
Related
Background
Suppose there are N clients and N nodes, each node emits a message that contains statistics and it's name . A single client n ∈ N only cares about a specific node n ∈ N.
Proposed Solution
Each client listens to a specific event node_name, when the server receives a message from a node with it's node_name, emit an event node_name
Question
When an event is emitted node_name,
will the message be sent to every client n ∈ N and then be thrown away if there is no listener for that specific node_name
OR
does the server keep meta data and knows which clients/connections are listening for the event node_name and only send to the specific client n
Modifications
Can the latter be achieved through namespaces
OR
Would it be advisable to create a room for each node ? Would these rooms act as the meta data
Events are filtered at the client level, while namespace and rooms are at the server level
All connected clients within the same namespace and room are sent all events
Looking within the source for we see that flask_socketio provides a layer on top of socketio which contains some wrapper functions in it's __init__.py which calls functions from instantiated properties of it's core files.
The emit function from base_manager.py is as such
def emit(self, event, data, namespace, room=None, skip_sid=None,
callback=None, **kwargs):
"""Emit a message to a single client, a room, or all the clients
connected to the namespace."""
if namespace not in self.rooms or room not in self.rooms[namespace]:
return
for sid in self.get_participants(namespace, room):
if sid != skip_sid:
if callback is not None:
id = self._generate_ack_id(sid, namespace, callback)
else:
id = None
self.server._emit_internal(sid, event, data, namespace, id)
It uses a function called get_participants which looks as such
def get_participants(self, namespace, room):
"""Return an iterable with the active participants in a room."""
for sid, active in six.iteritems(self.rooms[namespace][room].copy()):
yield sid
Finally a look at the function enter_room reveals the structure of the object that stores client hierarchy
def enter_room(self, sid, namespace, room):
"""Add a client to a room."""
if namespace not in self.rooms:
self.rooms[namespace] = {}
if room not in self.rooms[namespace]:
self.rooms[namespace][room] = {}
self.rooms[namespace][room][sid] = True
Given the structure, it would be best to use node_name as the name of the room under a specific predefined namespace.
I have successfully integeraed meteor with angular2 but while fetching the data from collection facing difficulties in getting at one shot, here is the steps:
Collection Name : OrderDetails
No Of records : 1000
Server:
Created publication file to subcribe the collection:
Meteor.publish('orderFilter', function() {
return OrderLineDetails.find({});
});
Client:
this.dateSubscription =
MeteorObservable.subscribe('orderFilter').subscribe(()=> {
let lines = OrderDetails.find({expectedShipDate:{$in:strArr}},{fields:
{"expectedShipDate":1,"loadNo":1},sort:{"expectedShipDate":1}}).fetch();
});
In this lines attribute fetches all the collection entries, but fails to subscribe for the changes
When I try with below one,
OrderDetails.find({expectedShipDate:{$in:strArr}},{fields:{"expectedShipDate":1,"loadNo":1},sort:{"expectedShipDate":1}}).zone().subscribe(results => {
// code to loop the results
});
In this am able to subscribe for the collection changes, but the results are looped for 1000 times , as 1000 entries in the colleciton.
Is there any way to get the whole collection entries in one single shot and mean time to subscribe the changes in the collection ?.
Yes, there are a couple of ways you can do it, mostly depending on how you want to handle the data.
If having everything at once is important, then use a Method such as:
MeteorObservable.call('getAllElements', (err, result) => {
// result.length === all elements
})
While on server side doing
Meteor.methods({
getAllElements:function(){return myCollection.find().fetch()}
})
Now, if you want to listen to changes, ofcourse you'll have to do a subscription, and if you want to lower the amount of subscriptions, use rxjs' debounceTime() function, such as (from your code):
this.theData.debounceTime(400).subscribe(value => ...., err =>)
This will wait a certain amount of time before subscribing to that collection.
Now, based on your intent: listening to changes and getting everything at once, you can combine both approaches, not the most efficient but can be effective.
As #Rager explained, observables are close to streams, so when you populate data on miniMongo (front end collection you use when you find() data and is populated when you subscribe to publications) it will start incrementing until the collection is in sync.
Since miniMongo is populated when you subscribe to a publication, and not when you query a cursor, you could either:
Try the debouceTime() approach
Use a Meteor.Method after subscribing to the publication, then sync both results, keeping the first response from the method as your starting point, and then using data from Collection.find().subscribe(collectionArray => ..., err=>) to do whatterver you want to do when changes apply (not that recommended, unless you have a specific use case for this)
Also, .zone() function is specific to force re-render on Angular's event cycle. I'd recomend not use it if you're processing the collections' data instead of rendering it on a ngFor* loop. And if you're using an ngFor* loop, use the async pipe instead ngFor="let entry of Collection | async"
I don't think that's possible. When you subscribe to an Observable it handles values as a "stream", not necessarily a loop. I have seen some makeshift helper methods that handle the data synchronously, though the time it takes to subscribe is not decreased. Check out this article for an under the hood look... A simple Observable implementation
However, you can set it up to only loop once.
The way that I've been setting up that scenario, the collection only gets looped through one time (in the constructor when the app starts) and detects changes in the collection. In your case it would look like:
values: YourModel[] = []; //this is an array of models to store the data
theData: Observable<YourModel[]>;
errors: string[];
subFinished: boolean = false;
constructor(){
this.theData = OrderDetails.find({expectedShipDate:{$in:strArr}},{fields:{"expectedShipDate":1,"loadNo":1},sort:{"expectedShipDate":1}}).zone();
MeteorObservable.subscribe('orderFilter').subscribe();
//push data onto the values array
this.theData.subscribe(
value => this.values = value,
error => this.errors.push("new error"),
() => this.subFinished = true
);
}
The "values" array is updated with whatever changes happen to the database.
We've been using SCORM in our previous e-learning 'engine' but we want to change the elements our Managed Learning Environment (MLE) tracks, namely each completable component in an e-learning module.
At runtime, we run the following code to set up our SCORM connection:
var vault = {}; //vault 'namespace' helps ensure no conflicts with possible other "SCORM" variables
vault.UTILS = {}; //For holding UTILS functions
vault.debug = { isActive: true }; //Enable (true) or disable (false) for debug mode
vault.SCORM = { //Define the SCORM object
version: null, //Store SCORM version.
handleCompletionStatus: true, //Whether or not the wrapper should automatically handle the initial completion status
handleExitMode: true, //Whether or not the wrapper should automatically handle the exit mode
API:{handle: null, isFound: false}, //Create API child object
connection: { isActive: false }, //Create connection child object
data: { completionStatus: null, exitStatus: null}, //Create data child object
debug:{} //Create debug child object
};
vault.SCORM.API.find('win');
vault.SCORM.connection.initialize();
if (vault.SCORM.data.get("cmi.core.lesson_status")=="not attempted") {
vault.SCORM.data.set("cmi.core.lesson_status" , "incomplete");
vault.SCORM.data.save();
}
There are many more functions in the SCORM.js file, but the point is this all works; When the module is loaded into our MLE, the following code triggers course completion:
vault.SCORM.data.set("cmi.core.lesson_status" , "completed");
So how would we register a completable component with SCORM? (Components in our 'engine' are jQuery objects usually called 'element'). Would something like the following work, or are custom calls in SCORM not possible?
vault.SCORM.data.set("cmi.interactions.n."+element.componentId() , "incomplete");
But then if I registered an interaction by specifying an id, as follows...
vault.SCORM.data.set("cmi.interactions.n.id", element.componentId());
...how do I then set or access 'completion' on that component?
I've been reading posts and pdf specs from various sites, but the explanations are sparse at best.
I know there aren't a lot of SCORM followers here, but if you have any info, I'd be keen to hear it.
FWIW, that's my pipwerks SCORM wrapper, but with the variable pipwerks changed to ncalt.
There is documentation on how to use my wrapper at http://pipwerks.com (search for "scorm wrapper" in the search field). The original source code can be found at https://github.com/pipwerks/scorm-api-wrapper.
Note your sample code is not using the wrapper the way it was intended to be used. For example, this:
ncalt.SCORM.data.set("cmi.core.lesson_status" , "completed");
should be this (data is an internal helper and not necessary):
ncalt.SCORM.set("cmi.core.lesson_status" , "completed");
You can shorten it even further via a reference variable, like so:
var scorm = ncalt.SCORM;
scorm.set("cmi.core.lesson_status" , "completed");
scorm.save();
scorm.get("cmi.core.lesson_status"); //returns "completed"
As for your 'components', if you'd like to use SCORM's cmi.interactions model, be sure you're using the correct syntax. The "n" in the SCORM documentation (cmi.interactions.n.id) is meant to represent a number, it's not a literal "n".
scorm.set("cmi.interactions.0.id", "myfirstinteraction");
scorm.save();
To retrieve data from that interaction, you need to specify the number in place of the n:
scorm.get("cmi.interactions.0.id"); //returns "myfirstinteraction"
Note the CMI data model doesn't provide a 'status' field for cmi.interactions. You'd need to use cmi.objectives.
scorm.set("cmi.objectives.0.status", "completed");
scorm.save();
scorm.get("cmi.objectives.0.status"); // returns "completed"
The CMI data model (as available in SCORM) is spelled out here: http://scorm.com/scorm-explained/technical-scorm/run-time/run-time-reference/
All,
I have an entity, that has several collections,- each collection is mapped lazily. When I run a criteria query, I get duplicate results for my root entity in the result set. How's that possible when all my collections are mapped lazily!
I verified, my collections, load lazily.
Here's my mapping:
Root entity 'Project':
[Bag(0, Lazy = CollectionLazy.True, Inverse = true, Cascade = "all-delete-orphan")]
[Key(1, Column = "job_id")]
[OneToMany(2, ClassType = typeof(ProjectPlan))]
public virtual IList<ProjectPlan> PlanList
{
get { return _planList; }
set { _planList = value; }
}
The criteria query is:
ICriteria criteria = session.Session.CreateCriteria<Entities.Project>()
.Add(Restrictions.Eq(Entities.Project.PROP_STATUS, !Entities.Project.STATUS_DELETED_FLAG));
.CreateAlias(Entities.Project.PROP_PLANLIST, "p")
.Add(Restrictions.Eq("p.County", 'MIDDLSEX'))
.setFirstResult(start).setMaxResults(pageSize)
.List<Entities.Project>();
I know, I can correct this problem w/ Distinct result transformer, I just want to know if this is normal behavior on lazy collections.
EDIT: I found the cause of this,- when looking at the raw SQL, the join, and where clause are correct but what baffles me is the generated Select clause,- it not only contains columns from the project entity (root entity) but also columns from the project plans entity which causes the issue I described above. I am not at work right now, but I'll try to do this: .SetProjection(Projections.RootEntity()), so I only get Project's columns in the select clause.
One way, how to solve this (I'd say so usual scenario) is: 1) not use fetching collections inside of the query and 2) use batch fetching, as a part of the mapping
So, we will always be querying the root entity. That will give us a flat result set, which can be correctly used for paging.
To get the collection data for each recieved row, and to avoid 1 + N issue (goign for collection of each record) we will use 19.1.5. Using batch fetching
The mapping would be like this
[Bag(0, Lazy = CollectionLazy.True
, Inverse = true
, Cascade = "all-delete-orphan"
, BatchSize = 25)] // Or something similar to batch-size="25"
[Key(1, Column = "job_id")]
[OneToMany(2, ClassType = typeof(ProjectPlan))]
public virtual IList<ProjectPlan> PlanList
{
...
Some other similar QA (with the almost same details)
How to Eager Load Associations without duplication in NHibernate?
NHibernate QueryOver with Fetch resulting multiple sql queries and db hits
Is this the right way to eager load child collections in NHibernate
And we still can filter over the collection items! but we have to use subqueries, an example Query on HasMany reference
I've read some sample code (especially from the Couchbase Model Views demo project link) and realized the map() function is so strange.
function(doc, meta) {
if (doc.type == "beer" && doc.name){
emit(doc.name, null);
}
}
Why the emit function's value is null, but the result from GetView("beers", "beer") is getting the value perfectly???
Please help me out !
In couchbase usally view's result set is built in background. If you have for example 1 million documents each 4Kb size without any views it take ~4Gb on disk. When you create a view with map function like
function(doc, meta) {
emit(doc.name, doc);
}
As a result it take additional 4Gb on disk for view results because view results are stored separately. And in most cases (if you query view with param Stale=Ok) couchbase returns result from that "precompiled" set of records, couchbase doesn't scan all docs on each query.
So emmiting null in map functions is used to prevent disk space usage and it also increase speed of indexing process.
Now the second question about couchbase magic when "result from GetView("beers", "beer") is getting the value perfectly". Couchbase get(key) and getMulti(keys) operations are very fast. So when you query view that emits null it returns not only nulls, also it returns document ids. Then you can manually use getMulti for that array of document ids to get doc's value or in some SDKs there is query param called IncludeDocs that will do the same automatically.