All,
I have an entity, that has several collections,- each collection is mapped lazily. When I run a criteria query, I get duplicate results for my root entity in the result set. How's that possible when all my collections are mapped lazily!
I verified, my collections, load lazily.
Here's my mapping:
Root entity 'Project':
[Bag(0, Lazy = CollectionLazy.True, Inverse = true, Cascade = "all-delete-orphan")]
[Key(1, Column = "job_id")]
[OneToMany(2, ClassType = typeof(ProjectPlan))]
public virtual IList<ProjectPlan> PlanList
{
get { return _planList; }
set { _planList = value; }
}
The criteria query is:
ICriteria criteria = session.Session.CreateCriteria<Entities.Project>()
.Add(Restrictions.Eq(Entities.Project.PROP_STATUS, !Entities.Project.STATUS_DELETED_FLAG));
.CreateAlias(Entities.Project.PROP_PLANLIST, "p")
.Add(Restrictions.Eq("p.County", 'MIDDLSEX'))
.setFirstResult(start).setMaxResults(pageSize)
.List<Entities.Project>();
I know, I can correct this problem w/ Distinct result transformer, I just want to know if this is normal behavior on lazy collections.
EDIT: I found the cause of this,- when looking at the raw SQL, the join, and where clause are correct but what baffles me is the generated Select clause,- it not only contains columns from the project entity (root entity) but also columns from the project plans entity which causes the issue I described above. I am not at work right now, but I'll try to do this: .SetProjection(Projections.RootEntity()), so I only get Project's columns in the select clause.
One way, how to solve this (I'd say so usual scenario) is: 1) not use fetching collections inside of the query and 2) use batch fetching, as a part of the mapping
So, we will always be querying the root entity. That will give us a flat result set, which can be correctly used for paging.
To get the collection data for each recieved row, and to avoid 1 + N issue (goign for collection of each record) we will use 19.1.5. Using batch fetching
The mapping would be like this
[Bag(0, Lazy = CollectionLazy.True
, Inverse = true
, Cascade = "all-delete-orphan"
, BatchSize = 25)] // Or something similar to batch-size="25"
[Key(1, Column = "job_id")]
[OneToMany(2, ClassType = typeof(ProjectPlan))]
public virtual IList<ProjectPlan> PlanList
{
...
Some other similar QA (with the almost same details)
How to Eager Load Associations without duplication in NHibernate?
NHibernate QueryOver with Fetch resulting multiple sql queries and db hits
Is this the right way to eager load child collections in NHibernate
And we still can filter over the collection items! but we have to use subqueries, an example Query on HasMany reference
Related
In EFCore5, implicit tables are saved as Dictionary<TKey, object> sets, knows as Property Bag Entity Types. However, I cannot figure out how to create a LINQ query with a Where() clause that compiles for MySQL for such a property bag entity type.
For example, this code successfully retrieves the IQueryable reference to an intermediate table (generated by EFcore5's implicity many-to-many feature) given the ISkipNavigation:
ISkipNavigation nav = // some many-to-many relationship
IQueryable<Dictionary<string, object>> intermediateTable =
context.Set<Dictionary<string, object>>(nav.JoinEntityType.Name);
And this code successfully retrieves all the entries in the intermediate table:
List<Dictionary<string, object>> joins = await intermediateTable.ToListAsync();
In the resulting List, each Dictionary has just one key/value (representing the row).
However, I cannot figure out how to add a .Where() clause to the LINQ query which will compile:
joinTable.Where(d => d.Keys.First() == "foo").ToList();
joinTable.Where(d => d.Keys.Any(k => k == "foo")).ToList();
The error is:
Translation of member 'Keys' on entity type 'MachinePartMachineProfile (Dictionary<string, object>)' failed. This commonly occurs when the specified member is unmapped. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'
I do not wish to do client-side parsing for performance reasons (the join table will be to big).
I note that the type reported by the error is MachinePartMachineProfile (Dictionary<string, object>). Some investigation showed that these types are being generated based upon the static Model.DefaultPropertyBagType (which is a Dictionary<string, object>). But despite staring at the EFCore code base, I cannot discern how to correctly query such a default property bag type.
I am using MySQL as my database, if it is relevant.
You can index the dictionary directly, with knowledge of the column name.
Working example would be:
joinTable.Where(d => d[columnName] == "foo").ToList();
And for the sake of completeness, if you have an ISkipNavigation instance, you can infer these keys as follows:
string foreignKey = nav.ForeignKey.Properties.First().GetColumnBaseName();
string localKey = nav.Inverse.ForeignKey.Properties.First().GetColumnBaseName();
I am fairly new in this realm and any help is appreciated
I have a table in Dynamodb database named Tenant as below:
"TenantId" is the hash primary key and I have no other keys. And I have a field named "IsDeleted" which is boolean
Table Structure
I am trying to run a query to get the record with specified "TenantId" while it is not deleted ("IsDeleted == 0")
I can get a correct result by running the following code: (returns 0 item)
var filter = new QueryFilter("TenantId", QueryOperator.Equal, "2235ed82-41ec-42b2-bd1c-d94fba2cf9cc");
filter.AddCondition("IsDeleted", QueryOperator.Equal, 0);
var dbTenant = await
_genericRepository.FromQueryAsync(new QueryOperationConfig
{
Filter = filter
}).GetRemainingAsync();
But no luck when I try to get it with following code snippet (It returns the item which is also deleted) (returns 1 item)
var queryFilter = new List<ScanCondition>();
var scanCondition = new ScanCondition("IsDeleted", ScanOperator.Equal, new object[]{0});
queryFilter.Add(scanCondition);
var dbTenant2 = await
_genericRepository.LoadAsync("2235ed82-41ec-42b2-bd1c-d94fba2cf9cc", new DynamoDBOperationConfig
{
QueryFilter = queryFilter,
ConditionalOperator = ConditionalOperatorValues.And
});
Any Idea why ScanCondition has no effect?
Later I also tried this: (throw exception)
var dbTenant2 = await
_genericRepository.QueryAsync("2235ed82-41ec-42b2-bd1c-d94fba2cf9cc", new DynamoDBOperationConfig()
{
QueryFilter = new List<ScanCondition>()
{
new ScanCondition("IsDeleted", ScanOperator.Equal, 0)
}
}).GetRemainingAsync();
It throws with: "Message": "Must have one range key or a GSI index defined for the table Tenants"
Why does it complain about Range key or Index? I'm calling
public AsyncSearch<T> QueryAsync<T>(object hashKeyValue, DynamoDBOperationConfig operationConfig = null);
You simply cant query a table only giving a single primary key (only hash key). Because there is one and only one item for that primary key. The result of the Query would be that still that single item, which is actually Load operation not Query. You can only query if you have composite primary key in this case (Hash (TenantID) and Range Key) or GSI (which doesn't impose key uniqueness therefore accepts duplicate keys on index).
The second code attempts to filter the Load. DynamoDBOperationConfig's QueryFilter has a description ...
// Summary:
// Query filter for the Query operation operation. Evaluates the query results and
// returns only the matching values. If you specify more than one condition, then
// by default all of the conditions must evaluate to true. To match only some conditions,
// set ConditionalOperator to Or. Note: Conditions must be against non-key properties.
So works only with Query operations
Edit: So after reading your comments on this...
I dont think there conditional expressions are for read operations. AWS documents indicates they are for put or update operations. However, not being entirely sure on this since I never needed to do a conditional Load. There is no such thing like CheckIfExists functionality as well in general. You have to read the item and see if it exists. Conditional load will still consume read throughput so your only advantage would be only NOT retrieving it in other words saving the bandwith (which is very negligible for single item).
My suggestion is read it and filter it in your application layer. Dont query for it. However what you can also do is if you very need it you can use TenantId as hashkey and isDeleted for range key. If you do so, you always have to query when you wanna get a tenant. With the query you can set rangeKey(isDeleted) to 0 or 1. This isnt how I would do it. As I said, would just read it and filter it at my application.
Another suggestion thing could be setting a GSI on isDeleted field and writing null when it is 0. This way you can only see that attribute in your table when its only 1. GSI on such attribute is called sparse index. Later if you need to get all the tenants that are deleted (isDeleted=1) you can simply scan that entire index without conditions. When you are writing null when its 0 dynamoDB wont put it in the index at the first place.
Cassandra support updating specific value in Collection by syntax
ALTER TABLE users ADD todo map<timestamp, text>
UPDATE users SET todo['2012-10-2 12:00'] = 'throw my precious into mount doom'
WHERE user_id = 'frodo';
http://www.datastax.com/documentation/cql/3.0/cql/cql_using/use_map_t.html
Did not see any example of using QueryBuilder to update specific row in Map. How it can be done?
I think you have several options.
1/ Build your own query based on the CQL one.
Example: Consider that you have a table named Interactions and in your schema a column of type named 'attributes'.
String update ="UPDATE demo.Interactions SET attributes=attributes + {'i':'j'} where id='ff';
SimpleStatement statement = new SimpleStatement(update);
session.execute(statement);
2/ Use Java API.
Java API is not that documented indeed.
Let's take an example
a- Create an update object using queryBuilder
Update update = QueryBuilder.update(keyspace, tableName);
b- Then populate with 'put' or 'putAll' functions. put/putAll will add/update content
update.with(QueryBuilder.putAll(key, map));
To remove a key, set the content of a key to null, like:
for (Object item : map.keySet()) {
update.with(
QueryBuilder.put(columName, item, null)
);
}
Then execute the query.
Following methods are available for different types:
LIST:
QueryBuilder.appendAll(...)
QueryBuilder.discardAll(...)
SET:
QueryBuilder.addAll(...)
QueryBuilder.removeAll(...)
MAP:
QueryBuilder.putAll(...)
QueryBuilder.put(...)
The list is not exhautive.
Have a look in QueryBuilder class if you do not find what you are looking for.
Best regards and best luck with Cassandra.
I got the following issue. I am trying to use the With or WithMany instruction
to retrieve a linked list of roles of an business relation via an
outer join. The referential integrity is in place on the database but
the primary key on the roles table is a composite key. That's the
reason i use an OuterJoin clause because I get an exception
otherwise .
When the query gets executed the results are exactly as I expected and
nicely filled with data. Nevertheless there are certain cases where
there are not yet roles available in the database. So I would expect
that in those cases the returned SimpleList (Roles below) would be
null, cause there is not data available. Instead Simple.Data returns a
SimpleLIst and if i expand the Dynamic View in debug then it says
'Empty: No further information on this object could be discovered".
Even if i traverse further down and i retrieve the first object in the
SimpleList, it even returns a SimpleRecord with the same information
as above in debug. Only after I request a property of the SimpleRecord
I get some information that the record was empty because then it
returns null.
To come to the bottom line... is there anybody who can tell me how to
check if a SimpleList or SimpleRecord is empty or null without
traversing down the hierarchy?
I am using Simple.Data 0.16.1.0 (due to policy i can't use the
beta yet)
Thanks in advance for reading the whole story...
Below is the code sample:
dynamic businessRelationRoles;
var query = db.Zakenrelaties.As("BusinessRelations")
.All()
.OuterJoin(db.Zakenrelaties_Rollen.As("Roles"), out businessRelationRoles)
.On(zr_ID: db.Zakenrelaties.zr_ID)
.With(businessRelationRoles);
foreach (var relation in query)
{
//Get the SimpleList as IEnumerable
IEnumerable<dynamic> roles = relation.Roles;
//Get the first available SimpleRecord
var role = roles.First();
//Check if any record was returned..This passes always?? Even if the SimpleList was empty
if (role != null)
{
//Get the id of the role. returns null if SimpleRecord was empty
var roleId = role.zrro_id;
}
}
Is there anybody who can help me out?
Belatedly, and for information purposes only, this was a bug and got fixed in the 0.17 (aka 1.0-RC0) release.
In RavenDB I can store objects of type Products and Categories and they will automatically be located in different collections. This is fine.
But what if I have 2 logically completely different types of products but they use the same class? Or instead of 2 I could have a generic number of different types of products. Would it then be possible to tell Raven to split the product documents up in collections, lets say based on a string property available on the Product class?
Thankyou in advance.
EDIT:
I Have created and registered the following StoreListener that changes the collection for the documents to be stored on runtime. This results in the documents correctly being stored in different collections and thus making a nice, logically grouping of the documents.
public class DynamicCollectionDefinerStoreListener : IDocumentStoreListener
{
public bool BeforeStore(string key, object entityInstance, RavenJObject metadata)
{
var entity = entityInstance as EntityData;
if(entity == null)
throw new Exception("Cannot handle object of type " + EntityInstance.GetType());
metadata["Raven-Entity-Name"] = RavenJToken.FromObject(entity.TypeId);
return true;
}
public void AfterStore(string key, object entityInstance, RavenJObject metadata)
{
}
}
However, it seems I have to adjust my queries too in order to be able to get the objects back. My typical query of mine used to look like this:
session => session.Query<EntityData>().Where(e => e.TypeId == typeId)
With the 'typeId' being the name of the new raven collections (and the name of the entity type saved as a seperate field on the EntityData-object too).
How would I go about quering back my objects? I can't find the spot where I can define my collection at runtime prioring to executing my query.
Do I have to execute some raw lucene queries? Or can I maybe implement a query listener?
EDIT:
I found a way of storing, querying and deleting objects using dynamically defined collections, but I'm not sure this is the right way to do it:
Document store listener:
(I use the class defined above)
Method resolving index names:
private string GetIndexName(string typeId)
{
return "dynamic/" + typeId;
}
Store/Query/Delete:
// Storing
session.Store(entity);
// Query
var someResults = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e => e.EntityId == entity.EntityId)
var someMoreResults = session.Advanced.LuceneQuery<EntityData>(GetIndexName(entityTypeId)).Where("TypeId:Colors AND Range.Basic.ColorCode:Yellow)
// Deleting
var loadedEntity = session.Query<EntityData>(GetIndexName(entity.TypeId)).Where(e =>
e.EntityId == entity.EntityId).SingleOrDefault();
if (loadedEntity != null)
{
session.Delete<EntityData>(loadedEntity);
}
I have the feeling its getting a little dirty, but is this the way to store/query/delete when specifying the collection names runtime? Or do I trap myself this way?
Stephan,
You can provide the logic for deciding on the collection name using:
store.Conventions.FindTypeTagName
This is handled statically, using the generic type.
If you want to make that decision at runtime, you can provide it using a DocumentStoreListner