I enjoy working with LiteDB in Xamarin.Forms. What is the best practice for getting list of objects from a table using list of ids or indexes in condition?
Currently, it looks for me like this:
_db.GetCollection<T>().FindAll().Where(q => listValues.Contains(Convert.ToInt32(q.GetProperty(idColumnName))));
listValues - list of searched ids.
idColumnName - column with indexes.
But FindAll retrives all records from LiteDB. Is there any more efficint options without full scanning?
Use the query in the Find method instead which will only retrieve entries based on your query rather than the full data set. This, according to the LiteDB's guidance, is more efficient than Linq queries on the full data returned: https://github.com/mbdavid/LiteDB/wiki/Queries
So your query would look like as follows
_d.GetCollection<T>().Find(q => listValues.Contains(Convert.ToInt32(q.GetProperty(idColumnName))))
I think you can just use collection.Query().Where("Column", 1,2,3) for this
Related
I've been reading a DynamoDB docs and was unable to understand if it does make sense to query on Global Secondary Index with a usage of 'contains' operator.
My problem is as follows: my dynamoDB document has a list of embedded objects, every object has a 'code' field which is unique:
{
"entities":[
{"code":"entity1Code", "name":"entity1Name"},
{"code":"entity2Code", "name":"entity2Name"}
]
}
I want to be able to get all documents that contain entities with entity.code = X.
For this purpose I'm considering adding a Global Secondary Index that would contain all entity.codes that are present in current db document separated by a comma. So the example above would look like:
{
"entities":[
{"code":"entity1Code", "name":"entity1Name"},
{"code":"entity2Code", "name":"entity2Name"}
],
"entitiesGlobalSecondaryIndex":"entityCode1,entityCode2"
}
And then I would like to apply filter expression on entitiesGlobalSecondaryIndex something like: entitiesGlobalSecondaryIndex contains entityCode1.
Would this be efficient or using global secondary index does not make sense in this way and DynamoDB will simply check the condition against every document which is similar so scan?
Any help is very appreciated,
Thanks
The contains operator of a query cannot be run on a partition Key. In order for a query to use any sort of operators (contains, begins with, > < ect...) you must have a range attributes- aka your Sort Key.
You can very well set up a GSI with some value as your PK and this code as your SK. However, GSIs are replication of the table - there is a slight potential for the data ina GSI to lag behind that of the master copy. If the query you're doing against this GSI isn't very often, then you're probably safe from that.
However. If you are trying to do this to the entire table at once then it's no better than a scan.
If what you need is a specific Code to return all its documents at once, then you could do a GSI with that as the PK. If you add a date field as the SK of this GSI it would even be time sorted. If you query against that code in that index, you'll get every single one of them.
Since you may have multiple codes, if they aren't too many per document, you maybe could use a Sparse Index - if you have an entity with code "AAAA" then you also have an attribute named AAAA (or AAAAflag or something.) It is always null/does not exist Unless the entities contains that code. If you do a GSI on this AAAflag attribute, it will only contain documents that contain that entity code, and ignore all where this attribute does not exist on a given document. This may work for you if you can also provide a good PK on this to keep the numbers well partitioned and if you don't have too many codes.
Filter expressions by the way are different than all of the above. Filter expressions are run on tbe data that would be returned, after it is already read out of the table. This is useful I'd you have a multi access pattern setup, but don't want a particular call to get all the documents associated with a particular PK - in the interests of keeping the data your code is working with concise. The query with a filter expression still retrieves everything from that query, but only presents what makes it past the filter.
If are only querying against a particular PK at any given time and you want to know if it contains any entities of x, then a Filter expressions would work perfectly. Of course, this is only per PK and not for your entire table.
If all you need is numbers, then you could do a count attribute on the document, or a meta document on that partition that contains these values and could be queried directly.
Lastly, and I have no idea if this would work or not, if your entities attribute is a map type you might very well be able to filter against entities code - and maybe even with entities.code.contains(value) if it was an SK - but I do not know if this is possible or not
Im still new with DynamoDB , How do I query something based on the previous query result?
This is how my table look like :
I want to query for the list of project info for an user.
From my first query , the result of USER#001 have [PROJECT#001,PROJECT#002].
Then I want to get a list project detail based on the first query.
How do I make an "nested" query ?? or is there anyway there I can query more efficiently ?
*The table structure is fix, I cant change it.
You can't. That's not the way DDB works...
All you could do is a BatchGetItem() with the pk & sk for the two projects.
Or if you don't happen to know the SK, you'd need to make two individual Query() calls with the pk only.
I have an SQLITE Database with the following tables
items(item_id,item_name,....);
tags(tag_id,tag_name);
items_tags(item_id,tag_id);
I'm using QSqlRelationalTableModel class to for the items table.
this class have this function QSqlTableModel::setFilter(const QString &filter);
The filter is a SQL WHERE clause without the keyword WHERE
for example: setFilter("item_name=arg");
what I want to do is to use the setFilter function to select only items with a specific tag/tags.
since I can only use the setFilter(QString &Filter) function which is basically a WHERE clause
I found this solution which I think is a bad one
I'll use this filter
setFilter(FilterString);
FilterString="item_id=arg1 or item_id=arg2 or item_id=argX or......"
the IDs will be obtained from this query
Query.prepare("select item_id from items_tags where tag_id=?");
the FilterString will be generated and incremented by a (do while loop) using Query.next() function that will loop over the resulted Query IDs
FilterString+="item=Query.value(item_id).toString();
and use like 2 if statements to determine when to add the or clause to the FilterString
this solution will definitely work but I think it's stupid because if there are many tagged items(say 5000) the Query string will contain 4999 or clause and another 5000 "id=x" so the Query will be more than 10000 characters long.
QString object can contain a lot more characters but is this Query will have a big impact on performance ?
what are the other alternatives, Even if it requires to switch to another DB like ORACLE or Mysql since SQLITE doesn't have many internal features
I am running the following query on Google BigQuery web interface, for data provided by Google Analytics:
SELECT *
FROM [dataset.table]
WHERE
hits.page.pagePath CONTAINS "my-fun-path"
I would like to save the results into a new table, however I am obtaining the following error message when using Flatten Results = False:
Error: Cannot query the cross product of repeated fields
customDimensions.value and hits.page.pagePath.
This answer implies that this should be possible: Is there a way to select nested records into a table?
Is there a workaround for the issue found?
Depending on what kind of filtering is acceptable to you, you may be able to work around this by switching to OMIT IF from WHERE. It will give different results, but, again, perhaps such different results are acceptable.
The following will remove entire hit record if (some) page inside of it meets criteria. Note two things here:
it uses OMIT hits IF, instead of more commonly used OMIT RECORD IF).
The condition is inverted, because OMIT IF is opposite of WHERE
The query is:
SELECT *
FROM [dataset.table]
OMIT hits IF EVERY(NOT hits.page.pagePath CONTAINS "my-fun-path")
Update: see the related thread, I am afraid this is no longer possible.
It would be possible to use NEST function and grouping by a field, but that's a long shot.
Using flatten call on the query:
SELECT *
FROM flatten([google.com:analytics-bigquery:LondonCycleHelmet.ga_sessions_20130910],customDimensions)
WHERE
hits.page.pagePath CONTAINS "m"
Thus in the web ui:
setting a destination table
allowing large results
and NO flatten results
does the job correctly and the produced table matches the original schema.
I know - it is old ask.
But now it can be achieved by just using standard SQL dialect instead of Legacy
#standardSQL
SELECT t.*
FROM `dataset.table` t, UNNEST(hits.page) as page
WHERE
page.pagePath CONTAINS "my-fun-path"
I'm trying to turn a method I have right now into a more "generic" method that returns a string. Right now, the method uses a statement like this:
var app = (from d in testContext.DAPPs
where d.sserID == (Guid)user.ProviderUserKey
select d).ToList();
I process the results of "app", add extra text etc. The piece that changes (that I need to make more "generic") is the table name (DAPPs). Is there a way I can do that, or, a better way to go around this all together?
You'll have to compose a Dynamic Linq Query.
There are a couple of ways to do this. Have a look at the following:
Dynamic Expressions in Linq to Sql
http://www.west-wind.com/Weblog/posts/143814.aspx
Using the LINQ Dynamic Query Library
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx