I have an index created like this:
ancestor: NONE
indexId: CICAgJiUpoMK
kind: Candidate
projectId: financialplanning-270210
properties:
- direction: ASCENDING
name: activity.lastupdatets
- direction: DESCENDING
name: id
state: READY
but when running the following query:
select activity.lastupdatets from candidate order by activity.lastupdatets asc, __key__ desc
It fails with an error like "...no suitable composite index found..."
Related
I have tried creating GSI with a PK that uses a composite value of business_id, type_id, partner_id fields. I did it in two different ways in the AWS console:
First: business_id#type_id#partner_id
Second: [business_id]#[type_id]#[partner_id]
and sort key: updated
Here is the query:
SELECT *
FROM "items"."composite_key-index"
WHERE business_id = 435634652 AND type_id = 2 AND partner_id = 69992528
ORDER BY updated ASC
In both cases it throws this error:
ValidationException: Must have at least one non-optional hash key
condition in WHERE clause when using ORDER BY clause.
And if I run it without the order by:
SELECT *
FROM "items"."composite_key-index"
WHERE business_id = 435634652 AND type_id = 2 AND partner_id = 69992528
it doesnt return any items, even though there are data matching those values.
What am I doing wrong here?
To use a composite value as a key, you have to build the values yourself.
Your application would have to store the value in a single attribute, ex GSI_PK, as 435634652#2#6992528
Then your query would look like
SELECT *
FROM "items"."composite_key-index"
WHERE GSI_PK = "435634652#2#6992528"
Hello I m new to DynamoDB, I have created a TABLE, with Partition Key "pk" and Sort Key "id"
in then item explorer I can query with the pk and sort key value and it seems to work.
In the PartiQL Editor I do
SELECT * FROM "dev" WHERE "pk" = 'config' AND "id" = "7b733512cc98445891dcb07dc4299ace"
and I get the error Filter Expression can only contain non-primary key attributes: Primary key attribute: id
I don't know how I can specify the sort key in the key conditions instead of the filter condition with the WHERE clause.
I found the error, if you use " instead of ' it doesn't work. so the correct query is :
SELECT * FROM "dev" WHERE "pk" = 'config' AND "id" = '7b733512cc98445891dcb07dc4299ace'
I recently moved to mariadb 10.5, and have encountered this specific output where a percentage is shown along with rows in explain output. I couldn't find any documentation for the same, probably it's a new feature.
What exactly does that mean? Is it the probability of some kind regarding rows being read?
MariaDB [c6b2c772b91fd3d8]> explain
select
`execute_action`, `state_type`
from
`tabSuperflow Document State`
where
`parent` = 'Check Point'
and `state` = 'Pending TSM Approval - Delivery'
order by
modified desc \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: tabSuperflow Document State
type: ref|filter
possible_keys: parent,index_on_state
key: index_on_state|parent
key_len: 563|563
ref: const
rows: 1 (17%)
Extra: Using index condition; Using where; Using filesort; Using rowid filter
1 row in set (0.001 sec)
Found out the answer in a rather unrelated documentation
https://mariadb.com/kb/en/rowid-filtering-optimization/
rows column shows the expected filter selectivity, it is 5%.
So basically that percentage shows expected filter selectivity, i.e. rows which will be filtered using where clause in this step. This output can also be seen in explain extended output in the filtered column.
MariaDB [c6b2c772b91fd3d8]> explain extended select `execute_action`, `state_type` from `tabSuperflow Document State` where `parent` = 'Check Point' and `state` = 'Pending TSM Approval - Delivery' order by modified desc \G
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: tabSuperflow Document State
type: ref|filter
possible_keys: parent,index_on_state
key: index_on_state|parent
key_len: 563|563
ref: const
rows: 1 (17%)
filtered: 16.67
Extra: Using index condition; Using where; Using filesort; Using rowid filter
1 row in set, 1 warning (0.001 sec)
While designing key-only queries to filter Google Datastore entities, I am generating many composite indexes that are subsets of another index. Is it possible to use the same composite index for queries that filter on a subset of the properties already indexed? For example, if I have the following key-only queries, would it be possible to have less than three indexes?
Query 1: Entities where a = 1, b = 1, c = 1;
Query 2: Entities where a = 1, b = 1;
Query 3: Entities where a = 1;
Here is a sample of the actual query I am working with:
Query<Key> query = Query.newKeyQueryBuilder()
.setKind("track")
.setFilter(CompositeFilter.and(PropertyFilter.eq("status", 1), PropertyFilter.eq("bpm", 138), PropertyFilter.eq("artist", "AVB"), PropertyFilter.eq("label", "Armada")))
.setOrderBy(OrderBy.asc("date"))
.build();
Datastore can merge smaller indexes together to support larger equality queries, see index merging. Using this feature, a minimal set of indexes for your set of queries would be something like:
index.yaml
indexes:
- kind: Albums
properties:
- name: artist
- name: date
- kind: Albums
properties:
- name: bpm
- name: date
- kind: Albums
properties:
- name: label
- name: date
- kind: Albums
properties:
- name: status
- name: date
This supports equality queries on any number of these properties, sorted by date. Note, however, that index merging has a performance trade-off in some cases.
I'm familiar with MySQL and am starting to use Amazon DynamoDB for a new project.
Assume I have a MySQL table like this:
CREATE TABLE foo (
id CHAR(64) NOT NULL,
scheduledDelivery DATETIME NOT NULL,
-- ...other columns...
PRIMARY KEY(id),
INDEX schedIndex (scheduledDelivery)
);
Note the secondary Index schedIndex which is supposed to speed-up the following query (which is executed periodically):
SELECT *
FROM foo
WHERE scheduledDelivery <= NOW()
ORDER BY scheduledDelivery ASC
LIMIT 100;
That is: Take the 100 oldest items that are due to be delivered.
With DynamoDB I can use the id column as primary partition key.
However, I don't understand how I can avoid full-table scans in DynamoDB. When adding a secondary index I must always specify a "partition key". However, (in MySQL words) I see these problems:
the scheduledDelivery column is not unique, so it can't be used as a partition key itself AFAIK
adding id as unique partition key and using scheduledDelivery as "sort key" sounds like a (id, scheduledDelivery) secondary index to me, which makes that index pratically useless
I understand that MySQL and DynamoDB require different approaches, so what would be a appropriate solution in this case?
It's not possible to avoid a full table scan with this kind of query.
However, you may be able to disguise it as a Query operation, which would allow you to sort the results (not possible with a Scan).
You must first create a GSI. Let's name it scheduled_delivery-index.
We will specify our index's partition key to be an attribute named fixed_val, and our sort key to be scheduled_delivery.
fixed_val will contain any value you want, but it must always be that value, and you must know it from the client side. For the sake of this example, let's say that fixed_val will always be 1.
GSI keys do not have to be unique, so don't worry if there are two duplicated scheduled_delivery values.
You would query the table like this:
var now = Date.now();
//...
{
TableName: "foo",
IndexName: "scheduled_delivery-index",
ExpressionAttributeNames: {
"#f": "fixed_value",
"#d": "scheduled_delivery"
},
ExpressionAttributeValues: {
":f": 1,
":d": now
},
KeyConditionExpression: "#f = :f and #d <= :d",
ScanIndexForward: true
}