According to the redux docs: "While there is less need to store the response in a normalized lookup table with RTK Query managing caching data, transformResponse can be leveraged to do so if desired."
Why there is "less need" with RTK Query managing caching data?
What are the trade-offs of normalizing state with RTK Query?
We cover the answers to both those questions in more detail in both the RTK Query docs, and the "Redux Essentials" tutorial:
https://redux-toolkit.js.org/rtk-query/usage/cache-behavior#no-normalized-or-de-duplicated-cache
https://redux.js.org/tutorials/essentials/part-8-rtk-query-advanced#normalized-vs-document-caches
Summarizing for here:
One of the main reasons to normalize data is to make it easy to update a specific item by its ID. With RTK Query, you're generally not "updating items" in the state any more. Instead, the server is the source of truth. In that scenario, you use an RTK Query mutation to send an update to the server, invalidate any corresponding queries, and re-fetch all of that data from the server. So, different use case, different usage pattern.
Related
I have a flat table with around 30 attributes in DynamoDB. I would like to expose an API for my end users/applications to query on a random combination of those attributes.
This is trivial to do in a typical RDBMS.
How can we do this in DynamoDB? What kind of modelling techniques and/or Key condition expressions can we use to achieve this.
Multi-faceted search like you describe can be challenging in DynamoDB. It can certainly be done, but you may be fighting the tool depending on your specific access patterns. Search in DynamoDB is supported through query (fast and cheap) and scan (slower and expensive) operations. You may want to take some time to read the docs to understand how each works, and why it's critical to structure your data to support your access patterns.
One options is to use ElasticSearch. DynamoDB Streams can be used to keep the ElasticSearch index updated when an operation happens in DynamoDb. There are even AWS docs on this particular setup.
I have use case where I write data in Dynamo db in two table say t1 and t2 in transaction.My app needs to read data from these tables lot of times (1 write, at least 4 reads). I am considering DAX vs Elastic Cache. Anyone has any suggestions?
Thanks in advance
K
ElastiCache is not intended for use with DynamoDB.
DAX is good for read-heavy apps, like yours. But be aware that DAX is only good for eventually consistent reads, so don't use it with banking apps, etc. where the info always needs to be perfectly up to date. Without further info it's hard to tell more, these are just two general points to consider.
Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache that can reduce Amazon DynamoDB response times from milliseconds to microseconds, even at millions of requests per second. While DynamoDB offers consistent single-digit millisecond latency, DynamoDB with DAX takes performance to the next level with response times in microseconds for millions of requests per second for read-heavy workloads. With DAX, your applications remain fast and responsive, even when a popular event or news story drives unprecedented request volumes your way. No tuning required. https://aws.amazon.com/dynamodb/dax/
AWS recommends that you use **DAX as solution for this requirement.
Elastic Cache is an old method and it is used to store the session states in addition to the cache data.
DAX is extensively used for intensive reads through eventual consistent reads and for latency sensitive applications. Also DAX stores cache using these parameters:-
Item cache - populated with items with based on GetItem results.
Query cache - based on parameters used while using query or scan method
Cheers!
I'd recommend to use DAX with DynamoDB, provided you're having more read calls using item level API (and NOT query level API), such as GetItem API.
Why? DAX has one weird behavior as follows. From, AWS,
"Every write to DAX alters the state of the item cache. However, writes to the item cache don't affect the query cache. (The DAX item cache and query cache serve different purposes, and operate independently from one another.)"
Hence, If I elaborate, If your query operation is cached, and thereafter if you've write operation that affect's result of previously cached query and if same is not yet expired, in that case your query cache result would be outdated.
This out of sync issue, is also discussed here.
I find DAX useful only for cached queries, put item and get item. In general very difficult to find a use case for it.
DAX separates queries, scans from CRUD for individual items. That means, if you update an item and then do a query/scan, it will not reflect changes.
You can't invalidate cache, it only invalidates when ttl is reached or nodes memory is full and it is dropping old items.
Take Aways:
doing puts/updates and then queries - two seperate caches so out of sync
looking for single item - you are left only with primary key and default index and getItem request (no query and limit 1). You can't use any indexes for gets/updates/deletes.
Using ConsistentRead option when using query to get latest data - it works, but only for primary index.
Writing through DAX is slower than writing directly to Dynamodb since you have a hop in the middle.
XRay does not work with DAX
Use Case
You have queries that you don't really care they are not up to date
You are doing few putItem/updateItem and a lot of getItem
I have a book collection for example, each book has 'genre' field. How to query all books which genre is "fantasy" or "historical"? Like SQL SELECT * FROM book WHERE genre in ("fantasy", "historical"). Pretty usual SQL query as my opinion.
I found this feature request in GitHub, which isn't resolved yet. The Firestore documentation says: "Logical OR queries aren't supported". So, I cannot do such simple query in Firestore? What is a workaround? Should I query each genre separately and join the result?
Note, that I found similar questions like "How to get multiple documents by set of ids?", but not about custom properties. For ids, there is 'getAll()' method in the js admin sdk for example.
It is not supported in firestore as you correctly identified. You can send two separate queries and merge the results locally. This is how it needs to be done unfortunately. If yours is a simple usecase, sending two separate requests shouldn't be a problem.
However, have a look at rxfire package which is officially supported by firebase. Note that it doesn't support or queries per se but with the help of rxjs, it makes such tasks easier to manage. Here's an excerpt from the link below on what it can do:
Firebase provides realtime streams and async callbacks to do all sorts
of awesome things in your app. Getting data from one realtime stream
is easy enough, but what if you want to join it from another?
Combining multiple async streams of data can get complex. At Firebase,
we wanted to help simplify things, so we created a new JavaScript
library: RxFire.
Link to introductory blogpost: RxFire
What is the conflict resolution strategy for DynamoDB ? The white paper on Dynamo talks about returning multiple versions by GetItem to be resolved by the client.
This SO Question says that Dynamo and DynamoDB are different and GetItem returns only one value. In that case, what is the conflict resolution strategy that DynamoDB employs ?
See this
"Conflicts can arise if applications update the same item in different regions at about the same time. To ensure eventual consistency, DynamoDB global tables use a “last writer wins” reconciliation between concurrent updates, where DynamoDB makes a best effort to determine the last writer. With this conflict resolution mechanism, all of the replicas will agree on the latest update, and converge toward a state in which they all have identical data."
So the latest write wins based on some for of consensus between the replicas.
As stated, your question is not very clear: "What is the conflict resolution strategy for DynamoDB" - what conflicts? Are you referring to potentially inconsistent reads?
DynamoDB, for GetItem queries, allows both eventual consistent and strongly consistent reads, configurable with a parameter on the request (as described in the docs here: http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html). For strongly consistent reads the value returned is the most recent value at the time the query was executed. For eventual consistent reads it is possible to read a slightly out of date version of an item but there is no "conflict resolution" per se.
You may be thinking about conditional updates which allow for requests to fail if an expected condition is not met at the time the query is executed.
Two firebase performance questions:
Docs refer to flat data is best practice when constructing data. However, if I wish to retrieve a few nodes of data together (a JOIN query in SQL), this means a few network requests. Is Firebase optimizing such use case (in server/client side)? How?
When fetching a specific node, using its full path, is there any need of indexing it? (Docs refer to actual queries, and I'm not sure this case applies as a query)
Thanks
Doing a "client-side join" in Firebase is not nearly as expensive as you might expect. See this answer: Speed up fetching posts for my social network app by using query instead of observing a single event repeatedly
If you directly access the node (only calling new Firebase() and child()), no query is needed, so you won't need an index. If you're calling orderByChild() or orderByValue() you should add an index.