Well, I have been using Relay for a while now, and I am realising that I am still not sure about some fundamental aspects of using Relay, Graphql, React and Dynamodb.
Im using Dynamodb as the database and Dynamodb discourages the use of uuids as identifiers. At the same time, the relay Nodedefinitions function expects to output an object with a type and an id field, so my question is, can you reconcile the best practice for Relay and for Dynamodb?
Or is it just me who does not understand?
Relay doesn't have any requirements of your database. You can use whatever kind of primary key in Dynamo that you want.
You'll end up with two functions:
f(relayId) -> dynamoEntity
g(dynamoEntity) -> relayId
Those functions can be implemented however you like. E.g. maybe you can base64-encode the Dynamo primary key to produce a relay id.
Related
Background: I have a relational db background and have never built anything for DynamoDB that wasn't just used for fast writes with very few reads. I am trying to learn DynamoDB patterns by migrating one of my help desk apps from MySQL to DynamoDB.
The application is a fairly simple one from a data storage perspective. A user submits a request and that request generates 1 or more tickets.
Setup: I have screens where people see initial requests and that request's tickets and search views that allow support to query on a bunch of attributes of a ticket (last name of user, status of ticket, use case of ticket, phone number of user, dept of user). This design in a SQL db is pretty straightforward but in Dynamo, I'm really being thrown for a loop on how to structure primary/sort keys and secondary indexes (if necessary).
I created one collection for requests and one collection for tickets. The individual requests have an array of ticket ids that belong to it. The ticket item has an attribute that stores the request id so that I can search that way. But what I am hung up on, is how do I incorporate searching on a ticket/request's attributes without having to do a full scan?
I read about composite keys and perhaps creating a composite sort key similar to: ## so that I can search on each of those fields directly without having to know the primary key (ticket id).
Question: How do you design dynamo collections/tables that require querying a lot of different attribute values without relying on a primary key?
This is typically something that DynamoDB is not good at, not to say it definitely cannot be done. The strength and speed for DynamoDB comes from having well known access patterns and designing your schema for these patterns. In general if you don't know what your users will search for, or there are many different possible queries, it's better to look at something like RDS or a native SQL DB. That being said a possible direction to solve this could be to create multiple lists for each of the fields and duplicate the data. This could all be done in the same table.
I have a flat table with around 30 attributes in DynamoDB. I would like to expose an API for my end users/applications to query on a random combination of those attributes.
This is trivial to do in a typical RDBMS.
How can we do this in DynamoDB? What kind of modelling techniques and/or Key condition expressions can we use to achieve this.
Multi-faceted search like you describe can be challenging in DynamoDB. It can certainly be done, but you may be fighting the tool depending on your specific access patterns. Search in DynamoDB is supported through query (fast and cheap) and scan (slower and expensive) operations. You may want to take some time to read the docs to understand how each works, and why it's critical to structure your data to support your access patterns.
One options is to use ElasticSearch. DynamoDB Streams can be used to keep the ElasticSearch index updated when an operation happens in DynamoDb. There are even AWS docs on this particular setup.
I have an application on AWS using DynamoDB with user sending messages to each other. I am not familiar with AWS and I a lacking best practice knowledge
My application has now started to get slow to retrieve messages for a user because I have more and more data in my database.
I am thinking that it is because of my primary key and I wonder what could be a good primary key in this case.
Currently I am using a random guid as a primary key.
I am looking to retrieve all messages corresponding to a user, I am doing a scan operation.
I would like to use a composite value based on username as a primary key but I wonder if it will be better. For instance if I need to retrieve the number of messages for a user and to increment it will probably be even longer to do the request to create the primary key.
What would be a good primary key here ?
Thanks!
It will be better since it appears you often query based on the userid. Scans are expensive and should be avoided where possible. AWS has a great article on best practices for choosing a partition key (primary key). The key takeaway is the following:
You should evaluate various approaches based on your data ingestion and access pattern, then choose the most appropriate key with the least probability of hitting throttling issues.
Using a guid for the partition/primary key is a waste if you never query the data using it. Since using the query operation (rather than using scan) requires querying using the partition/primary (and sort key), you want to ensure you choose a value that you use to retrieve the data often and also has the sufficient cardinality to ensure your data is distributed across a reasonable amount of partitions.
What other access patterns do you have in your application? From what you've mentioned so far, userid seems to be a reasonable choice.
I create a DynamoDB table that store orders from ecommerce front end. When a user places an order it is stored on a DynamoDB table. This table has a primary key (order_id) and tow global secondary index: (email, SSN).
I would like to query by order status too.
So i would like to retrieve all orders on specific status on specific date. Which is the best way to model this behavior?
Make another global secondary index with a sort key?
Yes, you'll need to add another GSI.
This will, however, cost you money. One question that you can ask yourself is, do you really need real-time/low-latency lookups?
If not, then you can consider copying your DynamoDB data to a datastore like Redshift and run your queries on it. This:
Might be more cost-efficient, depending on your application.
Will allow you to support a wider variety of query patterns in future. (Remember, you can only have 5 GSIs in DynamoDB, and you've already used 2 of them)
I'm trying Amazon's tutorial on dynamoDB: http://docs.aws.amazon.com/amazondynamodb/latest/gettingstartedguide/GettingStarted.DDBLocal.html
As I'm working through it, I can't figure out how to do simple things like:
print the names of the tables I've created or figure out what the primary keys are in a particular table, t.
I'm assuming that there is probably some really easy way to do this, I just haven't seen it.
DynamoDBLocal is essentially a DynamoDB instance running on your own computer with its own endpoint. The way to interact with it is the same way you would with the actual DynamoDB service.
The easiest way to do that is choose an API and make requests with the local endpoint. See here for some basic examples of how to set the endpoint.
In your case, it sounds like you want to use a few different API operations, which the syntax will differ depending on which language/SDK you use:
ListTables - self-explanatory
Scan - "The Scan operation returns one or more items and item attributes by accessing every item in a table or a secondary index".
DescribeTable - "Returns information about the table, including the current status of the table, when it was created, the primary key schema, and any indexes on the table."
I have a fairly full example of a few operations using the Java SDK in this answer if you want some reference.