I've been doing some research on DynamoDb and a common scenario I see is where a sort key is created in a form like ORDER#{orderId}. Such as the below image in the re:Invent 2019 talk about DyamodDb.
I have some confusion around this. Where would a "unique" order like this come from? Everything I read says you're not going to be able to efficiently check all Orders in the system to guarantee uniqueness of a new OrderId. In a real scenario would it be fine to just generate a UUID and use that here?
Related
I want to create a sqlite database with at least two tables: account and service. One account can have several services and one service can be used by several accounts.
I would like to be able to query all the services which one account uses.
Now, where it gets complicated for me is that I would like to avoid a third table which is usually used - as far as I know - to resolve this many-to-many relationship.
Is there an appropriate way to do this?
Thank you!
You can denormalize your schema and store relations in a field containing an array in each table. But this is going to be less efficient to query and not all database engines support array type. Actually sqlite3 doesn't. You can mitigate that by storing that as a JSON data type which will be then an array in JSON representation.
Background: I have a relational db background and have never built anything for DynamoDB that wasn't just used for fast writes with very few reads. I am trying to learn DynamoDB patterns by migrating one of my help desk apps from MySQL to DynamoDB.
The application is a fairly simple one from a data storage perspective. A user submits a request and that request generates 1 or more tickets.
Setup: I have screens where people see initial requests and that request's tickets and search views that allow support to query on a bunch of attributes of a ticket (last name of user, status of ticket, use case of ticket, phone number of user, dept of user). This design in a SQL db is pretty straightforward but in Dynamo, I'm really being thrown for a loop on how to structure primary/sort keys and secondary indexes (if necessary).
I created one collection for requests and one collection for tickets. The individual requests have an array of ticket ids that belong to it. The ticket item has an attribute that stores the request id so that I can search that way. But what I am hung up on, is how do I incorporate searching on a ticket/request's attributes without having to do a full scan?
I read about composite keys and perhaps creating a composite sort key similar to: ## so that I can search on each of those fields directly without having to know the primary key (ticket id).
Question: How do you design dynamo collections/tables that require querying a lot of different attribute values without relying on a primary key?
This is typically something that DynamoDB is not good at, not to say it definitely cannot be done. The strength and speed for DynamoDB comes from having well known access patterns and designing your schema for these patterns. In general if you don't know what your users will search for, or there are many different possible queries, it's better to look at something like RDS or a native SQL DB. That being said a possible direction to solve this could be to create multiple lists for each of the fields and duplicate the data. This could all be done in the same table.
I'm trying Amazon's tutorial on dynamoDB: http://docs.aws.amazon.com/amazondynamodb/latest/gettingstartedguide/GettingStarted.DDBLocal.html
As I'm working through it, I can't figure out how to do simple things like:
print the names of the tables I've created or figure out what the primary keys are in a particular table, t.
I'm assuming that there is probably some really easy way to do this, I just haven't seen it.
DynamoDBLocal is essentially a DynamoDB instance running on your own computer with its own endpoint. The way to interact with it is the same way you would with the actual DynamoDB service.
The easiest way to do that is choose an API and make requests with the local endpoint. See here for some basic examples of how to set the endpoint.
In your case, it sounds like you want to use a few different API operations, which the syntax will differ depending on which language/SDK you use:
ListTables - self-explanatory
Scan - "The Scan operation returns one or more items and item attributes by accessing every item in a table or a secondary index".
DescribeTable - "Returns information about the table, including the current status of the table, when it was created, the primary key schema, and any indexes on the table."
I have a fairly full example of a few operations using the Java SDK in this answer if you want some reference.
I'm fairly new to Access.
I have a DB table that needs to be normalized. I have some information about a person. These people are authorized to grant access to areas at our work site. Every person may be authorized several times to manage different areas, and of course different people can be authorized to manage different areas. My first try at it was to include the authorization and the areas together, but I realized that I was really repeating the data that way. After doing some study I decided that the best way to do this was to create 4 tables
tblPerson, tblPermission, tblArea, tblArea_Permission
The tblArea_Permission is a join table for the many-to-many relationship between tblPermission and tblArea (this is something that I just learned about). I seemingly set up the table relationships OK on the relationship tab. I also use a query for adding the records to the join table. When I try to do this, with a query that is getting the records from the tables, I get "You cannot add or change a record because a related record is required in table XXX." This would seem to be impossible.
I decided that I could probably live with the DB not enforcing referential integrity and took that away and used a combined primary key for the two records because every person with permission will control an area in only one combination. That seemed to work, but then I noticed that the records would randomly change. I decided that the DB must be corrupt. Parts of the DB seem to be working correctly, so I started with a new database and imported the tables and one form, then started to rebuild the new tables as described above. I got the same error.
Any help would be greatly appreciated. I've read through some different books, and used google, but nothing addresses this.
If a person is authorised to manage an area, you need a persons_area table:
PersonID ) Primary key
AreaID )
Which shows which areas the person can manage. I am not sure where the permissions table is coming from.
You will then not be able to add a record to person_areas table unless you have an ID in the area table and an ID in the persons table. If either of these IDs are missing, you will get the error above.
If you want more relevant comments on your DB design, you will need to post schemas.
One requirement is that when persisting my C# objects to the database I must decide the database ID (surrogate primary key) in code.
Second requirement is that the database type for the key must be int or char(x)... so no uniqueidentifier or binary(16) or the like.
These are unchangeable requirements.
What would be the best way to go about handling this?
One idea is the base64 encoded GUIDs looking like "XSiZtdXcKU68QWe7N96Dig". These are easily created in code and are to me acceptable in URLs if necessary. But will it be too expensive regarding performance (indexing, size) having all primary and foreign keys be char(22)? Off hand I really like this idea.
Another idea would be to create a code version of a database sequence creating incremented integers for me. But I don't know if this is plausible and would need some guidance to secure the reliability. The sequencer must know har far it has come and what about threads that I don't control etc.
I imagine that no table involved will ever exceed 1.000.000 rows... will probably be far less.
You could have a table called "sequences". For each table there would be a row with a counter. Then, when you need another number, fetch it from the counter table and increment it. Put it in a transaction and you will have uniqueness.
However this will suffer in terms of performance, of course.
A simple incrementing int would be the easiest way to ensure uniqueness. This is what the database will do if you let it. If you set the table row to auto_increment, the database will do this for you automatically.
There are no security issues with this, but since you will be handling it yourself instead of letting the database engine take care of it, you will need to ensure that you don't generate the same id twice. This should be simple if you are on a single threaded system, but if your program is distributed you will need to put some effort into ensuring the uniqueness.
Seeing that you have an ASP.NET app, you could do the following (hoping and assuming all users must authenticate themselves before using your app!):
Assign each user a unique "UserID" in your database (can be INT, or CHAR)
Assign each user a "HighestSequentialID" (INT) in your database
When the user logs on, read those values from the database and store them in e.g. a custom principal, or in a cookie, or something else
whenever the user is about to insert a row, create a segmented ID: (UserID).(User's sequential number) and store it as "VARCHAR(20)" - e.g. your UserID is 15 and thus this user's entries would have unique IDs of "15.00001", "15.00002" and so on.
when the user logs off (or at any other time), update its new, highest used sequential ID in the database so that next time around, you'll know what this user has used last
Again - you'll have to do a lot more housekeeping work yourself, and it's always prone to a mishap (assigning a duplicate user ID, or misinterpreting the highest sequential number for that user).
I would strongly recommend trying to get these requirements changed - with these in place, all solutions will be sub-optimal at best, while using the database to handle this would be totally painless.
Marc
For a table below 1.000.000 rows, I would not be too terribly concerned about a char(22) Primary key. Of course the ideal solution for a situation like this would be for each object to have something unique about it that you could leverage for the key, even if it is a multi-part key. The next ideal solution would be to have the requirements changed :)