I am trying to retrieve a list of items based on loggedInUserId. Below is a rough schema I have
{
personId: 1,
accountManagerId: 2,
accountExecId: 3,
successManagerId: 5
}
I want to retrieve all the records if loggedInUserId is any of above mentioned fields. What will be the best way to organize the data?
Thanks for your help
Typically in DynamoDB you denormalize the data, unlike the normalized schema you show. That would look like the following:
pk | sk | data
1 | personId | other info
1 | accountManagerId | other info
2 | personId | other info
4 | successManagerId | other info
Now if you have a loggedInUserId as 1 and you do a Query API call to your table you get all the items that user relates to.
SELECT * FROM mytable WHERE pk= 1
1 | personId | other info
1 | accountManagerId | other info
We you have two options.
One would look something like
PK SK PersonId AccountManagerId AccountExecId SuccessManagerId
<partition value> <sort value> 1 2 3 5
The other would be to just store the whole object in a json string
PK SK UserData
<partition value> <sort value> <user data>
I prefer the second one, but both are valid
Edit:
For querying, if you want to search by the loggedInUserId you can set it as a partition key
PK=LOGGED_IN_USER_ID#1
And a simple Java example for query
var dynamoClient = DynamoDbEnhancedClient.builder().dynamoDbClient(dynamoDbClient).build();
var table = dynamoClient.table("Users", TableSchema.fromClass(Users.class));
table.getItem(Key.builder().partitionValue("1").build());
Its easier to use string partition and sort keys
Related
I'm coding in python (PySide2), but the overall concept concerns the Qt Framework.
Here are two tables of mine:
Table "recordings":
| Column | Type |
| -------- | -------------------|
| id | int (primary key) |
| param1 | int |
| param2 | int |
| ... | int |
| paramN | int |
Table "analyzed_recs":
| Column | Type |
| -------- | -------------------|
| id | int (primary key) |
| rec_id | int (foreign key) | <-- Points to recordings.id
| paramN | int |
I need in my program to display param1 and param2 from the former. In Qt I used a QSqlRelationalTable to fulfill this objective:
def _init_db_models(self):
self.analyzed_recs_sql_model = QSqlTableModel(self, self.db)
self.analyzed_recs_sql_model.setTable("analyzed_recs")
rec_id = self.analyzed_recs_sql_model.fieldIndex('rec_id')
self.analyzed_recs_sql_model.setRelation(rec_id, QSqlRelation("recordings", "id", "param1"))
self.analyzed_recs_sql_model.setRelation(rec_id, QSqlRelation("recordings", "id", "param2"))
self.analyzed_recs_sql_model.select()
self.analyzed_data_table.setModel(self.analyzed_recs_sql_model)
This code works fine in displaying the desired fields.
However, when it comes to update a record in analyzed_recs:
record = self.analyzed_recs_sql_model.record()
record.remove(record.indexOf("id"))
record.setValue("rec_id", self.current_rec_id)
record.setValue("param1", self.param1)
record.setValue("param2", param2)
self.analyzed_recs_sql_model.insertRecord(-1, record)
self.analyzed_recs_sql_model.submitAll()
The column rec_id is not set (NULL) into the table (the other params are correctly inserted into the table).
On the contrary, if I avoid using QSqlRelationalTableModel and take QSqlTableModel instead, the insertion is performed correctly (as I expected), but I lose the INNER JOIN display feature.
I was thinking as a work around to create two distinct models, a QSqlRelationalTableModel only for displaying and a QSqlTableModel only for editing the data. However I don't like the extra workload of syncing the two.
I'm sure there is a Qt feature to achieve this, but unfortunately I'm missing it.
Any suggestion?
I've had the same problem using PYQT.
The record object returned by calling record() method has no fields named 'rec_id' because the QSqlRelationalTableModel changes it with the referenced field name 'param1'. We can verify the field names using:
fieldcount = record.count()
for i in range(fieldcount):
logging.info("field %s %s", i, record.fieldName(i))
so we need to add the field before assigning it:
record = self.analyzed_recs_sql_model.record()
record.remove(record.indexOf("id"))
record.append(QSqlField("rec_id"))
record.setValue("rec_id", self.current_rec_id)
I want the following table structure, to store an auto increment a row's pid* attribute
| id | timstamp | pid*
| 00000000-1111-2222-93bb-0371fcb45674 | 0 | 1
| 00000000-1111-2222-93bb-ee742a825e88 | 1 | 2
| 00000000-1111-2222-93bb-bfac8753c0ba | 2 | 3
PutItem -> autoId() | timestamp() | max(pid) + 1 = 4 ??
For PutItem operation, Is something like the following 1) possible, and 2) acceptable in DynamoDB land?
"pid" : $util.dynamodb.toDynamoDBJson(records.findMax(pid) + 1) # just pseudo code
3) How might one implement the above using DyanmoDB resolver mapper template?
Use case:
I'm trying to use AWS DynamoDB to back GraphQL, managed by AWS AppSync, the following is the request mapping template for Mutation.createFoo
{
"version" : "2018-05-29",
"operation" : "PutItem",
"key" : {
"id": $util.dynamodb.toDynamoDBJson($util.autoId()),
},
"attributeValues" : {
"timestamp" : $util.dynamodb.toDynamoDBJson($util.time.nowEpochMilliSeconds()),
"pid" : $util.dynamodb.toDynamoDBJson($ctx.args.input.pid), # currently client provided but it is not acceptable
...
}
}
The primary key id is an UUID auto-generated by DynamoDB which is fine. But our use-case requires a incrementing pid for each new Foo in our FooTable. The business model requires at least for show a unique pid, while under the hood, queries like GetItem the UUID and timestamp will be used instead and business as usual.
I'm also weary to call for a change in business model because of an implementation detail issue.
References:
Resolver Mapping Template Reference for DynamoDB
I have a table like below in Room, in a Android application, I use Raw Query to get data. Can it be sorted by second value in array sorting_field?
---------------------------------------------
| id | other_fields | sorting_field |
---------------------------------------------
| 1001 | … | ["24","0.02","2"] |
---------------------------------------------
Initially I did the sorting part in Repository with Transformations.switchMap, inside the function a MutableLiveData> and applied Collections.sort.
It worked like a charm:
Collections.sort(list, (o1, o2) -> Double.compare(Double.valueOf(o1.sorting_field().get(positionInList)), Double.valueOf(o2.sorting_field().get(positionInList))));
After Paging implementation, I took the sorting logic out, moved to queries builder and here I am.
I am having some issues designed a lookup table in dynamodb, without running into issues with GSI.
I have an object U, this object has 4 (that is of interest) attributes. Id, A, B and C. I have another object T that has 2 (that is of interest) attributes Id and a list of U.Id's.
I need to create a DynamoDB table where i can do quick looks up such as.
Given T.Id, give me all U objects that has A, or Given T.Id, give me all connected U objects that has B. Lastly given U.Id, give me the T.Id which has the U.Id in it's list.
I where thinking something like
| T.Id | Sort Key | U.Id | U.A | U.B | U.C |
| T1 | U1 | U1 | abc | rew | bgt |
| T1 | U2 | U2 | bgf | red | bcs |
| T2 | U3 | U3 | abc | rew | bgt |
There T.Id is the Primary Key, Sort Key is the U.Id.
Then a GSI on U.ID, U.A, U.B and U.C.
This approach should work for now, but i am limited to 5 GSI, and i know that later on more attributes on U would be added.
T could have up towards 2000 U.Id's in its list.
How should i do this DynamoDB design to have the quickest looks ups and not run into problems with a limit on GSI?
The use case like get all U.Id given T.Id, U.A and U.B would be nice to support but is not a requirements, i can ask the users of this table to do a intersection of multiple calls.
The idea of my solution is to spend space complexity to reduce time complexity.
Create a table, the hash key should be the U.id or T.id, which have prefix 'U.' or 'T.', then let's call the range key as Lookup. The Lookup column should be a compositing key, if you want to set a lookup for T.Id to U.Id, the Lookup key's prefix is 'id.', if the lookup is for T.Id to A, the lookup key's prefix is 'A.' and suffix is U.Id, same to B and C.
The attributes A,B,C should only store in the record whose id and lookup are both U.Id's value
To allow U.Id to refer back to T.Id, you should create a GSI for Lookup column.
|Id |Lookup |Attributes|
+----+-----------+----------+
|U.Id|U.Id |A,B,C |
|T.Id|id.value | |
|T.Id|A.value:Id | |
|T.Id|B.value:Id | |
|T.Id|C.value:Id | |
Given T.Id (value is 'tid'), give me all U objects that has A (value is 'a')
SELECT 'Lookup' FROM table WHERE Id = T.tid & Lookup.beginsWith('A.a');
Then we can get all U's id by split the lookup's value with ':'. And then use batch get to get all U's value
same idea to B and C
Lastly given U.Id (value is 'uid'), give me the T.Id which has the U.Id in it's list.
SELECT 'Id' FROM table WHERE Lookup = id.uid;
Then you have the T.Id
Hope my answer can help you, if you have any problems just let me know.
Notice: the SQL I wrote is just pseudo code, to provide you an idea only.
What you have described is a one-to-many relationship. Have a read up on how to model one-to-many relationships in NOSQL stores.
To make this less abstract I will assume u=user and t=team. One team can have many users. One user can only be in one team.
The way to model this is to have two tables - a user table and a team table. Note that if Teams have no attributes then just the user table will suffice.
User table:
Partiton Key: UserId
Attribute: A
Attribute: B
Attribute: C
Attribute: TeamId
User table GSI:
Partition Key: Team Id
Team table:
Partition Key: TeamId
Attribute: X
Given T.Id, give me all U objects that has A
Query the User table using the GSI, with partition key = TeamId, use a filterexpression on A
Given T.Id, give me all connected U objects that has B
Query the User table using the GSI, with partition key = TeamId, use a filterexpression on B
Lastly given U.Id, give me the T.Id which has the U.Id in it's list.
Use GetItem on the User table using the Primary Partition Key
The solution described by #Yu-Huang is a graph-node implementation. You can read about it in the context of DynamoDB at https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-adjacency-graphs.html
I do not recommend this implementation. It is suitable for many-to-many relationships and will add a great deal of complexity where it is not required.
I'm in visual studio, looking to create a composite index on 2 columns for several tables. There are 2 columns: UserID is in all tables and acts as the foreign key; then, each table has its own key to refer to the parts of the object, such as phone, address... Like this:
TablePhones:
PhoneID | UserID | PhonePrefix | PhoneNumber | PhoneExtention
TableAddresses:
AddressID | UserID | AddressStreet1 | AddressStreet2 | AddressCity...
Note that users can have more than 1 address and more than 1 phone number.
I'm using linq to sql and the where clauses queries to get the objects look like this:
read queries:
where x.UserID == TheUserID
update/delete queries:
where x.UserID == TheUserID && x.PhoneID = ThePhoneID
At the moment, the primary keys are on PhoneID and AddressID and I'm looking to create composite indexes on PhoneID/UserID and AddressID/UserID. Is the order of the columns in the database fine as it is or should I move UserID in first position for all tables.
Thanks for suggestions.
Order of columns in table doesn't matter; at least for SQLServer. The important thing is in which order fields are listed in an index. Queries with conditions on leading column[s] will very benefit from the index.
If your primary key is clustered, you can create index on only userID, no need for composite key. Anyway, it will have a reference to clustered key.