I am creating a product catalogue database with 10000s of products and wondered whats the best way to structure them in a NoSQL database?
I need to make sure that i can create a category structure to build the menu on the fly (which will be cached daily)
First I thought that the products could contain their own category data
EAN is the Primary Key
CatSlug is the Sort Key
Product json
Products: [
{
"EAN" : {"S" : "123456"},
"CatSlug" : {"S" : "drill"},
"CatParent" : {"S" : "drill"},
"Name" : {"S" : "Drill"},
"Img" : {"S" : "/img/img.png"},
"Desc" : {"S" : "Description"}
....
},
{NEXT PRODUCT}
]
But doing this would be difficult to create a menu structure.
Then I though having the CatSlug as the PK would be better to create the menu structure, but you would still need to go through loads of items to create this.
Having it as a separate table is quicker, but seems wrong for this type of database.
Is there a good way to do this in Dynamo DB?
You can express your category hierarchy in CatSlug using this pattern: cat#cat#cat...
then build a secondary global index with CatSlug as PK, and EAN as SK.
Related
I have the following basic test entities:
#Entity()
class Author {
#PrimaryKey()
public id!: number;
#Property()
public name!: string;
}
#Entity()
class Book {
#PrimaryKey()
public id!: number;
#Property()
public title!: string;
#ManyToOne({joinColumn: 'authorID'})
public author!: Author;
}
What i'm trying, is to select only a single Book record, with its 'author', but I care only about its ID, I don't want to actually load the entity.
If I simply call this, it won't work (no author data loaded at all):
em.getRepository(Book).findOne({id: 1}, {fields: ['id', 'title', 'author.id']});
'author.id' doesn't do the trick, the SQL doesn't even contain the 'authorID' field.
If I add 'author' to the fields list as well, it works, author is loaded (only with the ID), but as a separate entity, with a separate, additional SQL statement! That's what I'm trying to avoid.
em.getRepository(Book).findOne({id: 1}, {fields: ['id', 'title', 'author', 'author.id']})
#1. SQL
select `b0`.`id`, `b0`.`title`, `b0`.`authorID` from `book` as `b0` where `b0`.`id` = 1 limit 1
#2. SQL (this wouldn't be neccessary as I want only the ID)
select `a0`.`id` from `author` as `a0` where `a0`.`id` in (2)
--> Result:
Book: { id: 1, title: 'a book', author: { id: 2 } }
The only way I found is to add the specific 'authorID' field too to the Book entity:
#Property()
public authorID!: number;
But, I'd like to avoid introducing these foreign key columns, it would be better to handle through the already existing and used 'author' relation (only by the 'id' property).
Does any solution exists where I could retrieve a relation's ID without generating a 2nd SELECT statement (for the relation), and even avoid introducing the foreign key (next to the already existing relation property)? Would be great to receive through the relation without any extra sql statement.
Thanks in advance.
It is correct behaviour you see the second query, that is how population works, and the fact that you want just a single property from the entity does not change anything, you still populate the relation, and each relation will use its own query to load it. You can use LoadStrategy.JOINED if you want to use a single query. But that would still do a join for that relation, which is not needed for your use case.
Given you only want the FK to be present, you dont need to care about the target entity at all. This should do the trick too:
em.getRepository(Book).findOne(1, {
fields: ['id', 'title', 'author'],
populate: [],
});
This way you say you want those 3 properties to be part of what's selected from the Book entity. You already have the author property, which represents the FK. You will end up with what you want once you serialize such entity. During runtime, you will see entity reference there - an entity with just the PK. It is represented as Ref<Author> when you console.log such entity.
Note that you need that populate: [] there, as otherwise it would be inferred from your fields which contains author property, and that would trigger the full load of it.
I have two entities. Categories and Products. Each product can have many categories and each category can belong to many categories. I would like to be able to return all products and for each one to have all the categories it belongs to and their parent categories. Is datastore a good option for something like this?
example Product response:
{
id: 1,
name: "shoes",
categories: [{
id:1,
ordinal:1
},{
id:2,
ordinal:1
}]
}
I assume you meant 'Datastore' where you wrote 'dataflow'
It depends if you want to query later based on categories or not.
In case you won't need to query based on categories
I'd suggest defining your categories in a model (with the appropriate parent/child relationships) and then adding each category to a product in a LocalStructuredProperty. Datastore will save as a blob but will reconstruct the category entity model when you retrieve the product. You could also add a JsonProperty with a serialized string containing the category structure for each product. For example:
[
0: {
category: 'Electronics',
subcategories: ['Smartphones', 'Telephones', 'Gadgets']
},
1: {
category: 'Apple',
subcategories: ['iPhone']
}
]
Read more about the LocalStructuredProperty and the JsonProperty here (for Python client library).
If you need to query based on categories
Then you should use a StructuredProperty. Create a model to define your categories and their ancestor paths. Then you add one or more categories (along with their parents) to the Product entity when you instantiate it.
The Entity Property Reference in Datastore documentation has a good example of how to implement it (in Python, but also available for other languages). And here's how you filter for StructuredProperty values.
From what I've gathered, Symfony 2 / Doctrine uses the database definitions (foreign key constraints in my case) to map relations between entities. I have two tables in particular that I want to be able to relate from both sides, but I do not want to create redundant foreign keys in each table. In this case, I have an Account table, and a Transaction table.
Account Table
CREATE TABLE "account" (
"account_id" BIGSERIAL NOT NULL,
"name" VARCHAR (100) NOT NULL,
"date_created" TIMESTAMP (6) WITH TIME ZONE NOT NULL,
"date_modified" TIMESTAMP (6) WITH TIME ZONE,
CONSTRAINT "pk-account-account_id"
PRIMARY KEY ("account_id"),
);
Transaction Table
CREATE TABLE "transaction" (
"transaction_id" BIGSERIAL NOT NULL,
"account_id" BIGINT NOT NULL,
"amount" MONEY NOT NULL,
"date_created" TIMESTAMP (6) WITH TIME ZONE NOT NULL,
"date_modified" TIMESTAMP (6) WITH TIME ZONE,
CONSTRAINT "pk-transaction-transaction_id"
PRIMARY KEY ("transaction_id"),
CONSTRAINT "fk-transaction-account_id-account-account_id"
FOREIGN KEY ("account_id")
REFERENCES "account" ("account_id")
ON DELETE RESTRICT
ON UPDATE CASCADE,
);
When I generate the entities using php bin/console doctrine:generate:entities I see that the transaction entity has an $account property, but my account entity does not have a $transaction entity. I assume this is because I do not define a foreign key constraint in my account table.
In my code, I create my account object by the following:
$accounts = $this->getDoctrine()
->getRepository('BalancesBundle:Account')
->findAll();
I then would want to iterate over the array to get the total balance for each account. In the long-term, I'd like to create a helper method inside my account entity that would call getTransactions() to add up all of the transactions into one sum total.
Is this possible? I feel like I'm missing something, and that my only recourse would be to do this from within the transaction entity. I would like to avoid doing this from the transaction entity if possible.
From what I get, you can't get transaction entities of the account entity. The weird thing is that you don't have "transactions" property inside your ccount entity, am I right ? Seems like a bad mapping.
Take a look at Doctrine documentation, you have to define your "transcations" property inside your "account" entity and map it with the OneToMany association.
Then you can use "php bin/console do:ge:entities AppBundle:Account"
If your entities are setup correctly then you should be able to access the transactions from the account.
foreach ($accounts as $account) {
$transactions = $account->getTransactions();
$transaction_total = 0;
foreach($transactions as $transaction) {
$transaction_total += $transaction->getAmount();
}
echo 'Transactions Total: '.$transaction_total;
}
What I'm asking here is something weird. I'm using Symfony2 with Doctrine2. To bypass a issue between FOSRestBundle and JMSBundle (who don't like composite primary key) I need to know if it's possible - and how - to call a setter from the name of the field.
Here an example: imagine my Product entity have a primary key composed by 3 fields.
The trick is to send an array this kind of JSON into my API REST:
{
"id": {
"field1": "xxx",
"field2": "...",
"field3": "..."
},
// other fields
}
Then I use json_decode() to extract the ID field. I create a table of $key => $value with the 3 primary keys. And there is my problem, with this table I want, in a foreach loop, to do something like $myEntity->set$KEY($VALUE).
I'm doing this because I want to reuse this code with all my entities.
I know that this is something really weird, to sum up with only the name of the object field/property I want to call is appropriate setter().
Thanks ;-)
PS: sorry if my English isn't perfect, this isn't my birth langage.
You can use dynamic method name in PHP :
$myEntity->{'set'.$KEY}($VALUE);
or more readable version :
$method = 'set'.$KEY;
$myEntity->$method($VALUE);
Don't forget to check if method exists anyway :
$method = 'set'.$KEY;
if (!method_exists($myEntity, $method))
throw new \Exception('Something bad');
$myEntity->$method($VALUE);
I'm wondering if it's possible to querying doctrine entities by a property from a relation.
Here is an example :
Entity A fields :
-> title
-> content
-> description
-> date
Entity B fields :
-> title
-> link ( entity b )
-> date
Is it possible to querying Entity B by link->title property such as the following :
$this->getDoctrine()->getManager()->getRepository("acmeAppBundle:EntityB")->findBy(array( "title" => "test", "link.title" => "example" ) );
Currently I'm achieving this with a custom function from Entity B repository, but may be I'm missing something.
You can't use findBy like this. FindBy is only there to fetch very basic things. Generally it's considered as a best practice to use repository calls, because for example, if you here fetch all objects where title is test and then you get from A all B entity, then entity B will be fetched separately, while in a repository call you can use join, so only one query will be sent to your DB.