A call to the following local endpoint:-
curl -k --location --request POST 'https://localhost:8443/project_exports' \
--header 'Authorization: Bearer eyJ0eXA...YVhA' \
--header 'Content-Type: application/json' \
--data-raw '{
"email": "bob#test.com",
"projectid": 0,
"status": true,
"emailedat": "2022-08-22T09:56:28.487Z",
"generatedat": "2022-08-22T09:56:28.487Z"
}'
returns an error like thus:-
"message": "An exception occurred while executing a query: SQLSTATE[42703]: Undefined column: 7 ERROR: column \"projectid\" of relation \"project_export\" does not exist\nLINE 1: INSERT INTO project_export (id, email, projectid, status, em...\n ^",
"code": 500
But the field exists in both entity and db:-
<?php
namespace App\Entity;
use ApiPlatform\Core\Annotation\ApiResource;
use App\Repository\ProjectExportRepository;
use Doctrine\ORM\Mapping as ORM;
use Symfony\Component\Serializer\Annotation\Groups;
/**
* #ORM\Entity(repositoryClass=ProjectExportRepository::class)
*/
#[ApiResource]
class ProjectExport
{
...
/**
* #ORM\Column(type="integer")
*/
#[Groups('entry-index')]
private $projectid;
...
mysql> describe project_export
-> ;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| id | int | NO | PRI | NULL | auto_increment |
| email | varchar(255) | NO | | NULL | |
| projectid | int | NO | | NULL | |
| status | tinyint(1) | NO | | NULL | |
| emailedat | datetime | NO | | NULL | |
| generatedat | datetime | NO | | NULL | |
+-------------+--------------+------+-----+---------+----------------+
6 rows in set (0.01 sec)
I'm serving the api from a docker instance locally. Not sure if this affects it any way?
So, why then, no data gets persisted and the error thrown?
Related
As a lockdown project I'm introducing myself to the concept of multi-tenancy applications. My simple application has a tenant who has a an online shop front. The shop has product categories each containing many products. My initial thought on database schema is as follows:
+====================================================================================+
| Primary Key | Sort Key (GSI PK) | Attribute 1 (GSI SK) | Attribute 2 | Attribute 3 |
|-------------|-------------------|----------------------|-------------|-------------|
| TENANT-uuid | CATEGORY-uuid | categoryName | ... | ... |
| TENANT-uuid | PRODUCT-uuid | productName | ... | ... |
| TENANT-uuid | PRODUCT-uuid | productName | ... | ... |
+====================================================================================+
So our GSI looks like so:
+=======================================================================================+
| Primary Key | Sort Key | Attribute 1 (PK) | Attribute 2 (SK) | Attribute 3 |
|---------------|-------------------|------------------|------------------|-------------|
| CATEGORY-uuid | categoryName | TENANT-uuid | CATEGORY-uuid | ... |
| PRODUCT-uuid | productName | TENANT-uuid | PRODUCT-uuid | ... |
| PRODUCT-uuid | productName | TENANT-uuid | PRODUCT-uuid | ... |
+=======================================================================================+
If I were to implement the following role policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:GetItem",
"dynamodb:PutItem"
],
"Resource": [
"arn:aws:dynamodb:XXX:XXX:table/XXX"
],
"Condition": {
"ForAllValues:StringEquals": {
"dynamodb:LeadingKeys": [
"TENANT-uuid"
]
}
}
}
]
}
How does the LeadingKeys condition work if we're running a query on an index?
Update 1
So upon further inspection it seems one way to do this (for this situation) is to have a GSI with the partition key as the TENANT-uuid and the sort key as the item's parent. I've realised I should probably add slightly more information as follows.
Our desired outcomes are:
Get list of tenant's categories -> Query with PK = TENANT-uuid and SK BeginsWith "CATEGORY"
Get list of tenant's products -> Query with PK = TENANT-uuid and SK BegingsWith "PRODUCT"
Get list of products in a specific tenant's category -> ???
Get single tenant's category -> Query with PK = TENANT-uuid and SK = CATEGORY-uuid
Get single tenant's product -> Query with PK = TENANT-uuid and SK = PRODUCT-uuid
As it stands the only one that was an issue was number 3. A little reorganisation of the schema as follows seems to work. However it does limit our ability to sort our data slightly.
Table
+----------------------+---------------+-----------------+-------------+
| TenantID (PK/GSI PK) | ItemType (SK) | Data - (GSI SK) | Attribute 2 |
+----------------------+---------------+-----------------+-------------+
| TENANT-uuid | CATEGORY-1 | Category Name | ... |
+----------------------+---------------+-----------------+-------------+
| TENANT-uuid | PRODUCT-1 | CATEGORY-1 | ... |
+----------------------+---------------+-----------------+-------------+
| TENANT-uuid | PRODUCT-2 | CATEGORY-1 | ... |
+----------------------+---------------+-----------------+-------------+
Index
+---------------+---------------+------------+-------------+
| TenantID (PK) | Data (SK) | ItemType | Attribute 2 |
+---------------+---------------+------------+-------------+
| TENANT-uuid | Category Name | CATEGORY-1 | ... |
+---------------+---------------+------------+-------------+
| TENANT-uuid | CATEGORY-1 | PRODUCT-1 | ... |
+---------------+---------------+------------+-------------+
| TENANT-uuid | CATEGORY-1 | PRODUCT-2 | ... |
+---------------+---------------+------------+-------------+
So now, for number 3, to get a list of products in a specific tenant's category we query the index with PK = TENANT-uuid and SK=CATEGORY-uuid
This allows us to meet the leadingKeys condition.
However, I'm not sure if this it the best solution. For the time being, in my little project, it works.
After almost giving up, I have found a solution. See this SO post describing how you can use wildcards in the IAM policy. Then in your GSI's, you could prefix each of your Id's with a tenant ID. Using your second table as an example, replace CATEGORY-uuid with TENANT-uuid-CATEGORY-uuid
And then your policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"dynamodb:GetItem",
"dynamodb:PutItem"
],
"Resource": [
"arn:aws:dynamodb:XXX:XXX:table/XXX"
],
"Condition": {
"ForAllValues:StringLike": {
"dynamodb:LeadingKeys": [
"TENANT-uuid*"
]
}
}
}
]
}
I tested this quick, it works just fine, and this is the approach I plan to use in my multi-tenant app.
My Psql database has a table which has jsonb as type for some columns, when i tried to upload these tables in amazon quicksight for some analysis purpose, am getting an error says unsupported datatype and the columns are getting skipped in amazom Quicksight.
Please help me to convert these into some supported type in amazon Quicksight.
Column | Type | Collation | Nullable | Default
---------------+-----------------------------+-----------+----------+-----------------------------------------------
id | bigint | | not null | nextval('solera_progresses_id_seq'::regclass)
milestones | jsonb | | |
reference_id | character varying | | |
response_code | integer | | |
activity | jsonb | | |
response | jsonb | | |
user_id | bigint | | |
You can use custom SQL to convert the data to the supported type before loading to Quicksight.
For instance, if your jsonb column contains objects like {"name": "John"}, you can create a column name in Quicksight using the query:
SELECT column_name->'name' AS name
FROM table_name
I already have GSI set to author as partition key and status as a sort key and use them to query the data I need. However, I can't figure out to sort my return data by chronological order.
DynamoDb table
-----------------------------------------------
| id | post | author | status | createdAt |
-----------------------------------------------
| 1 | post1 | author1 | publish | 2019-12-10 |
-----------------------------------------------
| 2 | post2 | author2 | draft | 2019-12-11 |
-----------------------------------------------
| 3 | post3 | author1 | publish | 2019-12-12 |
-----------------------------------------------
and the query
var params = {
TableName : "TABLENAME",
IndexName: "gsi-authorStatus",
KeyConditionExpression: "author = :author AND status = :status",
ExpressionAttributeValues: {
":author": JSON.stringify(event.bodyJSON.author),
":status": JSON.stringify(event.bodyJSON.status)
}
};
dynamo.query(params, function (err, data) {
if (err) {
console.error('Error with ', err);
context.fail(err);
} else {
context.succeed(data);
}
});
My query give me the data where author and status are matching but it give me a data in a random order. It is possible to use createdAt to make the return data order by latest?
I'm trying to change the type of 2 columns. The first works but the second gives a syntax error for the same command:
> show full columns from KernelParams;
+-------+------------------+-------------------+------+-----+---------+----------------+---------------------------------+---------+
| Field | Type | Collation | Null | Key | Default | Extra | Privileges | Comment |
+-------+------------------+-------------------+------+-----+---------+----------------+---------------------------------+---------+
| id | int(10) unsigned | NULL | NO | PRI | NULL | auto_increment | select,insert,update,references | |
| param | varchar(256) | latin1_swedish_ci | YES | UNI | NULL | | select,insert,update,references | |
| desc | varchar(256) | latin1_swedish_ci | YES | | NULL | | select,insert,update,references | |
+-------+------------------+-------------------+------+-----+---------+----------------+---------------------------------+---------+
> ALTER TABLE KernelParams MODIFY param varchar(128);
Query OK, 6 rows affected (0.08 sec)
Records: 6 Duplicates: 0 Warnings: 0
> ALTER TABLE KernelParams MODIFY desc varchar(128);
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'desc varchar(128)' at line 1
Any ideas what is wrong there?
DESC is a reserved word, so you need to quote the column name, like OTTA said in their comment. The table and column quoting character in MySQL and MariaDB is the backtick (`)
ALTER TABLE KernelParams MODIFY `desc` varchar(128);
This works as expected:
MariaDB [test]> describe new_table;
+-------------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+-------------+------+-----+---------+-------+
| idnew_table | int(11) | NO | PRI | NULL | |
| desc | varchar(45) | YES | | NULL | |
+-------------+-------------+------+-----+---------+-------+
2 rows in set (0.02 sec)
MariaDB [test]> ALTER TABLE new_table MODIFY `desc` varchar(128);
Query OK, 0 rows affected (0.03 sec)
Records: 0 Duplicates: 0 Warnings: 0
MariaDB [test]> describe new_table;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| idnew_table | int(11) | NO | PRI | NULL | |
| desc | varchar(128) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
2 rows in set (0.02 sec)
When using Doctrine ORM in Symfony2, I have the following tables generated from three different entities, of which accessory has two foreign key constraints (marked A and B below).
describe publication;
+-------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| kid | varchar(10) | NO | | NULL | |
| title | varchar(255) | NO | | NULL | |
| title_canonical | varchar(255) | NO | | NULL | | <- A
| created | datetime | NO | | NULL | |
| modified | datetime | NO | | NULL | |
+-------------------+--------------+------+-----+---------+----------------+
describe accessory;
+-------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| publication_title | varchar(255) | YES | | NULL | | <- A
| index_id | int(11) | NO | | NULL | |
| index_alias | varchar(255) | NO | | NULL | |
| value | longtext | NO | | NULL | |
| attribute_name | varchar(255) | YES | | NULL | | <- B
+-------------------+--------------+------+-----+---------+----------------+
describe attribute;
+-------------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
| name_canonical | varchar(255) | NO | | NULL | | <- B
| parameter | varchar(16) | NO | | NULL | |
+-------------------+--------------+------+-----+---------+----------------+
The foreign keys are mapped with annotations:
Publication.php
/**
* #ORM\OneToMany(targetEntity="Accessory", mappedBy="publication")
*/
protected $accessories;
Accessory.php
/**
* #ORM\ManyToOne(targetEntity="Publication", inversedBy="accessories")
* #ORM\JoinColumn(name="publication_title", referencedColumnName="title_canonical")
*/
protected $publication;
/**
* #ORM\ManyToOne(targetEntity="Attribute", inversedBy="accessories")
* #ORM\JoinColumn(name="attribute_name", referencedColumnName="name_canonical")
*/
protected $attribute;
Attribute.php
/**
* #ORM\OneToMany(targetEntity="Accessory", mappedBy="attribute")
*/
protected $accessories;
but upon running php app/console doctrine:schema:update --force I got this exception
[Doctrine\DBAL\DBALException]
An exception occurred while executing 'ALTER TABLE accessory ADD CONSTRAINT FK_A1B1251CCEE83EE7 FOREIGN KEY (publication_title) REFERENCES publication (title_canonical)':
SQLSTATE[HY000]: General error: 1005 Can't create table 'publicationsapp.#sql-2a3c_2828' (errno: 150)
So I ran php app/console doctrine:schema:update --dump-sql
ALTER TABLE accessory ADD CONSTRAINT FK_A1B1251CCEE83EE7 FOREIGN KEY (publication_title) REFERENCES publication (title_canonical);
ALTER TABLE accessory ADD CONSTRAINT FK_A1B1251C5CBDA8E FOREIGN KEY (attribute_name) REFERENCES attribute (name_canonical);
CREATE INDEX IDX_A1B1251CCEE83EE7 ON accessory (publication_title);
CREATE INDEX IDX_A1B1251C5CBDA8E ON accessory (attribute_name);
What's the correct way to resolve this? Should I edit the tables manually or with Doctrine?
Based on what I've read about errno 150, the foreign column needs to be indexed, but can't Doctrine handle this automatically?
One could add unique=true to both $nameCanonical and $titleCanonical properties or whichever columns are referenced by the foreign key. Then drop and run the schema update command to recreate the tables.
In the case of $titleCanonical
/**
* #var string
*
* #ORM\Column(name="title_canonical", type="string", length=255, unique=true)
*/
private $titleCanonical;
But ideally with Doctrine, the foreign keys should be referenced to the other table's primary key to make it valid.