DynamoDB - UpdateItem where X <= Y - amazon-dynamodb

I have a DynamoDb table defined with a composite key of
$response = $this->dynamoDb->createTable([
...
'KeySchema' => [
[
'AttributeName' => 'product_code',
'KeyType' => 'HASH'
],
[
'AttributeName' => 'token',
'KeyType' => 'RANGE'
]
]
...
]);
I want to be able to update all records where "product_code" = "X" and "created_at" <= "Y". I assume it must be possible but I am a bit stumped. The updateItem method requires the full key but I want a conditional update without specifying a key. My latest stub reads as
$response = $this->dynamoDb->updateItem([
'TableName' => 'products',
'ExpressionAttributeValues' => [
':val1' => ['N' => (string)$this->input['product_code']]
':val2' => ['N' => (string)$this->product['created_at']['N']],
':val3' => ['N' => (string)strtotime("now")],
],
'ConditionExpression' => 'product_code = :val1 AND processed_at <= :val2',
'UpdateExpression' => 'set processed_at = :val3'
]);
But the generated error message reads as follows:
[Key] is missing and is a required parameter
Which command should I be using? Any help building my query is greatly appreciated.

The following is taken from AWS DynamoDB API reference for UpdateItem:
For the primary key, you must provide all of the attributes. For example, with a simple primary key, you only need to provide a value for the partition key. For a composite primary key, you must provide values for both the partition key and the sort key.
In your code (the update item part) you provide only hash key (product_code). You have to provide the range key (token) as well in your ConditionExpression property, because you defined a composite key.
EDIT:
(regarding the comment)
The following describes the parameters for UpdateItem:
Request Parameters
...
Key
The primary key of the item to be updated. Each element consists of an
attribute name and a value for that attribute.
For the primary key, you must provide all of the attributes. For
example, with a simple primary key, you only need to provide a value
for the partition key. For a composite primary key, you must provide
values for both the partition key and the sort key.
Type: String to AttributeValue object map
Required: Yes
...
As you can see, the primary key is required for an UpdateItem operation.
So you cannot use a secondary index to point UpdateItem to a collection of items to update.
However, you can query the table by a secondary index, and collect the primary keys of the resulted items, say in an array keysToUpdate.
Then you can update the desired items, whose keys are in keyToUpdate.

Related

Doctrine createQueryBuilder - How get data from foreign key

I have two database tables which were created with make:entity
participant (id, firstname, lastname, age, gender, school*, group*, ...)
school (id, name)
Participant has a field "school" which stores an id that is also a foreign key which points to an entry in school table.
In my frontend I want to show just a subset of the fields of the participant table.
An easy solution would be to skip the irrelevant fields, but the participant_table.twig template is used multiple times in my application.
There should be different views which show more or less participant data depending who's using the view.
This approach would lead to too much logic in the template in order control what's going to be shown/hidden.
So I need to build a custom query in the ParticipantRepository.php class which passes the data to the template.
Example:
$query = "participant.Firstname, participant.Lastname, participant.Age, participant.Gender ..."
And now the tricky part: how do I fetch the school's name by it's foreign key?
In the end I want to have an array with the following structure:
[
0 => [
"Firstname" => "Jon"
"Lastname" => "Doe"
"Age" => 6
"Gender" => "m"
"School" => "School X"
]
1 => [
"Firstname" => "Mike"
"Lastname" => "Doe"
"Age" => 10
"Gender" => "m"
"School" => "School Y"
]
...
]
Searched in Symfony, Docrtine docs, Stackoverflow..
You can achieve this by using a join in your custom query in the ParticipantRepository.php class. The join will allow you to fetch the school name from the school table based on the foreign key in the participant table. Here's an example:
$query = $this->createQueryBuilder('p')
->select('p.Firstname', 'p.Lastname', 'p.Age', 'p.Gender', 's.name as School')
->join('p.school', 's')
->getQuery();
$result = $query->getArrayResult();

Why does Analytics Data API V1 Beta not conform to the REST spec?

I'm adding GTM and GA4 to some website apps that need to produce detailed stats on the click-throughs of ads per advertiser.
This looks infeasible with standard GA reporting so am using the PHP implementation of Analytics Data API V1 Beta. Since there are few examples (eg analyticsdata/quickstart.php) of V1 reporting using PHP, I am translating other classes and operands from the REST API’s JSON .
<?php
namespace Google\Analytics\Data\V1beta;
require 'vendor/autoload.php';
$property_id = '<redacted>';
putenv('GOOGLE_APPLICATION_CREDENTIALS=Keyfile.json');
$client = new BetaAnalyticsDataClient();
// Make an API call.
$response = $client->runReport([
'property' => 'properties/' . $property_id,
'dateRanges' => [
new DateRange([
'start_date' => '2021-04-01',
'end_date' => 'today',
]
),
],
'dimensions' => [new Dimension(
[
'name' => 'customEvent:link_classes'
]
),
],
'dimensionFilter'=>[new FilterExpression(
[
'filter'=>[new Filter(
[
'field_name' => 'customEvent:Classes',
'string_filter' => [new Filter\StringFilter(
[
'match_type'=> '1',
'value'=> 'AdvertA',
'case_sensitive'=> false
])]])]])],
'metrics' => [new Metric(
[
'name' => 'eventCount',
]
)
]
]);
etc
The Quickstart example works but has endless trouble when a dimensionFilter is added.
For example, match_type should be an enum of one of a few character strings (EXACT, CONTAINS and so on). The JSON definition of match_type only shows the strings (enum 'members') and not any associated values (which would usually be integers). The GA4 migration guide has an example with
"matchType": "BEGINS_WITH"
PHP doesn’t have ‘enum’ but the equivalent would be to select one string and assign it to match_type (vide above). Wrong: StringFilter falls over unless it is given an integer operand, presumably the ordinal number of the desired match in the enum match string (and is the first one 0 or 1?). My understanding of the JSON schema was that an 'enum' list simply restricted the result to one of the unique operands, with an optional check on the operand type. (By comparison, the Python enumerate function returns an object containing a list of pairs with the ordinal number of an operand preceding the operand).
Custom dimensions appear not to conform to the API’s JSON. In Analytics, I specify a custom dimension with a dimension Name of Classes and User Property/Parameter of link_classes**.
However... in the API, dimension Name has to be customEvent:link_classes and not customEvent:Classes. Otherwise it falls over with ‘Field customEvent:Classes is not a valid dimension’
This occurs also when defining field_name in a Filter within a Filter Expression.
So is the API dimension Name not the name of the Analytics dimension Name but actually the Property/Parameter of an Analytics descriptive name? In one place I read the latter: "Custom dimensions are specified in an API report request by the dimension's parameter name and scope." but elsewhere it is implied that Name is the dimension name, e.g. /devguides/reporting/data/v1/advanced:
"dimensions": [{ "name": "customUser:last_level" }]
Finally, even falling in line with what the developers have implemented, dimensionFilter falls over with ‘Expect Google\Analytics\Data\V1beta\Filter\StringFilter’
It is Beta code but one would not expect overt deviations from the REST spec so perhaps I am reading the spec wrongly. Does anyone else have this problem?
** GTM has a ‘Click - Just Links’ trigger where the ‘click URL’ ‘contains’ the advertiser’s URL. The Classes custom dimension in the API dimension Filter has the class values of the adverts click-through links.
To answer the first part of your question, I believe the correct way to use an enum in PHP would be:
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH
As for the second question. Per the API schema documentation the name of a custom dimension is constructed as customEvent:parameter_name for event scoped dimensions and customUser:parameter_name for user scoped dimensions.
Where parameter_name, as you correctly noted, is not a descriptive name, but rather the event parameter name. In your example you seem to be using a user scoped dimension, so the dimension name in the API should be customUser:link_classes.
Here is a complete example that seems to be running fine:
require 'vendor/autoload.php';
use Google\Analytics\Data\V1beta\BetaAnalyticsDataClient;
use Google\Analytics\Data\V1beta\DateRange;
use Google\Analytics\Data\V1beta\Dimension;
use Google\Analytics\Data\V1beta\FilterExpression;
use Google\Analytics\Data\V1beta\Filter;
use Google\Analytics\Data\V1beta\Metric;
/**
* TODO(developer): Replace this variable with your Google Analytics 4
* property ID before running the sample.
*/
$property_id = 'YOUR-GA4-PROPERTY-ID';
$client = new BetaAnalyticsDataClient();
// Make an API call.
$response = $client->runReport([
'property' => 'properties/' . $property_id,
'dateRanges' => [
new DateRange([
'start_date' => '2020-03-31',
'end_date' => 'today',
]),
],
'dimensions' => [new Dimension(
[
'name' => 'customUser:link_classes'
]),
],
'dimensionFilter' => new FilterExpression(
[
'filter' => new Filter(
[
'field_name' => 'customUser:link_classes',
'string_filter' => new Filter\StringFilter(
[
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH,
'value' => 'AdvertA',
'case_sensitive' => false
]
)
])
]),
'metrics' => [new Metric(
[
'name' => 'eventCount',
]
)
]
]);
Many thanks Ilya for a most useful, prompt and correct reply.
Three points:
1.
Using
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH
instead of:
'match_type' => ‘BEGINS_WITH’
fixes the problem of “Uncaught Exception: Expect integer. in /vendor/google/protobuf/src/Google/Protobuf/Internal/GPBUtil.php” as the
MatchType::BEGINS_WITH (etc) constant returns an integer (in this case 2) from class MatchType.
2.
It would forestall errors if a reminder were added to the places in Custom Dimension documentation where dimension Name is used, such as
/devguides/reporting/data/v1/advanced: "dimensions": [{ "name": "customUser:last_level" }]
emphasising that name is not the dimension Name as defined to Analytics but rather the associated User Property/Parameter name. Or perhaps the Name heading in GA's Custom Dimension 'form' should be amended.
3.
Finally, dimensionFilter falling over with
‘Expect Google\Analytics\Data\V1beta\Filter\StringFilter’
error message was caused by my stupidity in instantiating FilterExpression, Filter and StringFilter as if they were array elements,
e.g
'string_filter' => [new Filter\StringFilter(
rather than
'string_filter' => new Filter\StringFilter(
something that is an unfortunately easy mistake to make when carelessly following the sample report code, where dateRanges, dimensions and so on correctly define arrays (since they can take multiple date ranges and dimensions), e.g.
'dimensions' => [new Dimension (

Database design to store values and arguments of functions

I'm facing the following problem, where I need to design a filter engine with nested conditional logic.
I'm representing the logic as a tree where each branch first value is "OR" or "AND"; the second value can either be
a name of a function
another branch with further conditional structure
For example:
$tree = [
'nodetype' => 'ruleset',
'conditional' => 'OR',
'children' => [
[
'nodetype' => 'method',
'methodName' => 'startsWith'
'arguments' => [
'startsWithThis' => 'john',
'subject' => 'john doe'
]
],
[
'nodetype' => 'ruleset'
'conditional' => 'AND',
'children' => [
...more nesting
]
]
]
];
This tree is then recursively evaluated using Symfony's Expression language component (I've registered custom expressions for methods like startsWith etc).
The issue is that methods will differ from one another in their number of arguments they accept and the order of those arguments. I'm not sure how to store this in a relational database, without serialising the whole tree to a json string; which I'd like to avoid.
What I came up with so far is the following database structure:
filters:
id
name
filter_arguments:
id
filter_id
name
filter_usecases:
id
filter_id
filter_usecase_values
id
filter_usecase_id
filter_argument_id
value
However this table design does not address the issue of storing the "OR" / "AND" nature of a branch; and it also cannot represent nested filters (e.g. parent-child relation of branches).
How do I go about this? Is there a specific term that describes what I'm trying to achieve here? I'd gladly read more about this but I don't even know what to google.
To take a quick stab at it, going just from the data:
node
id
nodetype
conditional
method_name
children
id
parent_node_id
child_node_id
arguments
id
node_id
key
value
Note that the relationships (children) and argument data are not in the node table, but rather are specified by cross reference tables you will have to join with when you retrieve nodes. I would expect that it is the "children" table which will become the central actor in your recursing the tree, while "node" and "arguments" will be the joined tables.
Please let us know the solution you end up using successfully.

Symfony - Get Entity Repository with multiple ID's

I have an entity that has multiple keys, how would I go about finding the proper object based on multiple ids?
$product = $em->getRepository('AcmeStoreBundle:Product')->find($id);
It's a little confusing what you're asking here. It sounds as though you have an entity with a compound key (primary key relates to multiple columns) and want to find it based on it's primary key values, yes?
If so, the find method will require an array containing values for each of the fields that make up the key:
$product = $em->getRepository('AcmeStoreBundle:Product')->find(array(
'key1' => 'value1',
'key2' => 'value2'
));
Alternatively, you could use findOneBy method. This would be useful for when the combination of the provided fields are not unique as you're able to provide a second argument to define the ordering.
$product = $em->getRepository('AcmeStoreBundle:Product')->findOneBy(array(
'key1' => 'value1',
'key2' => 'value2'
), array('updated_at' => 'DESC'));
See http://symfony.com/doc/2.0/book/doctrine.html#fetching-objects-from-the-database
$product = $em->getRepository('AcmeStoreBundle:Product')->findBy(
array('key1' => 'value1', 'key2'=>'value2')
);

Adding an array of an array at Amazon DynamoDB

I know that AmazonDB supports number, string, number set and string set as item types. But, how about a set of an string set (array of array, or multidimensional array)?
In case it's possible, this is the only way I found to do that, which didn't work (using PHP):
$units_frequencies["id"][0] = "400";
$units_frequencies["id"][1] = "401";
$units_frequencies["id"][2] = "402";
$units_frequencies["frequency"][0] = "20";
$units_frequencies["frequency"][1] = "30";
$units_frequencies["frequency"][2] = "50";
// item that will be inserted
$item = array(
'id' => array(AmazonDynamoDB::TYPE_STRING => $id),
'arrays_field' => array(
AmazonDynamoDB::TYPE_ARRAY_OF_STRINGS => array(
AmazonDynamoDB::TYPE_ARRAY_OF_STRINGS => $units_frequencies)));
I don't want to have two columns (one for $units_frequencies["id"] and $units_frequencies["frequency"]) because the second one can have two index with the same values, which is not allowed by Dynamo.
Thanks in advance.
It doesn't.
At least while looking at AttributeValue.class (AWS Java SDK)
I also couldn't find any hint on the documentation site except for those dealing with int, string, set of string or set of int
You can eventually serialize your object. More info at https://java.awsblog.com/post/Tx1K7U34AOZBLJ2/Using-Custom-Marshallers-to-Store-Complex-Objects-in-Amazon-DynamoDB

Resources