I'm facing the following problem, where I need to design a filter engine with nested conditional logic.
I'm representing the logic as a tree where each branch first value is "OR" or "AND"; the second value can either be
a name of a function
another branch with further conditional structure
For example:
$tree = [
'nodetype' => 'ruleset',
'conditional' => 'OR',
'children' => [
[
'nodetype' => 'method',
'methodName' => 'startsWith'
'arguments' => [
'startsWithThis' => 'john',
'subject' => 'john doe'
]
],
[
'nodetype' => 'ruleset'
'conditional' => 'AND',
'children' => [
...more nesting
]
]
]
];
This tree is then recursively evaluated using Symfony's Expression language component (I've registered custom expressions for methods like startsWith etc).
The issue is that methods will differ from one another in their number of arguments they accept and the order of those arguments. I'm not sure how to store this in a relational database, without serialising the whole tree to a json string; which I'd like to avoid.
What I came up with so far is the following database structure:
filters:
id
name
filter_arguments:
id
filter_id
name
filter_usecases:
id
filter_id
filter_usecase_values
id
filter_usecase_id
filter_argument_id
value
However this table design does not address the issue of storing the "OR" / "AND" nature of a branch; and it also cannot represent nested filters (e.g. parent-child relation of branches).
How do I go about this? Is there a specific term that describes what I'm trying to achieve here? I'd gladly read more about this but I don't even know what to google.
To take a quick stab at it, going just from the data:
node
id
nodetype
conditional
method_name
children
id
parent_node_id
child_node_id
arguments
id
node_id
key
value
Note that the relationships (children) and argument data are not in the node table, but rather are specified by cross reference tables you will have to join with when you retrieve nodes. I would expect that it is the "children" table which will become the central actor in your recursing the tree, while "node" and "arguments" will be the joined tables.
Please let us know the solution you end up using successfully.
Related
I'm adding GTM and GA4 to some website apps that need to produce detailed stats on the click-throughs of ads per advertiser.
This looks infeasible with standard GA reporting so am using the PHP implementation of Analytics Data API V1 Beta. Since there are few examples (eg analyticsdata/quickstart.php) of V1 reporting using PHP, I am translating other classes and operands from the REST API’s JSON .
<?php
namespace Google\Analytics\Data\V1beta;
require 'vendor/autoload.php';
$property_id = '<redacted>';
putenv('GOOGLE_APPLICATION_CREDENTIALS=Keyfile.json');
$client = new BetaAnalyticsDataClient();
// Make an API call.
$response = $client->runReport([
'property' => 'properties/' . $property_id,
'dateRanges' => [
new DateRange([
'start_date' => '2021-04-01',
'end_date' => 'today',
]
),
],
'dimensions' => [new Dimension(
[
'name' => 'customEvent:link_classes'
]
),
],
'dimensionFilter'=>[new FilterExpression(
[
'filter'=>[new Filter(
[
'field_name' => 'customEvent:Classes',
'string_filter' => [new Filter\StringFilter(
[
'match_type'=> '1',
'value'=> 'AdvertA',
'case_sensitive'=> false
])]])]])],
'metrics' => [new Metric(
[
'name' => 'eventCount',
]
)
]
]);
etc
The Quickstart example works but has endless trouble when a dimensionFilter is added.
For example, match_type should be an enum of one of a few character strings (EXACT, CONTAINS and so on). The JSON definition of match_type only shows the strings (enum 'members') and not any associated values (which would usually be integers). The GA4 migration guide has an example with
"matchType": "BEGINS_WITH"
PHP doesn’t have ‘enum’ but the equivalent would be to select one string and assign it to match_type (vide above). Wrong: StringFilter falls over unless it is given an integer operand, presumably the ordinal number of the desired match in the enum match string (and is the first one 0 or 1?). My understanding of the JSON schema was that an 'enum' list simply restricted the result to one of the unique operands, with an optional check on the operand type. (By comparison, the Python enumerate function returns an object containing a list of pairs with the ordinal number of an operand preceding the operand).
Custom dimensions appear not to conform to the API’s JSON. In Analytics, I specify a custom dimension with a dimension Name of Classes and User Property/Parameter of link_classes**.
However... in the API, dimension Name has to be customEvent:link_classes and not customEvent:Classes. Otherwise it falls over with ‘Field customEvent:Classes is not a valid dimension’
This occurs also when defining field_name in a Filter within a Filter Expression.
So is the API dimension Name not the name of the Analytics dimension Name but actually the Property/Parameter of an Analytics descriptive name? In one place I read the latter: "Custom dimensions are specified in an API report request by the dimension's parameter name and scope." but elsewhere it is implied that Name is the dimension name, e.g. /devguides/reporting/data/v1/advanced:
"dimensions": [{ "name": "customUser:last_level" }]
Finally, even falling in line with what the developers have implemented, dimensionFilter falls over with ‘Expect Google\Analytics\Data\V1beta\Filter\StringFilter’
It is Beta code but one would not expect overt deviations from the REST spec so perhaps I am reading the spec wrongly. Does anyone else have this problem?
** GTM has a ‘Click - Just Links’ trigger where the ‘click URL’ ‘contains’ the advertiser’s URL. The Classes custom dimension in the API dimension Filter has the class values of the adverts click-through links.
To answer the first part of your question, I believe the correct way to use an enum in PHP would be:
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH
As for the second question. Per the API schema documentation the name of a custom dimension is constructed as customEvent:parameter_name for event scoped dimensions and customUser:parameter_name for user scoped dimensions.
Where parameter_name, as you correctly noted, is not a descriptive name, but rather the event parameter name. In your example you seem to be using a user scoped dimension, so the dimension name in the API should be customUser:link_classes.
Here is a complete example that seems to be running fine:
require 'vendor/autoload.php';
use Google\Analytics\Data\V1beta\BetaAnalyticsDataClient;
use Google\Analytics\Data\V1beta\DateRange;
use Google\Analytics\Data\V1beta\Dimension;
use Google\Analytics\Data\V1beta\FilterExpression;
use Google\Analytics\Data\V1beta\Filter;
use Google\Analytics\Data\V1beta\Metric;
/**
* TODO(developer): Replace this variable with your Google Analytics 4
* property ID before running the sample.
*/
$property_id = 'YOUR-GA4-PROPERTY-ID';
$client = new BetaAnalyticsDataClient();
// Make an API call.
$response = $client->runReport([
'property' => 'properties/' . $property_id,
'dateRanges' => [
new DateRange([
'start_date' => '2020-03-31',
'end_date' => 'today',
]),
],
'dimensions' => [new Dimension(
[
'name' => 'customUser:link_classes'
]),
],
'dimensionFilter' => new FilterExpression(
[
'filter' => new Filter(
[
'field_name' => 'customUser:link_classes',
'string_filter' => new Filter\StringFilter(
[
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH,
'value' => 'AdvertA',
'case_sensitive' => false
]
)
])
]),
'metrics' => [new Metric(
[
'name' => 'eventCount',
]
)
]
]);
Many thanks Ilya for a most useful, prompt and correct reply.
Three points:
1.
Using
'match_type' => Filter\StringFilter\MatchType::BEGINS_WITH
instead of:
'match_type' => ‘BEGINS_WITH’
fixes the problem of “Uncaught Exception: Expect integer. in /vendor/google/protobuf/src/Google/Protobuf/Internal/GPBUtil.php” as the
MatchType::BEGINS_WITH (etc) constant returns an integer (in this case 2) from class MatchType.
2.
It would forestall errors if a reminder were added to the places in Custom Dimension documentation where dimension Name is used, such as
/devguides/reporting/data/v1/advanced: "dimensions": [{ "name": "customUser:last_level" }]
emphasising that name is not the dimension Name as defined to Analytics but rather the associated User Property/Parameter name. Or perhaps the Name heading in GA's Custom Dimension 'form' should be amended.
3.
Finally, dimensionFilter falling over with
‘Expect Google\Analytics\Data\V1beta\Filter\StringFilter’
error message was caused by my stupidity in instantiating FilterExpression, Filter and StringFilter as if they were array elements,
e.g
'string_filter' => [new Filter\StringFilter(
rather than
'string_filter' => new Filter\StringFilter(
something that is an unfortunately easy mistake to make when carelessly following the sample report code, where dateRanges, dimensions and so on correctly define arrays (since they can take multiple date ranges and dimensions), e.g.
'dimensions' => [new Dimension (
I am trying to search with the below params, and I am wondering why some cause this exception to be thrown.
Only a few params are not working. All others are working.
?q=220v+0+ph => Not working
?q=220v+1+ph => Not working
?q=220v+2+ph => Not working
?q=220v+3+ph => Not working
?q=220v+4+ph => Working
?q=220v+5+ph => Working
?q=220v+6+ph => Working
?q=220v+7+ph => Working
?q=220v+8+ph => Working
?q=220v+9+ph => Working
I am checking the center character. It is not working only in the cases of 0, 1, 2 and 3.
Query: {+(title:480v* content:480v title:3* content:3 title:ph* content:ph)
One or more of your wildcard queries is generating too many term matches. Wildcard queries are rewritten by enumerating all of the matching terms, and create a set of primitive queries matching them, combined in a BooleanQuery.
For instance, the query title:foo*, could be rewritten to title:foobar title:food title:foolish title:footpad, in an index containing those terms.
By default, a BooleanQuery allows a maximum of 1024 clauses. If you have over 1024 different terms in the index matching title:0*, for instance, that is likely your problem.
I have a problem, I want to create a simple search form with a filter assembly. These filters are attributes that belong to attribute groups.
group 1
[] Attribute 1
[] Attribute 2
[] Attribute 3
group 2
[] Attribute 1
[] Attribute 2
[] Attribute 3
But the problem is that I can not do (graphic aspect)
$builder->add('attribut', 'entity', array(
'class' => 'RestoFrontBundle:Attribut',
'group_by' => 'groupeAttribut.id',
'expanded' => true,
'multiple' => true,
'query_builder' => function(AttributRepository $er) {
return $er->createQueryBuilder('a')
->join("a.groupeAttribut", 'g')
->where("a.statut = 1");
}
))
->getForm();
Also I can not manage the game if the checkbox has been checked.
You note that the graphic aspect is the hard part. That is due to the way checkboxes are used in HTML. Unlike select inputs there is no notion of an optgroup. The closest analog for checkboxes would be a fieldset with a legend.
You may want explore using a Choice Field type rather than an entity type. Provide your choices via some provider function within which you format the options array(s) whether or not you retrieved them from a database. ( For the record, that's exactly how I populate the select at http://stayattache.com which has multiple places to retrieve options from. ) You may even want to explore creating your own form field type and template to format the output. Checkout the documentation on creating your own field type.
I hope that helps a bit. There are probably plenty of other ways to approach this as well.
I am trying to write test case for my Row classes and I don't really what is the proper way to do that.
I've seen many example use model & table, for example http://techportal.inviqa.com/2010/12/07/unit-testing-databases-with-zend-framework/.
The case doesn't use Row at all and the model is all about getters and setters. I don't want rewrite such things as the Zend_Db_Table_Row can do it automatically.
My row classes are extended from Zend_Db_Table_Row, I don't think there is necessary to write test cases like getters & setters.
And most my row classes are like following
class App_Table_Row_User extends Zend_Db_Table_Row_Abstract {
}
Then, in order to get a better test case coverage, what kind of test case I should write for a class like above?
I presume you need to populate object's attributes (table's column name => values) and run tests for your custom methods.
Zend_Db_Table_Row_Abstract constructor has one parameter - associative array with following keys:
table
data
stored
readOnly
To build useful for testing object you should use at least the "data" entries. It is associative array too. The "data" array has column names as keys and row data as values. So the testing object setUp may look as follows:
$this->object = new App_Table_Row_User(
array(
'data' => array(
'username' => 'Jon Doe',
'password' => 'qwerty',
'email' => 'j.doe#example.com'
)
)
);
You can pass NULL as values in the "data" array entries if you need something similar to the fetchNew return object.
If you are using "save" or any method that requires the table model, I will suggest using mock/stub object and passing it as the "table" parameter in the constructor. You can control any Db related operation that way.
Hope that helps :)
I am creating a form where i have to populate "X" form elements (text-fields to be specific) based on values in the database (X number of elements, properties of each element etc). Is there a way to do this using the Drupal Form API or should i start from somewhere else.
I tried using a for() loop in the form generating function but this doesn't work. Please post you suggestions and any related links.
sure you can, just put the #default_value with the value coming from the DB.
$form['text'] = array(
'#type' => 'textfield',
'#default_value' => $db->val,
);
EDIT
say you have a questionnaire object ($questionnaire) that contains a param named "question_list" where you have stored all your questions; what you can do is run a foreach on the list like:
$form['questionnaire'] = array('#type'=>'fieldset','#title'=>$questionnaire->title,);
foreach($questionnaire->question_list as $id=>$question_obj){
$form['questionnaire']['question_'.$id]=array(
'#type'=>'textfield',
'#title'=>$question->question,
'#default_value'=>$form_state['values']['question_'.$id],
);
}
At the end you will have an associative array where each question is identified by 'question_'.$id
I hope this is what you're looking for. :)