Elastic Search : How to get most researched terms - symfony

i m implemeting elasticsearch in a symfony2 project with fos_elastica.
everythings works fine ( indexing data, updating, etc.)
i m currently looking for user behavior analysis : i would like to get the 10 most user searches or keywords in order to re-query it .
for example :
if 45% of searches are about yellow balloons and 45% are about red balloons, i would like to suggest on my homepage some yellow or red balloons
firstly, i was thinking about creating a symfony2 entity to save user search with a timestamp then compute last 1000 searches to get the most famous keywords. although it would surely work , that would be resource killer.
i was wondering if elasticsearch is able to provide these and how to implement it .
i ve read that i could create an index to store my user queries ( and that would be awsome, cause i could use facets to compute them really easily ) , but i don t know how to do save it directly in elastic search from symfony2 without an dedicated entity.

Okay, i finally got it !
here are the different steps :
1) create a new index in config.yml with a specific mapping for your keywords search
in config.yml
indexes:
your_index:
types:
search:
mappings:
value: {type:string}
date : {type:date}
provider: acme\AppBundle\Service\SearchProvider
2) create a new class SearchProvider in Service directory
in acme\Appbundle\Service\SearchProvider
<?php
namespace acme\AppBundle\Service;
use FOS\ElasticaBundle\Provider\ProviderInterface;
use Elastica\Type;
use Elastica\Document;
class SearchProvider implements ProviderInterface
{
protected $searchType;
private $search;
public function __construct(Type $searchType)
{
$this->searchType = $searchType;
}
// the function you will call from your service
public function add( $search )
{
$this->search = $search;
$this->populate();
}
/**
* Insert the repository objects in the type index
*
* #param \Closure $loggerClosure
* #param array $options
*/
public function populate(\Closure $loggerClosure = null, array $options = array())
{
if ($loggerClosure) {
$loggerClosure('Indexing users');
}
$date = time();
$document = new Document();
$document->setData(array('value' => $this->search, 'date' => $date ) );
$this->userType->addDocuments(array($document));
$this->userType->getIndex()->refresh();
}
}
3) create a new service declaration in your service.yml
services:
acme.search_provider:
class: acme\AppBundle\Service\SearchProvider
arguments:
- #fos_elastica.index.recetas.search
tags:
- { name: fos_elastica.provider, index: your_index, type: search }
4) call your service to store new searches like this
$this->get("acme.search_provider")->add("kapoue");
kapoue will be added to the searches.
5) get all the search keywords and rank it with aggregation
$es = $this->get('fos_elastica.index.acme.search');
$query = new \Elastica\Query();
$aggregation = new \Elastica\Aggregation\Terms("top_hits");
$aggregation->setField('value');
$aggregation->setSize( 3 );
$query->addAggregation($aggregation);
$result = $es->search($query);
$mostResearched = $result->getAggregation("top_hits");
print_r ( $mostResearched ); die();

Related

One to many selected items

I have 2 entities:
Teams (id, name, short_name...)
TeamMembers (id, teams_id, player_id)
In admin, I have multiselect field, and on submit it sends ids array. How can I save it in related table?
Here is a part of code that I have in TeamsController
/**
* #param array<string, mixed> $data
*/
protected function mapDataToEntity(array $data, Teams $entity): void
{
$image = null;
if ($imageId = ($data['logo']['id'] ?? null))
{
$image = $this->mediaRepository->findMediaById($imageId);
}
$entity->setLogo($image);
$entity->setName($data['name']);
$entity->setCountry($data['country']);
$entity->setGender($data['gender']);
$entity->setShortName($data['shortName']);
$entity->setPlayers($data['players']); // <-- how do I handle THIS?
}
P.S. Also, I need to handle deletion...
Thanks in advance...
You need to load the entities you need to map them to your entity.
At the best you have here a look at the doctrine/orm documentation about association mapping: https://www.doctrine-project.org/projects/doctrine-orm/en/2.9/reference/association-mapping.html
You just have to load the entities as you load the image.
$entity->setPlayers($this->playerRepository->findById($data['players']));

Is there a way to define first item of pagination results?

I'm setting up a Symfony project displaying paginated blogposts with an admin interface to manage all this stuff. On this admin it is also possible to "highlight" one of those public blogposts so that this highlighted one is displayed at first position only on the first page.
I need the same item count on each page and that is the problem I'm dealing with.
I'm using PagerFanta so I created an AbstractRepository with a "paginate" function.
protected function paginate(QueryBuilder $qb, $limit = 20, $offset = 0)
{
if ($limit == 0) {
throw new \LogicException('$limit must be greater than 0.');
}
//Instantiates the pagination object with the result of the query
$pager = new Pagerfanta(new DoctrineORMAdapter($qb));
//Sets max data per page
$pager->setMaxPerPage($limit);
//Sets the current page
$pager->setCurrentPage($offset);
return $pager;
}
In my blogpost repository I made a querybuilder to get all public blogpost excluding the highlighted one because I can get it in another way to display it on top of the first page.
public function findAllVisible($id, $limit = 3, $offset = 1, $order = 'DESC')
{
$qb = $this
->createQueryBuilder('a')
->select('a')
->where('a.website = :website')
->setParameter('website', 'blog')
->andWhere('a.public = :public')
->setParameter('public', true)
->andWhere('a.id != :id')
->setParameter('id', $id)
->orderBy('a.dateInsert', $order)
;
return $this->paginate($qb, $limit, $offset);
}
So I first tried to change the limit and the offset according to the current page but I logically lost one item between the first and the second page.
Then I tried to include the highlighted blogpost in querybuilder but I don't know how to define it as the first result if the current page is the first one.
Any idea of how to force the first result to be the highlighted blogpost only on first page? Or another clean and appropriate way to display results as expected?
I answer to myself because I managed to do what I needed to. In case of someone is dealing with the same issue, here is how I did.
I don't use PagerFanta anymore but Doctrine Paginator tool.
Instead of excluding my highlighted article from my query I replaced my initial ORDER BY by a.id = :highlightedId DESC, a.dateInsert DESC.
Now it's working as expected.
Here is my new repository function:
/**
* Finds all visible articles
*
* #param int $highlightedTipId the highlighted tip id
* #param int $page current page
* #param int $limit max items per page
*
* #throws InvalidArgumentException
* #throws NotFoundHttpException
*
* #return Paginator
*/
public function findAllVisible($highlightedTipId, $limit = 3, $page)
{
if (!is_numeric($page)) {
throw new InvalidArgumentException('$page value is incorrect (' . $page . ').');
}
if ($page < 1) {
throw new NotFoundHttpException('Page not found');
}
if (!is_numeric($limit)) {
throw new InvalidArgumentException('$limit value is incorrect (' . $limit . ').');
}
$entityManager = $this->getEntityManager();
$query = $entityManager->createQuery(
"SELECT
a,
CASE WHEN a.id = :id THEN 1 ELSE 0 END AS HIDDEN sortCondition
FROM App\Entity\Item a
WHERE
a INSTANCE OF App\Entity\TipArticle
AND
a.website = :website
AND
a.public = :public
ORDER BY
sortCondition DESC,
a.dateInsert DESC
"
);
$query->setParameter(':website', 'blog');
$query->setParameter(':public', true);
$query->setParameter(':id', $highlightedTipId);
$firstResult = ($page - 1) * $limit;
$query
->setFirstResult($firstResult)
->setMaxResults($limit);
$paginator = new Paginator($query);
if (($paginator->count() <= $firstResult) && $page != 1) {
throw new NotFoundHttpException('Page not found');
}
return $paginator;
}
A word about this line CASE WHEN a.id = :id THEN 1 ELSE 0 END AS HIDDEN sortCondition: it is the only way I found to do a ORDER BY a.id = :highlightedId DESC with Doctrine. As you can see I made a DQL but I is also possible with QueryBuilder.
Hope it will help! :)
Nice, well done. If I may offer some advice though. In a repo you shouldn't need to get the ObjectManager ($this->getEntityManager()) as the repo is already for a type of Entity, Item in this case. You should use Criteria instead. practical example docs
You should have the ObjectManager in whichever controller you've got, so then you'd do:
$items = $this-getObjectManager()->getRepository(Item::class)->findAllVisible($params)
For use of the Paginator you should use the QueryBuilder, with Expressions, like here
As a practical example, an indexAction of mine:
use Doctrine\ORM\QueryBuilder;
use Doctrine\ORM\Tools\Pagination\Paginator as OrmPaginator;
use DoctrineORMModule\Paginator\Adapter\DoctrinePaginator as OrmAdapter;
use Zend\Paginator\Paginator;
public function indexAction()
{
// get current page, defaults to 1
$page = $this->params()->fromQuery('page', 1);
// get current page size, defaults to 10
$pageSize = $this->params()->fromQuery('pageSize', 10);
// get ordering, defaults to 'createdAt'
$orderBy = $this->params()->fromQuery('orderBy', 'createdAt');
// get order direction, defaults to Criteria::DESC
$orderDirection = ($this->params()->fromQuery('orderDirection') === Criteria::ASC)
? Criteria::ASC
: Criteria::DESC;
$criteria = new Criteria();
$criteria->setFirstResult($page * $pageSize);
$criteria->setMaxResults($pageSize);
$criteria->orderBy([$orderBy => $orderDirection]);
/** #var QueryBuilder $queryBuilder */
$queryBuilder = $this->getObjectManager()->createQueryBuilder();
$queryBuilder->select('a')->from(Article::class, 'a');
$queryBuilder->addCriteria($criteria);
$paginator = new Paginator(new OrmAdapter(new OrmPaginator($queryBuilder)));
$paginator->setCurrentPageNumber($page); // set current page
$paginator->setItemCountPerPage($pageSize); // set item count per page
return [
'paginator' => $paginator,
'queryParams' => $this->params()->fromQuery(), // pass these for your pagination uri's
];
}
NOTE: In the above the $this is an instance of a Zend Framework controller, where ->fromQuery returns (if present) a given key from the Query bit if a URI, else return the 2nd param default (or null). You should do something similar.

How to add custom property to Symfony Doctrine YAML mapping file

Can anyone tell me how to add custom property to doctrine ORM yml file?
My idea is to add a property like this:
fields:
name:
type: string
localizable: true
Then I would like to get information about this localizable property by using
protected function getEntityMetadata($entity)
{
$factory = new DisconnectedMetadataFactory($this->getContainer()->get('doctrine'));
return $factory->getClassMetadata($entity)->getMetadata();
}
and then:
$met = $this->getEntityMetadata($bundle.'\\Entity\\'.$entity);
$this->metadata = $met[0];
$fields = $this->metadata->fieldMappings;
if (isset($fields)) {
foreach ($fields as $field => $fieldMapping) {
if (isset($fieldMapping['localizable']) && $fieldMapping['localizable'] == true) {
// Do sth with it
}
}
}
The way doctrine is written makes this awkward. It seems like you'd like to keep the Yaml mapping but just add a single property. I think you can create your own custom driver extending from the one provided. The Yaml driver has mostly private methods so overriding a little bit of the functionality is difficult, but it is possible.
I created a custom driver that extends from the SimplifiedYamlDriver. The naming of the driver is important because doctrine extension will try to load one of their drivers based what comes before Driver. It also does a strpos check for Simplified in the name, so I think the safest bet is to keep the original name completely and give the original an alias.
use Doctrine\ORM\Mapping\Driver\SimplifiedYamlDriver as BaseDriver;
class SimplifiedYamlDriver extends BaseDriver
{
public function loadMetadataForClass($className, ClassMetadata $metadata)
{
parent::loadMetadataForClass($className, $metadata);
$element = $this->getElement($className);
if (!isset($element['fields'])) {
return;
}
foreach ($element['fields'] as $name => $fieldMapping) {
if (isset($fieldMapping['localizable'])) {
$original = $metadata->getFieldMapping($name);
$additional = ['localizable' => $fieldMapping['localizable']];
$newMapping = array_merge($original, $additional);
$metadata->fieldMappings[$newMapping['fieldName']] = $newMapping;
}
}
}
}
Then I told Symfony to use this driver by overriding the class inside app/config/parameters.yml
parameters:
doctrine.orm.metadata.yml.class: MyBundle\SimplifiedYamlDriver
Then I updated the mapping like in your example inside MyBundle/Resources/config/doctrine/Foo.orm.yml
MyBundle\Entity\Foo:
type: entity
id:
id:
type: integer
generator:
strategy: IDENTITY
fields:
text:
type: string
localizable: true
And I can fetch this mapping wherever I have access to doctrine with:
$mapping = $this
->getDoctrine()
->getEntityManager()
->getClassMetadata(Foo::class)
->getFieldMapping('text');
Will give me:
Array
(
[fieldName] => text
[type] => string
[columnName] => text
[localizable] => 1
)
Unfortunately, this is impossible without rewriting a significant part of Doctrine DBAL. This would impact drivers (YAML, annotation...), meta data generator...
In your case, the simplest I see would be to add a custom type let's say LocalizableString (I guess at most you will need that and maybe LocalizableText).
Adding a type is relatively straightforward, since you can extend a base type so you don't have to write any SQL. You can refer to Doctrine documentation here and Doctrine bundle one here.
Then you can just do:
$met = $this->getEntityMetadata($bundle.'\\Entity\\'.$entity);
$this->metadata = $met[0];
$fields = $this->metadata->fieldMappings;
if (isset($fields)) {
foreach ($fields as $field => $fieldMapping) {
if ($this->getClassMetadata()->getTypeOfField($field) === 'localized_string') {
// Do sth with it
}
}
}

Update a field after Linking / Unlinking Many-Many records in SilverStripe

I have created a Customer DataObject by extending Member. Customer has a many_many data relation with a Package DataObject.
I would like increment/decrement a Credits field in the Customer DataObject when a Package is linked / unlinked through the CMS based on the Limit field in the Package table.
Customer
class Customer extends Member {
private static $db = array(
'Gender' => 'Varchar(2)',
'DateOfBirth' => 'Date',
'Featured' => 'Boolean',
'Credits' => 'Int'
);
private static $many_many = array(
'Packages' => 'Package'
);
public function getCMSFields() {
$fields = new FieldList();
$config = GridFieldConfig_RelationEditor::create();
$config->removeComponentsByType('GridFieldAddNewButton');
$packageField = new GridField(
'Packages',
'Package',
$this->Packages(),
$config
);
$fields->addFieldToTab('Root.Package', $packageField);
Session::set('SingleID', $this->ID);
$this->extend('updateCMSFields', $fields);
return $fields;
}
}
Package
class Package extends DataObject {
private static $db = array(
'Title' => 'Varchar(255)',
'Limit' => 'Int'
);
private static $belongs_many_many = array(
'Customers' => 'Customer'
);
}
When you create or delete many to many relationship just one record is modified in your database - the one in table which joins elements of both sides of the relationship. Therefore neither object the relationship is based on is updated. This is why methods like: onBeforeWrite, onAfterWrite, onBeforeDelete and onAfterDelete will not be called at all and you cannot use them to detect such change.
However, Silverstripe provides class ManyManyList which is responsible for all operations connected to many to many relationships. There are two methods which are of your interest: add and remove. You can override them and put inside action to do what you need. These methods are obviously called on each link or unlink operation no matter object types are, so you should make some filtering on classes you are particularly interested in.
The proper way to override the ManyManyList class is to use Injector mechanism, so as not to modify anything inside the framework or cms folder. The example below uses relationship between Members and Groups in Silverstripe but you can easily adopt it to your need (Customer -> Member; Package -> Group).
app.yml
Injector:
ManyManyList:
class: ManyManyListExtended
ManyManyListExtended.php
/**
* When adding or removing elements on a many to many relationship
* neither side of the relationship is updated (written or deleted).
* SilverStripe does not provide any built-in actions to get information
* that such event occurs. This is why this class is created.
*
* When it is uses together with SilverStripe Injector mechanism it can provide
* additional actions to run on many-to-many relations (see: class ManyManyList).
*/
class ManyManyListExtended extends ManyManyList {
/**
* Overwritten method for adding new element to many-to-many relationship.
*
* This is called for all many-to-many relationships combinations.
* 'joinTable' field is used to make actions on specific relation only.
*
* #param mixed $item
* #param null $extraFields
* #throws Exception
*/
public function add($item, $extraFields = null) {
parent::add($item, $extraFields);
if ($this->isGroupMembershipChange()) {
$memberID = $this->getMembershipID($item, 'MemberID');
$groupID = $this->getMembershipID($item, 'GroupID');
SS_Log::log("Member ($memberID) added to Group ($groupID)", SS_Log::INFO);
// ... put some additional actions here
}
}
/**
* Overwritten method for removing item from many-to-many relationship.
*
* This is called for all many-to-many relationships combinations.
* 'joinTable' field is used to make actions on specific relation only.
*
* #param DataObject $item
*/
public function remove($item) {
parent::remove($item);
if ($this->isGroupMembershipChange()) {
$memberID = $this->getMembershipID($item, 'MemberID');
$groupID = $this->getMembershipID($item, 'GroupID');
SS_Log::log("Member ($memberID) removed from Group ($groupID)", SS_Log::INFO);
// ... put some additional actions here
}
}
/**
* Check if relationship is of Group-Member type.
*
* #return bool
*/
private function isGroupMembershipChange() {
return $this->getJoinTable() === 'Group_Members';
}
/**
* Get the actual ID for many-to-many relationship part - local or foreign key value.
*
* This works both ways: make action on a Member being element of a Group OR
* make action on a Group being part of a Member.
*
* #param DataObject|int $item
* #param string $keyName
* #return bool|null
*/
private function getMembershipID($item, $keyName) {
if ($this->getLocalKey() === $keyName)
return is_object($item) ? $item->ID : $item;
if ($this->getForeignKey() === $keyName)
return $this->getForeignID();
return false;
}
}
The solution provided by 3dgoo should also work fine but IMO that code does much more "hacking" and that's why it is much less maintainable. It demands more modifications (in both classes) and needs to be multiplied if you would like to do any additional link/unlink managing, like adding custom admin module or some forms.
The problem is when adding or removing items on a many to many relationship neither side of the relationship is written. Therefore onAfterWrite and onBeforeWrite is not called on either object.
I've come across this problem before. The solution I used isn't great but it was the only thing that worked for me.
What we can do is set an ID list of Packages to a session variable when getCMSFields is called. Then when an item is added or removed on the grid field we refresh the CMS panel to call getCMSFields again. We then retrieve the previous list and compare it to the current list. If the lists are different we can do something.
Customer
class Customer extends Member {
// ...
public function getCMSFields() {
// Some JavaScript to reload the panel each time a package is added or removed
Requirements::javascript('/mysite/javascript/cms-customer.js');
// This is the code block that saves the package id list and checks if any changes have been made
if ($this->ID) {
if (Session::get($this->ID . 'CustomerPackages')) {
$initialCustomerPackages = json_decode(Session::get($this->ID . 'CustomerPackages'), true);
$currentCustomerPackages = $this->Packages()->getIDList();
// Check if the package list has changed
if($initialCustomerPackages != $currentCustomerPackages) {
// In here is where you put your code to do what you need
}
}
Session::set($this->ID . 'CustomerPackages', json_encode($this->Packages()->getIDList()));
}
$fields = parent::getCMSFields();
$config = GridFieldConfig_RelationEditor::create();
$config->removeComponentsByType('GridFieldAddNewButton');
$packageField = GridField::create(
'Packages',
'Package',
$this->Packages(),
$config
);
// This class needs to be added so our javascript gets called
$packageField->addExtraClass('refresh-on-reload');
$fields->addFieldToTab('Root.Package', $packageField);
Session::set('SingleID', $this->ID);
$this->extend('updateCMSFields', $fields);
return $fields;
}
}
The if ($this->ID) { ... } code block is where all our session code happens. Also note we add a class to our grid field so our JavaScript refresh works $packageField->addExtraClass('refresh-on-reload');
As mentioned before, we need to add some JavaScript to reload the panel each time a package is added or removed from the list.
cms-customer.js
(function($) {
$.entwine('ss', function($){
$('.ss-gridfield.refresh-on-reload').entwine({
reload: function(e) {
this._super(e);
$('.cms-content').addClass('loading');
$('.cms-container').loadPanel(location.href, null, null, true);
}
});
});
})(jQuery);
Inside the if($initialCustomerPackages != $currentCustomerPackages) { ... } code block there are a number of things you can do.
You could use $this->Packages() to fetch all the current packages associated to this customer.
You could call array_diff and array_merge to get just the packages that have been added and removed:
$changedPackageIDs = array_merge(array_diff($initialCustomerPackages, $currentCustomerPackages), array_diff($currentCustomerPackages, $initialCustomerPackages));
$changedPackages = Package::get()->byIDs($changedPackageIDs);
The above code will add this functionality to the Customer side of the relationship. If you also want to manage the many to many relationship on the Package side of the relationship you will need to add similar code to the Package getCMSFields function.
Hopefully someone can come up with a nicer solution. If not, I hope this works for you.
note: Didn't actually check does the model work but by visually checking this should help you:
On the link you provided you are using
$customer = Customer::get()->Filter...
That returns a DataList of objects, not a singular object unless you specify what is the object you want from the DataList.
If you are filtering the Customers then you want to get a SPECIFIC customer from the DataList, e.g. the first one in this case.
$customer = Customer::get()->filter(array('ID' => $this->CustomerID))->first();
But You should be able to get the singular DataObject with:
$customer = $this->Customer();
As you are defining the Customer as "has_one". If the relation was a Has many, using () would get you a DataList of objects.
Protip:
You don't need to write our own debug files in SilverStripe. It has own functions for it. For example Debug::log("yay"); what writes the output to a file and Debug::dump("yay") that dumps it directly out.
Tip is that you can check what is the object that you accessing right. Debug::dump(get_class($customer)); would output only the class of the object.

Symfony2 Doctrine Join Entity

I have a entity with the next join:
class blogComment
{
....
/**
* #ORM\OneToMany(targetEntity="BlogComment", mappedBy="replyTo")
*/
protected $replies;
....
}
Now I get successfully all the replies. But I only want to get: where active = true
How to do that?
Oke if you guys recommend to get the comments by query in the controller how to build a nested array to get result like this:
For solving the part where you only want active replies there are a couple of options:
1) Use some custom DQL in a repository:
$dql = 'SELECT bc FROM BlogComment bc WHERE bc.replyTo = :id AND bc.active = :active';
$q = $em->createQuery($dql)
->setParameters(array('id' => $id, 'active' => true));
2) Using ArrayCollection::filter() in the getter:
public function getReplies()
{
return $this->replies
->filter(function ($reply) {
return $reply->isActive();
})
->toArray();
}
3) Using ArrayCollection::matching() (Collection Criteria API) in the getter:
use Doctrine\Common\Collections\Criteria;
// ...
public function getReplies()
{
$criteria = new Criteria::create()
->where(Criteria::expr()->eq('active', true));
return $this->replies
->matching($criteria)
->toArray();
}
4) Use Filters. These can add where clauses to queries regardless of where that query is generated. Please see the docs.
If you want to be able to fetch an entire set of replies, nested and all, in a single query, you need to implement some kind of "tree" of "nested set" functionality. I'd advise you to look at the Tree behavior of l3pp4rd/DoctrineExtensions.
http://doctrine-orm.readthedocs.org/en/latest/reference/dql-doctrine-query-language.html
Wherever you are obtaining your blog comments to display them (probably on a controller), you need to customise your query so that only the active replies are extracted. Something like:
$query = $em->createQuery('SELECT b FROM blogComment b JOIN b.replies r WHERE r.active = :active');
$query->setParameter('active', true);
$blogComments = $query->getResult();
EDIT:
For your new requirement of nested replies, you would need to specify a relationship between a comment entity and its parent comment.

Resources