ElasticSearch filter by wildcard - symfony

I'm trying to filter my results by wildcard character.
example of my records:
....
"_source" : {
"urlSlug" : "entry-title",
"revisions" : [
{
"title" : "Entry title",
"context" : "NKD"
}
]
}
each revision can have different context with different order.
And when I search for record I want to search only entities with context like "N". So I perform nested query with match_all and wildcard.
{"query":{"bool":{"must":[{"query_string":{"query":"*entry*"}},{"nested":{"path":"revisions","query":{"bool":{"should":[{"match_all":{}}],"filter":[{"wildcard":{"revisions.context":{"value":"*N*","boost":1}}}]}}}}]}},"size":10}
When I run query I get zero results. And Can't figure out how to restrict results.
I'm using for this FosElastica with following config:
indexes:
app:
types:
entity:
properties:
urlSlug: ~
revisions:
type: "nested"
properties:
title: { type: text,boost: 10 }
context: { type: text }
and my query builder looks like this:
$boolQuery = new ESQuery\BoolQuery();
$fieldQuery = new ESQuery\QueryString();
$fieldQuery->setQuery('*' . $query . '*');
$boolQuery->addMust($fieldQuery);
$nestedQuery = new ESQuery\Nested();
$nestedQuery->setPath('revisions');
$nestedBoolQuery = new ESQuery\BoolQuery();
$matchAllQuery = new ESQuery\MatchAll();
$nestedBoolQuery->addShould($matchAllQuery);
$filterQuery = new ESQuery\Wildcard();
$filterQuery->setValue('revisions.context','*N*');
$nestedBoolQuery->addFilter($filterQuery);
$nestedQuery->setQuery($nestedBoolQuery);
$boolQuery->addMust($nestedQuery);
$result = $finder->findHybrid($boolQuery,self::AUTOCOMPLETE_MAX_RESULTS);
ElasticSearch version 5.2.2

Well I did find out what is the problem in my query.
this is working one:
{"query":{"bool":{"must":[{"query_string":{"query":"*komi*"}}],"filter":[{"nested":{"path":"revisions","query":{"wildcard":{"revisions.context":{"value":"*n*","boost":1}}}}}]}},"size":10}
whole problem was uppercase wildcard search. I was looking for *N* and have zero results and with *n* it fine.

Related

Doctrine - Query builder - Select few column of a leftjoin

I have two entities :
NOTE : id, title, description, isPublic, user
USER: id, email, firstname, lastname, password, salt, roles, tag
I would like to select just few columns of the user when I get all notes of my database (in order to don't get the password of the user for example)
So in the NoteRepository I have made a doctrine request like this :
$qb = $this->createQueryBuilder('n');
//Get the owner of the knowledge
$qb
->leftJoin('n.user', 'owner')
->addSelect('owner.tag as ownerTag, owner.firstname as ownerFirstname, owner.lastname as ownerLastname')
;
return $qb->getQuery()->getResult(Query::HYDRATE_ARRAY);
(I insist to do it with a join clause, because we can imagine more than one user like owner of the note)
And I get this following JSON response :
[
{
"0":{
"id":6,
"title":"A1",
"description":"A1",
"isPublic":false,
"ownerTag":"#02a4c022d8",
"ownerFirstname":"ama",
"ownerLastname":"ama"
}
},
{
"1":{
"id":7,
"title":"Z1",
"description":"Z1",
"isPublic":false,
"ownerTag":"#00a7bd24g8",
"ownerFirstname":"z",
"ownerLastname":"z"
}
}
]
But I would like this following response :
[
{
"0":{
"id":6,
"title":"A1",
"description":"A1",
"isPublic":false,
"owner":{
"tag":"#02a4c022d8",
"firstname":"ama",
"lastname":"ama"
}
}
},
{
"1":{
"id":7,
"title":"Z1",
"description":"Z1",
"isPublic":false,
"owner":{
"tag":"#00a7bd24g8",
"firstname":"z",
"lastname":"z"
}
}
}
]
I don't know how to get it. Thank you to helping me ;)
Alright, I found a solution.
I need to use the clause PARTIAL like this:
$qb = $this->createQueryBuilder('n');
//Get the owner of the knowledge
$qb
->leftJoin('n.user', 'owner')
->addSelect('PARTIAL owner.{id,tag,firstname,lastname}')
;
return $qb->getQuery()->getResult(Query::HYDRATE_ARRAY);
I found this https://github.com/Atlantic18/DoctrineExtensions/issues/118
I'm not sure you're able to do it straight with your queryBuilder.
But you could surely do it with some PHP work on your array :)

How to add custom property to Symfony Doctrine YAML mapping file

Can anyone tell me how to add custom property to doctrine ORM yml file?
My idea is to add a property like this:
fields:
name:
type: string
localizable: true
Then I would like to get information about this localizable property by using
protected function getEntityMetadata($entity)
{
$factory = new DisconnectedMetadataFactory($this->getContainer()->get('doctrine'));
return $factory->getClassMetadata($entity)->getMetadata();
}
and then:
$met = $this->getEntityMetadata($bundle.'\\Entity\\'.$entity);
$this->metadata = $met[0];
$fields = $this->metadata->fieldMappings;
if (isset($fields)) {
foreach ($fields as $field => $fieldMapping) {
if (isset($fieldMapping['localizable']) && $fieldMapping['localizable'] == true) {
// Do sth with it
}
}
}
The way doctrine is written makes this awkward. It seems like you'd like to keep the Yaml mapping but just add a single property. I think you can create your own custom driver extending from the one provided. The Yaml driver has mostly private methods so overriding a little bit of the functionality is difficult, but it is possible.
I created a custom driver that extends from the SimplifiedYamlDriver. The naming of the driver is important because doctrine extension will try to load one of their drivers based what comes before Driver. It also does a strpos check for Simplified in the name, so I think the safest bet is to keep the original name completely and give the original an alias.
use Doctrine\ORM\Mapping\Driver\SimplifiedYamlDriver as BaseDriver;
class SimplifiedYamlDriver extends BaseDriver
{
public function loadMetadataForClass($className, ClassMetadata $metadata)
{
parent::loadMetadataForClass($className, $metadata);
$element = $this->getElement($className);
if (!isset($element['fields'])) {
return;
}
foreach ($element['fields'] as $name => $fieldMapping) {
if (isset($fieldMapping['localizable'])) {
$original = $metadata->getFieldMapping($name);
$additional = ['localizable' => $fieldMapping['localizable']];
$newMapping = array_merge($original, $additional);
$metadata->fieldMappings[$newMapping['fieldName']] = $newMapping;
}
}
}
}
Then I told Symfony to use this driver by overriding the class inside app/config/parameters.yml
parameters:
doctrine.orm.metadata.yml.class: MyBundle\SimplifiedYamlDriver
Then I updated the mapping like in your example inside MyBundle/Resources/config/doctrine/Foo.orm.yml
MyBundle\Entity\Foo:
type: entity
id:
id:
type: integer
generator:
strategy: IDENTITY
fields:
text:
type: string
localizable: true
And I can fetch this mapping wherever I have access to doctrine with:
$mapping = $this
->getDoctrine()
->getEntityManager()
->getClassMetadata(Foo::class)
->getFieldMapping('text');
Will give me:
Array
(
[fieldName] => text
[type] => string
[columnName] => text
[localizable] => 1
)
Unfortunately, this is impossible without rewriting a significant part of Doctrine DBAL. This would impact drivers (YAML, annotation...), meta data generator...
In your case, the simplest I see would be to add a custom type let's say LocalizableString (I guess at most you will need that and maybe LocalizableText).
Adding a type is relatively straightforward, since you can extend a base type so you don't have to write any SQL. You can refer to Doctrine documentation here and Doctrine bundle one here.
Then you can just do:
$met = $this->getEntityMetadata($bundle.'\\Entity\\'.$entity);
$this->metadata = $met[0];
$fields = $this->metadata->fieldMappings;
if (isset($fields)) {
foreach ($fields as $field => $fieldMapping) {
if ($this->getClassMetadata()->getTypeOfField($field) === 'localized_string') {
// Do sth with it
}
}
}

Elastic Search : How to get most researched terms

i m implemeting elasticsearch in a symfony2 project with fos_elastica.
everythings works fine ( indexing data, updating, etc.)
i m currently looking for user behavior analysis : i would like to get the 10 most user searches or keywords in order to re-query it .
for example :
if 45% of searches are about yellow balloons and 45% are about red balloons, i would like to suggest on my homepage some yellow or red balloons
firstly, i was thinking about creating a symfony2 entity to save user search with a timestamp then compute last 1000 searches to get the most famous keywords. although it would surely work , that would be resource killer.
i was wondering if elasticsearch is able to provide these and how to implement it .
i ve read that i could create an index to store my user queries ( and that would be awsome, cause i could use facets to compute them really easily ) , but i don t know how to do save it directly in elastic search from symfony2 without an dedicated entity.
Okay, i finally got it !
here are the different steps :
1) create a new index in config.yml with a specific mapping for your keywords search
in config.yml
indexes:
your_index:
types:
search:
mappings:
value: {type:string}
date : {type:date}
provider: acme\AppBundle\Service\SearchProvider
2) create a new class SearchProvider in Service directory
in acme\Appbundle\Service\SearchProvider
<?php
namespace acme\AppBundle\Service;
use FOS\ElasticaBundle\Provider\ProviderInterface;
use Elastica\Type;
use Elastica\Document;
class SearchProvider implements ProviderInterface
{
protected $searchType;
private $search;
public function __construct(Type $searchType)
{
$this->searchType = $searchType;
}
// the function you will call from your service
public function add( $search )
{
$this->search = $search;
$this->populate();
}
/**
* Insert the repository objects in the type index
*
* #param \Closure $loggerClosure
* #param array $options
*/
public function populate(\Closure $loggerClosure = null, array $options = array())
{
if ($loggerClosure) {
$loggerClosure('Indexing users');
}
$date = time();
$document = new Document();
$document->setData(array('value' => $this->search, 'date' => $date ) );
$this->userType->addDocuments(array($document));
$this->userType->getIndex()->refresh();
}
}
3) create a new service declaration in your service.yml
services:
acme.search_provider:
class: acme\AppBundle\Service\SearchProvider
arguments:
- #fos_elastica.index.recetas.search
tags:
- { name: fos_elastica.provider, index: your_index, type: search }
4) call your service to store new searches like this
$this->get("acme.search_provider")->add("kapoue");
kapoue will be added to the searches.
5) get all the search keywords and rank it with aggregation
$es = $this->get('fos_elastica.index.acme.search');
$query = new \Elastica\Query();
$aggregation = new \Elastica\Aggregation\Terms("top_hits");
$aggregation->setField('value');
$aggregation->setSize( 3 );
$query->addAggregation($aggregation);
$result = $es->search($query);
$mostResearched = $result->getAggregation("top_hits");
print_r ( $mostResearched ); die();

How to perform a Prefix query with Elastica

I'm attempting to create an autocomplete search using Elastica and Elastic Search (more specifically, FOSElasticaBundle for Symfony2).
I've created a simple index, populated it with objects.
A basic search works, ie:
$finder = $this->container->get('fos_elastica.finder.search.person');
$results = $finder->find('Mike');
I'm having some trouble getting my prefix query to work though and I can't find any examples online. Here's what I'm trying:
$finder = $this->container->get('fos_elastica.finder.search.person');
$prefixQuery = new \Elastica\Query\Prefix();
$prefixQuery->setPrefix('nameFirst','Mik');
$results = $finder->find($prefixQuery);
It doesn't kick any errors, but just doesn't return any results.
The field should be set properly...my configuration looks like this:
...
types:
person:
mappings:
nameFirst: { boost: 10 }
nameLast: { boost: 10 }
nameMiddle: { boost: 3 }
...
Can anyone tell what I'm doing wrong?
Also: Bonus Question: Is the best way to search the prefix on both the nameFirst AND nameLast fields going to be using a NestedQuery, or using addParam() to add both the nameFirst and nameLast?
Thanks a bunch-

Database Searching Using Doctrine and Symfony2

So I'm currently trying to perform a simple search using Symfony2 and Doctrine. Something similar to this: http://docs.doctrine-project.org/projects/doctrine1/en/latest/en/manual/searching.html
I've currently got the following YAML file setup to generate my entities. It generates my class Style entity correctly as a class.
...\Style:
type: entity
table: styles
id:
id:
type: integer
generator:
strategy: IDENTITY
actAs:
Searchable:
fields: [title]
batchUpdates: true
fields:
title:
type: string
length: 150
unique: true
In my controller, I'm trying to run a search on that table based on a string.
public function searchAction($pattern)
{
$repository = $this->getDoctrine()->getRepository('..:Style');
$search = $repository->search($pattern);
return $this->outputize($search);
}
However, when I try executing the code, I get the following exception.
Undefined method 'search'. The method name must start with either findBy or findOneBy!
Am I generating my entities correctly or is there something I'm clearly missing?
On a side note, when I look at my Entity/Style.php after generating, there is no clear method ->search(), is the function supposed to be generated by Symfony here?
search() is not a function supported in Symfony2. You're looking at the Symfony 1.x documentation, and Symfony2 is really different from Symfony 1.x so for reference, you should always use the doc.
There are several ways to fetch entities in Symfony2. Here are a few examples:
Find
$user = $this->getDoctrine()
->getRepository('UserBundle:User')
->find($user_id)
;
DQL:
$query = $em->createQuery(
'SELECT b FROM YourBundle:Bid b WHERE b.property_id = :property_id ORDER BY b.amount DESC'
)->setParameter('property_id', $property_id);
try {
$bids = $query->getResult();
} catch (\Doctrine\Orm\NoResultException $e) {
//Handle No Result Exception here
}
Refer to the Doctrine guide for Symfony2 here: http://symfony.com/doc/current/book/doctrine.html
Hello you can do it in symfony 3
$em = $this->getDoctrine()->getManager();
$query = $em->createQuery(
'SELECT p
FROM AppBundle:Hotel p
WHERE p.address like :location
ORDER BY p.address ASC'
)->setParameter('location','%'.$request->get('location').'%' );
$hotel = $query->getResult();

Resources