FOSElasticaBundle / Custom filter - symfony

I have an entity with
title, url, description, text, enabled,...
I want to search only through enabled = true articles.
How can I do it in elastica?
My config:
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
appletrh:
index_name: 'domain.com'
client: default
types:
products:
properties:
title: { type: string, analyzer: czech }
persistence:
driver: orm
model: Web\MagazineBundle\Entity\Article
elastica_to_model_transformer:
**query_builder_method: search**
provider: ~
listener: ~
finder: ~
EntityRepository search function
public function search()
{
$qb = $this->createQueryBuilder('p');
$qb->where('p.enabled = true');
return $qb;
}
Action:
public function searchAction(Request $request)
{
$keyword = $request->query->get('keyword');
$finder = $this->get('fos_elastica.finder.domain.articles');
$paginator = $this->get('knp_paginator');
$articles= $finder->createPaginatorAdapter($keyword);
$pagination = $paginator->paginate($articles, $request->query->get('page', 1), 12);
return $this->render('WebMagazineBundle:Search:search.html.twig', ['articles' => $pagination]);
}
I thought that this is the right solution, but it's still returning all of the data from DB.

I feel like you should use "provider", not "elastica_to_model_transformer" option in the bundle config for that purpose, like:
provider:
query_builder_method: search

Related

api_platform produces Error "no handler found for uri [/index/_doc/_search] and method [POST]"

When trying to implement elasticsearch (v7.9.3) via the fos_elastica-bundle (v6.0.0) into my Symfony (v5.3.10) - App with api_platform (v2.6.6), I keep on getting this error:
"{"error":"no handler found for uri [//posts/_doc/_search] and method [POST]"}",
My api_platform.yaml reads:
api_platform:
[...]
elasticsearch:
hosts: [ '%env(ELASTICSEARCH_URL)%' ]
mapping:
App\Document\Post:
index: posts
and my fos_elastica.yaml:
fos_elastica:
clients:
default: { url: '%env(ELASTICSEARCH_URL)%' }
indexes:
posts:
properties:
id:
"type": "keyword"
source: ~
title: ~
description: ~
body: ~
children: ~
tags: ~
originalContent: ~
persistence:
driver: mongodb
model: App\Document\Post
By debugging the fos-elastica Bundle, I found out that the Elastica-Connector correctly triggers a [POST]-Request to "/posts/_doc/_search" with this request body:
{"sort":[{"id":{"order":"asc"}}],"query":{"match_all":{}},"size":30,"from":0}
If I use the Kibana Dev Tools Console and trigger an identical request
POST /posts/_doc/_search
{"sort":[{"id":{"order":"asc"}}],"query":{"match_all":{}},"size":30,"from":60}
I do get results from elasticsearch as expected:
#! Deprecation: [types removal] Specifying types in search requests is deprecated.
{
"took" : 12,
"timed_out" : false,
"_shards" : {
"total" : 1,
"successful" : 1,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : {
"value" : 3082,
"relation" : "eq"
},
"max_score" : null,
"hits" : [
{
"_index" : "posts",
"_type" : "_doc",
[...]
Apart from the deprecation notice, everything seems fine.
Does anyone have an idea why the api_platform integration of the fos_elastica-bundle does not work as expected and keeps on returning the "no handler found"-error message?
I have now helped myself by creating a custom ApiResource - filter
#[ApiFilter(FulltextFilter::class, arguments: ['index' => 'post'], properties: ['body','description','tag'])]
My custom filter implements ApiPlatform\Core\Bridge\Doctrine\MongoDbOdm\Filter\FilterInterface, directly communicates with the ElasticSearch server, sends a query to search the specified index (posts) and adds another match()-directive to the aggregationBuilder with a set of IDs matching the original search:
<?php
declare(strict_types=1);
namespace App\Filter;
use ApiPlatform\Core\Bridge\Doctrine\MongoDbOdm\Filter\FilterInterface;
use Doctrine\ODM\MongoDB\Aggregation\Builder;
use Elastica\Result;
use Elastica\Client;
use Elastica\Query;
use Symfony\Component\PropertyInfo\Type;
/**
* Filter the collection by given properties.
*
*/
final class FulltextFilter implements FilterInterface
{
protected $index = '';
protected $properties = [];
protected $client;
protected $searchParameterName;
protected $maxResultsParameterName;
const DEFAULT_MAX_RESULTS = 200;
public function __construct(Client $client, string $index = '', string $maxResultsParameterName = 'amount', string $searchParameterName = 'query', array $properties = []) {
$this->index = $index;
$this->properties = $properties;
$this->client = $client;
$this->searchParameterName = $searchParameterName;
$this->maxResultsParameterName = $maxResultsParameterName;
}
public function getFilteredIds($searchterm, $index = null, $properties = null, $maxResults = null) {
$matches = [];
if (is_null($properties)) {
$properties = array_keys($this->properties);
}
foreach ($properties as $propertyName) {
array_push($matches, ['match'=>[$propertyName => $searchterm]]);
}
$queryObject = ['query' => ['bool' => ['should' => $matches]]];
$queryObject['size'] = (int) $maxResults >0 ? (int) $maxResults : self::DEFAULT_MAX_RESULTS;
$query = new Query();
$response = $this->client->getIndex($index ?? $this->index)
->search($query->setRawQuery($queryObject))
->getResults();
return array_map(function(Result $result) {return $result->getHit()['_source']['id'];}, $response);
}
public function apply(Builder $aggregationBuilder, string $resourceClass, string $operationName = null, array &$context = [])
{
$maxResults = $context['filters'][$this->maxResultsParameterName] ?? null;
$searchterm = $context['filters'][$this->searchParameterName] ?? false;
if ($searchterm !== false) {
$aggregationBuilder->match()->field('id')->in($this->getFilteredIds($searchterm, null, null, $maxResults));
}
}
public function getDescription(string $resourceClass): array
{
return [];
}
}
This solution might not be as elegant as using the ElasticSearch-Connector natively provided by api_platform, but it is fairly performant and it works.
However, if someone comes up with a solution to fix the depicted ES-Connector issue with api_platform, please feel free to share it.
The problem is that, FOS Elastica requires an ES URL with an ending slash. But Api Platform requires a URL without ending slash.
We usually define the URL in .env file and then recall it in config files.
To solve the problem, we could define the URL in .env without endling slash and add the slash to the FOS Elastica config.
# .env
###> friendsofsymfony/elastica-bundle ###
ELASTICSEARCH_URL=http://localhost:9200
###< friendsofsymfony/elastica-bundle ###
# config/packages/api_platform.yaml
api_platform:
elasticsearch:
enabled: true
hosts: [ '%env(ELASTICSEARCH_URL)%' ]
# config/packages/fos_elastica.yaml
fos_elastica:
clients:
default: { url: '%env(ELASTICSEARCH_URL)%/' }

elasticsearch doesn't returns all hits

I'm using Symfony 2.3 and ElasticSearchBundle 3.0. I implemented two fields for the search. The search works correctly but it doesn't display all results. For example: when I search for the a keyword, the number of hits are 33 hits but it returns only 10 results.
config.php
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
serializer:
callback_class: FOS\ElasticaBundle\Serializer\Callback
serializer: serializer
indexes:
hortis:
finder: ~
client: default
settings:
index:
analysis:
analyzer:
custom_search_analyzer:
type: custom
tokenizer: standard
filter : [standard, lowercase, asciifolding]
custom_index_analyzer:
type: custom
tokenizer: standard
filter : [standard, lowercase, asciifolding, custom_filter]
filter:
custom_filter:
type: edgeNGram
side: front
min_gram: 3
max_gram: 100
types:
business:
mappings:
name: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type:string }
enabled: ~
gouvernaurat: ~
delegation: ~
postal_code: ~
# activities.principal: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type:string }
activities :
type : object
properties :
principal : ~
persistence:
driver: orm
model: Toto\AdminBundle\Entity\EntityName
provider: ~
listener: ~
finder: ~
controller.php
public function searchEngineAction(Request $request) {
$finder = $this->container->get('fos_elastica.index.hortis.business');
// get data from both fields
$querystring = strip_tags($request->get('name'));
$querystring2 = strip_tags($request->get('location'));
$boolQuery = new \Elastica\Query\Bool();
// if both fields are empty then display all businesses
if (empty($querystring) and empty($querystring2)) {
$query = new \Elastica\Query\MatchAll();
$boolQuery->addMust($query);
} else {
// create a boolean query
if (!empty($querystring)) {
$fieldQuery = new \Elastica\Query\QueryString();
$fieldQuery->setFields(array('name', 'activities.principal'));
$fieldQuery->setQuery($querystring);
$boolQuery->addMust($fieldQuery);
}
if (!empty($querystring2)) {
$fieldQuery2 = new \Elastica\Query\QueryString();
$fieldQuery2->setFields(array(
'gouvernaurat', 'delegation', 'postal_code'));
$fieldQuery2->setQuery($querystring2);
$boolQuery->addMust($fieldQuery2);
}
}
// select only enbaled business
$enabled = new \Elastica\Query\Term();
$enabled->setTerm('enabled', true);
$boolQuery->addMust($enabled);
$findAll = \Elastica\Query::create($boolQuery);
$findAll->setSize(27);
// trigger search function
$elasticaResultSet = $finder->search($findAll);
dump($elasticaResultSet);
// get results from
$findbusinesses = $elasticaResultSet->getResults();
$noresult = '';
if (!$findbusinesses) {
$noresult = 'no result';
}
$em = $this->getDoctrine()->getManager();
$FrontSettings = $em->getRepository('TotoAdminBundle:FrontSettings')->getFrontSettings();
if (!$FrontSettings) {
throw $this->createNotFoundException('Unable to find frontSettings entity');
}
// get all categories and activities
$categories = $em->getRepository('TotoAdminBundle:Category')
->findBy(array(), array('order' => 'ASC'));
if (!$categories) {
throw $this->createNotFoundException('unable to find categories and activities');
}
$paginator = $this->get('knp_paginator');
$businesses = $paginator->paginate(
$findbusinesses, $this->get('request')->query->get('page', 1)/* page number */, 9/* limit per page */
);
return $this->render('TotoFrontBundle:Front:search_result.html.twig', array(
'querystring' => $querystring, 'businesses' => $businesses,
'FrontSettings' => $FrontSettings, 'noresult' => $noresult,
'categories' => $categories,
));
}
How can I display the all the hits?
Elasticsearch by default only returns the first 10 results. This setting can be modified by specifing the from and size parameters. Note that it rarely makes sense to display all results on one page, instead use a pagination with a controllable amount of viewed items.
If you want all hits on one page also consider using the scroll api as deep pagination can get very inefficiently when having a high amount of results.

Change ES data on nested object changes automatically

types:
product:
mappings:
title: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type: string }
status:
brand.name: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type: string }
brand:
type: "nested"
properties:
status: ~
persistence:
driver: orm
model: MyBundle\Entity\Product\Product
provider:
query_builder_method: customProductQueryBuilderElastica
listener: ~
finder: ~
This is my mappings for type product. customProductQueryBuilderElastica contains code which populates only products which have active status and have related brand status active. It is working perfectly if i change products from my admin.
what i want to do is when i change my brand status to inactive, all related products should be removed from ES.
For that i have used brand as nested of product and created listener for it as explained here and now i am able to change brand status for every products in my ES automatically but i want to remove such products when brand status sets to inactive. How can this be achieved in better way?.
After many tries. i finally achieved what i want. I am posting my code here and try to help others.
Thanks to #maercky. i have taken reference to his answer which is given here
Here is my config.yml file.
types:
product:
mappings:
title: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type: string }
status:
brand.name: { search_analyzer: custom_search_analyzer, index_analyzer: custom_index_analyzer, type: string }
brand:
type: "nested"
properties:
status: ~
persistence:
driver: orm
model: XXX\MyBundle\Entity\Product\Product
provider:
query_builder_method: customProductQueryBuilderElastica
listener: ~
finder: ~
This code will go to service.yml
fos_elastica.listener.brand.product:
class: 'XXX\MyBundle\Listener\ElasticaBrandListener'
arguments:
- #fos_elastica.object_persister.search.product
- ['postPersist', 'postUpdate', 'postRemove', 'preRemove']
- #fos_elastica.indexable
calls:
- [ setContainer, [ '#service_container', #fos_elastica.object_persister.search.product ] ]
tags:
- { name: 'doctrine.event_subscriber' }
and finally this is my Listener for Brand
<?php
namespace XXX\MyBundle\Listener;
use FOS\ElasticaBundle\Doctrine\Listener as BaseListener;
use Doctrine\Common\EventArgs;
use Symfony\Component\DependencyInjection\ContainerInterface;
use XXX\MyBundle\Entity\Supplier\Brand;
use FOS\ElasticaBundle\Persister\ObjectPersister;
class ElasticaBrandListener extends BaseListener
{
/** #var \Symfony\Component\DependencyInjection\ContainerInterface */
private $container;
private $objectPersisterProducts;
public function setContainer(ContainerInterface $container,ObjectPersister $objectPersisterProduct) {
$this->container = $container;
$this->objectPersisterProducts = $objectPersisterProduct;
}
/**
* #param Doctrine\Common\EventArgs $eventArgs
*/
public function postUpdate(EventArgs $eventArgs)
{
/** #var $brand Brand */
$brand = $eventArgs->getEntity();
if ($brand instanceof Brand) {
$this->scheduledForUpdate[] = $brand;
foreach ($brand->getProducts() as $product) {
$brand_status = $brand->getStatus();
$product_status = $product->getStatus();
if($brand_status == 'active' && $product_status == 'active'){
$this->objectPersisterProducts->replaceOne($product);
}else{
$this->objectPersisterProducts->deleteOne($product);
}
}
}
}
}
?>
All this works for me well and so i am contributing this for others.

How to add a tag on a aliased service?

I defined the following service:
my_project.widget_listing_content_resolver:
class: MyProject\Widget\ListingBundle\Resolver\WidgetListingContentResolver
arguments:
- "#router"
tags:
- { name: my_project.widget_content_resolver, alias: Listing }
And I want to declare an alias of this service, with a different tag:
my_project.widget_domain_operations_content_resolver:
alias: my_project.widget_listing_content_resolver
tags:
- { name: my_project.widget_content_resolver, alias: DomainOperations }
But in my ContentResolverChain, the service aliased "DomainOperations" is not present. Is there a way to solve this ?
EDIT:
I tried the following configuration:
my_project.widget_listing_content_resolver:
class: MyProject\Widget\ListingBundle\Resolver\WidgetListingContentResolver
arguments:
- "#router"
tags:
- { name: my_project.widget_content_resolver, alias: Listing }
- { name: my_project.widget_content_resolver, alias: DomainOperations }
It results that the "my_project.widget_listing_content_resolver" service is only tagged as "Listing". My problem now is: "How to tag a service with multiple tag aliases"
I found the solution to that problem. The service alias was tagged as espected, but the additionnal tag was not read by my CompilerPass:
This was the errored CompilerPass:
$taggedServices = $container->findTaggedServiceIds(
'my_project.widget_content_resolver'
);
foreach ($taggedServices as $id => $attributes) {
$definition->addMethodCall(
'addResolver',
array($attributes[0]['alias'], new Reference($id))
);
}
As you see, it took only the first alias found ($attributes[0])
I had to change it to:
$taggedServices = $container->findTaggedServiceIds(
'my_project.widget_content_resolver'
);
foreach ($taggedServices as $id => $attributes) {
foreach ($attributes as $attribute) {
$definition->addMethodCall(
'addResolver',
array($attribute['alias'], new Reference($id))
);
}
}

FOSElasticaBundle multiple nested query

I use FOSElasticaBundle to handle searching. All works great when I have one level of nesting. However, when I have two levels of nesting results which should match the innermost nest are not returned (e.g. searching for 'xx' category does produce results, but searching for 'yy' brand does not - and should).
Here's my fos_elastica configuration:
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
my_index:
client: default
types:
product:
mappings:
title: { boost: 1 }
articleNumber: ~
introductionDateSearch: { type: integer }
delistingDateSearch: { type: integer }
deleted: { type: boolean }
category:
type: "nested"
properties:
name: { boost: 1 }
brand:
type: "nested"
properties:
name: { boost: 1 }
persistence:
driver: orm
model: MyBundle\Entity\Product
provider: ~
finder: ~
listener: ~
And my query handler:
public function searchForKeyword($keyword, AbstractUser $user)
{
$this->setFilters($user);
$keyword = trim($keyword);
if ($keyword !== '') {
$mainQuery = new \Elastica\Query\Bool();
$mainProductQuery = new \Elastica\Query\Bool();
//searching in Product title
$productQuery = new \Elastica\Query\Text();
$productQuery->setFieldQuery('title', $keyword);
$productQuery->setFieldParam('title', 'boost', 5);
$productQuery->setFieldParam('title', 'type', 'phrase_prefix');
//searching in Product articleNumber
$articleNumberQuery = new \Elastica\Query\Text();
$articleNumberQuery->setFieldQuery('articleNumber', $keyword);
$articleNumberQuery->setFieldParam('articleNumber', 'boost', 5);
$articleNumberQuery->setFieldParam('articleNumber', 'type', 'phrase_prefix');
//searching in Category name
$categoryQuery = new \Elastica\Query\Text();
$categoryQuery->setFieldQuery('name', $keyword);
$categoryQuery->setFieldParam('name', 'boost', 3);
$categoryQuery->setFieldParam('name', 'type', 'phrase_prefix');
$nestedCategoryProductQuery = new \Elastica\Query\Nested();
$nestedCategoryProductQuery->setPath('category');
$nestedCategoryProductQuery->setQuery($categoryQuery);
//searching in Brand name
$brandQuery = new \Elastica\Query\Text();
$brandQuery->setFieldQuery('name', $keyword);
$brandQuery->setFieldParam('name', 'boost', 3);
$brandQuery->setFieldParam('name', 'type', 'phrase_prefix');
$nestedBrandCategoryQuery = new \Elastica\Query\Nested();
$nestedBrandCategoryQuery->setPath('category.brand');
$nestedBrandCategoryQuery->setQuery($brandQuery);
$mainProductQuery->addShould($productQuery);
$mainProductQuery->addShould($articleNumberQuery);
$mainProductQuery->addShould($nestedCategoryProductQuery);
$mainProductQuery->addShould($nestedBrandCategoryQuery);
$mainQuery->addMust($mainProductQuery);
$esFilteredQuery = new \Elastica\Query\Filtered($mainQuery, $this->filters);
} else {
$esFilteredQuery = new \Elastica\Query\Filtered(new \Elastica\Query\MatchAll(), $this->filters);
}
$this->query = new \Elastica\Query();
$this->query->setQuery($esFilteredQuery);
}
How is the $nestedBrandCategoryQuery added to the $mainProductQuery?
Thanks for your help!
gtb
FOSElasticaBundle uses the Elastica Library. So this should not be an issue of FOSElasticaBundle.
Have a Look at http://elastica.io/ for more Details about the Lib. In my experience,there is nothing you can not do with Elastica if it is supported by Elasticsearch. Even when there is no Mapper in Elastica, just use the Raw Array Query (http://elastica.io/example/raw-array-query.html) to build the desired Query.

Resources