FOSElasticaBundle indexing hidden objects - symfony

I have installed FOSElasticaBundle, my config fos_elastica.yml is as follows:
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
test:
client: default
index_name : %elastica_index_name%
types:
exhibition:
mappings:
name : { boost: 100 }
company: { boost: 9 }
directions: { boost: 8}
tags: { boost: 7 }
description : { boost: 6 }
status : ~
persistence:
driver: orm
model: Test\MainBundle\Entity\Exhibition
finder: ~
provider: ~
listener:
is_indexable_callback: "isPublic"
On persist is_indexable_callback works. But when I run fos:elastica:populate, the index is created even for hidden objects.
How can I index just active object? Maybe using analyzer or something like that?
Thanks in advance!

You should specify:
provider:
query_builder_method: createSearchIndexQueryBuilder
and then add method into related Entity\Repository:
public function createSearchIndexQueryBuilder()
{
$qb = $this->createQueryBuilder()
//->write_magic_here;
return $qb;
}

Related

Unrecognized option "types" | FOS Elastica & Symfony

I am able to create indexes 1 by 1 but my purpose is to have 1 big index with subindexes to search. (I am reproducing an existing app but upgrading dependencies)
Symfony : 5.4*
FOS-Elastica : 6.1
Elastic : 7*
This is my error message and if I change types to properties I have similar errors. I have been trying to switch and indent differently all day:
Unrecognized option "types" under "fos_elastica.indexes.app". Available options are "_id", "_routing", "_source", "analyzer", "client", "date_detection", "dynamic", "dynamic_date_formats", "dynamic_templates", "finder", "index_
name", "index_prototype", "indexable_callback", "numeric_detection", "persistence", "properties", "serializer", "settings", "use_alias".
What am I doing wrong please ?
#app/config/config.yml
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
app:
settings:
analysis:
analyzer:
fr:
tokenizer: standard
filter: [ lowercase, stop_fr, snowball_fr ]
autocomplete:
type: custom
tokenizer: whitespace
filter: [ lowercase, engram, elision ]
csv:
type: pattern
pattern: '\s*,\s*'
lowercase: false
filter:
snowball_fr:
type: "snowball"
language: "French"
stop_fr:
type: "stop"
stopwords: "_french_"
engram:
type: edge_ngram
min_gram: 2
max_gram: 15
types:
# ---------
# USER
# ---------
user:
properties:
username: ~
email: ~
organization:
type: object
properties:
id: { index: true }
code: { index: true }
name:
index: true
type: text
fields:
source: { type: text, index: true }
persistence:
driver: orm # orm, mongodb, propel are available
model: App\Entity\User
provider: ~
listener: ~
finder: ~
So I was also figuring out how to deal with the same issue as yours, and I do confirm lot of the docs and blogposts out there use the config struture you posted. However when I went back to the v6.1 documentation of the bundle I found this:
Dynamic templates
Dynamic templates allow to define mapping templates
that will be applied when dynamic introduction of fields / objects
happens.
Documentation
fos_elastica:
indexes:
user:
dynamic_templates:
my_template_1:
match: apples_*
mapping:
type: float
my_template_2:
match: *
match_mapping_type: text
mapping:
type: keyword
properties:
username: { type: text }
So in your case a working config would look like:
#app/config/config.yml
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
user:
settings:
analysis:
analyzer:
fr:
tokenizer: standard
filter: [ lowercase, stop_fr, snowball_fr ]
autocomplete:
type: custom
tokenizer: whitespace
filter: [ lowercase, engram, elision ]
csv:
type: pattern
pattern: '\s*,\s*'
lowercase: false
filter:
snowball_fr:
type: "snowball"
language: "French"
stop_fr:
type: "stop"
stopwords: "_french_"
engram:
type: edge_ngram
min_gram: 2
max_gram: 15
properties:
username: ~
email: ~
organization:
type: object
properties:
id: { index: true }
code: { index: true }
name:
index: true
type: text
fields:
source: { type: text, index: true }
persistence:
driver: orm # orm, mongodb, propel are available
model: App\Entity\User
provider: ~
listener: ~
finder: ~

Error "No providers were registered for index "apple"" after FOSElasticaBundle Update

I am trying to update my Symfony version and i have troubles with FOS elastica-bundle 5.0.3.
I have updated my ES server to 5.6.
If i dump my old ES datas and put them into the new server everythings works fine.
But if i try to do an
fos:elastica:populate
i have this error :
No providers were registered for index "apple".
My old configuration (FOSEB 3.0) was not needing any providers for this index and i don't understand how to add one for my new configuration.
I have searched all day how to configure this index provider to make the populate work on my new environnment and i have tried lot of things but with no success.
My elastica.yml :
fos_elastica:
clients:
default: { host: "%elasticsearch.ip%", port: "%elasticsearch.port%" }
indexes:
apple:
client: default
settings:
index:
analysis:
analyzer:
custom_index_analyzer:
type: custom
tokenizer: standard
filter : [lowercase, asciifolding, nGram]
custom_search_analyzer:
type: custom
tokenizer: standard
filter : [lowercase, asciifolding]
filter:
nGram:
type: nGram
min_gram: 3
max_gram: 20
token_chars: ['letter', 'digit', 'punctuation', 'symbol']
types:
banana:
properties:
id: { boost: 5 }
persistence:
identifier: id
driver: orm
model: apple\AppBundle\Entity\banana\banana
provider:
service: "#apple_app.elastica.provider.banana"
orange:
properties:
id: { boost: 5 }
persistence:
identifier: id
driver: orm
model: apple\AppBundle\Entity\banana\orange
provider:
service: "#apple_app.elastica.provider.orange"
My services :
services:
# manager
apple_app.elastica.manager.abstract:
class: apple\AppBundle\Manager\Common\ElasticaManager
abstract: true
calls:
- ['setTranslator', ['#translator']]
- ['setDefaultLocale', ['%locale%']]
- ['setLocales', ['%available_languages%']]
apple_app.elastica.manager.banana:
class: apple\AppBundle\Manager\Elastica\bananaManager
parent: apple_app.elastica.manager.abstract
calls:
- ['setIndexes', [{'banana': '#fos_elastica.index.apple.banana'}]]
- ['setContainer', ['#service_container']]
apple_app.elastica.manager.orange:
class: apple\AppBundle\Manager\Elastica\orangeManager
parent: apple_app.elastica.manager.abstract
calls:
- ['setIndexes', [{'orange': '#fos_elastica.index.apple.orange'}]]
# provider
apple_app.elastica.provider.abstract:
class: apple\AppBundle\Provider\Common\AbstractProvider
abstract: true
calls:
- ['setEntityManager', ['#doctrine.orm.entity_manager']]
apple_app.elastica.provider.apple:
class: apple\AppBundle\Provider\Common\AbstractProvider
abstract: true
calls:
- ['setEntityManager', ['#doctrine.orm.entity_manager']]
apple_app.elastica.provider.banana:
parent: apple_app.elastica.provider.abstract
class: apple\AppBundle\Provider\bananaProvider
tags:
- { name: fos_elastica.provider, index: apple, type: banana }
calls:
- ['setIndex', ['#fos_elastica.index.apple.banana']]
- ['setElasticaManager', ['#apple_app.elastica.manager.banana']]
apple_app.elastica.provider.orange:
parent: apple_app.elastica.provider.abstract
class: apple\AppBundle\Provider\orangeProvider
tags:
- { name: fos_elastica.provider, index: apple, type: orange }
calls:
- ['setIndex', ['#fos_elastica.index.apple.orange']]
- ['setElasticaManager', ['#apple_app.elastica.manager.orange']]
# listener
apple_app.elastica.listener.abstract:
class: apple\AppBundle\Listener\Common\ElasticaListener
calls:
- ['setDefaultLocale', ['%locale%']]
- ['setLocales', ['%available_languages%']]
apple_app.elastica.listener.banana:
parent: apple_app.elastica.listener.abstract
class: apple\AppBundle\Listener\Elastica\bananaListener
calls:
- ['setManager', ['#apple_app.elastica.manager.banana']]
- ['setIndex']
tags:
- { name: doctrine.event_subscriber }
apple_app.elastica.listener.orange:
parent: apple_app.elastica.listener.abstract
class: apple\AppBundle\Listener\Elastica\orangeListener
calls:
- ['setManager', ['#apple_app.elastica.manager.orange']]
- ['setIndex']
tags:
- { name: doctrine.event_subscriber }
Any help or idea would be wonderful.
Thank you in advance

How to configure correctly FOS Elastica analyzers and filters?

In my symfony2 application, I am using FOS Elastica bundle to perform searches.
I have tried to set up analyzers and filters but it seems they just have no effect. For instance, if I search for the word 'cake', the objects containing the sentence case 'Cake' won't be returned.
How can I configure those analyzers and filters correctly ?
My config :
#Elastic Search
fos_elastica:
default_manager: orm
clients:
default: { host: localhost, port: 9200 }
indexes:
website:
client: default
settings:
index:
analysis:
analyzer:
custom_index_analyzer :
type : custom
tokenizer: nGram
filter : [stopwords, asciifolding ,lowercase, snowball, elision, worddelimiter]
custom_search_analyzer :
type : custom
tokenizer: nGram
filter : [stopwords, asciifolding ,lowercase, snowball, elision, worddelimiter]
tokenizer:
nGram:
type: nGram
min_gram: 1
max_gram: 2
filter:
snowball:
type: snowball
language: French
elision:
type: elision
articles: [l, m, t, qu, n, s, j, d]
stopwords:
type: stop
stopwords: [_french_]
ignore_case : true
worddelimiter :
type: word_delimiter
index_name: foodmeup
types:
recipe:
mappings:
name:
boost: 5
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
nickName:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
content:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
recipeIngredients:
type: "nested"
properties:
name:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
product:
type: "nested"
properties:
name: { type: string, boost: 10}
nickName: { type: string }
content: { type: string }
tags:
type: "nested"
boost: 5
properties:
name: { type: string }
userRecipes:
type: "nested"
properties:
name:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
content:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
tags:
type: "nested"
boost: 5
properties:
name:
index_analyzer : custom_index_analyzer
search_analyzer : custom_search_analyzer
persistence:
driver: orm
model: AppBundle\Entity\FoodAnalytics\Recipe
repository: AppBundle\Repository\FoodAnalytics\RecipeRepository
provider: ~
finder: ~
listener: ~ # by default, listens to "insert", "update" and "delete"
product:
mappings:
name: { type: string, boost: 10}
nickName: { type: string }
content: { type: string }
userIngredients:
type: "nested"
properties:
name: { type: string }
content: { type: string }
tags:
type: "nested"
boost: 5
properties:
name: { type: string }
persistence:
driver: orm
model: AppBundle\Entity\MarketPlace\Product
repository: AppBundle\Repository\MarketPlace\ProductRepository
provider: ~
finder: ~
listener: ~ # by default, listens to "insert", "update" and "delete"
user:
mappings:
firstName: { type: string, boost: 3}
lastName: { type: string, boost: 10 }
content: { type: string }
username: { type: string }
email: { type: string }
jobSeeker:
type: "nested"
properties:
skills:
type: "nested"
properties:
name: { type: string }
experiences:
type: "nested"
properties:
position:
type: "nested"
properties:
name: { type: string }
content: { type: string }
trainings:
type: "nested"
properties:
name: { type: string }
content: { type: string }
diploma:
type: "nested"
properties:
name: { type: string }
persistence:
driver: orm
model: AppBundle\Entity\User\User
repository: AppBundle\Repository\User\UserRepository
provider: ~
finder: ~
listener: ~ # by default, listens to "insert", "update" and "delete"
organization:
mappings:
name: { type: string, boost: 10}
legalName: { type: string, boost: 10}
shortDescription: { type: string, boost: 3}
route: { type: string}
content: { type: string }
persistence:
driver: orm
model: AppBundle\Entity\User\Organization
repository: AppBundle\Repository\User\OrganizationRepository
provider: ~
finder: ~
listener: ~ # by default, listens to "insert", "update" and "delete"
offer:
mappings:
name: { type: string, boost: 10}
content: { type: string }
responsibilities: { type: string }
skills:
type: "nested"
properties:
name: { type: string }
contractType:
type: "nested"
properties:
name: { type: string }
position:
type: "nested"
properties:
name: { type: string, boost: 10 }
persistence:
driver: orm
model: AppBundle\Entity\Job\Offer
repository: AppBundle\Repository\Job\OfferRepository
provider: ~
finder: ~
listener: ~
post:
mappings:
name: { type: string, boost: 10}
content: { type: string }
summary: { type: string }
tags:
type: "nested"
boost: 5
properties:
name: { type: string }
comments:
type: "nested"
properties:
content: { type: string }
persistence:
driver: orm
model: AppBundle\Entity\Social\Post
repository: AppBundle\Repository\Social\PostRepository
provider: ~
finder: ~
listener: ~
The query is a basic one :
$finder = $this->container->get('website.recipe')
$elasticaResults = $finder->find($search);
Each time you update your fos_elastica config file, you need in a first time to update the index(es) with the command:
php app/console fos:elastica:populate --index website --type= <<you type>>
This command will reindex your data with erasing previous one. In a second time, you should have to check your mapping information to check everything is fine and synchronous.
You could check your analyser with this command:
curl 'http://<< yourdomain >>:9200/website/_analyze?pretty=true&analyzer=custom_french_analyzer' -d "Cake"

Auto index child objects in fos elastica

I have created an elastic search mapping in config.yml which is using doctrine orm, the issue is that when I create a new document the index is auto populated for the top most object which is "documents" but not for any child objects such as "Folder, Authors, Comments ...", can someone suggest if I am missing any flag or any settings. Below is my Elastica configuration:
# Elastica Configuration
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
website:
client: default
index_name: docova
types:
documents:
mappings:
Doc_Title: ~
Description: ~
Date_Created: ~
Doc_Status: ~
Keywords: ~
folder:
type: "object"
properties:
id: ~
Library:
type: "object"
properties:
id: ~
Author:
type: "object"
properties:
username: ~
Comments:
type: "object"
properties:
comment: ~
Form_Values:
type: "object"
properties:
Field_Value: ~
Bookmarks:
type: "object"
properties:
Target_Folder:
type: "object"
properties:
id: ~
DocType:
type: "object"
properties:
id: ~
Doc_Name: ~
Attachments:
type: "object"
properties:
File_Name:
content:
type: attachment
persistence:
driver: orm
model: Docova\DocovaBundle\Entity\Documents
provider: ~
listener: ~
finder: ~
Thnx.
i've found a solution here.
Symfony2 FOSElasticaBundle update index for all entities related to the entity updated
Note : In my version of FOSElastica, Listener is not in the ORM folder but in FOS\ElasticaBundle\Doctrine and the argument has changed from LifecycleEventArgs $eventArgs to \Doctrine\Common\EventArgs $eventArgs.
EDIT : For this case, you have to override postPersist too.
public function postPersist(\Doctrine\Common\EventArgs $eventArgs)
{
$entity = $eventArgs->getEntity();
if ($entity instanceof $this->objectClass && $this->isObjectIndexable($entity)) {
$this->scheduledForInsertion[] = $entity;
$this->inicializaPost();
$this->objectPersisterPost->replaceOne($entity->getPost());
}
}

exclude fields in _source mapping with foqelasticabundle

I have attachment plugin for Elasticsearch to index all my file stored in Document. I would like to excludes the file content from being stored in the _source.
My config file look likes:
document:
mappings:
id: { index: not_analyzed }
path: {}
name: { boost: 5}
file:
type: attachment
store: "yes"
fields:
title: { store : "yes" }
file : {term_vector: "with_positions_offsets", store: yes}
analyzer: standard
boost: 2
persistence:
driver: orm
model: ACF\CaseBundle\Entity\Document
listener:
finder:
provider:
batch_size: 100
_source:
excludes:
file: ~
When I run foq:elastica:populate I still see the "file" attribute being stored in _source. I can not figure out what is missing. Please help
If anyone else comes across this problem, if you exclude your properties as follows it should just work:
_source:
excludes:
[ file ]

Resources