I have attachment plugin for Elasticsearch to index all my file stored in Document. I would like to excludes the file content from being stored in the _source.
My config file look likes:
document:
mappings:
id: { index: not_analyzed }
path: {}
name: { boost: 5}
file:
type: attachment
store: "yes"
fields:
title: { store : "yes" }
file : {term_vector: "with_positions_offsets", store: yes}
analyzer: standard
boost: 2
persistence:
driver: orm
model: ACF\CaseBundle\Entity\Document
listener:
finder:
provider:
batch_size: 100
_source:
excludes:
file: ~
When I run foq:elastica:populate I still see the "file" attribute being stored in _source. I can not figure out what is missing. Please help
If anyone else comes across this problem, if you exclude your properties as follows it should just work:
_source:
excludes:
[ file ]
Related
I am able to create indexes 1 by 1 but my purpose is to have 1 big index with subindexes to search. (I am reproducing an existing app but upgrading dependencies)
Symfony : 5.4*
FOS-Elastica : 6.1
Elastic : 7*
This is my error message and if I change types to properties I have similar errors. I have been trying to switch and indent differently all day:
Unrecognized option "types" under "fos_elastica.indexes.app". Available options are "_id", "_routing", "_source", "analyzer", "client", "date_detection", "dynamic", "dynamic_date_formats", "dynamic_templates", "finder", "index_
name", "index_prototype", "indexable_callback", "numeric_detection", "persistence", "properties", "serializer", "settings", "use_alias".
What am I doing wrong please ?
#app/config/config.yml
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
app:
settings:
analysis:
analyzer:
fr:
tokenizer: standard
filter: [ lowercase, stop_fr, snowball_fr ]
autocomplete:
type: custom
tokenizer: whitespace
filter: [ lowercase, engram, elision ]
csv:
type: pattern
pattern: '\s*,\s*'
lowercase: false
filter:
snowball_fr:
type: "snowball"
language: "French"
stop_fr:
type: "stop"
stopwords: "_french_"
engram:
type: edge_ngram
min_gram: 2
max_gram: 15
types:
# ---------
# USER
# ---------
user:
properties:
username: ~
email: ~
organization:
type: object
properties:
id: { index: true }
code: { index: true }
name:
index: true
type: text
fields:
source: { type: text, index: true }
persistence:
driver: orm # orm, mongodb, propel are available
model: App\Entity\User
provider: ~
listener: ~
finder: ~
So I was also figuring out how to deal with the same issue as yours, and I do confirm lot of the docs and blogposts out there use the config struture you posted. However when I went back to the v6.1 documentation of the bundle I found this:
Dynamic templates
Dynamic templates allow to define mapping templates
that will be applied when dynamic introduction of fields / objects
happens.
Documentation
fos_elastica:
indexes:
user:
dynamic_templates:
my_template_1:
match: apples_*
mapping:
type: float
my_template_2:
match: *
match_mapping_type: text
mapping:
type: keyword
properties:
username: { type: text }
So in your case a working config would look like:
#app/config/config.yml
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
user:
settings:
analysis:
analyzer:
fr:
tokenizer: standard
filter: [ lowercase, stop_fr, snowball_fr ]
autocomplete:
type: custom
tokenizer: whitespace
filter: [ lowercase, engram, elision ]
csv:
type: pattern
pattern: '\s*,\s*'
lowercase: false
filter:
snowball_fr:
type: "snowball"
language: "French"
stop_fr:
type: "stop"
stopwords: "_french_"
engram:
type: edge_ngram
min_gram: 2
max_gram: 15
properties:
username: ~
email: ~
organization:
type: object
properties:
id: { index: true }
code: { index: true }
name:
index: true
type: text
fields:
source: { type: text, index: true }
persistence:
driver: orm # orm, mongodb, propel are available
model: App\Entity\User
provider: ~
listener: ~
finder: ~
In Symfony I try to merge a block of YAML from parameters.yml to config.yml. My question is how I can store some configurations in Symfony 3.4 and then insert into my config file. For now I got an error:
In Parser.php line 290:
Reference "'%insurance%'" does not exist at line 188 (near "<<:
*'%insurance%'").
parameters.yml
parameters:
Insurance: &insurance
list:
title: '<strong>Ubezpieczenia</strong>'
sort: ['sequence', 'ASC']
fields:
- { property: 'sequence', label: 'Kolejność'}
- { property: 'title', label: 'Tytuł'}
- { property: 'description', label: 'Opis'}
form:
fields:
- { property: 'title', type: 'text', label: 'Tytuł'}
- { property: 'description', type: 'ckeditor', label: 'Opis',
type_options: { config_name: 'simple_config' }}
- { property: 'sequence', type: 'integer', label: 'Kolejność'}
config.yml
imports:
- { resource: parameters.yml }
easy_admin:
[...]
entities:
[...]
MotorInsurance:
class: AppBundle\Entity\MotorInsurance
label: menu.motorInsurance
<<: *'%insurance%'
[...] there are non-related configurations
Am I did something wrong when I am calling Inusrance block?
So I try again and #xabbuh was right. I forgot that I have also block for parameters in my config.yml. My file now look like this:
parameters:
locale: pl
Insurance: &insurance
list:
title: '<strong>Ubezpieczenia</strong>'
sort: ['sequence', 'ASC']
fields:
- { property: 'sequence', label: 'Kolejność'}
- { property: 'title', label: 'Tytuł'}
- { property: 'description', label: 'Opis'}
form:
fields:
- { property: 'title', type: 'text', label: 'Tytuł'}
- { property: 'description', type: 'ckeditor', label: 'Opis',
type_options: { config_name: 'simple_config' }}
- { property: 'sequence', type: 'integer', label: 'Kolejność'}
easy_admin:
[...]
entities:
[...]
MotorInsurance:
class: AppBundle\Entity\MotorInsurance
label: menu.motorInsurance
<<: *insurance
And this works fine :) also everything can be overriden for your minds.
This does not work. Different files are parsed independently so you cannot reuse a reference defined in one YAML file in another YAML file.
The solution would be to place all the config that is required in the same file.
I use symfony with fos elastica bundle but i have some problems
I have an entity content with link with other entity categorie (with name and slugname) x content have only 1 category.
I have install the bundle and configure the yml files like this
fos_elastica:
clients:
default: { host: 127.0.0.1, port: 9200 }
serializer:
callback_class: FOS\ElasticaBundle\Serializer\Callback
serializer: serializer
indexes:
portail:
client: default
settings:
index:
analysis:
analyzer:
custom_analyzer :
type: custom
tokenizer: nGram
char_filter: [html_strip]
filter: [stopwords, lowercase, snowball, elision]
custom_analyzer_simple :
type: custom
tokenizer: whitespace
filter: [lowercase]
custom_search_analyzer :
type: custom
char_filter: [html_strip]
tokenizer: whitespace
filter: [stopwords, lowercase, snowball, elision]
custom_search_analyzer_simple :
type: custom
tokenizer: whitespace
filter: [lowercase]
tokenizer:
nGram:
type: nGram
min_gram: 4
max_gram: 20
filter:
snowball:
type: snowball
language: French
elision:
type: elision
articles: [l, m, t, qu, n, s, j, d]
stopwords:
type: stop
stopwords: [_french_]
ignore_case : true
worddelimiter :
type: word_delimiter
types:
Content:
mappings:
titre:
type: string
index_analyzer: custom_analyzer
search_analyzer : custom_search_analyzer
preview:
type: string
index_analyzer: custom_analyzer
search_analyzer : custom_search_analyzer
contenu:
type: string
index_analyzer: custom_analyzer
search_analyzer : custom_search_analyzer
categorie:
type: object
properties:
slugnom:
type: string
index: not_analyzed
search_analyzer : custom_search_analyzer_simple
persistence:
driver: orm
model: PO\EtablissementBundle\Entity\Content
provider: ~
listener:
logger: true
finder: ~
Page:
mappings:
titre:
type: string
index_analyzer: custom_analyzer
search_analyzer : custom_search_analyzer
persistence:
driver: orm
model: PO\EtablissementBundle\Entity\PageEtablissement
provider: ~
listener:
logger: true
finder: ~
Etablissement:
mappings:
nom:
type: string
index_analyzer: custom_analyzer
search_analyzer : custom_search_analyzer
contents:
type: object
categories:
type: nested
properties:
slugnom:
type: string
index: not_analyzed
search_analyzer : custom_search_analyzer_simple
sous_categories:
type: nested
properties:
slugnom:
type: string
index: not_analyzed
persistence:
driver: orm
model: PO\EtablissementBundle\Entity\Etablissement
provider: ~
listener:
logger: true
finder: ~
I want to have some flexibility but on the content categorie slugname i want to search exactly the word for example arts-de-la-rue but when i make a search i found result with categorie = arts or categorie = lecture, i don't understand the problem :(
If somebody can help me :)
You have to setup your mapping for that field
your field: { type: "string", index: "not_analyzed" }
And just to be sure destroy the type and populate it again.
You need to setup an keyword based analyzer for that.
The ngram on your analyzer will destroy any slug.
It is possible to setup multiple different analyzers per field, but I dont know if it is supported by the FOS\ElasticaBundle: https://www.elastic.co/guide/en/elasticsearch/reference/current/multi-fields.html
Later you can do a "filtered" query (term filter) against this new raw field.
Thanks for your help i have done some mistake in my configuration. I have put a type nested and in fact it's objet and the name of the properties are not good.
Now it's working perfectly :)
I have installed FOSElasticaBundle, my config fos_elastica.yml is as follows:
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
test:
client: default
index_name : %elastica_index_name%
types:
exhibition:
mappings:
name : { boost: 100 }
company: { boost: 9 }
directions: { boost: 8}
tags: { boost: 7 }
description : { boost: 6 }
status : ~
persistence:
driver: orm
model: Test\MainBundle\Entity\Exhibition
finder: ~
provider: ~
listener:
is_indexable_callback: "isPublic"
On persist is_indexable_callback works. But when I run fos:elastica:populate, the index is created even for hidden objects.
How can I index just active object? Maybe using analyzer or something like that?
Thanks in advance!
You should specify:
provider:
query_builder_method: createSearchIndexQueryBuilder
and then add method into related Entity\Repository:
public function createSearchIndexQueryBuilder()
{
$qb = $this->createQueryBuilder()
//->write_magic_here;
return $qb;
}
I have created an elastic search mapping in config.yml which is using doctrine orm, the issue is that when I create a new document the index is auto populated for the top most object which is "documents" but not for any child objects such as "Folder, Authors, Comments ...", can someone suggest if I am missing any flag or any settings. Below is my Elastica configuration:
# Elastica Configuration
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
website:
client: default
index_name: docova
types:
documents:
mappings:
Doc_Title: ~
Description: ~
Date_Created: ~
Doc_Status: ~
Keywords: ~
folder:
type: "object"
properties:
id: ~
Library:
type: "object"
properties:
id: ~
Author:
type: "object"
properties:
username: ~
Comments:
type: "object"
properties:
comment: ~
Form_Values:
type: "object"
properties:
Field_Value: ~
Bookmarks:
type: "object"
properties:
Target_Folder:
type: "object"
properties:
id: ~
DocType:
type: "object"
properties:
id: ~
Doc_Name: ~
Attachments:
type: "object"
properties:
File_Name:
content:
type: attachment
persistence:
driver: orm
model: Docova\DocovaBundle\Entity\Documents
provider: ~
listener: ~
finder: ~
Thnx.
i've found a solution here.
Symfony2 FOSElasticaBundle update index for all entities related to the entity updated
Note : In my version of FOSElastica, Listener is not in the ORM folder but in FOS\ElasticaBundle\Doctrine and the argument has changed from LifecycleEventArgs $eventArgs to \Doctrine\Common\EventArgs $eventArgs.
EDIT : For this case, you have to override postPersist too.
public function postPersist(\Doctrine\Common\EventArgs $eventArgs)
{
$entity = $eventArgs->getEntity();
if ($entity instanceof $this->objectClass && $this->isObjectIndexable($entity)) {
$this->scheduledForInsertion[] = $entity;
$this->inicializaPost();
$this->objectPersisterPost->replaceOne($entity->getPost());
}
}