I am on my way to figure out, how Elastica/FOSElastica is working. I got it running fine for like really simple search queries, where the product name exactly match my search string.
Since this is not a good way to search, I would have to add analyzers to perform better search results. But they are not working..
I tried to just copy&paste this example code (inserted my index/types of course) => http://obtao.com/blog/2013/10/configure-elasticsearch-on-an-efficient-way/#comment-16733
So, here is my question: How do I get my analyzers working? I guess, I have to apply them in my controller, but I am not entirely sure how I would do that...
This is my config
#app/config/config.yml
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
search:
client: default
settings:
index:
analysis:
analyzer:
custom_analyzer :
type : custom
tokenizer: nGram
filter : [stopwords, asciifolding ,lowercase, snowball, elision, worddelimiter]
custom_search_analyzer :
type : custom
tokenizer: standard
filter : [stopwords, asciifolding ,lowercase, snowball, elision, worddelimiter]
my_analyzer _:
type: snowball
language: German
tokenizer:
nGram:
type: nGram
min_gram: 2
max_gram: 20
filter:
snowball:
type: snowball
language: German
elision:
type: elision
articles: [l, m, t, qu, n, s, j, d]
stopwords:
type: stop
stopwords: [_german_]
ignore_case : true
worddelimiter :
type: word_delimiter
types:
article :
mappings:
articleNumber:
shortName: [ boost: 6, analyzer : custom_analyzer, custom_search_analyzer, my_analyzer]
shortDescription:
index_analyzer : custom_analyzer
search_analyzer : custom_search_analyzer
longDescription:
index_analyzer : custom_analyzer
search_analyzer : custom_search_analyzer
persistence:
driver: orm
model: IndexBundle\Entity\Articles
finder: ~
provider: ~
listener: ~
This is my controller:
$query = new \Elastica\Query\Match();
$query->setFieldQuery(‘shortName’, $searchTerm);
$query->setFieldFuzziness(‘shortName’, 0.7);
$query->setFieldMinimumShouldMatch(‘shortName’, ’80%’);
$boolQuery = new \Elastica\Query\Bool();
$boolQuery->addMust($query);
$baseQuery = $query;
$filtered = new \Elastica\Query\Filtered($baseQuery);
$query = \Elastica\Query::create($filtered);
$articles = $finder->find($query);
Related
My Web API method returns a binary PDF file in the response body and I've set this in the swagger file in the Custom Connector of Power Automate. However, despite the output being a binary PDF file AND that the property type is "file", the Test function in the Custom Connector UI still thinks its returning a "string" even through it clearly isn't!
Question: Any ideas what I'm doing wrong to make the tester think it's a string?
Here is a screenshot of the issue:
Here is my swagger for the Web API method (take note of the 200 response of schema: {type: file, format: binary}:
swagger: '2.0'
info: {title: xxxx, version: '1.0', description: xxxx}
host: xxxx.azurewebsites.net
basePath: /api
schemes: [https]
consumes: []
produces: []
paths:
/FrontPageGenerator:
post:
description: xxxx
operationId: FrontPageGenerator
summary: FrontPageGenerator
parameters:
- name: body
in: body
required: true
schema: {type: string}
consumes: [application/json]
produces: [application/json, application/pdf]
responses:
'200':
description: Successfully generated composite PDF file
schema: {type: file, format: binary}
'400':
description: Invalid input
schema: {$ref: '#/definitions/GenericJson'}
examples:
application/json: {message: No body}
'406':
description: Not Acceptable Accept Type
schema: {$ref: '#/definitions/BasicError'}
examples:
application/json: {message: Not Acceptable}
definitions:
BasicError:
type: object
properties:
message: {type: string}
GenericJson: {type: object}
I'm writing a service with a GET that can return one of five different but closely related types. Since the user wants the option of searching through all five types at once, it has to be a single get call. I'm returning JSON, which can easily handle any type.
I'm trying to do this in Swagger, using their polymorphism feature, which I've never tried before. I'm doing it just like in the example, except under "definitions" instead of "components/schemas". But I'm getting a strange error message that I can't understand. The swagger file is below. The error says this:
Schema error at definitions['Event'].discriminator should be string
It gives this on line 49, which says discriminator:
So, my two questions are: How can I fix it? And will this even give me what I need?
swagger: '2.0'
info:
description: RESTful API to retrieve Titles Metadata
version: 1.0.0
title: Swagger Mystery
schemes:
- https
paths:
/event:
get:
operationId: getEvent
summary: searches names
description: |
Search by names, across all types, or by a specific type.
produces:
- application/json
parameters:
- in: query
name: title
description: name to search for
required: true
type: string
- in: query
name: start
required: false
type: boolean
- in: query
name: type
required: false
type: string
description: |
May be "contest", "partner", "sponsor", or "dancer". If missing, will search for all types.
responses:
'200':
description: search results
# I also don't know why I need to comment these out.
# content:
# application/json:
# schema:
# type: array
# items:
# $ref: '#/definitions/Event'
'400':
description: bad input parameter
definitions:
Event:
type: object
discriminator:
propertyName: eventType
properties:
eventType:
type: string
id:
type: integer
format: int64
name:
type: string
description:
type: string
contests:
type: array
items:
$ref: '#/definitions/Contest'
required:
- id
- name
- description
- contests
- eventType
Contest:
allOf:
- $ref: '#/definitions/Event'
- type: object
properties:
parentEvent:
type: string
venue:
type: string
required:
- parentEvent
- venue
Dancer:
allOf:
- $ref: '#/definitions/Event'
- type: object
properties:
eventInvitationDate:
type: string
format: date
venue:
type: string
required:
- eventInvitationDate
- venue
# Sponsor:
# allOf:
# - $ref: '#/definitions/Event'
# - type: object
# properties:
# invitationDate:
# type: string
# format: date
# parentEvent:
# type: string
# partners:
# type: array
# items:
# $ref: '#/definitions/Partner'
Partner:
allOf:
- $ref: '#/definitions/Event'
- type: object
properties:
invitationDate:
type: string
format: date
parentEvent:
type: string
venue:
type: string
required:
- invitationDate
- parentEvent
- venue
# two problems:
# 1. Schema error at definitions['Event'].discriminator
# should be string on line 49 (discriminator:)
# 2. Resolver error:
# e is undefined
# (no line number)
# (This error goes away when I comment out Sponsor.)
The error occurs because you are mixing OpenAPI 2.0 and 3.0 syntax.
Your spec is swagger: '2.0' but the following is 3.0 syntax:
discriminator:
propertyName: eventType
In OpenAPI 2.0, the value of discriminator is the property name:
discriminator: eventType
Also, OpenAPI 2.0 assumes that the possible values of the discriminator property (in this case eventType) are exactly the same as the model names in definitions. That is:
If eventType can be event, partner etc. in lowercase, then the model names must also be in lowercase – event, not Event.
If eventType is some code like e, p, d etc., the model names must be e, p, d etc. instead of Event, Partner etc.
Check out questions for more examples of discriminator usage in OpenAPI 2.0:
Swagger Inheritance and Composition
“discriminator” in polymorphism, OpenAPI 2.0 (Swagger 2.0)
Swagger: variant schema shape dependant on field value
I have an application where the users (clients) can activate different modules.
On the search, the user should be able to search a value in the entire index (for the modules he has activated).
The problem is that I also need to boost some field.
Currently I have this.
mapping.yml
indexes:
traveler:
client: default
finder: ~
types:
Country:
mappings:
name: { boost: 10 }
code:
# additional info (irrelevant)
persistence:
driver: orm
model: CoreBundle\Entity\Country
provider: ~
listener: ~
finder: ~
serializer:
groups: [Default, elastica]
Places:
mappings:
name:
address:
type: "object"
properties:
city: ~
region: ~
country: ~
persistence:
driver: orm
model: CoreBundle\Entity\Places
provider: ~
listener: ~
finder: ~
serializer:
groups: [Default, elastica]
And in my service I have this :
$index = $this->get('fos_elastica.finder.traveler');
$query = new \Elastica\Query\QueryString($search);
# if the client has the module activated
foreach ($this->getClient()->getServices() as $service) {
switch ($service->getService()->getTag()) {
case "countries":
$filters->addShould(
new Type('Country')
);
break;
case "places":
$filters->addShould(
new Type('Places')
);
break;
}
}
$query = new \Elastica\Query\Filtered($query, $filters);
// set result limit
$globalQuery = new Query();
$globalQuery->setQuery($query);
$globalQuery->setSize($limit);
// return result
return $index->find($query);
Now, for example, if I search "Germany", the "Country" result should be first in the result set, because I applied a boost to it in the yml mapping.
Instead, the first 5 or 6 results are for "Places" which have in their country address "Germany".
The boost is not applied (I tried with higher values, removing it ... results are not changed).
I found a way where it works, if I change my service like this, but I would prefer that it use the boost property defined in the mapping.
$query = new QueryString($search);
$query->setFields(array(
'_all',
'Country.name^10'
));
Am I doing something wrong ?
Is there another way to do this ?
Thanks !
I use FOSElasticaBundle in my Symfony 2 project. Since today reindexing is resulting in the below error:
index: /app/hotel/1 caused MapperParsingException[failed to parse
[priceFrom]]; nested: NumberFormatException[For input string:
"410.00"];
In my doctrine orm yml the priceFrom field is defined as followed:
priceFrom:
type: decimal
nullable: true
precision: 7
scale: 2
comment: ''
column: price_from
My fos_elastica config looks like this (config.yml):
fos_elastica:
clients:
default: { host: localhost, port: 9200 }
indexes:
app:
types:
hotel:
mappings:
id: ~
active: ~
priceFrom: { type: integer }
persistence:
driver: orm
model: XXX\XXXBundle\Entity\Hotel
provider: ~
listener:
immediate: ~
finder: ~
The command I use to reindex: php app/console fos:elastica:populate
The above setup has worked until now. I hope someone can point my to the good direction to solve this problem.
Versions:
ruflin/elastica (2.1.0)
friendsofsymfony/elastica-bundle (v3.1.5)
symfony/symfony (v2.6.11)
PS: No other entities in my project are using a priceFrom field.
In mappings, you define PriceFrom as integer but then you pass a decimal.
I haven't tested it but it definitely seems the major candidate as the culprit.
Francesco Abeni is right with answer. If you are already pushed something in ES as integer (or ES defined it as integer) it will generate exception when you will try to save decimal data here.
I always explicitly specify type in mapping like:
id: {"type" : "integer"}
shop_id: {"type" : "integer"}
source: {"type" : "string", "index" : "not_analyzed"}
There I see two ways to solve problem.
index alias and index merge
specify type in mapping; kill index; populate in again
I used second variant on a dev :)
I’m trying to use Elasticsearch to index a city database and get autocomplete on a field.
Here is my FOSElasticaBundle configuration :
fos_elastica:
indexes:
xxxxxxx:
settings:
index:
analysis:
analyzer:
custom_analyzer:
type: custom
tokenizer: nGram
filter: [lowercase, asciifolding, stopwords_fr, elision, snowball_fr, word_delimiter]
custom_search_analyzer:
type: custom
tokenizer: standard
filter: [lowercase, asciifolding, stopwords_fr, elision, snowball_fr, word_delimiter]
tokenizer:
nGram:
type: nGram
min_gram: 4
max_gram: 20
filter:
snowball_fr:
type: snowball
language: French
elision:
type: elision
articles: [l, m, t, qu, n, s, j, d]
stopwords_fr:
type: stop
stopwords: [_french_]
ignore_case: true
types:
cities:
mappings:
id:
name: { search_analyzer: custom_analyzer, index_analyzer: custom_analyzer, type: string, store: yes }
persistence:
driver: orm
model: XXXXX\MainBundle\Entity\City
provider: { query_builder_method: createSearchIndexQueryBuilder }
listener: ~
finder: ~
repository: XXXXX\SearchBundle\Repository\CitySearchRepository
My query looks like this :
{
"query": {
"match": {
"name": "xxxxxx"
}
}
}
But my problem is here :
When I type “Pa”, I get 242 results (ok)
When I type “Par”, I get no results (WTF)
When I type “Pari”, I get 22 results (ok)
When I type “Paris”, I get 10 results (ok)
It’s my first time with Elasticsearch, I think the solution is nearby, but if somebody has the same issue, I’m curious to know more about it.
Thanks, have a good day ;)
In your query you are doing a match. It means that you will look for documents that have the (analyzed) value you are looking for.
You should have a look on the completion suggester, it is the best way to implement suggestions for autocomplete.
If you are lazy you can just use a prefix query:
{
"prefix" : { "user" : "Pa" }
}