Elasticsearch - foselastica Sorting by integer having strange results - symfony

i want to sort my result of companies based on the count of nested (employment)objects,
i added an extra field to the company entity that holds the count like:
private employeeCount;
getEmployeeCount(){
return count($this->employments);
}
and added it to the index like:
company:
mappings:
fullname: ~
employeeCount: ~
the field is correctly indexed and i get hits like:
"_hit": {
"_index": "search",
"_type": "company",
"_id": "2628",
"_source": {
"fullname": "acme",
"employeeCount": 9,
... },
"sort": [
"9"
]
i added the sort like:
$query->addSort(array('employeeCount' => array( 'order'=>'desc')));
and the result seems to be sorted correctly down from " 9, 8, 7, 6 ...",
but for some reason there are some results somewhere in the middle with higher employeeCount
for example this:
"_hit": {
"_index": "search",
"_type": "company",
"_id": "2668",
"_source": {
"fullname": "acme2",
"employeeCount": 18,
... },
"sort": [
"18"
]
i expect this result to be on top of my first example but it is somewhere between 2 and 1
so two guesses, it is sorting from 0-10 and everything greater than 10 is ignored
or there is some bug in elasticsearch or foselastica bundle,
heres the resulting query:
search/company/_search (GET) 5.59 ms {"sort":[{"employeeCount":{"order":"desc"}}],"query":{"wildcard":{"fullname":{"value":"**","boost":1}}},"size":"2000","from":0}
anybody any idea ?

Seems like it is sorting as a string instead of an integer. You can add a type to the sort params to specify integer.

Related

Azure CosmosDB - Partial Document Update (Patch) on Complex Structures

I'm trying to use the Partial Document Update (Patch API) to update a child object in my document, but I'm running into trouble. I found this Stack Overflow question which is the same question that I have. However, the accepted answer resolves the problem by referring to an object in an array by index. I don't believe that I have the luxury of being able to do that. So, to use the same example document as the other question...
{
"id": "SalesOrder2",
"ponumber": "PO15428132599",
"OrderDate": "2005-07-01T00:00:00",
"DueDate": "2005-07-13T00:00:00",
"ShippedDate": "2005-07-08T00:00:00",
"AccountNumber": "Account2",
"SubTotal": 6107.082,
"TaxAmt": 586.1203,
"Freight": 183.1626,
"TotalDue": 4893.3929,
"DiscountAmt": 1982.872,
"Items": [
{
"Id": 1,
"OrderQty": 3,
"ProductCode": "A-123",
"ProductName": "Product 1",
"CurrencySymbol": "$",
"CurrencyCode": "USD",
"UnitPrice": 17.1,
"LineTotal": 5.7
},
{
"Id": 2,
"OrderQty": 2,
"ProductCode": "A-456",
"ProductName": "Product 2",
"CurrencySymbol": "$",
"CurrencyCode": "USD",
"UnitPrice": 10,
"LineTotal": 20
}
],
"_rid": "BsMkAMc43s4CAAAAAAAAAA==",
"_self": "dbs/BsMkAA==/colls/BsMkAMc43s4=/docs/BsMkAMc43s4CAAAAAAAAAA==/",
"_etag": "\"00000000-0000-0000-e136-0dbec04601d7\"",
"_attachments": "attachments/",
"_ts": 1637760030
}
I have no guarantee that the item in the Items array with an Id of 1 would be in position 0 of the array. Similarly, the item with an Id of 2 is not guaranteed to be in position 1. Therefore I believe that I need to use the FilterPredicate parameter of the Patch API to filter my results. But when I attempt to do that, I keep getting the following exception:
Microsoft.Azure.Cosmos.CosmosException : Response status code does not
indicate success: PreconditionFailed (412); Substatus: 1110;
ActivityId: dbd258ae-0a0a-4a9b-8c25-1d36e137b7c5; Reason: ();
Any assistance you could provide on how to accomplish this would be appreciated.
As i answered in the attached link, Patch requires the user to pass the specific index of the object needs to be updated. We are working on enabling this particular feature in the coming months, However as an alternative, you should look at Conditional Patch
Code will be something like this,
response = patch(operation, Condition(check if item exists))
if(response == fail/precondition failed)
{
PatchOperation operation = PatchOperation.Add("/Items", [{"Id" : "P-1", "Description" : "My Product"}]);
}

Merging list of maps in gremlin

I have this relationship:
person --likes--> subject
This is my query:
g.V().
hasLabel('person').
has('name', 'Joe').
outE('likes').
range(0, 2).
union(identity(), inV().hasLabel('subject')).
valueMap('rating', 'name').
At this point, I get result that looks like this:
[
{
"rating": 3.236155563
},
{
"rating": 3.162886797
},
{
"name": "math"
},
{
"name": "history"
}
]
I'd like to get something like this:
[
{
"rating": 3.236155563,
"name": "math"
},
{
"rating": 3.162886797,
"name": "history"
},
]
I've tried grouping the results - which gives me the structure I want - but because of the identical keys, I only get 1 set of results back.
It always helps when you post the code to create the graph so we can give you a tested answer. Like so
g.addV('person').property('name', 'P1').as('p1').
addV('subject').property('name', 'Math').as('math').
addV('subject').property('name', 'History').as('history').
addV('subject').property('name', 'Geography').as('geography').
addE('likes').from('p1').to('math').property('rating', 1.2).
addE('likes').from('p1').to('history').property('rating', 2.3).
addE('likes').from('p1').to('geography').property('rating', 3.4)
I believe you are trying to write a traversal that starts from a certain person, go out along the first two "likes" edges and get the names of the subjects that he likes and the rating on the corresponding "likes" edge.
g.V().has('person', 'name', 'P1').
outE('likes').
range(0, 2).
project('SubjectName', 'Rating').
by(inV().values('name')).
by(values('rating'))

Serialized Entities displaying only ID

I'm using JMSSerializer and FOSRestBundle. I have a fairly typical object graph, including some recursion.
What I would like to accomplish is that included objects beyond a certain depth or in general are listed only with their ID, but when serialized directly, with all data.
So, for example:
Users => Groups => Users
when requesting /user/1 the result should be something like
{ "id": 1, "name": "John Doe", "groups": [ { "id": 10 }, { "id": 11 } ] }
While when I request /group/10 it would be:
{ "id": 10, "name": "Groupies", "users": [ { "id": 1 }, { "id": 2 }, { "id": 4 } ] }
With #MaxDeph I can hide the included arrays completely, so I get
{ "id": 1, "name": "John Doe", "groups": [] }
But I would like to include just the IDs so that the REST client can fetch them if it needs them, or consult his cache, or do whatever.
I know I can manually cobble this together using groups, but for consistency reasons I was wondering if I can somehow enable this behaviour in my entire application, maybe even with a reference to maxdepth so I can control where to include IDs and where to include full objects?
For the sake of those finding this:
I found no other solution, but doing this with groups works just fine and gives me the result I was looking for.

kibana histogram value field use string?

i have a task..
the Values field of the Histogram must be contain a numeric value
but.. my value field come from string ,like this
traceMsg: api_name=xxx&pageSize=20
and i need the 20 as value field..
{
"_index": "tracelog",
"_type": "tracelog",
"_id": "TS7s-gkUSsiH7gqnkHOjpA",
"_score": null,
"_source": {
"id": null,
"host": "10.153.192.105",
"appId": "32",
"spanIndex": 1,
"spanLevel": 1,
"spanFlag": 0,
"spanId": "1",
"parentSpanId": "0",
"taobaoId": 1659018141,
"traceId": "e7e2e60a6ade4f4ab23df4994b2392ae",
"traceLoc": "http",
"traceType": 0,
"traceUrl": "/trade/search",
"traceMsg": "api_name=trade_search&queryId=1&sys=true&timeType=created&field=created&order=asc&pageNo=1&pageSize=20",
"payTime": 1173,
"logTime": "2014-11-13T16:16:43+0800"
},
"sort": [
1415866603000
]
}
i don't know how to do. :)
This is what logstash is for. Check out the grok filter, which would let you parse the traceMsg field and make your own field for the pageSize value.
How do you insert data on elasticsearch without using logstash? You can use the mutate filter to change the field type.

Freebase MQL must sort on a single value

I'm trying to learn to use Freebase, however when I try and do a sort by "/people/person/date_of_birth" for a search for actors for a show, it returns:
"code": 400,
"message": "Must sort on a single value, not at /tv/tv_program/regular_cast./tv/regular_tv_appearance/actor./people/person/date_of_birth"
Here is the full MQL query:
[{
"id": "/m/0524b41",
"name": [],
"sort":"/tv/tv_program/regular_cast./tv/regular_tv_appearance/actor./people/person/date_of_birth",
"/tv/tv_program/regular_cast": [{
"/tv/regular_tv_appearance/actor": [{
"name": [],
"/people/person/date_of_birth": []
}]
}]
}]
But you're not asking to sort on /people/person/date_of_birth, you're asking to sort on that long nested expression which goes through multiple intermediary nodes, some of which can appear multiple times (as indicated by the [] array notation). It's this multiplicity that MQL is complaining about.
To fix it, take your query, paste it into the query editor, click on the innermost /person/date_of_birth and then click "Invert Query." That will turn the query inside out and give you something that looks like this:
[{
"name": [],
"/people/person/date_of_birth": [],
"type": "/tv/tv_actor",
"!/tv/regular_tv_appearance/actor": [{
"!/tv/tv_program/regular_cast": [{
"id": "/m/0524b41",
"name": [],
"sort": "/tv/tv_program/regular_cast./tv/regular_tv_appearance/actor./people/person/date_of_birth"
}]
}]
}]
which isn't exactly what you want, but indicates the general shape of your target query.
Getting rid of the array brackets for single valued properties and moving the sort clause to the outside gives us:
[{
"name": null,
"/people/person/date_of_birth": null,
"sort": "/people/person/date_of_birth",
"type": "/tv/tv_actor",
"!/tv/regular_tv_appearance/actor": [{
"!/tv/tv_program/regular_cast": [{
"id": "/m/0524b41",
"name": null
}]
}]
}]
which is functional and returns our 81 regular Game of Thrones actors sorted by birth date, but could still be cleaned up a bit more. The !inverse property notation isn't necessary since we have forward equivalents and we don't really need to get the Game of Thrones info over and over again since it's constant and we really just want to use it as a filter.
Incorporating these final tweaks gives us a final query like this which returns nice compact results:
[{
"name": null,
"/people/person/date_of_birth": null,
"sort": "/people/person/date_of_birth",
"type": "/tv/tv_actor",
"starring_roles": [{
"series": {
"id": "/m/0524b41"
},
"limit": 0
}]
}]
The "limit": 0 clause is a little trick to cause MQL to use that subquery for filtering, but not bother returning any of the (constant) information in the results. The /tv/tv_actor/starring_roles and /tv/regular_tv_appearance/series can be abbreviated to the simple property names because their types are implied by their context.
Since there are only 81 results, MQL's default limit of 100 is plenty and we don't need to worry about increasing it or using cursors.
Oldest Game of Thrones actor: Peter Vaughn, born 1923.
Youngest: Lino Facioli b. 2000
Note that 7 actors don't have birth dates in Freebase, so we don't know where they rank age-wise. Here's a bonus query which returns their names and ids as well as their character's name. If we were running a production system, we might use something like this to feed a human curation queue to fill in the gaps.
[{
"name": null,
"/people/person/date_of_birth": {
"value": null,
"optional": "forbidden"
},
"type": "/tv/tv_actor",
"starring_roles": [{
"series": {
"id": "/m/0524b41"
},
"character":null
}]
}]
The seven character/actor pairs are (were): Roose Bolton - Michael McElhatton,
Gregor Clegane - Conan Stevens,
Hizdahr zo Loraq - Joel Fry,
Rickon Stark - Art Parkinson,
Janos Slynt - Dominic Carter,
Hodor - Kristian Nairn,
Tommen Baratheon - Callum Wharry. I say "were" because I couldn't resist fixing Hodor's birth date. The strange thing is that it was in Wikipedia, so should have been picked up automatically by Freebase. I think there's a bug lurking there somewhere.

Resources