I am trying to setup Kibana dashboard.
In Elasticsearch, there is one type of document only.
We have two indices with the same mapping:
Indices product_1 and product_2:
"_index": "product_1",
"_type": "product",
"_id": "3da33451d10f095c4b8cd485133dc23639244538",
"_score": 1,
"_source": {
...
source": { "brand": "","category": "","store": "Store1"}
...
}
"_index": "product_1",
"_type": "product",
"_id": "3da33451d10f095c4b8cd485133dc23639244538",
"_score": 1,
"_source": {
...
source": { "brand": "","category": "","store": "Store2"}
...
}
...
We have managed to create follow visualization.
We need to count total quantity of products (documents) per field store:
Top 0 source.store Count
Store1 52,517
Store2 31,517
Store3 12,838
...
So, question is: How do we add data from second index here, in order to get result like that:
Top 0 source.store Count (product_1) Count (product_2)
Store1 52,517 42,517
Store2 31,517 56,517
Store3 12,838 13,890
...
Thanks a lot.
PS: We have managed to collect data from 2 indices:
["product_1","product_2"]
So kibana sources data, as if it is one index, and we get follow wrong result:
Top 0 source.store Count (product_1+product_2)
Store1 102,517
Store2 62,517
Store3 24,838
...
I am not entirely sure if what you are trying is even possible. From the elastic search index documentation:
The easiest and most familiar layout clones what you would expect from a relational database.
You can (very roughly) think of an index like a database.
MySQL => Databases => Tables => Columns/Rows
Elasticsearch => Indices => Types => Documents with Properties
And as I understand here, you are trying to get data from two databases(indices) in a single query, which to the best of my knowledge, is not possible - at least the way you want it, i.e. assort results according to indices.
Now you might (should) have a question that if that is entirely impossible, how did you get the following results?
Top 0 source.store Count (product_1+product_2)
Store1 102,517
Store2 62,517
Store3 24,838
This is where the Kibana magic lies. You can create an index pattern with a wild card. That will query every index that falls within the card. For example creating logstash-* will query all the logstash-[date] indices and give out consolidated results that you see. In your case it can be achieved by creating an index pattern as product_*. But I think you cannot get index wise assorted results with that (which I suppose, is exactly what you're looking for).
Related
I have a list in a DynamoDB table and would like to move items to different positions in the same list, is there a way to do this in a single update?
At the moment, I'm looking at having to read the list, modify it, then write it back again, but would prefer doing it all in a single update, is there a way to do this?
Edit to add example
So here's some noddy data that shows what I'd like to do:
If the data started like this:
Item: { COLUMN: [ "Element_0", "Element_1", "Element_2", "Element_3" ] }
Then I'd give it from and to indices and it would move the element. So for example if I gave it a from index of 0 and to index of 2 the data should end up like this:
Item: { COLUMN: [ "Element_1", "Element_2", "Element_0", "Element_3" ] }
You can do this with an Update Expression, but it's a little tricky, since you don't have the data.
Basically, you have to create a dynamic update statement that sets every value you want to move. Something like this works:
aws dynamodb update-item --table-name test --key '{"pk":{"S":"1"}}' --update-expression "SET #list[1] = #list[2], #list[2] = #list[1]" --region us-west-2 --profile jw-test --expression-attribute-names '{"#list": "list"}'
I created a table with a key of pk, with a value of 1. The list before the update was like this:
[
'one',
'two',
'three',
'four'
]
After the update it looks like this:
[
'one',
'three',
'two',
'four'
]
Default answer if this isn't possible in a single update.
Read out the list, modify it, then write it back. It's not elegant, but it works and isn't that ugly either.
It's not atomic though so any answer that can do it in a single update will get the check mark.
I'm new to DynamoDB and I'm trying to query a table from javascript using the Dynamoose library. I have a table with a primary partition key of type String called "id" which is basically a long string with a user id. I have a second column in the table called "attributes" which is a DynamoDB map and is used to store arbitrary user attributes (I can't change the schema as this is how a predefined persistence adapter works and I'm stuck working with it for convenience).
This is an example of a record in the table:
Item{2}
attributes Map{2}
10 Number: 2
11 Number: 4
12 Number: 6
13 Number: 8
id String: YVVVNIL5CB5WXITFTV3JFUBO2IP2C33BY
The numeric fields, such as the "12" field, in the Map can be interpreted as "week10", "week11","week12" and "week13" and the numeric values 2,4,6 and 8 are the number of times the application was launched that week.
What I need to do is get all user ids of the records that have more than 4 launches in a specific week (eg week 12) and I also need to get the list of user ids with a sum of 20 launches in a range of four weeks (eg. from week 10 to 13).
With Dynamoose I have to use the following model:
dynamoose.model(
DYNAMO_DB_TABLE_NAME,
{id: String, attributes: Map},
{useDocumentTypes: true, saveUnknown: true}
);
(to match the table structure generated by the persistence adapter I'm using).
I assume I will need to do DynamoDB "scan" to achieve this rather than a "query" and I tried this to get started and get a records where week 12 equals 6 to no avail (I get an empty set as result):
const filter = {
FilterExpression: 'contains(#attributes, :val)',
ExpressionAttributeNames: {
'#attributes': 'attributes',
},
ExpressionAttributeValues: {
':val': {'12': 6},
},
};
model.scan(filter).all().exec(function (err, result, lastKey) {
console.log('query result: '+ JSON.stringify(result));
});
If you don't know Dynamoose but can help with solving this via the AWS SDK tu run a DynamoDB scan directly that might also be helpful for me.
Thanks!!
Try the following.
const filter = {
FilterExpression: '#attributes.#12 = :val',
ExpressionAttributeNames: {
'#attributes': 'attributes',
'#12': '12'
},
ExpressionAttributeValues: {
':val': 6,
},
};
Sounds like what you are really trying to do is filter the items where attributes.12 = 6. Which is what the query above will do.
Contains can't be used for objects or arrays.
I'm new to NoSQL database. Currently I'm trying to use the Firebase and integrate it with iOS. When it comes to predefine the database, with trial and error, I try to make it look like this:
When I tried to retrieve the "stories" path in iOS, I get json structure like this:
[
<null>,
{
comments: [
<null>,
1,
2,
3
],
desc: "Blue versus red in a classic battle of good versus evil and right versus wrong.",
duration: 30,
rating: 4.42,
tags: [
<null>,
"fantasy",
"scifi"
title: "The Order of the Midnight Sun",
writer: 1
]
}
]
My question is, why there's always a null at the beginning of each array? What should I do in the database editor to avoid the null?
It looks like you start pushing data to index 1 and not 0, inserting/retrieving data to/from a list starts with index 0:
I am using knex and bookshelf, and my table consists of author, title, content, count, and each data looks like this:
author: 'John Doe',
title: 'aaaaa',
content: 'aaaaaaaa'
count: 54,
I want to retrieve data based on the value of count, and I want to get 4 data that has the highest count value.
If I want to retrieve all data, I am doing like this:
router.get('/', (req, res) => {
Article.forge().fetchAll().then(article => {
res.json(article);
})
})
Is there any way that I can do like forge({ count: 3 data that has the highest count value }) or
What should I add the code so that I can achieve this?
Combine orderBy with fetchPage
Article
.orderBy('-count')
.fetchPage({
pageSize: 3
})
.forge()
This highlights a reason why my team is removing bookshelf and just using basic knex. Unless you are wanting to fetch related models it's simpler to deal without the ORM layer. The knex equivalent knex code is:
knex('articles')
.orderBy('count', 'desc')
.limit(3)
Which is slightly simpler and the resulting rows' properties can be accessed directly, ie rows[0].id rather than rows[0].get('id')
I am trying to get a list of all venues available through the Here API by requesting a venue index. My url looks as follows(with the strings replaced where necessary):
static-3.venue.maps.cit.api.here.com/1/models-poi/index_bb.js?Policy={Policy}&Signature={Signature}&Key-Pair-Id={Key Pair}&app_id={App Id}&app_code={App Code}
This returns a JSON table, which I want, but I only have 155 entries, although there are clearly more. Does anyone know why I don't get the full list? Thanks. Below is the first couple lines of the output I get.
JSON.venues([{ "gml:id" : "DM_8961", "bb": [ [52.4564118412704,13.384279785476354],[52.454433435991014,13.388207656793229] ]},{ "gml:id" : "DM_10465", "bb": [ [52.43143833815419,13.453328297714588],[52.4288047627723,13.45769174285097] ]},{ "gml:id" : "DM_17394", "bb": [ [52.475570406808345,13.458645521436816],
and so on
You're connecting to a CIT Environment where not all Venue Data is exposed.