I'm trying to archieve something like this :
example, using kibana and/or Vega/Vega-lite.
The csv file I used to add the index to kibana was:
student1,90,80,85,95
student2,50,60,55,100
student3,40,70,50,60
At the moment I have this:
{
"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
"data": {
"url": {
%context%: true,
"index":"grades",
"body":{
"size":5
"_source":["StudentName","test1","test2","test3","test4"]
}
},
"format":{"property":"hits.hits"}
},
"mark": "line",
"encoding": {
"x": {"field": "_source.test1", "type": "quantitative"},
"y": {"field": "_source.StudentName", "type": "nominal"}
}
}
So my problem is trying to archieve what is on the picture. I know the "encoding" section of my Vega code isn't correct but I'm having problem finding a way of having multiple parameters in X-axis.
I think this : vega example
would do the trick if i managed to replace the hardcoded values in data with the data from the kibana index. Is there any way I can use the "_source.fields" inside the "values" or is any option in encoding that I can use in order to archieve my result?
Thanks in advance.
Note: My end result most likely only have 1 student. But I want that the visualization to be updated in real-time, therefore the need to use the field.
You posed your question here and answers have been posted - https://github.com/vega/vega/issues/1229#issuecomment-379593878
Related
This is probably more complicated as it sounds, at least with Grafana.
I have an experiment, where for every location (1-100) a value is changed over time. I want to show this with a line graph (or a bar graph), where x-axis correspond to the locations (1-100) and y-axis correspond to the average value for this location for the time interval that is set in Grafana in the upper right corner. Data comes from database. Please, suggest me, which type of graph (dashboard) should I choose in Grafana to achieve the goal. I can only see two kinds of them, those with time on x-axis and those of type histogram but none seems to be applicable.
seems Grafana builtin panels only support
time series, which mean x-axis must be 'time' type.
bilibala-echarts-panel
I use this third-part panel bilibala-echarts-panel
https://grafana.com/grafana/plugins/bilibala-echarts-panel/
achieved the goal: x-axis not use time value.
It can use a custom callback function, to handle data and render
setting
in grafana query setting:
format as time series
// time series need 3 column:
time // time or number (can be parse as time)
metric // string: series name
value // number (otherwise got value error)
assign the value to time column.
// it may rander as 1970-01 in table view, but in echart we can read it as number
in echart panel option:
data.series got the grafana data
// convert/adapte it to echart data here.
maybe format as table can have more flexible data,
and parse them in the callback js.
summary
grafana and echart have different theory,
have to understand both of them ,
and do the convert in js.
// bilibala-echarts-panel using echart v4 #2022-07
We have implemented the graph using natel-plotly-panel plugin.
"panels": [
{
"pconfig": {
"traces": [
{
"mapping": {
"color": "64",
"size": null,
"text": "metriccat",
"x": "loccat",
"y": "valuecat",
"z": null
},
"name": "My value",
"show": {
"line": true,
"lines": true,
"markers": false
}
},
...
]
},
"pluginVersion": "7.5.5",
"targets": [
{
"format": "table",
"group": [],
"metricColumn": "none",
"rawQuery": true,
"rawSql": "SELECT\n avg(column_with_values) AS valuecat,\n loc loccat,\n avg(column_with_values) as metriccat\nFROM ... \nWHERE\n $__timeFilter(timestamp)\nGROUP BY loc\n\n",
"select": [
[
{
"params": [
"value"
],
"type": "column"
}
]
],
"timeColumn": "time",
"where": [
{
"name": "$__timeFilter",
"params": [],
"type": "macro"
}
]
},
...
],
"title": "Average Values Along the Locations",
"type": "natel-plotly-panel",
"version": 1
}
]
Trying to setup schema markup for a simple math solver action with two fields. Let's say addition.
1+1=2
Here is Google's doc and example:
{
"#context": "https://schema.org",
"#type": ["MathSolver", "LearningResource"],
"name": "An awesome math solver",
"url": "https://www.mathdomain.com/",
"usageInfo": "https://www.mathdomain.com/privacy",
"inLanguage": "en",
"potentialAction": [{
"#type": "SolveMathAction",
"target": "https://mathdomain.com/solve?q={math_expression_string}",
"mathExpression-input": "required name=math_expression_string",
"eduQuestionType": ["Polynomial Equation","Derivative"]
}],
"learningResourceType": "Math solver"
}
How do we add multiple variables for two numbers?
return {
'#context': 'https://schema.org',
'#type': ['MathSolver', 'LearningResource'],
...
potentialAction: [
{
'#type': 'SolveMathAction',
target: `domain.com/?num1={num1}&num2={num2}`,
'mathExpression-input': 'required name=num1 name=num2',
eduQuestionType: ['addition', 'sum']
},
],
learningResourceType: 'Math solver'
};
Schema.org says about mathExpression (note: mathExpression-input doesnt seem to exist) but does fall under Thing > Intangible EntryPoint
A mathematical expression (e.g. 'x^2-3x=0') that may be solved for > a specific variable, simplified, or transformed. This can take many > formats, e.g. LaTeX, Ascii-Math, or math as you would write with a > keyboard.
But can this be setup for URL params to accept multiple fields within the mathExpression-input instead of a single math expression?
Here is a sample document in my index:
{"_id":"2018:05:03:19","_source":{"0":9926,"1":53436,"2":81151,"3":56466,"4":29904,...}}.
I am trying to build a bar chart where instead of using aggregation I need to take the keys (0, 1, 2, etc) to use as X axis and corresponding values as Y. All aggregation has already been done in my pipeline.
Has anyone tried something like that? I would really appreciate any suggestions as I am really stuck.
Thank you,
Dmitry
My data looks like the following format and works very well:
{ "_index": "my_index_name", "_type": "doc", "_id": "a_random_id", "_source": {"key1": 9926, "key2": 53416, ...}}
{ "_index": "my_index_name", "_type": "doc", "_id": "a_random_id", "_source": {"key1": 9926, "key2": 53416, ...}}
...
Hope it helps!
I am brand new to Vega and I was trying to plot some charts on Vega (plugin ElasticSearch and Kibana). Below is the simple visulization I am trying to plot. I am following through the documentation to connect the existing data, however I am unable to get the visuals. It just shows Y and X axis labeled from the code below with blank plots. What am I doing wrong?
{
"$schema": "https://vega.github.io/schema/vega-lite/v2.json"
"data": {
url: {
%context%: true
index: test-data
}
format: {property: "hits.hits"}
},
"mark": {"type":"bar"}
"encoding": {
"x": {"field": "DEPT", "type": "ordinal"},
"y": {"field": "SALES", "type": "quantitative"}
}
}
The specification needs to be valid JSON. There are numerous things in your specification that make it invalid; for example:
all strings need to be enclosed in quotes (e.g. url and format)
all items need to be separated by commas (applies to nearly every line of your specification)
Finally, even if you change those syntax errors, the content of your specification doesn't follow the schema: for example, the "url" and the "format" properties of "data" both should be strings.
I would suggest beginning with the vega-lite tutorials, and go from there, modifying what you learn to work with your own data.
** UPDATE **
Thanks to Alfred Fuller for pointing out that I need to create a manual index for this query.
Unfortunately, using the JSON API, from a .NET application, there does not appear to be an officially supported way of doing so. In fact, there does not officially appear to be a way to do this at all from an app outside of App Engine, which is strange since the Cloud Datastore API was designed to allow access to the Datastore outside of App Engine.
The closest hack I could find was to POST the index definition using RPC to http://appengine.google.com/api/datastore/index/add. Can someone give me the raw spec for how to do this exactly (i.e. URL parameters, what exactly should the body look like, etc), perhaps using Fiddler to inspect the call made by appcfg.cmd?
** ORIGINAL QUESTION **
According to the docs, "a query can combine equality (EQUAL) filters for different properties, along with one or more inequality filters on a single property".
However, this query fails:
{
"query": {
"kinds": [
{
"name": "CodeProse.Pogo.Tests.TestPerson"
}
],
"filter": {
"compositeFilter": {
"operator": "and",
"filters": [
{
"propertyFilter": {
"operator": "equal",
"property": {
"name": "DepartmentCode"
},
"value": {
"integerValue": "123"
}
}
},
{
"propertyFilter": {
"operator": "greaterThan",
"property": {
"name": "HourlyRate"
},
"value": {
"doubleValue": 50
}
}
},
{
"propertyFilter": {
"operator": "lessThan",
"property": {
"name": "HourlyRate"
},
"value": {
"doubleValue": 100
}
}
}
]
}
}
}
}
with the following response:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "FAILED_PRECONDITION",
"message": "no matching index found.",
"locationType": "header",
"location": "If-Match"
}
],
"code": 412,
"message": "no matching index found."
}
}
The JSON API does not yet support local index generation, but we've documented a process that you can follow to generate the xml definition of the index at https://developers.google.com/datastore/docs/tools/indexconfig#Datastore_Manual_index_configuration
Please give this a shot and let us know if it doesn't work.
This is a temporary solution that we hope to replace with automatic local index generation as soon as we can.
The error "no matching index found." indicates that an index needs to be added for the query to work. See the auto index generation documentation.
In this case you need an index with the properties DepartmentCode and HourlyRate (in that order).
For gcloud-node I fixed it with those 3 links:
https://github.com/GoogleCloudPlatform/gcloud-node/issues/369
https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/system-test/data/index.yaml
and most important link:
https://cloud.google.com/appengine/docs/python/config/indexconfig#Python_About_index_yaml to write your index.yaml file
As explained in the last link, an index is what allows complex queries to run faster by storing the result set of the queries in an index. When you get no matching index found it means that you tried to run a complex query involving order or filter. So to make your query work, you need to create your index on the google datastore indexes by creating a config file manually to define your indexes that represent the query you are trying to run. Here is how you fix:
create an index.yaml file in a folder named for example indexes in your app directory by following the directives for the python conf file: https://cloud.google.com/appengine/docs/python/config/indexconfig#Python_About_index_yaml or get inspiration from the gcloud-node tests in https://github.com/GoogleCloudPlatform/gcloud-node/blob/master/system-test/data/index.yaml
create the indexes from the config file with this command:
gcloud preview datastore create-indexes indexes/index.yaml
see https://cloud.google.com/sdk/gcloud/reference/preview/datastore/create-indexes
wait for the indexes to serve on your developer console in Cloud Datastore/Indexes, the interface should display "serving" once the index is built
once it is serving your query should work
For example for this query:
var q = ds.createQuery('project')
.filter('tags =', category)
.order('-date');
index.yaml looks like:
indexes:
- kind: project
ancestor: no
properties:
- name: tags
- name: date
direction: desc
Try not to order the result. After removing orderby(), it worked for me.