I have an application that handles payment network transactions (from MasterCard via a third party issuer processor). For an approved auth we're able to ascertain (via a separate third party) the latitude and longitude of the merchant store where the transaction was initiated.
I'm logging the latitude and longitude to the stdout as follows:
Latitude: {latitude}
Longitude: {longitude}
Logs like this (and may others) get propagated through to ElasticSearch (in Elastic Cloud).
Using the latitude/longitude; how would I go about creating a visualization (in Kibana) of a geographical map with a heat map overlay that shows where the all transactions are coming from?
I'm thinking something like this (which is just an image from Google images):
or:
Ideally as you scroll in, the heat map precision would also get more detailed.
FWIW I have complete control over the log message in the source app, as well as various components in the telemetry pipeline where I could preform transformations (e.g. Logstash and Elastic ingest pipelines).
The most performant solution would be to index latitude longitude data as geo_point field.
Another, less performant solution, is to create a geo_point runtime field.
Once your data is available as geo_point, you can use Maps to visualize the data on a map.
Related
I'm trying to visualize GPS points on PowerBI maps offline and wonder what would be the best way to achieve that.
Ideally, I'd like to see the offline mapping to have the same effect as directly using the "Map" visualization - GPS points (in data as longitude and latitude columns) plotted on the Bing map. However, our team will need this function offline, because our users are usually under bad internet connections. Also we are wary about Bing's data sharing policy as our GPS information is highly confidential.
So far I've tried PowerBI's shape map function, which allows me to upload a TopoJSON file and use it offline. However, it looks like shape map is better used for highlighting a region on the map, instead of plotting individual GPS points - shape map doesn't seem to allow longitude/latitude. Besides that, shape map does not have as much information like Bing map, such as showing city/road names and etc.
Is there a way to visualize GPS points offline on PowerBI, but still get enough information about what are around the GPS points? I'm referring to PowerBI here but all things I've been doing are on PowerBI Desktop. I don't plan to publish on the web as this will be my company's internal tool. Thanks in advance!
The built-in mapping functions such as the Map visual powered by the Bing maps API and the Eris mapping do not have an offline mode. Both require an online connection. Depending on the level of detail that you require, I would suggest looking into an R or Python visual with one of the mapping libraries. Here are some examples of the R mapping libraries. But getting the map json files into Power BI will also be an issue and may not meet your requirements on the level of detail that you need.
Also we are wary about Bing's data sharing policy as our GPS information is highly confidential.
I wouldn't worry about it, Bing & Power BI mapping is used by a wide number of organisation including heath and military customers. Your confidential data is more at risk with some one copying your Power BI report and sticking it on a USB drive.
However, our team will need this function offline, because our users are usually under bad internet connections
I would try some other options for offline mapping, for example Google Maps has the ability to down load a section of the map for offline/bad connections. I've downloaded the whole UK and its about 100meg.
Hope that helps
I am trying to construct a dataset including road links, their geometries, and information about the roads such as number of lanes, functional class, and average daily traffic. I have been unable to determine whether Here has an API to retrieve something approximating average annual daily traffic for each road link. Is it feasible to retrieve this data?
I've found an API to get general road link information which does not include traffic:
https://developer.here.com/documentation/platform-data/dev_guide/topics/quick-start-view-map-data.html
I've found an API to retrieve map tile images built from historic data (given a day of week and time of day) that shows traffic flow, but does not provide the underlying data:
https://developer.here.com/documentation/map-tile/dev_guide/topics/example-traffic.html
My desired result would be something structured as {road_link_id, annual_average_daily_traffic}, so that I can connect the link with other associated information.
The HERE Traffic API is what you are looking for. It offers:
Traffic Incident Data - provides aggregated information about traffic incidents in XML or JSON, including the type and location of each traffic incident, status, start and end time, and other relevant data. This data is useful to dynamically optimize route calculations.
Traffic Flow Data - provides access to real-time traffic flow data in XML or JSON, including information on speed and congestion for the region(s) defined in each request. The API can also deliver additional data such as the geometry of the road segments in relation to the flow.
Traffic Flow Availability - allows client applications to access traffic flow information (excluding incidents) in an area, if available.
Traffic Map Tile Overlays (Traffic Tiles) - delivers pre-rendered map tile overlays with traffic information that you can readily display with your mapping application. You can request map tiles that show traffic data for a specific area.
See documentation: https://developer.here.com/documentation/traffic/dev_guide/topics/what-is.html
I started exploring HERE maps few days back and I want to get road specify data (tunnel, bridge, overpass, underpass, speed limit) like the one returned using "Fleet Telematics Advanced Data Sets" after using specific layer. I see that this returns list of all the points that belong to particular tile.
But just want to for gps points that my path has and not on an area. Is it possible using any HERE maps api or any manipulations to the above API or any other WAR?
It is not possible to get road geometry for a single gps point buut rather for the link/road in which the gps point is. If you want to find get road geometry details for the links/roads of a particular route you can get it by using Fleet Telematics API. Adding "attributes=ROAD_GEOM_FCn(TUNNEL,BRIDGE)" can fetch you the details whether the road is a tunnel or a bridge etc. Hope this helps!
http://cre.cit.api.here.com/2/calculateroute.json?app_id=xxxx&app_code=xxxx&mode=fastest;truck;traffic:disabled&driver_cost=10&waypoint0=55.308989,10.805059&waypoint1=55.368920,11.288338&attributes=ROAD_GEOM_FCn(TUNNEL,BRIDGE)
I am trying to use HERE Maps to implement the eco-routing algorithm which we developed in our lab that requires real time traffic data as an input to the routing algorithm. So far, we were using INRIX dataset to verify our algorithms, but we decided to move to HERE Maps, and use the Traffic API to harness the power of real time traffic data.
Traffic API was very simple to use, and I managed to get the traffic data for a bounding box. It seems like that the traffic flow is passed for each TMC. My question is how to assign the traffic flow data to the links of the map using TMC. More specifically, I am wondering if there is a way to transform the TMC values to an adjacency matrix so that I can assign the traffics to each link and run my optimization algorithm over that graph? Also is it possible to overlay Open Street Maps (OSM) with the HERE Maps traffic data?
Thanks
You can associate the TMC data you get from the traffic data response to a particular link by using PDE layers LINK_TMC_FC1..5. You have look into our PDE documentation in developer site and tcs.ext.here.com/pde/maps?url_root=pde.api.here.com to see the list of layers provided.
Below is a sample PDE request for the Link TMC. Replace app id and code placeholders with actuals:
https://pde.api.here.com/1/tile.json?&layer=LINK_TMC_FC1&level=9&tilex=496&tiley=358®ion=WEU&release=18131&url_root=pde.api.here.com&app_id={YOUR-APP-ID}&app_code={YOUR-APP-CODE}
For all your other questions on creating a matrix and OSM overlay, you have to write your own algorithm. There is no straight forward function provided by Here.
I am building an Events site in which the address of a given event will be stored to be used within a map on the page. Whilst viewing an Individual Events' page i would like to provide a list of similar events on side of the page. I would also like to rank this list by distance and display a small map with a marker on it for each item on the list.
I was thinking of Geocoding the Events address upon content input and then storing this data within the sites database. Doing this would allow me to calculate the distance of an event on the server and then provide an ordered list to display.
My question is this:
Would this violate the Googlemaps Terms of Service?
These terms:
"the geocoder for any purpose other than obtaining locations that will be displayed using the Google Maps APIs is a violation of the Terms of Service. You may use the HTTP geocoder to geocode addresses outside of your Google Maps API application so that they may be cached and later displayed using one of the Google Maps APIs, but locations obtained using the Geocoding Web Service may not be used by any other application, distributed by other means, or resold."
Suggest that storing the data is not a problem but i am unsure that the process of ordering the list by long lat would not result in a violation.
Any advice would be greatly appreciated.
As long as the map your refer to in your question is a Google Map and you are using the geocoded data to display features on a Google Map, you are fine. The client-side google.maps.Geocoderdev-guide is intended for dynamic requests that are based on real-time user interaction that drives a need to perform on-the-fly geocoding. But Google encourages you to consider using the Geocoding Web Servicedev-guide on the server-side, when you wish to pre-geocode static addresses or coordinates prior to using the response values on a Google Map. Storing those values is assumed.
It is common to perform some form of post-response/post-storage processing on those values, just like the ordering by distance that you describe, to fit whatever use case is needed to populate your map with the appropriate data. I understand you want to be careful to remain within the TOS, but what you describe sounds pretty standard.