I am currently working on a large population-based database where it is required that I compute the distance between two addresses for each individual. My first train of thought was to obtain the latitude and longitude representing each address and then compute the distance. I used Google API's to obtain the latitude and longitude for these addresses via various packages in R (e.g. dismo). However Google has a restriction of 2500 requests in a 24hr period. I have about 300,000 addresses and by running 2,500/day, I will not be able to meet the deadline.
Would anyone have suggestions regarding other API's I could use to obtain the latitude and longitude using R?
Thank you.
I would recommend that you try the RDSTK package which interfaces with the Data Science Toolkit. It is available on github.
EDIT. To compute the Road Distance, I would recommend the Mapquest Directions API, which appears to have no preset limits as per this link. Let me know how it goes.
What you are doing is called "geocoding". Knowing that will greatly improve your ability to search for more info :) You may have already known that since you tagged the question properly, but I wanted to make sure.
Here's a really good previous Stack Overflow question, albeit not R specific.
More options:
The geonames package to retrieve information from the geonames service.
The osmar package to use the API of openstreetmap.
Besides, the gdistance package is useful for distance calculations on geographical grids.
Related
I have been creating a twitter bot to post satellite images of random coordinates and I have found that the vast majority are of the ocean, so I would like to somehow make the bot detect that the entire photo is blue and repeat the process. The bot code is available here.
Thanks in advance.
As an alternative to employing machine vision to assess the content of the image to determine whether to sample again, you could simply sample from coordinates that are not the ocean! There are many files produced by geographers that have sought to map the boundaries of the world's continents.
For example, see here for code to do so. You'll need to understand projections and other features of Geographic Information Systems if you want to get extremely accurate, but you can get reasonably accurate using the polygons for oceans. If you REALLY wanted to get accurate, you could obtain a set of shapefiles that partition all possible coordinates (rather than a single "world" shapefile) and use the more-detailed mapping that may accompany those more-zoomed-in shapefiles.
Alternatively, you can implement the model here, provided in a guide to water detection in satellite imagery using Python.
I'm trying to visualize GPS points on PowerBI maps offline and wonder what would be the best way to achieve that.
Ideally, I'd like to see the offline mapping to have the same effect as directly using the "Map" visualization - GPS points (in data as longitude and latitude columns) plotted on the Bing map. However, our team will need this function offline, because our users are usually under bad internet connections. Also we are wary about Bing's data sharing policy as our GPS information is highly confidential.
So far I've tried PowerBI's shape map function, which allows me to upload a TopoJSON file and use it offline. However, it looks like shape map is better used for highlighting a region on the map, instead of plotting individual GPS points - shape map doesn't seem to allow longitude/latitude. Besides that, shape map does not have as much information like Bing map, such as showing city/road names and etc.
Is there a way to visualize GPS points offline on PowerBI, but still get enough information about what are around the GPS points? I'm referring to PowerBI here but all things I've been doing are on PowerBI Desktop. I don't plan to publish on the web as this will be my company's internal tool. Thanks in advance!
The built-in mapping functions such as the Map visual powered by the Bing maps API and the Eris mapping do not have an offline mode. Both require an online connection. Depending on the level of detail that you require, I would suggest looking into an R or Python visual with one of the mapping libraries. Here are some examples of the R mapping libraries. But getting the map json files into Power BI will also be an issue and may not meet your requirements on the level of detail that you need.
Also we are wary about Bing's data sharing policy as our GPS information is highly confidential.
I wouldn't worry about it, Bing & Power BI mapping is used by a wide number of organisation including heath and military customers. Your confidential data is more at risk with some one copying your Power BI report and sticking it on a USB drive.
However, our team will need this function offline, because our users are usually under bad internet connections
I would try some other options for offline mapping, for example Google Maps has the ability to down load a section of the map for offline/bad connections. I've downloaded the whole UK and its about 100meg.
Hope that helps
I am trying to retrieve information about trees surrounding a given location from the Socrata API.
API Endpoint Description
I found two functions within_circle(...) and distance_in_meters(...) which I could use to filter the data set. The problem is, that those functions need either a location or a point data type which is not present in the data set.
There is, however a latitude and longitude field.
Is there any method to utilize those functions or get nearby trees other way?
Tried this, but POINT(0 0) must be the point of the tree.
https://data.cityofnewyork.us/resource/nwxe-4ae8.json?$where=within_circle( 'POINT(0 0)' ,0,0,400)
I need something like this.
https://data.cityofnewyork.us/resource/nwxe-4ae8.json?$where=within_circle( make_point(latitude, longitude) ,0,0,400)
SoQL
If you have not already done so, you may want to submit this question at https://support.socrata.com. In addition to the possibility that the people there will have an answer I do not, it would serve as some feedback that a function like what you had in mind would be useful.
I cannot think of a way to do exactly what you have in mind. Really, what I mean is a way that is within your power (or mine). The owner of the dataset could create a Point column -- and you may want to reach out to the NYC open data team to ask for that if you have not already done so.
However, since the X and Y coordinates, in feet, are present, you should be able to use the Pythagorean theorem to determine the distance from any given point. For that matter, the size of a degree of latitude or longitude cannot vary that much over an area as small as NYC so you could do the same thing with those values and save having to figure out the X and Y of your reference point.
Good luck!
Jon
I'm trying to get speed limit from Here API maps but I can't find the way to do it. I tryed few example on web site but the only one that works is the one which require point of start and point of stop of the route.
I would like to get the speed limit given only one point ( or a box ). Which api do I have to use? Is there an example?
https://route.cit.api.here.com/routing/7.2/calculateroute.json?jsonAttributes=1&waypoint0=51.31854,9.51183&waypoint1=50.11208,8.68342&departure=2019-01-18T10:33:00&routeattributes=sh,lg&legattributes=li&linkattributes=nl,fc&mode=fastest;car;traffic:enabled&app_code=appcode&app_id=appid
This is the waypoint one but is not what I'm looking for, I would like to pass it only a Latitude / Longitude.
Thank you
You can achieve this by using PDE API(Platform Data Extension)
You have to first map your geocordinates(lat,long) to a navigable position(lat,long) for the given coord and the Functional Class(FC1-5) it is located in. This you can achieve by a simple geocoder request
You can calculate tilexy values based on the navigable lat,long and pass it to PDE API for querying speed limit layer on that particular FC class.
Look at https://tcs.ext.here.com/examples/v3/link_speed_locator example which covers this exact usecase.
Below is a sample geocoder request. Here prox parameter is your lat,long
https://reverse.geocoder.cit.api.here.com/6.2/reversegeocode.json?app_id=xxxx&app_code=yyyy&prox=50.133848, 8.715332,500&mode=retrieveAddresses&maxResults=1&additionaldata=SuppressStreetType,Unnamed&locationattributes=linkInfo
Below is a sample pde request
https://pde.cit.api.here.com/1/tiles.json?layers=SPEED_LIMITS_VAR_FC1,SPEED_LIMITS_VAR_FC2,SPEED_LIMITS_VAR_FC3,SPEED_LIMITS_VAR_FC4,SPEED_LIMITS_VAR_FC5,ROAD_GEOM_FC1,ROAD_GEOM_FC2,ROAD_GEOM_FC3,ROAD_GEOM_FC4,ROAD_GEOM_FC5&levels=9,10,11,12,13,9,10,11,12,13&tilexy=536,398,1073,797,2147,1594,4294,3188,8588,6377,536,398,1073,797,2147,1594,4294,3188,8588,6377&app_id=xxxx&app_code=yyyy
Read more about it in the developer site - https://developer.here.com/documentation/platform-data/topics/quick-start-view-map-data.html
Hope you find this useful!
The ReverseGeocoder has rarely used mode "trackPosition" (mode=trackPosition).
In combination with locationattributes=linkInfo you may retrieve the speed limit for that matching road.
I'm pulling distance/time information for a large number of origin/destination pairs using the Google Maps API in R. I'm currently using the gmapsdistance package but have looked at a few others.
My premium API key includes 100k free queries per day. Are there any packages that can return how many are remaining? For example, the ggmap package has a geocodeQueryCheck(). The problem is I don't think this function actually returns the number remaining on your account. It doesn't ask for your API key. My guess is that it just keeps track of how many it has called today. The latest github version has a register_google() function that does allow you to set your API key, but when I make API requests with the gmapsdistance package, geocodeQueryCheck() doesn't update.
In summary, I just want to know how many are left. Even if I need to construct the URL address directly. When I look at the API documentation, I don't even see URL calls for it, which doesn't give me much hope.
As confirmed by #SymbolixAU, there is currently no way to do this.
Sorry, I guess this is late, but have you tried this?
sum(.GoogleDistQueryCount$elements)