I'm trying to create a heatmap of users' location in various world regions in R, but the dataset I'm working with contained a free text location field for users to fill on their own -- hence the quality of place names varies quite a bit. I'm OK with ignoring any string that won't parse as a known place (there's enough good data in here for me to get a sample), but I can't seem to find a failure-tolerant placename entity resolver to use in R.
Another issue is that some of these are cities and some of these are states and some of these are countries -- Ideally, I think, I'd like to make two maps, one map of the US in which all of the cities resolve to states, and one map of the world in which all of the locations on the first map resolve to the US. Is there a reasonably easy way of doing this?
Thanks!
Related
The below linked map contains a number of layers which I would like to be able to extract as polygons [if possible]. I've not previously done any web-scraping and realise that doing so in regards to the geographic data on this system represents a significant challenge.
Ideally I would only want to extract the data relating to the 'Shopping Local Centre' category.
Happy to try to use Phython or R to achieve such, just wondered if anyone had any ideas.....
Web scraping (using BeautifulSoup, for example) would get you the HTML objects from a webpage. You would need basic knowledge of Python for this.
Or you could avoid that by going this route:
With QGIS and Geofabrik you can gather retail location polygons and their attributes of a given area.
Use Geofabrik to download your area of interest in *.shp (shapefile) format. It looks like you're in Greater Manchester, so I navigated to the download page here (it's a 50MB file for the greater-manchester-latest-free.shp.zip).
Once you download that, open it in QGIS and you'll see in the attributes it has retail locations.
That site is using WMS to display the map (I work for the company that makes iShare) so there is no vector content for you to scrape as it works entirely with images.
The easiest way to get the data would be to ask the council to provide it, you might need to make it a freedom of information request but they should be happy to provide the data in a usable GIS format.
I am trying to retrieve information about trees surrounding a given location from the Socrata API.
API Endpoint Description
I found two functions within_circle(...) and distance_in_meters(...) which I could use to filter the data set. The problem is, that those functions need either a location or a point data type which is not present in the data set.
There is, however a latitude and longitude field.
Is there any method to utilize those functions or get nearby trees other way?
Tried this, but POINT(0 0) must be the point of the tree.
https://data.cityofnewyork.us/resource/nwxe-4ae8.json?$where=within_circle( 'POINT(0 0)' ,0,0,400)
I need something like this.
https://data.cityofnewyork.us/resource/nwxe-4ae8.json?$where=within_circle( make_point(latitude, longitude) ,0,0,400)
SoQL
If you have not already done so, you may want to submit this question at https://support.socrata.com. In addition to the possibility that the people there will have an answer I do not, it would serve as some feedback that a function like what you had in mind would be useful.
I cannot think of a way to do exactly what you have in mind. Really, what I mean is a way that is within your power (or mine). The owner of the dataset could create a Point column -- and you may want to reach out to the NYC open data team to ask for that if you have not already done so.
However, since the X and Y coordinates, in feet, are present, you should be able to use the Pythagorean theorem to determine the distance from any given point. For that matter, the size of a degree of latitude or longitude cannot vary that much over an area as small as NYC so you could do the same thing with those values and save having to figure out the X and Y of your reference point.
Good luck!
Jon
I want to use the script of Neveldo for Mapael in order to insert the regions in the map of France as well as the departments.
I can not nest the codes for the different JS files in each region provided in a single file by Neveldo: https://github.com/neveldo/mapael-maps/tree/master/france.
So I wish to keep the current interactivity of this map by including the regions of NEVELDO. http://cnecj.org/cnejc/annuaire/#cours-appel
The site VincentBroute.fr/Mapael does not give documentation and on the Github either.
Can you help me understand the architecture of this code and possibly give me a solution to nest: map of France, regions and departments?
thank you
I'm not sure what you are trying to do, so I will try my best to answer broadly.
In Mapael, a map is composed of areas. These areas are defined in the map file.
For instance, the map france_departments defines 95 areas, which corresponds to all (Metropolitan) France departments.
However, in the map france_regions_2016, it defines 12 areas which corresponds to all (Metropolitan) regions of France
This means that a map in Mapael cannot have the regions and its departments, because they would overlap and hide each other. This is because they are showing the same thing but with different level of details.
So, if you want to have the departments (in order to show details about it) as well as being able to recognize each regions, the best approach would be to use the map which has the most level of details you need (i.e. the one with departments) and group departments by regions by visually filling them with different colors (e.g. green for Normandie, yellow for Bretagne, and so on...).
I am using the Here API to search for roads where traffic accidents occur in my home town. My aim is to extract the full road coordinates (as sequence of of lat/lons) where the accident is active in order to draw that road on a reasonable scale.
Using the traffic/6.1/incidents API, I query for the data using a geographical bounding box. An accident in the response is described by one pair of TMC locations as well as one pair of lat/lon geographical points. This data is too coarse-grained for plotting the roadway. (Please correct me if I am wrong here, but I would like to add some more detail about turns of the road.)
My question regards what I found when I turned to examine the rest of the accident/incident response, containing so-called NAVTECH data on this (XML) form:
<NAVTECH><EDGE><EDGE_ID>1150578390</EDGE_ID><EDGE_ID>1150578391</EDGE_ID>
<EDGE_ID>...</EDGE_ID>
<VERSION_ID>201604</VERSION_ID></NAVTECH>
My question is: Is there any available method to map the NAVTECH edges to pairs of lat/lon coordinates?
I have searched Here.com, the here-api stackoverflow tag and tried to Google for an answer, but I find so little documentation on this type of data. I am looking for all information I can find to (i) Describe the NAVTECH edges and their meaning, (ii) How to map NAVTECH to geographical coordinates. In general, (iii) Can I use the Here API to get more geographical information than this about roads where there are accidents?
Kind regards,
Vilhelm from Sweden
You can pass a lat/lon to their routing API and get back the shape point coordinates (and other attributes) for the road edge at that location, like this:
https://route.cit.api.here.com/routing/7.2/getlinkinfo.xml?waypoint=43.066628,-71.471729&app_id=DemoAppId01082013GAL&app_code=AJKnXv84fjrb0KIHawS0Tg&linkattributes=all
Unfortunately, though, this may not be a long term solution, as seaBass states here:
getlinkinfo will depreciated March 31, 2017. In order to get speed limit information you will have to use PDE (Platform Data Extension) which requires an APP ID and Code
So what I'm trying to do is the following:
Have a map (such as Google Maps or questMaps). It doesn't matter at all which API I need to use.
On that map have an overlay on the streets. So say (for example) the street has bad lightning at night, it will be colored red. If it has good lightning it will have a green overlay.
Based on the overlay the map creates a custom route (for example the user only wants to walk on the green/well lit streets).
I have no idea how to accomplish this (especially step 3).
First, you'll have to decide what data you need. How do you categorize certain streets as lit or unlit? What if some parts of a street are well lit and some have no lights? Do you need to know the location of every streetlight in your area? What if lights burn out?
After figuring out what data you need, you need to build your dataset. I'd be VERY surprised if this data already exists, so you will probably need to gather it yourself. Either go around town and take notes, or crowdsource the project, or figure out some other way.
Once you have gathered your data, learn the drawing API of whatever mapping tool you wish to use. They all should have functions in their API for drawing colored lines (for streets) or points (for streetlights) on top of an existing map.
Finally, learn the navigational API of the mapping tool you chose. You're right, this is a hard step. I know Google Maps lets you specify certain waypoints when requesting directions; maybe your app can calculate well-lit waypoints and feed them to Google Maps' Directions service to influence the route it generates.
Good luck!
For custom routing, you need to read up on "Graph Theory". This ignores the geography of the street map, and considers it as a set of junctions (nodes or vertices in the graph theory jargon) connected by edges. You can assign weights to edges - these could be lengths, travel times, ones and zeroes etc. Anything. They can have no relation to the position on the map.
So for your application, you'd assign a large weight to unlit streets, and a small weight to lit streets, then use a standard minimum-weight algorithm to get a route from one node to another.