I'm using the newest version of meteor and I've created a schema, and I would like to add a location to the schema. I'm wondering what the best practices are for saving locations to a collection using autoform and ultimately query based on geo-location and proximity.
The latest Meteor comes with Mongo 3.2
Mongo 3.2 uses GeoJSON to let you describe locations using different types of geometry. For example, you can identify a location as a point in space like:
{
location: {
type: "Point",
coordinates: [-73.856077, 40.848447]
},
name: "Morris Park Bake Shop"
}
Or a polygon like:
{
geometry: {
type: "Polygon",
coordinates: [[
[ -73.99, 40.75 ],
...
[ -73.98, 40.76 ],
[ -73.99, 40.75 ]
]]
},
name: "Hell's Kitchen"
}
This allows you to make complicated queries like:
const neighborhood = Neighborhoods.findOne({geometry: {$geoIntersects: {$geometry: {type: "Point", coordinates: [ -73.93414657, 40.82302903 ]}}}});
const restaurants = Restaurants.find({location:{$geoWithin:{$geometry: neighborhood.geometry}}})
(example from this mongo tutorial)
This is mongo's recommended way to save geospatial data with indices and sharding support.
There is an autoform map addon: https://github.com/yogiben/meteor-autoform-map/ -- that has GeoJSON support
Related
Dear Stackoverflow team,
I'm impressed that after a bunch of hours digging the forum I still can't find any question/answer similar to my problem :
I have a GeoJson with a lot of Points features. I collect all Points (green in my example, see figure below) that follow some specification (distance between each of them)
Initial Data:
and I want to link all of them to build a Polygon (which represent an area).
What I'm looking for :
Or Solution accepted :
So I collect all coordinates from these Points, and to be sure the Polygon follows the GeoJson requirements, I'm using the "rewind" function
from geojson_rewind import rewind
But at the end, whatever I've tried I only get that kind of Polygon:
I probably don't use correctly the "rewind" function?
I'm looking for an (easy) automatic way to link all points together in a "convexion hull"
Thanks a lot for any help !
My initial coordinates are collected in a list :
[[4.3556672, 50.8538851], [4.3542534, 50.8546955], [4.3567798, 50.8547854], [4.3566527, 50.8541356], [4.3574286, 50.8552813], [4.3572234, 50.8551264], [4.3547752, 50.8545063], [4.3572736, 50.8560176], [4.3571226, 50.8546104]]
and the Polygon GeoJson I've managed to build, with the rewind function (recopying the last coordinates to get a Polygon) looks like that :
{
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"properties": {},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
4.357223,
50.855126
],
[
4.35678,
50.854785
],
[
4.355667,
50.853885
],
[
4.356653,
50.854136
],
[
4.357123,
50.85461
],
[
4.354253,
50.854695
],
[
4.354775,
50.854506
],
[
4.357429,
50.855281
],
[
4.357274,
50.856018
],
[
4.357223,
50.855126
]
]
]
}
}
]
}
The shapely library is very useful for doing these kinds of geometric manipulations.
For generating a polygon of the convex hull of a set of geometries you can use object.convex_hull:
import shapely.geometry as sg
points = [[4.3556672, 50.8538851], [4.3542534, 50.8546955], [4.3567798, 50.8547854], [4.3566527, 50.8541356], [4.3574286, 50.8552813], [4.3572234, 50.8551264], [4.3547752, 50.8545063], [4.3572736, 50.8560176], [4.3571226, 50.8546104]]
polygon = sg.MultiPoint(points).convex_hull
Which results in the following shape:
Converting it into a GeoJSON with the help of shapely.geometry.mapping:
feature_collection = {
"type": "FeatureCollection",
"features": [
{"type": "Feature", "properties": {}, "geometry": sg.mapping(polygon)}
],
}
import json
geojson = json.dumps(feature_collection)
Spatial indexing does not seem to be working on a collection which contains a document with GeoJson coordinates. I've tried using the default indexing policy which inherently provides spatial indexing on all fields.
I've tried creating a new Cosmos Db account, database, and collection from scratch without any success of getting the spatial indexing to work with ST_DISTANCE query.
I've setup a simple collection with the following indexing policy:
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/\"location\"/?",
"indexes": [
{
"kind": "Spatial",
"dataType": "Point"
},
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Range",
"dataType": "String",
"precision": -1
}
]
}
],
"excludedPaths": [
{
"path": "/*",
},
{
"path": "/\"_etag\"/?"
}
]
}
The document that I've inserted into the collection:
{
"id": "document1",
"type": "Type1",
"location": {
"type": "Point",
"coordinates": [
-50,
50
]
},
"name": "TestObject"
}
The query that should return the single document in the collection:
SELECT * FROM f WHERE f.type = "Type1" and ST_DISTANCE(f.location, {'type': 'Point', 'coordinates':[-50,50]}) < 200000
Is not returning any results. If I explicitly query without using the spatial index like so:
SELECT * FROM f WHERE f.type = "Type1" and ST_DISTANCE({'type': 'Point', 'coordinates':[f.location.coordinates[0],f.location.coordinates[1]]}, {'type': 'Point', 'coordinates':[-50,50]}) < 200000
It returns the document as it should, but doesn't take advantage of the indexing which I will need because I will be storing a lot of coordinates.
This seems to be the same issue referenced here. If I add a second document far away and change the '<' to '>' in the first query it works!
I should mention this is only occurring on Azure. When I use the Azure Cosmos Db Emulator it works perfectly! What is going on here?! Any tips or suggestions are much appreciated.
UPDATE: I found out the reason that the query works on the Emulator and not Azure - the database on the emulator doesn't have provisioned (shared) throughput among its collections, while I made the database in Azure with provisioned throughput to keep costs down (i.e. 4 collections sharing 400 RU/s). I created a non provisioned throughput database in Azure and the query works with spatial indexing!! I will log this issue with Microsoft to see if there is a reason why this is the case?
Thanks for following up with additional details with regards to a fixed collection being the solution but, I did want to get some additional information.
The Cosmos DB Emulator now supports containers:
By default, you can create up to 25 fixed size containers (only supported using Azure Cosmos DB SDKs), or 5 unlimited containers using the Azure Cosmos Emulator. By modifying the PartitionCount value, you can create up to 250 fixed size containers or 50 unlimited containers, or any combination of the two that does not exceed 250 fixed size containers (where one unlimited container = 5 fixed size containers). However it's not recommended to set up the emulator to run with more than 200 fixed size containers. Because of the overhead that it adds to the disk IO operations, which result in unpredictable timeouts when using the endpoint APIs.
So, I want to see which version of the Emulator you were using. Current version is azure-cosmosdb-emulator-2.2.2.
The title pretty much says what I intend to do.
I am using Firebase as backend for markers on a map and use the on('child_added') method to monitor these. For every node on a specific location in the database, the on('child_added') will fire once.
This also applies to new nodes being created, hence this is perfect for asynchronously adding new markers to the map as they are added to the database.
In order display these on a map, mapbox GL requires me to transform the data to geojson, create a source and then add this source to a layer. The code below shows this and it actually displays the markers on the map.
markersRef.on('child_added', function(childSnapshot) { //fires once for every child node
var currentKey = childSnapshot.key; //the key of current child
var entry = childSnapshot.val(); //the value of current child
//creates a geojson object from child
var geojson = {
"type": "FeatureCollection",
"features": [{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [entry.position.long, entry.position.lat]
}
}],
"properties": {
title: entry.title,
text: entry.text
}
};
//creates a source with the geojson object from above
map.addSource(currentKey, { //currentKey is the name of this source
"type": "geojson",
"data": geojson,
cluster: true //clusters the points in this source
});
//adds the source defined above to a layer that will be displayed on a map
map.addLayer({
"id": currentKey, // Sets id as current child's key
"source": currentKey, // The source layer defined above
});
});
The problem is that the markers will be in individual sources, making them appear on different layers. Therefore, I cannot cluster them or e.g. search across them.
What I look for is a way to add a source to an existing layer. This would enable me to create a layer outside the on('child_added') method and then add the sources to this layer.
I have looked at the mapbox GL docs but I cannot find anything in there that will enable me to do this. It seems very limited in this respect compared to mapbox js.
I see this as a pretty important feature and don't understand why this is not possible. I hope some of you have a workaround or a way to achieve asynchronously adding markers to a map in mapbox GL.
I have the same problem. I did some searching on this and I found the setData attribute for GeoJSONSource:
https://www.mapbox.com/mapbox-gl-js/api/#geojsonsource#setdata
map.addSource("points", markers);
map.addLayer({
"id": "points",
"type": "symbol",
"source": "points",
"layout": {
"icon-image": "{icon}-15",
"icon-allow-overlap": true,
"icon-ignore-placement": true,
"icon-size": 2,
"icon-offset": [0, -10],
}
});
Then later I update the source, without creating a new layer like so:
map.getSource('points').setData(newMarkers)
So this updates the source without creating a new layer. Then you can search over this layer. The only problem I encountered was that setData erases all the previous data (there is no "addData" functionality) so you need to save the previous markers and add them again. Let me know if you find a workaround for this.
As the documentation at https://www.mapbox.com/mapbox-gl-js/api/#geojsonsource states: A GeoJSON data object or a URL to one. The latter is preferable in the case of large GeoJSON files.
What happens here is that geojson sources loaded via a url are loaded using a background thread worker so they do not affect the main thread, basically always load your data via url or a mapbox style to offload all JSON parsing and layer loading to another thread. Thus anytime you have a change event fired from your firebase monitoring you can simply reload the url you are using to initially load the source.
In addition, the founder of Leaflet and amazing Mapbox Developer Vladimir Agafonkin discusses this here: https://github.com/mapbox/mapbox-gl-js/issues/2289, and it is essentially what they do in their real-time example: https://www.mapbox.com/mapbox-gl-js/example/live-geojson/.
Furthermore, here is an example with socket.io I use client side:
const url = {server url that retrieves geojson},
socket = {setup all your socket initiation, etc};
socket.on('msg', function(data) {
if (data) {
//Here is where you can manipulate the JSON object returned from the socket server
console.log("Message received is: %s", JSON.stringify(data));
if(data.fetch){
map.getSource('stuff').setData(url)
}
} else {
console.log("Message received is empty: so it is %s", JSON.stringify(data));
}
});
map.on('load', function(feature) {
map.addSource('events', {
type: 'stuff',
data: url
});
map.addLayer({
"id": "events",
"type": "symbol",
"source": "events",
"layout": {
"icon-image": "{icon}"
}
});
});
First things first: is this data in proper GeoJSON format?
According to the definition of GeoJSON data, as you can see by the MultiPoint & coordinates, I think it is.
It looks like this:
{
"lang": {
"code": "en",
"conf": 1.0
},
"group": "JobServe",
"description": "Work with the data science team to build new products and integrate analytics\ninto existing workflows. Leverage big data solutions, advanced statistical\nmethods, and web apps. Coordinate with domain experts, IT operations, and\ndevelopers. Present to clients.\n\n * Coordinate the workflow of the data science team\n * Join a team of experts in big data, advanced analytics, and visualizat...",
"title": "Data Science Team Lead",
"url": "http://www.jobserve.com/us/en/search-jobs-in-Columbia,-Maryland,-USA/DATA-SCIENCE-TEAM-LEAD-99739A4618F8894B/",
"geo": {
"type": "MultiPoint",
"coordinates": [
[
-76.8582049,
39.2156213
]
]
},
"tags": [
"Job Board"
],
"spider": "jobserveNa",
"employmentType": [
"Unspecified"
],
"lastSeen": "2015-05-13T01:21:07.240000",
"jobLocation": [
"Columbia, Maryland, United States of America"
],
"identifier": "99739A4618F8894B",
"hiringOrganization": [
"Customer Relation Market Research Company"
],
"firstSeen": "2015-05-13T01:21:07+00:00"
},
I want to visualize this as a "zoomable",viz. interactive, map, as in the examples on the d3js website.
I'm trying to use a tool called mapshaper.org to see an initial visualization of the data in map form, but when I load it up, nothing happens.
To me this doesn't make sense because, according to their website, one can simply
Drag and drop or select a file to import.
Shapefile, GeoJSON and TopoJSON files and Zip archives are supported.
However, in my case it is not working.
Does anyone have any intuition as to what might be going wrong, or a suggestion as to a tool comparable to create a zoomable map out of GeoJSON data?
According to the definition of GeoJSON data, I have what I think constitutes data in that format
Well, you don't have a proper GeoJSON object. Just compare what you've got against the example you've linked. It doesn't even come close. That's why mapshaper doesn't know what to do with the JSON you load into it.
A GeoJSON object with the type "FeatureCollection" is a feature collection object. An object of type "FeatureCollection" must have a member with the name "features". The value corresponding to "features" is an array. Each element in the array is a feature object as defined above.
A feature collection looks like this:
{
"type": "FeatureCollection",
"features": [
// Array of features
]
}
http://geojson.org/geojson-spec.html#feature-collection-objects
A GeoJSON object with the type "Feature" is a feature object. A feature object must have a member with the name "geometry". The value of the geometry member is a geometry object as defined above or a JSON null value. A feature object must have a member with the name "properties". The value of the properties member is an object (any JSON object or a JSON null value). If a feature has a commonly used identifier, that identifier should be included as a member of the feature object with the name "id".
A feature looks like this:
{
"id": "Foo",
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [0, 0]
},
"properties": {
"label": "My Foo"
}
}
http://geojson.org/geojson-spec.html#feature-objects
Here are examples of the different geometry objects a feature can support: http://geojson.org/geojson-spec.html#appendix-a-geometry-examples
Put those two together, it would look like this:
{
"type": "FeatureCollection",
"features": [{
"id": "Foo",
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [0, 0]
},
"properties": {
"label": "My Foo"
}
},{
"id": "Bar",
"type": "Feature",
"geometry": {
"type": "LineString",
"coordinates": [
[100.0, 0.0],
[101.0, 1.0]
]
},
"properties": {
"label": "My Bar"
}
}]
}
That really doesn't look like the JSON you've posted. You'll need to convert that to proper GeoJSON somehow via a custom script or manually. It's a format i've never seen before, sorry to say.
I'm having a hard time trying to get data about a person from Freebase using his social link - by a MQL query.
How could this be done?
Something like:
https://www.googleapis.com/freebase/v1/mqlread?query={
"*":[{}],
"/common/topic/social_media_presence":[{
"value":"http://twitter.com/JustinBieber"
}]
}
Those links are really stored as keys and the links are generated from templates with they key plugged in. You can see all the keys here: https://www.freebase.com/m/06w2sn5?keys=
A modified version of your query would be:
[{
"key": [{
"namespace": {
"id": "/authority/twitter"
},
"value": "JustinBieber"
}],
"*": [{}]
}]
You can do the same thing with other namespaces like /authority/facebook or /authority/musicbrainz as well as the various language wikipedias e.g. /wikipedia/en
I'm not sure how complete the coverage or currency of the social media info is though...