Fetch Latitude Longitude by passing postcodes to maps.google.com using Javascript - asp.net

I have Postcode in my large database, which contains values like SL5 9JH, LU1 3TQ etc.
Now when I am pasting above postcode to maps.google.com it's pointing to a perfect location..
My requirement is like I want to pass post codes to maps.google.com and it should return a related latitude and longitude of that pointed location, that I want to store in my database.
So, most probably there should be some javascript for that... If anybody have another idea regarding that please provide it..
Thanks in advance...

A quick note for those finding this SO answer. The answer by Daniel Vassallo uses the Google Geocoding API V2 which has now been deprecated. The new v3 API uses a request format like this:
http://maps.googleapis.com/maps/api/geocode/output?parameters
An example for a postcode lookup, returning the data in JSON format is:
http://maps.googleapis.com/maps/api/geocode/json?address=SL59JH,+UK&sensor=false
This returns a JSON array that includes the lat and long in results->geometry->location->lat and results->geometry->location->lng
Example response:
{
"results" : [
{
"address_components" : [
{
"long_name" : "SL5 9JH",
"short_name" : "SL5 9JH",
"types" : [ "postal_code" ]
},
{
"long_name" : "Windsor and Maidenhead",
"short_name" : "Windsor and Maidenhead",
"types" : [ "administrative_area_level_2", "political" ]
},
{
"long_name" : "United Kingdom",
"short_name" : "GB",
"types" : [ "country", "political" ]
},
{
"long_name" : "Ascot",
"short_name" : "Ascot",
"types" : [ "postal_town" ]
}
],
"formatted_address" : "Ascot, Windsor and Maidenhead SL5 9JH, UK",
"geometry" : {
"bounds" : {
"northeast" : {
"lat" : 51.39655490000001,
"lng" : -0.66024660
},
"southwest" : {
"lat" : 51.39457330,
"lng" : -0.6624574999999999
}
},
"location" : {
"lat" : 51.39539040,
"lng" : -0.66096740
},
"location_type" : "APPROXIMATE",
"viewport" : {
"northeast" : {
"lat" : 51.39691308029150,
"lng" : -0.6600030697084980
},
"southwest" : {
"lat" : 51.39421511970851,
"lng" : -0.6627010302915021
}
}
},
"types" : [ "postal_code" ]
}
],
"status" : "OK"
}
The API spec is available here: https://developers.google.com/maps/documentation/geocoding/

The technical term for the process you describe is called reverse geocoding. Google offers the The Google Geocoding Web Service New working Google Geocoding Link, where you can do reverse geocoding on the server side, instead of in JavaScript on the client-side.
For example, if you try the following URLs in your browser, you would get back the latitude and longitude of the postcode passed in the q parameter, in CSV format:
http://maps.google.com/maps/geo?q=SL59JH,+UK&output=csv&sensor=false
http://maps.google.com/maps/geo?q=LU13TQ,+UK&output=csv&sensor=false
This is how you would be able to reverse geocode your postcodes in php, for example:
$url = 'http://maps.google.com/maps/geo?q=SL59JH,+UK&output=csv&sensor=false';
$data = #file_get_contents($url);
$result = explode(",", $data);
echo $result[0]; // status code
echo $result[1]; // accuracy
echo $result[2]; // latitude
echo $result[3]; // longitude
Note that as Pekka suggested in another answer, the Google Maps API Terms of Use seem to prohibit the storage of the results, unless the store acts as a cache for data that will used in Google Maps. You may want to get in touch with Google and enquire on the Google Maps API Premier to have more flexible terms of use for your geocoding requirements.

The Ordnance Survey have released the postcode locations on a Creative Commons licence (CC-BY v3, IIRC). It would be a lot less hassle (and a lot clearer legally) to use that instead.
There's even a version with WGS84 (a.k.a. GPS) coordinates mirrored by mySociety

The Google Geocoding API does that, although if I remember correctly, their terms of service forbid local storage of the geocoding results.

I know this is an old question, but just chipping in here with how I managed to achieve the same thing (in PHP, but should be fairly simple):
I had a database of thousands of differently formatted postcodes. I cleaned each and every one of them up, uniformly, with this function and batch updates:
function clean_postcode($postcode)
{
$out = preg_replace("/[^a-zA-Z0-9]/", '',strtoupper($postcode));
if(strlen($out)>3)
{
$out = substr($out, 0, (strlen($out) -3)).' '.substr($out, -3);
}
return $out;
}
Now that all postcodes are formatted uniformly, I downloaded and imported the Ordnance Survey's Code-Point data set (free) - https://www.ordnancesurvey.co.uk/opendatadownload/products.html
I imported all their CSVs into my database in a separate codepoint table. From each CSV I imported the postcode, and the Eastings and Northings values.
I ran the clean_postcode() function again in batch on all the Ordnance Survey data in my DB. About 50% of the postcodes have spaces in, and 50% don't - after this they were uniformly set.
I ran the following PHP script on each and every postcode in the codepoint table and saved the Latitude and Longitude values returned into this table: http://bramp.net/blog/os-easting-northing-to-lat-long
All done! You can now match up and pull a Lat/Lon value based on well-formatted postcodes.
Further reading: http://www.deepbluesky.com/blog/-/converting-os-coodinates-into-longitude-latitude_7/

Check out ALgolia places here. Super fast, Open source and MIT

Related

React Native Firebase, equalTo() is being aplied wrong?

I have this database:
{
"interpreters" : [ null, {
"InterpreterID" : "1",
"Name" : "Pedro",
"languages" : [ "portuguese", "russian", "english", "german" ],
"latitude" : 37.633,
"latitudeDelta" : 0.0143,
"longitude" : -122.345,
"longitudeDelta" : 0.0134
}, {
"InterpreterID" : "2",
"Name" : "Paulo",
"languages" : [ "english", "portuguese" ],
"latitude" : 37.554,
"latitudeDelta" : 0.0143,
"longitude" : -122.245,
"longitudeDelta" : 0.0134
} ]
}
And I'm trying to get only interpreters that match with the language I want. For example, get only interpreters
that speak russian.
I'm doing like this:
database()
.ref()
.child('interpreters')
.orderByChild('languages')
.equalTo('russian')
.on('value', snapshot => {
setInterpreters(snapshot.val())
console.log(snapshot.val())
});
But it returns 'null'.
If I look for orderByChild('Name').equalTo('Pedro') then return Pedros data.
Maybe it's because languages is an array but I'm not sure how to get data from it and couldn't find how to do it.
The forEach method simple didn’t work without any error. The code ran fine but I couldn’t display anything.
I'm following firebase documentation and seems that there is nothing different.
What should I do ?
Thank you in advance.
Firebase doesn't natively support arrays, well it does in form of objects. An array that you send as
["Portuguese", "Russian"]
is converted to
{0: "Portuguese", 1: "Russian"}
Hence your queries don't work as expected. Firebase doesn't support querying inner values of arrays
The workaround
Firebase recommends changing your data to:
{
"portugese": true,
"russian": true
}
and then query as
.orderByChild('languages/portuguese')
.equalTo(true)

Decent data structure for Firebase messaging?

I'm trying to get started with Firebase and I just want to make sure that this data structure is optimized for Firebase.
The conversation object/tree/whatever looks like this:
conversations: {
"-JRHTHaKuITFIhnj02kE": {
user_one_id: "054bd9ea-5e05-442b-a03d-4ff8e763030b",
user_two_id: "0b1b89b7-2580-4d39-ae6e-22ba6773e004",
user_one_name: "Christina",
user_two_name: "Conor",
user_one_typing: false,
user_two_typing: false,
last_message_text: "Hey girl, what are you doing?",
last_message_type: "TEXT",
last_message_date: 0
}
}
and the messages object looks like so:
messages: {
"-JRHTHaKuITFIhnj02kE": {
conversation: "-JRHTHaKuITFIhnj02kE",
sender: "054bd9ea-5e05-442b-a03d-4ff8e763030b",
message: "Hey girl, what are you doing?",
message_type: "TEXT",
message_date: 0
}
}
Is storing the name relative to the user in the conversation object needed, or can I easily look up the name of the user by the users UID on the fly? Other than the name question, is this good? I don't want to get started with a really bad data structure.
Note: Yes, i know the UID for the conversation & message are the same, I got tired of making up variables.
I usually model the data that I need to show in a single screen in a single location in the database. That makes it possible to retrieve that data with a single read/listener.
Following that train of thought it makes sense to keep the user name in the conversation node. In fact, I usually keep the username in each message node too. The latter prevents the need for a lookup, although in this case I might be expanding the data model a bit far for the sake of keep the code as simple as possible.
For the naming of the chat: if this is a fairly standard chat app, then user may expect to have a persistent 1:1 chat with each other, so that every time you and I chat, we end up in the same room. A good approach for accomplishing that in the data model, can be found in this answer: Best way to manage Chat channels in Firebase
I don't think you structured it right. You should bare in mind "What if" complete analysis.
Though, I would recommend structuring it this way (I made it up for fun, not really tested in-terms of performance when getting a huge traffic. but you can always do denormalization to increase performance when needed):
{
"conversation-messages" : {
"--JpntMPN_iPC3pKDUX9Z" : {
"-Jpnjg_7eom7pMG6LDe1" : {
"message" : "hey! Who are you?",
"timestamp" : 1432165992987,
"type" : "text",
"userId" : "user:-Jpnjcdp6YXM0auS1BAT"
},
"-JpnjibdwWpf1k-zS3SD" : {
"message" : "Arya Stark. You?",
"timestamp" : 1432166001453,
"type" : "text",
"userId" : "user:-OuJffgdYY0jshTFD"
},
"-JpnkqRjkz5oT9sTrKYU" : {
"message" : "no one. a man has no name.",
"timestamp" : 1432166295571,
"type" : "text",
"userId" : "user:-Jpnjcdp6YXM0auS1BAT"
}
}
},
"conversations-metadata" : { // to show the conversation list from all users for each user
"-JpntMPN_iPC3pKDUX9Z" : {
"id": "-JpntMPN_iPC3pKDUX9Z",
"date":995043959933,
"lastMsg": "no one. a man has no name.",
"messages_id": "-JpntMPN_iPC3pKDUX9Z"
}
},
"users" : {
"user:-Jpnjcdp6YXM0auS1BAT" : {
"id" : "user:-Jpnjcdp6YXM0auS1BAT",
"name" : "many-faced foo",
"ProfileImg" : "...."
"conversations":{
"user:-Yabba_Dabba_Doo" : {
"conversation_id": "-JpntMPN_iPC3pKDUX9Z",
"read" : false
}
}
},
"user:-Yabba_Dabba_Doo" : {
"id" : "user:-Yabba_Dabba_Doo",
"name" : "Arya Stark",
"ProfileImg" : "...."
"conversations":{
"user:-Jpnjcdp6YXM0auS1BAT" : {
"conversation_id": "-JpntMPN_iPC3pKDUX9Z",
"read" : true
}
}
}
}
}

Firebase: structuring data via per-user copies? Risk of data corruption?

Implementing an Android+Web(Angular)+Firebase app, which has a many-to-many relationship: User <-> Widget (Widgets can be shared to multiple users).
Considerations:
List all the Widgets that a User has.
A User can only see the Widgets which are shared to him/her.
Be able to see all Users to whom a given Widget is shared.
A single Widget can be owned/administered by multiple Users with equal rights (modify Widget and change to whom it is shared). Similar to how Google Drive does sharing to specific users.
One of the approaches to implement fetching (join-style), would be to go with this advice: https://www.firebase.com/docs/android/guide/structuring-data.html ("Joining Flattened Data") via multiple listeners.
However I have doubts about this approach, because I have discovered that data loading would be worryingly slow (at least on Android) - I asked about it in another question - Firebase Android: slow "join" using many listeners, seems to contradict documentation .
So, this question is about another approach: per-user copies of all Widgets that a user has. As used in the Firebase+Udacity tutorial "ShoppingList++" ( https://www.firebase.com/blog/2015-12-07-udacity-course-firebase-essentials.html ).
Their structure looks like this:
In particular this part - userLists:
"userLists" : {
"abc#gmail,com" : {
"-KBt0MDWbvXFwNvZJXTj" : {
"listName" : "Test List 1 Rename 2",
"owner" : "xyz#gmail,com",
"timestampCreated" : {
"timestamp" : 1456950573084
},
"timestampLastChanged" : {
"timestamp" : 1457044229747
},
"timestampLastChangedReverse" : {
"timestamp" : -1457044229747
}
}
},
"xyz#gmail,com" : {
"-KBt0MDWbvXFwNvZJXTj" : {
"listName" : "Test List 1 Rename 2",
"owner" : "xyz#gmail,com",
"timestampCreated" : {
"timestamp" : 1456950573084
},
"timestampLastChanged" : {
"timestamp" : 1457044229747
},
"timestampLastChangedReverse" : {
"timestamp" : -1457044229747
}
},
"-KByb0imU7hFzWTK4eoM" : {
"listName" : "List2",
"owner" : "xyz#gmail,com",
"timestampCreated" : {
"timestamp" : 1457044332539
},
"timestampLastChanged" : {
"timestamp" : 1457044332539
},
"timestampLastChangedReverse" : {
"timestamp" : -1457044332539
}
}
}
},
As you can see, the copies of shopping list "Test List 1 Rename 2" info appears in two places (for 2 users).
And here is the rest for completeness:
{
"ownerMappings" : {
"-KBt0MDWbvXFwNvZJXTj" : "xyz#gmail,com",
"-KByb0imU7hFzWTK4eoM" : "xyz#gmail,com"
},
"sharedWith" : {
"-KBt0MDWbvXFwNvZJXTj" : {
"abc#gmail,com" : {
"email" : "abc#gmail,com",
"hasLoggedInWithPassword" : false,
"name" : "Agenda TEST",
"timestampJoined" : {
"timestamp" : 1456950523145
}
}
}
},
"shoppingListItems" : {
"-KBt0MDWbvXFwNvZJXTj" : {
"-KBt0heZh-YDWIZNV7xs" : {
"bought" : false,
"itemName" : "item",
"owner" : "xyz#gmail,com"
}
}
},
"uidMappings" : {
"google:112894577549422030859" : "abc#gmail,com",
"google:117151367009479509658" : "xyz#gmail,com"
},
"userFriends" : {
"xyz#gmail,com" : {
"abc#gmail,com" : {
"email" : "abc#gmail,com",
"hasLoggedInWithPassword" : false,
"name" : "Agenda TEST",
"timestampJoined" : {
"timestamp" : 1456950523145
}
}
}
},
"users" : {
"abc#gmail,com" : {
"email" : "abc#gmail,com",
"hasLoggedInWithPassword" : false,
"name" : "Agenda TEST",
"timestampJoined" : {
"timestamp" : 1456950523145
}
},
"xyz#gmail,com" : {
"email" : "xyz#gmail,com",
"hasLoggedInWithPassword" : false,
"name" : "Karol Depka",
"timestampJoined" : {
"timestamp" : 1456952940258
}
}
}
}
However, before I jump into implementing a similar structure in my app, I would like to clarify a few doubts.
Here are my interrelated questions:
In their ShoppingList++ app, they only permit a single "owner" - assigned in the ownerMappings node. Thus no-one else can rename the shopping list. I would like to have multiple "owners"/admins, with equal rights. Would such a keep-copies-per-user structure still work for multiple owner/admin users, without risking data corruption/"desynchronization" or "pranks"?
Could data corruption arise in scenarios like this: User1 goes offline, renames Widget1 to Widget1Prim. While User1 is offline, User2 shares Widget1 to User3 (User3's copy would not yet be aware of the rename). User1 goes online and sends the info about the rename of Widget1 (only to his own and User2's copies, of which the client code was aware at the time of the rename - not updating User3's copy). Now, in a naive implementation, User3 would have the old name, while the others would have the new name. This would probably be rare, but still worrying a bit.
Could/should the data corruption scenario in point "2." be resolved via having some process (e.g. on AppEngine) listening to changes and ensuring proper propagation to all user copies?
And/or could/should the data corruption scenario in point "2." be resolved via implementing a redundant listening to both changes of sharing and renaming, and propagating the changes to per-user copies, to handle the special case? Most of the time this would not be necessary, so it could result in performance/bandwidth penalty and complicated code. Is it worth it?
Going forward, once we have multiple versions deployed "in the wild", wouldn't it become unwieldy to evolve the schema, given how much of the data-handling responsibility lies with the code in the clients? For example if we add a new relationship, that the older client versions don't yet know about, doesn't it seem fragile? Then, back to the server-side syncer-ensurerer process on e.g. AppEngine (described in question "3.") ?
Would it seem like a good idea, to also have a "master reference copy" of every Widget / shopping-list, so as to give good "source of truth" for any syncer-ensurerer type of operations that would update per-user copies?
Any special considerations/traps/blockers regarding rules.json / rules.bolt permissions for data structured in such a (redundant) way ?
PS: I know about atomic multi-path updates via updateChildren() - would definitely use them.
Any other hints/observations welcome. TIA.
I suggest having only one copy of a widget for the entire system. It would have an origin user ID, and a set of users that have access to it. The widget tree can hold user permissions and change history. Any time a change is made, a branch is added to the tree. Branches can then be "promoted" to the "master" kind of like GIT. This would guarantee data integrity because past versions are never changed or deleted. It would also simplify your fetches... I think :)
{
users:[
bob:{
widgets:[
xxx:{
widgetKey: xyz,
permissions: *,
lastEdit...
}
]
}
...
]
widgets:[
xyz:{
masterKey:abc,
data: {...},
owner: bob,
},
...
]
widgetHistory:[
xyz:[
v1:{
data:{...},
},
v2,
v3
]
123:[
...
],
...
]
}

Different results between browser and R mapdist (google API)

I am using mapdist function from the ggmap package in R and I have a problem with results (more specifically by using the ReadLines function).
I give 2 addresses to the function, and then it connects to the Google API to return the distance between the 2 addresses.
I give an example below without calling my real addresses.
The result is empty (Status: "Not_found") with R.
$destination_addresses
[1] "Adresse numero 1, France"
$origin_addresses
[1] ""
$rows
$rows[[1]]
$rows[[1]]$elements
$rows[[1]]$elements[[1]]
$rows[[1]]$elements[[1]]$status
[1] "NOT_FOUND"
$status
[1] "OK"
Given that I was sure of my address, I copied the URL (to connect on Google API) on my browser (I tried on Mozilla, Google Chrome and Internet explorer), and I was surprised to obtain a non-empty result.
{
"destination_addresses" : [ "Adresse numero 1, France" ],
"origin_addresses" : [ "Adresse2, France" ],
"rows" : [
{
"elements" : [
{
"distance" : {
"text" : "21,0 km",
"value" : 21045
},
"duration" : {
"text" : "17 minutes",
"value" : 1003
},
"status" : "OK"
}
]
}
],
"status" : "OK"
}
If I had only one address, the solution would be obvious. My problem is I have approximately 100 addresses with empty results and I can not check all of them.
How could you explain that the return result of ReadLines is not the same that the results on my webpage ?
Thank you for helping me.
The troubles I usually find with ggmap + R is the API does not understand the format.
I always fix this trouble checking the way google writes the address while using maps.googles.com and afterwards I make it reproductible.
I could try by my own if I could check it with real data, although you did not write it.
On the other hand you can subset all the "rows" google didn't give you the data back and try again.

How cfs s3 uploading works

I've previously used slingshot and the process it pretty simple: we upload the image and it returns the uploaded url of the s3 bucket.
Now I want to resize the image and need to perform some operations on the image, so I switched to the cfs:s3 package. But when I try to upload the image, it returns some record with no url and in the db it stores it as:
{
"_id" : "Rwa7Xo65pv6cAP2aY",
"copies" : {
"thumbs" : {
"name" : "306032-facebook.jpg",
"type" : "image/jpeg",
"size" : 4262,
"key" : "thumbs/Rwa7Xo65pv6cAP2aY-306032-facebook.jpg",
"updatedAt" : ISODate("2015-02-14T06:44:04.476Z"),
"createdAt" : ISODate("2015-02-14T06:44:04.476Z")
}
},
"original" : {
"name" : "306032-facebook.jpg",
"updatedAt" : ISODate("2015-01-30T09:48:58.000Z"),
"size" : 4262,
"type" : "image/jpeg"
},
"uploadedAt" : ISODate("2015-02-14T06:43:59.062Z")
}
How can I get the URL from this record? (I suppose it is key in thumbs) Is it linking my server url to amazon s3 url?
What are the advantages of this method over slingshot?
How do I know the upload is completed? I can't figure out any ui helpers, are there any reactive helpers to track download percentage?
The URL will be your s3 end point (e.g. s3-us-west-2.amazonaws.com/) + the key. I suggest you to create a constant for your end point and a register helper to return the URL.
Something like this
Template.registerHelper('THUMBS_URL', function(key){
return S3_ENDPOINT + key;
})
I haven't used slingshot yet so can't comment on it.
There is a isUploaded helper function in collectionFS.
https://github.com/CollectionFS/Meteor-CollectionFS#isuploaded

Resources