I tried to find the states of USA with Wikidata API but there is no results.
For example: http://wdq.wmflabs.org/api?q=claim[150:30]
P150 - contains administrative territorial entity
Q30 - United States of America
What am I doing wrong?
You need all Wikidata items which have P31 with value Q35657: claim[31:35657]
P31 - instance of
Q35657 - state of the United States
With SPARQL, you can do that
SELECT ?state ?stateLabel
WHERE {
?state wdt:P31 wd:Q35657.
SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
try it here
Related
I have a cloud function that is subscribed to messages of a pubsub topic and scrapes the given url (which is part of the message) using puppeteer. I set the region to europe-west3 (my firebase project is based in this region as well) since I want to make sure that scraping is done on a server in this region (scraping the url from the US will yield different results). However, judging from the scraping output, the function is still run on a US server.
The Cloud Functions location documentation seems to imply that by setting the region I can determine the location where a given function runs.
I have also gone into the Firebase Console for my project and verified that the functions have the correct location.
Am I missing something here? Is it possible to specify the region where my web scraping logic should be executed?
exports.updateDocWithScrapingData = functions.runWith({ memory: '1GB', timeoutSeconds: 120 }).region('europe-west3').pubsub.topic('myTopic').onPublish(async (message) => {
const urlMap = JSON.parse(Buffer.from(message.data, 'base64').toString());
// Scraping is done using puppeteer in separate file
const scrapedData = await scraper.fetchData(urlMap['url']);
const docId = urlMap['id'];
const refDoc = db.collection('myCollection').doc(docId);
const doc = await refDoc.get();
if (doc.exists) {
// Update doc using scrapedData
} else {
// Create doc using scrapedData
}
return {
response: 'Success',
}
});
EDIT:
This is my scraping function in scraper.js:
const fetchData = async (url) => {
const browser = await puppeteer.launch({
headless: true,
args: ['--no-sandbox', '--disable-setuid-sandbox'],
});
const page = await browser.newPage();
await page.goto(url);
await page.waitForTimeout(2000);
const games = await page.evaluate(() => {
const gamesArr = Array.from(document.querySelector('.grid-wrapper').querySelectorAll('.grid-event-wrapper'));
return gamesArr.map(game => game.innerText);
});
await browser.close();
console.log(games);
return games;
}
This is an example url I am trying to scrape: https://sports.bwin.de/de/sports/basketball-7/wetten/nordamerika-9/nba-6004
Expected output for europe-west3 (Function should be running in Frankfurt, Germany):
['Chicago Bulls
#
Indiana Pacers
Morgen / 01:05
KON…95
-5,5
1.87
▲ 224,5
1.91
▼ 224,5
1.91
2.85
1.44', 'Brooklyn Nets
#
Sacramento Kings
Morgen / 04:05
K…87
+3,5
1.95
▲ 242,5
1.87
▼ 242,5
1.95
1.57
2.45']
This is the actual output from the cloud function:
['Chicago Bulls
#
Indiana Pacers
Morgen / 01:05
KON…05
-5,5
-115
▲ 224,5
-110
▼ 224,5
-110
+185
-225', 'Brooklyn Nets
#
Sacramento Kings
Morgen / 04:05
K…15
+3,5
-105
▲ 242,5
-115
▼ 242,5
-105
-175
+145']
Notice the difference between the betting odds. US uses Money Line whereas in Germany for example you simply get the odds as a multiple of 1.
Yes, the thread exactly describes the reason why you are obtaining values from US servers.
All the IP addresses Google provides to GCP user are registered with ARIN
under the Google HQ in Mountain View, California (in other words SWIP to be
Mountain View, CA). So, all geolocations lookup of the IP addresses will
resolve to the United States even though the actual server is located
somewhere else. In addition to that, in GCP, it's not uncommon to remap a
block of IPs from one location to another, especially given the elasticity of
IP addresses for the GCP and they way they are recycled or reused.
I'm trying to make a post to the /v1/customers/:customer_id/deliveries endpoint using firebase. I'm sending the query params from an angular application.
Query Object
{
pickup_name: 'Patrick O',
pickup_phone_number: '619-***-****',
dropoff_name: 'Wendell And Chad',
dropoff_phone_number: '619-***-****',
manifest: 'Cheese',
quote_id: 'Mc26u6TwH_qKkF',
dropoff_address: '4606 Market Street, San Diego, CA',
pickup_address: '1041 Market Street, San Diego, CA'
}
Response (Firebase Function)
Unhandled error { StatusCodeError: 400 - {"kind":"error","code":"invalid_params","message":"The parameters of your request were invalid.","params":{"__all__":"invalid price quote"}}
...
name: 'StatusCodeError',
statusCode: 400,
message: '400 - {"kind":"error","code":"invalid_params","message":"The parameters of your request were invalid.","params":{"__all__":"invalid price quote"}}',
error:
{ kind: 'error',
code: 'invalid_params',
message: 'The parameters of your request were invalid.',
params: { __all__: 'invalid price quote' } }, ...
I haven't encountered the error for __all__ in the Postmates Api Docs. Thank you for your time and consideration.
The issue was resolved by removing the quote_id property from the query params
Query Object
{
pickup_name: 'Patrick O',
pickup_phone_number: '619-***-****',
dropoff_name: 'Wendell And Chad',
dropoff_phone_number: '619-***-****',
manifest: 'Cheese',
dropoff_address: '4606 Market Street, San Diego, CA',
pickup_address: '1041 Market Street, San Diego, CA'
}
I haven't tested it but I believe the quote_id property can be used but the string has to be the full quote id.
quote_id: 'dqt_Mc26u6TwH_qKkF', as oppose to quote_id: 'Mc26u6TwH_qKkF', like I had it.
I will update the answer after verifying the latter
I am looking to use HEREs geocoding service to locate the Lat and Lon of a place based on a UK postcode. At the moment my request will return a rough location even though I have provided a full postcode.
The old "geocode" API that I used previously, would return relevant results however this has been put into maintenance and replaced with the "geocode and search" API. This new API seems like it just looks through a list of stored points of interest within HERE’s database and returns the closest it can to what you have searched for, rather than trying to find the exact location entered.
How can I get more accurate results using the below request? Bare in mind that I will only have access to the postcode.
https://geocode.search.hereapi.com/v1/geocode?q={postCode}&apiKey={key}
At the moment I receive a response similar to the below using postcode PE1 1QL. It should be pointing to a car park, however if you enter the lat and lon returned from the API into a map E.g Google Maps, it gives you a more general location, rather than an accurate one.
{
"title": "PE1 1, Peterborough, England",
"id": "here:cm:namedplace:22221149",
"resultType": "locality",
"localityType": "postalCode",
"address": {
"label": "PE1 1, Peterborough, England",
"countryCode": "GBR",
"countryName": "England",
"county": "Cambridgeshire",
"city": "Peterborough",
"postalCode": "PE1 1"
},
"position": {
"lat": 52.57362,
"lng": -0.24219
},
"mapView": {
"west": -0.23515,
"south": 52.56739,
"east": -0.25194,
"north": 52.57984
},
"scoring": {
"queryScore": 0.67,
"fieldScore": {
"postalCode": 0.95
}
}
},
I would expect the Lat and Lng to be much closer to the postcode entered than the above example.
Regarding on this release notes https://developer.here.com/documentation/geocoding-search-api/release_notes/topics/known-issues.html
you can read "High precision postal codes are not yet supported":
Known Issues
The following table lists issues known to be present in the current release.
Search for intersections is not yet supported
Search by telephone numbers is not yet supported
Political views are not yet supported. All views are “International”
Places detail views are not yet supported
High precision postal codes are not yet supported
The Geocoder API 6.2 will be supported at least until end of 2020 (maybe more) and "Maintenance" in documentation means: no new features.
We are using the places/search endpoint to look up addresses.
Being based in Ontario, a lot of addresses have French and English version.
For example, looking for 360 Lisgar Ottawa, on will return 2 addresses in French and 2 in English (https://places.demo.api.here.com/places/v1/discover/search?at=45%2C-75&q=360+Lisgar%2C+Ottawa%2C+on&addressFilter=countryCode%3DCAN&Accept-Language=en%3Bq%3D1%2Cfr%3Bq%3D0.1&app_id=DemoAppId01082013GAL&app_code=AJKnXv84fjrb0KIHawS0Tg).
We have been trying all kinds of combination of the "Accept-Language" header parameters, with different values and weight, and it always returns the 4 values. Is there any other way to only get the addresses in French or English?
We could filter in our code after, if there was any language field on the address returned, but we couldn't find any reference about it in the documentation. Is there any ways to do it?
Thanks,
Rene
The Accept-Language header is used in the here-api to indicate preference for language. However, if we have no translation/transliteration for a result in that language, we use a fallback mechanism where we try to return results in a language that would be intelligible to the user. In your case, I suspect that there are only French entries for some of the addresses and only English entries for others, which is why you get the same results no matter what you set the "Accept-Language" header to. There's no way to filter results so that if we don't have a translation in your preferred language then we don't return the result at all.
This might be worth trying as I can specify the language using the HERE Geocoder REST API by adding a parameter (the same format as the Google API) at the end:
&language=en
or
&language=ar
I work in Dubai so my other desired language is Arabic. See examples below.
Example REST API Call for English
https://geocoder.cit.api.here.com/6.2/geocode.json?app_id={your app id}&app_code={your app code}&searchText=1 Prince Fawaz St, Al Khobar&language=en
Example extract from the JSON return
"Address": {
"Label": "1 Prince Ahmad Street, Madinat Al Ummal, 34441 Al Khobar, Saudi Arabia",
"Country": "SAU",
"County": "Eastern Province",
"City": "Al Khobar",
"District": "Madinat Al Ummal",
"Street": "Prince Ahmad Street",
"HouseNumber": "1",
"PostalCode": "34441",
"AdditionalData": [
{
Example REST API Call for Arabic
https://geocoder.cit.api.here.com/6.2/geocode.json?app_id={your app id}&app_code={your app code}&searchText=1 Prince Fawaz St, Al Khobar&language=ar
Example extract from JSON return
"Address": {
"Label": "\u202e\u202a1\u202c شارع الأمير أحمد, مدينة العمال, \u202a34441\u202c الخبر, السعودية\u202c",
"Country": "SAU",
"County": "المنطقة الشرقية",
"City": "الخبر",
"District": "مدينة العمال",
"Street": "شارع الأمير أحمد",
"HouseNumber": "1",
"PostalCode": "34441",
"AdditionalData": [
{
I am trying to pull basic information from various websites such as a title and description. I can successfully get a response object and what I get is the entire website HTML. I have been playing around with the options in the HTTP.call method but can't figure out how to only return exactly what I want from the response object. Here are the two elements I am trying to get:
<meta property="og:description" content="Our unique teaching style lets students develop their creative potential while learning solid computing skills.">
and
<meta property="og:site_name" content="Goldsmiths, University of London">
I could get the title easy enough by searching the result for <title></title>, but there must be a better way using params or data in the call method options.
Meteor.methods({
getInfo: function (url) {
HTTP.call('GET', url, {}, function (error, result) {
if (!error) {
//console.log(result);
var titleStart = result.content.toLowerCase().indexOf('<title>'),
titleEnd = result.content.toLowerCase().indexOf('</title>'),
titleText = result.content.substring(titleStart + '<title>'.length, titleEnd)}
Check out the meteor-scrape package.
https://github.com/Anonyfox/meteor-scrape
Here is an example of how you may use it:
# scrape any website
websiteData = Scrape.website "http://example.com/article"
Results in:
{
title: 'The Avengers (2012 film)'
lang: 'en'
descriptions: [ '2012 superhero film produced by Marvel Studios' ]
tags: [ 'avengers' ]
url: 'http://en.wikipedia.org/wiki/The_Avengers_(2012_film)'
summary: '<p><i><b>Marvel\'s The Avengers</b></i> (classified under the name <i><b>Marvel Avengers Assemble</b></i> in the United Kingdom and Ireland), or simply <i><b>The Avengers</b></i>, is a 2012 American superhero film based on the Marvel Comics superhero team of the same name, produced by Marvel Studios and distributed by Walt Disney Studios Motion Pictures.<sup class="reference plainlinks nourlexpansion" id="ref_1">1</sup> It is the sixth installment in the Marvel Cinematic Universe. The film was written [...]'
meta:
caption: 'Theatrical release poster'
director: '[Joss Whedon](http://en.wikipedia.org/wiki/Joss_Whedon)'
producer: '[Kevin Feige](http://en.wikipedia.org/wiki/Kevin_Feige)'
screenplay: 'Joss Whedon'
based: '[The Avengers](http://en.wikipedia.org/wiki/Avengers_(comics))'
music: '[Alan Silvestri](http://en.wikipedia.org/wiki/Alan_Silvestri)'
cinematography: '[Seamus McGarvey](http://en.wikipedia.org/wiki/Seamus_McGarvey)'
studio: '[Marvel Studios](http://en.wikipedia.org/wiki/Marvel_Studios)'
runtime: '143 minutes'
country: 'United States'
language: 'English'
budget: '$220 million'
gross: '$1.518 billion'
}