Right now it looks like a mystery. Please help me in solving it.
I use iTunes public API to fetch an album: "Metallica" by Metallica (see it in browser: US region, MV region). I construct the following URLs to fetch it via API:
US region https://itunes.apple.com/lookup?id=579372950&country=US&entity=album - works
MV region https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album - doesn't work
Here's the actual behaviour I observe:
If I query GET https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album in a Spring app (using RestTemplate + Jackson HttpMessageConverter) I get an empty response:
{
"resultCount":0,
"results": []
}
If I navigate to https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album in a browser I get a file download prompt. The file contains an empty response:
{
"resultCount":0,
"results": []
}
If I query API using HttpPie http get https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album I get a non-empty response !!!
{
"resultCount": 1,
"results": [
{
"amgArtistId": 4906,
"artistId": 3996865,
"artistName": "Metallica",
"artistViewUrl": "https://music.apple.com/us/artist/metallica/3996865?uo=4",
"artworkUrl100": "https://is1-ssl.mzstatic.com/image/thumb/Music/v4/0b/9c/d2/0b9cd2e7-6e76-8912-0357-14780cc2616a/source/100x100bb.jpg",
"artworkUrl60": "https://is1-ssl.mzstatic.com/image/thumb/Music/v4/0b/9c/d2/0b9cd2e7-6e76-8912-0357-14780cc2616a/source/60x60bb.jpg",
"collectionCensoredName": "Metallica",
"collectionExplicitness": "notExplicit",
"collectionId": 579372950,
"collectionName": "Metallica",
"collectionPrice": 9.99,
"collectionType": "Album",
"collectionViewUrl": "https://music.apple.com/us/album/metallica/579372950?uo=4",
"copyright": "℗ 1991 Blackened Recordings",
"country": "USA",
"currency": "USD",
"primaryGenreName": "Metal",
"releaseDate": "1991-08-12T07:00:00Z",
"trackCount": 13,
"wrapperType": "collection"
}
]
}
I tried it multiple times and the results seem to be consistent. I compared the requests and they seem to be identical.
Why does iTunes respond differently to different clients? I can't understand. What important detail am I missing?
Similar questions:
Spring RestTemplate getForObject URL not working for Apple iTunes - there's another problem (double encoding of the whitespace character).
This problem happens to the following regions (it's a complete list):
LI https://itunes.apple.com/lookup?id=579372950&country=LI&entity=album
MV https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album
MM https://itunes.apple.com/lookup?id=579372950&country=MM&entity=album
ET https://itunes.apple.com/lookup?id=579372950&country=ET&entity=album
RS https://itunes.apple.com/lookup?id=579372950&country=RS&entity=album
I spotted a difference:
http get 'https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album' -> empty response
curl 'https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album' -> empty response
http get https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album -> 1 album in response
curl https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album -> 1 album in response
if I don't use quotes around URL, the request is interpreted as GET https://itunes.apple.com/lookup?id=579372950. the default country is US and therefore I see 1 US album in response.
Related
I am trying to interface with the Chroma SDK released by Razer and have been running into some issues. Following the documentation that razer provides, I have been trying to change the color of my RGB mouse for a while now, and hope that someone has the answer for me. I can successfully check to see if the ChromaSDK is working through:
url = 'http://localhost:54235/razer/chromasdk'
x = requests.get(url)
print(x.text)
Then, I can initialize the connection by sending a post to the URL, following the template given on their website:
data = {
"title": "Razer Chroma SDK RESTful Test Application",
"description": "This is a REST interface test application",
"author": {
"name": "Chroma Developer",
"contact": "www.razerzone.com"
},
"device_supported": [
"keyboard",
"mouse",
"headset",
"mousepad",
"keypad",
"chromalink"],
"category": "application"
}
x = requests.post(url, json=data)
print(x.text)
This post request returns:
{"sessionid":55105,"uri":"http://localhost:55105/chromasdk"}
Then, since the connection is initialized, I SHOULD be able to change the colors of the connected Razer devices using endpoints such as /mouse or /headset. This is where it gets funky, if I were to use the url(s): http://localhost:54235/chromasdk/mouse, http://localhost:54235/razer/mouse, or http://localhost:54235/mouse then I get the error "Not Implemented", whereas if I use the URI provided by the previous post request and tag /mouse onto the end of it, it get this error:
{"error":"Expecting a string","result":87}
Or if I use http://localhost:54235/razer/chromasdk/mouse, I get:
{"author":null,"category":null,"description":null,"device_supported":null,"error":"The parameter is incorrect.","result":87,"title":null}
The endpoints SHOULD follow the URL http://localhost:54235/razer/chromasdk, and I am following the documentation to a T, so what am I doing wrong??
I am trying to scrape AirBNB by plain HTTP requests and noticed something.
Let's say we use this search string: "New York, New York, United States".
The simplest working request (striped off from unnecessary headers and fields) I can use to get the desired results is this:
GET /api/v3/ExploreSections?operationName=ExploreSections&locale=en¤cy=USD&variables=%7B%22isInitialLoad%22%3Atrue%2C%22hasLoggedIn%22%3Afalse%2C%22cdnCacheSafe%22%3Afalse%2C%22source%22%3A%22EXPLORE%22%2C%22exploreRequest%22%3A%7B%22metadataOnly%22%3Afalse%2C%22version%22%3A%221.8.3%22%2C%22itemsPerGrid%22%3A20%2C%22placeId%22%3A%22ChIJOwg_06VPwokRYv534QaPC8g%22%2C%22query%22%3A%22New%20York%2C%20New%20York%2C%20United%20States%22%2C%22cdnCacheSafe%22%3Afalse%2C%22screenSize%22%3A%22large%22%2C%22isInitialLoad%22%3Atrue%2C%22hasLoggedIn%22%3Afalse%7D%2C%22removeDuplicatedParams%22%3Atrue%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%2282cc0732fe2a6993a26859942d1342b6e42830704b1005aeb2d25f78732275e7%22%7D%7D HTTP/2
Host: www.airbnb.com
X-Airbnb-Api-Key: d306zoyjsyarp7ifhu67rjxn52tv0t20
Accept-Encoding: gzip, deflate
At this point, that API key is pretty much public, so not a concern.
The readable content of the "variables" parameter is this:
{
"isInitialLoad": true,
"hasLoggedIn": false,
"cdnCacheSafe": false,
"source": "EXPLORE",
"exploreRequest": {
"metadataOnly": false,
"version": "1.8.3",
"itemsPerGrid": 20,
"placeId": "ChIJOwg_06VPwokRYv534QaPC8g",
"query": "New York, New York, United States",
"cdnCacheSafe": false,
"screenSize": "large",
"isInitialLoad": true,
"hasLoggedIn": false
},
"removeDuplicatedParams": true
}
The readable content of the "extensions" parameter is this:
{
"persistedQuery": {
"version": 1,
"sha256Hash": "82cc0732fe2a6993a26859942d1342b6e42830704b1005aeb2d25f78732275e7"
}
}
I am trying to figure out where that hash comes from.
It seems it's calculated from a GraphQL query but I don't know anything else and there is no documentation about it.
Any help?
I had the same issue (wanted to get the prices) and after investigating in the HAR files that you can get with Chrome, I found out that you get this value from a Javascript file called PdpPlatformRoute.xxx.js
The steps to get this hash are simply to load the file PdpPlatformRoute.xxx.js, then to parse the file to get an "operationId".
If this helps, this is how I did this.
// contentPage is the HTML content of the listing page (e.g. https://www.airbnb.com/rooms/1234567)
function getPdpPlatformRouteUrl(contentPage) {
return 'https://a0.muscache.com/airbnb/static/packages/web/en/frontend/gp-stays-pdp-route/routes/' + `${contentPage}`.match(/(PdpPlatformRoute\.\w+\.\js)/)?.[1];
}
// textContent is the JS content that you get when you fetch the previously found URL
function getSha256(textContent) {
return `${textContent}`.match(/name:'StaysPdpSections',type:'query',operationId:'(.*)'/)?.[1];
}
I am trying to fetch the posts of the company from the api, I have already applied to the marketing development platform and it was approved. I already got the token with the scope: r_organization_social and I'm calling the /shares api:
https://api.linkedin.com/v2/shares?q=owners&owners=urn:li:organization:{company_ID}&sharesPerOwner=100&count=25&sharesPerOwner=10
But I'm getting the following response:
{
"paging": {
"start": 0,
"count": 25,
"links": [
{
"type": "application/json",
"rel": "next",
"href": "/v2/shares?count=25&owners=urn%3Ali%3Aorganization%3A{company_ID}&q=owners&sharesPerOwner=10&sharesPerOwner=100&start=0"
}
],
"total": 242
},
"elements": []
}
I tried to change the query params and it's still the same
This end-point worked for me:
https://api.linkedin.com/v2/ugcPosts?q=authors&authors=List(urn%3Ali%3Aorganization%3A<ID_ORGANIZATION>)
See documentation: https://learn.microsoft.com/en-us/linkedin/marketing/integrations/community-management/shares/ugc-post-api?tabs=http#sample-request-6
Disclaimer: I've no access to the linkedin API and couldn't test. But these are some things I noticed:
Your url contains two times the paramater sharesPerOwner, try removing one.
In the docs it's recommended to set the sharesPerOwner to 1000 and the count to 50. I'd also include the start paramater, just to make sure:
Maybe try something like this:
GET https://api.linkedin.com/v2/shares?q=owners&owners=urn:li:organization:{id}&sharesPerOwner=1000&count=50&start=0
From the api-docs(https://learn.microsoft.com/en-us/linkedin/marketing/integrations/community-management/shares/share-api?tabs=http#find-shares-by-owner): "Note that the pagination excludes UGC and Direct Sponsored Content (DSC) posts". Make sure that the owner you are testing contains posts.
If this doesn't work. Could you provide some information on how you are sending the request? Have you tried accessing other parts of the api?
I'm using Google Analytics to track data across multiple domains in a single profile.
By default, reporting only shows the path, not the full URL. This makes it quite confusing where multiple pages on our different domains have the same paths (e.g. '/index' or '/about').
To get round this, I've implemented the filter advised by Google to display the full URL in reporting:
Filter Type: Custom filter > Advanced
Field A: Hostname Extract A: (.*)
Field B: Request URI Extract: (.*)
Output To: Request URI Constructor: $A1$B1
This works just fine ; the only downside is that using the 'preview link' button in the reporting always appends the domain, resulting in a 404 error.
....clicking the 'link preview' icon results in......
Does anyone know a way around this ; either by preventing GA from appending the domain or a better way of displaying the full URLs in reporting?
Thanks Eike - I took your advice and wrote a small browser extension for Chrome. Obviously this isn't an essential, but I wanted to address it as our marketing team use the feature so frequently.
The manifest json :
{
"manifest_version": 2,
"name": "Analytics cross-domain link shortcut",
"version": "1.0",
"description": "Makes the links shortcuts in analytics work when using a 'full url' filter!",
"content_scripts":
[
{
"matches": ["*://*/*"],
"js": ["myscript.js"],
"run_at": "document_start"
}
]
}
And the script:
if (window.opener && document.referrer == "") {
var currentLocation = window.location.href;
if(currentLocation.indexOf("www.appendedurl.com") > -1) {
var newLocation = currentLocation.substr(30); // where '30' is the length of the appended URL
window.location.href = "http://"+newLocation;
}
}
So it's essentially just snipping off the appended URL (if present) on freshly opened popup windows.
I am trying to retrieve events from a Google calendar. I am calling this URL:
GET https://www.googleapis.com/calendar/v3/calendars/riteshmehandiratta%40gmail.com/events
Authorization: Bearer ya29.AHES6ZSvz3O7V2p7z_k_ZWAiUZqWX35Eyx5V_J4XC5pVSNxLjS6CNzQ
and I am getting this response:
{
"kind": "calendar#events",
"etag": "\"ZrhdJMCgpoUK_a5fT7XOC6xn46g/_4_AQ2FVvAtAEiDFDj8_pH_usqc\"",
"summary": "riteshmehandiratta#gmail.com",
"updated": "2013-02-18T03:45:37.728Z",
"timeZone": "Asia/Calcutta",
"accessRole": "owner",
"defaultReminders": [
{
"method": "email",
"minutes": 10
},
{
"method": "popup",
"minutes": 10
}
],
"nextPageToken": "CigKGnR0ZG1xbjAyNHQ1Y3RycjYwY2x0ZGxtcTk0GAEggIDA28aNo-cT"
}
There are many events in the calendar for the past, future and for the current date.
Why its not giving the calendar event list?
there is definitely something weird here, in the response you are posting, i see you are getting the nextPageToken tag, so there actually ARE more results, you can call the next result "page", using a similar request with the variable pageToken added, something like
GET https://www.googleapis.com/calendar/v3/calendars/riteshmehandiratta%40gmail.com/events?pageToken=CigKGnR0ZG1xbjAyNHQ1Y3RycjYwY2x0ZGxtcTk0GAEggIDA28aNo-cT
that way you'll get more results.
Anyway this is weird because the nextPageToken doesn't show up if you don't limit the query with some variable like maxResults. Please check if you are not limiting the query somewhere else.
Maybe you can find useful for making test, the documentation page, that has at the bottom a request form that can build the query for you.
Hope this helps, regards
I saw the same problem. Sometimes the list API will return an empty item list with a nextPageToken, even if I did not limit the number of responses. The solution is to check if there is a nextPageToken, and make another request for the next page until you get a response without a nextPageToken.
I looped through the requests until there wasn't a nextPage token, appending the event lists to a local list, and this provided a full list of the events.