I'm currently using the Computer Vision API to get image dimensions and also to detect color & adult content.
The API returns back the correct metadata containing the width, height, and format, but all the values for Color or Adult Content come back as default values (zeros/null/false/"None").
If it makes any difference, I'm calling the service in US West which is listed as having these features enabled. Also, I specify that I want these features in the request, which is why I receive the objects for these two features.
How do I get this information to be populated accordingly?
Example request I just used with today's Bing image. I'm using the Microsoft.Azure.CognitiveServices.Vision.ComputerVision package from nuget which contains the AnalyzeImageAsync method.
POST https://westus.api.cognitive.microsoft.com/vision/v1.0/analyze?
visualFeatures=Adult%2CColor HTTP/1.1
Ocp-Apim-Subscription-Key: <REMOVED>
User-Agent: FxVersion/4.6.26515.07 Microsoft.Azure.CognitiveServices.Vision.ComputerVision.ComputerVisionAPI/1.1.0.0
x-ms-request-root-id: 5e7823bb-4cac1f594af57fcf
x-ms-request-id: |5e7823bb-4cac1f594af57fcf.2.
Request-Id: |5e7823bb-4cac1f594af57fcf.2.
Content-Type: application/json; charset=utf-8
Content-Length: 129
Host: westus.api.cognitive.microsoft.com
{
"url": "https://outerpulsemedia.blob.core.windows.net/uploads/images/0be893af-9edc-416a-a779-929d2e2d1428-1529100079.jpg"
}
Response:
{
"adult": {
"isAdultContent": false,
"adultScore": 0.0,
"isRacyContent": false,
"racyScore": 0.0
},
"color": {
"dominantColorForeground": "None",
"dominantColorBackground": "None",
"dominantColors": [],
"accentColor": "000000",
"isBwImg": false
},
"requestId": "5b57186d-9976-4af7-97fd-e06b33fb0ecb",
"metadata": {
"height": 1200,
"width": 1920,
"format": "Jpeg"
}
}
The JPEG image is not, strictly speaking, correctly encoded. Specifically the EOI (end of image) marker is missing. The image should end with FF D9, whereas your image is ending simply in FF. This is why the image classifier is returning default values instead of true values.
While a case could be made that either the (a) return response is deceptive, or (b) the API should have returned a 400 Bad Request, it is probably more beneficial for you to try and understand how the image got truncated by a byte.
Related
I am trying to scrape AirBNB by plain HTTP requests and noticed something.
Let's say we use this search string: "New York, New York, United States".
The simplest working request (striped off from unnecessary headers and fields) I can use to get the desired results is this:
GET /api/v3/ExploreSections?operationName=ExploreSections&locale=en¤cy=USD&variables=%7B%22isInitialLoad%22%3Atrue%2C%22hasLoggedIn%22%3Afalse%2C%22cdnCacheSafe%22%3Afalse%2C%22source%22%3A%22EXPLORE%22%2C%22exploreRequest%22%3A%7B%22metadataOnly%22%3Afalse%2C%22version%22%3A%221.8.3%22%2C%22itemsPerGrid%22%3A20%2C%22placeId%22%3A%22ChIJOwg_06VPwokRYv534QaPC8g%22%2C%22query%22%3A%22New%20York%2C%20New%20York%2C%20United%20States%22%2C%22cdnCacheSafe%22%3Afalse%2C%22screenSize%22%3A%22large%22%2C%22isInitialLoad%22%3Atrue%2C%22hasLoggedIn%22%3Afalse%7D%2C%22removeDuplicatedParams%22%3Atrue%7D&extensions=%7B%22persistedQuery%22%3A%7B%22version%22%3A1%2C%22sha256Hash%22%3A%2282cc0732fe2a6993a26859942d1342b6e42830704b1005aeb2d25f78732275e7%22%7D%7D HTTP/2
Host: www.airbnb.com
X-Airbnb-Api-Key: d306zoyjsyarp7ifhu67rjxn52tv0t20
Accept-Encoding: gzip, deflate
At this point, that API key is pretty much public, so not a concern.
The readable content of the "variables" parameter is this:
{
"isInitialLoad": true,
"hasLoggedIn": false,
"cdnCacheSafe": false,
"source": "EXPLORE",
"exploreRequest": {
"metadataOnly": false,
"version": "1.8.3",
"itemsPerGrid": 20,
"placeId": "ChIJOwg_06VPwokRYv534QaPC8g",
"query": "New York, New York, United States",
"cdnCacheSafe": false,
"screenSize": "large",
"isInitialLoad": true,
"hasLoggedIn": false
},
"removeDuplicatedParams": true
}
The readable content of the "extensions" parameter is this:
{
"persistedQuery": {
"version": 1,
"sha256Hash": "82cc0732fe2a6993a26859942d1342b6e42830704b1005aeb2d25f78732275e7"
}
}
I am trying to figure out where that hash comes from.
It seems it's calculated from a GraphQL query but I don't know anything else and there is no documentation about it.
Any help?
I had the same issue (wanted to get the prices) and after investigating in the HAR files that you can get with Chrome, I found out that you get this value from a Javascript file called PdpPlatformRoute.xxx.js
The steps to get this hash are simply to load the file PdpPlatformRoute.xxx.js, then to parse the file to get an "operationId".
If this helps, this is how I did this.
// contentPage is the HTML content of the listing page (e.g. https://www.airbnb.com/rooms/1234567)
function getPdpPlatformRouteUrl(contentPage) {
return 'https://a0.muscache.com/airbnb/static/packages/web/en/frontend/gp-stays-pdp-route/routes/' + `${contentPage}`.match(/(PdpPlatformRoute\.\w+\.\js)/)?.[1];
}
// textContent is the JS content that you get when you fetch the previously found URL
function getSha256(textContent) {
return `${textContent}`.match(/name:'StaysPdpSections',type:'query',operationId:'(.*)'/)?.[1];
}
I am trying to learn how to update values on Wikidata using the API. Login and csrf cookies work ok, but when I try to update a value I get an invalid-snak error.
The request body looks like this:
POST /w/api.php HTTP/1.1
Accept-Encoding: gzip
Content-Length: 178
User-Agent: Mojolicious (Perl)
Host: test.wikidata.org
Content-Type: application/x-www-form-urlencoded
Cookie: [omitted]
action=wbcreateclaim&bot=1&entity=Q3345&format=json&property=P9876&snaktype=value&token=[omitted]&value=%7B%22entity-type%22%3A%22Q1917%22%7D
and the response is:
{
"error": {
"code": "invalid-snak",
"info": "Invalid snak data.",
"messages": [
{
"name": "wikibase-api-invalid-snak",
"parameters": [],
"html": {
"*": "Invalid snak data."
}
}
],
"*": "See https://test.wikidata.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce> for notice of API deprecations and breaking changes."
},
"servedby": "mw1386"
}
I've tried various ways to succeed with an update by changing the value - no results. The only update I succeeded with is one with snaktype=novalue - which would confirm that the issue is with the snak vaue alone.
So the question is, what's the right way to make an update to a snakvalue?
The problem is that you are stating value={"entity-type":"Q1917"}, but Q1917 is not an entity-type!
You should instead state value={"entity-type":"item","numeric-id":1917}.
For deepen the topic, see the Wikidata API's documentation.
Right now it looks like a mystery. Please help me in solving it.
I use iTunes public API to fetch an album: "Metallica" by Metallica (see it in browser: US region, MV region). I construct the following URLs to fetch it via API:
US region https://itunes.apple.com/lookup?id=579372950&country=US&entity=album - works
MV region https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album - doesn't work
Here's the actual behaviour I observe:
If I query GET https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album in a Spring app (using RestTemplate + Jackson HttpMessageConverter) I get an empty response:
{
"resultCount":0,
"results": []
}
If I navigate to https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album in a browser I get a file download prompt. The file contains an empty response:
{
"resultCount":0,
"results": []
}
If I query API using HttpPie http get https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album I get a non-empty response !!!
{
"resultCount": 1,
"results": [
{
"amgArtistId": 4906,
"artistId": 3996865,
"artistName": "Metallica",
"artistViewUrl": "https://music.apple.com/us/artist/metallica/3996865?uo=4",
"artworkUrl100": "https://is1-ssl.mzstatic.com/image/thumb/Music/v4/0b/9c/d2/0b9cd2e7-6e76-8912-0357-14780cc2616a/source/100x100bb.jpg",
"artworkUrl60": "https://is1-ssl.mzstatic.com/image/thumb/Music/v4/0b/9c/d2/0b9cd2e7-6e76-8912-0357-14780cc2616a/source/60x60bb.jpg",
"collectionCensoredName": "Metallica",
"collectionExplicitness": "notExplicit",
"collectionId": 579372950,
"collectionName": "Metallica",
"collectionPrice": 9.99,
"collectionType": "Album",
"collectionViewUrl": "https://music.apple.com/us/album/metallica/579372950?uo=4",
"copyright": "℗ 1991 Blackened Recordings",
"country": "USA",
"currency": "USD",
"primaryGenreName": "Metal",
"releaseDate": "1991-08-12T07:00:00Z",
"trackCount": 13,
"wrapperType": "collection"
}
]
}
I tried it multiple times and the results seem to be consistent. I compared the requests and they seem to be identical.
Why does iTunes respond differently to different clients? I can't understand. What important detail am I missing?
Similar questions:
Spring RestTemplate getForObject URL not working for Apple iTunes - there's another problem (double encoding of the whitespace character).
This problem happens to the following regions (it's a complete list):
LI https://itunes.apple.com/lookup?id=579372950&country=LI&entity=album
MV https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album
MM https://itunes.apple.com/lookup?id=579372950&country=MM&entity=album
ET https://itunes.apple.com/lookup?id=579372950&country=ET&entity=album
RS https://itunes.apple.com/lookup?id=579372950&country=RS&entity=album
I spotted a difference:
http get 'https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album' -> empty response
curl 'https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album' -> empty response
http get https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album -> 1 album in response
curl https://itunes.apple.com/lookup?id=579372950&country=MV&entity=album -> 1 album in response
if I don't use quotes around URL, the request is interpreted as GET https://itunes.apple.com/lookup?id=579372950. the default country is US and therefore I see 1 US album in response.
I'm trying to implement the behaviour described in this CosmosDB document with the additional ResourcePartitionKey restrictions on the User Permissions to constrain a resource token to only accessing documents that belong to the specified partition key however I can't make it work.
With the SQL REST API, I receive no errors with the POST to create the UserPermission object with the resourcepartitionkey keypair and both the initial returned object as well as subsequent GET's also show the "resourcepartitionkey" present when fetching the resource token.
Using the resource token against the appropriate collection specified in the permission, I can list all documents in the collection. When using a "x-ms-documentdb-partitionkey" header, I can target any paritionkey I like. Without "x-ms-documentdb-partitionkey" header, it simply returns the whole collection.
The collection is a brand new, Unlimited, 1000 RU's with a partition key of '/rpk'. Post creation query of the collection shows the partition key configured as follows
"partitionKey": {
"paths": [
"\/rpk"
],
"kind": "Hash"
}
Below is the User Permission returned during creation showing the "resourcepartitionkey" present
{
"resource": "dbs/dbName/colls/collectionName/",
"id": "read-collection",
"resourcepartitionkey": "rpk1",
"permissionMode": "read",
"_rid": "lH9FACGGKwAhslfihB0pAA==",
"_self": "dbs\/lH9FAA==\/users\/lH9FACGGKwA=\/permissions\/lH9FACGGKwAhslfihB0pAA==\/",
"_etag": "\"0000ba07-0000-0000-0000-5b7418770000\"",
"_ts": 1534335095,
"_token": "type=resource&ver=1&sig=<resource token signature>"
}
The following is the request for documents using the resource token above. I would expect this to fail due to the missing "x-ms-documentdb-partitionkey" header against a partitioned collection but it both succeeds and proceeds to return records from all partition keys in the collection (only 2 in my test dataset)
GET https://accountname.documents.azure.com/dbs/dbName/colls/collectionName/docs HTTP/1.1
authorization: type%3dresource%26ver%3d1%26sig<resource token signature>
x-ms-version: 2017-02-22
x-ms-max-item-count: -1
x-ms-date: Wed, 15 Aug 2018 12:11:35 GMT
User-Agent: Mozilla/5.0 (Windows NT; Windows NT 10.0; en-AU) WindowsPowerShell/5.1.17134.165
Content-Type: application/json
Host: accountname.documents.azure.com
Response Body from request above showing documents from the partition keys rpk1 and rpk2 even though the user permission is configured to rpk1.
{
"_rid": "lH9FAKbDh4c=",
"Documents": [
{
"id": "blue",
"rpk": "rpk1",
"_rid": "lH9FAKbDh4cCAAAAAAAAAA==",
"_self": "dbs\/lH9FAA==\/colls\/lH9FAKbDh4c=\/docs\/lH9FAKbDh4cCAAAAAAAAAA==\/",
"_etag": "\"ec012ca1-0000-0000-0000-5b73ab440000\"",
"_attachments": "attachments\/",
"_ts": 1534307140
},
{
"id": "red",
"rpk": "rpk2",
"_rid": "lH9FAKbDh4cDAAAAAAAAAA==",
"_self": "dbs\/lH9FAA==\/colls\/lH9FAKbDh4c=\/docs\/lH9FAKbDh4cDAAAAAAAAAA==\/",
"_etag": "\"ec012da1-0000-0000-0000-5b73ab580000\"",
"_attachments": "attachments\/",
"_ts": 1534307160
}
],
"_count": 2
}
I'm assuming I've missed something obvious, or using an incorrect value for 'resourcepartitionkey' in the UserPermission but I can't determine what. Any thoughts greatly appreciated.
After many more hours of trial and error, I have finally resolved my issue which is caused during the POST creation of the user permission.
Firstly, whilst the creation of the user permission will validate the name "resourcePartitionKey", it does not check case sensitivity. With the incorrect sensitivity, the returned UserPermission object has the value present but does not provide any security controls (dangerous situation #1)
Secondly, the input value is not validated for being of type array. Once again it is accepted and returned to you in the user permission object but again provides no security control (dangerous situation #2)
A full working example is below where the permission ID is called "read-collection" and the resourcePartitionKey is configured to 'rpk1' finally exhibits the expected behaviour of requiring "x-ms-documentdb-partitionkey" for the request and only returns values from the specified partition key.
POST https://accountname.documents.azure.com/dbs/dbName/users/userName/permissions HTTP/1.1
authorization: type%3dmaster%26ver%3d1.0%26sig<signature>
x-ms-version: 2017-02-22
x-ms-date: Thu, 16 Aug 2018 04:09:44 GMT
User-Agent: Mozilla/5.0 (Windows NT; Windows NT 10.0; en-AU) WindowsPowerShell/5.1.17134.165
Content-Type: application/json
Host: accountname.documents.azure.com
Content-Length: 215
{
"resource": "dbs/dbName/colls/collectionName/",
"id": "read-collection",
"resourcePartitionKey": [
"rpk1"
],
"permissionMode": "read"
}
If someone knows where to log a DCR or bug for the CosmosDB SQL Rest API, please let me know as without the proper validation during resource permission creation, resource tokens may be distributed to low trust clients that can gain unexpected full access to collection data.
According to HAR 1.2 specification, the response size is calculated as:
var totalSize = entry.response.headersSize + entry.response.bodySize;
But how could the response size be zero (=1273+(-1273)), for example in the following HAR:
"response": {
"status": 200,
"statusText": "OK",
"content": {
"size": 4916,
"mimeType": "application/x-javascript",
"compression": 6189
},
"headersSize": 1273,
"bodySize": -1273
In the specification, there is no negetive value for bodySize other than -1:
bodySize [number] - Size of the received response body in bytes. Set to zero in case of responses coming from the cache (304). Set to -1 if the info is not available.
This is probably a bug on Chromium. There were similar one, in timings section, in the past: https://code.google.com/p/chromium/issues/detail?id=152201
You may open a issue about this one.