I have a problem with recreating the headers, everything seem identical, but it just doesn't work. Need those headers to access Instagram API
I tried to use Charles to intercept a traffic from mobile device and it's working as expected, but I'm struggling to recreate the same headers.
URL is https://i.instagram.com/api/v1/feed/user/7499201770/reel_media/
Headers are
:method: GET
:scheme: https
:path: /api/v1/feed/user/7499201770/reel_media/
:authority: i.instagram.com
content-type: application/json
authority: i.instagram.com
accept: */*
path: /api/v1/feed/user/7499201770/reel_media/
accept-language: en-IN;q=1.0
accept-encoding: gzip;q=1.0, compress;q=0.5
content-length: 2
user-agent: Instagram 10.29.0 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; scale=2.00; 750x1334) AppleWebKit/420+
referer: https://www.instagram.com/
x-ig-capabilities: 3w==
cookie: ds_user_id=6742557571; sessionid=IGSCf716eb61bf2a6d41f...
I tried to use Postman in order to recreate this request, but every time I get the same error "Login required". How should I paste those headers? I can't understand that
It was the user-agent: Instagram 10.29.0 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; scale=2.00; 750x1334) AppleWebKit/420+ that I didn't copy
With user-agent it works, so HTTP headers will look like this in case someone writing an instagram story saver)
["Content-Type": "application/json",
"Accept-encoding": "gzip, deflate",
"User-agent": "Instagram 10.29.0 (iPhone7,2; iPhone OS 9_3_3; en_US; en-US; scale=2.00; 750x1334) AppleWebKit/420+",
"Cookie": "ds_user_id=67425...; sessionid=IGSCf716eb61b....]
Related
I'm running GenHTTP server that is setup to to have CORS permissive (so allow all):
var inline = Inline.Create();
inline.Add(CorsPolicy.Permissive());
For some reason the request still fails by CORS. For context http://localhost:55409 is the server and http://localhost:55309 is the client.
Options (preflight):
Request URL: http://localhost:55409/api/v1//metadata
Request Method: OPTIONS
Status Code: 204 No Content
Remote Address: [::1]:55409
Referrer Policy: strict-origin-when-cross-origin
Access-Control-Allow-Credentials: true
Access-Control-Allow-Headers: *
Access-Control-Allow-Methods: *
Access-Control-Allow-Origin: http://localhost:55309
Access-Control-Expose-Headers: *
Access-Control-Max-Age: 86400
Connection: Keep-Alive
Content-Length: 0
Date: Mon, 13 Jun 2022 13:05:34 GMT
Server: GenHTTP/6.3.4.0
Vary: Origin
This returns 204 OK.
Get:
Request URL: http://localhost:55409/api/v1//metadata
Referrer Policy: strict-origin-when-cross-origin
Provisional headers are shown
Learn more
Accept: application/json, text/plain, */*
Authorization: Bearer <token>
Referer: http://localhost:55309/
sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Windows"
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36
Returns CORS error
This also feels weird as similar requests without Authorization header return 200 OK. The options seem to be correct in that it allows everything. Why does it fail then?
Also note that this works with the other server (more proper server running .Net Web Api), so client is not to blame. I think I'm missing some crucial server setup
Turns out I needed to specifically configure it to allow Authorization header, as if * does not include it?
inline.Add(new CorsPolicyBuilder().Default(new OriginPolicy(null, new List<string>() { "authorization" }, null, AllowCredentials: true, 86400u)));
This is the answer, but I'll mark the other answer whoever can explain why * does not include Authorization as OPTIONS has returned this
Access-Control-Allow-Headers: *
so I assume that should allow Authorization header as well
Working on some code that that I inherited from a non-responsive initial developer. My ASP.NET and web.config are at best "dated", thus I'm turning to the community for some help. One of the first things I had to change is to force this website to operate in SSL (https:) as it deals with sensitive data. The program immediately stopped working and I had to make some undesirable changes to code that "already worked". And it still seems broken, and the changes won't make the client happy.
This is an ASP.NET project that seems hand-rolled.
Sending a POST command with some body text that (I think) is JSon setting additional parameters to the POST command such as: "indexID=8379fcd1-5083-4d1c-a6ee-5812f134a505".
As far as I can tell, this works as intended on non SSL (i.e. http: requests). However, when running in SSL (i.e. https: requests), it appears that the BODY (Json text) isn't getting decoded into the HttpContect.Current.Request parameters (which seems to be happening in http:).
However the post_data that I can read from the input stream has the JSon body text (as clear text?) with the parameters, which my 'fix' adds to the incoming HttpContect.Current.Request parameters as a combined dictionary.
[Here is the RAW command intercepted with Fiddler] POST https://vmdev-xpp/BuilderQC/Services/Data.svc/QueryGridResults?typename=ImportReadyForDownload HTTP/1.1 Host: vmdev-xpp User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0 Accept: application/json, text/javascript, /; q=0.01 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate, br Content-Type: application/x-www-form-urlencoded X-Requested-With: XMLHttpRequest Content-Length: 378 Origin: https://vmdev-xpp Connection: keep-alive Referer: https://vmdev-xpp/BuilderQC/BREDFileManagement.aspx Cookie: ASP.NET_SessionId=ehi2ccsdmekvkgegmfh11n1v; .ASPXLMPTest=2226D5725FBC10FBCCD606108CE5A4E32990EEA8FF8A1864496F69874F116D7E1ABF48A8BFD05EE683FE3F456D4475E88A61B19B299CB557209129BD25E87AC38CECA5303C7E2035E64C1F5A4AD2605D8581181A9C7E48680371F83BC7A93D7A63D8748EA4761A608F424578C20127D01DE0E2FBFBD5F079575E86FD506925D541026B7C8713FDEFE108BCEADBFC1DA0 Sec-Fetch-Dest: empty Sec-Fetch-Mode: cors Sec-Fetch-Site: same-origin
_search=true&nd=1629052554858&rows=40&page=1&sidx=&sord=asc&QC_ProjectID=14&FileReadyForDownload=1&filters=%7B%22fields%22%3A%5B%7B%22field%22%3A%22QC_ProjectID%22%2C+%22op%22%3A+%22cn%22%2C+%22value%22%3A%2214%22%7D%2C%7B%22field%22%3A%22FileReadyForDownload%22%2C+%22op%22%3A+%22cn%22%2C+%22value%22%3A%221%22%7D%5D%7D&indexID=8379fcd1-5083-4d1c-a6ee-5812f134a505&entity=false
[Here is the post_data I obtained from the input stream, I think that this being in clear-text is suspect] Post_data = _search=true&nd=1629052767566&rows=40&page=1&sidx=&sord=asc&QC_ProjectID=14&FileReadyForDownload=1&filters=%7B%22fields%22%3A%5B%7B%22field%22%3A%22FileName%22%2C+%22op%22%3A+%22cn%22%2C+%22value%22%3A%22th%22%7D%2C%7B%22field%22%3A%22QC_ProjectID%22%2C+%22op%22%3A+%22cn%22%2C+%22value%22%3A%2214%22%7D%2C%7B%22field%22%3A%22FileReadyForDownload%22%2C+%22op%22%3A+%22cn%22%2C+%22value%22%3A%221%22%7D%5D%7D&indexID=8379fcd1-5083-4d1c-a6ee-5812f134a505&entity=false&FileName=th
Here is the incoming HttpContext.Current.Request.Params.AllKeys, Notice the lacking "indexID" among other parameters
I'm trying to integrate custom authentication service with micronaut security and to do this I've implemented my own AuthenticationProvider and that works fine for basic auth, however I also need to take care of authentication tokens passed in the request.
To do this I'm trying to implement my own AuthenticationFetcher and in the fetchAuthentication method I'm trying to get my custom authentication header and then authenticate the request.
#Override
public Publisher<Authentication> fetchAuthentication(HttpRequest<?> request) {
if (request.getHeaders().get(authConfiguration.getTokenHeader()) != null) {
The issue I'm having is that netty's request.getHeaders() doesn't return all headers that are being sent to the webservice (I confirmed from my browsers developer console)
GET /service/all HTTP/1.1
Accept: application/json, text/plain, */*
Cookie: m=2258:Z3Vlc3Q6Z3Vlc3Q%253D
Accept-Encoding: gzip, deflate
Host: localhost:4200
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1 Safari/605.1.15
Accept-Language: pl-pl
Referer: http://localhost:4200/campaigns
Connection: keep-alive
X-Token: my.token.here
And here are my app settings
micronaut:
server:
netty:
maxHeaderSize: 1024
worker:
threads: 4
parent:
threads: 4
childOptions:
autoRead: true
application:
name: appName
Any feedback appreciated.
It was caused by cors config:
micronaut:
server:
cors:
enabled: true
After adding this my request header was removed in the filter chains.
I'm trying to remove the unnecessary pre-flight requests in my application. For it I've simplified some parts of request, removed custom headers etc. But got stuck on a problem - GET requests now work fine without pre-flights, but POST requests still have them.
I've followed the requirements:
Request does not set custom HTTP headers.
Content type is "text/plain; charset=utf-8".
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain.
Both GET and POST requests go through the single httpinvoke call.
As an example - GET request that is not prefaced by pre-flight:
URL: http://mydomain/APIEndpoint/GETRequest?Id=346089&Token=f5h345
Request Method:GET
Request Headers:
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:uk-UA,uk;q=0.8,ru;q=0.6,en-US;q=0.4,en;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Content-Type:text/plain; charset=utf-8
Host: correct host
Origin:http://localhost
Pragma:no-cache
Referer: correct referer
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Query String Parameters:
Id=346089
Token=f5h345
And a POST request that looks very similar but is still prefaced with pre-flight:
URL: http://mydomain/APIEndpoint/GETRequest?param=TEST
Request Method:POST
Request Headers:
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:uk-UA,uk;q=0.8,ru;q=0.6,en-US;q=0.4,en;q=0.2
Cache-Control:no-cache
Connection:keep-alive
Content-Length:11
Content-Type:text/plain; charset=UTF-8
Host:
Origin:http://localhost
Pragma:no-cache
Referer:
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Query String Parameters:
param:TEST
Request Payload
{MyData: {}}
Any advice would be appreciated! Thanks!
==== Update ===
As requested, posting the pre-flight request for the POST request:
URL: http://mydomain/APIEndpoint/GETRequest?param=TEST
Request Method:OPTIONS
Status Code:200 OK
Response Header
Access-Control-Allow-Origin:*
Cache-Control:no-cache
Content-Length:0
Date:Wed, 09 Aug 2017 08:02:16 GMT
Expires:-1
Pragma:no-cache
Server:Microsoft-IIS/8.5
X-AspNet-Version:4.0.30319
X-Powered-By:ASP.NET
Request Headers
Accept:*/*
Accept-Encoding:gzip, deflate
Accept-Language:uk-UA,uk;q=0.8,ru;q=0.6,en-US;q=0.4,en;q=0.2
Access-Control-Request-Method:POST
Cache-Control:no-cache
Connection:keep-alive
Host:correct host
Origin:http://localhost
Pragma:no-cache
Referer: correct referer
User-Agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36
Query String Parameters
param:TEST
It's a legacy code which uses the httpinvoke library. Code snippet that actually does the call:
_converters: {
'text json': function (input, reviver) {},
'json text': JSON.stringify
};
headers = {
'Content-Type': 'text/plain; charset=utf-8'
};
data = {params: "test"};
httpinvoke(url, method.toUpperCase(), {
corsExposedHeaders: ['Content-Type'],
headers: headers,
input: data,
converters: _converters,
inputType: 'json',
outputType: 'json',
timeout: self._getMessageTimeout()
}).then(function (res) {}, function (error) {});
This could happen if there are event listeners registered on the XMLHttpRequestUpload object (that forces a preflight; see the note on the use-CORS-preflight flag in https://xhr.spec.whatwg.org/, and a related note in https://fetch.spec.whatwg.org/ and the updated documentation on CORS preflight requests in the MDN CORS article).
Does httpinvoke do that?
As #Anne mentioned the reason that POST were sending pre-flight requests despite the requests themselves conforming to the rules of "simple requests" (and thus not needing a pre-flight) was in the XMLHttpRequestUpload event listeners.
XMLHttpRequestUpload itself might not be mentioned in the code, but you can always find it in xhr.upload variable. This was the case for http-invoke library.
So totally innocent looking code like:
xhr.upload.onprogress = onuploadprogress;
actually causes mandatory pre-flight requests.
Thanks to all who helped solve this problem.
I'm debugging a program I wrote and noticed something strange. I set up an HTTP server on port 12345 that servers a simple OGG video file, and attempted to access it from Firefox.
Upon sniffing the network requests, I found these two requests were made:
GET /video.ogv HTTP/1.1
Host: 127.0.0.1:12345
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
GET /video.ogv HTTP/1.1
Host: 127.0.0.1:12345
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Range: bytes=8122368-
The video is almost 8 MB in size, so the fact that the second request specifics 8122368 bytes, which is 7932 KB, suggests it is requesting the very end of the file for some reason. Anyone have ideas?
In order to support seeking and playing back regions of the media that aren't yet downloaded, Gecko uses HTTP 1.1 byte-range requests to retrieve the media from the seek target position. So because Ogg files don't contain their duration, the initial download connection is terminated. Then there is a seek to the end of the Ogg file and read a bit of data to extract the time duration of the media. Info from here and here.
Some media format have meta data at the end of the file, and this data is usually required to allow proper seeking of the video.
Its actually requesting 8122368 bytes starting backwards from the end. Which is 7.74MB if I did my calcs correctly.
it might be something in how the buffering for that file type is done.