I have a page that I can get by GET request.
the response of that request is an html page that has a button called Search
when I called that button, a post request is fired and get response that is appended to the page. In other words, clicking on that button didn't give me a completely new page but It adds new content to that page.
I tried to use Live HTTP headers firefox extension to read the request in order to see the parameters that are being sent in the post request. This is what I get
POST /plugins/ad/buy.php?q=used+cars+dubai HTTP/1.1
Host: www.autodealer.ae
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://www.autodealer.ae/plugins/ad/buy.php?q=used+cars+dubai
Cookie: PHPSESSID=f2072a947619ef2d61b552f38e163d02; __utma=154876456.960352407.1397595567.1397595567.1397598041.2; __utmc=154876456; __utmz=154876456.1397595567.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); __gads=ID=7a5bda3c29913b41:T=1397595570:S=ALNI_MZg2J44DRK3D1j8CX4FpZWFHWIzuw; __utmb=154876456.1.10.1397598041
Connection: keep-alive
Content-Type: application/x-www-form-urlencoded
Content-Length: 131
vehcategory=All&make=All&model=&platename=&pricefrom=%3C500&priceto=All&city=All&sort=postdate&results_listing=1MONTH&Search=Search
I have read many questions on StackOverFlow website and I learned that the post parameters exit in the request body. and in my situation, the content type is application/x-www-form-urlencoded so the post parameters should be as a query string
My Question
where is the post parameters in the above request? I just can see the cookies
HTTP GET Parameters are in the query string.
HTTP POST would have them inside the content body - so looks as though they are:
Content-Length: 131
vehcategory=All&make=All&model=&platename=&pricefrom=%3C500&priceto=All&city=All&sort=postdate&results_listing=1MONTH&Search=Search
Related
I have some react code that is rendering content dynamically via React.createElement. As such, css is applied via an object. Elements in that dynamic generation can have background image, pointing to a public aws S3 bucket.
It seems that every time my components re-render, the background images are being fetched again from S3. This is delaying the page render. I have S3 meta-data for Cache-Control set on all the objects . Here are request and response headers for background image load -
Response header -
Accept-Ranges: bytes
Cache-Control: public, max-age=604800
Content-Length: 52532
Content-Type: application/octet-stream
Date: Sun, 06 Feb 2022 05:57:32 GMT
ETag: "f29655808a5f80627d9ea7f44058a5e3"
Last-Modified: Sun, 06 Feb 2022 05:55:10 GMT
Server: AmazonS3
x-amz-meta-filetype: IMAGE
Request Header -
Accept: image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.9,hi;q=0.8
Cache-Control: no-cache
Connection: keep-alive
Host: <bucket-name>s3.amazonaws.com
Pragma: no-cache
Referer: https://<my-domain>.com/
sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="97", "Chromium";v="97"
sec-ch-ua-mobile: ?0
sec-ch-ua-platform: "Linux"
Sec-Fetch-Dest: image
Sec-Fetch-Mode: no-cors
Sec-Fetch-Site: cross-site
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36
I can see in Network tab that images are being loaded multiple times and it also shows data transfers being done everytime. What am I doing wrong here? Can someone please help finding the root cause. Thanks.
could it be a forgotten "disable cache" option selected in the network tab in the dev tools ? Because it seems the server responds with the correct type of cache headers.
If images are optimized and not huge, using base64 data url is a good solution.
const getBase64FromUrl = async (url) => {
const data = await fetch(url);
const blob = await data.blob();
return new Promise((resolve) => {
const reader = new FileReader();
reader.readAsDataURL(blob);
reader.onloadend = () => {
const base64data = reader.result;
resolve(base64data);
}
});
}
const image = getBase64FromUrl('the image url')
While creating the element you can use
background-image: `url(${image})`;
In addition, rarely do we serve from S3 directly, you should probably use cloudfront as a proxy to
reduce get request
reduce bandwidth charges
cache at cdn
better control of cache headers
hide your s3 real url
The reason you're seeing a network request is probably because you're using the Cache-Control: no-cache header in your request.
As seen here:
The no-cache response directive indicates that the response can be
stored in caches, but the response must be validated with the origin
server before each reuse, even when the cache is disconnected from the
origin server.
Cache-Control: no-cache
If you want caches to always check for content
updates while reusing stored content, no-cache is the directive to
use. It does this by requiring caches to revalidate each request with
the origin server.
Note that no-cache does not mean "don't cache". no-cache allows caches
to store a response but requires them to revalidate it before reuse.
If the sense of "don't cache" that you want is actually "don't store",
then no-store is the directive to use.
See here: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control#response_directives
Here is what a full request for a cached asset looks like on my network tab, when the asset returns 304 Not Modified from the validation request. (from S3) This is in a background: url context.
I'm new to Cache Control Header implementation and I need someone to point out any of my mistakes and/or misunderstandings over the cache control effects on Firebase Cloud Functions.
My understanding & expectation on Cache Control over Firebase Functions
When the Cache Control Header has been successfully set using Express response object (confirmed by checking from the Chrome's Network
tab), regardless it is on localhost or production server, the Firebase
Https Functions (not callable functions) should not be invoked again
after the first reload until the cache is expired.
Am I right? But after a few rounds of testing, it seems like my cloud function on localhost still consistently get invoked (confirmed by server console logging) regardless the number of refresh on my web browser. Below is my current Http header:
**General:**
Request URL: http://localhost:5005/otk-web-solutions?id=B0Y0jp2x83WVYzWrpg5y
Request Method: GET
Status Code: 304 Not Modified
Remote Address: 127.0.0.1:5005
Referrer Policy: strict-origin-when-cross-origin
**Response Headers:**
Access-Control-Allow-Origin: *
cache-control: public, max-age=432000, s-maxage=432000
content-length: 9688
content-type: text/html; charset=utf-8
date: Mon, 05 Apr 2021 11:52:20 GMT
etag: W/"25d8-TxL0Q+ujhzDjys8IJ1mLigY7jT8"
vary: Origin, Accept-Encoding, Authorization, Cookie
x-powered-by: Express
**Request Headers:**
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en;q=0.9,en-US;q=0.8,zh;q=0.7
Cache-Control: max-age=0
Connection: keep-alive
Cookie: _ga=GA1.1.816734993.1603107580; _gid=GA1.1.223745218.1617606982; __atuvc=20%7C12%2C15%7C13%2C23%7C14; __atuvs=606aec5f76521aab00a
DNT: 1
Host: localhost:5005
If-None-Match: W/"25d8-TxL0Q+ujhzDjys8IJ1mLigY7jT8"
sec-ch-ua: "Google Chrome";v="89", "Chromium";v="89", ";Not A Brand";v="99"
sec-ch-ua-mobile: ?0
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36
**Query String Parameters:**
id: B0Y0jp2x83WVYzWrpg5y
On Firebase documentation:
You can, though, configure caching behavior for dynamic content. For
example, if a function generates new content only periodically, you
can speed up your app by caching the generated content for at least a
short period of time.
You can also potentially reduce function execution costs because the
content is served from the CDN rather than via a triggered function.
Could it be that, the cache control header has no effects on localhost except on Firebase CDN, which means only when we've deployed it to the production server for the caching to work on the cloud CDN? Is there a right way to implement such test to see the effectiveness of the cache control header in helping to save the Firebase Cloud Functions' execution costs?
Please advise, thanks a lot!
From my understanding of the documentation and after checking your test, only if the content is served from the CDN you will be able to save costs.
Even if your request is met with a status code 304 Not Modified, it seems like you're still making the request and invoking the function if you're not using the Firebase Hosting CDN.
So to make a test to see if you can save costs by not invoking the function many times you should set up Firebase Hosting and do that same test to see if a response from the CDN invokes the function.
I am getting the following error message on my angular/asp.net web api project.
XMLHttpRequest cannot load http://localhost:7291/api/products. No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin 'http://localhost:7305' is therefore not allowed access. The response had HTTP status code 500.
I know it has something to do with CORS not being implemented correctly, but I'm not sure what I'm doing wrong. I'm following a tutorial and as far as I can tell I've got everything right?
Here's the info from the network tab in chrome debug.
Remote Address:[::1]:7291
Request URL:http://localhost:7291/api/products
Request Method:GET
Status Code:500 Internal Server Error
Response Headers
(8)
Request Headers
view source
Accept:application/json, text/plain, /
Accept-Encoding:gzip, deflate, sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Host:localhost:7291
Origin:http://localhost:7305
Referer:http://localhost:7305/
User-Agent:Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36
Here is where I map the API url.
(function () {
"use strict";
angular
.module("common.services", ["ngResource"])
.constant("appSettings",
{
serverPath: "http://localhost:7291/"
})
}());
And this is how I'm setting up the EnableCOrsAttribute:
[EnableCorsAttribute("http://localhost:7305", "*", "*")]
Can anyone see what I'm doing wrong? If more code is needed please let me know. Thanks.
First Step, use a tool like Postman or Fiddler to verify your service endpoint first to ensure functionality.
It looks as if this is not a CORS issue at all, due to the 500 response:
Status Code:500 Internal Server Error
A CORS issue on a normal success response is usually manifested as a status 0 or -1 in the client, but what you are experiencing is an error on the server side. I have seen this in my own code and suspect that your implementation on the server side is only injecting the CORS headers at the end of processing the request and as the processing abnormally aborted the CORS headers didn't make it in there.
Once you have confirmed functionality do an OPTIONS request on your endpoint to verify CORS:
OPTIONS / HTTP/1.1
Host: localhost:7291
Cache-Control: no-cache
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
Then inspect the headers of the response, if your CORS is enabled correctly on the server you should see Access-Control-Allow headers similar to this:
Access-Control-Allow-Methods: GET, POST, PUT, PATCH, DELETE, OPTIONS, ETAG
Access-Control-Allow-Origin: http://localhost:7305
Can somebody tell me how to add asp.net cookie authorization to a request header (via MVC3)?
Cookie:
Cookie: __RequestVerificationToken_Lw__=8+WZPGAaKtgkIPfbBovP1ZRP2qQKE3u67ueltnzcoCPH0nN1tUHdtgUorjlweUvn+zTJhkFeRuMShCOrbyHR5Xi3DOL4HCspXuVEOsWIr4Ape+l5MYPiFsQ6Lnw8LstqNjceWW9EaV24eA0mVxq2xTG18h/INNKLB8cRUiEn9DI=; .ASPXAUTH=C64A69436A8FC4A6DF5BC222982030C3CCF9E43FBCE335A47173236B4BA4B1CE762CBE6C9E9FDBB035D46C8F36228A61117F22DD55CF787D5E23A728F68B49DDF1A5D70FF3D33C8D16B06FC81894201E86DF93754B6021C9031CB4FBC5236DED952FB7244CE3217B659325A0614763B2E123002E5291EE8D8CEA7B2D7441F3EBB8176A71CDD6FEF3E545CDF46858174451D38890861664A55AF681A36C0B7CF1
Snippet of actual request:
Accept: text/*
Content-Type: multipart/form-data; boundary=----------KM7Ij5Ij5ei4gL6KM7ae0cH2Ef1ae0
User-Agent: Shockwave Flash
Host: localhost:82
Content-Length: 36874
Connection: Keep-Alive
Pragma: no-cache
Cookie: __RequestVerificationToken_Lw__=8+WZPGAaKtgkIPfbBovP1ZRP2qQKE3u67ueltnzcoCPH0nN1tUHdtgUorjlweUvn+zTJhkFeRuMShCOrbyHR5Xi3DOL4HCspXuVEOsWIr4Ape+l5MYPiFsQ6Lnw8LstqNjceWW9EaV24eA0mVxq2xTG18h/INNKLB8cRUiEn9DI=; .ASPXAUTH=C64A69436A8FC4A6DF5BC222982030C3CCF9E43FBCE335A47173236B4BA4B1CE762CBE6C9E9FDBB035D46C8F36228A61117F22DD55CF787D5E23A728F68B49DDF1A5D70FF3D33C8D16B06FC81894201E86DF93754B6021C9031CB4FBC5236DED952FB7244CE3217B659325A0614763B2E123002E5291EE8D8CEA7B2D7441F3EBB8176A71CDD6FEF3E545CDF46858174451D38890861664A55AF681A36C0B7CF1
------------KM7Ij5Ij5ei4gL6KM7ae0cH2Ef1ae0
<more stuff here...>
You don't. Cookies are automatically provided on each request when the cookie is set.
You set the cookie using FormsAuthentication class. Typically, this looks something like this:
FormsAuthentication.SetAuthCookie(model.UserName, false /*createPersistentCookie*/);
I have the following situation:
The browser does a POST with a Json payload to my server endpoint
My server processes the data and then issues a redirect to a new location
The browser does the redirect, but it does it with the same headers as the original post which mean it arrives at my endpint such that my endpoint
thinks it is a json request.
-> I want the redirect to arrive at my server as a standard text/html request.
Is it possible to control the headers that the redirect uses so that the Get request arrives with text/html Accept headers?
Here are some snippets from fiddler to highlight what I am talking about:
Initial POST, json payload:
POST /App/Client/Index HTTP/1.1
Accept: application/json, text/plain, */*
X-Requested-With: XMLHttpRequest
The response:
<html><head><title>Object moved</title></head><body>
<h2>Object moved to here.</h2>
</body></html>
The subsequest GET request from the Redirect:
GET /App/Client/SingleEntity?entityId=f859a6ca-dbcf-49cf-8de0-3888b7011815 HTTP/1.1
Accept: application/json, text/plain, */*
X-Requested-With: XMLHttpRequest
I had a similar requirement. My solution was to not send a redirect, but a "200 OK" with the following JSON:
{"url": "http://example.com/redirect/to/some/resource"}
The client used the URL and redirected via Javascript, i.e. location.href = data.url. It might not be the answer you're looking for but it solves your problem...
See also: How to manage a redirect request after a jQuery Ajax call