I'm trying to do a simple HTTP POST on the following website:
https://oktap.tax.ok.gov/oktap/web/_/
(click on "Permits" in the bottom left hand side)
I simply want to post a number and get the results from the page returned. Looking at the page with chrome's web developer tools it looks like the POST goes to both
https://oktap.tax.ok.gov/oktap/web/_/Recalc
https://oktap.tax.ok.gov/oktap/web/_/EventOccurred
I'm not sure if it's one or the other, it looks like EventOccured takes a couple more parameters. However when I post to either one of them with the following code:
#!/usr/bin/env bash
wget --post-data="D0IHwpHb__0_0_Ful0QW=5&VIEW__=VS_PermitSearch&LASTFOCUSFIELD__=D0IHwpHb__0_0_Ful0QW&DOC_MODAL_ID__=0&EVENT__=D0IHwpHb__0_0_Ful0QW&DOC_ACTION__=false&TYPE__=1&CLOSECONFIRMED__=false&FAST_VERLAST__=9.CvUZWPROHiWR-EO6d9UAeHYv4m81" \
-U "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547 Safari/537.36" \
https://oktap.tax.ok.gov/oktap/web/_/EventOccurred
First I get 307 Temporary Redirect and then ERROR 405: Method Not Allowed.
I don't see how the a post is not allowed, when it's clearly being made from the browser just fine. Any Help?
Here's the headers:
Request URL:https://oktap.tax.ok.gov/oktap/web/_/Recalc
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:146
Content-Type:application/x-www-form-urlencoded
Cookie:wdc-session=w0srLHI6Tz9tDLtEDo0n33PNuXSFexxysEHBa9v5dtjXBt/X4cKb9zKdxoVrtyDKseewwZMbU41vn3DLmyf0QUUjtKwXdmEhHtS69aZf94Y26cqd95rsiCKg06SQVIm5p63me/C2chBBapoABa1lJ8lf4F3MbBIiBAnCnbKlgVfXtsjpijt9i2PMILjlAalr
Host:oktap.tax.ok.gov
Origin:https://oktap.tax.ok.gov
Referer:https://oktap.tax.ok.gov/oktap/web/_/
User-Agent:Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/31.0.1650.4 Chrome/31.0.1650.4 Safari/537.36
X-Requested-With:XMLHttpRequest
Form Dataview sourceview URL encoded
D0IHwpHb__0_0_Ful0QW:5
VIEW__:VS_PermitSearch
LASTFOCUSFIELD__:D0IHwpHb__0_0_Ful0QW
DOC_MODAL_ID__:0
FAST_VERLAST__:3.2sMmdbnwgvAQF41H3c_2XHozyeA1
Response Headersview source
Cache-Control:no-cache, no-store, max-age=0, must-revalidate
Content-Encoding:gzip
Content-Language:en, en-US
Content-Length:318
Content-Type:application/json; charset=utf-8
Date:Thu, 03 Oct 2013 19:52:52 GMT
Expires:Fri, 01 Jan 1990 00:00:00 GMT
Pragma:no-cache
Server:Microsoft-HTTPAPI/2.0
Set-Cookie:wdc-session=w0srLHI6Tz9tDLtEDo0n33PNuXSFexxysEHBa9v5dtjXBt/X4cKb9zKdxoVrtyDKseewwZMbU41vn3DLmyf0QUUjtKwXdmEhHtS69aZf94Y26cqd95rsiCKg06SQVIm5p63me/C2chBBapoABa1lJ8lf4F3MbBIiBAnCnbKlgVfXtsjpijt9i2PMILjlAalr; path=/oktap/web/; HttpOnly; Secure;
X-Frame-Options:DENY
and
Request URL:https://oktap.tax.ok.gov/oktap/web/_/EventOccurred
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:application/json, text/javascript, */*; q=0.01
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:226
Content-Type:application/x-www-form-urlencoded
Cookie:wdc-session=w0srLHI6Tz9tDLtEDo0n33PNuXSFexxysEHBa9v5dtjXBt/X4cKb9zKdxoVrtyDKseewwZMbU41vn3DLmyf0QUUjtKwXdmEhHtS69aZf94Y26cqd95rsiCKg06SQVIm5p63me/C2chBBapoABa1lJ8lf4F3MbBIiBAnCnbKlgVfXtsjpijt9i2PMILjlAalr
Host:oktap.tax.ok.gov
Origin:https://oktap.tax.ok.gov
Referer:https://oktap.tax.ok.gov/oktap/web/_/
User-Agent:Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/31.0.1650.4 Chrome/31.0.1650.4 Safari/537.36
X-Requested-With:XMLHttpRequest
Form Dataview sourceview URL encoded
D0IHwpHb__0_0_Ful0QW:5
VIEW__:VS_PermitSearch
LASTFOCUSFIELD__:D0IHwpHb__0_0_Ful0QW
DOC_MODAL_ID__:0
EVENT__:D0IHwpHb__0_0_Ful0QW
DOC_ACTION__:false
TYPE__:1
CLOSECONFIRMED__:false
FAST_VERLAST__:4.Ol_i_B9mDsWsP0Mg0e02_y7OZjM1
Response Headersview source
Cache-Control:no-cache, no-store, max-age=0, must-revalidate
Content-Encoding:gzip
Content-Language:en, en-US
Content-Length:3711
Content-Type:application/json; charset=utf-8
Date:Thu, 03 Oct 2013 19:52:52 GMT
Expires:Fri, 01 Jan 1990 00:00:00 GMT
Pragma:no-cache
Server:Microsoft-HTTPAPI/2.0
Set-Cookie:wdc-session=w0srLHI6Tz9tDLtEDo0n33PNuXSFexxysEHBa9v5dtjXBt/X4cKb9zKdxoVrtyDKseewwZMbU41vn3DLmyf0QUUjtKwXdmEhHtS69aZf94Y26cqd95rsiCKg06SQVIm5p63me/C2chBBapoABa1lJ8lf4F3MbBIiBAnCnbKlgVfXtsjpijt9i2PMILjlAalr; path=/oktap/web/; HttpOnly; Secure;
X-Frame-Options:DENY
As a developer for that company, I would not recommend looking up permits that way. For $150 per year you can get a file with this information.
Contact the OTC. http://www.tax.ok.gov/rules/rule6507.pdf - 710:65-9-6. Subscription to sales tax permit list
Related
The Apple homepage loads some fonts in the head using this tag:
<link rel="stylesheet" href="/wss/fonts?families=SF+Pro,v3|SF+Pro+Icons,v3" type="text/css" media="all">
However, if you visit https://www.apple.com/wss/fonts?families=SF+Pro,v3|SF+Pro+Icons,v3 or try to access via curl
curl --head 'https://www.apple.com/wss/fonts?families=SF+Pro,v3|SF+Pro+Icons,v3'
You get
HTTP/2 403
server: AkamaiGHost
mime-version: 1.0
content-type: text/html
content-length: 280
cache-control: max-age=0
expires: Fri, 17 Feb 2023 20:13:08 GMT
date: Fri, 17 Feb 2023 20:13:08 GMT
x-cache: TCP_DENIED from a2-22-225-45.deploy.akamaitechnologies.com (AkamaiGHost/11.0.0-46340752) (-)
set-cookie: geo=GB; path=/; domain=.apple.com
It would seem that they've set up the server to do something along the lines of the top suggestion here:
https://stackoverflow.com/questions/2187200/using-php-apache-to-restrict-access-to-static-files-html-css-img-etc
I wanted to duplicate exactly what my browser was sending, so I copied the exact request headers (as shown in dev tools) into Postman
accept:text/css,*/*;q=0.1
accept-encoding:gzip, deflate, br
accept-language:en-US,en;q=0.9
cache-control:no-cache
cookie:MY_COOKIE
pragma:no-cache
referer:https://www.apple.com/
sec-ch-ua:"Chromium";v="110", "Not A(Brand";v="24", "Google Chrome";v="110"
sec-ch-ua-mobile:?0
sec-ch-ua-platform:"macOS"
sec-fetch-dest:style
sec-fetch-mode:no-cors
sec-fetch-site:same-origin
user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36
But this gets a 404.
But what specifically is the stylesheet sending? Is anything missing?
I loaded an Youtube video, and was checking the Network tab in the Chrome inspector. The weird part is the response header expires show a wrong date, see:
expires:Tue, 27 Apr 1971 19:44:06 EST
Does some one understand if this is correctly implemented (some solution as "the response already arrives expired for security reasons") or is just a bug?
The entire request-response pair:
General
Request URL:https://www.youtube.com/watch?v=Y2bcZpjbimc
Request Method:GET
Status Code:200
Remote Address:216.58.222.14:443
Response Headers
alt-svc:quic=":443"; ma=2592000; v="35,34"
cache-control:no-cache
content-encoding:gzip
content-type:text/html; charset=utf-8
date:Mon, 16 Jan 2017 02:12:59 GMT
expires:Tue, 27 Apr 1971 19:44:06 EST
server:YouTubeFrontEnd
status:200
strict-transport-security:max-age=31536000
x-content-type-options:nosniff
x-frame-options:SAMEORIGIN
x-xss-protection:1; mode=block; report=https://www.google.com/appserve/security-bugs/log/youtube
Request Headers
:authority:www.youtube.com
:method:GET
:path:/watch?v=Y2bcZpjbimc
:scheme:https
accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
accept-encoding:gzip, deflate, sdch, br
accept-language:en-US,en;q=0.8
cache-control:max-age=0
cookie:YSC=tkkR7-gquIo; LOGIN_INFO=09a68a7966aeeb4c54ea2812d67ef17bcz4AAAB7IjMiOiA5ODI4NjMwLCAiMSI6IDEsICI4IjogMTUxMjAxNzQ2Mjg1LCAiNyI6IDAsICI0IjogIkdBSUEifQ==; llbcs=0; SID=KgRtjV-NqZqWb_Vtlx1ZVI4BGeOq6TO0kOwRjM63Y9zRlD8NZ14Ain0S7OHEdAude6Ql5w.; HSID=Ae2Oerx0Cx8cLGNN2; SSID=AAxm-sCogA2PcrWj-; APISID=l63qqbTbXYbA1SWI/ATu8oD872iyWdvAgn; SAPISID=jEgCzttgmiin9s_R/A0u9gLEfrGesFDkOu; _ga=GA1.2.1081395761.1467350952; wide=1; VISITOR_INFO1_LIVE=9ZvTZmoHPqs; C_YNe.resume=nPTuJcnwLro:132,CG1HnKT8khI:1282,0SARbwvhupQ:527; PREF=f1=50000000&f5=20030&al=en+pt&cvdm=grid
dnt:1
upgrade-insecure-requests:1
user-agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36
x-client-data:CJG2yQEIprbJAQipncoB
Query String Parameters
v:Y2bcZpjbimc
It is an easter egg left by the original author.
How to perform button click, named: "confirm", using python 3.x and requests library? What parameters should be passed with POST request.
In general i am interested how to guess what settings are needed to achieve my point.
Here are headers information from the browser network graph (with slightly modified values):
General
Request URL:SOMEURL
Request Method:POST
Status Code:302 Found
Remote Address:SOMEADDR
Response Headers
Cache-Control:private
Content-Length:203
Content-Type:text/html; charset=utf-8
Date:Wed, 20 Apr 2016 05:54:51 GMT
Location:SOMELOCATION
MicrosoftSharePointTeamServices:14.0.0.6123
Server:Microsoft-IIS/7.5
SPRequestGuid:04e5403d-3170-4c1d-a5a3-441776ca4e57
X-AspNet-Version:2.0.50727
X-MS-InvokeApp:1; RequireReadOnly
X-Powered-By:ASP.NET
X-SharePointHealthScore:0
Request Headers
Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding:gzip, deflate
Accept-Language:ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4,ka;q=0.2
Cache-Control:max-age=0
Connection:keep-alive
Content-Length:3836
Content-Type:application/x-www-form-urlencoded
Cookie:filterValue=f4298401-b95c-4f84-b127-c1237f65819d; ASP.NET_SessionId=t35zomew310lpn45bgy1kp55; SectionId=15; databaseBtnText=0; databaseBtnDesc=0; Ribbon.ListItem=1366667|-1|655|1597680975; stsSyncAppName=Client; stsSyncIconPath=; Ribbon.ListForm.Display=808667|-1|503|815204513
Host:HOST
Origin:ORIGIN
Referer:REFERER
Upgrade-Insecure-Requests:1
User-Agent:Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36
Query String Parameters
itemID:ID
List:LIST
activity:ACTION
workflowID:WFID
source:SOURCE
Form Data
MSOWebPartPage_PostbackSource:
MSOTlPn_SelectedWpId:
MSOTlPn_View:0
MSOTlPn_ShowSettings:False
MSOGallery_SelectedLibrary:
MSOGallery_FilterString:
MSOTlPn_Button:none
__EVENTTARGET:
__EVENTARGUMENT:
MSOSPWebPartManager_DisplayModeName:Browse
MSOSPWebPartManager_ExitingDesignMode:false
MSOWebPartPage_Shared:
MSOLayout_LayoutChanges:
MSOLayout_InDesignMode:
MSOSPWebPartManager_OldDisplayModeName:Browse
MSOSPWebPartManager_StartWebPartEditingName:false
MSOSPWebPartManager_EndWebPartEditing:false
_maintainWorkspaceScrollPosition:0
__REQUESTDIGEST:0x2D3C5B3A042C327D5B676EA782F2AB9623C1E03DD32F9884204134C32E8EB4287AB98B49EB235D3E27D0F8C0870E3F1CE69ABCDC2AB9C3797023D74CC1DC9AA9,20 Apr 2016 05:53:06 -0000
__VIEWSTATE:REMOVED
__VIEWSTATEGENERATOR:REMOVED
InputKeywords:Search this site...
ctl00$PlaceHolderSearchArea$ctl01$ctl03:0
ctl00$PlaceHolderMain$hiddRedirectURL:
ctl00$PlaceHolderMain$hidConfirm:checked
ctl00$PlaceHolderMain$hiddValidate:
ctl00$PlaceHolderMain$Button2:Confirm
__spText1:
__spText2:
I am using express to write a response.
My code is:
res.set('Cache-Control', 'public, max-age=300');
res.send(data);
The headers I see are:
Remote Address:127.0.0.1:9000
Request URL:http://localhost:9000/some/path
Request Method:GET
Status Code:304 Not Modified
Request Headersview source
Accept:application/json, text/plain, */*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8,he;q=0.6
Cache-Control:max-age=0
Connection:keep-alive
Cookie:express:sess=eyJ1c2VySWQiOiI1MzgxYTdjMDA0ZmIwMmIxMGI1NTdlZTMifQ==; express:sess.sig=lm-kq5ludtkWdRcFcVxNBL0BdT0
Host:localhost:9000
If-None-Match:W/"v9r1H7w4HiaXvycJ9FJ7lg=="
Referer:http://localhost:9000/
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36
Response Headersview source
access-control-allow-headers:Content-Type, X-API-KEY
access-control-allow-methods:GET, POST, DELETE, PUT
access-control-allow-origin:*
cache-control:public, max-age=300
connection:close
date:Sat, 03 Jan 2015 08:47:29 GMT
expires:Sat, 03 Jan 2015 08:52:29 GMT
etag:W/"v9r1H7w4HiaXvycJ9FJ7lg=="
x-powered-by:Express
However, I see my backend gets the request each and every time.
I have tried it with developers area open and closed and with "cache" on and off.
Nothing seems to actually cache the request
What am I doing wrong?
Here's an example JavaScript file request/response:
Request URL:http://local/index.js?time=1367958844038
Request Method:GET
Status Code:200 OK
Request Headers
Accept:*/*
Accept-Charset:ISO-8859-1,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
DNT:1
User-Agent:Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31
Response Headers
cache-control:max-age=31536000
content-encoding:gzip
content-type:application/javascript
expires:Wed, 07 May 2014 20:34:04 GMT
last-modified:Tue, 07 May 2013 20:34:04 GMT
transfer-encoding:chunked
As you can see, the server responds with cache control, expires and even last modified, but everytime I reload with either F5 or clicking enter in location bar the request looks the same (I'd expect browser to send if-modified-since, etc.)
This happens in Chrome and Firefox at least.
Probably because the URL's time parameter changes with every request.
Since the URL is different, the browser can't use the previously cached response.