Token = aiudhaiw
Target = 88192823
url = "https://example.com" + Token + Target
Dim o
Set o = CreateObject("MSXML2.XMLHTTP")
o.Open "GET", url, False
o.Send
Lets say I have this code for example, it works, but it only works 1 time. What I'm trying to do is to update Token or Target or both (if needed) in the next lines and then use o.Send again and it should send it again, and it should work right? But it doesn't.
if I try with the same Token, Target again, it doesn't work. Even if I change them, and change url again, it doesn't work.
The only way that I can get it to work is to make new o.Open, and it also doesn't work with the same Token and Target again.
Calling Send on an open request twice results in an error 0xC00C0240:
This method cannot be called after the send method has been called.
So you MUST open a new request before you can call Send again.
Demonstration:
This works:
>>> url = "https://www.example.org/"
>>> Set req = CreateObject("Msxml2.XMLHTTP.6.0")
>>> req.Open "GET", url, False
>>> req.Send
>>> req.Open "GET", url, False
>>> req.Send
This doesn't:
>>> url = "https://www.example.org/"
>>> Set req = CreateObject("Msxml2.XMLHTTP.6.0")
>>> req.Open "GET", url, False
>>> req.Send
>>> req.Send
This method cannot be called after the send method has been called.
(0xC00C0240)
Also, to change the URL you must re-open the request. The URL on an already open request cannot be changed.
>>> url = "https://www.example.org/"
>>> Set req = CreateObject("Msxml2.XMLHTTP.6.0")
>>> req.Open "GET", url, False
>>> req.Send
>>> url = "https://www.example.com/" 'this has no effect on req!
>>> req.Send
This method cannot be called after the send method has been called.
(0xC00C0240)
>>> req.Open "GET", url, False 'now the request uses the new URL
>>> req.Send
Whether a token can be re-used depends entirely on how the request is processed by the server. If a token is for one-time use only you cannot expect to be able to successfully send the same request twice.
Related
I wrote a Python script which makes several https requests one after the other by following redirections (its purpose is to login to AWS Cognito via a load balancer):
import requests
session = requests.session()
response = session.get(
f'https://<initial_url>',
allow_redirects=False,
verify=False,
)
response = session.get(
response.headers["Location"],
allow_redirects=False,
verify=False,
)
response = session.post(
response.headers["Location"],
allow_redirects=False,
verify=False,
data={
"_csrf": session.cookies["XSRF-TOKEN"],
"username": <user>,
"password": <password>,
},
)
This works as expected. I would like to use the urllib3 library instead of requests and I transformed the script as follows:
import urllib3
http = urllib3.PoolManager(cert_reqs="CERT_NONE")
response = session.request(
"GET",
f'https://<initial_url>',
redirect=False,
retries=False,
)
response = session.request(
"GET",
response.headers["Location"],
redirect=False,
retries=False,
)
csrf=<get the XSRF-TOKEN cookie from response.headers["Set-Cookie"]>
fields = {
"_csrf": csrf,
"username": <user>,
"password": <password>,
}
response = session.request(
"POST",
response.headers["Location"],
redirect=False,
retries=False,
fields=fields,
)
The GET requests work and the redirects are as expected, but the POST does not (I get an error from Cognito). Based on the documentation
https://docs.python-requests.org/en/latest/user/quickstart/
and
https://urllib3.readthedocs.io/en/latest/user-guide.html
I understand that the equivalent of requests/data is urllib3/fields, as both are used to form-encode data
In the first version of the code, I created a new requests.session() object before the POST request and used it for that and I got the same error as in the urrlib3 case, which led me to believe that urllib3.PoolManager() does not provide a session like requests.session() and that each request is made separately, which makes the POST request fail. Does anyone know if there is a way to have a session in urrlib3 ? I could not find anything in the documentation
I'm trying to authenticate the user after visiting the registration link
(link example: http://127.0.0.1:8000/confirm-email?token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9F)
My code:
#app.get("/confirm-email", status_code=200, )
def confirm_email(
token: str = fastapi.Query(..., min_length=64, max_length=256,
db: Session = fastapi.Depends(database.get_db)):
if user := crud.read_user_by(db, column='current_token', value=token):
if user.created_datetime + timedelta(minutes=30) > datetime.now(): # TODO change minutes to days
return fastapi.responses.RedirectResponse(
url="http://127.0.0.1:8000/users/me",
headers={"access_token": token, "token_type": "bearer"})
else:
raise fastapi.HTTPException(
status_code=fastapi.status.HTTP_410_GONE,
detail="Confirmation link is expired")
else:
raise fastapi.HTTPException(
status_code=fastapi.status.HTTP_401_UNAUTHORIZED,
detail="Wrong token")
#app.get("/users/me")
def read_users_me(token: str = fastapi.Depends(oauth2_scheme),
db: Session = fastapi.Depends(database.get_db)):
try:
return services.get_user_by_token(db=db, token=token)
except Exception as e:
raise fastapi.HTTPException(
status_code=fastapi.status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
headers={"WWW-Authenticate": "Bearer"},
)
But every time I'm failing when trying to use /users/me endpoint (getting 401 error, UNAUTHORIZED).
Maybe I put the token in the wrong place or using wrong headers?
If using OAuth 2.0 and wanting to set the access_token in a request, tipically, it goes into the Authorization header like the example in the RFC: Authorization: Bearer mF_9.B5f-4.1JqM - in the example, mF_9.B5f-4.1JqM would be the value of the token.
It seems to me that you are accessing the users/me endpoint with the headers access_token: [token value] and token_type: "bearer". Instead, I believe the following header should be set: Authorization: Bearer [token value]
After a little researching, I figured out that redirection by specification can't have authorization headers (browser/client will just ignore it mainly). So even if headers are correct - it's nonsense. One possible solution to use URL.
I want to use requests to web scrape on a login site. I already done the code using selenium but it is very inconvenient and slower to do it that way as I want to make it public(every user has to download chrome driver).
The problem is, there are multiple requests from the site and I don't have any experience processing that data and extracting the header data and name. Any help is great, thanks.
[Premise]
Using requests module you can send requests in these way:
import requests
url = "http://www.example.com" # request url
headers = { # headers dict to send in request
"header_name": "headers_value",
}
params = { # params to be encoded in the url
"param_name": "param_value",
}
data = { # data to send in the request body
"data_name": "data_value",
}
# Send GET request.
requests.get(url, params=params, headers=headers)
# Send POST request.
requests.post(url, params=params, headers=headers, data=data)
Once you perform a request, you can get much information from the response object:
>>> import requests
# We perform a request and get the response object.
>>> response = requests.get(url, params=params, headers=headers)
>>> response = requests.post(url, params=params, headers=headers, data=data)
>>> response.status_code # server response status code
>>> 200 # eg.
>>> response.request.method
>>> 'GET' # or eventually 'POST'
>>> response.request.headers # headers you sent with the request
>>> {'Accept-Encoding': 'gzip, deflate, br'} # eg.
>>> response.request.url # sent request url
>>> 'http://www.example.com'
>>> response.response.body
>>> 'name=value&name2=value2' # eg.
In conclusion, you can retrieve all the information that you can find in Dev Tools in the browser, from the response object. You need nothing else.
Dev Tools view
Dev Tool view 2
Once you send a GET or POST requests you can retrieve information from Dev Tools:
In General:
Request URL: the url you sent the request to. Corresponds to response.request.url
Request Method: corresponds to response.request.method
Status Code: corresponds to response.status_code
In Response Headers:
You find response headers which correspond to response.headers
eg. Connection: Keep-Alive,
Content-Length: 0,
Content-Type: text/html; charset=UTF-8...
In Requests Headers:
You find request headers which correspond to response.request.headers
In Form Data:
You can find the data you passed with data keyword in requests.post.
Corresponds to response.request.body
I'm trying to use the Groovy HTTPBuilder library to delete some data from Firebase via a HTTP DELETE request. If I use curl, the following works
curl -X DELETE https://my.firebase.io/users/bob.json?auth=my-secret
Using the RESTClient class from HTTPBuilder works if I use it like this:
def client = new RESTClient('https://my.firebase.io/users/bob.json?auth=my-secret')
def response = client.delete(requestContentType: ContentType.ANY)
However, when I tried breaking down the URL into it's constituent parts, it doesn't work
def client = new RESTClient('https://my.firebase.io')
def response = client.delete(
requestContentType: ContentType.ANY,
path: '/users/bob.json',
query: [auth: 'my-secret']
)
I also tried using the HTTPBuilder class instead of RESTClient
def http = new HTTPBuilder('https://my.firebase.io')
// perform a POST request, expecting TEXT response
http.request(Method.DELETE, ContentType.ANY) {
uri.path = '/users/bob.json'
uri.query = [auth: 'my-secret']
// response handler for a success response code
response.success = { resp, reader ->
println "response status: ${resp.statusLine}"
}
}
But this also didn't work. Surely there's a more elegant approach than stuffing everything into a single string?
There's an example of using HttpURLClient in the tests to do a delete, which in its simplest form looks like:
def http = new HttpURLClient(url:'https://some/path/')
resp = http.request(method:DELETE, contentType:JSON, path: "destroy/somewhere.json")
def json = resp.data
assert json.id != null
assert resp.statusLine.statusCode == 200
Your example is very close to the test for the delete in a HTTPBuilder.
A few differences I see are:
Your path is absolute and not relative
Your http url path doesn't end with trailing slash
You're using content type ANY where test uses JSON. Does the target need the content type to be correct? (Probably not as you're not setting it in curl example unless it's doing some voodoo on your behalf)
Alternatively you could use apache's HttpDelete but requires more boiler plate. For a HTTP connection this is some code I've got that works. You'll have to fix it for HTTPS though.
def createClient() {
HttpParams params = new BasicHttpParams()
HttpProtocolParams.setVersion(params, HttpVersion.HTTP_1_1)
HttpProtocolParams.setContentCharset(params, "UTF-8")
params.setBooleanParameter(ClientPNames.HANDLE_REDIRECTS, true)
SchemeRegistry registry = new SchemeRegistry()
registry.register(new Scheme("http", PlainSocketFactory.getSocketFactory(), 80))
ClientConnectionManager ccm = new PoolingClientConnectionManager(registry)
HttpConnectionParams.setConnectionTimeout(params, 8000)
HttpConnectionParams.setSoTimeout(params, 5400000)
HttpClient client = new DefaultHttpClient(ccm, params)
return client
}
HttpClient client = createClient()
def url = new URL("http", host, Integer.parseInt(port), "/dyn/admin/nucleus$component/")
HttpDelete delete = new HttpDelete(url.toURI())
// if you have any basic auth, you can plug it in here
def auth="USER:PASS"
delete.setHeader("Authorization", "Basic ${auth.getBytes().encodeBase64().toString()}")
// convert a data map to NVPs
def data = [:]
List<NameValuePair> nvps = new ArrayList<NameValuePair>(data.size())
data.each { name, value ->
nvps.add(new BasicNameValuePair(name, value))
}
delete.setEntity(new UrlEncodedFormEntity(nvps))
HttpResponse response = client.execute(delete)
def status = response.statusLine.statusCode
def content = response.entity.content
I adopted the code above from a POST version, but the principle is the same.
I'm creating an QuickTest script that get all URL's for a page and check if the status of http request where code is 200. But my question is how do I get the http request status code from a URL ?
This was my solution:
Set Http = CreateObject("Microsoft.XMLHttp")
Set oDesc = Description.Create()
oDesc("micclass").Value = "Link"
Set EditCollection = Browser("Browser").Page("Page").ChildObjects(oDesc)
URL_TEMP = EditCollection(i).GetROProperty("href")
Http.Open "GET" , URL_TEMP , false
Http.Send
msgbox Http.Status