As the title says the site in question is secure and I can't share my credentials but here's the outline of events.
The way the site security works is you send a POST to one url with user/pass and then it sends back a token. All requests then need to carry that token in their headers to work. I can get that to work once. On the first request after the login step I get the results I want. All subsequent requests result in a http 500 error of "Internal Server Error". Of course, in a perfect world, I could go to the server and get logs to see more verbosely what is going on. However, they aren't so accommodating on my planet so I'm left scratching my head.
Just to clarify I can send the exact same request the second time and I get the aforementioned error. So far my work around is to detach httr and then relibrary(httr) to start over. This doesn't seem like it's the best approach for this problem.
I'm guessing that the problem has to do with how httr reuses the same handle but I don't know what info is changing between the two requests.
In pseudo code let's say I do
resp<-POST('https://my.site.com/login', add_headers(.headers=c('user'='me', 'pass'='blah'))
mytoken<-content(resp)$token
qry<-POST('https://my.site.com/soap/qry', add_headers(.headers=c('token'=mytoken)),body=myxmlstring)
#qry will have status 200 and the content I expect
#If I run the same POST command again
qry2<-POST('https://my.site.com/soap/qry', add_headers(.headers=c('token'=mytoken)),body=myxmlstring)
#qry2 will be status code 500
#if I do
detach("package:httr", unload=TRUE)
library(httr)
#and then do the commands again from the top then it will work again.
Ideally, there'd be a parameter I can add to POST which will make each POST completely independent of the last. Short of that I'd be happy with something that makes more sense than detaching and reattaching the package itself.
Related
I'm trying to fetch data from a website (https://gesetze.berlin.de/bsbe/search). Using Mozilla, I've taken a look at the network analysis. Usually, I'm just messing around with the parameters of the POST-Request to see how I might influence the response of the server. But when I simply re-send the request (making no changes at all), I'm getting HTTP-response 500. The server answer states as message: security_notAuthenticated.
Can anyone explain that behaviour? The request is done by the same PC, the same browser in the same session, and there is no login function on that website. Pictures shown below.
Picture 1 - Code 200
Picture 2 - Code 500
The response security_notAuthenticated indicates, that your way of repeating the request omits some authentication-related information.
When I repeat the request, using Mozilla Firefox's "Resend" or "Edit and resend" function, the Cookie header is not sent with the request. Although it occurs in the editable header list when using "Edit and resend" it's missing when watching the actual sent request. I'm not sure whether this is a feature or a bug.
When using Firefox's "Use as Fetch in Console" function, the header will automatically be included and you still have the ability to change the headers and the body. The fetch API is a web standard and some introductory material about fetch can be found on MDN.
If you want to do custom requests, in the browser, fetch is a good option.
In other environments and languages you usually use some HTTP client (just search the web for "...your language... http request" or similar, you will find something).
I've encountered a weird situation, after registration we're sending an email with a verification link, pretty standard stuff, but somehow clicking on the link seems to make the request twice, looking at the logs, the first time it comes from my IP and the second request comes from some Google IP: 66.102.8.60 (doing a reverse lookup shows google-proxy-66-102-8-60.google.com).
Any idea what's going on and how to prevent this?
The server is running Nginx and the site is Ruby on Rails if that helps.
I do not know the root cause but my best guess is same as Tripleee wrote above - most probably google is scanning urls. This happens in all browsers (well at least in Chrome and Firefox), but only under following circumstances:
the url is clicked from gmail (if you copy paste it to browser tab, the second request is not issued)
the url is clicked for the first time... Subsequent clicks from the same email do not trigger second request
I know it is probably not the answer you expected, but after giving it some thought I figured that operation like this should be handled on server side. In my case I am tracking information about confirmation urls anyways, so the first time the request comes to my backend I am deleting it and proceeding with confirmation normally. Since the confirmation entry is missing in the database for the second request it returns immediately with status 404, 422 or something whatever suits you.
Hope that helps anyone who gets here looking for an answer to this problem ;)
I cannot get a success response using the example code shown in the here documentation: https://developer.here.com/documentation/routing/dev_guide/topics/request-constructing.html
I created a Freemium account and generated two each of JS and REST api keys. Regardless of which key I try, I keep getting errors that seem to change with each attempt:
Sometimes a 404
Sometimes a 502
Sometimes a 504
Sometimes a 403 with a message I might understand if it happened every time:
{
Message: "User: anonymous is not authorized to perform: es:ESHttpGet"
}
Sometimes a 200 with a page not found error message from something called
Platform's Radar
Sometimes a 302 that redirects to a login for Kibana using a Live Nation account
For reference, a specific request from the documentation that I have been trying:
https://route.ls.hereapi.com/routing/7.2/calculateroute.json?apikey={API_KEY}&waypoint0=geo!52.5,13.4&waypoint1=geo!52.5,13.45&mode=fastest;car;traffic:disabled
I am really not sure what is going on here.
On replicating the API call, it is resulting 200 every time. For reference find screenshot, please use correct APIkey
(This is sort of an abstract philosophical question. But I believe it has objective concrete answers.)
I'm writing an API, my API has a "status" page (like, https://status.github.com/).
If whatever logic I have in place to determine the status says everything is good my plan would be to return 200 OK, and a JSON response with more information about each service tested by my status page.
But what if my logic says the API is down? Say the database isn't responding or something.
I think I want to return 500 INTERNAL SERVER ERROR (or 503 SERVICE NOT AVAILABLE) along with a JSON response with more details.
However, is that breaking the HTTP Status Code spec? Would that confuse end users? My status page itself is working just fine in that case. So maybe it should return 200? But that would mean anyone using it would have to dig into the body looking for a specific parameter to determine the API's status vs. just checking the HTTP Status Code. (Also if my status page itself was broken, I'm fine with the end user taking that to mean the API is down since that's a pretty bad sign...)
Thoughts? Is there official protocol on how a status page should work?
https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
For me the page should return 200 unless has problems itself. Is true that is easier to check the status code of a response than parsing but using HTTP status codes to encode application informations breaks what people (and spiders) expect. If a spider passes for your page and sees a 500 or 503 will think your site has a page with problems, not that that page is ok and is signaling that the site is down.
Also, as you notice, it wont' be possible to distinguish between the service is down and the status page is down cases, with the last the only one that should send 500. Also, what if you show more than one service like the twitter status page ? Use 200.
Related: https://stackoverflow.com/a/943021/1536382 https://stackoverflow.com/a/34324179/1536382
I have an inherited VB6 program that runs each morning to download Google Analytics feed data for several clients. The process does the following:
Checks that my current OAuth 2.0 Access Token (saved in my database from yesterday) has not expired, and if so obtains a new one. Naturally, when the program runs for the first time each day it gets a new Access Token.
For each client, post a request for feed data.
Processes the XML data received from the Google server.
My problem is with step 2 above. The first post using the new Access Token always fails with a 400 Bad Request error. Making a second post using the exact same data always succeeds and my program can move to step 3.
Here is an example of my POST (with Client Id and Access Token in [ ] brackets):
https://www.google.com/analytics/feeds/data?
ids=ga%3A[Client Id]&
start-date=2016-01-10&
end-date=2016-01-10&
metrics=ga%3Asessions%2Cga%3Atransactions%2Cga%3AtransactionRevenue&
dimensions=ga%3Amedium%2Cga%3Asource%2Cga%3Akeyword&
filters=ga%3Asource%3D%3Dshopping&
access_token=[Access Token]
This has been occurring for several weeks.
The error description (Err.Description) from my code is "400 Bad Request". The entire response from the Google server (less HTML) is "400. That's an error. Your client has issued a malformed or illegal request. That's all we know."
Can anyone offer any suggestions as to why the first request fails, but subsequent requests don't? I have even built in a five minute delay between getting the new Access Token and making the first data request, but still get the 400 Bad Request Error.
Any help you can offer would be greatly appreciated.
Thanks.
Using Fiddler (thank you M Schenkel) I was able to finally track the source of the problem.
My VB6 program is using the IP*WORKS! SSL component from nsoftware. The form had a single HTTPS component that was being used for (1) getting a new Access Token, and then (2) for getting the feed data.
Fiddler showed that the second use of the component was using some of the parameters from the first use of it.
I added a second HTTPS component to the form so my feed data request would start off with a blank slate and it worked.
Thank you very much for your help!