I'm writing tests for my python code and need to find a web page that returns status code grater then 307, if someone knows pages like that, plz link them. Thanks
I also wanted this some time back but couldn't fine , however some of my test I used below linked for below HTTP error codes. Basically these are some rest api calls. you can try it.
HTTP error code 401 if you call
http://gdata.youtube.com/feeds/api/standardfeeds/r
HTTP error 400 gives if you call
http://gdata.youtube.com/feeds/api/standardfeeds/
Related
I am trying to crawl the website https://www.rightmove.co.uk/properties/105717104#/?channel=RES_NEW
but I get (410) error
INFO: Ignoring response <410 https://www.rightmove.co.uk/properties/105717104>: HTTP status code is not handled or not allowed
I am just trying to find the properties that have been sold using the notification on the page "This property has been removed by the agent."
I know the website has not blocked me because I am able to use the scrappy shell to get the data and also view(response) works fine too, I can directly go to the same URL using web browser so the 410 doesn't make sense I can also crawl pages from the same domain,
(ie) the pages without the notification "This property has been removed by the agent."
Any help would be much appreciated.
Seem's the when a listing has been marked as removed by and agent on Rightmove then the website will return status code 410 Gone (Which is quite weird). But to solve this, simply do something like this in your request:
def start_requests(self):
yield scrapy.Request(
url='https://www.rightmove.co.uk/properties/105717104#/?channel=RES_NEW',
meta={
'handle_httpstatus_list': [410],
}
)
EDIT
Explanation: Basically, Scrapy will only handle the status code from the response is in the range 200-299, since 2XX means that it was a successful response. In your case, you got a 4XX status code which means that some error happened. By passing handle_httpstatus_list = [410] we tell Scrapy that we want it to also handle 410 responses and not only 200-299.
Here is the docs: https://docs.scrapy.org/en/latest/topics/spider-middleware.html#std-reqmeta-handle_httpstatus_list
I cannot get a success response using the example code shown in the here documentation: https://developer.here.com/documentation/routing/dev_guide/topics/request-constructing.html
I created a Freemium account and generated two each of JS and REST api keys. Regardless of which key I try, I keep getting errors that seem to change with each attempt:
Sometimes a 404
Sometimes a 502
Sometimes a 504
Sometimes a 403 with a message I might understand if it happened every time:
{
Message: "User: anonymous is not authorized to perform: es:ESHttpGet"
}
Sometimes a 200 with a page not found error message from something called
Platform's Radar
Sometimes a 302 that redirects to a login for Kibana using a Live Nation account
For reference, a specific request from the documentation that I have been trying:
https://route.ls.hereapi.com/routing/7.2/calculateroute.json?apikey={API_KEY}&waypoint0=geo!52.5,13.4&waypoint1=geo!52.5,13.45&mode=fastest;car;traffic:disabled
I am really not sure what is going on here.
On replicating the API call, it is resulting 200 every time. For reference find screenshot, please use correct APIkey
(This is sort of an abstract philosophical question. But I believe it has objective concrete answers.)
I'm writing an API, my API has a "status" page (like, https://status.github.com/).
If whatever logic I have in place to determine the status says everything is good my plan would be to return 200 OK, and a JSON response with more information about each service tested by my status page.
But what if my logic says the API is down? Say the database isn't responding or something.
I think I want to return 500 INTERNAL SERVER ERROR (or 503 SERVICE NOT AVAILABLE) along with a JSON response with more details.
However, is that breaking the HTTP Status Code spec? Would that confuse end users? My status page itself is working just fine in that case. So maybe it should return 200? But that would mean anyone using it would have to dig into the body looking for a specific parameter to determine the API's status vs. just checking the HTTP Status Code. (Also if my status page itself was broken, I'm fine with the end user taking that to mean the API is down since that's a pretty bad sign...)
Thoughts? Is there official protocol on how a status page should work?
https://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html
For me the page should return 200 unless has problems itself. Is true that is easier to check the status code of a response than parsing but using HTTP status codes to encode application informations breaks what people (and spiders) expect. If a spider passes for your page and sees a 500 or 503 will think your site has a page with problems, not that that page is ok and is signaling that the site is down.
Also, as you notice, it wont' be possible to distinguish between the service is down and the status page is down cases, with the last the only one that should send 500. Also, what if you show more than one service like the twitter status page ? Use 200.
Related: https://stackoverflow.com/a/943021/1536382 https://stackoverflow.com/a/34324179/1536382
I'm a noob when it comes to ASP.NET. I know few basic commands such as Response.Redirect("URL") to redirect my application web page to a different location.
However i receive HTTP Error 400 - Bad Request, whenever i try to use the code shown below
Response.Redirect(Server.UrlEncode(this.Downloadlink));
where this.Downloadlink is a user defined property which returns something like this
http://mdn.vatsag.net/fp;files/DOWNLOAD/VTSetup.exe
If i post this link in the browser, the .exe file pops up (means the link is good)
However this error comes when i use the ASP.NET code.
Any form of response on this issue/reason is deeply appreciated.
See here: http://www.kirit.com/Response.Redirect%20and%20encoded%20URIs
In short: if you quickly want to fix the issue, remove the part of your code that is UrlEncoding the URL!
I use ASIHTTPRequest framework in my iPhone application to manage the HTTP communication. I followed their examples given in the project home page to send asynchronous requests. There we have to implement two callbacks, one for requestFinish and other for requestFailed.
However, 60% of the time requests sent by the iPhone ends up in the "requestFinish" callback method and gives a valid HTTP status code. But sometimes it goes to "requestFailed" callback and the status code become '0' which is confusing.
My investigations revealed that the internet connection is ok, and I am sending the request to the correct URL, however no log messages found in server log.
So why does the request gets dropped in the middle so randomly? Has anyone came across with this type of issue. Will be very helpful.
Thanks
Are you looking at the status code of the ASIHTTPRequest object? That code is simply an HTTP response code - if you didn't get a response, then that should be zero.
Instead, you want to look at the NSError object that the delegate failure callback gives you.
I would use something like:
NSLog(#"%#",[error localizedErrorDescription]);
To print out to the log what the error is. Of course, "error" is the name of the variable in the method signature - you should double-check that, I think that's what the default is.
In [ASHTTPRequest initialize], I changed
[sharedQueue setMaxConcurrentOperationCount:4]
to
[sharedQueue setMaxConcurrentOperationCount:10]
This work for me, but I don't know why.
MORE:
I found this.
As well as what phooze suggested, there is logging in ASIHTTPRequestConfig.h that you can enable, that may provide a clue as to what is happening.