Consider the following situation:
I have a MVC setup where the view invokes a mapper to get a specific URL for a href attribute. The mapper holds keys to make the referencing easy.
Now the view requests an invalid key, so the mapper responds with say /invalid_url. But the dispatcher knows this URL and when a user clicks the link an error message will be displayed about how bad we feel.
But what if the Google Bot visits this invalid URL?
What would a search engine friendly status code be? I feel like 500 would be appropriate because it is a server side failure. But then this has the feeling of being a temporary error or somehow not related to the URL but to the internal mechanics. The other option that comes to mind is 404. This is also valid because the requested page does not exist. However 4xx errors are client side errors ("You requested the wrong URL. So it's basically your error"). And it just doesn't feel like a client side error to me.
Am I overthinking things? Should I just go with 404?
When I have a question like this, I refer to a site like http://www.restapitutorial.com/httpstatuscodes.html
5xx error codes imply that the server made a mistake. But you're saying the view (client-side) requested the wrong key. That sounds like a client-side error to me. While it's not the user's fault, the server doesn't know this, and the requested URL really doesn't exist. So a 404 would be appropriate.
But this is a weird case, and you should still want to fix the underlying issue of the client consistently(?) requesting a bad url.
Related
I am at my wits end with the following problem:
My site www.sebastianthalhammer.com is available under that URL without any problems.
However Google Search Console as well as other external third party test tools return a 404 error.
Status report from Uptrends
It is just the main page that's affected. All the other subpages and blog content isn't affected.
I have been in contact with the server stuff but it seems alright to them. As mentioned. The site can be reached. The site runs on wordpress - latest version.
I have no real clue where to start as this error seems to be quite a tricky one. Does anyone here might have an idea what's going on?
Sebastian
The 4xx class of status code is intended for cases in which the
client seems to have erred. Except when responding to a HEAD
request, the server SHOULD include a representation containing an
explanation of the error situation, and whether it is a temporary or
permanent condition. These status codes are applicable to any
request method. User agents SHOULD display any included
representation to the user.
This leaves me with two possible explanations:
Explanation 1: it's a server error.
the server wrongly returns a 404 status code
the browser thinks the response body contains details about the error and displays it - for the end user this is the actual page
Explanation 2: it's done on purpose to defeat crawlers and page watchers.
the server returns 404 on purpose - non-browser user agents won't process the result as they interpret it as error
browsers are unaffected, the end user doesn't care as long as the page is being displayed
The second one would indeed be kind of clever if you don't want your page to be indexed.
Thanks to your feedback I could think about the problem in a different way.
Ultimately at the unholy depths of a certain plugin I could dig out a setting that caused the error.
It was a redirection plugin that (for whatever reason) sent out a 404 signal when the URL was requested.
I don't know what the purpose would be for something. All I know is that the setting was on default for quite a while now and that caused the weird situation.
thanks guys for getting me on the right track.
Sebastian
I'm a noob when it comes to ASP.NET. I know few basic commands such as Response.Redirect("URL") to redirect my application web page to a different location.
However i receive HTTP Error 400 - Bad Request, whenever i try to use the code shown below
Response.Redirect(Server.UrlEncode(this.Downloadlink));
where this.Downloadlink is a user defined property which returns something like this
http://mdn.vatsag.net/fp;files/DOWNLOAD/VTSetup.exe
If i post this link in the browser, the .exe file pops up (means the link is good)
However this error comes when i use the ASP.NET code.
Any form of response on this issue/reason is deeply appreciated.
See here: http://www.kirit.com/Response.Redirect%20and%20encoded%20URIs
In short: if you quickly want to fix the issue, remove the part of your code that is UrlEncoding the URL!
For example, day, in a somewhat REST-oriented environment, a request comes in for an object that doesn't exist, like:
GET http://example.com/thing/5
Is there anything wrong with sending back a 404 response who's body is the same as a a different page? For example, responding like:
404 body: [content from "http://example.com/thing/" which is a list of things]
Does it make any sense to do this? Will this cause any problems with certain browsers? Is it confusing to the user? Or is this perfectly fine to do?
Along these same lines, I would have the content of the 404 response match the request's accept headers as best I could. (ie. abide by content negotiation with the user agent)
For example, a xml or json request would get something along the lines of a simple error message and something that says "look here for similar things", while an html request would get an HTML page that has the error message as well as the content of the list page (as I indicated above)
I think it depends on how the Restful web services are being consumed. If I'm programmatically consuming the web service from a different application, then I would want the status code together and a plain text message instead of a message decorated with HTML tags. I mean, say for example, it doesn't make sense to return a bloated 404 content if your user makes the web service call using Curl because the message will not be readable to them.
You could have different "consumes" for each restful webservice. If it's an XML request, then you return 404 and a plain text message. Otherwise, you return the error page content.
I don't see anything wrong with it. In our webservice we always send back a json error object which includes stacktraces and other details about the response. Even on a regular web server, you get at least text which can be displayed in a browser saying that you got a 404 response.
Say I have a website that allows anyone to log in through oauth or similar, but only allows certain uses to create or modify content. Should they somehow make a request for page for creating a new post, I'll do a check and redirect them if they don't have the appropriate permissions.
It is considered acceptable to redirect to the "403 Error" page in this situation? There was no actual HTTP response with a 403 status code, there was no database- or server- level query that was failed - just my business logic. Am I misappropriating the idea of HTTP status codes if I serve an error 403 page with a specific explanatory message?
You are free to do so, but I think if you want to expose an API you would use an actual 403 response because they carry meaning that will be nicely handled by the client.
If you want to display a page to the client and will be using redirect, you will lose this meaning of the "403".
Isn't it better to just redirect them to an explanation page without including the "403" code. Or better yet, redirect them to a more helpful place, like the sign up page if that is what they have to do to make a post, or back to the original page with a floating message.
We want to help the user get closer to their goals instead of confusing them with technical error codes.
There is often a lot of discussion about this very topic and it comes down to the following choices:
a 5xx? Of course not. This is not a server error.
a 400? Not really, it wasn't a malformed request.
a 401? Probably not, 401 is generally for authorization in general, not application-level permissions. If your user has already logged in but has the wrong role, and you want to let the user know, then use something else.
a 404? Perhaps, as the server can't find the resource for this particular user, but if you want to tell the user "well such a resource is available but you can't have it because you lack permissions" then go with something else.
a 403? Actually, this one makes a lot of sense. Here is the definition from the RFC
403 Forbidden The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. If the server does not wish to make this information available to the client, the status code 404 (Not Found) can be used instead.
In your question you mention your intention to redirect the user. If you are making a RESTFUL web service then just return the 403. If you are doing an entire web app, you can control the 403 and redirect....
So, from here...
In ASP.NET, you have a choice about how to respond to that - it's in the web.config as CustomErrors. Turn that on, then redirect to a fancy 404 page (maybe you already do). The fancy 404 page, then, could be checking the requested querystring (which gets passed over to the custom error page as yet another querystring) to see if it's a valid redirect, lives in your database, etc. Just do a Response.Redirect() from there.
Then schooner writes:
Thanks, we do have a 404 now but we would prefer this not to be detected as a 404 in the process. We would like ot handle it directly and seperately if possible.
..and I'd like to know just how bad a practice this is. I don't expect to put my "pretty" URLs on the internet (just business cards) and I have a sample of 404-redirecting-to-a-helpful-site code working, but I don't want to get to production and have an issue with a browser that takes the initial 404 too seriously. Can anyone help me understand more about why I wouldn't want to use customErrors / 404 to flow users to the page they actually wanted?
The main problem with using customeErrors as your 404 error handler is that every time customErrors picks up an errored request rather than throwing a 404 error back to your browser and letting your browser know there was a bad request, it instead returns a 302 which indicates that a page has been relocated to whatever your customErrors page is. This isn't bad for most users because they don't know or even notice the difference, the problem comes from the fact that web crawlers DO know the difference and the status code they receive directly affects how their indexing works.
Consider the scenario where you have a page at http://mysite.com/MyAwesomePageAboutStuff.aspx for some period of time and then one day you decide you no longer need it and delete the file. If Google or some other crawler has already indexed that URL and goes back to it after you delete it the crawler will get a 302 status code instead of a 404 error and because of this status code the crawler will update the page's url to point to your error page rather deleting the non-existent link. Now, whenever someone finds that url by way of a search engine they'll end up at your error page.
It's not really a huge issue, but you can definitely see the headaches this can create for your users in the long run.
Look here for some corroborating data.
I created a vanity url system using the 404 as the handler. There's no need for a 302 on my side as the 404 dynamically loads the content and returns that. I am fully able to handle any and all POST / GET and SERVER data.
Works great. If you are interested TarantulaHawk is up on SourceForge.