HTTP to HTTPS issues - http

I have a question, I am a bit confused, I don't really understand why this is happening.
I have a website which works well over http. When I force redirect to https something happens. Even if I replace all my urls in my code, only GET request will work. Anybody has any idea why is this happening?
I also have admin part of the website. it works to login into the admin but it doesn't work to make any requests on it. I am trying to post or delete but I receive a 401 err, even if I am logged in and set the token right...
So bottom line is:
On Https, the website works, it shows all the resources from the db, I can login in the Admin but I can not post or delete.
On Http everything works.
I am in a huge need of advice or ideas.
thanks.

From my experience you cannot serve mixed content, that's my first suggestion is to call all your scripts/dependencies without the prefix; ie: script src="https://blahblah" to "script src="//blahblah"; you're going to make sure you are sticking consistently to one serving source; so that's the first thing I'd check (also look at console logs, they often give hints as to what failed);
Secondly I am unsure of the response or how the server handles traffic from non https, possibly there's a rule in htaccess or some form of redirection trying to force the call via https so http fails? these are all steps in debugging right you need to troubleshoot and play process of eliminations; first though I'd make sure we are serving everything from // or https; when on http I would look at console logs for clues but even more so I would force a redirect to use https exclusively (as most sites do now)
Check for mixed content issues first though, this is something that can have a multitude of solutions based on the many variations of what could be causing this issue.

Related

How to create a HTTPS->HTTPS subdirectory redirect using subdomains?

I am currently having issues with setting up an HTTPS domain redirect. I have a DNS URL redirect entry that points a few sub-domains to same-server URLs. For example:
docs.kipper-lang.org -> kipper-lang.org/docs/
play.kipper-lang.org -> kipper-lang.org/playground
The issue I am currently experiencing is that when using the subdomains, it mostly works, but it can only use HTTP. If I attempt to use HTTPS (like for example https://docs.kipper-lang.org) the redirect won't work and will get stuck apparently waiting for the HTTPS certificate (I think, but I don't know for sure, since it loads forever and gets a time-out).
So my DNS provider does its job for the most part as I want, but I am not sure how to add the HTTPS encryption to these redirects. Is there maybe even some DNS configuration or even middle-man service for redirects I can use, where these HTTPS encryptions are built-In? Since receiving a "Warning: Insecure connection" every time someone uses the sub-domains is a massive problem for me.
Note though that considering I am hosting on a GitHub Pages server, I am unable to do these redirects on the server side myself, as I can't use any code in this case.
I would greatly appreciate any ideas for fixing this or what I could use to achieve this another way.
Thanks in advance!

Serving 404 directly

So I have an Nginx server set up which is supposed to redirect all http to https (and non-www to www) using 4 server blocks.
The issue is that any 404 or non existent http URL first get a 301 redirect to what could have been an https version if it hypothetically existed (hence creating an extra URL and redirect).
See example:
1) http://example.com/thisurldoesntexit
301 Redirect
2) https://example.com/thisurldoesntexit
404
3) https://example.com/notfound
Is there a way to redirect user directly to a https 404 (URL 3)?
First of all, as already been pointed out, doing a 301 redirect from a non-existent page to a single /notfound moniker, is a really bad practice, and is likely against the RFCs.
What if the user simply mistyped a single character of a long URL? Modern browsers make it non-trivial to go back to what has been typed in order to correct it. The user would have to decide whether your site is worth a retyping from scratch, or whether your competitor might have thought of a better experience.
What if the user simply followed a broken link, which is broken in a very obvious way, and could be easily fixed? E.g., http://www.example.org/www.example.com/page, where an absolute URL was mistyped by the creator to be a relative one, or maybe a URI like /page.html., with an extra dot in the end. Likewise, you'll be totally confusing the user with what's going on, and offering a terrible user experience, where if left alone, the URL could easily have been corrected promptly.
But, more importantly, what real problem are you actually trying to solve?!
For better or worse, it's a pretty common practice to indiscriminately redirect from http to https scheme, without an account of whether a given page may or may not exist. In fact, if you employ HSTS, then content served over http effectively becomes meaningless; the browser with a policy would never even be requesting anything over http from there on out.
Undoubtedly, in order to know whether or not a given page exists, you must consult with the backend. As such, you might as well do the redirect from http to https from within your backend; but it'll likely tie up your valuable server resources for little to no extra benefit.
Moreover, the presence or absence of the page may be dictated by the contents of the cookies. As such, if you require that your backend must discern whether a page does or does not exist for an http request, then you'll effectively be leaking private information that was meant to be protected by https in the first place. (In turn, if your site has no such private information, then maybe you shouldn't be using https in the first place.)
So, overall, the whole approach is just a REALLY, REALLY bad idea!
Consider instead:
Do NOT do a 301 redirect from all non-existent pages to a single /notfound page. Very bad practice, very bad UX.
It is totally OK to do an indiscriminate redirect from http to https, without accounting for whether or not the page exists. In fact, it's not only okay, but it's the way God intended, because an adversary should not be capable of discerning whether or not a given page exists for an https-based site, so, if you do find and implement a solution for your "problem", then you'll effectively create a security vulnerability and a data leak.
Use https://www.drupal.org/project/fast_404 module for serving 404 pages directly without much overload.
I'd suggest redirecting to a 404 page is a poor choice, and you should instead serve the 404 on the incorrect URL.
My reasons for stating this are:
By redirecting away from the page, you are issuing headers that implicitly say "The content does not exist on this URL, but it does over here". I'm not sure how the various search engines would react to being redirected to a 404
I can speak from my own experience as a user when I say that having the URL change on me when I've mis-typed by a single character can be very frustrating. I then need to spend the time to type out the entire URL again.
You can avoid having logic in your .htaccess file or whatever to judge a page as a 404. This will greatly simplify your initial logic (which by-the-by gets computed on every single page load) - and will remove far more redirects than just the odd one of http://badurl to https://badurl to https://404

How do I generate a 403 error when someone tries to access a particular page

I may be barking up completely the wrong tree here but what I would like to do is protect my .js pages by having them return a 403 Forbidden http error status page if someone tries to access them directly via http. I use them to support my index.html page but would like for them to remain hidden.
The helpdesk guys at my ISP basically say they don't know if it's possible but it may be something you could do with a web.config file (which is not something I have used before).
Any help at all would be gratefully received - I am a bit out of my comfort zone with this one
I would like to […] protect my .js pages by having them return a 403 Forbidden http error status page if someone tries to access them directly via http.
Please note that if you include some resource, for example a script via the <script>-tag in HTML or an image via the <img>-tag, the browser does nothing else than simply run another HTTP request to get that resource. The whole communication already happens over HTTP.
While a browser may include additional details in its HTTP request when requesting additional resources, like the Referer-header, it definitely is not required to do so. So if you look out for the Referer-header, be advised that you may lock out other valid clients which do not send the Referer-header in their requests.
Also note that this will not give you any protection whatsoever. One can simply construct HTTP headers when requesting things, so “faking” requests your server would allow (because it thinks they are correct) is not a problem at all. And even without that; every resource you tell the client to use to make your website work will be downloaded by the client. And after that, the client can do whatever he wants with it. It can cache them on the hard disk, or allow the user to quickly look at it without having to run another request.
So if you want to do this for protecting your code, then just forget about it, and make it easier for everyone by just not adding a non-optimal protection. Code you put on the web can be made difficult to read, but if you want the user to see the end result, then you also give out your code in the same step.
In php you can do this with:
header("HTTP/1.0 403 Forbidden");

How can a bot get the contents of subsequent pages in a category listing in WordPress?

I'm writing a bot to automatically download pages from my WordPress blog. The bot gets most of the pages without a problem. For example, it can easily get the first page of the article listing of a given tag: http://example.com/myblog/index.php/archives/tag/mytag. However, for some reason it can't get the subsequent pages, like http://example.com/myblog/index.php/archives/tag/mytag/page/2.
I've tried to figure out what was going on, and here's what I found: while the server answers normally to most requests, upon such requests it answers with a 301 permanent redirect. Peculiarly, the Location header is set to the exact same URL as the request! Basically, the server tells me to redirect my request of the page http://example.com/myblog/index.php/archives/tag/mytag/page/2 to... the very same page :P
When trying to access the page from the browser I get the page without a problem. I thought maybe the browser sends some headers (including cookies) that my bot doesn't send, so I copied the headers (including the cookies) from my browser's web console, but the behaviour didn't change.
I would appreciate any suggestions regarding what might be causing this strange behaviour, what I can do in order to understand what's going on better, and of course what I can do in order to fetch those pages automatically, just like I fetch their brethren.
Thanks!
It seems this post hasn't generated much public interest. However, in case somebody ever runs into the same problem and finds this post, here's the solution I used. Important note: I still don't understand the behaviour I witnessed, and would appreciate it if somebody could explain it.
So the solution I've found is basically to use the URL http://example.com/myblog/archives/tag/mytag?paged=2 instead of http://example.com/myblog/index.php/archives/tag/mytag/page/2. Funnily enough, this URL gets redirected to the original one when browsed to from a browser! But when the bot requested it it got the page without redirection or anything. (So I managed to do what I wanted to do, but I've got no idea what happened there, why there was a problem in the first place, and why this solution worked: for one URL the bot gets infinite redirection and the browser just gets the page, while for the other the browser gets redirected [finitely] and the bot gets the page. I am yet to figure this one out...)

Going to a page without "www" in my app causes the page to not load

We've recently run into an issue with our ASP.NET application where if a user goes to ourcompany.com instead of www.ourcompany.com, they will sometimes end up on a page that does not load data from the database. The issue seems to be related to our SSL certificate, but I've been tasked to investigate a way on the code side to fix this.
Here's the specific use case:
There is a user registration page that new users get sent to after they "quick register" (enter name, email, phone). With "www" in the URL (e.g. "www.ourcompany.com") it works fine, they can proceed as normal. However, if they browsed to just "ourcompany.com" or had that bookmarked, when they go to that page some data is not loaded (specifically a list of states from the DB) and, worse, if they try to submit the page they are kicked out entirely and sent back to the home page.
I will go in more detail if necessary but my question is simply if there is an application setting I can say to keep the session for the app regardless of if the URL has the "www" or not? Buying a second SSL cert isn't an option at this point unless there is no recourse, and I have to look at a way to solve this without another SSL.
Any ideas to point me in the right direction?
When your users go to www.ourcompany.com they get a session cookie for the www subdomain. By default, cookies are not shared across subdomains, which is why users going to ourcompany.com do not have access to their sessions.
There is a useful thread discussing this issue here. The suggested solution is:
By the way, I implemented a fairly good fix/hack today. Put this code
on every page: Response.Cookies["ASP.NET_SessionId"].Value =
Session.SessionID; Response.Cookies["ASP.NET_SessionId"].Domain =
".mydomain.com";
Those two lines of code rewrite the Session cookie so it's now
accessible across sub-domains.
Doug, 23 Aug 2005
Surely you are trying to solve the wrong problem?
Is it possible for you to just implement URL rewriting and make it consistent?
So for example, http://example.com redirects to http://www.example.com ?
For an example of managing rewriting see:
http://paulstack.co.uk/blog/post/iis-rewrite-tool-the-pain-of-a-simple-rule-change.aspx
From the browsers point of view, www.mysite.com is a different site than mysite.com.
If you have a rewrite engine, add a rule to send all requests to www that don't already have it.
Or (this is what I did) add a separate IIS site with the "mysite.com" host header and set the IIS flag to redirect all traffic to www.
In either of these cases, any time a browser requests a page without the www prefix, it will receive a redirect response sending it to the correct page.
Here's the redirect site home directory properties:
And the relevant host header setting:
This fixes the issue without requiring code changes, and incidentally prevents duplicate search results from Google etc.
Just an update, I was able to fix the problem with a web.config entry:
<httpCookies domain=".mycompany.com" />
After adding that, the problem went away.

Resources