DetectContentType, JavaScript support ?
https://github.com/golang/go/blob/c3931ab1b7bceddc56479d7ddbd7517d244bfe17/src/net/http/sniff.go#L21
Is there a genuine reason behind the http Method DetectContentType to not support JavaScript ?
As the doc comment notes, DetectContentType implements the algorithm described at https://mimesniff.spec.whatwg.org/, which does not detect JavaScript. The question then becomes: why doesn't it?
The answer is given in the introduction of the spec:
These security issues are most severe when an "honest" server allows potentially malicious users to upload their own files and then serves the contents of those files with a low-privilege MIME type. For example, if a server believes that the client will treat a contributed file as an image (and thus treat it as benign), but a user agent believes the content to be HTML (and thus privileged to execute any scripts contained therein), an attacker might be able to steal the user’s authentication credentials and mount other cross-site scripting attacks. (Malicious servers, of course, can specify an arbitrary MIME type in the Content-Type header field.)
This document describes a content sniffing algorithm that carefully balances the compatibility needs of user agent with the security constraints imposed by existing web content.
Labelling untrusted input as JavaScript when it's not (or even when it is!) could lead to security disasters.
I have created an API endpoint, I have a user of that endpoint requesting from servers at stackoverflow.com. I want to verify that the request was made from stackoverflow.com servers. One way I could verify it came from stackoverflow.com is to ask the developer to sign the request with their let's encrypt domain private key. I can then use their public key to decrypt the message.
I'm not totally sure I can decrypt the privately encrypted message with their public key but even if I could, I would like to avoid having the developer do any special type of encryption. Could I use TLS to verify the origin domain?
TLS supports client authentication, also called '2-way' or 'mutual' authentication. (SSL3 also did, but you should not be using SSL3.) See e.g. TLS1.2
'updated' for ECC and TLS1.3.
How to use this depends on the software (typically library or middleware) being used for TLS, which you didn't indicate; it is even possible some TLS stack doesn't support it at all, though I've never heard of any. Some stacks or use-cases allow client auth to be invoked without any code change, and others with only minimal or localized code change.
Some details that may or may not matter:
this does not sign the request. It authenticates the TLS connection (to be exact, it normally signs a transcript of the handshake) and then the data transferred over the connection is MACed (as well as encrypted) using keys created (and thus authenticated) by the handshake. This provides authentication but not nonrepudiation for the data; you the receiver can reliably determine it came from the sender, but you can't reliably prove this to a third party. For the closely related case of 'proving' the server, see the numerous crossdupes linked at https://security.stackexchange.com/questions/205074/is-it-possible-to-save-a-verifiable-log-of-a-tls-session .
this authenticates the data was sent by the identified client; it says nothing about the origin which as Sam Jason points out is often different.
the client is not necessarily identified by a domain name; it can be a person, organization, or something else. However, many CAs issue a single cert for both TLS server auth and client auth (look at the ExtendedKeyUsage extension in your own or any sample cert(s) to see) and in that case with few exceptions the subject is identified by a domain name or name(s) or at least wildcard(s).
I'm pretty sure you should be using some sort of API key or maybe something similar to how twilio signs its requests
one reason for these patterns is that it's common for HTTP requests to be proxied, with static requests handled by something other than the code/application server. therefore the TLS connection would have been terminated at the proxy server, and the actual application code wouldn't be able to easily see anything about the TLS connection used by the remote server
I have a question related to Share cookie between subdomain and domain - what would happen if I set the domain while setting cookie as .com? Would the cookie be available to all .com websites?
Well-configured user-agents should reject such cookies, as explained in RFC 6265 section 5.3:
If the user agent is configured to reject "public suffixes" and the domain-attribute is a public suffix:
If the domain-attribute is identical to the canonicalized request-host:
Let the domain-attribute be the empty string.
Otherwise:
Ignore the cookie entirely and abort these steps.
NOTE: A "public suffix" is a domain that is controlled by a
public registry, such as "com", "co.uk", and "pvt.k12.wy.us".
This step is essential for preventing attacker.com from
disrupting the integrity of example.com by setting a cookie
with a Domain attribute of "com". Unfortunately, the set of
public suffixes (also known as "registry controlled domains")
changes over time. If feasible, user agents SHOULD use an
up-to-date public suffix list, such as the one maintained by
the Mozilla project at http://publicsuffix.org/.
I've recently enabled Digest Authentication on an intranet website/application I am creating for my company in ASP.NET.
The reason I have done so is because Windows Authentication seemed to only work for some users, and not for others. I could not figure out why nor do I know enough about IIS to try and trace the issue. After some trial and error, I found that digest authentication seemed to give me the behaviour that I wanted. That is: allow only users with a valid account on the domain to log in to the website with their credentials.
The problem now, is that Firefox (3+) seems to ask for the user to authenticate on every HTTP request sent to the server. This does not appear to occur in Internet Explorer (6+) or Chrome.
I've tried searching for solutions but I always arrive at dead-ends. I'll find a discussion about the issue, and every posted solution leads to a dead link...or it's on Experts Exchange and I don't have access to view to solution.
The issue appears to be related (from what I've read) to the way the different browsers send their authentication headers vs how IIS interprets them. I'm not sure what I can do to change this though? One of the solutions I had found mentioned writing an ISAPI filter to fix this, but of course the link to the finished filter was broken and I have no idea how to go about making one myself.
I've tried messing with the NTLM and other auth related strings in about:config to try and force Firefox to trust my server but that doesn't seem to work either.
From a few other sources I've read, it appears that everything should work if I switch back to Windows Authentication, but then I'm back at square one where the authentication would work only for some users and not others.
A solution for either problem would work for me, but I have very little information for the Windows Authentication issue. If someone could guide me through tracing the problem I'd gladly post more information for it as well.
Here are the URLs I've found discussing what seems like the same problem. (Sorry I couldn't make them all links, it wouldn't let me post otherwise)
support.mozilla.com/tiki-view_forum_thread.php?locale=pt-BR&forumId=1&comments_parentId=346851
www.experts-exchange.com/Software/Internet_Email/Web_Browsers/Mozilla/Q_24427378.html
channel9.msdn.com/forums/TechOff/168006-Twin-bugs-in-IIS-IE-unfair-competitive-advantage-EDIT-SOLVED/
www.derkeiler.com/Newsgroups/microsoft.public.inetserver.iis.security/2006-03/msg00141.html
This is a know bug in FF. See Advanced digest authentication works from Internet Explorer however we receive multiple authentication prompts on each GET request from fire fox
IE 6 had the same bug.A potential workaround would be to re-enable "old" Digest in IIS6:
http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/1d6e22ac-0215-4d12-81e9-c9262c91b797.mspx?mfr=true
Currently, if the server send an opaque directive, the IE client will return this directive value as specified in the RFC. Unfortunately, for follow-on requests from the client where the nonce count is incremented (count 2 and beyond) the opaque directive value is not sent. This then fails authentication on the server and a 401 Unauthorized is returned. The IE client now requests the username and password for the new challenge and the file is retrieved.
This requires an additional round trip and the user is prompted for credential each time.
The RFC states that the opaque must always be sent on requests from the client.
The Digest implementation that IE6 is using is not RFC compliant (http://www.ietf.org/rfc/rfc2617.txt).
3.2.2 The Authorization Request Header
The values of the opaque and algorithm fields must be those supplied
in the WWW-Authenticate response header for the entity being
requested.
3.3 Digest Operation
A client should remember the username, password, nonce, nonce count and
opaque values associated with an authentication session to use to
construct the Authorization header in future requests within that
protection space.
Because the client is required to return the value of the opaque
directive given to it by the server for the duration of a session,
the opaque data may be used to transport authentication session state
information.
-------- Edit addition -----
Windows Authentication seemed to only work for some users, and not for others.
How did it fail? Did you enable impersonation?
If on a login screen user submits a form with their username and password, the password is sent in plain text (even with POST, correct me if I am wrong).
What is the right way to protect the user and his password against the third party who might be eavesdropping on the communication data?
I am aware that HTTPS is a solution to the problem, but is there any way to ensure at least some level of security using the standard HTTP protocol (POST request)? (perhaps using javascript in some way)
What I was about was a page - that is a PHP-generated login page, which is of course sent to users in HTTP GET request as an HTML file. There is no (#Jeremy Powel) connection established between the server and the client so I can't create such a handshaking protocol. And I want the complete process to be transparent to the user - he wants to submit a password, not deal with cryptography.
Using HTTP with SSL will make your life much easier and you can rest at ease. Very smart people (smarter than me at least!) have scrutinized this method of confidential communication for years.
Secure authentication is a broad topic. In a nutshell, as #jeremy-powell mentioned, always favour sending credentials over HTTPS instead of HTTP. It will take away a lot of security related headaches.
TSL/SSL certificates are pretty cheap these days. In fact if you don't want to spend money at all there is a free letsencrypt.org - automated Certificate Authority.
You can go one step further and use caddyserver.com which calls letsencrypt in the background.
Now, once we got HTTPS out of the way...
You shouldn't send login and password via POST payload or GET parameters. Use an Authorization header (Basic access authentication scheme) instead, which is constructed as follows:
The username and password are combined into a string separated by a
colon, e.g.: username:password
The resulting string is encoded using
the RFC2045-MIME variant of Base64, except not limited to 76
char/line.
The authorization method and a space i.e. "Basic " is then
put before the encoded string.
source: Wikipedia: Authorization header
It might seem a bit complicated, but it is not.
There are plenty good libraries out there that will provide this functionality for you out of the box.
There are a few good reasons you should use an Authorization header
It is a standard
It is simple (after you learn how to use them)
It will allow you to login at the URL level, like this: https://user:password#your.domain.com/login (Chrome, for example will automatically convert it into Authorization header)
IMPORTANT:
As pointed out by #zaph in his comment below, sending sensitive info as GET query is not good idea as it will most likely end up in server logs.
The Authorization header value is traditionally a base64-encoded username/password. Base64 is not encryption. The original value can be obtained by an on-path attacked using a simple base64-decode.
You can use a challenge response scheme. Say the client and server both know a secret S. Then the server can be sure that the client knows the password (without giving it away) by:
Server sends a random number, R, to client.
Client sends H(R,S) back to the server (where H is a cryptographic hash function, like SHA-256)
Server computes H(R,S) and compares it to the client's response. If they match, the server knows the client knows the password.
Edit:
There is an issue here with the freshness of R and the fact that HTTP is stateless. This can be handled by having the server create a secret, call it Q, that only the server knows. Then the protocol goes like this:
Server generates random number R. It then sends to the client H(R,Q) (which cannot be forged by the client).
Client sends R, H(R,Q), and computes H(R,S) and sends all of it back to the server (where H is a cryptographic hash function, like SHA-256)
Server computes H(R,S) and compares it to the client's response. Then it takes R and computes (again) H(R,Q). If the client's version of H(R,Q) and H(R,S) match the server's re-computation, the server deems the client authenticated.
To note, since H(R,Q) cannot be forged by the client, H(R,Q) acts as a cookie (and could therefore be implemented actually as a cookie).
Another Edit:
The previous edit to the protocol is incorrect as anyone who has observed H(R,Q) seems to be able to replay it with the correct hash. The server has to remember which R's are no longer fresh. I'm CW'ing this answer so you guys can edit away at this and work out something good.
If your webhost allows it, or you will need to deal with sensitive data, then use HTTPS, period. (It's often required by the law afaik).
Otherwise if you want to do something over HTTP. I would do something like this.
The server embeds its public key into the login page.
The client populates the login form and clicks submit.
An AJAX request gets the current timestamp from the server.
Client side script concatenates the credentials, the timestamp and a salt (hashed from analog data eg. mouse movements, key press events), encrypts it using the public key.
Submits the resulting hash.
Server decrypts the hash
Checks if the timestamp is recent enough (allows a short 5-10 second window only). Rejects the login if the timestamp is too old.
Stores the hash for 20 seconds. Rejects the same hash for login during this interval.
Authenticates the user.
So this way the password is protected and the same authentication hash cannot be replayed.
About the security of the session token. That's a bit harder. But it's possible to make reusing a stolen session token a bit harder.
The server sets an extra session cookie which contains a random string.
The browser sends back this cookie on the next request.
The server checks the value in the cookie, if it's different then it destroys the session, otherwise all is okay.
The server sets the cookie again with different text.
So if the session token got stolen, and a request is sent up by someone else, then on the original user's next request the session will be destroyed. So if the user actively browsing the site, clicking on links often, then the thief won't go far with the stolen token. This scheme can be fortified by requiring another authentication for the sensitive operations (like account deletion).
EDIT: Please note this doesn't prevent MITM attacks if the attacker sets up their own page with a different public key and proxies requests to the server.
To protect against this the public key must be pinned in the browser's local storage or within the app to detect these kind of tricks.
About the implementation: RSA is probably to most known algorithm, but it's quite slow for long keys. I don't know how fast a PHP or Javascript implementation of would be. But probably there are a faster algorithms.
You can use SRP to use secure passwords over an insecure channel. The advantage is that even if an attacker sniffs the traffic, or compromises the server, they can't use the passwords on a different server. https://github.com/alax/jsrp is a javascript library that supports secure passwords over HTTP in the browser, or server side (via node).
I would use a server-side and client-side Diffie-Hellman key exchange system with AJAX or multiple form submits(I recommend the former), although I don't see any good implementations thereof on the internet. Remember that a JS library can always be corrupted or changed by MITM. Local storage can be used to help combat this, to an extent.
HTTPS is so powerful because it uses asymmetric cryptography. This type of cryptography not only allows you to create an encrypted tunnel but you can verify that you are talking to the right person, and not a hacker.
Here is Java source code which uses the asymmetric cipher RSA (used by PGP) to communicate:
http://www.hushmail.com/services/downloads/
you can use ssl for your host there is free project for ssl like letsencrypt
https://letsencrypt.org/
Using https sounds best option here (certificates are not that expensive nowadays). However if http is a requirement, you may use some encription - encript it on server side and decript in users browser (send key separately).
We have used that while implementing safevia.net - encription is done on clients (sender/receiver) sides, so users data are not available on network nor server layer.