I edited this question to clarify why I asked this question again (I had weak Google-Fu and found these rather old 1 2 3 pretty-much-duplicates only after posting).
Approaches to accessing a password-protected resources that I've seen in the wild.
Plaintext storage in script (might often end up being shared, or in a Dropbox)
Plaintext storage in a config script
You can do password = readline("Password: ") but of course the password ends up in plaintext in the console (and thus in console logs etc.), so might as well store it in a plaintext config file.
I found this little trick to avoid displaying the password in the Terminal, but running system("stty -echo") on OS X Mavericks leads to the error stty: stdin isn't a terminal, so I guess it wouldn't be particularly portable.
Using tcltk. Has the unfortunate effect of making Rstudio crash and being difficult to install.
keychain. It's not on CRAN, so I don't think I can use this as a first-line approach, I'd also like a bit more detail about where and how passwords are stored on various systems (i.e. will it end up in plaintext on Windows?).
Access tokens, OAuth etc. seem to have similar problems.
I don't know any R packages which use PGP for connections? Probably also a bit difficult for newbie users.
I'm not asking for myself mainly, but I want to provide somewhat sensible defaults for nontechnical users who might store plaintext passwords enabling access to sensitive data in their Dropbox.
Unlike others who asked similar questions, I could also change the server-side of things if I had a better approach.
Are there best-practice approaches that I'm currently missing? My focus on interactive sessions is because I assume that's how most nontechnical types use R, but of course it would be nice if it worked during e.g. knitr report generation too.
Some suggestions to solve your problem securely. These solutions match all programming languages.
Establish a secure connection to your resource without R, like a SSL tunnel.
If you need a secure password in R to establish a secure connection, then you can read this from a secure config file and remove this password variable if you don't use the password anymore. A secure config file is a config file that is not part of your code repository (Git, SVN, ...). You have to manage your secret independent of your code. This mean separate your code and your secrets. One simple way is to put your private and secure secret in your private and secure user home directory. Then you have delegated your security problem to your operating system. Your secret is now save as your OS and your home directory. Pleas check the rights of your home directory and enable the file system encryption if they are off. Notice, this is the way like Maven handle passwords.
You get more security if you encrypt your password/secret config file. Then you have second line of defense.
For most applications is point 2 enough.
Notice, be sure that your secret is not deployed with your code. You need a second way to manage and deploy your secret to production systems.
Notice, be sure that if your programs jams, that your secret is not in memory anymore.
Notice, use always strong algorithms for encryption. Don't implement your own security algorithm, is a high complexity task. Better use standard implementations of strong encryption algorithms.
Related
I am currently using Solr to perform search services over some sensitive records.
As Solr/lucene provides fast searching by storing inverted indexes of the sensitive information in plain text on a disk there is a requirement to encrypt these index files so that unauthorized people can't have access to them by bypassing the system's security.
I found there are similar patches open on Apache JIRA AES encrypted directory and Codec for index-level encryption.
AES encrypted directory looks promising but this patch has been implemented for lucene 3.1 as I am using the newer version, I am not sure if this patch can be used with lucene version 5 or higher.
I was wondering if there is a way to implement a security measure that encrypts the indexes or if it is possible to write some custom plugin which can encrypt/decrypt the indexes on I/O level(i.e FsDirectory)?
The discussion in the comment section of LUCENE-6966 you have shared is really interesting. I would reason with this quote of Robert Muir that there is nothing baked into Solr and probably will never be.
More importantly, with file-level encryption, data would reside in an unencrypted form in memory which is not acceptable to our security team and, therefore, a non-starter for us.
This speaks volumes. You should fire your security team! You are wasting your time worrying about this: if you are using lucene, your data will be in memory, in plaintext, in ways you cannot control, and there is nothing you can do about that!
Trying to guarantee anything better than "at rest" is serious business, sounds like your team is over their head.
So you should consider to encrypt the storage Solr is using on OS level. This should be transparent for Solr. But if someone comes into your system, he should not be able to copy the Solr data.
This is also the conclusion the article Encrypting Solr/Lucene indexes from Erick Erickson of Lucidwors draws in the end
The short form is that this is one of those ideas that doesn't stand up to scrutiny. If you're concerned about security at this level, it's probably best to consider other options, from securing your communications channels to using an encrypting file system to physically divorcing your system from public networks. Of course, you should never, ever, let your working Solr installation be accessible directly from the outside world, just consider the following: http://server:port/solr/update?stream.body=<delete><query>*:*</query></delete>!
One suggested way to run jobs is to save context parameters in properties files.
Like this one:
#
#Wed Dec 16 18:23:03 CET 2015
MySQL_AdditionalParams=noDatetimeStringSync\=true
MySQL_Port=3306
MySQL_Login=root
MySQL_Password=secret_password_to_cipher
MySQL_Database=talend MySQL_Server=localhost
This is really easy and useful, but the issue with this is that passwords are saved in clear.
So I'm looking for ways to do easily ciphering.
Here are 2 very insteresting questions already discussed in Stack overflow about password ciphering technics:
Encrypt passwords in configuration files
Securing passwords in properties file
But they are Java native and I'm searching for a better Talend integration. I've already tried different ways in my Talend jobs:
Simple obfuscation using base64 encoding of passwords
Using tEncrypt and tDecrypt components from the forge
Using Jasypt ot JavaXCrypto librairies
Using pwdstore routine from the forge
All these technics are described in a tutorial (in french, sorry) explaining how to crypt passwords in Talend
But another issue is encountered: keys used to cipher/uncipher are always in clear, so if you know good ways to address this point I'll be glad to experiment it.
Fundamentally, anything an application can reach can be reached by somebody breaking in into the system/taking over control of the application.
Even if you use obfuscation (such as base64 or more advanced), or real encryption where the keys are available (even if they too might be obfuscated).
So essentially there is no good enough way to do what you seek to do and worse: it simply cannot exist.
So what do you do instead ?
1. Limit the rights
MySQL_Login=root is a big problem ... a compromise of the application will lead to an immediate compromise of the database (and its data).
So, limit the rights to what is absolutely needed for the application.
This should really be done and is quite easy to achieve.
2. Separate user and admin level access
If certain things are only needed after user interaction, you can use secrets provided by the user (e.g. a password of the user can give a hash and that can be xor-ed with and get you a key that's not always present in the application nor configuration files).
You can use this e.g. to separate out permissions in two levels: the normal user level which only has the bare minimal rights to make the application work for the average user, (but e.g. not the application management rights that allow managing the application itself), and use the secrets kept by the user to keep (pert of) the key outside of the application while there's no admin logged in into the administrative part of the application.
This is rarely done to be honest, nor all that easy.
But even with all that you essentially have to consider the access to e.g. the database to be compromised if the application is compromised.
That's also why data such as application user password should not (must not) be stored in the database without proper precautions.
I have developed a tool that loads in an configuration file at runtime. Some of the values are encrypted with an AES key.
The tool will be scheduled to run on a regular basis from a remote machine. What is an acceptable way to provide the decryption key to the program. It has a command line interface which I can pass it through. I can currently see three options
Provide the full key via CLI, meaning the key is available in the clear at OS config level (i.e. CronJob)
Hardcode the key into the binary via source code. Not a good idea for a number of reasons. (Decompiling and less portable)
Use a combination of 1 and 2 i.e. Have a base key in exe and then accept partial key via CLI. This way I can use the same build for multiple machines, but it doesn't solve the problem of decompiling the exe.
It is worth noting that I am not too worried about decompiling the exe to get key. If i'm sure there are ways I could address via obfuscation etc.
Ultimately if I was really conscious I wouldn't be storing the password anywhere.
I'd like to hear what is considered best practice. Thanks.
I have added the Go tag because the tool is written in Go, just in case there is a magical Go package that might help, other than that, this question is not specific to a technology really.
UPDATE:: I am trying to protect the key from external attackers. Not the regular physical user of the machine.
Best practice for this kind of system is one of two things:
A sysadmin authenticates during startup, providing a password at the console. This is often extremely inconvenient, but is pretty easy to implement.
A hardware device is used to hold the credential. The most common and effective are called HSMs (Hardware Security Modules). They come in all kinds of formats, from USB keys to plug-in boards to external rack-mounted devices. HSMs come with their own API that you would need to interface with. The main feature of an HSM is that it never divulges its key, and it has physical safeguards to protect against it being extracted. Your app sends it some data and it signs the data and returns it. That proves that that the hardware module was connected to this machine.
For specific OSes, you can make use of the local secure credential storage, which can provide some reasonable protection. Windows and OS X in particular have these, generally keyed to some credential the admin is required to type at startup. I'm not aware of a particularly effective one for Linux, and in general this is pretty inconvenient in a server setting (because of manual sysadmin intervention).
In every case that I've worked on, an HSM was the best solution in the end. For simple uses (like starting an application), you can get them for a few hundred bucks. For a little more "roll-your-own," I've seen them as cheap as $50. (I'm not reviewing these particularly. I've mostly worked with a bit more expensive ones, but the basic idea is the same.)
With the included zip utility in CentOS5 (zip command): i.e. /usr/bin/zip -P $ZIP_PASS ...
I haven't been able to find any documentation on this. I am trying to find out how secure the password is. Is this encrypted? How is it protected if not with encryption?
Thanks
Ok I found the answer I was looking for.... From http://en.wikipedia.org/wiki/ZIP_%28file_format%29 and http://en.wikipedia.org/wiki/Known-plaintext_attack
Encrypted file archives such as ZIP are prone to this attack.[citation
needed] For example, an attacker with an encrypted ZIP file needs only
(part of) one unencrypted file from the archive which forms the
"known-plaintext".[4][5] Then using some publicly available software
they can quickly calculate the key required to decrypt the entire
archive. To obtain this unencrypted file the attacker could search the
website for a suitable file, find it from another archive they can
open, or manually try to reconstruct a plaintext file armed with the
knowledge of the filename from the encrypted archive.
So... the zip is not completely secure - but with random file names (when implemented well) for the file(s) inside the zip, and immediate deletion of the unencrypted file (which is also non-web-accessible) - this appears to be a POSSIBLE solution...
More resources:
http://linux.101hacks.com/archive-compression/password-protection-for-zip-files/
However, the more I read, AES 256bit encrypted zips by 7zip (once installed on the server) is much, much more secure. It is NOT susceptible to the known plaintext attack, either.
Yes, it is encrypted. Per the manual:
-P password
use password to encrypt zipfile entries (if any). THIS IS INSECURE! Many multi-user operating
systems provide ways for any user to see the current command line of any other user; even on
stand-alone systems there is always the threat of over-the-shoulder peeking. Storing the
plaintext password as part of a command line in an automated script is even worse. Whenever
possible, use the non-echoing, interactive prompt to enter passwords. (And where security is
truly important, use strong encryption such as Pretty Good Privacy instead of the relatively
weak encryption provided by standard zipfile utilities.)
This is just found from running "man zip" on a centOS machine.
As the manual entry notes, for high security it is not good to use the password in the command to zip the file, as others could check the process list and see the password as zipping.
I don't know what kind of encryption this uses, but I looked around some and it doesn't appear to be great. There are some similiar questions on stackoverflow to this. Look into using a different zip library if encryption is truly important, use something that allows AES, such as GPG.
I know there are plenty of questions here already about this topic (I've read through as many as I could find), but I haven't yet been able to figure out how best to satisfy my particular criteria. Here are the goals:
The ASP.NET application will run on a few different web servers, including localhost workstations for development. This means encrypting web.config using a machine key is out. Each "type" or environment of web server (dev, test, prod) has its own corresponding database (dev, test, prod). We want to separate these connection strings so that a developer working on the "dev" code is not able to see any "prod" connection string passwords, nor allow these production passwords to ever get deployed to the wrong server or committed to SVN.
The application will should be able to decide which connection string to attempt to use based on the server name (using a switch statement). For example, "localhost" and "dev.example.com" will should know to use the DevDatabaseConnectionString, "test.example.com" will use the TestDatabaseConnectionString, and "www.example.com" will use the ProdDatabaseConnectionString, for example. The reason for this is to limit the chance for any deployment accidents, where the wrong type of web server connects to the wrong database.
Ideally, the exact same executables and web.config should be able to run on any of these environments, without needing to tailor or configure each environment separately every time that we deploy (something that seems like it would be easy to forget/mess up one day during a deployment, which is why we moved away from having just one connectionstring that has to be changed on each target). Deployment is currently accomplished via FTP. Update: Using "build events " and revising our deployment procedures is probably not a bad idea.
We will not have command-line access to the production web server. This means using aspnet_regiis.exe to encrypt the web.config is out. Update: We can do this programmatically so this point is moot.
We would prefer to not have to recompile the application whenever a password changes, so using web.config (or db.config or whatever) seems to make the most sense.
A developer should not be able to get to the production database password. If a developer checks the source code out onto their localhost laptop (which would determine that it should be using the DevDatabaseConnectionString, remember?) and the laptop gets lost or stolen, it should not be possible to get at the other connection strings. Thus, having a single RSA private key to un-encrypt all three passwords cannot be considered. (Contrary to #3 above, it does seem like we'd need to have three separate key files if we went this route; these could be installed once per machine, and should the wrong key file get deployed to the wrong server, the worst that should happen is that the app can't decrypt anything---and not allow the wrong host to access the wrong database!)
UPDATE/ADDENDUM: The app has several separate web-facing components to it: a classic ASMX Web Services project, an ASPX Web Forms app, and a newer MVC app. In order to not go mad having the same connection string configured in each of these separate projects for each separate environment, it would be nice to have this only appear in one place. (Probably in our DAL class library or in a single linked config file.)
I know this is probably a subjective question (asking for a "best" way to do something), but given the criteria I've mentioned, I'm hoping that a single best answer will indeed arise.
Thank you!
Integrated authentication/windows authentication is a good option. No passwords, at least none that need be stored in the web.config. In fact, it's the option I prefer unless admins have explicity taken it away from me.
Personally, for anything that varies by machine (which isn't just connection string) I put in a external reference from the web.config using this technique: http://www.devx.com/vb2themax/Tip/18880
When I throw code over the fence to the production server admin, he gets a new web.config, but doesn't get the external file-- he uses the one he had earlier.
you can have multiple web servers with the same encrypted key. you would do this in machine config just ensure each key is the same.
..
one common practice, is to store first connection string encrypted somewhere on the machine such as registry. after the server connects using that string, it will than retrieve all other connection strings which would be managed in the database (also encrypted). that way connection strings can be dynamically generated based on authorization requirements (requestor, application being used, etc) for example the same tables can be accessed with different rights depending on context and users/groups
i believe this scenario addresses all (or most?) of your points..
(First, Wow, I think 2 or 3 "quick paragraphs" turned out a little longer than I'd thought! Here I go...)
I've come to the conclusion (perhaps you'll disagree with me on this) that the ability to "protect" the web.config whilst on the server (or by using aspnet_iisreg) has only limited benefit, and is perhaps maybe not such a good thing as it may possibly give a false sense of security. My theory is that if someone is able to obtain access to the filesystem in order to read this web.config in the first place, then they also probably have access to create their own simple ASPX file which can "unprotect" it and reveal its secrets to them. But if unauthorized people are trouncing around in your filesystem—well… then you have bigger problems at hand, so my whole concern is now moot! 1
I also realize that there isn’t a foolproof way to securely hide passwords within a DLL either, as they can eventually be disassembled and discovered, perhaps by using something like ILDASM. 2 An additional measure of security obscurity can be obtained by obfuscating and encrypting your binaries, such as by using Dotfuscator, but this isn’t to be considered “secure.” And again, if someone has read access (and likely write access too) to your binaries and filesystem, you’ve again got bigger problems at hand methinks.
To address the concerns I mentioned about not wanting the passwords to live on developer laptops or in SVN: solving this through a separate “.config” file that does not live in SVN is (now!) the blindingly obvious choice. Web.config can live happily in source control, while just the secret parts do not. However---and this is why I’m following up on my own question with such a long response---there are still a few extra steps I’ve taken to try and make this if not any more secure, then at least a little bit more obscure.
Connection strings we want to try to keep secret (those other than the development passwords) won’t ever live as plain text in any files. These are now encrypted first with a secret (symmetric) key---using, of course, the new ridiculous Encryptinator(TM)! utility built just for this purpose---before they get placed in a copy of a “db.config” file. The db.config is then just uploaded only to its respective server. The secret key is compiled directly into the DAL’s dll, which itself would then (ideally!) be further obfuscated and encrypted with something like Dotfuscator. This will hopefully keep out any casual curiosity at the least.
I’m not going to worry much at all about the symmetric "DbKey" living in the DLLs or SVN or on developer laptops. It’s the passwords themselves I’ll keep out. We do still need to have a “db.config” file in the project in order to develop and debug, but it has all fake passwords in it except for development ones. Actual servers have actual copies with just their own proper secrets. The db.config file is typically reverted (using SVN) to a safe state and never stored with real secrets in our subversion repository.
With all this said, I know it’s not a perfect solution (does one exist?), and one that does still require a post-it note with some deployment reminders on it, but it does seem like enough of an extra layer of hassle that might very well keep out all but the most clever and determined attackers. I’ve had to resign myself to "good-enough" security which isn’t perfect, but does let me get back to work after feeling alright about having given it the ol’ "College Try!"
1. Per my comment on June 15 here http://www.dotnetcurry.com/ShowArticle.aspx?ID=185 - let me know if I'm off-base! -and some more good commentary here Encrypting connection strings so other devs can't decrypt, but app still has access here Is encrypting web.config pointless? and here Encrypting web.config using Protected Configuration pointless?
2. Good discussion and food for thought on a different subject but very-related concepts here: Securely store a password in program code? - what really hit home is the Pidgin FAQ linked from the selected answer: If someone has your program, they can get to its secrets.