I've been trying to set up encrypted folder following the instructions in https://wiki.archlinux.org/index.php/ECryptfs#Without_ecryptfs-utils. In that case a mount passphrase is manually created and wrapped using a chosen password ("Arch" in that example). This is working as expected.
There is yet another possibility how to manually set up things. Namely using simple mount e.g.:
mount -t ecryptfs ~/.Private/ ~/Private -o key=passphrase,ecryptfs_cipher=aes,ecryptfs_key_bytes=32,ecryptfs_passthrough=n,ecryptfs_enable_filename_crypto=y
In this case I'm prompted for a [password] and after that folder is mounted.
My question is: where is the mount passphrase hidden in the second example? And how is my entered [password] related to it and to the FEKEK in this case.
Could anybody please explain?
Thank you in advance.
I think the entered passphrase becomes the "file encryption key, encryption key, or FEKEK" itself, and is temporarily stored in the kernel keyring (root's keyring if you need to mount with sudo) while the folder is mounted. You can see the keyring before & after mounting & unmounting with sudo keyctl show.
It does not store the passphrase/FEKEK anywhere on disk, unlike the method "Without ecryptfs-utils" in the ArchWiki you liked to, and the one in the ecryptfs-utils below.
FYI, using the ecryptfs-utils tools it will create a random (16 byte) passphrase to use, if you haven't specified your own passphrase, and it apparently is the file encryption key (FEKEK) / FNEK.
Looking in a couple of the ecryptfs-utils tools, (several are shell scripts) inside ecryptfs-migrate-home it collects some data and calls ecryptfs-setup-private, which has this function to generate a more secure 16 byte random passphrase/FEKEK/FNEK (that then gets "wrapped"/encrypted with your login passphrase and stored on disk):
random_passphrase () {
bytes=$1
# Pull $1 of random data from /dev/urandom,
# and convert to a string of hex digits
od -x -N $bytes --width=$bytes /dev/urandom | head -n 1 | sed "s/^0000000//" | sed "s/\s*//g"
}
Related
I have some files in my repository, and one contains a secret Adafruit key. I want to use Git to store my repository, but I don't want to publish the key.
What's the best way to keep it secret, without having to blank it out everytime I commit and push something?
Depending on what you're trying to achieve you could choose one of those methods:
keep file in the tree managed by git but ignore it with entry in gitignore
keep file content in environment variable,
don't use file with key at all, keep key content elsewhere (external systems like hashicorp's vault, database, cloud (iffy, I wouldn't recommend that), etc.)
First approach is easy and doesn't require much work, but you still has the problem of passing the secret key to different location where you'd use the same repository in a secure manner. Second approach requires slightly more work, has the same drawback as the first one.
Third requires certainly more work then 1st and 2nd, but could lead to a setup that's really secure.
This highly depends on the requirements of the project
Basically, the best strategy security-wise is not to store keys, passwords and in general any vulnerable information inside the source control system. If its the goal, there are many different approaches:
"Supply" this kind of information in Runtime and keep it somewhere else:
./runMyApp.sh -db.password=
Use specialized tools (like, for example Vault by Hashicorp) to manage secrets
Encode the secret value offline and store in git the encoded value. Without a secret key used for decoding, this encoded value alone is useless. Decode the value again in runtime, using some kind of shared keys infra / asymmetric key pair, in this case, you can use a public key for encoding, a private key for decoding
I want to use Git to store my repository, but I don't want to publish the key.
For something as critical as a secret key, I would use a dedicated keyring infrastructure located outside the development environment, optionally coupled to a secret passphrase.
Aside this case, I personnaly use submodules for this. Check out :
git submodule
In particular, I declare a global Git repository, in which I declare in turn another Git repository that will contain the actual project that will go public. This enables us to store at top level everything that is related to the given project, but not forcibly relevant to it and that is not to be published. This could be, for instance, all my drafts, automation scripts, worknotes, project specifications, tests, bug reports, etc.
Among all the advantages this facility provides, we can highlight the fact that you can declare as a submodule an already existing repository, this repository being located inside or outside the parent one.
And what's really interesting with this is that both main repository and submodules remains distinct Git repositories, that still can be configured independently. This means that you don't need your parent repository to have its remote servers configured.
Doing that way, you get all the benefits of a versioning system wherever you work, while still ensuring yourself that you'll never accidentally push outside something that is not stored inside the public submodule.
If you don't have too many secrets to manage, and you do want to keep the secrets in version control, I make the parent repository private. It contains 2 folders - a secrets folder, and a gitsubmodule for the public repository (in another folder). I use ansible crypt to encrypt anything in the secrets folder, and a bash script to pass the decrypted contents, and load those secrets as environment vars to ensure the secrets file always stays encrypted.
Ansible crypt can encrypt and decrypt an environment variable, which I wrap in a bash script to do these functions like so-
testsecret=$(echo 'this is a test secret' | ./scripts/ansible-encrypt.sh --vault-id $vault_key --encrypt)
result=$(./scripts/ansible-encrypt.sh --vault-id $vault_key --decrypt $testsecret)
echo $result
testsecret here is the encrypted base64 result, and it can be stored safely in a text file. Later you could source that file to keep the encrypted result in memory, and finally when you need to use the secret, you can decrypt it ./scripts/ansible-encrypt.sh --vault-id $vault_key --decrypt $testsecret
This bash script referenced above is below (ansible-encrypt.sh). It wraps ansible crypt functions in a way that can store encrypted variables in base64 which resolves some problems that can occur with encoding.
#!/bin/bash
# This scripts encrypts an input hidden from the shell and base 64 encodes it so it can be stored as an environment variable
# Optionally can also decrypt an environment variable
vault_id_func () {
if [[ "$verbose" == true ]]; then
echo "Parsing vault_id_func option: '--${opt}', value: '${val}'" >&2;
fi
vault_key="${val}"
}
secret_name=secret
secret_name_func () {
if [[ "$verbose" == true ]]; then
echo "Parsing secret_name option: '--${opt}', value: '${val}'" >&2;
fi
secret_name="${val}"
}
decrypt=false
decrypt_func () {
if [[ "$verbose" == true ]]; then
echo "Parsing secret_name option: '--${opt}', value: '${val}'" >&2;
fi
decrypt=true
encrypted_secret="${val}"
}
IFS='
'
optspec=":hv-:t:"
encrypt=false
parse_opts () {
local OPTIND
OPTIND=0
while getopts "$optspec" optchar; do
case "${optchar}" in
-)
case "${OPTARG}" in
vault-id)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
opt="${OPTARG}"
vault_id_func
;;
vault-id=*)
val=${OPTARG#*=}
opt=${OPTARG%=$val}
vault_id_func
;;
secret-name)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
opt="${OPTARG}"
secret_name_func
;;
secret-name=*)
val=${OPTARG#*=}
opt=${OPTARG%=$val}
secret_name_func
;;
decrypt)
val="${!OPTIND}"; OPTIND=$(( $OPTIND + 1 ))
opt="${OPTARG}"
decrypt_func
;;
decrypt=*)
val=${OPTARG#*=}
opt=${OPTARG%=$val}
decrypt_func
;;
encrypt)
encrypt=true
;;
*)
if [ "$OPTERR" = 1 ] && [ "${optspec:0:1}" != ":" ]; then
echo "Unknown option --${OPTARG}" >&2
fi
;;
esac;;
h)
help
;;
*)
if [ "$OPTERR" != 1 ] || [ "${optspec:0:1}" = ":" ]; then
echo "Non-option argument: '-${OPTARG}'" >&2
fi
;;
esac
done
}
parse_opts "$#"
if [[ "$encrypt" = true ]]; then
read -s -p "Enter the string to encrypt: `echo $'\n> '`";
secret=$(echo -n "$REPLY" | ansible-vault encrypt_string --vault-id $vault_key --stdin-name $secret_name | base64 -w 0)
unset REPLY
echo $secret
elif [[ "$decrypt" = true ]]; then
result=$(echo $encrypted_secret | base64 -d | /snap/bin/yq r - "$secret_name" | ansible-vault decrypt --vault-id $vault_key)
echo $result
else
# if no arg is passed to encrypt or decrypt, then we a ssume the function will decrypt the firehawksecret env var
encrypted_secret="${firehawksecret}"
result=$(echo $encrypted_secret | base64 -d | /snap/bin/yq r - "$secret_name" | ansible-vault decrypt --vault-id $vault_key)
echo $result
fi
Storing encrypted values as environment variables is much more secure than decrypting something at rest and leaving the plaintext result in memory. That's extremely easy for any process to siphon off.
If you only wish to share the code with others and not the secrets, you can use a git template for the parent private repo structure so that others can inherit that structure, but use their own secrets. This also allows CI to pickup everything you would need for your tests.
Alternatively, If you don't want your secrets in version control, you can simply use git ignore on a containing folder that will house your secrets.
Personally, this makes me nervous, its possible for user error to still result in publicly committed secrets, since those files are still under the root of a public repo, any number of things could go wrong that could be embarrassing with that approach.
For Django
add another file called secrets.py or whatever you want and also another called .gitignore
type secrets.py in .gitignore
paste the secret key into the secrets.py file and to import it in the settings file by using
from foldername.secrets import *
this worked for me.
There is one file I want to encrypt with GnuPG by
gpg2 --homedir=~/.gnupg --always-trust=true --recipient="BlahBlah" --encrypt=/path/to/file --output=/path/to/output_file
However this command seems to hang forever and never return. Interestingly, after I interrupt process, there is indeed /path/to/output_file created , however the bytes written there is much bigger than raw payload (for example my /path/to/file is only of 5 bytes but it turns out there are nearly 200 bytes written to /path/to/output_file).
There must be something wrong, but I really couldn't figure out what is it.
I have in advance imported the key for BlahBlah by gpg --import key.asc. It happens both for GnuPG 1 and GnuPG 2.
You're applying --encrypt in a wrong way. --encrypt does not expect any parameters, the file(s) to be worked on are passed as very last arguments. Additionally, following the documentation you should pass --output /path/to/output_file instead of --output=/path/to/output_file. Finally, GnuPG distinguishes between options and commands, and options should precede commands.
What you observe is that GnuPG starts writing header information, but then waits for input from STDIN (until interrupted).
The GnuPG command line you're looking for is
gpg2 --homedir=~/.gnupg --always-trust=true --recipient="BlahBlah" --output /path/to/output_file --encrypt /path/to/file
One last hint: the combination of --always-trust=true and resolving a recipient by user ID is a very bad idea, as any other key with the same user ID in the local keyring might be used. Pass the full key's fingerprint instead, which specifically selects a distinct key (using short key IDs is not secure, either).
I have a job that runs periodically and signs/encrypts a file like so:
$ gpg --homedir /path/to/.gnupg -r key1#mydomain.com -r key2#mydomain.com --local-user sig1#mydomain.com --batch --passphrase-file /path/to/gpg-password --sign -ea myfile
The command encrypts the file for two recipients: key1#mydomain.com and key2#mydomain.com. It signs the file with sig1#mydomain.com. It runs in batch mode as there is no human interactivity - this is an automated process. It gets the passphrase for the signature from /path/to/gpg-password.
What I would like to do is now sign the file with two signatures at the same time. Like so:
$ gpg --homedir /path/to/.gnupg -r key1#mydomain.com -r key2#mydomain.com --local-user sig1#mydomain.com --local-user sig2#mydomain.com --batch --passphrase-file /path/to/gpg-password --sign -ea myfile
gpg: skipped "sig2#mydomain.com": bad passphrase
It works fine in interactive mode (eg. without --batch), I just have to supply the two passphrases via the command line. However, in batch mode it fails as it tries to get the signatures from the file. The signature is only valid for one of the signing keys.
From the man page:
--passphrase-file file
Read the passphrase from file file. Only the first line will be read from file file. This can only be used if only one passphrase is supplied.
How do I tell it what the password is for each key?
You have different options.
Completely remove the passwords, since they're stored somewhere anyway.
Use the same password (as you already discovered).
Use the gpg-agent and preset the passphrase. I'm unsure whether this is GnuPG 2-only (usually installed as gpg2, maybe to be installed from a gnupg2 package). Presetting the passphrase is as easy as running gpg-preset-passphrase --preset [fingerprint]. You will have to run this command for each of the keys individually, and make sure to cache the passphrase for a given time (at least the processing time of adding all the passphrases, and then signing the file you want to sign).
For the sake of completeness, but impractical: sign the file individually for each key, then take apart the OpenPGP packets and recombine them adding all the signatures one after the other. Signing with multiple keys just creates multiple signature packets.
For anyone else in the same situation as me, I ended up working around this apparent deficiency of gpg by editing one of the signing keys to have the same password as the other (the password stored in the gpg-password file). This doesn't compromise security in this instance since the password is stored in a text file anyway - the real security is the password of the user that this commands runs from and the fact that the secret keys are kept secret). You can change the password on a key by doing gpg --edit-key <key_id>, then passwd. Don't forget to save after.
I encryptd a file using gpg, now I want to decrypt the file.
Is there any way to decrypt the file without need to import the secret file?
We have the secret key in a file called key.sec; can we pass the secret file to gpg as a parameter (when we run the decrypt command from the bash command line) to use when decrypting the encrypted file? Or must we import the secret key then decrypt the encrypted files?
You must add the secret key to a keyring. From the gpg(1) documentation:
--no-default-keyring
Do not add the default keyrings to the list of
keyrings. Note that GnuPG will not operate without any
keyrings, so if you use this option and do not provide
alternate keyrings via --keyring or --secret-keyring,
then GnuPG will still use the default public or secret
keyrings.
You could --import --no-default-keyring --secret-keyring temporary to import the key, use --secret-keyring temporary when decrypting the content, then delete the ~/.gnupg/temporary.gpg file when you're done. But that's just a work-around.
You have to import the secret key to use it but the way that secret keys are managed by GnuPG version 2.x has changed. There is a gpg-agent daemon that handles secret keys access and its use is mandatory from version 2.1.
Here is a way that you can quickly create a temporary keyring to decrypt with a secret key that is contained in a file:
$ mkdir -m 700 ~/.gnupg-temp
$ gpg --homedir .gnupg-temp --import key.sec
$ gpg --homedir .gnupg-temp -d an_ecrypted_file
If you want to clean up afterwards, stop the agent and remove the directory:
$ gpg-connect-agent --homedir .gnupg-temp KILLAGENT /bye
$ rm -r ~/.gnupg-temp
There used to be an option --secret-keyring about which the documentation for version 2.1 has this to say:
This is an obsolete option and ignored. All secret keys are stored in the private-keys-v1.d directory below the GnuPG home directory.
The private-keys-v1.d directory (wthin the --homedir or ~/.gnupg) is owned and operated by the agent.
The objective of the OP Mohammed appears to be keeping his PUBLIC and SECRET key apart. After all, do we want to keep the Secret key with the data it was used to encrypt? Thus, Mohammed's and 10,650+ others (at the time I write this) are interested in if/how it's possible. Indeed it is, and this is how you do it:
The publicly-facing host only has two keys: Both are Public Keys
Your GPG Public key used to encrypt data
Your SSH Public key in .ssh/authorized_keys to facilitate non-interactive logins.
Round-tripping an encrypted file using Public-Secret key separation:
The following bash snippet when executed on the host with the Secret Key will fetch the crypted file from the DMZ host via scp, and squirt the gpg decrypted standard output back onto the DMZ host into a file so it can be read/operated upon. This code is tested and known to work correctly:
echo "$(gpg -d $(scp myuser#192.168.1.10:/home/myuser/test-gpg.txt.asc .;ls ./test-gpg.txt.asc))" | ssh myuser#192.168.1.10 'cat > /home/myuser/test-gpg.txt'
Note that you will still be prompted for a password once decryption begins. But once the password is supplied, the script continues and injects the decrypted gpg stream into a file on DMZ host.
And don't forget to do an rm test-gpg.txt of the decrypted file once the operation that required it's contents to be readable has been completed.
So yes, very possible to keep your secret key apart from the publicly accessible host where encryption occurs and your secret key tucked safely away in a host outside of that DMZ. HTH- Terrence Houlahan
I set a passphrase when creating a new SSH key on my laptop. But, as I realise now, this is quite painful when you are trying to commit (Git and SVN) to a remote location over SSH many times in an hour.
One way I can think of is, delete my SSH keys and create new. Is there a way to remove the passphrase, while still keeping the same keys?
Short answer:
$ ssh-keygen -p
This will then prompt you to enter the keyfile location, the old passphrase, and the new passphrase (which can be left blank to have no passphrase).
If you would like to do it all on one line without prompts do:
$ ssh-keygen -p [-P old_passphrase] [-N new_passphrase] [-f keyfile]
Important: Beware that when executing commands they will typically be logged in your ~/.bash_history file (or similar) in plain text including all arguments provided (i.e. the passphrases in this case). It is, therefore, is recommended that you use the first option unless you have a specific reason to do otherwise.
Notice though that you can still use -f keyfile without having to specify -P nor -N, and that the keyfile defaults to ~/.ssh/id_rsa, so in many cases, it's not even needed.
You might want to consider using ssh-agent, which can cache the passphrase for a time. The latest versions of gpg-agent also support the protocol that is used by ssh-agent.
$ ssh-keygen -p worked for me
Opened git bash. Pasted : $ ssh-keygen -p
Hit enter for default location.
Enter old passphrase
Enter new passphrase - BLANK
Confirm new passphrase - BLANK
BOOM the pain of entering passphrase for git push was gone.
Thanks!
You might want to add the following to your .bash_profile (or equivalent), which starts ssh-agent on login.
if [ -f ~/.agent.env ] ; then
. ~/.agent.env > /dev/null
if ! kill -0 $SSH_AGENT_PID > /dev/null 2>&1; then
echo "Stale agent file found. Spawning new agent… "
eval `ssh-agent | tee ~/.agent.env`
ssh-add
fi
else
echo "Starting ssh-agent"
eval `ssh-agent | tee ~/.agent.env`
ssh-add
fi
On some Linux distros (Ubuntu, Debian) you can use:
ssh-copy-id -i ~/.ssh/id_dsa.pub username#host
This will copy the generated id to a remote machine and add it to the remote keychain.
You can read more here and here.
To change or remove the passphrase, I often find it simplest to pass in only the p and f flags, then let the system prompt me to supply the passphrases:
ssh-keygen -p -f <name-of-private-key>
For instance:
ssh-keygen -p -f id_rsa
Enter an empty password if you want to remove the passphrase.
A sample run to remove or change a password looks something like this:
ssh-keygen -p -f id_rsa
Enter old passphrase:
Key has comment 'bcuser#pl1909'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.
When adding a passphrase to a key that has no passphrase, the run looks something like this:
ssh-keygen -p -f id_rsa
Key has comment 'charlie#elf-path'
Enter new passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved with the new passphrase.
On the Mac you can store the passphrase for your private ssh key in your Keychain, which makes the use of it transparent. If you're logged in, it is available, when you are logged out your root user cannot use it. Removing the passphrase is a bad idea because anyone with the file can use it.
ssh-keygen -K
Add this to ~/.ssh/config
UseKeychain yes
On windows, you can use PuttyGen to load the private key file, remove the passphrase and then overwrite the existing private key file.
In windows for me it kept saying
"id_ed25135: No such file or directory" upon entering above commands. So I went to the folder, copied the path within folder explorer and added "\id_ed25135" at the end.
This is what I ended up typing and worked:
ssh-keygen -p -f C:\Users\john\.ssh\id_ed25135
This worked. Because for some reason, in Cmder the default path was something like this C:\Users\capit/.ssh/id_ed25135 (some were backslashes: "\" and some were forward slashes: "/")
If you have set a passphrase before and is using mac, use the keychain instead, you'll need to enter your passpharase for the last time and that's it
ssh-add --apple-use-keychain ~/.ssh/id_rsa
Enter passphrase for /Users/{{user_name}}/.ssh/id_rsa:
Identity added: /Users/{{user_name}}/.ssh/id_rsa(/Users/{{user_name}}/.ssh/id_rsa)
If you are using Mac
Go to .ssh folder
update config file by adding "UseKeychain yes"