I am working on an embedded Linux project that can run on multiple
platforms. One uses e.MMC for storage and another NAND flash. I want
to encrypt all the filesystems (mainly to protect against someone
unsoldering the flash chips and putting them in a reader). I want to
try and maintain a common approach across both hardware types as far
as possible. One big difference between the two is the wear levelling
is in the hardware for the e,MMC whereas for NAND I'll be using UBI.
For the root filesystem I am thinking of using squashfs which is
protected using dm-crypt. For the NAND device I have tried this out
and I can layer dm-crypt on top of ubiblock then use the device mapper
to load the squashfs. This maps nicely to the e.MMC world with the
only difference being that the device mapper is on a gpt partition
rather than a ubiblock device.
My challenge is for other read/ write filesystems. I want to mount an
overlay filesystem on top of the read-only root and a data partition.
I want both of these to also be encrypted. I have been investigating
how fscrypt can help me. (I believe dm-crypt won't work with ubifs).
For filesystems on the e.MMC I will be using ext4 and for NAND
ubifs. The documentation says both of these support fscrypt. I've struggled
a bit to find detailed documentation about how to use this with ubifs
(there is a lot more for the ext4) but I think that there are some
differences between how this has been implemented on each and I'd like
those who know more to confirm this.
On the NAND side I have only been able to get it to work by using the
fscryptctl tool (https://github.com/google/fscryptctl
) as opposed to the fuller featured fscrypt tool
(https://github.com/google/fscrypt). This was following instructions I
found in a patch to add fscrypt support to mkfs.ubifs here:
https://patchwork.ozlabs.org/project/linux-mtd/cover/20181018143718.26298-1-richard#nod.at/
This appears to encrypt all the files on the partition using the
supplied key. When I look at fscrypt on ext4 it seems here that you
can't do this. The root directory cannot itself be encrypted, only
sub-directories. Reading here:
https://www.kernel.org/doc/html/v4.17/filesystems/fscrypt.html it
says:
"Note that the ext4 filesystem does not allow the root directory to be
encrypted, even if it is empty. Users who want to encrypt an entire
filesystem with one key should consider using dm-crypt instead."
So this is different. It also seems that with ubifs I can't apply
encryption to the subdirectories like I could in ext4. The README.md
here https://github.com/google/fscryptctl gives an example using ext4.
This encrypts a subdirectory called test. I don't see how to do the
same thing using ubifs. Could someone help me?
I've been using the NANDSIM kernel module for testing. At the end of
this post is a script for building an encrypted overlay ubifs
filesystem. As you can see the mkfs.ubifs is taking the key directly
and it appears to apply it to all the files on the partition. You
can't then apply policies to any sub-directories as they are already
encrypted.
I would like to use some of the other features that the userspace
fscrypt tool provides e.g. protectors (so I don't need to use the
master key directly). I can't however see any way to get the userspace fscrypt
tool to setup encryption on a ubifs. The userspace fscrypt command
creates a .fscrypt directory in the root of the
partition to store information about policies and protectors. This
seems to fit more with the ext4 implementation where the root itself isn't encrypted.
When I try to set-up an unencrypted ubifs with "fscrypt setup" I run
into trouble as making a standard ubifs seems to create a v4 ubifs format
version rather than the required v5. This means the "fscrypt encrypt"
command fails. (Errors like this are seen in the dmesg output
[12022.576268] UBIFS error (ubi0:7 pid 6006): ubifs_enable_encryption
[ubifs]: on-flash format version 5 is needed for encryption).
Is there some way to get mkfs.ubifs to create an unencrypted v5 formatted
filesystem? Or does v5 mean encrypted?
Here is my script to create an encrypted ubifs using the fscryptctl tool:
#!/bin/bash
MTD_UTILS_ROOT=../../mtd-utils
FSCRYPTCTL=../../fscryptctl/fscryptctl
MOUNTPOINT=./mnt
dd if=/dev/urandom of=overlay.keyfile count=64 bs=1 # XTS needs a 512bit key
descriptor=`$FSCRYPTCTL get_descriptor < overlay.keyfile`
$MTD_UTILS_ROOT/mkfs.ubifs --cipher AES-256-XTS --key overlay.keyfile
-m 2048 -e 129024 -c 32 -r ./overlay -o overlay.enc.img
$MTD_UTILS_ROOT/ubiupdatevol /dev/ubi0_6 overlay.enc.img
# Try it out
$FSCRYPTCTL insert_key < overlay.keyfile
key=`keyctl show | grep $descriptor | awk '{print $1}'`
mount -t ubifs /dev/ubi0_6 $MOUNTPOINT
ls $MOUNTPOINT
umount $MOUNTPOINT
keyctl unlink $key
NB I've been working with mtd-utils v2.1.2 on a 5.4 kernel.
Related
So I created encrypted key using ansible-vault create my.key.
Then I use it as var:
my_key: "{{ lookup('file','{{ inventory_dir }}/group_vars/my.key') }}"
And then when running my playbook, like this:
- name: Create My Private Key
ansible.builtin.copy:
content: "{{ secrets.my_key }}"
dest: "{{ secrets_key }}"
no_log: true
It does properly create key on remote host and it is then unencrypted. But I'm thinking if this is the right way to do it? Does it unencrypt at the right time and I am not exposing sensitive data where it should not be?
I thought encrypted variables must also have !vault keyword specified. But if I do this for my my_key, I get this error:
fatal: [v14-test]: FAILED! => {"msg": "input is not vault encrypted data. "}
So this got me worried, that file is unencrypted at the wrong time or maybe message is misleading or something.
Is this the right way to do it? Or I should do it differently?
Firstly, a definitive answer as to whether this approach is appropriate, is directly linked to what you want to achieve from encryption. Therefore all the answers here can do is talk about how Vault works and then you can decide if it is right for your requirements.
Fundamentally what you are doing is a 'correct' usage of Ansible Vault, although I have not previously seen it used in quite this workflow (typically I have seen create used for encrypting YAML files of vars).
Using your method, your secret is turned into ciphertext and stored in my.key (which can be confirmed by using basic text tools such as cat, less or more). You will see the first line of the file, contains a bunch of metadata that allows Ansible to understand the file contents and decrypt on demand.
At runtime, Ansible will then use the password/key for the encrypted file (accessed through a number of methods) to decrypt the file contents into plain text and then store it in the variable my_key for use during the play.
A non-exhaustive list of things to consider when determining if Ansible Vault is the right approach for you:
Ansible Vault encryption is purely designed to protect secrets at rest (i.e. when they are stored on your hard disk)
At run time, the secrets are converted into plain text and treated like any other variable/string data, however the file on disk still contains ciphertext so the plaintext is only accessible within the running Ansible process (i.e. on a multi-user system, at no point can anybody view the plaintext simply by looking inside the my.key file. However, depending on their level of access, skills and what your Ansible tasks are doing, they may be able to access the plaintext from the running process.)
Given inside the process the data is just plain text, it is vulnerable to leakage (for example by writing the contents out into a log file - checkout the Ansible no_log option)
At run time, Ansible needs some way to access the key necessary to decrypt the ciphertext. It provides a variety of methods, including prompting the user, accessing it from a file stored on disk, accessing it from an Env var, using scripts/integrations to pull it from another secrets mgmt tool. Careful thought needs to be given about which option is chosen, relative to what you are looking to achieve from the encryption (e.g. if your goal is to protect your data in the event that your laptop gets stolen, then storing the key in a file on the same system, renders the whole operation pointless). Quite often, with more sophisticated methods, you can still end up in a 'chicken and egg' situation, once more relative to what your goal from using encryption is
I might be talking complete cobblers or be a nefarious individual trying to sow disinformation, so read the docs thoroughly if the value of the secrets if significant to you :)
Unfortunately there is no getting away from generally good security is harder to achieve than the illusion of good security :|
The POSIX shm_open() function returns a file descriptor that can be used to access shared memory. This is extremely convenient because one can use all the traditional mechanisms for controlling file descriptors to also control shared memory.
The only drawback is that shm_open() always wants a filename. So I need to do this:
// Open with a clever temp file name and hope for the best.
fd = shm_open(tempfilename, O_RDWR | O_CREAT | O_EXCL, 0600);
// Immediately delete the temp file to keep the shm namespace clean.
shm_unlink(tempfilename);
// Then keep using fd -- the shm object remains as long as there are open fds.
This use of tempfilename is difficult to do portably and reliably. The interpretation of the filename (what the namespace is, how permissions are handled) differs among systems.
In many situations the processes using the shared memory object have no need for a filename because the object can be accessed more simply and safely by just passing a file descriptor from one process to another. So is there something that's just like shm_open() but can be used without touching the shared memory filename namespace?
mmap() with MAP_ANON|MAP_SHARED is great but instead of a file descriptor it gives a pointer. The pointer doesn't survive over an exec boundary and can't be sent to another process over a Unix domain socket like file descriptors can.
The file descriptor returned by shm_open() also doesn't survive an exec boundary by default: the POSIX definition says that the FD_CLOEXEC file descriptor flag associated with the new file descriptor is set. But it is possible to clear the flag using fcntl() on MacOS, Linux, FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and possibly other operating systems.
A library to solve the problem
I managed to write a library that provides the simple interface:
int shm_open_anon(void);
The library compiles without warnings and successfully runs a test program on Linux, Solaris, MacOS, FreeBSD, OpenBSD, NetBSD, DragonFlyBSD and Haiku. You may be able to adapt it to other operating systems; please send a pull request if you do.
The library returns a file descriptor with the close-on-exec flag set. You can clear that flag using fcntl() on all supported operating systems, which will allow you to pass the fd over exec(). The test program demonstrates that this works.
Implementation techniques used in the library
The readme of the library has very precise notes on what was done and what wasn't done for each OS. Here's a summary of the main stuff.
There are several non-portable things that are more or less equivalent to shm_open() without a filename:
FreeBSD can take SHM_ANON as the pathname for shm_open() since 2008.
Linux has a memfd_create() system call since kernel version 3.17.
Earlier versions of Linux can use mkostemp(name, O_CLOEXEC | O_TMPFILE) where name is something like /dev/shm/XXXXXX. Note that we are not using shm_open() at all here -- mkostemp() is implicitly using a perfectly ordinary open() call. Linux mounts a special memory-backed file system in /dev/shm but some distros use /run/shm instead so there are pitfalls here. And you still have to shm_unlink() the temp file.
OpenBSD has a shm_mkstemp() call since release 5.4. You still have to shm_unlink() the temp file but at least it is easy to create safely.
For other OSes, I did the following:
Figure out an OS-dependent format for the name argument of POSIX shm_open(). Please note that there is no name you can pass that is absolutely portable. For example, NetBSD and DragonFlyBSD have conflicting demands about slashes in the name. This applies even if your goal is to use a named shm object (for which the POSIX API was designed) instead of an anonymous one (as we are doing here).
Append some random letters and numbers to the name (by reading from /dev/random). This is basically what mktemp() does, except we don't check whether our random name exists in the file system. The interpretation of the name argument varies wildly so there's no reasonable way to portably map it to an actual filename. Also Solaris doesn't always provide mktemp(). For all practical purposes, the randomness we put in will ensure a unique name for the fraction of a second that we need it.
Open the shm object with that name via shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600). In the astronomical chance that our random filename already exists, O_EXCL will cause this call to fail anyway, so no harm done. The 0600 permissions (owner read-write) are necessary on some systems instead of blank 0 permissions.
Immediately call shm_unlink() to get rid of the random name. The file descriptor remains for our use.
This technique is not quaranteed to work by POSIX, but:
The shm_open() name argument is underspecified by POSIX so nothing else is guaranteed to work either.
I'll let the above compatibility list speak for itself.
Enjoy.
No, there isn't. Since both System V shared memory model and POSIX shared file mapping for IPC require operations with a file, there is always need for a file in order to do mapping.
mmap() with MAP_ANON|MAP_SHARED is great but instead of a file
descriptor it gives a pointer. The pointer doesn't survive over an
exec boundary and can't be sent to another process over a Unix domain
socket like file descriptors can.
As John Bollinger says,
Neither memory mappings created via mmap() nor POSIX shared-memory
segments obtained via shm_open() nor System V shared-memory segments
obtained via shmat() are preserved across an exec.
There must be a well-known place on the memory to meet and exchange information. That's why a file is the requirement. By doing this, after exec, the child is able to reconnect to the appropriate shared memory.
This use of tempfilename is difficult to do portably and reliably. The interpretation of the filename (what the namespace is, how permissions are handled) differs among systems.
You can have mkstemp create a unique filename in /dev/shm/ or /tmp and open the file for you. You can then unlink the filename, so that no other process can open this file, apart from the process that have the file descriptor returned from mkstemp.
mkstemp(): CONFORMING TO 4.3BSD, POSIX.1-2001.
Why not creating it with access rights to 0?
Thus no process would be able to "open" it and let you unlink it safely just after?
I have searched lots of tutorials on web & Youtube, but no luck.
I want to configure Cisco switch via Ansible, I already have it setup, works flawlessly.. but I want to store the passwords (for vty lines, console, enable secret...) ideally in hosts file encrypted via Ansible-Vault as variables so in my .yml file I can access them. I want them in hosts file, because we have different passwords for ASW, DSW and CSW so it could be easier to manage.
I generated encrypted variable in CLI:
ansible-vault encrypt_string enable_password --ask-vault-pass
I copy the value to the variable in /etc/ansible/hosts:
...
[2960-X:vars]
ansible_become=yes
ansible_become_method=enable
ansible_network_os=ios
ansible_user=admin
enable_password= !vault |
$ANSIBLE_VAULT;1.1;AES256
.....
In config.yml:
- name: Set enable password
ios_config:
lines:
- enable secret "{{ enable_password }}"
Right now, the password is going to be set as " !vault |"
I am not sure if this is even best practise, I read recommendations for this but all I could find was about server automation, not networks.
I'm running Ansible 2.8.0
Any help is appreciated, thank you.
Let me quote from Variables and Vaults
When running a playbook, Ansible finds the variables in the unencrypted file and all sensitive variables come from the encrypted file.
A best practice approach for this is to start with a group_vars/ subdirectory named after the group. Inside of this subdirectory, create two files named vars and vault. Inside of the vars file, define all of the variables needed, including any sensitive ones. Next, copy all of the sensitive variables over to the vault file and prefix these variables with vault_. You should adjust the variables in the vars file to point to the matching vault_ variables using jinja2 syntax, and ensure that the vault file is vault encrypted.
This scheme isn't limited to group_vars/ only and can be applied to any place where the variables come from.
We have increased the -n parameter in broker/db.pf file.We restarted the server and when we check in promon its still showing the same number of users. How do we increase the -n parameter?
I know you answered this yourselves but for future users a real answer can be good. There are several ways to set parameters like -n. This answer really applies to changing all startup parameters (but not what values are "good").
How you change this value depends on how you start your database. See below.
NB 1: you should be aware of your licensing plan before changing this number and contact your sales contact if needed.
NB 2: you should be aware that changing startup parameters can affect performance etc. Test new values in a separate environment before moving them to production.
NB 3: backup all files before messing around...
Managed Database
A managed database is a database that is handled by the AdminServer. OE Management is not needed for this approach. A working installation of OE Explorer is however recommended.
The managed database is started (and stopped etc) via either the web based OE Explorer interface or the dbman command line utility.
Settings are stored in conmgr.properties under your Progress installation. You can edit this file manually (save a copy first...) or via the OE Explorer (recommended way).
You will have a line like this in the file:
maxusers=20 # -n
Edit the number to your liking with your favourite editor.
You can also change this in the OE Explorer:
Log in to OE Explorer. Default location is http://servername:9090/.
Locate and click on the database (if it's not there it's not handled by the adminserver - see below).
Select Configuration
Select Configuration (again, not "servergroup")
Click EDIT
-n (or Max users) is located in the first group of settings ("General"). See picture below.
Edit the value and don't forget to save.
Scripted Database
A scripted database is a database that started with a custom script (or also directly from command line). The actual startup could be handled by crontab, a user, the server generic startup script etc.
The OE AdminServer is not "aware" of this database. (You can make the AdminServer "a little" aware of it by running the dbagent command line utility with certain parameters. Read more about this in the manual).
You could generally divide into two ways of handling the script: with parameters in it or with parameters in a separate parameter file (often with the extension .pf).
Script with parameters in it
With this approach you store all parameters in the actual startup script.
proserve <dbname> -H <hostname> -S <serviceport> -n 10 -B 10000 -spin 10000 etc..
Script with a separate parameter file
With this approach you store the parameters in a separate file.
proserve <dbname> -pf /path/to/file/file.pf
The .pf-file can be formatted like the parameters in the command line:
-db <dbname> -H <hostname> -S <service> etc.
Or with newlines (this allows for comments in the file):
# Main database
-db <dbname>
-H <hostname>
-S <service>
You can also mix these two approaches.
Sources:
OE Management and OE Explorer
OE Database Management
I need to encrypt data while we take mysqldump from database through command prompt. My OS is windows7. Please help me.
Can't you just pipe the dump output directly though your encryption tool?
ie:
mysqldump mydb | some-encryption-tool.sh
btw, the only reason I suggested piping directly through an encryption tool is to the (unsafe) plain-text version never exists on disk, which is the only interpretation of the question that makes sense. Otherwise, just save the dump to a file and encrypt it - there is nothing to "answer".