I'm trying to incorporate end to end encryption using Apache Pulsar. So far the examples I've seen look similar to http://pulsar.apache.org/docs/en/security-encryption/
I saw that the encryption key (public / private key) is fetched every 4 hours in the key rotation section. It seems to indicate that you can only have a single key to encrypt the messages you send.
I’d like to be able to use different encryption keys for different topics/tenants. Does pulsar support different encryption keys for different topics?
Basically my client will pull a different key depending on which tenant the data belongs to and encrypt any messages pertaining to that specific key using pulsar's end to end encryption.
How would this look?
Is it as simple as creating a new producer any time I want to use a different key for encryption?
Producer producer = pulsarClient.newProducer()
.topic("persistent://my-tenant/my-ns/my-topic")
.addEncryptionKey("myTenant1Key")
.cryptoKeyReader(new RawFileKeyReader("tenant1_pubkey.pem", "tenant1_privkey.pem"))
.create();
So far I've used the pulsar client to send encrypted messages using a local private/public key pair. Can I just switch out the key pair and instantiate a new producer anytime I want a message to be encrypted differently? I've tried asking around and can't find the right answer.
Absolutely, you are creating separate producers for different topic and can use different keys for each producer.
Related
I've been doing some searching and still do not know if this is possible. What I want is for a message to by encrypted by our system and decrypted by a "master key" and also a 3rd party.
This encrypted message needs to be decrypted by 2 entities
-A 3rd party (which we want the control to shut off their ability to decrypt)
-Our system (which we want to always be able to decrypt no matter what, master key?)
From some research I was doing there is a concept of master key and derived keys
Does this following system exist?:
Master Key - can decrypt anything encrypted by derived keys
DerivedKey1 -> Encrypt data with this key and be able to decrypt with Master Key OR DerivedKey1
DerivedKey2 -> Encrypt data with this key and be able to decrypt with Master Key OR DerivedKey2 but NOT derivedKey1
Any terminology I should be using to search for answers would be helpful, also any crypto systems that do this already would be great to know.
Yes, I mean deny any new messages sent from our system to be decrypted
You can encrypt the content with a random key (data key).
Then you can encrypt the data key for each intended recipient (master key and any 3rd party) using its shared or public key.
Hi Crypto experts out there, are there any best practices around distributing an ecnrypted package to multiple end user systems, specially if the end system are offline ones? in context of assymetric crypto.
is it must to create unique pub/pvt key sets [ per end user system] and encrypt the same package many times uniquely with the pub keys, resulting in a specific package per end user system? how will this scale?
will it be a good practice to sign the original private key[ corresponding to pub keys used to encrypt the package] with senders private keys and then enrypt using end user systems pub keys and share it directly with end user? through trusted communication.
or, encrypt the pvt key with end user systems public key, sign with senders private key and re-encrypt[symmetric] this with the hash of certain string uniquely identifying a end user system? This hash should be programtically reproducible using system unique identifiers later during decryption processes. This way, to retreive the original private key to decrypt the package, it will require both a corresponding pub key[end user clients] as well as end user machine [the hash of string to be generated at runtime on end user system.] and senders public key to manage the authenticity?
Thank you for any feedback!
I am not an expert, but as I understand asymmetric encryption, you can generate a key pair in the distribution center.
The private key stays secret in the distribution center.
To each offline client you provide the public key (as a file).
Each client generates a secure password for symmetric encryption, and encrypts it using the public key.
The encrypted symmetric key is sent to the distribution center.
The distribution center should associate the encrypted symmetric password to the client that sent it.
At the time of encrypting the package for the specific client, the distribution center will decrypt the symmetric password using the private key, and use it to encrypt the package.
Then the package can be sent to the client, who will use it's own password to decrypt the package.
I am fairly new to PGP encryption. I generated a new key pair last week on a Yubikey, while previously I had only used a private key that I generated on my machine. I use thunderbird enigmail for key management. This is probably a stupid question but I don't actually understand how to encrypt an email using the private key stored on my Yubikey. I can decrypt just fine but can't figure out how to compose an encrypted email from the new key. Using my old key pair, it's automatic - how do I actually encrypt using the right private key? At what point do I enter the Yubikey and pin? (I'm not all that technical and I use encryption for one specific project, so I appreciate dumbing down something that is probably very obvious.)
Using keys generated on a Yubikey is a practically identical process to using your old keys, with one extra step of set up.
To use keys stored on a Yubikey, you need "key stubs" registered on your PGP system. Basically, these are references that tell your PGP software "This private key is located on smartcard number #123456." Once you have the stubs to the keys you need, Enigmail should treat the Yubikey-stored keys like any other PGP key.
I'm only familiar with GnuPG, so specifics may vary, but using GPG you need to
Have a copy of the public key imported to your system from a key server or the like.
Import key stubs from your smart card (in this case, your Yubikey) using:
gpg --card-status
After that, checking your private keyring should reflect that you have stubs pointing to private keys stored on a smart card (denoted by the > after sub). I.E.
gpg --list-secret-keys
----------------------------------------------------
sec# rsa2048 2018-01-02 [C] [expires: 2020-01-02]
ABCDEF123456789ABCDEF123456789ABCDEF1234
uid [ ultimate] My Name (My PGP key) <anAddress#somesite.com>
ssb> rsa2048 2018-01-02 [S] [expires: 2018-12-31]
ssb> rsa2048 2018-01-02 [E] [expires: 2018-12-31]
ssb> rsa2048 2018-01-02 [A] [expires: 2018-12-31]
At this point you can use sub keys of CDEF1234 as if there were on your system. Your PGP software ought to handle checking the card is inserted and dealing with PIN entry.
Is it possible to "retarget" keys generated via the ncipher JCE API to pkcs11? I know that you can retarget via the generatekey command but I don't see how to do it to an existing JCE key. The first prompt is for the "source application" and the options don't seem to include JCE. Does it support other options beyond the ones listed there or should I be looking at a different way of retargeting?
The ultimate goal here is to export a couple keys (asymmetric and symmetric) that were generated via nCipher's JCE API (yes, I know that an HSM's job is to secure the keys and exporting is usually not a good idea but it is a requirement here). We are able to export keys that were generated via the PKCS11 interface but not ones that were generated via the JCE so our thinking is that if we can retarget it from JCE to PKCS11 we might be able to export these keys as well. If there is another way to do this we are open to that as well.
Lastly, the JCE keys show up as "recovery enabled" when executing the nfkminfo on them. Does that mean that they are exportable or does recovery here mean something else?
Disclaimer: I work for Thales e-Security but do not speak for the company.
Yes you can retarget a jcecsp key to pkcs11. If you have any jcecsp keys in your kmdata/local, /opt/nfast/bin/generatekey will offer jcecsp as a source option. If you have no keys of that ilk, it will quietly omit that option from the source list. However, this retarget process may not do what you think it does. All retargeting does is change the application type and potentially the associated metadata: it doesn't change the fundamental capabilities of the key as those were baked into the protected key blob at generation time and cannot be changed.
The Security World uses nShield key ACLs to limit the key's capabilities (Sign, Verify, Encrypt, Decrypt, Wrap, Be Wrapped, etc.). PKCS#11 pulls its parameters (CKA_SIGN, etc.) directly from the key ACLs, and when generating keys through the API, the ACLs saved in the key blob are derived directly from the parameters in the key template. If you set CKA_SENSITIVE to FALSE, and your Security World allows it, you can generate and save an exportable key. JCE is not that sophisticated: it has no concept of key capabilities at all, so the Provider has to guess at the user's intent with the key and it defaults to a fairly generous set. However, since as you point out the whole idea of HSMs is to protect key bits and not let you have them, Export is not one of the defaults. And what's not baked into the key file when you create it, you don't get by retargeting the key.
One thing you could do if you want to use JCE is to generate the key using a different Provider and then store it in an nCipher.sworld KeyStore using the nCipherKM Provider: this will import the key into the Security World (if your World allows that) and save it as a key_jcecsp_* file. However this has nothing to do with key security so from an HSM perspective it's not recommended. Another thing you could do is to drop down to the native nCore API, generate the key with the ACL entries you require, and then polymorph it to a JCE Key Object and save it in the HSM-backed KeyStore. You can shoot yourself in the foot as many times as you want with the ACLs on the key you create. The polymorphing is very poorly documented: ask Thales Support and they can guide you.
Finally, the Recovery capability means that in addition to the Working Key blob which may be protected by an Operator Card Set, the key file has a Recovery Blob. This is in case that Operator Card Set is lost: the Recovery Blob can be opened up by the Administrator Card Set of the Security World using the rocs utility (Replace Operator Card Set), which will write a new key file under a new OCS. No, this does not mean the key is exportable. It just means that you are protected against losing the OCS. Of course losing the ACS is a non-starter as that is your Root of Trust.
I'm working on creating small relay stations out of tiny embedded Linux boxes. They have some sensors connected to them and transport data back to a server via HTTP POST. Right now the server just accepts their message, along with a unique ID (the MAC address of eth0).
I want to expand this to include some type of security. I want to be able to deploy these little devices with minimal configuration. I'd like to copy a base firmware to the device, hook them up in the field, and they self-register. The first time they connect, I'd like the server and device to do some type of negotiation where I can store a fingerprint. Subsequent requests I could then authentication/verify the device using that fingerprint.
That way, once a device registers with its unique ID, I can be assured all data from that ID is from the same device. If a rouge device or set of devices does register, I'll just delete them (I store IPs to so I can delete by unknown ranges and block them).
My question is what's the best way to go about doing this? I think back to the idea of SSH fingerprints, where the first time you connect to a server you get a server fingerprint. If a future request yields a different fingerprint, you get a huge warning and have to manually delete the fingerprint out of your authorized_keys file if the server's keys have actually regenerated (e.g. you did a reinstall without saving your old SSH keys).
Is something like this possibly with HTTP, possibly avoiding having to use preshared keys?
If it matters, the clients are running Python2 and the server they connect to is written mostly in Scala on Tomcat.
Basically, all you need to do is tell the server the public key, and then sign all of your messages with it. If you don't want pre-shared keys, then the server cannot be assured that someone new who is registering is actually one of your devices. You can still validate that the message came from the same device that originally registered with that identifier, however.
The process basically goes like this:
Client generates a new key pair (e.g. an RSA public/private key pair).
Client registers with server, sending its public key. The server stores this public key.
When the client sends a message, it generates a signature of its message, which it attaches to the message. When the server receives the message, it validates the signature to ensure that the message was sent by someone holding the corresponding private key.
The code for this in PyCrypto goes something like this:
Generate key pair
from Crypto.PublicKey import RSA
key = RSA.generate(2048)
private_key = key.exportKey()
public_key = key.publickey().exportKey()
# private_key is a string suitable for storing on disk for retrieval later
# public_key is a string suitable for sending to the server
# The server should store this along with the client ID for verification
Generate signature
from Crypto.PublicKey import RSA
from Crypto.Hash import SHA
key = RSA.importKey(private_key)
# where private_key is read from wherever you stored it previously
digest = SHA.new(message).digest()
signature = key.sign(digest, None)
# attach signature to the message however you wish
The server should load the public key as it has previously stored, and use a "verify" method provided by the Scala/Java crypto API you use, and accept the message only if it succeeds.
It is important to understand the caveats of each approach, as various techniques only protect against certain types of attacks. For instance, the above approach does not protect against a "replay attack", in which an attacker records a message with a certain meaning and then re-transmits it to the server at a later time. One way of protecting against this would be to include a timestamp in the message which is hashed; another would be to use an appropriately encrypted transport (e.g. SSL/TLS).