I hope, that you can help me here.
I'm trying to make a MQTT client (azure function in an app service environment) that extracts data from MQTT broker (hivemq), but I'm kind of lost when it comes to certificates (I'm not an expert).
Anyways, I received 3 files (look below) from our partner (that's them with the MQTT broker), but the question is. How should I use these PEM files in the application??
Client certificate == mqtt-client-cert.pem
Client key == mqtt-client-key_nopass.pem
CA certificate == server.pem
Here is the application and here is an example where I tested it locally (with a crt certificate) against an test broker (test.mosquitto.org). It worked perfectly, but now I just need to the same thing, just with 3 PEM files instead.
I also suspect that I need to do something on Azure (either on function app or app service environment level) in order to use these certificates?
// Create a new MQTT client.
var factory = new MqttFactory();
var mqttClient = factory.CreateMqttClient();
// Load certificate
X509Certificate caCertificate = new X509Certificate(#"..\mosquitto.org.crt");
// Create TLS based parameters.
var tlsParameters = new MqttClientOptionsBuilderTlsParameters
{
UseTls = true,
Certificates = new List<X509Certificate> { caCertificate },
SslProtocol = System.Security.Authentication.SslProtocols.Tls12
};
// Create TCP based options using the builder.
var connectOptions = new MqttClientOptionsBuilder()
.WithTcpServer("test.mosquitto.org", 8883)
.WithProtocolVersion(MQTTnet.Formatter.MqttProtocolVersion.V311)
.WithTls(tlsParameters)
.Build();
var conResult = await mqttClient.ConnectAsync(connectOptions);
mqttClient.UseConnectedHandler(e =>
{
Console.Write("Connected successfully with MQTT Brokers.");
});
mqttClient.UseDisconnectedHandler(e =>
{
Console.Write("Disconnected from MQTT Brokers.");
});
.crt files normally contain PEM encoded keys/certs so in this case the file extension doesn't make any difference.
You should be able to replace the mosquitto.org.crt with the server.pem.
The other 2 files are for what is known as mutual TLS authentication. For most TLS connections (e.g. when making a HTTPS request for a web page) only one side of the connection needs a certificate/private key. This is the server. The client uses a collections of CA certificates to validate that the service is what it claims to be. (This is what you are doing using the mosquitto.org.crt file).
In other cases we want to authenticate both ends of the connection (the client wants to know what the server is and the server wants to know who the client is). To do this the client need to also present a certificate to the server, this is what the other 2 files are for.
The MQTTNet documentation includes an example of setting up a connection using client certificates here but that uses a .pfx (a pfx is just another name for a PKCS12 container, if needed you can convert the .pem files into a .pfx/.p12 file using openssl e.g. openssl pkcs12 -export -out mqtt-client.p12 -inkey mqtt-client-key_nopass.pem -in mqtt-client-cert.pem -CAfile server.pem)
List<X509Certificate> certs = new List<X509Certificate>
{
new X509Certificate2("myCert.pfx")
};
var options = new MqttClientOptionBuilder()
.WithTcpServer(broker, port)
.WithTls(new MqttClientOptionsBuilderTlsParameters
{
UseTls = true,
Certificates = certs
})
.Build();
Related
I am trying to connect to an Apache Kafka server, through SSL connection and using an ASP.Net console application. This is my very first attempt at connecting to Kafka.
I have set up the producer config (shown below) but the people on Kafka side are Java shop and they use JKS certificate that apparently contains certificate and key. This is of no use to me. Is there a way to convert this to a .Net-friendly set of certificates?
var config = new ConsumerConfig
{
BootstrapServers = "kafka_broker_IP:9093",
SecurityProtocol = SecurityProtocol.Ssl,
SslCaLocation = "path/to/ca.crt",
SslCertificateLocation = "path/to/client.crt",
SslKeyLocation = "path/to/client.key"
};
Can I get ca.crt, client.cert and client.key from one single .jks file?
For some reasons our infra blocks mqtt.googleapis.com. That's why was deployed nginx proxy with such configuration
stream {
upstream google_mqtt {
server mgtt.googleapis.com:8883;
}
server {
listen 8883;
proxy_pass google_mqtt;
}
}
Also it has external IP with domain name fake.mqtt.com
Using example here I'm testing connectivity.
If script to run against mgtt.googleapis.com:8883 everything works fine.
But if domain switch to fake.mqtt.com got an error:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'fake.mqtt.com'
For client implementation was used paho.mqtt.client.
Auth to mqtt broker realized with JWT.
def create_jwt(project_id, private_key_file, algorithm):
token = {
# The time that the token was issued at
"iat": datetime.datetime.utcnow(),
# The time the token expires.
"exp": datetime.datetime.utcnow() + datetime.timedelta(minutes=20),
# The audience field should always be set to the GCP project id.
"aud": project_id,
}
# Read the private key file.
with open(private_key_file, "r") as f:
private_key = f.read()
print(
"Creating JWT using {} from private key file {}".format(
algorithm, private_key_file
)
)
return jwt.encode(token, private_key, algorithm=algorithm)
Set JWT
client.username_pw_set(
username='unused',
password=create_jwt(project_id, private_key_file, algorithm))
TLS configuration:
client.tls_set(ca_certs='roots.pem', tls_version=ssl.PROTOCOL_TLSv1_2,)
Could you advise what to configure on nginx/paho-client side and is it working solution at all?
Or may be 3party brokers can connect to mqtt.googleapis.com? (from information i read here and on another resources - no)
You can not just arbitrarily change the domain name if you are just stream proxying, it needs to match the one presented in the certificate by the remote broker or as you have seen it will not validate.
You can force the client to not validate the server name by setting client.tls_insecure_set(True) but this is a VERY bad idea and should only be used for testing and never in production.
I'm exhausted after looking for an answer for 3 days. I don't know if my suggested flow is wrong or my Google skills have really deteriorated.
My API needs to create a valid certificate from a CSR it received, by signing it with a private key that exists ONLY inside an HSM-like service (Azure KeyVault), which unfortunately doesn't offer Certificate Authority functions BUT does offer signing data with a key that exists there. My CA certificate's private key is stored in the HSM. I'm using ECDSA.
My suggested flow:
Client generates Key Pair + CSR and sends CSR to API
API creates a certificate from the CSR
API asks HSM to sign the CSR data and receives back a signature
API appends the signature to the certificate and returns a signed (and including CA in chain) certificate to the Client
I'm using C# .NET Core and would like to keep it cross-platform (as it runs in Linux containers), so I have to keep it as native as possible or using Bouncy Castle (which I'm still not sure if runs in Linux .NET Core).
I really appreciate your help!
I had faced a similar issue and found a solution. You'll have to use the PKCS11Interop.X509Store library.
The solution uses dotnet core native System.Security.Cryptography.X509Certificates.CertificateRequest::Create method
for generating a certificate.
As per the docs:
Pkcs11Interop is managed library written in C# that brings the
full power of PKCS#11 API to the .NET environment
Pkcs11Interop.X509Store is managed library built on top of
Pkcs11Interop. It's main goal is to provide easy to use PKCS#11 based
read-only X.509 certificate store that can be easily integrated with
standard .NET ecosystem.
Till v0.3.0, implementation for issuing a certificate (i.e signing a CSR) is not available.
With minor modifications in the PKCS11Interop library, I was able to sign the CSR.
Mentioned in Issue #30, the code is now added in the PKCS11Interop.X509Store library version 0.4.0.
The below code is taken from test cases for BasicEcdsaCertificateRequestTest. Test cases for RSA CertificateRequest are also there.
// Load PKCS#11 based store
using (var pkcs11Store = new Pkcs11X509Store(SoftHsm2Manager.LibraryPath, SoftHsm2Manager.PinProvider))
{
// Find signing certificate (CA certificate)
Pkcs11X509Certificate pkcs11CertOfCertificateAuthority = Helpers.GetCertificate(pkcs11Store, SoftHsm2Manager.Token1Label, SoftHsm2Manager.Token1TestUserEcdsaLabel);
// Generate new key pair for end entity
ECDsa ecKeyPairOfEndEntity = ECDsa.Create(ECCurve.NamedCurves.nistP256);
// Define certificate request
CertificateRequest certificateRequest = new CertificateRequest(
new X500DistinguishedName("C=SK,L=Bratislava,CN=BasicEcdsaCertificateRequestTest"),
ecKeyPairOfEndEntity,
HashAlgorithmName.SHA256);
// Define certificate extensions
certificateRequest.CertificateExtensions.Add(new X509BasicConstraintsExtension(false, false, 0, true));
certificateRequest.CertificateExtensions.Add(new X509SubjectKeyIdentifierExtension(certificateRequest.PublicKey, false));
certificateRequest.CertificateExtensions.Add(new X509KeyUsageExtension(X509KeyUsageFlags.DigitalSignature, false));
// Issue X.509 certificate for end entity
X509Certificate2 certificateOfEndEntity = certificateRequest.Create(
pkcs11CertOfCertificateAuthority.Info.ParsedCertificate.SubjectName,
X509SignatureGenerator.CreateForECDsa(pkcs11CertOfCertificateAuthority.GetECDsaPrivateKey()),
DateTimeOffset.UtcNow,
DateTimeOffset.UtcNow.AddDays(365),
new BigInteger(1).ToByteArray());
// Verify signature on X.509 certificate for end entity
Assert.IsTrue(CaCertSignedEndEntityCert(pkcs11CertOfCertificateAuthority.Info.ParsedCertificate.RawData, certificateOfEndEntity.RawData));
// Asociate end entity certificate with its private key
certificateOfEndEntity = certificateOfEndEntity.CopyWithPrivateKey(ecKeyPairOfEndEntity);
// Export end entity certificate to PKCS#12 file
string basePath = Helpers.GetBasePath();
string pkcs12FilePath = Path.Combine(basePath, "BasicEcdsaCertificateRequestTest.p12");
File.WriteAllBytes(pkcs12FilePath, certificateOfEndEntity.Export(X509ContentType.Pkcs12, "password"));
}
Hope this helps.
Is there any way to change a http server to https using the library http4s? (https://http4s.org/)
I found myself facing this same issue but I managed to solve it, here's the thing:
You need to look for the moment when you build your server, presumably with BlazeServerBuilder.
BlazeServerBuilder has the method "withSslContext(sslContext: SSLContext)" to enable SSL. Thus, all you need to do is create a SSLContext object and pass it to the server builder.
Remember that you will probably have to store your SSL certificate in a keystore using Java's keytool utility before using it.
SSL context and SSL certificate
How to create an SSL context with an SSL certificate is another question, but here is an interesting post that covers the process of getting a free certificate from Let's Encrypt, storing it in a keystore and using it from a Java application: Using Let's Encrypt certificates in Java applications - Ken Coenen — Ordina JWorks Tech Blog
And here's the code I used for creating a SSLContext in Scala:
val keyStorePassword: String = your_keystore_password
val keyManagerPassword: String = your_certificate_password
val keyStorePath: String = your_keystore_location
val keyStore = KeyStore.getInstance(KeyStore.getDefaultType)
val in = new FileInputStream(keyStorePath)
keyStore.load(in, keyStorePassword.toCharArray)
val keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm)
keyManagerFactory.init(keyStore, keyStorePassword.toCharArray)
val trustManagerFactory = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm)
trustManagerFactory.init(keyStore)
val sslContext = SSLContext.getInstance("TLS")
sslContext.init(keyManagerFactory.getKeyManagers, trustManagerFactory.getTrustManagers, new SecureRandom())
sslContext
Our setup includes a WCF service and a number of clients written by us. Some of the clients include Silverlight applications, whereas others include Web and Windows applications.
I (think) I would like to authenticate clients based on X.509 certificates. Typically you would install a private key on the client to encrypt (aka digitaly sign) the messages. The server can the use the clients public key to de-crypt it to ensure the message has not been changed and prove the message is from who we expect (aka authenticated).
I dont want to install a certificate on a client machine. Its a hassel to deploy, and we cant really ask our clients to do it. I was speaking to someone the other day who sugested embeding the cert in a client assembly, reading it and using that. Is that possible?
It would be great if someone could point me to an example.
Thanks in advance,
David
Yes, you can load X509certificate2 by passing a certificate byte array with a password like
var certificate = new X509Certificate2(theByteArrary, "password");
To get the certificate byte array, you can simply copy paste the contents in .pfx file, which is a combination of .cer (public key) and .pvk (private key)
and then you can load this certificate on your client by doing:
var channelFactory = new ChannelFactory<IYourService>();
channelFactory.Credentials.ClientCertificate.Certificate =
clientCertificate;
If you use auto-generated client proxy, or you prefer configure the certificate via .config file then you might want to have a look at this from codeproject
Here is a suggestion. Could also be tweaked to use an embedded certificate.
http://www.codeproject.com/KB/WCF/wcfcertificates.aspx