ClientDeviceAuthorizer: Device isn't authorized to connect - aws-iot-core

I have a Client Device (thing-is-1)
A Greengrass Core Device (Corething2)
I have deployed the componenets: MQTT Moquette Broker, MQTT Bridge, Client Device Authenticator, IP Detector.
My MQTT Broker is listening on port 8883
I have double checked all my AWS IOT Policies for Core:
It has everything that has been mentioned in the AWS documentations in terms of policies.
But When I try to connect my client to my core after discovery, it gives the following error on the client device:
Performing greengrass discovery...
awsiot.greengrass_discovery.DiscoverResponse(gg_groups=[awsiot.greengrass_discovery.GGGroup(gg_group_id='greengrassV2-coreDevice-Corething2', cores=[awsiot.greengrass_discovery.GGCore(thing_arn='arn:aws:iot:eu-west-1:...:thing/Corething2', connectivity=[awsiot.greengrass_discovery.ConnectivityInfo(id='', host_address='', metadata='', port=8883), awsiot.greengrass_discovery.ConnectivityInfo(id='', host_address='', metadata='', port=8883)])], certificate_authorities=['-----BEGIN CERTIFICATE-----\n..\n-----END CERTIFICATE-----\n'])])
Trying core arn:aws:iot:eu-west-1:..:thing/Corething2 at host port 8883
Connection failed with exception AWS_IO_SOCKET_TIMEOUT: socket operation timed out.
Trying core arn:aws:iot:eu-west-1:..:thing/Corething2 at host port 8883
Connection failed with exception AWS_ERROR_MQTT_UNEXPECTED_HANGUP: The connection was closed unexpectedly.
All connection attempts failed
Now if I go to my Core device and check the greengrass.log.. I see this:
2022-04-11T15:07:43.899Z [INFO] (nioEventLoopGroup-5-3) com.aws.greengrass.device.DeviceAuthClient: Creating new session. {}
2022-04-11T15:07:44.454Z [INFO] (nioEventLoopGroup-5-3) com.aws.greengrass.device.SessionManager: Created the session. {sessionId=d65a97e6-1919-4798-8c2d-bb9b44398856}
2022-04-11T15:07:44.473Z [INFO] (nioEventLoopGroup-5-3) io.moquette.broker.metrics.MQTTMessageLogger: C->B CONNECT . {}
2022-04-11T15:07:44.473Z [INFO] (nioEventLoopGroup-5-3) com.aws.greengrass.mqttbroker.ClientDeviceAuthorizer: Retrieved client session. {clientId=thing-is-1, sessionId=d65a97e6-1919-4798-8c2d-bb9b44398856}
2022-04-11T15:07:44.799Z [INFO] (nioEventLoopGroup-5-3) com.aws.greengrass.mqttbroker.ClientDeviceAuthorizer: Device isn't authorized to connect. {clientId=thing-is-1, sessionId=d65a97e6-1919-4798-8c2d-bb9b44398856}
2022-04-11T15:07:44.799Z [INFO] (nioEventLoopGroup-5-3) com.aws.greengrass.device.SessionManager: Closing the session. {sessionId=d65a97e6-1919-4798-8c2d-bb9b44398856}
2022-04-11T15:07:44.800Z [INFO] (nioEventLoopGroup-5-3) io.moquette.broker.MQTTConnection: Authenticator has rejected the MQTT credentials CId=thing-is-1, certificate chain=[[
[
Version: V3
Subject: CN=AWS IoT Certificate
bla bla bla
]]. {}
2022-04-11T15:07:44.800Z [INFO] (nioEventLoopGroup-5-3) io.moquette.broker.MQTTConnection: Client didn't supply any password and MQTT anonymous mode is disabled CId=thing-is-1. {}
2022-04-11T15:07:44.802Z [INFO] (nioEventLoopGroup-5-3) io.moquette.broker.metrics.MQTTMessageLogger: Channel Inactive. {}
2022-04-11T15:08:41.247Z [INFO] (pool-1-thread-4) com.aws.greengrass.detector.IpDetectorManager: Acquired host IP addresses. {IpAddresses=[/, /]}
What am I missing here? Or maybe if there is a checklist that I can refer to for scratching out possibilities.
One question arises with the certs, do I need to add my clients public to some place in the Core? I didnt find that anywhere in the aws docs.
Also I see that the session is created but then my Authenticator rejects the client.
My Device Authenticator has a complete permissive configuration.
My thing-is-1 is associated to my core device. But the core device and the client device do not belong to the same thing group. (And I don't think that makes any difference)
ClientDeviceAuth Component config:
{
"reset": [],
"merge": {
"reset": [],
"merge": {
"deviceGroups": {
"formatVersion": "2021-03-05",
"definitions": {
"MyDeviceGroup": {
"selectionRule": "thingName: thing-*",
"policyName": "MyClientDevicePolicy"
}
},
"policies": {
"MyClientDevicePolicy": {
"AllowConnection": {
"statementDescription": "Allow client devices.",
"operations": [
"*"
],
"resources": [
"*"
]
}
}
}
}
}
}
}
I tried to be informative and concise at the same time. Let me know if im missing any info that might help to get a better understanding of the issue and I'll update the question accordingly.

Your client device auth configuration seems to have "merge" as a child of "merge"? That isn't correct. The device groups and policies should be keys under the top level merge.

Related

Step timed out while step is verifying the SSM Agent availability on the target instance(s). SSM Agent on Instances: [i-xxxxxxx] are not functioning

I am working on "Patch an AMI and update an Auto Scaling group" and followed the AWS document to configure but I am stuck at "Task 3: Create a runbook, patch the AMI, and update the Auto Scaling group" with the below error. To fix it I have added "user data" while starting the instance(startInstances). As it's accepting only base64, converted and provided base64(UmVzdGFydC1TZXJ2aWNlIEFtYXpvblNTTUFnZW50Cg==).
I tried to execute with the below user data but both are not working, even I tried to apply a new step with the same commands but failed to patch AMI.
Tried the below script:
<powershell> powershell.exe -Command Start-Service -Name AmazonSSMAgent </powershell> <persist>true</persist>
Tried to start and restart SSM agent.
Restart-Service AmazonSSMAgent
base64: UmVzdGFydC1TZXJ2aWNlIEFtYXpvblNTTUFnZW50Cg==
YAML sample:
mainSteps:
- name: startInstances
action: 'aws:runInstances'
timeoutSeconds: 1200
maxAttempts: 1
onFailure: Abort
inputs:
ImageId: '{{ sourceAMIid }}'
InstanceType: m3.large
MinInstanceCount: 1
MaxInstanceCount: 1
SubnetId: '{{ subnetId }}'
UserData: UmVzdGFydC1TZXJ2aWNlIEFtYXpvblNTTUFnZW50Cg==
Still, I am seeing the below error.
Step timed out while step is verifying the SSM Agent availability on the target instance(s). SSM Agent on Instances: [i-xxxxxxxx] are not functioning. Please refer to Automation Service Troubleshooting Guide for more diagnosis details.
Your suggestion/solutions help me a lot. Thank you.
I have troubleshoot and fixed the issue.
The issue was security group is missing on the instance. To communicate the SendCommand API of SSM Service with SSM agent on instance needs a security group that allows HTTPS port 443. I have attached SG allowing 443 port then the SSM agent can communicate with the EC2 instance.
EC2 instance IAM role should have SSM agent full access policy attached to it.
We might get the same issue when the SSM agent is not running on the EC2 instance, for that we need to provide user-data or add a new step in YAML or JSON on systems manager Documents.
If you are working on a windows instance use the below script to start the SSM agent. If its Linux server uses Linux script/commands.
{
"schemaVersion": "2.0",
"description": "Start SSM agent on instance",
"mainSteps": [
{
"action": "aws:runPowerShellScript",
"name": "runPowerShellScript",
"inputs": {
"runCommand": [
"Start-Service AmazonSSMAgent"
]
}
}
]
}

Spring Cloud Data Flow Server OAuth2 with Azure AD

I am trying to setup OAuth authentication (OpenID Connect) with Azure AD and Spring Cloud Data Flow server. Following below documentation reference but no help.
https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#appendix-identity-provider-azure
https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-security-oauth2
When I push data flow server in open source cloud foundry, i get below error (if OAuth2 config is removed from application.yml, it gets deployed without issue):
redentials/instance.key and /etc/cf-instance-credentials/instance.crt
18:16:57.512: [HEALTH.0] Failed to make TCP connection to port 8080: connection refused
18:16:57.512: [CELL.0] Timed out after 1m0s: health check never passed.
18:16:57.515: [CELL/SSHD.0] Exit status 0
18:17:07.588: [APP/PROC/WEB.0] Exit status 137 (exceeded 10s graceful shutdown interval)
18:17:07.857: [API.3] Process has crashed with type: "web"
18:17:07.931: [API.3] App instance exited with guid ff60a149-d91f-4d9c-90b9-661c3bb8ad0f payload: {"instance"=>"e35f4a5d-a4f0-433d-6546-82ed", "index"=>0, "cell_id"=>"231ab214-d841-46ba-b20f-243aeac9bbfa", "reason"=>"CRASHED", "exit_description"=>"Instance never healthy after 1m0s: Failed to make TCP connection to port 8080: connection refused", "crash_count"=>3, "crash_timestamp"=>1622845027800626529, "version"=>"3a0686fb-a43a-4528-a425-21a544
From the logs, I do not see any hint of OAuth2.0 setup is taking effect by data flow server. Here is my application.yml entries related to OAuth2 config:
spring:
cloud:
dataflow:
security:
authorization:
provider-role-mappings:
dataflow-server:
map-oauth-scopes: true
role-mappings:
ROLE_VIEW: dataflow.view
ROLE_CREATE: dataflow.create
ROLE_MANAGE: dataflow.manage
ROLE_DEPLOY: dataflow.deploy
ROLE_DESTROY: dataflow.destroy
ROLE_MODIFY: dataflow.modify
ROLE_SCHEDULE: dataflow.schedule
security:
oauth2:
client:
registration:
dataflow-server:
provider: azure
redirect-uri: 'https://data-flow-server/dashboard'
client-id: 977-95bc-4f3645d77f43
client-secret: ~02K-5pf182_E-x-PWn
authorization-grant-type: authorization_code
scope:
- openid
- profile
- email
- offline_access
- api://dataflow-server/dataflow.view
- api://dataflow-server/dataflow.deploy
- api://dataflow-server/dataflow.destroy
- api://dataflow-server/dataflow.manage
- api://dataflow-server/dataflow.modify
- api://dataflow-server/dataflow.schedule
- api://dataflow-server/dataflow.create
provider:
azure:
issuer-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/v2.0
user-name-attribute: name
access-token-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/token
token-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/token
user-authorization-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/authorize
authorization-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/oauth2/v2.0/authorize
resourceserver:
jwt:
jwk-set-uri: https://login.microsoftonline.com/sdf3s-3244f65-b82d-5ec2fd32d5aa/discovery/v2.0/keys

Shutdown Hook Corda integration testing

I have the following Corda service. It starts up a Jetty Server
#CordaService
class IOUService(private val serviceHub: AppServiceHub): SingletonSerializeAsToken() {
init {
val port = serviceHub.myInfo.addresses.first().port - 1002
log.println("IOUService init was called...")
log.println("Port: $port")
val jettyServer = JettyServer()
jettyServer.start(port)
}
}
My problem is how to release the stared Jetty port when running integration tests. Here are two example test (basically the same test twice to illustrate the problem):
#Test
fun `node test`() = withDriver {
val (partyAHandle, partyBHandle) = startNodes(bankA, bankB)
assertEquals(bankB.name, partyAHandle.resolveName(bankB.name))
assertEquals(bankA.name, partyBHandle.resolveName(bankA.name))
}
#Test
fun `node test2`() = withDriver {
val (partyAHandle, partyBHandle) = startNodes(bankA, bankB)
assertEquals(bankB.name, partyAHandle.resolveName(bankB.name))
assertEquals(bankA.name, partyBHandle.resolveName(bankA.name))
}
The first test will start up 3 nodes: one notary, BankA and BankB nodes with the following details:
Notary:
Advertised P2P messaging addresses : localhost:10000
RPC connection address : localhost:10001
RPC admin connection address : localhost:10002
Jetty Port: 8998
BankA:
Advertised P2P messaging addresses : localhost:10004
RPC connection address : localhost:10005
RPC admin connection address : localhost:10006
Jetty Port: 9002
BankB:
Advertised P2P messaging addresses : localhost:10008
RPC connection address : localhost:10009
RPC admin connection address : localhost:10010
Jetty Port: 9006
Unfortunately the second test will fails since the Jetty ports are still bound:
[ERROR] 14:22:04,825 [driver-pool-thread-0] internal.Node.installCordaServices - Corda service com.example.flows.IOUService failed to instantiate. Reason was: Address already in use [errorCode=1pryyp4, moreInformationAt=https://errors.corda.net/OS/4.1/1pryyp4]
java.io.IOException: Failed to bind to 0.0.0.0/0.0.0.0:9006
How to register a shutdown hook during integration testing in order to shut down the Jetty servers?
The example code can be found here:
https://github.com/altfatterz/learning-corda
A proper lifecycle for corda services is currently being looked at. Hopefully, you can do this in the future.
For now, there is not a simple way to do this from within the node.

EUCA 4.4.5 VPCMIDO Instances Terminate at Launch

I have achieved a small test cloud on 3 pieces of hardware. It works fine when in EDGE mode but when I try to configure it for VPCMIDO, new instances begin to launch but then timeout and move to a terminated state. I can also see the instances' initial volume and config data appear in the NC and CC data directories. Below is my system layout and network.json.
HOST 1 : CLC/UFS/WALRUS/MIDO CLUSTER/MIDO GATEWAY/MIDOLMAN AGENT:
em1 (All Services including Mido Cluster): 10.0.0.21
em3 (Target VPCMIDO Adapter): 10.0.0.22
HOST 2 : CC/SC
em1 : 10.0.0.23
HOST 3 : NC/MIDOLMAN AGENT
em1 : 10.0.0.24
{
"Mido": {
"Gateways": [
{
"Ip": "10.0.0.22",
"ExternalDevice": "em3",
"ExternalCidr": "192.168.0.0/16",
"ExternalIp": "192.168.0.2",
"ExternalRouterIp": "192.168.0.1"
}
]
},
"Mode": "VPCMIDO",
"PublicIps": [
"10.0.100.1-10.0.100.254"
]
}
I may be misunderstanding the intent of reserving an interface just for the mido gateway. All of my eucalyptus/zookeeper/cassandra/midonet configs use the 10.0.0.21 interface and seem to communicate fine. The midonet tunnel zone reports my CLC host and NC host successfully in the tunnel zone. The only part of my config that references the interface I intend to use for the midonet gateway is the network.json. No errors were returned at any time during my config so I think I may be missing something conceptual.
You may need to start eucanetd as described here:
https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#install-guide/starting_euca_clc.html
The eucanetd component in vpcmido mode runs on the cloud controller and is responsible for controlling midonet.
When eucanetd is not running instances will fail to start as the required network resources will not be created.
I configured a bridge on the NC and instances were able to launch and I no longer got an error in my nc.log. Docs and the eucalyptus.conf file comments tell me I shouldn't need to do this in VPCMIDO netowrking mode: https://docs.eucalyptus.cloud/eucalyptus/4.4.5/index.html#install-guide/configuring_bridge.html
Despite all that adding the bridge fixed this issue.

Why does Meteor Up (MUP) fail on authentication?

I am currently trying to deploy a Meteor project to an external server for the first time. The server is hosted by DigitalOcean, running ubuntu 16.04, and has an SSH key set up for password-free access.
The error I am getting from MUP is:
[159.203.165.13] - Setup Docker
events.js:165
throw er; // Unhandled 'error' event
^
Error: All configured authentication methods failed
at tryNextAuth (/usr/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2/lib/client.js:290:17)
at SSH2Stream.onUSERAUTH_FAILURE (/usr/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2/lib/client.js:469:5)
at SSH2Stream.emit (events.js:180:13)
at parsePacket (/usr/lib/node_modules/mup/node_modules/ssh2-streams/lib/ssh.js:3647:10)
at SSH2Stream._transform (/usr/lib/node_modules/mup/node_modules/ssh2-streams/lib/ssh.js:551:13)
at SSH2Stream.Transform._read (_stream_transform.js:185:10)
at SSH2Stream._read (/usr/lib/node_modules/mup/node_modules/ssh2-streams/lib/ssh.js:212:15)
at SSH2Stream.Transform._write (_stream_transform.js:173:12)
at doWrite (_stream_writable.js:410:12)
at writeOrBuffer (_stream_writable.js:396:5)
at SSH2Stream.Writable.write (_stream_writable.js:294:11)
at Socket.ondata (_stream_readable.js:651:20)
at Socket.emit (events.js:180:13)
at addChunk (_stream_readable.js:274:12)
at readableAddChunk (_stream_readable.js:261:11)
at Socket.Readable.push (_stream_readable.js:218:10)
Emitted 'error' event at:
at tryNextAuth (/usr/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2/lib/client.js:292:12)
at SSH2Stream.onUSERAUTH_FAILURE (/usr/lib/node_modules/mup/node_modules/nodemiral/node_modules/ssh2/lib/client.js:469:5)
[... lines matching original stack trace ...]
at Socket.Readable.push (_stream_readable.js:218:10)
At this point I have tried several solutions involving the mup file as per other recommendations such as:
1) Adding in a password - Gives the exact same error as though the change didn't occur.
2) Adding in the same SSH key that I use for authentication to the server as per digital ocean - Says 'privateKey value does not contain a (valid) private key'. I have tried both the key that is used for authentication to the server and every other key I could find short of generating a new one just for Meteor's use.
3) Leaving both blank and allowing it to 'try' ssh-agent - pretends it doesn't know what ssh-agent is and throws an error saying the same thing as when I use a password.
I have looked through and followed the same instructions in the following article: http://meteortips.com/deployment-tutorial/digitalocean-part-1/
This article assumes that there are only two possible states. One being that an ssh key has NOT been used or set up so it needs to be generated. The second being that an ssh key exists and is set up exactly where they expect it. Unfortunately I seem to be in a different situation. I generated a key using putty prior to setting up the D.O server and created the droplet using that. After creation, the file did not exist. The only thing in the ~/.ssh/ directory was a single file named "authorized_keys" that held the key I would use to connect to the server. This file cannot be used, nor any file on the server in the other ssh key locations.I also tried copying over the file directly onto the server to no avail as well.
In some vain hope at finding a solution I also tried running these same commands in both the Meteor build bundle an the source code folder. Neither worked. I should mention that although this is the only article I still have open to try for a solution, I have tried every one I could find using MUP.
If anyone can point me in the right direction with this so I can stop flailing wildly in the dark I would be incredibly grateful.
Edit: As requested, below is the current mup.js file with removed credentials
module.exports = {
servers: {
one: {
// TODO: set host address, username, and authentication method
host: '111.111.111.11',
username: 'root',
// ssh-agent: '/home/Meteor/MeteorKey.pem'
pem: '~/.ssh/id_rsa.pub'
// password: 'password1'
// or neither for authenticate from ssh-agent
}
},
app: {
// TODO: change app name and path
name: 'app-name',
path: '../',
servers: {
one: {},
},
buildOptions: {
serverOnly: true,
},
env: {
// TODO: Change to your app's url
// If you are using ssl, it needs to start with https://
ROOT_URL: 'http://www.app-name.com',
MONGO_URL: 'mongodb://mongodb/meteor',
MONGO_OPLOG_URL: 'mongodb://mongodb/local',
},
docker: {
// change to 'abernix/meteord:base' if your app is using Meteor 1.4 - 1.5
image: 'abernix/meteord:node-8.4.0-base',
},
// Show progress bar while uploading bundle to server
// You might need to disable it on CI servers
enableUploadProgressBar: true
},
mongo: {
version: '3.4.1',
servers: {
one: {}
}
},
// (Optional)
// Use the proxy to setup ssl or to route requests to the correct
// app when there are several apps
// proxy: {
// domains: 'mywebsite.com,www.mywebsite.com',
The error message you are receiving:
Error: All configured authentication methods failed
Means that the SSH connection is failing. So the credentials you are using (pity you removed them from the config) are not working. Try using a command line ssh using these same credentials, and then trouble shoot that - once you can ssh into the server, then mup should be able to do it's work.
You can get more information out of ssh by specifying one or more -v parameters, eg:
ssh -v -v my_user#remote.com
and it will give you information about the authentication methods it is trying as it goes through them. This will help you narrow down the problem.

Resources