Openstack Keystone OTP - openstack

I need to write a TOTP plugin for Keystone. And now I follow this guide http://docs.openstack.org/developer/keystone/auth-totp.html
But I can't pass the first step and create the TOTP credential.
I got the following error.
{"error": {"message": "The request you have made requires authentication.", "code": 401, "title": "Unauthorized"}}
Debug:
sudo tail -n 50 /var/log/httpd/keystone.log
2016-03-09 12:04:55.808 1181 DEBUG keystone.middleware.auth [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] There is either no auth token in the request or the certificate issuer is not trusted. No auth context will be set. 2016-03-09 12:04:55.808 process_request /opt/stack/keystone/keystone/middleware/auth.py:171
2016-03-09 12:04:55.814 1181 INFO keystone.common.wsgi [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] POST http://localhost:5000/v3/credentials
2016-03-09 12:04:55.815 1181 DEBUG keystone.common.controller [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] RBAC: Authorizing identity:create_credential(credential={u'user_id': u'4725c2a6592c46b89bbd42da1731d5ed', u'type': u'totp', u'blob': u'OBQXG43XN5ZGI'}) 2016-03-09 12:04:55.815 _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:80
2016-03-09 12:04:55.816 1181 DEBUG keystone.common.controller [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] RBAC: building auth context from the incoming auth token 2016-03-09 12:04:55.816 _build_policy_check_credentials /opt/stack/keystone/keystone/common/controller.py:92
2016-03-09 12:04:55.817 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 140351135589296, thread 140351402075904: Acquiring connection 2016-03-09 12:04:55.817 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.817 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 140351135589296, thread 140351402075904: Acquired connection 140351418931208 2016-03-09 12:04:55.817 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.818 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 1
40351135589296, thread 140351402075904: Releasing connection 140351418931208 2016-03-09 12:04:55.818 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.819 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 140351135589296, thread 140351402075904: Acquiring connection 2016-03-09 12:04:55.819 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.819 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 140351135589296, thread 140351402075904: Acquired connection 140351418931208 2016-03-09 12:04:55.819 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.820 1181 DEBUG oslo.cache._memcache_pool [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Memcached pool 140351135589296, thread 140351402075904: Releasing connection 140351418931208 2016-03-09 12:04:55.820 _debug_logger /usr/lib/python2.7/site-packages/oslo_cache/_memcache_pool.py:116
2016-03-09 12:04:55.821 1181 WARNING keystone.common.controller [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] RBAC: Invalid token
2016-03-09 12:04:55.823 1181 WARNING keystone.common.wsgi [req-c8f883dd-cdb1-46fc-bca3-70ea60285050 - - - - -] Authorization failed. The request you have made requires authentication. from ::1
My scripts.
Encoding the TOTP secret:
cat base32_str_encoding.py
#!/usr/bin/python
import base64
secret = 'password'
print base64.b32encode(secret).rstrip('=')
./base32_str_encoding.py
OBQXG43XN5ZGI
Creating the TOTP credential:
cat create_totp_credential.sh
#!/usr/bin/bash
USER_ID=4725c2a6592c46b89bbd42da1731d5ed
SECRET=OBQXG43XN5ZGI
curl -i \
-H "Content-Type: application/json" \
-d '
{
"credential": {
"blob": "'$SECRET'",
"type": "totp",
"user_id": "'$USER_ID'"
}
}' \
http://localhost:5000/v3/credentials ; echo
Keystone user database.
mysql -uuser -h localhost -ppassword -Bse 'use keystone; select * from user where name="trex";'
| id | name | extra | password | enabled | domain_id | default_project_id |
4725c2a6592c46b89bbd42da1731d5ed trex {"description": "Test user", "email": "trex#trex.com"} $6$rounds=10000$GfVY/Ws6cxS43fVC$p.44zK6gskY9Y3Aa8MwMCpeIfmKaZVlpB2niv2ewDeQbCyuqBhwOuMwfHLY5Kl67I/QsQUmG5BuK5BB6UtaKe/ 1 default d2c1b6084c5a41ceb5582c736e3f03f8

curl -i \
-H "X-Auth-Token: $OS_TOKEN" \ ###########INSERT YOUR TOKEN############
-H "Content-Type: application/json" \
-d '
{
"credential": {
"blob": "'$SECRET'",
"type": "totp",
"user_id": "'$USER_ID'"
}
}' \
http://localhost:5000/v3/credentials ; echo

Related

Problems with Java moneta Money object when used in an Axon command

I have an Axon Command which has an moneta Money object.
import lombok.Getter;
import lombok.ToString;
import lombok.experimental.SuperBuilder;
import org.javamoney.moneta.Money;
import java.time.LocalDate;
import java.util.UUID;
#Getter
#SuperBuilder
#ToString
public class MyAxonCommand {
private final UUID id;
private final Money hoogte;
private final LocalDate opleggingsdatum;
}
When i send this command with axon there is an exception.
commandGateway.sendAndWait(myAxonCommand.builder()
.id(new UUID(1, 1))
.hoogte(Money.of(0, "EUR"))
.opleggingsdatum(LocalDate.now())
.build());
The exception thrown is Caused by:
18:07:37.456 [main] INFO org.javamoney.moneta.DefaultMonetaryContextFactory - Using custom MathContext: precision=256, roundingMode=HALF_EVEN
18:07:37.465 [main] INFO nl.ind.handhaving.adapter.messaging.incoming.IndigoListener kvk:987654321 zn:Z1-31190106952 - INDiGO bericht ontvangen op methode: receiveMaatregelOpgelegd
18:07:37.928 [docker-java-stream--1691755530] INFO docker.axonserver - STDOUT: 2023-01-18 17:07:37.925 WARN 1 --- [nio-8024-exec-3] A.i.a.a.rest.DevelopmentRestController : [<anonymous>] Request to delete all events in context "default".
18:07:37.941 [EventProcessor[nl.ind.handhaving.application.query]-0] WARN org.axonframework.eventhandling.TrackingEventProcessor - Error occurred. Starting retry mode.
org.axonframework.axonserver.connector.AxonServerException: The Event Stream has been closed, so no further events can be retrieved
at org.axonframework.axonserver.connector.event.axon.EventBuffer.peekNullable(EventBuffer.java:178)
at org.axonframework.axonserver.connector.event.axon.EventBuffer.hasNextAvailable(EventBuffer.java:144)
at org.axonframework.eventhandling.TrackingEventProcessor.processBatch(TrackingEventProcessor.java:401)
at org.axonframework.eventhandling.TrackingEventProcessor.processingLoop(TrackingEventProcessor.java:300)
at org.axonframework.eventhandling.TrackingEventProcessor$TrackingSegmentWorker.run(TrackingEventProcessor.java:1072)
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.cleanUp(TrackingEventProcessor.java:1263)
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.run(TrackingEventProcessor.java:1240)
at java.base/java.lang.Thread.run(Thread.java:833)
18:07:37.942 [EventProcessor[nl.ind.handhaving.application.query]-0] WARN org.axonframework.eventhandling.TrackingEventProcessor - Releasing claim on token and preparing for retry in 1s
18:07:37.945 [EventProcessor[nl.ind.handhaving.application]-0] WARN org.axonframework.eventhandling.TrackingEventProcessor - Error occurred. Starting retry mode.
org.axonframework.axonserver.connector.AxonServerException: The Event Stream has been closed, so no further events can be retrieved
at org.axonframework.axonserver.connector.event.axon.EventBuffer.peekNullable(EventBuffer.java:178)
at org.axonframework.axonserver.connector.event.axon.EventBuffer.hasNextAvailable(EventBuffer.java:144)
at org.axonframework.eventhandling.TrackingEventProcessor.processBatch(TrackingEventProcessor.java:401)
at org.axonframework.eventhandling.TrackingEventProcessor.processingLoop(TrackingEventProcessor.java:300)
at org.axonframework.eventhandling.TrackingEventProcessor$TrackingSegmentWorker.run(TrackingEventProcessor.java:1072)
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.cleanUp(TrackingEventProcessor.java:1263)
at org.axonframework.eventhandling.TrackingEventProcessor$WorkerLauncher.run(TrackingEventProcessor.java:1240)
at java.base/java.lang.Thread.run(Thread.java:833)
18:07:37.945 [EventProcessor[nl.ind.handhaving.application]-0] WARN org.axonframework.eventhandling.TrackingEventProcessor - Releasing claim on token and preparing for retry in 1s
18:07:37.947 [EventProcessor[nl.ind.handhaving.application]-0] INFO org.axonframework.eventhandling.TrackingEventProcessor - Released claim
18:07:37.949 [EventProcessor[nl.ind.handhaving.application.query]-0] INFO org.axonframework.eventhandling.TrackingEventProcessor - Released claim
org.axonframework.commandhandling.CommandExecutionException: org.javamoney.moneta.spi.JDKCurrencyAdapter
at org.axonframework.axonserver.connector.ErrorCode.lambda$static$11(ErrorCode.java:88)
at org.axonframework.axonserver.connector.ErrorCode.convert(ErrorCode.java:182)
at org.axonframework.axonserver.connector.command.CommandSerializer.deserialize(CommandSerializer.java:164)
at org.axonframework.axonserver.connector.command.AxonServerCommandBus.lambda$doDispatch$1(AxonServerCommandBus.java:161)
at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2147)
at io.axoniq.axonserver.connector.command.impl.CommandChannelImpl$CommandResponseHandler.onNext(CommandChannelImpl.java:372)
at io.axoniq.axonserver.connector.command.impl.CommandChannelImpl$CommandResponseHandler.onNext(CommandChannelImpl.java:359)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onMessage(ClientCalls.java:466)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInternal(ClientCallImpl.java:661)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1MessagesAvailable.runInContext(ClientCallImpl.java:646)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: AxonServerRemoteCommandHandlingException{message=An exception was thrown by the remote message handling component: org.javamoney.moneta.spi.JDKCurrencyAdapter, errorCode='AXONIQ-4002', server='134589#xxxxxxxxx}
at org.axonframework.axonserver.connector.ErrorCode.lambda$static$11(ErrorCode.java:86)
... 16 more}
The Axonserver logging - running in a docker :
2023-01-18T17:30:30.237808536Z _ ____
2023-01-18T17:30:30.237852159Z / \ __ _____ _ __ / ___| ___ _ ____ _____ _ __
2023-01-18T17:30:30.237857221Z / _ \ \ \/ / _ \| '_ \\___ \ / _ \ '__\ \ / / _ \ '__|
2023-01-18T17:30:30.237861155Z / ___ \ > < (_) | | | |___) | __/ | \ V / __/ |
2023-01-18T17:30:30.237864060Z /_/ \_\/_/\_\___/|_| |_|____/ \___|_| \_/ \___|_|
2023-01-18T17:30:30.237866979Z Standard Edition Powered by AxonIQ
2023-01-18T17:30:30.237869529Z
2023-01-18T17:30:30.237872060Z version: 4.5.16
2023-01-18T17:30:30.326181167Z 2023-01-18 17:30:30.321 INFO 1 --- [ main] io.axoniq.axonserver.AxonServer : Starting AxonServer using Java 11.0.14 on c32eb57825c4 with PID 1 (/app/classes started by root in /)
2023-01-18T17:30:30.331544104Z 2023-01-18 17:30:30.325 INFO 1 --- [ main] io.axoniq.axonserver.AxonServer : No active profile set, falling back to 1 default profile: "default"
2023-01-18T17:30:33.989108312Z 2023-01-18 17:30:33.988 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8024 (http)
2023-01-18T17:30:34.235755158Z 2023-01-18 17:30:34.231 INFO 1 --- [ main] A.i.a.a.c.MessagingPlatformConfiguration : Configuration initialized with SSL DISABLED and access control DISABLED.
2023-01-18T17:30:37.126812182Z 2023-01-18 17:30:37.125 INFO 1 --- [ main] io.axoniq.axonserver.AxonServer : Axon Server version 4.5.16
2023-01-18T17:30:39.285810090Z 2023-01-18 17:30:39.285 WARN 1 --- [ main] .s.s.UserDetailsServiceAutoConfiguration :
2023-01-18T17:30:39.285860293Z
2023-01-18T17:30:39.285865737Z Using generated security password: f23552a4-9623-4adb-831e-506eac6a10a9
2023-01-18T17:30:39.285868706Z
2023-01-18T17:30:39.285871675Z This generated password is for development use only. Your security configuration must be updated before running your application in production.
2023-01-18T17:30:39.285874618Z
2023-01-18T17:30:41.633817404Z 2023-01-18 17:30:41.633 INFO 1 --- [ main] io.axoniq.axonserver.grpc.Gateway : Axon Server Gateway started on port: 8124 - no SSL
2023-01-18T17:30:41.667366113Z 2023-01-18 17:30:41.667 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8024 (http) with context path ''
2023-01-18T17:30:42.561266508Z 2023-01-18 17:30:42.555 INFO 1 --- [ main] io.axoniq.axonserver.AxonServer : Started AxonServer in 12.861 seconds (JVM running for 13.412)
2023-01-18T17:30:57.342935513Z 2023-01-18 17:30:57.338 INFO 1 --- [grpc-executor-1] i.a.a.logging.TopologyEventsLogger : Application connected: handhaving-service, clientId = 149931#v2l1-xxxxl, clientStreamId = 149931#v2l1-xxxxx.87e0f589-66d8-41ee-ab4a-7bc599cc2c01, context = default
2023-01-18T17:31:02.213813565Z 2023-01-18 17:31:02.213 WARN 1 --- [nio-8024-exec-3] A.i.a.a.rest.DevelopmentRestController : [<anonymous>] Request to delete all events in context "default".
2023-01-18T17:31:04.567541554Z 2023-01-18 17:31:04.567 INFO 1 --- [grpc-executor-3] i.a.a.logging.TopologyEventsLogger : Application disconnected: handhaving-service, clientId = 149931#xxxxx.87e0f589-66d8-41ee-ab4a-7bc599cc2c01, context = default: Platform connection completed by client
The issue seems to be that Axon is storing this Money object in a database, according the errorCode='AXONIQ-4002'.
What can i do to fix this? Does Axon needs a hibernate UserType so Axon is able to store this Money object or some other kind of type converter?
It seems that the de-serilizer in the axon server has problems with the Money object.
In order to store this Money object in a view database - where i store the event generated by the command - i had to make a type conversion for hibernate. this seems to be related to the occurred exception.
The project uses:
Spring Boot 2.7.6
axon-spring-boot-starter 4.5.15
moneta 1.4.2
It al runs with Java Temurin 17.0.4
For axon we have no configuration for serializing so the default is used: XML

Apache Airflow : Dag task marked zombie, with background process running on remote server

**Apache Airflow version:**1.10.9-composer
Kubernetes Version : Client Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.12-gke.6002", GitCommit:"035184604aff4de66f7db7fddadb8e7be76b6717", GitTreeState:"clean", BuildDate:"2020-12-01T23:13:35Z", GoVersion:"go1.12.17b4", Compiler:"gc", Platform:"linux/amd64"}
Environment: Airflow, running on top of Kubernetes - Linux version 4.19.112
OS : Linux version 4.19.112+ (builder#7fc5cdead624) (Chromium OS 9.0_pre361749_p20190714-r4 clang version 9.0.0 (/var/cache/chromeos-cache/distfiles/host/egit-src/llvm-project c11de5eada2decd0a495ea02676b6f4838cd54fb) (based on LLVM 9.0.0svn)) #1 SMP Fri Sep 4 12:00:04 PDT 2020
Kernel : Linux gke-europe-west2-asset-c-default-pool-dc35e2f2-0vgz
4.19.112+ #1 SMP Fri Sep 4 12:00:04 PDT 2020 x86_64 Intel(R) Xeon(R) CPU # 2.20GHz GenuineIntel GNU/Linux
What happened ?
A running task is marked as Zombie after the execution time crossed the latest heartbeat + 5 minutes.
The task is running in background in another application server, triggered using SSHOperator.
[2021-01-18 11:53:37,491] {taskinstance.py:888} INFO - Executing <Task(SSHOperator): load_trds_option_composite_file> on 2021-01-17T11:40:00+00:00
[2021-01-18 11:53:37,495] {base_task_runner.py:131} INFO - Running on host: airflow-worker-6f6fd78665-lm98m
[2021-01-18 11:53:37,495] {base_task_runner.py:132} INFO - Running: ['airflow', 'run', 'dsp_etrade_process_trds_option_composite_0530', 'load_trds_option_composite_file', '2021-01-17T11:40:00+00:00', '--job_id', '282759', '--pool', 'default_pool', '--raw', '-sd', 'DAGS_FOLDER/dsp_etrade_trds_option_composite_0530.py', '--cfg_path', '/tmp/tmpge4_nva0']
Task Executing time:
dag_id dsp_etrade_process_trds_option_composite_0530
duration 7270.47
start_date 2021-01-18 11:53:37,491
end_date 2021-01-18 13:54:47.799728+00:00
Scheduler Logs during that time:
[2021-01-18 13:54:54,432] {taskinstance.py:1135} ERROR - <TaskInstance: dsp_etrade_process_etrd.push_run_date 2021-01-18 13:30:00+00:00 [running]> detected as zombie
{
textPayload: "[2021-01-18 13:54:54,432] {taskinstance.py:1135} ERROR - <TaskInstance: dsp_etrade_process_etrd.push_run_date 2021-01-18 13:30:00+00:00 [running]> detected as zombie"
insertId: "1ca8zyfg3zvma66"
resource: {
type: "cloud_composer_environment"
labels: {3}
}
timestamp: "2021-01-18T13:54:54.432862699Z"
severity: "ERROR"
logName: "projects/asset-control-composer-prod/logs/airflow-scheduler"
receiveTimestamp: "2021-01-18T13:54:55.714437665Z"
}
Airflow-webserver log :
X.X.X.X - - [18/Jan/2021:13:54:39 +0000] "GET /_ah/health HTTP/1.1" 200 187 "-" "GoogleHC/1.0"
{
textPayload: "172.17.0.5 - - [18/Jan/2021:13:54:39 +0000] "GET /_ah/health HTTP/1.1" 200 187 "-" "GoogleHC/1.0"
"
insertId: "1sne0gqg43o95n3"
resource: {2}
timestamp: "2021-01-18T13:54:45.401670481Z"
logName: "projects/asset-control-composer-prod/logs/airflow-webserver"
receiveTimestamp: "2021-01-18T13:54:50.598807514Z"
}
Airflow Info logs :
2021-01-18 08:54:47.799 EST
{
textPayload: "NoneType: None
"
insertId: "1ne3hqgg47yzrpf"
resource: {2}
timestamp: "2021-01-18T13:54:47.799661030Z"
severity: "INFO"
logName: "projects/asset-control-composer-prod/logs/airflow-scheduler"
receiveTimestamp: "2021-01-18T13:54:50.914461159Z"
}
[2021-01-18 13:54:47,800] {taskinstance.py:1192} INFO - Marking task as FAILED.dag_id=dsp_etrade_process_trds_option_composite_0530, task_id=load_trds_option_composite_file, execution_date=20210117T114000, start_date=20210118T115337, end_date=20210118T135447
Copy link
{
textPayload: "[2021-01-18 13:54:47,800] {taskinstance.py:1192} INFO - Marking task as FAILED.dag_id=dsp_etrade_process_trds_option_composite_0530, task_id=load_trds_option_composite_file, execution_date=20210117T114000, start_date=20210118T115337, end_date=20210118T135447"
insertId: "1ne3hqgg47yzrpg"
resource: {2}
timestamp: "2021-01-18T13:54:47.800605248Z"
severity: "INFO"
logName: "projects/asset-control-composer-prod/logs/airflow-scheduler"
receiveTimestamp: "2021-01-18T13:54:50.914461159Z"
}
Airflow Database shows the latest heartbeat as:
select state, latest_heartbeat from job where id=282759
--------------------------------------
state | latest_heartbeat
running | 2021-01-18 13:48:41.891934
Airflow Configurations:
celery
worker_concurrency=6
scheduler
scheduler_health_check_threshold=60
scheduler_zombie_task_threshold=300
max_threads=2
core
dag_concurrency=6
Kubernetes Cluster :
Worker nodes : 6
What was expected to happen ?
The backend process takes around 2hrs 30 minutes to finish. During
such long running jobs the task is detected as zombie. Eventhough the
worker node is still processing the task. The state of the job is
still marked as 'running'. State if the task is not known during the
run time.

MQTT Issue while running Thingsboard IOT gateway

I am trying to connect to thingsboard server using thingsboard IoT gateway. I have followed all steps given in the below link : https://thingsboard.io/docs/iot-gateway/install/source-installation/
When running tb-gateway with command : python3 ./thingsboard_gateway/tb_gateway.py
I am getting below error message :
INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected mapping handlers: 0"
INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted serverSideRpc handlers: 2"
INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected serverSideRpc handlers: 0"
INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted connectRequests handlers: 2"
INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected connectRequests handlers: 0"
INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted disconnectRequests handlers: 2"
INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected disconnectRequests handlers: 0"
ERROR - [mqtt_connector.py] - mqtt_connector - 130 - 'attributeRequests' section missing from configuration"
INFO - [mqtt_connector.py] - mqtt_connector - 153 - Number of accepted attributeUpdates handlers: 1"
INFO - [mqtt_connector.py] - mqtt_connector - 157 - Number of rejected attributeUpdates handlers: 0"
INFO - [tb_gateway_service.py] - tb_gateway_service - 135 - Gateway started."
ERROR - [tb_client.py] - tb_client - 132 - [WinError 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond"
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python39\lib\site-packages\thingsboard_gateway-2.5.4-py3.9.egg\thingsboard_gateway\gateway\tb_client.py", line 127, in run
self.client.connect(keepalive=keep_alive,
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python39\lib\site-packages\thingsboard_gateway-2.5.4-py3.9.egg\thingsboard_gateway\tb_client\tb_device_mqtt.py", line 167, in connect
self._client.connect(self.__host, self.__port, keepalive=keepalive)
I have done below changes in the respective files :
mqtt.json :
"broker": {
"name":"Default Local Broker",
"host":"IP Address",
"port":1883,
"clientId": "ThingsBoard_gateway",
"security": {
"type": "basic",
"username": "**********",
"password": "***********"
}
}
tb_gateway.yaml :
host: "IP Address"
port: 1883
remoteShell: false
remoteConfiguration: false
security:
accessToken: ********************
qos: 1
storage:
type: memory
read_records_count: 100
max_records_count: 100000
connectors:
-
name: MQTT Broker Connector
type: mqtt
configuration: mqtt.json
Appreciate your help. Thanks in advance.
Do you have any server ? I gonna explain my method. for example I have a TCP server. But I can not directly connect with my TCP server as a client to thingsboard. But everyone can connect to thingsboard as a mqtt client. Just one role, you need to know your device access token in thingsboard. if you know access token you can connect to thingsboard as a client by using mqtt://demo.thingsboard.io, { username: "ACCESS_TOKEN" } url and v1/devices/me/telemetry topic. example js code that running with node.js. var client = mqtt.connect('mqtt://demo.thingsboard.io' + { username: "AccessToken" }); and to publish client.publish('v1/devices/me/telemetry', JSON.stringify(data));

Advanced AWS CloudFormation - cfn-Init & cfn-Hup not working

I am experimenting with Cloudformation CFN-INIT & CFN-HUP based on below template but the wordpress stack doesn't get created. CFN-HUP process is not started and CFN-Init throws Code1 error. Please see Stack-template and error log details below. Can anyone help me understand what's going wrong here please?
Stack-Template:
**Parameters:
DecideEnvSize:
Type: String
Default: LOW
AllowedValues:
- LOW
- MEDIUM
- HIGH
Description: Select Environment Size (S,M,L)
DatabaseName:
Type: String
Default: DB4wordpress
DatabaseUser:
Type: String
Default: ***************
DatabasePassword:
Type: String
Default: *************
NoEcho: true
TestString:
Type: String
Default: Don't eat yourself up!!!
Mappings:
MyRegionMap:
us-east-1:
"AMALINUX" : "ami-c481fad3" # AMALINUX SEP 2016 - N. Verginia
us-east-2:
"AMALINUX" : "ami-71ca9114" # AMALINUX SEP 2016 - Ohio
InstanceSize:
LOW:
"EC2" : "t2.micro"
"DB" : "db.t2.micro"
MEDIUM:
"EC2" : "t2.small"
"DB" : "db.t2.small"
HIGH:
"EC2" : "t2.medium"
"DB" : "db.t2.medium"
Resources:
DBServer:
Type: "AWS::RDS::DBInstance"
Properties:
AllocatedStorage: 5
StorageType: gp2
DBInstanceClass: !FindInMap [InstanceSize, !Ref DecideEnvSize, DB] # Dynamic mapping + Pseudo Parameter
DBName: !Ref DatabaseName
Engine: MySQL
MasterUsername: !Ref DatabaseUser
MasterUserPassword: !Ref DatabasePassword
DeletionPolicy: Delete
EC2server:
Type: "AWS::EC2::Instance"
DependsOn: DBServer
Properties:
ImageId: !FindInMap [MyRegionMap, !Ref "AWS::Region", AMALINUX] # Dynamic mapping + Pseudo Parameter
InstanceType: !FindInMap [InstanceSize, !Ref DecideEnvSize, EC2]
KeyName: AdvancedCFN
**UserData:
"Fn::Base64":
!Sub |
#!/bin/bash
yum update -y aws-cfn-bootstrap # good practice - always do this.
/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2server --configsets wordpress --region ${AWS::Region}
yum -y update
Metadata:
AWS::CloudFormation::Init:
configSets:
wordpress:
- "configure_cfn"
- "install_wordpress"
- "config_wordpress"
configure_cfn:
files:
/etc/cfn/cfn-hup.conf:
content: !Sub |
[main-just some name]
stack=${AWS::StackId}
region=${AWS::Region}
verbose=true
interval=5
mode: "000400"
owner: root
group: root
/etc/cfn/hooks.d/cfn-auto-reloader.conf:
content: !Sub |
[cfn-auto-reloader-hook #just a name]
triggers=post.update
path=Resources.EC2server.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init -v --stack ${AWS::StackName} --resource EC2server --configsets wordpress --region ${AWS::Region}
mode: "000400"
owner: root
group: root
/var/www/html/index2.html:
content: !Ref TestString
services:
sysvinit:
cfn-hup:
enabled: "true"
ensureRunning: "true"
files:
- "/etc/cfn/cfn-hup.conf"
- "/etc/cfn/hooks.d/cfn-auto-reloader.conf"**
install_wordpress:
packages:
yum:
httpd: []
php: []
mysql: []
php-mysql: []
sources:
/var/www/html: "http://wordpress.org/latest.tar.gz"
services:
sysvinit:
httpd:
enabled: "true"
ensureRunning: "true"
config_wordpress:
commands:
01_clone_config:
cwd: "/var/www/html/wordpress"
test: "test ! -e /var/www/html/wordpress/wp-config.php"
command: "cp wp-config-sample.php wp-config.php"
02_inject_dbhost:
cwd: "/var/www/html/wordpress"
command: !Sub |
sed -i 's/localhost/${DBServer.Endpoint.Address}/g' wp-config.php
03_inject_dbname:
cwd: "/var/www/html/wordpress"
command: !Sub |
sed -i 's/database_name_here/${DatabaseName}/g' wp-config.php
04_inject_dbuser:
cwd: "/var/www/html/wordpress"
command: !Sub |
sed -i 's/username_here/${DatabaseUser}/g' wp-config.php
05_inject_dbpassword:
cwd: "/var/www/html/wordpress"
command: !Sub |
sed -i 's/password_here/${DatabasePassword}/g' wp-config.php
S3blob:
Type: "AWS::S3::Bucket"**
Error & log details
[root#ip-172-31-25-239 ec2-user]# cd /var/log
[root#ip-172-31-25-239 log]# ls
*audit btmp cfn-init-cmd.log cfn-wire.log cloud-init-output.log dmesg lastlog maillog ntpstats spooler wtmp
boot.log cfn-hup.log cfn-init.log cloud-init.log cron dracut.log mail messages secure tallylog yum.log
[root#ip-172-31-25-239 log]# cat cfn-hup.log
2017-12-30 10:48:15,923 [ERROR] Error: [main] section must contain stack option*
===========================================================================================
===========================================================================================
[root#ip-172-31-25-239 log]# cat cfn-init.log
2017-12-30 10:48:15,499 [DEBUG] CloudFormation client initialized with endpoint https://cloudformation.us-east-1.amazonaws.com
2017-12-30 10:48:15,501 [DEBUG] Describing resource EC2server in stack Yetagain-init-hup-try10
2017-12-30 10:48:15,616 [INFO] -----------------------Starting build-----------------------
2017-12-30 10:48:15,616 [DEBUG] Not setting a reboot trigger as scheduling support is not available
2017-12-30 10:48:15,617 [INFO] Running configSets: wordpress
2017-12-30 10:48:15,618 [INFO] Running configSet wordpress
2017-12-30 10:48:15,619 [INFO] Running config configure_cfn
2017-12-30 10:48:15,620 [DEBUG] No packages specified
2017-12-30 10:48:15,620 [DEBUG] No groups specified
2017-12-30 10:48:15,620 [DEBUG] No users specified
2017-12-30 10:48:15,620 [DEBUG] No sources specified
2017-12-30 10:48:15,620 [DEBUG] Parent directory /etc/cfn does not exist, creating
2017-12-30 10:48:15,625 [DEBUG] Writing content to /etc/cfn/cfn-hup.conf
2017-12-30 10:48:15,625 [DEBUG] Setting mode for /etc/cfn/cfn-hup.conf to 000400
2017-12-30 10:48:15,626 [DEBUG] Setting owner 0 and group 0 for /etc/cfn/cfn-hup.conf
2017-12-30 10:48:15,626 [DEBUG] Parent directory /etc/cfn/hooks.d does not exist, creating
2017-12-30 10:48:15,626 [DEBUG] Writing content to /etc/cfn/hooks.d/cfn-auto-reloader.conf
2017-12-30 10:48:15,626 [DEBUG] Setting mode for /etc/cfn/hooks.d/cfn-auto-reloader.conf to 000400
2017-12-30 10:48:15,626 [DEBUG] Setting owner 0 and group 0 for /etc/cfn/hooks.d/cfn-auto-reloader.conf
2017-12-30 10:48:15,626 [DEBUG] Parent directory /var/www/html does not exist, creating
2017-12-30 10:48:15,627 [DEBUG] Writing content to /var/www/html/index2.html
2017-12-30 10:48:15,627 [DEBUG] No mode specified for /var/www/html/index2.html. The file will be created with the mode: 0644
2017-12-30 10:48:15,627 [DEBUG] No commands specified
2017-12-30 10:48:15,627 [DEBUG] Using service modifier: /sbin/chkconfig
2017-12-30 10:48:15,627 [DEBUG] Setting service cfn-hup to enabled
2017-12-30 10:48:15,634 [INFO] enabled service cfn-hup
2017-12-30 10:48:15,635 [DEBUG] Restarting cfn-hup due to change detected in dependency
2017-12-30 10:48:15,635 [DEBUG] Using service runner: /sbin/service
*2017-12-30 10:48:15,941 [ERROR] Could not restart service cfn-hup; return code was 1
2017-12-30 10:48:15,941 [DEBUG] Service output: Stopping cfn-hup: [FAILED]
Starting cfn-hup: [FAILED]
2017-12-30 10:48:15,942 [ERROR] Error encountered during build of configure_cfn: Could not restart cfn-hup*
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 542, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 270, in build
CloudFormationCarpenter._serviceTools[manager]().apply(services, changes)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/service_tools.py", line 161, in apply
self._restart_service(service)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/service_tools.py", line 185, in _restart_service
raise ToolError("Could not restart %s" % service)
ToolError: Could not restart cfn-hup
2017-12-30 10:48:15,942 [ERROR] -----------------------BUILD FAILED!------------------------
2017-12-30 10:48:15,944 [ERROR] Unhandled exception during build: Could not restart cfn-hup
Traceback (most recent call last):
File "/opt/aws/bin/cfn-init", line 171, in <module>
worklog.build(metadata, configSets)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 129, in build
Contractor(metadata).build(configSets, self)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 530, in build
self.run_config(config, worklog)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 542, in run_config
CloudFormationCarpenter(config, self._auth_config).build(worklog)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/construction.py", line 270, in build
CloudFormationCarpenter._serviceTools[manager]().apply(services, changes)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/service_tools.py", line 161, in apply
self._restart_service(service)
File "/usr/lib/python2.7/dist-packages/cfnbootstrap/service_tools.py", line 185, in _restart_service
raise ToolError("Could not restart %s" % service)
ToolError: Could not restart cfn-hup
=============================================================================================================
==============================================================================================================
[root#ip-172-31-25-239 log]# cat /etc/cfn/cfn-hup.conf
[main-just some name]
stack=arn:aws:cloudformation:us-east-1:523324464109:stack/Yetagain-init-hup-try10/908305e0-ed4d-11e7-b9f7-500c285ebefd
region=us-east-1
verbose=true
interval=5
==========================================================================================================
=========================================================================================================
[root#ip-172-31-25-239 log]# cat /etc/cfn/hooks.d/cfn-auto-reloader.conf
[cfn-auto-reloader-hook #just a name]
triggers=post.update
path=Resources.EC2server.Metadata.AWS::CloudFormation::Init
action=/opt/aws/bin/cfn-init -v --stack Yetagain-init-hup-try10 --resource EC2server --configsets wordpress --region us-east-1
Looks like the main error is in cfn-hup.log:
2017-12-30 10:48:15,923 [ERROR] Error: [main] section must contain stack option*
Try changing [main-just some name] to [main] in your cfn-hup.conf. For reference, my /etc/cfn/cfn-hup.conf looks like something like this:
[main]
stack=arn:aws:cloudformation:us-west-1:acccount_id:stack/mystack-dev-ecs-EC2-1VF68LZMOLAIY/cb2a6a80-554a-11e8-b318-503dcab41efa
region=us-west-1
interval=5
verbose=true

When I am trying to execute azure template with extension? I am getting below error?

New - AzureRmResourceGroupDeployment: 5: 33: 31 PM - Resource Microsoft.Compute / virtualMachines / extensions 'dataLin1030Vm/script'
failed with message
{
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [{
"code": "VMExtensionProvisioningError",
"message": "VM has reported a failure when processing extension 'script'. Error message: \"Script returned an
error.\n-- - stdout-- - \n\ n-- - errout-- - \nhg - mysql - mongo - instalation.sh: 6: hg - mysql - mongo - instalation.sh: Syntax error: newline unexpected\ n\ n\ "."
}]
}
}
This:
https://github.com/harish-kumar-garg/JsonTemplates/blob/master/Templates/hg-appAndDbVMCreation.json#L687
Should be pointing to the raw location, e.g.
https://raw.githubusercontent.com/harish-kumar-garg/MyShellScripts/master/Scripts/hg-mysql-mongo-instalation.sh

Resources