Can someone help me finding correct application insight query to get Azure Storage Queue No of messages for particular interval .
AFAIK you should use Azure Service Bus if you want to monitor queue lengths. Check out this article https://brettmckenzie.net/2019/10/20/consider-using-service-bus-queues-instead-of-azure-storage-queues-when-using-application-insights/
Related
I have to create 28 stateless services under one service fabric application. The role of each state less service is just to listen to the service bus queue, retrieve the messages and post them to a Rest endpoint. Is there a hard limit on how many stateless services that can be created on a single azure service fabric application? Do we run into any memory issues by having numerous stateless services?
EDIT : We looked on the server itself and saw that each app takes up about 250 MB mem.
Nothing comes for free. Each stateless service will use some memory, also of course largely depending on what it is doing. There is no hard limit but the size of the underlying machine and possibly the scale of the cluster do define the limits. What is the instance count of each service anyway?
I have an application that is being monitored by AppDynamics. The application consumes from a JMS queue and publishes messages to a JMS topic.
I would like to create a widget that displays the number of messages it is consuming from the queue and the number of messages it is publishing to the topic.
How do I identify the data source of the queue and topic then retrieve the number of messages that the application is consuming and then publishing?
For the consumption of messages, these can be instrumented as Business Transactions (see https://docs.appdynamics.com/appd/22.x/latest/en/application-monitoring/configure-instrumentation/transaction-detection-rules/message-queue-entry-points) - then the BT count will be the number of messages consumed.
For the publishing to a topic, these will be represented as exit calls to a backend (assuming this is not instrumented) - then the backend (a.k.a Remote Service) call count will be the number of pushes to the topic.
For actually directly getting the number of messages on a queue, you may want to use a Machine Agent extension:
e.g.
WebSphere MQ: https://developer.cisco.com/codeexchange/github/repo/Appdynamics/websphere-mq-monitoring-extension
ActiveMQ: https://developer.cisco.com/codeexchange/github/repo/Appdynamics/activemq-monitoring-extension
RabbitMQ: https://developer.cisco.com/codeexchange/github/repo/Appdynamics/rabbitmq-monitoring-extension
Docs on using extensions can be found here: https://docs.appdynamics.com/appd/22.x/latest/en/infrastructure-visibility/machine-agent/extensions-and-custom-metrics
ingesting data to kakfka cluster that can send data to adx kusto db by using kafka-sink-azure-kusto .
iam successfully ingesting data to kafka cluster and its not transferring data to kusto db. how to debug this? any logs i can verify.
i have tried to check broker log no errors there
ref:https://github.com/Azure/kafka-sink-azure-kusto/blob/master/README.md
Could you please provide more information about how you are running Kafka, and how did you set up the connector?
Debugging steps would be:
Broker logs should mention that connector was picked up properly, did you see that line in the logs?
Looking at Connector logs should show more info about what is actually going on under the hood. Maybe you will see some errors there. /var/log/connect-distributed.log
Try to ingest data via any other method? like one of the SDK's
Try running the setup according to steps detailed under delpoy
Update: more info about connector setup in general can be found at this SO question: Kafka connect cluster setup or launching connect workers
Also, confluent has some helpful docs:https://docs.confluent.io/current/connect/userguide.html
Background-
I am extremely new to CORDA and Blockchain Platform. In the past few months i have had my share of experience working on a small project on Ethereum as platform. Ethereum blockchain was leveraged as ledger to mark Transaction as a proof of existence. It means for some action (success/failure) we have marked respective transaction on Blockchain. We may consider it as a proof of concept to show knowledge of interaction with nodes running on Ethereum Blockchain.
Infrastructure - Node.js web services, two ethereum (PoA) nodes
Question-
I would now like to port this running example on to CORDA blockchain. How would i achieve this with bare minimum changes. That means if i have a Corda network with two nodes running on my system and i want my web services to connect to one or both of the nodes and save the transaction (in its state). I understand that this certainly is not what CORDA might be meant for. Consider this question as my first step to interact with CORDA from Node.js web services.
Any inputs highly appreciated.
I recommend you go through the documentation first. your Tx will be a state. you need to build contracts and flows for a Tx to happen. Tx will happen using flows which will be initiated using Crash Shell or RPC Client. AFAIK, this client is in Kotlin or Java. so you'll have to create a JAVA or Kotlin application to instantiate this client. now in the Java application, you'll have to expose rest endpoints to communicate with the client which will initiate your flows. you can call these rest endpoints from your node application. All this has to be created in CodaApp. This is the bare minimum.
I just found there is a library.
look at this: https://gitlab.com/bluebank/braid
This can help you.
As per the KAA references, I understand that once should only use the Notification feature, When it required to send data from server (External apps) to endpoints and Events are only used when there is a need for endpoint to endpoint communication (kind of device binding requirement)
So, To achieve request/response functionality using KAA. I need to implement any hybrid solutions like as below.
1) In my server, I can run one KAA SDK instance and use the event feature for request to the endpoint and response from the endpoint.
OR
2) From my server, I use the notification REST API for request and get the response back through the data logger feature using any in-build appender by configuring "LogUploadStrategy" as to uploads every log record as soon as it is created.
Notes For Point 1
As per Andrew, Solutions Architect of Kaa IoT platform
"You can always embed an SDK to a standalone application and host in
on the same server where kaa-node is present. This application may
receive REST API calls and forward them to particular endpoints via
Kaa events feature. However, this is useful for test purposes. I
would not recommend this solution in production because it is hard to
scale and has potential security issues"
Notes For Point 2
It satisfies the KAA reference document as well as Andrew's suggestion for request only but how can i achieve the response.
Questions For Point 1
1) What causes to scale the application and what type of security issues it faces even through it uses RSA 2048 encryption for communication?
2) Can we embed more then one SDK in standalone application and host in on the same server where kaa-node is present.
Questions For Point 2
3) if device sends the notification response along with the telemetry data, can it increase the latency and any other performance issue.
Common Questions
4) Which one is the better approach to achieve request/response functionality?
Any help or suggestion is really appreciated.
1) What causes to scale the application and what type of security issues it faces even though it uses RSA 2048 encryption for communication?
It makes the EP on the server side as a single point of failure and does not allow load balancing.
About security issues, Andrew meant: This application may receive REST API calls and this forces one to provide additional security for this REST API calls and better use your first hybrid solutions using solely event feature.
2) Can we embed more then one SDK in standalone application and host in on the same server where kaa-node is present.
No, you can't use more than one SDK in one application, but you can run a couple of instance on one machine in different directories
in order to prevent collisions of autogenerated security keys and other files.
3) if device sends the notification response along with the telemetry data, can it increase the latency and any other performance issue.
Of course, you will face some delays if start sending very frequently and big portions of data on both sides. If you have a lot of devices that sends in total a big amount of telemetry data, you can increase performance on the server side by start-up KAA in the cluster mode or add new nodes for processing requests.
4) Which one is the better approach to achieve request/response functionality?
The second hybrid solution – data collection and notification features. This doesn't cause any problem with scale and you can easily launch Kaa server in cluster mode.