Does NATS provide some api for managing its topology, i e create new cluster, combine it in supercluster, create gateway etc.
In docs I see only file configuration, but is there another way for configuring nats?
No, there is no API to handle topology.
Leveraging gossip protocol should ease the process - when adding a new node to the cluster, you don't need to alter the configuration of the current ones, nor let clients know.
https://docs.nats.io/running-a-nats-service/nats_admin/upgrading_cluster
Related
I am new to nginx, and am wondering if it can help me to solve a use-case we've encountered.
I have n nodes,which are reading from from a kafka topic with the same group id, which means that each node has disjoint data, partitioned by some key.
Nginx has no way of knowing apriori which node has data corresponding to which keys. But we can build an API or have a redis instance which can tell us the node given the key.
Is there a way nginx can incorporate third party information of this kind to route requests?
I'd also welcome any answers, even if it doesn't involve nginx.
Nginx has no way of knowing apriori which node has data corresponding to which keys
Nginx doesn't need to know. You would need to do this in Kafka Streams RPC layer with Interactive Queries. (Spring-Kafka has an InteractiveQueryService interface, btw, that can be used from Spring Web).
If you want to present users with a single address for the KStreams HTTP/RPC endpoints, then that would be a standard Nginx upstream definition for a reverse proxy, which would route to any of the backend servers, which in-turn communicate with themselves to fetch the necessary key/value, and return the response back to the client.
I have no idea how Kafka partitions
You could look at the source code and see it uses a murmur2 hash, which is available in Lua, and can be used in Nginx.
But again, this is a rabbit hole you should probably avoid.
Other option, use Kafka Connect to dump data to Redis (or whatever database you want). Then write a very similar HTTP API service, then (optionally) point Nginx at that.
I am trying to check the HTTP network map protocol in the sample cordapp example, I have used network bootstrapper to generate node info files. But I need to check the Rest API end points mentioned in the docs: /network-map/publish.
Is it necessary to set up the network map for this? If required,I am not able to get the corda-network map jar file for the community edition. How to access those API endpoints ?
The cordapp samples available in the samples repository uses a locally bootstrapped network. The network bootstrapper is used for this purpose to generate the node-info and the certificates of the network participants. It's a quick and easy way to set up a static Corda network for the purpose of local deployment and testing.
It doesn't have a network map service. The node-info files of each participant is dropped into the additional-node-info folder of the nodes which serves as the network map cache for the node.
The REST end-points you mentioned are available in a real network map service running as part of a dynamic Corda network setup (mostly) used for production deployments.
R3 does not provide a community version of the network map.
However, there is a commercial offering called CENM (https://docs.corda.net/docs/cenm/1.2.html). It's not just a Network Map but also takes care of other aspects of Network Management like Identity and Signing.
In case you are looking for an open-source network map you could try out Cordite Network Map (https://marketplace.r3.com/solutions/cordite-network-map) developed by one of our partners.
There's an IIB HTTP SOAP service exposed to multiple channels - the service has 4 operations and one of them is being consumed very frequently by a particular channel (less than 1 transaction per second).
Is there any way within IBM Integration Bus (broker or service level) to limit number of HTTP requests per channel (IP address) to 1 or n transactions per second?
You could implement it manually using the standard facilities of IIB, but rate-limiting is an API management feature and best implemented using out-of-the-box features of IBM API Connect. It works well with IIB, btw.
As already suggested above, this kind of logic should be done outside of IIB if you need it globaly.
On IIB level, you can configure many things, like the maximum amount of connection, but there's no logic to have this kind of pool for each users.
The best solution, in my opinion, is to use a network component specialized in this kind of logic. On my side, I've decided to implement this rule on the load balancer I have in front of my IIB server. A proxy could probably also do it.
For your specific case, if it is the only case where you need this logic, you can also consider creating different entry point for each application. If this is SOAP, and that the users currently calls /kimbertService/, you can consider having multiple SOAP Input node, with the following routes instead : /kimbertService/App1, /kimbertService/App2, /kimbertService/App3, and then you'll be sure that App1 will never block App2 ...
IIB has a feature of throttling by limiting the number of messages processed through a given message flow per second.
For example, to set the maximumRateMsgsPerSec property for a message flow included in an application, you can use the following sample code:
mqsiapplybaroverride –b BARfile -k applicationName -m sampleFlow#maximumRateMsgsPerSec=100
You can also do it through workload management policies by using the IIB web user interface.
Below is the link:
https://www.ibm.com/support/knowledgecenter/en/SSMKHH_9.0.0/com.ibm.etools.mft.doc/bj58270_.htm
A WORK-A-ROUND SOLUTION
The ideal solution, as others have mentioned, would be to have a API management gateway sitting in front of IIB to manage your API.
Now, a work-around solution could be following:
1) Have your main service flow duplicated, making them two different message flows. These two are your back-end flows performing the same thing but on one of them you can enable throttling.
2) Build a new router IIB flow which takes HTTP requests from consumers. This flow identifies the requester and routes it to the back-end flows accordingly.
Hope this helps.
I am working on a application, which is proposed to be a set of webapps (being called agent), running on tomcat 7, configured on different nodes. I have been tasked, to make these webapps(agent) discover each other automatically. The idea is, that each webapp(say agent X) , once up, will communicate a 'request pattern' to all the other webapps. Other webapps(say agents A, B, C) in turn will store this information ('request pattern') and will use these to route any matching request to agent X using http call.
I am looking for some option where in each webapp will have some component listening on particular port, and the agent X while registering itself will send a multicast request to all the nodes on that particular port.
I think apache camel might be useful here.. but I am not sure.
It will be great if some body can tell the technical viability of this approach, or any other suggestions.
My first thought was that you could use apache httpd and the mod_proxy_loadbalancer to balance all requests over the available nodes. You can define different balancers for any kind of agents. Requests will be send to the balancer and the balancer will route it to any available node.
This is more of a messaging than a routing problem. Add Camel if you need complex routing or adapting to legacy protocols.
This looks like a classic publish and subscribe use case. You can do it with any messaging technology. Look at JMS - ActiveMQ is what Camel uses, or AMQP - I've used RabbitMQ very successfully for this, both use the "topic" paradigm for this, a quick search found this as an example: http://jmsexample.zcage.com/index2.html. Or Jabber.
Julian
I've read of an ESB being used as a SOA approach. What are some other approaches?
This is a very broad question, you may want to focus is.
If you are asking regarding approaches that are instead of ESB, then you may consider using direct access to services, instead of using a service bus.
This approach is often used with a directory or lookup service like UDDI to look up service end point location.
When using an ESB, you send the message to the ESB, who 's responsible to route it to the service provider.
When using direct access the client should know in advance the address of the service provider, and he sends the message directly to him.
When using a lookup service, you first query the address of the service provider (like using DNS to lookup IP addresses), and using this address you send the message to the service provider.
Beyond addressing and routing, the ESB may provide other functions that you loose (or have to implement in other way) if you use the direct access approach.
multi cast routing - sending the request to more then one service provider
context based routing - deciding to which service provider we should send the request, based on the content of the request
central logging
central policy enforcement
load balancing \ fault tolerance
format or protocol translation
buffering and asynchronous service invocation
First.... ask yourself which SOA philosophy are you adhering to. If you are in the IBM camp, then there are 4 different products that provide ESB functionality. Each product is optimized for a different scenarios but basically each one does similar functions.
Think.... SOA == a car. IBM is one manufacturer. Different products == different type of cars for different type of drivers.