We have two clusters, one is active, the other is passive, in case they are switched from one to another, our application will be notified in this case to change the cluster and the hosts accordingly. So is it possible to do the switch on live and how please?
Related
I am using AWS ElastiCache for Redis as the caching solution for my spring-boot application. I am using spring-boot-starter-data-redis and jedis client to connect with my cache.
Imagine that I am having my cache in cluster-mode-enabled and 3 shards with 2 nodes in each. I agree then the best way of doing it is using the configuration-endpoint. Alternatively, I can list all the endpoints of all nodes and let the job done.
However, even if I use a single node's endpoint from one of the shards, my caching solution works. That doesn't looks right to me. I feel even if it works, that might case problems in the cluster in long run. When there are all together 6 nodes partitioned into 3 shards but only using one node's endpoint. I have following questions.
Is using one node's endpoint create an imbalance in the cluster?
or
Is that handled automatically by the AWS ElastiCache for Redis?
If I use only one node's endpoint does that mean the other nodes will never being used?
Thank you!
To answer your questions;
Is using one node's endpoint create an imbalance in the cluster?
NO
Is that handled automatically by the AWS ElastiCache for Redis?
Somewhat
if I use only one node's endpoint does that mean the other nodes will never being used?
No. All nodes are being used.
This is how Cluster Mode Enabled works. In your case, you have 3 shards meaning all your slots (where key-value data is stored) are divided into 3 sub-clusters ie. shards.
This was explained in this answer as well - https://stackoverflow.com/a/72058580/6024431
So, essentially, your nodes are smart enough to re-direct your requests to the nodes that has the key-slot where your data needs to be stored. So, no imbalances. Redis handles the redirection for you.
Now, while using Node endpoints, you're going to be facing other problems.
Elasticache is running on cloud (which is essentially AWS Hardware). All hardware faces issues. You have 3 primaries (1p, 2p, 3p) and 3 (1r, 2r, 3r) replicas.
So, if a primary goes down due to hardware issue (lets say 1p), the replica will get promoted to become the new Primary for the cluster (1r).
Now the problem would be, your application is connected directly to 1p which has now been demoted to replica. So, all the WRITE operations will fail.
And you will have to change the application code manually whenever this happens.
Alternatively, if you were using configurational endpoint (or other cluster level endpoints) instead of node-endpoints, this issue would only be a blip to your application at most, perhaps for 1-2 seconds.
Cheers!
I have set up Akka.net actors running inside an ASP.net application to handle some asynchronous & lightweight procedures. I'm wondering how Akka scales when I scale out the website on Azure. Let's say that in code I have a single actor to process messages of type FooBar. When I have two instances of the website, is there still a single actor or are there now two actors?
By default, whenever you'll call ActorOf method, you'll order creation of a new actor instance. If you'll call that in two actor systems, you'll end up with two separate actors, having the same relative paths inside their systems, but different global addresses.
There are several ways to share an information about actors between actor systems:
When using Akka.Remote you can call actors living on another actor system given their addresses or IActorRefs. Requirements:
You must know the path to a given actor.
You must know the actual address (URL or IP) of an actor system, on which that actor lives.
Both actor systems must be able to communicate via TCP between actor system (i.e. open ports on firewall).
When using Akka.Cluster actor systems (also known as nodes) can form a cluster. They will exchange information about their localization in the network, track incoming nodes and eventually detect a dead or unreachable ones. On top of it, you can use higher level components i.e. cluster routers. Requirements:
Every node must be able to open TCP channel to every other (so again, firewalls etc.)
A new incoming node must know at least one node that is already part of the cluster. This is easily achievable as part of the pattern known as lighthouse or via plugins and 3rd party services like consul.
All nodes must have the same name.
Finally, when using cluster configuration you can make use of Akka.Cluster.Sharding - it's essentially a higher level abstraction over actor's location inside the cluster. When using it, you don't need to explicitly tell, where to find or when to create an actor. Instead, all you need is a unique actor identifier. When sending a message to such actor, it will be created ad-hoc somewhere in the cluster if it didn't exist before, and rebalanced to equally spread the workload in cluster. This plugin also handles all logic associated with routing the message to that actor.
I want to create a webapp with the microservices approach in mind.
It will consist of a set of independent services made as Erlang applications, that I will be able to start and stop separately. In front of them there will be nginx server working as a reverse proxy.
Should I put all these services under one god supervisor or should I keep them separate having a supervisor for each?
Based on what facts should I make this decision?
By making each service an Erlang application, you already have the answer: each application has a separate supervision tree.
In my mind the answer depends on the ability (or necessity) to release your apps in run independently.
If the services need to stay consistent due to interfaces that are likely to change, it will be easier to have one, or a limited number, of supervision tree (and application(s). The start and stop of services is then an interface of your application(s).
If your services are independent, or if they have very stable interfaces (using a standard for example), or if you don't care about releasing in run, then an application per service seems to be a preferred solution.
Supervisors are there with well defined goal in mind: to give you control over the way your state is managed in presence of errors.
If you have two parts (A and B) of application. Each stores some state: SA and SB. You can decide how to construct your supervision tree basing on one question: does this part of state make sense without other part of state? Can my app continue to work correctly if A is restarted and SA is cleared but B is not restarted and SB is not cleared?
When looking at supervisor, people often think about restarts and rate of restarts. Arguably, this is less important part of supervisor. What you should really care is what is started and in which order. What should be restarted and in which order?
Supervisor gives three strategies for restarts.
one_for_one - this assumes that states of managed children are independent. Default and least useful of all of them.
one_for_all - if one of children dies, state of other children have to be cleared as well
rest_for_one - children on the left should not be restarted, children on the right should be restarted.
Other concept that I really like is "error kernel". In case of rest_for_one children on the left are your "error kernel".
One more important thing you have to know is that supervisor starts all children in order left-to-right, depth-first. And stops them in reverse order.
I started exploring openHAB for my home automation. Looks to be a great application for the home automation. I want to automate two homes and want to run openHAB on one centrally placed server. Is it possible to segregate the data for my two homes and provide use based access for two homes.
Or I will have to have to instances running on my server.
Please suggest if anyone has done this earlier.
you can (I believe) provide different sitemaps, but the most importing question is how will a central openhab instance communicate with the "other" home?
Especially If you're going to use bindungs which require a piece of hardware like z-wave etc.
You can potentially play with MQTT and have a small Raspberry Pi running in the "other" home feeding the MQTT.
Assuming that there is not a hardware or range based issue with using OpenHab for two homes (e.g. z-wave USB dongle but second home is out of range) and there is network connectivity between the two houses, there are a number of ways you can accomplish this. Here is one.
The easiest would probably be to just use a naming convention for your items and groups to easily tell which house the item comes from. You would probably want to set up a separate sitemap for each house as well. If I understand your question this should segregate the data for you based on name and provide use based access for each home.
If you want to segregate the data even more thoroughly you can configure your persistence to save all the items from one house to one DB and all the others to a different one, though you will need two different persistence bindings set up (i.e. one uses rrd4j and another uses db4o). I'm not sure this provides any advantage.
The final step is getting the data from the remote house into openHab. How this is accomplished will depend on the nature of the sensors and triggers in the other house. You can use the HTTP binding, TCP/IP binding, or an MQTT broker. I've personally exposed a couple of my Raspberry Pi based sensors to openHAB using a python script and the paho library that publishes the sensor data read from the GPIO pins to an MQTT broker and it works great.
Centralization vs segregation - you have to decide, which one has more advantages and less risk.
The two houses will store data on the server (openhab2, mqtt, DB/rr4d) and each one have access to it - that must be clarified.
The network connectivity is obvious, it should be stable between the two sites. Security is another issue - not only digital, but safety of life (hvac controlling or safety appliances with network outage?).
Configuration is pretty supported in both ways, separate config files (items, rules, persistence, etc) and connectivity in a hierarchy has endless approaches and capabilities.
In the newest version of the android app you can add multiple openhab servers. Why don't just use two instances of openHAB?
What is the right approach to use to configure OpenSplice DDS to support 100,000 or more nodes?
Can I use a hierarchical naming scheme for partition names, so "headquarters.city.location_guid_xxx" would prevent packets from leaving a location, and "company.city*" would allow samples to align across a city, and so on? Or would all the nodes know about all these partitions just in case they wanted to publish to them?
The durability services will choose a master when it comes up. If one durability service is running on a Raspberry Pi in a remote location running over a 3G link what is to prevent it from trying becoming the master for "headquarters" and crashing?
I am experimenting with durability settings in a remote node such that I use location_guid_xxx but for the "headquarters" cloud server I use a Headquarters
On the remote client I might to do this:
<Merge scope="Headquarters" type="Ignore"/>
<Merge scope="location_guid_xxx" type="Merge"/>
so a location won't be master for the universe, but can a durability service within a location still be master for that location?
If I have 100,000 locations does this mean I have to have all of them listed in the "Merge scope" in the ospl.xml file located at headquarters? I would think this alone might limit the size of the network I can handle.
I am assuming that this product will handle this sort of Internet of Things scenario. Has anyone else tried it?
Considering the scale of your system I think you should seriously consider the use of Vortex-Cloud (see these slides http://slidesha.re/1qMVPrq). Vortex Cloud will allow you to better scale your system as well as deal with NAT/Firewall. Beside that, you'll be able to use TCP/IP to communicate from your Raspberry Pi to the cloud instance thus avoiding any problem related to NATs/Firewalls.
Before getting to your durability question, there is something else I'd like to point out. If you try to build a flat system with 100K nodes you'll generate quite a bit of discovery information. Beside generating some traffic, this will be taking memory on your end applications. If you use Vortex-Cloud, instead, we play tricks to limit the discovery information. To give you an example, if you have a data-write matching 100K data reader, when using Vortex-Cloud the data-writer would only match on end-point and thus reducing the discovery information by 100K times!!!
Finally, concerning your durability question, you could configure some durability service as alignee only. In that case they will never become master.
HTH.
A+