I've try to setup the simplify cluster for rundeck solution , the simplify mean no need for automatic failover , but the data protection and consistency is the main focus.
The system architecture will be
enter image description here
I'm not sure , is the architecture as I've designed is possible for rundeck community version or not .
The purpose of the architecture is the data protection , some downtime accept with manual failover. The database is MariaDB , any suggestion is welcome.
Thanks in advance
The best way to use Rundeck on Cluster environments is through Rundeck Enterprise (with Autotakeover, Remote Job Execution, and Replication capacities out of the box), you can check the architecture here. Take a look at this, this video and this is useful to see how to works.
Related
I tend to run simulations on a cluster that produces files larger than 100MB and I can't sync my computer with the cluster. So I considered setting up rsync between the two by following this link.
However, I believe this is just a cron job to sync the backup server with the main server and doesn't work in both directions. What will be the stepwise instructions to set up a bidirectional rsync ?
Both the systems run linux
Rsync isn't really the right tool for this job. You can sort of get it to work, using cron jobs and extremely carefully chosen parameters, but there's significant danger of data loss, especially if you want file deletion to propagate.
I'd recommend a tool like Syncthing for bidirectional sync. You want something that maintains an independent database of what's changed and what hasn't, and real-time updates are nice to have too.
I need to obtain information about my servers running on Linux Redhat, Windows Server 2003 and Windows Server 2008. I have found the SNMP protocol to be useful for my purpose, but I am not clear how to implement it in my application. I don't even know how to get started.
Also, is it possible to monitor the applications/programs running on the servers? It is my first time working with SNMP, so I am finding it extremely difficult to understand how it has to be implemented. I have understood the architecture of SNMP(SNMPManager, SNMPAgent, Trap messages, MIB), but finding it difficult to implement them.
Please guide me in as detailed way as possible and mention links to any tutorials.
SNMP is not so simple as should be... if you want to provide control to your system/resources the best choice is to use an external native tool like:
PRTG:You can use free evaluation sensor
CACTI: open-source and web-based tool
SolarWinds:expensive but powerful choice
After that you can monitor your system using WMI protocol (for Win OS), direct agent installed on it or script.
I havea business need related to a MariaDb instance that should work in a master-slave configuration with failover.
Looking at the documentation I have seen that is possible to conigure a multi- cluster-master (galera) or a simple master slave replica.
Any suggestion to configure master-slave + failover?
Many thanks in advance
Roberto
MySQL/MariaDB master-slave replication is great for handling read-heavy workloads. It's also used as a redundancy strategy to improve database availability, and as a backup strategy (i.e. take the snapshot/backup on the slave to avoid interrupting the master). If you don't need a multi-master solution with all the headaches that brings—even with MySQL Cluster or MariaDB Galera Cluster—it's a great option.
It takes some effort to configure. There are several guides out there with conflicting information (e.g. MySQL vs. MariaDB, positional vs. GTID) and several decision points that can affect your implementation (e.g. row vs. statement binlog formats, storage engine selection), and you might have to stitch various pieces together to form your final solution. I've had good luck with MariaDB 10.1 (GTID, row binlog format) and mixed MyISAM and InnoDB storage engines. I create one slave user on the master per slave, and I don't replicate the mysql database. YMMV. This guide is a good starting place, but it doesn't really cover GTID.
Failover is a whole separate ball of wax. You will need some kind of a reverse proxy (such as MaxScale or HAproxy) or floating IP address in front of your master that can adjust to master changes. (There might be a way to do this client-side, but I wouldn't recommend it.) Something has to monitor the health of the cluster, and when it comes time to promote a slave to the new master, there is a whole sequence of steps that have to be performed. MySQL provides a utility called mysqlfailover to facilitate this process, but as far as I know, it is not compatible with MariaDB. Instead, you might take a look at replication-manager, which seems to be MariaDB's Go-based answer to mysqlfailover. It appears to be a very sophisticated tool.
Master-Slave helps with failover, but does not provide it.
MariaDB Cluster (Galera) does provide failover for most cases, assuming you have 3 nodes.
I was playing with percona xtrabackup innobackupex for incremental backups. It is a cool tool and very efficient and effective for incremental backups. However, i could not help but wonder why doing incremental backups would be any better than just doing a regular mysql master-slave replication, and whenever needed to retrieve point-in-time data, just use the binary log?
What advantages would doing incremental backups have over doing master-slave replication? When should you choose to use over the other?
One disadvantage to using master-slave replication as a backup is that accidentally running data damaging commands like
DROP TABLE users;
would replicate to the slave.
They are solutions to two different problems; master-slave is redundancy and backup is resilience.
The MySQL JDBC driver has the ability to connect to many servers. If you look at the driver options (https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-url-format.html) you will notice that the host option is not only host, but hosts. If you specify the URL to both the master and the slave and something happens to the master the driver will automatically connect to the slave instead.
Backup, on the other hand, is, as was mentioned earlier, a way to recover from either a catastrophic crash (having your backups stored off-site is a must) or recover from a catastrophic mistake -- neither of which is served by a master-slave setup. (Well, technically you could have the slave at a different site but that still does not cover the mistake scenario)
Is there any way to know any information about the Network Interface Cards (NIC) of servers in EC2?
I've tried a lot of commands that typically work in Linux, but seems it's all abstracted out when you try them on EC2 VMs.
Alternately, is there any way to characterize the performance of a NIC on a physical server that is hosting my VM (eg, to measure max throughput)? I was thinking there should be some tools for testing such things on a single server but I couldn't find any! (tools like iperf measure the bandwidth between two machines).
Thanks!
I'm not entirely sure testing the throughput of a nic would do much good since it seems to be variable. There is no official documentation on the subject. If you are service static content, S3 is your best bet. Otherwise use some sort of caching with varnish or something similar that you can scale out incase you are running into bandwidth issues.