How can I have notary services with fault tolerance using database oracle? Is there any recommendation?
Thanks!
The use of Oracle RAC to create a HA notary has not been tested. In theory, any database that can be configured to have zero data loss can be used (if the DB doesn't guarantee zero data loss, i.e. it is not strongly consistent and/or doesn't have synchronous replication, data may be lost in failover, allowing double-spends). However, the MySQL HA notary implementation uses some MySQL-specific syntax that may not work with Oracle RAC.
Related
I am using Vault on AWS with the DynamoDB backend. The backend supports HA.
storage "dynamodb" {
ha_enabled = "true"
region = "us-west-2"
table = "vault-data"
}
Reading the HA concept documentation:
https://www.vaultproject.io/docs/concepts/ha.html
To be highly available, one of the Vault server nodes grabs a lock within the data store. The successful server node then becomes the active node; all other nodes become standby nodes. At this point, if the standby nodes receive a request, they will either forward the request or redirect the client depending on the current configuration and state of the cluster -- see the sections below for details. Due to this architecture, HA does not enable increased scalability.
I am not interested in having a fleet of EC2 instances behind a ELB, where only 1 instance behaves like a master and talks to DynamoDB.
I would like to run N Ec2 instances running Vault, that read and write independently from DynamoDB.
Because DynamoDB supports read/write from multiple EC2 instances, I would expect to be able to unseal Vault from multiple instances simultaneously and perform read and write operations. This should work even with ha_enabled = "false", without doing the leader election.
Why this architecture is not suggested in the documentation ? Why it should not work ? Is there any cryptographic limitation that I am missing ?
thank you
It is a feature of Vault Enterprise. With it, you can set up a primary cluster and as many "secondary" clusters, better known as performance replicas. Each cluster has its own storage and unseal mechanism. So you could have one cluster on Dynamo DB and the other on Raft. If both are on Dynamo DB, then you'll need a Dynamo DB table for each.
But keep in mind that performance replicas will always forward write operations to the primary cluster. A write operation is something that affect the global state of Vault. In that sense a POST to /transit is not considered a write operation.
Another possibility is to have your kv store mounted locally (with the -local flag). Then it will behave like a primary even when mounted on a performance replica, at the price of not being able to replicate to the other cluster.
A final note: DR clusters are an exact copy of any cluster. Each cluster, whether a primary or a replica, can have its DR cluster.
I havea business need related to a MariaDb instance that should work in a master-slave configuration with failover.
Looking at the documentation I have seen that is possible to conigure a multi- cluster-master (galera) or a simple master slave replica.
Any suggestion to configure master-slave + failover?
Many thanks in advance
Roberto
MySQL/MariaDB master-slave replication is great for handling read-heavy workloads. It's also used as a redundancy strategy to improve database availability, and as a backup strategy (i.e. take the snapshot/backup on the slave to avoid interrupting the master). If you don't need a multi-master solution with all the headaches that brings—even with MySQL Cluster or MariaDB Galera Cluster—it's a great option.
It takes some effort to configure. There are several guides out there with conflicting information (e.g. MySQL vs. MariaDB, positional vs. GTID) and several decision points that can affect your implementation (e.g. row vs. statement binlog formats, storage engine selection), and you might have to stitch various pieces together to form your final solution. I've had good luck with MariaDB 10.1 (GTID, row binlog format) and mixed MyISAM and InnoDB storage engines. I create one slave user on the master per slave, and I don't replicate the mysql database. YMMV. This guide is a good starting place, but it doesn't really cover GTID.
Failover is a whole separate ball of wax. You will need some kind of a reverse proxy (such as MaxScale or HAproxy) or floating IP address in front of your master that can adjust to master changes. (There might be a way to do this client-side, but I wouldn't recommend it.) Something has to monitor the health of the cluster, and when it comes time to promote a slave to the new master, there is a whole sequence of steps that have to be performed. MySQL provides a utility called mysqlfailover to facilitate this process, but as far as I know, it is not compatible with MariaDB. Instead, you might take a look at replication-manager, which seems to be MariaDB's Go-based answer to mysqlfailover. It appears to be a very sophisticated tool.
Master-Slave helps with failover, but does not provide it.
MariaDB Cluster (Galera) does provide failover for most cases, assuming you have 3 nodes.
How can we compare Amazon DynamoDB to Couchbase? Which is better in terms of performance, scalability, availability and fault tolerance ?
Thanks for the help.
Regards,
Vinay
This is too generic & strategical question to be part of stackoverflow and the answer could be either of it depending on use case. However I will just let you know what should be considered while deciding.
Remember Couchbase is a NoSQL software but DynamoDB is a NoSQL database as a service. So comparing them may be not a a great idea.
Couchbase can be hosted in-memory and be blazing fast if hosted in local network inside the datacentre. For DynamoDB you have to host your application in EC2 otherwise network lag would be a bottle neck.Most of the time you may not notice a lag as they promise 2 digit latency i.e. less than 99ms. Advantage of dynamo DB is that you don't manage it for backup and recovery . For couchbase in case you host it it could be a pain while upgrades or downtime when you have to take care of backups and version compatibility of data and balancing data across nodes in cluster.
I'm developing an application that works distributed, and I have a SQLite database that must be shared between distributed servers.
If I'm in serverA, and change sqlite row, this change must be in the other servers instantly, but if a server were offline and then it came online, it must update all info equal other servers.
I'm trying to develop a HA service with small SQLite databases.
I'm thinking on something like MongoDB or ReThinkDB, due to replication works fine and I have got data independently server online I had.
There are a library or other SQL methodology to share data between servers?
I used the Raft consensus protocol to replicate my SQLite database. You can find the system here:
https://github.com/rqlite/rqlite
Here are some options:
LiteReplica:
It supports master-slave replication for SQLite3 databases using a single master (writable node) and one or many replicas (read-only nodes).
If a device went offline and then it came online, the secondary/slave dbs are updated with the primary/master one incrementally.
LiteSync:
It implements multi-master replication so we can write to the db in any node, even when the device is off-line.
On both we open the database using a modified URI, like this:
“file:/path/to/app.db?replica=master&bind=tcp://0.0.0.0:4444”
AergoLite:
Blockchain based, it has the highest level of security. Stores immutable relational data, secured by a distributed consensus with low resource usage.
Disclosure: I am the author of these solutions
You can synchronize SQLite databases by embedding SymmetricDS in your application. It supports occasionally connected clients, so it will capture changes and sync them when a server comes online. It supports several different database platforms and can be used as a library or as a standalone service.
You can also use CopyCat, which support SQLite as well as a few other database types.
Marmot looks good:
https://github.com/maxpert/marmot
From their docs:
What & Why?
Marmot is a distributed SQLite replicator with leaderless, and eventual consistency. It allows you to build a robust replication between your nodes by building on top of fault-tolerant NATS Jetstream. This means if you are running a read heavy website based on SQLite, you should be easily able to scale it out by adding more SQLite replicated nodes. SQLite is probably the most ubiquitous DB that exists almost everywhere, Marmot aims to make it even more ubiquitous for server side applications by building a replication layer on top.
I'm using JDBC in my application with business logic(client). This JDBC connects to the database which is in another machine(server). In this case, my JDBC directly connects with the database and stores & retrieves data. This is TWO-TIER architecture right?
In another application, for example servlet programming, I'm simply having browser in my client machine which is the presentation Layer(Client tier). Let me consider my business logic as Application Layer(Second tier) and database as Data layer(Third tier). Still I'm using JDBC to connect my application(business logic) with the database. Second and third tiers reside at server now.
By the above example, in three tier architecture a browser only added additionally and kept my business logic at server. I'm not feeling any performance difference other than these. If I'm wrong please correct me and explain me the exact architecture of 2-tier and 3-tier with other examples. Thanks in advance dear friends.
What you say is right.
You first example is two-tier.
The second example is three-tier.
A three-tier architecture can represent an important performance gain if the link between browser and server is slower than the link between server and DBMS. This is because usually the business logic needs to make several calls to the DBMS and/or present to the user only a small part of the information returned by the DBMS. Having the business logic in the client while having a slow connection to the DBMS would represent an important performance penalty.
In a typical web scenario, the connection between client and server is usually several times slower than the connection between server and DBMS, and there is your performance gain.