We have a big application that uses 40 microservices (Spring boot) that each have about 5 database connections to a mariadb server. That causes too many connection errors on our mariadb server. Default is 151 however I was thinking of just setting the max connections to 1000 to be on the safe side. I cant find anywhere on the Internet if this is possible or even wise. Our MariaDB is running standalone on a VPS with 8GB memory. It is not running in a docker container or something like that. It is run directly on the VPS.
What is the maximum connections advisable taking into consideration that we might scale up with our microservices?
You can scale up your max_connections just fine. Put a line like
max_connections=250
in your MariaDB my.cnf file. But don't just set it to a very high number; each potential connection consumes RAM, and with only 8GiB you need to be a bit careful about that.
If you give this command you'll get a bunch of data about your connections.
SHOW STATUS LIKE '%connect%';
The important ones to watch:
Connection_errors_max_connections The number of connection attempts that failed because you ran out of connection slots.
Connections The total number of connections ever handled
Max_used_connections The largest number of simultaneous connections used.
Max_used_connections_time The date and time when the server had its largest number of connections.
The numbers shown are cumulative since the last server boot or the most recent FLUSH STATUS; statement.
Keep an eye on these. If you run short you can always add more. If you have to add many more connections as you scale up, you probably will need to provision your VPS with more RAM. The last two are cool because you can figure out whether you're getting hammered at a particular time of day.
And, in your various microservices be very careful to use connection pools of reasonable maximum size. Don't let your microservices grab more than ten connections unless you run into throughput trouble. You didn't say what client tech you use (nodejs? dotnet? php? Java?) so it's hard to give you specific advice how to do that.
Related
We are doing a POC to prove that Vitess can scale massively and meet our requirements. We are using around 40 application servers, 15 VTGates and 30 shards (each shard contains master, replica and rdonly). However we were able to scale up to a point and above that point getting a flat line.
Main dark point for us is connecting application server and multiple VTGates. We have tried loadbalancer (AWS nlb) in between them and seen increased QPS but much lower TPS (~15000 QPS, ~ 1500-2000 TPS). Then we tired each application use JDBC connection pooling to connect VTGate without loadbalancer. We got similar results. Then we tried without connection pooling. Then we were able to increase TPS, however saw massive dips in QPS which affect the TPS.
As you can see we have hit a certain roadblock and need few brilliant ideas to overcome this. Really appreciate valuable inputs
I use Moodle on centos7 with Php, Mariadb, Nginx. There are huge number of users that use this Moodle. If the number of users grows more than 300user per sec, the Moodle has delay in response and seems to be hanged!
I read about:
Galera (multi master clustering with 3nodes)
slave-master (separate read and write)
MaxScale
increase ram and cpu (I have up to: 288GB ram, 24coreCPU, SSD drive)
What is the best practice to serve huge number of requests without delay? How can I scale my database (because it is the bottleneck)? I want scale it for serve huge request (most of them is read from database)
MariaDB (and MySQL) can scale 'infinitely' for reads by using Replication and sending read requests to Slave servers.
500 connections per second is very high. (But I don't know what the practical limit is.)
There are several extra tools that can do "connection pooling". Search for this; it may let you go well past 500 logical connections on a single server.
In the case of Galera, you could have 3 read-write nodes, plus any number of Slaves hanging off each of the 3.
For simple Master-Slave, there can be any number of Slaves hanging off the one Master.
Obviously you can do generic MySQL/MariaDB tuning first, and use a recent version of Moodle (3.7 is current right now)
After that, one thing you can check is how you have sessions implemented.
https://docs.moodle.org/37/en/Session_handling
This page also has many more tips:
https://docs.moodle.org/37/en/Performance_recommendations
I installed Mariadb 10.3.15 and Maxscale 2.3.9
It works fine, but sometimes the query speed is slow.
When I look at the status of the service through MaxCtrl, I can see that the value of 'Total Connections' is very high. At first it was small, but the value is still increasing, is this normal state value?
What is the limit of connections from MaxScale? From the clients? What kind of clients -- web servers or what?
My point is that it not MariaDB's problem, it is something upstream.
If "query speed is slow", that sounds like a poorly indexed table or a poorly written query. Set long_query_time=1 and turn on the slowlog.
Total Connections displays the total number of connections made to backends since maxscale start. It is normal it grows as every new connection established is counted.
Connections are the currently connections established to your backends, active (serving queries) + idle. Your backends should be able to handle these (hardware + network + configuration), otherwise it can slowdown things.
https://mariadb.com/kb/en/mariadb-maxscale-24-maxctrl/#list-services
In the DynamoDB documentation and in many places around the internet I've seen that single digit ms response times are typical, but I cannot seem to achieve that even with the simplest setup. I have configured a t2.micro ec2 instance and a DynamoDB table, both in us-west-2, and when running the command below from the aws cli on the ec2 instance I get responses averaging about 250 ms. The same command run from my local machine (Denver) averages about 700 ms.
aws dynamodb get-item --table-name my-table --key file://key.json
When looking at the CloudWatch metrics in the AWS console it says the average get latency is 12 ms though. If anyone could tell me what I'm doing wrong or point me in the direction of information where I can solve this on my own I would really appreciate it. Thanks in advance.
The response times you are seeing are largely do to the cold start times of the aws cli. When running your get-item command the cli has to get loaded into memory, fetch temporary credentials (if using an ec2 iam role when running on your t2.micro instance), and establish a secure connection to the DynamoDB service. After all that is completed then it executes the get-item request and finally prints the results to stdout. Your command is also introducing a need to read the key.json file off the filesystem, which adds additional overhead.
My experience running on a t2.micro instance is the aws cli has around 200ms of overhead when it starts, which seems inline with what you are seeing.
This will not be an issue with long running programs, as they only pay a similar overhead price at start time. I run a number of web services on t2.micro instances which work with DynamoDB and the DynamoDB response times are consistently sub 20ms.
There are a lot of factors that go into the latency you will see when making a REST API call. DynamoDB can provide latencies in the single digit milliseconds but there are some caveats and things you can do to minimize the latency.
The first thing to consider is distance and speed of light. Expect to get the best latency when accessing DynamoDB when you are using an EC2 instance located in the same region. It is normal to see higher latencies when accessing DynamoDB from your laptop or another data center. Note that each region also has multiple data centers.
There are also performance costs from the client side based on the hardware, network connection, and programming language that you are using. When you are talking millisecond latencies the processing time on your machine can make a difference.
Another likely source of the latency will be the TLS handshake. Establishing an encrypted connection requires multiple round trips and computation on both sides to get the encrypted channel established. However, as long as you are using a Keep Alive for the connection you will only pay this overheard for the first query. Successive queries will be substantially faster since they do not incur this initial penalty. Unfortunately the AWS CLI isn't going to keep the connection alive between requests, but the AWS SDKs for most languages will manage this for you automatically.
Another important consideration is that the latency that DynamoDB reports in the web console is the average. While DynamoDB does provide reliable average low double digit latency, the maximum latency will regularly be in the hundreds of milliseconds or even higher. This is visible by viewing the maximum latency in CloudWatch.
They recently announced DAX (Preview).
Amazon DynamoDB Accelerator (DAX) is a fully managed, highly available, in-memory cache for DynamoDB that delivers up to a 10x performance improvement – from milliseconds to microseconds – even at millions of requests per second. For more information, see In-Memory Acceleration with DAX (Preview).
When writing a custom server, what are the best practices or techniques to determine maximum number of users that can connect to the server at any given time?
I would assume that the capabilities of the computer hardware, network capacity, and server protocol would all be important factors.
Also, do you think it is a good practice to limit the number of network connections to a certain maximum number of users? Or should the server not limit the number of network connections and let performance degrade until the response time is extremely high?
Dan Kegel put together a summary of techniques for handling large amounts of network connections from a single server, here: http://www.kegel.com/c10k.html
In general modern servers can handle very large numbers of concurrent connections. I've worked on systems having over 8,000 concurrently open TCP/IP sockets.
You will need a high quality servicing interface to handle that kind of load, check out libevent or libev.
That is a good question and it definitely is situational. What is your computer? Do you have a 4 socket machine filled with Quad Core Xeons, 128 GB of RAM, and Fiber Channel Connectivity (like the pair of Dell R900s we just bought)? Or are you running on a p3 550 with 256 MB of RAM, and 56K modem? How much load does each connection place on your server? What kind of response is acceptible?
These are the questions you need to answer. I guess the best way to find the answer is through load testing. Create a unit test of the expected (and maybe some unexpected) paths that your code will perform against your server. Find a load testing framework that will allow you to simulate 10, 100, 1000, 10000 users performing those tasks at the same time.
That will tell you how many connections your computer can support.
The great thing about the load/unit test scenario is that you can put in response time expectations in your unit tests and increase the load until you fall outside of your response time. If you have a requirement of supporting X number of Users with Y second response, you will be able to demonstrate it with your load tests.
One of the biggest setbacks in high concurrency connections is actually the routers involved. Home user oriented routers usually have a small NAT table, preventing the router from actually servicing the server the connections.
Be sure to research your router/ network infrastructure setup just as well.
I think you shouldn't limit the number of connections your server will allow - just catch and handle properly any exceptions that might occur when accepting and closing connections and you should be fine. You should leave that kind of lower level programming to the underlying OS layers - that way you can port your server easier etc.
This really depends on your operating system.
Different Unix flavors will support "unlimited" number of file handles / sockets others have high values like 32768.
A typical user limit is 8192 but it can usually be set higher.
I think windows is more limiting but the server version may have higher limits.