I use Moodle on centos7 with Php, Mariadb, Nginx. There are huge number of users that use this Moodle. If the number of users grows more than 300user per sec, the Moodle has delay in response and seems to be hanged!
I read about:
Galera (multi master clustering with 3nodes)
slave-master (separate read and write)
MaxScale
increase ram and cpu (I have up to: 288GB ram, 24coreCPU, SSD drive)
What is the best practice to serve huge number of requests without delay? How can I scale my database (because it is the bottleneck)? I want scale it for serve huge request (most of them is read from database)
MariaDB (and MySQL) can scale 'infinitely' for reads by using Replication and sending read requests to Slave servers.
500 connections per second is very high. (But I don't know what the practical limit is.)
There are several extra tools that can do "connection pooling". Search for this; it may let you go well past 500 logical connections on a single server.
In the case of Galera, you could have 3 read-write nodes, plus any number of Slaves hanging off each of the 3.
For simple Master-Slave, there can be any number of Slaves hanging off the one Master.
Obviously you can do generic MySQL/MariaDB tuning first, and use a recent version of Moodle (3.7 is current right now)
After that, one thing you can check is how you have sessions implemented.
https://docs.moodle.org/37/en/Session_handling
This page also has many more tips:
https://docs.moodle.org/37/en/Performance_recommendations
Related
We have a big application that uses 40 microservices (Spring boot) that each have about 5 database connections to a mariadb server. That causes too many connection errors on our mariadb server. Default is 151 however I was thinking of just setting the max connections to 1000 to be on the safe side. I cant find anywhere on the Internet if this is possible or even wise. Our MariaDB is running standalone on a VPS with 8GB memory. It is not running in a docker container or something like that. It is run directly on the VPS.
What is the maximum connections advisable taking into consideration that we might scale up with our microservices?
You can scale up your max_connections just fine. Put a line like
max_connections=250
in your MariaDB my.cnf file. But don't just set it to a very high number; each potential connection consumes RAM, and with only 8GiB you need to be a bit careful about that.
If you give this command you'll get a bunch of data about your connections.
SHOW STATUS LIKE '%connect%';
The important ones to watch:
Connection_errors_max_connections The number of connection attempts that failed because you ran out of connection slots.
Connections The total number of connections ever handled
Max_used_connections The largest number of simultaneous connections used.
Max_used_connections_time The date and time when the server had its largest number of connections.
The numbers shown are cumulative since the last server boot or the most recent FLUSH STATUS; statement.
Keep an eye on these. If you run short you can always add more. If you have to add many more connections as you scale up, you probably will need to provision your VPS with more RAM. The last two are cool because you can figure out whether you're getting hammered at a particular time of day.
And, in your various microservices be very careful to use connection pools of reasonable maximum size. Don't let your microservices grab more than ten connections unless you run into throughput trouble. You didn't say what client tech you use (nodejs? dotnet? php? Java?) so it's hard to give you specific advice how to do that.
I want to create a load test for a feature of my app. It’s using a Google App Engine and a VM. The user sends HTTP requests to the App Engine. It’s realistic that this Engine gets thousands of requests in a few seconds. So I want to create a load test, where I send 20.000 - 50.000 in a timeframe of 1-10 seconds.
How would you solve this problem?
I started to try using Google Cloud Task, because it seems perfect for this. You schedule HTTP requests for a specific timepoint. The docs say that there is a limit of 500 tasks per second per queue. If you need more tasks per second, you can split this tasks into multiple queues. I did this, but Google Cloud Tasks does not execute all the scheduled task at the given timepoint. One queue needs 2-5 minutes to execute 500 requests, which are all scheduled for the same second :thinking_face:
I also tried a TypeScript script running asynchronous node-fetch requests, but I need for 5.000 requests 77 seconds on my macbook.
I don't think you can get 50.000 HTTP requests "in a few seconds" from "your macbook", it's better to consider going for a special load testing tool (which can be deployed onto GCP virtual machine in order to minimize network latency and traffic costs)
The tool choice is up to you, either you need to have powerful enough machine type so it would be able to conduct 50k requests "in a few seconds" from a single virtual machine or the tool needs to have the feature of running in clustered mode so you could kick off several machines and they would send the requests together at the same moment of time.
Given you mention TypeScript you might want to try out k6 tool (it doesn't scale though) or check out Open Source Load Testing Tools: Which One Should You Use? to see what are other options, none of them provides JavaScript API however several don't require programming languages knowledge at all
A tool you could consider using is siege.
This is Linux based and to prevent any additional cost by testing from an outside system out of GCP.
You could deploy siege on a relatively large machine or a few machines inside GCP.
It is fairly simple to set up, but since you mention that you need 20-50k in a span of a few seconds, siege by default only allows 255 requests per second. You can make this larger, though, so it can fit your needs.
You would need to play around on how many connections a machine can establish, since each machine will have a certain limit based on CPU, Memory and number of network sockets. You could just increase the -c number, until the machine gives an "Error: system resources exhausted" error or something similar. Experiment with what your virtual machine on GCP can handle.
I Just want to understand how we should plan for the capacity of a NiFi instance.
We have a NiFi instance which is having around 500 flows. So, the total number of processors enabled on NiFi canvas is around 4000. We do run 2-5 flows simultaneously which does not take more than half an hour i.e. we do process data in MBs.
It was working fine till now but we are seeing outofMemory error very often. So we increased xms and xmx parameters from 4g to 8g which has resolved the problem for now. But going forward we will have more flows and we may face outofmemory issue again.
So, can anyone help with matrix of capacity planning or any suggestion to avoid such issues before happening? eg:- If we have 3000 processors enabled with/without any processing then Xg amount memory required.
Any input on NiFi capacity planning would be appreciated.
Thanks in Advance.
OOM errors can occur due to specific memory consuming processors. For example: SplitXML is loading your whole record to memory, so it could load a 1GiB file for instance.
Each processors can document what resource considerations should be taken. All of the Apache processors(as far as I can tell) are documented in that matter so you can rely on them.
In our example, by the way, SplitXML can be replaced with SplitRecord which doesn't load all of the record to memory.
So even if you use 1000 processors simultaneously, they might not consume as much memory as one processor that loads your whole FlowFile's content to memory.
Check which processors you are using and make sure you don't use one like that(there are more like this one that load the whole document to memory).
I am trying to make a simple general purpose multi-threaded async downloader in python.How many parallel connections can be generally be made to a server with minimum risk of being banned or rate limited.
I am aware that network will be a limiting in some cases but lets assume in this case that network isn't an issue in this case for the sake of discussion.I/O is also done asynchronously.
According to Browserscope , browsers make a maximum of 17 connections at a time.
However according to my research , most download managers download files in multi-part and make 8+ connections per file.
1.How many files can be downloaded at a time ?
2.How many chunks for a single can be downloaded at one time ?
3.What should be the minimum size of those chunks to make it worth creating the overhead of creating parallel connections ?
It depends.
While some servers tolerate a high number of connections, others don't. General web servers might be more on the high side (low two digit), file hosters might be more sensitive.
There's little to say unless you can check the server's configuration or just try and remember for the next time when your ban has timed out.
You should however watch your bandwidth. Once you max out your access line there's no gain in further increasing the connections.
Is there a way to stimulate 10000 concurrent HTTP request?
I try siege tool
but only have 2000 request limit for my laptop
How can I make 10000 request?
The most simple approach to generate a huge amount of concurrent requests, it probably Apache's ab tool.
For example, ab -n 100 -c 10 http://www.example.com/ would request the given websites a 100 times, with a concurrency of 10 requests.
It is true that the number of simultaneous requests is limited by nature. Keep in mind that TCP only has 65536 available ports, some of which are already occupied and the first 1024 are usually reserved, this leaves you with a theoretical maximum of around 64500 ports per machine for outgoing request.
Then there are the operating system limits. For example, in Linux there are the kernel parameters in the net.ipv4.* group.
Finally, you should of course configure your HTTP server to handle that amount of simultaneous requests. In Apache, those are StartServers and its friends, in nginx it's worker_processes and worker_connections. Also, if you have some stand-alone dynamic processor attached to your webserver (such as php-fpm), you must raise the number of idle processes in the connection pool, too.
After all, the purpose of massive parallel requests should be to find your bottle necks, and the above steps will give you a fair idea.
Btw. if you use ab, read its final report thoroughly. It may seem brief, but it carries a lot of useful information (e.g. "non-2xx responses" may indicate server-side errors due to overload.)
Jmeter allows distributed testing, which means that you can setup up a set of computers (one acting as a master and the rest as slaves) to run as many threads as you need. Jmeter has a very good doc explaining this here . . .
http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf
and some more info here . . .
http://digitalab.org/2013/06/distributed-testing-in-jmeter/
You can set this all up on the cloud as well if you do not have access to sufficient slave machines, there are a couple of services out there for this.
Have you tried using Apache JMeter? You can create a web test plan and there are several options which you can play with. You can wrap the requests in a ThreadGroup as outlined here. You can generate extensive reports and graphs as well. If the simple thread group is not enough you could potentially try using the UltimateThreadGroup plugin for JMeter.
When creating so many threads with JMeter on a single machine you run out of memory to allocate a new stack for a thread. For that you can potentially consider reducing the stack space for the thread. How to do that is explained in the SO answer here. The post has some other alternative approaches as well.
If there isn't an OS limit of the number of simultaneous TCP connections allowed, there is a registry setting that removes or increases that limit. After you made sure that isn't the case, you could write some JavaScript that includes AJAX requests and put it in a loop.
You would probably need node.js to execute the JavaScript.