Using xcache 3.1.0 on php 5.4.24 on a dual hexacore machine (12 cores):
[~]# cat /proc/cpuinfo |grep -c processor
12
I have entered 12 in the count property of the xcache object in the php.ini and the webadmin agrees:
However, I am getting 16 slots for php caching, all of which are being used:
Can anyone explain this? Is this supposed to happen?
One quick explanation to this situation might be that xcache is rounding 12 number to 16.
Reading your post, I also changed the xcache.count for my installation from 4 to 12, when I tested it, I got 16 threads instead of 12. I set it to 6, it rounded to 8 so I'm pretty sure xcache.count needs to be set to the power of 2.
Related
i have upgraded from mariadb 10.1.36 to 10.4.8 and i can see mysterious increasing ram ussage on that new version. I also edited innodb_buffer_pool_size ant seems there is no effect if its set to 15M or 4G, ram is just slowly increasing. After while it eat whole ram and oom killer kills mariadb and this is repeating.
My server has 8GB RAM and its increasing like 60-150MB per day. Its not terrible but i have around 150 database servers so its huge problem.
I can temporary fix problem by restarting mariadb and its start again.
Info about database server:
databases: 200+
tables: 28200(141 per database)
average active connections: 100-200
size of stored data: 100-350GB
cpu: 4
ram: 8GB
there is my config:
server-id=101
datadir=/opt/mysql/
socket=/var/lib/mysql/mysql.sock
tmpdir=/tmp/
gtid-ignore-duplicates=True
log_bin=mysql-bin
expire_logs_days=4
wait_timeout=360
thread_cache_size=16
sql_mode="ALLOW_INVALID_DATES"
long_query_time=0.8
slow_query_log=1
slow_query_log_file=/opt/log/slow.log
log_output=TABLE
userstat = 1
user=mysql
symbolic-links=0
binlog_format=STATEMENT
default_storage_engine=InnoDB
slave_skip_errors=1062,1396,1690innodb_autoinc_lock_mode=2
innodb_buffer_pool_size=4G
innodb_buffer_pool_instances=5
innodb_log_file_size=1G
innodb_log_buffer_size=196M
innodb_flush_log_at_trx_commit=1
innodb_thread_concurrency=24
innodb_file_per_table
innodb_write_io_threads=24
innodb_read_io_threads=24
innodb_adaptive_flushing=1
innodb_purge_threads=5
innodb_adaptive_hash_index=64
innodb_flush_neighbors=0
innodb_flush_method=O_DIRECT
innodb_io_capacity=10000
innodb_io_capacity_max=16000
innodb_lru_scan_depth=1024
innodb_sort_buffer_size=32M
innodb_ft_cache_size=70M
innodb_ft_total_cache_size=1G
innodb_lock_wait_timeout=300
slave_parallel_threads=5
slave_parallel_mode=optimistic
slave_parallel_max_queued=10000000
log_slave_updates=on
performance_schema=on
skip-name-resolve
max_allowed_packet = 512M
query_cache_type=0
query_cache_size = 0
query_cache_limit = 1M
query_cache_min_res_unit=1K
max_connections = 1500
table_open_cache=64K
innodb_open_files=64K
table_definition_cache=64K
open_files_limit=1020000
collation-server = utf8_general_ci
character-set-server = utf8
log-error=/opt/log/error.log
log-error=/opt/log/error.log
pid-file=/var/run/mysqld/mysqld.pid
malloc-lib=/usr/lib64/libjemalloc.so.1
I solved it! The problem is memory allocation library.
If you do this SQL query:
SHOW VARIABLES LIKE 'version_malloc_library';
You must to get value "jemalloc" library. If you get only "system", you may have problems.
To change that, you need edit any .conf file in this directory:
/etc/systemd/system/mariadb.service.d/
There, add this line:
Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so.1"
(this library file may be in other folder)
Then you must to restart mysqld
service mysqld stop && systemctl daemon-reload && service mysqld start
You got carried away in increasing values in my.cnf.
Many of the caches grow until hitting their limit, hence the memory growth you experienced.
What is the value from SHOW GLOBAL STATUS LIKE 'Max_used_connections';? Having a large max_connections accentuates several of the other values; lower it.
But perhaps the really bad one(s) involve table caches -- which have units of tables, not bytes. Crank these down a lot:
table_open_cache=64K
innodb_open_files=64K
table_definition_cache=64K
I have exactly the same problem. Is it due to a bad configuration? Or is it a bug of the new version?
mariadb 10.1 was updated to 10.3 just when I upgraded Debian 9 to Debian 10. I tried solve the problem with mariadb 10.4 but nothing changed.
I want to downgrade version but I think it's neccesary dump all database and restore it, and that means being hours without service.
I don't think Debian 10 has to do with the issue
Please read my previous comments about alternative memory allocators...
When jemalloc is used:
When default memory allocator used:
Try with tcmalloc
Environment="LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libtcmalloc_minimal.so.4.5.3.1"
I'm using RocksDB via the C API.
I have a test program that opens a database, does 1,000 writes (gathering timing data between initiation of write and callback), does 1,000 reads, and shuts down.
This works. Average time to do a write is about 1ms.
I modified the test program to turn on write syncing via this
rocksdb_writeoptions_set_sync(wri_u, 1);
and ran it again. Average time to do a write is about 8ms.
So far, so good.
HOWEVER, I then ran strace on both versions of the program to verify that fsync() or fdatasync() or msync() is getting called.
The no-sync program shows 4 invocations of fsync(), 2 of fdatasync() and 0 of msync(). Reasonable.
...but the sync version of the program shows the same 4, 2, and 0. Odd! Surprising! Worrying!
The sync version DOES show 2 interesting deltas from the no-sync version: (i) 2 calls to nanosleep() per write, (ii) an 80% increase in the time spent in mmap().
One out-of-my-butt theory is that perhaps msync() [ or a stand-in for it ] is actually implemented in terms of nanosleep() ?
This is on a desktop Linux 16.04
uname -a
Linux mithril 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Anyway, my question is, as per the subject line:
Am I properly forcing RocksDB to use fsync? ... because neither fsync() nor msync() shows in strace
Thanks.
Yes, this is the correct way to turn fsync() on.
The issue is that strace must be used with the -f flag to trace system calls in new threads ... and RocksDB was doing all syncs in other threads.
I am using percona-toolkit for analysing mysql-slow-query (logs). So the command is pretty basic:
pt-query-digest slowquery.log
Now the result(error) is:
18.2s user time, 100ms system time, 35.61M rss, 105.19M vsz
Current date: Thu Jul 7 17:18:43 2016
Hostname: Jammer
Files: slowquery.log
Pipeline process 5 (iteration) caused an error: Redundant argument in sprintf at /usr/bin/pt-query-digest line 2556.
Will retry pipeline process 4 (iteration) 2 more times.
..
..(same result prints twice)
..
The pipeline caused an error: Pipeline process 5 (iteration) caused an error: Redundant argument in sprintf at /usr/bin/pt-query-digest line 2556.
Terminating pipeline because process 4 (iteration) caused too many errors.
Now the specifics for the environment, I am using Ubuntu 16.04 , MariaDB 10.1.14, Percona-Toolkit 2.2.16
I found something here bug-report, but it is like a workaround and does not actually solve the error. Even after applying the patch the command result doesn't look satisfying enough.
I am facing same problem on ubuntu 16.04 MySql.
The contents of my slow query log is as follow.
/usr/sbin/mysqld, Version: 5.7.16-0ubuntu0.16.04.1-log ((Ubuntu)). started with:
Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
Time Id Command Argument
/usr/sbin/mysqld, Version: 5.7.16-0ubuntu0.16.04.1-log ((Ubuntu)). started with:
Tcp port: 3306 Unix socket: /var/run/mysqld/mysqld.sock
Time Id Command Argument
Time: 2016-12-08T05:13:55.140764Z
User#Host: root[root] # localhost [] Id: 20
Query_time: 0.003770 Lock_time: 0.000200 Rows_sent: 1 Rows_examined: 2
SET timestamp=1481174035;
SELECT COUNT(*) FROM INFORMATION_SCHEMA.TRIGGERS;
The error is same:
The pipeline caused an error: Pipeline process 5 (iteration) caused an
error: Redundant argument in sprintf at /usr/bin/pt-query-digest line 2556.
Ubuntu 16.04
MySql Ver 14.14 Distrib 5.7.16
pt-query-digest 2.2.16
The bug appears to be fixed in the current version of the toolkit (2.2.20), and apparently in previous ones, starting from 2.2.17.
This patch seems to do the trick for this particular place in pt-query-digest:
--- percona-toolkit-2.2.16/bin/pt-query-digest 2015-11-06 14:56:23.000000000 -0500
+++ percona-toolkit-2.2.20/bin/pt-query-digest 2016-12-06 17:01:51.000000000 -0500
## -2555,8 +2583,8 ##
}
return sprintf(
$num =~ m/\./ || $n
- ? "%.${p}f%s"
- : '%d',
+ ? '%1$.'.$p.'f%2$s'
+ : '%1$d',
$num, $units[$n]);
}
But as mentioned in the original question and bug report, quite a few tools/functions were affected, the full bugfix consisted of a lot of small changes:
https://github.com/percona/percona-toolkit/pull/73/files
I might be late here. I want to share how I overcame that same error as it might help someone who is searching for an answer. At this time the latest tag of Percona toolkit is 3.0.9
I tried to run pt-query-digest after installing via apt, by downloading deb file as methods provided by Percona documentation, but any of it didn't help. It was this same error.
Pipeline process 5 (iteration) caused an error:
Redundant argument in sprintf at /usr/bin/pt-query-digest line (some line)
1 - So I deleted/removed the installation of percona-toolkit
2 - first, I cleaned/updated perl version
sudo apt-get install perl
3 - then I installed Percona toolkit from source as mentioned in the repository's readme. like this. I used branch 3.0.
git clone git#github.com:percona/percona-toolkit.git
cd percona-toolkit
perl Makefile.PL
make
make test
make install
Thats it. Hope this help to someone.
i found error in this version percona-toolkit-3.0.12-1.el7.x86_64.rpm
and percona-toolkit-3.0.10-1.el7.x86_64.rpm is fine, percona-toolkit is very useful to me
at ./pt-query-digest line 9302.
Terminating pipeline because process 4 (iteration) caused too many errors.
Note that you will see the error message:
"Redundant argument in sprintf at"
if you forget to put a % in front of your format spec (first argument).
I'm trying to create more then 2 instances of Grid Gain (Just by running the shell script) in Red Hat Release 6.5 (Santiago), but i get the following error when i try to run the shell script a 3rd time:
java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:714)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:949)
at java.util.concurrent.ThreadPoolExecutor.prestartAllCoreThreads(ThreadPoolExecutor.java:1604)
at org.gridgain.grid.kernal.GridGainEx$GridNamedInstance.start0(GridGainEx.java:1507)
at org.gridgain.grid.kernal.GridGainEx$GridNamedInstance.start(GridGainEx.java:1289)
at org.gridgain.grid.kernal.GridGainEx.start0(GridGainEx.java:832)
at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:759)
at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:677)
at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:524)
at org.gridgain.grid.kernal.GridGainEx.start(GridGainEx.java:494)
at org.gridgain.grid.GridGain.start(GridGain.java:314)
at org.gridgain.grid.startup.cmdline.GridCommandLineStartup.main(GridCommandLineStartup.java:293)
I have set ulimit -n 4096 but still no joy
The box has 64GB of memory - ample amount to run more then 2 instances of GridGain
Can anyone help with this error? are there any configuration changes i can make in Red Hat?
Thanks
Most likely you are running out of allowed number of user processes. We have encountered the same issue on our CentOs servers and setting ulimit -u 10240 helped.
I'm running a debian stable ThinkPad X1 (1294-3QG) with exactly three packages from squeeze-backports needed for the GraphicsModi:
initramfs-tools 0.99~bpo60+1
linux-base 3.4~bpo60+1
linux-image-3.2.0-0.bpo.2-amd64 3.2.9-1~bpo60
While running that kernel, starting for example paraview results in those errors:
Unrecognized deviceID 126
X Error: BadAlloc (insufficient resources for operation) 11
Extension: 154 (Uknown extension)
Minor opcode: 3 (Unknown request)
Resource id: 0x3200273
X Error: GLXBadContext 169
Extension: 154 (Uknown extension)
Minor opcode: 5 (Unknown request)
Resource id: 0x32002b0
paraview: ../../src/xcb_io.c:183: process_responses: Zusicherung »!(req && current_request && !(((long) (req->sequence) - (long) (current_request)) <= 0))« nicht erfüllt.
Somewhere on the net, I found the hint to offer the memory settings in the xorg.conf, but that did not solve my problem.
Starting within the current stable kernel works fine.
Running glxgearsresults similar:
Unrecognized deviceID 126
X Error of failed request: BadAlloc (insufficient resources for operation)
Major opcode of failed request: 154 (GLX)
Minor opcode of failed request: 3 (X_GLXCreateContext)
Serial number of failed request: 27
Current serial number in output stream: 29
I further tried, to solve the problem by updating xserver-xorg-video-intel (and all dependencies libdrm-intel1 libxfont1, xserver-common, xserver-xorg, xserver-xorg-core, xserver-xorg-input-evdev, xserver-xorg-video-fbdev and xserver-xorg-video-vesa) to backports, but that was not prosperous.
Additional, I found the entry
[drm] MTRR allocation failed. Graphics performance may suffer.
in the output of dmesg.
I had the same issue on self-made server station with Intel i7 2700k (which has Intel HD 3000) running Debian Stable 6.0.4 (squeeze) x64. Basically I knew that this platform has loads of problems with unix systems (as always intel GPU does), but it purpose is server, so on-board graphic is fair enough for that. Anyways I wanted someday to run just a move (on TV connected via HDMI*/VGA), so I installed gnome-core with gdm3 to run manually.
With kernel 2.6.32-5-amd64 everything was excellent, besides few things, which forced me to upgrade kernel:
SSD support (added & improved from linux-image-2.6.33)
HDMI - no devices was recognized, couldn't add & change resolution (cvt xrandr).
So I added squeeze-backports to sources.list and upgraded only kernel (same what you did).
After that HDMI connection works great, but I noticed slow refresh rate - tearing during loading gdm3 login screen and after. I checked dmesg and kernel messages for some infos
cat dmesg | grep failed && cat dmesg | grep drm && cat /var/log/messages | grep failed && cat /var/log/messages | grep drm - found same. Than I run glxgears and found same error.
I was digging net for few days after some solutions and ideas.
Found many useless things about allocating RAM (enable_mtrr_cleanup) etc.
Basically for my hardly ever cinematic needs it wasn't tragedy, but I like when everything is perfect, so I still was working around to fix it.
And at last! Got it solved! Problem was not with the RAM or new kernel itself.
I have to mention here, that I compiled Debian kernel myself - 3.2 based on settings from previous install.
I removed also all unneeded libs for my architecture (i.e. libdrm for nvidia radeon and others - even VESA!!!)
I added just for a moment wheezy (testing) repositories, upgraded and installed new packages with dependences as root (only this ones):
echo deb http://ftp.pl.debian.org/debian/ testing main contrib non-free >> /etc/apt/sources.list
apt-get update
apt-get install --reinstall -t testing libdrm2 libdrm-intel1 xserver-xorg-video-intel xserver-xorg-core libgl1-mesa-glx libgl1-mesa-dri mesa-utils
dpkg-reconfigure xserver-xorg
That fixed all problems with rendering and allocation on Intel GPU :)
Think it should works for you and everyone with Intel GPU-s. Don't forget to remove wheeze (testing) from sources.list when you are done.
Regards, T_Send.
I solved it now on my own by updating some mesa concerning packages. I'm running debian stable with those following packages from backports:
initramfs-tools, libdrm-intel1, libgl1-mesa-dev, libgl1-mesa-dri,
libgl1-mesa-glx, linux-base, linux-headers-3.2.0-0.bpo.1-all-amd64,
linux-headers-3.2.0-0.bpo.1-amd64, linux-headers-3.2.0-0.bpo.1-common,
linux-headers-3.2.0-0.bpo.1-common-rt,
linux-headers-3.2.0-0.bpo.1-rt-amd64,
linux-headers-3.2.0-0.bpo.2-all-amd64,
linux-headers-3.2.0-0.bpo.2-amd64, linux-headers-3.2.0-0.bpo.2-common,
linux-headers-3.2.0-0.bpo.2-common-rt,
linux-headers-3.2.0-0.bpo.2-rt-amd64, linux-image-3.2.0-0.bpo.2-amd64,
linux-kbuild-3.2, mesa-common-dev
Hoping this info will help other, too.