Details
-
Type:
Bug
-
Status: Stalled
-
Priority:
Major
-
Resolution: Unresolved
-
Affects Version/s: 5.5.37-galera, 5.5.39-galera
-
Fix Version/s: None
-
Component/s: None
-
Labels:
-
Environment:Ubuntu 12.04.4 LTS (GNU/Linux 2.6.32-28-server x86_64)
Description
package versions:
mariadb-galera-server-5.5-5.5.37+maria-1~precise
galera-25.3.5-wheezy
3 or 4 servers have a similar memory leak profile to the memory-month-g1 image attached. Variables and settings attached are from the g1 server. The dates that an increase in memory usage occurs is the same on all server. The 4th server, memory-month-repl1 is only used as read only and has significantly lower traffic and lower memory usage overall.
The restart at the beginning of the graph was an upgrade from an upgrade from 5.5.32. Looking at the year graphs it appears there was a leak there too that I hadn't noticed before.
Client has another regional database on 5.5.37-MariaDB-1~precise-log (non-galera) with similar structured databases and code usage. If this non-galera instance has a leak is much slower.
top - 09:14:36 up 172 days, 7:56, 2 users, load average: 0.07, 0.02, 0.00
Tasks: 128 total, 1 running, 127 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 7970076k total, 7873132k used, 96944k free, 126140k buffers
Swap: 1073144k total, 6408k used, 1066736k free, 2927628k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7637 mysql 20 0 12.7g 5.5g 1.3g S 0 72.0 505:21.12 mysqld
13405 syslog 20 0 254m 4540 1040 S 0 0.1 3:04.63 rsyslogd
25822 openquer 20 0 17292 1428 1072 R 0 0.0 0:00.03 top
1 root 20 0 24356 1556 696 S 0 0.0 0:00.56 init
I've restarted repl with no query cache and lower usage settings as it was overallocated.
There is a UAT environment with this that I can test out patches with.
Would trying the patch from MDEV-4974 be of use?
Gliffy Diagrams
Attachments
Activity
- All
- Comments
- Work Log
- History
- Activity
- Transitions
I will take a look at the uploaded data. BTW, the fix for MDEV#4974 is already in mariadb-galera-5.5.37.
So, the leak could be because of some other issue.
ack. going to try valgrind running on running process today.
Valgrind report (picked from MDEV-6368).
i'm not sure that valgrind report is going to help.
I did attempt to get a valgrind on a UAT instance. Unfortunately there was insufficient RAM On the UAT instance to successfully run valgrind even with innodb_buffer_size set abnormally low.
This week the UAT instance RAM has been upgraded and I'm running valgrind on the instance as I type. Apologies for the delay and lack of communication.
Besides some seemingly acceptable valgrind errors, I am still not able to reproduce this. Also tried to populate tables from the uploaded script with some sample data but no luck so far.
MDEV-6300-leaks-track.sql.gz pushed to private ftp as the the output from the psergey-mdev4974-xpl1.diff patch in
MDEV-4974.graphing showed small increase of memory usage over a day on a test server. I'm not 100 I've captured this.
Happy to try other patches and test stuff out.