[MDEV-4089] LevelDB: Extensive memory usage on relatively small tables, valgrind warnings "blocks are definitely lost" - JIRA

Details

Type: Technical task
Status: Closed
Priority: Major
Resolution: Won't Fix
Affects Version/s: None
Fix Version/s: None
Component/s: None
Labels:
- leveldb

Description

A 5000-row table with one INT pk and one MEDIUMTEXT field (the field is populated with ~200K values) takes about 1.6 Gb memory while it is being populated and about 1 Gb afterwards. A 10K-row table might take up to 5 Gb while it's been populated.

With InnoDB and otherwise the same flow, there is no visible growth in memory usage. See the comparative output below.

LevelDB:

CREATE TABLE t1 (pk INT PRIMARY KEY, f MEDIUMTEXT) ENGINE=LevelDB;
# Before INSERTs
10723 elenst    20   0  312m  87m 9748 S    0  1.5   0:00.44 mysqld             
BEGIN;
# Inserting 5000 rows
# After INSERTs, before COMMIT
10723 elenst    20   0 1903m 1.0g 9976 S    0 17.8   0:09.02 mysqld             
# After COMMIT and having let things settle down a bit (5 sec wait)
10723 elenst    20   0 1268m 1.0g 9.9m S    0 17.4   0:12.64 mysqld

InnoDB:

CREATE TABLE t1 (pk INT PRIMARY KEY, f MEDIUMTEXT) ENGINE=InnoDB;
# Before INSERTs
10625 elenst    20   0  310m  87m 9.8m S    0  1.5   0:00.46 mysqld             
BEGIN;
# Inserting 5000 rows
# After INSERTs, before COMMIT
10625 elenst    20   0  312m  95m  10m S    0  1.6   1:58.42 mysqld             
# After COMMIT and having let things settle down a bit (5 sec wait)
10625 elenst    20   0  312m  95m  10m S    0  1.6   1:58.46 mysqld

Test case (please note it's just for experimenting, not for regression suite):

let $num = 5000;

CREATE TABLE t1 (pk INT PRIMARY KEY, f MEDIUMTEXT) ENGINE=LevelDB;

--echo # Before INSERTs
--exec top -b -n 1 | grep mysqld
--exec du -sk $MYSQLTEST_VARDIR

SELECT NOW();
BEGIN;
--echo # Inserting $num rows
--disable_query_log
while ($num)
{
  eval INSERT INTO t1 VALUES (10001-$num, REPEAT('a',200000));
  dec $num;
}
--echo # After INSERTs, before COMMIT
--exec top -b -n 1 | grep mysqld
--exec du -sk $MYSQLTEST_VARDIR
COMMIT;
SELECT NOW();

--sleep 5
--echo # After COMMIT and having let things settle down a bit
--exec top -b -n 1 | grep mysqld
--exec du -sk $MYSQLTEST_VARDIR
DROP TABLE t1;

The size of the datadir is comparable (about 1 Gb in each case).
Execution time for LevelDB is much less.

revision-id: psergey@askmonty.org-20130124165745-nm4yaxrsu8e4o2ll
revno: 4511
branch-nick: mysql-5.6-leveldb

Below is a smaller test case, which might not show a noticeable growth, but it's faster to execute and it still causes valgrind complaints about lost blocks:

let $num = 50;

CREATE TABLE t1 (pk INT PRIMARY KEY, f MEDIUMTEXT) ENGINE=LevelDB;

while ($num)
{
  eval INSERT INTO t1 VALUES (10001-$num, REPEAT('a',200000));
  dec $num;
}
--sleep 5

8 bytes in 1 blocks are still reachable in loss record 1 of 367
   at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
   by 0x4E69DAD: leveldb::InitModule() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x58D3712: pthread_once (pthread_once.S:104)
   by 0x4E6F288: leveldb::port::InitOnce(int*, void (*)()) (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x4E6A026: leveldb::BytewiseComparator() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x4E6EDD8: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0xA5D8AC: leveldb_init_func(void*) (ha_leveldb.cc:425)
   by 0x6148DB: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
   by 0x7AE3FF: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
   by 0x7AED78: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
   by 0x602B0A: init_server_components() (mysqld.cc:4746)
   by 0x60379D: mysqld_main(int, char**) (mysqld.cc:5320)
   by 0x5FA7E3: main (main.cc:25)

8 bytes in 1 blocks are definitely lost in loss record 2 of 367
   at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
   by 0xA3CBA5: my_malloc (my_malloc.c:38)
   by 0xA5EBA8: ha_leveldb::create_key_defs(TABLE*, bool) (ha_leveldb.cc:793)
   by 0xA5EDB4: ha_leveldb::create(char const*, TABLE*, st_ha_create_information*) (ha_leveldb.cc:859)
   by 0x61C231: handler::ha_create(char const*, TABLE*, st_ha_create_information*) (handler.cc:4392)
   by 0x61CAA5: ha_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, bool) (handler.cc:4628)
   by 0x86801E: rea_create_table(THD*, char const*, char const*, char const*, st_ha_create_information*, List<Create_field>&, unsigned int, st_key*, handler*, bool) (unireg.cc:527)
   by 0x804384: create_table_impl(THD*, char const*, char const*, char const*, st_ha_create_information*, Alter_info*, bool, unsigned int, bool, bool*, st_key**, unsigned int*) (sql_table.cc:4724)
   by 0x804851: mysql_create_table_no_lock(THD*, char const*, char const*, st_ha_create_information*, Alter_info*, unsigned int, bool*) (sql_table.cc:4825)
   by 0x80495A: mysql_create_table(THD*, TABLE_LIST*, st_ha_create_information*, Alter_info*) (sql_table.cc:4871)
   by 0x79E53A: mysql_execute_command(THD*) (sql_parse.cc:2962)
   by 0x7A6BE9: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6097)
   by 0x79AA26: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312)
   by 0x799BE9: do_command(THD*) (sql_parse.cc:1036)
   by 0x764FE0: do_handle_one_connection(THD*) (sql_connect.cc:969)
   by 0x764A05: handle_one_connection (sql_connect.cc:885)

8 bytes in 1 blocks are definitely lost in loss record 3 of 367
   at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
   by 0xA3CBA5: my_malloc (my_malloc.c:38)
   by 0xA5E909: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:744)
   by 0x617D62: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421)
   by 0x853FE3: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2152)
   by 0x738C4C: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035)
   by 0x73B124: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510)
   by 0x73BE0C: open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) (sql_base.cc:4945)
   by 0x73CE87: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633)
   by 0x77EF15: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:712)
   by 0x79F509: mysql_execute_command(THD*) (sql_parse.cc:3344)
   by 0x7A6BE9: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6097)
   by 0x79AA26: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312)
   by 0x799BE9: do_command(THD*) (sql_parse.cc:1036)
   by 0x764FE0: do_handle_one_connection(THD*) (sql_connect.cc:969)
   by 0x764A05: handle_one_connection (sql_connect.cc:885)

14 bytes in 1 blocks are definitely lost in loss record 4 of 367
   at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
   by 0xA3CBA5: my_malloc (my_malloc.c:38)
   by 0xA5E93C: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:746)
   by 0x617D62: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421)
   by 0x853FE3: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2152)
   by 0x738C4C: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035)
   by 0x73B124: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510)
   by 0x73BE0C: open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int, Prelocking_strategy*) (sql_base.cc:4945)
   by 0x73CE87: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633)
   by 0x77EF15: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:712)
   by 0x79F509: mysql_execute_command(THD*) (sql_parse.cc:3344)
   by 0x7A6BE9: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6097)
   by 0x79AA26: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312)
   by 0x799BE9: do_command(THD*) (sql_parse.cc:1036)
   by 0x764FE0: do_handle_one_connection(THD*) (sql_connect.cc:969)
   by 0x764A05: handle_one_connection (sql_connect.cc:885)

64 bytes in 1 blocks are still reachable in loss record 187 of 367
   at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
   by 0x4E6D012: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x58D3712: pthread_once (pthread_once.S:104)
   by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0xA5D8AC: leveldb_init_func(void*) (ha_leveldb.cc:425)
   by 0x6148DB: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
   by 0x7AE3FF: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
   by 0x7AED78: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
   by 0x602B0A: init_server_components() (mysqld.cc:4746)
   by 0x60379D: mysqld_main(int, char**) (mysqld.cc:5320)
   by 0x5FA7E3: main (main.cc:25)

336 bytes in 1 blocks are still reachable in loss record 259 of 367
   at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
   by 0x4E6CF81: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x58D3712: pthread_once (pthread_once.S:104)
   by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0xA5D8AC: leveldb_init_func(void*) (ha_leveldb.cc:425)
   by 0x6148DB: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
   by 0x7AE3FF: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
   by 0x7AED78: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
   by 0x602B0A: init_server_components() (mysqld.cc:4746)
   by 0x60379D: mysqld_main(int, char**) (mysqld.cc:5320)
   by 0x5FA7E3: main (main.cc:25)

512 bytes in 1 blocks are still reachable in loss record 267 of 367
   at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
   by 0x4E6D037: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x58D3712: pthread_once (pthread_once.S:104)
   by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
   by 0xA5D8AC: leveldb_init_func(void*) (ha_leveldb.cc:425)
   by 0x6148DB: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
   by 0x7AE3FF: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
   by 0x7AED78: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
   by 0x602B0A: init_server_components() (mysqld.cc:4746)
   by 0x60379D: mysqld_main(int, char**) (mysqld.cc:5320)
   by 0x5FA7E3: main (main.cc:25)

LEAK SUMMARY:
   definitely lost: 30 bytes in 3 blocks
   indirectly lost: 0 bytes in 0 blocks
     possibly lost: 0 bytes in 0 blocks
   still reachable: 920 bytes in 4 blocks
        suppressed: 67,240,557 bytes in 1,052 blocks

For counts of detected and suppressed errors, rerun with: -v
ERROR SUMMARY: 3 errors from 3 contexts (suppressed: 312 from 312)

revision-id: psergey@askmonty.org-20130125200959-a7gq7phbaiajqs12
revno: 4519
branch-nick: mysql-5.6-leveldb

Gliffy Diagrams

Attachments

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Elena Stepanova added a comment - 2013-01-26 15:22

Added a test case to run with valgrind, and example of valgrind warnings it produces.

Show

Elena Stepanova added a comment - 2013-01-26 15:22 Added a test case to run with valgrind, and example of valgrind warnings it produces.

Hide

Permalink

Elena Stepanova added a comment - 2013-02-06 14:48 - edited

The valgrind warnings are reproducible on

revision-id: psergey@askmonty.org-20130201180328-ocmbh9uvcoedmihp
revno: 4591
branch-nick: mysql-5.6-leveldb

8 bytes in 1 blocks are still reachable in loss record 1 of 365
at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
by 0x4E69DAD: leveldb::InitModule() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x58D3712: pthread_once (pthread_once.S:104)
by 0x4E6F288: leveldb::port::InitOnce(int*, void ()) (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x4E6A026: leveldb::BytewiseComparator() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x4E6EDD8: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466)
by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
by 0x604DF9: init_server_components() (mysqld.cc:4755)
by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329)
by 0x5FC303: main (main.cc:25)

8 bytes in 1 blocks are definitely lost in loss record 2 of 365
at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
by 0xA6702B: my_malloc (my_malloc.c:38)
by 0xA8F2B7: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:845)
by 0x61AC8E: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421)
by 0x866B9D: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2266)
by 0x741FF4: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035)
by 0x744542: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510)
by 0x74522A: open_tables(THD*, TABLE_LIST*, unsigned int, unsigned int, Prelocking_strategy*) (sql_base.cc:4945)
by 0x7462A5: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633)
by 0x78A577: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:702)
by 0x7AB99A: mysql_execute_command(THD*) (sql_parse.cc:3349)
by 0x7B30C7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6117)
by 0x7A6E47: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312)
by 0x7A600A: do_command(THD*) (sql_parse.cc:1036)
by 0x76F123: do_handle_one_connection(THD*) (sql_connect.cc:969)
by 0x76EB48: handle_one_connection (sql_connect.cc:885)

9 bytes in 1 blocks are definitely lost in loss record 3 of 365
at 0x4C28F9F: malloc (vg_replace_malloc.c:236)
by 0xA6702B: my_malloc (my_malloc.c:38)
by 0xA8F2F3: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:847)
by 0x61AC8E: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421)
by 0x866B9D: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2266)
by 0x741FF4: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035)
by 0x744542: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510)
by 0x74522A: open_tables(THD*, TABLE_LIST*, unsigned int, unsigned int, Prelocking_strategy*) (sql_base.cc:4945)
by 0x7462A5: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633)
by 0x78A577: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:702)
by 0x7AB99A: mysql_execute_command(THD*) (sql_parse.cc:3349)
by 0x7B30C7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6117)
by 0x7A6E47: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312)
by 0x7A600A: do_command(THD*) (sql_parse.cc:1036)
by 0x76F123: do_handle_one_connection(THD*) (sql_connect.cc:969)
by 0x76EB48: handle_one_connection (sql_connect.cc:885)

64 bytes in 1 blocks are still reachable in loss record 184 of 365
at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
by 0x4E6D012: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x58D3712: pthread_once (pthread_once.S:104)
by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466)
by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
by 0x604DF9: init_server_components() (mysqld.cc:4755)
by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329)
by 0x5FC303: main (main.cc:25)

336 bytes in 1 blocks are still reachable in loss record 256 of 365
at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
by 0x4E6CF81: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x58D3712: pthread_once (pthread_once.S:104)
by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466)
by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
by 0x604DF9: init_server_components() (mysqld.cc:4755)
by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329)
by 0x5FC303: main (main.cc:25)

512 bytes in 1 blocks are still reachable in loss record 264 of 365
at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261)
by 0x4E6D037: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x58D3712: pthread_once (pthread_once.S:104)
by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7)
by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466)
by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658)
by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126)
by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417)
by 0x604DF9: init_server_components() (mysqld.cc:4755)
by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329)
by 0x5FC303: main (main.cc:25)

Show

Elena Stepanova added a comment - 2013-02-06 14:48 - edited The valgrind warnings are reproducible on revision-id: psergey@askmonty.org-20130201180328-ocmbh9uvcoedmihp revno: 4591 branch-nick: mysql-5.6-leveldb 8 bytes in 1 blocks are still reachable in loss record 1 of 365 at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261) by 0x4E69DAD: leveldb::InitModule() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x58D3712: pthread_once (pthread_once.S:104) by 0x4E6F288: leveldb::port::InitOnce(int*, void ()) (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x4E6A026: leveldb::BytewiseComparator() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x4E6EDD8: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466) by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658) by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126) by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417) by 0x604DF9: init_server_components() (mysqld.cc:4755) by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329) by 0x5FC303: main (main.cc:25) 8 bytes in 1 blocks are definitely lost in loss record 2 of 365 at 0x4C28F9F: malloc (vg_replace_malloc.c:236) by 0xA6702B: my_malloc (my_malloc.c:38) by 0xA8F2B7: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:845) by 0x61AC8E: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421) by 0x866B9D: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2266) by 0x741FF4: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035) by 0x744542: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510) by 0x74522A: open_tables(THD*, TABLE_LIST* , unsigned int , unsigned int, Prelocking_strategy*) (sql_base.cc:4945) by 0x7462A5: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633) by 0x78A577: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:702) by 0x7AB99A: mysql_execute_command(THD*) (sql_parse.cc:3349) by 0x7B30C7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6117) by 0x7A6E47: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312) by 0x7A600A: do_command(THD*) (sql_parse.cc:1036) by 0x76F123: do_handle_one_connection(THD*) (sql_connect.cc:969) by 0x76EB48: handle_one_connection (sql_connect.cc:885) 9 bytes in 1 blocks are definitely lost in loss record 3 of 365 at 0x4C28F9F: malloc (vg_replace_malloc.c:236) by 0xA6702B: my_malloc (my_malloc.c:38) by 0xA8F2F3: ha_leveldb::open(char const*, int, unsigned int) (ha_leveldb.cc:847) by 0x61AC8E: handler::ha_open(TABLE*, char const*, int, int) (handler.cc:2421) by 0x866B9D: open_table_from_share(THD*, TABLE_SHARE*, char const*, unsigned int, unsigned int, unsigned int, TABLE*, bool) (table.cc:2266) by 0x741FF4: open_table(THD*, TABLE_LIST*, Open_table_context*) (sql_base.cc:3035) by 0x744542: open_and_process_table(THD*, LEX*, TABLE_LIST*, unsigned int*, unsigned int, Prelocking_strategy*, bool, Open_table_context*) (sql_base.cc:4510) by 0x74522A: open_tables(THD*, TABLE_LIST* , unsigned int , unsigned int, Prelocking_strategy*) (sql_base.cc:4945) by 0x7462A5: open_normal_and_derived_tables(THD*, TABLE_LIST*, unsigned int) (sql_base.cc:5633) by 0x78A577: mysql_insert(THD*, TABLE_LIST*, List<Item>&, List<List<Item> >&, List<Item>&, List<Item>&, enum_duplicates, bool) (sql_insert.cc:702) by 0x7AB99A: mysql_execute_command(THD*) (sql_parse.cc:3349) by 0x7B30C7: mysql_parse(THD*, char*, unsigned int, Parser_state*) (sql_parse.cc:6117) by 0x7A6E47: dispatch_command(enum_server_command, THD*, char*, unsigned int) (sql_parse.cc:1312) by 0x7A600A: do_command(THD*) (sql_parse.cc:1036) by 0x76F123: do_handle_one_connection(THD*) (sql_connect.cc:969) by 0x76EB48: handle_one_connection (sql_connect.cc:885) 64 bytes in 1 blocks are still reachable in loss record 184 of 365 at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261) by 0x4E6D012: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x58D3712: pthread_once (pthread_once.S:104) by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466) by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658) by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126) by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417) by 0x604DF9: init_server_components() (mysqld.cc:4755) by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329) by 0x5FC303: main (main.cc:25) 336 bytes in 1 blocks are still reachable in loss record 256 of 365 at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261) by 0x4E6CF81: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x58D3712: pthread_once (pthread_once.S:104) by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466) by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658) by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126) by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417) by 0x604DF9: init_server_components() (mysqld.cc:4755) by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329) by 0x5FC303: main (main.cc:25) 512 bytes in 1 blocks are still reachable in loss record 264 of 365 at 0x4C28B35: operator new(unsigned long) (vg_replace_malloc.c:261) by 0x4E6D037: leveldb::InitDefaultEnv() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x58D3712: pthread_once (pthread_once.S:104) by 0x4E6CF56: leveldb::Env::Default() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0x4E6EDEC: leveldb::Options::Options() (in /home/elenst/leveldb-1.7.0/libleveldb.so.1.7) by 0xA8DE15: leveldb_init_func(void*) (ha_leveldb.cc:466) by 0x6177E6: ha_initialize_handlerton(st_plugin_int*) (handler.cc:658) by 0x7BADF1: plugin_initialize(st_plugin_int*) (sql_plugin.cc:1126) by 0x7BB784: plugin_init(int*, char**, int) (sql_plugin.cc:1417) by 0x604DF9: init_server_components() (mysqld.cc:4755) by 0x605A8C: mysqld_main(int, char**) (mysqld.cc:5329) by 0x5FC303: main (main.cc:25)

Hide

Permalink

Sergei Petrunia added a comment - 2013-02-06 18:16

Some of these warnings are known, O(1) warnings due to permanently-initialized strucutres of leveldb
Just fixed the other warnings like loss record 2 of 365 and loss record 3 of 365

Show

Sergei Petrunia added a comment - 2013-02-06 18:16 Some of these warnings are known, O(1) warnings due to permanently-initialized strucutres of leveldb Just fixed the other warnings like loss record 2 of 365 and loss record 3 of 365

LevelDB: Extensive memory usage on relatively small tables, valgrind warnings "blocks are definitely lost"

Details

Description

Gliffy Diagrams

Attachments

Activity

People

Dates

Agile