main.mysqldump fails sporadically in buildbot

Description

http://buildbot.askmonty.org/buildbot/builders/kvm-deb-wheezy-amd64/builds/3197/steps/test_4/logs/stdio

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 main.mysqldump 'innodb_plugin' w3 [ fail ] Test ended at 2015-01-06 11:37:45 CURRENT_TEST: main.mysqldump mysqldump: Couldn't find table: "non_existing" mysqldump: Got error: 1356: "View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them" when using LOCK TABLES mysqldump: Couldn't execute 'SHOW FIELDS FROM `v1`': View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them (1356) mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: user2 has insufficent privileges to SHOW CREATE PROCEDURE `sp1`! mysqldump: Got error: 1146: "Table 'test.???????????????????????' doesn't exist" when using LOCK TABLES mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: user1 has insufficent privileges to SHOW CREATE FUNCTION `hello1`! mysqldump: Got error: 2005: "Unknown MySQL server host 'unknownhost' (-2)" when trying to connect mysqldump: Couldn't execute 'SHOW SLAVE STATUS': MySQL server has gone away (2006) --- /usr/share/mysql/mysql-test/r/mysqldump.result 2015-01-06 09:25:23.000000000 -0500 +++ /run/shm/var/3/log/mysqldump.reject 2015-01-06 11:37:44.806794848 -0500 @@ -4293,6 +4293,7 @@ aberration drop table words; mysqlimport: Error: 1146, Table 'test.words' doesn't exist, when using table: words +mysqlimport: Error: 2002 Can't connect to local MySQL server through socket '' (111 "Connection refused") drop table t1; drop table t2; drop table words2; mysqltest: Result length mismatch

The error message 'Error: 2002 Can't connect to local MySQL server through socket '' (111 "Connection refused")' can happen if the socket value is empty. In the test, mysqlimport is run with my.cnf from the vardir, and this config has a socket value in the [client] section. So, can it be that the file gets overridden due to some race condition?

Same log, a different failure with xtradb:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 *** glibc detected *** /usr/bin/mysqlimport: double free or corruption (!prev): 0x00007fc5209eb0f0 *** ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x75b46)[0x7fc51e98db46] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x6c)[0x7fc51e99287c] /usr/bin/mysqlimport(+0x4e9b5)[0x7fc5203199b5] /usr/bin/mysqlimport(+0x4bdab)[0x7fc520316dab] /usr/bin/mysqlimport(+0x22601)[0x7fc5202ed601] /usr/bin/mysqlimport(+0x231bb)[0x7fc5202ee1bb] /usr/bin/mysqlimport(+0x237a1)[0x7fc5202ee7a1] /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50)[0x7fc51fe93b50] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fc51e9f06dd] ======= Memory map: ======== 7fc518000000-7fc518021000 rw-p 00000000 00:00 0 7fc518021000-7fc51c000000 ---p 00000000 00:00 0 7fc51ccf3000-7fc51ccf4000 ---p 00000000 00:00 0 7fc51ccf4000-7fc51d4f4000 rw-p 00000000 00:00 0 7fc51d4f4000-7fc51d4ff000 r-xp 00000000 08:01 34 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7fc51d4ff000-7fc51d6fe000 ---p 0000b000 08:01 34 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7fc51d6fe000-7fc51d6ff000 r--p 0000a000 08:01 34 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7fc51d6ff000-7fc51d700000 rw-p 0000b000 08:01 34 /lib/x86_64-linux-gnu/libnss_files-2.13.so 7fc51d700000-7fc51d701000 ---p 00000000 00:00 0 7fc51d701000-7fc51df01000 rw-p 00000000 00:00 0 7fc51df01000-7fc51df02000 ---p 00000000 00:00 0 7fc51df02000-7fc51e702000 rw-p 00000000 00:00 0 7fc51e702000-7fc51e717000 r-xp 00000000 08:01 18 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fc51e717000-7fc51e917000 ---p 00015000 08:01 18 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fc51e917000-7fc51e918000 rw-p 00015000 08:01 18 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fc51e918000-7fc51ea95000 r-xp 00000000 08:01 16 /lib/x86_64-linux-gnu/libc-2.13.so 7fc51ea95000-7fc51ec95000 ---p 0017d000 08:01 16 /lib/x86_64-linux-gnu/libc-2.13.so 7fc51ec95000-7fc51ec99000 r--p 0017d000 08:01 16 /lib/x86_64-linux-gnu/libc-2.13.so 7fc51ec99000-7fc51ec9a000 rw-p 00181000 08:01 16 /lib/x86_64-linux-gnu/libc-2.13.so 7fc51ec9a000-7fc51ec9f000 rw-p 00000000 00:00 0 7fc51ec9f000-7fc51ed20000 r-xp 00000000 08:01 25 /lib/x86_64-linux-gnu/libm-2.13.so 7fc51ed20000-7fc51ef1f000 ---p 00081000 08:01 25 /lib/x86_64-linux-gnu/libm-2.13.so 7fc51ef1f000-7fc51ef20000 r--p 00080000 08:01 25 /lib/x86_64-linux-gnu/libm-2.13.so 7fc51ef20000-7fc51ef21000 rw-p 00081000 08:01 25 /lib/x86_64-linux-gnu/libm-2.13.so 7fc51ef21000-7fc51f009000 r-xp 00000000 08:01 264207 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 7fc51f009000-7fc51f209000 ---p 000e8000 08:01 264207 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 7fc51f209000-7fc51f211000 r--p 000e8000 08:01 264207 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 7fc51f211000-7fc51f213000 rw-p 000f0000 08:01 264207 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.17 7fc51f213000-7fc51f228000 rw-p 00000000 00:00 0 7fc51f228000-7fc51f22a000 r-xp 00000000 08:01 32 /lib/x86_64-linux-gnu/libdl-2.13.so 7fc51f22a000-7fc51f42a000 ---p 00002000 08:01 32 /lib/x86_64-linux-gnu/libdl-2.13.so 7fc51f42a000-7fc51f42b000 r--p 00002000 08:01 32 /lib/x86_64-linux-gnu/libdl-2.13.so 7fc51f42b000-7fc51f42c000 rw-p 00003000 08:01 32 /lib/x86_64-linux-gnu/libdl-2.13.so 7fc51f42c000-7fc51f5e2000 r-xp 00000000 08:01 264729 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fc51f5e2000-7fc51f7e2000 ---p 001b6000 08:01 264729 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fc51f7e2000-7fc51f7fd000 r--p 001b6000 08:01 264729 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fc51f7fd000-7fc51f80c000 rw-p 001d1000 08:01 264729 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fc51f80c000-7fc51f810000 rw-p 00000000 00:00 0 7fc51f810000-7fc51f864000 r-xp 00000000 08:01 264730 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7fc51f864000-7fc51fa64000 ---p 00054000 08:01 264730 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7fc51fa64000-7fc51fa67000 r--p 00054000 08:01 264730 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7fc51fa67000-7fc51fa6e000 rw-p 00057000 08:01 264730 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 7fc51fa6e000-7fc51fa75000 r-xp 00000000 08:01 38 /lib/x86_64-linux-gnu/librt-2.13.so 7fc51fa75000-7fc51fc74000 ---p 00007000 08:01 38 /lib/x86_64-linux-gnu/librt-2.13.so 7fc51fc74000-7fc51fc75000 r--p 00006000 08:01 38 /lib/x86_64-linux-gnu/librt-2.13.so 7fc51fc75000-7fc51fc76000 rw-p 00007000 08:01 38 /lib/x86_64-linux-gnu/librt-2.13.so 7fc51fc76000-7fc51fc8c000 r-xp 00000000 08:01 170 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fc51fc8c000-7fc51fe8b000 ---p 00016000 08:01 170 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fc51fe8b000-7fc51fe8c000 r--p 00015000 08:01 170 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fc51fe8c000-7fc51fe8d000 rw-p 00016000 08:01 170 /lib/x86_64-linux-gnu/libz.so.1.2.7 7fc51fe8d000-7fc51fea4000 r-xp 00000000 08:01 36 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fc51fea4000-7fc5200a3000 ---p 00017000 08:01 36 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fc5200a3000-7fc5200a4000 r--p 00016000 08:01 36 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fc5200a4000-7fc5200a5000 rw-p 00017000 08:01 36 /lib/x86_64-linux-gnu/libpthread-2.13.so 7fc5200a5000-7fc5200a9000 rw-p 00000000 00:00 0 7fc5200a9000-7fc5200c9000 r-xp 00000000 08:01 40 /lib/x86_64-linux-gnu/ld-2.13.so 7fc5202ba000-7fc5202c2000 rw-p 00000000 00:00 0 7fc5202c6000-7fc5202c8000 rw-p 00000000 00:00 0 7fc5202c8000-7fc5202c9000 r--p 0001f000 08:01 40 /lib/x86_64-linux-gnu/ld-2.13.so 7fc5202c9000-7fc5202ca000 rw-p 00020000 08:01 40 /lib/x86_64-linux-gnu/ld-2.13.so 7fc5202ca000-7fc5202cb000 rw-p 00000000 00:00 0 7fc5202cb000-7fc5205b1000 r-xp 00000000 08:01 262824 /usr/bin/mysqlimport 7fc5207b1000-7fc5207b9000 r--p 002e6000 08:01 262824 /usr/bin/mysqlimport 7fc5207b9000-7fc52081f000 rw-p 002ee000 08:01 262824 /usr/bin/mysqlimport 7fc52081f000-7fc520829000 rw-p 00000000 00:00 0 7fc5209eb000-7fc520a29000 rw-p 00000000 00:00 0 [heap] 7fffbea10000-7fffbea32000 rw-p 00000000 00:00 0 [stack] 7fffbeb26000-7fffbeb27000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] main.mysqldump 'xtradb' w2 [ fail ] Test ended at 2015-01-06 11:38:55 CURRENT_TEST: main.mysqldump mysqldump: Couldn't find table: "non_existing" mysqldump: Got error: 1356: "View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them" when using LOCK TABLES mysqldump: Couldn't execute 'SHOW FIELDS FROM `v1`': View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them (1356) mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' mysqldump: user2 has insufficent privileges to SHOW CREATE PROCEDURE `sp1`! mysqldump: Got error: 1146: "Table 'test.???????????????????????' doesn't exist" when using LOCK TABLES mysqltest: At line 1783: command "$MYSQL_IMPORT --silent --use-threads=2 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt $MYSQLTEST_VARDIR/std_data/words.dat $MYSQLTEST_VARDIR/std_data/words2.dat 2>&1" failed with wrong error: 134 The result from queries just before the failure was: < snip > abdomen abdomens abdominal abduct abducted abduction abductions abductor abductors abducts Abe abed Abel Abelian Abelson Aberdeen Abernathy aberrant aberration drop table words; More results from queries before failure can be found in /run/shm/var/2/log/mysqldump.log - skipping '/run/shm/var/2/log/main.mysqldump-xtradb/'

Another flavor:
http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/4569/steps/test_3/logs/stdio

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 main.mysqldump 'xtradb' w1 [ fail ] Test ended at 2015-01-15 09:52:01 CURRENT_TEST: main.mysqldump mysqldump: Couldn't find table: "non_existing" mysqldump: Got error: 1356: "View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them" when using LOCK TABLES mysqldump: Couldn't execute 'SHOW FIELDS FROM `v1`': View 'test.v1' references invalid table(s) or column(s) or function(s) or definer/invoker of view lack rights to use them (1356) mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' Warning: 3072 bytes lost at 0x10841c0, allocated by T@0 at mysys/my_malloc.c:164, mysys/string.c:107, client/mysqldump.c:5831, client/mysqldump.c:3462, client/mysqldump.c:3440, client/mysqldump.c:3620, client/mysqldump.c:4525, client/mysqldump.c:4342 Warning: 8152 bytes lost at 0x1082160, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1652, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991 Warning: 96 bytes lost at 0x107e0e0, allocated by T@0 at sql-common/client.c:1626, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa32af39c4d Warning: 160 bytes lost at 0x107dfc0, allocated by T@0 at sql-common/client.c:4222, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa32af39c4d, 0x40a7c9 Warning: 8152 bytes lost at 0x1080100, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1503, sql-common/client.c:4167, sql-common/client.c:4200, libmysql/libmysql.c:714, libmysql/libmysql.c:801, client/mysqldump.c:4007, client/mysqldump.c:4491 Memory lost: 19632 bytes in 5 chunks mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' Warning: 3072 bytes lost at 0xcf31c0, allocated by T@0 at mysys/my_malloc.c:164, mysys/string.c:107, client/mysqldump.c:5831, client/mysqldump.c:3462, client/mysqldump.c:3440, client/mysqldump.c:3621, client/mysqldump.c:4525, client/mysqldump.c:4342 Warning: 8152 bytes lost at 0xcf1160, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1652, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991 Warning: 96 bytes lost at 0xced0e0, allocated by T@0 at sql-common/client.c:1626, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa4c008dc4d Warning: 160 bytes lost at 0xcecfc0, allocated by T@0 at sql-common/client.c:4222, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa4c008dc4d, 0x40a7c9 Warning: 8152 bytes lost at 0xcef100, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1503, sql-common/client.c:4167, sql-common/client.c:4200, libmysql/libmysql.c:714, libmysql/libmysql.c:801, client/mysqldump.c:4007, client/mysqldump.c:4491 Memory lost: 19632 bytes in 5 chunks mysqldump: Got error: 1083: "Field separator argument is not what is expected; check the manual" when executing 'SELECT INTO OUTFILE' Warning: 3072 bytes lost at 0x239a1c0, allocated by T@0 at mysys/my_malloc.c:164, mysys/string.c:107, client/mysqldump.c:5831, client/mysqldump.c:3462, client/mysqldump.c:3440, client/mysqldump.c:3622, client/mysqldump.c:4525, client/mysqldump.c:4342 Warning: 8152 bytes lost at 0x2398160, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1652, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991 Warning: 96 bytes lost at 0x23940e0, allocated by T@0 at sql-common/client.c:1626, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa0290fcc4d Warning: 160 bytes lost at 0x2393fc0, allocated by T@0 at sql-common/client.c:4222, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7fa0290fcc4d, 0x40a7c9 Warning: 8152 bytes lost at 0x2396100, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1503, sql-common/client.c:4167, sql-common/client.c:4200, libmysql/libmysql.c:714, libmysql/libmysql.c:801, client/mysqldump.c:4007, client/mysqldump.c:4491 Memory lost: 19632 bytes in 5 chunks mysqldump: user2 has insufficent privileges to SHOW CREATE PROCEDURE `sp1`! mysqldump: Got error: 1146: "Table 'test.???????????????????????' doesn't exist" when using LOCK TABLES Warning: 8152 bytes lost at 0x298d1b0, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1652, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991 Warning: 96 bytes lost at 0x298d0d0, allocated by T@0 at sql-common/client.c:1626, sql-common/client.c:4233, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7f418b4cec4d Warning: 160 bytes lost at 0x298cfb0, allocated by T@0 at sql-common/client.c:4222, libmysql/libmysql.c:803, client/mysqldump.c:4007, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7f418b4cec4d, 0x40a7c9 Warning: 8152 bytes lost at 0x29919d0, allocated by T@0 at mysys/my_alloc.c:233, sql-common/client.c:1503, sql-common/client.c:4167, sql-common/client.c:4200, libmysql/libmysql.c:714, libmysql/libmysql.c:801, client/mysqldump.c:4007, client/mysqldump.c:4491 Warning: 1024 bytes lost at 0x298cb30, allocated by T@0 at mysys/string.c:39, client/mysqldump.c:5812, client/mysqldump.c:4491, client/mysqldump.c:4342, client/mysqldump.c:5991, 0x7f418b4cec4d, 0x40a7c9 Memory lost: 17584 bytes in 5 chunks mysqltest: At line 1783: command "$MYSQL_IMPORT --silent --use-threads=2 test $MYSQLTEST_VARDIR/tmp/t1.txt $MYSQLTEST_VARDIR/tmp/t2.txt $MYSQLTEST_VARDIR/std_data/words.dat $MYSQLTEST_VARDIR/std_data/words2.dat 2>&1" failed with wrong error: 139 The result from queries just before the failure was: < snip > abdomen abdomens abdominal abduct abducted abduction abductions abductor abductors abducts Abe abed Abel Abelian Abelson Aberdeen Abernathy aberrant aberration drop table words; More results from queries before failure can be found in /dev/shm/var_auto_d2bo/1/log/mysqldump.log - saving '/dev/shm/var_auto_d2bo/1/log/main.mysqldump-xtradb/' to '/dev/shm/var_auto_d2bo/log/main.mysqldump-xtradb/'

Environment

None

Status

Assignee

Sergei Golubchik

Reporter

Elena Stepanova

Labels

External issue ID

None

External issue ID

None

Components

Fix versions

Affects versions

10.0

Priority

Major
Configure