[MDEV-8536] Crash in os_event_free on shutdown - JIRA

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 10.0.20
Fix Version/s: 10.1, 10.0
Component/s: Platform Debian, Platform Power
Labels:
None

Description

This crash comes from debian build system: https://buildd.debian.org/status/package.php?p=mariadb-10.0

Server crashes during shutdown of mtr bootstrap instance. Looks like either double free or memory corruption.

(gdb) --no-defaults --bootstrap --basedir=/home/otto/mariadb-10.0-10.0.20 --datadir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/install.db --default-storage-engine=myisam --skip-plugin-archive --skip-plugin-aria --skip-plugin-blackhole --skip-plugin-federated --skip-plugin-feedback --skip-plugin-innodb --skip-plugin-innodb-buffer-page --skip-plugin-innodb-buffer-page-lru --skip-plugin-innodb-buffer-pool-stats --skip-plugin-innodb-changed-pages --skip-plugin-innodb-cmp --skip-plugin-innodb-cmp-per-index --skip-plugin-innodb-cmp-per-index-reset --skip-plugin-innodb-cmp-reset --skip-plugin-innodb-cmpmem --skip-plugin-innodb-cmpmem-reset --skip-plugin-innodb-ft-being-deleted --skip-plugin-innodb-ft-config --skip-plugin-innodb-ft-default-stopword --skip-plugin-innodb-ft-deleted --skip-plugin-innodb-ft-index-cache --skip-plugin-innodb-ft-index-table --skip-plugin-innodb-lock-waits --skip-plugin-innodb-locks --skip-plugin-innodb-metrics --skip-plugin-innodb-sys-columns --skip-plugin-innodb-sys-datafiles --skip-plugin-innodb-sys-fields --skip-plugin-innodb-sys-foreign --skip-plugin-innodb-sys-foreign-cols --skip-plugin-innodb-sys-indexes --skip-plugin-innodb-sys-tables --skip-plugin-innodb-sys-tablespaces --skip-plugin-innodb-sys-tablestats --skip-plugin-innodb-trx --skip-plugin-partition --skip-plugin-xtradb-internal-hash-tables --skip-plugin-xtradb-read-view --skip-plugin-xtradb-rseg --loose-innodb --loose-innodb-log-file-size=5M --disable-sync-frm --tmpdir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/tmp/ --core-file --lc-messages-dir=/home/otto/mariadb-10.0-10.0.20/builddir/sql/share/ --character-sets-dir=/home/otto/mariadb-10.0-10.0.20/sql/share/charsets --debug --stack-trace
Undefined command: "--no-defaults".  Try "help".
(gdb) run --no-defaults --bootstrap --basedir=/home/otto/mariadb-10.0-10.0.20 --datadir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/install.db --default-storage-engine=myisam --skip-plugin-archive --skip-plugin-aria --skip-plugin-blackhole --skip-plugin-federated --skip-plugin-feedback --skip-plugin-innodb --skip-plugin-innodb-buffer-page --skip-plugin-innodb-buffer-page-lru --skip-plugin-innodb-buffer-pool-stats --skip-plugin-innodb-changed-pages --skip-plugin-innodb-cmp --skip-plugin-innodb-cmp-per-index --skip-plugin-innodb-cmp-per-index-reset --skip-plugin-innodb-cmp-reset --skip-plugin-innodb-cmpmem --skip-plugin-innodb-cmpmem-reset --skip-plugin-innodb-ft-being-deleted --skip-plugin-innodb-ft-config --skip-plugin-innodb-ft-default-stopword --skip-plugin-innodb-ft-deleted --skip-plugin-innodb-ft-index-cache --skip-plugin-innodb-ft-index-table --skip-plugin-innodb-lock-waits --skip-plugin-innodb-locks --skip-plugin-innodb-metrics --skip-plugin-innodb-sys-columns --skip-plugin-innodb-sys-datafiles --skip-plugin-innodb-sys-fields --skip-plugin-innodb-sys-foreign --skip-plugin-innodb-sys-foreign-cols --skip-plugin-innodb-sys-indexes --skip-plugin-innodb-sys-tables --skip-plugin-innodb-sys-tablespaces --skip-plugin-innodb-sys-tablestats --skip-plugin-innodb-trx --skip-plugin-partition --skip-plugin-xtradb-internal-hash-tables --skip-plugin-xtradb-read-view --skip-plugin-xtradb-rseg --loose-innodb --loose-innodb-log-file-size=5M --disable-sync-frm --tmpdir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/tmp/ --core-file --lc-messages-dir=/home/otto/mariadb-10.0-10.0.20/builddir/sql/share/ --character-sets-dir=/home/otto/mariadb-10.0-10.0.20/sql/share/charsets --debug --stack-trace
Starting program: /home/otto/builddir-debug/sql/mysqld --no-defaults --bootstrap --basedir=/home/otto/mariadb-10.0-10.0.20 --datadir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/install.db --default-storage-engine=myisam --skip-plugin-archive --skip-plugin-aria --skip-plugin-blackhole --skip-plugin-federated --skip-plugin-feedback --skip-plugin-innodb --skip-plugin-innodb-buffer-page --skip-plugin-innodb-buffer-page-lru --skip-plugin-innodb-buffer-pool-stats --skip-plugin-innodb-changed-pages --skip-plugin-innodb-cmp --skip-plugin-innodb-cmp-per-index --skip-plugin-innodb-cmp-per-index-reset --skip-plugin-innodb-cmp-reset --skip-plugin-innodb-cmpmem --skip-plugin-innodb-cmpmem-reset --skip-plugin-innodb-ft-being-deleted --skip-plugin-innodb-ft-config --skip-plugin-innodb-ft-default-stopword --skip-plugin-innodb-ft-deleted --skip-plugin-innodb-ft-index-cache --skip-plugin-innodb-ft-index-table --skip-plugin-innodb-lock-waits --skip-plugin-innodb-locks --skip-plugin-innodb-metrics --skip-plugin-innodb-sys-columns --skip-plugin-innodb-sys-datafiles --skip-plugin-innodb-sys-fields --skip-plugin-innodb-sys-foreign --skip-plugin-innodb-sys-foreign-cols --skip-plugin-innodb-sys-indexes --skip-plugin-innodb-sys-tables --skip-plugin-innodb-sys-tablespaces --skip-plugin-innodb-sys-tablestats --skip-plugin-innodb-trx --skip-plugin-partition --skip-plugin-xtradb-internal-hash-tables --skip-plugin-xtradb-read-view --skip-plugin-xtradb-rseg --loose-innodb --loose-innodb-log-file-size=5M --disable-sync-frm --tmpdir=/home/otto/mariadb-10.0-10.0.20/builddir/mysql-test/var/tmp/ --core-file --lc-messages-dir=/home/otto/mariadb-10.0-10.0.20/builddir/sql/share/ --character-sets-dir=/home/otto/mariadb-10.0-10.0.20/sql/share/charsets --debug --stack-trace
warning: Could not load shared library symbols for linux-vdso32.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/powerpc-linux-gnu/libthread_db.so.1".
150723 18:59:05 [Note] /home/otto/builddir-debug/sql/mysqld (mysqld 10.0.20-MariaDB-0+deb8u1-debug) starting as process 13168 ...
150723 18:59:05 [Note] Plugin 'ARCHIVE' is disabled.
150723 18:59:05 [Note] Plugin 'Aria' is disabled.
150723 18:59:05 [Note] Plugin 'BLACKHOLE' is disabled.
150723 18:59:05 [Note] InnoDB: Using mutexes to ref count buffer pool pages
2015-07-23 18:59:05 f7fe78d0 InnoDB: !!!!!!!! UNIV_DEBUG switched on !!!!!!!!!
2015-07-23 18:59:05 f7fe78d0 InnoDB: !!!!!!!! UNIV_SYNC_DEBUG switched on !!!!!!!!!
150723 18:59:05 [Note] InnoDB: The InnoDB memory heap is disabled
150723 18:59:05 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
150723 18:59:05 [Note] InnoDB: GCC builtin __atomic_thread_fence() is used for memory barrier
150723 18:59:05 [Note] InnoDB: Compressed tables use zlib 1.2.8
150723 18:59:05 [Note] InnoDB: Using Linux native AIO
150723 18:59:05 [Note] InnoDB: Not using CPU crc32 instructions
150723 18:59:05 [Note] InnoDB: Initializing buffer pool, size = 128.0M
150723 18:59:05 [Note] InnoDB: Completed initialization of buffer pool
[New Thread 0xe53ff430 (LWP 13173)]
[New Thread 0xe4bff430 (LWP 13174)]
[New Thread 0xe43ff430 (LWP 13175)]
[New Thread 0xe3bff430 (LWP 13176)]
[New Thread 0xe33ff430 (LWP 13177)]
[New Thread 0xe2bff430 (LWP 13178)]
[New Thread 0xe23ff430 (LWP 13179)]
[New Thread 0xe1bff430 (LWP 13180)]
[New Thread 0xe13ff430 (LWP 13181)]
[New Thread 0xe0bff430 (LWP 13182)]
150723 18:59:05 [Note] InnoDB: Highest supported file format is Barracuda.
[New Thread 0xdfbff430 (LWP 13183)]
150723 18:59:06 [Note] InnoDB: 128 rollback segment(s) are active.
[Thread 0xdfbff430 (LWP 13183) exited]
[New Thread 0xdfbff430 (LWP 13184)]
[New Thread 0xdefff430 (LWP 13185)]
[New Thread 0xde7ff430 (LWP 13186)]
[New Thread 0xddfff430 (LWP 13187)]
[New Thread 0xdd7ff430 (LWP 13188)]
150723 18:59:06 [Note] InnoDB: Waiting for purge to start
[New Thread 0xdc7ff430 (LWP 13189)]
[New Thread 0xdbfff430 (LWP 13190)]
150723 18:59:06 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.24-72.2 started; log sequence number 1629494
[New Thread 0xdb7ff430 (LWP 13191)]
[New Thread 0xdafff430 (LWP 13192)]
[New Thread 0xda7ff430 (LWP 13193)]
150723 18:59:06 [Note] Plugin 'XTRADB_READ_VIEW' is disabled.
150723 18:59:06 [Note] Plugin 'XTRADB_INTERNAL_HASH_TABLES' is disabled.
150723 18:59:06 [Note] Plugin 'XTRADB_RSEG' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_TRX' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_LOCK_WAITS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_CMP_RESET' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_CMPMEM' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_CMPMEM_RESET' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_CMP_PER_INDEX_RESET' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_BUFFER_PAGE' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_BUFFER_PAGE_LRU' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_BUFFER_POOL_STATS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_METRICS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_DEFAULT_STOPWORD' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_DELETED' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_BEING_DELETED' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_CONFIG' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_INDEX_CACHE' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_FT_INDEX_TABLE' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_TABLES' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_TABLESTATS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_INDEXES' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_COLUMNS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_FIELDS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_FOREIGN' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_FOREIGN_COLS' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_TABLESPACES' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_SYS_DATAFILES' is disabled.
150723 18:59:06 [Note] Plugin 'INNODB_CHANGED_PAGES' is disabled.
150723 18:59:06 [Note] Plugin 'FEDERATED' is disabled.
150723 18:59:06 [Note] Plugin 'FEEDBACK' is disabled.
150723 18:59:06 [Note] Plugin 'partition' is disabled.
[New Thread 0xf7e87430 (LWP 13194)]
[New Thread 0xf7e27430 (LWP 13195)]

bt
exit;
ERROR: 1064  You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'bt
exit;' at line 1
150723 19:00:29 [ERROR] Aborting

[Thread 0xf7e27430 (LWP 13195) exited]
150723 19:00:29 [Note] InnoDB: FTS optimize thread exiting.
150723 19:00:29 [Note] InnoDB: Starting shutdown...
[Thread 0xda7ff430 (LWP 13193) exited]
[Thread 0xde7ff430 (LWP 13186) exited]
[Thread 0xdb7ff430 (LWP 13191) exited]
[Thread 0xdefff430 (LWP 13185) exited]
[Thread 0xdafff430 (LWP 13192) exited]
[Thread 0xdfbff430 (LWP 13184) exited]
[Thread 0xdd7ff430 (LWP 13188) exited]
[Thread 0xdc7ff430 (LWP 13189) exited]
[Thread 0xddfff430 (LWP 13187) exited]
[Thread 0xe3bff430 (LWP 13176) exited]
[Thread 0xe0bff430 (LWP 13182) exited]
[Thread 0xe53ff430 (LWP 13173) exited]
[Thread 0xe13ff430 (LWP 13181) exited]
[Thread 0xe23ff430 (LWP 13179) exited]
[Thread 0xe4bff430 (LWP 13174) exited]
[Thread 0xe43ff430 (LWP 13175) exited]
[Thread 0xe1bff430 (LWP 13180) exited]
[Thread 0xe33ff430 (LWP 13177) exited]
[Thread 0xe2bff430 (LWP 13178) exited]
[Thread 0xdbfff430 (LWP 13190) exited]

Program received signal SIGSEGV, Segmentation fault.
0x109e8b8c in arena_purge ()
(gdb) bt
#0  0x109e8b8c in arena_purge ()
#1  0x109e8f60 in arena_dalloc_bin_run ()
#2  0x109ea7d8 in je_arena_dalloc_bin_locked ()
#3  0x10a0566c in je_tcache_bin_flush_small ()
#4  0x109df5f8 in free ()
#5  0x10754780 in ut_free (ptr=ptr@entry=0xf77e58e0)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/ut/ut0mem.cc:223
#6  0x1065132c in os_event_free (event=0xf77e58e0)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/os/os0sync.cc:540
#7  0x106517f4 in os_sync_free ()
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/os/os0sync.cc:321
#8  0x10713c04 in innobase_shutdown_for_mysql ()
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/srv/srv0start.cc:3227
#9  0x105be570 in innobase_end (hton=<optimized out>, type=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/handler/ha_innodb.cc:3883
#10 0x1034a458 in ha_finalize_handlerton (plugin=0xf745c368)
    at /home/otto/mariadb-10.0-10.0.20/sql/handler.cc:440
#11 0x101a8e80 in plugin_deinitialize (plugin=0xf745c368, 
    ref_check=ref_check@entry=true)
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1176
#12 0x101acf58 in reap_plugins ()
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1249
#13 0x101ad96c in plugin_shutdown ()
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1895
#14 0x100ef320 in clean_up (print_message=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:2029
#15 0x100f6c54 in unireg_abort (exit_code=1)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:1954
#16 0x100fd178 in mysqld_main (argc=54, argv=0xf7409e30)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:5510
#17 0x100ee8d0 in main (argc=<optimized out>, argv=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/sql/main.cc:25
(gdb) x/10c 0xf77e58e0
0xf77e58e0:	0 '\000'	0 '\000'	0 '\000'	0 '\000'	0 '\000'	0 '\000'	0 '\000'	0 '\000'
0xf77e58e8:	0 '\000'	0 '\000'
(gdb) thr apply all bt full

Thread 23 (Thread 0xf7e87430 (LWP 13194)):
#0  0x0ffd25f0 in do_sigwait () from /lib/powerpc-linux-gnu/libpthread.so.0
No symbol table info available.
#1  0x0ffd26a8 in sigwait () from /lib/powerpc-linux-gnu/libpthread.so.0
No symbol table info available.
#2  0x100f35cc in signal_hand (arg=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:3346
        error = <optimized out>
        set = {__val = {548869, 0 <repeats 31 times>}}
        sig = 0
        _db_stack_frame_ = {func = 0x10b94f78 "?func", file = 0x10b94f80 "?file", 
          level = 2147483649, prev = 0x0}
#3  0x0ffc7c3c in start_thread () from /lib/powerpc-linux-gnu/libpthread.so.0
No symbol table info available.
#4  0x0fb56470 in clone () from /lib/powerpc-linux-gnu/libc.so.6
No symbol table info available.

Thread 1 (Thread 0xf7fe78d0 (LWP 13168)):
#0  0x109e8b8c in arena_purge ()
No symbol table info available.
#1  0x109e8f60 in arena_dalloc_bin_run ()
No symbol table info available.
#2  0x109ea7d8 in je_arena_dalloc_bin_locked ()
No symbol table info available.
#3  0x10a0566c in je_tcache_bin_flush_small ()
No symbol table info available.
#4  0x109df5f8 in free ()
No symbol table info available.
#5  0x10754780 in ut_free (ptr=ptr@entry=0xf77e58e0)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/ut/ut0mem.cc:223
        block = <optimized out>
#6  0x1065132c in os_event_free (event=0xf77e58e0)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/os/os0sync.cc:540
No locals.
#7  0x106517f4 in os_sync_free ()
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/os/os0sync.cc:321
        event = <optimized out>
        mutex = <optimized out>
#8  0x10713c04 in innobase_shutdown_for_mysql ()
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/srv/srv0start.cc:3227
        i = <optimized out>
#9  0x105be570 in innobase_end (hton=<optimized out>, type=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/storage/xtradb/handler/ha_innodb.cc:3883
        err = 0
        _db_stack_frame_ = {func = 0x10ac98d8 "ha_finalize_handlerton", 
          file = 0x10ac9418 "/home/otto/mariadb-10.0-10.0.20/sql/handler.cc", 
          level = 2147483652, prev = 0xfffeea58}
        __PRETTY_FUNCTION__ = "int innobase_end(handlerton*, ha_panic_function)"
#10 0x1034a458 in ha_finalize_handlerton (plugin=0xf745c368)
    at /home/otto/mariadb-10.0-10.0.20/sql/handler.cc:440
        hton = 0xf7416738
        _db_stack_frame_ = {func = 0x10a33bec "plugin_shutdown", 
          file = 0x10a3347c "/home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc", 
          level = 2147483651, prev = 0xfffeebf8}
        __PRETTY_FUNCTION__ = "int ha_finalize_handlerton(st_plugin_int*)"
#11 0x101a8e80 in plugin_deinitialize (plugin=0xf745c368, 
    ref_check=ref_check@entry=true)
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1176
        __PRETTY_FUNCTION__ = "void plugin_deinitialize(st_plugin_int*, bool)"
#12 0x101acf58 in reap_plugins ()
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1249
        count = <optimized out>
        idx = <optimized out>
        reap = 0xfffeeb00
        list = 0xfffeeaec
        __PRETTY_FUNCTION__ = "void reap_plugins()"
        plugin = <optimized out>
#13 0x101ad96c in plugin_shutdown ()
    at /home/otto/mariadb-10.0-10.0.20/sql/sql_plugin.cc:1895
        i = <optimized out>
        count = 47
        dl = <optimized out>
        plugins = <optimized out>
        plugin = <optimized out>
        _db_stack_frame_ = {func = 0x10a1126c "unireg_abort", 
          file = 0x10a107c0 "/home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc", 
          level = 2147483650, prev = 0xfffeec88}
#14 0x100ef320 in clean_up (print_message=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:2029
No locals.
#15 0x100f6c54 in unireg_abort (exit_code=1)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:1954
        _db_stack_frame_ = {func = 0x10b94f78 "?func", file = 0x10b94f80 "?file", 
          level = 2147483649, prev = 0x0}
#16 0x100fd178 in mysqld_main (argc=54, argv=0xf7409e30)
    at /home/otto/mariadb-10.0-10.0.20/sql/mysqld.cc:5510
        ho_error = <optimized out>
#17 0x100ee8d0 in main (argc=<optimized out>, argv=<optimized out>)
    at /home/otto/mariadb-10.0-10.0.20/sql/main.cc:25
No locals.

Gliffy Diagrams

Attachments

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Otto Kekäläinen added a comment - 2015-07-23 23:13

Important remarks:

This is a build regression on powerpc since the 10.0.20 release. See build history at https://buildd.debian.org/status/logs.php?pkg=mariadb-10.0&arch=powerpc
This is reproducible on both Debian sid and jessie, so the fault is most certainly in new MariaDB code (build dependency version changes cannot be blamed)
Platform is powerpc (not Power8 = ppc64el), uname -a = > Linux partch 3.16.0-4-powerpc64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) ppc GNU/Linux
Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792780

When you fix it, please link to the commit that was the fix, so that that commit can be backported as a patch in Debian packaging. We cannot wait for 10.0.21 to release, this should be fixed ASAP.

Show

Otto Kekäläinen added a comment - 2015-07-23 23:13 Important remarks: This is a build regression on powerpc since the 10.0.20 release. See build history at https://buildd.debian.org/status/logs.php?pkg=mariadb-10.0&arch=powerpc This is reproducible on both Debian sid and jessie, so the fault is most certainly in new MariaDB code (build dependency version changes cannot be blamed) Platform is powerpc ( not Power8 = ppc64el), uname -a = > Linux partch 3.16.0-4-powerpc64 #1 SMP Debian 3.16.7-ckt11-1 (2015-05-24) ppc GNU/Linux Debian bug report: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=792780 When you fix it, please link to the commit that was the fix, so that that commit can be backported as a patch in Debian packaging. We cannot wait for 10.0.21 to release, this should be fixed ASAP.

10 older comments

Hide

Permalink

Otto Kekäläinen added a comment - 2015-07-24 14:05

Here is a i386 build log where the test suite failed and it correctly stopped the whole build: https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=i386&ver=10.0.17-1~exp1&stamp=1425382881

I don't know why test suite failures don't stop builds on powerpc. Ideas?

Even a single test failure should always stop the build from completing.

Show

Otto Kekäläinen added a comment - 2015-07-24 14:05 Here is a i386 build log where the test suite failed and it correctly stopped the whole build: https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=i386&ver=10.0.17-1~exp1&stamp=1425382881 I don't know why test suite failures don't stop builds on powerpc. Ideas? Even a single test failure should always stop the build from completing.

Hide

Permalink

Otto Kekäläinen added a comment - 2015-07-27 17:58

Now the build passed successfully: https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=powerpc&ver=10.0.20-3&stamp=1437946958

There was however one failing test: connect.json_udf

Conclusions:

this error has certain randomness
build should not complete if there is even a single test suite failure, otherwise the false positives will trigger false negatives in build status changes later

Show

Otto Kekäläinen added a comment - 2015-07-27 17:58 Now the build passed successfully: https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=powerpc&ver=10.0.20-3&stamp=1437946958 There was however one failing test: connect.json_udf Conclusions: this error has certain randomness build should not complete if there is even a single test suite failure, otherwise the false positives will trigger false negatives in build status changes later

Hide

Permalink

Otto Kekäläinen added a comment - 2015-07-28 09:50

Here is a build log from ppc64 (big endian, just like powerpc?) that has ended in the same way, that is test suite never starts but stalls at "Installing system databases..":

make[5]: Leaving directory '/«PKGBUILDDIR»/builddir'
/usr/bin/make -f mysql-test/CMakeFiles/test-force.dir/build.make mysql-test/CMakeFiles/test-force.dir/build
make[5]: Entering directory '/«PKGBUILDDIR»/builddir'
cd /«PKGBUILDDIR»/builddir/mysql-test && echo Running tests
Running tests
cd /«PKGBUILDDIR»/builddir/mysql-test && echo OS=Linux
OS=Linux
cd /«PKGBUILDDIR»/builddir/mysql-test && export MTR_BUILD_THREAD=auto
cd /«PKGBUILDDIR»/builddir/mysql-test && perl ./mysql-test-run.pl --force --testcase-timeout=30 --suite-timeout=540
Logging: /«PKGBUILDDIR»/mysql-test/mysql-test-run.pl  --force --testcase-timeout=30 --suite-timeout=540
vardir: /«PKGBUILDDIR»/builddir/mysql-test/var
Removing old var directory...
Creating var directory '/«PKGBUILDDIR»/builddir/mysql-test/var'...
Checking supported features...
MariaDB Version 10.0.20-MariaDB-3
 - SSL connections supported
Using suites: main-,archive-,binlog-,csv-,federated-,funcs_1-,funcs_2-,handler-,heap-,innodb-,innodb_fts-,innodb_zip-,maria-,multi_source-,optimizer_unfixed_bugs-,parts-,percona-,perfschema-,plugins-,roles-,rpl-,sys_vars-,unit-,vcol-,connect,oqgraph,sequence,spider,spider/bg,sql_discovery,metadata_lock_info,query_response_time
Collecting tests...
Installing system database...
...
Build killed with signal TERM after 300 minutes of inactivity

http://buildd.debian-ports.org/status/fetch.php?pkg=mariadb-10.0&arch=ppc64&ver=10.0.20-3&stamp=1437975386

Show

Otto Kekäläinen added a comment - 2015-07-28 09:50 Here is a build log from ppc64 (big endian, just like powerpc?) that has ended in the same way, that is test suite never starts but stalls at "Installing system databases..": make[5]: Leaving directory '/«PKGBUILDDIR»/builddir' /usr/bin/make -f mysql-test/CMakeFiles/test-force.dir/build.make mysql-test/CMakeFiles/test-force.dir/build make[5]: Entering directory '/«PKGBUILDDIR»/builddir' cd /«PKGBUILDDIR»/builddir/mysql-test && echo Running tests Running tests cd /«PKGBUILDDIR»/builddir/mysql-test && echo OS=Linux OS=Linux cd /«PKGBUILDDIR»/builddir/mysql-test && export MTR_BUILD_THREAD=auto cd /«PKGBUILDDIR»/builddir/mysql-test && perl ./mysql-test-run.pl --force --testcase-timeout=30 --suite-timeout=540 Logging: /«PKGBUILDDIR»/mysql-test/mysql-test-run.pl --force --testcase-timeout=30 --suite-timeout=540 vardir: /«PKGBUILDDIR»/builddir/mysql-test/var Removing old var directory... Creating var directory '/«PKGBUILDDIR»/builddir/mysql-test/var'... Checking supported features... MariaDB Version 10.0.20-MariaDB-3 - SSL connections supported Using suites: main-,archive-,binlog-,csv-,federated-,funcs_1-,funcs_2-,handler-,heap-,innodb-,innodb_fts-,innodb_zip-,maria-,multi_source-,optimizer_unfixed_bugs-,parts-,percona-,perfschema-,plugins-,roles-,rpl-,sys_vars-,unit-,vcol-,connect,oqgraph,sequence,spider,spider/bg,sql_discovery,metadata_lock_info,query_response_time Collecting tests... Installing system database... ... Build killed with signal TERM after 300 minutes of inactivity http://buildd.debian-ports.org/status/fetch.php?pkg=mariadb-10.0&arch=ppc64&ver=10.0.20-3&stamp=1437975386

Hide

Permalink

Otto Kekäläinen added a comment - 2015-07-28 12:43

Test suite false positives is due to https://github.com/ottok/mariadb-10.0/blob/master/debian/rules#L68

ifneq (,$(filter $(ARCH), amd64 kfreebsd-amd64 i386 kfreebsd-i386 ia64 s390 s390x))
    TESTSUITE_FAIL_CMD:=exit 1
else
    TESTSUITE_FAIL_CMD:=true
endif

This is inherited from mysql-5.6 packaging. Since mysql-5.6 maintainers have even chosen to no fail on any platform due to test suite failures...
https://anonscm.debian.org/cgit/pkg-mysql/mysql-5.6.git/tree/debian/rules?id=refs/heads/master#n28

Show

Otto Kekäläinen added a comment - 2015-07-28 12:43 Test suite false positives is due to https://github.com/ottok/mariadb-10.0/blob/master/debian/rules#L68 ifneq (,$(filter $(ARCH), amd64 kfreebsd-amd64 i386 kfreebsd-i386 ia64 s390 s390x)) TESTSUITE_FAIL_CMD:=exit 1 else TESTSUITE_FAIL_CMD:=true endif This is inherited from mysql-5.6 packaging. Since mysql-5.6 maintainers have even chosen to no fail on any platform due to test suite failures... https://anonscm.debian.org/cgit/pkg-mysql/mysql-5.6.git/tree/debian/rules?id=refs/heads/master#n28

Hide

Permalink

Otto Kekäläinen added a comment - 2015-08-28 11:05

Jan Lindström For the record, ppc64 still crashes randomly when trying to start the test suite. See latest log at https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=ppc64&ver=10.0.21-2&stamp=1440738838

Show

Otto Kekäläinen added a comment - 2015-08-28 11:05 Jan Lindström For the record, ppc64 still crashes randomly when trying to start the test suite. See latest log at https://buildd.debian.org/status/fetch.php?pkg=mariadb-10.0&arch=ppc64&ver=10.0.21-2&stamp=1440738838

People

Assignee:

Jan Lindström

Reporter:

Sergey Vojtovich

Votes:

0 Vote for this issue

Watchers:

4 Start watching this issue

Dates

Created:

2015-07-23 23:02

Updated:

2015-08-31 15:50

Agile

View on Board