Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-2516

LP:902654 - MariaDB consistently crashes in collect_tables on Aria checkpoint execution

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      Hi,
      This issue occurs on MariaDB 5.2.10 and 5.3.2-beta

      OS info:
      $ cat /etc/redhat-release
      Red Hat Enterprise Linux Server release 5.6 (Tikanga)

      Memory (with MariaDB running):
      $ free -m
      total used free shared buffers cached
      Mem: 16050 14153 1897 0 242 12572
      -/+ buffers/cache: 1338 14712
      Swap: 4095 89 4006

      This occurs just about every 5 minutes on the dot, but sometimes it crashes out of band. mysqld_safe continuously restarts it.

      Attached info includes:
      cpuinfo (from /proc)
      mysql table creation script
      my.cnf
      mysqld.log

      To repro, simply create a db, run the data_tables.sql script to create all the tables and it will start to crash.

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Thank you, now I can reproduce the problem.

            I am attaching an MTR testcase, it is based on the flow that Scott provided, only converted in mysqltest format and reduced. The test fails for me reliably on 5.2 and 5.3 debug server. If it doesn't cause the crash, try the algorithm suggested by Scott, it worked for me too.

            Debug assertion:

            mysqld: ma_checkpoint.c:949: collect_tables: Assertion `state_copy < state_copies_end' failed

            #8 0xb7571014 in __assert_fail () from /lib/libc.so.6
            #9 0x0860aba7 in collect_tables (str=0xb754a23c, checkpoint_start_log_horizon=4300787999)
            at ma_checkpoint.c:949
            #10 0x086097a8 in really_execute_checkpoint () at ma_checkpoint.c:198
            #11 0x086095c1 in ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:132
            #12 0x0860a310 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621
            #13 0xb77ffb25 in start_thread () from /lib/libpthread.so.0

            Release crash:

            #3 0x0825ab96 in handle_segfault (sig=11) at mysqld.cc:2838
            #4 <signal handler called>
            #5 0x0847fb3a in collect_tables (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:948
            #6 really_execute_checkpoint (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:198
            #7 ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:132
            #8 0x08480716 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621
            #9 0xb77c9b25 in start_thread () from /lib/libpthread.so.0

            bug902654.test
            LPexportBug902654_bug902654.test

            Show
            elenst Elena Stepanova added a comment - Thank you, now I can reproduce the problem. I am attaching an MTR testcase, it is based on the flow that Scott provided, only converted in mysqltest format and reduced. The test fails for me reliably on 5.2 and 5.3 debug server. If it doesn't cause the crash, try the algorithm suggested by Scott, it worked for me too. Debug assertion: mysqld: ma_checkpoint.c:949: collect_tables: Assertion `state_copy < state_copies_end' failed #8 0xb7571014 in __assert_fail () from /lib/libc.so.6 #9 0x0860aba7 in collect_tables (str=0xb754a23c, checkpoint_start_log_horizon=4300787999) at ma_checkpoint.c:949 #10 0x086097a8 in really_execute_checkpoint () at ma_checkpoint.c:198 #11 0x086095c1 in ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:132 #12 0x0860a310 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621 #13 0xb77ffb25 in start_thread () from /lib/libpthread.so.0 Release crash: #3 0x0825ab96 in handle_segfault (sig=11) at mysqld.cc:2838 #4 <signal handler called> #5 0x0847fb3a in collect_tables (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:948 #6 really_execute_checkpoint (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:198 #7 ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:132 #8 0x08480716 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621 #9 0xb77c9b25 in start_thread () from /lib/libpthread.so.0 bug902654.test LPexportBug902654_bug902654.test
            Hide
            elenst Elena Stepanova added a comment -

            Re: MariaDB consistently crashes
            Thank you, now I can reproduce the problem.

            I am attaching an MTR testcase, it is based on the flow that Scott provided, only converted in mysqltest format and reduced. The test fails for me reliably on 5.2 and 5.3 debug server. If it doesn't cause the crash, try the algorithm suggested by Scott, it worked for me too.

            Debug assertion:

            mysqld: ma_checkpoint.c:949: collect_tables: Assertion `state_copy < state_copies_end' failed

            #8 0xb7571014 in __assert_fail () from /lib/libc.so.6
            #9 0x0860aba7 in collect_tables (str=0xb754a23c, checkpoint_start_log_horizon=4300787999)
            at ma_checkpoint.c:949
            #10 0x086097a8 in really_execute_checkpoint () at ma_checkpoint.c:198
            #11 0x086095c1 in ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:132
            #12 0x0860a310 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621
            #13 0xb77ffb25 in start_thread () from /lib/libpthread.so.0

            Release crash:

            #3 0x0825ab96 in handle_segfault (sig=11) at mysqld.cc:2838
            #4 <signal handler called>
            #5 0x0847fb3a in collect_tables (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:948
            #6 really_execute_checkpoint (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:198
            #7 ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001')
            at ma_checkpoint.c:132
            #8 0x08480716 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621
            #9 0xb77c9b25 in start_thread () from /lib/libpthread.so.0

            Show
            elenst Elena Stepanova added a comment - Re: MariaDB consistently crashes Thank you, now I can reproduce the problem. I am attaching an MTR testcase, it is based on the flow that Scott provided, only converted in mysqltest format and reduced. The test fails for me reliably on 5.2 and 5.3 debug server. If it doesn't cause the crash, try the algorithm suggested by Scott, it worked for me too. Debug assertion: mysqld: ma_checkpoint.c:949: collect_tables: Assertion `state_copy < state_copies_end' failed #8 0xb7571014 in __assert_fail () from /lib/libc.so.6 #9 0x0860aba7 in collect_tables (str=0xb754a23c, checkpoint_start_log_horizon=4300787999) at ma_checkpoint.c:949 #10 0x086097a8 in really_execute_checkpoint () at ma_checkpoint.c:198 #11 0x086095c1 in ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:132 #12 0x0860a310 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621 #13 0xb77ffb25 in start_thread () from /lib/libpthread.so.0 Release crash: #3 0x0825ab96 in handle_segfault (sig=11) at mysqld.cc:2838 #4 <signal handler called> #5 0x0847fb3a in collect_tables (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:948 #6 really_execute_checkpoint (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:198 #7 ma_checkpoint_execute (level=CHECKPOINT_MEDIUM, no_wait=1 '\001') at ma_checkpoint.c:132 #8 0x08480716 in ma_checkpoint_background (arg=0x1e) at ma_checkpoint.c:621 #9 0xb77c9b25 in start_thread () from /lib/libpthread.so.0
            Hide
            monty Michael Widenius added a comment -

            Re: MariaDB consistently crashes in collect_tables on Aria checkpoint execution
            The bug happens when you have more than 1024 open Aria tables during checkpoint.
            This is now fixed in 5.2. Fix will be in 5.2.11.
            Will push to 5.3 today.

            Show
            monty Michael Widenius added a comment - Re: MariaDB consistently crashes in collect_tables on Aria checkpoint execution The bug happens when you have more than 1024 open Aria tables during checkpoint. This is now fixed in 5.2. Fix will be in 5.2.11. Will push to 5.3 today.
            Hide
            scottfeldstein Scott Feldstein added a comment -

            Re: MariaDB consistently crashes in collect_tables on Aria checkpoint execution
            Just fyi, been running maria 5.3.5 GA for several days now and it is stable.

            I'm very impressed with the quick turnover. Thanks!

            Show
            scottfeldstein Scott Feldstein added a comment - Re: MariaDB consistently crashes in collect_tables on Aria checkpoint execution Just fyi, been running maria 5.3.5 GA for several days now and it is stable. I'm very impressed with the quick turnover. Thanks!
            Hide
            ratzpo Rasmus Johansson added a comment -

            Launchpad bug id: 902654

            Show
            ratzpo Rasmus Johansson added a comment - Launchpad bug id: 902654

              People

              • Assignee:
                monty Michael Widenius
                Reporter:
                scottfeldstein Scott Feldstein
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: