Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Cannot Reproduce
    • Affects Version/s: 10.0.13
    • Fix Version/s: N/A
    • Component/s: OTHER
    • Labels:
      None
    • Environment:
      CentOS 6.5 64-bit

      Description

      My MariaDB was working fine a few days then it happend:

      140825  0:05:42 [ERROR] mysqld got signal 11 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
      
      To report this bug, see http://kb.askmonty.org/en/reporting-bugs
      
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed,
      something is definitely wrong and this may fail.
      
      Server version: 10.0.13-MariaDB
      key_buffer_size=12884901888
      read_buffer_size=4194304
      max_used_connections=115
      max_threads=501
      thread_count=38
      It is possible that mysqld could use up to
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 16707820 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
      
      Thread pointer: 0x0x7ff042b27008
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7fefcc3b5c58 thread_stack 0x48000
      /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xb7054b]
      /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x727588]
      /lib64/libpthread.so.0(+0xf710)[0x7ff6bd72d710]
      /usr/sbin/mysqld(_ZN8MDL_lock13remove_ticketEMS_NS_11Ticket_listEP10MDL_ticket+0x3d)[0x69fdfd]
      /usr/sbin/mysqld(_ZN11MDL_context12acquire_lockEP11MDL_requestm+0x2a7)[0x6a0cc7]
      /usr/sbin/mysqld[0x58c4a6]
      /usr/sbin/mysqld(_Z10open_tableP3THDP10TABLE_LISTP11st_mem_rootP18Open_table_context+0x4b4)[0x5937c4]
      /usr/sbin/mysqld(_Z11open_tablesP3THDPP10TABLE_LISTPjjP19Prelocking_strategy+0x1317)[0x595227]
      /usr/sbin/mysqld(_Z20open_and_lock_tablesP3THDP10TABLE_LISTbjP19Prelocking_strategy+0x44)[0x595484]
      /usr/sbin/mysqld[0x5ca418]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x4c4f)[0x5d51af]
      /usr/sbin/mysqld[0x5d6cc2]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x1b20)[0x5d8e80]
      /usr/sbin/mysqld(_Z26threadpool_process_requestP3THD+0xa7)[0x6cd937]
      /usr/sbin/mysqld[0x6fc91d]
      /lib64/libpthread.so.0(+0x79d1)[0x7ff6bd7259d1]
      /lib64/libc.so.6(clone+0x6d)[0x7ff6bbe42b5d]
      
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7ff013397020): is an invalid pointer
      Connection ID (thread ID): 6407489
      Status: NOT_KILLED
      
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=on,exists_to_in=on
      
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      140825 00:05:43 mysqld_safe Number of processes running now: 0
      140825 00:05:43 mysqld_safe mysqld restarted
      

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            Spacedust Marek Zakrzewski added a comment -

            It's working fine so far

            Show
            Spacedust Marek Zakrzewski added a comment - It's working fine so far
            Hide
            elenst Elena Stepanova added a comment -

            Thanks for the update.
            I'll have to close it as 'cannot reproduce' for now, but by all means feel free to re-open if you encounter it again. Maybe we'll see some pattern then.

            Show
            elenst Elena Stepanova added a comment - Thanks for the update. I'll have to close it as 'cannot reproduce' for now, but by all means feel free to re-open if you encounter it again. Maybe we'll see some pattern then.
            Hide
            Spacedust Marek Zakrzewski added a comment - - edited

            Another crash when moving whole MariaDB to new machine:

            140929 20:23:40 mysqld_safe mysqld from pid file /var/lib/mysql/online.pl.pid ended
            140929 20:23:59 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql
            140929 20:23:59 [Note] InnoDB: Using mutexes to ref count buffer pool pages
            140929 20:23:59 [Note] InnoDB: The InnoDB memory heap is disabled
            140929 20:23:59 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
            140929 20:23:59 [Note] InnoDB: Compressed tables use zlib 1.2.3
            140929 20:23:59 [Note] InnoDB: Using Linux native AIO
            140929 20:23:59 [Note] InnoDB: Using CPU crc32 instructions
            140929 20:23:59 [Note] InnoDB: Initializing buffer pool, size = 2.0G
            140929 20:24:00 [Note] InnoDB: Completed initialization of buffer pool
            140929 20:24:00 [Note] InnoDB: Highest supported file format is Barracuda.
            140929 20:24:00 [Note] InnoDB: Log scan progressed past the checkpoint lsn 257493140753
            140929 20:24:00 [Note] InnoDB: Database was not shutdown normally!
            140929 20:24:00 [Note] InnoDB: Starting crash recovery.
            140929 20:24:00 [Note] InnoDB: Reading tablespace information from the .ibd files...
            140929 20:24:04 [Note] InnoDB: Restoring possible half-written data pages
            140929 20:24:04 [Note] InnoDB: from the doublewrite buffer...
            InnoDB: Doing recovery: scanned up to log sequence number 257493216556
            140929 20:24:04 [Note] InnoDB: Starting an apply batch of log records to the database...
            InnoDB: Progress in percent: 1 2 140929 20:24:04 [ERROR] mysqld got signal 11 ;
            This could be because you hit a bug. It is also possible that this binary
            or one of the libraries it was linked against is corrupt, improperly built,
            or misconfigured. This error can also be caused by malfunctioning hardware.
            
            To report this bug, see http://kb.askmonty.org/en/reporting-bugs
            
            We will try our best to scrape up some info that will hopefully help
            diagnose the problem, but since we have already crashed,
            something is definitely wrong and this may fail.
            
            Server version: 10.0.13-MariaDB
            key_buffer_size=2147483640
            read_buffer_size=2097152
            max_used_connections=0
            max_threads=501
            thread_count=0
            It is possible that mysqld could use up to
            key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4169964 K  bytes of memory
            Hope that's ok; if not, decrease some variables in the equation.
            
            Thread pointer: 0x0x0
            Attempting backtrace. You can use the following information to find out
            where mysqld died. If you see no messages after this, something went
            terribly wrong...
            3 4 stack_bottom = 0x0 thread_stack 0x48000
            5 6 7 8 9 10 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xb7054b]
            11 /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x727588]
            /lib64/libpthread.so.0(+0xf710)[0x7f7b06ace710]
            /usr/sbin/mysqld[0x8d8799]
            12 /usr/sbin/mysqld[0x8d7802]
            /usr/sbin/mysqld[0x8d8035]
            13 /usr/sbin/mysqld[0x8be5b1]
            /usr/sbin/mysqld[0x8c0294]
            14 /usr/sbin/mysqld[0x99cbc6]
            /usr/sbin/mysqld[0x9e4ae6]
            15 /usr/sbin/mysqld[0x93a455]
            /lib64/libpthread.so.0(+0x79d1)[0x7f7b06ac69d1]
            /lib64/libc.so.6(clone+0x6d)[0x7f7b051e386d]
            The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
            information that should help you find out what is causing the crash.
            140929 20:24:04 mysqld_safe mysqld from pid file /var/lib/mysql/onlinecity.pl.pid ended
            
            Show
            Spacedust Marek Zakrzewski added a comment - - edited Another crash when moving whole MariaDB to new machine: 140929 20:23:40 mysqld_safe mysqld from pid file /var/lib/mysql/online.pl.pid ended 140929 20:23:59 mysqld_safe Starting mysqld daemon with databases from /var/lib/mysql 140929 20:23:59 [Note] InnoDB: Using mutexes to ref count buffer pool pages 140929 20:23:59 [Note] InnoDB: The InnoDB memory heap is disabled 140929 20:23:59 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 140929 20:23:59 [Note] InnoDB: Compressed tables use zlib 1.2.3 140929 20:23:59 [Note] InnoDB: Using Linux native AIO 140929 20:23:59 [Note] InnoDB: Using CPU crc32 instructions 140929 20:23:59 [Note] InnoDB: Initializing buffer pool, size = 2.0G 140929 20:24:00 [Note] InnoDB: Completed initialization of buffer pool 140929 20:24:00 [Note] InnoDB: Highest supported file format is Barracuda. 140929 20:24:00 [Note] InnoDB: Log scan progressed past the checkpoint lsn 257493140753 140929 20:24:00 [Note] InnoDB: Database was not shutdown normally! 140929 20:24:00 [Note] InnoDB: Starting crash recovery. 140929 20:24:00 [Note] InnoDB: Reading tablespace information from the .ibd files... 140929 20:24:04 [Note] InnoDB: Restoring possible half-written data pages 140929 20:24:04 [Note] InnoDB: from the doublewrite buffer... InnoDB: Doing recovery: scanned up to log sequence number 257493216556 140929 20:24:04 [Note] InnoDB: Starting an apply batch of log records to the database... InnoDB: Progress in percent: 1 2 140929 20:24:04 [ERROR] mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see http://kb.askmonty.org/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.0.13-MariaDB key_buffer_size=2147483640 read_buffer_size=2097152 max_used_connections=0 max_threads=501 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 4169964 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... 3 4 stack_bottom = 0x0 thread_stack 0x48000 5 6 7 8 9 10 /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xb7054b] 11 /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x727588] /lib64/libpthread.so.0(+0xf710)[0x7f7b06ace710] /usr/sbin/mysqld[0x8d8799] 12 /usr/sbin/mysqld[0x8d7802] /usr/sbin/mysqld[0x8d8035] 13 /usr/sbin/mysqld[0x8be5b1] /usr/sbin/mysqld[0x8c0294] 14 /usr/sbin/mysqld[0x99cbc6] /usr/sbin/mysqld[0x9e4ae6] 15 /usr/sbin/mysqld[0x93a455] /lib64/libpthread.so.0(+0x79d1)[0x7f7b06ac69d1] /lib64/libc.so.6(clone+0x6d)[0x7f7b051e386d] The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. 140929 20:24:04 mysqld_safe mysqld from pid file /var/lib/mysql/onlinecity.pl.pid ended
            Hide
            elenst Elena Stepanova added a comment -

            The crash looks very different though.
            It happened on startup, while InnoDB was applying the log after an apparently dirty shutdown. I'm afraid it's going to be impossible to find out what happened there, since the stack trace is useless.
            I assume you tried to restart the server after the crash, did it start all right, or did you have to do something to get it up and running?
            Do you know what caused the previous dirty shutdown?

            Show
            elenst Elena Stepanova added a comment - The crash looks very different though. It happened on startup, while InnoDB was applying the log after an apparently dirty shutdown. I'm afraid it's going to be impossible to find out what happened there, since the stack trace is useless. I assume you tried to restart the server after the crash, did it start all right, or did you have to do something to get it up and running? Do you know what caused the previous dirty shutdown?
            Hide
            elenst Elena Stepanova added a comment -

            Closing as "Can't reproduce" for now, because, sadly, there's nothing even to start from. If you get any more information, please comment to re-open.

            Show
            elenst Elena Stepanova added a comment - Closing as "Can't reproduce" for now, because, sadly, there's nothing even to start from. If you get any more information, please comment to re-open.

              People

              • Assignee:
                elenst Elena Stepanova
                Reporter:
                Spacedust Marek Zakrzewski
              • Votes:
                1 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Due:
                  Created:
                  Updated:
                  Resolved: