Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 5.5.37, 5.5.38
    • Fix Version/s: 5.5.39, 10.0.13
    • Component/s: None
    • Labels:
    • Environment:
      Ubuntu 12.04.4 LTS
      Linux debrecen 3.2.0-60-generic #91-Ubuntu SMP Wed Feb 19 03:54:44 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

      Description

      From mariaddb 5.5.33a mariadb crash after one day uptime
      (with 5.5.33a dosnt crash).
      I uploaded to ftp.askmonty.org crash report and core files name: mariadb-bug-crash-5.5.38-2014060161434_02.tgz

      Error log :

      Server version: 5.5.38-MariaDB-1~precise-log
      key_buffer_size=3221225470
      read_buffer_size=8388608
      max_used_connections=77
      max_threads=122
      thread_count=73
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 7270511 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
      
      Thread pointer: 0x0x7f31e15f1000
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f30f63fee20 thread_stack 0xc8000
      /usr/sbin/mysqld(my_print_stacktrace+0x2b)[0xa7499b]
      /usr/sbin/mysqld(handle_fatal_signal+0x398)[0x6ccf88]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0xfcb0)[0x7f399f14fcb0]
      /lib/x86_64-linux-gnu/libc.so.6(_IO_vfprintf+0x29f1)[0x7f399d93a3b1]
      /lib/x86_64-linux-gnu/libc.so.6(__fprintf_chk+0xeb)[0x7f399d9f880b]
      /usr/sbin/mysqld[0x846dcc]
      /usr/sbin/mysqld[0x847fda]
      /usr/sbin/mysqld[0x849778]
      /usr/sbin/mysqld[0x83ec6f]
      /usr/sbin/mysqld[0x8095bc]
      /usr/sbin/mysqld(_Z14ha_show_statusP3THDP10handlerton12ha_stat_type+0x217)[0x6d4017]
      /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x3f13)[0x59fcc3]
      /usr/sbin/mysqld[0x5a1934]
      /usr/sbin/mysqld(_Z16dispatch_command19enum_server_commandP3THDPcj+0x159f)[0x5a301f]
      /usr/sbin/mysqld(_Z24do_handle_one_connectionP3THD+0x1fb)[0x65323b]
      /usr/sbin/mysqld(handle_one_connection+0x4c)[0x6532bc]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x7e9a)[0x7f399f147e9a]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7f399d9e33fd]
      
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query (0x7f3171026018): SHOW /*!50000 ENGINE*/ INNODB STATUS
      Connection ID (thread ID): 141550
      Status: NOT_KILLED
      
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,engine_condition_pushdown=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on,extended_keys=off
      
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file 
      

        Gliffy Diagrams

          Attachments

          1. cd1
            113 kB
          2. cd2
            278 kB
          3. cd5
            203 kB
          4. cd6
            435 kB
          5. cd7
            230 kB
          6. cd8
            520 kB
          7. log
            6 kB

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Thanks a lot for provided stack traces, we'll see if we can figure out the problem from there.
            Please do attach the log that munin produced if you still have it, and also your cnf file(s), I suppose they haven't changed after the downgrade to 5.5.33a,

            By SHOW GLOBAL STATUS I meant the literal command, it's different from SHOW ENGINE INNODB STATUS, and provides information about what's going on with the server in general. If you can enable it (e.g. if munin allows it), please do, at least for a day, and attach the log, it might be useful even though you are not getting crashes. If you can't do that, we'll try to go without it.

            Show
            elenst Elena Stepanova added a comment - Thanks a lot for provided stack traces, we'll see if we can figure out the problem from there. Please do attach the log that munin produced if you still have it, and also your cnf file(s), I suppose they haven't changed after the downgrade to 5.5.33a, By SHOW GLOBAL STATUS I meant the literal command, it's different from SHOW ENGINE INNODB STATUS , and provides information about what's going on with the server in general. If you can enable it (e.g. if munin allows it), please do, at least for a day, and attach the log, it might be useful even though you are not getting crashes. If you can't do that, we'll try to go without it.
            Hide
            jplindst Jan Lindström added a comment -

            Hi,

            Error: 140630 19:03:41 InnoDB: compressed page checksum mismatch (space 112227 page 33844): 33554432 != 4127160225

            Is indication of database corruption on either in-memory or on storage. This is bad, but there have been several complains about these compressed tables, that there is someting wrong with current implementation. Can you avoid using them ?

            The other crashes are related to long semaphore waits, there is something wrong with info output code, I will try to find out where and fix it.

            R: Jan

            Show
            jplindst Jan Lindström added a comment - Hi, Error: 140630 19:03:41 InnoDB: compressed page checksum mismatch (space 112227 page 33844): 33554432 != 4127160225 Is indication of database corruption on either in-memory or on storage. This is bad, but there have been several complains about these compressed tables, that there is someting wrong with current implementation. Can you avoid using them ? The other crashes are related to long semaphore waits, there is something wrong with info output code, I will try to find out where and fix it. R: Jan
            Hide
            bulepage bulepage added a comment -

            Hi,
            After Sig 6 I got InnoDB: Warning: CHECK TABLE on index `PRIMARY` of table `merleg`.`lista` returned 12.
            merleg.lista is a compressed table.
            After
            alter table merleg.lista engine innodb;
            table is fine.

            Show
            bulepage bulepage added a comment - Hi, After Sig 6 I got InnoDB: Warning: CHECK TABLE on index `PRIMARY` of table `merleg`.`lista` returned 12. merleg.lista is a compressed table. After alter table merleg.lista engine innodb; table is fine.
            Hide
            jplindst Jan Lindström added a comment -

            revno: 4225
            committer: Jan Lindström <jplindst@mariadb.org>
            branch nick: 5.5
            timestamp: Tue 2014-07-08 17:21:13 +0300
            message:
            MDEV-6348: mariadb crash signal 11

            Analysis: sync array output function, should make sure that all
            used pointers are valid before using them.

            Show
            jplindst Jan Lindström added a comment - revno: 4225 committer: Jan Lindström <jplindst@mariadb.org> branch nick: 5.5 timestamp: Tue 2014-07-08 17:21:13 +0300 message: MDEV-6348 : mariadb crash signal 11 Analysis: sync array output function, should make sure that all used pointers are valid before using them.
            Hide
            jplindst Jan Lindström added a comment -

            revno: 4274
            committer: Jan Lindström <jplindst@mariadb.org>
            branch nick: 10.0-innodb
            timestamp: Tue 2014-07-08 18:51:34 +0300
            message:
            MDEV-6348: mariadb crash signal 11

            Analysis: sync array output function, should make sure that all
            used pointers are valid before using them.

            Merge revision 4225 from lp:maria/5.5.

            Show
            jplindst Jan Lindström added a comment - revno: 4274 committer: Jan Lindström <jplindst@mariadb.org> branch nick: 10.0-innodb timestamp: Tue 2014-07-08 18:51:34 +0300 message: MDEV-6348 : mariadb crash signal 11 Analysis: sync array output function, should make sure that all used pointers are valid before using them. Merge revision 4225 from lp:maria/5.5.

              People

              • Assignee:
                jplindst Jan Lindström
                Reporter:
                bulepage bulepage
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 6 hours
                  6h