Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-6590

InnoDB recover ends in signal 11

        Details

        • Type: Bug
        • Status: Closed
        • Priority: Major
        • Resolution: Fixed
        • Affects Version/s: 10.0.13
        • Fix Version/s: 10.0.14
        • Component/s: None
        • Labels:
        • Environment:

          Description

          Some days ago, I need to reset my local system (press the reset button) because it was frozen.
          After reboot, mariadb won't start up, here is the output from the log file:

          140815 15:09:42 [Warning] No argument was provided to --log-bin and neither --log-basename or --log-bin-index where used;  This may cause repliction to break when this server acts as a master and has its hostname changed! Please use '--log-basename=exa' or '--log-bin=mysqld-bin' to avoid this problem.
140815 15:09:42 [Note] CONNECT: Version 1.03.0002 Aug 15 2014 15:05:37
140815 15:09:42 [Note] InnoDB: Using mutexes to ref count buffer pool pages
140815 15:09:42 [Note] InnoDB: The InnoDB memory heap is disabled
140815 15:09:42 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
140815 15:09:42 [Note] InnoDB: Compressed tables use zlib 1.2.8
140815 15:09:42 [Note] InnoDB: Using Linux native AIO
140815 15:09:42 [Note] InnoDB: Using CPU crc32 instructions
140815 15:09:42 [Note] InnoDB: Initializing buffer pool, size = 64.0M
140815 15:09:42 [Note] InnoDB: Completed initialization of buffer pool
140815 15:09:42 [Note] InnoDB: Highest supported file format is Barracuda.
140815 15:09:42 [Note] InnoDB: Log scan progressed past the checkpoint lsn 3359224610
140815 15:09:42 [Note] InnoDB: Database was not shutdown normally!
140815 15:09:42 [Note] InnoDB: Starting crash recovery.
140815 15:09:42 [Note] InnoDB: Reading tablespace information from the .ibd files...
140815 15:09:42 [Note] InnoDB: Restoring possible half-written data pages 
140815 15:09:42 [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number 3359225074
140815 15:09:42 [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent: 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position 0 203636560, file name ./mysqld-bin.000039
140815 15:09:42 [ERROR] mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.

To report this bug, see http://kb.askmonty.org/en/reporting-bugs

We will try our best to scrape up some info that will hopefully help
diagnose the problem, but since we have already crashed, 
something is definitely wrong and this may fail.

Server version: 10.0.13-MariaDB-log
key_buffer_size=67108864
read_buffer_size=1048576
max_used_connections=0
max_threads=153
thread_count=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 538697 K  bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

Thread pointer: 0x0x0
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0x0 thread_stack 0x48000
/usr/sbin/mysqld(my_print_stacktrace+0x29)[0xb30f79]
/usr/sbin/mysqld(handle_fatal_signal+0x398)[0x6f2658]
/lib64/libpthread.so.0(+0x11260)[0x7f3c9564a260]
:0(dict_get_first_path(unsigned long, char const*))[0x9d4a08]
:0(dict_check_tablespaces_and_store_max_id(dict_check_t))[0x9d59a4]
:0(innobase_start_or_create_for_mysql())[0x9533f1]
ha_innodb.cc:0(innobase_init(void*))[0x8ad168]
/usr/sbin/mysqld(_Z24ha_initialize_handlertonP13st_plugin_int+0x57)[0x6f4797]
sql_plugin.cc:0(plugin_initialize(st_mem_root*, st_plugin_int*, int*, char**, bool))[0x5ce8ca]
/usr/sbin/mysqld(_Z11plugin_initPiPPci+0x4ab)[0x5cf4ab]
mysqld.cc:0(init_server_components())[0x5492a7]
/usr/sbin/mysqld(_Z11mysqld_mainiPPc+0x3e0)[0x54d290]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f3c94255db5]
/usr/sbin/mysqld[0x5430e5]
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

          The startup fails every time I try.
          Moving /var/lib/mysql away and reinstall the tables works, so it seems to be a problem with the existing tables.

            Gliffy Diagrams

              Attachments

                Activity

                Ascending order - Click to sort in descending order
                Hide
                Permalink
                jplindst Jan Lindström added a comment -

                Hi,

                Do you still have the broken database somewhere, can you download it here ? Code tries to find filepath for a spaceid from SYS_DATAFILES and checks it against the contents of a link file, but something goes wrong.

                Show
                jplindst Jan Lindström added a comment - Hi, Do you still have the broken database somewhere, can you download it here ? Code tries to find filepath for a spaceid from SYS_DATAFILES and checks it against the contents of a link file, but something goes wrong.
                Hide
                Permalink
                tristan.lins Tristan Lins added a comment -

                Yes I still have the broken database files.
                I will send you an E-Mail with a download link later (the database contains some sensitive data). I'm still packing the archive and upload it to my server.

                Show
                tristan.lins Tristan Lins added a comment - Yes I still have the broken database files. I will send you an E-Mail with a download link later (the database contains some sensitive data). I'm still packing the archive and upload it to my server.
                Hide
                Permalink
                jplindst Jan Lindström added a comment -

                Hi,

                Have you migrated database from some older version, for some reason InnoDB data dictionary is missing table SYS_DATAFILES.

                Show
                jplindst Jan Lindström added a comment - Hi, Have you migrated database from some older version, for some reason InnoDB data dictionary is missing table SYS_DATAFILES.
                Hide
                Permalink
                tristan.lins Tristan Lins added a comment -

                Mh, yes I have transfered the database from my old laptop. If I remember correctly I have currently installed MariaDB 5.6 on it. It was already migrated from an MySQL 5.5 as I remember correctly.

                Show
                tristan.lins Tristan Lins added a comment - Mh, yes I have transfered the database from my old laptop. If I remember correctly I have currently installed MariaDB 5.6 on it. It was already migrated from an MySQL 5.5 as I remember correctly.
                Hide
                Permalink
                jplindst Jan Lindström added a comment -

                revno: 4362
                committer: Jan Lindström <jplindst@mariadb.org>
                branch nick: 10.0-innodb
                timestamp: Mon 2014-08-25 13:35:33 +0300
                message:
                MDEV-6590: InnoDB recover ends in signal 11

                Analysis: When database is migrated from 5.5 or earlier and
                database needs crash recovery, there is possibility that
                SYS_DATAFILES system table does not exists, but
                crash recovery in function dict_check_tablespaces_and_store_max_id()
                assumes that SYS_DATAFILES exists.

                Fix: If SYS_DATAFILES does not exists, create it before
                we end up to function dict_check_tablespaces_and_store_max_id()
                on crash recovery.

                Show
                jplindst Jan Lindström added a comment - revno: 4362 committer: Jan Lindström <jplindst@mariadb.org> branch nick: 10.0-innodb timestamp: Mon 2014-08-25 13:35:33 +0300 message: MDEV-6590 : InnoDB recover ends in signal 11 Analysis: When database is migrated from 5.5 or earlier and database needs crash recovery, there is possibility that SYS_DATAFILES system table does not exists, but crash recovery in function dict_check_tablespaces_and_store_max_id() assumes that SYS_DATAFILES exists. Fix: If SYS_DATAFILES does not exists, create it before we end up to function dict_check_tablespaces_and_store_max_id() on crash recovery.
                Hide
                Permalink
                tristan.lins Tristan Lins added a comment -

                Yeah, my database is running, great work!

                Show
                tristan.lins Tristan Lins added a comment - Yeah, my database is running, great work!

                  People

                  • Assignee:
                    jplindst Jan Lindström
                    Reporter:
                    tristan.lins Tristan Lins
                  • Votes:
                    0 Vote for this issue
                    Watchers:
                    4 Start watching this issue

                    Dates

                    • Created:
                      Updated:
                      Resolved:

                      Time Tracking

                      Estimated:
                      Original Estimate - Not Specified
                      Not Specified
                      Remaining:
                      Remaining Estimate - 0 minutes
                      0m
                      Logged:
                      Time Spent - 2 hours
                      2h