Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-8611

mariadb 5.5.45 crashes during 'DROP DATABASE'

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 5.5.43, 5.5.44, 5.5.45
    • Fix Version/s: None
    • Labels:
    • Environment:
      IBM x3650 M3 Centos 6.7 X64

      Description

      mysqld crashes every time during an attempt to DROP a specific database

      core file and other data included in file on the ftp://ftp.askmonty.org/private/mariadb-bug-crash-during-drop-database.tgz

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -
            Version: '5.5.45-MariaDB-log'  socket: '/tmp/mysql.sock'  port: 3306  Source distribution
            InnoDB: Error: trying to load index PRIMARY for table <dbname/tablename>
            InnoDB: but the index tree has been freed!
            150813 10:52:54  InnoDB: Assertion failure in thread 140667958499072 in file row0mysql.c line 3686
            InnoDB: Failing assertion: table
            
            /lib64/libpthread.so.0(+0xf790)[0x7fefcf70f790]
            /lib64/libc.so.6(gsignal+0x35)[0x7fefce5bb625]
            /lib64/libc.so.6(abort+0x175)[0x7fefce5bce05]
            /usr/local/mysql/lib/plugin/ha_innodb.so(+0xd796f)[0x7fefb925096f]
            /usr/local/mysql/lib/plugin/ha_innodb.so(+0xb9c3a)[0x7fefb9232c3a]
            sql/handler.cc:641(dropdb_handlerton)[0x69938f]
            sql/sql_plugin.cc:2344(plugin_foreach_with_mask(THD*, char (*)(THD*, st_plugin_int*, void*), int, unsigned int, void*))[0x584e3d]
            sql/sql_db.cc:856(mysql_rm_db(THD*, char*, bool, bool))[0x558b85]
            sql/sql_parse.cc:3390(mysql_execute_command(THD*))[0x579232]
            sql/sql_parse.cc:5911(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x57d5dc]
            sql/sql_parse.cc:1081(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x57eb68]
            sql/sql_connect.cc:1269(do_handle_one_connection(THD*))[0x624774]
            sql/sql_connect.cc:1187(handle_one_connection)[0x62480a]
            /lib64/libpthread.so.0(+0x7a51)[0x7fefcf707a51]
            /lib64/libc.so.6(clone+0x6d)[0x7fefce6719ad]
            

            As I understand, it most likely indicates tablespace corruption (Jan Lindström, is that right?)

            Benjamin Zwittnig, do you happen to know how the problem started? Did you have some kind of an outage, disk problems, maybe a crash upon altering the table, anything like that?

            Show
            elenst Elena Stepanova added a comment - Version: '5.5.45-MariaDB-log' socket: '/tmp/mysql.sock' port: 3306 Source distribution InnoDB: Error: trying to load index PRIMARY for table <dbname/tablename> InnoDB: but the index tree has been freed! 150813 10:52:54 InnoDB: Assertion failure in thread 140667958499072 in file row0mysql.c line 3686 InnoDB: Failing assertion: table /lib64/libpthread.so.0(+0xf790)[0x7fefcf70f790] /lib64/libc.so.6(gsignal+0x35)[0x7fefce5bb625] /lib64/libc.so.6(abort+0x175)[0x7fefce5bce05] /usr/local/mysql/lib/plugin/ha_innodb.so(+0xd796f)[0x7fefb925096f] /usr/local/mysql/lib/plugin/ha_innodb.so(+0xb9c3a)[0x7fefb9232c3a] sql/handler.cc:641(dropdb_handlerton)[0x69938f] sql/sql_plugin.cc:2344(plugin_foreach_with_mask(THD*, char (*)(THD*, st_plugin_int*, void*), int, unsigned int, void*))[0x584e3d] sql/sql_db.cc:856(mysql_rm_db(THD*, char*, bool, bool))[0x558b85] sql/sql_parse.cc:3390(mysql_execute_command(THD*))[0x579232] sql/sql_parse.cc:5911(mysql_parse(THD*, char*, unsigned int, Parser_state*))[0x57d5dc] sql/sql_parse.cc:1081(dispatch_command(enum_server_command, THD*, char*, unsigned int))[0x57eb68] sql/sql_connect.cc:1269(do_handle_one_connection(THD*))[0x624774] sql/sql_connect.cc:1187(handle_one_connection)[0x62480a] /lib64/libpthread.so.0(+0x7a51)[0x7fefcf707a51] /lib64/libc.so.6(clone+0x6d)[0x7fefce6719ad] As I understand, it most likely indicates tablespace corruption ( Jan Lindström , is that right?) Benjamin Zwittnig , do you happen to know how the problem started? Did you have some kind of an outage, disk problems, maybe a crash upon altering the table, anything like that?
            Hide
            bzwitt Benjamin Zwittnig added a comment -

            Hi,

            Since this is a test database I did 'DROP/CREATE DATABASE
            arnes_20150701' several times. Recreation of the data was done with
            command like 'zcat /data/mysql-arnes-dump-20150701-09:13:01.gz | mysql
            -S /tmp/mysql.sock arnes_20150701'. It happened once or twice that
            during the recreation of the data server went away:

            1. zcat /data/mysql-arnes-dump-20150701-09:13:01.gz | mysql -S
              /tmp/mysql.sock arnes_20150701
              ERROR 2013 (HY000) at line 6745: Lost connection to MySQL server during
              query

            It had helped to recreate the database in question until drop caused
            database crash.

            The server crashed once at the beginning of Aug. It could be power
            outage but I don't know since I was on vacation. After the crash I was
            able to DROP/CREATE the database in question. innodb files reside on 3TB
            FC disk (EMC VNX5300 array made of NL-SAS disks).

            As I remember innodb files were created with mysql-5.x (not mariadb).
            I have found attached messages in error log which dates in Jun 2015 and
            could be related to the problem we have.

            Best regards,

            Benjamin

            Show
            bzwitt Benjamin Zwittnig added a comment - Hi, Since this is a test database I did 'DROP/CREATE DATABASE arnes_20150701' several times. Recreation of the data was done with command like 'zcat /data/mysql-arnes-dump-20150701-09:13:01.gz | mysql -S /tmp/mysql.sock arnes_20150701'. It happened once or twice that during the recreation of the data server went away: zcat /data/mysql-arnes-dump-20150701-09:13:01.gz | mysql -S /tmp/mysql.sock arnes_20150701 ERROR 2013 (HY000) at line 6745: Lost connection to MySQL server during query It had helped to recreate the database in question until drop caused database crash. The server crashed once at the beginning of Aug. It could be power outage but I don't know since I was on vacation. After the crash I was able to DROP/CREATE the database in question. innodb files reside on 3TB FC disk (EMC VNX5300 array made of NL-SAS disks). As I remember innodb files were created with mysql-5.x (not mariadb). I have found attached messages in error log which dates in Jun 2015 and could be related to the problem we have. Best regards, Benjamin
            Hide
            elenst Elena Stepanova added a comment -

            Thank you.

            The error log shows that the server was started on an old datadir (MySQL datadir, I assume), and few hours later the server got stuck dropping the table.
            First question is – do you know whether mysql_upgrade was run between these two events?
            Not running it does not explain the problem right away, but it might be related.

            Please also attach your cnf file(s).

            Show
            elenst Elena Stepanova added a comment - Thank you. The error log shows that the server was started on an old datadir (MySQL datadir, I assume), and few hours later the server got stuck dropping the table. First question is – do you know whether mysql_upgrade was run between these two events? Not running it does not explain the problem right away, but it might be related. Please also attach your cnf file(s).
            Hide
            bzwitt Benjamin Zwittnig added a comment -

            Unfortunately I run mysql_upgrade after I had got problems dropping the
            database.

            Best regards,

            Benjamin

            Show
            bzwitt Benjamin Zwittnig added a comment - Unfortunately I run mysql_upgrade after I had got problems dropping the database. Best regards, Benjamin
            Hide
            elenst Elena Stepanova added a comment -

            Benjamin Zwittnig, do you still have the database which causes the crash?
            I'm afraid without the actual binary data there isn't much we can do to investigate the issue. Similar cases that I can find online suggest ibdata corruption, e.g. this one.
            I suppose if we had the ibdata* files and the schema directory, Jan Lindström could at the very least confirm that the reason is the same here, but otherwise only guessing is possible.

            Show
            elenst Elena Stepanova added a comment - Benjamin Zwittnig , do you still have the database which causes the crash? I'm afraid without the actual binary data there isn't much we can do to investigate the issue. Similar cases that I can find online suggest ibdata corruption, e.g. this one . I suppose if we had the ibdata* files and the schema directory, Jan Lindström could at the very least confirm that the reason is the same here, but otherwise only guessing is possible.

              People

              • Assignee:
                elenst Elena Stepanova
                Reporter:
                bzwitt Benjamin Zwittnig
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: