Uploaded image for project: 'MariaDB Server'
  1. MariaDB Server
  2. MDEV-5411

Sporadic failure in rpl.rpl_relayrotate 'row': Assertion `m_status == DA_ERROR' failed

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 5.3.12
    • Fix Version/s: 5.3.13
    • Component/s: None
    • Labels:

      Description

      The test failed in buildbot on 5.3 tree revno 3734. Retry passed, and the failure is not reproducible by re-running the test separately, but it is something to watch further.

      http://buildbot.askmonty.org/buildbot/builders/kvm-dgcov-jaunty-i386/builds/3408/steps/test_2/logs/stdio

      mysql-test-run.pl  --mem --testcase-timeout=45 --parallel=2 --force --retry=3 --max-save-core=0 --max-save-datadir=1
      
      rpl.rpl_relayrotate 'row'                w2 [ fail ]
              Test ended at 2013-12-07 19:11:49
      
      CURRENT_TEST: rpl.rpl_relayrotate
      mysqltest: In included file "./extra/rpl_tests/rpl_relayrotate.test": 
      included from /home/buildbot/build/mysql-test/suite/rpl/t/rpl_relayrotate.test at line 15:
      At line 47: query 'stop slave' failed: 2013: Lost connection to MySQL server during query
      
      The result from queries just before the failure was:
      include/master-slave.inc
      [connection master]
      stop slave;
      create table t1 (a int) engine=innodb;
      reset slave;
      start slave;
      stop slave;
      
      
      Server [mysqld.2 - pid: 23647, winpid: 23647, exit: 256] failed during test run
      Server log from this test:
      ----------SERVER LOG START-----------
      131207 19:11:40 [Warning] 'master-host' is deprecated and will be removed in a future release. Please use ''CHANGE MASTER'' instead.
      131207 19:11:40 [Warning] /home/buildbot/build/sql/mysqld: Option '-O' is deprecated. Use --variable-name=value instead.
      InnoDB: The InnoDB memory heap is disabled
      InnoDB: Mutexes and rw_locks use GCC atomic builtins
      InnoDB: Compressed tables use zlib 1.2.3
      131207 19:11:40  InnoDB: Initializing buffer pool, size = 128.0M
      131207 19:11:40  InnoDB: Completed initialization of buffer pool
      InnoDB: The first specified data file ./ibdata1 did not exist:
      InnoDB: a new database to be created!
      131207 19:11:40  InnoDB: Setting file ./ibdata1 size to 10 MB
      InnoDB: Database physically writes the file full: wait...
      131207 19:11:40  InnoDB: Log file ./ib_logfile0 did not exist: new to be created
      InnoDB: Setting log file ./ib_logfile0 size to 5 MB
      InnoDB: Database physically writes the file full: wait...
      131207 19:11:40  InnoDB: Log file ./ib_logfile1 did not exist: new to be created
      InnoDB: Setting log file ./ib_logfile1 size to 5 MB
      InnoDB: Database physically writes the file full: wait...
      InnoDB: Doublewrite buffer not found: creating new
      InnoDB: Doublewrite buffer created
      InnoDB: Creating foreign key constraint system tables
      InnoDB: Foreign key constraint system tables created
      131207 19:11:40 Percona XtraDB (http://www.percona.com) 1.0.17-14.1 started; log sequence number 0
      131207 19:11:40 [Warning] /home/buildbot/build/sql/mysqld: unknown option '--loose-skip-pbxt'
      131207 19:11:40 [Warning] /home/buildbot/build/sql/mysqld: unknown option '--loose-skip-feedback'
      131207 19:11:40 [Warning] /home/buildbot/build/sql/mysqld: unknown variable 'loose-feedback-user-info=mysql-test'
      131207 19:11:40 [Note] Event Scheduler: Loaded 0 events
      131207 19:11:40 [Note] /home/buildbot/build/sql/mysqld: ready for connections.
      Version: '5.3.13-MariaDB-debug-log'  socket: '/home/buildbot/build/mysql-test/var/tmp/2/mysqld.2.sock'  port: 16002  Source distribution
      131207 19:11:41 [Note] 'CHANGE MASTER TO executed'. Previous state master_host='127.0.0.1', master_port='16000', master_log_file='', master_log_pos='4'. New state master_host='127.0.0.1', master_port='16000', master_log_file='master-bin.000001', master_log_pos='4'.
      131207 19:11:41 [Note] Slave SQL thread initialized, starting replication in log 'master-bin.000001' at position 4, relay log './slave-relay-bin.000001' position: 4
      131207 19:11:41 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16000',replication started in log 'master-bin.000001' at position 4
      131207 19:11:42 [Note] Error reading relay log event: slave SQL thread was killed
      131207 19:11:42 [Note] Slave I/O thread killed while reading event
      131207 19:11:42 [Note] Slave I/O thread exiting, read up to log 'master-bin.000001', position 245
      131207 19:11:47 [Note] Slave I/O thread: connected to master 'root@127.0.0.1:16000',replication started in log 'FIRST' at position 4
      131207 19:11:47 [Note] Slave SQL thread initialized, starting replication in log 'FIRST' at position 0, relay log './slave-relay-bin.000001' position: 4
      mysqld: sql_class.h:1356: uint Diagnostics_area::sql_errno() const: Assertion `m_status == DA_ERROR' failed.
      131207 19:11:48 [ERROR] mysqld got signal 6 ;
      This could be because you hit a bug. It is also possible that this binary
      or one of the libraries it was linked against is corrupt, improperly built,
      or misconfigured. This error can also be caused by malfunctioning hardware.
      
      To report this bug, see http://kb.askmonty.org/en/reporting-bugs
      
      We will try our best to scrape up some info that will hopefully help
      diagnose the problem, but since we have already crashed, 
      something is definitely wrong and this may fail.
      
      Server version: 5.3.13-MariaDB-debug-log
      key_buffer_size=1048576
      read_buffer_size=131072
      max_used_connections=4
      max_threads=152
      thread_count=4
      connection_count=4
      It is possible that mysqld could use up to 
      key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61468 K  bytes of memory
      Hope that's ok; if not, decrease some variables in the equation.
      
      Thread pointer: 0x0x3710eb0
      Attempting backtrace. You can use the following information to find out
      where mysqld died. If you see no messages after this, something went
      terribly wrong...
      stack_bottom = 0x7f11700a69f0 thread_stack 0x48000
      /home/buildbot/build/sql/mysqld(my_print_stacktrace+0x47) [0x1020eaf]
      /home/buildbot/build/sql/mysqld(handle_fatal_signal+0x4df) [0xa47603]
      /lib/libpthread.so.0(+0xf8f0) [0x7f116f5388f0]
      /lib/libc.so.6(gsignal+0x35) [0x7f116e823a75]
      /lib/libc.so.6(abort+0x180) [0x7f116e8275c0]
      /lib/libc.so.6(__assert_fail+0xf1) [0x7f116e81c941]
      /home/buildbot/build/sql/mysqld(Diagnostics_area::sql_errno() const+0x5d) [0x73c575]
      /home/buildbot/build/sql/mysqld(Rows_log_event::do_apply_event(Relay_log_info const*)+0x556) [0x9932da]
      /home/buildbot/build/sql/mysqld(Log_event::apply_event(Relay_log_info const*)+0x40) [0xaef190]
      /home/buildbot/build/sql/mysqld(apply_event_and_update_pos(Log_event*, THD*, Relay_log_info*)+0x396) [0xae36be]
      /home/buildbot/build/sql/mysqld() [0xae3f3a]
      /home/buildbot/build/sql/mysqld(handle_slave_sql+0x109e) [0xae7d03]
      /lib/libpthread.so.0(+0x69ca) [0x7f116f52f9ca]
      /lib/libc.so.6(clone+0x6d) [0x7f116e8d670d]
      
      Trying to get some variables.
      Some pointers may be invalid and cause the dump to abort.
      Query ((nil)): Connection ID (thread ID): 9
      Status: KILL_CONNECTION
      Optimizer switch: index_merge=on,index_merge_union=on,index_merge_sort_union=on,index_merge_intersection=on,index_merge_sort_intersection=off,index_condition_pushdown=on,derived_merge=on,derived_with_keys=on,firstmatch=on,loosescan=on,materialization=on,in_to_exists=on,semijoin=on,partial_match_rowid_merge=on,partial_match_table_scan=on,subquery_cache=on,mrr=off,mrr_cost_based=off,mrr_sort_keys=off,outer_join_with_cache=on,semijoin_with_cache=on,join_cache_incremental=on,join_cache_hashed=on,join_cache_bka=on,optimize_join_buffer_size=off,table_elimination=on
      
      The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
      information that should help you find out what is causing the crash.
      Writing a core file
      ----------SERVER LOG END-------------
      
      
       - saving '/home/buildbot/build/mysql-test/var/2/log/rpl.rpl_relayrotate-row/' to '/home/buildbot/build/mysql-test/var/log/rpl.rpl_relayrotate-row/'
      
      Retrying test rpl.rpl_relayrotate, attempt(2/3)...
      
      ***Warnings generated in error logs during shutdown after running tests: rpl.rpl_relayrotate
      
      mysqld: sql_class.h:1356: uint Diagnostics_area::sql_errno() const: Assertion `m_status == DA_ERROR' failed.
      131207 19:11:48 [ERROR] mysqld got signal 6 ;
      Attempting backtrace. You can use the following information to find out
      
      rpl.rpl_row_func002 'row'                w1 [ pass ]   1697
      rpl.rpl_row_inexist_tbl 'row'            w1 [ pass ]   1037
      rpl.rpl_row_insert_delayed 'row'         w1 [ pass ]   6697
      rpl.rpl_relayrotate 'row'                w2 [ retry-pass ]  11018
      
      Retrying test rpl.rpl_relayrotate, attempt(3/3)...
      
      rpl.rpl_temporary_errors 'row'           w2 [ pass ]   1173
      rpl.rpl_row_max_relay_size 'row'         w1 [ pass ]   6699
      rpl.rpl_row_rec_comp_myisam 'row'        w1 [ pass ]   4354
      rpl.rpl_relayrotate 'row'                w2 [ retry-pass ]  10123
      

      Here is another vacation, on a valgrind build, revno 3732:
      http://buildbot.askmonty.org/buildbot/builders/work-amd64-valgrind/builds/4948/steps/test/logs/stdio

        Gliffy Diagrams

          Attachments

            Activity

            Hide
            elenst Elena Stepanova added a comment -

            Still observed sometimes in buildbot, but apparently only on 5.3.

            Show
            elenst Elena Stepanova added a comment - Still observed sometimes in buildbot, but apparently only on 5.3.

              People

              • Assignee:
                Unassigned
                Reporter:
                elenst Elena Stepanova
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: