[MDEV-5963] InnoDB: Assertion failure in file row0sel.cc line 2503, Failing assertion: 0 with "key ptr now exceeds key end by 762 bytes" - JIRA

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 10.0.9
Fix Version/s: 10.0.11
Component/s: None
Labels:
None

Description

Test case:


--source include/have_innodb.inc
--source include/have_partition.inc

CREATE TABLE t1 (f1 VARCHAR(512) CHARACTER SET utf8) ENGINE=InnoDB;
INSERT INTO t1 VALUES ('j');

CREATE TABLE t2 (
  f2 VARCHAR(5) CHARACTER SET latin1,
  f3 VARCHAR(5) CHARACTER SET utf8,
  f4 INT,
  f5 VARCHAR(512) CHARACTER SET utf8,
  f6 VARCHAR(256) CHARACTER SET utf8,
  key (f2),
  key (f3),
  key (f5)
) ENGINE=InnoDB PARTITION BY LIST COLUMNS (f4)
  SUBPARTITION BY KEY(f6) SUBPARTITIONS 4 (
    PARTITION p0 VALUES IN (1,3,9,null),
    PARTITION p1 VALUES IN (2,4,0)
);

INSERT INTO t2 VALUES  
  ('k','s',3,'b','j'),('a','b',NULL,'v','j'),('c','m',9,'t',NULL),
  ('b','l',9,'b',NULL),('i','y',3,'o','w'),('c','m',NULL,'a','m'),  
  ('f','o',9,'m','w'),('f','q',NULL,'o','a');

CREATE TABLE t3 LIKE t2;

SELECT * FROM t1 INNER JOIN t2 ON ( f5 = f1 );
INSERT INTO t3 SELECT * FROM t2 WHERE f3 = 'm' AND f2 ='c';

2014-03-27 15:56:21 7f7f87ee9700  InnoDB: Warning: using a partial-field key prefix in search.
InnoDB: index `f5` of table `test`.`t2` /* Partition `p0`, Subpartition `p0sp2` */. Last data field length 768 bytes,
InnoDB: key ptr now exceeds key end by 762 bytes.
InnoDB: Key value in the MySQL format:
 len 6; hex 000000000203; asc       ;
2014-03-27 15:56:21 7f7f87ee9700  InnoDB: Assertion failure in thread 140185718134528 in file row0sel.cc line 2503
InnoDB: Failing assertion: 0
InnoDB: We intentionally generate a memory trap.
InnoDB: Submit a detailed bug report to http://bugs.mysql.com.
InnoDB: If you get repeated assertion failures or crashes, even
InnoDB: immediately after the mysqld startup, there may be
InnoDB: corruption in the InnoDB tablespace. Please refer to
InnoDB: http://dev.mysql.com/doc/refman/5.6/en/forcing-innodb-recovery.html
InnoDB: about forcing recovery.
140327 15:56:21 [ERROR] mysqld got signal 6 ;

#5  0x00007f7f8602c6f0 in *__GI_abort () at abort.c:92
#6  0x0000000000c865cd in row_sel_convert_mysql_key_to_innobase (tuple=0x7f7f59d803d0, buf=0x7f7f59d6d81f "\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\2
45\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\24
5\245\245\245\245\245\245\245
\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245\245"..., buf_len=3136, index=0x7f7f59c837f8, key_ptr=0x7f7f59e0a38a "", key_len=6, trx=0x7f7f59cbd478) at 10.0/storage/xtradb/row/row0sel.cc:2503
#7  0x0000000000b6e6b5 in ha_innobase::index_read (this=0x7f7f59d6d088, buf=0x7f7f59c98088 "\375\001c", key_ptr=0x7f7f59e0a08a "", key_len=6, find_flag=HA_READ_KEY_EXACT) at 10.0/storage/xtradb/handler/ha_innodb.cc:8302
#8  0x0000000000b6f8d8 in ha_innobase::rnd_pos (this=0x7f7f59d6d088, buf=0x7f7f59c98088 "\375\001c", pos=0x7f7f59e0a08a "") at 10.0/storage/xtradb/handler/ha_innodb.cc:8854
#9  0x0000000000862614 in handler::ha_rnd_pos (this=0x7f7f59d6d088, buf=0x7f7f59c98088 "\375\001c", pos=0x7f7f59e0a08a "") at 10.0/sql/handler.cc:2530
#10 0x0000000000df7964 in ha_partition::rnd_pos (this=0x7f7f59d37888, buf=0x7f7f59c98088 "\375\001c", pos=0x7f7f59e0a088 "\002") at 10.0/sql/ha_partition.cc:5028
#11 0x00000000008625ca in handler::ha_rnd_pos (this=0x7f7f59d37888, buf=0x7f7f59c98088 "\375\001c", pos=0x7f7f59e0a088 "\002") at 10.0/sql/handler.cc:2530
#12 0x000000000097fe71 in QUICK_ROR_INTERSECT_SELECT::get_next (this=0x7f7f59ca72f0) at 10.0/sql/opt_range.cc:11424
#13 0x000000000098f15d in rr_quick (info=0x7f7f59c6b5e8) at 10.0/sql/records.cc:346
#14 0x00000000006d4a91 in join_init_read_record (tab=0x7f7f59c6b540) at 10.0/sql/sql_select.cc:18357
#15 0x00000000006d2a07 in sub_select (join=0x7f7f59c23700, join_tab=0x7f7f59c6b540, end_of_records=false) at 10.0/sql/sql_select.cc:17464
#16 0x00000000006d22ce in do_select (join=0x7f7f59c23700, fields=0x7f7f67fbf578, table=0x0, procedure=0x0) at 10.0/sql/sql_select.cc:17129
#17 0x00000000006afd5e in JOIN::exec_inner (this=0x7f7f59c23700) at 10.0/sql/sql_select.cc:3065
#18 0x00000000006ad270 in JOIN::exec (this=0x7f7f59c23700) at 10.0/sql/sql_select.cc:2355
#19 0x00000000006b0607 in mysql_select (thd=0x7f7f67fbb070, rref_pointer_array=0x7f7f67fbf6d8, tables=0x7f7f59c228d8, wild_num=1, fields=..., conds=0x7f7f59c234f8, og_num=0, order=0x0, group=0x0, having=0x0, proc_param=0x0, select_options=3489925888, result=0x7f7f59c23668, unit=0x7f7f67fbed78, select_lex=0x7f7f67fbf460) at 10.0/sql/sql_select.cc:3293
#20 0x00000000006a6d1f in handle_select (thd=0x7f7f67fbb070, lex=0x7f7f67fbecb0, result=0x7f7f59c23668, setup_tables_done_option=1073741824) at 10.0/sql/sql_select.cc:372
#21 0x0000000000676549 in mysql_execute_command (thd=0x7f7f67fbb070) at 10.0/sql/sql_parse.cc:3555
#22 0x000000000067e2dd in mysql_parse (thd=0x7f7f67fbb070, rawbuf=0x7f7f59c22088 "INSERT INTO t3 SELECT * FROM t2 WHERE f3 = 'm' AND f2 ='c'", length=58, parser_state=0x7f7f87ee8610) at 10.0/sql/sql_parse.cc:6446
#23 0x00000000006710a4 in dispatch_command (command=COM_QUERY, thd=0x7f7f67fbb070, packet=0x7f7f682a0071 "INSERT INTO t3 SELECT * FROM t2 WHERE f3 = 'm' AND f2 ='c'", packet_length=58) at 10.0/sql/sql_parse.cc:1308
#24 0x0000000000670446 in do_command (thd=0x7f7f67fbb070) at 10.0/sql/sql_parse.cc:1005
#25 0x000000000078ae82 in do_handle_one_connection (thd_arg=0x7f7f67fbb070) at 10.0/sql/sql_connect.cc:1379
#26 0x000000000078abd5 in handle_one_connection (arg=0x7f7f67fbb070) at 10.0/sql/sql_connect.cc:1293
#27 0x0000000000a307d8 in pfs_spawn_thread (arg=0x7f7f67fe6170) at 10.0/storage/perfschema/pfs.cc:1853
#28 0x00007f7f87bc6b50 in start_thread (arg=<optimized out>) at pthread_create.c:304
#29 0x00007f7f860d1a7d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112

Stack trace from:

revision-id: psergey@askmonty.org-20140327090800-l3je8eqjrfx21iki
date: 2014-03-27 13:08:00 +0400
build-date: 2014-03-27 16:00:24 +0400
revno: 4116
branch-nick: 10.0

Gliffy Diagrams

Attachments

Activity

Ascending order - Click to sort in descending order

Hide

Permalink

Sergei Petrunia added a comment - 2014-04-17 14:14

It is also possible to get this error: Incorrect key file for table 't2'; try to repair it

CREATE TABLE t1 (f1 VARCHAR(512) CHARACTER SET utf8) ENGINE=InnoDB;
INSERT INTO t1 VALUES ('j');

CREATE TABLE t2 (
  f2 VARCHAR(5) CHARACTER SET latin1,
  f3 VARCHAR(5) CHARACTER SET utf8,
  f4 INT,
  f5 VARCHAR(512) CHARACTER SET utf8,
  f6 VARCHAR(256) CHARACTER SET utf8,
  key (f2),
  key (f3),
  key (f5)
) ENGINE=InnoDB PARTITION BY LIST COLUMNS (f4)
  SUBPARTITION BY KEY(f6) SUBPARTITIONS 4 (
    PARTITION p0 VALUES IN (1,3,9,null),
    PARTITION p1 VALUES IN (2,4,0)
);

INSERT INTO t2 VALUES  
  ('k','s',3,'b','j'),('a','b',NULL,'v','j'),('c','m',9,'t',NULL),
  ('b','l',9,'b',NULL),('i','y',3,'o','w'),('c','m',NULL,'a','m'),  
  ('f','o',9,'m','w'),('f','q',NULL,'o','a');

CREATE TABLE t3 LIKE t2;
INSERT INTO t3 SELECT * FROM t2 WHERE f3 = 'm' AND f2 ='c';
ERROR 1034 (HY000): Incorrect key file for table 't2'; try to repair it

Show

Sergei Petrunia added a comment - 2014-04-17 14:14 It is also possible to get this error: Incorrect key file for table 't2'; try to repair it CREATE TABLE t1 (f1 VARCHAR(512) CHARACTER SET utf8) ENGINE=InnoDB; INSERT INTO t1 VALUES ('j'); CREATE TABLE t2 ( f2 VARCHAR(5) CHARACTER SET latin1, f3 VARCHAR(5) CHARACTER SET utf8, f4 INT, f5 VARCHAR(512) CHARACTER SET utf8, f6 VARCHAR(256) CHARACTER SET utf8, key (f2), key (f3), key (f5) ) ENGINE=InnoDB PARTITION BY LIST COLUMNS (f4) SUBPARTITION BY KEY(f6) SUBPARTITIONS 4 ( PARTITION p0 VALUES IN (1,3,9,null), PARTITION p1 VALUES IN (2,4,0) ); INSERT INTO t2 VALUES ('k','s',3,'b','j'),('a','b',NULL,'v','j'),('c','m',9,'t',NULL), ('b','l',9,'b',NULL),('i','y',3,'o','w'),('c','m',NULL,'a','m'), ('f','o',9,'m','w'),('f','q',NULL,'o','a'); CREATE TABLE t3 LIKE t2; INSERT INTO t3 SELECT * FROM t2 WHERE f3 = 'm' AND f2 ='c'; ERROR 1034 (HY000): Incorrect key file for table 't2'; try to repair it

Hide

Permalink

Sergei Petrunia added a comment - 2014-04-17 15:05

It looks like the problem is caused here:

#0 ha_partition::rnd_init (...
#1 0x00000000005eb62f in handler::ha_rnd_init (...
#2 0x0000000000850d6f in handler::ha_rnd_init_with_error (...
#3 0x000000000095d896 in QUICK_ROR_INTERSECT_SELECT::init_ror_merged_scan (...
#4 0x000000000095d95d in QUICK_ROR_INTERSECT_SELECT::reset (...
#5 0x00000000006b7a08 in join_init_read_record (...
#6 0x00000000006b5a94 in sub_select (...
#7 0x00000000006b5364 in do_select (...
#8 0x000000000069294c in JOIN::exec_inner (...
#9 0x000000000068fc1c in JOIN::exec (...
#10 0x000000000069320b in mysql_select (...
#11 0x000000000068964c in handle_select (...
#12 0x0000000000657860 in mysql_execute_command (...

Here, opt_range.cc calls:
if (need_to_fetch_row && head->file->ha_rnd_init_with_error(1))

1 here means "scan=1", i.e. we tell the storage engine that we will be doing a sequential scan with handler->rnd_pos() calls.

Actually, index merge will be doing random probes with handler->rnd_pos().

Ha_partition depends heavily on whether scan=1 or scan=0. When scan=1 it only starts the scan on the first underlying partition (because it assumes that it will init the scan on the second partition after it has finished the first one). When we first do ha_partition->rnd_init(scan=1), but then issue a ha_partition->rnd_pos(..) which hits the non-first partition, we attempt to do an rnd_pos() on a ha_innobase object that hasn't been initialized for rnd_pos calls. We pass it rowid (6-byte internal innodb PK), while it thinks that it should get an index tuple for index "b". This causes assertion and/or crash.

Show

Sergei Petrunia added a comment - 2014-04-17 15:05 It looks like the problem is caused here: #0 ha_partition::rnd_init (... #1 0x00000000005eb62f in handler::ha_rnd_init (... #2 0x0000000000850d6f in handler::ha_rnd_init_with_error (... #3 0x000000000095d896 in QUICK_ROR_INTERSECT_SELECT::init_ror_merged_scan (... #4 0x000000000095d95d in QUICK_ROR_INTERSECT_SELECT::reset (... #5 0x00000000006b7a08 in join_init_read_record (... #6 0x00000000006b5a94 in sub_select (... #7 0x00000000006b5364 in do_select (... #8 0x000000000069294c in JOIN::exec_inner (... #9 0x000000000068fc1c in JOIN::exec (... #10 0x000000000069320b in mysql_select (... #11 0x000000000068964c in handle_select (... #12 0x0000000000657860 in mysql_execute_command (... Here, opt_range.cc calls: if (need_to_fetch_row && head->file->ha_rnd_init_with_error(1)) 1 here means "scan=1", i.e. we tell the storage engine that we will be doing a sequential scan with handler->rnd_pos() calls. Actually, index merge will be doing random probes with handler->rnd_pos(). Ha_partition depends heavily on whether scan=1 or scan=0. When scan=1 it only starts the scan on the first underlying partition (because it assumes that it will init the scan on the second partition after it has finished the first one). When we first do ha_partition->rnd_init(scan=1), but then issue a ha_partition->rnd_pos(..) which hits the non-first partition, we attempt to do an rnd_pos() on a ha_innobase object that hasn't been initialized for rnd_pos calls. We pass it rowid (6-byte internal innodb PK), while it thinks that it should get an index tuple for index "b". This causes assertion and/or crash.

Hide

Permalink

Sergei Petrunia added a comment - 2014-04-17 15:06

Btw, MySQL 5.6 has it fixed:

mattias.jonsson@oracle.com-20120717134355-086hbd5maadz0h8i

Bug#13947868

Problem chain was:
1) ha_rnd_init(1) was called (start random access scanning,
    when using rnd_next)
2) the partitioning engine optimizes this to only init one
   partition at a time.
3) ha_rnd_pos was called for a row not in the initialized
    partitions -> InnoDB tried to read back a position with
    wrong index, since the partition never got the rnd_init call.

Note that it requires that the partitioned table does NOT have
a primary key! I.e. InnoDB uses a generated clustered key (row_id).

Fixed by using ha_rnd_init(false) instead (prepare for ha_rnd_pos
instead of ha_rnd_next).

Jorgen L also found another place where ha_rnd_init was used with
wrong argument, which is also fixed and has a test case.

also ending previous ha_rnd_init call since that could have been
a scan init.

Their fix lacks the testcase.

Show

Sergei Petrunia added a comment - 2014-04-17 15:06 Btw, MySQL 5.6 has it fixed: mattias.jonsson@oracle.com-20120717134355-086hbd5maadz0h8i Bug#13947868 Problem chain was: 1) ha_rnd_init(1) was called (start random access scanning, when using rnd_next) 2) the partitioning engine optimizes this to only init one partition at a time. 3) ha_rnd_pos was called for a row not in the initialized partitions -> InnoDB tried to read back a position with wrong index, since the partition never got the rnd_init call. Note that it requires that the partitioned table does NOT have a primary key! I.e. InnoDB uses a generated clustered key (row_id). Fixed by using ha_rnd_init(false) instead (prepare for ha_rnd_pos instead of ha_rnd_next). Jorgen L also found another place where ha_rnd_init was used with wrong argument, which is also fixed and has a test case. also ending previous ha_rnd_init call since that could have been a scan init. Their fix lacks the testcase.

InnoDB: Assertion failure in file row0sel.cc line 2503, Failing assertion: 0 with "key ptr now exceeds key end by 762 bytes"

Details

Description

Gliffy Diagrams

Attachments

Activity

People

Dates

Time Tracking

Agile